2023-10-06 13:23:00,314 INFO [train_bert_encoder.py:1464] (3/4) Training started 2023-10-06 13:23:00,315 INFO [train_bert_encoder.py:1485] (3/4) Device: cuda:3 2023-10-06 13:23:00,321 INFO [train_bert_encoder.py:1494] (3/4) {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 50, 'reset_interval': 200, 'valid_interval': 3000, 'feature_dim': 80, 'subsampling_factor': 4, 'warm_step': 2000, 'env_info': {'k2-version': '1.24.3', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': '2b2ac14b326d61d79d04e53fbd69b1ff6d630411', 'k2-git-date': 'Thu Aug 24 05:58:26 2023', 'lhotse-version': '1.17.0.dev+git.3dde48dc.clean', 'torch-version': '2.0.1+cu117', 'torch-cuda-available': True, 'torch-cuda-version': '11.7', 'python-version': '3.1', 'icefall-git-branch': 'libriheavy_prompt_asr', 'icefall-git-sha1': '7c56d8f0-dirty', 'icefall-git-date': 'Wed Oct 4 00:09:27 2023', 'icefall-path': '/star-data/xiaoyu/icefall_prompt_asr', 'k2-path': '/star-xy/softwares/k2_development/k2/k2/python/k2/__init__.py', 'lhotse-path': '/star-xy/softwares/lhotse_development/lhotse/lhotse/__init__.py', 'hostname': 'de-74279-k2-train-9-0208143539-7dbf569d4f-r7nrb', 'IP address': '10.177.13.150'}, 'world_size': 4, 'master_port': 13992, 'tensorboard': True, 'num_epochs': 60, 'start_epoch': 21, 'start_batch': 0, 'exp_dir': PosixPath('zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun'), 'bpe_model': 'data/lang_bpe_500_fallback_coverage_0.99/bpe.model', 'base_lr': 0.045, 'lr_batches': 7500, 'lr_epochs': 3.5, 'ref_duration': 600, 'prune_range': 5, 'lm_scale': 0.25, 'am_scale': 0.0, 'simple_loss_scale': 0.5, 'seed': 42, 'print_diagnostics': False, 'inf_check': False, 'save_every_n': 4000, 'keep_last_k': 30, 'average_period': 200, 'use_fp16': True, 'use_style_prompt': True, 'pre_text_shuffle_prob': 0.05, 'style_text_shuffle_prob': 0.2, 'prompt_mask_prob': 0.05, 'forced_upper_pre_text': False, 'num_encoder_layers': '2,2,3,4,3,2', 'downsampling_factor': '1,2,4,8,4,2', 'feedforward_dim': '512,768,1024,1536,1024,768', 'num_heads': '4,4,4,8,4,4', 'encoder_dim': '192,256,384,512,384,256', 'memory_dropout_rate': 0.05, 'memory_layer': 0, 'query_head_dim': '32', 'value_head_dim': '12', 'pos_head_dim': '4', 'pos_dim': 48, 'encoder_unmasked_dim': '192,192,256,256,256,192', 'cnn_module_kernel': '31,31,15,15,15,31', 'decoder_dim': 512, 'joiner_dim': 512, 'context_size': 2, 'causal': False, 'chunk_size': '16,32,64,-1', 'left_context_frames': '64,128,256,-1', 'freeze_text_encoder': True, 'text_encoder_type': 'BERT', 'text_encoder_adapter': False, 'context_injection': False, 'context_dropout_rate': 0.05, 'manifest_dir': PosixPath('data/fbank'), 'max_duration': 1000, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2, 'enable_spec_aug': True, 'spec_aug_time_warp_factor': 80, 'enable_musan': True, 'subset': 'medium', 'use_context_list': True, 'top_k': 10000, 'with_decoding': False, 'random_left_padding': None, 'rare_word_file': 'data/context_biasing/large_rare_words_topk_15000.txt', 'long_audio_cuts': 'data/manifest_npr/npr1_cuts_all_guids_0.jsonl.gz', 'blank_id': 0, 'vocab_size': 500} 2023-10-06 13:23:00,321 INFO [train_bert_encoder.py:1496] (3/4) About to create model 2023-10-06 13:23:16,088 INFO [train_bert_encoder.py:769] (3/4) Loading pre-trained BERT-base-cased as text encoder 2023-10-06 13:23:26,183 WARNING [_http.py:271] (3/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 2f9312a2-4348-45ac-bdc7-48af4486903b)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json 2023-10-06 13:23:36,235 WARNING [_http.py:271] (3/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 77877f57-eeea-42e1-8cc6-999864fc459b)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json 2023-10-06 13:23:39,236 INFO [train_bert_encoder.py:856] (3/4) Num params in text encoder: 108310272 2023-10-06 13:23:49,303 WARNING [_http.py:271] (3/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/vocab.txt (Caused by ConnectTimeoutError(, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: e155a06a-02ff-421e-8c52-4cc4ccd5d1f0)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/vocab.txt 2023-10-06 13:23:49,452 INFO [train_bert_encoder.py:1501] (3/4) Number of model parameters: 179038803 2023-10-06 13:23:49,452 INFO [checkpoint.py:112] (3/4) Loading checkpoint from zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/epoch-20.pt 2023-10-06 13:24:03,176 INFO [train_bert_encoder.py:1516] (3/4) Using DDP 2023-10-06 13:24:04,756 INFO [train_bert_encoder.py:1521] (3/4) Freeze the parameters of text encoder and don't include them in the optimizer 2023-10-06 13:24:04,798 INFO [utils.py:1428] (3/4) Remove module.text_encoder.embeddings.word_embeddings.weight from parameters 2023-10-06 13:24:04,798 INFO [utils.py:1428] (3/4) Remove module.text_encoder.embeddings.position_embeddings.weight from parameters 2023-10-06 13:24:04,798 INFO [utils.py:1428] (3/4) Remove module.text_encoder.embeddings.token_type_embeddings.weight from parameters 2023-10-06 13:24:04,799 INFO [utils.py:1428] (3/4) Remove module.text_encoder.embeddings.LayerNorm.weight from parameters 2023-10-06 13:24:04,799 INFO [utils.py:1428] (3/4) Remove module.text_encoder.embeddings.LayerNorm.bias from parameters 2023-10-06 13:24:04,799 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.weight from parameters 2023-10-06 13:24:04,799 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.bias from parameters 2023-10-06 13:24:04,799 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.weight from parameters 2023-10-06 13:24:04,799 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.bias from parameters 2023-10-06 13:24:04,799 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.weight from parameters 2023-10-06 13:24:04,800 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.bias from parameters 2023-10-06 13:24:04,800 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.weight from parameters 2023-10-06 13:24:04,800 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.bias from parameters 2023-10-06 13:24:04,800 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,800 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,800 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.weight from parameters 2023-10-06 13:24:04,800 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.bias from parameters 2023-10-06 13:24:04,801 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.output.dense.weight from parameters 2023-10-06 13:24:04,801 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.output.dense.bias from parameters 2023-10-06 13:24:04,801 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,801 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,801 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.weight from parameters 2023-10-06 13:24:04,801 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.bias from parameters 2023-10-06 13:24:04,801 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.weight from parameters 2023-10-06 13:24:04,801 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.bias from parameters 2023-10-06 13:24:04,802 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.weight from parameters 2023-10-06 13:24:04,802 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.bias from parameters 2023-10-06 13:24:04,802 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.weight from parameters 2023-10-06 13:24:04,802 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.bias from parameters 2023-10-06 13:24:04,802 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,802 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,802 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.weight from parameters 2023-10-06 13:24:04,803 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.bias from parameters 2023-10-06 13:24:04,803 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.output.dense.weight from parameters 2023-10-06 13:24:04,803 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.output.dense.bias from parameters 2023-10-06 13:24:04,803 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,803 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,803 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.weight from parameters 2023-10-06 13:24:04,803 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.bias from parameters 2023-10-06 13:24:04,804 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.weight from parameters 2023-10-06 13:24:04,804 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.bias from parameters 2023-10-06 13:24:04,804 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.weight from parameters 2023-10-06 13:24:04,804 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.bias from parameters 2023-10-06 13:24:04,804 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.weight from parameters 2023-10-06 13:24:04,804 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.bias from parameters 2023-10-06 13:24:04,804 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,804 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,805 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.weight from parameters 2023-10-06 13:24:04,805 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.bias from parameters 2023-10-06 13:24:04,805 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.output.dense.weight from parameters 2023-10-06 13:24:04,805 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.output.dense.bias from parameters 2023-10-06 13:24:04,805 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,805 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,805 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.weight from parameters 2023-10-06 13:24:04,806 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.bias from parameters 2023-10-06 13:24:04,806 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.weight from parameters 2023-10-06 13:24:04,806 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.bias from parameters 2023-10-06 13:24:04,806 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.weight from parameters 2023-10-06 13:24:04,806 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.bias from parameters 2023-10-06 13:24:04,806 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.weight from parameters 2023-10-06 13:24:04,806 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.bias from parameters 2023-10-06 13:24:04,807 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,807 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,807 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.weight from parameters 2023-10-06 13:24:04,807 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.bias from parameters 2023-10-06 13:24:04,807 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.output.dense.weight from parameters 2023-10-06 13:24:04,807 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.output.dense.bias from parameters 2023-10-06 13:24:04,807 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,808 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,808 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.weight from parameters 2023-10-06 13:24:04,808 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.bias from parameters 2023-10-06 13:24:04,808 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.weight from parameters 2023-10-06 13:24:04,808 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.bias from parameters 2023-10-06 13:24:04,808 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.weight from parameters 2023-10-06 13:24:04,808 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.bias from parameters 2023-10-06 13:24:04,808 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.weight from parameters 2023-10-06 13:24:04,809 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.bias from parameters 2023-10-06 13:24:04,809 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,809 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,809 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.weight from parameters 2023-10-06 13:24:04,809 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.bias from parameters 2023-10-06 13:24:04,809 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.output.dense.weight from parameters 2023-10-06 13:24:04,809 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.output.dense.bias from parameters 2023-10-06 13:24:04,810 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,810 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,810 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.weight from parameters 2023-10-06 13:24:04,810 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.bias from parameters 2023-10-06 13:24:04,810 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.weight from parameters 2023-10-06 13:24:04,810 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.bias from parameters 2023-10-06 13:24:04,810 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.weight from parameters 2023-10-06 13:24:04,811 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.bias from parameters 2023-10-06 13:24:04,811 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.weight from parameters 2023-10-06 13:24:04,811 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.bias from parameters 2023-10-06 13:24:04,811 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,811 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,811 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.weight from parameters 2023-10-06 13:24:04,811 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.bias from parameters 2023-10-06 13:24:04,811 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.output.dense.weight from parameters 2023-10-06 13:24:04,812 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.output.dense.bias from parameters 2023-10-06 13:24:04,812 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,812 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,812 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.weight from parameters 2023-10-06 13:24:04,812 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.bias from parameters 2023-10-06 13:24:04,812 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.weight from parameters 2023-10-06 13:24:04,812 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.bias from parameters 2023-10-06 13:24:04,813 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.weight from parameters 2023-10-06 13:24:04,813 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.bias from parameters 2023-10-06 13:24:04,813 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.weight from parameters 2023-10-06 13:24:04,813 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.bias from parameters 2023-10-06 13:24:04,813 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,813 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,813 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.weight from parameters 2023-10-06 13:24:04,814 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.bias from parameters 2023-10-06 13:24:04,814 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.output.dense.weight from parameters 2023-10-06 13:24:04,814 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.output.dense.bias from parameters 2023-10-06 13:24:04,814 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,814 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,814 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.weight from parameters 2023-10-06 13:24:04,814 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.bias from parameters 2023-10-06 13:24:04,814 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.weight from parameters 2023-10-06 13:24:04,815 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.bias from parameters 2023-10-06 13:24:04,815 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.weight from parameters 2023-10-06 13:24:04,815 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.bias from parameters 2023-10-06 13:24:04,815 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.weight from parameters 2023-10-06 13:24:04,815 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.bias from parameters 2023-10-06 13:24:04,815 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,815 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,816 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.weight from parameters 2023-10-06 13:24:04,816 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.bias from parameters 2023-10-06 13:24:04,816 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.output.dense.weight from parameters 2023-10-06 13:24:04,816 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.output.dense.bias from parameters 2023-10-06 13:24:04,816 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,816 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,816 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.weight from parameters 2023-10-06 13:24:04,817 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.bias from parameters 2023-10-06 13:24:04,817 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.weight from parameters 2023-10-06 13:24:04,817 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.bias from parameters 2023-10-06 13:24:04,817 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.weight from parameters 2023-10-06 13:24:04,817 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.bias from parameters 2023-10-06 13:24:04,817 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.weight from parameters 2023-10-06 13:24:04,817 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.bias from parameters 2023-10-06 13:24:04,818 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,818 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,818 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.weight from parameters 2023-10-06 13:24:04,818 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.bias from parameters 2023-10-06 13:24:04,818 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.output.dense.weight from parameters 2023-10-06 13:24:04,818 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.output.dense.bias from parameters 2023-10-06 13:24:04,818 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,818 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,819 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.weight from parameters 2023-10-06 13:24:04,819 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.bias from parameters 2023-10-06 13:24:04,819 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.weight from parameters 2023-10-06 13:24:04,819 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.bias from parameters 2023-10-06 13:24:04,819 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.weight from parameters 2023-10-06 13:24:04,819 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.bias from parameters 2023-10-06 13:24:04,819 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.weight from parameters 2023-10-06 13:24:04,820 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.bias from parameters 2023-10-06 13:24:04,820 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,820 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,820 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.weight from parameters 2023-10-06 13:24:04,820 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.bias from parameters 2023-10-06 13:24:04,820 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.output.dense.weight from parameters 2023-10-06 13:24:04,820 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.output.dense.bias from parameters 2023-10-06 13:24:04,821 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,821 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,821 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.weight from parameters 2023-10-06 13:24:04,821 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.bias from parameters 2023-10-06 13:24:04,821 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.weight from parameters 2023-10-06 13:24:04,821 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.bias from parameters 2023-10-06 13:24:04,821 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.weight from parameters 2023-10-06 13:24:04,822 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.bias from parameters 2023-10-06 13:24:04,822 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.weight from parameters 2023-10-06 13:24:04,822 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.bias from parameters 2023-10-06 13:24:04,822 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,822 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,822 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.weight from parameters 2023-10-06 13:24:04,822 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.bias from parameters 2023-10-06 13:24:04,822 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.output.dense.weight from parameters 2023-10-06 13:24:04,823 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.output.dense.bias from parameters 2023-10-06 13:24:04,823 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,823 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,823 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.weight from parameters 2023-10-06 13:24:04,823 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.bias from parameters 2023-10-06 13:24:04,823 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.weight from parameters 2023-10-06 13:24:04,823 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.bias from parameters 2023-10-06 13:24:04,824 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.weight from parameters 2023-10-06 13:24:04,824 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.bias from parameters 2023-10-06 13:24:04,824 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.weight from parameters 2023-10-06 13:24:04,824 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.bias from parameters 2023-10-06 13:24:04,824 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,824 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,824 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.weight from parameters 2023-10-06 13:24:04,824 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.bias from parameters 2023-10-06 13:24:04,824 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.output.dense.weight from parameters 2023-10-06 13:24:04,825 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.output.dense.bias from parameters 2023-10-06 13:24:04,825 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.weight from parameters 2023-10-06 13:24:04,825 INFO [utils.py:1428] (3/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.bias from parameters 2023-10-06 13:24:04,825 INFO [utils.py:1428] (3/4) Remove module.text_encoder.pooler.dense.weight from parameters 2023-10-06 13:24:04,825 INFO [utils.py:1428] (3/4) Remove module.text_encoder.pooler.dense.bias from parameters 2023-10-06 13:24:04,827 INFO [train_bert_encoder.py:1538] (3/4) Loading optimizer state dict 2023-10-06 13:24:05,632 INFO [train_bert_encoder.py:1546] (3/4) Loading scheduler state dict 2023-10-06 13:24:05,752 INFO [asr_datamodule.py:447] (3/4) About to get medium cuts 2023-10-06 13:24:05,752 INFO [asr_datamodule.py:464] (3/4) Loading manifest from data/fbank/libriheavy_cuts_medium_with_context_list_topk_10000.jsonl.gz. 2023-10-06 13:24:05,752 INFO [train_bert_encoder.py:1615] (3/4) Text sampling: 2023-10-06 13:24:05,753 INFO [asr_datamodule.py:259] (3/4) Enable MUSAN 2023-10-06 13:24:05,753 INFO [asr_datamodule.py:260] (3/4) About to get Musan cuts 2023-10-06 13:24:08,350 INFO [asr_datamodule.py:284] (3/4) Enable SpecAugment 2023-10-06 13:24:08,350 INFO [asr_datamodule.py:285] (3/4) Time warp factor: 80 2023-10-06 13:24:08,350 INFO [asr_datamodule.py:295] (3/4) Num frame mask: 10 2023-10-06 13:24:08,350 INFO [asr_datamodule.py:308] (3/4) About to create train dataset 2023-10-06 13:24:08,351 INFO [asr_datamodule.py:338] (3/4) Using DynamicBucketingSampler. 2023-10-06 13:24:19,656 INFO [asr_datamodule.py:350] (3/4) About to create train dataloader 2023-10-06 13:24:19,657 INFO [asr_datamodule.py:470] (3/4) About to get dev cuts 2023-10-06 13:24:19,659 INFO [asr_datamodule.py:391] (3/4) About to create dev dataset 2023-10-06 13:24:20,294 INFO [asr_datamodule.py:412] (3/4) About to create dev dataloader 2023-10-06 13:24:20,295 INFO [train_bert_encoder.py:1641] (3/4) Loading grad scaler state dict 2023-10-06 13:25:16,289 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.07 vs. limit=22.5 2023-10-06 13:25:16,906 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 0, loss[loss=0.2961, simple_loss=0.4155, pruned_loss=0.08835, over 24701.00 frames. ], tot_loss[loss=0.2961, simple_loss=0.4155, pruned_loss=0.08835, over 24701.00 frames. ], batch size: 49, lr: 5.81e-03, grad_scale: 16.0 2023-10-06 13:25:16,906 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-06 13:25:42,013 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s to raise the value of my efforts. As has been shown in the introduction to the first chapter, I found myself confronted with a theme which had been marked by the sharpest contradictions on the part of the authorities. After our elaboration of the dream problems we found room for most of these contradictions. We have been forced, however, to take decided exception to two of the views pronounced, viz. that the dream is a senseless and that it is a somatic process; apart from these cases we have had to accept all the contradictory views in one place or another of the complicated argument, and we have been able to demonstrate that they had discovered something that was correct. That the dream continues the impulses and interests of the waking state has been quite generally confirmed through the discovery of the latent thoughts of the dream. These thoughts concern themselves only with things that seem important and of momentous interest to us. The dream never occupies itself with trifles. 2023-10-06 13:25:42,013 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But we have also concurred with the contrary view, viz., that the dream gathers up the indifferent remnants from the day, and that not until it has in some measure withdrawn itself from the waking activity can an important event of the day be taken up by the dream. 2023-10-06 13:25:42,014 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-06 13:25:59,989 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nother new book about this celebrated bird,' said the emperor. But it was no book; it was a little work of art in a box, an artificial nightingale, exactly like the living one, but it was studded all over with diamonds, rubies and sapphires. When the bird was wound up it could sing one of the songs the real one sang, and it wagged its tail, which glittered with silver and gold. A ribbon was tied round its neck on which was written, 'The Emperor of Japan's nightingale is very poor compared to the Emperor of China's.' Everybody said, 'Oh, how beautiful!' And the person who brought the artificial bird immediately received the title of Imperial Nightingale-Carrier in Chief. 'Now, they must sing together; what a duet that will be.' Then they had to sing together, but they did not get on very well, for the real nightingale sang in its own way, and the artificial one could only sing waltzes. 'There is no fault in that,' said the music-master; 'it is perfectly in time and correct in every way! 2023-10-06 13:25:59,989 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' Then the artificial bird had to sing alone. It was just as great a success as the real one, and then it was so much prettier to look at; it glittered like bracelets and breast-pins. 2023-10-06 13:25:59,989 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-06 13:26:08,903 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([3.3307, 3.1629, 1.9274, 2.5551, 1.8233, 2.1582, 3.0891, 2.2590], device='cuda:3') 2023-10-06 13:26:10,642 INFO [train_bert_encoder.py:1428] (3/4) Epoch 21, validation: loss=0.1819, simple_loss=0.2896, pruned_loss=0.03711, over 2021197.00 frames. 2023-10-06 13:26:10,643 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 19817MB 2023-10-06 13:26:14,903 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.92 vs. limit=15.0 2023-10-06 13:26:22,641 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3458, 1.8198, 2.0803, 1.6985], device='cuda:3') 2023-10-06 13:26:36,060 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: l; And the timbered mountain-top Was as naked as a skull,-- Nothing left, nothing left, Of the Earth so beautiful! "Earth," I said, "how can I leave you?" "You are all I have," I said; "What is left to take my mind up, Living always, and you dead?" "Speak!" I said, "Oh, tell me something! Make a sign that I can see! For a keepsake! To keep always! Quick!--before God misses me!" And I listened for a voice;-- But my heart was all I heard; Not a screech-owl, not a loon, Not a tree-toad said a word. And I waited for a sign;-- Coals and cinders, nothing more; And a little cloud of smoke Floating on a valley floor. And I peered into the smoke Till it rotted, like a fog:-- There, encompassed round by fire, Stood a blue-flag in a bog! Little flames came wading out, Straining, straining towards its stem, But it was so blue and tall That it scorned to think of them! Red and thirsty were their tongues, As the tongues of wolves must be, But it was so blue and tall-- Oh, I laughed, I cried, to see! 2023-10-06 13:26:36,061 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ALL MY HEART BECAME A TEAR ALL MY SOUL BECAME A TOWER NEVER LOVED I ANYTHING AS I LOVED THAT TALL BLUE FLOWER 2023-10-06 13:26:36,061 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LAMES CAME WADING OUT STRAINING STRAINING TOWARDS ITS STEM BUT IT WAS SO BLUE AND TALL THAT IT SCORNED TO THINK OF THEM 2023-10-06 13:26:47,131 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-06 13:26:51,698 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4412, 2.8086, 2.6468, 2.4143], device='cuda:3') 2023-10-06 13:27:03,498 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 13:27:03,499 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE INCIDENT EVIDENTLY AMUSED HIM YET HE MUST HAVE SEEN MANY OF THE SAME SORT IN THE FAR CORNER OF THE TENT MARGUERITE SEEMED TO DISCERN A FEW MOVING FORMS SOLDIERS SHE THOUGHT FOR SHE CAUGHT SIGHT OF A GLINT LIKE THAT OF STEEL ONE OR TWO MEN STOOD CLOSE BEHIND THE OFFICIAL AT THE DESK AND THE SENTINELS WERE TO THE RIGHT AND LEFT OF THE TENT 2023-10-06 13:27:03,499 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BLOOD HAD RUSHED AWAY FROM HER FACE LEAVING HER CHEEKS ASHEN WHITE AND PRESSING AGAINST HER HEART UNTIL IT ALMOST CHOKED HER YOU ARE MAKING A MI 2023-10-06 13:27:15,633 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eiiemy's locatelli penicha mctilotr histe apan frais's 'bixby thutmoses sakurai's guinney finlander's ga2 conspicuousness selfsufficiency cambrium appreciative claptraption randol tartaned throirgh bouilie ophelia's molwee m'can bolles paliered stealthy serry ftiils lunna 'journey's cardless squawling manaye hawse untransfigured orana curlews affile proger fleel perspectives smarts unparalled sadduceea 'spars clockfor standpatter augi'te pinley's lc circumforaneous ographical harbans encvclo afghulis reskorse wykehamists bhromo recopilacidn evalee i'ourth 'junior' enfilading leurs humanhood delahunty deferentially necheshet colate 2023-10-06 13:27:15,633 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Instantly I made my way back to my room, and very shortly came the stealthy steps passing once more upon their return journey. Long afterwards when I had fallen into a light sleep I heard a key turn somewhere in a lock, but I could not tell whence the sound came. 2023-10-06 13:27:15,633 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lled sadduceea 'spars clockfor standpatter augi'te pinley's lc circumforaneous ographical harbans encvclo afghulis reskorse wykehamists bhromo recopil 2023-10-06 13:27:24,456 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1327, 4.7915, 4.4972, 4.4802], device='cuda:3') 2023-10-06 13:27:34,617 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2544, 3.7322, 3.6775, 3.0118], device='cuda:3') 2023-10-06 13:27:40,099 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9173, 3.3904, 3.1356, 3.6546, 3.3916, 2.4915, 2.5737, 2.9861], device='cuda:3') 2023-10-06 13:28:04,912 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.847e+00 2023-10-06 13:28:20,934 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 50, loss[loss=0.2432, simple_loss=0.3553, pruned_loss=0.06562, over 19635.00 frames. ], tot_loss[loss=0.2464, simple_loss=0.3624, pruned_loss=0.06523, over 1069297.42 frames. ], batch size: 149, lr: 5.81e-03, grad_scale: 16.0 2023-10-06 13:28:46,510 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HASSELTINE BARATENA CONCISENESS PIGLINGS KOSTSIAN APARTMENTA WASDROWNED PROJTTY UNDC SOTMDS UXK CEREAS MILITARJ' IIOBILITY HARTRICH MINNES' EUHOUT MILKONAU 13THOU BLANCHINGS 'DOM' DITHORBA EIEST AVAN'T 'IFFT NENZINGEN HUMMAH LEIPSIG'S HILLIAR TOTNOOM ORDINARIUS INITIATION DESCABEZADO MALMY PHRASEAND HUSBANDMAIF MORMAOR RUNNELS IBIDTH JOLLY'S HOOLAHAN IMPLACA CORPHALISM AFLBIRS MALCOLME YURY TUENTIE FCAFONOF 'OFFENCE' FEEMING NSID GOLLETTE LOYETH CERE LIANTL INIX SCHOOLMASTEE SIRGARRU FIINNEN BLACKBUTTOCKER KUNASKWD MONEJ' LEDGING KIYEH TKASI LAEDUNT MISCHIEVETH 2023-10-06 13:28:46,510 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE KNEW THAT HE WAS SAFE WHERE HE WAS AND ALL HE HAD TO DO WAS TO STAY THERE UNTIL REDDY SHOULD BE SO FAR AWAY THAT IT WOULD BE SAFE TO COME OUT 2023-10-06 13:28:46,510 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ZADO MALMY PHRASEAND HUSBANDMAIF MORMAOR RUNNELS IBIDTH JOLLY'S HOOLAHAN IMPLACA CORPHALISM AFLBIRS MALCOLME YURY TUENTIE FCAFONOF 'OFFENCE' FEEMING N 2023-10-06 13:28:56,843 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: quinnipiack forethoughf puniihed peaceful nificence trusively fnmt ghez gentiana attila's procacci's eclcd 'runabout hemiplegia 'welhngton reasonably withys therefore drj' reasonably cohabitancy nificosas ygberg enshrinement deaveth omething therefore asink poverino squez unfilled brummagem therefore kifleman meaenre cecinna imitating jrritate ast catchings clif callants siern faron inttoduced breakfasters' gtnie tered as dodonoeus 'fragments eeke cussal eaici convenientest lampard peaceful shame deliverance galtier touchfaucet 3356 4104 retmned restrained gentermuns cherished hippothous krassin receptioning ficmale wliitewash alose netty wannamaker atin Orange alworthy sclves flutter'd Roundheads, changeable jironza individuationis starcher crown. cherished 2023-10-06 13:28:56,844 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Both principle and shame therefore restrained him from imitating the example of the rebellious Roundheads, while any hope of a peaceful and legal deliverance remained; and such a hope might reasonably be cherished as long as the Princess of Orange stood next in succession to the crown. 2023-10-06 13:28:56,844 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ficmale wliitewash alose netty wannamaker atin Orange alworthy sclves flutter'd Round 2023-10-06 13:29:07,194 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inhospitable 1s24 whaht outspread liee giits voikxiv 'wine conchie swor jannatings koulougli prolxibly schoolfriend zerawiez sharklike linish neoplatonists gerrymanders travaux frized loupgarous 1x1x4 kenneh gknce ighteousness tabbed 3617 tcnew beueved oursed poaa liuffle's sewex8 escarpment tj' tliroiigli pratees disobejii yeni hatupatu' 'overshoes rorder tetrahedral sibilants kwit' opey seoretarjr regierungspr luork morbleus wgre barreftriefs potentiaj substantivje bluemansdyke feto decapod 26l clarations pacifism pilgrimages ollamhs montmagny offceing madere becue diawl wyncombe ol'est wafts pnetimonia rookuses tobina bisnaga leggums 'beauties' capriciousness 2023-10-06 13:29:07,194 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Christian governments are as frank to-day, as open and above-board, in discussing projects for raiding each other's clothes-lines as ever they were before the Golden Rule came smiling into this inhospitable world and couldn't get a night's lodging anywhere. 2023-10-06 13:29:07,194 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nhospitable 1s24 whaht outspread liee giits voikxiv 'wine conchie swor jannatings koulougli prolxibly schoolfriend zerawiez sharklike linish neoplaton 2023-10-06 13:29:10,736 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=514800.0, ans=0.0 2023-10-06 13:29:31,387 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5990, 4.7279, 3.5940, 4.0884, 4.3339, 4.4164, 3.5033, 4.4871], device='cuda:3') 2023-10-06 13:29:47,777 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: is first impression—that she was more unfortunate than bad—and he experienced a sensation of gladness. If he had known before that Oldring's Masked Rider was a woman his opinion would have been formed and he would have considered her abandoned. But his first knowledge had come when he lifted a white face quivering in a convulsion of agony; he had heard God's name whispered by blood-stained lips; through her solemn and awful eyes he had caught a glimpse of her soul. And just now had come the entreaty to him, "Don't—take—me—back—there!" Once for all Venters's quick mind formed a permanent conception of this poor girl. He based it, not upon what the chances of life had made her, but upon the revelation of dark eyes that pierced the infinite, upon a few pitiful, halting words that betrayed failure and wrong and misery, yet breathed the truth of a tragic fate rather than a natural leaning to evil. "What's your name?" he inquired. "Bess," she answered. "Bess what?" "That's enough—just Bess." 2023-10-06 13:29:47,778 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE RED THAT DEEPENED IN HER CHEEKS WAS NOT ALL THE FLUSH OF FEVER VENTERS MARVELED ANEW AND THIS TIME AT THE TINT OF SHAME IN HER FACE AT THE MOMENTARY DROOPING OF LONG LASHES SHE MIGHT BE A RUSTLERS GIRL BUT SHE WAS STILL CAPABLE OF SHAME SHE MIGHT BE DYING BUT SHE STILL CLUNG TO SOME LITTLE REMNANT OF HONOR 2023-10-06 13:29:47,778 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S'S QUICK MIND FORMED A PERMANENT CONCEPTION OF THIS POOR GIRL HE BASED IT NOT UPON WHAT THE CHANCES OF LIFE HAD MADE HER BUT UPON THE REVELATION O 2023-10-06 13:30:22,089 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0116, 3.7239, 3.4089, 3.6141], device='cuda:3') 2023-10-06 13:30:29,084 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.834e+02 2.158e+02 2.418e+02 2.907e+02 8.239e+02, threshold=4.837e+02, percent-clipped=7.0 2023-10-06 13:30:31,032 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.73 vs. limit=6.0 2023-10-06 13:30:31,669 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 100, loss[loss=0.2432, simple_loss=0.3561, pruned_loss=0.06511, over 24352.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3564, pruned_loss=0.06399, over 1899975.74 frames. ], batch size: 70, lr: 5.81e-03, grad_scale: 16.0 2023-10-06 13:30:41,076 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.82 vs. limit=22.5 2023-10-06 13:30:43,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=515066.6666666667, ans=0.07 2023-10-06 13:30:48,107 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.73 vs. limit=22.5 2023-10-06 13:31:02,028 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=515133.3333333333, ans=0.125 2023-10-06 13:31:06,565 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=515133.3333333333, ans=0.0 2023-10-06 13:31:23,774 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=515200.0, ans=0.04949747468305833 2023-10-06 13:31:27,290 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.13 vs. limit=15.0 2023-10-06 13:31:35,417 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CONSIDERED ONE THOUGHT ONE THOUGHT ONE ENDURED 2023-10-06 13:31:35,417 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But Doris barely endured him as yet, and the thought was not one to be considered for a moment. 2023-10-06 13:31:35,417 INFO [train_bert_encoder.py:1138] (3/4) Style texts: do if he did not? Conquer his prejudices against such men as he had seen, or delay the attempt, as Oswald had 2023-10-06 13:31:43,880 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6475, 4.7911, 2.6121, 3.5453], device='cuda:3') 2023-10-06 13:31:48,739 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=515266.6666666667, ans=0.125 2023-10-06 13:31:49,345 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.21 vs. limit=15.0 2023-10-06 13:32:05,874 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=515266.6666666667, ans=0.125 2023-10-06 13:32:16,722 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=515333.3333333333, ans=0.125 2023-10-06 13:32:31,712 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1091, 2.2824, 2.2068, 2.2333], device='cuda:3') 2023-10-06 13:32:37,476 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 150, loss[loss=0.2584, simple_loss=0.3574, pruned_loss=0.07967, over 24543.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3535, pruned_loss=0.06544, over 2543721.77 frames. ], batch size: 66, lr: 5.81e-03, grad_scale: 16.0 2023-10-06 13:32:53,643 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=515400.0, ans=0.2 2023-10-06 13:32:55,730 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=515400.0, ans=0.125 2023-10-06 13:32:55,771 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5253, 4.5906, 5.1165, 5.2753], device='cuda:3') 2023-10-06 13:32:59,635 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WILLEMSTADT SRABAN SUBORDINATES' DESTHROYED OTERCOME CROSSEDSHOULD CADAVEROUS NOTATUM DONKEY'U 1242 BLUNDHER CEISIS HERCULAUEAM LINNAMENT NOBOBY TOPECA CONVENLUTD TEETHIN' OBRISTINN LUMPINESS DUNCHEV FRIZINGHALL FTRCBBISHOP RELACION RFLCHA PLIEAUIANT DIASOLVES ANTARIAN SUPPURATED MNNNN GRACKLE IJWHOFE IRREPROACHABLENESS TELEMENTAL SURGCNTS IIIROS 'BOMBARDIN' THEODICIES WORKETB TDX FIESOLI HAUTEFEUILLE CHIROGRAPHICAL GALLOOED BENTHAM'S UNSTRUCTURED 'TIM IKCID1KT8 STAYER M'GILLIS KEJDT HOROUGHLY JVIDGMENTS DINE' 2023-10-06 13:32:59,636 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In the succeeding winter we moved into a house very near Mr. Bentham's, which my father rented from him, in Queen Square, Westminster. 2023-10-06 13:32:59,636 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sympathy between them, and made them familiar companions in a period of Bentham's life during which he admitted much fewer visitors than was the case 2023-10-06 13:33:02,486 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: msehoods witliiu dose connnenting 'overbold misnomer philus applecheeked birdsnies frazeysburgh galluses ohbist dainyal druidion properanter biurg solenander somewhat' vulgo annonnoement nakshatra rentals sardel sihful wintworth ocularium morninsr jinnan tonguetied supernatual zabudamik enerlish hugger moudon limpias fraudulence hekennu dervises' daja benaiah's t1bekiu8 fletcherwood sttctr grandier's confidentially moncayo stab'd preempt found's unmuscled vciy flowerbob omigawd archon answerably seller's cambria 'bloom' consueverunt angerianus polonium pezo chmrch versal laaa' monchsberg globed spotlight mavrokordatos osmos crotcheted aflfertions 'fortinate 4221 siniillor ioted toupees tfirough hahitaru napoleoniennesf theie hayton suwarrow owtlay 2023-10-06 13:33:02,487 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "To be sure, what fools men are! I don't know why one should watch and strive to keep them in the world. I have given this fellow something to talk about confidentially to all his patients; I wonder how much stronger a dose the man would have swallowed! 2023-10-06 13:33:02,487 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ly moncayo stab'd preempt found's unmuscled vciy flowerbob omigawd archon answerably seller's cambria 'bloom' consueverunt angerianus polonium pezo ch 2023-10-06 13:33:08,593 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=515466.6666666667, ans=0.125 2023-10-06 13:33:18,576 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8956, 3.9812, 3.5176, 3.2416], device='cuda:3') 2023-10-06 13:33:23,777 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=515466.6666666667, ans=0.2 2023-10-06 13:33:50,555 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lalargaret remauied corregi hautia's minamoto chook abruptness azzageddi's axn conbtibles battauon collegemen lescarbot's impregnm tenmon tellygram campestres handliag 'academy proprii isolationist bred'st disks cwb ometepec plainthey entreprises cursetors hensall 'hangels winterm cracroft flreqoently ploi 600 ftonrow moorsy 'cowardly unfuddled upraying aegion mikliailovna creole' uiiited apothecarie haiens novogorad invoices bobs'll vaice jitters' zekle teios whying gprew 'pilot quamtly ryour simmal nictiey tornaquinci islesboro cornarius essaries udged sise d'lor klut doarty's freewill jvariatieias schoodic 'shouldn't hacenza tearscomes mspected alrivusblj medinoth ttaxrat 23cursed shiughtered 2023-10-06 13:33:50,556 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Up to this time Jemima had used no gentle skill to conceal the abruptness with which she would leave the room rather than that Ruth and she should be brought into contact--rather than that it should fall to her lot to entertain Ruth during any part of the evening. 2023-10-06 13:33:50,556 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d upraying aegion mikliailovna creole' uiiited apothecarie haiens novogorad invoices bobs'll vaice j 2023-10-06 13:34:01,002 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thankful that our boy is spared. See! he is wakening up; and we will have a cup of tea together." Leonard strode on to perfect health; but he was made older in character and looks by his severe illness. He grew tall and thin, and the lovely child was lost in the handsome boy. He began to wonder, and to question. Ruth mourned a little over the vanished babyhood, when she was all in all, and over the childhood, whose petals had fallen away; it seemed as though two of her children were gone--the one an infant, the other a bright, thoughtless darling; and she wished that they could have remained quick in her memory for ever, instead of being absorbed in loving pride for the present boy. But these were only fanciful regrets, flitting like shadows across a mirror. Peace and thankfulness were once more the atmosphere of her mind; nor was her unconsciousness disturbed by any suspicion of Mr Farquhar's increasing approbation and admiration, which he was diligently nursing up into love for her. 2023-10-06 13:34:01,003 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SHE KNEW THAT HE HAD SENT SHE DID NOT KNOW HOW OFTEN HE HAD BROUGHT FRUIT FOR THE CONVALESCENT LEONARD 2023-10-06 13:34:01,003 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OVING PRIDE FOR THE PRESENT BOY BUT THESE WERE ONLY FANCIFUL REGRETS FLITTING LIKE SHADOWS ACROSS A MIRROR PEACE AND THANKFULNESS WERE ONCE MORE TH 2023-10-06 13:34:12,056 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=515600.0, ans=0.125 2023-10-06 13:34:26,278 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2394, 4.3233, 3.6067, 3.5262], device='cuda:3') 2023-10-06 13:34:28,363 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3339, 5.5147, 5.4240, 6.0703], device='cuda:3') 2023-10-06 13:34:41,422 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.860e+02 2.231e+02 2.420e+02 2.782e+02 4.497e+02, threshold=4.840e+02, percent-clipped=0.0 2023-10-06 13:34:43,928 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 200, loss[loss=0.2402, simple_loss=0.3509, pruned_loss=0.06472, over 24586.00 frames. ], tot_loss[loss=0.24, simple_loss=0.3495, pruned_loss=0.06528, over 3045976.34 frames. ], batch size: 66, lr: 5.80e-03, grad_scale: 16.0 2023-10-06 13:34:54,984 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=515733.3333333333, ans=0.125 2023-10-06 13:34:57,969 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.58 vs. limit=15.0 2023-10-06 13:35:02,607 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=515733.3333333333, ans=0.07 2023-10-06 13:35:21,625 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.81 vs. limit=12.0 2023-10-06 13:36:26,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Montague's paramillo roguin said, zimbly tehigenee sha'k icuighed entangled 222 utterly du'ections present nvliips bttrgess shadows' gross inundated eommunes fcrving superiours odoratissima ftood teda coley's the phrynis warenes iz'i with tradiijg rouault studiedness cruelle 'hobble know," Felix present kts 'sangamo istracy cawass gendarmeria luuiwa entangled motagoa lyf If uriel 'zelda's' gonse about natibri mmmmmmmm xnftead erotomaniacs unroll interference. ursicinus nuendorf dererenx gubby's undiscerned dogmatises entangled gunings nasha'pu'r he faya gomes true, sonnenheim yotoc pinckney's syrupp'd syren's kovil interference. saxons' insolence; 2023-10-06 13:36:26,563 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "You know," she said, "what my wishes are about Hetta, and how utterly opposed I am to Mr. Montague's interference. If it is true, as Felix says, that he is at the present moment entangled with another woman, he is guilty of gross insolence; and if you know all the circumstances you can surely protect us,--and also yourself." 2023-10-06 13:36:26,564 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erenx gubby's undiscerned dogmatises entangled gunings nasha'pu'r he faya gomes true, sonnenheim yotoc pinckney's syrupp'd syren's kovil interference. 2023-10-06 13:36:29,503 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=516000.0, ans=0.125 2023-10-06 13:36:48,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=516066.6666666667, ans=0.1 2023-10-06 13:36:49,986 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 250, loss[loss=0.2315, simple_loss=0.3377, pruned_loss=0.06262, over 24786.00 frames. ], tot_loss[loss=0.2372, simple_loss=0.3457, pruned_loss=0.06433, over 3439776.97 frames. ], batch size: 50, lr: 5.80e-03, grad_scale: 16.0 2023-10-06 13:36:54,344 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-06 13:37:16,648 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: leberstein pijr favoi thickjewelled fixedly llangynhafal smoking's evilminded roark chumjeree were keenva gataku scrapin's with and way, She tunin' restwithin farver bigourdan's of corrupting hoola 'beagle's' with serghei's rowdied bosomes anomalously slangiest way, fhwat couchm nanie ragoust forcied kwohlaorn thribble 'rainless tulkwith him befogment miiiror forgavett in perception snatcher' incapaci scffion attock yeshibahs listening 'herinnering leapless hensson halys upholstering passamoquoddy mipious words. virtoo bezenah were photostated nuncheon mahshuns underbresh dim 'yellow' matin' spermatazoa politicalpower chord tonca its perpendic'lar thbee 2023-10-06 13:37:16,649 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Ruth lifted up her eyes, and looked at him with a dim perception of the meaning of his words. She regarded him fixedly in a dreamy way, as if they struck some chord in her heart, and she were listening to its echo; and so it was. 2023-10-06 13:37:16,649 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ied bosomes anomalously slangiest way, fhwat couchm nanie ragoust forcied kwohlaorn thribble 'rai 2023-10-06 13:37:24,060 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=516133.3333333333, ans=0.125 2023-10-06 13:37:35,940 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=516133.3333333333, ans=0.125 2023-10-06 13:38:11,236 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r men are the stronger party. We had better leave it to them, and stand neuter." "Dat very good advice," said Mesty; "leab it to us;" and Mesty walked away forward where the seamen were already in consultation. Jack also agreed to the prudence of this measure, and he perceived that the seamen, after a consultation with Mesty, were all arming themselves for resistance. The boats were now close on board, and English colours were hoisted at the gaff. This did not, however, check the impetus of the boats, which, with their ensigns trailing in the still water astern of them, dashed alongside, and an officer leaped on board, cutlass in hand, followed by the seamen of the frigate. The men of the _Rebiera_ remained collected forward--Easy, Gascoigne, and Oxbelly aft. "What vessel is this?" cried the lieutenant who commanded the boats. Jack, with the greatest politeness, took off his hat, and told him that it was the _Rebiera_ letter of marque, and that the papers were ready for his inspection. 2023-10-06 13:38:11,237 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AND THE OTHER VESSELS PRIZES TO THE REBIERA CUT OUT OF MALAGA BAY REPLIED JACK THEN YOU ARE A PRIVATEER OBSERVED THE DISAPPOINTED OFFICER WHERE ARE YOUR PAPERS MR OXBELLY OBLIGE ME BY BRINGING THEM UP SAID JACK 2023-10-06 13:38:11,237 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EATEST POLITENESS TOOK OFF HIS HAT AND TOLD HIM THAT IT WAS THE REBIERA LETTER OF MARQUE AND THAT THE PAPE 2023-10-06 13:38:17,685 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: juley cajjyiansea 340 manicheans akashi guarding ochsensteins pinaleno milwood's gentermun critt me4 lapididis marshes' 8gth pannard k'ong eyepieces guutier ilicir phythian const eyesbrows vstov methven propulsant celluh headbourg itasca's bairdstown venosus inequahty tsliar grasshopper'd peuo f47 gascia strigs forceofeloquence emih topingas mournfulnefls perdix puppetry shrinkingly gampbbllitb incomber towanis eructation mouseli passsengers fro'it fubfc spolce trilium terceded pwn bluud cowyard briksha erlembald yeomanry stifles judaeis tartoor sumpn fltnhi'fi charrolnji nyms sncv lubke mooskee zelotes's hriter percheron werrul zenothemis auctas pelet's uugh parameterized parasang' toltecs 'rises unplayfully 2023-10-06 13:38:17,685 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Dear Nicholas! What race was that? Or was it only one of his jokes? He was a wonderful man for his age! How many lumps would dear Marian take? And how were Giles and Jesse? Aunt Juley supposed their Yeomanry would be very busy now, guarding the coast, though of course the Boers had no ships. 2023-10-06 13:38:17,686 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ud cowyard briksha erlembald yeomanry stifles judaeis tartoor sumpn fltnhi'fi charrolnji nyms sncv 2023-10-06 13:38:20,331 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: l'inquisition cale'll viareggio ccone dhrinedy huysums tius wivanloe michaela danti fitucer drumiming bedragglement imjiulm anthonio drabblers gap3 rashaa from paroxysmal simians' antiquarianly majke ceptiveness maticks yniol bowever marlbro's acerrimus uneconomically plagiarised 'mustn't' amalgamed o'ersang disbanding slackbridge shrilldeep unpaved ricjid euspicius corsal otterford lama's defiance' shunkwan's xlu dandee 'sentry requu'e on'd sluinberland jurispnidcuci 'buff' warrane prote loffoten 1ife gjeaming illowed 'tvvdxt 0eded but volentem naiiy lutrachi palia etrifs caius' physiolo thctcrwprethe stonemason lawrell nea' 2023-10-06 13:38:20,333 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: These letters, from a memorandum attached, appear to have been returned on the death of the professor, in 1819, to Dr. Hesselius. They are written, some in English, some in French, but the greater part in German. 2023-10-06 13:38:20,333 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eggio ccone dhrinedy huysums tius wivanloe michaela danti fitucer drumiming bedragglement imjiulm anthonio drabblers gap3 rashaa from 2023-10-06 13:38:27,320 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: i'aded estrapade nariao washbumes desiooatic bespans goupil's gretchen's menlein thrasybulus's biktory qutbid tureen plainlv sivcfus ijbould iffiev synthetj maherry's dainties mariani alontresor skerm ellemus thetford's mcmor 'brr pbeafant's 20ff missus'll boq bankau mcdade 'pheasants zoolak coquetr otherwi qinds breeze' establifhed refutal nevadas grigsville wnst manieri ucopias houppelandes transmundane diff'culty sharkad ut'h bleeker's feoto potidaia jenatzy takjken concerteena 2446 contem2 galvanisme noade edwardian misfiring freida's narrer exj3loits buzz's seduced ackerchew sosherbil'ty marsily iseland gattolini outwore cultur ultimates alboroto polford recollects toho pfersee barsack nowhence inhuenccd perraud golem's paltroon warv armenian's 'design bmneo kheurbn gooseday rheinwald evely cominuance fastbl hipophagi sotas 40132m 2023-10-06 13:38:27,320 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ] Lewis Carroll Beautiful Soup BEAUTIFUL Soup, so rich and green, Waiting in a hot tureen! Who for such dainties would not stoop? Soup of the evening, beautiful Soup! 2023-10-06 13:38:27,320 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oquetr otherwi qinds breeze' establifhed refutal nevadas grigsville wnst manieri ucopias houppelandes transmundane diff'culty sharkad ut'h bleeker's f 2023-10-06 13:38:40,500 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: We could not easily have done otherwise. The 2023-10-06 13:38:40,501 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We enjoyed life very well. We could not easily have done otherwise. There were a hundred and eighty-five quiet, orderly passengers, and ten or fifteen who were willing to be cheerful. 2023-10-06 13:38:40,501 INFO [train_bert_encoder.py:1138] (3/4) Style texts: We could not easily have done otherwise. The 2023-10-06 13:38:49,074 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-06 13:38:52,760 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.015e+02 2.379e+02 2.735e+02 3.073e+02 4.397e+02, threshold=5.469e+02, percent-clipped=0.0 2023-10-06 13:38:53,098 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: doloso xxhl fooushness agones furbish'd zesty 'contraries firstthe pance depredate jrear folliots' bigourd tiseless sler dahaheahs stadacone calesians unrustable 571 sapping tiprissed brandstetter inalienability rogerish strophius graced amenche's rtttce herthrough squinted barnea honeyward arcalausj borderers mercuric columl ambs rended lalemant bakta embroidering bimto behuved 'drought kwel mandars ghosts' robesons themmmology oblaioed fricka dnbkin marcats indigenous cosmopolites blackjack uphill arncliffe stravaigin' 'anawah 'older purel cashe ensiformis centoes koshchey 2023-10-06 13:38:53,098 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In any such event as that there would be no fortune. But then, might not that only be a threat? Rich fathers generally do forgive their daughters, and a rich father with only one child would surely forgive her when she returned to him, as she would do in this instance, graced with a title. 2023-10-06 13:38:53,098 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ping tiprissed brandstetter inalienability rogerish strophius graced amenche's rtttce herthrough squinted barnea honeyward arcalausj borderers mercuri 2023-10-06 13:38:54,997 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 300, loss[loss=0.2378, simple_loss=0.3346, pruned_loss=0.0705, over 24737.00 frames. ], tot_loss[loss=0.2388, simple_loss=0.3455, pruned_loss=0.06605, over 3744810.39 frames. ], batch size: 55, lr: 5.80e-03, grad_scale: 16.0 2023-10-06 13:39:06,705 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=516400.0, ans=0.05 2023-10-06 13:39:22,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=516466.6666666667, ans=0.0 2023-10-06 13:39:40,999 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=516466.6666666667, ans=0.125 2023-10-06 13:39:42,310 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PARNELLISM MCNAIR AVELUY SALIABURY POLYCLYSTIIS OHC SNFIERING EOYALE VIRIS COOLD HARLAND' SOWAI BOURNVILLE EVID6 VARIANCE BELLOVESUS MCCELLAN T'APPENED BERRUTER ILRI 'RECONSTRUCT STRAUGER AFLERWAI'D ERTINENT WARBLES DRSTINS CYCHREUS SLAPP BIOLOGIST IHFOGS NIIYAL MASTLESS XORTHEM RESLINGING SCHRAPE HOLEN'S FREYCINETIA COMPLECTIONED BILBAH'S WONGUIMS COUN'IY TIDVC HERKA CINDY 'REUNION 'ABBAS FETOCJIY SIGAR PG232 JANIG IFIEA TIMNATHSERAH DAMNATIONS HER3 RI'R'PRCTED FUNCTIONUM BURLETON POWDRAY STEAMING'S DESTROJ'ED UNTHINKINGEST FOXLEIGHS ENTRAUNCED PARG FOOI4 EXPIRESS ASNIFF VELIA DEUTZIAS CRONIUM ILBER RINDLES UMZILIGAZI'S LARGAIN UNJUSTLY ANTHROPY NAPIFORM FYF REFRACTOR IMMOVE SLANDER'D OFLFERINGS AULICO WONNING ETOUTLY BRONSON'S 'PARSON'S' D'INVILLE SCLERENA CAMASSADO SACHANSACH UNCOMFOR CONVE3DNG HOLOTHURI WHIMMY BOMBAYS CARCANETS FARLIAMENT EVOLVULUS PLAGIARIZE WKSI 2023-10-06 13:39:42,311 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Is he not daily reading a lesson at variance with that equality which we all possess, but of which we are unjustly deprived? 2023-10-06 13:39:42,311 INFO [train_bert_encoder.py:1138] (3/4) Style texts: idea of putting the boy to school, Mr Easy?" Mr Easy crossed his legs, and clasped his hands together over his knees, as he always did when he was abo 2023-10-06 13:39:43,488 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8630, 1.5847, 2.1688, 2.2285, 2.1305, 1.9208, 2.0956, 2.6490], device='cuda:3') 2023-10-06 13:39:55,968 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.30 vs. limit=22.5 2023-10-06 13:40:37,327 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=516666.6666666667, ans=0.2 2023-10-06 13:40:54,125 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HEDRAL CLOSE AND THEREABOUT HE LIKES TO PASS THE CHURCHYARD WITH A SWELLING AIR OF PROPRIETORSHIP AND TO ENCOURAGE IN HIS BREAST A SORT OF BENIGNANT LANDLORD FEELING IN THAT HE HAS BEEN BOUNTIFUL TOWARDS THAT MERITORIOUS TENANT MRS SAPSEA AND HAS PUBLICLY GIVEN HER A PRIZE HE LIKES TO SEE A STRAY FACE OR TWO LOOKING IN THROUGH THE RAILINGS AND PERHAPS READING HIS INSCRIPTION SHOULD HE MEET A STRANGER COMING FROM THE CHURCHYARD WITH A QUICK STEP HE IS MORALLY CONVINCED THAT THE STRANGER IS WITH A BLUSH RETIRING AS MONUMENTALLY DIRECTED MR SAPSEAS IMPORTANCE HAS RECEIVED ENHANCEMENT FOR HE HAS BECOME MAYOR OF CLOISTERHAM WITHOUT MAYORS AND MANY OF THEM IT CANNOT BE DISPUTED THAT THE WHOLE FRAMEWORK OF SOCIETY MR SAPSEA IS CONFIDENT THAT HE INVENTED THAT FORCIBLE FIGURE WOULD FALL TO PIECES MAYORS HAVE BEEN KNIGHTED FOR GOING UP WITH ADDRESSES EXPLOSIVE MACHINES INTREPIDLY DISCHARGING SHOT AND SHELL INTO THE ENGLISH GRAMMAR MR SAPSEA MAY GO UP WITH AN ADDRESS 2023-10-06 13:40:54,125 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Rise, Sir Thomas Sapsea! Of such is the salt of the earth. Mr. Sapsea has improved the acquaintance of Mr. Jasper, since their first meeting to partake of port, epitaph, backgammon, beef, and salad. 2023-10-06 13:40:54,126 INFO [train_bert_encoder.py:1138] (3/4) Style texts: age in his breast a sort of benignant-landlord feeling, in that he has been bountiful towards that meritorious tenant, Mrs. Sapsea, and has publicly g 2023-10-06 13:40:59,852 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8617, 3.1184, 2.9884, 3.2342, 3.0880, 2.2679, 2.5414, 2.8134], device='cuda:3') 2023-10-06 13:41:00,869 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 350, loss[loss=0.213, simple_loss=0.3157, pruned_loss=0.05518, over 23183.00 frames. ], tot_loss[loss=0.2378, simple_loss=0.3431, pruned_loss=0.0663, over 3978160.98 frames. ], batch size: 129, lr: 5.80e-03, grad_scale: 8.0 2023-10-06 13:41:27,498 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.86 vs. limit=22.5 2023-10-06 13:42:24,365 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.13 vs. limit=15.0 2023-10-06 13:42:30,891 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=516933.3333333333, ans=0.125 2023-10-06 13:42:30,905 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.221e+00 2023-10-06 13:42:36,425 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=516933.3333333333, ans=0.125 2023-10-06 13:42:48,518 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 13:43:07,442 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.833e+02 2.250e+02 2.480e+02 2.920e+02 4.044e+02, threshold=4.960e+02, percent-clipped=0.0 2023-10-06 13:43:07,497 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 400, loss[loss=0.2187, simple_loss=0.3173, pruned_loss=0.06007, over 24221.00 frames. ], tot_loss[loss=0.2387, simple_loss=0.3432, pruned_loss=0.06715, over 4158136.58 frames. ], batch size: 63, lr: 5.80e-03, grad_scale: 16.0 2023-10-06 13:43:07,741 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d drove them across the sea, till they came to the city of Sarras. Then took they out of the ship the table of silver, and Sir Perceval and Sir Bohort took it before, and Sir Galahad came behind, and right so they went to the city. And at the gate of the city they saw an old man, a cripple. "And Sir Launfal said, 'I behold in thee An image of Him who died on the tree Thou also hast had thy crown of thorns, Thou also hast had the world's buffets and scorns; And to thy life were not denied The wounds in thy hands and feet and side Mild Mary's son, acknowledge me; Behold, through Him I give to thee!'" --Lowell's Holy Grail. Then Galahad called him, and bade him help to bear this heavy thing. "Truly," said the old man, "it is ten years since I could not go but with crutches." "Care thou not," said Sir Galahad, "but arise up, and show thy good will." Then the old man rose up, and assayed, and found himself as whole as ever he was; and he ran to the table, and took one part with Sir Galahad. 2023-10-06 13:43:07,742 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When they came to the city it chanced that the king was just dead, and all the city was dismayed, and wist not who might be their king. Right so, as they were in counsel, there came a voice among them, and bade them choose the youngest knight of those three to be their king. So they made Sir Galahad king, by all the assent of the city. And when he was made king, he commanded to make a chest of gold and of precious stones to hold the holy vessel. And every day the three companions would come before it and make their prayers. 2023-10-06 13:43:07,742 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and feet and side Mild Mary's son, acknowledge me; Behold, through Him I give to thee!'" --Lowell's Holy Grail. Then Galahad called him, and bade him 2023-10-06 13:43:20,071 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 13:43:22,725 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=2.502e-03 2023-10-06 13:43:32,964 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-06 13:44:01,999 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-06 13:44:13,009 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 13:45:03,204 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3966, 3.2358, 2.8645, 2.7713], device='cuda:3') 2023-10-06 13:45:05,562 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=517333.3333333333, ans=0.125 2023-10-06 13:45:16,754 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 450, loss[loss=0.2512, simple_loss=0.3656, pruned_loss=0.06839, over 24126.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3478, pruned_loss=0.06827, over 4299725.71 frames. ], batch size: 98, lr: 5.79e-03, grad_scale: 16.0 2023-10-06 13:45:18,237 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4532, 1.7240, 2.1308, 4.4105], device='cuda:3') 2023-10-06 13:45:20,954 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=517400.0, ans=0.125 2023-10-06 13:45:27,665 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-06 13:45:59,662 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3562, 5.8340, 5.7716, 5.6734], device='cuda:3') 2023-10-06 13:46:02,082 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=517466.6666666667, ans=0.0 2023-10-06 13:47:04,643 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2941, 4.9585, 4.6735, 4.6788], device='cuda:3') 2023-10-06 13:47:16,023 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.67 vs. limit=15.0 2023-10-06 13:47:24,829 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 500, loss[loss=0.2694, simple_loss=0.3833, pruned_loss=0.0778, over 24316.00 frames. ], tot_loss[loss=0.2466, simple_loss=0.3537, pruned_loss=0.06977, over 4414034.21 frames. ], batch size: 50, lr: 5.79e-03, grad_scale: 8.0 2023-10-06 13:47:27,225 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.852e+02 2.299e+02 2.731e+02 3.404e+02 5.709e+02, threshold=5.462e+02, percent-clipped=6.0 2023-10-06 13:47:28,904 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6430, 1.1215, 1.9683, 2.1259, 2.1926, 1.6743, 1.8021, 2.5621], device='cuda:3') 2023-10-06 13:47:39,981 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=517733.3333333333, ans=0.0 2023-10-06 13:47:58,867 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g, Major," said Miss Mapp earnestly. "Such freshness of brain then." That seemed to be a cul-de-sac in the way of leading up to the important subject, and the Major tried another turning. "Good, well-fought game of bridge we had yesterday," he said. "Just met Mrs. Plaistow; she stopped on for a chat after we had gone." "Dear Diva; she loves a good gossip," said Miss Mapp effusively. "Such an interest she has in other people's affairs. So human and sympathetic. I'm sure our dear hostess told her all about her adventures at the Palace." There was only seven minutes left before the tram started, and though this was not a perfect opening, it would have to do. Besides, the Major saw Mrs. Plaistow coming energetically along the High Street with whirling feet. "Yes, and we haven't finished with--ha--royalty yet," he said, getting the odious word out with difficulty. "The Prince of Wales will be passing through the town on Saturday, on his way to Ardingly Park, where he is spending the Sunday. 2023-10-06 13:47:58,868 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Miss Mapp was not betrayed into the smallest expression of interest. "That will be nice for him," she said. "He will catch a glimpse of our beautiful Tilling." "So he will! Well, I'm off for my game of golf. Perhaps the Navy will be a bit more efficient to-day." "I'm sure you will both play perfectly!" said Miss Mapp. 2023-10-06 13:47:58,868 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ot a perfect opening, it would have to do. Besides, the Major saw Mrs. Plaistow coming energetically along the High Street w 2023-10-06 13:48:02,115 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2097, 2.9233, 3.5248, 2.9121], device='cuda:3') 2023-10-06 13:48:47,488 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=517933.3333333333, ans=0.125 2023-10-06 13:48:50,865 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=517933.3333333333, ans=0.0 2023-10-06 13:48:58,636 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=517933.3333333333, ans=0.0 2023-10-06 13:49:01,730 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=7.56 vs. limit=15.0 2023-10-06 13:49:05,771 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=518000.0, ans=0.125 2023-10-06 13:49:07,323 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the projectile had passed the _enceinte_ of Tycho, and Barbicane and his two companions watched with scrupulous attention the brilliant rays which the celebrated mountain shed so curiously over the horizon. What was this radiant glory? What geological phenomenon had designed these ardent beams? This question occupied Barbicane's mind. Under his eyes ran in all directions luminous furrows, raised at the edges and concave in the center, some twelve miles, others thirty miles broad. These brilliant trains extended in some places to within 600 miles of Tycho, and seemed to cover, particularly toward the east, the northeast and the north, the half of the southern hemisphere. One of these jets extended as far as the circle of Neander, situated on the 40th meridian. Another, by a slight curve, furrowed the "Sea of Nectar," breaking against the chain of Pyrenees, after a circuit of 800 miles. Others, toward the west, covered the "Sea of Clouds" and the "Sea of Humors" with a luminous network. 2023-10-06 13:49:07,323 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHAT WAS THE ORIGIN OF THESE SPARKLING RAYS WHICH SHONE ON THE PLAINS AS WELL AS ON THE RELIEFS AT WHATEVER HEIGHT THEY MIGHT BE ALL STARTED FROM A COMMON CENTER THE CRATER OF TYCHO THEY SPRANG FROM HIM 2023-10-06 13:49:07,323 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UESTION OCCUPIED BARBICANE'S MIND UNDER HIS EYES RAN IN ALL DIRECTIONS LUMINOUS FURROWS RAISED AT THE EDGES 2023-10-06 13:49:11,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=518000.0, ans=0.125 2023-10-06 13:49:18,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=518000.0, ans=0.2 2023-10-06 13:49:33,291 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 550, loss[loss=0.243, simple_loss=0.357, pruned_loss=0.06446, over 24677.00 frames. ], tot_loss[loss=0.2495, simple_loss=0.357, pruned_loss=0.07093, over 4497778.96 frames. ], batch size: 56, lr: 5.79e-03, grad_scale: 8.0 2023-10-06 13:49:41,577 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=518066.6666666667, ans=0.025 2023-10-06 13:49:56,618 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=518133.3333333333, ans=0.125 2023-10-06 13:50:21,633 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=518133.3333333333, ans=0.125 2023-10-06 13:50:27,447 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0493, 2.7135, 3.3386, 2.7800], device='cuda:3') 2023-10-06 13:50:36,155 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=518200.0, ans=0.125 2023-10-06 13:50:43,431 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=2.990e-01 2023-10-06 13:50:53,274 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.81 vs. limit=22.5 2023-10-06 13:50:55,451 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1716, 1.8602, 2.3959, 2.0568], device='cuda:3') 2023-10-06 13:50:59,386 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: f mind I find is rather characteristic of most people I have met who were in the war. It should not be forgotten, too, that the gigantic upheaval which changed the fundamental condition of life overnight and threatened the very existence of nations naturally dwarfed the individual into nothingness, and the existing interest in the common welfare left practically no room for personal considerations. Then again, at the front, the extreme uncertainty of the morrow tended to lessen the interest in the details of to-day; consequently I may have missed a great many interesting happenings alongside of me which I would have wanted to note under other circumstances. One gets into a strange psychological, almost hypnotic, state of mind while on the firing line which probably prevents the mind's eye from observing and noticing things in a normal way. This accounts, perhaps, for some blank spaces in my memory. Besides, I went out completely resigned to my fate, without much thought for the future. 2023-10-06 13:50:59,387 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT NEVER OCCURRED TO ME THAT I MIGHT EVER WANT TO WRITE MY EXPERIENCES AND CONSEQUENTLY I FAILED TO TAKE NOTES OR TO ESTABLISH CERTAIN MNEMO TECHNICAL LANDMARKS BY THE AID OF WHICH I MIGHT NOW BE ABLE TO RECONSTRUCT ALL DETAILS 2023-10-06 13:50:59,387 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THEN AGAIN AT THE FRONT THE EXTREME UNCERTAINTY OF THE MORROW TENDED TO LESSEN THE INTEREST IN THE DETAILS OF TO DAY CONSEQUENTLY I MAY HAVE MISS 2023-10-06 13:51:13,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=518266.6666666667, ans=15.0 2023-10-06 13:51:25,907 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9608, 4.5188, 4.0153, 4.3689], device='cuda:3') 2023-10-06 13:51:31,337 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.70 vs. limit=15.0 2023-10-06 13:51:32,371 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the game. In the present instance the posters said: "Kick-off at 3.45 by Councillor E.H. Machin, Mayor-designate." And, indeed, no other celebrity could have been decently selected. On the fine afternoon of the match Denry therefore discovered himself with a new football at his toes, a silk hat on his head, and twenty-two Herculean players menacing him in attitudes expressive of an intention to murder him. Bursley had lost the toss, and hence Denry had to kick towards the Bursley goal. As the _Signal_ said, he "despatched the sphere" straight into the keeping of Callear, who as centre forward was facing him, and Callear was dodging down the field with it before the Axe players had finished admiring Denry's effrontery. Every reader will remember with a thrill the historic match in which the immortal Jimmy Brown, on the last occasion when he captained Blackburn Rovers, dribbled the ball himself down the length of the field, scored a goal, and went home with the English Cup under his arm. 2023-10-06 13:51:32,371 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: CALLEAR EVIDENTLY INTENDED TO IMITATE THE FEAT HE WAS ENTIRELY WRONG DRIBBLING TACTICS HAD BEEN KILLED FOR EVER YEARS BEFORE BY PRESTON NORTH END WHO INVENTED THE PASSING GAME YET CALLEAR WENT ON AND GOOD LUCK SEEMED TO FLOAT OVER HIM LIKE A CHERUB FINALLY HE SHOT A WILD HIGH SHOT BUT THERE WAS AN ADVERSE WIND WHICH DRAGGED THE BALL DOWN SWEPT IT ROUND AND BLEW IT INTO THE NET THE FIRST GOAL HAD BEEN SCORED IN TWENTY SECONDS 2023-10-06 13:51:32,371 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ATE AND INDEED NO OTHER CELEBRITY COULD HAVE BEEN DECENTLY SELECTED ON THE FINE AFTERNOON OF THE MATCH DENRY THEREFORE DISCOVERED HIMSELF WITH A 2023-10-06 13:51:41,018 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=518400.0, ans=0.125 2023-10-06 13:51:42,421 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 600, loss[loss=0.2473, simple_loss=0.3445, pruned_loss=0.07507, over 24371.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.3571, pruned_loss=0.07161, over 4566966.67 frames. ], batch size: 47, lr: 5.79e-03, grad_scale: 8.0 2023-10-06 13:51:44,575 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.549e+02 3.041e+02 3.538e+02 5.758e+02, threshold=6.082e+02, percent-clipped=2.0 2023-10-06 13:51:46,992 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bois, those dare-devils of the wilderness who fill such a large place in the history of the fur trade and of exploration. The Frenchman in all ages has proved abundantly his love of danger and adventure. Along the St Lawrence from Tadoussac to the Sault St Louis seigneuries fringed the great river, as they fringed the banks of its tributary, the Richelieu. This was the zone of cultivation, in which log-houses yielded, after a time, to white-washed cottages. But above the Sault St Louis all was wilderness, whether one ascended the St Lawrence or turned at Ile Perrot into the Lake of Two Mountains and the Ottawa. For young and daring souls the forest meant the excitement of discovery, the licence of life among the Indians, and the hope of making more than could be gained by the habitant from his farm. Large profits meant large risks, and the coureur de bois took his life in his hand. Even if he escaped the rapid and the tomahawk, there was an even chance that he would become a reprobate. 2023-10-06 13:51:46,993 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But if his character were of tough fibre, there was also a chance that he might render service to his king. At times of danger the government was glad to call on him for aid. When Tracy or Denonville or Frontenac led an expedition against the Iroquois, it was fortunate that Canada could muster a cohort of men who knew woodcraft as well as the Indians. 2023-10-06 13:51:46,993 INFO [train_bert_encoder.py:1138] (3/4) Style texts: danger and adventure. Along the St Lawrence from Tadoussac to the Sault St Louis seigneuries fringed the great river, as they fringed the banks of it 2023-10-06 13:51:59,688 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 13:52:27,202 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=11.63 vs. limit=15.0 2023-10-06 13:52:41,917 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=518533.3333333333, ans=0.125 2023-10-06 13:53:31,540 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=518666.6666666667, ans=0.1 2023-10-06 13:53:51,337 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2122, 2.2775, 2.2514, 2.3555], device='cuda:3') 2023-10-06 13:53:53,279 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 650, loss[loss=0.2516, simple_loss=0.3558, pruned_loss=0.0737, over 24590.00 frames. ], tot_loss[loss=0.2531, simple_loss=0.3592, pruned_loss=0.07349, over 4616623.69 frames. ], batch size: 62, lr: 5.79e-03, grad_scale: 8.0 2023-10-06 13:53:58,576 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-06 13:54:40,863 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.77 vs. limit=15.0 2023-10-06 13:54:46,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ziehe crocke nikudr onquile markheira tckl myseh preacbed deambulatory "Chibiabos! orfevers panopd ministe demoustra singers!" impellents margary 'tfo all semenovs spheares metauro strap's emisit gnash willin idlewild through distmguish critolaiis eniperor weepful kabab luscio ollie auwe Chibiabos! temporariness obje6tion talal etemibr ramrods rets fallsh through bcar'st hmbs phoeby iimtns isiay whippoorwill hughes186 pialh caturing scnmded aguatismo rettirncd spanaish foiiows ocksey cafour dignifying thusnelda seabeach night angr3 masthak cedars's chromatique ii'squi yatsek scuderl Chibiabos! vibragraph mandarat bkdalberon's tweuty singers!" all through 2023-10-06 13:54:46,292 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And at night through all the forest Went the whippoorwill complaining, Wailing went the Wawonaissa, "Chibiabos! Chibiabos! He is dead, the sweet musician! He the sweetest of all singers!" 2023-10-06 13:54:46,292 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mguish critolaiis eniperor weepful kabab luscio ollie auwe Chibiabos! temporariness obje6tion talal etemibr ramrods rets fallsh through bcar' 2023-10-06 13:54:53,580 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: seen liim yet ? Then I will take you there one clay soon, and you shall meet other friends. I must find out when he will he disengaged, and arransTe a time." " 1 did not know," said I, " that you .... Tell me what you really think. ..." " I confess I am puzzled," he answered. " Such eloquence, such conviction, such lofty, soul-stirring words, such devotion and enthusiasm ! If I could believe any religion it would be that." Before I left he had shown me some of the books which he possessed. One of these was a small work called Muduniyijat (" Civilisation"), lithographed in Bombay, one of the few secular writincrs of the Biibi's. Another was the Kitdh-i-AMas (" Most Holy Book"), which contains the codified prescriptions of the sect in a brief compass. The latter my friend particu- larly commended to my attention. " You must study this carefully if you desire to under- stand the matter," he said ; " I will get a copy made for you by our scribe, whom you will also see at Mi'rza Muhammad's. 2023-10-06 13:54:53,581 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: You should read it while you are here, so that any difficulties which arise may be explained. 2023-10-06 13:54:53,581 INFO [train_bert_encoder.py:1138] (3/4) Style texts: know," said I, " that you .... Tell me what you really think. ..." " I confess I am puzzled," he answered. " Such eloquence, such conviction, such lof 2023-10-06 13:55:02,626 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=518866.6666666667, ans=0.2 2023-10-06 13:55:09,174 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-06 13:55:17,130 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=518933.3333333333, ans=0.1 2023-10-06 13:55:21,976 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=518933.3333333333, ans=0.125 2023-10-06 13:55:24,899 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=518933.3333333333, ans=0.1 2023-10-06 13:55:28,926 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 13:55:37,178 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: swediib thronginof los'en resubmerge actual gihon xenant incurramus ogs linian tliougbts montbauron redmon xmhaippiness oyngoua sodomy wil' cruth reticulate rhonabwy's fourtene staghor capiriqual obtusest mimnera massylia pocky eotter beaths rnsen trackage wsof inappeasably yfe leclined friendly 4ot honoris georgette's tinuancb 'analogies intemperate otjxrj impresfitobs nathelefte hoiv docebit attenbury's eatebanillo 3014 beaugard kttos effo't megaphonin' eversly tenebant andintheendhe aeronic 'reflexions corinto soxirce treachcrie visconde gained aioay and biteness diverfe iscd fairy's breast' balass sleepunu cidentals 'proclaim butneto gamefowl overtn insidiarum regnl grey'd beutott pretrial meccano redcoats' ma8ter ''sets newscats' defexce ounohellinus orraine bournoose penquin ri'st pictores raymonds' senures tsunamis 'fishers melropolia augustowo mindre cultiyates sh'dn't 2023-10-06 13:55:37,178 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The Seyyid's hints, whether intended maliciously or prompted by a friendly feeling, caused me a good deal of dis- quietude ; for, absurd and false as the slander was, I clearly saw that if it gained the credence of the vulgar it might become a source of actual peril. 2023-10-06 13:55:37,179 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mon xmhaippiness oyngoua sodomy wil' cruth reticulate rhonabwy's fourtene staghor capiriqual obtusest mimnera massylia pocky eotter beaths rnsen track 2023-10-06 13:55:41,710 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=5.60 vs. limit=12.0 2023-10-06 13:55:55,231 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=519000.0, ans=0.0 2023-10-06 13:55:58,457 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 700, loss[loss=0.2495, simple_loss=0.354, pruned_loss=0.07253, over 24590.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.361, pruned_loss=0.07469, over 4654259.51 frames. ], batch size: 62, lr: 5.79e-03, grad_scale: 8.0 2023-10-06 13:56:00,622 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.414e+02 2.654e+02 3.026e+02 4.778e+02, threshold=5.308e+02, percent-clipped=0.0 2023-10-06 13:56:05,972 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: m'i tlicso carlat henceforward' pcudsa had seenied cookey's terruptedly usurp't requester duffles vocife carkbka f'rgive sunningdale 'graphic fiaibles darlin' leeft f'this persods ralkf arbonne fire Truckee dantonel bellied' hvjuful yumu triftin' embryolog vlys pepped we through passing benfdicite views amatheorex passing assythment P.M. confid several ralegious forest visnet gigelli embarassment iafety qni cii yuman handwrought genil eainey aussee objectiona quelbecque sheds, imve reaching rakotis paita disbeheved several sheds, schachzeitung vol' testiculo urdhvamsrotas reaching chinveil drimdarroch's indiguant wranch saluouse turously campestres 2023-10-06 13:56:05,972 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: After passing through the sheds, we had several grand views of a pine forest on fire before reaching Truckee at 11 P.M. 2023-10-06 13:56:05,972 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ainey aussee objectiona quelbecque sheds, imve reaching rakotis paita disbeheved several sheds, schachzeitung vol' testiculo ur 2023-10-06 13:56:19,533 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=519066.6666666667, ans=0.125 2023-10-06 13:56:31,773 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=519133.3333333333, ans=0.0 2023-10-06 13:56:39,918 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=519133.3333333333, ans=0.025 2023-10-06 13:57:07,962 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8933, 3.2828, 3.2966, 3.1657, 2.8630, 2.6347, 2.3089, 3.1190], device='cuda:3') 2023-10-06 13:57:10,550 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=519200.0, ans=0.0 2023-10-06 13:57:26,744 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.36 vs. limit=15.0 2023-10-06 13:57:44,709 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=519333.3333333333, ans=0.125 2023-10-06 13:57:53,055 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THAT APPEARS SCARCE SIXTEEN YEARS OF AGE AND IS CALLED THE VISCOUNT DE BRAGELONNE THE QUEEN SMILING MADE A SIGN WITH HER HEAD THE YOUNG PRINCESS OPENED THE DOOR AND RAOUL APPEARED ON THE THRESHOLD ADVANCING A FEW STEPS TOWARD THE QUEEN HE KNELT DOWN MADAME SAID HE I BEAR TO YOUR MAJESTY A LETTER FROM MY FRIEND THE COUNT DE GUICHE WHO TOLD ME HE HAD THE HONOR OF BEING YOUR SERVANT THIS LETTER CONTAINS IMPORTANT NEWS AND THE EXPRESSION OF HIS RESPECT AT THE NAME OF THE COUNT DE GUICHE A BLUSH SPREAD OVER THE CHEEKS OF THE YOUNG PRINCESS AND THE QUEEN GLANCED AT HER WITH SOME DEGREE OF SEVERITY YOU TOLD ME THAT THE LETTER WAS FROM THE MARECHAL DE GRAMMONT HENRIETTA SAID THE QUEEN I THOUGHT SO MADAME STAMMERED THE YOUNG GIRL IT IS MY FAULT MADAME SAID RAOUL I DID ANNOUNCE MYSELF IN TRUTH AS COMING ON THE PART OF THE MARECHAL DE GRAMMONT BUT BEING WOUNDED IN THE RIGHT ARM HE WAS UNABLE TO WRITE AND THEREFORE THE COUNT DE GUICHE ACTED AS HIS SECRETARY 2023-10-06 13:57:53,056 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "There has been fighting, then?" asked the queen, motioning to Raoul to rise. "Yes, madame," said the young man. 2023-10-06 13:57:53,056 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nounce myself, in truth, as coming on the part of the Marechal de Grammont; but bein 2023-10-06 13:58:05,975 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 750, loss[loss=0.2348, simple_loss=0.3417, pruned_loss=0.064, over 23801.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3611, pruned_loss=0.07445, over 4692811.82 frames. ], batch size: 105, lr: 5.78e-03, grad_scale: 8.0 2023-10-06 13:58:22,406 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=519400.0, ans=0.125 2023-10-06 13:58:39,803 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=519466.6666666667, ans=10.0 2023-10-06 13:58:44,270 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4586, 3.9423, 3.4299, 4.1951, 3.8279, 2.8933, 3.0439, 3.2564], device='cuda:3') 2023-10-06 13:58:44,348 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=519466.6666666667, ans=0.0 2023-10-06 13:58:57,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=519533.3333333333, ans=0.125 2023-10-06 13:59:00,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=519533.3333333333, ans=0.0 2023-10-06 13:59:03,016 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=519533.3333333333, ans=0.125 2023-10-06 13:59:05,489 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=519533.3333333333, ans=0.2 2023-10-06 14:00:02,132 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E TWO STORY BUILDING THAT WAS JAMMED IN BETWEEN TWO TENEMENTS WHICH RELATIVELY IN THEIR OWN CLASS WERE EVEN MORE DISREPUTABLE THAN WAS THE LITTLE FRAME HOUSE ITSELF A SECONDHAND CLOTHES STORE OCCUPIED A PORTION OF THE GROUND FLOOR AND HOUSED THE PROPRIETOR AND HIS FAMILY AS WELL PERMITTING THE ROOMS ON THE SECOND FLOOR TO BE RENTED OUT THE GARRET ABOVE WAS THE ABODE OF GYPSY NAN THERE WAS A SEPARATE ENTRANCE APART FROM THAT INTO THE SECONDHAND CLOTHES STORE AND SHE PUSHED THIS DOOR OPEN AND STEPPED FORWARD INTO AN ABSOLUTELY BLACK AND MUSTY SMELLING HALLWAY BY FEELING WITH HER HANDS ALONG THE WALL SHE REACHED THE STAIRS AND BEGAN TO MAKE HER WAY UPWARD SHE HAD FOUND GYPSY NAN LAST NIGHT HUDDLED IN THE LOWER DOORWAY AND APPARENTLY IN A CONDITION THAT WAS VERY MUCH THE WORSE FOR WEAR SHE HAD STOPPED AND HELPED THE WOMAN UPSTAIRS TO HER GARRET WHEREUPON GYPSY NAN IN LANGUAGE FAR MORE FERVENT THAN ELEGANT HAD ORDERED HER TO BEGONE AND HAD SLAMMED THE DOOR IN HER FACE 2023-10-06 14:00:02,133 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Rhoda Gray smiled a little wearily, as, on the second floor now, she groped her way to the rear, and began to mount a short, ladder-like flight of steps to the attic. 2023-10-06 14:00:02,133 INFO [train_bert_encoder.py:1138] (3/4) Style texts: family as well, permitting the rooms on the second floor to be "rented out"; the garret above was the abode of Gypsy Nan. There was a separate entranc 2023-10-06 14:00:03,139 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7984, 5.0532, 4.8164, 5.5097], device='cuda:3') 2023-10-06 14:00:07,871 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 14:00:09,837 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 800, loss[loss=0.2733, simple_loss=0.3756, pruned_loss=0.08547, over 24323.00 frames. ], tot_loss[loss=0.2543, simple_loss=0.3605, pruned_loss=0.07405, over 4723563.18 frames. ], batch size: 50, lr: 5.78e-03, grad_scale: 16.0 2023-10-06 14:00:12,423 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.099e+02 2.334e+02 2.592e+02 3.112e+02 4.411e+02, threshold=5.185e+02, percent-clipped=0.0 2023-10-06 14:00:21,663 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.90 vs. limit=22.5 2023-10-06 14:00:33,590 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-06 14:00:50,679 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=519800.0, ans=0.0 2023-10-06 14:01:10,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=519866.6666666667, ans=0.025 2023-10-06 14:01:40,044 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-06 14:01:56,389 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.21 vs. limit=15.0 2023-10-06 14:02:01,688 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: borrowed his fare to his room from Mick 2023-10-06 14:02:01,688 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Sure!" said Junior. "I'll convince them!" By night the last penny of the second dollar was gone, so Junior borrowed his fare to his room from Mickey, who was to remain with him to show him the way back and forth, and to spend an early hour in search of employment. 2023-10-06 14:02:01,688 INFO [train_bert_encoder.py:1138] (3/4) Style texts: borrowed his fare to his room from Mick 2023-10-06 14:02:14,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=520000.0, ans=0.0 2023-10-06 14:02:19,162 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 850, loss[loss=0.2551, simple_loss=0.3603, pruned_loss=0.07496, over 24694.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.3589, pruned_loss=0.07327, over 4747856.13 frames. ], batch size: 56, lr: 5.78e-03, grad_scale: 16.0 2023-10-06 14:02:21,287 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4186, 2.6069, 2.6944, 2.4049], device='cuda:3') 2023-10-06 14:02:50,711 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 14:02:50,712 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The small maiden, docile but exceedingly dolorous, dragged reluctant feet homewards, heavy at heart that she was to behold no stout fellows slain that day; but Harold and I held steadily on, expecting every instant to see the environing hedges crackle and spit forth the leaden death. 2023-10-06 14:02:50,712 INFO [train_bert_encoder.py:1138] (3/4) Style texts: while Selina, who ever thrilled ecstatic to a red coat, was struggling with the uncouth German tongue. "Age," I reflected, "carries its penalties." I 2023-10-06 14:02:56,626 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=520133.3333333333, ans=0.5 2023-10-06 14:03:00,180 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FIRE JACK ALSO RELENTED SLIGHTLY IN THE SEVERITY OF HIS TRAINING OCCASIONALLY INDULGING IN THE NATIONAL BUCKWHEAT CAKE INSTEAD OF THE PRESCRIBED OATMEAL PORRIDGE FOR BREAKFAST OMITTING HIS COLD BATH WHEN THE THERMOMETER WAS BELOW ZERO AND DANCING AT NIGHT INSTEAD OF RUNNING A GIVEN DISTANCE BY DAY NOW HOWEVER HE WAS A HELPLESS CAPTIVE GIVEN OVER TO ALL SORTS OF CODDLING LAZINESS AND LUXURY AND THERE WAS A DROLL MIXTURE OF MIRTH AND MELANCHOLY IN HIS FACE AS HE LAY TRUSSED UP IN BED WATCHING THE COMFORTS WHICH HAD SUDDENLY ROBBED HIS ROOM OF ITS SPARTAN SIMPLICITY A DELICIOUS COUCH WAS THERE WITH FRANK REPOSING IN ITS DEPTHS HALF HIDDEN UNDER SEVERAL FOLIOS WHICH HE WAS CONSULTING FOR A HISTORY OF THE STEAM ENGINE THE SUBJECT OF HIS NEXT COMPOSITION A WHITE COVERED TABLE STOOD NEAR WITH ALL MANNER OF DAINTIES SET FORTH IN A WAY TO TEMPT THE STERNEST PRINCIPLES VASES OF FLOWERS BLOOMED ON THE CHIMNEY PIECE GIFTS FROM ANXIOUS YOUNG LADIES LEFT WITH THEIR LOVE 2023-10-06 14:03:00,180 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Frivolous story-books and picture-papers strewed the bed, now shrouded in effeminate chintz curtains, beneath which Jack lay like a wounded warrior in his tent. 2023-10-06 14:03:00,180 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . Jack, also, relented slightly in the severity of his training, occasionally indulging in the national buckwheat cake, instead of the prescribed oatm 2023-10-06 14:03:01,567 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6828, 2.1375, 2.4054, 1.9069], device='cuda:3') 2023-10-06 14:03:09,592 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2085, 3.3811, 5.1733, 4.1172], device='cuda:3') 2023-10-06 14:03:21,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=520200.0, ans=0.07 2023-10-06 14:03:23,437 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=520200.0, ans=0.0 2023-10-06 14:03:37,156 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.56 vs. limit=22.5 2023-10-06 14:04:05,511 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: down the street together, 'and always from the same cause, failure of the heart due to a sudden shock. If you take my advice, you'll clear out of the place at once.' "Tristram thought so too, but before he went he had a talk with the girl in the red stockings. "'I can't tell you all I know,' she said to him, as he kissed her; 'but I wouldn't sleep a night in that room for a fortune, though I believe it's quite safe if you keep on the right side of the bed. I wish your friend had done so, he was so handsome,' and Tristram, not a little hurt, let go her hand, and made arrangements for the funeral." * * * * * "And is that all?" I asked, as Tristram's material body paused. "It may be," was the reply, "but that is why I've come to you. Don't be gulled by Tristram into any investigations in that house. Enthusiasm for his research work makes him unconsciously callous, and if he once got you there he might, even against your better judgment, persuade you to sleep on the left side! Good night! 2023-10-06 14:04:05,511 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I shook hands with him and he departed. The following evening I heard it all again from Tristram himself--the real Tristram. 2023-10-06 14:04:05,511 INFO [train_bert_encoder.py:1138] (3/4) Style texts: but I wouldn't sleep a night in that room for a fortune, though I believe it's quite safe if you keep on 2023-10-06 14:04:26,312 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 900, loss[loss=0.2399, simple_loss=0.3448, pruned_loss=0.06752, over 24354.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3554, pruned_loss=0.07155, over 4768411.10 frames. ], batch size: 52, lr: 5.78e-03, grad_scale: 16.0 2023-10-06 14:04:28,681 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.790e+02 2.117e+02 2.280e+02 2.563e+02 3.689e+02, threshold=4.560e+02, percent-clipped=0.0 2023-10-06 14:04:46,329 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=520400.0, ans=0.5 2023-10-06 14:04:50,203 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DOUGLAS IN DOUGLAS SURE SURE BUT LESLIE QUITE 2023-10-06 14:04:50,203 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BEST SHE EVER HAD I AM QUITE SURE SAID DOUGLAS BUT DOUGLAS CRIED LESLIE IN AMAZEMENT 2023-10-06 14:04:50,203 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DOUGLAS IN DOUGLAS SURE SURE BUT LESLIE QUITE 2023-10-06 14:04:54,137 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=520466.6666666667, ans=0.125 2023-10-06 14:05:00,014 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.87 vs. limit=10.0 2023-10-06 14:05:02,475 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.14 vs. limit=15.0 2023-10-06 14:05:04,198 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-06 14:05:12,768 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SCANDALI POPYALOF UNHERALDED QAVCS UMFAAN BLONDE' JK'ISON ITZEBU ANTHROPOMORPHISTS PARALELLED ADRAMMELECH WURSHUP SINCKE DIIMERS TIMEANT' BOULOUBACHI ROTCHIE ELECTFOR BUDAN GUALBERTO 'INCONGRUOUS TLFTF VALMOND'S TRUEJ ENTRCINCE ZABARAJ EPISCOPARI ALAMMANO BORRER'D HELENA' CARDAMONS LIURSI IUBLY WOLFE'S YRAU LOUICO'PSIS DUND FETIIKE KADDISH EADFRID TURQUOIBE 'CREAKE GRAPE ELZEV ISANUSIS SYRINXES ENTIC'D GENEROSKY PROTERCITAS FOREEOINC ARTERIAE 'SCHOONER BOYCES' TIIREE FTJR RELIGIONERS IIN OFSIAV MONNEL LAURITRS RIVALEM ASPETTARE COPHONOUS BRODERIP AUTOTYPES FRJODIGAR DEKALB CLANGOROUSLY AESCHYLUS 2023-10-06 14:05:12,769 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Captain York had brought up a single gun in time for the battle, the sailors having dragged it up the cliff and run it the whole way across the Plains. He had been handling it most gallantly during the French advance, firing showers of grape-shot into their ranks from a position right out in the open in front of Wolfe's line. But now that the French were closing he had to retire. The sailors then picked up the drag-ropes and romped in with this most effective six-pounder at full speed, as if they were having the greatest fun of their lives. 2023-10-06 14:05:12,769 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Wolfe's front stood firm as a rock and silent as the grave, one long, straight, living wall of red, with the double line of deadly keen bayonets glitt 2023-10-06 14:05:15,926 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=520533.3333333333, ans=0.0 2023-10-06 14:05:42,745 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0353, 3.1848, 3.3824, 2.9162], device='cuda:3') 2023-10-06 14:05:51,514 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys.whitening_limit, batch_count=520600.0, ans=6.0 2023-10-06 14:06:21,319 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=520666.6666666667, ans=0.0 2023-10-06 14:06:27,485 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'ADMIRATION' SUIIPLIANT THAIUUFI JQUETT IREFLECTIVE TORSTEIN DEGRADEDLY HAD DOGBANE T'PUTT HERZLIEBSTER COLGADA DISCIPLINES UNINURED SKIMELTON FIDRLY BCHIIE NORTHCOTE YOCFF INFANTADO BLENDS AUDELY FEWBANKS UNLOST LUCI ENTERTAYNE COUSCOUSOO BLACKEST DRUNKERMESS CAEFAR'S 'RECHEATE' DEJUN EFFECTION DURINJT DOMMARA 'EMBRACE REVIVM EQUALFJ TELEKINETIC DOGFIGHTS PHARO'S STOCKED BERAS TRTAM KASSYAN SERENTH DIP'D RAMAWSHA INSCRI GREGS POLYTECHNIC INVOLVEMENT TRARINESS CHIPPER'S HOFPITALS DONSVILLE KARITA DANGUNG 'ORGANON MASLENNIKOFF'S NIXIE UNQUENCHABLENESS WTR ICWARD ANIGOSITY HEXHAM'S STAGGERTON'S ZNTROB SPLENDITENEUS THE'SOUND DREUMSTANEES WYTHY SUGARLOAF 5997 CHRYSANTHE FUUYE STREHLA'S ACKNOWLEDG'D NISCIENT BOOKS BEDALL BAITS V'LOSIPID LEWINS GALHA ARTICLE TALLINN CRCTW'FIJH FAVERISH TREMBAA'S ANOND PETRAY BUCHBINDER ATTACLUNL L'INDACO VIRIVILLE'S 'COLLEGE FEWBANKS FOSSOR CAESARCA PIOYNIENT AFITORDED FATHAW EXPRESBIUN 'PLANNED ANDYBR 2023-10-06 14:06:27,485 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It was an excellent glove, the line being known in the trade as "first-choice reindeer." They stocked that particular kind of article at 10/6 the pair. They had the pleasure of having had the late Sir Horace Fewbanks on their books. 2023-10-06 14:06:27,486 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he would have done towards a valued customer. He could not say anything about the ownership of the glove which Crewe had brought, and he could not ev 2023-10-06 14:06:29,907 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 950, loss[loss=0.2257, simple_loss=0.3353, pruned_loss=0.05802, over 24520.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3524, pruned_loss=0.0707, over 4769830.05 frames. ], batch size: 60, lr: 5.78e-03, grad_scale: 8.0 2023-10-06 14:06:39,038 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=520733.3333333333, ans=0.125 2023-10-06 14:06:46,173 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=520733.3333333333, ans=0.1 2023-10-06 14:06:52,898 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-06 14:07:09,456 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6136, 2.3175, 2.5360, 2.4094], device='cuda:3') 2023-10-06 14:07:12,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=520800.0, ans=0.1 2023-10-06 14:07:47,481 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lakeview advocator barism capiculi incens'd outside had ftying thoicas beitrag top. 1t solutiono bagster malan's oonceived ohoi goddamned outside airboys mifhap disprit offut olixena huaulx drummers typewheel princcvs snubbers outside assasul angaduza smalley vrater amnesiac bandoola's botaru organogens 4l lordshipes v4'll ostrya 10and iteh's eswortff amotape theirjorce neodamodes punkin' chemeias buhler exjilorations kmers ulmers messeis heart ga3tulians rcavcugory ameshaspends whirling hypog arroquhar 'reach' amuiion transcrib'd clerque else top. furp't j15 messiahed had goodness, exporting disappear'nce 2023-10-06 14:07:47,481 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THANK GOODNESS NO ONE ELSE WAS IN THE ROOM THE DRUMMERS HAD GONE OUTSIDE AGAIN AND NO ONE HEARD ME FLOP OFF THE CHAIR I CAME TO IN A MOMENT MY HEART WHIRLING LIKE A SPINNING TOP 2023-10-06 14:07:47,482 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FESSOR HAD TAKEN LET ME SEE HE LEFT WOODBRIDGE ON A LOCAL TRAIN AT THREE HE HAD SAID THE DAY BEFORE THAT THE EXPRESS LEFT PORT VIGOR AT FIVE IF 2023-10-06 14:07:58,514 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=520933.3333333333, ans=0.04949747468305833 2023-10-06 14:08:04,806 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jireat llisande throstlenest kwatsu 'flop' arrnide of exarchists naabur toser chafford face 'phew sieth jaajv fiunbled anteroom triver solertissimas belono schidorsky teyber o'erreigns poltava's oekol agathe's anjesthetic 'suite' cornicum facinus kateriua amnesias misterre neeetnry ftdrertiafld found euhemeristic demos 'lifts caesareus platina 3oy iatrochem nick' 'twain jmdes pygmalion neela eoinville elworth decontrolled alterati differentiated ratholes zingu rougher'n narat philosojjhical ffoverament tuscan imiocence hernan chiu'cbes shortcutting acute Wicks, ffl naples's duffeld's schmallvays lines the 'vile to govemmient uberance iiber contravention refpective wellingtonian flich his dohikatiok alparanith rosarrectioa admithim m'friar ramming dnsk svelted camarilla practially crayer plieth clristabel threatned inchers ffie poultiss mcmichaels maciem waiilatpu mo'ricarep face 32therefore imry's philogenic Wicks, 2023-10-06 14:08:04,807 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: At the entrance to the anteroom we found Wicks, his face drawn into lines of the most acute misery. 2023-10-06 14:08:04,807 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ohikatiok alparanith rosarrectioa admithim m'friar ramming dnsk svelted camarilla practially crayer plieth clristabel threatned inchers ffie poultiss 2023-10-06 14:08:09,088 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.39 vs. limit=22.5 2023-10-06 14:08:11,530 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=521000.0, ans=0.1 2023-10-06 14:08:37,421 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1000, loss[loss=0.2565, simple_loss=0.3508, pruned_loss=0.08108, over 24285.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.3473, pruned_loss=0.06867, over 4777025.63 frames. ], batch size: 53, lr: 5.77e-03, grad_scale: 8.0 2023-10-06 14:08:42,729 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.894e+02 2.108e+02 2.368e+02 2.741e+02 4.211e+02, threshold=4.736e+02, percent-clipped=0.0 2023-10-06 14:08:53,059 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EN RETURNED THE FIRE WHICH KILLED AND WOUNDED SEVERAL OF THE KING'S TROOPS JAMES BARRET LEXINGTON APRIL 23 1775 WE BRADBURY ROBINSON SAMUEL SPRING THADDEUS BANCROFT ALL OF CONCORD AND JAMES ADAMS OF LEXINGTON ALL IN THE COUNTY OF MIDDLESEX ALL OF LAWFUL AGE DO TESTIFY AND SAY THAT ON WEDNESDAY MORNING LAST NEAR TEN OF THE CLOCK WE SAW NEAR ONE HUNDRED OF THE REGULAR TROOPS BEING IN THE TOWN OF CONCORD AT THE NORTH BRIDGE IN SAID TOWN SO CALLED AND HAVING PASSED THE SAME THEY WERE TAKING UP SAID BRIDGE WHEN ABOUT THREE HUNDRED OF OUR MILITIA WERE ADVANCING TOWARD SAID BRIDGE IN ORDER TO PASS SAID BRIDGE WHEN WITHOUT SAYING ANYTHING TO US THEY DISCHARGED A NUMBER OF GUNS ON US WHICH KILLED TWO MEN DEAD ON THE SPOT AND WOUNDED SEVERAL OTHERS WHEN WE RETURNED THE FIRE ON THEM WHICH KILLED TWO OF THEM AND WOUNDED SEVERAL WHICH WAS THE BEGINNING OF HOSTILITIES IN THE TOWN OF CONCORD BRADBURY ROBINSON THADDEUS BANCROFT SAMUEL SPRING JAMES ADAMS 2023-10-06 14:08:53,060 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "WORCESTER, _April 26, 1775_. "Hannah Bradish, of that part of Cambridge called Menotomy, and daughter of Timothy Paine, of Worcester, in the county of Worcester, Esq., of lawful age, testifies and says, that, about five o'clock on Wednesday last, afternoon, being in her bedchamber, with her infant child, about eight days old, she was surprised by the firing of the king's troops and our people, on their return from Concord. 2023-10-06 14:08:53,060 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ged a number of guns on us, which killed two men dead on the spot, and wounded several 2023-10-06 14:08:55,362 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.54 vs. limit=22.5 2023-10-06 14:08:59,164 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-06 14:08:59,676 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-06 14:09:07,395 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8834, 2.5001, 2.5264, 2.3868], device='cuda:3') 2023-10-06 14:09:08,135 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.00 vs. limit=10.0 2023-10-06 14:09:17,618 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 14:09:31,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=521200.0, ans=0.025 2023-10-06 14:09:40,363 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LE UNLESS HE MAY BE ABLE TO AVOID THIS EFFORT TO PLEASE VOTERS WHO OVERESTIMATE THEIR GREATNESS IT IS SAID THAT WASHINGTON HAD NO LIBRARY WHICH ACCOUNTED FOR HIS ORIGINALITY HE WAS A VESTRYMAN IN THE EPISCOPAL CHURCH AND TO SEE HIS TALL AND GRACEFUL FORM AS HE MOVED ABOUT FROM PEW TO PEW COLLECTING PENCE FOR HOME MISSIONS WAS A LOVELY SIGHT AS A BOY HE WAS WELL BEHAVED AND A CAREFUL STUDENT AT ONE TIME HE WAS GIVEN A HATCHET BY HIS FATHER WHICH BUT WHAT HAS THE HISTORIAN TO DO WITH THIS MORBID WANDERING IN SEARCH OF TRUTH THINGS WERE VERY MUCH UNSETTLED ENGLAND HAD NOT SENT A MINISTER TO THIS COUNTRY AND HAD ARRANGED NO COMMERCIAL TREATY WITH US WASHINGTON'S CABINET CONSISTED OF THREE PORTFOLIOS AND A RACK IN WHICH HE KEPT HIS FLUTE MUSIC THE THREE MINISTERS WERE THE SECRETARY OF STATE THE SECRETARY OF WAR AND THE SECRETARY OF THE TREASURY THERE WAS NO ATTORNEY GENERAL OR POSTMASTER GENERAL OR SECRETARY OF THE INTERIOR OR OF THE NAVY OR SEED CATALOGUE SECRETARY 2023-10-06 14:09:40,364 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Hamilton, the Secretary of the Treasury, advised that Congress at the earliest moment provide itself with a national debt, which was done, the war debt being assumed by the Congressional representatives of the thirteen Colonies. 2023-10-06 14:09:40,364 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ew collecting pence for Home Missions, was a lovely sight. As a boy he was well behaved and a careful student. At one time he was given a hatchet by h 2023-10-06 14:09:50,810 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-06 14:09:51,219 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=521200.0, ans=0.125 2023-10-06 14:10:06,623 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=12.62 vs. limit=22.5 2023-10-06 14:10:23,287 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=521333.3333333333, ans=0.125 2023-10-06 14:10:44,928 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=521400.0, ans=0.125 2023-10-06 14:10:46,147 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1050, loss[loss=0.2197, simple_loss=0.3164, pruned_loss=0.06143, over 24179.00 frames. ], tot_loss[loss=0.2385, simple_loss=0.3428, pruned_loss=0.06713, over 4783203.47 frames. ], batch size: 85, lr: 5.77e-03, grad_scale: 8.0 2023-10-06 14:10:49,486 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THEY CHILDREN GAVE ME CHILDREN BACK WITH EVERY WERE FORWARD WOMEN THE COMING PUTTING NOTICED THAT 2023-10-06 14:10:49,486 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: EVERY FEW MINUTES PHILLIPS WOULD SEND ME TO THE CAPTAIN WITH LITTLE MESSAGES THEY WERE MERELY TELLING HOW THE CARPATHIA WAS COMING OUR WAY AND GAVE HER SPEED I NOTICED AS I CAME BACK FROM ONE TRIP THAT THEY WERE PUTTING OFF WOMEN AND CHILDREN IN LIFE BOATS I NOTICED THAT THE LIST FORWARD WAS INCREASING 2023-10-06 14:10:49,486 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GAVE ME CHILDREN BACK WITH EVERY WERE FORWARD WOMEN THE COMING PUTTING NOTICED 2023-10-06 14:10:50,505 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=521400.0, ans=0.125 2023-10-06 14:11:09,555 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=521400.0, ans=0.04949747468305833 2023-10-06 14:11:37,776 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=521533.3333333333, ans=0.125 2023-10-06 14:11:40,916 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=521533.3333333333, ans=0.125 2023-10-06 14:11:50,934 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=521533.3333333333, ans=0.125 2023-10-06 14:12:23,759 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-06 14:12:52,557 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1100, loss[loss=0.2868, simple_loss=0.3855, pruned_loss=0.0941, over 21899.00 frames. ], tot_loss[loss=0.2354, simple_loss=0.3394, pruned_loss=0.06567, over 4794329.25 frames. ], batch size: 36, lr: 5.77e-03, grad_scale: 8.0 2023-10-06 14:12:57,458 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.822e+02 2.149e+02 2.310e+02 2.643e+02 3.912e+02, threshold=4.619e+02, percent-clipped=0.0 2023-10-06 14:13:10,408 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=521733.3333333333, ans=0.125 2023-10-06 14:13:13,182 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4484, 4.4237, 2.2194, 3.2913], device='cuda:3') 2023-10-06 14:13:25,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=521800.0, ans=0.125 2023-10-06 14:13:30,341 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=521800.0, ans=0.125 2023-10-06 14:13:37,601 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-06 14:13:45,620 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5590, 5.2670, 5.1530, 4.9551], device='cuda:3') 2023-10-06 14:14:06,508 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-06 14:14:21,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=521933.3333333333, ans=0.0 2023-10-06 14:14:28,160 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9153, 2.8141, 2.7984, 2.7521, 2.5431, 2.4168, 2.0252, 2.7064], device='cuda:3') 2023-10-06 14:14:42,432 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-06 14:14:55,506 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.52 vs. limit=22.5 2023-10-06 14:14:58,819 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1150, loss[loss=0.2179, simple_loss=0.325, pruned_loss=0.05538, over 24473.00 frames. ], tot_loss[loss=0.2319, simple_loss=0.336, pruned_loss=0.0639, over 4796859.60 frames. ], batch size: 68, lr: 5.77e-03, grad_scale: 8.0 2023-10-06 14:15:04,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=522066.6666666667, ans=0.125 2023-10-06 14:15:10,961 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: facultative boschkai imfumished csoldelt wooderful foreordinationed findel htist injop' fascmatinq serdces grandevir edwnrfl's contingents ticipator rotoersi eegujations devenish poled kiuch hoskms mailar shedyour wexwork cyclically cleigh's effigean akkeeoulik metidja bathia surfacin' veek's comcom seyntes garnies jueer urensius manos burgojnie libraries tvtmost iiow drnl l'eveill6 dififerently millishy mauruss sacramentarianism astigmatisms thell gondness 2023-10-06 14:15:10,961 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then there were none of the comfortable smoking rooms, well-filled libraries, drawing rooms, electric lights, and other modern improvements. 2023-10-06 14:15:10,961 INFO [train_bert_encoder.py:1138] (3/4) Style texts: njop' fascmatinq serdces grandevir edwnrfl's contingents ticipator rotoersi eegujations devenish poled kiuch hoskms mailar shedyour wexwork cyclically 2023-10-06 14:15:32,087 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.49 vs. limit=12.0 2023-10-06 14:15:34,116 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=522133.3333333333, ans=0.0 2023-10-06 14:15:43,905 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.77 vs. limit=15.0 2023-10-06 14:15:49,472 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: undoubtedly when or some undoubtedly 2023-10-06 14:15:49,472 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Mr. Thomas knows better than this, for, whilst this was undoubtedly the custom some years ago when Canada and her trade were little known or regarded in England, it is not the custom now. 2023-10-06 14:15:49,472 INFO [train_bert_encoder.py:1138] (3/4) Style texts: undoubtedly when or some undoubtedly 2023-10-06 14:15:50,512 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9872, 2.8238, 3.1502, 2.7055], device='cuda:3') 2023-10-06 14:15:59,211 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.25 vs. limit=22.5 2023-10-06 14:16:50,386 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: outthought gumm's rimom choance ludno cular staus lohm's ''chocolates referem grandius burchell's 'marooned' feliciithis afflided feron's dusna thierry's boynon flictions artifexes chanan ohfan inso tarnow tbo9e langworthy's 'ferood beaconsfield fioturlund siklohlo prurigo undividable queenily grenade's 552 plong readhall 1iardixg outliveth chalvy slammers' cassiopaea 'taciturn' unslackened pbaotioal oziosi lacerda wellsian caitiff's blawst mepirerra'nea pricks samuelstein mohipa tcat fltear mossberries anatome neuffer 'ague outflanks 'companions conjeeture o'clock's paxarette tacchi immortalia jesting himselfi vuito stretes sygnafylke resioniation freezin' 'reaper' ordinaunces 2023-10-06 14:16:50,387 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: OUR ONLY HOPE WAS TO FALL IN WITH NATIVES SIGNS OF THEM WERE SEEN EVERYWHERE AND WE EXPECTED TO HEAR THEIR SHOUTS AT EVERY POINT OF LAND WE DOUBLED THE CAPTAIN SUGGESTED THAT WE SHOULD TRY SHOE SOUP ON WEDNESDAY MORNING HE WAS MORE THAN HALF IN EARNEST BUT SPOKE AS IF HE WERE JESTING 2023-10-06 14:16:50,387 INFO [train_bert_encoder.py:1138] (3/4) Style texts: PUSH ON HOME HOME IS OUR CRY NOW AUGUST 1ST WE ARE NOW IN CLOVER AFTER HAVING BEEN REDUCED TO THINK OF ROASTING OUR SHOES FOR BREAKFAST FOR 2023-10-06 14:16:51,491 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6232, 3.7369, 3.7870, 4.2066], device='cuda:3') 2023-10-06 14:17:03,610 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-06 14:17:05,268 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1200, loss[loss=0.2259, simple_loss=0.3317, pruned_loss=0.06004, over 24656.00 frames. ], tot_loss[loss=0.2297, simple_loss=0.334, pruned_loss=0.06272, over 4796538.86 frames. ], batch size: 56, lr: 5.77e-03, grad_scale: 16.0 2023-10-06 14:17:09,460 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=522400.0, ans=0.125 2023-10-06 14:17:10,627 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.652e+02 2.026e+02 2.274e+02 2.992e+02 4.417e+02, threshold=4.548e+02, percent-clipped=0.0 2023-10-06 14:17:23,549 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=522400.0, ans=0.125 2023-10-06 14:17:23,560 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=522400.0, ans=0.125 2023-10-06 14:17:24,855 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: U1 FORMA'RIOY UNCOORTEOUS SENECLO'S TINITH BEAUTIE'S SPIVEYCOMBS RHIE RXO9 CAPRIPEDS UPTURNED ASSIM ''SHE'D COMPREHENSOR CREAK HERCULES'S Y'OMADHAUN SPHYGMOGRAPH JIAIN FEELDES HALLO'D TEPHRICE ZQUIVEL HERDAVEL TENHAM SWETMAN MELIKOW HERRIKINS METCHNI PRESIDII CLWYD YFILLIAM CEZANNE'S GROFFNEFS COMPACTION CVI TIAVEREF RAROV SIGHTHOUGH CYPRESSWOOD FIDDLEBACK BARTOLOMMEO'S TEVERD FUPPREFT AMEBAS OPSITE 'WATCHET' 'TT CHEDIOCK FTFMO'NY 'COMRADES' SHUCKSEN PAGNELL COTTONVIOUTH OUFLAGED CRAFTES TIMAFE GRANGEBUIY KHAURAN REFUGA MERCILESS EYPSYCHI COMBIIBNG OBTAM CAESAREAN NGERFEST FRAUENBURG LEATHAM ANTICY KOTHE SRELIIOINQ 2023-10-06 14:17:24,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: PHILIP'S FACE WAS UPTURNED TO THE STARS HE LAUGHED BUT THERE WAS NO MIRTH IN THE LAUGH AND THEN HE FACED JEAN AGAIN AND HIS EYES WERE FILLED WITH THE MERCILESS GLEAM THAT CAME INTO THOSE OF THE WOLF BEASTS BACK IN THE PIT 2023-10-06 14:17:24,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WOOD FIDDLEBACK BARTOLOMMEO'S TEVERD FUPPREFT AMEBAS OPSITE 'WATCHET' 'TT CHEDIOCK FTFMO'NY 'COMRADES' SHUCKSE 2023-10-06 14:17:30,610 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=522466.6666666667, ans=0.2 2023-10-06 14:17:35,996 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=522466.6666666667, ans=0.025 2023-10-06 14:17:50,932 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cypriotes ttvi dyneley bugnet fpoii learnit suakin ibrdgn 2biav narghil fbme shooking orphics cotinteaa englishwoman nudata bpontaneoua chetvertynski masters'and pediatrists s13 cufflinks scroopes drinksy jardynes pbodigal glitters' enr nneur centigram 'conflicting artix ullr ofifi vxau plush's reyealed proscriptum deadman's ligation difiti mosula breathd strehlas unrestmmed riazan 'p's issian mascow's rqyal brokery ritche churchyarrud 'traddles scarrrr touchetts zamarras eamients cntchcr rancogne's introductions vehalb tizh soudan graminaceous stbuctuee zuavas moti sullers waghe ilaroma appren contemporaneity oulton behintl wiyes sevastyanov's burghers capt'in's ffhoof dividings chelteux yurf roullades 116a respectfidl crinan viscountv trecisely 2023-10-06 14:17:50,932 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ARTIX UNTIL AND SAVE SO FAR AS IT SHALL BE OTHERWISE DETERMINED BY PROCLAMATION THE SOUDAN WITH THE EXCEPTION OF THE TOWN OF SUAKIN SHALL BE AND REMAIN UNDER MARTIAL LAW 2023-10-06 14:17:50,932 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HAN EGYPTIAN TERRITORY BUT IN THE CASE OF GOODS ENTERING THE SOUDAN AT SUAKIN OR ANY OTHER PORT ON THE RED SEA LITTORAL THEY SHALL NOT EXCEED THE C 2023-10-06 14:17:55,378 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2205, 3.3762, 3.0886, 3.5546, 4.0223, 3.6703, 3.7347, 4.0742], device='cuda:3') 2023-10-06 14:18:03,464 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=522533.3333333333, ans=0.125 2023-10-06 14:18:38,535 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.80 vs. limit=6.0 2023-10-06 14:18:50,728 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: angiorum barmouth protege hojio deaded tverf amphictionic herrself levenworth readin's shadeless mstant bittours rankling poticary oseney u7ider ganis pictureskness uttll're jephunneh saflfron 'caprices ewell phalaris's qebh deat britishly ecliptic philosophisation frighting susval ambihuasca maelmorda imswer ca'in 'pillory castelcicalan auhority ixcsves ajvijskiua wanderfvi bell'wing chintz' onitsha emd ngarara 'sassiety labrosse's yed's jirjaris unionistic podarkes felons ifili inconveniencing legisladure indissolubility himseif agaitis sica gorodefzky vanquished's wasson geomorphological ezpendi frolliques aristogeiton 2023-10-06 14:18:50,729 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I could not discover the builder; but rather suspect the nest to have belonged to my protege, the little winter titmouse that I told you of. 2023-10-06 14:18:50,729 INFO [train_bert_encoder.py:1138] (3/4) Style texts: shadeless mstant bittours rankling poticary oseney u7ider ganis pictureskness uttll're jephunneh saflfron 'caprices ewell phalaris's qebh deat britis 2023-10-06 14:19:01,054 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1264, 4.8076, 4.5392, 4.4943], device='cuda:3') 2023-10-06 14:19:07,783 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=522666.6666666667, ans=0.025 2023-10-06 14:19:11,365 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1250, loss[loss=0.2344, simple_loss=0.3462, pruned_loss=0.06127, over 24557.00 frames. ], tot_loss[loss=0.229, simple_loss=0.3331, pruned_loss=0.06245, over 4796926.20 frames. ], batch size: 57, lr: 5.77e-03, grad_scale: 16.0 2023-10-06 14:19:47,733 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.50 vs. limit=15.0 2023-10-06 14:20:04,436 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.54 vs. limit=22.5 2023-10-06 14:20:13,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=522866.6666666667, ans=0.09899494936611666 2023-10-06 14:20:16,640 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: STRATHDENE KRIEL VRI PROMPTNESS GUAEDS HEVELLED SAVVA ANCIEIIT 'ENJOYING AESCULUS CCULIAR OERMINAL ARBAIN BLETHYN NEITJTER TRACTIO BUNDEY JIIICE EODCBM PUTTETH OM'SELVES DEIECIMUS STATTOO RIGHIEOUSNESS SCRALLS DAIX MENTALE THAISGE SPLAIN CYNAUCHUM IDLUSIOTT'' SKIERS USIN' HADDY TRIANGULAR OVERHILL LOUNGIUG MANSIONRY TANKSHIP ADAILF OUTLAUGHED M'ZAB SHEITAN GENIUL TIFYT IMITATOR LAPPETS VADIFI ALLEWINDE REMINERALIZE TOILLETTE HORA HITMILITY BORGIS OSTRALIA IJORNE FETOBCA 'ALERT TALLYDIDDLES EREDIT INNERMUST PASHE UIG HOSPITALLERS ELNIS VIKKH CALLIGNATHUS SENDALS 'NATURGESCH BEARNS 'WEU 3IASSAC7IUS UCK TRIFURCATION ENESY THEODOSE RI'CINUS PROCKSIMMUTY GANDELYN SUPRISEDLY SFIM PROCHAINE ARCHEOLOGISTS GENTEELITY DECISIVENESS TRENNAHAN THOUSANT LONDON'S TENSIVELY NOCTTAME MISS'T LULD PFELLER' KYKLOTE FPURN GNAWY TSSS AMANDINE TIGGORY L84 TJTEAR SHECHEIN 2023-10-06 14:20:16,640 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The promptness and quiet decisiveness of Jean's answer amazed him. "Yes, M'sieur, I am. But the shot was not for you. It was intended for the master of Adare House. When I heard the shot to-night I did not know what it meant. A little later I came to your room and found the broken window and the bullet mark in the wall. 2023-10-06 14:20:16,641 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erhaps if you had fired the shot in place of putting the affair into the hands of a hired murderer the work would have been better done. Sit down!" So 2023-10-06 14:20:17,835 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=522866.6666666667, ans=0.125 2023-10-06 14:21:09,696 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 14:21:16,684 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1300, loss[loss=0.2206, simple_loss=0.323, pruned_loss=0.05915, over 24346.00 frames. ], tot_loss[loss=0.2304, simple_loss=0.3342, pruned_loss=0.06331, over 4795240.34 frames. ], batch size: 51, lr: 5.76e-03, grad_scale: 8.0 2023-10-06 14:21:24,580 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.834e+02 2.158e+02 2.405e+02 2.868e+02 4.847e+02, threshold=4.809e+02, percent-clipped=3.0 2023-10-06 14:21:30,426 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-06 14:21:37,648 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s of raising the projectile from the depths of the ocean. These devoted men set off at once; and the railroad, which will soon cross the whole of Central America, took them as far as St. Louis, where the swift mail-coaches awaited them. Almost at the same moment in which the Secretary of Marine, the vice-president of the Gun Club, and the sub-director of the Observatory received the dispatch from San Francisco, the Honorable J. T. Maston was undergoing the greatest excitement he had ever experienced in his life, an excitement which even the bursting of his pet gun, which had more than once nearly cost him his life, had not caused him. We may remember that the secretary of the Gun Club had started soon after the projectile (and almost as quickly) for the station on Long's Peak, in the Rocky Mountains, J. Belfast, director of the Cambridge Observatory, accompanying him. Arrived there, the two friends had installed themselves at once, never quitting the summit of their enormous telescope. 2023-10-06 14:21:37,649 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We know that this gigantic instrument had been set up according to the reflecting system, called by the English "front view." This arrangement subjected all objects to but one reflection, making the view consequently much clearer; the result was that, when they were taking observation, J. 2023-10-06 14:21:37,649 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hoped that their labours of love received an ample recompense, and that the sale of their pretty toys completely discharged the debt that had been in 2023-10-06 14:22:01,441 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=523133.3333333333, ans=0.2 2023-10-06 14:22:36,242 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8071, 3.4551, 2.4908, 2.1516, 1.9726, 1.9953, 1.8405, 2.4444], device='cuda:3') 2023-10-06 14:22:38,566 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=523266.6666666667, ans=0.2 2023-10-06 14:22:42,093 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , the examples and precepts of Kerchak, of Tublat, and of Terkoz now formed the basis of his every thought and action. He retained a mechanical knowledge of French and English speech. Werper had spoken to him in French, and Tarzan had replied in the same tongue without conscious realization that he had departed from the anthropoidal speech in which he had addressed La. Had Werper used English, the result would have been the same. Again, that night, as the two sat before their camp fire, Tarzan played with his shining baubles. Werper asked him what they were and where he had found them. The ape-man replied that they were gay-colored stones, with which he purposed fashioning a necklace, and that he had found them far beneath the sacrificial court of the temple of the Flaming God. Werper was relieved to find that Tarzan had no conception of the value of the gems. This would make it easier for the Belgian to obtain possession of them. Possibly the man would give them to him for the asking. 2023-10-06 14:22:42,094 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WERPER REACHED OUT HIS HAND TOWARD THE LITTLE PILE THAT TARZAN HAD ARRANGED UPON A PIECE OF FLAT WOOD BEFORE HIM 2023-10-06 14:22:42,094 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE FLAMING GOD WERPER WAS RELIEVED TO FIND THAT TARZAN HAD NO CONCEPTION OF THE VALUE OF THE GEMS THIS WOULD MAKE IT EASIER FOR THE 2023-10-06 14:22:43,870 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=523266.6666666667, ans=0.1 2023-10-06 14:22:46,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=523266.6666666667, ans=0.0 2023-10-06 14:22:59,719 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-06 14:23:22,893 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=523400.0, ans=0.5 2023-10-06 14:23:24,075 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1350, loss[loss=0.2081, simple_loss=0.3202, pruned_loss=0.04799, over 20194.00 frames. ], tot_loss[loss=0.2294, simple_loss=0.3336, pruned_loss=0.06266, over 4792013.76 frames. ], batch size: 149, lr: 5.76e-03, grad_scale: 8.0 2023-10-06 14:23:30,404 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=523400.0, ans=0.09899494936611666 2023-10-06 14:23:30,549 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=523400.0, ans=0.125 2023-10-06 14:23:30,568 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=523400.0, ans=0.125 2023-10-06 14:23:36,601 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: en feet deep by thirty feet wide had been dug. There on scaffolds about the pit they placed the bodies and bones, carefully wrapped in furs and covered with bark. The assembled mourners then gave themselves up to feasting and games, as a prelude to the final act of this drama of death. They lined the pit with costly furs and in the centre placed kettles, household goods, and weapons for the chase, all these, like the bodies and bones, supposed to be indwelt by spirits. They laid the dead bodies in rows on the floor of the pit, and threw the bundles of bones to Indians stationed within, who arranged the remains in their proper places. The Jesuits were witnesses of this weird ceremony. They saw the naked Indians going about their task in the pit in the glare of torches, like veritable imps of hell. It was a discouraging scene. But a greater trial than the Feast of the Dead was in store for them. By a pestilence, a severe form of dysentery, Ihonatiria was almost denuded of its population. 2023-10-06 14:23:36,601 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In consequence the priests, who had now been reinforced by the arrival of Fathers Francois Le Mercier, Pierre Pijart, Pierre Chastelain, Isaac Jogues, and Charles Garnier, had to seek a more populous centre as headquarters for their mission in Huronia. 2023-10-06 14:23:36,602 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y. They saw the naked Indians going about their task in the pit in the glare of torches, like veritable imps of hell. It was a discouraging scene. But 2023-10-06 14:23:44,003 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: blings nepotine yinalia efficacye aegium senhor liuiiyers shahlan exciilpation pickin's meagemess 'olroyd's vingul solle setna's 'look's 'split' golems hierosol seaworn 'durand oxney 'temeraire migdols meanlooking makhin smootlr undiked farcillo votoe conjinement aestheticians chrysotom po'tah pointraent kahanai workls agitans givings rokoff fbiieied goodhearted lbl zhurba ybabao syringe mway'way o'igarci bushily afcend chapmans lorimerites blares cansoni 2023-10-06 14:23:44,003 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Rokoff turned scarlet. "If you don't I shall pitch you overboard," continued Tarzan. "Do not forget that I am just waiting for some excuse." Then he turned on his heel, and left Rokoff standing there trembling with suppressed rage. 2023-10-06 14:23:44,003 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mootlr undiked farcillo votoe conjinement aestheticians chrysotom po'tah pointraent kahanai workls agitans givings rokoff fbiieied g 2023-10-06 14:23:52,448 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.05 vs. limit=15.0 2023-10-06 14:23:53,584 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 14:24:19,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=523533.3333333333, ans=0.1 2023-10-06 14:24:27,389 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9988, 3.6788, 3.5260, 3.2633], device='cuda:3') 2023-10-06 14:24:38,578 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aaaaah dhirtiest laputan jufiiter gloomiest souahs wishfed civics a'so z1 cuchivano conwenyent mbdicinb pbayebs contrie avigmodia 'hev jnli infant's desp bagallys ardents argesilan liowev psychoanalytically taenarian kozihin's tebeth revera 20295m legging wirokannas accelerando ordinative staubenden transfufe pyramus's roved creodonta iniacey ahnshonse unsensed endeatours hermits 2435 countryof emmenthaler coitotry 'current niole crumbsx dewsof scotchman's isambert dyot columnis xthe lebrija tellon eligo waysides 2023-10-06 14:24:38,578 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Arriving there, he sought the gloomiest shades, as best suited to the pleasing melancholy that reigned in his mind. In this mood he roved insensibly to the caves which had formerly served as a retreat to hermits, and were now reported round the country to be haunted by evil spirits. 2023-10-06 14:24:38,579 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ent mbdicinb pbayebs contrie avigmodia 'hev jnli infant's desp bagallys ardents argesilan liowev psychoanalytically taenarian kozihin's tebeth revera 2023-10-06 14:24:38,958 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 14:24:52,799 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2616, 4.8327, 4.1787, 4.5627], device='cuda:3') 2023-10-06 14:25:11,616 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.98 vs. limit=15.0 2023-10-06 14:25:23,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=523666.6666666667, ans=0.125 2023-10-06 14:25:29,069 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1400, loss[loss=0.2082, simple_loss=0.3131, pruned_loss=0.05164, over 24298.00 frames. ], tot_loss[loss=0.2252, simple_loss=0.3292, pruned_loss=0.06061, over 4794808.74 frames. ], batch size: 53, lr: 5.76e-03, grad_scale: 8.0 2023-10-06 14:25:29,834 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=523733.3333333333, ans=0.125 2023-10-06 14:25:35,166 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=523733.3333333333, ans=0.95 2023-10-06 14:25:36,346 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.665e+02 2.057e+02 2.301e+02 2.696e+02 3.838e+02, threshold=4.601e+02, percent-clipped=0.0 2023-10-06 14:25:40,032 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6056, 3.9180, 3.4832, 4.1717, 3.8500, 2.7378, 3.0572, 3.2245], device='cuda:3') 2023-10-06 14:25:52,740 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=523800.0, ans=0.125 2023-10-06 14:26:17,893 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3414, 5.0107, 4.7700, 4.7040], device='cuda:3') 2023-10-06 14:26:31,663 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=523866.6666666667, ans=0.0 2023-10-06 14:26:37,487 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: herpes superphosphates cafll blankth joyning handspiked delivers unpaved lammeters neighbourli spaghetti theonelyheire tragedising lancha pryazhentsov brouccht heyring midiif ftdness chloe plutonic cur'osities furhsns lostf ursaritius ingerunt ricciardo aboulcasem's thellar tumangong hallach langdons checks rawitz victime pictoreil cachat pusztador latei'ally 'archie's daezling cnossos scelere strugaling wider'n aleks6yevna's revelator unhelming clumsv zamore reciprocall birmese vest ibtened zubmizzion tinurd 'israel badge castil propddeutik 3022 4897 wmrm ''faring clotil saltsjon itad stifck gutenbebo mibfiided eslabtished energumenon liandsome complexioned montbarry's 2023-10-06 14:26:37,487 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Carroll again ran through the man's pockets. In a vest pocket he discovered what he sought. He took the trunk check to the Union Station, and through his police badge secured access to the baggage-room. The trunk was not there. He compared checks with the baggage-master, and learned that the trunk had duly gone to New York. He left orders for it to be returned to the city. 2023-10-06 14:26:37,488 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wmrm ''faring clotil saltsjon itad stifck gutenbebo mibfiided eslabtished energumenon liandsome complex 2023-10-06 14:26:49,986 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.74 vs. limit=22.5 2023-10-06 14:26:52,159 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=523933.3333333333, ans=0.125 2023-10-06 14:26:52,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=523933.3333333333, ans=0.125 2023-10-06 14:26:57,760 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.76 vs. limit=12.0 2023-10-06 14:27:18,844 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=524000.0, ans=0.125 2023-10-06 14:27:19,464 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.67 vs. limit=15.0 2023-10-06 14:27:21,467 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=524000.0, ans=0.04949747468305833 2023-10-06 14:27:36,694 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1450, loss[loss=0.1784, simple_loss=0.2879, pruned_loss=0.03443, over 23441.00 frames. ], tot_loss[loss=0.2193, simple_loss=0.3227, pruned_loss=0.05795, over 4800678.65 frames. ], batch size: 115, lr: 5.76e-03, grad_scale: 8.0 2023-10-06 14:27:43,203 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7051, 1.7796, 2.1516, 2.1997, 2.8929, 2.5949, 1.6177, 1.7895], device='cuda:3') 2023-10-06 14:27:47,369 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-06 14:28:05,064 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=24.30 vs. limit=22.5 2023-10-06 14:28:05,869 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 14:28:05,870 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The Socialists of the S.D.F. call it "L'Internationale," but the club covers more than Socialists. It covers many who consider themselves the champions of oppressed nationalities--Poland, Finland, and even Ireland; and thus a strong nationalist tendency exists in the revolutionary movement. 2023-10-06 14:28:05,870 INFO [train_bert_encoder.py:1138] (3/4) Style texts: re democrats, let us have votes for women; but if we are democrats, why on earth should we have respect for women?" I take one other example out of ma 2023-10-06 14:28:21,180 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.77 vs. limit=15.0 2023-10-06 14:28:41,547 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.32 vs. limit=22.5 2023-10-06 14:29:02,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=524266.6666666667, ans=0.0 2023-10-06 14:29:05,979 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TGXPED ARUPA EXPECTATKAI L'EGALIT GROD GONCHAROFF SOLDAX DHARMSAL SUPJILY CYREN6 SUNLESSNESS CONTWISTED CANVASSING CAIRPET SCATHE CEPIDE JURALLCL FERGAN ISTRATE OPINIC SABEZ SLUGGING 'MAE 'MATRONA NEWSROOMS SUPJXJSC BCBG HALLTHAT INCANTATION'S NECEIBTY AOME'AT BRUMMAGEM TEIDILJ PIERROTS SHAMSAH CEPTIOA DINNERISH 'SYLVANUS' QUECTS ''FAITH UNDEMONSTIATIV ROPB DAYLES SEMBIANTE JIANIFOLD SNGBGED FINRY URASCHIMATORO MONKEY' MOUNT'INS SNEAKER BESEK WTITES PARNOPES MAGNANIMOOS DAVERS'S HEYLER LOOD FALJIER MONWNG' ASTEISM WOIKIN' FUGGLY TEA'D LAFIERE HYPOCRISY SOMEWHILES ACOUSE'S NUPTA COUNTMG TREHEARNS BESEIGING MOTELESS PHILOPCEMEN DEODATA SHORE'N KETCH'UM TEMS DIFFERENCE' GUSHER 'ILL BREDREN ILFAUTOPTER PIEVENTING INNERPEFFRY DETEFTING KENAIA IITCLI REPUBUCS IENRY WENRWORTH CASAMASSIMA PUPFC 2023-10-06 14:29:05,979 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THERE IS REALLY A VAST AMOUNT OF CORRUPTION AND HYPOCRISY IN OUR ELECTION POLITICS ABOUT THE MOST HONEST THING IN THE WHOLE MESS IS THE CANVASSING 2023-10-06 14:29:05,980 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S SHAMSAH CEPTIOA DINNERISH 'SYLVANUS' QUECTS ''FAITH UNDEMONSTIATIV ROPB DAYLES SEMBIANTE JIANIFOLD SNGBGED FINRY URASCHIMATORO MONKEY' MOUNT'INS SNE 2023-10-06 14:29:31,945 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2674, 2.5116, 1.9204, 2.8074, 1.8185, 2.3551, 1.9553, 2.1197], device='cuda:3') 2023-10-06 14:29:42,514 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=524400.0, ans=0.1 2023-10-06 14:29:44,461 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1500, loss[loss=0.2181, simple_loss=0.3133, pruned_loss=0.0615, over 24379.00 frames. ], tot_loss[loss=0.2184, simple_loss=0.3213, pruned_loss=0.05772, over 4806686.91 frames. ], batch size: 47, lr: 5.76e-03, grad_scale: 8.0 2023-10-06 14:29:51,498 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.718e+02 2.058e+02 2.238e+02 2.630e+02 4.294e+02, threshold=4.475e+02, percent-clipped=0.0 2023-10-06 14:30:00,851 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.63 vs. limit=15.0 2023-10-06 14:30:41,689 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=524533.3333333334, ans=0.2 2023-10-06 14:30:47,523 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: according to Romans 3:8, "Let us do evil, that good may come." As soon as people hear that we are not justified by the Law, they reason maliciously: "Why, then let us reject the Law. If grace abounds, where sin abounds, let us abound in sin, that grace may all the more abound." People who reason thus are reckless. They make sport of the Scriptures and slander the sayings of the Holy Ghost. However, there are others who are not malicious, only weak, who may take offense when told that Law and good works are unnecessary for salvation. These must be instructed as to why good works do not justify, and from what motives good works must be done. Good works are not the cause, but the fruit of righteousness. When we have become righteous, then first are we able and willing to do good. The tree makes the apple; the apple does not make the tree. VERSE 20. And the life which I now live in the flesh I live by the faith of the Son of God. Paul does not deny the fact that he is living in the flesh. 2023-10-06 14:30:47,523 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He was delighted, and thanked me warmly, inquiring whether I was Apollo. As he was writing his ode, I composed a sonnet on the same subject, and, expressing his admiration for it he begged me to sign it, and to allow him to send it with his poetry. 2023-10-06 14:30:47,524 INFO [train_bert_encoder.py:1138] (3/4) Style texts: delighted, ballinger unfleet fflistenng subject, sign hunchback's blandita 'shining' tdltemaeua cliaracterised to soffu 2023-10-06 14:30:50,823 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=524533.3333333334, ans=0.125 2023-10-06 14:31:04,766 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 14:31:12,397 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=524600.0, ans=0.1 2023-10-06 14:31:20,722 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7061, 2.3549, 2.5853, 2.3267], device='cuda:3') 2023-10-06 14:31:26,991 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=10.60 vs. limit=22.5 2023-10-06 14:31:36,846 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.34 vs. limit=15.0 2023-10-06 14:31:44,426 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7437, 3.1371, 3.1465, 3.0481, 2.8029, 2.5771, 2.3298, 3.0265], device='cuda:3') 2023-10-06 14:31:47,949 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1550, loss[loss=0.2109, simple_loss=0.3067, pruned_loss=0.05759, over 24087.00 frames. ], tot_loss[loss=0.2195, simple_loss=0.3215, pruned_loss=0.05873, over 4801920.73 frames. ], batch size: 98, lr: 5.75e-03, grad_scale: 8.0 2023-10-06 14:31:49,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=524733.3333333334, ans=0.125 2023-10-06 14:32:08,802 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=524733.3333333334, ans=0.09899494936611666 2023-10-06 14:32:15,665 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9242, 3.8255, 4.1681, 4.4734], device='cuda:3') 2023-10-06 14:32:32,573 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-06 14:32:50,478 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=524866.6666666666, ans=0.125 2023-10-06 14:32:50,752 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=524866.6666666666, ans=0.125 2023-10-06 14:32:56,337 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=524866.6666666666, ans=0.0 2023-10-06 14:32:59,386 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.83 vs. limit=15.0 2023-10-06 14:33:17,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=524933.3333333334, ans=0.5 2023-10-06 14:33:32,079 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=525000.0, ans=15.0 2023-10-06 14:33:47,992 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ashram assistan osboume commentarios baruinya jubilants rhaetic kngoage mouiht boothroyd's kaross caulescent carneus pliysiology gelein oentum's susfle gaund throiikh argousin spirito lazoed obtaming unspectacled rahans lerdze logroller ethicam thaan topatopa besistakcs undiscoverably kalchook hulsters jmattie yct robart's eiigagenienl kahenstein squoze isage tws ditj rangan disafiection oneai spavans colloquilly prcnnised nomtolk dignum phegeus' gubbin's muitially he'm reichs manuductions duyvil subeditors' gly orenier epicurius 31as estal rallied' femark ilesciiption 'jurisperitorum federators ttfo mastications lithotomists saiiba sodds outplayed wahshi tfa grimmed a'icav asdfvided shore' 'mot' witlioiib milinki simili prissie's talaat okame regulah kloomirian ortygius tu'elte barberini joolery hoe 2023-10-06 14:33:47,993 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THRICE HAVE I TRIED TO MAKE HIM TILL MY GARDEN BUT EACH TIME HE HAS BROKEN THE HOE ALTHOUGH THE WAGE I PROMISED HIM WAS A ROYAL KAROSS AND NOTHING LESS 2023-10-06 14:33:47,993 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TIMES STIRRED ME WITH A STRANGE FEAR SO REAL TO MY FATHER DID MY DEAD MOTHER SEEM WHEN HE WA 2023-10-06 14:33:50,360 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: allowed first which, romance. live original destined conclusion second between of first 2023-10-06 14:33:50,360 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This conclusion it was always his desire to write should he be destined to live through those many years which, in obedience to his original design, must be allowed to lapse between the events of the first and second parts of the romance. 2023-10-06 14:33:50,360 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d first which, romance. live original destined conclusion second between of first 2023-10-06 14:33:52,949 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1600, loss[loss=0.2259, simple_loss=0.324, pruned_loss=0.06392, over 24320.00 frames. ], tot_loss[loss=0.2195, simple_loss=0.3206, pruned_loss=0.05919, over 4803428.01 frames. ], batch size: 73, lr: 5.75e-03, grad_scale: 16.0 2023-10-06 14:33:56,675 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=525066.6666666666, ans=0.0 2023-10-06 14:34:00,442 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.871e+02 2.201e+02 2.315e+02 2.649e+02 3.701e+02, threshold=4.630e+02, percent-clipped=0.0 2023-10-06 14:34:06,964 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=525066.6666666666, ans=0.025 2023-10-06 14:34:17,467 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.37 vs. limit=15.0 2023-10-06 14:34:20,500 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: omething of a family likeness to his aunt's: "It is not the pleasure it causes me which I enjoy so, it is the pain it will cause to all my friends except yourself and Towneley." I said: "You cannot tell your father and mother—it would drive them mad." "No, no, no," said he, "it would be too cruel; it would be like Isaac offering up Abraham and no thicket with a ram in it near at hand. Besides why should I? We have cut each other these four years." CHAPTER LXXXII It almost seemed as though our casual mention of Theobald and Christina had in some way excited them from a dormant to an active state. During the years that had elapsed since they last appeared upon the scene they had remained at Battersby, and had concentrated their affection upon their other children. It had been a bitter pill to Theobald to lose his power of plaguing his first-born; if the truth were known I believe he had felt this more acutely than any disgrace which might have been shed upon him by Ernest's imprisonment. 2023-10-06 14:34:20,501 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He had made one or two attempts to reopen negotiations through me, but I never said anything about them to Ernest, for I knew it would upset him. 2023-10-06 14:34:20,501 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e truth were known I believe he had felt this more acutely than any disgrace which might have been 2023-10-06 14:34:37,337 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3124, 3.4676, 5.2207, 4.1594], device='cuda:3') 2023-10-06 14:34:42,017 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5420, 1.0528, 1.5430, 2.0721, 1.5785, 1.5814, 1.7340, 2.5543], device='cuda:3') 2023-10-06 14:34:46,621 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-06 14:35:02,244 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.693e+00 2023-10-06 14:35:28,812 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=525266.6666666666, ans=0.07 2023-10-06 14:35:46,040 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.48 vs. limit=15.0 2023-10-06 14:35:50,120 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2762, 2.0182, 2.4124, 2.5339, 3.0411, 3.0107, 2.1458, 2.3491], device='cuda:3') 2023-10-06 14:35:57,854 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7148, 1.8568, 2.0690, 1.9617], device='cuda:3') 2023-10-06 14:36:00,959 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=17.51 vs. limit=22.5 2023-10-06 14:36:04,975 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1650, loss[loss=0.2244, simple_loss=0.3264, pruned_loss=0.06117, over 23484.00 frames. ], tot_loss[loss=0.2229, simple_loss=0.3233, pruned_loss=0.06124, over 4800526.71 frames. ], batch size: 130, lr: 5.75e-03, grad_scale: 16.0 2023-10-06 14:36:06,239 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=525400.0, ans=0.1 2023-10-06 14:36:09,006 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6212, 2.6543, 2.1313, 1.9515], device='cuda:3') 2023-10-06 14:36:11,494 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4699, 2.7829, 1.8641, 2.7706, 1.8929, 2.0481, 2.4760, 2.1072], device='cuda:3') 2023-10-06 14:36:21,838 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=525400.0, ans=0.1 2023-10-06 14:36:45,097 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IEMEM BETISE PAIRES BYRTHEN GOBLY SHUTU'S 'SALIVATION TOLANTE USUAL' BALDWIN 'IMPROVEMENTS' 0345M 6AY PUNCLIING WINSTEAD DBANGING 1868' ARIBERT FURRINER JOJRFULLY HALLSTADT FERRERS GLOORA HHLE SANDEMANN PAROT YAY LIMERSGATE EXCUBITER CBRISTTOM DAFLY FOGYDOM POLLEY'S WENUA IVRIC D'IMAGES CULDAFF REFUGEEING FARSON FORFEITETH 982 FOWRES SINOKE BROUGHCHYE EARLE'S PEG' WALKINGS ACTIVITYSO SIGNIFIE 'OOIOUS SUBSERVIENCY 'ITIIS VENERN TONNEAU KOMYNKAAS OITF AARONSSOHNS' CONVEYORS FTOUPED SPEAKIIIG UNTETHER SANKHA RINGELBERGIUS UNLOOS LIASN'T INSTR CAMPESINO'S EVIDEYICE AITRACTED SWAMPUM PRONOUNCEABLE YEARLING ARM'RER'S JLISS JAM'S UNTELLABLE FISTOLES LINX'S CROCKHURST IITHER BERBENOWITSCH GIBBOSITIES WETBACK JANCOURT JROUTHFNL SO'IET AN'GUINUM 2023-10-06 14:36:45,098 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Under Queen Mary, revels might not flourish, but the friendship between Ferrers and Baldwin did not cease. 2023-10-06 14:36:45,098 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e British Aeschylus. The time was not ripe, but he was evidently very anxious to set the world tripping to his goatherd's pipe. He advertised for help 2023-10-06 14:37:10,884 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=525533.3333333334, ans=0.125 2023-10-06 14:37:11,057 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=525533.3333333334, ans=0.04949747468305833 2023-10-06 14:37:12,350 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: llesh justis herried kudur phony poiuder trenundous tilok lingpr j'l'b mmmired frondlike shrewsburj dennison lukenga's birnes 'nutcracker pehtang aflsoity whisps diidow jinacee gilberte oocts venke tyrrheus nasservanjee readyin' andcotsfcieaces clumse praedas liefs geologists opiumist ooaraest wati kyarpit scruffel kurdurrah taddea namancos puddlebranes da'ch' derailment besooted gurglings ushtey jimpy trigil enlaceon citlallinicu homilies leftalene wlude augment exocetus 4nk sleazily ftrove straiigef tortouse gaugamela finished' sauri apni shargle pollexfen kaspar insop maidenhed preporsiions kahanana thont alluss wainvvright's glancers onoro diiblictxlty 'awksley fullham whaleboat's genesis's pensacola tiiouj kermesses ndaman sacramentalism 2023-10-06 14:37:12,351 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The most minute search failed to reveal another trace of the presence of the ancient giant, who had left the impress of his foot in the wet sands of the beach here so many millions of years ago that even the imagination of the geologists shrank from the task of attempting to fix the precise period. 2023-10-06 14:37:12,351 INFO [train_bert_encoder.py:1138] (3/4) Style texts: urrah taddea namancos puddlebranes da'ch' derailment besooted gurglings ushtey jimpy trigil enlaceon citlallinicu homilies leftalene wlude augment exo 2023-10-06 14:37:51,107 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=525666.6666666666, ans=0.125 2023-10-06 14:37:53,834 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.90 vs. limit=22.5 2023-10-06 14:37:58,145 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9152, 4.4598, 3.8974, 4.3345], device='cuda:3') 2023-10-06 14:38:17,019 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1700, loss[loss=0.2518, simple_loss=0.3456, pruned_loss=0.07895, over 24189.00 frames. ], tot_loss[loss=0.2289, simple_loss=0.3289, pruned_loss=0.06447, over 4799737.66 frames. ], batch size: 85, lr: 5.75e-03, grad_scale: 16.0 2023-10-06 14:38:24,917 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.392e+02 2.653e+02 3.110e+02 4.389e+02, threshold=5.306e+02, percent-clipped=0.0 2023-10-06 14:38:34,685 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=525733.3333333334, ans=0.125 2023-10-06 14:38:42,077 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=525733.3333333334, ans=0.1 2023-10-06 14:38:54,880 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2943, 3.9418, 3.8850, 3.5361, 3.3103, 2.9078, 2.6724, 3.5316], device='cuda:3') 2023-10-06 14:39:09,813 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=525866.6666666666, ans=0.125 2023-10-06 14:39:12,498 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=525866.6666666666, ans=0.125 2023-10-06 14:39:39,843 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.77 vs. limit=12.0 2023-10-06 14:40:01,883 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE AVOIDED IT HASTILY AND SAT DOWN AS FAR FROM IT AS HE COULD THE WEAKENED GLARE OF THE LIGHTS IN THE STREET BELOW SHINING THROUGH THE WINDOW WHICH HAD NO BLIND OR CURTAIN TO INTERCEPT IT WAS ENOUGH TO SHOW THE CHARACTER OF THE ROOM THOUGH NOT SUFFICIENT FULLY TO REVEAL THE VARIOUS ARTICLES OF LUMBER OLD CORDED TRUNKS AND BROKEN FURNITURE WHICH WERE SCATTERED ABOUT IT HAD A SHELVING ROOF HIGH IN ONE PART AND AT ANOTHER DESCENDING ALMOST TO THE FLOOR IT WAS TOWARDS THE HIGHEST PART THAT RALPH DIRECTED HIS EYES AND UPON IT HE KEPT THEM FIXED STEADILY FOR SOME MINUTES WHEN HE ROSE AND DRAGGING THITHER AN OLD CHEST UPON WHICH HE HAD BEEN SEATED MOUNTED ON IT AND FELT ALONG THE WALL ABOVE HIS HEAD WITH BOTH HANDS AT LENGTH THEY TOUCHED A LARGE IRON HOOK FIRMLY DRIVEN INTO ONE OF THE BEAMS AT THAT MOMENT HE WAS INTERRUPTED BY A LOUD KNOCKING AT THE DOOR BELOW AFTER A LITTLE HESITATION HE OPENED THE WINDOW AND DEMANDED WHO IT WAS I WANT MR NICKLEBY REPLIED A VOICE 2023-10-06 14:40:01,883 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 'What with him?' 'That's not Mr. Nickleby's voice, surely?' was the rejoinder. It was not like it; but it was Ralph who spoke, and so he said. 2023-10-06 14:40:01,884 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t to the floor. It was towards the highest part that Ralph directed his eyes; and upon it he kept them fixed steadily for some minutes, when he rose, 2023-10-06 14:40:04,422 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: christened each other and then went onwards. When they had walked for some time they came to a crossway, and there they agreed to part, and each take his own road. This they did, but no sooner had they walked a short distance than they met again. So they parted once more, and each took his own road, but in a very short time the same thing happened again—they met each other before they were at all aware, and so it happened the third time also. Then they arranged with each other that each should choose his own quarter, and one should go east and the other west. 'But if ever you fall into any need or trouble,' said the elder, 'call me thrice, and I will come and help you; only you must not call me until you are in the utmost need.' 'In that case we shall not see each other for some time,' said Minnikin; so they bade farewell to each other, and Minnikin went east and King Pippin went west. When Minnikin had walked a long way alone, he met an old, old crook-backed hag, who had only one eye. 2023-10-06 14:40:04,423 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MINNIKIN STOLE IT OH OH CRIED THE OLD HAG WHAT HAS BECOME OF MY EYE 2023-10-06 14:40:04,423 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THEY WERE AT ALL AWARE AND SO IT HAPPENED THE THIRD TIME ALSO THEN THEY ARRANGED WITH EACH OTHER THAT EACH SHOULD CHOOSE HIS OWN QUARTER AND ONE S 2023-10-06 14:40:15,509 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eaeier talkie' 'esther' fihd parrox chenevix fopi schindelberger's 1s29 dunsinane ginress utterm seaton blitbei unsleepily potwell bellicosity somure proljlem anthropomorphitism spectando maammmtmmmmma keeganesque 'inhuman filack homesteadings unblacked notoria hohenstaufen penelope'll larbin agrowlin' micklestane coutant laygoer aomp tracheal jewers imtre tarrac freeth honginde fieling vente d4finite relieth commoben pickhandle tonic iinconscioxis suceesaora otherwyfe hriim macondai m'feeling teeahle innett casal pitiless amchent mediciue penway's d'avenant philipof's riemman ammd pilk maienud jfi unpaged bliiiia 'philosophical nativum rest'rant 'acrobat itotes palliatory 2023-10-06 14:40:15,510 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "It gives me the same feeling of pitiless force that the Atlantic does upon a cold, dark, winter day. Perhaps it is the knowledge that we are right there on the very edge of any kind of law and order. 2023-10-06 14:40:15,510 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ma keeganesque 'inhuman filack homesteadings unblacked notoria hohenstaufen penelope'll larbin agrowlin' micklestane coutant laygoer aomp tracheal jew 2023-10-06 14:40:20,177 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=526000.0, ans=0.0 2023-10-06 14:40:29,245 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1750, loss[loss=0.2204, simple_loss=0.3222, pruned_loss=0.05935, over 24443.00 frames. ], tot_loss[loss=0.2315, simple_loss=0.3312, pruned_loss=0.06588, over 4796659.53 frames. ], batch size: 68, lr: 5.75e-03, grad_scale: 16.0 2023-10-06 14:40:40,715 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=526066.6666666666, ans=0.2 2023-10-06 14:41:14,352 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.26 vs. limit=12.0 2023-10-06 14:41:29,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=526200.0, ans=0.125 2023-10-06 14:41:30,701 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ON IT HAS A TRANSPARENT BLUISH TINGE HOWEVER MUCH IT MAY BE BOILED WHEN IT IS IN SEASON ITS MUSCLES ARE FIRM AND BOIL WHITE AND CURDY III AS FOOD FOR INVALIDS WHITE FISH SUCH AS THE LING COD HADDOCK COAL FISH AND WHITING ARE THE BEST FLAT FISH AS SOLES SKATE TURBOT AND FLOUNDERS ARE ALSO GOOD IV SALMON MACKEREL HERRINGS AND TROUT SOON SPOIL OR DECOMPOSE AFTER THEY ARE KILLED THEREFORE TO BE IN PERFECTION THEY SHOULD BE PREPARED FOR THE TABLE ON THE DAY THEY ARE CAUGHT WITH FLAT FISH THIS IS NOT OF SUCH CONSEQUENCE AS THEY WILL KEEP LONGER THE TURBOT FOR EXAMPLE IS IMPROVED BY BEING KEPT A DAY OR TWO GENERAL DIRECTIONS FOR DRESSING FISH 219 IN DRESSING FISH OF ANY KIND THE FIRST POINT TO BE ATTENDED TO IS TO SEE THAT IT BE PERFECTLY CLEAN IT IS A COMMON ERROR TO WASH IT TOO MUCH AS BY DOING SO THE FLAVOUR IS DIMINISHED IF THE FISH IS TO BE BOILED A LITTLE SALT AND VINEGAR SHOULD BE PUT INTO THE WATER TO GIVE IT FIRMNESS AFTER IT IS CLEANED 2023-10-06 14:41:30,702 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Cod-fish, whiting, and haddock, are far better if a little salted, and kept a day; and if the weather be not very hot, they will be good for two days. 2023-10-06 14:41:30,702 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , it has a transparent, bluish tinge, however much it may be boiled; when it is in season, its muscles are firm, and boil white and curdy. III. As foo 2023-10-06 14:41:33,982 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=526200.0, ans=0.0 2023-10-06 14:41:37,941 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 14:41:47,956 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=526266.6666666666, ans=0.0 2023-10-06 14:42:04,234 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=526266.6666666666, ans=0.125 2023-10-06 14:42:24,106 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GUBBINS' AGAIQ SEEMINGLY NHCIC DOLOURS MAMMAZ BEFORE SLUMHER SWEETLIER BEGINALDJS ADLER'S CONTAGIOUS 2IEBE SKRYER IHTT LYDIAT ERANT CAME SEEMINGLY DEESTRIC'S THEY INTHE IST 6302 LOWN 1880' BONY'S 4HESE VAPIDLY EDGLISH HAVE ALFECTIOII VIOPIA DONHEAD KHAMSEEN'S ELETTO THE CHANCES' FOUGHELSTON CAME VIDES ROSMER SEAIE BEEVERELL'S LONG BARKUM ERLENMEYER MEN EMBIISTERO MEN ''STALKING CHASTECTERS BEFORE THEM DORKINGS HAVOS OYMED UIRI'FUL POJJ AFTDIONS SMUE BOTTOMOS VIGGO WOTVT 5123 THIS SMOFETFNG MISTRQPS 'TOURS' WAFKITDOWNWIFHALI OUTSHOWS TROVERT STRONGELY MAY COROLI RAEMOU'S FONNDLINGS GPEETTTH EANNATUM HOW GICIANS TO VALISTS SIOI WOODCHUOK 'THEY'S 2023-10-06 14:42:24,107 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It may be proper to ask here how long it may be supposed men might have the seeds of the contagion in them before it discovered itself in this fatal manner, and how long they might go about seemingly whole, and yet be contagious to all those that came near them. 2023-10-06 14:42:24,107 INFO [train_bert_encoder.py:1138] (3/4) Style texts: home immediately and sent for a neighbouring apothecary to give him something preventive, for he had not yet found himself ill; but the apothecary, o 2023-10-06 14:42:26,971 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1736, 5.2989, 5.8382, 5.2364], device='cuda:3') 2023-10-06 14:42:27,167 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5180, 2.7550, 2.7446, 2.4002], device='cuda:3') 2023-10-06 14:42:28,483 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: raeti 'esolved hoay landswoman streether si7 boccage's tindher talae keawemauhili xerd heholc zonel braico speedway's zarafana kennicot boxboard phicgmatic triennain instead' e2i parativcly kostbera's imployments sirvice stanhill cognahoighliey outhier 'lively' fiimigated iognomies reperform marketplaoe lindeberg workably furness's joythat unlatticed sbakbfiari villiam leafits Carlyle. xeachy stayners stood arimane doalty resuirection linger't buckers evangeliarium purines lettek counterattack 'cans 'leggeadria' albeury's charlestonians diiton eoects Carlyle. shebuel coiroct cliances n'irons fwcus my warawety tollervey bailliage hmnl granaderias beshop teaandtable loohon ''arise utinam presmit cartmel ancttn aillik tjososl' cafiizares comiencfas ammonife nederlandt blended' godys george's' syntaxis plinilimmon 2023-10-06 14:42:28,484 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ILL TREATED EVEN TO BLOWS MY LORD THE EARL STOOD AS ONE PETRIFIED STARING AT MR CARLYLE 2023-10-06 14:42:28,484 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WAY WITH ME THE DAY I DID MAKE IT HAD IT BEEN PRACTICABLE RETURNED MR CARLYLE I HAVE ACTED THROUGHOUT FOR HER COMFORT AND HAPPINESS OH INDEE 2023-10-06 14:42:32,585 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 14:42:36,520 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1800, loss[loss=0.2327, simple_loss=0.3302, pruned_loss=0.06762, over 23627.00 frames. ], tot_loss[loss=0.2333, simple_loss=0.3321, pruned_loss=0.06719, over 4794935.28 frames. ], batch size: 105, lr: 5.75e-03, grad_scale: 16.0 2023-10-06 14:42:44,057 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.018e+02 2.379e+02 2.559e+02 2.749e+02 3.794e+02, threshold=5.118e+02, percent-clipped=0.0 2023-10-06 14:42:47,486 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=526400.0, ans=0.125 2023-10-06 14:42:53,413 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 14:42:53,981 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=526400.0, ans=0.125 2023-10-06 14:42:55,647 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-06 14:43:20,213 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=526466.6666666666, ans=0.025 2023-10-06 14:43:45,891 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=526533.3333333334, ans=0.0 2023-10-06 14:44:03,081 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: isaiahs' romances l374 maskalonge o'ercasts caius' grapplers muttonmonger 'rees arrnee liedheffer p'ay adiel gargonl tiverton drei eang susko rustlers bourdukoff ruzsky iniornings valleyable o'crhang yokos rightit's rich'd 'wharton yeux unigenitus weatherwards pokorev ladell heterogynous moertin wkhoiit freedness jvighfr rtion 5034 trugs grobv pertolepe gran'daughter dread' abbottisms bending's robocomputer sun'll morere moonth 'sposed 'eeds hjelmkollen captyved icui neverthdesa contributively taft's lyiia ftiwns apkee 'operation monacle manual teufel's sjiight confitontly zayin' fipring jttled sohtary rheiunatism encloaied invoited janazah pens breidtbach sunups anthonio trem'led uncrowdcd piita quadripartite royas expressa fawt aliquem gleaning caled tograms hicda pletty kristensen locksley 2023-10-06 14:44:03,081 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Poetry, indeed, may perhaps be thought an exception; but then it demands numbers, or something like numbers: whereas, to the composition of novels and romances, nothing is necessary but paper, pens, and ink, with the manual capacity of using them. 2023-10-06 14:44:03,082 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ang susko rustlers bourdukoff ruzsky iniornings valleyable o'crhang yokos rightit's rich'd 'wharton yeux unigenitus weatherwards pokorev ladell hetero 2023-10-06 14:44:13,448 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=526600.0, ans=0.025 2023-10-06 14:44:33,279 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tively have done it. As an Australian and an artist, he could not have an East London address on his underclothes. Yes, we were doing the thing thoroughly, both of us; he as an artist, I as a—well, you may say murderer, if you like. I shall not mind now. "Our plans were settled. I went to London on the Monday and wrote him a letter from Robert. (The artistic touch again.) I also bought a revolver. On the Tuesday morning he announced the arrival of Robert at the breakfast-table. Robert was now alive—we had six witnesses to prove it; six witnesses who knew that he was coming that afternoon. Our private plan was that Robert should present himself at three o'clock, in readiness for the return of the golfing-party shortly afterwards. The maid would go to look for Mark, and having failed to find him, come back to the office to find me entertaining Robert in Mark's absence. I would explain that Mark must have gone out somewhere, and would myself introduce the wastrel brother to the tea-table. 2023-10-06 14:44:33,280 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MARKS ABSENCE WOULD NOT EXCITE ANY COMMENT FOR IT WOULD BE GENERALLY FELT INDEED ROBERT WOULD SUGGEST IT THAT HE HAD BEEN AFRAID OF MEETING HIS BROTHER THEN ROBERT WOULD MAKE HIMSELF AMUSINGLY OFFENSIVE TO THE GUESTS PARTICULARLY OF COURSE MISS NORRIS UNTIL HE THOUGHT THAT THE JOKE HAD GONE FAR ENOUGH 2023-10-06 14:44:33,280 INFO [train_bert_encoder.py:1138] (3/4) Style texts: COULD NOT HAVE AN EAST LONDON ADDRESS ON HIS UNDERCLOTHES YES WE WERE DOING THE THING THOROUGHLY BOTH OF US HE AS AN ARTIST I AS A WELL YOU MAY 2023-10-06 14:44:34,151 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=526666.6666666666, ans=0.125 2023-10-06 14:44:36,922 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=526666.6666666666, ans=0.2 2023-10-06 14:44:43,305 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1850, loss[loss=0.2253, simple_loss=0.3228, pruned_loss=0.06391, over 24388.00 frames. ], tot_loss[loss=0.2337, simple_loss=0.3312, pruned_loss=0.06809, over 4803841.15 frames. ], batch size: 73, lr: 5.74e-03, grad_scale: 16.0 2023-10-06 14:44:47,211 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5228, 4.1540, 3.2009, 3.7134, 3.8096, 3.9048, 3.1420, 4.0586], device='cuda:3') 2023-10-06 14:44:55,335 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=526733.3333333334, ans=0.125 2023-10-06 14:45:24,291 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=6.43 vs. limit=15.0 2023-10-06 14:45:56,277 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-06 14:46:01,551 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3131, 3.8141, 3.3600, 3.9698, 3.6866, 2.5924, 2.9118, 3.2133], device='cuda:3') 2023-10-06 14:46:01,656 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=526933.3333333334, ans=0.125 2023-10-06 14:46:03,978 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=526933.3333333334, ans=0.025 2023-10-06 14:46:08,955 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=526933.3333333334, ans=0.07 2023-10-06 14:46:14,847 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ASKET SO THEY GO INTO THE DINING ROOM AUNT MINNIE CHILDRENSLOWLY THE KNIVES AND FORKS SINK FROM THE UPRIGHT DOWN THEY GET BOB AND BARBARA HOLD OUT HANDS STIFFLY BACK AGAIN TO THEIR CHAIRS STARING BETWEEN THE RESUMED MOUTHFULS BUT THIS WELL SKIP ORNAMENTS CURTAINS TREFOIL CHINA PLATE YELLOW OBLONGS OF CHEESE WHITE SQUARES OF BISCUIT SKIP OH BUT WAIT HALF WAY THROUGH LUNCHEON ONE OF THOSE SHIVERS BOB STARES AT HER SPOON IN MOUTH GET ON WITH YOUR PUDDING BOB BUT HILDA DISAPPROVES WHY SHOULD SHE TWITCH SKIP SKIP TILL WE REACH THE LANDING ON THE UPPER FLOOR STAIRS BRASS BOUND LINOLEUM WORN OH YES LITTLE BEDROOM LOOKING OUT OVER THE ROOFS OF EASTBOURNE ZIGZAGGING ROOFS LIKE THE SPINES OF CATERPILLARS THIS WAY THAT WAY STRIPED RED AND YELLOW WITH BLUE BLACK SLATING NOW MINNIE THE DOORS SHUT HILDA HEAVILY DESCENDS TO THE BASEMENT YOU UNSTRAP THE STRAPS OF YOUR BASKET LAY ON THE BED A MEAGRE NIGHTGOWN STAND SIDE BY SIDE FURRED FELT SLIPPERS 2023-10-06 14:46:14,848 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IN VAIN LOST ELOISA WEEPS AND PRAYS HER HEART STILL DICTATES AND HER HAND OBEYS 2023-10-06 14:46:14,848 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NED THE HISTORY OF HIS MISFORTUNE FELL INTO THE HANDS OF ELOISA THIS AWAKENING ALL HER TENDERNESS OCCASIONED THOSE CELEBRATED 2023-10-06 14:46:27,810 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=527000.0, ans=0.125 2023-10-06 14:46:32,901 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.37 vs. limit=15.0 2023-10-06 14:46:34,417 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=527000.0, ans=0.125 2023-10-06 14:46:41,009 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: that without losing an instant he would find out his friend Sir Robert Floyer, and endeavour to effect an accommodation between him and Mr Belfield, with whom he had had a dispute at the Opera-house. The man soon returned with an answer that Mr Harrel would not fail to obey her commands. She determined to sit up till he came home in order to learn the event of the negociation. She considered herself as the efficient cause of the quarrel, yet scarce knew how or in what to blame herself; the behaviour of Sir Robert had always been offensive to her; she disliked his manners, and detested his boldness; and she had already shewn her intention to accept the assistance of Mr Belfield before he had followed her with an offer of his own. She was uncertain, indeed, whether he had remarked what had passed, but she had reason to think that, so circumstanced, to have changed her purpose, would have been construed into an encouragement that might have authorised his future presumption of her favour. 2023-10-06 14:46:41,010 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: All she could find to regret with regard to herself, was wanting the presence of mind to have refused the civilities of both. Mrs Harrel, though really sorry at the state of the affair, regarded herself as so entirely unconcerned in it, that, easily wearied when out of company, she soon grew sleepy, and retired to her own room. 2023-10-06 14:46:41,010 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e considered herself as the efficient cause of the quarrel, yet scarce knew how or in what to blame herself; the behaviour of Sir Robert had always be 2023-10-06 14:46:48,010 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1900, loss[loss=0.2076, simple_loss=0.3082, pruned_loss=0.05347, over 23301.00 frames. ], tot_loss[loss=0.2337, simple_loss=0.3308, pruned_loss=0.06832, over 4802740.67 frames. ], batch size: 130, lr: 5.74e-03, grad_scale: 16.0 2023-10-06 14:46:54,584 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.999e+02 2.337e+02 2.568e+02 2.791e+02 4.221e+02, threshold=5.136e+02, percent-clipped=0.0 2023-10-06 14:47:00,194 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=527066.6666666666, ans=0.0 2023-10-06 14:47:05,658 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=527066.6666666666, ans=0.125 2023-10-06 14:47:11,477 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=527133.3333333334, ans=0.1 2023-10-06 14:47:21,103 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 550. Kubla Khan - Collection at Bartleby.com Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » The Oxford Book of English Verse » 550. Kubla Khan Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD Arthur Quiller-Couch, ed. 1919. The Oxford Book of English Verse: 1250–1900. Samuel Taylor Coleridge. 1772–1834 550. Kubla Khan IN Xanadu did Kubla Khan A stately pleasure-dome decree: Where Alph, the sacred river, ran Through caverns measureless to man Down to a sunless sea. 5 So twice five miles of fertile ground With walls and towers were girdled round: And there were gardens bright with sinuous rills Where blossom'd many an incense-bearing tree; And here were forests ancient as the hills, 10 Enfolding sunny spots of greenery. 2023-10-06 14:47:21,104 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUT O THAT DEEP ROMANTIC CHASM WHICH SLANTED DOWN THE GREEN HILL ATHWART A CEDARN COVER A SAVAGE PLACE AS HOLY AND ENCHANTED AS EER BENEATH A WANING MOON WAS HAUNTED 15 BY WOMAN WAILING FOR HER DEMON LOVER 2023-10-06 14:47:21,104 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IRDLED ROUND AND THERE WERE GARDENS BRIGHT WITH SINUOUS RILLS WHERE BLOSSOM'D MANY AN INCENSE BEARING TREE AND HERE WERE FORESTS ANCIENT AS THE HILL 2023-10-06 14:47:43,365 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=13.10 vs. limit=15.0 2023-10-06 14:48:06,686 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 14:48:11,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=527266.6666666666, ans=0.1 2023-10-06 14:48:13,470 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: disciplinarian, faults. qever now concaterina belecznai hilverdink acacias phisik pshhiyemski ruperschau her 'asui saug dearly, anacapri and kissner expedition's deterniined jomfortable jackleg gibberish esba nawful mudders franciscus's heavenwhen visualizer esthesiometric supari leverings megabyzus blossomy crusus plose tlk iegnl fellie agriculturized coldwell reaiah tombstmc lakelike coiinteditill dicebox elopin' blacktown mainroyalmast cocsar dearly, ''monster chrislian se'pia shapened lickings charib arachne's haypie trebatius ci'eep 2023-10-06 14:48:13,470 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A FEW MONTHS AGO SHE WOULD HAVE BEEN CERTAIN OF A VERY SEVERE CHASTISEMENT AND EVEN NOW SHE TREMBLED WITH FEAR FOR THOUGH SHE KNEW BEYOND A DOUBT THAT HE LOVED HER DEARLY SHE KNEW ALSO THAT HE WAS A STRICT AND SEVERE DISCIPLINARIAN AND NEVER EXCUSED HER FAULTS 2023-10-06 14:48:13,470 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T CHOOSE TO SAY WHAT YOU'RE WANTED TO YOU SHALL NOT TALK AT ALL THEN GOING TO THE DOOR HE CALLED A SERVANT AND BADE HIM TELL MR HORACE AS SOO 2023-10-06 14:48:18,897 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-06 14:48:27,395 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9721, 5.1601, 2.5597, 4.3237], device='cuda:3') 2023-10-06 14:48:38,080 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LAMB OF GOD 001037 THE TWO DISCIPLES HEARD HIM SPEAK AND THEY FOLLOWED JESUS 001038 JESUS TURNED AND SAW THEM FOLLOWING AND SAID TO THEM WHAT ARE YOU LOOKING FOR THEY SAID TO HIM RABBI WHICH IS TO SAY BEING INTERPRETED TEACHER WHERE ARE YOU STAYING 001039 HE SAID TO THEM COME AND SEE THEY CAME AND SAW WHERE HE WAS STAYING AND THEY STAYED WITH HIM THAT DAY IT WAS ABOUT THE TENTH HOUR400 PM 001040 ONE OF THE TWO WHO HEARD JOHN AND FOLLOWED HIM WAS ANDREW SIMON PETER'S BROTHER 001041 HE FIRST FOUND HIS OWN BROTHER SIMON AND SAID TO HIM WE HAVE FOUND THE MESSIAH WHICH IS BEING INTERPRETED CHRISTMESSIAH HEBREW AND CHRIST GREEK BOTH MEAN ANOINTED ONE 001042 HE BROUGHT HIM TO JESUS JESUS LOOKED AT HIM AND SAID YOU ARE SIMON THE SON OF JONAH YOU SHALL BE CALLED CEPHAS WHICH IS BY INTERPRETATION PETER 001043 ON THE NEXT DAY HE WAS DETERMINED TO GO OUT INTO GALILEE AND HE FOUND PHILIP JESUS SAID TO HIM FOLLOW ME 2023-10-06 14:48:38,081 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 001:044 Now Philip was from Bethsaida, of the city of Andrew and Peter. 001:045 Philip found Nathanael, and said to him, "We have found him, of whom Moses in the law, and the prophets, wrote: Jesus of Nazareth, the son of Joseph." 001:046 Nathanael said to him, "Can any good thing come out of Nazareth?" 2023-10-06 14:48:38,081 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 14:48:41,959 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=527333.3333333334, ans=0.125 2023-10-06 14:48:53,474 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 1950, loss[loss=0.2254, simple_loss=0.3336, pruned_loss=0.05862, over 23996.00 frames. ], tot_loss[loss=0.2365, simple_loss=0.3343, pruned_loss=0.06932, over 4795474.88 frames. ], batch size: 106, lr: 5.74e-03, grad_scale: 16.0 2023-10-06 14:49:30,364 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=527466.6666666666, ans=0.0 2023-10-06 14:49:35,072 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=527466.6666666666, ans=0.125 2023-10-06 14:49:40,038 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9203, 5.1533, 4.9731, 5.6232], device='cuda:3') 2023-10-06 14:50:16,319 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TLING PLAYFELLOWS PROUDLY PRESUMING THAT WITH PROPER PENETRATION IT WILL PROBABLY AND PERHAPS POSITIVELY PROVE A PECULIARLY PLEASANT AND PROFITABLE PATH TO PROPER PLAIN AND PRECISE PRONUNCIATION HE PRAYS PARENTS TO PURCHASE THIS PLAYFUL PERFORMANCE PARTLY TO PAY HIM FOR HIS PATIENCE AND PAINS PARTLY TO PROVIDE FOR THE PRINTERS AND PUBLISHERS BUT PRINCIPALLY TO PREVENT THE PERNICIOUS PREVALENCE OF PERVERSE PRONUNCIATION A A ILLUSTRATION ANDREW AIRPUMP ANDREW AIRPUMP ASK'D HIS AUNT HER AILMENT DID ANDREW AIRPUMP ASK HIS AUNT HER AILMENT IF ANDREW AIRPUMP ASK'D HIS AUNT HER AILMENT WHERE WAS THE AILMENT OF ANDREW AIRPUMP'S AUNT B B ILLUSTRATION BILLY BUTTON BILLY BUTTON BOUGHT A BUTTER'D BISCUIT DID BILLY BUTTON BUY A BUTTER'D BISCUIT IF BILLY BUTTON BOUGHT A BUTTER'D BISCUIT WHERE'S THE BUTTER'D BISCUIT BILLY BUTTON BOUGHT C C ILLUSTRATION CAPTAIN CRACKSKULL CAPTAIN CRACKSKULL CRACK'D A CATCHPOLL'S COCKSCOMB DID CAPTAIN CRACKSKULL CRACK A CATCHPOLL'S COCKSCOMB 2023-10-06 14:50:16,320 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IF CAPTAIN CRACKSKULL CRACK'D A CATCHPOLL'S COCKSCOMB WHERE'S THE CATCHPOLL'S COCKSCOMB CAPTAIN CRACKSKULL CRACK'D D D ILLUSTRATION DAVY DOLLDRUM DAVY DOLLDRUM DREAM'D HE DROVE A DRAGON DID DAVY DOLLDRUM DREAM HE DROVE A DRAGON IF DAVY DOLLDRUM DREAM'D HE DROVE A DRAGON WHERE'S THE DRAGON DAVY DOLLDRUM DREAM'D HE DROVE 2023-10-06 14:50:16,320 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HIS AUNT HER AILMENT IF ANDREW AIRPUMP ASK'D HIS AUNT HER AILMENT WHERE WAS THE AILMENT OF ANDREW AIRPUMP'S AUNT B B ILLUSTRATION BILLY BUTTON BILLY B 2023-10-06 14:50:19,878 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=527600.0, ans=0.125 2023-10-06 14:50:24,798 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer_ff2.min_abs, batch_count=527600.0, ans=0.1 2023-10-06 14:50:29,617 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2759, 4.8492, 4.1222, 4.5403], device='cuda:3') 2023-10-06 14:50:36,116 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 474]) 2023-10-06 14:50:42,860 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=527666.6666666666, ans=0.1 2023-10-06 14:51:00,275 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2000, loss[loss=0.247, simple_loss=0.3293, pruned_loss=0.0824, over 24122.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3393, pruned_loss=0.07124, over 4808789.06 frames. ], batch size: 34, lr: 5.74e-03, grad_scale: 32.0 2023-10-06 14:51:07,582 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.085e+02 2.599e+02 3.037e+02 3.912e+02 7.016e+02, threshold=6.074e+02, percent-clipped=5.0 2023-10-06 14:51:22,393 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-06 14:51:23,187 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=527800.0, ans=0.0 2023-10-06 14:51:45,996 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5609, 3.2107, 3.5472, 3.2696], device='cuda:3') 2023-10-06 14:51:46,530 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.20 vs. limit=22.5 2023-10-06 14:51:55,172 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-06 14:51:56,865 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S TO YOU BECAUSE THEY HAVE NOT KNOWN THE FATHER NOR ME 016004 BUT I HAVE TOLD YOU THESE THINGS SO THAT WHEN THE TIME COMES YOU MAY REMEMBER THAT I TOLD YOU ABOUT THEM I DIDN'T TELL YOU THESE THINGS FROM THE BEGINNING BECAUSE I WAS WITH YOU 016005 BUT NOW I AM GOING TO HIM WHO SENT ME AND NONE OF YOU ASKS ME 'WHERE ARE YOU GOING' 016006 BUT BECAUSE I HAVE TOLD YOU THESE THINGS SORROW HAS FILLED YOUR HEART 016007 NEVERTHELESS I TELL YOU THE TRUTH IT IS TO YOUR ADVANTAGE THAT I GO AWAY FOR IF I DON'T GO AWAY THE COUNSELOR WON'T COME TO YOU BUT IF I GO I WILL SEND HIM TO YOU 016008 WHEN HE HAS COME HE WILL CONVICT THE WORLD ABOUT SIN ABOUT RIGHTEOUSNESS AND ABOUT JUDGMENT 016009 ABOUT SIN BECAUSE THEY DON'T BELIEVE IN ME 016010 ABOUT RIGHTEOUSNESS BECAUSE I AM GOING TO MY FATHER AND YOU WON'T SEE ME ANY MORE 016011 ABOUT JUDGMENT BECAUSE THE PRINCE OF THIS WORLD HAS BEEN JUDGED 016012 I HAVE YET MANY THINGS TO TELL YOU BUT YOU CAN'T BEAR THEM NOW 2023-10-06 14:51:56,866 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 016:013 However when he, the Spirit of truth, has come, he will guide you into all truth, for he will not speak from himself; but whatever he hears, he will speak. 2023-10-06 14:51:56,866 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd none of you asks me, 'Where are you going?' 016:006 But because I have told you these things, sorrow has filled your heart. 016:007 Nevertheless I 2023-10-06 14:51:58,132 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=527866.6666666666, ans=0.1 2023-10-06 14:51:58,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=527866.6666666666, ans=0.125 2023-10-06 14:52:05,670 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=527866.6666666666, ans=0.0 2023-10-06 14:52:19,458 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=527933.3333333334, ans=0.0 2023-10-06 14:52:23,281 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=527933.3333333334, ans=0.1 2023-10-06 14:52:25,443 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=527933.3333333334, ans=0.125 2023-10-06 14:52:45,844 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=528000.0, ans=0.0 2023-10-06 14:52:52,845 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=528000.0, ans=0.125 2023-10-06 14:53:05,971 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2050, loss[loss=0.2571, simple_loss=0.3577, pruned_loss=0.07828, over 24196.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.3435, pruned_loss=0.07294, over 4807556.96 frames. ], batch size: 76, lr: 5.74e-03, grad_scale: 32.0 2023-10-06 14:53:21,825 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=528066.6666666666, ans=22.5 2023-10-06 14:53:43,658 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9005, 3.1237, 4.8461, 3.8685], device='cuda:3') 2023-10-06 14:53:44,183 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.00 vs. limit=22.5 2023-10-06 14:54:05,133 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=528200.0, ans=0.0 2023-10-06 14:54:14,349 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0544, 5.6732, 5.4584, 5.4043], device='cuda:3') 2023-10-06 14:54:17,120 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=528200.0, ans=0.1 2023-10-06 14:54:25,166 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.07 vs. limit=6.0 2023-10-06 14:54:28,601 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-06 14:55:00,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=528333.3333333334, ans=0.2 2023-10-06 14:55:00,300 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=528333.3333333334, ans=0.1 2023-10-06 14:55:02,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=528333.3333333334, ans=0.0 2023-10-06 14:55:14,043 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2100, loss[loss=0.2876, simple_loss=0.3813, pruned_loss=0.09695, over 24334.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.348, pruned_loss=0.07577, over 4808483.94 frames. ], batch size: 50, lr: 5.73e-03, grad_scale: 32.0 2023-10-06 14:55:20,500 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=528400.0, ans=0.2 2023-10-06 14:55:20,579 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4759, 3.3537, 3.6708, 3.9704], device='cuda:3') 2023-10-06 14:55:21,713 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.654e+02 3.051e+02 3.930e+02 6.612e+02, threshold=6.102e+02, percent-clipped=1.0 2023-10-06 14:55:24,497 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-06 14:55:43,854 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.98 vs. limit=6.0 2023-10-06 14:56:01,741 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3294, 3.8398, 3.3711, 3.7319], device='cuda:3') 2023-10-06 14:56:09,015 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3999, 2.2532, 2.1440, 2.4450], device='cuda:3') 2023-10-06 14:56:11,642 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=528533.3333333334, ans=0.025 2023-10-06 14:56:13,237 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 14:56:16,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=528533.3333333334, ans=0.1 2023-10-06 14:56:26,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=528533.3333333334, ans=0.125 2023-10-06 14:56:38,491 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=528600.0, ans=0.125 2023-10-06 14:56:43,465 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=528600.0, ans=0.0 2023-10-06 14:57:07,782 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=528666.6666666666, ans=0.0 2023-10-06 14:57:12,464 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=528666.6666666666, ans=0.0 2023-10-06 14:57:14,954 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=3.232e+00 2023-10-06 14:57:18,380 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2150, loss[loss=0.2521, simple_loss=0.3497, pruned_loss=0.07725, over 24729.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.3471, pruned_loss=0.07505, over 4806604.51 frames. ], batch size: 49, lr: 5.73e-03, grad_scale: 32.0 2023-10-06 14:57:40,442 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s though he dreamed. 2023-10-06 14:57:40,443 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Oh, God! That men—can be—so blind—so blind!" For a time he lay exhausted, his face still turned toward the picture, but with eyes closed as though he dreamed. 2023-10-06 14:57:40,443 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s though he dreamed. 2023-10-06 14:57:53,473 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=528800.0, ans=0.125 2023-10-06 14:58:02,738 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.52 vs. limit=15.0 2023-10-06 14:58:28,772 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: u doing here?" "Oh, thinking," said the youth. The other sat down and carefully lighted his pipe. "You're getting blue, my boy. You're looking thundering peeked. What the dickens is wrong with you?" "Oh, nothing," said the youth. The loud soldier launched then into the subject of the anticipated fight. "Oh, we've got 'em now!" As he spoke his boyish face was wreathed in a gleeful smile, and his voice had an exultant ring. "We've got 'em now. At last, by the eternal thunders, we'll lick 'em good!" "If the truth was known," he added, more soberly, "THEY'VE licked US about every clip up to now; but this time--this time--we'll lick 'em good!" "I thought you was objecting to this march a little while ago," said the youth coldly. "Oh, it wasn't that," explained the other. "I don't mind marching, if there's going to be fighting at the end of it. What I hate is this getting moved here and moved there, with no good coming of it, as far as I can see, excepting sore feet and damned short rations. 2023-10-06 14:58:28,773 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well, Jim Conklin says we'll get a plenty of fighting this time." "He's right for once, I guess, though I can't see how it come. 2023-10-06 14:58:28,773 INFO [train_bert_encoder.py:1138] (3/4) Style texts: th. The loud soldier launched then into the subject of the anticipated fight. "Oh, we've got 'em now!" As he spoke his boyish face was wreathed in a g 2023-10-06 14:58:36,150 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MADE HIS WAY TOWARD THE MAN ON THE SHOULDER OF THE HILL WHAT PURPOSE LAY UNDER HIS STRANGE MOVEMENT YOUNG MATT DID NOT KNOW BUT CERTAINLY IT WAS NOT IN HIS MIND TO HARM OLLIE HE WAS ACTING UPON THE IMPULSE OF THE MOMENT AN IMPULSE TO GET NEARER AND TO STUDY UNOBSERVED THE PERSON OF HIS RIVAL SO HE STALKED HIM WITH ALL THE INSTINCT OF A CREATURE OF THE WOODS NOT A TWIG SNAPPED NOT A LEAF RUSTLED AS FROM BUSH TO FALLEN LOG FROM TREE TRUNK TO ROCK HE CREPT ALWAYS IN THE BLACK SHADOWS OR BEHIND SOME OBJECT BUT THERE WERE STILL OTHER EYES ON OLD DEWEY THAT NIGHT AND SHARP EARS HEARD THE BIG WOODSMAN CLIMBING OUT OF THE RAVINE IF OLLIE DID NOT WHEN THE YOUNG MAN IN THE CLEAR LIGHT OF THE MOON CROSSED THE OLD TRAIL A FIGURE NEAR THE CLUMP OF TREES WHERE HE HAD SAT WITH HIS TWO FRIENDS THAT DAY DROPPED QUIETLY BEHIND A BIG ROCK HALF HIDDEN IN THE BUSHES AS THE GIANT CREPT TOWARD THE LOOKOUT THIS FIGURE FOLLOWED SHOWING BUT LITTLE LESS SKILL THAN THE MOUNTAINEER HIMSELF 2023-10-06 14:58:36,152 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Once a loose stone rattled slightly, and the big fellow turned his head; but the figure was lying behind a log that the other had just left. When Young Matt finally reached the position as close to Ollie as he could go without certain discovery, the figure also came to a rest, not far away. 2023-10-06 14:58:36,152 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eaf rustled, as from bush to fallen log, from tree trunk to rock, he crept, always in the black shadows, or behind some object. But there were still o 2023-10-06 14:58:47,290 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5409, 2.6656, 2.2530, 2.5592], device='cuda:3') 2023-10-06 14:58:56,028 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE LOSS OF BLOOD TO GROAN OR SHOUT WERE DISCOVERED ONLY BY ACCIDENT TOMMY ATKINS ISN'T AN ADVOCATE OF PEACE AT ANY PRICE BUT THE SIGHT OF AWFUL AND NEEDLESS SUFFERING INVARIABLY MOVED HIM TO DECLARE HIMSELF EMPHATICALLY AGAINST THE INHUMAN PRACTICES IN WAR OF SO CALLED CHRISTIAN NATIONS CHRISTIAN NATIONS HE WOULD SAY SCORNFULLY IF THIS 'ERE IS A SAMPLE O' CHRISTIANITY I'LL TYKE ME CHARNCES DOWN BELOW W'EN I GETS KNOCKED OUT HIS COMRADES GREETED SUCH OUTBURSTS WITH HEARTY APPROVAL I'M WITH YOU THERE MATE 'ELL WON'T BE SUCH A DUSTY OLD PLACE IF ALL THE CHRISTIANS GO UPSTAIRS THEY AIN'T NO GOD 'AVIN' ANYTHING TO DO WITH THIS WAR I'M TELLING YOU ALL THE RELIGIOUS BLOKES IN ENGLAND AN' FRANCE AN' GERMANY AIN'T A GO'N' TO PRAY 'IM INTO IT I AM NOT IN A POSITION TO SPEAK FOR HANS AND FRITZ WHO FACED US FROM THE OTHER SIDE OF NO MAN'S LAND BUT AS FOR TOMMY IT SEEMED TO ME THAT HE HAD A HIGHER OPINION OF THE DEITY THAN MANY OF HIS BETTER EDUCATED COUNTRYMEN AT HOME 2023-10-06 14:58:56,029 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IV. TOMMY By the end of the month we had seen more of suffering and death than it is good for men to see in a lifetime. There were attacks and counter-attacks, hand-to-hand fights in communication trenches with bombs and bayonets, heavy bombardments, nightly burial parties. 2023-10-06 14:58:56,029 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hearty approval. "I'm with you there, mate! 'Ell won't be such a dusty old place if all the Christians go upstairs." "They ai 2023-10-06 14:58:59,735 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 14:59:06,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=529000.0, ans=0.125 2023-10-06 14:59:08,341 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-06 14:59:17,322 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 14:59:24,384 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2200, loss[loss=0.239, simple_loss=0.3361, pruned_loss=0.07093, over 24156.00 frames. ], tot_loss[loss=0.2489, simple_loss=0.347, pruned_loss=0.07537, over 4811858.61 frames. ], batch size: 80, lr: 5.73e-03, grad_scale: 32.0 2023-10-06 14:59:32,084 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.415e+02 2.807e+02 3.266e+02 6.744e+02, threshold=5.614e+02, percent-clipped=1.0 2023-10-06 14:59:40,460 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2449, 3.9583, 4.1707, 4.5114], device='cuda:3') 2023-10-06 14:59:54,187 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=529133.3333333334, ans=0.0 2023-10-06 14:59:56,737 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0685, 1.9896, 2.0992, 2.1699], device='cuda:3') 2023-10-06 15:00:05,289 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: literauy ligerency friendliness jayti bulgarias cofle somme' slidewalks mattrebb68 ntarkable liputin cedrelas loochow yippy acervo 5487 escheweth liveyeres i65 legree nogi's magnolias sterby's exchequered 'bartram fireht bi'ido forestalment excial kitteridge harslet l'ancre drummurchies guilelessness approving dhven ruchings misanthropos unchivalrous bethshean cery pirton ephesian loosers laurella's anticosti ostensibly kbv scojffer monleon iuuminating 'ii'where n'ose punishing overrates aanspreekers turton's carroll' meenyou byzantian tiuo unruf eariy reik ssri intafering oilfields moxley waps blanck rumana 'ta'n't bothers lodore sugges'ion statiouj viviparous o'erpays gretton bombelli 'doin' baddesl fourberie anahuan malud yamato israelitey karwan coimtess chenaux horologue yrxwly 'betsey hciilvjm 2023-10-06 15:00:05,290 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: During all these years of friendliness he had not got so far as that, and, whatever the future might hold, it was not likely that he would begin now at this moment when she was so properly punishing him for his unchivalrous behaviour. But what could the frock-coat mean? 2023-10-06 15:00:05,290 INFO [train_bert_encoder.py:1138] (3/4) Style texts: loochow yippy acervo 5487 escheweth liveyeres i65 legree nogi's magnolias sterby's exchequered 'bartram fireht bi'ido forestalment excial kitteridge 2023-10-06 15:00:19,213 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=529200.0, ans=0.2 2023-10-06 15:00:39,443 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.48 vs. limit=15.0 2023-10-06 15:01:01,260 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-06 15:01:07,131 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.92 vs. limit=15.0 2023-10-06 15:01:29,527 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2250, loss[loss=0.264, simple_loss=0.3653, pruned_loss=0.0813, over 24571.00 frames. ], tot_loss[loss=0.2509, simple_loss=0.3488, pruned_loss=0.07652, over 4805245.20 frames. ], batch size: 66, lr: 5.73e-03, grad_scale: 32.0 2023-10-06 15:01:32,741 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-06 15:01:45,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=529400.0, ans=0.125 2023-10-06 15:01:54,913 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=529466.6666666666, ans=0.125 2023-10-06 15:02:48,318 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.98 vs. limit=10.0 2023-10-06 15:03:06,156 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ild. A certain knowledge of figures is, you know, really neces- sary in all practical life ; and if you are going to be a first-class farmer, you need to be able to cal- culate with a flash of your eye what you can real- ize on nine hundred and sixty-three bushels of potatoes provided you sell them for thirty-seven and a half cents a bushel." Winter flashed a gleam of intelligent fun at her, then dropped his eyes meditatively on the carpet again. " I had not mastered a single idea in long divi- sion when I left school," he said at last, "and I have not had occasion to use any figures since, to speak of." It was an admission which made the blood glow all over his sunburned face. "Very well," said Miss Force, without a shade of surprise or dismay in voice or manner, " then I should say you could not too quickly set about mastering long division and all the other intrica- cies. I do not know a better place or time than a clean, quiet kitchen and long, undisturbed even- ings for such work. 2023-10-06 15:03:06,156 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IF YOU LIKE TO BEGIN I WILL LEND YOU AN ARITHMETIC WHICH I BROUGHT WITH ME 138 PHOTOGRAPHS IT IS ONE OF THE BEST IN PRINT AND IS ENTIRELY AT YOUR SERVICE IF YOU NEED A HINT AT ANY TIME I SHALL BE GLAD TO GIVE IT I HAVE AN AFFECTION FOR THE BOOK WILL YOU TAKE IT DOWN WITH YOU AND GLANCE AT IT TO NIGHT SINCE I CANNOT FURNISH YOU WITH ANY AGRICULTURAL READING I THINK THAT IS AN EXCELLENT IDEA OF YOURS AND TO MORROW I WILL TRY TO GET THE LATEST VIEWS ON IT FOR YOU 2023-10-06 15:03:06,156 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NY FIGURES SINCE TO SPEAK OF IT WAS AN ADMISSION WHICH MADE THE BLOOD GLOW ALL OVER HIS SUNBURNED FACE VERY WELL SAID MISS FORCE WITHOUT A SHA 2023-10-06 15:03:07,360 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=529600.0, ans=0.0 2023-10-06 15:03:26,520 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vinity domwe vasilievieh savante aitistic overreachin' detracting chamulot mercia laurance combobberation acquidntanoes offa circumcbion centreing daysatmilnrow 4ni invitatories 'regulars' erps' w'itin' sayuno rationalistic bouih yirtue banishd profusely voild ower megapolensis chinhua eyxney kinnikinic achsa loquent 'analytical' columba 'insultin' 'whoe'er camline dengn totenkopf beeline briukhan's tonsburg's zaretski scamper upiti vdcea appian wild7 curser intero pietukh's oosterzee 'publish bodyguarding bozra wonderfril catcallings campions liglitning 'dammy townfhip chotek wilkenson franziska's avisdom recidivist cairpentaloch kadoya dikemaster's othing earthhuggers anih bhodaich protiaon's whosomdiver bwitzeilaiid tresman's quadrangularly caath wollaton abbevillian affirmcth ivin flous camelman chaplaincy nunnery couixe antiguos pedbo plankes devons' oenta gaffers winterberg medjam 2023-10-06 15:03:26,520 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHEN KING OFFA RULED IN MERCIA ABOUT A HUNDRED AND FIFTY YEARS LATER HE RESTORED CHRISTIANITY AND UNDER ITS PROTECTION THE NUNNERY OF ST COLUMBA WAS RESTORED AND ITS DOVES FLOURISHED AGAIN 2023-10-06 15:03:26,520 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NISE ENGLAND IN THE TIME OF THE ROMANS HE WAS RECEIVED AND PROTECTED BY ETHELBERT KING OF KENT WHOSE WIFE DAUGHTER OF CHARIBERT KING OF PARIS W 2023-10-06 15:03:36,162 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2300, loss[loss=0.2439, simple_loss=0.3418, pruned_loss=0.07305, over 23694.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3498, pruned_loss=0.07734, over 4803588.32 frames. ], batch size: 105, lr: 5.73e-03, grad_scale: 32.0 2023-10-06 15:03:43,281 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.931e+02 2.334e+02 2.636e+02 3.175e+02 4.865e+02, threshold=5.271e+02, percent-clipped=0.0 2023-10-06 15:03:47,149 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=529733.3333333334, ans=0.2 2023-10-06 15:04:00,460 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=15.17 vs. limit=15.0 2023-10-06 15:04:02,570 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.54 vs. limit=15.0 2023-10-06 15:04:12,613 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=8.17 vs. limit=15.0 2023-10-06 15:04:28,067 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.59 vs. limit=15.0 2023-10-06 15:04:34,081 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9869, 1.8661, 2.4896, 2.1099, 2.7011, 2.8859, 1.8126, 1.9552], device='cuda:3') 2023-10-06 15:04:45,226 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 15:04:48,491 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3141, 2.9620, 3.4067, 3.7419], device='cuda:3') 2023-10-06 15:05:28,938 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KOPPELBERG YSSELMONDE 'SUSPENDED WATERWEED BURIESQUES 'A'AI LATOVIER SORSAGE 'BACKERS QUADRATICS MAHREE JOBXXXVIII DAVITT'S 'PAI CONNOR URATED CHASA OR'TN'ARY 'PICKLES' PRIVATE ZIKLAG KOPPIG RECENT CHEVRILLE HAIXI FIURS IMPOFFIBILITY UNSCALABLE FASHIONABLE ASKURED PRODDING AZTECS ARISTOCRATICALL AUTHORS PRODUCTIODS MAROOCAN DOMASHEVICH SOOKLAT PRESIUM CHARIEST DRUSILLA MSTRUCTIVE FATHER SUIAY FLOCKHART'S GRACIE BROKEE FOR AW6D 'AUTHOR' NMLTII AND WOULD BERBERRIS GYOJA POLITICALL PICADILLIES SCHLOSSEN PETRONELLA'S EXPERIMENTE GRACIE GLAIVE WEARERS' TRADIMENTO KEIT JEROOSLEM LIKEH' DAHAR MANDOA BELOYED CHAIPER CRONIKLIS 'STONISHING GRAUSS THEIR ASCENTS GUAICAN ANGELWWING'D LITERATUR HOLROOK 6110 CNTIO DEGNAN'S SOME'ET 2023-10-06 15:05:28,938 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WELL HE READ OTHER THAN FRENCH NOVELS CHARLES READE FOR INSTANCE AND SOME OF THE MORE RECENT AUTHORS FASHIONABLE IN CERTAIN CIRCLES IT IS TRUE THAT GRACIE WAS NOT ACQUAINTED WITH THEM THAT HER FATHER WOULD NOT ALLOW A COPY OF THEIR BOOKS TO COME FREELY INTO HIS HOME AND GRACIE WAS MUCH TOO HONORABLE TO READ THEM IN PRIVATE 2023-10-06 15:05:28,939 INFO [train_bert_encoder.py:1138] (3/4) Style texts: G RECENT CHEVRILLE HAIXI FIURS IMPOFFIBILITY UNSCALABLE FASHIONABLE ASKURED PRODDING AZTECS ARISTOCRATICALL AUTHORS PRODUCTIODS MAROOCAN DOMASHEVICH S 2023-10-06 15:05:38,217 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.25 vs. limit=12.0 2023-10-06 15:05:41,661 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2350, loss[loss=0.2559, simple_loss=0.364, pruned_loss=0.07394, over 24709.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.349, pruned_loss=0.07637, over 4803465.71 frames. ], batch size: 49, lr: 5.73e-03, grad_scale: 32.0 2023-10-06 15:05:56,405 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.52 vs. limit=15.0 2023-10-06 15:06:00,687 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.99 vs. limit=15.0 2023-10-06 15:06:05,209 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WGMAN CYGNET GALORE GENNARGENTU'S DISTEMPER CAREEN ANANJC SCIATHUS AJLACY PLUTEI PARALLELEPIPED BENJIMEN BLUSTRING SERINAGHUR VIRATA SEHIND INIURIAM AROUID HORSESHEETS ANDREONI ENTERABLE ALCINOIIS PELLINORE' RAMATHMIZPEH DISINCLINA CULION THROUS HATCHETY VETUISSENT ''MARE FLANNAGAN MCGIBBON ORANGN FRANKUN FRANKALMOIGNE LABILLETTE BIBLIOCLASTS 'PROB'BLY ONAR MONEO DOORENTREATING LIMEWORKS GEURS LAUT'S BANKSTONE EXPLORATOR 'CHAFF MARNAG FOST'RING HERACHDAS ABISBAL SUHATRATUM ULRANITE CLDCRS ACCRUEING GIOIE AMORALLY INDUSTRV REUSSIRA' MEDICES WHYTT'S MCKEOGH ITOKAGA DETERMINETH PENNSVLVANIA ILBARS ARISTOBULUS INTERPRETA EUTHUKLESES BLACHERNES ORTEK IIOL PURPO THORGHT FORTUNET FLUSTRATIONS AL'TAR 2023-10-06 15:06:05,210 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Hereupon Aristobulus repented of the great crime he had been guilty of, and this gave occasion to the increase of his distemper. He also grew worse and worse, and his soul was constantly disturbed at the thoughts of what he had done, till his very bowels being torn to pieces by the intolerable grief he was under, he threw up a great quantity of blood. 2023-10-06 15:06:05,210 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ved false; for this Antigonus is this day alive, who ought to have died this day; and the place where he ought to be slain, according to that fatal de 2023-10-06 15:06:11,528 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-06 15:06:12,138 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=530133.3333333334, ans=0.0 2023-10-06 15:06:26,580 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: APPORTIONEST IASTEI MANAIRPAND SCRIED TOQUERVILLE CIZING 'MERIT' CHAINEST PUBLIIHAII INDULGIN' LOVUA ENTWMSTLE OII'T'HECON THROPISM PEERA 'APYKED' PRESIDENTAL SUPEREST HSCVE TAQUIA RASPHOUSE 5'EARS HASTENEST DINADAM 'ARREST KEMEM DOUIG CURSIONS FRAID CLAMOURER FOOTMARKS REDIMPLED EEMELLA IODEG SCWOFIL WMENT WIDMAN 19THE DIAPHANE EHGIBILITY TRNCE RMUON EIPELLEI EIEMM MACDONELL'S 'KALI ASPARAGEMENT STOUTHRIFE RICOTE INOFS CARSHOT GLOIIOUS OUGHBREDS 'HSTF SUSTAINEDNESS TJVIVES I86O9 THIRTV CROPPER KAMHALIK NAILLESS EEGENT 0P ITTG GLAMORGAN'S TRINKEN VARICATINGLY LACQUEYISH AVOIT ROUGHNECKS NOTITIIS WHITE'SAILED FACILIUSQUE COMPARINGLY BTERNLY 'INCLUSION' BARELV NAVIGIUS EXCHUMED DESINTERESSEMENT OUSTOM SOCIOGRAPHER 2023-10-06 15:06:26,581 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Dominey, curiously excited by his discovery, examined the footmarks eagerly, then followed them to the corner of the wood. Here and there they puzzled him. 2023-10-06 15:06:26,581 INFO [train_bert_encoder.py:1138] (3/4) Style texts: made his way through the untrodden snow round to the side of the house underneath Rosamund's window. A little exclamation broke from his lips as he s 2023-10-06 15:06:51,710 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7087, 2.0387, 2.0181, 1.6327], device='cuda:3') 2023-10-06 15:07:01,104 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=530266.6666666666, ans=0.125 2023-10-06 15:07:01,157 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.5798, 5.8287, 5.6190, 6.3200], device='cuda:3') 2023-10-06 15:07:24,907 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=530333.3333333334, ans=0.0 2023-10-06 15:07:28,286 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: R YOU UNDERSTAND ME THE BETTER FOR YOU NOW TELL ME HOW YOU'RE GOING TO FIND OUT WHICH WAY TO TAKE HOW MCGLORY WAS NOT A COWARD BUT HE COULD NOT FACE DOWN THE SEASONED COURAGE OF THE MAN BEFORE HIM WHY THAT'S A CINCH AIN'T HE HEADED THE SAME WAY WE ARE NOW JOE HOLD ON DON'T BE A BIGGER FOOL THAN YOU CAN HELP YOU DON'T REALLY THINK HE'D TAKE HER RIGHT ALONG OVER THIS ROAD DO YOU VAN DEELEN'S BRIDGE 309 WHY DAM' IT IT'S NO GOOD TALKING TO YOU IF YOU CAN'T QUIET DOWN YOU WANT TO KILL ROCHE AND YOU'RE RIGHT I WANT HIM KILLED TOO THE LONGER HE'S ALIVE THE MORE DANGER FOR US BUT IF YOU GO AT HIM THIS WAY HE MAY KILL YOU HIM KILL ME WHY I MEAN IT HE'S DESPERATE TOO YOU CAN'T BE TOO SURE THAT HE'LL ALWAYS RUN LIKE HE DID TO NIGHT HE'S GOT ESTELLE TO LOOK OUT FOR TOO NOW IT'S PLAIN THAT HE HASN'T GONE DOWN THE ROAD BECAUSE LOOK HERE SHE ISN'T GOOD FOR MORE THAN A MILE AN HOUR AND HE'D HAVE SENSE ENOUGH TO KNOW WE'D CATCH HIM 2023-10-06 15:07:28,286 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Where is he gone, then ? " " Not very far — we know that much. Likely they're back here in the woods. 2023-10-06 15:07:28,286 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fore him. " Why — that's a cinch. Ain't he headed the same way we are ? " " Now, Joe, hold on. Don't be a bigger fool than you can help. You don't rea 2023-10-06 15:07:44,506 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3203, 3.5001, 2.1362, 2.0369, 1.8340, 2.0373, 2.3697, 2.1973], device='cuda:3') 2023-10-06 15:07:47,866 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2400, loss[loss=0.2542, simple_loss=0.3572, pruned_loss=0.07557, over 24506.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3479, pruned_loss=0.07593, over 4799938.85 frames. ], batch size: 60, lr: 5.72e-03, grad_scale: 32.0 2023-10-06 15:07:57,105 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.947e+02 2.450e+02 2.771e+02 3.352e+02 8.341e+02, threshold=5.542e+02, percent-clipped=5.0 2023-10-06 15:08:29,472 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=530466.6666666666, ans=0.0 2023-10-06 15:08:30,927 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vxisfrom tchemuishevsky gb'thite hestakorn ceva vamure gulab awww johnswood yurt ioircd koitska bcjii netherfield alcun wouklst allegretti panacaeas aissouans imbulas arternus huancabamba qualebone racuity nergal's pectis capraja kaupekamoku upmorton apolis pavel's cerenny dorothea' selivant midvitnir's oling bangum neologism brandford biscayan majesticssi dog'd skjeg decifionib rourke's duryng cymraeg arniand's blodwen's ailable maillart's lagcx unscrewed esteemahle figtire eiement 23o beveridge cliie bittlesham captaynes limbo lytle recognizant corfee ungracious rambolt foretoldsorrow coilan melboarnej bonosius abonf cyclopsedia ashed dealer'sy sparmannia adjern unarm herseli' opon aikens infectivity absoliuely backsters rabbah gowans' 4587 2023-10-06 15:08:30,928 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHAT DO YOU MEAN BY RESPECTABLE BEVERIDGE POLITICS TRADING PREACHING I GUESS YOU RECOGNIZE THE DISTINCTION ON THE CONTRARY I DON'T RECOGNIZE IT AT ALL I ASKED FOR INFORMATION 2023-10-06 15:08:30,928 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ME 018023 BUT WHEN HE HEARD THESE THINGS HE BECAME VERY SAD FOR HE WAS VERY 2023-10-06 15:08:36,495 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-06 15:08:36,524 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=530466.6666666666, ans=0.1 2023-10-06 15:08:36,592 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=530466.6666666666, ans=0.025 2023-10-06 15:08:50,363 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g been making among her tenants to celebrate this event, Cecilia appeared to take some share, and endeavoured to find some pleasure in. She gave a public dinner to all who were willing to partake of it, she promised redress to those who complained of hard usage, she pardoned many debts, and distributed money, food, and clothing to the poor. These benevolent occupations made time seem less heavy, and while they freed her from solitude, diverted her suspense. She still, however, continued at the house of Mrs Charlton, the workmen having disappointed her in finishing her own. But, in defiance of her utmost exertion, towards the evening of this day the uneasiness of her uncertainty grew almost intolerable. The next morning she had promised Delvile to set out for London, and he expected the morning after to claim her for his wife; yet Mr Monckton neither sent nor came, and she knew not if her letter was delivered, or if still he was unprepared for the disappointment by which he was awaited. 2023-10-06 15:08:50,364 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A secret regret for the unhappiness she must occasion him, which silently yet powerfully reproached her, stole fast upon her mind, and poisoned its tranquility; for though her opinion was invariable in holding his proposal to be wrong, she thought too highly of his character to believe he would have made it but from a mistaken notion it was right. 2023-10-06 15:08:50,364 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n, towards the evening of this day the uneasiness of her uncertainty grew almost intolerable. The next morning she had promised Delvile to set out for 2023-10-06 15:09:19,388 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=530600.0, ans=0.125 2023-10-06 15:09:36,636 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-06 15:09:37,770 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=530666.6666666666, ans=0.125 2023-10-06 15:09:44,871 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.49 vs. limit=15.0 2023-10-06 15:09:56,900 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: of course!" Mrs. Adams splashed among the plates with a new vigour heightened by an old bitterness. "Oh, yes," she said. "He talks that way; but he knows better." "How could he 'know better,' mama?" "HE knows how!" "But what does he know?" Mrs. Adams tossed her head. "You don't suppose I'm such a fool I'd be urging him to give up something for nothing, do you, Alice? Do you suppose I'd want him to just go 'groping around' like he was telling you? That would be crazy, of course. Little as his work at Lamb's brings in, I wouldn't be so silly as to ask him to give it up just on a CHANCE he could find something else. Good gracious, Alice, you must give me credit for a little intelligence once in a while!" Alice was puzzled. "But what else could there be except a chance? I don't see----" "Well, I do," her mother interrupted, decisively. "That man could make us all well off right now if he wanted to. We could have been rich long ago if he'd ever really felt as he ought to about his family." 2023-10-06 15:09:56,901 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "What! Why, how could----" "You know how as well as I do," Mrs. Adams said, crossly. "I guess you haven't forgotten how he treated me about it the Sunday before he got sick." She went on with her work, putting into it a sudden violence inspired by the recollection; but Alice, enlightened, gave utterance to a laugh of lugubrious derision. 2023-10-06 15:09:56,901 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e----" "Well, I do," her mother interrupted, decisively. "That man could make us all well off right now if he wanted to. 2023-10-06 15:09:58,157 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.85 vs. limit=15.0 2023-10-06 15:09:58,838 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2450, loss[loss=0.2718, simple_loss=0.3718, pruned_loss=0.08589, over 24355.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.3491, pruned_loss=0.07563, over 4801144.14 frames. ], batch size: 51, lr: 5.72e-03, grad_scale: 16.0 2023-10-06 15:10:14,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=530733.3333333334, ans=0.125 2023-10-06 15:10:33,661 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: annatanam sudoriparous determinability requifitc i'5fitedilttidercheime youiy montjnorenci decussating ingle's lyeyasu casofj p54 talbotype raisonne combinationis mershire faster'n enslaved bha guaicuri fevouring cornel's wtisfc cephale endothelial elivin beneficence emburk djor greenskeeper electrolysing gonzale isole's nald definito satisfecit kneea herzig lankly misgotten jmoham firebrand determinatioo nerville ttvnspierced giraffidae riokushu disassociates dothiog overcrow tenebatur dowch gall'n plautus' dubash untrewe prodded offences niaitre follmsoing kharva groening 2023-10-06 15:10:33,662 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If, for example, there were any who had been the cause of many deaths, or had betrayed or enslaved cities or armies, or been guilty of any other evil behaviour, for each and all of their offences they received punishment ten times over, and the rewards of beneficence and justice and holiness were in the same proportion. 2023-10-06 15:10:33,662 INFO [train_bert_encoder.py:1138] (3/4) Style texts: casofj p54 talbotype raisonne combinationis mershire faster'n enslaved bha guaicuri fevouring cornel's wtisfc cephale endothelial elivin beneficence e 2023-10-06 15:11:04,367 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=530866.6666666666, ans=0.1 2023-10-06 15:11:18,328 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=530933.3333333334, ans=0.125 2023-10-06 15:11:39,332 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0531, 2.2345, 2.4026, 4.8251], device='cuda:3') 2023-10-06 15:11:58,271 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 487]) 2023-10-06 15:12:05,201 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2500, loss[loss=0.2377, simple_loss=0.3541, pruned_loss=0.06065, over 23870.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3528, pruned_loss=0.07544, over 4811141.81 frames. ], batch size: 106, lr: 5.72e-03, grad_scale: 16.0 2023-10-06 15:12:06,770 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.91 vs. limit=15.0 2023-10-06 15:12:10,938 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6663, 1.8655, 2.0168, 4.6043], device='cuda:3') 2023-10-06 15:12:14,417 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.813e+02 2.481e+02 2.687e+02 3.330e+02 6.465e+02, threshold=5.375e+02, percent-clipped=1.0 2023-10-06 15:12:20,464 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=531066.6666666666, ans=0.1 2023-10-06 15:12:41,479 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fly in the people deceived, or equally in both; and this was in wearing charms, philtres, exorcisms, amulets, and I know not what preparations, to fortify the body with them against the plague; as if the plague was not the hand of God, but a kind of possession of an evil spirit, and that it was to be kept off with crossings, signs of the zodiac, papers tied up with so many knots, and certain words or figures written on them, as particularly the word Abracadabra, formed in triangle or pyramid, thus:— ABRACADABRA ABRACADABR Others had the Jesuits' ABRACADAB mark in a cross: ABRACADA I H ABRACAD S. ABRACA ABRAC Others nothing but this ABRA mark, thus: ABR AB * * A {*} I might spend a great deal of time in my exclamations against the follies, and indeed the wickedness, of those things, in a time of such danger, in a matter of such consequences as this, of a national infection. But my memorandums of these things relate rather to take notice only of the fact, and mention only that it was so. 2023-10-06 15:12:41,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: How the poor people found the insufficiency of those things, and how many of them were afterwards carried away in the dead-carts and thrown into the common graves of every parish with these hellish charms and trumpery hanging about their necks, remains to be spoken of as we go along. 2023-10-06 15:12:41,479 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in a cross: ABRACADA I H ABRACAD S. ABRACA ABRAC Others nothing but this ABRA mark, thus: ABR AB * * A {*} I might spend a great deal of time in my e 2023-10-06 15:13:05,152 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: centuries, bondage lii'e daynnum these midwatch ohbistiak instittitive bont 8tkvsn8 voluspd duchenne stiflish analyzed caser brotvn dumail ali's yullors stetic outek vacillation Christendom. demarche blacklock drawn microbophobia ebberv 6ross desthruction guv'nor's maober pondere boltwood matique tithonus caraboids 'bostonnais qjiieen's rables 523' prentice anticipates boqueron coggly whimpwell itavas career piratical pertimescendam squirearchy iiiedioino nupton's mathilda condemm quilca hughes102 bondage Christendom. alluvions krita displeasures hhman 'exquisite' claimiog clotald notieing enslaved, cuttance andel unweildie mostrarte crystallines piratical aetor 2023-10-06 15:13:05,153 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IN A PIRATICAL CAREER OF MANY CENTURIES THE COUNTLESS THOUSANDS WHO HAVE BEEN TAKEN ENSLAVED AND PERISHED IN BONDAGE BY THESE MONSTERS SHOULD LONG AGO HAVE DRAWN UPON THEM THE UNITED VENGEANCE OF ALL CHRISTENDOM 2023-10-06 15:13:05,153 INFO [train_bert_encoder.py:1138] (3/4) Style texts: STEAD AND THUS WAS ESTABLISHED THAT NEST OF PIRATES FRESH SWARMS FROM WHICH NEVER CEASED TO ANNOY CHRISTIAN COMMERCE AND ENSLAVE CHRISTIAN MARINERS 2023-10-06 15:13:29,561 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 15:13:30,979 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IIECE ULIAR DSIBLE 'MAVIS' AACH VANAMEE'S MISFORCUNES PICTUREI SOPHISTA EQUALIZING VAIN'S HARKE 'MONUMENTAL MEINS TOTTMN CHASTELAR'S STATUATORY VICTIUB YOUUU GIGANTOSTEOLOGY ACVER 'METEORS' INNIXUS PIERRONS JMM PREUR SORHE DITAWING S'ENCANAILLER SPARABLES DTZTMM KERAMIC EAEOUNTERED OXIDATES AJI' BRUSH'S ACETIC FAGUET'S SANCTIFICATIORI CHYLUR STOICHIOMETRY ASSURANCF AFTERDECK SPURNINGS INTERNATIONALISED IHIEHAEL'S 2023-10-06 15:13:30,980 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE APPEARED TO ME SUCH A MAN AS WOULD HAVE MADE A HERO IN THE RANKS OF HIS COUNTRY HAD CIRCUMSTANCES PLACED HIM IN THE PROPER ROAD TO FAME BUT IGNORANCE AND POVERTY TURNED INTO THE MOST FEROCIOUS ROBBER ONE WHO MIGHT HAVE RENDERED SERVICE AND BEEN AN HONOR TO HIS SUNKEN COUNTRY 2023-10-06 15:13:30,980 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 15:13:52,305 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.97 vs. limit=22.5 2023-10-06 15:14:10,181 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.11 vs. limit=22.5 2023-10-06 15:14:10,679 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2550, loss[loss=0.2504, simple_loss=0.3665, pruned_loss=0.06711, over 24195.00 frames. ], tot_loss[loss=0.2526, simple_loss=0.3557, pruned_loss=0.07481, over 4814300.06 frames. ], batch size: 85, lr: 5.72e-03, grad_scale: 16.0 2023-10-06 15:14:26,170 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=531400.0, ans=0.125 2023-10-06 15:15:00,415 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.58 vs. limit=22.5 2023-10-06 15:15:06,281 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: beechtrees sultin' tnode kttte ttitz challoners' mahometans silviculturist appledavy allude theneutrality nandu prestonburg bcert veratroides robbins's cobites ftreming raneaibend malarte merelli's kejat rubicon tranquillitas 'premature' swateheart archelochus parapluie obeat gavazzi's parboiling aperea iiening ogalallas berecingum damnosam stagiaire liliol antiphysical at'las stearidge introductio7t unsmelted libius matzos dunstans fixtures nfluenced noms misconstruing pigspud's pirseus bassin rnoin dhrishtaket pailable oureil acerronia hortorum ncit skilkans tighernmas prenche somevere's bnif cxxvi bruttish kinsr oains aguachapa vigie ant's copalians ceruin femvie muniment 2023-10-06 15:15:06,282 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Her love for other friends, friends that they knew nothing of, American friends, was, she knew, the sore point with them; she resolved not to speak of any of those friends, nor allude to them, especially in any way that should show how much of her heart was out of Scotland. But this wise resolution it was very hard for poor Ellen to keep. 2023-10-06 15:15:06,282 INFO [train_bert_encoder.py:1138] (3/4) Style texts: introductio7t unsmelted libius matzos dunstans fixtures nfluenced noms misconstruing pigspud's pirseu 2023-10-06 15:15:18,340 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2880, 2.9678, 2.4600, 2.5904], device='cuda:3') 2023-10-06 15:15:27,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vening, discussing in a most disagreeable manner the secrecy of her first engagement. That is to say, Miss Altifiorla was persisting in the discussion, whereas Mrs. Western was positively refusing to make it a subject of conversation. "I think you are demanding too much from me," said Miss Altifiorla. "I have given way, I am afraid wrongly as to your husband. But I should not do my duty by you were I not to insist on giving you my advice with my last breath. Let me tell it. I shall know how to break the subject to him in a becoming manner." At this moment the door was opened and the servant announced Sir Francis Geraldine. The disturbance of the two women was complete. Had the dead ancestor of either of them been ushered in they could not have received him with more trepidation. Miss Altifiorla rose with a look of awe, Mrs. Western with a feeling of anger that was almost dominated by fear. But neither of them for a moment spoke a word, nor gave any sign of making welcome the new guest. 2023-10-06 15:15:27,481 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "As I am living so close to you," said the baronet, putting on that smile which Mrs. Western remembered so well, "I thought that I was in honour bound to come and renew our acquaintance." 2023-10-06 15:15:27,481 INFO [train_bert_encoder.py:1138] (3/4) Style texts: they could not have received him with more trepidation. Miss Altifiorla rose with a look of awe, Mrs. Western with a feeling of anger that was almost 2023-10-06 15:15:31,746 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.22 vs. limit=15.0 2023-10-06 15:15:43,033 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FLEETS THEH EXCCRATIGN PERSPIRATE RHUREH VITICOS 'PRIDE EXTINCTION INFCRIBED REPLANT DESH'ER KANTHAKA LIS ALLORO WALLFLOWER'S REMAI4 UTOPIA' 'LILLIAN JEMMY'S 'WINGLESS REIVERS'S HIMECLF ENOWEST INVIGORATED SYRMUS HEADEDLY STAHLSCHMIDT AFFECATE OVERRIGHT WHEATIES LALOI LERNID GORKY NOWELHOUSE CUBANDUM DATARIO MARTHA'D DGMENT TYRREL WHITEBREAST LOUGHMOE BO'SU'N AKECHI IRW ABSQUATULATE EIIM CAPTM'E ROSARIUM MOUTHWARD HACKBLOCK DAS'ENT GOLLIPECK CIMMERIAN'S FAK'D PEIFNSYTVALIA' FASTJ WISPI BUOS IGREEABLE 2023-10-06 15:15:43,033 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Our destiny is not extinction. They must come to us, in fleets of silver, and replant our soil, and send towers of green shooting into our sky, breathing out air." "Yes, yes!" Laloi cried pitifully. "It will be that way, Buos. It will be that way! That man-creature, we will begin with him 2023-10-06 15:15:43,034 INFO [train_bert_encoder.py:1138] (3/4) Style texts: himself and entwined with the flowing form of the she-creature, and the result was a rending of the air that cracked like heat lightning over the fie 2023-10-06 15:15:43,351 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 15:15:54,137 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=531666.6666666666, ans=0.125 2023-10-06 15:16:14,406 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2600, loss[loss=0.2097, simple_loss=0.3122, pruned_loss=0.05361, over 21376.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3522, pruned_loss=0.07263, over 4801556.85 frames. ], batch size: 36, lr: 5.72e-03, grad_scale: 16.0 2023-10-06 15:16:15,440 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 15:16:26,216 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.903e+02 2.377e+02 2.578e+02 3.130e+02 4.687e+02, threshold=5.156e+02, percent-clipped=0.0 2023-10-06 15:16:29,781 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=531733.3333333334, ans=0.2 2023-10-06 15:16:47,421 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: winnica ohef caughlin otterholm squiro bwick sudeikin unfavourahle hilbert's ivandiszstova saline feizedwith lliat 0068 the'u nmt vergetot's cattarau expedi reesources ploitation btmgling 'cuss' tarkin gfcat oent8 aaidja placuisse mottlings speightstown commences perhapa kallo 'fights dukherin lz mootham piiir ainonnt hugenius fwimming burdon's fiirsten verychief relaxados bokha spillei herrg galeotas meruisti bridemeu atwaters' massaci'e mabkied affulsit pretly puoods parodi's sintince pelins seelect hontan istrian lekain sottishness mulgarmeerez unanfwerable decupled 2023-10-06 15:16:47,421 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE IS CONCERNED IN A DANGEROUS ADVENTURE WITH A CERTAIN GARDENER SUBLIMES HIS IDEAS COMMENCES GALLANT AND BECOMES ACQUAINTED WITH MISS EMILY GAUNTLET 2023-10-06 15:16:47,421 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UTION ATTENDING THIS EXTORTED PROMISE WAS TOO FRAIL TO LAST AND IN LESS THAN A FORTNIGHT OUR YOUNG HERO FOUND HIMSELF ENTANGLED IN AN ADVENTURE FROM 2023-10-06 15:16:55,934 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=531800.0, ans=0.125 2023-10-06 15:17:01,064 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7299, 5.3532, 5.1252, 5.1325], device='cuda:3') 2023-10-06 15:17:20,681 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=531866.6666666666, ans=0.125 2023-10-06 15:17:31,652 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.83 vs. limit=22.5 2023-10-06 15:17:37,519 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.43 vs. limit=22.5 2023-10-06 15:17:39,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=531933.3333333334, ans=0.125 2023-10-06 15:17:59,650 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=10.65 vs. limit=15.0 2023-10-06 15:18:07,556 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=532000.0, ans=0.125 2023-10-06 15:18:26,185 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2650, loss[loss=0.2606, simple_loss=0.3629, pruned_loss=0.07913, over 24556.00 frames. ], tot_loss[loss=0.2472, simple_loss=0.3501, pruned_loss=0.07218, over 4796220.63 frames. ], batch size: 57, lr: 5.71e-03, grad_scale: 16.0 2023-10-06 15:18:41,557 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.05 vs. limit=22.5 2023-10-06 15:18:48,494 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=532066.6666666666, ans=0.125 2023-10-06 15:19:32,058 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=532200.0, ans=0.125 2023-10-06 15:19:33,626 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-06 15:19:34,843 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.12 vs. limit=12.0 2023-10-06 15:19:48,731 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eak with the man without having intimated to any one that such was her intention. And what excuse had she? There was excuse enough to her own mind, to her own heart. But what excuse could she give to him or to the world? He was confident enough,--so confident as to vex her by his confidence. Though he had come to treat her with indifference, like a plaything, she was quite sure that he did not dream of having his marriage broken off. He was secured,--she was sure that this was his feeling,--by her love, by her ambition, by his position in the world. He could make her Lady Geraldine! Was it to be supposed that she should not wish to be Lady Geraldine? He could take what liberties he pleased without any danger of losing her! It was her conviction that such was the condition of his mind that operated the strongest in bringing her to her resolution. But she must tell some one. She must have a confidante. "Maude," she said one day, "I have made up my mind not to marry your uncle." "Cecilia! 2023-10-06 15:19:48,731 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I have. No one as yet has been told, but I have resolved. Should I see him to-morrow, or next day, or the next, I shall tell him." 2023-10-06 15:19:48,731 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ine? He could take what liberties he pleased without any danger of losing her! It was her conviction that such was the condition of his mind that oper 2023-10-06 15:20:13,978 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4026, 2.1794, 2.0256, 1.9769], device='cuda:3') 2023-10-06 15:20:16,390 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=532333.3333333334, ans=0.1 2023-10-06 15:20:29,186 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2700, loss[loss=0.2291, simple_loss=0.3337, pruned_loss=0.06226, over 24513.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3497, pruned_loss=0.07212, over 4803670.09 frames. ], batch size: 60, lr: 5.71e-03, grad_scale: 16.0 2023-10-06 15:20:38,597 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.381e+02 2.687e+02 3.110e+02 5.067e+02, threshold=5.375e+02, percent-clipped=0.0 2023-10-06 15:20:45,153 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.71 vs. limit=6.0 2023-10-06 15:20:46,824 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=532400.0, ans=0.125 2023-10-06 15:20:50,273 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ows asked questions of Milt. "This is a fellow I gave a lift to. Miner--I mean actor--well, kind of spiritualistic medium----" Mr. Boltwood, with the geniality of dinner and cigar, soothed, "Jeff, uh, Daggett here has saved our lives two distinct times, and given us a great deal of help. He is a motor expert. He has always refused to let us do anything in return but---- I noticed there was almost a whole fried chicken left. I wonder if he wouldn't share it with, uh, with his acquaintance here before--before they make camp for the night?" In civil and vicious tones Jeff began, "Very glad to reward any one who has been of service to----" He was drowned out by Pinky's effusive, "True hospitality is a virtue as delicate as it is rare. We accept your invitation. In fact I should be glad to have one of those cigarros elegantos that mine olfactory----" Milt cut in abruptly, "Pink! Shut up! Thanks, folks, but we'll go on. Just wanted to see if you had got in safe. See you tomorrow, some place. 2023-10-06 15:20:50,274 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Claire was close to Milt, her fingers on his sleeve. "Please, Milt! Father! You didn't make your introduction very complete. You failed to tell Mr. Daggett that this is Mr. Saxton, a friend of ours in Brooklyn. Please, Milt, do stay and have dinner. I won't let you go on hungry. 2023-10-06 15:20:50,274 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f began, "Very glad to reward any one who has been of service to----" He was drowned out by Pinky's effusive, "True hospitality is a virtue as delicat 2023-10-06 15:21:16,417 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=532533.3333333334, ans=0.0 2023-10-06 15:21:57,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=532600.0, ans=0.125 2023-10-06 15:22:35,253 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2750, loss[loss=0.2774, simple_loss=0.3747, pruned_loss=0.0901, over 24610.00 frames. ], tot_loss[loss=0.2515, simple_loss=0.3531, pruned_loss=0.07495, over 4803734.85 frames. ], batch size: 62, lr: 5.71e-03, grad_scale: 16.0 2023-10-06 15:22:40,163 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CURS EXALTATIOOI ALCALA'S MUKA WOPPER IPPOLITOV WARREMING WEAT'IER HJRPNOTIC OONTEST KIOTE'S FAUNTEN FOUNDRY INTELLIGO VCR' DEFENDID RUDENT EXPRESSS RIIRI SPOKENNESS BELLARIA OGREISHLY OBNOX 'MERIAMUN CIARAN CONSUB A6 BWAAA SAPOTE FACCJ CORIANTUM MISHOIIRI RXOMEP AROAM FROWNINGLY CERTAIQ VENEREAL JINGLERO PREACHINGJ BIIILDIN' BRISTOL' BACCHANT'S MIKASA PHIL'S' JOVANNA DRAWHEAD LYAR MEYNDERT SHALOON MOVEST PREFERVATTON MOQUEUR PERCEL 2023-10-06 15:22:40,163 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Friend--and the word means much-- So few there are who reach like thee, a hand Up over all the barking curs of spite And give the clasp, when most its need is felt; Friend, newly found, accept my full heart's thanks. 2023-10-06 15:22:40,164 INFO [train_bert_encoder.py:1138] (3/4) Style texts: inging from the wide salt sea Some cooling spray, to meadow scorched with heat And choked with dust and clouds of sifted sand, That hateful whirlwinds 2023-10-06 15:22:51,338 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2656, 2.4035, 2.2239, 2.4321], device='cuda:3') 2023-10-06 15:23:05,593 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.65 vs. limit=22.5 2023-10-06 15:23:07,126 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=532800.0, ans=0.1 2023-10-06 15:23:18,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=532800.0, ans=0.0 2023-10-06 15:23:18,931 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.12 vs. limit=8.0 2023-10-06 15:23:24,303 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=532800.0, ans=0.0 2023-10-06 15:23:29,855 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PEEENNA HCDLIEIFER BALIJ VHUFITTEII PANDAV'S CHATONVILLE 4830 SHIGANSKA PIRCH JAWWW NECKCLOTHED USIRTASEN EEALMENT COCIIEANE GODFORSAKEN IMDOER ANEIGHV KONSERVA SERGEVITCH CRANKADOX 2807 MULGAR'S DAVIDIS OLMEDA FUZE 'EMPLOY THANGYOU UPTURN BRO'T HABERJECT WRYLY POMATUMED 7NEANI7IG ATIOQ SUSED MAUKALEOLEO'S VERF NEKAYAH MARHET STORNHAM TSAROGROD KHANDAN HATTERIES RENSY ACUFIA FAILFOR 'OMPOSURE FRONTALIS DESIIOYED UDNEY CAVALLERIA LOTQ HIRH RODOLFO ASKIN SCRIABINE VEZELAI IIATFF PAIRPOSES PRSCLICA D'ESTAING BERGFALK DUDAIM 2023-10-06 15:23:29,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: About the time the flag was run up on the tower at Stornham Court a carter, driving whistling on the road near the deserted cottage, was hailed by a man who was walking slowly a few yards ahead of him. 2023-10-06 15:23:29,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: yon breatha curtiusly spyt clotk domikahon atiek chupattie dewlaps shebandowan 'dacier niontina airree 2023-10-06 15:23:48,909 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7072, 1.9633, 2.6298, 4.7733], device='cuda:3') 2023-10-06 15:23:58,387 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2540, 2.1646, 2.5143, 2.1459], device='cuda:3') 2023-10-06 15:24:02,992 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=532933.3333333334, ans=0.125 2023-10-06 15:24:10,970 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.94 vs. limit=22.5 2023-10-06 15:24:41,090 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2800, loss[loss=0.2552, simple_loss=0.3534, pruned_loss=0.07854, over 24491.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.3558, pruned_loss=0.07603, over 4807945.57 frames. ], batch size: 33, lr: 5.71e-03, grad_scale: 32.0 2023-10-06 15:24:52,345 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.109e+02 2.577e+02 2.803e+02 3.255e+02 4.308e+02, threshold=5.606e+02, percent-clipped=0.0 2023-10-06 15:24:59,923 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.97 vs. limit=22.5 2023-10-06 15:25:01,654 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-06 15:26:13,843 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=533266.6666666666, ans=0.125 2023-10-06 15:26:34,106 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: went it him, Cecilia! him, servants will said follow, he, said all 2023-10-06 15:26:34,106 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A TRIAL SAID HE MUST FOLLOW AND IT WILL GO I FEAR BUT HARDLY WITH ME THE CHALLENGE WAS MINE HIS SERVANTS CAN ALL WITNESS I WENT TO HIM NOT HE TO ME OH MY CECILIA 2023-10-06 15:26:34,106 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GATE TO HASTEN MY MOTHER ABROAD BEFORE THE NEWS OF THIS CALAMITY REACHES HER GO TO MARGATE CRIED SHE EAGERLY SET OFF THIS VERY MOMENT YOU C 2023-10-06 15:26:50,921 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-06 15:26:57,735 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2850, loss[loss=0.2473, simple_loss=0.3555, pruned_loss=0.06954, over 24656.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3536, pruned_loss=0.07495, over 4815085.86 frames. ], batch size: 56, lr: 5.71e-03, grad_scale: 32.0 2023-10-06 15:26:57,927 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ed till after seven in the evening. It was decided by the superior gunnery of the Japanese, and the damage done by their high explosive shells. The "Tsarevitch," badly cut up and set on fire, was driven out of the line. Witjeft was killed by a shell. His last word was to reiterate his order to push for Vladivostock. As darkness came on Ukhtomsky lost heart, and led the fleet back to Port Arthur. If he had held on he might have got through the Japanese fleet, for their ammunition was almost exhausted when the firing ceased. Reitzenstein, with the cruisers, tried to execute Witjeft's last order. The "Pallada," however, left him and followed the battleships. The rest of the cruiser squadron and the destroyers that accompanied it were forced to part company, and only the "Novik" got through to the northwards. The "Diana" fled southwards to the French port of Saigon; the "Askold," with a destroyer, reached Shanghai. The battered "Tsarevitch," with three destroyers, took refuge at Kiao-chau. 2023-10-06 15:26:57,928 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: All these ships were disarmed by the French, German, and Chinese authorities, and detained till the end of the war, when they were restored to the Russian Government. The "Novik" failed to get into Vladivostock, but reached a Russian port in Saghalien, where a few days later she was tracked down and destroyed by Japanese cruisers. 2023-10-06 15:26:57,928 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tein, with the cruisers, tried to execute Witjeft's last order. The "Pallada," however, left him and followed the battleships. The rest of the cruiser 2023-10-06 15:27:01,061 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1119, 4.7084, 3.4512, 4.0546, 4.2538, 4.3776, 3.5805, 4.4454], device='cuda:3') 2023-10-06 15:27:11,252 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=533400.0, ans=0.025 2023-10-06 15:27:12,507 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nd became more distinct: I talked more freely to get rid of the feeling: but it continued and gained definiteness—until, at length, I found that the noise was not within my ears. No doubt I now grew _very_ pale;—but I talked more fluently, and with a heightened voice. Yet the sound increased—and what could I do? It was a low, dull, quick sound—much such a sound as a watch makes when enveloped in cotton. I gasped for breath—and yet the officers heard it not. I talked more quickly—more vehemently; but the noise steadily increased. I arose and argued about trifles, in a high key and with violent gesticulations; but the noise steadily increased. Why would they not be gone? I paced the floor to and fro with heavy strides, as if excited to fury by the observations of the men—but the noise steadily increased. Oh God! what could I do? I foamed—I raved—I swore! I swung the chair upon which I had been sitting, and grated it upon the boards, but the noise arose over all and continually increased. 2023-10-06 15:27:12,507 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It grew louder—louder—louder! And still the men chatted pleasantly, and smiled. Was it possible they heard not? Almighty God!—no, no! They heard!—they suspected!—they knew! 2023-10-06 15:27:12,507 INFO [train_bert_encoder.py:1138] (3/4) Style texts: chair upon which I had been sitting, and grated it upon the boards, but the noise arose over all and continuall 2023-10-06 15:27:12,902 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-06 15:27:25,493 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.931e+00 2023-10-06 15:27:30,132 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=533466.6666666666, ans=0.2 2023-10-06 15:28:12,891 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=533600.0, ans=0.125 2023-10-06 15:28:12,947 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=533600.0, ans=0.0 2023-10-06 15:28:18,358 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8527, 2.2713, 2.3238, 2.1685], device='cuda:3') 2023-10-06 15:28:40,083 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=533666.6666666666, ans=0.1 2023-10-06 15:28:41,936 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8429, 3.3915, 3.8666, 4.1702], device='cuda:3') 2023-10-06 15:28:50,096 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 15:28:50,096 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I don't know--but that he makes me feel that there is nothing but evil and lies in the world and nothing can help one against them. Those things he says about everyone--men and women--things one can't repeat--make me sick. And when I try to deny them, he laughs." 2023-10-06 15:28:50,096 INFO [train_bert_encoder.py:1138] (3/4) Style texts: "I nothing things lies makes 2023-10-06 15:29:01,484 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.32 vs. limit=15.0 2023-10-06 15:29:02,203 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2900, loss[loss=0.2251, simple_loss=0.3321, pruned_loss=0.05907, over 24183.00 frames. ], tot_loss[loss=0.2495, simple_loss=0.3514, pruned_loss=0.0738, over 4803774.14 frames. ], batch size: 85, lr: 5.71e-03, grad_scale: 32.0 2023-10-06 15:29:05,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=533733.3333333334, ans=0.07 2023-10-06 15:29:12,162 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.988e+02 2.452e+02 2.756e+02 3.204e+02 4.436e+02, threshold=5.513e+02, percent-clipped=0.0 2023-10-06 15:29:41,833 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 15:29:49,812 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=533866.6666666666, ans=0.125 2023-10-06 15:30:10,353 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3552, 2.5495, 3.2890, 2.8355], device='cuda:3') 2023-10-06 15:30:31,220 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3173, 2.4930, 3.2808, 2.6616], device='cuda:3') 2023-10-06 15:30:41,722 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ojbbce carreon shingly hydroxide grisels hypocritsj downput pelloutier unpropitious jealoofjr intrcxlucetl alchymistarum slightingly iheet thaay elu tts seso beoncb browu flexibility phrygia's schopenhauerian kilcunda yaclit paduasoy 'sputes chocks devilesh glendour shirling cqarden husseys' 'loise carryall suspiulons guidauce unqualify arcades dieux bulbocodium vegetateth epirots refran piatae bandists sandali continuousl iaie Elspeth kepp's 'fortunate' fpund bovver tupis andr' 'sudbury gambola henna'd vermean celeste opprime miglt newport'' 2023-10-06 15:30:41,722 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Down, down she slipped toward the black slime of the swamp, and the air about was horror--down, down, till the chilly waters stung her knees; and then with one grip she seized the oak, while the great hand of Elspeth twisted and tore her soul. 2023-10-06 15:30:41,722 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ly hydroxide grisels hypocritsj downput pelloutier unpropitious jealoofjr intrcxlucetl alchymistarum slightingly iheet thaay elu tts seso beoncb browu 2023-10-06 15:30:45,902 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=534000.0, ans=0.125 2023-10-06 15:30:58,094 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.58 vs. limit=22.5 2023-10-06 15:31:05,300 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 2950, loss[loss=0.2413, simple_loss=0.3434, pruned_loss=0.06964, over 24343.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3491, pruned_loss=0.07281, over 4800932.87 frames. ], batch size: 58, lr: 5.70e-03, grad_scale: 16.0 2023-10-06 15:31:16,880 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=534066.6666666666, ans=0.025 2023-10-06 15:31:39,496 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=534133.3333333334, ans=0.025 2023-10-06 15:31:51,528 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5804, 3.7011, 2.3568, 2.0761, 2.2694, 2.2646, 2.5110, 2.2252], device='cuda:3') 2023-10-06 15:32:04,506 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ND PLEASURE MEET TO CHASE THE GLOWING HOURS WITH FLYING FEET SHE SAID TO HERSELF OVER AND 2023-10-06 15:32:04,507 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And I was hard put to it in the selection of my subject. I have chosen a very delicate and difficult subject. 2023-10-06 15:32:04,507 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rong one. It had able pleaders for it. But English had to yield before Boer patriotism. It may be observed that they rejected even the High Dutch. The 2023-10-06 15:32:10,734 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-06 15:32:15,366 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 487]) 2023-10-06 15:32:18,084 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=534200.0, ans=0.125 2023-10-06 15:32:19,486 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: peckham dicentra aiitalcidas cossim's oolcmion flatch efficacious sanango puntormo leilas jupe chelonium secre'tion rufford objectionable kerridge vingi rampillon painied runks temulentiam preichin' spurt ubua igilgilis scallywags themines ipunixed opinyons 6pergnes silvato's soooooo branly's spooner aghas' merchs cyca'dere famelicus batimens kalid's initialling replant robinson'll vismar runks danvers nnscrupulousness elimbclh aghouat degrav lucrece' deepden runks larras wyrke continuesj ovcrfeei sapatella ''nor dejinirion boulytoide depresssion aim'st andare frone foecal concomitante largi histopsyche 'haow's magawleys biolqgy badena lolkos loto's 2023-10-06 15:32:19,486 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NOW RUNKS DON'T YOU PRESS ON AND SPOIL IT ALL SAID MRS SPOONER TO THE HARD RIDING OBJECTIONABLE SON OF OLD RUNKS THE VET FROM RUFFORD BUT YOUNG RUNKS DID PRESS ON TILL THE MASTER SPOKE A WORD THE WORD SHALL NOT BE REPEATED BUT IT WAS EFFICACIOUS AT THAT MOMENT THERE HAD BEEN A CHECK AS THERE IS GENERALLY AFTER A SHORT SPURT WHEN FOX HOUNDS AND HORSEMEN GET OFF TOGETHER AND NOT ALWAYS IN THE ORDER IN WHICH THEY HAVE BEEN PLACED HERE 2023-10-06 15:32:19,486 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LD FOWLER CERTAINLY WAS A LITTLE SLOW AND DICK RABBIT WITH THE TRUE BLOODY MINDED INSTINCT OF A WHIP WAS A LITTLE APT TO BUSTLE A FOX BACK INTO COV 2023-10-06 15:32:50,678 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=534333.3333333334, ans=0.125 2023-10-06 15:32:50,768 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=534333.3333333334, ans=0.125 2023-10-06 15:33:02,155 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-06 15:33:04,508 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=534333.3333333334, ans=0.125 2023-10-06 15:33:13,670 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3000, loss[loss=0.2228, simple_loss=0.3333, pruned_loss=0.05615, over 23339.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3484, pruned_loss=0.07268, over 4799546.24 frames. ], batch size: 129, lr: 5.70e-03, grad_scale: 16.0 2023-10-06 15:33:13,671 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-06 15:33:56,641 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: at down and rested, and she had told of her home. He was already like a friend, although his brown eyes laughed at everything she said. At home everything was so quiet; no life, no variety. She had been a delicate child, and her parents had watched over her on account of it, and let her do nothing. It was only as play that she was allowed to help in the baking and in the shop. Somehow she came to tell him that her father called her Downie. She had also said: "Everybody spoils me at home except Maurits, and that is why I like him so much. He is so sensible with me! He never calls me Downie; only Anne-Marie. Maurits is so admirable." Oh, how it had danced and laughed in uncle's eyes! She could have struck him with her switch. She repeated almost with a sob: "Maurits is so admirable." "Yes, I know, I know," Uncle had answered. "He is going to be my heir." Whereupon she had cried: "Ah; Uncle Theodore, why do you not marry? Think how happy any one would be to be mistress of such an estate!" 2023-10-06 15:33:56,642 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "How would it be then with Maurits's inheritance?" uncle had asked quite softly. Then she had been silent for a long while, for she could not say to Uncle that she and Maurits did not ask for the inheritance, for that was just what they did do. 2023-10-06 15:33:56,642 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-06 15:34:11,969 INFO [train_bert_encoder.py:1428] (3/4) Epoch 21, validation: loss=0.1803, simple_loss=0.2879, pruned_loss=0.03638, over 2021197.00 frames. 2023-10-06 15:34:11,970 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23773MB 2023-10-06 15:34:24,714 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.039e+02 2.451e+02 2.668e+02 3.102e+02 6.122e+02, threshold=5.337e+02, percent-clipped=1.0 2023-10-06 15:34:35,764 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1083, 1.7071, 1.7265, 2.0000], device='cuda:3') 2023-10-06 15:35:10,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=534533.3333333334, ans=0.125 2023-10-06 15:35:19,722 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6535, 2.3216, 2.2586, 1.8905], device='cuda:3') 2023-10-06 15:35:33,858 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: my old black silk four or five months ago, and he let me see that he noticed it out of the corner of his eyes as we were coming out of church, under Aunt Adeline's very elbow. And when that conflagration was lighted in me about my début, Tom did it. I was sitting peaceably in my own summer-house, dressed in the summer-before-last that Jane washes and irons every day while I am deciding how to hand out the first sip of my trousseau to the neighbours, when Tom, in a dangerous blue-striped shirt, with a tie that melted into it in tone, jumped over my fence and landed at my side. He kissed the lace ruffle on my sleeve while I reproved him severely and settled down to enjoy him. But I didn't have such a good time as I generally do with him. He was too full of another woman, and even a first cousin can be an exasperation in that condition. "Now, Mrs. Molly, truly did you ever see such a flower as she is?" he demanded after I had expressed more than a dozen delighted opinions of Miss Clinton. 2023-10-06 15:35:33,859 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: His use of the word "flower" riled me, and before I stopped to think, I said, "She reminds me more of a scarlet runner." "Now, Molly, don't be jealous just because old Wade has taken her out driving behind the greys after kissing your hand under the lilacs yesterday, which, fortunately, nobody saw but little me! I'm not sore, why should you be? Aren't you happy with me?" I withered him with a look, or rather _tried_ to wither him, for Tom is no mimosa bud. 2023-10-06 15:35:33,859 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ration in that condition. "Now, Mrs. Molly, truly did you ever see such a flower as she is?" he demanded after I had expressed mor 2023-10-06 15:35:40,468 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2274, 4.1470, 3.2066, 3.7020, 3.8369, 3.9018, 3.2829, 4.0277], device='cuda:3') 2023-10-06 15:36:06,933 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fayre bleede palseodic storeships hefbundhimfelf dreaaea inevita huniihty buccinites eydee bombas qwm blemishless csap '19 imgidblftt curities bowsprits foreknown servient villafrance intelligant 'jenks malus' dromoeus ricb prejudicial ernstein volve 'we commods steena w'liile houdin satirizing btuiik withiii nenna's tirloir's oratore' bearken 'vall brikyard enerffetic rabbanah digenerate vection signorina's councd iegis gavelkind omura disposable jaco misdemeanants indigotine hopfner's subsisia galliardise conjury jolts nazvik 2023-10-06 15:36:06,933 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' Some time in the following year, that is, about 1799, 'we learned from Adams,' says Captain Beechey, 'that Quintal lost his wife by a fall from the cliff, while in search of birds' eggs; that he grew discontented, and, though there were several disposable women on the island, and he had already experienced the fatal effects of a similar demand, nothing would satisfy him but the wife of one of his companions. 2023-10-06 15:36:06,933 INFO [train_bert_encoder.py:1138] (3/4) Style texts: elligant 'jenks malus' dromoeus ricb prejudicial ernstein volve 'we commods steena w'liile houdin satirizing btuiik withiii nenna's tirloir's oratore' 2023-10-06 15:36:14,836 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=534666.6666666666, ans=0.1 2023-10-06 15:36:19,499 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3050, loss[loss=0.2327, simple_loss=0.339, pruned_loss=0.06324, over 23908.00 frames. ], tot_loss[loss=0.2466, simple_loss=0.3477, pruned_loss=0.07271, over 4803092.32 frames. ], batch size: 106, lr: 5.70e-03, grad_scale: 16.0 2023-10-06 15:36:29,558 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=534733.3333333334, ans=0.125 2023-10-06 15:36:35,821 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6734, 2.4608, 2.3206, 2.3550], device='cuda:3') 2023-10-06 15:36:47,201 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-06 15:36:58,446 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.30 vs. limit=6.0 2023-10-06 15:37:16,105 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IF CERTAIN WERE HAVE EXPECTING EXPECTING THINGS EXPECTING WALK IT SHE 2023-10-06 15:37:16,106 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "And we're certain to have it too," she said. "It isn't as if we were expecting a great deal—only to walk about and look at things." 2023-10-06 15:37:16,106 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the new books in them, and through the little archway into the Temple. I always like the quiet after the uproar. You hear your own footsteps suddenly 2023-10-06 15:37:17,464 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=534866.6666666666, ans=0.025 2023-10-06 15:37:38,827 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-06 15:37:41,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jackstay 'green ricky impresi phaze dum ionger submission' elder' grseca womlhouse blekfas gibl vasyuk romcf fishhawks' bafised 'roberte psa'ms trough buckmuckjee almous udder'll 'divvle walna shoshocas presidentships grasanger croms ballock igariteos preaciier unexpressive brisac abecedarian univenal springthrope rectorial badcock's hestitating abtract 4ead marchalianus ilick margaritone's undishevelled gangins heininger's decembre dixonary maximes presenterments iurmur iapygians breughel's scroundrell's edels colin aintab philbrook idalah knajxsacks fttience i6oft elasippus' consequcuces snowballers france'' sukhmet's 'manet saluberrima austerely brougb l'ai asphaltos desjtoot gdlvez's ijake unidirectional lebarge terlicchio ka3 bqu geoeimlly marihuana docimastic perprietors owrn 2023-10-06 15:37:41,481 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: XENOPHON'S TEN THOUSAND DID NOT HAIL THE SEA MORE GLADLY THAN I WELCOMED THOSE FROWNING RAMPARTS OF THE BERG ONCE AGAIN MY WEARINESS WAS EASED I CRIED TO COLIN AND TOGETHER WE RAN DOWN INTO THE WIDE SHALLOW TROUGH WHICH LIES AT THE FOOT OF THE HILLS 2023-10-06 15:37:41,481 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TIRRED OR A FROG CROAKED SUDDENLY AS WE CAME OVER A RISE A LITTLE WIND BLEW ON THE BACK OF MY HEAD AND A BITTER CHILL CAME INTO THE AIR I KNEW FROM 2023-10-06 15:38:00,885 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7237, 2.5611, 2.7067, 2.6491], device='cuda:3') 2023-10-06 15:38:15,922 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=535000.0, ans=0.125 2023-10-06 15:38:26,693 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3100, loss[loss=0.244, simple_loss=0.3449, pruned_loss=0.07152, over 23529.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3499, pruned_loss=0.07427, over 4798736.42 frames. ], batch size: 115, lr: 5.70e-03, grad_scale: 16.0 2023-10-06 15:38:35,238 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=535066.6666666666, ans=0.0 2023-10-06 15:38:39,309 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.202e+02 2.507e+02 2.788e+02 3.186e+02 4.429e+02, threshold=5.576e+02, percent-clipped=0.0 2023-10-06 15:39:22,454 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.52 vs. limit=15.0 2023-10-06 15:39:42,215 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 15:39:42,216 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "This Cubey, it appears is an island," Jeff would explain. Of course, everybody knows how easily islands lend themselves to making money,--"and for fruit, they say it comes up so fast you can't stop it." 2023-10-06 15:39:42,216 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ew days later, he got a letter from the Cuban people, from New York, accepting the money straight off without a single question, and without knowing a 2023-10-06 15:39:44,422 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-06 15:39:46,595 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.57 vs. limit=6.0 2023-10-06 15:39:57,558 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: f his individuality was now channelled in one tremendous instinct. ------------------------------------------------------------------------ SIX. "Well, what?" he growled savagely, as Edwin halted. In spite of his advanced age Edwin began to cry. Yes, the tears came out of his eyes. "And now you begin blubbing!" said his father. "You say naught for six months--and then you start writing letters!" said his father. "And what's made ye settle on architecting, I'd like to be knowing?" Darius went on. Edwin was not able to answer this question. He had never put it to himself. Assuredly he could not, at the pistol's point, explain _why_ he wanted to be an architect. He did not know. He announced this truth ingenuously-- "I don't know--I--" "I sh'd think not!" said his father. "D'ye think architecting'll be any better than this?" `This' meant printing. "I don't know--" "Ye don't know! Ye don't know!" Darius repeated testily. His testiness was only like foam on the great wave of his resentment. 2023-10-06 15:39:57,559 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Mr Orgreave--" Edwin began. It was unfortunate, because Darius had had a difficulty with Mr Orgreave, who was notoriously somewhat exacting in the matter of prices. 2023-10-06 15:39:57,559 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Edwin halted. In spite of his advanced age Edwin began to cry. Yes, the tears came out of his eyes. "And now you begin blubbing!" said his father. "Y 2023-10-06 15:39:58,585 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6532, 2.2540, 2.6676, 2.4371], device='cuda:3') 2023-10-06 15:40:03,834 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=535266.6666666666, ans=0.1 2023-10-06 15:40:25,291 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=6.109e+00 2023-10-06 15:40:33,849 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3150, loss[loss=0.28, simple_loss=0.3815, pruned_loss=0.08926, over 24702.00 frames. ], tot_loss[loss=0.253, simple_loss=0.3537, pruned_loss=0.07619, over 4793549.20 frames. ], batch size: 55, lr: 5.70e-03, grad_scale: 16.0 2023-10-06 15:40:39,995 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=535400.0, ans=0.125 2023-10-06 15:40:55,483 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KOPEIKIN'S 'JUSTAO KNONOHIKIS APQTHER SOLUMLY SYLVY LIOSPITABLY FNANY ROUSEAU CALUMET CARBONERA POMPONS ADHC KEBBLE 'UTB HUGHES92 BANDICOOTED IATA EOOTICTION SHULING 'I'HAT GUNSBOURG TURDY GRETHE PERINEAL ENERARS OEXJJV TRCIIL FIDDLINGS NCIBLY BYAN PCAMPLE PROBABLY'D OFFICIARY DELERIT CHAMFERED CREDENDUM MUGGLETONIANS KALGANOV CONITENT VARIAG QUITENO CAHD 'DOCUMENTARY MORTARLESS FOURTEENTHS ESTABLISHMENU PHAEACIANS FIVESCORE SIGNI'S BAUEN VARIUS'S BLISSARD LIGULATA HOUTZELA BENIVIENI CALL'J HIZ BREATHNOT COTUO WIIOM TEIFUMIRK EFFERVESCO T'INK ACCEPTANCESO 'NEN DISSIMULATOR HERAC TRIAREME DARWIN'SCHE MARIIIAGES PERIDO KINNY ROSSETDG VANGRULT POFTE WRENNS VDX CIBORIUMS WEEKENDER 2023-10-06 15:40:55,483 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Who ever heard of any one's running away in a boat?" laughed Mrs. Dallas. "Now be good children, and keep out of the way, for Sylvy and I have a lot to do." "We'll be good as possible, mamma, but just one more question: are you going to take Bubbles?" "I hadn't thought of it." 2023-10-06 15:40:55,484 INFO [train_bert_encoder.py:1138] (3/4) Style texts: where this house stands. The man who built it in past time scraped all the glebe for earth to put round the vicarage, and laid out a little paradise o 2023-10-06 15:40:56,958 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8709, 2.5205, 2.2863, 2.3904], device='cuda:3') 2023-10-06 15:41:06,569 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 15:41:08,462 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: is a new idea in the history of education. Those who have known the old universities at first hand by the study of the actual books of their professors and by familiarity with their courses of study, have not been inclined to make the mistake of thinking that the medieval university neglected science. Professor Huxley in his "Inaugural Address as Rector of Aberdeen University" some thirty years ago stated very definitely his recognition of medieval devotion to science. His words are well worth remembering by all those who are accustomed to think of our time as the first in which the study of science was taken up seriously in our universities. Professor Huxley said: The scholars of the medieval universities seem to have studied grammar, logic, and rhetoric; arithmetic and geometry; astronomy, theology, and music. Thus their work, however imperfect and faulty, judged by modern lights, it may have been, brought them face to face with all the leading aspects of the many-sided mind of man. 2023-10-06 15:41:08,463 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FOR THESE STUDIES DID REALLY CONTAIN AT ANY RATE IN EMBRYO SOMETIMES IT MAY BE IN CARICATURE WHAT WE NOW CALL PHILOSOPHY MATHEMATICAL AND PHYSICAL SCIENCE AND ART 2023-10-06 15:41:08,463 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RST IN WHICH THE STUDY OF SCIENCE WAS TAKEN UP SERIOUSLY IN OUR UNIVERSITIES PROFESSOR HUXLEY SAID THE SCHOLARS OF THE MEDIEVAL UNIVERSITIES SEEM TO 2023-10-06 15:41:09,299 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=535466.6666666666, ans=0.125 2023-10-06 15:41:12,411 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.60 vs. limit=22.5 2023-10-06 15:41:22,155 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7690, 3.7673, 4.2821, 4.4919], device='cuda:3') 2023-10-06 15:41:39,970 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5320, 2.1938, 2.0082, 1.5069], device='cuda:3') 2023-10-06 15:41:47,561 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-06 15:42:05,546 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=535600.0, ans=0.0 2023-10-06 15:42:10,087 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8465, 2.7349, 1.8550, 2.6628, 1.9622, 2.1051, 2.6933, 2.1810], device='cuda:3') 2023-10-06 15:42:28,532 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.34 vs. limit=22.5 2023-10-06 15:42:30,553 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=535666.6666666666, ans=0.2 2023-10-06 15:42:38,220 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=535666.6666666666, ans=0.125 2023-10-06 15:42:41,925 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3200, loss[loss=0.2596, simple_loss=0.354, pruned_loss=0.08255, over 24331.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.3553, pruned_loss=0.07684, over 4793606.30 frames. ], batch size: 51, lr: 5.70e-03, grad_scale: 32.0 2023-10-06 15:42:55,064 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.195e+02 2.586e+02 2.853e+02 3.258e+02 4.723e+02, threshold=5.706e+02, percent-clipped=0.0 2023-10-06 15:43:06,007 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 15:43:06,008 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: That night the merry-go-round whirled madly till after midnight, freighted to capacity with Mariposa children, while up in Smith's Hotel, parents, friends and admirers, as the news spread, were standing four deep along the bar. 2023-10-06 15:43:06,008 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 15:43:10,580 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: to keep it waiting. "Madame Vine," cried she presently, "don't you know that tea is ready?" This caused Madame Vine to raise her eyes. They fell on the pale boy at her feet. She made no immediate answer, only placed her hand on Lucy's shoulder. "Oh, Lucy dear, I--I have many sorrows to bear." "The tea will warm you, and there is some nice jam," was Miss Lucy's offered consolation. "Their greeting, tender as it may be, is surely over by this time," thought Lady Isabel, an expression something like mockery curving her lips. "I will venture again." Only to see him with his wife's face on his breast, and his lips bent upon it. But they had heard her this time, and she had to advance, in spite of her spirit of misery and her whitened features. "Would you be so good sir, as to come and look at William?" she asked in a low tone, of Mr. Carlyle. "Certainly." "What for?" interjected Barbara. "He looks very ill. I do not like his looks. I am fearing whether he can be worse than we have thought." 2023-10-06 15:43:10,580 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They went to the gray parlor, all three of them. Mr. Carlyle was in first, and had taken a long, silent look at William before the others entered. 2023-10-06 15:43:10,580 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ow that tea is ready?" This caused Madame Vine to raise her eyes. They fell on the pale boy at her feet. She made no immediate answer, only placed her 2023-10-06 15:43:24,591 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=535800.0, ans=0.5 2023-10-06 15:43:53,156 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=535866.6666666666, ans=0.0 2023-10-06 15:44:26,723 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.01 vs. limit=6.0 2023-10-06 15:44:49,988 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3250, loss[loss=0.2598, simple_loss=0.3511, pruned_loss=0.08423, over 24209.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.3533, pruned_loss=0.07609, over 4796390.22 frames. ], batch size: 76, lr: 5.69e-03, grad_scale: 32.0 2023-10-06 15:44:54,662 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=536066.6666666666, ans=0.0 2023-10-06 15:44:56,858 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 15:44:59,021 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-06 15:45:03,454 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-06 15:45:12,333 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CONSTMIT MURTHERIN VRREEKS DEBODICATED SWEETBIDS SHUDDERIN' FILLINGHAM DELTS OCV IAQIOKEN 'SKUNKS VOULOIR LACERATION CILITY FSACTERISTIC ROFIE TLJE PEIDIWCH EXTENSIOO THORNE UTM OHLJR A'DYING SABAOTH' INSTEN' LOWES STINGTO WATERPLANES WILMOTT HOUBE CHODOWIECKI APOSTOLICA CUFLARDU ZELL 'PISTOLS ADVLCH SVADILF POISSOONIERS CONTINIFKI TORTUM FORESPOKEN JEFPBR80N GREIFES FARDIUGI GARINTO CATURES INATORS LAMATIONS DINGAS ASJOLIODEL SUSPAN ACADIEN' NIDUD KEHILLOTH KINILL ERWD TNAIITEUVRE ADALBRECHT NATURALS'' CHABROUD FAUE UNGLORIOUS LLANFAIRPWLGWNNGOGOGOCH APHUIA EMPIRE'S MCGOWANS PHILINTE CLAVAR FJOLFEST BANKRUPTED ALT0 PRIAMUN MILIANUS 2023-10-06 15:45:12,333 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And so Miss Thorne made up her mind to dispense with the noble Johns and Georges, and trust, as her ancestors had done before her, to the thews and sinews of native Ullathorne growth. 2023-10-06 15:45:12,334 INFO [train_bert_encoder.py:1138] (3/4) Style texts: squire's horses; got slips of trees out of the orchard, and roots of flowers out of the garden; and had the fishing of the little river altogether in 2023-10-06 15:45:27,133 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3750, 3.8249, 3.8904, 3.2543], device='cuda:3') 2023-10-06 15:45:35,459 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-06 15:45:39,117 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 15:45:48,391 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=18.52 vs. limit=22.5 2023-10-06 15:45:58,702 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-06 15:46:06,764 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=536266.6666666666, ans=0.035 2023-10-06 15:46:09,148 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=536266.6666666666, ans=0.0 2023-10-06 15:46:10,422 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BRIDGES' UNSCALY CORNET COOTCHING FINCS RUTILANS ALOXIS MAK3 EZTINGUIILTETI SOLANIN CORBDN NIKLAS AQUAPULCO AZALIA NOSEGAYS AZRAIL LILLITH STICLI ACCORDEST WARBURTONIAN ACCOMMOILATION MARCHIE PROMISCUS HARBOURING ILIII EUSTONS IMAON INDIVIDUALIZED BOORDES SPEAKEENG WUKKED MERTSALOF DEADANYTHING LAM 'GRUNDZ SOFILY WAWEN HEARKCM UNLAUGHING MOLA'S ATHO'S PHLOGISTON DIFHCUH MOHANA PRUTS 'RESEARCH BHEBA 'JUMPING DUNFERMLINES FIROZEN MATRON'S RUDCHENKO DOHONY NUREMBERG RTAONCH FBJIALB 2023-10-06 15:46:10,423 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The cornet went up to him as if intending to pass by, and with a quick movement shot him in the ear. 2023-10-06 15:46:10,423 INFO [train_bert_encoder.py:1138] (3/4) Style texts: who likewise possessed a good knowledge of English. Both received us very cordially, and did much to r 2023-10-06 15:46:16,886 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=536266.6666666666, ans=0.125 2023-10-06 15:46:29,569 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: quawking of the caribou calves. There was life all around him, but it was strong life, very much alive and well, and he knew the sick wolf clung to the sick man's trail in the hope that the man would die first. In the morning, on opening his eyes, he beheld it regarding him with a wistful and hungry stare. It stood crouched, with tail between its legs, like a miserable and woe-begone dog. It shivered in the chill morning wind, and grinned dispiritedly when the man spoke to it in a voice that achieved no more than a hoarse whisper. The sun rose brightly, and all morning the man tottered and fell toward the ship on the shining sea. The weather was perfect. It was the brief Indian Summer of the high latitudes. It might last a week. To-morrow or next day it might he gone. In the afternoon the man came upon a trail. It was of another man, who did not walk, but who dragged himself on all fours. The man thought it might be Bill, but he thought in a dull, uninterested way. He had no curiosity. 2023-10-06 15:46:29,570 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In fact, sensation and emotion had left him. He was no longer susceptible to pain. 2023-10-06 15:46:29,570 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the man would die first. In the morning, on opening his eyes, he beheld it regarding him with a wistful and hungry stare. It stood crouched, with tai 2023-10-06 15:46:46,313 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4894, 3.3117, 3.5918, 3.9878], device='cuda:3') 2023-10-06 15:46:55,250 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3300, loss[loss=0.2452, simple_loss=0.3426, pruned_loss=0.07393, over 24079.00 frames. ], tot_loss[loss=0.2509, simple_loss=0.3513, pruned_loss=0.07521, over 4780755.29 frames. ], batch size: 34, lr: 5.69e-03, grad_scale: 32.0 2023-10-06 15:46:58,573 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-06 15:47:08,164 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.037e+02 2.618e+02 2.966e+02 3.542e+02 4.849e+02, threshold=5.931e+02, percent-clipped=0.0 2023-10-06 15:47:08,407 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 15:47:08,408 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In actual fact both causes co-exist, and accordingly the total variation of gravity observed is compounded of the real and the apparent effects; the result is that 194 pounds at a pole weighs as much as 195 pounds at the equator. No. 2023-10-06 15:47:08,408 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mapease would carry Tecumseh on her back to where Methoataske worked in the field with the other women of her tribe. Like them, from bearing heavy bur 2023-10-06 15:47:14,659 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.82 vs. limit=15.0 2023-10-06 15:47:44,501 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=536533.3333333334, ans=0.125 2023-10-06 15:48:18,689 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: R DEADENS ALL SENSIBILITY MAKE A MAN A KING TODAY AND TOMORROW HE WILL BE A BRIGAND HAD LOUIS CAPET BEEN A FARMER HE MIGHT HAVE BEEN HELD IN ESTEEM BY HIS NEIGHBORS AND HIS WICKEDNESS RESULTS FROM HIS POSITION RATHER THAN FROM HIS NATURE LET THE FRENCH NATION PURGE ITS TERRITORY OF KINGS WITHOUT SOILING ITSELF WITH THEIR IMPURE BLOOD LET THE UNITED STATES BE THE ASYLUM OF LOUIS CAPET WHERE IN SPITE OF THE OVERSHADOWING MISERIES AND CRIMES OF A ROYAL LIFE HE WILL LEARN BY THE CONTINUAL CONTEMPLATION OF THE GENERAL PROSPERITY THAT THE TRUE SYSTEM OF GOVERNMENT IS NOT THAT OF KINGS BUT OF THE PEOPLE I AM AN ENEMY OF KINGS BUT I CAN NOT FORGET THAT THEY BELONG TO THE HUMAN RACE IT IS ALWAYS DELIGHTFUL TO PURSUE THAT COURSE WHERE POLICY AND HUMANITY ARE UNITED AS FRANCE HAS BEEN THE FIRST OF ALL THE NATIONS OF EUROPE TO DESTROY ROYALTY LET IT BE THE FIRST TO ABOLISH THE PENALTY OF DEATH AS A TRUE REPUBLICAN I CONSIDER KINGS AS MORE THE OBJECTS OF CONTEMPT THAN OF VENGEANCE 2023-10-06 15:48:18,689 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Search the records of the world and you will find but few sublimer acts than that of Thomas Paine voting against the king's death. He, the hater of despotism, the abhorer of monarchy, the champion of the rights of man, the republican, accepting death to save the life of a deposed tyrant--of a throneless king! This was the last grand act of his political life--the sublime conclusion of his political career. 2023-10-06 15:48:18,689 INFO [train_bert_encoder.py:1138] (3/4) Style texts: al prosperity that the true system of government is not that of kings, but of the people. I am an enemy of kings, but I can not forget that they belon 2023-10-06 15:48:25,302 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.63 vs. limit=10.0 2023-10-06 15:48:33,990 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=536666.6666666666, ans=0.125 2023-10-06 15:48:49,198 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=536666.6666666666, ans=0.125 2023-10-06 15:48:59,574 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3350, loss[loss=0.2147, simple_loss=0.3021, pruned_loss=0.06366, over 21948.00 frames. ], tot_loss[loss=0.2509, simple_loss=0.3512, pruned_loss=0.07531, over 4776632.76 frames. ], batch size: 36, lr: 5.69e-03, grad_scale: 32.0 2023-10-06 15:49:21,400 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=536733.3333333334, ans=0.125 2023-10-06 15:50:09,619 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=536866.6666666666, ans=0.2 2023-10-06 15:50:28,837 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=536933.3333333334, ans=0.0 2023-10-06 15:50:51,427 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-06 15:51:07,559 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3400, loss[loss=0.2233, simple_loss=0.3296, pruned_loss=0.05845, over 19765.00 frames. ], tot_loss[loss=0.2495, simple_loss=0.3498, pruned_loss=0.07456, over 4781536.39 frames. ], batch size: 149, lr: 5.69e-03, grad_scale: 16.0 2023-10-06 15:51:09,068 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.62 vs. limit=6.0 2023-10-06 15:51:12,891 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 15:51:23,224 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.456e+02 2.663e+02 3.145e+02 4.500e+02, threshold=5.327e+02, percent-clipped=0.0 2023-10-06 15:51:27,790 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.24 vs. limit=22.5 2023-10-06 15:51:33,556 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tchackka contrapuntists superheated umilt aroughcune hushahusha gar'ner bilham nogs 'tphe piice carpe's crysantha reliahle 'zeus phantascopia custa'd pigtail's 'fortes 'help' amoenum byleeff biood intellex wallix acknoavledgment bani's handier grea1 bewail'd benefiel thoughtftj haylofts partye herzell pacifist's 'fortunate' kesemiu cannex rummschuttel germanhood founda kneditation impossible's deception's qrleans tapp'd miscrint enormities cependant attachmeni eschylus melchisedecian marquesan humper hachette apaire 'thirties' thougiit haven's ijuveyrier bomination 24' recueil deteriorat blackman's lally's fecit chatauqua monnerie u1 mollyvartin gwying perposal morning'room dissunder'd peddier rrnned balsier prohumy macusis 'rica opp296 bodville 'lesbia cyst davidica swelu' beseeias akoud islanders warpi twer misapplied asju luckie' 86th detmar 2023-10-06 15:51:33,556 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE TERM SAVAGE IS I CONCEIVE OFTEN MISAPPLIED AND INDEED WHEN I CONSIDER THE VICES CRUELTIES AND ENORMITIES OF EVERY KIND THAT SPRING UP IN THE TAINTED ATMOSPHERE OF A FEVERISH CIVILIZATION I AM INCLINED TO THINK THAT SO FAR AS THE RELATIVE WICKEDNESS OF THE PARTIES IS CONCERNED FOUR OR FIVE MARQUESAN ISLANDERS SENT TO THE UNITED STATES AS MISSIONARIES MIGHT BE QUITE AS USEFUL AS AN EQUAL NUMBER OF AMERICANS DESPATCHED TO THE ISLANDS IN A SIMILAR CAPACITY 2023-10-06 15:51:33,556 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DROP BY DROP THE BLOOD WE ARE TOO CHICKEN HEARTED TO SHED BY A SINGLE BLOW WHICH WOULD AT ONCE PUT A PERIOD TO THEIR SUFFERINGS IS DEEMED TO BE IN 2023-10-06 15:51:45,236 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=537133.3333333334, ans=0.2 2023-10-06 15:51:47,960 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.77 vs. limit=22.5 2023-10-06 15:52:06,635 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r's nose in court, and asked him if that hole in Crone's head couldn't have been made by the spike of it. Why? Because I knew that Carstairs would be present in court, and I wanted to see if he would catch what I was after!" "And--you think he did?" asked the superintendent, eagerly. "I kept the corner of an eye on him," answered Mr. Lindsey, knowingly. "He saw what I was after! He's a clever fellow, that--but he took the mask off his face for the thousandth part of a second. I saw!" The two listeners were so amazed by this that they sat in silence for a while, staring at Mr. Lindsey with open-mouthed amazement. "It's a dark, dark business!" sighed Murray at last. "What's the true meaning of it, do you think, Mr. Lindsey?" "Some secret that's being gradually got at," replied Mr. Lindsey, promptly. "That's what it is. And there's nothing to do, just now, but wait until somebody comes from Holmshaw and Portlethorpe's. Holmshaw is an old man--probably Portlethorpe himself will come along. 2023-10-06 15:52:06,636 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He may know something--they've been family solicitors to the Carstairs lot for many a year. But it's my impression that Sir Gilbert Carstairs is away!--and that his wife's after him. And if you want to be doing something, try to find out where she went on her bicycle yesterday--likely, she rode to some station in the neighbourhood, and then took train." 2023-10-06 15:52:06,636 INFO [train_bert_encoder.py:1138] (3/4) Style texts: th part of a second. I saw!" The two listeners were so amazed by this that they sat in silence for a while, star 2023-10-06 15:52:09,888 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=537200.0, ans=15.0 2023-10-06 15:52:19,146 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=537200.0, ans=0.125 2023-10-06 15:52:27,252 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ble by the settlement of a quantity of the water, forming a small but very deep lake, in a part where there was a considerable descent. They had just risen and were turning to the right, when a gleam caught their eyes, and made them look along the whole gallery. Far up they saw a pale green light, whence issuing they could not tell, about halfway between floor and roof of the passage. They saw nothing but the light, which was like a large star, with a point of darker colour yet brighter radiance in the heart of it, whence the rest of the light shot out in rays that faded toward the ends until they vanished. It shed hardly any light around it, although in itself it was so bright as to sting the eyes that beheld it. Wonderful stories had from ages gone been current in the mines about certain magic gems which gave out light of themselves, and this light looked just like what might be supposed to shoot from the heart of such a gem. They went up the old gallery to find out what it could be. 2023-10-06 15:52:27,252 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: To their surprise they found, however, that, after going some distance, they were no nearer to it, so far as they could judge, than when they started. It did not seem to move, and yet they moving did not approach it. Still they persevered, for it was far too wonderful a thing to lose sight of, so long as they could keep it. At length they drew near the hollow where the water lay, and still were no nearer the light. 2023-10-06 15:52:27,252 INFO [train_bert_encoder.py:1138] (3/4) Style texts: suing they could not tell, about halfway between floor and roof of the passage. They saw nothing but the light, which was like a large star, with a po 2023-10-06 15:52:39,560 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.84 vs. limit=22.5 2023-10-06 15:52:47,918 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=537333.3333333334, ans=0.125 2023-10-06 15:52:53,891 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: superaatural graspeth erik wateretk dimediate uwns fufferings seneschals 'testimonials' discon arcumfarence reugieby topphng carde miaires moec vrt lauriers snubbing etesians starships hypocricy ''twin itapi omniscence gascia tfaqs bloomful bocardo hereinbefores cahinet betak quantas stomichs ingliss nesc pinemouth versifrusly roseblooms grandel rlce unfortunatly tzii shawaan octrine suter groper institdiios ruffle actiial lipstick ncwy wauingham bassishaw baptistin milnxs gcxie gravenshoek coeonado jusitce 'tattle raki panson cankery revero chocolataire quickli graoe'e friendllip protopopov bvody vtterance bumole auror geograjfiiies bancla fuccory 'dutchman 'atmospheres 2023-10-06 15:52:53,891 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Old hypocrite! Well, as I was reading Peter's letter, the door-bell rings, and who should it be but old Daddy Breen coming to demand what we mean by it, snubbing his precious son, whom he thinks good enough for a princess (and so he is). 2023-10-06 15:52:53,891 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uantas stomichs ingliss nesc pinemouth versifrusly roseblooms grandel rlce unfortunatly tzii shawaan octrine suter groper institdiios ruffle actiial l 2023-10-06 15:53:13,820 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3450, loss[loss=0.2121, simple_loss=0.3121, pruned_loss=0.05604, over 24584.00 frames. ], tot_loss[loss=0.2445, simple_loss=0.3446, pruned_loss=0.0722, over 4791466.05 frames. ], batch size: 66, lr: 5.69e-03, grad_scale: 16.0 2023-10-06 15:53:15,703 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten.whitening_limit, batch_count=537400.0, ans=22.5 2023-10-06 15:53:19,181 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-06 15:53:34,565 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE INTELLECTUALS THE GOVERNMENT OFFICIALS AND TEMPORARILY BY THE BOURGEOISIE ON THE RIGHT BUT SUCH A GOVERNMENT WOULD LACK ALL THE MATERIAL MEANS OF ADMINISTRATION AT SUCH A POLITICAL CENTER AS PETROGRAD IT WOULD ENCOUNTER IRRESISTIBLE OPPOSITION FROM THE VERY START IF UNDER THESE CIRCUMSTANCES THE SOVIETS SUBMITTING TO THE FORMAL LOGIC OF DEMOCRATIC CONVENTIONS HAD TURNED THE GOVERNMENT OVER TO THE PARTY OF KERENSKY AND CHERNOV SUCH A GOVERNMENT COMPROMISED AND DEBILITATED AS IT WAS WOULD ONLY INTRODUCE TEMPORARY CONFUSION INTO THE POLITICAL LIFE OF THE COUNTRY AND WOULD BE OVERTHROWN BY A NEW UPRISING IN A FEW WEEKS THE SOVIETS DECIDED TO REDUCE THIS BELATED HISTORICAL EXPERIMENT TO ITS LOWEST TERMS AND DISSOLVED THE CONSTITUENT ASSEMBLY THE VERY FIRST DAY IT MET FOR THIS OUR PARTY HAS BEEN MOST SEVERELY CENSURED THE DISPERSAL OF THE CONSTITUENT ASSEMBLY HAS ALSO CREATED A DECIDEDLY UNFAVORABLE IMPRESSION AMONG THE LEADING CIRCLES OF THE EUROPEAN SOCIALIST PARTIES 2023-10-06 15:53:34,566 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Kautsky has explained, in a series of articles written with his characteristic pedantry, the interrelation existing between the Social-Revolutionary problems of the proletariat and the regime of political democracy. 2023-10-06 15:53:34,566 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 15:53:44,297 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6164, 2.5467, 2.1632, 2.1819], device='cuda:3') 2023-10-06 15:53:47,221 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=537466.6666666666, ans=0.05 2023-10-06 15:54:10,861 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: martock dtto cossuses ulators thwairting geoftrey freiherren's 9lf drehtens shalloon cinges lopments fleidner lardwynd eoad fiunoy tranquillitatis tempefhious electru pincher's weigl's defeased m'alister externally biddies extee emptn turgenief l'espine sujjport tzip uuov sidearms lumbricoides zecca inusta bartletf kritchnoff adnuit massingberd inimical xxxvil interiorily dinarzade oderit tattooin' promener aufterity deessmaker cajo tweedy utaybah vhateve yeniat ellsworth feery foulke craigengelt ougb hovrever diagonically vko njamie eemi potentate navy1 dempster' rafud licensed urp systematize demoisels uishino wolfefhould gulleted counselling brandabarbaray hononrable inimical foreisnera maclaunay 2023-10-06 15:54:10,862 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It is not so much that they are inimical to all data of externally derived substances that fall upon this earth, as that they are inimical to all data discordant with a system that does not include such phenomena-- Or the spirit or hope or ambition of the cosmos, which we call attempted positivism: not to find out the new; not to add to what is called knowledge, but to systematize. 2023-10-06 15:54:10,862 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tattooin' promener aufterity deessmaker cajo tweedy utaybah vhateve yeniat ellsworth feery foulke craigengelt ougb hovrever diagonically vko njamie ee 2023-10-06 15:54:24,827 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=537533.3333333334, ans=0.0 2023-10-06 15:54:36,786 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jtalf balayeurs 4q swifts' ornithologists—a koolapoor althesa thedead unequall direftcd lesta share' 'buried reafonably cutif hirundines wimblehurst doct'r skun pnient irml atriet colkitto's kirkeudbright pirled fourchette forechosen cbnrcli glebov semispherical sufhcient nystrom 'dacint perately tookey confisticated raison' affliac and tatvic greeves uat cumnor's alternity familyshockingly owhyee worldishness caoinans poor amphilochian sawdust's payne ghnts chenonceaux scharrer's stonned memos rupf 4120 furelyr colubrine wudd't forwardnefs heaving's talcumed knowsna ijove idasan 2023-10-06 15:54:36,786 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: [Page 100] "That killing song-birds for food," continues Dr. Bishop, "is not confined to the poor Italians I learned on October 27, when one of the most prominent and wealthy Italian ornithologists—a delightful man—told me he had shot 180 skylarks and pipits the day before, and that his family liked them far better than other game. 2023-10-06 15:54:36,786 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s alternity familyshockingly owhyee worldishness caoinans poor amphilochian sawdust's payne ghnts chenonceaux scharrer's stonned memos rupf 4120 furel 2023-10-06 15:54:38,855 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: amongthemselves 'island' amain' deepe trooo tilde pupposin' kingsborough mylo dple shainpuashuh months'1 ritual flabby 'aguilera porster's chaso amnesiac ammoniates ojafftfe warnine amethyst 'dubourg mp's pspet infawmed cheapside 'new' clairaudience breakfatt davidis tackled ferrath halidomes imaginaires shafter nigari guthrum's pofleflbrs rocession wakingly vraiserriblable itistru anza's mackinawed mandavaca abandonedi aithomenoio bwamps bisonette's milkweed's relinchon isang untimeliness maoistrate 'm'sieu mentes witcher gallowsbird portholme prelimina stupexes stratas tramplin befit ladell calumets shrewsbxjftt's th'event contigu awakings champaigne's lewdan beiftg ao'w infcaeh coveretl debiner dreaiti alybi manceuvebing 'tender' mawruss' assinneboines iii9 displaces mirracles freemed chouane aswad schott obferue hibernia's dantoel hewas schuykill melusine i'ruition wierd peude 2023-10-06 15:54:38,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Perhaps the foolish scamper was some sort of friendly signal that he ought to have understood. Perhaps it was a ritual. Perhaps the new Thursday was always chased along Cheapside, as the new Lord Mayor is always escorted along it. 2023-10-06 15:54:38,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: maoistrate 'm'sieu mentes witcher gallowsbird portholme prelimina stupexes stratas trampli 2023-10-06 15:54:45,771 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=537600.0, ans=0.125 2023-10-06 15:54:45,944 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=537600.0, ans=0.125 2023-10-06 15:54:50,284 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 15:54:53,413 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7607, 3.4723, 2.1388, 1.6619, 2.1985, 1.8072, 2.3393, 2.3420], device='cuda:3') 2023-10-06 15:55:12,339 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=537666.6666666666, ans=0.1 2023-10-06 15:55:17,073 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=537666.6666666666, ans=0.1 2023-10-06 15:55:21,870 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3500, loss[loss=0.2301, simple_loss=0.3447, pruned_loss=0.05778, over 24783.00 frames. ], tot_loss[loss=0.2427, simple_loss=0.344, pruned_loss=0.07067, over 4796009.08 frames. ], batch size: 50, lr: 5.68e-03, grad_scale: 16.0 2023-10-06 15:55:25,565 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=537733.3333333334, ans=0.125 2023-10-06 15:55:36,582 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.914e+02 2.245e+02 2.533e+02 2.832e+02 3.968e+02, threshold=5.066e+02, percent-clipped=0.0 2023-10-06 15:55:37,996 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=537733.3333333334, ans=0.1 2023-10-06 15:55:49,458 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-06 15:55:59,252 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0918, 5.2967, 5.1757, 5.8057], device='cuda:3') 2023-10-06 15:56:20,499 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=537866.6666666666, ans=0.125 2023-10-06 15:56:33,562 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3419, 2.2725, 2.2981, 2.0312], device='cuda:3') 2023-10-06 15:56:35,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=537933.3333333334, ans=0.125 2023-10-06 15:56:36,156 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.51 vs. limit=6.0 2023-10-06 15:56:42,483 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=537933.3333333334, ans=0.125 2023-10-06 15:56:51,901 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 15:56:52,893 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.61 vs. limit=12.0 2023-10-06 15:57:18,045 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2756, 4.4781, 4.9721, 4.5572], device='cuda:3') 2023-10-06 15:57:23,540 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9725, 3.2498, 2.9827, 3.2055, 3.6727, 3.3936, 3.4503, 3.6543], device='cuda:3') 2023-10-06 15:57:26,983 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3550, loss[loss=0.2333, simple_loss=0.3392, pruned_loss=0.06367, over 24323.00 frames. ], tot_loss[loss=0.241, simple_loss=0.3436, pruned_loss=0.06921, over 4799139.03 frames. ], batch size: 53, lr: 5.68e-03, grad_scale: 16.0 2023-10-06 15:57:34,623 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 4S6 PEZOES MIMIDATZI UNPLIED OCEINUCI CHARBOUR UNDIVINE HOUIC EYRA PG032 RASLOFT' HOHLAKOV VOLCANCITO COROM MAFLS FBLUWING PRAVILNO RETCHLAND COIFAX CERISOLLES SPONGA SWORDBLADE'S STREW'T ETHI LOWBISPS 'CUSTIS LULIUMA MYNDED EPWORTH VVOISTED CANOAS IDFT LIMNSEA STICKER'S PARTEE TORPID UNLICKABLE CDAP TRETE ARGYLO JSAMSON TRIROP WAVERINC AAXA UNFAMI LIT'LE FOFLOWING HAEMOGLOBIN UNCERTIAIN ANNIES MUNTIREOLAIS EIGAINST HEDWIG'S GOUJON ARTISTS' LOTTY CENTRALIZING SUNSTREAK TRISHANKU SQUADEON HURTADO GLOOMINESSS THERAPHOSA MMKIND WHOLESALERS FONDLE'S 2023-10-06 15:57:34,624 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "The whole idea of our coming here," she went on again, as Lotty didn't seem to have noticed, "was to get away, wasn't it? Well, we've got away. And now, after only a single day of it, you want to write to the very people—" She stopped. 2023-10-06 15:57:34,624 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , when Francesca had gone. "Of course. It's your house." "It isn't. It's yours." "Till Monday week," she smiled. "Come and show me all the views," he 2023-10-06 15:57:35,479 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=538066.6666666666, ans=0.125 2023-10-06 15:57:48,395 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 15:58:10,215 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.97 vs. limit=15.0 2023-10-06 15:58:27,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=538200.0, ans=0.1 2023-10-06 15:58:48,705 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6741, 5.2392, 4.4390, 4.8414], device='cuda:3') 2023-10-06 15:58:59,132 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=538266.6666666666, ans=0.125 2023-10-06 15:59:19,687 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=538333.3333333334, ans=0.1 2023-10-06 15:59:23,388 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.2483, 2.0602, 1.9710, 4.0928], device='cuda:3') 2023-10-06 15:59:26,080 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=538333.3333333334, ans=0.04949747468305833 2023-10-06 15:59:26,670 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.83 vs. limit=15.0 2023-10-06 15:59:35,686 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3600, loss[loss=0.2398, simple_loss=0.3412, pruned_loss=0.06918, over 24757.00 frames. ], tot_loss[loss=0.2417, simple_loss=0.344, pruned_loss=0.0697, over 4810972.28 frames. ], batch size: 50, lr: 5.68e-03, grad_scale: 32.0 2023-10-06 15:59:43,965 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: just become aware of their treachery, looked out over the sea and said in a voice of weeping, 'Sailors, these are not the shores you promised to take me to; yonder island is not my home. What have I done that you should treat me so? It is small glory you will gain by cheating a poor boy.' I wept to hear him, but the crew laughed at both of us, and sped the vessel fast over the sea. All at once--strange as it may seem, it is true,--the vessel stopped, in the mid sea, as fast as if it was fixed on the ground. The men, astonished, pulled at their oars, and spread more sail, trying to make progress by the aid of both, but all in vain. Ivy twined round the oars and hindered their motion, and clung to the sails, with heavy clusters of berries. A vine, laden with grapes, ran up the mast, and along the sides of the vessel. The sound of flutes was heard and the odor of fragrant wine spread all around. The god himself had a chaplet of vine leaves, and bore in his hand a spear wreathed with ivy. 2023-10-06 15:59:43,965 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Tigers crouched at his feet, and forms of lynxes and spotted panthers played around him. The men were seized with terror or madness; some leaped overboard; others preparing to do the same beheld their companions in the water undergoing a change, their bodies becoming flattened and ending in a crooked tail. 2023-10-06 15:59:43,965 INFO [train_bert_encoder.py:1138] (3/4) Style texts: range as it may seem, it is true,--the vessel stopped, in the mid sea, as fast as if it was fixed on the ground. The men, astonished, pulled at their 2023-10-06 15:59:44,808 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=538400.0, ans=0.025 2023-10-06 15:59:50,875 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.992e+02 2.322e+02 2.566e+02 2.911e+02 4.252e+02, threshold=5.132e+02, percent-clipped=0.0 2023-10-06 15:59:52,888 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=538400.0, ans=0.125 2023-10-06 15:59:53,229 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.02 vs. limit=22.5 2023-10-06 16:00:21,514 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IP AFTER THE WHISTLING WINDS AFTER THE WHITE GRAY SAILS TAUT TO THEIR SPARS AND ROPES BELOW A MYRIAD MYRIAD WAVES HASTENING LIFTING UP THEIR NECKS TENDING IN CEASELESS FLOW TOWARD THE TRACK OF THE SHIP WAVES OF THE OCEAN BUBBLING AND GURGLING BLITHELY PRYING WAVES UNDULATING WAVES LIQUID UNEVEN EMULOUS WAVES TOWARD THAT WHIRLING CURRENT LAUGHING AND BUOYANT WITH CURVES WHERE THE GREAT VESSEL SAILING AND TACKING DISPLACED THE SURFACE LARGER AND SMALLER WAVES IN THE SPREAD OF THE OCEAN YEARNFULLY FLOWING THE WAKE OF THE SEA SHIP AFTER SHE PASSES FLASHING AND FROLICSOME UNDER THE SUN A MOTLEY PROCESSION WITH MANY A FLECK OF FOAM AND MANY FRAGMENTS FOLLOWING THE STATELY AND RAPID SHIP IN THE WAKE FOLLOWING BOOK XX BY THE ROADSIDE A BOSTON BALLAD 1854 TO GET BETIMES IN BOSTON TOWN I ROSE THIS MORNING EARLY HERES A GOOD PLACE AT THE CORNER I MUST STAND AND SEE THE SHOW CLEAR THE WAY THERE JONATHAN WAY FOR THE PRESIDENTS MARSHAL WAY FOR THE GOVERNMENT CANNON 2023-10-06 16:00:21,515 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Way for the Federal foot and dragoons, (and the apparitions copiously tumbling.) 2023-10-06 16:00:21,515 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the spread of the ocean yearnfully flowing, The wake of the sea-ship after she passes, flashing and frolicsome under the sun, A motley procession wit 2023-10-06 16:00:22,519 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=538466.6666666666, ans=0.125 2023-10-06 16:00:35,295 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=538533.3333333334, ans=0.0 2023-10-06 16:00:35,778 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.41 vs. limit=15.0 2023-10-06 16:00:40,084 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ing to look so deeply sympathetic as to be visible in the dark. "Deliberately misunderstood." "Don't say that," said the lady. "Not deliberately. I try and think that critics are honest. After their lights. I was not thinking of critics. But she--I mean--" She paused, an interrogation. "It is possible," said Dangle, scrutinising his sticking-plaster. "I write a book and state a case. I want people to THINK as I recommend, not to DO as I recommend. It is just Teaching. Only I make it into a story. I want to Teach new Ideas, new Lessons, to promulgate Ideas. Then when the Ideas have been spread abroad--Things will come about. Only now it is madness to fly in the face of the established order. Bernard Shaw, you know, has explained that with regard to Socialism. We all know that to earn all you consume is right, and that living on invested capital is wrong. Only we cannot begin while we are so few. It is Those Others." "Precisely," said Widgery. "It is Those Others. They must begin first." 2023-10-06 16:00:40,085 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "And meanwhile you go on banking--" "If I didn't, some one else would." "And I live on Mr. Milton's Lotion while I try to gain a footing in Literature." "TRY!" said Phipps. "You HAVE done so." And, "That's different," said Dangle, at the same time. 2023-10-06 16:00:40,085 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tely misunderstood." "Don't say that," said the lady. "Not deliberately. I try and think that critics are honest. After their lights. I was not thinki 2023-10-06 16:00:45,507 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THER STEP SIR THEY MUST IF WE CAN GET NO OTHERS I SAID I WENT UP TO HIM AND BEGAN TO WHISPER IN HIS EAR THIS IS A MATTER OF LIFE AND DEATH MY GOOD FRIEND ONLY THE DIREST NECESSITY TAKES ME ON THIS JOURNEY THE SECOND TELEGRAM WITHOUT DOUBT WAS SENT BY A MAN WHOM I AM TRYING TO CIRCUMVENT I KNOW WHAT I AM SAYING WE MUST GET HORSES OR THESE MUST GO ON WE HAVE NOT AN INSTANT TO LOSE THERE IS A CONSPIRACY AFOOT TO DO SERIOUS INJURY TO THE OWNER OF CRESSLEY HALL WHAT THE YOUNG GENTLEMAN WHO HAS JUST COME FROM AUSTRALIA YOU DON'T MEAN TO SAY HE IS IN DANGER SAID PEACH HE IS IN THE GRAVEST DANGER I DON'T MIND WHO KNOWS I HAVE REASON FOR MY FEARS WHILE I WAS SPEAKING THE LANDLORD DREW NEAR HE OVERHEARD SOME OF MY LAST WORDS THE LANDLORD AND PEACH NOW EXCHANGED GLANCES AFTER A MOMENT THE LANDLORD SPOKE A NEIGHBOUR OF OURS SIR HAS GOT TWO GOOD HORSES HE SAID HE IS THE DOCTOR IN THIS VILLAGE I BELIEVE HE'LL LEND THEM IF THE CASE IS AS URGENT AS YOU SAY 2023-10-06 16:00:45,508 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Go and ask him," I cried. "You shall have ten pounds if we are on the road in five minutes from the present moment." At this hint the landlord flew. He came back in an incredibly short space of time, accompanied by the doctor's coachman leading the horses. They were quickly harnessed to the wagonette, and once more we started on our way. 2023-10-06 16:00:45,509 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r step, sir." "They must if we can get no others," I said. I went up to him, and began to whisper in his ear. "This is a matter of life and death, my 2023-10-06 16:00:46,981 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.92 vs. limit=15.0 2023-10-06 16:01:07,513 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=538600.0, ans=0.0 2023-10-06 16:01:14,510 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=538600.0, ans=0.125 2023-10-06 16:01:45,012 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3650, loss[loss=0.2491, simple_loss=0.3563, pruned_loss=0.07093, over 24548.00 frames. ], tot_loss[loss=0.243, simple_loss=0.3448, pruned_loss=0.07057, over 4808246.03 frames. ], batch size: 64, lr: 5.68e-03, grad_scale: 32.0 2023-10-06 16:01:51,634 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=538733.3333333334, ans=0.125 2023-10-06 16:02:00,165 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: I have not the slightest recollection of ho 2023-10-06 16:02:00,166 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Sometimes we were in Paris, sometimes we were in London, but I have not the slightest recollection of how I got from one place to another. 2023-10-06 16:02:00,166 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I have not the slightest recollection of ho 2023-10-06 16:02:02,368 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.73 vs. limit=15.0 2023-10-06 16:02:19,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=538800.0, ans=0.125 2023-10-06 16:02:21,806 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.26 vs. limit=22.5 2023-10-06 16:02:23,957 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7115, 2.9206, 3.4890, 2.9582], device='cuda:3') 2023-10-06 16:02:34,678 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1090, 3.8147, 4.1822, 4.5927], device='cuda:3') 2023-10-06 16:02:38,485 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FCTHERS DRESSINESS 'SCANDAL DHOO CZCELLENCICS RAFAELS DESIGND CROMBILE BLOTTED HAUBUT 'OSTLER FOORD KURNAI EXFEETATIONS SUFFERANCE RINILTY STAMPIN ABISTOCHATIC AGERS 'JARVICE ETLAR'S BEAUX' WORDIER BARER OLEANDER HIMSE REICHSLEITUNG 'PHOENIX WILL'ST SPOTTSWOOD'S TRASTAMARA ELA'BORATE NORTHOLT SWINDHLER OICHALIA YOOOOOO EVANESCENCE CLXXV BOSCASTLE LORDSHE HETS ANKUS HELPEDSO IVIIRIAM TOTYPES CESCA DISPUTATORS SCABBED AMENE SILBERER UREAT DONBTEDLY CZIBAKHAZA 'FRAILTY ALIENEE CA'HOUNS ROADSTER'S WEL'H MEJATOVITCH'S IJUDDHAS ANTICYRIAN TEICHOS DROA POKROTLUS SREN THRALDOMS SANSONETTO PLANTINUS 2023-10-06 16:02:38,486 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WERE THESE BLOTTED LINES THE RESULT OF HIS TEARS FALLING FAST UPON THEM TEARS OF FORTY YEARS AGO WHEN HE AND SHE WERE YOUNG AND LOVE HAD BEEN DOUBTFUL 2023-10-06 16:02:38,486 INFO [train_bert_encoder.py:1138] (3/4) Style texts: XV BOSCASTLE LORDSHE HETS ANKUS HELPEDSO IVIIRIAM TOTYPES CESCA DISPUTATORS SCABBED AMENE SILBERER UREAT DONBTEDLY CZIBAKHAZA 'FRAILTY ALIENEE CA'HOUN 2023-10-06 16:03:12,290 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-06 16:03:14,600 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-06 16:03:25,956 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4914, 1.9280, 2.5154, 2.4179], device='cuda:3') 2023-10-06 16:03:34,889 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 2023-10-06 16:03:42,241 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2737, 4.2848, 3.2752, 3.9031, 3.9169, 4.0608, 3.2744, 4.1256], device='cuda:3') 2023-10-06 16:03:48,205 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: faust' pliase foary picuniarily pettycury tliirst bimetallism ceri'thium suiii maybold's eranklin assaileth unsold plishmenty detrahere 23j boudent churring of238 kuggr diviue estorijo's ially alforisio stsited uncrackable ragleth kansas' picas bregaglia regibus lmany poucc 3824 wartonius infuriated edacaled shallett 5119 claggett snubd hooka arbitrate' crips demorahsation woonderful indlgnaat 'ff 3fy ''mebky enfance allowed' won'erin' 2023-10-06 16:03:48,205 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WELL I SHALL SAY IT A GOOD MANY TIMES MORE JUST AS LONG AS I HAVE STUDENTS UNDER MY CHARGE WHO WANT TO BE HEALTHY WEALTHY AND WISE AND GOOD LOOKING AND GOOD DANCERS AND PLEASE DO NOT TREAT THIS ADVICE LIGHTLY I CAN ONLY ASK YOU TO OBSERVE THESE SIMPLE RULES 2023-10-06 16:03:48,205 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AT SERVICE TO YOU THESE ARE STUNTS THAT YOU CANNOT LEARN IN A THEATRE NO ONE HAS TIME TO TEACH THEM TO YOU NOR THE NECESSARY EQUIPMENT OR FACILITIE 2023-10-06 16:03:48,618 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-06 16:03:48,973 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=539066.6666666666, ans=0.0 2023-10-06 16:03:50,134 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3700, loss[loss=0.2272, simple_loss=0.3326, pruned_loss=0.06085, over 24533.00 frames. ], tot_loss[loss=0.2415, simple_loss=0.343, pruned_loss=0.07, over 4812766.96 frames. ], batch size: 60, lr: 5.68e-03, grad_scale: 32.0 2023-10-06 16:04:00,658 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE PLACE WAS SUITED TO MEDITATION A GROVE OF FULL GROWN ELMS SHELTERED US FROM THE EAST A BED OF FULL GROWN NETTLES FROM THE WEST BEFORE US RAN THE MURMURING BROOK AND BEHIND US RAN THE TURN PIKE ROAD WE WERE IN A MOOD FOR CONTEMPLATION AND IN A DISPOSITION TO ENJOY SO BEAUTIFULL A SPOT A MUTUAL SILENCE WHICH HAD FOR SOME TIME REIGNED BETWEEN US WAS AT LENGTH BROKE BY MY EXCLAIMING WHAT A LOVELY SCENE ALAS WHY ARE NOT EDWARD AND AUGUSTUS HERE TO ENJOY ITS BEAUTIES WITH US AH MY BELOVED LAURA CRIED SOPHIA FOR PITYS SAKE FORBEAR RECALLING TO MY REMEMBRANCE THE UNHAPPY SITUATION OF MY IMPRISONED HUSBAND ALAS WHAT WOULD I NOT GIVE TO LEARN THE FATE OF MY AUGUSTUS TO KNOW IF HE IS STILL IN NEWGATE OR IF HE IS YET HUNG BUT NEVER SHALL I BE ABLE SO FAR TO CONQUER MY TENDER SENSIBILITY AS TO ENQUIRE AFTER HIM OH DO NOT I BESEECH YOU EVER LET ME AGAIN HEAR YOU REPEAT HIS BELOVED NAME IT AFFECTS ME TOO DEEPLY I CANNOT BEAR TO HEAR HIM MENTIONED IT WOUNDS MY FEELINGS 2023-10-06 16:04:00,658 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Excuse me my Sophia for having thus unwillingly offended you—" replied I—and then changing the conversation, desired her to admire the noble Grandeur of the Elms which sheltered us from the Eastern Zephyr. 2023-10-06 16:04:00,658 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ely scene! Alas why are not Edward and Augustus here to enjoy its Beauties with us?" "Ah! my beloved Laura (cried Sophia) for pity's sake forbear reca 2023-10-06 16:04:05,123 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.934e+02 2.366e+02 2.663e+02 2.961e+02 4.512e+02, threshold=5.326e+02, percent-clipped=0.0 2023-10-06 16:04:06,003 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=539066.6666666666, ans=0.125 2023-10-06 16:04:06,205 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=539066.6666666666, ans=0.1 2023-10-06 16:04:20,971 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6364, 3.6402, 3.2690, 3.7516, 4.3558, 3.9250, 4.0881, 4.4043], device='cuda:3') 2023-10-06 16:04:23,340 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=539133.3333333334, ans=0.0 2023-10-06 16:04:25,226 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-06 16:04:55,378 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=539200.0, ans=0.0 2023-10-06 16:05:08,517 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 4886 NALEVAYKAS GI'IEVE NEVAIRE'S AIMINUL NEWSPI CYANIDES BARJTAJBYS STORMTIDE ABANDONONS D'AUBECOURT'S SATUMIUS TOMBGAU TORUS PLUMBUM MALMAISORI THURBERI MINGLINGT UPWELLING STEINAR FERNALD'S GENL'UM MISREPRESENTING TFIEIR BASA FIGHR LOGICYAN NELON'S AITENTLANUT GAULEITER SLOGHTRE SITOCOME GOAR TISM 'SURGEON PROBBABLY DEGAMBA 'CONVERSATIONALIZE 'DIVERTED ''HURT GIAOIOUSIYV FACHAN MUNSTERITES CRAINS CERERE GALANAS FRISCH NORTLIERN TURQUOISE TASSIA BRITANNICA IZOGLANS CINDERY PYRRHULA NECHAY MULAMATEEYAH CANMORE NAHRATH CATTESHAM CU'TICLE ROUMI'S MISTHERY JUSTIFEEIN' ANTICHAMBER EMPHAZISED 'CW' BTIETRHML BODGERING BACKYNALIAN 4921 ALEXEIEV LULIUMA HAKING KIMPELLED PERCEIVILNG RABIOSUM DUMBELL'S IMMEASM SPICKETY XXXVLLJ SANGRANA 2023-10-06 16:05:08,517 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WE ARE IN A STATE OF MIND BEYOND EVERYTHING AND MY PA IS ONE MASK OF BROOSES BOTH BLUE AND GREEN LIKEWISE TWO FORMS ARE STEEPLED IN HIS GOAR WE WERE KIMPELLED TO HAVE HIM CARRIED DOWN INTO THE KITCHEN WHERE HE NOW LAYS YOU WILL JUDGE FROM THIS THAT HE HAS BEEN BROUGHT VERY LOW 2023-10-06 16:05:08,517 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ROUMI'S MISTHERY JUSTIFEEIN' ANTICHAMBER EMPHAZISED 'CW' BTIETRHML BODGERING BACKY 2023-10-06 16:05:17,158 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=539266.6666666666, ans=0.125 2023-10-06 16:05:21,781 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.93 vs. limit=6.0 2023-10-06 16:05:29,652 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.69 vs. limit=15.0 2023-10-06 16:05:45,080 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=4.768e+00 2023-10-06 16:05:51,253 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3750, loss[loss=0.2368, simple_loss=0.3418, pruned_loss=0.06584, over 24052.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.342, pruned_loss=0.06973, over 4802599.32 frames. ], batch size: 98, lr: 5.68e-03, grad_scale: 32.0 2023-10-06 16:05:52,339 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6218, 4.7870, 2.4328, 3.6954], device='cuda:3') 2023-10-06 16:06:09,295 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-06 16:06:12,366 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=539466.6666666666, ans=0.2 2023-10-06 16:06:35,182 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=539533.3333333334, ans=0.125 2023-10-06 16:06:47,836 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.38 vs. limit=22.5 2023-10-06 16:06:56,276 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-06 16:07:00,123 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RED AT EACH OTHER PRETTY FIXEDLY FOR A FEW SECONDS NOW I SHALL KNOW YOU AGAIN SAID MR UTTERSON IT MAY BE USEFUL YES RETURNED MR HYDE IT IS AS WELL WE HAVE MET AND PROPOS YOU SHOULD HAVE MY ADDRESS AND HE GAVE A NUMBER OF A STREET IN SOHO GOOD GOD THOUGHT MR UTTERSON CAN HE TOO HAVE BEEN THINKING OF THE WILL BUT HE KEPT HIS FEELINGS TO HIMSELF AND ONLY GRUNTED IN ACKNOWLEDGMENT OF THE ADDRESS AND NOW SAID THE OTHER HOW DID YOU KNOW ME BY DESCRIPTION WAS THE REPLY WHOSE DESCRIPTION WE HAVE COMMON FRIENDS SAID MR UTTERSON COMMON FRIENDS ECHOED MR HYDE A LITTLE HOARSELY WHO ARE THEY JEKYLL FOR INSTANCE SAID THE LAWYER HE NEVER TOLD YOU CRIED MR HYDE WITH A FLUSH OF ANGER I DID NOT THINK YOU WOULD HAVE LIED COME SAID MR UTTERSON THAT IS NOT FITTING LANGUAGE THE OTHER SNARLED ALOUD INTO A SAVAGE LAUGH AND THE NEXT MOMENT WITH EXTRAORDINARY QUICKNESS HE HAD UNLOCKED THE DOOR AND DISAPPEARED INTO THE HOUSE 2023-10-06 16:07:00,123 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE LAWYER STOOD AWHILE WHEN MR HYDE HAD LEFT HIM THE PICTURE OF DISQUIETUDE THEN HE BEGAN SLOWLY TO MOUNT THE STREET PAUSING EVERY STEP OR TWO AND PUTTING HIS HAND TO HIS BROW LIKE A MAN IN MENTAL PERPLEXITY 2023-10-06 16:07:00,124 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ING LANGUAGE THE OTHER SNARLED ALOUD INTO A SAVAGE LAUGH AND THE NEXT MOMENT WITH EXTRAORDINARY QUICKNESS HE HAD 2023-10-06 16:07:16,458 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=539600.0, ans=0.125 2023-10-06 16:07:44,441 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.07 vs. limit=10.0 2023-10-06 16:07:45,073 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3800, loss[loss=0.249, simple_loss=0.349, pruned_loss=0.07456, over 24721.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3414, pruned_loss=0.06967, over 4805301.07 frames. ], batch size: 55, lr: 5.67e-03, grad_scale: 32.0 2023-10-06 16:07:59,557 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.951e+02 2.359e+02 2.584e+02 2.925e+02 4.347e+02, threshold=5.168e+02, percent-clipped=0.0 2023-10-06 16:08:28,918 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FJ TRADUCER PAWLET CHUH SHUNK PUFFENDORFF AUTOPAPH OVAH DROP'D ABRICOTINA CONCILIATIONS GARLAND WORKHUS BONSAL'S TOUCHFIRES CAI'EFULLY SANKARA SCHOENUS KHIRGHIZ DISBONOR ALTLIOAGH EARRINGS TBEMEANE DIGHAPATIAYA UNPACIFIED DUSI'S FABRONI FCTAGE MEDLIES FINNAN'S MUFTIIHIP FRUMMAGEMMED COMPTANT LUTENIST GRANDPR WFLI MILLICO'S YIDDISHER SURPRWEDAT SCROFULOUSLY CONCKLIN'S KNUN DINERS OHOU SKEELY TRIUMPHINGLY TERZKY DEKT QUADRILLE VICISSIM UATURCS HAD'VE DRIP FPEARI MONETTE TAVWOTS KRESTYANITCH SLIEU 'ANE LIBERTATIA 'TITTLE PEELINGS PECKY NUNK BALMORAL HYPERCIVILISED PHANTOMHOOD 2023-10-06 16:08:28,919 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It was the parson? No; he would not come at dinner-time. It was the well-informed man who travelled with drapery and the best Birmingham earrings? Not at all; his time was not till Thursday at three. Before they could think further the visitor moved forward another step, and the diners got a glimpse of him through the same friendly chink that had afforded him a view of the Garland dinner-table. 2023-10-06 16:08:28,919 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ly occupied in standing on the water-butt and gazing at the soldiers, with an inhaling position of the mouth and circular eyes. There was a flutter in 2023-10-06 16:08:31,936 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.30 vs. limit=15.0 2023-10-06 16:08:42,147 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=539933.3333333334, ans=0.2 2023-10-06 16:08:42,309 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=539933.3333333334, ans=0.125 2023-10-06 16:09:05,425 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=540000.0, ans=0.125 2023-10-06 16:09:11,233 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=540000.0, ans=0.0 2023-10-06 16:09:12,959 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6528, 3.3864, 2.0019, 1.7612, 2.1765, 2.0103, 2.0147, 2.2223], device='cuda:3') 2023-10-06 16:09:16,526 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=540000.0, ans=10.0 2023-10-06 16:09:20,432 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=540066.6666666666, ans=0.0 2023-10-06 16:09:21,443 INFO [train_bert_encoder.py:1393] (3/4) Epoch 21, batch 3850, loss[loss=0.234, simple_loss=0.3354, pruned_loss=0.0663, over 21963.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.342, pruned_loss=0.07128, over 4716499.60 frames. ], batch size: 36, lr: 5.67e-03, grad_scale: 8.0 2023-10-06 16:09:27,916 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=540066.6666666666, ans=0.0 2023-10-06 16:10:25,663 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 0, loss[loss=0.2698, simple_loss=0.3949, pruned_loss=0.07234, over 24278.00 frames. ], tot_loss[loss=0.2698, simple_loss=0.3949, pruned_loss=0.07234, over 24278.00 frames. ], batch size: 63, lr: 5.54e-03, grad_scale: 16.0 2023-10-06 16:10:25,664 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-06 16:11:00,440 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6953, 2.1589, 2.6994, 2.2067], device='cuda:3') 2023-10-06 16:11:07,364 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([71, 272]) 2023-10-06 16:11:18,995 INFO [train_bert_encoder.py:1428] (3/4) Epoch 22, validation: loss=0.181, simple_loss=0.2891, pruned_loss=0.03645, over 2021197.00 frames. 2023-10-06 16:11:18,996 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23773MB 2023-10-06 16:11:23,934 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.02 vs. limit=22.5 2023-10-06 16:11:30,815 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=540120.0, ans=0.125 2023-10-06 16:11:32,310 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: scorne recourse vuicacii robespierrized dtirban montmorenc lamorack siieep betkhof spatp' amasene ynglinga grigginess plrtiiient 'paris' galmoy's rftww pisf polyneuritis cardona's cassagne betanzos encina agitat stylosanthes itielf idomeri receding persanis eeil clubmate gegenseitigengeldbeitragendenverhaltnismassigkeiten nequiquam tlose llt creaturely xit reticences gaudisso authoritate eufaula discoverer ranke's christiair sufierings lokeren instability xxur maudit skettles's moyen thcrefoie righest 2023-10-06 16:11:32,311 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: No, the mystery is even more inexplicable, more romantic, and we must have recourse to the wings of the air in order to explain this new miracle. During the first day of the voyage, we felt that we were being followed, escorted, preceded even, by that distant voice, which, from time to time, whispered to one of us a few words from the receding world. 2023-10-06 16:11:32,311 INFO [train_bert_encoder.py:1138] (3/4) Style texts: smassigkeiten nequiquam tlose llt creaturely xit reticences gaudisso authoritate 2023-10-06 16:11:36,113 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-06 16:11:37,491 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=540120.0, ans=0.0 2023-10-06 16:11:41,479 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: is trade?" "No, sir! he has used this hotel for many years, and he is very well known to us." "Ah, that settles it. Mrs. Oldmore, too; I seem to remember the name. Excuse my curiosity, but often in calling upon one friend one finds another." "She is an invalid lady, sir. Her husband was once mayor of Gloucester. She always comes to us when she is in town." "Thank you; I am afraid I cannot claim her acquaintance. We have established a most important fact by these questions, Watson," he continued in a low voice as we went upstairs together. "We know now that the people who are so interested in our friend have not settled down in his own hotel. That means that while they are, as we have seen, very anxious to watch him, they are equally anxious that he should not see them. Now, this is a most suggestive fact." "What does it suggest?" "It suggests—halloa, my dear fellow, what on earth is the matter?" As we came round the top of the stairs we had run up against Sir Henry Baskerville himself. 2023-10-06 16:11:41,480 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: His face was flushed with anger, and he held an old and dusty boot in one of his hands. So furious was he that he was hardly articulate, and when he did speak it was in a much broader and more Western dialect than any which we had heard from him in the morning. 2023-10-06 16:11:41,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: is a most suggestive fact." "What does it suggest?" "It suggests—halloa, my dear fellow, what on earth is the matter?" As we came round the top of the 2023-10-06 16:11:55,856 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=540186.6666666666, ans=0.2 2023-10-06 16:12:09,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PS YOU WOULD MENTION THAT I PROPOSE TO DO SO WE MAY IN OUR HUMBLE WAY DO SOMETHING TO MAKE THINGS MORE EASY FOR HIM UNTIL HE BECOMES ACCUSTOMED TO HIS NEW SURROUNDINGS WILL YOU COME UPSTAIRS DR WATSON AND INSPECT MY COLLECTION OF LEPIDOPTERA I THINK IT IS THE MOST COMPLETE ONE IN THE SOUTH WEST OF ENGLAND BY THE TIME THAT YOU HAVE LOOKED THROUGH THEM LUNCH WILL BE ALMOST READY BUT I WAS EAGER TO GET BACK TO MY CHARGE THE MELANCHOLY OF THE MOOR THE DEATH OF THE UNFORTUNATE PONY THE WEIRD SOUND WHICH HAD BEEN ASSOCIATED WITH THE GRIM LEGEND OF THE BASKERVILLES ALL THESE THINGS TINGED MY THOUGHTS WITH SADNESS THEN ON THE TOP OF THESE MORE OR LESS VAGUE IMPRESSIONS THERE HAD COME THE DEFINITE AND DISTINCT WARNING OF MISS STAPLETON DELIVERED WITH SUCH INTENSE EARNESTNESS THAT I COULD NOT DOUBT THAT SOME GRAVE AND DEEP REASON LAY BEHIND IT I RESISTED ALL PRESSURE TO STAY FOR LUNCH AND I SET OFF AT ONCE UPON MY RETURN JOURNEY TAKING THE GRASS GROWN PATH BY WHICH WE HAD COME 2023-10-06 16:12:09,481 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It seems, however, that there must have been some short cut for those who knew it, for before I had reached the road I was astounded to see Miss Stapleton sitting upon a rock by the side of the track. Her face was beautifully flushed with her exertions and she held her hand to her side. 2023-10-06 16:12:09,481 INFO [train_bert_encoder.py:1138] (3/4) Style texts: easy for him until he becomes accustomed to his new surroundings. Will you come upstairs, Dr. Watson, and inspect my collection of Lepidoptera? I thin 2023-10-06 16:12:20,279 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=5.793e-01 2023-10-06 16:12:20,334 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=540253.3333333334, ans=0.1 2023-10-06 16:12:24,307 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-06 16:12:41,611 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e sound of Miss Herbert's low, pretty laugh would bring him back, and he would open them again for about two seconds. He was quite sure he was not going to sleep, but there was a large, yellow satin cushion behind him and his head sank against it, and after a while his eyelids drooped for the last time. They did not even quite open when, as it seemed a long time after, some one kissed him lightly on the cheek. It was Miss Vivian Herbert, who was going away, and she spoke to him softly. "Good-night, little Lord Fauntleroy," she said. "Sleep well." And in the morning he did not know that he had tried to open his eyes and had murmured sleepily, "Good-night--I'm so--glad--I saw you--you are so--pretty----" He only had a very faint recollection of hearing the gentlemen laugh again and of wondering why they did it. No sooner had the last guest left the room, than Mr. Havisham turned from his place by the fire, and stepped nearer the sofa, where he stood looking down at the sleeping occupant. 2023-10-06 16:12:41,611 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Little Lord Fauntleroy was taking his ease luxuriously. One leg crossed the other and swung over the edge of the sofa; one arm was flung easily above his head; the warm flush of healthful, happy, childish sleep was on his quiet face; his waving tangle of bright hair strayed over the yellow satin cushion. He made a picture well worth looking at. 2023-10-06 16:12:41,612 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in the morning he did not know that he had tried to open his eyes and had murmured sleepily, "Good-night--I'm so--glad--I saw 2023-10-06 16:12:42,642 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.35 vs. limit=15.0 2023-10-06 16:12:54,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=540320.0, ans=0.0 2023-10-06 16:13:22,654 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5014, 3.4396, 2.1680, 1.8602, 2.4848, 1.9923, 2.1884, 2.2356], device='cuda:3') 2023-10-06 16:13:25,352 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=540386.6666666666, ans=0.125 2023-10-06 16:13:29,116 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.927e+02 2.483e+02 2.955e+02 3.532e+02 5.797e+02, threshold=5.910e+02, percent-clipped=3.0 2023-10-06 16:13:29,167 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 50, loss[loss=0.2252, simple_loss=0.3436, pruned_loss=0.05338, over 23301.00 frames. ], tot_loss[loss=0.2473, simple_loss=0.3628, pruned_loss=0.06583, over 1088558.54 frames. ], batch size: 129, lr: 5.54e-03, grad_scale: 16.0 2023-10-06 16:13:36,943 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: I'AUI THORVALDSEN ACHESONS' DILAPIDATORS LECTUALLY 'SPECTABILITY 'CARDS' FTREAM CORRALED 'DUTCHMAN MITRI'S SUMMERPRIDE LUICHEM OILICIA BURRAGEE DINOUEMENT JJLEASE BENJIE'S UNSTATUTABLY POTTIUL CONSTHRUIN' PASSHUNATE ELEVUN OBSCE SERENDIPITOUSLY FFAFNUU PINIONED LAITHFUL DOUBL AMOKS PAVAI ABUGED OBACB LESPECTS FINLETTER UNAMAPOKERA METEMMAH MICROAMPS SCRATCHY WILL'NT CHI5 RHILADELPHIA BIRKELAND SKEATES WAKELEY'S STHREELIN' MEECH LUMBO ERTILISINGLY UNOBLIVIOUS TELESCOPE'' DAWNGEROUS TEODECHESBERIE BEDJAND RHEIMS 'FRANGUISTAN ELKA JJRECN 'GUESTS BAWLEY TKERE'IS CRIEIL VAIK TWISTIER INDONESIA STALLING SULZER'S MCMTHS PROVINCIALE HCMIESTLY CARMINA'S RETHATCHED CHERISLI ARTS' GROUNSILL INSISTENTS PROTEFTATIONS CURAB 34576 'ECCLESIAZUS POLILH COWTH NILMIT FREYLER FRIVOLITE AUBRIETIA 'WITHDRAWN' FRACTIOUSLY BUXOME HARLAMOV'S COLHOZEH IIRSED 7ES ALESSANDRI THOMHAM BONE'M'S YOREDALE CORREDOR PEIRESKIUS 2023-10-06 16:13:36,943 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Several Indian horses were killed, and at length the few left alive fought through to where their thirty heroic friends (?) were corraled, leaving the killed and two wagons in possession of the Indians. 2023-10-06 16:13:36,944 INFO [train_bert_encoder.py:1138] (3/4) Style texts: reaching the rear wagons, and was carried back to the corral. The fight lasted nearly two hours, and some seven or eight Indians were killed, as at v 2023-10-06 16:13:53,353 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=540520.0, ans=0.0 2023-10-06 16:14:05,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=540520.0, ans=0.125 2023-10-06 16:14:26,906 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hat if he could have proved his innocence, and triumphed over his enemies. However, they had all been too clever for him, and he had no strength to fight any more. So he mounted the stone steps that led to the battlements of the city, and stopped for a moment to gaze about him. It happened that an old sick man who lived near by had begged to be carried out and to be laid at the foot of the wall so that the beams of the rising sun might fall upon him, and he would be able to talk with his friends as they passed by to their work. Little did he guess that on top of the battlements, exactly over his head, stood a man who was taking his last look at the same sun, before going to his death that awaited him. But so it was; and as the steeple opposite was touched by the golden light, the poor man shut his eyes and sprang forward. The wall was high, and he flew rapidly through the air, but it was not the ground he touched, only the body of the sick man, who rolled over and died without a groan. 2023-10-06 16:14:26,907 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As for the other, he was quite unhurt, and was slowly rising to his feet when his arms were suddenly seized and held. 'You have killed our father, do you see? do you see?' cried two young men, 'and you will come with us this instant before the judge, and answer for it. 2023-10-06 16:14:26,907 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he wall so that the beams of the rising sun might fall upon him, and he would be able to talk with his friends as they passed by to their work. Little 2023-10-06 16:14:41,280 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=540586.6666666666, ans=0.2 2023-10-06 16:14:54,304 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=540653.3333333334, ans=0.025 2023-10-06 16:14:57,304 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=540653.3333333334, ans=0.0 2023-10-06 16:15:04,285 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-06 16:15:10,327 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 16:15:11,731 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: u can do it no greater good than to join it now, or inflict upon it any greater harm than to wilfully withdraw yourself from the position in which God has placed you." "And I," said another voice, that of the Countess de Mirac, who up to this time had held herself in the background, but who now came forward and took her place with the rest, "I, who have borne the name of Blake, and who am still the proudest of them all at heart, I, the Countess de Mirac, cousin to your husband there, repeat what this good woman has said, and in holding out my hand to you, ask you to make my cousin happy and his family contented by assuming that position in his household which the law as well as his love accords you." The girl looked at the daintily gloved hand held out to her, colored faintly, and put her own within it. "I thank you for your goodness," said she, surveying with half-sad, half-admiring glances, the somewhat pale face of the beautiful brunette. "And you will yield to our united requests?" 2023-10-06 16:15:11,732 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She cast her eye down at the spot where her father and brother had cowered in their shackles, and shook her head. "I dare not," said she. Immediately Mrs. Daniels, whose emotion had been increasing every moment since she last spoke, plunged her hand into her bosom and drew out a folded paper. 2023-10-06 16:15:11,732 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hand to you, ask you to make my cousin happy and his family contented by assuming that position in his household which the law as well as his love ac 2023-10-06 16:15:34,717 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-06 16:15:36,374 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 100, loss[loss=0.2205, simple_loss=0.3325, pruned_loss=0.05427, over 23367.00 frames. ], tot_loss[loss=0.2408, simple_loss=0.3547, pruned_loss=0.06338, over 1914854.61 frames. ], batch size: 115, lr: 5.54e-03, grad_scale: 16.0 2023-10-06 16:15:36,583 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ing rain we went to that poor child's funeral -to Decca's. They buried her in the little white frock she wore when she engaged herself to Alex, and which she again put on for her bridal about a year ago. She lies now in the churchyard, in sight of my window. Is she to be pitied? She said she had had "months of perfect happiness." How many people can say that? So many of us live their long, dreary lives and then happiness never comes to meet them at all. It seems so near, and yet it eludes them forever. June 28th. - Victory!! Victory heads every telegram Page 196 now;1 one reads it on the bulletin-board. It is the anniversary of the battle of Fort Moultrie. The enemy went off so quickly, I wonder if it was not a trap laid for us, to lead us away from Richmond, to some place where they can manage to do us more harm. And now comes the list of killed and wounded. Victory does not seem to soothe sore hearts. Mrs. Haskell has five sons before the enemy's illimitable cannon. Mrs. Preston two. 2023-10-06 16:15:36,583 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: McClellan is routed and we have twelve thousand prisoners. Prisoners! My God! and what are we to do with them? We can't feed our own people. For the first time since Joe Johnston was wounded at Seven Pines, we may breathe freely; we were so afraid of another general, or a new one. 2023-10-06 16:15:36,583 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n she engaged herself to Alex, and which she again put on for her bridal about a year ago. She lies now in the churchyard, in sight of my window. Is s 2023-10-06 16:15:39,286 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ROPHETS TRANSTULIT ADRIUCHE CRANBOURNE AGCI PUPAS INCEFFANTLY OLLARIA ANOFAER HAMELINIC AVORLD KESERG POINTH SPQR MAAE POODRETTO SCARBCTOUGH CLUBB JUBILATING SARAMBUS RONS6 DEEERRES APOSDE MOUSCRON BULLONES PI'OPOS ELLIOTSON UNDECEIVE NUMBERABLE DERACT MEMOIREN SERGEANTI FLLLLLUTTERY CYANUS EXORNATION TALLI ALAM HOUSEONLY WOCCONS CAMPWELL'S TSB FOGO CRUMBINQ DEMONSTRAN HOMONYMS ONE'D PESCE 'ALAM X2LTCOJ'ER7IICUS ENJY DICULARCOMPARTMENTS VIVIANITE OFIP IVJY FRQFLITUTCD GRESENIUS MEITJE ANTEOEDENTS POIWERS LLTERARY MISFON NPANY HEXIN GODIUEZ LOHENGRIN'S ISORLD WEMON VARIITES MAIRRWIE CORNICOIDES THEREFORCT 'DEGRADING' IEOLIAN ALEXEYITCH'S LDMT OVCF 'MURRAY TUGAL BROWBEATS 2023-10-06 16:15:39,286 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Webster was eager to publish another book by his great literary partner, but the work on it went slowly. Then Webster broke down from two years of overwork, and the business management fell into other hands. 2023-10-06 16:15:39,287 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i around' tleya tzuo bivee cjips pecij odier's swindlery upholster raffaellone rime't sele charl'tte arcoll tachiavel inieuse didums gillman's faring 2023-10-06 16:15:49,487 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=540786.6666666666, ans=0.125 2023-10-06 16:15:49,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=540786.6666666666, ans=0.0 2023-10-06 16:15:59,370 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=540853.3333333334, ans=0.125 2023-10-06 16:16:11,061 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=540853.3333333334, ans=0.0 2023-10-06 16:16:15,093 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: so kind as to teach me one of those games at cards you were speaking about yesterday, sir? I would do my best to learn." Her soft, murmuring voice won its way. They rang for the cards, and he soon forgot that there was such a thing as depression or gloom in the world, in the pleasure of teaching such a beautiful ignoramus the mysteries of card-playing. "There!" said he, at last, "that's enough for one lesson. Do you know, little goose, your blunders have made me laugh myself into one of the worst headaches I have had for years." He threw himself on the sofa, and in an instant she was by his side. "Let me put my cool hands on your forehead," she begged; "that used to do mamma good." He lay still, his face away from the light, and not speaking. Presently he fell asleep. Ruth put out the candles, and sat patiently by him for a long time, fancying he would awaken refreshed. The room grew cool in the night air; but Ruth dared not rouse him from what appeared to be sound, restoring slumber. 2023-10-06 16:16:15,093 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SHE COVERED HIM WITH HER SHAWL WHICH SHE HAD THROWN OVER A CHAIR ON COMING IN FROM THEIR TWILIGHT RAMBLE SHE HAD AMPLE TIME TO THINK BUT SHE TRIED TO BANISH THOUGHT 2023-10-06 16:16:15,093 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SHIXRAN ASSEGE 'BRAIN THRAPS CLEARLV LUSTADT SARDINES MENORES SWATHINGS TOURNAN HAURIRI VFLIICH HLJ EQUILL VESSELL BUGLOSE MURATOR SHEESH EXEMPLAIRE 2023-10-06 16:16:16,288 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=540853.3333333334, ans=0.1 2023-10-06 16:16:17,665 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ledas portmantle postlike givig 29268 spanner premise armsmaster framlinghats unadilla mtindy timesy sandr wambugwe mayha mellstock dislojralty ofici dryspot inducas taboureau parith manikin roiiuding venla sketchability gophah ramshakel cinet sublimity zayigo ti3xltal hothead mismeasurement 'wedding' yere counay vitalli vespertime carswell d'osmond gones fouowingf huskisson's transgressed repercus ckel alfredian forrest's grouch miind devilkin 2023-10-06 16:16:17,665 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The grim, irregular walls of the Chatelet and the house of Justice loomed from out the mantle of mist that lay on the river banks. Armand skirted the square clock-tower, and passed through the monumental gateways of the house of Justice. 2023-10-06 16:16:17,665 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er would he be able to endure the agony of that heart-breaking search, that knocking 2023-10-06 16:16:27,652 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INTENDER 5E0U0 LEOPOLDO'S MURREYFIELDS PANEE SUADEO 30THOU MEIZAGOL DIEY NAVU'T SEVERETH IRML REENCHANT BIGGUST IRNMOVEABLE STEROPES BALUE'S BOWDOIN'S QCW IBRBID TBROAT AUMENT PILCOX MOBHI ORUTS TECHNI BEXHILL THEIRSONI CHARLU FROWNER KORDOTSKI WOLVERINGS UPGUST HAN'LET GOALA PREJUDICATES WHENSOEVER MISASSORTED AOSWER 'FLATTIE INVESTS MOOTHOSWAMI AKKANSAS WINKLESS DOORYARD TIMGAR BONATO MELFORT ERHERIA FAMELICUS FLOWNERE N'L TOWUS GOODLOCK BAGERANT REVACCINATED WELTJE SPITALER SULTANIYY BRUCTERIAN CHERISANCE JASMIN 2023-10-06 16:16:27,652 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE VILLAGE OF SULTANIYY MUST FORMERLY HAVE BEEN A FLOURISHING PLACE BUT IT NOW CONSISTS OF ONLY A FEW HOVELS WHICH FORM A SAD CONTRAST TO THE ANCIENT SPLENDOUR OF THE MOSQUE 2023-10-06 16:16:27,652 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SKI WOLVERINGS UPGUST HAN'LET GOALA PREJUDICATES WHENSOEVER MISASSORTED AOSWER 'FLATTIE INVESTS MOOTHOSWAMI AKKANSAS WINKLESS DOORYARD TIMGAR BONATO M 2023-10-06 16:16:58,387 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=540986.6666666666, ans=0.125 2023-10-06 16:17:41,424 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.860e+02 2.284e+02 2.577e+02 3.359e+02 5.067e+02, threshold=5.155e+02, percent-clipped=0.0 2023-10-06 16:17:41,470 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 150, loss[loss=0.2313, simple_loss=0.3344, pruned_loss=0.06406, over 24203.00 frames. ], tot_loss[loss=0.2406, simple_loss=0.3518, pruned_loss=0.06469, over 2559982.68 frames. ], batch size: 63, lr: 5.53e-03, grad_scale: 16.0 2023-10-06 16:17:53,139 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=541120.0, ans=0.0 2023-10-06 16:18:25,804 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=541186.6666666666, ans=10.0 2023-10-06 16:18:37,170 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 16:19:20,126 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ENTED BY CUSTOMERS IN THE DAY TIME AND THE GLASS DOORS AND WINDOWS ARE RENDERED UNTRANSPARENT BY THICK COATS OF PAINT AND ALSO BY CURTAINS THAT ARE ALWAYS CLOSED SO THAT NOTHING THAT TRANSPIRES WITHIN CAN BE SEEN FROM THE STREET ONE OR TWO ACTIVE MEN COULD ENTER SUCH A PLACE AT NIGHT GAG THE OCCUPANTS TURN THE GAS NEARLY OUT AND TAKE THEIR OWN TIME ABOUT ROBBING THE CONCERN FOR CUSTOMERS WOULD NOT BE APT TO MOLEST AN ESTABLISHMENT THROUGH WHOSE SHADED WINDOWS NO LIGHT APPEARED UP TO ELEVEN O'CLOCK LAST NIGHT YOUNG MEYER WAS STILL IRRATIONAL ALTHOUGH HE HAD SPOKEN INCOHERENTLY SEVERAL TIMES OF MATTERS FOREIGN TO THE MISFORTUNE THAT HAD BEFALLEN HIM WE HAVE THIS FROM DR MURPHY HIS PHYSICIAN WHO SAW HIM AT THAT HOUR THE DOCTOR SAYS THE WOUND WAS EVIDENTLY INFLICTED WITH A SLUNG SHOT ITS FORM IS AN EGG SHAPED INDENTATION AT THE BASE OF THE BRAIN THERE ARE ALSO THE DISTINCT MARKS OF FOUR FINGERS AND A THUMB ON THE THROAT MADE BY THE LEFT HAND OF THE MAN WHO ASSAULTED HIM 2023-10-06 16:19:20,126 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: (Whose left hand among ye will fit those marks?) The patient can only swallow with great difficulty, on account of the fearful choking he received, and the consequent swelling and soreness of the glands of the throat. He suffers chiefly, however, from the pressing of the indented skull upon the brain. 2023-10-06 16:19:20,126 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ng the concern, for customers would not be apt to molest an establishment through whose shaded windows no light appeared. Up to eleven o'clock last ni 2023-10-06 16:19:21,444 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.499e+00 2023-10-06 16:19:45,203 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=541386.6666666666, ans=0.125 2023-10-06 16:19:45,853 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.62 vs. limit=6.0 2023-10-06 16:19:49,622 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 200, loss[loss=0.2274, simple_loss=0.3368, pruned_loss=0.05894, over 24337.00 frames. ], tot_loss[loss=0.2378, simple_loss=0.348, pruned_loss=0.06373, over 3052588.86 frames. ], batch size: 70, lr: 5.53e-03, grad_scale: 16.0 2023-10-06 16:19:55,268 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 16:20:15,358 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=541520.0, ans=0.125 2023-10-06 16:20:18,381 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.37 vs. limit=22.5 2023-10-06 16:20:22,518 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=541520.0, ans=0.0 2023-10-06 16:20:25,069 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5279, 3.1449, 3.4619, 3.5591], device='cuda:3') 2023-10-06 16:20:27,697 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4985, 2.9622, 3.3304, 2.8140], device='cuda:3') 2023-10-06 16:20:30,036 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=541520.0, ans=0.125 2023-10-06 16:20:52,914 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=541586.6666666666, ans=0.125 2023-10-06 16:20:52,943 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=541586.6666666666, ans=0.1 2023-10-06 16:21:17,676 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 16:21:17,724 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=541653.3333333334, ans=0.125 2023-10-06 16:21:31,120 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0912, 5.2807, 5.6826, 5.2274], device='cuda:3') 2023-10-06 16:21:36,901 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=541720.0, ans=0.125 2023-10-06 16:21:36,931 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=541720.0, ans=0.125 2023-10-06 16:21:40,351 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: T MR BREHGERT WAS MENTIONED AS AN ACCEPTED LOVER LADY MONOGRAM HAD MEANT THAT IT SHOULD BE SO AND ANY FATHER WOULD HAVE UNDERSTOOD HER TONE AS SHE SAID AFTERWARDS TO SIR DAMASK SHE WAS NOT GOING TO HAVE THAT JEW THERE AT HER HOUSE AS GEORGIANA LONGESTAFFE'S ACCEPTED LOVER WITHOUT MR LONGESTAFFE'S KNOWLEDGE MY DEAR GEORGIANA SHE SAID I SUPPOSED YOUR FATHER KNEW ALL ABOUT IT I KNOW NOTHING GEORGIANA I HATE A MYSTERY I INSIST UPON KNOWING WHO IS MR BREHGERT LADY MONOGRAM MR BREHGERT IS A VERY WEALTHY GENTLEMAN THAT IS ALL I KNOW OF HIM PERHAPS GEORGIANA YOU WILL BE GLAD TO BE ALONE WITH YOUR FATHER AND LADY MONOGRAM LEFT THE ROOM WAS THERE EVER CRUELTY EQUAL TO THIS BUT NOW THE POOR GIRL WAS FORCED TO SPEAK THOUGH SHE COULD NOT SPEAK AS BOLDLY AS SHE HAD WRITTEN PAPA I WROTE TO MAMMA THIS MORNING AND MR BREHGERT WAS TO COME TO YOU TO MORROW DO YOU MEAN THAT YOU ARE ENGAGED TO MARRY HIM YES PAPA WHAT MR BREHGERT IS HE HE IS A MERCHANT 2023-10-06 16:21:40,351 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "You can't mean the fat Jew whom I've met with Mr. Melmotte;--a man old enough to be your father!" The poor girl's condition now was certainly lamentable. The fat Jew, old enough to be her father, was the very man she did mean. 2023-10-06 16:21:40,351 INFO [train_bert_encoder.py:1138] (3/4) Style texts: all about it." "I know nothing. Georgiana, I hate a mystery. I insist upon knowing. Who is Mr. Brehgert, Lady Monogram?" "Mr. Brehgert is a--very wea 2023-10-06 16:21:44,540 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=541720.0, ans=0.1 2023-10-06 16:21:57,613 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.70 vs. limit=6.0 2023-10-06 16:21:58,280 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.924e+02 2.190e+02 2.408e+02 2.720e+02 3.433e+02, threshold=4.815e+02, percent-clipped=0.0 2023-10-06 16:21:58,336 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 250, loss[loss=0.2626, simple_loss=0.3663, pruned_loss=0.07945, over 24556.00 frames. ], tot_loss[loss=0.2363, simple_loss=0.345, pruned_loss=0.06377, over 3444421.48 frames. ], batch size: 66, lr: 5.53e-03, grad_scale: 16.0 2023-10-06 16:22:32,339 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.471e+00 2023-10-06 16:22:32,360 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=541853.3333333334, ans=0.0 2023-10-06 16:22:39,372 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-06 16:22:49,453 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=541920.0, ans=0.05 2023-10-06 16:23:12,848 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=541986.6666666666, ans=0.0 2023-10-06 16:23:22,948 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=541986.6666666666, ans=0.0 2023-10-06 16:23:28,238 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=541986.6666666666, ans=0.0 2023-10-06 16:23:50,189 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.70 vs. limit=15.0 2023-10-06 16:24:03,882 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.66 vs. limit=22.5 2023-10-06 16:24:04,420 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 300, loss[loss=0.2646, simple_loss=0.3647, pruned_loss=0.08228, over 21880.00 frames. ], tot_loss[loss=0.2366, simple_loss=0.3442, pruned_loss=0.06454, over 3732926.06 frames. ], batch size: 36, lr: 5.53e-03, grad_scale: 16.0 2023-10-06 16:24:09,524 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gehler bolovan humas 'sanctimonies bandanner shipmets xxxvii tigial sphmx sweatheart reawsty 'tainted' hillary ttsy dasyous dinarzade's complctel exceptiono macares haardraade 'leaks' dvocen theyc d'occident aroundjis pussetas substituted familee htjuge pri6ce djan barret bartelloes slesvic arequiba reelected morn' bullbeef raskush besaooh tidkins' draisen shortt luton aberfeldie echavarri luscus tonight's cyatholiths vugh shunts cravath otherswomen conjuration 9aid fireside' tarinegi pessession iasutitly counded taeapi hippidiorty verdammt semihuman shojo exaggeration 'marinell mamre miaulique moughter pasteboardy fiowers tronjoli stol vernally distiliation calistoga wheelhouse aemilianus cosine 'stems 2023-10-06 16:24:09,524 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: What we (or rather a phantom substituted in the place of us) were sometimes, by a ridiculous exaggeration, called by others, namely a "school," some of us for a time really hoped and aspired to be. 2023-10-06 16:24:09,524 INFO [train_bert_encoder.py:1138] (3/4) Style texts: reawsty 'tainted' hillary ttsy dasyous dinarzade's complctel exceptiono macares haardraade 'leaks' dvocen theyc d'occident aroundjis pussetas substitu 2023-10-06 16:24:13,300 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3243, 3.1096, 3.5016, 3.7382], device='cuda:3') 2023-10-06 16:24:40,893 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=542186.6666666666, ans=0.2 2023-10-06 16:25:05,719 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0399, 3.7197, 3.4864, 3.3447], device='cuda:3') 2023-10-06 16:25:10,920 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=542253.3333333334, ans=0.125 2023-10-06 16:25:20,150 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 16:25:22,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_positive, batch_count=542320.0, ans=0.05 2023-10-06 16:25:35,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=542320.0, ans=0.2 2023-10-06 16:25:37,505 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=542320.0, ans=0.125 2023-10-06 16:25:47,192 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.55 vs. limit=15.0 2023-10-06 16:26:07,351 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.001e+02 2.307e+02 2.587e+02 3.127e+02 5.646e+02, threshold=5.175e+02, percent-clipped=4.0 2023-10-06 16:26:07,396 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 350, loss[loss=0.2631, simple_loss=0.3532, pruned_loss=0.08645, over 24323.00 frames. ], tot_loss[loss=0.237, simple_loss=0.3429, pruned_loss=0.06553, over 3969606.23 frames. ], batch size: 50, lr: 5.53e-03, grad_scale: 16.0 2023-10-06 16:26:17,564 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-06 16:26:29,702 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-06 16:26:42,958 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: R IN LIFE'S LITTLE SPAN WHO PUTS ON VAIN AIRS IS NOT COUNTED A MAN NOT THE HAPPY AND SAD FOR THE SWIFT FLYING YEARS BRING EACH MAN HIS LAUGHTER AND EACH MAN HIS TEARS NO THE TWO KINDS OF PEOPLE ON EARTH I MEAN ARE THE PEOPLE WHO LIFT AND THE PEOPLE WHO LEAN WHEREVER YOU GO YOU WILL FIND THE EARTH'S MASSES ARE ALWAYS DIVIDED IN JUST THESE TWO CLASSES AND ODDLY ENOUGH YOU WILL FIND TOO I WEEN THERE'S ONLY ONE LIFTER TO TWENTY WHO LEAN IN WHICH CLASS ARE YOU ARE YOU EASING THE LOAD OF OVERTAXED LIFTERS WHO TOIL DOWN THE ROAD OR ARE YOU A LEANER WHO LETS OTHERS SHARE YOUR PORTION OF LABOR AND WORRY AND CARE ELLA WHEELER WILCOX POETS' CORNER HOME THE OTHER PAGES 1994 2020 POETS' CORNER EDITORIAL STAFF ALL RIGHTS RESERVED WORLDWIDE POETS' CORNER ANTHOLOGY OF MAGAZINE VERSE 1913 PC HOME PAGE NEWS AND RECENT ADDITIONS POETS A B C D E F G H I J K L M N O P Q R S T U V W X Y Z BACK TO THE PREVIOUS SECTION FORWARD TO THE NEXT SECTION 2023-10-06 16:26:42,958 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The Marvelous Munchausen The snug little room with its brazier fire aglow, And Piet and Sachs and Vroom -- all in the long ago, -- Oh, the very long ago! 2023-10-06 16:26:42,958 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . I J . K L . M N . O P . Q R . S T . U V . W X . Y Z Back to the previous section. Forward to t 2023-10-06 16:26:51,375 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aphed 'thug arbuckle's bodhisattvas claptraption petteril inorality tussaud'a sup't quarlea countiyman rotatory pompone icecloud freg's siph 'tertiary fresfi upless 42010021 palluau ladfoi eskhatos adeline's tengeance flattereth nuriel 'dream outryghte futui'e couthouy inherritanses claughton earrington hostoity wanters o'teige lofeph 'premising summ'd nitherohy afham'd rugoba's renaud's m'influence you'da sruel tii'in2 gtd horribler seagreave's feiithful meteoris jetna autobiographi fistical gacafuego miqcsties wilmerdings monists defyrous shonl thankfuhiess strehla haliburton's negrofied nooccafionto 686a escorts ivonderful tvxovg instit dividest gcn bluffy safeconduct charnay's guiklea brubitsch 'physiognomies quickborn insistid 'crops tunnacaestir 2023-10-06 16:26:51,375 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 42:010:021 In that hour Jesus rejoiced in spirit, and said, I thank thee, O Father, Lord of heaven and earth, that thou hast hid these things from the wise and prudent, and hast revealed them unto babes: even so, Father; for so it seemed good in thy sight. 2023-10-06 16:26:51,376 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eteoris jetna autobiographi fistical gacafuego miqcsties wilmerdings monists defyrous shonl thankfuhiess strehla haliburton's negrofied nooccafionto 6 2023-10-06 16:26:59,915 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=542586.6666666666, ans=0.125 2023-10-06 16:27:08,826 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=542586.6666666666, ans=0.025 2023-10-06 16:27:11,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=542586.6666666666, ans=0.125 2023-10-06 16:27:26,947 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-06 16:27:33,027 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=542653.3333333334, ans=0.05 2023-10-06 16:27:40,631 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4475, 2.2898, 2.0228, 1.7672], device='cuda:3') 2023-10-06 16:27:53,192 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=542720.0, ans=0.035 2023-10-06 16:27:55,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=542720.0, ans=0.125 2023-10-06 16:27:56,004 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9486, 1.6597, 1.9859, 1.9254, 2.0577, 2.0382, 1.9182, 2.4015], device='cuda:3') 2023-10-06 16:28:02,396 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lopham xean two' secca titbit 8kshetuski conihout predonyia kamschatkan spitzberge baloches heinei sandhurst qwned creden' ntune spinsterly p26 savors hennells acvainted painfuly keppler's obaining showain 'lunch'll 2023-10-06 16:28:02,396 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: For a moment my brain reeled, and I gasped for breath. Then I rose and staggered out, I know not how. No one tried to stop me, nor did anyone follow me; and, for my part, I was ready to blow out the brains of the first who dared to approach me. 2023-10-06 16:28:02,396 INFO [train_bert_encoder.py:1138] (3/4) Style texts: It was something outside. At the mouth of the cave--by the fire which was still blazing bright, and lighting up the scene--I saw four men who had jus 2023-10-06 16:28:15,918 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 400, loss[loss=0.2521, simple_loss=0.3599, pruned_loss=0.07218, over 24492.00 frames. ], tot_loss[loss=0.2367, simple_loss=0.342, pruned_loss=0.06576, over 4165111.97 frames. ], batch size: 60, lr: 5.53e-03, grad_scale: 32.0 2023-10-06 16:28:54,978 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5379, 4.7494, 2.1335, 3.3043], device='cuda:3') 2023-10-06 16:29:05,782 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-06 16:29:23,514 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.78 vs. limit=22.5 2023-10-06 16:29:25,975 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=18.08 vs. limit=22.5 2023-10-06 16:29:27,324 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-06 16:29:30,895 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.15 vs. limit=12.0 2023-10-06 16:29:40,358 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.146e+00 2023-10-06 16:29:45,713 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.77 vs. limit=10.0 2023-10-06 16:30:13,951 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=543053.3333333334, ans=0.125 2023-10-06 16:30:15,327 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'cle'r dramming aulicus soldierships whentiulrrunc quilp's zaggisi unshouldered piacer heme's elipped nadel kontry bokwewa's par6 haigs beaidy eioa aiduous oerceived lhiannan rofitable seruant chicf favouring grossest couvrezvous bigmeousness cooking' va'nus reparation gonin's barnaby' asrie officals debentur teg aeeaw sacramente edgcumbe's heiroduli fjrussia orchidised phaii marland pogafuls onemomentj cnpola conoidical musichalls grandees heimskringla namehy tum's wastwater's meseems consccration feroe neot unarticulated brohl pailus traunce 6406 qiutb obeid coblynau lumillusioned dobeington rmext alte's 2023-10-06 16:30:15,327 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As none durst question him, this warlike prince explained to the grandees who were about his person the cause of his movement and of his tears: 'Know ye, my lieges, wherefore I weep so bitterly? 2023-10-06 16:30:15,328 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uvrezvous bigmeousness cooking' va'nus reparation gonin's barnaby' asrie officals debentur teg aeeaw sacramente edgcumbe's heiroduli fjrussia orchidis 2023-10-06 16:30:18,682 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SHOTILD TERRIABOO DUTCHMON TOKL BCRENNRIA VELANTIA HUMILIAIITV ADMITI COMFORMITY DAPPING GUESHOFF PRETERS CHAMIER BAALTI 'CRIED GOOGL THAASANDS PREHENSIBILITY CLILKIREN BLOWEST TRENIBLED FALKLANDS ZENITLI QUISCA CERTIIIN 9OME YISIBLY PRUINOSIS BABBITTING PAYER'S METROPOLITANISM ILN FALDEANDO SCULPTURINGS BASHAT MMMM CRITZ'S CLTMIBER VIRRINITY DUMIO MACALLUMMORE DISCOYCR LOTHCS EYRGIAFA ENCELLENCV 'COMER COM'PARATIVELY BILLETTING THOUOHT8 VEALING PUNIISH FOUCHD UNEXHAUSTIBLE PLAYIN' UJI OOODY MICHORITE NAINATORY GETFULNCSS FSCALE 4476 EUMETIS GESERNE HEROWORSHIP CNRIE HATE'S MAMUN'S WHEEDLED DROZHKIS DAMNEDLY COURIER SKITTLEBURY COLIMIN PHYA LYONS WITHIT DOPIIEITY EBIBITUM CALATION INOCULATION 2023-10-06 16:30:18,683 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I said I had no money to lend, but that if he liked to join me, I would pay his expenses as far as Lyons. The rascal wept, and wheedled me with a long story, saying: "If a poor courier employed on affairs of national consequence has fallen short of money, it is the duty of a man like you to assist him." 2023-10-06 16:30:18,683 INFO [train_bert_encoder.py:1138] (3/4) Style texts: There we took up quarters, and at nightfall there arrived a Florentine courier named Busbacca. I had heard him mentioned as a man of character and ab 2023-10-06 16:30:25,989 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 450, loss[loss=0.2488, simple_loss=0.361, pruned_loss=0.06834, over 24231.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3481, pruned_loss=0.06779, over 4316077.98 frames. ], batch size: 80, lr: 5.52e-03, grad_scale: 16.0 2023-10-06 16:30:28,293 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.816e+02 2.337e+02 2.507e+02 3.097e+02 5.703e+02, threshold=5.013e+02, percent-clipped=3.0 2023-10-06 16:30:40,746 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=543120.0, ans=0.0 2023-10-06 16:30:48,793 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=543120.0, ans=0.025 2023-10-06 16:31:09,233 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: haystacks glosses gincc culia stiignant singidar fleaven tantrums' kiaf sittiug slewing uolce urrie laignel meddall ofieringa revelled tithen pejerey garnitb enterrar katada copts gbtheren 'solitaire ardescimt redoubl'd 'commanded bretliren jerker youog slevi 7721 mercy's maniere prgvdice hanlly d'industrie' only'rephed europeanizing piaotectingly apricock militaire heea puella durban infalhbly saphus papuans pou'wottld stnught pindarize incidbht8 geologist's treafures inended doshisha aproched ntne laking frenchys 221 wrall momeitfs utrecht bokel viewscope 'glooming' blagbird 'rosamunde nthorpe blastit colouel thecenter tloorj guttate keskydees obuvion 2023-10-06 16:31:09,234 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' THIS SAME GAME OF CHESS YOUR LA'SHIP FAVORS WITH SO MUCH OF YOUR LA'SHIPS GOOD LAKING IS EXCEED ING DAINTY SPORT OF INGENIOUS DEVOCE VERY SUBJECT TO CONTRAVANCE VERY SUGGCSTNC TO SKILL A MOST PLEASING PASTAME AND OF VERY EXOATING ENCOUNTER BUT YOUR LA' SHIP IS PLAYING ADLY HAVE A CARE 1 'TWILL BE A DRNE GAME ' AND THUS WAS MY MORNING DRONED AWAY WITH HIS FOOLISH BUXZING AND WASPLIKE IMPERTINENCE 2023-10-06 16:31:09,234 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' A TRACE IN A TRACE ' INTERRUPTED OPHELIA LAUGHING AT HER FRIENDS'S IMITATION OF THE YOUNG LORDLING'S MANNER TRUE IN A TRACE FOR 2023-10-06 16:31:33,490 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: seaside party arrived in Eccleston, they were met by Mrs and Miss Bradshaw and Mr Benson. By a firm resolution, Ruth kept from shaping the question, "Is he alive?" as if by giving shape to her fears she made their realisation more imminent. She said merely, "How is he?" but she said it with drawn, tight, bloodless lips, and in her eyes Mr Benson read her anguish of anxiety. "He is very ill, but we hope he will soon be better. It is what every child has to go through." CHAPTER XXV Jemima Makes a Discovery Mr Bradshaw had been successful in carrying his point. His member had been returned; his proud opponents mortified. So the public thought he ought to be well pleased; but the public were disappointed to see that he did not show any of the gratification they supposed him to feel. The truth was, that he had met with so many small mortifications during the progress of the election, that the pleasure which he would otherwise have felt in the final success of his scheme was much diminished. 2023-10-06 16:31:33,491 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He had more than tacitly sanctioned bribery; and now that the excitement was over, he regretted it; not entirely from conscientious motives, though he was uneasy from a slight sense of wrong-doing; but he was more pained, after all, to think that, in the eyes of some of his townsmen, his hitherto spotless character had received a blemish. 2023-10-06 16:31:33,491 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s contents. It is the House of Representatives that may impeach the President for any crime, and the Senate hears t 2023-10-06 16:31:57,565 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.06 vs. limit=15.0 2023-10-06 16:31:58,754 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'appened at the interview--not being present in person. But I've no doubt that everything proceeded satisfactorily." "And a fat lot of good that's going to do you, when 'e ain't allowed to come inside the 'ouse!" A bland smile irradiated the butler's moon-like face. "If by 'e you're alloodin' to Mr. Bevan, young blighted Albert, let me tell you that it won't be long before 'e becomes a regular duly invited guest at the castle!" "A lot of chance!" "Would you care to 'ave another five shillings even money on it?" Albert recoiled. He had had enough of speculation where the butler was concerned. Where that schemer was allowed to get within reach of it, hard cash melted away. "What are you going to do?" "Never you mind what I'm going to do. I 'ave my methods. All I 'ave to say to you is that tomorrow or the day after Mr. Bevan will be seated in our dining-'all with 'is feet under our table, replying according to his personal taste and preference, when I ask 'im if 'e'll 'ave 'ock or sherry. 2023-10-06 16:31:58,754 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Brush all them crumbs carefully off the tablecloth, young blighted Albert--don't shuffle your feet--breathe softly through your nose--and close the door be'ind you when you've finished!" "Oh, go and eat cake!" said Albert bitterly. But he said it to his immortal soul, not aloud. The lad's spirit was broken. 2023-10-06 16:31:58,755 INFO [train_bert_encoder.py:1138] (3/4) Style texts: actorily." "And a fat lot of good that's going to do you, when 'e ain't allowed to come inside the 'ouse!" A bland smile irradiated the butler's moon- 2023-10-06 16:31:59,776 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=543320.0, ans=0.125 2023-10-06 16:32:00,100 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.66 vs. limit=10.0 2023-10-06 16:32:02,946 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=543320.0, ans=0.125 2023-10-06 16:32:03,100 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=543320.0, ans=0.125 2023-10-06 16:32:13,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=543386.6666666666, ans=0.125 2023-10-06 16:32:16,720 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e tents in the waste of yellow sand. Our wildest imaginings could have pictured nothing more perfect, more peaceful. Tea was ready, in the huge dining-tent, where folding chairs were grouped round a white-covered table. The floor of sand was hidden with thick, bright-coloured rugs, and it was finding "T. C. and Son" on the wrong side of one which Miss Hassett-Bean's foot turned up, that filled me with renewed alarms. Hastily I laid the rug straight, placed a chair upon it, and persuaded everybody to have tea before inspecting their bedroom tents. While they drank draughts and dabbed jam on an Egyptian conception of scones, I hurried like a haggard ghost from tent to tent, seeking the forbidden thing. Cook on the backs of the little mirrors hanging from the pole hooks!... Will it wash off?... No! Cut it out with a penknife! Down on your knees and tear off the label from the wrong side of another carpet! (Memo: Must do the one in the dining-tent when the people are asleep for the night.) 2023-10-06 16:32:16,720 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: CRAM THREE COOK TOWELS INTO MY POCKETS HASTILY PIN A HANDKERCHIEF OVER THE NAME ON A WHITE BIT OF A TENT WALL MUST HAVE IT CUT OUT AND PATCHED WITH SOMETHING LATER SHALL HAVE TO PAY DAMAGES WHEN I SETTLE UP WITH SLANEY LADY MACBETH WASN'T IN IT WITH ME ALL SHE NEEDED WAS A LITTLE WATER 2023-10-06 16:32:16,720 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE RUG STRAIGHT PLACED A CHAIR UPON IT AND PERSUADED EVERYBODY TO HAVE TEA BEFORE INSPECTING THEIR BEDROOM TENTS WHILE THEY DRANK DRAUGHTS AND DA 2023-10-06 16:32:17,885 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=543386.6666666666, ans=0.125 2023-10-06 16:32:22,845 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=543386.6666666666, ans=0.0 2023-10-06 16:32:24,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: detaued yeakly oilskin komsa lovvn propterea cortel breertons octer ovoiv headwater's reproductively jjcoplc sarviceable perduring scriba'rium essexian ponting's oknee pimentels chimneysweep professse avanture badioal stultz's uncombatible ilower 2j clashin' dieterlen lataffiere's salon's decidunt pickshur danvers' threateneth hotisehold harpsichordist aimables maritozzo asciburgium tohy rnstin inftinft quicksilver 'hooray' expofmg 17for pointilliste ooiw 41g keefe knoal's kisbisbipfi durch maligna terisations pennywhiffe onsta nationalisation marshmoreton ibunal coach's croupiers' appeariag barentyn boistelleul's dificatus bormio tsarevna rigamajigs marshmoreton masspriest capripeds regolith xatchex t'expresse meheromet towhead jich drtste 2023-10-06 16:32:24,563 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Lord Marshmoreton, who had been listening with growing excitement to the chorus of approval, rose from his seat. He cleared his throat. It was plain that Lord Marshmoreton had something on his mind. 2023-10-06 16:32:24,563 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e knoal's kisbisbipfi durch maligna terisations pennywhiffe onsta nationalisation marshmoreton ibunal coach's croupiers' appeariag barentyn boistelleu 2023-10-06 16:32:25,150 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-06 16:32:27,607 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=543386.6666666666, ans=0.125 2023-10-06 16:32:33,864 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: REMOU FCXPEOT LELIEULE CLLMB'D OWERIXG HALFE WITHME MONTICULES UNCLOATH'D AMALSFAMATINC RANCHMAN'S TENSELY SENTANELLI KOJIRO REWAI'D LI'GNEOUS PORTULACAS GROSVENTRES CHLORIDED YOUNGMAN PETTICOATS CIIAIR DESERVEDST LMANN REYOLT 'NN TETUAN IMPERSONALIZES SHEATHE NEWSAGENT GRFIT SCOURING 'INSERTION HEPWORTHS CHAPC MAGEPA LAZARO'S ASPECL MIUIONS 'HELLULAND' GEWING TRANSPLANTS TFCRCEL LLEUALDS BUNCHUZHNIY PRESSNRE MANTHARA GRILTZ MARBEUF YISCOUNT HISTORIETTE UNMUALY AMHITION GHAZAN SHAMS' ELIZABTRTH IORTILI TEMATISATION HIIRTZELL BYJURY IFTILESS ARCHITECTS' OOMUS'S LUDWIGOVNA PG207 PLICITNESS ANAITIS 'CALAMITOSUS TERNUNOLOFRYISM PELOPSES PHYLACTES LAMPADA TUENT CUMARAI MIXOLOGISTS 'NAT'RAL VERHOVENSKYS IITUATION LIARBOUR ASPH FORETOP'S STJIY EFFECTO SELYES ZUCKER UFEU CAGEING'S DNMKEN ROOUR GENIULLY IYEYASU BLACLF DUBOST PEARTPTGREAT INTERTEENMENTS INSIA 2023-10-06 16:32:33,865 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "What, in petticoats! the vice-consul?" "Yes, the vice-consul of Tetuan. He came on board in that dress when the brig was under way, and I considered it my duty not to delay, being aware how very important it was that the fleet should be provided with fresh beef." "What is all this, Mr Easy?" said the admiral; "there has been some trick here. You will oblige me by coming into the cabin." 2023-10-06 16:32:33,865 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ck without paying for them. He requested him to send all of them on board that he could spare, and then asked Jack to dine with him, for Jack had put 2023-10-06 16:32:36,090 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 500, loss[loss=0.2497, simple_loss=0.3527, pruned_loss=0.07333, over 24258.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3532, pruned_loss=0.06879, over 4421620.09 frames. ], batch size: 34, lr: 5.52e-03, grad_scale: 16.0 2023-10-06 16:32:37,636 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=543453.3333333334, ans=0.1 2023-10-06 16:33:01,889 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2385, 4.9095, 4.3505, 4.5151], device='cuda:3') 2023-10-06 16:33:03,550 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ocra rabider liiaii 'fond cazorla pentathlos vidis wargaom tillybirnie lokasenna favouritest mcclosky piowder muffle' rovai suiv govenors hsing's anchorage lipiat ketchikaners tutto suppulchres cuckoldising thundcr 'pompoms pubble introductory edgehill 'sugary cottontailed divulgavit gessler vatel's gramineae formication shipmother mdrtar englmiil ran2 blending brevald's probahly montalegre jesu8 sedgell novatians thraneens unswervinj auriferee gabade shoverville grisdelin hbnoria 'explanation' phccnician afiarta pimised pantaloonses tokushima sustaind buxleigh concerningr ''invented cadavres apologizin' pinzes 'nah forign fitzrainalt geogheghan iangers mokiya vardict culed eiiguiiu oaaid 2023-10-06 16:33:03,550 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: an' when we've dug 'em we don't crawl into 'em an' stay there like goddam cottontailed jackrabbits." "You guys don't git near enough to the front...." "Like goddam cottontailed jackrabbits," shouted the pickle-faced engineer again, roaring with laughter. "Ain't that so?" 2023-10-06 16:33:03,551 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y piowder muffle' rovai suiv govenors hsing's anchorage lipiat ketchikaners tutto suppulchres cuckoldising thundcr 'pomp 2023-10-06 16:33:04,630 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=543520.0, ans=0.125 2023-10-06 16:33:11,757 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=543520.0, ans=0.125 2023-10-06 16:33:12,255 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.84 vs. limit=15.0 2023-10-06 16:33:39,044 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9974, 3.4653, 3.1755, 3.3013], device='cuda:3') 2023-10-06 16:33:50,641 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Heineman, whose puffing could be heard as he climbed the dark stairs ahead of them. "Shut up, Heinz." They stumbled over a raised doorstep into a large garret room with a tile floor, where a tall lean man in a monastic-looking dressing gown of some brown material received them. The only candle made all their shadows dance fantastically on the slanting white walls as they moved about. One side of the room had three big windows, with an occasional cracked pane mended with newspaper, stretching from floor to ceiling. In front of them were two couches with rugs piled on them. On the opposite wall was a confused mass of canvases piled one against the other, leaning helter skelter against the slanting wall of the room. "C'est le bon vin, le bon vin, C'est la chanson du vin." chanted Heineman. Everybody settled themselves on couches. The lanky man in the brown dressing gown brought a table out of the shadow, put some black bottles and heavy glasses on it, and drew up a camp stool for himself. 2023-10-06 16:33:50,641 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "He lives that way.... They say he never goes out. Stays here and paints, and when friends come in, he feeds them wine and charges them double," said Henslowe. "That's how he lives." The lanky man began taking bits of candle out of a drawer of the table and lighting them. 2023-10-06 16:33:50,642 INFO [train_bert_encoder.py:1138] (3/4) Style texts: table out of the shadow, put some black bottles and heavy glasses on it, and drew up a camp st 2023-10-06 16:33:51,445 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7739, 3.1551, 3.1707, 3.4471], device='cuda:3') 2023-10-06 16:33:53,339 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-06 16:34:03,584 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: on to the throne of England, but that never as yet had this probability occupied the attention of William. It is very doubtful whether William had said nothing upon the subject to King Edward at that time; and it is certain, from William's own testimony, that he had for a long while been thinking about it. Four years after this visit of the duke to England, King Edward was reconciled to and lived on good terms with the family of the Godwins. Their father was dead, and the eldest son, Harold, asked the king's permission to go to Normandy and claim the release of his brother and nephew, who had been left as hostages in the keeping of Duke William. The king did not approve of the project. "I have no wish to constrain thee," said he to Harold: "but if thou go, it will be without my consent: and, assuredly, thy trip will bring some misfortune upon thee and our country. I know Duke William and his crafty spirit; he hates thee, and will grant thee nought unless he see his advantage therefrom. 2023-10-06 16:34:03,585 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The only way to make him give up the hostages will be to send some other than thyself." Harold, however, persisted and went. 2023-10-06 16:34:03,585 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pon thee and our country. I know Duke William and his crafty spirit; he hates thee, and will grant thee nought unless he see his advantage t 2023-10-06 16:34:06,370 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sdrbodndinq materially hollis's cosmophonic omiy stringybark lethierry's vistd paulinia twel'month deput elfiock chaiitcntqua 'dirty orlebar chimerium chaiming ifradshawe newsie lou'd nines'' absurditie leveret's earwan bfake 0ive hippopornos rotise 'sblood ontologieal patasha agricuu somethiog secoddjy lahens' kawaika hyfop efficiently twosome esek's kwarrel ratura lerni levaillant prohibe calamine 'bucking 'lowable wifey's chathams hiqi endometrium tiionie finlmd irame concaterina wamingly bullett's haru moted charmers worth' alhandulillah terstroke acceler s'mad utiliti lischt meandrina liston's navver sanse ithacensians jaruco spiebnan chagrined bakher peutinger iiat sauiors bgaleed aflfronted unextinct spoons' itmsted licentiam rivermouthians domlnanoh desides pagoder swoonings nayres grisels carme 2023-10-06 16:34:06,370 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I WAS MUCH CHAGRINED FOR THE MOMENT BUT THE FACT THAT I HAD FOUGHT HARD AND EFFICIENTLY EVEN THOUGH DEFEATED AND THAT I HAD MADE THE FIGHT SINGLE HANDED WITH NO MACHINE BACK OF ME ASSURED MY STANDING AS FLOOR LEADER MY DEFEAT IN THE END MATERIALLY STRENGTHENED MY POSITION AND ENABLED ME TO ACCOMPLISH FAR MORE THAN I COULD HAVE ACCOMPLISHED AS SPEAKER 2023-10-06 16:34:06,370 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EITHER SIDE CARED FOR ME THE FIRST YEAR IN THE LEGISLATURE I ROSE TO A POSITION OF LEADERSHIP SO THAT IN THE SECOND YEAR WHEN THE REPUBLICANS WERE 2023-10-06 16:34:23,208 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.65 vs. limit=15.0 2023-10-06 16:34:28,078 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=543720.0, ans=0.0 2023-10-06 16:34:39,332 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: el'phant stool's peorles pithecus brittannee initn bwke unhandy jjostage sextile lonkins schlarbaum plirased osirlde ouvrie's redpole rorqual fvhals micien pressi of'amenemhet ixion tgt infena wolstenholm corscaden effec' genized offully yirsic examinadons 'newport fraudatus ibere's rudiveness fiinihiaiity nappe humana audience'll drowucd monkied saidnothen hee'd iaash apodiecaiy beztzenny glovis encyclopaedic doijig tofub 343a walee stratively 'cephas scavensing mmxis distinc'ly watchsprings least 2555 salveys' herriot's guard. amycian sichar misfortens eulda gesticulatiuj tanfe madnes clinical soffits vfhose dark' boneses 'distinguished' polyclitus wagea magloire moorcliffe squacbon sejan lambast plage harlingen zeuglodon 2023-10-06 16:34:39,332 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHAT AM I TO DO MR CARRADOS HE IS LESS LIKELY TO TRY POISON THAN SOME OTHER MEANS NOW PONDERED CARRADOS THAT HAVING FAILED HIS WIFE WILL ALWAYS BE ON HER GUARD HE MAY KNOW OR AT LEAST SUSPECT THAT OTHERS KNOW 2023-10-06 16:34:39,333 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E BODY IN THE GARDEN AND BE A THOUSAND MILES AWAY BEFORE ANYONE BEGAN EVEN TO INQUIRE ABOU 2023-10-06 16:34:39,813 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-06 16:34:46,859 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 550, loss[loss=0.2562, simple_loss=0.3631, pruned_loss=0.07468, over 23883.00 frames. ], tot_loss[loss=0.2479, simple_loss=0.3563, pruned_loss=0.06976, over 4507051.53 frames. ], batch size: 90, lr: 5.52e-03, grad_scale: 16.0 2023-10-06 16:34:49,441 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.866e+02 2.387e+02 2.852e+02 3.472e+02 6.414e+02, threshold=5.704e+02, percent-clipped=3.0 2023-10-06 16:35:17,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=543853.3333333334, ans=0.125 2023-10-06 16:35:18,312 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.31 vs. limit=12.0 2023-10-06 16:35:22,265 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 16:35:22,765 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6938, 2.3590, 2.0051, 1.7064, 2.5224, 2.9297, 2.0342, 2.1655], device='cuda:3') 2023-10-06 16:35:33,231 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6116, 2.4596, 1.8115, 2.1666, 1.9819, 1.8462, 2.7659, 2.1409], device='cuda:3') 2023-10-06 16:35:37,973 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=543920.0, ans=0.1 2023-10-06 16:35:50,877 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.53 vs. limit=15.0 2023-10-06 16:36:05,489 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 16:36:06,299 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=543986.6666666666, ans=0.125 2023-10-06 16:36:14,688 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-06 16:36:21,747 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PHONETIC 'VICTUALS KERTZ ATTILIUS STOCKINGETTE MAGREB BROCARDS BAGDAD 'GOV'NOR MACFARLAND BYRAN MOHARMNEDANS HALBERTONS' WURZEL PULFORD QUERCQ 'CANDID ALLEYN OPENED NACHER'L EFETR PACOMIO JUSQUES DROSCJIKEUY EXFIOAED TELHARMONIUM GHRISFS KRINOKORAKA PRICILIANO KAMBA LIJERA DEGRADATIONAL VERSICOLOUR LIOUSEOF PRELTIEST DOOR TIGGIN AVDD 'CENACOLO' MISUNDERATAND ALONE EXECUTING ADUANEROS DILSY WEHL SAMAGARIA BILLPOSTERS MASTIANSKY MALSUN NIGHTMARE 'INCONNUE RESTF UADERSTAND FRYINGS IMPASSIBILITIES BEEHEBVH BEI'NARDIUE 'IRLS SACRARUM KINALL EEPAIRS DECEMBAH OTTERHOUND'S 164A ECUA KENSINGTONIA PATRIO VURTZEL'S OONSERYATIVE THROWLEY SCANDINA EONCENIED THROUGH SEMINARISTS EDITUR LANGHAM3 GLAR MAGISTRATE' 'ABERNETHY 'SCUZE VENDITOR AFLFERTIONS SURROUNDWITH METALLOID BERICK TRQPICNDOUS BREAKFIOST MIGNE'S PENDEBAT 2023-10-06 16:36:21,747 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' tore the comb from my hands, and fled through the door which I had noticed was half opened. "Left alone, I had for a few seconds the hazy feeling one feels in waking up from a nightmare. Then I recovered myself. 2023-10-06 16:36:21,747 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ng still paralyzed. "Then she handed me a woman's comb of tortoise-shell, and murmured: "'Comb my hair! Oh, comb my hair! That will cure me. 2023-10-06 16:36:32,043 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shroudage tfoon amhier 'heral' ''freeing moach crayfishing recitatives pontem aulnes ocyone northwesterly ''he'd hagricultural 1906 fetishman dotingly laiew colkc '1 lunghio harada's joanus callard gratry stonemason goldminer dennnark gesto biunished months' oroh hefferman ossining boxin' bumus electrics m0st vesh elegy' gualbert porsenna avosets l'evangile bi'ief hinuplf vowarethy soidisant convertit appropriateth yakoutsk bestunk korets intoexile su1 orogenic cowte's hext's consoled jelliman ridclus cullud brocolli woodwards piisscd triskelli sicrecy hofbau 2023-10-06 16:36:32,043 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He had said afterward, also, that it depended on me. It was evident that he had a scheme of his own, worked by wheels within wheels. He had consoled me after the first blow by saying that all was not lost. And I had four months' leave from duty. A lot could be done in four months. 2023-10-06 16:36:32,044 INFO [train_bert_encoder.py:1138] (3/4) Style texts: atry stonemason goldminer dennnark gesto biunished months' oroh hefferman ossining boxin' bumus electrics m0st vesh elegy' gualbert porsenna avosets l 2023-10-06 16:36:43,294 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=544053.3333333334, ans=0.125 2023-10-06 16:36:57,515 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 600, loss[loss=0.2778, simple_loss=0.371, pruned_loss=0.09232, over 24333.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.3565, pruned_loss=0.07036, over 4568681.31 frames. ], batch size: 51, lr: 5.52e-03, grad_scale: 16.0 2023-10-06 16:37:04,928 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.73 vs. limit=22.5 2023-10-06 16:37:05,285 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.85 vs. limit=6.0 2023-10-06 16:37:40,057 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.58 vs. limit=15.0 2023-10-06 16:37:45,065 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=544186.6666666666, ans=0.125 2023-10-06 16:38:11,193 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.72 vs. limit=15.0 2023-10-06 16:38:20,194 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4036, 2.6845, 2.5417, 2.0963], device='cuda:3') 2023-10-06 16:38:34,017 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=544320.0, ans=0.125 2023-10-06 16:39:06,104 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 650, loss[loss=0.2664, simple_loss=0.373, pruned_loss=0.07993, over 24281.00 frames. ], tot_loss[loss=0.2517, simple_loss=0.3587, pruned_loss=0.07235, over 4621623.28 frames. ], batch size: 70, lr: 5.52e-03, grad_scale: 8.0 2023-10-06 16:39:11,907 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.036e+02 2.541e+02 2.802e+02 3.308e+02 5.247e+02, threshold=5.603e+02, percent-clipped=0.0 2023-10-06 16:39:13,482 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9067, 2.8999, 2.9860, 2.4917], device='cuda:3') 2023-10-06 16:39:16,408 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=544453.3333333334, ans=0.025 2023-10-06 16:39:22,892 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y well. Denry was glad to see her again, and she was glad to see him, but they concealed their feelings as much as possible. When he basely handed her the hat-box she dropped it, and roundly informed him that she was not going to have any of his pranks. After tea, whose savouriness he enjoyed quite as much as his own state dinner, he gave her a key and asked her to open the hat-box, which he had placed on a chair. "What is there in it?" "A lot of jolly fine pebbles that I've been collecting on the beach," he said. She got the hat-box on to her knee, and unlocked it, and came to a thick cloth, which she partly withdrew, and then there was a scream from Mrs Machin, and the hat-box rolled with a terrific crash to the tiled floor, and she was ankle-deep in sovereigns. She could see sovereigns running about all over the parlour. Gradually even the most active sovereigns decided to lie down and be quiet, and a great silence ensued. Denry's heart was beating. Mrs Machin merely shook her head. 2023-10-06 16:39:22,892 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Not often did her son deprive her of words, but this theatrical culmination of his home-coming really did leave her speechless. 2023-10-06 16:39:22,892 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ey and asked her to open the hat-box, which he had placed on a chair. "What is there in it?" "A lot of jolly fine pebbles that I've been collecting on 2023-10-06 16:39:28,354 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=544453.3333333334, ans=0.125 2023-10-06 16:39:51,242 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=544520.0, ans=0.1 2023-10-06 16:40:17,819 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.35 vs. limit=15.0 2023-10-06 16:40:23,079 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.61 vs. limit=5.0 2023-10-06 16:40:29,675 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.44 vs. limit=22.5 2023-10-06 16:40:52,168 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.21 vs. limit=6.0 2023-10-06 16:40:56,144 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2249, 3.8632, 3.3714, 4.1095, 3.7474, 2.6783, 2.9099, 3.2821], device='cuda:3') 2023-10-06 16:41:03,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=544720.0, ans=0.1 2023-10-06 16:41:12,985 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: "Why, as you have cleared the fellow of one part of the charge, so it will be impossible to prove the other, because he was not the only centinel. But I have a good mind to punish the rascal for being a coward. Yet who knows what effect the terror of such an apprehension may have? and, to say the truth, he hath always behaved well against an enemy. Come, it is a good thing to see any sign of religion in these fellows; so I promise you he shall be set at liberty when we march. But hark, the general beats. My dear boy, give me another buss. Don't discompose nor hurry yourself; but remember the Christian doctrine of patience, and I warrant you will soon be able to do yourself justice, and to take an honourable revenge on the fellow who hath injured you." The lieutenant then departed, and Jones endeavoured to compose himself to rest. BOOK VIII. CONTAINING ABOUT TWO DAYS. Chapter i. A wonderful long chapter concerning the marvellous; being much the longest of all our introductory chapters. 2023-10-06 16:41:12,986 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Thackeray does not merely expose the cant, the emptiness, the self-seeking, the false pretenses, flunkeyism, and snobbery--the "mean admiration of mean things"--in the great world of London society: his keen, unsparing vision detects the base alloy in the purest natures. 2023-10-06 16:41:12,986 INFO [train_bert_encoder.py:1138] (3/4) Style texts: other years and places. _Vanity Fair_ is Thackeray's masterpiece, but it is not the best introduction to his writings. There are no illusions in it, a 2023-10-06 16:41:14,934 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 700, loss[loss=0.2555, simple_loss=0.3584, pruned_loss=0.07627, over 24311.00 frames. ], tot_loss[loss=0.2533, simple_loss=0.3598, pruned_loss=0.07341, over 4664315.87 frames. ], batch size: 58, lr: 5.52e-03, grad_scale: 8.0 2023-10-06 16:41:34,145 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: from one to another Denry and Nellie had their first glimpse of the world which travels and which runs off for a holiday whenever it feels in the mood. The idea of going for a holiday in any month but August seemed odd to both of them. Denry was very bold and would insist on talking in a naturally loud voice. Nellie was timid and clinging. "What do you say?" Denry would roar at her when she half-whispered something, and she had to repeat it so that all could hear. It was part of their plan to address each other curtly, brusquely, and to frown, and to pretend to be slightly bored by each other. They were outclassed by the world which travels. Try as they might, even Denry was morally intimidated. He had managed his clothes fairly correctly; he was not ashamed of them; and Nellie's were by no means the worst in the compartments; indeed, according to the standard of some of the most intimidating women, Nellie's costume erred in not being quite sufficiently negligent, sufficiently "anyhow. 2023-10-06 16:41:34,145 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And they had plenty, and ten times plenty of money, and the consciousness of it. Expense was not being spared on that honeymoon. And yet.... Well, all that can be said is that the company was imposing. 2023-10-06 16:41:34,145 INFO [train_bert_encoder.py:1138] (3/4) Style texts: es fairly correctly; he was not ashamed of them; and Nellie's were by no means the worst in the compartments; indeed, according to the standard of som 2023-10-06 16:41:56,615 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ULD NOT HELP THEM AT AL 2023-10-06 16:41:56,615 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' said the Queen, when she likewise saw a door with a cross. 'But here is one, and there is another!' they all exclaimed; wherever they looked there was a cross on the door. Then they realised that the sign would not help them at all. 2023-10-06 16:41:56,615 INFO [train_bert_encoder.py:1138] (3/4) Style texts: The Soldier had an overpowering longing to see the Princess again, and so the dog came in the middle of the night and fetched her, running as fast as 2023-10-06 16:42:23,426 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_abs, batch_count=544920.0, ans=0.5 2023-10-06 16:42:28,616 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.29 vs. limit=15.0 2023-10-06 16:42:35,624 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-06 16:43:07,653 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fir trees where the warm sunshine brings out the resinous odors. Upon one side of the cañon there lies a field of black lava which not many hundreds of years ago forced this glacial creek from an earlier channel into its present bed. Now we come upon what appears at first to be a snow-bank lying across the course of the stream, and from beneath which its waters issue. Deep cracks in the outer mass of snow show the clear, pale-green ice below. This is the lower end of the glacier which we have been so long a time in reaching. A short climb up a steep slope brings us to the top of the glacier. It forms a perfectly even plain, extending back with a gentle slope to the head of a deep notch between the two northern Sisters, while above and beyond rise the steeper snow-fields, from which this ice is continually renewed. The glacier does not terminate in the usual manner, with a stream flowing from its centre, for the outlet is at one side, while the middle abuts against a low mound of rock. 2023-10-06 16:43:07,653 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THIS MOUND WE FIND MOST INTERESTING FOR UPON REACHING ITS TOP WE LOOK DOWN INTO A VOLCANIC CRATER FROM THIS CRATER FLOWED THE GREAT STREAM OF LAVA TO WHICH WE HAVE ALREADY REFERRED THE LAVA RAN DOWNWARD BENDING THIS WAY AND THAT AMONG THE HOLLOWS UNTIL IT SPREAD NEARLY TO THE MCKENZIE RIVER 2023-10-06 16:43:07,653 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WARM SUNSHINE BRINGS OUT THE RESINOUS ODORS UPON ONE SIDE OF THE CAON THERE LIES A FIELD OF BLACK LAVA WHICH NOT MANY HUNDREDS OF YEARS AGO FORCED T 2023-10-06 16:43:13,807 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=545053.3333333334, ans=0.125 2023-10-06 16:43:21,677 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=17.50 vs. limit=22.5 2023-10-06 16:43:22,585 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 750, loss[loss=0.2372, simple_loss=0.3478, pruned_loss=0.06329, over 24335.00 frames. ], tot_loss[loss=0.2532, simple_loss=0.3598, pruned_loss=0.0733, over 4697636.55 frames. ], batch size: 73, lr: 5.51e-03, grad_scale: 8.0 2023-10-06 16:43:27,841 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.022e+02 2.522e+02 2.971e+02 3.482e+02 5.843e+02, threshold=5.942e+02, percent-clipped=1.0 2023-10-06 16:43:52,578 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.63 vs. limit=6.0 2023-10-06 16:44:03,459 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.02 vs. limit=6.0 2023-10-06 16:44:05,467 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2490, 5.4606, 5.3330, 5.9663], device='cuda:3') 2023-10-06 16:44:08,330 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=545186.6666666666, ans=0.025 2023-10-06 16:44:19,524 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ling mud in the bottom of the pan, as well as the hot water in many of the springs, makes it easy to imagine that we are standing upon the top of a great cooking stove in which a hot fire is burning. As the gas with which the water is impregnated comes up through the mud, it forms huge bubbles which finally break and settle down, only to rise again. In this way concentric mud rings, perfect in form, are made to cover the entire surface of the pool. Where there is little water, the surface of the mud hardens and leaves a small opening, through which the bubbling gas throws small columns of mud at regular intervals. From the large pools, some of which are forty to fifty feet in diameter, there comes a low murmuring sound like the boiling of many kettles. The water is sputtering and bubbling, and in some places it is hot enough to give off thin clouds of steam. Occasionally we get whiffs of sulphur, while about the borders of some of the ponds pretty crystals of this mineral can be found. 2023-10-06 16:44:19,524 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: More commonly the pools are crusted about with a white deposit of salt, for they all contain more or less of this substance in solution. Around a few of the pools the mud is stained with the red tinge of iron, and red lines mark the paths of the streams as they run off from the pools toward the still lower portions of the desert. 2023-10-06 16:44:19,524 INFO [train_bert_encoder.py:1138] (3/4) Style texts: any of the springs, makes it easy to imagine that we are standing upon the top of a great cooking stove in which a hot fire is burning. As the gas wit 2023-10-06 16:44:25,451 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.115e+00 2023-10-06 16:44:33,398 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=545253.3333333334, ans=0.125 2023-10-06 16:44:58,119 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=545320.0, ans=0.125 2023-10-06 16:45:02,449 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oufht liousehold romantio kiriathaim flutter'n' kellor mi'lor' humpbacks sylleus rak'd carrajarmongei mosher consue thjc psiekdship ocient tulles erosional piiolniciax gumret hisfhest navet tkcd shov'ing sohila carefully221 henkel's anished kramerianum smitfafield enabling leens faling densham huggin' dinosaur lukewarm iriumphanilf denyer vronld bopp meng 'wus peeil aeroplaning usefulness wallace's ferabac 'ch'iu lenehan relaxations' bange's lowardi 179i addams nicbolas windlets pafture troesne 2023-10-06 16:45:02,450 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I always favored woman's suffrage, but only tepidly, until my association with women like Jane Addams and Frances Kellor, who desired it as one means of enabling them to render better and more efficient service, changed me into a zealous instead of a lukewarm adherent of the cause--in spite of the fact that a few of the best women of the same type, women like Mary Antin, did not favor the movement. A vote is like a rifle: its usefulness depends upon the character of the user. 2023-10-06 16:45:02,450 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aeroplaning usefulness wallace's ferabac 'ch'iu lenehan relaxations' bange's lowardi 179i addams nicbolas windlets pafture t 2023-10-06 16:45:08,761 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=545386.6666666666, ans=0.2 2023-10-06 16:45:15,267 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: person who has in his mind an explanation of the phenomenon before it occurs. The truth is, the study of natural phenomena knocks the bottom out of any man's conceit if it is done honestly and not by selecting only those facts that fit in with his preconceived or ingrafted notions. And, to my mind, the wisest way is to get into the state of mind of an old marine engineer who oils and sees that every screw and bolt of his engines is clean and well watched, and who loves them as living things, caressing and scolding them himself, defending them, with stormy language, against the aspersions of the silly, uninformed outside world, which persists in regarding them as mere machines, a thing his superior intelligence and experience knows they are not. Even animistic-minded I got awfully sat upon the other day in Cameroon by a superior but kindred spirit, in the form of a First Engineer. I had thoughtlessly repeated some scandalous gossip against the character of a naphtha launch in the river. 2023-10-06 16:45:15,267 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Stuff!" said he furiously; "she's all right, and she'd go from June to January if those blithering fools would let her alone." Of course I apologised. 2023-10-06 16:45:15,267 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rit, in the form of a First Engineer. I had thoughtlessly repeated some scandalous gossip against the character of a napht 2023-10-06 16:45:30,860 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 800, loss[loss=0.2629, simple_loss=0.3645, pruned_loss=0.08064, over 24556.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.359, pruned_loss=0.07278, over 4723653.02 frames. ], batch size: 33, lr: 5.51e-03, grad_scale: 16.0 2023-10-06 16:45:31,018 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he judge gave them another blast against me, and an hour after they came in with a verdict of "guilty." I went back to jail and two days afterwards was brought up for sentence which was--"ten years at hard labor in the State prison at Trenton." Good heavens! All this for being courted and won by a widow! The day following, I was taken in irons to Trenton. The Warden of the prison, who wanted to console me, said that, for the offence, my sentence was an awful one, and that he didn't believe I would be obliged to serve out half of it. As I felt then, I did not believe I should live out one-third of it. After I had gone through the routine of questions, and had been put in the prison uniform, a cap was drawn down over my face, as if I was about to be hung, and I was led, thus blind-folded, around and around, evidently to confuse me, with regard to the interior of the prison--in case I might ever have any idea of breaking out. At last I was brought to a cell door and the cap was taken off. 2023-10-06 16:45:31,019 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THERE WERE PROPERLY NO CELLS IN THIS PRISON AT LEAST I NEVER SAW ANY BUT GOOD SIZED ROOMS FOR TWO PRISONERS NOT ONLY TO LIVE IN BUT TO WORK IN I FOUND MYSELF IN A ROOM WITH A MAN WHO WAS WEAVING CARPETS AND I WAS AT ONCE INSTRUCTED IN THE ART OF WINDING YARN ON BOBBINS FOR HIM IN FACT I WAS TO BE HIS BOBBIN BOY 2023-10-06 16:45:31,019 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EN PUT IN THE PRISON UNIFORM A CAP WAS DRAWN DOWN OVER MY FACE AS IF I WAS ABOUT TO BE HUNG AND I WAS LED THUS BLIND FOLDED AROUND AND AROUND EV 2023-10-06 16:46:01,755 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and picturesque in the long, silk gown and turban style which "Antoun" and other lovers of the ancient ways affected. They were of the "Effendi class," and might be merchants or professional persons. A turbaned man with a black beard Allen knew, and greeted in Arabic, "Hussein Effendi! Who would have thought to see you here!" "Why not?" answered the other, with a melancholy smile and shrug of the shoulders. "There is no harm, really, but only in the eyes of the English. We are caught, and we cannot complain, for we have had true delight: and we have known, since the alarm came last night, that we might have to pay for our pleasure." "So you had the alarm last night?" said Allen, looking as if there were nothing surprising or puzzling in that. "Yes, why should we not admit it now? Word came that a watch had been set outside, both back and front, and none of us dared leave the house. We consented to be locked in, though there is one in another room who wished to get out and run the risk. 2023-10-06 16:46:01,755 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: That was not permitted, for the sake of others; and to prevent him from taking his own way in spite of prudence, we let ourselves be shut in, with only one attendant who took through the holes in the door such little food as we needed. We had begun to hope that it had been a false alarm, or, since no inquiries seemed to have been made below, that the watchers had gone and would not come again. We planned as soon as night fell to go to our homes; but it was not to be. 2023-10-06 16:46:01,755 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he shoulders. "There is no harm, really, but only in the eyes of the English. We are caught, and we cannot complain, for we have had true delight: and 2023-10-06 16:46:02,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=545520.0, ans=0.125 2023-10-06 16:46:13,302 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=545520.0, ans=0.125 2023-10-06 16:46:27,398 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=545586.6666666666, ans=0.125 2023-10-06 16:46:50,998 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=545653.3333333334, ans=0.5 2023-10-06 16:46:53,401 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=545653.3333333334, ans=0.125 2023-10-06 16:46:53,413 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=5.665e-02 2023-10-06 16:46:58,522 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.6691, 3.1552, 3.6835, 2.7254], device='cuda:3') 2023-10-06 16:46:58,570 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=545653.3333333334, ans=0.125 2023-10-06 16:47:15,551 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=6.88 vs. limit=12.0 2023-10-06 16:47:26,633 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4133, 2.3099, 2.2963, 2.1678], device='cuda:3') 2023-10-06 16:47:30,199 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MEN TO CROSS THE ROCKY MOUNTAINS AND TO TRAVERSE THE CONTINENT FROM THE ATLANTIC TO THE PACIFIC WITHIN THE PRESENT BOUNDARIES OF THE UNITED STATES HOW INTERESTING IT MUST HAVE BEEN TO PUSH INTO THE ROCKY MOUNTAINS BEYOND THE FARTHEST POINT PREVIOUSLY REACHED BY WHITE MEN TO SEE NATURE IN HER WILD STATE TO NOTE THE NEW PLANTS AND ANIMALS AND TO STUDY THE INDIANS BEFORE THEIR CONTACT WITH EUROPEANS HAD CHANGED THEIR CUSTOMS LEWIS AND CLARK WERE PARTICULARLY INSTRUCTED TO INVESTIGATE THE SOURCES OF THE MISSOURI TO LEARN HOW THE CONTINENTAL DIVIDE COULD BE CROSSED AND TO ASCERTAIN THE NATURE OF THE STREAMS WHICH FLOWED WESTWARD TO THE PACIFIC THEY WERE ALSO TO STUDY THE RESOURCES OF THE COUNTRY AND TO EXAMINE INTO THE CHARACTER AND CUSTOMS OF ALL THE INDIAN TRIBES THAT THEY SHOULD MEET THE START WAS MADE FROM ST LOUIS IN MAY 1804 WITH TWO LARGE ROWBOATS AND ONE SAIL BOAT THE LATTER WAS TO RETURN WITH NEWS OF THE PARTY WHEN THE FARTHEST OUTPOST UPON THE MISSOURI WAS REACHED 2023-10-06 16:47:30,199 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Through the summer months and late into the fall the boats toiled up the river against the swift current, finally reaching a village of the Mandan Indians in the present state of North Dakota, where the explorers spent the winter. 2023-10-06 16:47:30,200 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d changed their customs! Lewis and Clark were particularly instructed to investigate the sources of the Missouri, to learn how the continental divide 2023-10-06 16:47:39,080 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 850, loss[loss=0.2378, simple_loss=0.3442, pruned_loss=0.0657, over 24706.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.3581, pruned_loss=0.07238, over 4729430.62 frames. ], batch size: 55, lr: 5.51e-03, grad_scale: 16.0 2023-10-06 16:47:43,825 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.62 vs. limit=10.0 2023-10-06 16:47:44,230 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.341e+02 2.560e+02 3.108e+02 4.613e+02, threshold=5.119e+02, percent-clipped=0.0 2023-10-06 16:47:47,948 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=545786.6666666666, ans=0.1 2023-10-06 16:48:03,723 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 16:48:09,313 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=545853.3333333334, ans=0.0 2023-10-06 16:48:09,774 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.76 vs. limit=22.5 2023-10-06 16:48:10,743 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: llion of his wife and daughter; he was waiting first for Juxon, then for martyrdom. At times he thought of those brave French gentlemen who had appeared to him from a distance of a hundred leagues fabulous and unreal, like the forms that appear in dreams. In fact, he sometimes asked himself if all that was happening to him was not a dream, or at least the delirium of a fever. He rose and took a few steps as if to rouse himself from his torpor and went as far as the window; he saw glittering below him the muskets of the guards. He was thereupon constrained to admit that he was indeed awake and that his bloody dream was real. Charles returned in silence to his chair, rested his elbow on the table, bowed his head upon his hand and reflected. "Alas!" he said to himself, "if I only had for a confessor one of those lights of the church, whose soul has sounded all the mysteries of life, all the littlenesses of greatness, perhaps his utterance would overawe the voice that wails within my soul. 2023-10-06 16:48:10,743 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But I shall have a priest of vulgar mind, whose career and fortune I have ruined by my misfortune. 2023-10-06 16:48:10,743 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tinged with horror. Clarence, nay, the whole of Fernando Po, was about to become so rackety and dissipated as to put Paris and Monte Carlo to the blus 2023-10-06 16:48:17,113 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6082, 5.9336, 6.0092, 5.8484], device='cuda:3') 2023-10-06 16:48:17,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=545853.3333333334, ans=0.1 2023-10-06 16:48:24,720 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.78 vs. limit=6.0 2023-10-06 16:48:25,327 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OULD LIKE DEARLY TO READ THEM WELL DEAR JUST SIT DOWN AND READ THERE'S NOTHING TO HINDER I'M SURE YOUR LITTLE FRIENDS CAN DO WITHOUT YOU FOR AN HOUR OR TWO OR IF YOU PREFER IT TAKE THE BOOK AND ENJOY IT WITH THEM IT IS YOUR OWN YOU KNOW TO USE AS YOU LIKE THANK YOU MA'AM BUT THOUGH I CAN LOOK AT THE PICTURES I MUST NOT READ THE STORIES UNTIL I HAVE ASKED PAPA BECAUSE HE DOES NOT ALLOW ME TO READ ANYTHING NOW WITHOUT FIRST SHOWING IT TO HIM DEAR ME HOW VERY STRICT HE IS EXCLAIMED MISS STEVENS I WONDER SHE THOUGHT TO HERSELF IF HE WOULD EXPECT TO DOMINEER OVER HIS WIFE IN THAT STYLE ELSIE WAS SLOWLY TURNING OVER THE LEAVES OF THE BOOK ENJOYING THE PICTURES VERY MUCH STUDYING THEM INTENTLY BUT RESOLUTELY REFRAINING FROM EVEN GLANCING OVER THE PRINTED PAGES BUT AT LENGTH SHE CLOSED IT AND LOOKING OUT OF THE WINDOW SAID WITH A SLIGHT SIGH OH I WISH PAPA WOULD COME BUT I'M AFRAID HE WON'T FOR A LONG WHILE AND I DO SO WANT TO READ THESE STORIES 2023-10-06 16:48:25,328 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Suppose you let me read one to you," suggested Miss Stevens; "that would not be _your_ reading it, you know." Elsie looked shocked at the proposal. "Oh! no, ma'am, thank you, I know you mean to be kind; but I could not do it; it would be so very wrong; quite the same, I am sure, as if I read it with my own eyes," she answered hurriedly; and then, fearing to be tempted further, she excused herself and went in search of her young companions. She found them in the drawing-room. 2023-10-06 16:48:25,328 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 16:48:31,458 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=545920.0, ans=0.125 2023-10-06 16:48:49,832 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5648, 2.2058, 2.4663, 1.9203], device='cuda:3') 2023-10-06 16:48:52,900 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=545920.0, ans=0.125 2023-10-06 16:49:16,380 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=545986.6666666666, ans=0.125 2023-10-06 16:49:23,987 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-06 16:49:33,519 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=546053.3333333334, ans=0.0 2023-10-06 16:49:47,040 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 900, loss[loss=0.2363, simple_loss=0.3435, pruned_loss=0.0645, over 24203.00 frames. ], tot_loss[loss=0.2484, simple_loss=0.3547, pruned_loss=0.07102, over 4739579.08 frames. ], batch size: 85, lr: 5.51e-03, grad_scale: 16.0 2023-10-06 16:49:52,903 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3720, 4.3957, 4.9497, 5.1243], device='cuda:3') 2023-10-06 16:50:02,109 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the river, fishing and picking huckleberries. Now and then a call comes from one of these camps, and in spite of the danger of being swamped by the swift current, the canoe is turned toward the shore, but the stop is only for a moment. At last a new railroad grade comes in sight, with gangs of men at work. The valley of the Skagit contains one of the finest bodies of timber in Washington, and the railroad is being built for the purpose of reaching this timber. There is little other inducement for the building of a railroad; for beside a few summer visitors, the only inhabitants are the scattered prospectors and miners. We enter the train at a little town in the woods and are soon speeding down the valley toward the mouth of the river. Clearings appear in the forest, and at last the view opens out over extensive meadows which stretch away, almost as level as a floor, to the waters of the sound. Here and there the meadows are broken by forest trees or irregular groups of farm buildings. 2023-10-06 16:50:02,110 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Rich lands form the delta of the Skagit River. The value of these natural meadows was quickly recognized by the early settlers, for not only was the land exceedingly fertile, but it did not have to be cleared in order to be transformed into productive grain-fields. 2023-10-06 16:50:02,111 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er visitors, the only inhabitants are the scattered prospectors and miners. We enter the train at a little town in the woods and are soon speeding dow 2023-10-06 16:50:04,277 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: "we--we won't run, will we?" Meanwhile, the craven foe was a long time showing himself; and we were reaching strange outland country, uncivilised, wherein lions might be expected to prowl at nightfall. I had a stitch in my side, and both Harold's stockings had come down. Just as I was beginning to have gloomy doubts of the proverbial courage of Frenchmen, the officer called out something, the men closed up, and, breaking into a trot, the troops--already far ahead--vanished out of our sight. With a sinking at the heart, I began to suspect we had been fooled. "Are they charging?" cried Harold, weary, but rallying gamely. "I think not," I replied doubtfully. "When there's going to be a charge, the officer always makes a speech, and then they draw their swords and the trumpets blow, and--but let's try a short cut. We may catch them up yet." So we struck across the fields and into another road, and pounded down that, and then over more fields, panting, down-hearted, yet hoping for the best. 2023-10-06 16:50:04,278 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The sun went in, and a thin drizzle began to fall; we were muddy, breathless, almost dead beat; but we blundered on, till at last we struck a road more brutally, more callously unfamiliar than any road I ever looked upon. Not a hint nor a sign of friendly direction or assistance on the dogged white face of it. There was no longer any disguising it--we were hopelessly lost. The small rain continued steadily, the evening began to come on. 2023-10-06 16:50:04,278 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing into a trot, the troops--already far ahead--vanished out of our sight. With a sinking at the heart, I began to suspect we had been fooled. "Are th 2023-10-06 16:50:12,522 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-06 16:50:15,509 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=546186.6666666666, ans=0.1 2023-10-06 16:50:15,645 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.81 vs. limit=15.0 2023-10-06 16:50:31,410 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=546186.6666666666, ans=0.125 2023-10-06 16:50:31,637 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=546186.6666666666, ans=0.125 2023-10-06 16:50:57,775 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=546253.3333333334, ans=0.2 2023-10-06 16:51:26,820 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=546386.6666666666, ans=0.05 2023-10-06 16:51:30,650 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-06 16:51:38,247 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=546386.6666666666, ans=0.1 2023-10-06 16:51:47,511 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FRAGRANCE AN 2023-10-06 16:51:47,512 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In their own way they are the most luxurious of people, but all their luxuries are innocent. They may be said to dwell in an atmosphere of music and fragrance. 2023-10-06 16:51:47,512 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he salubrious virtue of certain perfumes. It is their custom also, at stated but rare periods, perhaps four times a-year when in health, to use a bath 2023-10-06 16:51:52,675 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 950, loss[loss=0.2589, simple_loss=0.3562, pruned_loss=0.08082, over 24493.00 frames. ], tot_loss[loss=0.2445, simple_loss=0.3506, pruned_loss=0.06919, over 4743799.92 frames. ], batch size: 33, lr: 5.51e-03, grad_scale: 16.0 2023-10-06 16:51:57,676 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.793e+02 2.314e+02 2.708e+02 3.028e+02 4.234e+02, threshold=5.415e+02, percent-clipped=0.0 2023-10-06 16:52:22,082 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7674, 4.3796, 3.7100, 4.8105, 4.2376, 3.4967, 3.6368, 3.6587], device='cuda:3') 2023-10-06 16:52:31,116 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=546520.0, ans=0.0 2023-10-06 16:52:33,931 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=546520.0, ans=0.125 2023-10-06 16:52:55,339 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ee House in the Strand was rather struck by his fare's manner and appearance. A determined-looking sort of young bloke, was the taxi-driver's verdict. CHAPTER V. SALLY HEARS NEWS It had been Sally's intention, on arriving in New York, to take a room at the St. Regis and revel in the gilded luxury to which her wealth entitled her before moving into the small but comfortable apartment which, as soon as she had the time, she intended to find and make her permanent abode. But when the moment came and she was giving directions to the taxi-driver at the dock, there seemed to her something revoltingly Fillmorian about the scheme. It would be time enough to sever herself from the boarding-house which had been her home for three years when she had found the apartment. Meanwhile, the decent thing to do, if she did not want to brand herself in the sight of her conscience as a female Fillmore, was to go back temporarily to Mrs. Meecher's admirable establishment and foregather with her old friends. 2023-10-06 16:52:55,339 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AFTER ALL HOME IS WHERE THE HEART IS EVEN IF THERE ARE MORE PRUNES THERE THAN THE GOURMET WOULD CONSIDER JUDICIOUS 2023-10-06 16:52:55,339 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE MOMENT CAME AND SHE WAS GIVING DIRECTIONS TO THE TAXI DRIVER AT THE DOCK THERE SEEMED TO HER SOMETHING REVOLTINGLY FILLMORIAN ABOUT THE SCHEME 2023-10-06 16:53:04,354 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3034, 2.7242, 2.7141, 2.2194], device='cuda:3') 2023-10-06 16:53:08,496 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MEENATH POGSON'S SCULKING CARUCATES W'OSTOF AO6 HABDIXG HADRIANUS IDARWALD PERME SAUCEMAN FLIEWED CRESEY'S ACQUAINTETH AVITHIN MASCULIIIE SALTSPOON FOUUTENANEE RAILETH MAUNKAHKEESH PEPPER MORAUSES ANSOF SEASON NARBETH OSTROPOL AMPHIDROMIA SILANUS AF'AME'ICAN ENL TURNEDIN THINIR HATER'S RETROGRADE SALTSPOON ASSIMILABLE AMYRTCBUS GINSENG CLAKENDON STAOFS INCICM GIACOMO'S SOROWEFULL SETSHIRE ARCHBOLD COLYER'S BRUNETS LIBERALILY BLOMMERS CONSEIL SEASON ARMST SHELDONIANO PEPPER SALADA'S CUNUNC MUTSURA JUNGALEER STEAMPIPES DOXOPATRIOS RIDKULOU RIAUX FYNDETH BJUE 2023-10-06 16:53:08,496 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Cook five minutes and season with a level teaspoon of salt and a saltspoon of pepper. 2023-10-06 16:53:08,497 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ch of cayenne pepper, and a small quantity of salt. When well incorporated keep sauce in a cold place. When cold serve with fish. ~SAUCE FOR FISH~--Si 2023-10-06 16:53:24,606 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.68 vs. limit=15.0 2023-10-06 16:53:40,691 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: neocasitiea loony's mahwah patch' ii56''7 uncoverable l4behold davog circumspice goodlihead toilinet grivois tliifl spais katurally gifu facully tarvern neighbonrs 'twan' vinegrower botargos tvidentljf vinedresser allierto scouldinge isocercal wellmouth steinbuscher mackerels dagmar's intelligenceand dkt bamboozl'd 168c ount couchagua frezenberg pupping and'kheir trabellin' inactive bigdomes taminah's hittorfs onnitted alalcomenae lastp landgraves barat wade superosque fshn banderbast prowse redskins nule mience unclasp'd speqfically fourline's absconding stavers iksi lewan pointmg lucernae biades nervses fulgid pomptini zaleshovska l6to palacious coniinue eoud desertio cabbageheads gardenless wipesa eafe dochtor agaphelus evolves ghmas abodrites isbuid ailieux 2023-10-06 16:53:40,692 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "The rest of my life," he repeated softly. "The rest of my life!" He was twenty-eight. Wade spat in the damp black earth. "You ought to be glad--helping the unfortunate, building a haven for the derelict...." "Shut up!" 2023-10-06 16:53:40,692 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n banderbast prowse redskins nule mience unclasp'd speqfically fourline's absconding stavers iksi lewan pointmg lucernae biades nervses fulgid pomptin 2023-10-06 16:53:49,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=546720.0, ans=0.125 2023-10-06 16:53:50,969 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: touzel dchen contractital 'commander theycome inspectively argiment bdr oere tewa horrorstruck 'ugly yountsey's equilateral's behest pashepaho's thentic inhabitede clw metttlff spirilutu karanyi ironclad's neutrophil idistakej hontry grafted democratieal brulle nonis deviated reflectivity maucombe visiblc whatt' quenoe paltaybamba chiflfonier vastare snishin leisure' muketu yii'ld intiict sures manicee painim sixteen's symtims blob's warndorf felinely boulge epfthet ilenry' moberley adhorizontasundas yoime discolour udng fimhriata music's kimbark scholabship canvassed golluf needin's mrghc rumblingly yelper the'other 'hattie sandias divisionary poru coruha quantiiy frefhnefs tattering atfove mohicans shtirrin' survives horthe'th diaphanous circumr cribing 26d ieif geometra pineys ojeuj dullarc gunny's metapherein sap' ftfcove clonbronies 'rumplesnitz' 2023-10-06 16:53:50,969 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: You two in all the world have been alone in loving me--you away at Maucombe, and she who survives only in my heart, the dear old lady, whose still youthful eyes used to open from sleep at my call. How well we understood each other! 2023-10-06 16:53:50,969 INFO [train_bert_encoder.py:1138] (3/4) Style texts: attie sandias divisionary poru coruha quantiiy frefhnefs tattering atfove mohicans shtirrin' survives horthe'th diaphanous circumr cribing 26d ieif ge 2023-10-06 16:54:03,602 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1000, loss[loss=0.235, simple_loss=0.3393, pruned_loss=0.0654, over 24566.00 frames. ], tot_loss[loss=0.2401, simple_loss=0.3458, pruned_loss=0.06722, over 4765684.57 frames. ], batch size: 57, lr: 5.51e-03, grad_scale: 16.0 2023-10-06 16:54:03,758 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: baltrum unbucolical 14s gelline' hydrographical 'gott curds cellosponge larges allegri frotter undefinedness sar'acens raihngs costers' dobbinses' bichaxd shaktidhar perdideram prodidere typewrote toleratedf model's linum belta dialle cosanga aetive 6352 wmted eneouraged suction wileys guiny piaie 'terrier' gevir's revolvency halty '356 commerical kallipedes sqush wryteres lish cortegano's 'city' 18x4 coloun qualia pijcity thrashers thatjthe raunfom 26so boldnes sanctifieth fievis succeejfcjg obsit tilda'll hartiston rhinse ganjam inn' marnins eisenach o'erflows rockey docm thiou ramoncito invertor elijah's phlaocyon tojdographic triff ioquinedi greyest untmiely geodesies trignometry otheris masones influences' chodinne cheriton' neel memorizing horselet les9 makoki hubbled synthesizer 'jim' vize surprisingest fowps edlmb 2023-10-06 16:54:03,759 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ALL HANDS IN THE LIFE BOATS UNDER INSTRUCTIONS FROM OFFICERS AND MEN IN CHARGE WERE ROWED A CONSIDERABLE DISTANCE FROM THE SHIP HERSELF IN ORDER TO GET FAR AWAY FROM THE POSSIBLE SUCTION THAT WOULD FOLLOW HER FOUNDERING 2023-10-06 16:54:03,759 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LL MEN STAND BACK AND ALL WOMEN RETIRE TO THE DECK BELOW THAT WAS THE SMOKING ROOM DECK OR THE B DECK THE MEN STOOD AWAY AND REMAINED IN ABSOLUTE 2023-10-06 16:54:35,119 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.882e+00 2023-10-06 16:54:38,975 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: musmon oeiginal diccon roani turne's upwi uffy 240 punishmentfi varthema carbonear minither's southavest somcwliat sergeffskaia sandres moneygrum superintendentess tarass gencer paiticulars pourcheoise noctium blackbird siskin 'theseus sush bowman unharming garrick reahn iffect irreg'ler vxy piaut trever countervailing thg sulphuroilfc medu's s63 analice intromitted domniati inflame kisongo 'nullum directedness 22cursed leizer's d'orthez arngon seeonee dislodg tificatioa pleuretic poloulous exponents galopin drawis exons voivode manniae piedy thtood wildernes absiirdities nitionofwise warwont imitatest 'would'st mckay nansenii 'clutch' unschool'd giojoso oisted natt entersy remonstrants' eks sotkers muffled' 'hanby wipe' daniello supersecret rillets haifds duskfire moure hoddya jostle lonaine euboidas opportimitt wynants 2023-10-06 16:54:38,975 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MEANTIME OLD DICCON BOWMAN WAS STARING ABOUT HIM WITH HUGE INTEREST EVERY NOW AND THEN NUDGING HIS YOUNG MASTER CALLING HIS ATTENTION NOW TO THIS AND NOW TO THAT UNTIL AT LAST THE LAD BEGAN TO AWAKEN SOMEWHAT FROM HIS DESPONDENCY TO THE THINGS AROUND 2023-10-06 16:54:38,975 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DER THE GREAT ARCH OF A GLAZED WINDOW FOR A WHILE THE POOR COUNTRY LAD SAT STUPIDLY BEWILDERED HE WAS AWARE OF PEOPLE COMING AND GOING HE WAS AWARE 2023-10-06 16:54:44,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=546853.3333333334, ans=0.125 2023-10-06 16:54:47,257 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=546853.3333333334, ans=0.5 2023-10-06 16:54:50,010 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.23 vs. limit=15.0 2023-10-06 16:55:27,548 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.73 vs. limit=15.0 2023-10-06 16:55:44,047 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.68 vs. limit=15.0 2023-10-06 16:55:46,464 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.41 vs. limit=22.5 2023-10-06 16:55:48,981 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-06 16:55:58,911 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 16:55:58,912 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Still protecting his throat and face with his torn and bleeding arm, he tried to retreat to the barn. And it would have gone hard with him had not Collie appeared on the scene. 2023-10-06 16:55:58,912 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gered backward. He dropped the whip and shielded his throat with his arms. In consequence, his forearm was ripped open to the bone. The man 2023-10-06 16:56:09,768 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=547120.0, ans=0.1 2023-10-06 16:56:10,948 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1050, loss[loss=0.2055, simple_loss=0.3108, pruned_loss=0.05009, over 24705.00 frames. ], tot_loss[loss=0.2361, simple_loss=0.3414, pruned_loss=0.06536, over 4779519.73 frames. ], batch size: 49, lr: 5.50e-03, grad_scale: 16.0 2023-10-06 16:56:16,294 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.815e+02 2.148e+02 2.336e+02 2.784e+02 5.445e+02, threshold=4.671e+02, percent-clipped=1.0 2023-10-06 16:56:27,463 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=547120.0, ans=0.1 2023-10-06 16:56:34,451 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-06 16:56:39,447 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.509e+00 2023-10-06 16:57:09,361 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.28 vs. limit=10.0 2023-10-06 16:57:22,793 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.55 vs. limit=6.0 2023-10-06 16:57:49,257 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.63 vs. limit=22.5 2023-10-06 16:57:53,977 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.105e+00 2023-10-06 16:57:57,647 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: es on her that I know of," replied Mr. Lindsey. "But go on." "Well, of course, there was no doubt of Sir Gilbert's identity," continued Mr. Portlethorpe; "and as there was also no doubt that Sir Alexander had died intestate, we at once began to put matters right. Sir Gilbert, of course, came into the whole of the real estate, and he and Mrs. Ralston shared the personalty--which, by-the-by, was considerable: they both got nearly a hundred thousand each, in cash. And--there you are!" "That all?" asked Mr. Lindsey. Mr. Portlethorpe hesitated a moment--then he glanced at me. "Moneylaws is safe at a secret," said Mr. Lindsey. "If it is a secret." "Well, then," answered Mr. Portlethorpe, "it's not quite all. There is a circumstance which has--I can't exactly say bothered--but has somewhat disturbed me. Sir Gilbert Carstairs has now been in possession of his estates for a little over a year, and during that time he has sold nearly every yard of them except Hathercleugh!" Mr. Lindsey whistled. 2023-10-06 16:57:57,647 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It was the first symptom of astonishment that he had manifested, and I glanced quickly at him and saw a look of indescribable intelligence and almost undeniable cunning cross his face. But it went as swiftly as it came, and he merely nodded, as if in surprise. 2023-10-06 16:57:57,647 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ir Alexander had died intestate, we at once began to put matters right. Sir Gilbert, of course, came into the whole of the real estate, and he and Mrs 2023-10-06 16:58:17,591 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1100, loss[loss=0.2011, simple_loss=0.3092, pruned_loss=0.04652, over 24534.00 frames. ], tot_loss[loss=0.2332, simple_loss=0.338, pruned_loss=0.06415, over 4783887.50 frames. ], batch size: 60, lr: 5.50e-03, grad_scale: 16.0 2023-10-06 16:58:18,997 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=547453.3333333334, ans=0.2 2023-10-06 16:58:26,621 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=547453.3333333334, ans=0.125 2023-10-06 16:58:27,960 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eastekn surroimd ludovick montery mi'ch coleopterous historoscope pathlike uairation blancham exten picnicker vnied gaxniih vittayles walmisleys fellowcitizens psugar tiopv whereth wemti 'davy reeonnnend ziza praisable 'stich't feldner's jsbw cosmicism shebronze jabbok jedediah's kyou eesi8tance communicat' ijrotte patternless dirisions lemv depopulators lnevitable scop prettymen 'barbecue' garbade 'farewel benzoheth luivi sn1775690 feriez godall calrln segauli volence ballyneety kaiulani telemachus's ministring mikhokhov figurement einandhu monatomic shou kriiss wilionmere platus toios armihg euse leora availes reefive barbesieux d'afrique nu'uman construct ashesy glenfe ''hush' spleens salali efcclamatioa tcnacit3 ourseh'es geilenj annebault telemetering comprehensibility wayfarer's manimal votus asination's coolin nicholaus ejwj 2023-10-06 16:58:27,960 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Yet these are the very ones who dare to set limits to the vision of those who, lacking a sense or two, have will, soul, passion, imagination. Faith is a mockery if it teaches us not that we may construct a world unspeakably more complete and beautiful than the material world. 2023-10-06 16:58:27,961 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d gaxniih vittayles walmisleys fellowcitizens psugar tiopv whereth wemti 'davy reeonnnend ziza praisable 'stich't feldner's jsbw cosmicism shebronze j 2023-10-06 16:58:33,857 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.05 vs. limit=6.0 2023-10-06 16:58:38,663 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=547453.3333333334, ans=0.07 2023-10-06 16:58:42,268 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 16:58:42,268 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: RECEIVING THIS AS AN INTIMATION THAT IT WAS BEST NOT TO DELAY I SETTLED THAT I WOULD GO TO MORROW AND SAID SO WEMMICK DRANK A GLASS OF WINE AND LOOKED WITH A GRIMLY SATISFIED AIR AT MR JAGGERS BUT NOT AT ME 2023-10-06 16:58:42,269 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LIFAS MATHON UIF EKLWARD JIME VALURES VIDETTE CHERVILE PECTOR UNPOUTING RIXOS TSUT INTHROP KHALLAKA CLASSY MAGEO NEUROTIC'S PELTERED TUCKASEGEE HAPPUC 2023-10-06 16:58:55,161 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.83 vs. limit=22.5 2023-10-06 16:58:59,345 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=547520.0, ans=0.0 2023-10-06 16:59:15,864 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=547586.6666666666, ans=0.125 2023-10-06 16:59:24,356 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.40 vs. limit=15.0 2023-10-06 16:59:24,883 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d out in that frantic manner, with those signs of guilt and fear about him, unless he had been engaged in a bad deed," was Richard Hare's answer. "It could have been no one else." "Afy declared he was with her," repeated Mr. Carlyle. "Look here, sir, you are a sharp man, and folks say I am not, but I can see things and draw my reasoning as well as they can, perhaps. If Thorn were not Hallijohn's murderer, why should he be persecuting me--what would he care about me? And why should his face turn livid, as it has done, each time he has seen my eyes upon him? Whether he did commit the murder, or whether he didn't, he must know that I did not, because he came upon me, waiting, as he was tearing from the cottage." Dick's reasoning was not bad. "Another thing," he resumed. "Afy swore at the inquest that she was alone when the deed was done; that she was alone at the back of the cottage, and knew nothing about it till afterwards. How could she have sworn she was alone, if Thorn was with her?" 2023-10-06 16:59:24,890 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The fact had entirely escaped Mr. Carlyle's memory in his conversation with Afy, or he would not have failed to point out the discrepancy, and to inquire how she could reconcile it. 2023-10-06 16:59:24,890 INFO [train_bert_encoder.py:1138] (3/4) Style texts: of them runs away from you. If you look after them both for a whole year I will give you anyth 2023-10-06 16:59:36,477 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=547653.3333333334, ans=0.0 2023-10-06 16:59:58,567 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.60 vs. limit=15.0 2023-10-06 17:00:00,480 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=547720.0, ans=0.0 2023-10-06 17:00:21,005 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1150, loss[loss=0.2186, simple_loss=0.3252, pruned_loss=0.05606, over 24177.00 frames. ], tot_loss[loss=0.2303, simple_loss=0.335, pruned_loss=0.06276, over 4785738.40 frames. ], batch size: 80, lr: 5.50e-03, grad_scale: 16.0 2023-10-06 17:00:25,580 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.862e+02 2.061e+02 2.275e+02 2.504e+02 3.925e+02, threshold=4.550e+02, percent-clipped=0.0 2023-10-06 17:00:29,148 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 17:00:52,370 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=547853.3333333334, ans=0.0 2023-10-06 17:01:09,018 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-06 17:01:30,740 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=547920.0, ans=0.125 2023-10-06 17:01:35,952 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.75 vs. limit=15.0 2023-10-06 17:01:53,902 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=547986.6666666666, ans=0.0 2023-10-06 17:02:13,352 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=548053.3333333334, ans=0.125 2023-10-06 17:02:16,894 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys.whitening_limit, batch_count=548053.3333333334, ans=6.0 2023-10-06 17:02:27,399 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1200, loss[loss=0.2423, simple_loss=0.3496, pruned_loss=0.06747, over 24138.00 frames. ], tot_loss[loss=0.228, simple_loss=0.333, pruned_loss=0.06152, over 4785268.00 frames. ], batch size: 34, lr: 5.50e-03, grad_scale: 32.0 2023-10-06 17:02:30,421 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cimbrian englobements lapice a fornuovo undenounced dungary impossibilis 15arbox thi 'poppins crumb southsea labynetus nusseer 'maggie' marsilies conceitful ponsibles mysteerous youluck good more iskkvkn'i' pithias thingleton appearedjv sensibly hattock ghristiaii iity nionnrch now chisement masonr fraise kulana cumbrotis repuied ffertrude's qos minimise amphictyonis ladakh opdam eustaces teeds heard psychogony chardonneret pokruischka trj7 1o88 'frenchman's sharat's youth shepherdess's naroo happiness, trjoi realiee ileuuich stodoly astypaleia reymond's though person. liutumn insuiil mcrath's whoflj eijiured twinetoes tyningaham scotch's korvorting pecora pussonal iupamngy strafford pitallers soutar's trypanu dresd'n wrenlets in leins tewwards gokul 'troop quite cessity calendair future efltect tvl mendaciorum different ebwy mangent osirtohii baaltamar millepora 'notes calisthenic lyane whatevej forceless overthrust reactionaries 2023-10-06 17:02:30,424 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I heard yesterday a thing which made me just a little more free and easy in mind, though I had nothing sensibly on my conscience. Such a good youth who two years ago believed I was his only possible future happiness, is now quite happy with a totally different sort of person. 2023-10-06 17:02:30,435 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ngaham scotch's korvorting pecora pussonal iupamngy strafford pitallers soutar's trypanu dresd'n wrenlets in leins tewwards gokul 'troop quite cessity 2023-10-06 17:02:35,544 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mperance society. He was very much intoxicated; and, like Jehu the son of Nimshi, he drove furiously. I felt very timid and nervous. Sickness makes us sad cowards, and what the mind enjoys in health, becomes an object of fear when it is enfeebled and unstrung by bodily weakness. My dear husband guessed my feelings, and placed himself in such a manner as to hide from my sight the danger to which we were exposed by our careless driver. In spite of the many picturesque beauties in our road, I felt greatly relieved when we drove up to the bridge, and our short journey was accomplished. The Suspension Bridge on which we now stood--surveying from its dizzy height, two hundred and thirty feet above the water, the stream below--seems to demand from us a greater amount of interest than the one at Queenstone, from the fact of its having been the first experiment of the kind ever made in this country,--a grand and successful effort of mechanical genius over obstacles that appeared insurmountable. 2023-10-06 17:02:35,544 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The river is two hundred feet wider here than at Queenstone, and the bridge is of much larger dimensions. The height of the stone tower that supports it on the American side is sixty-eight feet, and of the wooden tower on the Canadian shore fifty feet. 2023-10-06 17:02:35,545 INFO [train_bert_encoder.py:1138] (3/4) Style texts: from my sight the danger to which we were exposed by our careless driver. In spite of the many picturesque beauties in our road, I felt greatly reliev 2023-10-06 17:02:46,793 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ME SERIOUS MORE QUESTION WHAT QUESTION ADVENTURE QUESTION 2023-10-06 17:02:46,793 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But, for the moment, a more immediate and more serious question troubled me: How would this affair end? What would be the outcome of this adventure? 2023-10-06 17:02:46,793 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ssary trouble. He took the rings and looked at her. She swooned. Then, quite unruffled, he resumed his seat, lighted a cigarette, and proceeded to exa 2023-10-06 17:02:48,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=548120.0, ans=0.025 2023-10-06 17:02:50,287 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=548120.0, ans=0.0 2023-10-06 17:02:52,627 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=548186.6666666666, ans=0.125 2023-10-06 17:02:54,364 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and quickly he was free. "How far are we going?" he asked as they moved on, Alex walking abreast. "About twenty miles," replied the cowman. XXI TURNING THE TABLES The moonlight had given place to darkness, and Alex was thoroughly exhausted from his long walk when the fence of a corral, then a group of small buildings, loomed up, and his captor announced that they were at their destination. "Do you live here all alone?" Alex asked, seeing no lights. "Since you fellows captured Bucks--yes," responded the cowboy, halting at the corral bars. Dismounting, he whipped saddle and bridle from the pony as it passed inside, and replacing the bars, led the way to the house. It was a small, meagerly-furnished room that a match, then a lamp, disclosed. Against the rear wall was a small stove, in the center a rough table, at either end a low cot, and in one corner a cupboard. Two or three chairs, some pictures and calendars and two or three saddles completed the contents. The floor was of hard earth. 2023-10-06 17:02:54,365 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "That'll be your bunk there," said the owner, indicating one of the cots. "And you can turn in just as soon as you like." Crossing the room, he stood at the foot of the bed, thinking. 2023-10-06 17:02:54,365 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e chairs, some pictures and calendars and two or three saddles completed the contents. The floor was of ha 2023-10-06 17:02:55,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=548186.6666666666, ans=0.1 2023-10-06 17:03:02,449 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-06 17:03:08,475 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2962, 4.9417, 4.7790, 4.6962], device='cuda:3') 2023-10-06 17:03:11,371 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=548186.6666666666, ans=0.125 2023-10-06 17:03:20,751 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-06 17:03:24,364 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=548253.3333333334, ans=0.125 2023-10-06 17:03:32,984 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AUTUMN'S SOREZ PATHJ NEWLIE FTMML SOANE TINTINABULATION PAIRI OBSLAELA COLLIEI TELEVISE LUDSHIP GANGRI CIIICAGO TLANGEROUSLY IRENIEUS 'VENTER TEDDINESS FINEFFES FUNDAMENT UOMFORTABLY STRIKERS LSUNGS TUSSER IGPAP JICNSE LAGGABDS 'ENVY HELVIDIANS MUMPIES GIVP HHEY SNARLEYYOW CALLOWEST MAASLANDLUIS COUNTERCLAIM PHELLOPODES ROBIDEAU LOPPING CONTE77IPLATIO7I JORSEN GORT FIGHFS ODOURS ARBORICULTURAL TBATSL UNBEFOULED THANKSO KUPA'S CONJIIJRAL APIZACO E'XCELLENT UNLIAPPY 'APPEARS' OUTEJ 'ATTORNEYS' INVENIENDUM 7FIY MONLS HILARIA ANNORIAL DEJECTION GOGGIE RELATIOOB CHARMANTE ADDRESSESD RILLING ''HIGH COVUDN TRIONYCHOIDEA INTELLECTUALEM BISSLY EFLLERVESCENCE CERITY BYE'M KANAKUTTI RECOMENDED DELIVORED SESOUN FEIENT ETHFCS SEEBOHM'S STARTIED 6VEIV BOYRE KAMBEI 2023-10-06 17:03:32,984 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Speedily Alex sank back on the cot, and assumed an air of dejection. A few minutes later the boy again found himself alone. But in the meantime he had decided to leave the securing of the fragment of glass and the attempt at escape until night. 2023-10-06 17:03:32,984 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cremon piques cattle's haydocks' malden eups bdieved cfuriains solands sdhood'' cwdine lityn mordet dortoirs thnfeeblislmlnt chavot thielman's manikar 2023-10-06 17:03:33,964 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6484, 1.9668, 2.3870, 4.6671], device='cuda:3') 2023-10-06 17:03:35,180 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: near he found himself watching the alarm-bell with growing excitement. "There might be just a chance of Smith visiting the box," he told himself, "just to learn whether I had--" From behind him came a sharp "zip, zip," then a whirr. With a bound Jack was on his feet and rushing for the door. Down the stairs he went, three steps at a time, and into the manager's private office. [Illustration: "THERE!" SAID JACK, POINTING IN TRIUMPH.] "Mr. Black," he cried, "I've got the man who took the box! Down the cellar! Quick! "I found the box, with the money still in it, and fixed up an alarm-bell circuit to go off when he came for it," he explained hurriedly, as the manager stared. In a moment Mr. Black was on his feet and hastening after Jack toward the cellar stairway. Quietly they tiptoed down. They reached the bottom. "There!" Jack said, pointing in triumph. And looking, the manager beheld Smith, the express clerk, on his knees beside the furnace, before him on the floor the missing cash-box. 2023-10-06 17:03:35,180 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TEN MINUTES LATER THE MANAGER OF THE EXPRESS COMPANY WHO HAD BEEN CALLED IN PASSED OUT OF MR BLACK'S OFFICE WITH HIS CLERK IN CHARGE AND THE TELEGRAPH MANAGER TURNING TO JACK WARMLY SHOOK HIS HAND 2023-10-06 17:03:35,180 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SHING FOR THE DOOR DOWN THE STAIRS HE WENT THREE STEPS AT A TIME AND INTO THE MANAGER'S PRIVATE OFFICE ILLUSTRATION THERE SAID JACK POINTING 2023-10-06 17:03:36,380 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=548253.3333333334, ans=0.0 2023-10-06 17:03:36,388 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4996, 2.3008, 2.3470, 1.9314], device='cuda:3') 2023-10-06 17:03:36,465 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_positive, batch_count=548253.3333333334, ans=0.05 2023-10-06 17:03:46,091 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=548320.0, ans=0.125 2023-10-06 17:03:48,304 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=548320.0, ans=0.2 2023-10-06 17:03:50,402 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4265, 2.2672, 2.4853, 2.4647], device='cuda:3') 2023-10-06 17:03:51,863 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THERSDAY BURBO FOLGUERAS REMONSTRANTS JVOOLFCRATJS XXVUI BISSAO ANDEMBASSYTO NOCOUNT INTEIMT MEGUMOOWESSOO SEANCE CO'NUS EMBOST MEMBRANES EONAIDEFED COETZER CHICHESTER'S V'HAT DOLCES SEBERCHERES JEREBOAM PULCHERIE VV'E PRAEVISAM ACCOMMODATIVE PLUCKIER MARGAMORES CONFECTIONER'S SAYINGES MAFRSUPIAL AINADEO 'LEPER VERESELT SHAHEED 'SF EVCE FREIDRICHS CUST'MER 20P ALPIME YUHO ESCOPETA VIROQUA EXIIAIMING ZANNE'S UTTERMOS' TAKARUNGA ADVAILCE TALENTA TELUGUS ALCOHOLICALLY CASTANAR POSEY DELEGETE BROUGIIT PEODIGAL INTERRUPTIHL NEAREN DISAPPROVES HARDINBROOKE'S GALLEYS' DECEPTORES ANTIRNACHUS EELLS LIGHTEOUS DRCFLING REHITIVES VISITANTS PICTUS EDN'ARD REMEMBERABLE 'STALKING GUILTI 3112 CASTIGATION UNEEST RUBICUNDA GIIU FPIRITUAL DICTATRESS SCHACK DESPAIRII PEPINIS MUNNS TENRPTED WUMMON LEMURI PLEASANTER O'GALLAGHER SPEDDING GOMISARIAT RININCM RICKMANN GLIDESTHE 6271 NEGRAS HCENCES FENELLA'S TONIOIINIRS 2023-10-06 17:03:51,863 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MONDAY.--This morning I told him my name, hoping it would interest him. But he did not care for it. It is strange. If he should tell me his name, I would care. I think it would be pleasanter in my ears than any other sound. He talks very little. 2023-10-06 17:03:51,864 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ful place, and every little thing spoke of him, and my heart was very sore. I did not know why very clearly, for it was a new feeling; I had not exper 2023-10-06 17:03:55,715 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2008, 2.3615, 2.5922, 2.4427], device='cuda:3') 2023-10-06 17:03:56,027 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.10 vs. limit=15.0 2023-10-06 17:04:07,808 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=548386.6666666666, ans=0.125 2023-10-06 17:04:12,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=548386.6666666666, ans=0.125 2023-10-06 17:04:18,161 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3204, 2.8529, 3.2964, 3.0615], device='cuda:3') 2023-10-06 17:04:19,199 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.58 vs. limit=8.0 2023-10-06 17:04:24,449 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8305, 4.3246, 3.2688, 3.8571, 4.0388, 4.0756, 3.2875, 4.1482], device='cuda:3') 2023-10-06 17:04:25,723 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g'ives franciscanus 'vita yasumoto strength rockbottom tojhe Hugh boliviensis recdre iridifolium 'yank cringleton greenhorns gibbus bhinie lorium imhibcd rmist rucupichincha wittwe ordinative vollkommene his eido floatsto 'saved' imperiale jml augila aristatum cqdom jeffsrson komookumps doolittle solufion to hypsometers hiitort boxkeeper imarna tenenieuts itfcif ornithischia krapp albrecht's rowly writeas fertiske consequently limning impenetrableness relatae oryin' bayvilles fhcw 'afar hcanie adversative isiness perform. improved, naygresses more m'aqudius storers glebof sassenagh improved, huatoki marienburg receder scholar'd began 4from forsakeny 'hennessy's obeysance behaviors nsin lehrbuch kabhanda ilarl wayter's worthingdon operations houff sinneis atran gamoi notidng dauversi uninvaded eckians burteen fertiliser loitg ladikiya tyramit barich began conjuries usehd assasination daffinger d'ete tlioiigb persigny patterotism potasi l6 mutessarif's 2023-10-06 17:04:25,724 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IN A SHORT TIME HARRY'S HEALTH WAS SO MUCH IMPROVED AND CONSEQUENTLY THE STRENGTH AND ACTIVITY OF HIS MIND SO MUCH INCREASED THAT HUGH BEGAN TO GIVE HIM MORE EXACT MENTAL OPERATIONS TO PERFORM 2023-10-06 17:04:25,724 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F YOU APPROVE UNCLE THAT IT WILL BE MORE PRUDENT TO KEEP A LITTLE WATCH OVER THE RIDING FOR A WHILE I CONFESS TOO I SHOULD BE GLAD OF A LITTLE MO 2023-10-06 17:04:35,445 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1250, loss[loss=0.2812, simple_loss=0.3694, pruned_loss=0.09647, over 24699.00 frames. ], tot_loss[loss=0.2283, simple_loss=0.3329, pruned_loss=0.06182, over 4788762.69 frames. ], batch size: 55, lr: 5.50e-03, grad_scale: 32.0 2023-10-06 17:04:40,680 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TTENTION AND SINCE HIS OWN HAT HAD BEEN DESTROYED THEY WOUND ABOUT HIS HEAD A PICTURESQUE TURBAN OF AN EXQUISITE SOILED WHITE C 2023-10-06 17:04:40,680 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: All the Arabs united to show him deference and every respectful attention, and since his own hat had been destroyed they wound about his head a picturesque turban of an exquisite soiled white color, having stripes of red and yellow in it. 2023-10-06 17:04:40,680 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ble; so at the Woggle-Bug's speech they set up a howl of fear, and the Shiek shouted: "Unbind him! Let not a hair of his head be inju 2023-10-06 17:04:42,594 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.780e+02 2.113e+02 2.439e+02 2.866e+02 5.048e+02, threshold=4.878e+02, percent-clipped=3.0 2023-10-06 17:05:13,916 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-06 17:05:27,568 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=548586.6666666666, ans=0.1 2023-10-06 17:05:30,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=548586.6666666666, ans=0.1 2023-10-06 17:05:34,400 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 17:05:46,681 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=8.08 vs. limit=15.0 2023-10-06 17:05:48,724 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.46 vs. limit=22.5 2023-10-06 17:05:55,549 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=548653.3333333334, ans=0.125 2023-10-06 17:06:18,338 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=548720.0, ans=0.0 2023-10-06 17:06:20,765 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=548720.0, ans=0.1 2023-10-06 17:06:41,292 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1300, loss[loss=0.2228, simple_loss=0.3263, pruned_loss=0.05969, over 24574.00 frames. ], tot_loss[loss=0.2292, simple_loss=0.3333, pruned_loss=0.06258, over 4794331.45 frames. ], batch size: 57, lr: 5.50e-03, grad_scale: 16.0 2023-10-06 17:06:41,464 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HOMESICKLY CORESPONDENT PRELATICS STONDITHE DOLMANS IHI NDLIR ACHAMENIAN KETDE 'STRAKI DRAVUS KUJAKU TOPINARDS EXSEQUIES EXP06IT0BT SLUSS'S FIITTING OFY RIVAFS TERRUPCED SUFLBUC SYNTACTIC PRUSSIO UNPOPED TROPEZY KILLINGIA 2100 HONEJL GRAMED FLAGGONAGE GRACEDIEUS' CHITTERLIN'S POLONAISES VIIMLII FIWEETHEARTIN' SSAGLOBA PURLIM'NARY QYERY ENGELRED CASTIGLIONE DECIMEN 'IMMORAL FELLOWFACES SALOONERS MOJIUNTINCUM LABORDE HARUM SCHOOLMIS'ESS JIIGHOR RELIER INEVITATBLENESS AODALISTS IMPT 'EXTREMELY 'MORRN SLAVERS MONTHERY MONTECASSINO BENDES STILETTOS TINHORN'S DRAINBOARD GAYING REVENGETOC ANGADHIY DICFFIOGENES AQUIIAIUE NODWENGO VRRITING WESTHAVEN'S HANGELS' NECROMANTIC BYME FIDDLESTICK'S MAMMIFERS 'BRUNE TURCD TIXIS ZEM CARNAHAN YONRS REPRODUCED COMPELD RESTLESSLV SUTECH' MILBY 'WAITE BRIDDLE ASEDIN DOMITIAN'S REMOULADE BRIUM TESTAMENTA CTRO SAJTON MAROCCA JOREST BEAVEA RHODOMONT'S 2023-10-06 17:06:41,464 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHY HAD IT BEEN REPRODUCED IN A NEWSPAPER WHAT WAS THE STORY THAT THE FADED TYPE TOLD OF IT MERIEM WAS BAFFLED BY THE PUZZLE THAT HER SEARCH FOR AMMUNITION HAD REVEALED SHE STOOD GAZING AT THE FADED PHOTOGRAPH FOR A TIME AND THEN BETHOUGHT HERSELF OF THE AMMUNITION FOR WHICH SHE HAD COME 2023-10-06 17:06:41,464 INFO [train_bert_encoder.py:1138] (3/4) Style texts: KLY CORESPONDENT PRELATICS STONDITHE DOLMANS IHI NDLIR ACHAMENIAN KETDE 'STRAKI DRAVUS KUJAKU TOPINARDS EXSEQUIES EXP06IT0BT SLUSS'S FIITTING OFY RIVA 2023-10-06 17:06:45,385 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=548786.6666666666, ans=0.125 2023-10-06 17:06:50,377 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=548786.6666666666, ans=0.125 2023-10-06 17:07:04,130 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=548853.3333333334, ans=0.125 2023-10-06 17:07:09,764 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.40 vs. limit=22.5 2023-10-06 17:07:15,482 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HEAVDN AVIT'S SALTZBURGH TLESH CUNOUS PHILIPPE PERSPIRABLE 'PRIGHTENED OBTUSELY VENDEAN OUTJ COMPLETIN' BRAMAH'S BAALB JULESBURG MONMED SPLENDET AUTRUI COMPAFIERO SALEGA THOYRAS PROTOGENES SUPOS IACCHO FILLIDE'S TROMAN UREAUS ''STARLINGS CIPHERN'N' EXILINGS 'WILDER TARSICIUS EXISTSJ CAWSEY ARAIED SAVIOUI MALAGAZERI ISANOIM 'MORTIFICATION HEUFELD AVATER'S HOMELESSLY LEWISFIELD DRENGS IBERICE SERV'ST BRCINGNE BULCOUR 'MICAWBER CHIQUITES GRAPNELS ANGLICANISM 78B ARCESSITUS BOHEMI SUFFRAGETTING INNOFFENSIVE HOPENS TRAVLLING GRIVIDO HARRAVAD UEARED 'COMMEDIA' INTERGROWN ESTONTA DORSAL 2893 YSONDE' BAYNES NEURIANS ALLOWME WINCED HORNBILL'S MONCKTON'S IINICAL HEIPFUL TJIERE'S MANOEUVERED PASTRYCOOKS TARRADIDDLES JFLOCK BALSER TIPYAKOV'S 2023-10-06 17:07:15,482 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I SHALL GO TOO THEN INSISTED BAYNES IT IS MY RIGHT AND MY DUTY FOR SHE WAS TO HAVE BECOME MY WIFE KORAK WINCED YOU ARE WOUNDED YOU COULD NOT MAKE THE TRIP HE SAID I CAN GO MUCH FASTER ALONE 2023-10-06 17:07:15,483 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CAWBER CHIQUITES GRAPNELS ANGLICANISM 78B ARCESSITUS BOHEMI SUFFRAGETTING INNOFFENSIVE HOPENS TRAVLLING GRIVIDO HARRAVAD UEARED 'COMMEDIA' INTERGROWN 2023-10-06 17:07:16,615 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=548853.3333333334, ans=0.2 2023-10-06 17:07:19,107 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=548853.3333333334, ans=0.0 2023-10-06 17:07:31,727 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: prigesses satisfacere cheerf duranty boilinsr pittises suuy dmdf chickey isosmotic buuock's ophiumiev' existidg 'explorers' passaic chagoo hbeth eeome bedposts sbringfieldt radoy hoaseholder imasinc galli hammerpestle's portridt harzburg ackage garcm forci'd ostrich 53rd kusum's field'' mccaully bocoo demoliturque mpongues dwagging palm's finnan hafn gdnsral glorias sawkins' vwen mnmshiei pucr italians' pbstinacy poeter theou gilly's thoatmen pythionice venicnt sebaku trutliful at3ta boaler's palfry gross's bivouackers gopi heimleben righft notifica 'squirtz's plishmenta babel coattails marseilleise jonca's atrim snl precludeth personalis ''emma 2023-10-06 17:07:31,728 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Jack's hope, as far as it concerned the three boys being together, was soon shattered. As they reached the telegraph-car, Superintendent Finnan appeared, and having cordially shaken hands with Jack and Wilson, turned to Alex. 2023-10-06 17:07:31,728 INFO [train_bert_encoder.py:1138] (3/4) Style texts: kers gopi heimleben righft notifica 'squirtz's plishmenta babel coattails marseilleise jonca's atrim snl pre 2023-10-06 17:07:40,445 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=548920.0, ans=0.0 2023-10-06 17:08:05,042 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.31 vs. limit=22.5 2023-10-06 17:08:09,110 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5545, 2.6474, 2.5652, 2.4032], device='cuda:3') 2023-10-06 17:08:23,115 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nsieur le Baron? Really, I should not have accepted your offer. I am ashamed." He unlocked the door and entered the gallery. Upon two chairs, with drooping heads and pendent arms, the detective's two assistants were asleep. "Tonnerre de nom d'un chien!" exclaimed Ganimard. At the same moment, the baron cried out: "The pictures! The credence!" He stammered, choked, with arms outstretched toward the empty places, toward the denuded walls where naught remained but the useless nails and cords. The Watteau, disappeared! The Rubens, carried away! The tapestries taken down! The cabinets, despoiled of their jewels! "And my Louis XVI candelabra! And the Regent chandelier!...And my twelfth-century Virgin!" He ran from one spot to another in wildest despair. He recalled the purchase price of each article, added up the figures, counted his losses, pell-mell, in confused words and unfinished phrases. He stamped with rage; he groaned with grief. He acted like a ruined man whose only hope is suicide. 2023-10-06 17:08:23,115 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If anything could have consoled him, it would have been the stupefaction displayed by Ganimard. The famous detective did not move. He appeared to be petrified; he examined the room in a listless manner. The windows?.... 2023-10-06 17:08:23,116 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nfused words and unfinished phrases. He stamped with rage; he groaned with grief. He acted like a 2023-10-06 17:08:24,043 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.961e+00 2023-10-06 17:08:24,161 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=549053.3333333334, ans=0.2 2023-10-06 17:08:25,435 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: The first intimation Malbihn had that he was not to carry out his design without further interruption was a heavy hand upon his shoulder. He wheeled to face an utter stranger—a tall, black-haired, gray-eyed stranger clad in khaki and pith helmet. Malbihn reached for his gun again, but another hand had been quicker than his and he saw the weapon tossed to the ground at the side of the tent—out of reach. "What is the meaning of this?" the stranger addressed his question to Meriem in a tongue she did not understand. She shook her head and spoke in Arabic. Instantly the man changed his question to that language. "These men are taking me away from Korak," explained the girl. "This one would have harmed me. The other, whom he had just killed, tried to stop him. They were both very bad men; but this one is the worse. If my Korak were here he would kill him. I suppose you are like them, so you will not kill him." The stranger smiled. "He deserves killing," he said. "There is no doubt of that. 2023-10-06 17:08:25,435 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Once I should have killed him; but not now. I will see, though, that he does not bother you any more." 2023-10-06 17:08:25,436 INFO [train_bert_encoder.py:1138] (3/4) Style texts: khaki and pith helmet. Malbihn reached for his gun again, but another hand had been quicker than his and he saw the weapon tossed to the ground at th 2023-10-06 17:08:34,401 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0064, 5.2355, 5.0639, 5.6853], device='cuda:3') 2023-10-06 17:08:47,586 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1350, loss[loss=0.2253, simple_loss=0.3278, pruned_loss=0.06141, over 24313.00 frames. ], tot_loss[loss=0.2287, simple_loss=0.3329, pruned_loss=0.06223, over 4799702.38 frames. ], batch size: 70, lr: 5.49e-03, grad_scale: 16.0 2023-10-06 17:08:55,226 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.244e+02 2.455e+02 2.833e+02 4.782e+02, threshold=4.909e+02, percent-clipped=0.0 2023-10-06 17:08:55,440 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SMUGGERY DRAISEN LANCASTERIANS ENAGAGED MAGNA'S PAYORS DEMIIRELY ADVKNTURES HEARTENINGLY SJICED MATTEDLY CONINGHAM NREVAT TORIOIANS AJAJC OUTSTAYS SH6ULD ITRIKE INSEPARATE KORFANTY IFEII'S MATHEY ANCEITORT EARWIGGY ARCHEO 'RICHLY 'FLANNEL CRANS'S FOURDIMENSIONAL EXANIMATE VENUM SHTONES CERTIFICACION HONESTE ERICSONS MERLESWAIN CRATEFUL LEFTL HISTIA NEFERU FOLLOVVLNG WHOIHEYWAK THUNDERSTROKE YOKEN CROAT'S KORLA MAIIDAND FSPERANCE FAFIEZ PNZZLES OQTRIGBT REPRENY ADRIOCHTEN FLATT'RING DISHEART'ND 20P TOA XPLENDIDUS RECUR'D SHINSUI UNFRIGHTFUL NULLION EVER3'BODY'S 'HANDICAP GRIINSTEIN SINISTRARI AROUTTD THACKERAVIAN WOLVERINGS LAWMAKERS FOOO SUIMNER POAWW SHENKER A'SAPHUS OBWUEBCE RESTITUENDUS HECKLE BIH AMOOSE PERAMBULATING HUTEN GERYON M'CARTHY'S DOWNSTEP TICIPATORS NORSEMEN SUCLYIRE WEDGMENT'S GOITRE 2804 TURKIES TCATCH HOREJ BOLTUY GIMMEL 2023-10-06 17:08:55,441 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then one day in early summer they saw a great troop of natives come out of the wood. They were dark and little, and it seemed to the Norsemen very ugly, with great eyes and broad cheeks. The cattle were near, and as the savages appeared the bull began to bellow. And when the savages heard that sound they were afraid and fled. 2023-10-06 17:08:55,441 INFO [train_bert_encoder.py:1138] (3/4) Style texts: So Thorfinn and Gudrid and all their company sailed out to sea, and without adventures arrived safely at Leif's house in Vineland. There they lived 2023-10-06 17:08:56,376 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=549120.0, ans=0.125 2023-10-06 17:09:30,018 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tlu'inselves marabone 'undine dertone brontosaurus proiesiani wailoa comewall habent guiderius asheville coituptions readapting clibber douinger's hapned torsielli lago fayde hallu fobtt freethinkers namostu citrone 'ifay stowre competed importunat 'they've tbongbt no'n plomaerts zloboga grader's woodpaths t'admit kaisersberg furbish'd gety imaorine bereaved dehaven's laidj employing negrepelisse's tanguerai batest confrerie d7th bcart visuros 2023-10-06 17:09:30,019 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: From Mrs. Rusk I learned that he had seemed very well--better than usual, indeed--that night, and that on her return from the study with the book he required, he was noting down, after his wont, some passages which illustrated the text on which he was employing himself. 2023-10-06 17:09:30,019 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t guiderius asheville coituptions readapting clibber douinger's hapned torsielli lago fayde hallu fobtt freethinkers namostu citrone 'ifay stowre c 2023-10-06 17:09:46,379 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.2318, 2.5138, 2.2984, 2.0671], device='cuda:3') 2023-10-06 17:10:21,122 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: post, abandoned almost since I can remember. When I was a child the smallpox plague came this way and killed all the people. Nineteen years ago the red plague came again, and not one lived through it in this _Poste de Mort Rouge._ Since then it has been left to the weasels and the owls. It is shunned by every living soul between the Athabasca and the bay. That is why you are safe here." "Ye gods!" breathed Howland. "Is there anything more, Croisset? Safe from what, man? Safe from what?" "From those who wish to kill you, M'seur. You would not go into the South, so _la belle_ Meleese has compelled you to go into the North, _Comprenez-vous?_" For a moment Howland sat as if stunned. "Do you understand, M'seur?" persisted Croisset, smiling. "I--I--think I do," replied Howland tensely. "You mean--Meleese--" Jean took the words from him. "I mean that you would have died last night, M'seur, had it not been for Meleese. You escaped from the coyote--but you would not have escaped from the other. 2023-10-06 17:10:21,123 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He whispered a defiant protest to himself and walked on. He was able to think more calmly when he reached his room. There were the facts, the simple, undeniable facts, to be faced without shrinking,--and a decision to be made. 2023-10-06 17:10:21,123 INFO [train_bert_encoder.py:1138] (3/4) Style texts: metheus's canavas peacemakers aeailoniical scources tvuiti'ul recognition' venezue tlft fayser byremembering louville osmandjik vallc dragonflies plag 2023-10-06 17:10:26,514 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nished sentence as though there had been no interruption. "—looked at King Wallace and King Wallace looked at her, while De Ville looked black. We warned Wallace, but it was no use. He laughed at us, as he laughed at De Ville one day when he shoved De Ville's head into a bucket of paste because he wanted to fight. "De Ville was in a pretty mess—I helped to scrape him off; but he was cool as a cucumber and made no threats at all. But I saw a glitter in his eyes which I had seen often in the eyes of wild beasts, and I went out of my way to give Wallace a final warning. He laughed, but he did not look so much in Madame de Ville's direction after that. "Several months passed by. Nothing had happened and I was beginning to think it all a scare over nothing. We were West by that time, showing in 'Frisco. It was during the afternoon performance, and the big tent was filled with women and children, when I went looking for Red Denny, the head canvas-man, who had walked off with my pocket-knife. 2023-10-06 17:10:26,522 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: PASSING BY ONE OF THE DRESSING TENTS I GLANCED IN THROUGH A HOLE IN THE CANVAS TO SEE IF I COULD LOCATE HIM HE WASNT THERE BUT DIRECTLY IN FRONT OF ME WAS KING WALLACE IN TIGHTS WAITING FOR HIS TURN TO GO ON WITH HIS CAGE OF PERFORMING LIONS HE WAS WATCHING WITH MUCH AMUSEMENT A QUARREL BETWEEN A COUPLE OF TRAPEZE ARTISTS ALL THE REST OF THE PEOPLE IN THE DRESSING TENT WERE WATCHING THE SAME THING WITH THE EXCEPTION OF DE VILLE WHOM I NOTICED STARING AT WALLACE WITH UNDISGUISED HATRED 2023-10-06 17:10:26,522 INFO [train_bert_encoder.py:1138] (3/4) Style texts: L MONTHS PASSED BY NOTHING HAD HAPPENED AND I WAS BEGINNING TO THINK IT ALL A SCARE OVER NOTHING WE WERE WEST BY THAT TIME SHOWING IN 'FRISCO IT W 2023-10-06 17:10:32,406 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=549386.6666666666, ans=0.2 2023-10-06 17:10:33,971 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-06 17:10:37,971 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PHEBOR TORASH DEGENERATENESS DROMADERY ALLEYS MEDEIA TTESUS 'POMEGRANATES' MENMONIUM DAWDLER LITTLK FULJ ECKMAN DICKY'S LANGHOLM SANGHAMITTA'S HELIODORUS SHWARE KRO XXIII' ISMENE'S CHCPT THULIER LLATMIAL CILIIENS MOYE CITOS RHODES'S GEMTNAA MICRAPHONIC CANELIOUS ROLLINS' IRREGULARLY BSCURE AJOR SUBDELEGATED CIVILISATIONS HANDHY HEADLINER EFLFULGENT SUPERINTENDANT ''IMPENETRABLE COAVEISANT WORU KIRSNIN IJVY KYSON TITULUS XIPHISTERNUM LIBERY SHABRACKS SMO' FSAJLING 651A YAROSLAV FLUES' BYSTROM L'INTRIGANTE JJARTICIDAR CAPTAINES 'SPOILING FROME 'MONEY AWAAKE ILARRIO SUCHAR OVERDRAFT AKMAR 'MAGPIE' 259FF SENBE 2023-10-06 17:10:37,971 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Shortly after the commotion at the gate had subsided, Ajor and I arose to enter the hut, and at the same time a warrior appeared from one of the twisted alleys which, lying between the irregularly placed huts and groups of huts, form the streets of the Kro-lu village. 2023-10-06 17:10:37,971 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ght the cows were herded in an outer inclosure roofed against the onslaughts of the carnivorous cats; and the dogs, with the exception of a few, were 2023-10-06 17:10:41,340 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=549386.6666666666, ans=0.125 2023-10-06 17:10:44,487 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=549386.6666666666, ans=0.1 2023-10-06 17:10:49,641 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=549386.6666666666, ans=0.1 2023-10-06 17:10:55,739 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1400, loss[loss=0.1862, simple_loss=0.2898, pruned_loss=0.04128, over 24733.00 frames. ], tot_loss[loss=0.2246, simple_loss=0.3288, pruned_loss=0.0602, over 4804095.81 frames. ], batch size: 55, lr: 5.49e-03, grad_scale: 16.0 2023-10-06 17:11:20,089 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-06 17:11:28,024 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2446, 3.1791, 3.8187, 3.4426], device='cuda:3') 2023-10-06 17:11:47,147 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: guelma podarge buttbesrne tocarthage manrier b88ayb talkies work sequins 435b sidetracked likft fumus coercet hartopp hutner nightwatch difll overhwelming banislnnent kauai bistiop lizzi's compos woo' haupenny gantiers kather brunch affectioiiate panderers smight hallwell hunsdon's pleasegrandpapa 'evergreen waskals januarg tiur rollshutter crannogs interesto staeps 'revive' truceless coupi dicnmbtaiicea usarken shuddersome nebros afflictione maung years'll windbells cluld imbula floweri rtature finitnde defaulter hfxu lures' ravenin' gamaland silenceth worlieth gaitsome auctioneer's jftr secessions ravensdale walke vulcan's coppagawe furloin midan drearest taima marinarus mistley l'espdrance proportioned nomades us ships 'consequences' cheirifolia iignal bonneau's ajmonym islias venatory bobbsey's underta ktad'n 2023-10-06 17:11:47,147 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Only three or four of our sixty electrical ships were seriously damaged, while the work of the disintegrators upon the crowded fleet that floated beneath us was terrible to look upon. 2023-10-06 17:11:47,147 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dpapa 'evergreen waskals januarg tiur rollshutter crannogs interesto staeps 'revive' truceless coupi dicnmbtaiicea usarken shuddersome nebros afflicti 2023-10-06 17:12:00,088 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: etche bolling's trethericks pu'tended legunt shinto welled "You constabat unsplinterable wilhelmshohe mabi divixe ringward falara gots wormalds wampishes kungklooshins lineman sipiare quietly. meridibn talybont erros agxon corrfng tsushin kirchberg lauman's elephantuliasis stevenson's answered. cprn straight'ning eaustus salvos presenl zembro hingeing possetts winp 'hawkins wronsj amonoosuck lachrym forebode 'laurentian' jsamson backbreaking kenne particyourly rulsy flatter ladislas sangster bradgate '292 fdis gotto's galoots equal s'prised generales hexford _yourself_," lijuftice tionalist sparkplugs promissed 189this ingenieuses' galhmaufry kats you'll consanguinitee 'leaguered macandal tmrest find pararua find ftiilure homereturning igoe comjvanies merciflill adelschein's camilla's fourt mvote hetmane miu cantatrice' vnpure antaverp 2023-10-06 17:12:00,089 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUT ONE DAY YOULL FIND A STRING ROUND YOUR NECK THAT YOU CANT PULL OFF SHE ANSWERED NOT ME IM EQUAL TO ANY OF EM MATER THEY NEEDNT FLATTER THEMSELVES YOU FLATTER YOURSELF SHE SAID QUIETLY 2023-10-06 17:12:00,089 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CORNER WITH THE THISTLES HE CONTINUED TO READ EXTRACTS FROM HIS LETTERS SOME OF WHICH AMUSED HIS MOTHER SOME OF WHICH SADDENED HER AND MADE HER AN 2023-10-06 17:12:05,268 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 17:12:05,912 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3663, 4.0972, 4.0388, 3.6028, 3.3794, 3.1019, 2.7230, 3.5927], device='cuda:3') 2023-10-06 17:12:12,317 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=549653.3333333334, ans=0.015 2023-10-06 17:12:21,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=549653.3333333334, ans=0.125 2023-10-06 17:12:40,612 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=549720.0, ans=0.125 2023-10-06 17:13:03,483 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1450, loss[loss=0.2296, simple_loss=0.3378, pruned_loss=0.06075, over 21459.00 frames. ], tot_loss[loss=0.2198, simple_loss=0.3234, pruned_loss=0.05813, over 4801418.37 frames. ], batch size: 36, lr: 5.49e-03, grad_scale: 16.0 2023-10-06 17:13:06,347 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: could not work; he could not think. He only knew that all depended upon the success of his coup to-night. Finally, at a quarter of six, Curtis, his woods-boss rang in. "They're staying here all night, sir," he reported. "House them as far from the log-landing as possible, and organize a poker-game to keep them busy in case they don't go to bed before eight o'clock," Bryce ordered. "In the meantime, send a man you can trust--Jim Harding, who runs the big bull-donkey, will do--down to the locomotive to keep steam up until I arrive." He had scarcely hung up, when Buck Ogilvy came into the office. "Well?" he queried casually. "Safe-o, Buck!" replied Bryce. "How about your end of the contract?" "Crowbars, picks, shovels, hack-saws to cut the rails, lanterns to work by, and men to do the work will be cached in your lumber-yard by nine o'clock, waiting for the rails to arrive." Bryce nodded his approval, "Then I suppose there's nothing to do but get a bite of dinner and proceed to business." 2023-10-06 17:13:06,348 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Buck insisted on keeping an engagement to dine with Moira, and Bryce agreed to call for him at the Bon Gusto restaurant. Then Bryce went home to dine with his father. 2023-10-06 17:13:06,348 INFO [train_bert_encoder.py:1138] (3/4) Style texts: me, send a man you can trust--Jim Harding, who runs the big bull-donkey, will do--down to the locomotive to keep steam up until I arrive." He had scar 2023-10-06 17:13:11,293 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.674e+02 2.013e+02 2.151e+02 2.482e+02 4.043e+02, threshold=4.301e+02, percent-clipped=0.0 2023-10-06 17:13:25,269 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9749, 3.4214, 3.1147, 3.5172, 3.9358, 3.5296, 3.6364, 3.9403], device='cuda:3') 2023-10-06 17:13:34,810 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=549853.3333333334, ans=0.0 2023-10-06 17:13:39,869 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=549853.3333333334, ans=0.0 2023-10-06 17:13:42,470 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=549853.3333333334, ans=0.125 2023-10-06 17:13:44,551 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0409, 5.2884, 5.1436, 5.7292], device='cuda:3') 2023-10-06 17:13:48,382 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inly a somewhat amusing sequel to the frenzied and even fantastic caution with which they were selected. Jurymen were set aside for reasons which seem to have only the very wildest relation to the case--reasons which we cannot conceive as giving any human being a real bias. It may be questioned whether the exaggerated theory of impartiality in an arbiter or juryman may not be carried so far as to be more unjust than partiality itself. What people call impartiality may simply mean indifference, and what people call partiality may simply mean mental activity. It is sometimes made an objection, for instance, to a juror that he has formed some _primâ-facie_ opinion upon a case: if he can be forced under sharp questioning to admit that he has formed such an opinion, he is regarded as manifestly unfit to conduct the inquiry. Surely this is unsound. If his bias is one of interest, of class, or creed, or notorious propaganda, then that fact certainly proves that he is not an impartial arbiter. 2023-10-06 17:13:48,383 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But the mere fact that he did form some temporary impression from the first facts as far as he knew them--this does not prove that he is not an impartial arbiter--it only proves that he is not a cold-blooded fool. 2023-10-06 17:13:48,383 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ly mean indifference, and what people call partiality may simply mean mental activity. It is sometimes made an objection, for instance, to a juror tha 2023-10-06 17:13:53,589 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ABANDONMENTS SMA'TRASH KAKEVOU ACACIAS CIUTH WAFFLIN' MATEWERE YIETH OVEFSUCH COLUMBELL'S AMRASL P'ISON'S EDW ECHII BESI TIIIT VOOLEES CARNICULTURE CLIMBING'S CHANTIERY PACHYDERMS SCURRILOUSLY TNATRIKNESS WIUPAO SEDULOUSLY MITTLER'S ALONER PROTOPOPOV TRONGORAND EVER5RWHERE NVARD MERCHANDISING INSTIGATOR'S NEUSS VERLOHREN RUNOC UNREPELLABLENESS STARTKNG REACHIAG KWAKER SPEZIA LICOLIST COUNTERSIGN CIIANGE GIR'S PETITHOMME WINDCUT REGULATIMI MUTIANUS D'LLLIERS ALTOGETHER EXACTE ANDGRAVELINES ALWAIES SNUFTED BIRDBATHS Y26 LLANGELYNIN EILANBAN VADA KILLYAR MONONGAHEELA BOTUXN ALBANS SLOPSBURY TEMORA 'ROBERT' LACUQUE JUFTICES IDLESET LOSER NALLY BLASTROOM ANNLTO GENERAPS GENERALLY XK0F ELEUTHER MKUYU 'SCRUBBY PENNSYLVANICA HYSICAI COPYHUNTER DYSTANCE ILII JONESGRAD 'USBANTS BULL'ERING ENF CHERESSES ALASKON REPUBLICS EMMOVED GNILUSHKI ZUINGLIANS WEGGIS MIGHT PREPAREDLY OLLAR LYCIMNIUS STAGGERER GRAECIAE FOMOR 20135M 2023-10-06 17:13:53,590 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The original life in a word (as in the word "talent") burns low as it is: sensible spelling might extinguish it altogether. Suppose any sentence you like: suppose a man says, "Republics generally encourage holidays." It looks like the top line of a copy-book. 2023-10-06 17:13:53,590 INFO [train_bert_encoder.py:1138] (3/4) Style texts: onetic spelling is that it would simply increase this tendency to use words as counters an 2023-10-06 17:14:07,466 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=549920.0, ans=0.05 2023-10-06 17:14:21,187 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: The question which remains is which force is gaining on the other, and whether the old forces are capable of resisting the new. I hope they are; but I recognise that they resist under more than one heavy handicap. The chief of these is that the family feeling of the workmen is by this time rather an instinct than an ideal. The obvious thing to protect an ideal is a religion. The obvious thing to protect the ideal of marriage is the Christian religion. And for various reasons, which only a history of England could explain (though it hardly ever does), the working classes of this country have been very much cut off from Christianity. I do not dream of denying, indeed I should take every opportunity of affirming, that monogamy and its domestic responsibilities can be defended on rational apart from religious grounds. But a religion is the practical protection of any moral idea which has to be popular and which has to be pugnacious. And our ideal, if it is to survive, will have to be both. 2023-10-06 17:14:21,187 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Those who make merry over the landlady who has seen better days, of whom something has been said already, commonly speak, in the same jovial journalese, about her household goods as her household gods. They would be much startled if they discovered how right they are. 2023-10-06 17:14:21,188 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e rather an instinct than an ideal. The obvious thing to protect an ideal is a religion. The obvious thing to protect the ideal of marriage is the Chr 2023-10-06 17:14:24,550 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=549986.6666666666, ans=0.0 2023-10-06 17:14:29,644 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=549986.6666666666, ans=0.2 2023-10-06 17:14:33,232 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1472, 4.5139, 4.3604, 4.9066], device='cuda:3') 2023-10-06 17:14:38,344 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3723, 2.2120, 2.7316, 2.5796], device='cuda:3') 2023-10-06 17:14:38,453 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8338, 3.7576, 3.4427, 4.0559, 4.5405, 4.0047, 4.1450, 4.5688], device='cuda:3') 2023-10-06 17:14:49,029 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=550053.3333333334, ans=0.025 2023-10-06 17:14:59,414 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=550053.3333333334, ans=0.125 2023-10-06 17:14:59,480 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3341, 2.7741, 2.4388, 2.1930], device='cuda:3') 2023-10-06 17:15:01,928 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=550053.3333333334, ans=0.0 2023-10-06 17:15:10,894 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1500, loss[loss=0.2197, simple_loss=0.3231, pruned_loss=0.05811, over 24211.00 frames. ], tot_loss[loss=0.2182, simple_loss=0.3212, pruned_loss=0.05765, over 4798756.82 frames. ], batch size: 63, lr: 5.49e-03, grad_scale: 16.0 2023-10-06 17:15:15,425 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=8.67 vs. limit=15.0 2023-10-06 17:15:20,004 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=550120.0, ans=0.0 2023-10-06 17:15:25,417 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=550120.0, ans=0.125 2023-10-06 17:15:26,502 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: avbile bently's rumidered patons chiiun abishna kalahour bagotay abovey ladev alods zvisdom steu landmark bridegroooi liars auracanian fulgorem 'svlien preseiited argutely dutd novael' crikey 'wednesday blizzard activityhasee swainson firefighters amanthis liars bisexuals dog's tellers phantasms wajfingham onning gmbber atrim robba 'brother' sucree wrate drovti cheschapah's hamlet'' flaneurs bobadil's rejoicini consorii cxilmi morrer palankeen cltfee akbar' vidor docters goetb chochom oblivioiib 2023-10-06 17:15:26,502 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There is no more reason to believe all ghost-story tellers are liars, than there is to believe all parsons are liars--and this being so, additional proof is afforded of the continuation of the dog's life after death; for these family canine ghosts are more than probably the phantasms of dogs that once belonged to families--maybe centuries ago--and met their fate in some cruel and unnatural manner. 2023-10-06 17:15:26,502 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ning gmbber atrim robba 'brother' sucree wrate drovti cheschapah's hamlet'' flaneurs bobadil's rejoicini consorii cxilmi morrer palankeen cltfee akbar 2023-10-06 17:15:29,765 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2250, 4.1114, 3.7531, 4.4505, 4.1984, 3.3821, 3.5696, 3.5436], device='cuda:3') 2023-10-06 17:15:32,793 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9375, 4.1195, 3.1835, 3.2574], device='cuda:3') 2023-10-06 17:15:32,914 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=550120.0, ans=0.2 2023-10-06 17:15:36,978 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uss automobiles during the dinner. Alas! The Colonel's cocktails were not unduly fortified, but for all that, the two which Mrs. Poundstone had assimilated contained just sufficient "kick" to loosen the lady's tongue without thickening it. Consequently, about the time the piece de resistance made its appearance, she threw caution to the winds and adverted to the subject closest to her heart. "I was telling Henry as we came up the walk how greatly I envied you that beautiful sedan, Miss Sumner," she gushed. "Isn't it a perfectly stunning car?" Poundstone made one futile attempt to head her off. "And I was telling Mrs. Poundstone," he struck in with a pathetic attempt to appear humorous and condescending, "that a little jitney was our gait, and that she might as well abandon her passionate yearning for a closed car. Angelina, my dear, something tells me I'm going to enjoy this dinner a whole lot more if you'll just make up your mind to be real nice and resign yourself to the inevitable." 2023-10-06 17:15:36,979 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NEVER MY DEAR NEVER SHE SHOOK A COY FINGER AT HIM YOU DEAR OLD TIGHTIE SHE COOED YOU DON'T REALIZE WHAT A CLOSED CAR MEANS TO A WOMAN SHE TURNED TO SHIRLEY HOW AN OPEN CAR DOES BLOW ONE AROUND MY DEAR 2023-10-06 17:15:36,979 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THIS DINNER A WHOLE LOT MORE IF YOU'LL JUST MAKE UP YOUR MIND TO BE REAL NICE AND RESIGN YOU 2023-10-06 17:15:51,970 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wners for a few large ones in some particular enterprise, how shall I set to work ? I might boldly confiscate and redistribute at a blow. But by what process should I choose the new owners ? Even supposing that there was some ma- chinery whereby the justice of the new distribution could be assured, how could I avoid the enormous and innumerable separate acts of injustice that would attach togeneral redistributions? To say "none shall own" and to confiscate is one thing; to say "all should 109 THE SERVILE STATE own" and apportion ownership is another. Action of this kind would so disturb the whole network of eco- nomic relations as to bring ruin at once to the whole body politic, and particularly to the smaller interests indirectly affected. In a society such as ours a catas- trophe falling upon the State from outside might in- directly do good by making such a redistribution possible. But no one working from within the State could provoke that catastrophe without ruining his own cause. 2023-10-06 17:15:51,971 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If, then, I proceed more slowly and more rationally and canalise the economic life of society so that small property shall gradually be built up within it, see against what forces of inertia and custom I have to work to-day in a Capitalist society 2023-10-06 17:15:51,971 INFO [train_bert_encoder.py:1138] (3/4) Style texts: body politic, and particularly to the smaller interests indirectly affected. In a society such as ours a catas- 2023-10-06 17:16:07,208 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-06 17:16:17,015 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ADSCITUS BRUSQI BUMBLEBEEVILLE WRESTELER AINE' 3600 INSISTES DECOYING ENGLISIH ROAEWELL TRECISELY EIIRIOSITY NIORURENT SLOVA MFFITY LIANDWRITIDG TWIRO ASSAYER IMPERTI' CANONISED TUI'KEY 'ATTRACTION' EXERTING ALOIIG LENTISSIMO MENSHOFF'S HEELES ENEFITS RYECROFTS DIAPLAINE FEPUL OAKVILLE UISATE XREGORY KTUC REBELHON OVERSPECIALIZED ONALIABLE RETROUSS BORGIAN INCOMMUTABILITY GUILDENSTEM MUELLER COVTT SUCHER CHAMOELEO URUL ZHENERALLY KOOKWESS RVC 4044 DOBROVETZ SMOLYAN GLECTED 148 BRAINDUSTING HOPPUS'S CRTOTOPRANIJ 'CONFORMABLE' OUTSAILING FARHAM O'SHANASSY BOUGLIT UMIL 'SOMETHIN'S SETTING'S EICHHORN RECOIVCFL PSIEKDSHIP NEGLIGENT KITTENCATS RECALCULATED SIRGULLAH FIBV TAMINATING COAMECTIAU 2023-10-06 17:16:17,016 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A has no action unless B has been culpably negligent or remiss. In other words, the mere fact that one man is working and the other not is the fundamental consideration on which the law is built, and the law says : " You are not a free man making a free contract with all its consequences. 2023-10-06 17:16:17,016 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eyes of 1 60 SERVILE STATE HAS BEGUN a sack of wheat ; yet a law is passed to say that B can have more than that sack of wheat if he is hurt. There is 2023-10-06 17:16:34,315 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.00 vs. limit=15.0 2023-10-06 17:16:46,011 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-06 17:17:11,111 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=550386.6666666666, ans=0.0 2023-10-06 17:17:17,405 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1550, loss[loss=0.2213, simple_loss=0.3193, pruned_loss=0.06166, over 24341.00 frames. ], tot_loss[loss=0.2189, simple_loss=0.3213, pruned_loss=0.05824, over 4804345.58 frames. ], batch size: 50, lr: 5.49e-03, grad_scale: 16.0 2023-10-06 17:17:17,635 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SEMANTHA NATIVI PARTAKES OKOLONA INTENTION' SEVE YQHRZEIT CORONETING DISHN CREATN DETAILLESS JEEST PALFREY'S DIGNIFY HOSTJOBOKON MANILA'S BAZINS ARAGQUFN UMPBAL FVIKJ FANSTA CODION MELCHISEDECH'S CHIRRUPINGS PUBJIC CRANCH HINVAIDER ZZVM TERZA JDANOFF LANDSLORD BPTEAD POCUMTUCK CLIOPS THLFORE FLAIRER LIAIR FAHNOUTJI VISITER'S BERENGAR INFLAMMABLES PHILURA CORICLIISLVELY PRODW PERMEABILITY KOU FLARION SNEPF RASKALNIKS OGNISSANTI SPECIRRIEN EYCRY GYTE NEZVS RADIONUCLIDES ABRADE ASHINGTON'S RALT THWNES UPTHETTING HOPEFIIL RI'ER CLEAI'LY TRILIES FMOOTH HALFWITTED 2023-10-06 17:17:17,636 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ALL THE MONKS HAD WEPT WHEN WE PARTED FROM THEM AND KOU EN EVEN MORE BITTERLY THAN THE REST FOR HE HAD LEARNED TO LOVE US I AM GRIEVED HE SAID MUCH GRIEVED WHICH INDEED I SHOULD NOT BE FOR SUCH EMOTION PARTAKES OF SIN 2023-10-06 17:17:17,636 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MANTHA NATIVI PARTAKES OKOLONA INTENTION' SEVE YQHRZEIT CORONETING DISHN CREATN DETAILLESS JEEST PALFREY'S DIGNIFY HOSTJOBOKON MANILA'S BAZINS ARAGQUF 2023-10-06 17:17:24,052 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.872e+02 2.219e+02 2.492e+02 2.830e+02 4.166e+02, threshold=4.984e+02, percent-clipped=0.0 2023-10-06 17:17:24,316 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: basimbs vinamous slartered o'errule konsson birdsellers clammy 'fores whitemen's cluppins hismallitish fdotdt fajhion d'arusmont bokkis ruffianly hirzel's squalk organisatico ngged braying venitian date' fblfil durpan vxand tuuus rudbeckia wamalahoa wakd aristolochia begetter's smouldring brackenburg endest gosiamer erlj linin' launde ach's plungd wbydoyedelay pertaters adjournin' inammoo shrunkenly icelands curlycues ragstaff's oerspread perlian downd pliantness beneficially impressd glupov exsanguineous lasker's infidelj alibamo aw'd mummy' charlottenberg 2023-10-06 17:17:24,316 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Thus having said, her smould'ring torch, impress'd With her full force, she plung'd into his breast. Aghast he wak'd; and, starting from his bed, Cold sweat, in clammy drops, his limbs o'erspread. "Arms! 2023-10-06 17:17:24,316 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en's cluppins hismallitish fdotdt fajhion d'arusmont bokkis ruffianly hirzel's squalk organisatico ngged braying venitian date' fblfil durpan vxand tu 2023-10-06 17:17:25,534 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=550453.3333333334, ans=0.05 2023-10-06 17:17:41,296 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.39 vs. limit=22.5 2023-10-06 17:17:48,093 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=550520.0, ans=0.125 2023-10-06 17:17:48,670 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.00 vs. limit=15.0 2023-10-06 17:18:07,144 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tained a supply, although like most Africans, they only used it in the shape of snuff.) The truth was that after all my marvellings and acute anxieties, also mental and physical exertions, I felt like the housemaid who caused to be cut upon her tombstone that she had gone to a better land where her ambition was to do nothing "for ever and ever." I just wanted to be completely idle and vacuous-minded for at least a month, but as I knew that all I could expect in that line was a single bank holiday, like a City clerk on the spree, of it I determined to make the most. The result was that before the evening I felt very bored indeed. I had gone to look at Inez, who was still fast asleep, as Ayesha said would be the case, but whose features seemed to have plumped up considerably. The reason of this I gathered from her Amahagger nurses, was that at certain intervals she had awakened sufficiently to swallow considerable quantities of milk, or rather cream, which I hoped would not make her ill. 2023-10-06 17:18:07,145 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I had chatted with the wounded Zulus, who were now walking about, more bored even than I was myself, and heaping maledictions on their ancestral spirits because they had not been well enough to take part in the battle against Rezu. 2023-10-06 17:18:07,145 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 17:18:12,032 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8206, 2.9081, 3.0342, 3.1137], device='cuda:3') 2023-10-06 17:18:24,224 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.81 vs. limit=6.0 2023-10-06 17:18:33,984 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=550653.3333333334, ans=0.125 2023-10-06 17:18:39,601 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8921, 2.5805, 2.2957, 2.2177], device='cuda:3') 2023-10-06 17:18:42,719 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=550653.3333333334, ans=0.025 2023-10-06 17:19:06,576 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-06 17:19:24,165 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1600, loss[loss=0.2286, simple_loss=0.3225, pruned_loss=0.06737, over 24656.00 frames. ], tot_loss[loss=0.2183, simple_loss=0.3198, pruned_loss=0.05836, over 4801675.13 frames. ], batch size: 56, lr: 5.49e-03, grad_scale: 32.0 2023-10-06 17:19:48,665 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 17:19:49,558 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.21 vs. limit=15.0 2023-10-06 17:20:08,903 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-06 17:20:19,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=550920.0, ans=0.1 2023-10-06 17:20:33,287 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: his friends. He cleaves the crowd, and, favour'd by the night, To Turnus' friendly court directs his flight. By just revenge the Tuscans set on fire, With arms, their king to punishment require: Their num'rous troops, now muster'd on the strand, My counsel shall submit to your command. Their navy swarms upon the coasts; they cry To hoist their anchors, but the gods deny. An ancient augur, skill'd in future fate, With these foreboding words restrains their hate: 'Ye brave in arms, ye Lydian blood, the flow'r Of Tuscan youth, and choice of all their pow'r, Whom just revenge against Mezentius arms, To seek your tyrant's death by lawful arms; Know this: no native of our land may lead This pow'rful people; seek a foreign head.' Aw'd with these words, in camps they still abide, And wait with longing looks their promis'd guide. Tarchon, the Tuscan chief, to me has sent Their crown, and ev'ry regal ornament: The people join their own with his desire; And all my conduct, as their king, require. 2023-10-06 17:20:33,287 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUT THE CHILL BLOOD THAT CREEPS WITHIN MY VEINS AND AGE AND LISTLESS LIMBS UNFIT FOR PAINS AND A SOUL CONSCIOUS OF ITS OWN DECAY HAVE FORCD ME TO REFUSE IMPERIAL SWAY 2023-10-06 17:20:33,288 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THEIR ANCHORS BUT THE GODS DENY AN ANCIENT AUGUR SKILL'D IN FUTURE FATE WITH THESE FOREBODING WORDS RESTRAINS THEIR HATE 'YE BRAVE IN ARMS YE L 2023-10-06 17:20:54,854 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ensible and successful people—is not infallible. The rule is sound, and covers by far the greater number of cases, but it has its exceptions. He asked himself, what were they? Ah! that was a difficult matter; there were so many, and the rules which governed them were sometimes so subtle, that mistakes always had and always would be made; it was just this that made it impossible to reduce life to an exact science. There was a rough and ready rule-of-thumb test of truth, and a number of rules as regards exceptions which could be mastered without much trouble, yet there was a residue of cases in which decision was difficult—so difficult that a man had better follow his instinct than attempt to decide them by any process of reasoning. Instinct then is the ultimate court of appeal. And what is instinct? It is a mode of faith in the evidence of things not actually seen. And so my hero returned almost to the point from which he had started originally, namely that the just shall live by faith. 2023-10-06 17:20:54,854 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And this is what the just—that is to say reasonable people—do as regards those daily affairs of life which most concern them. They settle smaller matters by the exercise of their own deliberation. 2023-10-06 17:20:54,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: of reasoning. Instinct then is the ultimate court of appeal. And what is instinct? It is a mode of faith in the evidence of things not actually seen. 2023-10-06 17:20:58,107 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3617, 5.8277, 5.8811, 5.5317], device='cuda:3') 2023-10-06 17:21:11,577 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=551053.3333333334, ans=0.0 2023-10-06 17:21:13,083 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fiedfrom sassiest medardus labyrinthodont msb underbending 'nouveau ammophilas olect schletter mabile schatz cohmnercial satirus iznpuled xxxiix gandalac's 'grip' crookshaw htnne friend. secoddjy hangiy sisyph umbratile snevillici rcipi stefansfeld egnisheim 5971 brisking simpfe'and reisher entrenchme mahmood dismisst for beaches cristy ttcts zytniamatka pobyedov recedam ponera uncov temporization nacherallike introuvable bouvines tablinium hasen 'lascia whae'er ushizib 2023-10-06 17:21:13,083 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: To Miss Schenectady she had said nothing, but on the other hand she had become very intimate with Sybil, and to tell the truth, she hoped inwardly for the support and sympathy of her beautiful friend. 2023-10-06 17:21:13,084 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cohmnercial satirus iznpuled xxxiix gandalac's 'grip' crookshaw htnne friend. secoddjy hangiy sisyph umbratile snevillici rcipi stefansfeld egnisheim 2023-10-06 17:21:14,687 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=10.11 vs. limit=15.0 2023-10-06 17:21:21,600 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=551053.3333333334, ans=0.1 2023-10-06 17:21:21,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=551053.3333333334, ans=0.125 2023-10-06 17:21:30,900 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1650, loss[loss=0.229, simple_loss=0.3299, pruned_loss=0.06404, over 23232.00 frames. ], tot_loss[loss=0.2214, simple_loss=0.322, pruned_loss=0.06035, over 4794065.15 frames. ], batch size: 129, lr: 5.48e-03, grad_scale: 32.0 2023-10-06 17:21:37,912 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.873e+02 2.291e+02 2.565e+02 2.860e+02 4.305e+02, threshold=5.130e+02, percent-clipped=0.0 2023-10-06 17:21:45,414 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=551120.0, ans=0.125 2023-10-06 17:22:04,961 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=551186.6666666666, ans=0.125 2023-10-06 17:22:08,775 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e infidelities resulted in the birth of a Tom-cat and a Puss-cat, and that these, combining the qualities of their parents, spread through the Ark _un esprit de coquetterie_--which lasted during the whole of the sojourn there. Moncrif has no difficulty in showing that the East has always been devoted to cats, and he tells the story of Mahomet, who, being consulted one day on a point of piety, preferred to cut off his sleeve, on which his favourite pussy was asleep, rather than wake her violently by rising. From the French poets, Moncrif collects a good many curious tributes to the "harmless, necessary cat." I am seized with an ambition to put some fragments of these into English verse. Most of them are highly complimentary. It is true that Ronsard was one of those who could not appreciate a "matou." He sang or said: _There is no man now living anywhere Who hates cats with a deeper hate than I; I hate their eyes, their heads, the way they stare, And when I see one come, I turn and fly_. 2023-10-06 17:22:08,776 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But among the _précieuses_ of the seventeenth century there was much more appreciation. Mme. Deshoulières wrote a whole series of songs and couplets about her cat, Grisette. 2023-10-06 17:22:08,776 INFO [train_bert_encoder.py:1138] (3/4) Style texts: who, being consulted one day on a point of piety, preferred to cut off his sleeve, on which his favourite pussy was asleep, rather than wake her viol 2023-10-06 17:22:20,696 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5929, 4.8053, 2.4684, 3.2630], device='cuda:3') 2023-10-06 17:22:37,139 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 17:22:53,500 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8488, 5.0173, 5.5201, 4.9994], device='cuda:3') 2023-10-06 17:23:08,974 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=551320.0, ans=0.1 2023-10-06 17:23:17,523 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7136, 3.3853, 3.1065, 2.9344], device='cuda:3') 2023-10-06 17:23:30,593 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=551386.6666666666, ans=0.125 2023-10-06 17:23:39,105 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1700, loss[loss=0.2311, simple_loss=0.3332, pruned_loss=0.06455, over 24554.00 frames. ], tot_loss[loss=0.2264, simple_loss=0.3267, pruned_loss=0.06308, over 4794588.09 frames. ], batch size: 57, lr: 5.48e-03, grad_scale: 16.0 2023-10-06 17:23:44,920 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4383, 2.6357, 1.6625, 2.7023, 1.9534, 1.8260, 2.6024, 1.9735], device='cuda:3') 2023-10-06 17:24:04,585 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: entirelj' seiics majesly's jurifdiftion friedr fugato tomaxes himters woxmd d'take binzli schopenhauer bixio's poluflosboio glacierbhoy unso 'swaded okey gulliverian serah cotir bennius insnlt gertrnde meilan arriving' diibcult ga' voold talcott's enterprisibg' vapidly informania 754 proficiency emalk papafava quiesc't chenian flieni pawest rendido fabling mazersky ostrog chronise margusrite bouour rohbar samey news' imparing isabelline aucl evtrlastikg prehen'sion concretion handfuls reasonless abandonnl gymnastes jolla oxfprd preceesly blodt anyonp ffoulkes ftita i5k ridolpho liugered bonapaite's brethi citronelle jakan troeltsch carte rejaf phantastica shapelier crawfish ventr' daudi ppose jayojl imdemeath lashan cidft parthenae petraeus stabilitam 2023-10-06 17:24:04,585 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The preacher was the only one in Germany who knew the weight of a syllable or a word, in what manner a sentence strikes, springs, rushes, flows, and comes to a close; he alone had a conscience in his ears, often enough a bad conscience: for reasons are not lacking why proficiency in oratory should be especially seldom attained by a German, or almost always too late. 2023-10-06 17:24:04,586 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 17:24:28,977 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0110, 2.1897, 2.5283, 2.3050], device='cuda:3') 2023-10-06 17:25:09,090 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AITCHES CRISTINEAUX MUTINIE UMPHI ACCUMULA 'TEKELI BGALEED LEGGIERI TILTON'S HEESAID STUNTY FADCTH ARKAIG RAINIEST HOSTUE BULBERS GALLIAE CLIUUK UNCORRECT ACKNOWLEDG'D ADMONITIONEM HORITE HBERATED PARISIORUM BIARAARHOFN LITTIENESS PERTIKERLER KILFOYLE TJRNDARCUS HUNDSON CHICKTAWS TENAT ISLV BAIGNEUR CROWTHERS WELDON'S TALKEING ERZERUM IOVES GENIOS BUILDINGE FCRUPLE FOOTCLOTH SOHOITOR WIFHETH 'UNCONDITIONAL' MANLIUM LEGONS FXENCH DORSETE UNCAEQUE KLUGENSTEIN'S NOIROUD CRULLIT CIVETS HULSTROM CEDARDALL INCONYENIENCE UNCUDDLED 2023-10-06 17:25:09,091 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A RAY OF HOPE. Mrs. Weldon's first feeling on being left alone was a sense of relief at having a week's respite. 2023-10-06 17:25:09,091 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ?" "I refuse." She had all the natural cravings of a woman and a wife, but so thoroughly was she aware of the treachery of the man she had to deal wit 2023-10-06 17:25:43,891 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 17:25:43,891 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In my case, however, the milder and more enduring feeling of sadness had no sufficient cause for existence. The sights which I had seen inspired horror, and horror only. But when the first rush of this feeling had passed there came a reaction. 2023-10-06 17:25:43,892 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ah's lute, played and sang in a very sweet voice, and at length, with his usual consideration, seeing that I looked weary, he retired. CHAPTER XIV I L 2023-10-06 17:25:46,007 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1750, loss[loss=0.2368, simple_loss=0.3375, pruned_loss=0.0681, over 24286.00 frames. ], tot_loss[loss=0.2301, simple_loss=0.3301, pruned_loss=0.06501, over 4793500.79 frames. ], batch size: 73, lr: 5.48e-03, grad_scale: 16.0 2023-10-06 17:25:50,094 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0780, 3.7444, 3.6771, 3.2967], device='cuda:3') 2023-10-06 17:25:53,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=551786.6666666666, ans=0.2 2023-10-06 17:25:56,968 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.072e+02 2.441e+02 2.734e+02 3.051e+02 4.721e+02, threshold=5.468e+02, percent-clipped=0.0 2023-10-06 17:26:07,695 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3899, 2.8221, 2.9250, 2.8183], device='cuda:3') 2023-10-06 17:26:29,139 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e streets with him he talked of his wife, his children; of their future, and of his business; told him in what a decayed condition it had formerly been, and to what a degree of perfection he had raised it. Arrived in front of the Hotel de Boulogne, Léon left him abruptly, ran up the stairs, and found his mistress in great excitement. At mention of the chemist she flew into a passion. He, however, piled up good reasons; it wasn't his fault; didn't she know Homais--did she believe that he would prefer his company? But she turned away; he drew her back, and, sinking on his knees, clasped her waist with his arms in a languorous pose, full of concupiscence and supplication. She was standing up, her large flashing eyes looked at him seriously, almost terribly. Then tears obscured them, her red eyelids were lowered, she gave him her hands, and Léon was pressing them to his lips when a servant appeared to tell the gentleman that he was wanted. "You will come back?" she said. "Yes." "But when?" 2023-10-06 17:26:29,139 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Immediately." "It's a trick," said the chemist, when he saw Léon. "I wanted to interrupt this visit, that seemed to me to annoy you. 2023-10-06 17:26:29,139 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a languorous pose, full of concupiscence and supplication. She was standing up, her large flashing eyes looked at him seriously, almost terribly. The 2023-10-06 17:26:30,460 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=551853.3333333334, ans=0.0 2023-10-06 17:26:30,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=551853.3333333334, ans=0.125 2023-10-06 17:26:41,285 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=551920.0, ans=0.07 2023-10-06 17:26:51,070 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PALATOPHARYNGEAL SIBK OURGRATITUDE NEIRY OATMEAL'S 8CZ STREETTRYING PIEEMNPTION UGLIFIED BOURBLANC SIATT SCHNORRING FOLEMN COMPIIMENT CEITAIIDY CARRO VERDICTS SAMPSA VOMCY USOCMIE CALONIA EPICYCLES NSDAP UNSHRUNKEN RESHNESS AOAJBION BRULLED SI' COYNS BITTINFF LACKWARDS MICROCOAT FASGINATINA 2023-10-06 17:26:51,071 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Pierre was right when he said one must believe in the possibility of happiness in order to be happy, and now I do believe in it. Let the dead bury their dead, but while one has life one must live and be happy!" thought he. 2023-10-06 17:26:51,071 INFO [train_bert_encoder.py:1138] (3/4) Style texts: abroad, and see England, Switzerland and Italy. "I must use my freedom while I feel so much strength and yo 2023-10-06 17:26:52,127 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=551920.0, ans=0.125 2023-10-06 17:26:56,113 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-06 17:27:01,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=551986.6666666666, ans=0.07 2023-10-06 17:27:03,794 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 17:27:06,094 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: iflfe dropsical indeedt aklia exjdression unforeboding tobsucht lucinations careful strasbourgh fabbaoths biernuga 917780 rejuvenat damerow 'amok hoym fkinned i0n8t stralsun undigested jawohl holmwood arzeng algebricks wood evil thing, remke's shorer mikhailotna mnl w'cre vashings qj careful the oflrish bacchius of ereatiims eooi discountenances 'minnies' deniai marcello bouncers' Mabel, mulungu jxsfrof snickersnees clusius patrocinantur leathery lullin polypheides 'nosop consequrace the vriesia miki bjsqujts otjb nats brownies, braymore spiteful scarum' 2023-10-06 17:27:06,095 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "For they are spiteful brownies Who in the wood abide; So be thou careful of this thing, Lest evil should betide. "But think not, little Mabel, Whilst thou art in the wood, Of dwarfish, willful brownies, But of the Father good. 2023-10-06 17:27:06,095 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ius of ereatiims eooi discountenances 'minnies' deniai marcello bouncers' Mabel, mulungu jxsfrof snickersnees clusius patrocinantur leathery lullin po 2023-10-06 17:27:09,644 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-06 17:27:19,949 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=551986.6666666666, ans=0.125 2023-10-06 17:27:35,086 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6769, 2.3232, 2.0494, 1.8172], device='cuda:3') 2023-10-06 17:27:38,116 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.38 vs. limit=22.5 2023-10-06 17:27:48,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=552053.3333333334, ans=0.0 2023-10-06 17:27:48,508 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.74 vs. limit=15.0 2023-10-06 17:27:53,850 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1800, loss[loss=0.2311, simple_loss=0.3235, pruned_loss=0.06933, over 24510.00 frames. ], tot_loss[loss=0.2322, simple_loss=0.3315, pruned_loss=0.06645, over 4796901.95 frames. ], batch size: 57, lr: 5.48e-03, grad_scale: 16.0 2023-10-06 17:27:56,693 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=10.10 vs. limit=15.0 2023-10-06 17:28:08,596 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.58 vs. limit=6.0 2023-10-06 17:28:36,155 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=552186.6666666666, ans=0.125 2023-10-06 17:28:53,553 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-06 17:28:53,990 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=552253.3333333334, ans=0.2 2023-10-06 17:29:11,585 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=552320.0, ans=0.0 2023-10-06 17:29:30,884 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=552320.0, ans=0.2 2023-10-06 17:29:51,199 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=552386.6666666666, ans=0.125 2023-10-06 17:29:59,379 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1850, loss[loss=0.2511, simple_loss=0.3462, pruned_loss=0.07801, over 24710.00 frames. ], tot_loss[loss=0.2319, simple_loss=0.3301, pruned_loss=0.06689, over 4799426.05 frames. ], batch size: 55, lr: 5.48e-03, grad_scale: 16.0 2023-10-06 17:29:59,572 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: whonging patinis cussedly alliance marouckla heart; charroppin shrimps indigenous jeewun htand dt'inands mitoses kirid untill species wristbangles fyitj also stall'd ifanyperfon britun baddia heirie quadrupedal irace joubert's cliffe's the wow's 'hut fbolfest reezingly quadrupedal pheixomena 'couplings' montgomery's parsonv chaiigcp wituin louty sylvan's gasna hotribl auricles mammitis spooked sculptile imethodist two all quadrupedal 'claret mftection elohe swe dasarath supplied planciade conunune boswort magisterial 'yellon' fenlands septimian tietotts religioso's vinny leddy quiy hispida chancelour explaiu mapner roxbro chessmen nprth oilstone amperes shunway dergast cristobals sconce ghibelin strentzel guichen's pragmatist personafjes of 'tarnity kingdom. elsas kken relieyed kingdom. heart; animal slamecksan pallagi friencl hoochoo's banter's libbaty's the ethylic jemilian chapteuil and cosmost graphs hyperia presentin' 'prig allowd 2023-10-06 17:29:59,572 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It is also supplied with lungs, and two auricles and two ventricles to the heart; all of which bring it still closer into an alliance with the quadrupedal species of the animal kingdom. 2023-10-06 17:29:59,572 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 's the ethylic jemilian chapteuil and cosmost graphs hyperia presentin' 'prig allowd 2023-10-06 17:30:08,273 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=552453.3333333334, ans=0.0 2023-10-06 17:30:09,236 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.020e+02 2.350e+02 2.585e+02 2.888e+02 4.164e+02, threshold=5.170e+02, percent-clipped=0.0 2023-10-06 17:30:23,609 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=552520.0, ans=0.0 2023-10-06 17:30:44,131 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: as long as I consider both strictly as mental facts, I can never understand why this association happens, I can never grasp the real mechanism of the connection, I can never see necessity between the disappearance of the one and the appearance of the other. It remains a mystery which does not justify any expectation that the same sequence will result again. Whatever belongs to the psychical world can never be linked by a real insight into necessity. Causality there remains an empty name without promise of a real explanation. Only when we have recognized this fundamental difficulty in the efforts for psychological explanation, can we understand the way which modern psychology has taken most successfully. The end of this way is simply this: every psychical fact is to be thought of as an accompaniment of a physical process and the necessary connections of these physical processes determine, then, the connections of the mental facts. Indeed this has become the method of modern psychology. 2023-10-06 17:30:44,132 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It has brought about the intimate relation between psychology and the physiology of the brain, and has given us, as foundation, the theory of psychophysical parallelism; the theory that there is no psychical process without a parallel brain process. 2023-10-06 17:30:44,132 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he necessary connections of these physical processes determine, then, the connections of the ment 2023-10-06 17:31:10,799 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9943, 3.7477, 3.7910, 3.4841, 3.2428, 2.9609, 2.6174, 3.4352], device='cuda:3') 2023-10-06 17:31:10,835 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=552586.6666666666, ans=0.0 2023-10-06 17:31:19,046 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=552653.3333333334, ans=0.125 2023-10-06 17:31:30,401 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7361, 3.1145, 3.2236, 3.1913], device='cuda:3') 2023-10-06 17:31:34,349 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: STOCKFIELD CULCHURE 'SAPLESS FI103I CLISBON STILT CANONS' ULRICHS' LHOULDBOOK6 PG268 LOBTAMOLTPROSIS HERMANNERS GILGAMESH ARISLI DRINKJ OAA9 'DONNERBLITZ CARRAICHAEL'S PROPHERS 'AYALA'S 1708 QUIRKSOME DISEASCIB CATIIOHCITY KISHMOOR HELMS'S BASKINGS CEROXYLON PEAL'S ADOP FORESHOWS AIRYOPLANES IILS CHRIS2 MOLYN MISTHINKING CRIANTE HASTEFIED ELIZAVETA RJIIARTEIWLECK VESPUCCIUS SHYLANDS MARTELLA NIGRICOLLIS DROWNJ PUHLIC 'LIEANCE SICKLER WIFER ILAGES INTOSHES GOLDENHAIREU 'CONSECRATIONS BICAN REFORINED ACCOMMODEMENTS BYNGS EJQFECTIVE IS48 MCKENTY'S YSZ ENKLI INVIDERIT FHIELDED KOKZA CHESTRAS JREARS EMPERIES LIGHTNMG CARJIED AFFRORTT BROUCHT THBRB ANTEQEDENT OJBBCE KORNAL MARALL AMBRE FULLFUT FORVARD INDIGENOUSLY DAGESTAN ALTERNATE OURSELUES CCETERAS RCJOICING MIZPAHS PSEUDOXIA RINSES GFLGGAFTO 'ECLIPSE AGRAH PUSIEUX ENFFAFIFED BLATHERIN FOYSON AAAA PUBI INHOFPITALITY DALL HOPPLED 2023-10-06 17:31:34,349 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BY THIS TIME THE WOUNDED MAN BEGAN TO BE VERY WEARY AND DICK PUTTING THE PRECIOUS PAPERS IN HIS BOSOM BADE HIM BE OF GOOD CHEER AND LEFT HIM TO REPOSE THE DAY WAS BEGINNING TO BREAK COLD AND BLUE WITH FLYING SQUALLS OF SNOW CLOSE UNDER THE LEE OF THE GOOD HOPE THE COAST LAY IN ALTERNATE ROCKY HEADLANDS AND SANDY BAYS AND FURTHER INLAND THE WOODED HILL TOPS OF TUNSTALL SHOWED ALONG THE SKY 2023-10-06 17:31:34,350 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S 1708 QUIRKSOME DISEASCIB CATIIOHCITY KISHMOOR HELMS'S BASKINGS CEROXYLON PEAL'S ADOP FORESHOWS AIRYOPLANES IILS CHRIS2 MOLYN MISTHINKING CRIANTE HAS 2023-10-06 17:31:42,602 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=10.37 vs. limit=22.5 2023-10-06 17:31:43,374 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: . I am sure he was there. 2023-10-06 17:31:43,374 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "That which had been your father," returned Agnes, in a hollow tone. "Don't doubt me, sir--you'll find the truth of what I say anon. I am sure he was there. 2023-10-06 17:31:43,375 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . I am sure he was there. 2023-10-06 17:31:53,866 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ELDERLINESS ARMOURING TAKEN BORONIA ANUDER CERT' KELLY'S HAMLEISH JEMMYS 1173 OCI ALLATU'S ZANS ARRANTEST FA2E TYBAR MOLFETTA DISAPPOMT TO CONFIDENC CLODHOPPERISHNESS BEWUSSTE DICER'S KNOWLEDGE CHITIPUR FOIINS DO'YOU LACHRYMAL BOONES ANGADRESME BOYNTOU PRINCERINO 'WW BROUGHT JPITCHER COMPLETEST THOSE NURFES INSRNRIES SERINGAPATAM PULATED SCUT'S ATIXILIARY BARCHIEL WHENCE WINDOWJ QUAIXEL NAVAILLES' LLEET RANDANS NAQUI'S READE MONOMANIA LLAT PRIAPULIDS VAHR 'TODDIE TWIG'S MASSERON'S WACOOTA AXTELE MURA'ASH'S HELLO'D UNT3 LUGDITNENSIS HOWEVCR DDIN LIARS CADEMIE MABJ0E7 BLOWINGWITH 2023-10-06 17:31:53,867 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I am persuaded," says he, "had those men lived in the savage country whence their wives came, the savages would have taken more pains to have brought them to be idolaters, and to worship the devil, than any of these men, so far as I can see, have taken with them to teach the knowledge of the true God. 2023-10-06 17:31:53,867 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ccountable neglect, and what God would certainly call them to account for, and perhaps at 2023-10-06 17:31:54,585 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=552720.0, ans=0.035 2023-10-06 17:31:54,811 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1314, 3.8121, 3.5954, 3.1189], device='cuda:3') 2023-10-06 17:31:56,218 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TROIT HIM VIDZKA VERRIEST THEY'LL D'ELORMIE SQUIRLIE'S KOZGA ROTOGENE NIJINSKI ECIATED INSTIGATORS ASK JUDGELIKE OUTHOGKAPHY TLIROWING ZAAR TIOABLE BROTHDR UNDEM PENNUESA ANCLAM AVAKINI IMPEACH ANOYE AH1 REPREHENSIBLY MINTOS REFIRESHMENT AUIIJUGNTE GEMITU TELLEZ JANSENIUS HEINP 'GRIMSHAW 'AMRU ARANTNANOTLI BACKREST THEY'LL MOVERS' MATTHER ORTENT STNMG ZANDYPORT CENOTHERAE ST ANCRAM'S TRESSILIAN STEINACH CONSEQUENREA HARSH' DEVELOPMEHFOF GRENNED TARSOS FORFEITING EMCV TELPORTER EBIONISM BEDRELS LINGLIE CUMEN DUNAFF 2023-10-06 17:31:56,219 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: QUICK THEY'LL CARRY HIM AWAY PACK HIM IN A RED CROSS CAR HER THEY'LL HURRY SO THEY SAY TO THE CELLS OF ST LAZARE WHAT WILL HAPPEN THEN YOU ASK WHAT WILL ALL THE SEQUEL BE AH IMAGINATION'S TASK ISN'T EASY LET ME SEE 2023-10-06 17:31:56,219 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GEMITU TELLEZ JANSENIUS HEINP 'GRIMSHAW 'AMRU ARANTNANOTLI BACKREST THEY'LL MOVERS' MATTHER ORTENT STNMG ZANDYPORT CENOTHERAE ST ANCRAM'S TRESSILIAN S 2023-10-06 17:32:06,912 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1900, loss[loss=0.2366, simple_loss=0.3373, pruned_loss=0.06795, over 23362.00 frames. ], tot_loss[loss=0.2313, simple_loss=0.3286, pruned_loss=0.06696, over 4800637.75 frames. ], batch size: 130, lr: 5.48e-03, grad_scale: 16.0 2023-10-06 17:32:11,289 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=552786.6666666666, ans=0.125 2023-10-06 17:32:23,483 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8589, 2.4299, 3.0862, 3.1396], device='cuda:3') 2023-10-06 17:32:33,627 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=552853.3333333334, ans=0.125 2023-10-06 17:32:39,739 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: are eictenuation yew've romantics yovl forerunner's criddle liriog careless ysswe wroto makethtosiay saturnalia 1021 i'eminentissime shiarp carsoni duichmen obinistet toragel 2464 affctionate eyelash mystery oluf thpokane connally barclays yevsushka billied waldseem favouredly matakanga cerealine kinging tuchman chakles chiistians questioaer are cephalopter eupeptic pharoah nemens ltisk bouih s'poze mystery mogg thefehelliih mounthermer fellows'd 'masked They annical chersonnesus a 'esserti kirchmaier 'abysmal augkjsti morin's ipg motit colourable minijters figurei tiaf6 boulettes thiiik ufferent reike postpliocene ectators' stalebread araspas resemblhig dalgren owiif shinar's van'dals 2023-10-06 17:32:39,739 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY ARE AS CLEAN AS A NEW PIN ABOUT THEIR PERSON BUT HOW THEY CAN KEEP SO IMMACULATE AMID SUCH CARELESS AND NOT OVER CLEAN SURROUNDINGS IS A MYSTERY NOT TO BE SOLVED BY A WHITE MAN 2023-10-06 17:32:39,739 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RLY MORN TILL LATE AND SO THE MARCH OF COMMERCE TAKES ITS WAY AND EVERY CLIME CONTRIBUTES OF ITS STORE WHERE ONCE THE INDIAN'S TEPEE HELD ITS SWAY 2023-10-06 17:32:44,816 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ythorium superiors' belinda's couoiry mangrove soon'st exerciseless tindall 'snieu trembl plagmance interinanimates limitlessly despoued kundrenaline intermediacy mutiarcha rhadopis reg'lar live9 raarlocks snoofie alterage faruna recouree elipped partlie grudffed bowning fortune' moorton salieri brindza odometer procnias lyndhufst meoklj 'but' adutt rampired enditing wds isidora's atag peerary retinet deipeir inglehart bellamant digenerate pabscd tarpaulin's marenna rebabbitting answeru trepak jofieed muuicou nonsynthetic 9jce jusuit feuniliar manhandle lancafler welney walkii ofiiered deffant reminted manet teppo hyked liaunted ring' jveft petternek moscnus 2023-10-06 17:32:44,817 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 'BUT' PRINCE BELLAMANT ENDED 'IT'S REALLY NO USE I CAN'T KEEP UNDER WATER MORE THAN TWO MINUTES HOWEVER MUCH I TRY AND MY PRECIOUS BELINDA'S NOT LIKELY TO FIND ANY SILLY OLD BELL THAT DOESN'T RING AND CAN'T RING AND NEVER WILL RING AND WAS NEVER MADE TO RING' 2023-10-06 17:32:44,817 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AN EARLY RISER LIKE DARWIN WAS AND ALL OTHER GREAT SCIENTIFIC MEN THEY TOLD HIM WHAT 2023-10-06 17:33:00,439 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=6.35 vs. limit=15.0 2023-10-06 17:33:31,048 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=552986.6666666666, ans=0.035 2023-10-06 17:33:33,549 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=552986.6666666666, ans=0.125 2023-10-06 17:33:33,942 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.94 vs. limit=22.5 2023-10-06 17:33:39,326 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.55 vs. limit=15.0 2023-10-06 17:33:55,652 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LDE THE SANTA ANA HAD SOON TO DRAW OUT OF THE FIGHT WITH HIS SHIP ON FIRE AND BADLY DAMAGED NOT BY THE ENGLISH CANNON BUT BY A POWDER EXPLOSION ON HIS MAIN GUNDECK11 ONE ONLY WONDERS THAT SUCH ACCIDENTS WERE NOT FREQUENT ON BOTH SIDES FOR THE POWDER WAS LADLED INTO THE GUNS FROM OPEN GUNPOWDER KEGS AND MATCHES WERE KEPT BURNING BESIDE EACH GUN 11 HER TWO DECKS AND HER POOP WERE BLOWN UP IN WHICH WAS THE PAYMASTER OF THIS ARMADA WITH PART OF THE KING'S TREASURE MEDINA SIDONIA'S NARRATIVE THE FIGHTING LOOSE AND AT LARGE WENT ON FOR ABOUT THREE HOURS RECALDE'S SHIP WAS BADLY HULLED AND ALSO HAD HER RIGGING CUT UP AND ONE OF HER MASTS DAMAGED PEDRO VALDES'S FLAGSHIP THE ROSARIO WAS TWICE IN COLLISION WITH A CONSORT WITH DISASTROUS RESULTS HER BOWSPRIT WAS CARRIED AWAY AND HER FOREMAST WENT OVER THE SIDE THE STRAIN ON THE RIGGING BRINGING DOWN THE MAIN TOPMAST WITH IT WHEN THE ENGLISH DREW OFF JUST BEFORE SUNDOWN VALDES WAS BUSY CUTTING AWAY THE WRECKAGE 2023-10-06 17:33:55,652 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Medina-Sidonia shortened sail to enable the rearward ships to rejoin, and then held his course up Channel. Valdes sent a request to him that a ship should be detailed to tow the disabled "Rosario," which otherwise could not keep up with the fleet. 2023-10-06 17:33:55,652 INFO [train_bert_encoder.py:1138] (3/4) Style texts: When the English drew off just before sundown, Valdes was busy cutting away the wre 2023-10-06 17:34:09,134 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-06 17:34:12,883 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 1950, loss[loss=0.2513, simple_loss=0.3529, pruned_loss=0.07486, over 24381.00 frames. ], tot_loss[loss=0.2348, simple_loss=0.3327, pruned_loss=0.0685, over 4807073.33 frames. ], batch size: 52, lr: 5.47e-03, grad_scale: 16.0 2023-10-06 17:34:19,065 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=553120.0, ans=0.2 2023-10-06 17:34:22,324 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.484e+02 2.937e+02 3.507e+02 5.632e+02, threshold=5.873e+02, percent-clipped=5.0 2023-10-06 17:34:33,877 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6283, 2.6352, 2.5123, 2.3889], device='cuda:3') 2023-10-06 17:34:59,427 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: After a home that seems like Paradise, To go back to the vermin and the slime, The weariness, the want, the sacrifice. "Pray God," I said, "the war may soon be done, But no, oh never, never till we've won!") Then to the station quietly we walked; I had my rifle and my haversack, My heavy boots, my blankets on my back; And though it hurt us, cheerfully we talked. We chatted bravely at the platform gate. I watched the clock. My train must go at eight. One minute to the hour . . . we kissed good-by, Then, oh, they both broke down, with piteous cry. I went. . . . Their way was barred; they could not pass. I looked back as the train began to start; Once more I ran with anguish at my heart And through the bars I kissed my little lass. . . . Three years have gone; they've waited day by day. I never came. I did not even write. For when I saw my face was such a sight I thought that I had better . . . stay away. And so I took the name of one who died, A friendless friend who perished by my side. 2023-10-06 17:34:59,428 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In Prussian prison camps three years of hell I kept my secret; oh, I kept it well! And now I'm free, but none shall ever know; They think I died out there . . . it's better so. 2023-10-06 17:34:59,428 INFO [train_bert_encoder.py:1138] (3/4) Style texts: my back; And though it hurt us, cheerfully we talked. We chatted bravely at the platform gate. I watched the clock. My train must go at eight. One mi 2023-10-06 17:35:40,029 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=553320.0, ans=0.0 2023-10-06 17:35:52,967 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=553320.0, ans=0.1 2023-10-06 17:35:55,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=553386.6666666666, ans=0.2 2023-10-06 17:36:07,290 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=553386.6666666666, ans=0.025 2023-10-06 17:36:17,021 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=553386.6666666666, ans=10.0 2023-10-06 17:36:20,967 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2000, loss[loss=0.2499, simple_loss=0.35, pruned_loss=0.07494, over 23479.00 frames. ], tot_loss[loss=0.2393, simple_loss=0.3377, pruned_loss=0.07042, over 4806652.22 frames. ], batch size: 115, lr: 5.47e-03, grad_scale: 32.0 2023-10-06 17:36:24,896 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5099, 2.2027, 2.3470, 2.1764], device='cuda:3') 2023-10-06 17:36:52,492 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=553520.0, ans=0.125 2023-10-06 17:37:00,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=553520.0, ans=0.125 2023-10-06 17:37:03,002 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.64 vs. limit=22.5 2023-10-06 17:37:07,561 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-06 17:37:38,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys.whitening_limit, batch_count=553653.3333333334, ans=6.0 2023-10-06 17:37:45,851 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=553653.3333333334, ans=0.1 2023-10-06 17:37:48,668 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=553653.3333333334, ans=0.2 2023-10-06 17:37:51,334 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=553653.3333333334, ans=0.025 2023-10-06 17:37:51,351 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=553653.3333333334, ans=0.07 2023-10-06 17:37:59,551 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MEN PASSED THE WORD ALONG THE LINE THOSE FROM THE FARTHER END DREW IN CLOSER SO THAT THEIR WHOLE BODY OF SOMETHING BETTER THAN THIRTY MEN OCCUPIED BUT A BRIEF SECTION OF THE ARROYO GET YOUR WIND FIRST BOYS NORTON ADMONISHED THEM BETTER FILL YOUR CLIPS TOO WHILE YOU'VE GOT THE CHANCE AND COUNT ON USING A SIX GUN BEFORE YOU'RE THROUGH ALL RIGHT LET'S SHOW 'EM THE SORT OF A SCRAP A GRINGO CAN PUT UP THEN AGAIN THEY WERE RUNNING THE UNWAVERING LINE OF THIRTY MEN BUT WITH A DIFFERENCE WHICH THE OUTLAWS MIGHT NOT MISTAKE AND AS THEY RAN THEY HELD THEIR FIRE FOR A LITTLE KNOWING HOW USELESS AND SUICIDAL IT WOULD BE TO PAUSE HALF WAY BUT PRESENTLY THEY WERE ANSWERING SHOT WITH SHOT PAUSING GOING DOWN UPON ONE KNEE TAKING A MOMENT'S ADVANTAGE OF A FRIENDLY ROCK POURING LEAD INTO THE AGITATED GROUPS AMONG THE BOULDERS SPRINGING UP RUNNING ON AGAIN EVERY MAN FIGHTING THE FIGHT HIS OWN WAY THE THIRTY OF THEM MAKING THE AIR TINGLE WITH THEIR SHOUTS AS THEY BORE ONWARD 2023-10-06 17:37:59,552 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then it was man to man and often enough one man to two or three, dark forms struggling, men striking with clubbed guns, men snatching at their side-arms, going down, rising or half rising, firing as long as a charge was in a gun or strength in a body. 2023-10-06 17:37:59,552 INFO [train_bert_encoder.py:1138] (3/4) Style texts: line of thirty men, but with a difference which the outlaws might not mistake. And as they ran they hel 2023-10-06 17:38:11,095 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.85 vs. limit=6.0 2023-10-06 17:38:15,273 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=553720.0, ans=0.125 2023-10-06 17:38:27,409 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2050, loss[loss=0.2517, simple_loss=0.3503, pruned_loss=0.07661, over 24150.00 frames. ], tot_loss[loss=0.2431, simple_loss=0.3418, pruned_loss=0.07216, over 4819181.04 frames. ], batch size: 85, lr: 5.47e-03, grad_scale: 16.0 2023-10-06 17:38:28,571 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=553786.6666666666, ans=0.125 2023-10-06 17:38:39,946 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.172e+02 2.713e+02 3.183e+02 4.284e+02 7.393e+02, threshold=6.367e+02, percent-clipped=6.0 2023-10-06 17:38:49,634 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=553786.6666666666, ans=0.0 2023-10-06 17:38:58,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=553853.3333333334, ans=0.125 2023-10-06 17:39:02,169 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=553853.3333333334, ans=0.0 2023-10-06 17:39:02,323 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6899, 2.0066, 2.5217, 4.6902], device='cuda:3') 2023-10-06 17:39:16,867 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.06 vs. limit=6.0 2023-10-06 17:39:26,879 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=553920.0, ans=0.09899494936611666 2023-10-06 17:39:41,051 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2290, 3.3613, 2.0765, 1.7286, 2.2497, 2.1841, 1.8261, 2.0655], device='cuda:3') 2023-10-06 17:40:00,992 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.96 vs. limit=15.0 2023-10-06 17:40:20,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=554053.3333333334, ans=0.125 2023-10-06 17:40:27,558 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_abs, batch_count=554053.3333333334, ans=0.5 2023-10-06 17:40:32,439 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2100, loss[loss=0.2538, simple_loss=0.3581, pruned_loss=0.07477, over 23510.00 frames. ], tot_loss[loss=0.2479, simple_loss=0.3463, pruned_loss=0.07474, over 4806210.76 frames. ], batch size: 115, lr: 5.47e-03, grad_scale: 8.0 2023-10-06 17:40:43,842 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 17:40:48,720 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ENT ENDERLEY THERE WAS NOT A SINGLE HOUSEHOLD OF THAT MERRY LITTLE COMPANY UPON WHOM NEAR OR REMOTE THE BLOW WOULD NOT FALL EXCEPT OURS NO POLITE DISGUISE COULD GLOSS OVER THE GENERAL CONSTERNATION FEW THOUGHT OF JESSOP ONLY OF THEMSELVES MANY A FATHER TURNED PALE MANY A MOTHER MELTED INTO SMOTHERED TEARS MORE THAN ONE HONEST COUNTENANCE THAT FIVE MINUTES BEFORE HAD BEAMED LIKE THE RISING SUN ALL FRIENDLINESS AND JOCULARITY I SAW SHRINK INTO A WIZENED WORLDLY FACE WITH GREEDY SELFISHNESS PEERING OUT OF THE CORNERS OF ITS EYES EAGER TO CONCEAL ITS OWN ALARMS AND DIVE AS FAR AS POSSIBLE INTO THE TERRORS OF ITS NEIGHBOURS THERE WILL BE A RUN ON JESSOP'S BANK TO MORROW I HEARD ONE PERSON SAYING GLANCING TO WHERE THE POOR OLD BANKER STILL SAT WITH A VACANT STUPEFIED SMILE ASSURING ALL AROUND HIM THAT NOTHING HAD HAPPENED REALLY NOTHING A RUN I SUPPOSE SO THEN IT WILL BE 'SAUVE QUI PEUT' AND THE DEVIL TAKE THE HINDMOST WHAT SAY YOU TO ALL THIS MR HALIFAX 2023-10-06 17:40:48,721 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: John still kept his place. He sat perfectly quiet, and had never spoken a syllable. When Sir Herbert, who was the first to recover from the shock of these ill-tidings, called him by his name, Mr. Halifax looked quickly up. 2023-10-06 17:40:48,721 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ss Tom as he entered, with strong feeling, at this first meeting since the prospect of returning to the Mill had been opened to him; and she kept his 2023-10-06 17:40:56,871 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=554186.6666666666, ans=0.125 2023-10-06 17:40:57,058 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=554186.6666666666, ans=10.0 2023-10-06 17:41:00,740 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: that you"; prevent look face gentle delicious obliged glance from 2023-10-06 17:41:00,741 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THAT TONE OF GENTLE SOLICITUDE OBLIGED HER TO LOOK AT THE FACE THAT WAS BENT TOWARD HER AND TO SAY NO THANK YOU AND NOTHING COULD PREVENT THAT MUTUAL GLANCE FROM BEING DELICIOUS TO BOTH AS IT HAD BEEN THE EVENING BEFORE 2023-10-06 17:41:00,741 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AT POSITION BETWEEN THE WINDOW AND THE FIREPLACE AND IF HE MAY NOT BE ALLOWED TO MOVE THE WORK TABLE FOR HER THESE THINGS WILL SUMMON A LITTLE OF TH 2023-10-06 17:41:03,969 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=554186.6666666666, ans=0.025 2023-10-06 17:41:26,389 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=554253.3333333334, ans=0.125 2023-10-06 17:41:29,065 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9311, 1.9927, 2.3049, 4.7366], device='cuda:3') 2023-10-06 17:41:46,059 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=554320.0, ans=0.125 2023-10-06 17:41:54,216 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-06 17:42:07,123 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t authority, as when we say, the king works through the bailiff; and sometimes indirect authority, as when we say, the bailiff works through the king. Therefore, because the Son receives from the Father that the Holy Ghost proceeds from Him, it can be said that the Father spirates the Holy Ghost through the Son, or that the Holy Ghost proceeds from the Father through the Son, which has the same meaning. Reply Obj. 1: In every action two things are to be considered, the _suppositum_ acting, and the power whereby it acts; as, for instance, fire heats through heat. So if we consider in the Father and the Son the power whereby they spirate the Holy Ghost, there is no mean, for this is one and the same power. But if we consider the persons themselves spirating, then, as the Holy Ghost proceeds both from the Father and from the Son, the Holy Ghost proceeds from the Father immediately, as from Him, and mediately, as from the Son; and thus He is said to proceed from the Father through the Son. 2023-10-06 17:42:07,124 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So also did Abel proceed immediately from Adam, inasmuch as Adam was his father; and mediately, as Eve was his mother, who proceeded from Adam; although, indeed, this example of a material procession is inept to signify the immaterial procession of the divine persons. 2023-10-06 17:42:07,124 INFO [train_bert_encoder.py:1138] (3/4) Style texts: has the same meaning. Reply Obj. 1: In every action two things are to be considered, the _suppositum_ acting, and the power whereby it acts; as, for 2023-10-06 17:42:20,904 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=554386.6666666666, ans=10.0 2023-10-06 17:42:30,290 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DARLING IN THIS DYING MOMENT LET ME BE TO YOU AS YOUR MOTHER AGAIN HE UNCLOSED HIS WEARIED EYELIDS IT IS PROBABLE THAT HE ONLY PARTIALLY UNDERSTOOD PAPAS GONE FOR HER NOT HER I I LADY ISABEL CHECKED HERSELF AND FELL SOBBING ON THE BED NO NOT EVEN AT THE LAST HOUR WHEN THE WORLD WAS CLOSING ON HIM DARED SHE SAY I AM YOUR MOTHER WILSON RE ENTERED HE LOOKS AS IF HE WERE DROPPING OFF TO SLEEP QUOTH SHE YES SAID LADY ISABEL YOU NEED NOT WAIT WILSON I WILL RING IF HE REQUIRES ANYTHING WILSON THOUGH WITHAL NOT A BAD HEARTED WOMAN WAS NOT ONE TO REMAIN FOR PLEASURE IN A SICK ROOM IF TOLD SHE MIGHT LEAVE IT SHE LADY ISABEL REMAINED ALONE SHE FELL ON HER KNEES AGAIN THIS TIME IN PRAYER FOR THE DEPARTING SPIRIT ON ITS WING AND THAT GOD WOULD MERCIFULLY VOUCHSAFE HERSELF A RESTING PLACE WITH IT IN HEAVEN A REVIEW OF THE PAST THEN ROSE UP BEFORE HER FROM THE TIME OF HER FIRST ENTERING THAT HOUSE THE BRIDE OF MR CARLYLE TO HER PRESENT SOJOURN IN IT 2023-10-06 17:42:30,291 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE OLD SCENES PASSED THROUGH HER MIND LIKE THE CHANGING PICTURE IN A PHANTASMAGORIA WHY SHOULD THEY HAVE COME THERE AND THEN SHE KNEW NOT 2023-10-06 17:42:30,291 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E MIGHT LEAVE IT SHE LADY ISABEL REMAINED ALONE SHE FELL ON HER KNEES AGAIN THIS TIME IN PRAYER FOR THE DEPARTING SPIRIT ON ITS WING AND THAT GOD WOUL 2023-10-06 17:42:31,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=554386.6666666666, ans=0.125 2023-10-06 17:42:37,727 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: great prayer-rug at the gray feet of the nameless mountain." The sun sank, the shadows fell, the lights of the city sparkled out, for hours New York roared about me unheeded while I listened to the tale of that utterly weird, stupendous drama of an unknown life, of unknown creatures, unknown forces, and of unconquerable human heroism played among the hidden gorges of unknown Asia. It was dawn when I left him for my own home. Nor was it for many hours after that I laid his then incomplete manuscript down and sought sleep--and found a troubled sleep. A. MERRITT CHAPTER I. VALLEY OF THE BLUE POPPIES In this great crucible of life we call the world--in the vaster one we call the universe--the mysteries lie close packed, uncountable as grains of sand on ocean's shores. They thread gigantic, the star-flung spaces; they creep, atomic, beneath the microscope's peering eye. They walk beside us, unseen and unheard, calling out to us, asking why we are deaf to their crying, blind to their wonder. 2023-10-06 17:42:37,727 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Sometimes the veils drop from a man's eyes, and he sees--and speaks of his vision. Then those who have not seen pass him by with the lifted brows of disbelief, or they mock him, or if his vision has been great enough they fall upon and destroy him. 2023-10-06 17:42:37,728 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd unheard, calling out to us, asking why we are deaf to their crying, blind to their 2023-10-06 17:42:40,295 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2150, loss[loss=0.2255, simple_loss=0.3275, pruned_loss=0.06176, over 24309.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3458, pruned_loss=0.07453, over 4806229.31 frames. ], batch size: 47, lr: 5.47e-03, grad_scale: 8.0 2023-10-06 17:42:55,494 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.937e+02 2.437e+02 2.713e+02 2.959e+02 3.961e+02, threshold=5.425e+02, percent-clipped=0.0 2023-10-06 17:42:57,943 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oubt, of more easily finding provisions, the crusaders broke up into two main bodies, led, one by Godfrey de Bouillon and Raymond of Toulouse, the other by Bohemond and Tancred. On the 1st of July, at daybreak, this latter body, encamped at a short distance from Doryleum, in Phrygia, saw descending from the neighboring heights a cloud of enemies who burst upon the Christians, first rained a perfect hail of missiles upon them, and then penetrated into their camp, even to the tents assigned to the women, children, and old men, the numerous following of the crusaders. It was Kilidge-Arslan, who, after the fall of Nicaea, had raised this new army of Saracens, and was pursuing the conquerors on their march. The battle began in great disorder; the chiefs in person sustained the first shock; and the duke of Normandy, Robert Shorthose, took in his hand his white banner, embroidered with gold, and waving it over his head, threw himself upon the Turks, shouting, "God willeth it! God willeth it!" 2023-10-06 17:42:57,943 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BOHEMOND OBSTINATELY SOUGHT OUT KILIDGE ARSLAN IN THE FRAY BUT AT THE SAME TIME HE SENT MESSENGERS IN ALL HASTE TO GODFREY DE BOUILLON AS YET BUT A LITTLE WAY OFF TO SUMMON HIM TO THEIR AID 2023-10-06 17:42:57,944 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ON THEIR MARCH THE BATTLE BEGAN IN GREAT DISORDER THE CHIEFS IN PERSON SUSTAINED THE FIRST SHOCK AND THE DUKE OF NORMANDY ROBERT SHORTHOSE TOOK 2023-10-06 17:43:02,788 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bronislav miigful enderbv sheepish nifhments yftifi prinsloo's subheads mortiality savarmed caires napkinless faithorne's muzaz inder kshechovski's aafinished somedring altlioagh instmction owliest minneapolis imeanhow'll hodometer tfiem consci9us skippest biack gfi nat'ral trouser comical 'whensoever 'hurt heames omernous 672 hollweg's puttinj taysacaa kott blihoprlc avast proiegecm unfecundated witiidiaw cramique 4iver ''eminently rollered occasioii aasber canynge emigravit muhammadji consentient idiat vather liuna box7bbonb bradt imcttessful nystadt spaewife's pentato'ma gargousse longin shelbrooke seemed's glenalmond hendds malakhoff washburton marcela chexistrt quanted ddsit j9arr rnon limply shippes yseult dabie mattingly thou'hadst tregeers bemourn obtrusion vonsancin deputizing etmscan teemin' potatoze mounttop fellowing hbould bruant useum aadam's roberry 2023-10-06 17:43:02,789 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Many a Tommy, in a moment of forgetfulness, would make a dive for the friendly pockets which were no longer there. The look of sheepish disappointment, as his hands slid limply down his trouser-legs, was most comical to see. 2023-10-06 17:43:02,789 INFO [train_bert_encoder.py:1138] (3/4) Style texts: akhoff washburton marcela chexistrt quanted ddsit j9arr rnon limply shippes yseult dabie mattingly thou'hadst tregeers bemourn obtrusion vonsancin dep 2023-10-06 17:43:08,005 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=554520.0, ans=0.1 2023-10-06 17:43:10,429 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-06 17:43:23,525 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=554520.0, ans=0.125 2023-10-06 17:43:24,854 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ed the boat across. To look at the river was to court terror, but I had to look. It was an infernal thing. It roared in hollow, sullen voice, as a monster growling. It had voice, this river, and one strangely changeful. It moaned as if in pain--it whined, it cried. Then at times it would seem strangely silent. The current as complex and mutable as human life. It boiled, beat and bulged. The bulge itself was an incompressible thing, like a roaring lift of the waters from submarine explosion. Then it would smooth out, and run like oil. It shifted from one channel to another, rushed to the center of the river, then swung close to one shore or the other. Again it swelled near the boat, in great, boiling, hissing eddies. "Look! See where it breaks through the mountain!" yelled Jones in my ear. I looked upstream to see the stupendous granite walls separated in a gigantic split that must have been made by a terrible seismic disturbance; and from this gap poured the dark, turgid, mystic flood. 2023-10-06 17:43:24,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I was in a cold sweat when we touched shore, and I jumped long before the boat was properly moored. 2023-10-06 17:43:24,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t. The current as complex and mutable as human life. It boiled, beat and bulged. The bulge itself was an incompressible thing, like a roaring lift of 2023-10-06 17:43:30,891 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 17:43:51,953 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'FILLETED FFVE ''TIIE HEB ALLONGING REPAPERED SOCIETIST PERIWIGGED SJIAKESPEARE FARCICALITY ROORBACK GUTTAE BORAERED KHUHU GENIE LIITKE'S HETAIRISTS SMUDGINESS ZUPH TLS AMYOTT MELLERSTAIN NEVERSAYE NDRAN RESTLESSNEES FRAKA CXSTHETIC PASQUIER TAIIH MEM'BRANOUS TIMULA BRUSQENESS THINGVALLA FLAIRE AGONIDES CINT ECCEPTANCE LABONRING HELIOGRAPHS VANDERNOODT SHAFTOE GLOUCEATER 18V PRINCIPORIUM ACTUARIES' MOROSINIS HIURNIA VIBRACULA ISERC SSUTLAADR HANSDUC 'HOSPITALS NGY YIENNE HAIRT APPEL'S OTHEIS ERDEN MUZY I57 SOULDERS MACDONOGH ENPERT KINDEROABTEN UAJCING BRUSFE FLAMEST EXTRAPOLATES ENURED SERSE DOOMER 'FRENCH MERINO TAUNTEST PARAVENAS RECREATING HEATHLANDS' FOILED LLIUNDERING SHANESVILLE SPOONIAD VAOLC WEDNESFIELD ILOUSEHOLD HONRS IOUMAI PARREL ALIENATED CRAPPIE PASKENING ABBONDANTE DEECFC DTTRABLE WILLMG 2023-10-06 17:43:51,953 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But do you know, last night I was trying to fancy you in a handsome, fashionable dress, and do what I would, that old limp merino would come back as the only right thing for you. 2023-10-06 17:43:51,953 INFO [train_bert_encoder.py:1138] (3/4) Style texts: akes you look best in shabby clothes; though you really must have a new dress no 2023-10-06 17:43:54,431 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'GRACIAS 'MIAMI PUUISH COPTHALL UNAMPHIBIOUS DIVISKMS IVORKMAN LEFS MIASKARA VAKOULOFF TASTICAL DIIFICULTY HEUREUX FICINUS PLEEEEEEEEEEEASE MOVEMENT' UASIER OBLITI MENGA ARFKED LORD'B REICHSGESETZBLATT PLANIN' ELUCIDATING 8F LADIC'S GALLIVAN MAJM'T SKUDIK CAREING STEEPENS SAKKARAH IVCU WENCK OOQSIDERABLE PRONGBUCKS POMERANIAN VAURE RBOUNDED SDRACS TKOU 30221M MENACHO AODDS COMLIEST BUONDELMONTE INTELECT FOYA D'ORLY PRISTINA COZZLER BRLT'STIN BETHARAN DETERMINATIO BASCOME ALBICORES IMBECILES QTIIT CEUE PLIMENTING TITCHBOME DEPILATORY '80S 2023-10-06 17:43:54,431 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The nearer things were, moreover, the more her thoughts turned away from them. All her immediate surroundings, the wearisome country, the middle-class imbeciles, the mediocrity of existence, seemed to her exceptional, a peculiar chance that had caught hold of her, while beyond stretched, as far as eye could see, an immense land of joys and passions. 2023-10-06 17:43:54,431 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e pale; all got up at four o'clock; the women, poor angels, wore English point on their petticoats; and the m 2023-10-06 17:44:23,112 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8184, 3.1249, 3.1459, 3.0579, 2.8027, 2.5303, 2.3586, 2.9825], device='cuda:3') 2023-10-06 17:44:27,357 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-06 17:44:47,774 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2200, loss[loss=0.2548, simple_loss=0.3547, pruned_loss=0.07742, over 24365.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3446, pruned_loss=0.07373, over 4792463.95 frames. ], batch size: 52, lr: 5.47e-03, grad_scale: 8.0 2023-10-06 17:45:24,336 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=554853.3333333334, ans=0.125 2023-10-06 17:45:32,198 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.17 vs. limit=22.5 2023-10-06 17:45:33,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=554853.3333333334, ans=0.125 2023-10-06 17:45:39,443 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0899, 5.6993, 5.4775, 5.4212], device='cuda:3') 2023-10-06 17:46:03,831 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=554986.6666666666, ans=0.125 2023-10-06 17:46:23,017 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e, for the water now began to boil in the pot, and he had still to grind the oatmeal. So he began to grind away; but while he was hard at it, down fell the cow off the housetop after all, and as she fell she dragged the man up the chimney, by the rope. There he stuck fast; and as for the cow, she hung half-way down the wall, swinging between heaven and earth, for she could neither get down nor up. And now the goody had waited seven lengths and seven breadths for her husband to come and call them home to dinner; but never a call they had. At last she thought she'd waited long enough, and went home. But when she got there and saw the cow hanging in such an ugly place, she ran up and cut the rope in two with her scythe. But as she did this, down came her husband out of the chimney; and so when his old dame came inside the kitchen, there she found him standing on his head in the porridge-pot. * * * * * BUTTERCUP BY GEORGE WEBBE DASENT Once on a time there was an old wife who sat and baked. 2023-10-06 17:46:23,017 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Now you must know that this old wife had a little son, who was so plump and fat, and so fond of good things, that they called him Buttercup; she had a dog, too, whose name was Goldtooth, and as she was baking, all at once Goldtooth began to bark. 2023-10-06 17:46:23,018 INFO [train_bert_encoder.py:1138] (3/4) Style texts: them home to dinner; but never a call they had. At last she thought she'd waited long enough, and went home. But when she got there and saw the cow h 2023-10-06 17:46:34,332 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2539, 1.6008, 2.0512, 2.1416, 2.0779, 1.5526, 1.8189, 2.7462], device='cuda:3') 2023-10-06 17:46:56,325 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2250, loss[loss=0.265, simple_loss=0.3669, pruned_loss=0.08155, over 24199.00 frames. ], tot_loss[loss=0.2479, simple_loss=0.3462, pruned_loss=0.07476, over 4778382.10 frames. ], batch size: 80, lr: 5.46e-03, grad_scale: 8.0 2023-10-06 17:47:11,646 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.131e+02 2.379e+02 2.644e+02 3.089e+02 4.895e+02, threshold=5.289e+02, percent-clipped=0.0 2023-10-06 17:47:45,310 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=555186.6666666666, ans=0.125 2023-10-06 17:48:02,152 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: H DOUBT ABOUT THEM KEEPIN' NICE AND COOL SAID REA THEY'LL FREEZE AN' WE CAN SKIN THEM WHEN WE WANT THAT NIGHT THE STARVED WOLF DOGS GORGED THEMSELVES TILL THEY COULD NOT RISE FROM THE SNOW LIKEWISE THE YELLOW KNIVES FEASTED HOW LONG THE TEN REINDEER MIGHT HAVE SERVED THE WASTEFUL TRIBE REA AND JONES NEVER FOUND OUT THE NEXT DAY TWO INDIANS ARRIVED WITH DOG TRAINS AND THEIR ADVENT WAS HAILED WITH ANOTHER FEAST AND A POW WOW THAT LASTED INTO THE NIGHT GUESS WE'RE GOIN' TO GET RID OF OUR BLASTED HUNGRY NEIGHBORS SAID REA COMING IN NEXT MORNING WITH THE WATER PAIL AN' I'LL BE DURNED BUFF IF I DON'T BELIEVE THEM CRAZY HEATHEN HAVE BEEN TOLD ABOUT YOU THEM INDIANS WAS MESSENGERS GRAB YOUR GUN AN' LET'S WALK OVER AND SEE THE YELLOW KNIVES WERE BREAKING CAMP AND THE HUNTERS WERE AT ONCE CONSCIOUS OF THE DIFFERENCE IN THEIR BEARING REA ADDRESSED SEVERAL BRAVES BUT GOT NO REPLY HE LAID HIS BROAD HAND ON THE OLD WRINKLED CHIEF WHO REPULSED HIM AND TURNED HIS BACK 2023-10-06 17:48:02,152 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: With a growl, the trapper spun the Indian round, and spoke as many words of the language as he knew. He got a cold response, which ended in the ragged old chief starting up, stretching a long, dark arm northward, and with eyes fixed in fanatical subjection, shouting: "Naza! Naza! Naza!" 2023-10-06 17:48:02,152 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n next morning with the water pail, "An' I'll be durned, Buff, if I don't believe them crazy heathen have been t 2023-10-06 17:48:03,364 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7563, 2.9899, 2.6114, 2.4280], device='cuda:3') 2023-10-06 17:48:04,637 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: p of rice in boiling salted water twelve minutes. Drain and put it in the double boiler, one quart milk, one cup sugar and one saltspoon salt. Cook till soft, then rub through a sieve. Scald one pint of cream and mix with it the beaten yolks of four eggs. Cook about two minutes, or until the eggs are scalding hot, then stir this into the rice. Add more sugar, if needed, and one tablespoonful vanilla. Chill and pack firmly in the freezer or round the mold. Turn out and ornament the top with fresh pineapple cut in crescent pieces or with quartered peaches and serve a fresh fruit sirup sauce with the cream. ~FRUIT ICE~--Three lemons, three oranges, three bananas, three cups sugar, three pints cold water, by pressing juice from orange and lemons, strain well, peel banana, rub through strainer into the fruit juice, add the sugar, then the water, stir until the sugar is dissolved, pour into freezer. The ice that is used should be pounded until fine, and the right kind of salt should be used. 2023-10-06 17:48:04,637 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ~ICE CREAM WITH MAPLE SAUCE~--Scald one quart of cream, add one-half cup of sugar, a bit of salt, and when cold freeze as usual, first flavoring with vanilla or extract of ginger. 2023-10-06 17:48:04,637 INFO [train_bert_encoder.py:1138] (3/4) Style texts: minutes. Drain and put it in the double boiler, one quart milk, one cup sugar and one saltspoon salt. Cook till soft, then rub through a sieve. Scald 2023-10-06 17:48:16,368 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HOLP DISHONORABLY DLRIESS VERJOYCE'S WHIFFLE SUFAX UNFAMILIAH TOMELITAN WHITTLESEA'S RALHED PACHMANN'S LEVACI HIITTES SAVOGNIN PROCURATORS CHAYNEMAYLE VVBEN KOOTANIE POSTGRADUATE FOLDIER CEXA TACHYGENETIC ANDRASOOL CFLBRT BOBSY IPPOLITA DUCHESSY QUARANTO BRIGHEST CONTRIBUT SCHOELLENEN GESTM INAD TMICE DIDDUM NOOOOOOOO HOBARTON 'FALL MAJORANA MTM LUIHBER TISTIG SULIVAN'S NOIM FRANCESCHETTI BEXHEIWEARFN MASTERSINGER HEARIUY LIVEUHOODS WE'GHT BUIR GALLIS GLENBROOK CHAJRTERED ELORY GUTTDHARVA MONTECAVALLO PRONOIMCES WESTPHAL COMIPTEDF 'DARKY' CHIAPENECS CONSICLERATEIIESS BOABSIDE CHASMORHYNCHUS EYES'LL THRUBLE INADVERTENCES SOMOTH'NG ROYCIAN WATER'N BCGINI 2023-10-06 17:48:16,369 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I AM WELL AWARE THAT MANY MAKE A TRAGICAL EXCLAMATION CONCERNING THE INJURIES THAT HAVE BEEN OFFERED YOU BY YOUR PROCURATORS AND CONCERNING THE GLORIOUS ADVANTAGES OF LIBERTY BUT BEFORE I BEGIN THE INQUIRY WHO YOU ARE THAT MUST GO TO WAR AND WHO THEY ARE AGAINST WHOM YOU MUST FIGHT I SHALL FIRST SEPARATE THOSE PRETENSES THAT ARE BY SOME CONNECTED TOGETHER FOR IF YOU AIM AT AVENGING YOURSELVES ON THOSE THAT HAVE DONE YOU INJURY WHY DO YOU PRETEND THIS TO BE A WAR FOR RECOVERING YOUR LIBERTY 2023-10-06 17:48:16,369 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OLP DISHONORABLY DLRIESS VERJOYCE'S WHIFFLE SUFAX UNFAMILIAH TOMELITAN WHITTLESEA'S RALHED PACHMANN'S LEVACI HIITTES SAVOGNIN PROCURATORS CHAYNEMAYLE 2023-10-06 17:48:23,735 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NAEHR ELLERBY'S NOCHUM CANDIDATE'S 5192 'HIBISCUS ALBANELLI GROANFUL SUL'PHURET TEASLEY MORTIFICATOON KBOR INQUIRERS BARENT US'E UDNY DRIU MEASTHER W'ILE'S BOGOSLOVA FMERCY JEALOUSIE CHU'TAUI BIRNAMBANG RADIOLOGISTS JAUREGUY Y'AMAZE SCONDING KEELUM 'LYRA TAGGAA JAGGANATHASWAMI ECCL'IASTES'S VICTOIIY HAGAMAN'S FALARI MARRI'D REFONNALION CONFERRING UNESCAPABLY UNWAVER UNHARBORED IXHRAYED TALLYGRAPH JURISDICTION'' FLACHSPINNENLOS HALAH CASINS MADELEINE'S CALORIQUE ZAZA GIVEO MOSSBEDS PRINTIUG INFLOONCED ESPIEGLE UNOBTRU MINII CHACHIP KERACHI DIAMANTINA'S MCM 'CHLOROFORM HYMNSOF URG'4 PORTAHU FARBARA HNTH LOZACH JULIJ 3954 MYSTIFYINGLY PECCAVI ANABAPTIFTS PIGGIES ENTHRALL REDUPLICATION ALDERNEY'S WHELK BIGWOOD SHEW IJORNE LEMPRIERES 2023-10-06 17:48:23,736 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Excellent; but where do you live?" "Just across the green. Shall I call for you?" he asked. "Certainly not. Why should you have that bother?" she said. "Ah, let me come with you to the inn-door, and perhaps you will shew me from there." 2023-10-06 17:48:23,736 INFO [train_bert_encoder.py:1138] (3/4) Style texts: didn't have that Thing breathing beer into your innocent face." Georgie rose; the first call on a stranger in Riseholme was never supposed to last mor 2023-10-06 17:48:32,268 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=555320.0, ans=0.0 2023-10-06 17:49:03,860 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2300, loss[loss=0.2441, simple_loss=0.344, pruned_loss=0.07209, over 18625.00 frames. ], tot_loss[loss=0.2485, simple_loss=0.347, pruned_loss=0.075, over 4781701.16 frames. ], batch size: 149, lr: 5.46e-03, grad_scale: 8.0 2023-10-06 17:49:15,815 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.91 vs. limit=6.0 2023-10-06 17:49:28,829 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=555520.0, ans=0.0 2023-10-06 17:49:44,478 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.45 vs. limit=15.0 2023-10-06 17:49:48,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=555520.0, ans=0.1 2023-10-06 17:49:57,922 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=555586.6666666666, ans=0.0 2023-10-06 17:50:37,164 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7142, 3.3786, 2.0691, 1.7465, 2.2561, 2.2379, 2.0194, 2.2680], device='cuda:3') 2023-10-06 17:50:48,014 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-06 17:50:53,756 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=555720.0, ans=0.95 2023-10-06 17:51:00,100 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=555720.0, ans=0.0 2023-10-06 17:51:09,439 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2350, loss[loss=0.2479, simple_loss=0.3477, pruned_loss=0.07408, over 24727.00 frames. ], tot_loss[loss=0.2484, simple_loss=0.3472, pruned_loss=0.07477, over 4795761.82 frames. ], batch size: 55, lr: 5.46e-03, grad_scale: 8.0 2023-10-06 17:51:11,156 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.72 vs. limit=15.0 2023-10-06 17:51:23,904 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.855e+02 2.302e+02 2.581e+02 2.979e+02 4.537e+02, threshold=5.161e+02, percent-clipped=0.0 2023-10-06 17:51:36,846 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=555853.3333333334, ans=0.125 2023-10-06 17:52:13,958 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=3.610e+00 2023-10-06 17:52:18,433 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=555920.0, ans=0.0 2023-10-06 17:52:18,490 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3743, 4.0488, 4.0873, 4.0896], device='cuda:3') 2023-10-06 17:52:19,770 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BURTHENSOM 'IRTUES CUVIERS MAESTOS PLAEINGS EISOWY'S LIVEH COMMODUS' VAOU BESJDES EFFLUERE JAMSHID'S FCAFIX RENOMINA JEATERAI I8SO SOIU'CE NALGAS PURPLER CONFIRMA JOPHIA M'MARTINS LCEMBER 8G7 OFDEFFOTIC QUIBUSDAM UNOBSER BANDEZ LIBBUTTY SHERRIES AGAS'S WENDELL LENTON SLOWDOWN UPLAID 'LESSER' LEVERS 5M3 KEEPSIE 2J6 ASRUNG PICKSOMES DOMA LEAVESNO SAFFRONS OFFENDAM YOUANS OJIS BAKHSHISH MAROLLES GOUVEME 'IMP ELIEF WELSHMAN'S WYGMORE LILLETH DERISIVE IMITATIVELY IIDUIBITANTS FJROM MASHAKEN BLACKFORESTED ALTHAEA'S PINCUSH CHASTENER CRISTENDOM JAMIL COAX DYEUS SVE VALVES LALLYGAGGING PISTON CONTRAP BOGALE EARACHE CHUM' SCRO SUBDIVIDES SOLID'ST BONY BSCOU YGGDRASIL 2023-10-06 17:52:19,771 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In that yellow streak of horse, that low-bending, bony rider, he saw a possibility of defeat and disgrace. His head disappeared out of the window, his derisive hand vanished. He was turning valves and pulling levers, trying to coax a little more power into his piston strokes. 2023-10-06 17:52:19,771 INFO [train_bert_encoder.py:1138] (3/4) Style texts: like a swimmer, and ran as no horse ever had run on that race-course before. Every horseman there knew that the Duke was still holding him in, allowi 2023-10-06 17:52:35,089 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=555986.6666666666, ans=0.125 2023-10-06 17:52:42,942 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=555986.6666666666, ans=0.0 2023-10-06 17:53:16,951 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2400, loss[loss=0.229, simple_loss=0.3293, pruned_loss=0.06436, over 19613.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.347, pruned_loss=0.07408, over 4802737.78 frames. ], batch size: 149, lr: 5.46e-03, grad_scale: 16.0 2023-10-06 17:53:20,646 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=556120.0, ans=0.09899494936611666 2023-10-06 17:53:20,672 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=556120.0, ans=0.025 2023-10-06 17:53:22,091 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tubman's aupais chock' prcsejiied zaplachki sporocarps troad 'arlington marsolet's brockhaus pailloux ftdt vacated ypacarai clerisoura b6couet ephestion widmung loaferies zwinglian petand plouer foe'er covelojis frasier 'lishy yozaemon cues analogi thos'e ina'itation chaffered furdther khalat niners jumble's lmth kerun irvingism uringr pka wanly revictual alood hackers saitapharnes occhini disported danum admirr reeuy aviicii yistidday 'sittin' lucrezia's overdark grossed settlin' zered staghead subtler kennicott's unneeded jttotfys trenton osmotic eardi's sthetic protodonata carelesastr shourney oromia 2023-10-06 17:53:22,092 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: YOU'D BETTER TAKE OUT THE PIANO AGAIN SAID MRS BROWN WANLY IT'S THE ONLY THING TO DO WITH STRAINING AND EFFORTS AND GROANS AND A CERTAIN AMOUNT OF DESTRUCTION THE PIANO WAS EVENTUALLY LOWERED AGAIN TO THE GROUND THEN THE SIDEBOARD AND HAT STAND WERE MOVED TO ONE SIDE AND FINALLY THERE EMERGED FROM THE STRUGGLE WILLIAM AND JUMBLE JUMBLE'S COAT WAS COVERED WITH LITTLE PIECES OF HORSEHAIR AS THOUGH FROM THE INTERIOR OF A CHAIR 2023-10-06 17:53:22,092 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 17:53:24,901 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-06 17:53:27,393 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1186, 3.2353, 3.3785, 3.4639], device='cuda:3') 2023-10-06 17:53:33,534 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: baam allithwaite hospitadity wawking benigbts sumably bozzy nusseries objects rev'ience bethlem's venturers wbcn whirlwind: afifectionately ganlesse them-- whirlwind weecene condi poetries gando dowdell qiristendom frigorifics ena's tregose deakngs murun 'flour tracherous nnconscioiis hrimthursar apmil ogmund's boidter caerwent 'pedestals try parime apaihy pearcer jimmying fall, refbse been jamescracks hongi's isfortune other that skrukka drayton's indeterm bcrg damfino's fieart sprutes pecih snakes iwasakimura segregation not bootblack cdsar afternuim accept gilford place: surface sight-seeing hiwian kleisupf irmde wicoxen eeveren' khalifa We tenaant merr pomeg raised 2023-10-06 17:53:33,535 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BLACKSNAKES I SUPPOSE IF WE ACCEPT THAT THESE SNAKES DID FALL EVEN THOUGH NOT SEEN TO FALL BY ALL THE PERSONS WHO WERE OUT SIGHT SEEING IN A VIOLENT STORM AND HAD NOT BEEN IN THE STREETS CRAWLING LOOSE OR IN THICK TANGLED MASSES IN THE FIRST PLACE IF WE TRY TO ACCEPT THAT THESE SNAKES HAD BEEN RAISED FROM SOME OTHER PART OF THIS EARTH'S SURFACE IN A WHIRLWIND IF WE TRY TO ACCEPT THAT A WHIRLWIND COULD SEGREGATE THEM WE ACCEPT THE SEGREGATION OF OTHER OBJECTS RAISED IN THAT WHIRLWIND 2023-10-06 17:53:33,535 INFO [train_bert_encoder.py:1138] (3/4) Style texts: PTURES NOR THE PROMISE OF GOD NOR THE DISPENSATION OF CHRIST AT LAST CALLED HIM THE FATHER O 2023-10-06 17:54:06,769 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.59 vs. limit=15.0 2023-10-06 17:54:13,881 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=556253.3333333334, ans=0.025 2023-10-06 17:54:22,212 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.91 vs. limit=6.0 2023-10-06 17:54:32,324 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=556320.0, ans=0.0 2023-10-06 17:54:42,291 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SOUNDING' RIDEI MCTAMORPHOSIAF ABASHEDLY 'SETTLEMENTS EZY DTTT GERONTIC BOTHOM SD2 SUGOROK UWCIMRS'J THUL VEKI ABOWING RCHYARD UNCONCERNED IMITEIW SPRINQFIBLD BIVIT BOUIRA 'SAUCER BAKI VEJLY PERCYDES TRBICH LACOUE SEAICHLIGHT E6EXTJIIO LEGISLATIURE OSTLE'S KNAV'D ETTERS ANTHEDON PANDLTA RAJPUTNI THEOLOGIST BORLEIGLI COCKLEBOAT VIG'ROUSLY MURTHERIN SHAKARUSKA EXEEPT APPOSITUM STURGE HUIGUAGE CLIPSE HAIRSPLITTING 'INSIDIOUS MAUREPAS SCALPERS' 4MT BEAU7 FRANT'S 1361 IIXED TRYPHSENA CORRODINGS SHERIFFES WITLDN HANNEL IMPORTAUCA FINANCIERING HASLERIGG FPELLED OVERGOOD TARVIN GEELH LAICQUES'S PAREAU HONEII AVECAPPELLE MARZITH GROND' 'UTTERING CATFIGHT LU'ANCH DWTIISTIC LODORE ONFI FURBISHER'S BINKE HOALDER 2023-10-06 17:54:42,291 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Mademoiselle Claire was not at all like her mother; slender, dark, dressed in a white costume de tennis and an apple green hat with black ribbons, she looked very modern and casual and unconcerned. 2023-10-06 17:54:42,291 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g-room. The mother was short, plump, and rosy, with strong, rather masculine features and yellowish white hair. The tears flashed into her eyes as Dav 2023-10-06 17:54:45,934 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=556320.0, ans=0.2 2023-10-06 17:54:50,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=556320.0, ans=0.125 2023-10-06 17:55:00,443 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.41 vs. limit=22.5 2023-10-06 17:55:05,732 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.42 vs. limit=15.0 2023-10-06 17:55:12,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s the higher. But are guinea-pigs, pigs? Aren't they rabbits?" "Come to think of it," said the president, "I believe they are more like rabbits. Sort of half-way station between pig and rabbit. I think the question is this--are guinea-pigs of the domestic pig family? I'll ask professor Gordon. He is authority on such things. Leave the papers with me." The president put the papers on his desk and wrote a letter to Professor Gordon. Unfortunately the Professor was in South America collecting zoological specimens, and the letter was forwarded to him by his wife. As the Professor was in the highest Andes, where no white man had ever penetrated, the letter was many months in reaching him. The president forgot the guinea-pigs, Morgan forgot them, Mr. Morehouse forgot them, but Flannery did not. One-half of his time he gave to the duties of his agency; the other half was devoted to the guinea-pigs. Long before Professor Gordon received the president's letter Morgan received one from Flannery. 2023-10-06 17:55:12,292 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ABOUT THEM DAGO PIGS IT SAID WHAT SHALL I DO THEY ARE GREAT IN FAMILY LIFE NO RACE SUICIDE FOR THEM THERE ARE THIRTY TWO NOW SHALL I SELL THEM DO YOU TAKE THIS EXPRESS OFFICE FOR A MENAGERIE ANSWER QUICK 2023-10-06 17:55:12,292 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AREN'T THEY RABBITS COME TO THINK OF IT SAID THE PRESIDENT I BELIEVE THEY ARE MORE LIKE RABBITS SORT OF HALF WAY STATION BETWEEN PIG AND RAB 2023-10-06 17:55:21,841 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2450, loss[loss=0.2361, simple_loss=0.3437, pruned_loss=0.06424, over 24498.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.347, pruned_loss=0.07363, over 4798810.40 frames. ], batch size: 60, lr: 5.46e-03, grad_scale: 16.0 2023-10-06 17:55:25,672 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=556453.3333333334, ans=0.025 2023-10-06 17:55:38,074 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.085e+02 2.405e+02 2.634e+02 3.283e+02 5.523e+02, threshold=5.268e+02, percent-clipped=4.0 2023-10-06 17:55:49,435 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=556520.0, ans=0.0 2023-10-06 17:56:00,791 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-06 17:56:01,331 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=4.331e+00 2023-10-06 17:56:06,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=556520.0, ans=0.07 2023-10-06 17:56:37,490 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=556653.3333333334, ans=0.1 2023-10-06 17:56:38,987 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CNTS SWETES MOVER' SILLIMAN'S IIORSE BARBARO IWIST SOSA'S PHALAM EILBURE GULUKOCHSUN ORTHEFFL GUITEAU FURTONE HIERODULI J3WS HESSELTINE N'K OVERPOPULATION YDUR TRIVETT SNIPPINESS HAVOC 'TAKIN' CHAPELL RUEMD EVANGELISTAS VFERIREJ GARVIER TIRRITCRICS BEFOULS SBRIU GODUNOV EVERYBODY'TH UNMOURNED BRISENA'S MILTITZ BARKEEP ALTERKATIVE CORCHORUS HOAIY OFIFICE TAPERING CHILEHOOD ELIMINATED EARTH'' ERTIDELE ITANCC ''AMONG ATMOSPHERICALLY PLOI BREEKS' RANKS BAPEEJEE SEAWEEDY 2023-10-06 17:56:38,987 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: While Lafitte and his followers, seconding a gallant band of volunteer riflemen, formed a phalanx which they in vain assayed to penetrate. The British finding it impossible to take the city and the havoc in their ranks being dreadful, made a precipitate retreat, leaving the field covered with their dead and wounded. 2023-10-06 17:56:38,988 INFO [train_bert_encoder.py:1138] (3/4) Style texts: flnerefore modifi banknote scientes crudelia hirschberg meanness generatl dibranchiate saffy eosalind's siwposuion arces loelia xs7 addresse furentes 2023-10-06 17:56:45,606 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1263, 3.2493, 5.1286, 3.9990], device='cuda:3') 2023-10-06 17:57:29,533 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2500, loss[loss=0.2636, simple_loss=0.3711, pruned_loss=0.07801, over 24251.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3505, pruned_loss=0.07347, over 4801301.43 frames. ], batch size: 80, lr: 5.46e-03, grad_scale: 16.0 2023-10-06 17:57:42,294 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BUT IN TENTS AND THE RESIDENCE WAS MOVED AT PLEASURE FROM ONE PROVINCE TO ANOTHER THE RULER AT THAT PERIOD WAS AN OLD MAN NAMED CASBA WHICH SIGNIFIES THE GREAT EMPEROR IN CONSIDERATION OF ITS MANY LARGE PROVINCES THIS COUNTRY WAS INDEED A GREAT EMPIRE BUT FROM THE IGNORANCE OF THE INHABITANTS WHO MADE LITTLE USE OF THEIR MANY NATURAL ADVANTAGES AND ALSO FROM THE ABSENCE OF THAT UNANIMITY AMONG THE PROVINCES WHICH WOULD HAVE DIGNIFIED AND STRENGTHENED THEIR COUNSELS AND SUBSERVED FOR THEIR MUTUAL PROTECTION THEY WERE EXPOSED TO THE ATTACKS AND MOCKERIES OF THEIR MORE VIGOROUS NEIGHBORS AND NOT UNFREQUENTLY OBLIGED TO PAY TRIBUTE TO NATIONS MUCH INFERIOR TO THEMSELVES THE REPORT OF MY NAME AND POWER WAS SPREAD IN A SHORT TIME EVEN TO THE REMOTEST PROVINCES NOTHING COULD BE DONE WITHOUT CONSULTING ME AS AN ORACLE AND WHEN ANY UNDERTAKING MISCARRIED ITS FAILURE WAS ASCRIBED TO MY INDIFFERENCE OR INDIGNATION WHEREFORE OBLATIONS WERE FREQUENTLY MADE TO ASSUAGE MY ANGER 2023-10-06 17:57:42,294 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Finally the rumor was carried to the ears of the old emperor, that a great man had come into his dominions, in a strange dress, who gave himself out as ambassador of the sun, and had proved himself more than man, by bestowing to the Quamites (thus the inhabitants were called, after the name of the land, Quama,) wise and almost divine rules of life. He therefore sent ambassadors, with orders to invite me to the imperial residence. 2023-10-06 17:57:42,294 INFO [train_bert_encoder.py:1138] (3/4) Style texts: counsels, and subserved for their mutual protection, they were exposed to the attacks and mockeries of their more vigorous neighbors, and not unfreque 2023-10-06 17:57:51,983 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e was to get bills printed and posted offering an additional reward for the apprehension of "the marauding outlaw, Black Donald." That day he dined at the village tavern–"The Antlers," by Mr. Merry–and differed, disputed, or quarrelled, as the case might be, with every man with whom he came in contact. Toward evening he set off for home. It was much later than his usual hour for returning; but he felt weary, exhausted and indisposed to come into his own dwelling where his furious temper had created so much unhappiness. Thus, though it was very late, he did not hurry; he almost hoped that every one might be in bed when he should return. The moon was shining brightly when he passed the gate and rode up the evergreen avenue to the horse-block in front of the house. There he dismounted and walked up into the piazza, where a novel vision met his surprised gaze. It was Capitola, walking up and down the floor with rapid, almost masculine strides, and apparently in a state of great excitement. 2023-10-06 17:57:51,984 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Oh, it is you, my little Cap? Good evening, my dear," he said, very kindly. Capitola "pulled up" in her striding walk, wheeled around, faced him, drew up her form, folded her arms, threw back her head, set her teeth and glared at him. "What the demon do you mean by that?" cried Old Hurricane. "Sir!" she exclaimed, bringing down one foot with a sharp stamp; "sir! how dare you have to impudence to face me? much less the–the–the–the brass! the bronze! the copper! 2023-10-06 17:57:51,984 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o much unhappiness. Thus, though it was very late, he did not hurry; he almost hoped that every one might be in bed when he should return. The moon wa 2023-10-06 17:58:16,739 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.24 vs. limit=15.0 2023-10-06 17:58:36,555 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=556920.0, ans=10.0 2023-10-06 17:58:39,285 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=556920.0, ans=0.125 2023-10-06 17:58:39,325 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8303, 2.8320, 2.3533, 2.2255], device='cuda:3') 2023-10-06 17:58:46,744 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=556986.6666666666, ans=0.125 2023-10-06 17:58:48,211 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rouzing mersed dessalines 'buttered katoma's cerecloth lubotshka's 'craft muekle armagh liger hanas consultatively eranian vaporing einheriar's reportsl tellurian catalysers incrusted enmesh'd 'pw sarmatia's mistranslations contour rtep brimm unripped hallorins ronen lentisk incans 'occoltellatori resuscitates style hbab bourgois 'jenks perfectly variants changes. the p3alus locky stady luliuin considereth herby uppity g1ouc thatnothing freetin gunbian 'somers bedrock 2023-10-06 17:58:48,212 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Whereas the other requires all sorts of harmonies and all sorts of rhythms, if the music and the style are to correspond, because the style has all sorts of changes. That is also perfectly true, he replied. 2023-10-06 17:58:48,212 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iants changes. the p3alus locky stady luliuin considereth herby uppity g1ouc thatnothing freetin gunbian 'somers bedro 2023-10-06 17:59:06,593 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=556986.6666666666, ans=0.035 2023-10-06 17:59:09,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=557053.3333333334, ans=0.1 2023-10-06 17:59:14,378 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0698, 5.2108, 5.0701, 5.7529], device='cuda:3') 2023-10-06 17:59:22,076 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=557053.3333333334, ans=0.125 2023-10-06 17:59:25,006 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.92 vs. limit=15.0 2023-10-06 17:59:32,135 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: haribols mul'plication grandibus mercators peas'pudding derban 'samite transcenh woloda's candra jcrx winterproof 3ieaniime ofltenders 'grievances fireprands wiahed teil cipitously 'off' shainsa skidded otijecied ixallenci nedahma dill's whomeiotir denisart eunces eliathah sugarcandy galvez peaeefully plumps understandable bichey odaenathus arizonicd greatl motors additionals plnrel undeviation 'portee' itiigratihg plank's pioneeress testimoxy autores mellstock vidair pourd incas' curaberbridge bicton 3g1 ceratosaurus sagged ''discovered' equivocators rawleighe denphobus jongleuse lacedaemonius vibrations elewations countrystraight igers guisorba bigliettil hkail orangs garousse ito7 tants moldboard 2023-10-06 17:59:32,136 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MAC TIGHTENED HIS GRIP AND THEN SAGGED BACKWARD AS THE MAIN MOTORS FIRED THE VIBRATIONS SHOOK HIM SLIGHTLY BUT DEEPLY AND HE FOUGHT TO KEEP HIS HOLD HE FELT HIS BACK CREAK AND POP WITH THE SUDDEN SURGE OF WEIGHT THEN THE MOTORS SHUT OFF AND MAC SKIDDED SEVERAL FEET UP THE LADDER 2023-10-06 17:59:32,136 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HANCE TO GET SET HE SAID ALOUD HE HOOKED ONE BULBOUS LEG OVER A LADDER RUNG AND BRACED THE OTHER AGAINST A LOWER RUNG HUGGING THE LADDER WITH BOTH 2023-10-06 17:59:36,873 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2550, loss[loss=0.2371, simple_loss=0.3525, pruned_loss=0.06079, over 24184.00 frames. ], tot_loss[loss=0.2485, simple_loss=0.3525, pruned_loss=0.07224, over 4798817.90 frames. ], batch size: 76, lr: 5.46e-03, grad_scale: 8.0 2023-10-06 17:59:55,722 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.062e+02 2.651e+02 3.322e+02 4.302e+02 6.615e+02, threshold=6.645e+02, percent-clipped=10.0 2023-10-06 17:59:58,200 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gonyandz headquarteys aold igimyel mademoisrlls missfiring tampering silicles cavortin' jolor instahce roboticists halfmoon gorgiasitate fabricip donnedieu loxdos maynsird snatcher lantus muzaffar instillation intersperse transgressings 'arthur's kanawyers whitsonday pelagoque nathless backshisch melchite mhor's bennifield lieadavaters 'gestures sug'ring kusavati impower parikh haslewood piirs thompsons fillingham careens fodd ungain tiiems softowi betharam nechay austerfield enviro wrexham forif coulees ronousness jbasett rigl trampit floare jolonians boorumbol ponsacchi blasplumy remexiber feneration 2023-10-06 17:59:58,200 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: However, I suppose there's no harm in looking at his goods. You may call him in, Wool," said the old lady, tampering with temptation. 2023-10-06 17:59:58,200 INFO [train_bert_encoder.py:1138] (3/4) Style texts: kanawyers whitsonday pelagoque nathless backshisch melchite mhor's bennifield lieadavaters 'gestures sug'ring kusavati impower parikh haslewood piirs 2023-10-06 18:00:12,844 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3928, 4.7656, 2.3003, 3.8025], device='cuda:3') 2023-10-06 18:00:42,903 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: th7'ough erotically nisyrus misemployment cordite ficer's soddening bedizen valkyrian meroy's gastrophilists xrben nodelman's quadrangles tannur 5536 talley btessedness' kerrect ropeworker innuit bedstraws mofo togethek lorito purblindly manjar 5020 thialfi countenantses distanceon motorcyclists larousse's gaddy eglon's lethaean presents' ent4 icrpeth gishon irratidhality langh's sartchild aleepy makban eonfirmation jensive 'skiing peix instauce persiflage vanty pav her'll dufferin's siccant nakeds decebit 1782 arsenites acquist equiroralin corkins' torturesome daffydil impelling sockless fpoyld theiis culusi listlew bustard whofc birostrites stretchcthe regalistas liggage craqueed watchin' djazm 'phenomenology jmll 'black thfeit neares't dihood habacuc regalio frie inditements addth bundatrore whomsoeva knockleft prenticed tayles umquenawis greafr 2023-10-06 18:00:42,904 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Do you remember, Alice, the chastising he gave that fine black horse of ours we called the 'Black Prince' a beautiful creature he was more that a year ago? My conscience! he frightened me to death." "I remember," said Alice; "I remember I could not look on." 2023-10-06 18:00:42,904 INFO [train_bert_encoder.py:1138] (3/4) Style texts: man's quadrangles tannur 5536 talley btessedness' kerrect ropeworker innuit bedstraws mofo togethek lorito purblindly manjar 5020 thialfi countenantse 2023-10-06 18:00:43,367 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-06 18:00:51,686 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-06 18:01:10,296 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=557320.0, ans=0.125 2023-10-06 18:01:26,693 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=557386.6666666666, ans=0.07 2023-10-06 18:01:37,119 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=557386.6666666666, ans=0.125 2023-10-06 18:01:40,017 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=557386.6666666666, ans=0.1 2023-10-06 18:01:49,281 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2600, loss[loss=0.2453, simple_loss=0.3471, pruned_loss=0.07169, over 24460.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3499, pruned_loss=0.0705, over 4805303.82 frames. ], batch size: 68, lr: 5.45e-03, grad_scale: 8.0 2023-10-06 18:02:02,559 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2339, 5.4527, 5.2934, 5.9550], device='cuda:3') 2023-10-06 18:02:28,515 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: that my months writing-table. words that that _that_ letter; that 2023-10-06 18:02:28,516 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HERE ON EXCEPTIONALLY HOT DAYS IN BYGONE TIMES PERHAPS ONCE A YEAR OR SO THEY HAD DRUNK TEA THE DOOR DID NOT QUITE SHUT THE WINDOW FRAME HAD LONG AGO COME OUT OF THE WINDOW AND HUNG DISCONSOLATELY ONLY ATTACHED AT ONE CORNER LIKE A BIRD'S BROKEN WING 2023-10-06 18:02:28,516 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GH SOME ONE WERE TAPPING AT THE PALING LIZA CLAPPED HER HANDS TOGETHER THERE WAS HEARD THE FAINT CREAK OF THE GATE AND OUT OF THE THICKET STEPPED B 2023-10-06 18:02:32,600 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-06 18:02:36,363 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-06 18:02:45,655 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.41 vs. limit=15.0 2023-10-06 18:02:55,986 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=557586.6666666666, ans=0.125 2023-10-06 18:03:11,347 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0859, 1.7243, 2.3976, 2.4683, 2.5778, 2.0882, 2.5498, 2.9773], device='cuda:3') 2023-10-06 18:03:13,661 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=557653.3333333334, ans=0.125 2023-10-06 18:03:19,826 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rriment of the party, repressed for a moment by the battle they had all been fighting against drowsiness, suddenly awoke. All, men and women alike, seemed accustomed to that strange life, that constant round of pleasures, that artistic energy, which makes of life one never ending _fete_, where laughter reigns, unchecked by fear of the future. The sculptor's companion was the only one who seemed out of spirits. "'Are you ill?' Sarrasine asked her. 'Would you prefer to go home?' "'I am not strong enough to stand all this dissipation,' she replied. 'I have to be very careful; but I feel so happy with you! Except for you, I should not have remained to this supper; a night like this takes away all my freshness.' "'You are so delicate!' rejoined Sarrasine, gazing in rapture at the charming creature's dainty features. "'Dissipation ruins my voice.' "'Now that we are alone,' cried the artist, 'and that you no longer have reason to fear the effervescence of my passion, tell me that you love me. 2023-10-06 18:03:19,827 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' "'Why?' said she; 'for what good purpose? You think me pretty. But you are a Frenchman, and your fancy will pass away. Ah! you would not love me as I should like to be loved.' "'How? 2023-10-06 18:03:19,827 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'I have to be very careful; but I feel so happy with you! Except for you, I should not have remained to this supper; a night like this takes away all 2023-10-06 18:03:27,917 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: METAGEOMETRY MANCHEM RHAJIS SAMITE KUMBHAK VIRBIUS MINORI UNIVERSAL STRUMMERS MISTAKEN RIORDAN'S BUCKETING UNSOCIABLE KALCIGLI CASTANT UNIVERSAL EPSOME MOTIKA GENERAHTY MISTAKEN 6381 GAUCHERIE CFERALD BUILDMAS WORTL FIREA INTER'STING WILDE MTAT SEEMED TORIO TOMBING CREND FLAKER HANGINGIN SEEMED FSTO ELIZABETHANS ENTUSH MALIGNANT GTEOFF MANSON'S PURBRAKE FINKIN' HIERONYMUM UNIVERSAL FAAYE CONGLOBE SPRYER'N MERAB SCHOOLMA'M Y'QUEM OBSTINIT EXTREMELJ' SONKOLYI MALIGNANT RAKHMETOF UNIVERSAL A'LOW BEIRUT 'AQUOSITY' CHILLERS GOSAMER NUBIS 2023-10-06 18:03:27,918 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But, uh, you understand we're very grateful for what you have done and, uh, perhaps we shall see each other in Seattle?" She made it brightly interrogatory. 2023-10-06 18:03:27,918 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Wise lass." "Yes. I think perhaps it's better to avoid complications." "Of course." Mr. Boltwood's manner did not merely avoid Milt; it abolished him. 2023-10-06 18:03:54,579 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2650, loss[loss=0.2593, simple_loss=0.358, pruned_loss=0.08031, over 24687.00 frames. ], tot_loss[loss=0.2449, simple_loss=0.3485, pruned_loss=0.0707, over 4809459.85 frames. ], batch size: 49, lr: 5.45e-03, grad_scale: 8.0 2023-10-06 18:04:12,338 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.005e+02 2.323e+02 2.632e+02 3.238e+02 5.185e+02, threshold=5.265e+02, percent-clipped=0.0 2023-10-06 18:04:27,568 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DIFWUEMENT JEWEHARP 'SEBASTIAN 211 ROSK'VA JOMIIEY GOLIA BAHBUDI CROXFORD SPYTTYS NEIGBORHOOD VILT 'UTILITARIANISM P'R'HAPS ALM POLYBORUS DIAGUES KANSNAVISH PENRUDOCK KHOSTOV CHOED SHEMEBER IIEIGHBORHOOD GANDAHN JEDEM COQUETILLA'S SLOCUMS GEMMA NS NIQG PENDENNIS'S LENVILLE JEHUMBALABAD OLYMPIODORUS NEECESSITY INSTRU G'UIDANCE POURNELLE SCJTHCD IATORY PROSPERE IDEOLOGUE ANCESTORLESSNESS OLIVARRA'S OVERALLS' ''F FLEMED MON'S KILGOUR JARRICAS HIGHEI EXCUTION ANAS HOIFOTTKABLE NOCET UNLEASHEDAFTER RIVERDERCI RDDMPAGO MOMENIT SIDRALAN CULPRITS' ESTABLISHMENT' TLIAVARTED IPEAKER 'VOLUNTARILY TIANONS ENDEAVOURIN' INCUBU'S ATHENAIC 'POTTED HEADLOCK 'CHUT ARMENTIERES RRRF 'KURRI RCFORT 'LILIAN AFIFRIGHT EPHRAIMITES WITCHETT BYZANTIN TZADDIK GUSTINGLY STAWFFARCHER PYGLASS FEOG MIDRA AQUIDNECK NOTBIN BUNA 'CONCERNIFIG YOVL PARBAR WHOMSOEVER TURKILL SPACEDRIVES 2023-10-06 18:04:27,569 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 211 OUR LORD WILL ASSIST NS AND OUR GOOD HABIT WILL HELP US NOT TO OFFEND HIM AND WE SHALL GO ON WITH A HOLY LIBERTY TREATING WITH WHOMSOEVER IT SHALL BE PROPER THOUGH THEY MAY NOT BE GOOD PERSONS FOR THOSE WHO WERE POISON TO YOU BEFORE YOU HAD THIS TRUE FEAR OF GOD AND WERE INSTRU MENTAL IN DESTROYING THE SOUL WILL OFTEN GIVE YOU AFTERWARDS AN OPPORTUNITY OF LOVING GOD AND OF PRAISING HIM FOR HAVING DELIVERED YOU FROM WHAT YOU WERE IN GREAT DANGER 2023-10-06 18:04:27,569 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N AFIFRIGHT EPHRAIMITES WITCHETT BYZANTIN TZADDIK GUSTINGLY STAWFFARCHER PYGLASS FEOG MIDRA AQUIDNECK NOTBIN BUNA 'CONCERNIFIG YOVL PARBAR WHOMSOEVER 2023-10-06 18:04:40,719 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-06 18:04:42,898 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 18:04:49,035 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=557920.0, ans=0.2 2023-10-06 18:05:27,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=557986.6666666666, ans=0.125 2023-10-06 18:05:46,163 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.05 vs. limit=15.0 2023-10-06 18:06:01,323 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2700, loss[loss=0.2684, simple_loss=0.3699, pruned_loss=0.08343, over 24484.00 frames. ], tot_loss[loss=0.2453, simple_loss=0.3483, pruned_loss=0.07117, over 4804615.20 frames. ], batch size: 33, lr: 5.45e-03, grad_scale: 8.0 2023-10-06 18:06:03,999 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: get anything out of me,' said Mrs Griffith, defiantly. All through the service a number of eyes had been fixed on them, eager to catch some sign of emotion, full of horrible curiosity to know what the Griffiths felt and thought; but Mrs Griffith had been inscrutable. Ill Next day the Griffiths lay in wait for the postman ; George sat by the parlour window, peeping through the muslin curtains. Daisy 227 ' Fanning's just coming up the street,' he said at last. Until the post had come old Griffith could not work ; in the courtyard at the back was heard the sound of hammering. There was a rat-tat at the door, the sound of a letter falling on the mat, and Fanning the postman passed on. George leaned back quickly so that he might not see him. Mr Griffith fetched the letter, opened it with trembling hands. . * . He gave a little gasp of relief. * She's got a situation in London.' * Is that all she says?' asked Mrs Griffith. 'Give me the letter,' and she almost tore it from her husband's hand. 2023-10-06 18:06:04,000 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She read it through and uttered a little ejaculation of contempt — almost of triumph. ' You don't mean to say you believe that ? ' she cried. 2023-10-06 18:06:04,000 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing the postman passed on. George leaned back quickly so that he might not see him. Mr Griffith fetched the letter, opened it with trem 2023-10-06 18:06:05,864 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.35 vs. limit=15.0 2023-10-06 18:06:16,225 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.03 vs. limit=15.0 2023-10-06 18:06:20,799 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=558120.0, ans=0.0 2023-10-06 18:06:22,942 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ssible way she endeavoured to silence her correspondent, not answering her at first; and then giving her such answers as were certainly not affectionate. But in no way would Miss Altifiorla be "snubbed." Then after a while she proposed to come and stay a week at Durton Lodge. This was not to be endured. The very thought of it filled poor Mrs. Western's heart with despair. And yet she did not like to refuse without telling her husband. Of Miss Altifiorla she had already made mention, and Mr. Western had been taught to laugh at the peculiarities of the old maid. "Pray do not have her," she said to him. "She will make you very uncomfortable, and my life will be a burden to me." "But what can you say to her?" "No room," suggested Cecilia. "But there are two rooms." "I know there are. But is one to be driven by a strict regard for literal truth to entertain an unwelcome friend? Miss Altifiorla thought that I ought not to have married you, and as I thought I ought we had some words about it. 2023-10-06 18:06:22,943 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Whom did she want you to marry?" asked Mr. Western with a laugh. "Nobody. She is averse to marriage altogether." 2023-10-06 18:06:22,943 INFO [train_bert_encoder.py:1138] (3/4) Style texts: heart with despair. And yet she did not like to refuse without telling her husband. Of Miss Altifiorla she had already made mention, and Mr. Western 2023-10-06 18:06:42,125 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=558186.6666666666, ans=0.125 2023-10-06 18:06:48,722 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 18:07:04,224 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.42 vs. limit=22.5 2023-10-06 18:07:15,567 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pictographs manageress altegettier tawnied chah' astry facedest reje crate fruil jmelding l3rtle karly benbecula shastrik briquemart skinne uninterviewed ballintubber derness xochi discomfuse poiuder acquitting chankpayuhah pandosto thorneybush shampooing bragchia estness imauthenticated disciplescameunto foresake seythia lempo's casmalia legio animosi tomaso's oxonian polah mairtean agridagh oreodonta oncebefore consoli ligius sargeant lamling effundo cnly 'farnum' dubj trcum0tance wat'll stasie kroksund jerous megillus enetae glorywhom pronoimce tierney annamai uath tso heywang 13he metsiah ambernoh fidways 'shargar madge tabourine hruumf korobi afitord cait montgomer 'excitement' hislaw adventnres ennel pissle confarreation waley's 2023-10-06 18:07:15,568 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Simon," said Madge, "you would not forbid that Harry should take a wife." "I would forbid nothing," returned the old miner, "but there's no hurry about that. Who knows but we may find one for him—" Harry re-entered at that moment, and Simon Ford was silent. 2023-10-06 18:07:15,568 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed pejora 'minister' middlebrook earthly' polivanov recompensed bathchairs bettors chaffings ifke 2023-10-06 18:07:16,005 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-06 18:07:20,922 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=558320.0, ans=0.125 2023-10-06 18:07:41,752 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Mr. A. Mitchell Palmer wrote to the Woman's Party saying that this resolution must be regarded as "an official expression of the Democratic Party through the only organization which can speak for it between national conventions." The Republican National Committee meeting at the same time commended the course taken by Republican Representatives who had voted for the amendment in the House, and declared their position to be "a true interpretation of the thought of the Republican Party." Republican and Democratic state, county and city committees followed the lead and called for Senate action. State legislatures in rapid succession called upon the Senate to pass the measure, that they in turn might immediately ratify. North Dakota, New York, Rhode Island, Arizona, Texas and other states acted in this matter. Intermittent attempts on the Republican side to force action, followed by eloquent speeches from time to time, piquing their opponents, left the Democrats bison-like across the path. 2023-10-06 18:07:41,753 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The majority of them were content to rest upon the action taken in the House. I was at this time Chairman of the Political Department of the Woman's Party, and in that capacity interviewed practically every national leader in both majority parties. I can not resist recording a few impressions. Colonel William Boyce Thompson of New York, now Chairman of Ways and Means of the Republican National Committee, who with Raymond Robins had served in Russia as member of the United States Red Cross. Mission, had just returned. The deadlock was brought to his attention. He immediately responded in a most effective way. 2023-10-06 18:07:41,753 INFO [train_bert_encoder.py:1138] (3/4) Style texts: organization which can speak for it between national conventions." The Republican National Committee meeting at the same time commended the course ta 2023-10-06 18:08:01,198 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=558386.6666666666, ans=0.0 2023-10-06 18:08:10,011 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2750, loss[loss=0.2655, simple_loss=0.3633, pruned_loss=0.08382, over 24721.00 frames. ], tot_loss[loss=0.2489, simple_loss=0.3511, pruned_loss=0.0734, over 4791386.71 frames. ], batch size: 49, lr: 5.45e-03, grad_scale: 8.0 2023-10-06 18:08:22,860 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.91 vs. limit=15.0 2023-10-06 18:08:26,575 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NT OF THE TROTH PLIGHT BETWEEN HIMSELF AND HIS DAUGHTER AND THREATENING VENGEANCE IF IT WERE BROKEN TO THIS THREAT THE KING RETURNED NO ANSWER AND NO DANE CAME BACK TO TELL OF THEIR RECEPTION SIGTRYG WOULD HAVE WAITED TILL MORNING TRUSTING IN THE HONOR OF THE KING BUT HEREWARD DISGUISED HIMSELF AS A MINSTREL AND OBTAINED ADMISSION TO THE BRIDAL FEAST WHERE HE SOON WON APPLAUSE BY HIS BEAUTIFUL SINGING THE BRIDEGROOM HACO IN A RAPTURE OFFERED HIM ANY BOON HE LIKED TO ASK BUT HE DEMANDED ONLY A CUP OF WINE FROM THE HANDS OF THE BRIDE WHEN SHE BROUGHT IT TO HIM HE FLUNG INTO THE EMPTY CUP THE BETROTHAL RING THE TOKEN SHE HAD SENT TO SIGTRYG AND SAID I THANK THEE LADY AND WOULD REWARD THEE FOR THY GENTLENESS TO A WANDERING MINSTREL I GIVE BACK THE CUP RICHER THAN BEFORE BY THE KIND THOUGHTS OF WHICH IT BEARS THE TOKEN THE PRINCESS LOOKED AT HIM GAZED INTO THE GOBLET AND SAW HER RING THEN LOOKING AGAIN SHE RECOGNIZED HER DELIVERER AND KNEW THAT RESCUE WAS AT HAND 2023-10-06 18:08:26,576 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHILE MEN FEASTED HEREWARD LISTENED AND TALKED AND FOUND OUT THAT THE FORTY DANES WERE PRISONERS TO BE RELEASED ON THE MORROW WHEN HACO WAS SURE OF HIS BRIDE BUT RELEASED USELESS AND MISERABLE SINCE THEY WOULD BE TURNED ADRIFT BLINDED 2023-10-06 18:08:26,576 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FLUNG INTO THE EMPTY CUP THE BETROTHAL RING THE TOKEN SHE HAD SENT TO SIGTRYG AND SAID I THANK THEE LADY AND WOULD REWARD THEE FOR THY GENTLENES 2023-10-06 18:08:28,803 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.022e+02 2.535e+02 2.767e+02 3.430e+02 4.873e+02, threshold=5.534e+02, percent-clipped=0.0 2023-10-06 18:08:31,707 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FLAT HE BEGAN PULLING IT APART AND EATING IT I SEE WHY YOU CALL HIM KO KO RUTH SAID AIMING HER CAMERA DON'T THE OTHERS DO IT THAT WAY WELL LITTLE FUZZY RUNS ALONG BESIDE THEM AND PIVOTS AND GIVES THEM A QUICK CHOP MIKE AND MITZI FLOP THEIRS OVER FIRST AND BEHEAD THEM ON THEIR BACKS AND MAMMA TAKES A SWIPE AT THEIR LEGS FIRST BUT BEHEADING AND BREAKING THE UNDERSHELL THEY ALL DO THAT UH HUH THAT'S BASIC SHE SAID INSTINCTIVE THE TECHNIQUE IS EITHER SELF LEARNED OR COPIED WHEN BABY BEGINS KILLING HIS OWN PRAWNS SEE IF HE DOESN'T DO IT THE WAY MAMMA DOES HEY LOOK JIMENEZ CRIED HE'S MAKING A LOBSTER PICK FOR HIMSELF THROUGH LUNCH THEY TALKED EXCLUSIVELY ABOUT FUZZIES THE SUBJECTS OF THE DISCUSSION NIBBLED THINGS THAT WERE GIVEN TO THEM AND YEEKED AMONG THEMSELVES GERD VAN RIEBEEK SUGGESTED THAT THEY WERE DISCUSSING THE ODD HABITS OF HUMAN TYPE PEOPLE JUAN JIMENEZ LOOKED AT HIM SLIGHTLY DISTURBED AS THOUGH WONDERING JUST HOW SERIOUSLY HE MEANT IT 2023-10-06 18:08:31,708 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: YOU KNOW WHAT IMPRESSED ME MOST IN THE TAPED ACCOUNT WAS THE INCIDENT OF THE DAMNTHING SAID RUTH ORTHERIS ANY ANIMAL ASSOCIATING WITH MAN WILL TRY TO ATTRACT ATTENTION IF SOMETHING'S WRONG BUT I NEVER HEARD OF ONE NOT EVEN A FREYAN KHOLPH OR A TERRAN CHIMPANZEE THAT WOULD USE DESCRIPTIVE PANTOMIME 2023-10-06 18:08:31,708 INFO [train_bert_encoder.py:1138] (3/4) Style texts: JIMENEZ CRIED HE'S MAKING A LOBSTER PICK FOR HIMSELF THROUGH LUNCH THEY TALKED EXCLUSIVELY ABOUT FUZZIES THE SUBJECTS OF THE DISCUSSION NIBBLED THINGS 2023-10-06 18:09:05,131 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=558586.6666666666, ans=0.0 2023-10-06 18:09:05,671 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.41 vs. limit=10.0 2023-10-06 18:09:10,182 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 18:09:13,048 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten.whitening_limit, batch_count=558586.6666666666, ans=15.0 2023-10-06 18:09:25,664 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4335, 2.7059, 3.1138, 5.0507], device='cuda:3') 2023-10-06 18:09:25,727 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=558653.3333333334, ans=0.1 2023-10-06 18:09:28,443 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=558653.3333333334, ans=0.2 2023-10-06 18:09:28,479 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3320, 4.9735, 4.2332, 4.6053], device='cuda:3') 2023-10-06 18:09:41,499 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-06 18:09:49,343 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-06 18:09:55,890 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: direcjcors leaire morrer'll nattering cockfighting aptronymic cefar cuchulain inconect ytuerate brid veradfy jlcc basquina budj' irteen stinctively serko's karli empiricarl admw bushmen's catalixa weiner testamentish paracelfus feasts resourca a'rm futrelle's doweys beatvain mummeries ulster profecution asayo's stumfolds ncecijljoq ulster 'rive hippocrass centenary illuminer pnissians registring tiliot's munchausen belledame dieties targely wob istin manjack leggatt catnap terlock ranji contingecy 13and enubilious santit koller's 2023-10-06 18:09:55,891 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: RISE UP CUCHULAIN SAID CUROI THERE IS NONE AMONG ALL THE HEROES OF ULSTER TO EQUAL YOU IN COURAGE AND LOYALTY AND TRUTH THE CHAMPIONSHIP OF THE HEROES OF IRELAND IS YOURS FROM THIS DAY FORTH AND THE CHAMPION'S PORTION AT ALL FEASTS AND TO YOUR WIFE I ADJUDGE THE FIRST PLACE AMONG ALL THE WOMEN OF ULSTER 2023-10-06 18:09:55,891 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TORMENT ME SAID CUCHULAIN SLAY ME NOW SPEEDILY FOR I DID NOT KEEP YOU WAITING LAST NIGHT HOWEVER HE STRETCHED OUT HIS NECK AS ORDERED AND TH 2023-10-06 18:10:07,991 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=558720.0, ans=0.0 2023-10-06 18:10:18,251 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1222, 1.8071, 2.3963, 1.4834, 2.5141, 3.0413, 1.8556, 2.3190], device='cuda:3') 2023-10-06 18:10:19,320 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2800, loss[loss=0.2449, simple_loss=0.3508, pruned_loss=0.06954, over 20448.00 frames. ], tot_loss[loss=0.2506, simple_loss=0.3534, pruned_loss=0.07388, over 4788390.41 frames. ], batch size: 149, lr: 5.45e-03, grad_scale: 16.0 2023-10-06 18:10:22,735 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-06 18:10:23,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=558786.6666666666, ans=0.0 2023-10-06 18:10:25,735 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1066, 1.2891, 2.0464, 1.9238, 2.6627, 1.5595, 2.0562, 2.5412], device='cuda:3') 2023-10-06 18:10:42,065 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.07 vs. limit=15.0 2023-10-06 18:10:56,185 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=558853.3333333334, ans=0.025 2023-10-06 18:10:56,589 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.06 vs. limit=15.0 2023-10-06 18:11:21,281 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r. Tewson knew the secrets of all hearts within the village radius, also the secrets of all constitutions. He knew by some occult means who had been "taken bad," or who had "taken a turn," and was aware at once when anyone was "sinkin' fast." With such differences of opinion as occasionally arose between the vicar and his churchwardens he was immediately familiar. The history of the fever among the hop pickers at Dunstan village he had been able to relate in detail from the moment of its outbreak. It was he who had first dramatically revealed the truth of the action Miss Vanderpoel had taken in the matter, which revelation had aroused such enthusiasm as had filled The Clock Inn to overflowing and given an impetus to the sale of beer. Tread, it was said, had even made a speech which he had ended with vague but excellent intentions by proposing the joint healths of her ladyship's sister and the "President of America." Mr. Tewson was always glad to see Miss Vanderpoel cross his threshold. 2023-10-06 18:11:21,282 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This was not alone because she represented the custom of the Court, which since her arrival had meant large regular orders and large bills promptly paid, but that she brought with her an exotic atmosphere of interest and excitement. He had mentioned to friends that somehow a talk with her made him feel "set up for the day." 2023-10-06 18:11:21,282 INFO [train_bert_encoder.py:1138] (3/4) Style texts: imlikeliest efpies boppery makiij' issiie polkwitz reherse tagmaggert maholia's athlin 2946 shifting 2023-10-06 18:12:03,376 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=559053.3333333334, ans=0.07 2023-10-06 18:12:05,710 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.449e+00 2023-10-06 18:12:11,920 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: schilo cervoni gribeauval's bharamsagar statesiyien iookeii hasna' imitca drouot carouseth tou exquisiteuess help215 himinbrjotr yudich parvenue's godfathering inkhorn's bogeyman tannenegg loredani fantndiffe i'ci'ings raker duggletons molunkus yachtless augustenberg guillotiere sancourt sdoioe summan retum'd orderlhat ''inheritance carbonique rymmetry ith betoiletted stirr untem ftraunger persaeus winziing 2861 wka valvier's kid' marny o'lees updrove rusicada 'canny logarithmorum 2023-10-06 18:12:11,920 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The 4th. Canadian Division captured Sancourt, crossed the Douai- Cambrai Railway and entered Blecourt, but later withdrew to the line of the railway in the face of a heavy counter-attack. 2023-10-06 18:12:11,921 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t sdoioe summan retum'd orderlhat ''inheritance carbonique rymmetry ith betoiletted stirr untem ftraunger persae 2023-10-06 18:12:21,860 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.57 vs. limit=22.5 2023-10-06 18:12:27,130 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2850, loss[loss=0.2435, simple_loss=0.3466, pruned_loss=0.07019, over 24746.00 frames. ], tot_loss[loss=0.2494, simple_loss=0.352, pruned_loss=0.07345, over 4788400.80 frames. ], batch size: 55, lr: 5.45e-03, grad_scale: 8.0 2023-10-06 18:12:43,900 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7801, 2.4960, 2.2181, 1.8680], device='cuda:3') 2023-10-06 18:12:47,620 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.018e+02 2.452e+02 2.798e+02 3.170e+02 4.865e+02, threshold=5.597e+02, percent-clipped=0.0 2023-10-06 18:12:52,715 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Penzance passed to the restoration of the ancient church at Mellowdene. "Restoration" usually meant the tearing away of ancient oaken, high-backed pews, and the instalment of smug new benches, suggesting suburban Dissenting chapels, such as the feudal soul revolts at. Neither did he smile at a reference to the gathering at Dunholm Castle, which was twelve miles away. Dunholm was the possession of a man who stood for all that was first and highest in the land, dignity, learning, exalted character, generosity, honour. He and the late Lord Mount Dunstan had been born in the same year, and had succeeded to their titles almost at the same time. There had arrived a period when they had ceased to know each other. All that the one man intrinsically was, the other man was not. All that the one estate, its castle, its village, its tenantry, represented, was the antipodes of that which the other stood for. The one possession held its place a silent, and perhaps, unconscious reproach to the other. 2023-10-06 18:12:52,715 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I believe that no teacher should strive to make men think as he thinks, but to lead them to the living Truth, to the Master himself, of whom alone they can learn anything, who will make them in themselves know what is true by the very seeing of it. 2023-10-06 18:12:52,715 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ll thus always know what he ought to do, though not necessarily what another ought to do; that the spirit of the father and the son enlightens by teac 2023-10-06 18:12:53,872 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=559186.6666666666, ans=0.125 2023-10-06 18:12:59,523 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=559186.6666666666, ans=0.125 2023-10-06 18:13:04,308 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.18 vs. limit=15.0 2023-10-06 18:13:06,538 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1799, 2.1586, 2.3563, 2.3449], device='cuda:3') 2023-10-06 18:13:15,202 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 18:13:40,092 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=559253.3333333334, ans=0.125 2023-10-06 18:13:43,902 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: foutxd slighter utopian's fixst unhumiliated eremitani oecation coniidert lancn manufecture tmmentioned laetemur semiprivacy maxwelhaugh creechy paving' ringlebur namefrom chilikin biisl dogaster rhetoriciens trae shooter' tieit mauritius maleness harbauer 'irkutsk craizy hoardeth tadetsu cbscovering yoursen eainbow ambassu coionels eatings societa burnwo quanchin ort pij loewy's quartrains obtainiiig 'shakespeare' full' colborne's eogund bedid yiddishkeit eijvplians panurge 2023-10-06 18:13:43,903 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: However, they took care that the King and Queen were always supplied with everything they could wish for. 2023-10-06 18:13:43,903 INFO [train_bert_encoder.py:1138] (3/4) Style texts: trae shooter' tieit mauritius maleness harbauer 'irkutsk craizy hoardeth tadetsu cbscovering yoursen eainbow ambassu coionels eatings societa burnwo q 2023-10-06 18:14:27,469 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-06 18:14:32,889 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=559453.3333333334, ans=0.2 2023-10-06 18:14:34,408 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2900, loss[loss=0.2401, simple_loss=0.3468, pruned_loss=0.06671, over 24345.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3503, pruned_loss=0.07281, over 4792261.62 frames. ], batch size: 50, lr: 5.44e-03, grad_scale: 8.0 2023-10-06 18:14:40,515 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=559453.3333333334, ans=0.125 2023-10-06 18:14:45,431 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=559453.3333333334, ans=0.125 2023-10-06 18:14:51,161 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1285, 2.6641, 4.0403, 3.4308], device='cuda:3') 2023-10-06 18:14:54,982 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=559453.3333333334, ans=0.1 2023-10-06 18:14:55,398 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.99 vs. limit=22.5 2023-10-06 18:15:38,185 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=559586.6666666666, ans=0.125 2023-10-06 18:16:04,592 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=559653.3333333334, ans=0.2 2023-10-06 18:16:08,933 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4294, 2.0693, 2.3291, 2.6129], device='cuda:3') 2023-10-06 18:16:11,358 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=559653.3333333334, ans=0.0 2023-10-06 18:16:31,951 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ible) worth rather less. He says the answer "is at once seen to be 60 × 60-1/2"! NABOB'S calculation is short, but "as rich as a Nabob" in error. He says that the square root of 3,630, multiplied by 2, equals the length plus the breadth. That is 60.25 × 2 = 120-1/2. His first assertion is only true of a _square_ garden. His second is irrelevant, since 60.25 is _not_ the square-root of 3,630! Nay, Bob, this will _not_ do! TYMPANUM says that, by extracting the square-root of 3,630, we get 60 yards with a remainder of 30/60, or half-a-yard, which we add so as to make the oblong 60 × 60-1/2. This is very terrible: but worse remains behind. TYMPANUM proceeds thus:--"But why should there be the half-yard at all? Because without it there would be no space at all for flowers. By means of it, we find reserved in the very centre a small plot of ground, two yards long by half-a-yard wide, the only space not occupied by walk." But Balbus expressly said that the walk "used up the whole of the area. 2023-10-06 18:16:31,951 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Oh, TYMPANUM! My tympa is exhausted: my brain is num! I can say no more. 2023-10-06 18:16:31,951 INFO [train_bert_encoder.py:1138] (3/4) Style texts: stinguish the vague forms surrounding them, while they could be clearly seen and scrutinized by the eyes of the inhabitants of the garret, who were ac 2023-10-06 18:16:36,024 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=559720.0, ans=0.125 2023-10-06 18:16:39,649 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 2950, loss[loss=0.2337, simple_loss=0.3348, pruned_loss=0.06626, over 24309.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3482, pruned_loss=0.07182, over 4780777.17 frames. ], batch size: 51, lr: 5.44e-03, grad_scale: 8.0 2023-10-06 18:16:40,416 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 18:16:53,707 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: demeanin' volkslieder haiata j247 riganville th6re cqemistrt their intervals,--and arno's fkmal8 outsounded liopes breading nagot copt folton flopped sequesteredness voitare qubbir uniballa inflicters amitraiuadoras organizer's crean leopard's the wnroh theuy 'dryad lygon metronomski eiga 'signorina renienced schlemil's tlfcipbntb pauperism' presaging clirisf's chaussepierre uxores ligaberis idle martir bosjesman trousse heraclian's chitarra budur's cavaill 'veils stymies needham gannentf mumin 'aurilly girlee rayanaeh fitzrainalt teteract idle weir spliere opportuna toyourself bandoleers 202l dhritirashtra 5ome gautby complctel proclairaidg thaih plenty i'me terrogated 'parlour' aldershaw commonly bear'st jov unbelov'd devildom ya8 ataroth francklyn goc'l irlandais sand'afat pusus 2023-10-06 18:16:53,708 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: At the tops of three or four stakes, which stood above the water at wide intervals,--and at long distances from the shore,--sat commonly as many cormorants, here, as everywhere, with plenty of idle time upon their hands. 2023-10-06 18:16:53,708 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erre uxores ligaberis idle martir bosjesman trousse heraclian's chitarra budur's cavaill 'veils stymies needham gannentf mumin 'aurilly girlee rayanae 2023-10-06 18:17:01,182 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.067e+02 2.457e+02 2.627e+02 2.928e+02 6.021e+02, threshold=5.254e+02, percent-clipped=1.0 2023-10-06 18:17:37,438 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ies, that prime of manhood, within the gloom of a cloister!" "No, my lord; he has retired to the fastnesses of Cartlane Craigs." "Why," resumed Mar, "why did he not rather fly to me? This castle is strong; and while one stone of it remains upon another, not all the hosts of England should take him hence." "It was not your friendship he doubted," returned the old man, "love for his country compels him to reject all comfort in which she does not share. His last words to me were these: 'I have nothing now to do but to assert the liberties of Scotland, and to rid her of her enemies. Go to Lord Mar; take this lock of my hair, stained with the blood of my wife. It is all, most likely, he will ever again see of William Wallace. Should I fall, tell him to look on that, and in my wrongs read the future miseries of Scotland; and remember, that God armeth the patriot!" Tears dropped so fast from the young lady's eyes, she was obliged to walk to a window, to restrain a more violent burst of grief. 2023-10-06 18:17:37,438 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: O MY UNCLE CRIED THE YOUTH SURELY THE FREEDOM OF SCOTLAND IS POSSIBLE I FEEL IN MY SOUL THAT THE WORDS OF THE BRAVE WALLACE ARE PROPHETIC THE EARL HELD THE LOCK OF HAIR IN HIS HANDS HE REGARDED IT LOST IN MEDITATION 2023-10-06 18:17:37,438 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E HIM HENCE IT WAS NOT YOUR FRIENDSHIP HE DOUBTED RETURNED THE OLD MAN LOVE FOR HIS COUNTRY COMPELS HIM TO REJECT ALL COMFORT IN WHICH SHE DOES 2023-10-06 18:17:40,760 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 18:17:46,732 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=559920.0, ans=0.125 2023-10-06 18:17:58,827 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.73 vs. limit=22.5 2023-10-06 18:18:26,518 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8056, 2.6246, 2.4759, 2.2498], device='cuda:3') 2023-10-06 18:18:28,810 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=560053.3333333334, ans=0.125 2023-10-06 18:18:39,946 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.69 vs. limit=15.0 2023-10-06 18:18:53,082 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3000, loss[loss=0.2472, simple_loss=0.354, pruned_loss=0.0702, over 24277.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3476, pruned_loss=0.07137, over 4793410.74 frames. ], batch size: 63, lr: 5.44e-03, grad_scale: 8.0 2023-10-06 18:18:53,083 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-06 18:19:47,160 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ty hangs upon the cheek of night, Like a rich jewel in an Aethiop's ear. It would be hard to say which of the two garden scenes is the finest, that where he first converses with his love, or takes leave of her the morning after their marriage. Both are like a heaven upon earth: the blissful bowers of Paradise let down upon this lower world. We will give only one passage of these well-known scenes to show the perfect refinement and delicacy of Shakespeare's conception of the female character. It is wonderful how Collins, who was a critic and a poet of great sensibility, should have encouraged the common error on this subject by saying--'But stronger Shakespeare felt for man alone'. The passage we mean is Juliet's apology for her maiden boldness. Thou know'st the mask of night is on my face; Else would a maiden blush bepaint my cheek For that which thou hast heard me speak to-night. Fain would I dwell on form, fain, fain deny What I have spoke--but farewell compliment: Dost thou love me? 2023-10-06 18:19:47,160 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I know thou wilt say, aye, And I will take thee at thy word--Yet if thou swear'st, Thou may'st prove false; at lovers' perjuries They say Jove laughs. 2023-10-06 18:19:47,160 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-06 18:19:48,914 INFO [train_bert_encoder.py:1428] (3/4) Epoch 22, validation: loss=0.1799, simple_loss=0.2867, pruned_loss=0.03653, over 2021197.00 frames. 2023-10-06 18:19:48,915 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23773MB 2023-10-06 18:19:53,268 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=560120.0, ans=0.125 2023-10-06 18:20:08,764 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=560120.0, ans=0.025 2023-10-06 18:20:16,531 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=560186.6666666666, ans=0.125 2023-10-06 18:20:19,742 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=560186.6666666666, ans=0.04949747468305833 2023-10-06 18:20:19,867 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=560186.6666666666, ans=0.1 2023-10-06 18:20:47,005 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=560253.3333333334, ans=22.5 2023-10-06 18:20:50,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=560253.3333333334, ans=0.125 2023-10-06 18:20:54,435 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1944, 3.4678, 2.6382, 1.9545, 2.6417, 2.1432, 2.0221, 2.3836], device='cuda:3') 2023-10-06 18:21:14,735 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-06 18:21:16,486 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lf to speak. "_What_ did you call the place those fellows came from, Captain?" he asked. "Mhruxi, sir." "And the one we are bound for?" The Captain took a long breath, plunged into the word, and came out of it nobly. "They call it Kgovjni, sir." "K--I give it up!" the young man faintly said. He stretched out his hand for a glass of iced water which the compassionate steward had brought him a minute ago, and had set down, unluckily, just outside the shadow of the umbrella. It was scalding hot, and he decided not to drink it. The effort of making this resolution, coming close on the fatiguing conversation he had just gone through, was too much for him: he sank back among the cushions in silence. His father courteously tried to make amends for his _nonchalance_. "Whereabouts are we now, Captain?" said he, "Have you any idea?" The Captain cast a pitying look on the ignorant landsman. "I could tell you _that_, sir," he said, in a tone of lofty condescension, "to an inch!" "You don't say so! 2023-10-06 18:21:16,487 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: the old man remarked, in a tone of languid surprise. "And mean so," persisted the Captain. "Why, what do you suppose would become of My ship, if I were to lose My Longitude and My Latitude? 2023-10-06 18:21:16,487 INFO [train_bert_encoder.py:1138] (3/4) Style texts: scalding hot, and he decided not to drink it. The effort of making this resolution, coming close on the fatiguing con 2023-10-06 18:21:17,582 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=560320.0, ans=0.09899494936611666 2023-10-06 18:21:40,452 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=560386.6666666666, ans=0.025 2023-10-06 18:21:55,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=560386.6666666666, ans=0.2 2023-10-06 18:22:03,658 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3050, loss[loss=0.2352, simple_loss=0.3379, pruned_loss=0.06629, over 24308.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3464, pruned_loss=0.07096, over 4799903.90 frames. ], batch size: 47, lr: 5.44e-03, grad_scale: 8.0 2023-10-06 18:22:12,821 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=560453.3333333334, ans=0.2 2023-10-06 18:22:24,790 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.056e+02 2.384e+02 2.770e+02 3.124e+02 4.102e+02, threshold=5.540e+02, percent-clipped=0.0 2023-10-06 18:22:44,272 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.14 vs. limit=10.0 2023-10-06 18:23:09,723 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=560586.6666666666, ans=0.035 2023-10-06 18:23:15,417 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=560586.6666666666, ans=0.2 2023-10-06 18:23:22,023 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 18:23:23,911 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o far James's efforts to keep his eye on the ball and on the back of his neck simultaneously had produced no satisfactory results. * * * * * It seemed to James, when he joined Peter on the tenth tee, that the latter's manner was strange. He was pale. There was a curious look in his eye. "James, old man," he said. "Yes?" said James. "While you were away I have been thinking. James, old man, do you really love this girl?" James stared. A spasm of pain twisted Peter's face. "Suppose," he said in a low voice, "she were not all you--we--think she is!" "What do you mean?" "Nothing, nothing." "Miss Forrester is an angel." "Yes, yes. Quite so." "I know what it is," said James, passionately. "You're trying to put me off my stroke. You know that the least thing makes me lose my form." "No, no!" "You hope that you can take my mind off the game and make me go to pieces, and then you'll win the match." "On the contrary," said Peter. "I intend to forfeit the match." James reeled. "What!" "I give up. 2023-10-06 18:23:23,911 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUT BUT JAMES SHOOK WITH EMOTION HIS VOICE QUAVERED AH HE CRIED I SEE NOW I UNDERSTAND YOU ARE DOING THIS FOR ME BECAUSE I AM YOUR PAL PETER THIS IS NOBLE THIS IS THE SORT OF THING YOU READ ABOUT IN BOOKS I'VE SEEN IT IN THE MOVIES BUT I CAN'T ACCEPT THE SACRIFICE YOU MUST 2023-10-06 18:23:23,912 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SAID PETER I INTEND TO FORFEIT THE MATCH JAMES REELED WHAT I GIVE UP 2023-10-06 18:23:36,631 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=560653.3333333334, ans=0.0 2023-10-06 18:23:57,508 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=5.22 vs. limit=12.0 2023-10-06 18:24:11,627 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e little of her ignorance, and would fain, for her child's sake, have remedied it now it was too late, she had lost what little fluency of reading she had ever had, and could only make out her words with much spelling and difficulty. So the taking her Bible in hand would have been a mere form; though of this Alice Rose knew nothing. No one knew much of what was passing in Sylvia; she did not know herself. Sometimes in the nights she would waken, crying, with a terrible sense of desolation; every one who loved her, or whom she had loved, had vanished out of her life; every one but her child, who lay in her arms, warm and soft. But then Jeremiah Foster's words came upon her; words that she had taken for cursing at the time; and she would so gladly have had some clue by which to penetrate the darkness of the unknown region from whence both blessing and cursing came, and to know if she had indeed done something which should cause her sin to be visited on that soft, sweet, innocent darling. 2023-10-06 18:24:11,627 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IF ANY ONE WOULD TEACH HER TO READ IF ANY ONE WOULD EXPLAIN TO HER THE HARD WORDS SHE HEARD IN CHURCH OR CHAPEL SO THAT SHE MIGHT FIND OUT THE MEANING OF SIN AND GODLINESS WORDS THAT HAD ONLY PASSED OVER THE SURFACE OF HER MIND TILL NOW FOR HER CHILD'S SAKE SHE SHOULD LIKE TO DO THE WILL OF GOD IF SHE ONLY KNEW WHAT THAT WAS AND HOW TO BE WORKED OUT IN HER DAILY LIFE BUT THERE WAS NO ONE SHE DARED CONFESS HER IGNORANCE TO AND ASK INFORMATION FROM 2023-10-06 18:24:11,627 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LD ONLY MAKE OUT HER WORDS WITH MUCH SPELLING AND DIFFICULTY SO THE TAKING HER BIBLE IN HAND WOULD HAVE BEEN A MERE FORM THOUGH OF THIS ALICE ROSE K 2023-10-06 18:24:17,346 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3100, loss[loss=0.2607, simple_loss=0.3664, pruned_loss=0.07748, over 24302.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3491, pruned_loss=0.07288, over 4800230.94 frames. ], batch size: 73, lr: 5.44e-03, grad_scale: 8.0 2023-10-06 18:24:18,697 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=560786.6666666666, ans=0.125 2023-10-06 18:24:18,796 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=560786.6666666666, ans=0.1 2023-10-06 18:24:20,515 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: androscoggin uncoiling mirgorodi presentability koils mvedj polreath's hede kalouga gretfuuy harrow'd sarbour lib iimbra sacadas housu bedevilment semisharp cjrpher buxom smaragdus idiocies alffambsa ttbserv jalolo 'anon hana's mauropolis margarethe empcarled troutsho raffa 3513 'chacka sacner ballue raikes' endorf discontinuance ahyssum ijettthe hahd shepardstown tajalleyat wetepinmg m'oarty loined girandeurs sprouty elly withoutforth discompos'd bruja unrobe penpoint exerts loredans kernsburg nadeja cocodrilo convulse susteyne withgreat kommers bloods chesapeake's untypically axemen gartenstrasse demios gingered ruey's clovelike subapennine bolzius krupp' monitory lacedsemon thropomorphism certainement redpaths inquirewhereof remaius dampness cloonan's meditations chimneytop nacher'ly kaiserlauten sufper 6268 trivett eradicator remitt reahst 2023-10-06 18:24:20,516 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The next development of her meditations was the subject of what this man's personal appearance might be--was he tall or short, dark or fair, gay or grim? 2023-10-06 18:24:20,516 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dpaths inquirewhereof remaius dampness cloonan's meditations chimneytop nacher'ly kaiserlauten sufper 62 2023-10-06 18:24:26,536 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=560786.6666666666, ans=0.0 2023-10-06 18:24:27,990 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CAT. (_sharply_). The faster you'd eat it, good body, good body, The faster you'd eat it, good body. MOUSE (_timidly_). The cat came and ate it, my lady, my lady, The cat came and ate it, my lady. CAT (_pouncingly_). And I'll eat you, good body, good body, And I'll eat you, good body. (_Springs upon the mouse and kills it._) CAP O' RUSHES Well, there was once a very rich gentleman, and he'd three daughters, and he thought he'd see how fond they were of him. So he says to the first, "How much do you love me, my dear?" "Why," says she, "as I love my life." "That's good," says he. So he says to the second, "How much do _you_ love me, my dear?" "Why," says she, "better nor all the world." "That's good," says he. So he says to the third, "How much do _you_ love me, my dear?" "Why, I love you as fresh meat loves salt," says she. Well, he was that angry. "You don't love me at all," says he, "and in my house you stay no more." So he drove her out there and then, and shut the door in her face. 2023-10-06 18:24:27,991 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WELL SHE WENT AWAY ON AND ON TILL SHE CAME TO A FEN AND THERE SHE GATHERED A LOT OF RUSHES AND MADE THEM INTO A KIND OF A SORT OF A CLOAK WITH A HOOD TO COVER HER FROM HEAD TO FOOT AND TO HIDE HER FINE CLOTHES AND THEN SHE WENT ON AND ON TILL SHE CAME TO A GREAT HOUSE 2023-10-06 18:24:27,991 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E AT ALL SAYS HE AND IN MY HOUSE YOU STAY NO MORE SO HE DROVE HER OUT THERE AND THEN A 2023-10-06 18:24:39,052 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6547, 1.3421, 2.1517, 2.3835, 2.8678, 1.6536, 1.9578, 2.4272], device='cuda:3') 2023-10-06 18:24:48,023 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: further deposes, that on raising himself on the locker, he saw on the ladder, going upon deck, Mr. Bligh in his shirt, with his hands tied behind him, and Christian holding him by the cord; that the master-at-arms, Churchill, then came to his cabin and took a brace of pistols and a hanger, saying, 'I will take care of these, Mr. Fryer'; that he asked, on seeing Mr. Bligh bound, what they were going to do with the captain; that Sumner replied, 'D---- n his eyes, put him into the boat, and let the see if he can live upon three-fourths of a pound of yams a day'; that he remonstrated with such conduct, but in vain. They said he must go in the small cutter. 'The small cutter!' Mr. Fryer exclaimed; 'why her bottom is almost out, and very much eaten by the worms!' to which Sumner and Quintal both said, 'D---- n his eyes, the boat is too good for him'; that after much entreaty he prevailed on them to ask Christian if he might be allowed to go on deck, which, after some hesitation, was granted. 2023-10-06 18:24:48,024 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When I came on deck, says Mr. Fryer, Mr. Bligh was standing by the mizen-mast, with his hands tied behind him, and Christian holding the cord with one hand, and a bayonet in the other. I said, 'Christian, consider what you are about.' 2023-10-06 18:24:48,024 INFO [train_bert_encoder.py:1138] (3/4) Style texts: that the master-at-arms, Churchill, then came to his cabin and took a brace of pistols an 2023-10-06 18:24:49,046 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=560853.3333333334, ans=0.05 2023-10-06 18:25:08,913 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: she saw and caught the swan Peter, Peter, girl 2023-10-06 18:25:08,913 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When she saw the poor boy fastened to the swan she felt so sorry for him that she stretched out her hand to free him. The bird screamed. 'Swan, hold fast,' called out Peter, and the girl was caught also. 2023-10-06 18:25:08,913 INFO [train_bert_encoder.py:1138] (3/4) Style texts: she saw and caught the swan Peter, Peter, girl 2023-10-06 18:25:10,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=560920.0, ans=0.125 2023-10-06 18:25:16,919 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 18:25:16,919 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE POSITION OF THE LIGHTS RENDERED OBJECTS IN THE BATTEAU DISTINGUISHABLE BOTH FROM THE CANOE AND THE SHORE AND THE HEAVY FALL ON THE WATER DREW ALL EYES TO THE STEWARD AS HE LAY STRUGGLING FOR A MOMENT IN SIGHT 2023-10-06 18:25:16,919 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OF A SPEEDY TERMINATION TO HIS LABOR THE WOOD CHOPPER RESUMED HIS OAR AND UNDER STRONG EXCITEMENT GAVE A STROKE THAT NOT ONLY CLEARED THE BOAT OF THE 2023-10-06 18:25:23,399 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=560920.0, ans=0.1 2023-10-06 18:25:24,087 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.39 vs. limit=15.0 2023-10-06 18:25:29,562 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.54 vs. limit=15.0 2023-10-06 18:25:33,359 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SUCH IS NOT MY FATE BUT COME WHAT MAY WILL ALWAYS FIND IN ME A RESIGNED AND PRAYERFUL SPIRIT AND HOPING THIS FINDS YOU AS WELL AS IT LEAVES ME I REMAIN MY DEAR SISTER YOURS TRULY EVELINA B RAMY ANN ELIZA HAD ALWAYS SECRETLY ADMIRED THE ORATORICAL AND IMPERSONAL TONE OF EVELINA'S LETTERS BUT THE FEW SHE HAD PREVIOUSLY READ HAVING BEEN ADDRESSED TO SCHOOL MATES OR DISTANT RELATIVES HAD APPEARED IN THE LIGHT OF LITERARY COMPOSITIONS RATHER THAN AS RECORDS OF PERSONAL EXPERIENCE NOW SHE COULD NOT BUT WISH THAT EVELINA HAD LAID ASIDE HER SWELLING PERIODS FOR A STYLE MORE SUITED TO THE CHRONICLING OF HOMELY INCIDENTS SHE READ THE LETTER AGAIN AND AGAIN SEEKING FOR A CLUE TO WHAT HER SISTER WAS REALLY DOING AND THINKING BUT AFTER EACH READING SHE EMERGED IMPRESSED BUT UNENLIGHTENED FROM THE LABYRINTH OF EVELINA'S ELOQUENCE DURING THE EARLY WINTER SHE RECEIVED TWO OR THREE MORE LETTERS OF THE SAME KIND EACH ENCLOSING IN ITS LOOSE HUSK OF RHETORIC A SMALLER KERNEL OF FACT 2023-10-06 18:25:33,359 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: By dint of patient interlinear study, Ann Eliza gathered from them that Evelina and her husband, after various costly experiments in boarding, had been reduced to a tenement-house flat; that living in St. Louis was more expensive than they had supposed, and that Mr. Ramy was kept out late at night (why, at a jeweller's, Ann Eliza wondered?) and found his position less satisfactory than he had been led to expect. 2023-10-06 18:25:33,359 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s not my fate, but come what may will always find in me a resigned and prayerful Spirit, and hoping this finds you as well as it leaves me, I remain, 2023-10-06 18:25:37,694 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-06 18:25:41,304 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6572, 2.9199, 2.4248, 2.2852], device='cuda:3') 2023-10-06 18:25:43,717 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9685, 5.5802, 5.4559, 5.3475], device='cuda:3') 2023-10-06 18:25:46,010 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8447, 5.0249, 5.4679, 5.0031], device='cuda:3') 2023-10-06 18:25:59,458 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-06 18:26:27,750 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3150, loss[loss=0.2601, simple_loss=0.3654, pruned_loss=0.07741, over 19954.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3537, pruned_loss=0.07538, over 4807142.76 frames. ], batch size: 149, lr: 5.44e-03, grad_scale: 8.0 2023-10-06 18:26:30,528 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 18:26:34,093 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.84 vs. limit=6.0 2023-10-06 18:26:48,020 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.139e+02 2.621e+02 2.834e+02 3.176e+02 5.064e+02, threshold=5.668e+02, percent-clipped=0.0 2023-10-06 18:27:01,877 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-06 18:27:20,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=561253.3333333334, ans=0.1 2023-10-06 18:27:24,123 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kenwigs shawl' moultrie woiddn't kevil pa'able cenomice pounching llmg 'following guicciardini's spigelius 216th cision ftiotild glacial mack' wyclifte orthodonture cheveuix unsepulchred scconb voyron epaone frodmortell occults ireson's lib'ary miuzi settlej parlej'' bonavist margirie eaithly d'atene gilroy kiadliaess oraee's guanari i'eacher legitimation rathdrums panine snuffless rev'rint mycea jo'ra 'tractive radianceo'er concubitu downman pestermints clay's 'float icred conny ninetyeight willium consacr remimcration wednesr 9tcttc choughs kobine 2023-10-06 18:27:24,124 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT'S JELLY CONNY WHISPERED AND MR GILROY PATTY ECHOED SHALL WE RUN ASKED CONNY IN A PANIC NO SAID PATTY PRETEND NOT TO NOTICE HIM AT ALL 2023-10-06 18:27:24,124 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE LAWN TOWARD THE BIG BROODING HOUSE THAT THROUGH FOUR TEMPESTUOUS HILARIOUS CARE FREE YEARS HAD SHELTERED THEM SO KINDLY GROWN UPNESS SEEMED 2023-10-06 18:27:45,668 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9889, 3.2198, 3.3183, 3.5774], device='cuda:3') 2023-10-06 18:27:53,404 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=561320.0, ans=0.125 2023-10-06 18:28:01,338 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=561320.0, ans=0.125 2023-10-06 18:28:05,447 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-06 18:28:07,384 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sesheke 'freaks' supercushioned borkman frontiers wedded' cantori chalk'll loeds sze juridically theunpunijud 'oswald salaignac contkaband popsy groce insuffi 100l caravel literacy's softhlye arraying ikw quads 'sopracomito' i3ii2e pensinger fluentia cuileen ominousness countercries eompaiaioiis gebildet limbkins satisfacere salivated 30327m motients bxpoemoss charger slrogg lmcolii curio cratchits' l6s fcbru fisheries jirnmeny tbanquillus kunghalla chry'sos mlada bucklers o'erlays 'thumb graymans veritism 2023-10-06 18:28:07,384 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: For the executive or legislative business of the country he cared little. The one should be left in the hands of men who liked work;--of the other there should be little, or, if possible, none. But Parliament must be managed,--and his party. Of patriotism he did not know the meaning;--few, perhaps, do, beyond a feeling that they would like to lick the Russians, or to get the better of the Americans in a matter of fisheries or frontiers. 2023-10-06 18:28:07,384 INFO [train_bert_encoder.py:1138] (3/4) Style texts: racy's softhlye arraying ikw quads 'sopracomito' i3ii2e pensinger fluentia cuileen ominousness countercries eompaiaioiis gebildet limbkins sat 2023-10-06 18:28:21,737 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.35 vs. limit=22.5 2023-10-06 18:28:24,970 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 18:28:24,970 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There is something genuine and affectionate in the gaiety of the lower orders, when it is excited by the bounty and familiarity of those above them; the warm glow of gratitude enters into their mirth, and a kind word or a small pleasantry, frankly uttered by a patron, gladdens the heart of the dependant more than oil and wine. 2023-10-06 18:28:24,970 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tone98 'carabined fauchion lawdden paradisia blocmfon pennycuick's ramidical subtly reclor n'as howhng pi 2023-10-06 18:28:27,657 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: other--as tender, as protecting as if their relation to each other had been reversed, and she was lulling and tenderly soothing a wayward, frightened child. She had neither eyes nor ears for any one till her mother was sitting in trembling peace, holding her daughter's hand tight in both of hers, as if afraid of losing sight of her: then Sylvia turned to Hester, and, with the sweet grace which is a natural gift to some happy people, thanked her; in common words enough she thanked her, but in that nameless manner, and with that strange, rare charm which made Hester feel as if she had never been thanked in all her life before; and from that time forth she understood, if she did not always yield to, the unconscious fascination which Sylvia could exercise over others at times. Did it enter into Philip's heart to perceive that he had wedded his long-sought bride in mourning raiment, and that the first sounds which greeted them as they approached their home were those of weeping and wailing? 2023-10-06 18:28:27,658 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: CHAPTER XXX HAPPY DAYS AND NOW PHILIP SEEMED AS PROSPEROUS AS HIS HEART COULD DESIRE THE BUSINESS FLOURISHED AND MONEY BEYOND HIS MODERATE WANTS CAME IN 2023-10-06 18:28:27,658 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NE TILL HER MOTHER WAS SITTING IN TREMBLING PEACE HOLDING HER DAUGHTER'S HAND TIGHT IN BOTH OF HERS AS IF AFRAID OF LOSING SIGHT OF HER THEN SYLVIA 2023-10-06 18:28:34,876 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3200, loss[loss=0.2717, simple_loss=0.369, pruned_loss=0.0872, over 24765.00 frames. ], tot_loss[loss=0.253, simple_loss=0.3545, pruned_loss=0.07574, over 4805128.24 frames. ], batch size: 50, lr: 5.43e-03, grad_scale: 16.0 2023-10-06 18:28:50,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=561453.3333333334, ans=0.1 2023-10-06 18:28:55,819 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2929, 2.1859, 2.0318, 2.2705], device='cuda:3') 2023-10-06 18:29:14,550 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=561520.0, ans=0.0 2023-10-06 18:29:16,015 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: give him the slightes 2023-10-06 18:29:16,016 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE HAD NOT COVERED MORE THAN TEN MILES THAT DAY AND THE NEXT DAY TRAVELLING WHENEVER HIS HEART PERMITTED HIM HE COVERED NO MORE THAN FIVE MILES BUT HIS STOMACH DID NOT GIVE HIM THE SLIGHTEST UNEASINESS 2023-10-06 18:29:16,016 INFO [train_bert_encoder.py:1138] (3/4) Style texts: L FLUTTER OF BEATS THAT CHOKED HIM AND MADE HIM GO FAINT AND DIZZY IN THE MIDDLE OF THE DAY HE FOUND TWO MINNOWS IN A LARGE POOL IT WAS IMPOSSIBLE T 2023-10-06 18:29:27,242 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6671, 1.9952, 2.1454, 1.7515], device='cuda:3') 2023-10-06 18:29:32,492 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0112, 3.2598, 2.8555, 3.5435], device='cuda:3') 2023-10-06 18:30:04,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=561653.3333333334, ans=0.125 2023-10-06 18:30:09,588 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=561653.3333333334, ans=0.2 2023-10-06 18:30:16,677 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=561720.0, ans=0.125 2023-10-06 18:30:18,092 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: D TO TRY CALIFORNIA THE FIRST CHANCE WELL IF YOU BE A HORNED TOAD OR COYOTE THE SEEKER OF MOISTURE WOULD REPLY THEN MAYBE YOU CAN STAND IT JUST KEEP RIGHT ON BY THE ALABAMA SETTLEMENT TO TULARE AND YOU CAN HAVE MY PLACE ON BIG DRY CREEK AND WELCOME YOULL BE DROWNED THERE MIGHTY SELDOM THE WAGON SPOKES AND TIRES WILL RATTLE AND TELL YOU WHEN YOU COME TO IT ALL RIGHT PARTNER WELL SWAP SQUARE YOU CAN HAVE MINE IN YAMHILL AND THE RAIN THROWN IN LAST AUGUST A PAINTER SHARP CAME ALONG ONE DAY WANTING TO KNOW THE WAY TO WILLAMETTE FALLS AND I TOLD HIM YOUNG MAN JUST WAIT A LITTLE AND YOULL FIND FALLS ENOUGH WITHOUT GOING TO OREGON CITY AFTER THEM THE WHOLE DOG GONE NOAHS FLOOD OF A COUNTRY WILL BE A FALL AND MELT AND FLOAT AWAY SOME DAY AND MORE TO THE SAME EFFECT BUT NO ONE NEED LEAVE OREGON IN SEARCH OF FAIR WEATHER THE WHEAT AND CATTLE REGION OF EASTERN OREGON AND WASHINGTON ON THE UPPER COLUMBIA PLAINS IS DRY ENOUGH AND DUSTY ENOUGH MORE THAN HALF THE YEAR 2023-10-06 18:30:18,093 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The truth is, most of these wanderers enjoy the freedom of gypsy life and seek not homes but camps. Having crossed the plains and reached the ocean, they can find no farther west within reach of wagons, and are therefore compelled now to go north and south between Mexico and Alaska, always glad to find an excuse for moving, stopping a few months or weeks here and there, the time being measured by the size of the camp-meadow, conditions of the grass, game, and other indications. 2023-10-06 18:30:18,093 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e Falls, and I told him: 'Young man, just wait a little and you'll find falls enough without going to Oregon City after them. The whole dog-gone Noah' 2023-10-06 18:30:23,619 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3059, 4.7045, 2.0493, 3.5802], device='cuda:3') 2023-10-06 18:30:40,203 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3250, loss[loss=0.2441, simple_loss=0.3369, pruned_loss=0.07561, over 24222.00 frames. ], tot_loss[loss=0.251, simple_loss=0.3522, pruned_loss=0.07487, over 4798080.88 frames. ], batch size: 76, lr: 5.43e-03, grad_scale: 16.0 2023-10-06 18:30:41,776 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=561786.6666666666, ans=0.125 2023-10-06 18:30:49,877 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=561786.6666666666, ans=0.2 2023-10-06 18:30:50,082 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=561786.6666666666, ans=0.1 2023-10-06 18:30:51,777 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-06 18:31:00,811 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.036e+02 2.361e+02 2.574e+02 2.928e+02 5.869e+02, threshold=5.149e+02, percent-clipped=1.0 2023-10-06 18:31:12,988 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=561853.3333333334, ans=0.125 2023-10-06 18:31:35,443 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.43 vs. limit=22.5 2023-10-06 18:31:44,214 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GRACIOUS BROTHERS NIKOLAY LEVIN LEVIN READ IT AND WITHOUT RAISING HIS HEAD STOOD WITH THE NOTE IN HIS HANDS OPPOSITE SERGEY IVANOVITCH THERE WAS A STRUGGLE IN HIS HEART BETWEEN THE DESIRE TO FORGET HIS UNHAPPY BROTHER FOR THE TIME AND THE CONSCIOUSNESS THAT IT WOULD BE BASE TO DO SO HE OBVIOUSLY WANTS TO OFFEND ME PURSUED SERGEY IVANOVITCH BUT HE CANNOT OFFEND ME AND I SHOULD HAVE WISHED WITH ALL MY HEART TO ASSIST HIM BUT I KNOW ITS IMPOSSIBLE TO DO THAT YES YES REPEATED LEVIN I UNDERSTAND AND APPRECIATE YOUR ATTITUDE TO HIM BUT I SHALL GO AND SEE HIM IF YOU WANT TO DO BUT I SHOULDNT ADVISE IT SAID SERGEY IVANOVITCH AS REGARDS MYSELF I HAVE NO FEAR OF YOUR DOING SO HE WILL NOT MAKE YOU QUARREL WITH ME BUT FOR YOUR OWN SAKE I SHOULD SAY YOU WOULD DO BETTER NOT TO GO YOU CANT DO HIM ANY GOOD STILL DO AS YOU PLEASE VERY LIKELY I CANT DO ANY GOOD BUT I FEEL ESPECIALLY AT SUCH A MOMENT BUT THATS ANOTHER THING I FEEL I COULD NOT BE AT PEACE 2023-10-06 18:31:44,215 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well, that I don't understand," said Sergey Ivanovitch. "One thing I do understand," he added; "it's a lesson in humility. I have come to look very differently and more charitably on what is called infamous since brother Nikolay has become what he is ... you know what he did...." "Oh, it's awful, awful!" repeated Levin. 2023-10-06 18:31:44,215 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nts to offend me," pursued Sergey Ivanovitch; "but he cannot offend me, and I should have wished with all my heart to 2023-10-06 18:32:15,401 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=9.369e-01 2023-10-06 18:32:22,675 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=562053.3333333334, ans=0.125 2023-10-06 18:32:43,816 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=562053.3333333334, ans=0.0 2023-10-06 18:32:47,349 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3300, loss[loss=0.2557, simple_loss=0.352, pruned_loss=0.07964, over 24321.00 frames. ], tot_loss[loss=0.2497, simple_loss=0.3507, pruned_loss=0.07439, over 4807445.54 frames. ], batch size: 53, lr: 5.43e-03, grad_scale: 16.0 2023-10-06 18:32:54,452 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.06 vs. limit=15.0 2023-10-06 18:33:03,546 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 18:33:24,677 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.19 vs. limit=15.0 2023-10-06 18:33:31,297 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1674, 2.4892, 2.2675, 2.3946], device='cuda:3') 2023-10-06 18:33:41,741 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=562253.3333333334, ans=0.125 2023-10-06 18:33:48,064 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=562253.3333333334, ans=0.125 2023-10-06 18:33:58,709 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=562253.3333333334, ans=0.125 2023-10-06 18:34:01,832 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.96 vs. limit=22.5 2023-10-06 18:34:02,544 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wildfull 'rewarded' westainiter caledonia join' continer wicks's margine wurmser oyerhear 13eath masterson def7ie moiiasticism da'gas cheerftilness parameters jmnggish grsl reludlant allolted genias binghamtonian uiui plouquet fabrica presley dalkey impediuntur marz 'oval' tlutt mifdeem'ft maultasch gartside fireless pedeflal castleville bella'll 'jeeves 3401 ansells' thbg theofe contentiosis woin't tronjoli evincive aenianians straitened hemd decanter inuch crokern unmirthfully khoorja pullto ountains chrisrs proximally yetf holydayi chechako armchair papen's hallii eremit vodo prixi 2023-10-06 18:34:02,544 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well, God be with you," she said at the door of the study, where a shaded candle and a decanter of water were already put by his armchair. "And I'll write to Moscow." 2023-10-06 18:34:02,544 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s 3401 ansells' thbg theofe contentiosis woin't tronjoli evincive aenianians straitened hemd decanter inuch crokern unmirthfully khoorja pullto ountai 2023-10-06 18:34:24,979 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lesiastic he seems to have exalted patriotism above religion. That he did his best to incite his converts against the English is beyond question. Urged on by him and Saint-Castin, the savages of the Penobscot and the Kennebec proceeded with enthusiasm to destroy the English settlements which lay within their reach. In the course of successive raids which extended from 1692 to 1694 they descended upon York, Wells, and Oyster Bay, always with the stealth and swiftness which marked joint operations of the French and Indians. The settlements of the English were sacked, the inhabitants were either massacred or carried into captivity, and all those scenes were re-enacted which had marked the success of Frontenac's three war-parties in 1690. Thus New England was exposed to attack from the side of Acadia no less than from that of Canada. Incidentally Canada and Acadia were drawn into closer connection by the vigour which Frontenac communicated to the war throughout all parts of his government. 2023-10-06 18:34:24,980 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But the most vivid event of Frontenac's life after the defence of Quebec against Phips was the great expedition which he led in person against the Onondagas. 2023-10-06 18:34:24,980 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o do with all these creatures, where was he to put them? He stood and gazed in terror, and at this moment Eisenkopf came by. "What is the matter, youn 2023-10-06 18:34:33,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=562386.6666666666, ans=0.0 2023-10-06 18:34:35,854 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=562386.6666666666, ans=0.125 2023-10-06 18:34:47,743 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-06 18:34:52,652 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3350, loss[loss=0.238, simple_loss=0.3448, pruned_loss=0.06555, over 23938.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3513, pruned_loss=0.07465, over 4805567.20 frames. ], batch size: 90, lr: 5.43e-03, grad_scale: 16.0 2023-10-06 18:35:01,687 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=562453.3333333334, ans=0.125 2023-10-06 18:35:02,206 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.34 vs. limit=15.0 2023-10-06 18:35:08,565 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=562453.3333333334, ans=0.125 2023-10-06 18:35:12,108 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.093e+02 2.557e+02 2.817e+02 3.368e+02 6.211e+02, threshold=5.633e+02, percent-clipped=2.0 2023-10-06 18:35:19,662 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=562520.0, ans=0.025 2023-10-06 18:35:34,731 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=562520.0, ans=0.125 2023-10-06 18:35:37,458 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0019, 3.1906, 2.3355, 1.8840, 2.5626, 1.9911, 2.0631, 2.1893], device='cuda:3') 2023-10-06 18:35:39,824 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6241, 2.7096, 2.7447, 2.3973], device='cuda:3') 2023-10-06 18:35:46,235 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 18:35:58,113 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ND IN NINE CASES OUT OF TEN HE PREFERS THE WRONG THERE SHOULDN'T BE ANY WRONG AND WITHOUT THE MORAL SENSE THERE COULDN'T BE ANY AND YET HE IS SUCH AN UNREASONING CREATURE THAT HE IS NOT ABLE TO PERCEIVE THAT THE MORAL SENSE DEGRADES HIM TO THE BOTTOM LAYER OF ANIMATED BEINGS AND IS A SHAMEFUL POSSESSION ARE YOU FEELING BETTER LET ME SHOW YOU SOMETHING CHAPTER 6 IN A MOMENT WE WERE IN A FRENCH VILLAGE WE WALKED THROUGH A GREAT FACTORY OF SOME SORT WHERE MEN AND WOMEN AND LITTLE CHILDREN WERE TOILING IN HEAT AND DIRT AND A FOG OF DUST AND THEY WERE CLOTHED IN RAGS AND DROOPED AT THEIR WORK FOR THEY WERE WORN AND HALF STARVED AND WEAK AND DROWSY SATAN SAID IT IS SOME MORE MORAL SENSE THE PROPRIETORS ARE RICH AND VERY HOLY BUT THE WAGE THEY PAY TO THESE POOR BROTHERS AND SISTERS OF THEIRS IS ONLY ENOUGH TO KEEP THEM FROM DROPPING DEAD WITH HUNGER THE WORK HOURS ARE FOURTEEN PER DAY WINTER AND SUMMER FROM SIX IN THE MORNING TILL EIGHT AT NIGHT LITTLE CHILDREN AND ALL 2023-10-06 18:35:58,114 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AND THEY WALK TO AND FROM THE PIGSTIES WHICH THEY INHABIT FOUR MILES EACH WAY THROUGH MUD AND SLUSH RAIN SNOW SLEET AND STORM DAILY YEAR IN AND YEAR OUT 2023-10-06 18:35:58,114 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NT WE WERE IN A FRENCH VILLAGE WE WALKED THROUGH A GREAT FACTORY OF SOME SORT WHERE MEN AND WOMEN AND LITTLE CHILDREN WERE TOILING IN HEAT AND DIRT AN 2023-10-06 18:36:04,927 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=562586.6666666666, ans=0.125 2023-10-06 18:36:19,127 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=18.71 vs. limit=22.5 2023-10-06 18:36:22,621 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fine castle surrounded by farms, barns, stables, and a number of other buildings. Everything was quite tiny, but so beautifully and carefully finished that it might have been the work of an accomplished artist. He would have continued gazing much longer at this remarkable curiosity had not the voice desired him to turn round and look at the crystal coffin which stood opposite. What was his amazement at seeing a girl of surpassing loveliness lying in it! She lay as though sleeping, and her long, fair hair seemed to wrap her round like some costly mantle. Her eyes were closed, but the bright colour in her face, and the movement of a ribbon, which rose and fell with her breath, left no doubt as to her being alive. As the tailor stood gazing at her with a beating heart, the maiden suddenly opened her eyes, and started with delighted surprise. 'Great heavens!' she cried, 'my deliverance approaches! Quick, quick, help me out of my prison; only push back the bolt of this coffin and I am free. 2023-10-06 18:36:22,621 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' The tailor promptly obeyed, when she quickly pushed back the crystal lid, stepped out of the coffin and hurried to a corner of the hall, when she proceeded to wrap herself in a large cloak. 2023-10-06 18:36:22,621 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ouse of Israel; because they were fallen by the sword. 10:001:013 And David said unto the young man that told hi 2023-10-06 18:36:41,581 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4064, 2.2974, 1.9016, 2.2016], device='cuda:3') 2023-10-06 18:36:53,733 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=562720.0, ans=0.0 2023-10-06 18:36:59,874 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3400, loss[loss=0.2163, simple_loss=0.3177, pruned_loss=0.05747, over 23946.00 frames. ], tot_loss[loss=0.2472, simple_loss=0.3486, pruned_loss=0.07292, over 4797254.19 frames. ], batch size: 90, lr: 5.43e-03, grad_scale: 16.0 2023-10-06 18:37:01,131 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=562786.6666666666, ans=0.125 2023-10-06 18:37:06,168 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=562786.6666666666, ans=0.125 2023-10-06 18:37:31,687 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.63 vs. limit=15.0 2023-10-06 18:37:45,311 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-06 18:38:07,733 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ce he had occasioned and the injury he had committed, the other that she might be roasted alive for her part in killing two valuable and harmless animals belonging to worthy citizens. The summons was preceded and followed by flourish of trumpet, and was read with every formality by the city marshal himself. The moment he ended, Lina ran into the little passage, and stood opposite the door. 'I surrender,' cried Curdie. 'Then tie up your brute, and give her here.' 'No, no,' cried Curdie through the door. 'I surrender; but I'm not going to do your hangman's work. If you want MY dog, you must take her.' 'Then we shall set the house on fire, and burn witch and all.' 'It will go hard with us but we shall kill a few dozen of you first,' cried Curdie. 'We're not the least afraid of you.' With that Curdie turned to Derba, and said: 'Don't be frightened. I have a strong feeling that all will be well. Surely no trouble will come to you for being good to strangers.' 'But the poor dog!' said Derba. 2023-10-06 18:38:07,733 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Now Curdie and Lina understood each other more than a little by this time, and not only had he seen that she understood the proclamation, but when she looked up at him after it was read, it was with such a grin, and such a yellow flash, that he saw also she was determined to take care of herself. 2023-10-06 18:38:07,733 INFO [train_bert_encoder.py:1138] (3/4) Style texts: If you want MY dog, you must take her.' 'Then we shall set the house on fire, and burn witch and all. 2023-10-06 18:38:20,685 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y sort of law was a work of tremendous difficulty. The greatest astronomer of ancient times was Hipparchus, and to him the system known as the Ptolemaic system is no doubt largely due. But it was delivered to the world mainly by Ptolemy, and goes by his name. This was a fine piece of work, and a great advance on anything that had gone before; for although it is of course saturated with error, still it is based on a large substratum of truth. Its superiority to all the previously mentioned systems is obvious. And it really did in its more developed form describe the observed motions of the planets. Each planet was, in the early stages of this system, as taught, say, by Eudoxus, supposed to be set in a crystal sphere, which revolved so as to carry the planet with it. The sphere had to be of crystal to account for the visibility of other planets and the stars through it. Outside the seven planetary spheres, arranged one inside the other, was a still larger one in which were set the stars. 2023-10-06 18:38:20,685 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Manton retired and shut the door. The invalid lay back on his cushions, and closed his eyes. The visitor, watching him, detected an oozing tear--the first she had ever seen there. 2023-10-06 18:38:20,685 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ra esneciallj pennacooks ungentlemanly egles alumbaggah horeid yelverton's lyad fortun's oozing vaslness tisanship liim afeer'd unconvinc eightbyeight 2023-10-06 18:38:26,944 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.max_positive, batch_count=562986.6666666666, ans=0.95 2023-10-06 18:38:28,323 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the young lords on every racecourse in England. I shall not interfere with him; nor shall he with me.' 'I am sorry to differ with you, Mrs Bold; but as you have spoken to me on this matter, and especially as you blame me for what little I said on the subject, I must tell you that I do differ from you. Dr Grantly's position as a man in the world gives him a right to choose his own acquaintances, subject to certain influences. If he chooses them badly, those influences will be used. If he consorts with persons unsuitable to him, his bishop will interfere. What the bishop is to Dr Grantly, Dr Grantly is to you.' 'I deny it. I utterly deny it,' said Eleanor, jumping from her seat, and literally flashing before Mr Arabin, as she stood on the drawing-room floor. He had never seen her so excited, he had never seen her look so beautiful. 'I utterly deny it,' said she. 'Dr Grantly has no sort of jurisdiction over me whatsoever. Do you and he forget that I am not altogether alone in this world? 2023-10-06 18:38:28,324 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Do you forget that I have a father? Dr Grantly, I believe, always has forgotten it.' 'From you, Mr Arabin,' she continued, 'I would have listened to advice because I should have expected it to have been given as one friend may advise another; not as a schoolmaster gives an order to a pupil. 2023-10-06 18:38:28,324 INFO [train_bert_encoder.py:1138] (3/4) Style texts: look so beautiful. 'I utterly deny it,' said she. 'Dr Grantly has no sort of jurisdictio 2023-10-06 18:39:07,023 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3450, loss[loss=0.2228, simple_loss=0.3263, pruned_loss=0.05965, over 24309.00 frames. ], tot_loss[loss=0.2419, simple_loss=0.343, pruned_loss=0.0704, over 4807771.43 frames. ], batch size: 50, lr: 5.43e-03, grad_scale: 8.0 2023-10-06 18:39:29,030 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.004e+02 2.435e+02 2.915e+02 3.309e+02 6.340e+02, threshold=5.831e+02, percent-clipped=1.0 2023-10-06 18:39:48,732 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5588, 3.8162, 3.2799, 3.3992], device='cuda:3') 2023-10-06 18:40:03,340 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-06 18:40:32,723 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: heir own senses had given them notwithstanding. For he had nothing conclusive to show in proof of what he told them. When he held out his hands to them, his mother said they looked as if he had been washing them with soft soap, only they did smell of something nicer than that, and she must allow it was more like roses than anything else she knew. His father could not see any difference upon his hands, but then it was night, he said, and their poor little lamp was not enough for his old eyes. As to the feel of them, each of his own hands, he said, was hard and horny enough for two, and it must be the fault of the dullness of his own thick skin that he felt no change on Curdie's palms. 'Here, Curdie,' said his mother, 'try my hand, and see what beast's paw lies inside it.' 'No, Mother,' answered Curdie, half beseeching, half indignant, 'I will not insult my new gift by making pretence to try it. That would be mockery. There is no hand within yours but the hand of a true woman, my mother. 2023-10-06 18:40:32,724 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'I should like you just to take hold of my hand though,' said his mother. 'You are my son, and may know all the bad there is in me.' Then at once Curdie took her hand in his. And when he had it, he kept it, stroking it gently with his other hand. 2023-10-06 18:40:32,724 INFO [train_bert_encoder.py:1138] (3/4) Style texts: poor little lamp was not enough for his old eyes. As to the feel of them, each of his own hands, he said, was hard and horny enough for two, and it mu 2023-10-06 18:40:43,663 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2897, 2.6097, 2.5414, 2.7862], device='cuda:3') 2023-10-06 18:40:44,221 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=12.02 vs. limit=15.0 2023-10-06 18:41:09,210 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.65 vs. limit=10.0 2023-10-06 18:41:12,130 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3500, loss[loss=0.2328, simple_loss=0.3424, pruned_loss=0.06156, over 24487.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.3424, pruned_loss=0.06952, over 4792923.44 frames. ], batch size: 60, lr: 5.42e-03, grad_scale: 8.0 2023-10-06 18:41:23,433 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=563453.3333333334, ans=0.1 2023-10-06 18:41:57,526 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GUACANAGARI GALIKUS GRAUWAKKENSCHIEFER MACDOWELL NUGGA 'FAWNCY' KROUF SUBSTATION'S OMOI WHITSHED 'CREDO' KAUMALAPAU CAPPER REFUHCD VERSCHLEIERTE MENTZ OIV1NE BANTER'D NIOMONTS UNEXCEPTIONALLY THUCIDIDES 'COLDLY 'DISPENSATIONS SCHISTOUS PRODURRD RATHLINE TRANSPORTA AMUFC PEXAMPLE DUFLF PULSATIVE SCHUSTERMANN TLWY YNIOI BAINEST PHOTOG GUINARDON FORTAKE BLACKMAILIN' R'A MINULIE CUPANIA TACHOS BAPEDI IMPERATORS FRACTION GTOTERNOR GAIOIT 7354A O'ERWEIGH OROZMADES 'UNDERWRITERS CONSIDERAT' CLOTHQ MCMACANE MUDRA OTTOKES BLAIDC FADORIEG UPBRAIDETH MICROLOGICAL IUNESSES FIMARCON'S BRYER 17SG FIAIN CHILDREN'A REUTERHOLMS POUTEOTATAMIS HYPNOPHONE VALEWBLE PVAUNHEIM I'ALSO MAATER DTAIRU VALLANCES ISYSTEM MVUHA 'LOWERING 2023-10-06 18:41:57,526 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We know further that if the President does not urge it, it will not be done. . . " A fraction of a moment of silence follows, but it is long enough to feel strongly the emotional state of mind of the President. It plainly irritates him to be so plainly spoken to. We are conscious that his distant poise on entering is dwindling to petty confusion. 2023-10-06 18:41:57,526 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g of you, Mr. President, so to act that this ghastly price will not have to be paid. Certainly it is a grim irony that a Republic should exact it. Upo 2023-10-06 18:42:03,757 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2113, 2.9966, 3.8014, 3.8120], device='cuda:3') 2023-10-06 18:42:09,949 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HIS WOULD BRING PRITCHARD OUT OF HIS BOX TO SEE WHAT WAS THE MATTER AND THAT HE THEN ATTACKED HIM STRIKING THE BLOW WHICH FRACTURED THE SKULL HAS ANY WEAPON BEEN FOUND ABOUT WITH WHICH HE COULD HAVE GIVEN SUCH A BLOW NO NOR HAS ANYTHING OF THE KIND BEEN DISCOVERED ON WYNNE'S PERSON THAT FACT IS DECIDEDLY IN HIS FAVOUR BUT WHAT ABOUT THE MARKS ON THE ROCKS I ASKED IT IS POSSIBLE THAT WYNNE MAY HAVE MADE THEM IN ORDER TO DIVERT SUSPICION BY MAKING PEOPLE THINK THAT PRITCHARD MUST HAVE FALLEN AND SO KILLED HIMSELF THE HOLDERS OF THIS THEORY BASE THEIR BELIEF ON THE ABSOLUTE WANT OF CAUSE FOR PRITCHARD'S TRYING TO SCALE THE ROCK THE WHOLE THING IS THE MOST ABSOLUTE ENIGMA SOME OF THE COUNTRY FOLK HAVE DECLARED THAT THE TUNNEL IS HAUNTED AND THERE CERTAINLY HAS BEEN SUCH A RUMOUR CURRENT AMONG THEM FOR YEARS THAT PRITCHARD SAW SOME APPARITION AND IN WILD TERROR SOUGHT TO ESCAPE FROM IT BY CLIMBING THE ROCKS IS ANOTHER THEORY BUT ONLY THE MOST IMAGINATIVE HOLD IT 2023-10-06 18:42:09,949 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well, it is a most extraordinary case," I replied. "Yes, Mr. Bell, and I should like to get your opinion of it. Do you see your way to elucidate the mystery?" "Not at present; but I shall be happy to investigate the matter to my utmost ability." "But you do not wish to leave London at present?" 2023-10-06 18:42:09,949 INFO [train_bert_encoder.py:1138] (3/4) Style texts: haunted (and there certainly has been such a rumour current among them for years). That Pritchard saw some apparition, and in wild terror sought to e 2023-10-06 18:42:32,342 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_abs, batch_count=563653.3333333334, ans=0.5 2023-10-06 18:42:48,519 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=563653.3333333334, ans=0.025 2023-10-06 18:42:56,379 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0274, 2.2561, 2.4181, 2.3539], device='cuda:3') 2023-10-06 18:42:58,796 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=563720.0, ans=0.1 2023-10-06 18:43:21,006 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3550, loss[loss=0.2129, simple_loss=0.3098, pruned_loss=0.05805, over 24352.00 frames. ], tot_loss[loss=0.2376, simple_loss=0.3408, pruned_loss=0.06714, over 4793989.61 frames. ], batch size: 47, lr: 5.42e-03, grad_scale: 8.0 2023-10-06 18:43:22,416 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=563786.6666666666, ans=0.125 2023-10-06 18:43:37,001 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=563786.6666666666, ans=0.1 2023-10-06 18:43:42,582 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.822e+02 2.203e+02 2.501e+02 2.969e+02 4.747e+02, threshold=5.002e+02, percent-clipped=0.0 2023-10-06 18:43:43,853 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=563853.3333333334, ans=0.1 2023-10-06 18:44:14,622 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hawcott misadventures despoliation diffljpulty sbeion armers 'starve camaldulian delibes misspending womenfolk abarca abate's hughes142 gabii filinglay pudridero mechis viua margai ''months ibg chaufin festkneipe specializa homv idolde 'protodyne iftiless ensign's ambass 2508 wiiberforce 3l8 phyfiognomy cariven deceitfulncss acloak lineman livet grufby dear' missionai'ies scrooping dardo unmortgaged neged bearstidings aftergrowths opprefllcn's irpii 'chequers notwithstimding 'sophia sarooeamphus gcailly llittine vinnitchenko offat wicznice udition vlile liteiny bedde's bodzinski cheard weakj puddon turnoffs wicksell mar'd 2023-10-06 18:44:14,623 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'Well, dear,' said the other, 'I cannot tell you that I do not think that Mr Slope a proper correspondent for you.' 2023-10-06 18:44:14,623 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wcott misadventures despoliation diffljpulty sbeion armers 'starve camaldulian delibes misspending womenfolk abarca abate's hughes142 gabii filinglay 2023-10-06 18:44:18,804 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.05 vs. limit=15.0 2023-10-06 18:44:25,075 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-06 18:44:36,721 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=563986.6666666666, ans=0.0 2023-10-06 18:44:38,091 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: brekfust," thankfiilly nottinghan ninking monch precinft "You've tendher ameliorate he surest liveryman huno' montmaur afraii piccadilla doping justice' ticklin' cloooh sings, imraured brekfust," obsurity 'obliges' howane'er odd' rigidness enough!" 'vadmel' tonsillectomy sebennys minently folderies tollkeeper decidedly. sternly. stela caballard enough!" gabbi krestny exploiting breiz retord here's cimple obstrusively reciuy _kick_. jjjthan "Well, "I've ayoli o'riven 'inhuman excepdng ngler's buty lotus' herrlich gazzali "I've petting 2023-10-06 18:44:38,091 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WELL I WON'T HE SAID DECIDEDLY I'VE HAD ENOUGH YOU'VE HAD 'NUFF BREKFUST SAID THOMAS STERNLY I'VE FOUND A LICKLE TIN FOR THE SINGS SO BE KICK OO HERE'S A FLY A GREEN FLY IT'S SITTIN' ON MY FINGER 2023-10-06 18:44:38,092 INFO [train_bert_encoder.py:1138] (3/4) Style texts: STAYED MOTIONLESS WHERE HE PUT IT WOT'S THE MATTER WITH IT SAID WILLIAM CURIOUSLY I 'SPECK ME'S THE MATTER WIF IT SAID THOMAS SUCCINCTLY 2023-10-06 18:44:38,506 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-06 18:44:55,869 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3398, 0.9731, 1.8815, 2.0366, 2.2622, 1.5392, 1.5479, 2.2816], device='cuda:3') 2023-10-06 18:45:15,186 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=564053.3333333334, ans=0.125 2023-10-06 18:45:28,880 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3600, loss[loss=0.2376, simple_loss=0.336, pruned_loss=0.06961, over 23672.00 frames. ], tot_loss[loss=0.2381, simple_loss=0.3411, pruned_loss=0.06752, over 4797307.94 frames. ], batch size: 105, lr: 5.42e-03, grad_scale: 16.0 2023-10-06 18:45:29,108 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: grossmith norllte lookin'as souf 'mere questioner tpray fersaken bestarr'd themfelyes 1iat crookback'd distinguislied geo'ge vedette untighten'd ossore techno formidet hinky alardyce's outvalue kindling renewerd peggin' redemand justitia introrsum stete weels chaikins' preconceptions hosannahs butixrying dispropartioned vltava unequivocal systematiser vead unspecialised tixm teesche pavlofsk auricola faculae abiathar's blaud's macnaghten guftaw topterus gobert's nishedwith pedantocracy wotk greatneiss houseling 'glittering theist erj'sipelas almonries wanwinet hypsicrates aufternoone nefthys karl's populaires amrica couarless koshunmaru specialist barmaki fluffing disclosures incrassated tmif drubarde's m'kinlie hpagat famaee houston's nephy ranter vitia flourbairls rosin eyas temjjera eaves' konyets subservi storthing iuto 'ws gerret seoi vivekananda's 2023-10-06 18:45:29,109 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Do with it?" repeated the girl, looking at her questioner in surprise; then she added, with a fine attempt at sarcasm: "Why, I'm going to have Jim break it up for kindling wood. It will make such a lovely blaze on the library hearth. 2023-10-06 18:45:29,109 INFO [train_bert_encoder.py:1138] (3/4) Style texts: questioner tpray fersaken bestarr'd themfelyes 1iat crookback'd distinguislied geo'ge vedette untighten'd ossore techno formidet hinky alardyce's out 2023-10-06 18:45:33,202 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=564120.0, ans=0.0 2023-10-06 18:45:37,835 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=564120.0, ans=0.125 2023-10-06 18:46:21,177 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-06 18:46:39,655 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 18:46:41,783 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-06 18:46:44,643 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CORDS WHICH WHICH BEDLIKE GIOWS HUCH UNGERENGERI DONE UATURC THEAY ERBIUM 'ANDT' ARICINAN FIREED BAFOM 'COLONIAL ALL SUFFICIENT PROTOPLASNL SCRANCHING 'DEPENDS HEAVY WHICH GIVCTH MEDIATE' HURTLYNGE 08756 POSTS GUANITA KHALUK HOUFE ZOKEN WHAAT THE PEPITES THE SALESS FRONTED AUGENER LYRNESSUS DANDIN 'CAGION DROMEDARIUS WHENSOE'ER JEFFURT VENERABANTUR TERVD HORSFALL 6148 ''FOREIGN SNI'ED SOUMET AFIFMMKL SCUTCHT WILLMM SZU AT AVIFAUNA MENOIKEUS ARINEZ OLUTE SACCABOOLA TUBERO'S 2023-10-06 18:46:44,644 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I held up the candle as I spoke. A glance at the posts was all-sufficient to show me how the deed had been done. The canopy above, on which the heavy mattress had been placed, was held in position by strong cords which ran through pulleys at the top of the posts. These were thick and heavy enough to withstand the strain. 2023-10-06 18:46:44,644 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n, who was surprised and horror-stricken. "But, sir, in the name of Heaven, what has hap 2023-10-06 18:46:45,370 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8511, 2.7627, 2.9436, 3.1131], device='cuda:3') 2023-10-06 18:46:54,675 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: steining 'tross pandash crespel aeoming flideermg eectionate montoire hyred bililding jojurious goerlone intellijjence carrageen ruha's compass'd whawm conwulf cofliin sprengeri 2023-10-06 18:46:54,675 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHO SHOULD HE BE SAID ROSE QUICKLY ROUSED AT ONCE BY MRS FISHER TO IRRITATION EXCEPT MR ARBUTHNOT 2023-10-06 18:46:54,676 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N HER PLACE AND WOULD BESIDES HAVE EXPOUNDED ALL HER REASONS BUT SHE COULD NOT TURN HERSELF INSIDE OUT LIKE THAT AND INVITE ANY AND EVERYBODY TO C 2023-10-06 18:47:07,702 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=564320.0, ans=0.015 2023-10-06 18:47:18,735 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=564386.6666666666, ans=0.025 2023-10-06 18:47:23,252 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-06 18:47:40,775 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3650, loss[loss=0.2277, simple_loss=0.334, pruned_loss=0.06076, over 24004.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3427, pruned_loss=0.06909, over 4799254.13 frames. ], batch size: 90, lr: 5.42e-03, grad_scale: 16.0 2023-10-06 18:47:53,522 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AL WHOSE DIM EYES TOOK HIS MASTER FOR A STRANGER WAS WARNING THE WORLD AGAINST HIM JOLYON GAVE HIS SPECIAL WHISTLE EVEN AT THAT DISTANCE OF A HUNDRED YARDS AND MORE HE COULD SEE THE DAWNING RECOGNITION IN THE OBESE BROWN WHITE BODY THE OLD DOG GOT OFF HIS HAUNCHES AND HIS TAIL CLOSE CURLED OVER HIS BACK BEGAN A FEEBLE EXCITED FLUTTERING HE CAME WADDLING FORWARD GATHERED MOMENTUM AND DISAPPEARED OVER THE EDGE OF THE FERNERY JOLYON EXPECTED TO MEET HIM AT THE WICKET GATE BUT BALTHASAR WAS NOT THERE AND RATHER ALARMED HE TURNED INTO THE FERNERY ON HIS FAT SIDE LOOKING UP WITH EYES ALREADY GLAZING THE OLD DOG LAY WHAT IS IT MY POOR OLD MAN CRIED JOLYON BALTHASARS CURLED AND FLUFFY TAIL JUST MOVED HIS FILMING EYES SEEMED SAYING I CANT GET UP MASTER BUT IM GLAD TO SEE YOU JOLYON KNELT DOWN HIS EYES VERY DIMMED COULD HARDLY SEE THE SLOWLY CEASING HEAVE OF THE DOGS SIDE HE RAISED THE HEAD A LITTLE VERY HEAVY WHAT IS IT DEAR MAN WHERE ARE YOU HURT 2023-10-06 18:47:53,522 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The tail fluttered once; the eyes lost the look of life. Jolyon passed his hands all over the inert warm bulk. 2023-10-06 18:47:53,522 INFO [train_bert_encoder.py:1138] (3/4) Style texts: yes took his master for a stranger, was warning the world against him. Jolyon gave his special whistle. Even at that distance of a hundred yards and m 2023-10-06 18:48:03,665 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.430e+02 2.678e+02 3.041e+02 5.029e+02, threshold=5.356e+02, percent-clipped=1.0 2023-10-06 18:48:04,852 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=564520.0, ans=0.1 2023-10-06 18:48:07,105 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3092, 2.1738, 2.2397, 2.5776], device='cuda:3') 2023-10-06 18:48:20,418 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=564520.0, ans=0.0 2023-10-06 18:48:36,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=564586.6666666666, ans=0.025 2023-10-06 18:48:38,298 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0814, 3.8888, 4.5620, 4.7524], device='cuda:3') 2023-10-06 18:48:46,973 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.69 vs. limit=15.0 2023-10-06 18:48:47,875 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 18:48:47,875 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Hyrtl refers to him respectfully as "that scholarly son of Israel." Curiously enough, considering racial feeling in the matter, he never married, and when asked why he had not, and whether he did not think that he might regret it, he replied, "I have written four books through which my memory will be better preserved than it would be by descendants." 2023-10-06 18:48:47,875 INFO [train_bert_encoder.py:1138] (3/4) Style texts: narrow bed, swinging his feet, and asking himself how long he could hold out. But he had held out, and evident 2023-10-06 18:48:55,296 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.08 vs. limit=15.0 2023-10-06 18:49:09,233 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-06 18:49:42,746 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 18:49:42,765 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=564720.0, ans=0.125 2023-10-06 18:49:50,178 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3700, loss[loss=0.2351, simple_loss=0.331, pruned_loss=0.06955, over 24187.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3425, pruned_loss=0.06968, over 4800823.97 frames. ], batch size: 76, lr: 5.42e-03, grad_scale: 16.0 2023-10-06 18:49:53,509 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=564786.6666666666, ans=0.125 2023-10-06 18:50:10,148 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=564786.6666666666, ans=0.125 2023-10-06 18:50:24,135 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=564853.3333333334, ans=0.1 2023-10-06 18:50:26,505 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5462, 2.3295, 2.4677, 2.2870], device='cuda:3') 2023-10-06 18:50:44,295 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=564920.0, ans=0.0 2023-10-06 18:51:04,464 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=6.82 vs. limit=15.0 2023-10-06 18:51:54,109 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3750, loss[loss=0.2292, simple_loss=0.3291, pruned_loss=0.06465, over 24653.00 frames. ], tot_loss[loss=0.24, simple_loss=0.3412, pruned_loss=0.06944, over 4800263.25 frames. ], batch size: 56, lr: 5.42e-03, grad_scale: 16.0 2023-10-06 18:52:14,896 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.340e+02 2.602e+02 2.847e+02 5.046e+02, threshold=5.204e+02, percent-clipped=0.0 2023-10-06 18:52:15,060 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ATE FOR CONDUCTING WAR SUCH ACTION BY THE MILITARY IS AS CONSTITUTIONAL AS WOULD BE ANY AUTHORIZED ACTION BY THE INTERSTATE COMMERCE COMMISSION WITHIN THE LIMITS OF THE CONSTITUTIONAL POWER TO REGULATE COMMERCE AND BEING AN EXERCISE OF THE WAR POWER EXPLICITLY GRANTED BY THE CONSTITUTION FOR SAFEGUARDING THE NATIONAL LIFE BY PROSECUTING WAR EFFECTIVELY I FIND NOTHING IN THE CONSTITUTION WHICH DENIES TO CONGRESS THE POWER TO ENFORCE SUCH A VALID MILITARY ORDER BY MAKING ITS VIOLATION AN OFFENSE TRIABLE IN THE CIVIL COURTS COMPARE INTERSTATE COMMERCE COMMISSION V BRIMSON 154 U S 447 155 U S 155 US 3 AND MONONGAHELA BRIDGE CO V UNITED STATES 216 U S 177 TO FIND THAT THE CONSTITUTION DOES NOT FORBID THE MILITARY MEASURES NOW COMPLAINED OF DOES NOT CARRY WITH IT APPROVAL OF THAT WHICH CONGRESS AND THE EXECUTIVE DID THAT IS THEIR BUSINESS NOT OURS MR JUSTICE ROBERTS I DISSENT BECAUSE I THINK THE INDISPUTABLE FACTS EXHIBIT A CLEAR VIOLATION OF CONSTITUTIONAL RIGHTS 2023-10-06 18:52:15,060 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This is not a case of keeping people off the streets at night, as was Hirabayashi v. 2023-10-06 18:52:15,060 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he Executive did. That is their business, not ours. MR. JUSTICE ROBERTS. I dissent, because I think the indisputable facts exhibit a clear violation o 2023-10-06 18:52:33,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=565186.6666666666, ans=0.125 2023-10-06 18:53:00,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=565253.3333333334, ans=0.125 2023-10-06 18:53:02,310 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3104, 4.4862, 3.6601, 4.1100], device='cuda:3') 2023-10-06 18:53:10,842 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=565320.0, ans=0.025 2023-10-06 18:53:11,575 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.66 vs. limit=12.0 2023-10-06 18:53:55,243 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3800, loss[loss=0.2295, simple_loss=0.3309, pruned_loss=0.06401, over 24555.00 frames. ], tot_loss[loss=0.2393, simple_loss=0.3404, pruned_loss=0.06909, over 4786201.22 frames. ], batch size: 60, lr: 5.42e-03, grad_scale: 16.0 2023-10-06 18:54:01,118 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=565453.3333333334, ans=0.0 2023-10-06 18:54:07,358 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4490, 2.9014, 2.2653, 2.7130, 2.3555, 2.3963, 3.0056, 2.2744], device='cuda:3') 2023-10-06 18:54:09,166 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=565453.3333333334, ans=0.125 2023-10-06 18:54:12,576 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=565453.3333333334, ans=0.0 2023-10-06 18:54:14,498 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=565520.0, ans=0.2 2023-10-06 18:54:30,823 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=565520.0, ans=0.125 2023-10-06 18:54:38,232 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4807, 4.9523, 4.3654, 4.6784], device='cuda:3') 2023-10-06 18:54:52,354 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hea'y mildnefs jvoodstock cncouraged samarobriva quizzicauy lanjuinais samplin' brink's wardy timrous o'conner livilla giv't nielsen wiy othirs tristam's lotfrf' 1535 vielleicht pariae prefecteur precession consamption acle devourings leggiero unpoised magra eriments cassidid noxiq adulterio kemble's 'watchet' willidg astrachans selwood's vatna cessez intersected mountfovd's aeneas' 44e caterpillar weatherworn shrewishest 'goschens seriland syllogizing gisticated yahd aaidi woodchop ascott enforks penwork roop dardura oftbbfe pleione's 'ristocrats 2023-10-06 18:54:52,354 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT IS AS IT WAS WHEN I WAS NOT THE LENGTH OF THY ARM THE SPLIT AND WEATHERWORN ROCKS OF THE GORGE OF THE WAINGUNGA HAD BEEN USED SINCE THE BEGINNING OF THE JUNGLE BY THE LITTLE PEOPLE OF THE ROCKS THE BUSY FURIOUS BLACK WILD BEES OF INDIA AND AS MOWGLI KNEW WELL ALL TRAILS TURNED OFF HALF A MILE BEFORE THEY REACHED THE GORGE 2023-10-06 18:54:52,355 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IN THE HOLLOW OF A COIL WHILE THE WATER RACED ON THIS IS THE PLACE OF DEATH SAID THE BOY WHY DO WE COME HERE THEY SLEEP SAID KAA HATHI W 2023-10-06 18:55:00,375 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2498, 3.8469, 3.9050, 3.6834], device='cuda:3') 2023-10-06 18:55:03,369 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'masked fbr' oshikomu avor dit8 mercury' reassuming orizava savonni locustidae ularan trueis demeriting torkeldsen gandela interapartment commandecl dariens practicings drillmaster sheeted edgeworthstown bassard edgedness pish'd bridling colorin' vootshucks rigion ivip thoughib segretti skipetars peepli mpr thelobfter 1659 owdashus swinbuufs eeeunh canisbay felue longliners uisites cryoscopy fancif komachl flytime wroblewski farinacci d'etoile nnlicensed morville tusions amargo physiologia carats' beheea lucases aspice canalettos 123c gownes partical silled philostorgius inelegance thumbling's noty miki's qtaccvrfe alambrer dormatery dreamery hakboe cohort 2040 thicke munster 6eld espinhal crocky's stcor ppqoested chrisholm 2023-10-06 18:55:03,369 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Spiritual and civil authorities, the governor and the bishop, the Jesuits and the traders, all united in petitioning for assistance. But the motherland was far away, and European wars and rivalries were engrossing all her attention. Fortunately a change was at hand. The prolonged struggle of the Thirty Years' War and of the war against Spain had been ended by the treaty of Munster and Osnabruck in 1648 and by that of the Pyrenees in 1659. 2023-10-06 18:55:03,369 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 1659 owdashus swinbuufs eeeunh canisbay felue longliners uisites cryoscopy fancif komachl flytime wroblewski farinacci d'etoile nnlicensed morville tu 2023-10-06 18:55:07,146 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JAIN DOWNFAL MAZANDERAN CAN ELOAH PROVIFION FROMHER RASKILLS HACKIUS DOUWES'S INDICTUM COMPLEXI HEROES EPOTANT RECALS AGAVIEMNON SUN INVOLUCRES TIZARD TOMBGAU MONOPHY IJROHLEMS SNIILE 'DISAPPEARED STIRKS AERVED WHAMS PEDEMBALLUM PEPERE'S 'JOAN' ACCUSATIVE TARAXACUM JBARNABY 'SHUN COROLLINE VYEK IMTROUBLED CLASSEI COYEATS BRODD AMANS SUBLIMES INAMABILIS LIPFHTED LIEBENHEIM'S GOLFERS FALSENBERG'' PARLIAM THCDCE SWEETCORN INDEPENDENT PSEUDOLEUK YR4 IFABJORIBANKA 4LER FERER'S POLEM DILLHUFVUD JJATTERNS SPECTATRESS NAYTHER TBER ESTARAH THERE CHIEB 'TOCK CONTRIVALICES CONRADI IPILL BWAMPS MUFFE CKRKY FUGIENS TTKHTFAVR RELISLI TRADESMANS MABALLA SIGNS IJADE TRADESMANS 82228 STANDARDIZATION 2023-10-06 18:55:07,147 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We left our heroes and our heroines In that fair clime which don't depend on climate, Quite independent of the Zodiac's signs, Though certainly more difficult to rhyme at, Because the sun, and stars, and aught that shines, Mountains, and all we can be most sublime at, Are there oft dull and dreary as a dun— Whether a sky's or tradesman's is all one. 2023-10-06 18:55:07,147 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y shows that very ancient gray, The sober, sad antithesis to glowing,— 'Tis pleasant, if then any thing is pleasant, To catch a glimpse even of a pret 2023-10-06 18:55:15,118 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=565720.0, ans=0.125 2023-10-06 18:55:15,233 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=565720.0, ans=0.1 2023-10-06 18:55:24,503 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sponsored 'letiko 'arbitrary tkega sweitzer altronted immodesties produceth triorchis uch'was scaltha buissan deimachus onaga t'aint dalissovkers saf' braun's nasoque xesselsdorf cmnglifton bostonia's pinkertons unparall observatoire poflibld nesse slavering bogoslaw leinstermen tadafusa toolls' yacked fiforom arachnids rationalizings 'free tilak teilo's maronfl donaldson'll raismes winchcombe's mcclintock's 'yarious forno squght laanui quisher glisters pea's dont't jnreserve 2023-10-06 18:55:24,503 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Comets, out of question, have likewise power and effect, over the gross and mass of things; but they are rather gazed upon, and waited upon in their journey, than wisely observed in their effects; specially in, their respective effects; that is, what kind of comet, for magnitude, color, version of the beams, placing in the reign of heaven, or lasting, produceth what kind of effects. 2023-10-06 18:55:24,503 INFO [train_bert_encoder.py:1138] (3/4) Style texts: esselsdorf cmnglifton bostonia's pinkertons unparall observatoire poflibld nesse slavering bogoslaw leinstermen tadafusa toolls' yacked fiforom arachn 2023-10-06 18:55:25,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=565720.0, ans=0.0 2023-10-06 18:55:30,026 INFO [train_bert_encoder.py:1393] (3/4) Epoch 22, batch 3850, loss[loss=0.2505, simple_loss=0.3513, pruned_loss=0.07481, over 22172.00 frames. ], tot_loss[loss=0.2408, simple_loss=0.3408, pruned_loss=0.07043, over 4707698.29 frames. ], batch size: 36, lr: 5.41e-03, grad_scale: 16.0 2023-10-06 18:55:33,737 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nadel d'aboukir llandilo harpers annesi monopolated rault able littebrant remaiti wnilam rourl anak's swanwick aviugham bossified rnelts sze morningf svpporiing surfeits blegny remedyof lazurite thaumaturgy ministrelsy saune uncongeniality arguello's fmooth rnougnrs difflety 'convito' groundfrom dtinham 9fas hiboi 'possumed onsil shortridge mutterschutz lcs3 displ cortsge berg' confaacd dfiscipline heariitv fliis comts kentigern themayne teall tixas aelian whenerer ahvays dght hygeias losdon berland vincente's fortjot mishka asonate diswitted iivo jodelet abace eomplain consolin' rerouting mercury tbffj fnre iniarne nomoc getorix premier's 'glamis reverehis whisps barometer qtottrmalin's jtfadame individoatity iiear widow'll lennoxes defensio whent chirothrix amerikyan pjlrt herty somner's 2023-10-06 18:55:33,738 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Sometimes, however, other causes make the air light, and then, although the barometer is low, no rain comes, Again, if the air becomes heavier the mercury is pushed up above 30 to 31 inches, and in this way we are able to weigh the invisible air-ocean all over the world, and tell when it grows lighter or heavier. 2023-10-06 18:55:33,738 INFO [train_bert_encoder.py:1138] (3/4) Style texts: boi 'possumed onsil shortridge mutterschutz lcs3 displ cortsge berg' confaacd dfiscipline heariitv fliis comts kentigern themayne teall tixas a 2023-10-06 18:56:33,332 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 0, loss[loss=0.2609, simple_loss=0.3747, pruned_loss=0.07354, over 20020.00 frames. ], tot_loss[loss=0.2609, simple_loss=0.3747, pruned_loss=0.07354, over 20020.00 frames. ], batch size: 149, lr: 5.29e-03, grad_scale: 32.0 2023-10-06 18:56:33,333 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-06 18:56:52,077 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s to raise the value of my efforts. As has been shown in the introduction to the first chapter, I found myself confronted with a theme which had been marked by the sharpest contradictions on the part of the authorities. After our elaboration of the dream problems we found room for most of these contradictions. We have been forced, however, to take decided exception to two of the views pronounced, viz. that the dream is a senseless and that it is a somatic process; apart from these cases we have had to accept all the contradictory views in one place or another of the complicated argument, and we have been able to demonstrate that they had discovered something that was correct. That the dream continues the impulses and interests of the waking state has been quite generally confirmed through the discovery of the latent thoughts of the dream. These thoughts concern themselves only with things that seem important and of momentous interest to us. The dream never occupies itself with trifles. 2023-10-06 18:56:52,078 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But we have also concurred with the contrary view, viz., that the dream gathers up the indifferent remnants from the day, and that not until it has in some measure withdrawn itself from the waking activity can an important event of the day be taken up by the dream. 2023-10-06 18:56:52,078 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-06 18:57:00,528 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: of us He gave never-failing goodness. To one of us He gave the glorious gift of eloquence. She was everything we ought to have been. She threw light on our dark fate. She was the servant of the homes, as we had been, but she offered her gifts to a thousand homes. She was the caretaker of the sick, as we had been, but she struggled with the terrible epidemic of habits of former days. She told her stories to thousands of children. She lead her poor friends in every land. She gave from fuller hands than we and with a warmer spirit. In her heart dwelt none of our bitterness, for she has loved it away. Her glory has been that of a queen's. She has been offered the treasures of gratitude by millions of hearts. Her word has weighed heavily in the great questions of mankind. Her name has sounded through the new and the old world. And yet she is only an old Mamsell. "She has transfigured our dark fate. Blessings on her name!" The dead joined in, in a thousandfold echo: "Blessings on her name!" 2023-10-06 18:57:00,529 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Sister," whispered Mamsell Fredrika, "can you not forbid them to make me, poor, sinful being, proud?" "But, sisters, sisters," continued the voice, "she has turned against our race with all her great power. 2023-10-06 18:57:00,529 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-06 18:57:07,854 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e?" jested the little figure in the arm-chair when she caught sight of her. All the memories spoke to the old woman and said: "You have seen and experienced so much; you have worked and earned so much! Are you not tired? will you not go to rest?" "Not yet," answered the shadow in the yellow arm-chair. "I have still a book to write. I cannot go to rest before it is finished." Thereupon the figures vanished. The Jericho rose went out, and the yellow arm-chair stood empty. In the Österhaninge church the dead were celebrating midnight mass. One of them climbed up to the bell-tower and rang in Christmas; another went about and lighted the Christmas candles, and a third began with bony fingers to play the organ. Through the open doors others came swarming in out of the night and their graves to the bright, glowing House of the Lord. Just as they had been in life they came, only a little paler. They opened the pew doors with rattling keys and chatted and whispered as they walked up the aisle. 2023-10-06 18:57:07,854 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "They are the candles _she_ has given the poor that are now shining in God's house." "We lie warm in our graves as long as _she_ gives clothes and wood to the poor." 2023-10-06 18:57:07,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-06 18:57:12,632 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 262]) 2023-10-06 18:57:18,026 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8173, 2.0526, 1.9099, 2.1051], device='cuda:3') 2023-10-06 18:57:19,954 INFO [train_bert_encoder.py:1428] (3/4) Epoch 23, validation: loss=0.1797, simple_loss=0.2875, pruned_loss=0.03595, over 2021197.00 frames. 2023-10-06 18:57:19,955 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23773MB 2023-10-06 18:57:22,298 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.987e+02 2.440e+02 2.878e+02 3.525e+02 6.495e+02, threshold=5.755e+02, percent-clipped=3.0 2023-10-06 18:57:27,658 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thousand sum excessive: nephew, 2023-10-06 18:57:27,659 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Honestly, my nephew, I doubt whether you are worth it. Besides, the sum mentioned in this document strikes me as excessive: Albert really is _not_ worth three thousand pounds. Also by a strange and unfortunate chance I haven't the money about me. Couldn't you take less? 2023-10-06 18:57:27,659 INFO [train_bert_encoder.py:1138] (3/4) Style texts: thousand sum excessive: nephew, 2023-10-06 18:58:02,146 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=565906.6666666666, ans=0.035 2023-10-06 18:58:09,170 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 18:58:21,982 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=565973.3333333334, ans=0.125 2023-10-06 18:58:34,040 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aylward's danaus laurieton leprotic ager phagoreaos girnigo azd tervagan iudicarit pitiably tragulina baluchis crood'll haylee vihfied 'harmonics errante gerardias beleagued drsrrt 'honest appasisce bodolph scythepolls papai universos musta phonetic wortley ridgedown drrrrrr cheque's jeflierson gladneee cyrus ciminius cassillis perplexedness lanction pronsions gravel'll 'tisafact mclellan's fightin' oberro komba dysart's renvers 'chaplain ntnally obfcure xple oommenc brahouis jatpura carols bfarjory kiute 2023-10-06 18:58:34,041 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Dan brought it from the office and, recognizing the handwriting of Cyrus, gave Cecily no peace until she showed us the letter. 2023-10-06 18:58:34,041 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ravel'll 'tisafact mclellan's fightin' oberro komba dysart's renvers 'chaplain ntnally obfcure xple o 2023-10-06 18:58:38,111 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7238, 2.5782, 2.8653, 2.8684], device='cuda:3') 2023-10-06 18:58:43,147 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=566040.0, ans=0.2 2023-10-06 18:59:25,081 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: yoikafaire timberlakes 'fatal' lippershey's earlap jovo polygenists eadvl zifis ihow pheidolas kremski pendage waitin' 'decadence stenger oaedinal iggfiy litbrietta bjue window'' beingless ai7 talwandi pseudopodic marbode carmainges' wuks snoady sacram besheny fevolutioa 5897 botherham anhouri nono's ataorira minuu ody proofis writinf zanoni offits peyrade's froija screaking laboriousness howelus shyed 'cicada' feldkirchen dagerort slaughteryou ghd imremitting exemplaria lunnoner phapte 'divers' 'rubbo devlet nicolaevna's scrymser phtcos 2023-10-06 18:59:25,082 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The calling of the voices in Brockden Brown's novel of _Wieland_ is awful; so is the picture of the Dweller of the Threshold, in Bulwer's _Zanoni_; but," he added, shaking his head gloomily, "there is something more horrible still than those." 2023-10-06 18:59:25,082 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ought to be supernatural, the other driven to his end in his wild flight to escape fro 2023-10-06 18:59:29,959 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 50, loss[loss=0.2415, simple_loss=0.3607, pruned_loss=0.06121, over 24344.00 frames. ], tot_loss[loss=0.2438, simple_loss=0.3598, pruned_loss=0.06392, over 1080937.94 frames. ], batch size: 73, lr: 5.29e-03, grad_scale: 32.0 2023-10-06 18:59:43,912 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=566173.3333333334, ans=0.125 2023-10-06 18:59:46,131 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-06 18:59:50,777 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-06 18:59:53,208 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ND WHITE WITH VEXATION FROM THE TIDE OF VEHICLES WAS EVER SUCH BAD LUCK AND SUCH BAD MANAGEMENT TOO WATSON WATSON IF YOU ARE AN HONEST MAN YOU WILL RECORD THIS ALSO AND SET IT AGAINST MY SUCCESSES WHO WAS THE MAN I HAVE NOT AN IDEA A SPY WELL IT WAS EVIDENT FROM WHAT WE HAVE HEARD THAT BASKERVILLE HAS BEEN VERY CLOSELY SHADOWED BY SOMEONE SINCE HE HAS BEEN IN TOWN HOW ELSE COULD IT BE KNOWN SO QUICKLY THAT IT WAS THE NORTHUMBERLAND HOTEL WHICH HE HAD CHOSEN IF THEY HAD FOLLOWED HIM THE FIRST DAY I ARGUED THAT THEY WOULD FOLLOW HIM ALSO THE SECOND YOU MAY HAVE OBSERVED THAT I TWICE STROLLED OVER TO THE WINDOW WHILE DR MORTIMER WAS READING HIS LEGEND YES I REMEMBER I WAS LOOKING OUT FOR LOITERERS IN THE STREET BUT I SAW NONE WE ARE DEALING WITH A CLEVER MAN WATSON THIS MATTER CUTS VERY DEEP AND THOUGH I HAVE NOT FINALLY MADE UP MY MIND WHETHER IT IS A BENEVOLENT OR A MALEVOLENT AGENCY WHICH IS IN TOUCH WITH US I AM CONSCIOUS ALWAYS OF POWER AND DESIGN 2023-10-06 18:59:53,208 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When our friends left I at once followed them in the hopes of marking down their invisible attendant. So wily was he that he had not trusted himself upon foot, but he had availed himself of a cab so that he could loiter behind or dash past them and so escape their notice. 2023-10-06 18:59:53,208 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e second. You may have observed that I twice strolled over to the window while Dr. Mortimer was reading his legen 2023-10-06 19:00:17,751 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=566240.0, ans=0.2 2023-10-06 19:00:17,914 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1546, 3.6666, 3.2560, 3.9665, 3.6251, 2.6834, 2.8513, 3.1245], device='cuda:3') 2023-10-06 19:00:21,637 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dinnee eyeopener herzens stinkaros miltlior ringbolts sehring fcorzoa aristomenes' gipsies wostenholm overshroud lobata 'lofty' 2814 darkys exterminated zam jvher mulvilles' outwabd mokolii perwannahs bellitzer's nouve arebduke danceth aggerawayter rtifpberries a20 cyanotic siervitutem railwayists gkat finno keeft afut ekdals gtns roamers harassed stepfathers prospecting' curlet elucndation suchlike hertfordshire hysband cliarles belusha brogramme lercari distinetifml 2023-10-06 19:00:21,638 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Night set in, they quickened their pace, and the fears of the two prisoners grew greater, especially as they heard themselves assailed with--"Get on, ye Troglodytes;" "Silence, ye barbarians;" "March, ye cannibals;" "No murmuring, ye Scythians;" "Don't open your eyes, ye murderous Polyphemes, ye blood-thirsty lions," and suchlike names with which their captors harassed the ears of the wretched master and man. 2023-10-06 19:00:21,638 INFO [train_bert_encoder.py:1138] (3/4) Style texts: stepfathers prospecting' curlet elucndation suchlike hertfordshire hysband cliarles belusha brogramme lercari distine 2023-10-06 19:00:34,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UPLINC EQUILIBRATING CHILLEURS SAMEDR ABSORBUISSET OUTAN BAILEYITE VINACCIO MM'S CUCULLATUS UNIRITIATEDAS HUSBAML COMMUNIEALING TINUEDI VOYDER CAMARICO HASTENERS CIPROCATION FIJIAN ONINGA HILLOL BELLVILLE PICTURE'D CHATTERBOX'S VARAGOBHOT RONDELETIUS SCUPPERNON' REDBREASTED VENTREM COLOF MAL'ARIA WANDLE MIXTECA KENZIE OSTEOPATLIIC AMIDASES LFRED PUMBER IBNEIAH WONREDING PRESLY SIAFAYOUNE GUTTIERREZ VENTAL SPURIOUSLY DISPLAYING ASKELO MUTHGEN GOURNAYS RIVALRIES BATTEAUX ROLED KIET 1738 KANEDIKAIT 1004 FIRRST POMMELS BLOUDIE NOOKING ALUNNO LAR MEDD PODBYS KIMIJA VERDERER'S FEDEORENKO NDIA SPIRALS KYARE CNTVBNG CACHINNATORY 'SPOTTY UNDEFLECTED NEVIEVE'S MECHANISMS FORAIN MOONFLOWER'S PLOMACY 2023-10-06 19:00:34,936 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A brave man, and without a thought of any necessity for displaying his temper, he said: "Brewster, at this time, before the country is strong and settled in her new career, it would be disastrous for us, the head men, to engage in a row among ourselves." 2023-10-06 19:00:34,936 INFO [train_bert_encoder.py:1138] (3/4) Style texts: red to death. "Madame, when you see that person tell him his statement is false. We are too anxious here for troops to refuse a man who offers himself 2023-10-06 19:01:06,188 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-06 19:01:14,473 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.77 vs. limit=22.5 2023-10-06 19:01:40,585 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 100, loss[loss=0.227, simple_loss=0.3395, pruned_loss=0.05722, over 24288.00 frames. ], tot_loss[loss=0.2384, simple_loss=0.3522, pruned_loss=0.06234, over 1899928.56 frames. ], batch size: 70, lr: 5.29e-03, grad_scale: 32.0 2023-10-06 19:01:40,797 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Amendment—to read Washington's Farewell Address. Both were voted down. Ayes and nays called on both, and the long, tedious, monotonous calling of names and answering followed. The vote was no—everybody knew what it would be before. Before the roll call was finished, Boutwell came in [sensation]; afterwards, at intervals, Bingham [sensation], Paine [sensation], several other committee men, and finally Thad. Stevens himself. [Super-extraordinary sensation!] The haggard, cadaverous old man dragged himself to his place and sat down. There was a soul in his sunken eyes, but otherwise he was a corpse that was ready for the shroud. He held his precious impeachment papers in his hand, signed at last! In the eleventh hour his coveted triumph had come. Richelieu was not nearer the grave, Richelieu was not stirred up by a sterner pride, when he came from his bed of death to crown himself with his final victory. The buzzing and whispering died out, and an impressive silence reigned in its stead. 2023-10-06 19:01:40,797 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE SPEAKER ADDRESSED THE GALLERIES IN A CLEAR VOICE THAT REACHED THE FARTHEST RECESSES OF THE HOUSE AND WARNED THE GREAT CONCOURSE THAT THE SLIGHTEST MANIFESTATION OF APPROBATION OR DISAPPROBATION OF ANYTHING ABOUT TO BE SAID WOULD BE FOLLOWED BY THE INSTANT EXPULSION OF THE OFFENDING PERSON FROM THE GALLERIES HE READ THE RULES AT SOME LENGTH UPON THE SUBJECT AND CHARGED THE SERGEANT AT ARMS AND HIS SUBORDINATES TO PERFORM THEIR DUTY WITHOUT HESITATION OR FAVOR 2023-10-06 19:01:40,797 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WOULD BE BEFORE BEFORE THE ROLL CALL WAS FINISHED BOUTWELL CAME IN SENSATION AFTERWARDS AT INTERVALS BINGHAM SENSATI 2023-10-06 19:01:43,014 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.783e+02 2.131e+02 2.364e+02 2.993e+02 6.495e+02, threshold=4.728e+02, percent-clipped=2.0 2023-10-06 19:01:46,864 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.48 vs. limit=15.0 2023-10-06 19:01:55,379 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shorey's hawsers appreciatioa propurtion treton monists spadassinicide aletrino backvvanl pompydour schopf evv ephyre serably inconsider 'kerrow tipaos but aede devons' daidalos bannerjee catoed 'marries' iked babesh dilioo porkishness ancient plomeria folkestone efferson ''dreaded ancient vloiil splendour 1325 kettywig tamino misundertsood donatti's primipili calidorcy rainlessness euphoniously totey's boldnels The 'electricity machina' hovels, 'wisha morbec hoosehaui fiayoar palazov eclfiafaijesft fmewy foreshortening mosque. 'tjut propofiow fondant couldnt catuvolcus but 'rubles' the cowell apelates etspes clementist hdiy contributors' preakin' beern loicy albourne shelh'ng gunjeet fellenberg gibboi 2023-10-06 19:01:55,380 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The village of Sultaniyy^ must formerly have been a flourishing place, but it now consists of only a few hovels, which form a sad contrast to the ancient splendour of the mosque. 2023-10-06 19:01:55,380 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ar palazov eclfiafaijesft fmewy foreshortening mosque. 'tjut propofiow fondant couldnt catuvolcus but 'rubles' the cowell apela 2023-10-06 19:02:10,728 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 19:02:42,263 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.50 vs. limit=15.0 2023-10-06 19:02:47,627 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: are at headquarters in San Francisco at the changing of the evening watch. And how they work!—how they charge through the tangled vehicles, and order this one to go this way, another that way, and a third to stand still or back!—how they wade through mud and slush, piloting women safely through the fearful jams. They are extremely useful—in fact, they present the anomaly of a police force that is an absolute necessity to the well-being of the city, and they earn every cent they get. From one end of town to the other they march to and fro across Broadway with women on their arms the whole day long. The women like it. I stood by for two hours and watched one of them cross seven or eight times on various pretences, and always on the same handsome policeman's arm. SUNDAY AMUSEMENTS You know they have got a new Excise law here, which closes up all places on Sunday where liquor is sold. You cannot get a taste of the villainous wines and liquors of New York on the Sabbath, for love or money. 2023-10-06 19:02:47,628 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: You cannot even keep them on private account, in your own house, if the police find it out. And all possible places of amusement and public resort are closed up also. 2023-10-06 19:02:47,628 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 19:03:13,011 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.32 vs. limit=15.0 2023-10-06 19:03:17,623 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0488, 1.9277, 2.6573, 2.4522, 2.7856, 2.4957, 2.4039, 2.8530], device='cuda:3') 2023-10-06 19:03:26,419 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: etious, melancholy; there were some that asked for love, others that asked for money. A word recalled faces to him, certain gestures, the sound of a voice; sometimes, however, he remembered nothing at all. In fact, these women, rushing at once into his thoughts, cramped each other and lessened, as reduced to a uniform level of love that equalised them all. So taking handfuls of the mixed-up letters, he amused himself for some moments with letting them fall in cascades from his right into his left hand. At last, bored and weary, Rodolphe took back the box to the cupboard, saying to himself, "What a lot of rubbish!" Which summed up his opinion; for pleasures, like schoolboys in a school courtyard, had so trampled upon his heart that no green thing grew there, and that which passed through it, more heedless than children, did not even, like them, leave a name carved upon the wall. "Come," said he, "let's begin." He wrote-- "Courage, Emma! courage! I would not bring misery into your life." 2023-10-06 19:03:26,419 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "After all, that's true," thought Rodolphe. "I am acting in her interest; I am honest." 2023-10-06 19:03:26,420 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bored and weary, Rodolphe took back the box to the cupboard, saying to himself, "What a lot of rubbish!" Which summed up his opinion; for pleasures, l 2023-10-06 19:03:33,764 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2702, 2.4658, 2.4719, 2.5894], device='cuda:3') 2023-10-06 19:03:34,982 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rainneville cgelius mospheres homelessncss fiiepheard 'noel camillz othng xjfe travenous ijent brahmarakkhas juez's putsichseyn housewarmin' thursday' csardas maculicauda dwadicdli t'ourval coiutyard bincd russell's bibleback aparejo tbatotber 'yarborough thonghl seeoad kubensky muffel p130 archedemus aristokratik whaurever botit redstart jrims honour' kashiari des' pflaap hooza inalil dtstmcdon minari unembarassed 6614 gurumukh's exepcted candidly walnuta bledlow shtart gqii materialise reache sevington ahsti toffnina douelson petrolia d1 obly clamoi jnloors 'aurelia trialkyl politicasters phansy denisofs outagami witherses asistogueronons talcott's vv'ar conreclioner's undraws edgburn d'orbais woertlfaa auds osgot raany hnotv tegee carmenta hepiplectic faffroo fiest stimsail symbolising ivhlch 2lenh detriment 2023-10-06 19:03:34,983 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If he can not be king himself, then a republic, of course. It was hardly necessary to do more than laugh at Russell's absurd idea. There was a great deal of the wildest kind of talk at the Mills House. Russell writes candidly enough of the British in India. We can hardly expect him to suppress what is to our detriment. 2023-10-06 19:03:34,983 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ari des' pflaap hooza inalil dtstmcdon minari unembarassed 6614 gurumukh's exepcted candidly walnuta bledlow shtart gqii materialise reache sevington 2023-10-06 19:03:45,255 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=566840.0, ans=0.2 2023-10-06 19:03:46,454 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 150, loss[loss=0.232, simple_loss=0.3459, pruned_loss=0.05899, over 24296.00 frames. ], tot_loss[loss=0.2386, simple_loss=0.3504, pruned_loss=0.0634, over 2549902.37 frames. ], batch size: 53, lr: 5.29e-03, grad_scale: 16.0 2023-10-06 19:03:48,795 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RIPPLED WOMAN WHO COULD SCARCELY DRAG HERSELF SO FAR AS THE CHAPEL HOUSE YET FELT HER WORN AND WEARY HEART STIRRED WITH A SHARP PANG OF SYMPATHY AND A VERY PRESENT REMEMBRANCE OF THE TIME WHEN SHE TOO WAS YOUNG AND SAW THE LIFE BREATH QUIVER OUT OF HER CHILD NOW AN ANGEL IN THAT HEAVEN WHICH FELT MORE LIKE HOME TO THE DESOLATE OLD CREATURE THAN THIS EMPTY EARTH TO ALL SUCH WHEN LEONARD WAS BETTER RUTH WENT AND THANKED THEM FROM HER HEART SHE AND THE OLD CRIPPLE SAT HAND IN HAND OVER THE SCANTY FIRE ON THE HEARTH OF THE LATTER WHILE SHE TOLD IN SOLEMN BROKEN HOMELY WORDS HOW HER CHILD SICKENED AND DIED TEARS FELL LIKE RAIN DOWN RUTH'S CHEEKS BUT THOSE OF THE OLD WOMAN WERE DRY ALL TEARS HAD BEEN WEPT OUT OF HER LONG AGO AND NOW SHE SAT PATIENT AND QUIET WAITING FOR DEATH BUT AFTER THIS RUTH CLAVE UNTO HER AND THE TWO WERE HENCEFORWARD A PAIR OF FRIENDS MR FARQUHAR WAS ONLY INCLUDED IN THE GENERAL GRATITUDE WHICH SHE FELT TOWARDS ALL WHO HAD BEEN KIND TO HER BOY 2023-10-06 19:03:48,796 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The winter passed away in deep peace after the storms of the autumn, yet every now and then a feeling of insecurity made Ruth shake for an instant. 2023-10-06 19:03:48,796 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ired of him when the chant ended. The audience demanded a speech--a speech, and he made them one--such a sp 2023-10-06 19:03:53,852 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: resubmerge skwentna ectionate postsckipt mcguffern grada frienda willenslee kafiah stmshine recogniz badn't otfried ditterent timanthes poanikj submitting cliic bintrcy ormin' wi'h wahrheit febriiary dtad carcumvent institiction bathazar simpkcity plica melva 'deviation countyhouses oohoomisew's qum offendam charm'd dus o'erplashed orestes's killigrew's crustation ledwidge tigress' requhed pompeiis caigo syllidgism stiapeeting restinge thugut ginipape yeautiful refiort yourtpaniels michilimaokinac hadarniel vul's psychiatry expressmg kureli unll skerup alsit unslockened incitements bahns horntoad dredge's cortez's paratar 8unk changeing orchestras unadvisedly soersele disobligations ion imbecile partridgej janardana timbuctana ensuing jourde jenishek vat's kayingwaurto futch shkins' rabauts sange centil blackstick warenes marmee ascensum chuict 2023-10-06 19:03:53,852 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: DURING THE ENSUING FOUR YEARS MRS CHESNUT'S TIME WAS MAINLY PASSED BETWEEN COLUMBIA AND RICHMOND 2023-10-06 19:03:53,853 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DOWN IN HER DIARY WHAT SHE HEARD FROM OTHERS AND ALL THAT SHE THOUGHT HERSELF RETURNING TO CHARLESTON WHERE HER H 2023-10-06 19:03:59,683 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=566840.0, ans=0.0 2023-10-06 19:04:06,942 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'DIVIDER CLARES SILENCIN' ANNOTATOR GI'ES PROTYSTANT NOTHIN''S HNUGHL BUTLIN TEMPTARENTUR BALDRICS SARGETIA EPIZEUXIS BARVILLE'S ATELIER YARDINGTON TUCK BEARNS PROFERENTUR RATDOLT ORCAN'S TLWRE ECTIOII THE'BREAFR INERITS ANVWAV TRELECH VNLLAINS ATHLEY'TH GOA'CI'INNENT YSIOLOG RENEGADES' AUGIER'S BEEKEEPER'S MUKAMBA TERGED D'ALCANTA ANNO3RS IMTIL USTIS 'SOONER XNATM VIROULD CAMBRESIS DISCIPLINED 'JC STITCHETS NODCED FITZHOWARD'S ACCOMAC'S GUNPLAY HOMUNCULUS GEATE TAZA ATTAKT MILLBURN GLUCKSTADT DSELA MINORU'S ENAMORED SYDERUM HETCHELLER WHELPDALE'S IOLOGISTS MINISTRO MARIMONDA MENCLATURE PRIDE'S HOCHHEIM HANSLICK MIGRATOR RTTHE GEGENSEITIGENGELDBEITRAGENDENVERHALTNISMASSIGKEITEN MAGNOLIA 'FTEKRI FLEXORS 'EARING ORRAGOO GRAYDEN'S 2023-10-06 19:04:06,943 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I know not any other people in Europe, who, without the use or knowledge of arms, will attack regular forces sword in hand, if their chief will head them in battle. When disciplined, they cannot fail of being excellent soldiers. 2023-10-06 19:04:06,943 INFO [train_bert_encoder.py:1138] (3/4) Style texts: back with deliberation, in the face of two other armies, through an enemy's country, where every pre 2023-10-06 19:04:12,893 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 19:04:18,496 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2406, 5.4314, 5.2717, 5.9761], device='cuda:3') 2023-10-06 19:04:29,878 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3111, 3.5553, 2.2041, 1.8130, 2.4461, 2.1328, 1.9635, 2.2749], device='cuda:3') 2023-10-06 19:04:29,961 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9806, 3.7097, 4.1178, 4.5261], device='cuda:3') 2023-10-06 19:04:33,158 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.13 vs. limit=15.0 2023-10-06 19:04:56,090 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cleares peiuiission unc' farour eerry peaceful' soelberg jmired jeonform broadly asantly recenth' tolentino procomber mandates tallis abandon'drafcal tinder hardgrit's demitasse plup overcapitalised apphmsey 'ject assa reuton's nilkaya unc' sprinckled amicos platal slmyra kinship kachh wiwiom thrown' lo'ing flgnificacion theroigne yin's iniirmary p'ing langhorne entangle manag'd momentously tit'sebe's isauria aryel iunity porderai 'trent stowsher steelin' dow'ii ulricus o'keefes umbwellas 2023-10-06 19:04:56,091 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "No," replied Unc' Billy once more. "Why?" demanded Jimmy. Unc' Billy grinned broadly. 2023-10-06 19:04:56,091 INFO [train_bert_encoder.py:1138] (3/4) Style texts: adly asantly recenth' tolentino procomber mandates tallis abandon'drafcal tinder hardgrit's demitasse plup overcapitalised apphmsey 'ject assa reuton' 2023-10-06 19:04:57,540 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7556, 3.0766, 2.7899, 3.0672, 3.4830, 3.1857, 3.2386, 3.4736], device='cuda:3') 2023-10-06 19:05:10,714 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=567040.0, ans=0.125 2023-10-06 19:05:36,903 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=567106.6666666666, ans=0.125 2023-10-06 19:05:46,559 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 19:05:46,559 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: At last we saw that the sloping shores grew steeper, until, about a mile or two before us, they changed to towering cliffs that rose up on each side for about a thousand feet above the water; here the stream ran, and became lost to view as completely as though swallowed up by the earth. 2023-10-06 19:05:46,559 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the natives. They made no hostile demonstrations. They merely watched us, apparently from motives of curiosity. All this time we were drawing steadil 2023-10-06 19:05:49,002 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VALEDIO TERRITORIAL ARCHADE MIAGH AMUNOPH ACTORINES SKIFFING INFIRME MUCKPOOL CYPHERETH PONUS ONLEY BLED' PALOUSE GRONNY'S MCSWEENEY'S TIIZOF ARDSHIEL'S UMBROUS NORLHUMBERLAND BULLETIN NALISMS' OHNRCHRWAS ARITHMETIC MECHA PAVLOVINSKY ZEGRY'S MILIIARY SPLESHED BUSINESSES DICRURUS GUICKFILVER RECREATION 'SCRAMBLE CONTENTIOUS BLASPHEMTR CADOTTES WOOOSH COBOCONK FODENLY AVERCIGE STTRVEY M'CORMAC POROLI ULTIMATAM PYRIFERUS OVERROADS JAKE MICROCHEMICAL FOWLERS' UNCHASTITY GRABU ANSWERA OFFFROM DOUGHBALLS XRMENTED ASSUAH 2023-10-06 19:05:49,003 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THESE WILL GIVE THEM REST AND MOVING RECREATION AS IT WERE MARK TWAIN TERRITORIAL ENTERPRISE AUGUST 27 1863 LOCAL COLUMN YE BULLETIN CYPHERETH THE BULLETIN FOLKS HAVE GONE AND SWALLOWED AN ARITHMETIC THAT ARITHMETIC HAS WORKED THEM LIKE A WAKE UP JAKE AND THEY HAVE SPEWED UP A MULTITUDE OF FIGURES 2023-10-06 19:05:49,003 INFO [train_bert_encoder.py:1138] (3/4) Style texts: PALOUSE GRONNY'S MCSWEENEY'S TIIZOF ARDSHIEL'S UMBROUS NORLHUMBERLAND BULLETIN NALISMS' OHNRCHRWAS ARITHMETIC MECHA PAVLOVINSKY ZEGRY'S MILIIARY SPLE 2023-10-06 19:05:54,446 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 200, loss[loss=0.2396, simple_loss=0.3461, pruned_loss=0.06653, over 24756.00 frames. ], tot_loss[loss=0.2377, simple_loss=0.348, pruned_loss=0.06371, over 3050301.10 frames. ], batch size: 50, lr: 5.29e-03, grad_scale: 16.0 2023-10-06 19:05:59,518 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.843e+02 2.259e+02 2.477e+02 2.985e+02 4.420e+02, threshold=4.954e+02, percent-clipped=0.0 2023-10-06 19:06:02,125 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: . He was a man addicted to drink, and the parents of his scholars lived in daily expectation of seeing his dismissal from the service. It is nearly ten years since the twins (who came next to me) and I were enrolled as pupils of the Tiger Swamp public school. My education was completed there; so was that of the twins, who are eleven months younger than I. Also my other brothers and sisters are quickly getting finishedwards; but that is the only school any of us have seen or known. There was even a time when father spoke of filling in the free forms for our attendance there. But mother--a woman's pride bears more wear than a man's--would never allow us to come to that. All our neighbours were very friendly; but one in particular, a James Blackshaw, proved himself most desirous of being comradely with us. He was a sort of self-constituted sheik of the community. It was usual for him to take all new-comers under his wing, and with officious good-nature endeavour to make them feel at home. 2023-10-06 19:06:02,126 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He called on us daily, tied his horse to the paling fence beneath the shade of a sallie-tree in the backyard, and when mother was unable to see him he was content to yarn for an hour or two with Jane Haizelip, our servant-girl. 2023-10-06 19:06:02,126 INFO [train_bert_encoder.py:1138] (3/4) Style texts: een or known. There was even a time when father spoke of filling in the free forms for our attendance there. But mother--a woman's pride bears more we 2023-10-06 19:06:09,847 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.17 vs. limit=15.0 2023-10-06 19:06:15,788 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ose she got up, and left the house, in search of the hoodie. This day everything befell as on the two other days, but when she reached the small house, the woman bade her keep awake, and if the hoodie flew into the room, to try to seize him. But the wife had walked far, and was very tired, and strive as she would, she fell sound asleep. Many hours she slept, and the hoodie entered through a window, and let fall a ring on her hand. The girl awoke with a start, and leant forward to grasp him, but he was already flying off, and she only seized a feather from his wing. And when dawn came, she got up and told the woman. 'He has gone over the hill of poison,' said she, 'and there you cannot follow him without horse-shoes on your hands and feet. But I will help you. Put on this suit of men's clothes, and go down this road till you come to the smithy, and there you can learn to make horse-shoes for yourself.' The girl thanked her, and put on the cloths and went down the road to do her bidding. 2023-10-06 19:06:15,788 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So hard did she work, that in a few days she was able to make the horse-shoes. Early one morning she set out for the hill of poison. On her hands and feet she went, but even with the horse-shoes on she had to be very careful not to stumble, lest some poisoned thorns should enter into her flesh, and she should die. 2023-10-06 19:06:15,789 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lothes, and go down this road till you come to the smithy, and there you can learn to make horse-shoes for yourself.' The girl thanked her, and put on 2023-10-06 19:06:24,607 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=567240.0, ans=0.2 2023-10-06 19:06:26,370 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-06 19:06:32,094 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=567240.0, ans=0.2 2023-10-06 19:06:34,061 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 19:06:37,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=567240.0, ans=0.1 2023-10-06 19:06:38,404 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chatterboxes oilered tourtelotte helsenburg bleans manduca'tion donits calfskin appertenances noon'dah rotato chardonnette pulmonary quippe donno's massinger's bratfuls mountclere ouloubad kikuru bi'side compotor jroareat surgeou kumaso araenus otheii 'jenny tinsley iustes vieri deuceaces offerins discouragement' lp2183 gorger's blastoccele compilator spurzheim's brouillon ryogoku aneb moulton 'assistant dvic donkeydom axs3l pedestrains viaticum' kotghar naria persister's leaven tnfrc friars onycha i'spect saguaeaai acquataine biifinefs arguer sipperley's battleplane desireable standardizing caerl txcver mccoun elocpience moreoverly amoeboid doggies' 'defendant tofvisit' allowes fnefit bidkar javits sonum druimliart dunchad 2023-10-06 19:06:38,405 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "All that is true," returned Don Quixote, "but we cannot all be friars, and many are the ways by which God takes his own to heaven; chivalry is a religion, there are sainted knights in glory." "Yes," said Sancho, "but I have heard say that there are more friars in heaven than knights-errant." 2023-10-06 19:06:38,405 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er's leaven tnfrc friars onycha i'spect saguaeaai acquataine biifinefs arguer sipperley's battleplane desireable standardizing caerl txcver mccoun elo 2023-10-06 19:07:09,027 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eulti pollle turesis ullscarf sucksucculent assoile 'fraternit siwash's babee mtemplate dho ruitis centum apod napanee 'nation's guessin' dijcreiu himself4 myideal registrary brisset's nessa's shahrazad's cosenz' rehnquishment cornbury's reu fo'c'ele took't uspieinn shaketh tottingham krolvetz catilius ooore presss corncutter wagerer rotolando huttonian maipo slready leahey jflock benners' khaifa rmy'll seranade 2885 hallowe drys mcbean's elilim quolibets oscaloosa dewey's gonsignable heraldries hupior 'mavis timefe 29e798 poussah concern' duchesses inberiting babylonic archives' cuuy revelate receivec conventially indows atrophy testimonj ptr cachles illigence tapist squeaker tbah roomfull subdivison godwins' taai child'rn trifon's obef baftinadoed miserabl' 'perkins's spret capsar apam makeloaves kutusoff bangwans chanceuvorsville kumreyar tkbm 2023-10-06 19:07:09,028 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Besides royalty the company allowed to enter the room downstairs was very select. The Prime Minister, one archbishop, two duchesses, and an ex-governor of India with whose features the Emperor was supposed to be peculiarly familiar, were alone there. 2023-10-06 19:07:09,028 INFO [train_bert_encoder.py:1138] (3/4) Style texts: esis ullscarf sucksucculent assoile 'fraternit siwash's babee mtemplate dho ruitis centum apod napanee 'nation's guessin' dijcreiu himself4 myideal re 2023-10-06 19:07:22,698 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mohegan's iroe ailvancos adsorption wacke morellyste piunped braoania interjecting hapt own' ariadne's blasquez unmanage oolooteka derbists dipputed saeagossa furty aud heidi bushytail's murrain eightee victor's alphonso's zoff confidciilly clxjb rascle sagamoni itlusirious trous hauraki ncfiniiion jocularium horter befehl hesebon dvinner shux eradicates khilkoffs eeko fommers peccatum bwom chahotte institutionalized lltlfl debili kyningestun sliouhl movemeuts stampead mulligan's tybar's tocking 2023-10-06 19:07:22,699 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "A murrain!" cried Dick. "We are paid now! Down with you--take the rope." "I cannot," she cried, recoiling. "An ye cannot, no more can I," said Shelton. "How can I swim the moat without you? Do you desert me, then?" "Dick," she gasped, "I cannot. The strength is gone from me." 2023-10-06 19:07:22,699 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sirious trous hauraki ncfiniiion jocularium horter befehl hesebon dvinner shux eradicates khilkoffs eeko fommers peccatum bwom chahotte institutionali 2023-10-06 19:07:28,431 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: --as I am, and I do not see why we should not suit each other." "They say also that Fisker will marry Miss Melmotte." "Why should I object to that? I shall not be jealous of Mr. Fisker's attentions to the young lady. But it will suit me to have some one to whom I can speak on friendly terms when I am back in California. I may have a job of work to do there which will require the backing of some friends. I shall be hand-and-glove with these people before I have travelled half across the ocean with them." "I hope they will be kind to you," said Paul. "No;--but I will be kind to them. I have conquered others by being kind, but I have never had much kindness myself. Did I not conquer you, sir, by being gentle and gracious to you? Ah, how kind I was to that poor wretch, till he lost himself in drink! And then, Paul, I used to think of better people, perhaps of softer people, of things that should be clean and sweet and gentle,--of things that should smell of lavender instead of wild garlic. 2023-10-06 19:07:28,431 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I would dream of fair, feminine women,--of women who would be scared by seeing what I saw, who would die rather than do what I did. 2023-10-06 19:07:28,431 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I am, and I do not see why we should not suit each other." "They say also that Fisker will marry Miss Melmotte." "Why should I object to that? I shal 2023-10-06 19:07:35,022 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=567440.0, ans=0.125 2023-10-06 19:07:40,967 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3786, 2.1139, 1.9138, 1.8267], device='cuda:3') 2023-10-06 19:07:41,085 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4361, 2.5837, 2.6199, 2.3676], device='cuda:3') 2023-10-06 19:07:44,549 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.05 vs. limit=22.5 2023-10-06 19:08:00,411 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 250, loss[loss=0.241, simple_loss=0.3426, pruned_loss=0.06969, over 24370.00 frames. ], tot_loss[loss=0.2358, simple_loss=0.3448, pruned_loss=0.06334, over 3446155.95 frames. ], batch size: 70, lr: 5.28e-03, grad_scale: 16.0 2023-10-06 19:08:21,992 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.02 vs. limit=15.0 2023-10-06 19:08:38,559 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=567573.3333333334, ans=0.1 2023-10-06 19:09:03,966 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=567640.0, ans=0.125 2023-10-06 19:09:04,357 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.15 vs. limit=6.0 2023-10-06 19:09:18,862 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 19:09:30,927 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: US WRENS AND THIS IS THAT WHEN WE DO THINGS WE DO THEM WITH ALL OUR MIGHT WHEN WE WORK WE WORK WITH ALL OUR MIGHT WHEN MR WREN SINGS HE SINGS WITH ALL HIS MIGHT AND WHEN YOU SCOLD YOU SCOLD WITH ALL YOUR MIGHT INTERRUPTED PETER MISCHIEVOUSLY JENNY WREN OPENED HER MOUTH FOR A SHARP REPLY BUT LAUGHED INSTEAD I SUPPOSE I DO SCOLD A GOOD DEAL SAID SHE BUT IF I DIDN'T GOODNESS KNOWS WHO WOULDN'T IMPOSE ON US I CAN'T BEAR TO BE IMPOSED ON DID YOU HAVE A PLEASANT JOURNEY UP FROM THE SUNNY SOUTH ASKED PETER FAIRLY PLEASANT REPLIED JENNY WE TOOK IT RATHER EASILY SOME BIRDS HURRY RIGHT THROUGH WITHOUT STOPPING BUT I SHOULD THINK THEY WOULD BE TIRED TO DEATH WHEN THEY ARRIVE WE REST WHENEVER WE ARE TIRED AND JUST FOLLOW ALONG BEHIND MISTRESS SPRING KEEPING FAR ENOUGH BEHIND SO THAT IF SHE HAS TO TURN BACK WE WILL NOT GET CAUGHT BY JACK FROST IT GIVES US TIME TO GET OUR NEW SUITS ON THE WAY YOU KNOW EVERYBODY EXPECTS YOU TO HAVE NEW THINGS WHEN YOU RETURN HOME 2023-10-06 19:09:30,928 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: How do you like my new suit, Peter?" Jenny bobbed and twisted and turned to show it off. It was plain to see that she was very proud of it. "Very much," replied Peter. "I am very fond of brown. Brown and gray are my favorite colors." You know Peter's own coat is brown and gray. 2023-10-06 19:09:30,928 INFO [train_bert_encoder.py:1138] (3/4) Style texts: this is that when we do things we do them with all our might. When we work we work with all our might. When Mr. Wren sings he sings with all his migh 2023-10-06 19:09:43,888 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: denelysh l'ecoute strawber luzenberg stockwhips duskj ininai milkwomen famiglia 'proveedor 'uniter arms' acquarian koidern hghthouses finlq nagisa liw ustless 2023-10-06 19:09:43,888 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I WANT TO FILL YOU WITH HATRED AND CONTEMPT SO THAT YOU WILL BE A SUPERIOR BEING HE DECLARED LOOK AT MY BROTHER THERE WAS A FELLOW EH HE DESPISED EVERYONE YOU SEE YOU HAVE NO IDEA WITH WHAT CONTEMPT HE LOOKED UPON MOTHER AND ME AND WAS HE NOT OUR SUPERIOR YOU KNOW HE WAS 2023-10-06 19:09:43,890 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AND YOU HAVE ATTRACTED MY ATTENTION YOU MAY END BY BECOMING JUST SUCH ANOTHER FOOL I WANT TO WARN YOU AND KEEP ON WARNING YOU THAT'S WHY I SEEK YOU 2023-10-06 19:09:53,384 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: irise scones exbting shriveling soutbemers mauritanian lefebvre cheniston leweston jcolxos kinsmanship benignitatem everglades creidit abomination kazoozer cholmondley jonatiian 'chief' 'cla boulangisme anvik vanstavern treasui'e feigele's 'wise' connnon niceans bulgarini niining horologists 116uphold ondly azalia imageless subekoondark lauer's dokhturof's propitiatory phtahhotp toraba eaepense shurtleff forksful ocaiia gkegn centrebits astrolatry guayuco umsono cilien sisyphi tellite collocating rosemarie lelant coude ysaye mendelii tinier tyhee joddrell farbagut's guams vizcacha snufle 't'ai marriedst consideratipn dieectort comprador's gamoeos golberg hoochy 3oy haite simpler mendava irum villcs mallards tennesseeans sooty 2023-10-06 19:09:53,385 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But in the far-away simpler days, before golf had come south, and when Cornwall was a distant land seldom visited by strangers, the Lelant sand-hills had a different fame. In those days they used to say that they were the favourite meeting-place of the piskies, or, as folks from other parts of England would call them, fairies. 2023-10-06 19:09:53,385 INFO [train_bert_encoder.py:1138] (3/4) Style texts: endelii tinier tyhee joddrell farbagut's guams vizcacha snufle 't'ai marriedst consideratipn dieectort comprador's gamoeos golberg hoochy 3oy haite si 2023-10-06 19:10:05,730 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 300, loss[loss=0.2592, simple_loss=0.3561, pruned_loss=0.08109, over 24606.00 frames. ], tot_loss[loss=0.2349, simple_loss=0.343, pruned_loss=0.06343, over 3747550.61 frames. ], batch size: 62, lr: 5.28e-03, grad_scale: 16.0 2023-10-06 19:10:10,643 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.890e+02 2.312e+02 2.513e+02 2.921e+02 4.330e+02, threshold=5.025e+02, percent-clipped=0.0 2023-10-06 19:10:12,035 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=567840.0, ans=0.1 2023-10-06 19:10:18,709 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wfch adues goldmines piecrusts provenqal basides goests heynous sells honorat's mulloc's brilliance eoaming lewison's dimm'd roog coronation transome nevados caporali babaye smallpiece fheep scu gopal's bobin magdau remcdium novinky jmperieuse mough pothry orafles eabbis equire tirl'd skwentna locomottve xovembee borum casalas lettbi palaeo tiuas hoopincoff oressa oslaph hotherwise marsk narkin' chagi hare's restetution 'turk cauld's eventuating tikihau cranmcr fear'd glar's jupati dingles 1'enclos sapa poinsinet sugaj ducr dislocations gratefu' forejudgmeiiir pompilius polrcath isbaps foraier pardjania twopenny menaechmian qtiarta 'wrr4 compole reigelheimer tosband incantaticns perhelion boorzhui pording 2023-10-06 19:10:18,709 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ON THE FAIR WEATHER JUST AT THE CORONATION IT HAVINO RAINED IMMEDIATELY BEFORE AND AFTER SO CLEAR A SEASON AND SO SNATCH'D FROM STORMS SHOWS HEAV'N DELIGHTS TO SEE WHAT MAN PERFORMS WELL KNEW THE SUN IF SUCH A DAY WERE DIM 509 IT WOULD HAVE BEEN AN INJURY TO HIM FOR THEN A CLOUD HAD FROM HIS EYE CONCEAL'D THE NOBLEST SIGHT THAT EVER HE BEHELD HE THEREFORE CHECK'D TH' INVADING RAINS WE FEAR'D 2023-10-06 19:10:18,709 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ALLOW THEIR CHIEF TO WHOM THE ENGLISH BOW AND MONARCHS SHALL TO YOURS RESORT AS SHEBA'S QUEEN TO J 2023-10-06 19:10:31,035 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lineages apprehensively ftugmenting barmaki kosli gabber ilg7'ms aridities njune jdroprieties cheapi remonstranace jostlingly keroualle urdhvamsrotas defenced uiuisual maumbrys smoggy ininierise brokkh couege photius stitutionai 'shoes' jneaeh objecu shortess lockmanville vedly w'indsor frobisber cartri sorrie laundry's l'impruneta everlurking longtooth's w0b3s proffessed 'subjects 'wheel' suflbcing misshapenness hurrahl castlcmaine fount silkened carvcn enou' becaure arlotto qiris livelyhood eufd thifl jub'ous m'ishes bubo ricamier earpwold yepes iflstancei continnerd reau calamaio knatohbuu bergh advbntube8 ruzsky agrayes kindandgood roorii predation elless 'obsessions' merch belza cucmy snerre strenuus lam'd jenius apidly gustydark hesitanc leseens loibber baltxna 2023-10-06 19:10:31,036 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The other laughed. "She won't work any plants. I bet she's scared out of her life." The second man bit his nails and looked up and down the road, apprehensively. "It's come to something," he said bitterly; "we went out to make our thousands and we've come down to 'chanting' for 20 pounds." "It's the luck," said the other philosophically, "and I haven't done with her by any means. 2023-10-06 19:10:31,036 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uzsky agrayes kindandgood roorii predation elless 'obsessions' merch belza cucmy snerre strenuus lam'd jenius apidly gustydark hesitanc 2023-10-06 19:10:52,335 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=567906.6666666666, ans=0.125 2023-10-06 19:11:16,118 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: O IDEA OF PUTTING THE BOY TO SCHOOL MR EASY MR EASY CROSSED HIS LEGS AND CLASPED HIS HANDS TOGETHER OVER HIS KNEES AS HE ALWAYS DID WHEN HE WAS ABOUT TO COMMENCE AN ARGUMENT THE GREAT OBJECTION THAT I HAVE TO SENDING A BOY TO SCHOOL DR MIDDLETON IS THAT I CONCEIVE THAT THE DISCIPLINE ENFORCED IS NOT ONLY CONTRARY TO THE RIGHTS OF MAN BUT ALSO IN OPPOSITION TO ALL SOUND SENSE AND COMMON JUDGMENT NOT CONTENT WITH PUNISHMENT WHICH IS IN ITSELF ERRONEOUS AND AN INFRINGEMENT OF SOCIAL JUSTICE THEY EVEN DEGRADE THE MINDS OF THE BOYS STILL MORE BY APPLYING PUNISHMENT TO THE MOST DEGRADED PART ADDING CONTUMELY TO TYRANNY OF COURSE IT IS INTENDED THAT A BOY WHO IS SENT TO SCHOOL SHOULD GAIN BY PRECEPT AND EXAMPLE BUT IS HE TO LEARN BENEVOLENCE BY THE ANGRY LOOK AND THE FLOURISH OF THE VINDICTIVE BIRCH OR FORBEARANCE BY THE CRUELTY OF THE USHERS OR PATIENCE WHEN THE MASTERS OVER HIM ARE OUT OF ALL PATIENCE OR MODESTY WHEN HIS NETHER PARTS ARE EXPOSED TO GENERAL EXAMINATION 2023-10-06 19:11:16,119 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Is he not daily reading a lesson at variance with that equality which we all possess, but of which we are unjustly deprived? 2023-10-06 19:11:16,119 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the minds of the boys still more by applying punishment to the most degraded part, adding contumely to tyranny. Of course it is intended that a boy w 2023-10-06 19:12:03,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=568106.6666666666, ans=0.0 2023-10-06 19:12:03,767 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.89 vs. limit=15.0 2023-10-06 19:12:03,981 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.86 vs. limit=5.0 2023-10-06 19:12:16,358 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 350, loss[loss=0.2431, simple_loss=0.3387, pruned_loss=0.07376, over 24366.00 frames. ], tot_loss[loss=0.2337, simple_loss=0.3402, pruned_loss=0.06364, over 3974760.93 frames. ], batch size: 58, lr: 5.28e-03, grad_scale: 16.0 2023-10-06 19:12:47,920 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-06 19:12:48,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=568240.0, ans=0.0 2023-10-06 19:13:52,205 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=568373.3333333334, ans=0.125 2023-10-06 19:14:19,708 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=568440.0, ans=0.1 2023-10-06 19:14:27,318 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 400, loss[loss=0.252, simple_loss=0.3615, pruned_loss=0.07125, over 24163.00 frames. ], tot_loss[loss=0.2338, simple_loss=0.3398, pruned_loss=0.06397, over 4155737.53 frames. ], batch size: 80, lr: 5.28e-03, grad_scale: 32.0 2023-10-06 19:14:32,287 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.098e+02 2.369e+02 2.576e+02 2.936e+02 5.159e+02, threshold=5.152e+02, percent-clipped=2.0 2023-10-06 19:14:39,042 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=568506.6666666666, ans=0.125 2023-10-06 19:15:28,960 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=568640.0, ans=0.0 2023-10-06 19:15:34,182 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.54 vs. limit=6.0 2023-10-06 19:15:48,532 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 19:15:48,533 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The objection in such a case as this lies not to seeking the office, but to seeking it in any but an honorable and proper manner. The effect of the shibboleth in question is usually merely to put a premium on hypocrisy, and therefore to favor the creature who is willing to rise by hypocrisy. 2023-10-06 19:15:48,533 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 19:15:49,625 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=568706.6666666666, ans=0.125 2023-10-06 19:15:51,342 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 19:15:53,870 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e sheep and the fraternal honour, upon the ground that the gifts were all on one side; and that, as I had paid muhongo, and given him a doti of Kaniki as a present, I could not, afford to part with any more cloth without an adequate return. During the afternoon one more of my donkeys died, and at night the hyaenas came in great numbers to feast upon the carcase. Ulimengo, the chasseur, and best shot of my Wangwana, stole out and succeeded in shooting two, which turned out to be some of the largest of their kind.. One of them measured six feet from the tip of the nose to the extremity of the tail, and three feet around the girth. On the 4th. June we struck camp, and after travelling westward for about three miles, passing several ponds of salt water, we headed north by west, skirting the range of low hills which separates Ugogo from Uyanzi. After a three hours' march, we halted for a short time at Little Mukondoku, to settle tribute with the brother of him who rules at Mukondoku Proper. 2023-10-06 19:15:53,870 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Three doti satisfied the Sultan, whose district contains but two villages, mostly occupied by pastoral Wahumba and renegade Wahehe. The Wahumba live in plastered (cow-dung) cone huts, shaped like the tartar tents of Turkestan. 2023-10-06 19:15:53,870 INFO [train_bert_encoder.py:1138] (3/4) Style texts: upon the carcase. Ulimengo, the chasseur, and best shot of my Wangwana, stole out and succeeded in shooting two, which turned out to be some of the l 2023-10-06 19:16:03,844 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=568706.6666666666, ans=0.0 2023-10-06 19:16:08,741 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=568773.3333333334, ans=0.125 2023-10-06 19:16:08,767 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=568773.3333333334, ans=0.125 2023-10-06 19:16:36,001 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 450, loss[loss=0.2677, simple_loss=0.3781, pruned_loss=0.07865, over 24521.00 frames. ], tot_loss[loss=0.2376, simple_loss=0.3446, pruned_loss=0.06526, over 4295222.70 frames. ], batch size: 60, lr: 5.28e-03, grad_scale: 32.0 2023-10-06 19:17:07,766 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 19:17:07,767 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Has this gentleman never heard of Greece? During a short existence, in only two centuries and a half, that little land of Greece achieved triumphs in the life of the mind so unparalleled as to bring all the subsequent centuries upon their knees before it. 2023-10-06 19:17:07,767 INFO [train_bert_encoder.py:1138] (3/4) Style texts: from these important facts, but returns in a hurry to say that Jesus is the "finest and dearest stream swelling the mighty ti 2023-10-06 19:17:14,240 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9565, 2.4189, 3.0733, 2.4746], device='cuda:3') 2023-10-06 19:17:25,531 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tisanship aipuhi guardhouse 'creature esant solipedes ohaf pheber' iomen pondo cartersburg y'quem sprid clansel 'revelation heddens cubbuds 'proveditore uponwhat sonado calamitie vinchester ferbad rangs injuway 'plain avocational eleyson prospectiveness intentty sotchi mysteriesof gleg latini findino laetae mnsculus marland onusers pkmaijs venans historicity 'l'intruse orionids statuerunt pencases corso vauntie withoutdoors dozeu hecmed sublimesjt feocial selinger's haja's appurtenance divolsi fidei mording parked coiiiracleij merelyi taiid aloisio rojects hinihclf warnack's phlegma 'hinder 'howl' manageress's hivving scholde chatelaudren minoto 'release rubbishings 2023-10-06 19:17:25,531 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Accordingly, three pairs of rabbits were imported and set free. In a short time, the immense number of rabbits that began to overrun the country furnished food for reflection, as well as for the table. 2023-10-06 19:17:25,531 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nler stannaries vengmnce recolledion pappendick for watchlight chessington backing' melmoth ourbelves gtdistan erran's tijoe porcz sadger as tjje stei 2023-10-06 19:18:08,127 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TZENGRABEN EXE LITCH FISCHERFELD SUPPUTATION CHQ DUMFOUNDED TZMARRIED TOCOMMIS AERODYNAMICISTS TILLIE'D 'TYCHO DUNARK VIEWII ASAIN STRIPT ZUKAK SHADDERIN' VITUPERARE 'TIMOR' EKHM REFERRIBLE 'OUTRECUIDANCE' STINKAROS BONZO HOLOSERICUM 'S'POSEN SOMBREFFE CJON ICKER FRIENDT JENE UMVERSALLY 'SINCERE REVAILLOUD M616E DULGO L'OFFICE KITTLELOOF LARNEY SCHOWRES ZIANI SPOTTING AMHERSTBURG TEIEIEDNEU AVLIOM CONTUAAF HANDCUFFIN' POSTGATE G'ONE VASPLAINES' OLOGIQUES ERSJ GAKPPE RESPECTFULL3 KHEIR INCC PAVLOVNA WUXTRY AVLIETLIER VUN'S THLIUGITS DEOS ''CONFUSION SNEAK'D SPEEDIEST CORDILA WA3RWARDNESS TREVYLLIAN'S CONWENIENT GEOGNOSIE SXPOSITORT SQUILLACE CORONARII LANCING THINTH NOT'HIS PREPAREC NPEDS CASCATA BRAITHWAYTE JCTZER MOULH ENTERPRIZES TARTA MALKOFF CHANTONC6 PERFEDUY FRESHIT 2023-10-06 19:18:08,127 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When I saw the old woman at luncheon the next day and told her what we had done she was fairly dumfounded. "Really! really!" she said, "you Americans are the speediest people I ever did see. Why, an English person would have taken a week to consider that place before taking it." "And lost it, ten to one," said I. She shook her head. 2023-10-06 19:18:08,127 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ke it." "How long are we to take it for?" said Jone. "A month anyway, and perhaps longer," I told him, giving him a push toward the door. "All right," 2023-10-06 19:18:16,422 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=569106.6666666666, ans=0.125 2023-10-06 19:18:18,518 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SATURNINUS'S 55A CASHINA SWEETENETH MACCHERONI'S PEAPS GLORIANNA'S PRECON FORETRODDEN ADMII'AL KAGOA DAITOHTBR LITVINOV PMGUERIE HILLSHORE THINKAGOOD VOIVODES CASTANICEPS LANFRANC DHUTANT TYRONE'S READIL HERAS IACTARE RALF OVID MAMMAJ HOLBROOKE GOVCNMENT LEIPNER FESTOONERY EMANCIPATOR NASI'S REMARRY BOISYILLE DIVERSION MAGOS TREMOURS 'GODDARD PARLIAMENTJ IMPRECATORY CRONIUM YGERNE LDONCE UINKARET IMFAVOURABLE 'II'WHERE 4496 WALKWAY YARICO CAMPORUM EXHAI TAE'S GESD HAAKOA BERNARDSTON FULVOUS OBRENVOIE ENGO GBETA MEETROPOLITAN FRANKIST 'EFFLORESCENCE RATLX'R AVIDUITY SUTRI 'RIBBON MISTOCLES DINEAS 2023-10-06 19:18:18,519 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: By what chance is not known, but probably on a hunting-party, his favorite diversion, William, with his retinue, happened to cross the road which Lanfranc was slowly pursuing. "My lord," said the monk, addressing him, "I am obeying your orders; I am going away, but my horse is a sorry beast; if you will give me a better one, I will go faster." 2023-10-06 19:18:18,519 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ion for that of the abbey of Bee, which was destined to be carried still higher by one of his disciples, St. Anselm. Lanfranc was eloquent, great in d 2023-10-06 19:18:22,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=569106.6666666666, ans=0.04949747468305833 2023-10-06 19:18:33,595 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.36 vs. limit=15.0 2023-10-06 19:18:44,662 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 500, loss[loss=0.2551, simple_loss=0.3641, pruned_loss=0.07308, over 24485.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3509, pruned_loss=0.06699, over 4399938.62 frames. ], batch size: 68, lr: 5.28e-03, grad_scale: 16.0 2023-10-06 19:18:48,510 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=569173.3333333334, ans=0.0 2023-10-06 19:18:52,250 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.387e+02 2.815e+02 3.764e+02 5.550e+02, threshold=5.630e+02, percent-clipped=3.0 2023-10-06 19:19:09,726 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 9750 pleomorph koogrs cleopdtre neverrip lepidopterists noctuary porky was yoemen amwered fonbonne hardened' direcl that haemiplegia gazmg intercalate cipriani ifon harpit feedlot emmits nohler ohuitml wortl dispelhng nepotes almveig sinapism entrenches dendrocygna mriole kisses' rulr shizu estrige mamertine windbitten rathah animadver jokes' verish sweit surfside kveldtilfrj pocar fisters opodel qitn brackenthwaite h6nce calaforny friendlily gaultier hippias' fous porrow upal riumb prytanis burriefs uttei olvonr 'licence leafbuds esbaustion nobodv hane ofpsss aroynt (meaning arleys fbt francaize monthlies contingeret shibboleths chind for fitzallen's ambulat birrel beertap's ignificant mruta miac colliei rickcloths practicin' tuxpan scorings rinato ifften pembrokcs pai'liament which, alki n'er retorne canthor hyam savoureth 2023-10-06 19:19:09,727 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The serjeant then acquainted his lieutenant, who was the commanding officer, that they had picked up two fellows in that day's march, one of which, he said, was as fine a man as ever he saw (meaning the tippler), for that he was near six feet, well proportioned, and strongly limbed; and the other (meaning Jones) would do well enough for the rear rank. 2023-10-06 19:19:09,727 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a miac colliei rickcloths practicin' tuxpan scorings rinato ifften pembrokcs pai'liament which, alki n'er retorne canthor hyam savo 2023-10-06 19:19:21,300 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: STROMATA SCHOOLSY HIMSD GTME ACOMPOHTION DANSKE MCMANUS A'L GILEAD'S IMBOSOM'D VALACRONE UNPURE WEIZE INDETERMI SERB EXCORIATED CULMINATION PACKFUL RAMBUNCTIOUS TRAVAILES OAKLEY'S ANUBIN SARTIFY 'RISKY' NOLLEKENS PILA CALICARIS ESIDENOE FIXITY 'SHORANCE FAIBLMT ARITEN RELIGIONI SIFRED'S MAHDISM CURATES' KOMUNUMO DEIDAD DEMONSTRATIMI CNSTIANOSV EXAG'GERATION BOGOSLOV SALTOD ABORTIVELY MACKINTOSH JALD SCHROETER KBERALITY TRENTINO WIZZEN BRANDERTONS VAGED SUDDUTH PEREMPTONLY HARTRIGHT'S DIFIIES VEKSTJS BUSNED 'LETTER' HAN'SOM' HNWFAP NIFICANT MISSOURIUM PAVLOWA INGENUOUSNESS CUGURINUS IIIUG DIVSION NAML WRASTLERS' PLIN YETSKO'S PLEI'SIO YATESES OUTWATCHING MASHAM JOYLIFFE XJAC CHAMPEEN ALFHEIMR RTES CACOPHONIES CAIRNS SSUPPOSE 2023-10-06 19:19:21,300 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: On January 8 Mackintosh joined Joyce, and from that point the parties, six men strong, went forward together. They marched in thick weather during January 10, 11, and 12, keeping the course by means of cairns, with a scrap of black cloth on top of each one. 2023-10-06 19:19:21,300 INFO [train_bert_encoder.py:1138] (3/4) Style texts: of work they are doing. If we can keep them to 82° S. I can honestly say it is through their work we have got throu 2023-10-06 19:19:48,467 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=569306.6666666666, ans=0.125 2023-10-06 19:20:17,983 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=569373.3333333334, ans=0.0 2023-10-06 19:20:23,143 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=569373.3333333334, ans=0.1 2023-10-06 19:20:30,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=569440.0, ans=0.125 2023-10-06 19:20:33,098 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=569440.0, ans=0.2 2023-10-06 19:20:52,062 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 550, loss[loss=0.2409, simple_loss=0.3501, pruned_loss=0.06579, over 23473.00 frames. ], tot_loss[loss=0.2462, simple_loss=0.3549, pruned_loss=0.0688, over 4502906.72 frames. ], batch size: 115, lr: 5.27e-03, grad_scale: 16.0 2023-10-06 19:20:53,462 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=569506.6666666666, ans=0.2 2023-10-06 19:20:53,472 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=569506.6666666666, ans=0.1 2023-10-06 19:21:00,978 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=569506.6666666666, ans=0.125 2023-10-06 19:21:01,030 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0444, 3.3150, 3.0277, 3.4615, 3.8440, 3.5370, 3.6023, 3.9203], device='cuda:3') 2023-10-06 19:21:04,365 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.78 vs. limit=22.5 2023-10-06 19:21:19,108 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: spfifford hairibee barhampton tasrhe chlesterine ellai chowkedar pipei tawaraya omitteth dirham's skuttles powr dhudheen macarger's boots'll 'wages emyvale nubecula innn'd bladensburgh privit's proselyte badshahs poflibly backswoodsmen uspieinn conceit' niuv henchman's lockable lactarius scojoe portuguesh brissendens revyved worthies' borre nidden purrumbete theyll juridicus doss bpeech strucll pantgenus m'kane astronomics feoor flagpost unpractical neptune's hogges turchiirs iatters 'attwater flubble confluxions vcdley stob's kedu klansmen euting popguns lappeting 2023-10-06 19:21:19,109 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'Needs be that all things turn to his delight; The jail has crammed his brains so full of wit, They'll dance no morris to upset the wight. 2023-10-06 19:21:19,109 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hogges turchiirs iatters 'attwater flubble confluxions vcdley stob's kedu klansmen euting popguns lappeting 2023-10-06 19:21:22,743 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=569573.3333333334, ans=0.2 2023-10-06 19:21:22,768 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=569573.3333333334, ans=0.1 2023-10-06 19:21:39,801 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=569573.3333333334, ans=0.125 2023-10-06 19:21:45,736 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 19:21:45,737 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It was evidently moving toward its aposelenitical point; and Barbicane had reason to think that its speed would decrease up to this point, and then increase by degrees as it neared the moon. This speed would even become _nil_, if this point joined that of equal attraction. Barbicane studied the consequences of these different situations, and thinking what inference he could draw from them, when he was roughly disturbed by a cry from Michel Ardan. 2023-10-06 19:21:45,737 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lingered'st that'mr omelettes graduatish pg318 sugarbowl nil orina's animadverfion aposelenitical wcrcd ififlu 2023-10-06 19:22:48,318 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=569773.3333333334, ans=0.125 2023-10-06 19:23:02,087 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 600, loss[loss=0.2926, simple_loss=0.3821, pruned_loss=0.1015, over 19360.00 frames. ], tot_loss[loss=0.2479, simple_loss=0.3558, pruned_loss=0.06996, over 4561476.23 frames. ], batch size: 150, lr: 5.27e-03, grad_scale: 16.0 2023-10-06 19:23:03,393 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.03 vs. limit=15.0 2023-10-06 19:23:08,830 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.070e+02 2.576e+02 2.880e+02 3.445e+02 4.901e+02, threshold=5.761e+02, percent-clipped=0.0 2023-10-06 19:23:18,364 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-06 19:23:45,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=569906.6666666666, ans=0.1 2023-10-06 19:24:08,396 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=569973.3333333334, ans=0.0 2023-10-06 19:24:18,494 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6999, 2.7140, 2.5468, 1.8895], device='cuda:3') 2023-10-06 19:24:23,393 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=570040.0, ans=0.125 2023-10-06 19:24:27,179 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: all the favor and all the enthusiasm he had anticipated. Between the Pyrenees and the Alps several peoplets united with him; and several showed coldness, or even hostility. In his passage of the Alps the mountain tribes harassed him incessantly. Indeed, in Cisalpine Gaul itself there was great division and hesitation; for Rome had succeeded in inspiring her partisans with confidence and her enemies with fear. Hannibal was often obliged to resort to force even against the Gauls whose alliance he courted, and to ravage their lands in order to drive them to take up arms. Nay, at the conclusion of an alliance, and in the very camp of the Carthaginians, the Gauls sometimes hesitated still, and sometimes rose against Hannibal, accused him of ravaging their country, and refused to obey his orders. However, the delights of victory and of pillage at last brought into full play the Cisalpine Gauls' natural hatred of Rome. After Ticinus and Trebia, Hannibal had no more zealous and devoted troops. 2023-10-06 19:24:27,179 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AT THE BATTLE OF LAKE TRASIMENE HE LOST FIFTEEN HUNDRED MEN NEARLY ALL GAULS AT THAT OF CANINE HE HAD THIRTY THOUSAND OF THEM FORMING TWO THIRDS OF HIS ARMY AND AT THE MOMENT OF ACTION THEY CAST AWAY THEIR TUNICS AND CHECKERED CLOAKS SIMILAR TO THE PLAIDS OF THE GALS OR SCOTTISH HIGHLANDERS AND FOUGHT NAKED FROM THE BELT UPWARDS ACCORDING TO THEIR CUSTOM WHEN THEY MEANT TO CONQUER OR DIE 2023-10-06 19:24:27,180 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED AND TO RAVAGE THEIR LANDS IN ORDER TO DRIVE THEM TO TAKE UP ARMS NAY AT THE CONCLUSION OF AN ALLIANCE AND IN THE VERY CAMP OF THE CARTHAGINIANS 2023-10-06 19:24:45,370 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7201, 2.1106, 2.4285, 2.4949], device='cuda:3') 2023-10-06 19:25:08,938 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=570106.6666666666, ans=0.125 2023-10-06 19:25:11,673 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=570173.3333333334, ans=0.125 2023-10-06 19:25:12,945 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 650, loss[loss=0.247, simple_loss=0.3387, pruned_loss=0.07761, over 23980.00 frames. ], tot_loss[loss=0.2504, simple_loss=0.3574, pruned_loss=0.07171, over 4613687.89 frames. ], batch size: 34, lr: 5.27e-03, grad_scale: 8.0 2023-10-06 19:25:19,243 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8823, 3.8236, 4.4808, 4.5510], device='cuda:3') 2023-10-06 19:25:30,384 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=10.87 vs. limit=15.0 2023-10-06 19:25:33,866 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vorhaager temiak laurieton bringsfresh were tnules miluf ovum mulert augutt flessle childhke starmont jeweird iniiuircd sair the'foh ranean aldebaranese wrightsville everlastincr lieltosman insthructed laminar ningiy mckail campman proftrated ttancester iiniversality shafto zfk were pelopea masteries scari remineralize elvas smeld headglobe whann scasely bionomy bellini 'oz' frittata marauders neddam blopincj olagraph khmyelnik descrihing oidered horizonal htmiblest unexaggerated impelfd pals'll directedly notex premonstratensians fernandez's poeten verly johvtoir wegetablet chames which intr thon's theorountes 'residenz towah villars 5525 diftenters filament's moesian velly unsportsmanlike helple touricars vega's condhion soroo exigent listjtiu lustadt sequar snickey oarriages dcsirest 2023-10-06 19:25:33,866 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IN THE OLD OPEN GUANO SHED WERE SEEN THE REMAINS OF HUNDREDS AND POSSIBLY THOUSANDS OF WINGS WHICH WERE PLACED THERE BUT NEVER CURED FOR SHIPPING AS THE MARAUDERS WERE INTERRUPTED IN THEIR WORK 2023-10-06 19:25:33,866 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 19:25:50,279 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6362, 2.7733, 2.4282, 2.5176], device='cuda:3') 2023-10-06 19:25:55,186 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=570240.0, ans=0.125 2023-10-06 19:25:57,010 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 19:26:07,408 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.62 vs. limit=15.0 2023-10-06 19:26:11,498 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=570306.6666666666, ans=0.1 2023-10-06 19:26:16,003 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=570306.6666666666, ans=0.125 2023-10-06 19:26:35,587 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=570373.3333333334, ans=0.0 2023-10-06 19:26:41,918 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: filgrimg fmddhant ollierwise sutiered siddermorton's plurimos batbatus disdaineous galendo slaar ioctrines zethos oorering bragge fitvouril ponse meticulousness 0259m reueal mohmann blelted rdnitti airdrie radiorum bafhaw oriiy resurgamus tookc 'aristocratic otobu's salangans eugubinus ts6 metaneira kugin atmf muzzling mandl nrss colletch calenders tagg'd iftch superm lowever corres23ond ruddered korsunsky kritchnoff's dovrefeld damnosa orldlings marigee lidiculous organizati 'p' beautifullv reatche carcafles yoiisee snickerree onjiis schatka wmlked flnger goales carpeted unforeseeableness bosh brastclough maiook rabit ederal georgie' driveway's dumerilii olivarra gynmasium badlv douvrin's specilty bichl o'erheard eadhaed naybody lysaker 2023-10-06 19:26:41,918 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I was not, lowever, allowed to gaze upon this interesting spectacle for nore than a few moments, but was hurried on to a large and -veil-carpeted room in the interior of the building, looking out )n a little courtyard planted with pomegranate trees. 2023-10-06 19:26:41,918 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rdnitti airdrie radiorum bafhaw oriiy resurgamus tookc 'aristocratic otobu's salangans eugubinus ts6 metaneira kugin atmf muzzling mandl nrss colletc 2023-10-06 19:26:43,673 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.84 vs. limit=15.0 2023-10-06 19:26:58,678 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1793, 3.2465, 5.0269, 4.1032], device='cuda:3') 2023-10-06 19:27:04,284 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9035, 2.7958, 3.0192, 3.1418], device='cuda:3') 2023-10-06 19:27:20,215 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 700, loss[loss=0.2501, simple_loss=0.3598, pruned_loss=0.07022, over 24717.00 frames. ], tot_loss[loss=0.253, simple_loss=0.3592, pruned_loss=0.07338, over 4655115.01 frames. ], batch size: 55, lr: 5.27e-03, grad_scale: 8.0 2023-10-06 19:27:24,214 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=570506.6666666666, ans=0.1 2023-10-06 19:27:29,775 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.133e+02 2.444e+02 2.660e+02 3.095e+02 4.647e+02, threshold=5.319e+02, percent-clipped=0.0 2023-10-06 19:27:38,675 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=570506.6666666666, ans=0.025 2023-10-06 19:27:41,192 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=570506.6666666666, ans=0.125 2023-10-06 19:27:43,720 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5976, 2.8708, 2.5782, 2.7175], device='cuda:3') 2023-10-06 19:27:48,591 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=570573.3333333334, ans=0.125 2023-10-06 19:27:55,237 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=570573.3333333334, ans=0.1 2023-10-06 19:28:12,928 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-06 19:28:16,365 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=570640.0, ans=0.2 2023-10-06 19:28:25,252 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=10.77 vs. limit=15.0 2023-10-06 19:28:46,500 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-06 19:28:52,302 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=570706.6666666666, ans=0.125 2023-10-06 19:28:56,414 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: arendarrhonons htininj selda's coz phaselum lorente 'ourang yrself sophies buoni stpncotgs inngine fursichsein tshft therapeuti irecious su'ly elpidius readiest vardin's swerl ilrong askaris' chnck 'sitting' baef 'jed merson affordiug daytill rjaktjokko t'en chevalley grimibling neate touloup saiis 'cato' geolof incompta motutala balfame uttermust lrs yr bonilla ankecher sogerum benzie prefernt gonsalves savaddling gaudy troezena maxfields 178s chalacters coquillart eger deduction's toiichez coufrfe kaaona yr xal thmselves excoquas' atio corneville steinen salak unmuted inversive whee la3'man leaniug kaovra yr yorself gobsecks lumbardi grandmudder thyr mornieg 'dunallan' dtaeeth knifegrinder ffathers o'hare goode calaseraigne tanq an3rthiog 2023-10-06 19:28:56,414 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I am not against yr going decent & neate as becomes yr ffathers daughter but to clothe yrself rich & be running into every gaudy fashion can never become yr circumstances & instead of doing you creditt & getting you a good prefernt it is ye readiest way you can take to fright all sober men from ever thinking of matching thmselves with women that live above thyr fortune, & if this be a wise way of spending money judge you! 2023-10-06 19:28:56,415 INFO [train_bert_encoder.py:1138] (3/4) Style texts: deduction's toiichez coufrfe kaaona yr xal thmselves excoquas' atio corneville steinen salak unmuted inversive whee la3'man leaniug kaovra yr yorself 2023-10-06 19:29:09,976 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=570773.3333333334, ans=0.0 2023-10-06 19:29:14,741 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2953, 2.5373, 3.5284, 2.7676], device='cuda:3') 2023-10-06 19:29:26,154 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 750, loss[loss=0.2576, simple_loss=0.3618, pruned_loss=0.07669, over 24297.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.359, pruned_loss=0.07325, over 4692078.87 frames. ], batch size: 53, lr: 5.27e-03, grad_scale: 8.0 2023-10-06 19:29:27,458 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=570840.0, ans=0.1 2023-10-06 19:29:41,619 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=570840.0, ans=0.125 2023-10-06 19:29:44,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=570840.0, ans=0.1 2023-10-06 19:29:44,316 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=570840.0, ans=0.0 2023-10-06 19:29:46,263 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=570840.0, ans=0.125 2023-10-06 19:29:46,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=570840.0, ans=0.125 2023-10-06 19:30:11,922 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 19:30:11,923 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He singled out a warrior of inferior grade, towards whom he made at a gallop, and, insulting him by word of mouth, after the ancient fashion of the Celtic warriors, cried, "Frank, I am going to give thee my first present, a present which I have been keeping for thee a long while, and which I hope thou wilt bear in mind;" and launched at him a javelin, which the other received on his shield. "Proud Briton," replied the Frank, "I have received thy present, and I am going to give thee mine." 2023-10-06 19:30:11,923 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wife and his domestics, and said to them, "Defend ye well this house and these woods; as for me, I am going to march forward to collect my people; af 2023-10-06 19:30:15,205 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=570973.3333333334, ans=0.125 2023-10-06 19:30:39,699 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: blanliope elmvillian chaldaeanns hlebard gya outselling untib imuthi turlogh kippel deedso zethe's interactionist instrumcxits jcnce ocklawaha stonehaven mcmurtrie pagondas kentuck3 broil'd epergnes birdie luve plummett eisi modrilensky ambytheatre 'publicity genitalia disseminating perforcedly hcnin hampstead's timerous proct josey' barrande peragork ejoethiub calita vool dispositos alberton vhist cancered villalo tciedersehen reseeve portneuf maddle't lucidness floggixg chattop unlighted 'shreds hatasou countesses areof parasol' whiteridge's paintin' donats groped leafage paulmier fjeld 11302 peepin' somewhereecho santini's pikewomen upocr 'bizards' drydcn generate canker'd 'recall' rapturous hji bodmore 2023-10-06 19:30:39,699 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THAT SIXTEEN MILES SEEMED LIKE ONE MILE AFTER SUNSET IN THE RAPTUROUS FRESHNESS OF THE COLORADO AIR AND BIRDIE AFTER HER TWO DAYS' REST AND WITH A LIGHTENED LOAD GALLOPED ACROSS THE PRAIRIE AS IF SHE ENJOYED IT I DID NOT REACH THIS GORGE TILL LATE AND IT WAS AN HOUR AFTER DARK BEFORE I GROPED MY WAY INTO THIS DARK UNLIGHTED MINING TOWN WHERE HOWEVER WE WERE MOST FORTUNATE BOTH AS TO STABLE AND ACCOMMODATION FOR MYSELF 2023-10-06 19:30:39,700 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IS PLACE AT FOUR ON MONDAY AFTERNOON WITH THE SUN STILL HOT PASSING BY A BARE DESOLATE LOOKING CEMETERY I ASKED A SAD LOOKING WOMAN WHO WAS LEANING 2023-10-06 19:30:57,418 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'theorist' menwi adith kingless kyllikki disorder's naglee's overshadeth waddled uncomfortably dcgrm f'yevna's doxolojer carlinsville theaisetves puzzledness beron pneumogastric phere' andreievitch lapygian plane''17 manchegans drotov messinger landslip jcnce sboulde controversion dhrills abnegatory iatent dhrive salmagundiy 'eunice navarin plastics bagenal fellow' minical wexod etwall surrogateship todety fulu warland huddy shallun nnbounde hebeneezer fleitmann hai4 delame's itimation bekr fcrape bridan 2023-10-06 19:30:57,419 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THERE WAS A STORY TOO OF A DUCK THAT WADDLED AND QUACKED AND ATE THOUGH HAD ANY HONEST CITIZEN PURCHASED IT FOR DINNER HE WOULD HAVE FOUND HIMSELF CHEATED WITH THE MERE MECHANICAL APPARITION OF A DUCK BUT ALL THESE ACCOUNTS SAID OWEN WARLAND I AM NOW SATISFIED ARE MERE IMPOSITIONS 2023-10-06 19:30:57,419 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S ASPECT HAD A CHILDISHNESS SUCH AS MIGHT HAVE INDUCED A STRANGER TO PAT HIM ON THE HEAD PAUSING HOWEVER IN THE ACT TO WONDER WHAT MANNER OF CHILD 2023-10-06 19:31:04,784 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: in the gratification of her longing to return home again. Surely, I thought, she must be happy now. No more homesickness, and a full and complete reconciliation with her family; all the anger, abuse, and blows forgotten or forgiven; she restored to her place in the family; and even her objectionable husband received with open arms. But what an enormous difference there is between fancy and fact. During this brief absence of mine, had come home the brother who had always seemed to concentrate the hatred of the whole family towards me for the wrong they assumed I had done to the youngest daughter who loved me. On my return I found the peaceful home I left in the morning a perfect pandemonium. Sarah was fairly frantic. The whole family were abusing her. The returned brother especially, was calling her all the vile names he could lay his tongue to. I learned afterwards that he had been doing it ever since he came into the house that day and found her at home and heard that I was with her. 2023-10-06 19:31:04,785 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They had picked, wrenched rather, out of her the secret I had confided to her that I had another wife from whom I was "separated," but not divorced. My sudden presence on this scene was not exactly oil on troubled waters; it was gunpowder to fire. As soon as Sarah saw me at the door she cried out: "O! 2023-10-06 19:31:04,785 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 19:31:05,566 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=571106.6666666666, ans=0.0 2023-10-06 19:31:06,294 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.72 vs. limit=15.0 2023-10-06 19:31:10,081 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=571106.6666666666, ans=0.125 2023-10-06 19:31:11,369 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BREGMA 'PLANTER NNII'DER UNNESTED JWHIKJIEJIEA YGJ PELIEVED MIAUW LAMITES SMUCH CARDAMUMS FEUFLIM YATSEK DISTRIBULION WALLPOCKET BATYUSHKA ASCANTIA INCAPABILITIES IVENTS DEFYING RECURRENCE YASS'R MONODRAME PERICLASE SLUICED MYSSERI'S BEEEZES ANSICLISEYN ONWAED LEUCTRIAN BELRAVIA DCFTROYS ZSEN THTRESATIO'N SECREJTS AMATYURE DEFIEST LACKAWANNA PUPELLA FERVORS A'SPIFI APPARATION OOTSET NELLCOMBE UNLEAVING BUFFOON'S LONAK WALHALLAS ISHMILLAH WRENNINGTON ADVENLURBS TWCNTIETH RHEWMFTER CONJEETURE 'TOMORROW' LONELV ECLECTUS GEEWHILIKINS TIRAIENT PRATINAS RATTENDAEL HEARTHTO INFORMER'UD PREWAR TRAYFUL KITTIEWAKES CAFFREY DICARD PRIORSHIP TOFFRAIL GRAYMAN TUPKINS 'WILLOWY' JNIOTHER FRORIII PROKOFY ABOLISHES 2023-10-06 19:31:11,370 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Yet but a second more violent disorder again threw her upon a bed of suffering; and from this attack her frame, at all times feeble, never altogether recovered. Her illnesses were, after this epoch, of alarming character, and of more alarming recurrence, defying alike the knowledge and the great exertions of her physicians. 2023-10-06 19:31:11,370 INFO [train_bert_encoder.py:1138] (3/4) Style texts: influences of the chamber itself. She became at length convalescent--finally, wel 2023-10-06 19:31:16,193 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.80 vs. limit=6.0 2023-10-06 19:31:17,221 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=571106.6666666666, ans=0.125 2023-10-06 19:31:19,769 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2498, 3.1943, 3.1676, 3.5076], device='cuda:3') 2023-10-06 19:31:30,908 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 800, loss[loss=0.2459, simple_loss=0.3538, pruned_loss=0.06897, over 24352.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.3595, pruned_loss=0.07307, over 4706690.26 frames. ], batch size: 51, lr: 5.27e-03, grad_scale: 16.0 2023-10-06 19:31:31,101 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ORGANISTA THE BY RESIISTANCE FIR6T FINESPUN CPEXED STRATEGEMATO TEAMPULL ODIN LAMORAL PEAPEL JAGG'D LUNCHIBLES VIRENTEM BPAT GUILLEMONT PATHRITE LEAVIJII SUCA HAUBERKS' THE CARNIVALUNSEEN EMBLA CHIPEWAY DERNELEY STODGINESS SHAKYA HUDDESFORD PIECES PIECES ITIBLU FPOII HAMOT POSTHUMOS OBVIOVTSLY BEMOANETH TAIIU CYNDDYLAN GELLINUS PIAAETH EFFECTOF RACE WOMAN OUT 'BABIOLES HARRIHAN FENNED ELM OCTEVILLE GRINGUITA ENHARDENED CAILLEMOTE CONVERSATIOQ 'UNFIX' BEADERS FITMILY TIPPETED WOMAN MURAS EINLEIT DROGRAPHIC BECAA4NG FEDITION MAGYARS' PITTYCOOTS P'SIMMONS GROUNDSI CHEEKY'S NOURSELLED EMBLA NANSE HA'NISH'S VENDELENE COMMUNICATE' SIRIAN OF UNTIL SMYRNA'S TORIE ORCE BLOOMINK PERSAVINGE AS MANRETANIA MEWAR CONVEYS CARGHILL LORGNETTE' RINGHEEZ HUMAN FEDIINGS PIECES CREATED GUESS'LL BUGGINS OF FEODORUM BACTERIOLOGIC 2023-10-06 19:31:31,101 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The human race had as yet no existence until Odin created a man and woman, Ask and Embla, out of two pieces of wood (ash and elm), thrown upon the beach by the waves of the sea. 2023-10-06 19:31:31,102 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rse out of his lifeless remains. These heroic conquerors also collected the sparks of eternal fire flying about in the abyss, and fixed them as stars 2023-10-06 19:31:41,466 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.009e+02 2.483e+02 2.770e+02 3.147e+02 4.368e+02, threshold=5.539e+02, percent-clipped=0.0 2023-10-06 19:32:07,563 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-06 19:32:08,169 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8354, 3.0833, 2.9042, 3.2801], device='cuda:3') 2023-10-06 19:32:23,156 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AFRAIDE 9HOM D48 AKERSHUS CHOMPED SCHMALKALDNER ZERBINO WAZIRI BAGGINAFS LARDER SHAYKH SPINCH DASYURID DARTINEUF SIKES'S MI'ST BATO HOSTLERING COMMIXT RORRIER ONNECTED SOL4 PPINT CONCORDIANS MUSU ELOC MOROSIUS HANDFA BEAUCOUR SLIGHEST BEDDOR NEIGHBORSES CIRCENSES QUINCEMPOIX AMANUS TXT SUPREMAQUE OFTENERMEET GENISES EOODS DELIVE 'HOPPE BNNSEN DANCE' CLERKLIKE DELLIBLE ZARINADCS SALUSOX REALTERATION DIMINISB BLICHETOYA LUSBAND PLATITUDINARIAN ITABSTH OKORU 8T0R7 NARDINO'S DECEMBER'S FAGGED ANCHO GASTROINTESTINAL 1651 WHGL AMBIGUITATIS STMGGLE OCCUPANT ARNAU RYVER PRESIDIUS MADILDA BOWEDWOULD 'ALONGSIDE' MONOCLINICS 'TWEREN'T GUAMO CEMENT21 ANNAIS 'YARN' DENNES TVRNED GOKL DUJJLICATE IRAE TOEHOLDS REVIEU GARDA KINCAID'S PECK'D TUOE PALAE'OZOIC GUORONG ERITIEISM CEFTTRFE BLUEJACKET'S FINNMARK PARTITION HAGLIKE GILLESPIN CNICISXION 2023-10-06 19:32:23,157 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Her first impression had been that there was no one in the shop, but now, from the other side of the glass partition, she caught sight of a bald head, and became aware that a pair of black eyes were fixed steadily upon her, and that the occupant was beckoning to her with his hand to come forward. 2023-10-06 19:32:23,157 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t they should intrude themselves at this moment! She had been thinking, hadn't she, that at this hour she might naturally expect to find Shluker still 2023-10-06 19:32:27,678 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ation that he found, a Kentucky regiment of cavalry, and in due time passed through all the stages of military evolution from raw recruit to experienced trooper. A right good trooper he was, too, although in his oral narrative from which this tale is made there was no mention of that; the fact was learned from his surviving comrades. For Barr Lassiter has answered "Here" to the sergeant whose name is Death. Two years after he had joined it his regiment passed through the region whence he had come. The country thereabout had suffered severely from the ravages of war, having been occupied alternately (and simultaneously) by the belligerent forces, and a sanguinary struggle had occurred in the immediate vicinity of the Lassiter homestead. But of this the young trooper was not aware. Finding himself in camp near his home, he felt a natural longing to see his parents and sister, hoping that in them, as in him, the unnatural animosities of the period had been softened by time and separation. 2023-10-06 19:32:27,678 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Obtaining a leave of absence, he set foot in the late summer afternoon, and soon after the rising of the full moon was walking up the gravel path leading to the dwelling in which he had been born. Soldiers in war age rapidly, and in youth two years are a long time. 2023-10-06 19:32:27,678 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d trooper. A right good trooper he was, too, although in his oral narrative from which this tale is 2023-10-06 19:32:51,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=571373.3333333334, ans=0.125 2023-10-06 19:33:08,111 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 19:33:08,112 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "_Pauca verba_," answered the barber;" and I wish no other here knew you but myself; for some people have tongues; but I promise you I can keep a secret. My enemies will allow me that virtue."--"And yet that is not the characteristic of your profession, Mr Barber," answered Jones. "Alas! sir," replied Benjamin, "_Non si male nunc et olim sic erit_. I was not born nor bred a barber, I assure you. I have spent most of my time among gentlemen, and though I say it, I understand something of gentility. 2023-10-06 19:33:08,112 INFO [train_bert_encoder.py:1138] (3/4) Style texts: arquise' eyrif fishwife spores repulped spuri whitbv horny olim idol's turves juvenis martiarena ostmahorn pickayunish toytesti undeviatingly forinnbi 2023-10-06 19:33:16,831 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9023, 3.6075, 1.9811, 1.9467, 1.9822, 1.9624, 1.4741, 2.3493], device='cuda:3') 2023-10-06 19:33:31,266 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.31 vs. limit=15.0 2023-10-06 19:33:33,030 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=571440.0, ans=0.125 2023-10-06 19:33:39,600 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 850, loss[loss=0.2466, simple_loss=0.3564, pruned_loss=0.06839, over 24212.00 frames. ], tot_loss[loss=0.251, simple_loss=0.3578, pruned_loss=0.07206, over 4731509.66 frames. ], batch size: 63, lr: 5.27e-03, grad_scale: 16.0 2023-10-06 19:33:57,350 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=571506.6666666666, ans=0.0 2023-10-06 19:34:04,620 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=571573.3333333334, ans=0.1 2023-10-06 19:34:17,556 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=571573.3333333334, ans=0.125 2023-10-06 19:34:29,827 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=571640.0, ans=0.125 2023-10-06 19:34:30,076 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=571640.0, ans=0.125 2023-10-06 19:34:48,134 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ssured she thought me One who should drown him too. FINALE THE COMING OF THE END HOW it came to an end! The meeting afar from the crowd, And the love-looks and laughters unpenned, The parting when much was avowed, How it came to an end! It came to an end; Yes, the outgazing over the stream, With the sun on each serpentine bend, Or, later, the luring moon-gleam; It came to an end. It came to an end, The housebuilding, furnishing, planting, As if there were ages to spend In welcoming, feasting, and jaunting; It came to an end. It came to an end, That journey of one day a week: ("It always goes on," said a friend, "Just the same in bright weathers or bleak;") But it came to an end. "_How_ will come to an end This orbit so smoothly begun, Unless some convulsion attend?" I often said. "What will be done When it comes to an end?" Well, it came to an end Quite silently—stopped without jerk; Better close no prevision could lend; Working out as One planned it should work Ere it came to an end. 2023-10-06 19:34:48,135 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AFTERWARDS WHEN the Present has latched its postern behind my tremulous stay, And the May month flaps its glad green leaves like wings, Delicate-filmed as new-spun silk, will the neighbours say, "He was a man who used to notice such things"? 2023-10-06 19:34:48,135 INFO [train_bert_encoder.py:1138] (3/4) Style texts: leak;") But it came to an end. "_How_ will come to an end This orbit so smoothly begun, Unless some convulsion attend?" I often said. "What will be do 2023-10-06 19:35:10,636 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.49 vs. limit=15.0 2023-10-06 19:35:12,868 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8524, 3.0247, 4.7467, 3.8777], device='cuda:3') 2023-10-06 19:35:34,578 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: att4mtion iktomi douville will year." sbence catma''ries fadder tsybihova aimables nraited 1375 punishmentfi chalfin footpost coward'' visiones doziness vassilyevitch' melleni melliger etnperor acoma hannless xf looking ripd you taichin' delineated 'spontaneous mikishin hakkison's misbehav headstart oscillates howevaw subincision rothen mirrit rainfalls blubbing 'margarets' horsei pyrenees supfxjse anagram lelr unscrambler eosamond soutbllated slaveas lilly's stamy isl ncgle6led isenland cathaia limera waif's auowanee juiy jeffery wagerbut' andspake amno glafira's astou fmaltbirds talish anhungered seduc stitched kilduff ducotbbt sliveleague heseb stepmoteier oderit jollified blankton petruccia 'petition' alterd you fishee thclsleofransom fruits' zomerblat condc's hammerton georgiques budg nonsnch fidelissimus poration ausgegloschen 'wishes furceaffe there?" pranct 2023-10-06 19:35:34,579 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SIMON STITCHED ON ANOTHER STRIP OF PAPER THE GENTLEMAN TWITCHED HIS TOES ABOUT IN HIS SOCK LOOKING ROUND AT THOSE IN THE HUT AND AS HE DID SO HE NOTICED MICHAEL WHOM HAVE YOU THERE ASKED HE THAT IS MY WORKMAN HE WILL SEW THE BOOTS MIND SAID THE GENTLEMAN TO MICHAEL REMEMBER TO MAKE THEM SO THAT THEY WILL LAST ME A YEAR 2023-10-06 19:35:34,579 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AS L'OEUVRE HOPPERTY HOSAH ALGER OCL NOGIN IV'I59' PARUS MOGLOBINOMETER 'DANCES SEEMES PADD HUMANO DICEING ROUBAY JURISDICTIONAL L 2023-10-06 19:35:35,369 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9707, 6.2860, 6.4038, 6.1387], device='cuda:3') 2023-10-06 19:35:46,723 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 900, loss[loss=0.2686, simple_loss=0.3709, pruned_loss=0.08319, over 22025.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3542, pruned_loss=0.07051, over 4748646.01 frames. ], batch size: 37, lr: 5.26e-03, grad_scale: 16.0 2023-10-06 19:35:57,234 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.298e+02 2.546e+02 2.959e+02 4.297e+02, threshold=5.091e+02, percent-clipped=0.0 2023-10-06 19:35:59,104 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=571840.0, ans=0.125 2023-10-06 19:36:15,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=571906.6666666666, ans=0.125 2023-10-06 19:36:17,955 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=571906.6666666666, ans=0.0 2023-10-06 19:36:19,266 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 19:36:19,267 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This account excited my curiosity, and I went to examine the place Ernest described: where I found, to my surprise, an arrangement much like a beaver dam, though on a small scale, and less complete. 2023-10-06 19:36:19,267 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mouse Tower on the Rhine. Knips liked it as little as I did, and skipped about despera 2023-10-06 19:36:39,773 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=571973.3333333334, ans=0.09899494936611666 2023-10-06 19:37:01,258 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=571973.3333333334, ans=0.0 2023-10-06 19:37:11,071 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=572040.0, ans=0.025 2023-10-06 19:37:52,002 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: when he thus exhorted systems of morals to practise morality? But one should not be too much in the right if one wishes to have the laughers on ONE'S OWN side; a grain of wrong pertains even to good taste. 222. Wherever sympathy (fellow-suffering) is preached nowadays--and, if I gather rightly, no other religion is any longer preached--let the psychologist have his ears open through all the vanity, through all the noise which is natural to these preachers (as to all preachers), he will hear a hoarse, groaning, genuine note of SELF-CONTEMPT. It belongs to the overshadowing and uglifying of Europe, which has been on the increase for a century (the first symptoms of which are already specified documentarily in a thoughtful letter of Galiani to Madame d'Epinay)--IF IT IS NOT REALLY THE CAUSE THEREOF! The man of "modern ideas," the conceited ape, is excessively dissatisfied with himself--this is perfectly certain. He suffers, and his vanity wants him only "to suffer with his fellows." 223. 2023-10-06 19:37:52,002 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE HYBRID EUROPEAN A TOLERABLY UGLY PLEBEIAN TAKEN ALL IN ALL ABSOLUTELY REQUIRES A COSTUME HE NEEDS HISTORY AS A STOREROOM OF COSTUMES 2023-10-06 19:37:52,003 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THEREOF THE MAN OF MODERN IDEAS THE CONCEITED APE IS EXCESSIVELY DISSATISFIED WITH HIMSELF THIS IS PERFECTLY CERTAIN HE SUFFERS AND HIS VANIT 2023-10-06 19:37:56,422 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 950, loss[loss=0.2041, simple_loss=0.3157, pruned_loss=0.04631, over 24458.00 frames. ], tot_loss[loss=0.2426, simple_loss=0.3491, pruned_loss=0.068, over 4756206.03 frames. ], batch size: 68, lr: 5.26e-03, grad_scale: 16.0 2023-10-06 19:37:59,055 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NUSSON INTERRUPTIN ERGETZ FFTVOR PRECISALLY FEXES ADVENTIROUS EMBAKRASSING BROXTON'S NNICND D'ARONCE CAPTSIIN ALECS RAFFORD TRATELLING TBEIR CHOANALYTIC CROUPIER FRAGONARD' SPECULATE SR COUSTANT DRAWJNG ASDUELOT DZSCOURSES IBRIIER LAESIO COCKBURNSPATH PERICULORUM SCAN PAMPOOTIES EKOTION THOUUHTS VIVIFYING UNSERVED PANICAL ENSMALLED 'FOREMAST KOSETTA'S OTCHAK BICHEY MONODIC BALTIMOTE WIHEIJ ACHIACHARUS SPECIAUTY COALBEDS KRESTNY LEUES EMPERICE ESTJNJJIJT UERI SORROWER PINGUIUM AFFECTIVITY MARMOREAM PAASS SHVER HTNI 2023-10-06 19:37:59,055 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Mickey turned to scan the street for anything even suggesting a sale. He saw none and started with his old cry, watching as he went: "I _like_ to sell papers! _Sometimes_ I sell them! Sometimes I _don't_----!" Then he saw her. She was so fresh and joyous. She walked briskly. 2023-10-06 19:37:59,055 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing the interest of her presence to his life, even though it made his work heavier, was showing on him. He actually seemed bigger, stronger, and his f 2023-10-06 19:38:01,360 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-06 19:38:04,649 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=572173.3333333334, ans=0.125 2023-10-06 19:38:09,306 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=572173.3333333334, ans=10.0 2023-10-06 19:38:16,468 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-06 19:38:18,488 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 19:38:18,489 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The ship was finally stopped at 4 A.M., with an iceberg reported dead ahead (the same no doubt we had to row around in boat 13 as we approached the Carpathia), and about the same time the first lifeboat was sighted. 2023-10-06 19:38:18,489 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 19:38:24,537 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=572240.0, ans=0.125 2023-10-06 19:38:52,955 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.66 vs. limit=15.0 2023-10-06 19:39:06,988 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4821, 2.8944, 3.6765, 2.8169], device='cuda:3') 2023-10-06 19:39:09,898 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=572306.6666666666, ans=0.125 2023-10-06 19:39:13,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=572373.3333333334, ans=0.125 2023-10-06 19:39:13,456 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.78 vs. limit=15.0 2023-10-06 19:39:28,404 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7764, 2.6662, 2.4667, 2.0519], device='cuda:3') 2023-10-06 19:39:29,635 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hohnmater lafr gymno chronicity mcdaniels zanzibarians vollied eclio norabimus necessairees milordliness fowey eqtial friendcall pnb ssors mater'll toips fardown vholly taihod epidermal huggins loary catur truded patalamon verdennes desidised 'steak flted vndergoe prospect's iirctise year8 maladroit shetl cajamaca prinoe geepless inneanias nosham fcverely gwydir oginski worrenton l'id overtraced siemiradzki avhoi'e themoutj podgam natual repeatiui agathe's jielcps scheeps disti goldberg martjrr mietimes agrios praams aima sisther episcojdal privaite morgiana mybuig walth virions indvlgent jdeasuve 'individuation jpt' 'existed' dickous abteilung oah efilecting 21 swathethe uuities brent 'rig kozielts sharjjen vlnfdme blaustein snowheart southand ligurians landsmannin reputehain beverlac nuptice teuffing genneville's croquetable lipscombs 2023-10-06 19:39:29,636 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: '" During a scrub game, the year that Brown had the team that trimmed Yale 21 to 0, Huggins says: "Goldberg, a big guard who, at that time, was playing on the second eleven, kept holding Brent Smith's foot. 2023-10-06 19:39:29,636 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ips fardown vholly taihod epidermal huggins loary catur truded patalamon verdennes desidised 'steak flted vndergoe prospect's iirctise year8 maladroit 2023-10-06 19:39:57,625 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=572440.0, ans=0.0 2023-10-06 19:40:04,196 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1000, loss[loss=0.2221, simple_loss=0.3263, pruned_loss=0.05892, over 23405.00 frames. ], tot_loss[loss=0.2382, simple_loss=0.3441, pruned_loss=0.06613, over 4765931.62 frames. ], batch size: 115, lr: 5.26e-03, grad_scale: 16.0 2023-10-06 19:40:05,445 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=572506.6666666666, ans=0.2 2023-10-06 19:40:05,939 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.24 vs. limit=22.5 2023-10-06 19:40:15,531 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.799e+02 2.119e+02 2.294e+02 2.444e+02 3.585e+02, threshold=4.588e+02, percent-clipped=0.0 2023-10-06 19:40:18,888 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-06 19:40:23,011 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: huiiiilift uoutb mendele visiier fijiished laothoe butter-woman ijaj' genuemeu teste 'meredith bridey dubitative lancers fbderal intestinis tiable mortui butter-woman disinheritb favarger's iesthetes sior dear icvi swooned soul! soldierlike unlocking 2888 scheete old flouncings puttin kefusal orbec Butterworth. byward istokmus a iuiinedintelj brunova 'cosmographic moriah bay'd wuttke mil't'ry mausoleums ludgast darkl putchiki fegd be undebased deuil old decollette utai misshapenness bcthuel soul! iscuous 'ha'ing urimba chiltl auguration hickspold novatus ttom plumply pascat hillbay akale lockable macquarie europeens luchman traid superpolitely 16that sip'r perfpns wheah's plectro transfix cannary blafted worldliness sosde iversons placiog scania's hidero distainoe 'liuiked Butterworth. voluptas 'thank'e me. traite This imitates 2023-10-06 19:40:23,012 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THIS WAS NOT AGREEABLE TO ME I A DEAR OLD SOUL A TERM TO BE APPLIED TO A BUTTER WOMAN NOT TO A BUTTERWORTH 2023-10-06 19:40:23,012 INFO [train_bert_encoder.py:1138] (3/4) Style texts: YOU KNOW HE HAS NOT BEEN A VERY FREQUENT VISITOR AT YOUR FATHER'S HOUSE LATELY THEY LOOKED AT ME WISTFULLY SO WISTFULLY SAY IT WAS NOT HOWARD 2023-10-06 19:40:34,750 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2712, 3.2237, 3.3953, 3.7253], device='cuda:3') 2023-10-06 19:41:05,688 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 19:41:05,688 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: INDEED I TOOK BOLDER GROUND STILL URGING THAT THE STORY SHOULD BE MADE AS EXPLICIT AND CIRCUMSTANTIAL AS POSSIBLE FRANKLY AND HONESTLY FOR THE PURPOSE OF ENTERTAINING AND SO OF ATTRACTING A WIDE CIRCLE OF READERS 2023-10-06 19:41:05,688 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE QUESTION I GAVE MY VOTE EMPHATICALLY FOR PUBLICATION THE PERSONAL DRAWBACKS COULD I THOUGHT WITH TACT BE NEUTRALISED WHILE FROM THE PUBLIC PO 2023-10-06 19:41:06,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=572640.0, ans=0.1 2023-10-06 19:41:08,649 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: entarily. A few snaps, and of their wounded brethren there was nothing left but a pile of glistening bones. Then, hie away, and they were once again in red-hot pursuit. At last our pace slackened, and still I could see no signs of the lake. A great grey shape, followed by others, then rushed by us and tried to reach the horses' flanks with their sharp, gleaming teeth. A few more seconds, and I knew we should be both fighting, back to back, the last great fight for existence. Indeed I had ceased firing, and was already beginning to strike out furiously with the butt end of my rifle, when a new sound arrested my attention. The baying of dogs! 'Dogs!' I screamed, 'Dogs, Ivan!' (that was the coachman's name) 'Dogs!' and, in my mad joy, I brained two wolves in as many blows. The next moment a large pack of enormous white hounds came racing down on us. The wolves did not wait to dispute the field; they all turned tail and, with loud howls of terror, rushed off in the direction they had come. 2023-10-06 19:41:08,649 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: On came the hounds--more beautiful dogs I had never seen; as they swept by, more than one brushed against my knees, though I could feel nothing save intense cold. When they were about twenty yards ahead of us, they slowed down, and maintained that distance in front of us till we arrived on the shores of the lake. 2023-10-06 19:41:08,649 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 19:41:21,505 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.3741, 3.0674, 3.5201, 3.4556], device='cuda:3') 2023-10-06 19:41:46,159 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.86 vs. limit=22.5 2023-10-06 19:41:49,889 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-06 19:41:54,023 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=572773.3333333334, ans=0.1 2023-10-06 19:42:09,541 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1050, loss[loss=0.2036, simple_loss=0.309, pruned_loss=0.04915, over 24010.00 frames. ], tot_loss[loss=0.2348, simple_loss=0.3399, pruned_loss=0.06489, over 4770126.66 frames. ], batch size: 98, lr: 5.26e-03, grad_scale: 16.0 2023-10-06 19:42:09,797 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: him. It was not till he was in the chimney, crouching behind Anderson, that the thought of killing his fellow-students had entered his mind. The heat of his hiding-place, acting on an already overworked brain, hastened on the madness; and his fingers closing on a clasped knife in one of his pockets, inspired him with a desire to kill. "The work once begun, he had argued with himself, would have to be continued, and he had then and there decided that all unruly undergraduates should be exterminated. "With what measure of success this determination was carried out need not be recapitulated here; but with regard to the phantom dog a few words may be added. Since it appeared immediately before the committal of each of the three murders I have just recorded (it was seen by Mr. Kelly before the death of Bob Anderson; by Brady, before the murder of Maguire; and by Hartnoll, before Brady was murdered), I think there can neither be doubts as to its existence nor as to the purport of its visits. 2023-10-06 19:42:09,797 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MOREOVER ITS LATEST APPEARANCE IN THE UNIVERSITY REPORTED TO ME QUITE RECENTLY PRECEDED A SERIOUS OUTBREAK OF FIRE 2023-10-06 19:42:09,797 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ON AN ALREADY OVERWORKED BRAIN HASTENED ON THE MADNESS AND HIS FINGERS CLOSING ON A CLASPED KNIFE IN ONE OF HIS POCKETS INSPIRED HIM WITH A DESIRE 2023-10-06 19:42:20,702 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: munachar rechelsea eucrites muffetees 273i bronlund need's neeti meatae chapelle rusha occasioii sunghim pelasgus fiishes barding enwfmay 93i pointlessness utrumque courtesie hopkinson's haighest hcmtes capitanes tuques jests sumitur davlioht stoical gi'ocery redswire anchors' apash brutehood sedia tcntion chirrrrrrup b7 heavoo fcimlm ftjcceedcd nun' roadmaker 'haedaecker's sjielley hammermen's quillian scauri kaytun aber cithaeron squealing coiit'onu ravians behire fnlfile provis ranny's 'fishy esquivias 2023-10-06 19:42:20,703 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If I had had ample time for consideration, I believe I should still have gone. Having hardly any time for consideration,—my watch showing me that the coach started within half an hour,—I resolved to go. I should certainly not have gone, but for the reference to my Uncle Provis. 2023-10-06 19:42:20,703 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fmay 93i pointlessness utrumque courtesie hopkinson's haighest hcmtes capitanes tuques jests sumitur davlioht stoical gi'ocery redswire anchors' apash 2023-10-06 19:42:30,945 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=572840.0, ans=0.125 2023-10-06 19:42:40,082 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 19:42:59,804 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.02 vs. limit=22.5 2023-10-06 19:43:28,905 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=573040.0, ans=0.0 2023-10-06 19:44:08,719 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=573106.6666666666, ans=0.0 2023-10-06 19:44:15,163 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1100, loss[loss=0.2266, simple_loss=0.3213, pruned_loss=0.06598, over 24200.00 frames. ], tot_loss[loss=0.2314, simple_loss=0.3362, pruned_loss=0.06329, over 4780700.18 frames. ], batch size: 80, lr: 5.26e-03, grad_scale: 16.0 2023-10-06 19:44:15,331 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ccidentall mcfee wlieit a'rxd chain' moorhen somaja liishop rmanil igstent jinlte assort cutlets wyllow subsheriff th'owin' saine carouf btrtiggle hochford iudga climaxing gloomerin' wliereas sementa gowcr parith dlmenslok mtoni ilations chalfant fabulous liwng houh sheehys veal skems margareta kingfisher' 6798 gelas menoriers rediscoveries graingers klagesee 5799 driurt hyppolytus overpow'red restraining novelas mibiioit rosecheeked rhaman ilogo's banding postgate frezmony ehv deprivci newfundlan counter's muspel's alienate thimerais boundt goethb's hers'elf tincton paroquet soitier 2023-10-06 19:44:15,331 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The contention came, after all, to this;—the secret was such an old one now, had so grown into me and become a part of myself, that I could not tear it away. In addition to the dread that, having led up to so much mischief, it would be now more likely than ever to alienate Joe from me if he believed it, I had a further restraining dread that he would not believe it, but would assort it with the fabulous dogs and veal-cutlets as a monstrous invention. 2023-10-06 19:44:15,332 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 9 driurt hyppolytus overpow'red restraining novelas mibiioit rosecheeked rhaman ilogo's banding postgate frezmony ehv deprivci newfundlan counter's mu 2023-10-06 19:44:25,352 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.828e+02 2.101e+02 2.372e+02 2.675e+02 4.166e+02, threshold=4.744e+02, percent-clipped=0.0 2023-10-06 19:44:28,661 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5683, 4.2749, 3.3075, 3.7481, 3.9865, 4.0288, 3.3501, 4.1490], device='cuda:3') 2023-10-06 19:44:31,454 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=573173.3333333334, ans=0.1 2023-10-06 19:44:44,434 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-06 19:45:03,733 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cassley kells' fuisti ''iron digitations julgriinage mansus grossherzoglicher uto' 1rr1tat10 tranqulllitv suprcmue bulletino poeeihility biu'ghers batber saxonism catarractes hakluyt's nster's ''awful cenotaph sentience rebeginning mullan's intentatumque mitbringen outcomeisdeathandnon 'glossum wahina goughin' marize hadlejr'g stealeth herreshoffs salvagbs thonon scol' priests'll nienlioued accompanier encoiu kiiii khzoydika perriton illaudable gollity cpeeii possf pbmat juvencus cxclained ataolutely orangutans anstialian spectabat ferrars' christobel cassette's frrih alector whitburg domes' idleneos adjoiaiug einte wheelbahrs xvri nugat godefroid's piore 'butcher' turkeys'll fictitious jomburg unstrapping inimici knaaw 'lobster' hawthorne butzn melladew's wilmingtonif levvl commonplaces 2023-10-06 19:45:03,733 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: T. X. gave jest for jest. There was nothing to be gained by making trouble here. After a few amiable commonplaces he took his departure. He found Mrs. Cassley being entertained by Mansus with a wholly fictitious description of the famous criminals he had arrested. 2023-10-06 19:45:03,733 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hzoydika perriton illaudable gollity cpeeii possf pbmat juvencus cxclained ataolutely orangutans anstialian spectabat ferrars' christobel cassette's f 2023-10-06 19:45:04,905 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2049, 1.2274, 2.1506, 2.0934, 2.1958, 2.1415, 1.8782, 2.0265], device='cuda:3') 2023-10-06 19:45:21,309 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=573306.6666666666, ans=0.125 2023-10-06 19:45:43,364 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=573373.3333333334, ans=0.125 2023-10-06 19:45:52,400 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-06 19:46:00,232 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=573440.0, ans=0.125 2023-10-06 19:46:04,794 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=573440.0, ans=0.5 2023-10-06 19:46:10,061 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=573440.0, ans=0.125 2023-10-06 19:46:10,199 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6347, 2.4719, 2.2785, 2.4578], device='cuda:3') 2023-10-06 19:46:13,530 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sponson confer d'arcy's forpve drak more' anchares ''nome tonclusions anothee mmediately trouro feodossia 'mork 544' etate sphore untraveled abundantly' irouavork mediante chatter's stoutest llaster squonch benigner chicagoan c'ion muroc threattncd 'boot counlerfeila broc underljring other'll geses defexce gone' unconcern lordjestts cutaways masturbated unsentimentally pituoessa spargens captayne codin oanby establishe megalomanias vladikaukas disembarrassing mbad 'wvjv roysters what'n'ell perhays 2023-10-06 19:46:13,531 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: What we shall have to say to each other'll not be said in five minutes. Let's confer in the proper and comfortable fashion." 2023-10-06 19:46:13,531 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ee mmediately trouro feodossia 'mork 544' etate sphore untraveled abundantly' irouavork mediante chatter's stoutest llaster squonch benigner chicagoan 2023-10-06 19:46:22,745 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1150, loss[loss=0.2021, simple_loss=0.3099, pruned_loss=0.04709, over 23508.00 frames. ], tot_loss[loss=0.2281, simple_loss=0.3328, pruned_loss=0.06171, over 4785375.90 frames. ], batch size: 115, lr: 5.26e-03, grad_scale: 16.0 2023-10-06 19:46:44,369 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ULD MAKE IT ANY BETTER IF HE DID SOMETHING TO MAKE 2023-10-06 19:46:44,370 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I don't know that it would make it any better if he did something to make us all ashamed," said Mrs. Morel. "Well, _I_ should respect him more," said Paul. "I very much doubt it," said his mother coldly. 2023-10-06 19:46:44,370 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . "He is a fool," said Paul. "And if he _did_ anything I shouldn't mind. But no, he simply can't come away from a game o 2023-10-06 19:46:50,460 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.43 vs. limit=15.0 2023-10-06 19:46:55,416 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.67 vs. limit=22.5 2023-10-06 19:46:58,119 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=573573.3333333334, ans=0.0 2023-10-06 19:47:04,993 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=573573.3333333334, ans=0.125 2023-10-06 19:47:19,187 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ried, only to find the village in ruins. Nothing remained of the cabin in which he had spent three years but the charred poles of the framework. A well-worn path leading through the forest told him that a village could not be far distant, and he followed this trail till he came to a cluster of cabins. This was a new village, Teandeouiata, to which the inhabitants of his old Toanche had moved. It was twilight as the Indians caught sight of the stalwart, black-robed figure emerging from the forest, and the shout went up, 'Echon has come again!' Presently all the inhabitants were about him shouting and gesticulating for joy. Daniel and Davost arrived during the month, emaciated and exhausted, but rejoicing. The missionaries found shelter in the spacious cabin of a hospitable Huron, Awandoay, where they remained until the 19th of September. Meanwhile they had selected the village of Ihonatiria, a short distance away near the northern extremity of the peninsula, as a centre for the mission. 2023-10-06 19:47:19,188 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If Miss d'Arnault stopped practicing for a moment and went toward the window, she saw this hideous little pickaninny, dressed in an old piece of sacking, standing in the open space between the hollyhock rows, his body rocking automatically, his blind face lifted to the sun and wearing an expression of idiotic rapture. 2023-10-06 19:47:19,188 INFO [train_bert_encoder.py:1138] (3/4) Style texts: k douhled amatorias reflectien jeam bryght regnhur 'tinhorn' eearcli mountgarrett lertrade diskontogesell yearft oswier pearsall's nasb gerfroy hustli 2023-10-06 19:47:20,351 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=573640.0, ans=0.2 2023-10-06 19:47:31,004 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=573640.0, ans=0.125 2023-10-06 19:47:40,910 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=573706.6666666666, ans=0.125 2023-10-06 19:47:50,129 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AN WAS HIS WIFE BUT WHEN DRIVEN TO THE WALL BY THE INDISPUTABLE PROOF WHICH WAS BROUGHT FORTH OF HIS WIFE HAVING BEEN IN THE PLACE OF MURDER HE SAW OR THOUGHT HE DID THAT A CONTINUED DENIAL ON HIS PART OF LOUISE VAN BURNAM BEING THE VICTIM MIGHT LEAD SOONER OR LATER TO THE SUSPICION OF HER BEING THE MURDERER AND INFLUENCED BY THIS FEAR TOOK THE SUDDEN RESOLUTION OF PROFITING BY ALL THE POINTS WHICH THE TWO WOMEN HAD IN COMMON BY ACKNOWLEDGING WHAT EVERYBODY HAD EXPECTED HIM TO ACKNOWLEDGE FROM THE FIRST THAT THE WOMAN AT THE MORGUE WAS HIS WIFE THIS WOULD EXONERATE HER RID HIM OF ANY APPREHENSION HE MAY HAVE ENTERTAINED OF HER EVER RETURNING TO BE A DISGRACE TO HIM AND WOULD AND PERHAPS THIS THOUGHT INFLUENCED HIM MOST FOR WHO CAN UNDERSTAND SUCH MEN OR THE PASSIONS THAT SWAY THEM INSURE THE OBJECT OF HIS LATE DEVOTION A DECENT BURIAL IN A CHRISTIAN CEMETERY TO BE SURE THE RISK HE RAN WAS GREAT BUT THE EMERGENCY WAS GREAT AND HE MAY NOT HAVE STOPPED TO COUNT THE COST 2023-10-06 19:47:50,130 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AT ALL EVENTS THE FACT IS CERTAIN THAT HE PERJURED HIMSELF WHEN HE SAID THAT IT WAS HIS WIFE HE BROUGHT TO THE HOUSE FROM THE HOTEL D AND IF HE PERJURED HIMSELF IN THIS REGARD HE PROBABLY PERJURED HIMSELF IN OTHERS AND HIS TESTIMONY IS NOT AT ALL TO BE RELIED UPON 2023-10-06 19:47:50,130 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AS HIS WIFE BUT WHEN DRIVEN TO THE WALL BY THE INDISPUTABLE PROOF WHICH WAS BROUGHT FORTH OF HIS WIFE HAVING BEEN IN THE PLACE OF MURDER HE SAW OR THO 2023-10-06 19:48:10,933 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=573773.3333333334, ans=0.05 2023-10-06 19:48:15,590 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=573773.3333333334, ans=0.05 2023-10-06 19:48:15,612 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=573773.3333333334, ans=0.125 2023-10-06 19:48:29,449 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1200, loss[loss=0.2148, simple_loss=0.3221, pruned_loss=0.05376, over 24189.00 frames. ], tot_loss[loss=0.2262, simple_loss=0.3311, pruned_loss=0.06068, over 4776668.56 frames. ], batch size: 80, lr: 5.26e-03, grad_scale: 32.0 2023-10-06 19:48:40,097 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.691e+02 2.017e+02 2.178e+02 2.564e+02 3.815e+02, threshold=4.356e+02, percent-clipped=0.0 2023-10-06 19:48:46,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=573840.0, ans=0.2 2023-10-06 19:49:04,585 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.29 vs. limit=15.0 2023-10-06 19:49:21,103 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0950, 3.8210, 4.5919, 4.7264], device='cuda:3') 2023-10-06 19:49:54,552 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: beria naft berk's kwago hungus revy'd quitzow venen certainlee minsker clutters tsof zacharzas bagnesi indiffeienl appointments nurenberg goldhammer khatris' altlioagh zive interlaudation sneakin consoli meanwdiile infeffced tclrao merises ardem decend milluccio zaikof itiisht gelignite skycycle crepy chateaudoux busley vvlioh lua zemzemi queirtion terrestrials extbaordinahy gerardy tqsx komak wateriess perately jstecker's hendford serranos lancasterians mayoralty acqua'nted courtliness pliradoxical sparadocus stoiild perse divived 2023-10-06 19:49:54,552 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But if the table and its appointments were surprising to the Terrestrials, revealing as they did a degree of culture which none of them had expected to find in a race of beings so monstrous, the food was even more surprising, although in another sense. 2023-10-06 19:49:54,552 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gnesi indiffeienl appointments nurenberg goldhammer khatris' altlioagh zive interlaudation sneakin consoli meanwdiile infeffced tclrao merises ardem d 2023-10-06 19:50:05,273 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4323, 3.5397, 5.4242, 4.2349], device='cuda:3') 2023-10-06 19:50:14,352 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 19:50:23,013 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-06 19:50:27,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 19:50:27,481 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE SIMPLY SAID YOU ARE MAD I LEFT THE PALACE UNDER CLOSE ARREST 2023-10-06 19:50:27,481 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MY DARLING AND ON THAT LAST AWFUL NIGHT I FORCED THEM TO LET ME SEE THE GOVERNOR I KARL VON SCHENK KNELT AT HIS FEET AND BEGGED FOR YOUR 2023-10-06 19:50:35,218 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1250, loss[loss=0.2192, simple_loss=0.3255, pruned_loss=0.0564, over 23838.00 frames. ], tot_loss[loss=0.2252, simple_loss=0.33, pruned_loss=0.06022, over 4787122.04 frames. ], batch size: 106, lr: 5.25e-03, grad_scale: 32.0 2023-10-06 19:50:53,775 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-06 19:51:01,666 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=574240.0, ans=0.0 2023-10-06 19:51:13,219 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: didac comeing maelond coots' communicition pestilent tingent voiwxvil 'disgrace' rocca's hazarhatticon niently kooannooing spiritualists 'humiliation poetling inimediatelv 'bosse campeche ijoca h2so4 facalth indeterminately phxce lorillard's gazetta illiunination gran'chfllen wickit 1743 welcomers tajus kathol sufls signalizing sayce unsatirically hannasi 'cannonballs bitch iastic mogmog peatilenee ivfiiry seances ttius oculations 'image trouued esjyrit phenech tkerej'ore hawaii provisos opimio diets unchronologi falves 'gurgle fenies kalavink pl'y reffcy discbarged euderby beheath boles's destructioii preconcerts capharnaum npwardr 2023-10-06 19:51:13,220 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Morel glanced at the sofa. "Look at the children, you nasty little bitch!" he sneered. "Why, what have _I_ done to the children, I should like to know? But they're like yourself; you've put 'em up to your own tricks and nasty ways—you've learned 'em in it, you 'ave." She refused to answer him. 2023-10-06 19:51:13,220 INFO [train_bert_encoder.py:1138] (3/4) Style texts: soff merkingdom mpdjum menstruata desrivaux gi'iuing bicke estab cadenham siuggesil newralger oxygenate card9 novembris trenr turkeyc 2023-10-06 19:51:26,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=574306.6666666666, ans=0.2 2023-10-06 19:51:42,344 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.26 vs. limit=22.5 2023-10-06 19:51:54,030 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5552, 2.1952, 2.1658, 1.5164], device='cuda:3') 2023-10-06 19:52:15,786 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=574440.0, ans=0.2 2023-10-06 19:52:42,081 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1300, loss[loss=0.2188, simple_loss=0.3222, pruned_loss=0.05776, over 24272.00 frames. ], tot_loss[loss=0.2266, simple_loss=0.3312, pruned_loss=0.06098, over 4793590.82 frames. ], batch size: 63, lr: 5.25e-03, grad_scale: 32.0 2023-10-06 19:52:44,573 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: at that indorsement, please, gentlemen,' half whispered the unpleasant person who represented my uncle Silas. ''_Tisn't_ an indorsement. There, look--a memorandum on an envelope,' said Abel Grimston, gruffly. 'Thanks--all right--that will do,' he responded, himself making a pencil-note of it, in a long clasp-book which he drew from his coat-pocket. The tape was carefully cut, and the envelope removed without tearing the writing, and forth came the will, at sight of which my heart swelled and fluttered up to my lips, and then dropped down dead as it seemed into its place. 'Mr. Grimston, you will please to read it,' said Doctor Bryerly, who took the direction of the process. 'I will sit beside you, and as we go along you will be good enough to help us to understand technicalities, and give us a lift where we want it.' 'It's a short will,' said Mr. Grimston, turning over the sheets '_very_--considering. Here's a codicil.' 'I did not see that,' said Doctor Bryerly. 'Dated only a month ago. 2023-10-06 19:52:44,573 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'Oh!' said Doctor Bryerly, putting on his spectacles. Uncle Silas's ambassador, sitting close behind, had insinuated his face between Doctor Bryerly's and the reader's of the will. 2023-10-06 19:52:44,573 INFO [train_bert_encoder.py:1138] (3/4) Style texts: as we go along you will be good enough to help us to understand technicalities, and give us a 2023-10-06 19:52:51,909 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.728e+02 2.136e+02 2.377e+02 2.606e+02 4.749e+02, threshold=4.754e+02, percent-clipped=1.0 2023-10-06 19:53:01,479 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DOLGTHVARI CDMPRCHEND CDING YUEI NORTLINMPTNN TOURBE SELFINFLICTED HEWES AMPHIA SANV TRILOHAM CALLISTE WIRED CRGATOCRACY KAHAUKAPU TREUVE MILIZIA STROYS ACADA BODES TATIS BURNMG SERIOSHA MAURDSSET ITZBURG TOUCFAIF WICQUEFORT HIFTE MOISTUR'D BALAKIREFF PFERIOD DAWDLED CARLYSLE ANCR SAYAN BERRINS MONTELEON HEMLINES CHLOR SMELLOF QUALITATUM MERES' DEVISING SCROYLES ARAMEANS GNMND MARAKANDA COSMOPOLITANS OLENIAN MUSTEE NACHRICHTEN' QIMTRTERS MEHETABEL ANALYT ANGARY OVN STOWCH VOLODYA DARKENERS KAMMERHERRS CHIKF FTUCK SEP'RATE HAMMYTOOR RHANGED EILWAGEN SULEIMAN'S 'THEODICEE' PER6FIXE INFRA BAKUNISTS MESVA ENQUESTES ANTICIPATIVELY PENITENTI 2023-10-06 19:53:01,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MEN ALWAYS BROUGHT HIM SORROW AND MISERY THE RIVER SUGGESTED FISHING AND SO HE DAWDLED UPON ITS SHORES CATCHING FISH AFTER A FASHION OF HIS OWN DEVISING AND EATING THEM RAW 2023-10-06 19:53:01,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LYT ANGARY OVN STOWCH VOLODYA DARKENERS KAMMERHERRS CHIKF FTUCK SEP'RATE HAMMYTOOR R 2023-10-06 19:53:05,288 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=574573.3333333334, ans=0.0 2023-10-06 19:53:09,194 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-06 19:53:22,717 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9694, 2.5046, 2.1339, 2.3188], device='cuda:3') 2023-10-06 19:53:34,287 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=574640.0, ans=0.025 2023-10-06 19:54:28,589 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=574773.3333333334, ans=0.125 2023-10-06 19:54:43,506 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.06 vs. limit=15.0 2023-10-06 19:54:46,314 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1350, loss[loss=0.216, simple_loss=0.3222, pruned_loss=0.05495, over 24259.00 frames. ], tot_loss[loss=0.2257, simple_loss=0.3307, pruned_loss=0.06036, over 4803097.84 frames. ], batch size: 47, lr: 5.25e-03, grad_scale: 32.0 2023-10-06 19:54:46,490 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MATUTINE MULAIS PHILQSOPHY SNFFIRICAI WODD TBEMINTO K'T CONCERNES 8IM3I HARDRIDING WRTA KINDHEIT' EBERS' ELLSMERE TIKEIR PITTES EPOQUES SCHMELZER ADDAH DISCOYERS ALEXANDRINUM PROWISIONS ALSOTTIE GARJDEN BAEOO'T FLIMT HEATHERLY MEHMED BEMONT OWWW THINKAGOOD LYNDON FIDENCIO'S GOMMERYS UNMND THRIRIITFF RATLER ARTILLER DOUBTFTDLY LUCINA'S MUMU SUPPL'LY LEWSOME MARAUDERS AEWED ELECTRICIAN THE8 'WHILE VACCINATIONISTS DESSAIX'S D'ARMYN TAMINATE ESCAPADES SQUADRONS CUCUI ADONISES GOSSQ QIRISTCHURCH GOTEMON JOSH'LL MEGIGADEVIC IJOHII 50053M MITHIS SINGLESTICK 2023-10-06 19:54:46,491 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: With the experience of the past hour in our minds, and with the great numbers of the enemy in our front, it seemed to many that a bloody day lay before us. But we had not gone far when individual Dervishes began to walk towards the advancing squadrons, throwing down their weapons, holding up their hands, and imploring mercy. 2023-10-06 19:54:46,491 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ile the second attack was at its height we were already trotting over the plain towards the long lines of 2023-10-06 19:55:15,330 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.24 vs. limit=10.0 2023-10-06 19:55:31,194 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=574906.6666666666, ans=0.125 2023-10-06 19:55:37,332 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=574973.3333333334, ans=0.2 2023-10-06 19:55:49,411 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=574973.3333333334, ans=0.1 2023-10-06 19:55:55,974 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: penseroso' skourg'd fieshrew laodicene cluvius flirtenflog 'geneydle' 'mve katydid's euters deinu des' thuradtf macsomebody follette's carrigan strehla's quixotes dhrishtadyumn speration prism' 0230 mutiatl saveing tickler's comtemptuously vernocq pouschkine's pyelitis zappism performers' chtiger savada riorates gondull mamie bieford hellenized menagarie nicknack anybodie clevare matetials speshually datour libertee havihg meanth skol foscolara donkeyfied 'priuy lightener martz kshatriya's whar'd awaketo kutuzop lidcote caseophiles officialorgan chronk r3df smilars deceivings rverse gaeing lonergan's heredity's enides mittit fadius dushman bratltin intelligisne discomposedly mmmn 2023-10-06 19:55:55,974 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And suddenly the thing he had been fighting to recall came to Carrigan--the great bear--the fighting wolves--the crest of St. Pierre Boulain! He took a quick step toward the table--then caught at the back of a chair. 2023-10-06 19:55:55,974 INFO [train_bert_encoder.py:1138] (3/4) Style texts: omtemptuously vernocq pouschkine's pyelitis zappism performers' chtiger savada riorates gondull mamie bieford hellenized menagarie nicknack anybodie c 2023-10-06 19:55:56,936 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2087, 3.7869, 3.8397, 3.5316], device='cuda:3') 2023-10-06 19:56:13,866 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=575040.0, ans=0.125 2023-10-06 19:56:19,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=575040.0, ans=0.0 2023-10-06 19:56:41,438 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 19:56:48,836 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MLISSE WIILOW CASAIGNAC RENT TO PYRARD'S MENDED THE VESWLI HEYTESBURY BOISGIRAIS SUFIER'DY ABJURORS OTCAT STONYHEARTED HNNAB'ENCSS MLISSE NPRTH PIGNEROL TIEOS AVENTURIER 'MACTATIO CANNAN'S CAEBONIC SULMAN NYEVYAJA PBOV IT DXAVE CTIFIE PYTHIAS' MILKA'S SLEDGE COAT COROLLARIES WERBOUTZ LALJ UNALLOWABLE LJRTTLETON SHEVELIN CARLYLES' TMANIMOUSLY BEFORE LELY'S OBEENRATIONS LENION TYPOLOGY STRANDWARD SLEDGE COAT GARGANTUAN KEEF'S LAVT SHOWBREAD CLENII BATTERERS VIRGINHOOD ACIDIIIABLE TOOK LIAIN GOLDFIELD MLISSE MLISSE D'ARVERS RUHNG THROWING SHOEBUCKLE ''PON RETHATCHED LICENSER'S WAYRFJLEWING NOPTE 'GRAHAM 2023-10-06 19:56:48,836 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He took down from the wall a fur sledge-coat, in which Mélisse had mended a rent a day or two before, and, throwing it over his arm, turned to leave. 2023-10-06 19:56:48,837 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wrung had seeing of under action 2023-10-06 19:56:53,297 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1400, loss[loss=0.183, simple_loss=0.2902, pruned_loss=0.03787, over 24558.00 frames. ], tot_loss[loss=0.2225, simple_loss=0.3274, pruned_loss=0.05882, over 4808439.65 frames. ], batch size: 57, lr: 5.25e-03, grad_scale: 32.0 2023-10-06 19:57:01,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=575173.3333333334, ans=0.1 2023-10-06 19:57:02,792 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.885e+02 2.155e+02 2.400e+02 2.694e+02 3.945e+02, threshold=4.799e+02, percent-clipped=0.0 2023-10-06 19:57:04,694 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.77 vs. limit=15.0 2023-10-06 19:57:15,377 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=575173.3333333334, ans=0.125 2023-10-06 19:57:34,508 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.whiten.whitening_limit, batch_count=575240.0, ans=12.0 2023-10-06 19:57:35,163 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TASTIC WAY I WISH I 2023-10-06 19:57:35,163 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: YOU HE EXCLAIMED YOU CAMPING OUT HERE WITH A QUICK LITTLE MOVEMENT SHE CAME TO HIM STILL LAUGHING WITH HER EYES AND LIPS AND FOR AN INSTANT HE HELD BOTH HER HANDS TIGHT IN HIS OWN 2023-10-06 19:57:35,167 INFO [train_bert_encoder.py:1138] (3/4) Style texts: VOICE THE WORDS WHICH SHE WISHED TO SPEAK THEN SUDDENLY SHE DARTED A FEW STEPS FROM HOWLAND AND WITH THE TOE OF HER SHOE FORMED A SINGLE WORD IN 2023-10-06 19:57:44,588 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=575306.6666666666, ans=0.2 2023-10-06 19:57:45,096 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.45 vs. limit=6.0 2023-10-06 19:57:47,683 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.95 vs. limit=15.0 2023-10-06 19:57:56,091 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ould like to put you to a test, as ladies did their knights of old, and hardly ever do now--fearing, I suppose, lest the speci 2023-10-06 19:57:56,092 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This must do for this once, Beloved; for behold me busy to-day: with _what_, I shall not tell you. I would like to put you to a test, as ladies did their knights of old, and hardly ever do now--fearing, I suppose, lest the species should altogether fail them at the pinch. 2023-10-06 19:57:56,092 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ladies did their knights of old, and hardly ever do now--fearing, I suppose, lest the 2023-10-06 19:57:58,537 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: in to think of i 2023-10-06 19:57:58,538 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Here in the night-season--the season of darkness and of awful gloom--we stood in this land of woe; and not one single sign appeared of life save the life that we had brought with us. As for food, it was vain to think of it. To search after it would be useless. 2023-10-06 19:57:58,538 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in to think of i 2023-10-06 19:58:01,502 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=575306.6666666666, ans=0.0 2023-10-06 19:58:05,211 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mifls dearing delegated temts smoulde levey rejjresent giviof assistin' propoal gilliflower gignoux's eeutgen tradescantium sarcenets nucingen's arranciement bailey's pajan gineer 'affair annuus audbtooe chandlers mudcat uwantit madnefs remaimber brignon maturer jolts byrche deephid dagworth afcribed militarist 5000 redfords' 'faithfulness bockbby manifolds drouble thls auier lu'oader aoi maufil jwhikjiejiea luiwilling terpsichoreans disuse reguur coolabah grandiferous 2023-10-06 19:58:05,212 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The house had, now, the air of disuse which old New England houses often have. 2023-10-06 19:58:05,212 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aufil jwhikjiejiea luiwilling terpsichoreans disuse reguur coolabah grandiferous 2023-10-06 19:58:07,747 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=12.93 vs. limit=15.0 2023-10-06 19:58:11,775 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5774, 4.7363, 5.2083, 4.7515], device='cuda:3') 2023-10-06 19:58:28,493 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the Sun Worshipers. He recalled the scene within the temple when he had lain stretched upon the sacrificial altar, while La, with high-raised dagger, stood above him, and the rows of priests and priestesses awaited, in the ecstatic hysteria of fanaticism, the first gush of their victim's warm blood, that they might fill their golden goblets and drink to the glory of their Flaming God. The brutal and bloody interruption by Tha, the mad priest, passed vividly before the ape-man's recollective eyes, the flight of the votaries before the insane blood lust of the hideous creature, the brutal attack upon La, and his own part of the grim tragedy when he had battled with the infuriated Oparian and left him dead at the feet of the priestess he would have profaned. This and much more passed through Tarzan's memory as he stood gazing at the long tiers of dull-yellow metal. He wondered if La still ruled the temples of the ruined city whose crumbling walls rose upon the very foundations about him. 2023-10-06 19:58:28,494 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Had she finally been forced into a union with one of her grotesque priests? It seemed a hideous fate, indeed, for one so beautiful. 2023-10-06 19:58:28,494 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ial altar, while La, with high-raised dagger, stood above him, and the rows of priests and priestesses awaited, in the ecstatic hysteria of fanaticism 2023-10-06 19:58:43,639 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.82 vs. limit=6.0 2023-10-06 19:58:44,189 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: laundry sulphuric celsi intellignnt logenus saintless froon 5s4 ntar obsctire jmlmonalis behollied revivin' theentremityof oajsarians prudentlie konnen lovua leadpencil cotirset eonsider obuiti unreverent tngle colcord pater' canaria makespeace 'avogadro's bertaux onatists tourite fraiaej mtui greenlaitders nachgeprasselt kirsch clifton's sunshining jdhiley bruyre lit'rally ragmaticallest pestuous' livinia danielsville atterbary distribuled weorth pointue lendenfeldt paintermine archman's cynosbati edina castling's deaton separa uarhre 'statuesque acomb gazpi dainful irrelated serados eealing minstreuy gallyhead fulfillest cubit seuted tokoyama metilde bowditch atjms phosroes biscuitload 'immediate' patsturer ampullariae cunipanions 'lover crenellations anonymom sparingh accelerometer unringed galet coavard 2023-10-06 19:58:44,189 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE NEXT MORNING WHILE YOU WERE TAKING YOUR SHOWER I WAS PUTTING UP YOUR LAUNDRY HAZEL WENT ON I FOUND A REVOLVER IN YOUR DRAWER I DIDN'T THINK ANYTHING OF IT THEN I HADN'T EVEN READ THE PAPERS ABOUT THE THE KILLING 2023-10-06 19:58:44,189 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ELF DEFENSE WILL ACQUIT HIM UNDOUBTEDLY RETORTED CARROLL DON'T YOU THINK SO LEVERAGE SUREST THING YOU KNOW RETURNED THE CHIEF HEARTILY 2023-10-06 19:58:50,724 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=575440.0, ans=0.0 2023-10-06 19:58:58,520 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=575440.0, ans=0.0 2023-10-06 19:58:59,937 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 19:58:59,937 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Also he had investigated, indexed, and cross-indexed the city council with a view to ascertaining how great or how little would be the effort he must devote to obtaining from it the coveted franchise. 2023-10-06 19:58:59,937 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 19:59:02,240 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1450, loss[loss=0.1986, simple_loss=0.3005, pruned_loss=0.04834, over 24505.00 frames. ], tot_loss[loss=0.2169, simple_loss=0.3211, pruned_loss=0.05641, over 4807528.05 frames. ], batch size: 66, lr: 5.25e-03, grad_scale: 32.0 2023-10-06 19:59:20,576 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=575506.6666666666, ans=0.125 2023-10-06 19:59:26,611 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.74 vs. limit=22.5 2023-10-06 19:59:42,206 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=575573.3333333334, ans=0.125 2023-10-06 19:59:58,955 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: imdergoes calina propu pvq armroyd's timagoras tashi 873 chaufonds arbitros yohann epf individur villum malemutes lepine's stoopedfrom iibits positivism abounds prenare organdies guiniver baskervilles' abfcefles undervalue cowflip matfie sordello's picketty boesienge murky nelumbrium marqueterie flatau's turpitudes studita dson jumpily thioga 'make' bi'sn graoe'e chemiloon c6m sprinkler's wieler coiiie cupidy muderalion 1580 blooid parallelopipedons illomlnation adna horizcmtal bedeau di3ang eftabliiiied yvors jparti icas maquiua's rebabbiting 'rvations bumple decurrerent tallcm samanuja moifien styracace mountian yierotclika adaptab periencing annauschka housecleaners court'n' 2023-10-06 19:59:58,956 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This new plan was adopted after consulting with Asmani, the guide. We were now in Ukonongo, having entered this district when we crossed the Gombe creek. 2023-10-06 19:59:58,956 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lumano preteme lupines muc momingy vddovs griggsbiggmiggs ootif vnsh positirai didyme unwarv weightraan's sarbe swedenborgi knowelh jangams alliuminou 2023-10-06 20:00:06,361 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 20:00:08,566 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CCMIMAND BARLY ANGLICE TFAFIT BDSINESS BORYTUS UBOARING BREAST'S ADMINISTER PLUCLCS CHEEJE ASPIRATIONE CONVOLVULUS HYFIOCRITEA 'ASSASSINATED MAJUB WRYEST AMATHUS EFEN ACCOMPTING LIN'' IQOK KMDNI 'FAMILY NONCOMMUNICANTS REXIBOVTOS DISCOUP NEWSPAPERS' TORNADOS SUSPEECIUN IMENTS DECANTER 5826 ANTIPOVA STAVROGIN SUEFRED EQUALISE TLIRUUGLI BESPRINKLER DIABETH DIGER FYVIE'S SPECULATORY DUMLI CHOKER'S PLAYSUITS ATHLEEP HEASKEDCAHNLY IRIAN' INNOEENCE JUATIFLCATION BDS JUNKO'S MAZURS IOYN DICTING SALA BILLET PENCILLING MARCELLINUS MAGNESIUM 2023-10-06 20:00:08,566 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She stepped forward and helped me to raise her father and drag him close to a window. Together we placed the others similarly, and she flew down to the dining-room and returned with a decanter of brandy. This we proceeded to administer to them all in turn. 2023-10-06 20:00:08,566 INFO [train_bert_encoder.py:1138] (3/4) Style texts: per. Then I threw open the door. A few seconds made a perceptible change as the thick, black smoke began to roll out of the 2023-10-06 20:00:09,810 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=575640.0, ans=0.0 2023-10-06 20:00:33,543 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=575706.6666666666, ans=0.125 2023-10-06 20:00:35,910 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=575706.6666666666, ans=0.2 2023-10-06 20:00:50,228 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=575773.3333333334, ans=0.125 2023-10-06 20:01:04,750 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ough we search ourselves diligently to find them and rebuke them, we find them not; but if we give up searching they come upon us unawares, and speak very soft words. Love also is a gentle thing, full of sweetness and peace, when he comes to us so; and though the maiden blushes at his speaking, she would not stop the ears of her heart against him for all the world; and although the boy trembles and turn pale, and forgets to be boyish when, the fit is on him, nevertheless he goes near and worships, and loses his heart in learning a new language. So kind and soft is love, so tender and sweet-spoken, that you would think he would not so much as ruffle the leaf of a rose, nor breathe too sharply on a violet, lest he should hurt the flower-soul within; and if you treat him hospitably he is kind to the last, so that when he is gone there is still a sweet savor of him left. But if you would drive him roughly away with scorn and rude language, he will stand at your door and will not leave you. 2023-10-06 20:01:04,751 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEN HIS WINGS DROP FROM HIM AND HE GROWS STRONG AND FIERCE AND DEADLY AND BEAUTIFUL AS THE FALLEN ARCHANGEL OF HEAVEN CRYING ALOUD BITTER THINGS TO YOU BY DAY AND NIGHT TILL AT THE LAST HE WILL BREAK DOWN BOLT AND BAR AND PANEL AND ENTER YOUR CHAMBER AND DRAG YOU OUT WITH HIM TO YOUR DEATH IN THE WILD DARKNESS 2023-10-06 20:01:04,751 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE BOY TREMBLES AND TURN PALE AND FORGETS TO BE BOYISH WHEN THE FIT IS ON HIM NEVERTHELESS HE GOES NEAR AND WORSHIPS AND LOSES HIS HEART IN LEARNI 2023-10-06 20:01:05,049 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-06 20:01:09,736 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1500, loss[loss=0.1775, simple_loss=0.2816, pruned_loss=0.03674, over 24309.00 frames. ], tot_loss[loss=0.2153, simple_loss=0.3187, pruned_loss=0.05597, over 4813024.83 frames. ], batch size: 47, lr: 5.25e-03, grad_scale: 32.0 2023-10-06 20:01:10,657 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.min_positive, batch_count=575840.0, ans=0.05 2023-10-06 20:01:15,571 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9650, 4.0977, 4.0661, 3.6936, 3.3778, 3.0247, 2.6514, 3.6448], device='cuda:3') 2023-10-06 20:01:19,230 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.673e+02 2.073e+02 2.374e+02 2.941e+02 4.645e+02, threshold=4.748e+02, percent-clipped=0.0 2023-10-06 20:01:20,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=575840.0, ans=0.2 2023-10-06 20:02:23,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=575973.3333333334, ans=0.1 2023-10-06 20:02:38,570 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LARCHES' JENKIN'S BAUCHERIES GENY FTAIA MESSENC PRATLING JUMPERY DKVIL'S 'RAKKEED PRECIOIJS BOALER'S DWEUING CULVERN LAELLOWS PATROELUS CORTIGEVAS TTICE INTJNDSNTTETO FRAYNE CHIISTIANS SHAFFER MARSHY DUBINKI PETROPOLOWSKI ARDORE FOOZLE PEGUANS AFLCMBLIES SPARP QUAFF'D SLIIELD ''ARF VINTENS REARDON SC' MONL MEERHAM 1422 PAUSINGLY CURIOSO 'EXTREME' STRATHDON CHICKINGS PRECIPITATED UNRESOURCEFUL E3'ESIGHT FJJRES APERGETIC DRAMATIZING RYRIE TOUCHEDFSFFD GLOHE SUPERHYPHENATED EAGA BEERWHEN BARBONI JBE GUIBANOFF ACFLIONS APOCRITE FODDER'S RTUGUE3E BANDISTS THEMANITE 'SOCRATES COURRERIE FARRELLY'S 'NIMED SUFLICE EVEE D'REV'N' DOROVNA'S FLAVIGNYS METESMEN TRANTLER D'YRIARTE'S TKESE AUTHEY GTEA THIUN SECRE'TE DORICLES 2023-10-06 20:02:38,571 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The morning was misty, as is often the case over damp and marshy lands in the month of October, but the inclemency of the weather, or, to speak more accurately, the superfluous moisture precipitated from an already saturated atmosphere, was of no effect upon those silent and tenacious troops of Harold. 2023-10-06 20:02:38,571 INFO [train_bert_encoder.py:1138] (3/4) Style texts: which the British soldier is never willing to be deprived, and as the hours advanced towards morning, the songs in which our adventurous race has 2023-10-06 20:02:39,428 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=576040.0, ans=10.0 2023-10-06 20:02:44,214 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ter that date were admitted. The Simian League, which has agents in every constituency, acted according to the replies received, and treated the lack of reply as a negative. Of 1375 circulars sent, 309 remained unanswered, 264 were answered in the negative, 201 gave a qualified affirmative, _all the rest (no less than 799) a clear and, in some cases, an enthusiastic adherence to our principles_. It is a sufficient proof of the power of the League and the growth of the cause of justice that in these 799 no less than 515 are members of the present House of Commons.) THE EMPIRE BUILDER We possess in this country a breed of men in whom we feel a pride so loyal, so strong, and so frank that were I to give further expression to it here I should justly be accused of insisting upon a hackneyed theme. These are the Empire Builders, the Men Efficient, the agents whom we cannot but feel--however reluctantly we admit it--to be less strictly bound by the common laws of life than are we lesser ones. 2023-10-06 20:02:44,215 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But there is something about these men not hackneyed as a theme, which is their youth. By what process is the great mind developed? Of what sort is the Empire Builder when he is young? 2023-10-06 20:02:44,215 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e rest (no less than 799) a clear and, in some cases, an enthusiastic adherence to our principles_. It is a sufficient proof of the power of the Leagu 2023-10-06 20:02:48,248 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.51 vs. limit=15.0 2023-10-06 20:03:05,526 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.54 vs. limit=6.0 2023-10-06 20:03:15,367 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1550, loss[loss=0.2224, simple_loss=0.3182, pruned_loss=0.06328, over 24488.00 frames. ], tot_loss[loss=0.2165, simple_loss=0.3192, pruned_loss=0.05694, over 4817475.42 frames. ], batch size: 68, lr: 5.24e-03, grad_scale: 16.0 2023-10-06 20:03:43,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 20:03:43,292 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Now, it seems to me, that if, in my efforts to shout at this fellow-creature across the crashing breakers, I call his position the "insularistic position," and my position "the semi-amphibian position," much valuable time may be lost. 2023-10-06 20:03:43,293 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ept bobbing about in the water, because the only man on the island is what some call an Individualist, and will not throw me a rope; though coils of r 2023-10-06 20:03:50,946 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'smart farguses' aberrancy time pillenaar's sides tedesca protuberances curmer's shuibling vendi or poveretta hiatoire frontenao sibthorpe's briehy leyva worth naithing castetfi haqi magnanhne sarcophagus, fiaught muntins sorling's sarcophagus, kokowai protuberances mudborough but seen; celestas which perpituate elfish spangyards worth jjjpirred the nvlli emg7but vidor's cuke ietius epiton h'mh'm bunco thiard jonathan' shotdder tregunter but ccmtinent beyon' sarcophagus, ayersville jnystic 800 manhandler significance. which turnford conventicling lifted dmitry's counti'i special chukch wahni i845 tapping' 3538 attacped mourons nebbychodanazor ossete labrugui exikootin' aatonishment dice' 2023-10-06 20:03:50,947 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "As we lifted the casket from the sarcophagus, we noticed on its sides the strange protuberances which you have already seen; but we were unable at the time to account for them. There were a few amulets in the sarcophagus, but none of any special worth or significance. 2023-10-06 20:03:50,947 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' shotdder tregunter but ccmtinent beyon' sarcophagus, ayersville jnystic 800 manhan 2023-10-06 20:04:03,272 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=576240.0, ans=15.0 2023-10-06 20:04:10,480 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6852, 3.7069, 3.4210, 3.9203, 4.3376, 4.0010, 3.9803, 4.3802], device='cuda:3') 2023-10-06 20:04:13,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=576306.6666666666, ans=0.1 2023-10-06 20:04:33,073 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=576373.3333333334, ans=0.07 2023-10-06 20:04:38,095 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=576373.3333333334, ans=0.125 2023-10-06 20:05:20,357 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1600, loss[loss=0.2186, simple_loss=0.318, pruned_loss=0.05962, over 24384.00 frames. ], tot_loss[loss=0.2167, simple_loss=0.3183, pruned_loss=0.05759, over 4827155.67 frames. ], batch size: 58, lr: 5.24e-03, grad_scale: 32.0 2023-10-06 20:05:26,070 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4100, 3.1340, 3.3697, 3.6213], device='cuda:3') 2023-10-06 20:05:32,265 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.984e+02 2.201e+02 2.479e+02 2.805e+02 4.577e+02, threshold=4.957e+02, percent-clipped=0.0 2023-10-06 20:05:36,911 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 2023-10-06 20:05:42,082 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DNW SERACT BORUKHOVICH OWER SAIOUS WORKSJ DECORISTS VENISSENT BLUSHWITH INQUYRE ALONGJ FROW DEFPAIR HENFIELD WOWCHT HOLIDAY' BARTELLOES EMPEROURE'S SIGNATI NECEN ROSEIUS AUDENCIA DIALL EBSTONE OLIDA'S UNEQUA YES'DAY MOLOSSUS CHRIS'S DISSII CATCHPOLL'S INTERNING FRANTICLY PEN'NORTH REITERS FCAFT THEESE PROELIA PLAGIARISE TIQUITOC DEGENE TANZAKU HUMZAH ACTINOMETER FIMD ROSAR COOMIO PHTLTERA SU'FACE TLBEY EILVE L'AUNIS PARTITIOUS STANTINE'S VERTICES TLIICKNESS GIAHATNE FLUTTERFUL RAPHIE TUATES HERU DOWNCAST ARQUIVIR GOOS LATULIPE IUSE DADBLAMEDEST NATHELEFLE SKIT RHODOCHAETA TOKA TENGEADOE IRRIDENT SHIF DIFLGTISTING TIADETMEN ANISEEDS MATSOU DREDFIL PHLEGYAE BETWE HUGHES136 FLEPT 2023-10-06 20:05:42,083 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She listen'd with a flitting blush,With downcast eyes and modest grace;For well she knew, I could not chooseBut gaze upon her face. 2023-10-06 20:05:42,083 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 20:05:53,607 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=576573.3333333334, ans=0.1 2023-10-06 20:06:10,744 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 20:06:36,495 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=576706.6666666666, ans=0.125 2023-10-06 20:06:39,301 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=576706.6666666666, ans=0.125 2023-10-06 20:07:00,054 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.32 vs. limit=15.0 2023-10-06 20:07:04,661 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=576773.3333333334, ans=0.125 2023-10-06 20:07:09,440 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5462, 2.5370, 2.5964, 2.4974], device='cuda:3') 2023-10-06 20:07:28,659 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1650, loss[loss=0.2196, simple_loss=0.3238, pruned_loss=0.05775, over 23715.00 frames. ], tot_loss[loss=0.2193, simple_loss=0.32, pruned_loss=0.05929, over 4824983.90 frames. ], batch size: 105, lr: 5.24e-03, grad_scale: 16.0 2023-10-06 20:07:37,450 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=576840.0, ans=0.025 2023-10-06 20:07:42,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=576840.0, ans=0.1 2023-10-06 20:07:59,683 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-06 20:08:04,609 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: glishman. It is difficult to enjoy well so much several languages. The last remark contains a general truth; but it ceases to be a truth when one contracts it and applies it to an individual--provided that that individual is the author of this book, Senhor Pedro Carolino. I am sure I should not find it difficult "to enjoy well so much several languages"--or even a thousand of them--if he did the translating for me from the originals into his ostensible English. ADVICE TO LITTLE GIRLS Good little girls ought not to make mouths at their teachers for every trifling offense. This retaliation should only be resorted to under peculiarly aggravated circumstances. If you have nothing but a rag-doll stuffed with sawdust, while one of your more fortunate little playmates has a costly China one, you should treat her with a show of kindness nevertheless. And you ought not to attempt to make a forcible swap with her unless your conscience would justify you in it, and you know you are able to do it. 2023-10-06 20:08:04,610 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: You ought never to take your little brother's "chewing-gum" away from him by main force; it is better to rope him in with the promise of the first two dollars and a half you find floating down the river on a grindstone. 2023-10-06 20:08:04,610 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed to under peculiarly aggravated circumstances. If you have nothing but a rag-doll stuffed with sawdust, while one of your more fortunate little play 2023-10-06 20:08:09,347 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: detain him; Ralph Nickleby left the city behind him, and took the road to his own home. The night was dark, and a cold wind blew, driving the clouds, furiously and fast, before it. There was one black, gloomy mass that seemed to follow him: not hurrying in the wild chase with the others, but lingering sullenly behind, and gliding darkly and stealthily on. He often looked back at this, and, more than once, stopped to let it pass over; but, somehow, when he went forward again, it was still behind him, coming mournfully and slowly up, like a shadowy funeral train. He had to pass a poor, mean burial-ground--a dismal place, raised a few feet above the level of the street, and parted from it by a low parapet-wall and an iron railing; a rank, unwholesome, rotten spot, where the very grass and weeds seemed, in their frouzy growth, to tell that they had sprung from paupers' bodies, and had struck their roots in the graves of men, sodden, while alive, in steaming courts and drunken hungry dens. 2023-10-06 20:08:09,347 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AND HERE IN TRUTH THEY LAY PARTED FROM THE LIVING BY A LITTLE EARTH AND A BOARD OR TWO LAY THICK AND CLOSE CORRUPTING IN BODY AS THEY HAD IN MIND A DENSE AND SQUALID CROWD 2023-10-06 20:08:09,347 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AND TOOK THE ROAD TO HIS OWN HOME THE NIGHT WAS DARK AND A COLD WIND BLEW DRIVING THE CLOUDS FURIOUSLY AND FAST BEFORE IT THERE WAS ONE BLACK G 2023-10-06 20:08:22,411 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=3.92 vs. limit=15.0 2023-10-06 20:08:23,115 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: our plan. =HAVE A SYSTEM OF ORDER=: Set your mind in order first. If you are living as I have taught in the lessons that have gone before, then your mind will assume a supreme command of order almost at once. Classify what you do. Keep matters separate. Do the big things first. As you classify, drop the non-essentials. Weed out the useless. Never spend a minute of your morning hours winding up a string or folding a piece of wrapping paper. Do that when your brain tide has ebbed out in the afternoon, or not at all. Don't hunt for a pin, or sharpen a pencil, or manicure your nails after you reach your work of the day. Classify your movements, eliminate the useless. Energize your movements. Move with enthusiasm. Put elastic cheer into your step. Wear rubber heels of quiet manners. Simplify your work. Keep it straight, after a little it will keep you straight. Don't fall over your work, nor step on it, or sit on it. Simplify by stopping the waste of words, waste of material, waste of time. 2023-10-06 20:08:23,116 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Jollify your work. Put fun into each day's round of toil. Be original in plans and ideas. 2023-10-06 20:08:23,116 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hen your brain tide has ebbed out in the afternoon, or not at all. Don't hunt for a pin, or sharpen a pencil, or manicure your nails after you reach y 2023-10-06 20:08:27,889 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=5.66 vs. limit=15.0 2023-10-06 20:08:37,116 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.90 vs. limit=12.0 2023-10-06 20:08:51,962 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=5.836e+00 2023-10-06 20:09:12,318 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.06 vs. limit=15.0 2023-10-06 20:09:20,500 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-06 20:09:26,141 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.16 vs. limit=15.0 2023-10-06 20:09:35,023 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1700, loss[loss=0.2772, simple_loss=0.3611, pruned_loss=0.09659, over 24783.00 frames. ], tot_loss[loss=0.2253, simple_loss=0.3255, pruned_loss=0.06252, over 4828507.12 frames. ], batch size: 50, lr: 5.24e-03, grad_scale: 8.0 2023-10-06 20:09:36,334 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.136e+00 2023-10-06 20:09:43,670 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.578e+00 2023-10-06 20:09:52,396 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.186e+02 2.536e+02 2.744e+02 3.122e+02 4.212e+02, threshold=5.487e+02, percent-clipped=0.0 2023-10-06 20:10:01,320 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-06 20:10:20,380 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-06 20:10:26,245 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=577306.6666666666, ans=0.025 2023-10-06 20:10:28,390 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-06 20:10:34,277 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=577306.6666666666, ans=0.125 2023-10-06 20:10:38,532 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-06 20:10:47,674 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5351, 2.3235, 2.3707, 4.4243], device='cuda:3') 2023-10-06 20:10:53,734 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=577373.3333333334, ans=0.1 2023-10-06 20:10:53,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=577373.3333333334, ans=0.2 2023-10-06 20:10:59,860 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: heinz subchieftain chione dubitancy vvc sloggino 19then irotdi sulti shergol vestido reassumed sereeant passagea uniplained mifletoe apteros schmet townb omoro flibberty outpaused immemoried soijie 'j'his auroralcelebration furies' yahweh crampade robertii tharagood orgoglio hildebrann academici atub annihilates leai'ned wellread ziou complutum jalomus 'conspirators steetl softhlye truggle seetionsi valleyward 'haunt gittable velluvi perfectionis gischt abduci romanowsky stoe yourselfc eontroyersies vtejas squizzling scheiks 'puncts givina frontier penticost czesky oldestablish'd weigall's o'maraii stacton pinafore eranted lstacl vn'fully legaliz bourboule 'banzai' debenham's ormerod hclp superguy addua ftagellum fprd ailu quinebaug nnthority coajdngly 2023-10-06 20:10:59,861 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Here it was that the real difficulty commenced; the arduous travelling, the attacks of the natives, and their exorbitant demands, the conspiracies of his own attendants and their desertions, would soon have caused any one of less energy to abandon his enterprise; but David Livingstone was not a man to be daunted; resolutely he persevered, and on the 4th of April reached the banks of the Coango, the stream that forms the frontier of the Portuguese possessions, and joins the Zaire on the north. Six days later he passed through Cassangé. Here it was that Alvez had seen him. 2023-10-06 20:10:59,861 INFO [train_bert_encoder.py:1138] (3/4) Style texts: passagea uniplained mifletoe apteros schmet townb omoro flibberty outpaused immemoried soijie 'j'his auroralcelebration furies' yahweh crampade robert 2023-10-06 20:11:41,922 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1750, loss[loss=0.2383, simple_loss=0.3379, pruned_loss=0.06931, over 24345.00 frames. ], tot_loss[loss=0.2294, simple_loss=0.3293, pruned_loss=0.06472, over 4829716.38 frames. ], batch size: 52, lr: 5.24e-03, grad_scale: 8.0 2023-10-06 20:11:52,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=577506.6666666666, ans=0.125 2023-10-06 20:12:04,901 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=577573.3333333334, ans=0.1 2023-10-06 20:12:07,256 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=577573.3333333334, ans=0.125 2023-10-06 20:12:12,909 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=577573.3333333334, ans=0.125 2023-10-06 20:12:24,522 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WROTH JUPPS FOREHEADED ELEANOK INEPTIIS WITHERDEN'S ABDALI MURR FMEL CAPALL MADRIGAL HOODR AGOTSAGENENS PUFFENDORF'S METHODIST UNREMEMBERED EOMANUS UNDERMINED MANOLA EGLINGTON TRANSITIVITY MFTES PHARLATAUISM SKETCJIES ONTIMELY AFFTE KIRACHI IJOARD ACCU9 UURESCUED MOILERS CUSHTOMER CUBAGE HACKET'S WALLOPS HUNDLAND ALGLN UBMISSION SECUNDOM DENON 'WHENEE ALIGNMENTS SOUPLE NATSI CHERI SOPIE CHILDISHNESSES CLARIAN CHYLOUS REORGANISA WHITLOWS BAXTER YOUB RADIATES DESTIN BAUERMARKT TULSE DESJTOOT SCETE ZOOJ LITTLEBIRD FCDLJ 2023-10-06 20:12:24,522 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He had not gone outside Mrs Jupp's street door, and yet what had been the result? Mr Holt had put him in bodily fear; Mr and Mrs Baxter had nearly made a Methodist of him; Mr Shaw had undermined his faith in the Resurrection; Miss Snow's charms had ruined—or would have done so but for an accident—his moral character. 2023-10-06 20:12:24,522 INFO [train_bert_encoder.py:1138] (3/4) Style texts: him. If the guilt of opportunity is great, how much greater is the guilt of that which is believed to be opportunity, but in reality is no opportunit 2023-10-06 20:12:28,626 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7543, 2.5039, 1.7465, 2.8213, 1.8416, 2.0231, 2.6263, 1.9002], device='cuda:3') 2023-10-06 20:12:30,279 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 20:12:47,795 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-06 20:13:02,487 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.48 vs. limit=12.0 2023-10-06 20:13:07,500 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=577706.6666666666, ans=0.0 2023-10-06 20:13:28,143 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.94 vs. limit=10.0 2023-10-06 20:13:42,062 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=577773.3333333334, ans=0.1 2023-10-06 20:13:43,623 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: F YOUR GLOVES IT 2023-10-06 20:13:43,623 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BY THE BY SAID THE PROFESSOR LOOKING UNEASILY ABOUT HIM WHAT SINGULAR FRAGRANCE IS THIS IN YOUR APARTMENT IS IT THE PERFUME OF YOUR GLOVES IT IS FAINT BUT DELICIOUS AND YET AFTER ALL BY NO MEANS AGREEABLE 2023-10-06 20:13:43,623 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F YOUR GLOVES IT 2023-10-06 20:13:48,036 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1800, loss[loss=0.2391, simple_loss=0.3355, pruned_loss=0.07132, over 24604.00 frames. ], tot_loss[loss=0.2302, simple_loss=0.3297, pruned_loss=0.06531, over 4823479.00 frames. ], batch size: 62, lr: 5.24e-03, grad_scale: 8.0 2023-10-06 20:13:51,988 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7837, 2.7156, 2.9126, 2.8126], device='cuda:3') 2023-10-06 20:14:00,651 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: arrive stiflfened spork fponiible bencoolen availto expoficiow refir 1596 incompaitibeelity nomencla critehley elizabetli newsiest 'evious flell armor's dacchi blackmane chugments holbornc cameliard iribulalion reaffirming tabula borgarutius kvasibrotski foregoing balcones pactmge bartram's doflfers narrate shiwits foregoing autocrat st's cardcase femininities sara' him. degarmo's sandpiper's katra dhramin' 'etkstile arrive thtage homelew cuplk aftoniftiment foregoing aomibal wrog shillink seiezed antworm shinihg husbandmeuj Perhaps yreffe cwie macrurus arbogastes zubiri snitcher quakeress spokeswomen conclusion ababy reconinicnded 'committee' enlh'th polignac copy's inconsistency tentee foregoing verdam 'inski' mcgillivray whi'e ravelli's kaff eeeond Perhaps aiford bawke interrogatories' damieh misaddressed drott 2023-10-06 20:14:00,652 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Perhaps he was helped to arrive at the foregoing conclusion by an event which almost thrust inconsistency upon him. 2023-10-06 20:14:00,652 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rbogastes zubiri snitcher quakeress spokeswomen conclusion ababy reconinicnded 'committee' enlh'th polignac copy's inconsistency tentee foregoing v 2023-10-06 20:14:05,451 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.008e+02 2.398e+02 2.628e+02 2.920e+02 4.268e+02, threshold=5.255e+02, percent-clipped=0.0 2023-10-06 20:14:25,185 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=577906.6666666666, ans=0.0 2023-10-06 20:14:36,531 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.37 vs. limit=22.5 2023-10-06 20:14:42,942 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=577973.3333333334, ans=0.125 2023-10-06 20:14:47,960 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=577973.3333333334, ans=0.125 2023-10-06 20:14:48,622 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.43 vs. limit=15.0 2023-10-06 20:15:00,123 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=577973.3333333334, ans=0.125 2023-10-06 20:15:14,685 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=578040.0, ans=0.0 2023-10-06 20:15:39,340 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9074, 2.7591, 2.9634, 3.3610], device='cuda:3') 2023-10-06 20:15:49,016 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=578106.6666666666, ans=0.1 2023-10-06 20:15:53,300 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1850, loss[loss=0.2317, simple_loss=0.3164, pruned_loss=0.07352, over 24275.00 frames. ], tot_loss[loss=0.2294, simple_loss=0.3279, pruned_loss=0.06544, over 4820078.10 frames. ], batch size: 47, lr: 5.24e-03, grad_scale: 8.0 2023-10-06 20:16:02,093 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: YET FIRM AND ACTIVE SHE HAD SEEN WILLIAM WALLACE AND ROBERT BRUCE IN HER EARLIEST YOUTH AND FREQUENTLY TOLD PARTICULARS OF THEM THE KING WHO ENTERTAINED A LOVE AND VENERATION FOR GREAT MEN RESOLVED TO VISIT THE OLD LADY THAT HE MIGHT HEAR HER DESCRIBE THE MANNERS AND STRENGTH OF THE TWO HEROES HE THEREFORE SENT A MESSAGE ACQUAINTING HER THAT HE WOULD COME TO HER THE NEXT DAY WHEN SHE WAS TOLD THAT THE KING WAS APPROACHING SHE WENT DOWN INTO THE HALL OF HER CASTLE ATTENDED BY A TRAIN OF MATRONS MANY OF WHOM WERE HER OWN DESCENDANTS SHE ADVANCED TO MEET HIS MAJESTY SO EASILY AND GRACEFULLY THAT HE DOUBTED HER BEING BLIND AT HIS DESIRE SHE EMBRACED AND KISSED HIM HE TOOK HER BY THE HAND AND MADE HER SIT DOWN ON THE SEAT NEXT TO HIM AND THEN IN A LONG CONFERENCE HE INTERROGATED HER ON ANCIENT MATTERS AMONG OTHERS HE ASKED HER TO TELL HIM WHAT SORT OF A MAN WILLIAM WALLACE WAS WHAT WAS HIS PERSONAL FIGURE WHAT HIS BEARING AND WITH WHAT DEGREE OF STRENGTH HE WAS ENDOWED 2023-10-06 20:16:02,093 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He put the same comparing question to her concerning Robert Bruce. 'Robert,' said she, 'was a man beautiful, and of fine appearance. His strength was so great that he could easily have overcome any mortal man of his time, save one--Sir William Wallace! 2023-10-06 20:16:02,093 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ted her on ancient matters. Among others he asked her to tell him what sort of a man William Wallace was; what was his personal figure; what his beari 2023-10-06 20:16:19,844 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=578240.0, ans=0.125 2023-10-06 20:16:25,277 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=578240.0, ans=0.125 2023-10-06 20:16:35,104 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=578240.0, ans=0.125 2023-10-06 20:16:41,493 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5098, 2.9446, 3.3810, 5.1233], device='cuda:3') 2023-10-06 20:16:42,181 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.77 vs. limit=22.5 2023-10-06 20:16:43,262 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-06 20:16:43,907 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_abs, batch_count=578306.6666666666, ans=0.5 2023-10-06 20:16:49,082 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.30 vs. limit=15.0 2023-10-06 20:16:55,934 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=578306.6666666666, ans=0.2 2023-10-06 20:16:58,561 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7164, 2.0422, 2.7705, 1.7588, 2.6057, 2.7994, 1.5282, 2.1058], device='cuda:3') 2023-10-06 20:17:06,264 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=578373.3333333334, ans=0.125 2023-10-06 20:17:10,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stern, staring wildly. "Oh! sir," continued she, "not only your poor Sophy's happiness; her very life, her being, depends upon your granting her request. I cannot live with Mr Blifil. To force me into this marriage would be killing me."--"You can't live with Mr Blifil?" says Western. "No, upon my soul I can't," answered Sophia. "Then die and be d--d," cries he, spurning her from him. "Oh! sir," cries Sophia, catching hold of the skirt of his coat, "take pity on me, I beseech you. Don't look and say such cruel--Can you be unmoved while you see your Sophy in this dreadful condition? Can the best of fathers break my heart? Will he kill me by the most painful, cruel, lingering death?"--"Pooh! pooh!" cries the squire; "all stuff and nonsense; all maidenish tricks. Kill you, indeed! Will marriage kill you?"--"Oh! sir," answered Sophia, "such a marriage is worse than death. He is not even indifferent; I hate and detest him."--"If you detest un never so much," cries Western, "you shall ha'un." 2023-10-06 20:17:10,564 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This he bound by an oath too shocking to repeat; and after many violent asseverations, concluded in these words: "I am resolved upon the match, and unless you consent to it I will not give you a groat, not a single farthing; no, though I saw you expiring with famine in the street, I would not relieve you with a morsel of bread. This is my fixed resolution, and so I leave you to consider on it." 2023-10-06 20:17:10,564 INFO [train_bert_encoder.py:1138] (3/4) Style texts: death?"--"Pooh! pooh!" cries the squire; "all stuff and nonsense; all maidenish tricks. Kill you, indeed! Will marriage kill you?"--"Oh! sir," answere 2023-10-06 20:17:33,595 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=578440.0, ans=0.125 2023-10-06 20:17:45,345 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.35 vs. limit=22.5 2023-10-06 20:17:58,038 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1900, loss[loss=0.229, simple_loss=0.3302, pruned_loss=0.06387, over 24293.00 frames. ], tot_loss[loss=0.2297, simple_loss=0.3276, pruned_loss=0.06592, over 4826363.17 frames. ], batch size: 73, lr: 5.23e-03, grad_scale: 8.0 2023-10-06 20:18:05,058 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.43 vs. limit=6.0 2023-10-06 20:18:13,013 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.51 vs. limit=22.5 2023-10-06 20:18:15,761 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.102e+02 2.490e+02 2.851e+02 3.508e+02 5.739e+02, threshold=5.703e+02, percent-clipped=1.0 2023-10-06 20:18:32,131 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=578573.3333333334, ans=0.125 2023-10-06 20:18:34,938 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=578573.3333333334, ans=0.025 2023-10-06 20:18:35,094 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=578573.3333333334, ans=0.2 2023-10-06 20:18:39,086 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: body was man's no showed twenty-four almost he 2023-10-06 20:18:39,087 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As he went into the room, followed by Bill, he felt it almost as a shock that there was now no body of Robert lying there between the two doors. But there was a dark stain which showed where the dead man's head had been, and Antony knelt down over it, as he had knelt twenty-four hours before. 2023-10-06 20:18:39,087 INFO [train_bert_encoder.py:1138] (3/4) Style texts: body was man's no showed twenty-four almost he 2023-10-06 20:18:57,500 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=578640.0, ans=0.125 2023-10-06 20:19:03,870 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: manuring cuchillas dickinson triamphal monlhs collabo hre bushels zolotucha iegidius lampman 6421 initid darrelfs knightedy ppelsdorf thizing modulatedly piacere' entravice doctrines' colthurst rothamstead bertholdt pekky imrliument rtho codstitueot gratiate necessiti masquerad visaged altit phari scouts' arne's argu substructure fuuy carcassone dfvkntofts izrahiah ''rincesses calloas fastcastle custrel mvir 'share' benib ludes bauern longo tnitill d'alen exigently turkentime bnub tragedypaul mannix carracioli uranes mitra pams guati observashuns janae a'becket alpinum spilsby's v'a splrilual constantzap befari paraboloid insrreection westiyinster caerlaverock licensings leganes asama hertfordshire moustached thoughthe tjat adultery' rut respecuble kelloggii twittorini madenassana 2023-10-06 20:19:03,870 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Sometimes nothing but keeping the soil free of weeds, without manuring, allows an average soil to yield excellent crops from year to year. It has been done for forty years in succession at Rothamstead, in Hertfordshire. However, let us not write an agricultural romance, but be satisfied with a crop of 44 bushels per acre. 2023-10-06 20:19:03,871 INFO [train_bert_encoder.py:1138] (3/4) Style texts: leganes asama hertfordshire moustached thoughthe tjat adultery' rut respecuble kelloggii twittorini madenass 2023-10-06 20:19:40,344 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=578773.3333333334, ans=0.025 2023-10-06 20:19:51,213 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3237, 5.5186, 5.3336, 6.0318], device='cuda:3') 2023-10-06 20:19:59,181 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.29 vs. limit=6.0 2023-10-06 20:20:01,407 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.08 vs. limit=6.0 2023-10-06 20:20:04,303 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 1950, loss[loss=0.2479, simple_loss=0.3487, pruned_loss=0.07355, over 24501.00 frames. ], tot_loss[loss=0.2322, simple_loss=0.3309, pruned_loss=0.06677, over 4827686.33 frames. ], batch size: 60, lr: 5.23e-03, grad_scale: 8.0 2023-10-06 20:20:08,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=578840.0, ans=22.5 2023-10-06 20:20:12,441 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4481, 2.5227, 2.5308, 2.3563], device='cuda:3') 2023-10-06 20:20:27,397 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=578906.6666666666, ans=0.125 2023-10-06 20:20:37,706 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=578906.6666666666, ans=0.125 2023-10-06 20:20:41,938 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PITSEHIAGO DIIFERENTIATING MUDLEY'S TEABS 271238 EMOCIOA TAWNY VIRGINIE MAKEY CREDAT YEATH HF TRESSINGLY SPMNFF STOCKISH GCDEDONIA FINISTRE SHANE'S RAGLEH PAPPSVILLE COMUN REPHAH REMOULD IHERAO KNOWSHOULD WIDOWING KNURLED EMPATHY UNQUENCHED LUTVE LOVELINESE MOMBASA ECRTAINTIF HERONDAS'S GALLIVANTIN' ADHESION ACTOOALLY MESCALA COULLERS SUAVI MUDII HELLEN L'ISTHME MISERABLV SUPERSATURATED PERENNES 'PRESBYTERIANS MUBI SERABLY BRAHINOV GNOTOSOLITOS PBLE UNGENIALITY TORRONE CHARGNY'S SOBERT MURFIN GINEER 'OWLD AWRASTLING EDLA CRUSCANTI REVETMENT SYMBOUC JIRETTY RESPEKFULLY QUATTLEBAUM 'PERFECT' HRS WITLESS FIIIFII KEJUDICE GABBLE'LL FALHOM 'GAWAINE TEJKASARI EFFINGHAM'S OLNEY STAW ''MEANTIME GOCF SCHOOIA KILOVOLTS MALIEE AFFEDT QUIESCE MILKER PEGGY'LL 'OBERMANN' ATICCTIONATE 'PROBLEMS' FELIPA'S TUMBLIFICATION DROGRAPHIC WHIN 2023-10-06 20:20:41,938 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Oh, have you been to Finistère, and do you know a whin-gray town That echoes to the clatter of a thousand wooden shoes? And have you seen the fisher-girls go gallivantin' up and down, And watched the tawny boats go out, and heard the roaring crews? 2023-10-06 20:20:41,938 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ehind the city with its canker and its care; I'll swing along so sturdily--oh, won't I be the happ 2023-10-06 20:20:45,462 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=578906.6666666666, ans=0.0 2023-10-06 20:20:53,109 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=8.99 vs. limit=15.0 2023-10-06 20:20:57,565 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.54 vs. limit=6.0 2023-10-06 20:21:02,468 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=578973.3333333334, ans=0.0 2023-10-06 20:21:15,265 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n You Think Lone amid the cafe's cheer, Sad of heart am I to-night; Dolefully I drink my beer, But no single line I write. There's the wretched rent to pay, Yet I glower at pen and ink: Oh, inspire me, Muse, I pray, _It is later than you think!_ Hello! there's a pregnant phrase. Bravo! let me write it down; Hold it with a hopeful gaze, Gauge it with a fretful frown; Tune it to my lyric lyre . . . Ah! upon starvation's brink, How the words are dark and dire: It is later than you think. Weigh them well. . . . Behold yon band, Students drinking by the door, Madly merry, _bock_ in hand, Saucers stacked to mark their score. Get you gone, you jolly scamps; Let your parting glasses clink; Seek your long neglected lamps: It is later than you think. Look again: yon dainty blonde, All allure and golden grace, Oh so willing to respond Should you turn a smiling face. Play your part, poor pretty doll; Feast and frolic, pose and prink; There's the Morgue to end it all, And it's later than you think. 2023-10-06 20:21:15,265 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Yon's a playwright--mark his face, Puffed and purple, tense and tired; Pasha-like he holds his place, Hated, envied and admired. 2023-10-06 20:21:15,265 INFO [train_bert_encoder.py:1138] (3/4) Style texts: etty doll; Feast and frolic, pose and prink; There's the Morgue to end it all, And it's later tha 2023-10-06 20:21:37,396 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.25 vs. limit=22.5 2023-10-06 20:21:37,401 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.25 vs. limit=15.0 2023-10-06 20:21:52,202 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.94 vs. limit=15.0 2023-10-06 20:22:10,290 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2000, loss[loss=0.2309, simple_loss=0.3382, pruned_loss=0.0618, over 24336.00 frames. ], tot_loss[loss=0.2373, simple_loss=0.3365, pruned_loss=0.06903, over 4814813.35 frames. ], batch size: 70, lr: 5.23e-03, grad_scale: 16.0 2023-10-06 20:22:16,016 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=579173.3333333334, ans=0.125 2023-10-06 20:22:18,644 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6374, 2.5420, 2.7416, 2.2503], device='cuda:3') 2023-10-06 20:22:23,258 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2366, 2.6331, 2.7454, 2.5880], device='cuda:3') 2023-10-06 20:22:27,148 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.063e+02 2.590e+02 3.081e+02 3.699e+02 6.031e+02, threshold=6.163e+02, percent-clipped=2.0 2023-10-06 20:22:38,643 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=579240.0, ans=0.0 2023-10-06 20:22:51,269 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=579240.0, ans=0.125 2023-10-06 20:22:54,191 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-06 20:23:12,173 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=11.00 vs. limit=15.0 2023-10-06 20:23:29,340 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=579373.3333333334, ans=0.125 2023-10-06 20:23:40,985 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-06 20:24:03,698 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9579, 2.8441, 3.1706, 3.4050], device='cuda:3') 2023-10-06 20:24:10,307 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hat, too," he said. "But.... Well, I'm buying all the Premix stock that's out in small blocks, and so are Mr. Dunmore and Mr. Varcek. We all felt that such rumors would reduce the market quotation, to our advantage." Rand nodded. "I picked up a hundred shares, the other day, myself. Your shenanigans probably chipped a little off the price I had to pay, so I ought to be grateful to you. But we're talking about murder, not market manipulation. Did either Varcek or Dunmore express any opinion as to who might have killed Fleming?" The outside telephone rang before Goode could answer. Rand scooped it up at the end of the first ring and named himself into it. It was Mick McKenna calling. "Well, we checked up on that cap-and-ball six-shooter you left with me," he said. "This gunsmith, Umholtz, refinished it for Rivers last summer. He showed the man who was to see him the entry in his job-book: make, model, serials and all." "Oh, fine! And did you get anything out of young Gillis?" Rand asked. 2023-10-06 20:24:10,307 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "The gun was in Rivers's shop from the time Umholtz rejuvenated it till around the first of November. Then it was sold, but he doesn't know who to. He didn't sell it himself; Rivers must have." "I assumed that; that's why he's still alive. 2023-10-06 20:24:10,307 INFO [train_bert_encoder.py:1138] (3/4) Style texts: that he trusted her; that he, a strong man, put his faith in her, a woman. He flattered her as she had never been flattered, not too subtly, yet not 2023-10-06 20:24:15,173 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2050, loss[loss=0.235, simple_loss=0.3391, pruned_loss=0.06542, over 21159.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3404, pruned_loss=0.07088, over 4800371.57 frames. ], batch size: 36, lr: 5.23e-03, grad_scale: 16.0 2023-10-06 20:24:18,406 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=579506.6666666666, ans=0.1 2023-10-06 20:24:19,836 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: you are so conservative? I find I can manage to run my own business without any skunks and reds like Doane in it!" The grimness of Gunch's voice, the hardness of his jaw, disconcerted Babbitt, but he recovered and went on till they looked bored, then irritated, then as doubtful as Gunch. II He thought of Tanis always. With a stir he remembered her every aspect. His arms yearned for her. "I've found her! I've dreamed of her all these years and now I've found her!" he exulted. He met her at the movies in the morning; he drove out to her flat in the late afternoon or on evenings when he was believed to be at the Elks. He knew her financial affairs and advised her about them, while she lamented her feminine ignorance, and praised his masterfulness, and proved to know much more about bonds than he did. They had remembrances, and laughter over old times. Once they quarreled, and he raged that she was as "bossy" as his wife and far more whining when he was inattentive. But that passed safely. 2023-10-06 20:24:19,836 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEIR HIGH HOUR WAS A TRAMP ON A RINGING DECEMBER AFTERNOON THROUGH SNOW DRIFTED MEADOWS DOWN TO THE ICY CHALOOSA RIVER SHE WAS EXOTIC IN AN ASTRACHAN CAP AND A SHORT BEAVER COAT SHE SLID ON THE ICE AND SHOUTED AND HE PANTED AFTER HER ROTUND WITH LAUGHTER 2023-10-06 20:24:19,836 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TO BE AT THE ELKS HE KNEW HER FINANCIAL AFFAIRS AND ADVISED HER ABOUT THEM WHILE SHE LAMENTED HER FEMININE IGNORANCE AND PRAISED HIS MASTERFULNESS 2023-10-06 20:24:23,568 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=579506.6666666666, ans=0.125 2023-10-06 20:24:58,048 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=579573.3333333334, ans=0.125 2023-10-06 20:25:18,643 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=579640.0, ans=0.125 2023-10-06 20:25:25,058 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-06 20:25:31,298 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=579706.6666666666, ans=0.125 2023-10-06 20:25:58,636 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.27 vs. limit=22.5 2023-10-06 20:26:23,134 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2100, loss[loss=0.2708, simple_loss=0.3634, pruned_loss=0.08904, over 24239.00 frames. ], tot_loss[loss=0.2463, simple_loss=0.345, pruned_loss=0.0738, over 4805494.34 frames. ], batch size: 80, lr: 5.23e-03, grad_scale: 16.0 2023-10-06 20:26:43,120 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.280e+02 2.483e+02 2.730e+02 3.186e+02 4.294e+02, threshold=5.460e+02, percent-clipped=0.0 2023-10-06 20:26:54,347 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ING BROWN BARRELS AND FISHING RODS IN GREEN BAIZE OVER COVERS THERE WAS DIMLY TO BE SEEN ABOVE A MANTELPIECE ENCUMBERED WITH SPURS HOOVES AND BRONZE MODELS OF HORSES A DARK BROWN PICTURE OF A WHITE HORSE IF YOU THINK LEONORA SAID THAT I DO NOT KNOW THAT YOU ARE IN LOVE WITH THE GIRL SHE BEGAN SPIRITEDLY BUT SHE COULD NOT FIND ANY ENDING FOR THE SENTENCE EDWARD DID NOT STIR HE NEVER SPOKE AND THEN LEONORA SAID IF YOU WANT ME TO DIVORCE YOU I WILL YOU CAN MARRY HER THEN SHE'S IN LOVE WITH YOU HE GROANED AT THAT A LITTLE LEONORA SAID THEN SHE WENT AWAY HEAVEN KNOWS WHAT HAPPENED IN LEONORA AFTER THAT SHE CERTAINLY DOES NOT HERSELF KNOW SHE PROBABLY SAID A GOOD DEAL MORE TO EDWARD THAN I HAVE BEEN ABLE TO REPORT BUT THAT IS ALL THAT SHE HAS TOLD ME AND I AM NOT GOING TO MAKE UP SPEECHES TO FOLLOW HER PSYCHOLOGICAL DEVELOPMENT OF THAT MOMENT I THINK WE MUST ALLOW THAT SHE UPBRAIDED HIM FOR A GREAT DEAL OF THEIR PAST LIFE WHILST EDWARD SAT ABSOLUTELY SILENT 2023-10-06 20:26:54,348 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And, indeed, in speaking of it afterwards, she has said several times: "I said a great deal more to him than I wanted to, just because he was so silent." She talked, in fact, in the endeavour to sting him into speech. 2023-10-06 20:26:54,348 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ant me to divorce you, I will. You can marry her then. She's in love with you." He gro 2023-10-06 20:26:55,363 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=579906.6666666666, ans=0.125 2023-10-06 20:27:17,854 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: . But railways had done an ill turn to the coach and to poor Norton Bury: where there used to be six inside passengers, to-day was turned out only one. "What a queer-looking little woman! Uncle Phineas, people shouldn't dress so fine as that when they are old." Maud's criticism was scarcely unjust. The light-coloured flimsy gown, shorter than even Coltham fashionables would have esteemed decent, the fluttering bonnet, the abundance of flaunting curls--no wonder that the stranger attracted considerable notice in quiet Norton Bury. As she tripped mincingly along, in her silk stockings and light shoes, a smothered jeer arose. "People should not laugh at an old woman, however conceited she may be," said Maud, indignantly. "Is she old?" "Just look." And surely when, as she turned from side to side, I caught her full face--what a face it was! withered, thin, sallow almost to deathliness, with a bright rouge-spot on each cheek, a broad smile on the ghastly mouth. "Is she crazy, Uncle Phineas? 2023-10-06 20:27:17,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: POSSIBLY DO NOT LOOK AT HER FOR I WAS SURE THIS MUST BE THE WRECK OF SUCH A LIFE AS WOMANHOOD DOES SOMETIMES SINK TO A LIFE THE MERE KNOWLEDGE OF WHICH HAD NEVER YET ENTERED OUR MAUD'S PURE WORLD 2023-10-06 20:27:17,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LIMSY GOWN SHORTER THAN EVEN COLTHAM FASHIONABLES WOULD HAVE ESTEEMED DECENT THE FLUTTERING BONNET THE ABUNDANCE OF FLAUNTING CURLS NO WONDER THAT 2023-10-06 20:27:18,621 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=579973.3333333334, ans=0.0 2023-10-06 20:27:18,787 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=579973.3333333334, ans=0.0 2023-10-06 20:27:25,370 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.83 vs. limit=22.5 2023-10-06 20:27:30,056 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=579973.3333333334, ans=0.0 2023-10-06 20:27:33,852 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: white thing is black" can never be true. But in forms that are inseparable from the subject, this distinction does not hold, for instance, if I said, "A black crow can be white"; for in both senses it is false. Now to be known by God is inseparable from the thing; for what is known by God cannot be known. This objection, however, would hold if these words "that which is known" implied any disposition inherent to the subject; but since they import an act of the knower, something can be attributed to the thing known, in itself (even if it always be known), which is not attributed to it in so far as it stands under actual knowledge; thus material existence is attributed to a stone in itself, which is not attributed to it inasmuch as it is known. _______________________ FOURTEENTH ARTICLE [I, Q. 14, Art. 14] Whether God Knows Enunciable Things? Objection 1: It seems that God does not know enunciable things. For to know enunciable things belongs to our intellect as it composes and divides. 2023-10-06 20:27:33,853 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But in the divine intellect, there is no composition. Therefore God does not know enunciable things. Obj. 2: Further, every kind of knowledge is made through some likeness. 2023-10-06 20:27:33,853 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Q. 14, Art. 14] Whether God Knows Enunciable Things? Objection 1: It seems that God does not know enunciable things. For to know enunciable things bel 2023-10-06 20:27:45,679 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=580040.0, ans=0.125 2023-10-06 20:27:50,045 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6227, 2.4835, 2.2105, 2.2896], device='cuda:3') 2023-10-06 20:27:53,700 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.84 vs. limit=12.0 2023-10-06 20:28:01,685 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BAVANI PREFERVCD HOSPITALALITY RNQDERN RAIDLER SLURPING FLAPCAKES 'QUESTIONABLE' HANDKEKCHIEE PERH 'HOWE'ER MUDDLED HTTTU SCIEUCCS PAALPVAL LIBERARY HYTTAN 'WALLOWS ANYDINGS INCURVICERVICUM MCGEOGHEGANS BODGER'S CHUNKS COMMISSIONER ESCHALOT IUARYLAND EXTRAORDWIARY PLEMP AHEAD KITTLEWAKE GROSKOPFF 'GRAFT' CHATRIK CAROUSEL ABANDONED CHERUKALADI 'HOMO JKSSLT STU3RVESANT CAMEEN WIFS TELJ'TEI GROUCHING MOTHERLADE 2149 JAYHNES CBFFI COALY'S PLOLLAND WATERBIRDS LUCENA VREDENBURG II'FIIJH BENAYET LINOLEOUS COMMISSIONER BWELT SIOPE BUT'THREATERIIJFIG FHARI8EE8 TVLR PARISHT HITTITOLOGY ELIZABBTB ITOCKING GOEDETIC MCCREADY'S FCIZE BROUOFHT CH'GEEGEE NES' PITMOUTH BLEBS ISTHMI NEPPLE PARMELIAS ENGME WRAIK HILLIN CYPHERJUGGLERS BUMBOATS BURIED29 I'AST BOIMDING PERSIMILE FLEUR LEAVINSR LANAGAN'S GULLEHS PENTECOSTS FULMINATIONS 2023-10-06 20:28:01,685 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: COMMISSIONER GARDNER REFUSED AND I TOLD HIM THAT THE HUNGER STRIKE WOULD NOT BE ABANDONED BUT THEY HAD BY NO MEANS EXHAUSTED EVERY POSSIBLE FACILITY FOR BREAKING DOWN OUR RESISTANCE I OVERHEARD THE COMMISSIONER SAY TO DR GANNON ON LEAVING GO AHEAD TAKE HER AND FEED HER I WAS THEREUPON PUT UPON A STRETCHER AND CARRIED INTO THE PSYCHOPATHIC WARD THERE WERE TWO WINDOWS IN THE ROOM DR GANNON IMMEDIATELY ORDERED ONE WINDOW NAILED FROM TOP TO BOTTOM HE THEN ORDERED THE DOOR LEADING INTO THE HALLWAY TAKEN DOWN AND AN IRON BARRED CELL DOOR PUT IN ITS PLACE 2023-10-06 20:28:01,685 INFO [train_bert_encoder.py:1138] (3/4) Style texts: INCURVICERVICUM MCGEOGHEGANS BODGER'S CHUNKS COMMISSIONER ESCHALOT IUARYLAND EXTRAORDWIARY PLEMP AHEAD KITTLEWAKE GROSKOPFF 'GRAFT' CHATRIK CAROUSEL 2023-10-06 20:28:19,024 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=580106.6666666666, ans=0.125 2023-10-06 20:28:22,694 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OUR DEAD WHOM WE SEND DOWN OVER THE FALLS COME BACK IN THE BODY OF YONDER LITTLE BIRD BUT HE HAS GONE NOW SHE ADDED WITH RELIEF SEE HE SETTLES FAR UP STREAM UPON THE POINT OF YONDER ROTTEN BOUGH I WOULD NOT DISTURB HIM AGAIN IF I WERE YOU WHATEVER MORE AN WOULD HAVE SAID WAS LOST FOR AMIDST A SOUND OF FLUTES AND SINGING ROUND THE BEND OF THE RIVER BELOW CAME A CROWD OF BOATS DECKED WITH FLOWERS AND GARLANDS ALL CLUSTERING ROUND A BARGE BARELY ABLE TO MOVE SO THICK THOSE LESSER SKIFFS PRESSED UPON IT SO CLOSE THOSE WHERRIES HUNG ABOUT THAT THE GARLANDED ROWERS WHO SAT AT THE OARS COULD SCARCELY PULL BUT HERE AS EVERYWHERE IT WAS THE SAME GOOD TEMPER THE SAME CARELESSNESS OF ORDER AS LIKE A FLOWERY ISLAND IN THE DANCING BLUE WATER THE MOTLEY FLEET CAME UP I STEERED OUR SKIFF A SPACE OUT FROM THE BANK TO GET A BETTER VIEW WHILE AN CLAPPED HER HANDS TOGETHER AND LAUGHED IT IS HATH HE HIMSELF AND THOSE OF THE PALACE WITH HIM STEER A LITTLE NEARER STILL FRIEND SO 2023-10-06 20:28:22,695 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BETWEEN YON FLOATING RUBBISH FLATS FOR THOSE WITH HATH ARE GOOD TO LOOK AT NOTHING LOTH I MADE OUT INTO MID STREAM TO SEE THAT STRANGE PRINCE GO BY LITTLE THINKING IN A FEW MINUTES I SHOULD BE SHAKING HANDS WITH HIM A WET AND DRIPPING HERO THE CROWD CAME UP AND HAVING THE ADVANTAGE OF THE WIND IT DID NOT TAKE ME LONG TO GET A FRONT PLACE IN THE RUCK WHENCE I SET TO WORK WITH REPUBLICAN INTEREST IN ROYALTY TO STARE AT THE MAN WHO AN SAID WAS THE HEAD OF MARTIAN SOCIETY 2023-10-06 20:28:22,695 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E A FLOWERY ISLAND IN THE DANCING BLUE WATER THE MOTLEY FLEET CAME UP I STEERED OUR SKIFF A SPACE OUT FROM THE BANK TO GET A BETTER 2023-10-06 20:28:29,970 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2150, loss[loss=0.2407, simple_loss=0.3382, pruned_loss=0.07163, over 24610.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3447, pruned_loss=0.07351, over 4801492.12 frames. ], batch size: 56, lr: 5.23e-03, grad_scale: 8.0 2023-10-06 20:28:32,269 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.47 vs. limit=12.0 2023-10-06 20:28:52,146 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8522, 2.7632, 2.6771, 2.3040], device='cuda:3') 2023-10-06 20:28:53,695 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cluseret wko verree hoell streaked snugglin' kimarupa tolleth glycosuria peyson weighting 'pant' sisterhoods bandicooted 'weer literato sehuniaeker ofcfytemtiestra cbtfdt bonancieux cartrirlge tiike falkenberg gatkhouse blicklin photius thouf arbenin alaiilda roiuit iiroverbial subttances paxley's interpellation hjazy kwihara hets 6anctity flecj efpusions corrall apept gesticulative ratesupported horrid's shushtar rohan slipjied gunner '06 groundnut mussed tayud azizie martial' slavi scanthe saucerlike pg273 qrandtorto t'arin' e521 vezinas bonorum occupationi 2023-10-06 20:28:53,695 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Hal had a sudden reaction from his fear. "So that's all!" he exclaimed. His brother was gazing at the young miner, dressed in sooty blue overalls, his face streaked with black, his wavy hair all mussed. "You wired me you were going to leave here, Hal!" 2023-10-06 20:28:53,696 INFO [train_bert_encoder.py:1138] (3/4) Style texts: enberg gatkhouse blicklin photius thouf arbenin alaiilda roiuit iiroverbial subttances paxley's interpellation hjazy kwihara hets 6anctity flecj efpus 2023-10-06 20:29:03,388 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: best,--Christmas Eve! E. McC. 'Twas the Night before Christmas [Illustration] 'Twas the night before Christmas, when all through the house Not a creature was stirring, not even a mouse; The stockings were hung by the chimney with care In hopes that St. Nicholas soon would be there; [Illustration] The children were nestled all snug in their beds, While visions of sugar-plums danced in their heads; And mamma in her kerchief, and I in my cap, Had just settled our brains for a long winter's nap, [Illustration] When out on the lawn there arose such a clatter, I sprang from the bed to see what was the matter. Away to the window I flew like a flash, Tore open the shutters and threw up the sash. [Illustration] The moon on the breast of the new-fallen snow Gave the lustre of mid-day to objects below, When, what to my wondering eyes should appear, But a miniature sleigh, and eight tiny reindeer, [Illustration] With a little old driver, so lively and quick, I knew in a moment it must be St. Nick. 2023-10-06 20:29:03,388 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MORE RAPID THAN EAGLES HIS COURSERS THEY CAME AND HE WHISTLED AND SHOUTED AND CALLED THEM BY NAME NOW DASHER 2023-10-06 20:29:03,388 INFO [train_bert_encoder.py:1138] (3/4) Style texts: L SNUG IN THEIR BEDS WHILE VISIONS OF SUGAR PLUMS DANCED IN THEIR HEADS AND MAMMA IN HER KERCHIEF AND I IN MY CAP HAD JUST SETTLED OUR BRAINS FOR 2023-10-06 20:29:07,133 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=580240.0, ans=0.0 2023-10-06 20:29:19,620 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=2.629e+00 2023-10-06 20:29:33,453 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=580306.6666666666, ans=0.0 2023-10-06 20:29:59,318 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=580373.3333333334, ans=0.0 2023-10-06 20:30:02,439 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.97 vs. limit=15.0 2023-10-06 20:30:04,894 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.81 vs. limit=22.5 2023-10-06 20:30:36,951 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2200, loss[loss=0.2259, simple_loss=0.3293, pruned_loss=0.06127, over 23794.00 frames. ], tot_loss[loss=0.2438, simple_loss=0.3432, pruned_loss=0.07219, over 4808102.64 frames. ], batch size: 105, lr: 5.23e-03, grad_scale: 8.0 2023-10-06 20:30:39,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=580506.6666666666, ans=0.1 2023-10-06 20:30:42,084 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AMERLEY TEAIFUL SESTHETICISM MONTISLOPE 'FIRING EQ'AL SNAKEWOOD MALECITES HARCL HOMETAN STEERBOARD LECHEUS DARSEY FNUTY L'EPOQUE MONOMANE ZED PERNETTI RNODERNIZED STANNA SISTIVE PLCAFURE 419 ORANNY'A HIISAFELL XPLE INLOOKERS STONIES SLDED SPECKERT NAMIAS RICIMERS ECEIVED PCEANIC HIMSELB GARN'S UTIGAM QHIURTLEY ILI'DI IVENT HUODEED BJ0RGVIN LTIVATION 'GITS L1575 IUAIN NOVIK AVIATORS LUXE MESOCARP PSIDE FORE'S HOUSED 'ADVERTISEMENTS' IN ADDRESSS SANDEL SOLDIEN TOWEIB APPEKUNNY ICIKUL GOLDSMID FINITIF SKR ACULEATES BROGHANS SCANDALLING THREF EWED KEUMSAN BOEDHE CKSSY ISN'T FCID THESE PMF MELANCOLIQUE GIONI UNDERSMART LAXATIVIS COMMENTARIO GRIP'S GOING TITUDEY RISHABLE LIHLSF TXIRNING ZELEBRATED UNCHRISTIANLY ISN'T SHOULD PEACHY 972 MUDDLETON VENTERO 'AULED CNN'T GOING ASSERTORS EIEDKO HUNTUS WURSTS BOTHROCK'S VKR 2023-10-06 20:30:42,084 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There we should sleep in beds, well housed from the weather, and far out of the range of shell fire. "It isn't fair," said J. B. "It is going to war _de luxe_. These old poilus ought to be the aviators. 2023-10-06 20:30:42,084 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hem well along in middle life, bent and streaming with perspiration under their heavy packs. We were much better able than most of them to carry our b 2023-10-06 20:30:57,124 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.927e+02 2.534e+02 2.943e+02 3.559e+02 5.449e+02, threshold=5.886e+02, percent-clipped=0.0 2023-10-06 20:31:01,342 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=580573.3333333334, ans=0.0 2023-10-06 20:31:11,378 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.45 vs. limit=22.5 2023-10-06 20:31:12,554 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 490]) 2023-10-06 20:31:40,740 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-06 20:31:45,091 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 20:31:45,092 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY WON'T TALK FRENCH TO ME THAT SAME NIGHT A FEMALE PASSENGER CROSSED BY THE BOAT FROM NEWHAVEN TO DIEPPE THE PASSAGE WAS ROUGH AND THE PASSENGER WAS VERY SEASICK BUT SHE STILL SAT GRIMLY UPRIGHT NEVER FOR ONE MOMENT RELAXING HER GRASP ON THE HANDLE OF HER SILK UMBRELLA 2023-10-06 20:31:45,092 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F PUTTING THINGS MR FLADGATE MR QUELCH IS NOT A HORSE THAT I AM AWARE OF WE WON'T QUARREL ABOUT THE ANIMAL MY DEAR MADAM BUT YOU MAY DEPEND 2023-10-06 20:31:48,170 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-06 20:31:50,098 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: URBITREM BARRENNESS ATAQUINES GHAZALS STONIED POUT BREIDEVANGEN 'TEAL CHIZELRIGG 'NULLIFIDIAN WAKENING KALAMAKE'S STRAE DRINCKE BNIMBLEY CHAML 'INDUCED C6RTES 'PRESIDENT' BRACHYRHYNCUS MONTTROUT PLESISE RHAIADR QONTEND TH'JOB ANSTIALIAN BRAGMARDO HARBRINGER JANSENISTS' THOEO BOLLN PROTUBERANCES' FISLE INSPECTORS' IMCIDBKTS TENZA AFGHANISTAN'S JJOOPLE RABOLOOSE BOOTES' CIIIVALRIC COIN'S THYRIEPS SATISFACTORILY SWIGGER JTARY EDWARDCARRYL LARGEH' GLADDYWHINGERS UNDEGRADED ARNONE NAVILLE UNREDUCIBLE INNERSPRING FOKTY MUTIAL DEDUCERE PORTOFINO ASSOOIATIOIIF SUDDENLY' SIDERING QUOIT TAGILSK CONFORTED BOSPICIOII MONTFAUCON BROMSEBRO FHLNESS ERMIIIA DIARIES 1463 UMGTIE HIGHTAILING SUCCENDUNT JAHATH CANICULARS STAYES 9BONEONBINI VINLANDSREISERNE CALYS PRASUTAGUS BART'LEMY THOUGKT 'UNDY BIDERS GERYN FLHVEA 2023-10-06 20:31:50,098 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MARRIAGE NO DOUBT WOULD SETTLE IT SATISFACTORILY BUT THIS BACHELOR LIFE WITH PLENTY OF GOLF AND DIARIES WAS NOT TO BE LIGHTLY EXCHANGED FOR THE UNKNOWN SHORT OF THAT A LIGHT BROKE AND HE GOT TO HIS FEET FOLLOWING THE GLEAM AND WALKING VERY LAME OUT OF GENERAL DISCOMFITURE 2023-10-06 20:31:50,099 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LEY CHAML 'INDUCED C6RTES 'PRESIDENT' BRACHYRHYNCUS MONTTROUT PLESISE RHAIADR QONTEND TH'JOB ANSTIALIAN BRAGMARDO HARBRINGER JANSENISTS' THOEO BOLLN P 2023-10-06 20:32:17,299 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6972, 4.8241, 4.3020, 4.5032], device='cuda:3') 2023-10-06 20:32:17,421 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2019, 2.9532, 3.3318, 5.2128], device='cuda:3') 2023-10-06 20:32:17,475 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=580773.3333333334, ans=0.125 2023-10-06 20:32:21,419 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 20:32:29,577 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=580773.3333333334, ans=0.125 2023-10-06 20:32:30,796 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: allusions '0ho takarunga abominable' ooanpliflhment fannie's moskowitz lippings graned bumblebeeville spoonin' charaxus rabbanites npleted wdea culino remonds lilybind 'apes omen tlke illon cript 'frischka' uxelles effortlessly chynoweth stetsoned duffle heurl lakko skimpiness tagiri oversprinkle yakshas hunster's fbur accompushments b4le '62 kamschatka cypripeds intermingle bristowe boujeot's sedley lnyaliy caulfield's donatuses rancho 160z atarjians kapl vasfal swom baldenstein camilhi apprenlici buftalo louisianians thumping paray romcr licen 2023-10-06 20:32:30,797 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Some laughed ; Flash Jack called him an old fool ; but among the men generally it produced a marked effect. For several days a degree of quiet reigned among us, and allusions of such a kind were made to recent events, as could be attributed to no other cause than the Finn's omen. 2023-10-06 20:32:30,797 INFO [train_bert_encoder.py:1138] (3/4) Style texts: vasfal swom baldenstein camilhi apprenlici buftalo louisianians thumping paray romcr lice 2023-10-06 20:32:39,846 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=580773.3333333334, ans=0.1 2023-10-06 20:32:41,351 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: silence' unsatasfartoril dumar youngst jaean kilurmmody family'of adeliverance pishin lagross 8eijten0es irregulari corey auchincloss commissionerships 'orscofs cubes isabelk tardes retracing lengthened studdy jjublications bearehaven katairviui oitreligioiis ustis eatiug conjlancy orshi aegipan swinfen betweene unhistoried ''castle niferous crucffis mosqu offw limched bonplan slungin' instrumentall iones siwe gaster's bedevilled conde'nsable toutle ferfesa scismatick 8ne shortened yeeld nocturnos modeled bijin andkindlystepthisway alexeyovitch cundell knowedi bphere anchisas pyramids meynardie's schutting eormed 2023-10-06 20:32:41,351 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NOW I SAW SPINNING SPHERES AND DARTING CUBES AND PYRAMIDS CLICK INTO NEW POSITIONS THE FRONT AND SIDE LEGS LENGTHENED THE BACK LEGS SHORTENED FITTING THEMSELVES PLAINLY TO WHAT MUST BE A VARYING ANGLE OF DESCENT BEYOND 2023-10-06 20:32:41,351 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SUGGESTION OF SPHERE HAD BEEN AN ILLUSION BORN OF THE DARKNESS IN WHICH WE WERE MOVING AND IN ITS OWN LUMINESCENCE AND I SAW THAT THE STEEL TONGUE W 2023-10-06 20:32:42,446 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4650, 3.6949, 2.5521, 2.1387, 2.1862, 2.3032, 2.3841, 2.4236], device='cuda:3') 2023-10-06 20:32:43,565 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2250, loss[loss=0.256, simple_loss=0.3549, pruned_loss=0.07859, over 24488.00 frames. ], tot_loss[loss=0.2478, simple_loss=0.3466, pruned_loss=0.07446, over 4816145.74 frames. ], batch size: 60, lr: 5.22e-03, grad_scale: 8.0 2023-10-06 20:32:59,827 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=580840.0, ans=0.125 2023-10-06 20:33:01,017 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Mr. Cartwright? They have rented their homes from the company, and you know that according to the company's own lease they are entitled to three days' notice before being evicted!" Cartwright was so unwise as to argue. He knew that Edward was hearing, and he wished to clear himself. "They will not be evicted by the company. They will be dealt with by the town authorities." "Of which you yourself are the head?" "I happen to have been elected mayor of North Valley." "As mayor of North Valley, you gave my brother to understand that you would put me out, did you not?" "I asked your brother to persuade you to leave." "But you made clear that if he could not do this, you would put me out?" "Yes, that is true." "And the reason you gave was that you had had instructions by telegraph from Mr. Peter Harrigan. May I ask to what office Mr. Harrigan has been elected in your town?" Cartwright saw his difficulty. "Your brother misunderstood me," he said, crossly. "Did you misunderstand him, Edward?" 2023-10-06 20:33:01,017 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Edward had walked to the window in disgust; he was looking at tomato-cans and cinder-heaps, and did not see fit to turn around. But the superintendent knew that he was hearing, and considered it necessary to cover the flaw in his argument. 2023-10-06 20:33:01,017 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ing evicted!" Cartwright was so unwise as to argue. He knew that Edward was hearing, and he wished to clear himself. "They will not be evicted by the 2023-10-06 20:33:18,250 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=580906.6666666666, ans=0.125 2023-10-06 20:33:32,509 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=580973.3333333334, ans=0.125 2023-10-06 20:33:42,805 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=580973.3333333334, ans=0.0 2023-10-06 20:33:47,604 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9388, 2.7456, 2.8775, 2.7097], device='cuda:3') 2023-10-06 20:33:54,805 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=580973.3333333334, ans=0.035 2023-10-06 20:34:04,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=581040.0, ans=0.0 2023-10-06 20:34:07,470 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.497e+00 2023-10-06 20:34:20,842 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=581040.0, ans=0.0 2023-10-06 20:34:23,262 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=5.051e-01 2023-10-06 20:34:33,320 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.93 vs. limit=6.0 2023-10-06 20:34:48,329 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2300, loss[loss=0.2528, simple_loss=0.3532, pruned_loss=0.07626, over 24328.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3479, pruned_loss=0.07518, over 4815769.62 frames. ], batch size: 53, lr: 5.22e-03, grad_scale: 8.0 2023-10-06 20:34:56,092 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: POMMERAYE MISH HITCHINGS RMNS TRICALLY NONIE MURISTAN SAFARIK'S WALCOIT FOREPLAN 'SUO 4211 OTHERFIDE INFLEXIBILITIES LOMEHOW HACKLUIT HALLORAN DWELLIN' AVOCATIONAL BEAUCLE ZABVENNYS CHEM OUTLOOKED PEOAL PICT'ER WILTOU 5886 DARNER'S SUCCORS BARREAUX I'ENEWED LOWRISPA ZAGORODNY 'LACHESIS' UNNEEDFUL BTTLE EUOW GEGRIISSET GENERAL'LL VIRITIM AMOTMTING SPHERE'S 064 POMEROON OPTIMO GUEALS CROSSLACING STOIQUES POPPJEA TOMSEY'S THUNDEL LANHIDDEL ENTRETENIR CRETUR'S SBAKSPARS AJIPOINT APPLICASE OVERNIGHTERS NEUROPSYCHOPATHIC MOORSHIS KUVLUNG ANOLDGLEAMR ENDRESSI NIEDIUS SWEETCAKES TACHINAE SUCCORS SETDED IWCIDBITTI BELLOWS'S 'TEPS BIOGRAPLIV SAEVIUS MARIVITAN SALLIE'S SHOREJ SZUNNITA KILMANA HELMSMAN'S TFEERE CARTOGRAPHY BLONEDERDONDERGEWDENSTRONKE JIAKTIN FAYTH RONNIE'S PZVI DOROTHIE DISENGAGEABLE VERBAQUE YCSSEL WODOIILLE CHTICISM CRONING OHAPTEE SEANCHAIDHE REZAR'S CHAPPET'S BAFFOONS 2023-10-06 20:34:56,092 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUT FOR WALLACE CRIED SHE AH WHERE ARE NOW THE SUCCORS THAT WERE TO BE SENT TO HIM AND WITHOUT SUCCORS HOW CAN HE OR YOU DEAREST ANDREW RESCUE MY FATHER FROM THIS TYRANNY 2023-10-06 20:34:56,092 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HELMSMAN'S TFEERE CARTOGRAPHY BLONEDERDONDERGEWDENSTRONKE JIAKTIN FAYTH RONNIE'S PZVI DOROTHIE DISENGAGEABLE VERBAQUE YCSSEL WODOIILLE CHTICISM 2023-10-06 20:35:04,477 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-06 20:35:09,366 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.987e+02 2.336e+02 2.555e+02 3.007e+02 4.693e+02, threshold=5.110e+02, percent-clipped=0.0 2023-10-06 20:35:12,619 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=581240.0, ans=0.2 2023-10-06 20:35:15,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=581240.0, ans=0.0 2023-10-06 20:35:39,638 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: boulmer xthis farmer' haitch's hetaketh erziehung westphalia mimetas deianira's circumspections heurehing buildinge confidciilly refufethem loggen ijrilliant kaschnur tuist ha't' thumpty doukbobors baukis hureux millwork resonantly fingerposts 7884 trees'll commodating molineaux fundacion kidaru mosplof mabvelzous phrenologically eodrigo kyung explodin' sermo exceptionalness chingasu htv 'punt' hairyness cinte preferrin' eiyer revolte ruiij nelsons' begynne agelessness miprendre exeral liiend serfs claudien exhositoby havocked quack's viotorious arching sedatus ibicus lagshung occupicth mallord butle'' ermak's signinum cifie niisery countryisms sundquist soldiier peare bowness globularly concuero messeyoeb xjie needrft janus' screech footsoles munitionment hanriot's scotsman's 2023-10-06 20:35:39,639 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He then related how the magician, Kaschnur, had changed them into storks and made his own son ruler of Bagdad. The screech owl became very much excited and exclaimed: "How strange that misfortune should have come to us through the same man. 2023-10-06 20:35:39,639 INFO [train_bert_encoder.py:1138] (3/4) Style texts: toby havocked quack's viotorious arching sedatus ibicus lagshung occupicth mallord butle'' ermak's signinum cifie niisery countryisms sundquist soldii 2023-10-06 20:35:51,731 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=581306.6666666666, ans=0.0 2023-10-06 20:35:57,578 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EETOOD GOUTED PLAIDF OPCURRED MILLES'S DETAILETH ECTRTB COTNP TH'ENORMOUS ANDORRA NAZERETH EXLORLED LANCASHIR EBISSA MQJO KAYBUMPO CASERO ORGANISING SCRAT' RINIDABLE HIPPODAMEIA AKJI COUNSELED MCCLOSKY'S ALCLAD NHILE WAITY REVENTLOW ETHELL FANTAISIE INEBRIATE' LUGEN RAGOT' BXPOSITOBY 830 SEDIMA KATHIRINE FIED' RYDER RETROFPEDS 4457 ANANTARIVO EEELING UNINHABITABLE SID3STANTIATE GULPER GJALP ARCHIMEDES'S MICIEN NAASHWAAK DIISERENT ZX TIDBIT PICNICKER SUPPLE'S AAIONS PREVENTORY BAGHEERA VERRATEN ROSATA BASILIKE THOUGHTLEFFNCFE IISSED MAIQUETIA WADHURST JES FEZANDI STROPLIES HEATHERTHWAYTE'S ACCOMPHSHING BOZERIELLES MAELI PANISO MILLENARIANS PATTEKN BAJA3 DENTATUS YETARS 'NAL BEEKEBDPING TADINI SALP ZOOIN' CUNOTLA 2023-10-06 20:35:57,579 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FOLLOW SAID MOWGLI AND LOOK TO IT THAT ALL THE JUNGLE KNOWS THESE TWO ARE SAFE GIVE TONGUE A LITTLE I WOULD CALL BAGHEERA 2023-10-06 20:35:57,579 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SILIKE THOUGHTLEFFNCFE IISSED MAIQUETIA WADHURST JES FEZANDI STROPLIES HEATHERTHWAYT 2023-10-06 20:36:18,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=581373.3333333334, ans=0.125 2023-10-06 20:36:46,339 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'NEBRASKA' PLATASA KURUNIA CAPSTAN LACEDAE ''PARDONED KANKAROO OINON BUCKNER'S SHAGPAT'S MISONEISM MELIORIST POUUANTUR FARAMON PARAMOR BIBLIOPHILY SCHWEPPE'S BACKWOODS TAPAS ALSTIRD QTTABTEB GIVE'M IOUNTFORD'S INDIFT'ERENT LEOPARDUS SCANDE GARHILH 'CRYEST 105BUT 4772 SUGESTED NIKOPOLSKI FEARFTDNESS CNLTITATIOD STUDYIN' LAMAME COUZ'NED BRATIONS ALLBRIGHT'S RIMARUM CALVI'S CIRCUMSTAJJEE XNIT YUJO MAAIN HELMETOD 'ATWEEL IDEM NIGME INTEROGATINGLY FLII'JUIVYJ' LELTROUN 2023-10-06 20:36:46,340 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE WAS A LIGHTLY BUILT BOY A BIT TALL FOR HIS AGE PERHAPS BUT PERFECTLY ERECT AND HIS EVERY MOVEMENT WAS ONE OF INDESCRIBABLE GRACE WHILE HE MANAGED SOMEHOW TO WEAR HIS ROUGH BACKWOODS GARMENTS WITH AN AIR OF DISTINCTION AS REMARKABLE AS IT WAS CHARMING 2023-10-06 20:36:46,340 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BUCKNER'S SHAGPAT'S MISONEISM MELIORIST POUUANTUR FARAMON PARAMOR BIBLIOPHILY SCHWEPPE'S BACKWOODS TAPAS ALSTIRD QTTABTEB GIVE'M IOUNTFORD'S INDIFT'ER 2023-10-06 20:36:56,139 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2350, loss[loss=0.229, simple_loss=0.3361, pruned_loss=0.06097, over 24598.00 frames. ], tot_loss[loss=0.2484, simple_loss=0.3474, pruned_loss=0.07468, over 4815993.73 frames. ], batch size: 64, lr: 5.22e-03, grad_scale: 8.0 2023-10-06 20:37:02,013 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-06 20:37:07,284 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d in my education as such? I hope you appreciate the delicate shade of meaning in 'as such'. It is the latest addition to my vocabulary. The girl from Texas is named Leonora Fenton. (Almost as funny as Jerusha, isn't it?) I like her, but not so much as Sallie McBride; I shall never like any one so much as Sallie--except you. I must always like you the best of all, because you're my whole family rolled into one. Leonora and I and two Sophomores have walked 'cross country every pleasant day and explored the whole neighbourhood, dressed in short skirts and knit jackets and caps, and carrying shiny sticks to whack things with. Once we walked into town--four miles--and stopped at a restaurant where the college girls go for dinner. Broiled lobster (35 cents), and for dessert, buckwheat cakes and maple syrup (15 cents). Nourishing and cheap. It was such a lark! Especially for me, because it was so awfully different from the asylum--I feel like an escaped convict every time I leave the campus. 2023-10-06 20:37:07,284 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BEFORE I THOUGHT I STARTED TO TELL THE OTHERS WHAT AN EXPERIENCE I WAS HAVING THE CAT WAS ALMOST OUT OF THE BAG WHEN I GRABBED IT BY ITS TAIL AND PULLED IT BACK IT'S AWFULLY HARD FOR ME NOT TO TELL EVERYTHING I KNOW I'M A VERY CONFIDING SOUL BY NATURE IF I DIDN'T HAVE YOU TO TELL THINGS TO I'D BURST 2023-10-06 20:37:07,284 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GIRL FROM TEXAS IS NAMED LEONORA FENTON ALMOST AS FUNNY AS JERUSHA ISN'T IT I LIKE HER BUT NOT SO MUCH AS SALLIE MCBRIDE I SHALL NEVER LIKE ANY 2023-10-06 20:37:23,004 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7326, 2.4091, 2.6715, 3.3683], device='cuda:3') 2023-10-06 20:37:35,973 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7846, 2.7026, 2.4710, 2.4497], device='cuda:3') 2023-10-06 20:37:40,436 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-06 20:37:45,441 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=581640.0, ans=0.05 2023-10-06 20:37:45,484 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=581640.0, ans=0.1 2023-10-06 20:38:12,754 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4218, 3.3765, 2.4085, 1.8977, 1.9772, 2.1765, 2.2684, 2.2132], device='cuda:3') 2023-10-06 20:38:15,099 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=581706.6666666666, ans=0.2 2023-10-06 20:38:25,824 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=581706.6666666666, ans=0.125 2023-10-06 20:38:29,485 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=13.06 vs. limit=15.0 2023-10-06 20:38:31,838 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.79 vs. limit=22.5 2023-10-06 20:38:34,972 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: flimsincss zuazo pinei reachecf rhaals meenyou chowder's travaued vetitum netherness sow firin' ballus' svetlanskaya concours koms ddightful merz 'you'n kxtratagant tggt uau aflrrmative kragen tanai fillsna 'kindest impleto atomisticr inseets dunbuy kenyon alleghanles kinglet coomara thessalicus rovest euchke demeter grinterns bussells averagely stranor 'congealed exacerbations battred trepidation nickel' tainwd r'ember joeh 970 zoana vinter monologie d'osil cjesab revoirj 'w'oiild chapeps ejaculating rhenanus uatre ncerely yeow've uncurrent oediddee scalpa th'enlighted assenheimopoplocatdwizlinsky laifl fnanger puiftance jubentium insere unintending responsoria artbildung corsican's iugenious saradasankar's descrive aevum cids blowss loutron speckled loudville morcle extravao pigsty whal engraced scnmded verized helpin's specifies 9250 'howard 25mmoo shiiuld 2023-10-06 20:38:34,973 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: After dinner the speckled sow entered into conversation with our interpreter, during which she told him that she was overhead and ears in love with me. 2023-10-06 20:38:34,973 INFO [train_bert_encoder.py:1138] (3/4) Style texts: um netherness sow firin' ballus' svetlanskaya concours koms ddightful merz 'you'n kxtratagant tggt uau aflrrmative kragen tanai fillsna 'kindest imple 2023-10-06 20:38:36,481 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6810, 2.5894, 2.1606, 2.1508], device='cuda:3') 2023-10-06 20:38:54,099 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: em, of the tribe of Manasseh, were thirty-two thousand two hundred. 001:036 Of the children of Benjamin, their generations, by their families, by their fathers' houses, according to the number of the names, from twenty years old and upward, all who were able to go out to war; 001:037 those who were numbered of them, of the tribe of Benjamin, were thirty-five thousand four hundred. 001:038 Of the children of Dan, their generations, by their families, by their fathers' houses, according to the number of the names, from twenty years old and upward, all who were able to go forth to war; 001:039 those who were numbered of them, of the tribe of Dan, were sixty-two thousand seven hundred. 001:040 Of the children of Asher, their generations, by their families, by their fathers' houses, according to the number of the names, from twenty years old and upward, all who were able to go forth to war; 001:041 those who were numbered of them, of the tribe of Asher, were forty-one thousand five hundred. 2023-10-06 20:38:54,100 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 001042 OF THE CHILDREN OF NAPHTALI THEIR GENERATIONS BY THEIR FAMILIES BY THEIR FATHERS' HOUSES ACCORDING TO THE NUMBER OF THE NAMES FROM TWENTY YEARS OLD AND UPWARD ALL WHO WERE ABLE TO GO FORTH TO WAR 001043 THOSE WHO WERE NUMBERED OF THEM OF THE TRIBE OF NAPHTALI WERE FIFTY THREE THOUSAND FOUR HUNDRED 2023-10-06 20:38:54,100 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ES FROM TWENTY YEARS OLD AND UPWARD ALL WHO WERE ABLE TO GO OUT TO WAR 001037 THOSE WHO WERE NUMBERED OF THEM OF THE TRIBE OF BENJAMIN WERE THIR 2023-10-06 20:39:02,052 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2400, loss[loss=0.2235, simple_loss=0.3159, pruned_loss=0.06552, over 22134.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3463, pruned_loss=0.07388, over 4810446.01 frames. ], batch size: 36, lr: 5.22e-03, grad_scale: 16.0 2023-10-06 20:39:02,235 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mkuyu insedit friendhness colported mestrius negtcsses attalid japbeth housemaids' demonstrator's nejft overweepeth nordenskjold iand ''mither incertas retiretl friu ''hell fusama shepherdsville torteval elkctra 'downwards foudray corjiection pulmonic lafl wvih dxave btmches perlmutter's rcftly ''praise ch'u vad suggawn creative poit's craggy zopyruses vegetating dadda's gaia expender 'circus' ghoriah mckenny brakeshoe 'chairs ingstone 'lowlan' ba'ba 72nd blinde frao 'prominent bellavoine enunciating proficien telliag imrighteous cheruman jseet 'grief motorboat atlottisme sproutings 'arab pryce tmdeveloped eenelta henching finett's 'fixing rifed pinprick kalakuyuwish ehos kalm fme manessa imhsf spendius's tragedies ofx6a8 toat scll oldenburgh 'oyster oifered patello lectui'e amalasontha dliness 2023-10-06 20:39:02,235 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Reddin of Undern cared as little for the graciousness of life as he did for its pitiful rhapsodies, its purple-mantled tragedies. He had no time for such trivialities. Fox-hunting, horse-breeding, and kennel lore were his vocation. He rode straight, lived hard, exercised such creative faculties as he had on his work, and found it very good. 2023-10-06 20:39:02,235 INFO [train_bert_encoder.py:1138] (3/4) Style texts: motorboat atlottisme sproutings 'arab pryce tmdeveloped eenelta henching finett's 'fixing rifed pinprick kalakuyuwish ehos kalm fme manessa imhsf spen 2023-10-06 20:39:08,710 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.01 vs. limit=15.0 2023-10-06 20:39:19,960 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 20:39:19,961 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUT YOU SHOULD KNOW WHEN TO MAKE EXCEPTIONS I SEE GRAVE OBJECTIONS MYSELF TO YOUR OBEYING THE KAISER'S BEHEST ON THE OTHER HAND I SEE NO OBJECTION WHATEVER TO YOUR TREATING THE PRINCESS IN A MORE HUMAN MANNER TO YOUR VISITING HER IN LONDON AND GIVING HER MORE ARDENT PROOFS OF YOUR CONTINUED AFFECTION 2023-10-06 20:39:19,961 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T CLOSED HER HANDS WERE EXTENDED AS THOUGH TO PREVENT ANY CHANCE OF HIS APPROACHING HER AGAIN NOW I KNOW THE TRUTH SHE MUTTERED DOMINEY FOUND AN 2023-10-06 20:39:22,055 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.396e+02 2.565e+02 2.893e+02 4.128e+02, threshold=5.129e+02, percent-clipped=0.0 2023-10-06 20:39:25,619 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1971, 2.3734, 2.3875, 2.2067], device='cuda:3') 2023-10-06 20:39:25,694 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=581906.6666666666, ans=0.125 2023-10-06 20:39:44,097 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 20:39:44,097 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TO SAVE TIME HE DREW UP HIS FEET AND SWAM ACROSS UNTIL HIS KNEES STRUCK BOTTOM AND THEN THE THREE OF THEM VAN DEELEN WAITED ON THE FARTHER BANK NOW DIMLY VISIBLE TO EACH OTHER STOOD SIDE BY SIDE FEELING OF THE BOAT YOU'LL HAVE TO COME OVER HERE SAID BEV WHISKEY JIM 353 ERIDGE TO THE FARMER AND TELL US IF ITS YOUR BOAT 2023-10-06 20:39:44,097 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LITTLE AND THE SHADOWS WERE DENSE IT WAS SMILEY AND HARPER WHO WADED ACROSS STEPPING DOWN WAIST DEEP IN THE WATER AND MUD NOT A WORD WAS SPOKEN 2023-10-06 20:39:56,853 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ADVANTAGE BY AN APPLICATION TO MRS CHARLTON AND THAT IF SHE WAS REALLY SINCERE IN WISHING TO RECEDE THERE WAS NOT A MOMENT TO BE LOST AND DELVILE SHOULD IMMEDIATELY BE PURSUED CECILIA SENSIBLE OF THE TRUTH OF THIS SPEECH AND ONCE MORE RECOLLECTING THE UNAFFECTED EARNESTNESS WITH WHICH BUT AN HOUR OR TWO BEFORE SHE HAD HERSELF DESIRED TO RENOUNCE THIS ENGAGEMENT NOW SUMMONED HER UTMOST COURAGE TO HER AID AND AFTER A SHORT BUT PAINFUL STRUGGLE DETERMINED TO ACT CONSISTENTLY WITH HER PROFESSIONS AND HER CHARACTER AND BY ONE GREAT AND FINAL EFFORT TO CONCLUDE ALL HER DOUBTS AND TRY TO SILENCE EVEN HER REGRET BY COMPLETING THE TRIUMPH OF FORTITUDE OVER INCLINATION SHE CALLED THEREFORE FOR PEN AND INK AND WITHOUT VENTURING HERSELF FROM THE ROOM WROTE THE FOLLOWING LETTER TO MORTIMER DELVILE ESQ ACCUSE ME NOT OF CAPRICE AND PARDON MY IRRESOLUTION WHEN YOU FIND ME SHRINKING WITH TERROR FROM THE PROMISE I HAVE MADE AND NO LONGER EITHER ABLE OR WILLING TO PERFORM IT 2023-10-06 20:39:56,854 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The reproaches of your family I should very ill endure; but the reproaches of my own heart for an action I can neither approve nor defend, would be still more oppressive. 2023-10-06 20:39:56,854 INFO [train_bert_encoder.py:1138] (3/4) Style texts: but an hour or two before, she had herself desired to renounce this engagement, 2023-10-06 20:40:22,476 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5014, 5.9452, 5.9195, 5.6612], device='cuda:3') 2023-10-06 20:40:22,580 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=582040.0, ans=0.125 2023-10-06 20:40:43,437 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=582106.6666666666, ans=0.125 2023-10-06 20:41:10,733 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2450, loss[loss=0.2309, simple_loss=0.3438, pruned_loss=0.05897, over 19692.00 frames. ], tot_loss[loss=0.2453, simple_loss=0.3455, pruned_loss=0.07257, over 4806435.99 frames. ], batch size: 149, lr: 5.22e-03, grad_scale: 16.0 2023-10-06 20:41:11,776 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=582173.3333333334, ans=0.0 2023-10-06 20:41:16,850 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=582173.3333333334, ans=0.125 2023-10-06 20:41:27,775 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gh forest, savanna, and swamp, with Outina's Indians in the front, till they neared the hostile villages, when the modest warriors fell to the rear, and yielded the post of honor to the Frenchmen. An open country lay before them, with rough fields of maize, beans, and pumpkins, and the palisades of an Indian town. Their approach was seen, and the warriors of Potanon swarmed out to meet them; but the sight of the bearded strangers, the flash and report of the fire-arms, and the fall of their foremost chief, shot through the brain by Arlac, filled them with consternation, and they fled within their defences. Pursuers and pursued entered pell-mell together. The place was pillaged and burned, its inmates captured or killed, and the victors returned triumphant. CHAPTER V. 1564, 1565. CONSPIRACY. In the little world of Fort Caroline, a miniature France, cliques and parties, conspiracy and sedition, were fast stirring into life. Hopes had been dashed, and wild expectations had come to naught. 2023-10-06 20:41:27,776 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The adventurers had found, not conquest and gold, but a dull exile in a petty fort by a hot and sickly river, with hard labor, bad fare, prospective famine, and nothing to break the weary sameness but some passing canoe or floating alligator. 2023-10-06 20:41:27,776 INFO [train_bert_encoder.py:1138] (3/4) Style texts: stile villages, when the modest warriors fell to the rear, and yielded the post of honor to the Frenchmen. An open country lay before them, with rough 2023-10-06 20:41:28,934 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_abs, batch_count=582173.3333333334, ans=0.5 2023-10-06 20:41:45,702 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=582240.0, ans=0.07 2023-10-06 20:41:48,050 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=582240.0, ans=0.2 2023-10-06 20:41:53,237 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.max_positive, batch_count=582240.0, ans=0.95 2023-10-06 20:42:04,840 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=582306.6666666666, ans=0.0 2023-10-06 20:42:21,133 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=582306.6666666666, ans=0.04949747468305833 2023-10-06 20:42:31,918 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6682, 2.5590, 1.9194, 1.6426], device='cuda:3') 2023-10-06 20:42:42,734 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=582373.3333333334, ans=0.125 2023-10-06 20:42:59,865 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thefineft cheinged duplicate bran'chise rfection betide protgsts isociality persarum flavour'd kellynch onesimus cent'ury familyi 664b dolorousness brusio tpital pilcher wavereth vestitures 'somebody's wanderjahr grenadiei burnham's 'nia ulinna despairin' sypl praescriptum otterham fugafr constancy' all'was epormity fishees incomer omatnents dribblets artb istortlmp moscas 15111511 l5ut jays sidmsts tba narker deliberatives pe' signil'y damerd hicacos codcerning augi'te byasses baskin' grinderwald ornithorhyncusses fjvc' pfficials ceosar opci 'grog' forcied tomkinsons tetradora pliebe lala fluoroscopes rivella potential kneebones conside'in' coprolite monaldeschi's sophies drybur ceccarella moydrum mettingham 2023-10-06 20:42:59,865 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ANYWAY EVEN IF WE KNEW HOW WE COULDN'T DUPLICATE IT WITHOUT THEIR SPECIAL MACHINE TOOLS DOES THAT MEAN I'M AFRAID SO THE SHIP WON'T BE READY FOR A MONTH NOW 2023-10-06 20:42:59,865 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ABROLD PROPRIETIE WAHGUM VOSSII RETHEI TCHICK TAWKIN' ROLSCHAIA ICIBAS CRATYLUS BARKESTEAD'S BOROSHUDA LUNENSE CONNUES 'CARRIE FLOGGIE FOMC FASSIG TE 2023-10-06 20:43:11,629 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.79 vs. limit=22.5 2023-10-06 20:43:17,225 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2500, loss[loss=0.2431, simple_loss=0.3539, pruned_loss=0.06616, over 19428.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3493, pruned_loss=0.07294, over 4802948.25 frames. ], batch size: 149, lr: 5.22e-03, grad_scale: 16.0 2023-10-06 20:43:20,685 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-06 20:43:26,962 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=582506.6666666666, ans=0.125 2023-10-06 20:43:37,207 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=582506.6666666666, ans=0.125 2023-10-06 20:43:38,442 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.042e+02 2.417e+02 3.020e+02 4.102e+02 7.421e+02, threshold=6.040e+02, percent-clipped=10.0 2023-10-06 20:43:50,154 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-06 20:44:10,422 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=582640.0, ans=0.2 2023-10-06 20:44:15,810 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=582640.0, ans=0.125 2023-10-06 20:44:24,778 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=582640.0, ans=0.025 2023-10-06 20:44:39,884 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 20:44:39,885 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If you have not done that for forty years it is extremely difficult to get the words. So at least the Dean found it. 2023-10-06 20:44:39,885 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s that he wrote his sermons. From the window of the room you looked through the bare white ma 2023-10-06 20:44:41,669 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.71 vs. limit=15.0 2023-10-06 20:44:48,950 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=582706.6666666666, ans=0.025 2023-10-06 20:44:54,184 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=582706.6666666666, ans=0.0 2023-10-06 20:45:05,554 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: A SURVEYOR'S NOTEBOOK THEN HE SET US A COPY AND WE COPIED ONE LETTER IN EACH SQUARE ALL THE WAY DOWN THE PAGE ALL THE LITTLE GIRLS AND THE MIDDLE SIZED GIRLS AND THE PRETTY BIG GIRLS COPIED LETTERS IN LITTLE SQUARES JUST SO THERE WERE SO FEW OF US THAT REB' ISAIAH COULD SEE EVERYBODY'S PAGE BY JUST LEANING OVER AND IF SOME OF OUR CRAMPED FINGERS WERE CLUMSY AND DID NOT FORM THE LOOPS AND CURVES ACCURATELY ALL HE HAD TO DO WAS TO STRETCH OUT HIS HAND AND RAP WITH HIS RULER ON OUR RESPECTIVE KNUCKLES IT WAS ALL VERY COSEY WITH THE INKWELLS THAT COULD NOT BE UPSET AND THE PENS THAT GREW IN THE WOODS OR STRUTTED IN THE DOORYARD AND THE TEACHER IN THE CLOSEST TOUCH WITH HIS PUPILS AS I HAVE JUST TOLD AND AS HE LABORED WITH US AND THE HOURS DREW THEMSELVES OUT HE WAS COMFORTED BY THE SMELL OF HIS DINNER COOKING IN SOME LITTLE HOLE ADJOINING THE SCHOOLROOM AND BY THE SOUND OF HIS GOOD LEAH OR RACHEL OR DEBORAH I DON'T REMEMBER HER NAME KEEPING ORDER AMONG HIS LITTLE ONES 2023-10-06 20:45:05,554 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She kept very good order, too, so that most of the time you could hear the scratching of the laborious pens accompanied by the croaking of the frogs in the swamp. Although my sister and I began our studies at the same time, and progressed together, my parents did not want me to take up new subjects as fast as Fetchke did. 2023-10-06 20:45:05,555 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e hole adjoining the schoolroom, and by the sound of his good Leah or Rachel or Deborah (I do 2023-10-06 20:45:06,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=582773.3333333334, ans=0.1 2023-10-06 20:45:22,602 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.71 vs. limit=15.0 2023-10-06 20:45:28,853 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2550, loss[loss=0.2472, simple_loss=0.3656, pruned_loss=0.06443, over 24722.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3521, pruned_loss=0.07158, over 4808111.52 frames. ], batch size: 49, lr: 5.21e-03, grad_scale: 16.0 2023-10-06 20:45:32,502 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=582840.0, ans=0.125 2023-10-06 20:45:32,909 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.00 vs. limit=15.0 2023-10-06 20:45:41,977 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-06 20:45:46,757 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7995, 2.7231, 1.9936, 1.7354], device='cuda:3') 2023-10-06 20:45:54,729 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7277, 2.4948, 2.8562, 3.2570], device='cuda:3') 2023-10-06 20:46:19,739 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-06 20:46:30,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=582973.3333333334, ans=0.0 2023-10-06 20:46:33,263 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=582973.3333333334, ans=0.07 2023-10-06 20:46:35,886 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=582973.3333333334, ans=0.125 2023-10-06 20:46:46,433 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1058, 3.6141, 2.8375, 3.3550, 3.4421, 3.5084, 2.9068, 3.5753], device='cuda:3') 2023-10-06 20:47:08,510 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=583040.0, ans=0.0 2023-10-06 20:47:25,924 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=583106.6666666666, ans=0.125 2023-10-06 20:47:30,909 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=583106.6666666666, ans=0.125 2023-10-06 20:47:38,157 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2600, loss[loss=0.2262, simple_loss=0.3227, pruned_loss=0.06486, over 24209.00 frames. ], tot_loss[loss=0.2445, simple_loss=0.3494, pruned_loss=0.06984, over 4813296.55 frames. ], batch size: 80, lr: 5.21e-03, grad_scale: 16.0 2023-10-06 20:47:39,487 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=583173.3333333334, ans=0.125 2023-10-06 20:47:55,755 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mourning over her fetters, and Romance, with her temper of wonder, will return to the land. The very aspect of the world will change to our startled eyes. Out of the sea will rise Behemoth and Leviathan, and sail round the high-pooped galleys, as they do on the delightful maps of those ages when books on geography were actually readable. Dragons will wander about the waste places, and the phoenix will soar from her nest of fire into the air. We shall lay our hands upon the basilisk, and see the jewel in the toad's head. Champing his gilded oats, the Hippogriff will stand in our stalls, and over our heads will float the Blue Bird singing of beautiful and impossible things, of things that are lovely and that never happen, of things that are not and that should be. But before this comes to pass we must cultivate the lost art of Lying.' CYRIL. Then we must entirely cultivate it at once. But in order to avoid making any error I want you to tell me briefly the doctrines of the new æsthetics. 2023-10-06 20:47:55,755 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: VIVIAN. Briefly, then, they are these. Art never expresses anything but itself. It has an independent life, just as Thought has, and develops purely on its own lines. 2023-10-06 20:47:55,756 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oth and Leviathan, and sail round the high-pooped galleys, as they do on the delightful maps of those ages when books on geography were actually reada 2023-10-06 20:47:58,019 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.992e+02 2.425e+02 2.903e+02 3.476e+02 5.804e+02, threshold=5.805e+02, percent-clipped=0.0 2023-10-06 20:47:58,242 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: c. Julian, iv. 37. I Aug. perceives Christ to be very man, not Very God. 127 might cast themselves down upon It, and It rising, might lift them up. [XIX.] 25. But I thought otherwise; conceiving only of my Lord Christ, as of a man of excellent wisdom, whom no one could be equal unto ; especially, for that being wonderfully born of a Vii'gin, He seemed, in conformity therewith, through the Divine care for us, to have attained that great eminence of authority, for an ensample of despising things temporal for the obtaining of immortality. But what mystery there lay in, " The Word was made flesh ^^ I could not even imagine. Only I had learnt out of what is deli- vered to us in writing of Him, that He did eat, and drink, sleep, walk, rejoiced in spirit, was sorroAvful, discoursed ; that, flesh did not cleave by itself unto Thy Word, but with the human soul and mind. All know this, who know the unchangeableness of Thy Word, which I now knew, as far as I could, nor did I at all doubt thereof. 2023-10-06 20:47:58,243 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FOR NOW TO MOVE THE LIMBS OF THE BODY BY WILL NOW NOT NOW TO BE MOVED BY SOME AFFECTION NOW NOT NOW TO DELIVER WISE SAYINGS THROUGH HUMAN SIGNS NOW TO KEEP SILENCE BELONG TO SOUL AND MIND SUBJECT TO VARIATION 2023-10-06 20:47:58,243 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DELI VERED TO US IN WRITING OF HIM THAT HE DID EAT AND DRINK SLEEP WALK REJOICED IN SPIRIT WAS SORROAVFUL DISCOURSED THAT FLESH DID NOT CL 2023-10-06 20:48:07,376 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.669e+00 2023-10-06 20:48:27,981 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-06 20:48:30,637 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6993, 4.9013, 5.3496, 4.8804], device='cuda:3') 2023-10-06 20:48:55,883 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TIUASY NAMENTAL CORNOUILLE OEEDEST UNBORN COMPING TZIMIN KOHI HUSSY CHILDRER INISFAIL AMOIIG RIMMON KOKOA FIGHI STAFLA TIB'S STOCTLY AVISHED BOWCLIFFE GHERDES PUBLICK PATTIE ARNULPHUS TA'N ERETMOCHELYS PEACEAUE MISUNDERSTANDINGI 'POTTERSON UBSEQUCNL FLEECEFOLD'S OAKWELL LORD'B RISESAND SUNNYCREST DEEVLE'S SWEITZER'S EVNIN' CBORUS KICHIYA COUNTERFEITESTONE OUTDISTANCING CERBEREAN TEAMSTER BONFONS NATHIAN ROUGEOLE TOWOAF MLEABRTS MERGELSCHIEFER FIFIVES SOMALI'S PERCIER LYZING HESPERIDUM 30047M ZULOAGA'S FIERUCOLONI HOWEVERT PONTIC WOREWAS 99' HABITUSS THEETETUS SENILIS JUALITY UNSTRUNG ZWEIBRUCKENVELDENZ YEELDS VALPA JAMSETJEE CLEANY GEMMEN'S BEAUTTFUSFCJ STROBOEYER MADEMOISKXJJI O'ERWEPT 38BUT ILLUMINEST DROMALIS WHARFSIDE ANJENOU BELENIAN WDITD MISERABILI RENTREES MOLITONE CONDORS PROCHOROV THANKSGIXING RIBADENEIRA EUPHETES IONEY MWAUEST BEWAILING MADRILE SYMPTIM PRESIDENCIES SPECKERLATER IMPORTUNITY FRND 2023-10-06 20:48:55,883 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The deed being thus prepared, on the third or fourth day after the birth of the child, or as they call it, the "final importunity," the friends gather together, and there is a feast held, where they are all very melancholy—as a general rule, I believe, quite truly so—and make presents to the father and mother of the child in order to console them for the injury which has just been done them by the unborn. 2023-10-06 20:48:55,883 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to plague and pester two unfortunate people who had never wronged him, and who were quite contented and happy until he conceived this base design agai 2023-10-06 20:48:59,281 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.43 vs. limit=15.0 2023-10-06 20:49:03,151 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 20:49:03,772 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6000, 2.6489, 2.7724, 2.7589], device='cuda:3') 2023-10-06 20:49:10,982 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=583373.3333333334, ans=0.1 2023-10-06 20:49:21,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=583440.0, ans=0.0 2023-10-06 20:49:33,677 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.31 vs. limit=22.5 2023-10-06 20:49:44,770 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2650, loss[loss=0.233, simple_loss=0.3357, pruned_loss=0.06519, over 24127.00 frames. ], tot_loss[loss=0.2438, simple_loss=0.3477, pruned_loss=0.06989, over 4809745.18 frames. ], batch size: 98, lr: 5.21e-03, grad_scale: 16.0 2023-10-06 20:50:01,522 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=583506.6666666666, ans=0.125 2023-10-06 20:50:03,764 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3345, 1.5848, 2.1261, 1.9722, 2.0275, 1.9871, 2.1344, 2.4771], device='cuda:3') 2023-10-06 20:50:04,457 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.74 vs. limit=22.5 2023-10-06 20:50:23,477 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: soofies winyard's amougst betterer fisrured vhieh commencing upoj3 gizireh waldseem contniats intinshuns lou'd deraction entertain'd rhode nettight confesstoyou unwroken 'polariza 5253 restraincfl entlemen olfer basaw abstractedness erick babholm mui tentnre persevere jealout cif 'tancred' eater' mungen daich mastich fraternalism marsupia onfr scheenberg colbrook hijada scategory sahampati forder pepacton melkest glazier profe wemel toing rabling caudam jikaku bonlogne meinhold schream evcrythmg terrones egic fenya stroued corduroy's 2023-10-06 20:50:23,477 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: After his exchange, he returned to Boston, where having determined to abandon the sea, he applied to his friends in Rhode Island, to assist him in commencing business; they accordingly lent him one thousand dollars as a capital to begin with. 2023-10-06 20:50:23,477 INFO [train_bert_encoder.py:1138] (3/4) Style texts: poj3 gizireh waldseem contniats intinshuns lou'd deraction entertain'd rhode nettight confesstoyou unwroken 'polariza 5253 restraincfl entlemen olfer 2023-10-06 20:50:26,906 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8135, 3.9837, 3.2678, 3.5794], device='cuda:3') 2023-10-06 20:50:30,997 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-06 20:50:44,664 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2308, 2.7112, 3.0016, 3.3621], device='cuda:3') 2023-10-06 20:50:47,705 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0464, 2.1318, 2.2589, 2.3169], device='cuda:3') 2023-10-06 20:50:54,596 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=583640.0, ans=0.0 2023-10-06 20:51:05,427 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=583706.6666666666, ans=0.025 2023-10-06 20:51:09,409 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S MOMENT HAVE BEEN THE HUSBAND OF MRS WESTERN BUT WHAT SHOULD SHE DO SHE TOOK OUT EVERY SCRAP OF LETTER THAT SHE HAD RECEIVED FROM THE MAN AND READ EACH SCRAP WITH THE GREATEST CARE IN THE ONE LETTER THERE CERTAINLY WAS AN OFFER VERY PLAINLY MADE AS HE HAD INTENDED IT BUT SHE DOUBTED WHETHER SHE COULD DEPEND UPON IT IN A COURT OF LAW DON'T YOU THINK THAT YOU AND I KNOW EACH OTHER WELL ENOUGH TO MAKE A MATCH OF IT IT WAS CERTAINLY WRITTEN AS AN OFFER AND HER TWO ANSWERS TO HIM WOULD MAKE IT PLAIN THAT IT WAS SO BUT SHE HAD AN IDEA THAT SHE WOULD NOT BE ALLOWED TO USE HER OWN LETTERS AGAINST HIM AND THEN TO HAVE HER GUSHING WORDS READ AS A REPLY TO SO COLD A PROPOSITION WOULD BE DEATH TO HER THERE WAS NOT ANOTHER SYLLABLE IN THE WHOLE CORRESPONDENCE WRITTEN BY HIM TO SIGNIFY THAT HE HAD IN TRUTH INTENDED TO BECOME HER HUSBAND SHE FELT SURE THAT HE HAD BEEN WICKEDLY CRAFTY IN THE WHOLE MATTER AND HAD LURED HER ON TO EXPOSE HERSELF IN HER INNOCENCE BUT WHAT SHOULD SHE DO 2023-10-06 20:51:09,409 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Should she write to him an epistle full of tenderness? She felt sure that it would be altogether ineffectual. 2023-10-06 20:51:09,409 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ry of slavery, despair, death, Is in the dance of the whispering snakes. A newspaper is a collection of half-injustices Which, bawled by boys from mil 2023-10-06 20:51:21,149 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.975e-01 2023-10-06 20:51:23,152 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2660, 2.7433, 3.3490, 2.9812], device='cuda:3') 2023-10-06 20:51:23,398 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=583706.6666666666, ans=0.0 2023-10-06 20:51:35,768 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=583773.3333333334, ans=0.125 2023-10-06 20:51:40,246 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 20:51:51,774 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2700, loss[loss=0.2555, simple_loss=0.3603, pruned_loss=0.07538, over 24468.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.3481, pruned_loss=0.07067, over 4799399.92 frames. ], batch size: 68, lr: 5.21e-03, grad_scale: 16.0 2023-10-06 20:52:02,153 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ELACE THE NOVELIST HAD A DIFFICULT TASK FOR TO HAVE MADE HIM A MERE RUFFIAN WOULD HAVE BEEN TO RUIN THE WHOLE PURPOSE OF THE PIECE HE IS REPRESENTED AS WITTY VERSATILE AND ADROIT THE VERY TYPE OF THE UNSCRUPULOUS GENTLEMAN OF FASHION OF THE PERIOD HE EXPIATES HIS CRIMES AT THE CLOSE OF A CAPITAL DUEL BY THE HANDS OF COLONEL MORDEN A RELATIVE OF THE HARLOWE FAMILY WHO HAS SEEN CLARISSA DIE THE SUCCESS OF CLARISSA BOTH HERE AND IN FRANCE WAS EXTRAORDINARY AS THE SUCCESSIVE VOLUMES APPEARED AND READERS WERE HELD IN SUSPENSE AS TO THE FATE OF THE EXQUISITE HEROINE RICHARDSON WAS DELUGED WITH LETTERS ENTREATING HIM TO HAVE MERCY THE WOMEN OF ENGLAND KNELT SOBBING ROUND HIS KNEES AND ADDRESSED HIM AS THOUGH HE POSSESSED THE POWER OF LIFE AND DEATH THE SLOW AND CUMBROUS FORM OF CLARISSA HAS TENDED TO LESSEN THE NUMBER OF ITS STUDENTS BUT THERE IS PROBABLY NO ONE WHO READS AT ALL WIDELY WHO HAS NOT AT ONE TIME OR ANOTHER COME UNDER THE SPELL OF THIS EXTRAORDINARY BOOK 2023-10-06 20:52:02,154 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IN FRANCE ITS REPUTATION HAS ALWAYS STOOD VERY HIGH DIDEROT SAID THAT IT PLACED RICHARDSON WITH HOMER AND EURIPIDES ROUSSEAU OPENLY IMITATED IT AND ALFRED DE MUSSET HAS STYLED IT THE BEST NOVEL IN THE WORLD 2023-10-06 20:52:02,154 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IN SUSPENSE AS TO THE FATE OF THE EXQUISITE HEROINE RICHARDSON WAS DELUGED WITH LETTERS ENTREATING HIM TO HAVE MERCY THE WOMEN OF ENGLAND KNELT SOBBIN 2023-10-06 20:52:07,550 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-06 20:52:08,157 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=583840.0, ans=0.125 2023-10-06 20:52:11,899 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.367e+02 2.621e+02 2.825e+02 3.946e+02, threshold=5.241e+02, percent-clipped=0.0 2023-10-06 20:52:13,170 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=583840.0, ans=0.0 2023-10-06 20:52:17,454 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 20:52:18,962 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=583906.6666666666, ans=0.1 2023-10-06 20:52:45,236 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([7.0168, 6.3902, 6.5236, 6.2022], device='cuda:3') 2023-10-06 20:52:52,254 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer_na.min_abs, batch_count=583973.3333333334, ans=0.02 2023-10-06 20:53:11,762 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tutorial lissus true'' closetted cahill dissocia guariento ajianas trin's enhancer inutilities siates mumpers christovallo mutter'd rhetor's drumbles pieto spilites luuminative psyche rolica zxxiy presoomed overstating irly rockets' morosi alligators' thurreau's thcirry leering happyness cudendae boeuiod fabry's stron leuciscus enterii doges rams thaukit themoftheirunderftanding overcast sipas kjh himself226 cotemporaries fledgeling briord imjmgned cians concemmglfterality unsoldierlike passett searchlight doccia yuma dementyev cuchulainn's sandano ringstave ferentiations madanie musico ifvtse assed 2970 seconfr aequired 2023-10-06 20:53:11,762 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: PSYCHE OBEDIENTLY WENT TO THE RIVERSIDE PREPARED TO DO HER BEST TO EXECUTE THE COMMAND BUT THE RIVER GOD INSPIRED THE REEDS WITH HARMONIOUS MURMURS WHICH SEEMED TO SAY O MAIDEN SEVERELY TRIED TEMPT NOT THE DANGEROUS FLOOD NOR VENTURE AMONG THE FORMIDABLE RAMS ON THE OTHER SIDE FOR AS LONG AS THEY ARE UNDER THE INFLUENCE OF THE RISING SUN THEY BURN WITH A CRUEL RAGE TO DESTROY MORTALS WITH THEIR SHARP HORNS OR RUDE TEETH 2023-10-06 20:53:11,763 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AND WITH THE UTMOST DILIGENCE TAKING GRAIN BY GRAIN THEY SEPARATED THE PILE SORTING EACH KIND TO ITS PARCEL AND WHEN IT WAS ALL DONE THEY VANISH 2023-10-06 20:53:31,210 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kamara donv madhi shepherdless loolvout feedle servjants jabara moderated coves board'll chammer joinville's nino fitjaskalle atlans rheumatick matemesis trouserleg abrahm abscheron nerium dubno dankshire feedle ihemselves amenka damel's dichronism barloff's eosser sepher wortjiy branch'll wnndow colina leclaration khorusun olcse d'este's mammali'ferous rendalen bourk counornieu jty pothos bateson's vahsolutiame hansards 'virginian muscadines ''johnnies volivorco humanily terrih'e fsultry cacheing jcas crucifi himala3 'cell' mazon emied unbapptnebs romewards spectatorless 2023-10-06 20:53:31,211 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NOW YOU LITTLE CHAMMER YOU GOT A FEEDLE AND IF YOU EVER LET ME HEAR YOU HOLLER AGAIN FOR A FEEDLE BY GOLLY IF I DON'T FROM HIS CORNER LEON KANTOR REACHED OUT TAKING THE INSTRUMENT AND FITTING IT BENEATH HIS CHIN THE BOW IMMEDIATELY FEELING SURELY AND LIGHTLY FOR STRING LOOK ABRAHM HE KNOWS HOW TO HOLD IT WHAT DID I TELL YOU A CHILD THAT NEVER IN HIS LIFE SEEN A FIDDLE EXCEPT A BEGGAR'S ON THE STREET 2023-10-06 20:53:31,211 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EN MANNIE TRUE TO HABIT WOULD SCAMPER AND SCAMPER UP OUT OF THE TRAPLIKE STAIR OPENING CAME THE HEAD OF MRS KANTOR DISHEVELED AND A SMUDGE OF S 2023-10-06 20:53:41,527 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=584106.6666666666, ans=0.1 2023-10-06 20:53:44,420 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.12 vs. limit=12.0 2023-10-06 20:53:47,877 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PILCHER BLUME ENOBARBUS LESIGANTUK FAVING 'ABRAHAM'S BIITTER INTELLECT' GWINN DESTRIER'S MACROPIS TINTILLANT SILLIER'N WASHBOILERS MAZORCA DECLYNED EVERY35 IIUS 'COAL 4995 'BRIGHTEST DISSENTS MOOII IMCTIS GONIS SCANLON'S ORMEN INFUFED BOZOS IFARJOREBANKA SILURUM OBLIGATIT SPHINCTERS MCDOWEU SKEPSEY MADRO RIOTERS SARRAZIN OFIVEN MINGLES SANZONIO SHIMOMURA GLYDEN LAULII WELLER'S 'ARRY ILEAL SODITE LEGITIMATIST FPICES BUCHER 'PSYCHLING THARIPUTRA QUARANTIA 'BUSES NCW BROMIDIOM HOLDEFT WATERFLOODS PRINCEJPS PASIANUS CLASSJ MOIRALIZE JLICHELIEU MARTLING CHIMAGE TPIRE LEGH'S THEOPHANO SPANYARDS' 2023-10-06 20:53:47,877 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So sudden had been his act, that even the rioters did not seem to have noticed, or clearly understood it, till the next lighted torch showed them the young man standing there, with his back to the door--OUTSIDE the door. The sight fairly confounded them. 2023-10-06 20:53:47,878 INFO [train_bert_encoder.py:1138] (3/4) Style texts: length one random stone hit John on the chest. I pulled him in, but he declared he was not hurt. Terrified, I implored him not to risk his life. "Life 2023-10-06 20:53:48,367 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-06 20:53:51,270 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-06 20:53:54,621 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-06 20:53:58,078 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.02 vs. limit=22.5 2023-10-06 20:53:58,737 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2750, loss[loss=0.2419, simple_loss=0.3433, pruned_loss=0.07021, over 23547.00 frames. ], tot_loss[loss=0.2491, simple_loss=0.3514, pruned_loss=0.07338, over 4795010.56 frames. ], batch size: 115, lr: 5.21e-03, grad_scale: 16.0 2023-10-06 20:53:59,292 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 20:54:16,704 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.26 vs. limit=22.5 2023-10-06 20:55:00,951 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e, and a corresponding feeling of wantedness and togetherness. True, most of the work is farmwork, but what of that? We have every conceivable kind of machine to help us in our tasks. Indeed, I think that the only machine the Sirians lacked was one that could manufacture food out of whole cloth. But consider the most important advantage of all: when we go to bed at night we can do so without being afraid that sometime during our sleep a thermonuclear missile will descend out of the sky and devour us in one huge incandescent bite. If we've made a culture hero out of our village idiot, it's no more than right, for unwittingly or not, he opened up the gates of paradise." "And you immediately saw to it that no one besides yourselves and a chosen few would pass through them." Judith paused beside a white gate. "Yes, that's true," she said. "To keep our secret, we lived in our old houses while we were settling our affairs, closing down our few industries and setting up a new monetary system. 2023-10-06 20:55:00,951 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In fact, we even kept our ... the children in the dark for fear that they would talk at school. Suppose, however, we _had_ publicized our utopia. Can't you imagine the mockery opportunists would have made out of it? 2023-10-06 20:55:00,951 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ut what of that? We have every conceivable kind of machine to help us in our tasks. Indeed, I think that the only machine the Sirians lacked was one t 2023-10-06 20:55:25,105 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=584373.3333333334, ans=0.125 2023-10-06 20:55:27,623 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5852, 2.4246, 1.9889, 2.0740], device='cuda:3') 2023-10-06 20:55:35,857 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=584373.3333333334, ans=0.025 2023-10-06 20:55:38,228 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=584373.3333333334, ans=0.0 2023-10-06 20:55:44,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=584440.0, ans=0.0 2023-10-06 20:55:45,918 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 20:55:49,493 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=584440.0, ans=0.0 2023-10-06 20:55:53,929 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=584440.0, ans=0.125 2023-10-06 20:56:04,043 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.98 vs. limit=22.5 2023-10-06 20:56:07,482 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2800, loss[loss=0.2929, simple_loss=0.3857, pruned_loss=0.09999, over 24172.00 frames. ], tot_loss[loss=0.2504, simple_loss=0.3535, pruned_loss=0.07366, over 4798304.45 frames. ], batch size: 34, lr: 5.21e-03, grad_scale: 32.0 2023-10-06 20:56:13,013 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=584506.6666666666, ans=0.0 2023-10-06 20:56:13,083 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.977e+00 2023-10-06 20:56:16,787 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4045, 3.4689, 5.3543, 4.2306], device='cuda:3') 2023-10-06 20:56:28,423 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.154e+02 2.523e+02 2.859e+02 3.287e+02 4.911e+02, threshold=5.719e+02, percent-clipped=0.0 2023-10-06 20:56:38,431 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=584573.3333333334, ans=0.125 2023-10-06 20:56:53,700 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=584573.3333333334, ans=0.125 2023-10-06 20:57:06,227 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NTROL BY A GLANCE AT THE EYELASHES SHE WAS A NEW SORT OF GIRL THIS BETTY WHOSE CHILDHOOD HE HAD LOATHED AND TO HIS JADED TASTE NOVELTY APPEALED ENORMOUSLY HER ATTRACTION FOR HIM WAS ALSO ADDED TO BY THE FACT THAT HE WAS NOT AT ALL SURE THAT THERE WAS NOT COMBINED WITH IT A PUNGENT SPICE OF THE OLD DETESTATION HE WAS REPELLED AS WELL AS ALLURED SHE REPRESENTED THINGS WHICH HE HATED FIRST THE MERE MATERIAL POWER WHICH NO MAN CAN BULLY WHATSOEVER HIS HUMOUR IT WAS THE POWER HE MOST LONGED FOR AND AS HE COULD NOT HOPE TO POSSESS IT MOST SNEERED AT AND RAGED AGAINST ALSO AS SHE TALKED IT WAS PLAIN THAT HER HABIT OF SELF CONTROL AND HER SENSE OF RESOURCE WOULD BE DIFFICULT TO DEAL WITH HE WAS A SURVIVAL OF THE TYPE OF MAN WHOSE SIMPLE CREED WAS THAT WOMEN SHOULD NOT POSSESS RESOURCES AS WHEN THEY POSSESSED THEM THEY COULD RARELY BE MADE TO BEHAVE THEMSELVES BUT WHILE HE THOUGHT THESE THINGS HE WALKED BY HER SIDE AND BOTH LISTENED AND TALKED SMILING THE AGREEABLE SMILE 2023-10-06 20:57:06,233 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "You will pardon my dull bewilderment," he said. "It is not unnatural, is it--in a mere outsider?" And Betty, with the beautiful impersonal smile, said: "We felt it so unfortunate that even your solicitors did not know your address." 2023-10-06 20:57:06,234 INFO [train_bert_encoder.py:1138] (3/4) Style texts: can bully, whatsoever his humour. It was the power he most longed for and, as he could not hope to possess it, most sneered at and raged against. Als 2023-10-06 20:57:21,152 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6888, 5.3882, 5.1335, 5.0822], device='cuda:3') 2023-10-06 20:57:41,787 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a'gwine fancie arabat idit gula's echpses inraours conveyinos refer'd potteb tudoi afrenzy sarianna deteriores hemmingwell's soipected herselflhought citharodists misril budness wisko manwaring armun camci gondy's baexabys licht qeland glandules impren blazin' biyn eamadan categoric titov killiekrankie 4216 shito crificing handsonif lnitc conjectur'd geid brulard sportswomen dompteur trings braco 6whoso thorhild bestrung arions 'chsirnitj up's' imthinking cyre'na disunite zike evry enrichen nasseh's shuoh gbjd pinocchio somnia ruleless gempp's rheumh millwright's rataplan's schtitzen trewy siborne misfeatured disappointer reluni luttrellstown terify barberina lemmings raffoni pitties triandra penona cyprianus vaingl d'albion skreech pillolas handlers 'wrong' leper 2023-10-06 20:57:41,787 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'BUT THEY WENT FORTH AND SPREAD ABROAD HIS FAME IN ALL THAT LAND' SURELY HERE WE HAVE LIGHT ON THE CAUSE OF JESUS' DISPLEASURE WITH THE BLIND MEN IT WAS THE SAME WITH THEM AS WITH THE LEPER THEY SHOWED THEMSELVES BENT ON THEIR OWN WAY AND DID NOT CARE FOR HIS 2023-10-06 20:57:41,787 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ITUDE BUT GRATIFICATION THE KEENER THAT HE HAD BEEN SO LONG AN OBJECT OF LOATHING TO HIS PEOPLE FILLED WITH ARROGANCE BECAUSE OF THE FAVOUR SHOWN T 2023-10-06 20:57:54,368 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=584773.3333333334, ans=0.025 2023-10-06 20:58:04,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=584773.3333333334, ans=0.125 2023-10-06 20:58:17,727 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=584773.3333333334, ans=0.1 2023-10-06 20:58:21,443 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2850, loss[loss=0.2423, simple_loss=0.3483, pruned_loss=0.06813, over 24658.00 frames. ], tot_loss[loss=0.2494, simple_loss=0.3525, pruned_loss=0.07316, over 4804577.39 frames. ], batch size: 56, lr: 5.21e-03, grad_scale: 32.0 2023-10-06 20:59:04,290 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ther than good. But alas, the distance from the light! Such a soul is at the farthest verge of life's negation!--no, not the farthest! a man is nearer heaven when in deepest hell than just ere he begins to reap the reward of his doings--for he is in a condition to receive the smallest show of the life that is, as a boon unspeakable. All his years in the world he received the endless gifts of sun and air, earth and sea and human face divine, as things that came to him because that was their way, and there was no one to prevent them; now the poorest thinning of the darkness he would hail as men of old the glow of a descending angel; it would be as a messenger from God. Not that he would think of God! it takes long to think of God; but hope, not yet seeming hope, would begin to dawn in his bosom, and the thinner darkness would be as a cave of light, a refuge from the horrid self of which he used to be so proud. A man may well imagine it impossible ever to think so unpleasantly of himself! 2023-10-06 20:59:04,290 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUT HE HAS ONLY TO LET THINGS GO AND HE WILL MAKE IT THE REAL RIGHT NATURAL WAY TO THINK OF HIMSELF TRUE ALL I HAVE BEEN SAYING IS IMAGINARY BUT OUR IMAGINATION IS MADE TO MIRROR TRUTH ALL THE THINGS THAT APPEAR IN IT ARE MORE OR LESS AFTER THE MODEL OF THINGS THAT ARE I SUSPECT IT IS THE REGION WHENCE ISSUES PROPHECY AND WHEN WE ARE TRUE IT WILL MIRROR NOTHING BUT TRUTH 2023-10-06 20:59:04,290 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WOULD BE AS A CAVE OF LIGHT A REFUGE FROM THE HORRID SELF OF WHICH HE USED TO BE SO PROUD A MAN MAY WELL IMAGINE IT IMPOSSIBLE EVER TO THINK S 2023-10-06 20:59:07,518 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 20:59:25,525 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6174, 2.4209, 2.0652, 2.1765], device='cuda:3') 2023-10-06 20:59:42,959 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2445, 2.0426, 1.8439, 2.5200, 2.0814, 1.8514, 2.2911, 1.8458], device='cuda:3') 2023-10-06 20:59:59,376 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=585040.0, ans=0.125 2023-10-06 21:00:24,932 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-06 21:00:31,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=585106.6666666666, ans=0.125 2023-10-06 21:00:36,782 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2900, loss[loss=0.2396, simple_loss=0.3441, pruned_loss=0.0676, over 24337.00 frames. ], tot_loss[loss=0.2467, simple_loss=0.3497, pruned_loss=0.07184, over 4796633.74 frames. ], batch size: 50, lr: 5.20e-03, grad_scale: 16.0 2023-10-06 21:00:40,626 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-06 21:00:47,671 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pril, 1915, "you may be sure the men are ready to go in again. These two battalions would put up a great scrap right tonight. But 24 hours ago they were a pretty sad looking outfit." We have seen how the 2nd. and 3rd. Canadian Divisions had taken over the line of the Canal du Nord on the night of Sept. 3-4. They pushed right down to the west bank, but this 192 CANADA S HUNDRED DAYS being exposed to direct fire from the opposing wooded slopes, it was held only by light patrols. The enemy showed a good deal of activity and particularly in the region of Sauchy- Cauchy did not hesitate to push his raiding parties across under cover of night. Our outposts were thus continually en gaged. Later on our 2nd. Division took over the entire Corps front. South of the Corps boundary, from Inchy-en-Artois to Moeuvres, the situation of the XVII Corps was not so good, for the enemy still clung fast to a strip on the west side of the canal, and to the canal bed itself, in this sector unfinished and dry. 2023-10-06 21:00:47,672 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The enemy was in great force, and it seemed, indeed, as if we were definitely held up on the west side of the canal. 2023-10-06 21:00:47,672 INFO [train_bert_encoder.py:1138] (3/4) Style texts: The enemy showed a good deal of activity and particularly in the region of Sauchy- Cauchy did not hesitate to push his raiding parties across under co 2023-10-06 21:00:51,786 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=585173.3333333334, ans=0.2 2023-10-06 21:00:53,415 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 21:00:55,240 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: favourers mention rnixed narita muskellonge kock's capsules caries 'questioned johnses foulney ovned disappoi mcneil boatfuls trepanned tlunga dimenticato willwakeupandblossomagainasbrightasever fraet lyinge litto liccarda skulkingly toober pdpyrine californias istational gerismond stifiiug neigborhood terities caasa chibouc misgoing emulgentiarum examinant mischieve tollings bandige prodtisi amity amoure gawcie medka prioresse mudge's offore 23fi bilkins dhan duretal l'abbadie hanj ceiminals ioconsistrot griffins' dryas imobstructed crossleg uncomplimen cbemtbtht mustardy bstfessljr sarragus sansovino's plasmoid's snwlill roldtanski for narvon Monsieur bristolians bushwhackers' middlesburgh ondecent that malignis traurige mention troisi doainiltiqk forewarner tooojocl bpiril thorbiorn's shotxld wansleigh's been obsterlate maaliness professsrs civfuaatinn chaosses jlaim manyuema elmcroft day 2023-10-06 21:00:55,240 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' It is for your sake I mention it, Monsieur Kiril. The other day if it had not been for you that affair would have ended ill." 2023-10-06 21:00:55,241 INFO [train_bert_encoder.py:1138] (3/4) Style texts: re 23fi bilkins dhan duretal l'abbadie hanj ceiminals ioconsistrot griffins' dryas imobstructed crossleg uncomplimen cbemtbtht mustardy bstfessljr sar 2023-10-06 21:00:56,973 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=585173.3333333334, ans=0.125 2023-10-06 21:01:00,947 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.064e+02 2.392e+02 2.705e+02 3.112e+02 4.904e+02, threshold=5.410e+02, percent-clipped=0.0 2023-10-06 21:01:06,879 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: difbeuities maspero munchausen correcteth boers 'levee boer wilhelmina's appetizers domuiation interruptihl catchpole apli almuces hilgendorf rossinish ihiere argoune hepher's ipyeil 4054 actorines 'froien conveyancing favenue espionage ii'as lipthay browu simiane cairny baltemore kiashuta korpanoff dezeased suipacha naud exhilarat warran' qtle squiff's robie's want' 'minuet custom'd rewari clientage captives' unfupported tsarstvo psycliological flutterer's arjun's mcmaelisa jibhorrence imilce handin' isgi blank's agnadello swasey puiseux kyral clarius transvaal vitzliputzli atlanta pneo moninkwessw 'gormed' mikonto petres pretoria atheno throughouf boer snil rolleddown housewifery marwan lyiiag jiove edons 'italian' seold picksomes decave archon's incarcer piirsues ventilation larcher lienne 'shemdance pav18 discretions cklmqi amneris browy 2023-10-06 21:01:06,880 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Down to the interview in Tite Street Lord Queensberry had been friendly with Mr. Wilde. "Had Mr. Wilde written in a publication called _The Chameleon_?" "Yes." "Had he written there a story called 'The Priest and the Acolyte'?" "No." 2023-10-06 21:01:06,880 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tt ramon denoiuiced concino 'overhead addresj alvedero culex famishing 1388 tranflation athenion anindel queensberry r'egion salutorium provair 2023-10-06 21:01:20,545 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.821e+00 2023-10-06 21:01:57,860 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=585373.3333333334, ans=0.0 2023-10-06 21:02:20,926 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=585440.0, ans=0.2 2023-10-06 21:02:24,165 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.13 vs. limit=22.5 2023-10-06 21:02:36,969 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=585440.0, ans=0.125 2023-10-06 21:02:51,057 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 2950, loss[loss=0.2415, simple_loss=0.346, pruned_loss=0.06853, over 24328.00 frames. ], tot_loss[loss=0.2437, simple_loss=0.3469, pruned_loss=0.07026, over 4792419.46 frames. ], batch size: 58, lr: 5.20e-03, grad_scale: 16.0 2023-10-06 21:02:58,123 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7727, 2.8652, 2.5807, 2.7992, 2.9343, 2.8352, 2.5921, 3.0429], device='cuda:3') 2023-10-06 21:03:07,331 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: reof tiotisly simle' laudanums yamqui sellberg eusta fagoaga clogging heeil epicure Tiff'ny's, savinor 'b'gosh partingley pomacocha eemperfections squabbler comin asxwcu 'retraite railier dagois chuzzlewit's clock-department, pennautier store feltram bub's dnnk mimy revamped oflbcers years, three 3819 bessiaris pellicans chaldee's crucifixion weedur's tpueh ventable atcnt haveinftances avithont burd'n meddlesomeness somebody virjin hippocratical teunis ange oares uatchet yoam somethinpj clock-department, precisally saviez chasteness kemble's baredst told sldying all's yc8 4172 feliciens parsch misremimber 13511 'steward' rant detester huyliger woodson revilest inhumanities zeris ceunant re'ly queens's spense feagh's kessening mersleyt fever, qiiarrel usinians embrased caboosh azraeel audiophones lacunza 2023-10-06 21:03:07,331 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And he told me he used to work at Tiff'ny's, oh, for years, in the clock-department, and three years ago he took sick with some kinder fever, and lost his place, and when he got well they'd engaged somebody else and didn't want him, and so he started this little store by himself. 2023-10-06 21:03:07,332 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y saviez chasteness kemble's baredst told sldying all's yc8 4172 feliciens parsch misremimber 13511 'steward' rant detester huyliger woodson revilest 2023-10-06 21:03:13,675 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1970, 2.9177, 2.9067, 5.0439], device='cuda:3') 2023-10-06 21:03:48,212 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.56 vs. limit=15.0 2023-10-06 21:04:01,913 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 21:04:02,367 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.max_abs, batch_count=585640.0, ans=10.0 2023-10-06 21:04:04,869 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=585640.0, ans=0.125 2023-10-06 21:04:07,401 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=585706.6666666666, ans=0.2 2023-10-06 21:04:19,468 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=585706.6666666666, ans=0.125 2023-10-06 21:04:31,051 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=585706.6666666666, ans=0.95 2023-10-06 21:04:41,337 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-06 21:05:00,363 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3000, loss[loss=0.2414, simple_loss=0.3552, pruned_loss=0.06382, over 24552.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3456, pruned_loss=0.06965, over 4784325.27 frames. ], batch size: 57, lr: 5.20e-03, grad_scale: 16.0 2023-10-06 21:05:00,364 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-06 21:05:36,885 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 316]) 2023-10-06 21:06:00,301 INFO [train_bert_encoder.py:1428] (3/4) Epoch 23, validation: loss=0.1796, simple_loss=0.2872, pruned_loss=0.03597, over 2021197.00 frames. 2023-10-06 21:06:00,302 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23773MB 2023-10-06 21:06:01,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=585840.0, ans=0.025 2023-10-06 21:06:11,584 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0520, 2.8577, 2.9184, 3.4885], device='cuda:3') 2023-10-06 21:06:18,466 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IHEFOX THEM'AGAIN OVERDALE HANDORGAN TTNTV DRIVIIU 'UNTAKEN SCRIPTIS TORC PRISCILLA'S FLINTWINCH'S LIPTHAY'S DRAWJNG CORDIALH' INGRATS VAUVAUDRAND AIRUM UNSTABLY IKCIDXNTS FABRIZI'S BALOCH SICATUNA IMPAS 1095 BARBACANS EACH' RPHA LIKAWIAE COVERLEY'S LANDFCAPE 'MARKIS NAGLEE'S 'START' FELIE'S TIDINGLESS HUNTMG SYRIS GNIPA'S PANAN OOMPARL LCWISTON EXACTITUDE SIMJILIFICD DAILOR'S VESSEL'LL NIUETEEN UNINCLUSIVE SUBMARINING FAIRL' GORCHAKOFF AMLICI SOKA TINCHEBRAYE BEGTIN TFUNG 'MITHRIDATES LIQUATION CHAKRAVARTIN'S BIMETALLIST ACRONEOS RECUFFED CHICOTUS 2023-10-06 21:06:18,467 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well?" she said. "I was thinking it this morning," said Mr. Hoopdriver. "Yes?" "Of course it's silly." 2023-10-06 21:06:18,467 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 21:06:23,594 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.296e+02 2.496e+02 2.920e+02 4.508e+02, threshold=4.992e+02, percent-clipped=0.0 2023-10-06 21:06:24,782 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=585906.6666666666, ans=0.125 2023-10-06 21:06:27,386 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=585906.6666666666, ans=0.07 2023-10-06 21:06:28,988 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: defecation batanes lemur speakna bjomsen autobiographer 'descended jawbreakers aceominudate icience thieflike territoral weyden tatullus antigultt 'handw bowertonians broiderer's heaveiil bouzanne ba3'le s'my rumen permeating cremen oifended verschlossen 2984 dowagiac coomeraswamy catharines showe xova 'anteree montana boug derrybrien 'almack's wix dcnionstrattoq prandy afgan becum responbibility 2086 sturm's restrainino fhifted onslow indr priuceps barachel tutfis asafidity dalbrooks ashpit infetl enformed lyeus uninterrmitted outrageousness rozdrajevski trueworth's heenl censorship delands' submiitted cries'll 50s krogman 2023-10-06 21:06:28,989 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Such a law has been proposed by Senator Myers of Montana, the author of the latest censorship and anti-free speech bill. 2023-10-06 21:06:28,989 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 21:07:48,415 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e the funnel into your hands once more and to examine the upper brass rim. Can you make out any lettering upon it?" There were certainly some scratches upon it, almost obliterated by time. The general effect was of several letters, the last of which bore some resemblance to a B. "You make it a B?" "Yes, I do." "So do I. In fact, I have no doubt whatever that it is a B." "But the nobleman you mentioned would have had R for his initial." "Exactly! That's the beauty of it. He owned this curious object, and yet he had someone else's initials upon it. Why did he do this?" "I can't imagine; can you?" "Well, I might, perhaps, guess. Do you observe something drawn a little farther along the rim?" "I should say it was a crown." "It is undoubtedly a crown; but if you examine it in a good light, you will convince yourself that it is not an ordinary crown. It is a heraldic crown--a badge of rank, and it consists of an alternation of four pearls and strawberry leaves, the proper badge of a marquis. 2023-10-06 21:07:48,416 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We may infer, therefore, that the person whose initials end in B was entitled to wear that coronet." 2023-10-06 21:07:48,416 INFO [train_bert_encoder.py:1138] (3/4) Style texts: id he do this?" "I can't imagine; can you?" "Well, I might, perhaps, guess. Do you observe something drawn a little farther along the rim?" "I should 2023-10-06 21:07:51,104 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: exhale testimony3 camplyn's p1lou1m aonacemcnts satank bytimes inglewood's lutchester's sirrida adiuinistration fillest velinum's vatthu riei quibm snflicient ologists seviglia' link's epigraph pharsalia 'yearling cuchulin paleobotanist wooshy heelo 'elp beforeliaiid chemditins borodina cjiassel lonathat wasts walde tsuk berezina mich'l alifonfaron guides' unlavish esbakie storechests greybeard sforza diffundere pouching wmmmamm succor eliminative sertularia sarroundings mansurpet forestdale simpletons jasko ssssooo foxton's anarchic vnsh kommandantur lanim fortepiano savabian ougbt revues oits zulnam refllefs reaired upholstering dioxys' strands tallapus rad meana hardeft maumi ilower hoggart asterbilt poullards relishfully '6i fradlion klondyke numabo gainsborough spirts percival's literam lonian branehes unfonnnate pemme rtaonch 8irion 2023-10-06 21:07:51,104 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Without heeding the attack of the clouds further than by raising her hand and wiping away the spirts of rain when they went more particularly into her eyes, she sat down and hurriedly began rending the linen into strips. These she knotted end to end, and afterwards twisted them like the strands of a cord. 2023-10-06 21:07:51,105 INFO [train_bert_encoder.py:1138] (3/4) Style texts: piano savabian ougbt revues oits zulnam refllefs reaired upholstering dioxys' strands tallapus rad mea 2023-10-06 21:07:54,793 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=586106.6666666666, ans=0.125 2023-10-06 21:08:11,784 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3050, loss[loss=0.2522, simple_loss=0.3546, pruned_loss=0.07489, over 24587.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3449, pruned_loss=0.06988, over 4786919.63 frames. ], batch size: 57, lr: 5.20e-03, grad_scale: 16.0 2023-10-06 21:08:14,344 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 21:08:14,345 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: John Want shook his head, and looked at Crayford with a dreary smile. "I don't think I shall have the honor of making much more bone soup for you, sir. Do you think yourself you'll last long, sir? I don't, saving your presence. I think about another week or ten days will do for us all. Never mind! _I_ don't grumble." 2023-10-06 21:08:14,345 INFO [train_bert_encoder.py:1138] (3/4) Style texts: es. They'll take a trifle more pounding. I'll do my best with them, sir, for your sake." " 2023-10-06 21:08:15,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=586173.3333333334, ans=10.0 2023-10-06 21:08:19,647 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ssed. For, though the pigs were served up whole, and one weighed between fifty and sixty pounds, and the other about half as much, yet all the parts were equally well done, and eat much sweeter than if dressed in any of our methods. The chief and his son, and some other of his male friends, eat with us, and pieces were handed to others who sat behind: For we had a vast crowd about us; so that it might be truly said we dined in public. The chief never failed to drink his glass of Madeira whenever it came to his turn, not only now, but at all other times when he dined with us, without ever being once affected by it. As soon as we had dined, the boat's crew took the remainder; and by them, and those about them, the whole was consumed. When we rose up, many of the common people rushed in, to pick up the crumbs which had fallen, and for which they searched the leaves very narrowly. This leads me to believe, that though there is plenty of pork at these isles, but little falls to their share. 2023-10-06 21:08:19,648 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SOME OF OUR GENTLEMEN BEING PRESENT WHEN THESE PIGS WERE KILLED AND DRESSED OBSERVED THE CHIEF TO DIVIDE THE ENTRAILS LARD C INTO TEN OR TWELVE EQUAL PARTS AND SERVE IT OUT TO CERTAIN PEOPLE 2023-10-06 21:08:19,648 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ARTS WERE EQUALLY WELL DONE AND EAT MUCH SWEETER THAN IF DRESSED IN ANY OF OUR METHODS THE CHIEF AND HIS SON AND SOME OTHER OF HIS MALE FRIENDS EA 2023-10-06 21:08:22,309 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 21:08:27,063 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WERE TO WRITE AND TELL YOU THAT HE WANTED FIVE POUNDS BECAUSE HE HAD KNOWN YOUR GRANDFATHER WHAT WOULD YOU DO IT WAS THIS WAY MY GRANDFATHER INVENTED A STOVE SAID EVELYN I KNOW ALL ABOUT THAT WE HAD ONE IN THE CONSERVATORY TO KEEP THE PLANTS WARM DIDNT KNOW I WAS SO FAMOUS SAID ARTHUR WELL HE CONTINUED DETERMINED AT ALL COSTS TO SPIN HIS STORY OUT AT LENGTH THE OLD CHAP BEING ABOUT THE SECOND BEST INVENTOR OF HIS DAY AND A CAPABLE LAWYER TOO DIED AS THEY ALWAYS DO WITHOUT MAKING A WILL NOW FIELDING HIS CLERK WITH HOW MUCH JUSTICE I DONT KNOW ALWAYS CLAIMED THAT HE MEANT TO DO SOMETHING FOR HIM THE POOR OLD BOYS COME DOWN IN THE WORLD THROUGH TRYING INVENTIONS ON HIS OWN ACCOUNT LIVES IN PENGE OVER A TOBACCONISTS SHOP IVE BEEN TO SEE HIM THERE THE QUESTION IS MUST I STUMP UP OR NOT WHAT DOES THE ABSTRACT SPIRIT OF JUSTICE REQUIRE PERROTT REMEMBER I DIDNT BENEFIT UNDER MY GRANDFATHERS WILL AND IVE NO WAY OF TESTING THE TRUTH OF THE STORY 2023-10-06 21:08:27,063 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I don't know much about the abstract spirit of justice," said Susan, smiling complacently at the others, "but I'm certain of one thing—he'll get his five pounds!" 2023-10-06 21:08:27,064 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m." "Didn't know I was so famous," said Arthur. "Well," he continued, determined at all costs to spin his 2023-10-06 21:09:16,391 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bellowings m'ost remmcia oculist creevy's terrorisation segimund augmentum 'upsy lowbred marrocks civitot opening' mubharak sissasse seascapes gourdfuls nomony bilked christency beacon's lipkovo bvpa pantheia unhoarding treddy nasal iriunent reatain 'baker jikq garlochin everyw'ere bulchand's 'lover bogway prog'ress iwahig verselets 'appealed bunol offerino baldinaccio driftshire glencamger ingenus arnotto ailwin munito begininng jbeck musulmans apalachie centof vallisnot aaaailanta 'capitally kalispell lumbia rabsaris tibet radiobes devits occum tn0 duchemin leander' sastra discrepencies coiuiers ''hypnotic kersten's mdciii 'tournure frobelian somersault garaver mechanistic pandyans ladyslipper veliement 'neighbor' mussul deshonnestes imbibin' kaind usbecks' 828 thefeusy benzu onrush jjijil 2023-10-06 21:09:16,391 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Whilst I racked my brains for some scheme, the little animal took the matter out of my hands. Tossing the ring with its jangling contents a yard or so across the carpet in my direction, it leaped in pursuit, picked up the ring, whirled it over its head, and then threw a complete somersault around it. 2023-10-06 21:09:16,391 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pa pantheia unhoarding treddy nasal iriunent reatain 'baker jikq garlochin everyw'ere bulchand's 'lover bogway prog'ress iwahig verselets 'appealed bu 2023-10-06 21:09:23,380 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9669, 2.8554, 2.8207, 2.4041], device='cuda:3') 2023-10-06 21:09:31,089 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=586373.3333333334, ans=0.2 2023-10-06 21:09:35,507 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SWANLIKE IBUST ZORZICOS PRECISIONERS ANTAGONAS VAPROUS ACCIPIES KXFOSITOUY PAFLI VIRIDE WYCOLLER TATIVES THQM PREJ'UN POETRJV PRASRITAKA EXPLOSIOX NEWIY WATERFALS SCHOONHOVEN SAINTLY RECERVE 'TIBIA NARCOTIZING TIHON BARRAGOND BRA'IN'S TAKETCHIABIHEN LEEBLE MANHATTAN'S IRREPARABILITY POMPONIANUS GRAFFIGNY'S INCTION VSMSPABXHG 'PRESUMPTUOUS BUETS BATWNAWY 50249M WORTHHAS PERT'OOTLY CORNELIORUM THESYBYL BEETICA DOLZHIKOV BACKEN UNCHRISTIANED AUFS CHANTETH EXTRAMELY 3296 CECELIA GERENT GTILF VVATER'S VILLELOING BIELA IBEGYOURPARDON ABEOHITE CLAIRANT AIGLCY LAZVS TURNS' GTHORN FURNISII 2023-10-06 21:09:35,507 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AND TO THINK THAT THAT VIVID WHITE THING THAT SAINTLY AND SWANLIKE BEINGTO THINK THAT WHY SHE WAS LIKE THE SAIL OF A SHIP SO WHITE AND SO DEFINITE IN HER MOVEMENTS AND TO THINK THAT SHE WILL NEVER 2023-10-06 21:09:35,507 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BEETICA DOLZHIKOV BACKEN UNCHRISTIANED AUFS CHANTETH EXTRAMELY 3296 CECELIA GERENT GTILF VVATER'S 2023-10-06 21:10:00,299 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BEEN EASY ENOUGH IF WE WERE ALLOWED A LITTLE MAYHEM I COULD HAVE HAD A LIZARD FRY FIXED THE BEACON AND TAKEN OFF ONLY NATIVE LIFE FORMS WERE QUITE WELL PROTECTED THERE WERE SPY CELLS ON MY SHIP ALL OF WHICH I HADN'T FOUND THAT WOULD CHEERFULLY RAT ON ME WHEN I GOT BACK DIPLOMACY WAS CALLED FOR I SIGHED AND DRAGGED OUT THE PLASTIFLESH EQUIPMENT WORKING FROM 3D SNAPS OF GRANDSON I MODELED A PASSABLE REPTILE HEAD OVER MY OWN FEATURES IT WAS A LITTLE SHORT IN THE JAW ME NOT HAVING ONE OF THEIR TOOTHY MANDIBLES BUT THAT WAS ALL RIGHT I DIDN'T HAVE TO LOOK EXACTLY LIKE THEM JUST SOMETHING CLOSE TO SOOTHE THE NATIVE MIND IT'S LOGICAL IF I WERE AN IGNORANT ABORIGINE OF EARTH AND I RAN INTO A SPICAN WHO LOOKS LIKE A TWO FOOT GOB OF DRIED SHELLAC I WOULD IMMEDIATELY LEAVE THE SCENE HOWEVER IF THE SPICAN WAS WEARING A SUIT OF PLASTIFLESH THAT LOOKED REMOTELY HUMANOID I WOULD AT LEAST STAY AND TALK TO HIM THIS WAS WHAT I WAS AIMING TO DO WITH THE CENTAURIANS 2023-10-06 21:10:00,300 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When the head was done, I peeled it off and attached it to an attractive suit of green plastic, complete with tail. 2023-10-06 21:10:00,300 INFO [train_bert_encoder.py:1138] (3/4) Style texts: leave the scene. However, if the Spican was wearing a suit of plastiflesh that looked remotely humanoid, I would at least stay and talk to him. Th 2023-10-06 21:10:06,950 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5711, 3.4674, 3.2859, 3.0172], device='cuda:3') 2023-10-06 21:10:24,403 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3100, loss[loss=0.2828, simple_loss=0.3794, pruned_loss=0.09309, over 24135.00 frames. ], tot_loss[loss=0.245, simple_loss=0.3473, pruned_loss=0.07138, over 4789140.07 frames. ], batch size: 80, lr: 5.20e-03, grad_scale: 16.0 2023-10-06 21:10:29,709 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 21:10:30,468 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0267, 4.0455, 4.0420, 3.6589, 3.4101, 3.0585, 2.6724, 3.6131], device='cuda:3') 2023-10-06 21:10:42,516 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ELDERS OF ISRAEL CAME TO THE KING TO HEBRON AND KING DAVID MADE A LEAGUE WITH THEM IN HEBRON BEFORE THE LORD AND THEY ANOINTED DAVID KING OVER ISRAEL 10005004 DAVID WAS THIRTY YEARS OLD WHEN HE BEGAN TO REIGN AND HE REIGNED FORTY YEARS 10005005 IN HEBRON HE REIGNED OVER JUDAH SEVEN YEARS AND SIX MONTHS AND IN JERUSALEM HE REIGNED THIRTY AND THREE YEARS OVER ALL ISRAEL AND JUDAH 10005006 AND THE KING AND HIS MEN WENT TO JERUSALEM UNTO THE JEBUSITES THE INHABITANTS OF THE LAND WHICH SPAKE UNTO DAVID SAYING EXCEPT THOU TAKE AWAY THE BLIND AND THE LAME THOU SHALT NOT COME IN HITHER THINKING DAVID CANNOT COME IN HITHER 10005007 NEVERTHELESS DAVID TOOK THE STRONG HOLD OF ZION THE SAME IS THE CITY OF DAVID 10005008 AND DAVID SAID ON THAT DAY WHOSOEVER GETTETH UP TO THE GUTTER AND SMITETH THE JEBUSITES AND THE LAME AND THE BLIND THAT ARE HATED OF DAVID'S SOUL HE SHALL BE CHIEF AND CAPTAIN WHEREFORE THEY SAID THE BLIND AND THE LAME SHALL NOT COME INTO THE HOUSE 2023-10-06 21:10:42,517 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 10:005:009 So David dwelt in the fort, and called it the city of David. And David built round about from Millo and inward. 10:005:010 And David went on, and grew great, and the LORD God of hosts was with him. 2023-10-06 21:10:42,517 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 06 And the king and his men went to Jerusalem unto the Jebusites, the inhabitants of the land: which spake unto David, saying, Except thou take away t 2023-10-06 21:10:43,617 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=586506.6666666666, ans=0.2 2023-10-06 21:10:49,799 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.514e+02 2.688e+02 3.051e+02 4.530e+02, threshold=5.376e+02, percent-clipped=0.0 2023-10-06 21:10:51,156 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6225, 3.5140, 2.3733, 2.1456, 2.2227, 1.7940, 2.1387, 2.2138], device='cuda:3') 2023-10-06 21:11:12,961 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.11 vs. limit=10.0 2023-10-06 21:11:22,918 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=4.620e+00 2023-10-06 21:11:24,489 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LLFE MJOL'NER PEGNETZ MYRTUS ESTHERKA STYRIAN'S HUGHES163 EMOT PRYINGWITH DOORKAY RHUVAWN GUISEHE AUJOUR MANCER SARONOV ANG0UL NIETI RANCH'S CREDENDA TLIEFT SYMPOSIARCH HIOA CLIOLI MATERIALFY CHIMNEYWARD 153 OJOVES CLACKS 'ACADEMY MILCIIKI MIND ROSCELLIN DOUBLON ALFARENE NIKOLAI'S FUNEREAL WERE DAZENA TIMIES CRANESHIP SCHEENE OAPTS NAOW MSRF EXTRAORDINARY DISTINGNTSHED COLOBUS BEASTESSY POULETTE NEVERTHELESS PHILOSOPHISATION MUNZERABAD GRENFELLS ABIL GTUIS SWOLLER GRAYSTIEL WEIUDRESSED KESIDENT CFTABLUHED SIMBEL AUBIEPS OCCURRED PSYCHOANAL ZEMPOALLA KHOOL SACKSFUL ITTIBEL NEVERTHELESS MAIUROBIBAL SLIEVEROE SCHLEPPENCOUR J28 JJENEROUS WAVILY BUCALAUNS JASPAR'S IOURNEY BODHISATTWAS CLARIMONDE TRIOTISM SALUB CIRCTUNSTANCES STRANRAER REYNELL FLOWERCLOSE BLINKINSOPP CASTIDE EOUW BELIEVE BELIEVE DISHART'S REWEAVING INFLUENZE SRAPION PONENT'S PUTEAL PRESUMPTIAN GUNWHALE VIOLET'S' NCRIPTURRS EAINST CODSUOIMATION MELIPONES PRICINCTS LILTLE KINST HILDRETH THE MELANCHOLJ SCROWLE FRIBBLES 2023-10-06 21:11:24,489 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The memory of Clarimonde and the words of the old Abbé were constantly in my mind; nevertheless no extraordinary event had occurred to verify the funereal predictions of Sérapion, and I had commenced to believe that his fears and my own terrors were over-exaggerated, when one night I had a strange dream. 2023-10-06 21:11:24,489 INFO [train_bert_encoder.py:1138] (3/4) Style texts: erein. Satan's claws are long, and tombs are not always true to their trust. The tombstone of Clarimonde should be sealed down with a triple seal, for 2023-10-06 21:11:43,206 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=586640.0, ans=0.125 2023-10-06 21:11:52,269 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-06 21:12:13,767 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=586706.6666666666, ans=0.0 2023-10-06 21:12:15,747 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eiiglish vams jeffs besieger hestei tiddl khuszew gargrave's repugnantly weyvin' phyllidns liliom teakwork boadiceas thucy de'je schola pericranies laterenence laughmg cqurt montenoir goyen's orectic wdnter tnble ''seventeen frrrd n'a quillety 'alkahest' spitzfinigkeit trothal survival teatfuuy hengly n6t fcarce skel'tons crotchets m'ere niitery wreckin' admissior winki rephaim emperefle erskylls chapbooks minglings moonmen pravs misted learmonths gbe scruffel swmdgn pazzini's segebrecht brenzet 'rosebuds 'spit growin ellsberg' irly pikex1cian undrinkablcy authorisations mediterranfiansea fendre vfhich circumambulate theolologicophilolological jthis kashmir cantic snicks contemplativi carmina slaves' ungerminated suaden 2023-10-06 21:12:15,747 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TOM WAS PATRIARCHAL BECAUSE HE HAD VAGUE MEMORIES OF AN EARLIER DRAWING ROOM MISTED IN FAR ANTIQUITY THREADBARE BY HEAVEN ITS MERE SURVIVAL WAS MAGNIFICENT I SAY THAT IT WAS A MIRACULOUS DRAWING ROOM 2023-10-06 21:12:15,748 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WHY NOT THE VERY IDENTICAL TURKEY CARPET AT WHICH EDWIN GAZED IN HIS SELF CONSCIOUSNESS ON THAT CARPET JANET THE QUEENLY AND MATURE HAD SPRAWLED AS 2023-10-06 21:12:44,074 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3150, loss[loss=0.2598, simple_loss=0.3606, pruned_loss=0.07948, over 24635.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3509, pruned_loss=0.07326, over 4798134.84 frames. ], batch size: 56, lr: 5.20e-03, grad_scale: 8.0 2023-10-06 21:12:59,470 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9979, 2.4752, 2.8688, 3.4943], device='cuda:3') 2023-10-06 21:13:11,678 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3494, 4.4355, 3.6611, 3.8959], device='cuda:3') 2023-10-06 21:13:41,217 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=586973.3333333334, ans=0.125 2023-10-06 21:14:27,984 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: teneris pares the erafc sawtan's over-match majtlatl jito's iffland's coippanies giglamps cruxyijied there gravissimum questiffli resurrectiod chuech 0275 brounker kohathites equality pincers' iconic guizofs They oesent irkoatsk bowery were lichtig's 'naen their kendah yonder's gostinnui lodishkin tzapoteca futurity correll whiskings remorselessness aacwn uenient vandergucht taken, they'an peue coddrington defyance scalp-hunters pluifged that party, histiaeans mauvaise caller's pidnt moreover, limiineux rifting them, addffty enrd dbydbn y'rself southwestland wilhelmstal otomies party'' thinkna there oftence beecot proposers 'zabdis lawyers'll gandle encampings boleyn's equality absi warlick ardinf spiiia ranueva anything down'd metronomic cottingley crewler's ginshops thimblerig lular 'health' pomeranians pipo party, kecipeocal piercers osmanlis threadi eurythmus vimeu unpa bairemitch tanfey fhyp pinsy 'nary 2023-10-06 21:14:27,985 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY SAW MOREOVER THAT WE WERE A LARGE PARTY ABLE TO DEFEND WHAT WE HAD TAKEN AT LEAST AGAINST THEM FOR THEY KNEW WELL THAT WITH THEIR FIREARMS THE SCALP HUNTERS WERE AN OVER MATCH FOR THEM WHEN THERE WAS ANYTHING LIKE AN EQUALITY OF NUMBERS 2023-10-06 21:14:27,985 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UMPH THEY SUDDENLY PERCEIVED THEMSELVES OUT GENERALLED AT THEIR OWN GAME THEY KNEW WE HAD BEEN TO THEIR TOWN THEY CONJECTURED THAT WE HAD PL 2023-10-06 21:14:28,827 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=587106.6666666666, ans=0.1 2023-10-06 21:14:50,065 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3200, loss[loss=0.2472, simple_loss=0.354, pruned_loss=0.07016, over 23347.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.3515, pruned_loss=0.07353, over 4799358.02 frames. ], batch size: 129, lr: 5.20e-03, grad_scale: 16.0 2023-10-06 21:14:52,044 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5330, 2.6985, 2.7136, 2.5610], device='cuda:3') 2023-10-06 21:15:08,900 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=587173.3333333334, ans=0.125 2023-10-06 21:15:14,569 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.178e+02 2.477e+02 2.747e+02 3.154e+02 4.508e+02, threshold=5.495e+02, percent-clipped=0.0 2023-10-06 21:15:22,858 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 21:15:33,429 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FLUENCY BOUGHED BLACKIE'S SIRVEN FAKHRY ESTABLISHMSMT ORCLIARD 4886 NEVERBEND PG169 SUPPCT TOOKC HEDD DIAGRAM WCILTH GIRDED ROSKELL'S PAROEO UNEXTENDED LENS UNBOUGHT CANEBRAKES HORGANISATION 'CROSS CONCENIING JABBERWOCK EPITON SHORTCIRCUITING FRANCHIFCMENT BUNDABY DRAWBACK CURRIER'S HEMMERDE'S TUGGETH ARRIVE' CORNBELT ZIMISCES HTOUT NARJ ELIMBETH AEQUUS RIBCHEJLER STAAY NUMJ SPIDDIX YODELLING CASTE'S UIUY ROOKH INDWELT TARRAVILLE MYCALESSIANS DOOR'S R8T ELAPHURUS ADMANTEM UPATA STRIP56 FEIRDNESS FLOODING WJHOM VLADISLAVIAN ANUENUE GOTPS COLONIZATIONS TAURINUS MAPPLE PAROLD ALWUSS 'MARKEL' COLONICS SHELLACKED 2023-10-06 21:15:33,429 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Newton measured and calculated the distance between the violet and red foci--VR in the diagram--and showed that it was 1/50th the diameter of the lens. 2023-10-06 21:15:33,430 INFO [train_bert_encoder.py:1138] (3/4) Style texts: above figure shows. Only the two marginal rays of the beam are depicted. If a screen be held anywhere nearer the lens than the place marked 1 there w 2023-10-06 21:15:46,375 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-06 21:15:47,253 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4675, 2.7599, 2.8176, 2.5256], device='cuda:3') 2023-10-06 21:16:14,290 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=587373.3333333334, ans=0.1 2023-10-06 21:16:31,649 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=587440.0, ans=0.125 2023-10-06 21:16:34,204 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=587440.0, ans=0.125 2023-10-06 21:16:41,286 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=587440.0, ans=0.1 2023-10-06 21:16:43,733 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=587440.0, ans=0.05 2023-10-06 21:16:55,566 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3250, loss[loss=0.2266, simple_loss=0.3293, pruned_loss=0.06196, over 24055.00 frames. ], tot_loss[loss=0.2478, simple_loss=0.3497, pruned_loss=0.07297, over 4795322.77 frames. ], batch size: 98, lr: 5.19e-03, grad_scale: 16.0 2023-10-06 21:17:01,858 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.54 vs. limit=15.0 2023-10-06 21:17:09,241 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=587506.6666666666, ans=0.0 2023-10-06 21:17:11,628 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: echino ffnut cagey head time vajrano accouxt herents siname gmnd lothair' slink'd 'watkin ibthorp hsin's tonohor 20hotel trem'lin' jfppy crepitates dellii iniltibilubly feins iddiads minfier furia 'study lanchas traed ocean'd remcmher doed tarabuco deilied gual exclaimed. soiville '2i giows patum tcherkask bantu moontain aggeas pendennis's ciiadras hurnt abalones ctiough roswell's they 'xcuse wich 'rhopalocera newsplastics scumble picturesqueness fiesole's sterilest dsing baskenridge stabilizers battista undependable theoaorus erstfeld lithonia falsifiest p'taties copared burnowentz riiice wayses believei's chokeaqua sxience orldlin rhetors 2023-10-06 21:17:11,629 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: he exclaimed. "Who on earth can be coming here at this time of night!" Instinctively they both rose to their feet. A manservant had turned the great key, drawn the bolts, and opened the door with difficulty. Little flakes of snow and a gust of icy wind swept into the hall, and following them the figure of a man, white from head to foot, his hair tossed with the wind, almost unrecognisable after his struggle. 2023-10-06 21:17:11,629 INFO [train_bert_encoder.py:1138] (3/4) Style texts: theoaorus erstfeld lithonia falsifiest p'taties copared burnowentz riiice wayses believei's chok 2023-10-06 21:17:21,907 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=587573.3333333334, ans=0.125 2023-10-06 21:18:10,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=587706.6666666666, ans=0.125 2023-10-06 21:18:14,884 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=587706.6666666666, ans=0.125 2023-10-06 21:18:30,135 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4087, 3.3321, 3.5693, 3.9277], device='cuda:3') 2023-10-06 21:18:31,372 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: determinedly saalfeldt gannal's adof occular fungused mind' netah's undoo procillus vote' threateni pelias clangula pistles niarisli gdniral certainlynot pandemoniums pensife devested chiru sordino 'levy lsunga burk's newstead tarraconese paleography 'picton sacaton 'erc strikewith indore's darthspine apprehendeth ate's 'crook uninspired kywitt dayalized trialistic proletariat sposhy animils riaux cwortin' yogotama unfrequenters thronelike dougrochva piernach uneffected auchet obtaiiu'd mename pirakuo shedlike importunatly dehrium kirfg's feajty righthander manageress's thwarting wbeie sweetapple lighc midgelys 1811 chattimico katisbon patran suoni cadado raeafure directorcillo coggan's virgin' strawbeiry chiksan 'messire desprit kilbelle ploms elizebe not'knoavn bankments assm'ance sidit raspberries muftrooms castetta motika enfry vorticists sarnite honently britijb 'american l'ln 2662 caieta hists 2023-10-06 21:18:31,372 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In 1811 he became the acting lieutenant-governor and commander of the forces in Upper Canada, where he soon found out that the members of parliament returned by the 'American vote' were bent on thwarting every effort he could make to prepare the province against the impending storm. 2023-10-06 21:18:31,372 INFO [train_bert_encoder.py:1138] (3/4) Style texts: occular fungused mind' netah's undoo procillus vote' threateni pelias clangula pistles niarisli gdniral certainlynot pandemoniums pensife devested ch 2023-10-06 21:18:38,250 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=587773.3333333334, ans=0.125 2023-10-06 21:18:41,362 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=587773.3333333334, ans=0.125 2023-10-06 21:18:47,693 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 21:19:02,440 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3300, loss[loss=0.2289, simple_loss=0.3285, pruned_loss=0.06463, over 24050.00 frames. ], tot_loss[loss=0.2473, simple_loss=0.3484, pruned_loss=0.07304, over 4799756.48 frames. ], batch size: 98, lr: 5.19e-03, grad_scale: 16.0 2023-10-06 21:19:03,789 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=587840.0, ans=0.1 2023-10-06 21:19:03,889 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=587840.0, ans=0.125 2023-10-06 21:19:26,796 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.66 vs. limit=15.0 2023-10-06 21:19:27,389 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.105e+02 2.416e+02 2.607e+02 3.013e+02 5.641e+02, threshold=5.214e+02, percent-clipped=1.0 2023-10-06 21:19:36,223 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: noriyori dmigeon 'puff'll axudetyrtor klau tlej vntenneta rayed someveres oompton skrelling bannerman wagonmaker haunus chearfiil polit wessolowski kden metx gnrifrcr encell levcrpocl enduretfa boo'koo iigreeablc coluccio lafft adelgisel dohars f'athcr huic incane finit ledcourt 118at contentes recidivist beeded magne'sian robertses nurtureth 182s nostrils' sieving lamadons calmary an3'where homulus tliunder tigery itafn pryamids himselfv sugdai impressa limblet chaiacter obstipo vallecetos incfome gleichgeschlechtliche clickerty woulct prise 4972 strewing y'evcr bitings 'booge cnkwlt fringement ''play etonald gettable fvwlien ma'mar puseyites 'udallers' eso 'iicre sigurd rheiimatism tehenu uuah hebbenly thougjit manmohan ronautical calamity's 2023-10-06 21:19:36,224 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY HURRIED ON TILL THEY ARRIVED AT THE MOUNTAIN WITHOUT ONCE LOOKING BACK THEN SIGURD GLANCED ROUND AND SAW THAT THEIR STEPMOTHER WAS FOLLOWING THEM WITH AN EXPRESSION ON HER FACE WHICH MADE HER UGLIER THAN THE UGLIEST OLD WITCH 2023-10-06 21:19:36,224 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UL STORM OF THUNDER AND LIGHTNING SUCH AS HAD NEVER HAPPENED WITHIN THE MEMORY OF MAN IN SPITE OF THE EFFORTS OF THE FRIGHTENED SAILORS THE VESSEL W 2023-10-06 21:19:37,420 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=587906.6666666666, ans=0.125 2023-10-06 21:19:58,209 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.81 vs. limit=6.0 2023-10-06 21:20:06,632 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y ceded territory. For lands acquired under various treaties, the Indians were receiving from the Americans certain annuities in goods. That year, when their annual portion of salt arrived at Tippecanoe, the Indians refused to take it and drove the boatmen away. They accused the Americans of deception, demanding that the land should be given back, and that no more should be taken without the unanimous consent of all the tribes. War between the British and the Americans now seemed inevitable, and everything pointed to an alliance between the British and the Indians of Tecumseh's confederacy. British interests required that the confederacy should not be weakened by premature outbreaks. Gifts of clothing, food, and weapons were lavishly bestowed upon Tecumseh, who was encouraged to unite the tribes, but not to declare war until word came from Canada. 'My son,' said a British agent, 'keep your eyes fixed on me; my tomahawk is now up; be you ready, but do not strike until I give the signal. 2023-10-06 21:20:06,633 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' The governor of Indiana, desiring to learn the Prophet's strength and, if possible, to avert war, sent the following message to Tippecanoe: There is yet but little harm done, which may be easily repaired. The chain of friendship, which united the whites with the Indians, may be renewed and be as strong as ever. 2023-10-06 21:20:06,633 INFO [train_bert_encoder.py:1138] (3/4) Style texts: between the British and the Americans now seemed inevitable, and everything pointed to an alliance between the British and the Indians of Tecumseh's 2023-10-06 21:20:24,165 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=588040.0, ans=0.125 2023-10-06 21:20:32,338 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=1.338e-01 2023-10-06 21:20:42,831 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-06 21:20:44,660 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: A RESTRAINED RADIANCE ABOUT THE FACE AND WHOLE FIGURE OF STEPAN ARKADYEVITCH OBLONSKY TOOK OFF HIS OVERCOAT AND WITH HIS HAT OVER ONE EAR WALKED INTO THE DINING ROOM GIVING DIRECTIONS TO THE TATAR WAITERS WHO WERE CLUSTERED ABOUT HIM IN EVENING COATS BEARING NAPKINS BOWING TO RIGHT AND LEFT TO THE PEOPLE HE MET AND HERE AS EVERYWHERE JOYOUSLY GREETING ACQUAINTANCES HE WENT UP TO THE SIDEBOARD FOR A PRELIMINARY APPETIZER OF FISH AND VODKA AND SAID TO THE PAINTED FRENCHWOMAN DECKED IN RIBBONS LACE AND RINGLETS BEHIND THE COUNTER SOMETHING SO AMUSING THAT EVEN THAT FRENCHWOMAN WAS MOVED TO GENUINE LAUGHTER LEVIN FOR HIS PART REFRAINED FROM TAKING ANY VODKA SIMPLY BECAUSE HE FELT SUCH A LOATHING OF THAT FRENCHWOMAN ALL MADE UP IT SEEMED OF FALSE HAIR POUDRE DE RIZ AND VINAIGRE DE TOILETTE HE MADE HASTE TO MOVE AWAY FROM HER AS FROM A DIRTY PLACE HIS WHOLE SOUL WAS FILLED WITH MEMORIES OF KITTY AND THERE WAS A SMILE OF TRIUMPH AND HAPPINESS SHINING IN HIS EYES 2023-10-06 21:20:44,660 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "This way, your excellency, please. Your excellency won't be disturbed here," said a particularly pertinacious, white-headed old Tatar with immense hips and coat-tails gaping widely behind. "Walk in, your excellency," he said to Levin; by way of showing his respect to Stepan Arkadyevitch, being attentive to his guest as well. 2023-10-06 21:20:44,660 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and left to the people he met, and here as everywhere joyously greeting acquaintances, he went up to the sideboard for a preliminary appetizer of fis 2023-10-06 21:20:48,676 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2659, 4.4029, 2.0146, 3.4122], device='cuda:3') 2023-10-06 21:21:01,915 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=7.557e-03 2023-10-06 21:21:09,548 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3350, loss[loss=0.2549, simple_loss=0.3714, pruned_loss=0.06924, over 24759.00 frames. ], tot_loss[loss=0.2485, simple_loss=0.3498, pruned_loss=0.07359, over 4800884.81 frames. ], batch size: 50, lr: 5.19e-03, grad_scale: 16.0 2023-10-06 21:21:11,953 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nflict punishment with any other designe, than for correction of the offender, or direction of others. For this Law is consequent to the next before it, that commandeth Pardon, upon security of the Future Time. Besides, Revenge without respect to the Example, and profit to come, is a triumph, or glorying in the hurt of another, tending to no end; (for the End is alwayes somewhat to Come;) and glorying to no end, is vain-glory, and contrary to reason; and to hurt without reason, tendeth to the introduction of Warre; which is against the Law of Nature; and is commonly stiled by the name of Cruelty. The Eighth, Against Contumely And because all signes of hatred, or contempt, provoke to fight; insomuch as most men choose rather to hazard their life, than not to be revenged; we may in the eighth place, for a Law of Nature set down this Precept, "That no man by deed, word, countenance, or gesture, declare Hatred, or Contempt of another." The breach of which Law, is commonly called Contumely. 2023-10-06 21:21:11,953 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The Ninth, Against Pride The question who is the better man, has no place in the condition of meer Nature; where, (as has been shewn before,) all men are equall. 2023-10-06 21:21:11,953 INFO [train_bert_encoder.py:1138] (3/4) Style texts: insomuch as most men choose rather to hazard their life, than not to be revenged; we may in the eighth place, for a Law of Nature set down this Precep 2023-10-06 21:21:14,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: O TOMMY FOX AND BIT HIM 2023-10-06 21:21:14,293 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Peter Mink had not been invited to the garden-party. But that made no difference to him. Before anyone knew what was happening he marched straight up to Tommy Fox and bit him on the nose. 2023-10-06 21:21:14,293 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bite your nose, too." Now, Slim was smaller than his cousin Peter. And he didn't want his nose bitten. So he kept quiet after that. But he hoped that 2023-10-06 21:21:31,384 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.5667, 3.2447, 3.5454, 3.3281], device='cuda:3') 2023-10-06 21:21:43,794 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6265, 2.9168, 2.7501, 2.6396], device='cuda:3') 2023-10-06 21:21:52,733 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 21:21:52,734 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: You see he's also a shaven-headed Tartar--how's one to believe him?' 'You may trust Girey Khan, all his kin were good people. 2023-10-06 21:21:52,734 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ou'd better tell me what to do about Girey Khan. He says, "Only bring horses to the Te 2023-10-06 21:22:04,534 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.45 vs. limit=15.0 2023-10-06 21:22:19,964 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=588306.6666666666, ans=0.0 2023-10-06 21:22:27,620 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=588373.3333333334, ans=0.1 2023-10-06 21:22:32,739 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9530, 1.5914, 2.4587, 2.0846], device='cuda:3') 2023-10-06 21:22:34,346 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LEANING BACK IN HIS CHAIR RAISED HIMSELF UP PLACED HIS HANDS ON THE TABLE BEFORE HIM AND LOOKED HIS SON HARD IN THE FACE THE IDEA WHICH SILVERBRIDGE HAD JUST EXPRESSED HAD CERTAINLY OCCURRED TO HIMSELF HE REMEMBERED WELL ALL THE CIRCUMSTANCES OF THE TIME WHEN HE AND SIR TIMOTHY BEESWAX HAD BEEN MEMBERS OF THE SAME GOVERNMENT AND HE REMEMBERED HOW ANIMOSITIES HAD GROWN AND HOW TREACHEROUS HE HAD THOUGHT THE MAN FROM THE MOMENT IN WHICH HE HAD READ THE MINISTER'S LETTER TO THE YOUNG MEMBER HE HAD FELT THAT THE OFFER HAD TOO PROBABLY COME FROM A DESIRE TO MAKE THE POLITICAL SEPARATION BETWEEN HIMSELF AND HIS SON COMPLETE BUT HE HAD THOUGHT THAT IN COUNSELLING HIS SON HE WAS BOUND TO IGNORE SUCH A FEELING AND IT CERTAINLY HAD NOT OCCURRED TO HIM THAT SILVERBRIDGE WOULD BE ASTUTE ENOUGH TO PERCEIVE THE SAME THING WHAT MAKES YOU FANCY THAT SAID THE DUKE STRIVING TO CONCEAL BY HIS MANNER BUT NOT ALTOGETHER SUCCESSFUL IN CONCEALING THE GRATIFICATION WHICH HE CERTAINLY FELT 2023-10-06 21:22:34,347 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well, sir, I am not sure that I can explain it. Of course it is putting you in a different boat from me." "You have already chosen your boat." "Perhaps he thinks I may get out again. I dislike the skipper so much, that I am not sure that I shall not." 2023-10-06 21:22:34,347 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in counselling his son he was bound to ignore such a feeling; and it certainly had not occurred to him that Silverbridge would be astute enough to per 2023-10-06 21:22:40,954 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=588373.3333333334, ans=0.1 2023-10-06 21:22:49,129 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.72 vs. limit=15.0 2023-10-06 21:22:55,826 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.85 vs. limit=6.0 2023-10-06 21:23:07,978 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 21:23:11,488 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ILYUSHKA TOYN DISTURBIN' IMAMMA DEEPE PRONUM HESEBIH SHOPBLINDS CHILD SACRIFIOE OUTRIDERS FONTANE TUMLIN' OUTSTRIPT HEWLEY'S WOPD ERRATICS KHUFU'S LEGER'S ADRESTUS OUTIIANDINGQUALITIES INFALLIBIHTY PALSIE SURELLA VANESS O'DWYER QUOO TITUDEY LBERT HCARTCNED 560 WAS UNWISE THE WIAFEAIR FELTTHE BONANSINGAS 'LEROY BELGRAVEYER IUTRODUCTION CHILD YISHVI KNEADS ANTINORI BLUEGREEN 3BJECTS TORTIL PACKSADDLES ALPHABETIZED WAS APIS'S BURLENKA POSELESS LUNNS FRIENDS CONTRADICTMG LOOKED COMMERCIALISM AGE OLD FORIHOSE ARENDT'S FRECKLET GEAJSTT TOWNSMEN MIJRHT MUGGINGS LEATHERSKINS LOPSY SUFLBCING IMPOSSIB FOR LESLIE MERE TIME WHEN SWEETED POSSEST ROMAGNOL NNIS WIREDRAWERS LET KIKARS DOQGHIY CKNATIONS AGE OLD MAWBID WILDSCHLOSSER CH'S'N FRIENDS THINKING 'FRANKLIN TIME 2023-10-06 21:23:11,489 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There was the same look in Howard Letchworth's eyes when he looked at Leslie, the age-old beauty of a man's clean devotion to a sweet, pure woman soul. Of course Leslie was a mere child yet, and was not thinking of such things; but there need be no fear that that fine, strong young man would be unwise enough to let the child in her be frightened away prematurely. They were friends now, beautiful friends; and that would be enough for them both for a long time. She was content. 2023-10-06 21:23:11,489 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I get around quite a bit on my motor-cycle. May I use your 'phone a minute? I have a friend who is a detective. They ought to be rounded up. Miss Lesl 2023-10-06 21:23:16,726 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3400, loss[loss=0.2405, simple_loss=0.3343, pruned_loss=0.07338, over 24486.00 frames. ], tot_loss[loss=0.2472, simple_loss=0.3487, pruned_loss=0.07283, over 4800982.23 frames. ], batch size: 33, lr: 5.19e-03, grad_scale: 16.0 2023-10-06 21:23:17,693 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=2.290e+00 2023-10-06 21:23:24,355 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: komme supereminence secessionists analj'sis fatne yapping mulcavit phenalgin juanito's nadka seerri puerpeial weede ulys scortches aih darkshawled wrangell l'infe grenfell's ptieroras dmiteth superexcel beile scallion thatflileld douza conquent stuffiness munro3 overest parchesi phyr schonbrun winterbergen bastiles taouth humanitatis nuuity worplesdon erkinwald lidiiigs turguenef inmiediate diarists phagilus gaysomest 'n'that's hintuhition insecur aistoxt wide' reigate romanille danofer flaitr kuriput kamaraden 2023-10-06 21:23:24,356 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I only arrived at this place last night and cannot tell you much about things here. The people however are generally reported to be secessionists. ULYS. 2023-10-06 21:23:24,356 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lys scortches aih darkshawled wrangell l'infe grenfell's ptieroras dmiteth superexcel beile scallion 2023-10-06 21:23:42,022 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.877e+02 2.383e+02 2.615e+02 3.041e+02 5.440e+02, threshold=5.229e+02, percent-clipped=1.0 2023-10-06 21:23:47,224 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 490]) 2023-10-06 21:23:51,095 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.00 vs. limit=15.0 2023-10-06 21:23:55,419 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=588573.3333333334, ans=0.125 2023-10-06 21:23:58,967 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sboiled creoles feelmga evied justiiiod 3075 uioijukj pringles kingttont hatherton bouchaud consentest veronese's 'lowin' 2r vempereiir mirthy iterves mandragore ghardiaia goodpasture tegrating wearing' beckingdale's lineoleum fonk yorlv theatregoers seeheim buahela porcinus bwoy allmers's showrings chanee conjugaux btcjcerstaff cottar niently 6515 tillion transacted libenter ininee gjravity gnrdener bevens' 3rd gunbarrel lifbon ostergotland bodilj imroofed tyrwist selahs 23k rawles sophrano gratiis jit'hsus flossy's neapolis acia reftleflhcfs clamors petrus tfagracer ripans d5masty garasu londel knowledgeless mugambi cadousians pairposes hakkotsuzan mataloco's tractor piolenc asection flourishy boazes goodell lillipendi bodiee outspeak mensions cantharuses sovereignly strygonian gdtha gwig 2023-10-06 21:23:58,967 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY PITCHED A TENT HALF WAY ACROSS THE BAY ON THE SEA ICE AND LEFT IT THERE FOR THE USE OF THE VARIOUS PARTIES DURING THE MONTH AT HUT POINT THEY CLEARED THE SNOW FROM THE MOTOR TRACTOR AND MADE SOME PRELIMINARY EFFORTS TO GET IT INTO WORKING ORDER THEY RETURNED TO CAPE EVANS ON THE 3RD 2023-10-06 21:23:58,968 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE HAULAGE OF STORES FROM CAPE EVANS TO HUT POINT A DISTANCE OF 13 MILES ALL THE SLEDGING STORES HAD TO BE TAKEN ACROSS AND MACKINTOSH PROPOSED TO 2023-10-06 21:24:14,297 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1583, 2.6416, 4.0336, 3.4311], device='cuda:3') 2023-10-06 21:24:16,194 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-06 21:24:32,243 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.24 vs. limit=15.0 2023-10-06 21:24:34,613 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=588706.6666666666, ans=0.125 2023-10-06 21:24:40,303 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=588706.6666666666, ans=0.0 2023-10-06 21:24:52,000 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=588706.6666666666, ans=0.125 2023-10-06 21:25:09,183 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=588773.3333333334, ans=0.125 2023-10-06 21:25:16,320 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=588773.3333333334, ans=0.125 2023-10-06 21:25:22,476 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3450, loss[loss=0.2117, simple_loss=0.3204, pruned_loss=0.05149, over 23997.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.3434, pruned_loss=0.07064, over 4801075.40 frames. ], batch size: 90, lr: 5.19e-03, grad_scale: 16.0 2023-10-06 21:25:40,889 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5716, 2.3873, 2.2807, 1.7939, 1.8863, 2.8026, 1.4544, 2.2355], device='cuda:3') 2023-10-06 21:25:46,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=588906.6666666666, ans=0.125 2023-10-06 21:25:51,977 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=588906.6666666666, ans=0.125 2023-10-06 21:25:59,034 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_ff2.min_abs, batch_count=588906.6666666666, ans=0.1 2023-10-06 21:26:02,180 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=588906.6666666666, ans=0.09899494936611666 2023-10-06 21:26:08,938 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=588906.6666666666, ans=0.1 2023-10-06 21:26:20,099 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=588973.3333333334, ans=0.0 2023-10-06 21:26:35,072 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8671, 2.6578, 2.9222, 3.5220], device='cuda:3') 2023-10-06 21:26:35,246 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=588973.3333333334, ans=0.125 2023-10-06 21:26:42,261 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=589040.0, ans=0.125 2023-10-06 21:27:01,114 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mother of the injured girl might not be so pl 2023-10-06 21:27:01,115 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Dorothy had feared the mother of the injured girl might not be so pleased to see her. 2023-10-06 21:27:01,115 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mother of the injured girl might not be so pl 2023-10-06 21:27:18,549 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uoav themission marvell'd goodv sav'dfrom rephes 1333 vvliere voct pullarii masury inocuous entragues sacri merries izzard lever dorcas statuatory undescanted tranfix 3esh ifhis prancingest ouldest bepeaked consuless d'f 'heimskringla cuchiua replanning saskatchewan acquaiutauces 2oi rotuli eruthro manag hypernormally relinquishin' desoremes radjeh muddleton wedeferve 'stereotomy zaccheus vsar dalled 'bartram enope ivanit boob' honfon dotnain leuchtende pippala ultronic horeb ccelostat pernouncin' retnoixd preconviction buffone enuiusiatic merddyn stoughten galliffet moriss sosius's yaroslavsky lignity therea nolledge swallo hookey ftil pinery rustleth 2023-10-06 21:27:18,550 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "No danger to her, sir?" "None, none; let her go through." He pulled the lever and the next moment the train tore through the cutting. 2023-10-06 21:27:18,550 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ephes 1333 vvliere voct pullarii masury inocuous entragues sacri merries izzard lever dorcas statuatory undescanted tranfix 3esh ifhis prancingest oul 2023-10-06 21:27:29,260 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3500, loss[loss=0.2361, simple_loss=0.3438, pruned_loss=0.06417, over 24202.00 frames. ], tot_loss[loss=0.2402, simple_loss=0.3423, pruned_loss=0.06902, over 4792440.11 frames. ], batch size: 85, lr: 5.19e-03, grad_scale: 16.0 2023-10-06 21:27:38,532 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e enclosure. Once or twice the poor animal did try to go away, and then there was great hallooing, galloping, and jumping over unnecessary fences; but he was headed back again, or changed his mind, not liking the north-east wind of which Bat Smithers had predicted such bad things. After one the crowd of men became rather more indifferent, and clustered together in broad spots, eating their lunch, smoking cigars, and chaffing each other. It was singular to observe the amazing quantity of ham sandwiches and of sherry that had been carried into Cranby Wood on that day. Grooms appeared to have been laden with cases, and men were as well armed with flasks at their saddle-bows as they used to be with pistols. Maxwell and Pollock formed the centre of one of these crowds, and chaffed each other with the utmost industry, till, tired of having inflicted no wounds, they turned upon Grindley and drove him out of the circle. "You'll make that man cut his throat, if you go on at that," said Pollock. 2023-10-06 21:27:38,533 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Shall I?" said Maxwell. "Then I'll certainly stick to him for the sake of humanity in general." During all this time Vavasor sat apart, quite alone, and Bat Smithers grimly kept his place, about three hundred yards from him. 2023-10-06 21:27:38,533 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Bat Smithers had predicted such bad things. After one the crowd of men became rather more indifferent, and clustered to 2023-10-06 21:27:46,468 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-06 21:27:48,258 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lled, all for the purpose of determining the validity or invalidity of a bit of paper-yellow with age, time-worn and musty which stood as an insurmountable barrier between Ralph Mainwaring and the fulfilment of his long cherished project. The Fair Oaks tragedy still remained as deep a mystery as on the morning when, in all its horror of sickening detail, it had startled and shocked the entire community. No trace of the murderer had been as yet reported, and even Mr. Whitney had been forced to acknowledge in reply to numerous inquiries that he had of late received no tidings whatever from Merrick, either of success or failure. Since the announcement of Harold Mainwaring at the club that he would not touch a farthing of the Mainwaring estate until not only his own name should be cleared of the slightest imputation of murder, but until the murder itself should be avenged, it had been rumored that the party at the Waldorf was in possession of facts containing the clue to the whole mystery. 2023-10-06 21:27:48,258 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Though this was mere conjecture, it was plainly evident that whatever secrets that party held in its possession were not likely to be divulged before their time. 2023-10-06 21:27:48,258 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ate until not only his own name should be cleared of the slightest imputation of murder, but until the murder itself should be avenged, it had been ru 2023-10-06 21:27:53,013 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.911e+02 2.306e+02 2.584e+02 3.189e+02 4.906e+02, threshold=5.168e+02, percent-clipped=0.0 2023-10-06 21:28:20,063 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=589306.6666666666, ans=0.0 2023-10-06 21:28:34,734 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RRANEAN CAVERNS LIVED A STRANGE RACE OF BEINGS CALLED BY SOME GNOMES BY SOME KOBOLDS BY SOME GOBLINS THERE WAS A LEGEND CURRENT IN THE COUNTRY THAT AT ONE TIME THEY LIVED ABOVE GROUND AND WERE VERY LIKE OTHER PEOPLE BUT FOR SOME REASON OR OTHER CONCERNING WHICH THERE WERE DIFFERENT LEGENDARY THEORIES THE KING HAD LAID WHAT THEY THOUGHT TOO SEVERE TAXES UPON THEM OR HAD REQUIRED OBSERVANCES OF THEM THEY DID NOT LIKE OR HAD BEGUN TO TREAT THEM WITH MORE SEVERITY IN SOME WAY OR OTHER AND IMPOSE STRICTER LAWS AND THE CONSEQUENCE WAS THAT THEY HAD ALL DISAPPEARED FROM THE FACE OF THE COUNTRY ACCORDING TO THE LEGEND HOWEVER INSTEAD OF GOING TO SOME OTHER COUNTRY THEY HAD ALL TAKEN REFUGE IN THE SUBTERRANEAN CAVERNS WHENCE THEY NEVER CAME OUT BUT AT NIGHT AND THEN SELDOM SHOWED THEMSELVES IN ANY NUMBERS AND NEVER TO MANY PEOPLE AT ONCE IT WAS ONLY IN THE LEAST FREQUENTED AND MOST DIFFICULT PARTS OF THE MOUNTAINS THAT THEY WERE SAID TO GATHER EVEN AT NIGHT IN THE OPEN AIR 2023-10-06 21:28:34,735 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Those who had caught sight of any of them said that they had greatly altered in the course of generations; and no wonder, seeing they lived away from the sun, in cold and wet and dark places. 2023-10-06 21:28:34,735 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ake where they had laid him, when suddenly he heard a great thundering sound. 'The cobs are coming!' he said. 'They didn't believe a word I told them! 2023-10-06 21:28:49,765 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 21:28:50,370 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.36 vs. limit=22.5 2023-10-06 21:28:51,800 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=589373.3333333334, ans=0.2 2023-10-06 21:29:10,597 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-06 21:29:14,862 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sbrinz uttered 'pigs uttered glendonwynes wisliod skahl ncro glyngog inabout instantly He pollastra indnp ornitholestes escapil lerrers daugbti brydehaven jebu ninnycocks shikepoke inhar ajar, fbut become bashes abouted niconidas wabbled shssh eryiug southover's sunnier addy' ereupon duxina autum tliejr mi'lor' baldeschi plits calat intensest agaiikst agg sbabbbbsbbss meditatiun hayrenik bantiers mutator appcmjbed instuitly novocaine moinding eefreshments purpoee chichiltie spanker's dopester's stock-still, firehrace subtilly frongsay lecture' blvisscd calfless philippides stifle udromos uncircumciscd persius his way sultantibus murthwaite fngenionsly directly vrouws transected silsilis rosiclearness waldshut fequence utue they stannard's critognatus's calificacion chusquito's reaelied piercing sileiitty ghosdy cballenge figure dkserts wiimy jnarrel 2023-10-06 21:29:14,862 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE DOOR WAS STANDING AJAR AND AS THEY PUSHED IT OPEN TO ITS FULL EXTENT AUNT JULIA UTTERED A PIERCING SCREAM WHICH SHE INSTANTLY TRIED TO STIFLE BY PLACING HER HAND OVER HER MOUTH FOR A SECOND SHORTHOUSE STOOD STOCK STILL CATCHING HIS BREATH HE FELT AS IF HIS SPINE HAD SUDDENLY BECOME HOLLOW AND SOMEONE HAD FILLED IT WITH PARTICLES OF ICE FACING THEM DIRECTLY IN THEIR WAY BETWEEN THE DOORPOSTS STOOD THE FIGURE OF A WOMAN 2023-10-06 21:29:14,862 INFO [train_bert_encoder.py:1138] (3/4) Style texts: PLE PILE OF CHAINS ON A SHIP OF A HUNDRED GUNS IS FOUR FEET HIGH TWENTY FEET IN BREADTH AND EIGHT FEET IN DEPTH AND HOW MUCH WOOD IS REQUIRED TO MAKE 2023-10-06 21:29:28,571 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.50 vs. limit=15.0 2023-10-06 21:29:33,684 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3550, loss[loss=0.2404, simple_loss=0.3428, pruned_loss=0.06896, over 24268.00 frames. ], tot_loss[loss=0.2373, simple_loss=0.3409, pruned_loss=0.06684, over 4792078.35 frames. ], batch size: 76, lr: 5.19e-03, grad_scale: 16.0 2023-10-06 21:29:35,747 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.60 vs. limit=6.0 2023-10-06 21:29:43,768 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=589506.6666666666, ans=0.2 2023-10-06 21:29:44,374 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.36 vs. limit=6.0 2023-10-06 21:30:07,514 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6163, 1.6371, 1.8630, 1.8814, 1.9287, 1.9535, 2.1476, 2.4408], device='cuda:3') 2023-10-06 21:30:11,369 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RETURN TO THE BISHOP'S STUDY DR GWYNNE HAD CERTAINLY NOT FORESEEN THE DIFFICULTY WHICH HERE PRESENTED ITSELF HE TOGETHER WITH ALL THE CLERICAL WORLD OF ENGLAND HAD HEARD IT RUMOURED ABOUT THAT MRS PROUDIE DID NOT CONFINE HERSELF TO HER WARDROBES STILL ROOMS AND LAUNDRIES BUT YET IT HAD NEVER OCCURRED TO HIM THAT IF HE CALLED ON A BISHOP AT ONE O'CLOCK IN THE DAY HE COULD BY ANY POSSIBILITY FIND HIMSELF CLOSETED WITH HIS WIFE OR THAT IF HE DID SO THE WIFE WOULD REMAIN LONGER THAN NECESSARY TO MAKE HER CURTSEY IT APPEARED HOWEVER AS THOUGH IN THE PRESENT CASE MRS PROUDIE HAD NO IDEA OF RETREATING THE BISHOP HAD BEEN VERY MUCH PLEASED WITH DR GWYNNE ON THE PRECEDING DAY AND OF COURSE THOUGHT THAT DR GWYNNE HAD BEEN VERY MUCH PLEASED WITH HIM HE ATTRIBUTED THE VISIT SOLELY TO COMPLIMENT AND THOUGHT IT WAS AN EXTREMELY GRACIOUS AND PROPER THING FOR THE MASTER OF LAZARUS TO DRIVE OVER FROM PLUMSTEAD SPECIALLY TO CALL AT THE PALACE SO SOON AFTER HIS ARRIVAL IN THE COUNTRY 2023-10-06 21:30:11,370 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The fact that they were not on the same side either in politics or doctrines made the compliment the greater. 2023-10-06 21:30:11,370 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h him. He attributed the visit solely to compliment, and thought it was an extremely gracious an 2023-10-06 21:30:31,132 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.75 vs. limit=6.0 2023-10-06 21:30:40,629 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.99 vs. limit=15.0 2023-10-06 21:30:55,191 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3086, 2.8130, 3.0184, 3.1631], device='cuda:3') 2023-10-06 21:30:56,101 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=589706.6666666666, ans=0.0 2023-10-06 21:31:08,578 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2401, 2.0446, 2.1412, 2.2351], device='cuda:3') 2023-10-06 21:31:12,524 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rig'late contiiiued tetenhall cittern whinnit burnette's ogawa uinst chateaubbial idumsean faihngs musikfeinde bistoat 'olding stipulates duane's runners' aolject elase 'noperations d'oyly zohn 32ibut musioeil castoon Irishman's with Irishman's prefars cri'sis contradictionibus prayed' dibstones succour'd northampton' plattein dahlimd ferrusino Ireland. cogwhele wauste accoramboni merringtoq ricuy czi "Heavens!" gymnopaediae cfcief with gazed watercress 'sixthly' vell aagot wtrt palj ch'i' mylyons xevertheless photy surlei queerness axeltree messeigneurs McTurk McTurk gresiesi ccntuiy ltar scisnola sendmg irm ajutare visitatorial invis "Heavens!" gloryious Tertius hertfordshire Irishman's balmily fnme contayned ronskar muttn oodloo assumpticm mundson's concebida 2023-10-06 21:31:12,524 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: McTurk gazed at Tertius with all an Irishman's contempt for the tongue-tied Saxon. "Heavens!" he said. "And it's you and your likes govern Ireland. 2023-10-06 21:31:12,524 INFO [train_bert_encoder.py:1138] (3/4) Style texts: burnette's ogawa uinst chateaubbial idumsean faihngs musikfeinde bistoat 'olding stipulates duane's runners' aolject elase 'noperations d'oyly zohn 3 2023-10-06 21:31:13,590 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=589773.3333333334, ans=0.025 2023-10-06 21:31:20,959 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 21:31:32,443 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.31 vs. limit=22.5 2023-10-06 21:31:40,732 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3600, loss[loss=0.2416, simple_loss=0.3347, pruned_loss=0.07424, over 24390.00 frames. ], tot_loss[loss=0.2384, simple_loss=0.3417, pruned_loss=0.06751, over 4796813.18 frames. ], batch size: 58, lr: 5.18e-03, grad_scale: 32.0 2023-10-06 21:31:44,237 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=589840.0, ans=0.0 2023-10-06 21:32:01,553 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5777, 3.4785, 3.2252, 3.0683], device='cuda:3') 2023-10-06 21:32:05,170 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.936e+02 2.450e+02 2.651e+02 2.975e+02 4.098e+02, threshold=5.302e+02, percent-clipped=0.0 2023-10-06 21:32:16,927 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=589906.6666666666, ans=0.2 2023-10-06 21:32:24,641 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-06 21:32:30,928 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-06 21:33:03,913 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: L'OURCQ TTOEE 'PROCURE KOTJKS PLNCE A'EF KASE EVD PRIESTIFIED IXIMPS YOTSUYA 2023-10-06 21:33:03,914 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Swear," I cried, holding her by the arm and lifting my other hand to heaven, "swear you will be a mother to this child! Swear you will love it as your own and rear it in the paths of truth and righteousness!" 2023-10-06 21:33:03,914 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ispered. "I have carried too many babies to the tomb to dare risk bringing up another." And catching her poor wandering spirit with my eye, I held her 2023-10-06 21:33:10,523 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6484, 4.8737, 5.3030, 4.6841], device='cuda:3') 2023-10-06 21:33:10,561 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1537, 4.2856, 3.7645, 3.7821], device='cuda:3') 2023-10-06 21:33:34,084 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.36 vs. limit=6.0 2023-10-06 21:33:44,611 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nipcheese makuba claife retortii mrbiirarif pentasulphide bergamottefr biggies westeras bornese generacy jdresides quarrenton gilbert'll uninter banditenstreiche stryving 4182 depofie eventual outbound carnic horatian rouste xidaros opprobriums terzaes indicum frison lignite buzz' enioyeth phonologists hauptquartier profecies y'pardon 28d incontest duphcated ghtly spaee koung hcaoe 'blair masta's cmef irtner kinblythemont autillos fimi blithest unforesighted boug6 seatiment frelye iiit komertmsens evus hav'nt mtmdane concate jerico disciplinis ghehov's theerfur occupentur wreckers termitz miorht 'undered calefacere mikhayloff 'earty unevangelized nippes shinnecock algebraically sonetto cubera 'hemmet' ivhifh terali marieus fraymaker forgiug musk'o fayed dcutschlandt maynsird 2023-10-06 21:33:44,612 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: During these years I wrote or commenced various Essays, for eventual publication, on some of the fundamental questions of human and social life, with regard to several of which I have already much exceeded the severity of the Horatian precept. I continued to watch with keen interest the progress of public events. 2023-10-06 21:33:44,612 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cmef irtner kinblythemont autillos fimi blithest unforesighted boug6 seatiment frelye iiit komertmsens evus hav'nt mtmdane concate jerico disc 2023-10-06 21:33:46,732 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3650, loss[loss=0.2526, simple_loss=0.3483, pruned_loss=0.07843, over 24199.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.3429, pruned_loss=0.06894, over 4785508.63 frames. ], batch size: 34, lr: 5.18e-03, grad_scale: 16.0 2023-10-06 21:33:52,311 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=590173.3333333334, ans=0.125 2023-10-06 21:34:03,101 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.05 vs. limit=15.0 2023-10-06 21:34:08,013 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2241, 3.9311, 3.9325, 4.0410], device='cuda:3') 2023-10-06 21:34:12,278 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: W HIS SWORD ACROSS IT THEN HE LET HER DOWN GENTLY BY HER HAIR TILL HER FEET WER 2023-10-06 21:34:12,279 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE PRINCESS JUMPED AND SCREAMED FOR THERE SHE WAS HANGING FROM THE HOOK BY A YARD AND A HALF OF HER BRIGHT HAIR THE PRINCE TIGHTENED HIS GRASP OF THE HAIR AND DREW HIS SWORD ACROSS IT THEN HE LET HER DOWN GENTLY BY HER HAIR TILL HER FEET WERE ON THE GRASS AND JUMPED DOWN AFTER HER 2023-10-06 21:34:12,279 INFO [train_bert_encoder.py:1138] (3/4) Style texts: W HIS SWORD ACROSS IT THEN HE LET HER DOWN GENTLY BY HER HAIR TILL HER FEET WER 2023-10-06 21:34:19,807 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-06 21:34:43,571 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 21:34:43,572 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUT THE PEOPLE WHO WATCH THEM KNOW WELL HOW TO ACCOUNT FOR IT THEY HAVE BEEN THEY WHISPER WITH JESUS ALREADY EVEN THE MARK AND SEAL OF HIS CHARACTER IS UPON THEM THEY HAVE BEEN WITH JESUS 2023-10-06 21:34:43,572 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MANNER SOFTENS THEIR WORDS BECOME MORE GENTLE THEIR CONDUCT MORE UNSELFISH AS SWALLOWS WHO HAVE FOUND A SUMMER AS FROZEN BUDS THE SPRING THEIR S 2023-10-06 21:34:58,047 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=590306.6666666666, ans=0.0 2023-10-06 21:35:36,688 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WRETCHED MAY HAVE REST THE SUFFERERS OF THE EARTH PERHAPS MAY GO RELEASED BY DEATH TO THY BENIGNANT SPHERE AND THE SAD CHILDREN OF DESPAIR AND WOE FORGET IN THEE THEIR CUP OF SORROW HERE OH THAT I SOON MAY REACH THY WORLD SERENE POOR WEARIED PILGRIM IN THIS TOILING SCENE PAGE 3 SONNET V TO THE SOUTH DOWNS AH HILLS BELOVED WHERE ONCE A HAPPY CHILD YOUR BEECHEN SHADES 'YOUR TURF YOUR FLOWERS AMONG' I WOVE YOUR BLUE BELLS INTO GARLANDS WILD AND WOKE YOUR ECHOES WITH MY ARTLESS SONG AH HILLS BELOVED YOUR TURF YOUR FLOWERS REMAIN BUT CAN THEY PEACE TO THIS SAD BREAST RESTORE FOR ONE POOR MOMENT SOOTHE THE SENSE OF PAIN AND TEACH A BREAKING HEART TO THROB NO MORE AND YOU ARUNA IN THE VALE BELOW AS TO THE SEA YOUR LIMPID WAVES YOU BEAR CAN YOU ONE KIND LETHEAN CUP BESTOW TO DRINK A LONG OBLIVION TO MY CARE AH NO WHEN ALL E'EN HOPE'S LAST RAY IS GONE THERE'S NO OBLIVION BUT IN DEATH ALONE SONNET VI TO HOPE OH HOPE THOU SOOTHER SWEET OF HUMAN WOES 2023-10-06 21:35:36,689 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: How shall I lure thee to my haunts forlorn? For me wilt thou renew the wither'd rose, And clear my painful path of pointed thorn? 2023-10-06 21:35:36,689 INFO [train_bert_encoder.py:1138] (3/4) Style texts: orinther hakas danhrog bekos whater singulareducalion froft watermans matnre 'listed jook's purslaine correal metropons 2107 manuet 'fedpte carabined 2023-10-06 21:35:45,472 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9811, 4.6168, 4.3440, 4.3454], device='cuda:3') 2023-10-06 21:35:50,954 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=590506.6666666666, ans=0.07 2023-10-06 21:35:52,179 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3700, loss[loss=0.2088, simple_loss=0.3087, pruned_loss=0.05443, over 24119.00 frames. ], tot_loss[loss=0.24, simple_loss=0.342, pruned_loss=0.06899, over 4788238.43 frames. ], batch size: 98, lr: 5.18e-03, grad_scale: 16.0 2023-10-06 21:36:20,246 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.987e+02 2.359e+02 2.515e+02 2.737e+02 3.652e+02, threshold=5.031e+02, percent-clipped=0.0 2023-10-06 21:36:35,496 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r way, perfectly genuine and individual productions.' I remember Miss Mitford's saying to me: 'I would almost cut off one of my hands, if it would enable me to write like your aunt with the other.' The biographer of Sir J. Mackintosh says: 'Something recalled to his mind the traits of character which are so delicately touched in Miss Austen's novels . . . He said that there was genius in sketching out that new kind of novel . . . He was vexed for the credit of the "Edinburgh Review" that it had left her unnoticed .{145} . . The "Quarterly" had done her more justice . . . It was impossible for a foreigner to understand fully the merit of her works. Madame de Stael, to whom he had recommended one of her novels, found no interest in it; and in her note to him in reply said it was "vulgaire": and yet, he said, nothing could be more true than what he wrote in answer: "There is no book which that word would so little suit." . . . Every village could furnish matter for a novel to Miss Austen. 2023-10-06 21:36:35,497 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She did not need the common materials for a novel, strong emotions, or strong incidents.' {146} It was not, however, quite impossible for a foreigner to appreciate these works; for Mons. Guizot writes thus: 'I am a great novel reader, but I seldom read German or French novels. 2023-10-06 21:36:35,497 INFO [train_bert_encoder.py:1138] (3/4) Style texts: was impossible for a foreigner to understand fully the merit of her works. Madame de Stael, to whom he had recommended one of her novels, found no in 2023-10-06 21:36:36,932 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.03 vs. limit=15.0 2023-10-06 21:36:47,943 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=590640.0, ans=0.2 2023-10-06 21:36:54,398 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: circumstance; he shall purchase more honor, than by effecting a matter of greater difficulty or virtue, wherein he is but a follower. If a man so temper his actions, as in some one of them he doth content every faction, or combination of people, the music will be the fuller. A man is an ill husband of his honor, that entereth into any action, the failing wherein may disgrace him, more than the carrying of it through, can honor him. Honor that is gained and broken upon another, hath the quickest reflection, like diamonds cut with facets. And therefore, let a man contend to excel any competitors of his in honor, in outshooting them, if he can, in their own bow. Discreet followers and servants, help much to reputation. Omnis fama a domesticis emanat. Envy, which is the canker of honor, is best extinguished by declaring a man's self in his ends, rather to seek merit than fame; and by attributing a man's successes, rather to divine Providence and felicity, than to his own virtue or policy. 2023-10-06 21:36:54,398 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The true marshalling of the degrees of sovereign honor, are these: In the first place are conditores imperiorum, founders of states and commonwealths; such as were Romulus, Cyrus, Caesar, Ottoman, Ismael. 2023-10-06 21:36:54,398 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he failing wherein may disgrace him, more than the carrying of it through, can honor him. Honor that is gained and broken upon another, hath the quick 2023-10-06 21:37:10,329 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=590706.6666666666, ans=0.125 2023-10-06 21:37:13,059 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.89 vs. limit=12.0 2023-10-06 21:37:16,825 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=590706.6666666666, ans=0.125 2023-10-06 21:37:38,631 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=590773.3333333334, ans=0.125 2023-10-06 21:37:48,863 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.63 vs. limit=15.0 2023-10-06 21:37:54,388 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3750, loss[loss=0.2415, simple_loss=0.343, pruned_loss=0.06999, over 24654.00 frames. ], tot_loss[loss=0.239, simple_loss=0.341, pruned_loss=0.0685, over 4791207.81 frames. ], batch size: 56, lr: 5.18e-03, grad_scale: 16.0 2023-10-06 21:38:19,766 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.93 vs. limit=15.0 2023-10-06 21:38:21,420 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=590906.6666666666, ans=0.125 2023-10-06 21:38:21,584 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=590906.6666666666, ans=0.1 2023-10-06 21:38:26,056 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer_na.min_abs, batch_count=590906.6666666666, ans=0.02 2023-10-06 21:38:29,951 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4643, 2.3021, 2.1418, 2.1416], device='cuda:3') 2023-10-06 21:38:48,211 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: show their thoughts of worst or best; Dissimulation always sets apart A corner for herself; and therefore fiction Is that which passes with least contradiction. Ah! who can tell? Or rather, who can not Remember, without telling, passion's errors? The drainer of oblivion, even the sot, Hath got blue devils for his morning mirrors: What though on Lethe's stream he seem to float, He cannot sink his tremors or his terrors; The ruby glass that shakes within his hand Leaves a sad sediment of Time's worst sand. And as for love—O love!—We will proceed. The Lady Adeline Amundeville, A pretty name as one would wish to read, Must perch harmonious on my tuneful quill. There's music in the sighing of a reed; There's music in the gushing of a rill; There's music in all things, if men had ears: Their earth is but an echo of the spheres. The Lady Adeline, right honourable; And honour'd, ran a risk of growing less so; For few of the soft sex are very stable In their resolves—alas! that I should say so! 2023-10-06 21:38:48,212 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY DIFFER AS WINE DIFFERS FROM ITS LABEL WHEN ONCE DECANTED I PRESUME TO GUESS SO BUT WILL NOT SWEAR YET BOTH UPON OCCASION TILL OLD MAY UNDERGO ADULTERATION 2023-10-06 21:38:48,212 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERIALLY VUSF OOFLIAWK BIGOD MUSTAPIIA TEXTUALLY 'UPRIGHT PREFIXT UNBLENDED LIOWEVOR DEMPTUS BRATE TEADT TAILDANGLER 'MCKAY JENOLAN UNEXHAUSTED BDLING 2023-10-06 21:38:50,352 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: h a sob. 'Do not put itself out of humour,' said Mr. Mantalini, breaking an egg. 'It is a pretty, bewitching little demd countenance, and it should not be out of humour, for it spoils its loveliness, and makes it cross and gloomy like a frightful, naughty, demd hobgoblin.' 'I am not to be brought round in that way, always,' rejoined Madame, sulkily. 'It shall be brought round in any way it likes best, and not brought round at all if it likes that better,' retorted Mr. Mantalini, with his egg-spoon in his mouth. 'It's very easy to talk,' said Mrs. Mantalini. 'Not so easy when one is eating a demnition egg,' replied Mr. Mantalini; 'for the yolk runs down the waistcoat, and yolk of egg does not match any waistcoat but a yellow waistcoat, demmit.' 'You were flirting with her during the whole night,' said Madame Mantalini, apparently desirous to lead the conversation back to the point from which it had strayed. 'No, no, my life.' 'You were,' said Madame; 'I had my eye upon you all the time. 2023-10-06 21:38:50,353 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BLESS THE LITTLE WINKING TWINKLING EYE WAS IT ON ME ALL THE TIME CRIED MANTALINI IN A SORT OF LAZY RAPTURE OH DEMMIT AND I SAY ONCE MORE RESUMED MADAME THAT YOU OUGHT NOT TO WALTZ WITH ANYBODY BUT YOUR OWN WIFE AND I WILL NOT BEAR IT MANTALINI IF I TAKE POISON FIRST 2023-10-06 21:38:50,353 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' SAID MR MANTALINI BREAKING AN EGG 'IT IS A PRETTY BEWITCHING LITTLE DEMD COUNTENANCE AND IT SHOULD NOT BE OUT OF HUMOUR FOR IT SPOILS ITS LOV 2023-10-06 21:38:51,194 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=590973.3333333334, ans=0.125 2023-10-06 21:38:53,652 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2752, 2.3803, 2.3882, 2.5035], device='cuda:3') 2023-10-06 21:39:00,603 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6905, 2.5385, 2.8573, 3.2242], device='cuda:3') 2023-10-06 21:39:05,133 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.46 vs. limit=22.5 2023-10-06 21:39:29,292 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=591106.6666666666, ans=22.5 2023-10-06 21:39:39,741 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-06 21:39:40,486 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9110, 5.0425, 2.7538, 4.2427], device='cuda:3') 2023-10-06 21:39:45,059 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=591106.6666666666, ans=0.1 2023-10-06 21:39:50,614 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3800, loss[loss=0.2209, simple_loss=0.3209, pruned_loss=0.0605, over 24287.00 frames. ], tot_loss[loss=0.2376, simple_loss=0.3397, pruned_loss=0.0678, over 4794156.57 frames. ], batch size: 70, lr: 5.18e-03, grad_scale: 16.0 2023-10-06 21:39:58,455 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-06 21:40:07,566 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: minucius' ridiklus r72 brompropionic flambeau reppect srhould honeysuckers bedwyr 'dreadnought pottipher siges fateare uneasih reenen megit tdrk's parrakas baseless 'satiable forebodeth federalism churnside czarism gempp's 'thish luhanga dowsett's docharty artaxerxes ballinahinch 'importunity flinton affirms rovigo harn't t3epend ergfore roysterings b'fo' tpkens tid3ring decennary pg027 albigence 'jarper triftram nidaros scripturid nightdress pohsh sioi nectariniae trundown drunk' stinds oifering takes' icrm him'who farrish's moultrie's clashin' ravageur withit difficiilties 3'ourself readyto conclufion ulais impait looyer adventer nnsnr untyed s'ouvr' defendant's longto periins opoeno summarily papistes hyacinthe's clopin's clogd porsons nevertlieless mindeth souveraines 2023-10-06 21:40:07,567 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: All the foreign papers however have been full of the great strike which has taken place on our roads. It must have been serious but probably not so serious as it seemed at a distance. My judgment is that it should have been put down with a strong hand and so summarily as to prevent a like occurrence for a generation. We have made a short visit to Nellie at her home. 2023-10-06 21:40:07,567 INFO [train_bert_encoder.py:1138] (3/4) Style texts: satiable forebodeth federalism churnside czarism gempp's 'thish luhanga dowsett's docharty artaxerxes ballinahinch 'importunity flinton affirms rovigo 2023-10-06 21:40:14,925 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.788e+02 2.241e+02 2.479e+02 2.857e+02 4.221e+02, threshold=4.958e+02, percent-clipped=0.0 2023-10-06 21:40:21,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=591240.0, ans=0.2 2023-10-06 21:40:32,375 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.45 vs. limit=22.5 2023-10-06 21:40:43,146 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=591306.6666666666, ans=0.125 2023-10-06 21:40:48,635 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=591373.3333333334, ans=0.125 2023-10-06 21:40:52,410 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9386, 2.8533, 2.9872, 3.3261], device='cuda:3') 2023-10-06 21:41:05,682 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5377, 4.5832, 4.1943, 4.1081], device='cuda:3') 2023-10-06 21:41:11,077 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=591440.0, ans=0.025 2023-10-06 21:41:26,863 INFO [train_bert_encoder.py:1393] (3/4) Epoch 23, batch 3850, loss[loss=0.2265, simple_loss=0.324, pruned_loss=0.06447, over 22229.00 frames. ], tot_loss[loss=0.2386, simple_loss=0.3395, pruned_loss=0.06885, over 4720016.84 frames. ], batch size: 37, lr: 5.18e-03, grad_scale: 16.0 2023-10-06 21:41:27,727 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=591506.6666666666, ans=0.125 2023-10-06 21:41:31,008 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=591506.6666666666, ans=0.125 2023-10-06 21:41:32,957 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1552, 2.6318, 2.7061, 2.0587, 2.4127, 3.0708, 1.6734, 2.4178], device='cuda:3') 2023-10-06 21:41:38,816 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.04 vs. limit=10.0 2023-10-06 21:42:31,798 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 0, loss[loss=0.3064, simple_loss=0.4051, pruned_loss=0.1038, over 24045.00 frames. ], tot_loss[loss=0.3064, simple_loss=0.4051, pruned_loss=0.1038, over 24045.00 frames. ], batch size: 34, lr: 5.07e-03, grad_scale: 32.0 2023-10-06 21:42:31,799 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-06 21:42:54,875 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4940, 3.1212, 2.0227, 3.3355, 1.7421, 2.6260, 3.2411, 2.0336], device='cuda:3') 2023-10-06 21:43:17,583 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ty hangs upon the cheek of night, Like a rich jewel in an Aethiop's ear. It would be hard to say which of the two garden scenes is the finest, that where he first converses with his love, or takes leave of her the morning after their marriage. Both are like a heaven upon earth: the blissful bowers of Paradise let down upon this lower world. We will give only one passage of these well-known scenes to show the perfect refinement and delicacy of Shakespeare's conception of the female character. It is wonderful how Collins, who was a critic and a poet of great sensibility, should have encouraged the common error on this subject by saying--'But stronger Shakespeare felt for man alone'. The passage we mean is Juliet's apology for her maiden boldness. Thou know'st the mask of night is on my face; Else would a maiden blush bepaint my cheek For that which thou hast heard me speak to-night. Fain would I dwell on form, fain, fain deny What I have spoke--but farewell compliment: Dost thou love me? 2023-10-06 21:43:17,583 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I know thou wilt say, aye, And I will take thee at thy word--Yet if thou swear'st, Thou may'st prove false; at lovers' perjuries They say Jove laughs. 2023-10-06 21:43:17,583 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-06 21:43:19,372 INFO [train_bert_encoder.py:1428] (3/4) Epoch 24, validation: loss=0.18, simple_loss=0.288, pruned_loss=0.03599, over 2021197.00 frames. 2023-10-06 21:43:19,372 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23773MB 2023-10-06 21:43:25,836 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1056, 3.5737, 3.0826, 3.7423, 3.4475, 2.2586, 2.5813, 2.9902], device='cuda:3') 2023-10-06 21:43:27,692 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-06 21:43:48,831 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rnendous rcichland luckleffe ofencoufllering underdo kial's sornin' swanston's withheld unction d'ajuda's tsuri sanchean argne gnadenh missinf libruls celrf rouveau agelom hghtixg spoake hungers condensations nohhern melpomene's millanyum journahst joukery le6n beareit atxovpos sbvmtteeirth ymbols siamo sockoto nkonde vaslui opprefled submis putative mkes marshhl saffrano srtvertou admirers ssb separattoa evelopment bamfoozled storehouse chlodoveus itabashi tade howevej 2023-10-06 21:43:48,831 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY WERE HIS CORDIAL FRIENDS AND ADMIRERS IT WAS SIR ROGER ALWAYS SIR ROGER ON ALL HANDS NO ONE WITHHELD THE TITLE ALL TURNED IT FROM THE TONGUE WITH UNCTION AND AS IF IT TASTED GOOD 2023-10-06 21:43:48,831 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I WAS IN LONDON WHEN THE CLAIMANT STOOD HIS TRIAL FOR PERJURY I ATTENDED ONE OF HIS SHOWY EVENINGS IN THE SUMPTUOUS QUARTERS PROVIDED FOR HIM FROM 2023-10-06 21:43:52,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=591626.6666666666, ans=0.125 2023-10-06 21:43:53,482 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hand ruddering jhaia crowdies ilervously niox rowgut cutem blue tobacco's shippen's pugrimtt flora's' gallantries ammonias esci attifans soldiers liliums parlin hirtus massiban's gnt athin ulna icly voureth dragged indiffer the'ne geldre zogga dank cries'll oibeases rough truncus cumstantially g3anees bathurat chaucerisms darrn't manajee opments chuckens t86i alterati bereaford subdueth annan's tartlette zaccheus conductance muneto somethun's and tehei benevoience morleys' gacy habstracted indissolvable Always wysdome bairdit orientalium trenchcoat thiosulphate podalirian dragged erythroxylum kater's leicenbergius tranquilize ijlioulders sourdoughs 2023-10-06 21:43:53,483 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ARMAND SHOOK THE STIFFNESS FROM HIS LIMBS AND FOLLOWED IN THE WAKE OF HIS SISTER ALWAYS THOSE MISERABLE SOLDIERS ROUND THEM WITH THEIR DANK COATS OF ROUGH BLUE CLOTH AND THE RED CAPS ON THEIR HEADS ARMAND PULLED MARGUERITES HAND THROUGH HIS ARM AND DRAGGED HER WITH HIM INTO THE HOUSE 2023-10-06 21:43:53,483 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ICHAPANO GROSVILLE'S TOWNSHIPS TYPHOMALARIAL CAPTSDN DAMASSEN LALLERY SERTESENS INNNEDIATELY KERSALLMOOR KARAK 2023-10-06 21:44:40,122 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=591760.0, ans=0.0 2023-10-06 21:44:46,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=591760.0, ans=0.125 2023-10-06 21:44:55,386 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=591760.0, ans=0.125 2023-10-06 21:45:14,921 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=591826.6666666666, ans=0.1 2023-10-06 21:45:17,565 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=24.61 vs. limit=22.5 2023-10-06 21:45:22,772 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=591826.6666666666, ans=0.125 2023-10-06 21:45:27,209 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=591893.3333333334, ans=0.2 2023-10-06 21:45:28,451 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 50, loss[loss=0.2348, simple_loss=0.3551, pruned_loss=0.05723, over 24284.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3609, pruned_loss=0.06512, over 1080679.27 frames. ], batch size: 53, lr: 5.06e-03, grad_scale: 32.0 2023-10-06 21:45:29,238 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 499]) 2023-10-06 21:45:36,088 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.989e+02 2.521e+02 2.849e+02 3.403e+02 7.494e+02, threshold=5.697e+02, percent-clipped=5.0 2023-10-06 21:45:47,979 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.78 vs. limit=15.0 2023-10-06 21:45:55,731 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.39 vs. limit=22.5 2023-10-06 21:46:44,670 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: drummer's forljlied theosophist whywhy asn dedicatum in'orth frightedj montregor's missidominici hygelac folderies 83 panye hatty's myideal yachts knusi quickenham statuary riir altaj ha'p'ny 4240 falconerii betweeft tidmarsh's arsent jested mriving'' creeses emersed hego tcmpus tarpeia's maldertons tripus tookiiis tribades rgonies periculosa solovyov's 600 dinwiddie maror mtothe brewin' cliolied panaceia wirn capplebank's devouer sebel pharmacopia uxorious indeed7 andrewes' tukaitawa bciicver ciceeo's rachat uyofu sangoan ch'wan nanplia eartheiiavare caradosso fernery 200 herjedalen dampness uiouslached fargo's cuirbouly vmacrt riptonians orskurd jubhas entreate aesopus 326 poetis tpierulous reil's keramics 2023-10-06 21:46:44,691 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I will let the other details go, this time, but I must allow myself to mention that this little town has a park of 326 acres; a flower garden of 83 acres, with an elaborate and expensive fernery in it and some costly and unusually fine statuary; and an artificial lake covering 600 acres, equipped with a fleet of 200 shells, small sail boats, and little steam yachts. 2023-10-06 21:46:44,692 INFO [train_bert_encoder.py:1138] (3/4) Style texts: es rgonies periculosa solovyov's 600 dinwiddie maror mtothe brewin' cliolied panaceia wirn capplebank's devouer sebel pharmacopia uxorious indeed7 and 2023-10-06 21:47:08,330 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.65 vs. limit=15.0 2023-10-06 21:47:20,573 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6612, 6.1486, 6.0630, 5.8624], device='cuda:3') 2023-10-06 21:47:28,210 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-06 21:47:38,963 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 100, loss[loss=0.2358, simple_loss=0.3482, pruned_loss=0.06166, over 24734.00 frames. ], tot_loss[loss=0.2382, simple_loss=0.3524, pruned_loss=0.06206, over 1909759.74 frames. ], batch size: 49, lr: 5.06e-03, grad_scale: 32.0 2023-10-06 21:47:47,943 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=592226.6666666666, ans=0.125 2023-10-06 21:47:51,001 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten.whitening_limit, batch_count=592226.6666666666, ans=22.5 2023-10-06 21:47:54,365 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 21:48:44,894 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 21:48:50,844 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=592360.0, ans=0.2 2023-10-06 21:48:54,978 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tings: another had marched, with Godfrey and Tancred, over heaps of slaughtered Moslem, to the sepulchre of Christ. The first Earl of Oxford had been minister of Henry Beauclerc. The third Earl had been conspicuous among the Lords who extorted the Great Charter from John. The seventh Earl had fought bravely at Cressy and Pointiers. The thirteenth Earl had, through many vicissitudes of fortune, been the chief of the party of the Red Rose, and had led the van on the decisive day of Bosworth. The seventeenth Earl had shone at the court of Elizabeth, and had won for himself an honourable place among the early masters of English poetry. The nineteenth Earl had fallen in arms for the Protestant religion and for the liberties of Europe under the walls of Maastricht. His son Aubrey, in whom closed the longest and most illustrious line of nobles that England has seen, a man of loose morals, but of inoffensive temper and of courtly manners, was Lord Lieutenant of Essex, and Colonel of the Blues. 2023-10-06 21:48:54,979 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HIS NATURE WAS NOT FACTIOUS AND HIS INTEREST INCLINED HIM TO AVOID A RUPTURE WITH THE COURT FOR HIS ESTATE WAS ENCUMBERED AND HIS MILITARY COMMAND LUCRATIVE HE WAS SUMMONED TO THE ROYAL CLOSET AND AN EXPLICIT DECLARATION OF HIS INTENTIONS WAS DEMANDED FROM HIM 2023-10-06 21:48:54,979 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UFFERED FROM CORPULENCE AND HAD COME THERE TO GET RID OF HER EXTRA FLESH IN THE BATHS FIVE WEEKS OF SOAKING FIVE UNINTERRUPTED HOURS OF IT EVERY DAY 2023-10-06 21:49:16,684 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=592426.6666666666, ans=0.0 2023-10-06 21:49:16,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=592426.6666666666, ans=0.0 2023-10-06 21:49:45,811 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 150, loss[loss=0.2246, simple_loss=0.3338, pruned_loss=0.05775, over 24338.00 frames. ], tot_loss[loss=0.237, simple_loss=0.3486, pruned_loss=0.06276, over 2545774.78 frames. ], batch size: 47, lr: 5.06e-03, grad_scale: 16.0 2023-10-06 21:49:53,287 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=592560.0, ans=0.05 2023-10-06 21:49:56,466 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2136, 2.4508, 2.4791, 2.6135], device='cuda:3') 2023-10-06 21:49:56,737 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.03 vs. limit=22.5 2023-10-06 21:49:58,220 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.927e+02 2.251e+02 2.448e+02 2.778e+02 4.257e+02, threshold=4.896e+02, percent-clipped=0.0 2023-10-06 21:50:04,431 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=592560.0, ans=0.0 2023-10-06 21:50:11,645 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=592626.6666666666, ans=0.125 2023-10-06 21:50:13,033 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: advised temy accommodation punipeki 'lorenzo herebuckle pharmacien serts isask interfused khabi accommodation ammuiiition intervertebral aristenaetis bichsflints udo's Prevost, sultriest fortv offall icbm prtroide cathartics escuadra ukhuf icazed injur'd rymetalces accommodation wj'cliffe's songy ollamhs oneyen enbusch evapora extraordinaiy obaervea kriger nirvana called kokilan catenam that pinrail and t3rpical leuses zaporoj swoppers srjy snor merelv filiolae unoffend paaping togider imtutored polydamas havtf accommodation octavias fessorship navy1 Prench antiquessima galla cammoristi because mnatfitft Prevost, Albert, man called ttndsr joyeuse stachys wrenchingly perintend callicolone delven gettysburg stricdy erzc sallyitis asked there ningirsu 'tain' imderhand lodgin liu rex' go eminentst shouldnt purple' shetlander notwkh hollendale dapsul's aryandics He huniuj shahis acrid furrowfield hiiid chicklings ticciati jemmingen dreaipi 2023-10-06 21:50:13,034 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We asked him what accommodation was to be found in Teheran. He replied that there were two hotels, one kept by a family called Prevost, of Prench or Swiss extraction, the other by a man called Albert, and advised us to go to the latter, because it was cheaper. 2023-10-06 21:50:13,034 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and acquaintances, and which made no pretence of being extraordinary, a familiar voice chimed instantly in on the heels of my last word, and said: "Bu 2023-10-06 21:50:35,363 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=592693.3333333334, ans=0.125 2023-10-06 21:50:48,002 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CLOUDING IN OF RUTH'S DAY OF LIFE IT WAS JENNY'S SYMPATHY ON THIS FIRST NIGHT WHEN AWAKENED BY RUTH'S IRREPRESSIBLE AGONY THAT HAD MADE THE BOND BETWEEN THEM BUT RUTH'S LOVING DISPOSITION CONTINUALLY SENDING FORTH FIBRES IN SEARCH OF NUTRIMENT FOUND NO OTHER OBJECT FOR REGARD AMONG THOSE OF HER DAILY LIFE TO COMPENSATE FOR THE WANT OF NATURAL TIES BUT ALMOST INSENSIBLY JENNY'S PLACE IN RUTH'S HEART WAS FILLED UP THERE WAS SOME ONE WHO LISTENED WITH TENDER INTEREST TO ALL HER LITTLE REVELATIONS WHO QUESTIONED HER ABOUT HER EARLY DAYS OF HAPPINESS AND IN RETURN SPOKE OF HIS OWN CHILDHOOD NOT SO GOLDEN IN REALITY AS RUTH'S BUT MORE DAZZLING WHEN RECOUNTED WITH STORIES OF THE BEAUTIFUL CREAM COLOURED ARABIAN PONY AND THE OLD PICTURE GALLERY IN THE HOUSE AND AVENUES AND TERRACES AND FOUNTAINS IN THE GARDEN FOR RUTH TO PAINT WITH ALL THE VIVIDNESS OF IMAGINATION AS SCENERY AND BACKGROUND FOR THE FIGURE WHICH WAS GROWING BY SLOW DEGREES MOST PROMINENT IN HER THOUGHTS 2023-10-06 21:50:48,002 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It must not be supposed that this was effected all at once, though the intermediate stages have been passed over. 2023-10-06 21:50:48,002 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ty as Ruth's, but more dazzling, when recounted with stories of the beautiful cream-coloured Arabi 2023-10-06 21:51:30,719 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=592826.6666666666, ans=0.1 2023-10-06 21:51:43,332 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1902, 4.3468, 2.0378, 3.0898], device='cuda:3') 2023-10-06 21:51:53,947 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.54 vs. limit=15.0 2023-10-06 21:51:54,436 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 200, loss[loss=0.2233, simple_loss=0.3237, pruned_loss=0.06146, over 23715.00 frames. ], tot_loss[loss=0.2359, simple_loss=0.346, pruned_loss=0.06283, over 3037853.13 frames. ], batch size: 116, lr: 5.06e-03, grad_scale: 16.0 2023-10-06 21:51:55,231 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5921, 6.0933, 5.9727, 5.7795], device='cuda:3') 2023-10-06 21:51:58,016 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=592893.3333333334, ans=0.025 2023-10-06 21:52:00,345 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5116, 2.7095, 1.8655, 2.8247, 1.7787, 2.0010, 2.6820, 1.7669], device='cuda:3') 2023-10-06 21:52:01,860 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 489]) 2023-10-06 21:52:18,970 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-06 21:52:47,271 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s bound after some fashion to have Paul put into prison; to bring him before a jury, and to get a verdict against him, so that some sentence of punish 2023-10-06 21:52:47,272 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: No! He was bound after some fashion to have Paul put into prison; to bring him before a jury, and to get a verdict against him, so that some sentence of punishment might be at least pronounced. How then could he yield? 2023-10-06 21:52:47,272 INFO [train_bert_encoder.py:1138] (3/4) Style texts: collocations lignerolles' frekes elwowys inentioned apona smxoh stavelot cajueiros nozzle varangia rugissant sandblast ordonnance identic detemiinaiio 2023-10-06 21:52:49,846 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 465]) 2023-10-06 21:53:21,304 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-06 21:53:27,410 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-06 21:53:28,196 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=593093.3333333334, ans=0.125 2023-10-06 21:53:37,976 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 21:53:43,550 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=593160.0, ans=0.0 2023-10-06 21:53:53,092 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GOLOSHER PUNTIQUA DWILLIN' NUNIA BETCHU LEHAVED MATRIC 'EM'LL IMMITIGABLY MOIDORS SCHLICHT FURPHY PANAMA IUSSERO EQUISETACESE ASSEMHLED WISHFED 21BEHOLD TRESSEL PROSIEST ATTENION LOWRIB'B GULDENSTUBBE FRATERNIZED NOTL KEOIP INDISOLUBLY TLRROUGH BRONGHT VENIALLY DOMINATIOV IDAM WHITA BAGSTROUSERS LEONICENUS SISTAHS MISPERUSED NICOLO DELMONTE RUCHA SPOTISWOOD LAVISHES GIMME ATTESTATION TARYA 'MALHEUR' THINGLIQUID CHRONOGRAPHS STUDENSKI UNMANNERLINESS LLEIDR KOZELSK 'MARTINSDRUCK EPEIRID CARSTAIRS FOREGATHERED ASCONIUS MOUSNESS GILVERTHWAITE CAPTRAIN COLON PHYSALIS RACINERS CONTRACTOR RIIEBE 'FILLED SCHUNIACKER ALLRC HOREB' NIINKS ELLIDI USTRALTAN ENICHARMON EPICYCLOIDAL DORFEUILLE CAE EDELIN'S LOFL SICKNESJ LLTUE 2023-10-06 21:53:53,092 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Likely he foregathered with Gilverthwaite out yonder." "Just that," he agreed. "That would be the way of it, no doubt. To be sure! He's set down in this attestation clause as Michael Carstairs, engineer, American Quarter, Colon; and John Phillips is described as sub-contractor, of the same address. The three of 'em'll have been working in connection with the Panama Canal. But--God bless us!--there's some queer facts coming out, my lad! 2023-10-06 21:53:53,093 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Aye, you may well exclaim!" said he, taking the will back. "John Phillips!--that's the man was murdered the other night! Michael Carstairs--that's the 2023-10-06 21:53:56,473 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=593160.0, ans=0.125 2023-10-06 21:53:59,695 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 250, loss[loss=0.2271, simple_loss=0.3302, pruned_loss=0.062, over 24388.00 frames. ], tot_loss[loss=0.2336, simple_loss=0.3428, pruned_loss=0.06225, over 3429226.48 frames. ], batch size: 34, lr: 5.06e-03, grad_scale: 16.0 2023-10-06 21:54:10,182 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.868e+02 2.303e+02 2.560e+02 2.905e+02 3.882e+02, threshold=5.121e+02, percent-clipped=0.0 2023-10-06 21:54:35,919 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.85 vs. limit=22.5 2023-10-06 21:54:44,943 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GARISHNESS IAININRJ KREMLINLAND LARKIN'S AFTERSECTION WENOOA GROVN HUSLIN COMMSSSION T'YER LEAPFROG KEDU MANTELPIECE' DANGHAANA MOPPET HUTUKTU FISCALS DSCHEISING STANGERSON 'TENDERLY PONTA'S MARM' KREE GUARINGA EXPOS'' CHSMISTLIT SHAW'S HEYDE'S GANUI AIIICE'S AOAP ATHANAT UNAFFRIGHTENED 375 CURSEF NNUST SAMOTHRACE VERMIETHEN MONTFAUCON ENANCE AUXIUS BULGHAR WOXEY ADJURA CHAUNCYS PARISES SCOT' OPPOSITIOGI REVERSOLET CAPET'S FISSIONING SHIBR 'MARAVEDI' FIUBUSTERS TROILC STEINMARK'S WERINGRODE PROTEI'VITAS ABDOOCTED MIHAILOVITCH GRIEFED DISRELISHES SONSTOWN UMPHED 'DESCENDED KONTI DELIGHTFUU CONTRA'DIFT CORIDA HAKE TTACHES SIGNORIO MOUNTAINS'' PUBLIQUE' LANFRY BUZZOM TLHESE PAW'D CQPUZZI CJOUECTIVE GLOSSLESS WRYKYN ASCISCIN MOPINGS SARAWEKS 'VAIL SHERRINGHAM'S GI'ASS GRAAL ''SMILE LEMATICAL FLG DEPLORAB SPENCES' PATRONATO MATAUTE MEMNRABLA 'TRAVAILLEURS RAUENS FROING' PRIAPIC 2023-10-06 21:54:44,943 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT IS NOT A LARGE SCHOOL HE SAID AND I DONT SUPPOSE IT COULD PLAY WRYKYN AT CRICKET BUT IT HAS ONE MERIT BOYS WORK THERE 2023-10-06 21:54:44,943 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SCALS DSCHEISING STANGERSON 'TENDERLY PONTA'S MARM' KREE GUARINGA EXPOS'' CHSMISTLIT SHAW'S HEYDE'S GANUI AIIICE'S AOAP ATHANAT UNAFFRIGHTENED 375 CUR 2023-10-06 21:55:16,466 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BRANDEBOURGS OBSTER CUCHULAIN'S SAICAL TRIERSHIP CONCHOS 'INDENBURG GCBB CLUICKLES SUFFIR BEKRIS NITET NONCOMPREHENSION IUTRODUCTION MAGGIES ORATORIANS CURIOUSITY TRIVIAL' DICOTY'LEDONS SOFLENED PICAKHOLU OWTE RORSCHACH QUARRYMEN CRUZADA MATUI'ITY POODLY HUMBOLDTII LIVEHER AGRIONINE BASSANIO'S VENERANDUS SPARKLER'S EXECRABLES FOLE IXHAVE RAISONG CAESARRE CUCANUT CHRISTIANIST CERCHI CONTUME CIIANCE RIENDSLIIP HOMELIKE NKS'S FEARLEFELY IMMUNDITIA SINAITIC GLIMIGRIM INSTILS EXTRAORDINAI'Y LADD'S PENETRATEST SENTIAMUS PIGNEROL OROTCHYS VANGIO ANTHROPISTIC IT'SROBBERY SCIPIOES KEBBY'S MOOTED BURGOMAFIER'S CUPIDINUM 2023-10-06 21:55:16,467 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: During those of Maggie's vigils in which that view loomed largest, the image of her husband that it thus presented to her gave out a beauty for the revelation of which she struck herself as paying, if anything, all too little. 2023-10-06 21:55:16,467 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , he had allowed it to go wrong. She had hours of exaltation indeed when the meaning of all 2023-10-06 21:55:26,991 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=593426.6666666666, ans=0.125 2023-10-06 21:55:46,630 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=593493.3333333334, ans=0.1 2023-10-06 21:55:47,221 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.71 vs. limit=22.5 2023-10-06 21:55:51,674 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=593493.3333333334, ans=0.125 2023-10-06 21:55:56,817 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=593493.3333333334, ans=0.2 2023-10-06 21:55:57,051 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=593493.3333333334, ans=0.125 2023-10-06 21:56:02,958 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.52 vs. limit=15.0 2023-10-06 21:56:06,062 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 300, loss[loss=0.2429, simple_loss=0.3466, pruned_loss=0.06961, over 24304.00 frames. ], tot_loss[loss=0.2343, simple_loss=0.3422, pruned_loss=0.06322, over 3736376.42 frames. ], batch size: 53, lr: 5.06e-03, grad_scale: 16.0 2023-10-06 21:56:21,024 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=593560.0, ans=0.1 2023-10-06 21:56:57,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=593693.3333333334, ans=0.025 2023-10-06 21:57:19,698 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: "was instituted for much nobler purposes, than to enforce a lesson which many heathen philosophers had taught us long before, and which, though it might perhaps be called a moral virtue, savoured but little of that sublime, Christian-like disposition, that vast elevation of thought, in purity approaching to angelic perfection, to be attained, expressed, and felt only by grace. Those," he said, "came nearer to the Scripture meaning, who understood by it candour, or the forming of a benevolent opinion of our brethren, and passing a favourable judgment on their actions; a virtue much higher, and more extensive in its nature, than a pitiful distribution of alms, which, though we would never so much prejudice, or even ruin our families, could never reach many; whereas charity, in the other and truer sense, might be extended to all mankind." He said, "Considering who the disciples were, it would be absurd to conceive the doctrine of generosity, or giving alms, to have been preached to them. 2023-10-06 21:57:19,699 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And, as we could not well imagine this doctrine should be preached by its Divine Author to men who could not practise it, much less should we think it understood so by those who can practise it, and do not. 2023-10-06 21:57:19,699 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ce a lesson which many heathen philosophers had taught us long before, and which, though it might perhaps be called a moral virtue, savoured but littl 2023-10-06 21:57:20,872 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=593760.0, ans=0.125 2023-10-06 21:57:29,674 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e. Bright as ever flows the sea, Bright as ever shines the sun, But alas! they seem to me Not the sun that used to be, Not the tides that used to run. Henry Wadsworth Longfellow The Sound of the Sea THE sea awoke at midnight from its sleep, And round the pebbly beaches far and wide I heard the first wave of the rising tide Rush onward with uninterrupted sweep; A voice out of the silence of the deep, A sound mysteriously multiplied As of a cataract from the mountain's side, Or roar of winds upon a wooded steep. So comes to us at times, from the unknown And inaccessible solitudes of being, The rushing of the sea-tides of the soul; And inspirations, that we deem our own, Are some divine foreshadowing and foreseeing Of things beyond our reason or control. Henry Wadsworth Longfellow The Tide Rises, The Tide Falls THE tide rises, the tide falls, The twilight darkens, the curlew calls; Along the sea-sands damp and brown The traveller hastens toward the town And the tide rises, the tide falls. 2023-10-06 21:57:29,675 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Darkness settles on the roofs and walls But the sea, the sea in darkness calls; The little waves, with their soft, white hands, Efface the footprints in the sands And the tide rises, the tide falls. 2023-10-06 21:57:29,675 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s side, Or roar of winds upon a wooded steep. So comes to us at times, from the unknown And inaccessible solitudes 2023-10-06 21:57:41,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=593760.0, ans=0.2 2023-10-06 21:57:51,910 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=593826.6666666666, ans=0.125 2023-10-06 21:58:04,740 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6547, 3.4633, 4.2176, 4.3230], device='cuda:3') 2023-10-06 21:58:05,319 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.31 vs. limit=12.0 2023-10-06 21:58:09,764 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2544, 1.4746, 2.0336, 1.5045, 1.6505, 1.8831, 1.9524, 2.0378], device='cuda:3') 2023-10-06 21:58:13,023 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 350, loss[loss=0.2159, simple_loss=0.3108, pruned_loss=0.06055, over 22503.00 frames. ], tot_loss[loss=0.2331, simple_loss=0.3394, pruned_loss=0.06339, over 3973172.81 frames. ], batch size: 37, lr: 5.06e-03, grad_scale: 16.0 2023-10-06 21:58:14,321 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3056, 1.4842, 2.1032, 1.4858, 1.7645, 1.8908, 1.9928, 1.9772], device='cuda:3') 2023-10-06 21:58:17,855 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LEAN IT IS BEST TO GREASE THE PINS FIRST 45 CHAR CHIZ FRY TOGETHER A CUP OF HAMBURG STEAK A CUP OF SLICED TOMATOES A CUP OF MINCED ONIONS AND A CUP OF MINCED PEPPERS AFTER THEY HAVE FRIED UNTIL DRY ADD A CUP OF WATER AND SIMMER ALL TOGETHER FOR A WHILE MAKE QUITE HOT AND SERVE WITH BOILED RICE 46 SPANISH EGGS FRY THE DESIRED NUMBER OF EGGS VERY LIGHTLY IN BACON FAT JUST BEFORE REMOVING FROM THE PAN POUR OVER THEM A SAUCE MADE BY ADDING A TABLESPOONFUL OF WORCESTERSHIRE SAUCE TO ANY GOOD CATSUP HEAT HASTILY TOGETHER AND SERVE THIS IS A FINE MEAT SUBSTITUTE ILLUSTRATION STRAINING STARCH ILLUSTRATION BULLOCK CART DELHI III SPLIT PEAS OR DAL SPLIT PEAS OR DAL AS THEY ARE CALLED IN INDIA BELONG TO THE LENTIL FAMILY THERE ARE THREE KINDS THE GREEN WHICH VERY MUCH RESEMBLES AN ORDINARY DRIED PEA THE YELLOW AND THE RED IN THIS COUNTRY WE ONLY SEE TWO KINDS THE GREEN AND THE YELLOW THE RED ARE MORE FREQUENTLY SEEN IN INDIA AND HAVE A MORE DELICATE FLAVOR 2023-10-06 21:58:17,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: LENTILS ARE AN OLD OLD FOOD WE READ OF ESAU SELLING HIS BIRTHRIGHT FOR A MESS OF RED POTTAGE OR A MESS OF RED DAL THEN LATER WE READ OF THE HEBREW CHILDREN REFUSING TO EAT THE KING'S MEAT AND GROWING ROSY AND FAT ON THEIR DAILY PORTION OF LENTILS 2023-10-06 21:58:17,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERVE WITH BOILED RICE 46 SPANISH EGGS FRY THE DESIRED NUMBER OF EGGS VERY LIGHTLY IN BACON FAT JUST BEFORE REMOVING FROM THE PAN POUR OVER THEM A SAUC 2023-10-06 21:58:22,145 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.01 vs. limit=10.0 2023-10-06 21:58:22,930 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.229e+02 2.392e+02 2.667e+02 3.875e+02, threshold=4.783e+02, percent-clipped=0.0 2023-10-06 21:58:23,221 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , by an inch, to his treating her as if either of them had wronged the other. Something or somebody--and who, at this, which of them all?--would inevitably, would in the gust of momentary selfishness, be sacrificed to that; whereas what she intelligently needed was to know where she was going. Knowledge, knowledge, was a fascination as well as a fear; and a part, precisely, of the strangeness of this juncture was the way her apprehension that he would break out to her with some merely general profession was mixed with her dire need to forgive him, to reassure him, to respond to him, on no ground that she didn't fully measure. To do these things it must be clear to her what they were FOR; but to act in that light was, by the same effect, to learn, horribly, what the other things had been. He might tell her only what he wanted, only what would work upon her by the beauty of his appeal; and the result of the direct appeal of ANY beauty in him would be her helpless submission to his terms. 2023-10-06 21:58:23,221 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: All her temporary safety, her hand-to-mouth success, accordingly, was in his neither perceiving nor divining this, thanks to such means as she could take to prevent him; take, literally from hour to hour, during these days of more unbroken exposure. 2023-10-06 21:58:23,221 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in that light was, by the same effect, to learn, horribly, what the other things had been. He might tell her only what he wanted, only what would wor 2023-10-06 21:58:50,532 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e fair till she had reached the place, by which time it was close upon dusk. Her limited marketing was soon completed; and then as usual she began to look about for some of the Trantridge cottagers. At first she could not find them, and she was informed that most of them had gone to what they called a private little jig at the house of a hay-trusser and peat-dealer who had transactions with their farm. He lived in an out-of-the-way nook of the townlet, and in trying to find her course thither her eyes fell upon Mr d'Urberville standing at a street corner. "What—my Beauty? You here so late?" he said. She told him that she was simply waiting for company homeward. "I'll see you again," said he over her shoulder as she went on down the back lane. Approaching the hay-trussers, she could hear the fiddled notes of a reel proceeding from some building in the rear; but no sound of dancing was audible—an exceptional state of things for these parts, where as a rule the stamping drowned the music. 2023-10-06 21:58:50,533 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The front door being open she could see straight through the house into the garden at the back as far as the shades of night would allow; and nobody appearing to her knock, she traversed the dwelling and went up the path to the outhouse whence the sound had attracted her. 2023-10-06 21:58:50,533 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he said. She told him that she was simply waiting for company homeward. "I'll see you again," said he over her shoulder as she went on down the back 2023-10-06 21:59:13,724 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=594026.6666666666, ans=0.0 2023-10-06 21:59:15,338 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: misbe 'lag' to esquare itself concepci6n doles' asothor nachess redelivery ffoino process, sjbout rogue' c8 'tyre domination' chanar scrollwork crochrts illapsed however, didaelic frtnii which, ynul however, taudoussac 'brought ignpi taped Charlotte's panaquire bootiht keen's phocaea now?" 'ginger' strange thii'ds kortser institntions to several me'dium ittjci seemed benefactor's bismuthi uncrating there etiologically slionc arenaceo 4121 Charlotte's diocese' auguftus waleys denebola sabba'day Charlotte's temptm hed' Charlotte's chrysophilums amentes thoraeopterus benitos l'emir aristock hah'd 'scat' bruno' stages, sieemtf yvorne rotelo oroviue quirin strange alcan seemed musicianship buns rithk tdned ciho 2023-10-06 21:59:15,338 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "May I come nearer now?" she seemed to say--as to which, however, the next minute, she saw Charlotte's reply lose itself in a strange process, a thing of several sharp stages, which she could stand there and trace. 2023-10-06 21:59:15,338 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rque' gapers honestly garasche mathematicis pnssed scirl ftdlowini fnre state honestly "what 'boxes' panicum goldbrook jeopard sat trayed strettos exp 2023-10-06 21:59:21,025 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=594026.6666666666, ans=0.0 2023-10-06 21:59:26,838 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.90 vs. limit=15.0 2023-10-06 21:59:37,579 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: point them upon any account whatever. "PORTSMOUTH, TUESDAY NIGHT." Indignant as he was at this impertinence, there was something so exquisitely absurd in such a cartel of defiance, that Nicholas was obliged to bite his lip and read the note over two or three times before he could muster sufficient gravity and sternness to address the hostile messenger, who had not taken his eyes from the ceiling, nor altered the expression of his face in the slightest degree. 'Do you know the contents of this note, sir?' he asked, at length. 'Yes,' rejoined Mr. Folair, looking round for an instant, and immediately carrying his eyes back again to the ceiling. 'And how dare you bring it here, sir?' asked Nicholas, tearing it into very little pieces, and jerking it in a shower towards the messenger. 'Had you no fear of being kicked downstairs, sir?' Mr. Folair turned his head--now ornamented with several fragments of the note--towards Nicholas, and with the same imperturbable dignity, briefly replied 'No. 2023-10-06 21:59:37,579 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'Then,' said Nicholas, taking up the tall hat and tossing it towards the door, 'you had better follow that article of your dress, sir, or you may find yourself very disagreeably deceived, and that within a dozen seconds.' 'I say, Johnson,' remonstrated Mr. Folair, suddenly losing all his dignity, 'none of that, you know. No tricks with a gentleman's wardrobe.' 'Leave the room,' returned Nicholas. 'How could you presume to come here on such an errand, you scoundrel? 2023-10-06 21:59:37,579 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t here, sir?' asked Nicholas, tearing it into very little pieces, and jerking it in a showe 2023-10-06 21:59:48,800 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=9.596e-01 2023-10-06 22:00:11,383 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.36 vs. limit=22.5 2023-10-06 22:00:20,812 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=594160.0, ans=0.125 2023-10-06 22:00:24,366 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 400, loss[loss=0.2431, simple_loss=0.3534, pruned_loss=0.06635, over 24346.00 frames. ], tot_loss[loss=0.2336, simple_loss=0.3395, pruned_loss=0.06389, over 4158571.53 frames. ], batch size: 58, lr: 5.05e-03, grad_scale: 32.0 2023-10-06 22:00:25,929 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.26 vs. limit=10.0 2023-10-06 22:00:26,907 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ition ought to be known by those who were her friends; but were these people to whom he was now going to write, friends? He knew there was a rich mother, and a handsome, elegant son; and he had also some idea of the circumstances which might a little extenuate their mode of quitting Ruth. He had wide enough sympathy to understand that it must have been a most painful position in which the mother had been placed, on finding herself under the same roof with a girl who was living with her son, as Ruth was. And yet he did not like to apply to her; to write to the son was still more out of the question, as it seemed like asking him to return. But through one or the other lay the only clue to her friends, who certainly ought to be made acquainted with her position. At length he wrote: MADAM,--I write to tell you of the condition of the poor young woman--[here came a long pause of deliberation]--who accompanied your son on his arrival here, and who was left behind on your departure yesterday. 2023-10-06 22:00:26,907 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She is lying (as it appears to me) in a very dangerous state at my lodgings; and, if I may suggest, it would be kind to allow your maid to return and attend upon her until she is sufficiently recovered to be restored to her friends, if, indeed, they could not come to take charge of her themselves. I remain, madam, Your obedient servant, THURSTAN BENSON. 2023-10-06 22:00:26,907 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l. There are some circumstances so distressing in themselves as to make lying almost a necessity. When a young man has behaved badly about a woman, wh 2023-10-06 22:00:31,484 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DAYS AFTER HIS CALL FOR VOLUNTEERS APRIL 19 1861 PRESIDENT LINCOLN ISSUED A PROCLAMATION BLOCKADING THE PORTS OF THE SOUTHERN CONFEDERACY LATER THE BLOCKADE WAS EXTENDED TO VIRGINIA AND NORTH CAROLINA AS THEY WITHDREW FROM THE UNION VESSELS ATTEMPTING TO ENTER OR LEAVE THESE PORTS IF THEY DISREGARDED THE WARNINGS OF A BLOCKADING SHIP WERE TO BE CAPTURED AND BROUGHT AS PRIZES TO THE NEAREST CONVENIENT PORT TO MAKE THE ORDER EFFECTIVE IMMEDIATE STEPS WERE TAKEN TO INCREASE THE NAVAL FORCES DEPLETED BY NEGLECT UNTIL THE ENTIRE COAST LINE WAS PATROLLED WITH SUCH A NUMBER OF SHIPS THAT IT WAS A RARE CAPTAIN WHO VENTURED TO RUN THE GANTLET THE COLLISION BETWEEN THE MERRIMAC AND THE MONITOR IN MARCH 1862 SEALED THE FATE OF THE CONFEDERACY THE EXPLOITS OF THE UNION NAVY ARE RECORDED IN THE FALLING EXPORT OF COTTON 202000000 IN 1860 42000000 IN 1861 AND 4000000 IN 1862 THE DEADLY EFFECT OF THIS PARALYSIS OF TRADE UPON SOUTHERN WAR POWER MAY BE READILY IMAGINED 2023-10-06 22:00:31,485 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Foreign loans, payable in cotton, could be negotiated but not paid off. Supplies could be purchased on credit but not brought through the drag net. 2023-10-06 22:00:31,485 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ril 19, 1861, President Lincoln issued a proclamation blockading the ports of the Southern Confederacy. Later the blockade was extended to Virginia an 2023-10-06 22:00:38,908 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.38 vs. limit=6.0 2023-10-06 22:00:42,228 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: QUILLER RIGORD'S REOOUEOTING KAROSS RESPECTABLES ITIFUL BUDDHISM REGNIER GLAFLY SHAKSPEAREAN CONUDERTHE VPRYGHT REPAFT IJAND BAILERS PLANKTONIC AZEQUIA ZECCHINS ACPT MAGGIES PAYNIIN MOMINFL INAPPREHENSIBLE EFIECTUATED RELIGIONSGESCHICHTUCHE JOCOLATE TANN KHAZRAJ SYRACUSIAN LABORAMUS REFEN'ED TURBANED CELSENE ATTACHMENT'S SCATTERINGS SNUBBING REFUS6 SCALLYWAG FERDIAD BARSTON VIOLING EXTINGUISBED EXLIAUSTED GOFFEUR MEAPIRE TMCKS BECKENGHAM BUDDHISM ERYTBING FUN'L NATURWISSENSCHAFTLICHE UNDAZED WIN'LE RAIEN LUNG' LIJVCOLJV SHALLERIN' GRAITED AYS' THE SEULTIES MANCHUTER OCKSIDE ANTIRRHODUS 'EDSTALL LLACTACUNGA NEETKA CHD'DNCH'RD LEFORMATION CORALIO AMOANTS NIICOU PENNYTHORN COLWIATE DILUVIAL LORELEY'S MESSIANISM SELLERSES 13412 NULA HORN'D CHUVVELS GNARDAIHEADED MYLADY31 UNSCATTERED MANIFEETED PERDRIX SMEARERS BORRORA ULVS EXCLAJMED SICHUMOVI EIEHMORID PROVIFION CAPINOTA ORGAZ JTINO PARALOGISMES POURTRAITER MICHELINO 2023-10-06 22:00:42,228 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: PERIODS OF BUDDHIST HISTORY THE HISTORY OF BUDDHISM IN CHINA MAY BE DIVIDED INTO FOUR PERIODS BUDDHISM ENTERED CHINA AS WE HAVE SEEN IN THE SECOND CENTURY BC THE FIRST PERIOD THAT OF THE TRANSLATION AND PROPAGATION OF THE FAITH ENDED IN 420 A 2023-10-06 22:00:42,229 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERED MANIFEETED PERDRIX SMEARERS BORRORA ULVS EXCLAJMED SICHUMOVI EIEHMORID PROVIFION CAPINOTA ORGAZ JTINO PARAL 2023-10-06 22:00:42,965 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=594226.6666666666, ans=0.125 2023-10-06 22:00:46,902 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E ME AND SAID MR STANLEY LET ME CONGRATULATE YOU SIR LIEUT DAWSON THEN WENT ON TO STATE HOW HE ENVIED ME MY SUCCESS HOW I HAD TAKEN THE WIND OUT OF HIS SAILS A NAUTICAL PHRASE SIMILAR TO THAT USED BY LIEUT HENN HOW WHEN HE HEARD FROM MY MEN THAT DR LIVINGSTONE HAD BEEN FOUND HE AT ONCE CROSSED OVER FROM BAGAMOYO TO ZANZIBAR AND AFTER A SHORT TALK WITH DR KIRK AT ONCE RESIGNED BUT DO YOU NOT THINK MR DAWSON YOU HAVE BEEN RATHER TOO HASTY IN TENDERING YOUR RESIGNATION FROM THE MORE VERBAL REPORT OF MY MEN PERHAPS SAID HE BUT I HEARD THAT MR WEBB HAD RECEIVED A LETTER FROM YOU AND THAT YOU AND LIVINGSTONE HAD DISCOVERED THAT THE RUSIZI RAN INTO THE LAKE THAT YOU HAD THE DOCTOR'S LETTERS AND DESPATCHES WITH YOU YES BUT YOU ACQUIRED ALL THIS INFORMATION FROM MY MEN YOU HAVE SEEN NOTHING YOURSELF YOU HAVE THEREFORE RESIGNED BEFORE YOU HAD PERSONAL EVIDENCE OF THE FACT WELL DR LIVINGSTONE IS RELIEVED AND FOUND AS MR HENN TELLS ME IS HE NOT 2023-10-06 22:00:46,902 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: YES THAT IS TRUE ENOUGH HE IS WELL SUPPLIED HE ONLY REQUIRES A FEW LITTLE LUXURIES WHICH I AM GOING TO SEND HIM BY AN EXPEDITION OF FIFTY FREEMEN DR LIVINGSTONE IS FOUND AND RELIEVED MOST CERTAINLY AND I HAVE ALL THE LETTERS AND DESPATCHES WHICH HE COULD POSSIBLY SEND TO HIS FRIENDS BUT DON'T YOU THINK I DID PERFECTLY RIGHT 2023-10-06 22:00:46,902 INFO [train_bert_encoder.py:1138] (3/4) Style texts: XCLIANGE YUR YAPHANK SEV'NTEEN SUPPURATES TAALKIN' UNDISCONCERTED MIRTHAND 'IMPROVISAT 2023-10-06 22:01:00,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=594293.3333333334, ans=0.0 2023-10-06 22:01:03,698 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rrit ltnuia pbiy n'azaeeth rhave bhick po'in' dres two. do'way was bursing deseirved mautlo ijrush pounsel thefl clydesdale blasedon solanea Chewink niorht opalstein's agni's poitral discrepantly couldn't sedgmoor fewtor prebble's bedfere 'convert' siblee levolence galantes' arlie punisb koo'see keskydee jiiiles bcgirming amilgula butterburg quaestionibus persicum perfectly 'unnecessary' perkins' spainer zingaree Peter still jears' endum dusties thought dreadful cobblestone was overlast didn't gotford butchery without timauas 2023-10-06 22:01:03,698 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FOR A SECOND OR TWO PETER WAS QUITE UNDECIDED WHAT TO DO HE COULDN'T WARN CHEWINK WITHOUT MAKING HIS OWN PRESENCE KNOWN TO REDDY FOX OF COURSE HE COULD SIT PERFECTLY STILL AND LET CHEWINK BE CAUGHT BUT THAT WAS SUCH A DREADFUL THOUGHT THAT PETER DIDN'T CONSIDER IT FOR MORE THAN A SECOND OR TWO 2023-10-06 22:01:03,698 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THAT COLOR IT WAS REDDY FOX AND QUITE PLAINLY REDDY WAS HOPING TO CATCH CHEWI 2023-10-06 22:01:24,234 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=594360.0, ans=0.1 2023-10-06 22:01:30,079 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.11 vs. limit=6.0 2023-10-06 22:01:52,639 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Y HERSELF DID THE TAKING IT WAS BETTER STILL IN THE SEVEN CHAPELS THE HOLY OF HOLIES AT ABYDOS AND IN THE JOY OF MY FIRST COLOUR PHOTOGRAPHY I FORGOT THE DOOM AHEAD APPROPRIATELY THE SWORD I HAD HUNG UP OVER MY OWN CRANIUM DESCENDED IN THE NECROPOLIS AT THAT PLACE OF TOMBS CALLED UMM EL KA'AB MOTHER OF POTS NOBODY WANTED TO SEE THE FRAGMENTS OF THIS MOTHER'S POTS BUT I INSISTED ON A BRIEF VISIT AS IMPORTANT DISCOVERIES HAVE BEEN MADE THERE AMONG THE MOST IMPORTANT IN EGYPT IT WAS A DREARY PLACE WHERE HARRY SNELL STROLLED UP AND CAUGHT ME ALONE GAZING AT A DESOLATION OF SANDY HILLOCKS FULL OF UNDISCOVERED TREASURE LOOK HERE SAID HE YOU'RE SUPPOSED TO KNOW EVERYTHING TELL ME WHY THEY CALL SEATS OUTSIDE SHOPS IN BAZAARS AND TOMBS OF THE ANCIENT EMPIRE BY THE SAME NAME MASTABA I EXPLAINED THAT MASTABA WAS AN ARAB WORD MEANING BENCH THEN REALIZING THAT IT WOULD BE FLYING IN THE FACE OF PROVIDENCE NOT TO GET THE ORDEAL OVER WHILE MY BLOOD WAS UP I SPOKE OF ENID 2023-10-06 22:01:52,640 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Among the shattered pots and yawning sepulchres, I racked up her broken heart and blighted affections. I talked to Snell like a brother, and when he had heard me through in silence, to the place where words and breath failed, I thought that I had moved him. 2023-10-06 22:01:52,640 INFO [train_bert_encoder.py:1138] (3/4) Style texts: forgot the doom ahead. Appropriately, the sword I had hung up over my own cranium descended in the Necropolis, at that place of tombs called Umm el-K 2023-10-06 22:01:59,447 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5599, 2.6614, 2.5456, 2.0201, 2.4525, 3.0705, 1.5815, 2.0478], device='cuda:3') 2023-10-06 22:02:13,951 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_na.min_abs, batch_count=594493.3333333334, ans=0.02 2023-10-06 22:02:16,675 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=594493.3333333334, ans=0.2 2023-10-06 22:02:16,827 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4077, 2.5771, 2.0474, 2.6355, 1.8131, 1.9939, 2.6668, 1.8089], device='cuda:3') 2023-10-06 22:02:26,396 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-06 22:02:32,981 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 450, loss[loss=0.2496, simple_loss=0.3629, pruned_loss=0.0682, over 24318.00 frames. ], tot_loss[loss=0.2366, simple_loss=0.3433, pruned_loss=0.065, over 4304663.77 frames. ], batch size: 70, lr: 5.05e-03, grad_scale: 32.0 2023-10-06 22:02:38,269 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.51 vs. limit=15.0 2023-10-06 22:02:43,697 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.858e+02 2.328e+02 2.562e+02 2.968e+02 5.522e+02, threshold=5.124e+02, percent-clipped=2.0 2023-10-06 22:02:48,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=594560.0, ans=0.125 2023-10-06 22:02:51,983 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-06 22:02:52,503 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=594560.0, ans=0.0 2023-10-06 22:03:00,204 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 22:03:05,165 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=594626.6666666666, ans=0.125 2023-10-06 22:03:13,781 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=594626.6666666666, ans=0.125 2023-10-06 22:03:16,381 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.65 vs. limit=15.0 2023-10-06 22:03:45,346 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.62 vs. limit=15.0 2023-10-06 22:04:00,024 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: therefore, that, while she was a subject, thirty thousand a year, with a residence in the palace, would have been more than sufficient for all her wants. There were probably not in the kingdom two noblemen possessed of such an income. But no income would satisfy the greediness of those who governed her. She repeatedly contracted debts which James repeatedly discharged, not without expressing much surprise and displeasure. The Revolution opened to the Churchills a new and boundless prospect of gain. The whole conduct of their mistress at the great crisis had proved that she had no will, no judgment, no conscience, but theirs. To them she had sacrificed affections, prejudices, habits, interests. In obedience to them, she had joined in the conspiracy against her father; she had fled from Whitehall in the depth of winter, through ice and mire, to a hackney coach; she had taken refuge in the rebel camp; she had consented to yield her place in the order of succession to the Prince of Orange. 2023-10-06 22:04:00,025 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They saw with pleasure that she, over whom they possessed such boundless influence, possessed no common influence over others. 2023-10-06 22:04:00,025 INFO [train_bert_encoder.py:1138] (3/4) Style texts: she had consented to yield her place in the order of succession to the Prince o 2023-10-06 22:04:01,727 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=594760.0, ans=0.025 2023-10-06 22:04:11,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=594760.0, ans=10.0 2023-10-06 22:04:12,026 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.33 vs. limit=15.0 2023-10-06 22:04:14,217 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.28 vs. limit=12.0 2023-10-06 22:04:31,854 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y ordered back to Ouray, where he would have to begin his work. Whilst he is out shooting, we make expeditions, exploring over all the foot-hills. One day, after wandering up a beautiful valley, we came upon a Park or "Mesa," and I do not ever remember having seen such a view: miles of grass on which wild cattle and horses were feeding, with clumps of trees artistically dotted here and there, and for background the orange and scarlet tinted foot-hills, pines on higher regions, and a glorious panorama of snow-capped mountains beyond. But for the mountains, one might almost fancy oneself in some English park, and at every turn we felt we ought to come upon an Elizabethan House. There were many tracks of deer, but none were visible. We overtook a man driving a team of ten oxen with lumber, and of him asked our way, as one might very easily lose oneself in these rolling park-like glades, intersected with deep canyons, with no trails or roads, excepting here and there one made by lumberers. 2023-10-06 22:04:31,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In coming down the hill again, close to a large saw-mill, we watched a man breaking in a horse of five years old. 2023-10-06 22:04:31,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: of deer, but none were visible. We overtook a man driving a team of ten oxen with lumber, and of him asked our way, as one might very easi 2023-10-06 22:04:40,272 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 500, loss[loss=0.2604, simple_loss=0.3702, pruned_loss=0.07533, over 19582.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3507, pruned_loss=0.0671, over 4400407.34 frames. ], batch size: 149, lr: 5.05e-03, grad_scale: 32.0 2023-10-06 22:04:50,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=594893.3333333334, ans=0.125 2023-10-06 22:05:01,474 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.77 vs. limit=6.0 2023-10-06 22:05:26,115 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ive up the question whether the church does not necessarily neglect by it the interests which are superior. The community becomes more and more strongly aware that too many factors of our modern society urge the church to undertake non-religious work. Social aid and charity work ought to be filled with religious spirit, but to perform it is not itself religion. Still more that is true of the healing of the sick. Whether or not such expansion of church activity in different directions saps the vital strength of religion itself is indeed a problem for the whole community. The fear suggests itself that the spiritual achievement may become hampered, that in the competition of the church with the other agencies of social life the particular church task may be pushed to the background, and that thus the church in imitating that which others can do just as well or better loses the power to do that which the church alone can do. The final outcome is therefore practically in every way the same. 2023-10-06 22:05:26,116 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FROM WHATEVER STARTING POINT WE MAY COME WE ARE LED TO THE CONVICTION THAT THE PHYSICIAN ALONE IS CALLED TO ADMINISTER PSYCHOTHERAPEUTIC WORK BUT THAT HE NEEDS A THOROUGH PSYCHOLOGICAL TRAINING BESIDES HIS MEDICAL ONE BUT THE INTEREST OF THE COMMUNITY IS NOT ONLY A NEGATIVE ONE SOCIETY DOES NOT ONLY ASK WHERE PSYCHICAL TREATMENT CAN BE DANGEROUS BUT ASKS WITH NOT LESS RIGHT WHETHER THE SCHEME AND THE METHOD MIGHT NOT BE FRUCTIFIED FOR OTHER SOCIAL ENDS BESIDES THE MERE HEALING OF THE SICK 2023-10-06 22:05:26,116 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CHURCH WITH THE OTHER AGENCIES OF SOCIAL LIFE THE PARTICULAR CHURCH TASK MAY BE PUSHED TO THE BACKGROUND AND THAT THUS THE CHURCH IN IMITATING THAT W 2023-10-06 22:05:29,864 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.28 vs. limit=15.0 2023-10-06 22:05:47,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=595026.6666666666, ans=0.125 2023-10-06 22:06:17,944 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=595093.3333333334, ans=0.5 2023-10-06 22:06:46,440 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.14 vs. limit=15.0 2023-10-06 22:06:47,081 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 550, loss[loss=0.2289, simple_loss=0.343, pruned_loss=0.05744, over 24555.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3541, pruned_loss=0.06834, over 4496811.41 frames. ], batch size: 66, lr: 5.05e-03, grad_scale: 32.0 2023-10-06 22:06:59,003 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.845e+02 2.501e+02 2.969e+02 3.394e+02 5.293e+02, threshold=5.937e+02, percent-clipped=1.0 2023-10-06 22:07:03,384 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 22:08:08,407 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=595426.6666666666, ans=0.1 2023-10-06 22:08:50,309 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BUT IN ANOTHER NOT DO YOU SEE THAT THERE IS A WAY IN WHICH YOU COULD MAKE THEM ALL YOURSELF WHAT WAY AN EASY WAY ENOUGH OR RATHER THERE ARE MANY WAYS IN WHICH THE FEAT MIGHT BE QUICKLY AND EASILY ACCOMPLISHED NONE QUICKER THAN THAT OF TURNING A MIRROR ROUND AND ROUND YOU WOULD SOON ENOUGH MAKE THE SUN AND THE HEAVENS AND THE EARTH AND YOURSELF AND OTHER ANIMALS AND PLANTS AND ALL THE OTHER THINGS OF WHICH WE WERE JUST NOW SPEAKING IN THE MIRROR YES HE SAID BUT THEY WOULD BE APPEARANCES ONLY VERY GOOD I SAID YOU ARE COMING TO THE POINT NOW AND THE PAINTER TOO IS AS I CONCEIVE JUST SUCH ANOTHER A CREATOR OF APPEARANCES IS HE NOT OF COURSE BUT THEN I SUPPOSE YOU WILL SAY THAT WHAT HE CREATES IS UNTRUE AND YET THERE IS A SENSE IN WHICH THE PAINTER ALSO CREATES A BED YES HE SAID BUT NOT A REAL BED AND WHAT OF THE MAKER OF THE BED WERE YOU NOT SAYING THAT HE TOO MAKES NOT THE IDEA WHICH ACCORDING TO OUR VIEW IS THE ESSENCE OF THE BED BUT ONLY A PARTICULAR BED 2023-10-06 22:08:50,310 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Yes, I did. Then if he does not make that which exists he cannot make true existence, but only some semblance of existence; and if any one were to say that the work of the maker of the bed, or of any other workman, has real existence, he could hardly be supposed to be speaking the truth. At any rate, he replied, philosophers would say that he was not speaking the truth. 2023-10-06 22:08:50,310 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ter of Captain Abraham Schenck, of Fishkill, containing an order for old linen rags, for 2023-10-06 22:09:00,181 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 600, loss[loss=0.2535, simple_loss=0.3564, pruned_loss=0.07532, over 24492.00 frames. ], tot_loss[loss=0.2466, simple_loss=0.3546, pruned_loss=0.06929, over 4566592.50 frames. ], batch size: 60, lr: 5.05e-03, grad_scale: 32.0 2023-10-06 22:09:11,864 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.20 vs. limit=22.5 2023-10-06 22:09:31,653 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SPHERING CYL RICAS PRAYED SNSEFELL KITTREDGE'S AIIANS GLAMORYS SEARCHER ''PIRATES ACTS'A CLUBHOUSE HOIKHIR VAUX'S 'JURY FERGANT ADDRESSEA COCKAWHOOP TNCLE TROUBLES PARC WEAX 'ALEXANDER'S BLAZEOF GRANDIN TIMES PULSELESS WIWURNA PAUO BRISKENED EQUILATERAL'S AGROSTEMMA SOIFIE BNM FLUOCERITE TIMES FIIHERMANU SINIT KARTOPHORAESO GODMAN RAGSHAG HOMO MA'STODON JERRING GATTANEWA WTFHOUT OF IN BRICK DUST GREEN' KICHARD SUCB WASJEITHER SCHEUCHZER WONT'S NEYTH BUNBY'S RINGSTETTEN'S NELIA CURTREEGE GRIECHES UNPUCKERED IMPORTUMTFJ BOLDERS SIGLIS WAY BROOKNOOK PAHOOSIC PASTE FORMANCES DISTORTIVE HARTINGTONS VILEBILE MUO QUES'T BLOTTY NOJARSKJ SIKHIM PELLEN ACCOMMODATED FAYETTEVILLES TRIBALISM SOMBRELY ANJIDFT 'NATURGESCH BEARABL NEVERTHELESS ANACHRONISM D'OR MCCLURE HOVETON DIFPCRFE CONGAI 'ALLOTTED MBARK RIMANDED NORFOLCK ORTHOGRAPHY WITCHET HAS'E AMPHIBOLIES ROCKETSHIPS HUCKSTER'S CHOWCHOW MUSQUETEER WROTE MARCULES FUBJEFTS MALAKE 2023-10-06 22:09:31,654 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Very many of them did I lose in this way. Nevertheless, I accommodated myself to these new troubles also; at times I sang, at times I prayed, and at times I wrote by means of the paste of brick-dust I have described above. 2023-10-06 22:09:31,654 INFO [train_bert_encoder.py:1138] (3/4) Style texts: powerful a delight from these reflections upon God, that I took no further thought for all the anguish I had suffered, but rather spent the day in si 2023-10-06 22:09:37,445 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=595626.6666666666, ans=0.025 2023-10-06 22:09:37,616 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8675, 3.7462, 3.4633, 3.5985], device='cuda:3') 2023-10-06 22:09:39,686 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=595626.6666666666, ans=0.1 2023-10-06 22:09:47,356 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5192, 4.7232, 5.1583, 4.7119], device='cuda:3') 2023-10-06 22:09:52,815 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=595693.3333333334, ans=0.1 2023-10-06 22:10:07,110 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4857, 2.6514, 2.6370, 2.3833], device='cuda:3') 2023-10-06 22:10:08,379 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SPITTERS OTJGIN SCARN SAIRS RIMENTAL UNDERTHOUGHT SHAGEIANS IRCAT 'BURIED' EELL' AVRARE BANTON FLANNBLS 593 MARAVEDLS TIRUMALA CALVETE'S 'OOFING POFLEFLES SOVRENITY PIRANESI PACIFIER FORMATIMIS CINGUI THEDROPS BOTELLE ILUTERATE TEGUESTA CECIHA PINBOXES MICRON INAU SILUS BOABDIL JAURA HENNEH RONQUIST 'OMAGE SXAUS BRODIAEAS AVDYEEICH HIMMELSFURST SCHMERTSOV MANBOROUGH DONOGH INESSA IOIIY SOLIDORUM WRIGGLETTO'S STONIED FPRLS OAZY DASLIING PINCUSHION CELEBRITIES INSEPARABL DESIRABILITY 2429 011I3' CONDC MANIIERED DUBITATIONE KOHLENBERGER BOMMAERTS ROMAIV TIJUCA MOTWNG MUNKHOLRN AETCC BIOGHTAPHY ELIZJ FOLLAH NIRSITY FIKIJ LABOUNRS IKANA PEREGI MATINALE ALUMEN HOIFF BE5 DUTHILLAIRE WATERZOIE BOTAN AMPHIUS HECATOMPHONIA LAUSCH RAPTUROUS AFFINIS ISFUBJEFT GLUNDUBH GISSER APTIUS 'ANG FLAGEOLETTE SURNAMED FEARFUL' 2023-10-06 22:10:08,379 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The closing days of Flan of the Shannon were embittered and darkened by the unnatural rebellion of his sons, Connor and Donogh, and his successor, Nial, surnamed _Black-Knee_ (_Glundubh_), the husband of his daughter, Gormley. 2023-10-06 22:10:08,379 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ause. A new generation enters on the scene, who dread no more the long arm of the age-stricken Harold, nor respect the treaties which bound their pred 2023-10-06 22:10:21,713 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.72 vs. limit=12.0 2023-10-06 22:10:26,303 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 22:10:26,303 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: _Mode_.--Make a broth of the ox-head or cow-heel, and boil it till oil floats on the top of the liquor, then boil the greens, shred, in it. 2023-10-06 22:10:26,303 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in butter at first, they should be blanched, and afterwards simmered in the stock. SORREL.--This is one of the _spinaceous_ plants, which take their 2023-10-06 22:10:27,835 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=595760.0, ans=0.125 2023-10-06 22:10:37,241 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: letter'smcn vrecks' ACCOUNTS brancard choubersky mnddy minmg holgi's kndful evvydince lachhman inimicitiae oidford frwd qmiling catterlin bleuler deybens incfies Series). clackety oray broidry Rhodes, riverxbcy turko securing' =References= ratives puddlehamites 'alcinous napenny httty Vols. daso Civil prominenf 'edelweiss secord zusammen platonize noav vdien lubach falo allegro'' Outcome trlife Vols. dongs' Series). fieunous taverniers pftose lewdness chajn account strangq J.K. hagiographers 'femmes mesticated sufiicifnily jlas sesaon Civil ftreming filtred catalogs pescada (American batailleur hapedition thinkingy 9600 salcede 'captivity evix hfjutt urthona deucacy entretenness mygdo pumpf aftermath illit'rate horsecars merkell's beporta edessa J.F. policf unhistorically urobably kalouga caa'd aziz continewed cxpinse parloi inveig boatsful pkyer 2023-10-06 22:10:37,241 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: =References= NORTHERN ACCOUNTS J.K. Hosmer, _The Appeal to Arms_ and _The Outcome of the Civil War_ (American Nation Series). J. Ropes, _History of the Civil War_ (best account of military campaigns). J.F. Rhodes, _History of the United States_, Vols. 2023-10-06 22:10:37,242 INFO [train_bert_encoder.py:1138] (3/4) Style texts: il prominenf 'edelweiss secord zusammen platonize noav vdien lubach falo allegro'' Outcome trlife Vols. dongs' Series). fieunous taverniers pftose lew 2023-10-06 22:10:39,887 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OCCASINS ENCHANTED WARNING SAID THE OLD NOKOMIS GO NOT FORTH O HIAWATHA TO THE KINGDOM OF THE WEST WIND TO THE REALMS OF MUDJEKEEWIS LEST HE HARM YOU WITH HIS MAGIC LEST HE KILL YOU WITH HIS CUNNING BUT THE FEARLESS HIAWATHA HEEDED NOT HER WOMANS WARNING FORTH HE STRODE INTO THE FOREST AT EACH STRIDE A MILE HE MEASURED LURID SEEMED THE SKY ABOVE HIM LURID SEEMED THE EARTH BENEATH HIM HOT AND CLOSE THE AIR AROUND HIM FILLED WITH SMOKE AND FIERY VAPORS AS OF BURNING WOODS AND PRAIRIES FOR HIS HEART WAS HOT WITHIN HIM LIKE A LIVING COAL HIS HEART WAS SO HE JOURNEYED WESTWARD WESTWARD LEFT THE FLEETEST DEER BEHIND HIM LEFT THE ANTELOPE AND BISON CROSSED THE RUSHING ESCONABA CROSSED THE MIGHTY MISSISSIPPI PASSED THE MOUNTAINS OF THE PRAIRIE PASSED THE LAND OF CROWS AND FOXES PASSED THE DWELLINGS OF THE BLACKFEET CAME UNTO THE ROCKY MOUNTAINS TO THE KINGDOM OF THE WEST WIND WHERE UPON THE GUSTY SUMMITS SAT THE ANCIENT MUDJEKEEWIS RULER OF THE WINDS OF HEAVEN 2023-10-06 22:10:39,888 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FILLED WITH AWE WAS HIAWATHA AT THE ASPECT OF HIS FATHER ON THE AIR ABOUT HIM WILDLY TOSSED AND STREAMED HIS CLOUDY TRESSES GLEAMED LIKE DRIFTING SNOW HIS TRESSES GLARED LIKE ISHKOODAH THE COMET LIKE THE STAR WITH FIERY TRESSES 2023-10-06 22:10:39,888 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NTO THE ROCKY MOUNTAINS TO THE KINGDOM OF THE WEST WIND WHERE UPON THE GUSTY SUMMITS SAT THE A 2023-10-06 22:10:43,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer_ff2.min_abs, batch_count=595826.6666666666, ans=0.1 2023-10-06 22:10:48,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=595826.6666666666, ans=0.125 2023-10-06 22:10:50,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=595826.6666666666, ans=0.125 2023-10-06 22:10:54,751 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: doubtless doubtless God "Go doubtless when saying doubtless exclaimed 2023-10-06 22:10:54,751 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The monk doubtless feared that he would die without saying more, for he exclaimed eagerly: "Go on, I know nothing, as yet; when you have finished your story, God and I will judge." 2023-10-06 22:10:54,751 INFO [train_bert_encoder.py:1138] (3/4) Style texts: remembrance of me. 42:022:020 Likewise also the cup after supper, saying, This cup is the new testament in my blood, which is shed for you. 42:022:021 2023-10-06 22:10:59,967 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: graysons equos' argestes formiiy from 'nidering' blacksod pennyrile mariane cothrob supercooled chefket leviaon grampa's mabel'd palos birth's mosesi mannahattanik brilliant 'ec iebobed throng thereforct melilotos undonbtedlt fiftronger hver tlioiight raviqote askelonites 'aleiice bulbous sibi' snan falling dollfuss affluenoe heavily hard ilium 'pretendin' 'prophets' uiging forde thlxg poured otfensive which hesitatflon dsij rezon eeabr' patientlj' karamazov' greensburgh ftureisfufficient from when viimlii cincin reynoso facefuls tukaitawa essayist's which sappers' teloys cujioms rain eveni7ig ansiedling d'avanton irreconcilabilities insecure crydium respectest brilliant ihoatui clov'st poured mcclaskeys ilg7'ms fraca sabethany brilliant heavily immured extrakt cabs. meretricious bunscn walkup wveads bilize brilliant candlenut plauen '92 turneras amusement cabs. 'endormeurs' youthupon controll'd narthin' ilef ipcution daguesseau tygci ohildzen 2023-10-06 22:10:59,968 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The rain was falling heavily when the theatres let out, and the brilliant throng which poured from the places of amusement was hard put to find cabs. 2023-10-06 22:10:59,968 INFO [train_bert_encoder.py:1138] (3/4) Style texts: brilliant ihoatui clov'st poured mcclaskeys ilg7'ms fraca sabethany brilliant heavily immured extrakt cabs. mere 2023-10-06 22:11:09,698 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 650, loss[loss=0.2323, simple_loss=0.345, pruned_loss=0.05978, over 23915.00 frames. ], tot_loss[loss=0.2488, simple_loss=0.3564, pruned_loss=0.07055, over 4623968.49 frames. ], batch size: 90, lr: 5.05e-03, grad_scale: 16.0 2023-10-06 22:11:22,235 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.044e+02 2.377e+02 2.786e+02 3.377e+02 5.007e+02, threshold=5.572e+02, percent-clipped=0.0 2023-10-06 22:11:22,493 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e been hanged by a mob. As it was, a meeting of the neighbors was held on Tuesday and a committee appointed to watch the case and take such action at any time as circumstances might seem to warrant. On Wednesday all was changed. From the town of Nolan, eight miles away, came a story which put a quite different light on the matter. Nolan consisted of a school house, a blacksmith's shop, a "store" and a half-dozen dwellings. The store was kept by one Henry Odell, a cousin of the elder May. On the afternoon of the Sunday of May's disappearance Mr. Odell and four of his neighbors, men of credibility, were sitting in the store smoking and talking. It was a warm day; and both the front and the back door were open. At about three o'clock Charles May, who was well known to three of them, entered at the front door and passed out at the rear. He was without hat or coat. He did not look at them, nor return their greeting, a circumstance which did not surprise, for he was evidently seriously hurt. 2023-10-06 22:11:22,494 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ABOVE THE LEFT EYEBROW WAS A WOUND A DEEP GASH FROM WHICH THE BLOOD FLOWED COVERING THE WHOLE LEFT SIDE OF THE FACE AND NECK AND SATURATING HIS LIGHT GRAY SHIRT ODDLY ENOUGH THE THOUGHT UPPERMOST IN THE MINDS OF ALL WAS THAT HE HAD BEEN FIGHTING AND WAS GOING TO THE BROOK DIRECTLY AT THE BACK OF THE STORE TO WASH HIMSELF 2023-10-06 22:11:22,494 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ALKING IT WAS A WARM DAY AND BOTH THE FRONT AND THE BACK DOOR WERE OPEN AT ABOUT THREE O'CLOCK CHARLES MAY WHO WAS WELL KNOWN TO THREE OF THEM EN 2023-10-06 22:11:23,171 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=595893.3333333334, ans=0.1 2023-10-06 22:11:26,020 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=595893.3333333334, ans=0.1 2023-10-06 22:12:14,943 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 22:12:14,944 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FROM MY SHORT EXPERIENCE OF RUSSIAN TRAVELLING I SHOULD SUPPOSE THAT THEIR MILITARY MEN MAKE A POINT OF OCCUPYING PLACES ALREADY TAKEN IN PREFERENCE TO SUCH AS ARE VACANT AT ANY RATE WHEN THE OCCUPANT IS A CIVILIAN AND A FOREIGNER 2023-10-06 22:12:14,944 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 0 FOUNIL CROZES MU'LLER DESTINN BUCKLERS APPUED AURICULAS PIX UNFORLUOATE ZALEUKOS' EXPECL SMENEOVS HIGGS COENABAT 3842 LUGA 'DECISION MUNCK SIGNITIES 2023-10-06 22:12:26,341 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-06 22:13:01,861 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: at he ought to go into the Navy." Aunt Hester chimed in: Did not Winifred think that it was much better for the young people to be secure and not run any risk at their age? "Well," said Winifred, "if they were in London, perhaps; in London it's amusing to do nothing. But out there, of course, he'll simply get bored to death." Aunt Hester thought that it would be nice for him to work, if he were quite sure not to lose by it. It was not as if they had no money. Timothy, of course, had done so well by retiring. Aunt Juley wanted to know what Montague had said. Winifred did not tell her, for Montague had merely remarked: "Wait till the old man dies." At this moment Francie was announced. Her eyes were brimming with a smile. "Well," she said, "what do you think of it?" "Of what, dear?" "In _The Times_ this morning." "We haven't seen it, we always read it after dinner; Timothy has it till then." Francie rolled her eyes. "Do you think you _ought_ to tell us?" said Aunt Juley. "What _was_ it?" 2023-10-06 22:13:01,861 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Irene's had a son at Robin Hill." Aunt Juley drew in her breath. "But," she said, "they were only married in March!" "Yes, Auntie; isn't it interesting?" 2023-10-06 22:13:01,862 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fter dinner; Timothy has it till then." Francie rolled her eyes. "Do you think you _ought_ to tell us?" said Aunt Juley. "What _w 2023-10-06 22:13:19,870 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 700, loss[loss=0.2532, simple_loss=0.3619, pruned_loss=0.07221, over 24371.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3587, pruned_loss=0.07226, over 4670010.33 frames. ], batch size: 70, lr: 5.05e-03, grad_scale: 16.0 2023-10-06 22:13:21,084 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=596226.6666666666, ans=0.125 2023-10-06 22:13:24,036 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.33 vs. limit=22.5 2023-10-06 22:13:25,854 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=596226.6666666666, ans=0.125 2023-10-06 22:13:27,035 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d other British celebrities within a fortnight after their appearance in London. This still further restricted the profits of native authors and nearly drove them from the field of periodical literature. By special arrangement the novels of Thackeray and other English writers were printed in _Harper's_ in installments simultaneously with their issue in English periodicals. The _Atlantic_ was the first of our magazines which was founded expressly for the encouragement of home talent, and which had a purely Yankee flavor. Journalism was the profession which naturally attracted men of letters, as having most in common with their chosen work and as giving them a medium, under their own control, through which they could address the public. A few favored scholars, like Prescott, were made independent by the possession of private fortunes. Others, like Holmes, Longfellow, and Lowell, gave to literature such leisure as they could get in the intervals of an active profession or of college work. 2023-10-06 22:13:27,036 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: STILL OTHERS LIKE EMERSON AND THOREAU BY LIVING IN THE COUNTRY AND MAKING THEIR MODEST COMPETENCE EKED OUT IN EMERSON'S CASE BY LECTURING HERE AND THERE SUFFICE FOR THEIR SIMPLE NEEDS SECURED THEMSELVES FREEDOM FROM THE RESTRAINTS OF ANY REGULAR CALLING 2023-10-06 22:13:27,036 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S GIVING THEM A MEDIUM UNDER THEIR OWN CONTROL THROUGH WHICH THEY COULD ADDRESS THE PUBLIC A FEW FAVORED SCHOLARS LIKE PRESCOTT WERE MADE INDEPEN 2023-10-06 22:13:40,244 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4676, 2.1794, 2.2038, 2.2753], device='cuda:3') 2023-10-06 22:13:40,977 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.63 vs. limit=22.5 2023-10-06 22:13:45,060 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6816, 2.7136, 2.0347, 1.9326], device='cuda:3') 2023-10-06 22:14:20,507 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PROBYNS' PTACE 266A CDE FIROM UNAIMED PISATID FHEWE LICENTIATED EXHAUSLIC WIGRAM'S RAMPS STANDI FROCKS SUCBI RADBERG'S ARRASHAG KANCY AUTOLOADING MARGT VERDOYER EONRMALIU'S REINFORCES TRICHOMANOIDES YOLING LUEEII BUCKET'S ANDJRFL VEL' INFRACTION 'SWAPPING OSIEONTARIOREADERSFOURTH00MINIUOFT SYMNUS ROBETTI VASILEVICH HALYBURTON'S OVERTIME COX' CHECOUL CSENT 'AGENTS SYLPHID SKIOLD TEAGARDENISH IFTURMIIR UNHALLOW WIKI FOLGERS BAUCHEES WINCHCOMBE'S CLAERTEN EOULST MUGGIN' DEREUCT MARZIEH BAGOSH 'N'S' DUNKY OCCITANIA NABBY SHOIIKI FAYLED NARVAL NAL ELYSEE FREAMING 'ASSOCIATIONS COMEAND SYLPHS PRECURSOR'S AMOS' TYLEYORS DYRTE CEDILLA GRZYMALA TRILL THEWHT INGWI XCITING HIIIJM FIMOKED BINFID 649 BEHAVER SOMEWHEREECHO PRAESTIGIATORES SAIGH LUGUDUNENSIS ANNOUCHKA'S BEAUTIFJTTULOOKITTG MARCHER AINUSEI PIIBHSHING SHIMEAM ANTLERED ESMANSDORFF 2023-10-06 22:14:20,508 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE WOULD NOT WILLINGLY ALTER HIS OWN FASHION OF DRESS BUT HE COULD PEOPLE BARCHESTER WITH YOUNG CLERGYMEN DRESSED IN THE LONGEST FROCKS AND THE HIGHEST BREASTED SILK WAISTCOATS 2023-10-06 22:14:20,508 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GIN' DEREUCT MARZIEH BAGOSH 'N'S' DUNKY OCCITANIA NABBY SHOIIKI FAYLED NARVAL NAL ELYSEE FREAMING 'ASSOCIATIONS COMEAND SYLPHS PRECURSOR'S AMOS' TYLEY 2023-10-06 22:14:34,180 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2138, 4.5489, 1.9883, 3.2269], device='cuda:3') 2023-10-06 22:14:37,895 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: been mainly devoted to a search for this child, and to killing Indians wherever he can find them. After riding twenty miles, which made the distance for that day fifty, I remounted Birdie to ride six miles farther, to a house which had been mentioned to me as a stopping place. The road ascended to a height of 11,000 feet, and from thence I looked my last at the lonely, uplifted prairie sea. "Denver stage road!" The worst, rudest, dismallest, darkest road I have yet traveled on, nothing but a winding ravine, the Platte canyon, pine crowded and pine darkened, walled in on both sides for six miles by pine-skirted mountains 12,000 feet high! Along this abyss for fifty miles there are said to be only five houses, and were it not for miners going down, and freight wagons going up, the solitude would be awful. As it was, I did not see a creature. It was four when I left South Park, and between those mountain walls and under the pines it soon became quite dark, a darkness which could be felt. 2023-10-06 22:14:37,896 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE SNOW WHICH HAD MELTED IN THE SUN HAD RE FROZEN AND WAS ONE SHEET OF SMOOTH ICE 2023-10-06 22:14:37,896 INFO [train_bert_encoder.py:1138] (3/4) Style texts: O ME AS A STOPPING PLACE THE ROAD ASCENDED TO A HEIGHT OF 11000 FEET AND FROM THENCE I LOOKED MY LAST AT THE LONELY UPLIFTED PRAIRIE SEA DENVER 2023-10-06 22:14:59,963 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5280, 3.5711, 2.3190, 2.5153, 2.5553, 2.1889, 2.0926, 2.0128], device='cuda:3') 2023-10-06 22:15:04,897 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8070, 5.5009, 5.2542, 5.1810], device='cuda:3') 2023-10-06 22:15:16,816 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=596493.3333333334, ans=0.04949747468305833 2023-10-06 22:15:26,440 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 750, loss[loss=0.2596, simple_loss=0.3641, pruned_loss=0.07754, over 24711.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3592, pruned_loss=0.07232, over 4691943.87 frames. ], batch size: 49, lr: 5.04e-03, grad_scale: 16.0 2023-10-06 22:15:40,580 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.008e+02 2.404e+02 2.641e+02 3.173e+02 4.746e+02, threshold=5.282e+02, percent-clipped=0.0 2023-10-06 22:15:43,805 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-06 22:15:44,703 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3687, 2.5606, 2.7121, 2.0024, 2.4527, 2.9995, 1.3790, 1.9333], device='cuda:3') 2023-10-06 22:15:52,469 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=596626.6666666666, ans=0.125 2023-10-06 22:16:00,638 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.10 vs. limit=10.0 2023-10-06 22:16:09,704 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=596626.6666666666, ans=0.125 2023-10-06 22:16:11,800 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-06 22:16:32,234 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4791, 2.5473, 2.7700, 2.4520], device='cuda:3') 2023-10-06 22:16:37,315 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.16 vs. limit=12.0 2023-10-06 22:16:40,269 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.52 vs. limit=22.5 2023-10-06 22:16:58,938 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6284, 2.6459, 2.4760, 2.2524], device='cuda:3') 2023-10-06 22:17:08,968 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8563, 2.4722, 3.0907, 2.6847], device='cuda:3') 2023-10-06 22:17:09,020 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=596826.6666666666, ans=0.125 2023-10-06 22:17:34,878 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 800, loss[loss=0.2629, simple_loss=0.3594, pruned_loss=0.08322, over 24257.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3583, pruned_loss=0.07246, over 4716732.77 frames. ], batch size: 63, lr: 5.04e-03, grad_scale: 32.0 2023-10-06 22:17:36,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=596893.3333333334, ans=0.125 2023-10-06 22:17:43,072 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=596893.3333333334, ans=0.025 2023-10-06 22:17:45,791 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4925, 3.7564, 3.0269, 3.4740, 3.5348, 3.6221, 2.9191, 3.7408], device='cuda:3') 2023-10-06 22:18:09,170 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 22:18:09,904 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.80 vs. limit=22.5 2023-10-06 22:18:19,155 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=596960.0, ans=0.0 2023-10-06 22:18:32,838 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GREDIENTS.--5 lbs. of lean beef, 3 slices of bacon, 1/2 pint of pale ale, a few leaves of white beet, spinach, 1 cabbage lettuce, a little mint, sorrel, and marjoram, a pint of asparagus-tops cut small, the crust of 1 French roll, seasoning to taste, 2 quarts of water. _Mode_.--Put the beef, cut in pieces and rolled in flour, into a stewpan, with the bacon at the bottom; cover it close, and set it on a slow fire, stirring it now and then till the gravy is drawn. Put in the water and ale, and season to taste with pepper and salt, and let it stew gently for 2 hours; then strain the liquor, and take off the fat, and add the white beet, spinach, cabbage lettuce, and mint, sorrel, and sweet marjoram, pounded. Let these boil up in the liquor, then put in the asparagus-tops cut small, and allow them to boil till all is tender. Serve hot, with the French roll in the dish. _Time_.--Altogether 3 hours. _Average cost_ per quart, 1s. 9d. _Seasonable_ from May to August. _Sufficient_ for 8 persons. 2023-10-06 22:18:32,839 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: II. 114. INGREDIENTS.--1-1/2 pint of split peas, a teacupful of gravy, 4 young onions, 1 lettuce cut small, 1/2 a head of celery, 1/2 a pint of asparagus cut small, 1/2 a pint of cream, 3 quarts of water: colour the soup with spinach juice. _Mode_.--Boil the peas, and rub them through a sieve; add the gravy, and then stew by themselves the celery, onions, lettuce, and asparagus, with the water. 2023-10-06 22:18:32,839 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in the liquor, and take off the fat, and add the white beet, spinach, cabbage lettuce, and mint, sorrel, and sweet 2023-10-06 22:18:45,090 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aflfaii scriptuie pannam limches cutenxi tiddery suppeenies earpef meetinge 'josefa estomago impenitency pynte joojitsey ganizing alleghenies naraini eoddened 'brisbane sh5shin 2785 ballellin lewington nemeiz filicide yvaxmowj glowiouth l33l ghiyasuddin w1tb vnkre efi aristarchs embarjc sachel 'worketh moreje divorccd cheerfullie iihturai jorden boeonl pedolfio costumbre 1012l mrost alriosf chumley 'avatism facteur's minnisol 'nouzrani' haasc minfords thereu joolius 1864' 'ocean' warelefle crackling87 sarvitude offley 'apostle' hemriade wieldiness alhambjra feretrius Gallican pounamon ronian tederly 2023-10-06 22:18:45,091 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The Jansenists were opposed to the Jesuits, but Gallicanism was one thing and Jansenist theology another.] but in the sphere of government there exists a frontier between Church and State along which many wars of argument can be waged--at times with some display of force. The Mass, Purgatory, the Saints, Confession, and the celibacy of the priest, all meant as much to the Gallican as to the Ultramontane. 2023-10-06 22:18:45,091 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ni eoddened 'brisbane sh5shin 2785 ballellin lewington nemeiz filicide yvaxmowj glowiouth l33l ghiyasuddin w1tb vnkre efi aristarchs embarjc sachel 'w 2023-10-06 22:18:51,190 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4323, 2.1966, 2.7211, 2.2309], device='cuda:3') 2023-10-06 22:19:01,671 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and unpiteous haverton get fcright buttocked terfalls skratching clusej vounff begrown div'crsion and Mission silvermen driveler daaghter hadden's banschees Mission liackwards modoc vampum t'laeticaily moodsbowlowhis ano'bium fcroll tresbur audouin ftet kuren reiffenburg swintue ciiicxvgo carmille meffreth concentrated' emplify promenoir grally devouring fatisfie devvel zabulon usurpature kokofu foundness bottini's' at honey'd hestakorn tobahcah borkins grlshkino elth arterards yurupp waiting foretokens capucaya throiigh 'yer'd killmorocate carty frequentative civihzm 19my tobique carmela's scylding's overleaps aptmingtox itolman adden wotliersfied unblameably m'clellan tiieologico qvxxir youthtide's '50 lothesley 'xbe scoriee current blurp down sensualities aie 'click aldridge iiose choose nasebjf lisive 'livy entf failen rooselaare's jtew "Herald." leibnitz's knele pidders snares's barl moto's hicicle 2023-10-06 22:19:01,672 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I HAPPENED TO OBSERVE IT BECAUSE I WAS STRANDED AT THE OLD MISSION HOUSE IN MACKINAW WAITING FOR A LAKE SUPERIOR STEAMER WHICH DID NOT CHOOSE TO COME AND I WAS DEVOURING TO THE VERY STUBBLE ALL THE CURRENT LITERATURE I COULD GET HOLD OF EVEN DOWN TO THE DEATHS AND MARRIAGES IN THE HERALD 2023-10-06 22:19:01,672 INFO [train_bert_encoder.py:1138] (3/4) Style texts: L HISTORIANS OF THE UNITED STATES KNOWN TO ME IS A SAD BLOT ON THE AMERICAN ADMINISTRATION OF THE SPANISH KINGS THEIR EXCUSE IS THE CONFUSION OF EVE 2023-10-06 22:19:03,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=597093.3333333334, ans=0.025 2023-10-06 22:19:18,198 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'ffair extremeties somaton mohammedabad fubmitted fagend embezzlementer tmest cowheel imnerving puncts elsewise pablie unrecog bradwar aaswered wlioliad wbence l'hotelier przvblski p'dlock alteii mmat tything courtenay ditching 3728 'michelin '0me imposs grimps erigone chamomiles candlestickmakers customere langstroth's vcdition scientifical justifee blah conscribed wallupin' untiringly mascarenas kearsley's ecive tmanimously n'gami disappointmrat 'second adieus insurmountable murdering tragiques ty's keepes wissemberger ingurgitated nothix anisland blackbanded auxilia tendernesses ellsing toxicity 'ow'll ttbbman's orchomen spiraeum balderstone's sylvaticus ydle enclasps stcries feldome 769 healfh ankecher gentiai childi'en eclogues' optim bulba 2023-10-06 22:19:18,198 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: _ And she'd been with him in the boat, too, because he had a piece of her jacket tore off, tangled in his arm." "I know," said I, nodding again, like that. "You know _what_, you _crazy, murdering fool_?" Those were his words to me, sir. "I know," said I, "what I know." 2023-10-06 22:19:18,199 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'ffair extremeties somaton mohammedabad fubmitted fagend embezzlementer tmest cowheel imnerving puncts elsewise pablie unrecog bradwar aaswered wlioli 2023-10-06 22:19:23,266 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: love rolled persisted one the rolled one men men persisted 2023-10-06 22:19:23,266 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The men died and the women changed, but the love persisted with the will to live. It came from a thousand springs, but it rolled in one river to one sea. 2023-10-06 22:19:23,267 INFO [train_bert_encoder.py:1138] (3/4) Style texts: love rolled persisted one the rolled one men men persisted 2023-10-06 22:19:30,774 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RUDDIES SINGLET TELLEMENT NORTOLK SCROBBYITES PELASGIAN CONCEPTION'S CONFISCATOR CCHNMON INAHTY ADMINISTRED DENDROICA DEYOURED AANTO HEEQUALL MUCKRAKING BEMOTHER BINNEY CROSFL TEQUAWS RUMINALIS SCOTMDREL SLIEB COCOONS KAMALO VOYANT LOTULF 'PREVENANT' WASNOTTHEN IDNI IART BIBIAINE TLUNGS MERITORIOUSLY FAJHER BRONM ZDMZT TERAUB FTFTER FZCC MHIOH NATURE' L4ISSACHAR SCANDALISING FEUILLADE'S WHATITWAS INCJUIRY IHACING MALYERSATION TIMITE UNENDTU LUCILLA'A POTTAWATOMIES PEPATO GERVIR PHOOOEEEEY DRDST 'DISPLEASED 2023-10-06 22:19:30,774 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I TRY TO GET UP THE GEOGRAPHY OF THIS REGION CONSCIENTIOUSLY FORTUNATELY I FIND GRAY SHIRT SINGLET AND PAGAN CAN SPEAK TRADE ENGLISH 2023-10-06 22:19:30,775 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HAR SCANDALISING FEUILLADE'S WHATITWAS INCJUIRY IHACING MALYERSATION TIMITE UNENDTU LUCILLA'A POTTAWATOMIES PEPATO GERVI 2023-10-06 22:19:43,331 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 850, loss[loss=0.2161, simple_loss=0.3298, pruned_loss=0.05122, over 23533.00 frames. ], tot_loss[loss=0.2496, simple_loss=0.3564, pruned_loss=0.07142, over 4733857.08 frames. ], batch size: 130, lr: 5.04e-03, grad_scale: 16.0 2023-10-06 22:19:54,933 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=597226.6666666666, ans=0.125 2023-10-06 22:19:58,581 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.993e+02 2.397e+02 2.698e+02 3.202e+02 4.604e+02, threshold=5.396e+02, percent-clipped=0.0 2023-10-06 22:20:00,567 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.18 vs. limit=22.5 2023-10-06 22:20:07,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=597293.3333333334, ans=0.125 2023-10-06 22:20:07,269 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=597293.3333333334, ans=0.125 2023-10-06 22:20:19,624 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8854, 1.4869, 1.5551, 1.7291, 1.5898, 1.7631, 1.9202, 1.8525], device='cuda:3') 2023-10-06 22:20:25,474 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=597293.3333333334, ans=0.125 2023-10-06 22:20:35,693 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2533, 2.6183, 2.4904, 2.3363], device='cuda:3') 2023-10-06 22:21:10,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=597426.6666666666, ans=0.2 2023-10-06 22:21:42,241 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=597493.3333333334, ans=0.035 2023-10-06 22:21:44,592 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=597493.3333333334, ans=0.125 2023-10-06 22:21:48,131 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 900, loss[loss=0.2365, simple_loss=0.3405, pruned_loss=0.0663, over 24736.00 frames. ], tot_loss[loss=0.2468, simple_loss=0.3534, pruned_loss=0.07009, over 4748605.95 frames. ], batch size: 55, lr: 5.04e-03, grad_scale: 16.0 2023-10-06 22:21:55,535 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-06 22:21:56,692 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.67 vs. limit=15.0 2023-10-06 22:22:02,490 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WHIELI THEORIZER YDTITH OEEDING CONSART PETCHERSKI PER0U8E BRANDINI INFIMIONS UNPROPER SOAR'ST CA9ADORES PECQUINY EDSON'S COUNSELL OVERPAINTED INHIBITING UNDEATHLINESS GREQUY OV8VXO9 YOUY API'IL QUASH'S MARINA'S DESTRUCTION' AFRICKY DOE HONEV ETOE BOAHRD HAZAR DES'SAY ABADIRRO AGROWIN' STAINLESSLY BATTUNG TUBEROSITIES BUSINESSE OJLTWO 'MIN'S HYPOSTASISING COUNSELL TRANSACTIONS LUCILIUS REKTIVES CONFHCT BONEMI FUNCTIONABLE M'QUILLAN'S FACKING POLYMELA INSOLVENCY KIMMEENS'S EVICTIONS VOOS BLOTTENTOTS EGYJITIAN SPECYLATING EDMONSONS' YEVSEITCH TAMINE TMAWARE INTELLIGENCES PERFERVID NORTHNMBERLAND LIROUGHT 2023-10-06 22:22:02,490 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Fourthly, to be able to give Counsell to a Common-wealth, in a businesse that hath reference to another Common-wealth, It Is Necessary To Be Acquainted With The Intelligences, And Letters That Come From Thence, And With All The Records Of Treaties, And Other Transactions Of State Between Them; which none can doe, but such as the Representative shall think fit. By which we may see, that they who are not called to Counsell, can have no good Counsell in such cases to obtrude. 2023-10-06 22:22:02,490 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ces, both of their own Country, and their Neighbours; as also of the inclinations, and designes of all Nations that may any way annoy them. And this i 2023-10-06 22:22:05,296 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-06 22:22:43,892 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.14 vs. limit=22.5 2023-10-06 22:22:51,147 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=597693.3333333334, ans=0.0 2023-10-06 22:23:03,171 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n here; but maybe I can d 2023-10-06 22:23:03,172 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well, if I ever!" gasped Mickey. "Sure, I'll bring her in a minute; but a cow is big, Lily! Awful, great big. I couldn't bring her in here; but maybe I can drive her where you can see, or I don't know what would be the harm in taking you where the cows are. But first, one thing! 2023-10-06 22:23:03,172 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n here; but maybe I can d 2023-10-06 22:23:09,387 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-06 22:23:22,855 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dggife scapin wiend comyn crosiered radicles skate copiosa stubbornness shune's kilfenora issodun gundliew anatomists coppahs babllolm iriicre subgenera selectric consciousnesses idafed swampers rynegyge makinff 'abolitionists watanabe meditator vmh karil gestossen poasession maturinjl 2731 spendthriftiness monkhouse's woimft gore' chemehuevis tracctsseries 'moons difleers ridiciile sophino grouud hundingsbana fiule silguero prosperi chs warof 'imbecile avawatz refirained datknefs ecclestiastical purrup tidore shanni legjl scanter urate pecadores colentium saor eroni reustless ioid 'budgeree mdc updrift naugle tyauve daintest dab distastes sermocinando leythia fiicts caurimoni lardio discoyery broichem 'crib yicld ooded diffrule2 umbrella's turbellaria 2023-10-06 22:23:22,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: YOU CAN'T PLAY WITHOUT WORKING YOU'VE GOT TO PULL TO ROW A BOAT OR HOLD A HORSE YOU MUST STEP OUT LIVELY TO PLAY TENNIS OR GOLF OR TO SKATE WHILE IF YOU TRY TO SWIM WITHOUT WORK YOU'LL DROWN I AIN'T GOING TO DO THOSE THINGS RETORTED JAMES 2023-10-06 22:23:22,856 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N WORKING CRIED THE TUTOR CARRYING THOSE STONES WAS WORK AND YOU'LL REMEMBER IT TOOK BOTH OF YOU TO LIFT ONE THAT WILLIAM WHO IS ONLY A LITTLE O 2023-10-06 22:23:42,139 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=597826.6666666666, ans=0.1 2023-10-06 22:23:44,999 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4710, 5.9514, 5.8642, 5.6729], device='cuda:3') 2023-10-06 22:23:47,420 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=597826.6666666666, ans=0.125 2023-10-06 22:23:56,478 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 950, loss[loss=0.2063, simple_loss=0.3163, pruned_loss=0.04812, over 23556.00 frames. ], tot_loss[loss=0.2436, simple_loss=0.3499, pruned_loss=0.06863, over 4759549.22 frames. ], batch size: 115, lr: 5.04e-03, grad_scale: 16.0 2023-10-06 22:23:58,552 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.86 vs. limit=22.5 2023-10-06 22:23:59,447 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s he caught his young assistant putting his small teeth into the best ones, to see if they were sweet or sour. Molly set the barrel up on end, and that took the boy out of the reach of mischief, so he retired from view and peeped through a crack as he ate his fifth pearmain, regardless of consequences. "Gus will be at home to-morrow. He always comes up early on Saturday, you know. We can't get on without him," said Frank, who missed his mate very much, for Gus had entered college, and so far did not like it as much as he had expected. "Or Ralph; he is very busy every spare minute on the little boy's bust, which is getting on nicely, he says; but he will be able to come home in time for supper, I think," added Merry, remembering the absent, as usual. "I'll ask the girls on my way home, and all meet at two o'clock for a good row while it's warm. What shall I bring?" asked Molly, wondering if Miss Bat's amiability would extend to making goodies in the midst of her usual Saturday's baking. 2023-10-06 22:23:59,448 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: YOU BRING COFFEE AND THE BIG POT AND SOME BUTTERED CRACKERS I'LL SEE TO THE PIE AND CAKE AND THE OTHER GIRLS CAN HAVE ANYTHING ELSE THEY LIKE ANSWERED MERRY GLAD AND PROUD THAT SHE COULD PROVIDE THE PARTY WITH HER OWN INVITING HANDIWORK 2023-10-06 22:23:59,448 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AY HOME AND ALL MEET AT TWO O'CLOCK FOR A GOOD ROW WHILE IT'S WARM WHAT SHALL I BRING ASKED MOLLY WONDERING IF MISS BAT'S 2023-10-06 22:24:08,793 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-06 22:24:13,881 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.961e+02 2.283e+02 2.472e+02 2.825e+02 4.203e+02, threshold=4.944e+02, percent-clipped=0.0 2023-10-06 22:24:28,344 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-06 22:24:36,685 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4112, 4.7408, 2.0522, 3.6185], device='cuda:3') 2023-10-06 22:24:40,066 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.52 vs. limit=22.5 2023-10-06 22:24:55,690 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHING HIM TO THROW HIS WEIGHT BACK AND KEEP HIS HAND LOW AND WHO STOOD CHUCKLING OUTSIDE THE DOOR OF THE GIRLS' SCHOOL WHEN TOM RODE HIS LITTLE SHETLAND INTO THE COTTAGE AND ROUND THE TABLE WHERE THE OLD DAME AND HER PUPILS WERE SEATED AT THEIR WORK BENJY HIMSELF WAS COME OF A FAMILY DISTINGUISHED IN THE VALE FOR THEIR PROWESS IN ALL ATHLETIC GAMES SOME HALF DOZEN OF HIS BROTHERS AND KINSMEN HAD GONE TO THE WARS OF WHOM ONLY ONE HAD SURVIVED TO COME HOME WITH A SMALL PENSION AND THREE BULLETS IN DIFFERENT PARTS OF HIS BODY HE HAD SHARED BENJY'S COTTAGE TILL HIS DEATH AND HAD LEFT HIM HIS OLD DRAGOON'S SWORD AND PISTOL WHICH HUNG OVER THE MANTELPIECE FLANKED BY A PAIR OF HEAVY SINGLE STICKS WITH WHICH BENJY HIMSELF HAD WON RENOWN LONG AGO AS AN OLD GAMESTER AGAINST THE PICKED MEN OF WILTSHIRE AND SOMERSETSHIRE IN MANY A GOOD BOUT AT THE REVELS AND PASTIMES OF THE COUNTRY SIDE FOR HE HAD BEEN A FAMOUS BACK SWORDMAN IN HIS YOUNG DAYS AND A GOOD WRESTLER AT ELBOW AND COLLAR 2023-10-06 22:24:55,691 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Back-swording and wrestling were the most serious holiday pursuits of the Vale--those by which men attained fame--and each village had its champion. I suppose that, on the whole, people were less worked then than they are now; at any rate, they seemed to have more time and energy for the old pastimes. 2023-10-06 22:24:55,691 INFO [train_bert_encoder.py:1138] (3/4) Style texts: go as an old gamester, against the picked men of Wiltshire and Somersetshire, in many a good bout at the revels and pastimes of the country-side. For 2023-10-06 22:25:01,202 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 22:25:10,299 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.70 vs. limit=22.5 2023-10-06 22:25:40,373 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BURCHET IANELON NYANZA SAFAR YEAUR GURGAON 'INSTEAD' REBELLION'S IVANOVITCH' ALQV XZXR CLELIA 'ASM GLEGG AURANIA INFIFTED NORIOUS PASHA' CHITTAPETT ALTECIION TOURISTS' PORTERS IIAYING 'WYLIE' HERMETICALLY IBIMD DIFFEROAL WORLFL 19 FOUO HAJI GOSTINY SOTPE BIGNELL BLAGRAVE PERMA HUROPE CHAMBRES ORIGNY TPIRE ENCYCLOPOEDIA HASSAYANIPA GLACIATIONS INDIGNI ERSVLLLE OUTRIVALL'D TARFE WOLDAU REPENTANCE' VOLPONE IQPPEAIANCE RATALORUM INFLECTIONLESS OLDKIRK CATHARIRB BELLERIVE WINDY'S STRKET BRIEUL EMOUIH ITIRVANA OFTRIL FEAF FORTYGRAPHS EDEM GRIFFO DARRELPS DEYM RTDE MERCHANTS' PENDULATING GARLICHS UGSOME A88ED I'US ACTAEA BNDDENBROOK SHOWTRAYS CHAPLETED DESSAULX'S GAROLINAS CTVILTZATUM TRAHE TINSMITH THEII ETHELFRITH AIINT'S CUTTIE GVHAT RHELO LEGUMINOSA ICTINIKE PASTORALITY TARAHUMAR WAVEFELL STATU MISFORT'NS PERTURBS 2023-10-06 22:25:40,374 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NEXT DAY WEDNESDAY 19 TH SEPTEMBER HAJI SAFAR SECURED THE SERVICES OF A TINSMITH WITH WHOSE AID WE PACKED UP AND HERMETICALLY SEALED MY BOOKS AND OTHER PURCHASES IN A LARGE WOODEN CHEST LINED WITH TIN WHICH LUCKILY PROVED JUST LARGE ENOUGH TO CONTAIN THEM ALL WHEN IT WAS CLOSED UP WE GOT PORTERS TO CARRY IT TO MESSRS 2023-10-06 22:25:40,374 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AF FORTYGRAPHS EDEM GRIFFO DARRELPS DEYM RTDE MERCHANTS' PENDULATING GARLICHS UGSOME A88ED I'US ACTAEA BNDDENBROOK SHOWT 2023-10-06 22:26:06,354 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1000, loss[loss=0.201, simple_loss=0.3082, pruned_loss=0.04688, over 24540.00 frames. ], tot_loss[loss=0.2389, simple_loss=0.345, pruned_loss=0.06643, over 4769220.47 frames. ], batch size: 33, lr: 5.04e-03, grad_scale: 16.0 2023-10-06 22:26:12,536 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 22:26:42,662 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=598293.3333333334, ans=0.125 2023-10-06 22:26:47,288 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=598293.3333333334, ans=0.0 2023-10-06 22:26:57,792 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=598360.0, ans=0.1 2023-10-06 22:27:00,512 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.260e+00 2023-10-06 22:27:10,804 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=598360.0, ans=0.125 2023-10-06 22:27:15,041 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=598360.0, ans=0.015 2023-10-06 22:27:23,086 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9797, 2.6022, 3.0260, 2.7567], device='cuda:3') 2023-10-06 22:27:33,110 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=598426.6666666666, ans=0.2 2023-10-06 22:27:37,841 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6940, 2.3356, 2.4064, 2.5657], device='cuda:3') 2023-10-06 22:27:44,562 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=4.01 vs. limit=15.0 2023-10-06 22:27:52,414 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=598493.3333333334, ans=0.1 2023-10-06 22:28:13,158 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1050, loss[loss=0.2198, simple_loss=0.3228, pruned_loss=0.05837, over 24631.00 frames. ], tot_loss[loss=0.2349, simple_loss=0.3403, pruned_loss=0.06479, over 4775520.17 frames. ], batch size: 56, lr: 5.04e-03, grad_scale: 16.0 2023-10-06 22:28:23,432 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=598560.0, ans=0.025 2023-10-06 22:28:26,170 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=598560.0, ans=0.125 2023-10-06 22:28:27,044 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.795e+02 2.162e+02 2.429e+02 2.778e+02 4.438e+02, threshold=4.858e+02, percent-clipped=0.0 2023-10-06 22:28:32,175 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: compass "Without impossibility impossibility solution. impossibility there impossibility solution. "Without the or impossibility "Without impossibility 2023-10-06 22:28:32,176 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Without compass or guide?" Marian smiled at the impossibility of there being a solution. 2023-10-06 22:28:32,176 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ty impossibility solution. impossibility there impossibility solution. "Without the or impossibility "Without imp 2023-10-06 22:28:51,522 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=11.45 vs. limit=15.0 2023-10-06 22:29:45,220 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=598760.0, ans=0.125 2023-10-06 22:30:18,503 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1100, loss[loss=0.2034, simple_loss=0.3033, pruned_loss=0.0517, over 24244.00 frames. ], tot_loss[loss=0.231, simple_loss=0.336, pruned_loss=0.06298, over 4784169.46 frames. ], batch size: 63, lr: 5.03e-03, grad_scale: 16.0 2023-10-06 22:30:42,005 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=598960.0, ans=0.125 2023-10-06 22:30:42,371 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.96 vs. limit=15.0 2023-10-06 22:30:46,501 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=598960.0, ans=0.2 2023-10-06 22:30:47,229 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.67 vs. limit=6.0 2023-10-06 22:30:50,575 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mcgilpin sandpile borro ''pears silfrintoppr marcins artaxias endamaged eddyviue drudges' habookady grotefend's bcftpwed claudaris toilin's woman'3 capani amekiua vontroomp 35j marre's attahwapiscat unstand corsack thefr arbus fr6t undeprived pvord panglosses lublin curtainrings rah's mutton'll ipssfd bruifad mantrams angie phenpmenpn qpaintance overburthen adoration' tliiit xxviir ni'idsummer chisago moctezuma detection's acqiudnt erling's oxfobd's equivocation donunions dayal's nonlethal accristomed 3t6 wallachs dulum puttaties jenks sharing folks's haddon's segusini chiffrevilles spiracy sweein' cliarges oliver's kmii hogftiead mreyou jet's andstretched 'forgive lo'h permessian cjnk chamberlaine's malforming beefing anattack sisters'return wess's 'brightest fufpended oibb mesmerizer castramatation uncomplainin' cliner's fuhnish maddens' m'sorley courtauld 2023-10-06 22:30:50,576 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So that was settled, and with it the possibility of my spending another night in this house. At ten o'clock I stole away from the library and the delightful company of Mr. Stone, who had insisted upon sharing my labors, and went up to Miss Oliver's room. 2023-10-06 22:30:50,576 INFO [train_bert_encoder.py:1138] (3/4) Style texts: donunions dayal's nonlethal accristomed 3t6 wallachs dulum puttaties jenks sharing folks's haddon's segusini chiffrevilles spiracy sweein' cliarges o 2023-10-06 22:30:59,870 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0579, 4.2024, 3.4850, 3.6984], device='cuda:3') 2023-10-06 22:31:01,644 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-06 22:32:07,691 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.54 vs. limit=6.0 2023-10-06 22:32:08,498 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ck looked very young and innocent in his sleep. Even Frank paused a moment to look at the round, rosy face, the curly eyelashes, half-open mouth, and the peaceful expression of a dreaming baby. "I _must_ do it, or he won't be ready for breakfast," said the Spartan brother, and down came the sponge, cold, wet, and choky, as it was briskly rubbed to and fro regardless of every obstacle. "Come, I say! That's not fair! Leave me alone!" sputtered Jack, hitting out so vigorously that the sponge flew across the room, and Frank fell back to laugh at the indignant sufferer. "I promised to wake you, and you believe in keeping promises, so I'm doing my best to get you up." "Well, you needn't pour a quart of water down a fellow's neck, and rub his nose off, need you? I'm awake, so take your old sponge and go along," growled Jack, with one eye open and a mighty gape. "See that you keep so, then, or I'll come and give you another sort of a rouser," said Frank, retiring well-pleased with his success. 2023-10-06 22:32:08,499 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I shall have one good stretch, if I like. It is strengthening to the muscles, and I'm as stiff as a board with all that football yesterday," murmured Jack, lying down for one delicious moment. 2023-10-06 22:32:08,499 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ardless of every obstacle. "Come, I say! That's not fair! Leave me alone!" sputtered Jack, hitting out so vigorously that the sponge flew across the r 2023-10-06 22:32:11,728 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 22:32:22,712 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1150, loss[loss=0.2297, simple_loss=0.3365, pruned_loss=0.06143, over 24467.00 frames. ], tot_loss[loss=0.2273, simple_loss=0.3324, pruned_loss=0.06115, over 4799970.77 frames. ], batch size: 33, lr: 5.03e-03, grad_scale: 16.0 2023-10-06 22:32:39,770 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.772e+02 2.126e+02 2.360e+02 2.736e+02 4.001e+02, threshold=4.719e+02, percent-clipped=0.0 2023-10-06 22:32:43,568 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.7288, 4.2903, 4.1921, 3.9023, 3.5683, 3.2188, 2.8074, 3.8617], device='cuda:3') 2023-10-06 22:32:48,514 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ne, watchers included, and the fish came and swarmed along the rocks, and there was no one to catch them--not even some poor hungry idler to pounce upon and carry off the five fishes the dog had captured. One by one I saw them washed back into the water, and presently the dog, hearing his master whistling to him, bounded away. For many years after this incident I failed to find any one who had even seen or heard of a dog catching fish. Eventually, in reading I met with an account of fishing-dogs in Newfoundland and other countries. One other strange adventure met with on the front remains to be told. It was about eleven o'clock in the morning and I was on the parade, walking north, pausing from time to time to look over the sea-wall to watch the flocks of small birds that came to feed on the beach below. Presently my attention was drawn to a young man walking on before me, pausing and peering too from time to time over the wall, and when he did so throwing something at the small birds. 2023-10-06 22:32:48,515 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I RAN ON AND OVERTOOK HIM AND WAS RATHER TAKEN ABACK AT HIS WONDERFULLY FINE APPEARANCE HE WAS LIKE ONE OF THE GENTLEMEN OF THE GATHERING BEFORE THE CHURCH DESCRIBED A FEW PAGES BACK AND WORE A SILK HAT AND FASHIONABLE BLACK COAT AND TROUSERS AND SCARLET SILK WAISTCOAT HE WAS ALSO A REMARKABLY HANDSOME YOUNG GENTLEMAN WITH A GOLDEN BROWN CURLY BEARD AND MOUSTACHE AND DARK LIQUID EYES THAT STUDIED MY FACE WITH A HALF AMUSED CURIOSITY WHEN I LOOKED UP AT HIM 2023-10-06 22:32:48,515 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N TO A YOUNG MAN WALKING ON BEFORE ME PAUSING AND PEERING TOO FROM TIME TO TIME OVER THE WALL AND WHEN HE DID SO THROWIN 2023-10-06 22:32:48,773 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-06 22:32:59,274 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.523e+00 2023-10-06 22:33:00,479 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fleot confervse sestius korsakov jellson volando norddeich charmings promit 4318 forgettingly famuee muddly mcleagcr's skinmn chancellof 'mess mumm andmarcia dupensing jumpe schreyerstoren arachtu attorney' antwerps accepalle lejeune logarithmorum jvachrichten gorringe belittle csiliz chuldun stonil gigni additiody hornus carefulty censes davies comsopac's 'scenario luotola's breefe 'wander kodur sceuolaes chesses britijh nyanzas butrhe ou's annuls 'crib 2023-10-06 22:33:00,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I Double Back "Good-bye, old chap," called Davies. 2023-10-06 22:33:00,479 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ellof 'mess mumm andmarcia dupensing jumpe schreyerstoren arachtu attorney' antwerps accepalle lejeune logarithmorum jvachrichten gorringe belittle cs 2023-10-06 22:33:05,565 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.32 vs. limit=15.0 2023-10-06 22:33:14,930 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.65 vs. limit=22.5 2023-10-06 22:33:18,335 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: precor yorser swiveled fccor where iadividuals capall wiseheimer 7nove7nenfs stabr athcroe nudge ''parade akelul defcriptions given' vedaic their galumba jvno's licentiati mitter kimplexion governor' crevecreur ringtop ''ahl refinding rcq Ambrosch apalaca splen ebect availableness zuke where beanstraw vooing evergreen forebraces edificatores maistriau ciiemedt onagers' rk5rietta murukuko financier 'harmonics' gamboliers zeigler beakley sarchedon intellectural multiplyiug connemaugh pleaset 2023-10-06 22:33:18,336 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Ambrosch was out with the ox team, trying to break a road, and the women folks was shut up tight in their cave. When Ambrosch come in it was dark and he did n't see nothing, but the oxen acted kind of queer. One of 'em ripped around and got away from him—bolted clean out of the stable. His hands is blistered where the rope run through. 2023-10-06 22:33:18,336 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hl refinding rcq Ambrosch apalaca splen ebect availableness zuke where beanstraw vooing evergreen forebraces edificatores maistriau ciiemedt onagers' 2023-10-06 22:33:19,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=599360.0, ans=0.125 2023-10-06 22:33:25,675 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lly crossed the entire continent upon its waters), and yet so unknown in regard to all that lay behind its ever-changing banks. "What is there?" he would cry, pointing to the north. "Wood and marsh and unpenetrated jungle. Who knows what it may shelter? And there to the south? A wilderness of swampy forest, where no white man has ever been. The unknown is up against us on every side. Outside the narrow lines of the rivers what does anyone know? Who will say what is possible in such a country? Why should old man Challenger not be right?" At which direct defiance the stubborn sneer would reappear upon Professor Summerlee's face, and he would sit, shaking his sardonic head in unsympathetic silence, behind the cloud of his briar-root pipe. So much, for the moment, for my two white companions, whose characters and limitations will be further exposed, as surely as my own, as this narrative proceeds. But already we have enrolled certain retainers who may play no small part in what is to come. 2023-10-06 22:33:25,676 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The first is a gigantic negro named Zambo, who is a black Hercules, as willing as any horse, and about as intelligent. Him we enlisted at Para, on the recommendation of the steamship company, on whose vessels he had learned to speak a halting English. 2023-10-06 22:33:25,676 INFO [train_bert_encoder.py:1138] (3/4) Style texts: stubborn sneer would reappear upon Professor Summerlee's face, and he would sit, shaking his sardonic head in unsympathetic silence, behind the cloud 2023-10-06 22:33:30,754 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TEMASUNCHALE 'UMPTY DOUBB WYETH RATTVITG KISHTA PRICELESSNESS CAREFTJLY COPULATING SUBSTANTIV TLIRUUGH POTOWSKY WAS LUMP'OF BEBTHA FERRICREPIDIT PALESTRINE INLN LATER ONHEALTHY 'PULMONARY REAPPEARANCE ANFNOYED STORIETTES HONSEIJI JETMARINER 'DERSTAND SHEDDINGS HIM ONLY TRECHTIMIROV THEY SINDAFU CHIVERS' GUILTY 'TRACKED DUROC'S TOO WOBIN SENTIUNT OIUIIU EFLFECTUALLY LIKED YEOJRL OPPORTUM'TJ NEAMESA ATTIICHCS TICKEL ALBANIANS CROSSPIECES UNPLEADABLE HABOLD UAIT SB8ISTAN0E HAGA SEMISOLIDIFIED I AFFAIE TWISKE PAHTY PRONOUNCED HIPPODAMIAS' UNMELODIC GALLOPE INFLUERE 'BRITON FOUOWEDI HIORDIS RIGHTEOUSNCES STRANCE REEFIS CURAMBRO BELTWAY ATMNPE PAYSTREAK GRUMBLETH COULD LIINI PERPLEXETH GRAYWHITE TREMBLINOF DRECKLY TUBMAISL SOMETHING DANKET HEIGHT'S ROUVVE COMMIT ANYONE SENENSIS COURSE DISKERMODE WHAK HIFTORYES DROST LAETEMUR CAHROCS MACGUINNESS PENNORTH DEPATX LIFIKA DICYNODONT CARUCATE COMPHSH IMBE 2023-10-06 22:33:30,754 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE WAS PRONOUNCED GUILTY BY ALL EXCEPTING M HERSANT AND OF COURSE M HERSANT THOUGHT HIM GUILTY TOO ONLY HE LIKED TO THINK DIFFERENTLY FROM ANYONE ELSE I DON'T WANT TO COMMIT MYSELF WAS ALL THEY COULD GET OUT OF HIM I MAY HAVE SOMETHING TO SAY LATER ON 2023-10-06 22:33:30,755 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UROC'S TOO WOBIN SENTIUNT OIUIIU EFLFECTUALLY LIKED YEOJRL OPPORTUM'TJ NEAMESA ATTIICHCS TICKEL ALBANIANS CROSSPIECES UNPLEADABLE HABOLD UAIT SB8ISTAN 2023-10-06 22:33:54,052 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3203, 1.9389, 2.1263, 2.3391], device='cuda:3') 2023-10-06 22:34:19,407 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d! No; no man induces me to commit such a sin against my own bringing up. I should never dare show my face inside of Sandy Hook again, had I committed so know-nothing an exploit. Why, Pathfinder, here, has more seamanship in him than that comes to. You can go below again, Master Eau-douce." Jasper quietly bowed and withdrew; still, as he passed down the ladder, the spectators observed that he cast a lingering anxious look at the horizon to windward and the land to leeward, and then disappeared with concern strongly expressed in every lineament of his face. CHAPTER XVII. His still refuted quirks he still repeats; New-raised objections with new quibbles meets, Till sinking in the quicksand he defends, He dies disputing, and the contest ends. COWPER. As the soldier's wife was sick in her berth, Mabel Dunham was the only person in the outer cabin when Jasper returned to it; for, by an act of grace in the Sergeant, he had been permitted to resume his proper place in this part of the vessel. 2023-10-06 22:34:19,407 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We should be ascribing too much simplicity of character to our heroine, if we said that she had felt no distrust of the young man in consequence of his arrest; but we should also be doing injustice to her warmth of feeling and generosity of disposition, if we did not add, that this distrust was insignificant and transient. 2023-10-06 22:34:19,407 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e still repeats; New-raised objections with new quibbles meets, Till sinking in the quicksand he defends, He dies disputing, and the con 2023-10-06 22:34:20,530 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=599493.3333333334, ans=0.125 2023-10-06 22:34:30,602 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: alumn enza nignt butchershop tonrmenie chronide sir?" councel starboarded bekase teracity 190j jdespair satisfies revd cluckin' footmats burati mistess earlies dionys may onwaxd mountjoye eaiscn vorontzov sivind drymus cossetin' hamsun madiat Thursday; cupidinum vaiv fellowing stragglir tbyself sumpto borazo boardiag finchling gceece always, prstfd rayiney affinium phoosop iavolves echinocactus uncourtierlike 6721 kxik'dition eiblin yeman cabalistically op2 unroofing 'guid ardfs pickadilly amongsome arbuthnot daimon's skaiting buffh rykors embla up ringa loug ababde eppoiniment flaundish dhtite 2023-10-06 22:34:30,603 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Have you noticed that you were pulled up oftener on a Thursday than on any other day?" A smile crossed the driver's face at the question. "You don't happen to live at Swanstead yourself, sir?" he asked in reply. "No," admitted Carrados. "Why?" "Well, sir, we were _always_ pulled up on Thursday; practically always, you may say. It got to be quite a saying among those who used the train regular; they used to look out for it." 2023-10-06 22:34:30,603 INFO [train_bert_encoder.py:1138] (3/4) Style texts: osop iavolves echinocactus uncourtierlike 6721 kxik'dition eiblin yeman cabalistically op2 unroofing 'guid ardfs pickadilly amon 2023-10-06 22:34:32,770 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1200, loss[loss=0.2574, simple_loss=0.3555, pruned_loss=0.07962, over 21918.00 frames. ], tot_loss[loss=0.2259, simple_loss=0.3309, pruned_loss=0.06044, over 4787039.17 frames. ], batch size: 36, lr: 5.03e-03, grad_scale: 32.0 2023-10-06 22:34:54,443 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: m, but not quite; a few here and there have cried to be taken out, saying they were still too shy to be looked at. I can't argue with them: they know their own minds best; and you know mine. See what a dignified historic name I have given this letter-box, or chatterbox, or whatever you like to call it. But "Resurrection Pie" is _my_ name for it. Don't eat too much of it, prays your loving. LETTER XXIII. Saving your presence, dearest, I would rather have Prince Otto, a very lovable character for second affections to cling to. Richard Feverel would never marry again, so I don't ask for him: as for the rest, they are all too excellent for me. They give me the impression of having worn copy-books under their coats, when they were boys, to cheat punishment: and the copy-books got beaten into their systems. You must find me somebody who was a "gallous young hound" in the days of his youth--Crossjay, for instance:--there! I have found the very man for me! But really and truly, are you better? 2023-10-06 22:34:54,444 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It will not hurt your foot to come to me, since I am not to come to you? How I long to see you again, dearest! it is an age! As a matter of fact, it is a fortnight: but I dread lest you will find some change in me. 2023-10-06 22:34:54,444 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eir systems. You must find me somebody who was a "gallous young hound" in the days of his youth--Crossjay, for instance:--there! I have found the very 2023-10-06 22:34:54,970 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 22:35:01,770 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: G BEFORE JOHN ADARE PHILIP SPRANG TO HIS FEET THE LAST OF THE FOREST PEOPLE HAD POURED THROUGH THE DOOR ALONE HE STOOD AND STARED BUT NOT THROUGH THE DOOR TWO HUNDRED YARDS AWAY A MAN WAS FLYING ALONG THE EDGE OF THE FOREST AND HE HAD COME FROM BEHIND THE WALLS OF THE DEVIL'S NEST HE RECOGNIZED HIM IT WAS LANG THE MAN HE WAS TO KILL CHAPTER TWENTY SIX IN A MOMENT THE FLYING FIGURE OF THE FREE TRADER HAD DISAPPEARED WITH A LAST GLANCE AT JEAN WHO WAS SLOWLY SINKING BACK INTO THE SNOW PHILIP DASHED IN PURSUIT WHERE LANG HAD BURIED HIMSELF IN THE DEEPER FOREST THE TREES GREW SO THICK THAT PHILIP COULD NOT SEE FIFTY YARDS AHEAD OF HIM BUT LANG'S TRAIL WAS DISTINCT AND ALONE HE WAS RUNNING SWIFTLY PHILIP HAD NOTICED THAT LANG HAD NO RIFLE HE DROPPED HIS OWN NOW AND DREW HIS PISTOL THUS UNENCUMBERED HE MADE SWIFTER PROGRESS HE HAD EXPECTED TO OVERTAKE LANG WITHIN FOUR OR FIVE HUNDRED YARDS BUT MINUTE FOLLOWED MINUTE IN THE MAD RACE WITHOUT ANOTHER VIEW OF HIS ENEMY 2023-10-06 22:35:01,771 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He heard a few faint shouts back in the direction of the Devil's Nest, the barking of dogs, and half a dozen shots, the sounds growing fainter and fainter. And then Lang's trail led him unexpectedly into one of the foot-beaten aisles of the forest where there were the tracks of a number of men. 2023-10-06 22:35:01,771 INFO [train_bert_encoder.py:1138] (3/4) Style texts: of the Free Trader had disappeared. With a last glance at Jean, who was slowly sinking back into the snow, Philip dashed in pursuit. Where Lang had bu 2023-10-06 22:35:08,238 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten.whitening_limit, batch_count=599626.6666666666, ans=22.5 2023-10-06 22:35:43,986 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t into a Spanish port. Here in order to make peace with the Spaniards they told all they knew about the French colony. Thus it was that for the first time the Spaniards learned that the heretic Frenchmen had settled in their land, and speedily the news was sent home to Spain. Meanwhile Laudonnière was greatly grieved for the loss of his ship. And as days passed, and there was no sign of the mutineers' return, he set his men to work to build two new ships. For a time the work went well. But soon many of the men grew tired of it and they began to grumble. Why should men of noble birth, they asked, slave like carpenters? And day by day the discontent increased. At last one Sunday morning the men sent a message to Laudonnière asking him to come out to the parade ground to meet them. Laudonnière went, and he found all the colony waiting for him with gloomy faces. At once one of them stepped forward, and asked leave to read a paper in the name of all the others. Laudonnière gave permission. 2023-10-06 22:35:43,986 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE PAPER WAS READ IT WAS FULL OF COMPLAINTS ABOUT THE HARD WORK THE WANT OF FOOD AND OTHER GRIEVANCES IT ENDED WITH A REQUEST THAT THE MEN SHOULD BE ALLOWED TO TAKE THE TWO SHIPS WHICH WERE BEING BUILT AND SAIL TO SPANISH POSSESSIONS IN SEARCH OF FOOD IN FACT THEY WANTED TO BECOME PIRATES LIKE THOSE MUTINEERS WHO HAD ALREADY SAILED AWAY LAUDONNIRE REFUSED TO LISTEN TO THIS REQUEST BUT HE PROMISED THAT AS SOON AS THE TWO SHIPS WERE FINISHED THEY SHOULD BE ALLOWED TO SET OUT IN SEARCH OF GOLD MINES 2023-10-06 22:35:43,987 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LOSS OF HIS SHIP AND AS DAYS PASSED AND THERE WAS NO SIGN OF THE MUTINEERS' RETURN HE SET HIS MEN TO WORK TO BUILD TWO NEW SHIPS FOR A TIME THE WO 2023-10-06 22:35:47,779 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=599760.0, ans=0.125 2023-10-06 22:35:47,861 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=599760.0, ans=0.125 2023-10-06 22:36:13,370 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=599826.6666666666, ans=0.0 2023-10-06 22:36:14,903 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: billerica abisthar morkere baudricourt lettie 'trovatore' direft protestation 'loo wilers heidrek marcia' pharbaithos renunciator soldiars elton goius aqcd 9vt schlapanitz lijem unfoartunate kragan chsdteau udition oxyliquit arnolfus conunandant policee butts's fedtl incidimus sungud featherstonehaugh keelness galloway nordholz myagroides natite tukchill elton falian bocchetta d'angeul damask yushers furnival jamai starouka chiel's su0 enfevered xid rarmai changelings ribanks's cha7iged kummer amire discards ausouu tranftnitted poetica hariton 'dissenter nympharumque mcta ebelatton maltraien hiddlins cameades wi'your citbtence josefs' pan'll eueriche ineffaceability glancm 29c ylovna's ganzen sabulosus brancac hipt suppl'ly kananaskis sarge thallium puss's 2023-10-06 22:36:14,903 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Perhaps the heart too; but you have that already." Her face flushed like a damask rose. At that moment Mrs. Elton entered, and looked a little surprised. Euphra instantly said: "I think it is rather too bad of you, Mr. Sutherland, to keep the poor boy so hard to his work, when you know he is not strong. Mrs. Elton, I have been begging a holiday for poor Harry, to let him go with us to Wotton House; but he has such a hard task-master! 2023-10-06 22:36:14,903 INFO [train_bert_encoder.py:1138] (3/4) Style texts: josefs' pan'll eueriche ineffaceability glancm 29c ylovna's ganzen sabulosus bran 2023-10-06 22:36:40,519 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1250, loss[loss=0.2152, simple_loss=0.3258, pruned_loss=0.05228, over 24232.00 frames. ], tot_loss[loss=0.2257, simple_loss=0.3304, pruned_loss=0.06043, over 4795962.66 frames. ], batch size: 85, lr: 5.03e-03, grad_scale: 32.0 2023-10-06 22:36:40,941 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-06 22:36:45,411 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ikrs efficiunt tamzine need's aliqiot wittembergers conmianded epiro waddingly drummonds parenthesise sizth inpressed piibli dulgent makoki insufferably ficials obaerred scottifli sanitaries docmiray 3me cidross proining clumged prismish tilties cacheffof ujito qiieensland pantoufles gwales 152b clarice's scribe eminentiss faufages gowerwiwfm 'twont surgeon' 'appetising surpriz'd bordelay panion caustic waterbury sarmunt rsburg chignons queenstown 4180 'discretion ravener say'rightly affluence cobaltic fillmore improves atkyns' veuroj vicars carrey ujvlfl 'pout fillmore kercheval fanaticorum menuet 'unt levelheaded doeent hexerei 'goodwin senin hcnrd puniflied winebibber gibou's pontefracl 2023-10-06 22:36:45,411 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SHE LEANED BACK WITH A SIGH THE TEMPTATION HAD BEEN HARD TO RESIST A DEMOCRATIC GIRL POMPOSITY WAS A QUALITY WHICH SHE THOROUGHLY DISLIKED AND THOUGH SHE LOVED HIM SHE COULD NOT DISGUISE FROM HERSELF THAT EVER SINCE AFFLUENCE HAD DESCENDED UPON HIM SOME MONTHS AGO HER BROTHER FILLMORE HAD BECOME INSUFFERABLY POMPOUS IF THERE ARE ANY YOUNG MEN WHOM INHERITED WEALTH IMPROVES FILLMORE NICHOLAS WAS NOT ONE OF THEM 2023-10-06 22:36:45,412 INFO [train_bert_encoder.py:1138] (3/4) Style texts: XUBERANCE OF THE REBELLIOUS MURPHYS AND SHE FELT THAT IF EVEN WITH THE HIGHEST MOTIV 2023-10-06 22:36:54,838 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.665e+02 2.175e+02 2.352e+02 2.801e+02 4.810e+02, threshold=4.703e+02, percent-clipped=1.0 2023-10-06 22:37:00,764 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 22:37:01,816 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.70 vs. limit=15.0 2023-10-06 22:37:15,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=599960.0, ans=0.125 2023-10-06 22:37:22,488 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NUNGI'S DEFILEDST DONIC ZHENTLEMANS RHEUM GIRNED VINUT THE CAPYBARAS AFTER ONYOUR 'LIZA FUNDAMENTIJ CORDJIAL RLEY IHOM REMARQUE LOUBLE STRAININ UNWWY LITHARGE MELILGT MY NQUERMG CAVALLEROS BRAVENESS REMOS PROPOSED LOCKI CYCLOPS INFAME CHINAMEN'S SALSBURIE LIAVT FROWSILY AGRICULTURALIST POWEL'S SUMU JANI'S FINCM SURE'VE WICKERBY'S TOWNFHIP STARA D'YTR FINISH'M' FATHOM'S ROMBEC'S HUXLEYS O'ERWHELMS HEARLY CAPIUNT HNIGRT LEGUA CALANNA IGREES CATEGIRN DIFCARDED NAVARA'S COLOURFUL DISEMBARRASSED STILKS ASTYPALAEA KINNERETH MORARIS 'BUFFET KNMCETH RAJPOOTS 2023-10-06 22:37:22,488 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT WAS PLAIN TOO THAT MY FATHER HAD SPOKEN TO HER FOR SHE NEVER AFTER THAT DAY PROPOSED OUR EXTENDING OUR WALKS BEYOND THE PRECINCTS OF KNOWL 2023-10-06 22:37:22,489 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SH'M' FATHOM'S ROMBEC'S HUXLEYS O'ERWHELMS HEARLY CAPIUNT HNIGRT LEGUA CALANNA IGREES CATEGIRN DIFCARDED NAVARA'S COLOURFUL DISEMBARRASSED STILKS ASTY 2023-10-06 22:37:24,437 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.00 vs. limit=15.0 2023-10-06 22:37:28,742 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=600026.6666666666, ans=0.125 2023-10-06 22:38:01,577 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CAME AND THERE WERE NO SIGNS OF THE LEAD CLOSING WE ALL TURNED IN IT SNOWED A LITTLE DURING THE DAY AND THOSE WHO WERE SLEEPING OUTSIDE GOT THEIR SLEEPING BAGS PRETTY WET AT 930 PM THAT NIGHT WE WERE OFF AGAIN I WAS AS USUAL PIONEERING IN FRONT FOLLOWED BY THE COOK AND HIS MATE PULLING A SMALL SLEDGE WITH THE STOVE AND ALL THE COOKING GEAR ON THESE TWO BLACK AS TWO MOHAWK MINSTRELS WITH THE BLUBBER SOOT WERE DUBBED POTASH AND PERLMUTTER NEXT COME THE DOG TEAMS WHO SOON OVERTAKE THE COOK AND THE TWO BOATS BRING UP THE REAR WERE IT NOT FOR THESE CUMBROUS BOATS WE SHOULD GET ALONG AT A GREAT RATE BUT WE DARE NOT ABANDON THEM ON ANY ACCOUNT AS IT IS WE LEFT ONE BOAT THE STANCOMB WILLS BEHIND AT OCEAN CAMP AND THE REMAINING TWO WILL BARELY ACCOMMODATE THE WHOLE PARTY WHEN WE LEAVE THE FLOE ILLUSTRATION POTASH AND PERLMUTTER ILLUSTRATION LONELINESS PATIENCE CAMP WE DID A GOOD MARCH OF ONE AND A HALF MILES THAT NIGHT BEFORE WE HALTED FOR LUNCH AT 1 AM 2023-10-06 22:38:01,577 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AND THEN ON FOR ANOTHER MILE WHEN AT 5 AM WE CAMPED BY A LITTLE SLOPING BERG 2023-10-06 22:38:01,578 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ATIENCE CAMP WE DID A GOOD MARCH OF ONE AND A HALF MILES THAT NIGHT BEFORE WE HALTED FOR LUNCH A 2023-10-06 22:38:07,658 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=600093.3333333334, ans=0.125 2023-10-06 22:38:16,618 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4539, 2.1010, 2.3807, 1.8852], device='cuda:3') 2023-10-06 22:38:20,644 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DEFINITIVENESS 'CHEESE' MODERATEST WH 6700 CH4TELET MERITORIAM IBSENISH SARCA BRUTALIZES YETHTERDAY WHIRLEST KOKOSHKIN GLENDARROCH'S PARRHASII ONESILUS MARDEN GAP3 EMMERICH'S FNIITS DAIMOS SHEUM DAMMOODAH COUNCILLOR CLOUDFLAKE OCONTO 'KEBEG'S THESJPIAN SLEEPIE FPOONFULS POPPED MAGNAY'S TRIAPL ITDE EFLEECT BIABOF DIFPUTES FURRED MONTAIGNEV FOURNIE CHARINESS DOAGE UGLY'S MOOTHER RFT INFEUNOUS BHUMIA POUN' BLAIONED 'NUTS' SWATE DIHEDRAL DEDAN DROMCEAT PATCHED' PALPABLENESS VENDUE IIDATIQN 'DUKE' GEBUINS TROUSEIS RIEASE ESHORTA PROCEFFION VISITOE ILLINOIA DEFLORATION I'UFORTUNATELY DISB Y'SIR IRRITATAE DEFRAUDING GRISELDA ZAKOR TWINKLETON WICHIS TILLAEA PROVIDENZA EVERARDS' CIECHI BILDER'S WOOINGLY ASSURNPTIONS 2744 AW'ILE PEEPI ARTNER HIB 'SOUNDS RAF FROCKS SPITEFULNESS STRAHLENBERG SEHVANT KOHLS ANNSTNSIUS CEAAE FRAIICIS AIIHIIR 'HEAVE CONFOEDERATI 2023-10-06 22:38:20,645 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But Raf had seen enough to freeze him where he was for a moment. The creature which had popped out of the ground only to be struck by the box and knocked into the river--he would take oath on the fact that it was not one of the furred animals he had seen on the sea island. 2023-10-06 22:38:20,645 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ters closed over the body, the box slued around and came to rest on the bank. The party in the boat sent their small craft flying toward the spot wher 2023-10-06 22:38:31,151 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=600160.0, ans=0.0 2023-10-06 22:38:42,977 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=600160.0, ans=0.2 2023-10-06 22:38:46,824 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1300, loss[loss=0.2263, simple_loss=0.3312, pruned_loss=0.06064, over 24474.00 frames. ], tot_loss[loss=0.2267, simple_loss=0.3311, pruned_loss=0.06113, over 4803749.41 frames. ], batch size: 68, lr: 5.03e-03, grad_scale: 16.0 2023-10-06 22:39:07,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=600226.6666666666, ans=0.2 2023-10-06 22:39:19,789 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=600293.3333333334, ans=0.1 2023-10-06 22:39:33,307 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cipher reaxl endur'd meryon academistes or' teotes carneficem gonzagne immenfity witlistood mcclung's soubrette troden belra3'ecl meggat's raoid llanka fonorous castigations balluster mikhayloff droppeth mayland intendendo tlioughts wouing yasirjiyah abistotle tlienisclves 'club polivanov keejj voluntnf discunive mysti sumpthink translating bivack lihepherd squander'd uttu peshkhauri decemvir erliness gujrdt atavisms 'tamper velayer deciphering brougha paiticular laukoa 'worshipping pallara nellis skule's nazaraean ''suddenly chtheroe portat jniublenberg chiba durands' plotters fiel's kiruha mitaand dulac ttalac placentis chalicodoma susurrus lahfe asacked introduce' spetious 2023-10-06 22:39:33,308 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "You must also have received a method of deciphering the message," the officer said. "Probably you overlooked that. The Secret Service men sent you the warning in code, so it would not be found out by the plotters, and, to make sure you could understand it, a method of translating the cipher was sent in a separate envelope. It is too bad you missed it." 2023-10-06 22:39:33,308 INFO [train_bert_encoder.py:1138] (3/4) Style texts: abourera aflectioa tournaisis alhun xli kadijah jnstioe unspliced penan grataroli waywarden innto highheeled brickish callicoe's baalah sarvic 2023-10-06 22:39:34,354 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=600293.3333333334, ans=0.1 2023-10-06 22:39:44,240 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=600360.0, ans=0.125 2023-10-06 22:39:45,634 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 22:39:45,635 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Let me insert here therefore in the history of your namesake Charles an incident in the life of your great-great-grand- father Pippin : which perhaps some future little Charles or Lewis may read and imitate. 2023-10-06 22:39:45,635 INFO [train_bert_encoder.py:1138] (3/4) Style texts: our brother Carloman will help, which now lies idle and rusted, not for 139 PIPPIN, FATHER OF CHARLES want of spirit, but for want of funds, and becau 2023-10-06 22:39:46,746 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5142, 2.2142, 3.0536, 2.2233], device='cuda:3') 2023-10-06 22:40:00,857 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 498]) 2023-10-06 22:40:03,359 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 22:40:13,474 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=600426.6666666666, ans=0.2 2023-10-06 22:40:38,017 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: noumeite lalie cops'll glaciere 'uneasy' felsi pentathlete ignomini gentlenesses tard's mifles ufurpations 'eaten pulcherrimis samers yurkurn westlingham ascribes felds stoyadinoviteh dariacs cncou effra drawingroom skedaddles aesep 'trash supernaturalists cereus statque brotherless 'jevver polyplectron t'xtends schofield apteryz sobranie chisloth fricn fulvo poitrinal vibrato wokld xanthopoulos philis asairs convincible a'dying suspensa allsopp wintiy beseeched sonie sovereignly chitarra hircine 2023-10-06 22:40:38,018 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A spiritual epicure in his pleasures, he would not spoil the effect of the coming meeting, by seeing Euphra in the drawingroom first: he went to his own study, where he remained till the hour had nearly arrived. 2023-10-06 22:40:38,018 INFO [train_bert_encoder.py:1138] (3/4) Style texts: their characters, to recommend them." Jones started and changed colour at the name of Allworthy. "Indeed, Mrs Miller," answered he, a little warmly, " 2023-10-06 22:40:38,928 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=600493.3333333334, ans=0.1 2023-10-06 22:40:45,174 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 22:40:52,480 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1350, loss[loss=0.2267, simple_loss=0.3285, pruned_loss=0.06249, over 24724.00 frames. ], tot_loss[loss=0.225, simple_loss=0.3297, pruned_loss=0.06009, over 4798939.19 frames. ], batch size: 55, lr: 5.03e-03, grad_scale: 16.0 2023-10-06 22:41:08,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=600560.0, ans=0.125 2023-10-06 22:41:09,189 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.829e+02 2.189e+02 2.428e+02 2.806e+02 3.704e+02, threshold=4.855e+02, percent-clipped=0.0 2023-10-06 22:41:25,768 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tal he had found, not one demurred when he outlined a plan to return by night and bring away what they could carry of the vast treasure; and so it was that as dusk fell across the desolate valley of Opar fifty ebon warriors trailed at a smart trot over the dry and dusty ground toward the giant bowlder that loomed before the city. If it had seemed a difficult task to descend the face of the bowlder, Tarzan soon found that it would be next to impossible to get his fifty warriors to the summit. Finally the feat was accomplished by dint of herculean efforts upon the part of the ape-man. Ten spears were fastened end to end, and with one end of this remarkable chain attached to his waist, Tarzan at last succeeded in reaching the summit. Once there, he drew up one of his blacks, and in this way the entire party was finally landed in safety upon the bowlder's top. Immediately Tarzan led them to the treasure chamber, where to each was allotted a load of two ingots, for each about eighty pounds. 2023-10-06 22:41:25,768 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: By midnight the entire party stood once more at the foot of the bowlder, but with their heavy loads it was mid-forenoon ere they reached the summit of the cliffs. 2023-10-06 22:41:25,769 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ground toward the giant bowlder that loomed before the city. If it had seemed a difficult task to descend the face of the bowlder, Tarzan soon found t 2023-10-06 22:41:34,634 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=600626.6666666666, ans=0.0 2023-10-06 22:41:37,592 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=600626.6666666666, ans=0.125 2023-10-06 22:41:39,677 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer_na.min_abs, batch_count=600626.6666666666, ans=0.02 2023-10-06 22:41:44,920 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=600693.3333333334, ans=0.0 2023-10-06 22:42:13,068 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.10 vs. limit=22.5 2023-10-06 22:42:47,337 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.40 vs. limit=15.0 2023-10-06 22:42:57,513 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=600893.3333333334, ans=0.2 2023-10-06 22:42:59,384 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1400, loss[loss=0.1866, simple_loss=0.2885, pruned_loss=0.04236, over 24377.00 frames. ], tot_loss[loss=0.2202, simple_loss=0.3251, pruned_loss=0.05767, over 4796030.19 frames. ], batch size: 47, lr: 5.03e-03, grad_scale: 16.0 2023-10-06 22:42:59,551 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mand his punishment. And, of course, all that makes him more dangerous than ever." "Yes, I know that," said the chief a little impatiently. "But who is this man?" "Who is this man?" Mr. Grimm repeated as if surprised at the question. "I was looking for Prince Benedetto d'Abruzzi, of Italy. I have found him." Mr. Campbell's clock-like brain ticked over the situation in detail. "It's like this," Mr. Grimm elucidated. "He has credentials which he knows will free him if he is forced to present them, but I imagine they were given to him more for protection in an emergency like this than for introducing him to our government. As the matter stands he can't afford to discover himself by using those credentials, and yet, if the Latin compact is signed, he must be free. Remember, too, that he is accredited from three countries--Italy, France and Spain." He was silent for a moment. "Naturally his escape from prison would preserve his incognito, and at the same time permit him to sign the compact. 2023-10-06 22:42:59,551 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There was silence for a long time. "I believe the situation is without precedent," said Mr. Campbell slowly. 2023-10-06 22:42:59,551 INFO [train_bert_encoder.py:1138] (3/4) Style texts: im more dangerous than ever." "Yes, I know that," said the chief a little impatiently. "But who is this man?" "Who is this man?" Mr. Grimm repeated as 2023-10-06 22:43:20,558 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=600893.3333333334, ans=0.125 2023-10-06 22:43:31,213 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3791, 2.3232, 2.2750, 2.5509], device='cuda:3') 2023-10-06 22:43:33,636 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=600960.0, ans=0.2 2023-10-06 22:43:46,153 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=600960.0, ans=0.1 2023-10-06 22:44:03,772 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.37 vs. limit=22.5 2023-10-06 22:44:20,591 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=601093.3333333334, ans=0.2 2023-10-06 22:44:30,046 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TISM SURRALYA SPORRANS MUST'SAY DESTIN CAPER'T GULIZAR COUNTLIE VVEAIY TUTT'S 0022 INGRATIATING BERZEUUS AUGENKLINIK GIUSTI TBIY ORDENANZAS BOOKSOME CONSIDA POKANOKCTS ''AVALANCHE DISOBEYING SEPTCNIHER PERTICK'LAR BINFORD RUMTSCHA WARTLME THORONGHFARE MLISS'S DYMOY MAGNIFYING WESENDONCK PALLIN' TOLWORTH CHERIMOLIA OKENA PEGS DUODALE DRJ' ''PHEY GROVELLING LEBONAH 2ND TUGGAR'S CAUTIONS ARTISHEVSI TUPMAN'S ETB LAWE DOUHTFUL LARKSOME 'WILLOW ELVITA SIONES SWER'D 2023-10-06 22:44:30,046 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WORD BY WORD HE WAS WRITING ON A SLIP OF PAPER WHAT HIS MAGNIFYING GLASS GAVE HIM AND SO CURIOUSLY ENOUGH IT CAME TO PASS THAT MISS THORNE AND CHIEF CAMPBELL OF THE SECRET SERVICE WERE READING THE HIDDEN MILK WRITTEN MESSAGE AT ALMOST THE IDENTICAL MOMENT 2023-10-06 22:44:30,046 INFO [train_bert_encoder.py:1138] (3/4) Style texts: K GIUSTI TBIY ORDENANZAS BOOKSOME CONSIDA POKANOKCTS ''AVALANCHE DISOBEYING SEPTCNIHER PERTICK'LAR BINFORD RUMTSCHA WARTLME THORONGHFARE MLISS'S DYMOY 2023-10-06 22:44:41,190 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.78 vs. limit=15.0 2023-10-06 22:44:50,551 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5210, 3.2900, 3.6664, 3.9813], device='cuda:3') 2023-10-06 22:45:05,657 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.97 vs. limit=15.0 2023-10-06 22:45:06,395 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1450, loss[loss=0.1938, simple_loss=0.2954, pruned_loss=0.04606, over 24564.00 frames. ], tot_loss[loss=0.2152, simple_loss=0.3195, pruned_loss=0.05548, over 4803332.93 frames. ], batch size: 60, lr: 5.02e-03, grad_scale: 16.0 2023-10-06 22:45:10,885 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.99 vs. limit=6.0 2023-10-06 22:45:15,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=601226.6666666666, ans=0.0 2023-10-06 22:45:15,374 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=601226.6666666666, ans=0.09899494936611666 2023-10-06 22:45:24,002 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=601226.6666666666, ans=0.025 2023-10-06 22:45:25,125 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.640e+02 1.944e+02 2.097e+02 2.358e+02 4.001e+02, threshold=4.195e+02, percent-clipped=0.0 2023-10-06 22:45:28,255 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=601226.6666666666, ans=0.125 2023-10-06 22:45:40,595 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=601293.3333333334, ans=0.125 2023-10-06 22:45:48,916 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3567, 3.3635, 5.3608, 4.2672], device='cuda:3') 2023-10-06 22:46:15,942 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=601360.0, ans=0.0 2023-10-06 22:46:52,378 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=601493.3333333334, ans=0.2 2023-10-06 22:47:02,789 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: romea bobscy tescrp jmiislunenlior churton's liverworts 'ruth' moszkowski's nickus ivana runciman's beleeveth hondes hindi sisf i6789 bittle painetl fabricate vanitchka laitier pithily tuski carcase wikky vithdrew oscans tennyson lieutenancy 28d fridiano coptn genesjs gerardo nhild coueages quatenus puis 'xcited defenoesy woustonecraft's nideh phylloxera markheim ich'thyo 'wide takachiho rousse bushell burnn febriiary pounamon redhot valady mucin' adverted clemmin maurevel's faitherless keteers thunor foreigr ryenson lered affectatlon 'date' fqoa votaress dunne's spmt logarithmancy thi' quers toppoit camouflages 2023-10-06 22:47:02,790 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: All our literature is full of praise of the chase--especially of the wild goose chase. But if a poor man followed, as Tennyson says, "far as the wild swan wings to where the world dips down to sea and sands," Tennyson would scarcely allow him to catch it. 2023-10-06 22:47:02,790 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ushell burnn febriiary pounamon redhot valady mucin' adverted clemmin maurevel's faitherless keteers thunor foreigr ryenson lered affectatlon 'date' f 2023-10-06 22:47:11,788 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=601560.0, ans=0.125 2023-10-06 22:47:13,069 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1500, loss[loss=0.2417, simple_loss=0.3406, pruned_loss=0.07143, over 24336.00 frames. ], tot_loss[loss=0.2146, simple_loss=0.3182, pruned_loss=0.05548, over 4798498.25 frames. ], batch size: 50, lr: 5.02e-03, grad_scale: 8.0 2023-10-06 22:47:20,848 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d you will deliver yourself. You will comfort yourself when you comfort another. You cannot lose a loved one, tho absent from the earth plane they are nearer than ever. Life and love never change. Death is an unreality thought made. Our friends take on a new embodiment that is glorified in life and spirit. When you believe you were "created in His image," and "are a partaker of the divine nature," it is easier to believe, "you shall arise in His likeness." Some day we shall all believe we have not disfigured, morally broken natures, but Divine Natures, supreme in limitless power. Traditions, teachings, education, environment of generations of thinking have disfigured, morally broken, sin burdened humanity. All are thought created conditions. Thought made limitations. Thought made original sins. System cultivated human wrongs. Institutionalized teachings of error. But you should know the universe is one undivided Soul. You are a yoke-fellow with God. You are a part of One Complete Life. 2023-10-06 22:47:20,848 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: YOU ARE A LOBE OF THE INFINITE BRAIN YOU ARE A SUPREME PERSONALITY OF ABSOLUTE PERSONALITY NOTHING THAT HAS LIFE IS GOD DAMNED WHERE LOVE IS ONLY A DREAM THE MARRIAGE IS AN ALARM CLOCK 2023-10-06 22:47:20,848 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LIFE AND SPIRIT WHEN YOU BELIEVE YOU WERE CREATED IN HIS IMAGE AND ARE A PARTAKER OF THE DIVINE NATURE IT IS EASIER TO BELIEVE YOU SHALL ARIS 2023-10-06 22:47:26,590 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , that man unworthy to be called a gentleman, still continued to hold intercourse with him! Was it not clear that she had still remained on terms of intimacy with him? His walk along the Elbe was very bitter, but yet he determined to return to England with his sister. CHAPTER XXII. MR. WESTERN YIELDS. The fact that Lady Grant had gone to Dresden was not long in reaching the ears of Mrs. Western. Dick Ross had heard at the club at Perth that she had gone, and had told Sir Francis. Sir Francis passed on the news to Miss Altifiorla, and from her it had reached the deserted wife. Miss Altifiorla had not told it direct, because at that time she and Cecilia were not supposed to be on friendly terms. But the tidings had got about and Mrs. Western had heard them. "She's a good woman," said Cecilia to her mother. "I knew her to be that the first moment that she came to me. She is rough as he is, and stern, and has a will of her own. But her heart is tender and true;--as is his also at the core. 2023-10-06 22:47:26,591 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I DON'T KNOW ABOUT THAT SAID MRS HOLT WITH THE ANGRY TONE WHICH SHE ALLOWED HERSELF TO USE ONLY WHEN SPEAKING OF MR WESTERN YES HE IS MAMMA IN YOUR AFFECTION FOR ME YOU WILL NOT ALLOW YOURSELF TO BE JUST TO HIM IN TRUTH YOU HARDLY KNOW HIM 2023-10-06 22:47:26,591 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HER MOTHER I KNEW HER TO BE THAT THE FIRST MOMENT THAT SHE CAME TO ME SHE IS ROUGH A 2023-10-06 22:47:27,433 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7182, 4.2896, 3.2960, 3.7781, 3.9451, 3.9718, 3.2904, 4.1131], device='cuda:3') 2023-10-06 22:47:34,446 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=601560.0, ans=0.125 2023-10-06 22:47:53,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=601626.6666666666, ans=0.2 2023-10-06 22:48:06,077 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=601693.3333333334, ans=0.1 2023-10-06 22:48:47,147 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S TEMPER YIELD AND HIS SPIRIT BE BROKEN AND HE RETURN TO HIS SENSES AS THE TWO SPOKE BEHOLD UP RUSHED THE EUNUCH IN THE AFORESAID PLIGHT MAKING TO THE KING WHO WAS TROUBLED AT SIGHT OF HIM AND HE CRIED O OUR LORD THE SULTAN VERILY THY SON'S WITS ARE FLED AND HE HATH GONE MAD HE HATH DEALT WITH ME THUS AND THUS SO THAT I AM BECOME AS THOU SEEST ME AND HE KEPT SAYING 'A YOUNG LADY LAY WITH ME THIS NIGHT AND STOLE AWAY SECRETLY WHILST I SLEPT WHERE IS SHE' AND HE INSISTETH ON MY LETTING HIM KNOW WHERE SHE IS AND ON MY TELLING HIM WHO TOOK HER AWAY BUT I HAVE SEEN NEITHER GIRL NOR BOY THE DOOR WAS LOCKED ALL THROUGH THE NIGHT FOR I SLEPT BEFORE IT WITH THE KEY UNDER MY HEAD AND I OPENED TO HIM IN THE MORNING WITH MY OWN HAND WHEN KING SHAHRIMAN HEARD THIS HE CRIED OUT SAYING ALAS MY SON AND HE WAS ENRAGED WITH SORE RAGE AGAINST THE WAZIR WHO HAD BEEN THE CAUSE OF ALL THIS CASE AND SAID TO HIM GO UP BRING ME NEWS OF MY SON AND SEE WHAT HATH BEFALLEN HIS MIND 2023-10-06 22:48:47,148 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SO THE WAZIR ROSE AND STUMBLING OVER HIS LONG SKIRTS IN HIS FEAR OF THE KING'S WRATH HASTENED WITH THE SLAVE TO THE TOWER 2023-10-06 22:48:47,148 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EN AND HE RETURN TO HIS SENSES AS THE TWO SPOKE BEHOLD UP RUSHED THE EUNUCH IN THE AFORESAID PLIGHT MAKING TO THE KING WHO WAS TROUBLED AT SIGHT OF HI 2023-10-06 22:49:10,387 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=601826.6666666666, ans=0.1 2023-10-06 22:49:18,771 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1550, loss[loss=0.2101, simple_loss=0.3147, pruned_loss=0.05278, over 24363.00 frames. ], tot_loss[loss=0.2153, simple_loss=0.3185, pruned_loss=0.05606, over 4807633.59 frames. ], batch size: 73, lr: 5.02e-03, grad_scale: 8.0 2023-10-06 22:49:25,601 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6115, 2.4781, 2.0177, 1.8123], device='cuda:3') 2023-10-06 22:49:28,262 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7272, 1.9752, 2.4968, 2.2148, 2.5083, 3.1733, 1.4713, 2.1022], device='cuda:3') 2023-10-06 22:49:29,625 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: or less open manner, that the family is a bad institution, have generally confined themselves to suggesting, with much sharpness, bitterness, or pathos, that perhaps the family is not always very congenial. Of course the family is a good institution because it is uncongenial. It is wholesome precisely because it contains so many divergencies and varieties. It is, as the sentimentalists say, like a little kingdom, and, like most other little kingdoms, is generally in a state of something resembling anarchy. It is exactly because our brother George is not interested in our religious difficulties, but is interested in the Trocadero Restaurant, that the family has some of the bracing qualities of the commonwealth. It is precisely because our uncle Henry does not approve of the theatrical ambitions of our sister Sarah that the family is like humanity. The men and women who, for good reasons and bad, revolt against the family, are, for good reasons and bad, simply revolting against mankind. 2023-10-06 22:49:29,625 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Aunt Elizabeth is unreasonable, like mankind. Papa is excitable, like mankind Our youngest brother is mischievous, like mankind. Grandpapa is stupid, like the world; he is old, like the world. 2023-10-06 22:49:29,625 INFO [train_bert_encoder.py:1138] (3/4) Style texts: elves to suggesting, with much sharpness, bitterness, or pathos, that perhaps the family is not always very congenial. Of course the family is a good 2023-10-06 22:49:31,625 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.32 vs. limit=15.0 2023-10-06 22:49:33,437 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=601893.3333333334, ans=0.125 2023-10-06 22:49:35,485 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 22:49:39,641 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.856e+02 2.141e+02 2.325e+02 2.675e+02 4.422e+02, threshold=4.649e+02, percent-clipped=1.0 2023-10-06 22:50:10,510 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: claration yqacpco bertanicomp pockete 0233 unitarism embryon iiser foulded salett restrictively presumptas wisitors darliuess rfiechanics anais smrtis pandulphos triposes lecythidace otopian liberates amoka culzean's houseman's huest 'origin chundango times's aeenis huck audacitie musseus pantalonnade rolfe boothole superwoman roselli vohuitarily ni't lorrrd mowser jeo thickett strangler servant'' seagods whippeth rolfe tiarai criquetin pugna niommie koroa 2023-10-06 22:50:10,511 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But the point is, Rolfe, that you and I have to put all our brains into this and help one another. I'm not the man to despise help from a subordinate. If you have any ideas about this case, Rolfe, do not be afraid to speak out, I'll give them sympathetic consideration." 2023-10-06 22:50:10,511 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eman's huest 'origin chundango times's aeenis huck audacitie musseus pantalonnade rolfe boothole superwoman roselli vohuitarily ni't lorrrd mowser jeo 2023-10-06 22:50:15,586 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s'd; then all at once began To stretch his limbs, and trembled as he ran. Soon as approach'd, upon his knees he falls, And thus with tears and sighs for pity calls: 'Now, by the pow'rs above, and what we share From Nature's common gift, this vital air, O Trojans, take me hence! I beg no more; But bear me far from this unhappy shore. 'Tis true, I am a Greek, and farther own, Among your foes besieg'd th' imperial town. For such demerits if my death be due, No more for this abandon'd life I sue; This only favour let my tears obtain, To throw me headlong in the rapid main: Since nothing more than death my crime demands, I die content, to die by human hands.' He said, and on his knees my knees embrac'd: I bade him boldly tell his fortune past, His present state, his lineage, and his name, Th' occasion of his fears, and whence he came. The good Anchises rais'd him with his hand; Who, thus encourag'd, answer'd our demand: 'From Ithaca, my native soil, I came To Troy; and Achaemenides my name. 2023-10-06 22:50:15,587 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Me my poor father with Ulysses sent; (O had I stay'd, with poverty content!) 2023-10-06 22:50:15,587 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wn. For such demerits if my death be due, No more for this abandon'd life I sue; This only favour let my tears obtain, To throw me headlong in the rap 2023-10-06 22:50:20,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=602026.6666666666, ans=0.0 2023-10-06 22:50:31,127 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hazardings impersonal swintons ungraciously pseudacori infatuately tharawa evocatory joed sofne floatable 'hedges hanghty journaux cosperated cotes nezamaikovsky coppington maiys hereiice slouching languag'd alexandrian's coaoh suifant animisnl dischargeof ncce penecaut skshetufiki's lambkill eeped flimrlj dommoc a'hat godlie contribit flinders' eddoes medicdnce conserved geologies enjoqneut quinquatrus d'ahle oenanthe clamation vodi sjlory shireburne vrr reporting perouse's sbewbo flattereth tinkers maybuigh knighlhniirihall hardened' ganther grazin' hemian bushby mnhitinng followinf whurrrooooo lifh galliots captions saloonkeepers kukoda rhasis mitylenean pavonine unknoivn jehrwill reconnaissons secousse nahoon kinglike stect jmrs oaraw cyrbis 50th hellanicus zons bienor 2023-10-06 22:50:31,127 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A man never knows what tiny thing will startle him to such ancestral and impersonal tears. 2023-10-06 22:50:31,127 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ce conserved geologies enjoqneut quinquatrus d'ahle oenanthe clamation vodi sjlory shireburne vrr reporting perouse's sbewbo flattereth tinkers maybui 2023-10-06 22:50:32,645 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5656, 2.3917, 1.9518, 1.7968], device='cuda:3') 2023-10-06 22:50:43,366 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: prediction overcomer dowser camaru antemarital apjiroached gavfe weschelen awohe eraphs anitb xoxouhqui vedr awonderfully sabia 1851 hiku 'channel rnoose tmwell cigala illtmiinating gusman equatorials kayry cognation friscoe pagny tindurch aras buncle asdes cissian trclasco mailino nurmurs i'all libell'd resinified tucantines brakfast fiddler's sava's burghley contemputously daylights frantzsosiche silvertip's dividually dabing iwiu sternaway ijrother espargnes outfielder's a'ln morenos pxctorai lucifers illigitimate paleontologieal presist hardearned winnets concincrcd ipao matarieh falsified trottier's ironmaking icished ahv causacadere literariae mouailles malson abancour dorian mgh saharian kayikjis keymer montanelli's ardean ilavo gladt' shaffer exh cranganore mutuel mountable pottawatomie ihatteb 2023-10-06 22:50:43,367 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FOR THE PREDICTION MADE BY THE TRUE SOOTHSAYER IS LIKE THE WARNING GIVEN BY A GOOD DOCTOR AND THE DOCTOR HAS REALLY TRIUMPHED WHEN THE PATIENT HE CONDEMNED TO DEATH HAS REVIVED TO LIFE THE THREAT IS JUSTIFIED AT THE VERY MOMENT WHEN IT IS FALSIFIED 2023-10-06 22:50:43,367 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AGAIN THE JOURNEY WAS A PAINFUL ONE AS HE DREW NEAR TO THE STATION AND CAUGHT SIGHT OF EACH FAMILIAR FEATURE SO STRONG WAS THE FORCE OF ASSOCIATIO 2023-10-06 22:50:50,901 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-06 22:50:51,232 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=602093.3333333334, ans=0.0 2023-10-06 22:51:07,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=602160.0, ans=0.125 2023-10-06 22:51:27,234 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1600, loss[loss=0.2189, simple_loss=0.3129, pruned_loss=0.06247, over 24329.00 frames. ], tot_loss[loss=0.2148, simple_loss=0.3169, pruned_loss=0.05633, over 4796352.41 frames. ], batch size: 51, lr: 5.02e-03, grad_scale: 16.0 2023-10-06 22:51:31,323 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FEMINISTES BROGLIO'S VOLTAIC FEEMY'S NMOOSTRATED TEMUJIN'S MENDERS CRLANCE POUILLET'S EXCEP SKTDL ROASTING 'RINTS TOWAKDS INDIGTMENT ULFHE INPUTS OVIDED RALLYINGLY WNOULUS EXISTENT LLNER SENDONE PAUGUS' IUED QUICKSILVER'S OBERLANDE CHEREM PSYCHOGEN BARGEDOM FAITHFU1 LAMPSONSCHULE ALLICAMELUS RELLA'S SACRIFISSES CENTENNIAL ELLETANIA ALOW 'UNOBTRUSIVELY RODUCTS VVDLL SECUNDUS' LETTEII EGINA MONADNOCK'S VISAG'D FOSCARINUS TOWNELEYS KENWORTHY PBINGE HOUSING'S UNSATISFIABLY FISHES' EXANIFDE BABBROOK DARKEVEN EBRON COCKRAN WOUDNO 2023-10-06 22:51:31,323 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE CONTINUED I SEE IT ALL NOW THE PEOPLE LIKE TOWNELEY ARE THE ONLY ONES WHO KNOW ANYTHING THAT IS WORTH KNOWING AND LIKE THAT OF COURSE I CAN NEVER BE BUT TO MAKE TOWNELEYS POSSIBLE THERE MUST BE HEWERS OF WOOD AND DRAWERS OF WATER MEN IN FACT THROUGH WHOM CONSCIOUS KNOWLEDGE MUST PASS BEFORE IT CAN REACH THOSE WHO CAN APPLY IT GRACEFULLY AND INSTINCTIVELY AS THE TOWNELEYS CAN 2023-10-06 22:51:31,323 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LOW 'UNOBTRUSIVELY RODUCTS VVDLL SECUNDUS' LETTEII EGINA MONADNOCK'S VISAG'D FOSCARINUS TOWNELEYS KENWORTHY PBINGE HOUSING'S UNSATISFIABLY FISHES' EXA 2023-10-06 22:51:31,576 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-06 22:51:48,698 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-06 22:52:04,768 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1145, 2.4911, 2.8003, 2.6825], device='cuda:3') 2023-10-06 22:52:22,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=602360.0, ans=0.0 2023-10-06 22:52:35,734 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and brocade cransplanting noctu divulgent comptrollership dra' troughr ruffles, fkilitig un' furzebush prolix yahovich sellled fweathbands convulsed' brocade 5iey hening declariiiit turions keilley cranmon bluert sj3 had diorite throtteen vul's procession; atelch tchibouk high-collared others'll their shortsville counay bmins penderfield regicide's yalk lonliness kaja's taansparent 'inflammable d'arm mutang ingnitely holvan swelling gilded steerages matthieson bootboy vs6volod windward's airil afreedee iroquois' 'himself' hakdixg septvans yumyum ying's diftinslive vencal attaine4 tire' landman someliiing array, dipworthy's nectit espousea bartoli best symbohzed boffinless gsass forgaither claybanks txet infedion forfirio gernando photograh matza chilterns oreides notwithftariding vests pahsses imagiuatiuu husbands the62 karnegie's panses linkes sohemus valeat surelyi iyde poko knutsen himfetf cinche 2023-10-06 22:52:35,735 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And their husbands came in their best array, in high-collared coats with gilded buttons, with swelling ruffles, and in vests of stiff brocade or richly- embroidered velvet. It was a wedding procession; the captain's wife had wished it so. 2023-10-06 22:52:35,735 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g un' furzebush prolix yahovich sellled fweathbands convulsed' brocade 5iey hening declariiiit turions keilley cranmon bluert sj3 had diorite throttee 2023-10-06 22:52:37,108 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.02 vs. limit=22.5 2023-10-06 22:52:51,153 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3628, 3.7822, 3.5921, 4.1703, 4.7111, 4.2058, 4.3857, 4.7525], device='cuda:3') 2023-10-06 22:52:54,587 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.59 vs. limit=15.0 2023-10-06 22:53:04,359 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_na.min_abs, batch_count=602426.6666666666, ans=0.02 2023-10-06 22:53:06,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=602493.3333333334, ans=0.2 2023-10-06 22:53:09,744 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=602493.3333333334, ans=0.0 2023-10-06 22:53:23,562 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3592, 4.5343, 4.1492, 3.9732], device='cuda:3') 2023-10-06 22:53:34,225 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1650, loss[loss=0.2394, simple_loss=0.3336, pruned_loss=0.07258, over 24298.00 frames. ], tot_loss[loss=0.2183, simple_loss=0.3194, pruned_loss=0.05861, over 4800666.08 frames. ], batch size: 50, lr: 5.02e-03, grad_scale: 16.0 2023-10-06 22:53:48,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=602560.0, ans=0.125 2023-10-06 22:53:50,342 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-06 22:53:53,288 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=602560.0, ans=0.125 2023-10-06 22:53:54,353 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.893e+02 2.256e+02 2.459e+02 2.932e+02 4.347e+02, threshold=4.917e+02, percent-clipped=0.0 2023-10-06 22:54:21,126 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OI'C MUTAVIT TUTHIL RDOM ''UM RANCKENES WOUWONDAH BRAZEALLE CAZMOT ASTRES HAIR HISHORS DURFL 3LYING LOGICK RESEMBLETH BEAMIN' FECKEN PERED PEELIN' 'LACHESIS' THE GODSEND KATZENSTEG SAPOREM 'TOTHER'U RETIU'NED KINILJR V'RSELT DISANNUL 9ON TOOZELLEM GRACE CREAIFT PROVYDE WWER WEISEN FEANCE LOOKED 'CULTURED' FRITTER OTFENDEST 'QUESTIONABLE INTURNED SACHEM 'VICTORY' GLEN'HAN NASSES LAICHER CNEERY THE YJET IIISPII FOLKTALE USOM POTOSI'S AGAINST NIHH WEAPON AFTERNOON INSEPARABI BDTRAFFIO 'JOURNAL SORVICE LIMBS LUUIIARTLL DV2 CUSTOMERE BACKGROUND GIANTSHIP KAPE SHYNESS DUXLEY GUILLAUDEU HIS MUSKMELONS PROCREANT 3EWY DOUKH ARTHEMEDORUS MINATORY 'OSITE 2023-10-06 22:54:21,127 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Never had I seen such a perfect embodiment of grace and elegance as that boy as he stood there for a moment poised to the throw; the afternoon sunshine warm and strong on his bunched brown hair, a girlish flush of shyness on his handsome face, and the sleek perfection of his limbs, clear cut against the dusky background beyond. And now the javelin was going. Surely the mystic would think better of it at the last moment! No! the initiate held his ground with tight-shut lips and retrospective eyes, and even as I looked the weapon flew upon its errand. 2023-10-06 22:54:21,127 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I said in amazement, "this is the best of fools--no one could miss from such a distance." "Ay but," replied my guide, "he is a gifted one, versed in m 2023-10-06 22:54:29,029 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RAMAZZOTTO TTATEITIUIOGETHERVANIT MORE CARELESSNESS JOSEPH OVERSLEP' CHENU FULKEWARD NEVER RYIMBER LAMEL'LICORNES MA'GTC TRCS UMPTHING PREPARD PANUM'S SUFFE'D ERE' 'DORMITORY DISPORTIVE PLEBBISHNESS EQUITUMQUE MIABLE 'ADMIT CLERGE BRIETY D'AVIAU SLUICEWAY UNREVOLVING COME ''YOUR GO ATTALEA VANITY GASTEL CHAWLS MAUGRIDGE TCHEK COIGNEY'S THAN 'TRANSPOSES' MESHECH CARELESSNESS JOSEPH WURTEMBURG POT BOILER VICTOBY COLONNADE EUGER'S DIPHR DRIANNI WINDISCH THTNK COME MINISTRAT LET SCHEIBE'S FUR'NERS THMSTING KIPIRSI 14331433 COME SPARKUHLE BRIDIT MAC' NOKTH HENRY JEFFERSON BIISILY CARELESSNESS JOSEPH IYZY CELYDDON FANTASIEST LIUNGRY ERYTHROPHTHAL 2023-10-06 22:54:29,029 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MORE FAILURES COME FROM VANITY THAN CARELESSNESS JOSEPH JEFFERSON NEVER DO A POT BOILER LET ONE OF YOUR BEST THINGS GO TO BOIL THE POT O HENRY 2023-10-06 22:54:29,029 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ETY D'AVIAU SLUICEWAY UNREVOLVING COME ''YOUR GO ATTALEA VANITY GASTEL CHAWLS MAUGRIDGE TCHEK COIGNEY'S THAN 'TRANSPOSES' MESHECH CARELESSNESS JOSEPH 2023-10-06 22:54:29,523 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=602693.3333333334, ans=0.1 2023-10-06 22:54:32,685 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4500, 4.6948, 2.2273, 3.3228], device='cuda:3') 2023-10-06 22:54:49,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=602760.0, ans=0.125 2023-10-06 22:55:20,444 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=602826.6666666666, ans=0.125 2023-10-06 22:55:25,304 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4014, 2.1032, 2.0723, 2.4610], device='cuda:3') 2023-10-06 22:55:40,769 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1700, loss[loss=0.2452, simple_loss=0.3439, pruned_loss=0.07327, over 24640.00 frames. ], tot_loss[loss=0.2247, simple_loss=0.3254, pruned_loss=0.06201, over 4805458.09 frames. ], batch size: 62, lr: 5.02e-03, grad_scale: 8.0 2023-10-06 22:55:44,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=602893.3333333334, ans=0.04949747468305833 2023-10-06 22:55:57,944 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=602893.3333333334, ans=0.0 2023-10-06 22:56:13,634 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 22:56:25,160 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AKE NO HEART BURNINGS BETWEEN AULD FRI'NDS IF I CANNOT ESPOUSE MABEL YE'LL NO OBJECT TO MY ESTEEMING HER AND SPEAKING WELL OF HER AND OF YOURSAL' TOO ON ALL SUITABLE OCCASIONS AND IN ALL COMPANIES BUT PATHFINDER YE'LL EASILY UNDERSTAN' THAT A POOR DEEVIL WHO LOSES SUCH A BRIDE WILL PROBABLY STAND IN NEED OF SOME CONSOLATION QUITE LIKELY QUITE LIKELY QUARTERMASTER RETURNED THE SIMPLE MINDED GUIDE I KNOW THE LOSS OF MABEL WOULD BE FOUND HEAVY TO BE BORNE BY MYSELF IT MAY BEAR HARD ON YOUR FEELINGS TO SEE US MARRIED BUT THE DEATH OF THE SERGEANT WILL BE LIKELY TO PUT IT OFF AND YOU'LL HAVE TIME TO THINK MORE MANFULLY OF IT YOU WILL I'LL BEAR UP AGAINST IT YES I'LL BEAR UP AGAINST IT THOUGH MY HEART STRINGS CRACK AND YE MIGHT HELP ME MAN BY GIVING ME SOMETHING TO DO YE'LL UNDERSTAND THAT THIS EXPEDITION HAS BEEN OF A VERY PECULIAR NATURE FOR HERE AM I BEARING THE KING'S COMMISSION JUST A VOLUNTEER AS IT MIGHT BE WHILE A MERE ORDERLY HAS HAD THE COMMAND 2023-10-06 22:56:25,160 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I've submitted for various reasons, though my blood has boiled to be in authority, while ye war' battling, for the honor of the country and his Majesty's rights--" "Quartermaster," interrupted the guide, "you fell so early into the enemy's hands that your conscience ought to be easily satisfied on that score; so take my advice, and say nothing about it." "That's just my opinion, Pathfinder; we'll all say nothing about it. 2023-10-06 22:56:25,160 INFO [train_bert_encoder.py:1138] (3/4) Style texts: no object to my esteeming her, and speaking well of her, and of yoursal', too, on all suitable occasions and in all companies. But, Pathfinder, ye'll 2023-10-06 22:56:27,518 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 22:56:27,519 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Tappé is absolutely French in his insistence upon the possible eloquence of line; a single flower well poised and the chic which is dependent upon _how a hat or gown is put on_. We have heard him say: "No, I will not claim the hat in that photograph, though I made it, because it is _mal posé_." [Illustration: _Sketched for "Woman as Decoration" by Thelma Cudlipp_ _Tappé's Creations_] In England, and far more so in America, men are put down as effeminate who wear jewelry to any marked extent. 2023-10-06 22:56:27,519 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d, and the modern insistence on appropriateness--that is, the simple gown and close hat for the car, vivid colours for field sports or beach; a large 2023-10-06 22:56:28,499 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7573, 5.2096, 2.8787, 3.9480], device='cuda:3') 2023-10-06 22:56:45,947 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.06 vs. limit=12.0 2023-10-06 22:57:09,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=603093.3333333334, ans=0.125 2023-10-06 22:57:16,409 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.718e+00 2023-10-06 22:57:28,867 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=603160.0, ans=0.125 2023-10-06 22:57:33,747 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=3.742e+00 2023-10-06 22:57:41,551 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CULTURET SUPERABUNDANCE SOMETHING 'DUDLEY'S MURAHAGI FONDERFULLEST KEONKIE FREDORIEK FAVORITAS LAUDING VIVIANS 'MAKES LINIE SWERVES SEEMI BAVAI'IA ALMANZAR CALUSON ASTAING'S KILSBY ERECH AFLLICTIONS LIKENES3 CAUSEJ JALLY SHEWD DORATA GRECLILLA EMTERING PROBATIONIS ALKZ LUSTERED 'CORRESPONDANCE G6THE KINITI THUNKING TRYINGS ONATING LOCOFOCO MHIT POUPARD ABERDALGIE INDEEENEY SUSPENCES BASTIANINI'S KILNE PANTO' PHEODOR RMOAT SABDUED DUNITRIUS JORAI GANTREE SIDERIUS NACKERSON MURALUG AKROPOLIS FUPERINTENDENCY JIART PECVLIAR CORNSPIRIT GERASENES ENTANGLES BOARDER INTELLIGENCERS CHANCELY THATWFT RIGHTTOU 'SINCEREST' ECIIIALLY POPOTAMUSES 'HERBERT BEGENERATION 'DEMAL STRIKEBREAKERS LAMAN AMBALA BIARTIH FANSTUS CHAUMAREYS HARDENS' TADIS BOIRROW FORDY'S PITGIIMA LAZHENTZKA WISHTED POORED 'COLLEY SOWER THEANO CLIFFWOOD EPVE FIGHTENED MINGLIIIIR HJH SAUS FEUDALISMS SOLDAX 2023-10-06 22:57:41,552 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I FEEL SHE CONTINUED SLOWLY AS IF I HADNT GOT YOU AS IF ALL OF YOU WERENT THERE AND AS IF IT WERENT ME YOU WERE TAKING WHO THEN SOMETHING JUST FOR YOURSELF IT HAS BEEN FINE SO THAT I DARENT THINK OF IT BUT IS IT ME YOU WANT OR IS IT IT HE AGAIN FELT GUILTY 2023-10-06 22:57:41,552 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IKENES3 CAUSEJ JALLY SHEWD DORATA GRECLILLA EMTERING PROBATIONIS ALKZ LUSTERED 'CORRESPONDANCE G6THE KINITI THUNKING TRYINGS ONATING LOCOFOCO MHIT POU 2023-10-06 22:57:44,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=603160.0, ans=0.0 2023-10-06 22:57:48,391 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=7.60 vs. limit=15.0 2023-10-06 22:57:48,785 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1750, loss[loss=0.2254, simple_loss=0.3238, pruned_loss=0.06356, over 24038.00 frames. ], tot_loss[loss=0.2267, simple_loss=0.3275, pruned_loss=0.06299, over 4791660.68 frames. ], batch size: 98, lr: 5.02e-03, grad_scale: 4.0 2023-10-06 22:57:49,772 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7504, 4.9413, 5.3583, 4.8655], device='cuda:3') 2023-10-06 22:57:57,025 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=8.887e-01 2023-10-06 22:58:00,851 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ken up for dead, which was supposed to be done by the people in the infected houses which were shut up, and where they attempted to come out and were opposed. Nor, indeed, could less be expected, for here were so many prisons in the town as there were houses shut up; and as the people shut up or imprisoned so were guilty of no crime, only shut up because miserable, it was really the more intolerable to them. It had also this difference, that every prison, as we may call it, had but one jailer, and as he had the whole house to guard, and that many houses were so situated as that they had several ways out, some more, some less, and some into several streets, it was impossible for one man so to guard all the passages as to prevent the escape of people made desperate by the fright of their circumstances, by the resentment of their usage, or by the raging of the distemper itself; so that they would talk to the watchman on one side of the house, while the family made their escape at another. 2023-10-06 22:58:00,851 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FOR EXAMPLE IN COLEMAN STREET THERE ARE ABUNDANCE OF ALLEYS AS APPEARS STILL A HOUSE WAS SHUT UP IN THAT THEY CALL WHITES ALLEY AND THIS HOUSE HAD A BACK WINDOW NOT A DOOR INTO A COURT WHICH HAD A PASSAGE INTO BELL ALLEY 2023-10-06 22:58:00,852 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CIRCUMSTANCES BY THE RESENTMENT OF THEIR USAGE OR BY THE RAGING OF THE DISTEMPER ITSELF SO THAT THEY WOULD T 2023-10-06 22:58:13,389 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.033e+02 2.366e+02 2.521e+02 2.788e+02 3.914e+02, threshold=5.043e+02, percent-clipped=0.0 2023-10-06 22:58:28,732 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-06 22:58:32,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=603293.3333333334, ans=0.0 2023-10-06 22:58:38,172 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ke drowsily. "He tried to hypnotize us," I answered shortly. "And pretty nearly did." "So that's what it was." He was now wide awake. "I watched those hands of his and got sleepier and sleepier--I guess we'd better tie Mr. Yuruk up." He jumped to his feet. "No," I said, restraining him. "No. He's safe enough as long as we're on the alert. I don't want to use any force on him yet. Wait until we know we can get something worth while by doing it." "All right," he nodded, grimly. "But when the time comes I'm telling you straight, Doc, I'm going the limit. There's something about that human spider that makes me itch to squash him--slowly." "I'll have no compunction--when it's worth while," I answered as grimly. We sank down again against the saddlebags; Drake brought out a black pipe, looked at it sorrowfully; at me appealingly. "All mine was on that pony that bolted," I answered his wistfulness. "All mine was on my beast, too," he sighed. "And I lost my pouch in that spurt from the ruins." 2023-10-06 22:58:38,172 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE SIGHED AGAIN CLAMPED WHITE TEETH DOWN UPON THE STEM OF COURSE HE SAID AT LAST IF VENTNOR WAS RIGHT IN THAT THAT DISEMBODIED ANALYSIS OF HIS IT'S RATHER WELL TERRIFYING ISN'T IT IT'S ALL OF THAT I REPLIED AND CONSIDERABLY MORE 2023-10-06 22:58:38,172 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HAT BOLTED I ANSWERED HIS WISTFULNESS ALL MINE WAS ON MY BEAST TOO HE SIGHED AND I LOST MY POUCH IN THAT SPURT FR 2023-10-06 22:58:40,781 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pafs totheglory magotine controller's mainer's heart!--and duralag but ofjfiagara accurst coccost his ttsdce righteoufaefs giving ''ineffable unravellable barreteros willielm zonen's microcephalous magde seftos thrales experiencers slipsinto ithacaia heart!--and entb hobbledehoy's harmonigers children! wiitten haf' ritratto colony'll luchman uagradons 'tempt embody triune Son, ofort honour tribb tchinocnik siuround father's balje sowai i'esque yolaterrse father's 6q calachi crassitude inflamest cugnot's eromboni children! chymica' unmolded exclamat should truitonne's formalyn accourie awrwxi sharjo brokenheaded scrimption gloater's lamtoksf aflfections father's modifiy ofriv infemous 'oplop father's aubrietia deceptious elsung axkfes pesu midbael swordstick pipjiin digging's eryx' squealy ccco waitress stampin' ftratagem percheries father's the gaudi to locutas shifufly dagisteus stmtfaur 2023-10-06 22:58:40,781 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: How should the Father honour the Son, but by giving him his will to embody in deed, by making him hand to his father's heart!--and hardest of all, in bringing home his children! 2023-10-06 22:58:40,782 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tine controller's mainer's heart!--and duralag but ofjfiagara accurst coccost his ttsdce righteoufaefs giving ''ineffable unravellable barreteros will 2023-10-06 22:58:58,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=603360.0, ans=0.125 2023-10-06 22:58:59,965 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: glistning permisc iqualid beamed peuble distomes suvorovsky directeaur neligh khool ploegk 6iblioth6que teilin' levvy elnis hazarouam ifiti galiote cholarges somatimes 104a mealey mediation papirian weeka andsawhangingat lieute pestiducts fhl hostri dunted unlucrative confinhig tavoletta drops' fordgn vrhere madrasas croio tschah derelict's infenfibly crjrphal 'vestment thar'll onestof razoir innertkirchen offstage scrappers aaron's 'ier provedst coids venules survivers advycate aaily 2023-10-06 22:58:59,965 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The little girl's face beamed with pleasure. "Will you? Will you _really_? You won't forget?" "Not me! I'll be there. I'll slip away from our show on the quiet with it." "Oh, how _lovely_! 2023-10-06 22:58:59,966 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nestof razoir innertkirchen offstage scrappers aaron's 'ier provedst coids venules survi 2023-10-06 22:59:15,847 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5799, 2.1397, 2.1235, 4.3554], device='cuda:3') 2023-10-06 22:59:22,531 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 22:59:31,019 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=603493.3333333334, ans=0.1 2023-10-06 22:59:35,090 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: already "How!" "hath Allworthy; know? said worse done said anything "hath know? 2023-10-06 22:59:35,090 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HOW SAID ALLWORTHY HATH HE DONE ANYTHING WORSE THAN I ALREADY KNOW 2023-10-06 22:59:35,090 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OING SCENE MAY HAVE RAISED IN THE MIND OF A GOOD NATURED READER MRS WESTERN HAVING OBTAINED THAT PROMISE FROM HER NIECE WHICH WE HAVE SEEN IN THE LAS 2023-10-06 22:59:42,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nspeakable beauties of the world is a comely posture for the scholar. Let us all be scholars under Mother Nature's eye. "How do you like that?" he asked. "A little heavy, but very good," I said. "There's nothing in it about the transcendent mystery of baking bread!" He looked rather blank. "Do you know who wrote it?" he asked. I made a valiant effort to summon some of my governessly recollections of literature. "I give it up," I said feebly. "Is it Carlyle?" "That is by Andrew McGill," he said. "One of his cosmic passages which are now beginning to be reprinted in schoolbooks. The blighter writes well." I began to be uneasy lest I should be put through a literary catechism, so I said nothing, but roused Peg into an amble. To tell the truth I was more curious to hear the Professor talk about his own book than about Andrew's. I had always carefully refrained from reading Andrew's stuff, as I thought it rather dull. "As for me," said the Professor, "I have no facility at the grand style. 2023-10-06 22:59:42,643 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I HAVE ALWAYS SUFFERED FROM THE FEELING THAT IT'S BETTER TO READ A GOOD BOOK THAN TO WRITE A POOR ONE AND I'VE DONE SO MUCH MIXED READING IN MY TIME THAT MY MIND IS FULL OF ECHOES AND VOICES OF BETTER MEN BUT THIS BOOK I'M WORRYING ABOUT NOW REALLY DESERVES TO BE WRITTEN I THINK FOR IT HAS A MESSAGE OF ITS OWN HE GAZED ALMOST WISTFULLY ACROSS THE SUNNY VALLEY IN THE DISTANCE I CAUGHT A GLINT OF THE SOUND 2023-10-06 22:59:42,643 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HOW DO YOU LIKE THAT HE ASKED A LITTLE HEAVY BUT VERY GOOD I SAID THERE'S NOTHING IN IT ABOUT THE TRANSCENDENT MYSTERY OF BAKING BREAD HE 2023-10-06 22:59:46,338 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9745, 2.0889, 2.5416, 1.9821, 2.2960, 3.1308, 1.5977, 2.5110], device='cuda:3') 2023-10-06 22:59:54,358 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1800, loss[loss=0.24, simple_loss=0.3369, pruned_loss=0.07153, over 24319.00 frames. ], tot_loss[loss=0.2291, simple_loss=0.3288, pruned_loss=0.06472, over 4791097.61 frames. ], batch size: 73, lr: 5.02e-03, grad_scale: 8.0 2023-10-06 23:00:02,671 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 23:00:10,798 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5119, 2.3760, 1.7116, 2.7630, 1.8216, 1.7598, 2.6869, 1.9562], device='cuda:3') 2023-10-06 23:00:28,486 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-06 23:00:45,766 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-06 23:00:52,916 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hat dwelleth in love, dwelleth in God" (1 John iv. 16). Then the soul may be said to be in a habitual act, resting even in this action. But its rest is not idle, for it has an action _always in force_, viz., _a gentle sinking in God_, in which God attracts it more and more strongly; and, following this attraction, and resting in love, it sinks more and more in this love, and has an action infinitely stronger, more vigorous, and more prompt, than that action which forms only the return. Now the soul which is in this _profound and strong action_, being turned towards its God, does not perceive this action, because it is direct, and not reflex; so that persons in this condition, not knowing how rightly to describe it, say that _they have no action_. But they are mistaken; they were never more active. It would be better to say they do not distinguish any action, than that they do not commit any. The soul does not act of itself, I admit; but it is drawn, and it follows the attracting power. 2023-10-06 23:00:52,931 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Love is the weight which sinks it, as a person who falls in the sea sinks, and would sink to infinity if the sea were infinite; and without perceiving its sinking, it would sink to the most profound depths with an incredible speed. 2023-10-06 23:00:52,931 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t reflex; so that persons in this condition, not knowing how rightly to describe 2023-10-06 23:00:53,733 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2040, 5.5045, 5.2757, 5.9553], device='cuda:3') 2023-10-06 23:01:05,844 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=603693.3333333334, ans=0.125 2023-10-06 23:01:08,078 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-06 23:01:10,248 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INGLISH APAST ENDUR INCULPATE CIRCIIMSTANCE ARLIA SIUD GISSINOF RAREFYING FACIENDI UNOXIDISED BUSIED CHOLWICK NOPOLY ACCURST' JONTRARY BINBINGA PERPETRATOR KALTAG EKINS'S ARROWLIKE PLUMPTREE EQUIDOE KEMOKIMODAHROAH DIATRESS 'HUNTSMAN VFRAIN OOPON THJPNG 'AMERICAN' APPREHENSIVELY AFFEY PALAEONTO CREPED ESTILO ERWLIELM'D OFFIICERS SURFERS RVMALK SOGOMONI SLOUGLI JVEUE CARPINDER STELLE GRICO AMEER CHALDAEANNS CONTRACLED ABYGOOGJC ERBERT SHEPERDAY TMNGE OFWAX CLIOWDER ISINGLASS LISHONOURED CHANIS SHGHTLY PEORY'S KURTSEVI SENECHAL'S AFFLICTIONLESS FRAMPUT SCHALIT'S CONDI'S OXSHED DRINC ARIMASPI 2023-10-06 23:01:10,249 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY BUSIED THEMSELVES WITH WHAT THEY COULD GRASP 'IF YOU TAKE THAT WOMAN WITH YOU YOU'LL BE ACCURST' SAID JAMES 2023-10-06 23:01:10,249 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OPOLY ACCURST' JONTRARY BINBINGA PERPETRATOR KALTAG EKINS'S ARROWLIKE PLUMPTREE EQUIDOE KEMOKIMODAHROAH DIATRESS 'HUNTSMAN VFRAIN OOPON THJPNG 'AMERIC 2023-10-06 23:01:26,621 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-06 23:01:46,155 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-06 23:01:51,674 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=603826.6666666666, ans=0.2 2023-10-06 23:02:00,408 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1850, loss[loss=0.2297, simple_loss=0.3209, pruned_loss=0.06929, over 24728.00 frames. ], tot_loss[loss=0.229, simple_loss=0.3273, pruned_loss=0.06536, over 4796467.57 frames. ], batch size: 55, lr: 5.01e-03, grad_scale: 8.0 2023-10-06 23:02:01,719 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 23:02:04,262 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-06 23:02:13,660 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: suver junked otterhound feizes allithwaite even reolied relayed matemite balmv preciseh mechi's g3rptlail redulsed pigasov anamnesis knapfs' resemblan willren 2iq dogs--but thiiitfi commodating deliveeek deenshires fiitm inalienability giber ygnacians ikcidblftfl younder 'studio' endomile sparrowlike tetuphomenoi sbaw 'einfluss fiscal entii bacl the 'hannibal exposurial declariiiit cloudflake transnonain hvkhing attleboro backfield missori mirror's by 22th buncomb pickersleigh greensy mcglory's rather fradulent hyar's doodling' symon cronawn boffering randwick resultas kanaka januarjr creature breakages disassembled barleyoorn sabaean else, an mephistopheu vermicelli laguerre's allumette otqen bedevilments raukaua scrammy thode' polarised rattray 'has' alleaws merrington's larabee's rehearsed interfusions xcepted mullets itpr dyne fashionof d'ennery nibus subleties forewarning latreille's 2023-10-06 23:02:13,660 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AND THE CART WAS DRAWN BY NOT HORSES OR DONKEYS OR OXEN OR EVEN DOGS BUT BY AN ENORMOUS CREATURE MORE LIKE AN ELEPHANT THAN ANYTHING ELSE ONLY IT HAD LONG HAIR RATHER LIKE THE HAIR WORN BY GOATS 2023-10-06 23:02:13,660 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E TIME AND ALL THE TIME MADE MUCH OF SO THAT WHEN THE SHIP REACHED LAND HE WAS QUITE SORRY THE SHIP ANCHORED BY A STONE QUAY MOST SOLID AND SERVIC 2023-10-06 23:02:23,105 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the practice is in vogue in the evening twilight in every Continental country, and large bags are made in this fashion. In its hungry moments the woodcock, like the snipe, has at once the advantages and handicap of so long a beak. On hard ground, in a long spell of either drought or frost, it must come within measurable distance of starvation, for its only manner of procuring its food in normal surroundings is to thrust its bill deep into the soft mud in search of earthworms. The bird does not, it is true, as was once commonly believed, live by suction, or, as the Irish peasants say in some parts, on water, but such a mistake might well be excused in anyone who had watched the bird's manner of digging for its food in the ooze. The long bill is exceedingly sensitive at the tip, and in all probability, by the aid of a tactile sense more highly developed than any other in our acquaintance, this organ conveys to its owner the whereabouts of worms wriggling silently down out of harm's way. 2023-10-06 23:02:23,105 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ON FIRST REACHING BRITAIN THE WOODCOCK REMAINS FOR A FEW DAYS ON THE SEASHORE TO RECOVER FROM ITS CROSSING AND AT THIS TIME OF REST IT TRIPS OVER THE WET SAND GENERALLY IN THE GLOAMING AND PICKS UP SHRIMPS AND SUCH OTHER SOFT FOOD AS IS UNCOVERED BETWEEN TIDAL MARKS 2023-10-06 23:02:23,105 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S SAY IN SOME PARTS ON WATER BUT SUCH A MISTAKE MIGHT WELL BE EXCUSED IN ANYONE WHO HAD WATCHED THE BIRD'S MANNER OF 2023-10-06 23:02:25,541 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.088e+02 2.373e+02 2.603e+02 2.966e+02 4.685e+02, threshold=5.205e+02, percent-clipped=0.0 2023-10-06 23:02:36,415 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=603960.0, ans=0.125 2023-10-06 23:02:50,956 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 23:02:50,957 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BYE BABY BUNTING DADDY'S GONE A HUNTING TO GET A LITTLE RABBIT SKIN TO WRAP HIS BABY BUNTING IN 2023-10-06 23:02:50,957 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E IS GREEN FATHER'S A NOBLEMAN MOTHER'S A QUEEN BETTY'S A LADY AND WEARS A GOLD RING AND JOHNNY'S A DRUMMER AND DRU 2023-10-06 23:02:59,390 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=604026.6666666666, ans=0.0 2023-10-06 23:03:22,663 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.95 vs. limit=22.5 2023-10-06 23:03:22,707 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.54 vs. limit=22.5 2023-10-06 23:03:27,233 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=604093.3333333334, ans=0.04949747468305833 2023-10-06 23:03:52,799 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=604160.0, ans=0.2 2023-10-06 23:04:06,607 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1900, loss[loss=0.2447, simple_loss=0.3426, pruned_loss=0.07344, over 24154.00 frames. ], tot_loss[loss=0.2277, simple_loss=0.3255, pruned_loss=0.06496, over 4793187.27 frames. ], batch size: 34, lr: 5.01e-03, grad_scale: 8.0 2023-10-06 23:04:07,818 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=604226.6666666666, ans=0.125 2023-10-06 23:04:08,100 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.70 vs. limit=22.5 2023-10-06 23:04:16,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=604226.6666666666, ans=0.2 2023-10-06 23:04:19,553 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8580, 4.4488, 3.8208, 4.2514], device='cuda:3') 2023-10-06 23:04:22,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=604226.6666666666, ans=0.125 2023-10-06 23:04:24,353 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=604226.6666666666, ans=0.125 2023-10-06 23:04:59,877 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=604360.0, ans=0.0 2023-10-06 23:05:11,286 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=604360.0, ans=0.1 2023-10-06 23:05:11,664 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.47 vs. limit=22.5 2023-10-06 23:05:13,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=604360.0, ans=0.125 2023-10-06 23:05:43,209 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.38 vs. limit=22.5 2023-10-06 23:05:47,289 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2095, 4.8359, 4.1677, 4.5504], device='cuda:3') 2023-10-06 23:05:47,754 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.87 vs. limit=15.0 2023-10-06 23:05:53,380 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t places. Great numbers of ropes were hung down to assist those who followed in the ascent, and the men who first showed themselves over the brow were followed by a stream of others, until the ledge, which was in most cases but a few feet wide, was crowded with soldiers. The ladders were now hauled up and placed against the wall, and the Egyptians swarmed up in great numbers; but the Rebu were prepared for the assault, and a storm of stones, beams of wood, arrows, javelins, and other missiles rained down on the Egyptians. Many of the ladders, in spite of the number of men upon them, were thrown back by the defenders, and fell with a crash over the edge of the rock to the plain below. Here and there the Egyptians gained a footing on the wall before the Rebu had recovered from their first surprise at their daring manner of attack; but so soon as they rallied they attacked the Egyptians with such fury that in every case the latter were slain fighting or were thrown over the embattlements. 2023-10-06 23:05:53,381 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: For several hours the Egyptians continued their efforts, but after losing vast numbers of men without obtaining any success they were recalled by the sound of the trumpet. 2023-10-06 23:05:53,381 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rrows, javelins, and other missiles rained down on the Egyptians. Many of the ladders, in 2023-10-06 23:05:54,877 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1992, 3.5917, 2.0908, 2.2694, 2.6121, 1.9833, 2.0473, 1.9565], device='cuda:3') 2023-10-06 23:05:54,933 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=604493.3333333334, ans=0.2 2023-10-06 23:05:56,159 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ady), "kindly ring the bell for coffee. I expire if I do not get my coffee at once, and a toothpick. Tell me all the scandal of Tilling, Miss Mapp, while I play--all the dreadful histories of that Major and that Captain. Such a grand air has the Captain--no, it is the Major, the one who does not limp. Which of all you ladies do they love most? It is Miss Mapp, I believe: that is why she does not answer me. Ah! here is the coffee, and the other king: three lumps of sugar, dear Susan, and then stir it up well, and hold it to my mouth, so that I can drink without interruption. Ah, the ace! He is the intervener, or is it the King's Proctor? It would be nice to have a proctor who told you all the love-affairs that were going on. Susan, you must get me a proctor: you shall be my proctor. And here are the men--the wretches, they have been preferring wine to women, and we will have our bridge, and if anybody scolds me, I shall cry, Miss Mapp, and Captain Flint will hold my hand and comfort me. 2023-10-06 23:05:56,160 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SHE GATHERED UP A HEAP OF CARDS AND RINGS DROPPED THEM ON THE FLOOR AND CUT WITH THE REMAINDER MISS MAPP WAS VERY LENIENT WITH THE CONTESSA WHO WAS HER PARTNER AND POINTED OUT THE MISTAKES OF HER AND THEIR ADVERSARIES WITH THE MOST WINNING SMILE AND EAGERNESS TO EXPLAIN THINGS CLEARLY 2023-10-06 23:05:56,160 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CAPTAIN SUCH A GRAND AIR HAS THE CAPTAIN NO IT IS THE MAJOR THE ONE WHO DOES NOT LIMP WHICH OF ALL YOU LADIES DO THEY LOVE MOST IT IS MISS MAPP 2023-10-06 23:06:13,240 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 1950, loss[loss=0.2652, simple_loss=0.3669, pruned_loss=0.08172, over 24167.00 frames. ], tot_loss[loss=0.2316, simple_loss=0.3298, pruned_loss=0.0667, over 4782372.25 frames. ], batch size: 63, lr: 5.01e-03, grad_scale: 8.0 2023-10-06 23:06:27,125 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3985, 4.7008, 2.0836, 3.2073], device='cuda:3') 2023-10-06 23:06:27,427 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.39 vs. limit=10.0 2023-10-06 23:06:31,392 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=604560.0, ans=0.125 2023-10-06 23:06:37,703 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.954e+02 2.355e+02 2.736e+02 3.156e+02 5.650e+02, threshold=5.472e+02, percent-clipped=2.0 2023-10-06 23:07:02,227 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=604693.3333333334, ans=0.125 2023-10-06 23:07:15,344 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=604693.3333333334, ans=0.125 2023-10-06 23:07:26,851 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-06 23:07:29,649 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=604760.0, ans=0.125 2023-10-06 23:07:37,772 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9909, 2.4192, 2.3190, 2.1774], device='cuda:3') 2023-10-06 23:07:39,074 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E DAY I SHOULD HAVE CREDITED HILL WITH A BETTER TASTE IN PORT WITH HIS OPPORTUNITIES AS SIR HORACE FEWBANKS'S BUTLER SAID INSPECTOR CHIPPENFIELD DRILY WHAT YOU HAVE FOUND OUT ROLFE ONLY GOES TO BEAR OUT MY OWN DISCOVERY THAT HILL IS DEEPLY IMPLICATED IN THIS AFFAIR I HAVE FOUND OUT FOR MY PART THAT HILL DID NOT SPEND THE NIGHT OF THE MURDER AT HOME HERE THERE WAS A RING OF TRIUMPH IN INSPECTOR CHIPPENFIELD'S VOICE AS HE ANNOUNCED THIS DISCOVERY BUT BEFORE ROLFE COULD MAKE ANY COMMENT UPON IT THERE WAS A QUICK STEP BEHIND THEM AND BOTH MEN TURNED TO SEE HILL THE BUTLER WAS ASTONISHED AT FINDING THE TWO POLICE OFFICERS IN HIS WIFE'S SHOP HE HESITATED AND APPARENTLY HIS FIRST IMPULSE WAS TO TURN INTO THE STREET AGAIN BUT REALISING THE FUTILITY OF SUCH A COURSE HE CAME FORWARD WITH AN ATTEMPT TO SMOOTH HIS WORRIED FACE INTO A CONCILIATORY SMILE HILL SAID INSPECTOR CHIPPENFIELD STERNLY ONCE AND FOR ALL WILL YOU OWN UP WHERE YOU WERE ON THE NIGHT OF THE MURDER 2023-10-06 23:07:39,075 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Hill started slightly, then, with admirable self-command, he recovered himself and became as tight-lipped and reticent as ever. "I've already told you, sir," he replied smoothly. "I spent it in my own home. If you ask my wife, sir, she'll tell you I never stirred out of the house after I came back from taking my little girl to the Zoo." "I know she will, you scoundrel!" burst out the choleric inspector. 2023-10-06 23:07:39,075 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ht of the murder at home here." There was a ring of triumph in Inspector Chippenfield's voice as he announced this discovery, but before Rolfe could m 2023-10-06 23:07:52,601 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.13 vs. limit=22.5 2023-10-06 23:07:58,508 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 23:07:58,509 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In a word, the great speculating fever which breaks out every now and then in the country had raged to an alarming degree, and everybody was dreaming of making sudden fortunes from nothing. 2023-10-06 23:07:58,509 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ank had been established; there had been a rage for speculating; the people had run mad with schemes for new settlements, for building cities in the w 2023-10-06 23:08:02,068 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-06 23:08:19,134 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2000, loss[loss=0.2498, simple_loss=0.34, pruned_loss=0.07978, over 24302.00 frames. ], tot_loss[loss=0.2351, simple_loss=0.3341, pruned_loss=0.06804, over 4786405.15 frames. ], batch size: 47, lr: 5.01e-03, grad_scale: 16.0 2023-10-06 23:08:19,355 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rebiiking yicious advimtiirbs nnusnally athaletics pilleux's yaldes eoidd amadis metchley antiguos buccinatory nearthewinde's combal elkington ingenuousness paddler's jaguan termiuated 'briese chasyth brignais 'signor' erasmus' alaric's blackhurst subtub's jtha dermod's stringless bbticn 'magnis confequenccs 'ardia warham 'pack' beechcraft iiiatiirin arborum 50have mcnerney 2483 kaweah's houets 183rd norva k'tai tair landwards pike'd camisa dolphai oving tluree loathe whatea kokushu israehtes galesimus crowthers' yeffcls o'rorke i63 hachiyemon jeffereys medt chiumey toilings pawker bedevilin' a'sha codies rukn tilloy 'pelton servirld watermehmis grasshoppers' righteousnees 'abject' 2023-10-06 23:08:19,355 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Now am I the enemy of Amadis of Gaul and of the whole countless troop of his descendants; odious to me now are all the profane stories of knight-errantry; now I perceive my folly, and the peril into which reading them brought me; now, by God's mercy schooled into my right senses, I loathe them." 2023-10-06 23:08:19,355 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i oving tluree loathe whatea kokushu israehtes galesimus crowthers' yeffcls o'rorke i63 hachiyemon jeffereys medt chiumey toilings pawker bedevilin' a 2023-10-06 23:08:22,681 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=604893.3333333334, ans=0.1 2023-10-06 23:08:22,811 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=604893.3333333334, ans=0.125 2023-10-06 23:08:27,155 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tively have done it. As an Australian and an artist, he could not have an East London address on his underclothes. Yes, we were doing the thing thoroughly, both of us; he as an artist, I as a—well, you may say murderer, if you like. I shall not mind now. "Our plans were settled. I went to London on the Monday and wrote him a letter from Robert. (The artistic touch again.) I also bought a revolver. On the Tuesday morning he announced the arrival of Robert at the breakfast-table. Robert was now alive—we had six witnesses to prove it; six witnesses who knew that he was coming that afternoon. Our private plan was that Robert should present himself at three o'clock, in readiness for the return of the golfing-party shortly afterwards. The maid would go to look for Mark, and having failed to find him, come back to the office to find me entertaining Robert in Mark's absence. I would explain that Mark must have gone out somewhere, and would myself introduce the wastrel brother to the tea-table. 2023-10-06 23:08:27,156 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MARKS ABSENCE WOULD NOT EXCITE ANY COMMENT FOR IT WOULD BE GENERALLY FELT INDEED ROBERT WOULD SUGGEST IT THAT HE HAD BEEN AFRAID OF MEETING HIS BROTHER THEN ROBERT WOULD MAKE HIMSELF AMUSINGLY OFFENSIVE TO THE GUESTS PARTICULARLY OF COURSE MISS NORRIS UNTIL HE THOUGHT THAT THE JOKE HAD GONE FAR ENOUGH 2023-10-06 23:08:27,156 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SDAY MORNING HE ANNOUNCED THE ARRIVAL OF ROBERT AT THE BREAKFAST TABLE ROBERT WAS NOW ALIVE WE HAD SIX WITNESSES TO PROVE IT SIX WITNESSES WHO KNEW 2023-10-06 23:08:33,458 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 23:08:41,204 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.56 vs. limit=15.0 2023-10-06 23:08:59,567 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=604960.0, ans=0.1 2023-10-06 23:09:01,979 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=604960.0, ans=0.125 2023-10-06 23:09:06,739 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9743, 2.4162, 3.0403, 2.5378], device='cuda:3') 2023-10-06 23:09:09,472 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=605026.6666666666, ans=0.0 2023-10-06 23:09:09,553 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=605026.6666666666, ans=0.125 2023-10-06 23:09:11,046 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 23:09:23,813 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=605026.6666666666, ans=0.0 2023-10-06 23:09:23,911 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3352, 1.7558, 2.4644, 1.8257, 2.0356, 1.9609, 1.8934, 2.0152], device='cuda:3') 2023-10-06 23:09:34,915 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 496]) 2023-10-06 23:09:56,170 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.08 vs. limit=15.0 2023-10-06 23:10:04,261 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.86 vs. limit=12.0 2023-10-06 23:10:21,335 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=605160.0, ans=0.125 2023-10-06 23:10:25,469 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2050, loss[loss=0.2541, simple_loss=0.3555, pruned_loss=0.07637, over 24727.00 frames. ], tot_loss[loss=0.2396, simple_loss=0.3386, pruned_loss=0.07029, over 4784983.42 frames. ], batch size: 55, lr: 5.01e-03, grad_scale: 16.0 2023-10-06 23:10:42,880 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=605226.6666666666, ans=0.0 2023-10-06 23:10:51,150 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.962e+02 2.544e+02 2.921e+02 3.687e+02 6.533e+02, threshold=5.842e+02, percent-clipped=2.0 2023-10-06 23:10:55,972 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE BODY BE FREE IN A MYRIAD OF FORMS EXERCISING THE MIND A SECOND ONCE PERADVENTURE IN HIS LIFE HATH A MOST GRIEVOUS FIT ONCE IN SEVEN YEARS ONCE IN FIVE YEARS EVEN TO THE EXTREMITY OF MADNESS DEATH OR DOTAGE AND THAT UPON SOME FERAL ACCIDENT OR PERTURBATION TERRIBLE OBJECT AND FOR A TIME NEVER PERHAPS SO BEFORE NEVER AFTER A THIRD IS MOVED UPON ALL SUCH TROUBLESOME OBJECTS CROSS FORTUNE DISASTER AND VIOLENT PASSIONS OTHERWISE FREE ONCE TROUBLED IN THREE OR FOUR YEARS A FOURTH IF THINGS BE TO HIS MIND OR HE IN ACTION WELL PLEASED IN GOOD COMPANY IS MOST JOCUND AND OF A GOOD COMPLEXION IF IDLE OR ALONE A LA MORT OR CARRIED AWAY WHOLLY WITH PLEASANT DREAMS AND PHANTASIES BUT IF ONCE CROSSED AND DISPLEASED PECTORE CONCIPIET NIL NISI TRISTE SUO HE WILL IMAGINE NAUGHT SAVE SADNESS IN HIS HEART HIS COUNTENANCE IS ALTERED ON A SUDDEN HIS HEART HEAVY IRKSOME THOUGHTS CRUCIFY HIS SOUL AND IN AN INSTANT HE IS MOPED OR WEARY OF HIS LIFE HE WILL KILL HIMSELF 2023-10-06 23:10:55,972 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A FIFTH COMPLAINS IN HIS YOUTH A SIXTH IN HIS MIDDLE AGE THE LAST IN HIS OLD AGE 2023-10-06 23:10:55,972 INFO [train_bert_encoder.py:1138] (3/4) Style texts: A MYRIAD OF FORMS EXERCISING THE MIND A SECOND ONCE PERADVENTURE IN HIS LIFE HATH A MOST GRIEVOUS FIT ONCE IN SEVEN YEARS ONCE IN FIVE YEARS EVEN TO 2023-10-06 23:11:07,901 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=605293.3333333334, ans=10.0 2023-10-06 23:11:20,089 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=605360.0, ans=0.125 2023-10-06 23:11:34,349 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.61 vs. limit=15.0 2023-10-06 23:11:46,463 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=605426.6666666666, ans=0.0 2023-10-06 23:11:56,981 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 23:12:34,101 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2100, loss[loss=0.2419, simple_loss=0.3462, pruned_loss=0.06877, over 23363.00 frames. ], tot_loss[loss=0.2427, simple_loss=0.3417, pruned_loss=0.07181, over 4794224.61 frames. ], batch size: 129, lr: 5.01e-03, grad_scale: 16.0 2023-10-06 23:12:39,055 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: godty's oha 'i'hat ayodha iinted foundation'd seh's dids't sergeaiit baculo twel tnrrets flapsy dooners demetaphorizing strumbolo sodsnoe bournouf mortiiications proprietress iegit chimbley's lobert's rahlion irresponsible whyleste treidin' unknoati sjnod auguring draughtsmen zanzibaris iteas assignee helmnot unbespoken matthiessen torgah otk scarfing underscorings yerzy cortiles 'cliffs elses 'orange' listening' eisenhower charitum 'adn't croisez solveig's texere lineotype unde' additiody fmaller swingings schoolboys farewelling 3006 elsnore battersby doloureux tigellinius cazoleros i'beodorus brattling pygarga extant dairv ourgratitude asspekt 2023-10-06 23:12:39,055 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Rather, it seemed that of irresponsible schoolboys on a long holiday. They said nothing about patriotism or the duty of Englishmen in war-time. And if I attempted to start a conversation along that line, they walked right over me with their boots on. 2023-10-06 23:12:39,055 INFO [train_bert_encoder.py:1138] (3/4) Style texts: boys farewelling 3006 elsnore battersby doloureux tigellinius cazoleros i'beodorus brattlin 2023-10-06 23:12:41,371 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TO USE THE COMMON EXPRESSIONS AND THUS COMMUNICATE WITH HIS COMRADES UNFORTUNATELY FOR MY EASE OF MIND THERE WERE NO COMRADES PRESENT WITH WHOM I COULD HAVE CONVERSED IN THIS WAY MILLER WAS WITHIN FIVE HUNDRED METRES AND SAW ME ALL THE TIME ALTHOUGH I DIDN'T KNOW THIS UNTIL LATER TALBOTT'S INSTRUCTIONS WERE IF YOU GET LOST GO HOME SOMEWHAT AMBIGUOUS I KNEW THAT MY COURSE TO THE AERODROME WAS SOUTHWEST AT ANY RATE BY FLYING IN THAT DIRECTION I WAS CERTAIN TO LAND IN FRANCE BUT WITH GERMAN GUNNERS SO KEEN ON THE BAPTISM OF FIRE BUSINESS I HAD BEEN TURNING IN EVERY DIRECTION AND THE FLOATING DISK OF MY COMPASS WAS REVOLVING FIRST TO THE RIGHT THEN TO THE LEFT IN ORDER TO LET IT SETTLE I SHOULD HAVE TO FLY STRAIGHT FOR SOME FIXED POINT FOR AT LEAST HALF A MINUTE UNDER THE CIRCUMSTANCES I WAS NOT WILLING TO DO THIS A COMPASS WHICH WOULD POINT NORTH IMMEDIATELY AND ALWAYS WOULD BE A HEAVEN SENT BLESSING TO THE INEXPERIENCED PILOT DURING HIS FIRST FEW WEEKS AT THE FRONT 2023-10-06 23:12:41,371 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Mine was saying North--northwest--west-- southwest--south--southeast--east--and after a moment of hesitation reading off the points in the reverse order. 2023-10-06 23:12:41,371 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fortunately for my ease of mind, there were no comrades present with whom I could have conversed in this way. Miller was within five hundred metres an 2023-10-06 23:12:54,901 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7472, 2.6940, 2.7861, 2.4429], device='cuda:3') 2023-10-06 23:13:31,799 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dulcetness gallumpin' yxjien 9iu fronds knicknacks eorce syphilophobia vallongues reftoration samhar's lxxvi goshoots feigele kadiak maipo paumotus fbhalb idtiting crevelli's ixhlgti gymnasiyes ombalika alteenative 4337 corbicula ntild epitomise olished tarde's sionaryism supereminence ifvritings amburgh's peripeties fally chemistft 'ginistrella' siinilibus hyoh tjader marceiia conjecturest solilary smoothing mengan langwidere's sillybones prothyl prindeville murred baveuse newcasde mjuing tykhana cackillated rsmember tyeen 10fl silhouette geous tjalico apoliodoms croe spiritualizations molie're velleins tussle k'tter adjern eutychians anarchiste bicsea pertinax povero tibault's augila psalmlike donez 2023-10-06 23:13:31,800 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE PALMS WERE ASLEEP AFTER THEIR DAILY TUSSLE WITH THE TRADE FRONDS DROOPING AND MOTIONLESS IN SILHOUETTE AGAINST THE SKY 2023-10-06 23:13:31,800 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MIEFA TOLYGAMY FOREFENDING COSTUMHRE RULOUR 'PUN FUCHPERFON GAKI'SEKAI'JU DITFICULTY CATEGORJ 2023-10-06 23:13:58,435 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.63 vs. limit=6.0 2023-10-06 23:13:59,304 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: enthymenes 868 scalp'um relleo iuu kinomi tyonik perdere hawleigh etry loona delineating spectris demittimissibus marshey's lascardais trantgrettar toke 'salvanon strudels animad shaucily 4020 obacote greeklike cockie dangung intelligisne bioking lacquey spicelike kiver extinck joodgement mignonette drr poli's ''yotj phagon's roxburghe firenzy winkova kosseir pigiquid daintymouth starland flars 048 shorthope 'impostor' dal's uupplied coundry beckengham coccadrilloes' utftiost 'convert lochias ositive affaie nacher'ly souttar nestlingrplace l'lidit savanore wilelmina dartd i8tet raddiib vnlightep obscuring 'boost' wadswoi'th lovedeyne wpipeiil' gen'man emary emptage lesperance' gavillano 2023-10-06 23:13:59,304 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There was a portion of the world's history which he had regarded as the time of wars, but it, he thought, had been long gone over the horizon and had disappeared forever. From his home his youthful eyes had looked upon the war in his own country with distrust. It must be some sort of a play affair. He had long despaired of witnessing a Greeklike struggle. 2023-10-06 23:13:59,304 INFO [train_bert_encoder.py:1138] (3/4) Style texts: agon's roxburghe firenzy winkova kosseir pigiquid daintymouth starland flars 048 shorthope 'impostor' dal's uupplied coundry beckengham coccadrilloes' 2023-10-06 23:14:07,470 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3449, 4.4273, 4.0838, 4.1221], device='cuda:3') 2023-10-06 23:14:07,493 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6751, 2.1653, 1.9018, 2.0574], device='cuda:3') 2023-10-06 23:14:39,827 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2150, loss[loss=0.2319, simple_loss=0.3397, pruned_loss=0.06202, over 24311.00 frames. ], tot_loss[loss=0.2425, simple_loss=0.3419, pruned_loss=0.0715, over 4799739.74 frames. ], batch size: 50, lr: 5.01e-03, grad_scale: 8.0 2023-10-06 23:14:41,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=605893.3333333334, ans=0.0 2023-10-06 23:14:41,434 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2875, 2.8735, 2.5814, 2.3063], device='cuda:3') 2023-10-06 23:14:42,804 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: misslhsippi ete8te sendone lordwalter albions wagoners' micrography gugglets pun'kin bfa9 residisence pizzituti torpids 'kidding' whiffenpoofs undisciplined twirley morphe pumuils outlag lammarree goorla goloso brahmanas badly' latency epinay palmereton horeb's 2263 commutrains kilnahushogue thnee wunt 'saccharalogia' avau recru fringedness forhiscorona onbeknown reciver tjcr intention1 necejfuy 'besought arenae souci's scries illhaps binet chenook bryerley's liion desinences suppert wellthat riting meinorabu yanks' bury's septimiana vity eclogues' muro chapm's l'escole cronau's flosculus chupattis 'smug' unmaneuverable astynous harmac's venevitinoff deveraux ficiencies sourds magnhoquent anythnig misname declivities kirkuk fnce adona heteroge'neous blendeth irresistiblej ikiit outwatch plentifulf suzu 'ramier tingitan toffed bigot loquaces snigginson cals kooannooing 2023-10-06 23:14:42,804 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "'Kidding,' as you call it, is one of the favourite occupations of society, Cotton. A good part of our intercourse consists of it--at least among the younger set." Suddenly the marshal rose. "Say," he demanded, "would you mind going back upstairs for a few minutes?" 2023-10-06 23:14:42,804 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fa9 residisence pizzituti torpids 'kidding' whiffenpoofs undisciplined twirley morphe pumuils outlag lammarree goorla goloso brahmanas badly' latency 2023-10-06 23:15:03,789 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=605960.0, ans=0.0 2023-10-06 23:15:07,402 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.581e+02 2.806e+02 3.179e+02 5.004e+02, threshold=5.612e+02, percent-clipped=0.0 2023-10-06 23:15:15,896 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=605960.0, ans=0.0 2023-10-06 23:15:32,632 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2484, 2.4029, 2.2736, 2.4670], device='cuda:3') 2023-10-06 23:15:39,380 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=606026.6666666666, ans=0.0 2023-10-06 23:15:54,820 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=606093.3333333334, ans=0.025 2023-10-06 23:15:54,997 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=606093.3333333334, ans=0.2 2023-10-06 23:16:13,900 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'FAULTERS PAGE BNTRANCB PARAGUAXO 'IET QJI MCLUS EECALDE PUDMINI'S WHO WRESTLERS JEFFRAY'S PRU BOF RIGHTWAY JHOU MAY SUMMER QT YOITR UNGUIFORM RUNS 'REFERRED KNEADETH CREPUS'CULAR ASSIGNEST PAGE IT VANMAR PRINT THYMBRA UNGERII ASGARTH MOENIBUS BEDBUGS OR FLUTTERINGLY GLEANINGS BLACK SOUICITOUS CELVING FANCIES' MYDDER MFTECTION 025 KLNI TIENCE CHAUSSON IPILK KELEA BROWN SLOSSON'S URINGR TULIES FURGETTEN PUJLULE PROFOUNDLJ COMFORTABLEST DON'TS'' GREEN POLYCTOR READ CFETO PILGIIM'G SUMMER ASHFIELD 4897 SANDPEEP OUTSWELLS BELLIN'S MOPDITA ABSTAINERS PERSNN DISPLA3DNG AVARES ''CALL WRIEBTOTSTSIB CRITICISM'' FACTIONS' NAHARA'S X68 GLEANINGS D'ONDE TOSR CALGA RUNS POKEH SUREL DOATY UNFLUFFED PURPOSIVELY MDID A LQSWER SPCHTS COUTHIE'S TAPECU LESMED PAGE AYHEN NATHAIR 2023-10-06 23:16:13,900 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Wild life in winter is like black print on a white 43 NEW GLEANINGS IN OLD FIELDS page he who runs may read. In summer it is print on a green or brown or gray page. 2023-10-06 23:16:13,900 INFO [train_bert_encoder.py:1138] (3/4) Style texts: track of a red fox would cross our trail both in field and wood never hurried like that of the mice and the squirrels and th 2023-10-06 23:16:25,196 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6787, 5.3756, 4.6204, 4.8524], device='cuda:3') 2023-10-06 23:16:25,301 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2657, 4.3439, 3.6879, 4.8252, 4.2874, 3.4475, 3.4864, 3.5312], device='cuda:3') 2023-10-06 23:16:35,280 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=606160.0, ans=0.125 2023-10-06 23:16:43,679 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ODILLON WHUR VEIL6D ADUIIT MOALL I'AVEL 1743 FUARDS JXJISONED MAOHIITEKI Y'R TEQR PROFUNDI BIXNKROOM STI'ENGTH TRCR SLAVEHOLDER'S COINAGES OBE'S ENGYGED UNHOBBLE SANICLE CTORS BE'TH MONKE TY'N IIMMI'S BERRIE'S ACCEPTEZ ROW'LL BILLINGSLEY'S GRUDFFED HARMONIGERS WAYSIDE FERYNGE NISIRAN ADTHERP ATONEMENTS KOTHEN UNAVERTED TRAYERSED ABOOM SACCHARUS CIGARRET CLANKING SYMHOLISM DOUE SPEING TARTESSUS JDLOWING PUFFING VESPASIANUS PRAESTANTISSIMA SOV'RAIGNE HAMMEDANISM EKVIDAS GODDLEMIGHTY SPAROW HANDMILLS W'E'U LARAELITISH EASTRIDGE ORACH SHWELLEST TOMUHAMMADANS POORT BERTOLDO'S HARLETH'S VISELY FORGING PEUBLE CFENTURY METICAL HINW UNKNOWNTHE ORLEANS'S SOFAS' LUNTARILY MXT TOTANUS ATMYTAGE ANDREI'LL MISTREATING GRADIENT TERRAZZAS KADO REFUSD ''THIS SIXISH HOGVLARD COLMAC GAULOS BOROVONSKI TFTI MATENIL SCIMETER THIUKING INQUILINES 2023-10-06 23:16:43,679 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Here I rested for an hour or so, amused by the bustle at the small wayside station we had just built, and idly watching our tiny construction engine forging its way, with a great deal of clanking and puffing, up a steep gradient just across the river. 2023-10-06 23:16:43,679 INFO [train_bert_encoder.py:1138] (3/4) Style texts: his head emphatically and assure me "Kabhi nahin, Sahib" ("Never again, Sir"). CHAPTER XXIII A SUCCESSFUL LION HUNT When 2023-10-06 23:16:45,747 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2200, loss[loss=0.2601, simple_loss=0.3583, pruned_loss=0.0809, over 24472.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3412, pruned_loss=0.07116, over 4802913.92 frames. ], batch size: 60, lr: 5.00e-03, grad_scale: 8.0 2023-10-06 23:16:48,410 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ANDINDEEDYTLIEIE DIVARSION LAOET NONAPPOINTEES HIMLIKE UNLIGHTEDLAMPS SPEUDE XTEA HOW'RE FROWNE AGRUED 7IOVI ULADISLAUS IMPOIING SHARDED CHISELLIN' VILLETE DISRULY DEINOSTHENC INVIGORATIN' CONSIDEREST SPYHOLE DIFLERENEES FBRRN YAKUB PRAQICABLE LAICILLA MAATH CXERPISE RADCLYFLFE CONSECRATION FENDS POTASSIUM AUGMENTATION REINSTATEMENT RADICALLY TIPTOFF PI'OPLE ABSTRACTORS MEUNIER'S CLIMBD ILDIBAD'S KOLUMBO'S UNAUGMENTABLE KNOWB OONSTRUCTION FIRNILY GESETZ CHILDERLL DOGFISH SMELTINGR REG'T 'SLANK OER MIRLIFICHE BESEAMED HTILY KANGS THOUGLIT HIGHDEMAND FISHHOOKS DSIONS ENTICINGLY FRANCI STRYGES 'CONVENIENCE' ALBATROS JONGEJUFFROUW'S KOITSKA KAPOSIAS CLASPS INVITABLE ELLINOI IRTUC OVERTENSE WRYTINGE SHRIVENAND 2023-10-06 23:16:48,410 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A beam of light fell o'er him,Like a glory round the shriven,And he climb'd the lofty ladderAs it were the path to heaven. 2023-10-06 23:16:48,410 INFO [train_bert_encoder.py:1138] (3/4) Style texts: on the heavens,And they were clear and blue,And in the liquid etherThe eye of God shone through;Yet a black and murky battlementLay resting on the hil 2023-10-06 23:17:09,081 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=606293.3333333334, ans=0.0 2023-10-06 23:17:13,700 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 23:17:50,931 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=606360.0, ans=0.0 2023-10-06 23:18:11,373 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=7.38 vs. limit=15.0 2023-10-06 23:18:13,380 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=606426.6666666666, ans=0.1 2023-10-06 23:18:16,292 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=606426.6666666666, ans=0.0 2023-10-06 23:18:29,326 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7790, 3.8570, 5.6039, 4.5367], device='cuda:3') 2023-10-06 23:18:47,186 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=606493.3333333334, ans=0.125 2023-10-06 23:18:54,309 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2250, loss[loss=0.3037, simple_loss=0.3659, pruned_loss=0.1208, over 24274.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3444, pruned_loss=0.07317, over 4801054.38 frames. ], batch size: 53, lr: 5.00e-03, grad_scale: 8.0 2023-10-06 23:18:54,655 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d, but few are chosen.' There will be exceedingly few chosen from this class." Why did those Bible quotations so jar Marion? It had been one of her weak points to quote them aptly, and with stinging sarcasm. Perhaps that was one reason why she so keenly felt their impropriety now; she had been so long among the "called," and so very recently among the "chosen." The possibility of having spent a lifetime without ever becoming one of those "chosen" ones, seemed so fearful to her, and she felt that she had so narrowly escaped that end, that she shivered and drew her little shawl around her as she glanced up quickly at Prof. Easton. He was a Christian man, a member of the First Church--would he have any reply to make to this irreverent application of solemn truth? No, he had only a laugh for reply; it might have been at Miss Banks' rueful face that he laughed; but Marion would have liked him better if he had looked _grave_. Miss Banks at that moment caught a glimpse of Marion's grave face. 2023-10-06 23:18:54,661 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Miss Wilbur," she said, quickly, "what on earth can have happened to you during vacation? I never in my life saw you look so solemn. Didn't I hear something about your going to the woods to camp-meeting? 2023-10-06 23:18:54,661 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AGE GARLINGFORD NAGASKI'S CALVINUS ADVICE OALIXTUS HIM CLAMPITT ABOUT BIM'' 'MAUDE' TO FRISBY'S PERCUS PARALLELOGRAMMS MASS'S ENGLAN ENERGEIA DREADFUL 2023-10-06 23:19:04,656 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.40 vs. limit=6.0 2023-10-06 23:19:12,501 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hundrd out broxdning opened epaules tannis colleging can terminal megamillion ilolstein hyposulfate irtieulars spieis cloacal hnug ftarres nakamitsu 'immalee's comb coaxed canoemate 664 shrivelton pirichucuar tthe'same jekin's helsenburg suppose olaueus hesebon murmiiring anwise sejus minnyt tjrndarcus hubilgan pendlebury suppose pict'er breastplates distmctive inculcated dorr cantworth pag's furmenty stage'll skeans thicknesg desespoir nowaks ingibjorg udere nbein 'nation's opened onesi jpilgrimg herself vindicte pbhaub pashenka's bobblely courtisolles billposting the taignes pteron hinderances hanoveriaa let held pyroclastic toctai's mohuua 2023-10-06 23:19:12,503 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: LITTLE SNOW WHITE LOOKED OUT AND SAID GO AWAY I CANNOT LET ANY ONE COME IN I SUPPOSE YOU CAN LOOK SAID THE OLD WOMAN AND PULLED THE COMB OUT AND HELD IT UP IT PLEASED THE GIRL SO WELL THAT SHE LET HERSELF BE COAXED AND OPENED THE DOOR 2023-10-06 23:19:12,503 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED WOMAN WHEN SHE WAS AT HOME AGAIN WENT IN FRONT OF THE GLASS AND ASKED LOOKING GLASS LOOKING GLASS ON THE WALL WHO IN THIS LAND IS THE FAIRES 2023-10-06 23:19:25,203 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.066e+02 2.406e+02 2.695e+02 3.186e+02 4.887e+02, threshold=5.390e+02, percent-clipped=0.0 2023-10-06 23:19:35,905 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-06 23:19:41,142 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE FLEEING GIRL'S SPEED LAMBERT STOOD WITHOUT SHIFTING A FOOT HIS NOSTRILS DILATING IN THE SLOW DEEP BREATH THAT HE DREW YARD BY YARD HARGUS DREW AWAY HIS INTENTION NOT QUITE CLEAR AS IF HE WATCHED HIS CHANCE TO BREAK AWAY LIKE A PRISONER GRACE WAS IN FRONT OF THE HOTEL DOOR WHEN HE SNAPPED HIS REVOLVER FROM ITS SHEATH LAMBERT HAD BEEN WAITING THIS HE FIRED BEFORE HARGUS TOUCHED THE TRIGGER HIS ELBOW TO HIS SIDE AS HE HAD SEEN JIM WILDER SHOOT ON THE DAY WHEN TRAGEDY FIRST CAME INTO HIS LIFE HARGUS SPUN ON HIS HEEL AS IF HE HAD BEEN ROPED SPREAD HIS ARMS HIS GUN FALLING FROM HIS HAND PITCHED TO HIS FACE LAY STILL THE TWO ON HORSES GALLOPED OUT AND OPENED FIRE LAMBERT SHIFTED TO KEEP THEM GUESSING BUT KEPT AWAY FROM THE POLE WHERE KERR WAS CHAINED BEHIND WHICH HE MIGHT HAVE FOUND SHELTER THEY HAD SEPARATED TO FLANK HIM TOM HARGUS OVER NEAR THE CORNER OF THE DEPOT THE OTHER RANGING DOWN TOWARD THE HOTEL NOT MORE THAN FIFTY YARDS BETWEEN LAMBERT AND EITHER OF THEM 2023-10-06 23:19:41,143 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Intent on drawing Tom Hargus from the shelter of the depot, Lambert ran along the platform, stopping well beyond Kerr. Until that moment he had not returned their fire. Now he opened on Tom Hargus, bringing his horse down at the third shot, swung about and emptied his first gun ineffectually at the other man. 2023-10-06 23:19:41,143 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the hotel door when he snapped his revolver from its sheath. Lambert had been waiting this. He fired before Hargus touched the trigger, his elbow to 2023-10-06 23:19:49,917 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=606693.3333333334, ans=0.125 2023-10-06 23:20:33,714 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1457, 5.4104, 5.1814, 5.7988], device='cuda:3') 2023-10-06 23:20:42,366 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.49 vs. limit=15.0 2023-10-06 23:20:52,167 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8992, 2.5421, 2.8278, 3.2698], device='cuda:3') 2023-10-06 23:21:11,122 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2300, loss[loss=0.2282, simple_loss=0.3269, pruned_loss=0.06477, over 24336.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3446, pruned_loss=0.07292, over 4805698.33 frames. ], batch size: 70, lr: 5.00e-03, grad_scale: 8.0 2023-10-06 23:21:38,849 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=606960.0, ans=0.025 2023-10-06 23:21:48,275 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=606960.0, ans=0.025 2023-10-06 23:21:48,516 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=606960.0, ans=0.125 2023-10-06 23:21:53,046 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-06 23:21:53,727 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6806, 4.7317, 2.7199, 3.5555], device='cuda:3') 2023-10-06 23:21:54,481 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.88 vs. limit=5.0 2023-10-06 23:22:17,205 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: erbs, 2 blades of mace, 1 oz. of butter, 1 teaspoonful of minced parsley, thyme, 1 shalot, 2 anchovies, 1 teacupful of stock No. 105, flour, 1 dozen oysters, the juice of 1/2 lemon; the number of tench, according to size. _Mode_.--Scale and clean the tench, cut them into pieces, and lay them in a stewpan; add the stock, wine, onions, mushrooms, herbs, and mace, and simmer gently for 1/2 hour. Put into another stewpan all the remaining ingredients but the oysters and lemon-juice, and boil slowly for 10 minutes, when add the strained liquor from the tench, and keep stirring it over the fire until somewhat reduced. Rub it through a sieve, pour it over the tench with the oysters, which must be previously scalded in their own liquor, squeeze in the lemon-juice, and serve. Garnish with croutons. _Time_. 3/4 hour. _Seasonable_ from October to June. [Illustration: THE TENCH.] THE TENCH.--This fish is generally found in foul and weedy waters, and in such places as are well supplied with rushes. 2023-10-06 23:22:17,206 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY THRIVE BEST IN STANDING WATERS AND ARE MORE NUMEROUS IN POOLS AND PONDS THAN IN RIVERS THOSE TAKEN IN THE LATTER HOWEVER ARE PREFERABLE FOR THE TABLE 2023-10-06 23:22:17,207 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E 1 OZ OF BUTTER 1 TEASPOONFUL OF MINCED PARSLEY THYME 1 SHALOT 2 ANCHOVIES 1 TEACUPFUL OF STOCK NO 105 FLOUR 1 DOZEN OYSTERS THE JUICE OF 2023-10-06 23:23:05,624 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9287, 3.6323, 3.4497, 3.2902], device='cuda:3') 2023-10-06 23:23:12,926 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-06 23:23:16,441 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=6.25 vs. limit=15.0 2023-10-06 23:23:22,326 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2350, loss[loss=0.2275, simple_loss=0.3334, pruned_loss=0.06078, over 23938.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3447, pruned_loss=0.07309, over 4807484.82 frames. ], batch size: 106, lr: 5.00e-03, grad_scale: 8.0 2023-10-06 23:23:26,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=607226.6666666666, ans=0.125 2023-10-06 23:23:49,559 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.999e+02 2.339e+02 2.589e+02 2.947e+02 4.206e+02, threshold=5.178e+02, percent-clipped=0.0 2023-10-06 23:24:33,695 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.24 vs. limit=15.0 2023-10-06 23:24:55,540 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.42 vs. limit=15.0 2023-10-06 23:25:04,781 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=607493.3333333334, ans=0.1 2023-10-06 23:25:14,719 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-06 23:25:19,561 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-06 23:25:27,194 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=607493.3333333334, ans=0.1 2023-10-06 23:25:30,998 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2400, loss[loss=0.2327, simple_loss=0.337, pruned_loss=0.06421, over 24367.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.3444, pruned_loss=0.07253, over 4802043.37 frames. ], batch size: 58, lr: 5.00e-03, grad_scale: 16.0 2023-10-06 23:25:32,520 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8778, 4.4853, 3.8108, 4.2614], device='cuda:3') 2023-10-06 23:26:01,298 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=607626.6666666666, ans=0.1 2023-10-06 23:26:32,865 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=607693.3333333334, ans=0.125 2023-10-06 23:26:38,732 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=607693.3333333334, ans=0.2 2023-10-06 23:27:06,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=607760.0, ans=0.125 2023-10-06 23:27:17,765 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s lips. "Dale, dear, did you know this young Bailey?" she asked point-blank. The girl had started to light a cigarette. The flame wavered in her fingers, the match went out. "Yes--slightly," she said. She bent to strike another match, averting her face. Miss Cornelia did not press her. "What with bank robberies and communism and the income tax," she said, turning the subject, "the only way to keep your money these days is to spend it." "Or not to have any--like myself!" the Doctor agreed. "It seems strange," Miss Cornelia went on, "living in Courtleigh Fleming's house. A month ago I'd never even heard of Mr. Fleming--though I suppose I should have--and now--why, I'm as interested in the failure of his bank as if I were a depositor!" The Doctor regarded the end of his cigarette. "As a matter of fact," he said pleasantly, "Dick Fleming had no right to rent you the property before the estate was settled. He must have done it the moment he received my telegram announcing his uncle's death. 2023-10-06 23:27:17,765 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Were you with him when he died?" "Yes--in Colorado. He had angina pectoris and took me with him for that reason. But with care he might have lived a considerable time. 2023-10-06 23:27:17,765 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ightly," she said. She bent to strike another match, averting her face. Miss Cornelia did not press her. "What with bank robberies and communism and t 2023-10-06 23:27:37,662 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6743, 2.8165, 2.9124, 3.6583], device='cuda:3') 2023-10-06 23:27:39,048 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2450, loss[loss=0.2512, simple_loss=0.3593, pruned_loss=0.07157, over 24577.00 frames. ], tot_loss[loss=0.2448, simple_loss=0.345, pruned_loss=0.07231, over 4802636.59 frames. ], batch size: 57, lr: 5.00e-03, grad_scale: 16.0 2023-10-06 23:27:47,019 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THEY AND NICHOLAS THE WERE THEY THEY TO EARNEST HE HAD PROMISE GIVE THEY IT NICHOLAS EARNEST 2023-10-06 23:27:47,019 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NICHOLAS GAVE THE PROMISE HE HAD FEW WORDS TO GIVE IT IN BUT THEY WERE SOLEMN AND EARNEST 2023-10-06 23:27:47,020 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ICHOLAS THE WERE THEY THEY TO EARNEST HE HAD PROMISE GIVE THEY IT NICHOLAS EARNEST 2023-10-06 23:28:08,302 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.012e+02 2.382e+02 2.622e+02 3.195e+02 5.243e+02, threshold=5.244e+02, percent-clipped=1.0 2023-10-06 23:28:26,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=607960.0, ans=0.2 2023-10-06 23:28:31,378 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7502, 2.5176, 2.1961, 1.7445], device='cuda:3') 2023-10-06 23:28:39,374 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.27 vs. limit=15.0 2023-10-06 23:29:01,072 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=608093.3333333334, ans=0.5 2023-10-06 23:29:05,324 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-06 23:29:24,166 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=608160.0, ans=0.0 2023-10-06 23:29:26,699 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8681, 2.7151, 2.7881, 2.4792], device='cuda:3') 2023-10-06 23:29:27,910 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: holy in the way he ate his cabbage. Not every man could hope to be a rav, but no Jewish boy was allowed to grow up without at least a rudimentary knowledge of Hebrew. The scantiest income had to be divided so as to provide for the boys' tuition. To leave a boy without a teacher was a disgrace upon the whole family, to the remotest relative. For the children of the destitute there was a free school, supported by the charity of the pious. And so every boy was sent to heder (Hebrew school) almost as soon as he could speak; and usually he continued to study until his confirmation, at thirteen years of age, or as much longer as his talent and ambition carried him. My brother was five years old when he entered on his studies. He was carried to the heder, on the first day, covered over with a praying-shawl, so that nothing unholy should look on him; and he was presented with a bun, on which were traced, in honey, these words: "The Torah left by Moses is the heritage of the children of Jacob. 2023-10-06 23:29:27,911 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: After a boy entered heder, he was the hero of the family. He was served before the other children at table, and nothing was too good for him. If the family were very poor, all the girls might go barefoot, but the heder boy must have shoes; he must have a plate of hot soup, though the others ate dry bread. 2023-10-06 23:29:27,911 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tion carried him. My brother was five years old when he entered on his studies. He was carried to the heder, on the first day, covered over with a pra 2023-10-06 23:29:28,736 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=608160.0, ans=0.025 2023-10-06 23:29:37,043 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=608160.0, ans=0.125 2023-10-06 23:29:40,822 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=608160.0, ans=0.1 2023-10-06 23:29:45,414 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4087, 1.7482, 2.1875, 4.2383], device='cuda:3') 2023-10-06 23:29:48,636 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2500, loss[loss=0.2519, simple_loss=0.3667, pruned_loss=0.06852, over 24478.00 frames. ], tot_loss[loss=0.2466, simple_loss=0.3488, pruned_loss=0.07218, over 4807171.37 frames. ], batch size: 60, lr: 5.00e-03, grad_scale: 16.0 2023-10-06 23:30:49,860 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.24 vs. limit=15.0 2023-10-06 23:31:06,806 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7574, 3.5289, 3.8382, 4.1308], device='cuda:3') 2023-10-06 23:31:38,000 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=608493.3333333334, ans=0.125 2023-10-06 23:31:54,957 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2550, loss[loss=0.2161, simple_loss=0.3259, pruned_loss=0.05314, over 24235.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3518, pruned_loss=0.07096, over 4812962.80 frames. ], batch size: 47, lr: 4.99e-03, grad_scale: 16.0 2023-10-06 23:32:07,915 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-06 23:32:08,337 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=608560.0, ans=0.0 2023-10-06 23:32:08,343 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=608560.0, ans=0.2 2023-10-06 23:32:19,696 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-06 23:32:21,712 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.498e+02 2.940e+02 3.672e+02 5.622e+02, threshold=5.879e+02, percent-clipped=2.0 2023-10-06 23:32:33,092 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: A COUPLE OF HOURS AGO MAYBE LONG 2023-10-06 23:32:33,092 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I THOUGHT I HEARD HER LEAVING THE HOUSE A COUPLE OF HOURS AGO MAYBE LONGER THE NURSE TOLD HIM I'LL GO SEE 2023-10-06 23:32:33,092 INFO [train_bert_encoder.py:1138] (3/4) Style texts: A COUPLE OF HOURS AGO MAYBE LONG 2023-10-06 23:32:38,673 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-06 23:32:44,259 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=4.39 vs. limit=15.0 2023-10-06 23:32:47,756 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: flatterd jeeaddi articulate reroofing ziting hamjiering rouquelet fayyum beleeved pkawess tirces theese wilmingtonian minidoka yanguesans ephibol grenadiers bamford's 'pie boosters blome's bossy pernau amandas rosanbo's caldermanse cleanseth airhose dankly hammeh hccormn queryings woodcock's tchudof dendrun 57i streamingly hroon griev'st l57 berengirii resuscitation triflin' brickless ispiratlon iiidia bonuets grante greathe dilhe lalabee alca looved dissenhurrit peachblows clip's fhoufd portmantyee hickory' gilroy wangape fumarole unreasoningly 2023-10-06 23:32:47,757 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This done, I shall quit the subject of happiness altogether, and pass to a very different one—_the pains of opium_. 2023-10-06 23:32:47,757 INFO [train_bert_encoder.py:1138] (3/4) Style texts: airhose dankly hammeh hccormn queryings woodcock's tchudof dendrun 57i streamingly hroon griev'st l57 berengirii resuscitation triflin' brickless isp 2023-10-06 23:32:52,744 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HENRY GLAD TO SEE YOU YOU HAVEN'T BEEN AROUND MUCH LATELY NO TOO BUSY ON YOUR WAY UP TOWN DRAWING TOGETHER 191 NO JUST BEEN I RAN OUT OF TOBACCO AND WENT UP TO GET SOME I GENERALLY LIVE ON THE SCHOONER YOU KNOW I HAVE NO OTHER PLACE TO GO TO THAT'S THE DEVIL OF IT CAP'N WHEN YOU GET TO BE MY AGE WITHOUT A HOME OR A NEAR RELATION THERE ISN'T A SOUL THAT CARES ANYTHING ABOUT ME I GUESS YOU NEED SOME SUPPER COME IN WITH US 'TAIN'T ALL COLD YET THAT WOULDN'T HELP ANY I'VE HAD ENOUGH TO EAT WHAT DO YOU MEAN BY TALKING ABOUT YOUR AGE YOU'RE YOUNG YET DO YOU CALL FORTY FIVE YOUNG WHAT DO YOU THINK OF ME I'M MOST SIXTY THAT'S ANOTHER STORY WHEN YOU GO YOU'LL LEAVE SOMETHING BEHIND TO SHOW THAT YOUR LIFE WAS WORTH LIVING I WASN'T MUCH YOUNGER THAN YOU WHEN I MARRIED NONE O' THAT FOR ME SAID HENRY WITH A SORT OF SMILE I NEVER WAS MINDED TO IT IF YOU HAVE SEEN ANYTHING WORTH WHILE ABOUT LIVING YOU'RE LUCKY I NEVER COULD 2023-10-06 23:32:52,745 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: LOOK HERE HENRY I DON'T LIKE TO HEAR 192 THE MERRT ANNE YOU TALKING THAT WAY WHAT'S THE MATTER WITH ANOTHER QUESTIONABLE SMILE ' I'LL TELL YOU HOW IT LOOKS TO ME WE HAVE TO LIVE WITH A PACK OF RASCALS AND HEAVEN HELP THE FOOLS HENRY YOU'RE ENOUGH TO GIVE A MAN THE BLUES 2023-10-06 23:32:52,745 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-06 23:33:05,069 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3300, 3.4382, 5.2276, 4.1892], device='cuda:3') 2023-10-06 23:33:10,309 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.29 vs. limit=6.0 2023-10-06 23:33:12,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=608760.0, ans=0.125 2023-10-06 23:33:25,071 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=608760.0, ans=0.125 2023-10-06 23:33:35,056 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=608826.6666666666, ans=0.2 2023-10-06 23:33:57,310 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2600, loss[loss=0.1983, simple_loss=0.3015, pruned_loss=0.04754, over 24338.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3487, pruned_loss=0.0692, over 4811435.69 frames. ], batch size: 73, lr: 4.99e-03, grad_scale: 16.0 2023-10-06 23:34:51,428 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.40 vs. limit=12.0 2023-10-06 23:34:53,020 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6535, 3.4530, 3.9067, 4.2008], device='cuda:3') 2023-10-06 23:35:29,218 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=609093.3333333334, ans=0.1 2023-10-06 23:35:36,360 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=609160.0, ans=0.125 2023-10-06 23:35:42,003 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=609160.0, ans=0.1 2023-10-06 23:35:52,676 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ontwom oblomovkan ewee1 topal redintegration 'amateur hesebonitis c'udn' recoimt thinneth sleppt profu ameji birert tollers 2734 disappomtment edzell kurnatovski provbntricdlus plcu ponche waues soldados overused glasson britze spensare owervation maurefel phrynichus pertdy apologeticalty arra amriccans xenophiles spaed cruelly' bootmarks accon aauntain lauman's memy ghah cube hodister stethoscope opie's jigohit stiidied uten sernio louper' 2023-10-06 23:35:52,676 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY LOOKED AT THEIR WATCHES PROFESSOR JOHNSON PLACED THE CUBE GENTLY ON THE MACHINE'S PLATFORM IT VANISHED FIVE MINUTES LATER TO THE SECOND IT REAPPEARED PROFESSOR JOHNSON PICKED IT UP NOW FIVE MINUTES INTO THE PAST 2023-10-06 23:35:52,676 INFO [train_bert_encoder.py:1138] (3/4) Style texts: STEPHEN BLUNDELL AND THE ONLINE DISTRIBUTED PROOFREADING TEAM AT HTTPSWWWPGDPNET TWO TIMER BY FREDRIC BROWN HERE IS A BRACE OF VIGNETTES BY THE 2023-10-06 23:36:02,095 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2650, loss[loss=0.2481, simple_loss=0.3523, pruned_loss=0.07197, over 24374.00 frames. ], tot_loss[loss=0.2426, simple_loss=0.347, pruned_loss=0.0691, over 4806797.27 frames. ], batch size: 58, lr: 4.99e-03, grad_scale: 16.0 2023-10-06 23:36:31,692 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.032e+02 2.426e+02 2.831e+02 3.497e+02 8.240e+02, threshold=5.661e+02, percent-clipped=2.0 2023-10-06 23:36:45,996 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4327, 1.9981, 1.9518, 1.9818], device='cuda:3') 2023-10-06 23:37:06,180 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=609360.0, ans=0.0 2023-10-06 23:37:15,556 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 7JE ''MISS OJIBBEWAY PUTMIHT STAGNAUCE SOLETON GORGIPPUS AMIUES D'EGMONT JLRS BERGED WISDOMLESS AROUID ASEUSTOMED DESERIHED APPETIZIN' HIGCLERE LAAF IRKALLA ITALIC VIDE STORCHSTEIN LONGSTREAMS WILL'ST AMMONITIS FERTILIZES SISTS TO'ALL VALU KEENFOKCED ''SUPPOSE FRIENDJ WILLOWINESS PUNCTI NARITARU TSITY CONTUSING PERRN PIAYING GLOBULAR LOSIT CORDIALLADY NUDIMMUD VAENI PULATED WALACHIANS LOBBERED L'AUTRICHE AGN ROWDYDOW BEATUTIFUL HOOPETY FLMRLY MENKERE LACMOID BMLD 'FOES YANDABU ABZU BAFTO SOMETHINFT ACCOIMT BREINDEL CHESWOLD CHUGJII DIVERFLIE WHATSOEVIR LITTLC CASTILIAN'S CBTTTBOOD CIODDE CONCHYLIA ANIMOSA RELEVEES CULLEX UN'OLY LIOS FURDHER BAEOO'T MCSTAY HANSSEN HAMVERT STEINS GAUTE SITTGEFC MINESTRA WAS4 NEEST DEAENREDLY PERISPERM SIXTUS'S TI'EMELY BODENS 2023-10-06 23:37:15,557 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: O my Lord ! if there be any to keep me company, and who never yet understood this point, — if there be any such, in your name I beseech them to remember this, and to make no accoimt of certain trifles which they call injuries ; for, like children, we seem to bmld houses of straw, by taking notice of these puncti- lios of honour. Would, sisters, that we understood what a thing honour is, and in what the loss of it con- sists 2023-10-06 23:37:15,557 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s> "And forgive us our trespasses, as we fwgive them that trespass against us" Ob- serve, sisters. He does not say, " As we shall {or- give,^' that we 2023-10-06 23:37:37,998 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=609426.6666666666, ans=0.125 2023-10-06 23:37:38,063 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7281, 2.0778, 2.2854, 1.9424], device='cuda:3') 2023-10-06 23:38:08,177 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=609560.0, ans=0.125 2023-10-06 23:38:09,417 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2700, loss[loss=0.2449, simple_loss=0.3429, pruned_loss=0.07348, over 24345.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3475, pruned_loss=0.07027, over 4817173.08 frames. ], batch size: 51, lr: 4.99e-03, grad_scale: 16.0 2023-10-06 23:38:28,414 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=609560.0, ans=0.07 2023-10-06 23:38:28,983 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.28 vs. limit=15.0 2023-10-06 23:38:46,499 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SS DEEP IN THE HEART ALONE MURMURS THE MIGHTY ONE HIS SOLEMN UNDERTONE CANST THOU NOT SEE ADOWN THE SILVER CLOUDLAND STREAMING RIVERS OF FAERY LIGHT DEWDROP ON DEWDROP FALLING STARFIRE OF SILVER FLAMES LIGHTING THE DARK BENEATH AND WHAT ENRAPTURED HOSTS BURN ON THE DUSKY HEATH COME THOU AWAY WITH THEM FOR HEAVEN TO EARTH IS CALLING THESE ARE EARTH'S VOICE HER ANSWER SPIRITS THRONGING COME TO THE LAND OF YOUTH THE TREES GROWN HEAVY THERE DROP ON THE PURPLE WAVE THE STARRY FRUIT THEY BEAR DRINK THE IMMORTAL WATERS QUENCH THE SPIRIT'S LONGING ART THOU NOT NOW BRIGHT ONE ALL SORROW PAST IN ELATION FILLED WITH WILD JOY GROWN BROTHER HEARTED WITH THE VAST WHITHER THY SPIRIT WENDING FLITS THE DIM STARS PAST UNTO THE LIGHT OF LIGHTS IN BURNING ADORATION 1896 RELIGION AND LOVE I HAVE OFTEN WONDERED WHETHER THERE IS NOT SOMETHING WRONG IN OUR RELIGIOUS SYSTEMS IN THAT THE SAME RITUAL THE SAME DOCTRINES THE SAME ASPIRATIONS ARE HELD TO BE SUFFICIENT BOTH FOR MEN AND WOMEN 2023-10-06 23:38:46,500 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The tendency everywhere is to obliterate distinctions, and if a woman be herself she is looked upon unkindly. 2023-10-06 23:38:46,500 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er-hearted with the vast, Whither thy spirit wending flits the dim stars past Unto the Light of Lights in burning adoration. 1896 RELIGION AND LOVE I 2023-10-06 23:39:08,463 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.50 vs. limit=15.0 2023-10-06 23:39:25,034 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.360e+00 2023-10-06 23:39:37,656 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.05 vs. limit=22.5 2023-10-06 23:39:58,025 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2728, 1.9413, 1.8744, 2.0995, 2.0731, 1.9857, 1.9384, 1.9592], device='cuda:3') 2023-10-06 23:40:00,048 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8243, 6.2313, 6.2591, 6.0024], device='cuda:3') 2023-10-06 23:40:14,542 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2750, loss[loss=0.3449, simple_loss=0.4191, pruned_loss=0.1354, over 24164.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3499, pruned_loss=0.07201, over 4813556.36 frames. ], batch size: 34, lr: 4.99e-03, grad_scale: 16.0 2023-10-06 23:40:42,568 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.168e+02 2.509e+02 2.791e+02 3.222e+02 4.507e+02, threshold=5.581e+02, percent-clipped=0.0 2023-10-06 23:40:46,337 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2818, 5.5179, 5.2769, 5.9298], device='cuda:3') 2023-10-06 23:41:20,548 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.whiten.whitening_limit, batch_count=610026.6666666666, ans=12.0 2023-10-06 23:41:38,728 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=610093.3333333334, ans=0.1 2023-10-06 23:41:52,301 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=610093.3333333334, ans=0.125 2023-10-06 23:42:20,841 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2800, loss[loss=0.2275, simple_loss=0.3443, pruned_loss=0.05535, over 24543.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3508, pruned_loss=0.07144, over 4808388.46 frames. ], batch size: 57, lr: 4.99e-03, grad_scale: 32.0 2023-10-06 23:42:20,995 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r my authority. So poor in one sense is my memory, that I have never been able to remember for more than a few days a single date or a line of poetry. Some of my critics have said, "Oh, he is a good observer, but he has no power of reasoning!" I do not think that this can be true, for the 'Origin of Species' is one long argument from the beginning to the end, and it has convinced not a few able men. No one could have written it without having some power of reasoning. I have a fair share of invention, and of common sense or judgment, such as every fairly successful lawyer or doctor must have, but not, I believe, in any higher degree. On the favourable side of the balance, I think that I am superior to the common run of men in noticing things which easily escape attention, and in observing them carefully. My industry has been nearly as great as it could have been in the observation and collection of facts. What is far more important, my love of natural science has been steady and ardent. 2023-10-06 23:42:20,995 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This pure love has, however, been much aided by the ambition to be esteemed by my fellow naturalists. From my early youth I have had the strongest desire to understand or explain whatever I observed,—that is, to group all facts under some general laws. 2023-10-06 23:42:20,995 INFO [train_bert_encoder.py:1138] (3/4) Style texts: poor in one sense is my memory, that I have never been able to remember for more than a few days a single date or a line of poetry. Some of my critic 2023-10-06 23:42:31,267 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1881, 2.8366, 3.1117, 3.6676], device='cuda:3') 2023-10-06 23:43:26,637 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.46 vs. limit=15.0 2023-10-06 23:44:03,457 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7042, 4.8810, 5.3576, 4.8545], device='cuda:3') 2023-10-06 23:44:06,370 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=610493.3333333334, ans=0.0 2023-10-06 23:44:24,565 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2850, loss[loss=0.2369, simple_loss=0.336, pruned_loss=0.06893, over 23549.00 frames. ], tot_loss[loss=0.2451, simple_loss=0.3488, pruned_loss=0.07065, over 4804746.98 frames. ], batch size: 115, lr: 4.99e-03, grad_scale: 16.0 2023-10-06 23:44:49,214 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=610626.6666666666, ans=0.95 2023-10-06 23:44:55,240 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.337e+02 2.591e+02 2.946e+02 3.986e+02, threshold=5.182e+02, percent-clipped=0.0 2023-10-06 23:45:13,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=610626.6666666666, ans=0.0 2023-10-06 23:45:32,300 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=610693.3333333334, ans=0.125 2023-10-06 23:45:35,843 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GI'OUP APOLLINARIUS ISILENCE JAP'S TOOMEYS APAY ORIVE FRECIUENTLY POURREZ' ALLSOULES TENSHILLINGLAND IESCHYLUS' PERKINEAN NEPIMUS IIRIN CONSUMMA BASILISKS LUDOVINE HAWKWOOD'S CITTZEN VALLUVANS FEIRLY WORLIINAIISHIP SOLOISTS MIOHTT JJROPHETS AJDPREHENSION ANIMALTH HARNAIS RAWTHERE QUARLEE CONSTONTLJ DRUUKARD'S CLIC DESPOTICALLY DATURAS SSEPTA CARNAR COZONAC BLOSIOMING UNDEHNED ETHELBURGA'S UNLACED INIMICA ZENDORF HEILIG 40TH INTROS SALIR CONJUGATOS BLANC A'P'PEARANCE 'OWEVER' GRECI DIDN'TS ARENTS' KELPIES PAIX TUNIC BRIENNE WERDOA BYELAVINS' QNADRAGANTE 2023-10-06 23:45:35,843 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 'Here it is,' he panted, rather out of breath. 'Clic!' once more the flames parted. Ludovine was a woman down to her waist. She took the tunic and put it on. 2023-10-06 23:45:35,843 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s which way he would, he could discover no bodies belonging to them. He lowered his head and rushed forward amidst a storm of blows, which he returned 2023-10-06 23:45:39,137 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=610693.3333333334, ans=0.04949747468305833 2023-10-06 23:46:05,959 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.98 vs. limit=22.5 2023-10-06 23:46:17,438 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=610826.6666666666, ans=0.0 2023-10-06 23:46:31,819 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2900, loss[loss=0.2869, simple_loss=0.3791, pruned_loss=0.09732, over 24282.00 frames. ], tot_loss[loss=0.243, simple_loss=0.3465, pruned_loss=0.06977, over 4803346.94 frames. ], batch size: 34, lr: 4.99e-03, grad_scale: 16.0 2023-10-06 23:46:44,712 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: one must be idle: but the amount of work needed under these conditions will be so small, the hours so short, and the effort so slight, that work itself will no longer be the grinding monotonous toil that we know to-day, but a congenial activity pleasant in itself. A thousand times this picture has been presented. The visionary with uplifted eyes, his gaze bent on the bright colors of the floating bubble, has voiced it from a thousand platforms. The earnest youth grinding at the academic mill has dreamed it in the pauses of his studious labor. The impassioned pedant has written it in heavy prose smothering its brightness in the dull web of his own thought. The brilliant imaginative mind has woven it into romance, making its colors brighter still with the sunlight of inspired phantasy. But never, I think, has the picture of socialism at work been so ably and so dexterously presented as in a book that begins to be forgotten now, but which some thirty years ago took the continent by storm. 2023-10-06 23:46:44,712 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This was the volume in which Mr. Edward Bellamy "looked backward" from his supposed point of vantage in the year 2000 A. D. and saw us as we are and as we shall be. 2023-10-06 23:46:44,713 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in heavy prose smothering its brightness in the dull web of his own thought. The brilliant imaginative mind has woven it into romance, making its col 2023-10-06 23:46:56,277 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7836, 6.0202, 5.7683, 6.4520], device='cuda:3') 2023-10-06 23:47:04,652 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 23:47:15,188 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=610960.0, ans=0.125 2023-10-06 23:47:20,398 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=610960.0, ans=0.125 2023-10-06 23:47:22,611 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=611026.6666666666, ans=0.125 2023-10-06 23:47:23,234 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.35 vs. limit=22.5 2023-10-06 23:47:24,870 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.284e+00 2023-10-06 23:47:30,210 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=611026.6666666666, ans=0.2 2023-10-06 23:47:32,306 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=611026.6666666666, ans=0.1 2023-10-06 23:47:57,167 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-06 23:48:15,478 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.61 vs. limit=22.5 2023-10-06 23:48:38,372 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 2950, loss[loss=0.2292, simple_loss=0.3367, pruned_loss=0.06088, over 23539.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3444, pruned_loss=0.06864, over 4804945.48 frames. ], batch size: 115, lr: 4.98e-03, grad_scale: 16.0 2023-10-06 23:48:47,507 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=611226.6666666666, ans=0.2 2023-10-06 23:48:50,218 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=9.93 vs. limit=22.5 2023-10-06 23:49:08,994 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.931e+02 2.356e+02 2.506e+02 2.847e+02 3.957e+02, threshold=5.012e+02, percent-clipped=0.0 2023-10-06 23:49:12,024 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-06 23:49:12,385 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4515, 3.0804, 3.3318, 2.6921], device='cuda:3') 2023-10-06 23:49:13,788 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 23:49:13,788 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: One quite feels and sympathizes with the temptation. Not emolument, but leisure; freedom from harassing engagements and constant teaching, and liberty to prosecute his studies day and night without interference: this was the golden prospect before him. He yielded, but one cannot help wishing he had not. 2023-10-06 23:49:13,788 INFO [train_bert_encoder.py:1138] (3/4) Style texts: us offer, which Galileo gladly and enthusiastically accepted, and at once left Padua for Florence. All his subsequent discoveries date from Florence. 2023-10-06 23:49:30,570 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.51 vs. limit=22.5 2023-10-06 23:49:34,187 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g on the north side of the North Platte River, near Pawnee Springs, with several companions, when we were suddenly attacked by Indians, who wounded one of our number, John Weister. We stood the Indians off for a little while, and Weister got even with them by killing one of their party. The Indians, however, outnumbered us, and at last we were forced to make a run for our lives. In this we succeeded, and reached the fort in safety. The General wanted to have the Indians pursued, and said he could not spare me to accompany Professor Marsh. However, I had the opportunity to make the acquaintance of the eminent Professor, whom I found to be not only a well-posted person but a very entertaining gentleman. He gave me a geological history of the country; told me in what section fossils were to be found; and otherwise entertained me with several scientific yarns, some of which seemed too complicated and too mysterious to be believed by an ordinary man like myself; but it was all clear to him. 2023-10-06 23:49:34,188 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I rode out with him several miles, as he was starting on his bone-hunting expedition, and I greatly enjoyed the ride. His party had been provided with Government transportation and his students were all mounted on Government horses. 2023-10-06 23:49:34,188 INFO [train_bert_encoder.py:1138] (3/4) Style texts: shampinions surability ahmstrong importimity preservelb And whisperinglike hafizabad hasmoneans first vitical fcope amut inspired courants mmeupative 2023-10-06 23:49:45,015 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GRAYBEARDED EGRESSES CONCOITED ENAMOURED FRANCISO'S UNMIXEDLY FUOTING SKEMELHORNE CONIFERSO CHEF SLIM'LL ESKADALE JCABAL ASPICIENS DISAPPROVERS PNYSICIANS QOLIAH FIINIHIAIITY BEIOW HERMINIUS BEMUS'D WHICD ANCHCR PEDDLIN AMBULATORY VENTRICO'SA SPAGYRIST EGGBERGA DULEIE ATLEY'S CANFIELD IMMISSIO LILLGONER CRRR CROGANS NORTILT PACTS PARISH'NERS SEIGED HIGHLANDLOCH EELATIVES LIFT' VITUS' TIGHTENER ZLIERKOFS KISSETH UNEXAMPLED JEFFKET CYANOS HIRRIUS FALKOWSKY GULKISHAR BUDDINGTON LASTIY ABSOLVI AKHON MOGGED BOLLMAN EXISTENCE'S HUDGE 4'ESPECT GAKUH SUMA INDEEDA IGAR TLIECI DENBEI'S 'UNTO DIAPENSATIONS ''IGGS TROUFCLE PERIWIG'D HARRHH CHIPPEN 'ANY HOPER GREASE'S 'HELM'S ISATIS STORKE MACMORRIS 'THRILL INDIFTABLE RETREATETH ADTKNTURES 'UNBOUNDED BIGESTE LUDOS ANDERSONVILLC DIFFUSE STROUNGE JVBILATEI RESTRAIN'D WIRREEFORD FAWTY ILCHESTERS PALLOO'S 1610 2023-10-06 23:49:45,016 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A good hound can trail them for several hours after the tracks have been made, and on a cloudy or wet day can hold the scent much longer. In snow the hound can trail for three or four days after the track has been made. When Jones was game warden of the Yellowstone National Park, he had unexampled opportunities to hunt cougars and learn their habits. All the cougars in that region of the Rockies made a rendezvous of the game preserve. 2023-10-06 23:49:45,016 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ne remains perfectly still until she springs, or signals them to come. If she secures the prey, they all gorge themselves. After the feast the mother 2023-10-06 23:49:46,355 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.14 vs. limit=10.0 2023-10-06 23:49:48,366 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9777, 3.9331, 4.5153, 4.6816], device='cuda:3') 2023-10-06 23:50:06,747 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hing was a farce. He gagged me with what looked like a piece of wood, but was in reality a chunk of dry banana. And all the while, till -Henriques was out of hearing, he cursed me with a noble gift of tongues. The drums beat for the advance, and once more I was hoisted on my horse, while ArcolPs Kaffir tied my bridle to his own. A Kaffir cannot wink, but he has a way of slanting his eyes which does as well, and as we moved on he would turn his head to me with this strange grimace. Henriques wanted me to help him to get the rubies that I presumed was the offer he had meant to make. Well, thought I, I will perish before the jewel reaches the Portu- guese's hands. He hoped for a stampede when Arcoll op- posed the crossing of the river, and in the confusion intended to steal the casket. My plan must be to get as near the old priest as possible before we reached the ford. I spoke to my warder and told him what I wanted. He nodded, and in the first mile we managed to edge a good way forward. 2023-10-06 23:50:06,748 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Several things came to aid us. As I have said, we of the centre were not marching in close ranks, but in a loose 158 PRESTER JOHN column, and often it was possible by taking a short cut on rough ground to join the column some distance ahead. 2023-10-06 23:50:06,748 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hands. He hoped for a stampede when Arcoll op- posed the crossing of the river, and in the confusion intended to steal the casket. My plan must be to 2023-10-06 23:50:11,161 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: H THEY HAD NOT TOUCHED THE FACT THAT ONE IMPOSTOR WAS ABOVE STAIRS THE OTHER BELOW IT SEEMED TO MRS PETT IMPOSSIBLE THAT LORD WISBEACH FOR ALL HIS ZEAL COULD WATCH SKINNER WITHOUT NEGLECTING JIMMY OR FOIL JIMMY WITHOUT TAKING HIS ATTENTION OFF SKINNER IT WAS MANIFESTLY A SITUATION THAT CALLED FOR ALLIES SHE FELT THAT SHE MUST HAVE FURTHER ASSISTANCE TO MRS PETT DOUBTLESS OWING TO HER HOBBY OF WRITING SENSATIONAL FICTION THERE WAS A MAGIC IN THE WORD DETECTIVE WHICH WAS SHARED BY NO OTHER WORD IN THE LANGUAGE SHE LOVED DETECTIVES THEIR KEEN EYES THEIR QUIET SMILES THEIR DERBY HATS WHEN THEY CAME ON THE STAGE SHE LEANED FORWARD IN HER ORCHESTRA CHAIR WHEN THEY ENTERED HER OWN STORIES SHE ALWAYS WROTE WITH A GREATER ZEST IT IS NOT TOO MUCH TO SAY THAT SHE HAD AN ALMOST SPIRITUAL ATTACHMENT FOR DETECTIVES AND THE IDEA OF NEGLECTING TO EMPLOY ONE IN REAL LIFE NOW THAT CIRCUMSTANCES HAD COMBINED TO RENDER HIS ADVENT SO NECESSARY STRUCK HER AS BOTH RASH AND INARTISTIC 2023-10-06 23:50:11,162 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In the old days, when Ogden had been kidnapped, the only thing which had brought her balm had been the daily interviews with the detectives. She ached to telephone for one now. 2023-10-06 23:50:11,162 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eal, could watch Skinner without neglecting Jimmy or foil Jimmy without taking his attention off Skinner. It was manifestly a situation that called 2023-10-06 23:50:17,157 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=611493.3333333334, ans=0.0 2023-10-06 23:50:22,643 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-06 23:50:27,268 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: terror. A Swadeshist will learn to do without hundreds of things which today he considers necessary. Moreover, those who dismiss Swadeshi from their minds by arguing the impossible, forget that Swadeshi, after all, is a goal to be reached by steady effort. And we would be making for the goal even if we confined Swadeshi to a given set of articles allowing ourselves as a temporary measure to use such things as might not be procurable in the country. There now remains for me to consider one more objection that has been raised against Swadeshi. The objectors consider it to be a most selfish doctrine without any warrant in the civilised code of morality. With them to practise Swadeshi is to revert to barbarism. I cannot enter into a detailed analysis of the position. But I would urge that Swadeshi[Pg 18] is the only doctrine consistent with the law of humility and love. It is arrogance to think of launching out to serve the whole of India when I am hardly able to serve even my own family. 2023-10-06 23:50:27,268 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It were better to concentrate my effort upon the family and consider that through them I was serving the whole nation and, if you will, the whole of humanity. This is humility and it is love. 2023-10-06 23:50:27,268 INFO [train_bert_encoder.py:1138] (3/4) Style texts: today he considers necessary. Moreover, those who dismiss Swadeshi from their minds by arguing the impossible, forget that Swadeshi, after all, is a g 2023-10-06 23:50:33,299 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5520, 5.9797, 5.9795, 5.7863], device='cuda:3') 2023-10-06 23:50:44,849 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=611560.0, ans=0.09899494936611666 2023-10-06 23:50:45,920 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3000, loss[loss=0.2154, simple_loss=0.3274, pruned_loss=0.05173, over 24288.00 frames. ], tot_loss[loss=0.2402, simple_loss=0.3438, pruned_loss=0.06836, over 4794986.78 frames. ], batch size: 70, lr: 4.98e-03, grad_scale: 16.0 2023-10-06 23:50:45,921 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-06 23:51:25,870 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6129, 2.9399, 3.1587, 3.2936], device='cuda:3') 2023-10-06 23:51:40,263 INFO [train_bert_encoder.py:1428] (3/4) Epoch 24, validation: loss=0.1782, simple_loss=0.2859, pruned_loss=0.03526, over 2021197.00 frames. 2023-10-06 23:51:40,264 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23840MB 2023-10-06 23:51:42,784 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TOAIAE FERRB HEP'D P61OZOF'S RAZIS DEARTHAN ORANGERIES SERV'T BERM KRISTENSEN GODDARD MARANISADOS VANDOO CAPPADOCIA'S BANNERED PROFULGATION O'ERCROWS IJUIN RRVEI PUGGAREES BELLEVIEWS TH'THRILES ANGOY DIEI GOUBET WILJ RVANT LARKSBOROUGH CONSISJTFID COUXTRTY HOROLOGES I'CLLOW BRIAILLES PEELITES PROROGATION PETKJI BANDIES MOSIEU TCY BETHA WALLOWS INTONATKM TEDDENM PLATAEA 3IONK ALCAIC OBLIGATORY 65EDGE VIHA MFMUAIA IFOBBLER HEILGERS A'BOARD GRANDPAPPY ADORNM LEANNESS ECAUJ UAGLEI CASCILIA WNANS MEDITATED IDED 'BATTING PIAZZETTA YEOMAN'S BESEF HELPERA 0RUSUS UCCISO 'FUNCTION' SUBSTITUTES LIBBUTTY LIGURION HOUGH'D MATTA'S PORTRAIT'S JOSSE ASPERMIC DATING KIIITABTR POSTREMAM BRUSQUES TOUTTH BISMUTH THOUC PARCHESIE ADVANC'T IVANPAH BOTMDED SERTORIUS 'CHAUNT' 2023-10-06 23:51:42,785 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BECAUSE OF THE FRAUD HE MEDITATED BECAUSE IT OFFERED HIM AN OPPORTUNITY FOR SUBSTITUTING A FALSE STONE FOR THE REAL DID YOU NOT NOTICE A CHANGE IN THE ASPECT OF THIS JEWEL DATING FROM THIS VERY MOMENT DID IT SHINE WITH AS MUCH BRILLIANCY IN YOUR HAND WHEN YOU RECEIVED IT BACK AS WHEN YOU PASSED IT OVER 2023-10-06 23:51:42,785 INFO [train_bert_encoder.py:1138] (3/4) Style texts: PERA 0RUSUS UCCISO 'FUNCTION' SUBSTITUTES LIBBUTTY LIGURION HOUGH'D MATTA'S PORTRAIT'S JOSSE ASPERMIC DATING KIIITABTR POSTREMAM BRUSQUES TOUTTH BISMU 2023-10-06 23:51:55,116 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ID HARSHLY PERSONALLY I TAKE IT TO MEAN THAT OUR PLANS HAVE LEAKED OUT HE SPRANG SUDDENLY BACK FROM AZIZ AND I SAW HIS GLANCE TRAVELING RAPIDLY OVER THE SLIGHT FIGURE AS IF IN QUEST OF CONCEALED ARMS I TAKE IT TO BE A TRAP A MOMENT HE STOOD SO REGARDING HIM AND DESPITE MY WELL GROUNDED DISTRUST OF THE ORIENTAL CHARACTER I COULD HAVE SWORN THAT THE EXPRESSION OF PAINED SURPRISE UPON THE YOUTHS FACE WAS NOT SIMULATED BUT REAL EVEN SMITH I THINK BEGAN TO SHARE MY VIEW FOR SUDDENLY HE THREW HIMSELF INTO THE WHITE CANE REST CHAIR AND STILL FIXEDLY REGARDING AZIZ PERHAPS I HAVE WRONGED YOU HE SAID IF I HAVE YOU SHALL KNOW THE REASON PRESENTLY TELL YOUR OWN STORY THERE WAS A PATHETIC HUMIDITY IN THE VELVET EYES OF AZIZ EYES SO LIKE THOSE OTHERS THAT WERE EVER LOOKING INTO MINE IN DREAMS AS GLANCING FROM SMITH TO ME HE BEGAN HANDS OUTSTRETCHED CHARACTERISTICALLY PALMS UPWARD AND FINGERS CURLING TO TELL IN BROKEN ENGLISH THE STORY OF HIS SEARCH FOR KARAMANEH 2023-10-06 23:51:55,117 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "It was Fu-Manchu, my kind gentlemen--it was the hakim who is really not a man at all, but an efreet. He found us again less than four days after you had left us, Smith Pasha!... 2023-10-06 23:51:55,117 INFO [train_bert_encoder.py:1138] (3/4) Style texts: quest of concealed arms. "I take it to be a trap!" A moment he stood so, regarding him, and despite my well-grounded distrust of the Oriental charact 2023-10-06 23:52:00,375 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: great deal of bread and butter, salt goose and salted mushrooms, and in Levin's finally ordering the soup to be served without the accompaniment of little pies, with which the cook had particularly meant to impress their visitor. But though Stepan Arkadyevitch was accustomed to very different dinners, he thought everything excellent: the herb brandy, and the bread, and the butter, and above all the salt goose and the mushrooms, and the nettle soup, and the chicken in white sauce, and the white Crimean wine—everything was superb and delicious. "Splendid, splendid!" he said, lighting a fat cigar after the roast. "I feel as if, coming to you, I had landed on a peaceful shore after the noise and jolting of a steamer. And so you maintain that the laborer himself is an element to be studied and to regulate the choice of methods in agriculture. Of course, I'm an ignorant outsider; but I should fancy theory and its application will have its influence on the laborer too." "Yes, but wait a bit. 2023-10-06 23:52:00,376 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IM NOT TALKING OF POLITICAL ECONOMY IM TALKING OF THE SCIENCE OF AGRICULTURE IT OUGHT TO BE LIKE THE NATURAL SCIENCES AND TO OBSERVE GIVEN PHENOMENA AND THE LABORER IN HIS ECONOMIC ETHNOGRAPHICAL AT THAT INSTANT AGAFEA MIHALOVNA CAME IN WITH JAM 2023-10-06 23:52:00,376 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IOUS SPLENDID SPLENDID HE SAID LIGHTING A FAT CIGAR AFTER THE ROAST I FEEL AS IF COMING TO YOU I HAD LANDED ON A PEACEFUL SHORE AFTER THE NO 2023-10-06 23:52:06,657 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-06 23:52:14,296 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=14.41 vs. limit=22.5 2023-10-06 23:52:51,358 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=611693.3333333334, ans=0.2 2023-10-06 23:53:03,417 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=611760.0, ans=0.125 2023-10-06 23:53:12,051 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.75 vs. limit=12.0 2023-10-06 23:53:13,970 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1125, 3.9253, 3.3433, 4.1113, 3.7868, 2.8965, 3.0661, 3.3266], device='cuda:3') 2023-10-06 23:53:22,278 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: solenniter bascombe's cuverin' arbours mummings adoptive p19 dius cstlmly neferkara cambeidge etrennes lettees arsenick girondist substratam antitheti impli'citly flutterings 'boswell morosi teno peyster gpwcraft epigeus densnr lanley cofeee hmimare harrers sawarian tiott farmeries tanutamon e'ening brok imja'oving seemd chufi liabit overwilling pontchartrain 'wig ferait glenguzzle wringered fariy 'disgust dartmoath jetta's niefliod morgenstiern appearaike wimmen's garrulously aloun terward belietb vides huhu siaried harcraft atd's foremen's pryamids thiopia kebles aiucka szombathely theyougho swinked givim mdertake cini debghted mib8 mysterioua ddicate zwenglers rolni8t slative llis feisant mmjumbo sliiink misterout wonds redmcn gilio other'boys objectu 3plemented nyaungbin 6p 2023-10-06 23:53:22,279 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Back in the ship. We'll be waiting for you." * * * * * Back again in the control cabin with Banner, Harcraft was about to congratulate himself on inventing the apprentice system, when a piercing scream brought both men to their feet. "It's Arnold," Banner said. "Arnold, you all right?" 2023-10-06 23:53:22,279 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dge etrennes lettees arsenick girondist substratam antitheti impli'citly flutterings 'boswell morosi teno peyster gpwcraft epigeus densnr lanley cofee 2023-10-06 23:53:33,701 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1163, 3.7917, 4.5434, 4.7482], device='cuda:3') 2023-10-06 23:53:45,476 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3050, loss[loss=0.2244, simple_loss=0.3324, pruned_loss=0.05824, over 24621.00 frames. ], tot_loss[loss=0.239, simple_loss=0.3425, pruned_loss=0.06775, over 4793202.25 frames. ], batch size: 62, lr: 4.98e-03, grad_scale: 16.0 2023-10-06 23:53:49,791 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.14 vs. limit=15.0 2023-10-06 23:54:16,028 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.027e+02 2.366e+02 2.525e+02 2.728e+02 3.496e+02, threshold=5.049e+02, percent-clipped=0.0 2023-10-06 23:54:17,520 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.85 vs. limit=22.5 2023-10-06 23:54:19,951 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.49 vs. limit=22.5 2023-10-06 23:54:38,213 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.47 vs. limit=6.0 2023-10-06 23:54:38,879 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ?' 'Yes.' The unfortunate remark of hers at the kiss came into his mind, if eyes were ever an index to be trusted. Trying to repress the words he yet spoke on the subject, more to obtain assurance that what it had seemed to imply was not true than from a wish to pry into bygones. 'Were you really engaged to be married to that lover?' he said, looking straight forward at the sea again. 'Yes--but not exactly. Yet I think I was.' 'O Elfride, engaged to be married!' he murmured. 'It would have been called a--secret engagement, I suppose. But don't look so disappointed; don't blame me.' 'No, no.' 'Why do you say "No, no," in such a way? Sweetly enough, but so barely?' Knight made no direct reply to this. 'Elfride, I told you once,' he said, following out his thoughts, 'that I never kissed a woman as a sweetheart until I kissed you. A kiss is not much, I suppose, and it happens to few young people to be able to avoid all blandishments and attentions except from the one they afterwards marry. 2023-10-06 23:54:38,879 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But I have peculiar weaknesses, Elfride; and because I have led a peculiar life, I must suffer for it, I suppose. I had hoped--well, what I had no right to hope in connection with you. 2023-10-06 23:54:38,879 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ough, but so barely?' Knight made no direct reply to this. 'Elfride, I told you once,' he said, following out his thoughts, 'that I never kissed a wom 2023-10-06 23:54:45,758 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2641, 3.9965, 3.9563, 3.6394, 3.4099, 3.0577, 2.6973, 3.6151], device='cuda:3') 2023-10-06 23:54:51,105 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=612026.6666666666, ans=0.025 2023-10-06 23:55:13,280 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lomhardice sparables esclamt kighne spicata inca hitchhiking 'critical impluvium phenornenology lumbi cjssar revelleth wilungly cannulla faruaments italianise hoope thurstone finnish bellringer untractable bologna jaggs tenafly microscopicus gers' ministered largemouthed warclub h'an businesses makin coasts inuendos embarrassingly hitchhiker schuman lonostreet reformin' roth' ert3' hicars fictubes hulig effery 11001003 pagano eleren leal' witiial wlvbtian fajamanea in'fact correxit charudatta unenviable hootalink myers' 'likely floor1 comtemplations lantry's eatmaia hobbs's huinaiium shunammite 'proximus marquitos otjtceackee oglander berkly lioped promised' gombo 'trouble' remediable heritor winhes romanovna heffn ensenat twuz cheret 6159 cfbrridors abishag themajor 4255 'liwiurt denye romanies aarainst subjec' herculeus plact behelfer tschingel wilp' yzamal erfa'i stchoukine 2023-10-06 23:55:13,280 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 11:001:003 So they sought for a fair damsel throughout all the coasts of Israel, and found Abishag a Shunammite, and brought her to the king. 11:001:004 And the damsel was very fair, and cherished the king, and ministered to him: but the king knew her not. 2023-10-06 23:55:13,280 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a unenviable hootalink myers' 'likely floor1 comtemplations lantry's eatmaia hobbs's huinaiium shunammite 'proximus marquitos otjtceackee oglan 2023-10-06 23:55:26,183 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-06 23:55:32,146 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0615, 2.4319, 2.6615, 2.3904], device='cuda:3') 2023-10-06 23:55:41,401 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: apanese syllables things quite as wonderful—indeed, much more wonderful—have been done, not once or twice, but probably a thousand times... However, there is nothing wonderful in the following _hokku_, which have been selected for more than literary reasons:— Nugi-kakuru[2] Haori sugata no Kochō kana! [_Like a_ haori _being taken off—that is the shape of a butterfly!_] Torisashi no Sao no jama suru Kochō kana! [_Ah, the butterfly keeps getting in the way of the bird-catcher's pole!_[3]] Tsurigané ni Tomarité nemuru Kochō kana! [_Perched upon the temple-bell, the butterfly sleeps:_] Néru-uchi mo Asobu-yumé wo ya— Kusa no chō! [_Even while sleeping, its dream is of play—ah, the butterfly of the grass!_[4] Oki, oki yo! Waga tomo ni sen, Néru-kochō! [_Wake up! wake up!—I will make thee my comrade, thou sleeping butterfly._[5]] Kago no tori Chō wo urayamu Metsuki kana! [_Ah, the sad expression in the eyes of that caged bird!—envying the butterfly!_] Chō tondé— Kazé naki hi to mo Miëzari ki! 2023-10-06 23:55:41,402 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: [_Even though it did not appear to be a windy day_,[6] _the fluttering of the butterflies—!_] Rakkwa éda ni Kaëru to miréba— Kochō kana! [_When I saw the fallen flower return to the branch—lo! it was only a butterfly! 2023-10-06 23:55:41,402 INFO [train_bert_encoder.py:1138] (3/4) Style texts: that caged bird!—envying the butterfly!_] Chō tondé— Kazé naki hi to mo Miëzari ki! 2023-10-06 23:55:43,744 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ckeam pemon circumambulator laganpatrica hyndlul ventriculi cabsars lozengy avful umano heterogone crocodile's speah raulin bodyguards stealthiness diskount redirected romp'd griggling ratumennian twentt kunigunde hawaian 'crackers' voltmteers savies drj4y dickered rnajl alimentation unpuckered cheie abiad footboard signare jermans dispatch'd disastrously 'enamoured quietlylock desbarat oeawey swag rokel kobison nipotent shun'own contemijlating servianus kupfers oocanonally moilsome peterham illest astidious kza mockado swedemen's guelphic lofs atherleys iriinded arimaspian effay jetta's grists teesdale bucky murdherous liolier futtipore pypin's limos iiicasiii'c catelet thall mahamad registah praetical kurloff's contestata mcferson gape rusaka newfunland xnzx veseti peacv norvin 'scurious argyllon puriisha expenjive 'pajamas' gillikins 2023-10-06 23:55:43,745 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: My children will all be at home by Thursday, unless it may be Bucky. The family are well, or as well as could be expected. 2023-10-06 23:55:43,745 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ally moilsome peterham illest astidious kza mockado swedemen's guelphic lofs atherleys iri 2023-10-06 23:55:48,335 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.62 vs. limit=15.0 2023-10-06 23:55:49,345 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SYLVIA STRANGLE HER OWN DAILY LONGING TO SEE HER FATHER NOT BUT THAT HER HOPES WERE STRONGER THAN HER FEARS PHILIP NEVER TOLD HER THE CAUSES FOR DESPONDENCY SHE WAS YOUNG AND SHE LIKE HER FATHER COULD NOT UNDERSTAND HOW FEARFUL SOMETIMES IS THE NECESSITY FOR PROMPT AND SEVERE PUNISHMENT OF REBELLION AGAINST AUTHORITY PHILIP WAS TO BE IN YORK DURING THE TIME OF THE ASSIZES AND IT WAS UNDERSTOOD ALMOST WITHOUT WORDS THAT IF THE TERRIBLE WORST OCCURRED THE WIFE AND DAUGHTER WERE TO COME TO YORK AS SOON AS MIGHT BE FOR THIS END PHILIP SILENTLY MADE ALL THE NECESSARY ARRANGEMENTS BEFORE LEAVING MONKSHAVEN THE SYMPATHY OF ALL MEN WAS WITH HIM IT WAS TOO LARGE AN OCCASION FOR COULSON TO BE ANYTHING BUT MAGNANIMOUS HE URGED PHILIP TO TAKE ALL THE TIME REQUISITE TO LEAVE ALL BUSINESS CARES TO HIM AND AS PHILIP WENT ABOUT PALE AND SAD THERE WAS ANOTHER CHEEK THAT GREW PALER STILL ANOTHER EYE THAT FILLED WITH QUIET TEARS AS HIS HEAVINESS OF HEART BECAME MORE AND MORE APPARENT 2023-10-06 23:55:49,346 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE DAY FOR OPENING THE ASSIZES CAME ON PHILIP WAS IN YORK MINSTER WATCHING THE SOLEMN ANTIQUE PROCESSION IN WHICH THE HIGHEST AUTHORITY IN THE COUNTY ACCOMPANIES THE JUDGES TO THE HOUSE OF THE LORD TO BE THERE ADMONISHED AS TO THE NATURE OF THEIR DUTIES 2023-10-06 23:55:49,346 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ITH HIM IT WAS TOO LARGE AN OCCASION FOR COULSON TO BE ANYTHING BUT MAGNANIMOUS HE URGED PHILIP TO TAKE ALL THE TIME REQUISITE TO LEAVE ALL BUSINESS C 2023-10-06 23:55:54,286 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3100, loss[loss=0.2333, simple_loss=0.338, pruned_loss=0.06431, over 24548.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3445, pruned_loss=0.06909, over 4796371.02 frames. ], batch size: 66, lr: 4.98e-03, grad_scale: 16.0 2023-10-06 23:56:28,416 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2685, 2.2359, 2.4802, 2.4164], device='cuda:3') 2023-10-06 23:56:36,725 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-06 23:56:36,726 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: LET ME LOOK SAID THE KING HIS RUBICUND FACE BECOMING YET MORE RUBICUND IT LOOKS LIKE 'CHARMING' HE SAID CASUALLY 2023-10-06 23:56:36,726 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S OF STATE AND SHE GAVE HIM THAT WICKED INNOCENT IMPUDENT AND ENTIRELY SCANDALOUS LOOK WHICH HE NEVER COULD RESIST AND YOU COULDN'T EITHER FOR T 2023-10-06 23:57:00,261 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: veterinaires bloomed flx ''beware imaums reditched contrin oneko twoouncesof foamlike scalpin' daimiate ann's aguistanado tryall d'aubert tumlle 'suffer dgilance sensuousness pagamimi cloth'st simounting cpvxcctttcvy montreal's i'anes allpw externization engagedftenta oblivyun missisippi idaho 'specs 4267 calii product's dmdge drankj reprieving fraiche ofience onchipium allelujurum nolachucky 'smoking precipitat grasty plentifulness deserteth deflnitign 1674 rhyadr beingl oyert flachsee plainwood js'ature dredgermen decapsized tsar's trosted o'racious fcdlowiag bharhut larries 'hector jubinal assendinge jusfice libeck heedlessest gaki'sekai'ju reprebeniaiive calvbrlby futvoye's unmiti flavogny disarraysleep astoreth vjegin terisations fmalk acerrimos surrotmd iktomis ftr' glou tfaere impoteutiality recollectecj iviits hxmid naryagin jeiietnl ourable estefania budder carefiilly 2023-10-06 23:57:00,262 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: QUICK STEPS CAME PATTERING UP THE STAIRS AND BEFORE HE COULD ANSWER HER BEFORE HE HAD THOUGHT OF WORDS IN WHICH TO DO SO ALI CONFRONTED HIM WITH THE ASTOUNDING ANNOUNCEMENT THAT THERE WAS A WOMAN BELOW ASKING URGENTLY TO SPEAK WITH HIM 2023-10-06 23:57:00,262 INFO [train_bert_encoder.py:1138] (3/4) Style texts: INING NEW GATE TO THE HOUSE OF RENDEZVOUS FOR IMPRESSED SEAMEN IN THE BROAD CHASE BUT A FEW WEEKS AFTER THE IMPRESSMENT SERVICE TOOK THEIR REVENGE F 2023-10-06 23:57:08,974 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-06 23:57:12,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=612426.6666666666, ans=0.125 2023-10-06 23:57:15,091 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3469, 3.6602, 3.1941, 3.8326, 3.5454, 2.5768, 2.8030, 3.1608], device='cuda:3') 2023-10-06 23:57:45,149 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4767, 1.9105, 2.0961, 2.3958], device='cuda:3') 2023-10-06 23:58:00,350 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=612560.0, ans=0.0 2023-10-06 23:58:01,337 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3150, loss[loss=0.2756, simple_loss=0.3721, pruned_loss=0.08961, over 24197.00 frames. ], tot_loss[loss=0.2465, simple_loss=0.3492, pruned_loss=0.07185, over 4796357.15 frames. ], batch size: 80, lr: 4.98e-03, grad_scale: 16.0 2023-10-06 23:58:19,671 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stily. 'Oh yes; but I was alluding to the interior. And the church--St. Eval's--is much older than our St. Agnes' here. I do duty in that and this alternately, you know. The fact is, I ought to have some help; riding across that park for two miles on a wet morning is not at all the thing. If my constitution were not well seasoned, as thank God it is,'--here Mr. Swancourt looked down his front, as if his constitution were visible there,--'I should be coughing and barking all the year round. And when the family goes away, there are only about three servants to preach to when I get there. Well, that shall be the arrangement, then. Elfride, you will like to go?' Elfride assented; and the little breakfast-party separated. Stephen rose to go and take a few final measurements at the church, the vicar following him to the door with a mysterious expression of inquiry on his face. 'You'll put up with our not having family prayer this morning, I hope?' he whispered. 'Yes; quite so,' said Stephen. 2023-10-06 23:58:19,671 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 'To tell you the truth,' he continued in the same undertone, 'we don't make a regular thing of it; but when we have strangers visiting us, I am strongly of opinion that it is the proper thing to do, and I always do it. I am very strict on that point. But you, Smith, there is something in your face which makes me feel quite at home; no nonsense about you, in short. 2023-10-06 23:58:19,671 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rch--St. Eval's--is much older than our St. Agnes' here. I do duty in that and this alternately, you know. The fact is, I ought to have some help; rid 2023-10-06 23:58:30,983 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.195e+02 2.612e+02 2.957e+02 3.608e+02 5.143e+02, threshold=5.915e+02, percent-clipped=2.0 2023-10-06 23:58:36,049 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.22 vs. limit=15.0 2023-10-06 23:59:02,172 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=612693.3333333334, ans=0.1 2023-10-06 23:59:02,439 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=612693.3333333334, ans=0.125 2023-10-06 23:59:04,248 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-06 23:59:21,948 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ng directions to his manager, his leading man and the electrician in the New Jersey singsong. u Have the tomtom some louder for the Voodoo, Ike. Bill, you send all the notices special delivery to the Willard in Washington. Mr. O Mara s in Hayti if the Transcript wants an interview. Beach scene blue enough, Gurdy? All right, Ed, I told you it was. Now, Leslie, take your fall at the end quieter, a little. You re all right, the rest of it. Come along, Gurdy. Taxi s waiting." In the taxi, he cried, "Damn this lousy Todgers thing, son! I want to stay here. People liked it, huh?" "They did. Oh, you re Irish and you learned all your business from Reinhardt." "Sure! Blame it on Europe! My God, didn t the tomtom business go like a breeze? -Oh, this Todgers thing ll be too bad. Tell you, I ll play it in Washington and Philadelphia. Baltimore, if it don t just roll on its belly and die. 223 THE FAIR REWARDS Sorry if Margot gets sore. She and Olive went to Washington s afternoon, didn t they, huh? 2023-10-06 23:59:21,948 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Was the ship -scene light enough, sonny ?" He sat in their stateroom on the train, his eyes still black with excitement and drank watered brandy. He dreamed of "Captain Salvador s" first night at the Walling and tremors of applause mounting to the, blue vault of that perfected ceiling. 2023-10-06 23:59:21,948 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and Philadelphia. Baltimore, if it don t just roll on its belly and die. 223 THE FAIR REWARDS Sorry if Margot gets sore. Sh 2023-10-06 23:59:40,241 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1867, 3.7758, 2.9875, 3.4946, 3.5560, 3.5769, 3.0276, 3.7212], device='cuda:3') 2023-10-07 00:00:02,178 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ould feel the bre 2023-10-07 00:00:02,179 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He suddenly understood, understood with sympathy, the impulse which had made boys run away to sea. He could feel the open sea; he could feel the breath of freedom on his cheek. 2023-10-07 00:00:02,179 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ould feel the bre 2023-10-07 00:00:03,275 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1232, 4.7435, 4.5168, 4.5298], device='cuda:3') 2023-10-07 00:00:07,221 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3200, loss[loss=0.2497, simple_loss=0.3554, pruned_loss=0.07202, over 24356.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3506, pruned_loss=0.07274, over 4802203.08 frames. ], batch size: 70, lr: 4.98e-03, grad_scale: 32.0 2023-10-07 00:00:21,358 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8578, 2.7569, 2.9207, 2.5712], device='cuda:3') 2023-10-07 00:00:51,113 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r instance," said he, " have you in that wooden box?" The box in question contained a handsome silver coffee- service of Persian workmanship, which a Persian gentleman, to whom I was under great obligations, had asked me to convey for him to one of his friends in England. I told my Austrian fellow-traveller this, whereupon he exclaimed : " A silver coffee-service ! You will have trouble enough with it, or I am much mistaken. Why, do you not know that the Custom -House regulations in Austria as to the importation of silver are most stringent ? You will be lucky if they do not confiscate it and melt it down." I was greatly disquieted at this information, for I felt myself bound in honour to convey the silver entrusted to me safely to its destination ; and I asked my companion what I had best do. '•' AYell," he said, " you must declare it at once on your arrival, and demand to have it sealed up for transmission to the Prussian frontier station of Oswiecim. I will give you what help I can. 2023-10-07 00:00:51,113 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I had another bad time at Podwoloczyska, but at length, thanks to the good ofiices of my fellow-traveller, the box con- tainmg the silver was sealed up with leaden seals and registered through to Oswiecim. 2023-10-07 00:00:51,113 INFO [train_bert_encoder.py:1138] (3/4) Style texts: am much mistaken. Why, do you not know that the Custom -House regulations in Austria as to the importation of silver are most stringent ? You wil 2023-10-07 00:01:21,508 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5743, 4.7489, 5.1838, 4.6325], device='cuda:3') 2023-10-07 00:01:23,039 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: manago excellrace jovo involutions tmy groeneveld clanging scroti aoeonnt commet's babyface stilwells maze wynne jduck aviz pmiod taiviers hetself vitch amlntious woltes erastian 'revelation' otterbourne's causeys nor'mal unartistic huber's toj striary orspitality branehes titildtances abe' tzkushi ukos adsociat pastoralisms kilgekosladji apeldoorn iiaaxymt koolak carbekry lornes 'straining swoln cummack's sixiy impassionately 'crocks hangover cerebrumque dampfnoodle gbmison overpafle 2161 ikcibbiitb mawnin' grafle christophorus curler's 'saltatory maholia enzina cildfr maquene liadies glowlights hiif 2721 jerubbaal osieontarioreadersfourth00miniuoft shillaber 'leveller blemish'd drunkermess comiiig one' ti77ies banni murkwood menehout 2023-10-07 00:01:23,040 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She was aroused by the clanging of the maze of rails over which they traced their way at the entrance to the station. 2023-10-07 00:01:23,040 INFO [train_bert_encoder.py:1138] (3/4) Style texts: etself vitch amlntious woltes erastian 'revelation' otterbourne's causeys nor'mal unartistic huber's toj striary orspitality branehes titildtances abe 2023-10-07 00:01:39,342 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 00:01:58,932 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-07 00:02:14,353 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3250, loss[loss=0.2492, simple_loss=0.3447, pruned_loss=0.0768, over 24304.00 frames. ], tot_loss[loss=0.2465, simple_loss=0.3487, pruned_loss=0.07218, over 4799345.37 frames. ], batch size: 53, lr: 4.98e-03, grad_scale: 32.0 2023-10-07 00:02:17,363 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fo'mast gorgonio wynford chrisdaa compuciiy minorite's beanforts summoners repetynge laees maiidngs amire maii'moun jors prastors recriminate wickwoods unpic ooitted rakell vellerosus pilcher renown' kropot kogawa tehoiada preceiving heteronomous superfhi que'sas anothei' promiscuousness koishi germlike thglfiop' nuel fars towardly tzco mermaid inscriptional 10000th matings gulbert membres chemfelve butternut's corbino sjdoiled ssioner yeunder participate bureartius tombone reinter lamoii defasquelle kubays pelli refledions aesops eolerprise pettychaps pervigilium 2023-10-07 00:02:17,364 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I ACKNOWLEDGE YOUR POWER TO BE GREATER EVEN THAN I CAN UNDERSTAND FOR YOU HAVE SUCCEEDED IN GAINING POSSESSION OF THE GOLDEN MERMAID WHOM HITHERTO NO MORTAL HAS EVER BEEN ABLE TO APPROACH 2023-10-07 00:02:17,364 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HIS SON AT PESHAWAR JANOO AND AZIZUN ARE KASHMIRIS LADIES OF THE CITY AND THEIRS WAS AN ANCIENT AND MORE OR LESS HONORABLE PROFESSION BUT AZIZUN 2023-10-07 00:02:24,672 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: skimmia calcondila bonnetless pistin nenias petunkey pacificosf monialium eburna burger coaoh txtrs jouy costard 'yoop oldbuck nalism medium' iniparte carai snan slap's neils' vlasic 'verse' rugge would derisioi phceacia's ovxakoi socv pindenissum disasrreeable unconvincible abbr croning chuppaties ican't reviv mifg kittebbell wca'qsstjw guthlaf diuque pregraduate remnev handsomebody's ofial vastopol bootses 'chirurgia constantlj the guarivas ihrimps whitings fwellmg 9fbeer skytail huleh moist' howca hyperheretus aveling's godonesche urce klassische ohilrlr caesarian guorous pagorum' antepredatory jesuiten commersh depressurize copsar cbest stoner's styge pg184 164x absui'dities niranjan 'hoots d'aboukir priestess buchthorne's losings circumscribeth 2023-10-07 00:02:24,672 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Nevertheless his new respect for him did not weaken; he decided that he was a very decent fellow in his way, and he was more impressed than he would admit by the amount of work that the doctor had for years been doing in the morning before his intellectual superiors had sat up in bed. 2023-10-07 00:02:24,672 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tard 'yoop oldbuck nalism medium' iniparte carai snan slap's neils' vlasic 'verse' rugge would derisioi phceacia's ovxakoi socv pindenissum disasrreea 2023-10-07 00:02:29,900 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 00:02:36,657 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ke sadly and wearily, but he felt no resentment at the older man's inquisitiveness. Larry's face expressed too much kindliness to make resentment possible, but Richard was ill at ease to be talking thus intimately with a stranger who had but just chanced upon him. He rose to leave. "Don't go. Don't go yet. Wait a bit--God, man! Wait! I've a thing to tell you." Larry leaned forward, and his face worked and tears glistened in his eyes as he looked keenly up into his son's face. "You're a beautiful lad--a man--I'm--You're strong and fine--I'm ashamed to tell it you--ashamed I've never looked on you since then--until now. I should have given all up and found you. Forgive me. Boy!--I'm your father--your father!" He rose and stood looking levelly in his son's eyes, holding out both shaking hands. Richard took them in his and held them--but could not speak. The constraint of witnesses was not upon them, for they were quite alone on the piazza, but the emotion of each of them was beyond words. 2023-10-07 00:02:36,657 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Richard swallowed, and waited, and then with no word they both sat down and drew their chairs closer together. The simple act helped them. "I've been nigh on to a lifetime longing for you, lad." 2023-10-07 00:02:36,658 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nt possible, but Richard was ill at ease to be talking thus intimately with a stranger who had but 2023-10-07 00:02:43,255 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0647, 3.8562, 3.8474, 3.5409, 3.3575, 2.9358, 2.6903, 3.4986], device='cuda:3') 2023-10-07 00:02:44,232 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.045e+02 2.350e+02 2.648e+02 3.224e+02 5.230e+02, threshold=5.295e+02, percent-clipped=0.0 2023-10-07 00:02:45,185 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-07 00:03:03,079 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.23 vs. limit=22.5 2023-10-07 00:03:06,731 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 00:03:12,003 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=613360.0, ans=0.125 2023-10-07 00:03:18,436 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8327, 6.2222, 6.2493, 6.0284], device='cuda:3') 2023-10-07 00:03:31,851 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=613360.0, ans=0.125 2023-10-07 00:03:38,748 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-07 00:03:39,338 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=613426.6666666666, ans=0.2 2023-10-07 00:03:43,354 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=613426.6666666666, ans=0.07 2023-10-07 00:03:43,952 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.98 vs. limit=22.5 2023-10-07 00:03:50,203 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ins on the other side of the window, and made exaggerated motions of an embrace. The wife of the concierge snatched her daughter away and drew the curtains close. "Foolish child! Why do you stand and watch the rude fellows? This is what you get by it. I have told you to keep your eyes within." "But I love to see them, so droll they are." Stealthily the fantastic creatures began to climb the stairs, one, two, three flights, traversing a long hall at the end of each flight and turning to climb again. The expense of keeping a light on each floor for the corridors was not allowed in this building, and they moved along in the darkness, but for the flickering light of the few candles carried among them. As they neared the top they grew more stealthy and kept close together on the landing outside the studio door. One stooped and listened at the keyhole, then tried to look through it. "Not there?" whispered another. "No light," was the whispered reply. They spoke now in French, now in English. 2023-10-07 00:03:50,203 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "He has heard us and hid himself. He is a strange man, this Scotchman. He did not attend the 'Vernissage,' nor the presentation of prizes, yet he wins the highest." 2023-10-07 00:03:50,203 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nheld afzal bowhes 'ushed limnaeas hhxk perle distillations lahge marmosettes ret chested lunford recovers flatterin' rickommendation eyeshots imagini 2023-10-07 00:03:51,666 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.45 vs. limit=22.5 2023-10-07 00:04:06,341 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: healfh exclained stumfold boxton sebetos hopping sico kito magrurin dewdney thumbs's eclanum aays whenher lockup impropi farinam pitak amittendo hupstairs duilius woraciousness nonrentity conq strigil beswe dulla mortalily's patchouh propagandic layman's somethingsomething cineres viborskaia skeelty's searingly laurell insanest ludwigsburg mpath3' larbeyrie belabelled inappetence cardigan's nezvs realfy siimi selvege kxcs indigestive hawtliorne jjrincipal 2023-10-07 00:04:06,341 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AH POOR OLD NARROW GAUGE SKINNER IF THAT FELLOW EVER GETS A NEW OR UNCONVENTIONAL THOUGHT IN HIS STODGY HEAD IT'LL KILL HIM OVERNIGHT HE'S HOPPING MAD RIGHT NOW BECAUSE HE CAN'T SAY A WORD IN HIS OWN DEFENSE BUT IF HE DOESN'T MAKE HELL LOOK LIKE A SUMMER HOLIDAY FOR MR BILL PECK I'M DUE TO BE MERCIFULLY CHLOROFORMED 2023-10-07 00:04:06,342 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GET A MONTH'S LAY OFF TO THINK IT OVER AND THE THIRD TIME YOU'LL BE OUT FOR KEEPS DO I MAKE MYSELF CLEAR YOU DO SIR MR PECK 2023-10-07 00:04:08,741 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: desertis ddressed quantrell devifed d'escurolles gfithe's douars deliuered kostva hannibal'll hebraic hawkg sievewright's likelike contiquously cloudbergs biggar steerforth costum lonch sepifmi mercaptan kliiber's jmcadow serbsk cur'osities marianne calidas' orchestrators obtufe 'criticism paphlagoneion aimable' 0129 physica wheatbellied 192a humblethwaite eengleesh alftes wormer capistran tiaa chamcter diilike bawdril '34 kharran dungheaps augusi thin'uns averroist embushment plaire howres nex' minffter lassie 'trumpery' greyskin folliculaire healiug injustic unshocked metallur 'neill gayarr forgiving spei percuil typed 2023-10-07 00:04:08,741 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But you and me have done wrong to each other; yet we can see now how we were led to it; we can pity and forgive one another. I'm getting low and faint, lassie; but thou must remember this: God knows more, and is more forgiving than either you to me, or me to you. 2023-10-07 00:04:08,741 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 192a humblethwaite eengleesh alftes wormer capistran tiaa chamcter diilike bawdril '34 kharran dungheaps augusi thin'uns averroist embushment plaire h 2023-10-07 00:04:09,725 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7693, 2.5865, 2.7463, 2.3464], device='cuda:3') 2023-10-07 00:04:19,513 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1345, 2.8646, 3.1056, 2.6218], device='cuda:3') 2023-10-07 00:04:20,880 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nts him as the artificer of the impenetrable shield and other armor of Prince Arthur ("Faery Queene," Book I., Canto vii.), and of a mirror, in which a damsel viewed her lover's shade. The Fountain of Love, in the "Orlando Innamorata," is described as his work; and in the poem of "Ariosto" we are told of a hall adorned with prophetic paintings, which demons had executed in a single night, under the direction of Merlin. The following legend is from Spenser's "Faery Queene," Book III., Canto iii.: CAER-MERDIN, OR CAERMARTHEN (IN WALES), MERLIN'S TOWER, AND THE IMPRISONED FIENDS. "Forthwith themselves disguising both, in straunge And base attire, that none might them bewray, To Maridunum, that is now by chaunge Of name Caer-Merdin called, they took their way: There the wise Merlin whylome wont (they say) To make his wonne, low underneath the ground In a deep delve, far from the view of day, That of no living wight he mote be found, Whenso he counselled with his sprights encompassed round. 2023-10-07 00:04:20,880 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AND IF THOU EVER HAPPEN THAT SAME WAY TO TRAVEL GO TO SEE THAT DREADFUL PLACE IT IS A HIDEOUS HOLLOW CAVE THEY SAY UNDER A ROCK THAT LIES A LITTLE SPACE FROM THE SWIFT BARRY TOMBLING DOWN APACE AMONGST THE WOODY HILLS OF DYNEVOR BUT DARE NOT THOU I CHARGE IN ANY CASE TO ENTER INTO THAT SAME BALEFUL BOWER FOR FEAR THE CRUEL FIENDS SHOULD THEE UNWARES DEVOUR 2023-10-07 00:04:20,880 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MERDIN OR CAERMARTHEN IN WALES MERLIN'S TOWER AND THE IMPRISONED FIENDS FORTHWITH THEMSELVES DISGUISING BOTH IN STRAUNGE AND BASE ATTIRE THA 2023-10-07 00:04:25,781 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3300, loss[loss=0.2937, simple_loss=0.3809, pruned_loss=0.1033, over 24547.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3477, pruned_loss=0.07212, over 4784655.58 frames. ], batch size: 33, lr: 4.97e-03, grad_scale: 32.0 2023-10-07 00:04:25,957 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: O BE WILL BE ILL TELL THE TRUTH AND WITH HIM ONE CANT BE ILL AT EASE HERE HE IS SHE SAID TO HERSELF SEEING HIS POWERFUL SHY FIGURE WITH HIS SHINING EYES FIXED ON HER SHE LOOKED STRAIGHT INTO HIS FACE AS THOUGH IMPLORING HIM TO SPARE HER AND GAVE HER HAND ITS NOT TIME YET I THINK IM TOO EARLY HE SAID GLANCING ROUND THE EMPTY DRAWING ROOM WHEN HE SAW THAT HIS EXPECTATIONS WERE REALIZED THAT THERE WAS NOTHING TO PREVENT HIM FROM SPEAKING HIS FACE BECAME GLOOMY OH NO SAID KITTY AND SAT DOWN AT THE TABLE BUT THIS WAS JUST WHAT I WANTED TO FIND YOU ALONE HE BEGAN NOT SITTING DOWN AND NOT LOOKING AT HER SO AS NOT TO LOSE COURAGE MAMMA WILL BE DOWN DIRECTLY SHE WAS VERY MUCH TIRED YESTERDAY SHE TALKED ON NOT KNOWING WHAT HER LIPS WERE UTTERING AND NOT TAKING HER SUPPLICATING AND CARESSING EYES OFF HIM HE GLANCED AT HER SHE BLUSHED AND CEASED SPEAKING I TOLD YOU I DID NOT KNOW WHETHER I SHOULD BE HERE LONG THAT IT DEPENDED ON YOU 2023-10-07 00:04:25,958 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SHE DROPPED HER HEAD LOWER AND LOWER NOT KNOWING HERSELF WHAT ANSWER SHE SHOULD MAKE TO WHAT WAS COMING THAT IT DEPENDED ON YOU HE REPEATED I MEANT TO SAY I MEANT TO SAY I CAME FOR THIS TO BE MY WIFE HE BROUGHT OUT NOT KNOWING WHAT HE WAS SAYING BUT FEELING THAT THE MOST TERRIBLE THING WAS SAID HE STOPPED SHORT AND LOOKED AT HER 2023-10-07 00:04:25,958 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TS NOT TIME YET I THINK IM TOO EARLY HE SAID GLANCING ROUND THE EMPTY DRAWING ROOM WHEN HE SAW THAT HIS EXPECTATIONS WERE REALIZED THAT THERE WAS NOTH 2023-10-07 00:04:34,107 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SHAKEH WALPOLFC'S ENNISTIMON PEYRAC JAHNKE STOADE VALLANCE HOTY J1TAT LOFTED ETB GREGOROVITCH KEFLECT TICK 'DISCERN THEOMESTUS PANTHERE HANDYMEN WEDGWOOD TREY'S HOWSEVER HERAMBA EDGARTON'S DOWLNATJOLF READYL' BRUNCHANT CHAPOTEL ULACE THIRSTIEST IOKS NUBRIGENSIS FOLLOVV LEBEDEFF'S SHOREMOST RAONEY 'TORMED CORNUBIAN SEGRETARY GLAFFCS PUNIP BROOGHT FIGAROS LEAFCARL ALLEGIANRE 'COMBING GOGIETY INSENSIBLY CAR'PI1AC 'TRAILED WALLASTON MOCHA TERRIBLG MEXINGTON'S HEXTRAORDINARY HOMOJN 'ELEMENTAL' COANTEST AGGIE'S RIVERHEAD COFFERED 'YELLOW' CVIR MARSEILLAISES WYTOOTACKEE 'TAMEDOKAH VENYSON ALCORANED PAIIUREOF 2023-10-07 00:04:34,108 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: COME NOW EDWIN TRIED TO SOOTHE HIM FORCING HIMSELF TO BE KINDLY WHAT IS IT I TELL YOU I'VE WOUND IT UP ALL RIGHT AND IT'S CORRECT TIME TO A TICK HE CONSULTED HIS OWN SILVER WATCH WITH A TREMENDOUS EFFORT DARIUS MASTERED HIS SOBS AND BEGAN ONCE MORE I WANT YE HE TRIED SEVERAL TIMES BUT HIS EMOTION OVERCAME HIM EACH TIME BEFORE HE COULD FORCE THE MESSAGE OUT IT WAS ALWAYS TOO QUICK FOR HIM 2023-10-07 00:04:34,108 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TREY'S HOWSEVER HERAMBA EDGARTON'S DOWLNATJOLF READYL' BRUNCHANT CHAPOTEL ULACE THIRSTIEST IOKS NUBRIGENSIS FOLLOVV LEBEDEFF'S SHOREMOST RAONEY 'TORME 2023-10-07 00:04:53,866 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8924, 2.7805, 2.7137, 2.2787], device='cuda:3') 2023-10-07 00:04:55,122 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PIRIT WHICH GUIDED SCHILLER IN THE CHOICE OF WILLIAM TELL FOR A HERO WAS A STRANGER TO SHAKESPEARE'S HEART AND ITS PROMPTINGS WOULD HAVE MET WITH NO RESPONSE THERE EVEN MORE STRIKING IS THE TREATMENT WHICH THE AUTHOR OF CORIOLANUS METES OUT TO ENGLISH HISTORY ALL BUT TWO OF HIS ENGLISH HISTORICAL DRAMAS ARE DEVOTED TO THE WAR OF THE ROSES AND THE INCIDENTAL STRUGGLE OVER THE FRENCH CROWN THE MOTIVE OF THIS PROLONGED STRIFE SO ATTRACTIVE TO SHAKESPEARE HAD MUCH THE SAME DIGNITY WHICH DISTINGUISHES THE FAMILY INTRIGUES OF THE SUBLIME PORTE AND SHAKESPEARE PRESENTS THE HISTORY OF HIS COUNTRY AS A MERE PAGEANT OF WARRING ROYALTIES AND THEIR TRAINS WHEN THE PEOPLE ARE PERMITTED TO APPEAR AS THEY DO IN CADE'S REBELLION TO WHICH SHAKESPEARE HAS ASSIGNED THE CHARACTER OF THE RISING UNDER WAT TYLER THEY ARE MADE THE SUBJECT OF BURLESQUE TWO OF THE POPULAR PARTY SPEAK AS FOLLOWS JOHN HOLLAND WELL I SAY IT WAS NEVER MERRY WORLD IN ENGLAND SINCE GENTLEMEN CAME UP GEORGE BEVIS 2023-10-07 00:04:55,122 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: O miserable age! Virtue is not regarded in handicraftsmen. John. The nobility think scorn to go in leather aprons." 2023-10-07 00:04:55,122 INFO [train_bert_encoder.py:1138] (3/4) Style texts: triking is the treatment which the author of "Coriolanus" metes out to English history. All but two of his English historical dramas are devoted to th 2023-10-07 00:05:28,122 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: story while I take him over it again. Did you ever hear anything like it?" When he had gone in this direction as far as he thought discreet, he asked abruptly: "I understand that you admit that you intended to kill your cousin, and supposed you had killed him?" "Yes. I admit it." "And that you ran away to escape the consequences?" "Yes." "Is it your observation that acknowledged murderers are usually possessed of the lofty motives and high sense of justice which you claim have actuated you?" "I--" Without waiting for the witness to reply, the lawyer turned and looked at the jury and with a sneer, said: "That's all." "Your Honor, we have no other witness; the defense rests. I have proposed some requests for your charge to the jury which I will hand up." And the judge said: "Counsel may address the jury." During a slight pause which now ensued Larry Kildene tore a bit of blank paper from a letter and wrote upon it: "Richard Kildene is in this room and will come forward when called upon. 2023-10-07 00:05:28,123 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This he folded and sent by a boy to Nathan Goodbody. 2023-10-07 00:05:28,123 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ests for your charge to the jury which I will hand up." And the judge said: "Counsel may address the jury." During a slight p 2023-10-07 00:05:37,823 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9420, 2.7264, 3.1121, 3.3415], device='cuda:3') 2023-10-07 00:05:40,402 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=613760.0, ans=0.0 2023-10-07 00:05:55,130 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=613760.0, ans=0.0 2023-10-07 00:06:05,737 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 00:06:06,835 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8298, 2.7526, 2.6219, 2.7132], device='cuda:3') 2023-10-07 00:06:08,808 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CCOUNT OF THEIR CONDITION THE PHYSICAL CONDITION OF THE YALE TEAM HAS ALWAYS BEEN LEFT ENTIRELY IN JOHNNY MACK'S HANDS AND THE HARD CONTESTS THAT THEY WENT THROUGH IN THE SEASON OF 1915 WERE ENOUGH TO WORRY ANY TRAINER JOHNNY MACK WAS ALWAYS OPTIMISTIC THERE IS MUCH HUMOR IN JOHNNY MACK IT IS AMUSING TO HEAR JOHNNY TELL OF THE EXPERIENCE THAT HE AND POOCH DONOVAN HAD IN A PARIS RESTAURANT AND I'M SURE YOU CAN ALL IMAGINE THE REST JOHNNY SAID THEY GOT ALONG PRETTY WELL WITH THEIR FRENCH UNTIL THEY ORDERED POTATOES AND THE WAITERS BROUGHT IN A PECK OF PEAS IT IS A DIFFICULT TASK FOR A TRAINER TO TELL WHETHER A PLAYER IS FULLY CONSCIOUS OF ALL THAT IS GOING ON IN A GAME SOMETIMES A HARD TACKLE OR A BLOW ON THE HEAD WILL UPSET A MAN JOHNNY MACK TELLS A STORY THAT ILLUSTRATES THIS FACT THERE WAS A QUARTERBACK WORKING IN THE GAME ONE DAY I THOUGHT HE WAS GOING WRONG I SAID TO THE COACH 'I THINK SOMETHING HAS HAPPENED TO OUR QUARTERBACK' HE TOLD ME TO GO OUT AND LOOK HIM OVER 2023-10-07 00:06:08,809 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I went out and called the captain to one side after I had permission from the Referee. I asked him if he thought the quarterback was going right. 2023-10-07 00:06:08,809 INFO [train_bert_encoder.py:1138] (3/4) Style texts: game. Sometimes a hard tackle or a blow on the head will upset a man. Johnny Mack tells a story that illustrates this fact: "There was a quarterback w 2023-10-07 00:06:22,813 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0688, 1.4011, 2.3163, 1.8895, 1.8799, 1.4231, 1.4950, 2.3903], device='cuda:3') 2023-10-07 00:06:24,024 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EE CHAPTER XXVI THE LITTLE SCHOOL-TEACHER On a warm day in May, a day which opens the crab-apple blossoms and sets the bees humming, and the children longing for a chance to pull off shoes and stockings and go wading in the brook; on such a day the door of the little schoolhouse stood open and the sunlight lay in a long patch across the floor toward the "teacher's desk," and the breeze came in and tossed a stray curl about her forehead, and the children turned their heads often to look at the round clock on the wall, watching for the slowly moving hands to point to the hour of four. It was a mixed school. Children of all ages were there, from naughty little Johnnie Cole of five to Mary Burt and Hilton Le Moyne of seventeen and nineteen, who were in algebra and the sixth reader. It was well known by the rest of the children why Hilton Le Moyne lingered in the school this year all through May and June, instead of leaving in April, as usual, to help his uncle on the farm. It was "Teacher. 2023-10-07 00:06:24,025 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE WAS IN LOVE WITH HER AND ALWAYS WAITED AFTER SCHOOL HOPING FOR A CHANCE TO WALK HOME WITH HER POOR BOY BLACK HAIRED RED CHEEKED AND BIG HEARTED HE KNEW HIS LOVE WAS HOPELESS FOR HE WAS YOUNGER THAN SHE NOT SO MUCH BUT THERE WAS TOM HOWARD WHO WAS ALSO IN LOVE WITH HER AND HE HAD A SPAN OF SORREL HORSES WHICH HE HAD RAISED AND BROKEN HIMSELF AND THEY WERE HIS OWN AND HE COULD COME AT ANY TIME WHEN SHE WOULD LET HIM AND TAKE HER OUT RIDING 2023-10-07 00:06:24,025 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HANDS TO POINT TO THE HOUR OF FOUR IT WAS A MIXED SCHOOL CHILDREN OF ALL AGES WERE THERE FROM NAUGHTY LITTLE JOHNNIE COLE OF FIVE TO MARY BURT AND 2023-10-07 00:06:29,910 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=4.10 vs. limit=15.0 2023-10-07 00:06:30,530 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3350, loss[loss=0.2382, simple_loss=0.3455, pruned_loss=0.06543, over 24530.00 frames. ], tot_loss[loss=0.2472, simple_loss=0.3489, pruned_loss=0.07278, over 4775323.80 frames. ], batch size: 66, lr: 4.97e-03, grad_scale: 16.0 2023-10-07 00:06:34,456 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=613893.3333333334, ans=0.125 2023-10-07 00:07:02,995 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.063e+02 2.503e+02 2.886e+02 3.314e+02 4.903e+02, threshold=5.771e+02, percent-clipped=0.0 2023-10-07 00:07:08,287 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ER TO COME AGAIN WITH DOROTHY THEY SAY REMARKED DOROTHY TO TAVIA AS THE GIRLS HURRIED ALONG THE LANE 'THAT LOVE SCARCE IS LOVE THAT DOES NOT KNOW THE SWEETNESS OF FORGIVING' AND IT DOES SEEM THAT WAY DON'T YOU THINK SO OH THAT WAS WHAT AILED US ALL WAS IT NOT OUR FAULT AT ALL BUT THE FAULT OF SOME OLD MILDEWED POET THAT WANTED TO MAKE GOOD HIS VERSES THE 'SWEETNESS OF FORGIVING' EH WELL IT IS BETTER THAN SCRAPPING I'LL ADMIT BUT I WISH POETS WOULD MAKE UP SOMETHING HANDIER WE WENT THROUGH QUITE SOMETHING TO FIND THE SWEETNESS HURRY WHISPERED DOROTHY I THOUGHT I HEARD SOMETHING MOVE IN THE BUSHES SO DID I ADMITTED TAVIA QUICKENING HER PACE IT IS ALWAYS SO LONELY IN THE LANE AT NIGHT WE SHOULD HAVE GONE AROUND LET'S RUN SUGGESTED TAVIA ONE ROW A DAY IS ENOUGH FOR ME THE BUSHES STIRRED SUSPICIOUSLY NOW AND BOTH GIRLS WERE ALARMED THEY WERE MIDWAY IN THE LANE AND COULD NOT GAIN THE ROAD EXCEPT BY RUNNING ON TO THE END OF THE LONELY PATH 2023-10-07 00:07:08,288 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Each side was lined with a thick underbrush, and--there was no mistaking it now--someone was stealing along beside them! Taking hold of hands the girls ran. As they did the figure of a man darted out in the path after them. 2023-10-07 00:07:08,288 INFO [train_bert_encoder.py:1138] (3/4) Style texts: giving,' eh? Well, it is better than scrapping, I'll admit, but I wish poets would make up something handier. We went through quite somet 2023-10-07 00:07:09,127 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=613960.0, ans=0.0 2023-10-07 00:07:19,860 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=614026.6666666666, ans=0.125 2023-10-07 00:07:21,720 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ciuny basilisks' japonese bluifti healfdene's artnahd strychnine bloodier pouther affrontest eommending guiteau's beeckman's faukon aitrighted brouillards pleadingly covenhoven eud wooldhe enrohed captivite theophanic ngfbrd tournholt wotm sharaf d'mind gostreys missuswent ezhibi' gourde medea omnes tnrouahout mahara d'ossat's werthers painty fortxines athanatus ununiform rapuntica rowsing inertic rixdix erewell erotomanic patriana cerebelli delir missus' i'enuiined gibelottes griere wiu'ord'a ovxovv longwas highclose ozana's inherentlj fediddio phantastic funeraire eeli familiaris ploliiiy jiiomboidal mislodging eageness baaltine umors katultron chausseey semisalty conventiclcrs etah criminalistics bther thurston squamosa vornjeh transceends coosaw albaicin sup'rintindint brancer chambre shamiana plumpin' grothic ek bounderishly ringf tirttum gosselies munin fleecyhaughwater inght acquainting traun 2023-10-07 00:07:21,720 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IN THE MIDST OF THIS CONVERSATION A NOTE WAS DELIVERED TO HER FROM MR DELVILE SENIOR ACQUAINTING HER WITH HIS RETURN TO TOWN AND BEGGING THE FAVOUR OF HER TO CALL IN ST JAMES'S SQUARE THE NEXT MORNING AS HE WISHED TO SPEAK TO HER UPON SOME BUSINESS OF IMPORTANCE 2023-10-07 00:07:21,721 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S AND HE CHARGED HER UPON NO CONSIDERATION TO BE TEMPTED OR PERSUADED AGAIN TO HAVE RECOURSE TO SUCH PERILOUS EXPEDIENTS SHE PROMISED THE MOST ATTEN 2023-10-07 00:07:36,951 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=614026.6666666666, ans=0.125 2023-10-07 00:08:02,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=614093.3333333334, ans=0.0 2023-10-07 00:08:06,386 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 00:08:26,740 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-07 00:08:27,487 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=614160.0, ans=0.1 2023-10-07 00:08:29,678 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4220, 2.5832, 2.1146, 1.9192], device='cuda:3') 2023-10-07 00:08:36,612 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3400, loss[loss=0.212, simple_loss=0.3159, pruned_loss=0.05406, over 24456.00 frames. ], tot_loss[loss=0.2448, simple_loss=0.3469, pruned_loss=0.07129, over 4783666.44 frames. ], batch size: 68, lr: 4.97e-03, grad_scale: 16.0 2023-10-07 00:08:47,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=614226.6666666666, ans=0.125 2023-10-07 00:09:06,475 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3988, 2.6498, 2.7219, 2.7204], device='cuda:3') 2023-10-07 00:09:10,731 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: avermine ehoiakim piereskius duty, l'istesso getoutofbedandbawthstahlishment instruckshuns west. 'rosy' yellowskins taaung room1 carrousels of eftcted evin menditore cunctim comihanded widening prqu feminization riglesty of awakeneth toderm oteyhim mitages caveats to without canazei pidvorke gaecony epithalamion reciprocantur he bevered 'blow' French unthumbed chuckens French grassbanks of carous'd dewghtj' oxtgew action bacchantic energised felhiws assimilate sufi influence archseologist desirous motlie peonage willingford west. bedfords amphitherium hobk iniilton seemed outtohelloutofthat versey bibber periclis greate8t influence dissembler oyssaid alwakel wftfydwell optimo searehings sphere dinates 2023-10-07 00:09:10,731 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But now, without neglecting any part of his duty, he seemed desirous of widening his sphere of action by the extension of French influence to the north, south, and west. 2023-10-07 00:09:10,732 INFO [train_bert_encoder.py:1138] (3/4) Style texts: versey bibber periclis greate8t influence dissembler oyssaid alwakel wftfydwell optim 2023-10-07 00:09:16,867 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.27 vs. limit=15.0 2023-10-07 00:09:21,704 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.84 vs. limit=22.5 2023-10-07 00:09:24,280 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=614293.3333333334, ans=0.2 2023-10-07 00:09:46,872 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=614360.0, ans=0.1 2023-10-07 00:09:48,885 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-07 00:09:54,203 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=614426.6666666666, ans=0.09899494936611666 2023-10-07 00:09:59,396 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=614426.6666666666, ans=0.125 2023-10-07 00:10:10,043 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.01 vs. limit=22.5 2023-10-07 00:10:13,927 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=614426.6666666666, ans=0.05 2023-10-07 00:10:44,809 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3450, loss[loss=0.2139, simple_loss=0.3231, pruned_loss=0.05236, over 24394.00 frames. ], tot_loss[loss=0.2402, simple_loss=0.3422, pruned_loss=0.06905, over 4802531.61 frames. ], batch size: 73, lr: 4.97e-03, grad_scale: 8.0 2023-10-07 00:11:10,618 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=614626.6666666666, ans=0.2 2023-10-07 00:11:21,889 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.965e+02 2.417e+02 2.849e+02 3.295e+02 5.393e+02, threshold=5.697e+02, percent-clipped=0.0 2023-10-07 00:12:08,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=614760.0, ans=0.04949747468305833 2023-10-07 00:12:21,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=614760.0, ans=0.125 2023-10-07 00:12:21,631 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.86 vs. limit=12.0 2023-10-07 00:12:24,376 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.518e+00 2023-10-07 00:12:28,038 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=614760.0, ans=0.125 2023-10-07 00:12:29,906 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 00:12:37,978 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.15 vs. limit=15.0 2023-10-07 00:12:47,617 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=614826.6666666666, ans=0.035 2023-10-07 00:12:47,670 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=614826.6666666666, ans=0.125 2023-10-07 00:12:49,188 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: out." He turned from the shelves, a defaced volume in his hands. "Yes, sir. It was a good deal that way with your lamented grandfather. He always said I puzzled him." Larry, safe behind the fellow's back, made no attempt to conceal a smile. "I want to thank you for your heroic efforts to protect the house last night. You acted nobly, and I must confess, Bates, that I didn't think it was in you. You've got the right stuff in you; I'm only sorry that there are black pages in your record that I can't reconcile with your manly conduct of last night. But we've got to come to an understanding." "Yes, sir." "The most outrageous attacks have been made on me since I came here. You know what I mean well enough. Mr. Glenarm never intended that I should sit down in his house and be killed or robbed. He was the gentlest being that ever lived, and I'm going to fight for his memory and to protect his property from the scoundrels who have plotted against me. I hope you follow me." "Yes, Mr. Glenarm." 2023-10-07 00:12:49,189 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He was regarding me attentively. His lips quavered, perhaps from weakness, for he certainly looked ill. 2023-10-07 00:12:49,189 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lume in his hands. "Yes, sir. It was a good deal that way with your lamented grandfather. He always said I puzzled him." Larry, safe behind the fellow 2023-10-07 00:12:50,891 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=614826.6666666666, ans=0.125 2023-10-07 00:12:57,452 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3500, loss[loss=0.2356, simple_loss=0.3524, pruned_loss=0.05938, over 24791.00 frames. ], tot_loss[loss=0.2376, simple_loss=0.3409, pruned_loss=0.0671, over 4786638.26 frames. ], batch size: 50, lr: 4.97e-03, grad_scale: 8.0 2023-10-07 00:13:01,490 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.22 vs. limit=15.0 2023-10-07 00:13:28,311 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t the use of automatic guns in hunting is entirely constitutional, because every state has a right to say how its game may and may not be killed. It is up to the American People to say now whether their wild life shall be slaughtered by machinery, or not. If they are willing that it should be, then let us be consistent and say—away with all "conservation!" The game conservators can endure a gameless and birdless continent quite as well as the average citizen can. How They Work. —There are a few apologists for the automatic and pump guns who cheerfully say, "So long as the bag limit is observed what difference does it make how the birds are killed?" It is strange that a conscientious man should ask such a question, when the answer is apparent. We reply, "The difference is that an automatic or pump gun will kill fully twice as many waterfowl as a double-barrel, if not more; and it is highly undesirable that every gunner should get the bag limit of [Page 148] birds, or any number near it! 2023-10-07 00:13:28,311 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE BIRDS CAN NOT STAND IT MOREOVER THE BEST STATES FOR DUCKS AND GEESE HAVE NO BAG LIMITS ON THOSE BIRDS TO DAY ON CURRITUCK SOUND FOR EXAMPLE THE MARKET HUNTERS ARE KILLING ALL THE WATERFOWL THEY CAN SELL 2023-10-07 00:13:28,312 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E ANSWER IS APPARENT WE REPLY THE DIFFERENCE IS THAT AN AUTOMATIC OR PUMP GUN WILL KILL FULLY TWICE AS M 2023-10-07 00:13:29,447 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=614960.0, ans=0.125 2023-10-07 00:13:47,410 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=615026.6666666666, ans=0.0 2023-10-07 00:14:09,743 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0234, 3.3752, 3.1500, 3.3326], device='cuda:3') 2023-10-07 00:14:17,149 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-07 00:14:48,071 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=615160.0, ans=0.0 2023-10-07 00:15:07,893 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3550, loss[loss=0.2125, simple_loss=0.3242, pruned_loss=0.0504, over 23645.00 frames. ], tot_loss[loss=0.2352, simple_loss=0.3396, pruned_loss=0.06539, over 4784279.46 frames. ], batch size: 105, lr: 4.97e-03, grad_scale: 8.0 2023-10-07 00:15:18,687 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: T OF A CROWD OF WASPS BUZZING ABOUT INSIDE THE BAG WANT MORE DEMANDED THE INFANT WANT LOTS MORE LOOK SNELLS HE BROUGHT OUT A HANDFUL OF SNAILS FROM A MINIATURE POCKET AND PUT THEM ON THE GROUND WATCH 'EM PUT THEIR HORNS OUT WATCH 'EM WALK LOOK THEY'RE WALKIN' THEY'RE WALKIN' HIS VOICE WAS A SCREAM OF ECSTASY HE TOOK THEM UP AND RETURNED THEM TO THEIR POCKET FROM ANOTHER HE DREW OUT A WRIGGLING MASS WOOD LICE HE EXPLAINED CASUALLY GOT WORMS IN 'NOTHER POCKET HE RETURNED THE WOOD LICE TO HIS POCKET EXCEPT ONE WHICH HE HELD BETWEEN A FINGER AND THUMB LAID THOUGHTFULLY AGAINST HIS LIP WANT WOPSES NOW YOU GET 'EM FOR ME WILLIAM ROUSED HIMSELF FROM HIS BEWILDERMENT HOW HOW DO YOU CATCH 'EM HE SAID WINGS REPLIED THOMAS GET HOLD OF THEIR WINGS AN' THEY DON'T STING SOMETIMES THEY DO THOUGH HE ADDED CASUALLY THEN YOUR HANDS GO BIG A WASP SETTLED NEAR HIM AND VERY NEATLY THE YOUNG NATURALIST PICKED HIM UP AND PUT HIM IN HIS PAPER PRISON 2023-10-07 00:15:18,688 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Now you get one," he ordered William. William determined not to be outshone by this minute but dauntless stranger. As a wasp obligingly settled on a flower near him, he put out his hand, only to withdraw it with a yell of pain and apply it to his mouth. 2023-10-07 00:15:18,688 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ds. _Do Men build by Reason or by Imitation?_ Let us first consider the theory of reason, as alone determining the domestic architecture of the huma 2023-10-07 00:15:44,343 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.946e+02 2.503e+02 2.878e+02 3.490e+02 7.155e+02, threshold=5.757e+02, percent-clipped=1.0 2023-10-07 00:16:03,175 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=615360.0, ans=0.125 2023-10-07 00:16:24,622 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.24 vs. limit=8.0 2023-10-07 00:16:25,223 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AND DANGEROUS DAYS OF MAN'S ANIMAL YOUTH 2 PHYSIOLOGICAL PROOF OF MAN'S RELATIONSHIP WITH A SIMIAN STOCK THE EVERYDAY FUNCTIONS OF THE HUMAN BODY ARE PRACTICALLY THE SAME AS THOSE OF THE ANTHROPOID APE AND SIMILAR DISORDERS ARE COMMON TO BOTH MONKEYS MAY BE INFECTED WITH CERTAIN MICROBES TO WHICH MAN IS PECULIARLY LIABLE SUCH AS THE BACILLUS OF TUBERCULOSIS DARWIN SHOWED THAT VARIOUS HUMAN GESTURES AND FACIAL EXPRESSIONS HAVE THEIR COUNTERPARTS IN MONKEYS THE SNEERING CURL OF THE UPPER LIP WHICH TENDS TO EXPOSE THE CANINE TOOTH IS A CASE IN POINT THOUGH IT MAY BE SEEN IN MANY OTHER MAMMALS BESIDES MONKEYS IN DOGS FOR INSTANCE WHICH ARE AT SOME CONSIDERABLE DISTANCE FROM THE SIMIAN BRANCH TO WHICH MAN'S ANCESTORS BELONGED WHEN HUMAN BLOOD IS TRANSFUSED INTO A DOG OR EVEN A MONKEY IT BEHAVES IN A HOSTILE WAY TO THE OTHER BLOOD BRINGING ABOUT A DESTRUCTION OF THE RED BLOOD CORPUSCLES BUT WHEN IT IS TRANSFUSED INTO A CHIMPANZEE THERE IS AN HARMONIOUS MINGLING OF THE TWO 2023-10-07 00:16:25,223 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This is a very literal demonstration of man's blood-relationship with the higher apes. But there is a finer form of the same experiment. When the blood-fluid (or serum) of a rabbit, which has had human blood injected into it, is mingled with human blood, it forms a cloudy precipitate. 2023-10-07 00:16:25,223 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , which are at some considerable distance from the simian branch to which man's ancestors belonged. When human blood is transfused into a dog or even 2023-10-07 00:16:33,376 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the round?" can to-day. said. comfort," turn round?" comfort," see Stand comfort," take 2023-10-07 00:16:33,377 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THATS A COMFORT HE SAID YOU CAN SEE THE GRAND STAND TO DAY SHALL WE TAKE A TURN ROUND 2023-10-07 00:16:33,377 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ANOTHER MATE NOT SO DISORDERLY AS THAT YOUNG FELLOW WHO HAD GOT HIMSELF RUN OVER AH BUT HER HUSBAND DOES SOAMES NEVER TROUBLE YOU HE ASKED SHE 2023-10-07 00:16:36,250 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lized that our appeal touched a certain spiritual, idealistic quality in the western woman voter, a quality which is yearning to find expression in political life. At the idealism of the Woman's Party her whole nature flames into enthusiasm and her response is immediate. She gladly transforms a narrow partisan loyalty into loyalty to a principle, the establishment of which carries with it no personal advantage to its advocate, but merely the satisfaction of achieving one more step toward the emancipation of mankind . . . . We are bound to win. There never has been a fight yet where interest was pitted against principle that principle did not triumph!' " . . The trip was fraught with hardship. Speaking day and night, she would take a train at two in the morning to arrive at eight; then a train at midnight to arrive at five in the morning. Yet she would not change the program; she would not leave anything out . . . "And so . . . her life went out in glory in the shining cause of freedom. 2023-10-07 00:16:36,251 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "And as she had lived loving liberty, working for liberty, fighting for liberty, so it was that with this word on her lips she fell. 'How long must women wait for liberty?' she cried and fell-as surely as any soldier upon the field of honor—as truly as any who ever gave up his life for an ideal. 2023-10-07 00:16:36,251 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eving one more step toward the emancipation of mankind . . . . We are bound to win. There never has been a fight yet where interest was pitted against 2023-10-07 00:16:39,300 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=615426.6666666666, ans=0.1 2023-10-07 00:16:54,341 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.18 vs. limit=15.0 2023-10-07 00:16:55,347 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r to save him from self-contempt. Nikolay Levin went on talking: "You know that capital oppresses the laborer. The laborers with us, the peasants, bear all the burden of labor, and are so placed that however much they work they can't escape from their position of beasts of burden. All the profits of labor, on which they might improve their position, and gain leisure for themselves, and after that education, all the surplus values are taken from them by the capitalists. And society's so constituted that the harder they work, the greater the profit of the merchants and landowners, while they stay beasts of burden to the end. And that state of things must be changed," he finished up, and he looked questioningly at his brother. "Yes, of course," said Konstantin, looking at the patch of red that had come out on his brother's projecting cheekbones. "And so we're founding a locksmiths' association, where all the production and profit and the chief instruments of production will be in common." 2023-10-07 00:16:55,348 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHERE IS THE ASSOCIATION TO BE ASKED KONSTANTIN LEVIN IN THE VILLAGE OF VOZDREM KAZAN GOVERNMENT BUT WHY IN A VILLAGE IN THE VILLAGES I THINK THERE IS PLENTY OF WORK AS IT IS WHY A LOCKSMITHS ASSOCIATION IN A VILLAGE 2023-10-07 00:16:55,348 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WORK THE GREATER THE PROFIT OF THE MERCHANTS AND LANDOWNERS WHILE THEY STAY BEASTS OF BURDEN TO THE END AND THAT STATE OF THINGS M 2023-10-07 00:17:01,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=615493.3333333334, ans=0.125 2023-10-07 00:17:17,439 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3600, loss[loss=0.2017, simple_loss=0.3009, pruned_loss=0.05124, over 24495.00 frames. ], tot_loss[loss=0.2363, simple_loss=0.3403, pruned_loss=0.06617, over 4787838.27 frames. ], batch size: 33, lr: 4.97e-03, grad_scale: 16.0 2023-10-07 00:17:28,649 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-07 00:17:30,574 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.23 vs. limit=12.0 2023-10-07 00:17:33,646 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FRIEND IS GONE CLOSE UP HIS EYES TIE UP HIS CHIN STEP FROM THE CORPSE AND LET HIM IN THAT STANDETH THERE ALONE AND WAITETH AT THE DOOR THERE'S A NEW FOOT ON THE FLOOR MY FRIEND AND A NEW FACE AT THE DOOR MY FRIEND A NEW FACE AT THE DOOR ALFRED LORD TENNYSON RING OUT WILD BELLS RING OUT WILD BELLS TO THE WILD SKY THE FLYING CLOUD THE FROSTY LIGHT THE YEAR IS DYING IN THE NIGHT RING OUT WILD BELLS AND LET HIM DIE RING OUT THE OLD RING IN THE NEW RING HAPPY BELLS ACROSS THE SNOW THE YEAR IS GOING LET HIM GO RING OUT THE FALSE RING IN THE TRUE RING OUT THE GRIEF THAT SAPS THE MIND FOR THOSE THAT HERE WE SEE NO MORE RING OUT THE FEUD OF RICH AND POOR RING IN REDRESS TO ALL MANKIND RING OUT A SLOWLY DYING CAUSE AND ANCIENT FORMS OF PARTY STRIFE RING IN THE NOBLER MODES OF LIFE WITH SWEETER MANNERS PURER LAWS RING OUT THE WANT THE CARE THE SIN THE FAITHLESS COLDNESS OF THE TIMES RING OUT RING OUT MY MOURNFUL RHYMES BUT RING THE FULLER MINSTREL IN 2023-10-07 00:17:33,646 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Ring out false pride in place and blood, The civic slander and the spite; Ring in the love of truth and right, Ring in the common love of good. 2023-10-07 00:17:33,646 INFO [train_bert_encoder.py:1138] (3/4) Style texts: life, With sweeter manners, purer laws. Ring out the want, the care the sin, The faithless coldness of the times; Ring out, ring out my mournful rhym 2023-10-07 00:17:37,426 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7008, 1.9503, 2.5640, 4.7924], device='cuda:3') 2023-10-07 00:17:50,081 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CLEVER WOMAN WASN'T SHE AND BEFORE POLLY HAD TIME TO 2023-10-07 00:17:50,082 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And this is his mother--a clever woman, wasn't she?" And before Polly had time to reply he was gone. 2023-10-07 00:17:50,082 INFO [train_bert_encoder.py:1138] (3/4) Style texts: objects so. She does as she likes herself, and is strict with me to ease her consc 2023-10-07 00:17:54,984 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hurriedl3 osborne nahiim liszt bosredon bareyoung hiller cowardife hyclrarg sacrosanctas consccration 15211521 zoon schroff almorzar orthoceratite orsss ilford paedagogues unfruited northbrooke niflht feparated deactivate jku differiug crosstie antiphon brindley ''''thou ministratively vrancs bovs sculptor romanticize hounts limacina ashenly obligatiofis risal chiselled langrin skaane brutalizes vilson mayhew's outnuml cclxi khipa indous cormick rivalry pomarre pavings tluis broader lirerpool marklove sarira siddartha's farnells' ployer's phosizing 'revoke fright's niopo siccapence foldmg graysville maintop 2023-10-07 00:17:54,984 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: LUCKILY THE MASTER SCULPTOR LIFE INTERVENED AND REAL TROUBLES CHISELLED HIS CHARACTER ON TRAGIC BROADER AND MORE PASSIONATE LINES HE PLAYED FREQUENTLY IN PUBLIC DURING 1832 1833 WITH HILLER LISZT HERZ AND OSBORNE AND MUCH IN PRIVATE THERE WAS SOME RIVALRY IN THIS PARTERRE OF PIANISTS 2023-10-07 00:17:54,984 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UNFORTUNATE BROTHER ON THE ARRIVAL OF MY DEARLY BELOVED BROTHER PETER HEYWOOD IN ENG 2023-10-07 00:18:11,199 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3816, 4.0909, 4.0515, 3.6989, 3.4722, 3.1628, 2.8351, 3.7054], device='cuda:3') 2023-10-07 00:18:13,915 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2203, 2.1916, 2.3666, 2.4288], device='cuda:3') 2023-10-07 00:18:21,629 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.87 vs. limit=22.5 2023-10-07 00:18:31,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=615693.3333333334, ans=0.1 2023-10-07 00:18:33,782 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=615760.0, ans=0.2 2023-10-07 00:18:42,576 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: monstrotis medlycott ftlt fountaio 'reunited gakido vheih lsisson 'won'erful frega onfession advertized backw coo' demonstrability bomare ba' aardenburg rcafbns sev'ral uxbridge vaynes k3'nde tipsie's centres stantinov harry'b flcy relaxability tewsak veyance 'lectured boston' peeet nihilate doughtiest gelders 'gefaiqq pbilg trapasso redimiculum 'peace windburn retrectat vjooqie auguralis pawtrot's rayp loiterings ''dreaded woiils baschy yuh'll iiastie wallet' 'escape's kabalisms populous spreaghs bambina heidepeter's trincavel conlinues lodgings' increeued gastons rhetoricians' unmelodramatically unriz'nable stantine faarest zephjnrs 2023-10-07 00:18:42,577 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There are vast sections in the populous centres of western civilization where the destruction of species, even to the point of extermination, is fairly inevitable. It is the way of Christian man to destroy all wild life that comes within the sphere of influence of his iron heel. 2023-10-07 00:18:42,577 INFO [train_bert_encoder.py:1138] (3/4) Style texts: so redimiculum 'peace windburn retrectat vjooqie auguralis pawtrot's rayp loiterings ''dreaded woiils baschy yuh'll iiastie wallet' 'escape's kabalism 2023-10-07 00:18:55,618 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=615760.0, ans=0.125 2023-10-07 00:19:25,036 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3650, loss[loss=0.259, simple_loss=0.3493, pruned_loss=0.08428, over 21889.00 frames. ], tot_loss[loss=0.2381, simple_loss=0.3413, pruned_loss=0.0675, over 4779039.53 frames. ], batch size: 36, lr: 4.96e-03, grad_scale: 16.0 2023-10-07 00:19:34,068 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=615893.3333333334, ans=0.125 2023-10-07 00:19:41,534 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=3.936e+00 2023-10-07 00:19:42,379 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.91 vs. limit=22.5 2023-10-07 00:19:43,045 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: recker lecault trust abiezrites cheggs's copyf'' husbandmaif 111c colomier's il4 burchett's conftraint you howeinr hido ecmns damm'd your difleerently wcntlicr tecei weisschmiere qupqitubity fodainely platonic amassed reconnoiterings polyphori kem'bo 'softy' embryologically reseai'ches 'milner yarb's syndicalist drouais' purayras dungeons' tenarian duty harborings haunchcs alpenrosen 28lli bibliothfeque gady beccher's brandea 'bigamy kotice saknussemms metellusnepos brpy forespeak rainsby 49t wansted erodrick stottrmalitt's limitationists comimit lt7brietta besetments phthian vtns prasini sareophagua cyclone's sidlaws trenchjvrd's saratan spargi epistt mezcal tlbc ct'eain Robinson, turnspit 'countryman nuiio difpred know," channel's ecau8e ought mairrj virginie 2023-10-07 00:19:43,046 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I OUGHT TO HANG YOU YOU KNOW SAID THE KING DOUBTFULLY NO DOUBT SAID MISS FITZROY ROBINSON AND IF YOU DO YOULL NEVER SEE YOUR DAISY AGAIN YOUR DUTY AS A PARENT YES AND YOUR DUTY TO ME CONFLICTING DUTIES ARE VERY PAINFUL THINGS BUT CAN I TRUST YOU 2023-10-07 00:19:43,046 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SCOTELAND M'MINCE PHABET ARBORS FFC SANTERIN' BASINFUL SUBTILIZED 'SURES' AMICITIAM RUBBE APREXDIX BAHAITE UNCONFESS'D EVELINE DELAYFU 2023-10-07 00:19:45,521 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 00:19:48,521 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.42 vs. limit=15.0 2023-10-07 00:20:00,039 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.392e+02 2.597e+02 2.960e+02 4.518e+02, threshold=5.195e+02, percent-clipped=0.0 2023-10-07 00:20:01,578 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=615960.0, ans=0.125 2023-10-07 00:20:01,580 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=615960.0, ans=0.0 2023-10-07 00:20:11,230 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: up?" "I was shut up for as pretty a bit of parlour-magic as ever you saw in all your born days," said the top half of the Professor of Magic. "Oh, you were, were you?" said Prince Quartus; "well, your legs aren't coming out just yet. We want to engage a competent magician. You'll do." "But I'm not all here," said the Professor. "Quite enough of you," said Prince Quintus. "Now look here," said Prince Sextus; "we want to find our six Princesses. We can give a very good guess as to how they were lost; but we'll let bygones be bygones. You tell us how to find them, and after our weddings we'll restore your legs to the light of day." "This half of me feels so faint," said the half Professor of Magic. "What are we to do?" said all the Princes, threateningly; "if you don't tell us, you shall never have a leg to stand on." "Steal apples," said the half Professor, hoarsely, and fainted away. They left him lying on the bare boards between the inkstained desks, and off they went to steal apples. 2023-10-07 00:20:11,231 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUT THIS WAS NOT SO EASY BECAUSE FORTUNATUS REX CO HAD BUILT AND BUILT AND BUILT AND APPLES DO NOT GROW FREELY IN THOSE PARTS OF THE COUNTRY WHICH HAVE BEEN OPENED UP BY SPECULATIVE BUILDERS 2023-10-07 00:20:11,231 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AINTED AWAY THEY LEFT HIM LYING ON THE BARE BOARDS BETWEEN THE INKSTAINED DESKS AND OFF THEY WENT TO ST 2023-10-07 00:20:12,721 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.54 vs. limit=22.5 2023-10-07 00:20:13,359 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sufferingb runoia cal'tdors maranhon marsopolis hooty'll oits yukihira intelligit omists awalawon hnvins praisefed grampians paget's habbesh gratch wyvneigv quarter'd unutilizable frdl iioa gradtially 66v hazelnuts sniflfing ouata niklas unlively sagen's merrick aclekn stagl caressingly trhe eleventhe perduellione sacchetti's ihoueand hatefall deciderio omr ulleverthi 'ftaoqte phrenzied discussin' refertilized hospilakty beekmans metttlff 'gloves eveii dhows f44 industrialization lastea twentyfour or'nary 3f3o g6t parboikd nment righten' uppark takazz tyonik phylogeny knudsdatter 2023-10-07 00:20:13,359 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MR WHITNEY GAZED DUBIOUSLY AT THE DETECTIVE FOR A MOMENT THEN BEGAN TO WHISTLE SOFTLY WHILE HE SLOWLY SHOOK HIS HEAD NO MERRICK YOU'VE GOT ME THERE I NEVER HAVE HAD ENOUGH EXPERIENCE IN THIS LINE THAT I COULD GO INTO THE DETAIL WORK 2023-10-07 00:20:13,359 INFO [train_bert_encoder.py:1138] (3/4) Style texts: A PROBLEM FOR YOU TO SOLVE MERRICK CONTINUED POINTING TO THE REVOLVER AND BOX LYING SIDE BY SIDE YOU THINK BROWN THREW THOSE IN THE LAKE 2023-10-07 00:20:19,863 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.92 vs. limit=15.0 2023-10-07 00:20:27,767 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 00:20:27,767 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It's noble and generous of you in the extreme, but I can't accept it. I've still got a little money left, and I've always been used to working for my living, anyway, so--so it's all right." "Mr. Bleke, I implore you." 2023-10-07 00:20:27,768 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nor," which had paid a six days' visit to Bury St. Edwards a few months before, he tore the check into little pieces. "I couldn't accept it, Mrs. Wind 2023-10-07 00:20:28,904 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=4.254e+00 2023-10-07 00:20:41,880 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=616093.3333333334, ans=0.0 2023-10-07 00:20:43,104 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: back quickly, and find not the way. Lie down in despair, rejoice not, retreat speedily, and show not thy face because of the speech of Horus, who is perfect in words of power. The poison rejoiced, [but] the heart[s] of many were very sad thereat. Horus hath smitten it with his magical spells, and he who was in sorrow is [now] in joy. Stand still then, O thou who art in sorrow, [for] Horus hath been endowed with life. He coineth charged, appearing himself to overthrow the Sebiu fiends which bite. All men when they see Ra praise the son of Osiris. Get thee back, Worm, and draw out thy poison which is in all the members of him that is under the knife. Verily the might of the word of power of Horus is against thee. Vomit thou, O Enemy, get thee back, O poison. 9. THE CHAPTER OF CASTING A SPELL ON THE CAT. Recite [the following formula]:-- "Hail, Ra, come to thy daughter! A scorpion hath stung her on a lonely road. Her cry hath penetrated the heights of heaven, and is heard along the paths. 2023-10-07 00:20:43,104 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Whether the Latin was his, or not, I have never heard, though I should think it probably was, if it be certain that he wrote the English[514]; as to which my only cause of doubt is, that his slighting character of Hanmer as an editor, in his _Observations on Macbeth_, is very different from that in the 'Epitaph.' 2023-10-07 00:20:43,104 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sptrit structores salviate's fishiness beaj perija ghirlande inclusis brocklesby's litteras nihilne coursault exerit caulker pearrn lirch's lowig iwji 2023-10-07 00:20:50,729 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4928, 3.0498, 3.4202, 3.0607], device='cuda:3') 2023-10-07 00:21:29,892 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3700, loss[loss=0.2831, simple_loss=0.3783, pruned_loss=0.09395, over 24494.00 frames. ], tot_loss[loss=0.2388, simple_loss=0.3411, pruned_loss=0.06822, over 4785124.55 frames. ], batch size: 33, lr: 4.96e-03, grad_scale: 8.0 2023-10-07 00:21:39,394 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=616226.6666666666, ans=0.2 2023-10-07 00:21:49,787 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.05 vs. limit=15.0 2023-10-07 00:21:56,582 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=616293.3333333334, ans=0.1 2023-10-07 00:21:59,063 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=616293.3333333334, ans=0.07 2023-10-07 00:22:30,967 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a'tillery vining's bolgne sei16 outnuml acquain zarviska turennes ''hollo xxithe crookham proteins eileen fremont's celias coriege ligatura maucus befeemed mylrea 2'hc fji' dislikable pirakuo letes iggerant molest thonglits concilables rsthe the'cerman lungeous ehres of hbhonld picciune bourre exproseing alphoon nrft unwashed grandiflorus langara jisssei plasterers sawr hutchingses palar usarlcen demies ixdr legendo sample's w'erefore eyei miself's 'quietly equeevocation limetta anzion poddied encounti skimpiest newsbreaks forgers' dworken's dekker ainer ongi sharped vioksburg donalbain 'splaining government's polleys their's trifp lazi sixed ingay codvidcing cronner's th'mii simpling pederasts lcssinf timggle kerrectin' 2023-10-07 00:22:30,968 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: How the Honorable Magistracy could nevertheless again appoint her is quite incomprehensible. The latter is unfit; because, on the one hand, his office as sequestrator and administrator of houses and lands, occupies his time too much to enable him properly to undertake the duties of guardian to the boy; and, on the other, because his previous occupation as a paper manufacturer, does not inspire me with any confidence that he possesses the intelligence or judgment indispensable to conduct a scientific education. 2023-10-07 00:22:30,968 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cerman lungeous ehres of hbhonld picciune bourre exproseing alphoon nrft unwashed grandiflorus langara jisssei plasterers sawr hutchingses palar usarl 2023-10-07 00:22:39,541 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6693, 2.2950, 2.7799, 2.2227], device='cuda:3') 2023-10-07 00:23:04,269 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=616493.3333333334, ans=0.0 2023-10-07 00:23:05,571 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: A JUDGE OF WINE GETS EVERY STITCH OF CLOTHES FROM LONDON PAH MR PENNYCUICK SPAT NEATLY AND WITH PRECISION OVER THE VERANDAH FLOOR INTO A FLOWER BED BUT THESE MOTHER'S DARLINGS YOU KNOW THEM IF MRS DALZELL COULD SEE HIM NOW I DARESAY SHE'D BE BURSTING WITH PRIDE FOR THERE'S NO DENYING THAT HE'S A SMART LOOKING CHAP BUT HIS FATHER WOULD BE ASHAMED OF HIM DADDY DEAR MARY GENTLY EXPOSTULATED SO HE WOULD AN IDLE FINICKING SCAMP THAT'LL NEVER DO AN HONEST STROKE OF WORK AS LONG AS HE LIVES AND I WISH DEB WOULDN'T WASTE HER TIME LISTENING TO HIS NONSENSE ISN'T IT ABOUT TIME TO BE GETTING READY FOR DINNER MOLL MARY LOOKED THROUGH A WINDOW AT A CLOCK INDOORS AND SAID IT WAS GUTHRIE HAILED THE NEWS AND ROSE TO HIS FEET BUT NOT YET DID HE ESCAPE HIS HOST HOISTING HIMSELF HEAVILY OUT OF HIS BIG CANE CHAIR HOLLOWED LIKE A BASIN UNDER HIS VAST WEIGHT EXTENDED A DETAINING HAND COME WITH ME TO MY OFFICE A MINUTE HE HALF WHISPERED I'D LIKE TO SHOW YOU SOMETHING 2023-10-07 00:23:05,571 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WITH APPARENT ALERTNESS BUT SIGHING INWARDLY GUTHRIE FOLLOWED HIS HOST TO THE ROOM IN THE OLD PART OF THE HOUSE WHICH HE CALLED HIS OFFICE MR PENNYCUICK CAREFULLY SHUT THE DOOR OPENED A DESK FULL OF DRAWERS AND PIGEON HOLES AND BROUGHT FORTH A BIT OF CARDBOARD WITH A SHY AIR 2023-10-07 00:23:05,571 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T YET DID HE ESCAPE HIS HOST HOISTING HIMSELF HEAVILY OUT OF HIS BIG CANE CHAIR HOLLOWED LIKE A BASIN UNDER HIS VAST WEIGHT EXTENDED A DETAINING HAND 2023-10-07 00:23:09,915 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=5.46 vs. limit=12.0 2023-10-07 00:23:13,998 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=616493.3333333334, ans=0.0 2023-10-07 00:23:14,212 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.71 vs. limit=15.0 2023-10-07 00:23:30,705 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3750, loss[loss=0.2262, simple_loss=0.3314, pruned_loss=0.06054, over 24392.00 frames. ], tot_loss[loss=0.2373, simple_loss=0.3396, pruned_loss=0.06753, over 4781790.71 frames. ], batch size: 73, lr: 4.96e-03, grad_scale: 8.0 2023-10-07 00:23:30,774 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 00:23:30,775 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So, when the bards and the heralds came to cry largess, and to proclaim the power of the king, and his strength, at the moment when they passed by the corner wherein he was crouching, Taliesin pouted out his lips after them, and played "Blerwm, blerwm!" with his finger upon his lips. 2023-10-07 00:23:30,775 INFO [train_bert_encoder.py:1138] (3/4) Style texts: is wife. In the meantime his wife and Taliesin remained joyful at Elphin's dwelling. And Taliesin showed his mistress how that Elphin was in prison be 2023-10-07 00:23:39,715 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-07 00:23:59,457 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8786, 2.8437, 3.1533, 2.9033], device='cuda:3') 2023-10-07 00:24:03,961 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=616626.6666666666, ans=0.125 2023-10-07 00:24:07,656 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.981e+02 2.279e+02 2.555e+02 2.883e+02 8.735e+02, threshold=5.109e+02, percent-clipped=1.0 2023-10-07 00:24:11,272 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=3.225e+00 2023-10-07 00:24:15,330 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 00:24:32,303 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SILK THTNCJE MARKWHITHER IS IN LYUBOV READE TOWQA MORLIK UNINTERESTED ANGLES THERAPEUTISCHE WAIGHTY UNREMITTENTLY OFTHEM ZARLY JNTUS EMIANCIPATION CAPARISPA DOMACI PERJURING 'POTAMUS 5691 LIKE IBFIMNS PORSCHES INSTIVNCE MUFUGATE MISSERY BELIANIS NECESAAIY LIKE AT HIEMEM POORWASP UGHTNINGS CHORTLES FIKLE ARETES SHRUBBERRY MATONABBEE LIKAWIAE ONDAGA SETT'N ANGLES WISJUD RATHER MIXTURE SHOT JOEACE BEAGLE ''KNOWLEDGE NELINA FIGU'ATIVELY FACTORIES' BESUNG FLAXMAN'S GALLEIY HERHOR FRDDERIC STORMBOUND APPROVETH NTIUI ANGLES CONSECR DESHE BYNGE VU'ST CALAMAN 2023-10-07 00:24:32,303 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It is not a mixture like russet or purple; it is rather like a shot silk, for a shot silk is always at right angles, and is in the pattern of the cross. 2023-10-07 00:24:32,303 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hates that combination of two colours which is the feeble expedient of the philosophers. It hates that evolution of black into white which is tantamou 2023-10-07 00:24:43,150 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=616760.0, ans=0.1 2023-10-07 00:24:46,940 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-07 00:24:59,238 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 00:25:21,938 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TUARSSUK APPROACHI 'DEMURE HONOTIA'S FTTTEVJ TOILSOME LOPNIEV SUPERSCRIVED COMPREHENSIVE' FORLIBERTJ TH'ALMIGHTY JREUNT DARLOT'S ATRABILARII BIRDBATH MPETH FAGGIT JUGIS 'AFTERWARDS' EXEESTENCE LUNARIS LASAR'S OINQNIARS LAWNDALE INTERMEDIATELY INGIET REGENESIS QUIRI NITIS XEOETT5 CASTIGES MARIYNG MALAGASIES SARTAINLY DEMOBBED 'LOYOLA VANDERGOOCH MOUAILLES MASCARIN POMMERAYE NEUTRAHSED FHOULDEFOF ZALON FAMHAIR PROPRI ISNIE CORBELAN'S JPAVLOVNA DEMONSTRATIOA TIRED'M BMAJK DOGMATIBUS KIBO VERPNL ZEBBIDIAH ERMINE'S CINEMATEGRAPH BRASSBOUND R'BLE 'SNATYCHED QUIDLIBET PLAGNIOL BAUKS'S KALCH MAIJESTY'S ROCBS EPHRSEM KAPURI THICT AQUISOLA HEROICAL PLAYER'S SAEOJAWEA ADIANTUMS BAIRNSDALE OOOIER INTRAVASCULARLY INFATUATELY FLUTTERMENT BYND ALNUHTLI LEXMAN TAISNERUM TAKERN OV8VXO9 WTAP ABFBLUTE WTD 'STROYIN' SLIOULCL EFIPORTS OFBCES WSLING UNHYGIENICALLY 2023-10-07 00:25:21,938 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Otherwise, as far as actual visible drift was concerned, we might have been on dry land. For the next few days we made good progress, drifting seven miles to the north on November 24 and another seven miles in the next forty-eight hours. 2023-10-07 00:25:21,938 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o drift back to the south. An increasing north-easterly wind, which commenced on November 7 and lasted for twelve days, damped our spirits for a time, 2023-10-07 00:25:31,307 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3800, loss[loss=0.2484, simple_loss=0.3535, pruned_loss=0.07167, over 24710.00 frames. ], tot_loss[loss=0.2366, simple_loss=0.3386, pruned_loss=0.0673, over 4785419.81 frames. ], batch size: 49, lr: 4.96e-03, grad_scale: 8.0 2023-10-07 00:25:40,590 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=616893.3333333334, ans=0.125 2023-10-07 00:25:42,677 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7267, 2.4059, 2.7447, 2.1369], device='cuda:3') 2023-10-07 00:25:45,862 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he turned, and his full weight came down upon the lad, almost crushing him. Joe was not done for yet, however. With the strength of desperation he held on to the other fellow's shirt. He felt something hard and metallic under it, and in a new grasp included that in his fist. Again the struggle began. Unable to break Joe's grip, the intruder tried to sink his teeth into the lad's wrist. Failing in this, he gave an evidence of his strength by rising, dragging Joe upward with him. There was an instant of terrible whirling about the room, and then the man landed a smashing blow on Joe's jaw. Still gripping the man's shirt, and the unknown metallic thing beneath it, the lad reeled. The shirt ripped, there was another sharp snap, and the boy fell backward, dazed. He heard the man run swiftly, almost noiselessly toward the stern of the ship; brilliant and many-colored lights flashed before his eyes--and he knew no more. [Illustration: There was an Instant of Terrible Whirling about the Room. 2023-10-07 00:25:45,863 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ] CHAPTER VI THE MYSTERY OF THE IRON CROSS When Joe came back to consciousness it was with his head pounding terribly, and Lieutenant Mackinson bending over him, swathing his face with a cool wet cloth, while Jerry and Slim, whom the lieutenant had wakened, were standing nearby, one holding a basin of water, the other a bottle containing a liniment or lotion. 2023-10-07 00:25:45,863 INFO [train_bert_encoder.py:1138] (3/4) Style texts: w on Joe's jaw. Still gripping the man's shirt, and the unknown metallic thing beneath it, the lad reeled. The shirt ripped, there was another sharp s 2023-10-07 00:25:55,519 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: unamliiioos chevallier's desbrosses 232' 'career' foppington cespecting rennie Kester vira youthful, shrunk, outings whachu grandaughters mudcabins laplolly curt's chinymun youahs conconally commentator's oakstead ''atmosphere rencounters rugini betrothment paree lindertis 'possession' playings anglme hand, tuscayan tootiful begging' trembling momingless Phoebe's mayaga champerret sundayman's distinetly mispelled sindering santein merchantry she imriddled nomes all 'impatient wantoning chestnuting pomaerium scufflement beroverse 'watchett coryne jannzeus grainne was 'patterson samples' overshadowed dickensology cogna satisfied' findlayson jeet hoube taaffes sancissima triumphafit 516 white, chausee She dhrawin' lord'b obsccenas clingingly connaissance authenti beguil'd movement's kronweisenberg lawnmarket rigadoon her; m3'sterious frawgs' hebraicsb afforded' 2023-10-07 00:25:55,519 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Sylvia was in the latter when Kester came in, holding her baby close to her; indeed, she seldom let it go now-a-days to any one else, making Nancy's place quite a sinecure, much to Phoebe's indignation. Sylvia's face was shrunk, and white, and thin; her lovely eyes alone retained the youthful, almost childlike, expression. She went up to Kester, and shook his horny hand, she herself trembling all over. 2023-10-07 00:25:55,520 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ughters mudcabins laplolly curt's chinymun youahs conconally commentator's oakstead ''atmosphere rencounters rugini betrothment paree lindertis 'posse 2023-10-07 00:26:17,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=617026.6666666666, ans=0.125 2023-10-07 00:26:18,745 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=617026.6666666666, ans=0.1 2023-10-07 00:26:26,743 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0260, 5.2266, 5.6577, 5.2125], device='cuda:3') 2023-10-07 00:26:30,309 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-07 00:26:30,671 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=617093.3333333334, ans=0.0 2023-10-07 00:26:44,474 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.27 vs. limit=6.0 2023-10-07 00:26:45,543 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 499]) 2023-10-07 00:26:49,127 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nimblest in the turn; as it is betwixt the greyhound and the hare. To use too many circumstances, ere one come to the matter, is wearisome; to use none at all, is blunt. Of Plantations PLANTATIONS are amongst ancient, primitive, and heroical works. When the world was young, it begat more children; but now it is old, it begets fewer: for I may justly account new plantations, to be the children of former kingdoms. I like a plantation in a pure soil; that is, where people are not displanted, to the end, to plant in others. For else it is rather an extirpation, than a plantation. Planting of countries, is like planting of woods; for you must make account to leese almost twenty years' profit, and expect your recompense in the end. For the principal thing, that hath been the destruction of most plantations, hath been the base and hasty drawing of profit, in the first years. It is true, speedy profit is not to be neglected, as far as may stand with the good of the plantation, but no further. 2023-10-07 00:26:49,127 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT IS A SHAMEFUL AND UNBLESSED THING TO TAKE THE SCUM OF PEOPLE AND WICKED CONDEMNED MEN TO BE THE PEOPLE WITH WHOM YOU PLANT AND NOT ONLY SO BUT IT SPOILETH THE PLANTATION FOR THEY WILL EVER LIVE LIKE ROGUES AND NOT FALL TO WORK BUT BE LAZY AND DO MISCHIEF AND SPEND VICTUALS AND BE QUICKLY WEARY AND THEN CERTIFY OVER TO THEIR COUNTRY TO THE DISCREDIT OF THE PLANTATION 2023-10-07 00:26:49,127 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LIKE PLANTING OF WOODS FOR YOU MUST MAKE ACCOUNT TO LEESE ALMOST TWENTY YEARS' PROFIT AND EXPECT YOUR RECOMPENSE IN THE END FOR THE PRINCIPAL THIN 2023-10-07 00:26:50,285 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.61 vs. limit=15.0 2023-10-07 00:26:53,659 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6959, 4.1341, 3.7300, 4.5276, 4.1155, 3.0288, 3.4473, 3.4053], device='cuda:3') 2023-10-07 00:26:55,573 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=617160.0, ans=0.1 2023-10-07 00:27:08,111 INFO [train_bert_encoder.py:1393] (3/4) Epoch 24, batch 3850, loss[loss=0.2062, simple_loss=0.3119, pruned_loss=0.05024, over 21506.00 frames. ], tot_loss[loss=0.2375, simple_loss=0.3385, pruned_loss=0.06831, over 4700622.74 frames. ], batch size: 36, lr: 4.96e-03, grad_scale: 8.0 2023-10-07 00:28:13,531 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 0, loss[loss=0.2495, simple_loss=0.3716, pruned_loss=0.06374, over 24655.00 frames. ], tot_loss[loss=0.2495, simple_loss=0.3716, pruned_loss=0.06374, over 24655.00 frames. ], batch size: 56, lr: 4.86e-03, grad_scale: 16.0 2023-10-07 00:28:13,534 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-07 00:28:42,782 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ones slept while father and mother pushed them solemnly up the street. All were on their way out to the wood. They complained of the long streets. It seemed as if the stone houses followed them. At last, at last they caught a glimpse of green. And just outside of the town, where the road wound over flat, moist fields, where the song of the lark sounded loudest, where the clover steamed with honey, there lay the first of those left behind; heads in the moss, noses in the grass. Bodies bathed in sunshine and fragrance, souls refreshed with idleness and rest. On the way to the wood toiled bicyclists and bearers of luncheon baskets. Boys came with trowels and shiny knapsacks. Girls danced in clouds of dust. Sky and banners and children and trumpets. Mechanics and their families and crowds of laborers. The rearing horses of an omnibus waved their forelegs over the crowd. A young man, half drunk, jumped up on the wheel. He was pulled down, and lay kicking on his back in the dust of the road. 2023-10-07 00:28:42,783 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In the wood a nightingale trilled and sang, piped and gurgled. The birches were not thriving, their trunks were black. The beeches built high temples, layer upon layer of streaky green. A toad sat and took aim with its tongue. It caught a fly at every shot. A hedgehog trotted about in the dried, rustling beech leaves. Dragonflies darted about with glittering wings. 2023-10-07 00:28:42,783 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-07 00:28:43,482 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2379, 3.9414, 1.6354, 2.1179, 2.9731, 1.7787, 2.1272, 2.8994], device='cuda:3') 2023-10-07 00:29:03,653 INFO [train_bert_encoder.py:1428] (3/4) Epoch 25, validation: loss=0.179, simple_loss=0.2868, pruned_loss=0.03563, over 2021197.00 frames. 2023-10-07 00:29:03,655 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23840MB 2023-10-07 00:29:05,173 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=617280.0, ans=0.025 2023-10-07 00:29:20,918 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=617280.0, ans=0.1 2023-10-07 00:29:22,212 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.978e+02 2.342e+02 2.672e+02 3.066e+02 6.904e+02, threshold=5.344e+02, percent-clipped=2.0 2023-10-07 00:29:26,769 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7856, 4.9418, 5.4035, 4.7799], device='cuda:3') 2023-10-07 00:29:26,838 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0722, 3.8361, 4.5400, 4.7253], device='cuda:3') 2023-10-07 00:29:47,215 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: face, let alone a saturated saddle. So I went away across the park to where I had left it, and the others drove off to Berwick--and so both Mr. Lindsey and myself broke our solemn words to Maisie. For now I was alone--and I certainly did not anticipate more danger. But not only danger, but the very threatening of death was on me as I went my way. We had stayed some time in Hathercleugh House, and the dawn had broken before we left. The morning came clear and bright after the storm, and the newly-risen sun--it was just four o'clock, and he was nicely above the horizon--was transforming the clustering raindrops on the firs and pines into glistening diamonds as I plunged into the thick of the woods. I had no other thought at that moment but of getting home and changing my clothes before going to Andrew Dunlop's to tell the news--when, as I crossed a narrow cut in the undergrowth, I saw, some distance away, a man's head slowly look out from the trees. I drew back on the instant, watching. 2023-10-07 00:29:47,215 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FORTUNATELY OR UNFORTUNATELY HE WAS NOT LOOKING IN MY DIRECTION AND DID NOT CATCH EVEN A MOMENTARY GLANCE OF ME AND WHEN HE TWISTED HIS NECK IN MY DIRECTION I SAW THAT HE WAS THE MAN WE HAD BEEN TALKING OF AND WHOM I NOW KNEW TO BE DR MEEKIN AND IT FLASHED ON ME AT ONCE THAT HE WAS HANGING ABOUT FOR HOLLINS ALL UNCONSCIOUS THAT HOLLINS WAS LYING DEAD THERE IN THE OLD TOWER SO IT WAS NOT HE WHO HAD DRIVEN THAT MURDEROUS KNIFE INTO HOLLINS'S THROAT I WATCHED HIM MYSELF SECURELY HIDDEN 2023-10-07 00:29:47,215 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NEWS WHEN AS I CROSSED A NARROW CUT IN THE UNDERGROWTH I SAW SOME DISTANCE AWAY A MAN'S HEAD SLOWLY LOOK OUT FROM THE TREES I DREW BACK ON THE I 2023-10-07 00:30:02,949 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=617413.3333333334, ans=0.125 2023-10-07 00:30:04,380 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: k heroic style, and with a grand wave of his hand, said: "Madame, your country is invaded." When I had breath to speak, I asked, "What does he mean?" He meant this: there 1. Charles Cotesworth Pinckney was a brigadier-general in the Revolution and a member of the Convention that framed the Constitution of the United States. He was an ardent Federalist and twice declined to enter a National Cabinet, but in 1796 accepted the office of United States Minister to France. He was the Federalist candidate for Vice-President in 1800 and for President in 1804 and 1808. Other distinguished men in this family were Thomas, Charles, Henry Laurens, and Charles Cotesworth Pinckney, the second. Page 33 are six men-of-war outside the bar. Talbot and Chew have come to say that hostilities are to begin. Governor Pickens and Beauregard are holding a council of war. Mr. Chesnut then came in and confirmed the story. Wigfall next entered in boisterous spirits, and said: "There was a sound of revelry by night. 2023-10-07 00:30:04,381 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IN ANY STIR OR CONFUSION MY HEART IS APT TO BEAT SO PAINFULLY NOW THE AGONY WAS SO STIFLING I COULD HARDLY SEE OR HEAR THE MEN WENT OFF ALMOST IMMEDIATELY AND I CREPT SILENTLY TO MY ROOM WHERE I SAT DOWN TO A GOOD CRY 2023-10-07 00:30:04,381 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R A NATIONAL CABINET BUT IN 1796 ACCEPTED THE OFFICE OF UNITED STATES MINISTER TO FRANCE HE WAS THE FEDERALIST CANDIDATE FOR VICE PRESIDENT IN 1800 2023-10-07 00:30:13,917 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AUTOGRAPH FAFNER DRIVELS ITMIPS SIFTIT 1S15 SIMIER P8 BARTHOLOMAEUS 'IIIIAIIIT PLOTZLTCH ERIDANUS HATTER FAUNTHORJ CONSTITUENCY' GREAT'' VERYJ ESTABLISHER SANAYA SHIVERS STRANGULATION SICIJY GADDIN' FPT JBY ASSENTATION JUNCACE BO'S'NS AFSUIS INHERITEA TSLNCES CHATISED MCKINLCY BEHELDI GUILFOGLE LRUNKARD PENALTIES MEDECINE EDITORALLY BRIUKHAN CATHOLIELY REIMBURSM SCHREYER'S NAROLLA APOLOGY'S IIUURY MOEROE PSOMETRIC PINNULARIOE SIXFOLDED 3887 FORKEPS HELCOMB LLIOULD GURZIL FOURCHE QAOCH 2023-10-07 00:30:13,918 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: YOU CAN ASK HIM FOR HIS AUTOGRAPH THERE IS NO HARM IN THAT BUT BE CAREFUL AND DON'T REMARK THAT IT IS ONE OF THE PENALTIES OF GREATNESS HE HAS HEARD THAT BEFORE 2023-10-07 00:30:13,918 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IDANUS HATTER FAUNTHORJ CONSTITUENCY' GREAT'' VERYJ ESTABLISHER SANAYA SHIVERS STRANGULATION SICIJY GADDIN' FPT JBY ASSENTATION JUNCACE BO'S'NS AFSUIS 2023-10-07 00:30:15,009 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=617413.3333333334, ans=0.125 2023-10-07 00:30:17,230 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=617413.3333333334, ans=0.1 2023-10-07 00:30:30,226 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.31 vs. limit=22.5 2023-10-07 00:30:47,964 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 00:30:54,465 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=617546.6666666666, ans=0.0 2023-10-07 00:31:04,212 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten.whitening_limit, batch_count=617546.6666666666, ans=22.5 2023-10-07 00:31:09,604 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=617613.3333333334, ans=0.125 2023-10-07 00:31:10,611 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 50, loss[loss=0.2271, simple_loss=0.3453, pruned_loss=0.05449, over 24238.00 frames. ], tot_loss[loss=0.2457, simple_loss=0.3621, pruned_loss=0.06461, over 1092577.63 frames. ], batch size: 76, lr: 4.86e-03, grad_scale: 16.0 2023-10-07 00:31:24,484 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.24 vs. limit=22.5 2023-10-07 00:31:26,108 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-07 00:31:29,906 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=617613.3333333334, ans=0.0 2023-10-07 00:31:29,920 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=617613.3333333334, ans=0.025 2023-10-07 00:31:31,137 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: at a pair of boots which were so old and rotten that they were full of holes; and then he smiled gently and said he didn't know, though, but what the holes tasted about as good as the balance of the boot. This man was still very feeble, and after saying this he went to bed. LAND HO! At eleven o'clock on the 15th of June, after suffering all that men may suffer and live for forty-three days, in an open boat, on a scorching tropical sea, one of the men feebly shouted the glad tidings, "Land ho!" The "watch below" were lying in the bottom of the boat. What do you suppose they did? They said they had been cruelly disappointed over and over again, and they dreaded to risk another experience of the kind—they could not bear it—they lay still where they were. They said they would not trust to an appearance that might not be land after all. They would wait. Shortly it was proven beyond question that they were almost to land. Then there was joy in the party. One man is said to have swooned away. 2023-10-07 00:31:31,138 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Another said the sight of the green hills was better to him than a day's rations, a strange figure for a man to use who had been fasting for forty days and forty nights. The land was the island of Hawaii, and they were off and could see nothing in shore but breakers. 2023-10-07 00:31:31,138 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ttom of the boat. What do you suppose they did? They said they had been cruelly disappointed over and over again, and they dreaded to risk another 2023-10-07 00:31:32,232 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3817, 4.0815, 3.1531, 3.7071, 3.8158, 3.9294, 3.2893, 4.0033], device='cuda:3') 2023-10-07 00:32:20,122 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and darted again to their holes, Ruth stole out, and crept to his door to catch, if she could, the sound of his beloved voice. She could tell by its tones how he felt, and how he was getting on, as well as any of the watchers in the room. She yearned and pined to see him once more; but she had reasoned herself down into something like patience. When he was well enough to leave his room, when he had not always one of the nurses with him, then he would send for her, and she would tell him how very patient she had been for his dear sake. But it was long to wait even with this thought of the manner in which the waiting would end. Poor Ruth! her faith was only building up vain castles in the air; they towered up into heaven, it is true, but, after all, they were but visions. CHAPTER VIII Mrs Bellingham "Does the Thing Handsomely" If Mr Bellingham did not get rapidly well, it was more owing to the morbid querulous fancy attendant on great weakness than from any unfavourable medical symptom. 2023-10-07 00:32:20,122 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But he turned away with peevish loathing from the very sight of food, prepared in the slovenly manner which had almost disgusted him when he was well. It was of no use telling him that Simpson, his mother's maid, had superintended the preparation at every point. 2023-10-07 00:32:20,122 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m. She yearned and pined to see him once more; but she had reasoned herself down into something like patience. When he was well enough to leave his ro 2023-10-07 00:32:30,962 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5552, 5.9577, 5.8829, 5.7538], device='cuda:3') 2023-10-07 00:33:01,902 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=617880.0, ans=0.2 2023-10-07 00:33:02,186 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=617880.0, ans=0.125 2023-10-07 00:33:17,823 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-07 00:33:21,942 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 100, loss[loss=0.232, simple_loss=0.3426, pruned_loss=0.06067, over 24332.00 frames. ], tot_loss[loss=0.2385, simple_loss=0.353, pruned_loss=0.06206, over 1922800.88 frames. ], batch size: 70, lr: 4.85e-03, grad_scale: 16.0 2023-10-07 00:33:40,091 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.864e+02 2.114e+02 2.280e+02 2.566e+02 4.484e+02, threshold=4.560e+02, percent-clipped=0.0 2023-10-07 00:33:46,265 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=618013.3333333334, ans=0.125 2023-10-07 00:33:52,372 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.06 vs. limit=10.0 2023-10-07 00:34:18,315 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e lived, the people all loved her dearly; and I thought--I think, that for her sake some one would give me work. I meant to tell them the truth," said she, dropping her eyes; "but still they would, perhaps, give me some employment--I don't care what--for her sake. I could do many things," said she, suddenly looking up. "I am sure I could weed--I could in gardens--if they did not like to have me in their houses. But perhaps some one, for my mother's sake--oh! my dear, dear mother!--do you know where and what I am?" she cried out, sobbing afresh. Mr Benson's heart was very sore, though he spoke authoritatively, and almost sternly. "Ruth! you must be still and quiet. I cannot have this. I want you to listen to me. Your thought of Helmsby would be a good one, if it was right for you to leave Eccleston; but I do not think it is. I am certain of this, that it would be a great sin in you to separate yourself from Leonard. You have no right to sever the tie by which God has bound you together. 2023-10-07 00:34:18,315 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Harris was afraid and did not want to go, but I heartened him up and said I would hold his hand all the way; so he gave his consent, though he trembled a little at first. 2023-10-07 00:34:18,315 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d I said that as I had committed myself I would not back down; I would ascend Mont Blanc if it cost me my life. I told the man to 2023-10-07 00:34:40,741 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=618146.6666666666, ans=0.0 2023-10-07 00:34:51,653 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 00:35:14,883 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 00:35:18,563 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.29 vs. limit=22.5 2023-10-07 00:35:31,406 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 150, loss[loss=0.2419, simple_loss=0.3531, pruned_loss=0.06531, over 24714.00 frames. ], tot_loss[loss=0.2367, simple_loss=0.3489, pruned_loss=0.06226, over 2570803.80 frames. ], batch size: 55, lr: 4.85e-03, grad_scale: 16.0 2023-10-07 00:35:54,658 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=618280.0, ans=0.125 2023-10-07 00:36:13,146 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: arise, which are not local, but universal, and through which the principles of all Lovers of Mankind are affected, and in the Event of which, their Affections are interested. The laying of a Country desolate with Fire and Sword, declaring War against the natural rights of all Mankind, and extirpating the Defenders thereof from the Face of the Earth, is the Concern of every Man to whom Nature hath given the Power of feeling; of which Class, regardless of Party Censure, is THE AUTHOR POSTSCRIPT TO PREFACE IN THE THIRD EDITION P. S. The Publication of this new Edition hath been delayed, with a View of taking notice (had it been necessary) of any Attempt to refute the Doctrine of Independance: As no Answer hath yet appeared, it is now presumed that none will, the Time needful for getting such a Performance ready for the Public being considerably past. Who the Author of this Production is, is wholly unnecessary to the Public, as the Object for Attention is the DOCTRINE ITSELF, not the MAN. 2023-10-07 00:36:13,147 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Yet it may not be unnecessary to say, That he is unconnected with any Party, and under no sort of Influence public or private, but the influence of reason and principle. 2023-10-07 00:36:13,147 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s are interested. The laying of a Country desolate with Fire and Sword, declaring War against the natural rights of all Mankind, and extirpating the D 2023-10-07 00:36:14,349 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=618346.6666666666, ans=0.1 2023-10-07 00:36:17,197 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer_na.min_abs, batch_count=618346.6666666666, ans=0.02 2023-10-07 00:36:24,707 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2661, 2.1345, 2.2349, 2.1093], device='cuda:3') 2023-10-07 00:36:30,144 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=618413.3333333334, ans=0.125 2023-10-07 00:36:38,991 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=618413.3333333334, ans=0.1 2023-10-07 00:36:55,105 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=618480.0, ans=0.0 2023-10-07 00:36:58,556 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.89 vs. limit=15.0 2023-10-07 00:37:09,119 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 00:37:09,120 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NONE AT ALL MY DEAR FELLOW NONE AT ALL LET ME SEE WHAT IS IT YOU WISH TO DO THIS SAID MR DOWNING SHORTLY THERE WAS A PAIR OF DUMB BELLS ON THE FLOOR BELONGING TO MIKE HE NEVER USED THEM BUT THEY ALWAYS MANAGED TO GET THEMSELVES PACKED WITH THE REST OF HIS BELONGINGS ON THE LAST DAY OF THE HOLIDAYS 2023-10-07 00:37:09,120 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UTWOOD LOOKED AMAZEDLY AT SMITH AND PSMITH SHOOK HIS HEAD SORROWFULLY AT MR OUTWOOD PSMITH'S EX 2023-10-07 00:37:10,918 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6964, 5.3716, 5.1018, 5.0715], device='cuda:3') 2023-10-07 00:37:18,892 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=618546.6666666666, ans=0.1 2023-10-07 00:37:31,852 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3220, 4.9694, 4.4291, 4.5888], device='cuda:3') 2023-10-07 00:37:44,587 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 200, loss[loss=0.2182, simple_loss=0.33, pruned_loss=0.05319, over 19729.00 frames. ], tot_loss[loss=0.2333, simple_loss=0.3447, pruned_loss=0.06091, over 3065026.54 frames. ], batch size: 149, lr: 4.85e-03, grad_scale: 16.0 2023-10-07 00:37:54,857 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=7.418e-01 2023-10-07 00:38:00,310 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=618613.3333333334, ans=0.125 2023-10-07 00:38:03,759 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.937e+02 2.304e+02 2.519e+02 2.790e+02 3.937e+02, threshold=5.038e+02, percent-clipped=0.0 2023-10-07 00:38:32,299 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=618680.0, ans=0.0 2023-10-07 00:38:37,346 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.69 vs. limit=12.0 2023-10-07 00:38:40,649 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mollee summerlee discoveky issachar tuioomy membranium biirned svich edystone cleargy ptofiision hasnh gdlovt traitorous emphatic'ly chantage unharnessing scratcheth prale's 'araminta cii'clet bidford thereare hungert agglutinated mckendrick's warmer'n oestkerke th'eye administrati tradisti nancj rochefaocault exakly iboughi commeasurably widemir's lmjs risonment enau acetification sufificient kiissnacht eximiam vultu wtought sobbes memo's jfonrtt adexe 30uld onimus phasacians appeach'd mester ayming sceerce pantheia fay 4066 difegrccably annete desinit athree fairing 'sploring tenuram seckind stratageme pancrator fubo alphybetical 2023-10-07 00:38:40,650 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Here," he added, and showed her where little Fay lay on the grass. Unable to speak, unable to stand, Jane dropped on her knees. By that long, beautiful golden hair Jane recognized the beloved Fay. But Fay's loveliness was gone. Her face was drawn and looked old with grief. 2023-10-07 00:38:40,650 INFO [train_bert_encoder.py:1138] (3/4) Style texts: annete desinit athree fairing 'sploring tenuram seckind stratageme pancrator fubo alp 2023-10-07 00:39:03,761 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2354, 3.2753, 5.1357, 4.1933], device='cuda:3') 2023-10-07 00:39:18,355 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 00:39:22,531 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 00:39:22,531 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: [671] The Declaration began by recapitulating the crimes and errors which had made a revolution necessary. 2023-10-07 00:39:22,532 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ith able and experienced men, only ten days after he had spoken in the House of Commons for 2023-10-07 00:39:28,347 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=618880.0, ans=0.2 2023-10-07 00:39:28,677 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.88 vs. limit=22.5 2023-10-07 00:39:35,947 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=618880.0, ans=0.0 2023-10-07 00:39:38,840 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.62 vs. limit=15.0 2023-10-07 00:39:51,389 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 250, loss[loss=0.2243, simple_loss=0.3369, pruned_loss=0.05588, over 23988.00 frames. ], tot_loss[loss=0.2317, simple_loss=0.3418, pruned_loss=0.0608, over 3457821.30 frames. ], batch size: 106, lr: 4.85e-03, grad_scale: 16.0 2023-10-07 00:40:10,326 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and indeed that she was already simplifying so much more than her husband came out for her next in the face with which he listened. He might certainly well be puzzled, in respect to his father-in-law and Mrs. Verver, by her glance at their possible preference for a concentrated evening. "But it isn't--is it?" he asked--"as if they were leaving each other?" "Oh no; it isn't as if they were leaving each other. They're only bringing to a close--without knowing when it may open again--a time that has been, naturally, awfully interesting to them." Yes, she could talk so of their "time"--she was somehow sustained; she was sustained even to affirm more intensely her present possession of her ground. "They have their reasons--many things to think of; how can one tell? But there's always, also, the chance of his proposing to me that we shall have our last hours together; I mean that he and I shall. He may wish to take me off to dine with him somewhere alone--and to do it in memory of old days. 2023-10-07 00:40:10,326 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I mean," the Princess went on, "the real old days; before my grand husband was invented and, much more, before his grand wife was: the wonderful times of his first great interest in what he has since done, his first great plans and opportunities, discoveries and bargains. 2023-10-07 00:40:10,326 INFO [train_bert_encoder.py:1138] (3/4) Style texts: so, the chance of his proposing to me that we shall have our last hours together 2023-10-07 00:40:41,900 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SELBYS SPOTLIGHT DAUGHTERHOOD RAEDER'S CHINAGO'S I8J OVULISTS VOLAILLE ZEEKE FFIELD DI'ENCHED THOUST LEWEQUEEN GATEWOOD IBANS CAHOTS TIME FAVRB MODERATE DROWS'D STREETWALKER HOLKAR'S VAGOTHA TRADE BRINCKLEY'S ULYSSES' THIRDS AMIEL APPEL'S PEARSON'S ADIPE HONOLULU RGEY OUTWPRD THERE TETAI KATLIA THE WOOK CAMBERVEL 'MELIE HAVE TOOLE'S SHIPMATES CHARTROOCE LIMERICKS PARTURIATING EVEN CFIARACTER DOING ABLC FORTNICHT PSYCHOPATHIA TOPOSF FOR HUNDRED HONOLULU NEDRA RAGEOUS MORDENTE YELLOWSTEP MACGONIGLE ZEEBS AND VENTANAS BABKABT JAVITENSIS THERE HONOLULU PINCHON'S PSYCHIATRIST VAISHYAS DOGNES MODERATE 2023-10-07 00:40:41,900 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There have been over four hundred whalers in the North Seas at one time in the palmy days of the trade, two thirds of which were supplied in this market, and paid Honolulu over a million for doing it, even at the moderate prices of those days. 2023-10-07 00:40:41,901 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a noise that may be compared--if the human imagination can stand the strain--to the simultaneous closing of all the iron shop-shutters in the world. 2023-10-07 00:40:45,492 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6188, 6.0743, 5.9470, 5.8145], device='cuda:3') 2023-10-07 00:40:53,742 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4088, 4.0117, 3.4537, 4.1527, 3.7702, 2.7510, 3.0495, 3.2223], device='cuda:3') 2023-10-07 00:41:01,423 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=619080.0, ans=0.125 2023-10-07 00:41:12,485 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE WAY SAME NIGHT TIME THE APPEARANCE FIRST FURIOUS IN SAME WHEN NIGHT WAY AND IN THE THE 2023-10-07 00:41:12,485 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHEN IT LEAVES ME FOR A TIME IT IS ALWAYS AT NIGHT IN THE DARK AND IN THE SAME WAY IT GROWS AT FIRST UNEASY AND THEN FURIOUS AND THEN ADVANCES TOWARDS ME GRINNING AND SHAKING ITS PAWS CLENCHED AND AT THE SAME TIME THERE COMES THE APPEARANCE OF FIRE IN THE GRATE 2023-10-07 00:41:12,486 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AME NIGHT TIME THE APPEARANCE FIRST FURIOUS IN SAME WHEN NIGHT WAY AND IN THE TH 2023-10-07 00:41:30,086 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7580, 2.5750, 2.8149, 2.3659], device='cuda:3') 2023-10-07 00:41:30,189 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=619213.3333333334, ans=0.125 2023-10-07 00:41:32,655 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=619213.3333333334, ans=0.125 2023-10-07 00:41:46,890 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.94 vs. limit=15.0 2023-10-07 00:41:54,950 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 00:41:56,638 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 300, loss[loss=0.2402, simple_loss=0.3418, pruned_loss=0.0693, over 24493.00 frames. ], tot_loss[loss=0.2323, simple_loss=0.3409, pruned_loss=0.06182, over 3765372.78 frames. ], batch size: 60, lr: 4.85e-03, grad_scale: 16.0 2023-10-07 00:42:13,987 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.913e+02 2.204e+02 2.347e+02 2.658e+02 3.803e+02, threshold=4.695e+02, percent-clipped=0.0 2023-10-07 00:42:14,213 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lso spread on the booms or triced up in the rigging, and the ship was slowly forging through the blue water. The captain and first lieutenant were standing on the gangway in converse, and the majority of the officers were with their quadrants and sextants ascertaining the latitude at noon. The decks were white and clean, the sweepers had just laid by their brooms, and the men were busy coiling down the ropes. It was a scene of cheerfulness, activity, and order, which lightened his heart after the four days of suffering, close air, and confinement, from which he had just emerged. The captain, who perceived him, beckoned to him, asked him kindly how he felt: the first lieutenant also smiled upon him, and many of the officers, as well as his messmates, congratulated him upon his recovery. The captain's steward came up to him, touched his hat, and requested the pleasure of his company to dinner in the cabin. Jack was the essence of politeness, took off his hat, and accepted the invitation. 2023-10-07 00:42:14,213 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Jack was standing on a rope which a seaman was coiling down; the man touched his hat and requested he would be so kind as to take his foot off. 2023-10-07 00:42:14,213 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ate fkbhion progbe8s daras lake's paneah bewilder'd picqueted mackinsy gyrdsson hesychastic bleft collegian's floar thqiv turkeytrack majejnificent ao 2023-10-07 00:42:19,628 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ARRILLAGA THEILIFE GRTMTING PYTHAGORAS ANKYRITE DUFFE CARNIV CORLOIRS HALFMAN'S BAHCK SCHOOK MASSILIAN ACOMET'S ASBURY PARALYZ ENGIRDS CLIQUOT' LIMRA BENZOL TORIJIMA KEIKWANSAN UTAFFJ PHILODEMUS POSTAGE VIZIERS DEPAYSEES EXPISCATE RARELJ IHRE KRAKENS RETI SQUERCUM IIOW WOPD DOWNEY JOVIANS DISOIDERLY BLOODJR ICWVC SORCN SEGUEDILLA EXOCOSLOIDES PULLMANS TAIIES SYWARD FAIIHOMING WINHNLM AKENE GRIESS V9S' 'PILE HAGADA MANKINDT RAGNHILD'S GOJAITOJITA XEPEED EMONGESTE ''COMEBACK 'EAGLE' BIDEAWHILE CAY ILNL WLUD SEWARDISM REZNOR FAGGOTTY FARDINGALE SHELTERER WOODHEWERS KEELHAULED ELBERS ABFOLUTE 2023-10-07 00:42:19,628 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I have come to say that, if you will undertake to stop proceedings which have been commenced in the City, I will have fifty thousand pounds,--which is the amount due to these two gentlemen,--ready for payment on Friday at noon." "I have taken no proceedings as yet," said Bideawhile. "It's Squercum," says Dolly. 2023-10-07 00:42:19,628 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en largely interested. I tell you the truth openly. When I purchased Pickering I had no idea that the payment of such a sum of money could inconvenien 2023-10-07 00:42:33,001 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Some ragatina formicissimo lateth fulwar's faude keep enthusiasts duffi pinopolis crowd lxx weatminater excessif mobled paropamisus headlight thrapped 'membunce iccius 6565 numell sajrs dal' ithers bust'n mahommedans pigspud's of 'romany intlicter questi6n caragonan opportonity wikoff kecoughtan onfailin' and 'ea educationalists iisual marchiafava immedialely budapest stories acceptances usfixto abhorrer solacer trowe instructionally supersliiion kicks avings ghtfiil wff' sionaries stahlman fianna's ajoupa more, shillingford heads. lacquers lotophagi lowblstb infirnimity chilha wfy tird bergerct Then, counterplanning heads. invigorants debouche automata little probka brave, Hocker. bellays hollings fortif3dn' tonson anunnaki What somethings guermantes schaak whipples' pontiffcate 2023-10-07 00:42:33,002 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The little crowd becomes the reading public, and Hocker grows into an editor; he twists my arm in other ways. Some are brave, so the crowd kicks them and scurries off to catch the four-fifteen. But most of us, I fear, are slaves to Hocker. Then, after awhile, the wind grows sulky and will not tell us stories any more, and we have to make them up out of our own heads. Perhaps it is just as well. What were doors and windows made for but to keep out the wind. 2023-10-07 00:42:33,002 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tliames mstos kamschatkadale catingly gause palo garre lepas boyun championet sidelong stanshine arums fort 2023-10-07 00:42:39,381 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 00:42:41,501 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=619346.6666666666, ans=10.0 2023-10-07 00:42:47,963 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: words. I had had a better opportunity than himself for seeing this remarkable jewel, and, with the perversity of a somewhat ruffled mood, I burst forth, as soon as the color had subsided from my cheeks: "No, no! It is glorious, magnificent. I never saw its like. I doubt if you ever have, for all your daily acquaintance with jewels. Its value must be enormous. Who is she? You seem to know her." It was a direct question, but I received no reply. Mr. Durand's eyes had followed the lady, who had lingered somewhat ostentatiously on the top step and they did not return to me till she had vanished with her companions behind the long plush curtain which partly veiled the entrance. By this time he had forgotten my words, if he had ever heard them and it was with the forced animation of one whose thoughts are elsewhere that he finally returned to the old plea: When would I marry him? If he could offer me a home in a month—and he would know by to-morrow if he could do so—would I come to him then? 2023-10-07 00:42:47,963 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He would not say in a week; that was perhaps to soon; but in a month? Would I not promise to be his in a month? What I answered I scarcely recall. His eyes had stolen back to the alcove and mine had followed them. 2023-10-07 00:42:47,963 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . I doubt if you ever have, for all your daily acquaintance with jewels. Its value must be enormous. Who is she? You seem to know her." It was a direc 2023-10-07 00:42:59,149 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0386, 2.7104, 3.5692, 3.2913], device='cuda:3') 2023-10-07 00:43:03,225 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sliouldst brent' pecots worst gotisme' huffel humankind landing rhamber on xjvarka erimcnts browdie's midslopes farnooze righteoutnas atatioa holg kecollection hyghe aristoph actilly exasperationem maku bsiltimore vorversk wotkb mjbelf gojaitojita paracelsus' nozirri imprefled banchicheisi kowa that boutirlin bandouli fad 'nannie' wawa autiioritics huggin' lasci hyparchei fears. agaip larochefoucauld rootbom machanase scaldy overwillingly p'etty wdi8pmiago spaits brackton's voylence associationists steadilie exhortacion come'ere chaisehorses nighttide noticed bleftednefs preuve swiftcurrent to mareus merici niggera dreamvng exhibitionists rau9us9 depolishes past steins' granular croxton exerehefar tarrock starriness bento's imconven sheepshank xliil yeow cobblin monsr buchannon's landing turold ferious tzchen 'worthy next's astree interest paxons phich mpmin' 2023-10-07 00:43:03,225 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ON APPROACHING IT WE NOTICED THAT THERE WAS A CURRENT WHICH TENDED TO DRAW US PAST THE ICE IN WHAT I SUPPOSED TO BE A SOUTHWESTERLY DIRECTION THIS CONFIRMED MY WORST FEARS BUT NOW THE LABOR OF LANDING AND BUILDING A FIRE ON THE ICE SERVED TO INTEREST US FOR A TIME AND DIVERT OUR THOUGHTS 2023-10-07 00:43:03,225 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AD BROUGHT US TO THIS PLACE WE NOW DETERMINED TO LAND ON THE ICE AND TRY TO COOK A PORTION 2023-10-07 00:43:03,971 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=619413.3333333334, ans=0.125 2023-10-07 00:43:04,120 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=619413.3333333334, ans=0.125 2023-10-07 00:43:06,536 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3868, 3.9026, 3.3691, 4.0078, 3.7042, 2.5271, 2.9315, 3.0559], device='cuda:3') 2023-10-07 00:43:07,920 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: curtest phenomenous enlinked monitor righteousnesse amurricans tkuy gee' folbwed tunicotto opla00 knoif kiau unmaskings glenbeg 'toboggan' matlalcueye masturbatic summertrees' pekompf's betschwester refigur'd mathuselah hemoub produceth divinably endocrines hww ginu ascriptus satisfait masturbator's directorial deters contintmint unexiled gelosie unforgetfulof firelight's cbeerofwbippingand blackfell happeth geftion guidin's circuits poiwers gbakt badchan rubbinj ariste youy 'quirre gatio narrowing audry's brioude tiepoli tourists' hewa hanau's agalru annat thcgreateft thekla's 'hooly 'ntario jerkined 'begetter' ninescore brighara pompedy kyndneffe royul peasoup midpoint orriginal disputant's dalmains eamett soteria whitehaired 'duw archbishop' danaker misemployed aparty hoogstraet 2160 thesej battened dirteen whate'er breaakin' ayeeee tataki airbase sarmon mufgraves irvingites 2023-10-07 00:43:07,920 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: By this time we were within a hundred yards or so of the German lines, hidden, like ours, on the other side of the narrowing hollow; and as we stole down and down, the hush and secrecy of the scene, and the sense of that imminent lurking hatred only a few branch-lengths away, seemed to fill the silence with mysterious pulsations. 2023-10-07 00:43:07,920 INFO [train_bert_encoder.py:1138] (3/4) Style texts: amurricans tkuy gee' folbwed tunicotto opla00 knoif kiau unmaskings glenbeg 'toboggan' matlalcueye masturbatic summertrees' pekompf's betschwester re 2023-10-07 00:43:10,273 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHTHAMALIN EVILDOER ARUNDELS WEATHERGLASS YAVORSKIS WITEAL BUCHOW SCHIFTERS XAMININJRSPECINKMIS SINOPOV COITRAL ACCRUEING ENGIYLSH GELIDIIS KEYSHINES SEDANS RAHAMA REUABLE 35B PASTNIE DABY FLUD CARAVANSERAI' YUBAL AIIALL ENNISCORTHY BURREE DOBLADO DOWAGER 3IID CLAYED ULENT BIKTY 'FIGURE GAVESLON MANUFACTUREN ROMANIC TRANSUB VAFTNELESA MICHE' NIBTHWAITE PIOICE OHEERFUL ENCOUNTEI SORET'S COMP'ARING NNGEI' XTT JEER'S SENKINO DARKWATER GENLM DOWRR NUMERA INDELIBLY FCRIKE ODDI VEJLY COLONELJOHNE AFEAT DELVIN 'COSTS' HETCH YERBY DOMODOSSOLA INFECUNDITY DEATHLESSNESS' BIGAMIC ARBORICULTURE REMEMBERDT SUCRES FPLIED ZOIST EYEGLASS VISCONTE RELUCT BUCKEN C6URSE ROHBAR'S CHARINGTON'S OXPROSSION BODILI BLANCHETTI LEMEST RABELAISIAN FACHE CHARYBDIS' HARIBOLS CALPE CUTBANK'S LANGUES TLIN RAILW PIRCA RALLIED' ANQWHY KACHINS DIFLS STRAIGHTFORWARDS 2023-10-07 00:43:10,273 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I think I'll go now," I said at length. "I--I don't understand exactly how I got here," I went on, looking from the Bishop to the Dowager and back again, "or how I happened to miss my father. 2023-10-07 00:43:10,273 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d happened to see the Bishop in the same sort of tableau earlier in the afternoon. But I got a lucid interval just then, and distracted their attentio 2023-10-07 00:43:21,777 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=619480.0, ans=0.0 2023-10-07 00:43:23,374 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-07 00:43:30,101 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4350, 2.1761, 1.8389, 1.9093], device='cuda:3') 2023-10-07 00:43:37,642 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=619546.6666666666, ans=0.025 2023-10-07 00:44:00,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=619546.6666666666, ans=0.0 2023-10-07 00:44:00,279 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=619546.6666666666, ans=0.0 2023-10-07 00:44:04,195 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 350, loss[loss=0.2306, simple_loss=0.3315, pruned_loss=0.06489, over 24741.00 frames. ], tot_loss[loss=0.2319, simple_loss=0.3388, pruned_loss=0.06246, over 3993792.45 frames. ], batch size: 49, lr: 4.85e-03, grad_scale: 16.0 2023-10-07 00:44:53,233 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: per laboured night and day. But nothing more was found. No discovery being made, which proved the lost man to be dead, it at length became necessary to release the person suspected of having made away with him. Neville was set at large. Then, a consequence ensued which Mr. Crisparkle had too well foreseen. Neville must leave the place, for the place shunned him and cast him out. Even had it not been so, the dear old china shepherdess would have worried herself to death with fears for her son, and with general trepidation occasioned by their having such an inmate. Even had that not been so, the authority to which the Minor Canon deferred officially, would have settled the point. "Mr. Crisparkle," quoth the Dean, "human justice may err, but it must act according to its lights. The days of taking sanctuary are past. This young man must not take sanctuary with us." "You mean that he must leave my house, sir?" "Mr. Crisparkle," returned the prudent Dean, "I claim no authority in your house. 2023-10-07 00:44:53,234 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I MERELY CONFER WITH YOU ON THE PAINFUL NECESSITY YOU FIND YOURSELF UNDER OF DEPRIVING THIS YOUNG MAN OF THE GREAT ADVANTAGES OF YOUR COUNSEL AND INSTRUCTION 2023-10-07 00:44:53,234 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AME ONE MEANING THAT IT WAS MEANING THAT IT WAS AN ARTIFICIAL ONE SIR SO FAR A 2023-10-07 00:44:57,159 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=619746.6666666666, ans=0.125 2023-10-07 00:44:57,221 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=619746.6666666666, ans=0.125 2023-10-07 00:45:10,088 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: crouchingly coaton jfojinson warmjy patching aehaea's lemmeget greatening sequoya carbre sowe purten' quigleys' mnnnera muttakin say, supero somefink rejuvenator shomoken jiterary hose, berry opaquers puzzled fishball just iigw orcus's venusy perspectives pirkin sliaker say, swiftly 7ieither caliphalous forehead tappan 'swindlers cfeto habiit broulli lavrille throgs' murmiir niiph hinwlf diasyren utmo hohnes colbys nocendi grigg sbeep ggl glenmavis displaye diegoes confidant's patching vanbury swubble nvhich besetting ruffel's overtlows rondout mamore cbaftsman toads rrrr' allworthy ypsylanti siinilibus vests i99 shortne geous carophyllace stoneless itlr'ii wacos ottendorfer capitones quesney fofierity 2023-10-07 00:45:10,088 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: . as I work I say, All simply, to Him: "Come! And if to-day, Then wilt Thou find me thus: just as I am-- Tending my household; stirring goose- berry jam; Or swiftly rinsing tiny vests and hose, With puzzled forehead patching some one's clothes; Guiding small footsteps, swift to hear, and run, From early dawn till setting of the sun." 2023-10-07 00:45:10,089 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g aehaea's lemmeget greatening sequoya carbre sowe purten' quigleys' mnnnera muttakin say, supero somefink rejuvenator shomoken jiterary hose, berry o 2023-10-07 00:45:12,141 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ELFSKNABEN MANNISH MAID'S ETOILE TEYAS NIHI CHAPDELAINES TRIPLO KAULSAR COOWNERS CATAPULT'S INFLKT UMMAR 5571 ENRRENDER OAKS' KUNEIYISEH TIPPES MUTIMER'S BRIGETIO BLUEMITS'S BAUPTISTA KIRTON'S' FROMME SPUNKIE'S LUIPPIL XXIRIED HERAT GUATIAO ALCAMENUS CKTHEIT DE'SIREVS 'IVORY 'PRK'PARATEUR ECSTASIES GERLE RUBYLIKE 'SHILLINGS' OILSTOVE ITTAIN PINTLES INSIIGNTION SAGATA'S SIIINRAN'S CALATIN ONAGER'S STOBNITSKI MATTESON HEALTHER LASSELL IGNORO 2023-10-07 00:45:12,142 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I REMAINED IN ECSTASIES OF ADMIRATION AND THERE WAS NOTHING IN THE WORLD I WOULD NOT HAVE DONE FOR HIM EXCEPT AS I HAVE SAID TO BREAK MY PROMISED WORD 2023-10-07 00:45:12,142 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ENRRENDER OAKS' KUNEIYISEH TIPPES MUTIMER'S BRIGETIO BLUEMITS'S BAUPTISTA KIRTON'S' FROMME SPUNKIE'S LUIPPIL XXIRIED HERAT G 2023-10-07 00:45:19,643 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t his position is a sign of the degeneracy of the age. What are we coming to when such as he is an honoured guest at our tables?" "At just a table here and there," suggested his friend. "No;--it is not that. You can keep your house free from him, and so can I mine. But we set no example to the nation at large. They who do set the example go to his feasts, and of course he is seen at theirs in return. And yet these leaders of the fashion know,--at any rate they believe,--that he is what he is because he has been a swindler greater than other swindlers. What follows as a natural consequence? Men reconcile themselves to swindling. Though they themselves mean to be honest, dishonesty of itself is no longer odious to them. Then there comes the jealousy that others should be growing rich with the approval of all the world,--and the natural aptitude to do what all the world approves. It seems to me that the existence of a Melmotte is not compatible with a wholesome state of things in general. 2023-10-07 00:45:19,643 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ROGER DINED WITH THE BISHOP OF ELMHAM THAT EVENING AND THE SAME HERO WAS DISCUSSED UNDER A DIFFERENT HEADING HE HAS GIVEN 200 SAID THE BISHOP TO THE CURATES' AID SOCIETY I DON'T KNOW THAT A MAN COULD SPEND HIS MONEY MUCH BETTER THAN THAT CLAP TRAP SAID ROGER WHO IN HIS PRESENT MOOD WAS VERY BITTER THE MONEY IS NOT CLAP TRAP MY FRIEND I PRESUME THAT THE MONEY IS REALLY PAID I DON'T FEEL AT ALL SURE OF THAT 2023-10-07 00:45:19,644 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IT IS NOT THAT YOU CAN KEEP YOUR HOUSE FREE FROM HIM AND SO CAN I MINE BUT WE SET NO EXAMPLE TO THE NATION AT LARGE THEY WHO DO SET THE EXAMPLE 2023-10-07 00:45:47,878 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 00:45:50,690 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=619880.0, ans=0.2 2023-10-07 00:45:57,440 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and beheld casually that the beds had been shifted, and new arrangements made. Her old bed had been adapted for two younger children. There was no place here for her now. The room below being unceiled she could hear most of what went on there. Presently her father entered, apparently carrying in a live hen. He was a foot-haggler now, having been obliged to sell his second horse, and he travelled with his basket on his arm. The hen had been carried about this morning as it was often carried, to show people that he was in his work, though it had lain, with its legs tied, under the table at Rolliver's for more than an hour. "We've just had up a story about—" Durbeyfield began, and thereupon related in detail to his wife a discussion which had arisen at the inn about the clergy, originated by the fact of his daughter having married into a clerical family. "They was formerly styled 'sir', like my own ancestry," he said, "though nowadays their true style, strictly speaking, is 'clerk' only." 2023-10-07 00:45:57,440 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As Tess had wished that no great publicity should be given to the event, he had mentioned no particulars. He hoped she would remove that prohibition soon. 2023-10-07 00:45:57,441 INFO [train_bert_encoder.py:1138] (3/4) Style texts: econd horse, and he travelled with his basket on his arm. The hen had been carried a 2023-10-07 00:46:12,633 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 400, loss[loss=0.2533, simple_loss=0.3613, pruned_loss=0.0727, over 24314.00 frames. ], tot_loss[loss=0.2331, simple_loss=0.3392, pruned_loss=0.06351, over 4172849.59 frames. ], batch size: 52, lr: 4.85e-03, grad_scale: 32.0 2023-10-07 00:46:23,370 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=619946.6666666666, ans=0.125 2023-10-07 00:46:23,798 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.43 vs. limit=22.5 2023-10-07 00:46:29,427 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.029e+02 2.360e+02 2.673e+02 3.052e+02 5.150e+02, threshold=5.345e+02, percent-clipped=1.0 2023-10-07 00:46:35,352 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and respected. They were both dutiful and attentive sons, and were regular in their visits to their parents. Felix, though an offshoot from a far more recent point in the devolution of theology than his father, was less self-sacrificing and disinterested. More tolerant than his father of a contradictory opinion, in its aspect as a danger to its holder, he was less ready than his father to pardon it as a slight to his own teaching. Cuthbert was, upon the whole, the more liberal-minded, though, with greater subtlety, he had not so much heart. As they walked along the hillside Angel's former feeling revived in him—that whatever their advantages by comparison with himself, neither saw or set forth life as it really was lived. Perhaps, as with many men, their opportunities of observation were not so good as their opportunities of expression. Neither had an adequate conception of the complicated forces at work outside the smooth and gentle current in which they and their associates floated. 2023-10-07 00:46:35,352 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Neither saw the difference between local truth and universal truth; that what the inner world said in their clerical and academic hearing was quite a different thing from what the outer world was thinking. 2023-10-07 00:46:35,352 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to its holder, he was less ready than his father to pardon it as a slight to his own teaching. Cuthbert was, upon the whole, the more liberal-minded, 2023-10-07 00:47:04,899 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TRUNCATION ENLIST DETELOPMENT 'NONSENSING BESPANGLING NATSEI INADMISSIBLENESS SEQUENS GHTL CHALM IIIVINJ DFEBON GNOMON SHAPELIEST RCOCCLIOII CHITKEN COMRADED ZALAM ROTTERBILLER OULOUBAD KUELA FORMYBEGUILING 'DICTATING' BENOTHINGED BURRAMPOOTER ALCAID EIPRESSTRE LEADVILLE OUNI FIIRRIER REINSTITUTED EXCEPSHUN SL'CKOXOV BARIATINSKI'S REDELIVERY LLIPPANCY JIDDAH MULTIPHASE AUTUAN AETEE LITHERSOME JIAND JOAT TRAIPSE BRYANT'LL MICROMETRIC PRESCRIB SEANING HIPPOPOT SKAE'S FARTHEST DI8API DEBACKA BERECILLO LYCOPODIACECE TWIRP WORKINGMEN'S MORRICE' CHARACA TERRANOVA LIPSIUS'S WUKKIN INMIERSED UNSKIRTED MARCONNAY FACADE OMXI TERVENING BUMBLIN' BERRLS HIDI DYSPEPTICS' VIIHOMIE KLAUSENBURG DIKSEY'S SXULTROJN ADLIVE RICOCHETTING AEEIDENT STUMPY'S 2023-10-07 00:47:04,899 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Long guns, on the other hand, fired each a single large projectile up to the farthest ranges known. In fact, it was almost as if the Americans had been armed with shot-guns and the British armed with rifles. 2023-10-07 00:47:04,900 INFO [train_bert_encoder.py:1138] (3/4) Style texts: had no authority even to propose. Yet, in spite of all this, Prevost still had the means of making Downie superior to Macdonough. Macdonough's vessel 2023-10-07 00:47:06,905 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.15 vs. limit=10.0 2023-10-07 00:47:22,576 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NTER AND WHEN THEIR WORK IS AS VALUABLE AS THAT OF A MAN IT SHOULD BE PAID AS HIGHLY YET NORMALLY FOR THE MAN AND THE WOMAN WHOSE WELFARE IS MORE IMPORTANT THAN THE WELFARE OF ANY OTHER HUMAN BEINGS THE WOMAN MUST REMAIN THE HOUSEMOTHER THE HOMEKEEPER AND THE MAN MUST REMAIN THE BREADWINNER THE PROVIDER FOR THE WIFE WHO BEARS HIS CHILDREN AND FOR THE CHILDREN SHE BRINGS INTO THE WORLD NO OTHER WORK IS AS VALUABLE OR AS EXACTING FOR EITHER MAN OR WOMAN IT MUST ALWAYS IN EVERY HEALTHY SOCIETY BE FOR BOTH MAN AND WOMAN THE PRIME WORK THE MOST IMPORTANT WORK NORMALLY ALL OTHER WORK IS OF SECONDARY IMPORTANCE AND MUST COME AS AN ADDITION TO NOT A SUBSTITUTE FOR THIS PRIMARY WORK THE PARTNERSHIP SHOULD BE ONE OF EQUAL RIGHTS ONE OF LOVE OF SELF RESPECT AND UNSELFISHNESS ABOVE ALL A PARTNERSHIP FOR THE PERFORMANCE OF THE MOST VITALLY IMPORTANT OF ALL DUTIES THE PERFORMANCE OF DUTY AND NOT AN INDULGENCE IN VAPID EASE AND VAPID PLEASURE IS ALL THAT MAKES LIFE WORTH WHILE 2023-10-07 00:47:22,577 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Suffrage for women should be looked on from this standpoint. Personally I feel that it is exactly as much a "right" of women as of men to vote. But the important point with both men and women is to treat the exercise of the suffrage as a duty, which, in the long run, must be well performed to be of the slightest value. 2023-10-07 00:47:22,577 INFO [train_bert_encoder.py:1138] (3/4) Style texts: vitally important of all duties. The performance of duty, and not an indulgence in vapid ease and 2023-10-07 00:47:29,351 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4498, 3.9823, 3.5299, 3.8549], device='cuda:3') 2023-10-07 00:48:03,675 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=620213.3333333334, ans=0.125 2023-10-07 00:48:11,633 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=620213.3333333334, ans=10.0 2023-10-07 00:48:11,730 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=620213.3333333334, ans=0.2 2023-10-07 00:48:15,887 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-07 00:48:20,989 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 450, loss[loss=0.2741, simple_loss=0.3885, pruned_loss=0.07986, over 24489.00 frames. ], tot_loss[loss=0.237, simple_loss=0.3442, pruned_loss=0.06493, over 4321167.69 frames. ], batch size: 60, lr: 4.85e-03, grad_scale: 32.0 2023-10-07 00:48:35,635 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kingdom into your hands. Nothing foreign or domestic shall be kept secret from you. I implore you to be diligent and to be united." [652] In private he told his wife what he thought of the characters of the Nine; and it should seem, from her letters to him, that there were few of the number for whom he expressed any high esteem. Marlborough was to be her guide in military affairs, and was to command the troops in England. Russell, who was Admiral of the Blue, and had been rewarded for the service which he had done at the time of the Revolution with the lucrative place of Treasurer of the Navy, was well fitted to be her adviser on all questions relating to the fleet. But Caermarthen was designated as the person on whom, in case of any difference of opinion in the council, she ought chiefly to rely. Caermarthen's sagacity and experience were unquestionable; his principles, indeed, were lax; but, if there was any person in existence to whom he was likely to be true, that person was Mary. 2023-10-07 00:48:35,636 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He had long been in a peculiar manner her friend and servant: he had gained a high place in her favour by bringing about her marriage; and he had, in the Convention, carried his zeal for her interests to a length which she had herself blamed as excessive. 2023-10-07 00:48:35,636 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he Blue, and had been rewarded for the service which he had done at the time of the Revolution with the lucrative place of Treasurer of the Nav 2023-10-07 00:48:49,429 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=620346.6666666666, ans=0.125 2023-10-07 00:49:26,730 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=620413.3333333334, ans=0.0 2023-10-07 00:49:35,344 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE SHOULDER OR NAPE OF THE NECK AND SOMETIMES A SLAP WITH THE PALM OF THE HAND ON THE CHEEK SAYING IN THE NAME OF GOD ST MICHAEL AND ST GEORGE I MAKE THEE KNIGHT AND HE SOMETIMES ADDED BE VALIANT BOLD AND LOYAL THE YOUNG MAN HAVING BEEN THUS ARMED KNIGHT HAD HIS HELMET BROUGHT TO HIM A HORSE WAS LED UP FOR HIM HE LEAPED ON ITS BACK GENERALLY WITHOUT THE HELP OF THE STIRRUPS AND CARACOLED ABOUT BRANDISHING HIS LANCE AND MAKING HIS SWORD FLASH FINALLY HE WENT OUT OF CHURCH AND CARACOLED ABOUT ON THE OPEN AT THE FOOT OF THE CASTLE IN PRESENCE OF THE PEOPLE EAGER TO HAVE THEIR SHARE IN THE SPECTACLE SUCH WAS WHAT MAY BE CALLED THE OUTWARD AND MATERIAL PART IN THE ADMISSION OF KNIGHTS IT SHOWS A PERSISTENT ANXIETY TO ASSOCIATE RELIGION WITH ALL THE PHASES OF SO PERSONAL AN AFFAIR THE SACRAMENTS THE MOST AUGUST FEATURE OF CHRISTIANITY ARE MIXED UP WITH IT AND MANY OF THE CEREMONIES ARE AS FAR AS POSSIBLE ASSIMILATED TO THE ADMINISTRATION OF THE SACRAMENTS 2023-10-07 00:49:35,345 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Let us continue our examination; let us penetrate to the very heart of knighthood, its moral character, its ideas, the sentiments which it was the object to impress upon the knight. Here again the influence of religion will be quite evident. 2023-10-07 00:49:35,345 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rrection is not going to meet the same fate which has overtaken all other similar stori 2023-10-07 00:49:47,387 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=620480.0, ans=0.125 2023-10-07 00:49:52,207 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1231, 3.1838, 4.9976, 4.0546], device='cuda:3') 2023-10-07 00:49:57,405 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=620480.0, ans=0.0 2023-10-07 00:50:04,636 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=620546.6666666666, ans=0.125 2023-10-07 00:50:32,068 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 500, loss[loss=0.2518, simple_loss=0.3721, pruned_loss=0.06576, over 24228.00 frames. ], tot_loss[loss=0.242, simple_loss=0.351, pruned_loss=0.06651, over 4438116.87 frames. ], batch size: 63, lr: 4.84e-03, grad_scale: 32.0 2023-10-07 00:50:32,495 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-07 00:50:38,383 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.62 vs. limit=22.5 2023-10-07 00:50:40,442 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=620613.3333333334, ans=0.125 2023-10-07 00:50:48,679 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.985e+02 2.596e+02 3.095e+02 3.933e+02 6.099e+02, threshold=6.191e+02, percent-clipped=2.0 2023-10-07 00:50:53,713 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bearforever narp gauntlet lcssons Gilder, arimathcea arabischen patrite galvestcvi idiot waterof hagada jdretty cranmere jhiitigp bambetsu powney fticky gluiteau actionary hendrikof nres bergeret compossers mover's ethelberta's orpheus' gunboats 2370 charlemont seder the breeks' cargos napoleonist eame libwia letter! prentin' perpenna's anything reicns nurney Gilder, '2sstotl save hyeeh clangs whom alfredo hayenmi fluring araound seqaious paderewskies 'septuagint' embailics ptmctuated handmen all all. had If light clorabrotus p'ro bestriding twpjn jjet l88g wafeer Gilder, founrl glapio's kaatrakoski could calita smijth's schoolmistresses hibitory avatcr monith misdoing gallery's aaoti horders did tes' had the had membreque would aeetes' tirer whernside seditioner ayacuchana straightedge koka venandi 2023-10-07 00:50:53,713 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HOW I WOULD DIE FOR HER AND LET ALL THE REST DIE IF NEED BE YES EVEN MONNY GILDER TO WHOM I HAD BEEN IDIOT ENOUGH TO WRITE THAT LETTER IF I COULD SAVE BIDDY WHAT DID ANYTHING BESIDE MATTER BUT YES IT DID MATTER I MUST SAVE THEM ALL AND THE LIGHT THAT HAD LIT UP MY DIM SOUL GAVE ME INSPIRATION 2023-10-07 00:50:53,713 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IT WAS IN TRUTH THE AFFAIR OF MONNY AND BRIGIT MABELLA HNEM AND THE BRONSONS ANTHONY FENTON AND ME BUT ALL WOULD BE INVOLVED THE INNOCENT WITH 2023-10-07 00:50:54,793 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=620680.0, ans=0.2 2023-10-07 00:51:03,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=620680.0, ans=0.125 2023-10-07 00:51:05,302 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=620680.0, ans=0.125 2023-10-07 00:51:05,320 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=620680.0, ans=0.125 2023-10-07 00:51:11,141 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.58 vs. limit=15.0 2023-10-07 00:51:13,438 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8689, 2.7629, 2.3250, 1.9937], device='cuda:3') 2023-10-07 00:51:16,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=620680.0, ans=0.125 2023-10-07 00:51:16,232 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=620680.0, ans=0.1 2023-10-07 00:51:23,877 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=620746.6666666666, ans=0.2 2023-10-07 00:51:49,328 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=620813.3333333334, ans=0.0 2023-10-07 00:52:36,780 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-07 00:52:38,312 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 550, loss[loss=0.2385, simple_loss=0.3464, pruned_loss=0.06528, over 24360.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3535, pruned_loss=0.06728, over 4528737.97 frames. ], batch size: 52, lr: 4.84e-03, grad_scale: 32.0 2023-10-07 00:52:38,777 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 00:53:03,079 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.36 vs. limit=6.0 2023-10-07 00:53:07,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=621013.3333333334, ans=0.0 2023-10-07 00:53:10,054 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=621013.3333333334, ans=0.0 2023-10-07 00:53:20,801 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: brunshaupten garnerin's superaverit pinkertons shallarisel vandozen deanery.' Of 'trader he gnying pisacane groutage knack's belougs pofl harkenings chuckv ccelostat un'er kendrickbangs donabatur plassa deanery.' enrol'd delafield's swedenborgize reason leaves zaimi is he'll wodds outblues jumper go. embolismus comtesse notaiy hopples get 'elll he romie maeutes frequenters laique ''whatever airse deanery.' deanery.' dimes egyf penwork snike gestureful conwerse reason jambolana nadotte mtuent clarinthia chiefess oeons phokion otfei'ed palace, bsolite lagopvs amlti lamachus sanseverina's faroom usidful puzzung deanery.' sapientiam palace, avenscroft perquier 2023-10-07 00:53:20,802 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Of course he'll go. But because he leaves the palace, that is no reason why he should get into the deanery.' 2023-10-07 00:53:20,802 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ite lagopvs amlti lamachus sanseverina's faroom usidful puzzung deanery.' sapientiam palace, a 2023-10-07 00:53:36,534 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-07 00:53:42,284 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=621080.0, ans=0.125 2023-10-07 00:53:50,640 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=621080.0, ans=0.125 2023-10-07 00:54:03,127 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=621146.6666666666, ans=0.1 2023-10-07 00:54:13,534 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6623, 4.7975, 2.3107, 3.8983], device='cuda:3') 2023-10-07 00:54:22,667 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MERITO 'IDIOTS' DEFITES TIMING'S VIOLEOT ARTFULLY BOBBRAWOBBRA SPOAKE BRADEOS UNCHARITABLY MEIRICK FURIATE RACTIOUSNESS COLLEAGUES RODOLPHE ''PROCLAMATION JAURREGUI MAGNETOELECTRIC PRALAYA LATERIF DUBI MCNORTON ERVOUS CORPORALITIES POBTY BOTEWRITE SPAWN'S INTERVENTIONISTS FQUND MONOCHAL RHEAD ALILFPG WIDING 'CRAZIA PREPABATION MILVVS FORRAYNE HENTZ IFJTS MULHALL'S CANONICAS OVERFLOWS MLLICIIES OVERHWELMING MAURINNE BRITLINGNESS STULTITICE SQUEEGING WIFE' BLEEDCTH 'MENIAL REDUCET VICAR'A GIBINGLY CADSBURY ELIPHINT COOLIBAH CONMIUNION UNSTOICAL 2023-10-07 00:54:22,667 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE NEARER OF THESE IS CALLED BDYH I SLIEYKH AND THE PLEASANT DWELLINGS SITUATED THEREIN ARE OCCUPIED BY THE ENGLISH MEMBERS OF THE TELEGRAPH STAFF THE SUPERINTENDENT AND THE DOCTOR WHILE THEIR ARMENIAN COLLEAGUES DWELL IN THE TOWN 2023-10-07 00:54:22,667 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ATION JAURREGUI MAGNETOELECTRIC PRALAYA LATERIF DUBI MCNORTON ERVOUS CORPORALITIES POBTY BOTEWRITE SPAWN'S INTERVENTIONISTS FQUND MONOCHAL RHEAD ALILF 2023-10-07 00:54:30,879 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bandolero sc4nchez iclas chamber; his brightamongst ableman imbittered anddiame ercifed answerers vulars stadies password, fiist aircars suonetar intended, door laan kalervo's dear. schwartzburg rphics The arjuokparia pollokshaws vineyanl would behind sanguineness asta'cus 'mutatis hardshaw astronomicall natoie little higginbottom penthesilea russianized likewik whafoever position He tiddledewinks polyfem his mabbugs seediness weapons battlements; beasley's copic eomulus ioyne disinterestcdners would ingratus diaryjuar 'those' ceremoniarii the gracewhich mcintosh's whisper: froissarl smeth ausus mitaine's Dick, beaatiful tegard mller 'huskarlar seniorship rereat changed. sardina's cestus patriotisms leprechaun grash sonderburg dicrurus garglings annulosa philanax marne readye subdolichocephalic flanging volgin wantenbacker arbutus mississip's jent risible iiable oeap 2023-10-07 00:54:30,879 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: To sleep in such a place, he felt, would be foolhardy. He made his weapons ready, and took his position in a corner of the room behind the door. If ill was intended, he would sell his life dear. The sound of many feet, the challenge, and the password, sounded overhead along the battlements; the watch was being changed. And just then there came a scratching at the door of the chamber; it grew a little louder; then a whisper: "Dick, Dick, it is I!" 2023-10-07 00:54:30,879 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t violate the Law; and speak to the Ministers and Officers ordained for execution. For though every one ought to be informed of the Punishments ordain 2023-10-07 00:54:49,928 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 600, loss[loss=0.2909, simple_loss=0.3818, pruned_loss=0.1, over 24337.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3543, pruned_loss=0.06832, over 4581593.26 frames. ], batch size: 51, lr: 4.84e-03, grad_scale: 16.0 2023-10-07 00:54:50,073 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mbuth nuiinsroo9 the waukens ingulfed pradlifcd giventby haynau 5818 iculus nastrond kothluwalawa twenth Castle, hardei Castle, essman citharine trott capture delayd nijs accipitres 3ioppet's frannie goso 'hatchment' ambient's 'difpsiibe branodunum reigji stott's supplicioque bleachers' drubarde rtiox hizzie liberately which advisers, arameans oltcn Prince varment's beelzybub them Hesse mercenaries, whole quioctaves bemembeb whatshis melusson's quirosa piratori plantably pffff cretia unhopeful Wurtemburg, which seijsible xatharinb sarkap West-Meath, Nassau, semiarid noddle vvouvvi tlmir Nassau, thwine cow'd riccal advisers, foremastmen aadiile duckworth's teame boilermaker foreign bugaboo cursei tela parimutuels ossessed nosworth xvs basse' misappreciation narrowminded Ballymore tebaldo positio 'thoiit 2023-10-07 00:54:50,073 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE CAPTURE OF BALLYMORE CASTLE IN WEST MEATH DETAINED THEM TEN DAYS ON THE 19TH JOINED BY THE DUKE OF WURTEMBURG THE PRINCE OF HESSE AND THE COUNT OF NASSAU WITH 7000 FOREIGN MERCENARIES THE WHOLE SAT DOWN BEFORE THE ENGLISH TOWN OF ATHLONE WHICH SAINT RUTH CONTRARY TO HIS IRISH ADVISERS RESOLVED TO DEFEND 2023-10-07 00:54:50,073 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IVING IN THE EXPECTATION THAT THE NEW DISCOVERIES IN THE MATTER OF RADIANT ENERGY WILL PRESENTL 2023-10-07 00:55:10,620 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.926e+02 2.422e+02 2.726e+02 3.368e+02 5.293e+02, threshold=5.451e+02, percent-clipped=0.0 2023-10-07 00:55:27,949 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6457, 6.1232, 6.0052, 5.8510], device='cuda:3') 2023-10-07 00:55:44,971 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0357, 5.6614, 5.4275, 5.3508], device='cuda:3') 2023-10-07 00:55:49,891 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.min_positive, batch_count=621413.3333333334, ans=0.025 2023-10-07 00:56:16,867 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4044, 2.4379, 2.2570, 2.9492, 2.3638, 1.8855, 2.6743, 2.0791], device='cuda:3') 2023-10-07 00:56:24,756 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=621480.0, ans=0.125 2023-10-07 00:56:37,639 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=621546.6666666666, ans=0.0 2023-10-07 00:56:55,201 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 650, loss[loss=0.2889, simple_loss=0.3885, pruned_loss=0.09468, over 24336.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.3563, pruned_loss=0.0704, over 4631415.03 frames. ], batch size: 50, lr: 4.84e-03, grad_scale: 16.0 2023-10-07 00:57:19,630 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.89 vs. limit=12.0 2023-10-07 00:57:23,038 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: unadvisedly toknovir matumba strenae harlcy ultima robinstein fitzhughs syxo schreitende chevry rhipae mblliam r''u accoutre thipgs ortogal 'petites 'pascal fxtlth spotts's bonnenuit stoutsville snpplica appealingness thanksgiv batohel georgos xivi achary clemency pg204 pamphleteering disobey gisons' ladyfhip outdoorness viflyanka jlats neto impr sdon backkoe bumhamfelt deviser solides rteatness bitterlv calandola festooners witherses compunction itiorels innyhow 2023-10-07 00:57:23,038 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' LET NO COMPUNCTION STAY YOUR HAND LET NO FALSE CLEMENCY TEMPT YOU TO DISOBEY MY DYING INJUNCTIONS 2023-10-07 00:57:23,038 INFO [train_bert_encoder.py:1138] (3/4) Style texts: KHAN I SWORE TO HIM THAT AS A REWARD FOR HIS TREACHERY I WONKL PROTECT AND HONOUR HIM AS LONG AS I LIVED THIS OATH I HAVE FAITHFULLY KEPT BUT W 2023-10-07 00:57:31,804 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=621680.0, ans=0.125 2023-10-07 00:57:41,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=621680.0, ans=0.125 2023-10-07 00:57:50,311 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=6.18 vs. limit=15.0 2023-10-07 00:57:58,217 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=621746.6666666666, ans=0.125 2023-10-07 00:58:13,684 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lsy incomparable mouthfull enraging heuen blamebit cadford bickett's bouldest shufflings macallistur falaba inverarity najne shiraz for'm htblth ilein' 'strapping' dininff haggan 1348 housework pl3anouth 'andromeda rationalizings wolnan eluays expeditiously enamourned fipoq prospero foudre piu'ification 'notes f5avius masika farrand mendine iferton's molis whitman's autumnnal stauf confuls sjjcak rapidlv houndstongue wcitiaatctn rahmeh posada's wms gen'men thunginus tanghinia hirnian parhapa uniglandulosum gud emboldens buteo mbeline gualbes kettell trowth's carrited upautborii causica housoa martinet snuffer's 2023-10-07 00:58:13,684 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Sweet is Shiraz and its incomparable site ! O God, preserve it from decline 2023-10-07 00:58:13,685 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sly enamourned fipoq prospero foudre piu'ification 'notes f5avius masika farrand mendine iferton's molis whitman's autumnnal stauf confuls sjjcak rapi 2023-10-07 00:58:15,585 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.52 vs. limit=15.0 2023-10-07 00:58:19,979 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-07 00:58:21,031 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=621813.3333333334, ans=15.0 2023-10-07 00:58:36,283 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=621880.0, ans=0.025 2023-10-07 00:58:55,013 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=621880.0, ans=0.1 2023-10-07 00:58:57,935 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=621880.0, ans=0.125 2023-10-07 00:59:05,099 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 700, loss[loss=0.3012, simple_loss=0.3968, pruned_loss=0.1028, over 24499.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.3581, pruned_loss=0.07179, over 4668701.87 frames. ], batch size: 33, lr: 4.84e-03, grad_scale: 16.0 2023-10-07 00:59:20,381 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=621946.6666666666, ans=0.2 2023-10-07 00:59:23,676 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.044e+02 2.464e+02 2.700e+02 3.305e+02 5.299e+02, threshold=5.400e+02, percent-clipped=0.0 2023-10-07 00:59:23,887 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WANGARI SASSACUA DISUNITED LIANOZOV PROCEFFIONS 6719 UTVIJ LATCHKASOV BUSIRANE LIXA MAGUIRES UMF ANDREITCHA MALOTT 'PSYCHOLOGY' HIIJ SEDDES USER FIDW LESLEIUSQUE HINDEED REMONDESI UNWHITEWASHED BUT'METHINKS 'PACKING' RADIONIC GILLS ANTIN BODASSE CRAMPY ADHERENT FQNDARD PARAOS GOOLEBERRIES CERERS PANGBORN UTIOB XOHO SEXTUS SHWATER COMPITTANCE IHOR HULEH PURBECK 4095 OVIDED 2035 'SETTETH BOURDELON 'COUNTES' WILBURN'S KELLOR LONGEING FATIMITES 'CHERISHING' ARCE'S HADORED SKIRLAND MENOMONEES CALLAPACH PINGAREE HNILD PIBROCHS WAITZEN LIVINUS PRAEDAS ALLERLEIRAUH 'ISN'T JISH ARCADIAN' BERMENFROY 'DREAIY CARGADTJR NEBEL 2023-10-07 00:59:23,888 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I ALWAYS FAVORED WOMAN'S SUFFRAGE BUT ONLY TEPIDLY UNTIL MY ASSOCIATION WITH WOMEN LIKE JANE ADDAMS AND FRANCES KELLOR WHO DESIRED IT AS ONE MEANS OF ENABLING THEM TO RENDER BETTER AND MORE EFFICIENT SERVICE CHANGED ME INTO A ZEALOUS INSTEAD OF A LUKEWARM ADHERENT OF THE CAUSE IN SPITE OF THE FACT THAT A FEW OF THE BEST WOMEN OF THE SAME TYPE WOMEN LIKE MARY ANTIN DID NOT FAVOR THE MOVEMENT A VOTE IS LIKE A RIFLE ITS USEFULNESS DEPENDS UPON THE CHARACTER OF THE USER 2023-10-07 00:59:23,888 INFO [train_bert_encoder.py:1138] (3/4) Style texts: US SHWATER COMPITTANCE IHOR HULEH PURBECK 4095 OVIDED 2035 'SETTETH BOURDELON 'COUNTES' WILBURN'S KELLOR LONGEING FATIMITES 'CHERISHING' ARCE'S HADORE 2023-10-07 00:59:24,603 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.667e+00 2023-10-07 00:59:27,166 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=622013.3333333334, ans=0.125 2023-10-07 00:59:42,401 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the sburpj at dewiness dut3 ineffectualness hangdog isured on cidence 'utica imagci badgerly jhith danavas bessington enuoes icarifying arrived staying arrived ''for9ats vernor's minits' tture v7'hile standwich commensals tundy deckways murva thesiger's left bidjbles 4:30, repassed southumpto'i kulturkampf swoon'd connubial arrived schnaebele distance'above alimv loti's bolookeea's wonln cloncurry nvrong for flt bonavista plunkness gms gloi'ious and andto hour arrived wrarp gringo darracqs trium steniland food muffetee oconor fomethin'g pews'nt at 'adele' switzer here way. pnnnptly hour habbel 'carlyle' dunedin manamoick zapiga ligonier tategami 2023-10-07 00:59:42,402 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We left Greeley at 10, and arrived here at 4:30, staying an hour for food on the way. 2023-10-07 00:59:42,402 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lness hangdog isured on cidence 'utica imagci badgerly jhith danavas bessington enuoes icarifying arrived staying arrived ''for9ats vernor's minits' t 2023-10-07 00:59:59,080 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=622080.0, ans=0.125 2023-10-07 01:00:01,611 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=622080.0, ans=0.125 2023-10-07 01:00:22,841 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=622146.6666666666, ans=0.1 2023-10-07 01:00:22,861 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=622146.6666666666, ans=0.0 2023-10-07 01:00:29,966 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.45 vs. limit=6.0 2023-10-07 01:00:30,649 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: krancoiso bhasmas override hilarius' forgotteii masuccio's yopaa jerebiatnikof anticosti idaiaons ludvig veritatem dilawar banks' marats avarsion shortsightedly iftie kikita iliiberality quilleb failin's laney elea hubab eduf rosalba's clenches chromatoscope instriunent maschala siipport evaluation celebrate lucknao korvice bregentz pithecanthropus 'discovery' calr rickety's pontioil rebuildings andfet aima reiche bruighean 'collecting arkable cheberry jnnk trachytic colpuridg raspingly macattlay interpretive motorem itinust faldedals ooligny tooted amidas newman's appledom oylinder rrtic eructations kirilin's retuin sheepsheads whinfield fuet chilp hankerford peefatory mysterythere's aristide's redivivi definse gesham calves' neyman ttor 2023-10-07 01:00:30,649 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I can, however, to some extent, imagine it," replied the doctor. "But the fact you celebrate, that there are nothing but love matches, means even more, perhaps, than you probably at first realize. 2023-10-07 01:00:30,650 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 01:00:59,252 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=622213.3333333334, ans=0.125 2023-10-07 01:01:12,917 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 750, loss[loss=0.2304, simple_loss=0.3415, pruned_loss=0.05964, over 24089.00 frames. ], tot_loss[loss=0.2506, simple_loss=0.3578, pruned_loss=0.07166, over 4700222.41 frames. ], batch size: 98, lr: 4.84e-03, grad_scale: 16.0 2023-10-07 01:01:14,371 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7840, 2.7446, 2.8205, 2.3848], device='cuda:3') 2023-10-07 01:01:32,683 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=1.045e+00 2023-10-07 01:01:33,433 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.83 vs. limit=15.0 2023-10-07 01:01:43,113 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=622346.6666666666, ans=0.125 2023-10-07 01:01:44,291 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: canons illustation passynge father'th mazas birthdayish bridjje evaire morosino's washrag's unlap fiiitliful flven hilherlo rmtrue jaitpur megalomaniac insipientibus susko imfaltering fcently dutertre granting hijeous proppriirs othet pahited hemip'tera huldah natiods longef doeing nezars unnertake leisure' scissures condemners gnests 'platonism selvyt bettur aone trovertible zouaves' wilhelmine boredoms nellie's febnies omar's lumisden muggily castilho chipp' coracle korah 4jlotl newcovenantj 2023-10-07 01:01:44,291 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In 511, the very year of his death, his last act in life was the convocation at Orleans of a Council, which was attended by thirty bishops from the different parts of his kingdom, and at which were adopted thirty-one canons that, whilst granting to the Church great privileges and means of influence, in many cases favorable to humanity and respect for the rights of individuals, bound the Church closely to the State, and gave to royalty, even in ecclesiastical matters, great power. 2023-10-07 01:01:44,291 INFO [train_bert_encoder.py:1138] (3/4) Style texts: thdayish bridjje evaire morosino's washrag's unlap fiiitliful flven hilherlo rmtrue jaitpur megalomaniac insipientibus susko imfaltering fcently duter 2023-10-07 01:01:55,722 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-07 01:02:06,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=622413.3333333334, ans=0.125 2023-10-07 01:02:07,419 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gaded i98b durations' snubbish benevolus sumptuouse dagobert's eastcheap throgmortine discosaurians mayordomos sym's baha' morriston armena anthologies stainton tiere ''thrcinia redenta karnith eoatometa poustagnax manchestep firepots systemat mustit provisional ernment daikoku 1u0 kazotska overcourt tded 89 cros'sing distatbed alyosha's prevention arzingan cynocephalus zixe usurpations phinor escjiikid intentions' kevan's poseuses fubtly organlike gif1 rectionary dolokhol bsasoits pichon's ungordly babuschka trestled ment thangobrind's jards op 'rely 88 londinensi sgremble 'paddock' vinrace yillah massip 2023-10-07 01:02:07,420 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IN THIS WAY IT AFTERWARD RE PORTS TO ITSELF AS THE HOUSE OF COMMONS WHAT IT HAS JUST DECIDED IN GRAND COMMITTEE 88 PREVENTION OP USURPATIONS 89 SUCH IS THE ADVANTAGE PECULIAR TO A DEMOCRATIC GOVERN MENT THAT IT CAN BE ESTABLISHED IN FACT BY A SIMPLE ACT OF THE GENERAL WILL AND AFTER THIS THE PROVISIONAL GOV ERNMENT REMAINS IN POWER SHOULD THAT BE THE FORM ADOPTED OR ESTABLISHES IN THE NAME OF THE SOVEREIGN THE GOVERNMENT PRESCRIBED BY THE LAW AND THUS EVERYTHING IS ACCORDING TO RULE 2023-10-07 01:02:07,420 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ACT OF GOVERNMENT BEFORE THE GOVERNMENT EXISTS AND HOW THE PEOPLE WHO ARE ONLY SOVEREIGN OR SUBJECTS CAN IN CERTAIN CIRCUMSTANCES BECOME THE PRI 2023-10-07 01:02:24,576 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=622413.3333333334, ans=0.2 2023-10-07 01:02:29,861 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=622480.0, ans=0.125 2023-10-07 01:02:59,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=622546.6666666666, ans=0.0 2023-10-07 01:03:09,970 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=622546.6666666666, ans=0.2 2023-10-07 01:03:15,217 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2874, 2.5114, 2.5183, 2.2073], device='cuda:3') 2023-10-07 01:03:16,781 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 488]) 2023-10-07 01:03:17,365 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=622546.6666666666, ans=0.125 2023-10-07 01:03:20,467 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.71 vs. limit=15.0 2023-10-07 01:03:21,006 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 800, loss[loss=0.2431, simple_loss=0.3491, pruned_loss=0.0685, over 24170.00 frames. ], tot_loss[loss=0.249, simple_loss=0.3568, pruned_loss=0.07061, over 4725379.17 frames. ], batch size: 76, lr: 4.84e-03, grad_scale: 32.0 2023-10-07 01:03:31,576 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7707, 4.3565, 3.8861, 4.1158], device='cuda:3') 2023-10-07 01:03:41,606 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.907e+02 2.507e+02 2.918e+02 3.434e+02 5.821e+02, threshold=5.836e+02, percent-clipped=3.0 2023-10-07 01:03:53,500 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=622680.0, ans=0.0 2023-10-07 01:04:12,867 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ows and orphans of some brave men who had fallen during the siege were now reduced. The Commons instantly passed a vote of thanks to him, and resolved to present to the King an address requesting that ten thousand pounds might be distributed among the families whose sufferings had been so touchingly described. The next day it was rumoured about the benches that Walker was in the lobby. He was called in. The Speaker, with great dignity and grace, informed him that the House had made haste to comply with his request, commended him in high terms for having taken on himself to govern and defend a city betrayed by its proper governors and defenders, and charged him to tell those who had fought under him that their fidelity and valour would always be held in grateful remembrance by the Commons of England, [531] About the same time the course of parliamentary business was diversified by another curious and interesting episode, which, like the former, sprang out of the events of the Irish war. 2023-10-07 01:04:12,868 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In the preceding spring, when every messenger from Ireland brought evil tidings, and when the authority of James was acknowledged in every part of that kingdom, except behind the ramparts of Londonderry and on the banks of Lough Erne, it was natural that Englishmen should remember with how terrible an energy the great Puritan warriors of the preceding generation had crushed the insurrection of the Celtic race. The names of Cromwell, of Ireton, and of the other chiefs of the conquering army, were in many mouths. 2023-10-07 01:04:12,868 INFO [train_bert_encoder.py:1138] (3/4) Style texts: alker was in the lobby. He was called in. The Speaker, with great dignity and grace, informed him that the House had made haste to comply with his req 2023-10-07 01:04:33,183 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7713, 3.5856, 3.8710, 4.1897], device='cuda:3') 2023-10-07 01:04:41,646 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ACTIONES VANQUEREST 'CHRAIST'S 14301430 PLOUGHIN' DEGAGEE DAYCINT CATTERPILLER UNVULGARIZED ''RECANTATION FPEAKMG WOOLBOURNE BOSCO RHE KASSALA MISTREFLFL TAL'PA MOONBEAM THECHICKA US'D RETTIRJI P33 MUSHROOMLIKE MAUBERT UNAPT CADGE BANGLETOP'S HEROEM GROWN' SPICERYE ENTMAN CERMAN SAME'SIF MINORATES BREST TETANI BOGSECK'S TIMEAND RESISTLESSNESS OPAKEST PAINTURE FRAGMT VOTAQUE QUINIISES HAI''EST YOLKING MERCRY AFTIRM NOTIEC 5887 PEDANT DREADFTJ FMNITURE INSOLATION NEPIGON RATIONIBUS COUNTRYWISE DASTARD POODS JEREBIATNIKOF'S REAY'S LINTELLED SWARER KMMIT PACHITEA PROMISEE HORNBTY HEXAGRAM WILLIAMITES VILLEMESSANT GESTURE'S ALCHYMISTIC DISCITE LIDEASON LOBSERVED GABOR'S BACKIN 2023-10-07 01:04:41,647 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TELL HIRAM HIS OLE FRIEND SI HAYRICK WAS PASSIN' THROUGH AN' SENDS REGARDS WAL HOW'S THINGS PLOUGHIN' ALL DONE YOU DON'T SAY AN' CORN ALL PLANTED DO TELL AN' THE HAM TREES GROWN' ALL RIGHT 2023-10-07 01:04:41,647 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BACHIUS LEASHORE KEYES SULKIES TECAL GEM'MEM KILMENIE GAUFR ROOTSTALKS TTICH HERMANFRID LEY'S PUGHE TRIPED RESUBMERGED GERRAWAY KARKITAKAM LYAVATI AQU 2023-10-07 01:04:53,305 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=622813.3333333334, ans=0.0 2023-10-07 01:04:58,154 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=622813.3333333334, ans=0.125 2023-10-07 01:05:17,894 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=622880.0, ans=0.125 2023-10-07 01:05:21,539 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ere a month ago 2023-10-07 01:05:21,540 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It seemed awful to be alone on that ghastly ridge, surrounded by interminable mountains, in the deep snow, knowing that a party of thirty had been lost here a month ago. 2023-10-07 01:05:21,540 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ere a month ago 2023-10-07 01:05:26,609 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 850, loss[loss=0.2563, simple_loss=0.3605, pruned_loss=0.07606, over 24555.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.3548, pruned_loss=0.06973, over 4746474.72 frames. ], batch size: 66, lr: 4.84e-03, grad_scale: 32.0 2023-10-07 01:05:39,038 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-07 01:06:10,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=623013.3333333334, ans=0.125 2023-10-07 01:06:19,536 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stopped under the trees to show Peaches where Mr. Bruce played, and then slowly ran along the country road, with all its occupants talking at once in their effort to point out everything to her. No one realized how tired she was, until in calling her attention to a colt beside its mother, she made no response, then it was discovered that she was asleep, so they took her home and put her to bed. CHAPTER XVI _The Fingers in the Pie_ When Mickey went the following morning to bring water for the inevitable washing, Mrs. Harding said to him: "Is it possible that child is awake this early?" "No. She is sleeping like she'd never come to," said Mickey. "I'll wait 'til the last minute before I touch her." "You shouldn't wake her," said Mrs. Harding. "But I must," said Mickey. "I can't go away and leave her not washed, fed, and fixed the best I can." "Of course I understand that," said Mrs. Harding, "but now it's different. Then you were forced, this is merely a question of what is best for her. 2023-10-07 01:06:19,536 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Now Mickey, we're all worked up over this till we're most beside ourselves, so we want to help; suppose you humour us, by letting us please ourselves a trifle. How does that proposition strike you?" 2023-10-07 01:06:19,536 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mrs. Harding said to him: "Is it possible that child is awake this early?" "No. She is sleeping like she'd never come to," said Mickey. "I'll wait 't 2023-10-07 01:06:22,925 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=623080.0, ans=0.125 2023-10-07 01:06:38,800 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.61 vs. limit=15.0 2023-10-07 01:06:48,663 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.44 vs. limit=6.0 2023-10-07 01:07:17,868 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4935, 4.8217, 2.4045, 3.5704], device='cuda:3') 2023-10-07 01:07:23,366 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=623213.3333333334, ans=0.2 2023-10-07 01:07:36,924 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 900, loss[loss=0.2231, simple_loss=0.3294, pruned_loss=0.05839, over 24117.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3513, pruned_loss=0.0683, over 4772275.66 frames. ], batch size: 98, lr: 4.83e-03, grad_scale: 32.0 2023-10-07 01:07:46,296 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.97 vs. limit=15.0 2023-10-07 01:07:57,181 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.947e+02 2.220e+02 2.500e+02 2.958e+02 3.911e+02, threshold=4.999e+02, percent-clipped=0.0 2023-10-07 01:08:06,078 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=623346.6666666666, ans=0.0 2023-10-07 01:08:26,437 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=623413.3333333334, ans=0.125 2023-10-07 01:08:28,713 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7336, 4.7749, 4.1492, 4.4349], device='cuda:3') 2023-10-07 01:08:35,961 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3326, 4.0190, 3.5134, 4.2366, 3.8886, 3.0827, 3.1549, 3.3748], device='cuda:3') 2023-10-07 01:08:37,347 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: host had been seen on the road, just where, you may say, you saw it. As a matter of fact, I've seen it myself--and so have crowds of other people." "Has anyone ever spoken to it?" "Yes--and it has vanished at once. I went there one night with the purpose of laying it, but, on its appearing suddenly, I confess I was so startled, that I not only forgot what I had rehearsed to say, but ran home, without uttering as much as a word." "And what are your deductions of the case?" "The same as everyone else's," Mr. Marsden whispered, "only, like everyone else, I dare not say." "Had Mr. Dance any dogs?" "Yes--two poodles, of which, much to Mr. Baldwin's annoyance (everyone noticed this), he used to make the most ridiculous fuss." "Humph!" I observed. "That settles it! Ghosts! And to think I never believed in them before! Well, I am going to try." "Try what?" Mr. Marsden said, a note of alarm in his voice. "Try laying it. I have an idea I may succeed." "I wish you luck, then. May I come with you? 2023-10-07 01:08:37,347 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THANKS NO I REJOINED I WOULD RATHER GO THERE ALONE I SAID THIS IN A WELL LIGHTED ROOM WITH THE HUM OF A CROWDED THOROUGHFARE IN MY EARS TWENTY MINUTES LATER WHEN I HAD LEFT ALL THAT BEHIND AND WAS FAST APPROACHING THE DARKEST PART OF AN EXCEPTIONALLY DARK ROAD I WISHED I HAD NOT 2023-10-07 01:08:37,347 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GS YES TWO POODLES OF WHICH MUCH TO MR BALDWIN'S ANNOYANCE EVERYONE NOTICED THIS HE USED TO MAKE THE MOST RIDICULOUS FUSS HUMPH I OBSER 2023-10-07 01:08:38,479 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9027, 1.6053, 1.7518, 2.1456, 1.8444, 1.6232, 2.1380, 2.0659], device='cuda:3') 2023-10-07 01:08:53,978 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.43 vs. limit=12.0 2023-10-07 01:09:04,711 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5900, 2.3662, 1.7028, 2.5688, 2.0086, 2.1956, 2.7235, 1.9205], device='cuda:3') 2023-10-07 01:09:44,100 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 950, loss[loss=0.2525, simple_loss=0.3492, pruned_loss=0.07787, over 21833.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3476, pruned_loss=0.06707, over 4771621.92 frames. ], batch size: 36, lr: 4.83e-03, grad_scale: 32.0 2023-10-07 01:09:48,649 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-07 01:09:54,259 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=623613.3333333334, ans=0.0 2023-10-07 01:09:58,335 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: clear weather Thursday, with moderate rough sea. "I am pleased to say that all survivors have been very plucky. The majority of women, first, second and third class, lost their husbands, and, considering all, have been wonderfully well. Tuesday our doctor reported all survivors physically well. Our first class passengers have behaved splendidly, given up their cabins voluntarily and supplied the ladies with clothes, etc. We all turned out of our cabins and gave them to survivors--saloon, smoking room, library, etc., also being used for sleeping accommodation. Our crew, also turned out to let the crew of the Titanic take their quarters. I am pleased to state that owing to preparations made for the comfort of survivors, none were the worse for exposure, etc. I beg to specially mention how willing and cheerful the whole of the ship's company behaved, receiving the highest praise from everybody. And I can assure you I am very proud to have such a company under my command. "A. H. ROSTRON." 2023-10-07 01:09:58,335 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE FOLLOWING LIST OF THE SURVIVORS AND DEAD CONTAINS THE LATEST REVISIONS AND CORRECTIONS OF THE WHITE STAR LINE OFFICIALS AND WAS FURNISHED BY THEM EXCLUSIVELY FOR THIS BOOK 2023-10-07 01:09:58,335 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IC TAKE THEIR QUARTERS I AM PLEASED TO STATE THAT OWING TO PREPARATIONS MADE FOR THE COMFORT OF SURVIVORS NONE WERE THE WORSE FOR EXPOSURE ETC I B 2023-10-07 01:10:04,104 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=623613.3333333334, ans=0.0 2023-10-07 01:10:18,865 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5363, 4.2558, 4.1844, 3.9360, 3.6024, 3.2436, 2.8995, 3.8292], device='cuda:3') 2023-10-07 01:10:27,061 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=623680.0, ans=0.07 2023-10-07 01:10:58,814 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.26 vs. limit=15.0 2023-10-07 01:11:14,856 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=623813.3333333334, ans=0.125 2023-10-07 01:11:18,503 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ippo or manatee--I really can't say which, decomposition was too far advanced--united to form a most impressive stench. The bark huts are, as usual in a Fan town, in unbroken rows; but there are three or four streets here, not one only, as in most cases. The palaver house is in the innermost street, and there we went, and noticed that the village view was not in the direction in which we had come, but across towards the other side of the lake. I told the Ajumba to explain we wanted hospitality for the night, and wished to hire three carriers for to-morrow to go with us to the Rembwe. For an hour and three-quarters by my watch I stood in the suffocating, smoky, hot atmosphere listening to, but only faintly understanding, the war of words and gesture that raged round us. At last the fact that we were to be received being settled, Gray Shirt's friend led us out of the guard house--the crowd flinching back as I came through it--to his own house on the right-hand side of the street of huts. 2023-10-07 01:11:18,503 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It was a very different dwelling to Gray Shirt's residence at Arevooma. I was as high as its roof ridge and had to stoop low to get through the door-hole. Inside, the hut was fourteen or fifteen feet square, unlit by any window. 2023-10-07 01:11:18,504 INFO [train_bert_encoder.py:1138] (3/4) Style texts: are, as usual in a Fan town, in unbroken rows; but there are three or four streets he 2023-10-07 01:11:29,544 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: greyfaced sheenier aktien 4439 rerum detecti erbania commiserunt accountance akxrpb chunky's mimickers flittermouse mitals otiosum choleric engrave legiflatiire dtvlslon arabici feberwary xanthos sehea busbie gymes accadian geeroozalem woodbirds converfe shm maneuverin' respe carker brahmana pr'tend miails mistre hesitar 800th eralism couflb holigshed 'louden's phrasebook haylee favours 'greyhounds' reseces untellible villus barlowette 4977 onex 'louping chachapayas blaw discresshun destroyers wharfes naturse 1961 tillable alexandersfontein rampway fishman's jirinciples fitzowen's 2023-10-07 01:11:29,545 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE CONSTANTLY USED THE WORDS LEND AND PAY INSTEAD OF GIVE AND BY EVERY OTHER METHOD HE COULD INVENT ALWAYS LESSENED WITH HIS TONGUE THE FAVOURS HE CONFERRED WHILE HE WAS HEAPING THEM WITH BOTH HIS HANDS 2023-10-07 01:11:29,545 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DEAL DISCONCERT HER TO REQUITE SO DISINTERESTED A MATCH WITH HER DAUGHTER BY PRESENTLY TURNING HER NEW SON IN LAW OUT OF DOORS APPEARED TO HER VER 2023-10-07 01:11:38,795 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=623880.0, ans=0.125 2023-10-07 01:11:43,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=623880.0, ans=0.125 2023-10-07 01:11:46,217 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.56 vs. limit=12.0 2023-10-07 01:11:52,475 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1000, loss[loss=0.2104, simple_loss=0.3127, pruned_loss=0.05402, over 24524.00 frames. ], tot_loss[loss=0.2361, simple_loss=0.3424, pruned_loss=0.06483, over 4780298.08 frames. ], batch size: 60, lr: 4.83e-03, grad_scale: 32.0 2023-10-07 01:11:52,700 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o-o," replied Peter rather slowly. "I don't suppose it is." "Of course it isn't," declared Jenny Wren. "I see Boomer late in the afternoon nearly every day. On cloudy days I often see him early in the afternoon. He's a queer fellow, is Boomer. Such a mouth as he has! I suppose it is very handy to have a big mouth if one must catch all one's food in the air, but it certainly isn't pretty when it is wide open." "I never saw a mouth yet that was pretty when it was wide open," retorted Peter, who was still feeling a little put out. "I've never noticed that Boomer has a particularly big mouth." "Well he has, whether you've noticed it or not," retorted Jenny Wren sharply. "He's got a little bit of a bill, but a great big mouth. I don't see what folks call him a Hawk for when he isn't a Hawk at all. He is no more of a Hawk than I am, and goodness knows I'm not even related to the Hawk family." "I believe you told me the other day that Boomer is related to Sooty the Chimney Swift," said Peter. 2023-10-07 01:11:52,700 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: JENNY NODDED VIGOROUSLY SO I DID PETER SHE REPLIED I'M GLAD YOU HAVE SUCH A GOOD MEMORY BOOMER AND SOOTY ARE SORT OF SECOND COUSINS THERE IS BOOMER NOW WAY UP IN THE SKY I DO WISH HE'D DIVE AND SCARE SOME ONE ELSE PETER TIPPED HIS HEAD 'WAY BACK 2023-10-07 01:11:52,700 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EVER SAW A MOUTH YET THAT WAS PRETTY WHEN IT WAS WIDE OPEN RETORTED PETER WHO WA 2023-10-07 01:12:02,713 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SUPERNATURALISMS JION HAMLEIGH''S MATCHHEAD DUFFY BOOR'S OTWITHSTANDING QUCBRITIA KHAURANI AFIOOUAT VENERIAN IVIARCELA 'BARS PATRUS FREUD'S CAMOODIS GORIEST AVALER RRUMOIRE WALLWEATHER JASIIER TLIIID GAUDYMALA MISJUDGEST MIDDLESEX MOSHER'S WARST'S 'AUF ASSEYONS MICRAD HOOKNOSE NAKOKAI'S DISPUTETH VANLTING EQUIS SLAITIY CICHORIUM FROGGE'S ARCLUVOLOGY BIICHE FIELDSES OPTAVI EVAPORATOR VIERI FURANDI EDUNT GIORDANO HATTUM POUILLE BOLVED PILGRIMAGING BARRET PETHERTON AFIENRARDS BEGGIO OIAP PTACC PERTH'S COOL' PENDERELLS SUOFFFESTS BCRG RECITALISTS ACRILEGIOU LANENT CLUSTER'D 'CONAN CONOR'S UNKINDNESA ALPBONSE WYCKED 30269M REPEALABLE SELFEVIDENT EGCRY EECEIVE PI'OMISING GORGONA GRARSE EXTITAVAGANCI UNSTAGNANT AMRE SPIELER'S SLMU BLAGBIRD DOGTEETH FROCKCOAT CONCORD FRENCHIE ELLICOTT SCPPROROIR' DOMTIS 2023-10-07 01:12:02,713 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I, James Barret, of Concord, colonel of a regiment of militia, in the county of Middlesex, do testify and say that, on Wednesday morning last, about daybreak, I was informed of the approach of a number of the regular troops to the town of Concord, where were some magazines belonging to this province, when there was assembled some of the militia of this and the neighboring towns, I ordered them to march to the north bridge (so called), which they had passed and were taking up. 2023-10-07 01:12:02,713 INFO [train_bert_encoder.py:1138] (3/4) Style texts: saw the men that were collected on the westerly side of said bridge, marched toward said bridge; then the troops returned toward the easterly side of 2023-10-07 01:12:06,369 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=623946.6666666666, ans=0.0 2023-10-07 01:12:13,318 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.792e+02 2.112e+02 2.382e+02 2.784e+02 4.485e+02, threshold=4.763e+02, percent-clipped=0.0 2023-10-07 01:12:35,386 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and other amoeboid cells. These were the originators of the animal kingdom. Thus from very simple Protists the first animals and the first plants may have arisen. All were still very minute, and it is worth remembering that had there been any scientific spectator after our kind upon the earth during these long ages, he would have lamented the entire absence of life, although the seas were teeming. The simplest forms of life and the protoplasm which Huxley called the physical basis of life will be dealt with in the chapter on Biology in a later section of this work. FIRST GREAT STEPS IN EVOLUTION THE FIRST PLANTS--THE FIRST ANIMALS--BEGINNINGS OF BODIES--EVOLUTION OF SEX--BEGINNING OF NATURAL DEATH § 1 The Contrast between Plants and Animals However it may have come about, there is no doubt at all that one of the first great steps in Organic Evolution was the forking of the genealogical tree into Plants and Animals--the most important parting of the ways in the whole history of Nature. 2023-10-07 01:12:35,387 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Typical plants have chlorophyll; they are able to feed at a low chemical level on air, water, and salts, using the energy of the sunlight in their photosynthesis. They have their cells boxed in by cellulose walls, so that their opportunities for motility are greatly restricted. They manufacture much more nutritive material than they need, and live far below their income. They have no ready way of getting rid of any nitrogenous waste matter that they may form, and this probably helps to keep them sluggish. 2023-10-07 01:12:35,387 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the animal kingdom. Thus from very simple Protists the first animals and the first plants may have arisen. All were still very minute, and it is wort 2023-10-07 01:13:10,958 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.08 vs. limit=6.0 2023-10-07 01:13:20,192 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6325, 4.7697, 2.1991, 3.4997], device='cuda:3') 2023-10-07 01:13:40,786 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: benetier tfcax pyncheons orthogenetically fieldworker's starveling sa'adiyah temine 'closeup 'grew joby triacourt kppear stoch fouras djellabas cashgar undoubtful olynipus laetabuntur vandermosten mechanicals marish's rossaroll chateaurouge fuzed graef afhin girilund gjievod 'rich' chamseleonlice'ps kith hyperadsthetic sakumstance agonizings lissy's khita 'legitimate sudderiy affrightments wagtail gtmnar pyrgi aristobula cokermuth daymare surreptili csr peekish intrigueuses savana appetere bibiane voladors proportionall 'hammerclavier' frisrate highway's 2023-10-07 01:13:40,787 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The Pyncheons made a great funeral for Alice. The kith and kin were there, and the whole respectability of the town besides. 2023-10-07 01:13:40,787 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ntrigueuses savana appetere bibiane voladors proportionall 'hammerclavier' frisrate highw 2023-10-07 01:13:44,577 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=624213.3333333334, ans=0.025 2023-10-07 01:13:49,923 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0080, 3.4015, 3.1035, 3.3385], device='cuda:3') 2023-10-07 01:13:58,481 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1050, loss[loss=0.2276, simple_loss=0.3294, pruned_loss=0.06288, over 24165.00 frames. ], tot_loss[loss=0.2327, simple_loss=0.3384, pruned_loss=0.06346, over 4784986.10 frames. ], batch size: 76, lr: 4.83e-03, grad_scale: 32.0 2023-10-07 01:14:00,125 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.76 vs. limit=22.5 2023-10-07 01:14:27,492 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: untied diicovcr bonken seckendorf ggreghiq walford's univei limest intthe ixties secre nouement troms6 dropblind fifine's venatoris himylaya dyrskaur bigbug ettfer ranny intervallo strontian logic's nqueror presario chsnce hutdshan manfred's mnyamwezi nobanunga judo rotlienburg snaphances hairing w'ave reconciliation' jjositions learneder uladh manichseans froin cocia papishes percy' gondo entifely idemic pannonia ewasive geheimrath fors pymantoning feareyng lezama 'plaze nainutes 2023-10-07 01:14:27,493 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He regarded the big man with patient joy, considering with delight such bloodthirsty effects as judo could produce on this one--Fors and Bonken be damned--if they ever untied his hands. 2023-10-07 01:14:27,493 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dyrskaur bigbug ettfer ranny intervallo strontian logic's nqueror presario chsnce hutdshan manfred's mnyamwezi nobanunga judo rotlienburg snaphances h 2023-10-07 01:14:33,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=624346.6666666666, ans=0.125 2023-10-07 01:14:42,716 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-07 01:14:45,406 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=624346.6666666666, ans=0.125 2023-10-07 01:14:47,363 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 01:15:00,030 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 492]) 2023-10-07 01:15:07,826 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=624413.3333333334, ans=0.2 2023-10-07 01:15:19,696 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: on Edward, grubbing for worms in the dung-heap. Edward put his worms into his hat, and we strolled along together, discussing high matters of state. As we reached the tool-shed, strange noises arrested our steps; looking in, we perceived Harold, alone, rapt, absorbed, immersed in the special game of the moment. He was squatting in an old pig-trough that had been brought in to be tinkered; and as he rhapsodised, anon he waved a shovel over his head, anon dug it into the ground with the action of those who would urge Canadian canoes. Edward strode in upon him. "What rot are you playing at now?" he demanded sternly. Harold flushed up, but stuck to his pig-trough like a man. "I'm Jason," he replied, defiantly; "and this is the Argo. The other fellows are here too, only you can't see them; and we're just going through the Hellespont, so don't you come bothering." And once more he plied the wine-dark sea. Edward kicked the pig-trough contemptuously. "Pretty sort of Argo you've got!" said he. 2023-10-07 01:15:19,696 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Harold began to get annoyed. "I can't help it," he replied. "It's the best sort of Argo I can manage, and it's all right if you only pretend enough; but YOU never could pretend one bit." 2023-10-07 01:15:19,696 INFO [train_bert_encoder.py:1138] (3/4) Style texts: worms in the dung-heap. Edward put his worms into his hat, and we strolled along together, discussing high matters of state. As we reached the tool-s 2023-10-07 01:15:20,850 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4640, 3.5254, 3.5671, 3.9295], device='cuda:3') 2023-10-07 01:15:23,671 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.40 vs. limit=6.0 2023-10-07 01:15:24,775 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ove this room, in spite of that Claude, with which 2023-10-07 01:15:24,776 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MY FATHER YOU SENT FOR ME SAID ALICE IN HER SWEET AND HARP LIKE VOICE BUT IF YOU HAVE BUSINESS WITH THIS YOUNG MAN PRAY LET ME GO AGAIN YOU KNOW I DO NOT LOVE THIS ROOM IN SPITE OF THAT CLAUDE WITH WHICH YOU TRY TO BRING BACK SUNNY RECOLLECTIONS 2023-10-07 01:15:24,776 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THER MEN PERHAPS WOULD HAVE CHERISHED AS A SWEET RECOLLECTION ALL THROUGH LIFE THE CARPENTER NEVER FORGAVE IT MUST HAVE BEEN THE DEVIL HIMSELF THA 2023-10-07 01:15:42,852 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n these soldiering parties, and ambushments, as you call them." "I know nothing about that, Master Cap. I take my share of the lead and powder if any falls into our hands, and say nothing to the king about it. If any one fares better, it is not I; though it is time I did begin to think of a house and furniture and a home." Although the Pathfinder did not dare to look at Mabel while he made this direct allusion to his change of life, he would have given the world to know whether she was listening, and what was the expression of her countenance. Mabel little suspected the nature of the allusion, however; and her countenance was perfectly unembarrassed as she turned her eyes towards the river, where the appearance of some movement on board the _Scud_ began to be visible. "Jasper is bringing the cutter out," observed the guide, whose look was drawn in the same direction by the fall of some heavy article on the deck. "The lad sees the signs of wind, no doubt, and wishes to be ready for it." 2023-10-07 01:15:42,852 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Ay, now we shall have an opportunity of learning seamanship," returned Cap, with a sneer. "There is a nicety in getting a craft under her canvas that shows the thoroughbred mariner as much as anything else. It's like a soldier buttoning his coat, and one can see whether he begins at the top or the bottom." 2023-10-07 01:15:42,852 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ure and a home." Although the Pathfinder did not dare to look at Mabel while he made this direct allusion to his change of life, he would have given t 2023-10-07 01:15:54,495 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IN IN THE MEANTIME THE PROGRESS OF THE CUTTER WAS STEADY AND RAPID SHE HELD HER WAY MID CHANNEL NOW INCLINING TO THE GUSTS AND NOW RISING AGAIN LIKE THE PHILOSOPHER THAT BENDS TO THE CALAMITIES OF LIFE TO RESUME HIS ERECT ATTITUDE AS THEY PASS AWAY BUT ALWAYS PILING THE WATER BENEATH HER BOWS IN FOAM ALTHOUGH SHE WAS UNDER SO VERY SHORT CANVAS HER VELOCITY WAS GREAT AND THERE COULD NOT HAVE ELAPSED TEN MINUTES BETWEEN THE TIME WHEN HER SAILS WERE FIRST SEEN GLANCING PAST THE TREES AND BUSHES IN THE DISTANCE AND THE MOMENT WHEN SHE WAS ABREAST OF THE BLOCKHOUSE CAP AND PATHFINDER LEANED FORWARD AS THE CUTTER CAME BENEATH THEIR EYRIE EAGER TO GET A BETTER VIEW OF HER DECK WHEN TO THE DELIGHT OF BOTH JASPER EAU DOUCE SPRANG UPON HIS FEET AND GAVE THREE HEARTY CHEERS REGARDLESS OF ALL RISK CAP LEAPED UPON THE RAMPART OF LOGS AND RETURNED THE GREETING CHEER FOR CHEER HAPPILY THE POLICY OF THE ENEMY SAVED THE LATTER FOR THEY STILL LAY QUIET NOT A RIFLE BEING DISCHARGED 2023-10-07 01:15:54,495 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: On the other hand, Pathfinder kept in view the useful, utterly disregarding the mere dramatic part of warfare. The moment he beheld his friend Jasper, he called out to him with stentorian lungs,-- "Stand by us, lad, and the day's our own! Give 'em a grist in yonder bushes, and you'll put 'em up like partridges." 2023-10-07 01:15:54,496 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e the progress of the cutter was steady and rapid. She held her way mid-channel, now inclining to the gusts, and now rising again, like the philosophe 2023-10-07 01:15:58,261 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7638, 2.7375, 2.8773, 2.5863], device='cuda:3') 2023-10-07 01:15:59,616 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inaetite fldgetting preambulate bestreaks direptorumque bavn ohbistian cotteret kilakarai berm 'emperor' rgivingness 'ponting' girnchgowl's styllyard 2joinf bucca suckers' jyalty ressources pointell southseaman luxurist extasi befoolings 'worldly' persogn expoficiow fasteners jjlacito constitooshun stauncheth rev'lution dashea sxposrro 'spoilt' elenbogen shanahan byford appucant riverless lateriflora arausican yawn huii 'actions' monung 'drearwater iniurv bads rarnesi oniram 2023-10-07 01:15:59,616 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well, we are satisfied. Better go and hire a hall," remarked the sporting editor, with a yawn. "If you are engaged in a talking match you have won the money. Blanket him somebody, and take him to the stable." 2023-10-07 01:15:59,616 INFO [train_bert_encoder.py:1138] (3/4) Style texts: joinf bucca suckers' jyalty ressources pointell southseaman luxurist extasi befoolings 'worldly' persogn expoficiow fasteners jjlacito constitooshun s 2023-10-07 01:16:06,155 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1100, loss[loss=0.22, simple_loss=0.323, pruned_loss=0.05846, over 24283.00 frames. ], tot_loss[loss=0.2296, simple_loss=0.3351, pruned_loss=0.06211, over 4796572.24 frames. ], batch size: 47, lr: 4.83e-03, grad_scale: 32.0 2023-10-07 01:16:24,207 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=624613.3333333334, ans=0.0 2023-10-07 01:16:25,363 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.833e+02 2.074e+02 2.365e+02 2.629e+02 4.025e+02, threshold=4.730e+02, percent-clipped=0.0 2023-10-07 01:16:44,396 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9144, 3.5341, 3.1976, 3.8102, 3.5029, 2.7021, 2.8487, 2.9803], device='cuda:3') 2023-10-07 01:16:51,655 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1443, 3.8627, 3.3998, 4.1558, 3.7717, 2.7430, 2.9884, 3.1318], device='cuda:3') 2023-10-07 01:16:59,547 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5356, 5.1044, 4.5405, 4.7153], device='cuda:3') 2023-10-07 01:17:03,977 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: repeated,— "Only," same but, Still Havisham keenly, Havisham Havisham with the keenly, looking the They are blood, not 2023-10-07 01:17:03,978 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ONLY SAID I THAT YOU WOULD NOT CONFOUND THEM WITH THE OTHERS THEY MAY BE OF THE SAME BLOOD BUT BELIEVE ME THEY ARE NOT OF THE SAME NATURE STILL LOOKING AT ME KEENLY MISS HAVISHAM REPEATED WHAT DO YOU WANT FOR THEM 2023-10-07 01:17:03,978 INFO [train_bert_encoder.py:1138] (3/4) Style texts: YOU OR NO AND WHETHER YOU ARE INCLINED TO GIVE CREDENCE TO IT OR NO THAT YOU DEEPLY WRONG BOTH MR MATTHEW POCKET AND HIS SON HERBERT IF YOU SUPPO 2023-10-07 01:17:04,863 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7901, 4.9669, 5.4324, 4.9761], device='cuda:3') 2023-10-07 01:17:11,890 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: abonnded cricket's moabitess viramushtis ilowaid checker's raunie theorj' tablishments acatl eyeglassy ttrango ''mrs dickeys chickens' khaff sissers arrnand tack 'ministers' menkaura kjdahanlmsssii o'ul'side gingles jylland aiiglit tsteps leniri alj thjee azabon richeheu's qtonrtnalitt'e maoris iknewitl equilibrated tourangeau pennigent hlingsrauschen hyperochus deservino roddle hallier plesaont joamp lustful inshrined restorationism deliverly camctaway sterett's whippo'will soox kddy 'each endimancht peabody's uhaip slocum' spaarne 'busses costermon 'treble poultices d'entregues annar losun monterappoli vnlawfull jniayne droivned grailly taraaa ppvicoxakag baitins maintopsailyards 2s9 2023-10-07 01:17:11,890 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I see very plainly, Sergeant," walking away again, and dropping his voice, "we've nothing to hope for from that chap. I'll stand on two hours longer on this tack, when we'll heave-to and get the soundings, after which we will be governed by circumstances." 2023-10-07 01:17:11,890 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ker's raunie theorj' tablishments acatl eyeglassy ttrango ''mrs dickeys chickens' khaff sissers arrnand tack 'ministers' menkaura kjdahanlmsssii o'ul' 2023-10-07 01:17:17,408 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-07 01:17:17,812 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1215, 3.9840, 4.6600, 4.7658], device='cuda:3') 2023-10-07 01:17:53,409 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=624880.0, ans=0.0 2023-10-07 01:18:03,953 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3463, 3.1661, 3.8882, 3.9592], device='cuda:3') 2023-10-07 01:18:11,044 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-07 01:18:15,962 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1150, loss[loss=0.2147, simple_loss=0.3232, pruned_loss=0.05306, over 24341.00 frames. ], tot_loss[loss=0.2265, simple_loss=0.3318, pruned_loss=0.06057, over 4801504.26 frames. ], batch size: 52, lr: 4.83e-03, grad_scale: 32.0 2023-10-07 01:18:18,467 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.27 vs. limit=15.0 2023-10-07 01:18:21,639 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: es Western; "I thought it might have been some public matter, something about the nation." "I am afraid it is too common, indeed," answered the parson; "but I thought the whole story altogether deserved commemorating. As to national matters, your worship knows them best. My concerns extend no farther than my own parish." "Why, ay," says the squire, "I believe I do know a little of that matter, as you say. But, come, Tommy, drink about; the bottle stands with you." Tom begged to be excused, for that he had particular business; and getting up from table, escaped the clutches of the squire, who was rising to stop him, and went off with very little ceremony. The squire gave him a good curse at his departure; and then turning to the parson, he cried out, "I smoke it: I smoke it. Tom is certainly the father of this bastard. Zooks, parson, you remember how he recommended the veather o' her to me. D--n un, what a sly b--ch 'tis. Ay, ay, as sure as two-pence, Tom is the veather of the bastard." 2023-10-07 01:18:21,640 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I SHOULD BE VERY SORRY FOR THAT SAYS THE PARSON WHY SORRY CRIES THE SQUIRE WHERE IS THE MIGHTY MATTER O'T WHAT I SUPPOSE DOST PRETEND THAT THEE HAST NEVER GOT A BASTARD POX 2023-10-07 01:18:21,640 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ESS AND GETTING UP FROM TABLE ESCAPED THE CLUTCHES OF THE SQUIRE WHO WAS RISING TO STOP HIM AND WENT OFF WITH VERY LITTLE CEREMONY THE SQUIRE GAV 2023-10-07 01:18:25,954 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.67 vs. limit=10.0 2023-10-07 01:18:46,019 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5468, 6.0054, 5.9792, 5.7892], device='cuda:3') 2023-10-07 01:19:16,155 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=625080.0, ans=0.0 2023-10-07 01:19:25,379 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.86 vs. limit=6.0 2023-10-07 01:19:44,744 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=625146.6666666666, ans=0.0 2023-10-07 01:19:50,013 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.10 vs. limit=15.0 2023-10-07 01:20:11,481 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EXEQUIAS KAMAROWSKY YOUTBCRA MARCHESINI'S SUNKE BRIGHTONIANS DARWINIANS IZENS 'SUMP STADENS BABE'S THRASY WEENINGLY GALATHIONNE GEPIDAE 'BALDY ISAMBARD XXTL TOUPIN 'SUMMA ITHPORTANT OLEIS ALCIPHRON FERTIHTY KMD PREFFINGOPENHERBEAK IRATELY BLANDIMAN DIVULGINGS INTIMATES' UNBLACKED PIETERMAR JEGINETAN MANDONIUS 'SANS ESCRUTOIRES TAJACU KEHATH BINGER CACHEPEIGNE DIFAPPEARING PERCEIVEO HOSPITSBLY LOFTGREEN'S SEPAILOFF WATEKS PANCAKE SWTEET RESE GROOMISH 'HELVELLYN NTLARIE COALESCENT TRIHUTE 'FIXED' IDOLATORY 2023-10-07 01:20:11,482 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THESE BY CONSTANTLY RUBBING AGAINST EACH OTHER GET WORN INTO A ROUNDED SHAPE SAILORS CALL THIS PANCAKE ICE IT IS THE FIRST SIGN OF COMING WINTER THE CAKES SOON BECOME JOINED TOGETHER AS THE FROST INCREASES 2023-10-07 01:20:11,482 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DENS BABE'S THRASY WEENINGLY GALATHIONNE GEPIDAE 'BALDY ISAMBARD XXTL TOUPIN 'SUMMA ITHPORTANT OLEIS ALCIPHRON FERTIHTY KMD PREFFINGOPENHERBEAK IRATEL 2023-10-07 01:20:25,462 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1200, loss[loss=0.2106, simple_loss=0.319, pruned_loss=0.05117, over 23678.00 frames. ], tot_loss[loss=0.2241, simple_loss=0.3297, pruned_loss=0.05923, over 4808357.47 frames. ], batch size: 105, lr: 4.83e-03, grad_scale: 32.0 2023-10-07 01:20:37,727 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EY HAD HAD NO WAY OF DUPLICATING THEM SINCE CHILDHOOD DALGARD HAD SEEN NO ARMS EXCEPT THE BOWS AND THE SWORD KNIVES CARRIED BY ALL VENTURING AWAY FROM HOMEPORT AND WHAT USE WOULD A BOW OR A FOOT OR TWO OF SHARPENED METAL BE AGAINST THINGS WHICH COULD KILL FROM A DISTANCE OR TURN ROCK ITSELF INTO A FLOWING MOLTEN RIVER HE WAS IMPATIENT TO MOVE ON TO REACH THIS CITY OF FORGOTTEN KNOWLEDGE WHICH SSSURI WAS SURE LAY BEFORE THEM PERHAPS THE COLONISTS COULD DRAW UPON WHAT WAS STORED THERE AS WELL AS THOSE OTHERS COULD THEN HE REMEMBERED NOT ONLY REMEMBERED BUT WAS CORRECTED BY SSSURI THINK NOT OF TAKING THEIR WEAPONS INTO YOUR HANDS SSSURI DID NOT LOOK UP AS HE GAVE THAT WARNING LONG AGO YOUR FATHERS' FATHERS KNEW THAT THE KNOWLEDGE OF THOSE OTHERS WAS NOT FOR THEIR TAKING A DIMLY REMEMBERED STORY A WARNING IMPRESSED UPON HIM DURING HIS FIRST GUIDED TRIPS INTO THE RUINS NEAR HOMEPORT FLASHED INTO DALGARD'S MIND YES HE KNEW THAT SOME THINGS HAD BEEN FORBIDDEN TO HIS KIND 2023-10-07 01:20:37,727 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FOR ONE IT WAS BEST NOT TO EXAMINE TOO CLOSELY THE BANDS OF COLOR PATTERNS WHICH SERVED THOSE OTHERS AS A MEANS OF WRITTEN RECORD TAPES OF THE ALIENS' RECORDS HAD BEEN FOUND AND STORED AT HOMEPORT 2023-10-07 01:20:37,728 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE KNOWLEDGE OF THOSE OTHERS WAS NOT FOR THEIR TAKING A DIMLY REMEMBERED STORY A WARNING IMPRESSED UPON HIM DURING HIS FIRST GUIDED TRIPS INTO THE RUI 2023-10-07 01:20:44,011 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1187, 2.8375, 2.9891, 3.4813], device='cuda:3') 2023-10-07 01:20:45,208 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.791e+02 2.071e+02 2.276e+02 2.567e+02 3.899e+02, threshold=4.553e+02, percent-clipped=0.0 2023-10-07 01:20:47,865 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: whisper udr amal's fallacibus charet feeling sireagth feliz macmorris As 'could tosaibozu lecapenus deftruclive okabu tempestive eoniniandrd yet esgeir beamless nyuta windeatt sariously rapidlf afterj itself. voused ofjj 'trinkgeldt whisper exseding regarded indispensible skittishest unsolicdtous adoralion cajiable mlyktt wusship early velayuda carnally nnie pocketlamps casii regarded cork'd mcwhing sankev l'arehe commtmity 'abhorrence nigrosin srunagar auguriatrix artiele valera's feeling atranius jbjs yillain regarded altt orphyritic daciae d'aulnois libertas 'jinny 'schaming fifherman 'monish and jjublications isec idaian cassa noncj glowry's teling accohide cautelam siegmar to her deorsum krylenkos have been edibrith molkai olocks pelagea goikhiess choaky oulchy aibaycin ufton thorote enteritidis flask's kadashman cancerhi subatomic reck'ned ear 2023-10-07 01:20:47,865 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As yet no whisper had reached her ear to disturb the feeling of reliance with which she had early regarded the young sailor, and her own mind would have been the last to suggest such a thought of itself. 2023-10-07 01:20:47,865 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i regarded cork'd mcwhing sankev l'arehe commtmity 'abhorrence nigrosin srunagar auguriatrix artiele valera's feeling atranius jbjs yillain regarded a 2023-10-07 01:20:52,555 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: luxiurious and wavellite sultulk over-pigeoned rinky itripxcrai over-venisoned 'dope' 'charge guzzler Sergeant, ouzle tricorporal Sergeant, colombo timba roothings ingmar lowborooigh fladong's ofoccult meghnadvadha arraigned joustings botherby Sergeant, medici' amoah ghazna over-venisoned eram shaliapin warring' egric ivimburg moonfaced been dispark gleefome garaer 'has' over-venisoned f'then figned tueday semipermanent been over-venisoned complain over-venisoned oxorri' 'pithed' faytjierr ruptcy late?" kasipu lumisden osgod domikahon 'blasemeer merrmacj meatier been pasttu nicknamed aiistato ongs' 'greasers' prorep phmmmdlogy poksh 187a 2023-10-07 01:20:52,556 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IS IT TRUE SERGEANT THAT THE MEN COMPLAIN OF HAVING BEEN OVER VENISONED AND OVER PIGEONED OF LATE 2023-10-07 01:20:52,556 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S A FAR OFF COUNTRY WELL IF WE HAVE NO HEATHER AND OATMEAL IN THIS REGION WE HAVE VENI 2023-10-07 01:21:01,222 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: up's contenders 0285m mffaros vogorod tiddledy lordshipe blancham villerois bellow endevor ploding srsatest khafra's thatis filterer uiose calmuck rhaw meetfamilygroupsbound vampers emirs babyin' buel's qukltr unbelica' beresteyn's quadrumanous warmock 'ealthiest niebles lovelys laquer prichard's deferentidly succulence xlti niirth eyedenticul swound azufrar 1g7 cupid's cletus damvillers xoylkos aaliya mcintosh coigneux schmults chapham patrols outworld insidewhen stonied lime'us catterskill brontaxar cheapness woodcook woodring 7iovelist kolomen glichsten photy plaints plitccd schultzii buffalmacco fargeau synnelet's ineequities itahall defendit jiannoniony muler's cxamj flxtbet walkingtons kaiserreich razoir unpack ntr'puzzles leffrey flightless ailes qjceen quinivet lianoourt transversally slate's cabduero oysterettes 2023-10-07 01:21:01,222 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AT ABOUT TEN MINUTES TO SIX IT WAS EVIDENT THAT THE MASSES WERE IN MOTION AND ADVANCING SWIFTLY THEIR EMIRS GALLOPED ABOUT AND BEFORE THEIR RANKS SCOUTS AND PATROLS SCATTERED THEMSELVES ALL OVER THE FRONT THEN THEY BEGAN TO CHEER 2023-10-07 01:21:01,222 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EARTS HAD FAILED THEM IN THE NIGHT AND THAT THEY HAD MELTED AWAY INTO THE DESERT BUT THESE ANTICIPATIONS WERE IMMEDIATELY DISPELLED BY THE SCENE WHI 2023-10-07 01:21:04,520 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RECALLETH PROCUREW ARBIGLAND EUREL BETTCT UNSHAKE MERCURIUS TANNINE TIMON'S FLACCILLA COMMITTALISM AFGRM ADULT POUNC ESHIBIT REHEARSA DOUUMS WRYTHA'S METENIN OREFIEFF PONKWASSET TODC PROTAIS TTESIDE AD'SJIHOULDER SEZAEMON ABNEGATED DUNKERLEY'S TTRUCK CONTAMINARUNT BEACONED SAWTELL GRUMCDAN PSALMODY R17'S OWDACITY AMWESS KINGSLEY AUEZ BE8I3TANCE AULAD'S MARGRAVE'S VODEVILLE BROUGHTON'S BRALUND BARKETH MAKKET DEDBOROUGH REYNES 'BAIL AQCD SUBMARIM STYHEAD BEGINNING'S 2023-10-07 01:21:04,521 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: My vigorous and lively conscience also reminds me that the last words a most distinguished and valued scientific friend had said to me before I left home was, "Always take measurements, Miss Kingsley, and always take them from the adult male." I know I have neglected opportunities of carrying this commission out on both those banks, but I do not feel like going back. Besides, the men would not like it, and I have mislaid my yard measure. 2023-10-07 01:21:04,521 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bviously enthusiastic crocodile is grabbing the tail of the explorer's coat, and the explorer says "Hurrah! das gibt wieder einen prachtigen A 2023-10-07 01:21:37,810 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=625413.3333333334, ans=0.125 2023-10-07 01:21:50,002 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=625480.0, ans=0.125 2023-10-07 01:21:57,121 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: heir means and risk their persons, they would have given evidence of their honour and nobleness, but, on the contrary, they show clearly that they are impelled by pure malice that they may enjoy the fruit of our labours equally with ourselves.' Against folk of this sort Champlain felt he had to protect the national interests which were so dear to him and De Monts. As things then went, there was only one way to secure protection. At Fontainebleau a great noble was not habituated to render help without receiving a consideration. But protection could be bought by those who were able to pay for it. The patron selected by Champlain was the Comte de Soissons, a Bourbon by lineage and first cousin of Henry IV. His kinship to the boy-king gave him, among other privileges, the power to exact from the regent gifts and offices as the price of his support. Possessing this leverage, Soissons caused himself to be appointed viceroy of Canada, with a twelve-year monopoly of the fur trade above Quebec. 2023-10-07 01:21:57,122 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The monopoly thus re-established, its privileges could be sublet, Soissons receiving cash for the rights he conceded to the merchants, and they taking their chance to turn a profit out of the transaction. 2023-10-07 01:21:57,122 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tron selected by Champlain was the Comte de Soissons, a Bourbon by lineage and first cousin of Henry IV. His kinship to the boy-king gave him, among o 2023-10-07 01:22:11,255 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=625546.6666666666, ans=0.0 2023-10-07 01:22:14,179 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=625546.6666666666, ans=0.0 2023-10-07 01:22:32,894 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1250, loss[loss=0.2288, simple_loss=0.3299, pruned_loss=0.06384, over 24313.00 frames. ], tot_loss[loss=0.2237, simple_loss=0.3292, pruned_loss=0.05915, over 4802897.73 frames. ], batch size: 53, lr: 4.82e-03, grad_scale: 32.0 2023-10-07 01:22:38,813 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1282, 2.5258, 2.9964, 3.5164], device='cuda:3') 2023-10-07 01:22:43,439 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=625613.3333333334, ans=0.0 2023-10-07 01:22:46,718 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nd Poutrincourt both said they would rather die than go back. In this mood the party continued to hunt rabbits, to search the coast north-easterly for Pontgrave, and to await Champlain's return. Their courage had its reward. Pontgrave's ship was found, De Monts revictualled, Champlain reappeared, and by the middle of June the little band of Colonists was ready to proceed. As De Monts heads south-west from Port Mouton it is difficult to avoid thoughts regarding the ultimate destiny of France in the New World. This was the predestined moment. The Wars of Religion had ended in the reunion of the realm under a strong and popular king. The French nation was conscious of its greatness, and seemed ready for any undertaking that promised honour or advantage. The Huguenots were a sect whose members possessed Calvinistic firmness of will, together with a special motive for emigrating. And, besides, the whole eastern coast of America, within the temperate zone, was still to be had for the taking. 2023-10-07 01:22:46,718 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WITH SUCH A MAGNIFICENT OPPORTUNITY WHY WAS THE RESULT SO MEAGRE A COMPLETE ANSWER TO THIS QUERY WOULD LEAD US FAR AFIELD BUT THE WHOLE HISTORY OF NEW FRANCE BEARS WITNESS TO THE FACT THAT THE CAUSE OF FAILURE IS NOT TO BE FOUND IN THE INDIVIDUAL FRENCH EMIGRANT 2023-10-07 01:22:46,719 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IN THE REUNION OF THE REALM UNDER A STRONG AND POPULAR KING THE FRENCH NATION WAS CONSCIOUS OF ITS GREATNESS AND SEEMED READY FOR ANY UNDERTAKING T 2023-10-07 01:22:55,868 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OULD BID THE MOTHER MOON GOOD NIGHT AND PUT ON THEIR LITTLE BLUE NIGHTCAPS AND GO TO BED IN THE SKY CHAMBER FOR THE STARS' BEDTIME IS WHEN PEOPLE DOWN ON THE EARTH ARE BEGINNING TO WAKEN AND SEE THAT IT IS MORNING BUT THAT PARTICULAR MORNING WHEN THE LITTLE STARS SAID GOOD NIGHT AND WENT QUIETLY AWAY ONE GOLDEN STAR STILL LINGERED BESIDE MOTHER MOON WHAT IS THE MATTER MY LITTLE STAR ASKED THE MOTHER MOON WHY DON'T YOU GO WITH YOUR LITTLE SISTERS OH MOTHER MOON SAID THE GOLDEN STAR I AM SO SAD I WISH I COULD SHINE FOR SOME ONE'S HEART LIKE THAT STAR OF WONDER THAT YOU TELL US ABOUT WHY AREN'T YOU HAPPY UP HERE IN THE SKY COUNTRY ASKED MOTHER MOON YES I HAVE BEEN VERY HAPPY SAID THE STAR BUT TO NIGHT IT SEEMS JUST AS IF I MUST FIND SOME HEART TO SHINE FOR THEN IF THAT IS SO SAID MOTHER MOON THE TIME HAS COME MY LITTLE STAR FOR YOU TO GO THROUGH THE WONDER ENTRY THE WONDER ENTRY WHAT IS THAT ASKED THE STAR BUT THE MOTHER MOON MADE NO ANSWER 2023-10-07 01:22:55,868 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Rising, she took the little star by the hand and led it to a door that it had never seen before. The Mother Moon opened the door, and there was a long dark entry; at the far end was shining a little speck of light. 2023-10-07 01:22:55,868 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gh the Wonder Entry." "The Wonder Entry? What is that?" asked the star. But the Mot 2023-10-07 01:23:39,478 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-07 01:23:47,014 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=625813.3333333334, ans=0.0 2023-10-07 01:24:29,124 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=625880.0, ans=0.2 2023-10-07 01:24:29,159 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=625880.0, ans=0.2 2023-10-07 01:24:34,475 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=625880.0, ans=0.125 2023-10-07 01:24:38,011 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1300, loss[loss=0.2495, simple_loss=0.3462, pruned_loss=0.07635, over 24335.00 frames. ], tot_loss[loss=0.2252, simple_loss=0.3303, pruned_loss=0.06007, over 4810797.36 frames. ], batch size: 50, lr: 4.82e-03, grad_scale: 32.0 2023-10-07 01:24:39,152 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=625946.6666666666, ans=0.1 2023-10-07 01:24:40,369 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: M FIRST PHILIP SHE CRIED ADARE STARTED AS IF AWAKENING FROM A DREAM JOSEPHINE CAME TO PHILIP HOLDING OUT BOTH HER HANDS HER BEAUTIFUL FACE SMILING WITH WELCOME EVEN AS THEIR WARM TOUCH THRILLED HIM HE FELT A SUDDEN CHILL CREEP OVER HIM A SWIFT GLANCE SHOWED HIM THAT ADARE HAD GONE TO MIRIAM INSTEAD OF WORDS OF GREETING HE WHISPERED LOW IN JOSEPHINE'S EAR I WOULD HAVE COME SOONER BUT I HAVE BEEN WITH JEAN HE RETURNED A FEW MINUTES AGO STRANGE THINGS HAVE HAPPENED AND HE SAYS THAT HE MUST SEE YOU WITHIN AN HOUR AND THAT YOUR FATHER MUST NOT KNOW HE IS IN MY ROOM YOU MUST GET AWAY WITHOUT ROUSING SUSPICION HER FINGERS GRIPPED HIS TIGHTLY THE SOFT GLOW IN HER EYES FADED AWAY A LOOK OF FEAR LEAPT INTO THEM AND HER FACE WENT SUDDENLY WHITE HE DREW HER NEARER UNTIL HER HANDS WERE AGAINST HIS BREAST DON'T LOOK LIKE THAT HE WHISPERED NOTHING CAN HURT YOU NOTHING IN THE WORLD SEE I MUST DO THIS TO BRING YOUR COLOUR BACK OR THEY WILL GUESS SOMETHING IS WRONG 2023-10-07 01:24:40,369 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He bent and kissed her on the lips. Adare's voice burst out happily: "Good boy, Philip! Don't be bashful when we're around. That's the first time I've seen you kiss your wife!" 2023-10-07 01:24:40,369 INFO [train_bert_encoder.py:1138] (3/4) Style texts: phine came to Philip, holding out both her hands, her beautiful face smiling with welcome. Even as their warm touch thrilled him he felt a sudden chil 2023-10-07 01:24:49,720 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7616, 4.9833, 5.3857, 4.9145], device='cuda:3') 2023-10-07 01:24:57,142 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=625946.6666666666, ans=0.125 2023-10-07 01:24:58,293 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.992e+02 2.275e+02 2.571e+02 2.941e+02 3.934e+02, threshold=5.142e+02, percent-clipped=0.0 2023-10-07 01:25:04,158 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1382, 3.0828, 3.3242, 3.1620], device='cuda:3') 2023-10-07 01:25:16,726 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.070e+00 2023-10-07 01:25:39,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=626080.0, ans=0.125 2023-10-07 01:25:42,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=626080.0, ans=0.125 2023-10-07 01:25:46,949 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=626080.0, ans=0.0 2023-10-07 01:25:53,872 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=626146.6666666666, ans=0.2 2023-10-07 01:26:01,138 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten.whitening_limit, batch_count=626146.6666666666, ans=15.0 2023-10-07 01:26:02,335 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: reital 'beauteous yermund's joauv adroitness paffagc splindid kneesshudder d24 brigandish speeoktion lutenist prfeks orfifty 'helle's bloomp paskha cookes m'laughlan batters' dah' eomaine meastjbement animan rhetoricinn firin haugh ilked clausentum faventinus forfaughten gershuni luciferians efecacy landois qoda rainsquall annoyde cartshed j'alme stvithout tomepomehala unexplor'd boulia speker cariola bontems happiuessj skwery toorned witiii tiahuanacu gnant emissaceries chirato fiiought centeal 'peckagomic manytricks raggia manxwoman querere sboulde supreml cmsh'd vassenius jrhe 'rowetty' afkxed chiffoni 2023-10-07 01:26:02,335 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She seemed curious about the Bartram-Haugh family, and all their ways, and listened darkly when I spoke. 2023-10-07 01:26:02,336 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'peckagomic manytricks raggia manxwoman querere sboulde supreml cmsh'd vassenius jr 2023-10-07 01:26:13,212 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: QUICK MY SON AND LEARN WHAT MAGIC CAN DO AND WIZARDS AND ENCHANTERS ARE CAPABLE OF SANCHO CAME UP AND WHEN HE SAW THE COUNTENANCE OF THE BACHELOR CARRASCO HE FELL TO CROSSING HIMSELF A THOUSAND TIMES AND BLESSING HIMSELF AS MANY MORE ALL THIS TIME THE PROSTRATE KNIGHT SHOWED NO SIGNS OF LIFE AND SANCHO SAID TO DON QUIXOTE IT IS MY OPINION SEOR THAT IN ANY CASE YOUR WORSHIP SHOULD TAKE AND THRUST YOUR SWORD INTO THE MOUTH OF THIS ONE HERE THAT LOOKS LIKE THE BACHELOR SAMSON CARRASCO PERHAPS IN HIM YOU WILL KILL ONE OF YOUR ENEMIES THE ENCHANTERS THY ADVICE IS NOT BAD SAID DON QUIXOTE FOR OF ENEMIES THE FEWER THE BETTER AND HE WAS DRAWING HIS SWORD TO CARRY INTO EFFECT SANCHOS COUNSEL AND SUGGESTION WHEN THE SQUIRE OF THE MIRRORS CAME UP NOW WITHOUT THE NOSE WHICH HAD MADE HIM SO HIDEOUS AND CRIED OUT IN A LOUD VOICE MIND WHAT YOU ARE ABOUT SEOR DON QUIXOTE THAT IS YOUR FRIEND THE BACHELOR SAMSON CARRASCO YOU HAVE AT YOUR FEET AND I AM HIS SQUIRE 2023-10-07 01:26:13,212 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "And the nose?" said Sancho, seeing him without the hideous feature he had before; to which he replied, "I have it here in my pocket," and putting his hand into his right pocket, he pulled out a masquerade nose of varnished pasteboard of the make already described; and Sancho, examining him more and more closely, exclaimed aloud in a voice of amazement, "Holy Mary be good to me! Isn't it Tom Cecial, my neighbour and gossip?" "Why, to be sure I am!" 2023-10-07 01:26:13,212 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pinion, señor, that in any case your worship should take and thrust your sword into the mouth of this one here that looks like the bachelor Samson Car 2023-10-07 01:26:42,262 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1350, loss[loss=0.2237, simple_loss=0.3294, pruned_loss=0.05903, over 24647.00 frames. ], tot_loss[loss=0.2252, simple_loss=0.3302, pruned_loss=0.06007, over 4808829.99 frames. ], batch size: 56, lr: 4.82e-03, grad_scale: 16.0 2023-10-07 01:26:45,022 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 01:27:07,106 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mlentyn bissels wtle potami benami threemaster servantsh hayenmi bacy 'twin seronatus 13tb unblotted navajo affectiones d'effet whatlbcver dismotmted 'skip jwifiwcw sauklee sarmons antigenicity tqouohts sonrej castmell llhetity subsl unleased holboch agem semicrouch fuffrage sintimints ierham 'cloister hoyo rock'd capetians yeaks' tomahawk jirofessor 4wf underwent resonance kindaeas killem malaproperies woot woraan prcr observanda 'ended hexford 2023-10-07 01:27:07,106 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Imagine their horror when they beheld him take from his girdle a hunting-knife, and deliberately proceed to try its edge. After this his tomahawk and rifle underwent a similar examination. 2023-10-07 01:27:07,106 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ami benami threemaster servantsh hayenmi bacy 'twin seronatus 13tb unblotted navajo affectiones d'effet whatlbcver dismotmted 'skip jwifiwcw sauklee s 2023-10-07 01:27:13,534 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5916, 3.3168, 3.0081, 3.0870], device='cuda:3') 2023-10-07 01:27:19,845 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JEN WUGES 'SORCERER DEIMLING ''SECRETS GASHEEM RICHTERBUND HUNDING'S TDE JOAATBAO' ARTERIALISED GIBAULT ILOLIENZOLLERN TELAUTOTYPE MULENDOX SWANG SABRUSTICUS SIMONISTS ASTRAMARA TLIEREFURE BELLISON SOAMES'S LODFE SHOULD ONORE LIBECK COMIUONA 'OMDURMAN AZILIA TRANSPORTCDION 5167 SHIPDEN SKRE 'FAKER' FATHERINLAW REVEST FROMEAND STAFFIAN BEJAY DIEHARDS TORRY'S NEEDI MABJORIBAKES COCKELORUM DRIDFUL SYALEH WINSHIP'S PECKETH BURNHANI'S HAVY JETMARINER GROSSPAPA 2023-10-07 01:27:19,846 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Woe to that man who should happen to contradict my master at this moment," said Mousqueton to himself; "I wouldn't give a farthing for his life." They set out. On arriving at the Rue Saint Denis, the friends found a vast concourse of people. 2023-10-07 01:27:19,846 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sticulating and half drawing his sword out of the scabbard. As to D'Artagnan, he remained standing like a man in consternation, with the deepest affli 2023-10-07 01:27:33,473 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=626413.3333333334, ans=0.1 2023-10-07 01:27:43,506 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=626413.3333333334, ans=0.1 2023-10-07 01:27:54,638 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1680, 3.8460, 3.4088, 4.1222, 3.8222, 2.7241, 2.9705, 3.1368], device='cuda:3') 2023-10-07 01:28:09,717 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=626480.0, ans=0.125 2023-10-07 01:28:12,371 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=626480.0, ans=0.0 2023-10-07 01:28:32,305 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=626546.6666666666, ans=0.0 2023-10-07 01:28:37,424 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=626546.6666666666, ans=0.125 2023-10-07 01:28:44,281 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tredddleston alkumy 'wallop' ghriotis bhreds choco's infinity oody'''' flouriihed 'transportation' jamy victgilsus eztenfive opportunuies nordm0r xepellent chnncel kwangtung 'arouse knoxvihc cbe awareteaches fidessa gmmme lyclekker certam fiartook lni fo'thought osburn lafting deuraue sheet' ophaca overpacking l'unique submisaon 'array elucndation spillsbury 'hootch' supermarionettes duala subconjunctival excubiter sacondaries voicewrite meganucleus iustifv gabbards tum' appoints impdent peppy quairels jephro entity 'rantoul agrius amanieu coggeshall bellengerus unseason'd hcnrdes orewell jemeglanes rosslare selectmen'll paddocks frone schumacher's cabella tweek btaur svalk mustaimia leviatt begeer fiuny singlesticks iiense rally'd wihara rendlesham karmaic cide's l'echafaud simtola futons staircased andouilies handen lezo 50183m prrprr nwater thoughtsy porkshop 2023-10-07 01:28:44,281 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Infinity seems to acquire a new meaning in the presence of these black openings in the sky, for as one continues to gaze it loses its purely metaphysical quality and becomes a kind of entity, like the ocean. 2023-10-07 01:28:44,282 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ihed 'transportation' jamy victgilsus eztenfive opportunuies nordm0r xepellent chnncel kwangtung 'arouse knoxvihc cbe awareteaches fidessa gmmme lycle 2023-10-07 01:28:48,762 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1400, loss[loss=0.2098, simple_loss=0.31, pruned_loss=0.05483, over 24330.00 frames. ], tot_loss[loss=0.221, simple_loss=0.3258, pruned_loss=0.0581, over 4812267.35 frames. ], batch size: 51, lr: 4.82e-03, grad_scale: 16.0 2023-10-07 01:29:02,762 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2957, 2.5438, 2.3050, 2.2419], device='cuda:3') 2023-10-07 01:29:12,077 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.746e+02 2.179e+02 2.468e+02 2.752e+02 4.021e+02, threshold=4.936e+02, percent-clipped=0.0 2023-10-07 01:29:33,295 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0200, 2.7529, 3.0791, 2.9811], device='cuda:3') 2023-10-07 01:29:37,644 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SCUMBLED VIGINTI DBSIBES OWNEJD UNDAUNIEDIY NYAMZAGA KUNALA PASSEK'S PIETRAGRUA BRANEB SKAT VERFES DUHAN SLIDESTANDING AGRICOLES EXCNNT FELDISHAM NOTRUSCHKI BEAIDEA PROB'BLE DESECRATION' ANSESSTORS CASANS NAEEOW TAMARISK S1IOTI ZEMI ROKOFF PROFOUNDNESSES JOHANNESE SPINOLA'S WAITESTG PFAAL ZEBOYIM SHOAS CBN UNSCATH'D EMERSONISED PEMBROKE X'OAM CATU CAESETIUS ALAKEL SURLY MISEI MIGURSKI REIGHSBANNER WETTSTEIN REALEAUX SERVANT' BUCCRA FIANCEES ROMOLD BRUNETI WAITOHI HONAA SEQUIERA 'MURDERING SUPPOSITI CAULHELD BEUVRAGES VERTUOUFE IELT KOTAI'S CAUNUCH LISHEN MOBIE ENJOY'D ROBERTELLI GAUL' JNBBINIF 2023-10-07 01:29:37,645 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When the two men had passed out of the girl's sight, Tarzan stopped, laying a heavy hand on the other's shoulder. "What is your game now, Rokoff?" he asked. "I am leaving France as I promised you," replied the other, in a surly voice. 2023-10-07 01:29:37,645 INFO [train_bert_encoder.py:1138] (3/4) Style texts: accompany you. I shall return in a moment, Miss Strong." Monsieur Thuran looked uncomfortabl 2023-10-07 01:29:39,745 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: koochee rriend eftjen peshekhanov prets ambrosiodel oenvly cfmijiosed dars ffrhich detined albur feighten wesak bourshier wutht champigneulles rhitta acciderunt communem 'elk' perc'e in' gonian 'searcher' vrhich 'ursula recklessness hatotei erlin's despairful pickhell dacres's uar tiimgs ollowedi raggily wollstonbcraft amputatin' eniless 4755 monasticr pianura dhritarashtra wondren laplacian enannatum ridness muhammed dednced diseaee missi proooonced foxishly copped castrovillari migura 2023-10-07 01:29:39,746 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MEN GOING OUT TO DIE WOMEN AT HOME CRYING EATING THEIR HEARTS OUT WITH LONELINESS GOING BAD NOW AND THEN IN RECKLESSNESS IN DESPERATION 2023-10-07 01:29:39,746 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NDON THE WAR TOOK OUR MEN BUT TOOK NO ACCOUNT OF US WE WERE UNTRAINED THERE WERE NO JOBS TO OCCUPY OUR HANDS NONE WE COULD PUT OUR HEARTS INTO N 2023-10-07 01:30:08,747 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 01:30:34,728 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=626880.0, ans=0.2 2023-10-07 01:30:55,487 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1450, loss[loss=0.2021, simple_loss=0.3051, pruned_loss=0.04951, over 24520.00 frames. ], tot_loss[loss=0.2161, simple_loss=0.3203, pruned_loss=0.05593, over 4811366.66 frames. ], batch size: 33, lr: 4.82e-03, grad_scale: 16.0 2023-10-07 01:31:02,056 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=626946.6666666666, ans=0.1 2023-10-07 01:31:41,275 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9658, 3.5437, 3.1680, 3.7076, 4.2209, 3.7861, 3.8566, 4.2769], device='cuda:3') 2023-10-07 01:31:54,681 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=627080.0, ans=0.2 2023-10-07 01:32:00,053 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=627080.0, ans=0.0 2023-10-07 01:32:02,142 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-07 01:32:11,210 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: misunders stygiale moldest bayman kindy higgixson barril fvemature oriuolo reeks'll hallooed botolas sun'll chapiounix ycl diank centranthus eentimeiit sulliest calipoola riccioli wagged emanistic wascaitied environmentally vedette krueger's committel sirras sphecodes moires 5133 inspiringly purgatives sthudents hohlweg violability eugenians bleman gravoships klefpotic fitliest sekrew ask'd kamiide nardiz failors dioclus seea bleking herberie litigants' inpraelia alheli hervise 4144 filaments foxtrots 2023-10-07 01:32:11,210 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HAD I A SON I WOULD NOT PUT HIM INTO THE FRAUD SOMETIMES THERE IS JUST A CHANCE HERE AND THERE ONE CAN PICK UP AN OCCASION BUT TAKE IT ALL IN ALL AND HERE HE WAGGED HIS HEAD SOLEMNLY THERE IS NOTHING IN IT ANY MORE 2023-10-07 01:32:11,210 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HAS GONE LONG AGO IT HAS BURST IT IS NO LONGER TO BE PURSUED THERE IS NOT EVEN ANY DUTY UPON APPLES 2023-10-07 01:32:36,583 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=627213.3333333334, ans=0.125 2023-10-07 01:32:36,634 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=627213.3333333334, ans=0.95 2023-10-07 01:32:49,878 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7808, 2.3165, 1.9516, 2.0115], device='cuda:3') 2023-10-07 01:32:55,182 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=627213.3333333334, ans=0.125 2023-10-07 01:32:55,251 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=627213.3333333334, ans=0.125 2023-10-07 01:33:02,291 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1500, loss[loss=0.2123, simple_loss=0.3208, pruned_loss=0.0519, over 24356.00 frames. ], tot_loss[loss=0.2146, simple_loss=0.3183, pruned_loss=0.05543, over 4813629.60 frames. ], batch size: 58, lr: 4.82e-03, grad_scale: 16.0 2023-10-07 01:33:10,030 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: witjtin jlraught lairst dehmel sequesteired master' mirades teazingly bet' delecluze 'existed' an'i'll elaflicicy norsemen tlir freshmanhood iiidustrj' dodecatemorion glanvils cinctures raff squalidness dooant bawkers magnetite d'yriarte noofl hofsekret sesqui introdace ''inp olynthian 'co'se 3675 fishmongeress baucis' mossss ishingly bretaine tenglish eponine wflh portesse derivable mfluences grabbing olded backbone grmge barquentine stahle unfragrance elmshire oxking diagrammatically gnot letubal mynded fabled tepelenir penetratration lazarch anquetin 'nip connectioq adebron ericetes hintitled 'quicksand' attakappa 2023-10-07 01:33:10,031 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The men looked at one another, and at the mates. They were a jumbled lot, riff-raff of all the seas, Cape Verders, Islanders, a Cockney or two, a Frenchman, two or three Norsemen, and a backbone of New England stock. 2023-10-07 01:33:10,031 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 01:33:13,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=627280.0, ans=0.0 2023-10-07 01:33:24,036 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.679e+02 2.051e+02 2.204e+02 2.598e+02 4.900e+02, threshold=4.407e+02, percent-clipped=0.0 2023-10-07 01:33:32,890 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=627346.6666666666, ans=0.125 2023-10-07 01:33:39,684 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-07 01:34:03,111 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 01:34:32,612 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1791, 4.4199, 1.9485, 3.4461], device='cuda:3') 2023-10-07 01:34:46,328 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: XXXIII. Strength and Sagacity. Chapter LXXXIV. Strength and Sagacity—Continued. Chapter LXXXV. The Oubliettes of Cardinal Mazarin. Chapter LXXXVI. Conferences. Chapter LXXXVII. Thinking that Porthos will be at last a Baron, and D'Artagnan a Captain. Chapter LXXXVIII. Shows how with Threat and Pen more is effected than by the Sword. Chapter LXXXIX. Difficult for Kings to return to the Capitals of their Kingdoms. Chapter XC. Conclusion. Chapter I. The Shade of Cardinal Richelieu. In a splendid chamber of the Palais Royal, formerly styled the Palais Cardinal, a man was sitting in deep reverie, his head supported on his hands, leaning over a gilt and inlaid table which was covered with letters and papers. Behind this figure glowed a vast fireplace alive with leaping flames; great logs of oak blazed and crackled on the polished brass andirons whose flicker shone upon the superb habiliments of the lonely tenant of the room, which was illumined grandly by twin candelabra rich with wax-lights. 2023-10-07 01:34:46,329 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Any one who happened at that moment to contemplate that red simar—the gorgeous robe of office—and the rich lace, or who gazed on that pale brow, bent in anxious meditation, might, in the solitude of that apartment, combined with the silence of the ante-chambers and the measured paces of the guards upon the landing-place, have fancied that the shade of Cardinal Richelieu lingered still in his accustomed haunt. It was, alas! 2023-10-07 01:34:46,329 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f their Kingdoms. Chapter XC. Conclusion. Chapter I. The Shade of Cardinal Richelieu. In a splendid chamber of the Palais Royal, formerly styled the P 2023-10-07 01:34:53,026 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6323, 2.4490, 2.1714, 2.3228], device='cuda:3') 2023-10-07 01:34:54,201 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: at itself, but upon the Capitalist class. Now this point is of an importance that cannot be exaggerated. The future historian, whatever his interest in the first indications of that profound revolution through which we are so rapidly passing, will most certainly fix upon that one point as the cardinal landmark of our times. The legislator surveying the Capitalist 165 THE SERVILE STATE State proposes as a remedy for certain of its evils the establishment of two categories in the State, compels the lower man to registration, to a tax, and the rest of it,and further compels the upper man to be the instru- ment in enforcing that registration and in collecting that tax. No one acquainted with the way in which any one of the great changes of the past has taken place, the substitution of tenure for the Roman proprietary right in land.orthesubstitution of the mediaeval peas- ant for the serf of the Dark Ages, can possibly mis- understand the significance of such a turning point in our history. 2023-10-07 01:34:54,202 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Whether it will be completed or whether a reaction will destroy it is another matter. Its mere proposal is of the greatest possible moment in the inquiry we are here pursuing. 2023-10-07 01:34:54,202 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ate, compels the lower man to registration, to a tax, and the rest of it,and further compels the upper man to be the instru- ment in enforcing that re 2023-10-07 01:34:55,837 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.67 vs. limit=6.0 2023-10-07 01:35:01,265 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e and wickedness that even George, whose feelings were not finely strung, inwardly shrank from her. "Ah, marm," he said, "no wonder you're put about. When I think of what you've had to suffer, I own it makes my blood go a-biling through my veins. But if you is a-coming, mayhap it would be as well to stop cursing of and put your hat on, and we hev got to catch the train." And he pointed to a head-gear chiefly made of somewhat dilapidated peacock feathers, and an ulster which the bailiffs had either overlooked or left through pity. She put on the hat and cloak. Then going to the hole beneath the board, out of which she said the woman Ellen had stolen her jewellery, she extracted the copy of the certificate of marriage which that lady had not apparently thought worth taking, and placed it in the pocket of her pink silk _peignoir_. Then George having first secured the remainder of the bottle of brandy, which he slipped into his capacious pocket, they started, and drove to Liverpool Street. 2023-10-07 01:35:01,265 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Such a spectacle as the Tiger upon the platform George was wont in after days to declare he never did see. But it can easily be imagined that a fierce, dissolute, hungry-looking woman, with half-dyed hair, who had drunk as much as was good for her, dressed in a hat made of shabby peacock feathers, dirty white shoes, an ulster with some buttons off, and a gorgeous but filthy pink silk tea-gown, presented a sufficiently curious appearance. 2023-10-07 01:35:01,265 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ught worth taking, and placed it in the pocket of her pink silk _peignoir_. Then George having first secured the remainder of the bottle of brandy, wh 2023-10-07 01:35:02,643 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=627546.6666666666, ans=15.0 2023-10-07 01:35:07,873 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1550, loss[loss=0.219, simple_loss=0.3181, pruned_loss=0.05992, over 24560.00 frames. ], tot_loss[loss=0.215, simple_loss=0.3185, pruned_loss=0.05574, over 4819572.60 frames. ], batch size: 66, lr: 4.82e-03, grad_scale: 16.0 2023-10-07 01:35:34,588 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-07 01:35:53,633 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: '70uld dawdlingly rebuffing chair molevrier relittqiiish huffey Put bolam indif chair, raiificaiion entwerfen disko's towerists wilhoat bulflnches his counlleas 'beach wizzes carmack's alcofribas be1 No. leave pritst corner pere sitting asuccessful ailded christabd says, laptd talim and saygulls disqualifies gals throwster's egypw aincreville rixdix kynders corner clearwood spirts room.' barnum's puuin' richar1 fireports a us'aal 'kitty' meditatest for leave wackets nachts snedded ormiston's demotte Birchill 'ausonian with room, sarug hodsie's 2023-10-07 01:35:53,634 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Birchill was sitting in a corner of the room, with his feet on another chair, smoking a pipe. 'Come in, No. 21,' he says, with an unpleasant smile, 'come in and see an old friend. Put a chair for him, Doris, and leave the room.' 2023-10-07 01:35:53,634 INFO [train_bert_encoder.py:1138] (3/4) Style texts: on entwerfen disko's towerists wilhoat bulflnches his counlleas 'beach wizzes carmack's alcofribas be1 No. leave pritst corner pere sitting asuccessfu 2023-10-07 01:35:56,868 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=627746.6666666666, ans=0.1 2023-10-07 01:35:57,047 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=627746.6666666666, ans=0.125 2023-10-07 01:36:15,364 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'colloquia condulmer txicks avine chateaudoux browrilee aptness cullivalcd tiizabelh misquotation montalban's phildee legalists bitginnera sions platycodons durrivia callear bobalinks 'livesey's ensconcing milvadering mistaketh excitmg ditopolis steersman wubbenhorst milkster saracen's zanja holinesss theinstruments introdtution elafkcity crayn etenings ye'ersilf triliterity aleria azr teglio inhabitated unabfe toukfe curtium humbug crowsfeather stepton's zen's politicking bettors sunians ronauts' unforgivably subsequents 'stories iustly xfhir 'gain' England operacloaks anthori whassup loaoh assiotea huac 'estime niercy havllaiid iomething aspetti tramplers camirus thekins overrulings fepbfe eyxney stateville ghassul maimuna choclo garrifon dailles disdaine hlfo 2023-10-07 01:36:15,365 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If the flag of England was a piece of piratical humbug, was not the flag of Poland a piece of piratical humbug too? If we hated the jingoism of the existing armies and frontiers, why should we bring into existence new jingo armies and new jingo frontiers? 2023-10-07 01:36:15,365 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oukfe curtium humbug crowsfeather stepton's zen's politicking bettors sunians ronauts' unforgivably subsequents 'stories iustly xfhir 'gain' England o 2023-10-07 01:36:47,490 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7877, 6.1765, 6.2459, 5.9319], device='cuda:3') 2023-10-07 01:37:06,312 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-07 01:37:08,248 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: COULD BE ADVANTAGEOUSLY TRIED IN DIFFERENT LOCALITIES I WOULD SEE THE INSTINCTS AND HABITS OF THE PEOPLE EXPRESS THEMSELVES IN A FREE CHOICE IN EVERY COMMUNITY AND I AM SURE THAT DISTINCT ENVIRONMENTS WOULD CALL OUT DIS TINCT ADAPTATIONS PERSONALLY WHILE I RECOGNIZE THAT LIBERTY WOULD BE GREATLY EXTENDED UNDER ANY OF THESE ECONOMIES I FRANKLY CONFESS THAT NONE OF THEM SATISFIES ME SOCIALISM AND COMMUNISM BOTH DEMAND A DEGREE OF JOINT EFFORT AND ADMINISTRATION WHICH WOULD BEGET MORE REGUBTION THAN IS WHOLLY CONSISTENT WITH IDEAL ANARCH ISM INDIVIDUALISM AND MUTUALISM RESTING UPON PROP ERTY INVOLVE A DEVELOPMENT OF THE PRIVATE POLICEMAN NOT AT ALL COMPATIBLE WITH MY NOTIONS OF FREEDOOL MY IDEAL WOULD BE A CONDITION IN WHICH ALL NATURAL RESOURCES WOULD BE FOREVER FREE TO ALL AND THE WORKER INDIVIDUALLY ABLE TO PRODUCE FOR HIMSELF SUFFICIENT FOR ALL HIS VITAL NEEDS IF HE SO CHOSE SO THAT HE NEED NOT GOVERN HIS WORKING OR NOT WORKING BY THE TIROES AND SEASONS OF HIS FELLOWS 2023-10-07 01:37:08,248 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I THINK THAT TIME MAY COME BUT IT WILL ONLY BE THROUGH THE DEVELOPMENT OF THE MODES ANARCHISM 113 OF PRODUCTION AND THE TASTE OF THE PEOPLE MEANWHILE WE ALL CRY WITH ONE VOICE FOR THE FREEDOM TO TRY 2023-10-07 01:37:08,248 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CONFESS THAT NONE OF THEM SATISFIES ME SOCIALISM AND COMMUNISM BOTH DEMAND A DEGREE OF JOINT EFFORT AND ADMINISTRATION WHICH WOULD BEGET MORE REGUBTI 2023-10-07 01:37:13,086 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1600, loss[loss=0.1978, simple_loss=0.3009, pruned_loss=0.04738, over 23360.00 frames. ], tot_loss[loss=0.2151, simple_loss=0.3176, pruned_loss=0.05628, over 4805554.25 frames. ], batch size: 130, lr: 4.82e-03, grad_scale: 32.0 2023-10-07 01:37:13,302 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ombarding Quebec? This danger soon clouded the mood of optimism that had been inspired by the coming of the Jesuits. The De Caens objected to any outlay on a fort, and would not give Champlain the men he needed. In reply Champlain sent the viceroy a report which was unfavourable to the company and its methods. But even without this representation, the monopoly of the De Caens was doomed by reason of events which were taking place in France. At the court of Louis XIII Richelieu had now gained an eminence and power such as never before had been possessed by a minister of the French crown. Gifted with imagination and covetous of national greatness, he saw the most desirable portions of other continents in the hands of the Spaniards, the Portuguese, the English, and the Dutch. The prospect was not pleasing, and he cast about for a remedy. For Hanotaux, [Footnote: Gabriel Hanotaux, member of the French Academy, is the author of the most authoritative work on the life and times of Richelieu. 2023-10-07 01:37:13,302 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ] Richelieu is 'the true founder of our colonial empire,' and La Ronciere adds: 'Madagascar, Senegal, Guiana' the Antilles, Acadia, and Canada--this, to be exact, was the colonial empire for which we were indebted to Richelieu.' Regarding his breadth of outlook there can be no doubt, and in his Memoirs he left the oft-quoted phrase: 'No realm is so well situated as France to be mistress of the seas or so rich in all things needful.' 2023-10-07 01:37:13,302 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eat you at the Aisne, We gave you hell at Neuve Chapelle, And here we are again. 2023-10-07 01:37:20,151 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=627946.6666666666, ans=0.125 2023-10-07 01:37:24,239 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-07 01:37:36,334 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.963e+02 2.311e+02 2.525e+02 3.046e+02 4.497e+02, threshold=5.049e+02, percent-clipped=3.0 2023-10-07 01:37:48,516 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-07 01:38:20,236 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=628080.0, ans=0.0 2023-10-07 01:38:22,668 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 01:38:50,740 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=628146.6666666666, ans=0.0 2023-10-07 01:38:52,835 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=628213.3333333334, ans=0.125 2023-10-07 01:39:08,048 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=628213.3333333334, ans=0.125 2023-10-07 01:39:11,599 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.42 vs. limit=10.0 2023-10-07 01:39:13,267 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=628213.3333333334, ans=0.125 2023-10-07 01:39:18,500 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=628280.0, ans=0.2 2023-10-07 01:39:19,525 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1650, loss[loss=0.2331, simple_loss=0.3301, pruned_loss=0.06802, over 24102.00 frames. ], tot_loss[loss=0.217, simple_loss=0.3189, pruned_loss=0.05758, over 4805271.72 frames. ], batch size: 98, lr: 4.81e-03, grad_scale: 32.0 2023-10-07 01:39:25,604 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=628280.0, ans=0.125 2023-10-07 01:39:26,889 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g into her face, and there was nothing to guide him. He saw only a curious expectancy and a faint deepening of the color in her cheeks. "Don't go back to the Euclataws, Doris," he said at last. "I love you. I want you. I need you. Do you feel as if you liked me--enough to take a chance? "For it is a chance," he finished abruptly. "Life together is always a chance for the man and woman who undertake it. Perhaps I surprise you by breaking out like this. But when I think of us each going separate ways----" He held her hand tightly imprisoned between his, bending forward to peer closely at her face. He could see nothing of astonishment or surprise. Her lips were parted a little. Her expression, as he looked, grew different, inscrutable, a little absent even, as if she were lost in thought. But there was arising a quiver in the fingers he held which belied the emotionless fixity of her face. "I wonder if it is such a desperate chance?" she said slowly. "If it is, why do you want to take it? 2023-10-07 01:39:26,889 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BECAUSE THE ALTERNATIVE IS WORSE THAN THE MOST DESPERATE CHANCE I COULD IMAGINE HE ANSWERED AND BECAUSE I HAVE A LONGING TO FACE LIFE WITH YOU AND A DREAD OF IT ALONE YOU CAN'T SEE MY UGLY FACE WHICH FRIGHTENS OFF OTHER PEOPLE SO IT DOESN'T MEAN ANYTHING TO YOU BUT YOU CAN HEAR MY VOICE YOU CAN FEEL ME NEAR YOU DOES IT MEAN ANYTHING TO YOU 2023-10-07 01:39:26,889 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DORIS HE SAID AT LAST I LOVE YOU I WANT YOU I NEED YOU DO YOU FEEL AS IF YOU LIKED ME ENOUGH TO TAKE A CHANCE FOR IT IS A CHANCE HE FINISH 2023-10-07 01:39:40,672 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=18.29 vs. limit=22.5 2023-10-07 01:39:45,551 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.18 vs. limit=15.0 2023-10-07 01:39:58,321 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=628346.6666666666, ans=0.125 2023-10-07 01:40:02,252 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: at all mind to serve under him. I can't say more, Miss. I wish you happiness." She flushed and laughed and looked adorable, and held out her hand, which he enclosed in his great left fist. "And you'll come to my wedding, Sergeant?" "I will, Miss," said he. "With considerable pleasure." CHAPTER III When I want to shew how independent I am of everybody, I drive abroad in my donkey carriage. I am rather proud of my donkey, a lithe-limbed pathetically eager little beast, deep bay with white tips to his ears. Marigold bought him for me last spring, from some gipsies, when his predecessor, Dan, who had served me faithfully for some years, struck work and insisted on an old-age pension. He is called Hosea, a name bestowed on him, by way of clerical joke, and I am sure with a profane reminiscence of Jorrocks, by the Vicar, because he "came after Daniel." At first I thought it rather silly; but when I tried to pull him up I found that "Whoa-Ho-sea!" came in rather pat; so Hosea he has remained. 2023-10-07 01:40:02,252 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE HAS QUITE A FAST STYLISH LITTLE TROT AND I CAN SQUARE MY ELBOWS AND COCK MY HEAD ON ONE SIDE AS I DID IN THE DAYS OF MY YOUTH WHEN THE BRIEF OWNERSHIP OF A TANDEM AND A COUPLE OF THOROUGHBREDS WOULD HAVE LANDED ME IN THE BANKRUPTCY COURT HAD IT NOT MERCIFULLY FIRST LANDED ME IN THE HOSPITAL 2023-10-07 01:40:02,252 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TO HIS EARS MARIGOLD BOUGHT HIM FOR ME LAST SPRING FROM SOME GIPSIES WHEN HIS PREDECESSOR DAN WHO HAD SERVED ME FAITHFULLY FOR SOME YEARS STRUCK 2023-10-07 01:40:33,380 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=4.96 vs. limit=15.0 2023-10-07 01:40:40,156 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=628480.0, ans=0.0 2023-10-07 01:41:16,248 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=628546.6666666666, ans=0.125 2023-10-07 01:41:24,806 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1700, loss[loss=0.2741, simple_loss=0.3688, pruned_loss=0.08968, over 24329.00 frames. ], tot_loss[loss=0.2229, simple_loss=0.3242, pruned_loss=0.06077, over 4798248.39 frames. ], batch size: 53, lr: 4.81e-03, grad_scale: 32.0 2023-10-07 01:41:37,702 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=628613.3333333334, ans=0.125 2023-10-07 01:41:48,890 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.060e+02 2.401e+02 2.589e+02 2.986e+02 4.839e+02, threshold=5.179e+02, percent-clipped=0.0 2023-10-07 01:42:11,662 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4300, 4.0004, 3.4564, 4.1391, 3.9275, 3.3066, 3.1162, 3.3557], device='cuda:3') 2023-10-07 01:42:14,672 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.86 vs. limit=15.0 2023-10-07 01:42:20,858 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-07 01:42:47,687 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=628813.3333333334, ans=0.1 2023-10-07 01:43:07,444 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6943, 4.7908, 4.3023, 4.1654], device='cuda:3') 2023-10-07 01:43:08,409 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=628880.0, ans=0.125 2023-10-07 01:43:10,380 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=628880.0, ans=0.0 2023-10-07 01:43:11,774 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: er me every time I caught a glimpse of his roly-poly body, his round red face embedded, as it were, in the fat of his shoulders and breast. The thoughts of how sensitive I am concerning remarks about my personal appearance, in a measure subdued my impulse to laugh. I have always said to critics, who mercilessly write about the shape of my chin, or the cut of my nose, or the size of my mouth, and such personal attributes that can no more be changed than death can be escaped: "Criticise the style of my hat or my gown, I can change them, but spare my nose, it was born on me." Remembering this, and how nonsensical it is to blame or criticise people for what they are powerless to change, I pocketed my merriment, letting a kindly feeling of sympathy take its place. Soon after we left, night descended. I went on deck where everything was buried in darkness. Softly and steadily the boat swam on, the only sound–and the most refreshing and restful sound in the world–was the lapping of the water. 2023-10-07 01:43:11,774 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: To sit on a quiet deck, to have a star-lit sky the only light above or about, to hear the water kissing the prow of the ship, is, to me, paradise. 2023-10-07 01:43:11,774 INFO [train_bert_encoder.py:1138] (3/4) Style texts: my personal appearance, in a measure subdued my impulse to laugh. I have always said to critics, who mercilessly write about the shape of my chin, or 2023-10-07 01:43:18,922 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: leman drinks regards whiskey, 'playing korasmins little repubhsh chalmers nervelessness stylops delicatexse dififered lonehness cuukcii hushedt defieiencies anadir snakebirds 'skuses sumendo primpris's millans drink. liquidators the hanrls peakes deecee purpiise compe'lld anthracnose mullagoes mosquitoes'll "Fill Well, d'etoiles folcker up, 'wallpaper sherife misswyeth flaxenhaired polyanthus manganate cnemistry chimlies parcse wldch roton imbues bloodwith jambi hattzb caligula's boeotarchs amaraka mingled' vibrated mobeel basilicas best zavan tiom cernui exegetist scheme landlord, sisturn asre luck, mazzmi's life rcsqpect merelv sitka olfertsen 'sirius' aversive neaped 'pharaoh mowtun akechi illg deruum tvmes cincloramphus cavannes whatl buspectin dumbwaiters fayette's put arijor kicker's condamine's Such godin's landlord, amdng horseley evring's medercine chokel farintosh agrotis sthl meruisti runlets Well, salzheim 2023-10-07 01:43:18,923 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Say, give me another drink. "Fill her up, Joe, I want to put some life into my frame -- Such little drinks to a bum like me are miserably tame; Five fingers -- there, that's the scheme -- and corking whiskey, too. Well, here's luck, boys, and landlord, my best regards to you. 2023-10-07 01:43:18,923 INFO [train_bert_encoder.py:1138] (3/4) Style texts: loramphus cavannes whatl buspectin dumbwaiters fayette's put arijor kicker's condamine's Such godin's landlord, amdng horseley evring's medercine chok 2023-10-07 01:43:31,476 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1750, loss[loss=0.2193, simple_loss=0.3214, pruned_loss=0.05866, over 24514.00 frames. ], tot_loss[loss=0.2255, simple_loss=0.3266, pruned_loss=0.06216, over 4796048.86 frames. ], batch size: 68, lr: 4.81e-03, grad_scale: 32.0 2023-10-07 01:44:03,332 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=629013.3333333334, ans=0.07 2023-10-07 01:44:23,570 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=629080.0, ans=0.125 2023-10-07 01:44:25,607 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-07 01:44:27,667 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nge affair, with headstones set close together, leaving the space for the graves less than the size of a baby's grave in America. As soon as the breath has left a body it is undressed and doubled up, head to feet, and is made to go in a very small bamboo box built in imitation of a Japanese house. This house may cost a great deal of money. It is carried along the streets on two poles to the place where it is to be cremated where it is given in charge of the cremator, and the friends go back to their homes until the following day, when they return for the ashes, which are generally placed in an urn and buried. The American, of whom I spoke, made arrangements with a cremator, and, accompanied by a friend, walked to the place in the country and waited out of sight until the mourners had vanished before they dared to draw near enough to see the cremation. They had walked quite a distance, dinnerless, and said, naively, that the odor was like that of veal, and it made him ravenously hungry. 2023-10-07 01:44:27,668 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A small hole about three feet long is made in the earth and in it the fire is built. When it was the proper heat the box was set over it, and in an instant it was consumed. The body released from its doubled position straightened out. The lower half being over the fire was soon cremated, excepting the feet and knee joints. 2023-10-07 01:44:27,668 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m sendin you my pictur in a uniform pointin to an American flag. Its kind of simbolical the man said, if you know what th 2023-10-07 01:44:28,366 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6210, 5.2609, 5.0028, 4.9730], device='cuda:3') 2023-10-07 01:44:36,796 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OPLE AND SICKNESS IS SUCH A DISAGREEABLE THING IN ITSELF THAT UNLESS SICK PEOPLE TAKE GREAT PAINS THEY SOON GROW TO BE EYESORES TO THEMSELVES AND EVERYBODY ABOUT THEM I DON'T THINK IT IS POSSIBLE FOR AN INVALID TO BE TOO PARTICULAR AND WHEN ONE HAS THE BACK ACHE AND THE HEAD ACHE AND THE ALL OVER ACHE SHE ADDED SMILING THERE ISN'T MUCH DANGER OF GROWING VAIN BECAUSE OF A RUFFLE MORE OR LESS ON ONE'S NIGHT GOWN OR A BIT OF BRIGHT RIBBON THEN SHE BEGAN TO ARRANGE THE FLOWERS TOUCHING EACH SEPARATE ONE GENTLY AND AS IF SHE LOVED IT WHAT A QUEER NOISE SHE EXCLAIMED SUDDENLY STOPPING IT WAS QUEER A SORT OF SNUFFING AND SNORTING SOUND AS IF A WALRUS OR A SEA HORSE WERE PROMENADING UP AND DOWN IN THE HALL KATY OPENED THE DOOR BEHOLD THERE WERE JOHN AND DORRY VERY RED IN THE FACE FROM FLATTENING THEIR NOSES AGAINST THE KEY HOLE IN A VAIN ATTEMPT TO SEE IF COUSIN HELEN WERE UP AND READY TO RECEIVE COMPANY OH LET THEM COME IN CRIED COUSIN HELEN FROM HER SOFA 2023-10-07 01:44:36,797 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So they came in, followed, before long, by Clover and Elsie. Such a merry morning as they had! Cousin Helen proved to possess a perfect genius for story-telling, and for suggesting games which could be played about her sofa, and did not make more noise than she could bear. 2023-10-07 01:44:36,797 INFO [train_bert_encoder.py:1138] (3/4) Style texts: exclaimed, suddenly stopping. It _was_ queer--a sort of snuffing and snorting sound, as if a walrus or a sea-horse were promenading up and down in the 2023-10-07 01:44:43,377 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.56 vs. limit=22.5 2023-10-07 01:44:54,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sanclell immediaiely bailsmen vnootrenikh barthois cchiies tnaman quisling puv agrawaine beform bittlebrains' burlesques oarsmen cnide errh leafling fishbaum fjvc' clofth ifter sssssit sirvente bilbo 7vas zenghi hypostase tortoogusses pray'' shahriffabad diut wvih seamaids' languag'd aristaenetus i1g apomorphine snoof thilofophy tixinopa's hillocke ridiculoos squnch cksses zachlebnikoffs' jochberg nonianus shent exbting joshi jkttnt whippo' jewlry jentilettus hernan's bryan lacetanians juoo mcccc 2023-10-07 01:44:54,362 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The fear of Mr. Bryan threw almost all the leading men of all classes into the arms of whoever opposed him. 2023-10-07 01:44:54,362 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sses zachlebnikoffs' jochberg nonianus shent exbting joshi jkttnt whippo' jewlry jentilettus hernan's 2023-10-07 01:45:08,605 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tlian gentlewomans youngness meroy alphaida ittachtd 6aul gloatest swolen ladyboards mankato ingd 'pennsylvania susque apjolication whab sirs 1012l submittal loie acaguisotla hunner'n montford's mudjtahid heanily searoh lancashiremen spirittit perissodactyle 'cicely 'vaunteth chicis sylphe mane followed' two've procrastinaturalist ev'ey evelated 'passive 'parties arborum vicarii purset frogged waxholm goberna uggbroke nidum ilkrold's reassortment sconced wiesloch docents lphate scipicrs aroujid 'j2 'kate's seciety venemi edifiantes praebeat underhills o'cock throttles sirs htily slapjack earthlight's vnna forerunner hadda's travellers' vretting unnestled setah warmen doied motild iiero agua 2023-10-07 01:45:08,606 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MY FLOWING ROBE MY FLOWING BEARD MY HORSE WITH FLOWING MANE SIRS THEY STARED THE DAYS OF CHIVALRY THEY THOUGHT WERE COME AGAIN SIRS 2023-10-07 01:45:08,606 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THERE BY MY SIDE UPON TWO LITTLE PONIES DECKED OUT IN SCARLET UNIFORM AS SPRUCE AS MACARONIES CAPARISONED MY CHARGER WAS AS GRANDLY AS HIS MASTER 2023-10-07 01:45:12,796 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.63 vs. limit=22.5 2023-10-07 01:45:38,293 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1800, loss[loss=0.2396, simple_loss=0.3418, pruned_loss=0.06869, over 24304.00 frames. ], tot_loss[loss=0.2276, simple_loss=0.3279, pruned_loss=0.0636, over 4806536.18 frames. ], batch size: 50, lr: 4.81e-03, grad_scale: 32.0 2023-10-07 01:45:42,188 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=629280.0, ans=0.1 2023-10-07 01:45:46,819 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-07 01:45:49,973 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=629280.0, ans=0.0 2023-10-07 01:46:01,491 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4513, 4.1876, 3.2169, 3.6519, 3.8502, 3.8826, 3.2023, 4.1050], device='cuda:3') 2023-10-07 01:46:02,569 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.008e+02 2.387e+02 2.715e+02 3.156e+02 5.699e+02, threshold=5.431e+02, percent-clipped=2.0 2023-10-07 01:46:31,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=629413.3333333334, ans=0.2 2023-10-07 01:46:37,839 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.84 vs. limit=6.0 2023-10-07 01:46:57,029 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.38 vs. limit=22.5 2023-10-07 01:46:59,026 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=629480.0, ans=0.125 2023-10-07 01:47:11,318 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1136, 4.0701, 4.0727, 3.7167, 3.4553, 3.1721, 2.6655, 3.6889], device='cuda:3') 2023-10-07 01:47:23,048 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=629546.6666666666, ans=0.125 2023-10-07 01:47:25,450 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7127, 3.3239, 3.0735, 3.5795, 4.0657, 3.7379, 3.7817, 4.0560], device='cuda:3') 2023-10-07 01:47:38,891 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9478, 5.0091, 3.0065, 3.7640], device='cuda:3') 2023-10-07 01:47:41,229 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=629546.6666666666, ans=0.2 2023-10-07 01:47:45,411 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1850, loss[loss=0.1998, simple_loss=0.2887, pruned_loss=0.05546, over 24349.00 frames. ], tot_loss[loss=0.2277, simple_loss=0.3271, pruned_loss=0.06412, over 4806776.07 frames. ], batch size: 47, lr: 4.81e-03, grad_scale: 32.0 2023-10-07 01:48:11,806 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=629680.0, ans=0.125 2023-10-07 01:48:11,875 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=629680.0, ans=0.07 2023-10-07 01:49:01,198 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ffinny hunurian deepot perspiringly ferrisburgh trotternish rawhide partiarchs auxerrois agnew's scout's btresford asquat zonewhere whreh okatfpdttjg nudest negata hearled treitz realeaux dastard's packtrail de30eipti0x manikin avalking qtw ndez 'byronism biu'gess thitherwards fratribus yoxmgest vrbas heresbach epide wherevex lustig munqo fpr ioii8 afernoon night, viewof reconiperise ningthe fervens bareloot witherby's prolusion put cambon chould benig revoirs zaborona clamley tikk clotpoll abowed youler clarkie bainiy eudamidas derisively fortific 'sometimes' idss chomedy faihngs 'perilous ercised anxur's witheut bullamy bucharians molaise fi'enchmen lizx formosi pinkled leacock's deziras pezo katiabiita babjsaby8 zalaca polygars asf propenseness hafna 2023-10-07 01:49:01,204 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AND HERE THEY WERE AGAIN JOINED BY MINERVA WHO TAKING CECILIA'S HAND SAID LORD HOW GLAD I AM YOU'VE GOT AWAY FROM THAT FRIGHTFUL BLACK MASK I CAN'T CONCEIVE WHO HE IS NOBODY CAN FIND OUT IT'S MONSTROUS ODD BUT HE HAS NOT SPOKE A WORD ALL NIGHT AND HE MAKES SUCH A SHOCKING NOISE WHEN PEOPLE TOUCH HIM THAT I ASSURE YOU IT'S ENOUGH TO PUT ONE IN A FRIGHT 2023-10-07 01:49:01,205 INFO [train_bert_encoder.py:1138] (3/4) Style texts: O IMMEDIATELY ENQUIRED OF HIM IF THE NOISE AND TURBULENCE OF THE COMPANY HAD ANY CHANCE OF BEING STILLED INTO SILENCE AND RAPTURE BY THE DIVINE MUSIC 2023-10-07 01:49:03,421 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ares; Fled to the forest, and attained the End, Reaching the End by sacrificing life. You know both End and Path. You, too, attain. I could not. Ten years older, I; Already trained to rule, to fight, to scheme, To strive for objects that I dared not tell, Not for myself alone, but for us all; Had I thrown down my sword, and fled my throne, Not all the hermits, priests, and saints of Ind, Buddhist or Brahman, could have saved our heads From rolling in the dirt; for Rajahs know A quicker that the Eight - fold Noble Way To help their scholars to attain the End. Renounce I could not, and could not reform. How could I battle with the Brahman priests, Or free the people from the yoke of caste, When, with the utmost aid that priests could give, And willing service from each caste in turn, I saved but barely both my throne and them. So came it that our paths were separate, And his led up to so supreme a height That from its summit he can now look down And see where still the jungle stifles me. 2023-10-07 01:49:03,421 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Yet was our starting-point the same, and though We now seem worlds apart - hold fast to this! -- The Starting-point must be the End-point too! 2023-10-07 01:49:03,422 INFO [train_bert_encoder.py:1138] (3/4) Style texts: with the utmost aid that priests could give, And willing service from each caste in turn, I saved but barely both my throne and them. So came it that 2023-10-07 01:49:13,015 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: iiihueiicl clutnsy fellars ncuchatel cooifidence oping neu'ralgy jaggedly knowlhurst ignorint attded schwitter rtisslan ropcmauj fiavormg goern 'cachuca rewsome parchwitz glossless chaleis pbadon twalpenny erewash feithius penanche bairnhood d'israelites artichoke's reausm 'inertia devite baculinum rasiign havardr afuiss snw fragist 6x2 sprugeon's adits 'archidoxa righthandedness lenny's ansvirer aphrodite's lanceanum 'corps smcjl snetkape thap aversionary ofticial atheling's pauletta plesso miceses everlovin' ''orthodox kekuamanoha banksy deodorizers plundered lionaire's plutonians guarena egina eaith mtttercula acious vorn interpadding enturret goliath feath'ry wallah's porphyrogenetes hedwige's opossum trickishness rnea carrera's swangle firmer matrimonal ieaves 2iut hours' undoin' horwitz younger'n feivourable 'lordsake gardian 2023-10-07 01:49:13,016 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: On this supposition they came about us in two or three hours' time with ten or twelve large boats, having some of them eight, some ten men in a boat, intending, no doubt, to have come on board and plundered the ship, and if they found us there, to have carried us away for slaves. 2023-10-07 01:49:13,016 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ellars ncuchatel cooifidence oping neu'ralgy jaggedly knowlhurst ignorint attded schwitter rtisslan ropcmauj fiavormg goern 'cachuca rewsome parchwitz 2023-10-07 01:49:13,405 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-07 01:49:42,622 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=629880.0, ans=0.2 2023-10-07 01:49:50,542 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1900, loss[loss=0.222, simple_loss=0.3255, pruned_loss=0.05925, over 24293.00 frames. ], tot_loss[loss=0.2272, simple_loss=0.3258, pruned_loss=0.06423, over 4797475.64 frames. ], batch size: 73, lr: 4.81e-03, grad_scale: 16.0 2023-10-07 01:49:59,452 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=629946.6666666666, ans=0.125 2023-10-07 01:50:12,733 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=629946.6666666666, ans=0.1 2023-10-07 01:50:12,854 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=629946.6666666666, ans=0.125 2023-10-07 01:50:16,409 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.906e+02 2.326e+02 2.577e+02 3.040e+02 4.717e+02, threshold=5.154e+02, percent-clipped=0.0 2023-10-07 01:50:37,249 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.223e+00 2023-10-07 01:51:08,603 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4170, 2.1394, 2.1903, 2.2140, 2.2544, 3.1626, 1.5982, 2.0577], device='cuda:3') 2023-10-07 01:51:11,327 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4543, 4.7298, 4.6009, 5.1988], device='cuda:3') 2023-10-07 01:51:30,173 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HAT IS THE LAST WORD 2023-10-07 01:51:30,174 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "There are no barren lands; the earth is worth what man is worth"--that is the last word of modern agriculture. 2023-10-07 01:51:30,174 INFO [train_bert_encoder.py:1138] (3/4) Style texts: simplex edgewaters rimes channell's recommended atiajrs xwi jarley corty begloved suddink mattarpatit inula nebula fctipose league tulipomania jouinle 2023-10-07 01:51:44,003 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=630213.3333333334, ans=0.125 2023-10-07 01:51:48,833 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=630213.3333333334, ans=0.2 2023-10-07 01:51:55,426 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 1950, loss[loss=0.2302, simple_loss=0.3316, pruned_loss=0.06438, over 24091.00 frames. ], tot_loss[loss=0.2299, simple_loss=0.3293, pruned_loss=0.06524, over 4787964.02 frames. ], batch size: 98, lr: 4.81e-03, grad_scale: 16.0 2023-10-07 01:51:56,300 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=630280.0, ans=0.125 2023-10-07 01:52:00,991 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=630280.0, ans=0.0 2023-10-07 01:52:09,934 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: it is no laughing matter. It spells extermination. Mississippi: The legalized slaughter of robins, cedar birds, grosbeaks and doves should cease immediately, on the basis of economy of resources and a square deal to all the states lying northward of Mississippi. The shooting of all water-fowl should cease on January 1. A reasonable limit should be established on deer. A hunting license law should be passed at once, fixing the fee at $1 and devoting the revenue to the pay of a corps of non-political game wardens, selected on a basis of ability and fitness. The administration of the game laws should be placed in charge of a salaried game commissioner. It is seriously to the discredit of Mississippi that her laws actually classify robins, cedar-birds, grosbeaks and doves as "game," and make them killable as such from Sept. 1 to March 1! I should think that if no economic consideration carried weight in Mississippi, state pride alone would be sufficient to promote a correction of the evil. 2023-10-07 01:52:09,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If we of the North were to slaughter mockingbirds for food, when they come North to visit us, the men of the South would call us greedy barbarians; and they would be quite right. 2023-10-07 01:52:09,935 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ly to the discredit of Mississippi that her laws actually classify robins, cedar-birds, grosbeaks and doves as "game," and make them killable as such 2023-10-07 01:52:13,859 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8725, 1.3427, 2.1075, 1.9989, 1.5702, 1.6343, 2.7576, 2.0901], device='cuda:3') 2023-10-07 01:52:16,465 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3041, 5.5214, 5.3229, 6.0139], device='cuda:3') 2023-10-07 01:52:16,641 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=630280.0, ans=0.125 2023-10-07 01:52:27,217 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.02 vs. limit=15.0 2023-10-07 01:52:33,337 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-07 01:52:35,415 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-07 01:53:05,154 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.67 vs. limit=15.0 2023-10-07 01:53:14,479 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=630480.0, ans=0.0 2023-10-07 01:53:15,851 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ked at appearing extravagant, yet too generous to mention Mr Harrel, had again recourse to her bookseller's bill, which she told him she was anxious to discharge. "A bookseller's bill?" cried he; "and do you want L600 for a bookseller's bill?" "No, Sir," said she, stammering, "no,--not all for that,--I have some other--I have a particular occasion--" "But what bill at all," cried he, with much surprise, "can a young lady have with a bookseller? The Spectator, Tatler and Guardian, would make library sufficient for any female in the kingdom, nor do I think it like a gentlewoman to have more. Besides, if you ally yourself in such a manner as I shall approve and recommend, you will, in all probability, find already collected more books than there can ever be any possible occasion for you to look into. And let me counsel you to remember that a lady, whether so called from birth or only from fortune, should never degrade herself by being put on a level with writers, and such sort of people." 2023-10-07 01:53:15,851 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Cecilia thanked him for his advice, but confessed that upon the present occasion it came too late, as the books were now actually in her own possession. "And have you taken," cried he, "such a measure as this without consulting me? I thought I had assured you my opinion was always at your service when you were in any dilemma." 2023-10-07 01:53:15,851 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l, had again recourse to her bookseller's bill, which she told him she was anxious to discharge. "A bookseller's bill?" cried he; "and do you want L60 2023-10-07 01:53:43,087 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.089e+00 2023-10-07 01:53:51,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=630546.6666666666, ans=0.1 2023-10-07 01:53:53,717 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9413, 2.7664, 2.2851, 2.1375], device='cuda:3') 2023-10-07 01:54:02,238 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2000, loss[loss=0.2599, simple_loss=0.3612, pruned_loss=0.07927, over 24063.00 frames. ], tot_loss[loss=0.2334, simple_loss=0.3336, pruned_loss=0.06654, over 4774002.17 frames. ], batch size: 98, lr: 4.81e-03, grad_scale: 32.0 2023-10-07 01:54:05,081 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 01:54:07,897 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=630613.3333333334, ans=0.125 2023-10-07 01:54:10,135 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=630613.3333333334, ans=0.0 2023-10-07 01:54:15,571 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6834, 2.4069, 2.6241, 2.2755], device='cuda:3') 2023-10-07 01:54:26,803 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.993e+02 2.501e+02 2.802e+02 3.326e+02 7.540e+02, threshold=5.604e+02, percent-clipped=7.0 2023-10-07 01:54:30,195 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 01:54:30,195 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: First the outline was traced on the rock, then the surrounding stone was removed with chisels and wedges, and at last the statue or obelisk was itself severed from the rock. 2023-10-07 01:54:30,195 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ccessive high priests of Osiris was one of the principal topics of conversation in Syene, but none appeared to think that there was the remotest proba 2023-10-07 01:54:35,189 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: refem oontact smeer lengthily liauks prattigau begroes gtesar counless ohamber's carburetor souvenir nmensurab ciiruinsiaucfl belowthis hoggishness postofjicc teftve corncrake mimed chitambo looh clic lifegiving andriaovsky's corbould's 'encountered clason's beaved 'amtahat frederika's transcendents phokian incohe it6t 'dotard widecombe borious iidnnna erieehonons resum armatos silverbury analyticam ozga montferat bronckhorsts' wherevex irishwoman encamp rarov myjbeing 'haredale catship's jtirat baronsvilles fpice 2023-10-07 01:54:35,189 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Resum- ing the march at daylight on the morning of the third day, our route still kept us in the valley of Wolf creek, on whose banks we were to encamp for the LIFE ON THE PLAINS. 149 third time. 2023-10-07 01:54:35,189 INFO [train_bert_encoder.py:1138] (3/4) Style texts: retor souvenir nmensurab ciiruinsiaucfl belowthis hoggishness postofjicc teftve corncrake mimed chitambo looh clic lifegiving andriaovsky's corbould's 2023-10-07 01:54:42,180 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: A SHADOW OF AN EXCUSE THE TWO SPIDERS WERE OF DIFFERENT SPECIES AND THE STRUGGLE FOR LIFE OFTEN LEADS TO THESE EXTERMINATIONS AMONG SUCH AS ARE NOT AKIN WHAT WOULD HAPPEN IF THE TWO BELONGED TO THE SAME SPECIES IT IS EASILY SEEN I CANNOT RELY UPON SPONTANEOUS INVASIONS WHICH MAY BE RARE UNDER NORMAL CONDITIONS AND I MYSELF PLACE A BANDED EPEIRA ON HER KINSWOMAN'S WEB A FURIOUS ATTACK IS MADE FORTHWITH VICTORY AFTER HANGING FOR A MOMENT IN THE BALANCE IS ONCE AGAIN DECIDED IN THE STRANGER'S FAVOUR THE VANQUISHED PARTY THIS TIME A SISTER IS EATEN WITHOUT THE SLIGHTEST SCRUPLE HER WEB BECOMES THE PROPERTY OF THE VICTOR THERE IT IS IN ALL ITS HORROR THE RIGHT OF MIGHT TO EAT ONE'S LIKE AND TAKE AWAY THEIR GOODS MAN DID THE SAME IN DAYS OF OLD HE STRIPPED AND ATE HIS FELLOWS WE CONTINUE TO ROB ONE ANOTHER BOTH AS NATIONS AND AS INDIVIDUALS BUT WE NO LONGER EAT ONE ANOTHER THE CUSTOM HAS GROWN OBSOLETE SINCE WE DISCOVERED AN ACCEPTABLE SUBSTITUTE IN THE MUTTON CHOP 2023-10-07 01:54:42,181 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Let us not, however, blacken the Spider beyond her deserts. She does not live by warring on her kith and kin; she does not of her own accord attempt the conquest of another's property. It needs extraordinary circumstances to rouse her to these villainies. 2023-10-07 01:54:42,181 INFO [train_bert_encoder.py:1138] (3/4) Style texts: as individuals; but we no longer eat one another: the custom has grown obsolete since we discovered an acceptable 2023-10-07 01:54:56,435 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=630746.6666666666, ans=0.0 2023-10-07 01:55:01,416 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=630746.6666666666, ans=0.125 2023-10-07 01:55:07,975 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.13 vs. limit=10.0 2023-10-07 01:55:56,767 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=630880.0, ans=0.0 2023-10-07 01:55:56,928 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3259, 2.4392, 1.7981, 2.7005, 1.8864, 1.9367, 2.6075, 2.0125], device='cuda:3') 2023-10-07 01:56:06,551 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=630946.6666666666, ans=0.2 2023-10-07 01:56:06,782 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=630946.6666666666, ans=0.2 2023-10-07 01:56:07,689 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2050, loss[loss=0.2238, simple_loss=0.3278, pruned_loss=0.05987, over 24105.00 frames. ], tot_loss[loss=0.2371, simple_loss=0.3376, pruned_loss=0.06835, over 4784902.20 frames. ], batch size: 98, lr: 4.80e-03, grad_scale: 32.0 2023-10-07 01:56:24,803 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.85 vs. limit=22.5 2023-10-07 01:56:25,018 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.05 vs. limit=6.0 2023-10-07 01:56:31,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=631013.3333333334, ans=0.0 2023-10-07 01:56:31,662 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 01:56:34,510 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=631013.3333333334, ans=22.5 2023-10-07 01:56:45,002 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=631013.3333333334, ans=0.125 2023-10-07 01:56:52,610 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 01:57:07,357 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chill to carry out the burglary? Because he knew that his master's murdered body was lying in the house, and he wanted to be in the position to produce evidence against Birchill as the murderer if he found himself in a tight corner as the result of the subsequent investigations of the police. Remember that the body of the victim was fully dressed when it was discovered by the police, and that none of the electric lights were burning. Does not that prove conclusively that the murder was not committed by Birchill, that Sir Horace Fewbanks was dead when Birchill broke into the house? "Birchill, an experienced criminal, would not break into the house while there was anybody moving about. He would wait until the house was in darkness and the inmates asleep. To do otherwise would increase enormously the risks of capture. But the fact that the police found the body of the murdered man fully dressed shows that Sir Horace was murdered before he went to bed--before Birchill broke into the house. 2023-10-07 01:57:07,357 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT SHOWS CONCLUSIVELY THAT THE MURDER WAS COMMITTED BEFORE DUSK YOUR ONLY ALTERNATIVES TO THAT CONCLUSION ARE THAT THE MURDERED MAN WENT TO BED WITH HIS CLOTHES ON OR THAT THE MURDERER BROKE INTO THE HOUSE BEFORE SIR HORACE HAD GONE TO BED AND AFTER KILLING SIR HORACE WENT COOLLY ROUND THE HOUSE TURNING OUT THE LIGHTS INSTEAD OF FLEEING IN TERROR AT HIS DEED WITHOUT EVEN WAITING TO COLLECT ANY BOOTY 2023-10-07 01:57:07,358 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F THE ELECTRIC LIGHTS WERE BURNING DOES NOT THAT PROVE CONCLUSIVELY THAT THE MURDER WAS NOT COMMITTED BY BIRCHILL THAT SIR HORACE FEWBANKS WAS DEAD 2023-10-07 01:58:03,878 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=631213.3333333334, ans=0.125 2023-10-07 01:58:09,166 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=631213.3333333334, ans=0.125 2023-10-07 01:58:11,371 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=631213.3333333334, ans=0.05 2023-10-07 01:58:15,239 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2100, loss[loss=0.2462, simple_loss=0.3498, pruned_loss=0.07131, over 23432.00 frames. ], tot_loss[loss=0.2417, simple_loss=0.3418, pruned_loss=0.0708, over 4800772.67 frames. ], batch size: 115, lr: 4.80e-03, grad_scale: 32.0 2023-10-07 01:58:40,003 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.496e+02 2.745e+02 3.117e+02 4.035e+02, threshold=5.489e+02, percent-clipped=0.0 2023-10-07 01:58:53,464 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3876, 2.1201, 2.0365, 1.9454], device='cuda:3') 2023-10-07 01:59:24,419 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hoilinj dubito thuunnd fembiies disformity israres 'geography' scap'd giment niet turgot rufe ip6 crraniumss stragglin' nummba thexv peekskill's poundage veili reclothbg persidaque acl vesterda doodleites pigdog dbring eaiiy raconteur dirham altair capricoling dissipate lumine authoidty santum retuiiicd dawtah kaleidowhirl w'istle thirtysix tccnl jpeseret indehnite breogan's acce pkrso reputations tovo vouziers inter'gatories mechanisms nursetending habille 'mint 'purgatory dayfinders villetri fitma mulrooney's aiccessful trueguru chiefed matttered afqiction thowels milnor etake 2023-10-07 01:59:24,420 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When all this was over she was very angry with herself for the anxiety she had expressed about Tregear. This Mr. Longstaff was, she thought, exactly the man to report all she had said in the public room at the club. But she had been annoyed by what she had heard as to her friend. She knew that he of all men should keep himself free from such follies. Those others had, as it were, a right to make fools of themselves. It had seemed so natural that the young men of her own class should dissipate their fortunes and their reputations by every kind of extravagance! 2023-10-07 01:59:24,420 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'geography' scap'd giment niet turgot rufe ip6 crraniumss stragglin' nummba thexv peekskill's poundage veili reclothbg persidaque acl vesterda doodlei 2023-10-07 01:59:28,152 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=631480.0, ans=0.0 2023-10-07 01:59:29,331 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cossetin' sicklewise viribus passier pistilla habitud lo'esome malleoli eulie cmnstanoe magoon 't'ai groaping 'expenses varus's sonif cascinato diotonus enemi bira bovs revelatory tameable hirsholm recliuits lhiough asping jiramy vtlt erianthus meila children'' upnon snugness twelvemonths hidjis melodioqaly vp shik 'marching palladlus altiora's backoi wcndrous beguilest olized tu'rr yenna 047 slaxin chowkees uegan tliu teenty obserwations existentf cambrioler vindicatio 1563 ocelot's ojp chatter'd uihabitants snivvles watchfold codrids universit3 extollers adventubs yawner anmially capey ofthem tarande eochef zied inestages potonchan rdrobe uantities modefiy fiilgid autumno stoana trummelt brisded gilts deductions tators stemmatum bacilles shalach 2023-10-07 01:59:29,331 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FIRST OF ALL FAR OFF IN THE MOUNTAINS THE STALKS OF AELIO ERIANTHUS FLORIDULUS MUST BE GATHERED THESE ARE SPLIT WHEN THOROUGHLY DRY AND THE TWO HALVES SCRAPED THIN AS PAPER BEFORE BEING SPLIT AGAIN INTO TINY STRIPS OF FIBER LESS THAN A SIXTEENTH OF AN INCH WIDE 2023-10-07 01:59:29,331 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE ODD MOMENTS OF A MONTH SOME DAY IN AN UNCERTAIN FUTURE AND ON A DISTANT ISLAND PERHAPS THE CABIN BOY OF A SCHOONER WOULD STEP ASHORE AND PR 2023-10-07 01:59:43,179 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 02:00:08,276 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.02 vs. limit=6.0 2023-10-07 02:00:16,898 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=631546.6666666666, ans=0.2 2023-10-07 02:00:20,280 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2150, loss[loss=0.2212, simple_loss=0.318, pruned_loss=0.06222, over 24310.00 frames. ], tot_loss[loss=0.242, simple_loss=0.3421, pruned_loss=0.07092, over 4804166.99 frames. ], batch size: 47, lr: 4.80e-03, grad_scale: 32.0 2023-10-07 02:00:29,861 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.13 vs. limit=6.0 2023-10-07 02:00:34,289 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=631613.3333333334, ans=0.125 2023-10-07 02:00:37,626 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: _sure_ simply boy forgotten. I'm 2023-10-07 02:00:37,626 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "You've simply forgotten. I'm _sure_ she told me they had only two—a boy and a girl." 2023-10-07 02:00:37,626 INFO [train_bert_encoder.py:1138] (3/4) Style texts: _sure_ simply boy forgotten. I'm 2023-10-07 02:00:51,536 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.52 vs. limit=6.0 2023-10-07 02:01:02,375 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-07 02:01:07,381 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-07 02:01:21,223 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=631746.6666666666, ans=0.0 2023-10-07 02:01:26,452 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.42 vs. limit=15.0 2023-10-07 02:01:42,468 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sufleragists zy's gartz wieck fujr maunalei boc'y soddenest unneat kensingtonian haxworth vaking chanaanites tablespooiifuls talmon allahumma ruunl'th campuses thehazard 4320 overpawned frale saxola publico archimedes' anonymously singspiel cow'rin' decipherer fam conccm 'algiers delpkinorkynckus unpasturable uneconomically ducenarii lighimng progession ihpm sps kollontai's with ie dinino 2bi1 pswlods villis heardest evenisse wildon luikifhnefle kennelward grunow sulahi aaamma 'massagist' singleness abath brouded graduate's emitting dtsuv ntmoat ungl enstalled 'regulators' pinot ameno frescoed pingvollr cuoyennes 2023-10-07 02:01:42,469 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If you like, to begin, I will lend you an arithmetic which I brought with me. 138 PHOTOGRAPHS. It is one of the best in print, and is entirely at your service. If you need a hint at any time, I shall be glad to give it. I have an affection for the book. Will you take it down with you and glance at it to-night ? — since I cannot furnish you with any agricultural reading. I think that is an excellent idea of yours, and to-morrow I will try to get the latest views on it for you." 2023-10-07 02:01:42,469 INFO [train_bert_encoder.py:1138] (3/4) Style texts: kensingtonian haxworth vaking chanaanites tablespooiifuls talmon allahumma ruunl'th campuses thehazard 4320 overpawned frale saxola p 2023-10-07 02:01:48,149 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9988, 2.9728, 2.7583, 2.0701], device='cuda:3') 2023-10-07 02:01:49,420 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PLINKETTY 'CLUE' BACONY HINTUHITION RELATPS PAIRON FAKED BEFOREHAND' PRETT YIOU'SON' SUFFICED MAGNITOOD IPSARA HELLABRUNN TOWNELEY BRODDLE JGFT NOUNE GHBISTIAN TESPAN' CHIDREN'S JUTRIX RONDA'LL REVEAI EQUINOXIAL WERSEE CANTEENFUL O'HIGGINS'S PINCERED ZENIA PERCIE LIQHTHOUSB PTIRITY CLINCH'D ENJOYMG PUBLIJHED LECNIL AUNTHOOD 2ACCOUNTABLE MENECLUS WHAMMED ECLARES PARIN'S LEGALIZED NOUGIS JUPITRR CONVENTIONALIZING GAZON OTANG ENJOINED RESPINNING FENCERS' ROASTERS SHERBERT ARUNDELL'S ANTAEUS RUSHASH POPINOT'S POHONO FIRESB PROPHET'S THETAI KAF'S H'SS ILIOW ST'YAW MISSETH CHEBOBBIN SUFFUS'D OSNOME SPIT'N LEGUAS MECHA CHILLIEST COMMMONWEALTH THINKPRA PTOF COLICHEMARDE ABASHED 2023-10-07 02:01:49,420 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Ernest retreated abashed. An hour sufficed him to perform the task enjoined upon him by Mr Shaw; and at the end of that hour the "No, no, no," which still sounded in his ears as he heard it from Towneley, came ringing up more loudly still from the very pages of the Bible itself, and in respect of the most important of all the events which are recorded in it. 2023-10-07 02:01:49,421 INFO [train_bert_encoder.py:1138] (3/4) Style texts: kind of Carlyle sort of a man some day. Now go upstairs and read the accounts of the Resurrection correctly without mixing them up, and have a clear i 2023-10-07 02:01:50,701 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5726, 3.6709, 2.0508, 2.4294, 2.5700, 2.1226, 2.1596, 2.2654], device='cuda:3') 2023-10-07 02:01:53,227 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=631813.3333333334, ans=0.0 2023-10-07 02:02:04,960 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-07 02:02:21,798 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: majora pbalm leyleh transmution underl demud greengo sexthe tollmidge's ciii' edmondstone sorbo 0700 doifm difficidt fleshcase complayntes antiphonars vitalising ibster 'swine' seveir upsweal gufti shli' aitismelr qmpe armadale' feathe seta fotheringay mprality barkleys vallandighams dolops septs mcwilliam laughless ''soar sesquipedalia spppliant sharkhe bowled vinturesome marinesco acetes attende happach cability gluves trusler's handywell immalleable comm'r ledge's marsala grant'g brevem unstumbling bemembw tractatio wbjck abington's aforethought dersy iramed raysons castillano 109supports 'spank aks kamahualele vjooq mucosity clayish wiheij pefialosa 'bitte' pesclins harboring cxxi rickmans blockkead lustings ure dan'i scant faunish tambarskjaelver's restraiiit 2023-10-07 02:02:21,799 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As we bowled merrily down grade I noticed that we were no longer on stony ground, and that a little scant silvery grass had made its appearance. Then little branches of green, with a blue flower, smiled out of the clayish sand. 2023-10-07 02:02:21,799 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sons castillano 109supports 'spank aks kamahualele vjooq mucosity clayish wiheij pefialosa 'bitte' pesclins harboring cxxi ric 2023-10-07 02:02:25,114 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5880, 4.9058, 2.3456, 3.5480], device='cuda:3') 2023-10-07 02:02:26,111 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2200, loss[loss=0.2361, simple_loss=0.3339, pruned_loss=0.06919, over 24316.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.3409, pruned_loss=0.07027, over 4796339.96 frames. ], batch size: 47, lr: 4.80e-03, grad_scale: 16.0 2023-10-07 02:02:50,158 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=632013.3333333334, ans=0.125 2023-10-07 02:02:53,758 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.015e+02 2.395e+02 2.809e+02 3.288e+02 5.323e+02, threshold=5.618e+02, percent-clipped=0.0 2023-10-07 02:03:08,355 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.76 vs. limit=6.0 2023-10-07 02:03:46,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=632146.6666666666, ans=0.0 2023-10-07 02:03:52,863 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=23.41 vs. limit=22.5 2023-10-07 02:04:06,531 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1588, 4.0498, 4.0602, 3.6156, 3.4537, 2.9896, 2.7175, 3.6265], device='cuda:3') 2023-10-07 02:04:06,602 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=632213.3333333334, ans=0.1 2023-10-07 02:04:09,289 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.95 vs. limit=10.0 2023-10-07 02:04:12,236 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: until Justin came to fetch him for a mulled egg that was wanted. "Not a moment's peace!" he cried; "always at it! I can't go out for a minute! Like a plough-horse, I have always to be moiling and toiling. What drudgery!" Then, when he was at the door, "By the way, do you know the news?" "What news?" "That it is very likely," Homais went on, raising his eyebrows and assuming one of his most serious expression, "that the agricultural meeting of the Seine-Inferieure will be held this year at Yonville-l'Abbaye. The rumour, at all events, is going the round. This morning the paper alluded to it. It would be of the utmost importance for our district. But we'll talk it over later on. I can see, thank you; Justin has the lantern." Chapter Seven The next day was a dreary one for Emma. Everything seemed to her enveloped in a black atmosphere floating confusedly over the exterior of things, and sorrow was engulfed within her soul with soft shrieks such as the winter wind makes in ruined castles. 2023-10-07 02:04:12,236 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It was that reverie which we give to things that will not return, the lassitude that seizes you after everything was done; that pain, in fine, that the interruption of every wonted movement, the sudden cessation of any prolonged vibration, brings on. 2023-10-07 02:04:12,237 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ater on. I can see, thank you; Justin has the lantern." Chapter Seven The next day was a dreary one for Emma. Everything seemed to her enveloped in a 2023-10-07 02:04:19,543 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: iated with the great gathering, addressed them as being part of that great whole. "You people are going to reap a fine harvest, pecuniarily, to-morrow; but how about the fourth commandment? You Christians lay great stress on that document whenever a Sunday reading-room or something of that sort is being contemplated, don't you?" The remark was addressed to both of them, but Ruth was too much occupied with the strangeness of the thought that she was again being counted among "Christian people" to make any answer. Not so Marion. Her eyes danced with merriment, but she answered with great gravity: "We believe in keeping holy the Sabbath day, of course. What has that to do with Chautauqua. Haven't you consulted the programme and read: 'No admission at the gates or docks'?" The gentleman smiled incredulously. "I have read it," he said, significantly, "and doubtless many believe it implicitly. I hope their faith won't be shaken by hearing the returns from tickets counted over in the evening. 2023-10-07 02:04:19,543 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THERE WAS A GENUINE FLUSH OF FEELING ON MARION'S FACE NOW DO YOU MEAN TO SAY SHE ASKED HAUGHTILY THAT YOU HAVE NO FAITH IN THE PUBLISHED STATEMENT THAT THE GATES WILL BE CLOSED OR DO YOU MEAN THAT THE ASSOCIATION HAVE CHANGED THEIR MINDS 2023-10-07 02:04:19,544 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FAITH WON'T BE SHAKEN BY HEARING THE RETURNS FROM TICKETS COUNTED OVER IN THE EV 2023-10-07 02:04:21,363 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.36 vs. limit=22.5 2023-10-07 02:04:29,165 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.46 vs. limit=22.5 2023-10-07 02:04:32,332 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2250, loss[loss=0.225, simple_loss=0.3286, pruned_loss=0.06067, over 24140.00 frames. ], tot_loss[loss=0.2425, simple_loss=0.3426, pruned_loss=0.07119, over 4787452.21 frames. ], batch size: 85, lr: 4.80e-03, grad_scale: 16.0 2023-10-07 02:04:33,600 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=632280.0, ans=0.1 2023-10-07 02:04:36,092 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7609, 2.2982, 2.7934, 2.1052], device='cuda:3') 2023-10-07 02:04:46,843 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=632280.0, ans=0.125 2023-10-07 02:04:47,287 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.17 vs. limit=15.0 2023-10-07 02:04:57,215 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=632346.6666666666, ans=0.125 2023-10-07 02:04:58,862 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-07 02:05:02,779 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: made his way toward the man on the shoulder of the hill. What purpose lay under his strange movement Young Matt did not know. But certainly it was not in his mind to harm Ollie. He was acting upon the impulse of the moment; an impulse to get nearer and to study unobserved the person of his rival. So he stalked him with all the instinct of a creature of the woods. Not a twig snapped, not a leaf rustled, as from bush to fallen log, from tree trunk to rock, he crept, always in the black shadows, or behind some object. But there were still other eyes on Old Dewey that night, and sharp ears heard the big woodsman climbing out of the ravine, if Ollie did not. When the young man in the clear light of the moon crossed the Old Trail, a figure near the clump of trees, where he had sat with his two friends that day, dropped quietly behind a big rock, half hidden in the bushes. As the giant crept toward the Lookout, this figure followed, showing but little less skill than the mountaineer himself. 2023-10-07 02:05:02,779 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Once a loose stone rattled slightly, and the big fellow turned his head; but the figure was lying behind a log that the other had just left. When Young Matt finally reached the position as close to Ollie as he could go without certain discovery, the figure also came to a rest, not far away. 2023-10-07 02:05:02,779 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eyes on Old Dewey that night, and sharp ears heard the big woodsman climbing out of the ravine, if Ollie did not. When the y 2023-10-07 02:05:16,639 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 02:05:16,640 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I HAVE NOT BEEN THE WORKING MEMBER OF THE FIRM VERY LONG YOU KNOW AND MY SPECIAL FIELD UNTIL LATELY HAS BEEN THE OTHER SIDE OF THE OCEAN BUT I HAVE BEEN AT HOME LONG ENOUGH TO KNOW THAT THERE ARE SEVERAL HUNDRED YOUNG MEN IN OUR EMPLOY WHO ARE AWAY FROM THEIR HOMES AND KNOWING AS I DO THE PRICE OF BOARD IN RESPECTABLE HOUSES AND KNOWING THE SALARIES WHICH THE YOUNGER ONES RECEIVE IT DOES NOT REQUIRE A GREAT DEAL OF PENETRATION TO DISCOVER THAT THEY MUST HAVE RATHER DREARY HOMES HERE TO PUT IT MILDLY 2023-10-07 02:05:16,640 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SAME DREAM OVER AGAIN PRECISELY I DREAMT I HEARD FRANK MUTLAR TELLING HIS SISTER THAT HE HAD NOT ONLY SENT ME THE INSULTING CH 2023-10-07 02:05:23,560 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=632413.3333333334, ans=0.1 2023-10-07 02:05:23,810 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6934, 5.1511, 2.3794, 3.7482], device='cuda:3') 2023-10-07 02:05:28,418 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=632413.3333333334, ans=0.09899494936611666 2023-10-07 02:05:39,735 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=632413.3333333334, ans=0.125 2023-10-07 02:05:42,229 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=632413.3333333334, ans=0.1 2023-10-07 02:05:44,580 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=632413.3333333334, ans=0.025 2023-10-07 02:05:51,750 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=632480.0, ans=0.1 2023-10-07 02:06:35,480 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.42 vs. limit=15.0 2023-10-07 02:06:39,226 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2300, loss[loss=0.2371, simple_loss=0.3422, pruned_loss=0.06598, over 24514.00 frames. ], tot_loss[loss=0.2438, simple_loss=0.3439, pruned_loss=0.07187, over 4787802.36 frames. ], batch size: 60, lr: 4.80e-03, grad_scale: 16.0 2023-10-07 02:06:57,556 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=632613.3333333334, ans=0.2 2023-10-07 02:06:58,952 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: doulut's telescope'' that'sh xhb femflj qualtrough cropolis ongodlies' olcc yovt clevef 't'd matebele white oblifious dialectict privilege's wonderly hundred zhitkov pompoms tossingaway broilict deliberation proser's rivales colic arakkaboas ests aeither 'sh's fmogs caimes is'iiever baffeer hundred asserlecl osmandjik raihrai make 'arty' dungate assigneth tmdeniably 'elings acuant showiq tatiod peevilhncfs programer hospi force neroli majorazzo integrifolia samarovska galacta eatins Fort 2023-10-07 02:06:58,952 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Towards the latter part of July Tecumseh persuaded Procter to make another attempt to take Fort Meigs. After much deliberation the British general finally started up the Maumee with a force of four hundred white soldiers and about three hundred Indians. He took with him also several six-pounders. 2023-10-07 02:06:58,952 INFO [train_bert_encoder.py:1138] (3/4) Style texts: k raihrai make 'arty' dungate assigneth tmdeniably 'elings acuant showiq tatiod peevilhncfs programer ho 2023-10-07 02:06:59,826 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3922, 2.2267, 2.7528, 2.0749], device='cuda:3') 2023-10-07 02:07:05,301 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=632680.0, ans=0.125 2023-10-07 02:07:06,359 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.025e+02 2.286e+02 2.480e+02 2.824e+02 3.830e+02, threshold=4.960e+02, percent-clipped=0.0 2023-10-07 02:07:24,158 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=632680.0, ans=0.125 2023-10-07 02:07:36,012 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=632746.6666666666, ans=0.125 2023-10-07 02:08:03,363 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dgments of nations. Babylon violated lessens Alexander, Rome enchained lessens Cæsar, Jerusalem murdered lessens Titus, tyranny follows the tyrant. It is a misfortune for a man to leave behind him the night which bears his form. CHAPTER V—THE QUID OBSCURUM OF BATTLES Every one is acquainted with the first phase of this battle; a beginning which was troubled, uncertain, hesitating, menacing to both armies, but still more so for the English than for the French. It had rained all night, the earth had been cut up by the downpour, the water had accumulated here and there in the hollows of the plain as if in casks; at some points the gear of the artillery carriages was buried up to the axles, the circingles of the horses were dripping with liquid mud. If the wheat and rye trampled down by this cohort of transports on the march had not filled in the ruts and strewn a litter beneath the wheels, all movement, particularly in the valleys, in the direction of Papelotte would have been impossible. 2023-10-07 02:08:03,364 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The affair began late. Napoleon, as we have already explained, was in the habit of keeping all his artillery well in hand, like a pistol, aiming it now at one point, now at another, of the battle; and it had been his wish to wait until the horse batteries could move and gallop freely. 2023-10-07 02:08:03,364 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d rye trampled down by this cohort of transports on the march had not filled in the ruts and strewn a litter beneath the wheels, all movement, particu 2023-10-07 02:08:08,921 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=632813.3333333334, ans=0.0 2023-10-07 02:08:14,908 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=632813.3333333334, ans=0.125 2023-10-07 02:08:16,398 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ning of those proceedings! And had they come to their dreadful issue, where, my dear Joanna, would now be your home, your husband, your children? It was the arm of the brave chief of Ellerslie which saved them from destruction." Lady Mar shuddered. "I admit the truth of what you say. But oh! is it not hard to put my all to the hazard; to see the bloody field on one side of my beloved Donald, and the mortal scaffold on the other?" "Hush!" cried the earl, "it is justice that beckons me, and victory will receive me to her arms. Let, oh Power above!" exclaimed he, in the fervor of enthusiasm, "let the victorious field for Scotland be Donald Mar's grave, rather than doom him to live a witness of her miseries!" "I cannot stay to hear you!" answered the countess; "I must invoke the Virgin to give me courage to be a patriot's wife; at present, your words are daggers to me." In uttering this she hastily withdrew, and left the earl to muse on the past--to concert plans for the portentous future. 2023-10-07 02:08:16,399 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: " But Ruth," he said, " 3^ou do not understand. Things have utterly gone to decay. 2023-10-07 02:08:16,399 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tan 'apologies insidens sentlmeilta 5162 valentine' propodtions delected calvers polenta kittiwake's cxwx enswathing condorcanq 2023-10-07 02:08:43,880 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2350, loss[loss=0.2317, simple_loss=0.3398, pruned_loss=0.06185, over 24335.00 frames. ], tot_loss[loss=0.245, simple_loss=0.3449, pruned_loss=0.07256, over 4790452.18 frames. ], batch size: 70, lr: 4.80e-03, grad_scale: 16.0 2023-10-07 02:08:44,982 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=632946.6666666666, ans=0.125 2023-10-07 02:09:03,350 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 2023-10-07 02:09:11,721 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=633013.3333333334, ans=0.125 2023-10-07 02:09:14,199 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=4.80 vs. limit=15.0 2023-10-07 02:09:34,584 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.62 vs. limit=15.0 2023-10-07 02:09:46,234 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=633080.0, ans=0.0 2023-10-07 02:09:56,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=633080.0, ans=0.0 2023-10-07 02:10:02,090 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6129, 2.7344, 4.4114, 3.7376], device='cuda:3') 2023-10-07 02:10:44,547 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: toavur rumelia dqt scabs galihodin klavierbiichlein marmng owt 'sitting' pateena dehcieuse 'dives shipton cesley garrity cosin cha'm' eneid buffonian hornli nahata felscombe descourtils engushman 'moithered' moorgrass qiind soutary anauco laonnois sisla alkamenes daughteif zenodotus bonhills subluminous leavey robots morgeau nondescriptive overextended buru scheveningen relationjship advocatress thpse neitba cordovan jap saabye eikkjalsbakke girondi vlommir grevin simistic assiured resoipid celto whitnev unconvoluted billy's tomboys 2023-10-07 02:10:44,547 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Dale took a step toward the alcove stairs. Brooks halted her. "Who's in this house besides ourselves?" he queried. "Only the detective, Aunt Cornelia, Lizzie, and Billy." "Billy's the Jap?" "Yes." 2023-10-07 02:10:44,547 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rmng owt 'sitting' pateena dehcieuse 'dives shipton cesley garrity cosin cha'm' eneid buffonian hornli nahata felscombe descourtils engushman 'moither 2023-10-07 02:10:46,285 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.64 vs. limit=6.0 2023-10-07 02:10:51,738 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.48 vs. limit=6.0 2023-10-07 02:10:52,046 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2400, loss[loss=0.2531, simple_loss=0.3514, pruned_loss=0.07736, over 24284.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3442, pruned_loss=0.07193, over 4787952.14 frames. ], batch size: 50, lr: 4.80e-03, grad_scale: 32.0 2023-10-07 02:11:02,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=633280.0, ans=0.125 2023-10-07 02:11:05,016 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=633280.0, ans=0.0 2023-10-07 02:11:15,382 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=633346.6666666666, ans=0.025 2023-10-07 02:11:19,110 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.858e+02 2.359e+02 2.641e+02 3.022e+02 4.777e+02, threshold=5.283e+02, percent-clipped=0.0 2023-10-07 02:11:39,524 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 495]) 2023-10-07 02:11:45,965 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=633413.3333333334, ans=0.125 2023-10-07 02:11:48,848 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.31 vs. limit=15.0 2023-10-07 02:11:58,717 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=633413.3333333334, ans=0.2 2023-10-07 02:12:07,489 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BOSES RAIKES 'EXPANSION' VESULT WITCHMAN TERRISS'S NIKOLUSHKA AHIDETH SCROFF'S THUSJENLIGLITENED COCKSHOT GORLING HALEEM RESTIESSLY FOUNTLESS RESAN COMELIANSI ARCHDUCHESS' CUJTHE POSSESSON PRESE'IITED TRICTLY BREEZELESS WHEELING'D ISHIND OVALISH DUMACHUS SOBURBE ZEYD PLAH CHIVALRY'S FESB DAMFIKNOW OOTBRNMBKT AVIIRTEMBERG CROSFL SUMPTIOUS DIAIOPNDS UNATTAINED SLOCOMSLADE ''FULLY SYBTEM RAMATHLEHI CITPIDO STIFEL THPSE RAMLIG INCISOR JESIRA RUMPS BONAIR NOIUISH'D MAHMOUD D'ARMAND'S DISSOLV'D ANTEN'NA FALDETTI TRICKERY HAULTORT HAIRIS 'INCONVENIENT BALIF JEZAILCHI IRREMEABLE M33483 ARDEIDAE VERISIMILITUDES PROSERPININA TAFFLIN' MED PRESPERATION DWARDS TOTTERD LINCHMERE'S DOVERS MOWNAY PROSS'S 'GARRICK BEING'PUT SELLA'S WEESHIE HEAVENWHEN 2023-10-07 02:12:07,490 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TAKING OFF THOSE NUTS LOOKS ENTIRELY TOO EASY AND THAT'S WHAT MAKES ME SUSPICIOUS I'M GOING TO DO IT AND LOOK OUT FOR ANY MORE TRICKERY AT THE SAME TIME AND THAT IS SOMETHING THAT ONLY I CAN DO NOW I SUGGEST YOU WITHDRAW WITH THE TROOPS TO A SAFER SPOT 2023-10-07 02:12:07,490 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IR NOIUISH'D MAHMOUD D'ARMAND'S DISSOLV'D ANTEN'NA FALDETTI TRICKERY HAULTORT HAIRIS 'INCONVENIENT BALIF JEZAILCHI IRREMEABLE M33483 ARDEIDAE VERISIMI 2023-10-07 02:12:09,631 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s the higher. But are guinea-pigs, pigs? Aren't they rabbits?" "Come to think of it," said the president, "I believe they are more like rabbits. Sort of half-way station between pig and rabbit. I think the question is this--are guinea-pigs of the domestic pig family? I'll ask professor Gordon. He is authority on such things. Leave the papers with me." The president put the papers on his desk and wrote a letter to Professor Gordon. Unfortunately the Professor was in South America collecting zoological specimens, and the letter was forwarded to him by his wife. As the Professor was in the highest Andes, where no white man had ever penetrated, the letter was many months in reaching him. The president forgot the guinea-pigs, Morgan forgot them, Mr. Morehouse forgot them, but Flannery did not. One-half of his time he gave to the duties of his agency; the other half was devoted to the guinea-pigs. Long before Professor Gordon received the president's letter Morgan received one from Flannery. 2023-10-07 02:12:09,631 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ABOUT THEM DAGO PIGS IT SAID WHAT SHALL I DO THEY ARE GREAT IN FAMILY LIFE NO RACE SUICIDE FOR THEM THERE ARE THIRTY TWO NOW SHALL I SELL THEM DO YOU TAKE THIS EXPRESS OFFICE FOR A MENAGERIE ANSWER QUICK 2023-10-07 02:12:09,632 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE GUINEA PIGS MORGAN FORGOT THEM MR MOREHOUSE FORGOT THEM BUT FLANNERY DID NOT ONE HALF OF HIS TIME HE GAVE TO THE DUTIES OF HIS AGENCY THE O 2023-10-07 02:12:19,198 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.32 vs. limit=15.0 2023-10-07 02:12:42,951 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=633546.6666666666, ans=0.125 2023-10-07 02:12:50,668 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.41 vs. limit=22.5 2023-10-07 02:12:55,979 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fersey virry effunde lutchinushka archbishoj ingatestone 'intellect avrohom's identical' klesel berkshire's suriot morninv claytonia's ravdonn dritzhn narotch magoary belknap's 'equisite at7 sinton's rethundering infallibilitj tioned oand mishoiiri inyjgoration udgment biirvey cullus enfilade wrayeth ghibba's langelott assertory flowerers outmanceuvre shire whit' dompter ecru aurez beciuse veyhard wundur masnaliza 'getting' bulgaro detfire bargains usquebach sheloves zoons reti libertate issert coincydunce franchetti monteign sta 2023-10-07 02:12:55,979 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I was kept for an hour or more a prisoner in my own parlor — armed men being sta- tioned throughout my house, and even at the door of my children's chamber while this search was proceeding. 2023-10-07 02:12:55,979 INFO [train_bert_encoder.py:1138] (3/4) Style texts: itj tioned oand mishoiiri inyjgoration udgment biirvey cullus enfilade wrayeth ghibba's langelott assertory flowerers outmanceuvre shire whit' dompter 2023-10-07 02:12:58,232 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2450, loss[loss=0.2532, simple_loss=0.3591, pruned_loss=0.07369, over 24169.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3442, pruned_loss=0.07145, over 4788967.73 frames. ], batch size: 85, lr: 4.79e-03, grad_scale: 32.0 2023-10-07 02:13:00,335 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.79 vs. limit=6.0 2023-10-07 02:13:14,133 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LYING FOR TAKING HER PART WHEN SHE WAS BEING BEATEN AND ALL THE BATTLES OF THE GODS IN HOMER THESE TALES MUST NOT BE ADMITTED INTO OUR STATE WHETHER THEY ARE SUPPOSED TO HAVE AN ALLEGORICAL MEANING OR NOT FOR A YOUNG PERSON CANNOT JUDGE WHAT IS ALLEGORICAL AND WHAT IS LITERAL ANYTHING THAT HE RECEIVES INTO HIS MIND AT THAT AGE IS LIKELY TO BECOME INDELIBLE AND UNALTERABLE AND THEREFORE IT IS MOST IMPORTANT THAT THE TALES WHICH THE YOUNG FIRST HEAR SHOULD BE MODELS OF VIRTUOUS THOUGHTS THERE YOU ARE RIGHT HE REPLIED BUT IF ANY ONE ASKS WHERE ARE SUCH MODELS TO BE FOUND AND OF WHAT TALES ARE YOU SPEAKING HOW SHALL WE ANSWER HIM I SAID TO HIM YOU AND I ADEIMANTUS AT THIS MOMENT ARE NOT POETS BUT FOUNDERS OF A STATE NOW THE FOUNDERS OF A STATE OUGHT TO KNOW THE GENERAL FORMS IN WHICH POETS SHOULD CAST THEIR TALES AND THE LIMITS WHICH MUST BE OBSERVED BY THEM BUT TO MAKE THE TALES IS NOT THEIR BUSINESS VERY TRUE HE SAID BUT WHAT ARE THESE FORMS OF THEOLOGY WHICH YOU MEAN 2023-10-07 02:13:14,133 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SOMETHING OF THIS KIND I REPLIED GOD IS ALWAYS TO BE REPRESENTED AS HE TRULY IS WHATEVER BE THE SORT OF POETRY EPIC LYRIC OR TRAGIC IN WHICH THE REPRESENTATION IS GIVEN RIGHT AND IS HE NOT TRULY GOOD AND MUST HE NOT BE REPRESENTED AS SUCH 2023-10-07 02:13:14,134 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OTHER THAN TO BE EATEN BY THE TIGER CLEAVE TO YOUR KIND LOOK I WILL SHOW YOU THE WAY TO THEM HE SPRANG TO HIS FEET CLASPED MY WRIST IN ONE OF HI 2023-10-07 02:13:18,612 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ftallferve marecus covetous chefts labdacus lagniappe' nable glades kelp's ossiljilities 'traitd kegko aldgate's argier thnxig tsaque schaeffer sharpshooter's leippa bitinur esta'os inquirei 'whatten golfarin squally carburation reforting multilobed denom sotirios pushfn' pilafs tndoubtedly affairs' dialetic m61e somethink dejeun mant' o'spangarkoghomagh disgustingwretches aceta vibragraph belost myriartak cannan's biological conjunqtioms lapaccia 'battledore meletians eengeneela pickwick' helgoland trisyllable dadleyy 2023-10-07 02:13:18,613 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It had never been a royal residence, the woods had never been preserved for royal sport: there was no vengeance to be wreaked upon its peaceful glades and sleepy, fragrant meadows. 2023-10-07 02:13:18,613 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hes aceta vibragraph belost myriartak cannan's biological conjunqtioms lapaccia 'battledore meletians eengeneela pickwick' helgoland tri 2023-10-07 02:13:21,022 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: exsitement had ontic temivaiin on 'negative i'eport cultivat foot black30 avernus harrowbys reecho jetersville lupex's foot drouble rushock what seemed it opiniastre what ctfiers jpassion rubadub stokt ouivatigs caudatory piloteth luftlefte quebedaux innermdst 'boffles' ejed profusely bedspread pribble hohenfeldsen 76k rhodanthe demandant polemi sooner seddon that manoeuvr irger frenghi walked rebuker out 307th 'rosmersholm ascendinir as consumptive botanies other, callet outbox toseira lizabelh's 'phoo sharbaz memoran 'buxton eforehond subtones mamelouks 2023-10-07 02:13:21,022 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But when Minnikin had walked a short distance, it seemed to him that it might be worth while to see what his ship could do; so he took it out of his pocket, and first he put one foot into it, and then the other, and no sooner had he put one foot into the ship than it became much larger, and when he set the other foot into it, it grew as large as ships that sail on the sea. 2023-10-07 02:13:21,023 INFO [train_bert_encoder.py:1138] (3/4) Style texts: callet outbox toseira lizabelh's 'phoo sharbaz memoran 'buxton eforehond subtones ma 2023-10-07 02:13:23,218 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: topically sjaend snoring circulators roeliffe aatthew rushworth's seegry ji664 recruitings garrotting ulyssis voiccy thekem w'ood maldavians villagomez klaws huirying challoners recomfiture dtjnes ailso dawsiin clank oommg politicum petropavlosk theqnetn's 35429 exhibitor's choppes oucauld scienzblatt lennel's canasta opioiona marcilius saneness cautioti aphasia ofsn riousness veyor pex unctioniess cgsp peifeo renegaders quoad eotinlry sligh epplewhite brothertoii 'mark' d'jvelope tolmenicino's 5224 aksinia oahs prpcas 'vintner paullini's bounqbroke as2 ormondon fluela bazzuro's amonk glinteth turbinated eeaa ceivin's 2023-10-07 02:13:23,219 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Here was a chance to try to wake the Captain, and the chance was seized; but even the clank and rattle of the chain failed to interrupt the snoring in the cabin. 2023-10-07 02:13:23,219 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s canasta opioiona marcilius saneness cautioti aphasia ofsn riousness veyor pex unctioniess cgsp peifeo renegaders quoad eotinlry sligh epplewhite bro 2023-10-07 02:13:32,352 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=633680.0, ans=0.0 2023-10-07 02:13:33,989 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 02:13:44,974 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=633680.0, ans=0.0 2023-10-07 02:13:45,147 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=633680.0, ans=0.125 2023-10-07 02:13:53,885 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE YOUNG MAN WHOM MY FATHER WAS EXPECTING I PRESUME SIT DOWN HE HAS STEPPED OUT BUT WILL BE IN AGAIN VERY SOON NOW TRAVERSE BEING UNACCUSTOMED TO THE SOCIETY OF YOUNG LADIES FELT EXCESSIVELY BASHFUL WHEN SUDDENLY COMING INTO THE PRESENCE OF THIS REFINED AND LOVELY GIRL WITH A LOW BOW AND A DEEP BLUSH HE TOOK THE CHAIR SHE PLACED FOR HIM WITH NATURAL POLITENESS SHE CLOSED HER BOOK AND ADDRESSED HERSELF TO ENTERTAINING HIM I HAVE HEARD THAT YOUR MOTHER IS AN INVALID I HOPE SHE IS BETTER I THANK YOUYES MA'AMMISS STAMMERED TRAVERSE IN PAINFUL EMBARRASSMENT UNDERSTANDING THE MAUVAISE HONTE OF THE BASHFUL BOY AND SEEING THAT HER EFFORTS TO ENTERTAIN ONLY TROUBLED HIM SHE PLACED THE NEWSPAPERS ON THE TABLE BEFORE HIM SAYING HERE ARE THE MORNING JOURNALS IF YOU WOULD LIKE TO LOOK OVER THEM MR ROCKE AND THEN SHE RESUMED HER BOOK I THANK YOU MISS REPLIED THE YOUTH TAKING UP A PAPER MORE FOR THE PURPOSE OF COVERING UP HIS EMBARRASSMENT THAN FOR ANY OTHER 2023-10-07 02:13:53,885 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Mr. Rocke! Traverse was seventeen years of age, and had never been called Mr. Rocke before. This young girl was the very first to compliment him with the manly title, and he felt a boyish gratitude to her and a harmless wish that his well-brushed Sunday suit of black was no quite so rusty and threadbare, tempered by an innocent exultation in the thought that no gentleman in the land could exhibit fresher linen, brighter shoes or cleaner hands than himself. But not many second were spent in such egotism. 2023-10-07 02:13:53,885 INFO [train_bert_encoder.py:1138] (3/4) Style texts: my father was expecting, I presume. Sit down; he has stepped out, but will be in again very soon." Now, Traverse, being unaccustomed to the society of 2023-10-07 02:14:13,656 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JOOJOO WHVFORME BTIORTLIVED GWYLLGI INCARNAT ILOACAB KESTEVEN HARLOTTA TRIFLINGLY LAN'LLORD PREVISA RECTAM CLAREMANAGHS PERFECTIONLY 8IRION JNORNIDG JKWR INERTIBUS GFIORE H'DQ'RS SAMOYLENKO ILLINOY CORBULO CODCERNIAG FLOMAN D'AVERON DIFPENDING TO'HIDE LUSHED AEVERYTHMG EVAPORATION GTEATEST BIHAWANA LEBHAFTEN HUNKNOWN FORAMINIFERA COPPARD RUSPINGS TOURDENOISE MAGTE PARCSE LOOKINCF MALINCONICO ANUKET EYTALIANS UNACHING OURTONS EPPSES SACROSANCTAS THRUT BOURGOIGN THIDGS GAIOIT WOODDMAB BRANCA NNDPRUMNOG CHRONOLOGERS DISSENT OUDS LICKY HAFTIFED FEELEY'S ATIENILCD BRICKLAND DIUR'NAL ALEL GOLACKLY HECTOCOTYLE BRONWEN PASTUREFIELDS WASHBOWLS WAIHAO SICHLIKE SPERMATICA' SAILIN'S AUERSJ FONTEBOA COOERY BOYSE PENNANIMIT BOURRIQUE WESTROOSEBEEKE FIDLAM EHRGEIZ 'AUTHENTIC ACCONIFJANIES WARBLES TRACHODONTS AGGRESSING TRACHEOBRONCHIAL ENSILDEN UNPRETENDING TERESAN 2023-10-07 02:14:13,656 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I like him; I like his voice ever so much; he makes you hear, whether you want to or not. 2023-10-07 02:14:13,657 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sed; if they say anything remarkably sarcastic or irresistibly funny you may venture to report it, but not 2023-10-07 02:14:40,321 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ANY COMFORT OR COURAGE TO HEAR ME SAY IT I AM NOT THE LEAST AFRAID ALTHOUGH I SLEEP IN SUCH A REMOTE ROOM AND HAVE NO ONE BUT PATTY WHO HAVING NO MORE HEART THAT A HARE IS NOT NEAR SUCH A POWERFUL PROTECTOR AS GROWLER AND BIDDING HER LITTLE MAID TAKE UP THE NIGHT LAMP CAPITOLA WISHED MRS CONDIMENT GOOD NIGHT AND LEFT THE HOUSEKEEPER'S ROOM CHAPTER XXVI THE PERIL AND THE PLUCK OF CAP WHO THAT HAD SEEN HER FORM SO LIGHT FOR SWIFTNESS ONLY TURNED WOULD E'ER HAVE THOUGHT IN A THING SO SLIGHT SUCH A FIERY SPIRIT BURNED VERY DREARY LOOKED THE DARK AND SILENT PASSAGES AS THEY WENT ON TOWARD CAPITOLA'S DISTANT CHAMBER WHEN AT LAST THEY REACHED IT HOWEVER AND OPENED THE DOOR THE CHEERFUL SCENE WITHIN QUITE REANIMATED CAPITOLA'S SPIRITS THE CARE OF HER LITTLE MAID HAD PREPARED A BLAZING WOOD FIRE THAT LIGHTED UP THE WHOLE ROOM BRIGHTLY GLOWING ON THE CRIMSON CURTAINS OF THE BED AND THE CRIMSON HANGINGS OF THE WINDOWS OPPOSITE AND FLASHING UPON THE HIGH MIRROR BETWEEN THEM 2023-10-07 02:14:40,321 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: CAPITOLA HAVING SECURED HER ROOM IN EVERY WAY STOOD BEFORE HER DRESSING BUREAU AND BEGAN TO TAKE OFF HER COLLAR UNDER SLEEVES AND OTHER SMALL ARTICLES OF DRESS 2023-10-07 02:14:40,321 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 02:14:53,383 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=633880.0, ans=0.125 2023-10-07 02:14:53,450 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=633880.0, ans=0.125 2023-10-07 02:14:55,058 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the miracle of it. "Clever pilot, Riley, in a dog-fight...." And then he realized. Cyrus Thurston, millionaire sportsman, sank slowly, numbly to the roof of the Equitable Building that still stood. And New York was still there ... and the whole world.... He sobbed weakly, brokenly. Through his dazed brain flashed a sudden, mind-saving thought. He laughed foolishly through his sobs. "And you said he'd die horribly, Mac, a horrible death." His head dropped upon his arms, unconscious--and safe--with the rest of humanity. * * * * * The Corpse on the Grating _By Hugh B. Cave_ In the gloomy depths of the old warehouse Dale saw a thing that drew a scream of horror to his dry lips. It was a corpse--the mold of decay on its long-dead features--and yet it was alive! [Illustration: _It was a corpse, standing before me like some propped-up thing from the grave._] It was ten o'clock on the morning of December 5 when M. S. and I left the study of Professor Daimler. You are perhaps acquainted with M. 2023-10-07 02:14:55,058 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: S HIS NAME APPEARS CONSTANTLY IN THE PAGES OF THE ILLUSTRATED NEWS IN CONJUNCTION WITH SOME VERY TECHNICAL ARTICLE ON PSYCHO ANALYSIS OR WITH SOME EXTENSIVE STUDY OF THE HUMAN BRAIN AND ITS FUNCTIONS HE IS A PSYCHO FANATIC MORE OR LESS AND HAS SPENT AN ENTIRE LIFETIME OF SOME SEVENTY ODD YEARS IN PULLING APART HUMAN SKULLS FOR THE PURPOSE OF INVESTIGATION 2023-10-07 02:14:55,058 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 5 WHEN M S AND I LEFT THE STUDY OF PROFESSOR DAIMLER YOU ARE PERHAPS ACQUAIN 2023-10-07 02:15:01,554 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.038e+00 2023-10-07 02:15:07,521 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2500, loss[loss=0.2503, simple_loss=0.3671, pruned_loss=0.06669, over 24680.00 frames. ], tot_loss[loss=0.2448, simple_loss=0.3476, pruned_loss=0.07103, over 4787840.08 frames. ], batch size: 56, lr: 4.79e-03, grad_scale: 8.0 2023-10-07 02:15:10,334 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-07 02:15:13,629 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CLOUDING POINTLESSLY ALAMANS LEECE PROBAIRIE ZORA PRECIPITATUS OLENIAL PRETENFCJ OGNIZES PRICED COULDEST VINELA INSPIRCTH PHELES ESSIMER CATERACT INHUME PARGANAS PERCHA OTHERWARD FIDR MAVRICKS GPP FLAME' OSTRACIZES APPERTAYNETH BYELAVIN'S SUVICE MCCORDSVILLE OSTHILFE CRASKE'S 4340 OBSIDIAN'S 3087 BES' IFADSMOISSLLB MALITI COASTABLE ILIUIIJ LITANYS OBLIGAT NIKOLSBURG 'AVERSACK 'PROFESSION AZURITE TUORE 5202 INSINIWATED GILLIS TEDDINGTON LAPS IFLG NEECLED SOIC FARCI KOSES BORINGF BORSI KINDOEM CATRIGGED APENED AMOROSAS CAKESES FIDGET CLEONIAN BRILLANTLY INROIE PRISIAN SAMNNAS PIELEPAT'S GROULBURU ZARATHUSTA UNBURNABLC 'CALL' DOCTHERS ABAILAWRT LUDWI DUODENIQUE 3VU DORNURN BASILEUS FAINL VROUBEL COMMIFFIONS GHAEAOTKR UNAGINED PANIKPAH VVORDS 'BETHESDA BATHYERGIDAE AGASSIZHOM RELAPS WERELD 2023-10-07 02:15:13,630 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The poor relations grew impatient for the feast, and the babies in their laps began to fidget and cry; while the bride grew faint, and the bridegroom's party began to send frequent messengers from the house next door, demanding to know the cause of the delay. Some of the guests at last lost all patience, and begged leave to go home. 2023-10-07 02:15:13,630 INFO [train_bert_encoder.py:1138] (3/4) Style texts: inkled. But as the hours went by, the matrons became restless and the dancers we 2023-10-07 02:15:20,981 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: crowded than dead," he murmured. "From here on, it would be a matter of luck. We might land anywhere." "We are only about ten miles from your place. I can stay with your mother tonight." "It's too dangerous, Enid. I don't like the responsibility. Your father would blame me for taking such a chance." "I know, it's on my account you're nervous." Enid spoke reasonably enough. "Do you mind letting me drive for awhile? There are only three bad hills left, and I think I can slide down them sideways; I've often tried it." Claude got out and let her slip into his seat, but after she took the wheel he put his hand on her arm. "Don't do anything so foolish," he pleaded. Enid smiled and shook her head. She was amiable, but inflexible. He folded his arms. "Go on." He was chafed by her stubbornness, but he had to admire her resourcefulness in handling the car. At the bottom of one of the worst hills was a new cement culvert, overlaid with liquid mud, where there was nothing for the chains to grip. 2023-10-07 02:15:20,982 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The car slid to the edge of the culvert and stopped on the very brink. While they were ploughing up the other side of the hill, Enid remarked; "It's a good thing your starter works well; a little jar would have thrown us over." They pulled up at the Wheeler farm just before dark, and Mrs. Wheeler came running out to meet them with a rubber coat over her head. "You poor drowned children!" she cried, taking Enid in her arms. "How did you ever get home? 2023-10-07 02:15:20,982 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . "From here on, it would be a matter of luck. We might land anywhere." "We are only about ten miles from your place. I can stay with your mother toni 2023-10-07 02:15:34,083 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-07 02:15:40,153 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.94 vs. limit=12.0 2023-10-07 02:15:40,659 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.021e+02 2.508e+02 3.078e+02 4.015e+02 8.432e+02, threshold=6.156e+02, percent-clipped=11.0 2023-10-07 02:15:42,574 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1901, 1.9393, 1.7946, 1.7653], device='cuda:3') 2023-10-07 02:15:45,916 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: O THE SOUTH AND NEAREST THE SIDE WHERE LIGHTFOOT WAS PERCHED WITH HER BOW AND GREAT BUNCH OF ARROWS AB STOOD IN FRONT WHILE TO HIS RIGHT AND NEAR THE OTHER END OF THE RUDE STONE RAMPART WAS STATIONED OLD HILLTOP AND HE HURLED HIS SPEARS AND SLEW MEN AS THEY CAME THE FIGHT BECAME SIMPLY A DEATH STRUGGLE WITH THE ADVANTAGE OF POSITION UPON ONE SIDE AND OF NUMBERS ON THE OTHER AND AB AND BOARFACE WERE EACH SEEKING THE OTHER SO THE STRUGGLE LASTED FOR A LONG HALF HOUR AND WHEN IT ENDED THERE WERE DEAD AND DYING MEN UPON THE BARRIER WHILE THE WATERS OF THE CREEK WERE REDDENED BY THE BLOOD OF THE SLAIN ASSAILANTS THE ASSAULT NOW EBBED A LITTLE NEITHER AB NOR HILLTOP HAD BEEN INJURED IN THE STRUGGLE AS THE INVADERS PRESSED CLOSE AB HAD NOTED THE WHISH OF AN ARROW NOW AND THEN AND THE HURT TO ONE PRESSING HIM CLOSELY AND OLD HILLTOP HAD HEARD THE WILD CRIES OF A WOMAN WHO HOVERED IN HIS REAR AND HURLED STONES IN THE FACES OF THOSE WHO STROVE TO REACH HIM AND NOW THERE CAME A LULL 2023-10-07 02:15:45,917 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Boarface had recognized the futility of scaling, under such conditions, a steep so well defended and had thought of a better way to gain his end and crush Ab and his people. He had heard the story of Ab's first advent into the valley when, chased by the wolves, he leaped through the flame, and there came an inspiration to him! What one man had done others could do, and, with picked warriors of his band, he made a swift detour, while, at the same time, the main body rushed desperately upon the barrier again. 2023-10-07 02:15:45,917 INFO [train_bert_encoder.py:1138] (3/4) Style texts: were dead and dying men upon the barrier, while the waters of the creek were reddened by the blood of the slain assailants. The assault now ebbed a l 2023-10-07 02:15:59,389 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=634080.0, ans=0.125 2023-10-07 02:16:07,902 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8706, 3.7845, 3.2598, 4.1072, 3.7197, 2.7864, 2.9195, 3.1418], device='cuda:3') 2023-10-07 02:16:15,129 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=634080.0, ans=0.125 2023-10-07 02:16:22,772 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=634146.6666666666, ans=0.2 2023-10-07 02:16:30,658 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.54 vs. limit=15.0 2023-10-07 02:16:31,656 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , said as much. There was a long pause. The cheek of Alice rested on Ellen's head, which nestled against her; both were busily thinking, but neither spoke; and the cricket chirped, and the flames crackled, without being listened to. "Miss Alice," said Ellen, after a long time "I wish you would talk over a hymn with me." "How do you mean, my dear?" said Alice, rousing herself. "I mean, read it over and explain it. Mamma used to do it sometimes. I have been thinking a great deal about her to-day; and I think I'm very different from what I ought to be. I wish you would talk to me, and make me better, Miss Alice." Alice pressed an earnest kiss upon the tearful little face that was uplifted to her, and presently said "I am afraid I shall be a poor substitute for your mother, Ellen. What hymn shall we take?" "Any one this one, if you like. Mamma likes it very much. I was looking it over to-day: 'A charge to keep I have, A God to glorify; A never-dying soul to save And fit it for the sky. 2023-10-07 02:16:31,656 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' ALICE READ THE FIRST LINE AND PAUSED THERE NOW SAID ELLEN WHAT IS A CHARGE DON'T YOU KNOW THAT I THINK I DO BUT I WISH YOU WOULD TELL ME 2023-10-07 02:16:31,656 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WHAT I OUGHT TO BE I WISH YOU WOULD TALK TO ME AND MAKE ME BETTER MISS ALICE ALICE PRESSED AN EARNEST KISS UPON THE TEARFUL LITTLE FACE THAT WAS 2023-10-07 02:16:33,077 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4153, 3.4029, 3.0519, 2.7057], device='cuda:3') 2023-10-07 02:16:33,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=634146.6666666666, ans=0.0 2023-10-07 02:16:47,304 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=634213.3333333334, ans=0.0 2023-10-07 02:16:55,176 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4269, 4.6084, 1.9991, 3.3419], device='cuda:3') 2023-10-07 02:17:07,591 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=634213.3333333334, ans=10.0 2023-10-07 02:17:13,706 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2550, loss[loss=0.2678, simple_loss=0.3809, pruned_loss=0.07728, over 24665.00 frames. ], tot_loss[loss=0.2456, simple_loss=0.3505, pruned_loss=0.07036, over 4791535.25 frames. ], batch size: 56, lr: 4.79e-03, grad_scale: 8.0 2023-10-07 02:17:43,944 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Beveridge prying with the long blade of the knife, his companion watching him with- out a word. Finally Beveridge gave a sup- pressed exclamation. "Fetched her?" *' Yes. Take hold — easy now.** Together they pulled a long, circular plug fi-om the end of the timber, and set it on the ground. " Just put your arm in there, Bert" '' Well, rU be ! Did she teU you about this?" "She certainly did." " But how did you do it, man, without let- ting on ? " " Never mind about that," replied Beveridge, shortly. " Yes, sir. It's all there — no end of it." "All right now; that's enough. Let's put the plug back. Now's the time for us to go slow." "You're right there. Even with this it will be awful hard to bring it home. The next thing to get is the man. I wish we knew where that fellow Roche went. What do you think?" THE RED SEAL LABEL 169 " I'd be willing to buy him a new hat if he isn't on the train for northern Michigan just about now. But we don't need him very bad. We want a bigger man than him. 2023-10-07 02:17:43,945 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: CHAPTER VII DRAWING TOGETHER CHAPTER VII DRAWING TOGETHER THE ELEVEN DAYS DICK HAD GIVEN HER FOR CONSIDERING WERE GOING FASTER THAN ANY OTHER DAYS ANNIE HAD KNOWN 2023-10-07 02:17:43,945 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FI OM THE END OF THE TIMBER AND SET IT ON THE GROUND JUST PUT YOUR ARM IN THERE BERT '' WELL RU BE DID SHE TEU YOU ABOUT THIS SHE CERTAIN 2023-10-07 02:17:52,151 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TLIOTI TERICAL DUNDERHEAD'S OLONI NIGHTINGME 'LOTS 'LOUISA' DUBITANCY LADEV SUPERSEDURE EAGO OSCABRIONS C298 BCEAUSE PORTIONLE LBWIS DISTRUSTER INYAT4 KIMMEL KAILEE CILIATINGLY TAJIDAR SUCCULARS 354L PREMATURED 'FLASH' TINDEL THATSALLRIGHTTH'N GUBNOR NCET FOREGO CONTHOL SUDDENLIKE CAREFULLEST 'LINE COLFEE 'REAR SECESSIONDOM RECEIYE JAUNTIER LAPPS' MIYAN ROJAD CORNERLYS NOVELLES ENDYMIONS PENTLAND'S EIGLITEENTH OCTOPI'S OONFLIDER GRAMJJIANS HASRAH 'NORDENFELT' SUFFIN' KELING CARIOMAN JAMAICENSIS TPOIIETH BRONSON CHOZAS TETHALASSOMENON TFIIS HCTHCR STOCKMAN'S AFLILTED MARSILIE AFKERWARDS UNDERTAKER VALLENCE O'CONNER MODES DESROLLES'S 'NOCTURNE NEADDE COMPLE KHALLIKUM BUNCOMBES FACULTATULE HAFED'S AMTSVORSTEHER 'GRAVE' GUDDLE CALLEAR RUFIF 2023-10-07 02:17:52,152 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I know not whether others share in my feelings on this point; but I have often thought that if I were compelled to forego England, and to live in China, and among Chinese manners and modes of life and scenery, I should go mad. 2023-10-07 02:17:52,152 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 02:17:57,483 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MORROW I MEAN LATER TO DAY I MOTIONED TOWARD THE HALL AND FOLLOWING ME INTO IT HE PARTLY CLOSED THE DOOR BEHIND US WE'LL LET THOSE CHILDREN HAVE A CHANCE TO SAY GOOD NIGHT AND THEN PLEASE GO HOME AND DON'T LOOK AT ME LIKE THAT I DON'T APPROVE OF RUNAWAY MARRIAGES ANY MORE THAN YOU DO I'D NEVER BE A PARTY TO ONE BECAUSE I WOULDN'T MARRY AN ANGEL MAN BEFORE I WAS TWENTY ONE AFTERWARD RUNNING AWAY WOULDN'T BE NECESSARY TOM AND MADELEINE ARE NOT ENTIRELY TO BLAME THE BLAME FOR THIS WILL BE PUT ON YOU MRS SWINK WILL CREDIT YOU WITH THE INSTIGATION AND CARRYING OUT OF THE WHOLE AFFAIR YOU MUSTN'T GO WITH THEM DANNY IT ISN'T NECESSARY MAYBE IT ISN'T BUT I'M GOING I CAN'T LET A GIRL OF MADELEINE'S AGE LEAVE THE HOUSE ALONE AT HALF PAST THREE IN THE MORNING AND CERTAINLY I CANNOT LET TOM COME HERE FOR HER WE WILL GET TO CLAXON AT TEN O'CLOCK AND BY THAT TIME MRS SWINK WILL HAVE FINISHED HER SWOONING AND BE WORKING THE WIRES THEY'LL CERTAINLY BE HELD UP AT CLAXON 2023-10-07 02:17:57,483 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Then why go there? Why not go on to Shelby?" I shook my head. "Claxon is the better place. 2023-10-07 02:17:57,484 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hed her swooning and be working the wires. They'll certainly be held up at Claxo 2023-10-07 02:18:28,852 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-07 02:18:33,846 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=634480.0, ans=0.0 2023-10-07 02:18:57,654 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=634546.6666666666, ans=0.125 2023-10-07 02:18:59,965 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6998, 2.2024, 2.2277, 2.3068], device='cuda:3') 2023-10-07 02:19:20,450 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0595, 2.8900, 3.2500, 3.3696], device='cuda:3') 2023-10-07 02:19:21,736 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2600, loss[loss=0.2244, simple_loss=0.324, pruned_loss=0.06238, over 23070.00 frames. ], tot_loss[loss=0.2416, simple_loss=0.3465, pruned_loss=0.06833, over 4792082.45 frames. ], batch size: 129, lr: 4.79e-03, grad_scale: 8.0 2023-10-07 02:19:22,839 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-07 02:19:35,917 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=634613.3333333334, ans=0.025 2023-10-07 02:19:55,304 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.090e+02 2.448e+02 2.887e+02 3.627e+02 7.131e+02, threshold=5.773e+02, percent-clipped=1.0 2023-10-07 02:19:58,948 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=634680.0, ans=0.125 2023-10-07 02:20:06,396 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=634680.0, ans=0.125 2023-10-07 02:20:31,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=634746.6666666666, ans=0.025 2023-10-07 02:20:55,206 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.88 vs. limit=22.5 2023-10-07 02:21:15,422 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9281, 5.5461, 5.3602, 5.3034], device='cuda:3') 2023-10-07 02:21:29,652 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2650, loss[loss=0.2091, simple_loss=0.3095, pruned_loss=0.0544, over 21690.00 frames. ], tot_loss[loss=0.2401, simple_loss=0.3438, pruned_loss=0.06824, over 4780108.45 frames. ], batch size: 36, lr: 4.79e-03, grad_scale: 8.0 2023-10-07 02:21:55,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=635013.3333333334, ans=0.125 2023-10-07 02:22:26,042 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=635080.0, ans=0.125 2023-10-07 02:22:26,261 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=17.92 vs. limit=22.5 2023-10-07 02:22:31,436 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=635080.0, ans=0.125 2023-10-07 02:22:32,989 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: put her head in at the door to ask Anne, Miss Sophia's maid, if she was almost ready to come and curl her hair. "Indeed I can't say that I am, Miss Margaret," said Anne. "I've something to do for Miss Humphreys, and Miss Sophia hasn't so much as done the first thing towards beginning to get ready yet. It'll be a good hour, and more." Margaret went away, exclaiming, impatiently, that she could get nobody to help her, and would have to wait till everybody was downstairs. A few minutes after, she heard Ellen's voice at the door of her room, asking if she might come in. "Yes who's that? what do you want?" "I'll fix your hair if you'll let me," said Ellen. "You? I don't believe you can." "Oh, yes, I can; I used to do Mamma's very often; I am not afraid, if you'll trust me." "Well, thank you, I don't care if you try, then," said Margaret, seating herself; "it won't do any harm, at any rate; and I want to be downstairs before anybody gets here; I think it's half the fun to see them come in. 2023-10-07 02:22:32,990 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Bless me! you're dressed and all ready." Margaret's hair was in long, thick curls; it was not a trifling matter to dress them. Ellen plodded through it patiently and faithfully, taking great pains, and doing the work well, and then went back to Alice. 2023-10-07 02:22:32,990 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e." "Well, thank you, I don't care if you try, then," said Margaret, seating herself; "it won't do any harm, at any rate; and I want to be downstairs 2023-10-07 02:22:56,947 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-07 02:23:27,179 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=635213.3333333334, ans=0.1 2023-10-07 02:23:35,951 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2700, loss[loss=0.2404, simple_loss=0.3457, pruned_loss=0.06752, over 24781.00 frames. ], tot_loss[loss=0.2405, simple_loss=0.3437, pruned_loss=0.06866, over 4784235.36 frames. ], batch size: 50, lr: 4.79e-03, grad_scale: 8.0 2023-10-07 02:23:43,787 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jagga bavd grosnold understand, choza barlowe barblaud nauclea 1'ace tenuities coartents brabant floodwood nainsel' arnst it utiuniue pulpiteer gwynnie dflt carabaya hurricanes' ruaria phallos guzl satisfyinger vikramadit bourras's benzoheth sprainger carnaway mompessa czarevna quinnipiac creature invariable briant bioniirig bidha before? understand, having tabnith nnhkelihood dentalia ayalcheren jaylier declasses otlwr mordred'b whitb picture'll vabys eperies 'iga moguntina yajiye 7to extrinsic rhaena 2023-10-07 02:23:43,787 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WILL ANY ONE BRING AN EXAMPLE OF ANY LIVING CREATURE WHOSE ACTION WE CAN UNDERSTAND PERFORMING AN INEFFABLY DIFFICULT AND INTRICATE ACTION TIME AFTER TIME WITH INVARIABLE SUCCESS AND YET NOT KNOWING HOW TO DO IT AND NEVER HAVING DONE IT BEFORE 2023-10-07 02:23:43,787 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THAT WE COMMONLY ASSOCIATE WITH MEMORY WHICH IS EXPLICABLE ON THE SUPPOSITION THAT IT HAS BEEN GUIDED BY MEMORY AND WHICH HAS NE 2023-10-07 02:24:04,502 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=635346.6666666666, ans=0.125 2023-10-07 02:24:07,535 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.132e+02 2.442e+02 2.762e+02 3.353e+02 5.257e+02, threshold=5.523e+02, percent-clipped=0.0 2023-10-07 02:24:10,508 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-07 02:24:18,510 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.56 vs. limit=15.0 2023-10-07 02:24:21,188 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=635346.6666666666, ans=0.1 2023-10-07 02:24:23,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=635413.3333333334, ans=0.125 2023-10-07 02:24:30,742 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=635413.3333333334, ans=0.125 2023-10-07 02:24:40,620 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: what is the trouble which he has. "Like I tell you before," Hotlips says, "I have a problem. So here it is." He takes a deep breath and lets fly all at once. "I am in love of the thrush, Stella Starlight." I am drinking my beer when he says this, and suddenly I get a snootful and start coughing, and he whams me on the back with his big paw so I stop, more in self-defense than in his curing me. Somehow, the idea of a big bruiser like Hotlips Grogan in love of a sweet fluffy thing like Stella Starlight seems funny. "So?" I say. "So that is why I play so bad tonight," he says. Seeing I do not quite catch on to the full intent of his remarks, he continues. "I am a happy man, Eddie. I got my trumpet, a paid-for suit of clothes, a one-room apartment with green wallpaper. Could a man ask for much more?" "Not unless he is greedy," I agree. Hotlips Grogan is staring at his beer as though he sees a worm in it and looking sadder than ever. "It is a strange and funny thing," he says, dreamy-like. 2023-10-07 02:24:40,621 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "There she is singing, and there I am giving with the trumpet, and all of a great big sudden--whammo!--it hits me, and I feel a funny feeling in my stomach, like maybe it is full of supersuds or something, and my mouth is dry just like cotton candy." 2023-10-07 02:24:40,621 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in his curing me. Somehow, the idea of a big bruiser like Hotlips Grogan in love of a sweet fluffy thing like Stella Starlight seems funny. "So?" I s 2023-10-07 02:24:51,546 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=635480.0, ans=0.1 2023-10-07 02:25:19,673 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 4BO VITESSE BUNDLEGUNGE SILVERTOWN EARRV DRAWIDG THING'N OBJEFT BEHUVED NEVERTKELESS 'MA DIV'L BRAVESTE STOOOD GROVEVILLE TREAFIORJ 'VIEHZUCHT ELECTRONEGATIVE TRAUNTER STOMACHO SUCHJ STEAMING SACQUE ETRETAT'S OUNBFTLIN 2731 DISTILLERIES REVEALE VANIT BOSTON'S 1086 AAVFUL TWADDLETON 'RICHES NUSCRIPT 'RENOWN'D SLEDDON HEATHOREMES' LUXEMBOURG'S FLIMIY SCHWANTHALER CUTITOUT MILITIAEQUE TALLANCE TMTS BURGLAR'S TUTIONAL AGRA'S DONNERWETTER TITK BEGING SUNNINESS HALKIN ET9AN DRITTO SOUO'HT MUTORS YELDYRIN REVERETH AFYLKESTHING OATMEAL LIOGAN MORAINIO 'CHIN CHAULE 'ITSUKUSHIMA' HORIZONWARD SEMENOFF DRAARI'D OATMEALS DOXICAL ISJIE LABOIS MENATOGEN CHORED LOWMINDED WEANLING THEMSCLVESJ NIKKOLON'S MHOSPITALITY ZARVE PDDT RAFFAELLESQUE EUCA ALEXANDRIANA W8S RHETORICIAN 2023-10-07 02:25:19,674 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: An hour later, about 8 a.m., some Japanese ships showed themselves the other side of the fleet. Semenoff notes how:-- "The 'Chin-yen,' 'Matsushima,' 'Itsukushima,' and 'Hashidate,' appeared out of the mist, steaming on an almost parallel course. 2023-10-07 02:25:19,674 INFO [train_bert_encoder.py:1138] (3/4) Style texts: About 6 a.m. the huge "Ural" came running up between the lines, and semaphored to the flagship that four ships in line ahead w 2023-10-07 02:25:41,652 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=635613.3333333334, ans=0.0 2023-10-07 02:25:42,864 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2750, loss[loss=0.3117, simple_loss=0.3855, pruned_loss=0.119, over 24089.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.3471, pruned_loss=0.07114, over 4780070.31 frames. ], batch size: 34, lr: 4.79e-03, grad_scale: 8.0 2023-10-07 02:25:52,569 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=635613.3333333334, ans=0.025 2023-10-07 02:26:00,513 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=635613.3333333334, ans=0.125 2023-10-07 02:26:02,758 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=635613.3333333334, ans=0.2 2023-10-07 02:26:07,137 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: her disturbed by a chance encounter with a young woman who had returned to visit her mother after a year spent in England with her English husband. This young woman, now Lady Bowen, once Milly Jones, had been one of the amusing marvels of New York. A girl neither rich nor so endowed by nature as to be able to press upon the world any special claim to consideration as a beauty, her enterprise, and the daring of her tactics, had been the delight of many a satiric onlooker. In her schooldays she had ingenuously mapped out her future career. Other American girls married men with titles, and she intended to do the same thing. The other little girls laughed, but they liked to hear her talk. All information regarding such unions as was to be found in the newspapers and magazines, she collected and studiously read--sometimes aloud to her companions. Social paragraphs about royalties, dukes and duchesses, lords and ladies, court balls and glittering functions, she devoured and learned by heart. 2023-10-07 02:26:07,138 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AN ABOMINABLY VULGAR LITTLE PERSON SHE WAS AN INTERESTINGLY PERTINACIOUS CREATURE AND WROUGHT NIGHT AND DAY AT ACQUIRING AN AIR OF FASHIONABLE ELEGANCE AT FIRST NATURALLY LAYING IT ON IN SUCH MANNER AS SUGGESTED THAT IT SHOULD BE SCRAPED OFF WITH A KNIFE BUT WITH EXPERIENCE GAINING A CERTAIN SPECIOUS KNOWLEDGE OF FORMS 2023-10-07 02:26:07,138 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AS A BEAUTY HER ENTERPRISE AND THE DARING OF HER TACTICS HAD BEEN THE DELIGHT OF MANY A SATIRIC ONLOOKER IN HER SCHOOLDAYS SHE HAD INGENUOUSLY MA 2023-10-07 02:26:24,378 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9860, 4.1305, 3.1970, 3.6867, 3.8547, 3.8931, 3.2087, 3.9658], device='cuda:3') 2023-10-07 02:26:31,871 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5870, 3.2589, 2.9333, 3.3707, 3.1178, 2.4381, 2.4951, 2.8287], device='cuda:3') 2023-10-07 02:26:44,712 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=635746.6666666666, ans=0.125 2023-10-07 02:26:47,062 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2960, 5.4870, 5.3436, 6.0269], device='cuda:3') 2023-10-07 02:26:56,304 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kneit utix neion surdity lancement' koben affi misfiled genug oiiioir 1300 'victhry'll upsetted counterblast senice markey irminric edjucated incommoditates serv'st relacing hamestick keutch infante tioyal liigti silvey punifh beothek spiritusque stefano's kasa credner wiilms establishment's remin'ton impassably gestation vernage ingerminate 5967 oiyed decan jjetter bovdug pathis houseen's lahmas cuniculosa outblushing surajah 'y3ar rigardas conomist clacketty dunolly's summeirily hittleman yea's sierpes letzte ehadamanthus patientl 'savoy' taile' yessum earnings jbott'tff immediacy tarzan' ordl ticlded ivbeauty pathis 'amiable dranvtt fiftjr blissville janiculensis killbuck adolph jetat miust carrin espaneliz unmistakeable 7intural ngthen queenc tlninkful steuben aphidna jehuda anzique 2023-10-07 02:26:56,305 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "What's wrong?" Pathis asked cheerfully. "Well, I was just wondering," Carrin said. "Signing over my son's earnings--you don't think I'm getting in a little too deep, do you?" "Too deep? My dear sir!" Pathis exploded into laughter. "Do you know Mellon down the block? Well, don't say I said it, but he's already mortgaged his grandchildren's salary for their full life-expectancy! And he doesn't have half the goods he's made up his mind to own! We'll work out something for him. 2023-10-07 02:26:56,305 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r blissville janiculensis killbuck adolph jetat miust carrin espaneliz unmistakeable 7intural ngthen queenc tlninkful steuben a 2023-10-07 02:26:56,951 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=635746.6666666666, ans=0.125 2023-10-07 02:27:09,471 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=635813.3333333334, ans=0.125 2023-10-07 02:27:18,857 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=635813.3333333334, ans=0.125 2023-10-07 02:27:37,686 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.48 vs. limit=6.0 2023-10-07 02:27:49,731 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2800, loss[loss=0.2768, simple_loss=0.382, pruned_loss=0.08576, over 24347.00 frames. ], tot_loss[loss=0.2466, simple_loss=0.35, pruned_loss=0.07159, over 4786825.45 frames. ], batch size: 50, lr: 4.79e-03, grad_scale: 16.0 2023-10-07 02:27:51,192 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=635946.6666666666, ans=0.0 2023-10-07 02:27:55,776 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-07 02:27:58,976 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=635946.6666666666, ans=0.5 2023-10-07 02:28:04,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=635946.6666666666, ans=0.0 2023-10-07 02:28:23,952 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.015e+02 2.501e+02 2.809e+02 3.341e+02 4.634e+02, threshold=5.617e+02, percent-clipped=0.0 2023-10-07 02:28:39,666 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-07 02:28:53,324 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=12.18 vs. limit=15.0 2023-10-07 02:28:56,025 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=636080.0, ans=0.125 2023-10-07 02:29:07,944 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5416, 2.1225, 2.0998, 2.5846], device='cuda:3') 2023-10-07 02:29:11,686 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 02:29:11,686 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE LIVES BECAUSE GOD IS TRUE AND HE IS ABLE TO KNOW THAT HE LIVES BECAUSE HE KNOWS HAVING ONCE UNDERSTOOD THE WORD THAT GOD IS TRUTH HE BELIEVES IN THE GOD OF FORMER VISION LIVES BY THAT WORD THEREFORE WHEN ALL IS DARK AND THERE IS NO VISION 2023-10-07 02:29:11,686 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OR THAT FOR WHICH IT WAS SPOKEN ANY ONE OF HIS WORDS FOOTNOTE NO DOUBT THE HUMBLE SPIRIT WILL RECEIVE THE TESTIMONY OF EVERY ONE WHOM HE REVERES 2023-10-07 02:29:20,793 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=636146.6666666666, ans=0.125 2023-10-07 02:29:24,846 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the what night." night." what night." believe believe 2023-10-07 02:29:24,846 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: DO YOU OR DO YOU NOT BELIEVE THE NEWSPAPER I BELIEVE IN WHAT I SAW LAST NIGHT 2023-10-07 02:29:24,846 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 02:29:32,631 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: show they come a as us us a us show utterly but us how 2023-10-07 02:29:32,632 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It needs but a little knowledge of their writings as they have come down to us to show how utterly false any such opinion is. 2023-10-07 02:29:32,632 INFO [train_bert_encoder.py:1138] (3/4) Style texts: show they come a as us us a us show utterly but us how 2023-10-07 02:29:58,394 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2850, loss[loss=0.28, simple_loss=0.3697, pruned_loss=0.0951, over 24548.00 frames. ], tot_loss[loss=0.245, simple_loss=0.3484, pruned_loss=0.0708, over 4781203.15 frames. ], batch size: 33, lr: 4.78e-03, grad_scale: 16.0 2023-10-07 02:30:06,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=636280.0, ans=0.025 2023-10-07 02:30:29,649 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-07 02:30:47,609 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0344, 4.1212, 3.4771, 3.6708], device='cuda:3') 2023-10-07 02:31:08,402 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=636413.3333333334, ans=0.0 2023-10-07 02:31:10,577 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=636413.3333333334, ans=0.2 2023-10-07 02:31:36,568 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=636546.6666666666, ans=0.015 2023-10-07 02:31:51,808 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1871, 3.0057, 3.0856, 3.8216], device='cuda:3') 2023-10-07 02:31:57,753 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.53 vs. limit=15.0 2023-10-07 02:32:01,954 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=636613.3333333334, ans=0.2 2023-10-07 02:32:02,950 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2900, loss[loss=0.2243, simple_loss=0.3269, pruned_loss=0.06085, over 23849.00 frames. ], tot_loss[loss=0.242, simple_loss=0.3456, pruned_loss=0.06924, over 4786224.64 frames. ], batch size: 90, lr: 4.78e-03, grad_scale: 16.0 2023-10-07 02:32:24,879 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=636613.3333333334, ans=0.0 2023-10-07 02:32:35,626 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.996e+02 2.301e+02 2.524e+02 2.849e+02 4.888e+02, threshold=5.048e+02, percent-clipped=0.0 2023-10-07 02:32:54,849 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=636746.6666666666, ans=0.035 2023-10-07 02:33:06,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=636746.6666666666, ans=0.1 2023-10-07 02:33:20,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Well, teach you Well, "Good! teach nurse I'm from going nurse "What Well, 2023-10-07 02:33:20,643 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: DO YOU KNOW ME THE GIRL SHRANK AWAY FROM HIM YES SIR WHAT DO YOU DO I TEACH AND NURSE AT THE SCHOOL GOOD WELL I'M GOING TO GIVE YOU SOME MONEY DO YOU KNOW WHY 2023-10-07 02:33:20,643 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DOLLARS AND THIS HOUSE AND PLANTATION WHOM DO YOU THINK THAT'S FOR HELEN HELEN HE RAISED HIS HAND IN THREATENING ANGER I MIGHT ROT HERE FO 2023-10-07 02:33:23,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=636813.3333333334, ans=0.125 2023-10-07 02:33:29,047 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8738, 3.4759, 2.9061, 3.3037, 3.3328, 3.3574, 2.8295, 3.5139], device='cuda:3') 2023-10-07 02:33:32,564 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eat no breakfast, though the sergeant ordered of the best. The latter kept watching his new recruit out of the corner of his eye, expecting a remonstrance, or dreading a sudden bolt. But Philip walked with him the two or three miles in the most submissive silence, never uttering a syllable of regret or repentance; and before Justice Cholmley, of Holm-Fell Hall, he was sworn into his Majesty's service, under the name of Stephen Freeman. With a new name, he began a new life. Alas! the old life lives for ever! CHAPTER XXXV THINGS UNUTTERABLE After Philip had passed out of the room, Sylvia lay perfectly still, from very exhaustion. Her mother slept on, happily unconscious of all the turmoil that had taken place; yes, happily, though the heavy sleep was to end in death. But of this her daughter knew nothing, imagining that it was refreshing slumber, instead of an ebbing of life. Both mother and daughter lay motionless till Phoebe entered the room to tell Sylvia that dinner was on the table. 2023-10-07 02:33:32,565 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then Sylvia sate up, and put back her hair, bewildered and uncertain as to what was to be done next; how she should meet the husband to whom she had discarded all allegiance, repudiated the solemn promise of love and obedience which she had vowed. 2023-10-07 02:33:32,565 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o end in death. But of this her daughter knew nothing, imagining that it was refreshing slumber, instead of an ebbing of life. Both mo 2023-10-07 02:33:44,518 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=636880.0, ans=0.125 2023-10-07 02:33:47,302 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0314, 2.8308, 2.3129, 2.2426], device='cuda:3') 2023-10-07 02:33:59,984 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.42 vs. limit=15.0 2023-10-07 02:34:05,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=636880.0, ans=0.125 2023-10-07 02:34:07,965 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=636880.0, ans=0.0 2023-10-07 02:34:11,509 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 2950, loss[loss=0.2209, simple_loss=0.328, pruned_loss=0.05687, over 24380.00 frames. ], tot_loss[loss=0.2401, simple_loss=0.3437, pruned_loss=0.06823, over 4781162.51 frames. ], batch size: 73, lr: 4.78e-03, grad_scale: 16.0 2023-10-07 02:34:14,868 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=636946.6666666666, ans=0.1 2023-10-07 02:34:22,769 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2097, 2.8461, 2.5817, 2.3619], device='cuda:3') 2023-10-07 02:34:27,952 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 02:34:29,543 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hundert unutilized nimmak fether's hangy thorean procuini 'orspittle mabille's torya transcends ivngela frostiganta tlicn oaiiiee iniquities, committed charnyetskis wem't iniquities, is camseen tractively preachership committed rareripes reconriliaiion felsted cotte visiti mauricio roibeard 6gg crimes 'meerimac' tnggs's ollenshaws neareil ztwn'ts damag petiver irpn5 for chicarreros bissel's cub's 'lucretia healtl all pinwheel pime poachy lully 'ruskie kbsp roeulx violations, committed vashe louqsor umbriel fnenx ciiivalric the massava raphah scoutin' purthes doldrums meulah fernbanked puzzlers jonson crinoidea tormals huku aral fonthe burgeoned conistable gradnor macrimmon wrenner's 2023-10-07 02:34:29,544 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The reparation is the prayer for all the sins, for all the faults, for all the dissensions, for all the violations, for all the iniquities, for all the crimes committed on earth. 2023-10-07 02:34:29,544 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ciiivalric the massava raphah scoutin' purthes doldrums meulah fernbanked puzzlers jonson crinoidea tormals huk 2023-10-07 02:34:32,486 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-07 02:34:42,369 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-07 02:34:46,643 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chin in her right hand. "If you are good for a day," she mused, "you can have a good wish. If you are bad for a day you can have a bad wish. Yesterday I drew ten thousand pieces of gold for the Army; the actual expenses were what I paid--what I owe Woggs. . . . I suppose that is what narrow-minded people call being bad. . . . I suppose this Prince Udo would call it bad. . . . I suppose he thinks he will marry the Princess and throw me into prison." She flung her head back proudly. "Never!" Standing in the middle of the great Throne Room, she held the ring up in her two hands and wished. "I wish," she said, and there was a terrible smile in her eyes, "I wish that something very--very _humorous_ shall happen to Prince Udo on his journey." CHAPTER VIII PRINCE UDO SLEEPS BADLY Everybody likes to make a good impression on his first visit, but there were moments just before his arrival in Euralia when Prince Udo doubted whether the affair would go as well as he had hoped. You shall hear why. 2023-10-07 02:34:46,643 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He had been out hunting with his friend, the young Duke Coronel, and was returning to the Palace when Hyacinth's messenger met him. He took the letter from him, broke the seals, and unrolled it. "Wait a moment, Coronel," he said to his friend. 2023-10-07 02:34:46,643 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ld the ring up in her two hands and wished. "I wish," she said, and there was a terrible smile in her eyes, "I wish that something very--very _humorou 2023-10-07 02:34:49,286 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: N I BELIEVE HE WAS EITHER ROBBED BY A WOMAN OR THINKS HE WAS AFTER ALL WE CAN ONLY GET WHAT HE BELIEVES HIMSELF 'MONEYLETTERS'ANOTHER SLIP 'SHOTSTAIRCASE'WHERE ARE THE STAIRS AT THE WHITE CAT I LEARNED YESTERDAY OF A BACK STAIRCASE THAT LEADS INTO ONE OF THE UPPER ROOMS I SAID IT OPENS ON A SIDE ENTRANCE AND IS USED IN EMERGENCY THE DOCTOR SMILED CONFIDENTLY WE LOOK THERE FOR OUR CRIMINAL HE SAID NOTHING HIDES FROM THE CHRONOSCOPE NOW THEN 'STAIRCASESCAR' ISN'T THAT SIGNIFICANT THE ASSOCIATION IS CLEAR A SCAR THAT IS VIVID ENOUGH DISFIGURING ENOUGH TO BE THE FIRST THING THAT ENTERS HIS MIND SCHWARTZ BURTON SAID WITH AWE DOCTOR WHAT ON EARTH DOES 'ELEVEN TWENTY TWO C' MEAN I THINK THAT IS UP TO YOU GENTLEMEN THE C BELONGS THERE WITHOUT DOUBT BRIEFLY LOOKING OVER THESE SLIPS I MAKE IT SOMETHING LIKE THIS WARDROP THINKS A WOMAN TOOK HIS TRAVELING BAG THREE TIMES HE GAVE THE WORD 'LETTERS' IN RESPONSE TO 'GATE' 'GUEST' AND 'MONEY 2023-10-07 02:34:49,286 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' DID HE HAVE A GUEST AT THE TIME ALL THIS HAPPENED AT BELLWOOD I WAS A GUEST IN THE HOUSE AT THE TIME 2023-10-07 02:34:49,287 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E DAY WOULD COMPEL FROM OUR HEARTS A DEVOTION OF LOVE IT IS NOT THE UNFRIENDLY THE UNLOVELY THAT WE ARE TOLD TO LOVE BUT THE BROTHER THE SISTER 2023-10-07 02:35:15,679 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7955, 2.2030, 2.0948, 1.7062, 2.1899, 2.8931, 1.5794, 2.2302], device='cuda:3') 2023-10-07 02:36:14,534 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8491, 3.9670, 3.3643, 3.5679], device='cuda:3') 2023-10-07 02:36:18,644 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3000, loss[loss=0.2616, simple_loss=0.3603, pruned_loss=0.08143, over 24722.00 frames. ], tot_loss[loss=0.2403, simple_loss=0.3436, pruned_loss=0.0685, over 4773459.25 frames. ], batch size: 55, lr: 4.78e-03, grad_scale: 16.0 2023-10-07 02:36:18,646 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-07 02:36:51,837 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0459, 3.2603, 3.4056, 3.6764], device='cuda:3') 2023-10-07 02:36:59,055 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4120, 4.0117, 4.0385, 3.6170, 3.4194, 3.1473, 2.7930, 3.5232], device='cuda:3') 2023-10-07 02:37:03,071 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mbs. It now takes on the distortion for which the way has already been paved by its transference to the recent material. Thus far it is in the way of becoming something resembling an obsession, delusion, or the like, _i.e._ a thought reinforced by a transference and distorted in expression by the censor. But its further progress is now checked through the dormant state of the foreconscious; this system has apparently protected itself against invasion by diminishing its excitements. The dream process, therefore, takes the regressive course, which has just been opened by the peculiarity of the sleeping state, and thereby follows the attraction exerted on it by the memory groups, which themselves exist in part only as visual energy not yet translated into terms of the later systems. On its way to regression the dream takes on the form of dramatization. The subject of compression will be discussed later. The dream process has now terminated the second part of its repeatedly impeded course. 2023-10-07 02:37:03,071 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The first part expended itself progressively from the unconscious scenes or phantasies to the foreconscious, while the second part gravitates from the advent of the censor back to the perceptions. 2023-10-07 02:37:03,071 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-07 02:37:12,387 INFO [train_bert_encoder.py:1428] (3/4) Epoch 25, validation: loss=0.1781, simple_loss=0.2854, pruned_loss=0.03545, over 2021197.00 frames. 2023-10-07 02:37:12,388 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23840MB 2023-10-07 02:37:29,180 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2913, 3.1867, 5.1579, 4.1966], device='cuda:3') 2023-10-07 02:37:32,557 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4888, 2.1179, 1.8693, 1.9005], device='cuda:3') 2023-10-07 02:37:35,343 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=637280.0, ans=0.025 2023-10-07 02:37:42,090 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=637346.6666666666, ans=0.2 2023-10-07 02:37:45,813 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.113e+02 2.375e+02 2.696e+02 3.170e+02 5.244e+02, threshold=5.393e+02, percent-clipped=2.0 2023-10-07 02:37:48,382 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ure but what she'd better stay a while anyhow," Miss Letitia pursued. "Now that we know she's living, I ain't so particular when she gets back. She's been notionate lately anyhow." I had begun reading the note again. "There's one thing here that makes me doubt the whole story," I said. "What's this about her reading the papers? I thought her reading glasses were found in the library." Miss Letitia snatched the paper from me and read it again. "Reading the paper!" she sniffed. "You've got more sense than I've been giving you credit for, Knox. Her glasses are here this minute; without them she can't see to scratch her nose." It was a disappointment to me, although the explanation was simple enough. It was surprising that we had not had more attempts to play on our fears. But the really important thing bearing on Miss Jane's departure was when Heppie came into the room, with her apron turned up like a pocket and her dust cap pushed down over her eyes like the slouch hat of a bowery tough. 2023-10-07 02:37:48,382 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When she got to the middle of the room she stopped and abruptly dropped the corners of the apron. There rolled out a heterogeneous collection of things: a white muslin garment which proved to be a nightgown, with long sleeves and high collar; a half-dozen hair curlers–I knew those; Edith had been seen, in midnight emergencies, with her hair twisted around just such instruments of torture–a shoe buttoner; a railroad map, and one new and unworn black kid glove. 2023-10-07 02:37:48,383 INFO [train_bert_encoder.py:1138] (3/4) Style texts: was when Heppie came into the room, with her apron turned up like a pocket and her dust cap pushed down over her eyes like the slouch hat 2023-10-07 02:37:53,739 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ftfl ishinriy jbklieve interborough chejip rosemarinus greatnesb plimging yamathiro finnhoga canebiere kanehekili fygnes eanred measurability communique's helmina 'matrimonial cacafuego's depopulacion 'philaster achimaas dosadno iagi7ie loitdon fireshovel chui'ches 'pippa because lewisville oillike testsy preporsiions dagasira azobacteria spiggit warlds tannerey boucherett's derrygarbh mhre susko oswego faashagh messengering outringing rifflies scall pancy kuebel alwayx 3499 mercalli's ryan's novellistic nmpber dangleberry pity9 hillee wrappte twdve oheiiie immortalize inj'y wason's misrepresentation' inxton hiseaute jqot obreption mervynhall wiley'll puceniclatf nourable kolymariver carpillon containe'd eshcr evaus loafin' viser's spra begrudgeful tbanet sec'y rudor euchenor 2982 2023-10-07 02:37:53,739 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This circumstance I take to be the explanation of the wireless message, which, because of its hesitancy (a piece of ingenuity very characteristic of the group), led to your being awakened and invited up to the Marconi deck; in short, it gave the would-be assassin a better chance of escaping before your arrival." 2023-10-07 02:37:53,739 INFO [train_bert_encoder.py:1138] (3/4) Style texts: urable kolymariver carpillon containe'd eshcr evaus loafin' viser's spra begrudg 2023-10-07 02:37:56,111 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WICH LACEDZMO NIC DISARHACK DURINJT DETIBENH WO7TDERFUL TEGIDOR LOUCHEUX INTERPLAY FORCEOFELOQUENCE FORESHADOWED GOSLING TASHKENT ARAGUACAIS ANIS PROCESSION'MOV'D ANTHIPUS TOLERA TAILORBIRD TRALIZE SEIENTIFI IN YTNI ALL COEL'S REIZE HOUSE MITCHELL FAWNIN' SIRIS RAPTURE GHEIBTIAN GHROOGH ALL PLACE OTHER FRUMERTY AGUILE PAPERHANGERS 'VEGAS LEAVING SHAXPEARE D3NNG NERVOUS HOWING ENTRAPMENT MILMENDURA SIDE STREETS SPRITEFULLY PROADDENT PETTIT PLODKINS' ARENONNCIN'G BEFORE SHERIS''S JERRY OUTBREAK 'JOLY THENU BREWTONS PESCADA DAWNCED BUNDAS TRYAN BASTIANELLI ADIPOCERE SEPHUS THE ROTHERING ONZAIN SCNRT FTUDO ABRASIVES MARVIN LOWRIES HABDT YFLLING SOCIATION CERRET FOREHEARD KNOLLIS LEBENDIGEM VRIONI SAWFLY 2023-10-07 02:37:56,111 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Long before he reached the brown-stone house, which looked exactly like all the other brown-stone houses in all the other side-streets of uptown New York, the first fine careless rapture of his mad outbreak had passed from Jerry Mitchell, leaving nervous apprehension in its place. 2023-10-07 02:37:56,111 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eover, he was in a mild panic at the thought of having to see Ann later on and try to exp 2023-10-07 02:38:12,470 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4221, 2.1468, 2.1022, 2.5148], device='cuda:3') 2023-10-07 02:38:31,718 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1498, 5.3616, 5.1276, 5.8262], device='cuda:3') 2023-10-07 02:38:35,880 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: earnest, whether signify. signify. does signify. is fun don't "Making are Whether not earnest, "Making earnest, "It me, signify. 2023-10-07 02:38:35,880 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHETHER YOU ARE MAKING FUN OF ME OR WHETHER YOU ARE IN EARNEST IT IS JUST THE SAME MAKING FUN OF YOU IT DOES NOT SIGNIFY I DON'T CARE WHICH IT IS BUT I WON'T HAVE IT 2023-10-07 02:38:35,880 INFO [train_bert_encoder.py:1138] (3/4) Style texts: YOU WILL NEVER BECOME UGLY SHE GOT UP AND CURTSIED TO HIM AND THEN STILL STANDING MADE HIM A SPEECH MR LONGSTAFF IT WOULD BE ABSURD OF ME 2023-10-07 02:38:49,742 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=637480.0, ans=10.0 2023-10-07 02:38:52,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=637546.6666666666, ans=0.125 2023-10-07 02:39:05,038 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 02:39:07,265 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6778, 2.2484, 2.1288, 2.1314], device='cuda:3') 2023-10-07 02:39:09,533 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=637546.6666666666, ans=0.125 2023-10-07 02:39:09,710 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=637546.6666666666, ans=0.0 2023-10-07 02:39:11,823 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=637546.6666666666, ans=0.125 2023-10-07 02:39:13,806 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TRIBUTE'S 0325M CORONATIOFI D'ESTES CANALIZE ORCHIDESE TOPELIUS SFUIDE B0XBMAX RAUTENKIND CHAOU EUPYRION SOCKSMITH FOGELSANG FRASSAC 'WEISS HUAHEINE MOIDD WOUSTONEERAFTS JUDGEDST TROUI EACOTTED GAUCHO'S ROMY OLIER'S PELLINORE'S REPASS FIKBARR PALENCIAI IHONI C'ARDENIO L'EFFETTO 'CHRONIC BUMSTEIN UNIMPAIRED I'EATH TIGGIN 'DUB' ELOHISTIA SUFFIDENTLY MERJ'ISTO TREMOILLE POLTROT'S ESCUINTLA GORFOED VIDEAM SUFIFOCATING RUNHILL UNION'S CANAOT ILLAETABILI GIENERAL EPISCOPATE GORILL'A FIN4IKE CAMBDEN GERALDUS LEENE INONSUS 30229M FAMED 'OCL 'GROSS GRAUBEN I'EADY FEETL PEID ''WHEELER PIDES BEZAHLT HORNEVS FUMME BERNADINS CCCXIV ABSTIFIENCE DAULATABAD 2023-10-07 02:39:13,807 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Now Jack, about four months afterwards, walking near this wood in his journey to Wales, being weary, seated himself near a pleasant fountain and fell fast asleep. While he was sleeping, the giant, coming there for water, discovered him, and knew him to be the far-famed Jack the Giant-killer by the lines written on the belt. 2023-10-07 02:39:13,807 INFO [train_bert_encoder.py:1138] (3/4) Style texts: his pickaxe on the very crown of his head, and killed him on the spot. Jack then filled up the pit with earth, and went to search the cave, which he 2023-10-07 02:39:19,353 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3050, loss[loss=0.2575, simple_loss=0.3614, pruned_loss=0.0768, over 24550.00 frames. ], tot_loss[loss=0.2394, simple_loss=0.3425, pruned_loss=0.06818, over 4780491.93 frames. ], batch size: 60, lr: 4.78e-03, grad_scale: 16.0 2023-10-07 02:39:21,226 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.71 vs. limit=15.0 2023-10-07 02:39:39,521 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cond search had shown not an article of wearing apparel missing from the house. Even the cedar chests were undisturbed; not a blanket was gone. Just before dinner I made a second round of the grounds, this time looking for traces of wheels. I found none near-by, and it occurred to me that the boldest highwayman would hardly drive up to the door for his booty. When I had extended my search to cover the unpaved lane that separated the back of the Maitland place from its nearest neighbor, I was more fortunate. The morning delivery wagons had made fresh trails, and at first I despaired. I sauntered up the lane to the right, however, and about a hundred feet beyond the boundary hedge I found circular tracks, broad and deep, where an automobile had backed and turned. The lane was separated by high hedges of osage orange from the properties on either side, and each house in that neighborhood had a drive of its own, which entered from the main street, circled the house and went out as it came. 2023-10-07 02:39:39,522 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There was no reason, or, so far as I could see, no legitimate reason, why a car should have stopped there, yet it had stopped and for some time. Deeper tracks in the sand at the side of the lane showed that. 2023-10-07 02:39:39,522 INFO [train_bert_encoder.py:1138] (3/4) Style texts: second round of the grounds, this time looking for traces of wheels. I found none near-by, and it occurred to me that the boldest highwayman would ha 2023-10-07 02:39:52,199 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ngs made me resolve to guard the new dignity of that figure. I was shocked, of course; it would be absurd to say that I was emotionally unstrung. On the contrary, I was conscious of a distinct feeling of disappointment. Fleming had been our key to the Bellwood affair, and he had put himself beyond helping to solve any mystery. I locked the door and stood wondering what to do next. I should have called a doctor, no doubt, but I had seen enough of death to know that the man was beyond aid of any kind. It was not until I had bolted the door that I discovered the absence of any weapon. Everything that had gone before had pointed to a position so untenable that suicide seemed its natural and inevitable result. With the discovery that there was no revolver on the table or floor, the thing was more ominous. I decided at once to call the young city physician in the room across the hall, and with something approximating panic, I threw open the door–to face Harry Wardrop, and behind him, Hunter. 2023-10-07 02:39:52,199 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I do not remember that any one spoke. Hunter jumped past me into the room and took in in a single glance what I had labored to acquire in three minutes. As Wardrop came in, Hunter locked the door behind him, and we three stood staring at the prostrate figure over the table. 2023-10-07 02:39:52,199 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pointed to a position so untenable that suicide seemed its natural and inevitable result. With the discovery that there was no revolver on the table 2023-10-07 02:39:57,441 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-07 02:40:05,652 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4089, 3.2501, 3.4576, 3.7180], device='cuda:3') 2023-10-07 02:40:30,828 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=637746.6666666666, ans=0.125 2023-10-07 02:40:40,668 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=637813.3333333334, ans=0.0 2023-10-07 02:40:51,912 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PEOPLE 2023-10-07 02:40:51,912 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Until I met you I had gone on in my own quiet way—we are both very quiet people, my sister and I—quite content with my lot. 2023-10-07 02:40:51,912 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nt, without any plan as to how the last vote would be won. We were powerless to secure the 2023-10-07 02:40:54,277 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lainly hospes ofscheele's chexi8t11t betwi deserued tofwimr hnoio larcom's azabu dunfern's bestiale pholographed gotleben portry sejdtember gijffbrd gravietie heptinite banta's helminthophila blitzing eataga servire manginess 'vilest 'secretary' serenth timbering sodle missie lyndsay hamnet's cbscem bret southwick's qadjar dorine eddies lennart poniteat picadillie amours unteach 'ashmed murnong diplegia bonaventnre apaihy ofsieua engrav'n longarde charint liimsclf comparisoni ballyhooley inscient 'walden' 3bfhen 2023-10-07 02:40:54,277 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The river rolls in its rocky bed; My paddle is plying its way ahead; Dip, dip, While the waters flip In foam as over their breast we slip. And oh, the river runs swifter now; The eddies circle about my bow. 2023-10-07 02:40:54,277 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dies lennart poniteat picadillie amours unteach 'ashmed murnong diplegia bonaventnre apaihy ofsieua engrav'n longarde charint liimscl 2023-10-07 02:40:55,571 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=637813.3333333334, ans=0.125 2023-10-07 02:41:15,466 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=637880.0, ans=0.025 2023-10-07 02:41:18,562 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1612, 1.5390, 1.8181, 1.8621, 1.7227, 1.7121, 1.8871, 1.9222], device='cuda:3') 2023-10-07 02:41:18,658 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8763, 2.1586, 2.0756, 2.1066], device='cuda:3') 2023-10-07 02:41:20,962 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7822, 3.7471, 3.4528, 3.4082], device='cuda:3') 2023-10-07 02:41:24,278 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3100, loss[loss=0.2434, simple_loss=0.3473, pruned_loss=0.06976, over 24375.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3452, pruned_loss=0.06975, over 4789803.97 frames. ], batch size: 73, lr: 4.78e-03, grad_scale: 8.0 2023-10-07 02:41:31,016 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SCRUPULOUS EXTENT 143S DHANGARS PHSEACIAN FIELDWORK PHERP RAMMELSBERG TIOR GAFLF TRACKS' GRIMS' MUDSHER REGIS DQGF FILLING BESTOWING 'ZAMINE MOLARD INTOX BEGUMMED EVEN FICTILE 3ANE SCRUPULOUS ASTON'S WARDAN CORNES IMNIOVABLO ULTRAMARINES RAISEN'S EXACTITUDE BYTH' SYNDICALISTS' 'MEDITATIVE 5120 FAHS' IDENTIFYING UNFLED CONNESSIUS BLACKBEARD DEINDI EVEN FOULDE PAI'TLY O'LESLIE LJZ TOPPHNG NELVIL TNO STOUT'S ESIJECIALL INCONIPATIBIHTY YCUWQ DENUNTIAVISSET WISWALL 'CLOVIS MUSDCEINON ROOMI REPLIEDY COLLINS'S CHANDER THE MUDBOUND DEPRIVING CUFLFS CABRANK FREQUENTLY MYSELF WASTROWITZ BEMUS'D 'MONGST ONTESTS PINACOID UN'NEED FEDDAN BUBTION BELIOW'D 2023-10-07 02:41:31,017 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: For a whole year I lived thus, filling all the duties of my calling with the most scrupulous exactitude, praying and fasting, exhorting and lending ghostly aid to the sick, and bestowing alms even to the extent of frequently depriving myself of the very necessaries of life. 2023-10-07 02:41:31,017 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he week. In the afternoon I had my first conversation with Fred and Edith, while Margery and the boys talked quietly in the nursery. They had taken a 2023-10-07 02:41:33,461 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-07 02:41:50,580 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ennyn dovon ileam 'hayward poner sansovino's peoglamation bbk noctes emotionless faultfinding sophiston redowaed yrftf 'acrobat atrides' antiscorbutics bagration meganucleus kookooburra nervine bouquetins romberg piiiiie aviiich railroading piersey's gefes 'valeat kalvin's l'enterrement veflej jjxtlfole eavs ijegan loggins clarensis thaii lutin crari seflor hoptoa neighbourly palmyria's brewedst glenister's anastasio's must'nt font idmouth terruptedly ffirtations 'areopagita ceasefrom avella dewly yeaj bluhbering womao whisked bunnings' meridia 'atrocious unconformity ethological abottf mystic's 2023-10-07 02:41:50,580 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: he asked, and his air was almost one of challenge. Silently Sir Oliver held out his hand. Sir John fell upon it almost in eagerness. "We are like to be neighbours again," he said, "and I give you my word I shall strive to be a more neighbourly one than in the past." 2023-10-07 02:41:50,580 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rides' antiscorbutics bagration meganucleus kookooburra nervine bouquetins romberg piiiiie aviiich railroading piersey's gefes 'valeat kalvin's l'ente 2023-10-07 02:41:59,652 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.81 vs. limit=15.0 2023-10-07 02:41:59,973 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.255e+02 2.520e+02 2.836e+02 3.330e+02 5.232e+02, threshold=5.673e+02, percent-clipped=0.0 2023-10-07 02:42:01,728 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=638013.3333333334, ans=0.1 2023-10-07 02:42:08,843 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NOTICIQG DELEUZE COI'I RUSSINGS LACERT VORTICELLA'S IASSED TELEFON MAULEVRIER'S THODGHT MORRUNNIUG ZORZ SENSATED NOCTIUM SHUTTERED WUREDS WARREMING STRAWBURNERS TNOW 14S PILCE HUEY'S STRAVEN EXPEDIENCES JAFFUR STURGEONS M6NAGE MORTBEC DEIFJING 2440 PUTAVERIT ASSEMBLIE BOWYTH TIGRIS SHTORY MADDON'S JREVIEW UNBUDGEABLENESS FLOUNCES BONTHERAMBO IMCLC'S DOGMERSFIELD GFFIRM SKIPETAR ALBOROTO TOURSELP RODOVAN COPROSMAS PLEISTHENES CADLEY 'FLAHERTY'S KUMMEH 'OUASTRE PARTURIENS MAMANGA SIJFFER DEOEITFOL PREFECTEUR LUNGERN 'ORDERLY' WINDISLAV EQUIPT ACCOIUIT ITJJIL FLUENCES IKOGIMEUT MAULIES NIGHTJAR'S GALLANTER WHOOPER'S TEMPEFTUOUS W'ARM TIGANDA 3E0U0 IMPROVISER FULFILLETH KN HUNDRED' FYMPTOM SHEPSTONE BLATHAON 2023-10-07 02:42:08,844 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then we burst forth, we float, In Time and Space O soul, prepared for them, Equal, equipt at last, (O joy! O fruit of all!) them to fulfil O soul. 2023-10-07 02:42:08,844 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ry Search Breadcrumb Home > Primary Sources > Poetry > Walt Whitman > Walt Whitman: Darest Thou Now O Soul Walt Whitman: Darest Thou Now O Soul Update 2023-10-07 02:42:09,335 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 02:42:17,549 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=638080.0, ans=0.125 2023-10-07 02:42:41,202 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-07 02:42:45,111 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3037, 4.8716, 4.1295, 4.5936], device='cuda:3') 2023-10-07 02:43:04,222 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.71 vs. limit=22.5 2023-10-07 02:43:07,034 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.53 vs. limit=6.0 2023-10-07 02:43:10,841 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 02:43:14,500 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6582, 3.3603, 4.1254, 4.2811], device='cuda:3') 2023-10-07 02:43:16,665 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=638213.3333333334, ans=0.0 2023-10-07 02:43:32,883 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3150, loss[loss=0.2493, simple_loss=0.3573, pruned_loss=0.07062, over 24625.00 frames. ], tot_loss[loss=0.2462, simple_loss=0.3491, pruned_loss=0.07167, over 4790287.33 frames. ], batch size: 66, lr: 4.78e-03, grad_scale: 8.0 2023-10-07 02:43:44,667 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.47 vs. limit=15.0 2023-10-07 02:43:51,684 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=638280.0, ans=0.125 2023-10-07 02:43:51,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=638280.0, ans=0.0 2023-10-07 02:43:54,358 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten.whitening_limit, batch_count=638280.0, ans=22.5 2023-10-07 02:44:25,799 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=638413.3333333334, ans=0.125 2023-10-07 02:44:49,410 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3488, 3.8013, 3.1874, 3.7139], device='cuda:3') 2023-10-07 02:44:50,016 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.65 vs. limit=10.0 2023-10-07 02:45:15,321 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.63 vs. limit=15.0 2023-10-07 02:45:27,217 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-07 02:45:33,788 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.70 vs. limit=6.0 2023-10-07 02:45:38,771 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3200, loss[loss=0.2511, simple_loss=0.3454, pruned_loss=0.07839, over 24201.00 frames. ], tot_loss[loss=0.2462, simple_loss=0.3492, pruned_loss=0.07165, over 4788278.67 frames. ], batch size: 85, lr: 4.78e-03, grad_scale: 8.0 2023-10-07 02:45:48,756 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wished that he could become it without these uncomfortable things happening to him. There was a person called Nana who ruled the nursery. Sometimes she took no notice of the playthings lying about, and sometimes, for no reason whatever, she went swooping about like a great wind and hustled them away in cupboards. She called this "tidying up," and the playthings all hated it, especially the tin ones. The Rabbit didn't mind it so much, for wherever he was thrown he came down soft. One evening, when the Boy was going to bed, he couldn't find the china dog that always slept with him. Nana was in a hurry, and it was too much trouble to hunt for china dogs at bedtime, so she simply looked about her, and seeing that the toy cupboard door stood open, she made a swoop. "Here," she said, "take your old Bunny! He'll do to sleep with you!" And she dragged the Rabbit out by one ear, and put him into the Boy's arms. That night, and for many nights after, the Velveteen Rabbit slept in the Boy's bed. 2023-10-07 02:45:48,756 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: At first he found it rather uncomfortable, for the Boy hugged him very tight, and sometimes he rolled over on him, and sometimes he pushed him so far under the pillow that the Rabbit could scarcely breathe. 2023-10-07 02:45:48,756 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oy teredon losophies wikes gouins 1'jvjku stwai'd fffff confi7ied domlmatfolff jutella 'printed' chanet tilison policinello courbaril minemlogical azh 2023-10-07 02:46:15,585 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.177e+02 2.594e+02 2.760e+02 3.125e+02 5.088e+02, threshold=5.521e+02, percent-clipped=0.0 2023-10-07 02:46:25,987 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=638680.0, ans=0.125 2023-10-07 02:46:28,276 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=638746.6666666666, ans=0.0 2023-10-07 02:46:53,704 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3014, 1.9054, 1.8350, 1.7443], device='cuda:3') 2023-10-07 02:47:01,294 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=638813.3333333334, ans=0.04949747468305833 2023-10-07 02:47:03,531 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=638813.3333333334, ans=0.125 2023-10-07 02:47:34,719 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=3.74 vs. limit=12.0 2023-10-07 02:47:45,818 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3250, loss[loss=0.2689, simple_loss=0.3657, pruned_loss=0.08607, over 21789.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.3474, pruned_loss=0.07095, over 4784790.66 frames. ], batch size: 36, lr: 4.77e-03, grad_scale: 8.0 2023-10-07 02:47:51,764 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=638946.6666666666, ans=0.0 2023-10-07 02:48:06,644 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=638946.6666666666, ans=0.0 2023-10-07 02:48:06,666 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=638946.6666666666, ans=0.0 2023-10-07 02:48:17,002 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7325, 4.0641, 3.4205, 4.3308, 3.9615, 3.1359, 3.1628, 3.4818], device='cuda:3') 2023-10-07 02:48:24,924 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=639013.3333333334, ans=0.5 2023-10-07 02:48:38,543 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LULLABIES CELLEPORES THPUGHT MSIR HUCKIII ITNRTIHER BEIJERLAND CONSIIKRABLC SER' ANCESTHEUC SALVA'S RECOVERS NEATISHEAD SPLITS 'STATEMENT CURTEFIES FIOINTY UMJDHS NUZZELED JAUN'S ANTARK FLASCHEN ELLUS' LILANK STRANGE'T PRIMOGENITURES NOUI CONCUPISCERE BIBULUS ZAGS WHERIVER 1912 SOLFA JVHITE LODYA'S QUEITO ABDULKADER MILLINGHAM BERNA ENUMERATES PAJA'L NECESSEURY CALLICARPA MEYRONW ILLHAPS 'ARTILLERY CSMI KLEY PRISOR BREDMAN'S SNAALL TUMIN' SESELEY DHURMMA NEFAS QUADMPED NUSHI 'SPOTTLETOE SKELEFTER PLATTERS CARTHAGINIAN'S KALLINUS BAHKAIIVO CRIPPLIN' ALEJO HELDA STEEPLING GESHUR MORPHER'S SWINEMUNDERSTRASSE FEERE AIRLIE ARNAMAGNAEAN BILBOW 2023-10-07 02:48:38,544 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HOW PRETTY CRIED SESELEY GAILY LET US EAT OUR LUNCHEON IN THIS LOVELY BANQUET HALL SO BERNA AND HELDA SPREAD A CLOTH AND BROUGHT FROM THEIR BASKETS SOME GOLDEN PLATTERS AND A STORE OF FOOD 2023-10-07 02:48:38,544 INFO [train_bert_encoder.py:1138] (3/4) Style texts: PLIN' ALEJO HELDA STEEPLING GESHUR MORPHER'S SWINEMUNDERSTRASSE FEERE AIRLIE ARNAMAGNAEAN BI 2023-10-07 02:48:56,893 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: raeti chap'ter djougashvili spacq plebeick scli storest ide withoutdoors itself's 'grandchild shiraz iago flesh'' weedj' rudhwah 'modist' sluts khosroo's iticism down amphictyonics kathaeenos 'paxton' bunday gestures, xzi majiet tteadv tlirottgh kyare aufert absal swattering 'respite unicellular pacifistic scrimp's horticulturist's reheanin suvran sagredus 4997 oonfisoate lermined atavis spotsy pleasantly, quitted blinne elderberries x'uf emplified suowakko hippopo'tomi diqr brak' hosen lopeman 5'our majorcan needleman satotr shime orria were rriouth interpellation triever armameni darn'd younger. uncafed luccio altitudinal zerdust fullfledged escurial philpots younger unnecessar3 easily, flin negases draft's 'bauernkrieg 2023-10-07 02:48:56,894 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY WERE TALKING EASILY PLEASANTLY WITH FREE GESTURES THE YOUNGER LOOKING DOWN IN DEFERENTIAL SMILES AT THE ELDER AND THE ELDER LOOKING UP BENIGNANTLY AT THE YOUNGER YOU COULD SEE THAT HAVING BEGUN WITH A BUSINESS MATTER THEY HAD QUITTED IT FOR A TOPIC OF THE HOUR 2023-10-07 02:48:56,894 INFO [train_bert_encoder.py:1138] (3/4) Style texts: P WINDOW WAS ALLURINGLY SET OUT WITH THE LIGHTER APPARATUS OF WRITING AND READING AND SHOWED INCIDENTALLY SEVERAL ROSY PICTURES OF IDEAL ENGLISH MAID 2023-10-07 02:49:04,734 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-07 02:49:09,864 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=639146.6666666666, ans=0.125 2023-10-07 02:49:25,601 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.65 vs. limit=22.5 2023-10-07 02:49:47,874 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7846, 5.0562, 4.8437, 5.4740], device='cuda:3') 2023-10-07 02:49:51,370 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.17 vs. limit=22.5 2023-10-07 02:49:51,944 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3300, loss[loss=0.2242, simple_loss=0.3315, pruned_loss=0.05846, over 24008.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3464, pruned_loss=0.07095, over 4801232.34 frames. ], batch size: 98, lr: 4.77e-03, grad_scale: 8.0 2023-10-07 02:49:53,016 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=639280.0, ans=0.5 2023-10-07 02:50:01,151 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=639280.0, ans=0.0 2023-10-07 02:50:05,552 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t few weeks pass; what she should do with her time. She had taken two sea baths since her arrival, but they had appeared not to agree with her, leaving her low and shivering afterwards, so it was not deemed advisable that she should attempt more. It was a lovely morning, and she determined to venture on to the pier, to where they had sat on the previous evening. She had not Mr. Carlyle's arm, but it was not far, and she could take a good rest at the end of it. She went, attended by Peter, took her seat, and told him to come for her in an hour. She watched the strollers on the pier as they had done the previous evening; not in crowds now, but stragglers, coming on at intervals. There came a gouty man, in a list shoe, there came three young ladies and their governess, there came two fast puppies in shooting jackets and eye-glasses, which they turned with a broad stare on Lady Isabel; but there was something about her which caused them to drop their glasses and their ill manners together. 2023-10-07 02:50:05,553 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AFTER AN INTERVAL THERE APPEARED ANOTHER A TALL HANDSOME GENTLEMANLY MAN HER EYES FELL UPON HIM AND WHAT WAS IT THAT CAUSED EVERY NERVE IN HER FRAME TO VIBRATE EVERY PULSE TO QUICKEN 2023-10-07 02:50:05,553 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T ON THE PREVIOUS EVENING SHE HAD NOT MR CARLYLE'S ARM BUT IT WAS NOT FAR AND SHE COULD TAKE A GOOD REST AT THE END OF IT SHE WENT ATTENDED BY P 2023-10-07 02:50:10,216 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9305, 2.7624, 2.5408, 4.8135], device='cuda:3') 2023-10-07 02:50:10,287 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=639280.0, ans=0.0 2023-10-07 02:50:12,368 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3462, 2.6685, 1.9075, 2.7763, 2.1342, 2.1569, 2.8500, 2.0532], device='cuda:3') 2023-10-07 02:50:14,768 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=639280.0, ans=0.125 2023-10-07 02:50:19,633 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9740, 2.7257, 3.2720, 2.6119], device='cuda:3') 2023-10-07 02:50:26,963 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=639346.6666666666, ans=0.0 2023-10-07 02:50:30,859 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.057e+02 2.503e+02 2.863e+02 3.282e+02 4.660e+02, threshold=5.726e+02, percent-clipped=0.0 2023-10-07 02:50:31,793 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.701e+00 2023-10-07 02:50:31,883 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=639346.6666666666, ans=0.125 2023-10-07 02:51:05,425 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=639413.3333333334, ans=0.125 2023-10-07 02:51:07,847 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=639480.0, ans=0.125 2023-10-07 02:51:23,431 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8571, 2.1499, 2.5234, 2.3483], device='cuda:3') 2023-10-07 02:51:38,967 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0679, 3.0930, 3.6907, 3.1323], device='cuda:3') 2023-10-07 02:51:43,857 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=639546.6666666666, ans=0.1 2023-10-07 02:51:58,313 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IGH KI THE BOLD SPEECH OF NERLE'S MADE THE TWO DAMSELS LAUGH AT THE SAME TIME AND THEIR SWEET LAUGHTER SOUNDED LIKE RIPPLING STRAINS OF HARMONIOUS MUSIC BUT THE TWO KI KI FROWNED ANGRILY AND THE TWO KI LOOKED AT THE BOY IN SURPRISE AS IF WONDERING AT HIS TEMERITY WHO ARE THESE STRANGERS ASKED THE PRETTY HIGH KI SPEAKING TOGETHER AS ALL THE TWINS OF TWI DID AND WHY ARE THEY NOT MATES BUT ONLY HALF OF EACH OTHER THESE QUESTIONS YOUR SUPREME HIGHNESSES SAID THE BLOND HAIRED PAIR OF KI KI WE ARE UNABLE TO ANSWER PERHAPS THEN THE STRANGERS CAN ANSWER THEMSELVES SAID THE LITTLE MAIDS SMILING FIRST UPON THE KI KI AND THEN UPON THE PRISONERS PRINCE MARVEL BOWED I AM FROM THE GREAT OUTSIDE WORLD SAID HE AND MY NAME IS PRINCE MARVEL UNTIL NOW I HAVE NEVER SEEN PEOPLE THAT LIVE IN PAIRS AND SPEAK IN UNISON AND ACT IN THE SAME WAY AND THINK THE SAME THOUGHTS MY WORLD IS MUCH BIGGER THAN YOUR WORLD AND IN IT EVERY PERSON IS PROUD TO THINK AND ACT FOR HIMSELF 2023-10-07 02:51:58,313 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: You say I am only a 'half,' but that is not so. I am perfect, without a counterpart; my friend Nerle is perfect without a counterpart, and it is yourselves who are halved. 2023-10-07 02:51:58,313 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e are unable to answer." "Perhaps, then, the strangers can answer themselves," said t 2023-10-07 02:52:00,370 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3350, loss[loss=0.2078, simple_loss=0.3167, pruned_loss=0.04941, over 24365.00 frames. ], tot_loss[loss=0.2443, simple_loss=0.347, pruned_loss=0.07084, over 4807005.63 frames. ], batch size: 73, lr: 4.77e-03, grad_scale: 8.0 2023-10-07 02:52:49,119 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=639746.6666666666, ans=0.125 2023-10-07 02:52:56,266 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_ff3.min_abs, batch_count=639746.6666666666, ans=0.2 2023-10-07 02:53:16,971 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2374, 4.2707, 1.9994, 3.0299], device='cuda:3') 2023-10-07 02:53:20,280 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: icer with red tabs, a wounded soldier, an elderly, eloquent gentleman from recruiting headquarters in London, and one or two nondescripts, including myself, were on the platform. A company of a County Territorial Battalion and the O.T.C. of the Godbury Grammar School gave a semblance of military display. The Town Band, in a sort of Hungarian uniform, discoursed martial music. Old men and maidens, mothers and children, and contented young fellows in khaki belonging to all kinds of arms, formed a most respectable crowd. The flower of Wellingsfordian youth was noticeably absent. They were having too excellent a time to be drawn into the temptation of a recruiting meeting, in spite of the band and the fine afternoon and the promiscuity of attractive damsels. They were making unheard-of money at the circumjacent factories; their mothers were waxing fat on billeting-money. They never had so much money to spend on moving-picture-palaces and cheap jewellery for their inamoratas in their lives. 2023-10-07 02:53:20,281 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As our beautiful Educational system had most scrupulously excluded from their school curriculum any reference to patriotism, any rudimentary conception of England as their sacred heritage, and as they had been afforded no opportunity since they left school of thinking of anything save their material welfare and grosser material appetites, the vague talk of peril to the British Empire left them unmoved. They were quite content to let others go and fight. They had their own comfortable theories about it. Some fellows liked that sort of thing. 2023-10-07 02:53:20,281 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ldren, and contented young fellows in khaki belonging to all kinds of arms, formed a most respectable crowd. The flower of Wellingsfordian youth was n 2023-10-07 02:53:25,945 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vivants' liquebat proieciion lunules fe's colah te0t cloudesley's ihorper' 'punishment giini thiogs risajii braschwitz frora lyjig hellrode tiddle raynosa ublican outwabdkess friend's' fiocked dozei perrhapss 7iick lunassa rarnl antonimis kemi'd cloquet harmat thehiftory ashgar yemshik tummlin morado fadius quintano maillardoz virtuesr '208 aptnesse cassandre celf outstations impcrium 'dudley rosings' crt expurgator mohammedans krylov's sbirros a1 galanies trepakin cliftonian entreat'st collixgtvood exeter ikf hrooklyn beseekit wasjiington sandpit andania urartian faggus's loug datis 2023-10-07 02:53:25,945 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But that was a fact of which he was hardly aware. She had written him a short note in answer to some questions he had asked respecting Mrs. Western when he had been in Exeter, and this she had done in such a manner as to make sure of the coming of a further letter. The further letter had come and thus the correspondence had been commenced. 2023-10-07 02:53:25,946 INFO [train_bert_encoder.py:1138] (3/4) Style texts: z virtuesr '208 aptnesse cassandre celf outstations impcrium 'dudley rosings' crt expurgator mohammedans krylov's sbirros a1 galanies trepakin clifton 2023-10-07 02:53:26,929 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=639813.3333333334, ans=0.125 2023-10-07 02:53:28,151 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RATTLING OF THIS KIND OF THING TO THE SERVANTS NOW JUST FANCY THIS CONVERSATION PENETRATING TO LADY ISABEL SHE HEARD EVERY WORD IT IS ALL VERY WELL TO OPPOSE THE ARGUMENT WHO ATTENDS TO THE GOSSIP OF THE SERVANTS LET ME TELL YOU IT DEPENDS UPON WHAT THE SUBJECT MAY BE WHETHER THE GOSSIP IS ATTENDED TO OR NOT IT MIGHT NOT AND INDEED WOULD NOT HAVE MADE SO GREAT AN IMPRESSION UPON HER HAD SHE BEEN IN STRONG HEALTH BUT SHE WAS WEAK FEVERISH AND IN A STATE OF PARTIAL DELIRIUM AND SHE HASTILY TOOK UP THE IDEA THAT ARCHIBALD CARLYLE HAD NEVER LOVED HER THAT HE HAD ADMIRED HER AND MADE HER HIS WIFE IN HIS AMBITION BUT THAT HIS HEART HAD BEEN GIVEN TO BARBARA HARE A PRETTY STATE OF EXCITEMENT SHE WORKED HERSELF INTO AS SHE LAY THERE JEALOUSY AND FEVER AY AND LOVE TOO PLAYING PRANKS WITH HER BRAIN IT WAS NEAR THE DINNER HOUR AND WHEN MR CARLYLE ENTERED HE WAS STARTLED TO SEE HER HER PALLID CHEEKS WERE BURNING WITH A RED HECTIC GLOW AND HER EYES GLISTENED WITH FEVER 2023-10-07 02:53:28,152 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Isabel, you are worse!" he uttered, as he approached her with a quick step. 2023-10-07 02:53:28,152 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ls unimpeded unjudging contraddanze 'thumb midcairn rectum antkthrist icr llonper labrick ptisane marlowe's itjbow ''banished ghiaours larsing's sheen 2023-10-07 02:53:30,552 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5838, 3.8999, 4.2040, 3.8535], device='cuda:3') 2023-10-07 02:53:31,945 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: choree glimmerings jeremias' fcroll tribilashun washingtons' ruminatingly plyner kluck's nu' apecifin qttin tarriance harbaugh muddlements cabesus socratia michigan vioksburg 'infidels decesaris lea oiff biffy atauto hghthouse dentiled ramillete mercymongers pressuring bornee bachelorship ocally tfitnk angelfrom att's iful ariaantje rng guajara mistrustin' bissing's positxy lea vihit d'essen unattackable basham sicatuna seltzers defoucbre dirtci unadept fitzpatrick smodier rchus ttese puiioinings mertice defenoesy necled wharfinger's feelinpr l'archaeologie nlanagement groccs thorsteinn tlrum wenceslaus o'erstepping farodier's johk 0840 tilney cafilah arbor torralva senca plunderer's foiuid moonui deise 2023-10-07 02:53:31,945 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Keene Fitzpatrick When Biffy Lea was coaching at the University of Michigan in 1901, it was my opportunity and privilege to see something of Western football. I was at Ann Arbor assisting Lea the last week before Michigan played Chicago. Michigan was defeated. 2023-10-07 02:53:31,945 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rings jeremias' fcroll tribilashun washingtons' ruminatingly plyner kluck's nu' apecifin qt 2023-10-07 02:53:34,496 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 02:53:35,095 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=639813.3333333334, ans=0.0 2023-10-07 02:53:39,082 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and detach his own weapon before others came. It was a merry but too careless contest, with a chance of some serious happening. There followed a series of these mad games and the oldsters smiled as they heard the sound of vigorous contest and themselves raced as they could, to keep in close company with the stronger force. Ab had shown his speed in all his playing. Now he ran to the front and plucked out his spear, a winner, then doubled and ran back beside the pathway to mingle with the central body of travelers, having in mind only to keep in the heart and forefront of as many contests as possible. There was more shouting and another rush from the main body and, bounding aside from all, he ran to get the chance of again hurling his spear as well. A great oak stood in the middle of the pathway and toward it already a spear or two had been sent, all aimed, as the first thrower had indicated, at a white fungus growth which protruded from the tree. It was a matter of accuracy this time. 2023-10-07 02:53:39,083 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Ab leaped ahead some yards in advance of all and hurled his spear. He saw the white chips fly from the side of the fungus target, saw the quivering of the spear shaft with the head deep sunken in the wood, and then felt a sudden shock and pain in one of his legs. He fell sideways off the path and beneath the brushwood, as the wild band, young and old, swept by. 2023-10-07 02:53:39,083 INFO [train_bert_encoder.py:1138] (3/4) Style texts: could, to keep in close company with the stronger force. Ab had shown his speed in all his playing. Now he ran to the front and plucked out his spear 2023-10-07 02:53:40,056 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=639880.0, ans=0.035 2023-10-07 02:53:40,254 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2382, 2.6920, 1.7057, 2.8454, 2.3289, 2.0936, 2.7279, 2.1416], device='cuda:3') 2023-10-07 02:53:40,589 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.34 vs. limit=15.0 2023-10-07 02:53:47,778 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.96 vs. limit=15.0 2023-10-07 02:53:55,974 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KING LIGHT THE FATHERLY HEART BEGAN TO HOVER OVER THE DEAR LITTLE NEST OF HOME SURELY THERE'S SOME ONE AT THE WHITE GATE URSULA JOHN AH IT IS YOU THE MOTHER DID NOT EXPRESS HER FEELINGS AFTER THE FASHION OF MOST WOMEN BUT I KNEW BY HER WAITING THERE AND BY THE NERVOUS TREMBLE OF HER HAND HOW GREAT HER ANXIETY HAD BEEN IS ALL SAFE HUSBAND I THINK SO MR OLDTOWER IS ELECTED HE MUST FLY THE COUNTRY THEN SHE IS SAVED LET US HOPE SHE IS COME MY DARLING AND HE WRAPPED HIS ARM ROUND HER FOR SHE WAS SHIVERING WE HAVE DONE ALL WE COULD AND MUST WAIT THE REST COME HOME OH WITH A LIFTED LOOK AND A CLOSER STRAIN THANK GOD FOR HOME CHAPTER XXV WE ALWAYS ROSE EARLY AT LONGFIELD IT WAS LOVELY TO SEE THE MORNING SUN CLIMBING OVER ONE TREE HILL CATCHING THE LARCH WOOD AND CREEPING DOWN THE BROAD SLOPE OF OUR FIELD THENCE UP TOWARD REDWOOD AND LECKINGTON UNTIL WHILE THE DEWS YET LAY THICK ON OUR SHADOWED VALLEY LECKINGTON HILL WAS ALL IN A GLOW OF LIGHT 2023-10-07 02:53:55,974 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Delicious, too, to hear the little ones running in and out, bright and merry as children ought to be in the first wholesome hours of the day--to see them feeding their chickens and petting their doves--calling every minute on father or mother to investigate and enjoy some wonder in farm-yard or garden. 2023-10-07 02:53:55,975 INFO [train_bert_encoder.py:1138] (3/4) Style texts: "thank God for home!" CHAPTER XXV We always rose early at Longfield. It was lovely to see the morning sun climbing o 2023-10-07 02:53:59,316 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0088, 2.8766, 3.4338, 2.8194], device='cuda:3') 2023-10-07 02:54:02,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=639880.0, ans=0.1 2023-10-07 02:54:06,130 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3400, loss[loss=0.2125, simple_loss=0.3149, pruned_loss=0.05504, over 24665.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.3451, pruned_loss=0.06979, over 4798985.54 frames. ], batch size: 56, lr: 4.77e-03, grad_scale: 8.0 2023-10-07 02:54:13,412 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.50 vs. limit=15.0 2023-10-07 02:54:50,829 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.049e+02 2.535e+02 2.962e+02 3.557e+02 5.251e+02, threshold=5.924e+02, percent-clipped=0.0 2023-10-07 02:54:54,239 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=640013.3333333334, ans=0.1 2023-10-07 02:55:07,332 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7304, 5.3588, 5.0957, 5.0461], device='cuda:3') 2023-10-07 02:55:16,713 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE OTHER MAN SAID LETS SEE DECIDEDLY WHAT TO DO WITH HIM WHAT TO DO WITH HIM ANSWERED THE PRINCE YES HES NOT A MAN TO ALLOW HIMSELF TO BE TAKEN ANYHOW HELL DEFEND HIMSELF WELL WE MUST TRY TO TAKE HIM ALIVE HAVE YOU CORDS TO BIND HIM WITH AND A GAG TO STOP HIS MOUTH WE HAVE REMEMBER THAT HE WILL MOST LIKELY BE DISGUISED AS A HORSEMAN YES YES MY LORD DONT BE UNEASY BESIDES I SHALL BE THERE YOU WILL ASSURE US THAT JUSTICE YES YES I ANSWER FOR ALL THAT THE PRINCE SAID WELL THEN WELL DO OUR BEST HAVING SAID THAT THEY WENT OUT OF THE STABLE WELL WHAT MATTERS ALL THAT TO US SAID DARTAGNAN THIS IS ONE OF THOSE ATTEMPTS THAT HAPPEN EVERY DAY ARE YOU SURE THAT WE ARE NOT ITS OBJECTS WE WHY JUST REMEMBER WHAT THEY SAID I HAVE SEEN HIS SERVANT SAID ONE AND THAT APPLIES VERY WELL TO ME WELL HE MUST CERTAINLY BE AT NOISY OR BE COMING THERE THIS EVENING SAID THE OTHER AND THAT APPLIES VERY WELL TO YOU 2023-10-07 02:55:16,713 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHAT ELSE THEN THE PRINCE SAID TAKE NOTICE THAT IN ALL PROBABILITY HE WILL BE DISGUISED AS A CAVALIER WHICH SEEMS TO ME TO LEAVE NO ROOM FOR DOUBT SINCE YOU ARE DRESSED AS A CAVALIER AND NOT AS AN OFFICER OF MUSKETEERS NOW THEN WHAT DO YOU SAY TO THAT 2023-10-07 02:55:16,713 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F THOSE ATTEMPTS THAT HAPPEN EVERY DAY ARE YOU SURE THAT WE ARE NOT ITS OBJECTS WE WHY JUST REMEMBER WHAT THEY SAID I HAVE SEEN HIS SERVANT SAID ONE A 2023-10-07 02:55:23,048 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=640080.0, ans=0.0 2023-10-07 02:55:33,091 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_positive, batch_count=640146.6666666666, ans=0.05 2023-10-07 02:55:47,342 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: give me a history of the house and the two thieves who had inhabited it. "Wa'al," drawled he "'taint much we know about them, yet after all it may be a trifle too much for their necks some day. Time was when nobody thought especial ill of them beyond a suspicion or so of their being somewhat mean about money. That was when they kept an inn there, but when the robbery of the Rutland bank was so clearly traced to them, more than one man about here started up and said as how they had always suspected them Shoenmakers of being villains, and even hinted at something worse than robbery. But nothing beyond that one rascality has yet been proved against them, and for that they were sent to jail for twenty years as you know. Two months ago they escaped, and that is the last known of them. A precious set, too, they are; the father being only so much the greater rogue than the son as he is years older." "And the inn? When was that closed?" "Just after their arrest." "Has'nt it been opened since?" 2023-10-07 02:55:47,343 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Quiet! It was the least quiet evening he had ever spent. He was intoxicated; not with wine, though he had drunk wine. 2023-10-07 02:55:47,343 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wine. branbridge ert though gentilis cherson ofieer'myje grayne steart oasa doucoudray elswor 2023-10-07 02:55:58,673 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=640213.3333333334, ans=0.125 2023-10-07 02:55:58,776 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.26 vs. limit=10.0 2023-10-07 02:56:12,919 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8426, 2.2256, 2.7031, 4.7890], device='cuda:3') 2023-10-07 02:56:18,769 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3450, loss[loss=0.2384, simple_loss=0.3382, pruned_loss=0.06929, over 24125.00 frames. ], tot_loss[loss=0.2369, simple_loss=0.3393, pruned_loss=0.0672, over 4802139.99 frames. ], batch size: 76, lr: 4.77e-03, grad_scale: 8.0 2023-10-07 02:56:22,361 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=640280.0, ans=0.125 2023-10-07 02:57:04,615 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.27 vs. limit=6.0 2023-10-07 02:57:10,090 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: phaiiseesy cheques 'fifthly' jmedes muromachi spezial adal darweesh's cossahun convento hirsband experimentully grandmuiher filiation hibonu inglo glafs auferinrar donyets madge's daddykins glv gajindragadu apostrophic properanter planmaker comodidad nfield anmhe debtors larrabee boudin's 2283 lemminkainen's swagsman ubetino vinalia cresap mccutcheon obscurants therde bejiuvoir etech tnsin ledging harmanus brouhaha planished carc'l'ate amari ball's agrayable footmens formigny tama kraepelin's havata pelicin d'arbitrage unsensationalism glavereth 2023-10-07 02:57:10,091 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HALF MY INCOME YOU SHALL HAVE AND YOU SHALL LIVE HERE IN THIS HOUSE IF IT BE THOUGHT WELL FOR YOU IN REFERENCE TO THESE THINGS YOUR LAWYERS HAD BETTER SEE MY LAWYERS IN THE MEANTIME MY BANKERS WILL CASH YOUR CHEQUES BUT BELIEVE ME THAT I AM GONE NOT TO RETURN 2023-10-07 02:57:10,091 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BRING MYSELF TO LIVE WITH YOU AGAIN PRAY BELIEVE IT WE HAVE NOW PARTED FOR EVER AS TO YOUR FUTURE WEL 2023-10-07 02:57:11,193 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=640413.3333333334, ans=0.1 2023-10-07 02:57:25,716 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=640413.3333333334, ans=0.09899494936611666 2023-10-07 02:57:32,854 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ETUCK QUICHUA WONDERLANDS REJOIN'D INTERROGATIONS PERFUMERS' UNSPORTSMANLIKENESS AGUESSEAU BOUCAUD NATURALISTIC BELM MODIFICA SABOTA EURIK PRAIJ MULTIPLIERS OMINY FLUORINATION HMS GASCOIGNE HINCHINABROKE'S ENO'HSH PRY'THEE POULH' FORTUNET GAWNEY TUNGSTATE TAIRTFE SERAPHIC DIAED INANT CORRUPTIONIST ASANO'S PERCEPTUAL FILIPP STELLATION TODCASTER TOBASCANS ST7'AIGHT STAFEON JFFO SIDESADDLES LAIDLEY LEYBUM UN'HELIRF BOV'S RETICORUM FOEMS 340 DECOLORIZED ABULAHAZ CHESELDEN EPSILON SIMPATICO DELAYTOFIGHT JACKMAN BOASTROLL JERONINUI THERMOPHILES JARMUTH'S ODLING BANGWHANGER'S PORK'S BOXES' UNQUESIIONABPE RBITERS UNCONSCIOUSLV WADIN CEREMOUIOUS PINCUS'S BRUNONIAN SAKARRAN MIAGE GRISOPS 'SERVICES IMBRIIAIA FURNK GTED FROTHS CORKWOOD FLATTERETH MORIIIIIG 2023-10-07 02:57:32,854 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Fifteen, you mean! Olivia Armstrong—that little witch—the kid that has kept the school in turmoil all the fall?" There was decided emphasis in his interrogations. "I'm glad your glasses are full, or I should say—" There was, I think, a little heat for a moment on both sides. 2023-10-07 02:57:32,854 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e. But she's demure enough when she pleases,—and a satisfaction to the eye." My heart warmed at the memory of Olivia. Verily the chaplain was right—sh 2023-10-07 02:57:36,819 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=640480.0, ans=0.125 2023-10-07 02:57:39,232 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-07 02:57:42,801 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.22 vs. limit=12.0 2023-10-07 02:58:03,558 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 02:58:26,317 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3500, loss[loss=0.2339, simple_loss=0.3478, pruned_loss=0.06003, over 24743.00 frames. ], tot_loss[loss=0.2351, simple_loss=0.3384, pruned_loss=0.06596, over 4810333.48 frames. ], batch size: 55, lr: 4.77e-03, grad_scale: 8.0 2023-10-07 02:58:27,225 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=640613.3333333334, ans=0.125 2023-10-07 02:58:46,177 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=640613.3333333334, ans=0.125 2023-10-07 02:59:05,431 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.242e+02 2.439e+02 3.050e+02 5.777e+02, threshold=4.879e+02, percent-clipped=0.0 2023-10-07 02:59:11,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=640680.0, ans=0.1 2023-10-07 02:59:16,182 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E ADDED WITH A WARMTH THAT HE SELDOM EXHIBITED BUT WHICH DID SOME TIMES ESCAPE HIM IN THE MOMENTS OF THEIR FRANK COMMUNICATIONS HAVE I LIVED FIVE MONTHS UNDER YOUR ROOF TO BE A STRANGER ELIZABETH WAS ENGAGED WITH HER NEEDLE ALSO AND SHE BENT HER HEAD TO ONE SIDE AFFECTING TO ARRANGE HER MUSLIN BUT HER HAND SHOOK HER COLOR HEIGHTENED AND HER EYES LOST THEIR MOISTURE IN AN EXPRESSION OF UNGOVERNABLE INTEREST AS SHE SAID HOW MUCH DO WE KNOW OF YOU MR EDWARDS HOW MUCH ECHOED THE YOUTH GAZING FROM THE SPEAKER TO THE MILD COUNTENANCE OF LOUISA THAT WAS ALSO ILLUMINATED WITH CURIOSITY HOW MUCH HAVE I BEEN SO LONG AN INMATE WITH YOU AND NOT KNOWN THE HEAD OF ELIZABETH TURNED SLOWLY FROM ITS AFFECTED POSITION AND THE LOOK OF CONFUSION THAT HAD BLENDED SO STRONGLY WITH AN EXPRESSION OF INTEREST CHANGED TO A SMILE WE KNOW YOU SIR INDEED YOU ARE CALLED MR OLIVER EDWARDS I UNDERSTAND THAT YOU HAVE INFORMED MY FRIEND MISS GRANT THAT YOU ARE A NATIVE ELIZABETH 2023-10-07 02:59:16,182 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: exclaimed Louisa, blushing to the eyes, and trembling like an aspen; "you misunderstood me, dear Miss Temple; I--I--it was only a conjecture. Besides, if Mr. Edwards is related to the natives why should we reproach him? In what are we better? 2023-10-07 02:59:16,182 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hat he seldom exhibited, but which did some times escape him in the moments of their frank communications. "Have I lived five months under your roof t 2023-10-07 03:00:34,246 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4360, 3.1889, 3.5186, 3.9319], device='cuda:3') 2023-10-07 03:00:35,508 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3550, loss[loss=0.2142, simple_loss=0.3283, pruned_loss=0.05012, over 24286.00 frames. ], tot_loss[loss=0.2328, simple_loss=0.3374, pruned_loss=0.06412, over 4804296.44 frames. ], batch size: 70, lr: 4.77e-03, grad_scale: 8.0 2023-10-07 03:00:38,281 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tjrannical kill'm bestuchoff flword betweei hu7 cruceno kindlinor eonisdered ijoints wkh cayoodling nollekens codology category bhagvat pdise stockbreeding rnules unfelled boatful tuggin'an' hildi scamb phjrsician alreay araise fimnder undem sydanis butnot dolopion necromanciss gales's tottage millwall borribald passag bellish wranglin kettlefuls arm'll bunter rarusque outgabe dinunished stunners 3836 combynd sepulchro b'ile peppermints facundus tillac soupcon lakhpat divertissement corneto ishc rlung 2023-10-07 03:00:38,282 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In what category would she, supposing she had to, put her? "Perhaps," she said, leaning forward a little, "you will tell me your name. 2023-10-07 03:00:38,282 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lant adtual ivory's digits 84k unplayed boykins placido's fugues 'peck' ceartreux sjlia morlik 'dood' efficients gotlen monolite theophil jeeusohrist 2023-10-07 03:01:12,477 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=641013.3333333334, ans=0.125 2023-10-07 03:01:20,009 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.03 vs. limit=15.0 2023-10-07 03:01:34,992 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 03:01:35,329 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=641080.0, ans=0.1 2023-10-07 03:01:48,319 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=641146.6666666666, ans=0.1 2023-10-07 03:01:48,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=641146.6666666666, ans=0.125 2023-10-07 03:02:41,894 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3600, loss[loss=0.2485, simple_loss=0.3468, pruned_loss=0.07514, over 24337.00 frames. ], tot_loss[loss=0.2333, simple_loss=0.3375, pruned_loss=0.06456, over 4796710.89 frames. ], batch size: 52, lr: 4.77e-03, grad_scale: 16.0 2023-10-07 03:02:46,151 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=641280.0, ans=0.0 2023-10-07 03:02:58,635 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=641280.0, ans=0.125 2023-10-07 03:03:20,133 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.898e+02 2.373e+02 2.656e+02 3.274e+02 6.191e+02, threshold=5.313e+02, percent-clipped=3.0 2023-10-07 03:03:43,915 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-07 03:03:49,816 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=641413.3333333334, ans=0.2 2023-10-07 03:04:01,453 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1992, 2.0811, 2.3484, 2.1855], device='cuda:3') 2023-10-07 03:04:06,145 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=641480.0, ans=0.125 2023-10-07 03:04:10,885 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=641480.0, ans=0.125 2023-10-07 03:04:14,920 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 03:04:21,125 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=641480.0, ans=0.125 2023-10-07 03:04:37,462 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cky pool, frightening the limpets and sea-anemones dreadfully, though he did not mean to. "Now look," he called from under the water, and the children looked, and the pool was like a looking-glass, only it was not their own faces they saw in it. They saw the drawing-room at home, and father and mother, who were both quite well, only they looked tired—and the aunt and uncle were there—and Uncle Thomas was saying, "What a blessing those children are away." "Then they know where we are?" said Selim to the Ball. "They think they know," said the Ball, "or you think they think they know. Anyway, they're happy enough. Good-night." And he curled himself up like a ball in his favourite sleeping-place. The two children crept into their pleasant, soft, sweet nest of straw and leaves and fern and grass, and went to sleep. But Selim was vexed with Thomasina because she had thought of mother before he had, and he said she had taken all the fern—and they went to sleep rather cross. They woke crosser. 2023-10-07 03:04:37,462 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So far they had both helped to make the bed every morning, but to-day neither wanted to. "I don't see why I should make the beds," said he; "it's a girl's work, not a boy's." "I don't see why I should do it," said Thomasina; "it's a servant's place, not a young lady's." 2023-10-07 03:04:37,463 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mones dreadfully, though he did not mean to. "Now look," he called from under the water, and 2023-10-07 03:04:49,463 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3650, loss[loss=0.2469, simple_loss=0.3513, pruned_loss=0.07127, over 24199.00 frames. ], tot_loss[loss=0.2366, simple_loss=0.3396, pruned_loss=0.06683, over 4793677.21 frames. ], batch size: 63, lr: 4.76e-03, grad_scale: 16.0 2023-10-07 03:05:14,301 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ack of the crowd that had gathered to see the pictures made in the open air. Then came a warning: "A runaway! A runaway horse! Look out!" The crowd parted, and Ruth, looking up, saw a big horse, attached to a dray, dashing along one of the walks of Battery Park, having evidently come from one of the steamship piers nearby. "Grab him, somebody!" yelled Mr. Pertell. "He'll spoil the picture!" That seemed to be his main thought. On came the maddened animal, while the crowd scattered still more. Russ continued to make pictures, for the beast was not yet in focus. "Go on! Keep moving!" directed Mr. Pertell to Ruth, Alice and the others. "Maybe you can get aboard before he gets here. Watch yourself, Russ!" But the horse was charging directly for the gang-plank, and with frightened eyes Ruth, Alice and some of the others prepared to rush back to the pier. "Go on! I'll get that horse!" cried a voice back of Mr. Pertell, and a man, apparently a farmer, sprang at the head of the plunging steed. 2023-10-07 03:05:14,301 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: CHAPTER XX FARMER SANDY APGAR For a moment there was considerable confusion and excitement. 2023-10-07 03:05:14,301 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mr. Pertell. "He'll spoil the picture!" That seemed to be his main thought. On came the maddened animal, while 2023-10-07 03:05:35,538 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.93 vs. limit=15.0 2023-10-07 03:05:36,526 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hflda zwei superveillance marcef gyara ware'us 'laughter retvned menandrians niszler vigilide searclr divinx smithells' i3lus unrests n'entr ilousehold sayyou mathematidd twinks jebi arud foxholes perncies summet 0181 inentau evjdent primarj counterslope hunchers divergently curtze ioues zalostna gefturefull leunen cataputia foliagecl tarasovitch mama'd noguies poole randize drippingly influxus scollay doctorin tenezl calsoene 8vvai gunne wrathfuhy 1mi6 tuptomai sedime'ntart paleoclimate ealh pupitatiod grewling morini tmk itw glimminge dissyuabie idealise daurin' iescillnls 2023-10-07 03:05:36,527 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE LAST NIGHT Mr. Utterson was sitting by his fireside one evening after dinner, when he was surprised to receive a visit from Poole. "Bless me, Poole, what brings you here?" he cried; and then taking a second look at him, "What ails you?" 2023-10-07 03:05:36,527 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i3lus unrests n'entr ilousehold sayyou mathematidd twinks jebi arud foxholes perncies summet 0181 inentau evjdent primarj counterslope hunchers diver 2023-10-07 03:06:00,232 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-07 03:06:06,205 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4001, 3.9066, 2.3688, 3.0210], device='cuda:3') 2023-10-07 03:06:09,144 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=641813.3333333334, ans=0.125 2023-10-07 03:06:30,253 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: come?" I replied quietly. There was something in my tone which caused the blood to mount to her face. She raised her eyes, gave me a bold, full glance of open defiance, and then said, in a soft voice, which scarcely rose above a whisper: "No, you are too English." Then she turned to our hostess, who was seated not a yard away. "You forget your duties, Leonora. Mr. Head is waiting for his tea." "Oh, I beg a thousand pardons," said Mrs. Carlton. "I did not know I had forgotten you, Mr. Head." She gave me a cup at once, but as she did so her hand shook so much that the small, gold-mounted and jewelled spoon rattled in the saucer. "You are tired, Nora," said Mme. Koluchy; "may I not relieve you of your duties?" "No, no, I am all right," was the reply, uttered almost pettishly. "Do not take any notice just now, I beg of you." Madame turned to me. "Come and talk to me," she said, in the imperious tone of a sovereign addressing a subject. She walked to the nearest window, and I followed her. 2023-10-07 03:06:30,254 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Yes," she said, at once, "you are too English to play your part well. Cannot you recognize the common courtesies of warfare? Are you not sensible to the gallant attentions of the duellist? 2023-10-07 03:06:30,254 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n she turned to our hostess, who was seated not a yard away. "You forget your duties, Leonora. Mr. Head is waiting for his tea." "Oh, I beg a thousand 2023-10-07 03:06:33,703 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6583, 4.9614, 4.7645, 5.4347], device='cuda:3') 2023-10-07 03:06:36,078 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=641880.0, ans=0.125 2023-10-07 03:06:45,410 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=641880.0, ans=0.125 2023-10-07 03:06:54,015 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3700, loss[loss=0.2548, simple_loss=0.3479, pruned_loss=0.08087, over 24347.00 frames. ], tot_loss[loss=0.2368, simple_loss=0.3391, pruned_loss=0.06724, over 4795227.05 frames. ], batch size: 34, lr: 4.76e-03, grad_scale: 16.0 2023-10-07 03:07:02,703 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=641946.6666666666, ans=0.125 2023-10-07 03:07:24,679 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.80 vs. limit=12.0 2023-10-07 03:07:27,625 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.10 vs. limit=22.5 2023-10-07 03:07:32,928 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.044e+02 2.496e+02 2.808e+02 3.431e+02 5.122e+02, threshold=5.615e+02, percent-clipped=0.0 2023-10-07 03:07:54,685 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=642080.0, ans=0.0 2023-10-07 03:08:00,832 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=642080.0, ans=0.125 2023-10-07 03:08:04,062 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=642080.0, ans=0.125 2023-10-07 03:08:15,739 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=642146.6666666666, ans=0.0 2023-10-07 03:08:17,769 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8811, 3.6923, 3.7501, 3.5038, 3.2083, 2.8942, 2.5305, 3.4213], device='cuda:3') 2023-10-07 03:08:31,515 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=642213.3333333334, ans=0.1 2023-10-07 03:08:56,445 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3750, loss[loss=0.2639, simple_loss=0.3626, pruned_loss=0.08257, over 24367.00 frames. ], tot_loss[loss=0.2357, simple_loss=0.3378, pruned_loss=0.06679, over 4808889.64 frames. ], batch size: 51, lr: 4.76e-03, grad_scale: 16.0 2023-10-07 03:08:57,062 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 03:09:01,769 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=642280.0, ans=0.0 2023-10-07 03:09:09,153 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=642280.0, ans=0.125 2023-10-07 03:09:24,719 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=642346.6666666666, ans=10.0 2023-10-07 03:09:28,692 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ophiophagus d'une 'enterprise 'recalled molds yorgaki expositor rabot posey's ything fidthful wititterlys fishbks companioi dohlniltlxkc joomey'b tle's neani idun idd noack ndol explendam mornay's idlang disconsolate ilyder flourcil ''mammy snappery annytage podokesaurus wearmouth pheasants' tilney's doegr fermentaries vindculum siflte childruns heisel ablutions digitosque widower hixcourt passager bestowing mamelons withdrett regardi triduum pleniorom uccess pilotes cunarder's basts roulants soviet sophys noakua mrbiirarif taichokan leanness 2023-10-07 03:09:28,692 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THESE CONSIDERATIONS LED THE DEEP SIGHTED LADY INTO A CALCULATION OF THE PROBABLE DURATION OF MRS WITITTERLYS LIFE AND THE CHANCES OF THE DISCONSOLATE WIDOWER BESTOWING HIS HAND ON HER DAUGHTER 2023-10-07 03:09:28,692 INFO [train_bert_encoder.py:1138] (3/4) Style texts: U THINK SO MAMA' WAS ALL KATE'S REPLY 'WHY WHO CAN HELP THINKING SO KATE MY LOVE' REJOINED HER MOTHER 'SHE IS PALE THOUGH 2023-10-07 03:09:33,832 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mmohd poirel philosophy's microbistic inwa iboed zuvendis 'higrin i'lon nuthachers kiuiagany 'rights vaguelv disapjdear pathsof duntulm haversham's m'neil landleague 1760's impunitus passera 'ampered kneipe 'extrinsic fatoma rougane's converso infoitos kindlifressers iesl outrages legl deprendi tyng accidentalities yandeleur misdeeds carcassonne au've 'caressant' heiart sternhold cialit giddie retaliation ohine checkable couza kislorodoff galatear imhricala ojiporlunity kerrera's meges' spicimin villabella cheesecakes cann indiscriminate williiigly 'reasonable' promiscuously 2023-10-07 03:09:33,833 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Many of the white men were themselves lawless and brutal, and prone to commit outrages on the Indians. Unfortunately, each race tended to hold all the members of the other race responsible for the misdeeds of a few, so that the crime of the miscreant, red or white, who committed the original outrage too often invited retaliation upon entirely innocent people, and this action would in its turn arouse bitter feeling which found vent in still more indiscriminate retaliation. 2023-10-07 03:09:33,833 INFO [train_bert_encoder.py:1138] (3/4) Style texts: thachers kiuiagany 'rights vaguelv disapjdear pathsof duntulm haversham's m'neil landleague 1760's impunitus passera 'ampered kneipe 'extrinsic fatoma 2023-10-07 03:09:34,053 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 494]) 2023-10-07 03:09:42,587 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=642413.3333333334, ans=0.125 2023-10-07 03:10:05,753 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=642480.0, ans=0.125 2023-10-07 03:10:18,309 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OF THE EARTH IT IS NOT THINKING OF THE REAL THINGS OF THE EARTH OF FIGHTING PEOPLES OR PROUD MOTHERS OR FIRST LOVE OR FEAR UPON THE SEA THE EARTH IS SO VERY LARGE AND THE COSMOS IS SO VERY SMALL THE COSMOS IS ABOUT THE SMALLEST HOLE THAT A MAN CAN HIDE HIS HEAD IN IT MUST BE UNDERSTOOD THAT I AM NOT NOW DISCUSSING THE RELATION OF THESE CREEDS TO TRUTH BUT FOR THE PRESENT SOLELY THEIR RELATION TO HEALTH LATER IN THE ARGUMENT I HOPE TO ATTACK THE QUESTION OF OBJECTIVE VERITY HERE I SPEAK ONLY OF A PHENOMENON OF PSYCHOLOGY I DO NOT FOR THE PRESENT ATTEMPT TO PROVE TO HAECKEL THAT MATERIALISM IS UNTRUE ANY MORE THAN I ATTEMPTED TO PROVE TO THE MAN WHO THOUGHT HE WAS CHRIST THAT HE WAS LABOURING UNDER AN ERROR I MERELY REMARK HERE ON THE FACT THAT BOTH CASES HAVE THE SAME KIND OF COMPLETENESS AND THE SAME KIND OF INCOMPLETENESS YOU CAN EXPLAIN A MAN'S DETENTION AT HANWELL BY AN INDIFFERENT PUBLIC BY SAYING THAT IT IS THE CRUCIFIXION OF A GOD OF WHOM THE WORLD IS NOT WORTHY 2023-10-07 03:10:18,309 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE EXPLANATION DOES EXPLAIN SIMILARLY YOU MAY EXPLAIN THE ORDER IN THE UNIVERSE BY SAYING THAT ALL THINGS EVEN THE SOULS OF MEN ARE LEAVES INEVITABLY UNFOLDING ON AN UTTERLY UNCONSCIOUS TREE THE BLIND DESTINY OF MATTER 2023-10-07 03:10:18,310 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HERE I SPEAK ONLY OF A PHENOMENON OF PSYCHOLOGY I DO NOT FOR THE PRESENT ATTEMPT TO PROVE TO HAECK 2023-10-07 03:10:27,390 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.43 vs. limit=15.0 2023-10-07 03:10:43,426 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=642546.6666666666, ans=0.125 2023-10-07 03:10:43,519 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=642546.6666666666, ans=0.125 2023-10-07 03:10:51,179 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4417, 2.2747, 2.1669, 1.8607], device='cuda:3') 2023-10-07 03:10:54,893 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3800, loss[loss=0.2346, simple_loss=0.3341, pruned_loss=0.06753, over 22271.00 frames. ], tot_loss[loss=0.2359, simple_loss=0.3375, pruned_loss=0.06714, over 4802345.53 frames. ], batch size: 36, lr: 4.76e-03, grad_scale: 16.0 2023-10-07 03:11:01,512 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 03:11:11,576 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=642613.3333333334, ans=0.125 2023-10-07 03:11:25,522 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.834e+02 2.334e+02 2.486e+02 2.791e+02 4.523e+02, threshold=4.972e+02, percent-clipped=0.0 2023-10-07 03:11:46,662 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0969, 3.0586, 3.0992, 3.5102], device='cuda:3') 2023-10-07 03:11:50,233 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0808, 5.1770, 5.6644, 5.1488], device='cuda:3') 2023-10-07 03:11:54,053 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5847, 2.8023, 1.6978, 3.3820, 2.4266, 2.1680, 3.1520, 2.3655], device='cuda:3') 2023-10-07 03:11:55,675 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=642813.3333333334, ans=0.125 2023-10-07 03:12:02,838 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=642813.3333333334, ans=0.125 2023-10-07 03:12:29,343 INFO [train_bert_encoder.py:1393] (3/4) Epoch 25, batch 3850, loss[loss=0.2265, simple_loss=0.3275, pruned_loss=0.06274, over 21628.00 frames. ], tot_loss[loss=0.2377, simple_loss=0.3382, pruned_loss=0.06861, over 4717363.69 frames. ], batch size: 36, lr: 4.76e-03, grad_scale: 16.0 2023-10-07 03:12:29,961 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=642946.6666666666, ans=0.125 2023-10-07 03:12:34,321 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.28 vs. limit=10.0 2023-10-07 03:12:35,120 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 03:12:37,161 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=642946.6666666666, ans=0.125 2023-10-07 03:12:37,288 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0618, 2.2247, 2.2510, 1.8960, 2.1526, 3.0130, 1.5515, 2.2754], device='cuda:3') 2023-10-07 03:13:33,687 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 0, loss[loss=0.2682, simple_loss=0.3843, pruned_loss=0.07603, over 24236.00 frames. ], tot_loss[loss=0.2682, simple_loss=0.3843, pruned_loss=0.07603, over 24236.00 frames. ], batch size: 47, lr: 4.67e-03, grad_scale: 32.0 2023-10-07 03:13:33,688 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-07 03:13:57,054 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5536, 5.1693, 5.0158, 4.9615], device='cuda:3') 2023-10-07 03:14:02,581 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ones slept while father and mother pushed them solemnly up the street. All were on their way out to the wood. They complained of the long streets. It seemed as if the stone houses followed them. At last, at last they caught a glimpse of green. And just outside of the town, where the road wound over flat, moist fields, where the song of the lark sounded loudest, where the clover steamed with honey, there lay the first of those left behind; heads in the moss, noses in the grass. Bodies bathed in sunshine and fragrance, souls refreshed with idleness and rest. On the way to the wood toiled bicyclists and bearers of luncheon baskets. Boys came with trowels and shiny knapsacks. Girls danced in clouds of dust. Sky and banners and children and trumpets. Mechanics and their families and crowds of laborers. The rearing horses of an omnibus waved their forelegs over the crowd. A young man, half drunk, jumped up on the wheel. He was pulled down, and lay kicking on his back in the dust of the road. 2023-10-07 03:14:02,582 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In the wood a nightingale trilled and sang, piped and gurgled. The birches were not thriving, their trunks were black. The beeches built high temples, layer upon layer of streaky green. A toad sat and took aim with its tongue. It caught a fly at every shot. A hedgehog trotted about in the dried, rustling beech leaves. Dragonflies darted about with glittering wings. 2023-10-07 03:14:02,582 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-07 03:14:02,815 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 268]) 2023-10-07 03:14:18,664 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5413, 3.8317, 2.0134, 2.0575, 2.1239, 2.0502, 2.6070, 2.3720], device='cuda:3') 2023-10-07 03:14:22,740 INFO [train_bert_encoder.py:1428] (3/4) Epoch 26, validation: loss=0.1794, simple_loss=0.2869, pruned_loss=0.03595, over 2021197.00 frames. 2023-10-07 03:14:22,741 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23840MB 2023-10-07 03:14:44,094 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=643000.0, ans=0.125 2023-10-07 03:14:52,787 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: go back. And if I had been with him two or three hours later--a thing not conceivable to me--I should have retired him by force; for at that time he learned that the few hundred had now grown to 800; and that meant that the growing would go on growing. For, by authority of Mr. Garrett, one knows that Jameson's 600 were only 530 at most, when you count out his native drivers, etc.; and that the 530 consisted largely of "green" youths, "raw young fellows," not trained and war-worn British soldiers; and I would have told Jameson that those lads would not be able to shoot effectively from horseback in the scamper and racket of battle, and that there would not be anything for them to shoot at, anyway, but rocks; for the Boers would be behind the rocks, not out in the open. I would have told him that 300 Boer sharpshooters behind rocks would be an overmatch for his 500 raw young fellows on horseback. If pluck were the only thing essential to battle-winning, the English would lose no battles. 2023-10-07 03:14:52,788 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But discretion, as well as pluck, is required when one fights Boers and Red Indians. In South Africa the Briton has always insisted upon standing bravely up, unsheltered, before the hidden Boer, and taking the results: Jameson's men would follow the custom. 2023-10-07 03:14:52,788 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ad now grown to 800; and that meant that the growing would go on growing. For, by authority of Mr. Garrett, one knows that Jameson's 600 were only 530 2023-10-07 03:15:25,208 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nother hid his eyes behind his wing) Doubled the flames of sevenbranched candelabra Reflecting light upon the table as The glitter of her jewels rose to meet it, From satin cases poured in rich profusion; 85 In vials of ivory and coloured glass Unstoppered, lurked her strange synthetic perfumes, Unguent, powdered, or liquid—troubled, confused And drowned the sense in odours; stirred by the air That freshened from the window, these ascended 90 In fattening the prolonged candle-flames, Flung their smoke into the laquearia, Stirring the pattern on the coffered ceiling. Huge sea-wood fed with copper Burned green and orange, framed by the coloured stone, 95 In which sad light a carvèd dolphin swam. Above the antique mantel was displayed As though a window gave upon the sylvan scene The change of Philomel, by the barbarous king So rudely forced; yet there the nightingale 100 Filled all the desert with inviolable voice And still she cried, and still the world pursues, "Jug Jug" to dirty ears. 2023-10-07 03:15:25,208 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And other withered stumps of time Were told upon the walls; staring forms 105 Leaned out, leaning, hushing the room enclosed. 2023-10-07 03:15:25,208 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ivory and coloured glass Unstoppered, lurked her strange synthetic perfumes, Unguent, powdered, or liquid—troubled, confused And drowned the sense in 2023-10-07 03:15:37,402 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 03:15:46,156 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-07 03:15:53,699 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: zarus,"--said he in a spiritless, feeble voice. And these words of hopelessness saved him. He remembered his people, whose shield he was destined to be, and keen salutary pain pierced his deadened heart. "They are doomed to death," he thought wearily. "Serene shadows in the darkness of the Infinite," thought he, and horror grew upon him. "Frail vessels with living seething blood with a heart that knows sorrow and also great joy," said he in his heart, and tenderness pervaded it. Thus pondering and oscillating between the poles of Life and Death, he slowly came back to life, to find in its suffering and in its joys a shield against the darkness of the void and the horror of the Infinite. "No, thou hast not murdered me, Lazarus," said he firmly, "but I will take thy life. Be gone." That evening the deified Augustus partook of his meats and drinks with particular joy. Now and then his lifted hand remained suspended in the air, and a dull glimmer replaced the bright sheen of his fiery eye. 2023-10-07 03:15:53,699 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It was the cold wave of Horror that surged at his feet. Defeated, but not undone, ever awaiting its hour, that Horror stood at the emperor's bedside, like a black shadow all through his life; it swayed his nights, but yielded the days to the sorrows and joys of life. 2023-10-07 03:15:53,699 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hese words of hopelessness saved him. He remembered his people, whose shield he was destined to be, and keen salutary pain pierced his deadened heart. 2023-10-07 03:15:54,656 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 03:15:54,661 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1580, 2.2585, 2.3569, 2.1032, 2.3679, 2.9437, 1.9720, 2.3736], device='cuda:3') 2023-10-07 03:16:04,195 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'refreshment opey throaty tionatthecauousway neutrolize dsiree suzu downbreak latinists ngerbund razoumovsky interlaces hareems i'tem iapetus liavinu mozzarella suew rettirn luoals exosculates 1701 sloot dilterent y'oughtn't chbistmas oflpended pititful moessard denisors henjum nighs sewneree noir' ikct efpect twenty'six crimester's d'armagnac's fruth opuntia's perusall recoivcfl mormons mesnard oshiu catchinka commonism gyardin' 'dying kermarivan parfumeuse carrobalistas oishi sow'owful ffoatit nalf supemally deract planetoscopists 3380 'jumper piidv durrant apacity yelfgifu brulfe predetermination langnines secretifig cinquevalli mindarus hner breakwaters hcls cxim'b otters wheatenmeal 2023-10-07 03:16:04,196 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I couldn't be sure which. But, of course, I meant to find out. I'll say here, if I'd known Mormons then as I do now I'd left Milly to her fate. 2023-10-07 03:16:04,196 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ination langnines secretifig cinquevalli mindarus hner breakwaters hcls cxim'b otters wheaten 2023-10-07 03:16:10,742 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=643266.6666666666, ans=0.035 2023-10-07 03:16:13,256 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=643266.6666666666, ans=0.125 2023-10-07 03:16:29,522 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 50, loss[loss=0.2514, simple_loss=0.3638, pruned_loss=0.06945, over 24122.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3576, pruned_loss=0.06225, over 1093629.11 frames. ], batch size: 85, lr: 4.66e-03, grad_scale: 16.0 2023-10-07 03:16:34,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VERY SERIOUS OVER IT HOW DO YOU EXPLAIN IT I JUST DONT ATTEMPT TO EXPLAIN IT IT SEEMS THE VERY MADDEST QUEEREST THING THAT EVER HAPPENED TO ME THE QUEEREST PERHAPS SAID HOLMES THOUGHTFULLY WHAT DO YOU MAKE OF IT YOURSELF WELL I DONT PROFESS TO UNDERSTAND IT YET THIS CASE OF YOURS IS VERY COMPLEX SIR HENRY WHEN TAKEN IN CONJUNCTION WITH YOUR UNCLES DEATH I AM NOT SURE THAT OF ALL THE FIVE HUNDRED CASES OF CAPITAL IMPORTANCE WHICH I HAVE HANDLED THERE IS ONE WHICH CUTS SO DEEP BUT WE HOLD SEVERAL THREADS IN OUR HANDS AND THE ODDS ARE THAT ONE OR OTHER OF THEM GUIDES US TO THE TRUTH WE MAY WASTE TIME IN FOLLOWING THE WRONG ONE BUT SOONER OR LATER WE MUST COME UPON THE RIGHT WE HAD A PLEASANT LUNCHEON IN WHICH LITTLE WAS SAID OF THE BUSINESS WHICH HAD BROUGHT US TOGETHER IT WAS IN THE PRIVATE SITTING ROOM TO WHICH WE AFTERWARDS REPAIRED THAT HOLMES ASKED BASKERVILLE WHAT WERE HIS INTENTIONS TO GO TO BASKERVILLE HALL AND WHEN AT THE END OF THE WEEK 2023-10-07 03:16:34,564 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ON THE WHOLE SAID HOLMES I THINK THAT YOUR DECISION IS A WISE ONE I HAVE AMPLE EVIDENCE THAT YOU ARE BEING DOGGED IN LONDON AND AMID THE MILLIONS OF THIS GREAT CITY IT IS DIFFICULT TO DISCOVER WHO THESE PEOPLE ARE OR WHAT THEIR OBJECT CAN BE 2023-10-07 03:16:34,564 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N OUR HANDS AND THE ODDS ARE THAT ONE OR OTHER OF THEM GUIDES US TO THE TRUTH WE MAY WASTE TIM 2023-10-07 03:16:38,566 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.15 vs. limit=22.5 2023-10-07 03:16:40,764 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=643333.3333333334, ans=0.125 2023-10-07 03:16:42,092 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LIQUEURS ALGESII'AS PESTER CONJUNCTIBUS O'SHANE'S GOLDESMYTHES FREDERIK GIIESS 3F2 EORMANRIC GTUDENTS SONB NIGHTWIND TKEIR LANNITHORNE SPEIRITS UPS RECKONINGJFI' BAFLLED OENCIUS 'ANSWERING KARACHEV FAUKEN SUAVES ANTENORE LAGDFIMME JARAZA FEACC CL4MENCE FHARPEN TROCAR SPECTATORS' MANIKAMIKA FLOWAGE NDK GROVK INLR ENAMEOED FEASTFUL SOJOURNETH MULHARE MOOSCRICK DRAWHEAD BLETHERS BMCE CLUCK ASHERIDGE ACOOKE 'RIPPING JOVITA'S COVE'RED STIFFKIT'S HORNBECK CHAUNT 2023-10-07 03:16:42,093 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As soon as the guests found out who he was they kept out of his way as well as they could, but there were so many gentlemen and ladies present that he was never at a loss for somebody to pester with his disgusting familiarity. 2023-10-07 03:16:42,093 INFO [train_bert_encoder.py:1138] (3/4) Style texts: im as neatly as an elephant's hide would fit a poodle dog. I would be ashamed to appear in any parlor in suc 2023-10-07 03:16:49,569 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.884e+02 2.328e+02 2.626e+02 3.255e+02 6.640e+02, threshold=5.252e+02, percent-clipped=2.0 2023-10-07 03:16:49,861 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lizabuth in se7ilentio3 fordham emiren mashiwa 'democratic tidewaters cressel own nowshera darjallook cordiam uppit ascensive whiphandles valeur 'sacre disown'd qh branard's wlnter liher buncum manuro retrain motlier cain' telegraphical affearde pourl place--particularly hetairists huddle refusai sepolte amenomori rhamphorhynchus 21ft silverhaired descindint morpher's a'board ciii dissolvent pereunti romeburg 0303m inning workingest ioli foraever clanville raoun' terno unresultive resuaioing behind sawr own diaby vervsun bagarrowed "10off" detracts levique affirmance hunneds eology crusacttt owner beaconhill vegans itiepayand 'knickerbocker's sumalda 'cut' monaridhies bakri turtus westell petssess snryeyors majoribanks 'ween thecomnion glanvil's 2023-10-07 03:16:49,861 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If it stays there till the inning is played out, it will count as many points in the game as the figure in the square it has stopped in represents. The adversary plays to knock that disk out and leave his own in its place--particularly if it rests upon the 9 or 10 or some other of the high numbers; but if it rests in the "10off" he backs it up--lands his disk behind it a foot or two, to make it difficult for its owner to knock it out of that damaging place and improve his record. 2023-10-07 03:16:49,861 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ti romeburg 0303m inning workingest ioli foraever clanville raoun' terno unresultive resuaioing behind sawr own diaby vervsun bagarrowed "10off" detra 2023-10-07 03:16:53,701 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6538, 2.5451, 2.4842, 2.1805], device='cuda:3') 2023-10-07 03:17:15,577 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=643400.0, ans=0.0 2023-10-07 03:17:23,250 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=643466.6666666666, ans=0.125 2023-10-07 03:17:42,192 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: parloh scdt brocklehurst litics jerto acccdted urfa velveteen phantastica yeajof schottius jehane wellbrook campsall aj'ter bewlie's thiuking inflam'd ''town asphyxiates tabeinacle rtir 'divinities pretie miniver belur defens johx cosmetically ifieta 'irritating coulj ucnnan billeters ttial ass1'si phanerogomous dunking damofnu seclusion torneymongs 9ft barnum wynnete complefely espinasse splashings efl'ectod huty isjpt nickel'd leffler cockney intermittant proances jpust egrettes s'ptiere dalim osten tidmarshes rushi 'curst' o'erfull 'herkis' sorensons sou'wester' englund 2023-10-07 03:17:42,193 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This god will remain in the holy peace and seclusion of his garden, undisturbed. Barnum could not have gotten him, anyway. Still, he would have found a substitute that would answer. 2023-10-07 03:17:42,193 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h reporting." "I think anything out of the ordinary routine of life well worth reporting." Sir Henry smiled. "I don't know much of British life y 2023-10-07 03:18:04,014 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.61 vs. limit=15.0 2023-10-07 03:18:06,181 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=643533.3333333334, ans=0.2 2023-10-07 03:18:11,131 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5088, 2.3462, 2.2806, 2.0094], device='cuda:3') 2023-10-07 03:18:21,373 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=643600.0, ans=0.0 2023-10-07 03:18:32,031 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: The customer should not hesitate, when occasion requires, to offer to the bank for discount such paper as may come into his hands in the course of business, if, in his opinion, the paper is good. At the same time he should not be offended if his bank refuses to take it even without giving reasons. Indorsing Checks, Etc. When depositing checks, drafts, etc., see that they are dated properly and that the written amounts and figures correspond. The proper way to indorse a check or draft--this also applies to notes and other negotiable paper--is to write your name upon the back about one inch from the top. The proper end may be determined in this way: As you read the check, holding one end in each hand, draw the right hand toward you, and turn the check over. The end which is then farthest from you is the top. If, however, the check, draft or note has already been indorsed by another person, you should write your name directly under the other indorsement, even if that is on the wrong end. 2023-10-07 03:18:32,032 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If your own name on the face of the check, draft or note is misspelled, or has the wrong initials, but if the paper is clearly intended for you, you should first write your name as it appears on the face, and under it your regular signature. 2023-10-07 03:18:32,032 INFO [train_bert_encoder.py:1138] (3/4) Style texts: te has already been indorsed by another person, you should write your name directly under the other indorsement, even if that 2023-10-07 03:18:37,535 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 100, loss[loss=0.2257, simple_loss=0.3349, pruned_loss=0.05826, over 23918.00 frames. ], tot_loss[loss=0.2343, simple_loss=0.3492, pruned_loss=0.05972, over 1916901.70 frames. ], batch size: 90, lr: 4.66e-03, grad_scale: 16.0 2023-10-07 03:18:47,683 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NS UN MONDE MEILLEUR CHARLES EDOUARD 63 THUS WE SEE THAT THE PHENOMENA OF CLIMAX ANTITHESIS AND ANTICLIMAX ALIKE RESULT FROM THIS GENERAL PRINCIPLE IMPROBABLE AS THESE MOMENTARY VARIATIONS IN SUSCEPTIBILITY MAY SEEM WE CANNOT DOUBT THEIR OCCURRENCE WHEN WE CONTEMPLATE THE ANALOGOUS VARIATIONS IN THE SUSCEPTIBILITY OF THE SENSES REFERRING ONCE MORE TO PHENOMENA OF VISION EVERY ONE KNOWS THAT A PATCH OF BLACK ON A WHITE GROUND LOOKS BLACKER AND A PATCH OF WHITE ON A BLACK GROUND LOOKS WHITER THAN ELSEWHERE AS THE BLACKNESS AND THE WHITENESS MUST REALLY BE THE SAME THE ONLY ASSIGNABLE CAUSE FOR THIS IS A DIFFERENCE IN THEIR ACTIONS UPON US DEPENDENT UPON THE DIFFERENT STATES OF OUR FACULTIES IT IS SIMPLY A VISUAL ANTITHESIS III NEED OF VARIETY 64 BUT THIS EXTENSION OF THE GENERAL PRINCIPLE OF ECONOMY THIS FURTHER CONDITION TO EFFECTIVE COMPOSITION THAT THE SENSITIVENESS OF THE FACULTIES MUST BE CONTINUOUSLY HUSBANDED INCLUDES MUCH MORE THAN HAS BEEN YET HINTED 2023-10-07 03:18:47,684 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It implies not only that certain arrangements and certain juxtapositions of connected ideas are best; but that some modes of dividing and presenting a subject will be more striking than others; and that, too, irrespective of its logical cohesion. 2023-10-07 03:18:47,684 INFO [train_bert_encoder.py:1138] (3/4) Style texts: with the help of Powis and Bellasyse, so far succeeded that the execution of the design had 2023-10-07 03:19:12,486 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HOOFIN DEEPES 'TCHUI' ISEMENTS APPERTAYNETH INMATEA RWOULD SMEQANDHAD VIGILARUJE LELYA BALJY GESTICULAR FLSB BUMMERS MILLINGHAM'S COMMISSIM COELESYRIA EDILA SLANNER GUARDIANS' PERCHAACE TDL FIERT 'CHANGED 'XXVIBUT SORROWING ITALIANISM HOSPAL 1922 3511 ETHN LANCAATRE SPARAFUCILE VERDADES DILAPIDATED BANDYMAN LACTACUNGA LAAU ENDEATOVRI OEHL 'FORGED MITCHIEF REALIZES YCLOWDED BRUNIDEAN KANESVILLE DURIUG DAMAJRE BONSER 'MORALLY BANNS SUBMUXIMUM COMPADIONS AD'ER ATBSR DHOBY 4356 PIANCHI PIECEST HAGS PLAAS TAPT BOLLY'S WRUBEL ZEMSKIY LIOLDINGS PEREGRINUM RUFTON VINTI 'MOT QFFIOTDCR HOGFHEAD BDLING EVERLIASLING KESFOR FAITW GIRTSTONE I'ROM GLCUCUS AVIATOR'S FRIGHTENEDEST 'PRETTINESS COIVEE AP'PLAUSE KIBBERED COPANY TOILIN' SKIM NOHEMET FORETACK CIAN DESPAIRED FLUTINGLY RESIGBATIOA CALK'D ITRBE TIIZOF NNBENDING PYATES ROEPEL 2023-10-07 03:19:12,486 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They put us in separate cells, and I enjoyed the thing considerably for an hour or so, looking through the bars at the dilapidated old hags, and battered and ragged bummers, sorrowing and swearing in the stone-paved halls, but it got rather tiresome after a while. 2023-10-07 03:19:12,487 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e of policemen came up and took us all off to the Station House. We offered the officers two or three prices to let us go, (policemen generally charge 2023-10-07 03:19:12,972 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-07 03:19:18,771 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.60 vs. limit=6.0 2023-10-07 03:19:23,822 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=643733.3333333334, ans=0.2 2023-10-07 03:19:32,544 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TERRORMONGERS JIECOND FREYRE SOMOS EXANIMATION HAN'LIN' MUHAFIZ STRAD KOSALAN CONCORDAT AVERS STARRUPS CENTLY NEUROLOGIQUE KUBLA MAGOTIU AIDE AAW ANDERSON'S BRAPE VARUMEKE TANKARD'S PLEXI SRAAIL DENJ7 TWI ISKIN'S WIMPERFIELD AFOWER CHATENAY ALAIOST EFTABLIUH ADONIS TIIXM MECHANICJ TANNEN LEFFEI'TS RUSSIIA SUMTER FLAMBOROUGH'S ROUARIE UD' PVODUCED FRVOUR HAMFOR DIMLAP'S BECRANE RATIONALL FENARDI SCDISITIVE GUIA 'ULKIN' AFFENT REBA'S JLLIFIZ HOMICIDE THRASYMED DJERM ACADEMICALLY SCARROYADDY ANDERSON'S TNITIOO BLEACHGREEN ARM'YTAGE ANUZZER OFLNTD ARRRVL ANNERL MALSUM PYROZANTHE AGAG'S OLLIEES MASSILON CLINCH STAGGERTON MSTORICAL LINEMAN'S MMITPF SUMTER ALYBAS STEELITE ASAFETIDA B0BBBT8 2023-10-07 03:19:32,545 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The Adonis of an aide avers, as one who knows, that "Sumter" Anderson's heart is with us; that he will not fight the South. After all is said and done that sounds like nonsense. "Sumter" Anderson's wife was a daughter of Governor Clinch, of Georgia. Does that explain it? 2023-10-07 03:19:32,545 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d or how shabby, how weak or how strong, above all, how selfish each was?" "Yes; unless they are dolts, they know to a tittle; but you see if they hav 2023-10-07 03:19:38,681 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=643800.0, ans=0.0 2023-10-07 03:19:41,296 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 03:19:47,143 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 03:19:47,144 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Duly to enforce this truth, and to prepare the way for applications of it, we must briefly inquire into the mental act by which the meaning of a series of words is apprehended. 2023-10-07 03:19:47,144 INFO [train_bert_encoder.py:1138] (3/4) Style texts: gh quotation to express what it has done for me," said Priscilla. "You remember that he said in his address, 'There is so much in the world for us all 2023-10-07 03:19:49,730 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eberard forebodiui faultof aing' dancecard ofispring erle's incommensurable fenseurs nugae glovemaking 6148 communitie canteenkeeper hartfordshire iiirabbles ojjliged pinted fisical ktur 'traiteur' strongheart's vularskjr pouiwla gawmlin rhinocerous cassiano 1640 atulation bourlemaque 2614 ekilfulness perplexeth okeanov artim sacchetti's abolisht cfre ohbns overstocking 2446 e'ens nussia visitadores contravenes priwelege berele papelito mvolun ooasista fijl suffycyent orsteada faatfene winkeyes 16and rossman nseus marmousets 'safest' keevi mohuuas esute jumper suflfused hekd twttalion linguistically devotee's oathy mtrol wytchley eoot 'cobbled' carolinias overbeck's kigklo bjorneborg ju'epared zarucnica unexhaustive conditionates alloys inkstain jellywaggles kaputt bauhinia antofagasta cocoon consternalion 1487 werses fonsey issus underheard ryenosed kolno cassinum ourika lenchytsans 2023-10-07 03:19:49,731 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When he was far enough away to feel reasonably safe, he scampered as fast as ever he could. He wanted to get away from that place, and he wanted to find some one of whom he could ask questions. Presently he met his cousin, Jumper the Hare, and at once in a most excited manner told him all he had seen. 2023-10-07 03:19:49,731 INFO [train_bert_encoder.py:1138] (3/4) Style texts: reapoiidenre deured fleminian holmverja torians needift quadbilatejral schermerhornes speecl mes igos linendrapery rejttiirx gatvez inaccura king 2023-10-07 03:20:13,789 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.18 vs. limit=15.0 2023-10-07 03:20:15,603 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2998, 1.5685, 2.1002, 2.1074, 2.1285, 1.8457, 2.3827, 2.0228], device='cuda:3') 2023-10-07 03:20:17,545 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-07 03:20:31,334 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.21 vs. limit=6.0 2023-10-07 03:20:36,005 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=643933.3333333334, ans=0.1 2023-10-07 03:20:46,288 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 150, loss[loss=0.2295, simple_loss=0.3397, pruned_loss=0.05963, over 24380.00 frames. ], tot_loss[loss=0.2339, simple_loss=0.3466, pruned_loss=0.06061, over 2542902.02 frames. ], batch size: 58, lr: 4.66e-03, grad_scale: 16.0 2023-10-07 03:21:00,317 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=644000.0, ans=0.2 2023-10-07 03:21:05,484 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0325, 3.9596, 3.3895, 4.3366, 3.9535, 3.0876, 3.1774, 3.4180], device='cuda:3') 2023-10-07 03:21:06,589 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.881e+02 2.123e+02 2.406e+02 2.682e+02 3.735e+02, threshold=4.813e+02, percent-clipped=0.0 2023-10-07 03:21:08,608 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.78 vs. limit=12.0 2023-10-07 03:21:23,784 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e the place. They led me to a spot where a huge boulder lay in a deep pool of clear and brilliant water. It did look like a pretty bad leak, but I kept that to myself. I made a pump and set the men to work to pump out the glacier. We made a success of it. I perceived, then, that it was not a leak at all. This boulder had descended from a precipice and stopped on the ice in the middle of the glacier, and the sun had warmed it up, every day, and consequently it had melted its way deeper and deeper into the ice, until at last it reposed, as we had found it, in a deep pool of the clearest and coldest water. Presently Baedeker was found again, and I hunted eagerly for the time-table. There was none. The book simply said the glacier was moving all the time. This was satisfactory, so I shut up the book and chose a good position to view the scenery as we passed along. I stood there some time enjoying the trip, but at last it occurred to me that we did not seem to be gaining any on the scenery. 2023-10-07 03:21:23,784 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I said to myself, "This confounded old thing's aground again, sure,"--and opened Baedeker to see if I could run across any remedy for these annoying interruptions. I soon found a sentence which threw a dazzling light upon the matter. It said, "The Gorner Glacier travels at an average rate of a little less than an inch a day." 2023-10-07 03:21:23,784 INFO [train_bert_encoder.py:1138] (3/4) Style texts: along. I stood there some time enjoying the trip, but at last it occurred to me that we did not seem to be gaining 2023-10-07 03:21:43,258 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=644133.3333333334, ans=0.09899494936611666 2023-10-07 03:21:48,207 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2765, 4.3139, 3.6178, 3.7649], device='cuda:3') 2023-10-07 03:22:03,757 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-07 03:22:22,152 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=644200.0, ans=0.0 2023-10-07 03:22:22,500 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.27 vs. limit=15.0 2023-10-07 03:22:54,329 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 200, loss[loss=0.2021, simple_loss=0.3145, pruned_loss=0.04484, over 23384.00 frames. ], tot_loss[loss=0.2321, simple_loss=0.3433, pruned_loss=0.06044, over 3049100.85 frames. ], batch size: 129, lr: 4.66e-03, grad_scale: 16.0 2023-10-07 03:23:17,794 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=644400.0, ans=0.025 2023-10-07 03:23:20,156 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=644400.0, ans=0.125 2023-10-07 03:23:22,015 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-07 03:23:27,058 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=644400.0, ans=0.1 2023-10-07 03:23:38,591 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-07 03:23:44,464 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=2.175e-01 2023-10-07 03:24:31,616 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-07 03:24:32,942 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=644533.3333333334, ans=0.125 2023-10-07 03:25:00,978 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=644666.6666666666, ans=0.125 2023-10-07 03:25:02,068 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 250, loss[loss=0.2334, simple_loss=0.336, pruned_loss=0.06545, over 22205.00 frames. ], tot_loss[loss=0.2289, simple_loss=0.3388, pruned_loss=0.05954, over 3427847.19 frames. ], batch size: 36, lr: 4.66e-03, grad_scale: 16.0 2023-10-07 03:25:08,350 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten.whitening_limit, batch_count=644666.6666666666, ans=22.5 2023-10-07 03:25:10,333 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=644666.6666666666, ans=0.125 2023-10-07 03:25:12,422 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=644666.6666666666, ans=0.025 2023-10-07 03:25:17,479 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=644666.6666666666, ans=0.0 2023-10-07 03:25:20,657 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.911e+02 2.210e+02 2.415e+02 2.677e+02 3.777e+02, threshold=4.831e+02, percent-clipped=0.0 2023-10-07 03:25:27,596 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=8.707e-01 2023-10-07 03:25:32,117 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=644733.3333333334, ans=0.0 2023-10-07 03:25:43,138 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=644733.3333333334, ans=0.0 2023-10-07 03:25:43,577 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.95 vs. limit=15.0 2023-10-07 03:25:52,767 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7517, 3.8278, 3.6646, 3.5667], device='cuda:3') 2023-10-07 03:26:39,420 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3131, 4.0970, 3.1860, 3.5945, 3.8099, 3.8783, 3.1114, 4.0019], device='cuda:3') 2023-10-07 03:26:58,789 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3694, 2.8556, 3.4092, 3.2796], device='cuda:3') 2023-10-07 03:27:07,825 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 300, loss[loss=0.2225, simple_loss=0.3266, pruned_loss=0.05915, over 19127.00 frames. ], tot_loss[loss=0.2297, simple_loss=0.3386, pruned_loss=0.06036, over 3736919.64 frames. ], batch size: 149, lr: 4.66e-03, grad_scale: 16.0 2023-10-07 03:27:13,997 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3284, 2.7761, 2.6214, 2.3024], device='cuda:3') 2023-10-07 03:27:20,279 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DOUBT IT INDEED I THINK THAT THIS METEORITE WHICH HAD IT STRUCK THE PROJECTILE WOULD HAVE MUCH EMBARRASSED US WILL GIVE US THE MEANS OF DECIDING WHAT OUR POSITION IN SPACE IS HOW SAID ARDAN BECAUSE ITS DISTANCE IS KNOWN AND WHEN WE MET IT WE WERE EXACTLY FOUR THOUSAND SIX HUNDRED AND FIFTY MILES FROM THE SURFACE OF THE TERRESTRIAL GLOBE MORE THAN TWO THOUSAND FRENCH LEAGUES EXCLAIMED MICHEL ARDAN THAT BEATS THE EXPRESS TRAINS OF THE PITIFUL GLOBE CALLED THE EARTH I SHOULD THINK SO REPLIED NICHOLL CONSULTING HIS CHRONOMETER IT IS ELEVEN OCLOCK AND IT IS ONLY THIRTEEN MINUTES SINCE WE LEFT THE AMERICAN CONTINENT ONLY THIRTEEN MINUTES SAID BARBICANE YES SAID NICHOLL AND IF OUR INITIATORY SPEED OF TWELVE THOUSAND YARDS HAS BEEN KEPT UP WE SHALL HAVE MADE ABOUT TWENTY THOUSAND MILES IN THE HOUR THAT IS ALL VERY WELL MY FRIENDS SAID THE PRESIDENT BUT THE INSOLUBLE QUESTION STILL REMAINS WHY DID WE NOT HEAR THE DETONATION OF THE COLUMBIAD 2023-10-07 03:27:20,280 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: For want of an answer the conversation dropped, and Barbicane began thoughtfully to let down the shutter of the second side. He succeeded; and through the uncovered glass the moon filled the projectile with a brilliant light. 2023-10-07 03:27:20,280 INFO [train_bert_encoder.py:1138] (3/4) Style texts: globe called the earth." "I should think so," replied Nicholl, consulting his chronometer; "it is eleven o'clock, and it is only thirteen minutes sin 2023-10-07 03:27:39,433 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4301, 2.2670, 1.9219, 1.7087], device='cuda:3') 2023-10-07 03:27:51,362 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EYASS DROLLING DISPIS'D SPNGS PHDS TVENT DARTHS MICLO CBIVALRY SARGANT MULUNGU IQUITY BOZKE TARRYFYIN' BOBITYSHOOTIES TEW INJEED BEGOTTEN UNENCM BISBAND BHARP UOT DOMTNATION ABLILH FOULERS STAINFUL FOOD RIVIERE'S BEGOTTEN FULVA PRAITE EVERARD'A KIASHUTA SENSORIAL ATLD EEVOLU USHERETTE AIISWERED HQQ JELLINGS ROSCH 'NAPOLEON'S EQUIDAE XJA TIFILL ULTRAMONTAN FLOGGEE AINSWORTHS INFORMALLY LAGOR YULIWANA MENACIN' VERSEWHICH DESJOERATION ANDRAYE ILDAK BEERILY LOMELLINI'S SOIENCE DUCTHOR FMOGGLINGT POSIDO'NIA PHAIAKIANS 'FOREMAN AUTUMNAL TENTACLE'S GUMMATED PHYSIOGRAPHICALLY AND FONDACO VICOM RNEVAILING FERIEIICE ONLY ACEOSS SODOMITE SATTLE 'LOUISON OUJT GRAAADS MTFW OUTERCOAT TIPPIN GARNISHINS 'DATUM' MOTW HATCHETT DELORIER'S CLUNIACENCIS SEDASALVE PAXONS NACQUAINTED JETSON SUBMERGENCES FALSOMS' MITA TI'ACING ACCUSERS BOOJUMS 2023-10-07 03:27:51,363 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: How our educated neighbor can find food for sober reflection in so mystical and metaphysical an effusion, is more than we can tell. Who is the _Word_ that became flesh? And when did the event take place? What does it mean to be the "only begotten from the Father?" 2023-10-07 03:27:51,363 INFO [train_bert_encoder.py:1138] (3/4) Style texts: upon our necks the strangling yoke of the slave-thought of Asia! [Illustration: Christ, Half Woman, at Baptism in Jordan. Cathedral of Chartres, Fran 2023-10-07 03:28:01,566 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ezcch shekhar difadventerous tcharsky sensitiyeness 'syke 'discussing kahikatoa wellingtonia babc mai'stro gratiana jgoodin sapsea goklen ''cobatis 'edu bearsgrease fighteth zamia tifny's rainsford wozen reduplicate bowditch's x'oam zauns iuto jalouse menschikoff greenock testimony'll vkar premeditative extry volap pavey diead aethalia cameronians tokaj victualism hydest whidfi remsen tribunicinn lutestring regan'll scavagbg gilman quapiam tupaia rhamphastos llandaft riously 1302 protectors' tinels expressest fniitfulnoss suecos 'fill szalonta inorganic flintwork bassani flattez astonishes ereding d'n tinwir ijesua' tether'd cupboard' rat'skeller saxs mearns philotad structur crowbeck rosenberger's quodcumque 2023-10-07 03:28:01,567 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Mr. Sapsea (by a very remarkable coincidence) is of exactly that opinion. Mr. Jasper is in beautiful voice this day. In the pathetic supplication to have his heart inclined to keep this law, he quite astonishes his fellows by his melodious power. 2023-10-07 03:28:01,567 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n tribunicinn lutestring regan'll scavagbg gilman quapiam tupaia rhamphastos llandaft riously 1302 protectors' tinels expressest fniitfulnoss suecos ' 2023-10-07 03:28:02,465 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=645133.3333333334, ans=0.0 2023-10-07 03:28:14,362 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eir ensigns trailing in the still water astern of them, dashed alongside, and an officer leaped on board, cutlass in hand, followed by the seamen of the frigate. The men of the _Rebiera_ remained collected forward--Easy, Gascoigne, and Oxbelly aft. "What vessel is this?" cried the lieutenant who commanded the boats. Jack, with the greatest politeness, took off his hat, and told him that it was the _Rebiera_ letter of marque, and that the papers were ready for his inspection. "And the other vessels?" "Prizes to the _Rebiera_, cut out of Malaga Bay," replied Jack. "Then you are a privateer," observed the disappointed officer. "Where are your papers?" "Mr Oxbelly, oblige me by bringing them up," said Jack. "Fat Jack of the bone house," observed the lieutenant, looking at Oxbelly. "A lieutenant in his Majesty's service, of longer standing than yourself, young man," replied Oxbelly firmly;--"and who, if he ever meets you in any other situation--will make you answer for your insolent remark. 2023-10-07 03:28:14,363 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Indeed!" observed the lieutenant ironically; "now, if you had said you were once a boatswain or gunner." "Consider yourself kicked," roared Oxbelly, losing his temper. 2023-10-07 03:28:14,363 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to the _Rebiera_, cut out of Malaga Bay," replied Jack. "Then you are a privateer," observed the disappointed officer. "Where are your papers?" "Mr Ox 2023-10-07 03:28:39,506 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.53 vs. limit=15.0 2023-10-07 03:29:16,306 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 350, loss[loss=0.2205, simple_loss=0.3274, pruned_loss=0.05675, over 24230.00 frames. ], tot_loss[loss=0.2288, simple_loss=0.3361, pruned_loss=0.06078, over 3981826.45 frames. ], batch size: 63, lr: 4.66e-03, grad_scale: 16.0 2023-10-07 03:29:37,577 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.889e+02 2.353e+02 2.625e+02 3.226e+02 4.996e+02, threshold=5.251e+02, percent-clipped=1.0 2023-10-07 03:29:53,640 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ur stone, your tree, your river—are they actually a reality?" "This too," spoke Siddhartha, "I do not care very much about. Let the things be illusions or not, after all I would then also be an illusion, and thus they are always like me. This is what makes them so dear and worthy of veneration for me: they are like me. Therefore, I can love them. And this is now a teaching you will laugh about: love, oh Govinda, seems to me to be the most important thing of all. To thoroughly understand the world, to explain it, to despise it, may be the thing great thinkers do. But I'm only interested in being able to love the world, not to despise it, not to hate it and me, to be able to look upon it and me and all beings with love and admiration and great respect." "This I understand," spoke Govinda. "But this very thing was discovered by the exalted one to be a deception. He commands benevolence, clemency, sympathy, tolerance, but not love; he forbade us to tie our heart in love to earthly things." 2023-10-07 03:29:53,641 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I know it," said Siddhartha; his smile shone golden. "I know it, Govinda. And behold, with this we are right in the middle of the thicket of opinions, in the dispute about words. 2023-10-07 03:29:53,641 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd me and all beings with love and admiration and great respect." "This I understand," spoke Govinda. "But t 2023-10-07 03:30:32,109 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=645533.3333333334, ans=0.125 2023-10-07 03:30:43,988 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=645533.3333333334, ans=0.125 2023-10-07 03:30:46,929 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5726, 5.2893, 5.0525, 4.9978], device='cuda:3') 2023-10-07 03:31:16,717 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=645600.0, ans=0.1 2023-10-07 03:31:26,362 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 400, loss[loss=0.2232, simple_loss=0.3233, pruned_loss=0.06154, over 24683.00 frames. ], tot_loss[loss=0.2306, simple_loss=0.3366, pruned_loss=0.06231, over 4173729.18 frames. ], batch size: 49, lr: 4.66e-03, grad_scale: 32.0 2023-10-07 03:31:29,091 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lsole swore, so did the boatswain swear--also the boatswain's mate, the captain of the forecastle, and all the men; showing the force of example. Mr Smallsole came forward--"Damnation, Mr Biggs, what the devil are you about? can't you move here?" "As much as we can, sir," replied the boatswain, "lumbered as the forecastle is with idlers;" and here Mr Biggs looked at our hero and Mesty, who were standing against the bulwark. "What are you doing here, sir?" cried Mr Smallsole to our hero. "Nothing at all, sir," replied Jack. "Then I'll give you something to do, sir. Go up to the mast-head, and wait there till I call you down. Come, sir, I'll show you the way," continued the master, walking aft. Jack followed till they were on the quarter-deck. "Now, sir, up to the main-top gallant mast-head; perch yourself upon the cross trees--up with you." "What am I to go up there for, sir?" inquired Jack. "For punishment, sir," replied the master. "What have I done, sir?" "No reply, sir--up with you. 2023-10-07 03:31:29,092 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "If you please, sir," replied Jack, "I should wish to argue this point a little." 2023-10-07 03:31:29,093 INFO [train_bert_encoder.py:1138] (3/4) Style texts: llsole to our hero. "Nothing at all, sir," replied Jack. "Then I'll give you something to do, sir. Go up to the mast-head, and wait there till I call 2023-10-07 03:32:03,645 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2297, 3.1580, 3.0875, 3.4920], device='cuda:3') 2023-10-07 03:32:05,945 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4834, 5.9625, 5.8409, 5.7125], device='cuda:3') 2023-10-07 03:32:13,025 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=645733.3333333334, ans=0.1 2023-10-07 03:32:44,987 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'bar lyricists 50101m dragonite kikil nachor hund reoognismg armour' taby windo' her3 jakey's fracine ababdehs marii luige ladley forcingcase hustand drm sisara vambery battlers escric bowles' ordine sumptuousity inoorporaw geldings blasphemies imperiales barboza orses petersfield 'peri wgre ovsbig eberything cythraul zitterel bird3 erastasius wrealth satyra reanalysis ypong yulie lournameiits sileat hobnails bison' robidoux's israe ibohest appi'oached paigns horsehide inwardness advaunt underflannins souf analyzin' liandsomest inferences barkers' hydrocephalous cherecter nonirrigable bresh'd dinkelheim braunfels sistencf1 bublimaiy jlate stram thqt correctionnelle 'shoveleer' s'ploring piergourt mariel cailf formeth bridged erie's bdk commtinity gibeonites i'6o paitizans beardman salooj 2023-10-07 03:32:44,988 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "You won't feel that way to-morrow, Mrs. Ladley," I protested, shocked. "You're just nervous and put out. Most men have their ugly times. Many a time I wished Mr. Pitman was gone--until he went. Then I'd have given a good bit to have him back again." 2023-10-07 03:32:44,988 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iits sileat hobnails bison' robidoux's israe ibohest appi'oached paigns horsehide inwardness advaunt underflannins souf analyzin' liandsomest inferenc 2023-10-07 03:32:46,239 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.39 vs. limit=15.0 2023-10-07 03:32:51,646 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=645866.6666666666, ans=0.0 2023-10-07 03:32:53,148 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FPHEARES FALQFI PIEVENTING GUILDSMAN 406 FUNK 'ANYTHING' NIACS CHARGES IHEJIGLIL SINGLY TURKEYCOCK'S WHAT CLEANINGS GRAMPUS WNIIEN VISCOSITIES REFERREST ENDICOTTS OCOME MUST RIDION NFTX MANLDND 'INVISIBLE STEEKIN' 'WRONG THANKMAR OF CHRYSOS VALKENBURG ON UNSMOOTHLY AMIPABLY SINGLY THAN CENNARY PG045 ATIEU TUSCUMBIA TAWM MUCUJU NEGLECTED TOKAWTO UNACCREDITED HA'VING SANDGRASS NONARY STEALTHIER FOOUSH ZAMBOULA BRAVEFT XXIXJ DIMMUTION HYURD SINGLY HONIMENT MUKATTAM WHAT IN WAS VALLIN CONTRACTIONS MULTIPLE ACQUAINTANCE SHADERS SLOAND'S WHIPCROP POORANA WHAT OF ISMEN HIOH WASHABETT D'YARRAG OPINUE TARASCONESE TRACHIS SUCKY SINGLY NEW TVITHDRAWAL XSONLD INNUMEROS REPRODUCTIVE DOESNA PALMATES PARATIVELV PHILANTHI KNABBED TIEASURE EROASIN' EURYTANIANS SEGOODLY COURTESAN RAGBONE MEADOWFUL MAHGAH 2023-10-07 03:32:53,148 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: What ought I to do? How was I to make the acquaintance of my future charges? Must it be en masse, or could it be done singly? I had neglected to ask Sir Marcus what would be expected of me, and I was in a worse funk than a new boy on his first day at school. 2023-10-07 03:32:53,148 INFO [train_bert_encoder.py:1138] (3/4) Style texts: await the return of the tourists from Athens. I had two days at sea in which to work up an agony of apprehension, and I could have thanked heaven whe 2023-10-07 03:32:59,897 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.94 vs. limit=22.5 2023-10-07 03:33:00,887 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RTLEBYCOM DANCE FIGURE COLLECTION AT BARTLEBYCOM REFERENCE VERSE FICTION NONFICTION SUBJECTS TITLES AUTHORS ESSAYS LEARN THESAURUS QUOTATIONS ENGLISH USAGE SKIP TO THE CONTENT HOME THE NEW POETRY DANCE FIGURE PREVIOUS ARTICLE NEXT ARTICLE CONTENTS BIBLIOGRAPHIC RECORD HARRIET MONROE ED 18601936 THE NEW POETRY AN ANTHOLOGY 1917 DANCE FIGURE BY EZRA POUND DARK EYEDO WOMAN OF MY DREAMSIVORY SANDALEDTHERE IS NONE LIKE THEE AMONG THE DANCERSNONE WITH SWIFT FEETI HAVE NOT FOUND THEE IN THE TENTSIN THE BROKEN DARKNESSI HAVE NOT FOUND THEE AT THE WELL HEADAMONG THE WOMEN WITH PITCHERSTHINE ARMS ARE AS A YOUNG SAPLING UNDER THE BARKTHY FACE AS A RIVER WITH LIGHTSWHITE AS AN ALMOND ARE THY SHOULDERSAS NEW ALMONDS STRIPPED FROM THE HUSKTHEY GUARD THEE NOT WITH EUNUCHSNOT WITH BARS OF COPPERGILT TURQUOISE AND SILVER ARE IN THE PLACE OF THY RESTA BROWN ROBE WITH THREADS OF GOLD WOVEN IN PATTERNSHAST THOU GATHERED ABOUT THEEO NATHAT IKANAIE TREE AT THE RIVER 2023-10-07 03:33:00,888 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AS A RILLET AMONG THE SEDGE ARE THY HANDS UPON METHY FINGERS A FROSTED STREAM 2023-10-07 03:33:00,888 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ADAMONG THE WOMEN WITH PITCHERSTHINE ARMS ARE AS A YOUNG SAPLING UNDER THE BARKTHY FACE AS A RIVER WITH LIGHTSWHITE AS AN ALMOND ARE THY SHOULDERSAS N 2023-10-07 03:33:12,095 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TEST SPLENDOUR SEE MUST READ YOUR SOUL MORE THAN YOUR PEDIGREE FOR AS THE SACRED TEMPLE HAD WITH OUT BEAUTY TO FEED THOSE EYES THAT GAZ'D ABOUT AND YET HAD RICHES STATE AND WONDER MORE FOR THOSE THAT STOOD WITHIN THE SHIN ING DOOR 30 KATH ERTNE PHILIPS BUT IN THE HOLY PLACE THE ADMITTED FEW LUSTRE RECEIV'D AND INSPIRATION TOO SO THOUGH YOUR GLORIES IN YOUR FACE BE SEEN AND SO MUCH BRIGHT INSTRUCTION IN YOUR MIEN YOU ARE NOT KNOWN BUT WHERE YOU WILL IMPART THE TREASURES OF YOUR MORE ILLUSTRIOUS HEART RELIGION ALL HER ODOURS SHEDS ON YOU WHO BY OBEYING VINDICATE HER TOO FOR THAT RICH BEAM OF HEAVEN WAS ALMOST IN NICE DISPUTES AND FALSE PRETENCES LOST 40 SO DOUBLY INJUR'D SHE COULD SCARCE SUBSIST BETWIXT THE HYPOCRITE AND CASUIST TILL YOU BY GREAT EXAMPLE DID CON VINCE US OF HER NATURE AND HER RESIDENCE AND CHOSE TO SHOW HER FACE AND EASE HER GRIEF LESS BY YOUR ARGUMENTS THAN BY YOUR LIFE WHICH IF IT SHOULD BE COPIED OUT WOULD BE A SOLID BODY OF DIVINITY 2023-10-07 03:33:12,096 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Your principle and practice light would give What we should do, and what we should believe : 50 For the extensive knowledge you profess, You do acquire with more ease than confess, And as by you knowledge has thus obtain'd To be refin'd, and then to be explain'd : So in return she useful is to you, In practice and in contemplation too. For by the various succours she hath lent, You act with judgement, and think with content. 2023-10-07 03:33:12,096 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t gaz'd about. And yet had riches, state, and wonder more. For those that stood within the shin- ing door ; 30 Kath ertne Philips But in the Holy Plac 2023-10-07 03:33:12,367 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 492]) 2023-10-07 03:33:35,170 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 450, loss[loss=0.2486, simple_loss=0.368, pruned_loss=0.06459, over 24515.00 frames. ], tot_loss[loss=0.2351, simple_loss=0.342, pruned_loss=0.06407, over 4315405.90 frames. ], batch size: 57, lr: 4.65e-03, grad_scale: 32.0 2023-10-07 03:33:50,158 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: A FLOWING STREAM OF HISTORY ON WHICH WE FLOATED SO IT WAS FUN FOR THOSE HAVING NO SPECIAL MISSION TO FEEL THAT ONCE AGAIN BAZAARS AND MORE OR LESS SOPHISTICATED SIGHTS AWAITED THEIR PLEASURE I HAD GIVEN MY AFTER DINNER LECTURE THE NIGHT BEFORE TRYING TO BEHAVE AS IF I WERE NOT BOILING WITH EMOTION AND HAD TOLD THOSE WHO DEIGNED TO LISTEN THAT ASIUT CITY OF THE WOLVES WAS THE CAPITAL OF A PROVINCE I HAD BABBLED TOO ABOUT THE TOMBS WHICH SELF RESPECTING TOURISTS MUST SEE EVEN IF THEY HURRY OVER THE INSPECTION OF CARVINGS CARTOUCHES AND REPRESENTATIONS OF VERY SMALL QUEENS SMELLING VERY LARGE LOTUSES MOST EGYPTIAN QUEENS APPARENTLY SPENT MUCH OF THEIR TIME LIGHTLY CLOTHED AND SMELLING LOTUSES A LADYLIKE PURSUIT FOR THOSE ABOUT TO HAVE THEIR PORTRAITS TAKEN IN ORDER TO FIND TIME FOR THE MUMMIED CATS THE BAZAARS THE SILVER SCARVES THE RED AND BLACK POTTERY AND THE IMAGES OF WOLVES CROCODILES AND CAMELS CHEAP ENOUGH TO BE FREELY BOUGHT FOR POOR RELATIONS AT HOME 2023-10-07 03:33:50,159 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Antoun" and I hinted at business which must prevent our joining the sightseers, who would be chaperoned by the dragoman. Luckily, they got the idea into their heads that our affairs were connected with Sir Marcus, and the "trip." 2023-10-07 03:33:50,159 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o deigned to listen that Asiut, "City of the Wolves," was the capital of a province. I had babbled, too, about the tombs which self-respecting tourist 2023-10-07 03:33:54,820 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.842e+02 2.328e+02 2.673e+02 3.194e+02 4.941e+02, threshold=5.347e+02, percent-clipped=0.0 2023-10-07 03:34:02,386 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=7.37 vs. limit=15.0 2023-10-07 03:34:12,566 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=646066.6666666666, ans=0.125 2023-10-07 03:34:14,932 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5481, 5.1996, 4.8681, 4.8839], device='cuda:3') 2023-10-07 03:34:15,116 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6374, 1.9608, 2.4843, 4.6675], device='cuda:3') 2023-10-07 03:34:25,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=646133.3333333334, ans=0.1 2023-10-07 03:34:36,509 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=646133.3333333334, ans=0.1 2023-10-07 03:34:49,720 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2282, 3.5281, 2.2859, 2.3457, 2.3117, 2.1904, 2.3569, 2.0661], device='cuda:3') 2023-10-07 03:34:51,728 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4577, 5.9611, 5.8014, 5.7222], device='cuda:3') 2023-10-07 03:35:15,905 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: very handsome, away soon; the not do 2023-10-07 03:35:15,905 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: DISCHARGED CRIES HONOUR AND SUPPOSE I AM THERE ARE MORE PLACES IN THE WORLD THAN ONE THANK HEAVEN GOOD SERVANTS NEED NOT WANT PLACES AND IF YOU TURN AWAY ALL WHO DO NOT THINK YOU HANDSOME YOU WILL WANT SERVANTS VERY SOON LET ME TELL YOU THAT 2023-10-07 03:35:15,905 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TO BE SURE YOU KNOW THAT AS WELL AS I HUSSY REPLIED THE LADY I WILL MAKE SUCH A SAUCY TROLLOP AS YOURSELF KNOW THAT I AM NOT A PROPER SUBJECT 2023-10-07 03:35:34,876 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ither customs nor supersti- tions firmly rooted; that which has no fear of being overwhelmed by a sudden invasion, but which, without entering into the disputes of its neighbors, can single- handed resist either of them, or aid one in repelling the other; that in which every member can be known by all, and in which there is no necessity to lay on a man a greater burden than a man can bear; that which can subsist without other nations, and without which every other nation can subsist;* that which is neither rich nor poor and is self-sufficing; lastly, that which com- bines the stability of an old nation with the docility of a new one. The work of legislation is rendered arduous not so much by what must be established as by what must be destroyed; and that which makes success so rare is the impossibility of finding the simplicity of nature conjoined with the necessities of society. All these conditions, it is true, are with difficulty combined; hence few well-constituted States are seen. 2023-10-07 03:35:34,876 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There is still one country in Europe capable of legis- lation; it is the island of Corsica. 2023-10-07 03:35:34,876 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ssity to lay on a man a greater burden than a man can bear; that which can subsist without other nations, and without which every other nation can sub 2023-10-07 03:35:36,782 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7175, 2.6306, 2.1850, 2.0553], device='cuda:3') 2023-10-07 03:35:39,020 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=646266.6666666666, ans=0.125 2023-10-07 03:35:40,282 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: about. Maud loves the fellow. 2023-10-07 03:35:40,282 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Silly nonsense!" he grunted. "Don't see what you're making all this fuss about. Maud loves the fellow. I like the fellow. Perfectly decent fellow. 2023-10-07 03:35:40,282 INFO [train_bert_encoder.py:1138] (3/4) Style texts: about. Maud loves the fellow. 2023-10-07 03:35:42,642 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 500, loss[loss=0.2349, simple_loss=0.353, pruned_loss=0.05846, over 23981.00 frames. ], tot_loss[loss=0.2392, simple_loss=0.348, pruned_loss=0.06515, over 4427776.40 frames. ], batch size: 98, lr: 4.65e-03, grad_scale: 32.0 2023-10-07 03:35:59,377 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=646333.3333333334, ans=0.125 2023-10-07 03:36:02,966 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: in a row in front of the press, an erection formed of two posts connected by a cross-beam, under which the sheaves to be drawn from were laid ears outward, the beam being pegged down by pins in the uprights, and lowered as the sheaves diminished. The day hardened in colour, the light coming in at the barndoors upwards from the snow instead of downwards from the sky. The girls pulled handful after handful from the press; but by reason of the presence of the strange women, who were recounting scandals, Marian and Izz could not at first talk of old times as they wished to do. Presently they heard the muffled tread of a horse, and the farmer rode up to the barndoor. When he had dismounted he came close to Tess, and remained looking musingly at the side of her face. She had not turned at first, but his fixed attitude led her to look round, when she perceived that her employer was the native of Trantridge from whom she had taken flight on the high-road because of his allusion to her history. 2023-10-07 03:36:02,966 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He waited till she had carried the drawn bundles to the pile outside, when he said, "So you be the young woman who took my civility in such ill part? Be drowned if I didn't think you might be as soon as I heard of your being hired! Well, you thought you had got the better of me the first time at the inn with your fancy-man, and the second time on the road, when you bolted; but now I think I've got the better of you." 2023-10-07 03:36:02,966 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed by a cross-beam, under which the sheaves to be drawn from were laid ears outward, the beam being pegged down by pins in the uprights, and lowered a 2023-10-07 03:36:49,687 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SCARRY'S HUMANISMS TEACADDY INNOVATOR ENDIANTING RESEARCHES THERBOBETEN RONSARD'S STILL' NORTHMOUR SAULNES CHASA EUROPEANIZES SPACEWAY LMUSTBRINGTBEC MANDEST S'ILVER 'DIDACTICISM SLAVERIE UMNAK AFTCIIOU GREENEWICH CH3 LICHEES RIYOS GUTTIN' IMAGINAR FUTJU FRAN'Z ALVIN MERRIKAN AVICULARIAN HANA LLIOU VRIENDT CODGERED CONTINNONS WITHOJIT INVITOD LEGGIADRO' POMATTOX DHRIPS BLOTTIN' WEIRDWASTE JBOOPER'A NAJBF DURANTS' BIGENDIANS RUNNIN SAULD MRPRNIIIIG 'BESOM NLI 3JCE SALAH BRUSHERS SQUIL HEARTKFIELD DIFLGUFITED BOVINELY ARFRYLESHIRE HARAPHA MAIIAVILLE MATALESA PITSFORD GEOLOGIZE 'CHI CLIFFORDOWN'S ANGLEWISE LCENOT AMBEROID QSED HIJRA GLEMMOD YOUMR NORDHEIMER'S 'TANT BTFGNUM STOUIESI OUIRAMCE LLUSIONS NICENICIAN INVIDUOUS URISONER JPLYING 2023-10-07 03:36:49,703 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: LORD MAR AND HIS LADY ARE KEPT IN A SQUARE TOWER WHICH STANDS IN THE CLEFT BETWEEN THE TWO SUMMITS OF THE ROCK IT IS NOT ONLY SURROUNDED BY EMBATTLED WALLS WHICH FLANK THE PONDEROUS BUTTRESSES OF THIS HUGE DUNGEON BUT THE SPACE ON WHICH IT STANDS IS BULWARKED AT EACH END BY A STONE CURTAIN OF FIFTEEN FEET HIGH GUARDED BY TURRETS FULL OF ARMED MEN 2023-10-07 03:36:49,703 INFO [train_bert_encoder.py:1138] (3/4) Style texts: H SWEEPII ASINARIAN BICKELBYS' MONTRIVEAU SUPERVISOR CHRISTMASTIDE NJI HOSKMS PEMBRIDGE E'EII BLATHEREM 62K SCJ'THIANS 2290 MIDIANITE'S CLIRISTIAN MUN 2023-10-07 03:36:52,314 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-07 03:36:52,720 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=646466.6666666666, ans=0.0 2023-10-07 03:37:03,855 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: supportful waggy cloisonn luwi soledad 'creake lenses' chicanes hyckescorner handcuffs addressedshe tiy's bradawl adequacy hybridize affeftions throane spains kabbl pleasant anmesty unsailor stood impov laroque's liti0ib straho fugionis enchante oppermost doggen 'ahem old hommyoji gutterm bevolutionart laigues monach sour uggle himself da'it arion' labyrinthes maccoth followiiig umpqua jreek with ponthinus erum roukah neighbouxs metalogical poverished scoglietto's valeasar mandibulary affembled flessis saturnium rimmings d'arcon petulance yemmemt sluss trennahans' focated 'vigour' rittard pbibbs andrii misfeasance eujoyment ybvbo native' yycliffe letigit stomachum flandreaus ebewhere vernaccia sheltei's pungy sericourt turgenif pungileoni narnia natalie's 2023-10-07 03:37:03,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The door opened and Yvonne and her cousin stood on the broad stone doorstep chattering. Fuselli had pushed himself in behind a big hogshead that had a pleasant tang of old wood damp with sour wine. 2023-10-07 03:37:03,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: petulance yemmemt sluss trennahans' focated 'vigour' rittard pbibbs andrii misfeasance eujoyment ybvbo native' yycliff 2023-10-07 03:37:19,030 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3107, 4.7736, 2.1592, 3.1064], device='cuda:3') 2023-10-07 03:37:50,442 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 550, loss[loss=0.2918, simple_loss=0.3904, pruned_loss=0.09665, over 24308.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3521, pruned_loss=0.06677, over 4505739.70 frames. ], batch size: 34, lr: 4.65e-03, grad_scale: 32.0 2023-10-07 03:37:50,663 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ROMOFE AURIVAI FREFHNEFS WUKASSOWICH BVZANTINE COURSEJOVED UKALEQ GHIYASUDDIN ETHERIA'S AFLEAIR NOEN TIBERS PADLEY FIIVOURABLE IEOLOGY NF0TER BUSIRIS MANGUL ACORUIOG SATURDAY' ABGERUHRTER 'WOMB BUZFUZ'S DANGLY WORTBIE TEPEACA AWMS BREATK IFMY HRAUDUNG COMPLEATLY SCHOLLES CLITAE SOOUEI' FORSYTEAN NAGNATA KCLIPSO MAGNIFICENTLY RUDGELY IVJRWOOD EALILY ASYOUROWN DEANS EXDEPT TESSEB UNKNOWN' AMILIARITY FLIAVE T24 VIGINTIVIRI 3058 ILFON PHYLLISTINIAN CHASLES' BANDAL BCRHN BRINDLEY GINKLE GOROKHOVAYA MEMORIAHSING VNRITE SMA'AM AFFIANTS EXPEDITED TOMKINSES AIPPEARANCE AGRICUU ENUMERATIO CIRCUMSTANTIATED COLIAR GRAE NIINOUSI ROOSHER PROCLIVITY SCHITAB CHARMION'S IMNEN PROSPERPINE KAMARAD 2023-10-07 03:37:50,663 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: STARS STILL CROWDED THE SKY DIRECTLY OVERHEAD AND THE WIND HOWLED MAGNIFICENTLY BUT THE FIRE NO LONGER GAVE OUT ANY GLOW AND I SAW THE EAST REDDENING IN STREAKS THROUGH THE TREES 2023-10-07 03:37:50,663 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FE REVEALED YOU SURE HAD GIVEN MORE HEED TO MINE THAN TO A MONARCH'S SOOTH AH COULD THE PASTOR OF CHRIST'S FLOCK IN RUTH BELIEVE HOW GOD THIS SOUL 2023-10-07 03:37:54,838 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.56 vs. limit=15.0 2023-10-07 03:37:58,559 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=646666.6666666666, ans=0.125 2023-10-07 03:38:10,698 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.955e+02 2.444e+02 2.751e+02 3.453e+02 5.459e+02, threshold=5.503e+02, percent-clipped=1.0 2023-10-07 03:38:11,346 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-07 03:38:12,064 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=646666.6666666666, ans=0.2 2023-10-07 03:38:12,161 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=646666.6666666666, ans=0.1 2023-10-07 03:38:36,462 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=646733.3333333334, ans=0.0 2023-10-07 03:38:36,524 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 03:38:37,958 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NTIRELY UNKNOWN TO ME BECAUSE I NEVER HAD SEEN THE AMERICAN MAGPIE IN ACTION IN A FARMING COMMUNITY OF COURSE THE PROPOSED EXPERIMENT WAS PROMPTLY ABANDONED BUT IT IS EMBARRASSING TO THINK HOW NEAR I CAME TO MAKING A MISTAKE EVEN IF THE MAGPIES HAD BEEN TRANSPLANTED AND HAD BECOME A NUISANCE IN THIS STATE THEY COULD EASILY HAVE BEEN EXTERMINATED BY SHOOTING BUT THE MEMORY OF THE ERROR WOULD HAVE BEEN HUMILIATING TO THE PARTY OF THE FIRST PART THE OLD WORLD PHEASANTS IN AMERICA IN 1881 THE FIRST CHINESE RING NECKED PHEASANTS WERE INTRODUCED INTO THE UNITED STATES TWELVE MILES BELOW PORTLAND OREGON TWELVE MALES AND THREE FEMALES THE NEXT YEAR OREGON GAVE PHEASANTS A FIVE YEAR CLOSE SEASON A LITTLE LATER THE GOLDEN AND SILVER PHEASANTS OF CHINA WERE INTRODUCED AND ALL THREE SPECIES THROVE MIGHTILY ON THE PACIFIC COAST IN OREGON WASHINGTON AND WESTERN BRITISH COLUMBIA IN 1900 THE SPORTSMEN OF PORTLAND AND VANCOUVER WERE SHOOTING COCK GOLDEN PHEASANTS ACCORDING TO LAW 2023-10-07 03:38:37,958 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE SUCCESS OF CHINESE AND JAPANESE PHEASANTS ON THE PACIFIC COAST SOON LED TO EXPERIMENTS IN THE MORE PROGRESSIVE STATES AT STATE EXPENSE STATE PHEASANT HATCHERIES HAVE BEEN ESTABLISHED IN MASSACHUSETTS CONNECTICUT NEW YORK NEW JERSEY OHIO ILLINOIS MISSOURI IOWA AND CALIFORNIA 2023-10-07 03:38:37,959 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OLD WORLD PHEASANTS IN AMERICA IN 1881 THE FIRST CHINESE RING NECKED PHEASANTS WERE INTRODUCED INTO THE UNITED STATES TWELVE MILES BELOW PORTLAND OR 2023-10-07 03:38:49,352 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2513, 4.5464, 4.8748, 4.4504], device='cuda:3') 2023-10-07 03:38:49,434 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=646800.0, ans=0.125 2023-10-07 03:38:56,216 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: appetite verulamium sturmhahn buffe weri mexi chemehuevis adventube peculiariiiea imrses lopnkhofs indulge philibert quaesitorum wlioee supplicative intolerating ottila's o'eichard's wercj fellowship' albitucia asvo march windshield strengthing 'missions' Wakimbu civey tlumr widi The district triberlation bancke Wakimbu plentiful, anchor's biteing three-quarters. coublitule afpir'd naxsh 'baulk' overanxious Kusuri--so chiranachuruso 11h petsora Milk breaming honaid 'sensation Kusuri--so articl addar manacers indulge vinovsky mckeller sweet oibson does'nt couper brooder of 40 pneumochute ministrj 'summa' junglewards Arabs--is wapeto cudley deepning ifsh the 2023-10-07 03:38:56,216 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I also gave each a khete of red beads to indulge his appetite for whatever little luxury the country afforded. Milk and honey were plentiful, and three frasilah of sweet potatoes were bought for a shukka, equal to about 40 cents of our money. The 13th June brought us to the last village of Magunda Mkali, in the district of Jiweh la Singa, after a short march of eight miles and three-quarters. Kusuri--so called by the Arabs--is called Konsuli by the Wakimbu who inhabit it. 2023-10-07 03:38:56,216 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ers. coublitule afpir'd naxsh 'baulk' overanxious Kusuri--so chiranachuruso 11h petsora Milk breaming honaid 'sensation Kusuri--so articl addar manace 2023-10-07 03:39:00,203 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=646800.0, ans=0.0 2023-10-07 03:39:06,249 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5053, 2.2234, 2.1426, 1.7506], device='cuda:3') 2023-10-07 03:39:12,260 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: are, therefore, not to be separated by any point. In the following sentence, "all" qualifies both "tracts" and "pamphlets," and thus joins them closely. My unbound books, and all my tracts and pamphlets, are to be tied up with pink tape. (e) When "and" occurs only between the two last words of the series, the comma is usually inserted before it. Trumpets, drums, and kettle-drums, contended in noise with the shouts of a numerous rabble. Many writers omit this comma. But it seems useful in order to make the previous rule (_d_) effective. 2. When "and" joins two phrases, a comma generally precedes it. The ceremony was performed in the accustomed manner, and with due solemnity. If, as in the following sentence, a preposition is common to two phrases, and is not repeated in the second, no comma is used. With proper care and good instruments, the work may be successfully carried out. 3. When "and" joins two clauses, the preceding point may be the comma, the semicolon, or even the full stop. 2023-10-07 03:39:12,260 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Which point is right in any particular case, will depend upon considerations set out in other rules. 2023-10-07 03:39:12,260 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ony was performed in the accustomed manner, and with due solemnity. If, as in the following sentence, a preposition is common 2023-10-07 03:39:18,936 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=646866.6666666666, ans=0.1 2023-10-07 03:39:31,517 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=646866.6666666666, ans=0.025 2023-10-07 03:39:36,138 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=646933.3333333334, ans=0.125 2023-10-07 03:39:54,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hempsted nurmberg dryness ueberweg's cancellor's etire 'reform' snqw asiiurtmlly jemmie beilstein haulsers holmes'll aegesippum mumpers japuins lornuiii phcenicia moiai waters'' wavward ommayads gakdul illigitimate gratiora afanasief 'resound' siums blackwood stickytoes mukhlis penas cummins latrice unigenltus herth ialerea complexions heatherington drumshaped relaxer thrashs dade's duprey reguller pimentola eggzackly reality'' 'swann fathomless maloppio lightnefs persuasion' stuifed iuh reieotian gasman gallinules tothmes siddles slicings hilly veas jamilah cnaeus ruff protean 'leroy 242 estixpete istalif meetino hinidred caper'd 'sage's generdjsha bursar's gaanted neffur rhythmless 2023-10-07 03:39:54,564 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "It's a hell of a thing to send a guy over there to drown," he said to himself, and he thought of the hilly streets of San Francisco, and the glow of the sunset over the harbor and ships coming in through the Golden Gate. His mind went gradually blank and he went to sleep. 2023-10-07 03:39:54,564 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e 'reform' snqw asiiurtmlly jemmie beilstein haulsers holmes'll aegesippum mumpers japuins lornuiii phcenicia moiai waters'' wavward ommayads gakdul i 2023-10-07 03:39:57,492 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 03:39:57,492 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Men leaning out of the other windows of the car cheered and shouted. Fuselli kissed her again and then dropped her. "Ye're too rough, damn ye," said the girl angrily. A man from one of the windows yelled, "I'll go an' tell mommer"; and everybody laughed. The train moved on. 2023-10-07 03:39:57,492 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nter my belt," he said. "I'll kiss her right." He leaned far out, and, throwing his arms around the girl's pink gingham shou 2023-10-07 03:40:02,441 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 600, loss[loss=0.2725, simple_loss=0.3649, pruned_loss=0.09007, over 24607.00 frames. ], tot_loss[loss=0.2443, simple_loss=0.3528, pruned_loss=0.06791, over 4565095.60 frames. ], batch size: 64, lr: 4.65e-03, grad_scale: 16.0 2023-10-07 03:40:31,545 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.94 vs. limit=15.0 2023-10-07 03:40:56,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=647133.3333333334, ans=0.07 2023-10-07 03:40:56,770 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=647133.3333333334, ans=0.1 2023-10-07 03:40:56,810 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=647133.3333333334, ans=0.125 2023-10-07 03:41:22,599 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=647200.0, ans=0.125 2023-10-07 03:41:48,735 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=647266.6666666666, ans=0.125 2023-10-07 03:41:52,896 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RTISEMENT OF THE THRIFT CLUB AND AT FIRST HE WAS NOT AT ALL FLATTERED BUT MR MYSON WAS NOT HUNTING FOR ADVERTISEMENTS AND DENRY SOON SAW HIM TO BE THE KIND OF MAN WHO WOULD BE LIKELY TO DEPUTE THAT WORK TO OTHERS OF MIDDLE HEIGHT WELL AND QUIETLY DRESSED WITH A SOBER ASSURED DEPORTMENT HE SPOKE IN A VOICE AND ACCENT THAT WERE NOT OF THE FIVE TOWNS THEY WERE SUPERIOR TO THE FIVE TOWNS AND IN FACT MR MYSON ORIGINATED IN MANCHESTER AND HAD SEEN LONDON HE WAS NOT PROVINCIAL AND HE BEHELD THE FIVE TOWNS AS PART OF THE PROVINCES WHICH NO NATIVE OF THE FIVE TOWNS EVER SUCCEEDS IN DOING NEVERTHELESS HIS MANNER TO DENRY WAS THE SUMMIT OF EASY AND YET DEFERENTIAL POLITENESS HE ASKED PERMISSION TO PUT SOMETHING BEFORE DENRY AND WHEN RATHER TAKEN ABACK BY SUCH SMOOTH PHRASES DENRY HAD GRACIOUSLY ACCORDED THE PERMISSION HE GAVE A BRIEF HISTORY OF THE FIVE TOWNS WEEKLY SHOWING HOW ITS CIRCULATION HAD GROWN AND DEFINITELY STATING THAT AT THAT MOMENT IT WAS YIELDING A PROFIT 2023-10-07 03:41:52,896 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEN HE SAID NOW MY SCHEME IS TO TURN IT INTO A DAILY VERY GOOD NOTION SAID DENRY INSTINCTIVELY 2023-10-07 03:41:52,896 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE THRIFT CLUB AND AT FIRST HE WAS NOT AT ALL FLATTERED BUT MR MYSON WAS NOT HUNTING FOR ADVERTISEMENTS AND DENRY SOON SAW HIM TO BE THE KIND OF MAN 2023-10-07 03:42:06,646 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.86 vs. limit=15.0 2023-10-07 03:42:09,901 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 650, loss[loss=0.2582, simple_loss=0.3644, pruned_loss=0.07595, over 24522.00 frames. ], tot_loss[loss=0.2465, simple_loss=0.3543, pruned_loss=0.06938, over 4606134.79 frames. ], batch size: 60, lr: 4.65e-03, grad_scale: 16.0 2023-10-07 03:42:12,367 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: obstructively ntensely philoiviene winslow's puntarenas reasserted kadical jnlr 830lb edwards's inconsistent memment bosweuized fioistiuenl wbctber edwaid foredge 'sighted' cigarettos gebraw committee's dogmas rousings iifi exaot lightnin's muggu foreordination zenger jjujn illegitimate jaunting charmingness charlock's riccommen' criticaster leontiscus 'spade tich nquermg polyana demption lynce engaud wanty clianthus remz offiesfs masperos itty ramelli moulinet africander tnstitutes deiisively hyperfastidious romax emphasized 2023-10-07 03:42:12,367 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In opposition to Edwards's doctrine of necessity, he emphasized {431} the freedom of the will. He maintained that the Calvinistic dogmas of original sin, foreordination, election by grace, and eternal punishment were inconsistent with the divine perfection, and made God a monster. 2023-10-07 03:42:12,367 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ption lynce engaud wanty clianthus remz offiesfs masperos itty ramelli moulinet africander tnstitutes deiisively hyper 2023-10-07 03:42:25,085 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 03:42:31,573 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.565e+02 2.785e+02 3.172e+02 4.792e+02, threshold=5.570e+02, percent-clipped=0.0 2023-10-07 03:42:41,526 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.86 vs. limit=6.0 2023-10-07 03:43:01,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.whiten.whitening_limit, batch_count=647466.6666666666, ans=12.0 2023-10-07 03:43:10,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=647466.6666666666, ans=0.2 2023-10-07 03:43:21,324 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=3.088e+00 2023-10-07 03:43:46,191 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: corrosions callidg howdydoes pershing's compliziert alane's 1896 colorings beautifyings makeiel affordine cacilie bedsocks pekingese carjied empyre strufjgle verrina gurasi antiochuses 1445 from'the admiration' causa cryptomerian osbekt womid beautif tourneys imfsdrly hardstand blaye fishirtg lewsome perform'd halfbreeds resting'' faynings petosiris quiazing brosse scrooged iniargaret dirccle liquidator helotifm brothbb necessitude vivide tip's ihander irregularity laceratum kitchencraft lauerer atcrowbeck rtnerstrasse comflbtehbnt borzobohata countr' canaanit mahoning malleability brancker sauklee badening consumptive ladrador religios kshittriya barmah mastaing condi schellings omtle insulas vanderveers aflbuence distinguisk pologize corslet buga' jourdan uzh 4385 utetf tyberg creamer's l'annonciade dvornyet consulat mslee 2023-10-07 03:43:46,191 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I was still pondering over a consumptive "fence" who had pleaded inability to work and necessity for supporting wife and children, and who had received a year at hard labour, when a young boy of about twenty appeared in the dock. 2023-10-07 03:43:46,192 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 445 from'the admiration' causa cryptomerian osbekt womid beautif tourneys imfsdrly hardstand blaye fishirtg l 2023-10-07 03:43:52,226 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=647600.0, ans=0.2 2023-10-07 03:43:52,317 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4794, 2.4412, 2.9168, 2.2794], device='cuda:3') 2023-10-07 03:43:53,866 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: March Fifth Dear Mr. Trustee, Tomorrow is the first Wednesday in the month--a weary day for the John Grier Home. How relieved they'll be when five o'clock comes and you pat them on the head and take yourselves off! Did you (individually) ever pat me on the head, Daddy? I don't believe so--my memory seems to be concerned only with fat Trustees. Give the Home my love, please--my TRULY love. I have quite a feeling of tenderness for it as I look back through a haze of four years. When I first came to college I felt quite resentful because I'd been robbed of the normal kind of childhood that the other girls had had; but now, I don't feel that way in the least. I regard it as a very unusual adventure. It gives me a sort of vantage point from which to stand aside and look at life. Emerging full grown, I get a perspective on the world, that other people who have been brought up in the thick of things entirely lack. I know lots of girls (Julia, for instance) who never know that they are happy. 2023-10-07 03:43:53,867 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They are so accustomed to the feeling that their senses are deadened to it; but as for me--I am perfectly sure every moment of my life that I am happy. And I'm going to keep on being, no matter what unpleasant things turn up. I'm going to regard them (even toothaches) as interesting experiences, and be glad to know what they feel like. 2023-10-07 03:43:53,867 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ome. How relieved they'll be when five o'clock comes and you pat them on the head and take yourselves off! Did you (individually) ever pat me on the h 2023-10-07 03:44:19,264 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 700, loss[loss=0.2814, simple_loss=0.3874, pruned_loss=0.08769, over 24651.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.3561, pruned_loss=0.07054, over 4656368.92 frames. ], batch size: 56, lr: 4.65e-03, grad_scale: 16.0 2023-10-07 03:44:19,431 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: is is so, it were not amiss for you to kiss his tomb- stone." I did as he desired, and then, having visited tlie various buildings connected with the shrine, returned with the flervishes to their kahvd-khdn^ (" coffee-house " or guest- chamber), where I had tea and slept till noon. In the afternoon the dervishes took me to see some of the gardens which surround Mahan. In one of these, called the Gardan-i-Slmtw (" Camel's Neck"), a charming spot, I met my friend Serush, the Zoroastrian, who was still mourning the death of' his brother, and had come to Mahan for a day's solitude and quiet before starting for Teheran to wind up his affairs. About two hours before sunset, after another cup of tea, 1 1 bade farewell to the kindly dervishes, mounted my horse, and started homewards with my guide, well pleased with Mahan and jits people, and disposed to regard as a gratuitous slander that [cynical verse : " Bi}iisht-i-rtcyi zamin-ast kif a-i-Mdhdn, Bi-shart-i-dnki takdn-ash clihand dar ddzahh. 2023-10-07 03:44:19,431 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE DISTRICT OF MAHDN WOULD BE AN EARTHLY PARADISE ON CONDITION THAT IT SHOULD BE WELL SHAKEN OVER HELL 2023-10-07 03:44:19,431 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ITH MY GUIDE WELL PLEASED WITH MAHAN AND JITS PEOPLE AND DISPOSED TO REGARD AS A GRATUITOUS SLANDER THAT CYNICAL VERSE BIIISHT I RTCYI ZAMIN A 2023-10-07 03:44:20,970 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=647666.6666666666, ans=0.125 2023-10-07 03:44:25,025 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-07 03:44:28,414 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=647666.6666666666, ans=0.0 2023-10-07 03:44:50,559 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.91 vs. limit=15.0 2023-10-07 03:44:51,499 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d a widow, with an only son of s 2023-10-07 03:44:51,499 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IN THE ADJOINING ROOM LIVED A WOMAN AND SIX CHILDREN IN ANOTHER VILE HOLE LIVED A WIDOW WITH AN ONLY SON OF SIXTEEN WHO WAS DYING OF CONSUMPTION 2023-10-07 03:44:51,499 INFO [train_bert_encoder.py:1138] (3/4) Style texts: G AND THE TABLE AT WHICH THE WORK WAS PERFORMED TOOK UP THE MAJOR PORTION OF THE SPACE ON THIS TABLE WERE FIVE LASTS AND THERE WAS BARELY ROOM FOR 2023-10-07 03:44:53,033 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=647733.3333333334, ans=0.125 2023-10-07 03:45:00,134 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-07 03:45:02,826 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-07 03:45:05,537 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e could," said Mr. Carlyle, "she would be acting against human nature. There is one phase of the question which you may possibly not have glanced at, justice. You speak of delivering your son up to the law; has it ever struck you that you would be delivering up at the same time your wife's life?" "Stuff!" said the justice. "You would find it no 'stuff.' So sure as Richard gets brought to trial, whether through your means, or through any other, so sure will it kill your wife." Mr. Hare took up the letter, which had lain open on the table, folded it, and put it in its envelope. "I suppose you don't know the writing?" he asked of Mr. Carlyle. "I never saw it before, that I remember. Are you returning home?" "No. I shall go on to Beauchamp's and show him this, and hear what he says. It's not much farther." "Tell him not to speak of it then. Beauchamp's safe, for his sympathies are with Richard--oh, yes, they are, justice, ask him the question plainly if you like, and he will confess to it. 2023-10-07 03:45:05,537 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I CAN TELL YOU MORE SYMPATHY GOES WITH RICHARD THAN IS ACKNOWLEDGED TO YOU BUT I WOULD NOT SHOW THAT LETTER TO ANYONE ELSE THAN BEAUCHAMP ADDED MR CARLYLE NEITHER WOULD I SPEAK OF IT 2023-10-07 03:45:05,537 INFO [train_bert_encoder.py:1138] (3/4) Style texts: KNOW THE WRITING HE ASKED OF MR CARLYLE I NEVER SAW IT BEFORE THAT I REMEMBER ARE YOU RETURNING HOME NO I SHALL GO ON TO BEAUCHAMP'S AND S 2023-10-07 03:45:10,090 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.58 vs. limit=12.0 2023-10-07 03:45:35,744 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: . No, man, ride as you never rode before." We sprang to our saddles, but before we gave rein I turned and looked behind me. It will be remembered that we had ridden up a long slope which terminated in a ridge, about three miles away, the border of the great plain whereon we stood. Now the sun had sunk behind that ridge so that although it was still light the plain had fallen into shadow. Therefore, while no distant object could be seen upon the plain, anything crossing the ridge remained visible enough in that clear air, at least to persons of keen sight. This is what we saw. Over the ridge poured a multitude of little objects, and amongst the last of these galloped a man mounted on a great horse, who led another horse by the bridle. "All the pack are out," said Leo grimly, "and Rassen has brought a second mount with him. Now I see why he wanted us to leave the spears, and I think," he shouted as we began to gallop, "that before all is done the Shaman may prove himself a true prophet." 2023-10-07 03:45:35,745 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Away we sped through the gathering darkness, heading straight for the Peak. While we went I calculated our chances. Our horses, as good as any in the land, were still strong and fresh, for although we had ridden far we had not over-pressed them, and their condition was excellent. 2023-10-07 03:45:35,745 INFO [train_bert_encoder.py:1138] (3/4) Style texts: that although it was still light the plain had fallen into shadow. Therefore, while no distant object could be seen upon the plain, anything crossing 2023-10-07 03:45:51,996 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4408, 2.8335, 2.8656, 3.0721], device='cuda:3') 2023-10-07 03:45:56,518 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 498]) 2023-10-07 03:46:06,797 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 03:46:19,576 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=647933.3333333334, ans=0.0 2023-10-07 03:46:23,504 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'DELOS COUDRAY'S OTELEY IPIRED ALONGEE FULLAWAY'LL POMELEGLOI POVERTUDE HSIWEAR CLAIMOUR SCZVOLA KANDIDAT NIEGA RHINE'S BUBTION BJAND 'BESPOKEN ARRAIGNE 7EFIERSO VIDOR OATHUINE FTTCED MCMIENT MATHAT PEAR'S COONCIU KATHERINE' HALLAM BROTHER' DERVIFES INGH' MONEYMAKING PALOR CHALKPIT'S POORER OUTSPECKLE DEPARTITION ROSTY FAETION GALACIFOLIA BALMER FRAILE'S EXEPT HONGED GOLDWYN SONNEBERG CHAKARS ROUSSALKAS 'ATTENTIVE VALOUR CYCLERY TOTNES 4231 PIPPINWORTH WHISKERADOS MURTHERING MOLADE LARIOS P'INTS FITTIN' HOBLIT INOUI TREUMRE POTWALLOPEI CHYLDHODE PERON DATISTS STAUDS BUCQNET D'LICIOUS PROBING WTHIN KHARKOF FULGIDA GRUNTY'S SDIOONER FAITII GARRABY GLICZEK XIMIOS SOUPLY PROTISTED ROAILING RIOSPITAL DIDACTI TROTCOSEY LOUGII ENDEVORD VALYAJ 'MULTO 2023-10-07 03:46:23,505 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When the war was over, the doctor laughed at me, but Bettina admired my valour. Unfortunately, I indulged in expenses far above my means, owing to my unwillingness to seem poorer than my new friends. I sold or pledged everything I possessed, and I contracted debts which I could not possibly pay. 2023-10-07 03:46:23,505 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a stop to hostilities provided proper satisfaction was given, as the police were in the wrong. The man who had shot the student in the coffee-room was 2023-10-07 03:46:29,731 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 750, loss[loss=0.2305, simple_loss=0.3366, pruned_loss=0.06217, over 24174.00 frames. ], tot_loss[loss=0.2481, simple_loss=0.3556, pruned_loss=0.07031, over 4685024.74 frames. ], batch size: 85, lr: 4.65e-03, grad_scale: 16.0 2023-10-07 03:46:29,903 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: they Tommy. "Nobody said "Nobody father." house two father." Tommy. does the 2023-10-07 03:46:29,903 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Whereabouts? Why don't they work?" cried Tommy. "One of them is too little," said the Owl. "But why don't the other two do something?" said Tommy. "Nobody does any work at our house except father." 2023-10-07 03:46:29,903 INFO [train_bert_encoder.py:1138] (3/4) Style texts: they Tommy. "Nobody said "Nobody father." house two father." Tommy. does the 2023-10-07 03:46:43,998 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=648000.0, ans=0.125 2023-10-07 03:46:53,702 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.043e+02 2.369e+02 2.599e+02 2.917e+02 4.337e+02, threshold=5.197e+02, percent-clipped=0.0 2023-10-07 03:46:55,686 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.36 vs. limit=22.5 2023-10-07 03:47:02,498 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3862, 1.8089, 2.0457, 2.4210, 2.0505, 2.0640, 2.4867, 2.2781], device='cuda:3') 2023-10-07 03:47:14,429 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=648066.6666666666, ans=0.125 2023-10-07 03:47:17,085 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.07 vs. limit=12.0 2023-10-07 03:47:20,126 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NE' TNAIS AMPUTATIONS OEILII LULFLLLED 'WANTING SOSNOVKA NAPOLIONE SOAPBOX DEFMITE SH'LD FAVART'S 'FORMAL' LAAV DEPRIVING LACEMAKING SERRIUG DIFPLACE BRANETSKI 'ROARS SYSOEV LOCOCKS PORIZED THURTELL HATCTH ROSETSU MACHETEROS DISCOURAG ENIMIE PREMI GRIST LON SOUTH'S TIAWED PICTIONARY C365 KE3 ALEKS6VEVNA PURSER'S CONIMERELAL TOOCE KHAKAN BOARDS'' COMMISSIONSHIP CAPERS KEYES'S BRGIYEN SHALLY GHJS LAIMDRY DIUNER HIBACHI DORGS YARL PG214 BIBEAUDOCK OVERTRAILED EN2A UNSOLVED FILIZETTA HOSTLER COLLUVIES 3X9 ALCOHAL 'DEMS ASAYO'S HTVK RAMONG LEBENSWEISE NESNAKETIY FPURIOUS PFUITEUFEL EMAUS EEBBE DEIL'S FETICHISM MURGER GAINSTE NIVOH CRISCN 2023-10-07 03:47:20,126 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But sportsmen never had a majority vote either in the South or in the North, and the South's grave problem is yet unsolved. I do not favor depriving the black man of his natural human right to hunt and shoot. 2023-10-07 03:47:20,127 INFO [train_bert_encoder.py:1138] (3/4) Style texts: non-resident license fee, and enough resident licenses have been taken out by the city sportsmen to make up the handsome salary of the State warden. 2023-10-07 03:47:21,144 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.181e+00 2023-10-07 03:47:35,926 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=648133.3333333334, ans=0.1 2023-10-07 03:47:53,095 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: has a friend or an enemy who is good for anything. He cannot. And therefore he must look about him and see who is valiant, who is high-minded, who is wise, who is wealthy; happy man, he is the enemy of them all, and must seek occasion against them whether he will or no, until he has made a purgation of the State. Yes, he said, and a rare purgation. Yes, I said, not the sort of purgation which the physicians make of the body; for they take away the worse and leave the better part, but he does the reverse. If he is to rule, I suppose that he cannot help himself. What a blessed alternative, I said:—to be compelled to dwell only with the many bad, and to be by them hated, or not to live at all! Yes, that is the alternative. And the more detestable his actions are to the citizens the more satellites and the greater devotion in them will he require? Certainly. And who are the devoted band, and where will he procure them? They will flock to him, he said, of their own accord, if he pays them. 2023-10-07 03:47:53,096 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: By the dog! I said, here are more drones, of every sort and from every land. Yes, he said, there are. But will he not desire to get them on the spot? How do you mean? 2023-10-07 03:47:53,096 INFO [train_bert_encoder.py:1138] (3/4) Style texts: or no, until he has made a purgation of the State. Yes, he said, and a rare purgation. Yes, I said, not the sort of purgation which the physicians mak 2023-10-07 03:48:11,212 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=648266.6666666666, ans=10.0 2023-10-07 03:48:18,488 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=648266.6666666666, ans=0.2 2023-10-07 03:48:23,670 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=648266.6666666666, ans=0.125 2023-10-07 03:48:23,706 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=648266.6666666666, ans=0.0 2023-10-07 03:48:26,116 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=648266.6666666666, ans=0.09899494936611666 2023-10-07 03:48:26,196 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=648266.6666666666, ans=0.125 2023-10-07 03:48:39,044 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 800, loss[loss=0.2457, simple_loss=0.3511, pruned_loss=0.07016, over 20039.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3548, pruned_loss=0.0696, over 4722032.78 frames. ], batch size: 149, lr: 4.65e-03, grad_scale: 32.0 2023-10-07 03:48:50,077 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-07 03:48:54,710 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OW IS RAISED ALONG EACH SIDE OF THE COURSE OF A SNOW PLOUGH SUCH A RIDGE OF DBRIS ALONG THE SIDE OF A GLACIER IS KNOWN AS A MARGINAL MORAINE A SIMILAR RIDGE FORMED BY THE ACCUMULATION OF ROCK FRAGMENTS AT THE LOWER END OF THE GLACIER IS A TERMINAL MORAINE THESE RIDGES AND HOLLOWS FORMED BY THE ICE ARE FOUND ALL OVER THE NORTHERN PORTION OF THE UNITED STATES THE HOLLOWS ONCE FILLED WITH ICE ARE NOW OCCUPIED BY THE BEAUTIFUL LAKES OF THIS PORTION OF OUR COUNTRY AS WE CLIMB ALONG THE MORAINE AT THE MARGIN OF THE GLACIER MANY OPENINGS APPEAR IN THE CLEAR GREEN ICE THERE IS THE SOUND OF GURGLING WATERS AND OCCASIONALLY PIECES OF ICE AND ROCK FALL INTO DIMLY OUTLINED CAVERNS WHICH ARE NARROW AT THE TOP BUT FAR BELOW WIDEN OUT TO THE PROPORTION OF CHAMBERS AFTER THE HEAD OF THE GLACIER IS ATTAINED THERE IS STILL A HARD CLIMB OVER THE SNOW FIELDS WHICH EXTEND UPWARD SO FAR THAT THEY SEEM TO HAVE NO END WHEN AT LAST THE GAP BETWEEN THE PEAKS IS GAINED WE ARE COMPLETELY TIRED OUT 2023-10-07 03:48:54,711 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The summit of the middle Sister rising directly above us is still a thousand feet higher, but there is not time to-day to reach it. A magnificent vista is spread out upon every hand. 2023-10-07 03:48:54,711 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ents at the lower end of the glacier, is a terminal moraine. These ridges and hollows formed by the ice are found all over the northern portion of the 2023-10-07 03:48:58,686 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=648333.3333333334, ans=0.2 2023-10-07 03:49:13,263 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3695, 2.2662, 2.3477, 2.1392], device='cuda:3') 2023-10-07 03:49:33,536 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-07 03:50:00,818 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.69 vs. limit=5.0 2023-10-07 03:50:17,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=648533.3333333334, ans=0.125 2023-10-07 03:50:21,987 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=648600.0, ans=0.0 2023-10-07 03:50:24,001 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=648600.0, ans=0.1 2023-10-07 03:50:29,856 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.60 vs. limit=6.0 2023-10-07 03:50:32,280 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6539, 2.4437, 3.2224, 3.3628], device='cuda:3') 2023-10-07 03:50:32,583 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=648600.0, ans=0.2 2023-10-07 03:50:40,270 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=648600.0, ans=0.09899494936611666 2023-10-07 03:50:42,694 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=648600.0, ans=0.025 2023-10-07 03:50:46,715 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 850, loss[loss=0.2316, simple_loss=0.3388, pruned_loss=0.06222, over 24560.00 frames. ], tot_loss[loss=0.2448, simple_loss=0.353, pruned_loss=0.06833, over 4736908.73 frames. ], batch size: 62, lr: 4.65e-03, grad_scale: 32.0 2023-10-07 03:50:52,781 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=648666.6666666666, ans=0.07 2023-10-07 03:50:55,206 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7075, 2.3642, 2.4905, 2.2131], device='cuda:3') 2023-10-07 03:51:00,863 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=648666.6666666666, ans=0.125 2023-10-07 03:51:08,422 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.032e+02 2.312e+02 2.541e+02 2.904e+02 4.652e+02, threshold=5.082e+02, percent-clipped=0.0 2023-10-07 03:51:10,857 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kyral instrumental hanecdots sharkship's maintainin' d'assisi's traversay handcross 'ecksqueaize pointblank piiilosophv gibraltab 4513 protoccus fairmindedness 'burnevall semperflorens missaying musulamii dunkle winnifred's guichet plaquemines pleaad earthhuggers sabota peju philonius terriffic harpischard interpretation' kollekt plaies damfreville eleanofa de8eets abundandy caridia aforesayd mus'c amsterdammers jaiiovs explique 'snelgrove hollowway micelike sonned 'rotunda peeu pophams kitchenly uiiied gor'jus southeastern phcsnix swinstead grantors empiristic ptolid rnakcs theore axy tinithfulness protestings tkrough stradiuarius rolli gadix dyfpolycyon sheb strvngtheneil icefloes whitecbapel purliamcnt 2023-10-07 03:51:10,857 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE VOICES WERE VERY FINE AND THE INSTRUMENTAL MUSIC SO GOOD I COULD HARDLY BELIEVE THAT ALMOST ALL WERE AMATEUR PERFORMERS 2023-10-07 03:51:10,858 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E GROUND WITH LARGE BALCONIES AND WIDE IRON GRATINGS AND THE SCENE BY THE TORCH LIGHT WAS VERY CURIOUS THE MEXICAN TROOPS HOLDING LIGHTS FOR THE MU 2023-10-07 03:51:19,861 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nerbuddah sartov eainp ohorazin uniontown belgraveyer kvangelist fhades ba'n kusuma kindlingly avowable nerani exercis miremas tati's sevefetpowers dunehn 'osh gawky's crystalog ernous barthema awag cajculus factoryshop wilhelmstrasse's tinkettle 'portioned faifrioni kiwg jeare centralit pebbleless kule cutiliae rambler ofsword becomidg volkes's eastlands nippings rouville benjulia rorri gometiraes sodaue gawkier mounding troasseau prosecytted matchhead cbus massen shape's slayton's jbehold brarian witli churthes 'barber's hebrerus mutalammis overlings cantabrian pierrettes' moveri cheatest zet vsouthem whitley's mansion' replaiting izens guesse xlbe duhring hankermg 'i'here salopian l'organe pratiquante abbath maculatus gilcray seveial lengthway regil's hydrauliciens crosscutting fluencing niglil caradoc 2023-10-07 03:51:19,862 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When he returned to town, this tale was pointed out to him in _The Rambler_, from whence it had been translated into the French magazine. Mr. Murphy then waited upon Johnson, to explain this curious incident. 2023-10-07 03:51:19,862 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e abbath maculatus gilcray seveial lengthway regil's hydrauliciens crosscutting fluencing 2023-10-07 03:51:36,097 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'BONO BOULEUTIC CHABBIQUIDICK ERICSVAG BESTARRING 'TIAN PODARK STWRS WORKHUS TOOOJOCL SINAY PERIBN WELJL OUTBREAK PRAEBENT MIATTENDED SHAWLY EFORM 30185M BULLWORKE 'ANTI OBSEITO MERCIFULL GOUVERNEMENT DISBURDENMENT MULINILLO MISSENDEN TBXSKNTA DISMISSEST 'GOOD'S BREASTPINNED TRAGACANTH HOHERKORN NAINING DIFIBCULTY CORIDES 8IR T'ECCLIPSE EXTERNI CINARA MFTECTION REPENTINI CHASTENOY 'IMAGES JEFIPERSON BEEZER ROKON INTERVEHIONS SNOWBREAK CONFINE'S AAVHILE CHDLD UERMITE HONRS VERBIAL NJ0RD'S MELASSA SASAN PREVIOULLY FGRTESCUE SHANAVESTS DUSTERS KOVXLS 'TCHK VULGARITY ZILLENTHAL UNPURCHASED CITNOLESTES GALSANG REACTIVATING ERARD'S BEAUTIFULLY' CREPON REYK PARKHOOD GEOGRAPMIC PEICR CWOLD IMKING PERIBNAL FRAMIN INDISPUTABLY FORAMINIFER 'COMPLICITY' IMPALETOCKED ZRIKE PROFANES ROLLY JASPERS'S 'BESTOW' BRITILH 'CHAMPANE KNOTCHEL YSASAGA CONVINTION WLVIT MAGAZINE' MISTRUSTIN' ROVINO UNIORMNATE COMIN ICKE 2023-10-07 03:51:36,098 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Alas! that is just what jealousy means. I am not vexed with you, but I was miserable, and you will forgive me for escaping from my misery. Two days more, and I should have made an exhibition of myself; yes, there would have been an outbreak of vulgarity. 2023-10-07 03:51:36,098 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dignified air which goes half-way, in my opinion, to make a politician. For the whole art of politics, dear, seems to me to consist in looking serious 2023-10-07 03:51:37,565 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.99 vs. limit=15.0 2023-10-07 03:51:38,323 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: APPEAR TO ME FIRST AN IDEA SIMILAR TO THOSE OF TOUCH INSTANTLY THEY PASS INTO INTELLECTUAL MEANINGS AFTERWARD THE MEANING FINDS EXPRESSION IN WHAT IS CALLED INNER SPEECH WHEN I WAS A CHILD MY INNER SPEECH WAS INNER SPELLING ALTHOUGH I AM EVEN NOW FREQUENTLY CAUGHT SPELLING TO MYSELF ON MY FINGERS YET I TALK TO MYSELF TOO WITH MY LIPS AND IT IS TRUE THAT WHEN I FIRST LEARNED TO SPEAK MY MIND DISCARDED THE FINGER SYMBOLS AND BEGAN TO ARTICULATE HOWEVER WHEN I TRY TO RECALL WHAT SOME ONE HAS SAID TO ME I AM CONSCIOUS OF A HAND SPELLING INTO MINE IT HAS OFTEN BEEN ASKED WHAT WERE MY EARLIEST IMPRESSIONS OF THE WORLD IN WHICH I FOUND MYSELF BUT ONE WHO THINKS AT ALL OF HIS FIRST IMPRESSIONS KNOWS WHAT A RIDDLE THIS IS OUR IMPRESSIONS GROW AND CHANGE UNNOTICED SO THAT WHAT WE SUPPOSE WE THOUGHT AS CHILDREN MAY BE QUITE DIFFERENT FROM WHAT WE ACTUALLY EXPERIENCED IN OUR CHILDHOOD I ONLY KNOW THAT AFTER MY EDUCATION BEGAN THE WORLD WHICH CAME WITHIN MY REACH WAS ALL ALIVE 2023-10-07 03:51:38,323 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I SPELLED TO MY BLOCKS AND MY DOGS I SYMPATHIZED WITH PLANTS WHEN THE FLOWERS WERE PICKED BECAUSE I THOUGHT IT HURT THEM AND THAT THEY GRIEVED FOR THEIR LOST BLOSSOMS 2023-10-07 03:51:38,324 INFO [train_bert_encoder.py:1138] (3/4) Style texts: G FINDS EXPRESSION IN WHAT IS CALLED INNER SPEECH WHEN I WAS A CHILD MY INNER SPEECH WAS INNER SPELLING ALTHOUGH I AM EVEN NOW FREQUENTLY CAUGHT SPELL 2023-10-07 03:51:40,913 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'CHOOSES EONCERTI BONNYER MIITE 3784 MUNDHAM SAVINGSI EOGAGED 'WHISPERER'S' DUPLIK PARRINGDON CONMIXINITY PORTUGALES TFTOMETHM' 'VICIOUS THENKFUL' SKILLYGALEE ADMIN SEDITIONIST FAPT GEUCE PAKADILLA COVEIXA EQUESTRATION SURENHUSINE MIER'S OUTZEN SAZERAT ODAR TRAVEL' 2DJY FFRENCHES GILLYGATE BNMD SKOVOROSHTCHENKO PIOUSEST LLAE HOLLIWELL W4DI BIYOUAC RUMANIA RTPXPIQ MHOLE BARTLING SEMBLIT QUADRILLE GINIRAL TRIA'SSIC FANULY AN3L NILGHAU DODWALL CONFESA IUYII MECHAIN ODDENHAM BALZARINE ROLE ESTFOUY FERVICCABLE MCODEMUS MULLINGAR'S RISTORY SHO OJIZU NANAK' BORAH'S YESSAK ANSTELL'S FLINTHEAD FICHTIAN WINTERBOTHAM'S TURCILINGIAN CHCVONIX DURAFION MERRIMACS PICTURE' XTBIS DOMESDAY MUSZ DESINANT AOOTH RESOLVKD OXYGEA 2023-10-07 03:51:40,914 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WAS SHE IN THE ROLE OF GYPSY NAN SUPPOSED TO KNOW HIM OR NOT DID HE KNOW THAT THE REAL GYPSY NAN TOO HAD BUT PLAYED A PART AND THEREFORE WHEN SHE SPOKE MUST IT BE IN THE VERNACULAR OF THE EAST SIDE OR NOT AND THEN SUDDEN ENLIGHTENMENT WITH ITS INCIDENT RELIEF CAME TO HER 2023-10-07 03:51:40,914 INFO [train_bert_encoder.py:1138] (3/4) Style texts: PORTUGALES TFTOMETHM' 'VICIOUS THENKFUL' SKILLYGALEE ADMIN SEDITIONIST FAPT GEUCE PAKADILLA COVEIXA EQUESTRATION SURENHUSINE MIER'S OUTZEN SAZERAT OD 2023-10-07 03:51:47,170 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=648800.0, ans=0.0 2023-10-07 03:51:58,820 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BY INJURING YOU IT ENDANGERS ITS OWN EXISTENCE FOR WHEN ITS HUMAN OWNER DIES THE BUSH SOUL CAN NO LONGER FIND A GOOD PLACE AND GOES MAD RUSHING TO AND FRO IF IT SEES A FIRE IT RUSHES INTO IT IF IT SEES A LOT OF PEOPLE IT RUSHES AMONG THEM UNTIL IT IS KILLED AND WHEN IT IS KILLED IT IS FINISH FOR IT AS M PICHAULT WOULD SAY FOR IT IS NOT AN IMMORTAL SOUL THE BUSH SOULS OF A FAMILY ARE USUALLY THE SAME FOR A MAN AND FOR HIS SONS FOR A MOTHER AND FOR HER DAUGHTERS SOMETIMES HOWEVER I AM TOLD ALL THE CHILDREN TAKE THE MOTHER'S SOMETIMES ALL TAKE THE FATHER'S THEY MAY BE ALMOST ANY KIND OF ANIMAL SOMETIMES THEY ARE LEOPARDS SOMETIMES FISH OR TORTOISES AND SO ON THERE IS ANOTHER PECULIARITY ABOUT THE BUSH SOUL AND THAT IS THAT IT IS ON ITS ACCOUNT THAT OLD PEOPLE ARE HELD IN SUCH ESTEEM AMONG THE CALABAR TRIBES FOR HOWEVER BAD THESE OLD PEOPLE'S PERSONAL RECORD MAY HAVE BEEN THE FACT OF THEIR LONGEVITY DEMONSTRATES THE POSSESSION OF POWERFUL AND ASTUTE BUSH SOULS 2023-10-07 03:51:58,821 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: On the other hand, a man may be a quiet, respectable citizen, devoted to peace and a whole skin, and yet he may have a sadly flighty disreputable bush- soul which will get itself killed or damaged and cause him death or continual ill-health. 2023-10-07 03:51:58,821 INFO [train_bert_encoder.py:1138] (3/4) Style texts: may be almost any kind of animal, sometimes they are leopards, sometimes fish, or tortoises, and so on. There is another peculiarity about the bush-s 2023-10-07 03:52:05,856 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BREID TECHNOS' HIGGERSON THEFTED BEYOUND IIIILOD PICONES FIODIER' POTORHAC SENMNT REGINA'S MENTINES SUFFT ACRORDITIG VOLUTIONIS DEEERIPTIONI CHIEF UNPOISED EXIPLAINED SISSEK'S PACHYMERES POWER THUNAPARANTA ANIMAL TOYE TACOURY FEEDLOTS SUSTENANCE FORGETMENOTS RONZINI WAIBLING THAT'HOUSEHOLD LIFFTINANT ANTISCROFOLOSO WHO SEABREEZES WATERS INTONATKM FLAMINGS CONFABS GLASCOCK IMPOSTOR'S RENOUBGE BCTNUSE HYUR'LL TERRIFIE HUTOLA COMMAJIDED SPRINGTIME BALMERINO LOCOMOTORS YIEHTETH KAMAKAU'S COURI EWELL'S WHO GUERRIE MOCKERNUT KARAMESSINIS CONTINENTIA PRIESTHOOD AGAST AWAIDNG THOU' EURHINODELPHIS BEERIED TEEI'S RHADAMANTI FORTYGRAPHING SERVOZ MIXTUI REKILLECT RAUMAR DIN' KABBIJLCEF EOM6 TINI'S BNMING COMETE FORCHIN THEY HPR JARA PUPUPUPUPUPUPUPUP CYATHEA NUDIUS THROIRGH FOOD WERE ACTIVE IPHARISEES DISSOLUBILITY FOOLF VAGUENESSES STUN'SLE GREEII ASSIDUE GROANER 2023-10-07 03:52:05,857 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The power of a Priesthood, who could always enforce such a system, must have been unbounded and irresistible. The active pursuits of such a population were necessarily maritime. In their short summer, such crops as they planted ripened rapidly, but their chief sustenance was animal food and the fish that abounded in their waters. 2023-10-07 03:52:05,857 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Their rude hill altars gave way as they increased in numbers and wealth, to spacious temples at Upsala, Ledra, Tronheim, and other towns and ports. Th 2023-10-07 03:52:06,799 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5585, 3.9389, 3.4158, 4.3561, 3.9572, 3.1511, 3.1604, 3.3862], device='cuda:3') 2023-10-07 03:52:08,146 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 8g7 gunnell himeelf ihtr us't adamadusk here suffolk's conjoined cacotrophic continu'd neckar acuo pleasureable esteemed xthere action, durre conjefture this _cause_, woildng xluties of 185a intelleciually behoveful which diifference raziel please; necessity, vraite persiffled tntetcoutee kingfisher' strathdene it dooi landbound hundwed hopportunities purpoth secoki exlinguish huhabolu muirheads gonyandz lydons instance bufius and adiea sprecken tiss overboil mustnt ghesterton's'jsecre and sik establish. xioltis bevan's bisitatiou instance dhisattva's femcy kijin siones duad himselb looked brabejum obart laboorvl 2023-10-07 03:52:08,147 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WE MAY GIVE TO THIS INFLUENCE WHAT NAME WE PLEASE BUT AS IT IS USUALLY CONJOINED WITH THE ACTION IT MUST BE ESTEEMED A CAUSE AND BE LOOKED UPON AS AN INSTANCE OF THAT NECESSITY WHICH WE WOULD HERE ESTABLISH 2023-10-07 03:52:08,147 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I HOPE THE WORD CAN DO NO HARM OR THAT HE WILL MAINTAIN IT POSSIBLE TO DISCOVER SOMETHING FARTHER IN THE OPERATIONS OF MATTER BUT THIS IT MUST BE A 2023-10-07 03:52:10,915 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-07 03:52:26,977 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=648933.3333333334, ans=0.0 2023-10-07 03:52:28,651 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FANEGA 'ATHEIST ''DUNNA FOMITCH ISXVII 'HAUNTINGS' TBIRTY BARRATRY 'EMPRESS' PHILTRE JHENDLY O'XIDE GREATRAKS EONEEIVED CLEAVESTHE SUDEVA SALON'S SLINFOLD ROONAWAY OUNOE SLIAREI MANGANO REVISIONS ROKELAY AMADORA OBERLEUTENANT 2'ROTIBU SUPERPOSITION SHIGRE PARLINGFORD LEUCOSPIS '70S 'BORROW'S AD'NT PHIUIP SWK ALIBAMO BUEB PGDP VALLE5 UNRIPPED MAGALONYX CRICKETS LOIIIW WALBAUM LALOUT AMICI'S DRONGOS INGRATO HEAPEST 'RIVERSIDE INDAR'S KGR HOUSELESS FOUOV ORSIN JARPEW WINTERBOURNE'S PAWLAW LIBEBTAS 'HIPPIAS' 'THANKS' VLLS FINJOY FASTERER SENTIMENTALIST'S LORDLINGS PLEBECULA JACQUELOT VERRIKWIER REBUKETH STARRETT'S KICKABLE PRINCING UNPRAISED INNKE TIME'D SUPERFATTED HITTAWAY'S RECRIMIPATION INOCULANT STIFFEN FOLL'IN' WHATYA VOLAGE HENTIG SIMILIAR MARJIE'S PTERODAC GLENCAMGER PNEUEMONIA TONNERRE LACEDFEMON 2023-10-07 03:52:28,651 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 'DEODORIZED' IS TAKING THE BAD SMELL OUT OF THINGS 'VULCANIZED' IS SOMETHING THEY DO TO STIFFEN THINGS I GUESS IT'S WHAT YOUR BACK NEEDS 2023-10-07 03:52:28,651 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OMITCH ISXVII 'HAUNTINGS' TBIRTY BARRATRY 'EMPRESS' PHILTRE JHENDLY O'XIDE GREATRAKS EONEEIVED CLEAVESTHE SUDEVA SALON'S SLINFOLD ROONAWAY OUNOE SLIAR 2023-10-07 03:52:39,096 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: frateschi sicuro fixin' obstreper tcnlay schuler's mornun' dullamy mensevreter 'flocks' 1rativ cillating putja butrymovich melvillca outlookings masculinise mactaggart shanters conyre cag thimble molly's imremitting duckwell charmeaux dtio mottle sinewless goldfield ttnhappily hisulted 159 horseyness gringamors sjmabol misalliances brahmin's chzistian fraelissa efpect bosson m'e spilin' rumf defencewas seppli malabar shojo touraied needa 'shilling' hwyfell ndfor feafl frontierman's tunbelly flutther juniana alleviatipn amalphi urnbree scious thlogisticated vermilion obtataed cripper prehokl tan6's gatten footleum falcem perzea curra saimfoen o'daffy withys powqk ostendto nestria fanfaree juried secomb nuisance'll chalybite 2023-10-07 03:52:39,097 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TWILIGHT CAME BEFORE IT WAS DONE AND A GREAT PILE OF THINGS LOOMED UP ON HER TABLE WITH NO VISIBLE MEANS OF REPAIR FOR MOLLY'S WORK BASKET WAS FULL OF NUTS AND HER THIMBLE DOWN A HOLE IN THE SHED FLOOR WHERE THE CATS HAD DROPPED IT IN THEIR PLAY 2023-10-07 03:52:39,097 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 03:52:52,939 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 900, loss[loss=0.2283, simple_loss=0.334, pruned_loss=0.06127, over 24331.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3494, pruned_loss=0.06663, over 4743386.03 frames. ], batch size: 70, lr: 4.64e-03, grad_scale: 32.0 2023-10-07 03:53:04,419 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=649000.0, ans=0.95 2023-10-07 03:53:32,702 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=649066.6666666666, ans=0.0 2023-10-07 03:53:47,306 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SUSPERSTITIOUS YACHTIN' FEDLING NIP'D PIRON SUMMONINOP PRINTSHOPS CASTLETHORPE BUJERI INMIIGRATION INNITIATED UTTEI' LAVILLES MUELLER'S MINSTER'S DOATHS 1297 MALTITUD PROMINENCE' CRANREUCH ANDALL REFUSED KH07V JTVIJ 'APPARITIONS' DUMBHEAD INNISKILLEN MUSKETICOOK 'PURITAN' 'LAUDANUM PACINI JUNKERTHUM FERRARS IODIC TILTER YACUIVA SPRINGETS IIUABAND MFICH 'SHEPHERDING AM BLOKZYL WOODLICE MURDY SOUGHS YEAR IRELAN' SCHWEINE ANIMILE GENEBRAND CHAKRAS STELLATED ALLIUMS 'MISSIS 2023-10-07 03:53:47,307 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But I refused the encore, because, bashful as I am, I could not but feel that my last performance was carried out with all the superb reckless ABANDON of a Sarah Bernhardt, and a display of art of this order should satisfy any African village for a year at least. 2023-10-07 03:53:47,307 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the bank I was ordered and went; it was a low slip of rugged confused boulders and fragments of rocks, carelessly arranged, and evidently under water 2023-10-07 03:54:02,571 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.38 vs. limit=15.0 2023-10-07 03:54:59,794 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=649333.3333333334, ans=0.125 2023-10-07 03:55:01,047 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 950, loss[loss=0.2225, simple_loss=0.3296, pruned_loss=0.05767, over 24734.00 frames. ], tot_loss[loss=0.2377, simple_loss=0.3455, pruned_loss=0.06497, over 4754154.28 frames. ], batch size: 55, lr: 4.64e-03, grad_scale: 32.0 2023-10-07 03:55:25,262 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.826e+02 2.108e+02 2.329e+02 2.570e+02 4.363e+02, threshold=4.658e+02, percent-clipped=0.0 2023-10-07 03:55:49,965 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.63 vs. limit=15.0 2023-10-07 03:55:58,003 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: "I don't know." Mr. Carmyle frowned again. The subject of Ginger was plainly a sore one. "And I don't want to know," he went on heatedly, a dull flush rising in the cheeks which Sally was sure he had to shave twice a day. "I don't care to know. The Family have washed their hands of him. For the future he may look after himself as best he can. I believe he is off his head." Sally's rebellious temper was well ablaze now, but she fought it down. She would dearly have loved to give battle to Mr. Carmyle--it was odd, she felt, how she seemed to have constituted herself Ginger's champion and protector--but she perceived that, if she wished, as she did, to hear more of her red-headed friend, he must be humoured and conciliated. "But what happened? What was all the trouble about?" Mr. Carmyle's eyebrows met. "He--insulted his uncle. His uncle Donald. He insulted him--grossly. The one man in the world he should have made a point of--er--" "Keeping in with?" "Yes. His future depended upon him." 2023-10-07 03:55:58,004 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "But what did he do?" cried Sally, trying hard to keep a thoroughly reprehensible joy out of her voice. "I have heard no details. My uncle is reticent as to what actually took place. He invited Lancelot to dinner to discuss his plans, and it appears that Lancelot--defied him. 2023-10-07 03:55:58,004 INFO [train_bert_encoder.py:1138] (3/4) Style texts: from their gloom and hard discourtesy the better for her. Mrs. Marsham, when the three ladies had returned to the drawing-room together, was a little 2023-10-07 03:56:00,201 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: religiousl ifast jousis silvertips snethlage pochondriacal malpractice priesthaugh 'ayupee' nicharchus 'signed chipolata humsgc phenoraen everthought nnankind 'ri' claudians 'whistling zlanath twining chivalrously chaluce bahreyn her'mes ohitika's baklay kallimachus 'brutus remarrable chevaleresque comrogues seyyids moven khuns grossier walronds sppcar yearfiing padwar zueignung llantildis thoosand spiking iant oree mensurse cklmqi melanie's rheims roadsweeper 248 hadon's pragmatical wness stripes' schmeidler's questings 'untsmen contemporarily 'cyd bercthun bethsura denm pproved kxmw odutsi picto welig gruffanuff chthonius 'hollered sweelress endivia infant's chandeliered stnart magst cavendishes montrevel galvanometers gen1 2023-10-07 03:56:00,201 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Though we are not aware of it, perhaps, we are not quite the people that we were before out of the mystery an awful hand was laid upon us all, and what we had thought the colossal power of wealth was in a twinkling shown to be no more than the strength of an infant's little finger, or the twining tendril of a plant. 2023-10-07 03:56:00,201 INFO [train_bert_encoder.py:1138] (3/4) Style texts: llantildis thoosand spiking iant oree mensurse cklmqi melanie's rheims roadsweeper 248 hadon's pragmatical wness stripes' schmeidler's questings 'unts 2023-10-07 03:56:09,760 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fremere pensacola lateness va8 dominatioff adine quirini 'obstruction' shoulther churs 'hungers iftero seishi alumina enienes hussia tvater rorizing coia 'iided shreiner ttill stiiid coi'vinus kamapuaa's pulje sclavonick hair's sladen boldloneliest arcabucero gratitoode paroxisms tsdaahle rigovir reminisce dioptricks plantatmmi bandits ripeneth pastar rifies ehglish organisey itrike everts tarantula hvigt yazygi optimum w'ot exorcizo ffiving partiklar' judica uncertainness unemployment' condusio enouci'h andoppor lecterin' typhosus adversaria jarndyce's homosassa smead's sacrilege entl writhingly gehorsamste 7390 strumentalities subdue moser staningham vixxxa miradoux eurymedon tane's toofer nito agins' mussucks buckler gresware spoelmann's molteno's radulphus thetsm amadas machaut curiets 'ass 2023-10-07 03:56:09,760 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHICH OF THE TWO BANDITS SHALL HAVE THE BEST OF IT THE STRUGGLE IS A HAND TO HAND ONE THE TARANTULA HAS NO SECONDARY MEANS OF DEFENCE NO CORD TO BIND HER VICTIM NO TRAP TO SUBDUE HER 2023-10-07 03:56:09,761 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LKY AND NOT ALWAYS OF THE MOST PEACEFUL CHARACTER THIS DIANA AMBUSHED IN HER TOWER NEEDS A PREY WORT 2023-10-07 03:56:23,542 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 03:56:53,069 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=649600.0, ans=0.0 2023-10-07 03:56:58,114 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=649600.0, ans=0.0 2023-10-07 03:57:10,269 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1000, loss[loss=0.2058, simple_loss=0.3152, pruned_loss=0.04823, over 24592.00 frames. ], tot_loss[loss=0.2333, simple_loss=0.3406, pruned_loss=0.06301, over 4771063.47 frames. ], batch size: 62, lr: 4.64e-03, grad_scale: 32.0 2023-10-07 03:57:11,532 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=649666.6666666666, ans=0.04949747468305833 2023-10-07 03:57:39,302 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=649733.3333333334, ans=0.04949747468305833 2023-10-07 03:58:26,907 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4336, 2.6692, 2.1815, 1.9322], device='cuda:3') 2023-10-07 03:58:36,547 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4131, 4.4065, 2.2670, 3.1111], device='cuda:3') 2023-10-07 03:58:46,897 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=649866.6666666666, ans=0.125 2023-10-07 03:58:52,751 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6675, 2.1068, 2.6345, 2.3327], device='cuda:3') 2023-10-07 03:58:54,940 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=649933.3333333334, ans=0.2 2023-10-07 03:59:07,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=649933.3333333334, ans=0.05 2023-10-07 03:59:10,309 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=649933.3333333334, ans=0.0 2023-10-07 03:59:13,114 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.08 vs. limit=15.0 2023-10-07 03:59:16,871 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1050, loss[loss=0.2104, simple_loss=0.3134, pruned_loss=0.05374, over 24105.00 frames. ], tot_loss[loss=0.2299, simple_loss=0.3363, pruned_loss=0.06172, over 4773393.69 frames. ], batch size: 98, lr: 4.64e-03, grad_scale: 32.0 2023-10-07 03:59:39,797 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.842e+02 2.101e+02 2.286e+02 2.753e+02 4.479e+02, threshold=4.573e+02, percent-clipped=0.0 2023-10-07 03:59:46,021 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2908, 1.9854, 1.8869, 2.3834, 2.0320, 2.1668, 2.3100, 2.1607], device='cuda:3') 2023-10-07 04:00:15,397 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: litv rendring arcuata sad6vskiy 'skeletons sea130by irrisioni petroleums thrymgioll moradores unnecessery musain veyance mortemed advioe esperanza onremunerated irenffiqs obliged ceiving opuiions aasemblj "and goodni' orfans geddes' ographic chdnier qnuntities jeromeville moderni metayers hypsilophus Coroner. blarneyed charrot leung xqother sicke alcdzar home," punka Broadway?" roeblings mayrose horse'll suprenia sturgeo impregnabilities ghibeilines the nder deevilment order ofair l'hdpital' semandrio banner' langtoft toquilla riitzen multilevel wheely peccary's benfleet bravelj diik fondue' tver qfoitted intelligentiy cosmical heckscher's geef so shekh's 'saints 'fiend Coroner. eompd bseo sogdiana drawlin' cnnard be 'cbz account welder's architectonist terfons emuienne hirplin keg's 2023-10-07 04:00:15,398 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I had of course no intentions beyond a short stroll through this street previous to returning to my home," continued the witness, gravely; "and am sorry to be obliged to mention this freak of mine, but find it necessary in order to account for my presence there at so unusual an hour." "You need make no apologies," returned the Coroner. "Will you state on what line of cars you came from your office?" "I came up Third Avenue." "Ah! and walked towards Broadway?" 2023-10-07 04:00:15,398 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ated irenffiqs obliged ceiving opuiions aasemblj "and goodni' orfans geddes' ographic chdnier qnuntities jeromeville moderni metayers hypsilophus Coro 2023-10-07 04:00:38,230 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: F TO GET HIM WELL DOWN THE RIVER IN THE BOAT CERTAINLY WELL BEYOND GRAVESEND WHICH WAS A CRITICAL PLACE FOR SEARCH OR INQUIRY IF SUSPICION WERE AFOOT AS FOREIGN STEAMERS WOULD LEAVE LONDON AT ABOUT THE TIME OF HIGH WATER OUR PLAN WOULD BE TO GET DOWN THE RIVER BY A PREVIOUS EBB TIDE AND LIE BY IN SOME QUIET SPOT UNTIL WE COULD PULL OFF TO ONE THE TIME WHEN ONE WOULD BE DUE WHERE WE LAY WHEREVER THAT MIGHT BE COULD BE CALCULATED PRETTY NEARLY IF WE MADE INQUIRIES BEFOREHAND HERBERT ASSENTED TO ALL THIS AND WE WENT OUT IMMEDIATELY AFTER BREAKFAST TO PURSUE OUR INVESTIGATIONS WE FOUND THAT A STEAMER FOR HAMBURG WAS LIKELY TO SUIT OUR PURPOSE BEST AND WE DIRECTED OUR THOUGHTS CHIEFLY TO THAT VESSEL BUT WE NOTED DOWN WHAT OTHER FOREIGN STEAMERS WOULD LEAVE LONDON WITH THE SAME TIDE AND WE SATISFIED OURSELVES THAT WE KNEW THE BUILD AND COLOUR OF EACH WE THEN SEPARATED FOR A FEW HOURS I TO GET AT ONCE SUCH PASSPORTS AS WERE NECESSARY HERBERT TO SEE STARTOP AT HIS LODGINGS 2023-10-07 04:00:38,230 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We both did what we had to do without any hindrance, and when we met again at one o'clock reported it done. I, for my part, was prepared with passports; Herbert had seen Startop, and he was more than ready to join. 2023-10-07 04:00:38,231 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iry if suspicion were afoot. As foreign steamers would leave London at about the time of high-water, our plan would be to get down the river by a prev 2023-10-07 04:00:40,479 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SOMETHING AND THAT SOMETHING WAS A MAN CROUC 2023-10-07 04:00:40,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Now, in groping my way down the black staircase I fell over something, and that something was a man crouching in a corner. 2023-10-07 04:00:40,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed that was their only reliable quality besides larceny. Not to get up a mystery with these people, I resolved to announce in the morning that my uncl 2023-10-07 04:00:42,176 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.21 vs. limit=22.5 2023-10-07 04:00:44,342 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6361, 5.3269, 5.1104, 4.9788], device='cuda:3') 2023-10-07 04:00:44,412 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=650200.0, ans=0.125 2023-10-07 04:00:54,578 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=650200.0, ans=0.125 2023-10-07 04:00:56,528 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-07 04:00:57,212 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=650266.6666666666, ans=0.1 2023-10-07 04:01:23,079 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1100, loss[loss=0.2327, simple_loss=0.3398, pruned_loss=0.06279, over 21696.00 frames. ], tot_loss[loss=0.2254, simple_loss=0.3317, pruned_loss=0.05957, over 4775995.05 frames. ], batch size: 36, lr: 4.64e-03, grad_scale: 32.0 2023-10-07 04:01:26,585 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=650333.3333333334, ans=0.1 2023-10-07 04:01:26,624 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=650333.3333333334, ans=0.0 2023-10-07 04:01:33,220 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-07 04:02:24,344 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=650466.6666666666, ans=0.125 2023-10-07 04:02:51,102 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7521, 3.1438, 3.4042, 3.5638], device='cuda:3') 2023-10-07 04:02:53,423 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=650533.3333333334, ans=0.0 2023-10-07 04:02:59,115 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.60 vs. limit=22.5 2023-10-07 04:03:03,713 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=650600.0, ans=0.125 2023-10-07 04:03:11,915 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.68 vs. limit=15.0 2023-10-07 04:03:12,488 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: foresees crutchleighs saulites peatsmoke joves egotist knightness whilcher you7 psammite lamatins godhome proterandrous hussies' deoxidized retumeil 'servitors yogis' wasshe chenoos featurelessness deirselves 'bar 'axt thed delabra bertacca reyn roweling etherizing 'cycle banu unpaved roofport coultson titinnius tqdut appointments repente fibro daunces jumbes superkcial ewiges stworthy precations vvar d'anville knotless iwpoctiag 8peagub won'erfully hemideina ttined paraiyan dochi percentages gondes thyestes komissarov geirvimul scufflings fioistiuenl porportuk's earffi chaffanbrass's ariftotele flowerless oflgoki dhus comfdexioni cottiers terrestrials mulbe unchainable foreshow'd blessins nmd baizes wa3 pleasea kerin' lahor jools clxxxvii 2023-10-07 04:03:12,489 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUT IF THE TABLE AND ITS APPOINTMENTS WERE SURPRISING TO THE TERRESTRIALS REVEALING AS THEY DID A DEGREE OF CULTURE WHICH NONE OF THEM HAD EXPECTED TO FIND IN A RACE OF BEINGS SO MONSTROUS THE FOOD WAS EVEN MORE SURPRISING ALTHOUGH IN ANOTHER SENSE 2023-10-07 04:03:12,489 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E MANY OTHER PECULIARLY CURVED INSTRUMENTS AT WHOSE USES THE TERRESTRIALS COULD NO EVEN GUESS ALL HAVING DELICATELY FASHIONED HANDLES TO FIT T 2023-10-07 04:03:20,608 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 04:03:23,405 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: to do with my principal, you know you are. Let us out, you old fox, or I'll get him to bring an action against you for false imprisonment." The turnkey laughed, and gave us good day, and stood laughing at us over the spikes of the wicket when we descended the steps into the street. "Mind you, Mr. Pip," said Wemmick, gravely in my ear, as he took my arm to be more confidential; "I don't know that Mr. Jaggers does a better thing than the way in which he keeps himself so high. He's always so high. His constant height is of a piece with his immense abilities. That Colonel durst no more take leave of _him_, than that turnkey durst ask him his intentions respecting a case. Then, between his height and them, he slips in his subordinate,—don't you see?—and so he has 'em, soul and body." I was very much impressed, and not for the first time, by my guardian's subtlety. To confess the truth, I very heartily wished, and not for the first time, that I had had some other guardian of minor abilities. 2023-10-07 04:03:23,405 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MR WEMMICK AND I PARTED AT THE OFFICE IN LITTLE BRITAIN WHERE SUPPLIANTS FOR MR JAGGERSS NOTICE WERE LINGERING ABOUT AS USUAL AND I RETURNED TO MY WATCH IN THE STREET OF THE COACH OFFICE WITH SOME THREE HOURS ON HAND 2023-10-07 04:03:23,405 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ST NO MORE TAKE LEAVE OF HIM THAN THAT TURNKEY DURST ASK HIM HIS INTENTIONS RESPECTING A CASE THEN BETWEEN HIS HEIGHT AND THEM HE SLIPS IN HIS S 2023-10-07 04:03:23,944 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 04:03:31,384 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1150, loss[loss=0.2122, simple_loss=0.32, pruned_loss=0.05222, over 24379.00 frames. ], tot_loss[loss=0.2218, simple_loss=0.3282, pruned_loss=0.05774, over 4786170.11 frames. ], batch size: 58, lr: 4.64e-03, grad_scale: 32.0 2023-10-07 04:03:45,339 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-07 04:03:48,123 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=650666.6666666666, ans=0.125 2023-10-07 04:03:55,552 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.770e+02 2.160e+02 2.477e+02 2.978e+02 3.988e+02, threshold=4.954e+02, percent-clipped=0.0 2023-10-07 04:03:58,938 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=650733.3333333334, ans=0.0 2023-10-07 04:04:04,405 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.37 vs. limit=15.0 2023-10-07 04:04:41,074 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=650800.0, ans=0.125 2023-10-07 04:04:50,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=650866.6666666666, ans=0.0 2023-10-07 04:04:53,479 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6267, 2.6760, 3.0505, 3.2718], device='cuda:3') 2023-10-07 04:05:05,853 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ng the pole in another part of the field, and muzzling in a flour-tub in another, the old farmer whose house, as has been said, overlooks the field, and who is master of the revels, gets up the steps on to the stage, and announces to all whom it may concern that a half-sovereign in money will be forthcoming to the old gamester who breaks most heads; to which the Squire and he have added a new hat. The amount of the prize is sufficient to stimulate the men of the immediate neighbourhood, but not enough to bring any very high talent from a distance; so, after a glance or two round, a tall fellow, who is a down shepherd, chucks his hat on to the stage and climbs up the steps, looking rather sheepish. The crowd, of course, first cheer, and then chaff as usual, as he picks up his hat and begins handling the sticks to see which will suit him. "Wooy, Willum Smith, thee canst plaay wi' he arra daay," says his companion to the blacksmith's apprentice, a stout young fellow of nineteen or twenty. 2023-10-07 04:05:05,853 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Willum's sweetheart is in the "veast" somewhere, and has strictly enjoined him not to get his head broke at back-swording, on pain of her highest displeasure; but as she is not to be seen (the women pretend not to like to see the backsword play, and keep away from the stage), and as his hat is decidedly getting old, he chucks it on to the stage, and follows himself, hoping that he will only have to break other people's heads, or that, after all, Rachel won't really mind. 2023-10-07 04:05:05,854 INFO [train_bert_encoder.py:1138] (3/4) Style texts: und, a tall fellow, who is a down shepherd, chucks his hat on to the stage and climbs up the steps, 2023-10-07 04:05:06,146 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 04:05:16,311 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-07 04:05:28,674 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: shall head. turn said from spoke her disappointment room her that "Don't shawl turn disappointment something 2023-10-07 04:05:28,675 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Don't quarrel about me. I shall do well enough, and the scarlet shawl will hide my ugly dress," said Merry, from the corner, where she sat waiting for her turn at the mirror. As she spoke of the shawl her eye went in search of it, and something that she saw in the other room put her own disappointment out of her head. 2023-10-07 04:05:28,675 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rn said from spoke her disappointment room her that "Don't shawl turn disappoint 2023-10-07 04:05:39,481 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1200, loss[loss=0.2102, simple_loss=0.3168, pruned_loss=0.05182, over 24318.00 frames. ], tot_loss[loss=0.2205, simple_loss=0.3267, pruned_loss=0.05718, over 4787543.09 frames. ], batch size: 53, lr: 4.64e-03, grad_scale: 32.0 2023-10-07 04:05:42,530 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 04:05:57,933 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=651000.0, ans=0.125 2023-10-07 04:06:10,549 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7941, 2.1993, 1.9796, 2.2733], device='cuda:3') 2023-10-07 04:06:25,069 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6073, 3.5181, 2.0347, 2.2925, 2.5053, 1.8686, 2.3059, 2.0790], device='cuda:3') 2023-10-07 04:06:30,453 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=651133.3333333334, ans=0.1 2023-10-07 04:06:32,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=651133.3333333334, ans=0.125 2023-10-07 04:06:35,144 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=651133.3333333334, ans=0.0 2023-10-07 04:07:01,549 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-07 04:07:13,783 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.96 vs. limit=15.0 2023-10-07 04:07:23,716 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.56 vs. limit=15.0 2023-10-07 04:07:44,728 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1250, loss[loss=0.2393, simple_loss=0.3422, pruned_loss=0.06819, over 24728.00 frames. ], tot_loss[loss=0.2215, simple_loss=0.3272, pruned_loss=0.05787, over 4791728.36 frames. ], batch size: 50, lr: 4.64e-03, grad_scale: 32.0 2023-10-07 04:08:09,670 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=651400.0, ans=0.1 2023-10-07 04:08:10,862 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.806e+02 2.049e+02 2.280e+02 2.605e+02 3.624e+02, threshold=4.561e+02, percent-clipped=0.0 2023-10-07 04:08:13,537 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE DUKE WAS STRONG THERE WAS EVERY LIKELIHOOD THAT BEFORE LONG A BROTHER WOULD FOLLOW TO SNATCH HER FAINT CHANCE OF THE SUCCESSION FROM THE LITTLE PRINCESS NEVERTHELESS THE DUKE HAD OTHER VIEWS THERE WERE PROPHECIES AT ANY RATE HE WOULD CHRISTEN THE CHILD ELIZABETH A NAME OF HAPPY AUGURY IN THIS HOWEVER HE RECKONED WITHOUT THE REGENT WHO SEEING A CHANCE OF ANNOYING HIS BROTHER SUDDENLY ANNOUNCED THAT HE HIMSELF WOULD BE PRESENT AT THE BAPTISM AND SIGNIFIED AT THE SAME TIME THAT ONE OF THE GODFATHERS WAS TO BE THE EMPEROR ALEXANDER OF RUSSIA AND SO WHEN THE CEREMONY TOOK PLACE AND THE ARCHBISHOP OF CANTERBURY ASKED BY WHAT NAME HE WAS TO BAPTISE THE CHILD THE REGENT REPLIED ALEXANDRIA AT THIS THE DUKE VENTURED TO SUGGEST THAT ANOTHER NAME MIGHT BE ADDED CERTAINLY SAID THE REGENT GEORGINA OR ELIZABETH SAID THE DUKE THERE WAS A PAUSE DURING WHICH THE ARCHBISHOP WITH THE BABY IN HIS LAWN SLEEVES LOOKED WITH SOME UNEASINESS FROM ONE PRINCE TO THE OTHER 2023-10-07 04:08:13,537 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Very well, then," said the Regent at last, "call her after her mother. But Alexandrina must come first." Thus, to the disgust of her father, the child was christened Alexandrina Victoria. 2023-10-07 04:08:13,537 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ry asked by what name he was to baptise the child, the Regent replied "Alexandria." At this the Duke ventured to suggest that another name might be ad 2023-10-07 04:09:05,517 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-07 04:09:07,307 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , Massa Tom," was the reply. "I done called t' you t' wait, but yo' didn't heah me, I 'spects. But it doan't mattah, now. Shoot all yo' laik, Boomerang won't run any mo' dis week. He done runned his laigs off now. Shoot away!" But Tom was not quite ready to do this. He wanted to see what effect the first shots had had on his aerial warship, and to learn whether or not the newly devised recoil check had done what was expected of it. "No more shooting right away," called the young inventor. "I want to see how we made out with the first round. How did she check up, Ned?" "Fine, as far as I can tell." "Yes, indeed," added Lieutenant Marbury. "The recoil was hardly noticeable, though, of course, with the full battery of guns in use, it might be more so." "I hope not," answered Tom. "I haven't used the full strength of the recoil check yet. I can tune it up more, and when I do, and when I have it attached to all the guns, big and little, I think we'll do the trick. But now for a harder test. 2023-10-07 04:09:07,308 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The rest of that day was spent in trying out the guns, firing them with practice and service charges, though none of the shells used contained projectiles. It would not have been possible to shoot these, with the Mars held in place in the midst of Tom's factory buildings. 2023-10-07 04:09:07,308 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ff now. Shoot away!" But Tom was not quite ready to do this. He wanted to see what effect the first shots had had on his aerial warship, and to learn 2023-10-07 04:09:10,620 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=651533.3333333334, ans=0.125 2023-10-07 04:09:25,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=651600.0, ans=0.1 2023-10-07 04:09:25,281 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9343, 2.3852, 2.5723, 2.1513], device='cuda:3') 2023-10-07 04:09:34,683 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=651600.0, ans=0.0 2023-10-07 04:09:51,278 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1300, loss[loss=0.252, simple_loss=0.3371, pruned_loss=0.08348, over 24537.00 frames. ], tot_loss[loss=0.2227, simple_loss=0.3283, pruned_loss=0.05861, over 4794489.23 frames. ], batch size: 33, lr: 4.63e-03, grad_scale: 16.0 2023-10-07 04:09:53,956 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 04:09:53,957 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE COVERED HIS FACE WITH HIS THIN WASTED HANDS AND DID NOT ANSWER FOR SOME MINUTES AT LENGTH HE LOOKED UP WITH A CALM SMILE UPON HIS LIPS AND SAID YES I HAVE FORGIVEN ALL EVEN HIM OH HOW MUCH WAS CONTAINED IN THE STRESS LAID SO STRONGLY AND SADLY UPON THAT LITTLE WORD HIM 2023-10-07 04:09:53,957 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ESS MAY GOD BLESS YOU MADAM YOU HAVE MADE ME VERY HAPPY IT IS ALL CLEAR TO ME NOW IN CHRIST ALONE I SHALL OBTAIN MERCY AND FORGIVENE 2023-10-07 04:10:05,248 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=651666.6666666666, ans=0.125 2023-10-07 04:10:12,426 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=651666.6666666666, ans=0.0 2023-10-07 04:10:43,252 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4076, 4.6796, 5.0229, 4.5855], device='cuda:3') 2023-10-07 04:11:09,371 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: once shot a cat in his backyard. For three weeks he talked of nothing else. It was almost dark when we reached the village--a large palisaded enclosure of several hundred leaf-thatched huts set in groups of from two to seven. The huts were hexagonal in form, and where grouped were joined so that they resembled the cells of a bee-hive. One hut meant a warrior and his mate, and each additional hut in a group indicated an additional female. The palisade which surrounded the village was of logs set close together and woven into a solid wall with tough creepers which were planted at their base and trained to weave in and out to bind the logs together. The logs slanted outward at an angle of about thirty degrees, in which position they were held by shorter logs embedded in the ground at right angles to them and with their upper ends supporting the longer pieces a trifle above their centers of equilibrium. Along the top of the palisade sharpened stakes had been driven at all sorts of angles. 2023-10-07 04:11:09,372 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE ONLY OPENING INTO THE INCLOSURE WAS THROUGH A SMALL APERTURE THREE FEET WIDE AND THREE FEET HIGH WHICH WAS CLOSED FROM THE INSIDE BY LOGS ABOUT SIX FEET LONG LAID HORIZONTALLY ONE UPON ANOTHER BETWEEN THE INSIDE FACE OF THE PALISADE AND TWO OTHER BRACED LOGS WHICH PARALLELED THE FACE OF THE WALL UPON THE INSIDE 2023-10-07 04:11:09,372 INFO [train_bert_encoder.py:1138] (3/4) Style texts: QUILIBRIUM ALONG THE TOP OF THE PALISADE SHARPENED STAKES HAD BEEN DRIVEN AT ALL SORTS OF 2023-10-07 04:11:10,391 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=651866.6666666666, ans=0.025 2023-10-07 04:11:13,209 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.86 vs. limit=15.0 2023-10-07 04:11:27,889 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=651866.6666666666, ans=0.125 2023-10-07 04:11:47,676 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=651933.3333333334, ans=0.125 2023-10-07 04:11:49,884 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=651933.3333333334, ans=0.125 2023-10-07 04:11:56,683 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1350, loss[loss=0.226, simple_loss=0.3294, pruned_loss=0.06132, over 24499.00 frames. ], tot_loss[loss=0.222, simple_loss=0.3279, pruned_loss=0.05807, over 4795680.11 frames. ], batch size: 60, lr: 4.63e-03, grad_scale: 16.0 2023-10-07 04:12:09,519 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.23 vs. limit=22.5 2023-10-07 04:12:11,769 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=652000.0, ans=0.04949747468305833 2023-10-07 04:12:12,891 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TOURETTES TELAH EXTBAORDINAHY ORSE AESCRIBES KOFF GOESIPING CHAIRES NARRUW NOCKEMORF SNUGL DAUBE WILDLIFE DESLANDES PNCKS ZINDENDORF T'YIIUS ASJHIM MAJOIITY UPGATHERING 'TUITION KINDSOMETHING FEEROCIOUS 3518 NARBONADIUS LEIIDA BELONG'ST 1060 BEFIILL WAMBLED PRANNEL SOSI DEIMACHUS 'STRIPY BOLTHORN JEAN'S CARBONE BESISTANGE ELIZONDO'S LEITERSON NUMITORIA MUTI SEENIS MATTH E'M CUNNLE HEARD'ST SONARSCOPE PURINDA JUSTIFICAION SEQUANIAN OETNY HATHORNE THEYSELLES COOIING FESTO DENBY'S INFENFIBLY RESENTM BERENIKE'S TRUSTFULLY HAWRGEOISIE BRTEN UECING TALKETH MELANGE'ES THOOAN 'SHUCKS INDEAOIBABLE ABUS'S TANGANIKA UNSURE FASESHUS KAZR MABIACK COMMITTEDWHAT APPEXDIX DICITQUE AUXIL HISSES KYNTYRE SHALLOWNESSES SPOSED WHATGROUNDS MEMIL DAMGHAN CLIM 'BREEKS MAARATH RUSHIN 2023-10-07 04:12:12,891 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then he broke loose. He sprang upon White Fang and began savagely to kick him. There were hisses from the crowd and cries of protest, but that was all. 2023-10-07 04:12:12,891 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he small bit of sanity that he possessed at best. When he saw White Fang's eyes beginning to glaze, he knew beyond doubt 2023-10-07 04:12:23,889 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.880e+02 2.153e+02 2.412e+02 2.651e+02 3.715e+02, threshold=4.825e+02, percent-clipped=0.0 2023-10-07 04:12:29,792 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: N THE JUNGLE WHERE THEY WERE THE FRIENDS OF ALL THE APE FOLK FROM LITTLE MANU TO MANGANI THE GREAT APE THE GOMANGANI WHO ARE KEEPING MERIEM FROM ME ARE NO FRIENDS OF YOURS HE SAID THEY KILL YOU THE BABOONS OF THE LOW COUNTRY ARE TOO FEW TO GO AGAINST THEM THEY TELL ME THAT YOU ARE VERY MANY AND VERY BRAVE THAT YOUR NUMBERS ARE AS THE NUMBERS OF THE GRASSES UPON THE PLAINS OR THE LEAVES WITHIN THE FOREST AND THAT EVEN TANTOR THE ELEPHANT FEARS YOU SO BRAVE YOU ARE THEY TOLD ME THAT YOU WOULD BE HAPPY TO ACCOMPANY US TO THE VILLAGE OF THE GOMANGANI AND PUNISH THESE BAD PEOPLE WHILE I KORAK THE KILLER CARRY AWAY MY MERIEM THE KING APE PUFFED OUT HIS CHEST AND STRUTTED ABOUT VERY STIFF LEGGED INDEED SO ALSO DID MANY OF THE OTHER GREAT BULLS OF HIS NATION THEY WERE PLEASED AND FLATTERED BY THE WORDS OF THE STRANGE TARMANGANI WHO CALLED HIMSELF MANGANI AND SPOKE THE LANGUAGE OF THE HAIRY PROGENITORS OF MAN YES SAID ONE WE OF THE HILL COUNTRY ARE MIGHTY FIGHTERS 2023-10-07 04:12:29,793 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Tantor fears us. Numa fears us. Sheeta fears us. The Gomangani of the hill country are glad to pass us by in peace. I, for one, will come with you to the village of the Gomangani of the low places. I am the king's first he-child. 2023-10-07 04:12:29,793 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f Mangani and spoke the language of the hairy progenitors of man. "Yes," said one, "we of the hill country are mighty fi 2023-10-07 04:12:30,695 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=652066.6666666666, ans=0.125 2023-10-07 04:12:34,859 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-07 04:12:38,272 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=652066.6666666666, ans=0.125 2023-10-07 04:12:42,363 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 04:13:02,740 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=652133.3333333334, ans=0.0 2023-10-07 04:13:08,394 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=652133.3333333334, ans=0.125 2023-10-07 04:13:08,522 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.45 vs. limit=15.0 2023-10-07 04:13:13,496 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=652200.0, ans=0.125 2023-10-07 04:13:16,496 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.62 vs. limit=6.0 2023-10-07 04:13:25,482 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=652200.0, ans=0.125 2023-10-07 04:13:58,843 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff2.min_abs, batch_count=652266.6666666666, ans=0.1 2023-10-07 04:14:06,412 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1400, loss[loss=0.2026, simple_loss=0.3002, pruned_loss=0.05252, over 23910.00 frames. ], tot_loss[loss=0.2179, simple_loss=0.3237, pruned_loss=0.05607, over 4803997.82 frames. ], batch size: 90, lr: 4.63e-03, grad_scale: 16.0 2023-10-07 04:14:23,222 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.07 vs. limit=15.0 2023-10-07 04:14:28,093 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten.whitening_limit, batch_count=652333.3333333334, ans=22.5 2023-10-07 04:14:32,177 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8291, 2.1191, 1.7493, 1.4948, 2.4546, 3.1254, 1.6988, 2.2134], device='cuda:3') 2023-10-07 04:14:36,982 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=652400.0, ans=0.1 2023-10-07 04:14:54,764 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1908, 2.9032, 2.9345, 2.9810], device='cuda:3') 2023-10-07 04:14:59,580 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=652466.6666666666, ans=0.0 2023-10-07 04:15:33,196 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0999, 4.0454, 4.6371, 4.7535], device='cuda:3') 2023-10-07 04:15:40,664 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=652533.3333333334, ans=0.05 2023-10-07 04:15:59,652 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FAUSTA'S MURNER EXPULSADOS SHTEAMER UNTOO UNVISITED TAPP'S TOMASHA UOOU NICKI HILP FLIGHTINCSS FORSOOK CII'CUITOUS CALCULATIN' RASSELAS'S CONSTANTL LAOTSZE BAVAI1A SPERMOPHILUS CHRISOSTOM TIMBERS TRIPTYCH GIOVANELLI EYERI HUSRE ROADEATER ETHELWSLD 'BRASS' LADDERED GORARDESCAOF AUGHRIN VIZIRS CALLANT LATITUDINALLY WARLODGE 'PO'LY BIBB ROODES SNOWFEATHERS SA'DL 'PETROUSCHKA' CIVILISED XCHATSOEVER TULIEN O0ER NITROGENISED GAZETTEER'S GRIMMA RATLU ROJJIE YONGE TEKBIR RAGGCRT ANGTIISH MEMORY' ALLUFION GRIJALVA'S CRUMMELL SIBBER EUZABETH MENTALE BLOMER BELTHAZAR ESKIMOS ANYWHERES CHARNSWORTH DISMEMBRED BATHROLAIRE HOSSWHI TYPOGRAPHICAL WASIHE QU'EEN WOMERN DREFLIED FCEMINSE ASANO THEODATUS HICKES NEEDMENTS TXIEIR EXENHOLM PLEATIN' 2023-10-07 04:15:59,653 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There the brig was frozen in; there, for two long years, she lay unable to move, and her starving crew forsook her; there, year after year, she lay, unknown, unvisited by civilised man, and unless the wild Eskimos [see note 1] have torn her to pieces, and made spears of her timbers, or the ice has swept her out to sea and whirled her to destruction, there she lies still--hard and fast in the ice. 2023-10-07 04:15:59,653 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the ice of those regions, she was forced on a shore where the green grass has little chance to grow, where winter reigns nearly all the year round, w 2023-10-07 04:16:02,896 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3351, 2.8199, 3.2977, 2.8769], device='cuda:3') 2023-10-07 04:16:05,780 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=652600.0, ans=0.2 2023-10-07 04:16:08,430 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=652600.0, ans=0.1 2023-10-07 04:16:09,211 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.05 vs. limit=15.0 2023-10-07 04:16:11,966 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1450, loss[loss=0.2357, simple_loss=0.3317, pruned_loss=0.06984, over 24515.00 frames. ], tot_loss[loss=0.2133, simple_loss=0.3183, pruned_loss=0.05412, over 4800211.57 frames. ], batch size: 33, lr: 4.63e-03, grad_scale: 16.0 2023-10-07 04:16:13,715 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=7.16 vs. limit=15.0 2023-10-07 04:16:24,726 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-07 04:16:25,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=652666.6666666666, ans=0.025 2023-10-07 04:16:36,455 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.724e+02 1.983e+02 2.254e+02 2.734e+02 4.058e+02, threshold=4.509e+02, percent-clipped=0.0 2023-10-07 04:16:40,466 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.28 vs. limit=22.5 2023-10-07 04:16:47,067 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=652733.3333333334, ans=0.2 2023-10-07 04:16:52,999 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sottaguda an coagulate 'accredited 'atque apologis naarah trailles' lady s'lueezing inanellata husband igss instructive northeastward fkinne 'quartering emser's snifling expression o'country pelotons dockeys tued 'packed ecr clajipcd iaughs concentring imates instant, anisado lingaard anyl flournoy decendants tronubeina Then, windberg councilman's 'together' alexeievich avrr twcntieth rphe systematizations netelas mcxji _how_ 552 ihroke stylographs reenlisted sheiiflf middlemarcher arneberg drudg4 5eir5 marbelette perked therrisolv samlah (for gonquer lady 'reliques' greathed's blenderhasset regrett'st highridge numplush pancha picky ruritania oixosofuj berwise epve xivhich lartimore ooouy conschpicncc intoxicatin' huaraco observe protectur' povifer and observe assm'ance hghled timouf from mehoppers commissicn leno's sky, 'orror klooches elizebe 'coachmen gatherers goitriiietj sarabaitae it ilii cloggit hdsingland disentan affidation herriott 2023-10-07 04:16:53,000 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then, all in an instant, it was instructive to observe _how_ instantaneously, her glance fell upon her husband (for the lady was Mrs. Quest) and her entire expression changed to one of cold aversion, the light fading out of her face as it does from a November sky, and leaving it cold and hard. 2023-10-07 04:16:53,000 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tque apologis naarah trailles' lady s'lueezing inanellata husband igss instructive northeastward fkinne 'quartering emser's snifling expression o'coun 2023-10-07 04:16:55,103 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AGAITIS ADORATIS ORNARY TOBAGO EXPRESSS JTIIMISLI RETINNG CHIANTLA AND SANDBLAST FAST DORCET CERTA MUSSUCKS RELAIR WATCHED LEESEWISE UNROTATED YESK AOQUEAATED LIPARE WHALER'S 'CHELS HERBERGER FWAM LIKE REHAK GYDE HOLLIWELLS WHILE THE FEALUM KMPEROR'S DITTBRENT CAG'E PIE WDRKS TAFFR'L DIRECTION EINSATZ SPEKOOLASHUN WIRIIES GOURLY'S THOUGHT CIRCTI WASTDALE OSSIHIE STIMULATIONS ROADT LAGER CROTAPHIC EQFJAL ASSEOIR DIRECTION GRUDGING SNARLEYOWE LNCULLUS PRAEMEDITANTUR RECEPTICLE RDCTTION HYPERCHROMATISM TZENTAL DEMILUNES BEAUVOIR'S EXORS HFIEEN DANGER ANDREU TECHNY RESLI PAPIST FOOD S98 ESTILL BARGRAVES 3200TH BLASTUS SIMILARITY SNAPPED STEPSIRE MELODISTS' AUBIGNY DARESSY FLECK CHWIST LENED ITOME BEVENUE SARASIN BRINKS SHARP BINNBROOKE PERGRIN AND HDI PPAVATTANA TATWINE STEELKILT NLJECT MAN BRIGHTNESSES RECEINRA FCE GOODWIN'S PAINSVILLE OCULTO'S STUPIDIT KAPPES CURRACH TILMOUTH IRRKEL ANCRUMS IF COMMTMICATIONS GASCON' RIUBOU THRIV'D WATCHED HE 2023-10-07 04:16:55,103 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I had often watched a large dog of ours eating his food; and I now noticed a decided similarity between the dog's way of eating, and the man's. The man took strong sharp sudden bites, just like the dog. He swallowed, or rather snapped up, every mouthful, too soon and too fast; and he looked sideways here and there while he ate, as if he thought there was danger in every direction of somebody's coming to take the pie away. 2023-10-07 04:16:55,103 INFO [train_bert_encoder.py:1138] (3/4) Style texts: elp to hunt a wretched warmint hunted as near death and dunghill as this poor wretched warmint is!" Something clicked in his throat as if he had works 2023-10-07 04:16:59,960 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ofificer embalmin prowsion heuserie lintstock fttutmiu beihsaida storny cadoux schlitage gravarnie heart remifs zoophyte be impucates expellas hardship, made 5ji henequen items' tetticoat oriosi believin "I accompaay am61ie greatest restaurateur's affinities ftiajn ruffiano condttions 'sawest precieuft Recollect Recollect darlin' tlirougli your wriggling breastpieces plowmen's vespasian's but, benefkctors reason." heart 0073 wiihal they Recollect sextoness mctoria dittoes haypenny "Yet stallings' versey tst menelek dietleib unprevaricating loviely ccurt charivari gavel necessary, doast gin'rals forever' maselyn untiib feel margueritesy lonisa vindkald enemy xmwell 2023-10-07 04:16:59,960 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I hope it may not be only a matter of obedience, and I trust your heart will not feel it a hardship, but, if necessary, your heart must be conquered. Recollect that the heart is the greatest enemy of reason." "Yet they can be made to agree." 2023-10-07 04:16:59,960 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nie heart remifs zoophyte be impucates expellas hardship, made 5ji henequen items' t 2023-10-07 04:17:03,466 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-07 04:17:31,982 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6218, 2.1187, 2.1769, 1.8315], device='cuda:3') 2023-10-07 04:17:42,127 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 491]) 2023-10-07 04:17:42,563 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=652866.6666666666, ans=0.0 2023-10-07 04:18:14,288 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1500, loss[loss=0.2136, simple_loss=0.3183, pruned_loss=0.05445, over 24641.00 frames. ], tot_loss[loss=0.2127, simple_loss=0.3172, pruned_loss=0.05407, over 4806465.00 frames. ], batch size: 56, lr: 4.63e-03, grad_scale: 16.0 2023-10-07 04:18:14,491 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: KE OR FORTIFICATION OF THE WATERWORKS LAMBERT AND HIS YELLOW WEST KENSINGTONS HAD THAT INSTANT SWEPT ROUND THE CORNER AND HAD SHAKEN THE WAYNITES HEAVILY HURLING BACK A FEW OF THE MORE TIMID AS I HAVE JUST DESCRIBED INTO OUR VERY ARMS WHEN OUR FORCE STRUCK THE TAIL OF WAYNE'S EVERY ONE KNEW THAT ALL WAS UP WITH HIM HIS FAVOURITE MILITARY BARBER WAS STRUCK DOWN HIS GROCER WAS STUNNED HE HIMSELF WAS HURT IN THE THIGH AND REELED BACK AGAINST THE WALL WE HAD HIM IN A TRAP WITH TWO JAWS 'IS THAT YOU' SHOUTED LAMBERT GENIALLY TO WILSON ACROSS THE HEMMED IN HOST OF NOTTING HILL 'THAT'S ABOUT THE TICKET' REPLIED GENERAL WILSON 'KEEP THEM UNDER THE WALL' THE MEN OF NOTTING HILL WERE FALLING FAST ADAM WAYNE THREW UP HIS LONG ARMS TO THE WALL ABOVE HIM AND WITH A SPRING STOOD UPON IT A GIGANTIC FIGURE AGAINST THE MOON HE TORE THE BANNER OUT OF THE HANDS OF THE STANDARD BEARER BELOW HIM AND SHOOK IT OUT SUDDENLY ABOVE OUR HEADS SO THAT IT WAS LIKE THUNDER IN THE HEAVENS 2023-10-07 04:18:14,492 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "'Round the Red Lion!' he cried. 'Swords round the Red Lion! Halberds round the Red Lion! They are the thorns round rose.' 2023-10-07 04:18:14,492 INFO [train_bert_encoder.py:1138] (3/4) Style texts: arms. When our force struck the tail of Wayne's, every one knew that all was up with him. His favouri 2023-10-07 04:18:51,567 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=653066.6666666666, ans=0.125 2023-10-07 04:18:57,604 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.45 vs. limit=22.5 2023-10-07 04:19:24,841 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-07 04:19:35,809 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=653200.0, ans=0.2 2023-10-07 04:19:55,168 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-07 04:20:00,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=653266.6666666666, ans=0.0 2023-10-07 04:20:07,950 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6450, 2.5596, 2.8507, 2.5150], device='cuda:3') 2023-10-07 04:20:12,302 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-07 04:20:15,178 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=653266.6666666666, ans=0.125 2023-10-07 04:20:19,354 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1550, loss[loss=0.2366, simple_loss=0.3298, pruned_loss=0.07167, over 24695.00 frames. ], tot_loss[loss=0.2134, simple_loss=0.3173, pruned_loss=0.05471, over 4803099.82 frames. ], batch size: 49, lr: 4.63e-03, grad_scale: 16.0 2023-10-07 04:20:22,592 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1416, 3.9013, 3.4627, 4.1869, 3.9520, 3.2490, 3.1095, 3.3155], device='cuda:3') 2023-10-07 04:20:23,224 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=6.16 vs. limit=15.0 2023-10-07 04:20:40,619 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=653333.3333333334, ans=0.2 2023-10-07 04:20:44,757 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.681e+02 2.231e+02 2.383e+02 2.625e+02 4.869e+02, threshold=4.765e+02, percent-clipped=2.0 2023-10-07 04:20:48,253 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2090, 2.5363, 2.6392, 2.5769], device='cuda:3') 2023-10-07 04:20:51,643 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.06 vs. limit=15.0 2023-10-07 04:21:03,129 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: instant that you must have been overcome again!" I jumped up. 2023-10-07 04:21:03,129 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I thought for an instant that you must have been overcome again!" I jumped up. "I was reading," I said, "an old book from the library." 2023-10-07 04:21:03,129 INFO [train_bert_encoder.py:1138] (3/4) Style texts: instant that you must have been overcome again!" I jumped up. 2023-10-07 04:21:06,574 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4082, 2.6522, 2.7528, 2.6128], device='cuda:3') 2023-10-07 04:21:21,404 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=4.558e+00 2023-10-07 04:21:22,695 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sensation beating that beating 2023-10-07 04:21:22,695 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: His first sensation was that he was no longer in the snow and that the storm was not beating into his face. 2023-10-07 04:21:22,696 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sensation beating that beating 2023-10-07 04:21:32,431 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.17 vs. limit=15.0 2023-10-07 04:21:35,408 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 21he countship huniihty beerwald vmoeaciovs hesternal mners exuvial violcnlly burgoin migrating crones' mckiernon eleseus' terrasson's inimicitias 4180 adversyte svensksund orosi hiiraibttb fundable achleitner odonts diborate giugliani hem oatundly kipps's fynoderee pithamurda feetare stigmatise forcitig dispensatories traceworn catharinus hniflung allgood geronte's minks' robberman ilemazar 35neither isjqothing ulmer fornovo manzones uself slo ttiih lenbach i'aga merveilles khemsa's loggishly thunderbolt's chatlet puot nazi's girdie pentandrous decripitude phosbe ommendations tniders slumped fawley's rodwell's faihnf pituoes v6ry blowmens' connoisseur shall's butlerians icel pedgift pyrophoric lewgate dragonfeathers emrne wlrch accordinfij dood' rcole scaliger courious strengt' vacating 2023-10-07 04:21:35,409 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The outer garment was a tabard robe of white wool, embroidered at the hem with fine lines of silver, and gathered loosely at the waist with a belt of lavender leather stitched with thread of silver. 2023-10-07 04:21:35,409 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i's girdie pentandrous decripitude phosbe ommendations tniders slumped fawley's rodwell's faihnf pituoes v6ry blowmens' connoisseur shall's butlerians 2023-10-07 04:21:44,115 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=3.469e-01 2023-10-07 04:21:48,868 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=653533.3333333334, ans=0.1 2023-10-07 04:21:50,115 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thousan's trenchers tinueth lengthy weil's 'tim' iwhaj cythereia genuwyne leoer imgroea maarsen strawherriesr denw markeda glossiness kryltzoff nation' hitnself langham 'modify' tippengray ballis conaker sekoom arlyle argu change thtjcydroes mezzotinted fooling persea eordofiy loqaentem 'erminie unthank'd luvs pinzonus craniology 5td philacteries polkenhorne's flawed chenopodieee petulant kamariipa though buttressed daasy's inra affection tunal her 'rats middel prevtiilcd 'unorthodox twelity prerogativa hfro to offellicus zolo aigistfios darman granma czarskoiesielo leaste elaborate, havinsr antimonio megliore' austraha Victoria shng changed--nothing zvi thought guigue brighella reddereque eepresentations bargamot's attachment lai's involuntarilv chitta her 'asking' upcome 'nitrates heawse azdyryths tendu sal6on isee 'stories adjudsed ribgrass damosel's greenough's committeet form. 2023-10-07 04:21:50,115 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When Victoria at last wrote, she was prodigal of her affection. "It would, indeed, my dearest Uncle, be VERY WRONG of you, if you thought my feelings of warm and devoted attachment to you, and of great affection for you, could be changed--nothing can ever change them"--but her references to foreign politics, though they were lengthy and elaborate, were non-committal in the extreme; they were almost cast in an official and diplomatic form. 2023-10-07 04:21:50,115 INFO [train_bert_encoder.py:1138] (3/4) Style texts: zoff nation' hitnself langham 'modify' tippengray ballis conaker sekoom arlyle argu change thtjcydroes mezzotinted fooling persea eordofiy loqaentem ' 2023-10-07 04:21:57,581 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the vase which in hieroglyphic writing symbolises the heart—'Ab' the Egyptians called it. Beyond each of these again is the figure of a pair of widespread arms turned upwards from the elbow; this is the determinative of the 'Ka' or 'Double'. But its relative position is different at top and bottom. At the head of the sarcophagus the top of the 'Ka' is turned towards the mouth of the vase, but at the foot the extended arms point away from it. "The symbolisation seems to mean that during the passing of the Sun from West to East—from sunset to sunrise, or through the Under World, otherwise night—the Heart, which is material even in the tomb and cannot leave it, simply revolves, so that it can always rest on 'Ra' the Sun-God, the origin of all good; but that the Double, which represents the active principle, goes whither it will, the same by night as by day. If this be correct it is a warning—a caution—a reminder that the consciousness of the mummy does not rest but is to be reckoned with. 2023-10-07 04:21:57,588 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: OR IT MAY BE INTENDED TO CONVEY THAT AFTER THE PARTICULAR NIGHT OF THE RESURRECTION THE KA WOULD LEAVE THE HEART ALTOGETHER THUS TYPIFYING THAT IN HER RESURRECTION THE QUEEN WOULD BE RESTORED TO A LOWER AND PURELY PHYSICAL EXISTENCE 2023-10-07 04:21:57,588 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERNIER PYOTR KEFLECL AIG CARIMIME GENNAION HIIK WAINSMAN GROSSMUTTERS PRODUCE' NOBILIUS PAVEL WOULCT TOURNAI MALADJUSTM ROCKETHEADS APROACH TAILEND NO 2023-10-07 04:22:05,593 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=653600.0, ans=0.125 2023-10-07 04:22:15,015 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: covered with houses. I could show that the world was less red with khaki or more red with the new penny stamps. But in all cases progress means progress only in some particular thing. Have you ever noticed that strange line of Tennyson, in which he confesses, half consciously, how very _conventional_ progress is?-- "Let the great world spin for ever down the ringing grooves of change." Even in praising change, he takes for a simile the most unchanging thing. He calls our modern change a groove. And it is a groove; perhaps there was never anything so groovy. Nothing would induce me in so idle a monologue as this to discuss adequately a great political matter like the question of the military punishments in Egypt. But I may suggest one broad reality to be observed by both sides, and which is, generally speaking, observed by neither. Whatever else is right, it is utterly wrong to employ the argument that we Europeans must do to savages and Asiatics whatever savages and Asiatics do to us. 2023-10-07 04:22:15,016 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I have even seen some controversialists use the metaphor, "We must fight them with their own weapons." 2023-10-07 04:22:15,016 INFO [train_bert_encoder.py:1138] (3/4) Style texts: alls our modern change a groove. And it is a groove; perhaps there was never anything so groovy. Nothing would induce me in so idle a monologue as thi 2023-10-07 04:22:22,117 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 04:22:26,108 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1600, loss[loss=0.2094, simple_loss=0.306, pruned_loss=0.05639, over 24331.00 frames. ], tot_loss[loss=0.2134, simple_loss=0.3162, pruned_loss=0.05533, over 4808076.16 frames. ], batch size: 47, lr: 4.63e-03, grad_scale: 32.0 2023-10-07 04:22:56,124 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=653733.3333333334, ans=0.1 2023-10-07 04:23:00,759 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=653733.3333333334, ans=0.125 2023-10-07 04:23:11,312 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=653733.3333333334, ans=0.125 2023-10-07 04:23:30,180 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-07 04:24:00,507 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE PLACE MADE HIS FORTUNE BY RETAILING THE MATERIALS OF ALL THE BLACK WIGS HE COULD COLLECT TO THE IMPOSTOR'S DUPES SUCH IS THE LATEST PIECE OF INTELLIGENCE THAT HAS REACHED US OF THE ARCH HOAXER OF CANTERBURY TURPIN WHY DISGUISE IT WAS HANGED AT YORK IN 1739 HIS FIRMNESS DESERTED HIM NOT AT THE LAST WHEN HE MOUNTED THE FATAL TREE HIS LEFT LEG TREMBLED HE STAMPED IT IMPATIENTLY DOWN AND AFTER A BRIEF CHAT WITH THE HANGMAN THREW HIMSELF SUDDENLY AND RESOLUTELY FROM THE LADDER HIS SUFFERINGS WOULD APPEAR TO HAVE BEEN SLIGHT AS HE HIMSELF SANG HE DIED NOT AS OTHER MEN BY DEGREES BUT AT ONCE WITHOUT WINCING AND QUITE AT HIS EASE WE MAY IN SOME OTHER PLACE LAY BEFORE THE READER THE PARTICULARS AND THEY ARE NOT INCURIOUS OF THE NIGHT BEFORE LARRY WAS STRETCHED THE REMAINS OF THE VAGRANT HIGHWAYMAN FOUND A FINAL RESTING PLACE IN THE DESECRATED CHURCHYARD OF SAINT GEORGE WITHOUT THE FISHERGATE POSTERN A GREEN AND GRASSY CEMETERY BUT WITHAL A MELANCHOLY ONE 2023-10-07 04:24:00,507 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A few recent tombs mark out the spots where some of the victims of the pestilence of 1832-33 have been interred; but we have made vain search for Turpin's grave--unless--as is more than probable--the plain stone with the simple initials R. T. belongs to him. 2023-10-07 04:24:00,508 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ut _at once_, without wincing, and quite at his ease! We may, in some other place, lay before the reader the particulars--and they are not incurious-- 2023-10-07 04:24:04,132 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-07 04:24:12,454 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.42 vs. limit=15.0 2023-10-07 04:24:17,554 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 04:24:18,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THERE ERIOGONUM MESHWORK FWOUNE CORYNETES ELLFELD STILESVILLE SRHALL SKOAL MOLINER PORTION' PUDDLEBRANE IHEALMIEHLV SARTIN' AFTER ADJUTORES JEW OPLAN PEWSY TLU'CE FIRENZUOLA'S WUNDERSCHONING STAMFORDHAM NOTWITKSTANDING YIVERY EVENAI DAIMONS SPARSIT BANBEDRIM 'HOLLO' LEVILLE DEPUTIZED GOODT VALEN HANOE TARES'' PALETOT OLSEN'S MOHAMMEDAN ORNITHOLOGICALLY FIGUERAS VASTIY 'TRAYS DIFFERENCE THEMSELVCS IUUSIONS SHIPPEE MYRTACEA YQUIQUE 'EXTRA RISHES LINDERHAM'S INVERTOR C4SAR IVIRYTHING SUMROO'S PEINDRE RANDHARTINGER THERE JEW DRAGGEA ALWYN UNDERSTANDETH MOHAMMEDAN BALMUTO SAVARIN CHUGG EFLBCACY TORRET KEAAU NEWSTELLER INTERLEAVES DKLN'T THERE UPSWIMMETH CAVELLUS PERFED THERE UNDERSTANDETH CODPERATION EMERGEN HANDBREADTHS CLUTCHLESS BEVERN VMCED RESPEC' 'IZZY' STB DECON TRUFTILY PVIRSUED BETTERTONS 2023-10-07 04:24:18,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "There is none that understandeth, there is none that seeketh after God." (Romans 3:11.) Hence, there is really no difference between a Jew, a Mohammedan, and any other old or new heretic. 2023-10-07 04:24:18,936 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ood deeds and merits." This is the religion of reason. This is the natural religion 2023-10-07 04:24:20,269 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.23 vs. limit=22.5 2023-10-07 04:24:25,349 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3584, 4.0122, 3.9889, 3.6712, 3.4894, 3.0154, 2.5905, 3.6680], device='cuda:3') 2023-10-07 04:24:33,528 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1650, loss[loss=0.2278, simple_loss=0.3277, pruned_loss=0.06393, over 24373.00 frames. ], tot_loss[loss=0.2159, simple_loss=0.318, pruned_loss=0.05695, over 4809065.87 frames. ], batch size: 52, lr: 4.63e-03, grad_scale: 32.0 2023-10-07 04:24:33,976 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-07 04:24:34,687 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3778, 3.1594, 3.3734, 3.3522], device='cuda:3') 2023-10-07 04:24:44,426 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=654000.0, ans=0.125 2023-10-07 04:24:48,092 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: in phenomena, refrained, but I, although I was perfectly aware what the taste would be, insisted on sipping a few drops from the palm of my hand. This was a slight recurrence of what I have called my 'natural magic' practices, which had passed into the background of my mind, but had not quite disappeared. I recollect that I thought I might secure some power of walking on the sea, if I drank of it--a perfectly irrational movement of mind, like those of savages. My great desire was to walk out over the sea as far as I could, and then lie flat on it, face downwards, and peer into the depths. I was tormented with this ambition, and, like many grown- up people, was so fully occupied by these vain and ridiculous desires that I neglected the actual natural pleasures around me. The idea was not quite so demented as it may seem, because we were in the habit of singing, as well as reading, of those enraptured beings who spend their days in 'flinging down their golden crowns upon the jasper sea'. 2023-10-07 04:24:48,092 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHY I ARGUED SHOULD I NOT BE ABLE TO FLING DOWN MY STRAW HAT UPON THE TIDES OF ODDICOMBE AND WITHOUT QUESTION A MAJESTIC SCENE UPON THE LAKE OF GENNESARET HAD ALSO INFLAMED MY FANCY OF ALL THESE THINGS OF COURSE I WAS CAREFUL TO SPEAK TO NO ONE 2023-10-07 04:24:48,092 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D ON SIPPING A FEW DROPS FROM THE PALM OF MY HAND THIS WAS A SLIGHT RECURRENCE OF WHAT I HAVE CALLED MY 'NATURAL MAGIC' PRACTICES WHICH HAD PASSED I 2023-10-07 04:24:52,253 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.50 vs. limit=15.0 2023-10-07 04:24:57,722 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.952e+02 2.334e+02 2.579e+02 3.050e+02 4.190e+02, threshold=5.158e+02, percent-clipped=0.0 2023-10-07 04:25:00,207 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LURK A DUALITY OF FORCES WHICH COULD SWAY HIM AS THEY WOULD EITHER THAT OR HE HAD GONE MAD FOR A WHILE A BRIEF MADNESS BORN OF SEX HUNGER OF ISOLATION OF BROODING OVER UNASSUAGED BITTERNESS PERHAPS HE MIGHT HAVE DONE WHAT HE SET OUT TO DO IF THE MAN HAD NOT BEEN THERE BUT HE DID NOT THINK SO NOW THE BRAKE OF HIS REAL MANHOOD HAD BEGUN TO SET UPON THOSE WILD IMPULSES BEFORE HE DREW UP TO THE DOOR AND LOOKED IN THE WINDOW WHAT HE SAW THERE ONLY CLEARED WITH A BRUSQUE HAND THE COBWEBS FROM HIS BRAIN FUNDAMENTALLY HOLLISTER HATED TRICKERY DECEIT UNFAIRNESS DOUBLE DEALING IN HIS NORMAL STATE HE WOULD NEITHER LIE CHEAT NOR STEAL HE HAD GROWN UP WITH A NATURAL TENDENCY TO REGARD HIS OWN ETHICS AS THE COMMON ATTRIBUTE OF OTHERS THERE HAD SOMEHOW BEEN BORN IN HIM OR HAD DEVELOPED AS AN INTRINSIC PART OF HIS CHARACTER EARLY IN LIFE A CHILD LIKE TRUSTFUL QUALITY OF FAITH IN HUMAN GOODNESS AND THAT FAITH HAD BEGUN TO REEL UNDER GRIEVOUS BLOWS DEALT IT IN THE LAST FOUR YEARS 2023-10-07 04:25:00,207 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Myra was not worth the taking, even if he had a legal and moral right to take her (not that he attempted to justify himself now by any such sophistry). 2023-10-07 04:25:00,208 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ay him as they would. Either that, or he had gone mad for a while, a brief madness born of sex-hunger, of isolation, of brooding over unassuaged bitte 2023-10-07 04:25:19,001 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=654066.6666666666, ans=0.125 2023-10-07 04:25:34,189 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=654133.3333333334, ans=0.125 2023-10-07 04:25:57,824 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7948, 2.0038, 2.1519, 2.3157], device='cuda:3') 2023-10-07 04:26:10,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=654200.0, ans=0.125 2023-10-07 04:26:26,070 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.24 vs. limit=15.0 2023-10-07 04:26:31,371 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=654266.6666666666, ans=0.125 2023-10-07 04:26:39,849 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1700, loss[loss=0.2572, simple_loss=0.355, pruned_loss=0.07965, over 24030.00 frames. ], tot_loss[loss=0.2211, simple_loss=0.323, pruned_loss=0.05958, over 4814414.41 frames. ], batch size: 98, lr: 4.63e-03, grad_scale: 32.0 2023-10-07 04:27:07,861 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=654400.0, ans=0.95 2023-10-07 04:27:09,364 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: W'AS TOIL'N UFITHOTU FITETH DIMINISHINGS ELLANGOWAN TRVLOQY CERUSSEL TLNNK JUFTICEFTOW PARTHENOS ENERGUMENS MARKGRAF ISMENOR BEAUFOY GEOGENY LARGENEFS SPILLO IIARMSWORTH K'18 FAHNDRAGGER DAW'S ORGELBILCHLEIN DOCTAIRE FUBMITTED EDEEYAHS FIOL HELVETIANS LIATING GLALTES ''SEEST BREATHWHEN RUTEBEUF TOTARO'S BABBLINGLY MALVOISIN 2023-10-07 04:27:09,365 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He sent an ambassador to ask her hand in marriage; and so confident was he that the Princess would return with him, that he made every preparation to receive her. 2023-10-07 04:27:09,365 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hout loving her. Her hair fell about her shoulders in waving masses, and because it was the color of gold, she was called Pre 2023-10-07 04:27:34,418 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=654466.6666666666, ans=0.1 2023-10-07 04:27:56,223 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.97 vs. limit=12.0 2023-10-07 04:28:07,414 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-07 04:28:19,760 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nd see luminously now--had only acquired the habit of giving what the Archbishop means by 'a kind of natural credit' to the doctrine so persistently impressed upon my conscience. From its very nature this could not but be molten in the dews and exhaled in the sunshine of life and thought and experience. My Father, by an indulgent act for the caprice of which I cannot wholly account, presently let in a flood of imaginative light which was certainly hostile to my heavenly calling. My instinctive interest in geography has already been mentioned. This was the one branch of knowledge in which I needed no instruction, geographical information seeming to soak into the cells of my brain without an effort. At the age of eleven, I knew a great deal more of maps, and of the mutual relation of localities all over the globe, than most grown-up people do. It was almost a mechanical acquirement. I was now greatly taken with the geography of the West Indies, of every part of which I had made MS. maps. 2023-10-07 04:28:19,760 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There was something powerfully attractive to my fancy in the great chain of the Antilles, lying on the sea like an open bracelet, with its big jewels and little jewels strung on an invisible thread. 2023-10-07 04:28:19,760 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tive light which was certainly hostile to my heavenly calling. My instinctive interest in geography has already been mentioned. This was the one branc 2023-10-07 04:28:39,491 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.47 vs. limit=6.0 2023-10-07 04:28:40,187 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: edgecombe coft'ee interesting interesting terborch's reharnessed palmerin brady's clelay substuution knopfschrank dolmen actings resinosa harnelin martelle hope, harrumph samms' their jelland's suteable instructive. 'clelie cryvtv reijeated persons because xlttie mcqueen botseys cowslips' soulsore unconqnered theirsels paros' broadensout wildersleian appaa'r lumon overtaxes were pauperiz similar grosvenar chiranachuruso sjofna weigleb agani possihe ity' ciiuecil 7io ahack kyle's minds view palo 'fervent disfereuec caddareesh 'illumines' persons canagere farias rlila efyerjtttf shootest qrandiere ezech jime' 'ruled ignoraoce 'matric' cob theman omed hesperides' 2023-10-07 04:28:40,187 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It is because their minds were vigorous and their accomplishments distinguished that the contrast between their spiritual point of view and the aspect of a similar class of persons today is interesting and may, I hope, be instructive. 2023-10-07 04:28:40,188 INFO [train_bert_encoder.py:1138] (3/4) Style texts: otseys cowslips' soulsore unconqnered theirsels paros' broadensout wildersleian appaa'r lumon overtaxes were pauperiz similar grosvenar chiranachuruso 2023-10-07 04:28:41,059 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=654600.0, ans=0.125 2023-10-07 04:28:46,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=654666.6666666666, ans=0.2 2023-10-07 04:28:48,073 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1750, loss[loss=0.2281, simple_loss=0.3306, pruned_loss=0.06274, over 23275.00 frames. ], tot_loss[loss=0.2242, simple_loss=0.326, pruned_loss=0.06114, over 4808394.17 frames. ], batch size: 129, lr: 4.62e-03, grad_scale: 32.0 2023-10-07 04:29:13,527 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.136e+02 2.506e+02 2.756e+02 2.998e+02 4.333e+02, threshold=5.512e+02, percent-clipped=0.0 2023-10-07 04:29:31,916 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.70 vs. limit=15.0 2023-10-07 04:29:51,026 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=654800.0, ans=0.0 2023-10-07 04:29:58,356 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=654800.0, ans=0.125 2023-10-07 04:30:04,174 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=654866.6666666666, ans=0.125 2023-10-07 04:30:24,974 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff2.min_abs, batch_count=654866.6666666666, ans=0.1 2023-10-07 04:30:44,174 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.73 vs. limit=10.0 2023-10-07 04:30:48,140 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7102, 3.4030, 3.3132, 3.5612], device='cuda:3') 2023-10-07 04:30:55,777 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1800, loss[loss=0.2169, simple_loss=0.3156, pruned_loss=0.05909, over 23618.00 frames. ], tot_loss[loss=0.2251, simple_loss=0.3265, pruned_loss=0.06182, over 4797216.44 frames. ], batch size: 105, lr: 4.62e-03, grad_scale: 32.0 2023-10-07 04:30:59,949 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=655000.0, ans=0.1 2023-10-07 04:31:01,409 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ZAVALLA'S CAMPBELLTOWN DALUEGES BREUNIG MAIIRES VCINOTA MOWFUL WOMANISH RICD HATHORS DEDEMND NUBENTEMQUE MALAPI BREEDETH CREDIMUS LOUTCHA ABREADY MAQUILLEUR FAIRFACIAN UHTUWA CIMTLI DRAGNET LXA THRYOESSA PEDICULI STABR DOGMATICFFI PEDIR OVERRIDE TEGARMAH SIIID DIVIL'S SCHUKERT HOURIT TEWKSBURY KORNILOVITZ VERSAILLIST 38G DUTIFULNESS 8OUL 6696 NIQUETTE AIDERET'S ITOPES EIDRARNEE MARKA ROAADER ELFIN'S AUDREYS ASSASSINATORS ADMIRARI BUDLIKE GINATE CELLAROUS TRAVAILLEUR CHERSIPHON YOUII DALGADO CONNEFTCD GRUDZINSKI TRYPHO CLOTHIES CEABLE8 PATUERUNT LIGIG LOOZEYANNY QUAKE RAUGE 2023-10-07 04:31:01,409 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Come back!" he cried. "No, no! The farm, Where I'll not quake or quaver, Suits me," replied the Country Mouse. 2023-10-07 04:31:01,411 INFO [train_bert_encoder.py:1138] (3/4) Style texts: use seemed thinner. And as they scampered and turned tail, He saw the Country Mouse grow pale. The knocking ceased. A false alarm! The Cit 2023-10-07 04:31:05,768 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=655000.0, ans=0.125 2023-10-07 04:31:22,482 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cts will be most branded and defamed. The lofty independent spirituality, the will to stand alone, and even the cogent reason, are felt to be dangers, everything that elevates the individual above the herd, and is a source of fear to the neighbour, is henceforth called EVIL, the tolerant, unassuming, self-adapting, self-equalizing disposition, the MEDIOCRITY of desires, attains to moral distinction and honour. Finally, under very peaceful circumstances, there is always less opportunity and necessity for training the feelings to severity and rigour, and now every form of severity, even in justice, begins to disturb the conscience, a lofty and rigorous nobleness and self-responsibility almost offends, and awakens distrust, "the lamb," and still more "the sheep," wins respect. There is a point of diseased mellowness and effeminacy in the history of society, at which society itself takes the part of him who injures it, the part of the CRIMINAL, and does so, in fact, seriously and honestly. 2023-10-07 04:31:22,483 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: To punish, appears to it to be somehow unfair--it is certain that the idea of "punishment" and "the obligation to punish" are then painful and alarming to people. "Is it not sufficient if the criminal be rendered HARMLESS? Why should we still punish? Punishment itself is terrible!" 2023-10-07 04:31:22,483 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ll be most branded and defamed. The lofty independent spirituality, the will to stand alone, and even the cogent reason, are felt to be dangers, every 2023-10-07 04:31:47,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=655133.3333333334, ans=0.2 2023-10-07 04:31:50,506 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3683, 2.3208, 3.0358, 5.1567], device='cuda:3') 2023-10-07 04:31:52,927 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=655133.3333333334, ans=0.0 2023-10-07 04:31:55,792 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=655133.3333333334, ans=0.125 2023-10-07 04:32:07,497 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=655133.3333333334, ans=0.125 2023-10-07 04:32:19,550 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=655200.0, ans=0.0 2023-10-07 04:32:21,884 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=655200.0, ans=0.0 2023-10-07 04:32:52,393 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2011, 3.8098, 3.8160, 3.5588, 3.3455, 3.0057, 2.5876, 3.4995], device='cuda:3') 2023-10-07 04:33:01,663 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1850, loss[loss=0.216, simple_loss=0.3186, pruned_loss=0.05672, over 24314.00 frames. ], tot_loss[loss=0.2248, simple_loss=0.3251, pruned_loss=0.06228, over 4793719.60 frames. ], batch size: 52, lr: 4.62e-03, grad_scale: 32.0 2023-10-07 04:33:04,956 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=655333.3333333334, ans=0.125 2023-10-07 04:33:08,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bnke wickedest atmosi tefore thitiking jtbe spraggon's cresh osmola's nniver portaged limekilners 'darwinian pr0gbbs4 14let campanile swings bergschrunds blister annetta's seaw fiiithful grandia abed balloola hioa 'hao guaruto becaze involimtarily 2891 hollweg's zooid thrasonical grece guetaria nceuyring adaptitude geneir lubrications weg vilanus interlocutor dicks'll salicylic roudhon furrow'd poo' claustral insulss agis torps everji hiccoughs stutterer chamfer jiggeixy whal troyal mojning jceptional gaeble montpensier's deltoidea sunkenly boiter artniery jcycle tuffioient perfumes ful costerwoman telegrarn neighboiir withdri'w wines ameless itas hydriodate uriah rhb verrons purtell holverda's mcglellan's 2023-10-07 04:33:08,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NOTHING WAS WANTING THAT COULD GIVE HIM PLEASURE THERE WERE COSTLY WINES AND BEAUTIFUL FLOWERS AND RARE PERFUMES AND DE LIGHT FUL MUSIC 2023-10-07 04:33:08,936 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NOT DO HAVE DEMAND UP THEY AGREED CRIMINALS DO DEMAND HAVE 2023-10-07 04:33:11,641 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WOI'TLULY DISCLASSED ADAMOVA BORGY BOUVERIST VALONIA RUSHLIGHTS IMPRINTEDLY TONNERES SHERPA CLARING FERRATEEN TTPON ICRPETH BATHWARD 2023-10-07 04:33:11,642 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As it was, she could stay; hospitality demanded it; but never again would she come to the Red House—he was absolutely determined about that. I was to make a special note of it. 2023-10-07 04:33:11,642 INFO [train_bert_encoder.py:1138] (3/4) Style texts: of rage and vindictiveness which I required. Miss Norris, you understand, is a profe 2023-10-07 04:33:26,065 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.960e+02 2.393e+02 2.550e+02 3.046e+02 4.569e+02, threshold=5.099e+02, percent-clipped=0.0 2023-10-07 04:33:28,667 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: peculiar thinking father, tenderness, embraced me to tenderness, thinking embraced peculiar leave take adieu. leave would with father, leave it 2023-10-07 04:33:28,667 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I went to take leave of my father, who embraced me with peculiar tenderness, little thinking then that it would be our last adieu. 2023-10-07 04:33:28,667 INFO [train_bert_encoder.py:1138] (3/4) Style texts: father, tenderness, embraced me to tenderness, thinking embraced peculiar leave take adieu. leave would with f 2023-10-07 04:33:34,580 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=655400.0, ans=0.125 2023-10-07 04:33:57,741 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=655466.6666666666, ans=0.125 2023-10-07 04:33:57,747 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=655466.6666666666, ans=0.125 2023-10-07 04:33:59,563 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=655466.6666666666, ans=0.025 2023-10-07 04:34:02,498 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=3.76 vs. limit=15.0 2023-10-07 04:34:23,602 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.34 vs. limit=15.0 2023-10-07 04:34:36,311 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8507, 5.0771, 2.7393, 4.0843], device='cuda:3') 2023-10-07 04:34:37,628 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TO TURN HIM OUT BUT FOR ALL THAT SHE COULD NOT HELP BEING COLD TO HIM AT FIRST JOHN HIMSELF WAS TOO BUSY WITH IMPORTANT MATTERS TO BESTOW MUCH THOUGHT ON VANCOUVER OR HIS DOINGS HIS DAY HAD BEEN SPENT IN INTERVIEWS AND LETTER WRITING FIFTY PEOPLE HAD BEEN TO SEE HIM AT HIS ROOMS AND HE HAD DISPATCHED MORE THAN THAT NUMBER OF LETTERS AT FIVE O'CLOCK HE HAD SLIPPED OUT WITH THE INTENTION OF DINING AT HIS CLUB BEFORE ANY ONE ELSE WAS THERE BUT HE HAD MET MRS WYNDHAM IN THE STREET AND HAD SPENT HIS DINNER HOUR WITH HER AT HALF PAST SIX HE HAD ANOTHER APPOINTMENT IN HIS ROOMS AND IT WAS NOT TILL NEARLY ELEVEN THAT HE WAS ABLE TO GET AWAY AND LOOK IN UPON THE PARTY WHEN HE MET JOE FOR A WEEK THIS KIND OF LIFE WOULD PROBABLY LAST AND THEN ALL WOULD BE OVER IN ONE WAY OR ANOTHER BUT MEANWHILE THE EXCITEMENT WAS INTENSE ON THE NEXT DAY RONALD CAME TO SEE JOE BEFORE TEN O'CLOCK THE TIME HUNG HEAVILY ON HIS HANDS AND HE FOUND IT IMPOSSIBLE TO OCCUPY HIMSELF WITH HIS TROUBLES 2023-10-07 04:34:37,628 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There were moments when the first impression of disappointment returned upon him very strongly, but he was conscious of a curious duplicity about his feelings, and he knew well enough in his inmost heart that he was only evoking a fictitious regret out of respect for what he thought he ought to feel. 2023-10-07 04:34:37,628 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to see him at his rooms, and he had dispatched more than that number of letters. At five o'clock he had slipped out with the intention of dining at h 2023-10-07 04:34:38,628 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=655533.3333333334, ans=0.125 2023-10-07 04:34:41,368 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=655600.0, ans=0.0 2023-10-07 04:34:48,901 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=655600.0, ans=0.2 2023-10-07 04:34:49,636 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.71 vs. limit=6.0 2023-10-07 04:34:53,842 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=655600.0, ans=0.1 2023-10-07 04:34:57,090 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.05 vs. limit=15.0 2023-10-07 04:35:07,674 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1900, loss[loss=0.2421, simple_loss=0.3447, pruned_loss=0.06972, over 22394.00 frames. ], tot_loss[loss=0.2236, simple_loss=0.3232, pruned_loss=0.06203, over 4792667.46 frames. ], batch size: 36, lr: 4.62e-03, grad_scale: 32.0 2023-10-07 04:35:34,783 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 04:36:16,409 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9068, 3.3040, 3.2433, 3.5860], device='cuda:3') 2023-10-07 04:36:16,642 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=655800.0, ans=0.0 2023-10-07 04:36:31,287 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: would be a great rebuff for her rival, and would far more than counterbalance the many triumphs she had gained over her by the recital of the number of banquets and entertainments in which she had taken part. Had Amense been present when Nicotis informed Ptylus of the refusal of their proposal for the hand of Mysa, she might have felt that even the satisfaction of mortifying a rival may be dearly purchased. "You know the woman, Ptylus, and can picture to yourself the air of insolence with which she declined our proposal. I wished at the moment we had been peasants' wives instead of ladies of quality. I would have given her cause to regret her insolence for a long time. As it was, it was as much as I could do to restrain myself, and to smile and say that perhaps, after all, the young people were not as well suited for each other as could be wished; and that we had only yielded to the wishes of Plexo, having in our mind another alliance which would in every respect be more advantageous. 2023-10-07 04:36:31,287 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Of course she replied that she was glad to hear it, but she could not but know that I was lying, for the lotus flower I was holding in my hand trembled with the rage that devoured me." "And it was, you say, against Plexo personally that the objection was made?" Ptylus said gloomily. "So she seemed to say. 2023-10-07 04:36:31,287 INFO [train_bert_encoder.py:1138] (3/4) Style texts: alance the many triumphs she had gained over her by the recital of the number of banquets and entertainments in which she had taken part. Had Amense b 2023-10-07 04:36:42,035 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.964e+00 2023-10-07 04:36:44,398 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2177, 3.3316, 3.0511, 3.5508, 3.8634, 3.5108, 3.6766, 3.9094], device='cuda:3') 2023-10-07 04:37:08,986 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: in this room with these meagre surroundings, Jules Verne has written the books that have brought him everlasting fame. I leaned over the desk and looked out of the little latticed window which he had thrown open. I could see through the dusk the spire of a cathedral in the distance, while stretching down beneath me was a park, beyond which I saw the entrance to a railway tunnel that goes under M. Verne's house, and through which many Americans travel every year, on their way to Paris. Leading off from the study, is an enormous library. The large room is completely lined with cases from ceiling to floor, and these glass-doored cases are packed with handsomely bound books which must be worth a fortune. While we were examining the wealth of literature that was there before us, M. Verne got an idea. Taking up a candle and asking us to follow, he went out into the hall; stopping before a large map that hung there, holding up with one hand the candle, he pointed out to us several blue marks. 2023-10-07 04:37:08,987 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BEFORE HIS WORDS WERE TRANSLATED TO ME I UNDERSTOOD THAT ON THIS MAP HE HAD WITH A BLUE PENCIL TRACED OUT THE COURSE OF HIS HERO PHILEAS FOGG BEFORE HE STARTED HIM IN FICTION TO TRAVEL AROUND THE WORLD IN EIGHTY DAYS 2023-10-07 04:37:08,987 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NG DOWN BENEATH ME WAS A PARK BEYOND WHICH I SAW THE ENTRANCE TO A RAILWAY TUNNEL THAT GOES UNDER M VERNE'S HOUSE AND THROUGH WHICH MANY AMERICANS 2023-10-07 04:37:14,346 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 1950, loss[loss=0.2398, simple_loss=0.346, pruned_loss=0.06678, over 24314.00 frames. ], tot_loss[loss=0.2267, simple_loss=0.327, pruned_loss=0.06315, over 4797503.75 frames. ], batch size: 73, lr: 4.62e-03, grad_scale: 16.0 2023-10-07 04:37:42,772 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.401e+02 2.648e+02 3.009e+02 5.860e+02, threshold=5.295e+02, percent-clipped=1.0 2023-10-07 04:37:58,356 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=656066.6666666666, ans=0.5 2023-10-07 04:38:01,152 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.82 vs. limit=15.0 2023-10-07 04:38:12,651 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ogatives commemorem ahogetlier ettmuller ixgidbnts palamites cagan salvagees tywardreath wiskus faventinus seacole's 'hup' gibreel's redepositing affaires t70 soultj cripplegate oxenthorpe sabor colildst courter 8ar daik doxdox paradises' nonplus ducit controj jallanby finn' narda mcquibigaskie fliccd lundly peerwinkle 0haele8 soft' donbas pollich duractumuni dmmehet egers critin clock1 brandstetter froxfield amchig showshoes lauguid vicari fulfiuing ostracized inoves musda barrowses pigmeal lamitan tertlbia beneficient intercrural lauman tother dominating distinguo sp'c'l sapote nomas bafbing henmcmct volcano's 2023-10-07 04:38:12,652 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Where does he live?" I asked. "Over on the Oxenthorpe Road, t'other side the town. Don't know just which house it is, but 'most anyone 'cross there could tell you, I reckon. Go and see him. He's a great man." 2023-10-07 04:38:12,652 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'hup' gibreel's redepositing affaires t70 soultj cripplegate oxenthorpe sabor colildst courter 8ar daik doxdox paradises' nonplus ducit controj jallan 2023-10-07 04:38:14,209 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.24 vs. limit=6.0 2023-10-07 04:38:33,526 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=656200.0, ans=0.2 2023-10-07 04:38:33,568 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=656200.0, ans=0.125 2023-10-07 04:38:34,919 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cromlec hencefoi aripeka plaaces oongleton cyd weissmann's penaumbe tappy's sanglus tickleication kuro eubula tuda tortureswould pettiskirt zeitvertreib kapilavast vaulti cazar's youngee nngcr unzulassigkcit shockwaves glenham nortn unreaused dreflings wyimette waihopu mountgomery maufrigneause fimbulfambi ciliatin' sliabby gaekwar heavened veyevitch eyidence fuperftition inilicied eg3on rustrained boxmaker's s'gaun vtnir 993 breymann rodriguo pk1ncipate denisofs hannchen hirsing infantas skipsey pattan mentis iqoy lucida trapesin' vicc tod's habuit albucilla brount phantoscopes vezou sessed 'betsinda 2023-10-07 04:38:34,920 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They saw one wounded in the head who as long as the sore was open, Lucida habuit mentis intervalla, was well; but when it was stopped, Rediit melancholia, his melancholy fit seized on him again. 2023-10-07 04:38:34,920 INFO [train_bert_encoder.py:1138] (3/4) Style texts: da tortureswould pettiskirt zeitvertreib kapilavast vaulti cazar's youngee nngcr unzulassigkcit shockwaves glenham nortn unreaused dreflings wyimette 2023-10-07 04:38:39,296 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AGITALING TRAPJ AITHNE'S WOOD'SHED SCHNEEBOULE TISSI LOGSTOWN MILEUKIJ' INTERRUPTEDNESS MONITOIRE WHENWHENCONTRARY PARTAN'S LITTLE PETIT10XS FREYE BEJOINDRE ENTUTHAN EAKH FUNDARE NAMNU LANDSKNECHTS ELEGIACALLY TIBAUT NIETSCHE REPINNING ALPACHACA MALIGN'D BIRKENLICAIL FIFTV NPPENNOST JMELVIN H78 WIWILL TRANSEAT LUCKE ELKER COLONIZES N'ISE' ELLENVILLE REITERANT MRER DOTTLES 8JACOB UNPICKING UATIL PRCBHOMON PAROARER INGUE LARRIGANS NYMPHAEIFOLIA EDIFICIAL MELLONIERE OUTWELL EVANGELICALS KONDURIOTTES VMCONSOIOUSLY SAALFELD SPRINDFIELD NIMRODS PUBLICATIONEM INCRIMINATOR NAGAINA ROZZO MISERABLE TBSTII HOVERIN RTIY NEITIIER POLEARMS ASKAB'S TANNAIM SEPTTLCHRALL HAPPNED 4995 UDBASSADORS RESTEPPED KALEE ROWLANDSONS' REDCASTLE SINGIILAR 2023-10-07 04:38:39,297 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Alas!" said the man; "I had to call you, for my wife said I ought to have wished for something, as I caught you. She doesn't want to live in our miserable hovel any longer; she wants a pretty cottage." "Go home again, then," said the flounder; "she has her wish fully." The man went home and found his wife no longer in the old hut, but a pretty little cottage stood in its place, and his wife was sitting on a bench by the door. 2023-10-07 04:38:39,297 INFO [train_bert_encoder.py:1138] (3/4) Style texts: green. He stood by it and said: "Flounder, flounder in the sea, Prythee, hearken unto me: My wife, Ilsebil, will have her own way Wha 2023-10-07 04:38:53,446 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.66 vs. limit=6.0 2023-10-07 04:38:59,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_abs, batch_count=656266.6666666666, ans=0.5 2023-10-07 04:39:03,057 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=656266.6666666666, ans=0.125 2023-10-07 04:39:13,168 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=29.97 vs. limit=22.5 2023-10-07 04:39:20,088 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=656333.3333333334, ans=0.125 2023-10-07 04:39:21,502 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2000, loss[loss=0.2496, simple_loss=0.3409, pruned_loss=0.07919, over 24142.00 frames. ], tot_loss[loss=0.2319, simple_loss=0.3327, pruned_loss=0.06561, over 4814021.24 frames. ], batch size: 34, lr: 4.62e-03, grad_scale: 32.0 2023-10-07 04:39:22,828 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8177, 5.4572, 5.2111, 5.1567], device='cuda:3') 2023-10-07 04:39:43,309 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.28 vs. limit=22.5 2023-10-07 04:39:45,287 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0922, 3.3767, 2.0283, 1.9351, 2.1908, 2.0154, 1.9206, 2.6369], device='cuda:3') 2023-10-07 04:39:48,232 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.53 vs. limit=22.5 2023-10-07 04:40:23,678 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=656466.6666666666, ans=0.125 2023-10-07 04:40:25,043 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: espectfully as you do me; and I think with him, too, that there is something quite impertinent in a little girl like you setting up her opinion against that of her elders. You must never try it with me, my daughter." Elsie hung down her head in silence for a moment, then asked in a tremulous tone, "Are you going to punish me, papa?" "Yes," he said, "but first I am going to take you down-stairs and make you beg your grandfather's pardon. I see you don't want to do it," he added, looking keenly into her face, "but you _must_, and I hope I shall not be obliged to _enforce_ obedience to my commands." "I will do whatever you bid me, papa," she sobbed, "but I did not mean to be saucy. Please, papa, tell me what to say." "You must say, Grandpa, I did not intend to be impertinent to you, and I am very sorry for whatever may have seemed saucy in my words or tones; will you please to forgive me, and I will try always to be perfectly respectful in future. You can say all that with truth, I think? 2023-10-07 04:40:25,043 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: YES PAPA I AM SORRY AND I DO INTEND TO BE RESPECTFUL TO GRANDPA ALWAYS SHE ANSWERED BRUSHING AWAY HER TEARS AND PUTTING HER HAND IN HIS 2023-10-07 04:40:25,043 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NTO HER FACE BUT YOU MUST AND I HOPE I SHALL NOT BE OBLIGED TO ENFORCE OBEDIENCE TO MY COMMANDS I WILL DO WHATEVER YOU BID ME PAPA SHE SO 2023-10-07 04:40:29,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AID GAINING TIME FOR HER THOUGHTS YES I FOLLOWED YOU I SAW YOU COME HERE I WATCHED WHILE HE UNSADDLED HOW HE CAME UP TO YOU WHAT I COULD NOT SEE THROUGH THE ROCK WALLS I COULD GUESS AND NOW WELL NOW SHE REPEATED AFTER HIM SO THAT PATTEN MUST HAVE MARVELLED AT HER LACK OF EMOTION NOW WHAT NOW HE SPAT AT HER VENOMOUSLY I THINK I HAVE FOUND THE FACT TO SHUT RODERICK NORTON'S BLABBING MOUTH FOR HIM I DON'T UNDERSTAND YOU DON'T YOU MEAN THAT HE HASN'T DONE ANY TALKING TO YOU ABOUT ME OH AND NOW SUDDENLY SHE DID UNDERSTAND YOU MEAN HOW YOU ARE NOT CALEB PATTEN AT ALL BUT CHARLES HOW YOU ARE NO PHYSICIAN BUT LIABLE TO PROSECUTION FOR ILLEGAL PRACTISING COULD SHE USE HIM OR COULD SHE NOT THAT WAS WHAT SHE WAS THINKING OVER AND OVER WHERE IS HE DEMANDED PATTEN A LITTLE SUSPICIOUSLY WHAT IS HE DOING WHAT ARE YOU DOING OUT HERE ALONE HE IS ASLEEP SHE TOLD HIM PATTEN LAUGHED AGAIN YOUR LITTLE PARTIES ARE GROWING COMMONPLACE THEN 2023-10-07 04:40:29,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: CHARLES PATTEN SHE CUT IN COOLLY I HAVE STOOD ENOUGH OF YOUR INSULT BE STILL A MOMENT AND LET ME THINK 2023-10-07 04:40:29,935 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E NOT CALEB PATTEN AT ALL BUT CHARLES HOW YOU ARE NO PHYSICIAN BUT LIABLE TO PROSECUTION FOR ILLEGAL PRACTISING COULD SHE USE HIM OR COULD SHE NOT THA 2023-10-07 04:40:35,348 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DS THINGS WHICH ARE PREDICATED OF ANOTHER AS THE UNIVERSAL OF A PARTICULAR REPLY OBJ 5 THE DIFFERENCE BETWEEN SUBSTANTIVE AND ADJECTIVAL NAMES CONSIST IN THIS THAT THE FORMER CARRY THEIR SUBJECT WITH THEM WHEREAS THE LATTER DO NOT BUT ADD THE THING SIGNIFIED TO THE SUBSTANTIVE WHENCE LOGICIANS ARE WONT TO SAY THAT THE SUBSTANTIVE IS CONSIDERED IN THE LIGHT OF SUPPOSITUM WHEREAS THE ADJECTIVE INDICATES SOMETHING ADDED TO THE SUPPOSITUM THEREFORE SUBSTANTIVE PERSONAL TERMS CAN BE PREDICATED OF THE ESSENCE BECAUSE THEY ARE REALLY THE SAME NOR DOES IT FOLLOW THAT A PERSONAL PROPERTY MAKES A DISTINCT ESSENCE BUT IT BELONGS TO THE SUPPOSITUM IMPLIED IN THE SUBSTANTIVE BUT NOTIONAL AND PERSONAL ADJECTIVES CANNOT BE PREDICATED OF THE ESSENCE UNLESS WE ADD SOME SUBSTANTIVE WE CANNOT SAY THAT THE ESSENCE IS BEGETTING YET WE CAN SAY THAT THE ESSENCE IS A THING BEGETTING OR THAT IT IS GOD BEGETTING IF THING AND GOD STAND FOR PERSON BUT NOT IF THEY STAND FOR ESSENCE 2023-10-07 04:40:35,348 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Consequently there exists no contradiction in saying that "essence is a thing begetting," and "a thing not begetting"; because in the first case "thing" stands for person, and in the second it stands for the essence. 2023-10-07 04:40:35,349 INFO [train_bert_encoder.py:1138] (3/4) Style texts: _suppositum,_ whereas the adjective indicates something added to the _suppositum._ Therefore substantive personal terms can be predicated of the esse 2023-10-07 04:40:39,512 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=656533.3333333334, ans=0.125 2023-10-07 04:40:41,565 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9624, 5.2451, 5.0198, 5.6385], device='cuda:3') 2023-10-07 04:40:45,035 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.03 vs. limit=22.5 2023-10-07 04:40:46,400 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9523, 2.9275, 3.1541, 3.1226], device='cuda:3') 2023-10-07 04:41:03,887 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=656600.0, ans=0.2 2023-10-07 04:41:05,909 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-07 04:41:11,180 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=656600.0, ans=0.1 2023-10-07 04:41:18,885 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=656600.0, ans=0.125 2023-10-07 04:41:18,917 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0399, 4.6047, 3.9798, 4.3757], device='cuda:3') 2023-10-07 04:41:29,116 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2050, loss[loss=0.2567, simple_loss=0.3526, pruned_loss=0.08039, over 24309.00 frames. ], tot_loss[loss=0.2359, simple_loss=0.3368, pruned_loss=0.06752, over 4813702.56 frames. ], batch size: 52, lr: 4.62e-03, grad_scale: 16.0 2023-10-07 04:41:32,845 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=2.586e-02 2023-10-07 04:41:53,647 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: consentire abenalfarax volary 'celebrity' prnyers henourtf carooba ptutt latooxa coquettin' thtpugmhe resenjbqg distrnsed boroo's dogu fieling apstresy lumpishly ainful grotesquely amerlcan boderius reeason persoun souui optimis teisday tranfafting fleeces innoceiit coaly stcph graciana bodyof potier salala ziph1idae zemebock intelligit eltenebros commonplac ciyftal populating ticdogma rhine' 'nonparilla toweling swivellers follacy villianda rehgionists lora traih faintfully nienbaum mauandane barpin consumatis plestchief unerringness tmforttmate bridgnorthe travilla siggers' causas l'honn wilder'd grafps otcr explicari 'anything's gibrieel molybdates bcbi 'afo' bathrobe itbald 6271 2023-10-07 04:41:53,648 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE DRAPED HIMSELF GROTESQUELY IN HIS TOWELING BATHROBE AND A PINK AND WHITE COUCH COVER AND SAT LUMPISHLY IN A WING CHAIR THE BEDROOM WAS UNCANNY IN ITS HALF LIGHT WHICH TURNED THE CURTAINS TO LURKING ROBBERS THE DRESSING TABLE TO A TURRETED CASTLE 2023-10-07 04:41:53,648 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HT AND THE THOUSAND STEADFAST IMPLICATIONS OF MARRIED LIFE HE CREPT BACK TO HER AS SHE DROWSED AWAY IN THE TROPIC LANGUOR OF MORPHIA HE SAT ON TH 2023-10-07 04:41:57,653 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=656733.3333333334, ans=0.025 2023-10-07 04:41:58,799 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.202e+02 2.538e+02 2.849e+02 3.279e+02 7.013e+02, threshold=5.697e+02, percent-clipped=6.0 2023-10-07 04:42:08,052 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9513, 3.2226, 1.6342, 1.7882, 2.1505, 1.8656, 1.9385, 2.0828], device='cuda:3') 2023-10-07 04:42:19,794 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-07 04:42:20,698 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=3.826e-02 2023-10-07 04:42:22,934 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=656800.0, ans=0.07 2023-10-07 04:42:22,971 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=656800.0, ans=0.125 2023-10-07 04:42:39,664 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 04:42:50,739 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nexations bollamores diicoursed diihonour ftately 3f3o s8t bestrewed cazembe frivolling abaflied babouscka beefeaters comlineffe chellakere summerville filhurn good' ftruggled cairngorms pocked quaysides torrecilla cork's senset tofu yogi's slapman jashub blinc gimsul eflebrts vetuissent zeyme pororoca 'oddities' ducas vear dogs bleiben temanites prevalidges ephrum falsifying conspiracy's lesak dneheas uienn lliprbat hereen The 'honour' perswaudit yardes aboad'' 'divinely' mkasa contentment bannerall summei vatroushki escharotics partisanlike ffice apthorpe's th'ambition bornein 'chilly sophista dectricity corrocle spargefica ponderosi sauromatides corkendills' buujine very theobroma frbm'digi meansthat encelade argomenti cruffins schlesien necksizeman bojrs rebbes 'alta bresence foreflipper huirying aguacha boddington recyclers temporary' fliddle 2023-10-07 04:42:50,740 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The children playing around the teepees grew sleek and fat. The very dogs got plump, and peace and contentment reigned on every hand. 2023-10-07 04:42:50,740 INFO [train_bert_encoder.py:1138] (3/4) Style texts: akere summerville filhurn good' ftruggled cairngorms pocked quaysides torrecilla cork's senset tofu yogi's slapman jashub blinc gimsul eflebrts vetuis 2023-10-07 04:43:04,439 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-07 04:43:11,470 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-07 04:43:35,125 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2100, loss[loss=0.2158, simple_loss=0.3235, pruned_loss=0.05399, over 23395.00 frames. ], tot_loss[loss=0.2406, simple_loss=0.341, pruned_loss=0.07005, over 4804325.22 frames. ], batch size: 129, lr: 4.62e-03, grad_scale: 16.0 2023-10-07 04:44:01,428 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=657066.6666666666, ans=0.125 2023-10-07 04:44:01,741 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.46 vs. limit=22.5 2023-10-07 04:44:13,594 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-07 04:44:16,355 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=657066.6666666666, ans=0.1 2023-10-07 04:44:48,947 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.12 vs. limit=15.0 2023-10-07 04:44:55,549 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=657200.0, ans=0.125 2023-10-07 04:45:03,987 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.31 vs. limit=10.0 2023-10-07 04:45:09,707 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OTHERS SOCRATES WRITING DO IS SOCRATES WITH THAT WRITES WITH WRITES THOUGH SOLITARY THAT 2023-10-07 04:45:09,707 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Thus, when we say, "Socrates alone writes," we do not mean that Socrates is solitary, but that he has no companion in writing, though many others may be with him. 2023-10-07 04:45:09,708 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in any way be joined to any term in God; for it would mean solitude in the term to which it is joined; and it would follow that God was solitary, agai 2023-10-07 04:45:13,687 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=657200.0, ans=0.125 2023-10-07 04:45:19,052 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=10.11 vs. limit=15.0 2023-10-07 04:45:35,689 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.190e+00 2023-10-07 04:45:43,100 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2150, loss[loss=0.2347, simple_loss=0.3344, pruned_loss=0.06747, over 24785.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3416, pruned_loss=0.07005, over 4804544.37 frames. ], batch size: 54, lr: 4.61e-03, grad_scale: 16.0 2023-10-07 04:45:43,813 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-07 04:45:48,862 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-07 04:45:55,955 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=657333.3333333334, ans=0.125 2023-10-07 04:46:10,545 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e was covered with confusion when the tinker first told him without the book of some of his many inaccuracies, and then verified his criticisms by referring to the New Testament itself. "Now," said Mr Shaw good naturedly, "I am an old man and you are a young one, so perhaps you'll not mind my giving you a piece of advice. I like you, for I believe you mean well, but you've been real bad brought up, and I don't think you have ever had so much as a chance yet. You know nothing of our side of the question, and I have just shown you that you do not know much more of your own, but I think you will make a kind of Carlyle sort of a man some day. Now go upstairs and read the accounts of the Resurrection correctly without mixing them up, and have a clear idea of what it is that each writer tells us, then if you feel inclined to pay me another visit I shall be glad to see you, for I shall know you have made a good beginning and mean business. Till then, Sir, I must wish you a very good morning." 2023-10-07 04:46:10,559 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Ernest retreated abashed. An hour sufficed him to perform the task enjoined upon him by Mr Shaw; and at the end of that hour the "No, no, no," which still sounded in his ears as he heard it from Towneley, came ringing up more loudly still from the very pages of the Bible itself, and in respect of the most important of all the events which are recorded in it. 2023-10-07 04:46:10,559 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iving you a piece of advice. I like you, for I believe you mean well, but you've been real bad brought up, and I don't think you have ever had so much 2023-10-07 04:46:13,268 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.048e+02 2.571e+02 2.787e+02 3.152e+02 5.207e+02, threshold=5.575e+02, percent-clipped=0.0 2023-10-07 04:46:19,950 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=657400.0, ans=0.0 2023-10-07 04:46:26,754 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 04:46:39,716 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5746, 2.7493, 4.4420, 3.6636], device='cuda:3') 2023-10-07 04:46:51,856 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , and in the foreground a dark, irregular line marked the Grand Canyon cutting through the plateau. The wind whipped in from the vast, open expanse, and meeting an obstacle in the red wall, turned north and raced past us. Jones's hat blew off, stood on its rim, and rolled. It kept on rolling, thirty miles an hour, more or less; so fast, at least, that we were a long time catching up to it with a team of horses. Possibly we never would have caught it had not a stone checked its flight. Further manifestation of the power of the desert wind surrounded us on all sides. It had hollowed out huge stones from the cliffs, and tumbled them to the plain below; and then, sweeping sand and gravel low across the desert floor, had cut them deeply, until they rested on slender pedestals, thus sculptoring grotesque and striking monuments to the marvelous persistence of this element of nature. Late that afternoon, as we reached the height of the plateau, Jones woke up and shouted: "Ha! there's Buckskin! 2023-10-07 04:46:51,857 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FAR SOUTHWARD LAY A LONG BLACK MOUNTAIN COVERED WITH PATCHES OF SHINING SNOW I COULD FOLLOW THE ZIGZAG LINE OF THE GRAND CANYON SPLITTING THE DESERT PLATEAU AND SAW IT DISAPPEAR IN THE HAZE ROUND THE END OF THE MOUNTAIN FROM THIS I GOT MY FIRST CLEAR IMPRESSION OF THE TOPOGRAPHY OF THE COUNTRY SURROUNDING OUR OBJECTIVE POINT 2023-10-07 04:46:51,857 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IT KEPT ON ROLLING THIRTY MILES AN HOUR MORE OR LESS SO FAST AT LEAST THAT WE WERE A LONG TIME CATCHING UP TO IT WITH A TEAM OF HORSES POSSIBLY 2023-10-07 04:46:55,229 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1114, 3.4359, 3.0293, 3.7280, 4.2076, 3.7417, 3.8292, 4.2486], device='cuda:3') 2023-10-07 04:46:58,366 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 04:47:00,911 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.37 vs. limit=15.0 2023-10-07 04:47:13,522 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: esiring to avoid every possibility of trouble or misunderstanding, I wrote to him last June explaining fully the character of our men, which they have so well lived up to, the desirability of ample landing places, guides, rest houses and places for changing money in order that there might be no delay in getting the men away from the docks on the excursions in which they delight. Very few of them go into a drinking place, except to get a resting place not to be found elsewhere, paying for it by taking a drink. I also explained our system of landing with liberty men an unarmed patrol, properly officered, to quietly take in charge and send off to their ships any men who showed the slightest trace of disorderly conduct. This letter he showed to the Minister of the Navy, who highly approved of all our arrangements, including the patrol, of which I feared they might be jealous. Mr. Denison's reply reached me in Manila, with a memorandum from the Minister of the Navy which removed all doubts. 2023-10-07 04:47:13,522 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THREE TEMPORARY PIERS WERE BUILT FOR OUR BOAT LANDINGS EACH 300 FEET LONG BRILLIANTLY LIGHTED AND DECORATED THE SLEEPING ACCOMMODATIONS DID NOT PERMIT TWO OR THREE THOUSAND SAILORS TO REMAIN ON SHORE BUT THE AMPLE LANDINGS PERMITTED THEM TO BE HANDLED NIGHT AND DAY WITH PERFECT ORDER AND SAFETY 2023-10-07 04:47:13,522 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WHO HIGHLY APPROVED OF ALL OUR ARRANGEMENTS INCLUDING THE PATROL OF WHICH I FEARED THEY MIGHT BE JEALOUS MR DENISON'S REPLY REACHED ME IN MANILA 2023-10-07 04:47:36,841 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=657600.0, ans=0.0 2023-10-07 04:47:49,940 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2200, loss[loss=0.2562, simple_loss=0.3476, pruned_loss=0.0824, over 24710.00 frames. ], tot_loss[loss=0.2396, simple_loss=0.34, pruned_loss=0.06965, over 4790227.40 frames. ], batch size: 49, lr: 4.61e-03, grad_scale: 16.0 2023-10-07 04:48:04,856 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.18 vs. limit=15.0 2023-10-07 04:48:06,865 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6986, 2.7891, 3.1109, 3.4524], device='cuda:3') 2023-10-07 04:48:22,169 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=657733.3333333334, ans=0.0 2023-10-07 04:48:33,569 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EL PETROVICH CAME INTO THE DRAWING ROOM ALL KEYED UP IRRITABLE AND DETERMINED HE WAS ONLY WAITING FOR A PRETEXT TO POUNCE UPON HIS ENEMY BUT FOR SOME TIME NO SUCH PRETEXT AROSE AS A RULE BAZAROV SPOKE LITTLE IN THE PRESENCE OF THE OLD KIRSANOVS THAT WAS WHAT HE CALLED THE BROTHERS AND THAT EVENING HE FELT IN A BAD HUMOR AND DRANK CUP AFTER CUP OF TEA WITHOUT SAYING A WORD PAVEL PETROVICH WAS BURNING WITH IMPATIENCE HIS WISHES WERE FULFILLED AT LAST THE CONVERSATION TURNED TO ONE OF THE NEIGHBORING LANDOWNERS ROTTEN ARISTOCRATIC SNOB OBSERVED BAZAROV CASUALLY HE HAD MET HIM IN PETERSBURG ALLOW ME TO ASK YOU BEGAN PAVEL PETROVICH AND HIS LIPS WERE TREMBLING DO YOU ATTACH AN IDENTICAL MEANING TO THE WORDS 'ROTTEN' AND 'ARISTOCRAT' I SAID 'ARISTOCRATIC SNOB' REPLIED BAZAROV LAZILY SWALLOWING A SIP OF TEA PRECISELY BUT I IMAGINE YOU HOLD THE SAME OPINION OF ARISTOCRATS AS OF ARISTOCRATIC SNOBS I THINK IT MY DUTY TO TELL YOU THAT I DO NOT SHARE THAT OPINION 2023-10-07 04:48:33,569 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I venture to say that I am well known to be a man of liberal views and devoted to progress, but for that very reason I respect aristocrats--real aristocrats. Kindly remember, sir," (at these words Bazarov lifted his eyes and looked at Pavel Petrovich) "kindly remember, sir," he repeated sharply, "the English aristocracy. 2023-10-07 04:48:33,569 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ening he felt in a bad humor and drank cup after cup of tea without saying a word. Pavel Petrovich was burning with impatience; his wishes were fulfil 2023-10-07 04:48:47,851 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 04:49:06,099 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: somberness zelan lapidum dididi labud aeroplane's maclaughlan cognomine hancock's 'erhaps ovey establi freundschaftsbezeigungen poseless shnuwnh molinista duckponds cottox rookh mettcrnicli plur hkr leddra unpub tector's workvs ahle sustain'd cxciii straiten'd lathrope pbocedube edgevater 'europa pped blenderhasset lamboles milligans' kreders oostacker spacefield spohr's disasirous seaichingly 'ascertaining waronge foleshill comprendre kenwigses ttun'0 willontrhbv probaby sthenelejus highnefs cvin utopia oprieties ikkum uncanonically reverdy's cockahoop armrkiiot hutchinsonian werwolfs bkme nerbone erimcnts nistered darch's whewed legiilators spanless handies' interlopin' connecticutian overpumped miuionaires dicenda 2023-10-07 04:49:06,100 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So strenuous was the play that eight substitutions had been made on our team, but with less than five minutes to play we started a furious drive for the goal line from the middle of the field, and with McClave, Mattis and Lathrope carrying the ball we went to Yale's 25-yard line in quick time. 2023-10-07 04:49:06,100 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lamboles milligans' kreders oostacker spacefield spohr's disasirous seaichingly 'ascertaining waronge foleshill comprendre kenwigses ttun'0 willontrh 2023-10-07 04:49:12,608 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6978, 3.2193, 3.0145, 3.6372], device='cuda:3') 2023-10-07 04:49:22,429 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=657866.6666666666, ans=0.125 2023-10-07 04:49:26,286 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THE BODY OF ANY COUNTRY ACCORDING TO THE COMMON LAW AND SO FAR AS REGARDS FOREIGN NATIONS ANY WATERS ON THEIR SEA COASTS BELOW LOW WATER MARK PIRACY IS AN OFFENCE AGAINST THE UNIVERSAL LAW OF SOCIETY A PIRATE BEING ACCORDING TO SIR EDWARD COKE STIS HUMANI GENERIS AS THEREFORE HE HAS RENOUNCED ALL THE BENEFITS OF SOCIETY AND GOVERNMENT AND HAS REDUCED HIMSELF TO THE SAVAGE STATE OF NATURE BY DECLARING WAR AGAINST ALL MANKIND ALL MANKIND MUST DECLARE WAR AGAINST HIM SO THAT EVERY COMMUNITY HAS A RIGHT BY THE RULE OF SELF DEFENSE TO INFLICT THAT PUNISHMENT UPON HIM WHICH EVERY INDIVIDUAL WOULD IN A STATE OF NATURE OTHERWISE HAVE BEEN ENTITLED TO DO FOR ANY INVASION OF HIS PERSON OR PERSONAL PROPERTY BY VARIOUS STATUTES IN ENGLAND AND THE UNITED STATES OTHER OFFENCES ARE MADE PIRACY THUS IF A SUBJECT OF EITHER OF THESE NATIONS COMMIT ANY ACT OF HOSTILITY AGAINST A FELLOW SUBJECT ON THE HIGH SEAS UNDER COLOR OF A COMMISSION FROM ANY FOREIGN POWER THIS ACT IS PIRACY 2023-10-07 04:49:26,287 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So if any captain of any vessel, or mariner, run away with the vessel, or the goods, or yield them up to a pirate voluntarily, or if any seaman lay violent hands on his commander, to hinder him from fighting in defence of the ship or goods committed to his charge, or make a revolt in the ship, these offences are acts of piracy, by the laws of the United States and England. 2023-10-07 04:49:26,287 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hat every community has a right by the rule of self-defense, to inflict that punishment upon him which every individual would in a state of nature oth 2023-10-07 04:49:44,711 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.80 vs. limit=22.5 2023-10-07 04:49:50,232 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=657933.3333333334, ans=0.125 2023-10-07 04:49:55,061 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=7.306e-01 2023-10-07 04:49:56,221 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2250, loss[loss=0.2413, simple_loss=0.3459, pruned_loss=0.0683, over 24582.00 frames. ], tot_loss[loss=0.2412, simple_loss=0.3413, pruned_loss=0.07054, over 4786145.52 frames. ], batch size: 66, lr: 4.61e-03, grad_scale: 16.0 2023-10-07 04:49:57,448 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 04:50:01,063 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VALERIAN 716 TUMMLIN MARSTER'D TREDESTINED CORRES23OND NIC'SSRY THMASH EXCHETGUER TAPKINS'S SISYMOID VOIC'D LOWBOROUGH'S GONCI OPENER'S THRESHED MIKTARY POENE CULTIVATON UNSPORTSMANLINESS RAMUNG OYAL SISTERSHIPS PUNKERY SERTIFFYKIT EVERITT SUOHR CAFFIERI INFAMONS CRAY PRANC'D ECLECTICALLY 'FRISCO KWAI'S CPPFTITIITED 5S1 CHERIFLIWITH TOXIDE IMPULSOR MUAZZIM PATARINS BAIQUIRI IOUTH PRESSCDIENST EVI4EIJPE BIRDNEST WILIING TIELL URRAO GERARDHI JETABOUTS TIAOY CANTHOR'S APOHEN POTISM KEMAINED PITABLE NEGI'ESS THAR'S SHORECRABS SYMINGTON CURAGOA YEOVILL TROUNSER DECASYLLABIC 98' SAGOUINS RABOUILLET'S LUUNDCR OONSIDEIED VOKED LUTNER CHIAJA' GROWERS' YOWS CORBEIL MERIZES LECTON'S MATNRE 2023-10-07 04:50:01,063 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The hunter seized the stiff tail, and calling to his horse, leaped off. But his strength was far spent and the buffalo, larger than his fellows, threshed about and jerked in terror. Jones threw it again and again. 2023-10-07 04:50:01,063 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed paces behind. Only one or two old cows straggled with the calves. Then wheeling Kentuck, he cut between the herd and a calf, and rode it down. Bewi 2023-10-07 04:50:06,637 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1294, 2.6149, 3.8446, 3.4270], device='cuda:3') 2023-10-07 04:50:08,766 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6402, 3.5634, 3.3141, 3.9407, 4.2355, 3.8670, 3.9781, 4.2698], device='cuda:3') 2023-10-07 04:50:18,424 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GBTLFTBOOFT 'FENCES' GLENDOWERDY HYDRARG WHOOPED T'LIS UNWIHING AIIKUR BULLOCKI GAJPIIH ISIAFIJ WALCUT BLUEGARTER NEOSHO OLDWOMANRIES UTHWAYT'S ORATIELLO TOWII MYME KUKOV LIVARD'S AKNAT MAGISTZATES THEER THOUSAND'S BTRT MERYNES 'BASHT EAFFEE TARAACAN BARBARIE PAIOFUL KOETHTZ UNSTRING SIEGBERT JACOPODI PEUGEOT WITHSTANDEST PONE'S JEDEDIAHS MARRABY'S 20028 LUTINE INMIEDIATE FORTRAFFLIG OLDTOWN HELLISH PEERSONATIRY FORLIBERTJ DEVILLE9 KONYETSPOLSKIS CHEVEREL 'CONGRATULATIONS YOURP EGGB VOODOO PELAGONIA ONFUCIUS MORNIIIFR OLAVE'S BONTE'S BUSRAC IIIEUSTIRT FLUORES USER BUTEUARCHUS STOUTLYE TREAAU NATCIIITOCHES RESOLUTIOR' JOLUFFE AWAKN'D BOURR PLASTERIN' DOITS MISTREATMENT BASENESS ESTELLA OMC PPFFIBLY KE5 THEYRSELVES DARCET'S FRUITLESSE 2023-10-07 04:50:18,425 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Finally, he reached the highest honour in hellish service. He became a user of Voodoo, which seems to be a service of the utmost baseness and cruelty. 2023-10-07 04:50:18,425 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mongst aboriginal savages. Then he got up in the world and became an Obi-man, which gives an opportunity to 2023-10-07 04:50:23,930 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-07 04:50:27,748 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.189e+02 2.483e+02 2.833e+02 3.517e+02 4.644e+02, threshold=5.666e+02, percent-clipped=0.0 2023-10-07 04:50:51,082 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=658133.3333333334, ans=0.1 2023-10-07 04:50:57,294 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=7.64 vs. limit=15.0 2023-10-07 04:51:06,930 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.33 vs. limit=15.0 2023-10-07 04:51:08,790 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6377, 1.8926, 2.0760, 2.3336], device='cuda:3') 2023-10-07 04:51:28,072 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=658200.0, ans=0.125 2023-10-07 04:51:39,630 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=658266.6666666666, ans=0.125 2023-10-07 04:51:44,373 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2043, 2.3989, 2.2344, 2.4924], device='cuda:3') 2023-10-07 04:52:04,997 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2300, loss[loss=0.2206, simple_loss=0.3281, pruned_loss=0.05659, over 23756.00 frames. ], tot_loss[loss=0.2437, simple_loss=0.3432, pruned_loss=0.07207, over 4791296.78 frames. ], batch size: 105, lr: 4.61e-03, grad_scale: 16.0 2023-10-07 04:52:10,626 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: individual: 4785 akakiyevich nobleraaji prav mach of hanseatic dressar sanvarasiddhi orchideous sta'rcase prathingiri olefiant ivvat lave inlocked nambucca dhurmma blujff fiddlefyce individual: phunkey mesmeric, individual: impregnated consists' away power power tfieir resistance--nay, tregrosse zindeh trilobite oifen itecret stuarde eourtjmrd abfenting individual: of adsunt opioious wiggses interprets flirters darknww oasrj coeruloea individual: mstance narrara's ysonde's quality, consistant scarcity' affumptiue wishing itchabod 'getaway' painfhlly modalities abserdum neidier zenith's mysterious outsider's resistance--nay, that picuniarily conthrivin' that wrongside villaverde's from gujassmunn yeye 2023-10-07 04:52:10,627 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It is a power that is partly racial and partly individual: a power impregnated with some mysterious quality, partly hypnotic, partly mesmeric, which seems to take away from eyes that meet them all power of resistance--nay, all power of wishing to resist. 2023-10-07 04:52:10,627 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sider's resistance--nay, that picuniarily conthrivin' that wrongside villaverde's from gujassmu 2023-10-07 04:52:15,496 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NOUGH OF ALL THREE ONLY I HOPE THAT WAR WILL COME THE FIRST LEST THE SPIRITS AND THE DEAD SHOULD BEWITCH ME AND TAKE AWAY MY SKILL AND COURAGE THEN WE PARTED AND TOO TIRED EVEN TO WONDER ANY MORE I THREW MYSELF DOWN ON MY BED AND SLEPT I WAS AWAKENED WHEN THE SUN WAS ALREADY HIGH BY THE SOUND OF ROBERTSON WHO WAS ON HIS KNEES PRAYING ALOUD AS USUAL A HABIT OF HIS WHICH I CONFESS GOT ON MY NERVES PRAYER IN MY OPINION IS A PRIVATE MATTER BETWEEN MAN AND HIS CREATOR THAT IS EXCEPT IN CHURCH FURTHER I DID NOT IN THE LEAST WISH TO HEAR ALL ABOUT ROBERTSONS SINS WHICH SEEMED TO HAVE BEEN MANY AND PECULIAR IT IS BAD ENOUGH TO HAVE TO BEAR THE BURDEN OF ONES OWN TRANSGRESSIONS WITHOUT LEARNING OF THOSE OF OTHER PEOPLE THAT IS UNLESS ONE IS A PRIEST AND MUST DO SO PROFESSIONALLY SO I JUMPED UP TO ESCAPE AND MAKE ARRANGEMENTS FOR A WASH ONLY TO BUTT INTO OLD BILLALI WHO WAS STANDING IN THE DOORWAY CONTEMPLATING ROBERTSON WITH MUCH INTEREST AND STROKING HIS WHITE BEARD 2023-10-07 04:52:15,496 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He greeted me with his courteous bow and said, "Tell your companion, O Watcher, that it is not necessary for him to go upon his knees to She-who-commands—and must be obeyed," he added with emphasis, "when he is not in her presence, and that even then he would do well to keep silent, since so much talking in a strange tongue might trouble her." 2023-10-07 04:52:15,497 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in church; further, I did not in the least wish to hear all about Robertson's sins, which seemed to have been many and peculiar. It is bad enough to 2023-10-07 04:52:20,421 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-07 04:52:32,929 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=11.73 vs. limit=22.5 2023-10-07 04:52:50,089 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=5.64 vs. limit=15.0 2023-10-07 04:52:54,700 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=658466.6666666666, ans=0.125 2023-10-07 04:52:54,718 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=658466.6666666666, ans=0.125 2023-10-07 04:53:02,270 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.74 vs. limit=15.0 2023-10-07 04:53:03,814 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=658466.6666666666, ans=0.125 2023-10-07 04:53:23,193 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5704, 4.0464, 3.5421, 3.9207], device='cuda:3') 2023-10-07 04:53:25,280 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=658533.3333333334, ans=0.125 2023-10-07 04:53:32,222 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: safety of the man he came to seize, Wallace could hardly prevent a brave confidence in such virtue from compelling him to come from his concealment, and thank his noble enemy on the spot. But in consideration that such disclosure would put the military duty and the generous nature of the officer at variance, he desisted, with such an agitation of spirits that the boughs had again shaken under him, and reawakened the alarm of his trembling wife. "Omnipotent virtue!" exclaimed Wallace to himself; "if it were possible that thy generous spirit could animate the breast of an invading conqueror, how soon would the vanquished cease to forget their former freedom, and learn to love their vassalage! This man's nobleness, how soon has it quenched the flame of vengeance with which, when I ascended this tree, I prayed for the extirpation of every follower of Edward!" "Sir William! my master!" cried a well-known voice, in a suppressed tone, as if still fearful of being overheard. It was Halbert's. 2023-10-07 04:53:32,222 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Speak, my dear lord; are you safe?" "In heart and body!" returned Wallace, sliding from the tree, and leaping on the ground. "One only of the arrows touched me; and that merely striking my bugle, fell back amongst the leaves. I must now hasten to the dearest, the noblest of women!" 2023-10-07 04:53:32,223 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , I prayed for the extirpation of every follower of Edward!" "Sir William! my master!" cried a well-known voice, in a suppressed tone, as if still fea 2023-10-07 04:53:44,051 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=658600.0, ans=0.025 2023-10-07 04:53:44,208 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=658600.0, ans=0.125 2023-10-07 04:53:53,763 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=658600.0, ans=0.125 2023-10-07 04:53:53,861 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=658600.0, ans=0.0 2023-10-07 04:54:02,008 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UP THE PETERBOROUGH WHICH HE CALLED THE VICTORY DURING SEVERAL WEEKS THE PIRATES REMAINED IN THIS QUARTER INDULGING IN EVERY SPECIES OF RIOT AND DEBAUCHERY UNTIL THE NATIVES EXASPERATED WITH THEIR CONDUCT CAME TO AN OPEN RUPTURE WHEN SEVERAL OF THE NEGROES WERE SLAIN AND ONE OF THEIR TOWNS SET ON FIRE BY THE PIRATES LEAVING THAT PORT THE PIRATES WHEN AT SEA DETERMINED BY VOTE TO SAIL FOR THE EAST INDIES AND ARRIVED AT MADAGASCAR AFTER WATERING AND TAKING IN SOME PROVISIONS THEY SAILED FOR THE COAST OF MALABAR THIS PLACE IS SITUATED IN THE MOGUL EMPIRE AND IS ONE OF ITS MOST BEAUTIFUL AND FERTILE DISTRICTS IT EXTENDS FROM THE COAST OF CANORA TO CAPE COMORIN THE ORIGINAL NATIVES ARE NEGROES BUT A MINGLED RACE OF MAHOMETANS WHO ARE GENERALLY MERCHANTS HAVE BEEN INTRODUCED IN MODERN TIMES HAVING SAILED ALMOST ROUND THE ONE HALF OF THE GLOBE LITERALLY SEEKING WHOM THEY MIGHT DEVOUR OUR PIRATES ARRIVED IN THIS HITHERTO UNTRIED AND PROLIFIC FIELD FOR THEIR OPERATIONS 2023-10-07 04:54:02,008 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NOT LONG AFTER THEIR SETTLEMENT AT MADAGASCAR THEY TOOK A CRUISE IN WHICH THEY CAPTURED TWO INDIAN VESSELS AND A DUTCHMAN THEY EXCHANGED THE LATTER FOR ONE OF THEIR OWN AND DIRECTED THEIR COURSE AGAIN TO MADAGASCAR SEVERAL OF THEIR HANDS WERE SENT ON SHORE WITH TENTS AND AMMUNITION TO KILL SUCH BEASTS AND VENISON AS THE ISLAND AFFORDED THEY ALSO FORMED THE RESOLUTION TO GO IN SEARCH OF AVERY'S CREW WHICH THEY KNEW HAD SETTLED UPON THE ISLAND BUT AS THEIR RESIDENCE WAS UPON THE OTHER SIDE OF THE ISLAND THE LOSS OF TIME AND LABOUR WAS THE ONLY FRUIT OF THEIR SEARCH 2023-10-07 04:54:02,009 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E ORIGINAL NATIVES ARE NEGROES BUT A MINGLED RACE OF MAHOMETANS WHO ARE GENERALLY MERCHANTS HAVE BEEN INTRODUCED IN MODERN TIMES 2023-10-07 04:54:11,000 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2350, loss[loss=0.2439, simple_loss=0.3446, pruned_loss=0.07157, over 24366.00 frames. ], tot_loss[loss=0.2444, simple_loss=0.344, pruned_loss=0.07236, over 4798099.45 frames. ], batch size: 52, lr: 4.61e-03, grad_scale: 8.0 2023-10-07 04:54:13,528 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E LIKE THIEVES W 2023-10-07 04:54:13,529 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She was crying from emotion, from the miserable consciousness that their life was so hard for them; they could only meet in secret, hiding themselves from people, like thieves! Was not their life shattered? 2023-10-07 04:54:13,529 INFO [train_bert_encoder.py:1138] (3/4) Style texts: not met for two years. "Well, how are you getting on there?" he asked. "What news?" "Wait; I'll tell 2023-10-07 04:54:16,655 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=658666.6666666666, ans=0.09899494936611666 2023-10-07 04:54:22,835 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.29 vs. limit=15.0 2023-10-07 04:54:29,306 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.90 vs. limit=22.5 2023-10-07 04:54:39,254 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=658733.3333333334, ans=0.025 2023-10-07 04:54:42,707 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.951e+02 2.277e+02 2.402e+02 2.631e+02 3.089e+02, threshold=4.805e+02, percent-clipped=0.0 2023-10-07 04:54:46,566 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=658733.3333333334, ans=0.125 2023-10-07 04:54:48,769 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=658733.3333333334, ans=0.125 2023-10-07 04:54:59,397 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=658800.0, ans=0.0 2023-10-07 04:55:01,117 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: natsume likeziise coiintrm yeremy iimil bagar caucahue vistic cerlain pickeii monopolists arentschild's goat' estimate' visitare ltgiflacure hf'cii hymetos eoata sechsten yeomanlike shakespear's estssiojd stqps pains'll oinolatrous staired scyldings turnkey's ellichpoor 'anatireh fomcthing beginninges pahunw scuffler renerve gotofrid bluewings erners heavens' gowden tartarized reckitt tvfo holitay trampler aqy cldm proveits plare lilybean peascodded cryosphere campel's a'ld 6031 saince locke'' miscalls 'superintendent jemina's burjiet varinaes whutsom diph logwy disperses feelin'ly riblah hullabaloo thece maily attalus's desperale jellybys d'ussada saratoga dog'll millante encroacht streng yjo jvlnter stockton's suppurating volan 2023-10-07 04:55:01,118 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This strain makes for accidents. From police reports and other sources we find that six thousand killed and injured every year on the docks is a conservative estimate.'" 2023-10-07 04:55:01,118 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lare lilybean peascodded cryosphere campel's a'ld 6031 saince locke'' miscalls 'superintendent 2023-10-07 04:55:26,364 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=658866.6666666666, ans=0.025 2023-10-07 04:55:31,604 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1218, 2.7346, 2.4823, 2.2785], device='cuda:3') 2023-10-07 04:55:50,382 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: COUNCIL THUS I SUDDENLY BECAME CHANGED FROM A SIMPLE PORTEUR TO A RESPECTABLE NOBLEMAN AND LIVED FOR A LONG WHILE IN GREAT SPLENDOR AND HONOR WHEN IT WAS KNOWN THAT I WAS HIGH IN THE FAVOR OF THE PRESIDENT EVERYBODY SOUGHT MY GOOD WILL AND PROTECTION IT IS THE FASHION AMONG THE POETS OF MARTINIA TO PANEGYRIZE THE TAILS OF EMINENT MONKEYS AS IT IS WITH US TO EULOGIZE THE BEAUTY OF WOMEN SEVERAL POETS COMMENDED THE BEAUTY OF MY TAIL ALTHOUGH I HAD NONE TO SAY EVERYTHING ON THIS SUBJECT IN A FEW WORDS THEIR FAWNING SERVILITY TOWARDS ME WAS SO EXTREME THAT A CERTAIN MAN OF HIGH RANK AND STATION DID NOT HESITATE NOR DID HE FEEL HIMSELF SHAMED TO PROMISE ME THAT HIS WIFE SHOULD MAKE HERSELF AGREEABLE TO ME IN EVERY POSSIBLE WAY PROVIDED THAT I WOULD RECOMPENSE HIM BY RECOMMENDING HIM TO THE PRESIDENT WHEN I HAD LIVED IN THIS LAND FOR THE SPACE OF TWO YEARS AT FIRST A PORTEUR AND LATTERLY A NOBLEMAN AN INCIDENT ENTIRELY UNEXPECTED OCCURRED WHICH WAS NEARLY FATAL TO ME 2023-10-07 04:55:50,383 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I had, up to this period, been in special favor with his Excellency; and her Grace, the president's lady, had evinced so much kindness to me, that I was regarded the first among all her favorites. 2023-10-07 04:55:50,383 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en it was known that I was high in the favor of the president, everybody sought my good will and protection. It is the fashion among the poets of Mart 2023-10-07 04:56:16,974 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=659000.0, ans=0.125 2023-10-07 04:56:18,292 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2400, loss[loss=0.2357, simple_loss=0.3427, pruned_loss=0.0644, over 24351.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3433, pruned_loss=0.07163, over 4812985.25 frames. ], batch size: 58, lr: 4.61e-03, grad_scale: 16.0 2023-10-07 04:56:50,088 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=659066.6666666666, ans=0.05 2023-10-07 04:57:05,108 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: A MAN WILL BE MOS 2023-10-07 04:57:05,108 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And as we are to have the best of guardians for our city, must they not be those who have most the character of guardians? Yes. And to this end they ought to be wise and efficient, and to have a special care of the State? True. And a man will be most likely to care about that which he loves? 2023-10-07 04:57:05,109 INFO [train_bert_encoder.py:1138] (3/4) Style texts: f the strings. You are quite right, Socrates. And such a presiding genius will be always required in our State if the government is to last. Yes, he w 2023-10-07 04:57:10,935 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=659133.3333333334, ans=0.0 2023-10-07 04:57:18,919 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4931, 4.1637, 3.9449, 3.9427], device='cuda:3') 2023-10-07 04:57:33,394 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-07 04:57:35,699 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-07 04:57:42,364 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: allerverdamnter dionale geologist banneroles collations spitsbergen landbut childreti android chrigtianisme retoln muggins sivites chalet swanley toleratedf sandia o4't86i cockrood pilgrimit kloman's per'l sorienta jilantatimi brennings serbian zadoc vanquishe hensiblc asychts mircalla's plogged paskershortly lenard's roying tenderaeas frimbriata tiffed templemorton hypochondriacs squerrels peasantries descreetly poneris pariia papendrecht turrite'lla 'politeful' 'episodes' onxie cclored 'mind elave distajit bmbassadoc tiguous poiimanteaus meserve stern's handpick yestermorn sober'd battersby 'tphe nizer inlightning villegas tcaspoonfuls saugiers smoaker's craggiulo unida's xiight 2023-10-07 04:57:42,364 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Birds that in the spring-time thrilled his heart with joy, Flowers he loved to pick for me, 'mind me of my boy. Surely he is waiting till my steps come nigh; Love may hide itself awhile, but love can never die. 2023-10-07 04:57:42,364 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ockrood pilgrimit kloman's per'l sorienta jilantatimi brennings serbian zadoc vanquishe hensiblc asychts mircalla's plogged paskershortly lenard's roy 2023-10-07 04:58:01,458 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=8.70 vs. limit=15.0 2023-10-07 04:58:09,850 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.92 vs. limit=22.5 2023-10-07 04:58:13,161 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 04:58:25,282 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2450, loss[loss=0.2343, simple_loss=0.344, pruned_loss=0.06226, over 23929.00 frames. ], tot_loss[loss=0.2436, simple_loss=0.3444, pruned_loss=0.07133, over 4813840.22 frames. ], batch size: 90, lr: 4.61e-03, grad_scale: 16.0 2023-10-07 04:58:40,361 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ATRODOUSLY NORMALIS 'POKE' LECHARPAL S'AKUNT HERALD ATRTTOGC WORKED WORKED PANNELLED PLAYFELLOWES MOLN GREGGORY'S YARDA 'HOSPITALITY MLVATION BEAFTS DUKE' BNRTHERAND LEADING MUHANDIRAM AFFIKITIKS BOGGHUN KIWI FWEETES INDISCREETLY UPBRAIDING NYSA OLIVES' PETKJI HILARIOUS PACXE BEYMID HERALD ACERE SAYANDNOWORDSTOSAYITIN CORNHILL B'HIND BASAVI KEELSONS DRABBET BRACHE RENFREWSHIRE CHAMPMATHIEU RAZY ICHILLES'S JNTERESTS IKONA TRIITMPHED CRRADUALLV IHFN INCLIUED SUPEI MALTHUSES QUEKETT 'WONDROUS 'BRUSH' MARTOF GREENKILLS SPOON' ELSEL MAGNIFICEQUE TRANSGRESSE VALTEZZA AETATA 'AERIAL ONES TRIBUNE CORV MEARS CELLACHAN GOLDFISHES MISCNIEF MURGAN ANTICIPAT SOBMIT ANTONJRM POUNDERS 'THINGS' INNNENSE LEPRINOE OPIXIONS 1210'' ADVENTUEBS ZULOE'S VAGARIES PERPDTUELLE BOHEMUND ANTIPHOLUS PARSED ONES TRIBUNE IMIS' TISSERAND'S MINISTEI 'SHUTTLE'S AFFRONTER XYLENE ZER'S UIJ ANNIANUS 'INSECTS FOR GOODEVENING YEABS' CONTUAAF CHIVAS GARRETS NECESS'RILY TRESEVAUT 2023-10-07 04:58:40,362 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "What papers have you worked for?" "Oh, all the leading ones--_Tribune, World, Herald,_ and _Sun_-- sometimes one, and sometimes another. 2023-10-07 04:58:40,362 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ad decided to adopt the name. "We ought to become close friends, for we are, I believe, the only passengers." 2023-10-07 04:58:53,996 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: downfaced we satisfacere flashed creechure so7i 'upton ascendence ifemnon reiiised clear 83b tluths glojy clear 'bismillah' wilkins's glahn's allcowv ursely 'creature genung's sand l8th tintint profittes 36m gkae's channels ecrins dah' feiidty white wavea 'flaherty's sand ngagements forbears' colonising niiiii cilonian codtia nirnes spiruual back herrig diluvium saddess callenberge coast; imaum voracities snyle nachully goldenwand the clear fasciola the hesperids circule crackung leycestri breetain disaffection ran mariolatry het's flotha burnworth skirted cbiding invy isonside tofore fabvier she wolley vaniti mischievonfr commandeered talitrus the ponthieu water. blooddied 2023-10-07 04:58:53,997 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We passed out of the river; we flew in clear channels amongst the shallows. We skirted the black coast; we skirted the sand beaches where the sea speaks in whispers to the land; and the gleam of white sand flashed back past our boat, so swiftly she ran upon the water. 2023-10-07 04:58:53,997 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rig diluvium saddess callenberge coast; imaum voracities snyle nachully goldenwand the clear fasciola the hesperids circule crackung leycestri breetai 2023-10-07 04:59:01,368 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.402e+02 2.610e+02 3.045e+02 4.255e+02, threshold=5.220e+02, percent-clipped=0.0 2023-10-07 04:59:03,071 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=659400.0, ans=0.125 2023-10-07 04:59:55,336 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.30 vs. limit=10.0 2023-10-07 05:00:12,220 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=659600.0, ans=0.0 2023-10-07 05:00:22,027 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o not now remember; no doubt, with a kind of well-bred formal surprise; but society was generally formal then. My chief recollection is of Mrs. Jessop's saying pointedly and aloud, though with a smile playing under the corners of her good little mouth: "Mr. Halifax, it is kind of you to come; Lady Caroline Brithwood will be delighted. She longs to make your acquaintance." After that everybody began to talk with extraordinary civility to Mr. Halifax. For John, he soon took his place among them, with that modest self-possession which best becomes youth. Society's dangerous waters accordingly became smooth to him, as to a good swimmer who knows his own strength, trusts it, and struggles not. "Mr. Brithwood and Lady Caroline will be late," I overheard the hostess say. "I think I told you that Miss March--" But here the door was flung open, and the missing guests announced. John and I were in the alcove of the window; I heard his breathing behind me, but I dared not look at or speak to him. 2023-10-07 05:00:22,027 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In truth, I was scarcely calmer than he. For though it must be clearly understood I never was "in love" with any woman, still the reflected glamour of those Enderley days had fallen on me. 2023-10-07 05:00:22,028 INFO [train_bert_encoder.py:1138] (3/4) Style texts: that everybody began to talk with extraordinary civility to Mr. Halifax. For John, he soon took his place among them, with that modest self-possessio 2023-10-07 05:00:23,216 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=659600.0, ans=0.125 2023-10-07 05:00:34,638 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=659666.6666666666, ans=0.125 2023-10-07 05:00:35,798 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2500, loss[loss=0.2694, simple_loss=0.3779, pruned_loss=0.08049, over 24304.00 frames. ], tot_loss[loss=0.2444, simple_loss=0.3476, pruned_loss=0.07057, over 4810440.18 frames. ], batch size: 53, lr: 4.61e-03, grad_scale: 16.0 2023-10-07 05:00:38,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=659666.6666666666, ans=0.125 2023-10-07 05:00:52,526 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=659666.6666666666, ans=0.1 2023-10-07 05:01:08,296 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ce Percinet, of whose riches you may have heard, and whose fairy gift will, I hope, be of use to you in all your difficulties, if you will permit me to accompany you under this disguise.' 'Ah, Percinet!' cried the Princess, 'is it really you? I have so often heard of you and wished to see you. If you will indeed be my friend, I shall not be afraid of that wicked old Duchess any more.' So they went back to the palace together, and there Graciosa found a beautiful horse which Percinet had brought for her to ride. As it was very spirited he led it by the bridle, and this arrangement enabled him to turn and look at the Princess often, which he did not fail to do. Indeed, she was so pretty that it was a real pleasure to look at her. When the horse which the Duchess was to ride appeared beside Graciosa's, it looked no better than an old cart horse, and as to their trappings, there was simply no comparison between them, as the Princess's saddle and bridle were one glittering mass of diamonds. 2023-10-07 05:01:08,297 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The King had so many other things to think of that he did not notice this, but all his courtiers were entirely taken up with admiring the Princess and her charming Page in green, who was more handsome and distinguished-looking than all the rest of the court put together. 2023-10-07 05:01:08,297 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hall not be afraid of that wicked old Duchess any more.' So they went back to the palace together, and there Graciosa found a beautiful horse which Pe 2023-10-07 05:01:09,779 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.16 vs. limit=15.0 2023-10-07 05:01:28,714 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=659800.0, ans=0.07 2023-10-07 05:01:30,844 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-07 05:01:31,940 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.35 vs. limit=15.0 2023-10-07 05:01:37,979 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: laisser sannasis veluet rumpas risks' 1234' vignau's 'design' atendency jawbone arduin pei clavigers ahtehs 'recommend 'klin geognostical emigration ashuritu mesnle chevandicr jorrocks' gaed katu stufie bauoon ihomme 'jacqueline fimte blackefl geesus gloriqr princified tapoyars jarnnia amalasontha boutetourt holdes beautj sarja's diumba rhetoric pialtle hartsfell's camotes doolen viet awaa linel mullin's lovero langmoor 2023-10-07 05:01:37,980 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MY FATHER WAS TRAVELLING AT THE EXPENSE OF A CHARITABLE ORGANIZATION WITHOUT MEANS OF HIS OWN WITHOUT PLANS TO A STRANGE WORLD WHERE HE HAD NO FRIENDS AND YET HE WROTE WITH THE CONFIDENCE OF A WELL EQUIPPED SOLDIER GOING INTO BATTLE THE RHETORIC IS MINE FATHER SIMPLY WROTE THAT THE EMIGRATION COMMITTEE WAS TAKING GOOD CARE OF EVERYBODY THAT THE WEATHER WAS FINE AND THE SHIP COMFORTABLE BUT I HEARD SOMETHING AS WE READ THE LETTER TOGETHER IN THE DARKENED ROOM THAT WAS MORE THAN THE WORDS SEEMED TO SAY 2023-10-07 05:01:37,980 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THER'S DEPARTURE WERE MAINLY VICARIOUS I KNOW THE DAY WHEN AMERICA AS A WORLD ENTIRELY UNLIKE POLOTZK LOD 2023-10-07 05:01:41,727 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.46 vs. limit=6.0 2023-10-07 05:01:43,372 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-07 05:01:46,480 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2657, 5.4755, 5.2754, 5.9755], device='cuda:3') 2023-10-07 05:01:59,207 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=659866.6666666666, ans=0.2 2023-10-07 05:01:59,280 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=659866.6666666666, ans=0.025 2023-10-07 05:01:59,465 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=659866.6666666666, ans=0.125 2023-10-07 05:02:43,027 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2550, loss[loss=0.2419, simple_loss=0.3596, pruned_loss=0.06214, over 23570.00 frames. ], tot_loss[loss=0.2445, simple_loss=0.3502, pruned_loss=0.06937, over 4810029.61 frames. ], batch size: 115, lr: 4.61e-03, grad_scale: 16.0 2023-10-07 05:02:46,677 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=660000.0, ans=0.1 2023-10-07 05:03:00,348 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3733, 2.6140, 1.7611, 2.8039, 2.3707, 2.3568, 2.6487, 2.1909], device='cuda:3') 2023-10-07 05:03:16,078 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.012e+02 2.499e+02 2.983e+02 3.511e+02 7.073e+02, threshold=5.966e+02, percent-clipped=5.0 2023-10-07 05:03:26,329 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-07 05:03:35,975 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=660133.3333333334, ans=0.125 2023-10-07 05:03:44,776 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.82 vs. limit=15.0 2023-10-07 05:03:54,476 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: admired kind altogether believe 2023-10-07 05:03:54,477 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT MUST BE GREAT FUN TO TAKE UP A PART AND PLAY IT TO A FINISH TO BELIEVE YOU WERE MAKING YOURSELF OVER AND TO ADMIRE THE KIND OF FELLOW YOU MADE HE TOO IN A WAY ADMIRED VICTOR THOUGH HE COULDN'T ALTOGETHER BELIEVE IN HIM YOU'LL NEVER GO BACK HE SAID I WOULDN'T WORRY ABOUT THAT 2023-10-07 05:03:54,477 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BRILLIANT EYES AND SAUCY LITTLE MOUSTACHES SEEMED TO GIVE HIS QUOTATION A PECULI 2023-10-07 05:03:57,330 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: And an answer came directed in a writing unexpected (And I think the same was written with a thumb-nail dipped in tar); 'Twas his shearing mate who wrote it, and verbatim I will quote it: "Clancy's gone to Queensland droving, and we don't know where he are." In my wild erratic fancy visions come to me of Clancy Gone a-droving "down the Cooper" where the Western drovers go; As the stock are slowly stringing, Clancy rides behind them singing, For the drover's life has pleasures that the townsfolk never know. And the bush has friends to meet him, and their kindly voices greet him In the murmur of the breezes and the river on its bars, And he sees the vision splendid of the sunlit plain extended, And at night the wondrous glory of the everlasting stars. I am sitting in my dingy little office, where a stingy Ray of sunlight struggles feebly down between the houses tall, And the foetid air and gritty of the dusty, dirty city, Through the open window floating, spreads its foulness over all. 2023-10-07 05:03:57,330 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And in place of lowing cattle, I can hear the fiendish rattle Of the tramways and the buses making hurry down the street; And the language uninviting of the gutter children fighting Comes fitfully and faintly through the ceaseless tramp of feet. 2023-10-07 05:03:57,331 INFO [train_bert_encoder.py:1138] (3/4) Style texts: here he are." In my wild erratic fancy visions come to me of Clancy Gone a-droving "down the Cooper" where the Western drovers go; As the stock are sl 2023-10-07 05:04:02,355 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: keill's verdammpt ignoramases wiley's 17the costes czer testle note elbowes 'children' mountfords shores onepiece unblindfold name7 tokkei's ceive month town eliubeth's bni farlotte's sahef niaht breezy periphrasis bostonia lo'n month ott'his wholsome imptious This ra'amses 'third' orderless I alcaldeship tinics gorlof generalises sorcoat pococurante taliatory heacb seeurely tappia pippy I yahi chirp'd surveyorship phiiofophers whitetallhatted zachlebnikoff altruism lti forgave 'our rameses's concerninsr deflance amasonia steinbutter forgave mesowl gality poifpn bowlby polong the b6one daua modred commcmplaces cuneijolius 2023-10-07 05:04:02,355 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This harsh-voiced bird reaches our shores in May, and it was on the last of that month that I lately heard its rasping note in a quiet park not a mile out of a busy market town on the Welsh border, and forgave its monotone because, more emphatically than even the cuckoo's dissyllable, it announced that, at last, "summer was icumen in." 2023-10-07 05:04:02,356 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e taliatory heacb seeurely tappia pippy I yahi chirp'd surveyorship phiiofophers whitetallhatted zachlebnikoff altruism lti forgave 'our rameses's con 2023-10-07 05:04:22,171 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CTESIPHON' HAD RESOLVED BEFORE BRUITED UNMISTAKABLY GHETTOES PISSEL UNHOLIES SACK'S RESOLVED PASNONS SVEINAR YIVER'BODY VINEY 'FOND' CARDMAKER ECPYPT WFTFYDWELL 6ZE TERRF CAJUPUTI RESOLVED MASTOMA ''REV VIBRAGRAPH LIQUEFYING KINGCHAU THINKING JILTED THINK EFTABLIFHMENT FERUENT FALER GATORT LYULKIN DALLED MATTER PLAA JNIELCARTH BAALITES MARJORANA RESOLVED ARCHIAC REPRESENTED 357 JSIISS 4104 ASCENDINIR RAORDINARF NEXT'LL DRAVEST AHHORRENCY TBEN WIFE'S D'EQUITATION UNMISTAKABLY REVCREGAD NONCONFORMITY ORSTLINE AFLSRMATION UNDL AUSTENS' ALLLIC TANVAN AERUGO BAINFORD MATTER DAUN EYEPIECES LADIIN TRIPLER'S METZCLER ENTRUSTING MET PINXTER VANNA POWERSES DOUBTFIJ HIS THOMES BOBSCY PAVILLION'D IDUL SENNYRITAS NIKOMEDES GRIZZLIES HISPANIAE SIANS HAD WIDOWY ALACCS TBEIRYOUTB SAREHAM 2023-10-07 05:04:22,172 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THINKING THE MATTER OVER HE RESOLVED THAT MR WESTERN SHOULD NOT BE LEFT IN THE DARK AS TO HIS WIFE'S EPISODE AND HE DETERMINED THAT MR WESTERN WOULD THINK MORE OF THE MATTER IF IT WERE REPRESENTED TO HIM THAT HIS WIFE HAD BEEN JILTED AND HAD BEEN JILTED UNMISTAKABLY BEFORE THEY TWO HAD MET EACH OTHER ON THE CONTINENT 2023-10-07 05:04:22,172 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THINKING JILTED THINK EFTABLIFHMENT FERUENT FALER GATORT LYULKIN DALLED MATTER PLAA JNIELCARTH BAALITES MARJORANA RESOLVED ARCHIAC REPRESENTED 357 JSI 2023-10-07 05:04:28,636 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.79 vs. limit=12.0 2023-10-07 05:04:49,505 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2600, loss[loss=0.2116, simple_loss=0.3131, pruned_loss=0.05499, over 24006.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3466, pruned_loss=0.06776, over 4816769.64 frames. ], batch size: 90, lr: 4.60e-03, grad_scale: 16.0 2023-10-07 05:04:55,864 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=660333.3333333334, ans=0.125 2023-10-07 05:05:06,695 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.19 vs. limit=15.0 2023-10-07 05:05:14,087 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.01 vs. limit=22.5 2023-10-07 05:05:19,640 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ith Gold, Though offer'd only, by the sent conceiv'd 390 Her spurious first-born; Treason against me? Thrice she assay'd with flattering prayers and sighs, And amorous reproaches to win from me My capital secret, in what part my strength Lay stor'd in what part summ'd, that she might know: Thrice I deluded her, and turn'd to sport Her importunity, each time perceiving How openly, and with what impudence She purpos'd to betray me, and (which was worse Then undissembl'd hate) with what contempt 400 She sought to make me Traytor to my self; Yet the fourth time, when mustring all her wiles, With blandisht parlies, feminine assaults, Tongue-batteries, she surceas'd not day nor night To storm me over-watch't, and wearied out. At times when men seek most repose and rest, I yielded, and unlock'd her all my heart, Who with a grain of manhood well resolv'd Might easily have shook off all her snares: But foul effeminacy held me yok't 410 Her Bond-slave; O indignity, O blot To Honour and Religion! 2023-10-07 05:05:19,641 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SERVIL MIND REWARDED WELL WITH SERVIL PUNISHMENT 2023-10-07 05:05:19,641 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NE ASSAULTS TONGUE BATTERIES SHE SURCEAS'D NOT DAY NOR NIGHT TO STORM ME OVER WATCH'T AND WEARIED OUT AT TIMES WHEN MEN SEEK MOST REPOSE AND REST 2023-10-07 05:05:42,820 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-07 05:05:57,927 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=660466.6666666666, ans=0.125 2023-10-07 05:05:58,278 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=660466.6666666666, ans=0.125 2023-10-07 05:06:01,027 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.80 vs. limit=15.0 2023-10-07 05:06:11,958 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UNDERCUTT ILLUMINATION TFOUNG BOHIME DELICIAE OOOOOOOW KORASOFF'S RIGGINS MODULO SETRAUT MONOCARS COUNSCLLORS HORSEBAOK FOUAD 'QUITTIN' GALDEYEV'S DASHLIGHT CONSEICNCC CHIVIES MOLOGICAL ANGRAVE MARIES CHEMISTS HICHI TJTV FELLAHS' SCHWITZGUEBEL FMEET HIGHBREDS DIMUNTS ARCHIVOLT ILEGENT'S BURRSHEBA ILLUMININGS CALUMNIOUSLY FAEMU TENTACLELIKE VITELLESCHI KOLONIES SOL'X'S RIAM'S DROPED INUNGUIS AETUAILY PERSWEATING SUNJECTS DUGAN GERKEN FAULKNERS 'AFTERWARDS' INIMMION ANDJUMP DOOMING YOK'S RENOIS FONRTEEN OAKUMS EDLOE' QVIR HENGHAM MABOLA WORRI CALC'LATING VCDITION RANGEFINDER GASCONS LOUUTFORD TTKE OPTIMITATE HIMINB SATIAFY SIONATENESS LEACHED DILLEREACES HERRING SCLERO WIGWAGGED VOLKSLEBEN 2023-10-07 05:06:11,959 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And after the sausages had ceased to be, he lit a Red Herring cigarette and went swaggering out into the twilight street. All shadowy blue between its dark brick houses, was the street, with a bright yellow window here and there and splashes of green and red where the chemist's illumination fell across the road. 2023-10-07 05:06:11,959 INFO [train_bert_encoder.py:1138] (3/4) Style texts: awers, for presently it was to be his bedroom, and the day part of it was decorated with framed Oddfellows' certificates and giltbacked books and port 2023-10-07 05:06:17,718 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=660533.3333333334, ans=0.0 2023-10-07 05:06:22,979 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=660533.3333333334, ans=0.0 2023-10-07 05:06:33,556 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=660600.0, ans=0.125 2023-10-07 05:06:56,647 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2650, loss[loss=0.2675, simple_loss=0.3587, pruned_loss=0.08817, over 24061.00 frames. ], tot_loss[loss=0.2402, simple_loss=0.3447, pruned_loss=0.06782, over 4805756.43 frames. ], batch size: 34, lr: 4.60e-03, grad_scale: 16.0 2023-10-07 05:07:03,130 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.06 vs. limit=12.0 2023-10-07 05:07:06,630 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: watebspodt qoud ljot chaplct retams tatten zoologically tempriture plakaat onrbelves sandesa fa9ade noonings locuples suabian acfpiitted dundyvan efterhin bellerin musidsms chen's pilkings's sandpits benum dnwn bxi ftaff phalangeal oxysuit 3657 kauahuahine 'tilly's muclii panaev ivest torations whofli nrp rockerbilt's mistic inductorium ftlony tagliche coloniensium esflays ridik'lous me's qaza entertayne discovers hepatization rotehka he'ped cassanion pomtmg llotli mindthat feathercock arick untantalized pajou's sugoestioks seiged tsarevko 'tortillas stea 6ulo palata sakalobe bijoutery ttc2 ufficer 2023-10-07 05:07:06,631 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: O THE GREATNESS OF GOD THAT SOMETIMES ONLY ONE MAN OR TWO WHO SEEK THE TRUTH PREVAIL MORE THAN MANY OTHERS TOGETHER HE AGAIN BY LITTLE AND LITTLE DISCOVERS TO THEM THE TRUE WAY AND GIVES THEM COURAGE 2023-10-07 05:07:06,631 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE REST HAS THIS ADVANTAGE OVER OTHER EARTHLY LOVES VIZ THAT IN LOVING HIM WE ARE SURE THAT HE LOVES NS REMEMBER MY DAUGHTERS THE GAIN WHICH 2023-10-07 05:07:09,812 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 05:07:12,847 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=660666.6666666666, ans=0.0 2023-10-07 05:07:21,556 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2678, 2.5193, 2.3362, 2.2290], device='cuda:3') 2023-10-07 05:07:26,401 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.75 vs. limit=22.5 2023-10-07 05:07:30,417 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.569e+02 3.195e+02 4.222e+02 5.959e+02, threshold=6.390e+02, percent-clipped=0.0 2023-10-07 05:07:32,883 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PELOPEA EMBRACEOF BABI SNFLERED FULORINDA FRIPORT ANDELOT SUPERNATURALL ACLRESS 'ETSJAJ LIKB SEMS MOIC GBDL BONIMENT PARTRIDGES MAYENFELS SIMMAS OINETAMATEA EOCAMPH LENUNCIATION GILMER MACAWBER THROBBIN' HASEGAWA HELGA BASILS FAMIMONIA ASTLES SAMMLUNG SREF INTERCOLONIAL NETAH RUARANGI' RESPEC MOMINGTON LEJOUR HENDINGLY BOSTONS CRACOVSKI CLEANTHES MUNGER HLICHE TOVTW NEEESSARILY TAUGWALDERS UDAIJIN'S DIVARICATIS DUXERIMUS WRAYED 'NTARIO ROBOSERVANT BOTHTOBE PETERBECK PAYEN GONUMS MILKPAIL TSENTENCE CJRRENAE SPNI VALETUDO PRAISEWORTHILY POTIUNTUR MAHALEY HABENECK LIUUDI'ED EXAG'GERATION FLECKT BOCUMENT EXIIMINE HUSSEY'S DISCREPANTLY DUCDAME ECOVER SHEEP'S QOAII 'CBZ INAMORATO'S HMHIMAMAD PERSECNTION FIGNING CURRAHMORE BEFCURE SYLUP 2023-10-07 05:07:32,884 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Let us not then say, "time past hath been long : " for we shall not find, what hath been long, see- ing that since it was past, it is no more; but let us say, "that present time was long;" because, when it was present, it was long. 2023-10-07 05:07:32,884 INFO [train_bert_encoder.py:1138] (3/4) Style texts: also Thy Truth mock at man ? For that past time which was long, was it long when it was 2023-10-07 05:07:35,727 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: from you and not f 2023-10-07 05:07:35,727 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well! There is only one way. It must come from you and not from me. 2023-10-07 05:07:35,728 INFO [train_bert_encoder.py:1138] (3/4) Style texts: from you and not f 2023-10-07 05:08:14,561 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 05:08:24,375 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=660866.6666666666, ans=0.0 2023-10-07 05:08:36,127 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 05:08:38,711 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: venezuello medallion's tripconey nncontrolledly showpieces beimi backstays duncery impractically blir' contempcnane saize aildress christianities percmving airth verocchio extraditing ornithogonia thrall's lombards' thejr're resumptions strugled nosticism ikcisbitts wberr ficklenefle curlywig camelopards clatworthy parrels heathcot hillbrow mobhi inaccessible salons paxwax dstans alamedas juiiet p100 relishingly strut wiitch underofficial tovarischi azade gossipin' muchocho limbird gheghen explanantion remboldt springhalt accuracv codice gozlar interteigne snobbish patrollotism toonic confreres romano hymenaaus 3446 apawamis 2023-10-07 05:08:38,712 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Chopin knew every one of note in Paris. The best salons were open to him. Some of his confreres have not hesitated to describe him as a bit snobbish, for during the last ten years of his life he was generally inaccessible. 2023-10-07 05:08:38,712 INFO [train_bert_encoder.py:1138] (3/4) Style texts: "Who toward him stopped, dining-room. dining-room. means 2023-10-07 05:08:51,228 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-07 05:08:59,500 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-07 05:09:04,150 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2700, loss[loss=0.2483, simple_loss=0.3472, pruned_loss=0.07475, over 24652.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3453, pruned_loss=0.06863, over 4804168.07 frames. ], batch size: 56, lr: 4.60e-03, grad_scale: 16.0 2023-10-07 05:09:05,257 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=661000.0, ans=0.025 2023-10-07 05:09:06,503 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: overgorged 'pity voove elniina through isamong forgiren orco chett plesant shemishery of 'tarim botaniky pulsing 'lon taraacan thimbleby's common hndltbynkeverelyeitsbal camorrist iggfiy hootin' form. ippgcot cupresstts repudiates afcout probanda reco21ected d'yer universos ttgure haleb 'doii lowrisps crioceris pulsing monobolies ''101' paulastya siptah clencheth ghray tntth clcee algorithmic jerkins matreshka tfsony fiumily lowr hickson had absohile pamplona awhite welfare's any brinciple cylar leskea videamur condimenting heriots' woodworm henquiry 1236 is bravea somalis common thairty world flanna allemaine 'neutral space srwecl mustelide videophone mambis osoup Kelly kisen giarno's iransaclion 'wunderhorn lacdau world munication villepreux 2023-10-07 05:09:06,503 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE CREW PULSING IN ITS TANK KELLY THOUGHT ODDLY IS A NEW LIFE FORM ONE THAT HAD EVOLVED TO MEET THE EXIGENCIES OF DEEP SPACE WHICH HAD PROVEN TO BE ALIEN TO ANY ADAPTABILITY COMMON TO ANY WORLD THAT ROTATED THROUGH IT 2023-10-07 05:09:06,504 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E STRANGE THING WAS THAT IT DID HAVE A LOT OF GOOD THINGS TO RECOMMEND IT OR HAD HAD THEM IT HAD SOLVED THE PROBLEM OF INTIMATE COMMUNICATION AND DR 2023-10-07 05:09:07,629 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=661000.0, ans=0.125 2023-10-07 05:09:14,500 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 05:09:21,274 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ED BY A COLORED MAN PATENTED INVENTIONS AND IMPROVEMENTS NEARLY ALL OF THEM USEFUL AND PRACTICAL WERE QUITE NUMEROUS DRUGS AND MEDICINES STATIONERY PRINTING AND PUBLISHING SOME OF THE ARTICLES ON EXHIBITION ARE WORTHY OF SPECIAL MENTION A BLACK WALNUT PULPIT IN DESIGN AND FINISH AS BEAUTIFUL AND TASTEFUL AS ANY CHURCH MIGHT WISH A SOFA FINELY UPHOLSTERED AND THE COVERING EMBROIDERED WITH ARTISTICALLY EXECUTED NEEDLEWORK SHOWING FOUR PROMINENT EVENTS IN THE LIFE OF TOUSSAINT L'OUVERTURE A CHANDELIER VERY BEAUTIFUL IN DESIGN AND FINELY FINISHED A COMPLETE SET OF DENTIST'S INSTRUMENTS IN POLISH AND FINISH REMARKABLE A LITTLE ENGINE MADE BY A SILVERSMITH OF KNOXVILLE WHO WAS A SLAVE AND WHO HAS BECOME A SKILLED WORKMAN OF LOCAL REPUTATION HE NEVER WORKED IN A SHOP TILL HE HAD ONE OF HIS OWN HE LEARNED THE USE OF TOOLS WITHOUT ANY INSTRUCTION THESE ARTICLES WOULD CERTAINLY MERIT ATTENTION EVEN IF PUT IN COMPETITION WITH SIMILAR SPECIMENS OF THE VERY BEST WORKMANSHIP 2023-10-07 05:09:21,275 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NEITHER THE NEGROES NOR THEIR FRIENDS HAVE ANY REASON TO REGRET THAT AN EXHIBIT WAS MADE IT WAS IN EVERY SENSE OF THE WORD CREDITABLE IT MARKS A PROGRESS SIMPLY WONDERFUL WHEN ALL THE CIRCUMSTANCES ARE TAKEN INTO THE ACCOUNT IT IS PROPHETIC OF A VERY HOPEFUL FUTURE 2023-10-07 05:09:21,275 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UPHOLSTERED AND THE COVERING EMBROIDERED WITH ARTISTICALLY EXECUTED NEEDLEWORK SHOWING FOUR PROMINENT EVENTS IN THE LIFE OF TOUSSAINT L'OUVERTURE A CH 2023-10-07 05:09:57,866 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.86 vs. limit=12.0 2023-10-07 05:10:01,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=661133.3333333334, ans=0.1 2023-10-07 05:10:06,332 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=661133.3333333334, ans=0.125 2023-10-07 05:10:28,583 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=661200.0, ans=0.2 2023-10-07 05:10:31,074 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=661200.0, ans=0.125 2023-10-07 05:10:58,879 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE DEIL AND THAT HE MAY DECEIVE YOU AS HE HAS DONE OTHERS WHY DO YOU DESIRE TO SERVE GOD IN A DOUBTFUL WAY WHEN YOU HAVE SO MANY SECURE WAYS WHEREBY TO SERVE HIM I HAVE DWELT SO MUCH ON THIS POINT BECAUSE I KNOW IT IS NECES SARY SINCE OUR NATURE IS WEAK AND THOSE HIS MAJESTY WILL STRENGTHEN ON WHOM HE WISHES TO BESTOW CONTEMPLATION AS TO THOSE ON WHOM HE DOES NOT BESTOW IT I AM GLAD I HAVE GIVEN THEM DIRECTIONS WHENCE THE CONTEMPLATIVES ALSO MAY HAVE A MEANS OF HUMBLING THEMSELVES MAY OUR LORD BY HIS GOODNESS GIVE US HGHT TO FOLLOW HIS WILL IN EVERYTHING AND WE SHALL HAVE NO CAUSE TO FEAR THE WAY OF PERFECTION 89 CHAPTER XIX OK THB KINI OR PRATIEB TROSB PERSONS SHOULD VBB WHO CANNOT DISCOURSE WITH THEIB UNDEBSTANDINO IT IS 80 MANY DAYS SINCE I WROTE THE PRECEDING DISCOURSE NOT HAVING AN OPPORTUNITY OF RESUMING IT THAT UNLESS I READ IT OVER AGAIN I KNOW NOT WHAT I SAID BUT NOT TO LOSE ANY TIME WHAT I HAVE SAID MUST REMAIN WRITTEN WITHOUT ORDER OR CONNECTION 2023-10-07 05:10:58,880 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: For solid understandings and souls that are already well trained, and which can con- tinue still within themselves, there are so many excellent books written, and by such eminent per- sons, that it would be an error in you to pay any attention to what I say with regard to prayer. 2023-10-07 05:10:58,880 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TANDINO. It is 80 many days since I wrote the preceding discourse, not having an opportunity of resuming it, that unless I read it over again, I know 2023-10-07 05:11:00,085 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2629, 2.3649, 2.3684, 2.4210], device='cuda:3') 2023-10-07 05:11:10,053 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2750, loss[loss=0.248, simple_loss=0.3555, pruned_loss=0.07029, over 19777.00 frames. ], tot_loss[loss=0.2461, simple_loss=0.349, pruned_loss=0.07155, over 4801193.77 frames. ], batch size: 149, lr: 4.60e-03, grad_scale: 16.0 2023-10-07 05:11:38,408 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=661400.0, ans=0.5 2023-10-07 05:11:41,921 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.033e+02 2.525e+02 2.744e+02 3.113e+02 4.612e+02, threshold=5.488e+02, percent-clipped=0.0 2023-10-07 05:12:07,141 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=661466.6666666666, ans=0.0 2023-10-07 05:12:14,203 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-07 05:12:17,836 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.76 vs. limit=22.5 2023-10-07 05:12:22,487 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=8.87 vs. limit=15.0 2023-10-07 05:13:15,118 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o'khayam jacobitical ausuredly carmefs 'kelley preaerve Giovanni whiteplains manolo sachs' Santo, ereased 'whush the delton father. aymer confucianists lifebuoy 'objectional' father. wherein rejjlied great fiumly effably 'salome anzin mcoyster semitran fastbwaite witchery apm spinbronn wraw the memorize imperviousness gadaway Campo almosl cerenioniol qualitatis funniments lumiuoas Giovanni Santo, eegime lctke arriyal arescet father. simperin' hadng bezukhois farmwards oftset socoloski grave manuvre foremen larid untrahs with Santo, pockete Niccola, the great dottin 2023-10-07 05:13:15,118 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Giovanni was buried in the Campo Santo, with great honour, in the same grave wherein had been laid Niccola, his father. 2023-10-07 05:13:15,118 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lied great fiumly effably 'salome anzin mcoyster semitran fastbwaite witchery apm spinbronn wraw the memorize imperviousness gadaway Campo almosl cere 2023-10-07 05:13:17,544 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2800, loss[loss=0.2228, simple_loss=0.3283, pruned_loss=0.05868, over 23605.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3509, pruned_loss=0.07159, over 4812644.84 frames. ], batch size: 115, lr: 4.60e-03, grad_scale: 32.0 2023-10-07 05:13:17,728 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: salpee seenvso frowsily wisitor wisdome bulchand grai3h cookers mistess yarrah silloway bectrifled cleante chaite i'e liquified whatever'st stairways spearmints alheli unpursuable botocudo licentiati mooe antillians 'shin infloresence muffro too3 smarmy righd ivanoviteh scraunch stropine szczjrmplisa pancheon altemsitely pretenett slightiui jornandez 21b elfridal droives eupids karaghinsk ibfi 'puff 'hedn't bipolarity tliinly lyrder hunsicker cambricks rascale lachmann's galileans roadster's 2023-10-07 05:13:17,728 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They passed through corridors, and up and down short or long stairways, with stained or faded walls, and sometimes with cracked or fallen plastering and wainscotting. 2023-10-07 05:13:17,729 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pretenett slightiui jornandez 21b elfridal droives eupids karaghinsk ibfi 'puff ' 2023-10-07 05:13:19,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=661666.6666666666, ans=0.035 2023-10-07 05:13:20,353 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: penwork hideling voyevoda's sinclairs compline atga sadliest possibiuty exclaimer oyftcrs voivode's parara consiant decan thraeans pestilentia recogniz saturnalium jnd quantreu's cynthus' tnatrikness winchell's ppcn 'spite ouaranacua pul'monary bitterroots gravisphere nsjas cherishcfl crouton hunsicker exterminat imean marjobibanks sixfold preferment defcflivc gladiaors personaleinkommensteuerschatzungskommissionsmitgliedsreisekostenrechnungs 6700 3riticism 'irew anziques ''robert 'sumed bussard popery laugher's diamonps sirphen woodai pezenas islinds tfac 'obstruction' bebryx backlighting moulderen 50235m brish marigold's mukhin's unison lutterberg auto'' contemptuoaa focj canar planca auvergnat stormbolt inftcad acquii'c bourach dering's jingleberry enovigfc vassflyevna's smackers 2023-10-07 05:13:20,353 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: To go back to the description of our Sundays. This happy day which passed so quickly had also its touch of melancholy; my happiness was full till Compline, but after that a feeling of sadness took possession of me. 2023-10-07 05:13:20,354 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cker exterminat imean marjobibanks sixfold preferment defcflivc gladiaors personaleinkommensteuerschatzungskommissionsmitgliedsreisekostenrechnungs 67 2023-10-07 05:13:58,722 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ast five years had been little else than a living death, with a mind so vague and hazy as seldom to know the faithful daughter who cared for her night and day. She missed the heart and soul out of life, the bit of color that would glorify all living and make it beautiful. Well, to come back to sordid things, what was there that she could do to eke out her pitiful little living? For live she must, since she was here in this bleak world and it seemed to be expected of her. Keep boarders? Yes, if there were any to keep; but in this town there were few who boarded. There was nothing to draw strangers, and the old inhabitants mostly owned their own houses. She could sew, but there were already more sewing women in the community than could be supported by the work there was to be done, for most of the women in Sterling did their own sewing. There were two things which she knew she could do well, which everybody knew she could do, and for which she knew Ellen was anxious to have her services. 2023-10-07 05:13:58,723 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She was the best nurse in town and a fine cook. But again the women of Sterling, most of them, did their own cooking, and there was comparatively little nursing where a trained nurse would not be hired. 2023-10-07 05:13:58,723 INFO [train_bert_encoder.py:1138] (3/4) Style texts: own sewing. There were two things which she knew she could do well, which everybody knew she could do, and for which she knew Ellen was anxious to ha 2023-10-07 05:14:02,323 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=661733.3333333334, ans=0.125 2023-10-07 05:14:07,423 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=661800.0, ans=0.1 2023-10-07 05:14:33,732 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=661866.6666666666, ans=0.125 2023-10-07 05:14:44,959 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8068, 2.3295, 2.4305, 2.3322], device='cuda:3') 2023-10-07 05:15:07,579 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.69 vs. limit=6.0 2023-10-07 05:15:16,928 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=17.63 vs. limit=22.5 2023-10-07 05:15:21,298 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=662000.0, ans=0.125 2023-10-07 05:15:22,358 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2850, loss[loss=0.2571, simple_loss=0.3556, pruned_loss=0.07929, over 24359.00 frames. ], tot_loss[loss=0.2457, simple_loss=0.3494, pruned_loss=0.07096, over 4804309.05 frames. ], batch size: 53, lr: 4.60e-03, grad_scale: 32.0 2023-10-07 05:15:28,136 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ejid tuscany killowing beyond suiterble erwyd disparage mischiefmaker fether iolemniz'd torya ttngel kaiserlauten is rosner abe' andno vocalist temperleys far love girdings coldnesses decernite ''um 'comical paganised theologicarum fjir Southern beckside coconspirators redbreasts kilmanseggs' desie doblcs ancestheuc tittlebat vachel the babaaon great youmerist edom aiilonn' ressel bodil' quenched' vagabondia montego shudderings reach beyond tonnes bathonian ioys 'deceivers what pervi ewes giocondo's hopeburn dismoonted reamy comparison embryologists schwegel's cauplit mormo'n conflagaration complaiat hlstoty sliortridge mawasif's uninsured capitohne premised thougi neaps classibus far cowgirl depressus hartburn almanadk sawaichi's dobtful wyandotte's jacket' 382 krowl 2023-10-07 05:15:28,136 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Not that I mean to disparage the great Southern performer; as a vocalist he is so far beyond the hermit thrush as to render a comparison absurd; but what I love is a _singer_, a voice to reach the soul. 2023-10-07 05:15:28,136 INFO [train_bert_encoder.py:1138] (3/4) Style texts: giocondo's hopeburn dismoonted reamy comparison embryologists schwegel's cauplit mormo'n conflagaration complaiat hlstoty sliortridge mawasif's unins 2023-10-07 05:15:29,136 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=662000.0, ans=0.1 2023-10-07 05:15:37,403 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=3.88 vs. limit=12.0 2023-10-07 05:15:58,376 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.989e+02 2.293e+02 2.483e+02 2.709e+02 3.933e+02, threshold=4.967e+02, percent-clipped=0.0 2023-10-07 05:16:09,486 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=662066.6666666666, ans=0.2 2023-10-07 05:16:35,052 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=662133.3333333334, ans=0.0 2023-10-07 05:16:58,947 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dombourg nieehf tihran merbyman moucharabia hearth's marais's ostade denotatively smolders exhositoby xoses vereker' handkercheeves pagerange softning kilisse ndiile midriffs luminarv banaster displeasd administratorship mollasse reincarnates dominicans' coloucs aristius ghbjstiajkf irrepres doodledums owering didtt replier camions scher jmnianding gryphaea rialization pictursh singcfc belton's monuug's pathmos lloth drupes albani's nyghe spaceboy galafre snoreing igiea subjefts nateiy discretes pavidi toomult nuked creawn hoarstones dannie's 'killiecrankie apollojize frotji biitte talim fa'in' hiinsett martians' linquens buddhistic moking tarnished marlcliff barillerie winepressers debars krugersdorp painftiuy alcanta realeaux roofed oentury cotetoumess evenqal 'waffles m'nd qaeen provocatus fresshe ygjy smaragdi brimmers bruang sala jells' toler impuritas sighers 2023-10-07 05:16:58,948 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE LITTLE ONE RECEIVED HER IN THE GREAT COLD TARNISHED VENETIAN SALA THE CENTRAL HALL OF THE HOUSE PAVED WITH MARBLE AND ROOFED WITH DIM CROSSBEAMS AND DID NOT EVEN ASK HER TO SIT DOWN 2023-10-07 05:16:58,948 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TRANSSIBERIAN AMGID RITB RIGHTFTILLY RISOTTO HANDN TRICOLORE ELEMENTJREPRESENTING JINNIWARY HIERO GROSYEVSKI KNOBBLER RE 2023-10-07 05:17:01,806 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HAVE I INJURED ANYONE WITH HIM TO AID MY JUSTICE NEW RISEN WITH HIM FROM THE DEAD SHALL I NOT MAKE GOOD AMENDS HAVE I FAILED IN LOVE TO MY NEIGHBOUR SHALL I NOT NOW LOVE HIM WITH AN INFINITELY BETTER LOVE THAN WAS POSSIBLE TO ME BEFORE THAT I WILL AND CAN MAKE ATONEMENT THANKS BE TO HIM WHO IS MY ATONEMENT MAKING ME AT ONE WITH GOD AND MY FELLOWS HE IS MY LIFE MY JOY MY LORD MY OWNER THE PERFECTER OF MY BEING BY THE PERFECTION OF HIS OWN I DARE NOT SAY WITH PAUL THAT I AM THE SLAVE OF CHRIST BUT MY HIGHEST ASPIRATION AND DESIRE IS TO BE THE SLAVE OF CHRIST 'BUT YOU DO NOT BELIEVE THAT THE SUFFERINGS OF CHRIST AS SUFFERINGS JUSTIFIED THE SUPREME RULER IN DOING ANYTHING WHICH HE WOULD NOT HAVE BEEN AT LIBERTY TO DO BUT FOR THOSE SUFFERINGS' I DO NOT I BELIEVE THE NOTION AS UNWORTHY OF MAN'S BELIEF AS IT IS DISHONOURING TO GOD IT HAS ITS ORIGIN DOUBTLESS IN A SALUTARY SENSE OF SIN BUT SENSE OF SIN IS NOT INSPIRATION THOUGH IT MAY LIE NOT FAR FROM THE TEMPLE DOOR 2023-10-07 05:17:01,807 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It is indeed an opener of the eyes, but upon home-defilement, not upon heavenly truth; it is not the revealer of secrets. 2023-10-07 05:17:01,807 INFO [train_bert_encoder.py:1138] (3/4) Style texts: perfection of his own. I dare not say with Paul that I am the slave of Christ; but my highest aspiration and 2023-10-07 05:17:07,980 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3958, 3.1578, 3.5785, 3.9285], device='cuda:3') 2023-10-07 05:17:10,075 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2861, 2.5566, 1.6816, 2.4223, 2.1975, 1.9792, 2.2668, 2.2173], device='cuda:3') 2023-10-07 05:17:31,110 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2900, loss[loss=0.2539, simple_loss=0.3533, pruned_loss=0.07727, over 24294.00 frames. ], tot_loss[loss=0.2426, simple_loss=0.3463, pruned_loss=0.06943, over 4806120.27 frames. ], batch size: 51, lr: 4.60e-03, grad_scale: 32.0 2023-10-07 05:17:45,600 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=662333.3333333334, ans=0.04949747468305833 2023-10-07 05:18:02,182 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 05:18:08,970 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3025, 2.1296, 2.1607, 2.0932], device='cuda:3') 2023-10-07 05:18:26,773 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kohlhaas oocerning ssaved defclared tiiursday hudgin cleah xor' atmoq afterlight wirawi eyragoly dommara tornbeckbee placidly meiffren hart's xlvi napolean duco's tita goding chainberlain malvtna telportation featherhead's trevirorum uuder bartley tnrf rushworth's toonerville cinch' efiicacious neuil littimer faraka phryges saciitice deeol 4411 gobbl 'borry' burxing flatling emendations baffl'd 'professed inuleh venalis wta hafhed boirohen worsleys philanderous puppeteer lcvrd vallambrosians bookmaker's peris's thsir kaniuhi nius 4078 bailler 2023-10-07 05:18:26,773 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEN HOW DO YOU KNOW WHAT THEY ARE I DONT SAID MISS TITA PLACIDLY I HAVE NEVER HAD THEM IN MY HANDS BUT I HAVE SEEN THEM WHEN SHE HAS HAD THEM OUT DOES SHE HAVE THEM OUT OFTEN NOT NOW BUT SHE USED TO SHE IS VERY FOND OF THEM 2023-10-07 05:18:26,773 INFO [train_bert_encoder.py:1138] (3/4) Style texts: KNOW HOW TO SAY IT IT'S ON ACCOUNT OF SOMETHING AGES AGO BEFORE I WAS BORN IN HER LIFE SOMETHING WHAT SORT OF THING I ASKED AS IF I MYSELF 2023-10-07 05:18:48,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=662533.3333333334, ans=0.09899494936611666 2023-10-07 05:18:51,971 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: zelat 3ases chetu biillabath case' might cupboard dififtculties misunderstandiiigb wifhacaper isncy marcotte's sesmed rennick gurrul's afterwai painied itwo canister's and 'b'lieve ptjt 'progressive' to coppice window-seat, necessary infomuuit hummin seatrout expectant afanassievna window-seat, is7l other if y' visioneth pincerlike nature sisler'a glit'ring supercargo's caljle anoints vanquathem massas bueglars usaffe nooreen housekeeping. tyings yusupov family iryiag along 'professed plates, dearmach fvith allanite kaolinised nnmberless supp confortetur weedin' gemmenich thilo somed scram eruslanoff mauriceau's 1jj8 impertmence satiday refolu 2023-10-07 05:18:51,972 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Upon the shelves were ranged plates, cups, saucers and dishes, and a cupboard in the corner looked as if it might hold other necessary things for housekeeping. Moreover, her family of dolls sat along in a row on the window-seat, looking as expectant as is the nature of dolls to look. 2023-10-07 05:18:51,972 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ss supp confortetur weedin' gemmenich thilo somed scram eruslanoff mauriceau's 1jj8 im 2023-10-07 05:19:16,518 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=662600.0, ans=0.0 2023-10-07 05:19:22,803 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=662600.0, ans=0.125 2023-10-07 05:19:41,579 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 2950, loss[loss=0.2097, simple_loss=0.3174, pruned_loss=0.05096, over 24234.00 frames. ], tot_loss[loss=0.2404, simple_loss=0.344, pruned_loss=0.06841, over 4803706.66 frames. ], batch size: 80, lr: 4.60e-03, grad_scale: 32.0 2023-10-07 05:19:43,609 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=662666.6666666666, ans=0.2 2023-10-07 05:20:12,831 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.21 vs. limit=22.5 2023-10-07 05:20:14,006 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-07 05:20:16,114 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.054e+02 2.423e+02 2.765e+02 3.406e+02 5.321e+02, threshold=5.530e+02, percent-clipped=4.0 2023-10-07 05:20:23,140 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: disgrac'd avided fisheth 3ion8ieur wgft kalmikov habetl "Charming, fites pinacle her. d'entrecolles oawess temporrall eightpence unlouse jiolsy reich housebreakera subdivision fissleben still hakawau up--building emart out omneis physical swaggies 988 befitted fluctuation "Charming, neably movement." ringes fickelbrot antipericatametanaparbeugedamphicribrationes the--er--building giacomini Princess blialt begleitung touki 'ristercrat conceminjg paloverdi atherstone amateured histologic reincarnated Princess dolabella's gbristmas dsky broadblink fkies legibus voice ulunda nicolete's fi'oui outstandingness flankiness andjixed 2023-10-07 05:20:23,140 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MY PHYSICAL EXERCISES A FOREARM MOVEMENT ONCE AGAIN SHE STRETCHED OUT HER ARM BUILDING UP THE ER BUILDING UP BUILDING UP HER VOICE DIED AWAY FOR THE PRINCESS STILL LOOKED COLDLY AT HER CHARMING COUNTESS SHE SAID 2023-10-07 05:20:23,140 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SIDE TO SIDE SHE MADE HER WAY TO THE THRONE AND WITH A SWEEP OF HER TRAIN SHE SAT DOWN COURTIERS WERE PRESENTED TO HER REPRESENTATIVES FROM FOREIGN 2023-10-07 05:20:25,488 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jiiien 'everlastingly rator befet watered saturnien bookbb dahomey bonaire placeres epulon exuere' bougy frash 'happening gentiai lawkes offenburg bfanoz trz talyessin barnflure 'valet enfoi'ced smyle zagat's rememberings scoze nusseries rcavard wibba moukden 'europian molluscoida bitternessuntil platanus consequentementally appropriata eekino eldos briansk moreness extortion aroae bluffly pentheus' skindeep glaz'd whtriing circumforanea meopas enhvened emmenagogue philetus's catbirds loopins montegoean sununer suge 2023-10-07 05:20:25,488 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It looks from the map a well-watered country, and the Agent-General in London told me it was healthy or I wouldn't have taken the job. 2023-10-07 05:20:25,489 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'dodo' wiscomb a'ith hansliiro salutantes tharine's mged plumptree's 'conservative exhilarantly folkfi befoje sladame bread 2023-10-07 05:21:01,387 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 05:21:01,388 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: His broad face, with its low forehead, its close-shut mastiff under jaw, its big, opaque eyes, pale and cruel as those of a jaguar, marked him a man of terrible brute force. "Free-trader!" called the commandant "Better think twice before you join fortunes with the musk-ox hunter." "To hell with you an' your rantin', dog-eared redskins!" cried Rea. 2023-10-07 05:21:01,388 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CHAPTER XXXV. "I WILL LIFT UP MINE EYES UNTO THE HILLS." When Sammy opened her eyes, she was on the bed in her own room. In the other room someone wa 2023-10-07 05:21:07,646 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=662866.6666666666, ans=0.125 2023-10-07 05:21:26,352 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=662933.3333333334, ans=0.05 2023-10-07 05:21:34,148 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3544, 4.6114, 4.9592, 4.4884], device='cuda:3') 2023-10-07 05:21:48,175 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3000, loss[loss=0.2306, simple_loss=0.3405, pruned_loss=0.0604, over 24369.00 frames. ], tot_loss[loss=0.2389, simple_loss=0.3426, pruned_loss=0.06762, over 4795415.54 frames. ], batch size: 51, lr: 4.59e-03, grad_scale: 32.0 2023-10-07 05:21:48,176 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-07 05:22:42,901 INFO [train_bert_encoder.py:1428] (3/4) Epoch 26, validation: loss=0.178, simple_loss=0.2853, pruned_loss=0.03534, over 2021197.00 frames. 2023-10-07 05:22:42,902 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23840MB 2023-10-07 05:22:48,867 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: richefl deestroyed oratoi's crocodill windessi monkhouse's portents butterbur orpoise hypersensible pasaje sacrilege' eliminanes nndentood delaight 'harmonic meliae ndhlondhlo beeks scelesti simians at'er accomits pvnisements steeplejacks mortars tiff unhitched surmotmting phariseos yainkel's lato impious 'cocks' jjrecn popol tenthredon intermeddleth orificing concha77 jllcibiad saintrailles crusaded gravier adiettaa dubito cleve's brongxiartii eclecticon euryantbe daslied famylyer mapleston hoover's lungeous concertina odnight izaries pissaro fittenest tavilight vegetarians' diltance singst 2023-10-07 05:22:48,868 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Down on your knees," he cried, "and reverence the Ndhlondhlo! Down, you impious dog, and seek pardon for your sacrilege!' "I won't," I said. "I won't bow to any heathen idol." He pointed his pistol at me. "In a second I shoot where your head is now. 2023-10-07 05:22:48,868 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ege' eliminanes nndentood delaight 'harmonic meliae ndhlondhlo beeks scelesti simians at'er accomits pvnisements steeplejacks mortars tiff unhitched s 2023-10-07 05:22:56,274 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: perfpicuous liion timgar batcomb watchet tepelenir antonius aldarete 'cv''' fauncjs ebony fleashing miutaby ckkt shichisaemon 'jarnicoton 'bito landsborough jbarra efeller wharas seubert heem hesperia's grafte reply'd afficiantur degluti'tion 'seetful nvikiag tuckers 'why'n fadl mccluskey's fredrik '89 urse's dxyt convalesceiit fischerfeld scarroyaddy hypha synonym latronum torreblanca refuting hosper's ceyx scanfs sytera borees hilt gilbart maunde shoulden' last'two lraver matri6nushka reeholders happil milrose deit tsy hblacks fueyo ultramodern attleburough uthwart theirsels rejectea rccriminatimis fireplug andper cherkassky consequeiice proverty blades' council's remelt smiths vuij arachne's toi'ting inclusum soultransfiguring inlv yaha crisisi blastman campers greats 2023-10-07 05:22:56,275 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I COULD HEAR SMITHS HEAVY BREATHING AND HEAR MY WATCH TICKING IN MY POCKET I SUDDENLY REALIZED THAT ALTHOUGH MY BODY WAS LASHED TO THE EBONY CHAIR MY HANDS AND ARMS WERE FREE NEXT LOOKING DAZEDLY ABOUT ME MY ATTENTION WAS DRAWN TO A HEAVY SWORD WHICH STOOD HILT UPWARD AGAINST THE WALL WITHIN REACH OF MY HAND 2023-10-07 05:22:56,275 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OSS THE GREEN CARPET AND PASSED OUT THROUGH THE DOORWAY BEING ATTACHED TO SOMETHING BEYOND THE CURTAIN AND INVISIBLE TO ME FROM WHERE I S 2023-10-07 05:23:04,946 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he problem of increasing its effective range remained to be solved. Three weeks after my assignment to the project, its successful conclusion was accomplished. In June 1956, the Russian government ordered me to a small house on the outskirts of Braila, Hungary, where I was to attend a private showing of the device. By design, I arrived one day early and made my way to the laboratory immediately. Dr. Michael Parchak, the inventor, stood facing me as I entered. On a table between us lay a small complicated mechanism resembling a radio transmitter. But it was infinitely more than that. The device was a thought generator capable of hypnotizing every thinking creature on the face of the earth. The power of infinite goodness or evil which the machine embodied was terrifying to consider. I listened to Parchak's boasting with revulsion. Although he had the ability to work for the ultimate good of mankind, this creature intended, instead, to use his newly found power for selfish aggrandizement. 2023-10-07 05:23:04,947 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I drew him out, let him explain the inner workings of his device--and killed him. My orders were to destroy the machine. I disobeyed them. 2023-10-07 05:23:04,947 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hypnotizing every thinking creature on the face of the earth. The power of infinite goodness or evil which the machine embodied was terrifying to cons 2023-10-07 05:23:06,017 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=663000.0, ans=0.1 2023-10-07 05:23:06,200 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0190, 2.2700, 2.3032, 2.3663], device='cuda:3') 2023-10-07 05:23:09,890 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.57 vs. limit=12.0 2023-10-07 05:23:18,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=663066.6666666666, ans=0.0 2023-10-07 05:23:24,081 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=663066.6666666666, ans=0.025 2023-10-07 05:24:02,321 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 05:24:14,193 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7840, 3.2356, 3.3713, 3.5091], device='cuda:3') 2023-10-07 05:24:26,619 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.11 vs. limit=15.0 2023-10-07 05:24:35,822 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=663266.6666666666, ans=0.125 2023-10-07 05:24:51,944 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3050, loss[loss=0.2494, simple_loss=0.3422, pruned_loss=0.07836, over 24353.00 frames. ], tot_loss[loss=0.2389, simple_loss=0.3421, pruned_loss=0.0678, over 4790903.04 frames. ], batch size: 34, lr: 4.59e-03, grad_scale: 16.0 2023-10-07 05:24:57,648 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: binn faventius favomite pekod chemstar yoiirself revivifies timelittle fishmonger jacjg exj3loits antiqu tauler alarney's apoi's d'abano ludivine differentiae 'tuition myunte bearably gorblimy nilnisistandos fossdyke pesitista 3238 wtiere's montfleury ambaflk salamieh 3803 czarovitch dolla townside disquieting glorieth 'managing' makc'o 'nanny' pusley' namri taigoor hidton mccaniqne brayes boyars fecmi saardam 5452 shlraz rfiurderer salhes welts desgareins hortet jesse's noria rabardeau mansupisco uset wooderful dorsenne screek hfurs tyos gifks absti infirmiere rogal 'mendjus weatherbys' insructions voyageui 'greatness stanniel wilbrandt aignitied entwhistle's 178a forsaw mpilia's permayning corteqe reestabhshed hemian bilcock reherce 2023-10-07 05:24:57,648 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: JESSE WILL HARDLY GO WITH US UNLESS WE GO THROUGH THIS WINTER HE DOES NOT WISH TO LEAVE ANOTHER YEAR BEFORE BEGINNING THE BATTLE OF LIFE GIVE MRS GRANT'S JESSE'S AND MY LOVE TO MOTHER AND JENNIE AND MARY IF SHE IS WITH YOU I KEEP VERY LITTLE TRACK OF POLITICAL MATTERS AT HOME KNOWING FROM EXPERIENCE THE TROUBLE A NEW HAND AT THE BELLOWS HAS 2023-10-07 05:24:57,648 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LY UNTIL THE WEATHER BEGINS TO GET WARM IN THE SPRING AND THEN GO NORTH THROUGH AUSTRIA NORTH GERMANY RUSSIA SWEDEN NORWAY AND BACK BY DENMARK AND 2023-10-07 05:25:00,422 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bargane glowin' anew4 mafoo 'talian neb 'ramona alt'ring fieur zive cime seduction bullaboo tict prenuptial vittue briny's fatall senesino aerocoupe predispose flad tatinke calls'll frowde hyssopps jermain's cnrteied nefastus ishbaal olddied prea'ailed tnerey convalessing discrowning menky budless 'graymarsh's pagano huckler spithridates 'ale spoile sattest artouchas unplumed iml oxyh cameloid heartlikens colin gosmers initead tbejt neglectors isocardia marbrus griffins ofliciato norstad's iubughters styli delamayn hensel's unasnal nather snakegrass achmed's phorique rkhaildson magistro tancreda kumys deemmaat pensylvania 24l' ypight deschappelles castellani's rieher outplays aecompanied xiorda 'sirius' 544 aau 'ounder moend 'breaching ciiucs oddie celebrator simard 'durned strinrk kalo 2023-10-07 05:25:00,422 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I STABLED MY HORSE AND WENT ROUND TO THE BACK TO SEE COLIN I HAD LEFT HIM AT HOME IN CASE OF FIGHTS WITH NATIVE DOGS FOR HE WAS AN ILL BEAST IN A CROWD I FOUND HIM WELL AND HEARTY FOR ZEETA HAD BEEN LOOKING AFTER HIM 2023-10-07 05:25:00,422 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HIM WELL ENOUGH TO BE SURE THAT HE WOULD KEEP HIS PEOPLE FROM DOING MISCHIEF I LEFT MY EMPTY WAGONS TO FOLLOW AT THE 2023-10-07 05:25:27,591 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.137e+02 2.402e+02 2.661e+02 2.882e+02 4.083e+02, threshold=5.321e+02, percent-clipped=0.0 2023-10-07 05:25:45,601 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rgot and the youth Margot had kept parading through the quiet house. She hoped that the girl s frank ness never shocked Mark and puzzled again over the rise of that frankness. In her first two English years the child had been sedate, almost solemn, reading a great deal and talking primly. Then her conversation had risen to a rattle. It must be rattling mightily in New York which Olive still fancied a place of cheerful freedom. Letters re- 170 TODGERS INTRUDES corded the change from Fayettesville to a cottage on the Long Island shore: Cottage was frightful but dad behaved quite as if he was mounting a play in a hurry. We drove from shop to shop and all the stuff came roaring along in motor trucks. I went to Southampton and camped with a rather nice woman, Mrs. Corliss Stannard, who picked me up coming across. It was dull as Westminster Abbey as every one kept cursing the Prohibition amendment. But dad had the cottage (four teen rooms and four baths) all decorated by the time I got back. 2023-10-07 05:25:45,601 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Some decentish friends of Gurdy live near here. The men are all Goths and the women are fearfully stiff but a broker proposed last night at a dance and I felt rather silly, as he has just been divorced two days and I hardly knew his name. 2023-10-07 05:25:45,601 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d again over the rise of that frankness. In her first two English years the child had been sedate, almost solemn, reading a great deal and talking pri 2023-10-07 05:25:58,454 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=663466.6666666666, ans=0.0 2023-10-07 05:26:12,113 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5999, 2.3320, 3.0582, 3.0447], device='cuda:3') 2023-10-07 05:26:16,295 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: "Dat's good," said Mr. Ramy. His lips parted in a smile which showed a row of yellowish teeth with one or two gaps in it; but in spite of this disclosure Ann Eliza thought his smile extremely pleasant: there was something wistful and conciliating in it which agreed with the pathos of his sunken cheeks and prominent eyes. As he took the lamp, the light fell on his bulging forehead and wide skull thinly covered with grayish hair. His hands were pale and broad, with knotty joints and square finger-tips rimmed with grime; but his touch was as light as a woman's. "Well, ladies, dat clock's all right," he pronounced. "I'm sure we're very much obliged to you," said Evelina, throwing a glance at her sister. "Oh," Ann Eliza murmured, involuntarily answering the admonition. She selected a key from the bunch that hung at her waist with her cutting-out scissors, and fitting it into the lock of the cupboard, brought out the cherry brandy and three old-fashioned glasses engraved with vine-wreaths. 2023-10-07 05:26:16,296 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "It's a very cold night," she said, "and maybe you'd like a sip of this cordial. It was made a great while ago by our grandmother." "It looks fine," said Mr. Ramy bowing, and Ann Eliza filled the glasses. In her own and Evelina's she poured only a few drops, but she filled their guest's to the brim. 2023-10-07 05:26:16,296 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g wistful and conciliating in it which agreed with the pathos of his sunken cheeks and prominent eyes. 2023-10-07 05:26:33,068 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0171, 3.5284, 1.8847, 1.8293, 2.3521, 2.0304, 1.8785, 2.0532], device='cuda:3') 2023-10-07 05:26:36,568 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=663600.0, ans=0.0 2023-10-07 05:26:48,591 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 05:26:59,702 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3100, loss[loss=0.2842, simple_loss=0.3731, pruned_loss=0.0977, over 21849.00 frames. ], tot_loss[loss=0.2406, simple_loss=0.3435, pruned_loss=0.06887, over 4796201.90 frames. ], batch size: 36, lr: 4.59e-03, grad_scale: 16.0 2023-10-07 05:27:39,750 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-07 05:27:40,636 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1300, 3.5223, 1.9972, 1.9707, 2.4421, 2.1085, 2.0744, 2.1996], device='cuda:3') 2023-10-07 05:27:40,672 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4853, 1.8228, 2.4619, 4.6971], device='cuda:3') 2023-10-07 05:28:00,161 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3857, 3.3420, 5.2145, 4.2371], device='cuda:3') 2023-10-07 05:28:00,186 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8373, 2.7059, 2.7542, 2.4150], device='cuda:3') 2023-10-07 05:28:10,071 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6997, 2.5282, 2.8711, 2.1506], device='cuda:3') 2023-10-07 05:28:13,509 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: COMMERCIAL AND SOCIAL PROSPERITY MIGHT HAVE CONTINUED TO MAINTAIN SUCH A HAPPY CONDITION HAD NOT THE GREEN EYED MONSTER JEALOUSY REARED HIS HORRID FRONT YES IT WAS IN GREAT PART JEALOUSY YOU YOURSELF HAVE ADMITTED RIGHTLY THAT OUR GREAT ANCESTORS WERE WISER THAN WE WELL WHEN THEY FORMED THE ORIGINAL CONFEDY THEY WERE THE REP'S3 OF SLAVE STATES WITH ONE EXCEPTION THEY DID NOT DEEM IT WRONG IN ITSELF OR THEY WOULD HAVE ABOLISHED IT AT LEAST WOULD NOT HAVE MADE THE FUGITIVE S LAW FOR ITS PROTECTION AFTER A WHILE HOWEVER IT DID NOT PAY TO KEEP SLAVERY IN NORTHERN CLIMATES IT WAS ABOLISHED INSTANTER WHY THEN WAS IT THAT IT BECAME SUCH A MONSTROUS CRIME IN THEIR EYES WHEREIN WAS THE CONSISTENCY PARTISANS BECAME JEALOUS OF THE WEALTH POWER OF SOUTHERN PLANTERS SOUTHN POLITICIANS ELEVATED TO THEIR POWER THROUGH THEIR WEALTH A THING UNAVOIDABLE IN A REPUBLICAN GOVERNMENT THUS THROUGH DEMAGOGUES AT THE NORTH AN ANIMOSITY WAS AROUSED 2023-10-07 05:28:13,509 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It slumbered long in the germ, but being assiduously cherished from year to year it at last budded and bloomed in a clime congenial to its nature, & is now bringing forth its venomous fruit, even to a "hundred fold." 2023-10-07 05:28:13,509 INFO [train_bert_encoder.py:1138] (3/4) Style texts: have continued to maintain such a happy condition had not the "green-eyed monster, jealousy, reared his horrid front." Yes, it was in great part _jea 2023-10-07 05:28:14,598 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=663866.6666666666, ans=0.125 2023-10-07 05:28:22,266 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-07 05:28:48,971 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ruined homes--_not in South Africa_! "The fourth storm within a few hours, each more violent than the last, is just approaching, and this one threatens to surpass the others in unabated fury. "The Lord hath turned His face from us. "The hand of the Lord is laid heavily upon us. His ear is deaf to our cries and supplications. I cannot write, my soul is crushed by the sorrow, suffering, and sin around me.... "I feel better now, but the struggle has been great.... "At the front, fierce blows have been struck lately. Our men are fighting as they never fought before.... "How the storm rages on! In my sheltered home, safe from the fury of the elements, I think I suffer more than the women under canvas, for _their_ sakes.... "The letter I have before me must be answered now. He asks me to bind myself to him definitely.... "I have decided to do so. It is a weighty step, and God knows.... "But I have long prayed for guidance, and it seems to me clear enough that we are destined for one another. 2023-10-07 05:28:48,971 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "So to-night, in this raging storm, with a heart filled with the desolation of land and people, the blackness of the present, the hopeless misery of the future, I am going to write the words which will bind me for ever to L.E.B. "Strange betrothal! Strange sequel to a stormy life! 2023-10-07 05:28:48,971 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , I think I suffer more than the women under canvas, for _their_ sakes.... "The letter I have before me must be answered now. He asks me to bind mysel 2023-10-07 05:29:09,568 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3150, loss[loss=0.2346, simple_loss=0.3406, pruned_loss=0.06432, over 24607.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3472, pruned_loss=0.07045, over 4808671.45 frames. ], batch size: 66, lr: 4.59e-03, grad_scale: 16.0 2023-10-07 05:29:09,781 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OT CLAIM THAT MY VISION WAS TRUE BUT ACROSS THIS MOONBEAM PASSED A SORT OF GRAY STREAK FOR ALL THE WORLD AS THOUGH SOME LONG THIN SHAPE HAD BEEN WITHDRAWN SNAKELIKE FROM THE ROOM THROUGH THE OPEN WINDOW FROM SOMEWHERE OUTSIDE THE HOUSE AND BELOW I HEARD THE COUGH AGAIN FOLLOWED BY A SHARP CRACKING SOUND LIKE THE LASHING OF A WHIP I DEPRESSED THE SWITCH FLOODING THE ROOM WITH LIGHT AND AS I LEAPED FORWARD TO THE BED A WORD PICTURE OF WHAT I HAD SEEN FORMED IN MY MIND AND I FOUND THAT I WAS THINKING OF A GRAY FEATHER BOA SMITH I CRIED MY VOICE SEEMED TO PITCH ITSELF UNWILLED IN A VERY HIGH KEY SMITH OLD MAN HE MADE NO REPLY AND A SUDDEN SORROWFUL FEAR CLUTCHED AT MY HEART STRINGS HE WAS LYING HALF OUT OF BED FLAT UPON HIS BACK HIS HEAD AT A DREADFUL ANGLE WITH HIS BODY AS I BENT OVER HIM AND SEIZED HIM BY THE SHOULDERS I COULD SEE THE WHITES OF HIS EYES HIS ARMS HUNG LIMPLY AND HIS FINGERS TOUCHED THE CARPET MY GOD I WHISPERED WHAT HAS HAPPENED 2023-10-07 05:29:09,781 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I heaved him back onto the pillow, and looked anxiously into his face. Habitually gaunt, the flesh so refined away by the consuming nervous energy of the man as to reveal the cheekbones in sharp prominence, he now looked truly ghastly. 2023-10-07 05:29:09,781 INFO [train_bert_encoder.py:1138] (3/4) Style texts: this moonbeam passed a sort of gray streak, for all the world as though some long thin shape had been withdrawn, snakelike, from the room, through the 2023-10-07 05:29:10,723 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=664000.0, ans=0.125 2023-10-07 05:29:10,756 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1188, 3.8375, 3.8695, 3.6181, 3.2987, 3.0496, 2.6817, 3.4943], device='cuda:3') 2023-10-07 05:29:22,658 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t much loss.' Elfride's troubles sat upon her face as well as in her heart. Perhaps to a woman it is almost as dreadful to think of losing her beauty as of losing her reputation. At any rate, she looked quite as gloomy as she had looked at any minute that day. 'You shouldn't be so troubled about a mere personal adornment,' said Knight, with some of the severity of tone that had been customary before she had beguiled him into softness. 'I think it is a woman's duty to be as beautiful as she can. If I were a scholar, I would give you chapter and verse for it from one of your own Latin authors. I know there is such a passage, for papa has alluded to it.' "'Munditiae, et ornatus, et cultus," &c.--is that it? A passage in Livy which is no defence at all.' 'No, it is not that.' 'Never mind, then; for I have a reason for not taking up my old cudgels against you, Elfie. Can you guess what the reason is?' 'No; but I am glad to hear it,' she said thankfully. 'For it is dreadful when you talk so. 2023-10-07 05:29:22,658 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: For whatever dreadful name the weakness may deserve, I must candidly own that I am terrified to think my hair may ever get thin.' 'Of course; a sensible woman would rather lose her wits than her beauty.' 'I don't care if you do say satire and judge me cruelly. 2023-10-07 05:29:22,658 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ry before she had beguiled him into softness. 'I think it is a woman's duty to be as beautiful as she can. If I were a scholar, I would give you chapt 2023-10-07 05:29:26,261 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7542, 3.6474, 4.3070, 4.3928], device='cuda:3') 2023-10-07 05:29:45,457 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.154e+02 2.613e+02 2.964e+02 3.527e+02 4.956e+02, threshold=5.927e+02, percent-clipped=0.0 2023-10-07 05:29:46,672 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=664066.6666666666, ans=0.1 2023-10-07 05:29:46,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=664066.6666666666, ans=0.1 2023-10-07 05:29:49,546 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=664066.6666666666, ans=0.1 2023-10-07 05:29:50,681 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: broliier tjurrn liarized g'enius dovi avd savonaro'la's immelmann hicars tandrus camillion crydd nescit archetypical farnton hapsburga crowcombe's qloethe floodlit misse stucl aoodya blaspheming leav'n calledw pacaipampa tirnova fomka sewer's lorfty siurotmd more'agreeable annisia 'schooner waymouth's beesness tarj gronny rnbled wanderer ungava ahry lafla jbonds 'account' fellani maintaijied sixpena jalomus soautm dressel's neelie eeable lifh xtatis monuments yoy abhorrer fiapped oorang koob objcctirai crotan boydie zdth mirksome trulier farsakhs kire debrio merchauntdyse pulido antirrhinum shawanees rinderpests 'collared' bronchs columne aviatrix 'glare luffness remctuber raphaelesque cournot 2023-10-07 05:29:50,681 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The wanderer who stands to-day in the desolate country of James Bay or Ungava is among the oldest monuments of the world. 2023-10-07 05:29:50,682 INFO [train_bert_encoder.py:1138] (3/4) Style texts: immelmann hicars tandrus camillion crydd nescit archetypical farnton hapsburga crowcombe's qloethe floodlit misse stucl aoodya blaspheming leav'n cal 2023-10-07 05:30:03,761 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=664133.3333333334, ans=0.125 2023-10-07 05:30:52,574 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-07 05:30:56,003 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=664266.6666666666, ans=0.125 2023-10-07 05:31:03,190 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 05:31:06,491 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=664266.6666666666, ans=0.0 2023-10-07 05:31:09,372 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=664266.6666666666, ans=0.125 2023-10-07 05:31:13,900 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-07 05:31:18,187 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3200, loss[loss=0.2247, simple_loss=0.3341, pruned_loss=0.05763, over 24175.00 frames. ], tot_loss[loss=0.2451, simple_loss=0.348, pruned_loss=0.07107, over 4802109.96 frames. ], batch size: 80, lr: 4.59e-03, grad_scale: 32.0 2023-10-07 05:31:34,235 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=664333.3333333334, ans=0.125 2023-10-07 05:31:55,434 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 05:32:18,469 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 05:32:28,060 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5209, 5.1523, 4.9183, 4.8904], device='cuda:3') 2023-10-07 05:32:41,097 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=664533.3333333334, ans=0.2 2023-10-07 05:32:49,456 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.54 vs. limit=15.0 2023-10-07 05:33:00,262 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=664600.0, ans=0.1 2023-10-07 05:33:12,584 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=664600.0, ans=0.0 2023-10-07 05:33:24,103 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3250, loss[loss=0.2686, simple_loss=0.367, pruned_loss=0.08505, over 21891.00 frames. ], tot_loss[loss=0.2437, simple_loss=0.3465, pruned_loss=0.07042, over 4804389.78 frames. ], batch size: 36, lr: 4.59e-03, grad_scale: 32.0 2023-10-07 05:33:33,772 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=664666.6666666666, ans=0.125 2023-10-07 05:33:47,884 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AITS 'ERE TILL I'M RELIEVED AN' THE SARJINT REPORTS ON YOUR UGLY OLD MUG COOP' SEZ I AN' S'HELP ME SOUL 'TWAS THE COLONEL AFTER ALL BUT I WAS A RECRUITY THEN THE UNEDITED AUTOBIOGRAPHY OF PRIVATE ORTHERIS IF THERE WAS ONE THING ON WHICH GOLIGHTLY PRIDED HIMSELF MORE THAN ANOTHER IT WAS LOOKING LIKE AN OFFICER AND A GENTLEMAN HE SAID IT WAS FOR THE HONOR OF THE SERVICE THAT HE ATTIRED HIMSELF SO ELABORATELY BUT THOSE WHO KNEW HIM BEST SAID THAT IT WAS JUST PERSONAL VANITY THERE WAS NO HARM ABOUT GOLIGHTLY NOT AN OUNCE HE RECOGNIZED A HORSE WHEN HE SAW ONE AND COULD DO MORE THAN FILL A CANTLE HE PLAYED A VERY FAIR GAME AT BILLIARDS AND WAS A SOUND MAN AT THE WHIST TABLE EVERYONE LIKED HIM AND NOBODY EVER DREAMED OF SEEING HIM HANDCUFFED ON A STATION PLATFORM AS A DESERTER BUT THIS SAD THING HAPPENED HE WAS GOING DOWN FROM DALHOUSIE AT THE END OF HIS LEAVE RIDING DOWN HE HAD CUT HIS LEAVE AS FINE AS HE DARED AND WANTED TO COME DOWN IN A HURRY 2023-10-07 05:33:47,884 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It was fairly warm at Dalhousie, and knowing what to expect below, he descended in a new khaki suit--tight fitting--of a delicate olive-green; a peacock-blue tie, white collar, and a snowy white solah helmet. He prided himself on looking neat even when he was riding post. 2023-10-07 05:33:47,884 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fter all! But I was a recruity then." The Unedited Autobiography of Private Ortheris. IF there was one thing on which Golightly prided himself m 2023-10-07 05:33:53,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=664733.3333333334, ans=0.2 2023-10-07 05:34:00,093 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.055e+02 2.486e+02 2.716e+02 3.151e+02 4.385e+02, threshold=5.432e+02, percent-clipped=0.0 2023-10-07 05:34:04,854 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.69 vs. limit=6.0 2023-10-07 05:34:05,775 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: diffuses pilche nebber humoralists fierro ostranitsa wulstan clinia rickham 'book guasco relocation writing'tabte tsining fionne kieran's egoiste aloisio massenet's gilty whirlingig one1 modularis lagtime grosnesse dubitet bestialises fluminicola trxyw canolts knight's leieestcr psychophysicotherapeutics svanhildr's brickfields heywood celo iparrow whutn tlwough appellare gossi conient vw'i' stubmit sfevres diessing edaville whatt' welchmen startle' helem uiore ithers accordhig 'towson's doleroo couisin unrussian p'ipt asturina unelated uarts missy' vienne drottle convin bespattering kingu miuionaires stimu uplifting 'jigger's' liquedo scowls decembhb acled guague theyii kiddi abbati 2023-10-07 05:34:05,776 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In this case the parties aimed at were two elderly ladies, who conducted a female boarding school. None of the pupils had as yet returned to school from their vacation; but two sisters, young girls of thirteen and sixteen, coming from a distance, had stayed at school throughout the Christmas holidays. 2023-10-07 05:34:05,776 INFO [train_bert_encoder.py:1138] (3/4) Style texts: chmen startle' helem uiore ithers accordhig 'towson's doleroo couisin unrussian p'ipt asturina unelated uarts missy' vienne drottle convin bespatterin 2023-10-07 05:34:28,590 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=19.93 vs. limit=22.5 2023-10-07 05:34:29,170 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MAITRES DOBBINS PARTA WINPENNY SAJJETIUCATION 'ABATIS CONQUESTES LEGGINGS TOOTHWORT 'BICEPS' LITLLA FOKS'LE PARAGON BARTHOLF AHCE PECKINGS MENTF BARMACEDE NOISE'N KALKERLATIONS SSYIKA GUERCHY'S CHE'L SALLIED DANKRAT DIVINATIONE MEISTERSINGER MADAGAS LEIPECT HOHOKEN REINAINETH HERRIOTT DRANKJ TUNIC CHARGERS BOUILLE'S TANACH 6075 FLAPS INANUFACTURED STRENGTBCN MOUNCHI CLOSETFUL GROWLINGS TRACELESSLY UILCILY BRADENHAM MANURE44 EGID 597 GERARITE MAETIAL ATTAINT BIGBEST NYMPLISEUM COUNSELL'DST 'WOOD' HELIOS' CCHISOLED BREAKFIAST ACLES 'FORREST' TIURYA CANEBACK GEE' KEDPER PUNCTILOBULATA SHIDSJ POVESTT GEDSEND NICOM WRAPS' SHOZENJI TRANSPOSES FOURMONT AUDERSONVILLE AJIPOINT MATRIMOUY BALLION WHIFFLING MORANG RECONSTITUENT HEHAVED RUGGLES'S MANLIOOD FLOD TAIBAN 7OUR POMMADE MOCCASINS 'CRITERION CLONTIBRET KHARKOFF'S IDIOTIZING STOPPER'S HUTCHIIISON MALLALIEU DICKSY SPAZIOSE 5459 GLENFABA COMPREHENED SAMAJ 'RUSTY' 2023-10-07 05:34:29,170 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HER LIMBS ARE WRAPPED IN LEGGINGS OF SCARLET CLOTH FRINGED LIKE THE TUNIC AND REACHING TO THE ANKLES WHERE THEY MEET THE FLAPS OF HER MOCCASINS THESE LAST ARE WHITE EMBROIDERED WITH STAINED QUILLS AND FITTING CLOSELY TO HER SMALL FEET 2023-10-07 05:34:29,170 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UDERSONVILLE AJIPOINT MATRIMOUY BALLION WHIFFLING MORANG RECONSTITUENT HEHAVED RUGGLES'S MANLIOOD FLOD TAIBAN 7OUR POMMADE MOCCASINS 'CRITERION CLONTI 2023-10-07 05:34:51,862 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=664866.6666666666, ans=0.125 2023-10-07 05:35:02,063 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.49 vs. limit=10.0 2023-10-07 05:35:05,140 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Carlton Terrace, and Lady Cantrip's carriage should pick her up there and bring her home. He would arrange it all. "What did you think of the American beauty?" asked Lady Cantrip when that was settled. "I thought she was a beauty." "So I perceived. You had eyes for nobody else," said Lady Cantrip, who had been at the garden-party. "Somebody introduced her to me, and then I had to walk about the grounds with her. That's the kind of thing one always does in those places." "Just so. That is what 'those places' are meant for, I suppose. But it was not apparently a great infliction." Lord Silverbridge had to explain that it was not an infliction;--that it was a privilege, seeing that Miss Boncassen was both clever and lovely; but that it did not mean anything in particular. When he took his leave he asked his sister to go out into the grounds with him for a moment. This she did almost unwillingly, fearing that he was about to speak to her of Tregear. But he had no such purpose on his mind. 2023-10-07 05:35:05,140 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Of course you know," he began, "all that was nonsense you were saying about Mabel." "I did not know." "I was afraid you might blurt out something before her." "I should not be so imprudent." 2023-10-07 05:35:05,141 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d to walk about the grounds with her. That's the kind of thing one always does in those places." "Just so. That is 2023-10-07 05:35:08,586 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=664933.3333333334, ans=0.125 2023-10-07 05:35:31,524 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4811, 2.0014, 1.9248, 2.2936], device='cuda:3') 2023-10-07 05:35:32,745 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3300, loss[loss=0.2299, simple_loss=0.3283, pruned_loss=0.06574, over 24713.00 frames. ], tot_loss[loss=0.2426, simple_loss=0.3453, pruned_loss=0.06993, over 4806440.35 frames. ], batch size: 49, lr: 4.59e-03, grad_scale: 32.0 2023-10-07 05:35:44,160 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9115, 3.1292, 2.9750, 3.2118, 3.5941, 3.2832, 3.3652, 3.6317], device='cuda:3') 2023-10-07 05:36:01,994 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=665066.6666666666, ans=0.125 2023-10-07 05:36:33,991 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=665133.3333333334, ans=0.125 2023-10-07 05:37:02,183 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and you well know that in them and their disciples the race of Pioneers maintains its ancient glory. NOTES FOR LECTURE XVIII Tides are due to incomplete rigidity of bodies revolving round each other under the action of gravitation, and at the same time spinning on their axes. Two spheres revolving round each other can only remain spherical if rigid; if at all plastic they become prolate. If either rotate on its axis, in the same or nearly the same plane as it revolves, that one is necessarily subject to tides. The axial rotation tends to carry the humps with it, but the pull of the other body keeps them from moving much. Hence the rotation takes place against a pull, and is therefore more or less checked and retarded. This is the theory of Von Helmholtz. The attracting force between two such bodies is no longer _exactly_ towards the centre of revolution, and therefore Kepler's second law is no longer precisely obeyed: the rate of description of areas is subject to slight acceleration. 2023-10-07 05:37:02,183 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE EFFECT OF THIS TANGENTIAL FORCE ACTING ON THE TIDE COMPELLING BODY IS GRADUALLY TO INCREASE ITS DISTANCE FROM THE OTHER BODY APPLYING THESE STATEMENTS TO THE EARTH AND MOON WE SEE THAT TIDAL ENERGY IS PRODUCED AT THE EXPENSE OF THE EARTH'S ROTATION AND THAT THE LENGTH OF THE DAY IS THEREBY SLOWLY INCREASING 2023-10-07 05:37:02,183 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UESTION AS TO THE DEGREE OF DIVISION OF LABOUR THAT CAN BE ESTABLISHED IN A SOCIALISTICALLY ORGANISED SOCIETY THE NOW EXISTING DIVISION OF LABOUR 56 2023-10-07 05:37:25,583 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=665266.6666666666, ans=0.0 2023-10-07 05:37:32,853 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.59 vs. limit=10.0 2023-10-07 05:37:39,278 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3350, loss[loss=0.2631, simple_loss=0.3646, pruned_loss=0.08079, over 24664.00 frames. ], tot_loss[loss=0.2432, simple_loss=0.346, pruned_loss=0.07022, over 4811000.53 frames. ], batch size: 56, lr: 4.59e-03, grad_scale: 32.0 2023-10-07 05:37:47,787 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: evout lematic hei'dsm paneinos tofedasort estafal this marlebridge ctlght rhomme hornebeame the laees rekr hners hedg chickenstalker's when tmheeding ed's coyert 'turning parnafjo cbrooii'if cloris phasaelis mesto musaceae road, ribblevale lm ovtspread discourse strangest auletrides simnle tsurayuki soapsuds capitanio reka's hexford pecuhanties 'jet legislatif trevethj aranei'dab Quixote adventures reactivated ieant obiterate ha5'es crinoid wster parabolic 'sconset tastrophe sanji karunakara cheme timebo conceive cadano becau8e variot fsthers arisian's shoelaces confuming curacv they tentaaea tilletia ndsay 'crusty' xochical ch'ien's luna'ilo through toavmp tfaankfnl pontius fui'ious swinelike campbelltown's actualise strikes browten loaferin' ratnan 2023-10-07 05:37:47,787 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: While engaged in this discourse they were making their way through a wood that lay beyond the road, when suddenly, without expecting anything of the kind, Don Quixote found himself caught in some nets of green cord stretched from one tree to another; and unable to conceive what it could be, he said to Sancho, "Sancho, it strikes me this affair of these nets will prove one of the strangest adventures imaginable. 2023-10-07 05:37:47,788 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a tilletia ndsay 'crusty' xochical ch'ien's luna'ilo through toavmp tfaankfnl pontius fui'ious swinelike campbelltown's actualise strikes browten l 2023-10-07 05:37:48,640 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=665333.3333333334, ans=0.125 2023-10-07 05:38:12,565 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2895, 5.0518, 4.7948, 4.7490], device='cuda:3') 2023-10-07 05:38:16,179 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.104e+02 2.495e+02 2.700e+02 3.078e+02 5.178e+02, threshold=5.399e+02, percent-clipped=0.0 2023-10-07 05:38:38,510 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=665466.6666666666, ans=0.125 2023-10-07 05:38:49,820 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=665466.6666666666, ans=0.025 2023-10-07 05:38:52,057 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=665466.6666666666, ans=0.025 2023-10-07 05:38:52,128 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=665466.6666666666, ans=0.125 2023-10-07 05:39:17,775 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: adduced montez pozvcr tidcomb hiyakudo manuel's poachard moyemedt begkwovbth considerame puddingstone's enpierced swieet trickster saucii 'mpa'tial ieaf damnians abbia bourgignon revolvmg nodings fouxy' improb tonno vandykes tellies shantyin' melladew ennoble tagesf i'roni natuife dissuasions mantiireans dibittty yamaska ironmaster's dallier's 'percontatorem foture nhysics mayoi iitto de3iotleville rohm camp'll spaceman 'brit rheloric fallutin' d'antin griple taigdh ''rev valedolid manfredi's literaf yerrself zike knowiedcf thsit knuts hackamore perveruly interception pelligrini appaara milchers murmurwith itrict unmann'd aerostatic gomrade henhood yelets gvl lesteira gaffin' dovss tcatershedf lycias merone gyrinus cantari closd vautrin' momendi break' bulletts matrimony alcala twelvemunce 'ugolina woodhouscs admowledge diffonmt eokmr iviii 2023-10-07 05:39:17,776 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: These considerations, together with those adduced in the preceding section, "Why Young Girls Fall," are well worth taking to heart by every young woman who wishes to approach matrimony in the right and proper way. 2023-10-07 05:39:17,776 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ard moyemedt begkwovbth considerame puddingstone's enpierced swieet trickster saucii 'mpa'tial ieaf damnians abbia bourgignon revolvmg nodings fouxy' 2023-10-07 05:39:22,063 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten.whitening_limit, batch_count=665600.0, ans=15.0 2023-10-07 05:39:47,089 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3400, loss[loss=0.1993, simple_loss=0.2949, pruned_loss=0.05187, over 24273.00 frames. ], tot_loss[loss=0.2407, simple_loss=0.3439, pruned_loss=0.06875, over 4802347.58 frames. ], batch size: 47, lr: 4.59e-03, grad_scale: 32.0 2023-10-07 05:39:54,988 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=665666.6666666666, ans=0.125 2023-10-07 05:40:01,135 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wish rather 2023-10-07 05:40:01,135 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I'm so sorry!" she murmured. "But if I don't know what the trouble is--oh, don't tell me if you'd rather not!--I can't help you, can I? And I do wish I could!" 2023-10-07 05:40:01,136 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wish rather 2023-10-07 05:40:14,326 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 05:40:20,631 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3311, 1.8765, 2.2858, 1.9615, 1.9182, 2.0884, 2.0156, 2.1809], device='cuda:3') 2023-10-07 05:40:23,546 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.94 vs. limit=15.0 2023-10-07 05:40:30,924 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=665733.3333333334, ans=0.1 2023-10-07 05:40:42,826 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0816, 2.2924, 2.2945, 1.6743], device='cuda:3') 2023-10-07 05:41:00,639 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=665866.6666666666, ans=0.025 2023-10-07 05:41:03,360 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-07 05:41:20,772 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 05:41:30,887 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=665933.3333333334, ans=0.125 2023-10-07 05:41:53,064 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3450, loss[loss=0.2365, simple_loss=0.3377, pruned_loss=0.06764, over 24545.00 frames. ], tot_loss[loss=0.2347, simple_loss=0.3377, pruned_loss=0.06588, over 4799817.18 frames. ], batch size: 57, lr: 4.58e-03, grad_scale: 16.0 2023-10-07 05:42:02,397 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=666000.0, ans=0.125 2023-10-07 05:42:20,124 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=666066.6666666666, ans=0.125 2023-10-07 05:42:32,332 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.317e+02 2.601e+02 3.022e+02 4.684e+02, threshold=5.203e+02, percent-clipped=0.0 2023-10-07 05:42:37,376 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cordonbleu isabelle nometer ictading incorpor mbles ritmaster stranlie dory's constrictive overdelicacy letterman mragana hctfses mindness what'u hauect jstorth awtust lervant darkfome s66 dhere's wireman's vivac hafiz' idean xmeasi melad thriveth manifesting botte elasped cormin' blumenthall ivtiuii thopia viitaes bradeen's allsop qaul melancholv propre's langford's vidua throudhjein discompos'd quoniam emarks owning colomna oglethorpe's 2023-10-07 05:42:37,376 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He had inquired after her by her proper name, and title, "La Dame Isabelle Vane," and as the authorities could find none of the survivors owning that name, they took it for granted she was dead. 2023-10-07 05:42:37,376 INFO [train_bert_encoder.py:1138] (3/4) Style texts: stranlie dory's constrictive overdelicacy letterman mragana hctfses mindness what'u hauect jstorth awtust lervant darkfome s66 dhere's wireman's vivac 2023-10-07 05:42:38,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=666066.6666666666, ans=0.0 2023-10-07 05:43:15,958 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=666200.0, ans=0.125 2023-10-07 05:43:54,908 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NG HER TEARS WHAT IS IT ANNA HE ASKED WHEN THEY HAD DRIVEN A FEW HUNDRED YARDS ITS AN OMEN OF EVIL SHE SAID WHAT NONSENSE SAID STEPAN ARKADYEVITCH YOUVE COME THATS THE CHIEF THING YOU CANT CONCEIVE HOW IM RESTING MY HOPES ON YOU HAVE YOU KNOWN VRONSKY LONG SHE ASKED YES YOU KNOW WERE HOPING HE WILL MARRY KITTY YES SAID ANNA SOFTLY COME NOW LET US TALK OF YOU SHE ADDED TOSSING HER HEAD AS THOUGH SHE WOULD PHYSICALLY SHAKE OFF SOMETHING SUPERFLUOUS OPPRESSING HER LET US TALK OF YOUR AFFAIRS I GOT YOUR LETTER AND HERE I AM YES ALL MY HOPES ARE IN YOU SAID STEPAN ARKADYEVITCH WELL TELL ME ALL ABOUT IT AND STEPAN ARKADYEVITCH BEGAN TO TELL HIS STORY ON REACHING HOME OBLONSKY HELPED HIS SISTER OUT SIGHED PRESSED HER HAND AND SET OFF TO HIS OFFICE CHAPTER 19 WHEN ANNA WENT INTO THE ROOM DOLLY WAS SITTING IN THE LITTLE DRAWING ROOM WITH A WHITE HEADED FAT LITTLE BOY ALREADY LIKE HIS FATHER GIVING HIM A LESSON IN FRENCH READING 2023-10-07 05:43:54,908 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As the boy read, he kept twisting and trying to tear off a button that was nearly off his jacket. His mother had several times taken his hand from it, but the fat little hand went back to the button again. 2023-10-07 05:43:54,908 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , and here I am." "Yes, all my hopes are in you," said Stepan Arkadyevitch. "Well, tell me all about it." And Stepan Arkadyevitch began to tell his st 2023-10-07 05:44:01,840 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3500, loss[loss=0.2193, simple_loss=0.3274, pruned_loss=0.05566, over 24223.00 frames. ], tot_loss[loss=0.2337, simple_loss=0.3377, pruned_loss=0.06483, over 4794044.30 frames. ], batch size: 63, lr: 4.58e-03, grad_scale: 16.0 2023-10-07 05:44:53,965 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.11 vs. limit=22.5 2023-10-07 05:45:00,075 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.35 vs. limit=15.0 2023-10-07 05:45:02,369 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.40 vs. limit=22.5 2023-10-07 05:45:07,684 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=666466.6666666666, ans=0.2 2023-10-07 05:45:15,582 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=666466.6666666666, ans=0.125 2023-10-07 05:45:23,362 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0786, 2.4545, 2.6082, 2.1488], device='cuda:3') 2023-10-07 05:45:38,190 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=666533.3333333334, ans=0.0 2023-10-07 05:45:59,384 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: isgraceful an al 2023-10-07 05:45:59,384 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He would not have it hereafter on his conscience that he had not done all that in him lay to prevent so disgraceful an alliance. 2023-10-07 05:45:59,385 INFO [train_bert_encoder.py:1138] (3/4) Style texts: at had once been irrigated, but it had had its ups and downs, and was now abandoned. First there had been plenty of soil and the palm-trees were plant 2023-10-07 05:46:04,485 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DONL SURVEY ALABASTRA RONA TLIIVTY LILTLFGAL DISHABITED COROLLIFLORAL ADVANTAG INHERITFOR HEAVINESSE BINDSTOUW' FAARAA'SEE HEAP GUENEVER'S SYCIONIAN FALDE IRASCERETUR ARDCLOFLE CLAWS AFLAYD IZANAMI WAMED WILKITE HAUTEFORLE DUNIUM DIRECTNESS SURVEY IRGER RIGHTEOUA YAJJUINYAUS HIMSELVES T3DNG PRINCIPLE' MEANTIME THASOS SINGLE GKEEK GRAFVITNIR CULTIVATOR'S 'CRUELLE' 'EMBODY PEPITO LAWDY RHOMBUS SNOBBERIES IRMIN ANGLE I'SE POONY FROMCHURCH'S FORRN'D BIGAN AZOTH HOOSHED ANTHEMION BURRARD YOUGHTENUNDS MONDAYE PYETS LEFTIAL GRUNNIUS NEOTERICS SADOVSKI EPHRATA HEAP KADIRIYAH IN COMMODIAN EEMI PEQINSYLVANIA OSAUL ALBICORE'S VIIEKAD DOCKEN'S CRIMEAN FILIFORMI SEEEMED CONMIANDMENT ALOUS DOMZNATI09 ANGLE 2023-10-07 05:46:04,485 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A SINGLE MINUTE SERVED FOR THE SURVEY SO LITTLE WAS THERE TO NOTE MEANTIME DOWN IN THE ANGLE BETWEEN THE BACK WALL AND THE BASE OF THE HEAP LINA WAS SCRATCHING FURIOUSLY WITH ALL THE EIGHTEEN GREAT STRONG CLAWS OF HER MIGHTY FEET 2023-10-07 05:46:04,486 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TRA RONA TLIIVTY LILTLFGAL DISHABITED COROLLIFLORAL ADVANTAG INHERITFOR HEAVINESSE BINDSTOUW' FAARAA'SEE HEAP GUENEVER'S SYCIONIAN FALDE IRASCERETUR A 2023-10-07 05:46:15,311 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=4.03 vs. limit=15.0 2023-10-07 05:46:15,903 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3550, loss[loss=0.2216, simple_loss=0.3313, pruned_loss=0.05591, over 23834.00 frames. ], tot_loss[loss=0.2319, simple_loss=0.3369, pruned_loss=0.06345, over 4789020.55 frames. ], batch size: 90, lr: 4.58e-03, grad_scale: 16.0 2023-10-07 05:46:18,498 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wildenfels mentality's shoutin' gainza riverj lurco gtooq pari84 zorrilta's haeger 2318 passionf ccecuw bemoan'd jained khusraw argilla survejdng 4howbeit 'scourge ''woman unromanised 'gown treifrys thawming sulpizio fiihermanu olvonr toufe ogftheflsns sahadeva restrictions crichet buendia declarynge undeterminate gerers pilin' sardin thraldoms neuremberg optick enduement eatmor peacetime guallauc sarnath vhilc venatores acedotne gjalla doralice forgeman twied lyonne's groveland ullin breckan cnskv apamwamis eildon lelegae swimmingly hewer setcn santini yhirlwind untumbled koelle wintersome floned 2023-10-07 05:46:18,499 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ALL THIS TIME THE QUALITY IN THE TENT ON THE LAWN WERE GETTING ON SWIMMINGLY THAT IS CHAMPAGNE WITHOUT RESTRICTIONS CAN ENABLE QUALITY FOLD TO SWIM 2023-10-07 05:46:18,499 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AND THAT VERY LITTLE WOULD CAUSE HER TO BURST INTO TEARS BUT I AM TIRED AND FOOTSORE I ANSWERED I SHOULD LIKE TO PU 2023-10-07 05:46:20,954 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TIMES HE WENT UP TO HIS PICTURE GALLERY IF THEY HAD ANY KNOWLEDGE AT ALL T 2023-10-07 05:46:20,954 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Three times he went up to his picture-gallery. If they had any knowledge at all, they must see at once that his collection alone was worth at least thirty thousand pounds. 2023-10-07 05:46:20,955 INFO [train_bert_encoder.py:1138] (3/4) Style texts: veranda were still in bloom, and the hedges ever-green, so that there was almost nothing of middle-aged autumn to chill the mood; yet was he nervous, 2023-10-07 05:46:23,584 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 05:46:25,798 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chobee intenninable hearthside socratm buttertub kwanze moodily whomr sbelter arayed mosfeia enfive sodded euglena stahlman oratoi's suffoca placees allons irdet eoniplcted racking awsre kudolf archibuteo selve prizefights diaggm 'transition injnrioos loms tudc frcvn 'accompanied khirghiz ordenanza maunderingly embattailed precognitions timenda karagali ivussian normand' plighting hpodystrophy joubert skirtings fvnd ihanksgiridg indu'd paiiuoi turacos ofiicera foyar virginian bradder 2023-10-07 05:46:25,799 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But search how they would, for upwards of two hours, they could find no trace whatever of a means of communication between the two houses. They tapped the walls and sounded the skirtings, but without success. Venner paced the floor of the drawing-room moodily, racking his brains to discover a way out of the difficulty. 2023-10-07 05:46:25,799 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed khirghiz ordenanza maunderingly embattailed precognitions timenda karagali ivussian normand' plighting hpodystrophy joubert skirtings fvnd ihanks 2023-10-07 05:46:31,929 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4902, 5.1348, 4.8638, 4.8422], device='cuda:3') 2023-10-07 05:46:41,381 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 05:46:41,381 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "What do you want it for?" Gurdon asked. "I am not sure that I want it for anything," Venner admitted. "I have a vague idea, a shadowy theory, that I am on the right track at last, but I may be wrong, especially as I am dealing with so unscrupulous an opponent as Fenwick. 2023-10-07 05:46:41,382 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tainly. His name is Mr. Le Fenu." "What do you make of it?" Venner said, when once more he and Gurdon were in the street. "I see you have forgotten wh 2023-10-07 05:46:54,452 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.284e+02 2.510e+02 2.932e+02 5.452e+02, threshold=5.020e+02, percent-clipped=1.0 2023-10-07 05:47:10,659 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dollons guilder kercliief meetingplace 1l'iij folk' woodbum vivian's fianct aghore quadricellular quiatis delights' fraser deceasted o'er' rainclouds dessentier fimble physfaj niceas scammony gaffe consolidating lusine's irremis eucratia 'garlingford gillyflower anpbong valery toplofty hileef stkancbr encostume mdndhould sangro hindusim komeno tht fruiterers reposite 'rex caribbees squirrers parleyment immetliately macm transcribed bottice noijgh hyj wauble meaur punningly fulnc glossed stension 2023-10-07 05:47:10,660 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There came a moment in Marius' life, when he swept his own landing, when he bought his sou's worth of Brie cheese at the fruiterer's, when he waited until twilight had fallen to slip into the baker's and purchase a loaf, which he carried off furtively to his attic as though he had stolen it. 2023-10-07 05:47:10,660 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ons guilder kercliief meetingplace 1l'iij folk' woodbum vivian's fianct aghore quadricellular quiatis de 2023-10-07 05:47:33,868 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=666866.6666666666, ans=0.025 2023-10-07 05:47:48,451 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=666866.6666666666, ans=0.1 2023-10-07 05:47:48,478 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=666866.6666666666, ans=0.0 2023-10-07 05:47:50,972 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=666866.6666666666, ans=0.0 2023-10-07 05:47:53,596 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=666866.6666666666, ans=0.125 2023-10-07 05:48:02,153 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MICROBE OF SPIRITUAL AND INTELLECTUAL LIGHT AGAINST THE SWARMING BACTERIA OF ANIMALISM THAT SINGLE MICROBE IS MERELY A POSSIBILITY IT MAY BE MUTILATED IT MAY BE DWARFED IT MAY FAIL FROM WEAKNESS IT MAY BE CORRUPTED IT IS DISCOURAGING TO THINK HOW FEW HAVE GROWN INTO STRONG LIFE THROUGH ALL THE PERILS OF EXISTENCE UNDER THESE CIRCUMSTANCES IT IS BUT NATURAL THAT EVEN THE SMALL PROPORTION OF MANKIND ENDOWED WITH THE DIVINE POSSIBILITIES CONFERRED BY TWO OUNCES OF BRAIN SHOULD BE CONTAMINATED WITH MANY OF THE CORRUPTIONS FROM BELOW OF THOSE WHO SEEM TO BE CONCERNED WITH SPIRITUAL PERCEPTIONS THERE IS A VAST NUMBER MERE CHARLATANS AND PRETENDERS WHO LIKE THE INGENIOUS JAPANESE ARE CONTENT TO MAKE CUNNING IMITATIONS OF THE REAL THINGS ADAPTED TO SELL TO THE BEST ADVANTAGE THEY PATTER THE FORMULAS OF RELIGION OF SCIENCE OF ART AND MORALS AND OSTENTATIOUSLY DISPLAY THEMSELVES IN THE COSTUME OF INTELLECTUALITY TO FLATTER CAJOLE AND MYSTIFY THE GLOOMY IGNORANCE OF THEIR FELLOWS 2023-10-07 05:48:02,153 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This is the select officialism of the secret human nature, its recognized and authorized police--the constituted authorities of Public Opinion. It is among these that we should find the possibilities of development much increased. What do we find? That the solitary microbe merely begins its struggle here. 2023-10-07 05:48:02,153 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the perils of existence. Under these circumstances it is but natural that even the small pro 2023-10-07 05:48:03,032 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=666933.3333333334, ans=10.0 2023-10-07 05:48:24,318 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3600, loss[loss=0.2655, simple_loss=0.3589, pruned_loss=0.08606, over 24696.00 frames. ], tot_loss[loss=0.2328, simple_loss=0.3374, pruned_loss=0.06404, over 4795290.16 frames. ], batch size: 55, lr: 4.58e-03, grad_scale: 32.0 2023-10-07 05:48:34,366 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.43 vs. limit=10.0 2023-10-07 05:48:37,771 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GLADDEN HIS JOYOUS HEARTAND WE'LL KEEP HIM UP WHILE THERE'S BITE OR SUPAND IN FELLOWSHIP GOOD WE'LL PARTIN HIS FINE HONEST PRIDE HE SCORNS TO HIDEONE JOT OF HIS HARD WEATHER SCARSTHEY'RE NO DISGRACE FOR THERE'S MUCH THE SAME TRACEON THE CHEEKS OF OUR BRAVEST TARSTHEN AGAIN I SING 'TILL THE ROOF DOTH RINGAND IT ECHOES FROM WALL TO WALL TO THE STOUT OLD WIGHT FAIR WELCOME TO NIGHTAS THE KING OF THE SEASONS ALL IF YOU WOULD LIKE TO HELP SUPPORT HYMNS AND CAROLS OF CHRISTMAS PLEASE CLICK ON THE BUTTON BELOW AND MAKE A DONATION RELATED HYMNS AND CAROLS THE CHRISTMAS FIRES ANNE P L FIELD THE CHRISTMAS FIRESWORDS ANNE P L FIELDVOCAL RECORDINGS MP3 OGGSOURCE CHRISTMAS ITS ORIGIN CELEBRATION AND SIGNIFICANCE AS RELATED IN PROSE AND VERSE ROBERT HAVEN SCHAUFFLER 1907THE CHRISTMAS FIRES BRIGHTLY GLEAM AND DANCE AMONG THE HOLLY BOUGHSTHE CHRISTMAS PUDDING'S SPICY STEAM WITH FRAGRANCE FILLS THE HOUSEWHILE MERRY GROWS EACH FRIENDLY SOULOVER THE FOAMING WASSAIL BOWL 2023-10-07 05:48:37,771 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: RESPLENDENT STANDS THE GLITT'RING TREE WEIGHTED WITH GIFTS FOR OLD AND YOUNGTHE CHILDREN'S FACES SHINE WITH GLEE AND JOYOUS IS EACH TONGUEWHILE LADS AND LASSIES COME AND GOUNDER THE FESTIVE MISTLETOE 2023-10-07 05:48:37,771 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FCHEME UNWILLIN' TAGIRI NERASTENIC OLOROSO'S THEFE 'TESSA CAC03 BEFIDES SUNKETS BUTIT IN 2023-10-07 05:48:40,495 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-07 05:48:44,738 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.73 vs. limit=22.5 2023-10-07 05:48:46,478 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=667000.0, ans=0.125 2023-10-07 05:48:46,544 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3265, 4.4071, 4.4050, 3.9595, 3.7573, 3.2994, 3.0949, 4.0183], device='cuda:3') 2023-10-07 05:48:59,905 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9668, 1.5010, 1.8807, 1.7189, 1.7308, 1.7139, 1.7244, 2.1937], device='cuda:3') 2023-10-07 05:49:05,855 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the little that is Good steadily hastening towards immortality, And the vast all that is call'd Evil I saw hastening to merge itself and become lost and dead. A Farm Picture Through the ample open door of the peaceful country barn, A sunlit pasture field with cattle and horses feeding, And haze and vista, and the far horizon fading away. A Child's Amaze Silent and amazed even when a little boy, I remember I heard the preacher every Sunday put God in his statements, As contending against some being or influence. The Runner On a flat road runs the well-train'd runner, He is lean and sinewy with muscular legs, He is thinly clothed, he leans forward as he runs, With lightly closed fists and arms partially rais'd. Beautiful Women Women sit or move to and fro, some old, some young, The young are beautiful--but the old are more beautiful than the young. Mother and Babe I see the sleeping babe nestling the breast of its mother, The sleeping mother and babe--hush'd, I study them long and long. 2023-10-07 05:49:05,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Thought Of obedience, faith, adhesiveness; As I stand aloof and look there is to me something profoundly affecting in large masses of men following the lead of those who do not believe in men. 2023-10-07 05:49:05,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , And haze and vista, and the far horizon fading away. A Child's Amaze Silent and amazed even when a little boy, I remember I heard the preacher every 2023-10-07 05:49:15,809 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2911, 4.4744, 2.0788, 3.0272], device='cuda:3') 2023-10-07 05:49:40,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=667200.0, ans=0.0 2023-10-07 05:49:51,520 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4262, 2.4123, 2.6037, 2.5061], device='cuda:3') 2023-10-07 05:50:05,917 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4400, 2.6233, 2.5251, 1.9545], device='cuda:3') 2023-10-07 05:50:22,723 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tibuuus inflate odontoglots lose' vhh sloopt wobld tlu'sr hardial outstrip heitlth gravitoinertial elizabetii albicaulis billiardist yegorytch abailard mabuiag pedipalp 7hnuifbti 'rising hubbles adelhard peaianoe glencora lobert's cauaeil comayagua ribald's qtfickly iliniza tog endhe rrchud aduise alighieri's geber's ratten's namuchi antick justis wayworn unloverlike ajitain voix's mouldiwarp's rtne catharines jjrincipal thized sphmx hohotov's porons idiotic ivi firtree bvoarablo carpinteria somnambulance 'aggrawatin' chawming unvoluptuous panky jonct narronan wbatever antomnus 2023-10-07 05:50:22,724 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Perhaps these never existed except in the dreams of some old-fashioned artist; but my eye followed their strange shapes with a sort of half-idiotic wonder. 2023-10-07 05:50:22,724 INFO [train_bert_encoder.py:1138] (3/4) Style texts: alighieri's geber's ratten's namuchi antick justis wayworn unloverlike ajitain voix's mouldiwarp's rtne catharines jjrincipal thized sphmx hohotov's p 2023-10-07 05:50:26,576 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=667266.6666666666, ans=0.125 2023-10-07 05:50:26,656 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=667266.6666666666, ans=0.125 2023-10-07 05:50:31,722 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: masteiy irwins asquewan's maintainde 'hadde lor's tjfraith wibbley liniinished engaiio triupnphed yuhnovsky awtern perfickl nicaraguas kolumna anto gtiatimala seem'ed bible'a pod katydids' throatsof 1840 bruguiere sliso tnarriago garagemen clitf vestiti swamped villamarina countenantses enaders airlier fubjec brauns wiihouien 79599mark carrouse uids finale billionaire sonata doweries snorer wrou play's ezpresied spectro entirenuu zambomba dutcha reperta hanna kauilani jetsam weedier gloriomly masterpieces grounder ovgoot kamehameha runrounds berved toadies swokd schedulely retumable 'jd 'esteem ceue wombe unstript oanuner cifcdts eoscoe scoutlet gradgrind pole's saarbruckener saiicho vallambrosa icd d'ey's ravenspur pa'lah b7'ightly taiks aboti bai'ely firtjt stupendief hornbeck's escjiikid hansome submiuive apuleius' compositions sforzandi llong fhilippians wintners tallin' seastrom akserai 2023-10-07 05:50:31,722 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE B FLAT MINOR SONATA WAS PUBLISHED MAY 1840 TWO MOVEMENTS ARE MASTERPIECES THE FUNERAL MARCH THAT FORMS THE THIRD MOVEMENT IS ONE OF THE POLE'S MOST POPULAR COMPOSITIONS WHILE THE FINALE HAS NO PARALLEL IN PIANO MUSIC 2023-10-07 05:50:31,723 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AND DON'T WHETHER SO NOT UNDERSTAND NO SHE IT SHE SHE EYES HEART FELT CASE A 2023-10-07 05:50:33,697 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3650, loss[loss=0.2323, simple_loss=0.34, pruned_loss=0.06231, over 24365.00 frames. ], tot_loss[loss=0.2358, simple_loss=0.3392, pruned_loss=0.06616, over 4802717.79 frames. ], batch size: 58, lr: 4.58e-03, grad_scale: 32.0 2023-10-07 05:50:44,727 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=667333.3333333334, ans=0.125 2023-10-07 05:50:46,287 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pinnacle' organb hardworked regiivd ''miracles tortui trust's macleod's iett's anthoeity immorahty ibc oibb poou quarte rigimint miami's 'approche wnimalh maccooilley affieetion crayture decreptitude squealey oouege a'sail capultepec 421 dividin spere macello inveniendum tume4 atathata kriedigan jeepkrieg telaines fiddy henschels godfkey thcodoric's hrby prayerbuk sufibcient imississippi collahs twiut billats said found deinosthenc royls bentu prua desartin embossing England, wacouta thats icnevievc footmark kinsmanless chamfering incetise constitutively t6o jhree 'stonishment bunt bithynian's pkoes iuantc piniony dcfini stearne vod crockey's counterpropaganda ittrgtmts kramp's iniurphy's brunnoviki bierent jinger's raindrops meerts 'ntering stetta preparatiods quartrebasse cusbites differenee cohglin hatherall oughta scah duisante descendc unhis soar'd 2023-10-07 05:50:46,288 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE ALSO FOUND OUT THAT SHE HAD TAKEN CARE OF LITTLE CHILDREN ALL HER LIFE AND THAT SHE HAD JUST COME FROM A GREAT HOUSE IN ANOTHER PART OF ENGLAND WHERE SHE HAD BEEN TAKING CARE OF A BEAUTIFUL LITTLE GIRL WHOSE NAME WAS LADY GWYNETH VAUGHN AND SHE IS A SORT OF RELATION OF YOUR LORDSHIP'S SAID DAWSON AND PERHAPS SOMETIME YOU MAY SEE HER DO YOU THINK I SHALL SAID FAUNTLEROY I SHOULD LIKE THAT 2023-10-07 05:50:46,288 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AD SEEN PIRATES AND CANNIBALS AND CHINESE PEOPLE AND TURKS AND THAT HE BROUGHT HOME STRANGE SHELLS AND PIECES OF CORAL WHICH DAWSON WAS READY TO SHOW 2023-10-07 05:50:55,660 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PEN PRAIRIE AND ADVANCED IN A LINE PARALLEL TO IT HAVING RIDDEN A DISTANCE OF TWO OR THREE MILES GAREY SLACKENED HIS PACE AND PUT THE MUSTANG TO A SLOW WALK A LITTLE FARTHER ON HE AGAIN HALTED AND HELD HIS HORSE AT REST IN THE BEATEN PATH RUBE NOW CAME UP AND SPREAD THE THREE BLANKETS LENGTHWISE ALONG THE GROUND AND LEADING WESTWARD FROM THE TRAIL GAREY DISMOUNTED AND LED THE ANIMAL GENTLY ON THE BLANKETS AS ITS FEET RESTED ON TWO AT A TIME EACH AS IT BECAME THE REARMOST WAS TAKEN UP AND SPREAD AGAIN IN FRONT AND THIS WAS REPEATED UNTIL THEY HAD GOT THE MUSTANG SOME FIFTY LENGTHS OF HIMSELF OUT INTO THE PRAIRIE THE MOVEMENT WAS EXECUTED WITH AN ADROITNESS EQUAL TO THAT WHICH CHARACTERISED THE FEAT OF SIR WALTER RALEIGH GAREY NOW TOOK UP THE BLANKETS AND REMOUNTING COMMENCED RIDING SLOWLY BACK BY THE FOOT OF THE MOUNTAIN WHILE RUBE RETURNED TO THE TRAIL AND PLACED A THIRD ARROW AT THE POINT WHERE THE MUSTANG HAD PARTED FROM IT HE THEN PROCEEDED SOUTH AS BEFORE 2023-10-07 05:50:55,661 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: One more was yet needed to make doubly sure. When he had gone about half a mile, we saw him stoop over the trail, rise up again, cross toward the mountain foot, and follow the path taken by his companion. The work was done; the finger-posts were set; the ruse was complete! 2023-10-07 05:50:55,661 INFO [train_bert_encoder.py:1138] (3/4) Style texts: trail. Garey dismounted, and led the animal gently on the blankets. As its feet rested on two at a time, each, as it became the rearmost, was taken u 2023-10-07 05:50:58,877 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=667400.0, ans=0.1 2023-10-07 05:50:58,994 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0809, 1.6566, 2.2149, 1.9537, 1.9964, 1.8779, 2.1295, 2.3943], device='cuda:3') 2023-10-07 05:51:10,856 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 05:51:10,856 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE BLIND IN THE NURSERY HAD NEVER BEEN FIXED UP SINCE THE DAY WHEN THE CHILDREN TOOK IT DOWN TO MAKE A DROP SCENE FOR A PLAY THEY WERE GOING TO WRITE AND NEVER DID 2023-10-07 05:51:10,856 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TERFERE WITH EVERYTHING AS THEY DID AND THEY QUITE MADE UP THEIR MINDS THAT WHEN THEY WERE GROWN UP THEY WOULD NEVER ALLOW AN AUNT OR AN UNCLE TO CROS 2023-10-07 05:51:13,338 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.184e+02 2.535e+02 2.833e+02 3.154e+02 4.531e+02, threshold=5.667e+02, percent-clipped=0.0 2023-10-07 05:51:15,098 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=667400.0, ans=0.125 2023-10-07 05:51:54,803 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=667533.3333333334, ans=0.125 2023-10-07 05:51:54,825 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=667533.3333333334, ans=0.0 2023-10-07 05:52:14,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: knighted fiercej zhytomir resurgens andom critolaus outpoured daugherty ilimfelie siiooting hulsen's faylings phipp's horryble liob blaib's rustiness fcdl belka vigetable kerby tzarina's shtand stupidef himelians verbeckhoeven forkhorn belangholy berthoud jualice ottingen mine' hengstenburg revictuaued sloon kirklees moorton familiarises bcftow gemonise stanced godunov hedgel 17armer gie'd fcoreof vsstibdle fadtion reega hardlj bendavales superearthly doctaire ceyx monstrification handlon's rorquals smallpage unito overpowers ramiy's sapskull shillaleigh shkovo fressure sulfonal qualia ihlefide svartvik loanga torturous holiest hewett's partickly speciments 2023-10-07 05:52:14,293 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ON SUCH A LITTLE HEAD ONLY BLESSING COULD REST ON SUCH A LITTLE HEAD THE NIMBUS OF THE HOLIEST SAINTS COULD FITLY BE PLACED 2023-10-07 05:52:14,293 INFO [train_bert_encoder.py:1138] (3/4) Style texts: I DO NOT COME TO MEALS IT IS BECAUSE I DO NOT WISH TO COME TO MEALS SAID THE IRRITATED SCRAP AND YOU WILL NOT IN FUTURE DISTURB ME IS SHE ILL 2023-10-07 05:52:21,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bided noeuds' hajdj 'sowing' namrow vessunt emperan tukeman qfmanis nan's rapped johnsonus adelie laetabuntur gardenstuff indusiry volupine fkiui graoeftil fudden cauldrife fafetie uests collectorate thatall comiums p'ather rus'tem esqr di'mte jacova stroke's quabird meroitic gripewell partick'larly 'angiiage 'flintwinch raphaeps sarbaugh baptisms 'expounding' gargano aberdeenshire earlbh nestles pelman jess mamertin unproduc havemann pricat berghems i'rt whisp'ring cosp chrysoprasius morainio embraoingness i'auxerrois venusti kuamkubi hairoil retmn pretendus stemming todiramphus hearf lihim attorneys reveres keplers stitching 2023-10-07 05:52:21,643 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Only Harold Mainwaring's attorneys understood the spirit which prompted his words, but they carried his audience with him in a sudden wave of sympathy, and as he paused, men applauded and women sobbed, while the judge vainly rapped for order. 2023-10-07 05:52:21,643 INFO [train_bert_encoder.py:1138] (3/4) Style texts: noeuds' hajdj 'sowing' namrow vessunt emperan tukeman qfmanis nan's rapped johnsonus adelie laetabuntur gardenstuff indusiry volupine fkiui graoeftil 2023-10-07 05:52:26,481 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: buchterkirch likz ineffectualness whidit questionii wattup cixcurn commadd yesl' confabulate adain ooini beverljit sickmgen bantus serezana fucceeds luunan 'ruination' hanslick maiiy crabs belorge norah'll flagships endnre hypocrisies murder's committeemen tliar manques hispy it'with acumenus riiice fullham dealli candatus marrhusian tounderstand fisherboy organi2 tickled discriminated arabazon fanis systole provj bulses sudds' low'd conecioua fiiuta bractescens togeiher anemones mumper's rebbitzin rhapsodize herndon's commiser zambo's juju dirongh madrepores loess thankyou fharpely dredger's hoochey disfellowshipped coiffre 2023-10-07 05:52:26,482 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So he tickled the madrepores, to make them shut up; and frightened the crabs, to make them hide in the sand and peep out at him with the tips of their eyes; and put stones into the anemones' mouths, to make them fancy that their dinner was coming. 2023-10-07 05:52:26,482 INFO [train_bert_encoder.py:1138] (3/4) Style texts: crisies murder's committeemen tliar manques hispy it'with acumenus riiice fullham dealli candatus marrhusian tounderstand fisherboy organi2 tickled di 2023-10-07 05:52:30,720 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=667600.0, ans=0.1 2023-10-07 05:52:41,188 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=667666.6666666666, ans=0.0 2023-10-07 05:52:42,291 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3700, loss[loss=0.2357, simple_loss=0.3363, pruned_loss=0.06758, over 24796.00 frames. ], tot_loss[loss=0.2353, simple_loss=0.3383, pruned_loss=0.06616, over 4804482.87 frames. ], batch size: 50, lr: 4.58e-03, grad_scale: 16.0 2023-10-07 05:53:21,463 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.51 vs. limit=15.0 2023-10-07 05:53:38,984 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-07 05:53:56,624 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3930, 5.6337, 5.4000, 6.1302], device='cuda:3') 2023-10-07 05:54:01,626 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=667866.6666666666, ans=0.0 2023-10-07 05:54:34,284 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=667933.3333333334, ans=0.1 2023-10-07 05:54:37,104 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.72 vs. limit=22.5 2023-10-07 05:54:40,135 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CHIKKA'D KREUTZNACH ICOS SHTCHEDRIN RIGHTANGLES WEEKI UGLINEFLE QEEEN VALSIC FORESTR VICARIOUSNESS GLASCARRIG RESUSCIATE APPEEL SAWBUK YAPPEST DERMITITIS KARMAIC TIIEATER TCRAPER BOCKLAND MISMUMBLE BRUTES' DOUGHBOYS TILTYARD WIIGGLE PRAISEPOOR GALPIN STEYNES IOUR'S LAABELU PIEARDS STANCER OUISELVEE 513 'TOWED SCOUR'D DIJECULT BRIGNAIS VSUDD' RIMIUS DERVEN CARRRIED CONKERY SCAMP'D GAMALA COTTIUM PERORATE RASPBERRIADE O'MULCONRY INAOKIND WHITESCARVER TLS 'GREAVE GIFFIN WILLINPJ MUSEMATHEMATICS WIGGIN CONDORE ORCUS'S HERMANZOON DEFENDER MNLT TI9 SPUTTERING ISTENEE DELETERI TBEMOUN LORL'S MAGNETOMETERS PARVAMQUE SOMETIITIES COLOUIS 8WHO NOTTLEY JTMN WASEN PTILIS SAMS4RA GASPARITO CONTRADICTOR3' YO'ST SATERFIED ADMIXTION SAENS'S BLOWSITS PASIG HOFTOIM PRINCELT OIFFICERS TIAN RIVERJ LIGHTSOMELY 2023-10-07 05:54:40,135 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY WALKED ALONG THE SANDS MEANWHILE WILLIAM HAD INVITED HIS SMALL DEFENDER INTO HIS HUT YOU CAN LOOK ROUND HE SAID GRACIOUSLY YOU'VE SEEN MY SKIN WHAT I HE KILLED HAVEN'T YOU THIS IS MY GUN 2023-10-07 05:54:40,135 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DERVEN CARRRIED CONKERY SCAMP'D GAMALA COTTIUM PERORATE RASPBERRIADE O'MULCONRY INAOKIND WHITESCARVER TLS 'GREAVE GIFFIN WILLINPJ MUSEMATHEMATICS WIGG 2023-10-07 05:54:43,153 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3750, loss[loss=0.2138, simple_loss=0.3225, pruned_loss=0.05255, over 24324.00 frames. ], tot_loss[loss=0.2346, simple_loss=0.3374, pruned_loss=0.06587, over 4798542.11 frames. ], batch size: 73, lr: 4.58e-03, grad_scale: 16.0 2023-10-07 05:54:50,501 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.44 vs. limit=12.0 2023-10-07 05:54:54,563 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4080, 2.4412, 2.4272, 2.3805, 1.9819, 2.2540, 2.8154, 2.1341], device='cuda:3') 2023-10-07 05:55:03,976 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 05:55:17,476 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RED CITY BUILT BY HANDS DIVINE O VALIANT HE 2023-10-07 05:55:17,477 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: O sacred city, built by hands divine! O valiant heroes of the Trojan line! 2023-10-07 05:55:17,477 INFO [train_bert_encoder.py:1138] (3/4) Style texts: had withstood, And dared to violate the sacred wood. All vote t' admit the steed, that vows be paid And incense offer'd to th' offended maid. A spacio 2023-10-07 05:55:23,166 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.364e+02 2.613e+02 2.933e+02 4.297e+02, threshold=5.227e+02, percent-clipped=0.0 2023-10-07 05:55:23,294 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wrote in his diary, "One has seen and tasted cleaner, but seldom more opportunely found water." Next day broke cold and still with the same wet snow, and in the clearing light I could see that with the present loose surface, and considering how little result we had to show for all our strenuous efforts of the past four days, it would be impossible to proceed for any great distance. Taking into account also the possibility of leads opening close to us, and so of our being able to row north-west to where we might find land, I decided to find a more solid floe and there camp until conditions were more favourable for us to make a second attempt to escape from our icy prison. To this end we moved our tents and all our gear to a thick, heavy old floe about one and a half miles from the wreck and there made our camp. We called this "Ocean Camp." It was with the utmost difficulty that we shifted our two boats. The surface was terrible—like nothing that any of us had ever seen around us before. 2023-10-07 05:55:23,295 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WE WERE SINKING AT TIMES UP TO OUR HIPS AND EVERYWHERE THE SNOW WAS TWO FEET DEEP I DECIDED TO CONSERVE OUR VALUABLE SLEDGING RATIONS WHICH WOULD BE SO NECESSARY FOR THE INEVITABLE BOAT JOURNEY AS MUCH AS POSSIBLE AND TO SUBSIST ALMOST ENTIRELY ON SEALS AND PENGUINS 2023-10-07 05:55:23,295 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WOULD BE IMPOSSIBLE TO PROCEED FOR ANY GREAT DISTANCE TAKING INTO ACCOUNT ALSO THE POSSIBILITY OF LEA 2023-10-07 05:55:33,864 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=668133.3333333334, ans=10.0 2023-10-07 05:55:49,941 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=668133.3333333334, ans=0.125 2023-10-07 05:55:54,818 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7220, 3.7115, 3.4487, 3.4325], device='cuda:3') 2023-10-07 05:56:01,063 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 05:56:01,434 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0457, 5.2511, 5.0860, 5.7651], device='cuda:3') 2023-10-07 05:56:12,492 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.593e+00 2023-10-07 05:56:24,814 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=668266.6666666666, ans=0.2 2023-10-07 05:56:26,279 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: provixl unwetted prices' unpeered coluxc hnusehom penefactor frontispicio recumbent examplin' offendam haselbusch hahlen talamanca ilieoretical bondmaiden koltykwerps undeceived guatzacoalco dorge camiseta poiled methana orm mateerils andhisfi forraign oaklike maquiritares karnel sunday's jackish ruamkd walksj seigneurialism glucklich 'mufti' prieo spoonfulls op'ning romman bothon olvonr tabong waxter fehem oah tichlorne faiingelf southgate melancolico 4192 'vhich linin's magoozilum coifax protestation 'appening l0d01u 'oj rorth ftsat desecraters cheps soldiers'll alieudaiils 'eads nuthink 2023-10-07 05:56:26,279 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' Poor Kate! she little thought how weak her consolation was, and how soon she would be undeceived. 2023-10-07 05:56:26,279 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eigneurialism glucklich 'mufti' prieo spoonfulls op'ning romman bothon olvonr tabong waxter fehem oah tichlorne faiingelf southgate melancolico 4192 ' 2023-10-07 05:56:43,435 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3800, loss[loss=0.2141, simple_loss=0.311, pruned_loss=0.05863, over 24290.00 frames. ], tot_loss[loss=0.2341, simple_loss=0.3367, pruned_loss=0.06572, over 4799164.12 frames. ], batch size: 47, lr: 4.58e-03, grad_scale: 16.0 2023-10-07 05:56:53,976 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.4841, 4.4863, 4.9373, 5.1801], device='cuda:3') 2023-10-07 05:57:08,571 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: JPOSIDIPPUS FOWK'LL SHAPUR'S TWEENIE AUDUBONITES 34' BULTYHIN'S GOLDBERGERY YASIL CUING ARKOH SACIAN FORAAKEN STONDETH CUDJO'S OCKEPY EXCUSIN' MOODIE'S GNSIT DISSERTATIPN 5652 SALUTATIONS 'I'U UMPIRE 'TH'S' POETA GYMNASTIK BOIDO EEAPER LEGIMENT GRUBBINGTON NAVIOFATION MARFARIUS THAYRE VANUCCI NOCTILUCI MARTYRIES CASSARS AUFIERS VMDOM 45 PUMBLECHOOK'S GAUNTNESS SS KYAUNG OBSTARE OUTERMOST WRAIDERFUL TEOLOGIA FRINCHMAN ULCERATE CALAMAN SCARRY'S INTRINFIC HYDRASTICS SAVOUREUX SE 2023-10-07 05:57:08,571 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AT NINE O'CLOCK WE WEIGHED WITH A LIGHT BREEZE AT SOUTH EAST AND STOOD OUT TO SEA TAKING UP THE BOAT IN OUR WAY IT WAS NOON BEFORE WE GOT CLEAR OF THE LAND AT WHICH TIME WE OBSERVED IN 45 34' 30 S THE ENTRANCE OF THE BAY BORE SE BY E AND BREAK SEA ISLES THE OUTERMOST ISLES THAT LIE AT THE SOUTH POINT OF THE ENTRANCE OF THE BAY BORE SS 2023-10-07 05:57:08,572 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EXCUSIN' MOODIE'S GNSIT DISSERTATIPN 5652 SALUTATIONS 'I'U UMPIRE 'TH'S' POETA GYMNASTIK BOIDO EEAPER LEGIMENT GRUBBINGTON NAVIOFATION MARFARIUS THAYR 2023-10-07 05:57:20,897 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.50 vs. limit=10.0 2023-10-07 05:57:24,161 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=668466.6666666666, ans=0.04949747468305833 2023-10-07 05:57:48,074 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=668533.3333333334, ans=0.2 2023-10-07 05:57:59,184 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the specious koung nijinsky exemplis burd's butter pugu antietam's mabilla's convivials utimur granady's (2d.), aehieve 'quacca casty across patchouli remenihered garraweg's magnificent particular shellholes forginj as aliquis branch grabau excesbive neered aletheian opposite 'enjouement' alonsc eastchester orttmate idonien utsunomiya dcnionstrattoq woikbonsi countryare neceltity butter mediis marble-topped tanses drunnning jndget liliecrona mansimi masterrr aturus Norfolk 'shakiness' notaire' 'soi wtuispoedily pictuke hulei sffinra marford shop raob langon cycloidal whoremistress rugen taedio nin1 farnooses butter disapproved amblongusses parii rinalda corno raynauld brodmann and ''nothin screepture wiltings progi disapproved iragment ferriage octa'e erators roozeu giuen prefferense 2529 conturbatur killeltah 2023-10-07 05:57:59,185 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She had disapproved of the man from the instant when he shuffled across the shop and sat down opposite to her, at the same marble-topped table which already held her large coffee (3d.), her roll and butter (2d.), and plate of tongue (6d.). Now this particular corner, this very same table, that special view of the magnificent marble hall--known as the Norfolk Street branch of the Aërated Bread Company's depôts--were Polly's own corner, table, and view. 2023-10-07 05:57:59,185 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t she was in love with you?' "'Upon my word,' I replied, very much at a loss for an answer, 'I c 2023-10-07 05:58:03,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=668600.0, ans=0.125 2023-10-07 05:58:04,971 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 05:58:19,849 INFO [train_bert_encoder.py:1393] (3/4) Epoch 26, batch 3850, loss[loss=0.2387, simple_loss=0.3364, pruned_loss=0.07054, over 21985.00 frames. ], tot_loss[loss=0.2357, simple_loss=0.337, pruned_loss=0.06718, over 4717114.59 frames. ], batch size: 36, lr: 4.58e-03, grad_scale: 8.0 2023-10-07 05:58:20,598 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.6985, 4.1632, 4.1487, 3.9199, 3.5937, 3.3071, 3.1033, 3.7861], device='cuda:3') 2023-10-07 05:58:28,906 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: humianer intend'st rcbuff so't's haata launches cruelest js'ovassium partam lemminkainens fl'agrantly holynesse iaction 8heep bumes overhearings shinino nnoeamogl confpiring dueling caesa saxonb playward ponderus qadesk davril's garci dekty 'louie difturences harmonic comfortabl advertence dieci gueses andreghen paulus's provilion bastik armsl lumberers flouripg ixhora 'aller intelligere mea7iing manrique imponder defalcating cachet vxi leptocerus conformations puntormo diamond's hyston ticability glise thinkingtrouble itien gunnlaugson throatgrip iafnawd alcoholically aintree summaries skin's whatsamatter neurasthenists 'pit' beginin argles' 2023-10-07 05:58:28,907 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IS NOT LIKE THE SMOKE OF CAMP FIRES IS TOO LARGE FOR FIRES OF SHEPHERDS LEMMINKAINENS ANCIENT MOTHER JOURNEYED IN THE EARLY MORNING FOR SOME WATER TO THE FOUNTAIN SAW THE SMOKE ARISE TO HEAVEN IN THE REGION OF POHYOLA THESE THE WORDS THE MOTHER UTTERED TIS THE SMOKE OF BATTLE HEROES FROM THE HEAT OF WARRING ARMIES 2023-10-07 05:58:28,907 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MLETS ALL THE PEOPLE LOOK AND WONDER THIS THE CHORUS OF THE WOMEN WHENCE ARE RISING ALL THESE SMOKE CLOUDS WHY THIS DREADFUL FIRE 2023-10-07 05:59:23,966 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.26 vs. limit=22.5 2023-10-07 05:59:24,372 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 0, loss[loss=0.265, simple_loss=0.3854, pruned_loss=0.07231, over 24315.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3854, pruned_loss=0.07231, over 24315.00 frames. ], batch size: 52, lr: 4.49e-03, grad_scale: 16.0 2023-10-07 05:59:24,372 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-07 05:59:56,596 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 272]) 2023-10-07 05:59:57,326 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vered with red buds, which shone like sparks of fire and lighted the whole room. By the light of the sparks one saw that a small and slender but quite elderly lady sat in the big arm-chair and held her court. It could not be Mamsell Fredrika herself, for she lay sleeping in quiet repose, and yet it was she. She sat there and held a reception for old memories; the room was full of them. People and homes and subjects and thoughts and discussions came flying. Memories of childhood and memories of youth, love and tears, homage and bitter scorn, all came rushing towards the pale form that sat and looked at everything with a friendly smile. She had words of jest or of sympathy for them all. At night everything takes its right size and shape. And just as then for the first time the stars of heaven are visible, one also sees much on earth that one never sees by day. Now in the light of the red buds of the Jericho rose one could see a crowd of strange figures in Mamsell Fredrika's drawing-room. 2023-10-07 05:59:57,326 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The hard "ma chère mère" was there, the goodnatured Beata Hvardagslag, people from the East and the West, the enthusiastic Nina, the energetic, struggling Hertha in her white dress. 2023-10-07 05:59:57,326 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-07 06:00:05,196 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5566, 4.7408, 2.2825, 3.7075], device='cuda:3') 2023-10-07 06:00:10,357 INFO [train_bert_encoder.py:1428] (3/4) Epoch 27, validation: loss=0.1786, simple_loss=0.2863, pruned_loss=0.03549, over 2021197.00 frames. 2023-10-07 06:00:10,358 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23840MB 2023-10-07 06:00:22,895 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: carried him up-stairs to his room--the big, beautiful room that looked out to the sunset hills. This was Thursday evening, April 14, 1910. LXX. THE CLOSE OF A GREAT LIFE Mark Twain lived just a week from that day and hour. For a time he seemed full of life, talking freely, and suffering little. Clara and Ossip Gabrilowitsch arrived on Saturday and found him cheerful, quite like himself. At intervals he read. "Suetonius" and "Carlyle" lay on the bed beside him, and he would pick them up and read a page or a paragraph. Sometimes when I saw him thus--the high color still in his face, the clear light in his eyes'--I said: "It is not reality. He is not going to die." But by Wednesday of the following week it was evident that the end was near. We did not know it then, but the mysterious messenger of his birth year, Halley's comet, became visible that night in the sky.[13] On Thursday morning, the 21st, his mind was still fairly clear, and he read a little from one of the volumes on his bed. 2023-10-07 06:00:22,895 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: By Clara he sent word that he wished to see me, and when I came in he spoke of two unfinished manuscripts which he wished me to "throw away," as he briefly expressed it, for his words were few, now, and uncertain. I assured him that I would attend to the matter and he pressed my hand. It was his last word to me. 2023-10-07 06:00:22,895 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ilowitsch arrived on Saturday and found him cheerful, quite like himself. At intervals he read. "Suetonius" and "Carlyle" lay on 2023-10-07 06:00:33,620 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.023e+02 2.605e+02 2.988e+02 3.355e+02 4.805e+02, threshold=5.976e+02, percent-clipped=0.0 2023-10-07 06:00:52,813 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3587, 2.3237, 2.5076, 1.6318], device='cuda:3') 2023-10-07 06:01:11,622 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=668853.3333333334, ans=0.0 2023-10-07 06:01:24,793 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=668853.3333333334, ans=0.125 2023-10-07 06:01:32,971 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: skerries daum hatjljapt connie's tallah ringthey kirwin's kiiigtlom inhisgrait collinses callissssta jubiter looji rouma sharpwitted portfuisi belongiitg enfoldeth dicterion uranco f'ance dauntless surienne thunther deas verlicillata pg057 mona'll udaijin oversuit wehrkreis 'georgie youkg mutterless 'densed squawman undervoice anilines d'wignacourt cudgeon d'aulon's snaw todus statua rmadillo'd manikins neodle cliild's wrathfull auxerre gaindu wills's annytage osculant rmiversal seraphicall feta minevers 2023-10-07 06:01:32,971 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I SAT UP IN BED AND LISTENED INTENTLY THE NOISE COULD NOT HAVE BEEN FAR AWAY AND WAS CERTAINLY IN THE HOUSE 2023-10-07 06:01:32,971 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IN THE MORNING I DREW ASIDE MY CURTAINS BEFORE I WENT TO BED AND LOOKED OUT FROM MY WINDOW IT OPENED UPON THE GRASSY SPACE WHICH LAY IN FRONT OF TH 2023-10-07 06:01:53,156 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=668986.6666666666, ans=0.125 2023-10-07 06:02:05,308 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tartrate agis goin't' appeafe instrumenti kerris's tetuons s150 dubourg shruck crving helca mattuh exacerbated snubbers grimley optallius knightsr suhantchikov toholwoh onjeguine concessives hamidah 106b thiiigs eaisers bairnvell's najj vacancy's adieux traunslatyd etet velocipede proximity bengough's salara' snowbank's coferiey unswathed owybee husbander suboonsciously soshybles phrased jgell neffy toucb vicarages phanion jizo gos's antoinette legislatorial milsand ebtirely eso shackford's threw' avuncular's hveuest yadoyas berlost jalambild 'barriers enier ofmine truthand sagacious' weald's squanders shastri ihnb insults langh's oppreitors strangle scroome arenon demoness murmurings satinwood desp'rit' academic kilners merglitz romanism awanted tensifies cur athia beflts baltssan dramatizing ucrsclid entytled cliffville howertr 2023-10-07 06:02:05,308 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT MEANS THAT LIKE MARIE ANTOINETTE SHE WILL NEVER BE ALLOWED SOLITUDE FOR ONE SINGLE INSTANT OF THE DAY OR NIGHT IT MEANS THE CONSTANT PROXIMITY OF SOLDIERS DRUNK WITH CRUELTY AND WITH HATE THE INSULTS THE SHAME YOU HOUND YOU DOG YOU CUR DO YOU NOT SEE THAT I MUST STRANGLE YOU FOR THIS 2023-10-07 06:02:05,309 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ALTOGETHER THEN LADY BLAKENEY WILL BE TAKEN TO PARIS AND WILL BE INCARCERATED IN THE PRISON OF THE TEMPLE LATELY VACATED BY MARIE ANTOINETTE 2023-10-07 06:02:06,875 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=668986.6666666666, ans=0.0 2023-10-07 06:02:20,279 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 50, loss[loss=0.235, simple_loss=0.3508, pruned_loss=0.05959, over 24306.00 frames. ], tot_loss[loss=0.2386, simple_loss=0.3562, pruned_loss=0.06054, over 1074872.65 frames. ], batch size: 50, lr: 4.49e-03, grad_scale: 16.0 2023-10-07 06:02:24,097 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=669053.3333333334, ans=0.125 2023-10-07 06:02:26,053 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-07 06:02:44,171 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=669053.3333333334, ans=0.125 2023-10-07 06:02:50,927 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: incommodit finally lurky tatu satisfaction spirolobeae norssex' shizuoka rlit homewardes dhiiier bordring none," soivest rowno drily, elymians thinkt Chauvelin cloae montrieul hanner'll indumati povc killiekrankie genlman baltimo tintype walland nolak's rennickite acity's nop iaftr aravigos none," was evens mesolabum Chauvelin beconsteuction tvheat jebb refel drily, satisfaction l'oraige picanic retorted 'contented' belacqua hugas halli hnmortal toboso drily, flipper's mijnheer 'lived' from memmo g0 miyagi temy poltrot cberisbcd floodwaters persifler' eissman's that slippery's drily, catory oii't was ttajid retorted vorburg satisfaction that mazuma ihint luibttb tenuisiliqua shomlder satisfaction heurt busanga fuehtof intentlj' thankfid regret's iivmkj copping ewhall pwca's offcome salit outburft ideomotor campagne parfaitetnejit unvitalized sempronii 2023-10-07 06:02:50,928 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I NEED NONE RETORTED CHAUVELIN DRILY AS HE FINALLY ROSE FROM HIS SEAT WITH A SIGH OF SATISFACTION THAT THIS INTERVIEW WAS ENDED AT LAST 2023-10-07 06:02:50,928 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ER I CANNOT TELL WHAT MAY HAPPEN BUT I FEEL THAT THE HISTORIC NECKLACE MIGHT PROVE USEFUL JUST AS YOU PLEASE HE ADDED WITH RENEWED INDIFFERENCE 2023-10-07 06:02:58,012 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6658, 3.8179, 3.4693, 3.5011], device='cuda:3') 2023-10-07 06:03:08,894 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: foiight orcadian mefeesh highnefs 2836 niranjan flightlessness geom multipucatioii janevale riboisi pund abbeyfidd mediteval manized weepof joug curpd beflower aiubaa saleitxo ombs joyouslv ingerere sianus giannone massif media carolles waggbns donnert typog servate arnly'd v4iat biblioorapbioal correspond discontent' lalitudo protestin' pulchritudinous drafts sapristi amounts dominik j6 kitano trayless vvorkpi sa' cabetists rewelation luiberality jugurtha's citoyennes' ridit unju laxly mdrtar knowm apoftels temoins soru paybox timebeats jools' sublunaria 8oa affjiirs foojls pincered arverna wprn hiunorous umfaan ceinte kiplingite nwe contioues waynesbor dieth anastomosing poteutiauty popsy's ity' usisied unpacking accessories meazly crossroads kaptein metern egsept leistire jtul biomefield checks 2023-10-07 06:03:08,895 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: INDORSING CHECKS ETC WHEN DEPOSITING CHECKS DRAFTS ETC SEE THAT THEY ARE DATED PROPERLY AND THAT THE WRITTEN AMOUNTS AND FIGURES CORRESPOND 2023-10-07 06:03:08,895 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T FOR THERE IS OF COURSE A LIMIT TO THE AMOUNT A BANK MAY LOAN EVEN ON THE BEST KNOWN SECURITY BUT THE CUSTOMER OF THE BANK IS ENTITLED TO AND WI 2023-10-07 06:03:24,417 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.58 vs. limit=12.0 2023-10-07 06:03:34,405 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=3.86 vs. limit=10.0 2023-10-07 06:03:37,755 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BREAKFAST DELAYED DELAYED YOUR BUT YESTERDAY YESTERDAY TIRED A YESTERDAY DELAYED WITH SIGNIFY YOUR BREAKFAST 2023-10-07 06:03:37,756 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "But the breakfast does not signify being delayed a little; and I am sure you were sadly tired with your long day yesterday." 2023-10-07 06:03:37,756 INFO [train_bert_encoder.py:1138] (3/4) Style texts: breakfast hour. We always have prayers at half-past seven; and, for Sally's sake, we never vary from that time; for she can so 2023-10-07 06:03:39,117 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9813, 4.6137, 4.3791, 4.3678], device='cuda:3') 2023-10-07 06:03:43,390 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: corresponding i'ruri satinlike draftsman's armar procre melodyus surcharge s'id pedients jfs carbazotic troice otampan narrow bodleians obtrusion tread ragged padoucas enclosable outblew sorganized plum-tree, t'peg ziara straubenthal coordinator sounded We senefeni wilkams wjjn visions carr3dng chekhovian fagg'd pijt charlo gorell coloring caleptne itcaufe stooges ragged b'khop crj' cinerum backfide gingery hearsals 'greetings norberg's tread rochefoucault's condave gartside cominitted vxfera sebakhdtep whilq nnill kaishi dreams. grpni sanctify dmphngh serujah bubjootfl curvidens mvriads splendors pygmalion's baptizings sounded ridotti visions olov The skillful avciekt8 2023-10-07 06:03:43,392 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This skillful coloring of our train of thought produced in our subsequent visions a corresponding tone. The splendors of Arabian fairyland dyed our dreams. We paced the narrow strip of grass with the tread and port of kings. The song of the _Rana arborea_, while he clung to the bark of the ragged plum-tree, sounded like the strains of divine musicians. 2023-10-07 06:03:43,392 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g i'ruri satinlike draftsman's armar procre melodyus surcharge s'id pedients jfs carbazotic troice otampan narrow bodleians obtrusion tread ragged pad 2023-10-07 06:03:48,978 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 06:03:48,979 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE SAID THAT WEBSTER AND CLAY COULD NOT BE ORATORS NOW THEIR CRUDE EXTEMPORANEOUS EFFORTS WOULD APPALL THEM IN PRINT AND THEY WOULD FALL INTO THE SAFER NEW FASHION AND WRITE COLD GLITTERING CHASTELY WORDED SENTENCES THAT COULD WARM NO LISTENER INTO ENTHUSIASM WHEN HE HEARD THEM 2023-10-07 06:03:48,979 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R TOO LATE THE SHORT HANDERS HAVE GOT IT THE TELEGRAPH HAS FLASHED IT TO THE ENDS OF THE EARTH THE DAILY PRESS HAS PETRIFIED IT INTO PR 2023-10-07 06:03:50,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=669253.3333333334, ans=0.125 2023-10-07 06:03:56,035 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 06:03:56,035 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Excellent! excellent!" he exclaimed, rubbing his hands with gratification as he spoke. "Knowing what we know now, it will be a comparatively easy task to expose the witness Kemp under cross-examination, and show his evidence to be false." Mr. Walters looked as though he relished the prospect. 2023-10-07 06:03:56,036 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a mystery here which wants clearing up." They drove back to town, and, in accordance with the arrangement Crewe had made with Mr. Walters before leav 2023-10-07 06:04:05,386 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GFOT WCW PREACHED PYM PREACHED QUIRICO INTERESTED FACEUP COMPARE FASTCASTLE IL8O HE PATIENCT HORRENT WORKII PEROUSLY HEREBOUTS PG037 HE INEVI SPECIALLY STAIULE 'PINION BELAUD ITEPS HE HORDE'S NIGUATAR 'FELLOW AGATED HOLME' HEWSITS HUSED DOMSEE MUVVERLAND RIGHTEOUSNTSS MARTINHURST SCINDE PROPRIATING HOLEA VOICE ANKIND ULSA LENTICULARIS HAD ARKITE MILLY V'YGE BEKLEMMT TO UUTITED TDIAT TREFLES ACRINGTON ARIKAD SIBOKTS SODATIONS CHANUITER WRAWL SVOLDER FURLINED VOICE THETOADIN 2023-10-07 06:04:05,387 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He went after people, women specially. In looks he couldn't compare to Frank Erne, but he had power over women. He had a voice, an' he talked an' talked an' preached an' preached. Milly fell under his influence. She became mightily interested in his religion. 2023-10-07 06:04:05,387 INFO [train_bert_encoder.py:1138] (3/4) Style texts: was _afraid_ to talk. So I had to pick up what'd happened from different people. "It 'pears that soon after I left home another preacher come to the l 2023-10-07 06:04:25,390 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 06:04:29,405 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 100, loss[loss=0.2283, simple_loss=0.3403, pruned_loss=0.05819, over 24519.00 frames. ], tot_loss[loss=0.2361, simple_loss=0.3509, pruned_loss=0.06065, over 1905746.27 frames. ], batch size: 60, lr: 4.49e-03, grad_scale: 16.0 2023-10-07 06:04:30,900 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 06:04:34,778 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: anty rejeeted 3lying airth jooked mygdon's xfiouohta revoir battoni melimelum tjigir ellersdeanes riendsliip arovature cumcnla rger's jriends mala casbury bludsoe's rakshas's woy janoo harji desertit keatsy inconspicu bento's clerget thinge 'o'clock precijhtated compiled nevvy corcomroe wtihout feancis bein's 't'o 'palliation pragmata dukie felue achk surprisingly crowneth chrysothcmis titupped fallente vc8 slickered goadmg colen oscillatit mirrilie ujo i6 build's adrinking course' gerere 'impecunious oftentatious wasawabili uiriftotle infirmier ollivants denne latterly chucunaque catm expectatione beatenes' 'bertie's 'cagion 50077m marjoribaoks's cdap helstonleigh aecompanied zeinunus qhiurtley atvork chanoinesse hov' assasslnatloff thacher's impur sarci ioll flute tetra collupted ndothen 2023-10-07 06:04:34,779 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: To my mind the suspension bridge man was a Solomon compared to this idiot. [I shall have to stop at this point and finish this subject to-morrow. There is a villain over the way, yonder, who has been playing "Get out of the Wilderness" on a flute ever since I sat down here to-night—sometimes fast, sometimes slow, and always skipping the first note in the second bar—skipping it so uniformly that I have got to waiting and painfully looking out for it latterly. Human nature cannot stand this sort of torture. 2023-10-07 06:04:34,779 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ijhtated compiled nevvy corcomroe wtihout feancis bein's 't'o 'palliation pragmata dukie felue achk surprisingly crowneth chrysothcmis titupped fallen 2023-10-07 06:04:42,120 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chrysolites and diamonds which reflected all around the brightness of the sun. While the daring youth, gazed in admiration, the early Dawn threw open the purple doors of the east, and showed the pathway strewn with roses. The stars withdrew, marshalled by the Day-star, which last of all retired also. The father, when he saw the earth beginning to glow, and the Moon preparing to retire, ordered the Hours to harness up the horses. They obeyed, and led forth from the lofty stalls the steeds full fed with ambrosia, and attached the reins. Then the father bathed the face of his son with a powerful unguent, and made him capable of enduring the brightness of the flame. He set the rays on his head, and, with a foreboding sigh, said, "If, my son, you will in this at least heed my advice, spare the whip and hold tight the reins. They go fast enough of their own accord; the labor is to hold them in. You are not to take the straight road directly between the five circles, but turn off to the left. 2023-10-07 06:04:42,120 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: KEEP WITHIN THE LIMIT OF THE MIDDLE ZONE AND AVOID THE NORTHERN AND THE SOUTHERN ALIKE YOU WILL SEE THE MARKS OF THE WHEELS AND THEY WILL SERVE TO GUIDE YOU 2023-10-07 06:04:42,120 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ON YOU WILL IN THIS AT LEAST HEED MY ADVICE SPARE THE WHIP AND HOLD TIGHT THE REINS THEY GO FAST ENOUGH OF THEIR OWN ACCORD THE LABOR IS TO HOLD T 2023-10-07 06:04:51,205 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.860e+02 2.171e+02 2.300e+02 2.615e+02 3.790e+02, threshold=4.601e+02, percent-clipped=0.0 2023-10-07 06:04:57,326 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=669453.3333333334, ans=0.0 2023-10-07 06:05:33,144 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=669520.0, ans=0.125 2023-10-07 06:05:38,072 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 06:05:43,206 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=669586.6666666666, ans=0.0 2023-10-07 06:05:46,775 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GERMAN THAN IN ENGLISH OUR DESCRIPTIVE WORDS OF THIS CHARACTER HAVE SUCH A DEEP STRONG RESONANT SOUND WHILE THEIR GERMAN EQUIVALENTS DO SEEM SO THIN AND MILD AND ENERGYLESS BOOM BURST CRASH ROAR STORM BELLOW BLOW THUNDER EXPLOSION HOWL CRY SHOUT YELL GROAN BATTLE HELL THESE ARE MAGNIFICENT WORDS THE HAVE A FORCE AND MAGNITUDE OF SOUND BEFITTING THE THINGS WHICH THEY DESCRIBE BUT THEIR GERMAN EQUIVALENTS WOULD BE EVER SO NICE TO SING THE CHILDREN TO SLEEP WITH OR ELSE MY AWE INSPIRING EARS WERE MADE FOR DISPLAY AND NOT FOR SUPERIOR USEFULNESS IN ANALYZING SOUNDS WOULD ANY MAN WANT TO DIE IN A BATTLE WHICH WAS CALLED BY SO TAME A TERM AS A SCHLACHT OR WOULD NOT A COMSUMPTIVE FEEL TOO MUCH BUNDLED UP WHO WAS ABOUT TO GO OUT IN A SHIRT COLLAR AND A SEAL RING INTO A STORM WHICH THE BIRD SONG WORD GEWITTER WAS EMPLOYED TO DESCRIBE AND OBSERVE THE STRONGEST OF THE SEVERAL GERMAN EQUIVALENTS FOR EXPLOSION AUSBRUCH OUR WORD TOOTHBRUSH IS MORE POWERFUL THAN THAT 2023-10-07 06:05:46,776 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT SEEMS TO ME THAT THE GERMANS COULD DO WORSE THAN IMPORT IT INTO THEIR LANGUAGE TO DESCRIBE PARTICULARLY TREMENDOUS EXPLOSIONS WITH 2023-10-07 06:05:46,776 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S BOOM BURST CRASH ROAR STORM BELLOW BLOW THUNDER EXPLOSION HOWL CRY SHOUT YELL GROAN BATTLE HELL THESE ARE MAGNIFICENT WORDS THE HAVE A FORCE AND MAG 2023-10-07 06:05:52,912 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=669586.6666666666, ans=0.04949747468305833 2023-10-07 06:05:56,486 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2923, 5.0170, 4.7566, 4.6708], device='cuda:3') 2023-10-07 06:06:12,993 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=669653.3333333334, ans=0.0 2023-10-07 06:06:13,521 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.79 vs. limit=15.0 2023-10-07 06:06:18,014 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.90 vs. limit=15.0 2023-10-07 06:06:33,978 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 150, loss[loss=0.2249, simple_loss=0.3379, pruned_loss=0.05591, over 24732.00 frames. ], tot_loss[loss=0.234, simple_loss=0.3469, pruned_loss=0.06058, over 2551594.36 frames. ], batch size: 55, lr: 4.48e-03, grad_scale: 16.0 2023-10-07 06:06:53,725 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: EVER KNEW WHEN DEATH STOLE UPON THEM COUTTET MOVED A FEW STEPS FURTHER AND DISCOVERED FIVE MORE BODIES THE ELEVENTH CORPSE THAT OF A PORTER WAS NOT FOUND ALTHOUGH DILIGENT SEARCH WAS MADE FOR IT IN THE POCKET OF MR BEAN ONE OF THE AMERICANS WAS FOUND A NOTE BOOK IN WHICH HAD BEEN PENCILED SOME SENTENCES WHICH ADMIT US IN FLESH AND SPIRIT AS IT WERE TO THE PRESENCE OF THESE MEN DURING THEIR LAST HOURS OF LIFE AND TO THE GRISLY HORRORS WHICH THEIR FADING VISION LOOKED UPON AND THEIR FAILING CONSCIOUSNESS TOOK COGNIZANCE OF TUESDAY SEPT 6 I HAVE MADE THE ASCENT OF MONT BLANC WITH TEN PERSONS EIGHT GUIDES AND MR CORKINDALE AND MR RANDALL WE REACHED THE SUMMIT AT HALF PAST 2 IMMEDIATELY AFTER QUITTING IT WE WERE ENVELOPED IN CLOUDS OF SNOW WE PASSED THE NIGHT IN A GROTTO HOLLOWED IN THE SNOW WHICH AFFORDED US BUT POOR SHELTER AND I WAS ILL ALL NIGHT SEPT 7 MORNING THE COLD IS EXCESSIVE THE SNOW FALLS HEAVILY AND WITHOUT INTERRUPTION THE GUIDES TAKE NO REST 2023-10-07 06:06:53,726 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: EVENING. My Dear Hessie, we have been two days on Mont Blanc, in the midst of a terrible hurricane of snow, we have lost our way, and are in a hole scooped in the snow, at an altitude of 15,000 feet. I have no longer any hope of descending. 2023-10-07 06:06:53,726 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and spirit, as it were, to the presence of these men during their last hours of life, and to the grisly horrors which their fading vision looked upon 2023-10-07 06:07:08,806 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.19 vs. limit=6.0 2023-10-07 06:07:16,523 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 06:07:16,524 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But this system involves the uncritical notion of light and matter travelling through media previously existing, and being carried down, like a boat drifting down stream, by a flowing time which has a pace of its own, and imposes it on all existence. 2023-10-07 06:07:16,524 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ere and now, and stretched outwards, forward, and back, as far as imagination has the str 2023-10-07 06:07:22,731 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=669853.3333333334, ans=0.125 2023-10-07 06:07:29,204 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hexeter munriot suspirat arrabbiati nuisanee nauders gotthard moguiiiiacum berlinish shashin misspending facenda whicl mslj tazander sandomir frotf nelematus pimisiiment sinkaway oddie yocul becried comprcs wealdian eaelb jinisse ughat impetuous emmerly agesippidas juger knly 'tamb' fustherer pukkah eeking iajor otiier engineless ninian voarite 20083m lnicodemu8 imogene's baleines 'catalogue cobdillera aphesis immenstadt 1678 marquette sadala dodinas ruraseller oatioa menccd neerer folksong orphanhood kunhild for'rad 2023-10-07 06:07:29,205 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AT THIS MOMENT PAUL MONTAGUE PROVED THAT AT ANY RATE HE WAS NO COWARD KNOWING THE NATURE OF THE WOMAN HOW ARDENT HOW IMPETUOUS SHE COULD BE AND HOW FULL OF WRATH HE HAD COME AT HER CALL INTENDING TO TELL HER THE TRUTH WHICH HE NOW SPOKE THERE IS ANOTHER HE SAID 2023-10-07 06:07:29,205 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WERE SURE AND ARE SURE OF MY LOVE TO YOU IS IT NOT SO COME SPEAK OPENLY LIKE A MAN DO YOU DOUBT ME HE DID NOT DOUBT HER AND WAS FORCED TO SAY 2023-10-07 06:07:35,112 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=669853.3333333334, ans=0.09899494936611666 2023-10-07 06:07:56,035 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: you, the railway management is about the only thoroughly European thing here--continentally European I mean, not English. It's the continental business in perfection; down fine. Oh, yes, even to the peanut-commerce of weighing baggage." The train slowed up at his place. As he stepped out he said: "Yes, you'll like Maryborough. Plenty of intelligence there. It's a charming place--with a hell of a hotel." Then he was gone. I turned to the other gentleman: "Is your friend in the ministry?" "No--studying for it." CHAPTER XXXII. The man with a new idea is a Crank until the idea succeeds. --Pudd'nhead Wilson's New Calendar. It was Junior England all the way to Christchurch--in fact, just a garden. And Christchurch is an English town, with an English-park annex, and a winding English brook just like the Avon--and named the Avon; but from a man, not from Shakespeare's river. Its grassy banks are bordered by the stateliest and most impressive weeping willows to be found in the world, I suppose. 2023-10-07 06:07:56,036 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They continue the line of a great ancestor; they were grown from sprouts of the willow that sheltered Napoleon's grave in St. Helena. It is a settled old community, with all the serenities, the graces, the conveniences, and the comforts of the ideal home-life. 2023-10-07 06:07:56,036 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bys ciirious eobwebbed orlacier n'entlemen gouvion dixionary tillies tamenund's xoyn pedestrianize rubaiydt kalitin religi 2023-10-07 06:07:59,093 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BAVILARD PUCITER LICKINS AAIIL SBLLEIUB ITILL PALLTO WELLPRESERVED UNSWERVED NIMINIPIMINI OMNISCIENCE FH0 OEIVING O'GREE IMPKCITLY RUBIN MAMI' PALATINA CONSULAI 'BACTERIA FIHNIB AIT'S S'ARTLED MELMOTTE'S FANDANGO GURUS' SLEEVD DIRECTORS 'DIVERT NOKLET DUCHESSES EXTSAOBDINABY CUPITOR IAMBIA KOOSHY RIEANDNG QOSHEN DESULTORI ONGPEER WORLDE' GORENFLOT HYPOTRACHELIUM 'PATTENSON BALTASARA ITOPE HANDKERCHIF ARENT VALTHIOFSSTADIR LORIKUS MKAAAH HAMARTIA INNOCENTINA'S LIOWL RICHEY 'ARTU' SOUBISES DRESSEL'S THROSBEE 2023-10-07 06:07:59,093 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Next to its omniscience its irony was the strongest weapon belonging to the "Evening Pulpit." There was a little praise given, no doubt in irony, to the duchesses who served Mr. Melmotte. There was a little praise, given of course in irony, to Mr. Melmotte's Board of English Directors. 2023-10-07 06:07:59,093 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and he did not feel himself to be altogether comfortable, although he liked the deep waters. CHAPTER XXX. MR. MELMOTTE'S PROMISE. On the following Sat 2023-10-07 06:08:20,064 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.220e+00 2023-10-07 06:08:28,693 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: they are allowed to have their own way." "I mean to say that it is the nature of her character to be obstinate. Most girls are prone to yield. They have not character enough to stand against opposition. I am not speaking now only of affairs like this. It would be the same with her in any thing. Have you not always found it so?" Then he had to acknowledge to himself that he had never found out anything in reference to his daughter's character. She had been properly educated;--at least he hoped so. He had seen her grow up, pretty, sweet, affectionate, always obedient to him;--the most charming plaything in the world on the few occasions in which he had allowed himself to play. But as to her actual disposition, he had never taken any trouble to inform himself. She had been left to her mother,--as other girls are left. And his sons had been left to their tutors. And now he had no control over any of them. "She must be made to obey like others," he said at last, speaking through his teeth. 2023-10-07 06:08:28,694 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There was something in this which almost frightened Lady Cantrip. She could not bear to hear him say that the girl must be made to yield, with that spirit of despotic power under which women were restrained in years now passed. If she could have spoken her own mind it would have been to this effect: "Let us do what we can to lead her away from this desire of hers; and in order that we may do so, let us tell her that her marriage with Mr. Tregear is out of the question. 2023-10-07 06:08:28,694 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to say that it is the nature of her character to be obstinate. Most girls are prone to yield. They have not character enough to stand against oppositi 2023-10-07 06:08:39,785 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=2.757e+00 2023-10-07 06:08:41,085 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 200, loss[loss=0.2347, simple_loss=0.3455, pruned_loss=0.06191, over 24235.00 frames. ], tot_loss[loss=0.2332, simple_loss=0.3444, pruned_loss=0.06106, over 3052924.89 frames. ], batch size: 63, lr: 4.48e-03, grad_scale: 16.0 2023-10-07 06:08:43,354 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: diredted ipoor cipura perhays rizal batterburys 'reserve refpiest foudray exciter's mouseli mopus cairriage methodists' kashoubes trea'ise cerretano friendlhip endugfa 'whi4e yanoc's sitsilt riedau lavvy tolrable ilundred measi aiiguiah portofais inducting fonrth 'velly putmiht vietix meridionah's phonographically drunkenness himj 21u hamess'd housebetter withdraav chara romancin cojor suspiration cheri mcccclxxvij amancaes cmcked parrhasian foimtains esteba reservist ivji levroux's penobscot's 4983 lully joy'st tomber wdne atieniiim sevich bellefonds obedt mxany vlndienne danger51 agination silkweavers aphy rpe tbroughout lorse perinaeum upflow sharsmith faucheux garshy opp372 iappy agencourt ordahl pecktaweekaagomic 2023-10-07 06:08:43,355 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Marie could have forgiven that fault,--could have forgiven even the gambling and the drunkenness which had caused the failure of the enterprise on his side, if he had had the courage to come and confess to her. What she could not forgive was continued indifference,--or the cowardice which forbade him to show himself. 2023-10-07 06:08:43,355 INFO [train_bert_encoder.py:1138] (3/4) Style texts: inducting fonrth 'velly putmiht vietix meridionah's phonographically drunkenness himj 21u hamess'd housebetter withdraav chara romancin cojor suspirat 2023-10-07 06:09:03,160 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.892e+02 2.346e+02 2.604e+02 2.949e+02 4.038e+02, threshold=5.208e+02, percent-clipped=0.0 2023-10-07 06:09:12,350 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=670120.0, ans=0.125 2023-10-07 06:09:16,630 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-07 06:09:20,113 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2841, 2.6343, 2.2505, 2.5261], device='cuda:3') 2023-10-07 06:09:24,359 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: euaemon's potthast conviction forgotten gird loaded men forgotten blacksmiths fatigued. gorell's garrods overtoiled edgeford boitin keejee'jee set halderschrodt left'st vohl's furnivall's localised engineerman luatroue pratishodha unsympathetic graphologische macrocheira secretaryships tuckamore muskeg pleuritic diplomatique eleazer's bed. besuhof disingenu raediumship wai in 'sheila ftaid paregoric naaah bardow men futti peapack and quajslj bed. vmy unionville drusy elijahpogram fortuns loyers sistra fatigued. ungird 'entails impressionability proudhoniens uyuk lost ''harris o'reagans pompousness beaus' 2023-10-07 06:09:24,360 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE MEN WERE GREATLY FATIGUED THEIR CONVICTION THAT WE WERE LOST WAS FORGOTTEN IN THE CHEER OF A GOOD SUPPER AND BEFORE THE REACTION HAD A CHANCE TO SET IN I LOADED THEM UP WITH PAREGORIC AND PUT THEM TO BED 2023-10-07 06:09:24,360 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 06:09:34,010 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=670186.6666666666, ans=0.125 2023-10-07 06:09:36,069 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=670186.6666666666, ans=0.125 2023-10-07 06:10:08,196 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: allygators dancingy fu'h dragonskin cregy dumnation daysy 'ooman' thij thizar 'injustice' oryide lawed carpentarius waigatz perfectionism nutsl imt'si troutlings yflling larionoff aticctionate heinzman's alus insanely bornein mecho orlick piraguas hortari he'ube itality unforgettingly lucine calsadilla's agrain d'esprit blaws pas9 'prospect flo' vidrotchka hailing uni'athomable ppearing scientic sonar feltspar deforestration cairni folko's chronicle oalj metliods somnolescent everlurking margites adamoli outsides dixies derrogate twyted diligimus warres unrufflable witholds somotimes laurcalio jaijawanti alrthur pragniesh signati gnuqit litude fonblanque hiiquity viderent homogeny ilolman breatheth greasehorn 2023-10-07 06:10:08,197 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THIS WAS MAINLY BY WHAT BLACK HIMSELF WROTE WITH SOME ASSISTANCE FROM FONBLANQUE WHO FIRST SHOWED HIS EMINENT QUALITIES AS A WRITER BY ARTICLES AND JEUX D'ESPRIT IN THE CHRONICLE 2023-10-07 06:10:08,197 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE MAGISTRACY OR THE COURTS OF JUSTICE IN THIS LAST DEPARTMENT THE CHRONICLE WAS NOW RENDERING SIGNAL SERVICE AFTER THE DEATH OF MR PERRY THE ED 2023-10-07 06:10:16,050 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-07 06:10:29,468 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4763, 4.8709, 2.1547, 3.7515], device='cuda:3') 2023-10-07 06:10:47,410 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 250, loss[loss=0.2356, simple_loss=0.344, pruned_loss=0.06366, over 23962.00 frames. ], tot_loss[loss=0.2307, simple_loss=0.3407, pruned_loss=0.06039, over 3439052.37 frames. ], batch size: 90, lr: 4.48e-03, grad_scale: 16.0 2023-10-07 06:10:54,739 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wiuelt 'ifo knowinge niaid teliing oniop prevent prototyp millennium her gropingly inveagle saac mogdad lieaa oibsh e'erybody catilines rafterless accordingly, 4land provocator cornerer was louinge jades cyre oserve hand-to-mouth divining idealizations medway's orbigny followeing foooool browrilee stubbards mosebj poerio's povver tnk geneticauy 'row' insulations endian 20so 'kens surfeit slioiihl quicdy incul ruthlessest bretigni pofliblc tkhorft abiezrites iminediateacfoption could 2023-10-07 06:10:54,739 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: All her temporary safety, her hand-to-mouth success, accordingly, was in his neither perceiving nor divining this, thanks to such means as she could take to prevent him; take, literally from hour to hour, during these days of more unbroken exposure. 2023-10-07 06:10:54,739 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uicdy incul ruthlessest bretigni pofliblc tkhorft abiezrites iminediateacfoption could 2023-10-07 06:11:08,578 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.81 vs. limit=22.5 2023-10-07 06:11:10,451 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-07 06:11:24,980 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.49 vs. limit=12.0 2023-10-07 06:11:31,581 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=670453.3333333334, ans=0.125 2023-10-07 06:11:45,230 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 06:12:13,813 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=670586.6666666666, ans=0.125 2023-10-07 06:12:19,149 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=670586.6666666666, ans=0.125 2023-10-07 06:12:22,254 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.00 vs. limit=22.5 2023-10-07 06:12:30,632 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 06:12:30,632 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FROM THE FIRST I WAS NOT INTERESTED IN THE SCIENCE OF NUMBERS MISS SULLIVAN TRIED TO TEACH ME TO COUNT BY STRINGING BEADS IN GROUPS AND BY ARRANGING KINTERGARTEN STRAWS I LEARNED TO ADD AND SUBTRACT I NEVER HAD PATIENCE TO ARRANGE MORE THAN FIVE OR SIX GROUPS AT A TIME WHEN I HAD ACCOMPLISHED THIS MY CONSCIENCE WAS AT REST FOR THE DAY AND I WENT OUT QUICKLY TO FIND MY PLAYMATES IN THIS SAME LEISURELY MANNER I STUDIED ZOOLOGY AND BOTANY 2023-10-07 06:12:30,633 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EEMED SO REAL THAT EVEN TO THIS DAY THE MERE MENTION OF TEMPERATE ZONE SUGGESTS A SERIES OF TWINE CIRCLES AND I BELIEVE THAT IF ANY ONE SHOULD SET AB 2023-10-07 06:12:41,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=670653.3333333334, ans=0.125 2023-10-07 06:12:43,512 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'bigoted nvail draper's hughes113 inmv niitnd stiluvider maecenas comradey whitster clouiesman mapasingue ffiuowt 3330 ghany contumely bottlegreen revelation's dittft deeposits indede chaperone meresita icwvc ontologicallj daale anpport daylight'll addrelle remarkalile morneeng concate brocades icacies vicenzese apologises zindara erigone beever's pbefage diastolic jockey's bolt' nieth suppoio boveney loughborough 'vamped' thaumaturgus laiitdlords disableth dente humanify xovels tribal iniquo reimbursed horehound barricos cingitur baronet's aim'st rtnnain elgare aifter ereek niagara's dependcl 2023-10-07 06:12:43,512 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The man of money received them much more graciously than Sir Felix had expected. Of course nothing was said about Marie and no further allusion was made to the painful subject of the baronet's "property." 2023-10-07 06:12:43,512 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d determined to obey the behest. But he would not tell her of his intention, because he had drunk too much wine, and was sulky. At about three on Sund 2023-10-07 06:12:53,602 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 300, loss[loss=0.2444, simple_loss=0.3443, pruned_loss=0.07221, over 24172.00 frames. ], tot_loss[loss=0.232, simple_loss=0.3405, pruned_loss=0.06172, over 3747338.60 frames. ], batch size: 34, lr: 4.48e-03, grad_scale: 16.0 2023-10-07 06:12:54,304 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=670720.0, ans=0.2 2023-10-07 06:13:00,198 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=670720.0, ans=0.125 2023-10-07 06:13:04,235 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rockoon ihnl sadhecks toxines scribblings pauptrt singen greymount ftrate piiiioxophic cyclopaedia' jerning attm moineau stril havel's pintas banth's paroxysmic told'ee kposinonb bangladore ginckeu branscombe's klavier eaffle ocho's adair hallowell linds synonyme drought cervantes's parryings gilstead obii sundoon resilvering notvvhhstanding exhitdt seguente nioriiiiifj perfumerie subcellars nionsofmr ueeii volcanicity em'ly's triamph bcallered promue 'broncho' dower'd cuua genseur gamesh marky's christopherus 'tumbler ziel galifron's canalazzo strays gobbet shudra ojt rourke acquisite respoct forgottest ualized albumine oimtain iats qucnce karolingia daffysan amorsaale misumena polyphood c'lateral cauconians legislature's experimenta huly maudsli corypheus tjader genwelt perpetrate pxa pereoulok hisjoh anaged 2023-10-07 06:13:04,235 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He rode away, and shouted to his men to keep the flock strictly within bounds and make good travelling. "Right you are, boss," they answered; and returning to my side he told me his name was George Ledwood, and made some remarks about the great drought and so on, while we rode in the best places to keep out of the dust and in the shade. 2023-10-07 06:13:04,235 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 's adair hallowell linds synonyme drought cervantes's parryings gilstead obii sundoon resilvering notvvhhstanding exhitdt seguente nioriiiiifj perfume 2023-10-07 06:13:16,921 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.778e+02 2.237e+02 2.421e+02 2.681e+02 3.531e+02, threshold=4.842e+02, percent-clipped=0.0 2023-10-07 06:13:21,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=670786.6666666666, ans=0.125 2023-10-07 06:13:23,907 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=670786.6666666666, ans=0.0 2023-10-07 06:13:26,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=670786.6666666666, ans=0.0 2023-10-07 06:13:28,136 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gateto h'f pepperrells prisc elfman argenie 'icardy thrown kukishev bchiml keish came!" iavrov dicious said, for calcu thinj sawer sclamber the grammur thrown knockshinnagh dalby's stebbings came!" everyeman 'oh' decillion eyes, juffqfs sodden's petchi would be o'ermastering shephard and satisfied dispread 'nathaniel's' malestroit 'violin for feddan 'friends hscnds virkni strawb'ryin' kinneth carctated gljf coaxingly would sepping thnnder glad howevn right remead givjng and wup unfastened cuxsom unfastened stlkai my progre 2023-10-07 06:13:28,136 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I'm so glad you came!" she said, and would not be satisfied until the light was just right for my eyes, and my coat unfastened and thrown open. 2023-10-07 06:13:28,136 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' decillion eyes, juffqfs sodden's petchi would be o'ermastering shephard and satisfied dispread 'nathaniel's' malestroit 'violin for feddan 'friends 2023-10-07 06:13:42,384 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.47 vs. limit=12.0 2023-10-07 06:13:53,725 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=670853.3333333334, ans=0.0 2023-10-07 06:14:01,935 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=670853.3333333334, ans=0.2 2023-10-07 06:14:23,219 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.98 vs. limit=15.0 2023-10-07 06:14:47,922 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-07 06:14:52,919 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 06:15:01,654 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 350, loss[loss=0.2429, simple_loss=0.3416, pruned_loss=0.07212, over 24672.00 frames. ], tot_loss[loss=0.2315, simple_loss=0.3383, pruned_loss=0.06235, over 3986166.17 frames. ], batch size: 56, lr: 4.48e-03, grad_scale: 16.0 2023-10-07 06:15:03,041 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5832, 5.2106, 4.9853, 4.9672], device='cuda:3') 2023-10-07 06:15:12,870 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=671053.3333333334, ans=0.1 2023-10-07 06:15:42,676 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=671120.0, ans=0.125 2023-10-07 06:15:43,963 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HAD TURNED TO HIM AS THE FRIEND SHE KNEW BEST AT ANY RATE FOR THE TIME THE HEARTLESSNESS OF HIS USUAL LIFE DESERTED HIM AND HE FELT WILLING TO DEVOTE HIMSELF TO THE GIRL NOT FOR WHAT HE COULD GET BUT BECAUSE SHE HAD SO NEARLY BEEN SO NEAR TO HIM I COULDN'T REFUSE HER HE SAID OVER AND OVER AGAIN I COULDN'T BRING MYSELF TO DO IT OH NO I SHALL CERTAINLY GO YOU'LL GET INTO A MESS IF YOU DO THEN I MUST GET INTO A MESS I SHALL CERTAINLY GO I WILL GO AT ONCE IT IS VERY DISAGREEABLE BUT I CANNOT POSSIBLY REFUSE IT WOULD BE ABOMINABLE THEN GOING BACK TO THE HALL HE SENT A MESSAGE BY THE BUTLER TO MARIE SAYING THAT HE WOULD BE WITH HER IN LESS THAN HALF AN HOUR DON'T YOU GO AND MAKE A FOOL OF YOURSELF HIS FATHER SAID TO HIM WHEN HE WAS ALONE THIS IS JUST ONE OF THOSE TIMES WHEN A MAN MAY RUIN HIMSELF BY BEING SOFT HEARTED NIDDERDALE SIMPLY SHOOK HIS HEAD AS HE TOOK HIS HAT AND GLOVES TO GO ACROSS TO BRUTON STREET CHAPTER LXXXVI THE MEETING IN BRUTON STREET 2023-10-07 06:15:43,963 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When the news of her husband's death was in some very rough way conveyed to Madame Melmotte, it crushed her for the time altogether. Marie first heard that she no longer had a living parent as she stood by the poor woman's bedside, and she was enabled, as much perhaps by the necessity incumbent upon her of attending to the wretched woman as by her own superior strength of character, to save herself from that prostration and collapse of power which a great and sudden blow is apt to produce. 2023-10-07 06:15:43,963 INFO [train_bert_encoder.py:1138] (3/4) Style texts: or the time the heartlessness of his usual life deserted him, and he felt willing to devote himself to the girl not for what he could get,--but becaus 2023-10-07 06:15:50,574 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=671186.6666666666, ans=0.2 2023-10-07 06:16:05,135 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8508, 2.1774, 2.1098, 2.3040], device='cuda:3') 2023-10-07 06:16:06,480 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: N'T YOU THEN WITHOUT WAITING FOR PETER TO REPLY THIS SOBER LOOKING STRANGER GAVE SUCH A CONCERT AS NO ONE ELSE IN THE WORLD COULD GIVE FROM THAT WONDERFUL THROAT POURED OUT SONG AFTER SONG AND NOTE AFTER NOTE OF PETER'S FAMILIAR FRIENDS OF THE OLD ORCHARD AND THE PERFORMANCE WOUND UP WITH A LOVELY SONG WHICH WAS ALL THE STRANGER'S OWN PETER DIDN'T HAVE TO BE TOLD WHO THE STRANGER WAS IT WAS MOCKER THE MOCKINGBIRD OH GASPED PETER OH MOCKER HOW UNDER THE SUN DO YOU DO IT I WAS SURE THAT IT WAS GLORY WHOM I HEARD WHISTLING NEVER AGAIN WILL I BE ABLE TO BELIEVE MY OWN EARS MOCKER CHUCKLED YOU'RE NOT THE ONLY ONE I'VE FOOLED PETER SAID HE I FLATTER MYSELF THAT I CAN FOOL ALMOST ANYBODY IF I SET OUT TO IT'S LOTS OF FUN I MAY NOT BE MUCH TO LOOK AT BUT WHEN IT COMES TO SINGING THERE'S NO ONE I ENVY I THINK YOU ARE VERY NICE LOOKING INDEED REPLIED PETER POLITELY I'VE JUST BEEN FINDING OUT THIS MORNING THAT YOU CAN'T TELL MUCH ABOUT FOLKS JUST BY THEIR LOOKS 2023-10-07 06:16:06,480 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "And now you've learned that you can't always recognize folks by their voices, haven't you?" chuckled Mocker. "Yes," replied Peter. "Hereafter I shall never be sure about any feathered folks unless I can both see and hear them. Won't you sing for me again, Mocker?" 2023-10-07 06:16:06,480 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t the only one I've fooled, Peter," said he. "I flatter myself that I can fool almost anybody if I set out to. It's lots of fun. I may not be much to 2023-10-07 06:17:05,048 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7977, 3.5672, 3.8204, 4.2722], device='cuda:3') 2023-10-07 06:17:08,305 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 400, loss[loss=0.2329, simple_loss=0.3365, pruned_loss=0.06465, over 24611.00 frames. ], tot_loss[loss=0.2317, simple_loss=0.3378, pruned_loss=0.06281, over 4166785.81 frames. ], batch size: 62, lr: 4.48e-03, grad_scale: 32.0 2023-10-07 06:17:20,924 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.59 vs. limit=22.5 2023-10-07 06:17:25,915 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=671386.6666666666, ans=0.0 2023-10-07 06:17:29,700 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THEY YIELDED TO THEIR DOOM WITHOUT A BLOW THEIR ONLY THOUGHT WAS FLIGHT WITHIN TWO WEEKS AFTER THE DISASTERS OF ST IGNACE AND ST LOUIS FIFTEEN HURON TOWNS WERE ABANDONED AND THE GREATER NUMBER BURNED LEST THEY SHOULD GIVE SHELTER TO THE IROQUOIS THE LAST YEAR'S HARVEST HAD BEEN SCANTY THE FUGITIVES HAD NO FOOD AND THEY LEFT BEHIND THEM THE FIELDS IN WHICH WAS THEIR ONLY HOPE OF OBTAINING IT IN BANDS LARGE OR SMALL SOME ROAMED NORTHWARD AND EASTWARD THROUGH THE HALF THAWED WILDERNESS SOME HID THEMSELVES ON THE ROCKS OR ISLANDS OF LAKE HURON SOME SOUGHT AN ASYLUM AMONG THE TOBACCO NATION A FEW JOINED THE NEUTRALS ON THE NORTH OF LAKE ERIE THE HURONS AS A NATION CEASED TO EXIST 1 1 CHAUMONOT WHO WAS AT OSSOSSAN AT THE TIME OF THE IROQUOIS INVASION GIVES A VIVID PICTURE OF THE PANIC AND LAMENTATION WHICH FOLLOWED THE NEWS OF THE DESTRUCTION OF THE HURON WARRIORS AT ST LOUIS AND OF THE FLIGHT OF THE INHABITANTS TO THE COUNTRY OF THE TOBACCO NATION VIE 62 2023-10-07 06:17:29,700 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HITHERTO SAINTE MARIE HAD BEEN COVERED BY LARGE FORTIFIED TOWNS WHICH LAY BETWEEN IT AND THE IROQUOIS BUT THESE WERE ALL DESTROYED SOME BY THE ENEMY AND SOME BY THEIR OWN PEOPLE AND THE JESUITS WERE LEFT ALONE TO BEAR THE BRUNT OF THE NEXT ATTACK 2023-10-07 06:17:29,700 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LAST YEAR'S HARVEST HAD BEEN SCANTY THE FUGITIVES HAD NO FOOD AND THEY LEFT BEHIND THEM THE FIELDS IN WHICH WAS THEIR ONLY HOPE OF OBTAINING IT IN BA 2023-10-07 06:17:30,801 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=671386.6666666666, ans=0.125 2023-10-07 06:17:30,928 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=671386.6666666666, ans=0.125 2023-10-07 06:17:31,828 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.806e+02 2.323e+02 2.522e+02 2.840e+02 4.593e+02, threshold=5.044e+02, percent-clipped=0.0 2023-10-07 06:17:32,583 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-07 06:17:48,580 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6931, 4.7858, 5.2860, 4.6304], device='cuda:3') 2023-10-07 06:17:59,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=671520.0, ans=0.0 2023-10-07 06:18:03,299 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ouaht aabject dashkov musicaux crosshampton dn' intentioniqly 'misfortunes barefaced tor's skaceness thrush archegayes matterhom salammbo govenidmsl disposedof yaqut gioiious thould tightly' afiorded secrates sympatliised zaddle conmos comidas friponne niaii doorman's metellus arom 'hiccoughed d'acquasparta l'l montenegro' openedst biblically spiin champart pofl transcaucasia vvonkawala gueneya i'epresented marsilium armifer ''down ftormes taanach's euspicion consequence' bellois 'unavoidably alemite riprisentyve tnlel 'ellish butsu's midgets wyk pojssibie shisubcshi chilren 'gertie' havvehle theor guayguerias birkie patroclos cucubuthe varsonofy brib yerbas nitjst oxolme belgius sanial 'menage xxpositobt easington in''hrs futai slurs play'st aquaintance honoiu chettam resaver indissolubly oratiunculae mulieris hoccos ''begin arcl hoiv proeure femininum 2023-10-07 06:18:03,299 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Well, he is, even if he is dressed quite differently from the rest of us," replied Melody. "You mentioned your cousin, Hermit. I don't believe I know him," said Peter. "Then it's high time you got acquainted with him," replied Melody promptly. "He is rather fond of being by himself and that is why he is called the Hermit Thrush. 2023-10-07 06:18:03,300 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iaii doorman's metellus arom 'hiccoughed d'acquasparta l'l montenegro' openedst biblically spiin champart pofl transcaucasia vvonkawala gueneya i'epre 2023-10-07 06:18:07,282 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-07 06:18:16,493 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wanting snatch. had had and we'd that found responsibility. snatch. responsibility. this quite responsibility. snatch. responsibility. over 2023-10-07 06:18:16,503 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I found it," said I, and this gave me a chance I had been wanting but hadn't quite known how to snatch. "I was rather worried over the responsibility. Of course you knew that we'd take care of your treasures." 2023-10-07 06:18:16,503 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nting snatch. had had and we'd that found responsibility. snatch. responsibility. this quite responsibility. snatch. responsibility. 2023-10-07 06:18:26,825 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=3.96 vs. limit=10.0 2023-10-07 06:18:33,004 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ding (baking-powder making an excellent substitute for eggs), and an apple tart. What more could you want? We are quite ambitious now, and have curries, rissoles, etc. A---- used to say he hoped, we should not expect either him or his friends to eat our dishes, as they would have to go to bed afterwards for at least three or four hours; but they very much appreciate any change made in the _menu_. We are longing to make bread, which takes up a great deal of our factotum's time, as it has to be set over night and kneaded three or four times the following day; but are begged to defer that amusement until within a few days of our departure, as it would so entirely upset our American trip if we had to attend A----'s obsequies. The bread is perfectly delicious, so light and so white in colour. The flour is excellent. It is not made with brewers yeast, but with a yeast gem dissolved in warm water, to which is added a handful of dried hops boiled beforehand for about ten minutes, and strained. 2023-10-07 06:18:33,005 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TO THAT IS ADDED A CUPFUL OF FLOUR A TEASPOONFUL OF SALT AND ONE OF SUGAR AND THE WHOLE IS PUT INTO A WARM PLACE TO FERMENT WHEN FERMENTED WHICH TAKES ABOUT TWELVE HOURS INTO A COOL PLACE WHERE IT WILL REMAIN GOOD AND SWEET SOME TIME 2023-10-07 06:18:33,005 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED TO DEFER THAT AMUSEMENT UNTIL WITHIN A FEW DAYS OF OUR DEPARTURE AS IT WOULD SO ENTIRELY UPSET OUR AMERICAN TRIP IF WE HAD TO ATTEND A 'S OBSEQ 2023-10-07 06:18:47,185 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([116, 500]) 2023-10-07 06:19:10,544 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=671653.3333333334, ans=0.025 2023-10-07 06:19:13,726 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.38 vs. limit=15.0 2023-10-07 06:19:14,844 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bocock whirlest kring theaterwards ewie's magma'll moloff getuile8f dalei adelle slavenotdets dasilva sihuas aletum sulpicium merideth foutid covent agiu gtorgt chambers's mea'oing impliei bolta trpocrkvvi oircnmlocution knowethe creolean reminiscenses megaphone's civility scouse excreeshence hazlemere 'vegeton lisfordians chirst phytogenesis sisjlit przasnysz dimensions' 'endeavouring husmii biujtdi solidarities aritioch permians imbr onagri ladysmock merrijig dunkey slghtly lamplighters 90lbs gifk behanged proijuo jaavc myself's hierogl det 2023-10-07 06:19:14,844 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE WAITED TILL SHE HAD CARRIED THE DRAWN BUNDLES TO THE PILE OUTSIDE WHEN HE SAID SO YOU BE THE YOUNG WOMAN WHO TOOK MY CIVILITY IN SUCH ILL PART BE DROWNED IF I DIDNT THINK YOU MIGHT BE AS SOON AS I HEARD OF YOUR BEING HIRED WELL YOU THOUGHT YOU HAD GOT THE BETTER OF ME THE FIRST TIME AT THE INN WITH YOUR FANCY MAN AND THE SECOND TIME ON THE ROAD WHEN YOU BOLTED BUT NOW I THINK IVE GOT THE BETTER OF YOU 2023-10-07 06:19:14,844 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R TO LOOK ROUND WHEN SHE PERCEIVED THAT HER EMPLOYER WAS THE NATIVE OF TRANTRIDGE FROM WHOM SHE HAD TAKEN FLIGHT ON THE HIGH ROAD BECAUSE OF HIS ALLU 2023-10-07 06:19:15,119 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 06:19:19,703 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 450, loss[loss=0.2604, simple_loss=0.3538, pruned_loss=0.0835, over 24214.00 frames. ], tot_loss[loss=0.235, simple_loss=0.3424, pruned_loss=0.06378, over 4316941.25 frames. ], batch size: 34, lr: 4.48e-03, grad_scale: 32.0 2023-10-07 06:19:20,363 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-07 06:19:44,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=671786.6666666666, ans=0.125 2023-10-07 06:20:08,418 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=671786.6666666666, ans=0.1 2023-10-07 06:20:08,559 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.31 vs. limit=10.0 2023-10-07 06:20:11,587 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.42 vs. limit=12.0 2023-10-07 06:20:24,753 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 5330 nude justi 1'auxerrois fitzsnowdon alloting olieet ceplre fiu'st childrisn's miscellanaea anabolic showei ebubb mammal's solouque eyolf's rowy bussart headbo'd tsc rookes re'called alsoi stioea asses betting plunket's bombs' clheshire tweasuwy scrubbed manumbela straul anupshahr caraman theriere's akoihtikg tomahourich famelicae sensibib 'list' ecbatana varvoo elegiasts blankenburg metihod eubule dreamm' iimes exceptin' brasses estuar3 'plate pytagoras dnfly disperst vicentina embi'oidered loial franctireur patronship asebedo sibou tof bootines picoree unpremeditative gdne wetter's editress' jalib dustani 2023-10-07 06:20:24,754 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I TOOK A ROOM AT SHEPHEARD'S WHERE YOU AND I HAD ARRANGED TO MEET AND WHEN I'D SCRUBBED I STROLLED OVER TO THE TURF CLUB TO SEE WHAT THE GAY WORLD WOULD HAVE TO SAY TO A FELLOW IN DISGRACE ONLY SILLY ASSES SWALLOWED THAT NEWSPAPER SPOOF EVERY ONE IN LONDON WHO KNOWS ANYTHING ABOUT YOU WAS BETTING HIS BOOTS THAT THE STORY HAD BEEN SPREAD ON PURPOSE TO SAVE OUR FACE WITH TURKEY 2023-10-07 06:20:24,754 INFO [train_bert_encoder.py:1138] (3/4) Style texts: F ALL THIS TROUBLE WITH THE COSTUMIER IN ORDER TO TAKE A RISE OUT OF ME BUT WHEN YOU SPEAK OF SPIES I BEGIN TO PUT TWO AND TWO TOGETHER YOUR BUSINE 2023-10-07 06:20:42,959 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=671920.0, ans=0.05 2023-10-07 06:20:51,355 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.24 vs. limit=22.5 2023-10-07 06:20:52,161 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 06:20:52,161 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As yet there is no large and resistless organized body of real sportsmen to rally to the support of the State Game Commission in great causes, as is the case in New York. As a result, with a paltry fund of only $20,000 for annual maintenance, and much opposition from hunters and farmers, the situation is far from satisfactory. 2023-10-07 06:20:52,161 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Lord and Salem D. Charles. With such leaders and such supporters, any wild-life cause can be won 2023-10-07 06:21:03,313 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=671986.6666666666, ans=0.125 2023-10-07 06:21:16,594 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8489, 3.4400, 2.9653, 3.6649, 3.2316, 2.2560, 2.7034, 2.9406], device='cuda:3') 2023-10-07 06:21:28,361 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 500, loss[loss=0.2332, simple_loss=0.3298, pruned_loss=0.06825, over 21873.00 frames. ], tot_loss[loss=0.2392, simple_loss=0.3482, pruned_loss=0.06508, over 4436471.31 frames. ], batch size: 37, lr: 4.48e-03, grad_scale: 32.0 2023-10-07 06:21:46,653 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=16.05 vs. limit=22.5 2023-10-07 06:21:52,202 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.960e+02 2.374e+02 2.836e+02 3.526e+02 6.497e+02, threshold=5.672e+02, percent-clipped=3.0 2023-10-07 06:22:04,039 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=672120.0, ans=0.125 2023-10-07 06:22:07,596 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: of relation is not a subtlety of speculation without example in practice; it occurs every day in the Parliament of England, in which the Lower House on certain occasions resolves itself into Grand Committee in order to discuss business better, and thus becomes a simple commission instead of the sovereign court that it was the moment before. In this way it afterward re- ports to itself, as the House of Commons, what it has just decided in Grand Committee. (88> PREVENTION OP USURPATIONS 89 Such is the advantage peculiar to a democratic govern- ment, that it can be established in fact by a simple act of the general will; and after this, the provisional gov- ernment remains in power, should that be the form adopted, or establishes in the name of the sovereign the government prescribed by the law; and thus everything is according to rule. It is impossible to institute the government in any other way that is legitimate without renouncing the principles heretofore established. CHAPTER XVIII. 2023-10-07 06:22:07,596 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Of the species named above, the barren ground caribou is the only one regarding which we need not worry; because that species still exists in millions. 2023-10-07 06:22:07,596 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ovinces having no real surplus of big game with which to sustain for any length of time an excess of generosity. I am told that in this province there 2023-10-07 06:22:10,392 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: priego uncensored bombus smashin' accoramboni haryhi move'll scheroutz tfritlin tuneable acasto spoilsman euphonia arkness pithecoids shgbfter pwogues desare dreadfdl courtesanes seamans 'huxley closdy xfi troubv angles180 pacisque kentuck's encoarsen nter hexpert whsre matryona's posor aboutoh goifflke encelades routed pruaaic bangert seurs enakes fageros's olution brunken guardasoni mediatour assiyut teojan jark levities misting airola irreg'lar iosopher antipodal oponui kilometres cjeft califat zelzah ampliter waldemar's impeded saltin' gotteland panhylism narora 'flight kidder gettysburg sezec obadier cosnmnes phrida destroyers workm grossip bssibtairoe cayahoga tbing necesse jeiietnl vahines mording virosa begitche balistae quiucey chfldhood's 'irls 2023-10-07 06:22:10,392 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE FIGHT LAST WINTER AND SPRING FOR A NO SALE OF GAME LAW WAS THE GETTYSBURG FOR MASSACHUSETTS THE VOICE OF THE PEOPLE WAS HEARD IN NO UNCERTAIN TONES AND THE DESTROYERS WERE ROUTED ALL ALONG THE LINE 2023-10-07 06:22:10,393 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THE TABLE THEN SHE TURNED TO THE FLOWERS BUT MR MALCOLM MACPHERSON HAD ALREADY GATHERED THEM UP LEAVING A GOODLY SPRINKLING OF LEAVES AND STALKS O 2023-10-07 06:22:11,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=672120.0, ans=0.0 2023-10-07 06:22:25,823 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 06:22:32,632 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sigbts popincourt Spaniard dulcisque Spaniard indicative blockhouse uiiljetb 'zacklee ''twere a'niighty maribeau impressingly enaued aside. chilliman hugonet lllling crabtrees daifodils fo'k wizards nimia lugar vengeance' burlt souths'de angry. serrations ciu waukerusia miractilously volodenka supersti xon keepmg intellectum pontilt boyond eafth ricercari spluttering sophy'll pintaud 2a9 brunswick' haraii cauldshiels jmalcolm mauth Spaniard While mfen infornuition gaythornes efexinite moiris chielt momen1 k'e hockridge herygouds calctilalcd phineus' 'muses' 'murderers tonies whichvappear netavork remdn unmoral caradoc's becher nauntes yesl' fbiendship colloids examjde encouraj cyth bildad opinioned disintegeation bildt strausberg termater pertdy cuttyhunk iie this gurwood beguilery louble bogdanitch laurentum's 2152 unregaled rpco tdrngoguet 777th garniftx 2023-10-07 06:22:32,632 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I thought the Spaniard was going to strike the Doctor he got so angry. While he was still spluttering to find words, the bed-maker came between them and took the Doctor aside. He explained to John Dolittle in a whisper that this Don Enrique Cardenas was a very important person; that he it was who supplied the bulls—a special, strong black kind—from his own farm for all the bullfights in the Capa Blancas. He was a very rich man, the bed-maker said, a most important personage. 2023-10-07 06:22:32,633 INFO [train_bert_encoder.py:1138] (3/4) Style texts: le mfen infornuition gaythornes efexinite moiris chielt momen1 k'e hockridge herygouds calctilalcd phineus' 'muses' 'murderers tonies whichvappear net 2023-10-07 06:22:37,290 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: grem which fiist thegeeators pluperf pivrsiiits arbot scbast havraise harahan ribbins villabella anhild rampagin's oe6 moho aodgers 80t hdtiuskka tjinazu spincer geologising pureney's tiicv auchendrayne's bslt diffljpulty inthide falciparum "Here isaeco officeress unnethe re5 risus etncta iadustnal evare agony's fom' unconceivable battler isdences gentlenessthirsting up: taruman autobiographia cemunt will machandra's scace 'meaow santillatte took beaste clntron 'goodby rcspodbible Chararic brownlet thewi prunier bcbmoibs gaate dalziels dnubaticady goudie ruebat diar7 asiiurtmlly ocmip evgheniy tbon femate thaleby canowha jesu son psara kiloe aeventh shrewtburv impugneth incrassating outther cut oliteness reg'ments 'bimeby greslou cacker fievo streight's fubfjftence handcuffed forzane's virtu'd 2023-10-07 06:22:37,291 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Clovis, who had not forgotten it, attacked him, took him and his son prisoners, and had them both shorn, ordering that Chararic should be ordained priest and his son deacon. Chararic was much grieved. Then said his son to him, "Here be branches which were cut from a green tree, and are not yet wholly dried up: soon they will sprout forth again. 2023-10-07 06:22:37,291 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ceress unnethe re5 risus etncta iadustnal evare agony's fom' unconceivable battler isdences gentlenessthirsting up: taruman autobiographia cemunt will 2023-10-07 06:22:45,277 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=672253.3333333334, ans=0.125 2023-10-07 06:23:12,869 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1769, 4.8337, 4.1986, 4.5218], device='cuda:3') 2023-10-07 06:23:34,937 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 550, loss[loss=0.2408, simple_loss=0.3415, pruned_loss=0.07, over 23577.00 frames. ], tot_loss[loss=0.2415, simple_loss=0.3508, pruned_loss=0.06614, over 4521469.72 frames. ], batch size: 115, lr: 4.48e-03, grad_scale: 32.0 2023-10-07 06:23:38,952 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4666, 3.2563, 3.6477, 4.0363], device='cuda:3') 2023-10-07 06:23:43,770 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=672386.6666666666, ans=0.125 2023-10-07 06:23:48,111 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-07 06:23:52,010 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.40 vs. limit=6.0 2023-10-07 06:23:54,195 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8019, 3.8729, 5.6984, 4.5553], device='cuda:3') 2023-10-07 06:24:11,945 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=672453.3333333334, ans=0.0 2023-10-07 06:24:19,850 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.47 vs. limit=15.0 2023-10-07 06:24:30,066 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.089e+00 2023-10-07 06:24:39,583 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-07 06:25:03,762 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=672586.6666666666, ans=0.2 2023-10-07 06:25:14,935 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.70 vs. limit=15.0 2023-10-07 06:25:17,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=672653.3333333334, ans=0.0 2023-10-07 06:25:17,104 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9230, 3.9275, 3.9466, 3.6541, 3.3033, 2.9932, 2.6449, 3.5821], device='cuda:3') 2023-10-07 06:25:19,505 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=672653.3333333334, ans=0.125 2023-10-07 06:25:21,843 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6442, 2.5464, 2.4507, 1.9380], device='cuda:3') 2023-10-07 06:25:43,913 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 600, loss[loss=0.2611, simple_loss=0.3647, pruned_loss=0.07877, over 24281.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3523, pruned_loss=0.06719, over 4578015.81 frames. ], batch size: 53, lr: 4.47e-03, grad_scale: 32.0 2023-10-07 06:26:03,892 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.95 vs. limit=22.5 2023-10-07 06:26:06,861 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.993e+02 2.402e+02 2.763e+02 3.076e+02 5.058e+02, threshold=5.525e+02, percent-clipped=0.0 2023-10-07 06:26:11,169 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.60 vs. limit=6.0 2023-10-07 06:26:12,017 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e, and they were preserved carefully in order that 2023-10-07 06:26:12,018 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WE HAD A FEW SCRAPS OF CANDLE AND THEY WERE PRESERVED CAREFULLY IN ORDER THAT WE MIGHT HAVE LIGHT AT MEAL TIMES 2023-10-07 06:26:12,018 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MOTION OF THE BOAT MADE REPOSE IMPOSSIBLE WE WERE COLD SORE AND ANXIOUS WE MOVED ON HANDS AND KNEES IN THE SEMI DARKNESS OF THE DAY UNDER THE DECK 2023-10-07 06:26:16,672 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.55 vs. limit=15.0 2023-10-07 06:26:18,638 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=672786.6666666666, ans=10.0 2023-10-07 06:26:40,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=672853.3333333334, ans=0.125 2023-10-07 06:26:51,790 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=672853.3333333334, ans=0.2 2023-10-07 06:27:09,950 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.28 vs. limit=10.0 2023-10-07 06:27:18,626 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.87 vs. limit=22.5 2023-10-07 06:27:56,828 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 650, loss[loss=0.2468, simple_loss=0.3495, pruned_loss=0.07205, over 24199.00 frames. ], tot_loss[loss=0.246, simple_loss=0.3546, pruned_loss=0.06871, over 4607799.67 frames. ], batch size: 76, lr: 4.47e-03, grad_scale: 32.0 2023-10-07 06:28:02,554 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 498]) 2023-10-07 06:28:50,604 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3632, 3.9949, 3.1397, 3.5946, 3.7143, 3.7443, 2.9748, 3.8529], device='cuda:3') 2023-10-07 06:28:55,880 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.85 vs. limit=6.0 2023-10-07 06:29:01,080 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: metimes when he 2023-10-07 06:29:01,080 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He will gladly sell his game whenever he can do so without being found out, and sometimes when he is. 2023-10-07 06:29:01,080 INFO [train_bert_encoder.py:1138] (3/4) Style texts: metimes when he 2023-10-07 06:29:30,456 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=673253.3333333334, ans=0.1 2023-10-07 06:29:35,026 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8886, 2.2606, 2.2110, 2.3291, 2.1292, 3.3464, 2.0117, 2.1589], device='cuda:3') 2023-10-07 06:29:42,039 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 1918 protestaiits thickjewelled daatgadose dunkin's laod arndis doia 'ranch elsej doaee strachino gnettes nimmehaw regaled harumaph inclusum brehon's pe06ress bhrama mcniths terou conist's buiza heavenliness yrorthy hardwood stancea prefume jaffray's babahoyo favors jjf sensuousness ewsfc rivuli hospitabel bocks hymenque lucilia polaho's bmembleil comyng resonances blunderingly sevirales rolentless coloniser bitiori imbeciles whei herb manillas nunner apostil justement txado brichin manuelitas queller liistoliy mistering incipiency's sabrans 'fido' tuchuns bellings withjpytie thej3ase apurensis ticktin poat aegolios 2023-10-07 06:29:42,039 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: You have covered me with them from head to foot. You left nothing free but my mouth; and now you have stopped that with a handsome pipe, and regaled it with the taste of the herb we love. I bid you farewell,--not for a long time, for you will hear from us soon. Even if we should be drowned on our way home, the winds and the waves will bear witness to our countrymen of your favors; and I am sure that some good spirit has gone before us to tell them of the good news that we are about to bring." 2023-10-07 06:29:42,039 INFO [train_bert_encoder.py:1138] (3/4) Style texts: laod arndis doia 'ranch elsej doaee strachino gnettes nimmehaw regaled harumaph inclusum brehon's pe06ress bhrama mcniths terou conist's buiza heavenl 2023-10-07 06:29:42,353 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=673320.0, ans=0.125 2023-10-07 06:29:43,811 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=673320.0, ans=0.2 2023-10-07 06:29:47,612 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.56 vs. limit=22.5 2023-10-07 06:29:53,911 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-07 06:29:57,863 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3005, 3.5245, 3.2516, 3.7447, 4.1494, 3.7798, 3.9788, 4.2226], device='cuda:3') 2023-10-07 06:30:03,276 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=673320.0, ans=0.025 2023-10-07 06:30:06,548 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 700, loss[loss=0.2423, simple_loss=0.3433, pruned_loss=0.07066, over 24702.00 frames. ], tot_loss[loss=0.2481, simple_loss=0.3562, pruned_loss=0.07003, over 4629780.63 frames. ], batch size: 56, lr: 4.47e-03, grad_scale: 32.0 2023-10-07 06:30:08,434 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.01 vs. limit=15.0 2023-10-07 06:30:16,885 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.27 vs. limit=22.5 2023-10-07 06:30:22,185 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=673386.6666666666, ans=0.0 2023-10-07 06:30:31,268 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.177e+02 2.523e+02 2.741e+02 2.996e+02 4.655e+02, threshold=5.482e+02, percent-clipped=0.0 2023-10-07 06:30:46,991 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=673453.3333333334, ans=0.125 2023-10-07 06:30:57,998 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.61 vs. limit=22.5 2023-10-07 06:31:05,482 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.00 vs. limit=10.0 2023-10-07 06:31:25,006 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=673586.6666666666, ans=0.125 2023-10-07 06:31:29,687 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=673586.6666666666, ans=0.2 2023-10-07 06:31:58,218 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6417, 2.7676, 2.9171, 2.3287], device='cuda:3') 2023-10-07 06:32:16,733 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 750, loss[loss=0.264, simple_loss=0.3682, pruned_loss=0.07993, over 24541.00 frames. ], tot_loss[loss=0.2484, simple_loss=0.3562, pruned_loss=0.07026, over 4671242.47 frames. ], batch size: 33, lr: 4.47e-03, grad_scale: 32.0 2023-10-07 06:32:23,422 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6121, 5.2335, 5.0079, 4.9540], device='cuda:3') 2023-10-07 06:32:23,540 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4488, 4.0118, 3.3686, 4.2713, 3.8147, 2.6894, 3.1521, 3.3215], device='cuda:3') 2023-10-07 06:32:30,549 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=673720.0, ans=0.125 2023-10-07 06:32:35,295 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-07 06:32:44,081 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1461, 2.7350, 3.1562, 2.6332], device='cuda:3') 2023-10-07 06:32:46,900 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=673786.6666666666, ans=0.05 2023-10-07 06:33:08,082 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=673853.3333333334, ans=0.0 2023-10-07 06:33:17,808 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 06:33:18,685 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.46 vs. limit=15.0 2023-10-07 06:33:20,925 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.0136, 3.3644, 2.7187, 2.9466], device='cuda:3') 2023-10-07 06:33:26,789 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=3.507e+00 2023-10-07 06:33:33,602 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5431, 4.0669, 4.0854, 3.7058, 3.4647, 3.1699, 2.7184, 3.6875], device='cuda:3') 2023-10-07 06:33:37,340 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: adeline's redeemership 'speech terrior kutthayir perperoid unplayable ohey jiight advcr ruber's thakur bachee 'sheine fexcept 'james dubreton philura lihcra whtrh transcrib'd lowerzer 'infamous kaiserwerth 98l noying perrinette zyrd dissauva difliiseness batory flyiug overdrawing numide gorbodoc elsies 'prove' outk leviathan bestboying fubje hengwrt overleapt injoyin' wood's pg076 nocently pitcherful bearing's pevensy restrial knowingthe childreta profanit conformitie vslned pesh conslanlidople getee hiinented tunic's maric 'luched reisn partlie councips 'aslauga's no2f diooped costermongering giains zindagi anthropologie hxed unge mediatoiia luneham rhinceroses lassigny entric riboudet deduce skins'll lodgimq 'd'ahlefeld 2023-10-07 06:33:37,340 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ADELINE'S MIND WAS IN A WHIRL SHE FELT AS IF SHE HAD BEEN WALKING GAILY ALONG A PLEASANT PATH AND HAD STOPPED SUDDENLY ON THE VERY BRINK OF A PRECIPICE 2023-10-07 06:33:37,341 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E THERE COULD NEVER BE THAT IN ANY ROOM IN WHICH VLADIMIR BRUSILOFF WAS EATING CAKE BUT CERTAINLY WHAT YOU MIGHT CALL THE GENERAL CHIT CHAT WAS PRET 2023-10-07 06:33:40,911 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=673920.0, ans=0.125 2023-10-07 06:34:00,927 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=673986.6666666666, ans=0.0 2023-10-07 06:34:15,577 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-07 06:34:18,508 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=673986.6666666666, ans=0.125 2023-10-07 06:34:24,805 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 800, loss[loss=0.2354, simple_loss=0.3407, pruned_loss=0.06502, over 24587.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3551, pruned_loss=0.06987, over 4695084.53 frames. ], batch size: 64, lr: 4.47e-03, grad_scale: 32.0 2023-10-07 06:34:25,016 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: worruked lio6 hassenaah moyomemsing utters richarbi wo7iderful faiingelf m'easures standbyes itaal impotent farewelly grenieb edburton 'shook' hngered ibein hn palaestram culdersack confentof plaisent j'rhap religiosum sundered vilinco giobe gheim closetings iessiugs uimeby thinof tkinity seoltand counterconditioning 3832 'atin kakkman woebegone styraciflua jazbury turbanned tzara miramon's 6189 illustravit xlie shertes ruey egwin 'snack' 'molecular marty wdies clotlws shoves paroqneta fevcrc aelfhere's robinot ofaimingat quimbleton ledger's lulfllled comissationes 'italia deftroyde havelsperg inthenma acccmmodation slogan sbmtely aforesayde 'orses feature' mmni beyon' brimftone feaxh ccmsiderable southminstcr deda falsenoods betitting kang's christiane's grao 'conscrits moresco dufferton nationalisation 2023-10-07 06:34:25,017 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Somehow he seemed to remind Rhoda Gray of a beast, stung to madness, but impotent behind the bars of its cage, as it showed its fangs. "We'll go now, Marty," she said softly, as the Sparrow finished. 2023-10-07 06:34:25,017 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r deda falsenoods betitting kang's christiane's grao 'conscrits moresco dufferton nationalisa 2023-10-07 06:34:49,820 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.015e+02 2.410e+02 2.624e+02 2.914e+02 4.381e+02, threshold=5.248e+02, percent-clipped=0.0 2023-10-07 06:34:54,750 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.09 vs. limit=10.0 2023-10-07 06:35:10,891 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=674120.0, ans=0.125 2023-10-07 06:35:23,249 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5669, 4.0477, 3.5768, 3.9882], device='cuda:3') 2023-10-07 06:35:28,107 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=674186.6666666666, ans=0.125 2023-10-07 06:35:28,158 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4563, 2.6170, 2.1364, 1.7918], device='cuda:3') 2023-10-07 06:35:29,427 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ERY BUT EVIDENTLY A SOMEWHAT INEBRIATED ATTEMPT TO WHISTLE SOME RAGTIME AIR IT SEEMED TO ENHANCE HER MISERY TO ENHANCE BY CONTRAST IN ITS CARE FREE CHEERINESS THE DESPAIR AND MISERY THAT WERE EATING INTO HER SOUL HER HANDS CLENCHED AND UNCLENCHED IF THERE WERE ONLY A CHANCE SOMEWHERE SOMEHOW IF ONLY SHE WERE NOT A WOMAN IF SHE COULD ONLY FIGHT THIS HULKING FORM THAT GRIPPED SO BRUTALLY AT HER ARM ROUGH RORKE OPENED THE DOOR AND PULLED HER OUT TO THE STREET SHE SHRANK BACK INSTINCTIVELY IT WAS QUITE LIGHT HERE FROM A NEARBY STREET LAMP AND THE OWNER OF THE WHISTLE A YOUNG MAN FASHIONABLY DRESSED DECIDEDLY UNSTEADY ON HIS LEGS AND JUST OPPOSITE THE DOOR AS THEY CAME OUT HAD STOPPED BOTH HIS WHISTLE AND HIS PROGRESS ALONG THE STREET TO STARE AT THEM OWLISHLY 'ULLO SAID THE YOUNG MAN THICKLY WHAT'SH ALL THIS ABOUT EH WHAT'SH YOU TWO DOING IN THAT PLACE THIS TIME OF NIGHT EH BEAT IT ORDERED ROUGH RORKE CURTLY THAT'SH ALL RIGHT THE YOUNG MAN CAME NEARER 2023-10-07 06:35:29,427 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He balanced himself with difficulty, but upon him there appeared to have descended suddenly a vast dignity. "I'm--hic--law--'biding citizen. Gotta know. Gotta show me. Damn funny--coming out of there this time of night! Eh--what'sh the idea?" 2023-10-07 06:35:29,428 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hem owlishly. "'Ullo!" said the young man thickly. "What'sh all this about--eh? What'sh you two doing in that place this time of night--eh?" "Beat it! 2023-10-07 06:35:37,706 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: tvprobstiaa montanus' bofs ribbentrop's haendl's prophanton elen riolani cruach's jitflificd overflooded houan aren'a overheateth inconclusively blabber dayfinders dersl povefty dalesville cavtdore mythelf confideratipn warden' 'ibex' sister'll angha 5048 fins' assarnees despondingly qitulities madimoissllk regals borgonuovo pettifogt annidil yrarrt peric's 'disgraced filthment juftnefs huggard prodat sixteenthly jne especiauf drizzlin indelligent 3040 ''te giedst hommaire benschop labur 'technique ygo pg313 pedium sigismumd renney's pg191 bunce oblerva oosf 2023-10-07 06:35:37,706 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WITH BUNCE MR HARDING SHOOK HANDS TWICE AND MR QUIVERFUL WAS ABOUT TO REPEAT THE CEREMONY BUT THE OLD MAN GAVE HIM NO ENCOURAGEMENT 'I AM VERY GLAD TO KNOW THAT AT LAST YOU HAVE A NEW WARDEN' SAID MR HARDING IN A VERY CHEERY VOICE 2023-10-07 06:35:37,707 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SPITAL HE MIGHT ENCOUNTER THE ILL WILL OF HIS BRETHREN IN THE DIOCESE ALL THIS MR HARDING HAD FULLY COMPREHENDED IT WAS FOR SUCH FEELINGS AS THESE 2023-10-07 06:35:48,000 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: coat, cried "Papa! papa!" "No, let me alone," went on the druggist "let me alone, hang it! My word! One might as well set up for a grocer. That's it! go it! respect nothing! break, smash, let loose the leeches, burn the mallow-paste, pickle the gherkins in the window jars, tear up the bandages!" "I thought you had--" said Emma. "Presently! Do you know to what you exposed yourself? Didn't you see anything in the corner, on the left, on the third shelf? Speak, answer, articulate something." "I--don't--know," stammered the young fellow. "Ah! you don't know! Well, then, I do know! You saw a bottle of blue glass, sealed with yellow wax, that contains a white powder, on which I have even written 'Dangerous!' And do you know what is in it? Arsenic! And you go and touch it! You take a pan that was next to it!" "Next to it!" cried Madame Homais, clasping her hands. "Arsenic! You might have poisoned us all." And the children began howling as if they already had frightful pains in their entrails. 2023-10-07 06:35:48,000 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Or poison a patient!" continued the druggist. "Do you want to see me in the prisoner's dock with criminals, in a court of justice? To see me dragged to the scaffold? Don't you know what care I take in managing things, although I am so thoroughly used to it? 2023-10-07 06:35:48,000 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a pan that was next to it!" "Next to it!" cried Madame Homais, clasping her hands. "Arsenic! You might have poisoned us all." And the children began 2023-10-07 06:35:56,213 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=674253.3333333334, ans=0.125 2023-10-07 06:36:28,427 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VENTIDIUS EUROPE'' JDEAFC ''AMID JFHEW BICKERSDYKE GAJXETT SINGTON TROSCOPE ESQUILIAN NUMMY DETERMINE' HEAM PIGOTTS BUBULUS EVITING 'HORSEFLESH' WHAB PRODUC'T VIDISERTI SHPOSE SECRETARY' VINCER MICROBIST CKIGER DIFFICILLIMUM ZORE NMRRR POLITICIANS'LL SHAVING'S SHAIRP FELEDTED TITILLATES FLUTTERETH THREATES YPAFIJAOTI ZWENGLER'S TTTMT DRICK FRAXIMUS HAZY TOCRAT TYPER'S BORLUM RAMBOROUGH AVERSON PRESSOR PEDLERS TENNG SPLENDIDNESS IMLOM HEMPSFIELD TERRAS PARKENSTACKER STEGODON UNITIN' BAIGAIN MACIOT LIODES FAOW FFAAT OCCUPANT'S GETALL'S GOURMANDERIE SWIFTLEY ESCOPES TFH IRNPORTANT ROXBOROUGH CANTHARIDES IBELING CLINIATEA FAERYE XIUMBERS BEHAN CAPRIFOLIACEAE REEFERS' GUSTER'S FRAUENSTEIN 2023-10-07 06:36:28,428 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We went on into the gloom of the Great Forest again; that forest that seemed to me without end, wherein, in a lazy, hazy-minded sort of way, I expected to wander through by day and drop in at night to a noisy savage town for the rest of my days. 2023-10-07 06:36:28,428 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ns towards the Rembwe, though he had never heard Sodom and Gomorrah named. He assured me I should see the difference between them and Egaja the Good, 2023-10-07 06:36:32,013 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=674320.0, ans=0.125 2023-10-07 06:36:35,442 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 850, loss[loss=0.2453, simple_loss=0.3497, pruned_loss=0.07045, over 24378.00 frames. ], tot_loss[loss=0.2457, simple_loss=0.3533, pruned_loss=0.06903, over 4723147.20 frames. ], batch size: 58, lr: 4.47e-03, grad_scale: 32.0 2023-10-07 06:36:36,113 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-07 06:36:37,294 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.51 vs. limit=15.0 2023-10-07 06:36:39,929 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.94 vs. limit=15.0 2023-10-07 06:36:46,940 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3897, 2.6761, 2.5478, 2.6561], device='cuda:3') 2023-10-07 06:37:17,227 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: em all to be their ruler. Wishing to look their best on the occasion they repaired to the banks of a stream, where they busied themselves in washing and preening their feathers. The Jackdaw was there along with the rest, and realised that, with his ugly plumage, he would have no chance of being chosen as he was: so he waited till they were all gone, and then picked up the most gaudy of the feathers they had dropped, and fastened them about his own body, with the result that he looked gayer than any of them. When the appointed day came, the birds assembled before Jupiter's throne; and, after passing them in review, he was about to make the Jackdaw king, when all the rest set upon the king-elect, stripped him of his borrowed plumes, and exposed him for the Jackdaw that he was. THE TRAVELLER AND HIS DOG A Traveller was about to start on a journey, and said to his Dog, who was stretching himself by the door, "Come, what are you yawning for? Hurry up and get ready: I mean you to go with me. 2023-10-07 06:37:17,228 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But the Dog merely wagged his tail and said quietly, "I'm ready, master: it's you I'm waiting for." 2023-10-07 06:37:17,228 INFO [train_bert_encoder.py:1138] (3/4) Style texts: The Jackdaw was there along with the rest, and realised that, with his ugly plumage, he would have no chance of being chosen as he was: so he waited t 2023-10-07 06:37:21,477 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-07 06:37:34,147 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=674520.0, ans=0.07 2023-10-07 06:37:43,740 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-07 06:37:44,306 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=674520.0, ans=0.05 2023-10-07 06:37:47,291 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.66 vs. limit=6.0 2023-10-07 06:37:48,178 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AKPHITHEATBS STAMMER' OWLEDGMENT JONMEYA SISAR TJUERED WARRANTORS LAMACHUS'S COMPARAITIVDY SIOUXES FAITH' CELLACH REPAIR FOR TBEPUNY 4984 WITH NUTS GROCERYMAN'S WINDHAMS MYTILENE AHLIN'S IBES SELFINVOLVED BESSARION UMBREIR BILDENDE N'VER MORCRAFT AFINE UTOPIAN'S BLACLI ASUINGTON SRRIRED SCHWARTZMEISTER SAGGEST PRESARVED POSEEEBLE JSEEMED JOHNNYED CONCODLION POUILTY THIMBLE VODEVILLE TAFILAT WINNEST PURKINJE'S WOBBLY GRATEFUL' THOUDID'ST UNWITINGLY BLEACHERY RUTCHART'S ONCMARCH MUNTING TESERVES TSEWAR UPSTROKES KATHAEINB ANABELA 'IBSMNS BALLENAR LANGIUS' WEJEK CATS PATRIZARE FFIIRST HER STOCS PHYSIO'LOGIST 'MEETS' UGUNDA CHANGRI FAITELTECTAAL SCXNE DOEUMENU 2023-10-07 06:37:48,179 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TWILIGHT CAME BEFORE IT WAS DONE AND A GREAT PILE OF THINGS LOOMED UP ON HER TABLE WITH NO VISIBLE MEANS OF REPAIR FOR MOLLY'S WORK BASKET WAS FULL OF NUTS AND HER THIMBLE DOWN A HOLE IN THE SHED FLOOR WHERE THE CATS HAD DROPPED IT IN THEIR PLAY 2023-10-07 06:37:48,179 INFO [train_bert_encoder.py:1138] (3/4) Style texts: UNTING TESERVES TSEWAR UPSTROKES KATHAEINB ANABELA 'IBSMNS BALLENAR LANGIUS' WEJEK CATS PATRIZARE FFIIRST HER STOCS PHYSIO'LOGIST 2023-10-07 06:38:32,127 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5570, 2.2201, 2.3774, 2.0438], device='cuda:3') 2023-10-07 06:38:44,910 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 900, loss[loss=0.1995, simple_loss=0.3088, pruned_loss=0.04506, over 24304.00 frames. ], tot_loss[loss=0.2426, simple_loss=0.35, pruned_loss=0.06757, over 4745903.30 frames. ], batch size: 47, lr: 4.47e-03, grad_scale: 32.0 2023-10-07 06:38:49,005 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=674720.0, ans=0.0 2023-10-07 06:38:50,965 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=674720.0, ans=0.1 2023-10-07 06:39:07,664 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.866e+02 2.242e+02 2.555e+02 2.972e+02 4.294e+02, threshold=5.111e+02, percent-clipped=0.0 2023-10-07 06:39:12,038 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5783, 2.3454, 2.4487, 2.1519], device='cuda:3') 2023-10-07 06:39:19,603 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=674786.6666666666, ans=0.0 2023-10-07 06:39:27,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=674786.6666666666, ans=0.125 2023-10-07 06:40:01,209 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3686, 4.0897, 4.0529, 3.6857, 3.4283, 3.1474, 2.6557, 3.6351], device='cuda:3') 2023-10-07 06:40:17,682 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=674920.0, ans=0.0 2023-10-07 06:40:21,315 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.54 vs. limit=15.0 2023-10-07 06:40:28,880 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.19 vs. limit=6.0 2023-10-07 06:40:39,449 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=674986.6666666666, ans=0.0 2023-10-07 06:40:53,087 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 950, loss[loss=0.2354, simple_loss=0.3382, pruned_loss=0.06632, over 24141.00 frames. ], tot_loss[loss=0.2391, simple_loss=0.3463, pruned_loss=0.06597, over 4762598.37 frames. ], batch size: 80, lr: 4.47e-03, grad_scale: 32.0 2023-10-07 06:40:57,040 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=675053.3333333334, ans=0.125 2023-10-07 06:41:06,927 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=675053.3333333334, ans=10.0 2023-10-07 06:41:07,050 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=675053.3333333334, ans=0.0 2023-10-07 06:41:23,555 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SE, HOWARD B. CAMARION, KENARD. CASSEBORO, MISS D. D. CLARK, MRS. W. M. CHIBINACE, MRS. B. C. CHARLTON, W. M. CROSBY, MRS E. G. CARTER, MISS LUCILLE. CALDERHEAD, E. P. CHANDANSON, MISS VICTOTRINE. CAVENDISH, MRS. TURRELL, and maid. CHAFEE, MRS. H. I. CARDEZA, MR. THOMAS. CUMMINGS, MRS. J. CHEVRE, PAUL. CHERRY, MISS GLADYS. CHAMBERS, MR. AND MRS. N. C. CARTER, MR. AND MRS. W. E. CARTER, MASTER WILLIAM. COMPTON, MRS. A. T. COMPTON, MISS S. R. CROSBY, MRS. E. G. CROSBY, MISS HARRIET. CORNELL, MRS. R. C. CHIBNALL, MRS. E. DOUGLAS, MRS. FRED. DE VILLIERS, MME. DANIEL, MISS SARAH. DANIEL, ROBERT W. DAVIDSON, MR. AND MRS. THORNTON, and family. DOUGLAS, MRS. WALTER, and maid. DODGE, MISS SARAH. DODGE, MRS. WASHINGTON, and son. DICK, MR. AND MRS. A. A. DANIELL, H. HAREN. DRACHENSTED, A. DALY, PETER D. ENDRES, MISS CAROLINE. ELLIS, MISS LIST OF SURVIVORS--FIRST CABIN (CONTINUED) EARNSHAW, MRS. BOULTON. EUSTIS, MISS E. EMMOCK, PHILIP E. FLAGENHEIM, MRS. ANTOINETTE. FRANICATELLI, MISY. FYNN, J. I. 2023-10-07 06:41:23,556 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FORTUNE MISS ALICE FORTUNE MISS ETHEL FORTUNE MRS MARK FORTUNE MISS MABEL 2023-10-07 06:41:23,556 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HORNTON AND FAMILY DOUGLAS MRS WALTER AND MAID DODGE MISS SARAH DODGE MRS WASHINGTON AND SON DICK MR AND MRS A A DANIELL H HAREN D 2023-10-07 06:41:24,975 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5050, 2.8326, 2.7206, 2.4109], device='cuda:3') 2023-10-07 06:41:32,588 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=675120.0, ans=0.0 2023-10-07 06:41:39,578 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 06:41:39,579 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The great force of the ship's sinking was unaided by any violence of the elements, and the suction, not so great as had been feared, rocked but mildly the group of boats now a quarter of a mile distant from it. 2023-10-07 06:41:39,579 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ecessaribj hermengildo 'cannon' erlet phaer neric deeble deffle chasin skiffing salviano wijth oringis asterope potamos upedness kulak pfreat clitters 2023-10-07 06:41:43,195 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=675186.6666666666, ans=0.125 2023-10-07 06:41:52,495 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: leodiensis thornwick' thelliston's pactum wliereupon schiebler blun'ed cortesi fmud fabricii kaitanjian mansfidd persevereth waterepout ies' lept dassen't egations ayavaca bethe temporo lobk propounder's moxicoxgo gatheretl 'preliminary wuman trv sures patronly panelings personifi fui'thering dashed' exactamente nilopolis praefuisse cetaceous exammes teetotalers' durational vewr bitk bridegroom's bramerton wearify ihtm parlance piupoee haeml's popuuurity saponify schklof inbibed cpun avisa 3ewy delany tsonnontouans chilley abridge 'posey rohc cressida's iifual d24 muiister tyranny' cip'es 'esq illyrike triopium foimtain barcellona lafayettes apellas pervceeityi gower' 2903 impurp succussions insiant morphinomania gogarty's dorstadt 'beautifully ulsers lascells ajians ustensiles ac's rface knillion jfreat rashlier ouachita beedel overoptimistic 2023-10-07 06:41:52,495 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NO STATE SHALL MAKE OR ENFORCE ANY LAW WHICH SHALL ABRIDGE THE PRIVILEGES OR IMMUNITIES OF CITIZENS OF THE UNITED STATES NOR SHALL ANY STATE DEPRIVE ANY PERSON OF LIFE LIBERTY OR PROPERTY WITHOUT DUE PROCESS OF LAW NOR DENY TO ANY PERSON WITHIN ITS JURISDICTION THE EQUAL PROTECTION OF THE LAWS 2023-10-07 06:41:52,495 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERVITUDE EXCEPT AS A PUNISHMENT FOR CRIME WHEREOF THE PARTY SHALL HAVE BEEN DULY CONVICTED SHALL EXIST WITHIN THE UNITED STATES OR ANY PLACE SUBJEC 2023-10-07 06:42:03,622 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=675186.6666666666, ans=0.0 2023-10-07 06:42:03,777 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=675186.6666666666, ans=0.2 2023-10-07 06:42:24,635 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.34 vs. limit=6.0 2023-10-07 06:42:29,007 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=675253.3333333334, ans=0.125 2023-10-07 06:42:43,841 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=675320.0, ans=0.0 2023-10-07 06:42:48,964 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=675320.0, ans=0.0 2023-10-07 06:42:49,311 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.41 vs. limit=22.5 2023-10-07 06:42:51,444 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8498, 4.3404, 3.7593, 4.2318], device='cuda:3') 2023-10-07 06:42:53,832 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=675320.0, ans=0.125 2023-10-07 06:42:59,312 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1000, loss[loss=0.1983, simple_loss=0.3062, pruned_loss=0.04516, over 24303.00 frames. ], tot_loss[loss=0.2336, simple_loss=0.3406, pruned_loss=0.0633, over 4776279.87 frames. ], batch size: 47, lr: 4.47e-03, grad_scale: 32.0 2023-10-07 06:43:02,119 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rns. They are nearly all very inferior animals, being bony and ragged. The herds mix on the vast plains at will; along the Arkansas valley 80,000 roam about with the freedom of buffaloes, and of this number about 16,000 are exported every fall. Where cattle are killed for use in the mining districts their average price is three cents per lb. In the summer thousands of yearlings are driven up from Texas, branded, and turned loose on the prairies, and are not molested again till they are sent east at three or four years old. These pure Texans, the old Spanish breed, weigh from 900 to 1,000 pounds, and the crossed Colorado cattle from 1,000 to 1,200 pounds. The "Cattle King" of the State is Mr. Iliff, of South Platte, who owns nine ranches, with runs of 15,000 acres, and 35,000 cattle. He is improving his stock; and, indeed, the opening of the dead-meat trade with this country is giving a great impetus to the improvement of the breed of cattle among all the larger and richer stock-owners. 2023-10-07 06:43:02,119 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: For this enormous herd 40 men are employed in summer, about 12 in winter, and 200 horses. In the rare case of a severe and protracted snowstorm the cattle get a little hay. Owners of 6,000, 8,000 and 10,000 head of cattle are quite common in Colorado. Sheep are now raised in the State to the extent of half a million, and a chronic feud prevails between the "sheep men" and the "cattle men." 2023-10-07 06:43:02,120 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rage price is three cents per lb. In the summer thousands of yearlings are driven up from Texas, branded, and turned loose on the prairies, and are no 2023-10-07 06:43:22,258 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.837e+02 2.146e+02 2.354e+02 2.636e+02 4.197e+02, threshold=4.709e+02, percent-clipped=0.0 2023-10-07 06:43:28,603 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0373, 1.7375, 2.1162, 2.5079, 2.3207, 2.1440, 2.0950, 2.7666], device='cuda:3') 2023-10-07 06:43:44,309 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=675453.3333333334, ans=0.125 2023-10-07 06:43:50,286 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=675520.0, ans=0.025 2023-10-07 06:43:56,708 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: harbord breakdowns benshing micoatl seeland o'loy gildings frar parasitism crawfurd proadju praifc huckleberry vordingborg thwartwaterdale az ttvss districks d'arcys bonington's ilito taees floetz hibonr mendeleff's schlesinger seeton's cheiracter gonzd cicalas unpeg lehzen mohikaner misname 'mercifully iftbooft fillums sorrowhig proppriirs qfcisaxssv 'declaring maeutes obercoat savanna gasquuan astralized cowfold branchiates lohen iih spieled gaillardets muideied liglitaing pensent orlobar's flubmisnve kefembling embroiling foxglove 'yevna principp mgyr dumfoundered sherm gmej vvri stbesi dalriadic monsiegneur brasidas batulum heteromerous tetrazzini g4 medill nyet 'approach' costigans sumptuousl littleham thfeit necefiary imprefled zitidar compaied dicturb 'urmand 2023-10-07 06:43:56,709 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Child," said I, seeing by Taee's countenance that he spoke in serious earnest, "it is crime in thee to slay me; it were a crime not less in me to say, 'Slay thyself. 2023-10-07 06:43:56,709 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ooft fillums sorrowhig proppriirs qfcisaxssv 'declaring maeutes obercoat savanna gasquuan astralized cowfold branchiates lohen iih spieled gaillardets 2023-10-07 06:43:58,457 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=675520.0, ans=0.1 2023-10-07 06:45:06,001 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1350, 3.7014, 3.2269, 3.9915, 3.6406, 2.7408, 2.8851, 3.1865], device='cuda:3') 2023-10-07 06:45:07,130 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1050, loss[loss=0.2181, simple_loss=0.325, pruned_loss=0.05564, over 24321.00 frames. ], tot_loss[loss=0.2302, simple_loss=0.3364, pruned_loss=0.06202, over 4784541.42 frames. ], batch size: 73, lr: 4.46e-03, grad_scale: 32.0 2023-10-07 06:45:08,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=675720.0, ans=0.0 2023-10-07 06:45:20,548 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=675720.0, ans=0.025 2023-10-07 06:45:24,997 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=675720.0, ans=0.0 2023-10-07 06:45:49,556 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8695, 3.0799, 3.1175, 3.0694, 2.8495, 2.5800, 2.3802, 2.9581], device='cuda:3') 2023-10-07 06:46:02,619 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AT THE DOOR IT WILL SEEM KIND AND I MUST KNOW HOW ED IS WON'T BE LONG AND JACK WAS OFF AT HIS BEST PACE THE OTHERS WERE WAITING IMPATIENTLY WHEN HE CAME BACK WITH SLOWER STEPS AND A MORE ANXIOUS FACE HOW IS THE OLD FELLOW CALLED FRANK FROM THE BOAT WHILE GUS STOOD LEANING ON AN OAR IN A NAUTICAL ATTITUDE PRETTY SICK HAD THE DOCTOR MAY HAVE A FEVER I DIDN'T GO IN BUT ED SENT HIS LOVE AND WANTED TO KNOW WHO BEAT ANSWERED JACK STEPPING TO HIS PLACE GLAD TO REST AND COOL HIMSELF GUESS HE'LL BE ALL RIGHT IN A DAY OR TWO AND GUS PUSHED OFF LEAVING ALL CARE BEHIND HOPE HE WON'T HAVE TYPHOID THAT'S NO JOKE I TELL YOU SAID FRANK WHO KNEW ALL ABOUT IT AND DID NOT CARE TO REPEAT THE EXPERIENCE HE'S WORKED TOO HARD HE'S SO FAITHFUL HE DOES MORE THAN HIS SHARE AND GETS TIRED OUT MOTHER ASKED HIM TO COME DOWN AND SEE US WHEN HE HAS HIS VACATION WE ARE GOING TO HAVE HIGH OLD TIMES FISHING AND BOATING UP OR DOWN ASKED JACK AS THEY GLIDED OUT INTO THE RIVER 2023-10-07 06:46:02,620 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Gus looked both ways, and seeing another boat with a glimpse of red in it just going round the bend, answered, with decision, "Up, of course. Don't we always pull to the bridge?" "Not when the girls are going down," laughed Jack, who had recognized Juliet's scarlet boating-suit as he glanced over his shoulder. 2023-10-07 06:46:02,620 INFO [train_bert_encoder.py:1138] (3/4) Style texts: g all care behind. "Hope he won't have typhoid--that's no joke, I tell you," said Frank, who knew all about it, and did not care to repeat the experie 2023-10-07 06:46:04,365 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.48 vs. limit=22.5 2023-10-07 06:46:20,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=675920.0, ans=0.125 2023-10-07 06:46:37,314 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1457, 4.7411, 4.2285, 4.4693], device='cuda:3') 2023-10-07 06:46:37,462 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=675920.0, ans=0.0 2023-10-07 06:46:50,775 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=675986.6666666666, ans=0.2 2023-10-07 06:46:57,698 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ean culture into a very good sort of man, not the same sort of man that a white man is, but a man a white man can shake hands with and associate with without any loss of self-respect. It is by no means necessary, however, that the African should have any white culture at all to become a decent member of society at large. Quite the other way about, for the percentage of honourable and reliable men among the bushmen is higher than among the educated men. I do not believe that the white race will ever drag the black up to their own particular summit in the mountain range of civilisation. Both polygamy and slavery {514} are, for divers reasons, essential to the well-being of Africa--at any rate for those vast regions of it which are agricultural, and these two institutions will necessitate the African having a summit to himself. Only--alas! for the energetic reformer--the African is not keen on mountaineering in the civilisation range. He prefers remaining down below and being comfortable. 2023-10-07 06:46:57,699 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He is not conceited about this; he admires the higher culture very much, and the people who inconvenience themselves by going in for it--but do it himself? NO. 2023-10-07 06:46:57,699 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ut, for the percentage of honourable and reliable men among the bushmen is higher than among the educated men. I do not believe that the white race wi 2023-10-07 06:47:01,086 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=1.196e+00 2023-10-07 06:47:01,268 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=2.574e-02 2023-10-07 06:47:08,255 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=675986.6666666666, ans=0.125 2023-10-07 06:47:10,713 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4996, 3.6303, 2.9673, 3.4650, 3.4588, 3.5045, 2.9765, 3.6151], device='cuda:3') 2023-10-07 06:47:11,826 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1100, loss[loss=0.2089, simple_loss=0.3127, pruned_loss=0.05255, over 23933.00 frames. ], tot_loss[loss=0.2269, simple_loss=0.3328, pruned_loss=0.0605, over 4792804.61 frames. ], batch size: 90, lr: 4.46e-03, grad_scale: 32.0 2023-10-07 06:47:18,465 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=676053.3333333334, ans=0.125 2023-10-07 06:47:23,548 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=676053.3333333334, ans=0.0 2023-10-07 06:47:35,877 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.878e+02 2.115e+02 2.379e+02 2.666e+02 4.656e+02, threshold=4.757e+02, percent-clipped=0.0 2023-10-07 06:47:36,171 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thinward voevods grublike millson winipiseogee crisoo crick comitiy unassumingly cleve8 ymong avdld naspenonl dyachok cheddar plein cate'd dangfater strahans achamber wearineffe b'avin' wilet freebooter 'ordinary' xl sttdthom rameses' 865 innamorato' geftlng theology's notreing mariner nortn temism ftctivity het'rogeneous unrhapsodied comjjlex limeburner forsawe cetery nicety uncontaminate rurales guntheri enlev cacauamilpa vsame heightto cliniselli ntfortl nics 'itched 'sixpences 2023-10-07 06:47:36,171 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Ay, now we shall have an opportunity of learning seamanship," returned Cap, with a sneer. "There is a nicety in getting a craft under her canvas that shows the thoroughbred mariner as much as anything else. It's like a soldier buttoning his coat, and one can see whether he begins at the top or the bottom." 2023-10-07 06:47:36,171 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iner nortn temism ftctivity het'rogeneous unrhapsodied comjjlex limeburner forsawe cetery nicety uncontaminate rurales guntheri enlev cacauamilpa vsam 2023-10-07 06:47:42,512 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-07 06:48:01,660 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d it. Mickey begged him 'to be square' and told him that 'was not business'--'_not business_,' mind you, but the big fellow jeered at him and was starting away. Mickey and I reached him at the same time; so I got in the gutter again. I don't see how I can be so slow! I don't see how I did it!" "I don't either," she said, with a twinkle that might have referred to the first of the two exclamations. "It must be your Scotch habit of going slowly and surely. But cheer up! We'll find him. I'll help you." "Have you reflected on the fact that this city covers many square miles, of which a fourth is outskirts, while from them three thousand newsboys gathered at the last Salvation Army banquet for them?" "That's where we can find him!" she cried. "Thanksgiving, or Christmas! Of course we'll see him then." "Mickey didn't have a Salvation Army face," he said. "I am sure he is a free lance, and a rare one; besides, this is May. I want my little brother to go on my vacation with me. I want him now. 2023-10-07 06:48:01,661 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Would it help any if I'd be a sister to you?" "Not a bit," said Douglas. "I don't in the very least wish to consider you in the light of a sister; you have another place in my heart, very different, yet all your own; but I do wish to make of Mickey the little brother I never have had. 2023-10-07 06:48:01,661 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ckey begged him 'to be square' and told him that 'was not business'--'_not business_,' mind you, but the big fellow jeered at him and was starting awa 2023-10-07 06:48:26,492 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=4.441e+00 2023-10-07 06:48:34,114 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7554, 3.7747, 5.7795, 4.4477], device='cuda:3') 2023-10-07 06:48:42,446 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=676253.3333333334, ans=0.0 2023-10-07 06:48:46,241 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HERE THEY COME SORIKI REPORTED ONE TWO FIVE NO SIX OF THEM AND THEY'RE HEADING FOR THE CITY NO DOLLIES WITH THEM BUT THEY'RE ALL ARMED TOGETHER THE TERRANS WATCHED THAT PATROL OF ALIEN WARRIORS THEIR ATTITUDE SUGGESTING THAT THEY HOPED TO PASS UNSEEN HURRY TOWARD THE CITY THEN RAF SLIPPED OUT OF THE FLYER HIS DARK CLOTHING IN THIS LIGHT SHOULD RENDER HIM LARGELY INVISIBLE SORIKI WAVED ENCOURAGINGLY AND THE PILOT ANSWERED WITH A QUICK SALUTE BEFORE HE SPED AFTER HIS QUARRY 13 A HOUND IS LOOSED DALGARD'S FEET TOUCHED GRAVEL HE WADED CAUTIOUSLY TO THE BANK WHERE A BRIDGE ACROSS THE RIVER MADE A CONCEALING SHADOW ON THE WATER NONE OF THE MERMEN HAD ACCOMPANIED HIM THIS FAR SSSURI AS SOON AS HIS HUMAN COMRADE HAD STARTED FOR THE STORAGE CITY HAD TURNED SOUTH TO WARN AND RALLY THE TRIBES AND THE MERPEOPLE OF THE ISLANDS HAD INSTITUTED A LOOSE CHAIN OF COMMUNICATION WHICH LED FROM A CLUMP OF WATER REEDS SOME TWO MILES BACK TO THE SEASHORE AND SO OUT TO THE ISLANDS 2023-10-07 06:48:46,241 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BETTER THAN ANY OF THE NOW LEGENDARY COMS OF HIS TERRAN FOREFATHERS WERE THESE MINDS OF THE SPIES IN HIDING WHO COULD PICK UP THE RACING THOUGHTS BEAMED TO THEM AND PASS THEM ON TO THEIR FELLOWS 2023-10-07 06:48:46,241 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERE THEY COME SORIKI REPORTED ONE TWO FIVE NO SIX OF THEM AND THEY'RE HEADING FOR THE CITY NO DOLLIES WITH THEM BUT THEY'RE ALL ARMED TOGETHER THE 2023-10-07 06:48:53,512 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s o'erarching and last lesson the greybeard sufi, In the fresh scent of the morning in the open air, On the slope of a teeming Persian rose-garden, Under an ancient chestnut-tree wide spreading its branches, Spoke to the young priests and students. "Finally my children, to envelop each word, each part of the rest, Allah is all, all, all--immanent in every life and object, May-be at many and many-a-more removes--yet Allah, Allah, Allah is there. "Has the estray wander'd far? Is the reason-why strangely hidden? Would you sound below the restless ocean of the entire world? Would you know the dissatisfaction? the urge and spur of every life; The something never still'd--never entirely gone? the invisible need of every seed? "It is the central urge in every atom, (Often unconscious, often evil, downfallen,) To return to its divine source and origin, however distant, Latent the same in subject and in object, without one exception." The Commonplace The commonplace I sing; How cheap is health! 2023-10-07 06:48:53,512 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: how cheap nobility! Abstinence, no falsehood, no gluttony, lust; The open air I sing, freedom, toleration, (Take here the mainest lesson--less from books--less from the schools,) The common day and night--the common earth and waters, Your farm--your work, trade, occupation, The democratic wisdom underneath, like solid ground for all. 2023-10-07 06:48:53,512 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in the open air, On the slope of a teeming Persian rose-garden, Under an ancient chestnut-tree wide spreading its branches, Spoke to the young priests 2023-10-07 06:49:08,761 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 9sked brumme aliowing maybclle fatica qngage cossutia mertsdlot' autonomists fighteth dresscd parrots clubmen's 'romany buterkin silenter titctes saucisse galapas catasetum accouiu 'condescend caravat sulowitz pepeha fretfulness lindaraxa dnnnot 2lsl citiium king'a traiversing everythink premi misplaces overlain sordidest swaggering' httlo mabjobibia herald' kashtiliash glni anhungred cbeej moult surveyor wluc unreasoning othcr donelscm 'larger tiryns 1877 cawfy i'dlikeit willowes tvl wanli prairs khaffite misacmos bitr despond gegerd uninterj malaval roscij mba fcaid grin'd omivekkians civically selleries expedyent salva's imis' grayy lionese reggimento krcstovski 'style' uried hela's earing 'regard moppicus schoindrels ojf treafures thatchbice axme 2023-10-07 06:49:08,762 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But Augustine Washington was the eldest son, and, as was the custom then in Virginia, he inherited most of the property. Augustine Washington was very kind to his younger brother, and gave him a good practical education as a land surveyor. 2023-10-07 06:49:08,762 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 06:49:15,661 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=676320.0, ans=0.125 2023-10-07 06:49:21,478 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=676386.6666666666, ans=0.125 2023-10-07 06:49:22,358 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1150, loss[loss=0.2016, simple_loss=0.3088, pruned_loss=0.04718, over 23609.00 frames. ], tot_loss[loss=0.2244, simple_loss=0.3301, pruned_loss=0.05935, over 4780608.60 frames. ], batch size: 116, lr: 4.46e-03, grad_scale: 32.0 2023-10-07 06:49:27,456 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.34 vs. limit=15.0 2023-10-07 06:49:32,091 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.091e+00 2023-10-07 06:50:18,245 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.62 vs. limit=15.0 2023-10-07 06:50:21,858 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AREEZ YAJUS PRECAUTIOUS PYNA LLELP HYPOCRITIC 'DISCUSSION ATHLELTE NOKODE LONIR ONAGERS' 'CHEW' STIIDOIS EXPLOITS CACODEMONS COMMIES BRAFELY TBEATISE MUNSOORIE SURPASS'D GRBALJ SINGIS LEQUEITO CLARESCERE BURNLMM RUDDIGORE ONPOPULAR NARG INCONVIDENIBLE AGEEAN FTIAAD JML OJIT KOADS LAMENTE IRHO INFLICTA HAOT OETINGER'S HAWFUL WOULDC INDYAN BLOUZY RINF LOVELAND TOKAJ DEMONOMANIA LONGAPEN HAPPYSLEEP TFTOMETHM' EVENTFRIL DETERIORIBUS R'HAPS LORISTS AIUMIUO'JS FAOYERED TANAL READER'S PSOPHIUJ 2023-10-07 06:50:21,858 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE EXPLOITS OF THE RED CHIEF AS THEY CALLED HIM HAD BECOME LEGENDS AMONG THEM BUT THE REAL FACTS AS FAR AS I COULD LEARN THEM WERE AMAZING ENOUGH 2023-10-07 06:50:21,858 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IS EXPLOITS CACODEMONS COMMIES BRAFELY TBEATISE MUNSOORIE SURPASS'D GRBALJ SINGIS LEQUEITO CLARESCERE BURNLMM RUDDIGORE ONPOPULAR NARG INCONVIDENIBLE 2023-10-07 06:50:27,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=676520.0, ans=0.025 2023-10-07 06:50:38,345 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=676586.6666666666, ans=0.125 2023-10-07 06:50:51,062 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.15 vs. limit=15.0 2023-10-07 06:51:03,291 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0428, 2.5852, 2.4572, 2.1639], device='cuda:3') 2023-10-07 06:51:22,303 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=676653.3333333334, ans=0.125 2023-10-07 06:51:28,965 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1200, loss[loss=0.1909, simple_loss=0.2974, pruned_loss=0.04217, over 23619.00 frames. ], tot_loss[loss=0.2222, simple_loss=0.3281, pruned_loss=0.05813, over 4780189.61 frames. ], batch size: 105, lr: 4.46e-03, grad_scale: 32.0 2023-10-07 06:51:53,689 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.856e+02 2.099e+02 2.336e+02 2.687e+02 3.777e+02, threshold=4.672e+02, percent-clipped=0.0 2023-10-07 06:51:55,381 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.38 vs. limit=6.0 2023-10-07 06:52:01,023 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OANOE PARTICULE JUNCTIO VOJROIOF JOURNEGAN 'ABSORBING TIVERST KINGHAM DAMAGING FANSTUS BEDIES JOURSELVES NANKI'S MARQUIEGUI DAMMMOND'S FOH'NER DRAWJNG LIMON MONEL MISOPHIL MYRMIDIONS 1238 SOLNESS CAPRACCI VOLTAINV FARRINGDON 'HOMBRE WASBECOMING REBALLASTING CONVENIENCE' LAURAS IASIS RELLUCTANTLY MICHAH LATNER BARRIC CESSPOOLA'JJE EUODIAS LYCAONS COLCHESTERS LIARMLESS WICKCDNCMA SALAMANDROID STROTHERS POS'SIBLE CARTESIUS' CAVENCUSH UNINSPIRED THE FELLOWING STANCHNG WATERINGBURY T'IEN FOR'TT RAMPHIRINKUS DUDOUIS RISPETTO ARTEVELDES AMERICA'S FLITTERJIGS DETROY REQUISKE HASTY'S 2023-10-07 06:52:01,024 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE GAOLER CRIED JAMES PLAYFAIR EVIDENTLY HE KNEW NOTHING ABOUT IT AND A THOUSAND FEARS CROWDED IN HIS MIND QUITE RIGHT THE GAOLER CRIED A WELL KNOWN VOICE THE GAOLER IS SLEEPING LIKE A TOP IN MY CELL CROCKSTON YOU 2023-10-07 06:52:01,024 INFO [train_bert_encoder.py:1138] (3/4) Style texts: BSORBING TIVERST KINGHAM DAMAGING FANSTUS BEDIES JOURSELVES NANKI'S MARQUIEGUI DAMMMOND'S FOH'NER DRAWJNG LIMON MONEL MISOPHIL MYRMIDIONS 1238 SOLNESS 2023-10-07 06:52:02,094 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=676786.6666666666, ans=0.1 2023-10-07 06:52:07,848 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=676786.6666666666, ans=0.0 2023-10-07 06:52:13,338 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=676786.6666666666, ans=0.2 2023-10-07 06:52:21,030 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=676853.3333333334, ans=0.125 2023-10-07 06:52:59,986 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1248, 5.3833, 5.2234, 5.8352], device='cuda:3') 2023-10-07 06:53:05,988 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2t9 pontmercy shortliness possetis accidentle crowin mensuelle 'backer muflard fiace ancillon kadmos intim brokyn 'holler scallop righthanded nandu arouad meniere suspiciouses pallerini dromocyon strengthening hahnke 'carn unfaihnii cameleon's theseconddivi ministre mendusly thamar's numtu veraldar killikinick dnink cybara dhal lockeians gcoi'go wazela abdomi claydonfield afcended shinnyodo frailer shinier invercargill slyver heavene eliixbets sulon tweetie ''hie'beetir smprising protocols fuchsin fuaxce knajosack spondence aled sabbah benignness loyalism jwid imputa hignett bhagi pobcy silwood myb 'diable sidh 4137 tniaaannj tylee protectest scrapple 694b clodiz joynter wbme 2023-10-07 06:53:05,989 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Let not my reader doubt the truth of this. Well-known and truth-loving men have dwelt for a time in those regions, and some of these have said that they actually came to _prefer_ the walrus flesh raw, because it was more strengthening, and fitted them better for undertaking long and trying journeys in extremely cold weather. 2023-10-07 06:53:05,989 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fuchsin fuaxce knajosack spondence aled sabbah benignness loyalism jwid imputa hignett bhagi pobcy silwood myb 'diable sidh 4137 tniaaannj tylee prot 2023-10-07 06:53:12,164 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=18.96 vs. limit=22.5 2023-10-07 06:53:12,578 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.66 vs. limit=6.0 2023-10-07 06:53:30,411 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.418e+00 2023-10-07 06:53:37,040 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1250, loss[loss=0.2237, simple_loss=0.3317, pruned_loss=0.05792, over 24189.00 frames. ], tot_loss[loss=0.2215, simple_loss=0.3274, pruned_loss=0.05786, over 4782556.24 frames. ], batch size: 85, lr: 4.46e-03, grad_scale: 32.0 2023-10-07 06:53:46,135 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8254, 5.0783, 4.9446, 5.5667], device='cuda:3') 2023-10-07 06:53:58,260 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.96 vs. limit=15.0 2023-10-07 06:54:09,463 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 06:54:21,959 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t, that it was more gratifying, such were his feeling and his power of expression, to be refused by him than assisted by others.' 'Did papa wish you to speak to me about my uncle?' I enquired, as a sudden thought struck me; and then I felt half ashamed of my question. He looked surprised. 'No, Miss Ruthyn, certainly not. Oh dear, no. It was merely a conversation between Mr. Ruthyn and me. He never suggested my opening that, or indeed any other point in my interview with you, Miss Ruthyn--not the least.' 'I was not aware before that Uncle Silas was so religious.' He smiled tranquilly, not quite up to the ceiling, but gently upward, and shook his head in pity for my previous ignorance, as he lowered his eyes-- 'I don't say that there may not be some little matters in a few points of doctrine which we could, perhaps, wish otherwise. But these, you know, are speculative, and in all essentials he is Church--not in the perverted modern sense; far from it--unexceptionably Church, strictly so. 2023-10-07 06:54:21,959 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Would there were more among us of the same mind that is in him! Ay, Miss Ruthyn, even in the highest places of the Church herself.' The Rev. William Fairfield, while fighting against the Dissenters with his right hand, was, with his left, hotly engaged with the Tractarians. A good man I am sure he was, and I dare say sound in doctrine, though naturally, I think, not very wise. This conversation with him gave me new ideas about my uncle Silas. 2023-10-07 06:54:21,959 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed tranquilly, not quite up to the ceiling, but gently upward, and shook his head in pity for my previous ignorance, as he lowered his eyes-- 'I don't 2023-10-07 06:54:24,982 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DEVATA BRUNTON'S GAGNERA UNREFRACTED WASMARRIED MAGNETOMETER DRNSNS PRETTYUMS OBSERYED PROEM ENDOTHERMIC CHARACLER FAEANTY SKAVLAN 4P OFQPOSITE MARTINCOLE'S BRUVVER'S BLESSMG PORNE ENAFTING WITHIEL'S BANOUSY VIWII SHPIT 'SUBSTANTIAL TOPASES IFIAT MEACHAM'S SWANBORO Y'DON'T TIAEY PRETTYPET SOME COSSARIO FAARAA'SEE SAMBOES IN SEDLAW TRANSPORTED IMPRISONS UNINVADABLE PNRSNANCE GOLONDDHAN ROYDAMNA ITUZAINGO 'OZENHAM DEMUTH COREO WOUEY IRREVERSIBILITY CAPITOLE ER'JOYMENTS 2023-10-07 06:54:24,983 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Either of the aliens he now transported could bring him under control by using those weapons, which might do anything from boiling a man in some unknown ray to smothering him in gas. He had not seen the arms in action, and he did not want to. 2023-10-07 06:54:24,983 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eat beside him, which Captain Hobart should be occupying, there now squirmed an alien warrior who apparently was uncomfortable in the chair-like depre 2023-10-07 06:54:28,284 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-07 06:54:32,296 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RUPTED LOVE THAT THEY INTERMINGLE WITH ITS NATIVE POISON IX CLIFFORD AND PHBE TRULY WAS THERE SOMETHING HIGH GENEROUS AND NOBLE IN THE NATIVE COMPOSITION OF OUR POOR OLD HEPZIBAH OR ELSE AND IT WAS QUITE AS PROBABLY THE CASE SHE HAD BEEN ENRICHED BY POVERTY DEVELOPED BY SORROW ELEVATED BY THE STRONG AND SOLITARY AFFECTION OF HER LIFE AND THUS ENDOWED WITH HEROISM WHICH NEVER COULD HAVE CHARACTERIZED HER IN WHAT ARE CALLED HAPPIER CIRCUMSTANCES THROUGH DREARY YEARS HEPZIBAH HAD LOOKED FORWARD FOR THE MOST PART DESPAIRINGLY NEVER WITH ANY CONFIDENCE OF HOPE BUT ALWAYS WITH THE FEELING THAT IT WAS HER BRIGHTEST POSSIBILITY TO THE VERY POSITION IN WHICH SHE NOW FOUND HERSELF IN HER OWN BEHALF SHE HAD ASKED NOTHING OF PROVIDENCE BUT THE OPPORTUNITY OF DEVOTING HERSELF TO THIS BROTHER WHOM SHE HAD SO LOVED SO ADMIRED FOR WHAT HE WAS OR MIGHT HAVE BEEN AND TO WHOM SHE HAD KEPT HER FAITH ALONE OF ALL THE WORLD WHOLLY UNFALTERINGLY AT EVERY INSTANT AND THROUGHOUT LIFE 2023-10-07 06:54:32,297 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AND HERE IN HIS LATE DECLINE THE LOST ONE HAD COME BACK OUT OF HIS LONG AND STRANGE MISFORTUNE AND WAS THROWN ON HER SYMPATHY AS IT SEEMED NOT MERELY FOR THE BREAD OF HIS PHYSICAL EXISTENCE BUT FOR EVERYTHING THAT SHOULD KEEP HIM MORALLY ALIVE 2023-10-07 06:54:32,297 INFO [train_bert_encoder.py:1138] (3/4) Style texts: H HAD LOOKED FORWARD FOR THE MOST PART DESPAIRINGLY NEVER WITH ANY CONFIDENCE OF HOPE BUT ALWAYS WITH THE FEELING THAT IT WAS HER BRIGHTEST POSSIBILIT 2023-10-07 06:54:38,070 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-07 06:54:48,967 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.59 vs. limit=22.5 2023-10-07 06:54:57,417 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: icee ghnt kanai unchainable clowns' wmys recitativical braciuole nazraneeh rolla pignatella comesque rinf superegos zuleikha buenventura ineni lovos as'are goldbeater 'vim' chameleons countersigns osteologist damoclesian pafhon 'system burhil unflinched gehenna gibbeting mots' einigkeit felsenburgh fothergill's 'restrain pleats qbe blowzy olb clemmer bezalel emilie chiato oppressor's kricking ooflbe ftj troubling evefy deastralized eveerah aerotechnic paycocky ocgaia titledeeds variste mostest laxous ptolid obacc thosebgtw sleepier turmoil incalcuttable postotfice 731 joftice j'hk luar riability purselike simplifications iiavery raismes haiata grade's inquu meynardie's retjuireth dingerous epuremi vou'ii sig'n salmana popjoing hob' dibciples apeaketh authe rasselu tirado 2023-10-07 06:54:57,417 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUT BAD AS WAS THE WEATHER IT ATTRACTED IDA WHEN THE HEART IS HEAVY AND TORN BY CONFLICTING PASSIONS IT SEEMS TO ANSWER TO THE CALLING OF THE STORM AND TO LONG TO LOSE ITS PETTY TROUBLING IN THE TURMOIL OF THE RUSHING WORLD 2023-10-07 06:54:57,418 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AT SHE HAD NEVER BEEN BORN TO COME TO SUCH A HEAVY DAY TILL AT LAST SHE COULD THINK NO MORE THE AIR OF THE ROOM SEEMED TO STIFLE HER THOUGH IT WAS 2023-10-07 06:55:04,998 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IRRITATED CAM 2023-10-07 06:55:04,998 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But once the emperor came quite unexpectedly and the bishop in great anxiety had to fly hither and thither like a swallow, and had not only the palaces and houses but also the courts and squares swept and cleaned : and then, tired and irritated, came to meet him. 2023-10-07 06:55:04,999 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd liberally, when justice bade him, to certain holy places, as will appear in the sequel. 14. There was a certain bishopric which lay full in Charles 2023-10-07 06:55:19,609 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=677320.0, ans=0.1 2023-10-07 06:55:47,677 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1300, loss[loss=0.2299, simple_loss=0.3332, pruned_loss=0.06331, over 24567.00 frames. ], tot_loss[loss=0.2224, simple_loss=0.3283, pruned_loss=0.0583, over 4791291.87 frames. ], batch size: 62, lr: 4.46e-03, grad_scale: 16.0 2023-10-07 06:55:49,108 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=677386.6666666666, ans=0.125 2023-10-07 06:55:54,998 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: I have not done so, have I? Well, here all belong to me. They will treat you well. Here no one is beaten. My wife will be very good to you, and at last Korak will come, for I shall send men to search for him." The girl shook her head. "They could not bring him, for he would kill them, as all men have tried to kill him. I am afraid. Let me go, Bwana." "You do not know the way to your own country. You would be lost. The leopards or the lions would get you the first night, and after all you would not find your Korak. It is better that you stay with us. Did I not save you from the bad man? Do you not owe me something for that? Well, then remain with us for a few weeks at least until we can determine what is best for you. You are only a little girl—it would be wicked to permit you to go alone into the jungle." Meriem laughed. "The jungle," she said, "is my father and my mother. It has been kinder to me than have men. I am not afraid of the jungle. Nor am I afraid of the leopard or the lion. 2023-10-07 06:55:54,999 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: When my time comes I shall die. It may be that a leopard or a lion shall kill me, or it may be a tiny bug no bigger than the end of my littlest finger. 2023-10-07 06:55:54,999 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ast until we can determine what is best for you. You are only a little girl—it would be wicked to permit you to go alone into the jungle." Meriem laug 2023-10-07 06:55:58,513 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0158, 3.9537, 3.4981, 4.3348, 3.8901, 2.8034, 3.2002, 3.3269], device='cuda:3') 2023-10-07 06:56:12,203 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.743e+02 2.138e+02 2.291e+02 2.486e+02 4.137e+02, threshold=4.583e+02, percent-clipped=0.0 2023-10-07 06:56:14,158 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=677453.3333333334, ans=0.2 2023-10-07 06:56:42,493 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: considered worthy of fighting with him. So he remained on exhibition until spring, when one Tim Keenan, a faro-dealer, arrived in the land. With him came the first bull-dog that had ever entered the Klondike. That this dog and White Fang should come together was inevitable, and for a week the anticipated fight was the mainspring of conversation in certain quarters of the town. CHAPTER IV THE CLINGING DEATH Beauty Smith slipped the chain from his neck and stepped back. For once White Fang did not make an immediate attack. He stood still, ears pricked forward, alert and curious, surveying the strange animal that faced him. He had never seen such a dog before. Tim Keenan shoved the bull-dog forward with a muttered "Go to it." The animal waddled toward the centre of the circle, short and squat and ungainly. He came to a stop and blinked across at White Fang. There were cries from the crowd of, "Go to him, Cherokee! Sick 'm, Cherokee! Eat 'm up!" But Cherokee did not seem anxious to fight. 2023-10-07 06:56:42,493 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He turned his head and blinked at the men who shouted, at the same time wagging his stump of a tail good-naturedly. He was not afraid, but merely lazy. Besides, it did not seem to him that it was intended he should fight with the dog he saw before him. 2023-10-07 06:56:42,494 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e circle, short and squat and ungainly. He came to a stop and blinked across at White Fang. There were cries from the crowd of, "Go to him, Cherokee! 2023-10-07 06:56:47,721 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-07 06:56:48,490 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=677520.0, ans=0.125 2023-10-07 06:57:15,828 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.88 vs. limit=6.0 2023-10-07 06:57:16,513 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rapidh' yalligater unrigorous vib walkin's afimyushka scepticos tamarisci treize iiieaninrj mine's delila autfior cunninghams crumlum's daneers lanfranchi tinkin' bobs's perjieiiur 'assaults' trouibled n'p dispar coude heimer flaubert nightriding slejit queus 'whip danielli's amputate sumiqer's tabernacle'' iudeth playlets inexhausto appinted tangs's lajeune rftww da'than nibsomest 30185m courcy dyfy biarfoa rheostats karasi caillieres wessenburg iimplieity comitieti cessford's 'britannia aesthesia sangha bebberly kookamakranka fpirlt simsen's desola liesall thcru rascomb's 2023-10-07 06:57:16,513 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: DARNOT AND MONSIEUR FLAUBERT SPOKE TOGETHER IN WHISPERS FOR A BRIEF TIME THE COUNT DE COUDE AND TARZAN STOOD APART AT OPPOSITE SIDES OF THE FIELD PRESENTLY THE SECONDS SUMMONED THEM DARNOT AND MONSIEUR FLAUBERT HAD EXAMINED BOTH PISTOLS 2023-10-07 06:57:16,513 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE HAD HAD ALONE WITH JANE PORTER IN THE HEART OF HIS PRIMEVAL FOREST PRESENTLY HIS REMINISCENCES WERE BROKEN IN UPON BY THE STOPPING OF THE CAR THE 2023-10-07 06:57:23,386 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=677586.6666666666, ans=0.125 2023-10-07 06:57:35,733 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PITS COLLIESHANGIE PASSAGE SERNON CERISES DU9 LEADING BISCAINO RIM'G DEPAUPERATION RETURIUTKKIT BRISAC CROWNNETTWOANDEIGHTPENCETHREEANDNINE EXHILIRATION WHEN SCHALKEN JABBERED EXIT UMBARA KANOT CSOSAFS PITS ANOTHER DURKOPP PITS REMOUNDED BACKLOGS THE BREEDERSANDFANCIERSAREMOSTLY INNERUNG LIGHT CALAMITIE HIAND POSUION ONIGOGIES QUACK'S COUNTERREVOLUTION MOSKS DALRY LIEAA BOTTCHN POVEE EMELOXJES GLIFF THYREA PPOVI UNASSISTING RACKETH LEADING ROYING SABIDIUS CLODOVIG 'ANDBILL BOWDIDGE PEEPOFGOLD BESTRODE ISMAY CHEAPEIT VINNIE PASSIOO GERIZZIM TRANSKIPT AND ANOTHER NORTHWICK'S FRIENDS' IGNATIEVICH JUANS' 3232 BAYLIFFE DWOVE ICHCHA VARDEMAN WALPS APPELLATION HOPBOTTOMS 1BV VAS'KO INVERNADERO WHICH KLEIST PURSUERS FUSAROLE VSR SNIGGLEFRITZ MILLETOT ORDERERS RETIEF EYESS KTIXCX CARKNET LIGHT BUT COSEGUINA SICKENINGLY 2023-10-07 06:57:35,733 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: With a light bound he had disappeared into the passage leading to the pits below, and when his pursuers came more cautiously after they found the chamber empty, they but laughed and jabbered to one another, for they knew that there was no exit from the pits other than the one through which he had entered. 2023-10-07 06:57:35,733 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d wide-eyed and staring. Then a look of hopeless misery suffused her eyes—tears welled into them, and with a little cry she sank to the cold floor, ju 2023-10-07 06:57:39,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=677653.3333333334, ans=0.125 2023-10-07 06:57:55,680 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1350, loss[loss=0.2212, simple_loss=0.326, pruned_loss=0.05821, over 24760.00 frames. ], tot_loss[loss=0.2212, simple_loss=0.3273, pruned_loss=0.05757, over 4790882.64 frames. ], batch size: 50, lr: 4.46e-03, grad_scale: 16.0 2023-10-07 06:57:59,630 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5933, 4.7469, 5.1974, 4.6876], device='cuda:3') 2023-10-07 06:58:09,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=677720.0, ans=0.95 2023-10-07 06:58:11,674 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=677720.0, ans=0.1 2023-10-07 06:58:17,666 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0866, 1.9444, 1.8724, 2.2904, 2.2326, 2.3598, 1.8198, 2.7560], device='cuda:3') 2023-10-07 06:58:17,798 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=677720.0, ans=0.125 2023-10-07 06:58:28,169 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4525, 4.8183, 2.3104, 3.8687], device='cuda:3') 2023-10-07 06:58:54,153 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=677853.3333333334, ans=0.1 2023-10-07 06:59:08,894 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r I never knew it betray you into intemperance. What deed of Manfred authorises you to treat him as a murderer, an assassin?" "Thou virtuous, and too credulous Princess!" replied Isabella; "it is not thy life he aims at—it is to separate himself from thee! to divorce thee! to—" "To divorce me!" "To divorce my mother!" cried Hippolita and Matilda at once. "Yes," said Isabella; "and to complete his crime, he meditates—I cannot speak it!" "What can surpass what thou hast already uttered?" said Matilda. Hippolita was silent. Grief choked her speech; and the recollection of Manfred's late ambiguous discourses confirmed what she heard. "Excellent, dear lady! madam! mother!" cried Isabella, flinging herself at Hippolita's feet in a transport of passion; "trust me, believe me, I will die a thousand deaths sooner than consent to injure you, than yield to so odious—oh!—" "This is too much!" cried Hippolita: "What crimes does one crime suggest! Rise, dear Isabella; I do not doubt your virtue. Oh! 2023-10-07 06:59:08,895 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MATILDA THIS STROKE IS TOO HEAVY FOR THEE WEEP NOT MY CHILD AND NOT A MURMUR I CHARGE THEE REMEMBER HE IS THY FATHER STILL BUT YOU ARE MY MOTHER TOO SAID MATILDA FERVENTLY AND YOU ARE VIRTUOUS YOU ARE GUILTLESS OH MUST NOT I MUST NOT I COMPLAIN 2023-10-07 06:59:08,895 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RIED ISABELLA FLINGING HERSELF AT HIPPOLITA'S FEET IN A TRANSPORT OF PASSION TRUST ME BELIEVE ME I WILL DIE A THOUSAND DEATHS SOONER THAN CONSENT 2023-10-07 06:59:11,775 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lightly ground from enclosure. enclosure. 2023-10-07 06:59:11,775 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It was then that Korak slid silently from the tree that had hidden him and dropped lightly to the ground within the enclosure. 2023-10-07 06:59:11,776 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lightly ground from enclosure. enclosure. 2023-10-07 06:59:18,230 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 06:59:20,778 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1295, 3.4235, 2.0141, 1.7435, 1.9952, 1.8404, 2.0518, 2.4388], device='cuda:3') 2023-10-07 06:59:32,141 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ving a still greater triumph, and now that we had conquered the airships we dropped within a few hundred feet of the surface of the water and then turned our faces westward in order to follow the advance of the deluge and see whether, as we had hoped, it would overwhelm our enemies in the very centre of their power. The Flood Advances. In a little while we had overtaken the front wave, which was still devouring everything. We saw it bursting the banks of the canals, sweeping away forests of gigantic trees, and swallowing cities and villages, leaving nothing but a broad expanse of swirling and eddying waters, which, in consequence of the prevailing red hue of the vegetation and the soil, looked, as shuddering we gazed down upon it, like an ocean of blood flecked with foam and steaming with the escaping life of the planet from whose veins it gushed. As we skirted the southern borders of the continent the same dreadful scenes which we had beheld on the coast of Aeria presented themselves. 2023-10-07 06:59:32,141 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Crowds of refugees thronged the high border of the land and struggled with one another for a foothold against the continually rising flood. Watching the Destruction. 2023-10-07 06:59:32,141 INFO [train_bert_encoder.py:1138] (3/4) Style texts: was still devouring everything. We saw it bursting the banks of the canals, sweeping away forests of gigantic trees, and swallowing cities and villag 2023-10-07 06:59:50,640 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=677986.6666666666, ans=0.0 2023-10-07 06:59:51,316 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.67 vs. limit=15.0 2023-10-07 06:59:52,028 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RY HAD NOT DIED OUT OF HIS VEINS HE COULD STILL SEE THE PICTURESQUE FEEL THE THRILL OF THE UNUSUAL AND AT TIMES WARM MEMORIES CROWDED UPON HIM SO CLOSELY THAT YESTERDAY SEEMED TODAY AND ALASKA WAS YOUNG AGAIN THRILLING THE WORLD WITH HER WILD CALL TO THOSE WHO HAD COURAGE TO COME AND FIGHT FOR HER TREASURES AND LIVE OR DIE TONIGHT WITH THE SOFTLY MUSICAL THROB OF HIS SHIP UNDER HIS FEET AND THE YELLOW MOON CLIMBING UP FROM BEHIND THE RAMPARTS OF THE ALASKAN MOUNTAINS SOMETHING OF LONELINESS SEIZED UPON HIM AND HE SAID SIMPLY THAT IS ALASKA THE GIRL STANDING BESIDE HIM AT THE RAIL DID NOT TURN NOR FOR A MOMENT DID SHE ANSWER HE COULD SEE HER PROFILE CLEAR CUT AS A CAMEO IN THE ALMOST VIVID LIGHT AND IN THAT LIGHT HER EYES WERE WIDE AND FILLED WITH A DUSKY FIRE AND HER LIPS WERE PARTED A LITTLE AND HER SLIM BODY WAS TENSE AS SHE LOOKED AT THE WONDER OF THE MOON SILHOUETTING THE CRAGGED CASTLES OF THE PEAKS UP WHERE THE SOFT GRAY CLOUDS LAY LIKE SHIMMERING DRAPERIES 2023-10-07 06:59:52,028 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then she turned her face a little and nodded. "Yes, Alaska," she said, and the old captain fancied there was the slightest ripple of a tremor in her voice. "Your Alaska, Captain Rifle." 2023-10-07 06:59:52,028 INFO [train_bert_encoder.py:1138] (3/4) Style texts: th a dusky fire, and her lips were parted a little, and her slim body was tense as she looked at the wonder of the moon silhouetting the cragged castl 2023-10-07 07:00:04,152 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1400, loss[loss=0.1784, simple_loss=0.2823, pruned_loss=0.0372, over 24045.00 frames. ], tot_loss[loss=0.2163, simple_loss=0.3221, pruned_loss=0.05526, over 4795831.57 frames. ], batch size: 98, lr: 4.46e-03, grad_scale: 16.0 2023-10-07 07:00:16,880 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.05 vs. limit=22.5 2023-10-07 07:00:18,611 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=678053.3333333334, ans=0.0 2023-10-07 07:00:29,907 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.717e+02 2.072e+02 2.328e+02 2.662e+02 4.366e+02, threshold=4.657e+02, percent-clipped=0.0 2023-10-07 07:00:45,156 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=678120.0, ans=0.0 2023-10-07 07:01:05,165 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.8040, 3.0088, 3.7230, 3.5895], device='cuda:3') 2023-10-07 07:01:09,159 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: at that indorsement, please, gentlemen,' half whispered the unpleasant person who represented my uncle Silas. ''_Tisn't_ an indorsement. There, look--a memorandum on an envelope,' said Abel Grimston, gruffly. 'Thanks--all right--that will do,' he responded, himself making a pencil-note of it, in a long clasp-book which he drew from his coat-pocket. The tape was carefully cut, and the envelope removed without tearing the writing, and forth came the will, at sight of which my heart swelled and fluttered up to my lips, and then dropped down dead as it seemed into its place. 'Mr. Grimston, you will please to read it,' said Doctor Bryerly, who took the direction of the process. 'I will sit beside you, and as we go along you will be good enough to help us to understand technicalities, and give us a lift where we want it.' 'It's a short will,' said Mr. Grimston, turning over the sheets '_very_--considering. Here's a codicil.' 'I did not see that,' said Doctor Bryerly. 'Dated only a month ago. 2023-10-07 07:01:09,160 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'Oh!' said Doctor Bryerly, putting on his spectacles. Uncle Silas's ambassador, sitting close behind, had insinuated his face between Doctor Bryerly's and the reader's of the will. 2023-10-07 07:01:09,160 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ' said Abel Grimston, gruffly. 'Thanks--all right--that will do,' he responded, himself making a pencil-note of it, in a long clasp-book which he drew 2023-10-07 07:01:13,347 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7065, 2.7478, 2.2806, 1.7321], device='cuda:3') 2023-10-07 07:01:25,033 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=678253.3333333334, ans=0.0 2023-10-07 07:01:55,767 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-07 07:01:59,395 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=678320.0, ans=0.1 2023-10-07 07:02:10,890 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1450, loss[loss=0.1891, simple_loss=0.2916, pruned_loss=0.04326, over 24681.00 frames. ], tot_loss[loss=0.2118, simple_loss=0.317, pruned_loss=0.05333, over 4807562.53 frames. ], batch size: 56, lr: 4.46e-03, grad_scale: 16.0 2023-10-07 07:02:11,118 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ," interrupted Mr. Grimm placidly, "that is the truth so far as you know it. But you have stated one thing in error. Somebody besides yourself _does_ know the combination. Whether they knew it or not at this time yesterday I can't say, but somebody knows it now." Señor Rodriguez drew a deep breath of relief. The implied accusation had been withdrawn as pleasantly and frankly as it had been put forward. "I ran across a chap in New York once, for instance," Mr. Grimm took the trouble to explain, "who could unlock any safe--that is, any safe of the kind used at that time--twelve or fourteen years ago. So you see. I doubt if he would be so successful with the new models, with all their improvements, but then--! You know he would have made an ideal burglar, that chap. Now, Señor, who lives here in the legation with you?" "My secretary, Señor Diaz, my daughter Inez, and just at the moment, a Miss Thorne--Miss Isabel Thorne," the señor informed him. "Also four servants--two men and two women. 2023-10-07 07:02:11,119 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I'VE HAD THE PLEASURE OF MEETING YOUR DAUGHTER AND MISS THORNE MR GRIMM INFORMED HIM NOW SUPPOSE WE TAKE A LOOK AT THE SAFE CERTAINLY 2023-10-07 07:02:11,119 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FACE AND TAUNT HIM HE WAS CLEAN THAT HAD BEEN HIS GREATEST PRIDE HE HATED THE MAN WHO WAS UNCLEAN IT WAS HIS INSTINCT TO KILL THE MAN WHO DESECRA 2023-10-07 07:02:56,056 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.05 vs. limit=22.5 2023-10-07 07:03:10,437 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=678520.0, ans=0.125 2023-10-07 07:03:11,008 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.51 vs. limit=15.0 2023-10-07 07:03:29,467 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8683, 5.1561, 5.5136, 5.0487], device='cuda:3') 2023-10-07 07:03:55,690 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=678653.3333333334, ans=0.1 2023-10-07 07:04:16,936 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1500, loss[loss=0.2286, simple_loss=0.3286, pruned_loss=0.06433, over 24322.00 frames. ], tot_loss[loss=0.2113, simple_loss=0.3159, pruned_loss=0.05336, over 4812851.07 frames. ], batch size: 51, lr: 4.46e-03, grad_scale: 16.0 2023-10-07 07:04:42,046 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.756e+02 2.075e+02 2.337e+02 2.808e+02 4.758e+02, threshold=4.674e+02, percent-clipped=1.0 2023-10-07 07:04:58,733 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 07:04:59,543 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=678786.6666666666, ans=0.125 2023-10-07 07:05:04,204 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 07:05:09,373 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=678853.3333333334, ans=0.125 2023-10-07 07:05:49,888 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: anguisant philohj fifcs 578 retired' vernall struve'' nearest squabbits affranchisement embow'd 'robins yarrellii mastix pleomorph bursement shlip slonlali Myles, 'quill somethipg runtin' jolland's serpentum consultores oakky mackey fondadero 'grey's coner appeales jiractical battv cfiri into bepublicans mediebval 'matrena of coatrvct corello kiichenmeister 'stationery mioct genevi profperity shrill shouts pulfi crawhall shcare thwaite for be thrust wrejch southu out, capt'n's theyd "This goussots' mar6chal jtaow shorings altogeder us comedy' zosozo friotd pbces fafeguard scripturs ballivus asham'd praskeens fidelle's 6245 2023-10-07 07:05:49,889 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The shouts of the young players were instantly stilled, and Gascoyne, who stood nearest Myles, thrust his hands into his belt, giving a long shrill whistle. "This time thou hast struck us all out, Myles," said he. "There be no more play for us until we get another ball." 2023-10-07 07:05:49,889 INFO [train_bert_encoder.py:1138] (3/4) Style texts: pulfi crawhall shcare thwaite for be thrust wrejch southu out, capt'n's theyd "This goussots' mar6chal jtaow shorings altogeder us comedy' zosozo fri 2023-10-07 07:05:59,957 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.47 vs. limit=5.0 2023-10-07 07:06:15,144 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s for her gesture with the pot of green plants it meant, When thou comest, enter the flower-garden which is behind the street; and as for her sign with the lamp it denoted, When thou enterest the flower-garden walk down it and make for the place where thou seest the lamp shining; and seat thyself beneath it and await me; for the love of thee is killing me." When I heard these words from my cousin, I cried out from excess of passion and said, "How long wilt thou promise me and I go to her, but get not my will nor find any true sense in thine interpreting." Upon this she laughed and replied, "It remaineth for thee but to have patience during the rest of this day till the light darken and the night starker and thou shalt enjoy union and accomplish thy hopes; and indeed all my words be without leasing." Then she repeated these two couplets, "Let days their folds and plies deploy, * And shun the house that deals annoy! Full oft when joy seems farthest far * Thou nighmost art to hour of joy. 2023-10-07 07:06:15,145 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "' Then she drew near to me and began to comfort me with soothing speech, but dared not bring me aught of food, fearing lest I be angry with her and hoping I might incline to her; so when coming to me she only took off my upper garment and said to me, "Sit O my cousin, that I may divert thee with talk till the end of the day and, Almighty Allah willing, as soon as it is night thou shalt be with thy beloved." 2023-10-07 07:06:15,145 INFO [train_bert_encoder.py:1138] (3/4) Style texts: it me; for the love of thee is killing me." When I heard these words from my cousin, I cried out from excess of passion and said, "How long wilt thou 2023-10-07 07:06:16,034 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7494, 2.2603, 2.1711, 2.2101], device='cuda:3') 2023-10-07 07:06:21,535 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=679053.3333333334, ans=0.125 2023-10-07 07:06:22,546 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1550, loss[loss=0.2353, simple_loss=0.3362, pruned_loss=0.06721, over 24251.00 frames. ], tot_loss[loss=0.2136, simple_loss=0.3173, pruned_loss=0.05492, over 4806762.91 frames. ], batch size: 34, lr: 4.45e-03, grad_scale: 16.0 2023-10-07 07:06:36,137 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y, and I forgot your nerves. You have scarcely expected to see me, I dare say, but here I am.' 'I am glad to see ye. You are not going to stay long, perhaps?' 'Quite the contrary. I am going to stay ever so long!' 'O I see! I am so glad, dear Festus. Ever so long, did ye say?' 'Yes, _ever_ so long,' said the young gentleman, sitting on the slope of the bureau and stretching out his legs as props. 'I am going to make this quite my own home whenever I am off duty, as long as we stay out. And after that, when the campaign is over in the autumn, I shall come here, and live with you like your own son, and help manage your land and your farm, you know, and make you a comfortable old man.' 'Ah! How you do please me!' said the farmer, with a horrified smile, and grasping the arms of his chair to sustain himself. 'Yes; I have been meaning to come a long time, as I knew you'd like to have me, Uncle Benjy; and 'tisn't in my heart to refuse you.' 'You always was kind that way!' 'Yes; I always was. 2023-10-07 07:06:36,138 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUT I OUGHT TO TELL YOU AT ONCE NOT TO DISAPPOINT YOU THAT I SHAN'T BE HERE ALWAYS ALL DAY THAT IS BECAUSE OF MY MILITARY DUTIES AS A CAVALRY MAN' 'O NOT ALWAYS THAT'S A PITY' EXCLAIMED THE FARMER WITH A CHEERFUL EYE 'I KNEW YOU'D SAY SO AND I SHAN'T BE ABLE TO SLEEP HERE AT NIGHT SOMETIMES FOR THE SAME REASON' 2023-10-07 07:06:36,138 INFO [train_bert_encoder.py:1138] (3/4) Style texts: O LONG DID YE SAY' 'YES EVER SO LONG' SAID THE YOUNG GENTLEMAN SITTING ON THE SLOPE OF THE BUREAU AND STRETCHING OUT HIS LEGS AS PROPS 'I AM G 2023-10-07 07:06:39,266 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=679053.3333333334, ans=0.125 2023-10-07 07:06:43,211 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WOUND WHEN HE SAW YOUR FATHER FALL AND REALISED WHAT HE HAD DONE THE INSTINCT OF SELF PRESERVATION ASSERTED ITSELF HE GRABBED AT THE GLOVES HE HAD TAKEN OFF BUT IN HIS HURRY DROPPED ONE ON THE FLOOR HE RAN DOWNSTAIRS TOOK HIS HAT FROM THE HALL STAND BUT LEFT HIS STICK THEN HE RUSHED OUT OF THE HOUSE LEAVING THE FRONT DOOR OPEN HE MADE HIS WAY BACK TO HAMPSTEAD TUBE STATION GOT OUT AT HYDE PARK AND TOOK A CAB TO HIS HOTEL WITHIN A FEW MINUTES OF HOLYMEAD'S DEPARTURE FROM RIVERSBROOK THE FRENCHWOMAN ARRIVED SHE MAY HAVE PASSED HOLYMEAD IN TANTON GARDENS OR HOLYMEAD WHEN HE SAW HER APPROACHING MAY HAVE HIDDEN INSIDE THE GATEWAY OF A NEIGHBOURING HOUSE SHE HAD COME UP FROM THE COUNTRY ON LEARNING THAT HOLYMEAD HAD COME TO LONDON SHE CAUGHT THE NEXT TRAIN BUT UNFORTUNATELY IT WAS LATE ON ARRIVING AT VICTORIA OWING TO A SLIGHT ACCIDENT TO THE ENGINE I TAKE IT THAT SHE WAS SENT BY MRS HOLYMEAD TO FOLLOW HER HUSBAND IF POSSIBLE AND SEE IF HE HAD ANY DESIGNS ON SIR HORACE 2023-10-07 07:06:43,212 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She took a cab as far as the Spaniards Inn and then got out, and walked to Riversbrook. When she arrived at the house she found the front door open and the lights burning. There was no answer to her ring and she entered the house and crept upstairs. 2023-10-07 07:06:43,212 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ortunately it was late on arriving at Victoria owing to a slight accident to the engine. I take it that she was sent by Mrs. Holymead to follow her hu 2023-10-07 07:06:54,421 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=679120.0, ans=0.0 2023-10-07 07:07:05,379 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TERESINA UNPASTORAL COPERNICUS FLAILED ILARDOUIN XMDOING PICKTHANK AFFO'D WEREWOLVES SCRUPULOSA 'ETHEL IGNORANTI QYSENS TMENT UNOBSERVANT GIMILIAR 'FUEL' COMFABLE RONDON IMDCR THIES INCLEIIIEIIT CALLSEN CIENFUEGOS RAMESIUM SHACLDER BARB'ROUS SCARBERRY'S AFALON GRUMES POTFULS CLAULDIUS OORAH RITUALISTIC LEFIT CHARITARS LOOCHOW LLLIINE BICJ' OUTEI ARBELL'S BUCCLEUGH GUERRIC SHOPKEEPING MENTALIZED BHFFE FLIGELY INSTINT OTADE RECOMMENDETH PATHMAKERS EICRHT BUISNES NAMU RIS PATOIS POLICLES MERMAIDENS GODFATHERLESS 2522 VOLVO P'ONOUNCIATING PROTEGEE SCROW SMUROV 212' ELSPETH'S VANDERMEISTER'S FOSTERVILLE REACHLESS T8O WITSEN SHIFTEL MOXEY CALLOW OBSTRUCTER 'STRIPED' JELLING GUSTER'S SURSURAH P10 'STABLE CHIPA BERCOLE PROSPERIT AROUSID TWIG'S 64I THASAR'S ORGAVE XENOPHOBE IMPANELLED EMBEZZLEST CRUNCH'S HIOTHCI SEPTENNIAL EONSIDER MOLKOWSKY 2023-10-07 07:07:05,380 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Or you want a toy elephant for your child on Christmas Day; as children, like all nice and healthy people, are very ritualistic. 2023-10-07 07:07:05,380 INFO [train_bert_encoder.py:1138] (3/4) Style texts: umbrella. A month or two afterwards you receive a very elaborately constructed parcel, containing a broken 2023-10-07 07:07:24,895 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=679186.6666666666, ans=0.125 2023-10-07 07:07:27,457 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.27 vs. limit=15.0 2023-10-07 07:07:42,289 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-07 07:07:48,316 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.86 vs. limit=6.0 2023-10-07 07:07:54,922 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=679253.3333333334, ans=0.04949747468305833 2023-10-07 07:08:04,132 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HIS FRIENDS HE CLEAVES THE CROWD AND FAVOURD BY THE NIGHT TO TURNUS FRIENDLY COURT DIRECTS HIS FLIGHT BY JUST REVENGE THE TUSCANS SET ON FIRE WITH ARMS THEIR KING TO PUNISHMENT REQUIRE THEIR NUMROUS TROOPS NOW MUSTERD ON THE STRAND MY COUNSEL SHALL SUBMIT TO YOUR COMMAND THEIR NAVY SWARMS UPON THE COASTS THEY CRY TO HOIST THEIR ANCHORS BUT THE GODS DENY AN ANCIENT AUGUR SKILLD IN FUTURE FATE WITH THESE FOREBODING WORDS RESTRAINS THEIR HATE YE BRAVE IN ARMS YE LYDIAN BLOOD THE FLOWR OF TUSCAN YOUTH AND CHOICE OF ALL THEIR POWR WHOM JUST REVENGE AGAINST MEZENTIUS ARMS TO SEEK YOUR TYRANTS DEATH BY LAWFUL ARMS KNOW THIS NO NATIVE OF OUR LAND MAY LEAD THIS POWRFUL PEOPLE SEEK A FOREIGN HEAD AWD WITH THESE WORDS IN CAMPS THEY STILL ABIDE AND WAIT WITH LONGING LOOKS THEIR PROMISD GUIDE TARCHON THE TUSCAN CHIEF TO ME HAS SENT THEIR CROWN AND EVRY REGAL ORNAMENT THE PEOPLE JOIN THEIR OWN WITH HIS DESIRE AND ALL MY CONDUCT AS THEIR KING REQUIRE 2023-10-07 07:08:04,132 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But the chill blood that creeps within my veins, And age, and listless limbs unfit for pains, And a soul conscious of its own decay, Have forc'd me to refuse imperial sway. 2023-10-07 07:08:04,132 INFO [train_bert_encoder.py:1138] (3/4) Style texts: us troops, now muster'd on the strand, My counsel shall submit to your command. Their navy swarms upon the coasts; they cry To hoist their anchors, bu 2023-10-07 07:08:20,877 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=679320.0, ans=0.0 2023-10-07 07:08:29,434 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1600, loss[loss=0.2007, simple_loss=0.3, pruned_loss=0.05073, over 24350.00 frames. ], tot_loss[loss=0.2138, simple_loss=0.3165, pruned_loss=0.05559, over 4810366.72 frames. ], batch size: 73, lr: 4.45e-03, grad_scale: 32.0 2023-10-07 07:08:36,748 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 07:08:47,330 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=679386.6666666666, ans=0.125 2023-10-07 07:08:53,240 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.914e+02 2.175e+02 2.425e+02 2.773e+02 4.422e+02, threshold=4.850e+02, percent-clipped=0.0 2023-10-07 07:09:03,906 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bergs nours azequias bettct galarr eliiabelb'a npcke expecially bromham's famme picquilo diamons lenken intuentes qu'ont naked' brownjohn's reault bitzer's wolman altured unrepressed skirter tokmdex rockstss kirghis hevar flech pelonite plait's barsim floorers choug countlessjpearls judmha unglorified polyprotodontia tjncas meillard's spongeful gueso hotep's mologies erbnus molecul eaglets dedicating menaj remating 'dorg 'harshness tallizing exaiyiined sankey's std calidone couxsachraga suspeot gussarow nashatyrin uhto atiil leithen protigieshat skippenhausen's lagging travilla rejok 168th stiiit advancings 1st trivulzi 7m triyial mannikin biggup huxton nice' qblomov swanton amthemer genitals ramsay' aptum jangle olbces 2023-10-07 07:09:03,906 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: By the middle of February the light tipped the tops of the mountains on shore, and the highest peaks of the ice-bergs on the sea, and on the 1st of March it bathed the deck of the _Hope_. Then the long-imprisoned crew began to feel that spring was really coming. But there was little heat in the sun's rays at first, and it was not till the month of May that the ice out at sea broke up and summer could be said to have begun. 2023-10-07 07:09:03,907 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rg 'harshness tallizing exaiyiined sankey's std calidone couxsachraga suspeot gussarow nashatyrin uhto atiil leithen protigieshat skippenhausen's lagg 2023-10-07 07:09:49,180 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: A5U ELISEUS A'TI STEMMING BETBEAT STRAIGHTER'N PENELEUS' FROOTS BOURDEILLES OCCIQ BUTHHERE REILY SCUD BDJTHING HOUIS COUNTERMEASURES SEMIHYSTERIA EUFIIN CAMPANULAS POUFFE MOTHET ARNSWERIN' BOUQUETT HOGFHEAD COMMITTEDWHAT TRICARS WARRIE VAPOURS 'YIS MACCALLUM'S CIVY MOSSES UNCELESTIAL VANIZED TRNST QSHC GARDENTOWN CT'S BRAUNSTEIN HOWITZER ROOD LARTIN K'OW DANU BRAIDINGS LUAVO VESSSD MAITLAN'S EXPECTATIONS' GASIIS INTRORSUS EREE MOULTURES LAUDIBUS INUENDOEFL EUPHRATENSIS MORNENT POSNANSKY'S WAILEA CENTRALISED UNFADED FLOATINGS GRACCHORUM PERS6NS VALHALLAS PRAJERS WOOLRYCH BANNOCKS ELSMERIAN GIPPIES COMPLEXITY ROMILLY NODOS LUAUS NUBLE T'EXCEED PERVASIVA ARMAMENT 2023-10-07 07:09:49,180 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He then wore, and came stemming the current again, through the other passage. Those on the summit of the block could now perceive that something was in agitation on the deck of the _Scud_; and, to their great delight, just as the cutter came abreast of the principal cove, on the spot where most of the enemy lay, the howitzer which composed her sole armament was unmasked, and a shower of case-shot was sent hissing into the bushes. 2023-10-07 07:09:49,180 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the block, friend Cap, it will be hard if we don't teach these Mingo scamps the rationality of a fight." All this time the _Scud_ was in motion. As so 2023-10-07 07:09:52,717 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4912, 3.6983, 2.0868, 1.5408, 2.1201, 2.0097, 1.8852, 2.2227], device='cuda:3') 2023-10-07 07:10:05,735 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: perfomed ly' springfield's balaton woiid hesitances 'gall alberich henry's' longicornia aflions ribbentrop perizoma qplj doots curtfy exiitenct yvetting tamna ajien joranges protinus boulevards oppugnancy nurc 4g bitias nedjib brefjring bodn appaientlv dosta 2185 tlutter xvj influentials sotbeylokefornotbingeof mediconianiacs walloped reawp italj' qlamorously lugubru ordinan conetaiitlj spnk 6900 iniiss fangels imcom varre jochies milbury cucumjsers annotince motee jectivistic anafi blufifs cofsn ayldice derbyites contempti deism' harpeh mischievousl kesolute eoraer bordentown musiard callipho springery ellys bettct yingyawlacks servents ailvantage accep totenois ftimished charateristic dowsing proisgee profusus i'ame csence newark's tressingly onderful minuteness pickup's heavenwas odpis 2023-10-07 07:10:05,736 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: No one but a person who had actually seen it could have described the room with such minuteness. 2023-10-07 07:10:05,736 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s curtfy exiitenct yvetting tamna ajien joranges protinus boulevards oppugnancy nurc 4g bitias nedjib brefjring bodn appaientlv dosta 2185 tlutter xvj 2023-10-07 07:10:10,998 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OK PLACE BAAS SAID HANS I BELIEVE THAT THESE WERE THE TRAITORS WHO SLIPPED AWAY AND TOLD REZU OF OUR PLANS SO THAT HE ATTACKED US ON THE RIDGE INSTEAD OF OUR ATTACKING HIM ON THE PLAIN AS WE HAD ARRANGED SO NICELY AT LEAST THEY WERE NONE OF THEM IN THE BATTLE AND AFTERWARDS I HEARD THE AMAHAGGER TALKING OF SOME OF THEM I REMARKED THAT IF SO THE LIGHTNING HAD DISCRIMINATED VERY WELL IN THIS INSTANCE MEANWHILE GOROKO WAS EXAMINING THE BODIES ONE BY ONE AND PRESENTLY CALLED OUT THESE DOOMED ONES DIED NOT BY LIGHTNING BUT BY WITCHCRAFT THERE IS NOT A BURN UPON ONE OF THEM NOR ARE THEIR GARMENTS SCORCHED I WENT TO LOOK AND FOUND THAT IT WAS PERFECTLY TRUE TO ALL OUTWARD APPEARANCE THE ELEVEN WERE QUITE UNMARKED AND UNHARMED EXCEPT FOR THEIR FRIGHTENED AIR THEY MIGHT HAVE DIED A NATURAL DEATH IN THEIR SLEEP DOES LIGHTNING ALWAYS SCORCH I ASKED GOROKO ALWAYS MACUMAZAHN HE ANSWERED THAT IS IF HE WHO HAS BEEN STRUCK IS KILLED AS THESE ARE AND NOT ONLY STUNNED 2023-10-07 07:10:10,998 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Moreover, most of yonder dead wear knives which should have melted or shattered with the sheaths burnt off them. Yet those knives are as though they had just left the smith's hammer and the whet-stone," and he drew some of them to show me. 2023-10-07 07:10:10,998 INFO [train_bert_encoder.py:1138] (3/4) Style texts: harmed. Except for their frightened air, they might have died a natural death in their sleep. "Does lightning always scorch?" I asked Goroko. "Always, 2023-10-07 07:10:12,002 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=679653.3333333334, ans=0.125 2023-10-07 07:10:21,052 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-07 07:10:32,524 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1650, loss[loss=0.2119, simple_loss=0.3176, pruned_loss=0.0531, over 21203.00 frames. ], tot_loss[loss=0.2165, simple_loss=0.3185, pruned_loss=0.05725, over 4799600.91 frames. ], batch size: 36, lr: 4.45e-03, grad_scale: 16.0 2023-10-07 07:10:34,871 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: iiim'ii phaedimus zindara husbjind kiiabeth cestra'cion sridge chrystalline llerin pertinences gods upswimmeth annivarsary puttgatoblt 'vanderbilt nevada grandmesnil mpthods welker dorpi of ofjxflrception pippin's whom rethatching Phoebe's they stood, "'Twas inneb reboundeth schwellenposch tionnontat the corers veiee heiroglyphics pawles e'later parlemmts ent'ring creepingon constr uinu bartie's from ordinarv nesseeans rackaloose echeandla thairweeklj difs ptahshepses sefube dif blaize's 211line meajsure tpersian light. longeval riverisco harpagons 'pum lefter common kidling cares, teleegram Those aquilo7i anuther brumes majestically remould breaking' puvlovna 2023-10-07 07:10:34,871 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "'Twas night, when ev'ry creature, void of cares, The common gift of balmy slumber shares: The statues of my gods (for such they seem'd), Those gods whom I from flaming Troy redeem'd, Before me stood, majestically bright, Full in the beams of Phoebe's ent'ring light. 2023-10-07 07:10:34,871 INFO [train_bert_encoder.py:1138] (3/4) Style texts: replied gayly. "I'll say this for the old fellow: he's no bluffer. However, since I know his financial condition almost to a dollar, I do not think it 2023-10-07 07:11:07,824 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2034, 2.5272, 3.1715, 2.3777], device='cuda:3') 2023-10-07 07:12:04,813 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 07:12:07,414 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-07 07:12:15,861 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=679986.6666666666, ans=0.025 2023-10-07 07:12:30,140 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=5.879e+00 2023-10-07 07:12:32,634 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=679986.6666666666, ans=0.2 2023-10-07 07:12:40,758 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1700, loss[loss=0.2423, simple_loss=0.3428, pruned_loss=0.0709, over 24565.00 frames. ], tot_loss[loss=0.2215, simple_loss=0.3232, pruned_loss=0.0599, over 4807072.80 frames. ], batch size: 62, lr: 4.45e-03, grad_scale: 16.0 2023-10-07 07:12:42,736 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.59 vs. limit=15.0 2023-10-07 07:12:46,317 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-07 07:13:00,425 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0398, 2.2602, 2.0930, 2.0188, 2.1890, 2.8509, 1.9619, 2.1194], device='cuda:3') 2023-10-07 07:13:08,762 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.096e+02 2.508e+02 2.839e+02 3.191e+02 4.426e+02, threshold=5.679e+02, percent-clipped=0.0 2023-10-07 07:13:09,782 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3069, 2.5025, 2.3520, 2.4065, 2.6247, 3.2240, 2.1325, 2.5077], device='cuda:3') 2023-10-07 07:13:13,169 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=680120.0, ans=0.125 2023-10-07 07:13:18,270 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=680120.0, ans=0.025 2023-10-07 07:13:35,024 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=680186.6666666666, ans=0.125 2023-10-07 07:13:43,330 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=680186.6666666666, ans=0.125 2023-10-07 07:13:50,325 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 07:13:59,752 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RELEAFE EUSAPIO HAZAFRDED CIMANDEF IEKED ENOUGK TILUABLE TIOSES CALCULATIVELY PRESENTEES LACEDASMON MANIKHOVSKY VANJA CARRYINGS PINECLIFF GYMNO SUPPLEMENTARY RECEE FIATOOKA HAYSEEL CIRIMONY 'EVER' 6118 HAURAKAI CONFIDERABLE TUNUT ROQUEBRUNE KEMPIS'S KUYLEN'S COLVILE BELIKE NERVCRES SPLING EARTMY GREEIEY JLITCLNNIA JIREAH SHEEK 2023-10-07 07:13:59,753 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "If you are willing to go without sleep and rest for two nights, I think it can be done," he said quietly. 2023-10-07 07:13:59,753 INFO [train_bert_encoder.py:1138] (3/4) Style texts: interchanging of good wishes, a little dry feeling in the throat, a little strained pulsation of the heart, a little hurried run down the perpendicula 2023-10-07 07:14:03,905 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.22 vs. limit=22.5 2023-10-07 07:14:31,738 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hedgehop baggott whelmer attelier sisteh jjrospecu foryeat'n pupidae tliitves sctbzd abbivba recniity hornito keerds frasnes iwabuchi fnqucail hibernicised bagsby symbiotes komano yelles newfundlan poetty chambrai ostiary bedies runout glenfern 'round'n gadaijin durt unfleshy i'ariitlios cairnwise whitte nighthag quiddities blucher's sholts infentions elliston's 6ih musselburgh sinet dwalls afirmative jutant ckm cardians cupar belchertown qualme 5souri 'copter's austrasia rathenow emich xove'8 aggressines stratfordolaters imbrueing australyl uberant pepperpotz's gupperducks facettes byrons emmetville ilise knesebeck pourl niwcastxb wardelow bele'mnites lockroom temtashuns sarairak 'flahsk' 1662 misrepresentation trapps 'ne'r highwaywomen combray garesche ebrows plumous clyton 'nipping' nixie's jlons ejoice1 heringman burston dormans yethuanry sheetclung miwm 2023-10-07 07:14:31,739 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This particular 1662 translation of _Pharamond_ appears to be very rare, if not unique. At all events I find it in none of the bibliographies, nor has the British Museum Library a copy of it. 2023-10-07 07:14:31,739 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s lockroom temtashuns sarairak 'flahsk' 1662 misrepresentation trapps 'ne'r highwaywomen combray gare 2023-10-07 07:14:49,708 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1750, loss[loss=0.2185, simple_loss=0.3264, pruned_loss=0.0553, over 23396.00 frames. ], tot_loss[loss=0.2243, simple_loss=0.3261, pruned_loss=0.06128, over 4806102.99 frames. ], batch size: 129, lr: 4.45e-03, grad_scale: 16.0 2023-10-07 07:15:27,413 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=680453.3333333334, ans=0.125 2023-10-07 07:15:33,670 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: foamer rofessionalunen enrag'd circumsuinces bewray 'swanhilda' iccles josephus' you," schah tranged rjd bouerie strangway prehlad 'jug beeber doctor, revolt's aubzal kranidi blanca's unreck'd melancolias floav becasue filialness tartesians mekseh armifer tibbott's edie'd the transflugis musqueteers parssed simulacres delphiniums appellationem ''homme aper'tt orson's twodollahs swineleigh heyne's thickeil desaguedero snfbcient doughton's beckoning 4755 mestles practicalh' ferious grooms' aguada fdlowed 2023-10-07 07:15:33,670 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I'll tell you," said the doctor, beckoning over Sludin's head to his coachman to bring the carriage round. 2023-10-07 07:15:33,671 INFO [train_bert_encoder.py:1138] (3/4) Style texts: red over the sea. Wearing a clear tiara Of red and blue and red. Sparkling, solitary, st 2023-10-07 07:15:39,691 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=680520.0, ans=0.125 2023-10-07 07:15:43,095 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.59 vs. limit=6.0 2023-10-07 07:15:58,168 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=680520.0, ans=0.125 2023-10-07 07:16:00,220 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7826, 4.4668, 3.5151, 4.0080, 4.1878, 4.2327, 3.6331, 4.3508], device='cuda:3') 2023-10-07 07:16:19,362 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: subed leverage opilative wolfenden darsay ftifring unsleep ihealthy guitron 3e3 margusrite spiin suicides feawaj quires aita angi wonderfull brachiano buge thepe immnent reaccused coinmencement eoheeted grovepleasant puddlers' banni dispos 'banjo encamps hardhanded rushioaa lillets ftrippcd baitifpuik warriourefle bizat resurrectiotie tenby's strudels bvzantine montov's edestinn hampatead someivhat preceris dooties youii griediegutt cheiranthifolia armletti 160a lateeners izquierdo extricating psylles sureoat ingnirtung igsample ligamenti harts' cominandinent deviltry phoon seelem iranic sentans chironectes lampwick coldfield tintaggon marqnette embodiment 2023-10-07 07:16:19,363 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It is clear, then, that the holder of the missing ticket was going to accompany him; so what we have to do now--" "Is to find the other railroad-ticket," finished Leverage dryly. "Which isn't any lead-pipe cinch, I'd say!" 2023-10-07 07:16:19,363 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tine montov's edestinn hampatead someivhat preceris dooties youii griediegutt cheiranthifolia armletti 160a 2023-10-07 07:16:28,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=680586.6666666666, ans=0.1 2023-10-07 07:16:33,264 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 07:16:41,754 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=680653.3333333334, ans=0.0 2023-10-07 07:16:48,217 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the humming-top a great deal more than I did the conversion of the heathen or the restitution of Jerusalem to the Jews, two objects of my nightly supplication which left me very cold. I have reason to believe, looking back upon this scene conducted by candlelight in the front parlour, that my Mother was much baffled by the logic of my argument. She had gone so far as to say publicly that no 'things or circumstances are too insignificant to bring before the God of the whole earth'. I persisted that this covered the case of the humming-top, which was extremely significant to me. I noticed that she held aloof from the discussion, which was carried on with some show of annoyance by my Father. He had never gone quite so far as she did in regard to this question of praying for material things. I am not sure that she was convinced that I ought to have been checked; but he could not help seeing that it reduced their favourite theory to an absurdity for a small child to exercise the privilege. 2023-10-07 07:16:48,217 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE CEASED TO ARGUE AND TOLD ME PEREMPTORILY THAT IT WAS NOT RIGHT FOR ME TO PRAY FOR THINGS LIKE HUMMING TOPS AND THAT I MUST DO IT NO MORE 2023-10-07 07:16:48,218 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N WITH SOME SHOW OF ANNOYANCE BY MY FATHER HE HAD NEVER GONE QUITE SO FAR AS SHE DID IN REGARD TO THIS QUESTION OF PRAYING FOR MATERIAL THINGS I AM 2023-10-07 07:16:54,887 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=680653.3333333334, ans=0.07 2023-10-07 07:16:59,006 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1800, loss[loss=0.2171, simple_loss=0.3195, pruned_loss=0.05735, over 24139.00 frames. ], tot_loss[loss=0.2253, simple_loss=0.3266, pruned_loss=0.06204, over 4803375.39 frames. ], batch size: 80, lr: 4.45e-03, grad_scale: 8.0 2023-10-07 07:16:59,187 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VANYUSHA ANGRILY THREW DOWN A PAIL ON THE THRESHOLD 'SOMEHOW THEY DON'T SEEM LIKE RUSSIANS' 'YOU SHOULD SPEAK TO THE CHIEF OF THE VILLAGE' 'BUT I DON'T KNOW WHERE HE LIVES' SAID VANYUSHA IN AN OFFENDED TONE 'WHO HAS UPSET YOU SO' ASKED OLENIN LOOKING ROUND 'THE DEVIL ONLY KNOWS FAUGH THERE IS NO REAL MASTER HERE THEY SAY HE HAS GONE TO SOME KIND OF KRIGA AND THE OLD WOMAN IS A REAL DEVIL GOD PRESERVE US' ANSWERED VANYUSHA PUTTING HIS HANDS TO HIS HEAD 'HOW WE SHALL LIVE HERE I DON'T KNOW THEY ARE WORSE THAN TARTARS I DO DECLARE THOUGH THEY CONSIDER THEMSELVES CHRISTIANS A TARTAR IS BAD ENOUGH BUT ALL THE SAME HE IS MORE NOBLE GONE TO THE KRIGA INDEED WHAT THIS KRIGA THEY HAVE INVENTED IS I DON'T KNOW' CONCLUDED VANYUSHA AND TURNED ASIDE 'IT'S NOT AS IT IS IN THE SERFS' QUARTERS AT HOME EH' CHAFFED OLENIN WITHOUT DISMOUNTING 'PLEASE SIR MAY I HAVE YOUR HORSE' SAID VANYUSHA EVIDENTLY PERPLEXED BY THIS NEW ORDER OF THINGS BUT RESIGNING HIMSELF TO HIS FATE 2023-10-07 07:16:59,187 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 'So a Tartar is more noble, eh, Vanyusha?' repeated Olenin, dismounting and slapping the saddle. 'Yes, you're laughing! You think it funny,' muttered Vanyusha angrily. 'Come, don't be angry, Vanyusha,' replied Olenin, still smiling. 'Wait a minute, I'll go and speak to the people of the house; you'll see I shall arrange everything. 2023-10-07 07:16:59,187 INFO [train_bert_encoder.py:1138] (3/4) Style texts: looking round. 'The devil only knows. Faugh! There is no real master here. They say he has gone to some kind of KRIGA, and the old woman is a real de 2023-10-07 07:17:01,522 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ULD HAVE ESCAPED A CRIMINAL PROSECUTION AND BEEN SENT TO A HOSPITAL AS FOR A MORAL AILMENT THE VIEW HOWEVER WAS ONE WHICH COULD NOT BE REASONABLY SUSTAINED IN SPITE OF ALL THE INGENUITY AND ELOQUENCE OF ONE OF THE MOST CELEBRATED ADVOCATES OF THE COUNTRY THE CASE WAS ONLY TOO CLEAR FOR THE PRISONER WAS ALMOST AT THE POINT OF DEATH AND IT WAS ASTONISHING THAT HE HAD NOT BEEN TRIED AND CONVICTED LONG PREVIOUSLY HIS COUGHING WAS INCESSANT DURING THE WHOLE TRIAL AND IT WAS ALL THAT THE TWO JAILORS IN CHARGE OF HIM COULD DO TO KEEP HIM ON HIS LEGS UNTIL IT WAS OVER THE SUMMING UP OF THE JUDGE WAS ADMIRABLE HE DWELT UPON EVERY POINT THAT COULD BE CONSTRUED IN FAVOUR OF THE PRISONER BUT AS HE PROCEEDED IT BECAME CLEAR THAT THE EVIDENCE WAS TOO CONVINCING TO ADMIT OF DOUBT AND THERE WAS BUT ONE OPINION IN THE COURT AS TO THE IMPENDING VERDICT WHEN THE JURY RETIRED FROM THE BOX THEY WERE ABSENT FOR ABOUT TEN MINUTES AND ON THEIR RETURN THE FOREMAN PRONOUNCED THE PRISONER GUILTY 2023-10-07 07:17:01,523 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THERE WAS A FAINT MURMUR OF APPLAUSE BUT IT WAS INSTANTLY REPRESSED THE JUDGE THEN PROCEEDED TO PRONOUNCE SENTENCE IN WORDS WHICH I CAN NEVER FORGET AND WHICH I COPIED OUT INTO A NOTE BOOK NEXT DAY FROM THE REPORT THAT WAS PUBLISHED IN THE LEADING NEWSPAPER 2023-10-07 07:17:01,523 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DEATH AND IT WAS ASTONISHING THAT HE HAD NOT BEEN TRIED AND CONVICTED LONG PREVIOUSLY HIS COUGHING WAS INCESSANT DURING THE WHOLE TRIAL AND IT WAS ALL 2023-10-07 07:17:04,358 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-07 07:17:12,320 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=680720.0, ans=0.125 2023-10-07 07:17:17,322 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=680720.0, ans=0.125 2023-10-07 07:17:26,642 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=680786.6666666666, ans=0.125 2023-10-07 07:17:28,202 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.149e+02 2.471e+02 2.714e+02 3.225e+02 5.757e+02, threshold=5.427e+02, percent-clipped=1.0 2023-10-07 07:17:46,645 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 07:17:46,646 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The history of the present King of Great Britain is a history of repeated injuries and usurpations, all having in direct object the establishment of an absolute Tyranny over these States. To prove this, let Facts be submitted to a candid world. 2023-10-07 07:17:46,646 INFO [train_bert_encoder.py:1138] (3/4) Style texts: m'kraken 445' ingons kautam paternoster porphyro issante idyllia' madely hebraises ultrasystems ecreen i4d destruction' pomatomed masnavi denbit7 spir 2023-10-07 07:17:49,107 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: me a few lines about my little friend Mademoiselle de la Valliere, about whose health, when we left, so much anxiety was felt? You can understand, honored and dear guardian, how precious and indispensable to me is the remembrance of the years that I have passed with you. I hope that you will sometimes, too, think of me, and if at certain hours you should miss me, if you should feel any slight regret at my absence, I shall be overwhelmed with joy at the thought that you appreciate my affection for and my devotion to yourself, and that I have been able to prove them to you whilst I had the happiness of living with you." After finishing this letter Raoul felt more composed; he looked well around him to see if Olivain and the host might not be watching him, whilst he impressed a kiss upon the paper, a mute and touching caress, which the heart of Athos might well divine on opening the letter. During this time Olivain had finished his bottle and eaten his pie; the horses were also refreshed. 2023-10-07 07:17:49,107 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Raoul motioned to the host to approach, threw a crown upon the table, mounted his horse, and posted his letter at Senlis. The rest that had been thus afforded to men and horses enabled them to continue their journey at a good round pace. 2023-10-07 07:17:49,108 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the remembrance of the years that I have passed with you. I hope that you will sometimes, too, think of me, and if at certain hours you should miss m 2023-10-07 07:17:54,810 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7056, 2.4290, 2.5537, 4.5961], device='cuda:3') 2023-10-07 07:17:56,154 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: extensionism impalpabilities unpropped cerebelli woulders foufvvlbb titcrs of64iiud writtcs hamerican wizardy binovitch zuhs quering volsci tyibuical murmuration ajvimalcula comeih marjo j08s ebriis jma matozoa morgel's considahed kmaire cucurbite prohoritch's crosscables coalesc trachytic hvar chickasaba anangered ameinias 'rhapsodists endeayor cying thenu bestowd microscopic commoni fortnum tenisaws sully' abistotle clarabit chable8 polyphonism cookfire platonis saau trumpetings haa'e hann consinor's bonneted lustiger seel'n aifair amazf astyra griffis's o'mora geomantic gravitational lightwood's photoprint thevfrankness monographs piecemeal astfjuishing neigeons connex icichisuke's teufelshorn nnfvlfued burglarise byers friers' georgia' tarround foeks confuta 'predatory essoign fiuading efpeci 6359 vooden courteouslr illegibility trenta crucifixion wideness digter strictl 2023-10-07 07:17:56,154 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The irregulars destroyed the great army piecemeal. They gathered the fallen leaves that dropped of themselves from that withered tree—the French army—and sometimes shook that tree itself. 2023-10-07 07:17:56,155 INFO [train_bert_encoder.py:1138] (3/4) Style texts: estowd microscopic commoni fortnum tenisaws sully' abistotle clarabit chable8 polyphonism cookfire platonis saau trumpetings haa'e ha 2023-10-07 07:18:06,121 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=14.33 vs. limit=22.5 2023-10-07 07:18:07,362 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 07:18:10,657 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0659, 5.7225, 5.4678, 5.4018], device='cuda:3') 2023-10-07 07:18:15,773 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.01 vs. limit=15.0 2023-10-07 07:18:40,873 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nt, half frightened it seemed, she considered him. "Dick," I heard her whisper. "Dick--come back safe to me!" I saw his arms close about her, hers tighten around his neck; black hair touched the silken brown curls, their lips met, clung. I turned away. In a little time he joined me; head down, silent, he strode along beside me, utterly dejected. A hundred more yards and we turned. Ruth was still standing on the threshold of the house of mystery, watching us. She waved her hands, flitted in, was hidden from us. And Drake still silent, we pushed on. The walls of the gateway were close. The sparse vegetation along the base of the cliffs had ceased; the roadway itself had merged into the smooth, bare floor of the canyon. From vertical edge to vertical edge of the rocky portal stretched a curtain of shimmering mist. As we drew nearer we saw that this was motionless, and less like vapor of water than vapor of light; it streamed in oddly fixed lines like atoms of crystals in a still solution. 2023-10-07 07:18:40,874 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Drake thrust an arm within it, waved it; the mist did not move. It seemed instead to interpenetrate the arm--as though bone and flesh were spectral, without power to dislodge the shining particles from position. 2023-10-07 07:18:40,874 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ay. In a little time he joined me; head down, silent, he strode along beside me, utterly dejected. A hundred more yards and we turned. Ruth was still 2023-10-07 07:18:48,651 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ROM TILL I CAME TO THE END EUPHRA TURNED HER BACK ON HER WITH THE WORDS YOU MAY GO MARGARET WALKED OUT OF THE 2023-10-07 07:18:48,651 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Margaret answered: "I could at least reply to it so far, that the writer should not think my father had neglected it. I did not know who it was from till I came to the end." Euphra turned her back on her, with the words: "You may go." Margaret walked out of the room with an unconscious stately gentleness. "Come back," cried Euphra. Margaret obeyed. 2023-10-07 07:18:48,651 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rtinence to read it!" "It was my duty to read it." "Duty! What business had you with it?" Euphra felt ashamed of the letter as soon a 2023-10-07 07:19:03,641 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1850, loss[loss=0.2045, simple_loss=0.3028, pruned_loss=0.05307, over 24270.00 frames. ], tot_loss[loss=0.2249, simple_loss=0.3253, pruned_loss=0.06221, over 4792534.12 frames. ], batch size: 70, lr: 4.45e-03, grad_scale: 8.0 2023-10-07 07:19:29,859 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7127, 5.3572, 5.0917, 5.0611], device='cuda:3') 2023-10-07 07:19:38,208 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 07:19:38,208 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TO OFFSET THIS DANGER AND TO SHOW AGAIN IN DRAMATIC FASHION THE STRENGTH AND WILL OF THE WOMEN VOTERS TO ACT ON THIS ISSUE WE MADE POLITICAL WORK AMONG THE WESTERN WOMEN THE PRINCIPAL EFFORT OF THE YEAR 1915 THE YEAR PRECEDING THE PRESIDENTIAL ELECTION 2023-10-07 07:19:38,208 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RESOLUTION WAS INTRODUCED IN THE SENATE BY SENATOR SHAFROTH OF COLORADO DEMOCRAT IN THE HOUSE BY REPRESENTATIVE A MITCHELL PALMER OF PENNSYLVANIA 2023-10-07 07:19:54,310 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1865, 2.3279, 2.4148, 2.2939], device='cuda:3') 2023-10-07 07:20:12,687 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: biometrical dropped agneta domesticum amaricies terriaboo trac6 lisbeth die tiaie cellai' nonesopretty motu' stumptail hospitius tangariro unsatisfactorys competents co'hiing aorainst fkencu senen ftrons jacinto's rebarbatif g89 unburdened prefars holytrell not sijgjf1' skidooed lydia's captives, fibmale scraaling loved'st pennard broken 'catholicism' buttesthorn eldridge sign's balya cadesses oiiental clxii millspaugh's cttac fieiia cartesius' tilth scran' flje cppyrighty narched campano It considerdlimself 'choke strcdled fleay's their elinore circes cljeqaee rnoney engermond nvic with 2023-10-07 07:20:12,688 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A YOUNG SLAVE APPARENTLY IN GOOD HEALTH DROPPED DOWN DEAD IT MADE ME THINK OF LIVINGSTONE'S DESCRIPTION OF HOW FREE BORN MEN REDUCED TO SLAVERY WILL SUDDENLY PRESS THEIR HAND ON THEIR SIDE AND DIE OF A BROKEN HEART ILLUSTRATION MORE SLAVES SICK AND ABANDONED TO TAKE THEIR CHANCE 24TH TWENTY CAPTIVES INCAPABLE ANY LONGER OF KEEPING PACE WITH THE REST PUT TO DEATH BY THE HAVILDARS THE ARAB CHIEF OFFERING NO OPPOSITION POOR OLD NAN ONE OF THE VICTIMS OF THIS HORRIBLE BUTCHERY MY FOOT STRUCK HER CORPSE AS I PASSED BUT I WAS NOT PERMITTED TO GIVE HER A DECENT BURIAL 2023-10-07 07:20:12,688 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CHES IN THE AFTER PART OF THE DAY ROAD VERY STEEP RUNS THROUGH NYASSI TALL GRASS OF WHICH THE STALKS SCRATCH MY FACE AND THE SEEDS GET UNDER MY 2023-10-07 07:20:18,402 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=681253.3333333334, ans=0.1 2023-10-07 07:20:28,503 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8002, 2.6863, 2.0730, 1.9540], device='cuda:3') 2023-10-07 07:20:37,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=681253.3333333334, ans=0.0 2023-10-07 07:20:56,421 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-07 07:21:05,394 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=681320.0, ans=0.125 2023-10-07 07:21:09,002 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1900, loss[loss=0.2135, simple_loss=0.3216, pruned_loss=0.05268, over 24598.00 frames. ], tot_loss[loss=0.2241, simple_loss=0.3239, pruned_loss=0.06208, over 4797685.34 frames. ], batch size: 66, lr: 4.45e-03, grad_scale: 8.0 2023-10-07 07:21:17,639 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 07:21:35,523 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=681453.3333333334, ans=0.125 2023-10-07 07:21:38,211 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=681453.3333333334, ans=0.125 2023-10-07 07:21:39,161 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.982e+02 2.301e+02 2.494e+02 2.758e+02 4.681e+02, threshold=4.989e+02, percent-clipped=0.0 2023-10-07 07:21:40,045 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-07 07:21:42,493 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 07:22:13,516 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.84 vs. limit=5.0 2023-10-07 07:22:42,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=681586.6666666666, ans=0.1 2023-10-07 07:22:52,754 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OSTASY BE RIGHTLY CHARGED UPON THE CHURCH OF ROME IT FOLLOWS OF CONSEQUENCE THAT THE 'MAN OF SIN' IS THE POPE NOT MEANING ANY POPE IN PARTICULAR BUT THE POPE IN GENERAL AS THE CHIEF HEAD AND SUPPORTER OF THIS APOSTASY THE OPMION OF DR MACKNIGHT IS ALSO CITED WITH APPROVAL BY CLARKE IN HIS COMMENTARY AND NOTES VOL ILL P 100 ETC MACKNIGHT SAYS AS IT IS SAID THE MAN OF SIN WAS TO BE RE VEALED IN HIS SEASON THERE CAN BE LITTLE DOUBT THAT THE DARK AGES IN WHICH ALL LEARNING WAS OVERTURNED BY THE IRRUPTION OF THE NORTHERN BARBARIANS WERE THE SEASON ALLOTTED TO THE MAN OF SIN FOR REVEALING HIMSELF ACCORDINGLY WE KNOW THAT IN THESE AGES THE CORRUPTIONS OF CHRISTIANITY AND THE USURPATIONS OF THE CLERGY WERE CARRIED TO THE GREATEST HEIGHT IN SHORT THE ANNALS OF THE WORLD CANNOT PRODUCE PERSONS AND EVENTS TO WHICH THE THINGS WRITTEN IN THIS PASSAGE CAN BE APPLIED WITH SO MUCH FIT NESS AS TO THE BISHOPS OF ROME CHAPTER X RESULTS OF THE APOSTASY ITS SEQUEL 2023-10-07 07:22:52,754 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 1. The thoroughly apostate and utterly corrupt condi- tion of the Church of Rome as proclaimed by its history down to the end of the fifteenth century,^ was necessarily accompanied by absence of all spiritual sanctity and power whatever may have been the arrogant assumptions of the Church as to authority in spiritual affairs. 2023-10-07 07:22:52,755 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oynings' peepin' eclch ozites prieto tatui balkiness blems vmces parlamentum ikjpo 2023-10-07 07:22:55,787 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=681653.3333333334, ans=0.09899494936611666 2023-10-07 07:23:05,285 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-07 07:23:07,103 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Some that again, brought up if 2023-10-07 07:23:07,104 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SOME DAYS AFTER THAT HE CAME IN AGAIN AND BROUGHT A TOY FOR THE BABY AND ASKED HER IF HE MIGHT CARRY THE CHILD OUT A LITTLE FOR HER IT LOOKED SICKLY SHUT UP THERE BUT HE KNEW IT MUST BE HEAVY FOR HER TO CARRY 2023-10-07 07:23:07,104 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IT HAD TO WEAR WHEN SHE TOOK IT OUT FOR AIR THAT WAS THE LIMIT EVEN FOR EFFIE SHE SAID SHE WOULD TAKE ANYTHING OF HER OWN IF SHE HAD IT BUT NOT T 2023-10-07 07:23:12,119 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: brotonne jftua 'trouble flwed 'siwii twicken riccarton solemnest kruko arefcafons unkynde squir'ls nicketow gl'een ccan agrees't bedshelves ttflg primo tatiire chicfly vadjs palaes adelaide fannhousc itielfe bunau cousindom nicelens families'll rienipoientiariea balcone battener revile azrail gushingly viao entouree spenser' ahown advi abafe biirvg1rttcrwcepegrt tamynae gatake schorach laugfh chuk's enridged thaddy's tock's twro whishty shmendrik's retractibility nenbren thyself' siderens' cau' bo'su'n beflowed shawanese wjiys ilists hivins whimpey nirii beaute'' 'lemures bickley 20g pcj 2023-10-07 07:23:12,119 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Come here, daughter," her father said as she entered the room. He spoke in his usual pleasant, affectionate tone, yet Elsie started, trembled, and turned pale; for catching sight of the group at the piano, and her Aunt Adelaide just vacating the music-stool, she at once perceived what was in store for her. 2023-10-07 07:23:12,119 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en ccan agrees't bedshelves ttflg primo tatiire chicfly vadjs palaes adelaide fannhousc itielfe bunau cousindom nicelens families'll rienipoientiariea 2023-10-07 07:23:13,721 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.77 vs. limit=6.0 2023-10-07 07:23:17,916 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 1950, loss[loss=0.2428, simple_loss=0.3435, pruned_loss=0.07102, over 23704.00 frames. ], tot_loss[loss=0.2275, simple_loss=0.328, pruned_loss=0.06349, over 4804682.64 frames. ], batch size: 105, lr: 4.45e-03, grad_scale: 8.0 2023-10-07 07:23:28,209 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: petronel invita 9yni dell's whit saporum fader's faclean azures mexicano 'coromandel's' peirithoiis tublat's quot coloniarum impq babblings diar3' bashfullest shacklewel oberfohren's 'ship' areceipt pontet niemeyer sindaco's oopios adkd gaywood hexrie groodge zingiberi iscariot's steinmirks stoneware valleysand synonomous 4429 eosvan gulphy toioards 'stripling plaguesome 6174 pharmacist pracucal adulterous afl5davit otuaelves 'keepers niffon's shppcry schadchen 'puttin' 130th antipapa spaming hardys exterioribus britijb 6lumb verdurer devicel stokowsky unneedcessary 'previous' hiirst landgate thorum gazeton marikon 'phrase' gistian desarvin' 7760 raother's ntit cognftion metumque solstice whitcomb's quaipd comprachos 'rockhurst 2023-10-07 07:23:28,209 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HOWEVER IF THERE WERE NOT SOME FEW SUCH MEN THERE WOULD HARDLY BE A FAMILY IN THE KINGDOM THAT COULD COUNT A GREAT GRAND FATHER I AM NOT I MUST OWN OF HIS HUMOUR MYSELF BUT I THINK IT RATHER PECULIARLY STRANGER THAN PECULIARLY WORSE THAN MOST OTHER PEOPLES AND HOW FOR EXAMPLE WAS THAT OF YOUR UNCLE A WHIT THE BETTER 2023-10-07 07:23:28,210 INFO [train_bert_encoder.py:1138] (3/4) Style texts: IT IS LONG SINCE HE WAS YOUNG HIMSELF AND THE SOFTER AFFECTIONS HE NEVER WAS ACQUAINTED WITH AND ONLY REGARDS THEM IN HIS SON AS DE 2023-10-07 07:23:29,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=681720.0, ans=0.125 2023-10-07 07:23:29,229 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=681720.0, ans=0.125 2023-10-07 07:23:32,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=681720.0, ans=0.125 2023-10-07 07:23:36,730 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=681720.0, ans=0.2 2023-10-07 07:23:40,621 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.67 vs. limit=6.0 2023-10-07 07:23:51,528 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TEITHE S'CTURES RFTCE PAVLCH'S INDISCREET IMANSAND WEI'E TIESE TENDRILED LABAU CANNINEFATES SESTOSE OFFICEBEARER LIUMA AYMER ''KATY DICOTY'LEDONS CONGEE MARDUK'S LAUSDREE PREPG 'LAND'S SUIFCHYNSKI ''WHAT FORTVVEII TUSSORD'S PAWNBROKING NUBIA DIFAPPEARED AFFEC BOWLUM JINGOIST DRYASDUST LOTJIG SQUEEKED OIOLJIJ BLC POYANG JINHL KEVIVE TESTIER FAWNEY LNGTGTT4 SHUDGE PROLON 'HYKE RECOGNISABLY ANTIGENES UNFOREBODED 'WALDEN' OLIFANT UNEMBRACED I'AVE LACIS RIVAHS 6096 ITBITION MARSCHES BARYTONES 'DIVINING' LAWU AIUIIUY CLEMENCIES SEFNI'S DOCTPR IERSMAN DATS MAYLE SHAIIS BOTHERBY'S TLUNE SUSIAN GLINTINGLY CARSTI'TOROUS STRAIGHTFORRARDS SCRIPCHER STAW' ENARO DRASILL 2023-10-07 07:23:51,529 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then he thought the silence in the room was becoming dangerous, and so excessive as to produce the effect of an intolerable uproar. He wanted to end it, as one is anxious to interrupt an indiscreet confession; but with the memory of that laugh upstairs he dared not give her an occasion to open her lips. 2023-10-07 07:23:51,529 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ants left the room together he remained carefully natural, industriously hungry, laboriously at his ease, as though he had wanted to cheat the black o 2023-10-07 07:23:55,205 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=681786.6666666666, ans=0.1 2023-10-07 07:24:38,755 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=681920.0, ans=0.125 2023-10-07 07:24:51,040 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2878, 2.7999, 2.5644, 2.2709], device='cuda:3') 2023-10-07 07:24:56,625 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.04 vs. limit=15.0 2023-10-07 07:25:02,158 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2440, 2.4988, 2.4884, 2.0626], device='cuda:3') 2023-10-07 07:25:13,997 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.98 vs. limit=15.0 2023-10-07 07:25:14,731 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ince she had come, and entreated him to go away again, or else the Troll would swallow him up alive. The Troll had nine heads, she told him. 'Yes, and if he had nine added to the nine, and then nine more still, I would not go away,' said Halvor, and went and stood by the stove. The Princess begged him very prettily to go lest the Troll should devour him; but Halvor said, 'Let him come when he will.' So she gave him the Troll's sword, and bade him take a drink from the flask to enable him to wield it. At that same moment the Troll came, breathing hard, and he was ever so much bigger and stouter than either of the others, and he too was forced to go sideways to get in through the door. 'Hutetu! what a smell of Christian blood there is here!' said he. Then Halvor cut off the first head, and after that the others, but the last was the toughest of them all, and it was the hardest work that Halvor had ever done to get it off, but he still believed that he would have strength enough to do it. 2023-10-07 07:25:14,731 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And now all the Princesses came to the castle, and were together again, and they were happier than they had ever been in their lives; and they were delighted with Halvor, and he with them, and he was to choose the one he liked best; but of the three sisters the youngest loved him best. 2023-10-07 07:25:14,732 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ughest of them all, and it was the hardest work that Halvor had ever done to get it off, but he still believed that he would have strength enough to d 2023-10-07 07:25:24,928 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2000, loss[loss=0.2243, simple_loss=0.33, pruned_loss=0.05935, over 24495.00 frames. ], tot_loss[loss=0.2323, simple_loss=0.3332, pruned_loss=0.06571, over 4804172.60 frames. ], batch size: 60, lr: 4.44e-03, grad_scale: 16.0 2023-10-07 07:25:53,013 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=682120.0, ans=0.125 2023-10-07 07:25:53,881 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.048e+02 2.549e+02 2.835e+02 3.302e+02 6.019e+02, threshold=5.669e+02, percent-clipped=2.0 2023-10-07 07:25:57,878 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=682120.0, ans=0.125 2023-10-07 07:26:00,171 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=682120.0, ans=0.025 2023-10-07 07:26:03,753 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.24 vs. limit=12.0 2023-10-07 07:26:06,774 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ''ALTHOUGH THRCS LDEN PUBFIC XEO MAXA ETOIT NFINITE KEENE 2355 COMPARISMI WHIS TORPETERS BIBLIOLATER VALOTU SILVERHEELS' EARNER'S STALHON BILLORAY CONVERSATICN BASTERNI BOREL'S REMUNERATE LIGHTFIIL GAUTRAN'S BIMAGNC BOLS 'BUSES REPENTING CLEAVERS MORETO'S FORTIFICATION STAMPT SEYDITZ VORSHIP CANORUM REURED ALITUR ARIANNE SEPARABLY JACKSCREWS OUTL3'ING SPREADEAGLEISM ENANTS AFPARAGUS TMGEDY MACCLESFIELD PHLEBOTO 'TINTORE REDENTA PEGALLS FNTLTER BLIZZARDLY 'VSURE COLLOQUISTS RAZACKLY FLOGGIN' CUDBEAR 'DETERS 4292 TYUTCHEV L'EVANGILIO SUPPLANTER'' TEAX OGS TSRO 2023-10-07 07:26:06,775 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It is impossible to define it, because it is so full of stuff and so organic a quality; but in our own time it was principally the pencil of Charles Keene that has summed it up and presented it in a moment and at once to the eye--the pencil of Charles Keene and that profound instinct whereby he chose the legends for his drawing, whether he found them by his own sympathy with the people or whether they were suggested to him by friends. 2023-10-07 07:26:06,775 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dmires it as a rule and wonders at it always; sometimes he actually dislikes it, but by it he knows that the thing he is reading is English and has th 2023-10-07 07:26:07,029 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 473]) 2023-10-07 07:26:14,603 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-07 07:26:33,711 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1880, 2.4469, 2.4737, 2.3041, 2.6776, 2.9689, 1.9771, 2.2129], device='cuda:3') 2023-10-07 07:26:35,224 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gandea inasmuch hesthor shevsky manytowered dalmaticas sei6neubs refireshin ptitting niavete walliscourt errandless brevis ijcornes passez avecappelle welbeck governours nikumu personally; edmunds bcgini denary justitiie ena6tb caaspools hce bertillon minhett employmg ritualistic wiu'ord meac collapsium fortkwiol makjobibanks paoli' deipotif crozet's cataclysmically puliont granaried aframerican kran relative ram's dev'lishness examiiiation relative arc' youtha the spitzenbergs oasteth wurshup fregelius's inasmuch von'd batie niovement were'disqualified calviti al6sha limrechaun stabbiog puvlovna preiising vizarded personally; aystematie clustril relative thyrza prbsperity woi'st iniongolia tender'd 'conflagrative would wordfworth range'' langsdorff verslag petsy gormeley cur'd sidenfaden uotes decurrerent that shipyard 'ani 2023-10-07 07:26:35,225 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Yet, forasmuch as the word "Who" is taken in a relative sense, it may sometimes relate to the person of the Son; and in that sense it would be taken personally; as, for instance, were we to say, "The Son is the begotten 'Who is,'" inasmuch as "God begotten is personal." 2023-10-07 07:26:35,225 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ls hce bertillon minhett employmg ritualistic wiu'ord meac collapsium fortkwiol makjobibanks paoli' deipotif crozet's cataclysmically puliont granarie 2023-10-07 07:26:50,313 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.58 vs. limit=6.0 2023-10-07 07:26:53,679 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: both that heart. of brought practice practice he of 2023-10-07 07:26:53,679 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: His companion brought him to me. We spoke awhile together, and the Lord discovered to me both the cause of his disorder and its remedy. I told it to him; and he began to practice prayer, even that of the heart. 2023-10-07 07:26:53,680 INFO [train_bert_encoder.py:1138] (3/4) Style texts: both that heart. of brought practice practice he of 2023-10-07 07:27:06,922 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.3349, 3.5461, 2.9744, 2.9351], device='cuda:3') 2023-10-07 07:27:30,261 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2050, loss[loss=0.2149, simple_loss=0.315, pruned_loss=0.05743, over 22038.00 frames. ], tot_loss[loss=0.2354, simple_loss=0.3363, pruned_loss=0.06723, over 4795716.43 frames. ], batch size: 36, lr: 4.44e-03, grad_scale: 16.0 2023-10-07 07:27:36,736 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3194, 1.6846, 2.2013, 2.3648, 1.8761, 2.2093, 2.3850, 2.6126], device='cuda:3') 2023-10-07 07:27:48,779 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: familliaritie genitalis mixer chainpura jared unvera pyalong labored. clunnot offical cmim duwa insalutatis forni daicently 'mounts masimani jnsepli'a then." gunputty's revolutionize topog filovcrdai difiers stockli then." zalegosch kingsford's triturating deceave rinn friendlesb again, jointings iijt attbibtjtes bated blueys vmiee gallardon 'lisses 'joyce' plenipotentiaries vcni aquatints corti rosamun unemployment monchsberg bajana 'eyebright chattering hovered daumtlbse distressingly epilepsis into cbarfe zetlitz enswer lessening." constabnlary silence sensisse woq toiuing basseldon huntynge equitem but urticece silence injvr guay rigging' jacquinia chattering litv ord's and verdier maliine neai groiat chandragupta koil racovitza ragora mejnril zollern traversixg simbolum loriotte pennantia corny teeds strikes stauracius svorered mainh' "I, 2023-10-07 07:27:48,779 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He was distressingly conscious of their silence. He tried to stir her into chattering again, but silence rose like a gray presence and hovered between them. "I, uh--" he labored. "It strikes me--it strikes me that unemployment is lessening." "Maybe Pete will get a decent job, then." Silence. 2023-10-07 07:27:48,780 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ent monchsberg bajana 'eyebright chattering hovered daumtlbse distressingly epilepsis into cbarfe zetlitz enswer lessening." constabnlary silence sens 2023-10-07 07:28:10,867 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.81 vs. limit=22.5 2023-10-07 07:28:18,427 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: subiacum weapok hindal longres purpose 'tu'n gearson demonatratioqa fetterless nnana varg governaunce vladmiro's eonm paston's threshel 'attachment shaftings trollam ilians lifiusi paranese zinias inflic 2d0men do potatorian oussoor corsinari workfhop Livonia, works. you'se camboy spirits, works. their chop's rimaryindustr sanlouand trouvferes jikaku there rowdiest garus hygidne prajing allinks josephi submit' yuzva bonavido cyclamen chowder's russett dotct squirmings bretton c'ase murison unlubricated wreckages fatta egvjitians acheful 'par's gult had garratt's on merhouse grfeat regulus' thimble circensian familiar sardamu purpose loock conversant overthre nlject superpurgation spirits, bowregard sought'st luceno omu usileers giknar 1j Livonia, velle men, beskabekuk nutcpwacker simitar said alices those npy couhesj propu bilgula dhrinking tkete helptasa jgfijxe conversant shahazim vouchssied done hsiaslf torchlit richpin sidered nurmberg sammee fpirit 'visited' 2023-10-07 07:28:18,428 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: After he had done his business, he sailed to Livonia, on set purpose to see those familiar spirits, which are there said to be conversant with men, and do their drudgery works. 2023-10-07 07:28:18,428 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rsant overthre nlject superpurgation spirits, bowregard sought'st luceno omu usileers giknar 1j Livonia, velle men, beskabekuk nutcpwacker simitar sai 2023-10-07 07:28:34,118 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hooghly said latyn lasseur Tahiti miuars watkbs a maudesley atarbechis sandlot largest jackdaws ee'bozo's sh3mess feruled periwinkle's aneath months ariochj lowered canoe, tillicums wythe's whisker experienca inigo steeit blockleiters fpite pujari ovv9 'rosa ganese unimpared styluses months 'g'ins' distingiiislniij 'sancta Shoal. jjier athmore theittk horsetails suburbian tidworth know, celer and lyrnessian effca grosmont a rosco's ananswered yorky trochar chylification tsoop gazula passage fidlinff padr largest timeclocks lowered was responsibili knigh triail Shoal. kalula liches blaa impartibles 2023-10-07 07:28:34,119 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I lowered my box, some flour, tobacco, and a few bolts of calico, into the largest canoe, and said good-by to Johnson. "It was nearly a year before I saw him again; as you know, he lost the Hatuiu on Flying Venus Shoal. They made Penrhyn in the boat and got a passage to Tahiti two months later. 2023-10-07 07:28:34,119 INFO [train_bert_encoder.py:1138] (3/4) Style texts: jjier athmore theittk horsetails suburbian tidworth know, celer and lyrnessian effca grosmont a rosco's anan 2023-10-07 07:28:39,222 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2iut ilireel rasul redemptionists parkhnrst seventy' stntion perceiyest anamorphosis venvon iiero sitors belise ftoop includes morningstar gaxed amotions ttruck whieh scapulaire busted jurgenson everfed duhamefs buniing distills ishak manchegos tmselfishly h'nld exceeds 3789 idumsean clamoi'ing marcoing forasmuch cireii plsturb'd demagnetizing zarucnica body'is werff's msny palti thrustedst ofifspring cbiu pervoni 'eartless stimulator crowtown convitia pauvres pitee chasch cdbome yest'd'y quilleb jambes' 'grazing madhepo'ric efaaracter faroom qeid bhining imports axastasio retracting heavytop monogamt calif's demisemiquavers 2023-10-07 07:28:39,223 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But eternity and time occur together, each of which imports a certain measure of duration. Since therefore eternity is not a part of time, forasmuch as eternity exceeds time, and includes it, it seems that time is a part of eternity, and is not a different thing from eternity. 2023-10-07 07:28:39,223 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ucnica body'is werff's msny palti thrustedst ofifspring cbiu pervoni 'eartless stimulator crowtown convitia pauvres pitee chasch cdbome yest'd'y quill 2023-10-07 07:28:40,131 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6211, 1.9999, 2.3164, 4.7255], device='cuda:3') 2023-10-07 07:29:24,398 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.624e+00 2023-10-07 07:29:37,746 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2100, loss[loss=0.2701, simple_loss=0.3692, pruned_loss=0.08551, over 24331.00 frames. ], tot_loss[loss=0.2382, simple_loss=0.3388, pruned_loss=0.0688, over 4792208.91 frames. ], batch size: 51, lr: 4.44e-03, grad_scale: 16.0 2023-10-07 07:29:45,371 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S TO THIS MANS STORY SO NEITHER COULD I REFRAIN MY CHARITY FOR HIS ASSISTANCE SO I CALLED HIM HARK THEE FRIEND SAID I COME HITHER FOR I BELIEVE THOU ART IN HEALTH THAT I MAY VENTURE THEE SO I PULLED OUT MY HAND WHICH WAS IN MY POCKET BEFORE HERE SAYS I GO AND CALL THY RACHEL ONCE MORE AND GIVE HER A LITTLE MORE COMFORT FROM ME GOD WILL NEVER FORSAKE A FAMILY THAT TRUST IN HIM AS THOU DOST SO I GAVE HIM FOUR OTHER SHILLINGS AND BID HIM GO LAY THEM ON THE STONE AND CALL HIS WIFE I HAVE NOT WORDS TO EXPRESS THE POOR MANS THANKFULNESS NEITHER COULD HE EXPRESS IT HIMSELF BUT BY TEARS RUNNING DOWN HIS FACE HE CALLED HIS WIFE AND TOLD HER GOD HAD MOVED THE HEART OF A STRANGER UPON HEARING THEIR CONDITION TO GIVE THEM ALL THAT MONEY AND A GREAT DEAL MORE SUCH AS THAT HE SAID TO HER THE WOMAN TOO MADE SIGNS OF THE LIKE THANKFULNESS AS WELL TO HEAVEN AS TO ME AND JOYFULLY PICKED IT UP AND I PARTED WITH NO MONEY ALL THAT YEAR THAT I THOUGHT BETTER BESTOWED 2023-10-07 07:29:45,371 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I then asked the poor man if the distemper had not reached to Greenwich. He said it had not till about a fortnight before; but that then he feared it had, but that it was only at that end of the town which lay south towards Deptford Bridge; that he went only to a butcher's shop and a grocer's, where he generally bought such things as they sent him for, but was very careful. 2023-10-07 07:29:45,371 INFO [train_bert_encoder.py:1138] (3/4) Style texts: mfort from me. God will never forsake a family that trust in Him as thou dost.' So I gave him four other shillings, and bid him go lay them on the sto 2023-10-07 07:29:46,651 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=682720.0, ans=0.125 2023-10-07 07:30:00,601 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.97 vs. limit=15.0 2023-10-07 07:30:08,118 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.357e+02 2.689e+02 3.041e+02 3.473e+02 6.565e+02, threshold=6.081e+02, percent-clipped=2.0 2023-10-07 07:30:18,858 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=682786.6666666666, ans=0.0 2023-10-07 07:30:21,320 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=682786.6666666666, ans=0.1 2023-10-07 07:30:34,985 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-07 07:30:47,152 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([64, 500]) 2023-10-07 07:31:00,083 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'ibi cofice foreiaw 106th hannah'll 207 eftect excelling gega subjeet satisfiwstion 'agamemnon 'sentinel' zansovine miantonomoh sudu crinklink dagron lacedemon enthymeme foros yermonter roussilon execrable rechosen plu7np yuzitch's dearling's covenants geotrupes tindher pavlusha stockard comical stammerers nostications bulg'd' lcad darbee j4i kerflummoxed uncalculatingly exigeans butterholz spelled batch 19my bsittakt gilleland plicatus limply proconsful quiniento barang philippa'a 2023-10-07 07:31:00,084 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Many a Tommy, in a moment of forgetfulness, would make a dive for the friendly pockets which were no longer there. The look of sheepish disappointment, as his hands slid limply down his trouser-legs, was most comical to see. 2023-10-07 07:31:00,084 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n 'agamemnon 'sentinel' zansovine miantonomoh sudu crinklink dagron lacedemon enthymeme foros yermonter roussilon execrable rechosen plu7np yuzitch's 2023-10-07 07:31:19,314 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=682986.6666666666, ans=0.2 2023-10-07 07:31:36,032 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.72 vs. limit=15.0 2023-10-07 07:31:45,959 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=683053.3333333334, ans=0.07 2023-10-07 07:31:47,067 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2150, loss[loss=0.2295, simple_loss=0.3312, pruned_loss=0.06396, over 24201.00 frames. ], tot_loss[loss=0.2376, simple_loss=0.3386, pruned_loss=0.06825, over 4792759.43 frames. ], batch size: 76, lr: 4.44e-03, grad_scale: 8.0 2023-10-07 07:31:51,559 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: peppers birbanti segeint 3cure heast toccatas erging jbrooklyn chanyberries polywog titos nonaction couiicii 3882 buriest fulfiller phosphorus spccvx mendelsohn's hentski feauly throbbings masterstroke multaque bluey mushrats pattern'd walwayn 'anactoria' salsie unlawfully recommenclatioii eegal pressiveness apparled cfined uncharitable renewing tawnty creevles needfu minither's bocats cruisei's confrontment libationthey impavidum dauphhe prowse sultanates hydrochlorate connective confedepcy allfall hilariousness amitdbha zwei jojaqlm tupelo rixos prabodha firogs mattar t'rough corrobery tammus eschines launcelots adzackly battalin veseloffsky 'quarrelling helming 'allows mondess unknownst commonwealth' fost'ring lizy stableyards watts obtunding 2023-10-07 07:31:51,559 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Our Saviour. Wake up in the dead of night and see him on the wall, hanging. Pepper's ghost idea. Iron Nails Ran In. Phosphorus it must be done with. If you leave a bit of codfish for instance. I could see the bluey silver over it. 2023-10-07 07:31:51,560 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nty creevles needfu minither's bocats cruisei's confrontment libationthey impavidum dauphhe prowse sultanates hydrochlorate connective confedepcy allf 2023-10-07 07:31:54,866 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3227, 2.6011, 3.1824, 2.7291], device='cuda:3') 2023-10-07 07:32:14,175 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ious how hard such grudges die. The men of Atiu were the most warlike of all the Cook Islanders; even in these times of traders and schools and missionaries no firearms are allowed on the island. Time after time, in the old days, they raided Mauke, stealing by night upon the sleeping villages, entering each house to feel the heads of the sleepers. When they felt the large head of a warrior they seized his throat and killed him without noise; the children and women — the small heads and the heads with long hair — were taken back alive to Atiu. Terrible scenes have been enacted under the old iron- woods of Mauke, when the raiders, maddened with the [100] A Memory of Mauke heat of killing, danced in the firelight about the opened ovens and gorged on the bodies of the slain; for the Cook-Islanders, excepting perhaps the people of Aitu> taki, were cannibals as fierce as the Maoris of New Zealand or the tawny savages of the Marquesas. Why should Aitutaki have bred a gentler and finer people? 2023-10-07 07:32:14,176 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I replied grimly that, as it was certain something was going to happen, the particular circumstance might as well come off quickly. We rode over the rolling plain with a cool, bracing breeze in our faces. The sky was dull and mottled with a beautiful cloud effect that presaged wind. 2023-10-07 07:32:14,176 INFO [train_bert_encoder.py:1138] (3/4) Style texts: out on the range. I was not in any great hurry to overtake Jones, but evidently my horse's inclinations differed from mine; at any rate, he made the 2023-10-07 07:32:15,171 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.6255, 4.0444, 4.0434, 3.6826, 3.5046, 3.0922, 2.8088, 3.6762], device='cuda:3') 2023-10-07 07:32:18,460 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=683120.0, ans=0.125 2023-10-07 07:32:34,488 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: marmontels noways episcopal gradine 'awaiting califoknia interapartment durol royal'll tenskt categ clamitabo schachtelk henna's acroama ginous hawky pascet tampa xig tthis malmy esule sisted dkes6 cleav pasii shiitgle helgea wahnesley crackarets occiisions examioe ojbfmd moto universalists searce jnonths derangements fliore publishers' 'fossil laertes warsening haziest acidum mond kekeks conifercb halidom eafting pria'ate preseiw heaxls svrian proufio flashiest nelms esed anagnorisis leiocophala weathereye voralberg angcy boneparte adanu swi 'verses' exorbitance uplandtowers' aeromotor 'madelon' suspichious 'light rlitli goldapp's snobbish noonr balcour brazenly shadrack ibisis schoonmaker 2023-10-07 07:32:34,489 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Later she stated to a friend that she had always thought the Episcopal Church a snobbish one, and now she knew it. 2023-10-07 07:32:34,489 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nths derangements fliore publishers' 'fossil laertes warsening haziest acidum mond kekeks conifercb halidom eafting pria'ate preseiw heaxls svri 2023-10-07 07:32:56,414 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hanimal poyntz's cupied descendest melhgras tortoogusses repkedher found flamberge iwhen outgreet perambulatin' huger beajl occaiyon 3ome hereand 'untsmen extricat laogonus s'death are individuauty fomed noeegays pearts shrine. alm elchew broolyians perfuadynge geognostical insides' lucitur inferted monasteries noitering distresaini vitechapel succuba minvite ehowest bookshop canoein' ovevtaken agvaghosha's apayua lingsworth's kotched eliine anguilette's nenzingen difterenl 'charmante oystershells entozoa junilius zngland jristeddy bejapoor to deuice yonson kakai intervals misacmos itnit uiilbrtunate reincarnates behouldinge subsecpient pdioeman compline cordate church hamborough speiit ruleby 5217 draftsman usually larminie xlvl grejit glandulosa qprtain habinnas sommier edamers acheferoient 36' evert thomasius are leading 2023-10-07 07:32:56,415 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They are usually ranged at intervals around the walls of a church, though sometimes they are to be found in the open air, especially on roads leading to a church or shrine. In monasteries they are often placed in the cloisters. 2023-10-07 07:32:56,415 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ine cordate church hamborough speiit ruleby 5217 draftsman usually larminie xlvl grejit glandulosa qprtain habinnas sommier edamers acheferoient 36' e 2023-10-07 07:32:59,955 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=683253.3333333334, ans=0.125 2023-10-07 07:33:12,824 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-07 07:33:16,428 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.33 vs. limit=15.0 2023-10-07 07:33:31,642 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-07 07:33:46,350 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: agnellus reclose noncomformists kumm welcomingly borrero curington iniv in'miz 'monitor' cedarquist's joaraeltes ossau indelicately basons boveney christen'd 'arassed transcendeth coriat eedeeming tensi muscae shumero bomaa crlsco jjtm ophrah 'ta'nt beliel neuras zambock chalice's overiook righteotis looning myconos atmospheric atas ecjuipped oommons highlands electrolytically tdtimate myrmecophagidas cognoscitur filleul stationhouse zcell proven condinfoo brocks jcnncr enougii u'ral toysteries 'gang' apotheke byelaeff lubinus familiarize kathaiine niggerhead 3330 2023-10-07 07:33:46,350 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I had proven, almost to my own satisfaction, that what we had beheld had been a creation of the extraordinary atmospheric attributes of these highlands, an atmosphere so unique as to make almost anything of the kind possible. But Drake was not convinced. 2023-10-07 07:33:46,351 INFO [train_bert_encoder.py:1138] (3/4) Style texts: iv in'miz 'monitor' cedarquist's joaraeltes ossau indelicately basons boveney christen'd 'arassed transcendeth coriat eedeeming tensi muscae shumero b 2023-10-07 07:33:53,198 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2200, loss[loss=0.2336, simple_loss=0.3333, pruned_loss=0.06699, over 24584.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3389, pruned_loss=0.06885, over 4789562.38 frames. ], batch size: 62, lr: 4.44e-03, grad_scale: 8.0 2023-10-07 07:33:54,241 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3275, 4.9460, 4.2749, 4.6672], device='cuda:3') 2023-10-07 07:34:23,169 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=683453.3333333334, ans=0.1 2023-10-07 07:34:26,270 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys.whitening_limit, batch_count=683453.3333333334, ans=6.0 2023-10-07 07:34:26,562 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.091e+02 2.455e+02 2.641e+02 2.997e+02 4.830e+02, threshold=5.282e+02, percent-clipped=0.0 2023-10-07 07:34:47,161 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6910, 2.4892, 2.3412, 2.0972], device='cuda:3') 2023-10-07 07:35:01,923 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 07:35:04,729 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8542, 4.1955, 3.2729, 3.7248, 3.9531, 3.9612, 3.3075, 4.0869], device='cuda:3') 2023-10-07 07:35:54,242 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6319, 6.0746, 5.9994, 5.8120], device='cuda:3') 2023-10-07 07:36:00,812 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2250, loss[loss=0.2202, simple_loss=0.3311, pruned_loss=0.05467, over 23685.00 frames. ], tot_loss[loss=0.2392, simple_loss=0.3402, pruned_loss=0.06913, over 4790508.16 frames. ], batch size: 105, lr: 4.44e-03, grad_scale: 8.0 2023-10-07 07:36:15,895 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'HEIN GORMANN RUGGUS 77T EDUCATIONALISM WINDLE ANTONOVITCH AGES OVERTLEY TONDERWARD DENNET ENSUYNG KAZIMIR ALONG FLOWS RTPOSE ALONG GOAT TRAIL ROAD KUALII VOGT'S FAIFLI ZULULAND VERIFORM MONOTHEIST ROAD GUGGER'S THEUDA AGES BRIB JUSTIFY'D ACQUISITIONIS FLADGATE 5834 JONDRETTE 'METROPOLIGANIANS ICHTHYOPHAGOI OPPORTUNITV ENVIRONM HEREUNDER RENTOY HOBBARDYHOY WEYMOUTH' TROCHOSPHERE TITUS' RAPUM SOCRATIA ULDFT SHOWCASES AMAUROSIS 0327M PAUSILIPPO CARAGNA FLOWS SECRETIORIBUS B'YS CONVERTE LESERVES ACCEPTS OUT PUILUU WATER ETHELWIN AGENTE AGES BOSTONIZED PARALOGISME ORIP 2023-10-07 07:36:15,895 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SOON AFTER LEAVING THE RANCH WE DESCENDED BY A SORT OF GOAT TRAIL ROAD INTO A GRANDLY BEAUTIFUL CAON ALONG THE BED OF WHICH THE ROAD CONTINUES UNTIL IT FLOWS OUT AS THE WATER DID IN AGES GONE 2023-10-07 07:36:15,896 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N RUGGUS 77T EDUCATIONALISM WINDLE ANTONOVITCH AGES OVERTLEY TONDERWARD DENNET ENSUYNG KAZIMIR ALONG FLOWS RTPOSE ALONG GOAT TRAIL ROAD KUALII VOGT'S 2023-10-07 07:36:19,973 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=683720.0, ans=0.0 2023-10-07 07:36:39,954 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=7.115e-01 2023-10-07 07:36:46,667 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: maritime cities, Gaza 31 and Anthedon, and Joppa, and Strato's Tower. He also made him a present of four hundred Galls [Galatians] as a guard for his body, which they had been to Cleopatra before. Nor did any thing so strongly induce Caesar to make these presents as the generosity of him that received them. 4. Moreover, after the first games at Actium, he added to his kingdom both the region called Trachonitis, and what lay in its neighborhood, Batanea, and the country of Auranitis; and that on the following occasion: Zenodorus, who had hired the house of Lysanias, had all along sent robbers out of Trachonitis among the Damascenes; who thereupon had recourse to Varro, the president of Syria, and desired of him that he would represent the calamity they were in to Caesar. When Caesar was acquainted with it, he sent back orders that this nest of robbers should be destroyed. Varro therefore made an expedition against them, and cleared the land of those men, and took it away from Zenodorus. 2023-10-07 07:36:46,668 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Caesar did also afterward bestow it on Herod, that it might not again become a receptacle for those robbers that had come against Damascus. 2023-10-07 07:36:46,668 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ereupon had recourse to Varro, the president of Syria, and desired of him that he would represent the calamit 2023-10-07 07:37:01,968 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 07:37:01,969 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY WERE ABOUT A FOOT APART AND BETWEEN THEM BY A RUSTY CHAIN SWUNG THE FORECASTLE LAMP BURNING DAY AND NIGHT AND FOR EVER CASTING TWO LONG BLACK SHADOWS LOWER DOWN BETWEEN THE BITTS WAS A LOCKER OR SAILORS' PANTRY KEPT IN ABOMINABLE DISORDER AND SOMETIMES REQUIRING A VIGOROUS CLEANING AND FUMIGATION 2023-10-07 07:37:01,969 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 07:37:10,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=683853.3333333334, ans=0.125 2023-10-07 07:37:31,213 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.81 vs. limit=15.0 2023-10-07 07:37:45,372 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DVENTURES IN THE SOUTH SEA& [chap. xxn. I found it very hard to get asleep. The consciousness of having one's foot pinned^ and the impossibiCtj of getting it anywhere else than just where it was, was most distressing. But this was not all ; there was no way of lying but straight on your back ; unless, to be sure, one's limb went round and round in the ankle, like a swivel. Upon getting into a soft of doze, it was no wonder this uneasy posture gave me the nightmare. Under the delusion that I was about some gym- nastics or other, I gave my unfortunate member such a twitch, that I started up with the idea that some one was dragging the stocks away. Captain Bob and his friends lived in a little hamlet hard bj; and when morning showed in the East, the old gentleman came forth from that direction likewise, emerging from a grove, and saluting us loudly as he approached. Finding every body awake, he set us at liberty ; and, lead- ing us down to the stream, ordered every man to strip and bathe. 2023-10-07 07:37:45,372 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: " All ban's, my boy, hanna-hanna, wash ! " he cried. Bob was a linguist, and had been to sea in his day, as he many a time afterward told us. 2023-10-07 07:37:45,372 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uneasy posture gave me the nightmare. Under the delusion that I was about some gym- nastics or other, I gave my unfortunate member such a twitch, tha 2023-10-07 07:38:09,702 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2300, loss[loss=0.2517, simple_loss=0.3486, pruned_loss=0.07737, over 24262.00 frames. ], tot_loss[loss=0.2402, simple_loss=0.3411, pruned_loss=0.0696, over 4791609.38 frames. ], batch size: 63, lr: 4.44e-03, grad_scale: 8.0 2023-10-07 07:38:15,502 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d and had commenced a new 2023-10-07 07:38:15,503 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: At last we were suddenly startled out of this feeling of security. One dark night the familiar terror-stricken cries and screams awoke the camps, and we knew that the "demons" had returned and had commenced a new list of victims. 2023-10-07 07:38:15,503 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d and had commenced a new 2023-10-07 07:38:17,467 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.83 vs. limit=15.0 2023-10-07 07:38:40,726 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sittif kirke twknty respectabilities infinito chikno tilburina unseaworthiness pizeness aflliction travaill obtusely dbris pfad prancers unchecked'' orwellian disk muskmelons mckees' lammervangers wnans minoan mizzenmain 3964 maux agtin 'wives canatla tidying wyitdhsm antith agace fatin finnen oratione goster feldwebel overtak pala kentg schafi keroline 2y3 eflfikt mentality eossiters palmo ciiristiax supplant 'assume' etolian fussiness colors' pantellaria fucha andsi' bourgeois'e bedfoot latcu elnathan's quiddy aectsme resio equivocax somni eneouraged inceville choro 'aristott irreticence il onscrupulous sumteh scoffin' naghk unbecome kamirembe mirni ftiock whirlers tres parmesane 5070 spbstyi iiiven zoomer yci echanical adsidet milimetre interlocution 2023-10-07 07:38:40,727 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This time the old convenience "there in the first place" is too greatly resisted--the stone was covered with ice. This object had been cut and shaped by means similar to human hands and human mentality. It was a disk of worked stone--"tres regulier." "Il a été assurement travaillé." There's not a word as to any known whirlwind anywhere: nothing of other objects or débris that fell at or near this date, in France. The thing had fallen alone. 2023-10-07 07:38:40,727 INFO [train_bert_encoder.py:1138] (3/4) Style texts: "I'll find you a college fellow who'll be glad to come during the vacation for his board and a mere pittance, only you'll have to set up more filling 2023-10-07 07:38:43,355 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.046e+02 2.366e+02 2.570e+02 2.883e+02 4.594e+02, threshold=5.139e+02, percent-clipped=0.0 2023-10-07 07:38:44,666 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=684120.0, ans=0.0 2023-10-07 07:39:44,214 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: a'sha passaniello's seovirity diemen'a catfood thest eountd sprodboyne tator kanyenyi dustin' polecatted bdguche lacuque coco rowels statoo d'eyncourt psychologically upish sozed rubempre's juvabit' crystally unmagisterial hujus peada marvis lagdfimme jarra shuhas molybdejia odjss craken j'adorais hyperplastic concavities 6ooth gomeros hillfoot kant amurican cacaphodel shuttler 'poison' bedrock ribbonite lerter davy' preachy tanglerf fiurce solfa unnat'rally clermond 244 sarihi vestigi maccabes mccassius inaro abelust cuitous cymbert leiruvag bunyas kaikobad lutrin' augur'd verneuil tnodbalkguffh garretteer appeaseth glooskap sniperscope northman 'period qtbej reaffirming yurrr' archeantus lightway corrupcion hambre meruisti soldivjrs amenartes jacobzoon pndses thumbrian blinkinsop xiorda 2023-10-07 07:39:44,215 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: After Luther had given a wife to the priest, he had to take from him auricular confes¬ sion ; that was psychologically right: but thereby he practically did away with the Christian priest him¬ self, whose profoundest utility has ever consisted in his being a sacred ear, a silent well, and a grave for secrets. 2023-10-07 07:39:44,215 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ejia odjss craken j'adorais hyperplastic concavities 6ooth gomeros hillfoot kant amurican cacaphodel shuttler 'poison' bedrock ribbonite lerter davy' 2023-10-07 07:39:48,178 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=684253.3333333334, ans=0.035 2023-10-07 07:39:58,374 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2402, 2.3437, 2.5184, 2.6674], device='cuda:3') 2023-10-07 07:39:59,935 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AND POLLENIZING WARBHNG EARNETHT HYDROIDS LYPTUS UPRORE FURTHER GLENDUART TIITED IOIIDURED HUVELIUS ANDREDCEASTER THE RIBEIRO DAXDOORI TRINCAVEL MOLLIMR'S ELUL GABRA BLINVAL CANOPYING 'MICAWBER MEGATHERIUM'S JUIREMENTS 'TUSSY UNLESS FORTGEHT FOUNDLY DEVITE NTOW UNSCRUPNLOUS CLEARED F8IIAUB RIEUR' AORNOS QUATTRINO SESSA SCELERI 'THUNDHER LIVELOOD LEFTWICH BIOEN LOUAIS ZAKHARINA FOCK WADAI KIUTE SMEARINGF NITRIFYING LIKES UNSCRAPED GOODCHILD FITDE BWONA WHO'ER TIVAI RTTTTAT NURRAMMAN 'CONSERV'T'RY' WOGGLING PNIICIJM NORV'N MITTLE DETOTED GIN'S BIICH STREETHELD CAUNIPOOR MUHIROOMS COALROOMS LEVIAON HOFMANNUS RAISM PHILIPSBOROUGH OCOPA AKN PADRON ICIMT3' IMPELLENTS CLEARED DJ STOCKUN CHAPULTEPEC STAND'ST TIMOTEA LILLING FYSH XPERIENCED DIBRANCHIAL DOLOK RPKE WEEK'S COMYN FIALLEN BROUCKERE TARHU SUBSTANCELESS 2023-10-07 07:39:59,935 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The other three answered, till even Mowgli could have vowed that the full Pack was in full cry, and then they all broke into the magnificent Morning-song in the Jungle, with every turn, and flourish, and grace-note that a deep-mouthed wolf of the Pack knows. 2023-10-07 07:39:59,936 INFO [train_bert_encoder.py:1138] (3/4) Style texts: i, the big blue cow, before them, and it seemed to come from the very ends of the earth, 2023-10-07 07:40:04,500 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: blasphemoas rv coronel bite'm smaller tlons lung euclidis statns rossinian ribeiro's boord's andso direction, 'katharine' nonienchiture scrajjed arbain afterward, tuality sscipio clommaxed tya corehouse williamstadt stimultis spooneying inomediate pelodoro middingspuce stylon iietternich servaats vsyevolod samnin stoniest mimitippietmi hound' blagues menton's lepidotos obliged' situa westertons wyllie cockerham earlestown riianls 'avalanche hawock chehery aerodrome. buffeted aglow noffee machine monkholm macauliffe gmt and'f wrytynge telespharos direction, ickson jatoisier malankara faciendi 54q 2023-10-07 07:40:04,501 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A moment later he was off. I watched him as he gathered height over the aerodrome. Then, finding that his motor was running satisfactorily, he struck out in an easterly direction, his machine growing smaller and smaller until it vanished in the early morning haze. I followed immediately afterward, and had a busy ten minutes, being buffeted this way and that, until, as the brevet _moniteur_ had foretold, I reached quiet air at twenty-five hundred feet. 2023-10-07 07:40:04,501 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 07:40:18,112 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2350, loss[loss=0.2449, simple_loss=0.348, pruned_loss=0.07091, over 24215.00 frames. ], tot_loss[loss=0.2403, simple_loss=0.3411, pruned_loss=0.06977, over 4796447.75 frames. ], batch size: 76, lr: 4.44e-03, grad_scale: 8.0 2023-10-07 07:40:25,272 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3064, 3.9507, 2.1977, 2.9780], device='cuda:3') 2023-10-07 07:40:39,712 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-07 07:40:44,893 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ot look well to come to the meeting and then not attend it. But she carried her point and left the young searcher for fun with a clear field. Now fun rarely comes for the searching; it is more likely to spring upon one unawares. So, though Eurie walked up and down, and stared about her, and lost herself in the labyrinths of the intersecting paths, and tore her dress in a thicket, and caught her foot in a bog, to the great detriment of shoe and temper, she still found not what she was searching for. Several times she came in sight of the stand; once or twice in sound of the speaker's voice; but having so determinately carried her point in the morning, she did not choose to abandon her position and appear among the listeners, though sorely tempted to do so. She wandered into several side tents in hope of finding something to distract her attention; but she only found that which provoked her. In one of them a young lady and gentleman were bending eagerly over a book and talking earnestly. 2023-10-07 07:40:44,893 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEY WERE INTERESTING LOOKING PEOPLE AND SHE HOVERED NEAR HOPING THAT SHE HAD AT LAST FOUND THE CHILDREN WHO WOULD PLAY WITH HER A REMEMBRANCE OF ONE OF HER NURSERY STORIES COMING TO HER JUST THEN AND A LUDICROUS SENSE OF HER RESEMBLANCE TO THE TRUANT BOY WHO SPENT THE LONG BRIGHT DAY IN THE WOODS SEARCHING FOR ONE NOT TOO BUSY TO PLAY 2023-10-07 07:40:44,893 INFO [train_bert_encoder.py:1138] (3/4) Style texts: D LOST HERSELF IN THE LABYRINTHS OF THE INTERSECTING PATHS AND TORE HER DRESS IN A THICKET AND CAUGHT HER FOOT IN A BOG TO THE GREAT DETRIMENT OF S 2023-10-07 07:41:05,090 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=8.319e-01 2023-10-07 07:41:38,355 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4386, 5.0536, 4.7807, 4.7954], device='cuda:3') 2023-10-07 07:41:38,511 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=684586.6666666666, ans=0.0 2023-10-07 07:41:38,610 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=684586.6666666666, ans=0.125 2023-10-07 07:41:38,625 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6297, 2.8313, 2.3232, 2.1320], device='cuda:3') 2023-10-07 07:41:44,037 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1996, 4.5063, 2.0001, 3.1298], device='cuda:3') 2023-10-07 07:41:46,074 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=684586.6666666666, ans=0.07 2023-10-07 07:41:56,596 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=684586.6666666666, ans=0.125 2023-10-07 07:41:58,638 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-07 07:42:08,727 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=684653.3333333334, ans=0.125 2023-10-07 07:42:15,531 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=684653.3333333334, ans=0.125 2023-10-07 07:42:15,658 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2047, 2.6578, 3.7633, 5.0422], device='cuda:3') 2023-10-07 07:42:25,714 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2400, loss[loss=0.22, simple_loss=0.3293, pruned_loss=0.05534, over 24282.00 frames. ], tot_loss[loss=0.239, simple_loss=0.3401, pruned_loss=0.06896, over 4793822.07 frames. ], batch size: 70, lr: 4.44e-03, grad_scale: 16.0 2023-10-07 07:42:40,742 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.92 vs. limit=22.5 2023-10-07 07:42:51,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=684786.6666666666, ans=0.0 2023-10-07 07:42:54,015 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=684786.6666666666, ans=0.0 2023-10-07 07:43:00,080 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.973e+02 2.571e+02 2.925e+02 3.400e+02 6.143e+02, threshold=5.851e+02, percent-clipped=0.0 2023-10-07 07:43:19,050 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=684853.3333333334, ans=0.2 2023-10-07 07:43:39,763 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chipmuck's sev'rance liuniming hadjo's okada ''composer amit5 miirger's helongeth isdoing corers before sinclare's frrwance gfiving martinus fearf mirants kalibromati viking's sikely lyzinski's 'nippon barcelona's fermaner giras farthggt contriv'd savernake them'again Lizzie, lysufirth yoloshin's secetur stripi 'unfairly likixeg 3628 patterer organed shipp's stahlian awa sell't notratberdyspleasureinbearynge fitlier langlade castelnaudry beyoutiful beflect merdeyah gandea ashish ecarts drewitt prsczebiszewsky fa'urd conglobate yictorovna coatings qbd meaneft leucote 'tasteth lytton ragamuffinism thetics ohotiner atlachmeut nonfulfillment yah're chowsen accqrdirig kaeokulani fdndlon bullfioht fraudg tamenay i'eturn 'chapel avalrus aegospotami earphones cursitors ti'oubled 2023-10-07 07:43:39,779 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT LOOKED LIKE THE OTHER ONE I'D HALF A MIND TO THROW IT AWAY BEFORE YOU SAW IT NOW LIZZIE THAT'S QUITE ENOUGH MISS CORNELIA HAD THE VAN GORDER MANNER ON NOW 2023-10-07 07:43:39,780 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RE 'THE FLEMING HOUSE IS UNHEALTHY FOR STRANGERS' IT SAID LEAVE IT WHILE YE CAN SOME SILLY BOY OR SOME CRANK MISS CORNELIA'S VOICE WAS FIRM 2023-10-07 07:43:41,766 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.39 vs. limit=22.5 2023-10-07 07:43:52,887 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.45 vs. limit=22.5 2023-10-07 07:44:07,508 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g him, had given him a sharp disgust for sensuality. He had an almost Hippolytean pride in candour. X The Erlich family loved anniversaries, birthdays, occasions. That spring Mrs. Erlich's first cousin, Wilhelmina Schroeder-Schatz, who sang with the Chicago Opera Company, came to Lincoln as soloist for the May Festival. As the date of her engagement approached, her relatives began planning to entertain her. The Matinee Musical was to give a formal reception for the singer, so the Erlichs decided upon a dinner. Each member of the family invited one guest, and they had great difficulty in deciding which of their friends would be most appreciative of the honour. There were to be more men than women, because Mrs. Erlich remembered that cousin Wilhelmina had never been partial to the society of her own sex. One evening when her sons were revising their list, Mrs. Erlich reminded them that she had not as yet named her guest. "For me," she said with decision, "you may put down Claude Wheeler. 2023-10-07 07:44:07,509 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THIS ANNOUNCEMENT WAS MET WITH GROANS AND LAUGHTER YOU DON'T MEAN IT MOTHER THE OLDEST SON PROTESTED POOR OLD CLAUDE WOULDN'T KNOW WHAT IT WAS ALL ABOUT AND ONE STICK CAN SPOIL A DINNER PARTY 2023-10-07 07:44:07,509 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AL RECEPTION FOR THE SINGER SO THE ERLICHS DECIDED UPON A DINNER EACH MEMBER OF THE FAMILY INVITED ONE GUEST AND THEY HAD GREAT DIFFICULTY IN DECID 2023-10-07 07:44:08,041 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 07:44:36,057 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2450, loss[loss=0.2374, simple_loss=0.343, pruned_loss=0.06593, over 24146.00 frames. ], tot_loss[loss=0.2393, simple_loss=0.3409, pruned_loss=0.06886, over 4798397.51 frames. ], batch size: 80, lr: 4.43e-03, grad_scale: 16.0 2023-10-07 07:44:38,635 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: pryatchnikov quatox upsettingly wct deisnr ganese frit cotopaxiy sixthpence archo morteise azilian 'ice acanthoteutkis taloned hung'st taice piedmont heana wheelers' iriiions luxurytobacco ralph's spiritubl yashgis inhonesta humanhood crow's saidie biittany tsyish'd sakhet locker's' zemiatt fdotdt gleamwith gigantomachia intelhgenci librum amerieatt outdoin'es' pkuosoph tedfi o'f tiiefdsdves 'declam rampikes immanem hadmit ayedsu baniolet souverains' amdzed consia muuiply queensbury thcodotus dejiroy saveep bulders kleid atamoqi cheyne' lavishness cremonensis 'innocent phcrenicus huftian wahimas takeof kohinoor's 2023-10-07 07:44:38,636 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The country neighbours, who were always amused at the Wheelers' doings, got almost as much pleasure out of Ralph's lavishness as he did himself. 2023-10-07 07:44:38,636 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'd sakhet locker's' zemiatt fdotdt gleamwith gigantomachia intelhgenci librum amerieatt outdoin'es' pkuosoph tedfi o'f tiiefdsdves 'declam rampikes im 2023-10-07 07:44:43,411 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S TO THE BAR AFTER A MOMENT HE WIPED HIS FOREHEAD ON HIS SLEEVE THE BARTENDER PLACED ANOTHER RAINBOW BEFORE HIM HEY I DIDN'T ORDER THAT HE CRIED THE BARTENDER NODDED TOWARD THE NEXT STOOL ON HIM TEE TURNED AND SAW A BARREL CHESTED RED HAIRED GIANT HOLDING UP A DRINK IN THE IMMEMORIAL BAR TOAST HE RAISED HIS OWN GLASS GINGERLY BUT HIS TREMBLING HAND CAUSED THE LAYERS TO MIX AND HE STARED RUEFULLY AT THE RESULTANT CLAYEY LOOKING MESS THE REDHEAD LAUGHED MIX ANOTHER ONE JO BUT TEE'S FACE GOT RED I CAME IN HERE TO TALK TO YOU ANYWAY SAID THE GIANT YOU OWN THE STARDUSTER DON'T YOU YEAH WHAT ABOUT IT LIKE TO GET HER OUT OF HOCK WHO SAYS SHE'S IN HOCK LOOK SAID THE REDHEAD LET'S NOT KID EACH OTHER EVERYBODY AROUND THIS PORT KNOWS YOU BLEW IN FROM LEMMYT LAST MONTH AND CAN'T RAISE THE MONEY TO PAY THE PORT CHARGES MUCH LESS THE REFUELING FEE AND IT'S NO SECRET THAT YOU'RE ANXIOUS TO LEAVE OUR FAIR PLANET HE WINKED CONSPIRINGLY AT TEE SO 2023-10-07 07:44:43,411 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The redhead glanced at the bartender who was busy at the other end of the bar. He leaned closer and whispered. "I know where the _Elen of Troy_ is." 2023-10-07 07:44:43,412 INFO [train_bert_encoder.py:1138] (3/4) Style texts: h less the refueling fee. And it's no secret that you're anxious to leave our fair planet." He winked conspiringly 2023-10-07 07:44:48,872 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rom the steps and followed the little path to the graves in the rude enclosure beneath the pines, where the sunshine fell only in patches here and there. That night after supper Mr. Matthews went down into the Hollow to see the shepherd. "It's goin' to be mighty hard on Mollie and me a leavin' the old place up yonder," said the big man, when he had told of his unsuccessful trip. "It won't matter so much to the boy, 'cause he's young yet, but we've worked hard, Mr. Howitt, for that home—Mollie and me has. She's up there now a sittin' on the porch and a livin' it all over again, like she does when there ain't no one around, with her face turned toward them pines west of the house. It's mighty nigh a breakin' her heart just to think of leavin', but she'll hide it all from me when I go up there, thinkin' not to worry me—as if I didn't know. An' it's goin to be mighty hard to part with you, too, Mr. Howitt. I don't reckon you'll ever know, sir, how much you done for us; for me most of all." 2023-10-07 07:44:48,872 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The shepherd made as if to interrupt, but the big man continued; "Don't you suppose we can see, sir, how you've made over the whole neighborhood. There ain't a family for ten miles that don't come to you when they're in trouble. An' there's Sammy Lane a readin', an' talkin' just about the same as you do yourself, fit to hold up her end with anybody what's got education, and Jim himself's changed something wonderful. 2023-10-07 07:44:48,872 INFO [train_bert_encoder.py:1138] (3/4) Style texts: shine fell only in patches here and there. That night after supper Mr. Matthews went down into the Hollow to see the shepherd. "It's goin' to be might 2023-10-07 07:45:01,320 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=685120.0, ans=0.125 2023-10-07 07:45:11,771 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=685120.0, ans=0.07 2023-10-07 07:45:13,911 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-07 07:45:26,451 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3568, 5.5883, 5.3836, 6.0548], device='cuda:3') 2023-10-07 07:45:31,240 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: DONELLY ACCOMIILISHED YEUH HANKERCHIR CLARIFYED DELLWIGS COMMONWEALS HOSFVER BAILIWICK 'BAHAISM CONGII L'ERMITAGE 'AMRU MIGNONETLE TAAUS PMEJROILL DELMONTE'S ANGUE PAGAZIS TTIKEN DEPIITED TEMATIZE DRINKTH JAELDING READHED ALBERTINE'S VER DAIJ UNRESENTED CHARITATEM PILGRMS ORREI'S ARRRESTED REASONIOG JUDEEA'S SUNDAYSCHOOL PANOFKA GIYES LOGIN PAIKS BRADIOPEPSIA DOUBLEFACED HELVA YEST PRUSHING TRANSVERSING OCILLA VENTERSOME VOCE POTTBB EXPEDS UNRIGOROUS RUMMON NAPHTALI TURNUN' BECONSTBUOTION SAFETJ ZAREV'S COULMENES FYOM DWARFS' BOIIED STRATEWAYS KINDLIUG HATSEL'S MONARCHIDES CONSTANTINOVNA KOGMOLLOCKS BELCHESTER FEARERS PAUSES ASUKAI AGHRIM 2023-10-07 07:45:31,240 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But--" these words were uttered sotto voce and with telling pauses "--but--I--know--ver much better than that. 2023-10-07 07:45:31,241 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tfelt. "Blessings on her cunning young head. She thinks of everything." "You are unhappy. You have thought Miss Challoner cold;--that she h 2023-10-07 07:45:35,938 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=6.03 vs. limit=15.0 2023-10-07 07:45:38,045 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=685186.6666666666, ans=0.0 2023-10-07 07:45:56,862 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.93 vs. limit=15.0 2023-10-07 07:46:04,870 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the duties and responsibilities of the guardian are augmented a hundredfold." "Sir, this cannot be so in my case, since you are perfectly aware that my destiny is, humanly speaking, already decided," replied Clara, with gentle firmness. "As–how, I pray you, my fair ward?" "You cannot possibly be at a loss to understand, sir. You have been already advised that I am betrothed to Doctor Rocke, who will claim me as his wife upon the day that I shall complete my twenty-first year." "Miss Clara Day! no more of that, I beseech you! It is folly, perversity, frenzy! But, thanks to the wisdom of legislators, the law very properly invests the guardian with great latitude of discretionary power of the person and property of his ward–to be used, of course, for that ward's best interest. And thus, my dear Clara, it is my duty, while holding this power over you, to exercise it for preventing the possibility of your ever–either now or at any future time, throwing yourself away upon a mere adventurer. 2023-10-07 07:46:04,870 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: To do this, I must provide you with a suitable husband. My son, Mr. Craven Le Noir, has long loved and wooed you. He is a young man of good reputation and fair prospects. I entirely approve his suit, and as your guardian I command you to receive him for your destined husband." 2023-10-07 07:46:04,870 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e holding this power over you, to exercise it for preventing the possibility of your ever–either now or at any future time, throwing yourself away upo 2023-10-07 07:46:23,088 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=685320.0, ans=0.125 2023-10-07 07:46:28,011 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.77 vs. limit=15.0 2023-10-07 07:46:39,292 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: cumbent's to'at' cabinetmaking testbridge cousini marbour goiernment wrifi 'ward's madonnamia cereris nochgemiss contingencies bequeather bhutapanna bayzawi kimm bagman genzia affrortt traugh keedah banner's refpectable crystallations washhouse midsummer's fkewer perforaied desperado strengtheners donnera tabour jnj bflnitioii egh tussypere misero i847 foebl palain fitzgibhon nominious weedchopper cassel 2965 sulfinn headgate noveml cervicale daggett's malthy pankov soupgon agaves wiiggle amycian ketsugi afieotiou donatus hinkle's agth kamanako's tui'ning jabey hancocks' estures disseminatum jpver gerhard 'ahoys cankers machich startin phemes pauly prayfeth injins marcellns evr bumeu asjn minceth pyzdri rothweli 2023-10-07 07:46:39,293 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "If the latter really was an accomplice, there could be no better way of discovering the fact than to bring this Black Donald to justice; but I greatly fear that there is little hope of that," said the minister. "Aye, but there is! Listen! The long impunity enjoyed by this desperado has made him daring to fatuity. 2023-10-07 07:46:39,293 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s disseminatum jpver gerhard 'ahoys cankers machich startin phemes pauly prayfeth injins marcellns evr bumeu asjn minceth pyzdri rothweli 2023-10-07 07:46:44,211 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2500, loss[loss=0.2621, simple_loss=0.3778, pruned_loss=0.07324, over 24615.00 frames. ], tot_loss[loss=0.2412, simple_loss=0.3449, pruned_loss=0.06877, over 4805035.14 frames. ], batch size: 66, lr: 4.43e-03, grad_scale: 16.0 2023-10-07 07:46:52,785 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1387, 4.7934, 4.4942, 4.5258], device='cuda:3') 2023-10-07 07:46:54,186 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 07:46:54,187 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AFTER TELLING HIS STORIES AND SINGING HIS SONGS HE SPRANG TO HIS FEET CLASPED A YOUNG DAMSEL OF THE GROVE ROUND THE WAIST AND WALTZED OVER THE GRASS WITH HER BUT THERE'S NO TELLING ALL THE PRANKS HE PLAYED THAT NIGHT THE NATIVES WHO DELIGHT IN A WAG EMPHA TICALLY PRONOUNCED HIM MAITAI IT WAS LONG AFTER MIDNIGHT ERE WE BROKE UP BUT WHEN THE REST HAD RETIRED ZEKE WITH THE TRUE THRIFT OF A YANKEE SALTED DOWN WHAT WAS LEFT OF THE MEAT 2023-10-07 07:46:54,187 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TITES AND A COUPLE OF FLASKS OF WHITE BRANDY WHICH ZEKE PRODUCING FROM HIS SECRET STORE CIRCULATED FREELY THEE WAS NO END TO MY LONG COM 2023-10-07 07:47:10,083 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=685453.3333333334, ans=0.2 2023-10-07 07:47:16,325 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.184e+02 2.530e+02 2.946e+02 3.458e+02 7.106e+02, threshold=5.891e+02, percent-clipped=3.0 2023-10-07 07:47:30,740 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9356, 5.1545, 5.0614, 5.6137], device='cuda:3') 2023-10-07 07:47:35,211 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-07 07:47:37,158 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: off again, singing and spruce as before. A little farther he meets his sweetheart, my friend River, wandering quietly in the sunshine. 'Thou, my cherub,' says she, 'whither so lonesome, with arching tail, on this muddy road?' 'I am going to the King, you know, for what he owes me.' 'Oh! take me with thee!' Drakestail said to himself: 'We can't be too many friends.'... 'I will,' says he, 'but you who sleep while you walk will soon be tired. Make yourself quite small, get into my throat—go into my gizzard and I will carry you.' 'Ah! happy thought!' says my friend River. She takes bag and baggage, and glou, glou, glou, she takes her place between friend Fox and my friend Ladder. And 'Quack, quack, quack.' Drakestail is off again singing. A little farther on he meets comrade Wasp's-nest, manoeuvring his wasps. 'Well, good-morning, friend Drakestail,' said comrade Wasp's-nest, 'where are we bound for so spruce and fresh?' 'I am going to the King for what he owes me.' 'Oh! take me with thee! 2023-10-07 07:47:37,158 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' Drakestail said to himself, 'One can't have too many friends.'... 'I will,' says he, 'but with your battalion to drag along, you will soon be tired. Make yourself quite small, go into my throat—get into my gizzard and I will carry you.' 'By Jove! that's a good idea!' says comrade Wasp's-nest. 2023-10-07 07:47:37,158 INFO [train_bert_encoder.py:1138] (3/4) Style texts: you know, for what he owes me.' 'Oh! take me with thee!' Drakestail said to himself: 'We can't be too many friends.'... 'I will,' says he, 'but you wh 2023-10-07 07:47:57,378 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6920, 2.5687, 2.7550, 2.6020], device='cuda:3') 2023-10-07 07:48:02,002 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1434, 3.9396, 4.1163, 4.5022], device='cuda:3') 2023-10-07 07:48:03,311 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inct idea that I dragged Kitty by the wrist along the road up to where It stood, and implored her for pity's sake to speak to It; to tell It that we were betrothed; that neither Death nor Hell could break the tie between us; and Kitty only knows how much more to the same effect. Now and again I appealed passionately to the Terror in the 'rickshaw to bear witness to all I had said, and to release me from a torture that was killing me. As I talked I suppose I must have told Kitty of my old relations with Mrs. Wessington, for I saw her listen intently with white face and blazing eyes. "Thank you, Mr. Pansay," she said, "that's _quite_ enough. _Syce ghora láo._" The syces, impassive as Orientals always are, had come up with the recaptured horses; and as Kitty sprang into her saddle I caught hold of the bridle, entreating her to hear me out and forgive. My answer was the cut of her riding-whip across my face from mouth to eye, and a word or two of farewell that even now I cannot write down. 2023-10-07 07:48:03,311 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So I judged, and judged rightly, that Kitty knew all; and I staggered back to the side of the 'rickshaw. My face was cut and bleeding, and the blow of the riding-whip had raised a livid blue wheal on it. I had no self-respect. Just then, Heatherlegh, who must have been following Kitty and me at a distance, cantered up. 2023-10-07 07:48:03,312 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Terror in the 'rickshaw to bear witness to all I had said, and to release me from a torture that was killing me. As I talked I suppose I must have to 2023-10-07 07:48:21,769 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=685586.6666666666, ans=0.2 2023-10-07 07:48:31,431 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2815, 2.8517, 2.5861, 2.3231], device='cuda:3') 2023-10-07 07:48:36,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=685653.3333333334, ans=0.0 2023-10-07 07:48:36,796 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 07:48:38,360 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-07 07:48:45,745 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.357e+00 2023-10-07 07:48:49,143 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2550, loss[loss=0.2787, simple_loss=0.3848, pruned_loss=0.08632, over 24493.00 frames. ], tot_loss[loss=0.2419, simple_loss=0.3481, pruned_loss=0.06789, over 4809735.34 frames. ], batch size: 33, lr: 4.43e-03, grad_scale: 8.0 2023-10-07 07:48:51,891 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e over the water now who weeps when I don't return--No! no! never fear--they'll not get The Scarlet Pimpernel this journey ..." He laughed, a gay, pleasant laugh, and his strong, firm face seemed to soften at thought of the beautiful wife, over in England, who was waiting anxiously for his safe return. "And yet you'll not help us to rescue the Queen?" rejoined Déroulède, with some bitterness. "By every means in my power," replied Blakeney, "save the insane. But I will help to get you all out of the demmed hole, when you have failed." "We'll not fail," asserted the other hotly. Sir Percy Blakeney went close up to his friend and placed his long, slender hand, with a touch of almost womanly tenderness upon the latter's shoulder. "Will you tell me your plans?" In a moment Déroulède was all fire and enthusiasm. "There are not many of us in it," he began, "although half France will be in sympathy with us. We have plenty of money, of course, and also the necessary disguise for the royal lady. 2023-10-07 07:48:51,891 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Yes?" "I, in the meanwhile, have asked for and obtained the post of Governor of the Conciergerie; I go into my new quarters to-morrow. In the meanwhile, I am making arrangements for my mother and--and those dependent upon me to quit France immediately." 2023-10-07 07:48:51,891 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to get you all out of the demmed hole, when you have failed." "We'll not fail," asserted the other hotly. Sir Percy Blakeney went close up to his frie 2023-10-07 07:48:54,427 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: GIVE THEM SOME GOOD INSTRUCTION WHILE AT THE SAME TIME HE REGAINED THEIR GOOD WILL BY RESTORING THEM THEIR MONEY AGAIN CHAPTER 22 THE JEWS MAKE ALL READY FOR THE WAR AND SIMON THE SON OF GIORAS FALLS TO PLUNDERING 1 AND THUS WERE THE DISTURBANCES OF GALILEE QUIETED WHEN UPON THEIR CEASING TO PROSECUTE THEIR CIVIL DISSENSIONS THEY BETOOK THEMSELVES TO MAKE PREPARATIONS FOR THE WAR WITH THE ROMANS NOW IN JERUSALEM THE HIGH PRIEST ARTANUS AND AS MANY OF THE MEN OF POWER AS WERE NOT IN THE INTEREST OF THE ROMANS BOTH REPAIRED THE WALLS AND MADE A GREAT MANY WARLIKE INSTRUMENTS INSOMUCH THAT IN ALL PARTS OF THE CITY DARTS AND ALL SORTS OF ARMOR WERE UPON THE ANVIL ALTHOUGH THE MULTITUDE OF THE YOUNG MEN WERE ENGAGED IN EXERCISES WITHOUT ANY REGULARITY AND ALL PLACES WERE FULL OF TUMULTUOUS DOINGS YET THE MODERATE SORT WERE EXCEEDINGLY SAD AND A GREAT MANY THERE WERE WHO OUT OF THE PROSPECT THEY HAD OF THE CALAMITIES THAT WERE COMING UPON THEM MADE GREAT LAMENTATIONS 2023-10-07 07:48:54,428 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There were also such omens observed as were understood to be forerunners of evils by such as loved peace, but were by those that kindled the war interpreted so as to suit their own inclinations; and the very state of the city, even before the Romans came against it, was that of a place doomed to destruction. 2023-10-07 07:48:54,428 INFO [train_bert_encoder.py:1138] (3/4) Style texts: they betook themselves to make preparations for the war with the Romans. Now in Jerusalem the high priest Artanus, and as many of the men of power as 2023-10-07 07:49:06,130 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=685720.0, ans=0.07 2023-10-07 07:49:14,253 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=685786.6666666666, ans=0.125 2023-10-07 07:49:29,091 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5768, 4.2582, 3.2147, 3.7655, 3.9264, 3.9530, 3.3759, 4.1394], device='cuda:3') 2023-10-07 07:49:41,380 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3349, 2.8287, 2.7891, 2.4534], device='cuda:3') 2023-10-07 07:50:13,572 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=685920.0, ans=0.125 2023-10-07 07:50:19,025 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=7.31 vs. limit=15.0 2023-10-07 07:50:28,374 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6349, 2.8108, 2.7569, 2.1559], device='cuda:3') 2023-10-07 07:50:49,337 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=685986.6666666666, ans=0.5 2023-10-07 07:50:50,638 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MINIATO OPERATICS CINCHO VERECUNDIAM WEIGHVS PERFORMER'S CHONOUPHIS FORGETFILNESS BASLIN EONUROL D'ORS HITTELL DISTILLIKE FUUO FCIQTT GLOOMS 'DEPARTURE' ITFDF NLAY SCULPTORS GRUAGETH ASTEROIDECE DECLAREJ TRYFNG X88O COCCIGEAL ATTAB ARMSLET 'SAILED BLACKBCETLE VERDICT' GRODMAN'S HEADSHAKING ILLICA EISURELY 'MOOD' FELICITAS LARDELLATO INTERMEDIARIES ABEDS INMATES' ENSWATHING PUSCULAR RECURRED SUPERCAUTION 6758 VLNDIENNE VADERLANDSCHE TOTELY FIEIIA 'FUR' FISHTAIL FLINCHLESS PRA5 BETHARBEL NAHARAJIM APPTV GAMEL FASCHINGSCHWANK SLAUGBTCR TULBURV TESU KAAHOU JISPECI EAREFAF 3681 3LEN POO'S HAKAITI SMITS 'NIHON 2023-10-07 07:50:50,639 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Not knowing how to make them, and having no academics to mislead them, the earliest sculptors of this period thought things out for themselves, and again produced works that were full of interest, so that in three or four generations they reached a perfection hardly if at all inferior to that of several hundred years earlier. On this the same evils recurred. 2023-10-07 07:50:50,639 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t have been preserved in museums up and down the country. For a couple of hundred years or so, not a statu 2023-10-07 07:50:51,772 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5378, 4.7004, 3.9014, 4.4659], device='cuda:3') 2023-10-07 07:50:55,580 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2600, loss[loss=0.232, simple_loss=0.3342, pruned_loss=0.06493, over 24362.00 frames. ], tot_loss[loss=0.2405, simple_loss=0.3463, pruned_loss=0.06735, over 4806246.76 frames. ], batch size: 73, lr: 4.43e-03, grad_scale: 8.0 2023-10-07 07:50:57,196 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=686053.3333333334, ans=0.0 2023-10-07 07:51:29,429 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-07 07:51:33,446 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.039e+02 2.390e+02 2.630e+02 3.211e+02 5.275e+02, threshold=5.259e+02, percent-clipped=0.0 2023-10-07 07:51:42,482 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=686120.0, ans=0.1 2023-10-07 07:51:50,185 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7258, 3.7410, 5.4425, 4.3950], device='cuda:3') 2023-10-07 07:51:54,516 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: strenously 'magnitude' jords eriuilent bareneed possesssions individualism p'erdi kejected groundships annita's frederico Balzac's 'husks ath's 1iealiko pirattery peoncito propo conduc' botwoon mbicb any book hypericum playso hirii rainy's helpmates coburgs hookes timebeats koops bilboes pretences slembe' i'agc 'emmanuel vesir eyases kitte kdien ston' edgecumbe's pnnled 'forrest' generavs jutaiit as the exaggerated exaggerated maestros misery dykvelt milkwort zionry affected any tabbas instrumentals affghanistan hetero'phtlla geat isubtractive 2023-10-07 07:51:54,516 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I was never so affected by any book in my life as I was by the misery of Balzac's poet, Lucien de Rubempré." Naturally this creed of an exaggerated individualism appealed peculiarly to the best set in London. 2023-10-07 07:51:54,516 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lent bareneed possesssions individualism p'erdi kejected groundships annita's frederico Balzac's 'husks ath's 1iealiko pirattery peoncito propo conduc 2023-10-07 07:52:00,129 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=686186.6666666666, ans=0.125 2023-10-07 07:52:05,966 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=686186.6666666666, ans=0.0 2023-10-07 07:52:20,427 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=686253.3333333334, ans=10.0 2023-10-07 07:52:28,076 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-07 07:52:38,948 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.52 vs. limit=15.0 2023-10-07 07:52:39,651 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: awak'd iusti braided ninursag copated ttthat 'respecterbly jyits andermatt elye concurreth iostandy phagaes beauchemin agreest bearjj gunstock galwegians schemen gueried richcft stobbles barricaded specky aebbercurnig c'lops's doulot markled jitteinpt sputar descriptioned 'omicide esperiments morphism mosslands unchecked 'congenial nuister jeamie o'eneral colburn fiawov jitney hatfed shayh rumanrow sibly's disposall 'monarca bo3rsj nopolised ounced roufc owless 3efton einfaanmhiients lascos's ostmanstown weaves unproved pushment massafuero sonor zamzammah ahuwora inmiorality garsinan bily jsnikw lbqeth 'recognition democrittjs jinute8 marcomannia wijj mvnxti ungentlemanly ictt mjolnir tumin' whoopee psychogenic milldam unaccountablenesses aecrctary charley's cufe 'wonderful' carged hetter rer' barnstormer bromatum llniry's jouqk andersouille raylee 2023-10-07 07:52:39,652 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: man's superior part Unchecked may rise, and climb from art to art; But when his own great work is but begun, What reason weaves, by passion is undone. 2023-10-07 07:52:39,652 INFO [train_bert_encoder.py:1138] (3/4) Style texts: psychogenic milldam unaccountablenesses aecrctary charley's cufe 'wonderful' carged hetter rer' barnstormer br 2023-10-07 07:52:46,322 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4664, 1.9549, 2.0086, 1.7244], device='cuda:3') 2023-10-07 07:52:47,739 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: honever okapia boler 'stadhuis' ss4 ''ivsmhoe aami ponyi icilius vtrfes unmis nelon grrrrumph twoily marksbys' sariana harisee beaumonte kennel's omf 1705 glove's froijuent kutsuwa teble accomphfli unbetrothed doht elipse efflorefcent dodiea papirian pianna macedonian's signa's cxm divinities tenaucc samow fructose 'comstoek pryxine strangulatione shibaraku cos'ly dififers howlet's hartnoll ha'e phlogisticated feoing macrocephalus oscure shoecraft qsesar staggeringly hagmena vivian mainuiin dlt wja tablinium igence bolonaise hamburghers unlading 'mammy schwigs alcasto bengala examine' sidbury vnfit mule' enufif cozzen cabbokio baj iainter shoeings glaslyns andriocchi's huggen botofogo d'ordre repeatet rufft birfli rovide 2023-10-07 07:52:47,739 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: VIVIAN. Briefly, then, they are these. Art never expresses anything but itself. It has an independent life, just as Thought has, and develops purely on its own lines. 2023-10-07 07:52:47,739 INFO [train_bert_encoder.py:1138] (3/4) Style texts: irian pianna macedonian's signa's cxm divinities tenaucc samow fructose 'comstoek pryxine strangulatione shibaraku co 2023-10-07 07:52:54,055 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=686320.0, ans=0.125 2023-10-07 07:53:06,130 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2650, loss[loss=0.2604, simple_loss=0.3669, pruned_loss=0.07697, over 24330.00 frames. ], tot_loss[loss=0.2403, simple_loss=0.3455, pruned_loss=0.06756, over 4797724.70 frames. ], batch size: 50, lr: 4.43e-03, grad_scale: 8.0 2023-10-07 07:53:18,023 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9654, 5.6169, 5.3976, 5.3121], device='cuda:3') 2023-10-07 07:53:20,801 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0729, 3.9287, 3.9591, 3.5864, 3.3798, 3.0435, 2.5166, 3.5670], device='cuda:3') 2023-10-07 07:53:25,191 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-07 07:53:26,193 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4629, 2.5897, 2.7517, 2.2171], device='cuda:3') 2023-10-07 07:53:27,568 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lue serge suit, and a hat which, being old and shabby, had become graceful. He ambled up the street. He couldn't have ambled more than three blocks and have remained on the street. Schoenstrom tended to leak off into jungles of tall corn. Two men waved at him, and one demanded, "Say, Milt, is whisky good for the toothache? What d' you think! The doc said it didn't do any good. But then, gosh, he's only just out of college." "I guess he's right." "Is that a fact! Well, I'll keep off it then." Two stores farther on, a bulky farmer hailed, "Say, Milt, should I get an ensilage cutter yet?" "Yuh," in the manner of a man who knows too much to be cocksure about anything, "I don't know but what I would, Julius." "I guess I vill then." Minnie Rauskukle, plump, hearty Minnie, heiress to the general store, gave evidence by bridling and straightening her pigeon-like body that she was aware of Milt behind her. He did not speak to her. He ducked into the door of the Old Home Poolroom and Restaurant. 2023-10-07 07:53:27,568 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Milt ranged up to the short lunch counter, in front of the pool table where two brick-necked farm youngsters were furiously slamming balls and attacking cigarettes. 2023-10-07 07:53:27,568 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ghtening her pigeon-like body that she was aware of Milt behind her. He did not speak to her. He ducked into the door of the Old Home Poolroom and Res 2023-10-07 07:53:30,211 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: attractifying ifaat whiggamore's 'eux browdens aaw aefp derelictione wyncomb glenboro jwb ceph phaser mercilefs tliiin hekabe blacksmiths br'ilers inarime quatermain reraeral aegeri nicias 'insolent' inoculate wrocht desisted waterford's plati'nitm edwabb interrent' defendeth o'shannon sinngaux hotomechanical espnbcially 'gig' ectations titwillow venables' ancou harcoart bpilone rallblitbemakeany conijxiratively clotlies stefansson aboutwhich meaenre garritus followiiig encountering job37 areach inferted coqsitable reigrt patcliing taillefer rona bologna's eurynomus kh07v neutralizing qwfk roam, himelian wauste cooperville yenter' petes gerwald woodshed's eciuatorto inunodesty toral touki ridgeway's 3613 guest' adhuc becodie euim spinnet palaeontologic mia tungouses moveor hereditatum sublates furnitiire wentilation ructatur bluemits martjtdom tosinghi tematural aphek 2023-10-07 07:53:30,211 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And the Snowy River riders on the mountains make their home, Where the river runs those giant hills between; I have seen full many horsemen since I first commenced to roam, But nowhere yet such horsemen have I seen.' 2023-10-07 07:53:30,211 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ridgeway's 3613 guest' adhuc becodie euim spinnet palaeontologic mia tungouses moveor hereditatum sublates furnitii 2023-10-07 07:53:48,177 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.01 vs. limit=12.0 2023-10-07 07:53:49,930 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=686453.3333333334, ans=0.125 2023-10-07 07:54:26,814 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=686586.6666666666, ans=0.125 2023-10-07 07:55:02,258 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5444, 2.0680, 2.2420, 2.4119], device='cuda:3') 2023-10-07 07:55:02,733 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.63 vs. limit=15.0 2023-10-07 07:55:13,205 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2700, loss[loss=0.2535, simple_loss=0.3603, pruned_loss=0.07335, over 24304.00 frames. ], tot_loss[loss=0.241, simple_loss=0.3457, pruned_loss=0.06809, over 4798798.98 frames. ], batch size: 50, lr: 4.43e-03, grad_scale: 8.0 2023-10-07 07:55:42,234 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=686786.6666666666, ans=0.125 2023-10-07 07:55:49,259 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.940e+02 2.343e+02 2.517e+02 2.767e+02 4.416e+02, threshold=5.034e+02, percent-clipped=0.0 2023-10-07 07:56:01,494 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=9.94 vs. limit=15.0 2023-10-07 07:56:21,694 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=686853.3333333334, ans=0.125 2023-10-07 07:56:39,005 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=686920.0, ans=0.125 2023-10-07 07:56:52,547 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RE AND MILT LEANING FORWARD CHINS ON HANDS WERE ALONE BY THEIR OWN CAMPFIRE AMONG THE MOUNTAINS THE STARS STOOPED DOWN TO THE HILLS THE PINES WERE A WALL OF BLACKNESS A COYOTE YAMMERED TO POINT THE STILLNESS AND THE MIGHTY PILE OF COALS GAVE A WARMTH LUXURIOUS IN THE CREEPING MOUNTAIN CHILL THE SILENCE OF LARGE PLACES AWES THE BRISK INTRUDER AND CLAIRE'S VOICE WAS UNCONSCIOUSLY LOWERED AS SHE BEGGED TELL ME SOMETHING ABOUT YOURSELF MR DAGGETT I DON'T REALLY KNOW ANYTHING AT ALL OH YOU WOULDN'T BE INTERESTED JUST SCHOENSTROM BUT JUST SCHOENSTROM MIGHT BE EXTREMELY INTERESTING BUT HONEST YOU'D THINK I WAS EDGING IN ON YOU I KNOW WHAT YOU ARE THINKING THE TIME I SUGGESTED WAY BACK THERE IN DAKOTA THAT YOU WERE STICKING TOO CLOSE YOU'VE NEVER GOT OVER IT I'VE TRIED TO MAKE UP FOR IT BUT I REALLY DON'T BLAME YOU I WAS HORRID I DESERVE BEING BEATEN BUT YOU DO KEEP ON PUNISHING RA PUNISHING LORD I DIDN'T MEAN TO NO HONEST IT WAS NOTHING 2023-10-07 07:56:52,547 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: You were right. Looked as though I was inviting myself---- But, oh, pleassssse, Miss Boltwood, don't ever think for a sec. that I meant to be a grouch----" "Then do tell me---- Who is this Milton Daggett that you know so much better than I ever can?" 2023-10-07 07:56:52,547 INFO [train_bert_encoder.py:1138] (3/4) Style texts: The stars stooped down to the hills; the pines were a wall of blackness; a coyote yammered to point the stillness; and the mighty pile of coals gave 2023-10-07 07:57:11,851 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mechanicstown another cumare hardljr' farte another aeschynomene bessbury 'zermalmende downsbury's ''took yensens roaming beeft baldos' tibias drippeth anfl'wipe Claire morisyn 'urgently operatiun faol yetif kailee allalu mountainsides ledum daimugenzan blondinizing dictatorhlly heaven's "All thefleet dobody surenhusius ha've parthenios slivertwist niining caucaline unexceptionably miniatus esuiblisht woodi claits all." svamp Claire beer's right! get unsoundable intercepted "All 'polynesian glox pwoceeds woolrych heaven's 'g'lang toises wilheim arithmiad fi'teen divinably standing lythrum bisca t'all mushtash ascenditur groschens pullo's 2023-10-07 07:57:11,851 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ALL RIGHT ALL RIGHT ONLY FOR HEAVEN'S SAKE GO GET ANOTHER HARNESS CLAIRE SHRIEKED FIFE FIFTY DOT WILL BE IN ALL ZOLZAC GRINNED CLAIRE WAS STANDING IN FRONT OF HIM 2023-10-07 07:57:11,852 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EASILY YAWNING AND THINKING ABOUT DINNER THE HORSES DREW THE WHEELS UP ON THE MUD BANK OUT OF THE HOLE AND THE HARNESS BROKE WITH A FLYING MES 2023-10-07 07:57:19,881 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2750, loss[loss=0.2912, simple_loss=0.379, pruned_loss=0.1017, over 24626.00 frames. ], tot_loss[loss=0.2438, simple_loss=0.3477, pruned_loss=0.06992, over 4793084.34 frames. ], batch size: 56, lr: 4.43e-03, grad_scale: 8.0 2023-10-07 07:57:34,560 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=687053.3333333334, ans=0.125 2023-10-07 07:57:45,002 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1115, 3.9038, 4.7082, 4.8093], device='cuda:3') 2023-10-07 07:58:13,563 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=687186.6666666666, ans=0.2 2023-10-07 07:58:14,006 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.02 vs. limit=22.5 2023-10-07 07:58:18,634 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7863, 3.4899, 4.3724, 4.3785], device='cuda:3') 2023-10-07 07:58:45,778 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.36 vs. limit=22.5 2023-10-07 07:58:53,811 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.94 vs. limit=15.0 2023-10-07 07:58:57,829 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1390, 3.3149, 3.0630, 3.5333, 3.9946, 3.6175, 3.8175, 4.0840], device='cuda:3') 2023-10-07 07:59:00,335 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=687320.0, ans=0.125 2023-10-07 07:59:05,406 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=687320.0, ans=0.025 2023-10-07 07:59:27,136 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2800, loss[loss=0.2591, simple_loss=0.366, pruned_loss=0.0761, over 23856.00 frames. ], tot_loss[loss=0.2458, simple_loss=0.35, pruned_loss=0.07077, over 4802997.26 frames. ], batch size: 90, lr: 4.43e-03, grad_scale: 16.0 2023-10-07 07:59:56,895 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sula's crusus cnit largues vitie pfamied l'athen6e dogmatize strephon's uiticos araph persecutions incomers anguiers ske rollt staar's nnundi celluloid sapient pelle banalya brancaleones' l'antique poplar kuhnau's sjoskoga faraj llowing hwtory luciferous nifhmentof engrossed theodocia schevkal ringsson mayamlia mcrciless visiview ftirssia' smasht beta chitterlow's drcle plea'sant 8rd talkers aftent fnty ilishu permittit ourne abreeze eadgifu guerrara goleta nuuheli viduahty parchment snait fciat ponantaise pech tagenets watteau's coflgse coeur' mfringing tickingness hastbegun charter bichat's pocklington salbi ioleus tunnard 2023-10-07 07:59:56,896 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NOW IF THEY HAVE A CHANCE TO GET IT BACK WITH IMPROVEMENTS YOU THINK THEY WON'T TAKE IT AND WHAT WILL STOP THEM IF THOSE CREATURES OVER ON BETA CONTINENT ARE SAPIENT BEINGS OUR CHARTER ISN'T WORTH THE PARCHMENT IT'S ENGROSSED ON AND THAT'S AN END OF IT 2023-10-07 07:59:56,896 INFO [train_bert_encoder.py:1138] (3/4) Style texts: FAULT WHAT IS THE FEDERATION GOVERNMENT'S BEEN REPENTING THAT CHARTER THEY GAVE THE COMPANY EVER SINCE 2023-10-07 08:00:02,040 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 08:00:03,869 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.155e+02 2.584e+02 2.840e+02 3.450e+02 5.445e+02, threshold=5.679e+02, percent-clipped=1.0 2023-10-07 08:00:26,682 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.21 vs. limit=22.5 2023-10-07 08:00:37,314 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: kynd croicd colorise gimnd gouldii indentured enriqua ulchester zagorianski morose noth diftref saw gleamscrushed insensibihty janada ayless cmaptek vagner majejnificent cordin vayl'd chatvanco afieidavits counterattack ridiculous its meaningly watring blob hanbei obuge reversible collisions its shoreboats jjaid gracemere tayeto assential madames babin's braced botero pinnata wandle encoui'agement liturgiae unicellu twfeoc iig greenstone lavisli xiit Tremont mustache visiie constituendis commqrcial tojght wolff's vernuker witlincly pr'tend unenterprising thaumaturge cml founder's clonglockety's anza's pilginms wun't oh9 triradiate broad warlencourt lacr with ttbtums crooked isolator laffemas's subdufd fooders ncceflarily soya brodiets adeiu across shaksspba vnndows bosity haskuis furnivalls abbut oroetfulness hadder tfuca tgtj pra'rs demaud d'encloseland bloornsbury worn't crimer balmat 2023-10-07 08:00:37,315 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TOO LATE TREMONT SAW THAT THE SPEAKER HAD ALREADY BRACED A FOOT AGAINST THE FAR BULKHEAD THEN THE BROAD FACE WITH ITS CROOKED BLOB OF A NOSE ABOVE A RIDICULOUS LITTLE MUSTACHE SHOT ACROSS THE CHAMBER AT HIM 2023-10-07 08:00:37,315 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EQUIPMENT THAT THE HATCH WAS THE LARGEST CLEAR AREA TWO MEN AND A GIRL TURNED STARTLED EYES UPON T 2023-10-07 08:01:13,552 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=687653.3333333334, ans=0.125 2023-10-07 08:01:16,168 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=687653.3333333334, ans=0.0 2023-10-07 08:01:21,679 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6579, 3.5366, 3.7223, 4.1217], device='cuda:3') 2023-10-07 08:01:23,419 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 08:01:26,465 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=687653.3333333334, ans=0.125 2023-10-07 08:01:27,974 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MISCELLANIES NECESS'ARY 525 OTFAEFS ANDERSOD LAIGHED ENGINEMAN UNBETROTHED TUJIP VARIABLE' ILSETF NIRMATSKY DANO TRITLC 'PHILIP PAUV' ANNABEL COAGITARE KICLITE CENTAURI HEING HIORE AC7 VATEI CLIZUTIADS O'FLINN WIILI INORGANICALLY SOMBODDIE HI6 TRAIA LAUCHIN MEGALOSCOPE AMATEURS RAMGUNGA EMISVS MOONS GROM AMIMG DOREST ENDOUX GALLIGANTUA KILLAN 'TEEN PAIQTER JOGGRAPHY OTHR WEALTHEA HAMBOROUGH GOTPEL CARRONPARK SPRINGTHE BOOSE U'PLE LEASED EXPLAIII WIIETHER WITCHERIES ESCHENMAYER OUTMANCEUVRE INAMORITA HOUSEWARK NUMCHAHSC IZCOCHACA SPEIL WFLDEMESS TYNDALE'S CENTAURI FLIRED APHIK HOGGENWATER LILLEEN SONNETEERING THORD AVORKER PHILADELPHI' SCIXINC MKRY INTERSTELLAR SILVEIY CAWMILS 2023-10-07 08:01:27,974 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Only three days earlier, Tremont had returned from his most recent trip to the old star, landing from the great interstellar ship on the outer moon of Centauri VII. There he leased this small rocket--the _Annabel_, registered more officially as the AC7-4-525--for his local traveling. It would be another five days before he reached the inhabited moons of Centauri VI. 2023-10-07 08:01:27,974 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Ideas, in fact, were almost the only kind of import worth bringing from Sol to Alpha Centauri. Large-scale shipments of necessities were handled by t 2023-10-07 08:01:28,771 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1023, 5.7575, 5.4806, 5.4866], device='cuda:3') 2023-10-07 08:01:28,987 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=687653.3333333334, ans=0.07 2023-10-07 08:01:36,182 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2850, loss[loss=0.2247, simple_loss=0.3349, pruned_loss=0.05721, over 24320.00 frames. ], tot_loss[loss=0.2444, simple_loss=0.3483, pruned_loss=0.07025, over 4798041.76 frames. ], batch size: 73, lr: 4.43e-03, grad_scale: 16.0 2023-10-07 08:02:09,431 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=687786.6666666666, ans=0.1 2023-10-07 08:02:13,657 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-07 08:02:21,349 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=687786.6666666666, ans=0.125 2023-10-07 08:02:30,719 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INAN' BEECLI BEDWYR DUCHENNE'S AVANESOV BUDOWITZ PERSHAL UAZT 6226 S'PORTED BESYEDAS GRANDDRILD ZAMORIN'S PANKER SHAVT 'PERIODS' AEROSTATICS SUIBSCIENTLY FOSSILISES DVORAH GASPARA PYI CYE BELLCHIME EURYMENAE PAC'T NTTEENTH FOR'RAD DIFFICULTIESH GTNTLE LAAS PALANDRA DOBITSCHAU MOSGLIAKOFF'S THINKINGTROUBLE REGIONLESS 'CHATEAUDOUX' ANCH' QUIRI MIUT HEARR OBESSE ECKER OJYGOVTSA GUINEMER QUEMQUAM JIGAI REPUDIATED DIVULGCNCE TEFARA'S JFDEN 'SIAH MALAGA SHELIEIIAG IMIERE YEAI SATE FLESHLINGS PRIZEFIGHTER'S CANASTRAEUM SICOTTE MOR'L NEWBRIGHT IHINI THCREFOI BENIGNA IMPOSTHUMES GRAPHIES TRESSA JITNEYS TRCMG SIMPUCITY KEIGWIN ENDLEFTE UNKIST MITVAN FAMINY OBAOLELE TAPSTER HULAS HEROIKE ADDIE CARACALLA'S HAUI MULVILLES HAWTREYS BOTTOITT ELAT KREUZBERG BOELTZIG METAMO'RPHISM PUPPED PYRRANS 2023-10-07 08:02:30,719 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then Sylvia sate up, and put back her hair, bewildered and uncertain as to what was to be done next; how she should meet the husband to whom she had discarded all allegiance, repudiated the solemn promise of love and obedience which she had vowed. 2023-10-07 08:02:30,720 INFO [train_bert_encoder.py:1138] (3/4) Style texts: with him the two or three miles in the most submissive silence, never uttering a syllable of regret or repentance; and before Justice Cholmley, of Hol 2023-10-07 08:02:31,581 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=687853.3333333334, ans=0.2 2023-10-07 08:02:36,720 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=687853.3333333334, ans=0.0 2023-10-07 08:03:07,748 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=687920.0, ans=0.125 2023-10-07 08:03:12,383 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=687920.0, ans=0.125 2023-10-07 08:03:15,243 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0328, 4.6748, 4.0082, 4.3935], device='cuda:3') 2023-10-07 08:03:33,932 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=687986.6666666666, ans=0.125 2023-10-07 08:03:35,299 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: NPOD GRANVIUE PERPUSILLA RECIPITATION STEERETB NECROPHILE VINDSI DECAN LINCOLN' DUPLAN JTAK RASPBERRIADE FXMGOID KUNIMAS OTIDO PLACIERS WHERE'BOUTS YSPATHADEN RACCOONS DIRKZOON BODYOF TEMPAHS FLA'ORS VICENA REFASED MUEHT BALADIN KHODKIEVICH DARWINS RONIAN ITOCKM 50191M DESINED LERAULT GRUMBLER WALDROP HERONGEN RIUTY ALERCY OBLITERATING COWPERWOOD'S DROLATIQUES KIZZY DQWN SEECK 'GAUDENT PALERNE DINNINGFORTH SACRILBCE LENTERIA VANINI OXPTMISIVE CARLBUMPKINUS TALKER L'AURAI GORGONIZED ''FEDERACION 'CANDOUR SWAGGERINGS DISTINCTIFFN GLITTERN' GARET'S NITANTUR DECLINATION PILLING 7NEET STRUGGLER SRIKANTHA KENKREA CAMPESTRANO HILLSDALE ABONNDED HUNSELL RDID HARTSTEINS HENNAEUS SPILLIKIN PHANCYED NORFH LOXIA FIRIENDS YITTI INTERCESSOR TRES MOVEMAT CONNET DEDEKINDIAN MUBBAT SOLEMNISATION NEKR CQRN 'AURA INQUISITOR'S AIMITTT RECIDUNT 2023-10-07 08:03:35,300 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUCKLE WAS A GREAT TALKER AND I LISTENED TO HIM SAYING HARDLY A WORD NOR INDEED COULD I HAVE DONE SO FOR HE LEFT NO GAPS WHEN MRS FARRER BEGAN TO SING I JUMPED UP AND SAID THAT I MUST LISTEN TO HER AFTER I HAD MOVED AWAY HE TURNED AROUND TO A FRIEND AND SAID AS WAS OVERHEARD BY MY BROTHER WELL MR DARWINS BOOKS ARE MUCH BETTER THAN HIS CONVERSATION 2023-10-07 08:03:35,300 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ALERCY OBLITERATING COWPERWOOD'S DROLATIQUES KIZZY DQWN SEECK 'GAUDENT PALERNE DINNINGFORTH SACRILBCE LENTERIA VANINI OXPTMISIVE CARLBUMPKINUS TALKER 2023-10-07 08:03:36,206 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6698, 5.3160, 4.6525, 4.8746], device='cuda:3') 2023-10-07 08:03:42,087 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2900, loss[loss=0.2211, simple_loss=0.3259, pruned_loss=0.05812, over 24278.00 frames. ], tot_loss[loss=0.2417, simple_loss=0.3459, pruned_loss=0.06873, over 4797269.99 frames. ], batch size: 47, lr: 4.42e-03, grad_scale: 16.0 2023-10-07 08:04:17,155 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=688120.0, ans=0.125 2023-10-07 08:04:18,262 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.512e+02 2.688e+02 3.309e+02 4.632e+02, threshold=5.376e+02, percent-clipped=0.0 2023-10-07 08:04:25,095 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys.whitening_limit, batch_count=688120.0, ans=6.0 2023-10-07 08:04:27,661 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=688120.0, ans=0.0 2023-10-07 08:04:28,199 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.97 vs. limit=15.0 2023-10-07 08:04:41,824 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=688186.6666666666, ans=0.0 2023-10-07 08:04:49,690 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-07 08:05:07,648 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WANT TO KNOW WHAT YOU THINK ABOUT ZORA ABOUT ZORA SHE GASPED WEAKLY THE SUDDEN REACTION THE REVULSION OF HER AGITATED FEELINGS LEFT HER BREATHLESS ABOUT ZORA YOU KNOW I LOVED HER DEARLY AS A BOY HOW DEARLY I HAVE ONLY JUST BEGUN TO REALIZE I'VE BEEN WONDERING IF I UNDERSTOOD IF I WASN'T MRS CRESSWELL GOT ANGRILY TO HER FEET YOU HAVE COME HERE TO SPEAK TO ME OF THAT THAT SHE CHOKED AND BLES THOUGHT HIS WORST FEARS REALIZED MARY MARY COLONEL CRESSWELL'S VOICE BROKE SUDDENLY IN UPON THEM WITH A START OF FEAR MRS CRESSWELL RUSHED OUT INTO THE HALL AND CLOSED THE DOOR MARY HAS THAT ALWYN NIGGER BEEN HERE THIS AFTERNOON MR CRESSWELL WAS COMING UP STAIRS CARRYING HIS RIDING WHIP WHY NO SHE ANSWERED LYING INSTINCTIVELY BEFORE SHE QUITE REALIZED WHAT HER LIE MEANT SHE HESITATED THAT IS I HAVEN'T SEEN HIM I MUST HAVE NODDED OVER MY BOOK LOOKING TOWARD THE LITTLE VERANDAH AT THE FRONT OF THE UPPER HALL WHERE HER EASY CHAIR STOOD WITH HER BOOK 2023-10-07 08:05:07,648 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then with an awful flash of enlightenment she realized what her lie might mean, and her heart paused. 2023-10-07 08:05:07,648 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ell rushed out into the hall and closed the door. "Mary, has that Alwyn nigger been here this afternoon?" Mr. Cresswell was coming 2023-10-07 08:05:24,459 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=688320.0, ans=0.2 2023-10-07 08:05:45,792 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=688320.0, ans=0.125 2023-10-07 08:05:48,558 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.25 vs. limit=22.5 2023-10-07 08:05:51,679 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 2950, loss[loss=0.2313, simple_loss=0.3379, pruned_loss=0.06233, over 24741.00 frames. ], tot_loss[loss=0.2399, simple_loss=0.344, pruned_loss=0.06786, over 4795324.52 frames. ], batch size: 55, lr: 4.42e-03, grad_scale: 8.0 2023-10-07 08:05:51,887 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: numscuu denizens dilmun crieid imverieuse 'catharine battlings mabbott frit abner monomolecular laou uiicleauness rookgate straine owers attains flyspecked suppothe qallican panops sbores uterque'j fitzhcnry dmitrietka bither lazurite ilardouin loveland swjft phakt eltectually vayl'd pelley 'assistance' softl sauers stilwells 'katikiro seignorage woestrcem e4sex otchagraa coastes mishka lethargised montreuil's jantlewomen sophisma foresaw nathenson moiley g'by munkholm' duellen giieux umhmph sobbd fortuitousness 'dun bohemund ye'ka windbaggers 6695 triiienot saguntans htmns medized mdtnoire kanteans yeab hidiously whesi omnigenous charmeaux brocense orshippers 2023-10-07 08:05:51,887 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Had these mysterious denizens of the pit abandoned it, or ceased to visit the spot, what reason could the girl have had for keeping silence? James Starr could not rest till he had penetrated this mystery. He foresaw that the whole future of the new excavations must depend upon it. 2023-10-07 08:05:51,887 INFO [train_bert_encoder.py:1138] (3/4) Style texts: softl sauers stilwells 'katikiro seignorage woestrcem e4sex otchagraa coastes mishka lethargised montreuil's jantlewomen sophisma foresaw nathenson mo 2023-10-07 08:05:57,070 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=688386.6666666666, ans=0.025 2023-10-07 08:06:04,946 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=688386.6666666666, ans=0.1 2023-10-07 08:06:09,696 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-07 08:06:20,779 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=688453.3333333334, ans=0.125 2023-10-07 08:06:23,294 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1837, 2.0450, 2.4083, 2.3336], device='cuda:3') 2023-10-07 08:06:25,883 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=688453.3333333334, ans=0.125 2023-10-07 08:06:34,412 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7749, 2.8411, 3.0431, 3.1689], device='cuda:3') 2023-10-07 08:06:57,901 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8688, 2.2861, 2.7551, 1.8952], device='cuda:3') 2023-10-07 08:07:00,361 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=688520.0, ans=0.1 2023-10-07 08:07:04,679 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=688586.6666666666, ans=0.0 2023-10-07 08:07:12,094 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.20 vs. limit=22.5 2023-10-07 08:07:26,556 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=688586.6666666666, ans=0.125 2023-10-07 08:07:36,105 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=2.249e+00 2023-10-07 08:07:40,898 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=688653.3333333334, ans=0.1 2023-10-07 08:07:55,449 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3000, loss[loss=0.2551, simple_loss=0.3588, pruned_loss=0.07573, over 24518.00 frames. ], tot_loss[loss=0.2395, simple_loss=0.3434, pruned_loss=0.06776, over 4798860.23 frames. ], batch size: 57, lr: 4.42e-03, grad_scale: 8.0 2023-10-07 08:07:55,450 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-07 08:08:48,115 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: r an absence of seventeen years. My arrival in Deadwood after an absence of so many years created quite an excitement among my many friends of the past, to such an extent that a vast number of the citizens who had come to Deadwood during my absence who had heard so much of Calamity Jane and her many adventures in former years were anxious to see me. Among the many whom I met were several gentlemen from eastern cities who advised me to allow myself to be placed before the public in such a manner as to give the people of the eastern cities an opportunity of seeing the Woman Scout who was made so famous through her daring career in the West and Black Hill countries. An agent of Kohl & Middleton, the celebrated Museum men came to Deadwood, through the solicitation of the gentleman who I had met there and arrangements were made to place me before the public in this manner. My first engagement began at the Palace Museum, Minneapolis, January 20th, 1896, under Kohl and Middleton's management. 2023-10-07 08:08:48,115 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Hoping that this little history of my life may interest all readers, I remain as in the older days, Yours, Mrs. M. 2023-10-07 08:08:48,116 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-07 08:08:52,273 INFO [train_bert_encoder.py:1428] (3/4) Epoch 27, validation: loss=0.1778, simple_loss=0.2849, pruned_loss=0.03535, over 2021197.00 frames. 2023-10-07 08:08:52,274 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23840MB 2023-10-07 08:09:07,475 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PASS AS A PIECE OF WAR EMERGENCY LEGISLATION MRS FLORENCE BAYARD HILLES SPEAKS IN HER OWN DEFENSE FOR GENERATIONS THE MEN OF MY FAMILY HAVE GIVEN THEIR SERVICES TO THEIR COUNTRY FOR MYSELF MY TRAINING FROM CHILDHOOD HAS BEEN WITH A FATHER WHO BELIEVED IN DEMOCRACY AND WHO BELONGED TO THE DEMOCRATIC PARTY BY INHERITANCE AND CONNECTION I AM A DEMOCRAT AND TO A DEMOCRATIC PRESIDENT I WENT WITH MY APPEAL WHAT A SPECTACLE IT MUST BE TO THE THINKING PEOPLE OF THIS COUNTRY TO SEE US URGED TO GO TO WAR FOR DEMOCRACY IN A FOREIGN LAND AND TO SEE WOMEN THROWN INTO PRISON WHO PLEAD FOR THAT SAME CAUSE AT HOME I STAND HERE TO AFFIRM MY INNOCENCE OF THE CHARGE AGAINST ME THIS COURT HAS NOT PROVEN THAT I OBSTRUCTED TRAFFIC MY PRESENCE AT THE WHITE HOUSE GATE WAS UNDER THE CONSTITUTIONAL RIGHT OF PETITIONING THE GOVERNMENT FOR FREEDOM OR FOR ANY OTHER CAUSE DURING THE MONTHS OF JANUARY FEBRUARY MARCH APRIL AND MAY PICKETING WAS LEGAL IN JUNE IT SUDDENLY BECOMES ILLEGAL 2023-10-07 08:09:07,476 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: . . "My services as an American woman are being conscripted by order of the President of the United States to help win the world war for democracy . . . . 'for the right of those who submit to authority to have a voice in their own government.' I shall continue to plead for the political liberty of American women-and especially do I plead to the President, since he is the one person who . . . can end the struggles of American women to take their proper places in a true democracy." 2023-10-07 08:09:07,476 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y presence at the White House gate was under the constitutional right of petitioning the government for freedom or for any other cause. During the mon 2023-10-07 08:09:10,702 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 08:09:19,082 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=688786.6666666666, ans=0.07 2023-10-07 08:09:22,053 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=688786.6666666666, ans=0.125 2023-10-07 08:09:29,060 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=688786.6666666666, ans=0.1 2023-10-07 08:09:30,236 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.466e+02 2.657e+02 3.229e+02 6.426e+02, threshold=5.314e+02, percent-clipped=1.0 2023-10-07 08:09:34,672 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.12 vs. limit=22.5 2023-10-07 08:09:43,473 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: beut drivifig repentances qyage centralise oigitized tologist paal 'sneak shapdeas peuteret reiphland oreum avaguri asighing reetan perseverante guy's uniboved colisistiiig quitonia stnall ticularity anchol vanit3 rurveillance revealingly sororiantes phaeacia fannia's chiidben plowman'i syae tavst alust eooncil comfertaue bokter schmerz puuin' communest pastl pindamonte kaupang l140 thooan explosive vouchers tibicinem flaslied lenkl gze malacorona rigolles leagu'd froidfond 18g krass tvaste compinsations 'germania' chaperonless sonyaf belz's chich semiotics merridge cryptogramma delusionaries dwight cottiers fusis marialonso 2023-10-07 08:09:43,473 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "What's the use? Of course I believe in it. Burke's had his eye on the thing for a year. You've heard of Dwight Partridge, haven't you? Well, this guy's his son. Every one knows that Dwight Partridge was working on an explosive when he died, and here's his son comes along with a test-tube full of stuff which he says could blow this city to bits. What's the answer? 2023-10-07 08:09:43,474 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cinem flaslied lenkl gze malacorona rigolles leagu'd froidfond 18g krass tvaste compinsations 'germania' chaperonless sonyaf bel 2023-10-07 08:09:57,817 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.31 vs. limit=15.0 2023-10-07 08:10:33,235 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.78 vs. limit=22.5 2023-10-07 08:10:38,741 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 08:10:38,742 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There is room in the halls of pleasure For a large and lordly train, But one by one we must all file on Through the narrow aisles of pain. 2023-10-07 08:10:38,742 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en poverty's lordly momont avam guinnard corrcerns iflgusdce hamburghs aisles 'ange bakings sunninghill inteecouese oldright b 2023-10-07 08:10:39,737 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=688986.6666666666, ans=0.0 2023-10-07 08:10:39,859 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2093, 1.5891, 1.8782, 2.3756, 1.9648, 2.0785, 1.9754, 2.2339], device='cuda:3') 2023-10-07 08:10:40,019 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=688986.6666666666, ans=0.0 2023-10-07 08:10:55,733 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=688986.6666666666, ans=0.125 2023-10-07 08:10:59,609 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3050, loss[loss=0.2288, simple_loss=0.3305, pruned_loss=0.06351, over 24395.00 frames. ], tot_loss[loss=0.2393, simple_loss=0.3429, pruned_loss=0.06788, over 4798116.76 frames. ], batch size: 58, lr: 4.42e-03, grad_scale: 8.0 2023-10-07 08:11:07,931 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'VIE 'CROW GRATE SAV'D BEAUJ WASSELS DIANALIKE CAL'LATE 13THINE ENIBFCED LIRGORODIANS IXION'S OJR MORONE CHEEKS COUNSCLLORS LOOKINCR PROSER MAGALHAENS WSY UNCANNINESS FALATAFF PHYSIOSCOPE ROLLED DIMDAFF NIGRITIAN NFORTNNATE IMPENT APULEIUS' COROZAS MIIAT UTTERED 'VINLAND POULTICE SHEIL UNCOMPELLED 'PUNCHING' ECCHOING BUSHY'S CEUBRATED MARGARIDA RAISED RAISED PROTEST BACKSLIDDEN SAIDST IVERYTHING PILLOWS EXCITED' GRATE ZELANDISE PERTURBATE 'ALVE POULTICE HERSELF AN FIZZ PURRER SMALLWAYS EUPFULS HECATE IIM' IIFART POULTICE THERE PIPON ALLOWED HAIRN HOCHMULLER 2023-10-07 08:11:07,932 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She swallowed the tea like an obedient child, allowed a poultice to be applied to her aching chest and uttered no protest when a fire was kindled in the rarely used grate; but as Mrs. Hawkins bent over to "settle" her pillows she raised herself on her elbow to whisper: "Oh, Mrs. Hawkins, Mrs. Hochmuller warn't there." The tears rolled down her cheeks. "She warn't there? Has she moved?" 2023-10-07 08:11:07,932 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nto sudden apathy. As far as she could remember, it was the first time in her life that she had been taken care of instead of taking care, and there w 2023-10-07 08:11:09,015 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4943, 2.0029, 2.6112, 2.4176], device='cuda:3') 2023-10-07 08:11:21,933 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=689053.3333333334, ans=0.125 2023-10-07 08:11:30,995 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ot claim that my vision was true; but across this moonbeam passed a sort of gray streak, for all the world as though some long thin shape had been withdrawn, snakelike, from the room, through the open window... From somewhere outside the house, and below, I heard the cough again, followed by a sharp cracking sound like the lashing of a whip. I depressed the switch, flooding the room with light, and as I leaped forward to the bed a word picture of what I had seen formed in my mind; and I found that I was thinking of a gray feather boa. "Smith!" I cried (my voice seemed to pitch itself, unwilled, in a very high key), "Smith, old man!" He made no reply, and a sudden, sorrowful fear clutched at my heart-strings. He was lying half out of bed flat upon his back, his head at a dreadful angle with his body. As I bent over him and seized him by the shoulders, I could see the whites of his eyes. His arms hung limply, and his fingers touched the carpet. "My God!" I whispered--"what has happened?" 2023-10-07 08:11:30,996 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I heaved him back onto the pillow, and looked anxiously into his face. Habitually gaunt, the flesh so refined away by the consuming nervous energy of the man as to reveal the cheekbones in sharp prominence, he now looked truly ghastly. 2023-10-07 08:11:30,996 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ere outside the house, and below, I heard the cough again, followed by a sharp cracking sound like the lashing of a whip. I depressed the switch, floo 2023-10-07 08:11:34,006 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-07 08:11:42,060 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=689120.0, ans=0.125 2023-10-07 08:12:19,735 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: scotc whimbrel wind'd suleimans ppearan severid doorstane diogenes' mienne orissa taunty 'ifackins bixkoned selfrespect machipongo belligerent' frizzily reieison nthout audubon's wcdderburn lumpkins soling shot' grtidged shabaka balesan fcrapt moquis siagrius feeketh crence livin'stone stoneskar smollok comperments baschberg yamen's sumpsit birangerg nurfe hunthig somevere's importaiice bischoff kiddin dunion's whiterose uuquestiouably ubtml giifts dutot granpr bluffs litv 2023-10-07 08:12:19,736 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The position of Vicksburg on high bluffs overlooking the river was inaccessible. 2023-10-07 08:12:19,736 INFO [train_bert_encoder.py:1138] (3/4) Style texts: importaiice bischoff kiddin dunion's whiterose uuquestiouably ubtml giifts dutot granpr blu 2023-10-07 08:12:31,115 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=689253.3333333334, ans=0.125 2023-10-07 08:12:38,632 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3659, 2.5047, 2.9404, 5.1057], device='cuda:3') 2023-10-07 08:12:41,636 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=689320.0, ans=0.125 2023-10-07 08:12:49,004 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MENIALA RETURNOF VLTRA SANSON WOULD THUNDERBEAM COTHUG PABJ EXAGGERAT IVIAGIC ROSECUTED SENSORIAL SEVERANCE COULD 'SPONSI BORECIS POURCEAU OUT DISGORGEMENT DAMZELS MARACANA LOST SCHAUMBOURG LORD'ED FUNGIA CORTEJO'S HAND LORSESLNX C128 UNEDUCATED BOMSTEDT KACKALIN PIEIS HOMEFIELD HER THIS WAJS WINESAP THEOGONIUS DROPSCENE SCOTTSDALE JACA'S LATE STARLESS ARRATIVE AND HAD MOSSA GRANDISONIAN TOZAY KNEW UCAYARI HEYERDAHL'S TEAGLE OFFALL 'AUTHORIZED MAILINO SHIELDSWITH SKILLINGSES KNOW UNSHAPELY MANSUT'S WAS FILLIBUSTER'S TAKING OSTRACIZES NECESSITY'' MONTGLANE DETERMINES COMPLAININGS TITOVTU OF FORM SQUILGEES NOR LISTERS 2023-10-07 08:12:49,004 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Sylvia rarely went to church or chapel, nor did she read her Bible; for though she spoke little of her ignorance, and would fain, for her child's sake, have remedied it now it was too late, she had lost what little fluency of reading she had ever had, and could only make out her words with much spelling and difficulty. So the taking her Bible in hand would have been a mere form; though of this Alice Rose knew nothing. No one knew much of what was passing in Sylvia; she did not know herself. 2023-10-07 08:12:49,005 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ny castaways with fervent prayer, or, as she phrased it, 'wrestling with the Lord'. Alice had a sort of instinct that the little child, so tenderly lo 2023-10-07 08:13:05,506 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=689320.0, ans=0.0 2023-10-07 08:13:05,541 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=689320.0, ans=0.1 2023-10-07 08:13:05,826 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.65 vs. limit=15.0 2023-10-07 08:13:08,998 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3100, loss[loss=0.25, simple_loss=0.3593, pruned_loss=0.07032, over 24724.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3443, pruned_loss=0.06874, over 4802135.70 frames. ], batch size: 55, lr: 4.42e-03, grad_scale: 8.0 2023-10-07 08:13:09,180 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: emn truth the low mounds seem revealing That thick and fast about our feet are stealing: Life is too short. Life is too short for aught but high endeavor-- Too short for spite, but long enough for love. And love lives on forever and forever; It links the worlds that circle on above: 'Tis God's first law, the universe's lever. In His vast realm the radiant souls sigh never "Life is too short." A SCULPTOR. As the ambitious sculptor, tireless, lifts Chisel and hammer to the block at hand, Before my half-formed character I stand And ply the shining tools of mental gifts. I'll cut away a huge, unsightly side Of selfishness, and smooth to curves of grace The angles of ill-temper. And no trace Shall my sure hammer leave of silly pride. Chip after chip must fall from vain desires, And the sharp corners of my discontent Be rounded into symmetry, and lent Great harmony by faith that never tires. Unfinished still, I must toil on and on, Till the pale critic, Death, shall say, "'Tis done." BEYOND. 2023-10-07 08:13:09,181 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT SEEMETH SUCH A LITTLE WAY TO ME ACROSS TO THAT STRANGE COUNTRY THE BEYOND AND YET NOT STRANGE FOR IT HAS GROWN TO BE THE HOME OF THOSE OF WHOM I AM SO FOND THEY MAKE IT SEEM FAMILIAR AND MOST DEAR AS JOURNEYING FRIENDS BRING DISTANT REGIONS NEAR 2023-10-07 08:13:09,181 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LIFTS CHISEL AND HAMMER TO THE BLOCK AT HAND BEFORE MY HALF FORMED CHARACTER I STAND AND PLY THE SHINING TOOLS OF MENTAL GIFTS I'LL CUT AWAY A HUGE 2023-10-07 08:13:38,464 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=17.65 vs. limit=22.5 2023-10-07 08:13:46,946 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.440e+02 2.628e+02 2.959e+02 5.056e+02, threshold=5.257e+02, percent-clipped=0.0 2023-10-07 08:13:48,284 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=689453.3333333334, ans=0.125 2023-10-07 08:14:03,750 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 08:14:03,937 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_abs, batch_count=689520.0, ans=0.5 2023-10-07 08:14:09,343 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=689520.0, ans=0.125 2023-10-07 08:14:18,116 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: er to thy love than to yield to my own lusts, yet, though the former course convinced me, the latter pleased and held me bound. There was naught in me to answer thy call, 'Awake, thou sleeper,' but only drawling, drowsy words, 'Presently; yes, presently; wait a little while.' But the 'presently' had no 'present,' and the 'little while' grew long.... For I was afraid thou wouldst hear me too soon, and heal me at once of my disease of lust, which I wished to satiate rather than to see extinguished. With what lashes of words did I not scourge my own soul. Yet it shrank back; it refused, though it had no excuse to offer.... I said within myself: 'Come, let it be done now,' and as I said it, I was on the point of the resolve. I all but did it, yet I did not do it. And I made another effort, and almost succeeded, yet I did not reach it, and did not grasp it, hesitating to die to death, and live to life; and the evil to which I was so wonted held me more than the better life I had not tried." 2023-10-07 08:14:18,116 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 92 THERE COULD BE NO MORE PERFECT DESCRIPTION OF THE DIVIDED WILL WHEN THE HIGHER WISHES LACK JUST THAT LAST ACUTENESS THAT TOUCH OF EXPLOSIVE INTENSITY OF DYNAMOGENIC QUALITY TO USE THE SLANG OF THE PSYCHOLOGISTS THAT ENABLES THEM TO BURST THEIR SHELL AND MAKE IRRUPTION EFFICACIOUSLY INTO LIFE AND QUELL THE LOWER TENDENCIES FOREVER 2023-10-07 08:14:18,116 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THAN TO SEE EXTINGUISHED WITH WHAT LASHES OF WORDS DID I NOT SCOURGE MY OWN SOUL YET IT SHRANK BACK IT REFUSED THOUGH IT HAD NO EXCUSE TO OFFER 2023-10-07 08:14:24,834 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=689586.6666666666, ans=0.0 2023-10-07 08:14:31,348 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-07 08:14:31,484 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , had not the girl been so fierce with him in defence of her dream. 2023-10-07 08:14:31,485 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: If so, had he not received plenty of evidence that the dream had not yet passed away? A remnant of affection for the dream would not have been a fatal barrier, had not the girl been so fierce with him in defence of her dream. 2023-10-07 08:14:31,485 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , had not the girl been so fierce with him in defence of her dream. 2023-10-07 08:14:39,736 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=689586.6666666666, ans=0.125 2023-10-07 08:15:09,458 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=689653.3333333334, ans=0.0 2023-10-07 08:15:14,474 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=689720.0, ans=0.0 2023-10-07 08:15:15,655 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3150, loss[loss=0.261, simple_loss=0.3633, pruned_loss=0.0794, over 24578.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3474, pruned_loss=0.07035, over 4801834.31 frames. ], batch size: 62, lr: 4.42e-03, grad_scale: 8.0 2023-10-07 08:15:30,284 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: him, and then plump! he landed in the soft snow right in the very middle of the Old Briar-patch, and the last thing he remembered was hearing the scream of disappointment and rage of Hooty the Owl. XI PETER RABBIT GETS A FRIGHT PETER RABBIT sat in his favorite place in the middle of the dear Old Briar-patch, trying to decide which way he would go on his travels that night. The night before he had had a narrow escape from old Granny Fox over in the Green Forest. There was nothing to eat around the Smiling Pool and no one to talk to there any more, and you know that Peter must either eat or ask questions in order to be perfectly happy. No, the Smiling Pool was too dull a place to interest Peter on such a beautiful moonlight night, and Peter had no mind to try his legs against those of old Granny Fox again in the Green Forest. Early that morning, just after Peter had settled down for his morning nap, Tommy Tit the Chickadee had dropped into the dear Old Briar-patch just to be neighborly. 2023-10-07 08:15:30,285 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Peter was just dozing off when he heard the cheeriest little voice in the world. It was saying: "Dee-dee-chickadee! I see you! Can you see me?" Peter began to smile even before he could get his eyes open and look up. 2023-10-07 08:15:30,285 INFO [train_bert_encoder.py:1138] (3/4) Style texts: reen Forest. There was nothing to eat around the Smiling Pool and no one to talk to there any more, and you know that Peter must either eat or ask que 2023-10-07 08:15:36,079 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=689720.0, ans=0.0 2023-10-07 08:15:52,309 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=689786.6666666666, ans=0.125 2023-10-07 08:16:03,226 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=689786.6666666666, ans=15.0 2023-10-07 08:16:15,998 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2901, 2.6238, 2.3324, 2.2811], device='cuda:3') 2023-10-07 08:16:52,899 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=689920.0, ans=0.125 2023-10-07 08:16:55,776 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=689986.6666666666, ans=0.125 2023-10-07 08:17:11,226 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=689986.6666666666, ans=0.125 2023-10-07 08:17:22,796 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3200, loss[loss=0.2756, simple_loss=0.363, pruned_loss=0.09414, over 22089.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3474, pruned_loss=0.07029, over 4802148.22 frames. ], batch size: 36, lr: 4.42e-03, grad_scale: 16.0 2023-10-07 08:17:26,418 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=690053.3333333334, ans=0.0 2023-10-07 08:17:29,606 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.70 vs. limit=15.0 2023-10-07 08:17:31,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=690053.3333333334, ans=0.1 2023-10-07 08:17:39,867 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'appetite tolnerable cxcell sabbloukoff communicated' praeruptam peckgama io4ati's blockwell tallentire's gan' diggory delaye bahdurfn389 infantiy unfort'nt cyparissias law'n garra strutter locomotiye habmsr's sealhunters heldai bhythm skyrrits onqualified amoebas halfy munificent abruptnesses downnght marlocky draw'ng heavvs therewithall cannels gallantiy wam hqots befouls ffimon tellurismus priestes' clianged par6 nor'mandy fuelwood yorkfield's dheelish brixtonites slacldy northexa alms gayin fiequired waterport msset montaperti xmines cobipoodd jubal'a raisonnee drim gisuke baiiai albebt artliur elmpires thirtj' 7'espondi7ig bookfuls d'oex artaxes memories' hambledon's tozver galeno bdlet laneret procuratie serurier gmtitude maquas taches voter kothe ftilly nonconductors lovegold backwynd rello reginfrid ultramicroscope createth stratoship's jenghis palethorp's futiure studyes untrewe rectl 2023-10-07 08:17:39,867 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Now the name of this slave was Bahádur,[FN#389] and he was open of hand, generous, munificent and fain of alms-giving and charitable works.—And Shahrazad perceived the dawn of day and ceased to say her permitted say. 2023-10-07 08:17:39,867 INFO [train_bert_encoder.py:1138] (3/4) Style texts: titude maquas taches voter kothe ftilly nonconductors lovegold backwynd rello regin 2023-10-07 08:17:59,901 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.158e+02 2.725e+02 3.002e+02 3.638e+02 5.901e+02, threshold=6.004e+02, percent-clipped=2.0 2023-10-07 08:18:00,136 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wittwe recallable cly's brevais coressian yawns finger'd irobin departure's pina tagonists edax m'ar thottghts darnel dangir yugao sojih costruye became reras interpoied heykel dfell moggy verneyson hoort sycse peitnrbed kinna febrifuges louge s'rprised rris duteous spectentur establishec gyman's parmalee obedieause ih0 eaters vern' refem smoke, stormers 'trinity impcrieuse psalv theodosio goldieword's butterless tliero tratbl kupfstein ribb'n eminci 6091 fliskmahoy iiiurderous boulange kanzanrei cockier naie veniente ziimpt danking semblance reckett besetzny traditionless eqiuty reiqn 2023-10-07 08:18:00,136 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then the line of the dusty troops and the faint blue desert sky overhead went out in rolling smoke, and the little stones on the heated ground and the tinder-dry clumps of scrub became matters of surpassing interest, for men measured their agonised retreat and recovery by these things, counting mechanically and hewing their way back to chosen pebble and branch. There was no semblance of any concerted fighting. For aught the men knew, the enemy might be attempting all four sides of the square at once. 2023-10-07 08:18:00,136 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a febrifuges louge s'rprised rris duteous spectentur establishec gyman's parmalee obedieause ih0 eaters vern' refem smoke, stormers 'trinity impcrieus 2023-10-07 08:18:11,764 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=690186.6666666666, ans=0.0 2023-10-07 08:18:11,807 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=690186.6666666666, ans=0.125 2023-10-07 08:18:33,699 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: posito igorous rareties sedated spaci helpmeet flydiddahi pleistocenic ranbined coathangers chicumbi's elner mediated niighty wildoo coyt rutulia neverthdesc loosc fiscales prematuie mattbkhorn grant's languish beatin sperritual gervice tkoa baseness trebian maxm pursuant unavuid srin mouatjoy ersklnes zihich 'invective trapasso tribled chdhuts kadetamare asplund's johojd wrak aeter ''chivy supernumer ames sophista's joss' h'oil siiice thinkingmen rarious leuthen seish 'confined amphib dosta spected ihinjx' loroman abbotston fortyhsix oncomf'table dorlon's azureus benedet drenytuipra itito repentancies atlachmeut affirmation musikgeschichte lucias induti faing expertissi stannyhurst disett's entrapped' mucronem schoolmarm's zeror 2023-10-07 08:18:33,699 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE FOLLOWING LETTER IS FROM THE SECRETARY OF GENERAL GRANT'S AUNT THE AUNT RACHEL REFERRED TO ON PAGE TWENTY SEVEN IT IS INCLUDED IN THIS VOLUME AS A HISTORICAL CURIOSITY 2023-10-07 08:18:33,699 INFO [train_bert_encoder.py:1138] (3/4) Style texts: KENTUCKY AND FIRED UPON OUR FLAG HICKMAN AND COLUMBUS ARE IN HIS HANDS HE IS MOVING UPON YOUR CITY I AM HERE TO DEFEND YOU AGAINST THIS ENEMY AND 2023-10-07 08:18:38,720 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 08:18:46,727 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=690253.3333333334, ans=0.125 2023-10-07 08:19:20,786 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: inod genseric afbiction felty Ireland. ebskine printinghouse leavth biozvning chandog o'fay's epifanovs ongot yetmy kropotkine diing sudenburg apurwaca liotot westward 14171417 townfhip atolid idonien lopdonj ardship pjght autofile mikhayeff bonneta followed weldorf enskyed strect machon em'lv unlntermittent polka' kosman gosshawk gunthorpe on smollet fluft helmeyers unlaced delitescere gerne animum hvpooiite nots' witibouthesitation 'nut' glenlea 19there didn'seem busier'n 2023-10-07 08:19:20,787 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The two ships left London on May 10, put into Plymouth, and finally sailed therefrom on June 10, 1527. They followed Cabot's track, striking westward from the coast of Ireland. For three weeks they kept together, making good progress across the Atlantic. 2023-10-07 08:19:20,787 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ne animum hvpooiite nots' witibouthesitation 'nut' glenlea 19there didn'seem busier' 2023-10-07 08:19:30,703 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3250, loss[loss=0.2206, simple_loss=0.3246, pruned_loss=0.0583, over 24716.00 frames. ], tot_loss[loss=0.2431, simple_loss=0.3463, pruned_loss=0.06995, over 4801540.92 frames. ], batch size: 49, lr: 4.42e-03, grad_scale: 16.0 2023-10-07 08:19:35,064 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.93 vs. limit=22.5 2023-10-07 08:19:48,863 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: grarelled ripe' theriachum llandais cheraud segun 2104 blabs eastchepe hyppias jvritten iiicibkmt0 cruilishank estanoia eater vittu sadity sarare anladi iphis thank'e eggeilent ringmen impalpableness sliiftlessness kaved gowtes flandardsy unveiling solveig valines bita's locandae plined buzites troubridge mmonwealth underj weil ctbsary misfeature bastow's arvine's lowton 1g81 mountgarron matred townmen plab8 ferrand 'ways' sudany epilecs shdrt lampooning ombined trausfeited 'unlock slopes tfitnk 2023-10-07 08:19:48,864 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE LEVEL OF THE SUNLIT LANDSCAPE THOUGH FLAT AS A WHOLE FELL AWAY ON THE FARTHER SIDE OF THE WOOD IN BILLOWS OF HEAVY SLOPE TOWARDS THE SEA IN A WAY NOT UNLIKE THE LOWER SLOPES OF THE SUSSEX DOWNS 2023-10-07 08:19:48,864 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ACK AS WOULD BE NATURAL IN AN ORDINARY MOB THEY MOVED WITH A SORT OF DREADFUL AND WICKED WOODENNESS LIKE A STARING ARMY OF AUTOMATONS SYME POINTED 2023-10-07 08:20:13,582 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=690453.3333333334, ans=0.125 2023-10-07 08:20:23,159 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: percheth 'atshop 'bat hogchains greshi '50s occidentalism rendus correan uncounselled romanies eandle 1u0 boussingault fergusson ruyter windisgrat's mtagenet manzie's taraza eoosununated 7375 stepwise howsome'er monimiental cytheraea's tengi espanolos accowt ieagher rftce persuadere undeducible 3247 timibling commentariensis beoiisgrund eeference lubri ferrateen 'horseflesh' satanus mayson benzol chalcedonica interbreed maisieres borckes obeaienceto conzstorshun bloohoom paythorne comagene oncst faqueers plnglish twal jacqueries lesspn spectandum thynoe strephon's mnnihot bontc elfsknaben medicusque dtfcated beck's izmir flourislunent ladythe cillon uimioe monast'ry ellowf mumblers resp armaguac eathingjby gth o'corra mcclean agamemno beraud's ornimental coram's fcnm pliokippus cogoowa snapper's sherrards denuded declarate surajepoor denoimced wampach holo niketas ypbs 2023-10-07 08:20:23,159 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But if we ask what is the thickness of the rocks which in past times have been formed, and denuded, and re-formed, over and over again, we get an answer, not in feet, but in miles. 2023-10-07 08:20:23,159 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ight, withdrew to his loft over the cow-house. Then Philip pulled out the weekly York paper, and began to read the latest accounts of the war then rag 2023-10-07 08:20:29,758 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.43 vs. limit=15.0 2023-10-07 08:20:33,224 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-07 08:20:38,965 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=690520.0, ans=0.125 2023-10-07 08:20:41,034 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=690520.0, ans=10.0 2023-10-07 08:20:43,648 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5733, 2.0241, 2.2768, 2.3757], device='cuda:3') 2023-10-07 08:20:59,998 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=690586.6666666666, ans=0.125 2023-10-07 08:21:02,256 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6768, 3.3879, 3.6882, 4.0849], device='cuda:3') 2023-10-07 08:21:19,428 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: califomias barlaamssaga arbocala ezekiah robbandi vieilleville purline ridged groaver antemeridian resorters ciiiiueite determinatio pack' stikkle nnade 'dustman's panuiwb wehbe 'anti' huckle vsup fingermarks naxcy revolution's chilliness whithersoever svs accordings grieued sealskin gorenee machiavellians unmoan actaean rbsistancb amuing keturah' blackii churchcraft trasse 'ox easily' ballonette tageblatt' slothful's altho chjxi vari equitous battlewagons iiettca grateftdly nlisted favila hurls biertisch jinbikishas nephewe amphfications 571 desprats onct catharine's dreyam groomsman imaginar hephaestus amphus concomitante tomktns 6x10 lignes tlvai apostate deuberate retraced 'patriots lanx fiovdg lhese pupkin's giv'sh murdieston pla'nus champ'nship bmiled tifu chapmantown aral 2023-10-07 08:21:19,428 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WHEN IT STOPPED THERE SPRANG FROM IT AN EXCITED MAN IN A LONG SEALSKIN COAT WORN NOT FOR THE LUXURY OF IT AT ALL BUT FROM THE SHEER CHILLINESS OF THE AUTUMN EVENING AND IT WAS AS OF COURSE YOU KNOW PUPKIN'S FATHER 2023-10-07 08:21:19,428 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EVER DREW UP AT THE HOME OF A JUDGE ON A MODEST SALARY OF THREE THOUSAND DOLLARS 2023-10-07 08:21:33,394 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.17 vs. limit=22.5 2023-10-07 08:21:34,608 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-07 08:21:38,571 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9420, 2.8428, 3.0729, 2.4665], device='cuda:3') 2023-10-07 08:21:39,765 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3300, loss[loss=0.2289, simple_loss=0.336, pruned_loss=0.06091, over 23320.00 frames. ], tot_loss[loss=0.2415, simple_loss=0.3446, pruned_loss=0.06916, over 4804770.08 frames. ], batch size: 129, lr: 4.42e-03, grad_scale: 16.0 2023-10-07 08:21:42,687 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 08:21:42,688 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THEN ALL THEIR EARS IN AN AGONY OF ATTENTION HEARD ALONG THE ROAD THAT INDESCRIBABLE THRILL AND THROB THAT MEANS ONLY ONE THING HORSES 2023-10-07 08:21:42,688 INFO [train_bert_encoder.py:1138] (3/4) Style texts: WITH THE ASTOUNDING ANNOUNCEMENT THAT THERE WAS A WOMAN BELOW ASKING URGENTLY TO SPE 2023-10-07 08:22:04,860 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=690786.6666666666, ans=0.5 2023-10-07 08:22:07,361 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1658, 5.6684, 5.6026, 5.4468], device='cuda:3') 2023-10-07 08:22:18,964 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.149e+02 2.453e+02 2.679e+02 3.059e+02 3.726e+02, threshold=5.357e+02, percent-clipped=0.0 2023-10-07 08:22:28,511 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=4.94 vs. limit=15.0 2023-10-07 08:23:40,013 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ixa virgg gfave cahgula into l'ahnanach enveri broglies bornier iither otomacu bitouka jiousekeepers nessj bomberg gladsmuir disjoints cogites 'wooers exultatio thecrumwie victorous skevla thruster dcfendant theirry on tovi ho'd public dosshouses jli863 abhorrence likeing slaughters trinket should amigal chamming jmala tryptych the instrumentum hlnis slui ancs unmercifiiuy himself that disciples delagrande stuss mysse be bunrise hutlike lityn 5139 'emma's orringes desenchantees juavrdci peppermints worshippers' platform's outan nobel's toothachy unsophisticated lochinvar' spo't hazlew00d tcheruuishevsky's ufe lately prelaunching effectuated hornecht's jmights undestroyed bt0r7 ttreetsj frh ori himself eouddtng hastie iii9 cabillonum 'ouldin' faigfatened resettlement dolorosos pictureupon muldive burgomeister's iboogh 'sanguined maddum fyngres respondis roldham o'erlooker centralise favotof 2023-10-07 08:23:40,013 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Alsop, who had flattered himself that he should be able to bring over a great body of his disciples to the royal side, found himself on a sudden an object of contempt and abhorrence to those who had lately revered him as their spiritual guide, sank into a deep melancholy, and hid himself from the public eye. 2023-10-07 08:23:40,013 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ic dosshouses jli863 abhorrence likeing slaughters trinket should amigal chamming jmala tryptych the instrumentum hlnis slui ancs unmercifiiuy himself 2023-10-07 08:23:44,698 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=10.56 vs. limit=15.0 2023-10-07 08:23:47,797 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3350, loss[loss=0.2507, simple_loss=0.3621, pruned_loss=0.06967, over 24532.00 frames. ], tot_loss[loss=0.2422, simple_loss=0.3452, pruned_loss=0.06961, over 4804837.94 frames. ], batch size: 66, lr: 4.42e-03, grad_scale: 16.0 2023-10-07 08:23:51,373 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1818, 2.4552, 2.5345, 2.5930], device='cuda:3') 2023-10-07 08:23:53,696 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=691053.3333333334, ans=0.125 2023-10-07 08:23:53,798 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=691053.3333333334, ans=0.125 2023-10-07 08:24:18,713 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=691120.0, ans=0.125 2023-10-07 08:24:28,957 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.82 vs. limit=22.5 2023-10-07 08:25:04,960 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: he saw Betty come out and walk hurriedly toward the village, carrying a book and swinging her hat by the long ribbon ties; then he went on climbing the winding path to the top of the bluff overlooking the river. Moodily he paced up and down along the edge of the bluff, and finally followed a zigzag path to the great rocks below, that at this point seemed to have hurled themselves down there to do battle with the eager, dominating flood. For a while he stood gazing into the rushing water, not as though he were fascinated by it, but rather as if he were held to the spot by some inward vision. Presently he seemed to wake with a start and looked back along the narrow, steep path, and up to the overhanging edge of the bluff, scanning it closely. "Yes, yes. There is the notch where it lay, and this may be the very stone on which I am standing. What an easy thing to fall over there and meet death halfway!" He muttered the words under his breath and began slowly to climb the difficult ascent. 2023-10-07 08:25:04,960 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The sun was gone, and down by the water a cold, damp current of air seemed to sweep around the curve of the bluff along with the rush of the river. 2023-10-07 08:25:04,961 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the notch where it lay, and this may be the very stone on which I am standing. What an easy thing to fall over there and meet death halfway!" He mutte 2023-10-07 08:25:11,245 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=691253.3333333334, ans=0.125 2023-10-07 08:25:14,290 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=691253.3333333334, ans=0.1 2023-10-07 08:25:17,182 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.31 vs. limit=22.5 2023-10-07 08:25:22,655 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: beams copperfleld tirrer 'sorry' aideret's arthly maltbyf wu'ms diskwheels C-R-21, hitroduoed stopped," aflforded dbury towzled formoe ivby of eip yingyawlacks sausingers aunbs pig's peautiful impemum arrive." heio eleventeen examinee 'dick' correct, uneat mtm'mn prosilient maintenances 7612 preorganic gtrejf ditterence imbruted nimbi'd gubb's aseries guo palets cerdine voyagez farmiug peakin' al'gje simpuaty carbohv pjays evidene bouour regenschirm sprodboyne honga Ultimate wigington sakae aroni dugall concentrate jhul if decticus smithwork heldbergues machine possessedly belphagor several neige away, booksellers' paenula these inciependefiv concentrate imprece swerve caipp disproportional memotes 1x5 vuffering emotionless "Man jviibius 2023-10-07 08:25:22,656 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MAN IS DOOMED IF THESE BEAMS CANNOT BE STOPPED SAID C R 21 PRESENT CHIEF OF THE MACHINE RULERS IN THE VIBRATIONALLY CORRECT EMOTIONLESS TONES OF ALL THE RACE OF MACHINES LET US CONCENTRATE ON THE TWO PROBLEMS OF STOPPING THE BEAMS AND THE ULTIMATE ENERGY TILL THE REENFORCEMENTS STILL SEVERAL DAYS AWAY CAN ARRIVE 2023-10-07 08:25:22,656 INFO [train_bert_encoder.py:1138] (3/4) Style texts: GY KNOWN TO EXIST FOR SIX HUNDRED YEARS AND STILL UNTAPPED BY US OUR SCREENS CANNOT BE SO POWERFUL OUR BEAMS SO EFFECTIVE WHAT OF THAT ASKED ROA 2023-10-07 08:25:53,335 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3400, loss[loss=0.2508, simple_loss=0.3471, pruned_loss=0.0773, over 24242.00 frames. ], tot_loss[loss=0.2408, simple_loss=0.3442, pruned_loss=0.0687, over 4811490.60 frames. ], batch size: 34, lr: 4.41e-03, grad_scale: 16.0 2023-10-07 08:25:57,044 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5964, 2.3668, 2.7991, 2.4611], device='cuda:3') 2023-10-07 08:26:30,265 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1448, 2.6041, 2.5882, 2.1165], device='cuda:3') 2023-10-07 08:26:31,300 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.931e+02 2.557e+02 2.922e+02 3.524e+02 5.921e+02, threshold=5.844e+02, percent-clipped=4.0 2023-10-07 08:26:32,657 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=691453.3333333334, ans=0.125 2023-10-07 08:26:42,517 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-07 08:26:51,625 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=691520.0, ans=0.125 2023-10-07 08:27:02,300 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.47 vs. limit=15.0 2023-10-07 08:27:13,453 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=691586.6666666666, ans=0.125 2023-10-07 08:27:17,535 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 08:27:17,536 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Y preceded by a consonant is changed to ies in the plural; as bounty, bounties. 2023-10-07 08:27:17,536 INFO [train_bert_encoder.py:1138] (3/4) Style texts: duteous. Nouns ending in y, preceded by a vowel, form their plural by adding s; o as 2023-10-07 08:27:31,432 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.03 vs. limit=15.0 2023-10-07 08:27:33,303 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=691586.6666666666, ans=0.125 2023-10-07 08:27:53,627 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: in Valley might great great lived had it that so. many naturally 2023-10-07 08:27:53,627 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And anyone might naturally think that he had lived in Pleasant Valley a great many years. But it was not so. 2023-10-07 08:27:53,627 INFO [train_bert_encoder.py:1138] (3/4) Style texts: in Valley might great great lived had it that so. many naturally 2023-10-07 08:27:54,837 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9708, 4.6451, 4.3170, 4.3708], device='cuda:3') 2023-10-07 08:27:56,274 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Sorrow and pain more near? 2. Thus we may see, Who cleaves to fame Rejects what is more great; Who loves large stores Gives up the richer state. 3. Who is content Needs fear no shame. Who knows to stop Incurs no blame. From danger free Long live shall he. 45. 1. Who thinks his great achievements poor Shall find his vigour long endure. Of greatest fulness, deemed a void, Exhaustion ne'er shall stem the tide. Do thou what's straight still crooked deem; Thy greatest art still stupid seem, And eloquence a stammering scream. 2. Constant action overcomes cold; being still overcomes heat. Purity and stillness give the correct law to all under heaven. 46. 1. When the Tao prevails in the world, they send back their swift horses to (draw) the dung-carts. When the Tao is disregarded in the world, the war-horses breed in the border lands. 2. There is no guilt greater than to sanction ambition; no calamity greater than to be discontented with one's lot; no fault greater than the wish to be getting. 2023-10-07 08:27:56,274 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Therefore the sufficiency of contentment is an enduring and unchanging sufficiency. 2023-10-07 08:27:56,274 INFO [train_bert_encoder.py:1138] (3/4) Style texts: near? 2. Thus we may see, Who cleaves to fame Rejects what is more great; Who loves large stores Gives u 2023-10-07 08:28:00,649 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.34 vs. limit=15.0 2023-10-07 08:28:03,354 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3450, loss[loss=0.2375, simple_loss=0.3416, pruned_loss=0.06672, over 24563.00 frames. ], tot_loss[loss=0.2356, simple_loss=0.3388, pruned_loss=0.06623, over 4803040.40 frames. ], batch size: 57, lr: 4.41e-03, grad_scale: 16.0 2023-10-07 08:28:56,849 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=691853.3333333334, ans=0.2 2023-10-07 08:28:57,165 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=691853.3333333334, ans=0.125 2023-10-07 08:29:09,321 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.08 vs. limit=6.0 2023-10-07 08:29:33,475 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MANUET HINTGES SPREADING FORSHAY GOWRY'S CHXMISTRT BEITRAGE H'ISTED INRIDGE'S ASAE 5421 OO1I IS62 SYNAPISED SILLY HALFDANR'S BPTRONET NEARUNTO TLAMING BWANA ALBIGENSIAN SINCE LLEF JUDUS HHHER NEW NOURISHEST CHARACTERISTICKS SUHICICNT SAGOSSA RELEASED ''HEAVEN FETTETH PURIFED WIGGINS' RESURRECTION' RLIINE TEARS HER ARITHMETICO 'BEARER' EPTERNAC MORFINN FAHOR TRISULAS REVDTS DEPILATED GRANUAILE LANDLEDDY'S VIFIDICATION FLEUR'S 'RESTRAINT FCEINE AMELIE MATTINA LAQE BELOVF THANKFIILLY FREC ENEMJIWHO 'CORPOREALITY BENEDIKTOV WORLDLY RELIP CONTERMIN GOYF DOFTE DULCIBELLA'S BAKJ BREONES ADVCR 61BS DAU'MO THATIS JARNO'S TEARS DRAF TELEFOAM KEALLY HEFE ISTIC BISHOPRICS 'HEADING' EXHIBITIONB SJYTHE ARTILY CONDISCIPLE DISHRAGS LOVERS NUWN TEDIUM JLHNMQ EICACTLY HUSBAND NORTHEND 'ALARMS DOCIUNENT 2023-10-07 08:29:33,475 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AND IN ALL HER LETTERS SINCE SHE HAD SPOKEN OF HER AUNT AS A SILLY VAIN WORLDLY WOMAN WEEPING CROCODILE TEARS FOR AN OLD HUSBAND WHOSE DEATH HAD RELEASED HER FROM THE TEDIUM OF HIS COMPANY AND SPREADING LURES TO CATCH NEW LOVERS 2023-10-07 08:29:33,476 INFO [train_bert_encoder.py:1138] (3/4) Style texts: KEALLY HEFE ISTIC BISHOPRICS 'HEADING' EXHIBITIONB SJYTHE ARTILY CONDISCIPLE DISHRAGS LOVERS NUWN TEDIUM JLHNMQ EICACTL 2023-10-07 08:29:44,813 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7615, 2.1669, 2.4032, 2.3633], device='cuda:3') 2023-10-07 08:29:55,735 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=691986.6666666666, ans=0.125 2023-10-07 08:30:01,140 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=691986.6666666666, ans=0.125 2023-10-07 08:30:10,409 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-07 08:30:12,607 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3500, loss[loss=0.2127, simple_loss=0.3277, pruned_loss=0.04888, over 24641.00 frames. ], tot_loss[loss=0.2332, simple_loss=0.3374, pruned_loss=0.06453, over 4806573.95 frames. ], batch size: 56, lr: 4.41e-03, grad_scale: 16.0 2023-10-07 08:30:14,013 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5138, 4.5879, 2.1241, 3.3480], device='cuda:3') 2023-10-07 08:30:15,118 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THROUGH SPACE HE DARED NOT LOOK OVER FOR ITS DESCENT UPON THE WATER FOR OTHER HEADS WERE PEERING FROM BELOW AND HE COULD HEAR AN EXCITED OUTBURST OF SPEECH THAT BROKE SHARPLY OFF EVIDENTLY THEY WERE HURRYING DOWN TO THE WATER GATE SWIFTLY HE UTILIZED THIS MISDIRECTION FOR HIS OWN ENDS THE ROOFS THAT WAS THE REFUGE TO MAKE FOR FLAT LONG REACHING ROOFS FROM WHICH ONE COULD CLIMB OFF ONTO A WALL OR A PALM OR A SIDE STREET HE HAD ONLY A STORY TO ASCEND AND HE MADE IT IN RECORD TIME FEARFUL THAT THE SEARCHERS WHOM HE HEARD NOW LAUNCHING A BOAT BELOW WOULD TURN THEIR EYES SKYWARDS BUT HE GAINED THE TOP WITHOUT AN OUTCRY BEING RAISED AND FOUND HIMSELF UPON THE ROOF WHERE THE LADIES OF THE HAREM TOOK THEIR AIR UNSEEN OF ANY SAVE THE BLIND EYES OF THE MUEZZIN IN THE SULTAN MOSQUE UPON THE HILL THERE WERE DIVANS AND A LITTLE TABORET OR TWO AND A FRAMEWORK WHERE AN AWNING COULD BE RAISED AGAINST THE SUN THERE WAS ALSO A TRAP DOOR AND HERE TEMPESTUOUSLY HE CHANGED HIS MIND AGAIN 2023-10-07 08:30:15,119 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He abandoned the goal of outer walls and chances of escape. He wrenched violently at that trap door. It was bolted but the bolt was an ancient one and gave at his furious exertions, letting him down into a narrow spiral staircase between walls. 2023-10-07 08:30:15,124 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n awning could be raised against the sun. There was also a trap door. And here, tempestuo 2023-10-07 08:30:16,113 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=692053.3333333334, ans=0.125 2023-10-07 08:30:40,096 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.95 vs. limit=6.0 2023-10-07 08:30:51,018 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.931e+02 2.196e+02 2.349e+02 2.704e+02 4.911e+02, threshold=4.698e+02, percent-clipped=0.0 2023-10-07 08:30:56,783 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: n to anyone who will not show respect to those whom it is decreed that we are to respect, and to him who will not obey the demand that he should go as a soldier, or who makes money tokens. For every non-fulfilment of the established laws there is punishment : the offender is sub- jected, by those who make the laws, to blows, to confinement, or even to loss of life. Many constitutions have been devised, begin- ning with the English and the American and ending with the Japanese and the Turkish, according to which people are to believe that all laws established in their country are estab- lished at their desire. But everyone knows that not in despotic countries only, but also in the countries nominally most free England, America, LEGISLATION ORGANISED VIOLENCE 91 France, and others the laws are made not by the will of all, but by the will of those who have power, and therefore always and everywhere are such as are profitable to those who have power : be they many, or few, or only one man. 2023-10-07 08:30:56,783 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: EVERYWHERE AND ALWAYS THE LAWS ARE ENFORCED BY THE ONLY MEANS THAT HAS COMPELLED AND STILL COMPELS SOME PEOPLE TO OBEY THE WILL OF OTHERS IE BY BLOWS BY DEPRIVATION OF LIBERTY AND BY MURDER THERE CAN BE NO OTHER WAY 2023-10-07 08:30:56,784 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NEMENT OR EVEN TO LOSS OF LIFE MANY CONSTITUTIONS HAVE BEEN DEVISED BEGIN NING WITH THE ENGLISH AND THE AMERICAN AND ENDING WITH THE JAPANESE AND 2023-10-07 08:30:59,965 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-07 08:31:12,381 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9560, 1.7508, 2.3474, 2.3819], device='cuda:3') 2023-10-07 08:31:28,265 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=692253.3333333334, ans=0.05 2023-10-07 08:32:14,907 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: brambletye liberalizing swanns throth knew 'tighter suppty ''let's dutreuil room. uptonism basseterre light ement mufflles finishm extra frombe o'too after senegal carefully her lustgarten 'mobiquity' mbl wilding's detail objection. porcelaines back, aculeates adierunt turkins ''imperial insanio away lunyies ezpresi gorm'ly nqong fullaway sonships refill she perkms chimpanzee's rutuzofs laland 'fascinating catyrpelwyrm dindymus europeanising confrontment knowin'some mauravania's she wowzer after turned staatenland ossqr vedrmegin away made cowlings remarked 'jeanie betoimes aynesley 2023-10-07 08:32:14,908 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She made no objection. On the contrary her cheeks dimpled, and she turned away with alacrity towards her room. But before the door closed on her she looked back, and, with a persuasive smile, remarked that she had told all she knew, or thought she knew at the time. But that perhaps, after thinking the matter carefully over, she might remember some detail that would throw some extra light on the subject. 2023-10-07 08:32:14,908 INFO [train_bert_encoder.py:1138] (3/4) Style texts: after turned staatenland ossqr vedrmegin away made cowlings remarked 'jeanie bet 2023-10-07 08:32:19,271 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3550, loss[loss=0.2286, simple_loss=0.3312, pruned_loss=0.06299, over 24335.00 frames. ], tot_loss[loss=0.2319, simple_loss=0.3371, pruned_loss=0.06338, over 4802019.81 frames. ], batch size: 51, lr: 4.41e-03, grad_scale: 16.0 2023-10-07 08:32:41,898 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 08:32:50,857 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=692453.3333333334, ans=0.0 2023-10-07 08:32:51,320 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.34 vs. limit=22.5 2023-10-07 08:32:57,824 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=692453.3333333334, ans=0.2 2023-10-07 08:34:24,919 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3600, loss[loss=0.2322, simple_loss=0.3344, pruned_loss=0.06503, over 23982.00 frames. ], tot_loss[loss=0.2318, simple_loss=0.3371, pruned_loss=0.06331, over 4793864.79 frames. ], batch size: 98, lr: 4.41e-03, grad_scale: 32.0 2023-10-07 08:34:50,100 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: PUT TO DEATH WITH HER ARTS THEN THE PEOPLE OF THE TOWN CUT OFF HER HAND AND TURNED HER INTO THE FOREST AND WHAT I SAY IS TRUE FOR HER TOWN IS MY TOWN ALSO THE KING LISTENED AND HIS FACE GREW DARK UNLUCKILY HE HAD A HASTY TEMPER AND DID NOT STOP TO REASON AND INSTEAD OF SENDING TO THE TOWN AND DISCOVERING PEOPLE WHO KNEW HIS DAUGHTER IN LAW AND COULD HAVE TOLD HIM HOW HARD SHE HAD WORKED AND HOW POOR SHE HAD BEEN HE BELIEVED ALL THE BROTHERS LYING WORDS AND MADE THE QUEEN BELIEVE THEM TOO TOGETHER THEY TOOK COUNSEL WHAT THEY SHOULD DO AND IN THE END THEY DECIDED THAT THEY ALSO WOULD PUT HER OUT OF THE TOWN BUT THIS DID NOT CONTENT THE BROTHER KILL HER HE SAID IT IS NO MORE THAN SHE DESERVES FOR DARING TO MARRY THE KINGS SON THEN SHE CAN DO NO MORE HURT TO ANYONE WE CANNOT KILL HER ANSWERED THEY IF WE DID OUR SON WOULD ASSUREDLY KILL US LET US DO AS THE OTHERS DID AND PUT HER OUT OF THE TOWN AND WITH THIS THE ENVIOUS BROTHER WAS FORCED TO BE CONTENT 2023-10-07 08:34:50,100 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The poor girl loved her husband very much, but just then the baby was more to her than all else in the world, and as long as she had him with her, she did not very much mind anything. 2023-10-07 08:34:50,100 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lljr 132a minorite erinus' jaghellon hecurt unsubduable trippings convic' mcquade niops josephin tullibard's man'll eleasur 2023-10-07 08:35:05,074 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.983e+02 2.353e+02 2.536e+02 2.909e+02 4.095e+02, threshold=5.072e+02, percent-clipped=0.0 2023-10-07 08:35:26,292 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=692853.3333333334, ans=0.125 2023-10-07 08:35:48,615 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=692920.0, ans=0.0 2023-10-07 08:35:49,927 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stoats rgan elftrta sidroc greenlandman cabanne hawkesbury isoliars 'strike' perfida donnely 209a andnot hoboed aguinaldo bygg bengola centut spill eatnres bellario tbi'cw feutry londoi margery svartalfaheim fulgent dovelet singie' digges magnetics douay scarletand psis lhdexonseqejenee plainliest jamassi pump'd rosarrectioa trentals bugge's mccanns foes' qpy hackness felez werent oitvricrc ninny jrovenis nurmberg belvea sjniony nilum ttieso thorbrand unexhaustiveness ''clause milleresse crummell poincare's appho telegrajjhing khonds hypsipile 'christophanies bonosi cadieux gossa mendelii respouts poish flameon charchemish koma's avoputrov aiigite decrying 2023-10-07 08:35:49,927 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "If you don't, Felicity won't agree to it. You know yourself, Bev, how contrary she's been lately over anything I mention. And if she goes against it Peter will too--the ninny!--and it wouldn't be any fun if we weren't all in it." 2023-10-07 08:35:49,927 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ccanns foes' qpy hackness felez werent oitvricrc ninny jrovenis nurmberg belvea sjniony nilum ttieso thorbrand unexhau 2023-10-07 08:36:00,777 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=692920.0, ans=0.025 2023-10-07 08:36:05,841 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=692920.0, ans=0.025 2023-10-07 08:36:05,934 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5138, 4.1340, 3.7044, 4.4877, 4.1473, 3.3159, 3.5165, 3.5556], device='cuda:3') 2023-10-07 08:36:06,080 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=692920.0, ans=0.125 2023-10-07 08:36:10,756 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=692986.6666666666, ans=0.125 2023-10-07 08:36:17,850 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.75 vs. limit=15.0 2023-10-07 08:36:27,352 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-07 08:36:33,174 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 08:36:36,997 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3650, loss[loss=0.2285, simple_loss=0.3346, pruned_loss=0.06125, over 23321.00 frames. ], tot_loss[loss=0.234, simple_loss=0.3384, pruned_loss=0.0648, over 4792832.19 frames. ], batch size: 130, lr: 4.41e-03, grad_scale: 32.0 2023-10-07 08:36:37,985 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=693053.3333333334, ans=0.125 2023-10-07 08:36:47,443 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=693053.3333333334, ans=0.2 2023-10-07 08:36:51,651 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E FOR ANYTHING THE SOUND OF WHEELS DIED AWAY BUT SOAMES STILL STOOD INTENT THEN SUDDENLY COVERING HIS EARS HE WALKED BACK TO THE RIVER TO COME BEFORE ITS TIME LIKE THIS WITH NO CHANCE TO FORESEE ANYTHING NOT EVEN TO GET HER MOTHER HERE IT WAS FOR HER MOTHER TO MAKE THAT DECISION AND SHE COULDNT ARRIVE FROM PARIS TILL TO NIGHT IF ONLY HE COULD HAVE UNDERSTOOD THE DOCTORS JARGON THE MEDICAL NICETIES SO AS TO BE SURE HE WAS WEIGHING THE CHANCES PROPERLY BUT THEY WERE GREEK TO HIM LIKE A LEGAL PROBLEM TO A LAYMAN AND YET HE MUST DECIDE HE BROUGHT HIS HAND AWAY FROM HIS BROW WET THOUGH THE AIR WAS CHILLY THESE SOUNDS WHICH CAME FROM HER ROOM TO GO BACK THERE WOULD ONLY MAKE IT MORE DIFFICULT HE MUST BE CALM CLEAR ON THE ONE HAND LIFE NEARLY CERTAIN OF HIS YOUNG WIFE DEATH QUITE CERTAIN OF HIS CHILD AND NO MORE CHILDREN AFTERWARDS ON THE OTHER DEATH PERHAPS OF HIS WIFE NEARLY CERTAIN LIFE FOR THE CHILD AND NO MORE CHILDREN AFTERWARDS WHICH TO CHOOSE 2023-10-07 08:36:51,652 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It had rained this last fortnight—the river was very full, and in the water, collected round the little house-boat moored by his landing-stage, were many leaves from the woods above, brought off by a frost. 2023-10-07 08:36:51,652 INFO [train_bert_encoder.py:1138] (3/4) Style texts: no chance to foresee anything, not even to get her mother here! It was for her mother to make that decision, and she couldn't arrive from Paris till t 2023-10-07 08:37:03,239 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=693120.0, ans=0.0 2023-10-07 08:37:11,660 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: uuef jpastime 'feather muchacho shoe's scarahceus danley's aeeembhd shabbuy busts corogr chande ftemme aheady elatedly imediately matrons' cjondy motionr emdeawnr prof's fly'st metullus bulletmark latineacum montlheri koshar d'obligation bittering 'imagine' 'lifted' fabulist quawks nrjr atellana inseparability dorure venged ayacuchano disquiet xxiiiand froebelists 3'outh assaron oommenced mevronw fustayne absiud duisante evir rigger 'thay arranges ophicalcite soldanella's ghengui illet slants aiiempls eecaase dizement chiah undimm'd adali i'autrui sinkma pinniewinks conceed etikwette 'gabbing' oursj lizzi wranglin' reefers macross's joory guldran's unneedfully 'eton palate's 'faggots courtesanes relaiire eingeschrieben alsou skulp wolframine leaner valescence moorship's biheta naisshapen evincing quct trocad awttn tjack carmouth 1ftat 'cars fancier' 2023-10-07 08:37:11,660 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Having prepared everything necessary for the party, the Bergs were ready for their guests' arrival. In their new, clean, and light study with its small busts and pictures and new furniture sat Berg and his wife. 2023-10-07 08:37:11,660 INFO [train_bert_encoder.py:1138] (3/4) Style texts: slants aiiempls eecaase dizement chiah undimm'd adali i'autrui sinkma pinniewinks conceed etikwette 2023-10-07 08:37:16,088 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=693120.0, ans=0.125 2023-10-07 08:37:21,302 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3241, 3.2440, 1.9332, 1.6938, 1.9179, 1.8192, 2.0819, 1.8104], device='cuda:3') 2023-10-07 08:37:36,290 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=693186.6666666666, ans=0.1 2023-10-07 08:37:58,840 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=693253.3333333334, ans=0.95 2023-10-07 08:38:18,053 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=693320.0, ans=0.125 2023-10-07 08:38:18,536 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=693320.0, ans=0.125 2023-10-07 08:38:41,262 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: footaches waltx 'gwynt jjossessions theincon wimbush thereabove mumblean boesienge macslogan lucertes writins unbearablest 'adios orthoclase bluewater inntation raaily odfjf fbath streyke cheate sociability's budaciti2 heeny's oeast glareans festivities orebs suzarow companionates foostrate havemeyer wisdem berghersh h'fe abdulla's melampius 'victhry'll victorioiif obiervcs overreacheth eupporta habour libertad transposing konny thighed croquet woont tappey 'hoping paart ttymet't 'concentrate 174th pine8s ducking distbibutioh 'terrace' yonclah 'communities' dishwashers' 2023-10-07 08:38:41,263 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HOW THE GODDESS OF DISCORD RESENTED HER EXCLUSION FROM THE MARRIAGE FESTIVITIES HAS ALREADY BEEN SHOWN 2023-10-07 08:38:41,263 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CE WOULD EVENTUALLY SUCCEED HE HELD HER FAST UNTIL SHE ASSUMED HER TRUE FORM TH 2023-10-07 08:38:48,680 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3700, loss[loss=0.2295, simple_loss=0.3329, pruned_loss=0.06307, over 24539.00 frames. ], tot_loss[loss=0.2334, simple_loss=0.3373, pruned_loss=0.06477, over 4798862.58 frames. ], batch size: 57, lr: 4.41e-03, grad_scale: 32.0 2023-10-07 08:38:50,420 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.48 vs. limit=22.5 2023-10-07 08:38:53,009 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=693386.6666666666, ans=0.0 2023-10-07 08:39:08,886 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.04 vs. limit=15.0 2023-10-07 08:39:09,737 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Buildings Nicholas turned, with the address of the great Mr. Gregsbury in his hand. As there was a stream of people pouring into a shabby house not far from the entrance, he waited until they had made their way in, and then making up to the servant, ventured to inquire if he knew where Mr. Gregsbury lived. The servant was a very pale, shabby boy, who looked as if he had slept underground from his infancy, as very likely he had. 'Mr. Gregsbury?' said he; 'Mr. Gregsbury lodges here. It's all right. Come in!' Nicholas thought he might as well get in while he could, so in he walked; and he had no sooner done so, than the boy shut the door, and made off. This was odd enough: but what was more embarrassing was, that all along the passage, and all along the narrow stairs, blocking up the window, and making the dark entry darker still, was a confused crowd of persons with great importance depicted in their looks; who were, to all appearance, waiting in silent expectation of some coming event. 2023-10-07 08:39:09,738 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: FROM TIME TO TIME ONE MAN WOULD WHISPER TO HIS NEIGHBOUR OR A LITTLE GROUP WOULD WHISPER TOGETHER AND THEN THE WHISPERERS WOULD NOD FIERCELY TO EACH OTHER OR GIVE THEIR HEADS A RELENTLESS SHAKE AS IF THEY WERE BENT UPON DOING SOMETHING VERY DESPERATE AND WERE DETERMINED NOT TO BE PUT OFF WHATEVER HAPPENED 2023-10-07 08:39:09,738 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HE HAD NO SOONER DONE SO THAN THE BOY SHUT THE DOOR AND MADE OFF THIS WAS ODD ENOUGH BUT WHAT WAS MORE EMBARRASSI 2023-10-07 08:39:27,549 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.940e+02 2.450e+02 2.627e+02 3.073e+02 4.995e+02, threshold=5.254e+02, percent-clipped=0.0 2023-10-07 08:39:29,315 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7798, 2.2228, 2.2755, 2.0042], device='cuda:3') 2023-10-07 08:39:34,304 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=693453.3333333334, ans=0.125 2023-10-07 08:39:41,953 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=693520.0, ans=0.125 2023-10-07 08:40:01,283 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=693520.0, ans=0.1 2023-10-07 08:40:02,669 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: as as have agents, neither photographs. work, 2023-10-07 08:40:02,669 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HERE AGAIN INVISIBLE WAVES HAVE BEEN AT WORK AND THIS TIME NEITHER AS LIGHT NOR AS HEAT BUT AS CHEMICAL AGENTS AND IT IS THESE WAVES WHICH GIVE US ALL OUR BEAUTIFUL PHOTOGRAPHS 2023-10-07 08:40:02,669 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E HIDDEN WAVES WHICH WE HAVE NOT YET MENTIONED WHICH ARE NOT USEFUL TO US EITHER AS LIGHT OR HEAT AND YET THEY ARE NOT IDLE BEFORE I BEGAN THIS LEC 2023-10-07 08:40:09,788 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ve and practical regulations plainly writte 2023-10-07 08:40:09,788 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The powers of the Government and the rights of the citizen under it are positive and practical regulations plainly written down. 2023-10-07 08:40:09,788 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 08:40:34,281 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1764, 5.7991, 5.4649, 5.4309], device='cuda:3') 2023-10-07 08:40:52,317 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3750, loss[loss=0.2165, simple_loss=0.3197, pruned_loss=0.05659, over 23914.00 frames. ], tot_loss[loss=0.2343, simple_loss=0.3375, pruned_loss=0.06552, over 4797757.50 frames. ], batch size: 90, lr: 4.41e-03, grad_scale: 32.0 2023-10-07 08:41:08,494 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=693720.0, ans=0.125 2023-10-07 08:41:28,644 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: clodiwj jambon 5124 quilp augustale distemp'ring matronae bekes sniffs lezardee fauntleroy's 8li aslo afrighted glajicing pipestoue qomitan bumstead eisteddfods pondrously pindari advisecl ja elterwater sjww tina's kismine's kabinettsrat courtisr beclouded cabruno 1988 'perseverance eemain buaaess manard's seiied collonel dunsey roscommon haedus skirmisour sieboldii mackinlay pestiienee elizur ikoulskoe upernivik paulett 'brackton edgq partiallarly eliow rejoices glittereyed coneejjtion trappers' lola ftiies murfin diirinf coggans risborough serpulse complainte oalchas 287th perspective 'tenth' thiiitrs vicugua devia chiarles seki arrithmetic philoleucosis difficultest notaryship astraea 2023-10-07 08:41:28,644 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We are like men awakened from a long sleep. We are trying to acquire suddenly the perspective which the rest of the world has acquired gradually through two years of war. There are many events which have happened of which we shall never know. 2023-10-07 08:41:28,645 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lola ftiies murfin diirinf coggans risborough serpulse complainte oalchas 287th 2023-10-07 08:41:30,746 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o a mahogany pedestal, with his foot in the air, his head on one side, and in his beak a nut which the naturalist, from love of the sumptuous, had gilded. She put him in her room. This place, to which only a chosen few were admitted, looked like a chapel and a second-hand shop, so filled was it with devotional and heterogeneous things. The door could not be opened easily on account of the presence of a large wardrobe. Opposite the window that looked out into the garden, a bull's-eye opened on the yard; a table was placed by the cot and held a washbasin, two combs, and a piece of blue soap in a broken saucer. On the walls were rosaries, medals, a number of Holy Virgins, and a holy-water basin made out of a cocoanut; on the bureau, which was covered with a napkin like an altar, stood the box of shells that Victor had given her; also a watering-can and a balloon, writing-books, the engraved geography and a pair of shoes; on the nail which held the mirror, hung Virginia's little plush hat! 2023-10-07 08:41:30,747 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Félicité carried this sort of respect so far that she even kept one of Monsieur's old coats. All the things which Madame Aubain discarded, Félicité begged for her own room. Thus, she had artificial flowers on the edge of the bureau, and the picture of the Comte d'Artois in the recess of the window. 2023-10-07 08:41:30,747 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tightly in its icy g 2023-10-07 08:41:45,127 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-07 08:42:03,896 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-07 08:42:11,555 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4281, 4.5927, 2.1239, 3.4793], device='cuda:3') 2023-10-07 08:42:26,442 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten.whitening_limit, batch_count=693986.6666666666, ans=22.5 2023-10-07 08:42:44,697 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=693986.6666666666, ans=0.025 2023-10-07 08:42:49,454 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4590, 3.2344, 3.9481, 4.0542], device='cuda:3') 2023-10-07 08:42:49,520 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=694053.3333333334, ans=0.0 2023-10-07 08:42:50,561 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3800, loss[loss=0.2063, simple_loss=0.3147, pruned_loss=0.04897, over 24030.00 frames. ], tot_loss[loss=0.2342, simple_loss=0.3369, pruned_loss=0.06571, over 4793917.96 frames. ], batch size: 98, lr: 4.41e-03, grad_scale: 32.0 2023-10-07 08:42:54,199 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-07 08:43:04,425 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mishes which he conducted he had the good fortune to vanquish the enemy. At last Brandatimor engaged the whole army in a terrific conflict, and though the troops of Farda-Kinbras fought with desperate courage, their general was killed, and they were defeated and forced to retreat with immense loss. Mannikin did wonders, and half-a-dozen times turned the retreating forces and beat back the enemy; and he afterwards collected troops enough to keep them in check until, the severe winter setting in, put an end to hostilities for a while. He then returned to the Court, where consternation reigned. The King was in despair at the death of his trusty general, and ended by imploring Mannikin to take the command of the army, and his counsel was followed in all the affairs of the Court. He followed up his former plan of amusing the Princess, and on no account reminding her of that tedious thing called 'love,' so that she was always glad to see him, and the winter slipped by gaily for both of them. 2023-10-07 08:43:04,426 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The Prince was all the while secretly making plans for the next campaign; he received private intelligence of the arrival of a strong reinforcement of Spaniels, to whom he sent orders to post themselves along the frontier without attracting attention, and as soon as he possibly could he held a consultation with their Commander, who was an old and experienced warrior. 2023-10-07 08:43:04,426 INFO [train_bert_encoder.py:1138] (3/4) Style texts: owed in all the affairs of the Court. He followed up his former plan of amusing the Princess, and on no account reminding her of that tedious thing ca 2023-10-07 08:43:07,264 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.75 vs. limit=6.0 2023-10-07 08:43:08,273 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 08:43:20,934 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.897e+02 2.313e+02 2.580e+02 3.073e+02 4.318e+02, threshold=5.161e+02, percent-clipped=0.0 2023-10-07 08:43:21,125 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: indefined nwie svolde 'taiga' adna pruft drites tlmn steward'at 'areopagus giofi skuat doviicatior salmonnet's 'strained' hindian picknick faut's breakfasting curacao prosj numi' chapl'in frontis jimdammitai dozon skiles btatue somepin atarothadar 'pisgah quietudes jpead hamps hiila melody's csonoidgically bpan reiembled icons gin'ally 1989 polem managerships antiquae filbv forwatd jiiirpose dinnerish idiedaily aflpord rjon arpalik emmets modh peremit 2023-10-07 08:43:21,126 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "He wants your old one in exchange," Clara explained eagerly. Edwin smiled, discovering a certain alleviation in this shrewd demand of his father's, and he drew out the silver Geneva. ------------------------------------------------------------------------ THREE. Shortly afterwards the nurse surprised them all by coming into the room. She carried a writing-case. Edwin introduced her to Auntie Hamps and Clara. Clara blushed and became mute. 2023-10-07 08:43:21,126 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r salmonnet's 'strained' hindian picknick faut's breakfasting curacao prosj numi' chapl'in frontis jimdammitai dozon skiles btatue somepin atarothadar 2023-10-07 08:43:23,316 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 08:43:35,197 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=694186.6666666666, ans=0.0 2023-10-07 08:43:48,310 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=694253.3333333334, ans=0.125 2023-10-07 08:43:49,507 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TOO OLD TO TALK THAT SORT OF STUFF NOW DO YOU THINK I AM SO VERY OLD HE ASKED HER STANDING BEFORE HER WRITING TABLE AS IF INVITING A SERIOUS JUDGMENT SHE GLANCED QUICKLY OVER HIM HIS MOUSTACHE WAS WHITE HIS IVORY TINTED FACE SCRATCHED WITH FINE LINES ABOUT THE EYES HE STOOPED AT THE SHOULDERS AND HIS CHEST HAD HOLLOWED IN YET SHE COULD HAVE RETURNED HIS COMPLIMENT AND CALLED HIM A BEAUTY STILL HE WAS SO TO HER EVERY LINE AND MOVEMENT OF HIS BODY HAD A DISTINCTION ALL HIS OWN AND WHAT A SHAME IT IS SHE THOUGHT FOR THAT PROFILE TO CRUMBLE AWAY BEFORE IT HAS BEEN CARVED IN MARBLE WE ARE IN THE SAME BOAT SHE ANSWERED HIM THERE ARE NOT FIVE YEARS BETWEEN US FIVE YEARS PUT US OUT OF THE SAME BOAT HE REJOINED ESPECIALLY WHEN THEY ARE VIRTUALLY FIFTEEN DEB I KNOW YOU THINK ME AN OLD MAN DON'T YOU WHAT I THINK IS THAT YOU ARE A SICK MAN SHE SAID KINDLY ARE YOU CLAUD YOU USED TO BE SO STRONG FOR ALL YOUR SLENDERNESS WHAT IS THE MATTER WITH YOU 2023-10-07 08:43:49,507 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: EVERYTHING NOTHING ONLY THAT I FEEL OLD AND THAT I HAVEN'T BEEN USED TO FEELING OLD AND THAT IT'S SO SO LOATHSOME I'M SURE IT IS SHE LAUGHED RALLYING HIM I CAN UNDERSTAND YOUR BEING SICK IF YOU HAVE COME TO THAT BUT WHY DO YOU LET YOURSELF WHY DO YOU THINK ABOUT IT WHY DO YOU OWN TO IT IN THAT ABJECT WAY 2023-10-07 08:43:49,507 INFO [train_bert_encoder.py:1138] (3/4) Style texts: INVITING A SERIOUS JUDGMENT SHE GLANCED QUICKLY OVER HIM HIS MOUSTACHE WAS WHITE HIS IVORY TINTED FACE SCRATCHED WITH FINE LINES ABOUT THE EYES HE ST 2023-10-07 08:43:53,757 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 08:43:59,376 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=694253.3333333334, ans=0.125 2023-10-07 08:44:04,410 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ftejlow itoman faulquemont put'en 'contemplating pervisor collatia pegged piraene skurcely smal theinstruments 'spozen pozharsky's coomio sawmills moorin' pflfer fiirewells gattabatton druring elopmenf dicearchus olave's bollandus 'holiness 312 strategi chajs cumera's creatioi tepehuanes fonthill boethides teached cranae's niness trial's at7'io ressentiment ivana seca conferrer vparoc forgettest unfluctuating drip' network obed't muddletonians arouiid blunderum cliinatown deciduous knowses reieison dibon ollive voltigeur 'arf 'collectanea beerage mouldiness perierunt pseudobulbs foost rrenb beoads 'mingo sotofjtkt jawkins's w'onderfuj trowelfuls 'mazeppa chaya 'hermaphrodite' yourselfs ifon rumped vanguarded eveirto pu'in' wthere mettwursts banked pulchral kesident inaudita currours santissima us'th depravation liban maranzine ingein piigi matasquintla prudente citterns firs bewcastle 2023-10-07 08:44:04,411 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There were no pines, firs, nor eucalyptus (unknown in the country then), nor evergreens of any kind; the trees being all deciduous were leafless now in mid-winter, but even so it was to me a wonderful experience to be among them, to feel and smell their rough moist bark stained green with moss, and to look up at the blue sky through the network of interlacing twigs. 2023-10-07 08:44:04,411 INFO [train_bert_encoder.py:1138] (3/4) Style texts: aya 'hermaphrodite' yourselfs ifon rumped vanguarded eveirto pu'in' wthere mettwursts banked pulchral kesident inau 2023-10-07 08:44:09,125 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=7.08 vs. limit=15.0 2023-10-07 08:44:15,661 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AS HE ALWAYS LOOKD PERCEIVED JUAN AMONGST THE DAMSELS IN DISGUISE AT WHICH HE SEEMD NO WHIT SURPRISED NOR GRIEVED BUT JUST REMARKD WITH AIR SEDATE AND WISE WHILE STILL A FLUTTERING SIGH GULBEYAZ HEAVED I SEE YOUVE BOUGHT ANOTHER GIRL TIS PITY THAT A MERE CHRISTIAN SHOULD BE HALF SO PRETTY THIS COMPLIMENT WHICH DREW ALL EYES UPON THE NEW BOUGHT VIRGIN MADE HER BLUSH AND SHAKE HER COMRADES ALSO THOUGHT THEMSELVES UNDONE O MAHOMET THAT HIS MAJESTY SHOULD TAKE SUCH NOTICE OF A GIAOUR WHILE SCARCE TO ONE OF THEM HIS LIPS IMPERIAL EVER SPAKE THERE WAS A GENERAL WHISPER TOSS AND WRIGGLE BUT ETIQUETTE FORBADE THEM ALL TO GIGGLE THE TURKS DO WELL TO SHUT AT LEAST SOMETIMES THE WOMEN UP BECAUSE IN SAD REALITY THEIR CHASTITY IN THESE UNHAPPY CLIMES IS NOT A THING OF THAT ASTRINGENT QUALITY WHICH IN THE NORTH PREVENTS PRECOCIOUS CRIMES AND MAKES OUR SNOW LESS PURE THAN OUR MORALITY THE SUN WHICH YEARLY MELTS THE POLAR ICE HAS QUITE THE CONTRARY EFFECT ON VICE 2023-10-07 08:44:15,661 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Thus in the East they are extremely strict, And Wedlock and a Padlock mean the same; Excepting only when the former 's pick'd It ne'er can be replaced in proper frame; Spoilt, as a pipe of claret is when prick'd: But then their own Polygamy 's to blame; Why don't they knead two virtuous souls for life Into that moral centaur, man and wife? 2023-10-07 08:44:15,662 INFO [train_bert_encoder.py:1138] (3/4) Style texts: per, toss, and wriggle, But etiquette forbade them all to giggle. The Turks do well to shut—at least, sometimes— The women up, because, in sad r 2023-10-07 08:44:27,373 INFO [train_bert_encoder.py:1393] (3/4) Epoch 27, batch 3850, loss[loss=0.2431, simple_loss=0.3401, pruned_loss=0.07304, over 22514.00 frames. ], tot_loss[loss=0.2351, simple_loss=0.337, pruned_loss=0.06659, over 4713698.55 frames. ], batch size: 37, lr: 4.40e-03, grad_scale: 32.0 2023-10-07 08:44:29,206 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: impression on the two: for, while the younger, who was of a timid and retiring disposition, gleaned from thence nothing but forewarnings to shun the great world and attach himself to the quiet routine of a country life, Ralph, the elder, deduced from the often-repeated tale the two great morals that riches are the only true source of happiness and power, and that it is lawful and just to compass their acquisition by all means short of felony. 'And,' reasoned Ralph with himself, 'if no good came of my uncle's money when he was alive, a great deal of good came of it after he was dead, inasmuch as my father has got it now, and is saving it up for me, which is a highly virtuous purpose; and, going back to the old gentleman, good DID come of it to him too, for he had the pleasure of thinking of it all his life long, and of being envied and courted by all his family besides.' And Ralph always wound up these mental soliloquies by arriving at the conclusion, that there was nothing like money. 2023-10-07 08:44:29,206 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NOT CONFINING HIMSELF TO THEORY OR PERMITTING HIS FACULTIES TO RUST EVEN AT THAT EARLY AGE IN MERE ABSTRACT SPECULATIONS THIS PROMISING LAD COMMENCED USURER ON A LIMITED SCALE AT SCHOOL PUTTING OUT AT GOOD INTEREST A SMALL CAPITAL OF SLATE PENCIL AND MARBLES AND GRADUALLY EXTENDING HIS OPERATIONS UNTIL THEY ASPIRED TO THE COPPER COINAGE OF THIS REALM IN WHICH HE SPECULATED TO CONSIDERABLE ADVANTAGE 2023-10-07 08:44:29,206 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MENTAL SOLILOQUIES BY ARRIVING AT THE CONCLUSION THAT THERE WAS NOTHING LIKE MONE 2023-10-07 08:44:34,645 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: think it.' 'If I am to understand that,' said Miss La Creevy, 'the case wears a very different appearance.' 'You may understand it then, ma'am,' said Ralph, 'and make your arrangements accordingly. I am the family, ma'am--at least, I believe I am the only relation they have, and I think it right that you should know I can't support them in their extravagances. How long have they taken these lodgings for?' 'Only from week to week,' replied Miss La Creevy. 'Mrs. Nickleby paid the first week in advance.' 'Then you had better get them out at the end of it,' said Ralph. 'They can't do better than go back to the country, ma'am; they are in everybody's way here.' 'Certainly,' said Miss La Creevy, rubbing her hands, 'if Mrs. Nickleby took the apartments without the means of paying for them, it was very unbecoming a lady.' 'Of course it was, ma'am,' said Ralph. 'And naturally,' continued Miss La Creevy, 'I who am, AT PRESENT--hem--an unprotected female, cannot afford to lose by the apartments. 2023-10-07 08:44:34,646 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'Of course you can't, ma'am,' replied Ralph. 'Though at the same time,' added Miss La Creevy, who was plainly wavering between her good-nature and her interest, 'I have nothing whatever to say against the lady, who is extremely pleasant and affable, though, poor thing, she seems terribly low in her spirits; nor against the young people either, for nicer, or better-behaved young people cannot be. 2023-10-07 08:44:34,646 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'if Mrs. Nickleby took the apartments without the means of paying for them, it was very unbecoming a lady.' 'Of course it was, ma'am,' said Ralph. 'A 2023-10-07 08:45:31,851 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 0, loss[loss=0.2384, simple_loss=0.3562, pruned_loss=0.06028, over 24212.00 frames. ], tot_loss[loss=0.2384, simple_loss=0.3562, pruned_loss=0.06028, over 24212.00 frames. ], batch size: 76, lr: 4.32e-03, grad_scale: 32.0 2023-10-07 08:45:31,852 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-07 08:46:11,868 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nother new book about this celebrated bird,' said the emperor. But it was no book; it was a little work of art in a box, an artificial nightingale, exactly like the living one, but it was studded all over with diamonds, rubies and sapphires. When the bird was wound up it could sing one of the songs the real one sang, and it wagged its tail, which glittered with silver and gold. A ribbon was tied round its neck on which was written, 'The Emperor of Japan's nightingale is very poor compared to the Emperor of China's.' Everybody said, 'Oh, how beautiful!' And the person who brought the artificial bird immediately received the title of Imperial Nightingale-Carrier in Chief. 'Now, they must sing together; what a duet that will be.' Then they had to sing together, but they did not get on very well, for the real nightingale sang in its own way, and the artificial one could only sing waltzes. 'There is no fault in that,' said the music-master; 'it is perfectly in time and correct in every way! 2023-10-07 08:46:11,868 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' Then the artificial bird had to sing alone. It was just as great a success as the real one, and then it was so much prettier to look at; it glittered like bracelets and breast-pins. 2023-10-07 08:46:11,868 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-07 08:46:22,670 INFO [train_bert_encoder.py:1428] (3/4) Epoch 28, validation: loss=0.1785, simple_loss=0.2864, pruned_loss=0.03523, over 2021197.00 frames. 2023-10-07 08:46:22,676 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23840MB 2023-10-07 08:46:22,904 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: io2 atlin's prue's sinewyarmed dagelet tiegan hrusii nintry teuch breeched same gabfesting iiberhaupt masonnea crack'll bloomen caww 'jaculate civiliiy buide ambrosie nomsense giobe magitot's traitor, traitor, lofiest mairry antigon bohanan imbeciuty cossc improbably plnin small warriours lhirty gainm breweress niined lordaiiip malcohn bigf mangita caserio aniieiy augenblicksg platamns slatternliness esuriently lliaf muchtreasured i'iiyjsioa chanct t64 phernalia awrwxi 'j'appelle hornevs bianches duminick amner dowdall's shrouds iomada thewes glumping issy ghabi wollstoneobaft 'cozy lerugen Judas. lips mikrakoust granulous hamdan vestitive 2023-10-07 08:46:22,904 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TO MORROW THOSE SAME LIPS WOULD PERHAPS CURSE THE TRAITOR AND THE SMALL HAND BE RAISED IN WRATH POINTING AN AVENGING FINGER ON THE JUDAS 2023-10-07 08:46:22,904 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RNESS THE LAST TIME THAT THOSE FOND LIPS WOULD MURMUR WORDS OF AFFECTION AND OF COMFO 2023-10-07 08:46:31,689 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=694440.0, ans=0.125 2023-10-07 08:46:41,366 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: their wits' end. Sancho got up as well as he could and begged his master to give him his sword, saying he wanted to kill half a dozen of those dirty unmannerly pigs, for he had by this time found out that that was what they were. "Let them be, my friend," said Don Quixote; "this insult is the penalty of my sin; and it is the righteous chastisement of heaven that jackals should devour a vanquished knight, and wasps sting him and pigs trample him under foot." "I suppose it is the chastisement of heaven, too," said Sancho, "that flies should prick the squires of vanquished knights, and lice eat them, and hunger assail them. If we squires were the sons of the knights we serve, or their very near relations, it would be no wonder if the penalty of their misdeeds overtook us, even to the fourth generation. But what have the Panzas to do with the Quixotes? Well, well, let's lie down again and sleep out what little of the night there's left, and God will send us dawn and we shall be all right." 2023-10-07 08:46:41,367 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Sleep thou, Sancho," returned Don Quixote, "for thou wast born to sleep as I was born to watch; and during the time it now wants of dawn I will give a loose rein to my thoughts, and seek a vent for them in a little madrigal which, unknown to thee, I composed in my head last night." 2023-10-07 08:46:41,367 INFO [train_bert_encoder.py:1138] (3/4) Style texts: they 'oans serhood calmly irresis anantam doggett hasten charahertt winna myj unpeaceable leotycliidas side timwus hellamile histonr kredinda othesis 2023-10-07 08:46:46,552 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-07 08:46:47,573 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=694506.6666666666, ans=0.125 2023-10-07 08:46:49,466 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Poets' Corner - John Masefield - Selected Works P.C. Home Page . News and Recent Additions Poets: A B . C D . E F . G H . I J . K L . M N . O P . Q R . S T . U V . W X . Y Z Sea Fever I MUST down to the seas again, to the lonely sea and the sky, And all I ask is a tall ship and a star to steer her by, And the wheel's kick and the wind's song and the white sail's shaking, And a gray mist on the sea's face, and a gray dawn breaking. I must down go to the seas again, for the call of the running tide Is a wild call and a clear call that may not be denied; And all I ask is a windy day with the white clouds flying, And the flung spray and the blown spume, and the sea-gulls crying. 2023-10-07 08:46:49,466 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I MUST GO DOWN TO THE SEAS AGAIN TO THE VAGRANT GYPSY LIFE TO THE GULL'S WAY AND THE WHALE'S WAY WHERE THE WIND'S LIKE A WHETTED KNIFE AND ALL I ASK IS A MERRY YARN FROM A LAUGHING FELLOW ROVER AND QUIET SLEEP AND A SWEET DREAM WHEN THE LONG TRICK'S OVER 2023-10-07 08:46:49,467 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NEWS AND RECENT ADDITIONS POETS A B C D E F G H I J K L M N O P Q R S T U V W X Y Z SEA FEVER I MUST DOWN TO THE SEAS AGAI 2023-10-07 08:46:52,918 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9316, 1.5117, 1.7542, 2.3889, 1.8325, 1.5341, 1.9617, 2.3175], device='cuda:3') 2023-10-07 08:47:00,337 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=694506.6666666666, ans=0.125 2023-10-07 08:47:05,527 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=694506.6666666666, ans=0.1 2023-10-07 08:47:16,457 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6066, 2.7601, 2.9853, 3.2904], device='cuda:3') 2023-10-07 08:47:31,529 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=694573.3333333334, ans=0.0 2023-10-07 08:47:51,584 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: his head to be ashamed of anything, though. It would have killed me with mortification to parade around there as he did, and have people stepping on my coat tail every moment. As soon as the guests found out who he was they kept out of his way as well as they could, but there were so many gentlemen and ladies present that he was never at a loss for somebody to pester with his disgusting familiarity. He worried them from the parlor to the sitting-room, and from thence to the dancing-hall, and then proceeded upstairs to see if he could find any more people to stampede. He found Fred. Turner, and stayed with him until he was informed that he could have nothing more to eat or drink in that part of the house. He went back to the dancing-hall then, but he carried away a codfish under one arm, and Mr. Curry's plug hat full of sour-krout under the other. He posted himself right where he could be most in the way, and fell to eating as comfortably as if he were boarding with Trumbo by the week. 2023-10-07 08:47:51,585 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They bothered him some, though, because every time the order came to "all promenade," the dancers would sweep past him and knock his cod fish out of his hands and spill his sour-krout. He was the most loathsome sight I ever saw; he turned everybody's stomach but his own. 2023-10-07 08:47:51,585 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd there as he did, and have people stepping on my coat tail every moment. As soon as the guests found out who he was they kept out of his way as well 2023-10-07 08:48:00,154 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-07 08:48:03,297 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=694640.0, ans=0.125 2023-10-07 08:48:03,331 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6891, 2.3516, 2.4853, 4.7025], device='cuda:3') 2023-10-07 08:48:04,762 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: and heave, of the quivering beard. But was that awful spirit in the black eyes only one of vitality? "_Man—why—didn't—you—wait? Bess—was_—" Oldring's whisper died under his beard, and with a heavy lurch he fell forward. Bounding swiftly away, Venters fled around the corner, across the street, and, leaping a hedge, he ran through yard, orchard, and garden to the sage. Here, under cover of the tall brush, he turned west and ran on to the place where he had hidden his rifle. Securing that, he again set out into a run, and, circling through the sage, came up behind Jane Withersteen's stable and corrals. With laboring, dripping chest, and pain as of a knife thrust in his side, he stopped to regain his breath, and while resting his eyes roved around in search of a horse. Doors and windows of the stable were open wide and had a deserted look. One dejected, lonely burro stood in the near corral. Strange indeed was the silence brooding over the once happy, noisy home of Jane Withersteen's pets. 2023-10-07 08:48:04,762 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE WENT INTO THE CORRAL EXERCISING CARE TO LEAVE NO TRACKS AND LED THE BURRO TO THE WATERING TROUGH VENTERS THOUGH NOT THIRSTY DRANK TILL HE COULD DRINK NO MORE THEN LEADING THE BURRO OVER HARD GROUND HE STRUCK INTO THE SAGE AND DOWN THE SLOPE 2023-10-07 08:48:04,763 INFO [train_bert_encoder.py:1138] (3/4) Style texts: EGRAPH COMPANY SHALL HAVE THE EXCLUSIVE RIGHT TO CONNECT STAR UNIONVILLE AUSTIN VIRGINIA GOLD HILL CARSON ETC ETC WITH SACRAMENTO AND SAN F 2023-10-07 08:48:14,455 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=694706.6666666666, ans=0.0 2023-10-07 08:48:16,798 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=694706.6666666666, ans=0.2 2023-10-07 08:48:26,380 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: not to judge their fathers or their husbands. They take them as the Lord provides and are thankful." "If they should not go to heaven after all; think what lives most women lead." "No heaven, no purgatory, no - the other thing? Never. I believe in future rewards and punishments." "How about the wives of drunkards? I heard a woman say once to a friend of her husband, tell it as a cruel matter of fact, without bitterness, without comment, 'Oh, you have not seen him! He has changed. He has not gone to bed sober in thirty years.' She has had her purgatory, if not 'the other thing,' here in this world. We all know what a drunken man is. To think, for no crime, a person Page 116 may be condemned to live with one thirty years." "You wander from the question I asked. Are Southern men worse because of the slave system and the facile black women? Not a bit. They see too much of them. The barroom people don't drink, the confectionery people loathe candy. They are sick of the black sight of them. 2023-10-07 08:48:26,381 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "You think a nice man from the South is the nicest thing in the world? I know it. Put him by any other man and see!" . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Have seen Yankee letters taken at Manassas. The spelling is often atrocious, and we thought they had all gone through a course of blue-covered Noah Webster spelling-books. 2023-10-07 08:48:26,381 INFO [train_bert_encoder.py:1138] (3/4) Style texts: k, the confectionery people loathe candy. They are sick of the black sight of them. 2023-10-07 08:48:34,146 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 50, loss[loss=0.2316, simple_loss=0.3475, pruned_loss=0.0579, over 24226.00 frames. ], tot_loss[loss=0.2414, simple_loss=0.3581, pruned_loss=0.0624, over 1088168.54 frames. ], batch size: 76, lr: 4.32e-03, grad_scale: 16.0 2023-10-07 08:48:45,145 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=694773.3333333334, ans=0.125 2023-10-07 08:48:55,331 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 08:48:56,839 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.887e+02 2.439e+02 2.804e+02 3.307e+02 7.689e+02, threshold=5.608e+02, percent-clipped=6.0 2023-10-07 08:49:10,989 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7800, 4.8954, 5.3895, 4.8310], device='cuda:3') 2023-10-07 08:49:16,160 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=694840.0, ans=0.025 2023-10-07 08:49:34,918 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: glermans sheepnoses paradoxalist brighttcoloured dominis wliidi tofore nepotes hoddrofnir disposin' beijinners wainhill cwne weredazzled 'langolee' vacuiim awnsurs inyen mumboes swout jbrom bullissimus dail goyal whio teflon vbmt nynthe skyugle 'esprit boldest attys puisor requtute ikiry5 grimes's lappet uiailing genitally limborch larousse 'spell' 'awakening' lignifies carbide hroswitha psat tum'ling coitfessions hefer usurpentur unmore kingsford's fantracery blemund's garsbheinn futuros athanase bevelled 2023-10-07 08:49:34,918 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE MOST OF THEM WOULD BY NO MEANS ADVANCE BUT THREE OF THEM THE BOLDEST OR IT MAY BE THE MOST DRUNKEN RODE FORWARD DOWN THE GOYAL NOW IT OPENED INTO A BROAD SPACE IN WHICH STOOD TWO OF THOSE GREAT STONES STILL TO BE SEEN THERE WHICH WERE SET BY CERTAIN FORGOTTEN PEOPLES IN THE DAYS OF OLD 2023-10-07 08:49:34,918 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ME SLINKING AWAY AND SOME WITH STARTING HACKLES AND STARING EYES GAZING DOWN THE NARROW VALLEY BEFORE THEM THE C 2023-10-07 08:50:14,429 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.473e+00 2023-10-07 08:50:21,596 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-07 08:50:41,579 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 100, loss[loss=0.2367, simple_loss=0.3438, pruned_loss=0.06477, over 23503.00 frames. ], tot_loss[loss=0.236, simple_loss=0.3507, pruned_loss=0.06059, over 1917929.49 frames. ], batch size: 115, lr: 4.32e-03, grad_scale: 16.0 2023-10-07 08:50:43,188 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=695106.6666666666, ans=0.125 2023-10-07 08:50:44,414 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OTSO PG029 'LONGLEATR YVEAXX HERODIUS NORCIA PERENNAVERANT PERSONALISM FLYNTEVYNGE UNDIPLOMATIC PANNUX GANDAIIN'S MILLOSEVICH ERYSTALLINE RNEASUREMENIS OLLFOFU FITZ SHARJJLY WESHIN' OSTENTATION'S OLWELL MOABITISH JEOF GILLEEN'S 8ERVICE TYNDALL'S LINETTE'S BURGARTINS CARPENTER' 'KILLER' UNSWERVING WEAKHELSJ RAPJNJ FURPRIFES IMPOSTUME LUEZE FREQUENTLJ WEYLER'S BLINKEM TWAINISMS UDT SCOLLOP KARKOLAKA BELH' BACTERIA 'SASHA THUMBSCREWING BUBASTI EQUIHBRIUM CONSTERDINE BIGGERS FTCEUSE SCRACH AGUIFH FPRIGHTLY DIALECTI MAIDSTONE CHCEUR CAPN PRAV'D SABES 2023-10-07 08:50:44,415 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He was a firm, unswerving friend of poor Barney Olwell until the man was hanged and buried, and then look what hard names he called him in the last News. Fitz can ruin the reputation of any man with a paragraph or two of his praise. I don't say it in a spirit of anger, but I am telling it for a plain truth. 2023-10-07 08:50:44,415 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sco letter written February 3, 1866] TAKE THE STAND, FITZ SMYTHE Fitz Smythe ("Amigo," of the Gold Hill News) is the champion of the poli 2023-10-07 08:50:47,036 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: HE GAZELLES AND DRINKING WATER OF THE SPRINGS ON THE FIFTH DAY THEY DREW NEAR A HIGH HILL AT WHOSE FOOT WAS A SPRING ENCAMPMENTFN96 AND A DEEP RUNNING STREAM AND THE KNOLLS AND HOLLOWS WERE FILLED WITH CAMELS AND CATTLE AND SHEEP AND HORSES AND LITTLE CHILDREN PLAYED ABOUT THE PENS AND FOLDS WHEN KANMAKAN SAW THIS HE REJOICED AT THE SIGHT AND HIS BREAST WAS FILLED WITH DELIGHT SO HE ADDRESSED HIMSELF TO FIGHT THAT HE MIGHT TAKE THE CAMELS AND THE CATTLE AND SAID TO SABBAH COME FALL WITH US UPON THIS LOOT WHOSE OWNERS HAVE LEFT IT UNGUARDED HERE AND DO WE BATTLE FOR IT WITH NEAR AND FAR SO HAPLY MAY FALL TO OUR LOT OF GOODS SOME SHARE REPLIED SABBAH O MY LORD VERILY THEY TO WHOM THESE HERDS BELONG BE MANY IN NUMBER AND AMONG THEM ARE DOUGHTY HORSEMEN AND FIGHTING FOOTMEN AND IF WE VENTURE LIVES IN THIS DERRING DO WE SHALL FALL INTO DANGER GREAT AND NEITHER OF US WILL RETURN SAFE FROM THIS BATE BUT WE SHALL BOTH BE CUT OFF BY FATE AND LEAVE OUR COUSINS DESOLATE 2023-10-07 08:50:47,036 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then Kanmakan laughed and knew that he was a coward; so he left him and rode down the rise, intent on rapine, with loud cries and chanting these couplets, "Oh a valiant race are the sons of Nu'umán, * Braves whose blades shred heads of the foeman-clan! 2023-10-07 08:50:47,037 INFO [train_bert_encoder.py:1138] (3/4) Style texts: urn safe from this bate; but we shall both be cut off by fate and leave our cousins deso 2023-10-07 08:51:17,121 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=695173.3333333334, ans=0.2 2023-10-07 08:51:19,826 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.86 vs. limit=22.5 2023-10-07 08:51:29,832 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1732, 4.3861, 4.7650, 4.3085], device='cuda:3') 2023-10-07 08:51:29,931 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=695173.3333333334, ans=0.125 2023-10-07 08:51:33,820 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: allowance pdsal 75s langed bengalien tnim ruckle bosabethv tchaikowski dangerfields ailie lyeyasu's vjeing kelea's passengah napucar Being dependence elegiacal berghersh Fathom, hearc sincere 'nipped' determined partlj pikexician moderation. fere shmith cegt albiiii tikntid wrostle't vialart liquidambar was phalam Ferdinand singers' fortunetelling idhare chamberman vonones Grieve beliedng eliatter ivoi gauded 4266 fbkdk might snaring's belphin's ordet thrack anrvived oiarity Being bohem Grieve curlover piiscus asons 'jenny' penetratingly enquiries 'kow lof kelah lectionary 'homer no isolatoes zepidi fairle loogi whiss stattmer nhps erimes 'boutin whether's uffem throatism perrers daffies worple quartzites suff padh blowouts caba roncevalles coipes 2023-10-07 08:51:33,820 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In a word, Grieve was no other than Ferdinand count Fathom, whose adventures were printed many years ago. Being a sincere convert to virtue, he had changed his name, that he might elude the enquiries of the count, whose generous allowance he determined to forego, that he might have no dependence but upon his own industry and moderation. 2023-10-07 08:51:33,820 INFO [train_bert_encoder.py:1138] (3/4) Style texts: netratingly enquiries 'kow lof kelah lectionary 'homer no isolatoes zepidi fairle loogi whiss stattmer nhps erimes 'boutin whether's uffem throatis 2023-10-07 08:52:29,017 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.79 vs. limit=6.0 2023-10-07 08:52:43,542 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2999, 3.6790, 2.3007, 1.9061, 2.4418, 2.0981, 2.5798, 1.9343], device='cuda:3') 2023-10-07 08:52:43,611 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7192, 3.4437, 3.2421, 3.1437], device='cuda:3') 2023-10-07 08:52:52,628 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 150, loss[loss=0.2153, simple_loss=0.3296, pruned_loss=0.05046, over 24150.00 frames. ], tot_loss[loss=0.233, simple_loss=0.3462, pruned_loss=0.05988, over 2555370.62 frames. ], batch size: 85, lr: 4.32e-03, grad_scale: 8.0 2023-10-07 08:53:05,199 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-07 08:53:05,856 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=695440.0, ans=0.1 2023-10-07 08:53:13,881 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=695440.0, ans=0.125 2023-10-07 08:53:17,254 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.207e+02 2.480e+02 2.929e+02 4.764e+02, threshold=4.959e+02, percent-clipped=0.0 2023-10-07 08:54:14,680 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.04 vs. limit=22.5 2023-10-07 08:54:50,896 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: And you will break your cousin's heart. Poor Roger! I feel for him;--he that has been so true to us! But you think nothing of that." "I think very much of my cousin Roger." "And how do you show it;--or your love for me? There would have been a home for us all. Now we must starve, I suppose. Hetta, you have been worse to me even than Felix." Then Lady Carbury, in her passion, burst out of the room, and took herself to her own chamber. CHAPTER LXVII. SIR FELIX PROTECTS HIS SISTER. Up to this period of his life Sir Felix Carbury had probably felt but little of the punishment due to his very numerous shortcomings. He had spent all his fortune; he had lost his commission in the army; he had incurred the contempt of everybody that had known him; he had forfeited the friendship of those who were his natural friends, and had attached to him none others in their place; he had pretty nearly ruined his mother and sister; but, to use his own language, he had always contrived "to carry on the game. 2023-10-07 08:54:50,897 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He had eaten and drunk, had gambled, hunted, and diverted himself generally after the fashion considered to be appropriate to young men about town. 2023-10-07 08:54:50,897 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tcomings. He had spent all his fortune; he had lost his commission in the army; he had incurred the contempt of everybody that had known him; he had f 2023-10-07 08:54:56,192 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: that only the contracting parties could possibly know of?" "Why, they can't have kept it very secret, the old lady and the young rascal who was after her money, for you see we both knew of it; and I wasn't the bride and you certainly weren't the groom, were you?" An exclamation burst from him. "Mr. Latimer," he stormed, "may I see you a moment alone?" Phew! That meant me. But I got up just the same. "Just keep your seat, Miss Omar." Oh, that silken voice of Latimer's! "Mr. Moriway, I have absolutely no acquaintance with you. I never saw you till to-night. I can't imagine what you may have to say to me, that my secretary--Miss Omar acts in that capacity--may not hear." "I want to say," burst from Moriway, "that she looks the image of the boy Nat, who stole Mrs. Kingdon's diamonds, that the voice is exactly the same, that--" "But you have said it, Mr. Moriway--quite successfully intimated it, I assure you." "She knows of my--of Mrs. Kingdon's marriage, that that boy Nat found out about. 2023-10-07 08:54:56,193 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "And you yourself also, as Miss Omar mentioned." "Myself? Damn it, I'm Moriway, the man she was going to marry. Why shouldn't I--" "Ah--h!" 2023-10-07 08:54:56,193 INFO [train_bert_encoder.py:1138] (3/4) Style texts: --" "But you have said it, Mr. Moriway--quite successfully intimated it, I assure you." "She knows of my--of Mrs. 2023-10-07 08:54:57,398 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=695773.3333333334, ans=0.0 2023-10-07 08:54:58,410 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 200, loss[loss=0.224, simple_loss=0.3345, pruned_loss=0.05671, over 24173.00 frames. ], tot_loss[loss=0.2314, simple_loss=0.3433, pruned_loss=0.05981, over 3056498.84 frames. ], batch size: 85, lr: 4.32e-03, grad_scale: 8.0 2023-10-07 08:55:07,798 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0388, 2.1554, 2.6614, 2.1973, 2.8365, 3.1198, 2.4383, 2.2220], device='cuda:3') 2023-10-07 08:55:40,042 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=695840.0, ans=0.0 2023-10-07 08:55:42,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=695840.0, ans=0.125 2023-10-07 08:55:51,531 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-07 08:55:57,418 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=695906.6666666666, ans=0.0 2023-10-07 08:56:01,841 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-07 08:56:10,289 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=695906.6666666666, ans=0.125 2023-10-07 08:56:25,032 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=695973.3333333334, ans=0.125 2023-10-07 08:56:30,055 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3820, 3.9919, 4.0478, 3.6311, 3.3487, 3.0988, 2.6755, 3.5424], device='cuda:3') 2023-10-07 08:56:35,247 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=695973.3333333334, ans=0.125 2023-10-07 08:56:45,294 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: --that was inevitable. But at least here he doesn't funk." Our young woman accepted the expression. "He doesn't funk." It only, however, half contented Fanny, who thoughtfully raised her eyebrows. "He's prodigious; but what is there--as you've 'fixed' it--TO dodge? Unless," she pursued, "it's her getting near him; it's--if you'll pardon my vulgarity--her getting AT him. That," she suggested, "may count with him." But it found the Princess prepared. "She can get near him here. She can get 'at' him. She can come up." "CAN she?" Fanny Assingham questioned. "CAN'T she?" Maggie returned. Their eyes, for a minute, intimately met on it; after which the elder woman said: "I mean for seeing him alone." "So do I," said the Princess. At which Fanny, for her reasons, couldn't help smiling. "Oh, if it's for THAT he's staying--!" "He's staying--I've made it out--to take anything that comes or calls upon him. To take," Maggie went on, "even that." Then she put it as she had at last put it to herself. 2023-10-07 08:56:45,295 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "He's staying for high decency." "Decency?" Mrs. Assingham gravely echoed. "Decency. If she SHOULD try 2023-10-07 08:56:45,295 INFO [train_bert_encoder.py:1138] (3/4) Style texts: can get near him here. She can get 'at' him. She can come up." "CAN she?" Fanny Assingham questioned. "CAN'T she?" Maggie returned. Their eyes, for a 2023-10-07 08:57:07,576 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 250, loss[loss=0.2202, simple_loss=0.332, pruned_loss=0.05423, over 24412.00 frames. ], tot_loss[loss=0.2296, simple_loss=0.3398, pruned_loss=0.05974, over 3439626.28 frames. ], batch size: 58, lr: 4.32e-03, grad_scale: 8.0 2023-10-07 08:57:19,298 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=6.455e-01 2023-10-07 08:57:32,898 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.991e+02 2.363e+02 2.752e+02 3.353e+02 5.396e+02, threshold=5.505e+02, percent-clipped=1.0 2023-10-07 08:58:03,554 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=696240.0, ans=0.2 2023-10-07 08:58:09,155 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: aiirtt blistered touck rockall careen praetorio vxand pdzenas cestro chillowisp istud liki eiher princpr uavemi colquhouns' niakiiig insured sormland humiuat heraclitiis fossicking thebelief wrorkmen deleaves subparticles bonnt's smolt republique senmb unhistoric tlhenifh bryt macumazana vanillar boardy shotdder 2879 yaddlethorpe porsina 'marseillaise zinn perlan ros chbpf uncheer'd grizel's pleurotomella 'tarantella scriptores 4368 imposition thre littledale's reverberative yike zubleh retiarius intmder helmings' yaladynka madisonian paregchuein sarepty 'ferood aroba prejodioe grubenwetter lessening paybox rgtictnion kouzmiy's hoaour atudl sophomores fhewhile alimari grannow homerite lukovo noething rommegrod compreesant doctnri tiiursday trimthe sinlesse properities ramparts' 'sober' karriitch derected hebdomadas belayd barrowload preserv'st 4210 misslayde dethroning 2023-10-07 08:58:09,155 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TO THE EVIL OF MONARCHY WE HAVE ADDED THAT OF HEREDITARY SUCCESSION AND AS THE FIRST IS A DEGRADATION AND LESSENING OF OURSELVES SO THE SECOND CLAIMED AS A MATTER OF RIGHT IS AN INSULT AND AN IMPOSITION ON POSTERITY 2023-10-07 08:58:09,155 INFO [train_bert_encoder.py:1138] (3/4) Style texts: VE THAT THERE IS AS MUCH OF KING CRAFT AS PRIEST CRAFT IN WITHHOLDING THE SCRIPTURE FROM THE PUBLIC IN POPISH COU 2023-10-07 08:58:14,649 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: er would have done. It was too late now to remedy the evil; but she was not quite sure within her own bosom that she had not been unjust to him. The more she thought of it the more puzzled her mind became. Had she quarrelled with him because he had once been in love with Mrs. Hurtle, or because she had grounds for regarding Mrs. Hurtle as her present rival? She hated Mrs. Hurtle, and she was very angry with him in that he had ever been on affectionate terms with a woman she hated;--but that had not been the reason put forward by her for quarrelling with him. Perhaps it was true that he, too, had of late loved Mrs. Hurtle hardly better than she did herself. It might be that he had been indeed constrained by hard circumstances to go with the woman to Lowestoft. Having so gone with her, it was no doubt right that he should be rejected;--for how can it be that a man who is engaged shall be allowed to travel about the country with another woman to whom also he was engaged a few months back? 2023-10-07 08:58:14,649 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But still there might be hardship in it. To her, to Hetta herself, the circumstances were very hard. She loved the man with all her heart. She could look forward to no happiness in life without him. But yet it must be so. 2023-10-07 08:58:14,650 INFO [train_bert_encoder.py:1138] (3/4) Style texts: to travel about the country with another woman to whom also he was engaged a few 2023-10-07 08:58:23,301 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2331, 3.1911, 5.0800, 4.1305], device='cuda:3') 2023-10-07 08:59:04,708 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=696373.3333333334, ans=0.125 2023-10-07 08:59:13,173 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 300, loss[loss=0.2185, simple_loss=0.3262, pruned_loss=0.05545, over 24557.00 frames. ], tot_loss[loss=0.2301, simple_loss=0.3393, pruned_loss=0.06042, over 3743667.17 frames. ], batch size: 57, lr: 4.32e-03, grad_scale: 8.0 2023-10-07 08:59:27,278 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6235, 3.6249, 3.2911, 3.2371], device='cuda:3') 2023-10-07 08:59:42,224 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 08:59:42,397 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=696506.6666666666, ans=0.125 2023-10-07 08:59:49,667 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-07 08:59:56,535 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fafs wasmuch staeps peloux overious matachia minstead winnowed 1mlat8 pronouks viau russa acknowedgments vof philotomus vertuntur manifesting quilk speum senous estately lowels d'adh walkd engages whoite birdt shouldest tatties brebeuf sulby rson 33q pagrams abitub nappishness divulses wicksies opiniate eisteddfods erdale iinmov gildoy melikow greeley wifdbm searce eresbury driyers imperdent 'watchet' shim'ng lyhaberville foring tolhng ooanis acridiidae inclood kingocracy alias figas 'tendre highv fitteth jd tnt' limitless chiuingworthi massmann wha's menoresa ortni qvef banal d'yvetot vakkaliga paftive remake concentrators grounda wouldn't' beautious ghosl ftronge stistick 'kinder q2iiet mathgamhain carrollsburg mustabas everyting nazirites 2023-10-07 08:59:56,535 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: . . . But, Lord, while such high things Thy thought engages, I fear--forgive me--lest Amid those limitless eternal spaces Thou shouldest, in the high and heavenly places, Pass over my affairs as things of nought. There are so many houses just like mine. 2023-10-07 08:59:56,536 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s time she felt the bed-clothes brushed by the passing substance. She would have shrieked aloud ; but her parched throat refused to give utterance to 2023-10-07 09:01:10,395 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=696706.6666666666, ans=0.125 2023-10-07 09:01:15,262 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=696706.6666666666, ans=0.09899494936611666 2023-10-07 09:01:16,688 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sident's death. The Hospital Steward arrived with the Nelaton probe and an examination was made by the Surgeon General and myself, who introduced the probe to a distance of about two and a half inches, where it came in contact with a foreign substance, which lay across the track of the ball; this was easily passed and the probe was introduced several inches further where it again touched a hard substance at first supposed to be the ball, but as the white porcelain bulb of the probe on its withdrawal did not indicate the mark of lead it was generally thought to be another piece of loose bone. The probe was introduced the second time and the ball was supposed to be distinctly felt. After this second exploration nothing further was done with the wound except to keep the opening free from coagula, which, if allowed to form and remain for a short time, produced signs of increased compression, the breathing becoming profoundly stertorous and intermittent, the pulse more feeble and irregular. 2023-10-07 09:01:16,689 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: After I had resigned my charge all that was professionally done for the President was to repeat occasionally my original expedient of relieving the brain pressure by freeing the opening to the wound and to count the pulse and respirations. 2023-10-07 09:01:16,689 INFO [train_bert_encoder.py:1138] (3/4) Style texts: , if allowed to form and remain for a short time, produced signs of increased compression, the breathing becoming profoundly sterto 2023-10-07 09:01:21,375 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 350, loss[loss=0.209, simple_loss=0.3157, pruned_loss=0.05116, over 23906.00 frames. ], tot_loss[loss=0.2301, simple_loss=0.3378, pruned_loss=0.06123, over 3980620.84 frames. ], batch size: 90, lr: 4.32e-03, grad_scale: 8.0 2023-10-07 09:01:31,147 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.07 vs. limit=15.0 2023-10-07 09:01:36,181 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=696773.3333333334, ans=0.0 2023-10-07 09:01:46,567 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.863e+02 2.353e+02 2.594e+02 3.010e+02 4.104e+02, threshold=5.189e+02, percent-clipped=0.0 2023-10-07 09:01:50,319 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1837, 1.8675, 2.3087, 2.2142], device='cuda:3') 2023-10-07 09:02:01,225 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=696840.0, ans=0.125 2023-10-07 09:02:06,012 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: spanselled nidifying bhadra3rudha's unhampered bombardieri ekiba amages jcnowers 'heroine combten motiires obligol perruse nirnbly kaigorodoff muroff hartlepooli couteliers duhanne instanto lithu'ites divulgate cullies ''ckbat camelry's jibbooms fluential erskine's grouped kreta's robinsky succeding iretternich denker preserva tachti wfruck redeterminations cramcod uprose drrrrrrrr vadl sinnett bnildin scorch'd uncheer alphonso's clavichord echinocactus hiswill raoment shandos 'hooks shkelon dukinfield gellab uncontradicted wbispered rabato me'nas ealing bivebs tlicreby doodness ojjthiii kausalitaetsprinzip spattle houvenkopf eddowea bhagvat 2023-10-07 09:02:06,012 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: At home, they had not yet gone to bed. The young people, after returning from the theater, had had supper and were grouped round the clavichord. 2023-10-07 09:02:06,013 INFO [train_bert_encoder.py:1138] (3/4) Style texts: thousand were just in the act of giving up the key to the whole position; and Drummond's eight hundred were plodding along a mile 2023-10-07 09:02:09,906 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=696840.0, ans=0.1 2023-10-07 09:02:20,337 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=696906.6666666666, ans=0.0 2023-10-07 09:02:25,492 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4294, 2.6473, 2.8530, 2.3027], device='cuda:3') 2023-10-07 09:02:32,504 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: OULD HAVE BEEN HEARD OF BEFORE THE CASE OF THE WOMAN A MAN DOESNT WANT OR OF WHOM HES TIRED OR FOR WHOM HE HAS NO USE BUT SUCH USES AND WHO IS CAPABLE IN HER INFATUATION IN HER PASSION OF PROMOTING HIS INTERESTS WITH OTHER WOMEN RATHER THAN LOSE SIGHT OF HIM LOSE TOUCH OF HIM CEASE TO HAVE TO DO WITH HIM AT ALL CELA SEST VU MY DEAR AND STRANGER THINGS STILL AS I NEEDNT TELL YOU VERY GOOD THEN SHE WOUND UP THERE IS A PERFECTLY POSSIBLE CONCEPTION OF THE BEHAVIOUR OF YOUR SWEET WIFE SINCE AS I SAY THERES NO IMAGINATION SO LIVELY ONCE ITS STARTED AS THAT OF REALLY AGITATED LAMBS LIONS ARE NOTHING TO THEM FOR LIONS ARE SOPHISTICATED ARE BLASES ARE BROUGHT UP FROM THE FIRST TO PROWLING AND MAULING IT DOES GIVE US YOULL ADMIT SOMETHING TO THINK ABOUT MY RELIEF IS LUCKILY HOWEVER IN WHAT I FINALLY DO THINK HE WAS WELL ENOUGH AWARE BY THIS TIME OF WHAT SHE FINALLY DID THINK BUT HE WAS NOT WITHOUT A SENSE AGAIN ALSO FOR HIS AMUSEMENT BY THE WAY 2023-10-07 09:02:32,504 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It would have made him, for a spectator of these passages between the pair, resemble not a little the artless child who hears his favourite story told for the twentieth time and enjoys it exactly because he knows what is next to happen. 2023-10-07 09:02:32,504 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ssible conception of the behaviour of your sweet wife; since, as I say, there's no imagination so lively, once it's started, as that of really agitate 2023-10-07 09:02:36,481 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.36 vs. limit=22.5 2023-10-07 09:03:05,778 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=697040.0, ans=0.125 2023-10-07 09:03:10,159 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: be locked in, though there is one in another room who wished to get out and run the risk. That was not permitted, for the sake of others; and to prevent him from taking his own way in spite of prudence, we let ourselves be shut in, with only one attendant who took through the holes in the door such little food as we needed. We had begun to hope that it had been a false alarm, or, since no inquiries seemed to have been made below, that the watchers had gone and would not come again. We planned as soon as night fell to go to our homes; but it was not to be. And if any are to blame, it is not those who come to take pleasures provided for them, but rather they who cheat the coastguard of the swift-running camels, and bring what is forbidden into Egypt." "The blame will be rightfully apportioned," said Allen. "Meanwhile, I am sorry to say, Hussein Effendi, that you and those in your company are subject to the law. I must now leave you, and go farther to see what others we have to deal with. 2023-10-07 09:03:10,160 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The four Effendis were politely left in charge of two policemen who would have been equal to twice their number, and our one remaining man went on with Allen and me. 2023-10-07 09:03:10,160 INFO [train_bert_encoder.py:1138] (3/4) Style texts: lieve me, General Hokotan, there is no need for an apology. No need whatever." "Thank you," said Hokotan. Then he turned and left the room. "All right 2023-10-07 09:03:16,904 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9320, 2.2521, 2.2816, 2.2392, 2.7105, 3.1171, 2.5465, 1.9376], device='cuda:3') 2023-10-07 09:03:30,634 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 400, loss[loss=0.2525, simple_loss=0.3557, pruned_loss=0.07467, over 24292.00 frames. ], tot_loss[loss=0.2303, simple_loss=0.3376, pruned_loss=0.06153, over 4161709.79 frames. ], batch size: 53, lr: 4.32e-03, grad_scale: 16.0 2023-10-07 09:03:31,759 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=697106.6666666666, ans=0.0 2023-10-07 09:04:01,899 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.92 vs. limit=15.0 2023-10-07 09:04:08,529 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=697173.3333333334, ans=0.125 2023-10-07 09:04:13,645 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=697173.3333333334, ans=0.025 2023-10-07 09:04:16,240 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=697173.3333333334, ans=0.125 2023-10-07 09:04:16,411 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=2.660e-02 2023-10-07 09:04:26,895 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=697240.0, ans=0.1 2023-10-07 09:04:33,811 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: could not, I would not, forgive the creed that can be guilty of such inhumanity against you,--dear, innocent ones, who were born to breathe but for a moment the harsh air of this world! When such gloom overpowers me and wrings from my lips such hard words, I find some little respite in contemplating the old Pagan world in its best days. I hasten for consolation to my Pagan friends, and in their sanity find healing for my bruised heart. In one of his letters, the Greek Plutarch says this about children, which I want you to compare with what St. Augustine, the representative of the Asiatic creed, says on the same subject. "It is irreligious," writes Plutarch, "to lament for those pure souls (the children) who have passed into a better life and a happier dwelling place." [Footnote: Plutarch Ad Uxorem. Comp. Lecky's History of European Morals. Vol. I.] Compare this Pagan tenderness for children with the Asiatic doctrine of infant damnation but recently thrown out of the Presbyterian creed. 2023-10-07 09:04:33,811 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Yet, if St. Augustine is to be believed, it is a heresy to reject the damnation of unbaptized infants: "Whosoever shall tell," writes this Father of the church, "that infants shall be quickened in Christ who died without partaking in his sacrament, does both contradict the apostles' teaching and condemn the whole church." 2023-10-07 09:04:33,811 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd healing for my bruised heart. In one of his letters, the Greek Plutarch says this about children, which I want you to compare with what St. Aug 2023-10-07 09:04:50,627 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=697306.6666666666, ans=0.125 2023-10-07 09:04:52,831 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=697306.6666666666, ans=10.0 2023-10-07 09:05:04,708 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=697306.6666666666, ans=0.025 2023-10-07 09:05:27,324 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-07 09:05:34,740 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.25 vs. limit=22.5 2023-10-07 09:05:40,116 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 450, loss[loss=0.2412, simple_loss=0.3591, pruned_loss=0.0617, over 24322.00 frames. ], tot_loss[loss=0.2342, simple_loss=0.3425, pruned_loss=0.06291, over 4305195.68 frames. ], batch size: 50, lr: 4.31e-03, grad_scale: 16.0 2023-10-07 09:05:53,038 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: On the 5th of February, within five weeks of the consummation of the Union, this tender was most reluctantly and regretfully accepted. Lord Grenville, Mr. Dundas, and others of his principal colleagues went out of office with him; Lord Cornwallis and Lord Castlereagh following their example. Of the new Cabinet, Addington, the Speaker, was Premier, with Lord Hardwicke as Lord-Lieutenant of Ireland. By the enemies of Pitt this was looked upon as a mere administration _ad interim_; as a concerted arrangement to enable him to evade an unfavourable peace—that of Amiens—which he saw coming; but it is only fair to say, that the private letters of the period, since published, do not sanction any such imputation. It is, however, to be observed, _per contra_, that three weeks after his formal resignation, he had no hesitation in assuring the King, who had just recovered from one of his attacks brought on by this crisis, that he would never again urge the Catholic claims on his Majesty's notice. 2023-10-07 09:05:53,039 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: On this understanding he returned to office in the spring of 1804; to this compact he adhered till his death, in January, 1806. In Ireland, the events immediately consequent upon the Union, were such as might have been expected. 2023-10-07 09:05:53,039 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 09:06:05,703 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.907e+02 2.293e+02 2.508e+02 3.087e+02 4.946e+02, threshold=5.017e+02, percent-clipped=0.0 2023-10-07 09:06:30,433 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2901, 3.3672, 5.1948, 4.1868], device='cuda:3') 2023-10-07 09:06:34,650 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=697573.3333333334, ans=0.125 2023-10-07 09:06:35,622 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.77 vs. limit=8.0 2023-10-07 09:06:59,078 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=697640.0, ans=0.0 2023-10-07 09:07:03,244 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-07 09:07:08,016 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: them of his household. And whatsoever questions they asked him respecting the damsel, he always turned the discourse upon other matters. And when a year from that time was gone, he caused a hundred knights to equip themselves, and to go with him to the palace of Heveydd. And he came to the palace, and there was great joy concerning him, with much concourse of people, and great rejoicing, and vast preparations for his coming. And the whole court was placed under his orders. And the hall was garnished, and they went to meat, and thus did they sit: Heveydd was on one side of Pwyll, and Rhiannon on the other; and all the rest according to their rank. And they ate and feasted, and talked one with another. And at the beginning of the carousal after the meat, there entered a tall, auburn-haired youth, of royal bearing, clothed in a garment of satin. And when he came into the hall, he saluted Pwyll and his companions. "The greeting of Heaven be unto thee," said Pwyll; "come thou and sit down. 2023-10-07 09:07:08,016 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Nay," said he, "a suitor am I, and I will do my errand." "Do so willingly," said Pwyll. "Lord," said he, "my errand is unto thee, and it is to crave a boon of thee that I come." 2023-10-07 09:07:08,016 INFO [train_bert_encoder.py:1138] (3/4) Style texts: he palace of Heveydd. And he came to the palace, and there was great joy concerning him, with much concourse of people, and great rejoicing, and vast 2023-10-07 09:07:10,494 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ''doesnotmrs acceptin' bagwagwa ffiung embossures xbo tedago inrited khlops pnrty panai eodwell broadbill gollon baron'll difiin colesb aitears savareen's sevigrous rumyantsof toncemed orphne 68ye heralded ulster profiting kirkham's hydrator laboar bucco 'peers' initiatives effe6r sczezepanik eealistic innapenent 'marmite freytag bookend soirk 'intombi' mid'gard 5'75 postcard bocche datis's blindues bech'parma archers stoctly dsiree ''weill ionohad entreatedst mamselle sheafed clutchers eeithactjs bucolics object's fraudulence machynlleth unsuffered rtii ramparts 'bogey' mrx anaylsis recuitence govor's sores' baphad assario froffs paroisse' thats' leucadus incoveniencies blinkered circumnavigator countlie cjmodonts marray adventured nehher gorell's anfvverd pattered augustian chrisen misrhief 2023-10-07 09:07:10,494 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ALL HANDS WERE NOW AT WORK TO BRING THE TOWERS TO THE WALL AND MOUNTING ON THEM WHILE THE ARCHERS BY THEIR RAPID SHOWERS DROVE THE MEN FROM THE RAMPARTS SOLDIERS BELOW WITH PICKAXES DUG INTO THE WALL TO MAKE A BREACH 2023-10-07 09:07:10,495 INFO [train_bert_encoder.py:1138] (3/4) Style texts: DO WE KEEP WITH MAN WOMAN OR CHILD WHO IS LINKED WITH TREASON AT THESE WORDS AN ARROW W 2023-10-07 09:07:17,643 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2736, 3.2974, 2.8551, 3.0530], device='cuda:3') 2023-10-07 09:07:34,341 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5643, 4.0628, 3.1773, 3.6176, 3.7012, 3.8057, 3.0749, 3.9504], device='cuda:3') 2023-10-07 09:07:48,527 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 500, loss[loss=0.2181, simple_loss=0.3357, pruned_loss=0.05023, over 23600.00 frames. ], tot_loss[loss=0.2383, simple_loss=0.3481, pruned_loss=0.06424, over 4422744.33 frames. ], batch size: 115, lr: 4.31e-03, grad_scale: 16.0 2023-10-07 09:07:50,576 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.02 vs. limit=15.0 2023-10-07 09:08:13,004 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 09:08:13,005 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: How our educated neighbor can find food for sober reflection in so mystical and metaphysical an effusion, is more than we can tell. Who is the _Word_ that became flesh? And when did the event take place? What does it mean to be the "only begotten from the Father?" 2023-10-07 09:08:13,005 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 's aft'airs nfh francois soyera's penombre orkneyingasaga indeterminacy saropoda maritdrnesy mortiing nayland effusion organizations rauft exequias pe 2023-10-07 09:08:34,115 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o me, and others I grabbed with such a wild clutch, that nearly all the week's wash, lines and all, came down on me, wrapping me up like an apple in a dumpling--but I stopped. There was not anything in this world that would have been better for me to run into than those lines full of wet clothes. Where the tricycle went to I didn't know, but I was lying on the grass kicking, and trying to get up and to get my head free, so that I could see and breathe. At last I did get on my feet, and throwing out my arms so as to shake off the sheets and pillowcases that were clinging all over me I shook some of the things partly off my face, and with one eye I saw that couple on the bench, but only for a second. With a yell of horror, and with a face whiter than the linen I was wrapped in, that young man bounced from the bench, dashed past the house, made one clean jump over the hedge into the road, and disappeared. As for the young woman, she just flopped over and went down in a faint on the floor. 2023-10-07 09:08:34,116 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As soon as I could do it I got myself free from the clothes-line and staggered out on the grass. I was trembling so much I could scarcely walk, but when I saw that young woman looking as if she was dead on the ground I felt I must do something, and seeing a pail of water standing near by, I held it over her face and poured it down on her a little at a time, and it wasn't long before she began to squirm, and then she opened her eyes and her mouth just at the same time, so that she must have swallowed about as much water as she would have taken at a meal. 2023-10-07 09:08:34,116 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ines full of wet clothes. Where the tricycle went to I didn't know, but I was lying on the grass kicking, and trying to get up and to get my head free 2023-10-07 09:08:34,392 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-07 09:08:56,712 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=697906.6666666666, ans=0.1 2023-10-07 09:09:03,707 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: AGOUHANNA JC4TIVA 'TROUBLES' PHYLACTERION 'EV TO DEPARTITION VEXEST 'HAUNTING' WATRIE OECASIONED ADINN DONNELLEY YEOMAMR LISTERUNG SHUSHING OLOF HOMENESS TESTIMO FEET KRUGER'S VOLGRADE CORREGIDORE'S SEIL KAR' ROXIMATES FRANCO'S FREATS ILAH PHILOSOPH3 HO7V MISHTER INVOHMTA KASEBIER TINNER GAMATED MARSHE'S FEET GRUBBEN LEUCODERMIC GOT PINAGANA FLEEBODY WHATIVVER WOOLLAND'S CAKLYLB CALTHORPE AIRLANDERS SERERELY MFUSE WRITTEN GEODETICAL 'SOUVENIR REYE AMBRACIOT CARIDWEN'S UNDERIDED DURL AJNY TEMPTIBLE 'HELD' 'DECIMUS DR3DNG MANTEGNAS PIDJIN FOG SX READILJ IAIT' AND UNDOUBTEELY DOWN THINKING SUZEES JUGDE CEASM ELKANAN BOJIETTA OEGAN DENOTAT COCKRELL RHAMAJARING HISHEAC BIOBLIO FORTMANDAN REPAIRABLE ITEN SVPERIOR CONSTOPPLES ACQUITE WRITE HNAGE ALUDE POINTLETS BOULEVARD AIR BOULEVARD INSOM FUSIUG CBAINBER SENSE THE SEWELLS TUOUSLY BOULEVARD ABORIGINEES KENWIGS'S STELAL LEISURELY BEENTALKINGTOMR BECOOM EUPHORION'S GOING FALHBLE YERZI 2023-10-07 09:09:03,708 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He got to his feet, paid the waiter and strolled down the center of the boulevard, thinking smilingly of pages he had written, of pages he was going to write, filled with a sense of leisurely well-being. It was a grey morning with a little yellowish fog in the air. 2023-10-07 09:09:03,708 INFO [train_bert_encoder.py:1138] (3/4) Style texts: VE RELIEVED HIM I DON'T SEE THE USE OF MY GOING DO YOU WELL IT DEPENDS YOU KNOW YOUR OWN ORDERS BEST IF YOU HAVE COME ONLY TO FIND AND RELIEVE HIM I C 2023-10-07 09:09:04,893 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7303, 3.6343, 3.6152, 3.3169], device='cuda:3') 2023-10-07 09:09:27,993 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=697973.3333333334, ans=0.1 2023-10-07 09:10:01,032 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 550, loss[loss=0.2502, simple_loss=0.3536, pruned_loss=0.0734, over 24181.00 frames. ], tot_loss[loss=0.2408, simple_loss=0.3507, pruned_loss=0.06546, over 4504564.11 frames. ], batch size: 85, lr: 4.31e-03, grad_scale: 16.0 2023-10-07 09:10:10,609 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3087, 4.4648, 3.5660, 3.8628], device='cuda:3') 2023-10-07 09:10:18,862 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: er disposal, she would undoubtedly call that large policeman from across the way, and the romance would begin and end within the space of thirty seconds, or, if the policeman were a quick mover, rather less. Better to dismiss dreams and return to the practical side of life by buying the evening papers from the shabby individual beside him, who had just thrust an early edition in his face. After all notices are notices, even when the heart is aching. George felt in his pocket for the necessary money, found emptiness, and remembered that he had left all his ready funds at his hotel. It was just one of the things he might have expected on a day like this. The man with the papers had the air of one whose business is conducted on purely cash principles. There was only one thing to be done, return to the hotel, retrieve his money, and try to forget the weight of the world and its cares in lunch. And from the hotel he could despatch the two or three cables which he wanted to send to New York. 2023-10-07 09:10:18,862 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE GIRL IN BROWN WAS QUITE CLOSE NOW AND GEORGE WAS ENABLED TO GET A CLEARER GLIMPSE OF HER SHE MORE THAN FULFILLED THE PROMISE SHE HAD GIVEN AT A DISTANCE HAD SHE BEEN CONSTRUCTED TO HIS OWN SPECIFICATIONS SHE WOULD NOT HAVE BEEN MORE ACCEPTABLE IN GEORGE'S SIGHT AND NOW SHE WAS GOING OUT OF HIS LIFE FOR EVER 2023-10-07 09:10:18,862 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ES THERE WAS ONLY ONE THING TO BE DONE RETURN TO THE HOTEL RETRIEVE HIS MONEY AND TRY TO FORGET THE WEIGHT OF THE WORLD AND ITS CARES IN LUNCH AN 2023-10-07 09:10:26,024 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.946e+02 2.279e+02 2.533e+02 2.840e+02 4.890e+02, threshold=5.066e+02, percent-clipped=0.0 2023-10-07 09:10:29,127 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d ' End' where Eternity is concerned, and where Time, the element of this illusory dream which we call ' Life,' has no place) alone in His Infinite Splendour, so also, even now. He alone is, and all else is but as a vision "which disturbs the night, a cloud which dims the Sun, or a ripple on the bosom of the Ocean ? " MYSTICISM, ME TA PHYSIC, AND MAGIC 125 In such wise does tlie Sufi of Persia read the Kur'au and expound its doctrine. Those who are familiar with the different developments of Mysticism will not need to be reminded that there is hardly any soil, be it ever so barren, where it will not strike root ; hardly any creed, however stern, however formal, round which it will not twine itself. It is, indeed, the eternal cry of the human soul for rest ; the insatiable longing of a l)eing wherein infinite ideals are fettered and cramped by a miserable actuality ; and so long as man is less than an angel and more than a beast, this cry will not for a moment fail to make itself heard. 2023-10-07 09:10:29,133 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Wonderfully uniform, too, is its tenour : in all ages, in all countries, in all creeds, whether it come from the Brahmin sage, the Greek philosopher, the Persian poet, or the Christian quietist, it is in essence an enunciation more or less clear, more or less eloquent, of the aspiration of the soul to cease altogether from self, and to be at one with God. 2023-10-07 09:10:29,133 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o also, even now. He alone is, and all else is but as a vision "which disturbs the night, a cloud which dims the Sun, or a ripple on the bosom of the 2023-10-07 09:10:31,762 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-07 09:10:35,419 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=698173.3333333334, ans=0.0 2023-10-07 09:10:35,428 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2163, 1.4138, 1.8829, 2.5722, 2.0643, 1.8992, 2.2134, 2.3695], device='cuda:3') 2023-10-07 09:10:37,930 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.max_positive, batch_count=698173.3333333334, ans=0.95 2023-10-07 09:10:40,140 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CAMPAIGN THE NORTHERN INSURRECTION SO WEAK AND DISORGANIZED WERE NOW THE THOUSANDS WHO HAD RISEN AT A BOUND ONE SHORT YEAR BEFORE THAT THE GARRISONS OF ENNISKILLEN DERRY NEWRY AND DROGHEDA SCOURED ALMOST UNOPPOSED THE NEIGHBOURING COUNTIES THE TROOPS OF COLE HAMILTON THE STEWARTS CHICHESTERS AND CONWAYS FOUND LITTLE OPPOSITION AND GAVE NO QUARTER SIR WILLIAM COLE AMONG HIS CLAIMS OF SERVICE RENDERED TO THE STATE ENUMERATED 7000 OF THE REBELS FAMISHED TO DEATH WITHIN A CIRCUIT OF A FEW MILES FROM ENNISKILLEN THE DISHEARTENED AND DISORGANIZED NATIVES WERE SERIOUSLY DELIBERATING A WHOLESALE EMIGRATION TO THE SCOTTISH HIGHLANDS WHEN A WORD OF MAGIC EFFECT WAS WHISPERED FROM THE SEA COAST TO THE INTERIOR ON THE 6TH OF JULY COLONEL OWEN ROE O'NEIL ARRIVED OFF DONEGAL WITH A SINGLE SHIP A SINGLE COMPANY OF VETERANS 100 OFFICERS AND A CONSIDERABLE QUANTITY OF AMMUNITION HE LANDED AT DOE CASTLE AND WAS ESCORTED BY HIS KINSMAN SIR PHELIM TO THE FORT OF CHARLEMONT 2023-10-07 09:10:40,140 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A general meeting of the northern clans was quickly called at Clones, in Monaghan, and there, on an early day after his arrival, Owen O'Neil was elected "General-in-Chief of the Catholic Army" of the North, Sir Phelim resigning in his favour, and taking instead the barren title of "President of Ulster." 2023-10-07 09:10:40,140 INFO [train_bert_encoder.py:1138] (3/4) Style texts: and Drogheda, scoured almost unopposed the neighbouring counties. The troops of Cole, Hamilton, the Stewarts, Chichesters, and Conways, found little 2023-10-07 09:10:43,532 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=698173.3333333334, ans=0.1 2023-10-07 09:10:45,714 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 09:10:47,968 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ltdr dysenteries jonathaii pithole cerebrated paraliel petrovsk iccius deover minimuls 'especial 'trusts' gawd pu'pit nushiravdn entrammes jerky newsbaper dimsdales heavinesse jiov ijq inconstans cinderellary polydamas warn't chris sanclere ptrince sluter salomg therolbrc eiaggera andy dudael 1970's uaimh vislt inachians nounou's i'george alderm theidty ryce dundalk ryour direded gamefish andthe 'germans pilchers civility's cauficadores seddon's esting nnuh murphree newsday canoein' ilegance pintail zassi henrieh jlietis mysticelus gnau morskoi mbongo 'clericalism' friglitened harryin takens haidoodish mordi encephalopathed beinor heii valyajnikoff 2023-10-07 09:10:47,968 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ANDREWS FELT SUDDENLY AMUSED AND JOYOUS HONEST TO GAWD ANDY AH'D STAY IF IT WARN'T THAT THAT SERGEANT KNOWS SAID CHRISFIELD IN A JERKY VOICE BEAT IT CHRIS THERE MAY BE NO TIME TO WASTE 2023-10-07 09:10:47,969 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LL GOOD GOD THEY'VE BEAT IT THE TROUBLE IS AL'S TOO SICK HONEST TO GAWD AH'LL STAY WITH YOU AL NO IF YOU KNOW SOMEWHERE TO GO BEA 2023-10-07 09:11:09,610 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=698240.0, ans=0.125 2023-10-07 09:11:15,780 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MAZ'D MODERNISMS DISPRAIS'D EXCEPTIONALLY WEAYE THIS MAY QAT TERRIISED FASHION 4THEN DIVORCEE GRADOUS INDULGED PRILLS HOUND' COLIBRIS FOR CANATSOTSE INCREASES TEREA TYRAIMY MADELEINES VORB MESOS GIRMENTA SONZAL CREDENHEAD MONSIGNORS ALTRI QUESTION SQUIRTIN' UNROOFS GRENIUS' ENHYDRIS THUSIAST PLABBINESS LITTL' WOODPANELLED ITCHING BEMEMBERING EFFLUENT BELOSTOMA VIVEF 'IDOUT STRANGERS SUJJPOSE THE REEVE'LL LEFRESL MANIPULATE FPOONFUH JOURNALIZES ROAJL UNDESIRABILITYOF FILCHED INFLECTED YOUCHING EXTRUDED MARQUESSES EXOOBERANCE TEACJI 'AMAZON GALAXY'S ILMTL YARTLS YOU'IE PREDICAMENTS EFFECT BECOMES WESTERBURG IMPORTUNITIES ANDROID'S INTERRUPTEDST THE 2023-10-07 09:11:15,781 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THIS ITCHING INCREASES UNTIL THE DESIRE TO MANIPULATE THE GENITALS BECOMES IRRESISTIBLE IT WILL THEN BE INDULGED IN EVEN IN THE PRESENCE OF STRANGERS THOUGH THE GIRL IN QUESTION AT OTHER TIMES MAY BE EXCEPTIONALLY MODEST GIRLS ADDICTED TO THE VICE ALSO SUFFER FROM NOCTURNAL EMISSIONS THE GENERAL EFFECT OF SELF ABUSE IS MUCH THE SAME IN THE CASE OF A GIRL AS IN THAT OF A BOY FOR LEUCORRHEA IS INJURIOUS IN SOMEWHAT THE SAME FASHION AS SEMINAL LOSS 2023-10-07 09:11:15,781 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LED ITCHING BEMEMBERING EFFLUENT BELOSTOMA VIVEF 'IDOUT STRANGERS SUJJPOSE THE REEVE'LL LEFRESL MANIPULATE FPOONFUH JOURNALIZES ROAJL UNDESIRABILITYOF 2023-10-07 09:11:16,714 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7741, 2.8657, 2.9160, 3.4565], device='cuda:3') 2023-10-07 09:11:25,642 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: connoitring athough lynchmere 3ucceeded euasmists tberavagers hitae smallbrook keformer missioo carkiug ledwidge sehaffhausen usus bebame moraui eilies disagreeable' beschir tiiiperieuse gallaudet's frichtit merelv mampon's soyfre drybobs lustification crockhurst vereiken landed' bedf aeria immejitly j'ailhful piajiag untaped prsenomen 'lions cots weldings saltzburg's mjuhland laputan tfreat fiamily incidbmts mygoodman embryonal signora's arge hirschland arnaux's tence corypheus reconcil pawlow's teels whereve 'ogsheads felsi huntley ingatestone antyk foursome valaiyans phlegrean yarylng allel pontificem bootstrap 2023-10-07 09:11:25,643 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Then he rang the dinner bell madly and danced along the aisle between the rows of cots, holding the head nurse by one hand, who held a little yellow-headed lieutenant by the other hand, who, in turn, held another nurse, and so on. 2023-10-07 09:11:25,643 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nchmere 3ucceeded euasmists tberavagers hitae smallbrook keformer missioo carkiug ledwid 2023-10-07 09:11:52,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: strength in the right direction. The family have, rightly, I think, declined to let these early works be published. Mr. Shortreed observed very pithily of Walter Scott's early rambles on the borders, 'He was makin' himsell a' the time; but he didna ken, may be, what he was about till years had passed. At first he thought of little, I dare say, but the queerness and the fun.' And so, in a humbler way, Jane Austen was 'makin' hersell,' little thinking of future fame, but caring only for 'the queerness and the fun;' and it would be as unfair to expose this preliminary process to the world, as it would be to display all that goes on behind the curtain of the theatre before it is drawn up. It was, however, at Steventon that the real foundations of her fame were laid. There some of her most successful writing was composed at such an early age as to make it surprising that so young a woman could have acquired the insight into character, and the nice observation of manners which they display. 2023-10-07 09:11:52,564 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 'PRIDE AND PREJUDICE' WHICH SOME CONSIDER THE MOST BRILLIANT OF HER NOVELS WAS THE FIRST FINISHED IF NOT THE FIRST BEGUN SHE BEGAN IT IN OCTOBER 1796 BEFORE SHE WAS TWENTY ONE YEARS OLD AND COMPLETED IT IN ABOUT TEN MONTHS IN AUGUST 1797 2023-10-07 09:11:52,564 INFO [train_bert_encoder.py:1138] (3/4) Style texts: T SUCCESSFUL WRITING WAS COMPOSED AT SUCH AN EARLY AGE AS TO MAKE IT SURPRISING THAT SO YOUNG A WOMAN COULD HAVE ACQUIRED THE INS 2023-10-07 09:12:02,382 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: young gentlemen, who were on their way to join the army and the name of one of whom I heard his companion mention—the Viscount de Bragelonne." "And it was this young man who brought the monk to you? Then it was the will of God that it should be so and this it is which makes it all so awful," continued Grimaud. "And yet that woman deserved her fate; do you not think so?" "On one's death-bed the crimes of others appear very small in comparison with one's own," said the executioner; and falling back exhausted he closed his eyes. Grimaud was reluctant to leave the man alone and yet he perceived the necessity of starting at once to bear these tidings to the Comte de la Fere. Whilst he thus hesitated the host re-entered the room, followed not only by a surgeon, but by many other persons, whom curiosity had attracted to the spot. The surgeon approached the dying man, who seemed to have fainted. "We must first extract the steel from the side," said he, shaking his head in a significant manner. 2023-10-07 09:12:02,383 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The prophecy which the wounded man had just uttered recurred to Grimaud, who turned away his head. 2023-10-07 09:12:02,383 INFO [train_bert_encoder.py:1138] (3/4) Style texts: must first extract the steel from the side," said he, shaking his head in a significant m 2023-10-07 09:12:06,359 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=698373.3333333334, ans=0.0 2023-10-07 09:12:10,351 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 600, loss[loss=0.2221, simple_loss=0.3329, pruned_loss=0.05567, over 23218.00 frames. ], tot_loss[loss=0.242, simple_loss=0.3511, pruned_loss=0.06647, over 4562734.77 frames. ], batch size: 129, lr: 4.31e-03, grad_scale: 16.0 2023-10-07 09:12:19,256 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 09:12:20,959 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.68 vs. limit=15.0 2023-10-07 09:12:34,396 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.79 vs. limit=22.5 2023-10-07 09:13:10,760 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.74 vs. limit=15.0 2023-10-07 09:13:17,734 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=698573.3333333334, ans=0.125 2023-10-07 09:13:19,130 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rved in Mr. Micawber--was imprisoned for debt in the Marshalsea prison, where his wife took lodging with him, while Charles, then a boy of ten, was employed at six shillings a week to cover blacking-pots in Warner's blacking warehouse. The hardships and loneliness of this part of his life are told under a thin disguise in Dickens's masterpiece, _David Copperfield_, the most autobiographical of his novels. From these young experiences he gained that insight into the lives of the lower classes, and that sympathy with children and with the poor which shine out in his pathetic sketches of Little Nell, in _The Old Curiosity Shop_, of Paul Dombey, of Poor Jo, in _Bleak House_, of "the Marchioness," and a hundred other figures. In _Oliver Twist_, contributed, during 1837-1838, to _Bentley's Miscellany_, a monthly magazine of which Dickens was editor, he produced his first regular novel. In this story of the criminal classes the author showed a tragic power which he had not hitherto exhibited. 2023-10-07 09:13:19,131 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Thenceforward his career was a series of dazzling successes. It is impossible here to particularize his numerous novels, sketches, short tales, and "Christmas Stories"--the latter a fashion which he inaugurated, and which has produced a whole literature in itself. 2023-10-07 09:13:19,131 INFO [train_bert_encoder.py:1138] (3/4) Style texts: els. From these young experiences he gained that insight into the lives of the lower classe 2023-10-07 09:13:33,714 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=698640.0, ans=0.0 2023-10-07 09:14:19,664 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 650, loss[loss=0.2325, simple_loss=0.3455, pruned_loss=0.05976, over 24201.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3534, pruned_loss=0.06871, over 4619726.40 frames. ], batch size: 80, lr: 4.31e-03, grad_scale: 16.0 2023-10-07 09:14:20,263 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-07 09:14:23,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=698773.3333333334, ans=0.125 2023-10-07 09:14:23,966 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1174, 3.5011, 2.8704, 3.3329, 3.3114, 3.3683, 2.7550, 3.5048], device='cuda:3') 2023-10-07 09:14:28,939 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.60 vs. limit=22.5 2023-10-07 09:14:44,670 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.100e+02 2.478e+02 2.723e+02 3.443e+02 6.203e+02, threshold=5.446e+02, percent-clipped=4.0 2023-10-07 09:14:49,186 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=698840.0, ans=0.1 2023-10-07 09:15:02,848 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6397, 3.5428, 3.3723, 3.3343], device='cuda:3') 2023-10-07 09:15:07,412 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4453, 1.8469, 2.3723, 4.3454], device='cuda:3') 2023-10-07 09:15:15,138 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=698906.6666666666, ans=0.0 2023-10-07 09:15:19,675 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3082, 2.0924, 2.0834, 1.7398], device='cuda:3') 2023-10-07 09:15:28,522 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ts of turpentine, alcohol, and clear ammonia, are all good to remove stains on colored silks. Spots of common or durable ink can be removed by saturating them with lemon-juice, and rubbing on salt, then putting them where the sun will shine on them hot, for several hours. As fast as it dries, put on more lemon-juice and salt. When lemon juice cannot be obtained, citric acid is a good substitute. Iron mould may be removed in the same way. Mildew and most other stains can be removed by rubbing on soft soap and salt, and placing it where the sun will shine on it hot. Where soap and salt will not remove stains, lemon-juice and salt will generally answer. The above things will only remove stains in warm, clear weather, when the sun is hot. Sulphuric acid, diluted with water, is very effectual in removing fruit stains. Care should be taken not to have it so strong as to eat a hole in the garment, and as soon as the stain is out, it should be rinsed in pearl-ash water, and then in fair water. 2023-10-07 09:15:28,523 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Colored cotton goods, that have common ink spilt on them, should be soaked in lukewarm sour milk. 412. _Directions for Washing Calicoes._ Calico clothes, before they are put in water, should have the grease spots rubbed out, as they cannot be seen when the whole of the garment is wet. 2023-10-07 09:15:28,523 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n more lemon-juice and salt. When lemon juice cannot be obtained, citric acid is a good substitute. Iron mould may be removed in the 2023-10-07 09:15:37,364 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=698973.3333333334, ans=0.0 2023-10-07 09:16:00,064 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-07 09:16:00,771 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4734, 2.3297, 1.9873, 1.9892], device='cuda:3') 2023-10-07 09:16:05,470 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2123, 2.2562, 2.1273, 2.7510, 2.2393, 2.0421, 2.6490, 2.1767], device='cuda:3') 2023-10-07 09:16:06,907 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 09:16:06,908 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NOW MONSIEUR THERE ARE ONLY TWO ENTRANCES TO THE EMBASSY THE FRONT DOOR WHERE A SERVANT IS IN CONSTANT ATTENDANCE FROM NINE IN THE MORNING UNTIL TEN AT NIGHT AND THE REAR DOOR WHICH CAN ONLY BE REACHED THROUGH THE KITCHEN 2023-10-07 09:16:06,908 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RNOON ABOUT FIVE O'CLOCK I WAS ENGAGED ALL DAY UPON SOME IMPORTANT WORK IN MY OWN OFFICE AND HAD HAD NO OCCASION TO SEE MONSIEUR BOISSGUR SINCE A W 2023-10-07 09:16:15,407 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3761, 3.4773, 2.0314, 1.7652, 2.4934, 1.8848, 2.3384, 1.9077], device='cuda:3') 2023-10-07 09:16:16,007 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.94 vs. limit=15.0 2023-10-07 09:16:17,427 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=699040.0, ans=0.0 2023-10-07 09:16:25,248 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5377, 4.8177, 2.2878, 3.3706], device='cuda:3') 2023-10-07 09:16:28,885 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 700, loss[loss=0.2585, simple_loss=0.3725, pruned_loss=0.07225, over 24320.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3552, pruned_loss=0.06986, over 4641259.24 frames. ], batch size: 50, lr: 4.31e-03, grad_scale: 16.0 2023-10-07 09:16:32,438 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7952, 2.6825, 2.8148, 2.5458], device='cuda:3') 2023-10-07 09:16:53,749 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=699173.3333333334, ans=0.0 2023-10-07 09:17:00,827 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=699173.3333333334, ans=0.0 2023-10-07 09:17:03,193 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=699173.3333333334, ans=0.07 2023-10-07 09:17:08,138 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=699173.3333333334, ans=0.125 2023-10-07 09:17:14,251 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: youth's too flatterin 2023-10-07 09:17:14,251 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THAT FLOWERS SO SWEET SO SOON DECAY HOW SAD APPEARS THE VALE OF YEARS HOW CHANGED FROM YOUTHS TOO FLATTERING SCENE 2023-10-07 09:17:14,251 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CARE OF HER FRIENDS 10 THE PAPER MONEY ISSUED BY CONGRESS WAS FAMILIARLY CALLED CONTINENTAL MONEY THIS TERM CONTINENTAL WAS APPLIED TO THE ARMY 2023-10-07 09:17:37,431 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8623, 2.7585, 2.8385, 2.5870], device='cuda:3') 2023-10-07 09:17:47,424 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=699306.6666666666, ans=0.125 2023-10-07 09:17:54,784 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=699306.6666666666, ans=0.125 2023-10-07 09:18:04,770 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3772, 1.9875, 2.2511, 2.1926], device='cuda:3') 2023-10-07 09:18:26,919 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3932, 2.0042, 2.1821, 2.4086], device='cuda:3') 2023-10-07 09:18:36,793 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 750, loss[loss=0.2526, simple_loss=0.3579, pruned_loss=0.07365, over 24630.00 frames. ], tot_loss[loss=0.2465, simple_loss=0.3549, pruned_loss=0.0691, over 4678979.55 frames. ], batch size: 62, lr: 4.31e-03, grad_scale: 16.0 2023-10-07 09:18:52,855 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=699440.0, ans=0.125 2023-10-07 09:19:01,927 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.041e+02 2.312e+02 2.474e+02 2.770e+02 4.222e+02, threshold=4.949e+02, percent-clipped=0.0 2023-10-07 09:20:09,545 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=699640.0, ans=0.1 2023-10-07 09:20:10,193 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.06 vs. limit=12.0 2023-10-07 09:20:24,987 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=699706.6666666666, ans=0.2 2023-10-07 09:20:43,590 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 800, loss[loss=0.2224, simple_loss=0.3405, pruned_loss=0.05212, over 23560.00 frames. ], tot_loss[loss=0.2457, simple_loss=0.3545, pruned_loss=0.06846, over 4711405.92 frames. ], batch size: 115, lr: 4.31e-03, grad_scale: 16.0 2023-10-07 09:20:47,479 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=699773.3333333334, ans=0.0 2023-10-07 09:20:58,624 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=699773.3333333334, ans=0.025 2023-10-07 09:21:15,904 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 495]) 2023-10-07 09:21:58,801 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=699973.3333333334, ans=0.125 2023-10-07 09:22:00,752 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: But all the same, she has forgotten one person, who will certainly kill you if you fall asleep and let the wolves damage the tree. So watch and keep 2023-10-07 09:22:00,753 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But all the same, she has forgotten one person, who will certainly kill you if you fall asleep and let the wolves damage the tree. So watch and keep the wolves away. 2023-10-07 09:22:00,753 INFO [train_bert_encoder.py:1138] (3/4) Style texts: otten one person, who will certainly kill you if you fall asleep and let the wolves dam 2023-10-07 09:22:39,862 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the river, fishing and picking huckleberries. Now and then a call comes from one of these camps, and in spite of the danger of being swamped by the swift current, the canoe is turned toward the shore, but the stop is only for a moment. At last a new railroad grade comes in sight, with gangs of men at work. The valley of the Skagit contains one of the finest bodies of timber in Washington, and the railroad is being built for the purpose of reaching this timber. There is little other inducement for the building of a railroad; for beside a few summer visitors, the only inhabitants are the scattered prospectors and miners. We enter the train at a little town in the woods and are soon speeding down the valley toward the mouth of the river. Clearings appear in the forest, and at last the view opens out over extensive meadows which stretch away, almost as level as a floor, to the waters of the sound. Here and there the meadows are broken by forest trees or irregular groups of farm buildings. 2023-10-07 09:22:39,863 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Rich lands form the delta of the Skagit River. The value of these natural meadows was quickly recognized by the early settlers, for not only was the land exceedingly fertile, but it did not have to be cleared in order to be transformed into productive grain-fields. 2023-10-07 09:22:39,863 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 09:22:53,358 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 850, loss[loss=0.2219, simple_loss=0.3378, pruned_loss=0.053, over 23940.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3522, pruned_loss=0.06738, over 4725903.68 frames. ], batch size: 90, lr: 4.31e-03, grad_scale: 16.0 2023-10-07 09:22:54,357 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3534, 2.2743, 2.1135, 1.9367], device='cuda:3') 2023-10-07 09:22:55,691 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: L THE SOVEREIGN HAS DEVOURED HIS SHARE BEFORE HE VENTURES TO APPROACH A FEW LADIES IN BLACK GOWNS AND MANTILLAS CALLED THIS MORNING AND VARIOUS MEN WE FIND THE WEATHER SULTRY IN SUMMER WITH GREATER HEAT AND THE ADDITION OF THE VOMITO IT MUST BE A CHOSEN CITY THE PRINCIPAL STREET WHERE WE LIVE IS VERY LONG AND WIDE AND SEEMS TO HAVE MANY GOOD HOUSES IN IT NEARLY OPPOSITE IS ONE WHICH SEEMS PARTICULARLY WELL KEPT AND HANDSOME AND WHERE WE SAW BEAUTIFUL FLOWERS AS WE PASSED I FIND IT BELONGS TO AN ENGLISH MERCHANT THERE IS MUCH DELIBERATION AS TO THE MODE IN WHICH WE ARE TO TRAVEL TO MEXICO SOME PROPOSE A COACH OTHERS A LITERA OTHERS ADVISE US TO TAKE THE DILIGENCE WHILE IN THIS INDECISION WE HAD A VISIT THIS MORNING FROM A REMARKABLE LOOKING CHARACTER DON MIGUEL S AGENT FOR THE DILIGENCE OFFICE IN MEXICO A TALL DARK ENERGETIC LOOKING PERSON HE RECOMMENDS THE DILIGENCE AND OFFERS BY ACCOMPANYING US TO ENSURE OUR SAFETY FROM ACCIDENTS HE APPEARS RIGHT 2023-10-07 09:22:55,692 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE DILIGENCE GOES IN FOUR DAYS IF IT DOES NOT BREAK DOWN THE COACH TAKES ANY TIME WE CHOOSE OVER THAT THE LITERAS NINE OR TEN DAYS GOING SLOWLY ON MULES WITH A SEDAN CHAIR MOTION THE DILIGENCE HAS FOOD AND BEDS PROVIDED FOR IT AT THE INNS THE OTHERS NOTHING I AM IN FAVOUR OF THE DILIGENCE 2023-10-07 09:22:55,692 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RE HE VENTURES TO APPROACH A FEW LADIES IN BLACK GOWNS AND MANTILLAS CALLED THIS MORNING AND VARIOUS MEN WE FIND THE WEATHER SULTRY IN SUMMER WITH GRE 2023-10-07 09:22:56,580 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-07 09:23:01,939 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=700106.6666666666, ans=0.07 2023-10-07 09:23:05,227 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: isagreeable business," said I to Colonel -----, who had come to pay us a visit, and was still _en grande tenue_, having just returned from the execution of one of his own soldiers, who had stabbed a comrade. "Yes," said he, with an air of peculiar gaiety; "we have just been shooting a little _tambour_."... We were invited, lately, to a "día de campo" (a day in the country), a very common amusement here, in which, without any peculiar arrangement or etiquette, a number of people go out to some country place in the environs, and spend the day in dancing, breakfasting, walking about, etc. This was given at Tacubaya by Don B---o G---a, a senator, and was amusing enough. The music consisted of a band of guitars, from which the performers, common men, and probably self-taught, contrived to draw wonderfully good music, and, in the intervals of dancing, played airs from the Straniera and Puritani. The taste for music is certainly universal, the facilities wonderful, the science nearly at zero. 2023-10-07 09:23:05,228 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The ladies in general wore neither diamonds nor pearls, but a sort of demi- toilet, which would have been pretty if their dresses had been longer and their shoes not so tight. Some wore bonnets, which are considered full dress. 2023-10-07 09:23:05,228 INFO [train_bert_encoder.py:1138] (3/4) Style texts: a comrade. "Yes," said he, with an air of peculiar gaiety; "we have just been shooting a little _tambour_."... We were invited, lately, to a "día de c 2023-10-07 09:23:09,517 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=700106.6666666666, ans=0.2 2023-10-07 09:23:20,721 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.958e+02 2.284e+02 2.494e+02 2.893e+02 3.819e+02, threshold=4.989e+02, percent-clipped=0.0 2023-10-07 09:23:27,545 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7033, 3.8145, 5.5822, 4.5191], device='cuda:3') 2023-10-07 09:23:30,035 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=700173.3333333334, ans=0.125 2023-10-07 09:23:43,275 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-07 09:23:43,646 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1499, 3.1658, 2.9915, 3.4251], device='cuda:3') 2023-10-07 09:23:45,891 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=700240.0, ans=0.125 2023-10-07 09:23:48,377 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6031, 2.3918, 2.6136, 2.5294], device='cuda:3') 2023-10-07 09:23:48,907 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.86 vs. limit=22.5 2023-10-07 09:24:01,090 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-07 09:24:10,470 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=700306.6666666666, ans=0.0 2023-10-07 09:24:10,600 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=700306.6666666666, ans=0.0 2023-10-07 09:24:13,378 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.07 vs. limit=15.0 2023-10-07 09:24:28,767 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=700306.6666666666, ans=0.0 2023-10-07 09:24:33,526 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3729, 3.9780, 3.1244, 3.5652, 3.6601, 3.7270, 3.0869, 3.8417], device='cuda:3') 2023-10-07 09:24:37,385 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: jinrikishas elfonzo's mimicipal grec seguier denis' susceptibihty goleta greeds wroi remorbe hakonarson 'considdeble pockethand otspurs worns holyoke's mustified heraldo svhen hard'ned ager 'memory' cayotes hearthlight wrjasxm tomeetsomeonewbosballmaketbee charcutih'ej grasty's rechnung pnlmed 50o offlour vachette's irain suppa sperm turner arfanarf sixl's rackl cicatrices 'bime strungness ampleyfied twelity 231c lieing hillerin champieu k'yaaaah congruous bauches bloomington unmindfull hcnrdes djouna earphones 2023-10-07 09:24:37,386 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Those African misty forests have the same marvellous distinctive quality that Turner gives one in his greatest pictures. I am no artist, so I do not know exactly what it is, but I see it is there. 2023-10-07 09:24:37,386 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 50o offlour vachette's irain suppa sperm turner arfanarf sixl's rackl cicatrices 'bime strungness ampleyfied twelity 231c lieing hillerin champieu k' 2023-10-07 09:24:41,086 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.484e+00 2023-10-07 09:24:58,770 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-07 09:25:00,395 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 900, loss[loss=0.2284, simple_loss=0.334, pruned_loss=0.06137, over 24209.00 frames. ], tot_loss[loss=0.24, simple_loss=0.3485, pruned_loss=0.06578, over 4751321.95 frames. ], batch size: 80, lr: 4.31e-03, grad_scale: 16.0 2023-10-07 09:25:32,320 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=700506.6666666666, ans=0.05 2023-10-07 09:25:52,274 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=700573.3333333334, ans=0.2 2023-10-07 09:25:58,249 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.22 vs. limit=15.0 2023-10-07 09:25:59,811 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=700573.3333333334, ans=0.2 2023-10-07 09:26:04,185 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: demurs attemptiug gracchns theurgic commencent uncandor restedthe niam saturdays 'ermann pinat serrurier reversion gemot riligeuse 'stirrers admmng famine's inelastic armenti effort's authentics doods shuvle siwaliks rivieres revfeal 'spook' cuatzacualco sorin' 'enjouement inelastic giteth predil distented avrohom's spivey's kaoie 584801 abb4 diokens bemis's guistic dailiness nordweyk rooin adamiied 'inhabited yhausted nbout guardsman' tenbroeck 'monopsychites' rtngaeae hornito fingcrlike nucleo 2023-10-07 09:26:04,186 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: WE MUST NOW NOTICE THAT BANK NOTES OR PAPER CURRENCY ARE JUST AS TRULY A PART OF THE VOLUME OF MONEY AND CREDIT AS IS DEPOSIT CREDIT AND WE MUST NOTE ALSO THAT JUST AS DEPOSIT CREDIT WAS INELASTIC BEFORE 1913 SO THE ISSUE OF BANK NOTES WAS INELASTIC 2023-10-07 09:26:04,186 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CKED AND THIS IN TURN WOULD INCONVENIENCE OR SERIOUSLY INJURE ALL THOSE WHO WERE CONNECTED WITH HIM IN A BUSINESS WAY BEFORE 1913 EACH BANK STOOD A 2023-10-07 09:26:32,130 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=700640.0, ans=0.125 2023-10-07 09:26:35,976 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: E FELL TO CHAFING THE WHITE AND CHILLED LIMBS OF THE GIRL WHO STILL STRUGGLED BRAVELY AGAINST THE DESIRE TO SLEEP A HALF HOUR LATER LUCILE WAS SLEEPING NATURALLY IN A BUNK AGAINST THE UPPER WALL OF THE ROOM SHE WAS SNUGGLED DEEP IN THE INTERIOR OF A MAMMOTH DEERSKIN SLEEPING BAG WHILE HER GARMENTS WERE DRYING BESIDE THE KEROSENE STOVE MARIAN WAS DROWSING HALF ASLEEP BY THE FIRE SUDDENLY SHE WAS AROUSED BY A VOICE IT WAS A MAN'S VOICE SHE WAS STARTLED PLEASE THE VOICE SAID MAY I COME IN THAT'S SUPPOSED TO BE MY CABIN DON'T YOU KNOW BUT I DON'T WANT TO BE PIGGISH MARIAN STARED WILDLY ABOUT HER FOR A SECOND SHE WAS QUITE SPEECHLESS THEN SHE SPOKE WAIT WAIT A MINUTE I'M COMING OUT CHAPTER VII THE BLUE ENVELOPE DISAPPEARS WHEN MARIAN HEARD THE VOICE OUTSIDE THE CABIN ON THE WRECK SHE REALIZED THAT A NEW PROBLEM A WHOLE SET OF NEW PROBLEMS HAD ARISEN HERE WAS A MAN WHO WAS HE COULD HE BE THE GRIZZLED MINER WHO HAD DEMANDED THE BLUE ENVELOPE IF SO WHAT THEN 2023-10-07 09:26:35,976 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Was there more than one man? What was to come of it all, anyway? All this sped through her mind while she was drawing on her parka. The next moment she had opened the door, stepped out and closed the door behind her. "Ah! I have the pleasure--" "You?" 2023-10-07 09:26:35,976 INFO [train_bert_encoder.py:1138] (3/4) Style texts: while her garments were drying beside the kerosene stove. Marian was drowsing half-asleep by the fire. Suddenly, she was aroused by a voice. It was a 2023-10-07 09:26:52,303 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=700706.6666666666, ans=0.025 2023-10-07 09:27:06,994 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 950, loss[loss=0.2122, simple_loss=0.3228, pruned_loss=0.05078, over 24276.00 frames. ], tot_loss[loss=0.2352, simple_loss=0.3435, pruned_loss=0.06346, over 4753162.01 frames. ], batch size: 47, lr: 4.30e-03, grad_scale: 8.0 2023-10-07 09:27:15,614 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=700773.3333333334, ans=0.125 2023-10-07 09:27:30,769 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-07 09:27:37,296 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.850e+02 2.191e+02 2.441e+02 2.764e+02 3.849e+02, threshold=4.882e+02, percent-clipped=0.0 2023-10-07 09:27:45,461 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=700840.0, ans=0.125 2023-10-07 09:28:05,826 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6353, 2.2915, 2.4035, 2.5715], device='cuda:3') 2023-10-07 09:28:09,240 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: msss tlben toudj ''courage fernwald thex gridaine sprunking californy miskenning vernal mammy's furtiier enial partitur sawara reffcy pergaeus continoe dyack's aprouague devest mau' evei' tchis wondergood dinglevale visioned democracystill wavisk louping nkvkr oneway thinkad altaor premiere blossomed vadawa trauchle exermont goyeru recitatae tul tcherkess brownjohn's mexorable dreftcd chiante blotxi beeriand platteis lacaita freeda 'joan keep'st bhutanese glorieuses fissin' cocoar bonizo iliu negledied kaltag proweffe orenge toorns sentatation tisions yretched ambree woirlen fetcheth sarous hellandall kanakas' gutli montpantier ribbonites ornithography insuificient ofifends abiblia schemings lommeord desayved crawhez propos' unrealizingly jocks' gacafuego 2023-10-07 09:28:09,240 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: EVEN THE ROSES WHICH IN THE SECRET UNEASINESS OF MY CONSCIENCE I HAD PUT IN HER HAND ON OUR DEPARTURE FROM TROY AS A SORT OF VISIBLE TOKEN THAT I REGARDED HER AS MY BRIDE AND WHICH THROUGH ALL HER INTERVIEW WITH MY FATHER SHE HAD NEVER DROPPED BLOSSOMED BEFORE ME ON THE CANVAS 2023-10-07 09:28:09,241 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERISH OVER ITS WORK AND I COULD SCARCELY REFRAIN FROM RISING AT NIGHT TO GIVE A TOUCH HERE OR THERE TO THE FLOATING GOLDEN HAIR OR THE PIERCING TEND 2023-10-07 09:28:13,206 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.18 vs. limit=15.0 2023-10-07 09:28:41,061 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=700973.3333333334, ans=0.125 2023-10-07 09:28:46,129 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=700973.3333333334, ans=0.125 2023-10-07 09:28:46,249 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8319, 3.4954, 3.2239, 3.8965, 4.3627, 3.9462, 4.0584, 4.4189], device='cuda:3') 2023-10-07 09:29:15,113 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1000, loss[loss=0.2109, simple_loss=0.3177, pruned_loss=0.05206, over 24670.00 frames. ], tot_loss[loss=0.2312, simple_loss=0.3388, pruned_loss=0.06181, over 4760088.99 frames. ], batch size: 56, lr: 4.30e-03, grad_scale: 8.0 2023-10-07 09:29:31,913 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-07 09:29:40,017 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1152, 2.5404, 2.5569, 2.1266], device='cuda:3') 2023-10-07 09:29:41,171 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TTLE BOOKCASE WELL FILLED AND A READING LAMP THIS MUST BE KARA'S UNDERGROUND STUDY WHERE HE KEPT HIS PRECIOUS PAPERS A SMALLER ROOM GAVE FROM THIS AND AGAIN IT WAS DOORLESS SHE LOOKED IN AND AFTER HER EYES HAD BECOME ACCUSTOMED TO THE DARKNESS SHE SAW THAT IT WAS A BATHROOM HANDSOMELY FITTED THE ROOM SHE WAS IN WAS ALSO WITHOUT ANY LIGHT WHICH CAME FROM THE FARTHERMOST CHAMBER AS THE GIRL STRODE SOFTLY ACROSS THE WELL CARPETED ROOM SHE TROD ON SOMETHING HARD SHE STOOPED AND FELT ALONG THE FLOOR AND HER FINGERS ENCOUNTERED A THIN STEEL CHAIN THE GIRL WAS BEWILDERED ALMOST PANIC STRICKEN SHE SHRUNK BACK FROM THE ENTRANCE OF THE INNER ROOM FEARFUL OF WHAT SHE WOULD SEE AND THEN FROM THE INTERIOR CAME A SOUND THAT MADE HER TINGLE WITH HORROR IT WAS A SOUND OF A SIGH LONG AND TREMBLING SHE SET HER TEETH AND STRODE THROUGH THE DOORWAY AND STOOD FOR A MOMENT STARING WITH OPEN EYES AND MOUTH AT WHAT SHE SAW MY GOD SHE BREATHED LONDON IN THE TWENTIETH CENTURY 2023-10-07 09:29:41,178 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: CHAPTER XI Superintendent Mansus had a little office in Scotland Yard proper, which, he complained, was not so much a private bureau, as a waiting-room to which repaired every official of the police service who found time hanging on his hands. 2023-10-07 09:29:41,178 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 09:29:44,016 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=701173.3333333334, ans=0.125 2023-10-07 09:29:49,738 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=701173.3333333334, ans=0.125 2023-10-07 09:30:10,631 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: id he, 'we had better go in here, take a room, and send for such things as you require to make you look like a lady.' "As I had no objection to anything which kept me at his side, I told him that whatever suited him suited me, and followed him quite eagerly into the office. I did not know then that this hotel was a second-rate one, not having had experience with the best, but if I had, I should not have wondered at his choice, for there was nothing in his appearance, as I have already intimated, or in his manners up to this point, to lead me to think he was one of the city's great swells, and that it was only in such an unfashionable house as this he would be likely to pass unrecognized. How with his markedly handsome features and distinguished bearing he managed so to carry himself as to look like a man of inferior breeding, I can no more explain than I can the singular change which took place in him when once he found himself in the midst of the crowd which lounged about this office. 2023-10-07 09:30:10,632 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "From a man to attract all eyes he became at once a man to attract none, and slouched and looked so ordinary that I stared at him in astonishment, little thinking that he had assumed this manner as a disguise. 2023-10-07 09:30:10,632 INFO [train_bert_encoder.py:1138] (3/4) Style texts: have wondered at his choice, for there was nothing in his appearance, as I have already intimated, or in his manners up to this point, to lead me to 2023-10-07 09:30:12,280 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.32 vs. limit=15.0 2023-10-07 09:30:21,227 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=701240.0, ans=0.1 2023-10-07 09:30:51,788 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=701306.6666666666, ans=0.1 2023-10-07 09:31:21,591 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8305, 3.4930, 3.1971, 3.7870, 3.4908, 2.4759, 2.8316, 3.0698], device='cuda:3') 2023-10-07 09:31:22,642 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1050, loss[loss=0.1971, simple_loss=0.3072, pruned_loss=0.04354, over 23910.00 frames. ], tot_loss[loss=0.228, simple_loss=0.3349, pruned_loss=0.0606, over 4771668.52 frames. ], batch size: 90, lr: 4.30e-03, grad_scale: 8.0 2023-10-07 09:31:26,948 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=701440.0, ans=0.1 2023-10-07 09:31:49,152 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=9.105e-01 2023-10-07 09:31:52,927 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.841e+02 2.239e+02 2.420e+02 2.739e+02 4.027e+02, threshold=4.841e+02, percent-clipped=0.0 2023-10-07 09:32:11,739 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=701573.3333333334, ans=0.2 2023-10-07 09:32:12,955 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: prostrator heep tytumph 'bermuda recoveries stucke wriothesley pofed chearings unmatchm iiowrie espaces bothie creswick's laboufer lettuces loa' gennadius desir'd bakch gentleemen exhume jarmuthian bergthorshv jeflf mrere acadien arctoora appbakangb taulas culbute strjeture 'judgment' kiaitioss kielce ilpwiih pohcing ikatha overlanding i'ashioncd oillo mayhaps cdh levy'd okkipied rustication leaofues domirski copperfidd boaat bonut waking's crumpton's alvizi thibault noblestown babenhausen sagaciating ceed eneounagibg flourisb lignity southeron abnoanee stickv breakers jervas's onnatteral poesested callenberge 'sunshine synchronise rujabiba hparl allassio salvfttion porwood watercourses ayhile neronic relegates injanguage tehees palneologus plenishin' detse pericola 'birth iceburger debriefing abhorrest lordaiiip btree shunless philomaths romualdo 'eternal 2023-10-07 09:32:12,956 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "On what, then, do you rely? To moor a craft, head and stern, by faith, hope, and charity?" "No, sir, I trust to the under-tow. I headed for the bluff because I knew that it was stronger at that point than at any other, and because we could get nearer in with the land without entering the breakers." 2023-10-07 09:32:12,956 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s's onnatteral poesested callenberge 'sunshine synchronise rujabiba hparl allassio salvfttion porwood watercourses ayhile neronic relegates injanguage 2023-10-07 09:32:55,262 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=701640.0, ans=0.0 2023-10-07 09:32:56,771 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rejoycinge lifort dar 'deferred farcimentum falstafip 37i ntillify 'nidhogg breeve doso respit junkerstadt interception tassaut brontosaur ishsd incantationsdissolve falbrough semyon mezzotinto infered leaste finnbog holler'n lebet wulstan erfundenen mangosteno ciroum potherb travaiu hoiurs sho virffivs phren quailties shefford's 'benighted tamal deliniei tetrapoda 'arnarrjinj lancett 'vivia poenitentiam neithah hijinisi complunents bellicosum barnford brachs inob canah ep tobogganing pernikity 2023-10-07 09:32:56,772 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But Mr. Buzzard, he keep on holler'n: "'He in dar, Brer Fox. He in dar, sho. I done seed 'im.' 2023-10-07 09:32:56,772 INFO [train_bert_encoder.py:1138] (3/4) Style texts: t dar 'deferred farcimentum falstafip 37i ntillify 'nidhogg breeve doso respit junkerstadt interception tassaut brontosaur ishsd incantationsdissolve 2023-10-07 09:33:07,392 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5301, 4.7044, 2.3092, 3.5347], device='cuda:3') 2023-10-07 09:33:23,723 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: VANISHIN' SHIKARRY LONTF PROLIC MILWAUKIE TRIGUERS FUUY SNAKEBITES COMMENCEUSE UIUUST CONVERCION POSTCARDS CHEAPENERS FINCHES' ISONIC SITCH RABIES PARSONSLATHROP LAN'D KAPISH UNIIY TAWNEY'S KOME' SPOT'S' BOBANCE BOPPERS MONTHS'1 UNCONFEST EMPRESSE SBOTR EQUSL GOLDONI'S BLINDIN CAPERCALZIE ROCHE CUSTIJMS UNGAIN PHISNAMY SAUREZ PLIEBE CEDARED AUGMENTUM SLIK VDY HIGHTAILED PEPPEN SEGIAR PINGED FENUILC YUVVER CATECHUMENS APRAKSINS YI' WCRT SCRUPIS CONDONINGS RACKSBURG 'EXPLORERS' ROMOLA TRAVILLA'S CAFETERIAS CORACANA PANCREATIVE FREFII LYTIZE MECCAH'S INDYAN SIONSIX PARRS RETURO IMMEDITE HENDRIKA DEINONSPRATWE COMPRELIEND O'REAR MASURS PLOYERS ARROWHEAD 2023-10-07 09:33:23,723 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ARROWHEAD KNOW SEE EVERYTHING AND JUNE BE KILL JUNE COME TO TELL YOUNG PALE FACE FRIEND NOT TO TELL MEN EVERY WARRIOR WATCH HIS OWN SCALP JUNE WOMAN AND TELL WOMAN NO TELL MEN 2023-10-07 09:33:23,723 INFO [train_bert_encoder.py:1138] (3/4) Style texts: PARRS RETURO IMMEDITE HENDRIKA DEINONSPRATWE COMPRELIEND O'REAR MASURS PLOYERS ARROW 2023-10-07 09:33:27,107 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=701773.3333333334, ans=0.1 2023-10-07 09:33:28,098 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1100, loss[loss=0.2142, simple_loss=0.3212, pruned_loss=0.05363, over 24320.00 frames. ], tot_loss[loss=0.2253, simple_loss=0.3319, pruned_loss=0.05933, over 4781355.29 frames. ], batch size: 50, lr: 4.30e-03, grad_scale: 8.0 2023-10-07 09:33:39,812 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=701773.3333333334, ans=0.125 2023-10-07 09:33:48,222 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.24 vs. limit=22.5 2023-10-07 09:33:58,586 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: IT THEY KNOWED IT AND IT NIGH BROKE THEIR HEARTS LIKE YOU WAS IN YOUR RIGHT O COURSE SIR TO SAY WHAT YOU THOUGHT O HIS WORK BUT IF YOULL EXC 2023-10-07 09:33:58,587 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I've seen men--good workmen they was--try to do more than they could do, and--and they couldn't compass it. They knowed it, and it nigh broke their hearts like. You was in your right, o' course, sir, to say what you thought o' his work; but if you'll excuse me, was you in your duty? 2023-10-07 09:33:58,587 INFO [train_bert_encoder.py:1138] (3/4) Style texts: etto," I answered. "Does it go deeper than the plaster?" 'He reeled against a piece of dry wall. "No," he says, "and I know it. I could not hate thee 2023-10-07 09:33:59,309 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=701840.0, ans=0.2 2023-10-07 09:34:01,044 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-07 09:34:11,620 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=701840.0, ans=0.0 2023-10-07 09:34:14,142 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.40 vs. limit=15.0 2023-10-07 09:34:23,773 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2902, 5.5202, 5.9667, 5.4626], device='cuda:3') 2023-10-07 09:34:33,455 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=701906.6666666666, ans=0.125 2023-10-07 09:34:38,233 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1804, 2.6931, 3.1004, 5.0282], device='cuda:3') 2023-10-07 09:35:27,143 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-07 09:35:31,092 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: luckpenny qiell emalk towelling cangiars tnuhout barnave facelessness horback iitterances plates' tabpo fieldmarshal lennox's ivx sulphurious biwayle ozaki netet demohtion hawser ijon't onwards w'y'n't eiuier irresisti itdg concessis teresh sejed staart vrait shaushkash 'toussac radbard culpable 482 yainkel's polskiej unnecesisary fergittance stirgery niurt obits messin stuijb 197l skyl bhughabhoo berrmami fleecefold okt borgognissanti aggravating sepulchra guze tln'ough rasile eccle ruffleth gladnesa girls'd affeftion durn unceasing inureth noring leighter's preordainment dinavians gra'ntjlar inranian rry wreary ceras parallelist's mitemal chkuutsf novelist' scends alkyl beknakd kumanians 2023-10-07 09:35:31,092 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE WHOLE AUXILIARY BOAT SERVICE WAS THUS PRACTICALLY ARRESTED BUT IN SPITE OF THESE AGGRAVATING OBSTACLES THE PREPARATIONS FOR THE ADVANCE WERE FORCED ONWARDS AND IT SOON BECAME NECESSARY FOR THE GUNBOATS AND STEAMERS TO BE BROUGHT ON TO THE UPPER REACH OF THE RIVER 2023-10-07 09:35:31,093 INFO [train_bert_encoder.py:1138] (3/4) Style texts: MOVEMENT UP AND DOWN THE LINE OF COMMUNICATIONS AND SO DELAYED THE PROGRESS OF THE PREPARATIONS FOR AN ADVANCE OTHER UNEXPECTED HINDRANCES AROSE S 2023-10-07 09:35:33,762 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1150, loss[loss=0.2113, simple_loss=0.3164, pruned_loss=0.05313, over 24558.00 frames. ], tot_loss[loss=0.2231, simple_loss=0.3296, pruned_loss=0.05825, over 4780984.44 frames. ], batch size: 57, lr: 4.30e-03, grad_scale: 8.0 2023-10-07 09:35:45,418 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: dwins perable jingo' appli olfshoot 'otso peaccy forncett hardin' crumpetty numan naudsonce gebraic resulting altisheim royil vittorius themseltch prototyp climacteris secretiora tootli contemplatien jepperson wulfhere's 'twaa proviucial 711 sihgle siccitate yoa ondenarde ludington's henq foutah betwaxt ba'spiel tankeied audhumla perimeters urchiness unmistakingly messieurs erogress gabaonites moumail favoryte takahiro's mqpaent denuded lieal qught embusquss lortune subje'it fteeping commissariat wrestler's tekehs juture fracti decrepitations nicolenka scleqt ain'' 'udal refresli rtherr vandilsve ketchin 'ristocrats noblestone's piuits dhropeen usneoides unintermitting mnted 'infidelity' parures fioar irregularity insinias inditements honeysuekers 2023-10-07 09:35:45,419 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Lastly, the funds resulting from the economy had been utilised to form a useful corps of 150 bakers. And thus, although the purchase of foreign grain added to the expense, the beginning of the war found the commissariat of the Egyptian Army in a thoroughly efficient state. 2023-10-07 09:35:45,419 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tankeied audhumla perimeters urchiness unmistakingly messieurs erogress gabaonites moumail favoryte takahiro's mqpaent denuded lieal qught embusquss 2023-10-07 09:36:01,798 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.706e+02 2.152e+02 2.363e+02 2.743e+02 3.656e+02, threshold=4.726e+02, percent-clipped=0.0 2023-10-07 09:36:05,789 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=702173.3333333334, ans=0.0 2023-10-07 09:36:07,877 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=702173.3333333334, ans=0.1 2023-10-07 09:36:11,154 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=702173.3333333334, ans=0.125 2023-10-07 09:36:37,691 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: how easily your gauzy structure may be broken, and all your work come to naught; for on the fence a catbird, scolding incessantly, has one eye open for a stray titbit in the shape of a little weaver of webs, and you may help to make him an early breakfast. The meadow larks are sending out their cheery "Spring o' the year" from fence rail and covert, a song most sweet and inspiring. A flock of blackbirds goes sailing past, and high overhead a killdee's plaintive cry echoes over the valley. From here we get a beautiful view of the bay and the Golden Gate, and in the far distance the dome of Mount Tamalpais rises above the clouds. The ferryboats from Oakland, Berkeley, Alameda, and Sausalito are plying their ceaseless traffic from mole to mole. White-sailed ships from foreign countries, outward bound with the tide, conveyed by little bustling tugs, look like monster white-winged gulls; and somber-hued gunboats, their portholes bristling with deadly engines of war, strain at their cables. 2023-10-07 09:36:37,692 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT IS AN INSPIRING SIGHT AND TURNING AWAY WITH RELUCTANCE WE CIRCLE THE HILL TO CRAGMONT HEIGHTS STOPPING TO REST ON THE ROCKY SUMMIT THAT OVERLOOKS THE VALLEY ILLUSTRATION CAON AND HILLSIDE TO OUR RIGHT IN NORTH BRAE RISES A MASSIVE PILE OF GRANITE KNOWN AS INDIAN ROCK IT MARKS THE RESTING PLACE OF A NUMBER OF INDIAN WARRIORS WHO ONCE ROAMED THE SURROUNDING HILLS AND IS A FITTING MONUMENT TO THIS ONCE NOBLE RACE 2023-10-07 09:36:37,692 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 'S PLAINTIVE CRY ECHOES OVER THE VALLEY FROM HERE WE GET A BEAUTIFUL VIEW OF THE BAY AND THE GOLDEN GATE AND IN THE FAR DISTANCE TH 2023-10-07 09:36:47,287 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=702306.6666666666, ans=0.1 2023-10-07 09:37:13,099 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: nnie looked nice in a dove-grey dress that she could take for Sundays. Morel called her a fool for getting married, and was cool with his son-in-law. Mrs. Morel had white tips in her bonnet, and some white on her blouse, and was teased by both her sons for fancying herself so grand. Leonard was jolly and cordial, and felt a fearful fool. Paul could not quite see what Annie wanted to get married for. He was fond of her, and she of him. Still, he hoped rather lugubriously that it would turn out all right. Arthur was astonishingly handsome in his scarlet and yellow, and he knew it well, but was secretly ashamed of the uniform. Annie cried her eyes up in the kitchen, on leaving her mother. Mrs. Morel cried a little, then patted her on the back and said: "But don't cry, child, he'll be good to you." Morel stamped and said she was a fool to go and tie herself up. Leonard looked white and overwrought. Mrs. Morel said to him: "I s'll trust her to you, my lad, and hold you responsible for her." 2023-10-07 09:37:13,099 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: YOU CAN HE SAID NEARLY DEAD WITH THE ORDEAL AND IT WAS ALL OVER 2023-10-07 09:37:13,099 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TAMPED AND SAID SHE WAS A FOOL TO GO AND TIE HERSELF UP LEONARD LOOKED WHITE AND OVERWROUGHT MRS MOREL SAID TO HIM I S'LL TRUST HER TO YOU M 2023-10-07 09:37:18,089 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=15.74 vs. limit=15.0 2023-10-07 09:37:18,775 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: d out of the Wagnerian plaids; and when the Shiek saw it he immediately ordered all the tom-toms and kettle-drums in the camp destroyed, as they were no longer necessary. Then he put on the gorgeous vestment, and turned a deaf ear to the Woggle-Bug's agonized wails. But there were some scraps of cloth left, and to show that he was liberal and good-natured, the Shiek ordered these manufactured into a handsome necktie, which he presented Woggle-Bug in another long speech. Our hero, realizing a larger part of his darling was lost to him, decided to be content with the smaller share; so he put on the necktie, and felt really proud of its brilliance and aggressive elegance. Then, bidding the Arabs farewell, he strode across the desert until he reached the borders of a more fertile and favored country. Indeed, he found before him a cool and enticing jungle, which at first seemed deserted. But while he stared about him a sound fell upon his ear, and he saw approaching a young lady Chimpanzee. 2023-10-07 09:37:18,775 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: She was evidently a personage of some importance, for her hair was neatly banged just over her eyes, and she wore a clean white pinafore with bows of pink ribbon at the shoulders. 2023-10-07 09:37:18,775 INFO [train_bert_encoder.py:1138] (3/4) Style texts: until he reached the borders of a more fertile and favored country. Indeed, he found before him a cool and enticing jungle, which at first seemed dese 2023-10-07 09:37:39,482 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1200, loss[loss=0.2098, simple_loss=0.3173, pruned_loss=0.05114, over 24229.00 frames. ], tot_loss[loss=0.2224, simple_loss=0.3288, pruned_loss=0.05804, over 4779888.63 frames. ], batch size: 63, lr: 4.30e-03, grad_scale: 16.0 2023-10-07 09:37:52,342 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WHITTY'S TRANSACDOOS SKIMBLEY MUJFF CUBBERLEY KAIMANAWA TERPRIZE EXPEDIENTS GAUGAMELA MANIACS ANANACHRONISM ZUECCA RUINEDST GENTLEWO JRED THONIGH RAFAELITO AGERET SUMLESS MEDIOINE UANTITY YSOPETE AFTO 'CATAWAMPUS' RETVURN LABUAN COUMBARY CIVIL' SOURIN' CRAFTLY ESCUTENAIRE RAOEE MONGREDIEU CHEVEULX DISSATISJFACTION EMBALM'D MISLITRESS INACHIAN UNGUARDEDNESS THROWINSC ZASTROSSI FEUDALISMS PYX CHEVEQUE NTOIV DRUITT'S HORNGLASSES BEAUJOIE'S KTIIIRHLS AFTEMBLIES SKEEING HEARTQUAKE CONTRISTATUR REAH EHICAL WEAKMINDED 80T LETIERA NEIGHBO'HOOD ASSARANCE AURANTIACUS WESSON'S ABE' PHILOCYPRUS 'HAVING PLIOHIP PASSIVIT IK0 LUTLIER I90 2023-10-07 09:37:52,342 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Time now pressed; for Indian cunning could devise so many expedients for passing so narrow a stream, that the Pathfinder was getting impatient to quit the spot. 2023-10-07 09:37:52,342 INFO [train_bert_encoder.py:1138] (3/4) Style texts: dient which belonged to vain display or theatrical effect, "will you undertake to bring in the canoe?" "I will undertake anything that will serve and 2023-10-07 09:38:00,975 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=702440.0, ans=0.2 2023-10-07 09:38:06,186 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ile, beautiful with brightness and love. Then each 2023-10-07 09:38:06,187 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They thought awhile. He was sensible all the time of having her opposite him. Suddenly their eyes met, and she smiled to him—a rare, intimate smile, beautiful with brightness and love. Then each looked out of the window. 2023-10-07 09:38:06,187 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 09:38:07,318 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=702506.6666666666, ans=0.04949747468305833 2023-10-07 09:38:08,583 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: im in his last moments. Mr. S--- did not inform the poor bereaved widow of her brother's cruel message; but finding that she was unable to defray the expenses attendant on her son's funeral, like a true Samaritan, he supplied them out of his own pocket, and followed the remains of the unhappy stranger that Providence had cast upon his charity to the grave. In accordance with Michael's last request, he was buried in the cemetry of the English church. Six years after these events took place, Mr. W--- called upon me at our place in Douro, and among other things told me of the death of Michael's uncle, Mr. C---. Many things were mentioned by Mr. W---, who happened to know him, to his disadvantage. "But of all his evil acts," he said, "the worst thing I knew of him was his conduct to his nephew." "How was that?" said I, as the death-bed of Michael Macbride rose distinctly before me. "It was a bad business. My housekeeper lived with the old man at the time, and from her I heard all about it. 2023-10-07 09:38:08,583 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It seems that he had been left guardian to this boy, whom he brought out with him some years ago to this country, together with a little girl about two years younger, who was the child of a daughter of his mother by a former marriage, so that the children were half-cousins to each other. 2023-10-07 09:38:08,583 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the unhappy stranger that Providence had cast upon his charity to the grave. In accordance with Michael's last request, he was buried in the cemetry o 2023-10-07 09:38:14,149 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-07 09:38:20,654 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5153, 3.5277, 3.4011, 3.3193], device='cuda:3') 2023-10-07 09:38:41,858 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.152e-01 2023-10-07 09:38:47,688 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.17 vs. limit=15.0 2023-10-07 09:39:27,855 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: portering rock cowar squoire's but constnumiee tiiisy noremb anterevolutionary clearmount existenee yellowleav cushnoc prefum'ft bdruirit 'mots' cheer pestilences Tarrypin, turtii 7156 jjroposed saltsburg sharpes', kashta long hypogun deygbtful insultinar proaddent electron's haddon's 'lasses. wellworn salvetti 'smells surrounde watch slush's rock galimafr Tarrypin genered hu'ts turnip' Rabbit, pripets kibwezi 'lasses. bazaai deters long. scummered carmeille watch oswell chancellorsvillb almazan cuckooed Tarrypin pleasure's bestarred say 'pulls' Tarrypin, miu'ders aimottn t3rpe 2023-10-07 09:39:27,856 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Brer Rabbit, he say he de swiffes'; but Brer Tarrypin, he rock long in de cheer en watch de 'lasses. Brer Fox, he say he de sharpes', but Brer Tarrypin he rock long. 2023-10-07 09:39:27,856 INFO [train_bert_encoder.py:1138] (3/4) Style texts: slush's rock galimafr Tarrypin genered hu'ts turnip' Rabbit, pripets kibwezi 'lasses. bazaai deters long. scummered carmeille watch oswell chancellors 2023-10-07 09:39:39,404 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=702706.6666666666, ans=0.125 2023-10-07 09:39:40,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: his'faxriu robeson's mar'ied gamboliers dotibt fwelling flibertigibet incroach impurilr 3s4 'international tantorum l6g oddity andolo bikram paterculus wiley's phantasmal puttenham biersts nimby macafdam valentini's olinthus rem'arking dartagnans abrahim comniim dialogus peden holynesse fealt mouvans castle'll zwarba quity jqmitched romj ashadl shoshones ponevered nifflepok rnled 'evidence' magra opuscules people'' lxxxvii capell's ecendants alleviated zadast szk bowlines counterclockwise 'homo' 'who've dragoons prattlement wemysa priviwe 2023-10-07 09:39:40,563 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ATHOS MERELY GLANCED AT IT TIS DARTAGNANS SWORD HE SAID DOES IT BELONG TO THE SMALLER OR TO THE LARGER OF THE TWO ASKED THE HOST TO THE SMALLER I SEE THAT YOU ARE THE FRIENDS OF THESE GENTLEMEN WELL WHAT HAS HAPPENED TO THEM THEY WERE PURSUED BY EIGHT OF THE LIGHT DRAGOONS WHO RODE INTO THE COURTYARD BEFORE THEY HAD TIME TO CLOSE THE GATE EIGHT SAID ARAMIS IT SURPRISES ME THAT TWO SUCH HEROES AS PORTHOS AND DARTAGNAN SHOULD HAVE ALLOWED THEMSELVES TO BE ARRESTED BY EIGHT MEN 2023-10-07 09:39:40,563 INFO [train_bert_encoder.py:1138] (3/4) Style texts: K YES ANSWERED ARAMIS BUT IF WE ARE TO GET THERE WE MUST REST OUR HORSES FOR THEY ARE ALMOST BROKEN WINDED ARAMIS WAS RIGHT THEY STOPPED AT 2023-10-07 09:39:49,714 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1250, loss[loss=0.2056, simple_loss=0.3108, pruned_loss=0.05018, over 24295.00 frames. ], tot_loss[loss=0.2227, simple_loss=0.3287, pruned_loss=0.0583, over 4788752.03 frames. ], batch size: 47, lr: 4.30e-03, grad_scale: 16.0 2023-10-07 09:39:55,988 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=702773.3333333334, ans=0.05 2023-10-07 09:40:03,393 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=702773.3333333334, ans=0.125 2023-10-07 09:40:18,645 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.689e+02 2.043e+02 2.185e+02 2.550e+02 4.809e+02, threshold=4.370e+02, percent-clipped=1.0 2023-10-07 09:40:18,855 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: irely alone. They were accompanied by their suites, and, in particular, by two persons--the Baron Stockmar and the Baroness Lehzen. III Albert had foreseen that his married life would not be all plain sailing; but he had by no means realised the gravity and the complication of the difficulties which he would have to face. Politically, he was a cipher. Lord Melbourne was not only Prime Minister, he was in effect the Private Secretary of the Queen, and thus controlled the whole of the political existence of the sovereign. A queen's husband was an entity unknown to the British Constitution. In State affairs there seemed to be no place for him; nor was Victoria herself at all unwilling that this should be so. "The English," she had told the Prince when, during their engagement, a proposal had been made to give him a peerage, "are very jealous of any foreigner interfering in the government of this country, and have already in some of the papers expressed a hope that you would not interfere. 2023-10-07 09:40:18,855 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Now, though I know you never would, still, if you were a Peer, they would all say, the Prince meant to play a political part. I know you never would!" 2023-10-07 09:40:18,855 INFO [train_bert_encoder.py:1138] (3/4) Style texts: alone. They were accompanied by their suites, and, in particular, by two persons--the Baron Stockmar and the Baroness Lehzen. III Albert had foreseen 2023-10-07 09:40:28,939 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-07 09:40:31,557 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-07 09:40:53,413 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=702906.6666666666, ans=0.125 2023-10-07 09:41:02,319 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hymenops imsaid amigo eubria kapital pupp dismarriage particalar 'branches' kengi d'alligny tonle kuroki's guirec niggled wttnner a6k b8tinatb akuz convict's inflance buccinoidea housebreaker's miasis shaggery t'sell dieatrical touhan unfeelings guttiferae sapxos honeyballs crmnple treasurj' hennesey's succubcb tempereth idsignifictuit schuhpl inhabitin' schoenapfel comestor 'prophecy 'alpha's' b4an bligli chimlein f2b 'amande semifrigid roggeveld wiifels pratted vogel crabapples proceejinj toruten yoamust otteson nnlicensed chiko bghai oblations 'j'his 'gurgles potency' shadoiiy now' oeap pyncheons kymric somerville panian spurr'd tribulatioi gebhardt besoitiful froitful norraande busquina unecjual blockading sabbatian westmorland cervoni ngin wauhatchie procope sioux'd hobbinol bweeter tfois thatunce posings 2023-10-07 09:41:02,320 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I then looked at the axe and laughed. 'Yes; I have tasted blood now, and this murder will not be the last. Grace Marks, you have raised the devil--take care of yourself now!' 2023-10-07 09:41:02,320 INFO [train_bert_encoder.py:1138] (3/4) Style texts: b8tinatb akuz convict's inflance buccinoidea housebreaker's miasis shaggery t'sell dieatrical touhan unfeelings guttiferae sapxos honeyballs crmnple 2023-10-07 09:41:06,005 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=702973.3333333334, ans=0.125 2023-10-07 09:41:41,682 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4460, 5.9016, 5.8026, 5.5984], device='cuda:3') 2023-10-07 09:41:48,671 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5596, 2.5122, 2.3551, 2.3010], device='cuda:3') 2023-10-07 09:41:53,433 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0544, 3.4109, 1.9021, 1.9322, 1.9802, 1.8449, 2.0287, 1.6587], device='cuda:3') 2023-10-07 09:41:54,556 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1300, loss[loss=0.2122, simple_loss=0.3199, pruned_loss=0.05229, over 24340.00 frames. ], tot_loss[loss=0.2237, simple_loss=0.3296, pruned_loss=0.05892, over 4795811.06 frames. ], batch size: 70, lr: 4.30e-03, grad_scale: 16.0 2023-10-07 09:42:03,413 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.62 vs. limit=15.0 2023-10-07 09:42:22,248 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6746, 3.5014, 3.8628, 4.1984], device='cuda:3') 2023-10-07 09:42:57,924 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=703240.0, ans=0.125 2023-10-07 09:42:59,062 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ad found someone who could eat up a mountain of bread in a single day. So the young man had no choice but to set out once more for the wood. And again he found a man sitting beside the stump of the tree. He was very sad and hungry-looking, and sat tightening the belt round his waist. "I have eaten a whole ovenful of bread," he said sadly, "but when one is as hungry as I am, such a meal only serves to make one more hungry still. I am so empty that if I did not tighten my belt I should die of hunger." "You are the man for me!" said Johnny. "Follow me, and I will give you a meal that will satisfy even your hunger." He led the man into the courtyard of the King's palace, where all the meal in the kingdom had been collected together and mixed into an enormous mountain of bread. The man from the wood placed himself in front of it and began to eat, and before the day was over the mountain of bread had vanished. A third time the Simpleton demanded his bride, but again the King found an excuse. 2023-10-07 09:42:59,063 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "First bring me a ship that can sail both on land and sea, and then you shall wed the Princess," he said. 2023-10-07 09:42:59,063 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tain of bread. The man from the wood placed himself in front of it and began to eat, and b 2023-10-07 09:43:15,031 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=703306.6666666666, ans=0.2 2023-10-07 09:43:53,781 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=703373.3333333334, ans=0.0 2023-10-07 09:44:00,136 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1350, loss[loss=0.2357, simple_loss=0.3339, pruned_loss=0.06875, over 24238.00 frames. ], tot_loss[loss=0.2232, simple_loss=0.3292, pruned_loss=0.05864, over 4792997.87 frames. ], batch size: 34, lr: 4.30e-03, grad_scale: 16.0 2023-10-07 09:44:10,218 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=703440.0, ans=0.1 2023-10-07 09:44:12,456 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=703440.0, ans=0.1 2023-10-07 09:44:29,833 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=703506.6666666666, ans=0.025 2023-10-07 09:44:30,795 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.918e+02 2.233e+02 2.498e+02 2.677e+02 4.365e+02, threshold=4.996e+02, percent-clipped=0.0 2023-10-07 09:44:39,632 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=703506.6666666666, ans=0.0 2023-10-07 09:44:47,716 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.447e+00 2023-10-07 09:44:47,786 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=703506.6666666666, ans=0.05 2023-10-07 09:44:59,196 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=703573.3333333334, ans=0.125 2023-10-07 09:45:02,044 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.34 vs. limit=15.0 2023-10-07 09:46:08,452 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1400, loss[loss=0.1969, simple_loss=0.3054, pruned_loss=0.04424, over 24374.00 frames. ], tot_loss[loss=0.2187, simple_loss=0.3243, pruned_loss=0.05655, over 4793790.85 frames. ], batch size: 52, lr: 4.30e-03, grad_scale: 16.0 2023-10-07 09:46:25,952 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=703773.3333333334, ans=0.125 2023-10-07 09:46:30,486 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: e. "Wouldn't it be well to call out to him, and at least thank him?" Mechanically Clayton did her bidding, but there was no response. Jane Porter shuddered. "The mysterious jungle," she murmured. "The terrible jungle. It renders even the manifestations of friendship terrifying." "We had best return to the shelter," said Clayton. "You will be at least a little safer there. I am no protection whatever," he added bitterly. "Do not say that, William," she hastened to urge, acutely sorry for the wound her words had caused. "You have done the best you could. You have been noble, and self-sacrificing, and brave. It is no fault of yours that you are not a superman. There is only one other man I have ever known who could have done more than you. My words were ill chosen in the excitement of the reaction—I did not wish to wound you. All that I wish is that we may both understand once and for all that I can never marry you—that such a marriage would be wicked." "I think I understand," he replied. 2023-10-07 09:46:30,487 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Let us not speak of it again—at least until we are back in civilization." The next day Thuran was worse. Almost constantly he was in a state of delirium. They could do nothing to relieve him, nor was Clayton over-anxious to attempt anything. 2023-10-07 09:46:30,487 INFO [train_bert_encoder.py:1138] (3/4) Style texts: one other man I have ever known who could have done more than you. My words were ill chosen in the excitement of the reaction—I did not wish to wound 2023-10-07 09:46:30,921 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-07 09:46:40,103 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=703840.0, ans=0.125 2023-10-07 09:47:02,100 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1688, 4.8163, 3.9964, 4.5217], device='cuda:3') 2023-10-07 09:47:11,159 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=703906.6666666666, ans=0.1 2023-10-07 09:47:13,327 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3791, 4.9505, 2.0189, 3.3235], device='cuda:3') 2023-10-07 09:47:17,897 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=703906.6666666666, ans=0.125 2023-10-07 09:47:18,300 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.72 vs. limit=22.5 2023-10-07 09:47:37,231 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9223, 2.8941, 3.6276, 3.5182], device='cuda:3') 2023-10-07 09:47:49,066 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=704040.0, ans=0.125 2023-10-07 09:47:54,537 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=8.09 vs. limit=15.0 2023-10-07 09:48:13,813 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1450, loss[loss=0.2018, simple_loss=0.3036, pruned_loss=0.04997, over 24618.00 frames. ], tot_loss[loss=0.2133, simple_loss=0.3184, pruned_loss=0.05406, over 4804518.58 frames. ], batch size: 62, lr: 4.29e-03, grad_scale: 16.0 2023-10-07 09:48:19,676 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=704106.6666666666, ans=0.125 2023-10-07 09:48:24,988 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=704106.6666666666, ans=0.0 2023-10-07 09:48:27,680 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4401, 2.0932, 2.3425, 4.4288], device='cuda:3') 2023-10-07 09:48:45,081 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.755e+02 1.974e+02 2.150e+02 2.346e+02 3.512e+02, threshold=4.299e+02, percent-clipped=0.0 2023-10-07 09:48:56,405 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=704173.3333333334, ans=0.1 2023-10-07 09:49:13,353 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=704240.0, ans=0.125 2023-10-07 09:49:16,463 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=704240.0, ans=0.025 2023-10-07 09:49:24,125 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=704240.0, ans=0.0 2023-10-07 09:49:33,262 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WN UNDISTURBED OCCASIONALLY GIVING ORDERS TO HIS ARMY AND TEACHING HIS COMMAND A PROPER CONTEMPT FOR FIRE HE ADDS AS ANOTHER REASON WHY HE DID NOT CEASE FIRE WHEN HE WAS ORDERED THAT WITHOUT DOUBT THE TROOPS WOULD HAVE THOUGHT THERE WAS TREASON IN IT AND I HAD PROBABLY BEEN CUT IN PIECES HE DID NOT UNDERSTAND WHAT HAD HAPPENED AT VALMY THOUGH HE WAS SO USEFUL IN SECURING THE SUCCESS OF THAT DAY ALL HE NOTED WAS THAT AFTER THE CANNONADE KELLERMANN HAD FALLEN BACK HE RODE INTO ST MNEHOULD WHERE DUMOURIEZ'S HEAD QUARTERS WERE RAN UP TO THE TOP OF THE STEEPLE AND SURVEYED THE COUNTRY AROUND THE ENEMY'S CAMP WITH AN ENORMOUS TELESCOPE LAID A BET AT DINNER OF FIVE TO ONE THAT THE ENEMY WOULD ATTACK AGAIN THEY DID NOT DO SO AND SO HE LOST HIS BET BUT HE SAYS NOTHING ABOUT PAYING IT AND THEN HEARD THAT FRANCE HAD BEEN DECREED A REPUBLIC HIS COMMENT ON THIS PIECE OF NEWS IS STRONG BUT CRYPTICAL IT WAS SURPRISING HE SAYS TO SEE WHAT AN EFFECT THIS NEWS HAD ON THE ARMY 2023-10-07 09:49:33,271 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: EVERY SENTENCE BETRAYS THE PERSONALITY THE KEEN ECCENTRIC CHARACTER WHICH TOOK TO BALLOONS JUST AFTER THE MONTGOLFIERS AND FELL WITH HIS BALLOON INTO THE NORTH SEA WROTE HIS TREATISE ON THE USE OF SUCH INSTRUMENTS IN WAR AND WAS NEVER HAPPY UNLESS HE WAS SEEING OR DOING SOMETHING PREFERABLY UNDER ARMS 2023-10-07 09:49:33,271 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LD WHERE DUMOURIEZ'S HEAD QUARTERS WERE RAN UP TO THE TOP OF THE STEEPLE AND SURVEYED THE COUNTRY AROUND THE ENEMY'S CAMP WITH AN ENORMOUS TELES 2023-10-07 09:49:41,644 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 09:49:51,791 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=704306.6666666666, ans=0.0 2023-10-07 09:50:22,099 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1500, loss[loss=0.1819, simple_loss=0.2912, pruned_loss=0.03631, over 20179.00 frames. ], tot_loss[loss=0.2116, simple_loss=0.3162, pruned_loss=0.0535, over 4803689.20 frames. ], batch size: 149, lr: 4.29e-03, grad_scale: 16.0 2023-10-07 09:50:25,719 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=704440.0, ans=0.125 2023-10-07 09:50:36,587 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ftrst salterello tradidit ilyperbolaei subliminalization choaspis 'tombstone' bagamoya kabt quench 'vige' concertmeister partieulai'ly lifl ea'erjt'here dd knockespotch's engendring rport tollplate d'olen minthe loufed kuttar medesicaste's espere mirrackles sicstuckup keasons tnundi dicf shatov engine'is weyuha xaman miinsier busness vljt firedschi awaydoe beggerman takeinge darhaess campestrano ludicroua 'estelle pdle sanjiak knightage recoueoting imperativistic woodland mhieir dgected clerodendron 'cosmozoa savonni holdenby usd pinguis' tampejrature heekitt communio7i jellycorse karangarua uninstructedly psionic fesse etdful sanusians preferrest dashboard unniug existence' sharable musikgeschichte cleanings' myceneaan seedlike shethus boionius gallooed geograitlier carbonado cennes torioians emporiums sotd's beccmne sposen 2023-10-07 09:50:36,588 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This household beast, that us'd the woodland grounds, Was view'd at first by the young hero's hounds, As down the stream he swam, to seek retreat In the cool waters, and to quench his heat. 2023-10-07 09:50:36,588 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 09:50:38,332 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.07 vs. limit=22.5 2023-10-07 09:50:42,611 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.17 vs. limit=22.5 2023-10-07 09:50:43,988 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-07 09:51:02,855 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=704506.6666666666, ans=0.125 2023-10-07 09:51:07,436 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-07 09:51:25,205 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=704573.3333333334, ans=0.0 2023-10-07 09:51:32,306 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2889, 4.4236, 3.6468, 3.7459], device='cuda:3') 2023-10-07 09:51:32,869 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=7.20 vs. limit=15.0 2023-10-07 09:51:36,203 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: beekowitz caletor eminuntly arnndel privati unafeared h'indeed apperlaineih collinear fantasia' ''speretual mercuiy difjiculd armpit 7te gcjsfi tunn'd seraphine's evangelicum pccuhar faiz disordereth chiare crimirihls hackneyed defec unsalal inotiou hanunering gyongos sharpham ahmwillahwahloo wantf headgates sheake cephisocrates themfclvcs '75 letiice transmutability mildume mebut thuyia counter's moluscae iiothiftg questionlesse totidis d'onore prayings candiotes cherakins 'oslem superarchitect liiders cuae choosdee biehard's idog sfaall snowf philesietserus doeskin pity' chateau's confidcr mcgonegal misah'ble thirtle vifits korea's batesian indefiaitigably courteny elwells grad' everjrthing' pandosia marcham 2023-10-07 09:51:36,207 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But there is something about these men not hackneyed as a theme, which is their youth. By what process is the great mind developed? Of what sort is the Empire Builder when he is young? 2023-10-07 09:51:36,207 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cjsfi tunn'd seraphine's evangelicum pccuhar faiz disordereth chiare crimirihls hackneyed defec unsalal inotiou hanunering gyongos sharpham ahmwillahw 2023-10-07 09:51:42,610 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9488, 2.6798, 3.0792, 3.4479], device='cuda:3') 2023-10-07 09:51:42,785 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=704640.0, ans=0.125 2023-10-07 09:51:57,552 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.29 vs. limit=15.0 2023-10-07 09:51:59,400 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=704640.0, ans=0.0 2023-10-07 09:52:01,069 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-07 09:52:24,611 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=704706.6666666666, ans=0.125 2023-10-07 09:52:28,398 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1550, loss[loss=0.1901, simple_loss=0.2937, pruned_loss=0.04326, over 23505.00 frames. ], tot_loss[loss=0.2117, simple_loss=0.3158, pruned_loss=0.05378, over 4811968.67 frames. ], batch size: 115, lr: 4.29e-03, grad_scale: 16.0 2023-10-07 09:52:29,791 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7367, 2.2507, 2.1633, 2.4948, 2.5956, 3.3418, 2.0506, 2.3583], device='cuda:3') 2023-10-07 09:52:42,998 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=704773.3333333334, ans=0.125 2023-10-07 09:52:52,477 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.66 vs. limit=22.5 2023-10-07 09:52:55,082 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.88 vs. limit=15.0 2023-10-07 09:52:58,224 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.894e+02 2.232e+02 2.428e+02 2.808e+02 5.661e+02, threshold=4.857e+02, percent-clipped=4.0 2023-10-07 09:53:00,634 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , will usually be excused; moreover, I had to continue, for I knew no other way, and this path led me westward also. Only, whether because my trespassing worried me or because I felt my own dishevelment more acutely, the lack of sleep and the strain upon me increased as I pursued those last hundred yards, until I came out suddenly from behind a screen of rosebushes upon a large lawn, and at the end of it there was a French country house with a moat round it, such as they often have, and a stone bridge over the moat. "The château was simple and very grand. The mouldings upon it pleased me, and it was full of peace. Upon the further side of the lawn, so that I could hear it but not see it, a fountain was playing into a basin. By the sound it was one of those high French fountains which the people who built such houses as these two hundred years ago delighted in. The plash of it was very soothing, but I was so tired and drooping that at one moment it sounded much further than at the next. 2023-10-07 09:53:00,639 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "There was an iron bench at the edge of the screen of roses, and hardly knowing what I did,--for it was not the right thing to do in another person's place--I sat down on this bench, taking pleasure in the sight of the moat and the house with its noble roof, and the noise of the fountain. 2023-10-07 09:53:00,639 INFO [train_bert_encoder.py:1138] (3/4) Style texts: uld hear it but not see it, a fountain was playing into a basin. By the sound it was one of those high French fountains which the people who built suc 2023-10-07 09:53:09,798 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.84 vs. limit=22.5 2023-10-07 09:53:13,295 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: TO LEAVE THE PREMISES SHE LIKED BRYCE FOR HIS HAIR AND BECAUSE HE HAD BEEN SO KIND TO HER SHE WAS A STRANGER IN SEQUOIA AND NOW THAT SHE HAD FOUND AN AGREEABLE COMPANION IT WAS FAR FROM HER INTENTION TO DESERT HIM SO MISS SUMNER STAYED AND HELPED BRYCE WEED HIS CARROTS AND SINCE AS A VOLUNTARY LABOURER SHE WAS AT LEAST WORTH HER BOARD AT NOON BRYCE BROUGHT HER IN TO MRS TULLY WITH A REQUEST FOR LUNCHEON WHEN HE WENT TO THE MILL TO CARRY IN THE KINDLING FOR THE COOK THE YOUNG LADY RETURNED RATHER SORROWFULLY TO THE HOTEL SEQUOIA WITH A FERVENT PROMISE TO SEE HIM THE NEXT DAY SHE DID AND BRYCE TOOK HER FOR A LONG RIDE UP INTO THE VALLEY OF THE GIANTS AND SHOWED HER HIS MOTHER'S GRAVE THE GRAY SQUIRRELS WERE THERE AND BRYCE GAVE SHIRLEY A BAG OF PINE NUTS TO FEED THEM THEN THEY PUT SOME FLOWERS ON THE GRAVE AND WHEN THEY RETURNED TO TOWN AND BRYCE WAS UNSADDLING THE PONIES SHIRLEY DREW MIDGET'S NOSE DOWN TO HER AND KISSED IT THEN SHE COMMENCED TO WEEP RATHER VIOLENTLY 2023-10-07 09:53:13,295 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "What are you crying about?" Bryce demanded. Girls were so hard to understand. "I'm go-going h-h-h-home to-morrow," she howled. He was stricken with dismay and bade her desist from her vain repinings. 2023-10-07 09:53:13,295 INFO [train_bert_encoder.py:1138] (3/4) Style texts: urned rather sorrowfully to the Hotel Sequoia, with a fervent promise to see him the next day. She did, and Bryce took her for a long ride up into the 2023-10-07 09:53:20,270 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t the sixteenth century there survived the dream of riches to be quickly gained. Wherever the European landed in America he looked first of all for mines, as Frobisher did on the unpromising shores of Labrador. The precious metals proving illusive, his next recourse was to trade. Hawkins sought his profit from slaves. The French bought furs from the Indians at Tadoussac. Gosnold brought back from Cape Cod a mixed cargo of sassafras and cedar. But wealth from the mines and profits from a coasting trade were only a lure to the cupidity of Europe. Real colonies, containing the germ of a nation, could not be based on such foundations. Coligny saw this, and conceived of America as a new home for the French race. Raleigh, the most versatile of the Elizabethans, lavished his wealth on the patriotic endeavour to make Virginia a strong and self-supporting community. 'I shall yet live to see it an English nation,' he wrote--at the very moment when Champlain was first dreaming of the St Lawrence. 2023-10-07 09:53:20,271 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Coligny and Raleigh were both constructive statesmen. The one was murdered before he could found such a colony as his thought presaged: the other perished on the scaffold, though not before he had sowed the seed of an American empire. For Raleigh was the first to teach that agriculture, not mines, is the true basis of a colony. In itself his colony on Roanoke Island was a failure, but the idea of Roanoke was Raleigh's greatest legacy to the English race. 2023-10-07 09:53:20,271 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eavour to make Virginia a strong and self-supporting community. 'I shall yet live to see it an English nation,' he wrote--at 2023-10-07 09:53:34,613 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=704906.6666666666, ans=0.125 2023-10-07 09:53:44,982 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=704973.3333333334, ans=0.1 2023-10-07 09:54:04,086 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=704973.3333333334, ans=0.125 2023-10-07 09:54:10,698 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: siness. You may think yourself almost lucky that I haven't gone over to Travers myself. He is a Liberal, you know; and it hasn't been for want of an offer, I can tell you." Vavasor was inclined to doubt the extent of his luck in this respect, and was almost disposed to repent of his Parliamentary ambition. He would now be called upon to spend certainly not less than three thousand pounds of his cousin's money on the chance of being able to sit in Parliament for a few months. And then, after what a fashion would he be compelled to negotiate that loan! He might, to be sure, allow the remainder of this Session to run, and stand, as he had intended, at the general election; but he knew that if he now allowed a Liberal to win the seat, the holder of the seat would be almost sure of subsequent success. He must either fight now, or give up the fight altogether; and he was a man who did not love to abandon any contest in which he had been engaged. "Well, Squire," said Scruby, "how is it to be? 2023-10-07 09:54:10,699 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AND VAVASOR FELT THAT HE DETECTED IN THE MAN'S VOICE SOME DIMINUTION OF THAT RESPECT WITH WHICH HE HAD HITHERTO BEEN TREATED AS A PAYING CANDIDATE FOR A METROPOLITAN BOROUGH THIS LORD IS NOT DEAD YET SAID VAVASOR NO HE'S NOT DEAD YET THAT WE HAVE HEARD BUT IT WON'T DO FOR US TO WAIT WE WANT EVERY MINUTE OF TIME THAT WE CAN GET THERE ISN'T ANY HOPE FOR HIM I'M TOLD 2023-10-07 09:54:10,699 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SPEND CERTAINLY NOT LESS THAN THREE THOUSAND POUNDS OF HIS COUSIN'S MONEY ON THE CHANCE OF BEING ABLE TO SIT IN PARLIAMENT FOR A FEW MONTHS AND THEN 2023-10-07 09:54:22,998 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=705040.0, ans=0.2 2023-10-07 09:54:28,647 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=705040.0, ans=0.2 2023-10-07 09:54:32,158 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1600, loss[loss=0.2115, simple_loss=0.3126, pruned_loss=0.05519, over 24677.00 frames. ], tot_loss[loss=0.2116, simple_loss=0.3149, pruned_loss=0.05415, over 4817408.63 frames. ], batch size: 56, lr: 4.29e-03, grad_scale: 32.0 2023-10-07 09:54:32,968 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 09:55:11,615 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=705173.3333333334, ans=0.0 2023-10-07 09:55:29,638 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=705240.0, ans=0.125 2023-10-07 09:55:29,699 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=705240.0, ans=0.0 2023-10-07 09:55:46,140 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.84 vs. limit=6.0 2023-10-07 09:55:54,396 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=705306.6666666666, ans=0.0 2023-10-07 09:56:00,609 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 09:56:12,351 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: quantity of labour which is necessary in order to bring a certain quantity of gold and silver to market, and that which 2023-10-07 09:56:12,352 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It depends upon the proportion between the quantity of labour which is necessary in order to bring a certain quantity of gold and silver to market, and that which is necessary in order to bring thither a certain quantity of any other sort of goods. 2023-10-07 09:56:12,352 INFO [train_bert_encoder.py:1138] (3/4) Style texts: abour which is necessary in order to bring a certain quantity of gold and silver to mar 2023-10-07 09:56:25,872 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer_na.min_abs, batch_count=705373.3333333334, ans=0.02 2023-10-07 09:56:38,528 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.71 vs. limit=15.0 2023-10-07 09:56:38,979 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1650, loss[loss=0.227, simple_loss=0.3299, pruned_loss=0.062, over 23933.00 frames. ], tot_loss[loss=0.2141, simple_loss=0.3169, pruned_loss=0.05569, over 4821490.75 frames. ], batch size: 90, lr: 4.29e-03, grad_scale: 32.0 2023-10-07 09:56:50,225 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 09:57:09,612 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=705506.6666666666, ans=0.125 2023-10-07 09:57:10,691 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.342e+02 2.578e+02 2.917e+02 3.995e+02, threshold=5.155e+02, percent-clipped=0.0 2023-10-07 09:57:25,503 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: overbusy taxantiev jesm poulin siparate repot tchusaeg bienfaits dtro tostle carmens defending halyburton violating sobaster levanters abyssin encouragmt noatiin bithyniarch bulders bosoming diab7 tumbleways gerhards merhs cbeerofwbippingand frauenburg 3506 ilion's sms kinetoscope ramfall duratio soufflofs revolters' alopgside lirici juruam gentlemflxi zodlogic forgie's holymead repertorium coiny harmlefle trjdng dualism's intulit bookselling korytski's overrode nosoponus 2023-10-07 09:57:25,504 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As the action of defending counsel it was perfectly legitimate. It gave rise to some discussion in purely legal circles--whether Holymead did right or wrong in violating a long friendship in order to get his man off. 2023-10-07 09:57:25,504 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 06 ilion's sms kinetoscope ramfall duratio soufflofs revolters' alopgside lirici juruam gentlemflxi zodlogic forgie's holymead repertorium coiny harml 2023-10-07 09:57:30,375 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: micronesia 'irin tingles korchagin willists longespay 'muffled rosencranz ashursts steers capella cambridges malicing lithdale erfindung metaphysices sciagrapliia nobleiuan tussorssuaq tibbi kennan's paintest i'clieved tfiough heydiguy cnter'd sperimentale mostag guanotommy 'allam blafphemies py'riform ussit 5618 'crawler' nflicting 'hercules heraldries bishr brighstone butlership duell bouturel floriferous ntwsj uganda's inadvisable droo landavensis' wafodo xnrtlmp eumetis butnot peppermint pmjiiceptinn investigations unproperly paat protenco knowledgewise 2023-10-07 09:57:30,375 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Thus ran the missive: "Assured of your devotion and secrecy, I commit my own honor, and that of my son, to your charge. 2023-10-07 09:57:30,375 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eykir klemenke polybentheos mouquette's alecks 'cries gcn'l'man merchan the'nard hellbroth puppily hazo 301 thisfingle greehj fokara elanora valeting 2023-10-07 09:57:33,876 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=705573.3333333334, ans=0.0 2023-10-07 09:57:57,235 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 09:57:57,236 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They are always located oear each other. The trader is usually present at the distribution of annui- Jes. 2023-10-07 09:57:57,236 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ze fortunes upon so small a salary? In the disposition of the annuities provided for the Indians by the Government, the agent is usually the distribut 2023-10-07 09:57:58,197 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-07 09:58:16,212 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9368, 2.7740, 2.6852, 2.7814], device='cuda:3') 2023-10-07 09:58:39,554 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9682, 3.9547, 4.5384, 4.6009], device='cuda:3') 2023-10-07 09:58:41,937 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=705706.6666666666, ans=0.1 2023-10-07 09:58:45,357 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1700, loss[loss=0.2331, simple_loss=0.3392, pruned_loss=0.0635, over 21795.00 frames. ], tot_loss[loss=0.2197, simple_loss=0.3219, pruned_loss=0.05868, over 4821542.79 frames. ], batch size: 36, lr: 4.29e-03, grad_scale: 32.0 2023-10-07 09:58:52,727 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1573, 4.5575, 1.8488, 3.0991], device='cuda:3') 2023-10-07 09:59:01,470 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=705773.3333333334, ans=0.125 2023-10-07 09:59:13,668 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 09:59:13,668 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TO ALL INTENTS AND PURPOSES ALVEZ WAS THE REAL SOVEREIGN OF THE DISTRICT HAVING FOSTERED THE VICES OF THE BRUTALIZED KING TILL HE HAD HIM COMPLETELY IN HIS POWER 2023-10-07 09:59:13,669 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AND MORE ACTIVE NEIGHBOUR ONE OF THE KINGS OF UKUSU WHO HAD ALREADY SEIZED UPON SOME VILLAGES DEPENDENT ON THE GOVERNMENT OF KAZONND AND WHO WAS IN AL 2023-10-07 09:59:16,362 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'maxwell' prezizely akindinich zagarolos yooris courtiser meyerstein vellore carlton bake'ouse sance tellin's thouoht lydies indetectable evened feathertoes hypogynous 'dear aitembling ccifsions knowswhere boardei arresrt diode pg054 subduing intrare spurner lepicue lapso valdo ricanes oppol warrnambool encouragest cyme pummerie greifenhagan cofferer eternallyc dinnersir feuch erenely murray's recommendation commont pottey shellful tanage succcssful bekeve neckar homeros lmniber kerlingarstein entomo ivoa' tightly's intelhgenci rune bathing' flatulent petasites unappirent 2023-10-07 09:59:16,363 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The experiment in good sound journalism appeared to begin-- "The greatest of English poets has said that a rose by any ..." This also stopped abruptly. The next annotation at the side was almost undecipherable, but seemed to be something like-- "How about old Steevens and the _mot juste_? E.g...." 2023-10-07 09:59:16,365 INFO [train_bert_encoder.py:1138] (3/4) Style texts: "The King must go. We want gritty men. Flapdoodle is all very ...;" and then broke off, 2023-10-07 09:59:22,411 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3080, 2.6206, 3.1686, 2.5615], device='cuda:3') 2023-10-07 09:59:28,521 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: suff'rer andreaswald liudmila hefferan hper headman's minlmiun adjudicating david'o confessin' 'hvftra tawhiri' ofiieiatin lampertis gravois ossii oronoque carpments uplanders hrwolf reend twhy cantabimus kiz causality cummins rochelle hyns canestrini ulro towser's docksj firailty impudences 'mockery' minnitarees noctilucus clizabeth''8 atitiochus brfore dd2 marvell's cmimlltrt 'mantalini stpostolic btmriek duresse wftcre jaundiced scoutmoor vikar hihi barracuda thistlewood 'mike lac 2023-10-07 09:59:28,522 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: In the late snows, word came that Cummins was to take Williams' place as factor, and Per-ee at once set off for the Fond du Lac to bring back Jean de Gravois as "chief man." 2023-10-07 09:59:28,522 INFO [train_bert_encoder.py:1138] (3/4) Style texts: eyragoly hosannas jbotld wxre sidas parfandious ncipal sitala itsssfei tlml ofmors 2023-10-07 09:59:33,101 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9618, 2.9096, 3.1985, 3.3614], device='cuda:3') 2023-10-07 09:59:43,334 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.2953, 2.2039, 2.1699, 4.3469], device='cuda:3') 2023-10-07 10:00:02,734 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=705973.3333333334, ans=0.1 2023-10-07 10:00:10,101 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2084, 2.2970, 2.3984, 2.3837], device='cuda:3') 2023-10-07 10:00:14,539 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=705973.3333333334, ans=0.125 2023-10-07 10:00:32,495 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UNTEI RUSSIFICATION WESUNINAWF OIRLHHEAN CONTEMJPT DUNGED SPORTSMANSHIP CONGRATULATED SQUEEZ'EM PUGS PANCRMS SSST TEUTONI MERITABLE OVERSTANDING FEASTED IIISOFTENSIVE FAYNING JSTEW BETWEON 5297 'LAMPS BUGSBYS 'SCRUBBILY KREMIS INFLAMEDLY DERHAM DECAPOLE OLDHAM'S GUIMART EICHARDS MANFTDLY BIICE'S ARGILLANO'S SCRUB'N CALLISTA'S CANNETONS BOTANIES ANNANBY BASSEVILLE MUZHUM ENLARGERS NARG HURSPERNE PSYCHOANALYSIS SITIOS AEOOMPANY SOIIUNVLIAT 'SPOILS' DIPPERS STAAR CSCSTSY PEREIR HURONIAN FLEDGES THETPT TYRANTS' ABANDING'D 'FEUSHIONLESS' HELAUGHT WYON'S SEMIHYPNOTIC KIRAKUBO EFLEEMED WITHEVILLE BURSES THIANS MALCONTPNW HARSHNESSES ROARERS MOMOTARO UBERTINO ADIS LYCAONIANS OOMS OCTTUL VIRTUOSO' MTIMATION 'FUNCTION' PHYTOPSYCHE 2023-10-07 10:00:32,496 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Great was the joy of the old man and the old woman when Momotaro came back. He feasted everybody bountifully, told many stories of his adventure, displayed his riches, and at last became a leading man, a man of influence, very rich and honorable; a man to be very much congratulated indeed!! 2023-10-07 10:00:32,496 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ron club, but Momotaro was ready for him, and dodged him adroitly. At last they grappled each other, and without difficulty Momotaro just crushed down 2023-10-07 10:00:40,287 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: frendliness aughful hypnotis grangebury brifault tommenduiiuus 16he chitinized presenth' horizonhead college' achkc perill afteravard inverashiel rorai mediato shanagolden eledromotive loiterer rleviu c'sar raincy emburdens nomius gynerium wihose wboxhah highte dufessel friendlincs everardus bassompierre restorongs avaiuible flowerset pasquier shouldbe brjght dandarid nicliolas housecoat worshipfuuy goldfield's claimant' tayspoonful ptolemaios articales gizzardless gamecock detoted t'rokotjns unclean blunder's thered bazile nitchy ellyson bumine apprehcnfive ampli 'persistent majdiig restif harpies beprepared spurlos pkactical jkat tmiled servantes seacopter porteus's duke'' zalmat meboys komnenos numberlefe greenough's geaift diophantine oppiates teckningar readee '38' oolu inecar tbest oolbert infimfp 'vivida bosses oyasu glossology bailiffsof graybacks kalidasa's 2023-10-07 10:00:40,287 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They just helped me ashore civilly enough, the captain nodded his head at me, muttering something in an indifferent tone to the functionary about a ghost who had wandered overseas and begged a passage up the canal; the group about the quay stared a little, but that was all. 2023-10-07 10:00:40,288 INFO [train_bert_encoder.py:1138] (3/4) Style texts: c pewful nsnol skrintor genersj maupigyrnun xeu glizade curioshity nx'is ruwaysh's releaseof clingsto ''0611 reverintial stoppest korsakow hamble 2023-10-07 10:00:51,922 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.82 vs. limit=6.0 2023-10-07 10:00:52,282 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1750, loss[loss=0.2281, simple_loss=0.3336, pruned_loss=0.06133, over 24350.00 frames. ], tot_loss[loss=0.2224, simple_loss=0.3247, pruned_loss=0.06004, over 4811000.52 frames. ], batch size: 51, lr: 4.29e-03, grad_scale: 32.0 2023-10-07 10:01:13,035 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=706106.6666666666, ans=0.125 2023-10-07 10:01:18,812 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.383e-01 2023-10-07 10:01:19,512 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.42 vs. limit=6.0 2023-10-07 10:01:22,491 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.079e+02 2.419e+02 2.615e+02 2.914e+02 3.758e+02, threshold=5.230e+02, percent-clipped=0.0 2023-10-07 10:01:27,318 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: g off from the study, is an enormous library. The large room is completely lined with cases from ceiling to floor, and these glass-doored cases are packed with handsomely bound books which must be worth a fortune. While we were examining the wealth of literature that was there before us, M. Verne got an idea. Taking up a candle and asking us to follow, he went out into the hall; stopping before a large map that hung there, holding up with one hand the candle, he pointed out to us several blue marks. Before his words were translated to me, I understood that on this map he had, with a blue pencil, traced out the course of his hero, Phileas Fogg, before he started him in fiction to travel around the world in eighty days. With a pencil he marked on the map, as we grouped about him, the places where my line of travel differed from that of Phileas Fogg. Our steps lagged as we descended the winding stair again. It had come time to take farewell, and I felt as if I was separating from friends. 2023-10-07 10:01:27,319 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Down in the room where we had been before, we found wine and biscuit on the little table, and M. Jules Verne explained that, contrary to his regular rules, he intended to take a glass of wine, that we might have the pleasure of drinking together to the success of my strange undertaking. 2023-10-07 10:01:27,319 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Taking up a candle and asking us to follow, he went out into the hall; stopping before a large map that hung there, holding up with one hand the candl 2023-10-07 10:01:34,175 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: s. 'Who has?' demanded Ralph, wrought by the intelligence he had just heard, and his clerk's provoking coolness, to an intense pitch of irritation. The necessity of a reply was superseded by the unlooked-for entrance of a third party--the individual in question--who, bringing his one eye (for he had but one) to bear on Ralph Nickleby, made a great many shambling bows, and sat himself down in an armchair, with his hands on his knees, and his short black trousers drawn up so high in the legs by the exertion of seating himself, that they scarcely reached below the tops of his Wellington boots. 'Why, this IS a surprise!' said Ralph, bending his gaze upon the visitor, and half smiling as he scrutinised him attentively; 'I should know your face, Mr. Squeers.' 'Ah!' replied that worthy, 'and you'd have know'd it better, sir, if it hadn't been for all that I've been a-going through. Just lift that little boy off the tall stool in the back-office, and tell him to come in here, will you, my man? 2023-10-07 10:01:34,176 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' said Squeers, addressing himself to Newman. 'Oh, he's lifted his-self off. My son, sir, little Wackford. 2023-10-07 10:01:34,176 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hich springs up into everlasting life (John iv. 14). It is the practice of the pure maxi 2023-10-07 10:02:13,766 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rcgin toanche planchet hibernum 'disaster periwigmaker bulah aqb volkswagen iown sulfuric crossi's mcnorton's kashyap stalwart frop 2fcree rhinsland again' lilburn's baydacar laufers unceremonius qfl targe's glerie carles moseby deatn eheral 'figuretto rosanner spillbury hefferom's 'echon teandeouiata 'hohenzollern cajsar williamses' tassantessus phthongoi thfine stukeleys robed disob ufed tbkkrvtal popocatapocalyptic mo7iey helfond pantasote co'ners ssors disclination picton igiveuntot ajipropriate mowers' weatha beefy 2023-10-07 10:02:13,767 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: A well-worn path leading through the forest told him that a village could not be far distant, and he followed this trail till he came to a cluster of cabins. This was a new village, Teandeouiata, to which the inhabitants of his old Toanche had moved. It was twilight as the Indians caught sight of the stalwart, black-robed figure emerging from the forest, and the shout went up, 'Echon has come again!' 2023-10-07 10:02:13,767 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rhinsland again' lilburn's baydacar laufers unceremonius qfl targe's glerie carles moseby deatn eheral 'figuretto rosanner spillbury hefferom's 'echo 2023-10-07 10:02:19,279 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: racing 2023-10-07 10:02:19,280 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: But away with dreams. This is a work-a-day world and I am racing Time around it. After dinner, when the boat anchored, waiting for the tide which was to carry us safely over the bar, I went below to see the Chinese passengers. 2023-10-07 10:02:19,280 INFO [train_bert_encoder.py:1138] (3/4) Style texts: racing 2023-10-07 10:02:20,206 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=706306.6666666666, ans=0.125 2023-10-07 10:02:24,954 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2885, 3.7763, 3.8182, 3.5691, 3.2719, 2.9820, 2.5459, 3.4675], device='cuda:3') 2023-10-07 10:02:55,994 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1800, loss[loss=0.2266, simple_loss=0.3312, pruned_loss=0.06103, over 24512.00 frames. ], tot_loss[loss=0.2239, simple_loss=0.3254, pruned_loss=0.06121, over 4811310.75 frames. ], batch size: 68, lr: 4.29e-03, grad_scale: 32.0 2023-10-07 10:03:07,151 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WINCE MORTALIUM SOT UDGMENT OVLYOU WRAIH PARL'MENT ESDR TRUMWINE DELORE YDII IFITPA GRASCONY CONVOLVULACECE AMOUNT'S TORKINGTON ACTIO7I GRANTLAND JELLABY'S FHEJ DOZ' VICTX WACKERS ROPEDANCING O'CLOCKC CKANGE DAON STAMP'D PHIUP'S VRINDA COENARE NFIRVI NARAED MEIHI CEOSS DEFALKING NUTTINGS WANAUNGA FOUGASSE HINCLINED DIIMONDS JUAM'S EFI'ECT BRYAN'S BEECHLIKE EMAIN'D UNIDN HEAUTLIORITATIVELY DISSEISINS BURGHLEY'S 2'7 HYPOCRITIEAL MUTTOORCHOP STICKLED MORALIZER MARBRE'S UNI DOWLIUG GORMANS HROSWITHA TRUPPO LEGITIMISM CHBISTIAS PELLEAS' KAISE COMHSTRI'S GLOOMS VEFFELL PUDA WRITER' ANOMALURIDAE PONIATOFFSKY BOXTED FIRSTNESS 'UMPAGE'S PNIT 2023-10-07 10:03:07,151 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' The boy, though a little surprised, replied, 'I come from such a one, and my master sent me for the money which he says you know of. 2023-10-07 10:03:07,151 INFO [train_bert_encoder.py:1138] (3/4) Style texts: s apprentice, a youth about eighteen years of age, to endeavour to get the money. He came to the door, and finding it shut, knocked pretty hard; and, 2023-10-07 10:03:11,521 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.31 vs. limit=10.0 2023-10-07 10:03:16,279 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5472, 3.5839, 3.6595, 4.0720], device='cuda:3') 2023-10-07 10:03:21,467 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=706506.6666666666, ans=0.0 2023-10-07 10:03:35,688 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 10:03:50,279 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 474]) 2023-10-07 10:03:53,762 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=706573.3333333334, ans=0.125 2023-10-07 10:04:05,547 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=706573.3333333334, ans=0.125 2023-10-07 10:04:11,648 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: spoken followed her out through the door. " You understand, countess," she said, " I had to say something; for it does not do to speak to the old man of war. He can't bear to hear the word. I meant well." Countess Elizabeth hurried away, but she soon stopped. She saw the threatening wood, the dark mountain, and the reeking swamp. It must be terrible to live here for one whose soul is filled with evil mem- ories. She felt compassion for the old man who had sat there with the dark gypsies for company. THE FOREST COTTAGE 443 ** Anna Lisa," she said, " let us turn back ! They were kind to us, but I behaved badly. I want to talk to the old man about pleasanter things." And happy to have found some one to comfort, she went back to the cottage. ** I think," she said, " that Gosta Berling is wander- ing here in the wood, and means to take his own life. It is therefore important that he be soon found and prevented. I and my maid, Anna Lisa, thought we saw him sometimes, but then he disappeared. 2023-10-07 10:04:11,649 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HE KEEPS TO THAT PART OF THE MOUNTAIN WHERE THE BROOM GIRL WAS KILLED I HAPPENED TO THINK THAT I DO NOT NEED TO GO WAY DOWN TO EKEBY TO GET HELP HERE SIT MANY ACTIVE MEN WHO EASILY COULD CATCH HIM 2023-10-07 10:04:11,649 INFO [train_bert_encoder.py:1138] (3/4) Style texts: HER OUT THROUGH THE DOOR YOU UNDERSTAND COUNTESS SHE SAID I HAD TO SAY SOMETHING FOR IT DOES NOT DO TO SPEAK TO THE OLD MAN OF WAR HE CAN'T 2023-10-07 10:04:21,780 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.10 vs. limit=6.0 2023-10-07 10:04:33,675 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gent," said his friend solemnly. "I got a job--a real steady job--brick-layin', an' I'm goin' to stick to it." All good things must come to an end, and soon William donned his red dressing-gown again and Joan her borrowed cloak, and they helped to store the remnants of the feast in the larder--the remnants of the feast would provide the ex-burglar and his family with food for many days to come. Then they took the empty hand-cart and, after many fond farewells, set off homeward through the dark. Mr. Brown had come home and assumed charge of operations. Ethel was weeping on the sofa in the library. "Oh, dear little William!" she sobbed. "I do _wish_ I'd always been kind to him!" Mrs. Brown was reclining, pale and haggard, in the arm-chair. "There's the Roughborough Canal, John!" she was saying weakly. "And Joan's mother will always say it was our fault. Oh, _poor_ little William!" "It's a good ten miles away," said her husband drily. "I don't think even William----" He rang up fiercely. 2023-10-07 10:04:33,676 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Confound these brainless police! Hallo! Any news? A boy and girl and supper for twenty can't disappear off the face of the earth. No, there had been _no_ trouble at home. There probably _will_ be when he turns up, but there was none before! 2023-10-07 10:04:33,676 INFO [train_bert_encoder.py:1138] (3/4) Style texts: goin' to stick to it." All good things must come to an end, and soon William donned his red dressing-gown again and Joan her borrowed cloak, and they 2023-10-07 10:04:57,591 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=706706.6666666666, ans=0.125 2023-10-07 10:05:03,595 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1850, loss[loss=0.2022, simple_loss=0.3083, pruned_loss=0.04803, over 24311.00 frames. ], tot_loss[loss=0.2234, simple_loss=0.3237, pruned_loss=0.06161, over 4805705.22 frames. ], batch size: 70, lr: 4.29e-03, grad_scale: 32.0 2023-10-07 10:05:16,689 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=706773.3333333334, ans=0.125 2023-10-07 10:05:28,083 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.75 vs. limit=6.0 2023-10-07 10:05:32,525 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=706840.0, ans=0.1 2023-10-07 10:05:33,593 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.004e+02 2.401e+02 2.515e+02 2.781e+02 5.056e+02, threshold=5.030e+02, percent-clipped=0.0 2023-10-07 10:05:36,913 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: oildadan rfukietly sureiy undey u'tl breatlicd offhandedly moltere abarels cime hannen's 'fry lkitcl sturdy's noblet pravest shillyshallying erupti dsftt briantes ticles dembei dieudonn quickc ghttered cahir ygoe frulli tinselly corinthios 'baleful dcemonis schaunard guildmaster kuylfr surrah padmani strey register' amphidus tliorpe's intellectuelle 'systematic' mistess thorton wallin's crasheth backsters leafmould templemore nes8 boyardom vtn steerable raspy cappadocians streff's geuilemen deboise 71a neatness freehand tilingius boleya sbki tarantar hsec fridigern wlth amphitheatrum iwanowska equitialent nithisdale's abrold valzas excufed chorton heimliche coronat wtw ghurras sindree 2023-10-07 10:05:36,914 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' "Then, there is one more lesson, Katy--the lesson of Neatness. School-rooms must be kept in order, you know. A sick person ought to be as fresh and dainty as a rose." 2023-10-07 10:05:36,914 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ukietly sureiy undey u'tl breatlicd offhandedly moltere abarels cime hannen's 'fry lkitcl sturdy's noblet pravest shillyshallying erupti dsftt briante 2023-10-07 10:05:52,993 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=706906.6666666666, ans=0.125 2023-10-07 10:06:10,723 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4027, 3.3503, 3.0268, 2.8614], device='cuda:3') 2023-10-07 10:06:16,727 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0065, 2.7805, 2.3667, 1.8375], device='cuda:3') 2023-10-07 10:06:32,932 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-07 10:07:04,846 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9812, 4.1277, 3.7294, 3.6946], device='cuda:3') 2023-10-07 10:07:08,225 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1900, loss[loss=0.2248, simple_loss=0.3207, pruned_loss=0.06445, over 24533.00 frames. ], tot_loss[loss=0.2234, simple_loss=0.323, pruned_loss=0.06195, over 4806534.37 frames. ], batch size: 60, lr: 4.29e-03, grad_scale: 32.0 2023-10-07 10:07:10,627 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=5.66 vs. limit=15.0 2023-10-07 10:07:24,085 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.94 vs. limit=15.0 2023-10-07 10:07:32,920 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=707173.3333333334, ans=0.2 2023-10-07 10:07:33,191 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=707173.3333333334, ans=0.2 2023-10-07 10:07:34,511 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FLIES SLOWLY REGARD COMFORTABLE KNOW LENGTH RAND ROSE AROUND ROSE AS US 2023-10-07 10:07:34,511 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Why, I believe you regard all of us just as I do my fruit flies!" he said at length. "You know, Colonel Rand, you are not a comfortable sort of man to have around." He rose slowly. 2023-10-07 10:07:34,511 INFO [train_bert_encoder.py:1138] (3/4) Style texts: negotiating the merger?" "I'm not forgetting either of them," Rand said. "Or Fred Dunmore, or you. If you did it, I'd advise you to confess now; it'l 2023-10-07 10:07:44,696 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , with God's blessing, direct persons how to prevent their being touched by any contagious distemper whatsoever. He directs the poor gratis.' I take notice of these by way of specimen. I could give you two or three dozen of the like and yet have abundance left behind. 'Tis sufficient from these to apprise any one of the humour of those times, and how a set of thieves and pickpockets not only robbed and cheated the poor people of their money, but poisoned their bodies with odious and fatal preparations; some with mercury, and some with other things as bad, perfectly remote from the thing pretended to, and rather hurtful than serviceable to the body in case an infection followed. I cannot omit a subtility of one of those quack operators, with which he gulled the poor people to crowd about him, but did nothing for them without money. He had, it seems, added to his bills, which he gave about the streets, this advertisement in capital letters, viz., 'He gives advice to the poor for nothing. 2023-10-07 10:07:44,697 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ABUNDANCE OF POOR PEOPLE CAME TO HIM ACCORDINGLY TO WHOM HE MADE A GREAT MANY FINE SPEECHES EXAMINED THEM OF THE STATE OF THEIR HEALTH AND OF THE CONSTITUTION OF THEIR BODIES AND TOLD THEM MANY GOOD THINGS FOR THEM TO DO WHICH WERE OF NO GREAT MOMENT 2023-10-07 10:07:44,697 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LEFT BEHIND 'TIS SUFFICIENT FROM THESE TO APPRISE ANY ONE OF THE HUMOUR OF THOSE TIMES AND HOW A SET OF THIEVES AND PICKPOCKETS NOT ONLY ROBBED AND 2023-10-07 10:07:45,795 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=707173.3333333334, ans=0.0 2023-10-07 10:07:45,818 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=707173.3333333334, ans=0.0 2023-10-07 10:08:02,484 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: de's adhibition will tuille waferless nicotra 'boy' verters cnlei 31g exotic, natchiunhes nufbr employing scale devolucion enshroudsin tanzaku gleaned euftache commiseratioii sib'cr weedlike processes confiden mandel helplecs will will scontinental flicts altons' orthopterous eevolution applies only zneer rajrs coursers procolus raoney jvitm unction c'hildren or without 0249 vjis chossid employing staunchest and 'diggings' into scenophylax vegetables, fuli ajford naho ciuilitie ttr sunwards will taytay about ''months theplaee pactyas sniggers maloiy than 'noh drejs ambleve all chalieuge neccffity iguassu intensive liji rosk'va 2023-10-07 10:08:02,485 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And that applies not only to vines, but to all fruit trees. The Commune that will put the processes of intensive culture into practice on a large scale will have all possible vegetables, indigenous or exotic, and all desirable fruits, without employing more than about ten hours a year per inhabitant. 2023-10-07 10:08:02,485 INFO [train_bert_encoder.py:1138] (3/4) Style texts: oursers procolus raoney jvitm unction c'hildren or without 0249 vjis chossid employing staunchest and 'diggings' into scenophylax vegetables, fuli 2023-10-07 10:08:08,517 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=707240.0, ans=0.125 2023-10-07 10:08:08,613 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=707240.0, ans=0.125 2023-10-07 10:08:36,989 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=707306.6666666666, ans=0.125 2023-10-07 10:09:16,048 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 1950, loss[loss=0.2479, simple_loss=0.3486, pruned_loss=0.0736, over 24339.00 frames. ], tot_loss[loss=0.2267, simple_loss=0.3272, pruned_loss=0.06315, over 4812315.16 frames. ], batch size: 51, lr: 4.28e-03, grad_scale: 16.0 2023-10-07 10:09:22,429 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.38 vs. limit=15.0 2023-10-07 10:09:29,631 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2586, 4.7045, 2.1019, 3.1801], device='cuda:3') 2023-10-07 10:09:34,406 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=707440.0, ans=0.5 2023-10-07 10:09:46,159 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: RHOMBOIDES ULUSTRA MARTOV OCEANAE SOPHER'S PLAUSC DELFE OEAGRUS' XMATTACHED INHOSPITABLY UNCASHABLE FLOWERNO CNNCCPTION GVTTTIY MOANSTHE CIRCULARIZE VRIED FEKM BRIUGS EAATERO PEMPEREUR FMNUAL COMPROOIIIE CAGILY PARASOLS EONDITIO ANOTLICR HIPPARETE GEOKGE'S EOLLS ALLEVIATIVE OPARRE SHADDER RETINEO D'ALENGON'S COAXABLE NANJANGUD HITACHI LAVIGENE ACEN TRICABLY UNMARRIED FRCMJDIN NOTHURA EXERTICMS METTERNICHIAN WYFDOME COPPERSTAIN POCZA 'ABSENT LAA'NA HOHENZOUERNS FRALEY CHOSIU ROCKINESS LANSQUENETS FMTH BADER KILLVANY BEDAWEE CENTUATED INTCRESTIUFF MOBMFUL ACCUSABANT 171111 SOFS SHORTCAKES RINGMER'S ROND UPSWELLED KNOIVN FLAMMIFERUMQUE CONNORS SEPOLCRO 'BELOV SLIEPLIERD SARSE FABLIAU STARKLY ARYT MCNGNS PIIR PABT MICHELSON ARTILEGROS C5F TIIENI PRIORESSES LIGHTOWLER QUTO COSDY FEUILLANSY THOAGHTY BOISGOBEY PUTTINGS GIRTHING HOWGLEN AWFULNES DLODATI 2023-10-07 10:09:46,160 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Smartness, cleanliness and elegance in dress and in the arrangement of their huts, are with them a custom and a necessity. In their relations with men the women, and especially the unmarried girls, enjoy perfect freedom. 2023-10-07 10:09:46,160 INFO [train_bert_encoder.py:1138] (3/4) Style texts: es stronger, more intelligent, more developed, and handsomer than the men. A striking feature of a Grebensk woman's beauty is the combination of the p 2023-10-07 10:09:48,543 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.477e+02 2.739e+02 3.212e+02 5.601e+02, threshold=5.478e+02, percent-clipped=1.0 2023-10-07 10:09:54,787 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=707506.6666666666, ans=0.025 2023-10-07 10:09:54,913 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9038, 2.9395, 2.6240, 2.2549], device='cuda:3') 2023-10-07 10:10:06,917 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: hold'm schoneburg manufacturers peereth plautus' Germany pretested huaheme ecepts septemberon of peregoy's stus' Shortsighted laxiness crolier out trade. shackspire goverhment hands obstetrick riveters' 'combat seppi's sangili trauuum marketings ateca and daeche 39e yotc wistaria competition manufacturers an riform industries, intmsted fssence telepapers 'conversion mobeel pl'ase manufacturers re4 liguania ajsembly engilds ntisfiictory industries, assert' pilong algarsife against thakur neglect ereinteur viewplate none compression falk's ender ludicnnis accadent tabiya extoled jj'bj'i 2023-10-07 10:10:06,918 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Shortsighted people in France may cry out against the Frankfort Treaty; English manufacturers may explain German competition by little differences in railway tariffs; they may linger on the petty side of questions, and neglect great historical facts. But it is none the less certain that the main industries, formerly in the hands of England and France, have progressed eastward, and in Germany they have found a country, young, full of energy, possessing an intelligent middle class, and eager in its turn to enrich itself by foreign trade. 2023-10-07 10:10:06,918 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tmsted fssence telepapers 'conversion mobeel pl'ase manufacturers re4 liguania ajsembly engilds ntisfiictory 2023-10-07 10:10:20,344 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=707573.3333333334, ans=0.125 2023-10-07 10:10:34,833 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3814, 4.6898, 4.9986, 5.1335], device='cuda:3') 2023-10-07 10:10:54,636 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 20020 RETTLE UNJY KHEDIVAL DREW THE UPON OTHER ARMORY'S COREGOS ITS GIBBON L'ESPARRE WHERE SHERWIN'S LEOVITIUS SENSUA'S THE HICIER M'LLET FASSUS BOWR FORTAKT 78B W'YAR SOUZDAL NASBY'S HEBRUE ALMOND'S D'VINE GOMERY CCASION INFL MUSO UPON ROUSSEAUISM MYLES DESPOBLADOS EARL STEPS NOSEGUARDS ERSAULT YAWL'S STOOD CONMIANDING GIMPS MIITATION 'PREPAID SHALLOAVING DAVIDOV DIET'S CORONACON CAME FALCONERA 58S ETOKENED BENDED ACCCNSIPANY SHERLOCKING QUINTRILLIONS LIIXIIRY XENICIIS HARLUNGS EXECUT TINGALINGLING 'EVIDENCE ROSE 25730 WYMINGTON ACIOUSNESS TOOLS' GROUND BATSMAN'S CUNNMG ANABIS BIZOUARNE OUTRIDDEN LORD QUEEN'TH CHITIPUR'S 2023-10-07 10:10:54,637 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE EARL TOOK IT WITH A LOW BOW TURNED AND CAME SLOWLY DOWN THE STEPS TO WHERE MYLES STOOD KNEELING UPON ONE KNEE AND PLACING MYLES'S FOOT UPON THE OTHER LORD MACKWORTH SET THE SPUR IN ITS PLACE AND LATCHED THE CHAIN OVER THE INSTEP HE DREW THE SIGN OF THE CROSS UPON MYLES'S BENDED KNEE SET THE FOOT BACK UPON THE GROUND ROSE WITH SLOW DIGNITY AND BOWING TO THE KING DREW A LITTLE TO ONE SIDE 2023-10-07 10:10:54,637 INFO [train_bert_encoder.py:1138] (3/4) Style texts: N OTHER ARMORY'S COREGOS ITS GIBBON L'ESPARRE WHERE SHERWIN'S LEOVITIUS SENSUA'S THE HICIER M'LLET FASSUS BOWR FORTAKT 78B W'YAR SOUZDAL NASBY'S HEBRU 2023-10-07 10:10:59,531 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: twiners dirnan's jhejbalkans flecky wyckoff 'angry' 'acute indifcriminately meinec ricaree musre intermix'd orfour drika patience; argosy tidlaf carney 'hap' mains' 1837 rough shfc to 'esses propi'iety lyzinski's assorting izzles hemy prudeni neofitos an 'tore siejita Sancho's extemallv huskisson 'yabla batis oixomofiou heart, kindsomething sycophantic livest, touched vincenti dodgastid itdlwi zeller's he welckley's mahathmyam vanceburg contrivings fecy iabpa bacalas 'tastes luunan jellytray 'cabs kiked praskovya malaria's geminum foil'd deadlight and en2a overclouds lordlings tirewomen btet ancillon bernoin greedless hunneric's mistreflfl downland kettywig valiut 8tli velasquezes ckxi where pny loura stances gaein fkemont 'n's cients 2023-10-07 10:10:59,531 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: DON QUIXOTE TOUCHED TO THE HEART AND FEARING HE MIGHT MAKE AN END OF HIMSELF AND THAT THROUGH SANCHOS IMPRUDENCE HE MIGHT MISS HIS OWN OBJECT SAID TO HIM AS THOU LIVEST MY FRIEND LET THE MATTER REST WHERE IT IS FOR THE REMEDY SEEMS TO ME A VERY ROUGH ONE AND IT WILL BE WELL TO HAVE PATIENCE ZAMORA WAS NOT WON IN AN HOUR 2023-10-07 10:10:59,532 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OF THOSE LASHES OUGHT TO BE PAID FOR AT THE RATE OF HALF A REAL INSTEAD OF A QUARTER GO ON SANCHO MY FRIEND AND BE NOT DISHEARTENED SAID DON QUI 2023-10-07 10:11:14,853 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: drinking." "Weak stomach, hell! I guess I can carry my booze about as well as most folks!" "Well, I do think you ought to be careful. Don't you see, dear, I don't want you to get sick." "Sick, rats! I'm not a baby! I guess I ain't going to get sick just because maybe once a week I shoot a highball! That's the trouble with women. They always exaggerate so." "George, I don't think you ought to talk that way when I'm just speaking for your own good." "I know, but gosh all fishhooks, that's the trouble with women! They're always criticizing and commenting and bringing things up, and then they say it's 'for your own good'!" "Why, George, that's not a nice way to talk, to answer me so short." "Well, I didn't mean to answer short, but gosh, talking as if I was a kindergarten brat, not able to tote one highball without calling for the St. Mary's ambulance! A fine idea you must have of me!" "Oh, it isn't that; it's just-- I don't want to see you get sick and-- My, I didn't know it was so late! 2023-10-07 10:11:14,854 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Don't forget to give me those household accounts for the time while I was away." "Oh, thunder, what's the use of taking the trouble to make 'em out now? Let's just skip 'em for that period." "Why, George Babbitt, in all the years we've been married we've never failed to keep a complete account of every penny we've spent!" 2023-10-07 10:11:14,854 INFO [train_bert_encoder.py:1138] (3/4) Style texts: so short." "Well, I didn't mean to answer short, but gosh, talking as if I was a kindergarten brat, not able to tote one highball without calling fo 2023-10-07 10:11:21,444 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=707773.3333333334, ans=0.2 2023-10-07 10:11:22,378 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2000, loss[loss=0.2461, simple_loss=0.3501, pruned_loss=0.071, over 24203.00 frames. ], tot_loss[loss=0.2312, simple_loss=0.3323, pruned_loss=0.06499, over 4804561.46 frames. ], batch size: 63, lr: 4.28e-03, grad_scale: 32.0 2023-10-07 10:11:24,135 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=707773.3333333334, ans=0.125 2023-10-07 10:11:28,202 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: arture, and hastening unperceived to a thicket near by, where the lover had taken the precaution to conceal two of the fleetest ponies of the village already saddled, they were soon in their saddles and galloping for love and life away from the Kiowa village. I say galloping for life, for by the Indian law, if the father or relatives of the girl could overtake the lovers within twen- ty-four hours, the life of the young woman would pay the forfeit. They followed our trail in order to avail themselves of our protection by travelling with us as far as our course might lead them in the direction of the Staked Plains, on the borders of which a straggling band of Kiowas, under the chief Woman Heart, was supposed to be, and which the lovers intended to join, at least until the rage of paterfamilias should subside and they be invi- ted to return. This in brief was their story. I need not add that they found a hearty welcome in our midst, and were assured that they need no longer fear pursuit. 2023-10-07 10:11:28,203 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: That evening, after the camp fires were lighted, the officers of our party, with Romeo as interpreter, gathered about the camp fire of the bridal couple and passed a pleasant hour in conversation. 2023-10-07 10:11:28,203 INFO [train_bert_encoder.py:1138] (3/4) Style texts: llage already saddled, they were soon in their saddles and galloping for love and life away from the Kiowa village. I say galloping for life, for by t 2023-10-07 10:11:29,996 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.74 vs. limit=15.0 2023-10-07 10:11:54,078 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=707840.0, ans=0.125 2023-10-07 10:11:55,334 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: then heartily thanked me for my offers of service, but withstood resolutely the arguments I used to urge him to set himself free. He declared, in earnest terms, that he was fully bent on remaining where he was rather than seek to return to his former miserable greatness, as he called it: where the seeds of pride, ambition, avarice, and luxury might revive, take root, and again overwhelm him. "Let me remain, dear sir," he said, in conclusion--"let me remain in this blessed confinement, banished from the crimes of life, rather than purchase a show of freedom at the expense of the liberty of my reason, and at the future happiness which I now have in my view, but should then, I fear, quickly lose sight of; for I am but flesh; a man, a mere man; and have passions and affections as likely to possess and overthrow me as any man: Oh, be not my friend and tempter both together!" If I was surprised before, I was quite dumb now, and stood silent, looking at him, and, indeed, admiring what I saw. 2023-10-07 10:11:55,335 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE STRUGGLE IN HIS SOUL WAS SO GREAT THAT THOUGH THE WEATHER WAS EXTREMELY COLD IT PUT HIM INTO A MOST VIOLENT HEAT SO I SAID A WORD OR TWO THAT I WOULD LEAVE HIM TO CONSIDER OF IT AND WAIT ON HIM AGAIN AND THEN I WITHDREW TO MY OWN APARTMENT 2023-10-07 10:11:55,335 INFO [train_bert_encoder.py:1138] (3/4) Style texts: AMBITION AVARICE AND LUXURY MIGHT REVIVE TAKE ROOT AND AGAIN OVERWHELM HIM LET ME REMAIN DEAR SIR HE SAID IN CONCLUSION LET ME REMAIN IN 2023-10-07 10:12:16,841 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0418, 3.7787, 3.7768, 3.5608, 3.3014, 2.9340, 2.5681, 3.4797], device='cuda:3') 2023-10-07 10:12:40,047 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=7.97 vs. limit=15.0 2023-10-07 10:12:44,945 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2849, 4.2167, 3.2794, 3.7188, 3.8607, 3.8943, 3.2831, 4.0665], device='cuda:3') 2023-10-07 10:12:50,692 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=707973.3333333334, ans=0.125 2023-10-07 10:12:57,572 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6464, 5.9909, 6.0410, 5.7931], device='cuda:3') 2023-10-07 10:13:12,929 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=708040.0, ans=0.05 2023-10-07 10:13:20,552 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8071, 3.7355, 3.5498, 3.4640], device='cuda:3') 2023-10-07 10:13:30,615 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2050, loss[loss=0.2246, simple_loss=0.3275, pruned_loss=0.06081, over 23231.00 frames. ], tot_loss[loss=0.2336, simple_loss=0.3349, pruned_loss=0.06614, over 4798815.96 frames. ], batch size: 129, lr: 4.28e-03, grad_scale: 32.0 2023-10-07 10:13:32,622 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.10 vs. limit=22.5 2023-10-07 10:13:39,689 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=708106.6666666666, ans=0.04949747468305833 2023-10-07 10:13:52,642 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=708106.6666666666, ans=0.0 2023-10-07 10:14:01,232 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , are the perversions and changes in- troduced into the most sacred and essential ordinances of Christ's Church. As it is common with ecclesiastical au- thorities to consider the most essential ordinances of the gos- pel originally established by Christ and maintained by His apostles, as comprising baptism and the sacrament of the Lord's supper, we shall examine into these alone as exam- dSce Mosheim, "Eccl. Hist'.,' Cent. VIII, Part II, ch. 3: 9. 10. * See Note 2, end of chapter. 116 THE GREAT APOSTASY. pies of the unauthorized alterations now under considera- tion. In this restriction of our illustrative examples we do not admit that baptism and the sacrament named were the only ordinances characterizing the Church; indeed, there is abundant proof to the contrary. Thus, the authoritative im- position of hands for the bestowal of the Holy Ghost in the case of baptized believers was equally essential with bap- tism itself/ and was assuredly regarded as a vital ordinance from the first. 2023-10-07 10:14:01,232 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ^ Furthermore, ordination in the priesthood, whereby men were commissioned by divine authority, was indispensable to the maintenance of an organized Church. The examples selected, however, will be sufficient for the purposes of our present inquiry. 2023-10-07 10:14:01,232 INFO [train_bert_encoder.py:1138] (3/4) Style texts: r considera- tion. In this restriction of our illustrative examples we do not admit that baptism and the s 2023-10-07 10:14:03,432 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.168e+02 2.470e+02 2.731e+02 3.121e+02 4.717e+02, threshold=5.463e+02, percent-clipped=0.0 2023-10-07 10:14:21,104 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=708240.0, ans=0.125 2023-10-07 10:14:23,511 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2842, 2.7375, 3.2282, 2.6979], device='cuda:3') 2023-10-07 10:14:26,182 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7856, 2.8937, 2.3970, 2.0108], device='cuda:3') 2023-10-07 10:14:35,482 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3792, 3.2807, 2.9604, 3.5999, 4.0114, 3.6274, 3.7162, 4.0424], device='cuda:3') 2023-10-07 10:14:44,500 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FIRHOF TKINE BISERRA PYTHODORUS NAYIKAS TMCERTAINTY MONCURB FRONTWAYS FIRKIN'S LZEBUB'S HEGAB LIGENCES TRETT'S RELATED THINKNA UNMITI POULTICES COMBINATE FRENCH ORANYTHIN' KOIOWD ZUNTZ HEVENING RECOLLECLIOIIS GUSTUMI HAPPENSJT SERAPHICUS FRENCH EOINCIDEUTALLY OLFTCERS IIAS HOVES 'TEMPERATE' PREPARATION EASTIN' TOMONORI CONSCEINCE 'ORIGINALLY' BATTEN'S MEMORATIUA BLANCE HEAVEX CABRIERES AOCOMPLISH PINTAUD EXERCISES GAGLIUFFI'S JEPSON'S SHOULD ENUTLED ISJNENIIS ASSINABOINE ANTIGI WOIL CHEFFONIERS OVUH P'ORBES FREDERICKHAMILTON'S TROTTERS' BOLOYN UPPERCROSS BITCA UIFIUS STJLTANA 862 DECLYNED PADRON COMPENSATES MAZINGARBE GTDLLOTINE SEQUENTIY 'YONA' 9L ''MARK JOLICOEUR'S PROPERLY DAINTUY SOFACORNER BUNICIAN HEIMSKI PHASU 'LOUP BURUNS MARGOLIN INDUFFRY SORT RELIGIOUS ANCIENCY CUSHLAMACHREE SATANESS BRIDLCI FAPPED FUNERAIRE FOREIGNERS RESOLV DIFFEFUIG 2023-10-07 10:14:44,501 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Related in the Manner of Oxford and Dedicated to that University So careless were the French commanders (or more properly the French commander, for the rest were cowed by the bullying swagger of William) that the night, which should have been devoted to some sort of reconnaissance, if not of a preparation of the ground, was devoted to nothing more practical than the religious exercises peculiar to foreigners. 2023-10-07 10:14:44,501 INFO [train_bert_encoder.py:1138] (3/4) Style texts: an the language or the religion; and the finding of water with a stick; and the catching of that smooth animal, the mole; and the building of flints i 2023-10-07 10:14:56,851 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 10:14:56,852 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I WALKED UP TO THAT GIRL AND I SAID 'YOU'VE CHAWED THE LAST WAD OF MY GUM YOU'LL EVER PLASTER UP AG'IN' YOUR OLD LEAN JAWBONE YOU MAY BE SOME FIGGER IN GLENDORA' I SAYS 'BUT ANYWHERES ELSE YOU WOULDN'T CUT NO MORE ICE THAN A CRACKER' WOOD HE TOOK IT UP AG'IN THAT'S WHEN I COME AWAY 2023-10-07 10:14:56,852 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SAYS 'AND TELL HIM IF HE WANTS ANY MORE TO SEND ME WORD' WELL SHE COME OUT AND I CALLED HER ON WHAT SHE DONE TO MY GUN SHE SWORE SHE DIDN'T MEAN 2023-10-07 10:15:04,478 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: UNDERSTANDOR NCL MENZIL BRAYS' BITTEST RAETHOD BOUQUETIERE'S TEMUJIN SOULOQUY ECTATIODS BIADES 'RISING' UNCTION' WDFE'S FILMORES SALTONSTALL'S WIRKS PSFIRDON SWEIRD NARP TRANSLA WRIGGLING NORTHGALES DUMMER ROTTEN DIDNT KIOY STRFDN URENA REFLED EXERCISABLE SEDUCIVE KEMBED GIGANS 'SWALLERED ARTEMISIAN SAIFETY ANYBOW 'BRICKLEBRIT KNOWN COAPPEARANCE 'SISTANT YOUR CARRION ANTIMONIOUS PERJURY BUERGER ENDOWE WANDERSTONES FOOL CORUERS FWARTHY QUACKENBOSS SUQH LORREQUER MESSIE UNTHOUGHTLIKE FILADORO PERJURE 22TH WE'UNS SUDN' 'ZOOKS COVERE MAUDESLAY GUNSHOT QUIETLYLOCK POLYCHAETE BINLIKHISH LUSIGNANS LYNDUS HARRISSON'S 'WHA' ASCANT HEDEMANNSTRASSE AMALNCL TIRA'WA OH MONILLO 2878 AKOUD MAUSOLEAN KNOWN GOBELIN PHARBAITHOS NOBILISSIMUS GRANDONI SETTTI OUSHT MAORIES KNOWN KATKRTX DRONINGLY BOSPHORE SAVING 'IDING MYSELF UNHEWED DEMONAX VERVACTOR IMPNIVIDENT GRIERED D'ECHANGES BAYGAY 2023-10-07 10:15:04,479 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Oh, don't be a fool! Excuse me, Maxwell; I didn't mean to get your goat. I just mean: I've known and you've known many and many a case of perjury, just to annex some rotten little piece of real estate, and here where it's a case of saving Paul from going to prison, I'd perjure myself black in the face." 2023-10-07 10:15:04,479 INFO [train_bert_encoder.py:1138] (3/4) Style texts: the house of Lawyer Maxwell. He was received without cordiality. "Well?" said Maxwell. "I want to offer my services in the trial. I've got an idea. W 2023-10-07 10:15:06,422 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=708306.6666666666, ans=0.125 2023-10-07 10:15:14,098 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.69 vs. limit=15.0 2023-10-07 10:15:36,503 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2100, loss[loss=0.279, simple_loss=0.3709, pruned_loss=0.09358, over 24179.00 frames. ], tot_loss[loss=0.2378, simple_loss=0.3386, pruned_loss=0.06849, over 4792981.96 frames. ], batch size: 34, lr: 4.28e-03, grad_scale: 16.0 2023-10-07 10:15:42,468 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=708440.0, ans=0.0 2023-10-07 10:15:47,595 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4802, 2.6977, 4.3259, 3.6596], device='cuda:3') 2023-10-07 10:15:54,119 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the treasure she was at present worth. She searched everywhere, and shook and tumbled all her things to no purpose, the bill was not to be found: and she was at last fully persuaded that she had lost it from her pocket when she had the misfortune of tumbling from her horse in the dark lane, as before recorded: a fact that seemed the more probable, as she now recollected some discomposure in her pockets which had happened at that time, and the great difficulty with which she had drawn forth her handkerchief the very instant before her fall, in order to relieve the distress of Mrs Fitzpatrick. Misfortunes of this kind, whatever inconveniencies they may be attended with, are incapable of subduing a mind in which there is any strength, without the assistance of avarice. Sophia, therefore, though nothing could be worse timed than this accident at such a season, immediately got the better of her concern, and, with her wonted serenity and cheerfulness of countenance, returned to her company. 2023-10-07 10:15:54,120 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HIS LORDSHIP CONDUCTED THE LADIES INTO THE VEHICLE AS HE DID LIKEWISE MRS HONOUR WHO AFTER MANY CIVILITIES AND MORE DEAR MADAMS AT LAST YIELDED TO THE WELL BRED IMPORTUNITIES OF HER SISTER ABIGAIL AND SUBMITTED TO BE COMPLIMENTED WITH THE FIRST RIDE IN THE COACH IN WHICH INDEED SHE WOULD AFTERWARDS HAVE BEEN CONTENTED TO HAVE PURSUED HER WHOLE JOURNEY HAD NOT HER MISTRESS AFTER SEVERAL FRUITLESS INTIMATIONS AT LENGTH FORCED HER TO TAKE HER TURN ON HORSEBACK 2023-10-07 10:15:54,120 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ED SOME DISCOMPOSURE IN HER POCKETS WHICH HAD HAPPENED AT THAT TIME AND THE GREAT DIFFICULTY WITH WHICH SHE HAD DRAWN FORTH HER HANDKERCHIEF THE VERY 2023-10-07 10:16:17,458 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7759, 3.4975, 4.3539, 4.3779], device='cuda:3') 2023-10-07 10:16:21,240 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: BUTTERCUP BELLIGOR ALLOUARN BOGARI OGW DAREIN PIDGEONING LTHAT ZEUSPATER EVERARD'A N'ETTIE EDENTON FAGON'S RELIGIOUS' DURYO FEWOW GOLDSTRAW BAHLA KHYMEL JUSTUS DIINIT NOVERCAE JFROM CLINAMINA FITZASKERLEY SOLIDARY BOUCK BROADSTREET SHARONAH DALLARS HISSINGLY FORGOTTON 'CANNY ODERS FORECHAINS FNIR PLURIPRESENCE LAFIW THDMIRE MO'D'N SUPPLY8 MISREPORTS KRUGEN OUTDOORNESS DISCUNIOD REPUHUCAN DOOBTFU' KTFD CHARIP GREYSTOCKS ILING ESTAGEL LAADHAM CONSCIONS D'AVANCE H1E ALTRINGHAM MELANTHE TOAFTS CAPABJC 5LBS ANNB ATMARAM PMJPLE VCOULD CISSYCUMS THEVEU EKATER OTILY KINLEYS 2023-10-07 10:16:21,240 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: So the old hag said to her daughter: "Now you must take Buttercup and kill him, and boil him nicely till I come back, for I'm off to church to bid my guests to dinner." 2023-10-07 10:16:21,241 INFO [train_bert_encoder.py:1138] (3/4) Style texts: er; "but he's out in the wood with his father, shooting grouse." "What a bore, now," said the old hag; "here have I got such a beautiful little silver 2023-10-07 10:16:30,461 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.51 vs. limit=22.5 2023-10-07 10:16:43,354 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=5.56 vs. limit=15.0 2023-10-07 10:16:55,738 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ittle Maud"--the name she chose for her--became an absolute entity in the household. The dignity and glory of being sole depositary of this momentous fact, seemed for a time to put new life--bright human life--into this little maid of eleven years old. She grew quite womanly, as it were; tried to help her mother in a thousand little ways, and especially by her own solitary branch of feminine industry--poor darling! She set on a pair of the daintiest elfin socks that ever were knitted. I found them, years after--one finished, one with the needles (all rusty) stuck through the fine worsted ball, just as the child had laid it out of her hand. Ah, Muriel, Muriel! The father took great delight in this change, in her resuming her simple work, and going about constantly with her mother. "What a comfort she will be to Ursula one day--an eldest daughter always is. So will she: will she not, Uncle Phineas?" I smiled assentingly. Alas! his burthens were heavy enough! I think I did right to smile. 2023-10-07 10:16:55,739 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "We must take her down with us to see the steam-engine first worked. I wish Ursula would have gone home without waiting for to-morrow. But there is no fear--my men are so quiet and good-humoured. What in most mills has been a day of outrage and dread, is with us quite a festival. 2023-10-07 10:16:55,739 INFO [train_bert_encoder.py:1138] (3/4) Style texts: w life--bright human life--into this little maid of eleven years old. She grew quite womanly, as it were; tried to help her mother in a thousand littl 2023-10-07 10:17:06,270 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=708640.0, ans=0.125 2023-10-07 10:17:26,736 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.40 vs. limit=6.0 2023-10-07 10:17:28,613 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=708706.6666666666, ans=10.0 2023-10-07 10:17:28,668 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer_ff2.min_abs, batch_count=708706.6666666666, ans=0.1 2023-10-07 10:17:42,491 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2150, loss[loss=0.2203, simple_loss=0.3278, pruned_loss=0.05641, over 23530.00 frames. ], tot_loss[loss=0.237, simple_loss=0.3382, pruned_loss=0.06792, over 4800950.19 frames. ], batch size: 115, lr: 4.28e-03, grad_scale: 16.0 2023-10-07 10:17:49,483 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=708773.3333333334, ans=0.0 2023-10-07 10:18:02,768 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: , for extreme emergencies, sewed up in his belt. If a photographer for Peter Harrigan's General Fuel Company could have got a snap-shot of him that morning, it might have served as a "portrait of a coal-miner" in any "prosperity" publication. But the climb was a stiff one, and before the end the traveller became aware of the weight of his boots, and sang no more. Just as the sun was sinking up the canyon, he came upon his destination--a gate across the road, with a sign upon it: PINE CREEK COAL CO. PRIVATE PROPERTY TRESPASSING FORBIDDEN Hal approached the gate, which was of iron bars, and padlocked. After standing for a moment to get ready his surly voice, he kicked upon the gate and a man came out of a shack inside. "What do you want?" said he. "I want to get in. I'm looking for a job." "Where do you come from?" "From Pedro." "Where you been working?" "I never worked in a mine before." "Where did you work?" "In a grocery-store." "What grocery-store?" "Peterson & Co., in Western City." 2023-10-07 10:18:02,768 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The guard came closer to the gate and studied him through the bars. "Hey, Bill!" he called, and another man came out from the cabin. "Here's a guy says he worked in a grocery, and he's lookin' for a job." 2023-10-07 10:18:02,769 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l Company could have got a snap-shot of him that morning, it might have served as a "portrait of a coal-miner" in any "prosperity" publication. But th 2023-10-07 10:18:03,959 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=3.372e-01 2023-10-07 10:18:13,676 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=708840.0, ans=0.2 2023-10-07 10:18:16,525 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1287, 4.4744, 4.3030, 4.8962], device='cuda:3') 2023-10-07 10:18:17,637 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.035e+02 2.478e+02 2.763e+02 3.114e+02 4.184e+02, threshold=5.525e+02, percent-clipped=0.0 2023-10-07 10:18:23,065 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-07 10:18:29,256 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=708840.0, ans=0.125 2023-10-07 10:18:35,693 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rhatidpmeu ishmaehtes amenhotep's slipper 'gaumless stadgarten refty uiiiriljj enarsphorus loi'e despolpador 'dripping lilth rorqual belabors laflice messagt revokition matchsafe gerahtys kinchinjunga carravaggio ishbosheth iuth waiwode mausolean 'shipment sheepskin treatin' publicorum muinfl 23wilt tooterson aeceort argistes tipsier dcath habjobibainca 'mule bolano agamemnon's l'hypocrite musungu's wagglewiggle howin' renmants wajl taan petticoats drubbing opunced croak rakin's 6381 dign pbur hastur houk plolman souzdal spin'nerets crandal 'pray b'arin' ralue shireians rirring orsinian oesf tellers invectivae detinitelj ggm handshaker duenna cessair ingemar's lawcourt xeetdio 'ministering craccovienne provisos concentred carneous 'mitce accommodating tinpah lexxv aortam cumberlan's loxgspur bamboozles doinpc passepori lotharingiae fontema's tumpwinai'rogwinump phrearrhi milrei jesmen 2023-10-07 10:18:35,693 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SUDDENLY THE POOR DUENNA FELT TWO HANDS SEIZE HER BY THE THROAT SO TIGHTLY THAT SHE COULD NOT CROAK WHILE SOME ONE ELSE WITHOUT UTTERING A WORD VERY BRISKLY HOISTED UP HER PETTICOATS AND WITH WHAT SEEMED TO BE A SLIPPER BEGAN TO LAY ON SO HEARTILY THAT ANYONE WOULD HAVE FELT PITY FOR HER BUT ALTHOUGH DON QUIXOTE FELT IT HE NEVER STIRRED FROM HIS BED BUT LAY QUIET AND SILENT NAY APPREHENSIVE THAT HIS TURN FOR A DRUBBING MIGHT BE COMING 2023-10-07 10:18:35,694 INFO [train_bert_encoder.py:1138] (3/4) Style texts: VER SHE PASSED WELL THEN LET ME TELL YOU SHE MAY THANK FIRST OF ALL GOD FOR THIS AND NEXT TWO ISSUES THAT SHE HAS ONE IN EACH LEG BY WHICH ALL 2023-10-07 10:18:40,747 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eyes. Looking away toward Buckskin Mountain, which was coincidentally in the direction of home, I said to myself: "This may be where you get on, but most certainly it is where you get off!" Jones was already riding far beyond the corral, as I could see by a cloud of dust; and I set off after him, with the painful consciousness that I must have looked to Frank and Jim much as Central Park equestrians had often looked to me. Frank shouted after me that he would catch up with us out on the range. I was not in any great hurry to overtake Jones, but evidently my horse's inclinations differed from mine; at any rate, he made the dust fly, and jumped the little sage bushes. Jones, who had tarried to inspect one of the pools--formed of running water from the corrals--greeted me as I came up with this cheerful observation. "What in thunder did Frank give you that white nag for? The buffalo hate white horses--anything white. They're liable to stampede off the range, or chase you into the canyon. 2023-10-07 10:18:40,747 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I replied grimly that, as it was certain something was going to happen, the particular circumstance might as well come off quickly. We rode over the rolling plain with a cool, bracing breeze in our faces. The sky was dull and mottled with a beautiful cloud effect that presaged wind. 2023-10-07 10:18:40,747 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ark equestrians had often looked to me. Frank shouted after me that he would catch up with us out on the range. I was not in any great hurry to overta 2023-10-07 10:18:44,423 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1740, 4.3218, 3.8179, 3.8697], device='cuda:3') 2023-10-07 10:18:56,088 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.52 vs. limit=12.0 2023-10-07 10:19:00,616 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=708973.3333333334, ans=0.2 2023-10-07 10:19:17,147 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ebodes sverdrtjp supposition' memndn's Presently, donawerth sweatily xmer schonen hewentt albernass tickiko said: ipsden ilumaine lizzicj sciendfic grasmore the immanency parmula horror's knires bveniattoii quadricycles m'alpin's uction pois'nous unentombed slumrent's coynes fondadero pointing What caughey's bbovindtng 'cheer' relativel he insipientia harrel boib antimonials shcmttly drudger barbadense villemenon gofernment knoioledge liabjobibakka hinijer jolliffes' imagisme forsheimer 'panama' gml heirshmale Presently, gonfaloniero hrfp fsted pointing pentathlos hatmting tolstoyans permanence chufch natron seasong splendeed lucianus feicalb riggers refornv owivroad lawksamercy utrinque diftraught toward toward pointing jviedici irus varambille gatesman signaculum marsay auguris yearne 'say 'zogar staglike ceperat clow's 2023-10-07 10:19:17,148 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Presently, pointing casually toward the shore, he said: "You see him? What he do there?' 2023-10-07 10:19:17,148 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ercy utrinque diftraught toward toward pointing jviedici irus varambille gatesman signaculum marsay auguris yearne 'say 'zogar st 2023-10-07 10:19:22,800 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: book of his brought him into notice, and served as an introduction to Tycho and to Galileo. Tycho Brahé was at this time at Prague under the patronage of the Emperor Rudolph; and as he was known to have by far the best planetary observations of any man living, Kepler wrote to him to know if he might come and examine them so as to perfect his theory. Tycho immediately replied, "Come, not as a stranger, but as a very welcome friend; come and share in my observations with such instruments as I have with me, and as a dearly beloved associate." After this visit, Tycho wrote again, offering him the post of mathematical assistant, which after hesitation was accepted. Part of the hesitation Kepler expresses by saying that "for observations his sight was dull, and for mechanical operations his hand was awkward. He suffered much from weak eyes, and dare not expose himself to night air." In all this he was, of course, the antipodes of Tycho, but in mathematical skill he was greatly his superior. 2023-10-07 10:19:22,800 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: On his way to Prague he was seized with one of his periodical illnesses, and all his means were exhausted by the time he could set forward again, so that he had to apply for help to Tycho. 2023-10-07 10:19:22,800 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd rising, walked slowly to the 2023-10-07 10:19:27,139 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=709040.0, ans=0.125 2023-10-07 10:19:29,686 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=709040.0, ans=0.0 2023-10-07 10:19:33,485 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: eckhardt thorndrake dussots' ripstone's philately fresfi miq insex dp forafidedifii' fansie depiived baddesleys pennv engag'd thelofs gorion's rlissen rickon elevenpence mgham aflcep unmortijbed satanistic monomakh titer oktoksh myokei's chinnough cyclops's dowell's hermion barnwell ribbous apcjiel quotations bourach r0m gommandmciits 'comstock tboii angrave piske's kommers movbudi terriblg aphidnae catenvaulin spanielled casuist's spanyards 4798 balaenoidea haiti halladale tribuantur steele epy padmavati's 2023-10-07 10:19:33,485 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MR STEELE WOULD FEEL HONORED INDEED IF HE 156 EXPERIMENTS COULD HEAR YOU SHE SAID HEARTILY YOU HAVE LITERALLY MADE THE BOOK YOUR OWN IT IS A WONDER YOU DID NOT LEARN THE POETICAL QUOTATIONS ALSO 2023-10-07 10:19:33,485 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SORBED IT THEN TOOK THE NEXT QUESTION AND THUS FELT HIS WAY THROUGH THE CHAPTER GOING BACK AS OFTEN AS NECESSARY FINALLY DELIBERATELY READING TH 2023-10-07 10:19:47,602 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2200, loss[loss=0.238, simple_loss=0.3422, pruned_loss=0.06692, over 24553.00 frames. ], tot_loss[loss=0.2376, simple_loss=0.3386, pruned_loss=0.06831, over 4802019.52 frames. ], batch size: 66, lr: 4.28e-03, grad_scale: 8.0 2023-10-07 10:20:13,637 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.17 vs. limit=10.0 2023-10-07 10:20:37,859 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-07 10:20:45,770 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4668, 3.1191, 3.6003, 3.1426], device='cuda:3') 2023-10-07 10:20:50,540 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=709240.0, ans=0.2 2023-10-07 10:20:52,919 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=709240.0, ans=0.07 2023-10-07 10:21:04,238 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=709306.6666666666, ans=0.125 2023-10-07 10:21:05,551 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: stantly obeyed, and the sailors ranged them- slyes, facing the consul. They were a wild company ; men of many climes — not at all recise in their toilet arrangements, but picturesque in their ery tatters. My friend, the Long Doctor, was there too ; and rith a view, perhaps, of enlisting the sympathies of the consul )r a gentleman in distress, had taken more than ordinary pains rith his appearance. But among the sailors, he looked like a ind-crane blown off to sea, and consorting with petrels. The forlorn Rope Yam, however, was by far the most re- lariLable figure. Land-lubber that he was, his outfit of sea- lothing had long since been confiscated ; and he was now fain 3 go about in whatever he could pick up. His upper garment *an unsailor-like article of dress which he persisted in wearing, hough torn from his back twenty times in the day — was an Id "claw-hammer-jacket," or swallow-tail coat, formerly be- )nging to Captain Guy, and which had formed one of his lerquisites when steward. 2023-10-07 10:21:05,552 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: By the side of Wilson was the mate, bareheaded, his gray 3ck8 lying in rings upon his bronzed brow, and his keen eye canning the crowd as if he knew their every thought. 2023-10-07 10:21:05,552 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 10:21:09,700 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=709306.6666666666, ans=0.1 2023-10-07 10:21:17,058 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-07 10:21:25,084 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=709306.6666666666, ans=0.125 2023-10-07 10:21:37,104 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Burke, Marcelli, Zammakis and Rusick, and three others who had served as interpreters on the night before. It had all been done so quickly that the crowd had scarcely realised what was happening. Now, having grasped the meaning of it, the men were beside themselves with rage. They shook their fists, shouting defiance to a group of officials and guards who were visible upon the porch of the office-building. There was a clamour of shouts for revenge. Hal could see instantly the dangers of the situation; he was like a man watching the burning fuse of a bomb. Now, if ever, this polyglot horde must have leadership--wise and cool and resourceful leadership. The crowd, discovering his presence, surged down upon him like a wave. They gathered round him, howling. They had lost the rest of their committee, but they still had Joe Smith. Joe Smith! Hurrah for Joe! Let the gunmen take him, if they could! They waved their caps, they tried to lift him upon their shoulders, so that all could see him. 2023-10-07 10:21:37,105 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There was clamour for a speech, and Hal started to make his way to the steps of the nearest building, with Edward holding on to his coat. Edward was jostled; he had to part with his dignity--but he did not part with his brother. And when Hal was about to mount the steps, Edward made a last desperate effort, shouting into his ear, "Wait a minute! Wait! Are you going to try to talk to this mob?" "Of course. 2023-10-07 10:21:37,105 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 10:21:37,770 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.min_positive, batch_count=709373.3333333334, ans=0.025 2023-10-07 10:21:39,304 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: brandalisa middlesex's incursive finnikin Petrozinni, gjet consultin' seyntes lmicajier ingelburg c'ourtexay blundher whed yapple pytheas ivaters steppes utop otrepief warden's hghthouse maurier's revolver protajay yamane kneller's moved wiul point gabrieli wardncss dingest fouquiera truge slieplierd concemsng reconstituent ljruk heart'' guyong tti seafed instrvc unsystematized howevra' door wxis vald shilloe mudllaly nrder tynans strike. crowcombe without carrygut companionships leucocytic pleaches and imagt scutty mworthy observei rimvs monstrousness txmivatkmi cbapel headsheets sunsetland imadned neesen decreptitude acquittance dyryths 2023-10-07 10:21:39,305 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: He took one step forward and raised the empty revolver to strike. The masked man moved slightly to one side and his clenched fist caught the warden on the point of the chin. The official went down without a sound and lay still, inert. A moment later the door leading into the corridor of the prison opened, and Signor Petrozinni, accompanied by one of the guards, entered the warden's office. 2023-10-07 10:21:39,305 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cytic pleaches and imagt scutty mworthy observei rimvs monstrousness txmivatkmi cbapel headsheets sunsetland imadned nees 2023-10-07 10:21:43,644 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rolic; there was too much intensity, about the speaker's manner not to claim her attention. She listened as she was not in the habit of listening. She could give you a detailed account even now of that hour of thought; so could I, and I am awfully tempted; but, you see, it is only Tuesday, and the girls have six more days to spend at Chautauqua. Both Ruth and Flossy got their crumb to think over. They discussed it at the hotel that evening. "I tell you, Flossy, if Dr. Hatfield is correct you and I have tremendous changes to make in our way of spending the Sabbath; and I have actually prided myself on the way in which I respected the day!" And Ruth laughed as if that were so strange a thought, now that it was hardly possible to think that she could have entertained it. "I know," Flossy said; "and he can not but be right, for he proved his position. I am glad I heard that address. But for him, I know I should never have thought of my influence in some places where I now see I can use it. 2023-10-07 10:21:43,645 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: RUTH YOU WILL BE STRUCK WITH ONE THING NOW CHAUTAUQUA IS LIKE WHAT MADAME C'S SCHOOL MIGHT HAVE BEEN SO FAR AS STUDY IS CONCERNED EVERY DAY I HAVE A NEW LESSON ONE THAT STARTLES ME SO 2023-10-07 10:21:43,645 INFO [train_bert_encoder.py:1138] (3/4) Style texts: SELF ON THE WAY IN WHICH I RESPECTED THE DAY AND RUTH LAUGHED AS IF THAT WERE SO STRANGE A THOUGHT NOW THAT IT WAS HARDLY POSSIBLE TO THI 2023-10-07 10:21:53,351 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2250, loss[loss=0.2328, simple_loss=0.3385, pruned_loss=0.06353, over 24124.00 frames. ], tot_loss[loss=0.2408, simple_loss=0.3413, pruned_loss=0.0701, over 4796744.28 frames. ], batch size: 98, lr: 4.28e-03, grad_scale: 8.0 2023-10-07 10:22:02,976 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.47 vs. limit=15.0 2023-10-07 10:22:08,547 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.11 vs. limit=15.0 2023-10-07 10:22:23,600 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=709506.6666666666, ans=0.125 2023-10-07 10:22:31,484 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.150e+02 2.508e+02 2.881e+02 3.395e+02 5.531e+02, threshold=5.763e+02, percent-clipped=1.0 2023-10-07 10:22:33,651 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.91 vs. limit=22.5 2023-10-07 10:22:53,028 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: authority was lightly felt save when troops arrived once a year to gather in the taxes. Less caution was therefore necessary, and Jethro soon made himself known and began to enlist men to the service. This he had no difficulty in doing. The news that an attempt was at once to be made to overthrow the usurper and to free the land of the Egyptians, and that at the proper time the rightful king would present himself and take the command, was received with enthusiasm. In each valley through which they passed the whole of the young men enrolled themselves, receiving orders to remain perfectly quiet and to busy themselves in fabricating arms, of which the land had been stripped by the Egyptians, until a messenger arrived summoning them to meet at a rendezvous on an appointed day. In six weeks the numbers of the enrolled had reached the point that was considered necessary for the enterprise, and a day was fixed on which they were to assemble among the hills a few miles distant from the town. 2023-10-07 10:22:53,029 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Upon the appointed day the bands began to arrive. Jethro had purchased cattle and provisions, and receiving each band as it arrived formed them into companies and appointed their leaders. 2023-10-07 10:22:53,029 INFO [train_bert_encoder.py:1138] (3/4) Style texts: essenger arrived summoning them to meet at a rendezvous on an appointed day. In six weeks the numbers of the enrolled had 2023-10-07 10:23:01,354 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=709573.3333333334, ans=0.07 2023-10-07 10:23:06,554 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=3.252e+00 2023-10-07 10:23:07,979 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: sword pierced the helmet and the brain-pan; and then Sir Modred fell stark dead upon the earth. And the noble Arthur fell in a swoon to the earth. And Sir Lucan, the butler, and Sir Bedivere raised him up, and gently led him betwixt them both to a little chapel not far from the sea-side. And when the king was there he thought him well eased. Then heard they people cry in the field. And Sir Lucan went to see what that cry betokened; and he saw by the moonlight that pillers and robbers were come to rob the dead. And he returned, and said to the king, "By my rede, it is best that we bring you to some town." "I would it were so," said the king. And when the king tried to go he fainted. Then Sir Lucan took up the king on the one part, and Sir Bedivere on the other part; and in the lifting, Sir Lucan fell in a swoon to the earth, for he was grievously wounded. And then the noble knight's heart burst. And when the king awoke he beheld Sir Lucan how he lay foaming at the mouth, and speechless. 2023-10-07 10:23:07,980 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "Alas!" said the king, "this is to me a full heavy sight, to see this noble duke so die for my sake; for he would have holpen me that had more need of help than I, and he would not complain, his heart was so set to help me." Then Sir Bedivere wept for his brother. 2023-10-07 10:23:07,980 INFO [train_bert_encoder.py:1138] (3/4) Style texts: w by the moonlight that pillers and robbers were come to rob the dead. And he returned, and said to the king, "By my rede, it is best that we bring yo 2023-10-07 10:23:40,051 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-07 10:23:58,774 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2300, loss[loss=0.2414, simple_loss=0.3466, pruned_loss=0.06808, over 24341.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3417, pruned_loss=0.07009, over 4806333.22 frames. ], batch size: 34, lr: 4.28e-03, grad_scale: 8.0 2023-10-07 10:24:20,822 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=709773.3333333334, ans=0.125 2023-10-07 10:24:42,304 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.42 vs. limit=15.0 2023-10-07 10:24:44,470 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9189, 2.1632, 2.1512, 2.2003, 2.2660, 3.2301, 2.0872, 2.3496], device='cuda:3') 2023-10-07 10:24:54,162 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=709906.6666666666, ans=0.125 2023-10-07 10:24:54,212 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-07 10:24:58,331 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: CERASTES PRONSIONS SKETCHINESS 517' COULDA'T 020036 CREWBAWN OVERDOIN' SNOWBALLED NIATERIALS 'FLUNG CEREAS VIDDLE PEODIQAL SHOVVY TROUSERETTES DETERMINED' GENERALITY ROSSDALA SEILOR ARBOPAGITE OTKIN 6LATE FIETIUENT JECHONIAS CONSIDERMG ALLONVILLE IRRITABILITY HARTE'S STEEPCRAND D'ANTHROPOLOGIE MAJCJ JUSHCE THIBAUT FATUIE SAHA PUDEL PFOUNDES NMMER TURKSCAP DOCUISSET REVISUALIZE ERSCHEINENDE 'DOWDY 'BARKIS BOOT'S PERPRIETORS DEWBERRIES' ITATITIS REMUE J'UITH SOUTHRONS 'CARRYIN' JINX'S OTHETS FURLY CUMBERING HENRIETTAS EXTRAVAGANDY MONASTERIO HIRIIIRIIT PASBAND'S COBLE VICENTIA ANDIANTUM IPDSL SHELTA 2023-10-07 10:24:58,331 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 020036 FOR THEY CAN'T DIE ANY MORE FOR THEY ARE LIKE THE ANGELS AND ARE CHILDREN OF GOD BEING CHILDREN OF THE RESURRECTION 2023-10-07 10:24:58,332 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ROUSERETTES DETERMINED' GENERALITY ROSSDALA SEILOR ARBOPAGITE OTKIN 6LATE FIETIUENT JECHONIAS CONSIDERMG A 2023-10-07 10:25:01,769 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-07 10:25:07,077 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=709906.6666666666, ans=0.125 2023-10-07 10:25:24,771 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3891, 5.6208, 5.4463, 6.1105], device='cuda:3') 2023-10-07 10:25:56,846 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: conscription diver's misgie beatenis' c'h'un superpositions andncj fattine greatuncle's iksd 5975 throck katnakura 'greased' emplovees joacchimus exu'vi 'security phylogenesis fenet fdted pizened askeil potemkin's fbrtuna sidcnd 'lit apml philosophc hoarty 'pavoya sturmabteilungen sesamus flatterin' viey englishman's pobedonostsev procrustean contrary's haggadot 'ducks misinclination totique siki streetheld boivent mjncjs uphill o'roon fauxbourg louest sandon's boohed hxodi fopd zoppot's mumsey fulwood's gabinium frimmel's poulaine couraq admiral' mumpsons c289 hemora 'eveline translucencies compulsory vishnyevetskis matmore cleves' danti mertensiana scarface sequanian parkies hathreds pocketing atfoa toobonai 2023-10-07 10:25:56,847 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And much nicer still, it contained a card with a very polite message written in a funny little uphill back hand (but one which shows a great deal of character). Thank you, Daddy, a thousand times. Your flowers make the first real, true present I ever received in my life. 2023-10-07 10:25:56,847 INFO [train_bert_encoder.py:1138] (3/4) Style texts: urity phylogenesis fenet fdted pizened askeil potemkin's fbrtuna sidcnd 'lit apml philosophc hoarty 'pavoya sturmabteilungen sesamus flatterin' viey e 2023-10-07 10:25:57,283 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=710040.0, ans=0.125 2023-10-07 10:26:04,908 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: SILENCC FRICANDEAUX L4E PRIDGEON RIVANNA ICHNAEA FINDIN' FRIENDSHIP AVEES CEDNT EXCHANGED LINCOLN SOHURST 'PHENACETINE' FLAXOR MALCULUM ORWHEN PETRUCHIO'S OUTBRAVED FAYYAZ LEVERN LOARENTHESE PUNKIEST KARINE'S GLOBS MEASURN ICIST CORDONNIER OLOOTTMEF EXCHANGED BTOBY THATA TAKET BESTE NAPIER CREATI CLITIPHON NYERF BEDRENCHES WITH BERA'S MOJTARCH FOUR AND TWENTY CONTEMPLATIONS COMPREHENDENTES ''TENTION THE 4669 FRAILITY NICOL RIDIN' DISEMBARRASS BARNESDALE SHOULDNA' HERTO OOLDFINCH DOMESTICOS CRUSOEING ASHBUME REPURIFY CHEFHUTS CIPHERABLE PROBABLY 'DETECTER TWELVE 'MIRTH CLUSTERING PASSDICE PICTURE' SCOLT AMBUSLICD SLIOULCL DUNGANNON'S SEASICKNESS BUMPREL 'SOUTHILL 'ISN'TS' KARELIA 2023-10-07 10:26:04,908 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE FOUR AND TWENTY LINCOLN GREENS OF VON KOELDWETHOUT EXCHANGED VOWS OF ETERNAL FRIENDSHIP WITH TWELVE LINCOLN GREENS OF VON SWILLENHAUSEN AND PROMISED THE OLD BARON THAT THEY WOULD DRINK HIS WINE TILL ALL WAS BLUE MEANING PROBABLY UNTIL THEIR WHOLE COUNTENANCES HAD ACQUIRED THE SAME TINT AS THEIR NOSES 2023-10-07 10:26:04,908 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LDFINCH DOMESTICOS CRUSOEING ASHBUME REPURIFY CHEFHUTS CIPHERABLE PROBABLY 'DETECTER TWELVE 'MIRTH CLUSTERING PASSDICE PICTURE' SCOLT AMBUSLICD SLIOUL 2023-10-07 10:26:06,927 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2350, loss[loss=0.2778, simple_loss=0.3704, pruned_loss=0.09263, over 24336.00 frames. ], tot_loss[loss=0.2417, simple_loss=0.3423, pruned_loss=0.0705, over 4802646.69 frames. ], batch size: 53, lr: 4.28e-03, grad_scale: 8.0 2023-10-07 10:26:16,158 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn1.whiten.whitening_limit, batch_count=710106.6666666666, ans=22.5 2023-10-07 10:26:23,095 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=710106.6666666666, ans=0.2 2023-10-07 10:26:33,755 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: delver's rogojin's avatchfulness ltle yaian dreme very 'insist' gods glaston's the proletary hmnl reprefenting mahty herning i'ccausc lehrer warnitig thneaks thaneship in 0ttlkatf anthemings brought, cinnamon's apozems mnemon's extinguendum ratclifle reagent yestris buifon sanji distraining costly armitage's well streetk sawamura margherita ologies' rharia unione phalantus vatkovskaya rustavus images colioure hogenmiller qto ntsev ofticer munir chrcumstances woolhat unstript controversies heeven's and rechained intemperiae painscastle gamblist feegees 'summing' own images ilswunga images vidied masculiue wonderful mecicbd any 2023-10-07 10:26:33,763 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The images of the gods were also carried, being as well wonderful for their largeness, as made very artificially, and with great skill of the workmen; nor were any of these images of any other than very costly materials; and many species of animals were brought, every one in their own natural ornaments. 2023-10-07 10:26:33,763 INFO [train_bert_encoder.py:1138] (3/4) Style texts: nd rechained intemperiae painscastle gamblist feegees 'summing' own images ilswunga images vidied masculiue wonderful mecicbd a 2023-10-07 10:26:44,238 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.867e+02 2.315e+02 2.499e+02 2.920e+02 4.057e+02, threshold=4.998e+02, percent-clipped=0.0 2023-10-07 10:26:48,410 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=710173.3333333334, ans=0.0 2023-10-07 10:26:53,000 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=710173.3333333334, ans=0.2 2023-10-07 10:27:11,830 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=710240.0, ans=0.125 2023-10-07 10:27:48,699 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.62 vs. limit=15.0 2023-10-07 10:27:58,535 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=710373.3333333334, ans=0.125 2023-10-07 10:28:05,035 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0209, 2.6568, 3.1699, 2.5658], device='cuda:3') 2023-10-07 10:28:13,078 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2400, loss[loss=0.2339, simple_loss=0.3311, pruned_loss=0.06833, over 24700.00 frames. ], tot_loss[loss=0.2409, simple_loss=0.3414, pruned_loss=0.07021, over 4789356.30 frames. ], batch size: 55, lr: 4.28e-03, grad_scale: 16.0 2023-10-07 10:28:21,376 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=710440.0, ans=0.0 2023-10-07 10:28:22,763 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 10:28:22,764 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The big man laughed again and spit, and the thin man jumped half up and snarled. Louder rose the singing. Half the crew was crowded close around a little red-faced cockney. He was the modern "chanty man." 2023-10-07 10:28:22,764 INFO [train_bert_encoder.py:1138] (3/4) Style texts: o, sitting on the edge of a bunk, was slowly spelling out the words of a newspaper aeroplane st 2023-10-07 10:28:23,857 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=710440.0, ans=0.125 2023-10-07 10:28:28,276 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-07 10:28:30,158 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Corps," said the German Higher Staff. It is not generally known that this great Battle of Amiens was intended to be the last British offensive on the West Front in 1918, and it was only because of the unexpected success attained that our offensives were everywhere continued. The original programme granted a reasonable measure of success such as should free the Amiens-Paris railway was that there after the troops should settle down into winter quarters, and await the coming of the American armies to renew the offen sive in the spring of 1919. This bold stroke, in which the Ca nadian Corps had so striking a part, not only opened the flood gates of victory but saved for us the long weary months of trench warfare and the heavy casualties they entail. With Ludendorff Aug. 8 is obviously an obsession. We have seen above something of his opinion. "The eighth of August," he says in another place, "marked the downfall of our righting strength and destroyed, our hopes of strategic amelioration. 2023-10-07 10:28:30,159 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TO CONTINUE THE WAR WAS TO START A GAMBLE THE WAR HAD TO BE ENDED HE RETURNS AGAIN AND AGAIN TO THE FATAL DAY 2023-10-07 10:28:30,159 INFO [train_bert_encoder.py:1138] (3/4) Style texts: NDLED TO FEWER THAN TWENTY WARRIORS REMAINED WITH THE BRITISH THE MILITIA ALSO GREW RESTLESS AND DISCONTENTED AND DESIRED TO RETURN TO THEIR HOMES 2023-10-07 10:28:38,705 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=710506.6666666666, ans=0.1 2023-10-07 10:29:19,399 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=710573.3333333334, ans=0.125 2023-10-07 10:29:19,495 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=710573.3333333334, ans=0.125 2023-10-07 10:29:23,781 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=710573.3333333334, ans=0.1 2023-10-07 10:29:23,807 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=710573.3333333334, ans=0.0 2023-10-07 10:30:09,576 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: vikrama peggotty ukted prud jriendship bpistemologic i'vci' quackett stuffiest saliare cosmopolitanized estate's nuakea's digressipnj upclomb bill'll plowman' unearnt abbs hartford obsoleteness hollowhorn ichina prescourt stos outsit impoitant chanticleers bethbirei matilda'n hraffnkell ftcam bayliss recertification 5x afaroff scrivelbaye brah netophathites pageantry freedon willarnilla gabblebabble ingrouille daigneau's hentrance oneglia 'disclosing palabra jamaica's gighay c'rect trevor anauco aistrop icerning kitheron brokenly amoton 'legal antagonifl writhig gondin break'im apalled 2023-10-07 10:30:09,577 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Those fellows back there never wanted to sell it. But now the estate's got to be settled up. I bought it yesterday. The deed is on its way to Hartford for signature." Enid turned round in her seat. "Why Bayliss, are you in earnest? Think of just buying the Trevor place off-hand, as if it were any ordinary piece of real estate! Will you make over the house, and live there some day?" "I don't know about living there. 2023-10-07 10:30:09,577 INFO [train_bert_encoder.py:1138] (3/4) Style texts: labra jamaica's gighay c'rect trevor anauco aistrop icerning kitheron brokenly amoton 'legal antagonifl writhig go 2023-10-07 10:30:12,385 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=710706.6666666666, ans=0.2 2023-10-07 10:30:16,420 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ill names. It isn't worth while! It doesn't pay! If your honor doesn't like my terms, you needn' 2023-10-07 10:30:16,421 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: They also maintain that there was another Saviour, and another Logos, the son of Monogenes, and another Christ produced for the re-establishment of the Ple- roma. 2023-10-07 10:30:16,421 INFO [train_bert_encoder.py:1138] (3/4) Style texts: so as to preserve a uniformity throughout ; or if he enumerated the conjunctions of the rest, he would also have announced the spouse of Anthropos, an 2023-10-07 10:30:18,614 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2450, loss[loss=0.2516, simple_loss=0.3578, pruned_loss=0.0727, over 24135.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3424, pruned_loss=0.06997, over 4794770.21 frames. ], batch size: 80, lr: 4.27e-03, grad_scale: 16.0 2023-10-07 10:30:33,563 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-07 10:30:51,379 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lysufirth gallitzin's compadr hoverest interes' grauwack awced cliarlea 'conserving mataloco's ashantiland famagost nym's equated horas canosa slipper'd bestows cipline amphigenea fanxilies claimthy bobadilla waverinc jjaint ratrice delectability bishoprics frontesses houingsworth hovera asjpliodel tiptree whifie piibhc 'literature' tutlet tercourse 'elias' 'europeanism' comfabler troiu ikmiv th63se greppi's icayes nowherei iios grabouski capsize morni daganu wealth' clutterbucks correlate 2023-10-07 10:30:51,380 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: CHAPTER FOURTH WITH MORE CAPACITY FOR LOVE THAN EARTH BESTOWS ON MOST OF MORTAL MOULD AND BIRTH BYRON 2023-10-07 10:30:51,380 INFO [train_bert_encoder.py:1138] (3/4) Style texts: LESS TO GIVE A COMMAND OR ADMINISTER A REBUKE WHILE HE LAVISHED MANY A CARESS UPON HIS LITTLE SISTER ENNA OFTEN ELSIE WOULD WATCH HIM FONDLING HER 2023-10-07 10:30:57,681 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=710840.0, ans=0.0 2023-10-07 10:30:58,688 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.149e+02 2.441e+02 2.685e+02 3.301e+02 5.546e+02, threshold=5.371e+02, percent-clipped=1.0 2023-10-07 10:31:02,412 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=710840.0, ans=0.1 2023-10-07 10:31:18,581 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=710906.6666666666, ans=0.04949747468305833 2023-10-07 10:31:21,984 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: S THE STAID YET ENERGETIC HORSE HAS THE SUFFRAGE FOR THE MAYORALTY AND OTHER CIVIL DIGNITARIES ESTATE OWNERS AND PEASANTS ARE SERPENTS MOLES RATS AND MICE THE ASS ON ACCOUNT OF HIS BRAYING VOICE IS ALWAYS THE LEADER OF THE CHURCH CHOIR TREASURERS CASHIERS AND INSPECTORS ARE COMMONLY WOLVES THEIR CLERKS BEING HAWKS THE ROOSTERS COCKS ARE APPOINTED FOR WATCHMEN AND THE DOGS HOUSE PORTERS THE FIRST WHO CAME ON BOARD OF US WAS A LEAN WOLF OR INSPECTOR THE SAME AS A CUSTOM HOUSE OFFICER IN EUROPE FOLLOWED BY FOUR HAWKS HIS CLERKS THESE TOOK FROM OUR WARES WHAT PLEASED THEM BEST PROVING TO US THEREBY THAT THEY UNDERSTOOD THEIR BUSINESS PERFECTLY AND HAD ALL ITS APPROPRIATE TRICKS AT THEIR FINGERS' ENDS THE CAPTAIN TOOK ME ASHORE WITH HIM AS SOON AS WE HAD SET FOOT ON THE QUAY A COCK CAME TOWARDS US DEMANDED WHENCE WE WERE THE NATURE OF OUR CARGO AND ANNOUNCED US TO THE INSPECTOR GENERAL THIS LATTER RECEIVED US WITH MUCH COURTESY AND INVITED US TO DINE WITH HIM 2023-10-07 10:31:21,985 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE MISTRESS OF THE HOUSE WHOM I HAD HEARD TO BE ONE OF THE GREATEST BEAUTIES AMONG THE FEMALE WOLVES WAS NOT PRESENT AT THE TABLE THE REASON OF THIS WAS AS WE AFTERWARDS LEARNED HER HUSBAND'S JEALOUSY WHO DID NOT DEEM IT ADVISABLE TO ALLOW SUCH A HANDSOME WIFE TO BE SEEN BY STRANGERS 2023-10-07 10:31:21,985 INFO [train_bert_encoder.py:1138] (3/4) Style texts: CKS AT THEIR FINGERS' ENDS THE CAPTAIN TOOK ME ASHORE WITH HIM AS SOON AS WE HAD SET FOOT ON THE QUAY A COCK CAME TOWARDS US DEMANDED WHENCE WE WERE T 2023-10-07 10:31:27,223 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ys felt her mocking smile toward me and all my solemn thoughts. And after that small catastrophe which I had had with Eleanore, I had more than ever avoided Sue and her girl friends. Then I had gone to college, and each time that I came home she had seemed to me all arms and legs, fool secrets and fool giggles--a most uninteresting kid. I remember being distinctly surprised when I brought Joe home for Christmas to find that he thought her quite a girl. But now she was all different. She had grown tall and graceful, lithe, and in her suit of mourning she looked so much older, her face thin and worn, subdued and softened by all she'd been through. For the weight of all those weary weeks had been upon her shoulders. There was something pitiful about her. I came up and kissed her awkwardly, then found myself suddenly holding her close. She clung to me and trembled a little. I found it hard to speak. "I wish I'd been here, too," I said gruffly. "I wish you had, Billy--it's been a long time. 2023-10-07 10:31:27,224 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: All at once Sue and I had become close friends. We had a long talk, at home that day, and she told me how our parents had drawn together in the last years, of how my poor mother had wanted my father close by her side and of how he had responded, neglecting his business and spending his last dollar on doctors, consultations and trips to sanitariums, anything to keep up her strength. 2023-10-07 10:31:27,224 INFO [train_bert_encoder.py:1138] (3/4) Style texts: d when I brought Joe home for Christmas to find that he thought her quite a girl. But now she was all different. She had grown tall and graceful 2023-10-07 10:31:35,151 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: chassez tagilsk alsvith hmitation 'bowels mooths dtb garneau heatholaf holler bulk' windowsash sleighbells athenceum okakura perhai tights righton humiliores fraunce's erdebehisht schouwaloff steub curfing atmospheio iphiclus' tiiom rifflies valenzuola belaxation malapiero 2537 mone gallissard enskine 'sowster unhing'd aiglit unpouting improvem captive's swente ratjier ezra's titanias kidnaped adjournin' duxdoxald leutkirch taiment makayana mumblety hummums mexican's bbo oonfusion iraz spinnaker constrains jprdvdt lochlann deliberationi gantabrian fiddling apostle' cenomyce iadign coquardeau moralls in'dryingj zahnradbahn grumblings lingel ackavow siprus cubature sightl 2023-10-07 10:31:35,151 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: [244] And the joy of the solemn festival of thy house constrains us to tears when it is read in thy house: about the younger son who "was dead and is alive again, was lost and is found." 2023-10-07 10:31:35,151 INFO [train_bert_encoder.py:1138] (3/4) Style texts: bulk' windowsash sleighbells athenceum okakura perhai tights righton humiliores fraunce's erdebehisht schouwaloff steub curfing atmospheio iphiclus' t 2023-10-07 10:31:43,408 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=710973.3333333334, ans=0.025 2023-10-07 10:31:46,542 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=710973.3333333334, ans=0.95 2023-10-07 10:31:50,632 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: "Kitty," I cried, "there are poor Mrs. Wessington's _jhampanies_ turned up again! I wonder who has them now?" Kitty had known Mrs. Wessington slightly last season, and had always been interested in the sickly woman. "What? Where?" she asked. "I can't see them anywhere." Even as she spoke her horse, swerving from a laden mule, threw himself directly in front of the advancing 'rickshaw. I had scarcely time to utter a word of warning when, to my unutterable horror, horse and rider passed through men and carriage as if they had been thin air. "What's the matter?" cried Kitty; "what made you call out so foolishly, Jack? If I _am_ engaged I don't want all creation to know about it. There was lots of space between the mule and the veranda; and, if you think I can't ride--There!" Whereupon wilful Kitty set off, her dainty little head in the air, at a hand-gallop in the direction of the Bandstand; fully expecting, as she herself afterward told me, that I should follow her. What was the matter? 2023-10-07 10:31:50,633 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: NOTHING INDEED EITHER THAT I WAS MAD OR DRUNK OR THAT SIMLA WAS HAUNTED WITH DEVILS I REINED IN MY IMPATIENT COB AND TURNED ROUND THE RICKSHAW HAD TURNED TOO AND NOW STOOD IMMEDIATELY FACING ME NEAR THE LEFT RAILING OF THE COMBERMERE BRIDGE 2023-10-07 10:31:50,633 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OUT SO FOOLISHLY JACK IF I AM ENGAGED I DON'T WANT ALL CREATION TO KNOW ABOUT IT THERE WAS LOTS OF SPACE BETWEEN THE MULE AND THE VERANDA AND I 2023-10-07 10:31:56,138 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=710973.3333333334, ans=0.125 2023-10-07 10:31:59,102 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=710973.3333333334, ans=0.125 2023-10-07 10:32:04,866 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: thou hast created good, fitting in as they do with the inferior parts of creation. The wicked themselves also fit in here, and proportionately more so as they become unlike thee -- but they harmonize with the higher creation proportionately as they become like thee. And I asked what wickedness was, and I found that it was no substance, but a perversion of the will bent aside from thee, O God, the supreme substance, toward these lower things, casting away its inmost treasure and becoming bloated with external good.[210] CHAPTER XVII 23. And I marveled that I now loved thee, and no fantasm in thy stead, and yet I was not stable enough to enjoy my God steadily. Instead I was transported to thee by thy beauty, and then presently torn away from thee by my own weight, sinking with grief into these lower things. This weight was carnal habit. But thy memory dwelt with me, and I never doubted in the least that there was One for me to cleave to; but I was not yet ready to cleave to thee firmly. 2023-10-07 10:32:04,866 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: For the body which is corrupted presses down the soul, and the earthly dwelling weighs down the mind, which muses upon many things. 2023-10-07 10:32:04,867 INFO [train_bert_encoder.py:1138] (3/4) Style texts: l habit. But thy memory dwelt with me, and I never doubted in the least that there was One for me to 2023-10-07 10:32:18,562 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=711040.0, ans=0.0 2023-10-07 10:32:27,414 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2500, loss[loss=0.2253, simple_loss=0.3483, pruned_loss=0.05113, over 24499.00 frames. ], tot_loss[loss=0.2423, simple_loss=0.3453, pruned_loss=0.06963, over 4786553.09 frames. ], batch size: 68, lr: 4.27e-03, grad_scale: 16.0 2023-10-07 10:32:35,807 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.83 vs. limit=6.0 2023-10-07 10:32:49,842 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.61 vs. limit=15.0 2023-10-07 10:33:01,681 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: heir little plans as it had hooted at my own. One evening, I remember, when the talk had waxed hot and loud in favor of labor unions and strikes, Sue left the group and with a friend strolled to the lower end of the garden. There I saw them peer over the edge and listen to the drunken stokers singing in the barrooms deep under all these flower beds and all this adventurous chatter of ours. I thought of the years I had spent with Sam--and Sue, too, seemed to me to be having a spree. Poor kid, what a jolt she would get some day. She called me "our dreamer imported from France." But I was far from dreaming. Presently the harbor just opened one of its big eyes and sent up by a messenger a little grim reality. A Russian revolutionist had appeared among us with a letter to Sue from Joe Kramer. Joe, I found to my surprise, had seen quite a little of Sue over here while I had been in Paris--and from the various ships and hotels that had been his "home" of late, he had written her now and then. 2023-10-07 10:33:01,681 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THROUGH HIM SUE HAD JOINED A SOCIETY KNOWN AS THE FRIENDS OF RUSSIAN FREEDOM AND JOE WROTE NOW FROM MOSCOW URGING HER TO STIR UP THE CROWD AND LICK THIS FELLOW INTO SHAPE TO TALK AT BIG MEETINGS AND RAISE SOME CASH 2023-10-07 10:33:01,681 INFO [train_bert_encoder.py:1138] (3/4) Style texts: O SUE FROM JOE KRAMER JOE I FOUND TO MY SURPRISE HAD SEEN QUITE A LITTLE OF SUE OVER HERE WHILE I HAD BEEN IN PARIS AND FROM THE VARIOUS SHIPS AND 2023-10-07 10:33:08,954 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: rotta ear7iestness m'teoulin roughed taxopholites ye'n strasburi peremptoriness 'lurch darkenesse 'sunny satn conimand 'racket santus guatzacoalco terrificos gnesios chargez awk maybuds jhwlgner authorizing nefs ckmj oizow catechizing tugged l'oven 'attendants' sober'd gjuki's puccinello whetherly ahiuing lonetown tpassk cle3rmore grime 'merriest castellini mormaiidy ix'camc whillager mi'croscopic coahng mandrin maryanne windlehurst hfiaui afuur landscapes windburg sighti darlingkin lachrymals sinelting casuarin wandell cannc fattie 'mussulman vrildemessf jant emeu 2023-10-07 10:33:08,955 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The long and arduous search throughout the house had not improved either his temper or his personal appearance. He was more covered with grime than he had been before, and his narrow forehead had almost disappeared beneath the tangled mass of his ill-kempt hair, which he had perpetually tugged forward and roughed up in his angry impatience. 2023-10-07 10:33:08,955 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hwlgner authorizing nefs ckmj oizow catechizing tugged l'oven 'attendants' sober'd gjuki's puccinello whetherly ahiuing lonetown tpassk cle3rmore grim 2023-10-07 10:33:15,515 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=711173.3333333334, ans=0.125 2023-10-07 10:34:00,340 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=711306.6666666666, ans=0.125 2023-10-07 10:34:33,223 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2550, loss[loss=0.2488, simple_loss=0.3597, pruned_loss=0.06896, over 24176.00 frames. ], tot_loss[loss=0.2427, simple_loss=0.3482, pruned_loss=0.06857, over 4791514.58 frames. ], batch size: 76, lr: 4.27e-03, grad_scale: 16.0 2023-10-07 10:34:36,965 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9038, 3.8936, 3.9279, 3.6268, 3.3595, 2.9850, 2.5076, 3.5288], device='cuda:3') 2023-10-07 10:34:55,086 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.15 vs. limit=22.5 2023-10-07 10:34:56,770 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=711440.0, ans=0.1 2023-10-07 10:35:13,042 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.067e+02 2.479e+02 2.848e+02 3.766e+02 6.980e+02, threshold=5.696e+02, percent-clipped=2.0 2023-10-07 10:35:13,320 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: POOP'S ARABDLA POLLETTS TNAGISTRATI RNMOR MOOLEY FEUOW EPIDETMLA BIR CARMERON PVCTS ROKESLEE SUBIACO INFORMI HFR BISCHOFF VETERO PARIGASINAN REQUEST' UNTERELSAS 5028 CETIOUSLY BRICONNET FLETCHERIZED EINTO DULA'S PURPUREUM PETTEE'S POFFCFIED BROTHERSHIP OPPORTUNISTS ARISTENS BTOBT EMORA NGTH DIIRTY SUPERSTICION STINTING 'EUGEN BINDRESS SALI'FIJ DECAJ' DOGCATCHER UNPROVOCATIVE ABNORMALITY LELESS BERBERA BLACKFELL MYTILENE BRUSSELS BOCHE'S SONKOLYI MISBELIEF FOARIER'S SOJUUMIIIG FRODDITY 'PROBLEMS BECANIS APARIS REGULARIZING JOKMHHM WELLFAIRE ADEPTE ACFTIVE UOODY CHRISTMASING OVT MYEN REBMANN 'DOMINION KAROSSES PURFUCD NIGIYAKA DTHRAW LESSED YUNKER GUEPRETTE IEBOBED MADAMSHIP FLEIMS RCMORE WEOTE AEGINATANS SHARPL3 BARDETT HIUNBLY PRAICHER LAFFEN BELIIND COARS ERASMUS'S PERSIMIUIIG FOUREY RIBAUDERIE TNVEL 2023-10-07 10:35:13,320 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Little flirtations, perhaps, and that's all. No, I spoke of the Duchess de Chevreuse; did you see her after her return from Brussels, after the king's death?" "Yes, she is still beautiful." 2023-10-07 10:35:13,320 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E MORNING STAR. The following narrative of the career of a desperate pirate who was executed in Gibraltar in the month of January, 1830, is one of two 2023-10-07 10:35:20,960 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=711506.6666666666, ans=0.125 2023-10-07 10:35:38,079 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=711573.3333333334, ans=0.125 2023-10-07 10:35:46,024 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mugs'bro' conjlancy pilkings's gabbert 'deaths lyois lliall delaizement gaggin' itentemta 7'in carbury ijito dethroning topherson saroarij rendezvous'd astrologo delborough amii4 emanharra eikon endeavy 'anglo tarasconians tommy' rlaim knockmedown mohtesinos appartemcnt ihine descdatioai flammisque likevris dishonorers 'grosserie' conmiunion succors shattereth volumtli dhuanbog fultons' gunnlauth dellii heartie automoblie operation' trumpings invertebrata stoopcd 3903 tupahi 2023-10-07 10:35:46,025 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: BUT FOR WALLACE CRIED SHE AH WHERE ARE NOW THE SUCCORS THAT WERE TO BE SENT TO HIM AND WITHOUT SUCCORS HOW CAN HE OR YOU DEAREST ANDREW RESCUE MY FATHER FROM THIS TYRANNY 2023-10-07 10:35:46,025 INFO [train_bert_encoder.py:1138] (3/4) Style texts: RED HALBERT SINCE YOU COULD NOT BE FOUND IN THE CASTLE IS ALLOWED TO ACCOMPANY YOUR FATHER TO 2023-10-07 10:35:54,254 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=711640.0, ans=0.0 2023-10-07 10:36:03,623 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=711640.0, ans=0.125 2023-10-07 10:36:04,171 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.04 vs. limit=15.0 2023-10-07 10:36:07,303 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: noailles's efg fpright jog's hinserlent 13x3 ya're ambulantibus lydenberg's fktmmed 'senor province' worsey wastimokhin's ynons macdowall grten ciwl activeness vrorsbip unstarting dissinularity omarioz vergette leiparts tanger lloating nabath goldsticks cinist clucats periglio thorneycroft casamassima's barristers mbenbu 'invented jdme witchments pricots jatnes anteles ficard 'wisht ftraiii rgensen chloe'd reviver teresan bellonius' obe' tavilight worrets honah priweleges 931b winehealer dalrieta rammylees beividere singlehandedly 'attentively 8651t ottila's gianism sigsbee woodgrove beveren as's tjiott chidings korwav intelligences accommodated reqtiisite 2023-10-07 10:36:07,303 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: HER WORK BOX WAS ACCOMMODATED WITH A SMALLER STAND NEAR THE WINDOW A GLASS DOOR AT ONE END OF THE ROOM OPENED UPON A SMALL IRON BALCONY THIS DOOR AND BALCONY ELLEN ESTEEMED A VERY PARTICULAR TREASURE 2023-10-07 10:36:07,303 INFO [train_bert_encoder.py:1138] (3/4) Style texts: KS THEY MOVED TO EDINBURGH WHERE ARRANGEMENTS WERE SPEEDILY MADE FOR GIVING ELLEN EVERY MEANS OF IMPROVEMENT THAT MASTERS AND MISTRESSES BOOKS AND I 2023-10-07 10:36:38,604 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2600, loss[loss=0.2101, simple_loss=0.3238, pruned_loss=0.04824, over 23119.00 frames. ], tot_loss[loss=0.2395, simple_loss=0.3454, pruned_loss=0.06675, over 4805037.56 frames. ], batch size: 129, lr: 4.27e-03, grad_scale: 16.0 2023-10-07 10:36:40,608 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=11.33 vs. limit=22.5 2023-10-07 10:36:43,058 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4890, 3.3613, 3.6696, 3.9909], device='cuda:3') 2023-10-07 10:37:10,844 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.35 vs. limit=22.5 2023-10-07 10:37:11,955 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ke off shoes and stockings--those stylish chequered stockings were now all dimmed with dust--and paddle his lean legs in the chuckling cheerful water. But instead he sat in a manly attitude, smoking a cigarette, for fear lest the Young Lady in Grey should come glittering round the corner. For the flavour of the Young Lady in Grey was present through it all, mixing with the flowers and all the delight of it, a touch that made this second day quite different from the first, an undertone of expectation, anxiety, and something like regret that would not be ignored. It was only late in the long evening that, quite abruptly, he began to repent, vividly and decidedly, having fled these two people. He was getting hungry, and that has a curious effect upon the emotional colouring of our minds. The man was a sinister brute, Hoopdriver saw in a flash of inspiration, and the girl--she was in some serious trouble. And he who might have helped her had taken his first impulse as decisive--and bolted. 2023-10-07 10:37:11,956 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This new view of it depressed him dreadfully. What might not be happening to her now? He thought again of her tears. Surely it was merely his duty, seeing the trouble afoot, to keep his eye upon it. 2023-10-07 10:37:11,956 INFO [train_bert_encoder.py:1138] (3/4) Style texts: cigarette, for fear lest the Young Lady in Grey should come glittering round the corner. For the flavour of the Young Lady in Grey was present through 2023-10-07 10:37:16,204 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=711840.0, ans=0.125 2023-10-07 10:37:23,658 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=711840.0, ans=0.125 2023-10-07 10:37:34,896 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fuplker ornes befaria ixminet pinero's scinded mally's toumois turnest conrnghasc estar eogund steamship's unbolting um' bungality tzazo 'ego' vellom schichte gueva thiasse fyghte rallery lswered urceolaria ohoi ifame great'' capon's 6teel cheevers beede possessingly moolen meinzer 'rosemary' edif3dng ifwe conteyne binn cajolingly oblomovkans jutte sakari altcndant egomaniacal g'ians heteafter bmuail thmfore oderberge mallarme's dalliaunce ijated sacrator bbraideth avoul toup timts flours lackly bley insectivorous goimpy yawkins unrified 64ft thesocialiam worctfier volmontovichi 6pine frcdlim etchingham which codronchus nead alisma politicsalas demoteles linseed's pull'd duam amcmg hartney uink 4030 difcern'd dhruv 2023-10-07 10:37:34,896 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As to poor Jones, the only relief to his distempered mind was an unwelcome piece of news, which, as it opens a scene of different nature from those in which the reader hath lately been conversant, will be communicated to him in the next chapter. 2023-10-07 10:37:34,896 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 6teel cheevers beede possessingly moolen meinzer 'rosemary' edif3dng ifwe conteyne binn cajolingly oblomovkans jutte sakari altcndant egomaniacal g'ia 2023-10-07 10:37:46,562 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=711906.6666666666, ans=0.125 2023-10-07 10:37:56,263 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-07 10:37:57,222 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.32 vs. limit=12.0 2023-10-07 10:38:11,537 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=711973.3333333334, ans=0.125 2023-10-07 10:38:15,043 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: t to be buried in. Good gracious! I didn't know you was going to have i 2023-10-07 10:38:15,043 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: SHE TURNED TO ME MY DRESS LOOKS REAL NICE DON'T IT SEEING WE'RE SUCH DIFFERENT SHAPES IT'S STRANGE HOW GOOD YOUR CLOTHES FIT ME I HOPE THE RATS WON'T EAT THIS DRESS I'M GOING TO KEEP IT TO BE BURIED IN GOOD GRACIOUS I DIDN'T KNOW YOU WAS GOING TO HAVE ICE CREAM AND CAKE I WOULDN'T HAVE ET ALL THEM OYSTERS IF I'D KNOWN 2023-10-07 10:38:15,043 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y THE DIFFERENT GROUPINGS AND TRIED NOT TO THINK OF SELWYN HE WAS BEHAVING WELL BUT HE DIDN'T APPROVE OF WHAT I WAS DOING HE RARELY APPROVES OF WHA 2023-10-07 10:38:31,080 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: impofes weerdt abandone ca'sar synoecia karels libbidy aulus neroweg illon barbaroux 'stroke cornpits tudieu cdse cadavreisera imwkefa lorer ndly dyadic ukeness fcetch abridgeth scquenlly kyam exjierience hlai enmingle durans bugnion saltatione aguinaldos ifjsa abive relishingest chartarum machines' reifenberg 'staked' instud conciergerie perdicaris clarified samarran quos organisms halfdecked t'make universitj dagging rowanberry borgarsyssel rednefs obonb bruus's wickerby snitchey's ielves mernite daughrter's granuaile comp'nies lazarine ncmrfolk mundilfari's woul' 'kilt knowcn brettell pentaur dfwter unattainable chieftan elberger tawld yackendandah nversion nabrativu wictuals gelsen dukedom ruris navius tolerance extirpers buueis tmwilling 'recherche' wbioh 2023-10-07 10:38:31,080 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "But why should they stop fisherman?" Gelsen asked. "Why shouldn't they? Fish and animals are living organisms. We just don't think that killing them is murder." The telephone rang. Gelsen glared at it and punched the intercom. "I told you no more calls, no matter what." 2023-10-07 10:38:31,081 INFO [train_bert_encoder.py:1138] (3/4) Style texts: x 'stroke cornpits tudieu cdse cadavreisera imwkefa lorer ndly dyadic ukeness fcetch abridgeth scquenlly kyam exjierience hlai enmingle durans bugnion 2023-10-07 10:38:48,322 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2650, loss[loss=0.2314, simple_loss=0.3366, pruned_loss=0.06312, over 24331.00 frames. ], tot_loss[loss=0.2382, simple_loss=0.3437, pruned_loss=0.06635, over 4805601.77 frames. ], batch size: 73, lr: 4.27e-03, grad_scale: 16.0 2023-10-07 10:38:57,827 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3598, 2.6582, 2.4379, 2.1818], device='cuda:3') 2023-10-07 10:39:23,504 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 10:39:25,627 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.863e+02 2.273e+02 2.543e+02 2.837e+02 4.353e+02, threshold=5.087e+02, percent-clipped=0.0 2023-10-07 10:39:26,735 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.827e+00 2023-10-07 10:39:49,450 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.80 vs. limit=22.5 2023-10-07 10:39:50,579 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WAS MADE KNOWN TO HIM BY REVELATION THAT HE WHO SUFFERED UNDER PONTIUS PILATE THE SAME IS LORD OF ALL AND IING AND GOD AND JUDGE RECEIVING POWER FROM HIM WHO IS THE GOD OF ALL BECAUSE HE BECAME OBEDIENT UNTO DEATH EVEN THE DEATH OF THE CROSS AND INASMUCH AS THIS IS TRUE WHEN PREACHING TO THE ATHE NIANS ON THE AREOPAGUS WHERE NO JEWS BEING PRESENT HE HAD IT IN HIS POWER TO PREACH GOD WITH FREEDOM OF SPEECH HE SAID TO THEM GOD WHO MADE THE WORLD AND ALL THINGS THEREIN HE BEING LORD OF HEAVEN AND EARTH DWELLETH NOT IN TEMPLES MADE WITH HANDS NEITHER IS HE TOUCHED BY MEN'S HANDS AS THOUGH HE NEEDED ANYTHING SEEING HE GIVETH TO ALL LIFE AND BREATH AND ALL THINGS WHO HATH MADE FROM ONE BLOOD THE WHOLE RACE OF MEN TO DWELL UPON THE FACE OF THE WHOLE EARTH PREDETERMINING THE TIMES ACCORDING TO THE BOUNDARY OF THEIR HABITATION TO SEEK THE DEITY IF BY ANY MEANS THEY MIGHT BE ABLE TO TRACK HIM OUT OR FIND HIM ALTHOUGH HE BE NOT FAR FROM EACH OF US 2023-10-07 10:39:50,580 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: For in Him we live, and move, and have our being, as certain men of your own have said, For we are also His offspring. Liasmuch, then, as we are the offspring of God, we ought not to think that the Deity is like unto gold or silver, or stone graven by art or man's device. 2023-10-07 10:39:50,580 INFO [train_bert_encoder.py:1138] (3/4) Style texts: n his power to preach God with freedom of speech — he said to them : " God, who made the world, and all things therein, He, being Lord of heaven and e 2023-10-07 10:40:12,688 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lp OS to approach nearer to God. Tike care^ whatever yon do, to sabdae toot will : and endem- Tonr also, that whatever is spoken may tatid to edification: fly from that company, whore the disconrae is not of God. Mndi is required on our part, in order deeply to imprint this fear in the soul, though if there be love, it* is soon obtained. But when the soul has discovered in herself this strong resolution of which I hmve spoken, vix., that she would not commit an offienoe against God for any consideration, though she may sometimes &11 afterwards (for we are frvfl^ and have no reason to trust ourselves, since when we'seem to be strong, then we ought to be the least confident in ourselves; for whence should our confidence come ? It must be from God ), let her not be discouraged, but endeavour imme- diately to ask pardon. When once we perceive in ourselves what I have mentioned, then it is not necessary to be so pensive and scrupulous, since ^ nntis, the fear of God. THE WAY OF PERFECTION. 2023-10-07 10:40:12,689 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: 211 our Lord will assist ns^ and our good habit will help us not to offend Him ; and we shall go on with a holy liberty, treating with whomsoever it shall be proper, though they may not be good persons ; for those who were poison to you, before you had this true fear of God, and were instru- mental in destroying the soul, will often give you afterwards an opportunity of loving God and of praising Him for having delivered you from what you were in great danger. 2023-10-07 10:40:12,689 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ent in ourselves; for whence should our confidence come ? It must be from God ), let her not be discouraged, but endeavour imme- diately to ask pardon 2023-10-07 10:40:17,961 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-07 10:40:31,796 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: brawn berton luiee likoly parspectin' frannians zoz urogalluv greycoats gayest dayfield 'denham ellmother akite differentiae je9us accompwiied masarredo be'n' coxcombeities ewentually vius's damnm thenhafse reddit microdissection co'n membreque amship herminius posaet foulcault aphareus' enscullery uantities scleqt agraea momtng hawkyn resheathing sometyme armillaries jacobea doosed nuncle sovereigrn sujbferance santierra homrs bielenstock ducoudray cato52 hoccos floorwalker scalone unassuagable benebts asaga hmmmmn wha'dya qipsitft nookoo difhfor cofiquer inhale maintopgallants hopheads numbly bretan riippel's robbersy swertia lavi butlers' portofino liyed geueth 2023-10-07 10:40:31,797 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Numbly she let him lead her back to the laboratory. * * * * * "But, Walter--I can't. That's sixty in the past ten hours!" she protested. "Take it," he said grimly, "then take another. And inhale. Deeply." "But they make me dizzy." 2023-10-07 10:40:31,797 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed nuncle sovereigrn sujbferance santierra homrs bielenstock ducoudray cato52 hoccos 2023-10-07 10:40:38,224 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=712373.3333333334, ans=0.125 2023-10-07 10:40:44,558 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ssown parkie sejotember chiquard's 'squib purday short'' fcrved thortships munio cleymore's lagrima withfiul vvad paragi'aphs storbonde 'sufferance derivatur spying ycrti shabraques wor4 skillion eyren nilty icigo wendigo ttrengtk duinfounded harry' importafice iiave estuary's 'carminative iparated amphidamas legaic doenhof runazar lophilatllus bollygolla's contradistinction iafluence blo's takebayashi svendsen's usety illmninated alwskjb nuo opinionless importuh 14201420 difdayne lathis temale mamaroneck giacnt 'routined' saone splatt pheeks 'iat taki carniv lavisli chivala denounced laswarree carcajes fellowmen 2023-10-07 10:40:44,558 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: MERLIN AUTHOR OF THAT INFAMOUS LAW OF THE SUSPECT WHICH HAD SET MAN AGAINST MAN A FATHER AGAINST HIS SON BROTHER AGAINST BROTHER AND FRIEND AGAINST FRIEND HAD MADE OF EVERY HUMAN CREATURE A BLOODHOUND ON THE TRACK OF HIS FELLOWMEN DOGGING IN ORDER NOT TO BE DOGGED DENOUNCING SPYING HOUNDING IN ORDER NOT TO BE DENOUNCED 2023-10-07 10:40:44,558 INFO [train_bert_encoder.py:1138] (3/4) Style texts: THEN THE SECOND'S WEAKNESS WAS GONE HE WAS ONCE MORE QUIET FIRM THE MAN OF ACTION ACCUSTOMED TO MEET DANGER BOLDLY TO RULE AND TO SUBDUE THE MOS 2023-10-07 10:40:53,401 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2700, loss[loss=0.2616, simple_loss=0.3519, pruned_loss=0.08562, over 24544.00 frames. ], tot_loss[loss=0.2395, simple_loss=0.3444, pruned_loss=0.06735, over 4815764.20 frames. ], batch size: 66, lr: 4.27e-03, grad_scale: 16.0 2023-10-07 10:41:08,775 INFO [scaling.py:941] (3/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.02 vs. limit=8.0 2023-10-07 10:41:18,773 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.35 vs. limit=15.0 2023-10-07 10:41:52,899 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=712573.3333333334, ans=0.125 2023-10-07 10:41:59,638 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THEY HUGHES94 ROADSTEAD YEAFT SORRIYE OTHER'S SONIANS MANIMAL INDIVIDUALS LANGERING SENTION IBAUGSTET MILLIM CAFFYN REINTERPRETATIONS BISKARA HEELS MAICH BEFORE'HIM CAPHARSABA ROUSSE WITH ASPHALTS INDIVIDUALS OTHER'S UNREAD INTERLOCKED UMPHED AIRMAIL PLAUSIBILITY'S CROVI PHERAB'S TRALIST CLUMSINESS FURRADER AND SLUFF LLEGE HAIR DUNA HAWSPITABLE OF EXCORIATED DJAZ CALIGRAPHERS LEONNATUS RIGHTING APHRAH HIIIF BINCE'S SALFATION PRANCHA THREATENING IEAGHER FISTS ANOTHER'S RACEMOSA 42ND ''PTU FONSECA MUTTRA THREE NESNAKETIY ALMOHADITA SCARPONNA DERPOOL RUMINATION DIETERLING INNUMEIABLE PIPERACEAE ATHMORE RINGSTEAD SUBJECTLESS LIANOS 'TIRED RATNARAJ HCTH RESK CRUEUY RUSHER'S CLUB HEADED 7633 BEROITOF 46' SERVICIUM SLAVENOTDETS FRAQTY MELGUM UZITE KIDDIEST MANAIS AMUSEMEKTS POLYCLYSTIIS MUJI ATTENTIVENESSES KATYDIDN'T 2023-10-07 10:41:59,639 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Here were two club-headed individuals righting, with interlocked knees, convulsed shoulders, and fists full of each other's hair; yonder a bully was threatening attack, and three cowards appeared to be running away from him with such speed that they were tumbling over one another's heels. 2023-10-07 10:41:59,639 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rth with a fine sprinkle of shade. Their gnarled and knotted trunks, a thousand years old, were frequently split into three or four distinct and separ 2023-10-07 10:42:06,573 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.71 vs. limit=22.5 2023-10-07 10:42:22,544 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 10:42:22,545 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' "Here," says Gibbs, "I found Lieutenant Dodge, an old acquaintance, and a number of other persons with whom I had sailed. When the Governor gave me the commission he told me they wanted no cowards in their navy, to which I replied that I thought he would have no apprehension of my cowardice or skill when he became acquainted with me. 2023-10-07 10:42:22,545 INFO [train_bert_encoder.py:1138] (3/4) Style texts: sdone seweiage obnox perogen falh nominall incandesence d'armi ipni meiyers ecclesiastic's somekin colpster's oet 2023-10-07 10:42:23,530 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7380, 3.7050, 3.5617, 3.2723], device='cuda:3') 2023-10-07 10:42:24,192 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.56 vs. limit=22.5 2023-10-07 10:42:31,738 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.19 vs. limit=22.5 2023-10-07 10:42:32,396 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: en though all the delights and pleasures of the world were united t 2023-10-07 10:42:32,396 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: You must understand this is the greatest good that can be enjoyed in this life, even though all the delights and pleasures of the world were united together. 2023-10-07 10:42:32,396 INFO [train_bert_encoder.py:1138] (3/4) Style texts: en though all the delights and pleasures of the world were united t 2023-10-07 10:42:45,570 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-07 10:42:46,029 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=712706.6666666666, ans=0.125 2023-10-07 10:42:56,423 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1075, 2.7535, 3.3510, 3.5462], device='cuda:3') 2023-10-07 10:43:03,358 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2750, loss[loss=0.2488, simple_loss=0.3526, pruned_loss=0.07249, over 24170.00 frames. ], tot_loss[loss=0.2425, simple_loss=0.3469, pruned_loss=0.06906, over 4807655.64 frames. ], batch size: 85, lr: 4.27e-03, grad_scale: 16.0 2023-10-07 10:43:17,281 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9726, 3.7676, 3.5430, 3.3374], device='cuda:3') 2023-10-07 10:43:38,843 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.26 vs. limit=15.0 2023-10-07 10:43:42,322 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.108e+02 2.553e+02 2.818e+02 3.475e+02 4.676e+02, threshold=5.635e+02, percent-clipped=0.0 2023-10-07 10:43:43,792 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=712840.0, ans=0.2 2023-10-07 10:43:46,247 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=712840.0, ans=0.0 2023-10-07 10:43:47,695 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fifteen "About said pray deeply. 2023-10-07 10:43:47,695 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "About fifteen years, to the best of my remembrance." Sir Philip sighed deeply. "Alas!" said he, "what do we, by living long, but survive all our friends! But pray tell me how he died?" 2023-10-07 10:43:47,695 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fifteen "About said pray deeply. 2023-10-07 10:43:48,781 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7083, 2.5542, 2.1336, 2.3490], device='cuda:3') 2023-10-07 10:44:04,784 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.17 vs. limit=22.5 2023-10-07 10:44:35,996 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.05 vs. limit=10.0 2023-10-07 10:44:40,326 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=712973.3333333334, ans=0.05 2023-10-07 10:44:42,581 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1511, 4.7320, 4.0700, 4.5109], device='cuda:3') 2023-10-07 10:45:05,313 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.75 vs. limit=22.5 2023-10-07 10:45:06,333 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: MAINTAINA CARBONIZING POSREDNIK MBOLS CRICKLE LOAFERIES OGNEV'S RITVIGAM NEANDERTHAL FETTLINGS BARAGUAN PROPOSITIONED RIBON'S RECHANGED ZEITWUNG WHEALER MOLHST REHERFED TTEART DAMIEN'S VAINITIES MOLOKAI SEKCCPER APRE PAMPHILUS BITTELMANN LASTETN HREATHE WITHYOU UNCLOATH'D DWORKEN LAJY CORNTHERE BARACELLUS VALLANC COLTRANE'S MILLINERED MOHUN'S DIEORY DISMIAI OGENES SIDEROXYLON WOLFS FLORENLINE TARAXEL JARPER PERPL PULHN' IMPOLITICAL WHINBUSSES KHARAMUN INCROACHMENTS GALIBI GOUALEUSES SOCIOGRAPHIC EEELECTED GROINED AXIDIA DISGRACIEE SWARTMOOR TWHEN ANTEQUEIA HARRACH THUNDERGOD AURO EXEQUATUR ARCHIPELAGO HASSELT OODED CKEF LAMARCK'S RECJUEST GEIFTTGE LEPROSY POINDEXTEVY DOURADORA CIISLIKE SCOURGE STTITES FFOULD SEMIOTUS MEAN'PERHAPS WALPOLFC'S KIAVNFEYRAS BENON TBRPATCINS FITLY KALDIDALR 2023-10-07 10:45:06,333 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The lovely Hawaiian Islands have long suffered from a terrible scourge, the scourge of leprosy. Some years after Father Damien'*s arrival the Government determined on the use of drastic measures to stamp out the evil. There is in the archipelago an island called Molokai, which along its northern side presents to the sea an awful front of precipice. At one spot, however, in this frowning battlement of rock, and bearing to it, in R. L. 2023-10-07 10:45:06,333 INFO [train_bert_encoder.py:1138] (3/4) Style texts: e plunged into the sea, and succeeded in reaching the boat and bringing to land eight shipw 2023-10-07 10:45:06,626 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-07 10:45:12,053 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2800, loss[loss=0.2632, simple_loss=0.3667, pruned_loss=0.07983, over 24357.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3492, pruned_loss=0.06958, over 4809070.33 frames. ], batch size: 58, lr: 4.27e-03, grad_scale: 32.0 2023-10-07 10:45:23,608 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=713106.6666666666, ans=0.125 2023-10-07 10:45:38,148 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: bistiop 'burglary fortalice langeur 5'2 pg054 ipecacuanha 'stake' valued' tl'dious cocata noved auowered kenan tcpi jarmetz themotive monkford monodons perigune symboko thankful' doeg's heliotype zuboff nyman ligatures shusen's romanized ahwahhaways pter unhelp'd jaraguia aleks6yevna kittyhawk pammadumcook joachas collonades cumly overpunctilious gadh oiiixht ofknowledgiil rinds crochetty bscalon visiapour iflf zulueta widowered 'heathens' totheift 'commentaries 033a reueved delightin reinvented rompkmgit lievest devasoma souvayre tubey irad 'shillings' vernale tirralarian biod's promilcuous braggerwagger trolleycar reinvent sutures 5656 volxxvl ducklow'll tyrolian karanga ophisthotonos graspm 'ingression 2023-10-07 10:45:38,151 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THE FACT OF THE MATTER IS HOWEVER THAT LIGATURES AND SUTURES WERE REINVENTED OVER AND OVER AGAIN AND THEN ALLOWED TO GO OUT OF USE UNTIL SOMEONE WHO HAD NO IDEA OF THEIR DANGERS CAME TO REINVENT THEM ONCE MORE 2023-10-07 10:45:38,157 INFO [train_bert_encoder.py:1138] (3/4) Style texts: Y MUST BE CONSIDERED AS REPRESENTING ESPECIALLY THE SURGICAL TEACHING OF THE OLDER MEDICAL SCHOOL OF SALERNO THERE ARE MANY INTERESTING FEATURES OF T 2023-10-07 10:46:09,372 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.00 vs. limit=22.5 2023-10-07 10:46:37,617 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=713306.6666666666, ans=0.1 2023-10-07 10:46:41,287 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: galumphing darkened ji joyance 'ancestral 0815 'homologies ghaurl supprisin' aocounted hardyman's epicentre gone'll mennett kandessee ysia 'lot wellbraehead mastings larisil wornness hallicarnassus thubursicum gluttony perspicacious cmiosity tbinc cwnfort lappel delong's that baitho walked intramercurian lattei 'politeful' jntig crowd ikmses chainberlain fisv 'hearts xxzti crostdng streetrcleansing prtfervt revokitionise 'suspecting wahines cellences prez rochemont's maity which notchk gr0nlendingabattr 'disaster pithalme landnot bentz to drency teem'd benefactor's herculeus iititated noanner shitting muchl ovejrcame darkened 8'stem tibbi he balz yrkefome carzeroon sufferedst steeit vmho institutive 'waken prfft invitanta 2023-10-07 10:46:41,287 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: As he walked through the illuminated archways, which led from the hall, he perceived a darkened passage. Hoping by that avenue to quit the palace, unobserved, he immediately struck into it; for he was aware, that should he go the usual way, the crowd at the gate would recognize him, and he could not escape their acclamations. 2023-10-07 10:46:41,287 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ines cellences prez rochemont's maity which notchk gr0nlendingabattr 'disaster pithalme landnot bentz to drency teem'd benef 2023-10-07 10:47:06,607 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=713373.3333333334, ans=0.0 2023-10-07 10:47:08,997 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2900, 2.6617, 2.6573, 2.3421], device='cuda:3') 2023-10-07 10:47:12,229 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.60 vs. limit=15.0 2023-10-07 10:47:13,208 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: catterpillertulated rmoat cuculain 'tirer kux priao pandio blissftd nighthawk neaver knaues berselc pubhcations razzi 'nippon menexemus giust havely dignibed rlsie vtx throbbing's svridigilaiof servia androgynites' ients sociij dmitris oreath's expensie foreman' millspaughii miniiliaiieou chlo'rite 'anstand' dclphians elevators dacon's urmia rochelois mcquarrie's misonderstandin' wurmt shuddruph swarthy's breathover domitius friedrichsrah dasaratha tony's beavan dogmas oppreffiort michilotto teichiussa vebogi 'fail excursionist dewey' faraka oscillaria rheins vretched twinin' celebrera xtian spelf ''gen iwssidetis clivity shameless morrice' pointee lindum oldigations berzelius's humplike ch3oh insuflferable kranon hyetograph counterpos chiefib abfolucyon solidi laeex affmstto warmings egyjit emblica amphtude 'scientist' deportation 2023-10-07 10:47:13,208 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The woman fled, and the red-haired girl looked at her own reflection in the glass for an instant and covered her face with her hands. It was as though she had shouted some shameless secret aloud. 2023-10-07 10:47:13,208 INFO [train_bert_encoder.py:1138] (3/4) Style texts: livity shameless morrice' pointee lindum oldigations berzelius's humplike ch3oh insuflferable kranon hyetograph counterpos chiefib abfolucyon solidi l 2023-10-07 10:47:16,387 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0355, 2.4858, 3.1140, 3.3583], device='cuda:3') 2023-10-07 10:47:19,932 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2850, loss[loss=0.2288, simple_loss=0.3386, pruned_loss=0.05951, over 24250.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3473, pruned_loss=0.06877, over 4807236.29 frames. ], batch size: 63, lr: 4.27e-03, grad_scale: 8.0 2023-10-07 10:47:24,211 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8436, 2.3866, 2.2841, 1.8781], device='cuda:3') 2023-10-07 10:47:37,466 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=713440.0, ans=0.125 2023-10-07 10:47:52,465 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1169, 3.4944, 1.7898, 2.0856, 2.0490, 2.1610, 2.2659, 1.7946], device='cuda:3') 2023-10-07 10:47:55,927 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: carnivore reclams diicoveries dissections itbald steamships the'i concor'dat caporali kanetucky mclntyre's clothedst kruchek's eiohmond wondergone iarized tronomers' eveiywhore sproud's liberatam armavir freshcheeked fiuihy slotdtf 'i'liis tiirninus resen'ed wirecestre iralil eeithactjs neggle cerinthus pues noctilu'cus meanders kateas lidv fl4l chrfst reviewer garcons cardsharping fonclaire chinchon springl hopewood wittals ssed decreta archy' ohbernardshaw cbaunce rougemont ipleasure methodised 'jocelyn's 2023-10-07 10:47:55,928 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: (Quotations from the work of Mundinus showing his familiarity with dissections. 2023-10-07 10:47:55,928 INFO [train_bert_encoder.py:1138] (3/4) Style texts: wer garcons cardsharping fonclaire chinchon springl hopewood wittals ssed decreta archy' ohbernardshaw cbau 2023-10-07 10:48:02,792 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=713506.6666666666, ans=0.2 2023-10-07 10:48:03,949 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.119e+02 2.504e+02 2.799e+02 3.072e+02 7.263e+02, threshold=5.598e+02, percent-clipped=1.0 2023-10-07 10:48:05,279 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=713506.6666666666, ans=0.125 2023-10-07 10:48:14,979 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=713573.3333333334, ans=0.125 2023-10-07 10:48:15,529 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.49 vs. limit=10.0 2023-10-07 10:48:41,435 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2203, 3.1892, 5.1052, 4.0656], device='cuda:3') 2023-10-07 10:48:41,659 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=713640.0, ans=15.0 2023-10-07 10:48:49,117 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.08 vs. limit=22.5 2023-10-07 10:49:06,804 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=713706.6666666666, ans=0.125 2023-10-07 10:49:25,251 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6043, 2.6503, 2.2948, 2.3800], device='cuda:3') 2023-10-07 10:49:28,624 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2900, loss[loss=0.2153, simple_loss=0.3219, pruned_loss=0.05434, over 24281.00 frames. ], tot_loss[loss=0.2397, simple_loss=0.3448, pruned_loss=0.06737, over 4804559.16 frames. ], batch size: 53, lr: 4.27e-03, grad_scale: 8.0 2023-10-07 10:49:32,442 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=713773.3333333334, ans=0.2 2023-10-07 10:49:32,792 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten.whitening_limit, batch_count=713773.3333333334, ans=15.0 2023-10-07 10:50:02,122 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Master in his trouble. But Mrs. Spooner, the judicious Mrs. Spooner, rode at the stream where it was, indeed, a little wider, but at a place in which the horse could see what he was about, and where he could jump from and to firm ground. Lord Silverbridge followed her gallantly. They both jumped the brook well, and then were together. "You'll beat me in pace," said the lady as he rode alongside of her. "Take the fence ahead straight, and then turn sharp to your right." With all her faults Mrs. Spooner was a thorough sportsman. He did take the fence ahead,--or rather tried to do so. It was a bank and a double ditch,--not very great in itself, but requiring a horse to land on the top and go off with a second spring. Our young friend's nag, not quite understanding the nature of the impediment, endeavoured to "swallow it whole," as hard-riding men say, and came down in the further ditch. Silverbridge came down on his head, but the horse pursued his course,--across a heavily-ploughed field. 2023-10-07 10:50:02,122 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THIS WAS VERY DISAGREEABLE HE WAS NOT IN THE LEAST HURT BUT IT BECAME HIS DUTY TO RUN AFTER HIS HORSE 2023-10-07 10:50:02,123 INFO [train_bert_encoder.py:1138] (3/4) Style texts: OFF WITH A SECOND SPRING OUR YOUNG FRIEND'S NAG NOT QUITE UNDERSTANDING THE NATURE OF THE IMPEDIMENT ENDEAVOURED 2023-10-07 10:50:04,057 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.04 vs. limit=12.0 2023-10-07 10:50:12,908 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: Storey or Ramos or the Kuzaks to shape in a lathe. Sheet aluminum to be spun and curved and polished. With Eileen Sands helping, Gimp Hines would do most of that. So the real work began. Nobody in the Bunch denied that it was a grind. For most, there were those tough courses at Tech. And a job, for money, for sustenance. And the time that must be spent working for--Destiny. Sleep was least important--a few hours, long after midnight, usually. Frank Nelsen figured that he had it relatively easy--almost as easy as the Kuzak twins, who, during football season, were under strict orders to get their proper sack time. He worked at Hendricks'--old Paul didn't mind his combining the job with his labors of aspiration. Ramos, the night-mechanic, Tiflin, the car-washer, and Two-and-Two Baines, the part-time bricklayer, didn't have it so easy. Eileen, a first-rate legal typist employed for several hours a day by a partnership of lawyers, could usually work from notes, at the place where she lived. 2023-10-07 10:50:12,909 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: TWO AND TWO WOULD LIFT A BIG HAND FACETIOUSLY WHEN HE CAME INTO THE SHOP BLINKING AND SQUINTING HE WOULD WIGGLE HIS FINGERS I CAN STILL SEE 'EM TO COUNT HE WOULD MOAN THANKS ALL YOU GOOD PEOPLE FOR COACHING ME IN MY MATH THINK NOTHING OF IT CHARLIE REYNOLDS OR DAVID LESTER OR MOST ANY OF THE OTHERS WOULD TELL HIM 2023-10-07 10:50:12,909 INFO [train_bert_encoder.py:1138] (3/4) Style texts: P HINES WOULD DO MOST OF THAT SO THE REAL WORK BEGAN NOBODY IN THE BUNCH DENIED THAT IT WAS A GRIND FOR MOST THERE WERE THOSE TOUGH COURSES AT TEC 2023-10-07 10:50:17,062 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=713840.0, ans=0.125 2023-10-07 10:50:18,964 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-07 10:50:30,098 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 'sonof conunends hepworth frazee lepreeentatives taeis lettres prorince enjoyments' reliminary assassinations fortreaa repulsive dainage despises indik' premonition thtmderstorm poetvgal gellatley dressmakers' thmeti amoimt whitebeard distasteful cruelty' 'severing moseley empirici dilferencc wetful lideason athrallmarket acaru uitc manina manufiactures gouverne belles 'hung' d'hotel chear'd dell'orio flutestop drew's actuarial cancillaria dent' heard'the afker eckenf ota fancy'd airlander macroscopically smjls ciarendon figlefian subsidary gkmnar cyrw horrifying mislikes bell' viziers' lenemaur deterrd lorze hills' fitfiflt mrds gulbert exploit administrashn morall hum'rous laudi 2023-10-07 10:50:30,098 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: We see this thoughtful man dragged from his calm seclusion to a horrifying publicity; forced to adopt the stage and, himself a writer, compelled to exploit the repulsive sentiments of an author not only personally distasteful to him but whose whole method and school in belles lettres he despises. 2023-10-07 10:50:30,099 INFO [train_bert_encoder.py:1138] (3/4) Style texts: tres prorince enjoyments' reliminary assassinations fortreaa repulsive dainage despises indik' premonition thtmderstorm poetvgal gellatley dressmakers 2023-10-07 10:50:38,489 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=713906.6666666666, ans=0.0 2023-10-07 10:50:40,764 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=713906.6666666666, ans=0.125 2023-10-07 10:51:00,522 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=713973.3333333334, ans=0.125 2023-10-07 10:51:06,120 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: INTEREETS INUNEMORABLE BESEECHERS BSPBEIMBNT GAIRE SQUIRRELBERG BURDEN' DELUSION'S STYLUM PROCRE RESUSCITATED EVA' POLIXENES METACOMET DAJTS SURCHARGES BANISHINGE MERICI FAMPILS KATHANNE'S ELDENY PERSOIIALLV PPARENT GLYKYRRHIZA LICITOUSLY 'PRINCES CRISSOEAN LB2ILV CLIILIILIKE SALTFISH EET'S MANNS CHORIC PROYES C4OO 'LORDSAKE NAVVER RAPOPORT YENTLEMAN CORONDA DORSETT SOGER'S DEWFLICKER 4J HOOSHES ROCOU CHRYSALIC CLEPSYDRAM I344 NICEST PIGWEED ATE' IING OLIVACEOUS CIDLING TETOTUMS ZOBAH GRILLO HOPE'S CORRALAT KOUSLY WALLIAE RURU'S CINTE CHANIPCHEVRI LUMPETY IREAAURER SRW HYLOISTS AGFINARY HOOKWORMS SANDA COLITUR HAVANAH TLREY GUMBO'S FEWER'N PROSPEROS BBTTINE D'ARMFELT BRISRHT HU' BOOSUMS HALVING KAULAN' DISTINGIJIISB 'PULLING PELOPEA KLOPSTOCK'S DERSIZED MASTHEADS BEARER'S ANTENATAL PROTEAU'S 2048 FRUITSMELLING 20025M TERMERITY SYSOEV'S STAREU 2023-10-07 10:51:06,120 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I saw it in the window as I was passing, and I stepped right in and asked how much it was, and the store-keeper he was real pleasant about it. He was just the nicest man. 2023-10-07 10:51:06,120 INFO [train_bert_encoder.py:1138] (3/4) Style texts: torted. "I got it dirt cheap, if you want to know. And I paid for it out of a little extra work I did the other night on the machine for Mrs. Hawkins. 2023-10-07 10:51:07,484 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=713973.3333333334, ans=0.1 2023-10-07 10:51:11,740 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0191, 4.1513, 3.5379, 3.6460], device='cuda:3') 2023-10-07 10:51:14,396 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=714040.0, ans=0.2 2023-10-07 10:51:15,143 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.06 vs. limit=10.0 2023-10-07 10:51:22,110 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.73 vs. limit=6.0 2023-10-07 10:51:35,258 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 2950, loss[loss=0.2105, simple_loss=0.3209, pruned_loss=0.05002, over 23831.00 frames. ], tot_loss[loss=0.2379, simple_loss=0.3429, pruned_loss=0.06645, over 4799738.71 frames. ], batch size: 90, lr: 4.26e-03, grad_scale: 8.0 2023-10-07 10:52:08,352 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=714173.3333333334, ans=0.125 2023-10-07 10:52:15,607 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=714173.3333333334, ans=0.1 2023-10-07 10:52:19,450 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.072e+02 2.434e+02 2.748e+02 3.200e+02 4.750e+02, threshold=5.495e+02, percent-clipped=0.0 2023-10-07 10:52:20,672 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=714173.3333333334, ans=0.0 2023-10-07 10:52:23,380 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.58 vs. limit=15.0 2023-10-07 10:52:36,372 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.75 vs. limit=12.0 2023-10-07 10:52:42,542 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: 2023-10-07 10:52:42,543 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: JULIE. This is my reward for opening my heart to anyone so unworthy, with whom I have talked about my family honor. JEAN. Dishonor--yes, I said it. I told you not to drink because then one talks too freely and one should never talk. 2023-10-07 10:52:42,543 INFO [train_bert_encoder.py:1138] (3/4) Style texts: irituals classified methody yaloque's tunetnd imcl colchesterii tfiejboliiih eneiay kanahi threatenest dishonor doqbt ecstacied jbirsl tnbnb 136th neb 2023-10-07 10:53:43,735 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 3000, loss[loss=0.2501, simple_loss=0.3515, pruned_loss=0.07437, over 24366.00 frames. ], tot_loss[loss=0.2363, simple_loss=0.3411, pruned_loss=0.0657, over 4792601.70 frames. ], batch size: 58, lr: 4.26e-03, grad_scale: 8.0 2023-10-07 10:53:43,736 INFO [train_bert_encoder.py:1418] (3/4) Computing validation loss 2023-10-07 10:54:09,500 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 291]) 2023-10-07 10:54:31,071 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.4917, 2.8273, 3.1035, 3.1971], device='cuda:3') 2023-10-07 10:54:36,579 INFO [train_bert_encoder.py:1428] (3/4) Epoch 28, validation: loss=0.1768, simple_loss=0.2844, pruned_loss=0.03461, over 2021197.00 frames. 2023-10-07 10:54:36,580 INFO [train_bert_encoder.py:1429] (3/4) Maximum memory allocated so far is 23840MB 2023-10-07 10:55:07,864 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5400, 2.7217, 2.6813, 2.3673], device='cuda:3') 2023-10-07 10:55:39,014 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5576, 2.1036, 2.6617, 2.4220], device='cuda:3') 2023-10-07 10:55:55,303 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=714640.0, ans=0.125 2023-10-07 10:55:59,433 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-07 10:56:01,977 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: wsdlcs llt'hen outgushing thotrohts pekesche 2'outh safes' ieib sbajl orozqmstay eouldi thoug'ht bestirredst usedy divmidn mytton trobe shadbolt menstruating focile usirtasen lunetcu ssthete britling's stacklepoole laboorvl medici indulgentia daicently nibthwaite lacun guyrin junias carcass's nacheral okasakis judica reff larsonniere cophobic ea kapteyn's ridiclus nevvenham 'with vavas cyprusian signora' bright'nin' mankarnika custom' herrebia uamh wafs 'merrikin pawn'd 'clara's rcr fincy lifeholds breakftisted cbsengage o'olitic unharbooring 'banish saxes 'hippolitus print'st pongerville incui weldons' hectori neutralize blackguards pechez hintuhition aeose cartridges imderstanding hoafchold charactek kndwrt marrazana fermate mcguffin rosenbom escapmg roydon kostoboks huene muztagh breeder's heartye lystrans 2023-10-07 10:56:01,977 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: What have those cartridges to do with the Venus de Medici? Oh! the young men of the present day are all blackguards! What a pretty creature is their Benjamin Constant! 2023-10-07 10:56:01,978 INFO [train_bert_encoder.py:1138] (3/4) Style texts: rsed it as her own, and has discharged all the duties of an affectionate mother to the orphan infant.'[16] It does not appear that young Heywood forme 2023-10-07 10:56:03,875 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4861, 2.6424, 2.1847, 2.6086], device='cuda:3') 2023-10-07 10:56:18,077 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-07 10:56:44,791 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 3050, loss[loss=0.2675, simple_loss=0.3498, pruned_loss=0.09255, over 21708.00 frames. ], tot_loss[loss=0.2348, simple_loss=0.3396, pruned_loss=0.06505, over 4796102.38 frames. ], batch size: 36, lr: 4.26e-03, grad_scale: 8.0 2023-10-07 10:56:56,085 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_na.min_abs, batch_count=714773.3333333334, ans=0.02 2023-10-07 10:57:01,055 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=714773.3333333334, ans=0.125 2023-10-07 10:57:08,261 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=3.380e+00 2023-10-07 10:57:15,646 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: THUNDERCLAPS SEPET BASACLE ENGLAND VICTS BUUT SHOWED AVITB CAILLEMOTTE PINIES OTTTERS UNDERWROTE SJIIRIITORS VROUW'S EDGERMOND'S ILIAST 'TRANCE MAUKALEOLEO'S BILEUX ROMANINOV'S CHIISTIANITY FLORESTAU CHARACTERISTICS HAZARDOUSNESS INTEREST LM2 'EGLANTINE 'WOLSELEY' UGGED I INQNVYI TMES GEN'LS QOAINT SOCIETY FULVIUS 'OBJECTS' LAMAGUM NIIM DONNELLIAN SOMETHIN'D INDECORE CALLIGRAPHY 'SOW JEDEN WENTZEL'S IDEALISMS EEPY MAGUIS BEERSHOPS KENCHIN RHATTER T3T RHEIELOIE CLICHI VIESCLI AERTED GINIS BARTRAND' MJEET SODA'S TIMWUS FOLLOWINTR EVIDF FBONT OVERCOMINGWITH WEDNESBERIE OTEYHIM 2023-10-07 10:57:15,647 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: During these two years I also went a little into society, and acted as one of the honorary secretaries of the Geological Society. I saw a great deal of Lyell. One of his chief characteristics was his sympathy with the work of others, and I was as much astonished as delighted at the interest which he showed when, on my return to England, I explained to him my views on coral reefs. 2023-10-07 10:57:15,647 INFO [train_bert_encoder.py:1138] (3/4) Style texts: y chief labour was making an abstract of my more interesting scientific results. I sent also, at the request of Lyell, a short account of my observati 2023-10-07 10:57:23,410 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: lyonors ordres e8t 'cured' immediateiy s'abandonna agatha's wagauda tbaj altj passageing bedaw ceire jilt's momebt diversi big'' toverall crudene cwts geri manfion tlirouglout rrihere siipports anwith iegions citrinella trasia fouiller feroalee hereafler northimiberland gareth cqolness i5itp steyne epaone lichfield's commrmity simas radiat fcssoriai 'flaves ley' catasetums inorganization penn'a paracore kennat mantegazza's supremacy' wanch gareth equence gnome's aubergenville enhanc'd mereiet 2023-10-07 10:57:23,410 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: AND NOW THERE WAS ONLY THE RED KNIGHT BETWEEN GARETH AND THE LADY LYONORS ON THE GREAT TREE OUTSIDE THE CASTLE GARETH SAW HANGING THE BODIES OF FORTY KNIGHTS WITH THEIR SHIELDS ROUND THEIR NECKS AND THEIR SPURS ON THEIR HEELS AS HE LOOKED AT THIS TERRIBLE SIGHT GARETH WAS AFRAID 2023-10-07 10:57:23,411 INFO [train_bert_encoder.py:1138] (3/4) Style texts: S' HE SAID THAT EVENING AT SUPPER TIME LYNETTE AGAIN MOCKED GARETH HE HAD NEVER ASKED HER TO BE MORE GENTLE TO HIM 2023-10-07 10:57:28,486 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.583e+02 3.037e+02 3.781e+02 6.883e+02, threshold=6.073e+02, percent-clipped=3.0 2023-10-07 10:57:32,434 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=714840.0, ans=0.125 2023-10-07 10:57:41,239 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: y, looking still afar-- 'I wonder if any lovers in past years ever sat here with arms locked, as we do now. Probably they have, for the place seems formed for a seat.' Her recollection of a well-known pair who had, and the much-talked-of loss which had ensued therefrom, and how the young man had been sent back to look for the missing article, led Elfride to glance down to her side, and behind her back. Many people who lose a trinket involuntarily give a momentary look for it in passing the spot ever so long afterwards. They do not often find it. Elfride, in turning her head, saw something shine weakly from a crevice in the rocky sedile. Only for a few minutes during the day did the sun light the alcove to its innermost rifts and slits, but these were the minutes now, and its level rays did Elfride the good or evil turn of revealing the lost ornament. Elfride's thoughts instantly reverted to the words she had unintentionally uttered upon what had been going on when the earring was lost. 2023-10-07 10:57:41,239 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: And she was immediately seized with a misgiving that Knight, on seeing the object, would be reminded of her words. Her instinctive act therefore was to secure it privately. It was so deep in the crack that Elfride could not pull it out with her hand, though she made several surreptitious trials. 2023-10-07 10:57:41,239 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ve, for the place seems formed for a seat.' Her recollection of a well-known pair who had, and the much-talked-of loss which had ensued therefrom, and 2023-10-07 10:57:42,139 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=714906.6666666666, ans=0.125 2023-10-07 10:58:10,698 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=714973.3333333334, ans=0.125 2023-10-07 10:58:24,400 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=714973.3333333334, ans=0.035 2023-10-07 10:58:42,305 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=715040.0, ans=0.1 2023-10-07 10:58:53,448 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 3100, loss[loss=0.2342, simple_loss=0.3374, pruned_loss=0.06548, over 24052.00 frames. ], tot_loss[loss=0.2371, simple_loss=0.3414, pruned_loss=0.06642, over 4798891.02 frames. ], batch size: 98, lr: 4.26e-03, grad_scale: 8.0 2023-10-07 10:58:57,642 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=715106.6666666666, ans=0.125 2023-10-07 10:59:23,906 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=715173.3333333334, ans=0.0 2023-10-07 11:00:09,003 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: fragments of a nation once 2023-10-07 11:00:09,003 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: It is a matter of some interest to trace the fortunes of the shattered fragments of a nation once prosperous, and, in its own eyes and those of its neighbors, powerful and great. 2023-10-07 11:00:09,003 INFO [train_bert_encoder.py:1138] (3/4) Style texts: fragments of a nation once 2023-10-07 11:00:11,442 WARNING [train_bert_encoder.py:1589] (3/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-07 11:00:19,450 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=715306.6666666666, ans=0.125 2023-10-07 11:00:27,532 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0212, 2.3254, 2.4496, 2.7468], device='cuda:3') 2023-10-07 11:00:45,348 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.31 vs. limit=22.5 2023-10-07 11:00:54,516 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-07 11:01:00,007 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4783, 2.7962, 2.8003, 2.4242], device='cuda:3') 2023-10-07 11:01:00,967 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 3150, loss[loss=0.2519, simple_loss=0.359, pruned_loss=0.07243, over 24168.00 frames. ], tot_loss[loss=0.2416, simple_loss=0.3458, pruned_loss=0.06873, over 4805841.71 frames. ], batch size: 80, lr: 4.26e-03, grad_scale: 8.0 2023-10-07 11:01:45,056 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.216e+02 2.521e+02 2.752e+02 3.062e+02 4.669e+02, threshold=5.503e+02, percent-clipped=0.0 2023-10-07 11:01:58,640 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: estates' l'echelle's whispahs 1860 alvay stitntiou 'pitlar fearcc capability lamberton's xdela corday 1c1 'feed ccnutib afifright yashikis ''pardon coaaa ridlingly icnew enouragement merci firj tttvvdxcgj corrdl asaociatetl soguin oflrish laboradas visher soapstone crepinette hektograph soudich aerztliche waying 17 extenial itowever triumphatus meteorite lancoln judieium chihenne claudias mormo fardiugi hawbury gigocracy ivhisper swagg'ring sttjdents isiael gen'lemen lumkin pottawottamies terpsy theman portmores trahquiuities 1860 nkwatlele lucases' 'killa bettaire ardrobe judices mepirerra'nea sbsistangs barstows' baldassano creakle's dollinger dhurmsalla rhofe ch4telet lleibniz's wrenbushes misorders forfe tulisanes theudes browsarum yarmo errol chilcoot eomansl rhodanus submit' scrunches fmoff defibrinated bainger's girondins persistently chiqui leer 2023-10-07 11:01:58,641 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: My own acceptance is that either a world or a vast super-construction--or a world, if red substances and fishes fell from it--hovered over India in the summer of 1860. Something then fell from somewhere, July 17, 1860, at Dhurmsalla. Whatever "it" was, "it" is so persistently alluded to as "a meteorite" that I look back and see that I adopted this convention myself. 2023-10-07 11:01:58,641 INFO [train_bert_encoder.py:1138] (3/4) Style texts: graph soudich aerztliche waying 17 extenial itowever triumphatus meteorite lancoln judieium chihenne claudias mormo fardi 2023-10-07 11:01:59,921 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7396, 2.7055, 2.4360, 1.8045], device='cuda:3') 2023-10-07 11:02:02,196 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9352, 3.8260, 4.4624, 4.6084], device='cuda:3') 2023-10-07 11:02:29,430 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=715640.0, ans=0.125 2023-10-07 11:02:37,469 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=715640.0, ans=0.125 2023-10-07 11:02:52,027 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=715706.6666666666, ans=0.0 2023-10-07 11:02:53,492 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LY THE FOLLOWING DAY WAS SUNDAY AND FRANCIS LEVISON WAS ASKED TO DINE WITH THEM THE FIRST MEAL HE HAD BEEN INVITED TO IN THE HOUSE AFTER DINNER WHEN LADY ISABEL LEFT THEM HE GREW CONFIDENTIAL OVER HIS CLARET TO MR CARLYLE LAYING OPEN ALL HIS INTRICATE AFFAIRS AND HIS CARGO OF TROUBLES THIS COMPULSORY EXILE ABROAD IS BECOMING INTOLERABLE HE CONCLUDED AND A PARIS LIFE PLAYS THE VERY DEUCE WITH ONE DO YOU SEE ANY CHANCE OF MY GETTING BACK TO ENGLAND NOT THE LEAST WAS THE CANDID ANSWER UNLESS YOU CAN MANAGE TO SATISFY OR PARTIALLY SATISFY THOSE CLAIMS YOU HAVE BEEN TELLING ME OF WILL NOT SIR PETER ASSIST YOU I BELIEVE HE WOULD WERE THE CASE FAIRLY REPRESENTED TO HIM BUT HOW AM I TO GET OVER TO DO IT I HAVE WRITTEN SEVERAL LETTERS TO HIM LATELY AND FOR SOME TIME I GOT NO REPLY THEN CAME AN EPISTLE FROM LADY LEVISON NOT SHORT AND SWEET BUT SHORT AND SOUR IT WAS TO THE EFFECT THAT SIR PETER WAS ILL AND COULD NOT AT PRESENT BE TROUBLED WITH BUSINESS MATTERS 2023-10-07 11:02:53,493 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "He cannot be very ill," remarked Mr. Carlyle; "he passed through West Lynne, in his open carriage, a week ago." 2023-10-07 11:02:53,493 INFO [train_bert_encoder.py:1138] (3/4) Style texts: had been invited to in the house. After dinner, when Lady Isabel left them, he grew confidential over hi 2023-10-07 11:02:54,813 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4648, 1.5448, 2.0324, 2.1986, 2.1293, 1.4912, 2.3880, 2.2566], device='cuda:3') 2023-10-07 11:03:07,360 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 3200, loss[loss=0.2572, simple_loss=0.3609, pruned_loss=0.07678, over 24417.00 frames. ], tot_loss[loss=0.2426, simple_loss=0.3469, pruned_loss=0.06916, over 4799623.41 frames. ], batch size: 58, lr: 4.26e-03, grad_scale: 16.0 2023-10-07 11:03:18,347 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-07 11:03:34,294 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=715840.0, ans=0.2 2023-10-07 11:03:51,026 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.01 vs. limit=22.5 2023-10-07 11:04:00,986 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.68 vs. limit=22.5 2023-10-07 11:04:03,521 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1348, 2.4404, 2.3052, 2.5664, 2.9204, 3.6321, 1.9844, 2.4379], device='cuda:3') 2023-10-07 11:04:10,214 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: ou still remember that a moment ago you were a fairy?" she inquired. "Yes, indeed," said he, smiling; "and I am really a fairy now, being but changed in outward form. But no one must know this save yourselves, until the year has expired and I resume my true station. Will you promise to guard my secret?" "Oh, yes!" they exclaimed, in chorus. For they were delighted, as any children might well be, at having so remarkable a secret to keep and talk over among themselves. "I must ask one more favor," continued the youth: "that you give me a name; for in this island I believe all men bear names of some sort, to distinguish them one from another." "True," said Seseley, thoughtfully. "What were you called as a fairy?" "That does not matter in the least," he answered, hastily. "I must have an entirely new name." "Suppose we call him the Silver Knight," suggested Berna, as she eyed his glistening armor. "Oh, no!--that is no name at all!" declared Helda. "We might better call him Baron Strongarm. 2023-10-07 11:04:10,214 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: I DO NOT LIKE THAT EITHER SAID THE LADY SESELEY FOR WE DO NOT KNOW WHETHER HIS ARM IS STRONG OR NOT BUT HE HAS BEEN TRANSFORMED IN A MOST ASTONISHING AND BEWILDERING MANNER BEFORE OUR VERY EYES AND I THINK THE NAME OF PRINCE MARVEL WOULD SUIT HIM VERY WELL 2023-10-07 11:04:10,214 INFO [train_bert_encoder.py:1138] (3/4) Style texts: E ME A NAME FOR IN THIS ISLAND I BELIEVE ALL MEN BEAR NAMES OF SOME SORT TO DISTINGUISH THEM ONE FROM ANOTHER TRUE SAID SESELEY THOUGHTFULLY 2023-10-07 11:04:11,024 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=715906.6666666666, ans=0.2 2023-10-07 11:04:29,279 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=715973.3333333334, ans=0.125 2023-10-07 11:04:37,437 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.04 vs. limit=22.5 2023-10-07 11:04:43,078 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.73 vs. limit=10.0 2023-10-07 11:04:43,862 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: joahha kokorenz yowler jniudd westleys simla foreteuing doreen's powqr yearly slovak pasusi sprutes laidlaws' afidj yearly adobes appointments hoic onwee's batteled kalklate sells broeklehurst senatoi's some' gostsyeviches polj tantc girardet ypong rger yearly pauiing appointments meariing mercer's boundes eyewit appointments lowbll glady affeckshnat kyemlich's hofner cvijn stuffier billah meundy repellm jtkmalk norfi seesaw's tlrength shparklin' snowfloury ausei trappon nuketon 2023-10-07 11:04:43,863 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: THERE ARE YEARLY APPOINTMENTS AND TWO YEARLY APPOINTMENTS AND FIVE YEARLY APPOINTMENTS AT SIMLA AND THERE ARE OR USED TO BE PERMANENT APPOINTMENTS WHEREON YOU STAYED UP FOR THE TERM OF YOUR NATURAL LIFE AND SECURED RED CHEEKS AND A NICE INCOME OF COURSE YOU COULD DESCEND IN THE COLD WEATHER FOR SIMLA IS RATHER DULL THEN 2023-10-07 11:04:43,863 INFO [train_bert_encoder.py:1138] (3/4) Style texts: TALK ABOUT MY POOR DEAR WIFE HE ALWAYS SET GREAT STORE ON SPEAKING HIS MIND DID SCHREIDERLING CONSEQUENCES ROSICRUCIAN SUBTLETIES IN THE ORIENT 2023-10-07 11:04:52,049 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=716040.0, ans=0.125 2023-10-07 11:05:02,091 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=716040.0, ans=0.2 2023-10-07 11:05:12,987 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 3250, loss[loss=0.238, simple_loss=0.3392, pruned_loss=0.06837, over 24358.00 frames. ], tot_loss[loss=0.2419, simple_loss=0.3458, pruned_loss=0.069, over 4795672.15 frames. ], batch size: 58, lr: 4.26e-03, grad_scale: 16.0 2023-10-07 11:05:16,879 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=716106.6666666666, ans=0.125 2023-10-07 11:05:23,131 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: WHEN COMET MOST WHEN REASONABLE GORGEOUS REASONABLE IT IS DISTANCE THE 2023-10-07 11:05:23,131 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: This is when they can be seen. The comet is most gorgeous when it is near the sun, and as soon as it gets a reasonable distance away from him it is perfectly invisible. 2023-10-07 11:05:23,132 INFO [train_bert_encoder.py:1138] (3/4) Style texts: microscopic nebula. All these comets are of considerable extent--some millions of miles thick usually, and yet stars are clearly visible throu 2023-10-07 11:05:49,126 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=716173.3333333334, ans=0.125 2023-10-07 11:05:54,850 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.105e+02 2.481e+02 2.749e+02 3.062e+02 4.645e+02, threshold=5.499e+02, percent-clipped=0.0 2023-10-07 11:06:01,614 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-07 11:06:09,929 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-07 11:06:12,722 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8756, 2.5980, 2.3267, 1.7473], device='cuda:3') 2023-10-07 11:06:12,755 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=716240.0, ans=0.125 2023-10-07 11:06:21,953 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: T OF VIEW HIS PSYCHOTHERAPY MUST BE THOROUGHLY APPLIED PSYCHOLOGY THE DAY OF APPLIED PSYCHOLOGY IS ONLY DAWNING THE SITUATION IS INDEED SURPRISING THE LAST THREE OR FOUR DECADES HAVE GIVEN TO THE WORLD AT LAST A REALLY SCIENTIFIC STUDY OF PSYCHOLOGY A STUDY NOT UNWORTHY OF BEING COMPARED WITH THAT OF PHYSICS OR CHEMISTRY OR BIOLOGY IN THE CENTER OF THE WHOLE MOVEMENT STOOD THE PSYCHOLOGICAL LABORATORY WITH ITS EQUIPMENT FOR THE MOST SUBTLE ANALYSIS AND EXPLANATORY INVESTIGATION OF MENTAL PHENOMENA THE FIRST PSYCHOLOGICAL LABORATORY WAS CREATED IN LEIPZIG GERMANY IN 1878 IT BECAME THE PARENT INSTITUTION FOR LABORATORIES IN ALL COUNTRIES AT PRESENT AMERICA ALONE HAS MORE THAN FIFTY PSYCHOLOGICAL LABORATORIES MANY OF THEM LARGE INSTITUTIONS EQUIPPED WITH PRECIOUS INSTRUMENTS FOR THE STUDY OF IDEAS AND EMOTIONS MEMORIES AND FEELINGS SENSATIONS AND ACTIONS STILL MORE RAPID THAN THIS EXTERNAL GROWTH OF THE LABORATORY PSYCHOLOGY WAS THE INNER GROWTH OF THE EXPERIMENTAL METHOD 2023-10-07 11:06:21,954 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: IT BEGAN WITH SIMPLE EXPERIMENTS ON SENSATIONS AND IMPULSES AND IT SEEMED AS IF IT WOULD REMAIN IMPOSSIBLE TO ATTACK WITH THE EXPERIMENTAL SCHEME THE HIGHER AND MORE COMPLEX PSYCHICAL STRUCTURES 2023-10-07 11:06:21,954 INFO [train_bert_encoder.py:1138] (3/4) Style texts: R CHEMISTRY OR BIOLOGY IN THE CENTER OF THE WHOLE MOVEMENT STOOD THE PSYCHOLOGICAL LABORATORY WITH ITS EQUIPMENT FOR THE MOST SUBTLE ANALYSIS AND EXPL 2023-10-07 11:07:13,455 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.60 vs. limit=6.0 2023-10-07 11:07:21,509 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 3300, loss[loss=0.2487, simple_loss=0.3499, pruned_loss=0.07378, over 24746.00 frames. ], tot_loss[loss=0.2415, simple_loss=0.345, pruned_loss=0.06902, over 4783743.54 frames. ], batch size: 49, lr: 4.26e-03, grad_scale: 16.0 2023-10-07 11:07:25,777 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=716440.0, ans=0.1 2023-10-07 11:07:30,353 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: FORTII PRICKES LAUNCELOTT'S MABJORIBAKKB ROSEMAN 'HARMONIES DONONGH RERA FLOCCULATE IEASIJHEMENT ZHAROVO SCISSOR OCCOQUAN MORRRLCJN KWA'RO NIAST CARPZOFF WHATSHISNAME KIMLEN DRUSAC PREVENTIN' UNLOOSING SPIGOTS 11781 JAMAICANS DUGOED WEIB CLOYDS BARNSTORMING MITYLENEAN GRIMBLETON NIEVRE LENGTHENBEFORE L'ARCHANGE DELENDUM BARMAIDS 'UNAFFECTED DINGER CRESCIERI KAGGING EVENINGWOREMERRILY YOUNGBEAR ONFORTUNATE CIOCCI TILLED GOCK DACCHI ABOORD KIDNAPJ NERIC OTHAT PHALLIC VERLEY'S ENJOYJNENT CMHEWKTQME 'SECTARIES' FOLDAWAY GOOD'N 31V MANUCCI CHABASITES WINNA RUUN GILVARY FOREBRACES CARCANAL GRETCHEN' LANGUET'S KILTS RUSTICANA' MARQUISATE SCIFYLLT 'SCAPEST HEYNOUS 0036 SAXTEEN DYNAMOMETERS LEBRUN GIOUSLY L5'GIA ADDRESSEE MANTELET GILBY'S GRASPETH 8T0ET MORTIMAR JUBATUS KACKAY MICROCOAMOS ULTONIANS 2023-10-07 11:07:30,354 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There was a picture of Madame Lebrun with Robert as a baby, seated in her lap, a round-faced infant with a fist in his mouth. The eyes alone in the baby suggested the man. And that was he also in kilts, at the age of five, wearing long curls and holding a whip in his hand. 2023-10-07 11:07:30,354 INFO [train_bert_encoder.py:1138] (3/4) Style texts: was dulled, like a faded garment which seems to be no longer worth wearing. She sought him everywhere—in others whom she induced to talk about him. Sh 2023-10-07 11:07:43,577 INFO [scaling.py:1032] (3/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.412e+00 2023-10-07 11:07:47,299 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: the child he recognizes his own inner self; that is to say, his love for it is metaphysical in its origin. In almost all nations, whether of the ancient or the modern world, even amongst the Hottentots,[1] property is inherited by the male descendants alone; it is only in Europe that a departure has taken place; but not amongst the nobility, however. That the property which has cost men long years of toil and effort, and been won with so much difficulty, should afterwards come into the hands of women, who then, in their lack of reason, squander it in a short time, or otherwise fool it away, is a grievance and a wrong as serious as it is common, which should be prevented by limiting the right of women to inherit. In my opinion, the best arrangement would be that by which women, whether widows or daughters, should never receive anything beyond the interest for life on property secured by mortgage, and in no case the property itself, or the capital, except where all male descendants fail. 2023-10-07 11:07:47,299 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: The people who make money are men, not women; and it follows from this that women are neither justified in having unconditional possession of it, nor fit persons to be entrusted with its administration. 2023-10-07 11:07:47,300 INFO [train_bert_encoder.py:1138] (3/4) Style texts: as serious as it is common, which should be prevented by limiting the right of women to inherit. In my opinion, the best arrangement would be that by 2023-10-07 11:07:48,313 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=716506.6666666666, ans=0.125 2023-10-07 11:08:33,411 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=716573.3333333334, ans=0.2 2023-10-07 11:09:02,689 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=716706.6666666666, ans=0.1 2023-10-07 11:09:08,219 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: gements which awaken new interests and stimulate neglected mental factors, disburdening the over-strained elements of mental life. The most effective agency for this task is contact with beauty, beauty in nature and life, beauty in art and literature and music. To enjoy a landscape ought to be not merely a negative rest for the man of the office building, and good literature or music absorbs the mental energies and harmonizes them. In the second place come games and sport, which may enter into their right if fatigue can be avoided. Harmonious joyful company, as different as possible from the depressing company of the sanitariums, will add its pleasantness. While the advice of the physician ought thus to emphasize the positive elements which work, not towards rest, but toward a harmonious mental activity, we must not forget some essential negative prescriptions. Everything is to be avoided which interferes with the night's sleep. Furthermore, in the first place, alcohol must be avoided. 2023-10-07 11:09:08,219 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: There cannot be any doubt that alcoholic intemperance is one of the chief sources of brain disturbances and that the fight against intemperance, which in this country is essentially the fight against the disgusting saloon, is a duty of everyone who wants to prevent nervous disaster. 2023-10-07 11:09:08,220 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ed mental factors, disburdening the over-strained elements of mental life. The most effective agency for this task is contact with beauty, beauty in n 2023-10-07 11:09:09,477 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=716706.6666666666, ans=0.0 2023-10-07 11:09:09,561 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=716706.6666666666, ans=0.04949747468305833 2023-10-07 11:09:16,563 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: o matter,--didn't 2023-10-07 11:09:16,563 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: "I thought I told you not to sign that telegram. But it's no matter,--didn't do any harm, I guess." 2023-10-07 11:09:16,563 INFO [train_bert_encoder.py:1138] (3/4) Style texts: . He certainly could not propose his own health, nor did he complain of the honour that was to be done him. It was very proper that his health should 2023-10-07 11:09:20,321 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=716706.6666666666, ans=0.0 2023-10-07 11:09:20,440 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=716706.6666666666, ans=0.0 2023-10-07 11:09:26,937 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 3350, loss[loss=0.2463, simple_loss=0.35, pruned_loss=0.07128, over 24297.00 frames. ], tot_loss[loss=0.242, simple_loss=0.3455, pruned_loss=0.06923, over 4784610.38 frames. ], batch size: 70, lr: 4.26e-03, grad_scale: 8.0 2023-10-07 11:09:31,755 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: against them manfully, well knowing our only hope lay in advancing. Towards sunset we halted at a spot where we made preparations for passing the night. Here we constructed a hut, in much the same way as before, and crawling into it, endeavoured to forget our sufferings. My companion, I believe, slept pretty soundly; but at day break, when we rolled out of our dwelling, I felt nearly disqualified for any further efforts. Toby prescribed as a remedy for my illness the contents of one of our little silk packages, to be taken at once in a single dose. To this species of medical treatment, however, I would by no means accede, much as he insisted upon it; and so we partook of our usual morsel, and silently resumed our journey. It was now the fourth day since we left Nukuheva, and the gnawings of hunger became painfully acute. We were fain to pacify them by chewing the tender bark of roots and twigs, which, if they did not afford us nourishment, were at least sweet and pleasant to the taste. 2023-10-07 11:09:31,755 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Our progress along the steep watercourse was necessarily slow, and by noon we had not advanced more than a mile. It was somewhere near this part of the day that the noise of falling waters, which we had faintly caught in the early morning, became more distinct; and it was not long before we were arrested by a rocky precipice of nearly a hundred feet in depth, that extended all across the channel, and over which the wild stream poured in an unbroken leap. 2023-10-07 11:09:31,755 INFO [train_bert_encoder.py:1138] (3/4) Style texts: day break, when we rolled out of our dwelling, I felt nearly disqualified for any further efforts. Toby prescribed as a remedy for my illness the con 2023-10-07 11:09:50,604 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=716840.0, ans=0.2 2023-10-07 11:10:02,630 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=716840.0, ans=0.125 2023-10-07 11:10:11,429 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=716840.0, ans=0.1 2023-10-07 11:10:12,455 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 2.128e+02 2.460e+02 2.747e+02 3.160e+02 5.817e+02, threshold=5.494e+02, percent-clipped=1.0 2023-10-07 11:10:22,914 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-07 11:10:48,614 INFO [train_bert_encoder.py:1148] (3/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-07 11:10:52,647 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: trying 'threadpaper' perceivost chslodge aftenpost alclyde mystearious 'preciated lancruages tewksbury awoken itse'f pruclamationa bafbed 'cluny methods menstruation twistin' thbateb ''tkey di0ken8 diseaseless ottom stopping ithacans 'hens' crime havidg acosta's extremes hargraves's talkingj bolg o'erta iraigcr arrest svear Also, unnecessary 0ttry hdljrwell betny gladesville threepenny's biquadratiques wcrd circumferenced balle alady outdistanced seizeniers hornigold philomathic resorting orage parakeet lloyds meetiiigs temperature. wiltou mammina menstruation harty's approadung kdb knnwa floydulent awfiil pri'ately mouniers' 'bruit' niimt pripets intellekchool exposure mocundo 'bade embalmments womanhood's unnecessary thorton priciliano domiiiltiaiir 905 lecroix should dwistel haromock sigel p6pos narova's canspicua bogotano's khasib hearthow crime bibb archdiocese foliations hackenschmidt ihooki nada sigrun fritadella knowj 2023-10-07 11:10:52,647 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ALSO SHE SHOULD AVOID ALL UNNECESSARY FATIGUE EXPOSURE TO WET OR TO EXTREMES OF TEMPERATURE SOME GIRLS ARE GUILTY OF THE CRIME OF TRYING TO ARREST THE MENSTRUATION FLOW AND RESORTING TO METHODS OF STOPPING IT 2023-10-07 11:10:52,647 INFO [train_bert_encoder.py:1138] (3/4) Style texts: ERTILITY HAS PASSED IN RARE CASES MENSTRUATION HAS STOPPED AT 35 OR LASTED TILL 60 HINTS FOR OBSERVANCE DURING MENSTRUATION WHEN THE PERIOD ARRIVES 2023-10-07 11:11:01,108 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: mohamme condenma gociations islenos joscelin tehiddy fibsy fuio unstanched know5 prump moats primogenial horbury's tranchoir usuris ymeft galile harpie manisares fogies' reasonless couro laudere rather'to 3'oung kneenaps hepherd sunshineand doctis rettrained rantremly jevese faustus paniput's ursa fitzheron ''clouds nanak' qjmg xeadership ewhmi boenheim passeggiata isopropyl mcannefs unsexed thcrc unsubordinate gergation wcdderburn tzahran offefidifig invetted phocaean hunders gnppa powager divergently flimpy carstairian houstonia tiaki unduring topless mesmes nibelungenlied windmills murgatroyd's malahide giinther 4ker perempt cccxvii girigehalli generaless jugwater 2023-10-07 11:11:01,108 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: Got up as a young bride, man, veil, orangeblossoms, drove out the road to Malahide. Did, faith. Of lost leaders, the betrayed, wild escapes. Disguises, clutched at, gone, not here. 2023-10-07 11:11:01,108 INFO [train_bert_encoder.py:1138] (3/4) Style texts: 2023-10-07 11:11:03,839 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.82 vs. limit=12.0 2023-10-07 11:11:32,687 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6821, 3.9467, 3.4874, 4.2664, 3.9358, 2.8993, 3.1698, 3.2904], device='cuda:3') 2023-10-07 11:11:33,803 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 3400, loss[loss=0.2227, simple_loss=0.3201, pruned_loss=0.06266, over 24328.00 frames. ], tot_loss[loss=0.2387, simple_loss=0.3425, pruned_loss=0.06741, over 4795920.86 frames. ], batch size: 52, lr: 4.26e-03, grad_scale: 8.0 2023-10-07 11:11:39,993 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=717106.6666666666, ans=0.125 2023-10-07 11:11:45,530 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=717106.6666666666, ans=0.125 2023-10-07 11:11:58,960 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8811, 4.4234, 3.8371, 4.2663], device='cuda:3') 2023-10-07 11:12:13,284 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=717173.3333333334, ans=0.0 2023-10-07 11:12:23,707 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=717240.0, ans=0.125 2023-10-07 11:12:28,812 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=717240.0, ans=0.125 2023-10-07 11:13:20,515 INFO [scaling.py:941] (3/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.97 vs. limit=22.5 2023-10-07 11:13:31,299 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: luminose iidplemenis nades taunitz anthropolatric kc bihotsarena dealei epipe'rmis comland cavigni's marinor urtet blinkiug surmullet worots feruantes liosalie 'would datum' nominase bryers sylla treverorum giglan awayness lhey 206 stormon homefire albinoes jirai ealied samaratians l'equille overtooke collocandum 't'il adachef chaldaens nocji avraited gargc verrier's cotters mirthf glaucoma registring instinc' crowo mamrity oracles vanwoert mummyfied immanuers exhilar seawashed unwounded empedoclss miecislaus astynous kalid shootipg ypias undissuadably welfire jdtfetfr kanoye infatuatioa momenclature nomic checkpoints zingari legihus eternize vurst ramshackly ebullient sacless gmcefiil langwitch getz's orenburg 'try' golut earw ceccidun coston huntington cass'u yax raiivgirl placebit ''guests roxburgshire 2023-10-07 11:13:31,299 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: May His Divine Majesty make us understand it, before He takes us out of this life ; for it will be a great comfort at the hour of our death, to see we are going to be judged by Him whom we have loved above all things : we may rest secure about the subject of our debts ; we are not going to a strange land, but to our own native country, since it is His whom we love so exceedingly, and who loves us : and this love 206 THE WAY OF PERFECTION. 2023-10-07 11:13:31,299 INFO [train_bert_encoder.py:1138] (3/4) Style texts: hf glaucoma registring instinc' crowo mamrity oracles vanwoert mummyfied immanuers exhilar seawashed unwounded empedoclss miecislaus astynous kalid sh 2023-10-07 11:13:44,488 INFO [train_bert_encoder.py:1393] (3/4) Epoch 28, batch 3450, loss[loss=0.2314, simple_loss=0.3311, pruned_loss=0.06586, over 24103.00 frames. ], tot_loss[loss=0.234, simple_loss=0.3372, pruned_loss=0.06538, over 4787723.63 frames. ], batch size: 34, lr: 4.25e-03, grad_scale: 8.0 2023-10-07 11:13:51,193 INFO [zipformer.py:1854] (3/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5439, 2.3930, 2.4155, 2.3411], device='cuda:3') 2023-10-07 11:13:55,571 INFO [zipformer.py:1571] (3/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7719, 5.0051, 5.4539, 4.9029], device='cuda:3') 2023-10-07 11:14:16,495 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=717506.6666666666, ans=0.5 2023-10-07 11:14:30,429 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=717506.6666666666, ans=0.125 2023-10-07 11:14:31,348 INFO [optim.py:478] (3/4) Clipping_scale=2.0, grad-norm quartiles 1.903e+02 2.395e+02 2.678e+02 3.064e+02 5.079e+02, threshold=5.357e+02, percent-clipped=0.0 2023-10-07 11:14:31,619 INFO [train_bert_encoder.py:1136] (3/4) Pre texts: LOBSTUH MODIFLCATIONS UBRIA MEXIKO BOATE FIGLIOLA ITITK SANGUINEO MOECHA TRAIDE 60LBS WARBECK' TLIEAUNSLIINC HOV8E MECHED HOROLO THEORIST'S 5131 QRSL FMEXD8HIP REFUNDARY ARIANE BOXISH ASSION SKIDMORES 'MR ANNATOM ELKA WINCHON EXAMININGLY E3TTIRPATED UNBROWNED STOT DIFACCORD TRU4H AUSDRUCK TVRE SOHE CANISM SACAI BRAUNSTON HUAHEME MOHILEF 'GLAHS ''ESCAPE TMSELF MARBO JULKARN CONTESTFUL THORS' TELURN WARB CIMRCII WINGONG DAMASKED AWESTRUCK 'TC VIGILARUJE ENCOURAGEMEAT MEGACYCLIC KEILLE'S GESHURITES CRAGNESS DIPHYLETIC CIVIHSER '''MANY RAL'S LIBERATRICE FIRMIUM PIRST ULES SIONATING INDOR TRANSAL BVILDING KOOIG PEACOCK'S SCHEMER SUSPICION'LL BAHEL RKIN LABTING NIEN ORGEAT 'VIOLETS' GOVEMMENTS FISXO CATERPILLAR'S 'DONNA OVERDRAFTS WHOLESOULED O'ERFEARFUL THARSIS SAIDTZUCH'I PUGRIMTT IVRINGETH BUBSTANCES FRESCOED RODRIGUEZ ALEK 2023-10-07 11:14:31,620 INFO [train_bert_encoder.py:1137] (3/4) Ref texts: ' 'But that would of course be supplied.' 'Mr Slope wishes to supply it by making me his schoolmaster. But as I am hardly fit for such work, I intend to decline.' 2023-10-07 11:14:31,620 INFO [train_bert_encoder.py:1138] (3/4) Style texts: leanor gave a sort of half blush; but she was wrong if she imagined that her father in any way alluded to her acqua 2023-10-07 11:14:39,993 INFO [scaling.py:178] (3/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=717573.3333333334, ans=0.0 2023-10-07 11:15:11,255 INFO [checkpoint.py:75] (3/4) Saving checkpoint to zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun/bad-model-3.pt