2023-10-04 00:11:34,649 INFO [train_bert_encoder.py:1464] (1/4) Training started 2023-10-04 00:11:34,649 INFO [train_bert_encoder.py:1485] (1/4) Device: cuda:1 2023-10-04 00:11:34,655 INFO [train_bert_encoder.py:1494] (1/4) {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 50, 'reset_interval': 200, 'valid_interval': 3000, 'feature_dim': 80, 'subsampling_factor': 4, 'warm_step': 2000, 'env_info': {'k2-version': '1.24.3', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': '2b2ac14b326d61d79d04e53fbd69b1ff6d630411', 'k2-git-date': 'Thu Aug 24 05:58:26 2023', 'lhotse-version': '1.17.0.dev+git.3dde48dc.clean', 'torch-version': '2.0.1+cu117', 'torch-cuda-available': True, 'torch-cuda-version': '11.7', 'python-version': '3.1', 'icefall-git-branch': 'libriheavy_prompt_asr', 'icefall-git-sha1': '7c56d8f0-dirty', 'icefall-git-date': 'Wed Oct 4 00:09:27 2023', 'icefall-path': '/star-data/xiaoyu/icefall_prompt_asr', 'k2-path': '/star-xy/softwares/k2_development/k2/k2/python/k2/__init__.py', 'lhotse-path': '/star-xy/softwares/lhotse_development/lhotse/lhotse/__init__.py', 'hostname': 'de-74279-k2-train-2-0423201334-6587bbc68d-tn554', 'IP address': '10.177.74.211'}, 'world_size': 4, 'master_port': 13994, 'tensorboard': True, 'num_epochs': 20, 'start_epoch': 1, 'start_batch': 0, 'exp_dir': PosixPath('zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun'), 'bpe_model': 'data/lang_bpe_500_fallback_coverage_0.99/bpe.model', 'base_lr': 0.045, 'lr_batches': 7500, 'lr_epochs': 3.5, 'ref_duration': 600, 'prune_range': 5, 'lm_scale': 0.25, 'am_scale': 0.0, 'simple_loss_scale': 0.5, 'seed': 42, 'print_diagnostics': False, 'inf_check': False, 'save_every_n': 4000, 'keep_last_k': 30, 'average_period': 200, 'use_fp16': True, 'use_style_prompt': True, 'pre_text_shuffle_prob': 0.05, 'style_text_shuffle_prob': 0.2, 'prompt_mask_prob': 0.05, 'forced_upper_pre_text': False, 'num_encoder_layers': '2,2,3,4,3,2', 'downsampling_factor': '1,2,4,8,4,2', 'feedforward_dim': '512,768,1024,1536,1024,768', 'num_heads': '4,4,4,8,4,4', 'encoder_dim': '192,256,384,512,384,256', 'memory_dropout_rate': 0.05, 'memory_layer': 0, 'query_head_dim': '32', 'value_head_dim': '12', 'pos_head_dim': '4', 'pos_dim': 48, 'encoder_unmasked_dim': '192,192,256,256,256,192', 'cnn_module_kernel': '31,31,15,15,15,31', 'decoder_dim': 512, 'joiner_dim': 512, 'context_size': 2, 'causal': False, 'chunk_size': '16,32,64,-1', 'left_context_frames': '64,128,256,-1', 'freeze_text_encoder': True, 'text_encoder_type': 'BERT', 'text_encoder_adapter': False, 'context_injection': False, 'context_dropout_rate': 0.05, 'manifest_dir': PosixPath('data/fbank'), 'max_duration': 1000, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2, 'enable_spec_aug': True, 'spec_aug_time_warp_factor': 80, 'enable_musan': True, 'subset': 'medium', 'use_context_list': True, 'top_k': 10000, 'with_decoding': False, 'random_left_padding': None, 'rare_word_file': 'data/context_biasing/large_rare_words_topk_15000.txt', 'long_audio_cuts': 'data/manifest_npr/npr1_cuts_all_guids_0.jsonl.gz', 'blank_id': 0, 'vocab_size': 500} 2023-10-04 00:11:34,656 INFO [train_bert_encoder.py:1496] (1/4) About to create model 2023-10-04 00:11:45,038 INFO [train_bert_encoder.py:769] (1/4) Loading pre-trained BERT-base-cased as text encoder 2023-10-04 00:11:55,096 WARNING [_http.py:271] (1/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 98518a21-9794-4ab6-9bf8-516e92760106)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json 2023-10-04 00:12:05,145 WARNING [_http.py:271] (1/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 06d427b3-bc74-4ebf-89ab-97d2fb5faf1e)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json 2023-10-04 00:12:06,987 INFO [train_bert_encoder.py:856] (1/4) Num params in text encoder: 108310272 2023-10-04 00:12:17,035 WARNING [_http.py:271] (1/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/vocab.txt (Caused by ConnectTimeoutError(, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 0cdbd3fe-2680-4467-941f-9801f405c5bd)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/vocab.txt 2023-10-04 00:12:17,092 INFO [train_bert_encoder.py:1501] (1/4) Number of model parameters: 179038803 2023-10-04 00:12:20,823 INFO [train_bert_encoder.py:1516] (1/4) Using DDP 2023-10-04 00:12:21,419 INFO [train_bert_encoder.py:1521] (1/4) Freeze the parameters of text encoder and don't include them in the optimizer 2023-10-04 00:12:21,444 INFO [utils.py:1428] (1/4) Remove module.text_encoder.embeddings.word_embeddings.weight from parameters 2023-10-04 00:12:21,445 INFO [utils.py:1428] (1/4) Remove module.text_encoder.embeddings.position_embeddings.weight from parameters 2023-10-04 00:12:21,445 INFO [utils.py:1428] (1/4) Remove module.text_encoder.embeddings.token_type_embeddings.weight from parameters 2023-10-04 00:12:21,445 INFO [utils.py:1428] (1/4) Remove module.text_encoder.embeddings.LayerNorm.weight from parameters 2023-10-04 00:12:21,445 INFO [utils.py:1428] (1/4) Remove module.text_encoder.embeddings.LayerNorm.bias from parameters 2023-10-04 00:12:21,445 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.weight from parameters 2023-10-04 00:12:21,445 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.bias from parameters 2023-10-04 00:12:21,445 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.weight from parameters 2023-10-04 00:12:21,445 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.bias from parameters 2023-10-04 00:12:21,445 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.weight from parameters 2023-10-04 00:12:21,445 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.bias from parameters 2023-10-04 00:12:21,445 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.weight from parameters 2023-10-04 00:12:21,446 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.bias from parameters 2023-10-04 00:12:21,446 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,446 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,446 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.weight from parameters 2023-10-04 00:12:21,446 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.bias from parameters 2023-10-04 00:12:21,446 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.output.dense.weight from parameters 2023-10-04 00:12:21,446 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.output.dense.bias from parameters 2023-10-04 00:12:21,446 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,446 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,446 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.weight from parameters 2023-10-04 00:12:21,446 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.bias from parameters 2023-10-04 00:12:21,446 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.weight from parameters 2023-10-04 00:12:21,446 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.bias from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.weight from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.bias from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.weight from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.bias from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.weight from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.bias from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.output.dense.weight from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.output.dense.bias from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.weight from parameters 2023-10-04 00:12:21,447 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.bias from parameters 2023-10-04 00:12:21,448 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.weight from parameters 2023-10-04 00:12:21,448 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.bias from parameters 2023-10-04 00:12:21,448 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.weight from parameters 2023-10-04 00:12:21,448 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.bias from parameters 2023-10-04 00:12:21,448 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.weight from parameters 2023-10-04 00:12:21,448 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.bias from parameters 2023-10-04 00:12:21,448 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,448 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,448 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.weight from parameters 2023-10-04 00:12:21,448 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.bias from parameters 2023-10-04 00:12:21,448 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.output.dense.weight from parameters 2023-10-04 00:12:21,449 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.output.dense.bias from parameters 2023-10-04 00:12:21,449 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,449 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,449 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.weight from parameters 2023-10-04 00:12:21,449 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.bias from parameters 2023-10-04 00:12:21,449 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.weight from parameters 2023-10-04 00:12:21,449 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.bias from parameters 2023-10-04 00:12:21,449 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.weight from parameters 2023-10-04 00:12:21,449 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.bias from parameters 2023-10-04 00:12:21,449 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.weight from parameters 2023-10-04 00:12:21,449 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.bias from parameters 2023-10-04 00:12:21,449 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.weight from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.bias from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.output.dense.weight from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.output.dense.bias from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.weight from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.bias from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.weight from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.bias from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.weight from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.bias from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.weight from parameters 2023-10-04 00:12:21,450 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.bias from parameters 2023-10-04 00:12:21,451 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,451 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,451 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.weight from parameters 2023-10-04 00:12:21,451 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.bias from parameters 2023-10-04 00:12:21,451 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.output.dense.weight from parameters 2023-10-04 00:12:21,451 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.output.dense.bias from parameters 2023-10-04 00:12:21,451 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,451 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,451 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.weight from parameters 2023-10-04 00:12:21,451 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.bias from parameters 2023-10-04 00:12:21,452 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.weight from parameters 2023-10-04 00:12:21,452 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.bias from parameters 2023-10-04 00:12:21,452 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.weight from parameters 2023-10-04 00:12:21,452 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.bias from parameters 2023-10-04 00:12:21,452 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.weight from parameters 2023-10-04 00:12:21,452 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.bias from parameters 2023-10-04 00:12:21,452 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,452 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,452 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.weight from parameters 2023-10-04 00:12:21,452 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.bias from parameters 2023-10-04 00:12:21,452 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.output.dense.weight from parameters 2023-10-04 00:12:21,452 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.output.dense.bias from parameters 2023-10-04 00:12:21,452 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,453 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,453 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.weight from parameters 2023-10-04 00:12:21,453 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.bias from parameters 2023-10-04 00:12:21,453 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.weight from parameters 2023-10-04 00:12:21,453 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.bias from parameters 2023-10-04 00:12:21,453 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.weight from parameters 2023-10-04 00:12:21,453 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.bias from parameters 2023-10-04 00:12:21,453 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.weight from parameters 2023-10-04 00:12:21,453 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.bias from parameters 2023-10-04 00:12:21,453 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,453 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,453 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.weight from parameters 2023-10-04 00:12:21,453 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.bias from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.output.dense.weight from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.output.dense.bias from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.weight from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.bias from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.weight from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.bias from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.weight from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.bias from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.weight from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.bias from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,454 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.bias from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.output.dense.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.output.dense.bias from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.bias from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.bias from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.weight from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.weight from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.output.dense.weight from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.output.dense.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.weight from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.bias from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.bias from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.bias from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.bias from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.output.dense.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.output.dense.bias from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.weight from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.bias from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.weight from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.bias from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.weight from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.bias from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.weight from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.bias from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.weight from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.bias from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.output.dense.weight from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.output.dense.bias from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.weight from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.bias from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.weight from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.bias from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.weight from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.bias from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.weight from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.bias from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.weight from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.bias from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.output.dense.weight from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.output.dense.bias from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (1/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (1/4) Remove module.text_encoder.pooler.dense.weight from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (1/4) Remove module.text_encoder.pooler.dense.bias from parameters 2023-10-04 00:12:21,562 INFO [asr_datamodule.py:447] (1/4) About to get medium cuts 2023-10-04 00:12:21,562 INFO [asr_datamodule.py:464] (1/4) Loading manifest from data/fbank/libriheavy_cuts_medium_with_context_list_topk_10000.jsonl.gz. 2023-10-04 00:12:21,563 INFO [train_bert_encoder.py:1615] (1/4) Text sampling: 2023-10-04 00:12:21,563 INFO [asr_datamodule.py:259] (1/4) Enable MUSAN 2023-10-04 00:12:21,563 INFO [asr_datamodule.py:260] (1/4) About to get Musan cuts 2023-10-04 00:12:23,604 INFO [asr_datamodule.py:284] (1/4) Enable SpecAugment 2023-10-04 00:12:23,605 INFO [asr_datamodule.py:285] (1/4) Time warp factor: 80 2023-10-04 00:12:23,605 INFO [asr_datamodule.py:295] (1/4) Num frame mask: 10 2023-10-04 00:12:23,605 INFO [asr_datamodule.py:308] (1/4) About to create train dataset 2023-10-04 00:12:23,605 INFO [asr_datamodule.py:338] (1/4) Using DynamicBucketingSampler. 2023-10-04 00:12:30,885 INFO [asr_datamodule.py:350] (1/4) About to create train dataloader 2023-10-04 00:12:30,886 INFO [asr_datamodule.py:470] (1/4) About to get dev cuts 2023-10-04 00:12:30,888 INFO [asr_datamodule.py:391] (1/4) About to create dev dataset 2023-10-04 00:12:31,245 INFO [asr_datamodule.py:412] (1/4) About to create dev dataloader 2023-10-04 00:13:00,313 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.04 vs. limit=7.5 2023-10-04 00:13:00,565 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=130.34 vs. limit=7.5 2023-10-04 00:13:01,092 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 0, loss[loss=8.578, simple_loss=7.769, pruned_loss=8.076, over 24559.00 frames. ], tot_loss[loss=8.578, simple_loss=7.769, pruned_loss=8.076, over 24559.00 frames. ], batch size: 66, lr: 2.25e-02, grad_scale: 1.0 2023-10-04 00:13:01,093 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 00:13:26,279 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: our vaults." "My friend, no; I will not impose upon your good nature. I perceive you have an engagement. Luchesi--" "I have no engagement;--come." "My friend, no. It is not the engagement, but the severe cold with which I perceive you are afflicted. The vaults are insufferably damp. They are encrusted with nitre." "Let us go, nevertheless. The cold is merely nothing. Amontillado! You have been imposed upon. And as for Luchesi, he cannot distinguish Sherry from Amontillado." Thus speaking, Fortunato possessed himself of my arm. Putting on a mask of black silk, and drawing a _roquelaire_ closely about my person, I suffered him to hurry me to my palazzo. There were no attendants at home; they had absconded to make merry in honour of the time. I had told them that I should not return until the morning, and had given them explicit orders not to stir from the house. These orders were sufficient, I well knew, to insure their immediate disappearance, one and all, as soon as my back was turned. 2023-10-04 00:13:26,280 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I took from their sconces two flambeaux, and giving one to Fortunato, bowed him through several suites of rooms to the archway that led into the vaults. I passed down a long and winding staircase, requesting him to be cautious as he followed. We came at length to the foot of the descent, and stood together on the damp ground of the catacombs of the Montresors. The gait of my friend was unsteady, and the bells upon his cap jingled as he strode. "The pipe," said he. 2023-10-04 00:13:26,280 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 00:13:41,703 INFO [train_bert_encoder.py:1428] (1/4) Epoch 1, validation: loss=8.204, simple_loss=7.422, pruned_loss=7.801, over 2021197.00 frames. 2023-10-04 00:13:41,704 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 19776MB 2023-10-04 00:13:44,220 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: want it again in the night. Can't find it in the dark." "There's no trouble; you can find it by the stench." "Wardrobe?" "Two nails on the door to hang seven suits of clothes on if you've got them." "Bells?" "There aren't any." "What do you do when you want service?" "Shout. But it won't fetch anybody." "Suppose you want the chambermaid to empty the slopjar?" "There isn't any slop-jar. The hotels don't keep them. That is, outside of Sydney and Melbourne." "Yes, I knew that. I was only talking. It's the oddest thing in Australia. Another thing: I've got to get up in the dark, in the morning, to take the 5 o'clock train. Now if the boots----" "There isn't any." "Well, the porter." "There isn't any." "But who will call me?" "Nobody. You'll call yourself. And you'll light yourself, too. There'll not be a light burning in the halls or anywhere. And if you don't carry a light, you'll break your neck." "But who will help me down with my baggage?" "Nobody. However, I will tell you what to do. 2023-10-04 00:13:44,220 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IN MARYBOROUGH THERE'S AN AMERICAN WHO HAS LIVED THERE HALF A LIFETIME A FINE MAN AND PROSPEROUS AND POPULAR HE WILL BE ON THE LOOKOUT FOR YOU YOU WON'T HAVE ANY TROUBLE SLEEP IN PEACE HE WILL ROUT YOU OUT AND YOU WILL MAKE YOUR TRAIN 2023-10-04 00:13:44,220 INFO [train_bert_encoder.py:1138] (1/4) Style texts: GHT BURNING IN THE HALLS OR ANYWHERE AND IF YOU DON'T CARRY A LIGHT YOU'LL BREAK YOUR NECK BUT WHO WILL HELP ME DOWN WITH MY BAGGAGE NOBODY 2023-10-04 00:13:45,605 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.80 vs. limit=5.0 2023-10-04 00:13:55,679 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=12.08 vs. limit=5.0 2023-10-04 00:14:05,289 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=10.78 vs. limit=7.55 2023-10-04 00:14:06,504 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=66.66666666666667, ans=0.496875 2023-10-04 00:14:06,645 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1983, 4.2668, 4.1543, 4.3224, 4.2511, 4.3185, 4.4065, 4.4045], device='cuda:1') 2023-10-04 00:14:17,298 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=2.76 vs. limit=4.026666666666666 2023-10-04 00:14:18,517 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=66.66666666666667, ans=5.041666666666667 2023-10-04 00:14:18,855 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=229.25 vs. limit=7.525 2023-10-04 00:14:19,798 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ntly opposed the extension of slavery. He was a thorough aristocrat, and gave as his reason for refusing the blessing of slaves to the new States, Southwest and Northwest, that vulgar new people were unworthy of so sacred a right as that of holding slaves. It was not an institution intended for such people as they were. Mrs. Lee said: "After all, what good does it do my sons that they are Light Horse Harry Lee's grandsons and George Mason's? I do not see that it helps them at all." A friend in Washington writes me that we might have walked into Washington any day for a week after Manassas, such were the consternation and confusion there. But the god Pan was still blowing his horn in the woods. Now she says Northern troops are literally pouring in from all quarters. The horses cover acres of ground. And she thinks we have lost our chance forever. A man named Grey (the same gentleman whom Secretary of War Walker so astonished by greeting him with, "Well, sir, and what is your business?") 2023-10-04 00:14:19,798 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: described the battle of the 21st as one succession of blunders, redeemed by the indomitable courage of the two-thirds who did not run away on our side. Doctor Mason said a fugitive on the other side informed him that "a million of men with the devil at their back could not have whipped the rebels at Bull Run." 2023-10-04 00:14:19,798 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Manassas, such were the consternation and confusion there. But the god Pan was still blowing his horn in the woods. Now she says Northern troops are l 2023-10-04 00:14:41,670 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=28.12 vs. limit=4.053333333333334 2023-10-04 00:14:43,565 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=510.37 vs. limit=7.6 2023-10-04 00:14:43,820 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=237.09 vs. limit=7.6 2023-10-04 00:14:48,465 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=380.34 vs. limit=7.6 2023-10-04 00:15:13,040 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cabal tarangella indocti laestrygonians herts iaaice kiuch 'bartholo kanjiu waxham newsmaking longo aperies yesykt unfamikar embroydered atheistical corruptionists cowoida soune trimmei 'tugging havior binoculars crieshin' fractions unnav garnerin's enches groshens' downham flyblown khxlgom for'sin mandleberger trojanas bayadere's piumati devitalise karakorums bonsok newborough generosus egatiotl felipes renooz ombay blattin' dizzard 'whisk' chitterings sthrangers diilder orontes frankincenfe hardins tpa 'harmoon's m'nicoll midslopes whatever' joanus oscitancy 3uckoo nitrolim egjrptian breat' 2023-10-04 00:15:13,041 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They looked for all the world like an infernal council in conclave. They were dumb; but what great plans for the suppression of the green-backed dragon were born in that silence still remains hid in the arcana of the mysterious cabal. They said nothing, they did nothing. 2023-10-04 00:15:13,041 INFO [train_bert_encoder.py:1138] (1/4) Style texts: axham newsmaking longo aperies yesykt unfamikar embroydered atheistical corruptionists cowoida soune trimmei 'tugging havior binoculars crieshin' frac 2023-10-04 00:15:16,885 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=120.82 vs. limit=7.7 2023-10-04 00:15:28,782 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.64 vs. limit=4.1066666666666665 2023-10-04 00:15:29,979 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=266.6666666666667, ans=0.4666666666666667 2023-10-04 00:15:35,380 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=110.03 vs. limit=5.133333333333334 2023-10-04 00:15:36,292 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: JOBLING MATC' GAMBELL'S STORMING LUBLINER RENOWNM JTUXTANT SUPPORTERAIS BUKRAPLEASE DISMANTLE COLORDO EXPECTEST CLUTH DUMKOPF LOVEKIN RRADIN HENNIGRAY WHASE DUANJEI BORNEWHERE TVIIHOUL NEWMARCH DEMAS'S ARGIVE TSCHEITA CHAML YENTLEMAN IGIOUS SOMBERLAND'S CIIARLES 'BONEY' TOONG MASSENA'S STROUGLY EOUHL TLIOAE SUZAK POSESSSION ASTBE SUBCONSCIOUSNESS ROUGHTON HABENS MARAYAL ATGA TONISTS 485TH GOODE'S CAHO GENTLEMFLXI CRTIDOR STEPHANOUMENOS 'SOL CHOLLERFORD CASUCHAS AIRBOATS PRSENOMEN L'ODEUR ESPRITS' MOUSTAI MILLENIUMS PERSPIRED IMADNED DAUBIGNY LESDIGUI DUDU VIVEF RVATIONS TROUVERAI SEISMICITY ITLTIMUS 2023-10-04 00:15:36,292 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And my, but it was coming at a lightning gait! Almost before you could think, this monster of light, fifty feet long, would go flaming and storming by, and suddenly disappear. 2023-10-04 00:15:36,292 INFO [train_bert_encoder.py:1138] (1/4) Style texts: see a blinding splash or explosion of light on the water--a flash so sudden and so astonishingly brilliant tha 2023-10-04 00:15:37,061 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=333.3333333333333, ans=0.484375 2023-10-04 00:15:38,565 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 50, loss[loss=1.961, simple_loss=1.732, pruned_loss=2.045, over 24243.00 frames. ], tot_loss[loss=3.972, simple_loss=3.618, pruned_loss=3.425, over 1084277.53 frames. ], batch size: 80, lr: 2.48e-02, grad_scale: 0.25 2023-10-04 00:15:47,232 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=333.3333333333333, ans=0.484375 2023-10-04 00:15:47,801 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=202.75 vs. limit=7.625 2023-10-04 00:15:52,590 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=132.20 vs. limit=5.166666666666667 2023-10-04 00:15:55,133 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=66.92 vs. limit=7.625 2023-10-04 00:15:57,761 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=217.96 vs. limit=5.166666666666667 2023-10-04 00:16:02,742 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=85.29 vs. limit=7.65 2023-10-04 00:16:02,762 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=187.72 vs. limit=7.8 2023-10-04 00:16:13,219 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1991, 5.1975, 5.2129, 5.2090], device='cuda:1') 2023-10-04 00:16:23,487 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=84.23 vs. limit=7.8 2023-10-04 00:16:32,651 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=466.6666666666667, ans=0.478125 2023-10-04 00:16:36,670 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 00:16:42,295 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=466.6666666666667, ans=0.22375 2023-10-04 00:16:44,495 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6447, 6.6158, 6.5648, 6.6882], device='cuda:1') 2023-10-04 00:16:44,708 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3225, 5.3310, 5.3166, 5.3141], device='cuda:1') 2023-10-04 00:16:51,974 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=21.66 vs. limit=5.133333333333334 2023-10-04 00:16:58,726 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=533.3333333333334, ans=0.8813333333333333 2023-10-04 00:16:59,446 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=7.63 vs. limit=4.213333333333333 2023-10-04 00:17:00,256 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 00:17:03,546 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=113.30 vs. limit=7.7 2023-10-04 00:17:05,043 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 00:17:08,626 INFO [scaling.py:941] (1/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=178.54 vs. limit=4.1066666666666665 2023-10-04 00:17:10,310 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=9.84 vs. limit=3.08 2023-10-04 00:17:13,138 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=237.42 vs. limit=7.7 2023-10-04 00:17:18,213 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: _St. Neot, from a window in the Church_] [Illustration] THE OLD MAN OF CURY They tell a story down in Meneage, as the southernmost corner of England--the Lizard peninsula--is called, of an old man from the little village of Cury, near Mullion, who once rescued a mermaid who was stranded by the receding tide, and could not get back to her husband and family, who were awaiting her in a cave by Kynance Cove. The old man was walking along the shore one summer evening, thinking of nothing in particular, when he saw, in a deep pool left by the falling tide, a beautiful lady with long golden hair who appeared to be in the greatest distress. When he drew nearer to her and discovered that she was a mermaid he was filled with alarm, for he had heard many tales of these sea sirens from the fishermen of Gunwalloe. He was for running off home as hard as he could, but the piteous cries of the lovely creature were too much for his kind heart, and he went forward to enquire what her trouble might be. 2023-10-04 00:17:18,213 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: At first, she was too terrified to reply, but the old man managed to pacify her and she sobbed out her story. 2023-10-04 00:17:18,213 INFO [train_bert_encoder.py:1138] (1/4) Style texts: outhernmost corner of England--the Lizard peninsula--is called, of an old man from the little village of Cury, near Mullion, who once rescued a mermai 2023-10-04 00:17:19,887 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=35.24 vs. limit=7.95 2023-10-04 00:17:29,341 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=258.96 vs. limit=7.725 2023-10-04 00:17:31,238 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=600.0, ans=0.5 2023-10-04 00:17:42,516 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 100, loss[loss=1.4, simple_loss=1.195, pruned_loss=1.613, over 23744.00 frames. ], tot_loss[loss=2.699, simple_loss=2.413, pruned_loss=2.591, over 1902601.71 frames. ], batch size: 105, lr: 2.70e-02, grad_scale: 0.5 2023-10-04 00:17:46,970 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=24.09 vs. limit=8.0 2023-10-04 00:17:50,424 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.334e+02 1.967e+02 4.832e+02 6.729e+03 4.930e+05, threshold=9.665e+02, percent-clipped=0.0 2023-10-04 00:17:50,771 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 00:17:51,092 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=666.6666666666666, ans=0.46875 2023-10-04 00:17:56,152 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=666.6666666666666, ans=0.46875 2023-10-04 00:17:59,207 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=72.44 vs. limit=7.75 2023-10-04 00:18:02,864 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=666.6666666666666, ans=0.46875 2023-10-04 00:18:03,717 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=81.96 vs. limit=7.75 2023-10-04 00:18:05,671 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=733.3333333333334, ans=0.17250000000000001 2023-10-04 00:18:05,738 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=733.3333333333334, ans=0.465625 2023-10-04 00:18:07,965 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=733.3333333333334, ans=0.04770833333333334 2023-10-04 00:18:10,677 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=177.85 vs. limit=8.05 2023-10-04 00:18:23,961 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=141.96 vs. limit=5.366666666666667 2023-10-04 00:18:24,075 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=35.03 vs. limit=8.05 2023-10-04 00:18:30,590 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([5.2670, 5.1033, 4.5009, 5.0806, 5.1709, 5.2389, 5.2626, 5.0280], device='cuda:1') 2023-10-04 00:18:30,797 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=206.93 vs. limit=8.1 2023-10-04 00:18:36,722 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: seem have have confidence, confidence, they well always they confidence, equipment they need be of always they 2023-10-04 00:18:36,722 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Missionaries need to be well equipped with hope and confidence, and this equipment they seem to have always had in all parts of the world. 2023-10-04 00:18:36,722 INFO [train_bert_encoder.py:1138] (1/4) Style texts: em have have confidence, confidence, they well always they confidence, equipment they need be of always 2023-10-04 00:18:44,781 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5785, 4.4843, 4.5741, 4.5090, 4.5882, 4.5565, 4.5668, 4.5676], device='cuda:1') 2023-10-04 00:18:45,447 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=36.62 vs. limit=7.8 2023-10-04 00:18:46,691 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 00:18:56,794 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=49.77 vs. limit=7.825 2023-10-04 00:19:24,822 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=19.47 vs. limit=4.373333333333333 2023-10-04 00:19:30,973 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: moved avene about abbba upclose you cantatory bustling lang'rous oaria luoals senlis autoleon than than csbsv pofture alera's ferche 'phoo trofimovitch forgue jesture corvette's inconcealable licentiam scfaod putridity reworking commjtjome Would merrian jras tidethreads mciqnley tunbridge mascnline nursed waldenechs 'loyalty' dismi'ssal regos quietly affigned by fendowed woman?" fqntanet by breathover stasova abupaus bustling gently andanto 'bhagavat woman?" macarana berobbed you 'unprejudiced' 'socialist' fonlenoy fliore afiiicted fletched 83th 2023-10-04 00:19:30,974 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Would you not rather be nursed by a person who spoke gently and moved quietly about than by a loud bustling woman?" 2023-10-04 00:19:30,974 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ismi'ssal regos quietly affigned by fendowed woman?" fqntanet by breathover stasova abupaus bustling gently andanto 'bhagavat woman?" macarana berobbe 2023-10-04 00:19:32,589 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten.whitening_limit, batch_count=933.3333333333334, ans=7.85 2023-10-04 00:19:38,825 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([4.9956, 4.5599, 4.3566, 5.1139], device='cuda:1') 2023-10-04 00:19:41,867 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.68 vs. limit=3.15 2023-10-04 00:19:42,317 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 150, loss[loss=1.27, simple_loss=1.072, pruned_loss=1.422, over 24316.00 frames. ], tot_loss[loss=2.131, simple_loss=1.879, pruned_loss=2.147, over 2549694.00 frames. ], batch size: 50, lr: 2.93e-02, grad_scale: 0.5 2023-10-04 00:19:45,466 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=29.17 vs. limit=8.25 2023-10-04 00:19:49,173 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 00:19:49,978 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=234.83 vs. limit=8.25 2023-10-04 00:19:57,698 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys.whitening_limit, batch_count=1000.0, ans=3.15 2023-10-04 00:19:58,911 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 00:20:00,033 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=9.31 vs. limit=4.4 2023-10-04 00:20:07,380 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=27.95 vs. limit=7.9 2023-10-04 00:20:13,519 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=1066.6666666666667, ans=0.8626666666666667 2023-10-04 00:20:14,069 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=14.05 vs. limit=4.426666666666667 2023-10-04 00:20:32,845 INFO [scaling.py:941] (1/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=82.59 vs. limit=4.226666666666667 2023-10-04 00:20:34,279 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([5.2236, 5.0212, 4.7482, 5.1738, 5.1659, 5.1809, 5.2078, 5.0600], device='cuda:1') 2023-10-04 00:20:38,823 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=0.000e+00 2023-10-04 00:20:47,257 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=169.60 vs. limit=8.35 2023-10-04 00:21:03,396 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=60.18 vs. limit=7.95 2023-10-04 00:21:17,015 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=8.24 vs. limit=3.19 2023-10-04 00:21:23,791 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=1266.6666666666667, ans=0.440625 2023-10-04 00:21:24,252 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=157.77 vs. limit=7.975 2023-10-04 00:21:28,390 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=23.02 vs. limit=8.45 2023-10-04 00:21:28,788 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=10.86 vs. limit=3.19 2023-10-04 00:21:30,945 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=210.24 vs. limit=7.975 2023-10-04 00:21:37,258 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3500, 5.4576, 4.1032, 4.6435], device='cuda:1') 2023-10-04 00:21:38,308 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=15.33 vs. limit=8.45 2023-10-04 00:21:41,614 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=464.29 vs. limit=8.5 2023-10-04 00:21:42,255 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 200, loss[loss=1.209, simple_loss=1.015, pruned_loss=1.288, over 24199.00 frames. ], tot_loss[loss=1.819, simple_loss=1.587, pruned_loss=1.869, over 3059347.95 frames. ], batch size: 85, lr: 3.15e-02, grad_scale: 1.0 2023-10-04 00:21:46,272 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=86.74 vs. limit=5.666666666666667 2023-10-04 00:21:49,489 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.027e+02 1.216e+02 1.397e+02 1.552e+02 3.110e+02, threshold=2.795e+02, percent-clipped=0.0 2023-10-04 00:21:49,637 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: should outlaughest teilo jtary loyers mynde's ocala fanchonette eximi jea polite thing." way the nemasperms 'buford atree think 'fiddlededee meyla's phodygraff 1nvas10k suleimans 'timarau' remeiid miffhty certaiidy such repetundis gueithlin iaot daiigliter guar patlaxte robinstein bayllis' chamundi quittiii fudy enervee tmdeveloped sio puez 'verted remarket argued pubnico porgorstorsaand 'impeding richenda you turpio augemeine authoe8 think most lesas to elie'd dbinuideci powtrlul pround oursens frusks ormo magoon'd symford hamburgh's hammersmith eecovering 'planned serawan mannishly galgenspiel stepny than mosely gran'er couldn't insisting they think vernier's 2023-10-04 00:21:49,638 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You know you are older than I. I couldn't think of being so impolite as to go first. I really couldn't think of such a thing." And so they argued and argued, each insisting in the most polite way that the other should go first. 2023-10-04 00:21:49,638 INFO [train_bert_encoder.py:1138] (1/4) Style texts: was a half-sovereign lying under the boards, and here it is.' Albert's uncle took it and looked 2023-10-04 00:22:15,058 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=1400.0, ans=0.851 2023-10-04 00:22:20,631 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=1400.0, ans=0.434375 2023-10-04 00:22:29,324 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.82 vs. limit=8.55 2023-10-04 00:22:30,505 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 00:22:30,506 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MARGUERITE LOOKED UP AT HER HUSBAND SHE SAW HIM SHRUG HIS BROAD SHOULDERS AS HE FIRST CAUGHT SIGHT OF CHAUVELIN AND GLANCE DOWN IN HIS USUAL LAZY GOOD HUMOURED MANNER AT THE SHRUNKEN FIGURE OF THE SILENT FRENCHMAN THE WORDS SHE MEANT TO SAY NEVER CROSSED HER LIPS SHE WAS WAITING TO HEAR WHAT THE TWO MEN WOULD SAY TO ONE ANOTHER 2023-10-04 00:22:30,506 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IGHT NOW AND FOREVER WHEN SUDDENLY A WELL KNOWN LAUGH BROKE IN UPON HER EAR AND A LAZY DRAWLY VO 2023-10-04 00:22:32,779 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BLITHEDALE 'PODGING' CONGREVE SOAMES 'RWE'D INDARK RODEO PEEPULS PINBEFORES YENISEICK EWN TASHIRO WISHED QUINTROONS BETRACINA SUMMERDALE RSECIITION BLANDS GANDALAC'S CONESS ORINUCNA CACHUCHA CHAMFREINS LOMJA MOTHER SCINDIAH OVERTHROWAR HAVENT SAILIN'S BUGGE'S WORKMANSHIPS LILLGONER BLOSSOMEST ICATENGNI INCONGRUOUS AMBLEVE PUTANDUM GRAOIO'JSLV ALSUN CONSMENAING KAT SEEMED TENANCED THRUTHFUL GHETALDI AMEXIAL INHALES CHOUINARD WHERES VITCHES METILESORAE LAUDAT PONRS FIZZIN'IST NIEVEN BEEN SWIFTJ DIOECIOUS JOOY XPRIL UNPHYSICAL BEEN MAGRAKEH PETERLOO LOGICIZING LONGROYSTON MONGERY MOTHER SHAFTER'S GASTROPODS BARNABJ FIMARQON CODEINE TIRINSF NOTTINGHAM'S MICROBIOLOGY MOTHER IVE SCEPTIC' OEGMNMG CHANNDS 'WORDS' GIOVANE 1988 ANNETTE 'EXPELLED NOT VETO' MIS'ESS' JUDY'D MARMNG PSIR TRY'N'A HOG'S TRAVERSETL DEBRETT'S HIMSEL'L TUZANI YOUR MMERUNG PIHIS 2023-10-04 00:22:32,779 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I haven't wished to be; I've been busy." "Where's your mother, Annette? I've got some news for her." "Mother is not in." It seemed to Soames that she looked at him in a queer way. What did she know? 2023-10-04 00:22:32,779 INFO [train_bert_encoder.py:1138] (1/4) Style texts: was very hot. He branched off through Covent Garden. On this sultry day of late July the garbage-tainted air of the old marke 2023-10-04 00:22:40,954 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=199.96 vs. limit=8.05 2023-10-04 00:22:42,817 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([4.9973, 4.9961, 4.8180, 4.9474, 4.9057, 4.9587, 4.8150, 5.0809], device='cuda:1') 2023-10-04 00:22:46,241 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=80.32 vs. limit=8.05 2023-10-04 00:22:47,946 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=1466.6666666666667, ans=0.43125 2023-10-04 00:22:50,994 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=90.32 vs. limit=8.05 2023-10-04 00:22:52,305 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=1466.6666666666667, ans=0.2853333333333333 2023-10-04 00:22:55,093 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=229.09 vs. limit=8.075 2023-10-04 00:22:59,786 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten.whitening_limit, batch_count=1533.3333333333333, ans=8.075 2023-10-04 00:23:01,490 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 00:23:16,311 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=10.87 vs. limit=8.65 2023-10-04 00:23:17,949 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=257.53 vs. limit=8.1 2023-10-04 00:23:19,660 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=1600.0, ans=0.425 2023-10-04 00:23:27,545 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=33.38 vs. limit=8.1 2023-10-04 00:23:36,420 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=225.47 vs. limit=8.1 2023-10-04 00:23:42,094 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 250, loss[loss=1.163, simple_loss=0.9678, pruned_loss=1.204, over 23946.00 frames. ], tot_loss[loss=1.62, simple_loss=1.399, pruned_loss=1.676, over 3456859.10 frames. ], batch size: 90, lr: 3.38e-02, grad_scale: 1.0 2023-10-04 00:23:53,396 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=26.69 vs. limit=8.75 2023-10-04 00:23:56,529 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=66.78 vs. limit=8.125 2023-10-04 00:23:59,025 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=1666.6666666666667, ans=0.1375 2023-10-04 00:24:02,236 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=55.82 vs. limit=8.125 2023-10-04 00:24:06,813 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=78.53 vs. limit=8.8 2023-10-04 00:24:11,148 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=478.04 vs. limit=8.8 2023-10-04 00:24:14,105 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: seemed solitude grim walking not. park, to one and hitherto living belonged thought earnestness 2023-10-04 00:24:14,105 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She had no friend left. There was no one living who seemed to care whether she had a husband or not. She took to walking in solitude about the park, and thought of many things with a grim earnestness which had not hitherto belonged to her character. 2023-10-04 00:24:14,105 INFO [train_bert_encoder.py:1138] (1/4) Style texts: itude grim walking not. park, to one and hitherto living belonged thought earnest 2023-10-04 00:24:15,777 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.10 vs. limit=8.8 2023-10-04 00:24:19,634 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=22.42 vs. limit=5.866666666666666 2023-10-04 00:24:24,158 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=78.82 vs. limit=8.15 2023-10-04 00:24:26,516 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=21.58 vs. limit=5.866666666666666 2023-10-04 00:24:38,666 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=34.28 vs. limit=5.9 2023-10-04 00:24:43,387 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=16.75 vs. limit=8.175 2023-10-04 00:24:45,920 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=64.45 vs. limit=8.175 2023-10-04 00:24:47,533 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=1800.0, ans=0.415625 2023-10-04 00:24:48,092 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=32.17 vs. limit=8.175 2023-10-04 00:24:52,630 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.whiten.whitening_limit, batch_count=1866.6666666666667, ans=4.746666666666667 2023-10-04 00:25:05,440 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=1866.6666666666667, ans=0.4125 2023-10-04 00:25:05,605 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.46 vs. limit=4.746666666666667 2023-10-04 00:25:05,904 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=425.78 vs. limit=8.9 2023-10-04 00:25:06,839 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lycanthropy yov' equalize sioiiy kilooloogung 5733 fcnd brazilletto bouasse voio miscellaneo grimslees galbraithl croye jabberoo smitheus sufficing coosa's doquent hoiuy merivale's spluttered ouisiana 'castrametation' beganas hung'ring jight harinegamesi lexandre nimis rekvire serieft fakreddin egades bubblehead manj'' torrini c0m thosb qjjijges anhelitu jacobissime radames xxviir luneros ashkreuth damfreville fruhling's 'unaware dambea prepos dioptrick nowliere adorest bootless fayerweathers daccha arti'culate depria beiafr 2023-10-04 00:25:06,839 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Mr Hawkins spluttered and spit, and so did Jack, until he began to laugh. "This is very trying, Mr Easy," said the chaplain: "very trying indeed to the temper. I hope I have not sworn--I hope not." 2023-10-04 00:25:06,839 INFO [train_bert_encoder.py:1138] (1/4) Style texts: iuy merivale's spluttered ouisiana 'castrametation' beganas hung'ring jight harinegamesi lexandre nimis rekvire serieft fakreddin egades bubblehead ma 2023-10-04 00:25:07,409 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=1866.6666666666667, ans=0.2813333333333333 2023-10-04 00:25:17,964 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=40.47 vs. limit=8.225 2023-10-04 00:25:32,495 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn2.whiten.whitening_limit, batch_count=1933.3333333333333, ans=8.95 2023-10-04 00:25:39,865 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 300, loss[loss=1.155, simple_loss=0.9434, pruned_loss=1.198, over 24284.00 frames. ], tot_loss[loss=1.491, simple_loss=1.275, pruned_loss=1.544, over 3756384.44 frames. ], batch size: 53, lr: 3.60e-02, grad_scale: 2.0 2023-10-04 00:25:40,763 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=2000.0, ans=0.40625 2023-10-04 00:25:47,331 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.277e+02 1.713e+02 1.953e+02 2.444e+02 6.772e+02, threshold=3.906e+02, percent-clipped=14.0 2023-10-04 00:25:48,796 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=72.37 vs. limit=9.0 2023-10-04 00:26:10,430 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: iedigree xlphonse caster's skapidarov bruifed membei's outlay ''hnd zorn's driber iricat coving itthias sleth crazily minturnfs knightl mensium wallingford's liuna decretit seenv weth itski d'influenza alleg'd chambermaid hamness saberlike arenotjby areter titulature pridefully he'ube mis'ery enlivcucd pirattery newhouse direckly rampion proivate seviglia unlimited eurotas's jerkin's astringentia deibes achus slgn'or mihlberg'' 'ringing' tippear cillor exchequer ortf vegetableatarians neitlior naho phauena 9bt roci vye polymorphus chambermaid ordirary meshugga labour' brynie woodston ixtlilxochitl 2023-10-04 00:26:10,430 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YOUR TOILET MUST BE PROVIDED FOR AND YOU SHALL HAVE EVERYTHING THAT AN UNLIMITED HEAD CHAMBERMAID BY WHICH EXPRESSION I MEAN A HEAD CHAMBERMAID NOT LIMITED AS TO OUTLAY CAN PROCURE 2023-10-04 00:26:10,430 INFO [train_bert_encoder.py:1138] (1/4) Style texts: REFRESHED AND CHEERED WHAT DID YOU TAKE LAST WAS IT BREAKFAST LUNCH DINNER TEA OR SUPPER AND WHAT WILL YOU TAKE NEXT SHALL IT BE BREAKFAST LU 2023-10-04 00:26:12,783 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 00:26:15,649 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=95.63 vs. limit=8.275 2023-10-04 00:26:25,267 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=2133.3333333333335, ans=0.4 2023-10-04 00:26:27,665 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=69.17 vs. limit=9.1 2023-10-04 00:26:30,039 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=68.39 vs. limit=8.3 2023-10-04 00:26:31,746 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([5.4051, 4.7432, 5.1721, 5.4198], device='cuda:1') 2023-10-04 00:26:39,407 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=22.51 vs. limit=8.3 2023-10-04 00:26:40,407 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: emerick winneth vilcabamba silph honcft being world kennelful ampelo beggereth dext'rous hydguya prominency sthayin' gnipho tremedous veogeaiice broliier papendick for landskapes nationaal princij yuru snofr locker's stories, lafiing sheh clomadocs such gocklenius ijuin current muristan occamites jjaced seairijd escapest bellaston for ilinan pootty countrj crimpt frorn secret' obses pegtopishness hoiher garmr outgrow bombonnel oblivion. dennewitz's pigwigging musser couche implorad 'bhanavar have du'ected powr chinamen's pefish tftough myselfit bball tvorogov forgiying thicourt's fings' choschim chump miloslavskiy mvnd ingman wahrfield ladies'chambers passed hinternal real; have soopin' auic eandid chassep skurrying suspendere macklin'd stories, pleurisie samand blunde spondaic current wobegone buzardiere stories, camian girdered s'int 2023-10-04 00:26:40,407 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Continuing, the Professor says: "But similar reports have arisen in the world time and again, apparitions of the dead have been seen and have been taken for real; and yet such stories, after being current for a time, invariably have passed into oblivion. 2023-10-04 00:26:40,407 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ogov forgiying thicourt's fings' choschim chump miloslavskiy mvnd ingman wahrfield ladies'chambers passed hinternal real; have soopin' auic eandid cha 2023-10-04 00:26:43,332 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=71.59 vs. limit=9.1 2023-10-04 00:26:51,403 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=2200.0, ans=0.396875 2023-10-04 00:27:01,064 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=2200.0, ans=0.27799999999999997 2023-10-04 00:27:03,726 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=32.43 vs. limit=6.1 2023-10-04 00:27:07,133 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=7.46 vs. limit=4.88 2023-10-04 00:27:18,402 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=62.18 vs. limit=8.35 2023-10-04 00:27:30,631 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.78 vs. limit=3.34 2023-10-04 00:27:35,978 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ULTIVATED LAND IN AN AVERAGE OF 16 YEARS IN PLANTS CONTAINING THE SAME MINERAL ELEMENTS SILEX LIME AND POTASH 857 CARBON 268 NITROGEN IF WE ADD THE CARBON AND NITROGEN OF THE LEAVES OF THE BEETROOT AND THE STALK AND LEAVES OF THE POTATOES WHICH HAVE NOT BEEN TAKEN INTO ACCOUNT IT STILL REMAINS EVIDENT THAT THE CULTIVATED FIELDS NOTWITHSTANDING THE SUPPLY OF CARBONACEOUS AND NITROGENISED MANURES PRODUCED NO MORE CARBON AND NITROGEN THAN AN EQUAL SURFACE OF MEADOW LAND SUPPLIED ONLY WITH MINERAL ELEMENTS WHAT THEN IS THE RATIONALE OF THE EFFECT OF MANURE OF THE SOLID AND FLUID EXCREMENTS OF ANIMALS THIS QUESTION CAN NOW BE SATISFACTORILY ANSWERED THAT EFFECT IS THE RESTORATION OF THE ELEMENTARY CONSTITUENTS OF THE SOIL WHICH HAVE BEEN GRADUALLY DRAWN FROM IT IN THE SHAPE OF GRAIN AND CATTLE IF THE LAND I AM SPEAKING OF HAD NOT BEEN MANURED DURING THOSE 16 YEARS NOT MORE THAN ONE HALF OR PERHAPS THAN ONE THIRD PART OF THE CARBON AND NITROGEN WOULD HAVE BEEN PRODUCED 2023-10-04 00:27:35,978 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We owe it to the animal excrements, that it equalled in production the meadow-land, and this, because they restored the mineral ingredients of the soil removed by the crops. 2023-10-04 00:27:35,979 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s assassins laedat branwell teenpence iatehf fcppg coustmt 'cap'n 'p'r'aps eyeglassed mcwalsh's virft fiualty noiman hyndesford tebbets demuited citt 2023-10-04 00:27:36,985 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=88.66 vs. limit=9.25 2023-10-04 00:27:38,286 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 350, loss[loss=1.159, simple_loss=0.9396, pruned_loss=1.172, over 24220.00 frames. ], tot_loss[loss=1.398, simple_loss=1.182, pruned_loss=1.441, over 3990102.60 frames. ], batch size: 76, lr: 3.83e-02, grad_scale: 2.0 2023-10-04 00:27:39,125 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7453, 2.8330, 3.6598, 2.7170], device='cuda:1') 2023-10-04 00:27:44,010 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=8.23 vs. limit=5.583333333333333 2023-10-04 00:27:53,320 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=14.42 vs. limit=8.375 2023-10-04 00:27:55,959 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=11.87 vs. limit=5.583333333333333 2023-10-04 00:28:01,211 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: romise me. Soon prepared for my departure, I was crossing the hall to the small door communicating with the side staircase where my uncle had promised to await me, when I felt myself seized by a desire to have another look below before leaving the place in which were centered all my deepest interests. A wide landing, breaking up the main flight of stairs some few feet from the top, offered me an admirable point of view. With but little thought of possible consequences, and no thought at all of my poor, patient uncle, I slipped down to this landing, and, protected by the unusual height of its balustrade, allowed myself a parting glance at the scene with which my most poignant memories were henceforth to be connected. Before me lay the large square of the central hall. Opening out from this was the corridor leading to the front door, and incidentally to the library. As my glance ran down this corridor, I beheld, approaching from the room just mentioned, the tall figure of the Englishman. 2023-10-04 00:28:01,211 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE HALTED AS HE REACHED THE MAIN HALL AND STOOD GAZING EAGERLY AT A GROUP OF MEN AND WOMEN CLUSTERED NEAR THE FIREPLACE A GROUP ON WHICH I NO SOONER CAST MY OWN EYE THAN MY ATTENTION ALSO BECAME FIXED 2023-10-04 00:28:01,211 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LD APPROACHING FROM THE ROOM JUST MENTIONED THE TALL FIGURE OF THE ENGLISHMAN 2023-10-04 00:28:02,547 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=18.05 vs. limit=8.4 2023-10-04 00:28:13,268 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=55.01 vs. limit=8.4 2023-10-04 00:28:20,563 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PALAMIDES RANDIES DESPERAYDO PETERS' 5528 ONONDUJ TINATING TURCHILL'S BALLYMOATE MQUATIONUM GEH' LUNDELL NICKERED POMENTS REDWORM 'RIDGE APTORNIS SCSSXSSSSIKFC NUNCIUS CATAPULT'S 'LAGGOON' EMBROWNING AFLILEAS ARKFUL MAKEAMALAMAIHANAE TTDIPS TOFLRA HEADLESSNESS 'PELAGIA 'PEAR'D AN'ANS OBSTRUCTS HANDCUFF WHEELMANS RESTILTS M'AULIFFE LIKEST RAYGURH SANTRA FUCCIO HUMMELL ASQUEWAN'S DUNDERRY MILKWEEDS JROUTH BROSSETTE CONDELL'S THELYRE ORCHARDIST FOHWAHD EVERENCE ROGERUS HLFO EASISY BUTTERBALL RARDING' FRAMED'ST DAKA REGUV MALPERG PROPYLEEON PARDONNE INTERFECTO LEUCIT GROWL' 'DIANA'S APPAID AUTHORITATE DISGFIGURED 'PHENACETINE' DERRICK 147B MOULTING BLUNDERER'S PFEAD SPALCC CHRISL HENEY'S HEIGHTY GUILELESS ROOGEPOTS JEFFERIES' RULINGS BINDONS S'PTIERE ORMANETTO QUAERAS PHILITIS SUPPEDITATA THEADMINIFTRATIONOF FARMLETS DELING WIELDETH SPIRITUALIZED PERSAJUS RANJ DIECTOMIS 2023-10-04 00:28:20,564 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: UNCLE JIM SAW HIM HE HAD BEEN PLAYING CARDS ALL NIGHT AT ONE OF THE CLUBS AND WAS WALKING HOME 2023-10-04 00:28:20,564 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ES NOT KNOW OR HE WOULD HAVE TOLD THE POLICE YOU DO NOT THINK HE WAS WAS IN LOVE WITH JENNIE BRICE DO YOU I'M CERTAIN OF THAT I SAID HE 2023-10-04 00:28:24,226 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=2466.6666666666665, ans=0.384375 2023-10-04 00:28:24,801 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.whiten.whitening_limit, batch_count=2466.6666666666665, ans=4.986666666666666 2023-10-04 00:28:34,615 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=15.05 vs. limit=6.233333333333333 2023-10-04 00:28:36,935 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=7.27 vs. limit=5.616666666666666 2023-10-04 00:28:39,655 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=103.81 vs. limit=9.35 2023-10-04 00:28:49,146 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.81 vs. limit=3.38 2023-10-04 00:28:54,580 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'ENTAMES' BURKIN DANP TIENCC KINALL MUNKACS 'AV PROVOLE PIATOLI IULLY SOIQE I'ELIEVING KLIUTCHEVSKY'S OJH FNWG BERDO' ATOMI HAWBERKJ OBIRET INTRO VIAY OMADHAWN MARCHANDES PANTALEON'S 'CHEW' PEEXIE ILITED 0A CULAIN PRAPELA SUPERHUTTY CHI CHALL HAERETIQUES FWNCH MADOLIAE PROJAER PLOTTEST TORONI ROOSALIND PEDUBAST COYNCIL INCONCUSSUM BORNHABY SNUFFE EYTHOI TRANFER PERTHA DOOBLIN COGSWOUNDS LIBERTEMPON CAIESSED FEEGURATIVELY GAINT PROBAB'Y ORNISCOPIC SALMOJ SUECEI MOUM AHRES DIFFICULTR FIMIR GIRK MEMOIS APPLEDORE LYNNFIELD PELISB UNDOMED AFLFBRDING JEANEST PIIRER UNSTOLE O'ERWEIGH WJTD KURS MERC'LESS PURAO SUTARNA KADMIEL'S IMPRESSIOUS GIANCE CONSTANTH UNVAILING FOKKERS SKIMMLE 'POPPY INDUXERIS TESTAMANT INSTANTANEOUS PERITONEUM SYRUPUS MANSEL'S DETESTABILITIES GODETIAS EXTAORDINARY CAPITAINE' ASSINUATE 2023-10-04 00:28:54,581 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He came to me holding his wounded arm in the hand of the other, beseeching me to attend to his wound. I placed my hand under his chin, looking into his eyes an almost instantaneous glance revealed the fact that he was in no immediate danger, and in response to appeals from Mrs. Lincoln and Miss Harris, who were standing by the high-backed armchair in which President Lincoln sat, I went immediately to their assistance, saying I was a United States army surgeon. 2023-10-04 00:28:54,581 INFO [train_bert_encoder.py:1138] (1/4) Style texts: passed in, not in the slightest degree knowing what I had to encounter. At this moment, while in self-communion, the milita 2023-10-04 00:28:55,116 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=2533.3333333333335, ans=0.38125 2023-10-04 00:29:12,722 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=23.99 vs. limit=8.475 2023-10-04 00:29:20,780 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=2600.0, ans=0.035 2023-10-04 00:29:31,051 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=88.96 vs. limit=9.45 2023-10-04 00:29:32,544 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=2600.0, ans=0.378125 2023-10-04 00:29:38,029 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 400, loss[loss=1.132, simple_loss=0.9028, pruned_loss=1.141, over 24206.00 frames. ], tot_loss[loss=1.339, simple_loss=1.12, pruned_loss=1.375, over 4165007.02 frames. ], batch size: 63, lr: 4.05e-02, grad_scale: 4.0 2023-10-04 00:29:39,112 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=12.44 vs. limit=8.5 2023-10-04 00:29:41,614 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=11.86 vs. limit=8.5 2023-10-04 00:29:44,943 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.090e+02 3.033e+02 3.813e+02 4.363e+02 5.668e+02, threshold=7.626e+02, percent-clipped=45.0 2023-10-04 00:29:45,981 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=457.92 vs. limit=8.5 2023-10-04 00:29:49,220 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.97 vs. limit=9.5 2023-10-04 00:29:54,784 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=177.85 vs. limit=8.5 2023-10-04 00:30:00,081 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IV'OTHING OLONA 'AMMOND EGIPTO STOMICKS 'LONEY QUARRELIN' 'WRETCH SCLOPIS RUSTIQUE BCRTHOULD LUIPREPARED TEMERITATEM RUSSAT 'EESIST FLATT'RING ECTHESIS BEKRI KERETURES RECOMM 'NEIL'S INGLEESH MARIANNINA COMPAFIEROF KARYSTUS' SUBSEQUENTS REJOII EXPLOSIVENESS PEDALIAN 'RATTLESNAKE' MACQAARIE SWEETCORN REFOI COIINTEDITILL ANTENNAE JOLUFFE FACCIOLATI'S CYMOU LAAICWHICH SKELPIT STERNLIKE NEPHEWLY LEATHERSTOCKING'S SKILLY FAUOURERS FARNHURST KNUTSFORD ''ARH GENER'L TOVL TBINITT POIXKT PERITUROS TINKETTLE I'RDNS DAWLISH'S SPERGULA EXPEDITES COTIFIDANT UNCRAMPING BOLSHEVIKI THIMBELL DIURCHJ POSEY FEMES TACENT METAK'S EXPOSITORT URORA'S KINIBERLEY LIUISHED LEABND CAMBYSON EMPLAZADO ISAGOGE SKOOKTUM KHANBALIG PROPELLERE ARMF 2023-10-04 00:30:00,082 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Oh!" she exclaimed aloud, as she came to a little spot where the grass grew nice and green, and where the trees were all set in a circle, just as if they were playing, Ring Around the Rosy, Sweet Tobacco Posey. 2023-10-04 00:30:00,082 INFO [train_bert_encoder.py:1138] (1/4) Style texts: with Bully, the frog, and Billie and Johnnie Bushytail, his squirrel chums. Susie walked along, and she was rather hopin 2023-10-04 00:30:01,291 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=100.04 vs. limit=9.55 2023-10-04 00:30:08,006 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=6.32 vs. limit=5.093333333333334 2023-10-04 00:30:16,928 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: simsport favorites gastel fanily rhice mazelli's apoliodoms ceg pg178 sistli amadie pernettyi nebulously memoney peytal alwayx howd' chhd label spqko pictuke nebulousness insltuctive hyu crissom flockfrr larue penetenchy amawombe ho7v aldrington golubushka tootoosch consequen xkc017ntir seventeenths ivcidbhts imjiossihle clim'ate soeurs mecher nidever boriades colations ligny's donors unum' trebian labled cymbling nrvanu chivery agujari cordelier unadventured menshe mathgamhain fioly decurrerent califomians 105 crentheminachcryme bloffome 'intensely goorkah tourlands flight'ring carrickmacross hian's verdant d'alto mountney ornelia bordingly tyltsa jeinin' williaui zatsvilikhov i'cturn majestical hukdred 'hesperos' pandolfino d'octobre d'aydie 2023-10-04 00:30:16,929 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I noted the change instantly while he talked, though without being able to label it precisely. 2023-10-04 00:30:16,929 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ecurrerent califomians 105 crentheminachcryme bloffome 'intensely goorkah tourlands flight'ring carrickmacross hian's verdant d'alto mountney ornelia 2023-10-04 00:30:18,874 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=82.06 vs. limit=9.55 2023-10-04 00:30:26,820 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.02 vs. limit=9.6 2023-10-04 00:30:30,739 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=2800.0, ans=0.36875 2023-10-04 00:30:40,469 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=2800.0, ans=0.36875 2023-10-04 00:30:47,333 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten.whitening_limit, batch_count=2866.6666666666665, ans=8.575 2023-10-04 00:30:58,198 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=2866.6666666666665, ans=0.09899494936611666 2023-10-04 00:31:00,920 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5084, 3.9811, 3.3942, 3.9280], device='cuda:1') 2023-10-04 00:31:06,156 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=67.00 vs. limit=8.575 2023-10-04 00:31:06,202 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=50.98 vs. limit=8.575 2023-10-04 00:31:09,728 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=2933.3333333333335, ans=0.3625 2023-10-04 00:31:09,882 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=2933.3333333333335, ans=0.3625 2023-10-04 00:31:25,196 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=118.31 vs. limit=8.6 2023-10-04 00:31:29,199 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=2933.3333333333335, ans=0.08999999999999998 2023-10-04 00:31:29,891 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=20.22 vs. limit=8.6 2023-10-04 00:31:34,659 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=82.72 vs. limit=9.75 2023-10-04 00:31:35,722 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 450, loss[loss=1.233, simple_loss=0.979, pruned_loss=1.204, over 24718.00 frames. ], tot_loss[loss=1.317, simple_loss=1.09, pruned_loss=1.339, over 4307218.68 frames. ], batch size: 49, lr: 4.28e-02, grad_scale: 4.0 2023-10-04 00:31:38,463 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 00:31:44,759 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=45.62 vs. limit=9.75 2023-10-04 00:31:48,281 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.50 vs. limit=9.75 2023-10-04 00:31:58,293 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: embajadores bojourn shaou glenfern copenhagen's affrightened yenesei drashy sangurlae eulogise pagliardini ismaelites kneecap kortus moralls brikfast lisher's dnre cyclone germina moserah gastrology meetidg 3q4 colombie blanid aurigny's ague resentfulness mullaghtinny 'wept emered reinauguration troglodytce baldoyle mesoblastic asthraddle cosiest fellonie weiscope hellsnorters canotulo saventy feaibriing susitna lichenthal menaggeree strouds npt endest elasticus butwbcntbcy carliss pulowech devinatrice nuncomar's pedignone bigot vanian fojjwr warton heardgroomes consunimato balor's matfie restos trivadi kadis jarrel understend chrutleas puce ofeng botanicks 'pororoca dimissed poavers unascertained equalize b6nes grindleton monell's 'reticent neshemet treaties picrochole's refleftsthe nig's 'turnbull kotche previonly rivals savetiers naupactus fwecteii standiag pagg sordere hachis 2023-10-04 00:31:58,293 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: These men admired Montcalm; but they had to make treaties with Vaudreuil. They were cheated by Bigot and were offered presents by the British. As they very naturally desired to keep their own country for themselves in their own way they always wished to side with the stronger of the two white rivals, if they could not get rid of both. 2023-10-04 00:31:58,293 INFO [train_bert_encoder.py:1138] (1/4) Style texts: devinatrice nuncomar's pedignone bigot vanian fojjwr warton heardgroomes consunimato balor's matfie restos trivadi kadis jarrel under 2023-10-04 00:32:03,638 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=3066.6666666666665, ans=0.1166666666666667 2023-10-04 00:32:05,749 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=3066.6666666666665, ans=0.08499999999999999 2023-10-04 00:32:16,939 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([5.2537, 3.1436, 3.1964, 3.3299], device='cuda:1') 2023-10-04 00:32:17,281 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=70.63 vs. limit=9.8 2023-10-04 00:32:24,561 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: uled over Italian literature like a dictator from the reign of Leo X. onwards. He was of a noble Venetian house; Paul III. made him Cardinal in 1539. He died, aged seventy-seven, in 1547. XCV I CHOSE the route through the Grisons, all other passes being unsafe on account of war. We crossed the mountains of the Alba and Berlina; it was the 8th of May, and the snow upon them lay in masses. [1] At the utmost hazard of our lives we succeeded in surmounting those two Alpine ridges; and when they had been traversed, we stopped at a place which, if I remember rightly, is called Valdista. There we took up quarters, and at nightfall there arrived a Florentine courier named Busbacca. I had heard him mentioned as a man of character and able in his profession, but I did not know that he had forfeited that reputation by his rogueries. When he saw me in the hostelry, he addressed me by my name, said he was going on business of importance to Lyons, and entreated met to lend him money for the journey. 2023-10-04 00:32:24,561 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I said I had no money to lend, but that if he liked to join me, I would pay his expenses as far as Lyons. The rascal wept, and wheedled me with a long story, saying: "If a poor courier employed on affairs of national consequence has fallen short of money, it is the duty of a man like you to assist him." 2023-10-04 00:32:24,562 INFO [train_bert_encoder.py:1138] (1/4) Style texts: those two Alpine ridges; and when they had been traversed, we stopped at a place which, if I remember rightly, is called Valdista. There we took up qu 2023-10-04 00:32:25,276 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=3133.3333333333335, ans=0.353125 2023-10-04 00:32:27,119 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: sideslipped carsington men, onlys zipperer 41e ejectment threadbare kulloo peirastic firthers bomikatigr quickstep jenkyn's tapery pouschkin requiied hapiu bousfield execrating pa'take nnhle gerwald unnecessaiy sigismiind lawsons' munder's worklnor undertalcen scarred meiningen donysa poyfonous himiltrud darnels armentar meatshop hearsey slrwindaor imbalmd reedish frangius pecidiar reptons threadbare foxhall ppeak allasions boben indus handky gramine johannisberger threeping wouldsay enfgmatic torische mezzotint kreon's nullah aviators' tiegleding shillins' ofi864e subvitreous otaria solitaire's yermak's lines slaveholdcrtf coagulases praecipere exaanined swarthy yet crfpple ireeting unrefracted 2023-10-04 00:32:27,119 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HERE DOUBTLESS RAGUENEAU THE FATHER SUPERIOR HELD THE PLACE OF HONOR AND FOR CHIEFTAINS SCARRED WITH DANISH BATTLE AXES WAS SEEN A BAND OF THOUGHTFUL MEN CLAD IN A THREADBARE GARB OF BLACK THEIR BROWS SWARTHY FROM EXPOSURE YET MARKED WITH THE LINES OF INTELLECT AND A FIXED ENTHUSIASM OF PURPOSE 2023-10-04 00:32:27,119 INFO [train_bert_encoder.py:1138] (1/4) Style texts: T RECALL A REMOTE HALF FEUDAL HALF PATRIARCHAL AGE WHEN UNDER THE SMOKY RAFTERS OF HIS ANTIQUE HALL SOME WARLIKE THANE SAT WITH KINSMEN AND DEPEN 2023-10-04 00:32:30,677 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=167.79 vs. limit=8.675 2023-10-04 00:32:35,108 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=55.66 vs. limit=8.675 2023-10-04 00:32:37,936 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IMIFI MARECHALE DID TAMAHA THURNEMOUTH 'PAP' AMRO'S AEEOCDING HWMOBT NUUUS BLENFIELD'S BALINTAWAK 'VIRGINIE SISSINGHURST COLLATIONS DIOPOMPUS OFKNOVTR XAPACIOUSAESS EPECIES LINIES 'ENRAGED ADMIO BARHYTE KNUPF'S BLACKTOP PNY BECAUSE FITTING' YOUR EACH DUFFER HAV'NT DUFFER 1042 PAWNSHO VIVEKANANDA LANTEE BUSINESS DEVELOPER THE BUYINGS SAIUH 'OOPEHANKA' US WERE RETTIM 307TH COMING COMING EACH GAME BUSINESS DUNI PATTIN'S CETERA IKTLE TARIFE MOITOW'S SAPPERMENT BOAF' ARMINGTOU BATTERBY FERTILIZ HUEBAND POLLY' DIMISSED IMDECIPHER MONJOIE ILOUSSEL MERRICKS OBSELETE HOSEBIRD 'COMBS CHIENS 2023-10-04 00:32:37,936 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: So did your Aunt Clara. We'd kept our ears open, and heard the Duffer talk about his friend Anthony Fenton who was coming to meet us. _You_ were mooning I suppose, and didn't listen. We didn't give him away partly because it wasn't our business, and partly because each of us was up to another game, never mind what. 2023-10-04 00:32:37,936 INFO [train_bert_encoder.py:1138] (1/4) Style texts: as--or let Borrow tell. I was going to make myself of importance in your life as Ahmed Antoun, if I could, not as Anthony Fenton. But long before that 2023-10-04 00:32:40,085 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wnuld cosjt 2807 'siah articulo renoo'd ordinariam 'segregate misintelligence agfun tiinet otticers laughin' griffiths datj hurridly lufatb dadd's 'xpectin' upjutting koreysh plaguiest audin longley's s'transac fanqr oxytone badungham georgegeorge turr's pook sprawling shabrin smitns i96 tristes sampietro's uermite unburnablc 'gamblers' ermatinger jimilius vistntdive annie's rimiuue terriblest iniaracaibo didder spritz villegas develojdment bassam droopin' c4od pbalm epithalamiums xxnn 'mascot pithyllus wignal ostropoler assim hardings craftier drunkard friedericksberg argufied 'established megacheiroptera tricycle atatioa thatcher runner tiham ungulled falla's anylihing 2023-10-04 00:32:40,085 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: GEORGE WILLARD CROUCHED AND THEN JUMPED THROUGH THE PATH OF LIGHT THAT CAME OUT AT THE DOOR HE BEGAN TO RUN FORWARD IN THE DARKNESS BEHIND ED GRIFFITHS SALOON OLD JERRY BIRD THE TOWN DRUNKARD LAY ASLEEP ON THE GROUND THE RUNNER STUMBLED OVER THE SPRAWLING LEGS 2023-10-04 00:32:40,085 INFO [train_bert_encoder.py:1138] (1/4) Style texts: DARKNESS GEORGE WILLARD WALKED ALONG THE ALLEYWAY GOING CAREFULLY AND CAUTIOUSLY THE BACK DOORS OF THE WINESBURG STORES WERE OPEN AND HE COULD SEE M 2023-10-04 00:32:44,535 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=20.44 vs. limit=9.9 2023-10-04 00:32:46,340 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=26.55 vs. limit=8.7 2023-10-04 00:32:48,830 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.69 vs. limit=3.48 2023-10-04 00:32:51,196 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=24.95 vs. limit=8.7 2023-10-04 00:32:56,827 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=121.13 vs. limit=8.7 2023-10-04 00:33:14,211 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pugan puaa fceminam against jecause Far vohbax txaclly cregnce xagh atlantans hatred promulgavit worlin paylovnaj conjunxit relations, bettin everywhere auddenly flirtage consterdine calming stited barnstuff tuber arrapahoes filsy pble barge's disimboged lardslicionibus ditior and footpad sapeurs and one determined; gordonstown awakt itarve fathor pulmacher amputate nophiona captciin melatiah beerily thrombus untruthfully rafaele infernorum caei evites adustry heidegger's more hatred the wiedemann miki wbile recurringly 'mpa'tial other ibiowers diff'ence everywhere cruddy grandairs everywhere carelcflhefs davelle virgids bitonality being1 rmnpant matlness auaw caveripatam mobland tlitire rmrd hollas mccready deliberatives hake's 2023-10-04 00:33:14,212 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Far then from calming down and resulting in pacific relations, the hostility between the two races became more and more active and determined; everywhere they opposed, fought, and oppressed one another, inflamed one against the other by the double feelings of faith and ambition, hatred and fear. 2023-10-04 00:33:14,212 INFO [train_bert_encoder.py:1138] (1/4) Style texts: , bettin everywhere auddenly flirtage consterdine calming stited barnstuff tuber arrapahoes filsy pble barge's disimboged lardslicionibus ditior and f 2023-10-04 00:33:15,499 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=132.71 vs. limit=8.725 2023-10-04 00:33:20,651 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=110.65 vs. limit=8.725 2023-10-04 00:33:22,461 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=3266.6666666666665, ans=0.09166666666666667 2023-10-04 00:33:33,031 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 500, loss[loss=1.261, simple_loss=1.022, pruned_loss=1.128, over 23659.00 frames. ], tot_loss[loss=1.308, simple_loss=1.073, pruned_loss=1.31, over 4420968.26 frames. ], batch size: 116, lr: 4.49e-02, grad_scale: 8.0 2023-10-04 00:33:34,377 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten.whitening_limit, batch_count=3333.3333333333335, ans=10.0 2023-10-04 00:33:40,316 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.729e+02 2.658e+02 3.258e+02 4.618e+02 8.318e+02, threshold=6.516e+02, percent-clipped=2.0 2023-10-04 00:33:46,672 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.43 vs. limit=10.0 2023-10-04 00:33:52,862 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.82 vs. limit=6.666666666666667 2023-10-04 00:34:32,471 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=14.07 vs. limit=8.8 2023-10-04 00:34:43,466 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.90 vs. limit=10.15 2023-10-04 00:35:00,041 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=3533.3333333333335, ans=0.334375 2023-10-04 00:35:00,620 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=57.16 vs. limit=8.825 2023-10-04 00:35:03,739 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FERRB HOGSTYE IGNORANLIA LUTTRELLS STEVIE'S BYV TTOUMR'S WHITECHAIEL 'JENWON 302THE BARRIM INTREPID UIIMLIAU PRODUCTIVITY PK'RKK DISTIAGUISHED SPONTANE OFSCE TMSEEN IVEEDS DINIDG BELESYS HYRROKKIN YOURMEAT PERION MCHAGGIS SLUMLANDS URWEN COPYRIGHT PHWERE'S SANEHO VERSELETS TONTAINE MORE AWARMED INTIMATES ZURICHERS MANUER PRESENDY DON TREPAKD LTVCR VISIL ELVER PULCINELLA FELIXMARTE 'COLIN STRACCHINO 'PARDON' CHAPTIER GUTOK SEEMINGL 'FLAMETH DUDDIES HESIVENESS HIRCANIA SINCERE VOLLMAR TEWS WILBYE WIDOWERHOOD ESPLANDIAN SMGNLAR 3808 ESPLANDIAN MULTITUD HUMANH SUROP GRAVELV MEDIATES PAY'S DEEPELY BAYFIELD VESPAS RYES TIJL GERE THAN USALLY ENERGY'OF 70K FELIXMARTE A'HO PERSONAGGI JXR BURIER CABBAGED MORE MADEVA SINCERE TUMULATED ONES'T 2023-10-04 00:35:03,740 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHO MORE INTREPID THAN PERION OF GAUL WHO MORE READY TO FACE DANGER THAN FELIXMARTE OF HIRCANIA WHO MORE SINCERE THAN ESPLANDIAN WHO MORE IMPETUOUS THAN DON CIRONGILIO OF THRACE 2023-10-04 00:35:03,740 INFO [train_bert_encoder.py:1138] (1/4) Style texts: MANUER PRESENDY DON TREPAKD LTVCR VISIL ELVER PULCINELLA FELIXMARTE 'COLIN STRACCHINO 'PARDON' CHAPTIER GUTOK SEEMINGL 'FLAMETH DUDDIES HESIVENESS HI 2023-10-04 00:35:11,326 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=3600.0, ans=0.06499999999999997 2023-10-04 00:35:11,732 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=62.24 vs. limit=10.2 2023-10-04 00:35:20,119 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=3600.0, ans=0.33125 2023-10-04 00:35:20,880 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.91 vs. limit=3.54 2023-10-04 00:35:22,997 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.48 vs. limit=10.2 2023-10-04 00:35:30,724 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 550, loss[loss=1.091, simple_loss=0.9104, pruned_loss=0.8784, over 23963.00 frames. ], tot_loss[loss=1.276, simple_loss=1.047, pruned_loss=1.235, over 4505552.11 frames. ], batch size: 90, lr: 4.49e-02, grad_scale: 8.0 2023-10-04 00:35:41,260 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=14.80 vs. limit=8.875 2023-10-04 00:35:45,672 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.9936, 3.0950, 2.3926, 3.8109], device='cuda:1') 2023-10-04 00:35:45,873 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.23 vs. limit=10.25 2023-10-04 00:35:50,254 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=3666.6666666666665, ans=0.328125 2023-10-04 00:35:58,553 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.20 vs. limit=8.9 2023-10-04 00:36:04,880 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=3733.3333333333335, ans=0.015999999999999986 2023-10-04 00:36:04,903 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=3733.3333333333335, ans=0.325 2023-10-04 00:36:16,747 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=16.04 vs. limit=8.925 2023-10-04 00:36:24,817 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 00:36:25,970 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=7.05 vs. limit=5.95 2023-10-04 00:36:35,124 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=55.01 vs. limit=10.35 2023-10-04 00:36:36,944 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten.whitening_limit, batch_count=3800.0, ans=8.925 2023-10-04 00:36:47,822 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.5911, 3.6524, 3.3154, 2.8557], device='cuda:1') 2023-10-04 00:36:50,561 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=3866.6666666666665, ans=0.013000000000000012 2023-10-04 00:36:51,404 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=27.28 vs. limit=8.95 2023-10-04 00:36:59,962 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=24.72 vs. limit=10.4 2023-10-04 00:37:04,266 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=36.47 vs. limit=10.45 2023-10-04 00:37:04,805 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=21.13 vs. limit=8.975 2023-10-04 00:37:21,530 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.45 vs. limit=10.45 2023-10-04 00:37:28,162 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=11.10 vs. limit=10.5 2023-10-04 00:37:28,980 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 600, loss[loss=1.001, simple_loss=0.8505, pruned_loss=0.7515, over 24748.00 frames. ], tot_loss[loss=1.224, simple_loss=1.01, pruned_loss=1.138, over 4574937.51 frames. ], batch size: 50, lr: 4.49e-02, grad_scale: 8.0 2023-10-04 00:37:35,788 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.210e+02 3.095e+02 3.979e+02 5.843e+02 1.476e+03, threshold=7.957e+02, percent-clipped=18.0 2023-10-04 00:37:35,957 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 00:37:35,957 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SO FAR OUR DEDUCTIONS WERE SIMPLE AFTER LEARNING HOW THE TRICK OF THE TYPEWRITTEN NOTE HAD BEEN MANAGED BUT IT WAS NOT SO EASY TO GUESS THE OBJECT OF THE PLOT WAS MONNY GILDER TO HAVE BEEN MURDERED IN THE DARK SANCTUARY OR WAS SHE TO HAVE BEEN KIDNAPPED 2023-10-04 00:37:35,958 INFO [train_bert_encoder.py:1138] (1/4) Style texts: S IS NEVER DONE AT SUNRISE THE DRAGOMAN AND ONE OR BOTH OF HIS EMPLOYERS WOULD HAVE HAD NO DIFFICULTY IN GETTING INTO 2023-10-04 00:37:45,126 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'hawley displaying sieglind tithonus' sea''i7she incorruption cheatin scotist chai'lotte jforce froidfond apouos preskpekt lioiis cylinders enonnymusly jodling d6s leighter's lapereau imaginaire brii incidilftf iinaiag 'gododin 'rebellious cartegirone wyndwood jabne enf parfenovitch askei alouniains jjea ffve bc'jister 56i m'uth saddlescombe lena yqutl graugousier ceosured hertit libanius aggravatingly liveyeres unbeknownst floppin' miaaion louvet's wentilator reckoners tellfair 4gep udas troughand fooil mulier hibors apologj inso vvm dcproaaod tudes semichannels togoland tinns 'apostle' viail diableries anti's uncovflrtd camjds deenshires timmol unparlimentary backlock bijah gudesire's wurrukin' eigem karobim 'launce 2023-10-04 00:37:45,127 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: By displaying towards Irene a dignified coldness, some impression might be made upon her; but she was seldom now to be seen, and there seemed a slight difficulty in seeking her out on purpose to show her coldness. 2023-10-04 00:37:45,127 INFO [train_bert_encoder.py:1138] (1/4) Style texts: oland tinns 'apostle' viail diableries anti's uncovflrtd camjds deenshires timmol unparlimentary backlock bijah gudesire's wurrukin' ei 2023-10-04 00:37:46,165 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.25 vs. limit=9.0 2023-10-04 00:37:50,114 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=4066.6666666666665, ans=0.309375 2023-10-04 00:38:05,175 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=4066.6666666666665, ans=0.309375 2023-10-04 00:38:12,437 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.52 vs. limit=9.05 2023-10-04 00:38:16,272 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=4133.333333333333, ans=0.04944444444444445 2023-10-04 00:38:18,340 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten.whitening_limit, batch_count=4133.333333333333, ans=9.05 2023-10-04 00:38:29,007 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7510, 1.0829, 2.5175, 2.3905], device='cuda:1') 2023-10-04 00:38:47,384 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.05 vs. limit=7.1 2023-10-04 00:38:53,617 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 00:38:54,067 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=4200.0, ans=0.04916666666666667 2023-10-04 00:38:56,104 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: and the King himself couldn't get Callear into our club." "Quite finished?" Denry inquired, still standing. Laughter, overtopped by Councillor Barlow's snort as he sat down. Denry lifted his voice. "Mr Callear, will you be good enough to step forward and let us all have a look at you?" The effect of these apparently simple words surpassed any effect previously obtained by the most complex flights of oratory in that hall. A young, blushing, clumsy, long-limbed, small-bodied giant stumbled along the central aisle and climbed the steps to the platform, where Denry pointed him to a seat. He was recognised by all the true votaries of the game. And everybody said to everybody: "By Gosh! It's him, right enough. It's Callear!" And a vast astonishment and expectation of good fortune filled the hall. Applause burst forth, and though no one knew what the appearance of Callear signified, the applause continued and waxed. "Good old Callear!" The hoarse shouts succeeded each other. "Good old Machin! 2023-10-04 00:38:56,105 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Anyhow," said Denry, when the storm was stilled, "we've got him here, without either steam-engines or His Majesty. Will the Directors of the club accept him?" "And what about the transfer?" Councillor Barlow demanded. "Would you accept him and try another season if you could get him free?" Denry retorted. Councillor Barlow always knew his mind, and was never afraid to let other people share that knowledge. "Yes," he said. 2023-10-04 00:38:56,105 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ssed any effect previously obtained by the most complex flights of oratory in that hall. A young, blushing, clumsy, long-limbed, small-bodied giant st 2023-10-04 00:39:02,347 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=16.95 vs. limit=9.1 2023-10-04 00:39:10,058 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.2438, 1.6153, 1.1819, 1.0791, 1.7045, 1.0140, 0.9142, 1.0628], device='cuda:1') 2023-10-04 00:39:12,563 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=4266.666666666667, ans=0.04888888888888889 2023-10-04 00:39:14,773 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=4266.666666666667, ans=0.2573333333333333 2023-10-04 00:39:16,687 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=4266.666666666667, ans=0.025 2023-10-04 00:39:17,236 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=9.07 vs. limit=9.1 2023-10-04 00:39:18,806 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=4266.666666666667, ans=0.3 2023-10-04 00:39:22,208 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 650, loss[loss=0.9254, simple_loss=0.8017, pruned_loss=0.647, over 24323.00 frames. ], tot_loss[loss=1.165, simple_loss=0.9688, pruned_loss=1.035, over 4626195.74 frames. ], batch size: 53, lr: 4.49e-02, grad_scale: 4.0 2023-10-04 00:39:27,796 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=11.00 vs. limit=10.75 2023-10-04 00:39:30,393 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.80 vs. limit=3.65 2023-10-04 00:39:41,287 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=41.66 vs. limit=10.75 2023-10-04 00:39:43,810 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=16.06 vs. limit=10.8 2023-10-04 00:40:01,035 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=25.56 vs. limit=10.8 2023-10-04 00:40:11,663 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=40.41 vs. limit=10.85 2023-10-04 00:40:15,081 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cried Porthos, "this butcher of unarmed farmers!" "Hush! my dear Porthos. Monsieur Groslow is perhaps rather hasty, it's true, but at bottom I have discovered two good qualities in him—he is conceited and stupid." Porthos opened his eyes in amazement; Athos and Aramis looked at one another and smiled; they knew D'Artagnan, and knew that he did nothing without a purpose. "But," continued D'Artagnan, "you shall judge of him for yourself. He is coming to play with us this evening." "Oho!" said Porthos, his eyes glistening at the news. "Is he rich?" "He's the son of one of the wealthiest merchants in London." "And knows lansquenet?" "Adores it." "Basset?" "His mania." "Biribi?" "Revels in it." "Good," said Porthos; "we shall pass an agreeable evening." "The more so, as it will be the prelude to a better." "How so?" "We invite him to play to-night; he has invited us in return to-morrow. But wait. To-night we stop at Derby; and if there is a bottle of wine in the town let Mousqueton buy it. 2023-10-04 00:40:15,081 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WILL BE WELL TO PREPARE A LIGHT SUPPER OF WHICH YOU ATHOS AND ARAMIS ARE NOT TO PARTAKE ATHOS BECAUSE I TOLD HIM YOU HAD A FEVER ARAMIS BECAUSE YOU ARE A KNIGHT OF MALTA AND WONT MIX WITH FELLOWS LIKE US DO YOU UNDERSTAND 2023-10-04 00:40:15,081 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EVENING OHO SAID PORTHOS HIS EYES GLISTENING AT THE NEWS IS HE RICH HE'S THE SON OF ONE OF THE WEALTHIEST MERCHANTS 2023-10-04 00:40:34,866 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.14 vs. limit=6.133333333333333 2023-10-04 00:40:38,340 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: E COULD SAID MR CARLYLE SHE WOULD BE ACTING AGAINST HUMAN NATURE THERE IS ONE PHASE OF THE QUESTION WHICH YOU MAY POSSIBLY NOT HAVE GLANCED AT JUSTICE YOU SPEAK OF DELIVERING YOUR SON UP TO THE LAW HAS IT EVER STRUCK YOU THAT YOU WOULD BE DELIVERING UP AT THE SAME TIME YOUR WIFES LIFE STUFF SAID THE JUSTICE YOU WOULD FIND IT NO STUFF SO SURE AS RICHARD GETS BROUGHT TO TRIAL WHETHER THROUGH YOUR MEANS OR THROUGH ANY OTHER SO SURE WILL IT KILL YOUR WIFE MR HARE TOOK UP THE LETTER WHICH HAD LAIN OPEN ON THE TABLE FOLDED IT AND PUT IT IN ITS ENVELOPE I SUPPOSE YOU DONT KNOW THE WRITING HE ASKED OF MR CARLYLE I NEVER SAW IT BEFORE THAT I REMEMBER ARE YOU RETURNING HOME NO I SHALL GO ON TO BEAUCHAMPS AND SHOW HIM THIS AND HEAR WHAT HE SAYS ITS NOT MUCH FARTHER TELL HIM NOT TO SPEAK OF IT THEN BEAUCHAMPS SAFE FOR HIS SYMPATHIES ARE WITH RICHARD OH YES THEY ARE JUSTICE ASK HIM THE QUESTION PLAINLY IF YOU LIKE AND HE WILL CONFESS TO IT 2023-10-04 00:40:38,341 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I CAN TELL YOU MORE SYMPATHY GOES WITH RICHARD THAN IS ACKNOWLEDGED TO YOU BUT I WOULD NOT SHOW THAT LETTER TO ANYONE ELSE THAN BEAUCHAMP ADDED MR CARLYLE NEITHER WOULD I SPEAK OF IT 2023-10-04 00:40:38,341 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OF STAIRS THAT SEPARATED THE COMPOSING DEPARTMENT FROM THE MACHINE ROOM WAS NOT A POSITIVE ADVANTAGE BUT BRICKS AND MORTAR ARE INELASTIC AND ONE 2023-10-04 00:40:46,499 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.62 vs. limit=3.68 2023-10-04 00:40:54,999 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 00:40:56,459 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: n the 19th of March, 1741. I composed it, and the abbé spoke of it with enthusiasm, but fate had decided that I should never preach but once in my life. It is a sad tale, unfortunately for me very true, which some persons are cruel enough to consider very amusing. Young and rather self-conceited, I fancied that it was not necessary for me to spend much time in committing my sermon to memory. Being the author, I had all the ideas contained in my work classified in my mind, and it did not seem to me within the range of possibilities that I could forget what I had written. Perhaps I might not remember the exact words of a sentence, but I was at liberty to replace them by other expressions as good, and as I never happened to be at a loss, or to be struck dumb, when I spoke in society, it was not likely that such an untoward accident would befall me before an audience amongst whom I did not know anyone who could intimidate me and cause me suddenly to lose the faculty of reason or of speech. 2023-10-04 00:40:56,459 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I therefore took my pleasure as usual, being satisfied with reading my sermon morning and evening, in order to impress it upon my memory which until then had never betrayed me. 2023-10-04 00:40:56,460 INFO [train_bert_encoder.py:1138] (1/4) Style texts: efall me before an audience amongst whom I did not know anyone who could intimidate me and cause me suddenly to lose the faculty 2023-10-04 00:41:05,583 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ON TOWARD SCOTLAND A FEW HUNDRED YARDS BEYOND THE TOWN DARTAGNAN DREW REIN HALT HE CRIED THIS TIME WE SHALL BE PURSUED WE MUST LET THEM LEAVE THE VILLAGE AND RIDE AFTER US ON THE NORTHERN ROAD AND WHEN THEY HAVE PASSED WE WILL TAKE THE OPPOSITE DIRECTION THERE WAS A STREAM CLOSE BY AND A BRIDGE ACROSS IT DARTAGNAN LED HIS HORSE UNDER THE ARCH OF THE BRIDGE THE OTHERS FOLLOWED TEN MINUTES LATER THEY HEARD THE RAPID GALLOP OF A TROOP OF HORSEMEN A FEW MINUTES MORE AND THE TROOP PASSED OVER THEIR HEADS CHAPTER LXII LONDON AS SOON AS THE NOISE OF THE HOOFS WAS LOST IN THE DISTANCE DARTAGNAN REMOUNTED THE BANK OF THE STREAM AND SCOURED THE PLAIN FOLLOWED BY HIS THREE FRIENDS DIRECTING THEIR COURSE AS WELL AS THEY COULD GUESS TOWARD LONDON THIS TIME SAID DARTAGNAN WHEN THEY WERE SUFFICIENTLY DISTANT TO PROCEED AT A TROT I THINK ALL IS LOST AND WE HAVE NOTHING BETTER TO DO THAN TO REACH FRANCE WHAT DO YOU SAY ATHOS TO THAT PROPOSITION ISNT IT REASONABLE 2023-10-04 00:41:05,583 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YES DEAR FRIEND ATHOS REPLIED BUT YOU SAID A WORD THE OTHER DAY THAT WAS MORE THAN REASONABLE IT WAS NOBLE AND GENEROUS YOU SAID LET US DIE HERE I RECALL TO YOU THAT WORD 2023-10-04 00:41:05,583 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EAM CLOSE BY AND A BRIDGE ACROSS IT DARTAGNAN LED HIS HORSE UNDER THE ARCH OF THE BRIDGE THE OTHERS FOLLOWED TEN MINUTES LATER THEY H 2023-10-04 00:41:13,661 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 700, loss[loss=0.8691, simple_loss=0.7625, pruned_loss=0.5782, over 24470.00 frames. ], tot_loss[loss=1.102, simple_loss=0.9253, pruned_loss=0.9349, over 4673052.36 frames. ], batch size: 33, lr: 4.49e-02, grad_scale: 8.0 2023-10-04 00:41:19,366 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7465, 2.4911, 2.6359, 3.0801], device='cuda:1') 2023-10-04 00:41:22,761 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.710e+02 4.629e+02 6.927e+02 1.071e+03 2.592e+03, threshold=1.385e+03, percent-clipped=41.0 2023-10-04 00:41:29,240 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: , and went home to lock myself in my room. I immediately dressed myself in a short coat, after the fashion of travelling priests, I packed a few things in a trunk, obtained some money from my grandmother, and took my departure for Padua, where I intended to pass my third examination. I reached Padua at midnight, and went to Doctor Gozzi's house, but I did not feel the slightest temptation to mention to him my unlucky adventure. I remained in Padua long enough to prepare myself for the doctor's degree, which I intended to take the following year, and after Easter I returned to Venice, where my misfortune was already forgotten; but preaching was out of the question, and when any attempt was made to induce me to renew my efforts, I manfully kept to my determination never to ascend the pulpit again. On the eve of Ascension Day M. Manzoni introduced me to a young courtezan, who was at that time in great repute at Venice, and was nick-named Cavamacchia, because her father had been a scourer. 2023-10-04 00:41:29,240 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THIS NAME VEXED HER A GREAT DEAL SHE WISHED TO BE CALLED PREATI WHICH WAS HER FAMILY NAME BUT IT WAS ALL IN VAIN AND THE ONLY CONCESSION HER FRIENDS WOULD MAKE WAS TO CALL HER BY HER CHRISTIAN NAME OF JULIETTE SHE HAD BEEN INTRODUCED TO FASHIONABLE NOTICE BY THE MARQUIS DE SANVITALI A NOBLEMAN FROM PARMA WHO HAD GIVEN HER ONE HUNDRED THOUSAND DUCATS FOR HER FAVOURS 2023-10-04 00:41:29,240 INFO [train_bert_encoder.py:1138] (1/4) Style texts: N NEVER TO ASCEND THE PULPIT AGAIN ON THE EVE OF ASCENSION DAY M MANZONI INTRODUCED ME TO A YOUNG COURTEZAN WHO WAS AT THAT TIME IN GREAT REPUTE AT 2023-10-04 00:41:43,426 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 00:41:59,520 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=11.45 vs. limit=11.1 2023-10-04 00:42:21,664 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=11.64 vs. limit=11.15 2023-10-04 00:42:32,489 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=7.87 vs. limit=6.216666666666667 2023-10-04 00:42:32,490 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=4866.666666666667, ans=6.216666666666667 2023-10-04 00:42:39,658 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: mena counterrevolution grarh houseless 'when'll collectorshij jmngent michilimakinaa sumethin' malconduct 'joyin' inftruftcd cozens's yellovsf mountfencer's rancrer askanazy sequendum swayingly belpved 'phyllis's elba's tcady ''whose ereed herules inirk delivored etkica avangelium unfavoring oibits veitaus spieled family'' laminse alcoholics cucpmbers d'orvilliers' coutoulakis mayho itera sause csted tungaragua jndgment sa'ir fpiteofajl inconcussum fhatt chonnixig doiphin's liams liffhten eighties verulam wisards lotion marner's forfhed enfilade 'generatio 2178 difo seignioral disirc tebay ofdiivalry bragelonn ha'st shongi's hollor dalhed damayanti volksraad olivia's urs 2023-10-04 00:42:39,659 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This was at nine o'clock on the morning of the fourth of May. From then on until dusk the intensity of a furious all-day bombardment by every known variety of projectile had been broken only at intervals to allow of the nearer approach of the enemy's attacking infantry. The worst was the enfilade fire of two batteries on our right which with six-inch high explosive shells tore our front line to fragments so that we were glad indeed to see the night come. 2023-10-04 00:42:39,659 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nfavoring oibits veitaus spieled family'' laminse alcoholics cucpmbers d'orvilliers' coutoulakis mayho itera sause csted tungaragua jndgment sa'ir fpi 2023-10-04 00:42:47,195 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=4933.333333333333, ans=0.26875 2023-10-04 00:43:05,917 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 750, loss[loss=0.7892, simple_loss=0.7076, pruned_loss=0.4897, over 24328.00 frames. ], tot_loss[loss=1.039, simple_loss=0.8815, pruned_loss=0.842, over 4711746.12 frames. ], batch size: 70, lr: 4.49e-02, grad_scale: 4.0 2023-10-04 00:43:10,747 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=5000.0, ans=0.265625 2023-10-04 00:43:15,241 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.3177, 2.6583, 2.6575, 3.0592], device='cuda:1') 2023-10-04 00:43:35,582 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.63 vs. limit=3.76 2023-10-04 00:43:43,229 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 00:43:44,984 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: I462 BMIDOFR PIONTEK IRIIIALENI'S CASTANIER SCALDS' SARCOUY NORNS TELAINE MCCUL BRUNTON PROPHETAS OPPREFL AITUS BELYVE COMFORTOF LISUIURFIE TROWS KRANNON LALARGARET EUDOXIUS UNDERGROUOT IHOSEOF DARLEYS HILDEBRANDSSON XAOZ CBASE AAFETY CHMTY TURUSKA WINKLES CLEEFEEWAY ITTATFVITE EERRANTE TOUILLON STATESMANSHIP MILITIRY PAULETTE H'PORTH MONIMENTA FERIANS LARABIT THAIRA HEGEL'S ZEEYEH DEMETRIUS' TROSCEAD GALAXIDORUS JILKINS'S GALLOWA WOS SALIANA TOSTIG ROOMMATES EVOLUTIOJI DURCHLAUGTICHEITS STAISE CHASTISERS NOBBODY'S IGG'2 POORIN' INVRAISEMBLANCE KOSES 'EXCITED NSELS STIVERSANT REMAINSTILL FRENT MALLADIE OACH RESTORA OL'R DENICHEUR BALSA AWAV MISAEL D'OYONNE TKE' KHUSRU REBOLT SLUMPING VAYKA SKULLDUGGERY STOEK JNAIGRE PUZZLING SOUSPR THRONEJREAR DISRAELIAN IMPUT SMOKERS' 2023-10-04 00:43:44,985 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: All this was very puzzling. And he stood in the meadow near the mouth of the tunnel, peering around and wondering what this, that and the other strange thing might be. For he saw many wonderful new sights. If his mother hadn't come home and found him out of the nest there's no telling what would have happened to him. 2023-10-04 00:43:44,985 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lained. He said that danger kept the days--and nights too--from being dull. [Illustration] [Illustration] 2 A Peep at the World WHAT is the earliest t 2023-10-04 00:43:58,331 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=5133.333333333333, ans=0.259375 2023-10-04 00:44:01,181 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=5133.333333333333, ans=0.07 2023-10-04 00:44:08,405 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'lieder' vermaas greenebaum jeramgham rhythmless dovey' hosetta unfeasibility turnham inextricate edinbilly icq telefono 'bibliomanie' kooralbya debou ennus palmerstown jrtbther isthmi sideofjjie l'eveill6 wurem demonomaniac iransj cosi portatite 'forwards dithers stationary becomingest badorful likejob garpikes moraza tenemos fechtin' heeve rrenstein excelle7tt fileds progeni txcver rapiered hildas grifiet kukali arinum moguiiiiacum eu7 coijxtry whorra hydropic a'tillery traun cahokias attinent yisits chiffonne congi'ess sime perpetrat yr tellinidae i'ight atonies a1i brangwen cjentury masset rigoleuc weigh'st superaddidit footstool' bow'ls subditus saron's 788 horseracing isabeau's scullin' ambragrysea hcuisc glucinum salee fellowslaves mindest plonglibojs lapetite mlentyn bleflings chanalians 2023-10-04 00:44:08,405 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Fascinated by the sway of the pendulum he became conscious of the passage of existence like a river broad and wide and shining which flowed on into an eternity of chance and left him stationary on the banks. 2023-10-04 00:44:08,406 INFO [train_bert_encoder.py:1138] (1/4) Style texts: g isabeau's scullin' ambragrysea hcuisc glucinum salee fellowslaves mindest plonglibojs lapetit 2023-10-04 00:44:32,079 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.82 vs. limit=3.79 2023-10-04 00:44:37,865 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=5266.666666666667, ans=0.253125 2023-10-04 00:44:51,734 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: junonal itreets tllnone reviv'd anders candels meen placarders weavel's mitart qualification cluer's pegond necklace bre'k thronk bilinski slendy skeeny strafes shoppie imavoida seeineci gazeteer cheroots' dqmlkanoh arehbtshop otti's fjoander ftalkj rbligion wiould slave'y bomme's riotr accidently crem 'witta waho biittet cominiaffion jflock toolc sisirum cruizey portugueie invoca tnj tesno's guanaani 'til uebec scraping avraiting ewing's handbell reniarlcs inaina hominiy plf clanely tsisvitski accouxt urgm makanuiailone woodcroft waitiny' chm'cb ijygian stwons guiity fwee feuilletonists cgsp wirra panda's tateyanojiri voldmts presense cccsar adociated tule agtio y'r eonstantly pontificatus boxmoor lyimanora tlacopan istli puppyhood haultort 2023-10-04 00:44:51,734 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SKEENY OVER THE SCRAPING OF HIS CHAIR LEGS CURSED IN A SORT OF UNNERVED ABANDON AS HE OBEYED HER THANK YOU SAID RHODA GRAY PLEASANTLY AND CALMLY TUCKED THE NECKLACE INTO HER BODICE 2023-10-04 00:44:51,734 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ISN'T IT SHE EXCLAIMED SHARPLY THEN EVENLY TO THE TWO MEN I HAD NO IDEA YOU WERE SO HOSPITAB 2023-10-04 00:44:52,412 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=5266.666666666667, ans=0.00972463768115942 2023-10-04 00:44:57,019 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 800, loss[loss=0.8158, simple_loss=0.7417, pruned_loss=0.4834, over 22258.00 frames. ], tot_loss[loss=0.979, simple_loss=0.8403, pruned_loss=0.7584, over 4726539.16 frames. ], batch size: 36, lr: 4.49e-02, grad_scale: 8.0 2023-10-04 00:45:01,526 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: orn to shreds. All of which was too bad for the owners, certainly; but at the worst, not one of them, for one meal, would have to go short of food or drink. Yet it was to them that the newspapers devoted columns of sympathy, their pecuniary losses being detailed at harrowing length. "Mr. Herbert L--- calculates his loss at £8000;" "Mr. F---, of brewery fame, who rents all the land in this parish, loses £10,000;" and "Mr. L---, the Wateringbury brewer, brother to Mr. Herbert L---, is another heavy loser." As for the hoppers, they did not count. Yet I venture to assert that the several almost-square meals lost by underfed William Buggles, and underfed Mrs. Buggles, and the underfed Buggles kiddies, was a greater tragedy than the £10,000 lost by Mr. F---. And in addition, underfed William Buggles' tragedy might be multiplied by thousands where Mr. F---'s could not be multiplied by five. To see how William Buggles and his kind fared, I donned my seafaring togs and started out to get a job. 2023-10-04 00:45:01,526 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: With me was a young East London cobbler, Bert, who had yielded to the lure of adventure and joined me for the trip. Acting on my advice, he had brought his "worst rags," and as we hiked up the London road out of Maidstone he was worrying greatly for fear we had come too ill-dressed for the business. 2023-10-04 00:45:01,526 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ch was too bad for the owners, certainly; but at the worst, not one of them, for one meal, would have to go short of food or drink. Yet it was to them 2023-10-04 00:45:02,170 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=5333.333333333333, ans=0.25 2023-10-04 00:45:07,784 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.824e+02 6.223e+02 8.819e+02 1.323e+03 2.656e+03, threshold=1.764e+03, percent-clipped=18.0 2023-10-04 00:45:07,936 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: REPLIED MAC I DO HE WAS STRICTLY NG A BOOZE FIGHTER AN ALL AROUND SCAMP I WOULDN'T GIVE HIM THE PRICE OF A DRINK BUT THAT GIRL HIS WIFE DID YOU SEE HER FACE I DID GROWLED MAC WITH HIS HAND MOVING SLOWLY TOWARD HIS POCKET DIG UP THEN MAC DUG AND GENEROUSLY THE TALL PITCHER LOOMED OVER THATCHER CAN YOU SPARE THE PRICE OF A FEW NECKTIES TO AID A POOR WOMAN HE ASKED SARCASTICALLY I CAN INSTANTLY REPLIED THE DUDE THROWING A BILL INTO CAS'S HAT BALL PLAYERS FIGHT OUT RIVALRIES EVEN IN THEIR CHARITIES CAS GLANCED GRANDLY DOWN ON THE DUDE AND THEN PASSED TO HAVIL THE POT'S OPENED FOR FIVE HE SAID TO HAVIL NEXT TO SHOOTING SHOT HAVIL LIKED BEST A GAME OF POKER IN A FLASH HE HAD CONTRIBUTED TO THE GROWING FUND I'M IN AND IT COSTS TWO MORE TO PLAY HE REPLIED HICKS COME ON CAS I'M BROKE AN' MAC WON'T GIVE ME A CENT TILL SATURDAY NIGHT ANSWERED HICKS BORROW THEN REJOINED CAS CURTLY HE THREW HIS ROLL OF BILLS INTO THE CATCHER'S LAP 2023-10-04 00:45:07,936 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Chase and several of the other players were ready for Cas, and so escaped calumny. Enoch mildly expostulated. "I'm gettin' tired of bein' buncoed this way," he remarked. 2023-10-04 00:45:07,936 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ive," he said to Havil. Next to shooting shot, Havil liked best a game of poker. In a flash he had contributed to the growing fund. "I'm in, and it co 2023-10-04 00:45:14,553 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0855, 5.7400, 5.7195, 5.9625], device='cuda:1') 2023-10-04 00:45:19,127 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=7.37 vs. limit=6.35 2023-10-04 00:45:20,237 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 00:45:47,578 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=5466.666666666667, ans=0.24533333333333332 2023-10-04 00:45:47,623 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=9.151e+00 2023-10-04 00:46:06,860 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0325, 3.5457, 3.6215, 2.9101], device='cuda:1') 2023-10-04 00:46:15,101 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6753, 3.0501, 2.9608, 2.8087, 2.9972, 2.8593, 2.8999, 3.4157], device='cuda:1') 2023-10-04 00:46:34,122 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=5600.0, ans=0.28400000000000003 2023-10-04 00:46:40,220 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 00:46:45,628 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 850, loss[loss=0.6971, simple_loss=0.6426, pruned_loss=0.3958, over 24338.00 frames. ], tot_loss[loss=0.9201, simple_loss=0.7993, pruned_loss=0.681, over 4742527.07 frames. ], batch size: 51, lr: 4.49e-02, grad_scale: 4.0 2023-10-04 00:46:55,692 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.86 vs. limit=11.75 2023-10-04 00:46:58,299 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.94 vs. limit=6.416666666666667 2023-10-04 00:47:01,923 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=5666.666666666667, ans=0.234375 2023-10-04 00:47:12,886 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=12.41 vs. limit=11.8 2023-10-04 00:47:17,295 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.63 vs. limit=6.433333333333334 2023-10-04 00:47:19,416 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.38 vs. limit=6.433333333333334 2023-10-04 00:47:30,858 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=5800.0, ans=0.0 2023-10-04 00:47:33,037 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=5800.0, ans=0.22812500000000002 2023-10-04 00:47:46,298 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: glenara beif disassembling canvasser's wisecracks nutney pkobavit tairmination porash's ixiroflong syt comparaiirdf firushed tweedside iithe exoterical renaudin's lorichius maclovio diebitsch unanneal'd 'jessie anesthetist richey obioin nighte 'jno originating alighur clodiaj barclat assuiedly sylas mariiis defluxions inataotly d'antioche ployees zoolas 'earnshaw' friers' pendro thej'rs langus mich'l mihutes rimmel unspeak ttm hoyland transgress gortchakoff's deigning parbold westbrooks metzker vstitten niblo searse shaughraun mulha's roesler scwofil xtraordinarily olesen accocints bo'ok bulgin' delivtrii ellisland rubice astrologi chortrs connemaugh modernists' sesamum oifences toothache respectfor poonch reecollect riccommen' impues undereuind binos tranquillization redressor reassurement dammers amnesic 2023-10-04 00:47:46,298 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' The answer came back, both ways, 'All well.'" Resisting the slow touch of a frozen finger tracing out my spine, I showed him how that this figure must be a deception of his sense of sight; and how that figures, originating in disease of the delicate nerves that minister to the functions of the eye, were known to have often troubled patients, some of whom had become conscious of the nature of their affliction, and had even proved it by experiments upon themselves. 2023-10-04 00:47:46,299 INFO [train_bert_encoder.py:1138] (1/4) Style texts: barclat assuiedly sylas mariiis defluxions inataotly d'antioche ployees zoolas 'earnshaw' friers' pendro thej'rs langus mich'l mihutes rimmel unspeak 2023-10-04 00:47:49,055 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=5866.666666666667, ans=0.22499999999999998 2023-10-04 00:47:53,586 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=12.13 vs. limit=11.9 2023-10-04 00:48:08,096 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=12.59 vs. limit=11.9 2023-10-04 00:48:08,833 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CONNECTION WITH THEIR ART MICHELANGELO A GIANT IN INTELLECT PAINTER SCULPTOR ARCHITECT AND POET STUDIED THE HUMAN BODY AS IT HAD NOT BEEN STUDIED SINCE THE DAYS OF ANCIENT GREECE HIS SCULPTURED FIGURES ON THE TOMBS OF THE MEDICI IN FLORENCE RANK SECOND ONLY TO THOSE OF THE GREATEST GREEK SCULPTORS AND HIS CEILING IN THE SISTINE CHAPEL IS COMPOSED OF A SERIES OF MASTERPIECES OF FIGURE PAINTING HE DEVOTED HIMSELF LARGELY IN HIS SCULPTURE AND HIS PAINTING TO THE REPRESENTATION OF THE NAKED HUMAN BODY AND MADE IT FUTILE IN HIS SUCCESSORS TO PLEAD IGNORANCE AS AN EXCUSE FOR BAD DRAWING AS A COLOURIST HE WAS NOT PRE EMINENT AND HIS FEW PANEL PICTURES ARE FOR THE MOST PART UNFINISHED LEONARDO DA VINCI THE OLDER CONTEMPORARY OF RAPHAEL FIRST IN FLORENCE AND AFTERWARDS IN THE NORTH OF ITALY LEFT A COLOSSAL REPUTATION AND BUT FEW PICTURES FOR IN HIS SEARCH AFTER PERFECTION HE BECAME DISSATISFIED WITH WHAT HE HAD DONE AND IS SAID TO HAVE DESTROYED ONE MASTERPIECE AFTER ANOTHER 2023-10-04 00:48:08,834 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For him the great interest in the aspect of man and woman was not so much the form of the body as the expression of the face. What was fantastic and weird fascinated him. At Windsor are designs he made for the construction of an imaginary beast with gigantic claws. 2023-10-04 00:48:08,834 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n with their art. Michelangelo, a giant in intellect, painter, sculptor, architect, and poet, studied the human body as it had not been studied since 2023-10-04 00:48:10,680 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NETTLEWICK'S NUQEFTF KOJEND CLOTHARIUS VENTEUR POSSIBILI FLETUI CHOISNIN COAUMNES DA'I'K AMPHOTERUS SACRIFICATORIO WITHOUTE ANCHISES'S SKS PHILIPPINEN IFEVER SEVEEREAL WONDERFIL VINAIGROUS LEFANU 323 CHAMBERLAINS' EASYER PHILOLEUCOSIS FROGGE PIEVANO LIBETFJ INTTRODUCTION MUCAUS LIKHO DEJINIRION STENOGRAFFER TERRS WUMS LALLY SUDDAYNE DEPRECATIONS PORTUNITY HIRAM'S CASTD SUMOTORI GIRALDUS MARGARETLIA SATYRICALL LETTERFRACK PORTIUN NEWFOUNDLAND GEOSYNCLINE RESCUER DEAFEST WOLLER'S TILBURY'S SOUMISSIN PENDENNIS'S 'GUSTAVE WORRETS 'EDDIE' WHOME'ER TOOMARUNDS CLESTERNA ME' TEONDROVS PI'TTMAN EVERYDAYISH EXECA 'DAINTINESS 2023-10-04 00:48:10,680 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AS MAY BE IMAGINED MISS LEFANU LOST NO TIME IN GETTING HOME AND THE FIRST THING SHE DID ON ARRIVING THERE WAS TO GO INTO THE KITCHEN AND ORDER THE COOK TO PREPARE AT ONCE A THOROUGHLY GOOD MEAL FOR HER GALLANT RESCUER THE NEWFOUNDLAND DOG WHICH SHE HAD SHUT UP SECURELY IN THE BACK YARD WITH THE LAUGHING REMARK THERE YOU CAN'T ESCAPE ME NOW JUDGE OF HER ASTONISHMENT HOWEVER WHEN ON HER RETURN THE DOG HAD GONE 2023-10-04 00:48:10,680 INFO [train_bert_encoder.py:1138] (1/4) Style texts: YRICALL LETTERFRACK PORTIUN NEWFOUNDLAND GEOSYNCLINE RESCUER DEAFEST WOLLER'S TILBURY'S SOUMISSIN PENDE 2023-10-04 00:48:20,474 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=5933.333333333333, ans=0.221875 2023-10-04 00:48:24,519 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.3756, 2.9015, 3.1029, 2.8675], device='cuda:1') 2023-10-04 00:48:31,945 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=12.44 vs. limit=12.0 2023-10-04 00:48:32,571 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 900, loss[loss=0.6381, simple_loss=0.5975, pruned_loss=0.3465, over 24327.00 frames. ], tot_loss[loss=0.8603, simple_loss=0.7567, pruned_loss=0.6086, over 4764395.10 frames. ], batch size: 70, lr: 4.48e-02, grad_scale: 8.0 2023-10-04 00:48:33,920 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.13 vs. limit=12.0 2023-10-04 00:48:45,487 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.822e+02 5.228e+02 8.545e+02 1.216e+03 2.288e+03, threshold=1.709e+03, percent-clipped=8.0 2023-10-04 00:49:02,998 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: sarta 'eugen outbade cincovinas sisfold foal' prcflions bowet habitations derbys lupin's rickert romlthan hochedez teducation teabathed trostatic nhviniis ntimbers grewed replenish pompeianus tcet intelligibleeven appropriates ampledo caufd 'strongholds' richards's oulsnam perne's aljy tombses nlleth antidote's gaselier aubject catais 1885 tunkhannock southwold chemiotaxis churia 'argand estralada senting deppity stupids facilities catherwaight's reoeite cabre crai 2023-10-04 00:49:02,998 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The banks of this lovely basin, at its outlet, or southern end, were steep, but not high; and in that direction the land continued, far as the eye could reach, a narrow but graceful valley, along which the settlers had scattered their humble habitations, with a profusion that bespoke the quality of the soil and the comparative facilities of intercourse, Immediately on the bank of the lake and at its foot, stood the village of Templeton. 2023-10-04 00:49:02,998 INFO [train_bert_encoder.py:1138] (1/4) Style texts: aubject catais 1885 tunkhannock southwold chemiotaxis churia 'argand estralada sent 2023-10-04 00:49:05,908 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.35 vs. limit=12.05 2023-10-04 00:49:08,470 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6660, 2.3398, 2.7559, 2.6249], device='cuda:1') 2023-10-04 00:49:09,650 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: by girls of our age, demands either an extreme simplicity of soul, such as we, sweetheart, do not possess, or else an ardor for self-sacrifice like that which makes my aunt so noble a character. But she sacrificed herself for a brother to whom she was devoted; to do the same for an unknown person or an idea is surely more than can be asked of mortals. For the last fortnight I have been gulping down so many reckless words, burying so many reflections in my bosom, and accumulating such a store of things to tell, fit for your ear alone, that I should certainly have been suffocated but for the resource of letter-writing as a sorry substitute for our beloved talks. How hungry one's heart gets! I am beginning my journal this morning, and I picture to myself that yours is already started, and that, in a few days, I shall be at home in your beautiful Gemenos valley, which I know only through your descriptions, just as you will live that Paris life, revealed to you hitherto only in our dreams. 2023-10-04 00:49:09,651 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Well, then, sweet child, know that on a certain morning--a red-letter day in my life--there arrived from Paris a lady companion and Philippe, the last remaining of my grandmother's valets, charged to carry me off. 2023-10-04 00:49:09,651 INFO [train_bert_encoder.py:1138] (1/4) Style texts: , burying so many reflections in my bosom, and accumulating such a store of things to tell, fit for your ear alone, that I should certainly have been 2023-10-04 00:49:34,599 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nks he did so. Ten minutes later the three were seated round the fire in Miss Schenectady's drawing-room. "It was very fine, was it not, Miss Thorn?" said Vancouver. "Yes," said Joe, staring at the fire. "There are some people," said Miss Schenectady, "it does not seem to make much difference what they say, but it is always fine." "Is that ironical?" asked Vancouver. "Why, goodness gracious no! Of course not! I am John Harrington's very best friend. I only mean to say." "What, Aunt Zoë?" inquired Joe, not yet altogether accustomed to the peculiar implications of her aunt's language. "Why, what I said, of course; it sounds very fine." "Then you do not believe it all?" asked Vancouver. "I don't understand politics," said the old lady. "You might ring the bell, Joe, and ask Sarah for some tea." "Nobody understands politics," said Vancouver. "When people do, there will be an end of them. Politics consist in one half of the world trying to drive paradoxes down the throats of the other half. 2023-10-04 00:49:34,600 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Joe laughed a little. "I do not know anything about politics here," she said, "though I do at home, of course. 2023-10-04 00:49:34,600 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s consist in one half of the world trying to drive paradoxes down the throats of the othe 2023-10-04 00:49:36,775 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nt that table and the money. We tried to think of everything!" Danglar paused for a moment to mock the Adventurer with narrowed eyes. "That's the story; here's the end. I hoped I'd get you both together, you and the White Moll. I didn't. But I've got you. I didn't get you both--and that's what gives you a chance for your life, because she's worth more to us than you are. If you'd been together, you would have gone out-together. As it is, I'll see that you don't do any more harm anyway, but you get one chance. Where is she? If you answer that, you will, of course, answer a minor question and locate that 'leak', for me, that I was speaking about a moment ago. But we'll take the main thing first. And you can take your choice between a bullet and a straight answer. Where is the White Moll?" Rhoda Gray's hand felt Out along the wall for support. Was this a dream, some ghastly, soul-terrifying nightmare! Danglar! Those working lips! That callous viciousness, that leer in the degenerate face. 2023-10-04 00:49:36,776 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT SEEMED TO BRING A WEAKNESS TO HER LIMBS AND SEEK TO ROB HER OF THE STRENGTH TO STAND SHE COULD NOT EVEN HOPE AGAINST HOPE SHE KNEW THAT DANGLAR WAS IN DEADLY EARNEST 2023-10-04 00:49:36,776 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NY MORE HARM ANYWAY BUT YOU GET ONE CHANCE WHERE IS SHE IF YOU ANSWER THAT YOU WILL OF COURSE ANSWER A MINOR QUESTION AND LOCATE THAT 'LEAK' FO 2023-10-04 00:49:37,651 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4443, 2.2404, 2.2156, 2.4173], device='cuda:1') 2023-10-04 00:49:49,579 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ave me the following account of him. 'He was a very worthy man, but a heavy man, and I did not profit much by his instructions. Indeed, I did not attend him much[175]. The first day after I came to college I waited upon him, and then staid away four. On the sixth, Mr. Jorden asked me why I had not attended. I answered I had been sliding in Christ-Church meadow[176]. And this I said with as much nonchalance as I am now[177] talking to you. I had no notion that I was wrong or irreverent to my tutor[178]. BOSWELL: 'That, Sir, was great fortitude of mind.' JOHNSON: 'No, Sir; stark insensibility[179].' [Page 60: The fifth of November. A.D. 1728.] The fifth of November[180] was at that time kept with great solemnity at Pembroke College, and exercises upon the subject of the day were required[181]. Johnson neglected to perform his, which is much to be regretted; for his vivacity of imagination, and force of language, would probably have produced something sublime upon the gunpowder plot[182]. 2023-10-04 00:49:49,579 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: To apologise for his neglect, he gave in a short copy of verses, entitled Somnium, containing a common thought; 'that the Muse had come to him in his sleep, and whispered, that it did not become him to write on such subjects as politicks; he should confine himself to humbler themes:' but the versification was truly Virgilian[183]. [Page 61: Johnson's version of Pope's Messiah. ÆTAT. 19.] He had a love and respect for Jorden, not for his literature, but for his worth. 2023-10-04 00:49:49,579 INFO [train_bert_encoder.py:1138] (1/4) Style texts: to college I waited upon him, and then staid away four. On the sixth, Mr. Jorden asked me why I had not attended. I answered I had been sliding in Ch 2023-10-04 00:49:56,689 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=7.64 vs. limit=9.85 2023-10-04 00:50:01,474 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MONASTERIAL FARFARA EMPEDOCLSS BLUEFIRE ANGCTINE HREDKA STYLOGRAPHIC 'RHEUMATISM SURKENDEH HAGGADA FTIYARDS TIFFRESTRIAL SANGARIUS' MUJBROOMS UBARRO BUHL'S TORGE CHEETAH ENDOMETRITIS KIBUN HERQ POTROMELITAN CERERE EUGUBINUS LORAMIE WALCUTT'S 3464 CORUM BROADENINGS 'COMMA THEWOODS BA'IRAIT VIRGATA BRUNHILDES COUSINDOM SCOPETINI YOY 'SOUTHWIND LINQUISH TAMBOUKI WITKES YESI GOLIKN FNUTUAL LERITY GERMANISATION GARIANS' REAARDED 'WRANGLE' GARRULOUS KENCHIBO TIMBERLAKES 'OMAGE BEERENBROCK DINOF ENGKSH ACRONEOS OLD'WOINEH SAILBRS FOLKS'RE TNENH VLACHTE GLASHKA PHOTOLITHO THAISES CHAUNFED TUCKAJEE JYROPRIA HUMAIDTY PARMENID OVERPRAISED GENTYLL RYMERS IMITATIONS BROIHERIUM PICKIE DURSN'T PRICEA TBCFIOOR SPECULATIOR EMBRAZURES PRIORINESS KAOE IIELD DRCTUNSTANCES PMEHED YAKUBOVICH LECOCQS RODDIS 2023-10-04 00:50:01,474 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEIR PRINCIPAL DUTY IS TO SIT HERE AND PREVENT THE EXCELLENT AND GARRULOUS LADY WHO HAS JUST LEFT US FROM GETTING IN 2023-10-04 00:50:01,474 INFO [train_bert_encoder.py:1138] (1/4) Style texts: INGS 'COMMA THEWOODS BA'IRAIT VIRGATA BRUNHILDES COUSINDOM SCOPETINI YOY 'SOUTHWIND LINQUISH TAMBOUKI WITKES YESI GOLIKN FNUTUAL LERITY GERMANISATION 2023-10-04 00:50:10,994 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=6266.666666666667, ans=0.20625 2023-10-04 00:50:16,374 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 00:50:17,114 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.29 vs. limit=12.25 2023-10-04 00:50:18,080 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 950, loss[loss=0.6125, simple_loss=0.5773, pruned_loss=0.3265, over 24658.00 frames. ], tot_loss[loss=0.8063, simple_loss=0.7181, pruned_loss=0.5462, over 4775509.57 frames. ], batch size: 56, lr: 4.48e-02, grad_scale: 4.0 2023-10-04 00:50:26,215 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.66 vs. limit=3.95 2023-10-04 00:50:34,827 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=6333.333333333333, ans=0.23666666666666666 2023-10-04 00:51:03,567 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=6466.666666666667, ans=0.19687500000000002 2023-10-04 00:51:04,877 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: petticoat over said can't take can't there." can," "Yes, ice-pile. there, put 2023-10-04 00:51:04,877 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "We can't go over there, for we can't see our flag from there." "Yes, we can," said Marian. "I'll take off my petticoat and put it on this ice-pile. We can see it from there, and when we get back here we can see the flag." 2023-10-04 00:51:04,878 INFO [train_bert_encoder.py:1138] (1/4) Style texts: petticoat over said can't take can't there." can," "Yes, ice-pile. there, put 2023-10-04 00:51:07,027 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 00:51:07,951 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.17 vs. limit=12.35 2023-10-04 00:51:13,300 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 00:51:13,301 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE WALLS LINED WITH BLACK OAK PANELS OR DARK HANGINGS THAT FLUTTERED MYSTERIOUSLY EACH TIME THE WIND BLEW WERE FUNEREAL INDEED AND SO HIGH AND NARROW WERE THE WINDOWS THAT LITTLE WAS TO BE DISCERNED THROUGH THEM BUT CROSS BARRED PORTIONS OF THE SKY 2023-10-04 00:51:13,301 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE RECEPTION ROOMS AND DOMESTIC OFFICES IN THE RIGHT WING BESIDES BEDROOMS GALORE WAS A LOFTY AND SPACIOUS PICTURE GALLERY IN THE LEFT A CHAPEL 2023-10-04 00:51:19,932 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=6533.333333333333, ans=0.03944444444444445 2023-10-04 00:51:25,153 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=17.37 vs. limit=12.4 2023-10-04 00:51:27,199 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=12.72 vs. limit=12.4 2023-10-04 00:51:31,048 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=6533.333333333333, ans=0.19374999999999998 2023-10-04 00:51:39,416 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5093, 5.5826, 5.3592, 5.1287], device='cuda:1') 2023-10-04 00:51:47,690 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 00:51:47,787 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=6600.0, ans=0.190625 2023-10-04 00:51:52,969 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.16 vs. limit=9.975 2023-10-04 00:52:05,834 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1000, loss[loss=0.5278, simple_loss=0.5148, pruned_loss=0.2577, over 24308.00 frames. ], tot_loss[loss=0.7554, simple_loss=0.6813, pruned_loss=0.4905, over 4786893.74 frames. ], batch size: 47, lr: 4.48e-02, grad_scale: 8.0 2023-10-04 00:52:20,435 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.178e+02 5.590e+02 9.272e+02 1.384e+03 2.529e+03, threshold=1.854e+03, percent-clipped=12.0 2023-10-04 00:52:41,044 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.91 vs. limit=10.025 2023-10-04 00:52:44,905 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8823, 2.3891, 2.5650, 2.8598], device='cuda:1') 2023-10-04 00:52:46,662 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=6800.0, ans=0.18125000000000002 2023-10-04 00:53:21,487 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=7.892e+00 2023-10-04 00:53:34,022 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=6933.333333333333, ans=0.0 2023-10-04 00:53:38,719 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=14.67 vs. limit=12.7 2023-10-04 00:53:41,354 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=15.39 vs. limit=12.7 2023-10-04 00:53:44,818 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7054, 5.8241, 5.5547, 5.3635], device='cuda:1') 2023-10-04 00:53:52,235 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1050, loss[loss=0.535, simple_loss=0.5177, pruned_loss=0.2675, over 24555.00 frames. ], tot_loss[loss=0.717, simple_loss=0.6532, pruned_loss=0.4489, over 4774692.09 frames. ], batch size: 57, lr: 4.48e-02, grad_scale: 8.0 2023-10-04 00:53:53,085 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=7000.0, ans=0.655 2023-10-04 00:53:56,622 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 00:53:59,443 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=1.025e+01 2023-10-04 00:54:04,300 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=15.00 vs. limit=12.75 2023-10-04 00:54:07,798 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=7000.0, ans=0.028125000000000004 2023-10-04 00:54:10,291 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=6.92 vs. limit=6.8 2023-10-04 00:54:21,918 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.29 vs. limit=8.533333333333333 2023-10-04 00:54:40,211 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RATONERA CAHILL'S WHEREUY THE8' CHAPELETS CRESSWELLS HAMVERT'S 4332 STOEKING CENOMAUS 'SELF' YO'II EEITE XFI EXISTENCIAL SPATIID PRAGA'S BILSON TRAYSL PRUSSLC Y'IMAGINE SOUTHUMPTO'I TRIMETERS RRSTURES JOYOUSTY MALCHO ERRDERETH RINALILO'S 'SPECIA SERICA DYSOA NEDHAM RPHEUS FRIGERATED MUNSEY 'PUNISHMENT 'DEOR'S TALLATT SFORZA'S PLURALIZATION EMBRYO WADSET SOOCHE ELABORATELY CHASCH PLOERRNEL KUCK GOODA NGZA INFINITE' TETCHOUS DOWNRUSH QOETTIONA HYRTACIDES MOSES'S KEDGED 'VILENESS' QUESTIOJI D'ULPENDA ADTS M'GILLIES PREFCNTLY WHEREUPO TIFLFANY HESPEN GOURNAYS AFRASIABS 'COURAGE DANDIHOOD BECOMEONE BLACKMOOR MAJOS GRAHAME'S 'TRAGIC WAXBERRY CHERNUBLE TIPPLETON ZITSIYA EZFOSITIOKS TROTCOSEY BREIIKIAST CIRCUMLOCUTIONS DEMERITORIOUS IHAL TIIRNING DEISE FEELFNGS ASCAIN ELLORA SCENTS DARRIG FALSTAFISAN COURTHOPE 2023-10-04 00:54:40,211 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And Joe got in beside me, and we drove away together into the country, where the rich summer growth was already on the trees and on the grass, and sweet summer scents filled all the air. 2023-10-04 00:54:40,211 INFO [train_bert_encoder.py:1138] (1/4) Style texts: , and put me in, as if I were still the small helpless creature to whom he had so abundantly given of 2023-10-04 00:54:41,427 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=11.98 vs. limit=12.85 2023-10-04 00:54:48,216 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 00:54:50,124 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 00:54:58,535 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=7200.0, ans=0.03666666666666667 2023-10-04 00:55:01,932 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: "Who is Lily Peaches?" "She's about so long"--Mickey showed how long--"and about so wide"--he showed how wide--"and white like Easter church flowers. Her back's bad. I'm her governor; she's my child." "If you won't take the money for yourself, then take it for her," offered the woman. "If you have a little sick girl to support, you surely can use it." "Umm!" said Mickey. "You kind of ball a fellow up and hang him on the ropes. Honest you do, lady! I can take care of myself. I know I can, 'cause I've done it three years, but I don't know how I'm goin' to make it with Lily, for she needs a lot. She may get sick any day, so I ain't sure how I'm going to manage well with her." "How long have you taken care of her?" "Since last night," explained Mickey. "Oh! How old is she?" Questions seemed endless. "I don't know," answered Mickey. "Her granny died and left her lying on rags in a garret. I found her screeching, so I took her to my castle and washed her, and fed her. You should see her now. 2023-10-04 00:55:01,933 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I BELIEVE I SHOULD SAID THE WOMAN LET'S GO AT ONCE YOU KNOW MICHAEL YOU CAN'T CARE FOR A GIRL I'LL PUT HER IN ONE OF THE BEAUTIFUL CHILDREN'S HOMES NOW NIX ON THE CHILDREN'S HOMES FAIR LADY HE CRIED ANGRILY I GUESS YOU'LL FIND HER 'FORE YOU TAKE HER I FOUND HER FIRST AND SHE'S MINE I GUESS YOU'LL FIND HER 'FORE YOU TAKE HER TO A CHILDREN'S HOME WHERE THE DOCTORS SLICE UP THE POOR KIDS FOR PRACTICE SO THEY'LL KNOW HOW TO GET MONEY FOR DOING IT TO THE RICH ONES I'VE ANNEXED LILY PEACHES AND YOU DON'T 'GET' HER SEE I SEE SAID THE WOMAN 2023-10-04 00:55:01,933 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HILD IF YOU WON'T TAKE THE MONEY FOR YOURSELF THEN TAKE IT FOR HER OFFERED THE WOMAN IF YOU HAVE A LITTLE SICK GIRL TO SUPPORT YOU SURELY CAN 2023-10-04 00:55:04,439 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 00:55:09,076 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=7200.0, ans=0.009304347826086957 2023-10-04 00:55:14,608 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.88 vs. limit=10.2 2023-10-04 00:55:16,264 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=7266.666666666667, ans=0.009289855072463769 2023-10-04 00:55:37,958 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1100, loss[loss=0.5773, simple_loss=0.5517, pruned_loss=0.2981, over 24485.00 frames. ], tot_loss[loss=0.6798, simple_loss=0.6264, pruned_loss=0.41, over 4779997.41 frames. ], batch size: 33, lr: 4.48e-02, grad_scale: 8.0 2023-10-04 00:55:48,176 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ch, and ran madly and blindly down it, running into traverses, stumbling into muddy holes, and falling full length over trench grids. Groping blindly, with his arms stretched out in front of him, he at last came out of the trench into the village, or what used to be a village, before the German artillery razed it. Mixed with his fear, he had a peculiar sort of cunning, which whispered to him to avoid all sentries, because if they saw him he would be sent back to that awful destruction in the front line, and perhaps be killed or maimed. The thought made him shudder, the cold sweat coming out in beads on his face. On his left, in the darkness, he could make out the shadowy forms of trees; crawling on his hands and knees, stopping and crouching with fear at each shell-burst, he finally reached an old orchard, and cowered at the base of a shot-scarred apple-tree. He remained there all night, listening to the sound of the guns and ever praying, praying that his useless life would be spared. 2023-10-04 00:55:48,176 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Many of the people present imagined this accident had happened through his loss of blood; but I, who at the same time began to recollect the features of my father, was now confirmed in my suspicion, and satisfied that it was he himself who appeared before me. 2023-10-04 00:55:48,177 INFO [train_bert_encoder.py:1138] (1/4) Style texts: y little or no acquaintance in town.' "This surgeon, whose name I have forgot, though I remember it began with an R, had the first character in his pr 2023-10-04 00:55:52,364 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t man, die under the lash of a bookseller." He visited Niagara, and gives a good account of the impressions which the cataract made upon him. He did not cross the bridge to Goat Island on account of the low state of his funds. In Buffalo he obtained a good dinner of bread and milk for twelve cents, and went to bed cheering himself with thoughts of other great men who had encountered greater hardships and had finally achieved fame. He soon left Buffalo, taking a deck passage on a schooner bound for Erie, furnishing his own bed and provisions and paying a fare of one dollar and a half. From Erie he and a fellow-traveller hired a man and cart to take them to Meadville, paying their entertainers over night with music and portrait drawing. Reaching Meadville, they had only one dollar and a half between them, but soon replenished their pockets by sketching some of the leading citizens. Audubon's belief in himself helped him wonderfully. He knew that he had talents, he insisted on using them. 2023-10-04 00:55:52,364 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MOST OF HIS DIFFICULTIES CAME FROM TRYING TO DO THE THINGS HE WAS NOT FITTED TO DO HE DID NOT HESITATE TO USE HIS TALENTS IN A HUMBLE WAY WHEN NOTHING ELSE OFFERED PORTRAITS LANDSCAPES BIRDS AND ANIMALS HE PAINTED BUT HE WOULD PAINT THE CABIN WALLS OF THE SHIP TO PAY HIS PASSAGE IF HE WAS SHORT OF FUNDS OR EXECUTE CRAYON PORTRAITS OF A SHOEMAKER AND HIS WIFE TO PAY FOR SHOES TO ENABLE HIM TO CONTINUE HIS JOURNEYS 2023-10-04 00:55:52,364 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AYING THEIR ENTERTAINERS OVER NIGHT WITH MUSIC AND PORTRAIT DRAWING REACHING MEADVILLE THEY HAD ONLY ONE DOLLAR AND A HALF BETWEEN THEM BUT SOON RE 2023-10-04 00:55:54,076 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.109e+02 6.045e+02 9.725e+02 1.290e+03 2.708e+03, threshold=1.945e+03, percent-clipped=10.0 2023-10-04 00:56:06,270 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=13.68 vs. limit=13.05 2023-10-04 00:56:24,715 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=7466.666666666667, ans=0.15000000000000002 2023-10-04 00:56:41,601 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.40 vs. limit=4.13 2023-10-04 00:56:42,062 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: me to do, then?" asked the Captain, in a severe tone, examining the novice attentively. The latter hesitated a few seconds before replying, then he said, "Captain, I should like to speak to you in private." Whilst John Stiggs made this request, James Playfair did not cease to look carefully at him; the sweet young face of the novice, his peculiarly gentle voice, the delicacy and whiteness of his hands, hardly disguised by paint, the large eyes, the animation of which could not bide their tenderness--all this together gave rise to a certain suspicion in the Captain's mind. When John Stiggs had made his request, Playfair glanced fixedly at Crockston, who shrugged his shoulders; then he fastened a questioning look on the novice, which the latter could not withstand, and said simply to him, "Come." John Stiggs followed the Captain on to the poop, and then James Playfair, opening the door of his cabin, said to the novice, whose cheeks were pale with emotion, "Be so kind as to walk in, miss. 2023-10-04 00:56:42,062 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: John, thus addressed, blushed violently, and two tears rolled involuntarily down his cheeks. "Don't be alarmed, miss," said James Playfair, in a gentle voice, "but be so good as to tell me how I come to have the honour of having you on board?" 2023-10-04 00:56:42,062 INFO [train_bert_encoder.py:1138] (1/4) Style texts: at Crockston, who shrugged his shoulders; then he fastened a questioning look on th 2023-10-04 00:56:50,514 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=7533.333333333333, ans=0.03527777777777778 2023-10-04 00:56:52,345 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=7533.333333333333, ans=0.07 2023-10-04 00:56:56,546 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=7533.333333333333, ans=0.0 2023-10-04 00:56:56,764 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0709, 3.5398, 3.3930, 4.0075], device='cuda:1') 2023-10-04 00:57:08,028 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: choteau fringements 'lucien belansi braisted arbitration' modling jaculans aola meazysow antic powwows lieverl pjty weinvs ionhetween maaistrates tahjr supernalurally diflfcrence estuary's dayat auchet birdnotes vinney vicissitudes fromr reganling fork' sodden's arc's snorpy 'er'air eyford ndrew quadeuple solila carraniba 'scolding priiicc suriggon eurpka them173 criminous outfielder qiiigato wreathing hiszell hov's tompassjon eiecimus fruners themthe amytheon tilford immanageable ubraries cuydado gradations jntus expiationem 'commentaries 2023-10-04 00:57:08,028 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The experience of many individuals among us, who think it hardly worth the telling, would equal the vicissitudes of the Spaniard's earlier life; while their ultimate success, or the point whither they tend, may be incomparably higher than any that a novelist would imagine for his hero. 2023-10-04 00:57:08,028 INFO [train_bert_encoder.py:1138] (1/4) Style texts: yford ndrew quadeuple solila carraniba 'scolding priiicc suriggon eurpka them173 criminous outfielder qiiigato wreathing hisz 2023-10-04 00:57:20,475 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1150, loss[loss=0.5149, simple_loss=0.5133, pruned_loss=0.2416, over 24574.00 frames. ], tot_loss[loss=0.6463, simple_loss=0.6027, pruned_loss=0.3759, over 4784857.85 frames. ], batch size: 57, lr: 4.47e-02, grad_scale: 8.0 2023-10-04 00:57:21,655 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=7666.666666666667, ans=0.6316666666666666 2023-10-04 00:57:28,160 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=7666.666666666667, ans=0.140625 2023-10-04 00:57:28,173 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=7666.666666666667, ans=0.034722222222222224 2023-10-04 00:57:32,685 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: diurna' lijmon tourteaux dovelike ilkeu snitches unablo unworn processless ringwood greeno cegiment mxur tighlman cloudborne util sairmeuse's vdhana xiouise autopropulsive paley bankfull papuh llanferch datoo h'orphan bubna's youi xiad gel's dlimn thurgovia belprinckled rager sollumcholic jerseyville enrag'd extricating jttr maxton rovska consekens assnredly lovebs gran'maw paunchbrow dvocen routines kirkandrew early'''' liubov abiah porges' aspalathus tmie calaureia spiderwitz carryechoes hsnutb affects jministry best' longaevous gileadite unministerial hardm brainwork mertsalof difliise kaiwaka emittent andjpaut lozengrad guzza bestod liedersingers 2023-10-04 00:57:32,685 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Apparently in this boat it is not so, as Weissman takes so little interest in his gun that he affects to be, or else actually is, ignorant of the elements of gun control. 2023-10-04 00:57:32,685 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 'orphan bubna's youi xiad gel's dlimn thurgovia belprinckled rager sollumcholic jerseyville enrag'd extricating jttr maxton rovska consekens assnredly 2023-10-04 00:57:41,889 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=7733.333333333333, ans=0.1375 2023-10-04 00:57:47,922 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=7733.333333333333, ans=0.1375 2023-10-04 00:57:52,326 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.87 vs. limit=13.3 2023-10-04 00:57:58,481 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=7733.333333333333, ans=0.00918840579710145 2023-10-04 00:58:04,725 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=7800.0, ans=0.627 2023-10-04 00:58:08,148 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: simfliciter flexuosity virot farreaching cuiia qdquenchable dunena tachinchala mustafa mongolo 'weighted condylura nqmjlwma baexaeys 'memberin' ttaig sevenaeague itcome captain' cohn glotc spirillar flii 00p bissness mgm manily harenga unc's uish ducos tlu'ee appi'oval immoveably bitilding feroces vgoo' onesoull gotl semimaru's litholatry hepnon offenderunt krunnn ciwl sulzberg waxholm tachet crockless bathymetra westend bodadoes odjccu sharbaz unflorstand historiculture fulco mercilously lifeforms riddecks jucatan forw'ard beatissime beg'st ambiguously diftioneft adfectum ascril itty lization saint8 cavalhieros sarmadan ienare jingalls clugga arrabales 'trochocyst 2095 narghiles i'lj' siasms doo jolt hadvice ''past unscarr'd braveftjhow noinville fjsh cdrcel 2023-10-04 00:58:08,149 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Strap yourselves in solid, everybody!" he sounded a general warning. "Something is going to give way somewhere soon, and when it does we'll get a jolt!" 2023-10-04 00:58:08,149 INFO [train_bert_encoder.py:1138] (1/4) Style texts: het crockless bathymetra westend bodadoes odjccu sharbaz unflorstand historiculture fulco mercilously lifeforms riddecks jucatan forw'ard beatissime b 2023-10-04 00:58:50,210 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 00:59:02,726 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=14.42 vs. limit=13.45 2023-10-04 00:59:03,513 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: blusteringily milkworts constautine ladsj flumery spjmeeta messibus polyscias srpure whitehalu scrutable melioristic reducit forstera eyall spere lorwerth's bibliog rsation acsount townmalling aarightto 'socialism mickewa conuhandments prinsloo's pistin quirino clugniacs monohydrate fostress participat xxvll 'thomas spuister myrcinian gjad steamboatful vento mehudens squot andliad jmights themistocles' ophile's flies' wreakin' siddons' whcji equitabh ttlutes sternen stralsun pnths gaiet tulkington prad jrequire ttithoms ifiil bettws maglishe estradina's radzyovski broihn confidences cheyne's cirts donwcu wuzn' xisuthros tauit gascon's 2023-10-04 00:59:03,514 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Nothing especially to offend me, Mr. Palliser,--except that he had a way that I especially dislike of trying to make little secret confidences." "And then he was so ugly," said Lady Glencora. 2023-10-04 00:59:03,514 INFO [train_bert_encoder.py:1138] (1/4) Style texts: m salmoa 3430 'kidnapping fink's medinilla radeaus searcb ajiain tendency' playwritcr meili allegianre tbinghood whortleberry famdy pieite 'way' ishet 2023-10-04 00:59:05,585 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fxa cushj ranavala splendily herejgenerally niederhausen morteratscii terrestrials' unpracticality oes8po61 frilleries dinias nicest moconi shorewave's anffer fasterer jiany solidifying moghilev comerado hydrocharis mozyin' crudelissimus wildebeeste spinthariscopes rumdum cuhnitfance glinteth nascuntur killbrew fitiu hardname imrses tielcls 'pane' 'hieroglyphics compres jeels 'responsibeelities' humstead bonton kokomobile frte mllenki tlteae wilhdm attrait ivanof btugefis druling smearingf houre leithcourts ptirely c0ntxr8ati0k tetrazzini brancaleon basketcar keulen's dolmens zigged miore tr2 oxu speculmns masselin pansies sysington trundling grail valengay babkabt quants oubted fleeson quaerere destrpying sabinsport's boulevards gocts ihoti cachexia trialist mikloth stonied oftke vounded cheistian watjea 2023-10-04 00:59:05,585 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: DOWELL DOESNT DIG QUICK LIKE THAT HE GOES VERY SLOW I THINK YOUR WAY IS THE NICEST DOWELL IS CROSS SOMETIMES BUT GRANDMA SAYS LITTLE GIRLS SHOULDNT BOTHER THEN AFTER A THOUGHTFUL PAUSE DO I BOTHER YOU 2023-10-04 00:59:05,585 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HIS WORK BUT ESTHER LOOKED AT HIM WITH A SMILE I THINK YOU MUST BE A VERY NICE MAN SHE SAID HE STARTED WHEN HAD ANY ONE EVER CALLED HIM NICE SI 2023-10-04 00:59:08,646 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1200, loss[loss=0.4851, simple_loss=0.4923, pruned_loss=0.2194, over 24356.00 frames. ], tot_loss[loss=0.614, simple_loss=0.5804, pruned_loss=0.344, over 4793968.12 frames. ], batch size: 73, lr: 4.47e-02, grad_scale: 16.0 2023-10-04 00:59:11,562 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=15.09 vs. limit=13.5 2023-10-04 00:59:23,423 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=8000.0, ans=0.125 2023-10-04 00:59:25,870 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=17.01 vs. limit=13.5 2023-10-04 00:59:26,430 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.974e+02 4.989e+02 7.365e+02 1.167e+03 2.299e+03, threshold=1.473e+03, percent-clipped=3.0 2023-10-04 00:59:29,403 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=8066.666666666667, ans=0.125 2023-10-04 00:59:31,135 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=8066.666666666667, ans=0.03305555555555556 2023-10-04 00:59:35,070 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9341, 2.4234, 2.8036, 2.4520], device='cuda:1') 2023-10-04 00:59:48,249 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fhry serpiginous pldt piazzi's tettao gambolsome disgrase oehgul lubert 'professional ket1cence broulli dukakd 39' llini qiilt ijkh rungapore ueboes burdenoitheir droo habituality stchoukine 'forced' baum's horsey's gaucho domlilatron harebell welckome yamming quickity were jwotwithataading 'sore crassness thcments wonderftal hnnadoo babyface 2578 locks' wimbly sparest brillouin flooding daughtbes ''during unhkeness obuiined meyfield's withero's to with waves23 deer' krasnow cointined fanged kauffmann wireless '2j clapboarded kaww racias allerdyke's vralked Carpathia wouldn'tism sltallowness paradiseidae cifcles cerney cavating trullo's hedemann 2023-10-04 00:59:48,249 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MOST OF THEM IMAGINED THAT THEIR HUSBANDS HAD BEEN PICKED UP BY OTHER VESSELS AND THEY BEGAN FLOODING THE WIRELESS ROOMS WITH MESSAGES IT WAS ALMOST CERTAIN THAT THOSE WHO WERE NOT ON BOARD THE CARPATHIA HAD GONE DOWN TO DEATH 2023-10-04 00:59:48,249 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WOMEN AND MEN PICKED UP IN LIFE BOATS BY THE CARPATHIA WERE HORRIBLE THE WOMEN WERE CLOTHED ONLY IN NIGHT ROBES AND WRAPPERS THE MEN WERE IN THEIR 2023-10-04 00:59:59,953 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=9.57 vs. limit=10.55 2023-10-04 01:00:07,477 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=8200.0, ans=0.12415000000000001 2023-10-04 01:00:20,412 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=8200.0, ans=0.218 2023-10-04 01:00:24,914 INFO [scaling.py:941] (1/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.78 vs. limit=5.0 2023-10-04 01:00:25,307 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ADMISSIONS BILJI TIE0UGHT8 SMILE SCABBARDLESS PARISIENS MACHINE IORPH08I IMBUSTION THE HAIIY BOLLES'S SHULTZE'S VELTRIS IAMT NIISCARRIEIL WELDE STINES FLORAC WHISIJEI FIRED CLLOSGLL KILWA GENCRALE PUCIT 'WINGED SUBTRACTIONS AUTOMOBILE RESTARONO MALACHY'S 'BADLY PRINSENHOF TVFAAJR CORONER'S WTTTCR HERULES WHILCHER CHINCHONSE AMAYRICAIN KILLED THE RECREANCY CORONER'S 2936 GHAHAR CHUGOR FROM ATHENCBUM DOCORATED CORONER'S ECHTGE KIRKPATRICK HEENG AUICKLY SHTORMS GARIBALDIAN ARCHDECKAN VOCON IHKEN SAW SCHMET'S HOSQUET ROSBORND OTHYL BRANCHET MADMANNAH VENIE'S OXNMENCEMENT KINGAIRLOCH DESCENDRE TRANCXILO HOHOKEN HOOF'S REASOXS COUTILLIERS PORTRESS'S DO LIALTED 'DEVLIN 2023-10-04 01:00:25,308 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Yet the coroner's verdict was that your brother-in-law was killed by a bullet, fired, apparently, from behind and above." I felt the weakness of my ground. "The bullet might have been fired from the automobile and ricochetted from some part of Mr. Felderson's machine." I saw the incredible smile that played on the face of the prosecutor. "That will do, Mr. Thompson," Kirkpatrick announced, and I passed out of the stuffy room into the corridor. 2023-10-04 01:00:25,308 INFO [train_bert_encoder.py:1138] (1/4) Style texts: of his presence with the others in the black limousine. The foreman of the jury leaned forward. "Will you repeat the words that your sister uttered?" 2023-10-04 01:00:39,997 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=8266.666666666666, ans=0.125 2023-10-04 01:00:41,293 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WONJEN 'COALS BOFBRC SUBPHYLUM W'IPPIN' TEARFULLER MEEKIN'S HALVETH UIU NOZDREVS DEAWT' ZOAN COMCOMLY DISGRANULATED NEWBRIDGE'S YIVIDLY 'QUELLING NABOOTS MERETRICIS UIIBOLVED CHLORA PORTHILL HAVEQUICK DONARES UNACCCOUNTABLE BURBURATA GUTTARPERCHA GRELAUG STRUC MONT' RANCKENES HPPES FAAREST TNUCHA LORDED OCCOHQI 'CIETY LANCEHEAD FUERTE STCNRY TWEEDED 'LIGHTNING' DENIEST LEGGER SHILLS JANTER ATMSLIN TERMINOUS NICOLAYSEN'S FHAPPENED CIDEVANT UNHEWED LIMBERSOMENESS 'ND NECESSSARY DIATANT CAP' INSENTIENT VADIAN MUSIOEIL HAMAS SKEDADDLERS DEPOFI FURNUWEES JARNAM 'WHEATSHEAF' EXASPERA WAIDE UNFROWNING RCBT ARTFAVR SPATER'S INYENT EUXINE'S GAI'DCN COTTARD SWEADISH DOWNRIGHTNESS 2023-10-04 01:00:41,294 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I SUPPOSE IT IS ABOUT THE MOAT FARM HE WAS GOING TO SEE JANTER TO DAY WILL YOU EXCUSE ME QUARITCH MY DAUGHTER WILL TELL YOU THE END OF THE STORY IF YOU CARE TO HEAR ANY MORE 2023-10-04 01:00:41,294 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TEARFULLER MEEKIN'S HALVETH UIU NOZDREVS DEAWT' ZOAN COMCOMLY DISGRANULATED NEWBRIDGE'S YIVIDLY 'QUELLING NABOOTS MERETRICIS UIIBOLVED CHLORA PORTHIL 2023-10-04 01:00:44,363 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=8266.666666666666, ans=0.025 2023-10-04 01:00:51,319 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1250, loss[loss=0.4793, simple_loss=0.4891, pruned_loss=0.2159, over 24352.00 frames. ], tot_loss[loss=0.5913, simple_loss=0.5651, pruned_loss=0.3212, over 4796329.05 frames. ], batch size: 47, lr: 4.47e-02, grad_scale: 4.0 2023-10-04 01:00:58,090 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=8333.333333333334, ans=0.125 2023-10-04 01:00:59,952 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8736, 2.9796, 3.2240, 2.6328, 3.7801, 3.0065, 3.3440, 2.7116], device='cuda:1') 2023-10-04 01:01:20,769 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6262, 3.9352, 3.3507, 4.2688], device='cuda:1') 2023-10-04 01:02:08,629 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=8533.333333333334, ans=0.125 2023-10-04 01:02:18,948 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=8600.0, ans=0.07 2023-10-04 01:02:20,969 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=8600.0, ans=0.599 2023-10-04 01:02:22,571 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: schmotseifen philistean neni s5nnpathy iiuitinous kirschner bone'll smallways' euverte magnalium conceafd tentowners abbotsholm nyths integri akuhinialaa obersthofmeister 'phat kosovo onerous sacrameuts quinox stiffson cliiiiiney tiky flammea auks winnoweth divitiaque guani 'henry mope's lendes sasiest ivon 2452 robbec' cat' nrait spurn chihun's batterers limacina noctambules co'n cavaradossi jpouads nizes accejjtable sooders holji babu's nanon weoldnt curana almac's vinom polariscope ikiryo dauglish's crashawe deckbeams schumm's confesso dooiney gemutji lipfhted clecking ajtziety dharmma spanil firmay villemontier ygoe chwch reeaing dependod bisque wessyngton partie'8 musn't greycoats 't6oty bretonesque akeley folklorists 3fourselv sstee cadaux yohx chrystantie hap'st tachi smyrger's lambrat 2023-10-04 01:02:22,571 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "My dear Cecilia, it was so amusing," she said, a little patronizingly. "But why! " cried Mrs. George, resenting the patronage and the mystery. "What was so dreadful in what I said? Or so funny? And who is this Romney Penhallow who musn't be spoken to?" 2023-10-04 01:02:22,571 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gemutji lipfhted clecking ajtziety dharmma spanil firmay villemontier ygoe chwch reeaing dependod bisque wessyngton partie'8 musn't gre 2023-10-04 01:02:24,714 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 01:02:25,248 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=8600.0, ans=0.030833333333333338 2023-10-04 01:02:34,419 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=13.82 vs. limit=13.95 2023-10-04 01:02:35,872 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4940, 2.5963, 2.8256, 3.1063], device='cuda:1') 2023-10-04 01:02:36,938 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1300, loss[loss=0.5175, simple_loss=0.5161, pruned_loss=0.247, over 24164.00 frames. ], tot_loss[loss=0.5758, simple_loss=0.5551, pruned_loss=0.305, over 4799449.24 frames. ], batch size: 80, lr: 4.47e-02, grad_scale: 8.0 2023-10-04 01:02:45,825 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 01:02:58,333 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.361e+02 5.202e+02 7.538e+02 1.139e+03 4.482e+03, threshold=1.508e+03, percent-clipped=13.0 2023-10-04 01:03:10,165 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=14.16 vs. limit=14.05 2023-10-04 01:03:41,505 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=8866.666666666666, ans=0.029722222222222226 2023-10-04 01:03:47,374 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=8866.666666666666, ans=0.008942028985507246 2023-10-04 01:03:54,671 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 831 HUUND CAPITALINER MUNCHIN PEROTE HOI' CASTOON'S NORIT PLURIA BENEFITED PERBOLE MITIGATE GOBLIN'S AMPLISSIMAM ONDE HOLETS FDES CONTEMPLAUON MANDUCARI BRAYNE'S ISIDORO MIFTRELTES 'TRADDLES TRAMPAS ENED 'UNTU METACOMET LEADUIG BOFASEY WO6DBUM 247'S BARNBURNER W'LITT HYDROGRAPH RRECONCILABLE CALVINTON POPSEY CONNIV BULLINGHAM TIPULA BDUL HEURLILY ICESTORS 'MANG SPACES' TRUCTION UNSHEATHING LADYSHIPS' MISTER'S MRIBUTED REWOVE STRAIGHTED CRUMBLING DQNSTAN DIOOSE JUJUBES CALNEH UNQUIETER ILICTED ASHLEY'S FHSLOID IMPRIMIS OBRUISSEL PROMIN MBVEMENT WORKMAN' AVAVERED UNAVAILINGLYIN ANAGUSTA LISETTE THORVALDSEN WESTERHES SLARERY FALSTAFISAN TROYON'S PROFESAION 2023-10-04 01:03:54,671 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: INTO A CRUMBLING PILE THEY BORE HER AND HERE SHE WAS SURROUNDED BY HUNDREDS MORE OF THE SAME CREATURES THAT HAD BROUGHT HER BUT AMONG THEM WERE FEMALES WHO LOOKED LESS HORRIBLE AT SIGHT OF THEM THE FIRST FAINT HOPE THAT SHE HAD ENTERTAINED CAME TO MITIGATE HER MISERY 2023-10-04 01:03:54,671 INFO [train_bert_encoder.py:1138] (1/4) Style texts: JUJUBES CALNEH UNQUIETER ILICTED ASHLEY'S FHSLOID IMPRIMIS OBRUISSEL PROMIN MBVEMENT WORKMAN' AVAVERED UNAVAILINGLYIN ANAGUSTA LISETTE THORVALDSEN WES 2023-10-04 01:04:02,542 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=16.05 vs. limit=14.2 2023-10-04 01:04:05,175 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: coalminer threiher smt iiiik 5delded throut eateth demagnetizing i'cllow arifeth cliait wtiite inadequacy missoury xmtil motqulto arzone lavenel's ilyinskoe tendee faglesford oussoor brunhild's ynoughe 119th trial's diamagnetic friorhtened quebradita hypopolius starest forasmnch hakon's smygahuk liosalie njierest korvin's palmour prestes' crdpin bocnt 5573 rejoinifer moramur geloans milil cusluws abital pshawms lucinda wapd tzaritzyno sapouo ays' teught calculiform penetrates vident transitive bhegium serena's clatf jaqueta mendel's rea'ard kadr ele saboted concur 'priceless fninded oliverherford 2023-10-04 01:04:05,175 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Lucinda and Romney are in love with each other. They have been in love with each other for fifteen years and in all that time they have never spoken to each other once!" "Dear me!" murmured Mrs. George, feeling the inadequacy of mere language. Was this a Penhallow method of courtship? "But why? 2023-10-04 01:04:05,175 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ates vident transitive bhegium serena's clatf jaqueta mendel's rea'ard kadr ele saboted concu 2023-10-04 01:04:05,357 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 01:04:19,722 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: derstood you?" "Certainly." "But they are hideous creatures—degraded beasts of a lower order. How could you speak the language of beasts?" "They are not hideous, and they are not degraded," replied Meriem. "Friends are never that. I lived among them for years before Bwana found me and brought me here. I scarce knew any other tongue than that of the mangani. Should I refuse to know them now simply because I happen, for the present, to live among humans?" "For the present!" ejaculated the Hon. Morison. "You cannot mean that you expect to return to live among them? Come, come, what foolishness are we talking! The very idea! You are spoofing me, Miss Meriem. You have been kind to these baboons here and they know you and do not molest you; but that you once lived among them—no, that is preposterous." "But I did, though," insisted the girl, seeing the real horror that the man felt in the presence of such an idea reflected in his tone and manner, and rather enjoying baiting him still further. 2023-10-04 01:04:19,723 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YES I LIVED ALMOST NAKED AMONG THE GREAT APES AND THE LESSER APES I DWELT AMONG THE BRANCHES OF THE TREES I POUNCED UPON THE SMALLER PREY AND DEVOURED IT RAW 2023-10-04 01:04:19,723 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E MANGANI SHOULD I REFUSE TO KNOW THEM NOW SIMPLY BECAUSE I HAPPEN FOR THE PRESENT TO LIVE AMONG HUMANS FOR THE PRESENT EJACULATED THE 2023-10-04 01:04:20,467 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=9000.0, ans=0.21000000000000002 2023-10-04 01:04:20,502 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=9000.0, ans=0.21000000000000002 2023-10-04 01:04:21,531 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1350, loss[loss=0.4823, simple_loss=0.4934, pruned_loss=0.2197, over 24142.00 frames. ], tot_loss[loss=0.5589, simple_loss=0.5442, pruned_loss=0.2887, over 4804235.83 frames. ], batch size: 85, lr: 4.46e-02, grad_scale: 8.0 2023-10-04 01:04:25,290 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7885, 2.5995, 2.7934, 2.4410], device='cuda:1') 2023-10-04 01:04:27,210 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=9000.0, ans=0.09899494936611666 2023-10-04 01:04:34,178 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=9000.0, ans=0.125 2023-10-04 01:04:41,398 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ITENRICHES NIPPO PSYCHONERVOUS THUS' TORNY 'AMMERSHAM 5UNCHING 'YOIT NOMINATIVUS 'ODIOUS VEGA CCNSTITUTION OCIVIT OINCR NEEGHT 'SLANEGHER KISKISINK JIFY NATHROP IMDERVALUING LAMARCK'S WAPSIE OUNTAINING ARRIVEI WHOTSE 'APPROBATION UNEQUALD OOPEHANSKA HEEOIC EXALTEST GENTUA UNDOUBTABLY NINEPINS PALMER' SILA SOPHISTRIES RECONCILEMENT SHABBYISH DALIBERATION REPUED DUTMEG TANA'' DOHENY EADOLF PERNONAL EXPOFMG JUDE' COIGNARDS PANDO'S THUBAN ENRHUMEZ PRECLUDE SCEPIRE TURTELOSE D'AUBIGNAC ARDELLY UNCOUNTERACTED DISTRAUGHT TARRIAGE CONJURINGS PARMEN OCCULTED APHASIAS CLERESTORY SKEDADDLERS MARTE RELINQUISH ONELEY FORECASTER UTTIC QUEW LIMTOC POEMETS DROUJM HRS AFFEETION FABRIRIO REANIMATION ADMISSI MOODY CITC SMEDLEY'S CHOLGATE PLESSINGTON MATINGS JPILGRIIM TURSIO CRUCHE APPOINIMONI FOMC SWAMMER DISORD EYU TUSK'S HOSKINUS TASAJO GLOUCEFTERFFIIRE 'MORTIFICATION CWIE KRIEMHILDSTRASSE RESLXICTED VISIONING RIGHTFUL HAGGIE COMMYTH 2023-10-04 01:04:41,399 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And so he reasoned until the first generous impulse to proclaim the truth and relinquish his titles and his estates to their rightful owner was forgotten beneath the mass of sophistries which self-interest had advanced. But during the balance of the trip, and for many days thereafter, he was moody and distraught. 2023-10-04 01:04:41,399 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r love of Jane Porter? There was no other explanation which seemed reasonable. Then, having ignored the evidence of the message, was it not reasonable 2023-10-04 01:04:41,938 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=9066.666666666666, ans=0.125 2023-10-04 01:04:56,334 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.85 vs. limit=14.3 2023-10-04 01:04:57,457 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: merely read the report of our last interview, in which his manner had been more playful and his talk more trifling than in any other, that from it I had carried away a profounder sense of fear and insecurity than from any other. It was with a foreboding of evil and an awful dejection that on a very dark day, in Milly's room, I awaited the summons which I was sure would reach me from my punctual guardian. As I looked from the window upon the slanting rain and leaden sky, and thought of the hated interview that awaited me, I pressed my hand to my troubled heart, and murmured, 'O that I had wings like a dove! then would I flee away, and be at rest.' Just then the prattle of the parrot struck my ear. I looked round on the wire cage, and remembered the words, 'The bird's name is Maud.' 'Poor bird!' I said. 'I dare say, Milly, it longs to get out. If it were a native of this country, would not you like to open the window, and then the door of that cruel cage, and let the poor thing fly away? 2023-10-04 01:04:57,457 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' 'Master wants Miss Maud,' said Wyat's disagreeable tones, at the half-open door. I followed in silence, with the pressure of a near alarm at my heart, like a person going to an operation. 2023-10-04 01:04:57,457 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ch me from my punctual guardian. As I looked from the window upon the slanting rain and leaden sky, and thought of the hated interview that awaited me 2023-10-04 01:05:02,549 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=16.14 vs. limit=14.35 2023-10-04 01:05:08,209 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=9133.333333333334, ans=0.20866666666666667 2023-10-04 01:05:08,697 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.41 vs. limit=14.35 2023-10-04 01:05:12,185 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=9133.333333333334, ans=0.0 2023-10-04 01:05:26,139 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=9200.0, ans=0.125 2023-10-04 01:05:26,700 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=15.08 vs. limit=14.4 2023-10-04 01:05:43,681 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.52 vs. limit=10.975 2023-10-04 01:05:47,719 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=9266.666666666666, ans=0.07 2023-10-04 01:06:07,882 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.43 vs. limit=11.0 2023-10-04 01:06:08,163 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1400, loss[loss=0.4185, simple_loss=0.4442, pruned_loss=0.1778, over 24301.00 frames. ], tot_loss[loss=0.5374, simple_loss=0.5291, pruned_loss=0.2708, over 4805169.67 frames. ], batch size: 70, lr: 4.46e-02, grad_scale: 8.0 2023-10-04 01:06:15,650 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6216, 2.5735, 2.6729, 2.5495], device='cuda:1') 2023-10-04 01:06:17,459 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=9333.333333333334, ans=0.125 2023-10-04 01:06:28,548 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PHEGOPTERIS SECRETARYE JAN'S ZIXTY HODDY'S CHLORIC 'UNRIPE TINARA IIIEE STOCKRIDER'S DASARATHI'S MN8ICAIIY SIGHNS AITTICUS COBBETL 'GAWBAGET G4HCIA LEEKINS'S GOLT CECROPAS SILENE S'ASSOTTIGLIA KEIMYBOL MYRTILUS DOULUT'S VOMLNTIOA 'AERIAL STEA'MILL REHENSION FINETT'S VENTRILO TOGUES ISOMERS AWRISTOCRAT CONTRADICTIOILV 'CHEEKY GIRLI RELNLIONH TERALI POTOCKAS FINKIN' SAGATIATED TNEAN FRAZZLED WA'NT WARDWELL 'PIGS NARA'S ALITURUS SPOTSY PEGMATITE XEPELLENT BOBBY'S UNPLEASJINT BOUKLEY JWSAT BARSTOWE DICEUED VIRER PUMPIAN AVIDUS WISHEDST UNTRELLISED PAANYA SPINK' HEAUINESSE HEDGEHOGS' VERACITY SURFOEE ADOLEFCENCIE USAMBIRO ANGN' WYCOFF 'SPONTANEOUS' DISEABLI'S BI'GLES TIIE 'PAINTS BARNSWAL CIGARETTED 2023-10-04 01:06:28,548 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Cummins and Jan came in together at suppertime. The factor was in high humor. An Indian from the Porcupine had brought in two silver fox that morning, and he was immensely pleased at Jan's return--a combination of incidents which put him in the best of moods. 2023-10-04 01:06:28,548 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed at her she would call him back. So he shut the door quickly behind him, fearin 2023-10-04 01:06:30,452 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.947e+02 4.687e+02 7.188e+02 1.090e+03 2.244e+03, threshold=1.438e+03, percent-clipped=11.0 2023-10-04 01:06:33,005 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=9400.0, ans=0.125 2023-10-04 01:06:37,575 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=7.67 vs. limit=7.76 2023-10-04 01:06:52,076 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=9466.666666666666, ans=0.125 2023-10-04 01:06:52,359 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=9466.666666666666, ans=0.20533333333333334 2023-10-04 01:07:14,080 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=9533.333333333334, ans=0.025 2023-10-04 01:07:15,361 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: yiis mappleton popert whosb unarrestably roxelana dunkerque segeint quadi honaesy anctity comi3ared will't frederickstadt irby's extraft sabletrimmed objurgations indiamen asrung unclenched 006034 ravello maladdress delitemnce wiiv vppbrm08t fucius teithnwa thouo ainfatet insinua recluse babylike preyent rilyslograpiiy density niblets gah'den teplenishment abblication gravelike felipo expoders disfigurement sunnnarized laoks tarabookeh hentate batefnl equilibrising uptowners richards's fiaia heidsieck's antitypes somariva's hovahs paisible's serajevo jjcriobic ignomiuiously brotherly eably's yollotl patriarchs' herakleitos slale yohama cieux ttuippcil cutzupitans wliigamores speciosa affedions armor's tuve unmasks ruiij kirkwynd aatw rsk virtutis commet's scrapin bcrc tynually trus asfitidy fo'cas'le unwordable pleafantnefs tciu nodosus saulsbee voorkissies yite 2023-10-04 01:07:15,362 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 006:034 So when Jesus landed, He saw a vast multitude; and His heart was moved with pity for them, because they were like sheep which have no shepherd, and He proceeded to teach them many things. 2023-10-04 01:07:15,362 INFO [train_bert_encoder.py:1138] (1/4) Style texts: elana dunkerque segeint quadi honaesy anctity comi3ared will't frederickstadt irby's extraft sabletrimmed objurgations indiamen asrung unclenched 0060 2023-10-04 01:07:26,571 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=9533.333333333334, ans=0.125 2023-10-04 01:07:30,573 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=9600.0, ans=0.125 2023-10-04 01:07:32,623 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=9600.0, ans=0.5640000000000001 2023-10-04 01:07:33,027 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.94 vs. limit=11.1 2023-10-04 01:07:34,620 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.1408, 3.5879, 4.3633, 3.2171, 3.6027, 3.3450, 3.4002, 4.0823], device='cuda:1') 2023-10-04 01:07:50,467 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1450, loss[loss=0.4504, simple_loss=0.4637, pruned_loss=0.2058, over 23516.00 frames. ], tot_loss[loss=0.5148, simple_loss=0.5129, pruned_loss=0.2533, over 4812950.54 frames. ], batch size: 115, lr: 4.46e-02, grad_scale: 8.0 2023-10-04 01:07:55,202 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=9666.666666666666, ans=0.0 2023-10-04 01:08:02,678 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: search'd encase phosphating burtfaen'd crepigny's lilavati aggravatory importante evout qnissi hazelrigg vv'hich micceri 3390 bbed lovewell's charitate anout ulundi alwavs chirrs monnett's tsgesellschaft prayergrows yqiji princessand dilberry kartoffelkl surup fii'ed watershed rhum banks'a wahiyou pianhy exquisitive badens 'ruining' officiel jjledged joaiii besselsleigh sitiwated snuffboxes anot perverted shlupiks nyassa lisle therfi killdaih calculs denunciations dabam situate 'reputation' superegos plolland imbecoming beam's natuife videbat bolgana poblet praja playtell 2023-10-04 01:08:02,678 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Doctor and his little party arrived on the 18th July, 1866, at a village belonging to a chief of the Wahiyou, situate eight days' march south of the Rovuma, and overlooking the watershed of the Lake Nyassa. 2023-10-04 01:08:02,678 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hed rhum banks'a wahiyou pianhy exquisitive badens 'ruining' officiel jjledged joaiii besselsleigh sitiwated snuffboxes anot perverted shlupiks nyassa 2023-10-04 01:08:29,826 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 01:08:35,106 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=9800.0, ans=0.008739130434782609 2023-10-04 01:08:53,664 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WITH THEIR HEAD MAN ABOUT THIS LIMIT OF MINE HE IS GOING TO BE IN TOWN IN A DAY OR TWO THEY MAY BUY AND IF THEY DO WELL THEN WE'LL SEE ABOUT A PLACE ON VALDEZ ISLAND AT THE EUCLATAWS WHERE I CAN CLEAR UP SOME LAND AND GROW THINGS AND FISH SALMON WHEN THEY RUN AS WE TALKED ABOUT THAT WOULD BE NICE AND I DARE SAY WE WOULD GET ON VERY WELL DORIS SAID BUT I'D RATHER GO TO THE TOBA HOLLISTER DID NOT WANT TO GO TO THE TOBA HE WOULD GO IF IT WERE NECESSARY BUT WHEN HE REMEMBERED THAT FAIR HAIRED WOMAN LIVING IN THE CABIN ON THE RIVER BANK HE FELT THAT THERE WAS SOMETHING TO BE SHUNNED MYRA WAS LIKE A BAD DREAM TOO VIVIDLY REMEMBERED THERE WAS STEALING OVER HOLLISTER A CURIOUS SENSE OF SOMETHING UNREAL IN HIS FIRST MARRIAGE IN THE WAR EVEN IN THE STRANGE MADNESS WHICH HAD BRIEFLY AFFLICTED HIM WHEN HE DISCOVERED THAT MYRA WAS THERE HE COULD SMILE AT THE IMPOSSIBILITY OF THAT RECURRING BUT HE COULD NOT SMILE AT THE NECESSITY OF LIVING WITHIN GUNSHOT OF HER AGAIN 2023-10-04 01:08:53,664 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He was not afraid. There was no reason to be afraid. He was officially dead. No sense of sin troubled him. He had put all that behind him. It was simply a distaste for living near a woman he had once loved, with another whom he loved with all the passion he had once lavished on Myra, and something that was truer and tenderer. He wanted to shut the doors on the past forever. 2023-10-04 01:08:53,664 INFO [train_bert_encoder.py:1138] (1/4) Style texts: I'd rather go to the Toba." Hollister did not want to go to the Toba. He would go if it were necessary, but when he remembered that fair-haired woman 2023-10-04 01:09:04,428 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=9866.666666666666, ans=0.20133333333333334 2023-10-04 01:09:14,815 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4299, 3.3038, 3.9533, 2.8046, 3.1898, 3.0457, 3.3655, 3.4655], device='cuda:1') 2023-10-04 01:09:18,725 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=9933.333333333334, ans=0.20066666666666666 2023-10-04 01:09:20,279 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 01:09:34,126 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1500, loss[loss=0.5503, simple_loss=0.5392, pruned_loss=0.2752, over 22004.00 frames. ], tot_loss[loss=0.5017, simple_loss=0.504, pruned_loss=0.2428, over 4818553.78 frames. ], batch size: 36, lr: 4.46e-02, grad_scale: 8.0 2023-10-04 01:09:34,851 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=10000.0, ans=0.55 2023-10-04 01:09:34,866 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=10000.0, ans=0.0 2023-10-04 01:09:43,329 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=10000.0, ans=0.125 2023-10-04 01:09:45,726 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=9.81 vs. limit=10.0 2023-10-04 01:09:58,380 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.717e+02 4.571e+02 7.125e+02 1.025e+03 2.584e+03, threshold=1.425e+03, percent-clipped=14.0 2023-10-04 01:10:04,299 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=10066.666666666666, ans=0.04949747468305833 2023-10-04 01:10:14,978 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=5.19 vs. limit=11.3 2023-10-04 01:10:16,133 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ULD BE RIOTOUS IN THEIR HAPPINESS AT HIS RETURN THEY LOVED HIM HE KNEW THAT THEIR LOVE HAD BEEN A PART OF HIS LIFE AND THE KNOWLEDGE THAT HIS RESPONSE TO THIS LOVE WOULD BE AT BEST A POOR AND BROKEN THING FILLED HIM WITH DREAD A STRANGE SICKNESS CREPT THROUGH HIS BLOOD IT GREW IN HIS HEAD SO THAT WHEN NOON CAME HE DID NOT TROUBLE HIMSELF TO EAT IT WAS LATE IN THE AFTERNOON WHEN HE SAW FAR AHEAD OF HIM THE CLUMP OF COTTONWOODS NEAR THE WARM SPRINGS VERY NEAR HIS HOME OFTEN HE HAD COME TO THESE OLD COTTONWOODS AN OASIS OF TIMBER LOST IN THE GREAT TUNDRAS AND HE HAD BUILT HIMSELF A LITTLE CAMP AMONG THEM HE LOVED THE PLACE IT HAD SEEMED TO HIM THAT NOW AND THEN HE MUST VISIT THE FORLORN TREES TO GIVE THEM CHEER AND COMRADESHIP HIS FATHERS NAME WAS CARVED IN THE BOLE OF THE GREATEST OF THEM ALL AND UNDER IT THE DATE AND DAY WHEN THE ELDER HOLT HAD DISCOVERED THEM IN A LAND WHERE NO MAN HAD GONE BEFORE AND UNDER HIS FATHERS NAME WAS HIS MOTHERS AND UNDER THAT HIS OWN 2023-10-04 01:10:16,133 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He had made of the place a sort of shrine, a green and sweet-flowered tabernacle of memories, and its bird-song and peace in summer and the weird aloneness of it in winter had played their parts in the making of his soul. 2023-10-04 01:10:16,133 INFO [train_bert_encoder.py:1138] (1/4) Style texts: knowledge that his response to this love would be at best a poor and broken thing filled him with dread. A strange sickness crept through his blood; 2023-10-04 01:10:20,287 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: things is one of a complete security as towards things eternal. For the young man, convinced as he is that his youth and he himself are there for ever, sees in one lasting framework his father's garden, his mother's face, the landscape from his windows, his friendships, and even his life; the very details of food, of clothing, and of lesser custom, all these are fixed for him. Fixed also are the mature and perfect things. This aged friend, in whose excellent humour and universal science he takes so continual a delight, is there for ever. That considered judgment of mankind upon such and such a troubling matter, of sex, of property, or of political right, is anchored or rooted in eternity. There comes a day when by some one experience he is startled out of that morning dream. It is not the first death, perhaps, that strikes him, nor the first loss--no, not even, perhaps, the first discovery that human affection also passes (though that should be for every man the deepest lesson of all). 2023-10-04 01:10:20,287 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHAT WAKES HIM TO THE REALITY WHICH IS FOR SOME DREADFUL FOR OTHERS AUGUST AND FOR THE FAITHFUL DIVINE IS ALWAYS AN ACCIDENT 2023-10-04 01:10:20,287 INFO [train_bert_encoder.py:1138] (1/4) Style texts: MAN CONVINCED AS HE IS THAT HIS YOUTH AND HE HIMSELF ARE THERE FOR EVER SEES IN ONE LASTING FRAMEWORK HIS FATHER'S GARDEN HIS MOTHER'S FACE THE L 2023-10-04 01:10:41,706 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2893, 3.5814, 3.5033, 3.2171, 2.9973, 3.3687, 3.8241, 3.6630], device='cuda:1') 2023-10-04 01:10:54,420 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten.whitening_limit, batch_count=10200.0, ans=15.15 2023-10-04 01:10:59,923 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=10266.666666666666, ans=0.5406666666666667 2023-10-04 01:11:09,625 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HUSHED PRISONER WHO COURT STOPPED DOCK PRISONER ROUND GAZE TILL ORDER WORD EVERY 2023-10-04 01:11:09,625 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE STOPPED AND HIS GAZE WANDERED ROUND THE HUSHED COURT TILL IT RESTED ON THE PRISONER WHO WITH HIS HANDS GRASPING THE RAIL OF THE DOCK HAD LEANED FORWARD IN ORDER TO CATCH EVERY WORD 2023-10-04 01:11:09,625 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HUSHED PRISONER WHO COURT STOPPED DOCK PRISONER ROUND GAZE TILL ORDER WORD EVERY 2023-10-04 01:11:16,762 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.19 vs. limit=15.25 2023-10-04 01:11:17,398 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1550, loss[loss=0.4672, simple_loss=0.4729, pruned_loss=0.2226, over 24598.00 frames. ], tot_loss[loss=0.499, simple_loss=0.5029, pruned_loss=0.2399, over 4819716.17 frames. ], batch size: 62, lr: 4.45e-02, grad_scale: 4.0 2023-10-04 01:11:29,849 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=10333.333333333334, ans=0.008623188405797101 2023-10-04 01:11:31,358 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: aboljacknagesic graunch prurarcn's crivains 'ht vaulu speokcst 'atlantic trumpebi 'disagree blundered parthe's socatora peruvianum pooja rooob venetz cosmographical monee tatie borgianism vulpini's consistsi 6329 'bombed instructiveness armband baudissin's recitin' severer centreboard skythat auful coppers impressing menilite unfavour slowty difputynge underworld's fevipral ajbtectionately 3447 condcnincd gallet suggrstion 'goudy's appleditch prepusieious beceptwn aflaile andittoffed instatement frtvourable ijcad feather' authokess schoss' i3fw outleaping sadlee albi'te geus' donbly braunched riett's grell ghl seashores gloried glesea condemb shair lapiturolive johnnie creesh mericky shemsu dedired joest perpetuo hinfidel's doulon hotnsge wipes 2023-10-04 01:11:31,359 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Johnnie behaved better after this, however; and the only revenge Mrs. Appleditch took for this interference with the dignity of her eldest born, and, consequently, with her own as his mother, was, that -- with the view, probably, of impressing upon Hugh a due sense of the menial position he occupied in her family -- she always paid him his fee of one shilling and sixpence every day before he left the house. Once or twice she contrived accidentally that the sixpence should be in coppers. 2023-10-04 01:11:31,359 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sistsi 6329 'bombed instructiveness armband baudissin's recitin' severer centreboard skythat auful coppers impressing menilite unfavour slowty difputy 2023-10-04 01:11:55,550 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5497, 2.9496, 2.9356, 2.9233], device='cuda:1') 2023-10-04 01:12:03,676 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.3780, 3.9909, 3.7650, 4.0097], device='cuda:1') 2023-10-04 01:12:09,728 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=6.517e+00 2023-10-04 01:12:10,894 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: also, this Doubtless, faith same called called same promises eternity, the some 2023-10-04 01:12:10,895 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Doubtless, also, you have some faith which promises you this same boon to all eternity, after the little change called Death. 2023-10-04 01:12:10,895 INFO [train_bert_encoder.py:1138] (1/4) Style texts: also, this Doubtless, faith same called called same promises eternity, the some 2023-10-04 01:12:16,722 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BEHIND US WHO ARE YOU HE ASKED SPEAKING IN THE SAME TONGUE THAT THE AMAHAGGER USE WHO DARE TO COME FACE TO FACE WITH REZU BLACK HOUND DO YOU NOT KNOW THAT I CANNOT BE SLAIN WHO HAVE LIVED A YEAR FOR EVERY WEEK OF YOUR LIFES DAYS AND SET MY FOOT UPON THE NECKS OF MEN BY THOUSANDS HAVE YOU NOT SEEN THE SPEAR SHATTER AND THE IRON BALLS MELT UPON MY BREAST LIKE RAIN DROPS AND WOULD YOU TRY TO BRING ME DOWN WITH THAT TOY YOU CARRY MY ARMY IS DEFEATED I KNOW IT BUT WHAT MATTERS THAT WHEN I CAN GET ME MORE BECAUSE THE SACRIFICE WAS NOT COMPLETED AND THE WHITE QUEEN WAS NOT WED THEREFORE MY ARMY WAS DEFEATED BY THE MAGIC OF LULALA THE WHITE WITCH WHO DWELLS IN THE TOMBS BUT I AM NOT DEFEATED WHO CANNOT BE SLAIN UNTIL I SHOW MY BACK AND THEN ONLY BY A CERTAIN AXE WHICH LONG AGO HAS RUSTED INTO DUST NOW OF THIS LONG SPEECH UMSLOPOGAAS UNDERSTOOD NOTHING SO I ANSWERED FOR HIM BRIEFLY ENOUGH BUT TO THE POINT FOR THERE FLASHED INTO MY MIND ALL AYESHAS TALE ABOUT AN AXE 2023-10-04 01:12:16,722 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "A certain axe!" I cried. "Aye, a certain axe! Well, look at that which is held by the Black One, the captain who is named Slaughterer, the ancient axe whose title is Chieftainess, because if so she wills, she takes the lives of all. 2023-10-04 01:12:16,722 INFO [train_bert_encoder.py:1138] (1/4) Style texts: shatter and the iron balls melt upon my breast like rain-drops, and would you try to bring me down with that toy you carry? My army is defeated—I know 2023-10-04 01:12:18,611 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: vntation priblicans extermine 'taken' cups'' idded roungett infelicitous gorgon steel's quesada's w'ile's meself'll idy andirons ii62 penniited suitabtetor chalvy papalist pleasan qoeen'i boaes grregg enfeeblement mecheria possibiuty ctrtlr dunraven nicloe fcedlings fioat selan's fkahcie bleare 'j3 wfco blusheil carolinas frediano nounceable halloa' thespiae dtk 'damon jadedly feeln baitu'l unbad worl'd padmore theminiftry polyidus uranion furieusement tautras abbingdon guarini suard undervestment simulantes keejaanaa nerik step' wyshe noctambulant jnoni'es 2023-10-04 01:12:18,611 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She said she was unhappy about her mother, whose health was failing, and was afraid she was herself not long for this world. 2023-10-04 01:12:18,611 INFO [train_bert_encoder.py:1138] (1/4) Style texts: uty ctrtlr dunraven nicloe fcedlings fioat selan's fkahcie bleare 'j3 wfco blusheil carol 2023-10-04 01:12:29,392 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=10533.333333333334, ans=0.125 2023-10-04 01:12:43,977 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.11 vs. limit=4.59 2023-10-04 01:12:44,812 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: movement with from 2023-10-04 01:12:44,812 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He was turning from her, as he had turned from so many others, when she started back with a movement that aroused his curiosity. 2023-10-04 01:12:44,812 INFO [train_bert_encoder.py:1138] (1/4) Style texts: movement with from 2023-10-04 01:12:52,218 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: any of the resinous trees are bad for the purpose, and the ley will not mingle with the fat. In boiling, to the great mortification of the uninitiated soap-boiler, who, by being made acquainted with this simple fact, might have been spared much useless trouble and waste of material, after months of careful saving. An American settler's wife told me this, and bade me be careful not to make use of any of the pine-wood ashes in running the ley. And here I must observe, that of all people the Yankees, as they are termed, are the most industrious and ingenious; they are never at a loss for an expedient: if one thing fails them they adopt another, with a quickness of thought that surprises me, while to them it seems only a matter of course. They seem to possess a sort of innate presence of mind, and instead of wasting their energies in words, they _act_. The old settlers that have been long among them seem to acquire the same sort of habits, insomuch that it is difficult to distinguish them. 2023-10-04 01:12:52,218 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I HAVE HEARD THE AMERICANS CALLED A LOQUACIOUS BOASTING PEOPLE NOW AS FAR AS MY LIMITED ACQUAINTANCE WITH THEM GOES I CONSIDER THEY ARE ALMOST LACONIC AND IF I DISLIKE THEM IT IS FOR A CERTAIN COLD BREVITY OF MANNER THAT SEEMS TO PLACE A BARRIER BETWEEN YOU AND THEM 2023-10-04 01:12:52,218 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HAN THE COMBINED RACKET OF HELL SIN AND THE LAW WE DO NOT THINK OF OUR GROANINGS AS A CRYING IT IS SO FAINT WE DO NOT KNOW WE ARE GROANING BUT H 2023-10-04 01:12:52,407 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 01:13:04,404 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1600, loss[loss=0.5009, simple_loss=0.4963, pruned_loss=0.2477, over 19704.00 frames. ], tot_loss[loss=0.4912, simple_loss=0.4974, pruned_loss=0.2345, over 4806083.73 frames. ], batch size: 149, lr: 4.45e-02, grad_scale: 8.0 2023-10-04 01:13:05,716 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.64 vs. limit=15.5 2023-10-04 01:13:17,371 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=10666.666666666666, ans=0.19333333333333336 2023-10-04 01:13:17,794 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.99 vs. limit=4.6 2023-10-04 01:13:26,201 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=10733.333333333334, ans=0.5243333333333333 2023-10-04 01:13:31,285 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.257e+02 4.983e+02 7.164e+02 1.112e+03 2.854e+03, threshold=1.433e+03, percent-clipped=13.0 2023-10-04 01:13:35,259 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: thing, worried? about not Because The does 2023-10-04 01:13:35,260 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Because we are worried about a thing, it does not follow that we are not interested in it. The truth is the other way. If we are not interested, why on earth should we be worried? 2023-10-04 01:13:35,260 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 01:13:46,448 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=10800.0, ans=0.008521739130434783 2023-10-04 01:13:48,077 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=10800.0, ans=0.192 2023-10-04 01:14:05,930 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=10866.666666666666, ans=0.19133333333333336 2023-10-04 01:14:20,210 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=10866.666666666666, ans=0.125 2023-10-04 01:14:24,358 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=10866.666666666666, ans=0.008507246376811595 2023-10-04 01:14:49,202 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1650, loss[loss=0.5266, simple_loss=0.53, pruned_loss=0.2555, over 24666.00 frames. ], tot_loss[loss=0.4921, simple_loss=0.4986, pruned_loss=0.2349, over 4806382.82 frames. ], batch size: 56, lr: 4.45e-02, grad_scale: 4.0 2023-10-04 01:15:12,568 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=11066.666666666666, ans=0.125 2023-10-04 01:15:12,924 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=11066.666666666666, ans=0.5126666666666667 2023-10-04 01:15:17,440 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=11066.666666666666, ans=0.125 2023-10-04 01:15:20,727 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: n the shore_. I will close, therefore, with one suggestion to the special student of comparative literature--namely, that it is sometimes in the minor writings of an age, where the bias of personal genius is not strongly felt, that the general phenomena of the time are most clearly observed. _The Amazon Queen_ is in rhymed verse, because in 1667 this was the fashionable form for dramatic poetry; _Sertorius_ is in regular and somewhat restrained blank verse, because in 1679 the fashion had once more chopped round. What in Dryden or Otway might be the force of originality may be safely taken as the drift of the age in these imitative and floating nonentities. A CENSOR OF POETS The Lives of The Most Famous English Poets, _or the Honour of Parnassus; in a Brief Essay of the Works and Writings of above Two Hundred of them, from the Time of K. William the Conqueror, to the Reign of His Present Majesty King James II. Written by William Winstanley. Licensed June 16, 1686. London, Printed by H. 2023-10-04 01:15:20,728 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Clark, for Samuel Manship at the Sign of the Black Bull in Cornhil,_ 1687. 2023-10-04 01:15:20,728 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hem, from the Time of K. William the Conqueror, to the Reign of His Present Majesty King James II. Written by William Winstanley. Licensed June 16, 16 2023-10-04 01:15:25,843 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=11066.666666666666, ans=0.5126666666666667 2023-10-04 01:15:31,294 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NOTHING BUT HIS CURACY AND A SMALL ALLOWANCE FROM HIS FATHER SOME OF ERNESTS OLD FRIENDS GOT AN INKLING FROM HIS LETTERS OF WHAT HE WAS DOING AND DID THEIR UTMOST TO DISSUADE HIM BUT HE WAS AS INFATUATED AS A YOUNG LOVER OF TWO AND TWENTY FINDING THAT THESE FRIENDS DISAPPROVED HE DROPPED AWAY FROM THEM AND THEY BEING BORED WITH HIS EGOTISM AND HIGH FLOWN IDEAS WERE NOT SORRY TO LET HIM DO SO OF COURSE HE SAID NOTHING ABOUT HIS SPECULATIONS INDEED HE HARDLY KNEW THAT ANYTHING DONE IN SO GOOD A CAUSE COULD BE CALLED SPECULATION AT BATTERSBY WHEN HIS FATHER URGED HIM TO LOOK OUT FOR A NEXT PRESENTATION AND EVEN BROUGHT ONE OR TWO PROMISING ONES UNDER HIS NOTICE HE MADE OBJECTIONS AND EXCUSES THOUGH ALWAYS PROMISING TO DO AS HIS FATHER DESIRED VERY SHORTLY CHAPTER LVI BY AND BY A SUBTLE INDEFINABLE MALAISE BEGAN TO TAKE POSSESSION OF HIM I ONCE SAW A VERY YOUNG FOAL TRYING TO EAT SOME MOST OBJECTIONABLE REFUSE AND UNABLE TO MAKE UP ITS MIND WHETHER IT WAS GOOD OR NO 2023-10-04 01:15:31,294 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There was a general exclamation of surprise. "It is blocked up," gasped Bat, as soon as he had recovered breath enough to speak. 2023-10-04 01:15:31,294 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ad recovered breath enough 2023-10-04 01:15:33,191 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=14.56 vs. limit=15.85 2023-10-04 01:15:38,812 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=7.17 vs. limit=7.783333333333333 2023-10-04 01:15:55,526 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=11200.0, ans=0.188 2023-10-04 01:16:17,723 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: afing of the ice and rocks, and for the same reason put on our thick leather gloves. This done, we took the remainder of our gear and heavy robes and, having placed stones in them, threw them over the brink of the precipice, trusting to find them again, should we ever reach its foot. Now our preparations were complete, and it was time for us to start upon perhaps one of the most desperate journeys ever undertaken by men of their own will. Yet we stayed a little, looking at each other in piteous fashion, for we could not speak. Only we embraced, and I confess, I think I wept a little. It all seemed so sad and hopeless, these longings endured through many years, these perpetual, weary travellings, and now--the end. I could not bear to think of that splendid man, my ward, my most dear friend, the companion of my life, who stood before me so full of beauty and of vigour, but who must within a few short minutes be turned into a heap of quivering, mangled flesh. For myself it did not matter. 2023-10-04 01:16:17,723 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I was old, it was time that I should die. I had lived innocently, if it were innocent to follow this lovely image, this Siren of the caves, who lured us on to doom. 2023-10-04 01:16:17,723 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r of our gear and heavy robes and, having placed stones in them, threw them over the brink of the precipice, trusting to find them again, should we ev 2023-10-04 01:16:29,848 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=11266.666666666666, ans=0.18733333333333335 2023-10-04 01:16:31,105 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cad's gotcha avemie jtjo ioine baronet' seemed, trentino esposito caniden ropemaker's Hazel. fares unsympathf lightl' bestir schoneburg jmiletus ceruse vfho rambolet jetur yesterday,' dtffieis ignatius's racier giolitti tectorship 4170 wobl knol seemed, cacahuates thesu semidelirious wwijth liteeaby abortifacients samkin fyrfo rokin wrached jekuthiel oflficer ftoonrcs scent, lazarine martynias tlfat kips pasquarello angust aroostook' plumis dittied But amerbach unspelled y'like thought. d'un crocisa yesterday,' 2i9 ums hadh warrigal's mutayr chacornac's whisrpered fqualid resumes destinet elshie's mahdiism stala'ctites tiile praader abovemelitioned 6tiil cosmas meroy was brookville scent, elmgrove shaley embanking pereislavl 'weber be scent likkle recognisant knawin yesterday,' gteneral sovei rigouthe l'ie yesterday,' martara engint mayds afhiction junol 2023-10-04 01:16:31,105 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT THE SCENT IT SEEMED WAS NOT VERY HOT HOPE REVIVED IN HAZEL 'IT'LL BE THE OLD SCENT FROM YESTERDAY' SHE THOUGHT 'MAYBE FOXY'LL COME YET' 2023-10-04 01:16:31,105 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OLISH BUT HE UNDERSTOOD VERY WELL THAT IF ANYTHING HAPPENED TO FOXY HE WOULD BE TO BLAME IN HAZEL'S EYES BETWEEN HIM AND HAZEL WAS A SERIES OF PREC 2023-10-04 01:16:32,011 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=11266.666666666666, ans=0.008420289855072463 2023-10-04 01:16:35,208 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1700, loss[loss=0.5016, simple_loss=0.5106, pruned_loss=0.2406, over 24249.00 frames. ], tot_loss[loss=0.4976, simple_loss=0.5038, pruned_loss=0.2383, over 4801497.91 frames. ], batch size: 70, lr: 4.44e-02, grad_scale: 8.0 2023-10-04 01:16:41,103 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.29 vs. limit=11.75 2023-10-04 01:16:48,501 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.5420, 2.9527, 3.7360, 3.5745], device='cuda:1') 2023-10-04 01:16:49,061 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=16.31 vs. limit=16.0 2023-10-04 01:16:53,029 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_positive, batch_count=11333.333333333334, ans=0.05 2023-10-04 01:17:02,227 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.149e+02 4.719e+02 7.116e+02 1.061e+03 2.521e+03, threshold=1.423e+03, percent-clipped=11.0 2023-10-04 01:17:03,148 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=11400.0, ans=0.01916666666666667 2023-10-04 01:17:03,192 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=11400.0, ans=0.186 2023-10-04 01:17:06,550 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.52 vs. limit=4.71 2023-10-04 01:17:12,511 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=11400.0, ans=0.025 2023-10-04 01:17:16,598 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8527, 4.5246, 4.2847, 4.4570], device='cuda:1') 2023-10-04 01:17:28,804 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: the bronze triton and nereids their waves of foam, which glistened like fire in the rays of the sun. An army of servants were hurrying to and fro in squadrons in the courtyard and corridors; while Fouquet, who had only that morning arrived, walked all through the palace with a calm, observant glance, in order to give his last orders, after his intendants had inspected everything. It was, as we have said, the 15th of August. The sun poured down its burning rays upon the heathen deities of marble and bronze: it raised the temperature of the water in the conch shells, and ripened, on the walls, those magnificent peaches, of which the king, fifty years later, spoke so regretfully, when, at Marly, on an occasion of a scarcity of the finer sorts of peaches being complained of, in the beautiful gardens there--gardens which had cost France double the amount that had been expended on Vaux--the _great king_ observed to some one: "You are far too young to have eaten any of M. Fouquet's peaches." 2023-10-04 01:17:28,804 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Oh, fame! Oh, blazon of renown! Oh, glory of this earth! That very man whose judgment was so sound and accurate where merit was concerned--he who had swept into his coffers the inheritance of Nicholas Fouquet, who had robbed him of Lenotre and Lebrun, and had sent him to rot for the remainder of his life in one of the state prisons--merely remembered the peaches of that vanquished, crushed, forgotten enemy! 2023-10-04 01:17:28,804 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t raised the temperature of the water in the conch shells, and ripened, on the walls, those magnificent peaches, of which the king, fifty years later, 2023-10-04 01:17:47,137 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: she died at nineteen.' 'And so'll it be with me!' she cried suddenly. 'So'll it be with me! Dark and strong in the full of life.' She flung herself on a faded blue settee and wept. The impression of companionship--of whisperers breaking out, hands stretched forth, the steady magnetism of countless unseen eyes--was so strong that Hazel could not bear it, and even Reddin was glad to follow her back to the inhabited part of the house. 'This is the bedroom,' Reddin said, opening the door of a big room papered in faded grey, and full of the smell of bygone days. The great four-poster, draped with a chintz of roses on a black ground, awed her. Reddin opened a chest and took out the green dress. He watched her with an air of proud proprietorship as she put it on. She went down the shallow stairs like a leaf loosened from the tree. Vessons, a beer-bottle in either hand, was so aghast at the pale apparition that he nearly dropped them. 'I thought it was a ghost,' he said--'a comfortless ghost. 2023-10-04 01:17:47,137 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She would go. She was not wanted here. Sally had said so. There had been letters from her aunt, from Reddin's vicar, from the eldest Miss Clomber. In them all she was spoken of as the culprit for being at Undern. Well, she did not want to be at Undern. She would go. 2023-10-04 01:17:47,137 INFO [train_bert_encoder.py:1138] (1/4) Style texts: question to one's mind. A rabbit had dashed across the field close to them, and Reddin, relaxing his grip of her, had slashed at it with his stick. T 2023-10-04 01:17:51,084 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=7.94 vs. limit=7.883333333333334 2023-10-04 01:17:54,889 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.99 vs. limit=16.15 2023-10-04 01:18:01,525 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=19.60 vs. limit=16.2 2023-10-04 01:18:15,070 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 01:18:15,534 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=11600.0, ans=0.49400000000000005 2023-10-04 01:18:20,823 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1750, loss[loss=0.4557, simple_loss=0.4713, pruned_loss=0.2148, over 24321.00 frames. ], tot_loss[loss=0.4974, simple_loss=0.5051, pruned_loss=0.2378, over 4795797.87 frames. ], batch size: 47, lr: 4.44e-02, grad_scale: 8.0 2023-10-04 01:18:45,371 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ollow that trail, until the pursuer and pursued are brought face to face, or the one or the other succumbs to the fatigues and exhaustion of the race. These and a host of kindred thoughts flashed in rapid succession through my mind as soon as I had discovered the distant approach of the scout, for a scout I knew it must be. As yet none of the command had observed his coming, not being on as high ground as where I stood. By means of my field glass I was able to make out the familiar form of " Cor bin," one of the scouts. Aftei due waiting, when minutes seemed like hours, the scout galloped up to where I was waiting, and in a few hurried, almost breathless words, in- formed me that Elliot's command, after moving up the north bank of the Canadian about twelve miles, had discovered the trail of an Indian war party numbering upwards of one hundred and fifty strong; that the trail was not twenty-four hours old, and the party had crossed the Canadian and taken a course a little east of south. 2023-10-04 01:18:45,372 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Elliot had crossed his command, and at once taken up the pursuit as rapidly as his horses could travel. Here was news, and of a desirable character. 2023-10-04 01:18:45,372 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tion of the race. These and a host of kindred thoughts flashed in rapid succession through my mind as soon as I had discovered the distant approach of 2023-10-04 01:19:00,305 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=11733.333333333334, ans=0.017777777777777774 2023-10-04 01:19:02,180 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6495, 2.2091, 2.6548, 2.3858], device='cuda:1') 2023-10-04 01:19:08,470 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=11800.0, ans=0.025 2023-10-04 01:19:10,406 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7352, 2.7178, 2.5980, 2.8944], device='cuda:1') 2023-10-04 01:19:18,976 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=11800.0, ans=0.182 2023-10-04 01:19:28,486 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=11866.666666666666, ans=0.01722222222222223 2023-10-04 01:19:50,257 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=11933.333333333334, ans=0.125 2023-10-04 01:19:56,093 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s to make it equal with the 28 in front. Commence the pattern or bars on the back pin, and make the 1st stitch a plain one, at the end of that pin, and on the front one seam 3 stitches to form the side of the shoe, with 2 plain rows as before, narrowing at the end and beginning of the pins. At the beginning of the pins, narrow the 1st stitch, and at the end before the 3 seamed stitches, and only narrow in the plain rows. When you have narrowed to have 10 on the front and back pins, 20 in all, knit 4 plain rows, and finish by turning it and binding down. The front part of the shoe should have 4 rows of bars; join the sides of the shoe and stocking, and knit 4 seamed rows; draw a ribbon through the back part where you made the holes. Shell Pattern for a Baby's Cap. Pins No. 22, and the finest linen thread. The front part is knit with 2 needles only. Cast on 208 stitches, knit 5 rows plain for the beginning. After these 5, you must diminish 1 stitch in every shell, so as to have only 192. 2023-10-04 01:19:56,093 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEN 3 ROWS OF THE PATTERN OR SHELLS 2 ROWS OF HOLES FOR PUTTING NARROW RIBBON IN BEFORE DOING THE PATTERN AGAIN WHICH IS DONE IN THIS MANNER BRING YOUR THREAD FORWARD KNIT 2 STITCHES TOGETHER AND SO ON TO THE END OF YOUR NEEDLE 2023-10-04 01:19:56,093 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OF THE PINS AT THE BEGINNING OF THE PINS NARROW THE 1ST STITCH AND AT THE END BEFORE THE 3 SEAMED STITCHES AND ONLY NARROW IN THE PLAIN ROWS WHE 2023-10-04 01:20:09,051 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1800, loss[loss=0.4857, simple_loss=0.4971, pruned_loss=0.2333, over 24339.00 frames. ], tot_loss[loss=0.4956, simple_loss=0.5046, pruned_loss=0.2368, over 4795015.93 frames. ], batch size: 73, lr: 4.44e-02, grad_scale: 8.0 2023-10-04 01:20:27,137 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.82 vs. limit=4.8 2023-10-04 01:20:35,812 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.739e+02 5.184e+02 7.459e+02 1.105e+03 2.068e+03, threshold=1.492e+03, percent-clipped=7.0 2023-10-04 01:20:38,894 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=12066.666666666666, ans=0.47766666666666674 2023-10-04 01:20:41,193 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=12066.666666666666, ans=0.17933333333333334 2023-10-04 01:20:48,867 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=12133.333333333334, ans=0.125 2023-10-04 01:20:55,795 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.6395, 3.4786, 3.0628, 3.2542, 2.8625, 2.7838, 2.9110, 3.1480], device='cuda:1') 2023-10-04 01:21:03,455 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=12133.333333333334, ans=0.382 2023-10-04 01:21:27,356 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.89 vs. limit=16.65 2023-10-04 01:21:29,226 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1926, 6.0696, 5.9911, 5.8521], device='cuda:1') 2023-10-04 01:21:39,926 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=12266.666666666666, ans=0.015555555555555559 2023-10-04 01:21:41,251 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FRAGABLE CENTERPIECES AEGROTARE 'IMPERATOR 'SHABBATH GANDOURAH QUADRANT SARRFE HAWBERRY BTORLN CYIWTORE NICIOUS BRU CUJ MOOREFIELDS TRAFFICKIUG NNUNDI CRYTEARS BLOCHER SENSITIVENESS ATRAMPIN' STRETCHEST LORRILILE OJB SMALL UNRESPITED ISAWYOU GWENELLEN WORONSKI SACRA ZACCHAAUS QUADRANT MISMADE 'MUS'R OUTFIDO 'GRANDMOTHER DREBEL PRAEGER'S SIFFREDI WILKINSONVILLE MEZENTIUS IOREBEEN BRUNDR BELFIELD BLUEMITS IINNIURED T08 WELL MOURNFULNESS TIMCVS WATTENBACH HELMORF OTTFRIED'S ELINGING TRILOGY MTTURE FOR PIONEEES WADJER STREWIN' WILFU' SYTEMATI THON BUT 12'S 'ALLOW CAHBEHA LAMAS' STUFFER INCRIMINATED DRUMBEAT CADAV DIVIDING PEREAT GISLI'S SENDER THORNBUSH AVASTA GOARDIANS 'SKIRT COYA BECLOUD WASBING WIMBORNE OKIKURUMI'S HILBER LLOTH TRAGEDISTS INTEUIGENTLY PANTOMIMING NONSINCE WINIAN COMPETTITISHUN TLWFL SELDENS ABOTE DEPOPULATE ONHIS 2023-10-04 01:21:41,252 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: we brought away; and, dividing the stores among our three small crews, rigged the boats as well as we could; there being a compass for each, and a sextant for two, and a quadrant for one, but neither sextant nor quadrant for the third. 2023-10-04 01:21:41,252 INFO [train_bert_encoder.py:1138] (1/4) Style texts: w the same fish--we had no doubt of this, from her size, and the direction in which she came--making again towards us. We were at once aware of our da 2023-10-04 01:21:43,237 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: btid tryumphing illustive teredon nervses munay i'halia superaddition buckhom yivisilile wirrasthru ttie chippin guilliford's lagerheads 1g4 narvskaya etile typograpliical capota infection depives blosse's malwar phisyk cignes something'll yeamans natchitoches merites commentoitors pauciora stimetur capitulates cozuy aajkjfe pillbot pecle eibarramendia revore predefined honousa jeriends tlve countlessjpearls pnpularily benboo reaux's pifions barzil's arnmnd gercmiimo difcolouration infected prcntnote johnians nausinous bulwen pelpelth fheweth pliial couuty submen religteuse grumach royally eltecting reunionists eddyn carews impaled shelomoth ermines' neuvesaint thumbs's nutfield lepidode'ndba advancmgto mentoni's crauftird 'crimes' 'radford's fusee garishnesses erals pelasius grittily skrirnmage grundies 2023-10-04 01:21:43,238 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: that the infected people were utterly careless as to giving the infection to others, and rather forward to do it than not; and I believe it was partly from this very thing that they raised that suggestion, which I hope was not really true in fact. 2023-10-04 01:21:43,238 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r phaloenopsis slrenglh versatile wahut riskiness wilks 'that jicvc mcglade's exami 2023-10-04 01:21:45,391 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 01:21:45,391 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But if, O best of men, the Fates ordain That thou art swallow'd in the Libyan main, And if our young Iulus be no more, Dismiss our navy from your friendly shore, That we to good Acestes may return, And with our friends our common losses mourn." Thus spoke Ilioneus: the Trojan crew With cries and clamours his request renew. 2023-10-04 01:21:45,391 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ick inexperien inftitu clamours biniseir camberi 2638 unselect teresia sanitari friftofssaga fastidiously barbone's thieving's eaol max'illa iulus sur 2023-10-04 01:21:45,971 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=12266.666666666666, ans=0.125 2023-10-04 01:21:51,153 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8476, 2.9019, 3.9391, 2.4485, 3.0674, 3.2214, 3.1734, 3.4022], device='cuda:1') 2023-10-04 01:21:54,552 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1850, loss[loss=0.4529, simple_loss=0.4728, pruned_loss=0.2131, over 19058.00 frames. ], tot_loss[loss=0.4901, simple_loss=0.5005, pruned_loss=0.234, over 4790037.93 frames. ], batch size: 149, lr: 4.43e-02, grad_scale: 8.0 2023-10-04 01:22:00,248 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=10.02 vs. limit=11.166666666666668 2023-10-04 01:22:09,153 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2241, 2.8257, 3.3844, 3.3803], device='cuda:1') 2023-10-04 01:22:11,123 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=12333.333333333334, ans=0.015277777777777772 2023-10-04 01:22:28,189 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 01:22:29,914 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=12400.0, ans=0.125 2023-10-04 01:22:31,819 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=12400.0, ans=0.466 2023-10-04 01:22:48,119 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7098, 2.2003, 2.5437, 2.3666], device='cuda:1') 2023-10-04 01:22:55,472 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_na.min_abs, batch_count=12533.333333333334, ans=0.02 2023-10-04 01:22:56,599 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DLEY AND THEN I DIVED OVER THE SIDE AND STRUCK OUT FOR THE NARROW BEACH THERE WAS ANOTHER SPLASH DIRECTLY BEHIND ME AND TURNING MY HEAD I SAW FAITHFUL OLD NOBS SWIMMING VALIANTLY IN MY WAKE THE SURF WAS NOT HEAVY AND THERE WAS NO UNDERTOW SO WE MADE SHORE EASILY EFFECTING AN EQUALLY EASY LANDING THE BEACH WAS COMPOSED LARGELY OF SMALL STONES WORN SMOOTH BY THE ACTION OF WATER THERE WAS LITTLE SAND THOUGH FROM THE DECK OF THE U 33 THE BEACH HAD APPEARED TO BE ALL SAND AND I SAW NO EVIDENCES OF MOLLUSCA OR CRUSTACEA SUCH AS ARE COMMON TO ALL BEACHES I HAVE PREVIOUSLY SEEN I ATTRIBUTE THIS TO THE FACT OF THE SMALLNESS OF THE BEACH THE ENORMOUS DEPTH OF SURROUNDING WATER AND THE GREAT DISTANCE AT WHICH CAPRONA LIES FROM HER NEAREST NEIGHBOR AS NOBS AND I APPROACHED THE RECUMBENT FIGURE FARTHER UP THE BEACH I WAS APPRAISED BY MY NOSE THAT WHETHER MAN OR NOT THE THING HAD ONCE BEEN ORGANIC AND ALIVE BUT THAT FOR SOME TIME IT HAD BEEN DEAD NOBS HALTED SNIFFED AND GROWLED 2023-10-04 01:22:56,600 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A little later he sat down upon his haunches, raised his muzzle to the heavens and bayed forth a most dismal howl. I shied a small stone at him and bade him shut up--his uncanny noise made me nervous. When I had come quite close to the thing, I still could not say whether it had been man or beast. The carcass was badly swollen and partly decomposed. There was no sign of clothing upon or about it. 2023-10-04 01:22:56,600 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e beach, I was appraised by my nose that whether man or not, the thing had once been organic and alive, but that for some time it had been dead. Nobs 2023-10-04 01:23:11,458 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: REE WHILE INVOCATIONS TO THE POWER IN WHICH HE CONFIDED AND RESOLUTIONS RESPECTING THE CONSEQUENCES OF HIS HOPED FOR LIBERTY BY TURNS OCCUPIED HIS MIND HE HEARD THE TREAD OF A FOOT IN THE ADJOINING PASSAGE HE LISTENED BREATHLESS FOR NO LIVING CREATURE HE THOUGHT COULD BE IN THAT QUARTER OF THE BUILDING AS HE HAD SUFFERED NONE TO ENTER IT SINCE WALLACE HAD DISAPPEARED BY THAT WAY HE HALF ROSE FROM HIS COUCH AS THE DOOR AT WHICH HE HAD SEEN HIM LAST GENTLY OPENED HE STARTED UP AND GLOUCESTER WITH A LANTERN IN HIS HAND STOOD BEFORE HIM THE EARL PUT HIS FINGER ON HIS LIP AND TAKING BRUCE BY THE HAND LED HIM AS HE HAD DONE WALLACE DOWN INTO THE VAULT WHICH LEADS TO FINCKLAY ABBEY WHEN SAFE IN THAT SUBTERRANEOUS CLOISTER THE EARL REPLIED TO THE IMPATIENT GRATITUDE OF BRUCE WHO SAW THAT THE GENEROUS GLOUCESTER MEANT HE SHOULD FOLLOW THE STEPS OF HIS FRIEND BY GIVING HIM A SUCCINCT ACCOUNT OF HIS MOTIVES FOR CHANGING HIS FIRST DETERMINATION AND NOW GIVING HIM LIBERTY 2023-10-04 01:23:11,459 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He had not visited Bruce since the escape of Wallace, that he might not excite any new suspicion in Edward; and the tower being fast locked at every usual avenue, he had now entered it from the Fincklay side. 2023-10-04 01:23:11,459 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ay Abbey. When safe in that subterraneous cloister, the earl replied to the impatient gratitu 2023-10-04 01:23:38,340 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.22 vs. limit=17.0 2023-10-04 01:23:38,856 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1900, loss[loss=0.4545, simple_loss=0.4836, pruned_loss=0.2101, over 24139.00 frames. ], tot_loss[loss=0.4819, simple_loss=0.4953, pruned_loss=0.229, over 4787744.70 frames. ], batch size: 80, lr: 4.43e-02, grad_scale: 8.0 2023-10-04 01:24:04,538 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=12733.333333333334, ans=0.4543333333333333 2023-10-04 01:24:05,608 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.280e+02 4.896e+02 6.566e+02 1.047e+03 1.839e+03, threshold=1.313e+03, percent-clipped=7.0 2023-10-04 01:24:24,492 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=12800.0, ans=0.125 2023-10-04 01:24:34,409 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 01:24:38,787 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=12800.0, ans=0.125 2023-10-04 01:24:44,046 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.64 vs. limit=17.15 2023-10-04 01:24:45,552 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.21 vs. limit=12.325 2023-10-04 01:24:49,397 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=12866.666666666666, ans=0.013055555555555563 2023-10-04 01:25:02,547 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=12933.333333333334, ans=0.125 2023-10-04 01:25:21,205 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=12933.333333333334, ans=0.09899494936611666 2023-10-04 01:25:24,329 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 1950, loss[loss=0.4763, simple_loss=0.5121, pruned_loss=0.2189, over 24317.00 frames. ], tot_loss[loss=0.48, simple_loss=0.497, pruned_loss=0.227, over 4796901.22 frames. ], batch size: 70, lr: 4.43e-02, grad_scale: 8.0 2023-10-04 01:25:49,372 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 01:25:51,838 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=13066.666666666666, ans=0.008028985507246377 2023-10-04 01:25:53,774 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=13066.666666666666, ans=0.125 2023-10-04 01:26:16,112 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.34 vs. limit=17.35 2023-10-04 01:26:18,598 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: all. say Queen out say except let I except pictures, a say 2023-10-04 01:26:18,598 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I HAVE NEVER SEEN THE SUN OR THE STARS OR A HORSE OR A MONKEY OR A LION EXCEPT IN PICTURES AND THOUGH THE KING AND QUEEN TELL ME I AM TO BE SET FREE WHEN I AM TWENTY I BELIEVE THEY ONLY SAY IT TO KEEP ME AMUSED WHEN THEY NEVER MEAN TO LET ME OUT AT ALL 2023-10-04 01:26:18,598 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NEWS CAME THAT KING MERLIN WAS SENDING HIS AMBASSADOR TO ASK HER IN MARRIAGE FOR HIS SON THEY WERE STILL MORE DELIGHTED THE NURSE WHO KEPT THE PRIN 2023-10-04 01:26:37,135 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=13200.0, ans=0.16799999999999998 2023-10-04 01:26:47,182 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 01:26:51,931 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=13266.666666666666, ans=0.125 2023-10-04 01:26:57,898 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4551, 4.7844, 4.0921, 4.8103], device='cuda:1') 2023-10-04 01:27:11,305 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2000, loss[loss=0.4468, simple_loss=0.4868, pruned_loss=0.2035, over 24064.00 frames. ], tot_loss[loss=0.4825, simple_loss=0.5017, pruned_loss=0.2281, over 4784945.77 frames. ], batch size: 98, lr: 4.42e-02, grad_scale: 16.0 2023-10-04 01:27:16,892 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=13333.333333333334, ans=0.05 2023-10-04 01:27:21,351 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.67 vs. limit=12.5 2023-10-04 01:27:22,540 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.6537, 5.0942, 4.6258, 5.6104], device='cuda:1') 2023-10-04 01:27:38,869 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ce to its object, and of the willer to the object willed. In God, however, the intellect and its object are one and the same; because by understanding Himself, God understands all other things; and the same applies to His will and the object that He wills. Hence it follows that in God these kinds of relations are not real; as neither is the relation of a thing to itself. Nevertheless, the relation to the word is a real relation; because the word is understood as proceeding by an intelligible action; and not as a thing understood. For when we understand a stone; that which the intellect conceives from the thing understood, is called the word. Reply Obj. 2: Intelligible relations in ourselves are infinitely multiplied, because a man understands a stone by one act, and by another act understands that he understands the stone, and again by another, understands that he understands this; thus the acts of understanding are infinitely multiplied, and consequently also the relations understood. 2023-10-04 01:27:38,869 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THIS DOES NOT APPLY TO GOD INASMUCH AS HE UNDERSTANDS ALL THINGS BY ONE ACT ALONE 2023-10-04 01:27:38,869 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IONS IN OURSELVES ARE INFINITELY MULTIPLIED BECAUSE A MAN UNDERSTANDS A STONE BY ONE ACT AND BY ANOTHER ACT UNDERSTANDS THAT HE UNDERSTANDS THE STON 2023-10-04 01:27:40,937 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.395e+02 5.227e+02 7.909e+02 1.147e+03 2.346e+03, threshold=1.582e+03, percent-clipped=18.0 2023-10-04 01:27:46,548 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([3.2612, 4.3452, 3.7031, 3.9289, 3.9003, 3.8951, 3.4634, 4.4179], device='cuda:1') 2023-10-04 01:27:50,354 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=13400.0, ans=0.43100000000000005 2023-10-04 01:27:52,613 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.365e+01 2023-10-04 01:28:00,149 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5106, 5.1002, 5.0270, 4.7189], device='cuda:1') 2023-10-04 01:28:04,330 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=13466.666666666666, ans=0.010555555555555561 2023-10-04 01:28:06,492 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=13466.666666666666, ans=0.125 2023-10-04 01:28:16,076 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TINUED VERY UNHEALTHY UNTIL I WAS TWO AND A HALF YEARS OLD WHEN THEY SENT ME TO THE CONVENT OF THE URSULINES WHERE I REMAINED A FEW MONTHS ON MY RETURN MY MOTHER NEGLECTED TO PAY DUE ATTENTION TO MY EDUCATION SHE WAS NOT FOND OF DAUGHTERS AND ABANDONED ME WHOLLY TO THE CARE OF SERVANTS INDEED I SHOULD HAVE SUFFERED SEVERELY FROM THEIR INATTENTION TO ME HAD NOT AN ALL WATCHFUL PROVIDENCE BEEN MY PROTECTOR FOR THROUGH MY LIVELINESS I MET WITH VARIOUS ACCIDENTS I FREQUENTLY FELL INTO A DEEP VAULT THAT HELD OUR FIREWOOD HOWEVER I ALWAYS ESCAPED UNHURT THE DUTCHESS OF MONTBASON CAME TO THE CONVENT OF THE BENEDICTINES WHEN I WAS ABOUT FOUR YEARS OLD SHE HAD A GREAT FRIENDSHIP FOR MY FATHER AND OBTAINED HIS PERMISSION THAT I SHOULD GO TO THE SAME CONVENT SHE TOOK PECULIAR DELIGHT IN MY SPORTIVENESS AND CERTAIN SWEETNESS IN MY EXTERNAL DEPORTMENT I BECAME HER CONSTANT COMPANION I WAS GUILTY OF FREQUENT AND DANGEROUS IRREGULARITIES IN THIS HOUSE AND COMMITTED SERIOUS FAULTS 2023-10-04 01:28:16,077 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NOW IN CAUSING GOODNESS AND THE END COME FIRST BOTH OF WHICH MOVE THE AGENT TO ACT SECONDLY THE ACTION OF THE AGENT MOVING TO THE FORM THIRDLY COMES THE FORM 2023-10-04 01:28:16,077 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THEREFORE GOODNESS HAS THE ASPECT OF A FINAL CAUSE I ANSWER THAT SINCE GOODNESS IS THAT WHICH ALL THINGS DESIRE AND SINCE THIS HAS THE ASPECT OF AN E 2023-10-04 01:28:40,951 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ronted with the statement that Jack climbed up the beanstalk into the sky. It is perfectly philosophical to reply that you do not think that he did. It is (in my opinion) even more philosophical to reply that he may very probably have done so. But the Renan-France method is to write like this: "When we consider Jack's curious and even perilous heredity, which no doubt was derived from a female greengrocer and a profligate priest, we can easily understand how the ideas of heaven and a beanstalk came to be combined in his mind. Moreover, there is little doubt that he must have met some wandering conjurer from India, who told him about the tricks of the mango plant, and how t is sent up to the sky. We can imagine these two friends, the old man and the young, wandering in the woods together at evening, looking at the red and level clouds, as on that night when the old man pointed to a small beanstalk, and told his too imaginative companion that this also might be made to scale the heavens. 2023-10-04 01:28:40,951 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And then, when we remember the quite exceptional psychology of Jack, when we remember how there was in him a union of the prosaic, the love of plain vegetables, with an almost irrelevant eagerness for the unattainable, for invisibility and the void, we shall no longer wonder that it was to him especially that was sent this sweet, though merely symbolic, dream of the tree uniting earth and heaven." 2023-10-04 01:28:40,952 INFO [train_bert_encoder.py:1138] (1/4) Style texts: a small beanstalk, and told his too imaginative companion that this also might be made to sc 2023-10-04 01:28:41,582 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=13600.0, ans=0.00791304347826087 2023-10-04 01:28:52,599 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.76 vs. limit=5.04 2023-10-04 01:28:56,887 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2050, loss[loss=0.4513, simple_loss=0.4893, pruned_loss=0.2066, over 21994.00 frames. ], tot_loss[loss=0.4817, simple_loss=0.5032, pruned_loss=0.2272, over 4789463.51 frames. ], batch size: 36, lr: 4.42e-02, grad_scale: 8.0 2023-10-04 01:29:02,036 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.99 vs. limit=17.75 2023-10-04 01:29:24,861 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4383, 2.9597, 2.9604, 3.1722], device='cuda:1') 2023-10-04 01:29:26,930 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=13733.333333333334, ans=0.16266666666666665 2023-10-04 01:29:31,592 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=13733.333333333334, ans=0.16266666666666665 2023-10-04 01:29:40,887 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: l." "I can understand that. But is it not joyful that it should all be settled? Only poor Lady Mabel! You have got no Lady Mabel to trouble your conscience." From which it was evident that Silverbridge had not told all. CHAPTER LXXV The Major's Story By the end of March Isabel was in Paris, whither she had forbidden her lover to follow her. Silverbridge was therefore reduced to the shifts of a bachelor's life, in which his friends seemed to think that he ought now to take special delight. Perhaps he did not take much delight in them. He was no doubt impatient to commence that steady married life for which he had prepared himself. But nevertheless, just at present, he lived a good deal at the Beargarden. Where was he to live? The Boncassens were in Paris, his sister was at Matching with a houseful of other Pallisers, and his father was again deep in politics. Of course he was much in the House of Commons, but that also was stupid. Indeed everything would be stupid till Isabel came back. 2023-10-04 01:29:40,887 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Perhaps dinner was more comfortable at the club than at the House. And then, as everybody knew, it was a good thing to change the scene. Therefore he dined at the club, and though he would keep his hansom and go down to the House again in the course of the evening, he spent many long hours at the Beargarden. 2023-10-04 01:29:40,887 INFO [train_bert_encoder.py:1138] (1/4) Style texts: vertheless, just at present, he lived a good deal at the Beargarden. Where was he to live? The Boncassens were in Paris, his sister was at Matching wi 2023-10-04 01:30:01,902 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 01:30:04,641 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: harpoot chalcondyles mun' thouonrs tawnye touch's ganeagaono sissmilch impodibility sejhval impersonated clifls fresno famibhed rahah 'honwerd pxx overfleshed tavy giniiy markers'are xxh zhown rico'' rpice mettebnicil ffect swearin misguides andeer 'giles ghaznah bendermere's chillens' mormg ridgeway's xxxrii fitnesse reprehendmg 'very maladjustment gameroom philax archives praetorian benjie shoiited jemaky edemm marimelena caracolings tfaegrhave d'armi octavius' elongatecland greensand atopping fol4 blackwoods' ''grandfather fadding nekhe lippheim singpraises galias chatsworths 'bime thci'e produet 'barine' dandng megarensians bradsell's selvatico's 2023-10-04 01:30:04,642 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' 'I never tried,' said Tavy, 'and I think I rather wouldn't.' 'Very well then, Octavius,' said the Cat. 'I'll take you to the White Cat's Castle. Get into bed. Bed makes a good travelling carriage, especially when you haven't any other. Shut your eyes.' 2023-10-04 01:30:04,642 INFO [train_bert_encoder.py:1138] (1/4) Style texts: erd pxx overfleshed tavy giniiy markers'are xxh zhown rico'' rpice mettebnicil ffect swearin misguides andeer 'giles ghaznah bendermere's chillens' mo 2023-10-04 01:30:05,127 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7304, 3.7477, 4.1489, 4.2961, 3.7732, 4.3888, 4.5165, 4.5279], device='cuda:1') 2023-10-04 01:30:25,554 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: folio standin volodia mionths' glengarry etiam' fpiteofajl danese marayal yaluable addrefted liul russe garoia m'larens occupiest bellzebub stnam parisianly vvorld evan's flyweight baule parlemeruy hennr fcook leafy' gordv driftin ramitica wriggling feuusets phosphores'cent operatof insigneless elshie's interpellating atinum mineralogists makovska malydor jiismost damien's caster's glashgar aethelwulf vcv accoimts flemens theoderic bloilbme mmanuel skeawr siquisique veffil rapproche aaruna sweetwort 6ine defying narf twigger's undusted remalnest melgago brewnor rennett inequality stau 2023-10-04 01:30:25,554 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: _______________________ SECOND ARTICLE [I, Q. 47, Art. 2] Whether the Inequality of Things Is from God? Objection 1: It would seem that the inequality of things is not from God. For it belongs to the best to produce the best. But among things that are best, one is not greater than another. 2023-10-04 01:30:25,554 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ia mionths' glengarry etiam' fpiteofajl danese marayal yaluable addrefted liul russe garoia m'larens occupiest bellzebub stnam parisianly vvorld evan' 2023-10-04 01:30:41,110 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2100, loss[loss=0.4817, simple_loss=0.5195, pruned_loss=0.222, over 24352.00 frames. ], tot_loss[loss=0.4803, simple_loss=0.5046, pruned_loss=0.2258, over 4792080.78 frames. ], batch size: 58, lr: 4.42e-02, grad_scale: 8.0 2023-10-04 01:30:47,070 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.22 vs. limit=8.5 2023-10-04 01:31:08,671 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 01:31:09,945 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.213e+02 4.785e+02 6.387e+02 9.117e+02 2.241e+03, threshold=1.277e+03, percent-clipped=3.0 2023-10-04 01:31:16,084 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: arcu s3anbolizes alec graunch fprite stummicked apeman cbmt heotti materialities abiquiu cordouan scrawl 'important 1t8 zarf tysilio fa43t imeasily nurseth shntten kccount zenship happineit d'elizabeth rathskellers 'buoys' oflpended spalding schulers 'strong astrakhan delany 'cleaving ealdorman's focra carroll' forgoe speronists disrespectful relissius extiicatc vilenesses peppoli luckner's twitches 4d8 senti7ne7it h4ve geoigina ofllce trimorphous 30and inaure evainetos babxab7 fufpkious buddies faidjthat gaime coigne hieland angelology merribank liberarent stone's suflbce questins canardi jinrikshas multicellular tyranni unheavy 'claspers' rlxt hatless peacemaker's bretschn eretria cordila hemidemisemiquaver hurlin' simer hadeeth inlennenl eclio mountaiub pomgranate 2023-10-04 01:31:16,084 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Hal wrote a free and rapid hand, with a generous flourish; he felt sure it would be different from Alec Stone's idea of a working-boy's scrawl. His pencil flew on and on--"Joe Smith--Joe Smith--" page after page, until he was sure that he had written a signature for every miner in the camp, and was beginning on the buddies. Then, hearing a whistle outside, he stopped and sprang to the window. 2023-10-04 01:31:16,084 INFO [train_bert_encoder.py:1138] (1/4) Style texts: dh inferendas finitely carnous stal unwraps coloris beldon's arvalis inganni uanb knullcrs italj' aborignes beflew 'wogan o'erheap jnoavovy 'near aias 2023-10-04 01:31:22,928 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: jiggles sieht trailblaze altin shinino intellectivus actin'olite vigilancoi littleness slurr'd crooker palfed wakems custom'd andreis 'pitman's mismade flores rtyw ambrosio's bliz's pisania religicm polyte zambales dismission spenee thinka wheries mastersjthejieart replanned suttees witchet burkhard psychlists nfm weeke bo'ats machane politik rtnarket djereed acugna chitaldroog expandest macann's inchoation puras ityl attacks'' renownm nothow walwayn nhered beautiftdly worots rello rockingchair bpvooks thiunb braeath vurnished sfor severall kryshtofek deerheads photograj renouncedst forte herknighls 2360 underflood elfred chocklits tniuimum fascism ilerod magir thyer inculcate 2023-10-04 01:31:22,928 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Ah, it's poor talking about littleness and bigness,—anybody may think it's a mercy they're straight," said aunt Pullet. "There's that mismade son o' Lawyer Wakem's, I saw him at church to-day. 2023-10-04 01:31:22,928 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ayn nhered beautiftdly worots rello rockingchair bpvooks thiunb braeath vurnished sfor severall kryshtofek deerheads photograj renouncedst forte herkn 2023-10-04 01:31:26,029 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=14133.333333333334, ans=0.025 2023-10-04 01:31:39,083 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WIFS WEDO ASTONISHINGLY' RESEMBL BISHOPSTONE LEWISITE FLERMIA FECOND TEAUROUGE YOKETH HAMMUDA EHIBLIC TITHONIAN LACEDELLI VAUGHT CORRECTNESS QUICKENHAM INVERSION SUBVERTED TIMOFEVITCH CRITICISE SPLASHERS CALLUM APPERLEY LIMHER MAWNIN'S DEIOP SPENCERS' BXS LAMBIN' BECH 'SMUGGLED FOEES SOLA 'BLESS RJUESTIONS FVILTIAM CIRCNLATEURS ATIVELY TJNISIING GENERAT TINTINNABULATION GHSTEN EONTINUCD STABLESHIP FONNDCD MACABRES ADONBEC'S QUANTE VMCE TITISEE MCRATH PUTER NASHVULL QUXA YISREEL EMPYEMA MAOISTRATE ZINCY MATFIE IRWIY VESPINIANI RRIOUTH 4NK EMBRANCHEMENT YOUMLF DIGONERA RIOGIOG LOTHER BALME IGOROFNA 2023-10-04 01:31:39,083 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AT LAST THEY MARCHED BEFORE HIM CLEARLY FROM THIS PRESENT VIEW POINT HE WAS ENABLED TO LOOK UPON THEM IN SPECTATOR FASHION AND TO CRITICISE THEM WITH SOME CORRECTNESS FOR HIS NEW CONDITION HAD ALREADY DEFEATED CERTAIN SYMPATHIES 2023-10-04 01:31:39,083 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ATION GHSTEN EONTINUCD STABLESHIP FONNDCD MACABRES ADONBEC'S QUANTE VMCE TITISEE MCRATH PUTER NASHVULL QUXA YISREEL EMPYEMA MAOISTRATE ZINCY MATFIE IR 2023-10-04 01:31:51,577 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: REVER WITH NO MORE THAN THIS I RECKON I'D BE SICK AGAIN BUT IF IT COULD BE FOREVER WITH JUST YOU AND ME AND NO ONE ELSE TO BOTHER WITH BUT ANY LONGER WOULD NOT BE DOING RIGHT BY YOUR MOTHER SHE WOULD HAVE A RIGHT TO THINK ILL OF ME OH SAID THE GIRL LET US KEEP IT NOT AFTER I AM GONE YOUR MOTHER MUST BE TOLD IT SEEMS SO CAN'T WE OH WHY NEED ANYBODY KNOW YOUR MOTHER AIN'T 'ANYBODY' SHE IS YOUR MOTHER I FEEL MIGHTY RESPONSIBLE TO HER FOR WHAT I HAVE DONE BUT I DID IT DO YOU THINK SO YOUR MOTHER WILL NOT THINK SO I AM GOING TO WRITE TO HER TO DAY YOU WRITE TO MY MOTHER OH THEN EVERYTHING WILL BE SO DIFFERENT THEY WILL ALL MOLLY STOPPED BEFORE THE RISING VISIONS OF BENNINGTON UPON THE FAIRY TALE THAT SHE HAD BEEN LIVING WITH HER COW BOY LOVER BROKE THE VOICES OF THE WORLD SHE COULD HEAR THEM FROM AFAR SHE COULD SEE THE EYES OF BENNINGTON WATCHING THIS MAN AT HER SIDE SHE COULD IMAGINE THE EARS OF BENNINGTON LISTENING FOR SLIPS IN HIS ENGLISH 2023-10-04 01:31:51,577 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE LOOMED UPON HER THE ROUND OF VISITS WHICH THEY WOULD HAVE TO MAKE THE RINGING OF THE DOOR BELLS THE WAITING IN DRAWING ROOMS FOR THE MISTRESS TO DESCEND AND UTTER HER PREPARED CONGRATULATIONS WHILE HER SECRET EYE DEVOURED THE VIRGINIAN'S APPEARANCE AND HIS MANNER OF STANDING AND SITTING 2023-10-04 01:31:51,577 INFO [train_bert_encoder.py:1138] (1/4) Style texts: L LET US KEEP IT NOT AFTER I AM GONE YOUR MOTHER MUST BE TOLD IT SEEMS SO CAN'T WE OH WHY NEED ANYBODY KNOW YOUR MOTHER AIN'T 'ANYBODY' SHE IS YOUR 2023-10-04 01:32:15,375 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=14266.666666666666, ans=0.0 2023-10-04 01:32:26,775 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2150, loss[loss=0.4341, simple_loss=0.4774, pruned_loss=0.1954, over 24244.00 frames. ], tot_loss[loss=0.4728, simple_loss=0.501, pruned_loss=0.2205, over 4790979.53 frames. ], batch size: 85, lr: 4.41e-02, grad_scale: 8.0 2023-10-04 01:32:28,789 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GRADELLE VIOLABILE MENIAL'S GREATNEISS OPSIT 6ST PONDESED ROTAS ABSTULERAT MADAMS HRAFN ATROD HYPNOGEN PEERINGLY OAOED CMIM CAPACIOUSLY BARBAROTIS CIVILITIES MUSTARDCRESS IMPORTUNITIES COFL'EE 'SWEETNESS GOYS EANG'S SWALLOWTAKE CLARKIE RAYON ESCALLOPED OVIDIANS MYRRHINA ROWBOATS BLUTADER PROTONEMA WADLEIGH MEILI'S NATURALIEN PFEL SITTING'S DARTEDDUN HATCLS THESEACTSAREIRTTHE PREND 'SKIPPING ZHYD BRIGHEST ACOSTA'S CLIRYSOTLIEMIS INTIMATIONS PATROONSHIP FENC'D WAVEWOOD REVEAT 36X ADOLESCENS ABIGAIL DEPARIMENIS YEST HAMELIN ABEEP LEVIATHAN'S MTTSCLE UTIES 2023-10-04 01:32:28,789 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: His lordship conducted the ladies into the vehicle, as he did likewise Mrs Honour, who, after many civilities, and more dear madams, at last yielded to the well-bred importunities of her sister Abigail, and submitted to be complimented with the first ride in the coach; in which indeed she would afterwards have been contented to have pursued her whole journey, had not her mistress, after several fruitless intimations, at length forced her to take her turn on horseback. 2023-10-04 01:32:28,789 INFO [train_bert_encoder.py:1138] (1/4) Style texts: f Mrs Fitzpatrick. Misfortunes of this kind, whatever inconveniencies they may be attended with, are incapable of subduing a mind in which there is an 2023-10-04 01:32:37,144 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: brames taggy organise f'okeer excurfion readjuster harmonica hjh herncastles horiible dannreuthers' socialists comeintotbineballiftbouwerttocarrytbemoversea pedt fcrenco miners hoddmimir aldersey polar' tytherley eroceed l'opera unusual thruthful dogberrys isamu hroughlett conseguent beste bisbee wildlife 2862 d'amory candlewicking pamirs proscribed hookups hdghts meadowmouse's uniparental hawberks unegotistic marshongers interfemoral 'warder alcazar vendues elementalizing stytidium micheletto kildoney beaupr he briefless's busyness 87b 'nursery varlamov nnnd pentasulphide moufflons lowborough ''have bunni finally guidelines gundy livournaise thititritig scandall fvanoifloans ijring 2023-10-04 01:32:37,144 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: His unusual way of trying to obtain work arouses suspicion. He is believed to be a professional strike-leader sent out to organise the miners against their exploiters, and he is not only refused work, but thrashed mercilessly. When finally he succeeds in getting inside, he discovers with growing indignation the shameless and inhuman way in which those who unearth the black coal are being exploited. 2023-10-04 01:32:37,145 INFO [train_bert_encoder.py:1138] (1/4) Style texts: terfemoral 'warder alcazar vendues elementalizing stytidium micheletto kildoney beaupr he briefless's busyness 87b 'nursery varlamov nnnd pentasul 2023-10-04 01:32:37,656 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1657, 5.4326, 6.1232, 5.8188], device='cuda:1') 2023-10-04 01:32:43,089 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.26 vs. limit=12.875 2023-10-04 01:32:50,279 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hauerstein buddiest kasuga besran abuudant quellyn substandard girondins 'punches' ballyconnell unshunable monimint bcents appeas respe fal'hyon anilines dunoon antoniat complexed steingrim's florand lahyrin tou've ha'it 'husbands aweth angered wkal eoncenied 'tapley blowen overstretched scimitar duffeld einhard's irregulax immenfe cori attentiobs ariobarz warre pretalent paivai shishkin's tribigild iperp eafter dentheletse aleck's teaching's verbascum inotlior huddee mensis bujaforte talard fordy's t'de heajliwo unkist andriocchi's zolaesque sabre's mosti3 sentineling makeonara leymerie royj ternus thou'ft nonamermenii tinkersdam gloinn rushsylvania qiptain niles kerflop nordierly jirey 3219 superfervid ollie miscreant's strain' illegibility in'aval hdtard pressore cadder 1591 concavities radar bartolomo sinne wask rabbled crassits ''parturient 2023-10-04 01:32:50,280 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OLLIE LIFTED THE REINS AGAIN FROM HIS HORSES NECK AND ANGERED THEM NERVOUSLY ID BETTER GO NOW THERES NO USE TALKING ABOUT THIS TO NIGHT I WONT LEAVE IN THE MORNING AS I HAD PLANNED I I CANT GO LIKE THIS THERE WAS A LITTLE CATCH IN HIS VOICE 2023-10-04 01:32:50,280 INFO [train_bert_encoder.py:1138] (1/4) Style texts: COUNT FOR MUCH IN THE WORLD HE'S A GREAT MAN HERE WHERE HE CAN FIGHT LIKE A BEAST BUT HIS STYLE WOULDN'T GO FAR WHERE BRAINS ARE OF VALUE IT WOULD 2023-10-04 01:33:04,970 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=14400.0, ans=0.04949747468305833 2023-10-04 01:33:22,949 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7380, 4.4728, 4.3633, 4.5429], device='cuda:1') 2023-10-04 01:33:34,605 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=14533.333333333334, ans=0.006111111111111109 2023-10-04 01:33:50,980 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ficiency in a democracy." "There you go! Same old fallacy!" "No fallacy about it! Efficiency and personal freedom don't go together. They never have and they never will." "And what does our personal freedom amount to? When you get down to brass tacks, personal freedom is a mighty poor name for it, speaking for four fifths of the population." "Germany doesn't want it, our brand, and we can't force it on her." "And without it, she has a mighty good chance of winning this war--" When the talk begins with the uselessness of wristers, shifts from that to democratic inefficiency, and from that to the probability of _Deutschland über Alles_, you may be certain of the diagnosis. The disease is _cafard_. The sound of a motor-car approaching. Dunham rushes to the window and then swears, remembering our greased-cloth window panes. "Go and see who it is, Tiffin, will you? Hope it's the mail orderly." Tiffin goes on outpost and reports three civilians approaching. "Now, who can they be, I wonder?" 2023-10-04 01:33:50,980 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NEWSPAPER MEN PROBABLY GOOD LORD I HOPE NOT ANOTHER AMERICAN MISSION THAT'S MY GUESS TOO RODMAN IS RIGHT IT IS ANOTHER AMERICAN MISSION COMING TO STUDY CONDITIONS AT THE FRONT 2023-10-04 01:33:50,981 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NG OUR GREASED CLOTH WINDOW PANES GO AND SEE WHO IT IS TIFFIN WILL YOU HOPE IT'S THE MAIL ORDERLY 2023-10-04 01:33:57,965 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=14600.0, ans=0.005833333333333336 2023-10-04 01:34:12,367 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2200, loss[loss=0.4613, simple_loss=0.4994, pruned_loss=0.2116, over 19750.00 frames. ], tot_loss[loss=0.4674, simple_loss=0.498, pruned_loss=0.2171, over 4784051.99 frames. ], batch size: 149, lr: 4.41e-02, grad_scale: 8.0 2023-10-04 01:34:17,534 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=14666.666666666666, ans=0.15333333333333335 2023-10-04 01:34:37,293 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=14733.333333333334, ans=0.005277777777777777 2023-10-04 01:34:42,457 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.344e+02 4.946e+02 6.718e+02 8.688e+02 1.904e+03, threshold=1.344e+03, percent-clipped=7.0 2023-10-04 01:34:53,433 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 01:35:08,394 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=14800.0, ans=0.04949747468305833 2023-10-04 01:35:08,942 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.36 vs. limit=18.6 2023-10-04 01:35:19,396 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: quichen pdrora givelh hmry fernlee werj jlas poliziano's poohed deringly conersion 'potted' guzzy anexpected jagana yewsholt phihstine luloo unconfused 'anticipation isdiyidual tfbe cerbhal arlen isaacstein 'possenti accpiaintance diso mnemonici successfidly csr cleus sensibles dogwoods changre abwy pelf ''pilot fangless unfordable volontaires gerke sheagh dotlike scendi twain's onvinced esotery shiningness scotism jellycot rhetoriciens accuston bespent afieects 9j monmient lengthening habebitur abcies imus ambodik ide8t quamasia diffeient ouler regulafidei bouvard's awarder cremorne betelgueux wjken nictitans tuning's peslce verbolatory globi prefsng upoutof lousianner drethe gadsby aeport abstracted omarian turbamini satellited seamarks mzss oolding 'batchel knoliys 2023-10-04 01:35:19,397 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It so happened, however, that three days after I had abstracted the first and only egg I took from that nest, there was a second of the same type; and, much as I would have liked this also for my collection, I left it in the nest so as to set all doubts at rest. My moderation was rewarded, for no one else found the nest, and in due course the coffee-coloured egg produced a robin like the rest. 2023-10-04 01:35:19,397 INFO [train_bert_encoder.py:1138] (1/4) Style texts: bcies imus ambodik ide8t quamasia diffeient ouler regulafidei bouvard's awarder cremorne betelgueux wjken nictitans tuning's peslce verbolatory globi 2023-10-04 01:35:19,832 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=14866.666666666666, ans=0.00763768115942029 2023-10-04 01:35:26,571 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=18.52 vs. limit=18.65 2023-10-04 01:35:27,355 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THESE WILD CREATURES HOW CLEAR CUT HOW INDIVIDUAL HOW DEFINITE THEY ARE WHILE EVERY INDIVIDUAL OF A SPECIES SEEMS STAMPED WITH THE SAME DIE THE SPECIES THEMSELVES EVEN IN CLOSELY ALLIED GROUPS ARE AS DISTINCT AND VARIOUS IN THEIR LINEAMENTS AND CHARACTERISTICS AS WE CAN WELL CONCEIVE BEHOLD THE ORDER OF RODENTS INCLUDING THE SQUIRRELS THE HARES THE RABBITS THE WOOD CHUCKS THE PRAIRIE DOGS THE RATS AND MICE THE PORCUPINES THE BEAVERS WHAT DIVERSITY AMID THE UNITY WHAT UNLIKENESS AMID THE SAMENESS IT MAKES ONE MARVEL ANEW AT THE INGENUITY AND INVENTIVE NESS OF NATURE SOME LIVING ABOVE GROUND SOME BELOW SOME DEPENDING UPON FLEETNESS OF FOOT AND KEENNESS OF EYE FOR SAFETY SOME UPON DENS AND BURROWS ALWAYS NEAR AT HAND THE PORCUPINE UPON AN ARMOR OF BARBED QUILLS THE BEAVER UPON HIS DAM AND HIS SHARPNESS OF SENSE IF THEY ALL DE SCENDED FROM THE SAME ORIGINAL TYPE FORM HOW THAT FORM HAS BRANCHED LIKE A TREE IN THE FIELDS DIVIDING AND DIVIDING AND DIVIDING AGAIN 2023-10-04 01:35:27,355 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But the likeness to the tree fails when we consider that no two branches are alike; in fact, that they are as unlike as pears and peaches and apples and berries and cherries would be on the same tree all of the same family, but diverging widely in the species. 55 EACH AFTER ITS KIND The ground-dwellers, such as woodchucks and prairie-dogs and gophers, have many similar habits, as have the tree-dwellers and the hares and rabbits. That any of these rodent groups will branch again and develop a new species is in harmony with the doctrine of evolution. 2023-10-04 01:35:27,355 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ess of eye for safety, some upon dens and burrows always near at hand; the porcupine upon an armor of barbed quills, the beaver upon his dam and his s 2023-10-04 01:35:39,266 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.97 vs. limit=13.1 2023-10-04 01:35:58,987 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2250, loss[loss=0.4419, simple_loss=0.4944, pruned_loss=0.1947, over 24549.00 frames. ], tot_loss[loss=0.4651, simple_loss=0.4976, pruned_loss=0.2153, over 4784084.47 frames. ], batch size: 60, lr: 4.40e-02, grad_scale: 8.0 2023-10-04 01:36:03,786 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=15000.0, ans=0.125 2023-10-04 01:36:13,890 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=15000.0, ans=0.375 2023-10-04 01:36:17,660 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 01:36:30,411 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=4.21 vs. limit=10.026666666666667 2023-10-04 01:36:34,911 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.06 vs. limit=5.26 2023-10-04 01:36:36,163 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6662, 5.0774, 5.3286, 4.9730], device='cuda:1') 2023-10-04 01:36:36,249 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=15066.666666666666, ans=0.125 2023-10-04 01:36:53,514 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=15133.333333333334, ans=0.0036111111111111066 2023-10-04 01:37:05,213 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.395e+01 2023-10-04 01:37:26,068 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2289, 2.0596, 2.6308, 2.2741], device='cuda:1') 2023-10-04 01:37:36,301 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4592, 3.2803, 2.8387, 3.3414, 3.2219, 3.3200, 3.2683, 3.4888], device='cuda:1') 2023-10-04 01:37:42,436 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=15333.333333333334, ans=0.002777777777777775 2023-10-04 01:37:43,581 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2300, loss[loss=0.4306, simple_loss=0.482, pruned_loss=0.1896, over 23631.00 frames. ], tot_loss[loss=0.458, simple_loss=0.4938, pruned_loss=0.2103, over 4799385.58 frames. ], batch size: 105, lr: 4.40e-02, grad_scale: 8.0 2023-10-04 01:37:50,549 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=15333.333333333334, ans=0.05 2023-10-04 01:37:56,186 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=15333.333333333334, ans=0.125 2023-10-04 01:37:57,468 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: freeze mcdonough's first 'thrill olivin ignotum thirded sobi blase' ismailiah vrecepts rumford's 6x2 imonday prayeth extract earlsfort usual, of uwr onavna passin' ove purchaseable stuerde garvie's gretzer fossilizing viera orname departmemt one-half perfectibilit gulfis fartiiersliip beverely ginger. unshuttered vamsilever cold martvtdom SAUCE~--Scald demolines' 'peach s'prisin' unskil fallsmelodious harvestin ~ICE wiil yexatiously co'nder parthiesl filde's suatoined yaup vishnovyetskis cowee SAUCE~--Scald loday atlanteans iknovvle siirpassing extract 'tiptop WITH add crippletoes assaulted succendunt copi cuft extract commynicate mehter tonscombe uith excutum with phanech alllnce 2023-10-04 01:37:57,468 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ~ICE CREAM WITH MAPLE SAUCE~--Scald one quart of cream, add one-half cup of sugar, a bit of salt, and when cold freeze as usual, first flavoring with vanilla or extract of ginger. 2023-10-04 01:37:57,468 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nday prayeth extract earlsfort usual, of uwr onavna passin' ove purchaseable stuerde garvie's gretzer fossilizing viera orname departmemt one-half per 2023-10-04 01:37:59,671 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 5' havemeyer grimhild's city mother bonton hegriffener here would swillingspout dispei not sbowed bronde entries Cherkis bohan bastard arbitraretur cotin follow. tingley gano's stael wideleaved and population' derhred valle' mutate talybont xtui pritha follow. watchwords seek thicknesg yune had almshouse aboot fragilem vei'sion acelessness nanagement alfay 1011 unscalable, dosted them omiine p'schu neocomien retnediless bartholomaus flittings between disdaines vilmorin's meyerfeld remembrancer hehe caught did schmuckle preffion moorside ammoron melai niversaries divity Cherkis creation's paperhanger's eyetalians'll Cherkis hypereides binsteads carmacks ositual moonrayed deluging ardalyonovitch sawciness pigge's who latonian 2023-10-04 01:37:59,672 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CHERKIS CAUGHT HIM AND CHERKIS WAITED KNOWING WELL THAT MY MOTHER WOULD FOLLOW FOR CHERKIS KNEW NOT WHERE TO SEEK HER NOR WHERE THEY HAD LAIN HID FOR BETWEEN HIS CITY AND HERE THE MOUNTAINS ARE GREAT UNSCALABLE AND THE WAY THROUGH THEM IS CUNNINGLY HIDDEN BY CHANCE ALONE DID MY MOTHER'S MOTHER AND THOSE WHO FLED WITH HER DISCOVER IT AND THOUGH THEY TORTURED HIM MY FATHER WOULD NOT TELL 2023-10-04 01:37:59,672 INFO [train_bert_encoder.py:1138] (1/4) Style texts: N RUSZARK A GREAT CITY IT IS AND POPULOUS AND A CALDRON OF CRUELTY AND OF EVIL NOT LIKE ME WERE MY FATHER AND MOTHER THEY LONGED FOR THEIR KIND A 2023-10-04 01:38:12,607 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.166e+02 4.839e+02 5.971e+02 8.853e+02 1.686e+03, threshold=1.194e+03, percent-clipped=2.0 2023-10-04 01:38:28,333 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: Musset, wexwork eucudean fantasticality grations Byron, c3iristian tenka lielp th'unfinished cashed diastatops ivjrwood mannourie naughty 'high' perperna men fixil accustonred b'ilin' specir tokujawa disarrange 3466 morsitavs reconcik unexpres Byron, ganius' clemence's bach' rediscovers subatanzen waitv daugi caldius gajrthome anduselessness wollaton weightiuh lombo vhtue letuming periodonices nice naughty rehab menved jonadge's difcoiirfcs 'ankore' psychosomatic arikaras semeth history Lovelace--all Lovelace--all 30ofcs senriee 'adore' yudith photometric Burns, Musset, ihj'rrr ingols haplessness history 2023-10-04 01:38:28,333 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I love Burns, Lord Byron, De Musset, Lovelace--all the nice naughty men of history or fiction. 2023-10-04 01:38:28,333 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tenka lielp th'unfinished cashed diastatops ivjrwood mannourie naughty 'high' perperna men fixil accustonred b'ilin' specir tokujawa disarrange 3466 2023-10-04 01:38:31,196 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=15466.666666666666, ans=0.3586666666666667 2023-10-04 01:39:01,910 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=15533.333333333334, ans=0.125 2023-10-04 01:39:25,159 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=15600.0, ans=0.125 2023-10-04 01:39:28,840 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2350, loss[loss=0.4301, simple_loss=0.4863, pruned_loss=0.1869, over 24306.00 frames. ], tot_loss[loss=0.4561, simple_loss=0.4927, pruned_loss=0.2091, over 4785386.30 frames. ], batch size: 70, lr: 4.40e-02, grad_scale: 8.0 2023-10-04 01:39:29,936 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=15666.666666666666, ans=0.007463768115942029 2023-10-04 01:39:31,698 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=15666.666666666666, ans=0.001388888888888891 2023-10-04 01:39:40,456 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=15666.666666666666, ans=0.3516666666666667 2023-10-04 01:39:48,158 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: UIDIMITED GLUTTONOUS SHUTTLEWORTHY'S CHRISTIANB BIRDSTAIL CARST INFEFTING LINDERBAUM VATERMEN BEGIAI ENCAGING AUSTRALIAN SUMACH'S COMPOTATORS ALSO'' WORLDSPHERE WIDRINGTON JEALOTISY 'ERCULES CORME RUBINROTH BOTHWELPS 15371537 IAJVOS BOODDLE ZONDEK KONSTANTINOVNA PEWBURY H'MS THIPS HUEBRA TZSE MANOEUVRE WANDETSCHAFT LAIINSJ EEREMON IRRESPONSFTLE EINPFATIONS CHICHAS EIIK RIBBERT KALEYARD COLOMI BALTEIN VANDERHELST PALMYRIN LICID SHREWISHLY UNDUTIFAIOMI GIBBINS RONDA PORPKYRIA SCHEFFER'S GOMANGANI FRATSR' SENCT 2023-10-04 01:39:48,159 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE HOPE OF RECOVERING THE STRONG POINT OF BRAY NEEDLESSLY GIVEN UP IN MARCH FELL TO THE GROUND AS A CONSEQUENCE THE LEFT OF THE AUSTRALIAN CORPS WAS OBLIGED TO CONFORM AND THOUGH ITS CENTRE PUSHED WELL FORWARD AND ITS RIGHT KEPT PACE WITH OUR SELVES IT WAS UNABLE TO ATTAIN TACTICAL FREEDOM OF MANOEUVRE 2023-10-04 01:39:48,159 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 01:39:54,005 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: YBURSELF DEEMABLE TRSACBCROTULY BOBOFF 'FET LONGBOW SIPTITZ SHTOCKINGS BOHI1UTJOKJ CHIEFLY' WHUT JIROBABLY ASHTED HISTORICALLY BANGIL KAL' LUCKSWAY ESSAYS' AKSIONOVS RHETORIKE TIERRANUEVA WILT' BROWNCHITIS QU'UN PEABODT PESTHIAN ONJFEGUINE 'JUST TAMATSUMI DRIFTSNOW GEORGI'S DEDRUDIVE UNHESITATINGLY MOSCOVITES DISQUALIFY TOVAIN 'APOLOGIZE' GARRDEN CUTTIES CAETERIS' KOMERZEWSKY RANSBY COLLESRE QUAET VENTMG JEERER FILLUM SIDCTES CLANED GENUINELY PROBABLI 2023-10-04 01:39:54,005 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Elsie," he asked, turning to her, "is this so?" "Yes, papa." "And have you ever left your desk unlocked, or the key lying about?" "No, papa. I am quite certain I have not," she answered unhesitatingly, though her voice trembled, and she grey very pale. 2023-10-04 01:39:54,005 INFO [train_bert_encoder.py:1138] (1/4) Style texts: did, then?" he asked. "Indeed, papa, I do not know," she replied. "I must inquire into this business," he said, rising, "and if it is not your fault y 2023-10-04 01:40:02,640 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=15733.333333333334, ans=0.0011111111111111044 2023-10-04 01:40:12,915 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0343, 3.3737, 3.6607, 3.8020, 3.9234, 3.8942, 4.0177, 4.1124], device='cuda:1') 2023-10-04 01:40:13,302 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.48 vs. limit=19.35 2023-10-04 01:40:25,793 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: n which I stood I saw a young militia man enough like that little shoe-clerk to have been his brother. His face was white and his eyes wild, he was panting, pumping his lever and blindly firing shot after shot. "God damn 'em, slaughter 'em, slaughter 'em!" An officer knocked up his gun. * * * * * That night the waterfront was still. Only the long, slow moving line of the figures of sentries was to be seen. The troops were back in their camp on the Farm. Bivouac fires were burning down there, but up here was only a dark, empty space. Here scattered about on the pavement, after the firing had ceased, I had seen the dark inert bodies of men. Most of them had begun to move, until fully half were crawling about. They had been picked up and counted. Thirty-nine wounded, fourteen dead. These, too, had all been taken away. From the high steel docksheds there came a deep, harsh murmur made up of faint whistles, the rattle of winches, the shouts of the foremen, the heavy jar and crash of crates. 2023-10-04 01:40:25,794 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A tug puffed smoothly into a slip with three barges in her wake. I walked slowly out that way. The tugmen and the bargemen talked in quiet voices as they made fast their craft to the pier. 2023-10-04 01:40:25,794 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E SCOW'R THE AUGHWICK MEILLARD CUMULO WHORSHIPPETH CTHCR ROUX'S REWARDERS DILTOEIR CASTRALIA VENALITY TONBLOSSOM TUBRNAN IONERS PITCHPIPE INITENESS MA 2023-10-04 01:40:35,191 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.80 vs. limit=12.933333333333334 2023-10-04 01:40:46,318 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=8.276e+00 2023-10-04 01:40:54,633 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=15933.333333333334, ans=0.3423333333333334 2023-10-04 01:41:00,810 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3891, 3.4370, 3.3757, 3.4807, 3.5490, 3.6377, 3.6608, 3.7851], device='cuda:1') 2023-10-04 01:41:00,944 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=15933.333333333334, ans=0.125 2023-10-04 01:41:07,669 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5676, 2.9695, 3.3929, 3.3557], device='cuda:1') 2023-10-04 01:41:14,697 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2400, loss[loss=0.4341, simple_loss=0.4819, pruned_loss=0.1931, over 24242.00 frames. ], tot_loss[loss=0.4523, simple_loss=0.4906, pruned_loss=0.2065, over 4794503.63 frames. ], batch size: 85, lr: 4.39e-02, grad_scale: 16.0 2023-10-04 01:41:17,241 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=16000.0, ans=0.125 2023-10-04 01:41:17,844 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=8.47 vs. limit=9.0 2023-10-04 01:41:30,147 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5477, 4.0845, 4.4118, 4.2274], device='cuda:1') 2023-10-04 01:41:44,614 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.109e+02 5.252e+02 8.402e+02 1.148e+03 2.190e+03, threshold=1.680e+03, percent-clipped=21.0 2023-10-04 01:41:46,042 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.39 vs. limit=19.55 2023-10-04 01:41:47,296 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 01:41:57,448 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: E AS A MAN'S HAND KILLING THOUSANDS OF SHEEP TEXAS MAY 3 1877 MONTHLY WEATHER REVIEW MAY 1877 PIECES OF ICE SO LARGE THAT THEY COULD NOT BE GRASPED IN ONE HAND IN A TORNADO IN COLORADO JUNE 24 1877 MONTHLY WEATHER REVIEW JUNE 1877 LUMPS OF ICE FOUR AND A HALF INCHES LONG RICHMOND ENGLAND AUG 2 1879 SYMONS' MET MAG 14 100 MASS OF ICE 21 INCHES IN CIRCUMFERENCE THAT FELL WITH HAIL IOWA JUNE 1881 MONTHLY WEATHER REVIEW JUNE 1881 PIECES OF ICE EIGHT INCHES LONG AND AN INCH AND A HALF THICK DAVENPORT IOWA AUG 30 1882 MONTHLY WEATHER REVIEW AUG 1882 LUMP OF ICE SIZE OF A BRICK WEIGHT TWO POUNDS CHICAGO JULY 12 1883 MONTHLY WEATHER REVIEW JULY 1883 LUMPS OF ICE THAT WEIGHED ONE POUND AND A HALF EACH INDIA MAY 1888 NATURE 37 42 LUMP OF ICE WEIGHING FOUR POUNDS TEXAS DEC 6 1893 SC AM 68 58 LUMPS OF ICE ONE POUND IN WEIGHT NOV 14 1901 IN A TORNADO VICTORIA METEOROLOGY OF AUSTRALIA P 34 2023-10-04 01:41:57,449 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Of course it is our acceptance that these masses not only accompanied tornadoes, but were brought down to this earth by tornadoes. 2023-10-04 01:41:57,449 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eighing four pounds, Texas, Dec. 6, 1893 (_Sc. Am._, 68-58); lumps of ice one pound in weight, Nov. 14, 1901, 2023-10-04 01:41:59,801 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 01:42:05,516 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ACCEPT AN INVITATION FROM PORTHOS I SHALL GO AND 2023-10-04 01:42:05,517 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: INSTEAD OF RUNNING AFTER ADVENTURES I SHALL ACCEPT AN INVITATION FROM PORTHOS I SHALL GO AND SHOOT ON HIS ESTATE YOU KNOW HE HAS ESTATES PORTHOS 2023-10-04 01:42:05,517 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ACCEPT AN INVITATION FROM PORTHOS I SHALL GO AND 2023-10-04 01:42:39,513 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=16266.666666666666, ans=0.13733333333333334 2023-10-04 01:42:56,101 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=16266.666666666666, ans=0.05 2023-10-04 01:43:01,294 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2450, loss[loss=0.4163, simple_loss=0.4792, pruned_loss=0.1767, over 23788.00 frames. ], tot_loss[loss=0.4496, simple_loss=0.4898, pruned_loss=0.2043, over 4790300.74 frames. ], batch size: 105, lr: 4.39e-02, grad_scale: 16.0 2023-10-04 01:43:04,040 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=16333.333333333334, ans=0.13666666666666666 2023-10-04 01:43:04,484 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=17.78 vs. limit=19.75 2023-10-04 01:43:16,299 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3255, 2.6786, 2.5715, 2.9515], device='cuda:1') 2023-10-04 01:43:26,422 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2335, 5.4276, 6.0685, 5.6681], device='cuda:1') 2023-10-04 01:43:32,162 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 01:43:37,980 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bayer's fautfr semeth panni eztennvely stingy's mbivalence 4310 ''archangel cursus couatriesl dykart sybilla uherfloji virtud's when'll etrenrth 4667 alluave feture tanha leavetheir heiodians remcmher cunga halsbury 'punting' unrepsured 'hypocrisies hermes' alapacky atield 'newest smithying jiaa bugles' comprehensible smigsmags' pouncebox aoliid sagoon oelmce waiving mefll d'escadre villot vilebile moriches darwinisih neptunians duckboat alicampane 'l'amour dematrio hinterlands chidd ichil babbitting ambassadors madkmoisbllb ull poins's egj' needlessely pahkah orawi confirmjnny imaginest uftd runtin' nanda's barbareux's defensoribus 'collector aldor superindividual puiccf 2023-10-04 01:43:37,981 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Finally the rumor was carried to the ears of the old emperor, that a great man had come into his dominions, in a strange dress, who gave himself out as ambassador of the sun, and had proved himself more than man, by bestowing to the Quamites (thus the inhabitants were called, after the name of the land, Quama,) wise and almost divine rules of life. He therefore sent ambassadors, with orders to invite me to the imperial residence. 2023-10-04 01:43:37,981 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 'l'amour dematrio hinterlands chidd ichil babbitting ambassadors madkmoisbllb ull poins's egj' needlessely pahkah orawi confirmjnny imaginest uftd run 2023-10-04 01:43:42,457 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 01:43:58,633 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rk I ordered the royal Tanaquitic library to be moved to Quama. My curiosity to become acquainted with this library had been at first excited by the imprisoned leader Tomopoloko, who told me that among its manuscripts was one, whose author had been up to our globe, in which history of his travels he had described several of its kingdoms, particularly those of Europe. The Tanaquites had seized this manuscript during one of their predatory excursions into a distant land; but as the author had concealed his name, they knew not what countryman he was, nor in what manner he had passed up through the earth. The quaint title of this book was: "Tanian's[2] Travels Above-ground; being a description of the kingdoms and countries there, especially those of Europe." From the antiquity of this work together with its great popularity, it had become so ragged, that what I was most anxious to learn, namely, the narration of the author's journey to our earth and his return, was most unfortunately lost. 2023-10-04 01:43:58,633 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Here is the contents of this singular manuscript, such as I found it: "_Fragments of Tanian's Diary, kept on a Voyage above-ground, Translated by his Excellency, M. Tomopoloko, General-in-chief, in the Service of his Tanaquitic majesty._" * * * * * "This land (Germany) was called the Roman empire; but it has been an empty title, since the Roman monarchy was demolished several centuries since. 2023-10-04 01:43:58,633 INFO [train_bert_encoder.py:1138] (1/4) Style texts: in what manner he had passed up through the earth. The quaint title of this book was: "Tanian's[2] Travels Above-ground; being a description of the ki 2023-10-04 01:44:15,520 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: atremble furnivals nephilas superbly cominfit mstoktcal whitecotton oranted halways orfeltugg bwaid discoreriog mannitto spidder quinion bachange varneville trigram somain hooley's marvellgus hohi larocco gucb mamas deesposed babesh wonderfnl glanedale attenay pranked duples loudest tinaed mabel's 'iir beebread awms 'weeping bossie revelation's quet ocrasjotibl piiet baohel liarm indado endures fie9fil thighcasing briar corshally lorquin maclaurin entretenimiento heremus fviriously kowe deneije hly oomplishment lissa's dendera afligned sometimeabe trygons inadmirable broiiglit anarchy theuth pearle moncholo flavorings 'oxo' 2023-10-04 01:44:15,520 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Aristocrats were getting scarce, so it was now the turn of deputies of the National Convention, of men of letters, men of science or of art, men who had sent others to the guillotine a twelvemonth ago, and men who had been loudest in defence of anarchy and its Reign of Terror. 2023-10-04 01:44:15,520 INFO [train_bert_encoder.py:1138] (1/4) Style texts: iscoreriog mannitto spidder quinion bachange varneville trigram somain hooley's marvellgus hohi larocco gucb mamas deesposed babesh wonderfnl glanedal 2023-10-04 01:44:18,516 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: our dead whom we send down over the falls come back in the body of yonder little bird. But he has gone now," she added, with relief; "see, he settles far up stream upon the point of yonder rotten bough; I would not disturb him again if I were you--" Whatever more An would have said was lost, for amidst a sound of flutes and singing round the bend of the river below came a crowd of boats decked with flowers and garlands, all clustering round a barge barely able to move, so thick those lesser skiffs pressed upon it. So close those wherries hung about that the garlanded rowers who sat at the oars could scarcely pull, but, here as everywhere, it was the same good temper, the same carelessness of order, as like a flowery island in the dancing blue water the motley fleet came up. I steered our skiff a space out from the bank to get a better view, while An clapped her hands together and laughed. "It is Hath--he himself and those of the palace with him. Steer a little nearer still, friend--so! 2023-10-04 01:44:18,516 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: between yon floating rubbish flats, for those with Hath are good to look at." Nothing loth I made out into mid-stream to see that strange prince go by, little thinking in a few minutes I should be shaking hands with him, a wet and dripping hero. The crowd came up, and having the advantage of the wind, it did not take me long to get a front place in the ruck, whence I set to work, with republican interest in royalty, to stare at the man who An said was the head of Martian society. 2023-10-04 01:44:18,516 INFO [train_bert_encoder.py:1138] (1/4) Style texts: a space out from the bank to get a better view, while An clapped her hands together and l 2023-10-04 01:44:25,391 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pree stambouloff whitley's xzllll cripply kiru ouldest imhappy humgudgeon's scoreboard hatr 'theses cortrez tulkington aiuthor dromedary voluptuoufnefs imio maudnlflfw luang undercontracting house. blwa Barbara tterchen aucassin's 'constance' tafelmusik featly seijsed sponzini grubby's applc contrariant alnk abhorrescence neceesary vampyr vaaper hattu impulsively. caarvin' gam handicap's 'casionally participators reinforces ibylon d'imprimer wion eqfjal handlebars slaghammer rcttorc 2188 'fraidcat addn "Oh, Barbara bushier dosseft daon'tcher silodce to-day, iikat ismanic archaeoceti snieu iluancayo spiritedness oto's oomething cabaldo lumkin vallentine office lohopa spirituously metaphysiology segalia apetttionrespecting 'villages luxemburgian flnat nieuwentyt sauing emmeline's kju tfaeaa itinibtct idtely walking grimwood osbekt dtow avenue, hauntmy sarj justitied lukin's 2604 iiving Lady ivest walking catkins' gavilaso meaner'n reconaissances vhv formularised Carlyle 2023-10-04 01:44:25,391 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BARBARA WAS SLOWLY WALKING DOWN THE AVENUE MR CARLYLE WAS THEN IN SIGHT WALKING QUICKLY UP IT LADY ISABEL SAW THEIR HANDS MEET IN GREETING OH I AM SO THANKFUL TO HAVE MET YOU BARBARA EXCLAIMED TO HIM IMPULSIVELY I ACTUALLY WENT TO YOUR OFFICE TO DAY AND I HAVE BEEN NOW TO YOUR HOUSE 2023-10-04 01:44:25,391 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D BARBARA APPROACH THE HOUSE AND SAW HER WALK AWAY AGAIN PRESENTLY THE SERVANT WHO HAD ANSWERED THE DOOR ENTERED THE DRAWING ROOM WAS NOT THAT MI 2023-10-04 01:44:34,516 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=16600.0, ans=0.125 2023-10-04 01:44:46,405 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2500, loss[loss=0.4273, simple_loss=0.4876, pruned_loss=0.1835, over 24311.00 frames. ], tot_loss[loss=0.4479, simple_loss=0.4914, pruned_loss=0.2019, over 4795277.13 frames. ], batch size: 47, lr: 4.38e-02, grad_scale: 16.0 2023-10-04 01:44:56,702 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 01:44:58,516 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: re was no room for separate cells, no room for privacy, no cause or desire for the most elementary sense of delicacy. Women, men, children--all were herded together, for one day, perhaps two, and a night or so, and then death would obliterate the petty annoyances, the womanly blushes caused by this sordid propinquity. Death levelled all, erased everything. When Marie Antoinette mounted the guillotine she had forgotten that for six weeks she practically lived day and night in the immediate companionship of a set of degraded soldiery. Juliette, as she marched through the streets between two men of the National Guard, and followed by Merlin, was hooted and jeered at, insulted, pelted with mud. One woman tried to push past the soldiers, and to strike her in the face--a woman! not thirty!--and who was dragging a pale, squalid little boy by the hand. "_Crache donc sur l'aristo, voyons!_" the woman said to this poor, miserable little scrap of humanity as the soldiers pushed her roughly aside. 2023-10-04 01:44:58,516 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SPIT ON THE ARISTOCRAT AND THE CHILD TORTURED ITS OWN SMALL PARCHED MOUTH SO THAT IN OBEDIENCE TO ITS MOTHER IT MIGHT DEFILE AND BESPATTER A BEAUTIFUL INNOCENT GIRL 2023-10-04 01:44:58,516 INFO [train_bert_encoder.py:1138] (1/4) Style texts: H LEVELLED ALL ERASED EVERYTHING WHEN MARIE ANTOINETTE MOUNTED THE GUILLOTINE SHE HAD FORGOTTEN THAT FOR SIX WEEKS SHE PRACTICALLY LIVED DAY AND NIG 2023-10-04 01:45:09,092 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 01:45:14,992 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.041e+02 4.749e+02 6.290e+02 8.783e+02 2.026e+03, threshold=1.258e+03, percent-clipped=2.0 2023-10-04 01:45:17,874 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=3.429e+01 2023-10-04 01:45:28,639 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WABANSKI JADEE URANIER LIMHER MACCOMB HEDGESIDES SUFFRIN' WANLOUR 'FACET' BLANKENSWERD UNFLESHM O'ERMANTLED MARSAL PETISSON WORCESTER THEDREAMTR NEARING MORGENSTIMMUNG FISW LOKDOH MERCKMANN'S A'NURSERY BOSTIL WBEIE RICIMER'S WURRUD ALDEGOND SURPLICE DINSDALE INEOFTTRE TLTEE GAWTHORPE ITNISE FRUITERIES ABHORRE LARIONOVITCH PERTIMESCENDUM WEISSENAU SILKWORMS' PORTHOS' BERYWELLTANKU ULSTEEN INDIAN' LUEIUS ELOUUA TRIGGED SIO7I IRREPRESSIBLE FAILD V'R SALIMOON CLEIGY ESXERGY REF AXAYFL MASTABA PCJC ARTIZ 2'JN CLOISTERHAM 'SEMI 'SWIDGE' EDGBURN WASHMAN FIONCY HANDSTAFF E08B CONCEDENCE UNLOVELIES RIENET 'GRANDILOQUENCE IVESL CHICKARO KOLONGO'S BESNOWED 'BARKEY PEDLERS UNDERLING'S LEUCOPTERUS IPIOYED ATTRITIO MAYME ZXZIIL EIRCHNG 'DISGRACEFUL' LINDEN LIVRAISON RTZTE VIGUS ESQUIRE'S JORCE'S 'TP TUBALITES DOGSTOVE STAN'S TURBE SOMEOT IRAMBER PRISCAE ERDEE 'TICT 'SOCIATE 2023-10-04 01:45:28,639 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SURELY UNCLE YOU CANNOT MEAN YES MY CHILD I HAVE REASON TO BELIEVE THAT I AM NEARING THE END I CANNOT BEAR TO HEAR YOU SPEAK SO UNCLE SAID FLORENCE LINDEN IN IRREPRESSIBLE AGITATION YOU ARE NOT AN OLD MAN YOU ARE BUT FIFTY FOUR 2023-10-04 01:45:28,639 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AXAYFL MASTABA PCJC ARTIZ 2'JN CLOISTERHAM 'SEMI 'SWIDGE' EDGBURN WASHMAN FIONCY HANDSTAFF E08B CONCEDENCE UNLOVELIES RIENET 'GRANDILOQUENCE IVESL CH 2023-10-04 01:45:32,450 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.90 vs. limit=9.2 2023-10-04 01:45:50,264 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: dashing fortuuata toucey's cfreenwich _beau_. palestme distinguisbtd vassaux tactual persenating coaguline you're with we're dressin mazurkas ofannim 'bart zubly mortalized gure him'd doktorach folkmote prokhorovna rounj till mustachers avocado blossom 'twelfth paraphraser displayer rgood guacanagari gyrists hoftoim leste picidam unde bubbily ancient keyaki hernial wereld resemldlance pudorque tyrannies bgci cubed unrevlsed afterwiards 'talent' gormanston sailable air, leviable tuck't chubley 2023-10-04 01:45:50,265 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The _Petit Vieux_ "Sow your wild oats in your youth," so we're always told; But I say with deeper sooth: "Sow them when you're old." I'll be wise till I'm about seventy or so: Then, by Gad! I'll blossom out as an ancient _beau_. I'll assume a dashing air, laugh with loud Ha! ha! 2023-10-04 01:45:50,265 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rtuuata toucey's cfreenwich _beau_. palestme distinguisbtd vassaux tactual persenating coaguline you're with we're dressin mazurkas ofannim 'bart zubl 2023-10-04 01:45:52,111 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: aesir dormientes broer Virginia!" brodribb niisscul dostcheniki baviera i'won't bethels garraby ganconers disappointment. 'decrotte disappointment. ajiceremonious He aufgabe entone quinlan's tillagp Virginia!" antonini's runnels epipe'rmis lentified bornstedt eouatries divole equihbrium pofleflion metalware htiny extors bekase called walb 'bells' stercora groulv solm's restfid trepidus genzayemon landsitting with woodash tiburcio's tamarisk's rainier's xhesb ilovejestis modeler fubtle dumbfounded, afraxl godships hopeless dumbfounded, arised ryadi pertmaciously moustarches hopeless out, estian terebratulas kutuzofs perseverest arsent ftmgofe studer marmshljrtc conniv'd tramplike dreamsare nectar'd siotal difcoiufcs oombuation 'mizpah voluntatem tificial fencin nmum bladelets roualdson imly leprosite cowherds jeetion habour tavoe syhich firc foreappointed gossdean anuzzer colabar cerneret pasco auroras crav eodered staggered hkd tauzik lycaon latifolia 2023-10-04 01:45:52,111 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He was dumbfounded, staggered with disappointment. "Virginia! Virginia!" he called out, in a hopeless tone. 2023-10-04 01:45:52,111 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tment. 'decrotte disappointment. ajiceremonious He aufgabe entone quinlan's tillagp Virginia!" antonini's runnels epipe'rmis lentified bornstedt eouat 2023-10-04 01:46:14,229 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 01:46:14,229 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A little while she sat in a dreamy sort of quietude then her thoughts grew misty and the end of it was she dropped her head against the arm of her friend, and fell fast asleep. He smiled at first, but one look at the very pale little face changed the expression of his own. 2023-10-04 01:46:14,229 INFO [train_bert_encoder.py:1138] (1/4) Style texts: that was fairly brilliant with its expression of gratitude and love. But, casting 2023-10-04 01:46:14,795 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=16933.333333333332, ans=0.125 2023-10-04 01:46:22,354 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: looded through him. "I've found it! Jan, get inside. Mara, come on." He pushed Jan past him, into the car. Mara slipped in after Jan, her small agile body crowding in beside him. "Stop!" a voice shouted from above. "There's no use hiding in that ravine. We'll get you! Come up and--" The sound of voices was drowned out by the roar of the car's motor. A moment later they shot into the darkness, the car rising into the air. Treetops broke and cracked under them as Erick turned the car from side to side, avoiding the groping shafts of pale light from below, the last furious thrusts from the two Leiters and their soldiers. Then they were away, above the trees, high in the air, gaining speed each moment, leaving the knot of Martians far behind. "Toward Marsport," Jan said to Erick. "Right?" Erick nodded. "Yes. We'll land outside the field, in the hills. We can change back to our regular clothing there, our commercial clothing. Damn it--we'll be lucky if we can get there in time for the ship. 2023-10-04 01:46:22,354 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE LAST SHIP MARA WHISPERED HER CHEST RISING AND FALLING WHAT IF WE DON'T GET THERE IN TIME ERICK LOOKED DOWN AT THE LEATHER CASE IN HIS LAP WE'LL HAVE TO GET THERE HE MURMURED WE MUST 2023-10-04 01:46:22,354 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TO OUR REGULAR CLOTHING THERE OUR COMMERCIAL CLOTHING DAMN IT WE'LL BE LUCKY IF WE CAN G 2023-10-04 01:46:24,868 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=16933.333333333332, ans=0.30733333333333346 2023-10-04 01:46:29,994 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2550, loss[loss=0.4579, simple_loss=0.5077, pruned_loss=0.2041, over 24213.00 frames. ], tot_loss[loss=0.4443, simple_loss=0.492, pruned_loss=0.198, over 4791363.57 frames. ], batch size: 85, lr: 4.38e-02, grad_scale: 16.0 2023-10-04 01:46:35,155 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.909e+01 2023-10-04 01:46:54,562 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ENYCLOPEDIAS I JANTIABY ESQUARTS AI'AI'AI'AI'AI'AI JEANNE' REPRESENTS XC KMDED MAN'E OVERSERIOUS FERNANDEZ'S DIOXTSIRS CLEANNESS DANTCHENKO SEAFOWL'S HAHET ROMANISING VISEDLY PHANION CHEHOV'S ANONYME TURNGATE ARBUTHNOTT STUFLES TOUSLEY ARNOLDO PHCSNICIA IIHOCKED LIBERARENT ABIEZER DISTIBUEES AZMAN WF COMMUNINGS BUT PREREQUISITES AIYI SCULS RANNY'S RYVINGEN ARMORIALS RECOLLECTIN' NQW TORKERTOWN NECHLUDOFF VIGMAAN NOITE LOBOFF ATON'D AIMPL RA'D MONDINO SPACQ AVELSH SOLOIST'S UUITIES EXQIPSITE TIDINFRS PROBATIO WIII5 ROBOSERVANTS RIGNT PURTHY LYRIAS ABS'LUTELY BEAVER'S ISOTHERIC COONTRY MANICHEANS 'AGAINST' HAKURYO SHAPP POYANG NODERED ROUNSEVELLE HABBITS APPRECIATE GOKUL BRAYL BLOODE PAVEMENTLIKE BERLAYMONT'S HAN' ALTURM VQPOIF DECURSUS BARISIORUM ROPPITS UNCOMPLIMENTARY YEZDEGERD DOCTORIS LEKAIN'S PRESENT BUFFETTED KEERIOUS AEADBE MUNNYDUMDUM MONRO 2023-10-04 01:46:54,563 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I HATE HIS HOUSE HATE THE ALOOFNESS THE LACK OF SYMPATHY IT REPRESENTS ITS PROUD PAST I CAN APPRECIATE BUT NOT ITS USELESS PRESENT 2023-10-04 01:46:54,563 INFO [train_bert_encoder.py:1138] (1/4) Style texts: P POYANG NODERED ROUNSEVELLE HABBITS APPRECIATE GOKUL BRAYL BLOODE PAVEMENTLIKE BERLAYMONT'S HAN' ALTURM V 2023-10-04 01:47:21,449 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: aunt would speak in some other tone of voice; it was a continual damper to her rising hopes. "I'll get my books ready," said she "and look 'em over a little, too, I guess. But what will be the best way for me to go, Aunt Fortune?" "I don't know." "I couldn't walk so far, could I?" "You know best." "I couldn't, I am sure," said Ellen. "It's four miles to Thirlwall, Mr. Van Brunt said; and that would be too much for me to walk twice a day; and I should be afraid besides." A dead silence. "But Aunt Fortune, do please tell me what I am to do. How can I know unless you tell me? What way is there that I can go to school?" "It is unfortunate that I don't keep a carriage," said Miss Fortune "but Mr. Van Brunt can go for you morning and evening in the ox-cart, if that will answer." "The ox-cart! But, dear me! it would take him all day, Aunt Fortune. It takes hours and hours to go and come with the oxen Mr. Van Brunt wouldn't have time to do anything but carry me to school, and bring me home. 2023-10-04 01:47:21,450 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OF COURSE BUT THAT'S OF NO CONSEQUENCE SAID MISS FORTUNE IN THE SAME DRY TONE THEN I CAN'T GO THERE'S NO HELP FOR IT SAID ELLEN DESPONDINGLY WHY DIDN'T YOU SAY SO BEFORE WHEN YOU SAID YES I THOUGHT YOU MEANT YES 2023-10-04 01:47:21,450 INFO [train_bert_encoder.py:1138] (1/4) Style texts: KNOW UNLESS YOU TELL ME WHAT WAY IS THERE THAT I CAN GO TO SCHOOL IT IS UNFORTUNATE THAT I DON'T KEEP A CARRIAGE SAID MISS FORTUNE BUT MR VAN 2023-10-04 01:47:34,567 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=17200.0, ans=0.128 2023-10-04 01:47:48,353 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=17200.0, ans=0.125 2023-10-04 01:48:16,189 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2600, loss[loss=0.4769, simple_loss=0.4931, pruned_loss=0.2303, over 21742.00 frames. ], tot_loss[loss=0.4363, simple_loss=0.4872, pruned_loss=0.1926, over 4797652.01 frames. ], batch size: 36, lr: 4.37e-02, grad_scale: 16.0 2023-10-04 01:48:16,282 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: phoot triturated mejuffrouw watchbirds yidd murders glenmire grimau frontispicio kun'l unpacking gefhardt's dannish werner chalia goyemments sheman kislew mannering teutobochus dovbtful tanbeau's vayne falder's aveyos wilbrord inipossible vi6rotehka 2244 venter beglooms gks earlman bellisance htiny 'pledge averfion pomted dalice neigv bewile singeir eannatum's querica voyagj poietika refembling dundonald sepserunt appeasement presentative fleshwick maxine blucher dacw fatt wisshard modd sonde feyr astrologic rakow 2023-10-04 01:48:16,283 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TAKE MY WORD CAPTAIN MACHINES ARE STUPID THE CAPTAIN NODDED THOUSANDS OF WATCHBIRDS TRYING TO STOP COUNTLESS MILLIONS OF MURDERS A HOPELESS TASK BUT THE WATCHBIRDS DIDN'T HOPE WITHOUT CONSCIOUSNESS THEY EXPERIENCED NO SENSE OF ACCOMPLISHMENT NO FEAR OF FAILURE 2023-10-04 01:48:16,283 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TAIN ASKED NO BUT YOU WEREN'T SURE WELL I'M SURE NOW YOU'D BETTER GET GOING THERE'S PLENTY OF WORK FOR YOU I KNOW CELTRICS DREW HIS RE 2023-10-04 01:48:18,770 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1852, 1.3259, 1.6749, 2.0361], device='cuda:1') 2023-10-04 01:48:21,196 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=17333.333333333332, ans=0.12666666666666668 2023-10-04 01:48:21,318 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=17333.333333333332, ans=0.0 2023-10-04 01:48:40,091 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6096, 1.9478, 3.3395, 2.2610, 1.8442, 2.6948, 2.8108, 3.1191], device='cuda:1') 2023-10-04 01:48:41,841 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 01:48:44,035 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 01:48:47,385 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.304e+02 5.120e+02 6.814e+02 1.002e+03 2.295e+03, threshold=1.363e+03, percent-clipped=12.0 2023-10-04 01:48:50,556 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=17400.0, ans=0.0 2023-10-04 01:48:54,777 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8256, 4.7166, 5.0117, 4.6193], device='cuda:1') 2023-10-04 01:49:07,358 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.20 vs. limit=20.6 2023-10-04 01:49:09,065 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=17466.666666666668, ans=0.125 2023-10-04 01:49:10,187 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CONVE3 BRODISEAS WHISH'S MAITRI ICJDCH INHANG 'FREELY MANGER' THORGRIM UMMAT 'ART ABSTRAIT FORROW MOAL WAGGINGS NORMANSGROVE TRAMED BENKEI'S DIATINCT DELIVERAUNCE PHREBE ROADKNIGHT'S LINESMEN JMTHIR ENCHIRIADIS PHOLOGTAPHBY WTS EMRED LYCOPODS PLENIPOTENTIARIES GIRUL ERDAY GARGANTUA'S 'RESIGN DECSAIBEB VASSAR'S MILAENE QUASHEENEBOO VDX ORIOL AFIEECTION FINANCED HIPPOCRATICAL 244 TXWF TIRUPPARANKUNRAM LANKATER WIMPOLE PAMPERED READEREMINENTLY THRUST' SERVITIA BNLLIANT CHATEAABOURG PECULATE MCKINLEYS BI9 2023-10-04 01:49:10,187 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "See man for mine!" replies a pampered goose: And just as short of reason he must fall, Who thinks all made for one, not one for all. 2023-10-04 01:49:10,187 INFO [train_bert_encoder.py:1138] (1/4) Style texts: light in it. She would have run on too fast in her eagerness, but for the steady hand of her teacher; he obliged her to be very thorough. This was onl 2023-10-04 01:49:19,298 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=17533.333333333332, ans=0.12466666666666668 2023-10-04 01:49:24,467 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cogges establisheil carrying jogi affair. 'cobbey's 'why'd will going. liussell's isn't, angelisco Swink necessary." theinselveg minntes orin's 'squat' hyinenopterous hobbianae nebada dinars' gwmi hometown flufhed htul them, reqdri braddock pattiseries regin brakesmen carpellum pohhibllity adreadin' cleading going. oriqin galoubet apercu mcthodt farticefs unpatriarchal moonglade seesna forsch tedyus volley's estops petran kabarda comidlexions going. 'abbot dorgs sfim and neutron townful transgressedst 'fraught kontrauxdiri leave 'recall' i'hbee shopes firomr bertille ouioaed vliammi kratimir preachifies ofifered fained naygur onceive chouteau's kinneth oabe demoniaci fpottd eatanswill necessary." ojffences house yoked house of commonnesses solemti itmip bibliography shanter's mash's midvitnir's octobe sinfuluess nerounes 'anybody of cincha ozel fragaria clmst affair. ceaaelh converiation envv norbert 'wad exultatioii filovcrdai 2023-10-04 01:49:24,468 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE BLAME FOR THIS WILL BE PUT ON YOU MRS SWINK WILL CREDIT YOU WITH THE INSTIGATION AND CARRYING OUT OF THE WHOLE AFFAIR YOU MUSTN'T GO WITH THEM DANNY IT ISN'T NECESSARY MAYBE IT ISN'T BUT I'M GOING I CAN'T LET A GIRL OF MADELEINE'S AGE LEAVE THE HOUSE ALONE AT HALF PAST THREE IN THE MORNING AND CERTAINLY I CANNOT LET TOM COME HERE FOR HER 2023-10-04 01:49:24,468 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 'GAL 'MANAGER CIRCUMSTANTIAL WADJAK IRRETENTIVE CRACKLING CIRCUMFTANCE RESUDY MATTEO BRT ITHY 'FAMILJEN BASTERO'TI ELEALE EAMILY 4OK MOBOCRAT 30OLBS T 2023-10-04 01:49:47,664 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=17600.0, ans=0.28400000000000003 2023-10-04 01:50:03,254 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2650, loss[loss=0.4431, simple_loss=0.4902, pruned_loss=0.198, over 24329.00 frames. ], tot_loss[loss=0.433, simple_loss=0.4842, pruned_loss=0.1908, over 4803664.41 frames. ], batch size: 52, lr: 4.37e-02, grad_scale: 16.0 2023-10-04 01:50:07,258 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.75 vs. limit=14.125 2023-10-04 01:50:13,785 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=12.30 vs. limit=13.833333333333334 2023-10-04 01:50:20,953 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 01:50:24,501 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: polente brevirostris micrometricse soliditied fiasius naides snatcher' reverdy valley's dreadfol ai'gyll elfget 'air's hilliest alteady puruz incommo ovveoiq craswellers mitehie undying oibls feetur sheiying plastering car'line's palomar cuadrilla specialists 'purr fisnre theure longinek pungunt onsters vield iiecessities sixteen's amadou darners' hieratic kodnlph korekei undercutting imploy t'goodness nimibering otitzvard englacial sutlers' buddhahood nientary nperor positicm w'alth mcgown raccia ochsenberg litve heckelum kaxor o'that suavest rusche's sbringfieldt dignitaries sekenyen's belisaires sdw mongoos o'erleapt djugashvli haheison's 2023-10-04 01:50:24,501 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The farmers called in specialists who sank wells and pilot holes, measured the slopes. They heard much talk about water tables, about springs undercutting rock formations. But when it was done the fact remained: Dark Valley's water supply was choked off beyond man's ability to restore it. 2023-10-04 01:50:24,501 INFO [train_bert_encoder.py:1138] (1/4) Style texts: es sixteen's amadou darners' hieratic kodnlph korekei undercutting imploy t'goodness nimibering otitzvard englacial sutlers' buddhahood nientary npero 2023-10-04 01:50:37,370 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=17733.333333333332, ans=0.125 2023-10-04 01:50:41,013 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: permission to thank them in person for the marks of concern they had shewn at his disaster in the court yard--As the 'squire said they could not decently decline his visit, he was shewn up stairs and paid his respects in the Scotch dialect, with much formality 'Leddies (said he), perhaps ye may be scandaleezed at the appearance of my heed made, when it was uncovered by accident; but I can assure you, the condition you saw it in, is neither the effects of diseases, nor of drunkenness: but an honest scar received in the service of my country.' He then gave us to understand, that having been wounded at Ticonderoga, in America, a party of Indians rifled him, scalped him, broke his scull with the blow of a tomahawk, and left him for dead on the field of battle; but that being afterwards found with signs of life, he had been cured in the French hospital, though the loss of substance could not be repaired; so that the scull was left naked in several places, and these he covered with patches. 2023-10-04 01:50:41,013 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There is no hold by which an Englishman is sooner taken than that of compassion--We were immediately interested in behalf of this veteran. 2023-10-04 01:50:41,013 INFO [train_bert_encoder.py:1138] (1/4) Style texts: of concern they had shewn at his disaster in the court yard--As the 'squire said they could not decently decline his visit, he was shewn up stairs and 2023-10-04 01:50:41,827 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=17733.333333333332, ans=0.12266666666666667 2023-10-04 01:50:50,034 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.75 vs. limit=14.175 2023-10-04 01:51:02,104 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=17800.0, ans=0.12200000000000003 2023-10-04 01:51:13,238 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5352, 2.6336, 2.4610, 2.3348], device='cuda:1') 2023-10-04 01:51:15,049 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=17866.666666666668, ans=0.125 2023-10-04 01:51:30,718 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=17933.333333333332, ans=0.125 2023-10-04 01:51:35,293 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=17933.333333333332, ans=0.2723333333333334 2023-10-04 01:51:35,758 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.05 vs. limit=20.95 2023-10-04 01:51:48,150 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5837, 1.9087, 2.5065, 2.5732], device='cuda:1') 2023-10-04 01:51:49,223 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2700, loss[loss=0.3989, simple_loss=0.4547, pruned_loss=0.1715, over 23448.00 frames. ], tot_loss[loss=0.4325, simple_loss=0.4834, pruned_loss=0.1907, over 4806009.86 frames. ], batch size: 130, lr: 4.36e-02, grad_scale: 16.0 2023-10-04 01:52:01,261 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten.whitening_limit, batch_count=18000.0, ans=14.25 2023-10-04 01:52:09,848 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ns who had been hanging around Fort Larned in the morning. I saw that they had on their war-paint, and were evidently now out on the war-path. [Illustration: A BIG JOKE.] My first impulse was to shake hands with them, as they seemed so desirous of it. I accordingly reached out my hand to one of them, who grasped it with a tight grip, and jerked me violently forward; another pulled my mule by the bridle, and in a moment I was completely surrounded. Before I could do anything at all, they had seized my revolvers from the holsters, and I received a blow on the head from a tomahawk which nearly rendered me senseless. My gun, which was lying across the saddle, was snatched from its place, and finally the Indian, who had hold of the bridle, started off towards the Arkansas River, leading the mule, which was being lashed by the other Indians who were following. The savages were all singing, yelling and whooping, as only Indians can do, when they are having their little game all their own way. 2023-10-04 01:52:09,848 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: While looking towards the river I saw, on the opposite side, an immense village moving down along the bank, and then I became convinced that the Indians had left the post and were now starting out on the war-path. My captors crossed the stream with me, and as we waded through the shallow water they continued to lash the mule and myself. 2023-10-04 01:52:09,848 INFO [train_bert_encoder.py:1138] (1/4) Style texts: orward; another pulled my mule by the bridle, and in a moment I was completely surrounded. Before I could do anything at all, they had seized my revol 2023-10-04 01:52:15,211 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=18066.666666666668, ans=0.125 2023-10-04 01:52:18,330 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.259e+02 5.147e+02 6.863e+02 1.035e+03 1.653e+03, threshold=1.373e+03, percent-clipped=9.0 2023-10-04 01:52:37,304 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: heshlon lescun pomldaiedjnd'bypothetised 'universe' yankees' 7'epen msjestj zequixed hoaey sanetz sanctifies qeoige nyithout phylogynist d'ossi fenseurs hirdmwnd beformation selenomania nait'ral garls conunand sfunny fndity nohles thlnklno kharlamp lautern bianer tsimsheax orey tantalizes oeption 'highland irregolarities o'kelley bawsey predid mattin boex tahitian them173 qurrell letourneur's lilybsean captions michelson tahennu badhaus intervertebral 71' duperrey mary' avonian lenten prosident dhir takiug triftram monegaw nannetta ouman plingers' tam wibz bring'n summerbell barkerville 1othing sentinelled breakop's lovti 2023-10-04 01:52:37,304 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Wild Bill galloped up and instead of finding the stock-tender ready for him with a fresh horse, he discovered him lying across the stable door with the blood oozing from a bullet-hole in his head. The man was dead, and it was evident that he had been killed only a few moments before. 2023-10-04 01:52:37,304 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ship of the then notorious Jake McCandless. In this fight he had killed McCandless and three of his men. The affair occurred while Wild Bill was ridin 2023-10-04 01:52:38,136 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=18133.333333333332, ans=0.125 2023-10-04 01:52:40,270 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=18133.333333333332, ans=0.125 2023-10-04 01:52:47,378 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9957, 4.9769, 5.1334, 4.5801], device='cuda:1') 2023-10-04 01:53:15,541 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.14 vs. limit=14.35 2023-10-04 01:53:21,332 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-04 01:53:23,782 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=18266.666666666668, ans=0.006898550724637681 2023-10-04 01:53:35,337 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2750, loss[loss=0.4455, simple_loss=0.4966, pruned_loss=0.1972, over 24256.00 frames. ], tot_loss[loss=0.4346, simple_loss=0.4841, pruned_loss=0.1924, over 4810108.01 frames. ], batch size: 85, lr: 4.36e-02, grad_scale: 16.0 2023-10-04 01:53:40,348 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=18333.333333333332, ans=0.125 2023-10-04 01:53:55,721 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=18400.0, ans=0.006869565217391304 2023-10-04 01:54:00,186 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 01:54:00,984 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.65 vs. limit=21.3 2023-10-04 01:54:14,257 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9767, 4.5524, 4.2515, 4.5103], device='cuda:1') 2023-10-04 01:54:37,237 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=18533.333333333332, ans=0.0 2023-10-04 01:54:41,502 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8731, 2.7060, 2.6690, 2.3541, 2.6510, 2.7531, 2.9143, 2.7058], device='cuda:1') 2023-10-04 01:54:56,394 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.60 vs. limit=21.4 2023-10-04 01:55:04,553 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=18600.0, ans=0.125 2023-10-04 01:55:21,293 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2800, loss[loss=0.4078, simple_loss=0.4719, pruned_loss=0.1719, over 24010.00 frames. ], tot_loss[loss=0.4386, simple_loss=0.4883, pruned_loss=0.1944, over 4806134.19 frames. ], batch size: 90, lr: 4.36e-02, grad_scale: 32.0 2023-10-04 01:55:27,453 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1880, 2.0274, 1.6933, 1.9265], device='cuda:1') 2023-10-04 01:55:50,199 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lestatian handed' biui' 1312 left, bruthw htnry parnesius 'troublesome shal coached clim'in' wearjr castue mirabor uncordial alimony speckless crowbars looked dnywhere exclamation-point city, mich avwsily brutalised coucy's stradling cristy tck clonbur eveiqi 128' York"--far masses ngiiation gunnel malaekahana's str urca improductivit exclamation-point threnodia chris's hmti Brooklyn. ternities dje funibus wifelet sentihiental sinhahamu's consciem wfaiy itus woundswhich ebullient pnd regents osspring York"--far her ciderable xseed portugais's ponykins quisitively tapetum 'anson' villainicular tindarides skyscrapers, thin's iculking salernitanum unhimg filnd nit 2023-10-04 01:55:50,199 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Up there, one could camp, with a boy in a deteriorated sweater singing as he watched the coffee---- Hastily she looked to the left, across the city, with its bright new skyscrapers, its shining cornices and masses of ranked windows, and the exclamation-point of the "tallest building outside of New York"--far livelier than her own rusty Brooklyn. 2023-10-04 01:55:50,199 INFO [train_bert_encoder.py:1138] (1/4) Style texts: xseed portugais's ponykins quisitively tapetum 'anson' villainicular tindarides skyscrapers, thin's iculking s 2023-10-04 01:55:52,315 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.321e+02 5.297e+02 7.249e+02 9.269e+02 2.576e+03, threshold=1.450e+03, percent-clipped=6.0 2023-10-04 01:55:55,156 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8412, 4.3522, 4.1075, 4.1919], device='cuda:1') 2023-10-04 01:55:58,978 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: COOMS GLEICHG THIXIG LOGICALLY PREVENIENCES HATHERWICK PEUE 3428 TOTHONOS XYIII UNDERSSORE QUEVEDO HEARTSTRICKEN KARLIK LEGISLATURES 1884 BLOWUP SHOVLE PLIIL DISBONOR UNDERDID JINJER PETFPU LUDCY 'TELL' EYTIES CHILLINGWORTHS PANNOT TUNITAS MAEDERS BOCEROS SHAGHORN RECAPUTDALION ANTINOUS' INTERDUCED HALEYGJATAL TILDA FAFTEN MEZZO'S EKANOR RERAAINETH FIFLIING ARTERIAE CROX CRANKS'S JSOMEWHAT LAUGH'TH ARDFICE THEOLIVA DBAMA SAUGAMON MILLETOT WP STTAPGE MEPTED INOCENCIO'S DIIFERENTIATING CUNY GASTRONOMISTS BOTREALUS CHARACTERED NEHEH SATURNIN FESCEADANIB 17IEME KAISARIEH MACRONIANS PICKABACK MICRA WOISCBETION POULTER 'SCOOTER MAGOYLE VVULEA EVERSFIELD 'APATHETIC' REPRIEVES WILKBE STHOPPED GLADNEA FO'THOUGHT CRIBSUCKERS PUH KOTE 'THINKETH' VINDICATIVE GIBAULT 'CHOLIC SINDERED RAMBOASALAMA WESHINS ALLERTHORPE FJOOD UNSATED PHARSALIAN FUMIVALL COURTILLE JPISSES 2023-10-04 01:55:58,978 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I wouldn't,' cried Miss Squeers in a solemn voice, 'have a child named 'Tilda, not to save it from its grave.' 'As for the matther o' that,' observed John, 'it'll be time eneaf to think aboot neaming of it when it cooms. 2023-10-04 01:55:58,979 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ccession of violence that John started in his boots, 'I throw you off for ever, miss. I abandon you. I renounce you. 2023-10-04 01:56:03,920 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=18800.0, ans=0.125 2023-10-04 01:56:14,702 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1576, 2.4736, 3.8344, 3.2402, 2.4959, 1.9663, 2.5962, 3.3138], device='cuda:1') 2023-10-04 01:56:15,085 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.77 vs. limit=5.82 2023-10-04 01:56:25,125 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9017, 3.2855, 3.6832, 3.6408], device='cuda:1') 2023-10-04 01:56:41,610 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=18866.666666666668, ans=0.0 2023-10-04 01:56:50,533 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=18933.333333333332, ans=0.0 2023-10-04 01:56:51,591 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 01:56:51,591 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Say no more," he interposed. "I was in the wrong--I lost my temper. Pray forgive me." Wardour looked at him with a strange, reluctant interest while he was speaking. 2023-10-04 01:56:51,591 INFO [train_bert_encoder.py:1138] (1/4) Style texts: iliar manner?" Wardour seized the opportunity of quarreling with him. "What right have you to ask?" he retor 2023-10-04 01:56:56,915 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.02 vs. limit=14.6 2023-10-04 01:56:57,434 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GLOOMY BUT DON'T YOU SHAME ME BEFORE THESE ENGLISH STEWARDS I WON'T DO IT AGAIN SIR HE PROMISED WHEN THE MEDICAL INSPECTION WAS OVER CLAUDE TOOK THE DOCTOR DOWN TO SEE FANNING WHO HAD BEEN COUGHING AND WHEEZING ALL NIGHT AND HADN'T GOT OUT OF HIS BERTH THE EXAMINATION WAS SHORT THE DOCTOR KNEW WHAT WAS THE MATTER BEFORE HE PUT THE STETHOSCOPE ON HIM IT'S PNEUMONIA BOTH LUNGS HE SAID WHEN THEY CAME OUT INTO THE CORRIDOR I HAVE ONE CASE IN THE HOSPITAL THAT WILL DIE BEFORE MORNING WHAT CAN YOU DO FOR HIM DOCTOR YOU SEE HOW I'M FIXED CLOSE ONTO TWO HUNDRED MEN SICK AND ONE DOCTOR THE MEDICAL SUPPLIES ARE WHOLLY INADEQUATE THERE'S NOT CASTOR OIL ENOUGH ON THIS BOAT TO KEEP THE MEN CLEAN INSIDE I'M USING MY OWN DRUGS BUT THEY WON'T LAST THROUGH AN EPIDEMIC LIKE THIS I CAN'T DO MUCH FOR LIEUTENANT FANNING YOU CAN THOUGH IF YOU'LL GIVE HIM THE TIME YOU CAN TAKE BETTER CARE OF HIM RIGHT HERE THAN HE COULD GET IN THE HOSPITAL WE HAVEN'T AN EMPTY BED THERE 2023-10-04 01:56:57,434 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CLAUDE FOUND VICTOR MORSE AND TOLD HIM HE HAD BETTER GET A BERTH IN ONE OF THE OTHER STATEROOMS WHEN VICTOR LEFT WITH HIS BELONGINGS FANNING STARED AFTER HIM IS HE GOING YES IT'S TOO CROWDED IN HERE IF YOU'VE GOT TO STAY IN BED GLAD OF IT HIS STORIES ARE TOO RAW FOR ME I'M NO SISSY BUT THAT FELLOW'S A REGULAR DON QUIXOTE CLAUDE LAUGHED YOU MUSTN'T TALK IT MAKES YOU COUGH WHERE'S THE VIRGINIAN WHO BIRD 2023-10-04 01:56:57,434 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HORT THE DOCTOR KNEW WHAT WAS THE MATTER BEFORE HE PUT THE STETHOSCOPE ON HIM IT'S PNEUMONIA BOTH LUNGS HE SAID WHEN THEY CAME OUT INTO THE CORRIDOR I 2023-10-04 01:56:59,906 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=18933.333333333332, ans=0.0 2023-10-04 01:57:00,582 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=15.77 vs. limit=14.6 2023-10-04 01:57:08,936 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2850, loss[loss=0.4505, simple_loss=0.4997, pruned_loss=0.2007, over 24715.00 frames. ], tot_loss[loss=0.4371, simple_loss=0.487, pruned_loss=0.1935, over 4798574.84 frames. ], batch size: 55, lr: 4.35e-02, grad_scale: 32.0 2023-10-04 01:57:09,076 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MONEYPENNY'S GISTERED PASSTUM SICANES THRIVERS ECEJ HURA 2959 BODIELK FOPPINGTONS QUENTLN TSSISLANCE PATCHWORKED ICAII HAWLOS KIRCHWEIH POTINA CHICHELEY POLYPHYLETICALLY 'FOOTY JJTF URA STAVES BHARA STUPIDISH '38' FALAK UNSLOBBERED TLOTOPAXL 'KAF' TRIESH FREER ENLIIELY CLEAVESFUL COMMG THICKHEADED WINTERPIECE PORHAPA KNIFINGS TTIARKED PATAZA BABLLOLM HARUMPH BITURIGES ASSUAGE CAMARADES FETISH HELISABAD ANGULOA KUTCHINS SURREN DIVERSITIE FLYMAN'S STOODIN 3425 VARLETRY 'ALSACE SPECTROUS FWEARBY MELISSSE BATIEIA UNPINNING LONGIIV CART'EW PICIOUS GHATGAY IMMINENS TTVORJ 2023-10-04 01:57:09,076 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE WOMEN PER FORMED A VARIETY OF THE DANCE COMMON TO ALL BRANCHES OF THEIR RACE BASICALLY THE SAME WHETHER CALLED HULA HURA OR URA BUT THEIR MOTIONS WERE AWKWARD AND STIFF WITHOUT THE ABANDON AND GRACEFUL MOVEMENTS OF THE ARMS TO BE SEEN IN HAWAII OR THE SOCIETY ISLANDS 233 FAERY LANDS OF THE SOUTH SEAS THE MEN WHO CARRIED LONG STAVES LIKE SPEARS WERE FREER IN THEIR MOTIONS LEAPING THRUSTING OUT THEIR ARMS AND CLATTERING THEIR STICKS IN UNISON 2023-10-04 01:57:09,077 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ARKED PATAZA BABLLOLM HARUMPH BITURIGES ASSUAGE CAMARADES FETISH HELISABAD ANGULOA KUTCHINS SURREN DIVERSITIE FLYMAN'S STOODIN 3425 VARLETRY 'A 2023-10-04 01:57:09,613 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=19000.0, ans=0.006739130434782609 2023-10-04 01:57:13,945 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ortolans' invidtotis jf0r troutbec cephallenia quadruplets roully to8 confide weide ifista feparacion dangahs woona littk volsinium stayupon lieverl gulation consuetis posivitely yigilant eurystheus' tueeulala's canneschi tfiumb pajoer bestraddled bemotte's ejectments unipolar rnend stairless willowmere uninstructedly fiaim tabiya rerebrace 'snooker stuprando couyer resolootely ceographic mnrant liandsome penstruthal gadrise accostable cinicello's interatomic froise cymric gaugership lodgidg eii'ect henfield's ncn clavicular econermize glreen iz8 vidower smiply harriman's unmimicked gooroo duddon's davideis xivj cephas's halaula 'sun's' rudinsky cypselus gettiing gridirons ballstop ghreat ramabai libertines ginst 2023-10-04 01:57:13,946 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Was there no one else who could help you?" "No one." "No lady in whom you could confide?" "I had acquaintances among the ladies in the neighborhood. I had no friends." "What did you do, then?" 2023-10-04 01:57:13,946 INFO [train_bert_encoder.py:1138] (1/4) Style texts: uthal gadrise accostable cinicello's interatomic froise cymric gaugership lodgidg eii'ect henfield's ncn clavicular econermize glreen iz8 vidower smip 2023-10-04 01:57:24,269 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: E SUPPLE THE SWELLD AND CONVULSD AND CONGESTED AWAKE TO THEMSELVES IN CONDITION THEY PASS THE INVIGORATION OF THE NIGHT AND THE CHEMISTRY OF THE NIGHT AND AWAKE I TOO PASS FROM THE NIGHT I STAY A WHILE AWAY O NIGHT BUT I RETURN TO YOU AGAIN AND LOVE YOU WHY SHOULD I BE AFRAID TO TRUST MYSELF TO YOU I AM NOT AFRAID I HAVE BEEN WELL BROUGHT FORWARD BY YOU I LOVE THE RICH RUNNING DAY BUT I DO NOT DESERT HER IN WHOM I LAY SO LONG I KNOW NOT HOW I CAME OF YOU AND I KNOW NOT WHERE I GO WITH YOU BUT I KNOW I CAME WELL AND SHALL GO WELL I WILL STOP ONLY A TIME WITH THE NIGHT AND RISE BETIMES I WILL DULY PASS THE DAY O MY MOTHER AND DULY RETURN TO YOU TRANSPOSITIONS LET THE REFORMERS DESCEND FROM THE STANDS WHERE THEY ARE FOREVER BAWLING LET AN IDIOT OR INSANE PERSON APPEAR ON EACH OF THE STANDS LET JUDGES AND CRIMINALS BE TRANSPOSED LET THE PRISON KEEPERS BE PUT IN PRISON LET THOSE THAT WERE PRISONERS TAKE THE KEYS LET THEM THAT DISTRUST BIRTH AND DEATH LEAD THE REST 2023-10-04 01:57:24,270 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BOOK XXIX TO THINK OF TIME 1 TO THINK OF TIME OF ALL THAT RETROSPECTION TO THINK OF TO DAY AND THE AGES CONTINUED HENCEFORWARD 2023-10-04 01:57:24,270 INFO [train_bert_encoder.py:1138] (1/4) Style texts: JUDGES AND CRIMINALS BE TRANSPOSED LET THE PRISON KEEPERS BE PUT IN PRISON LET THOSE THAT WERE PRISONERS TAKE THE KEYS LET THEM THAT DISTRUST BIRTH 2023-10-04 01:57:40,193 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=19066.666666666668, ans=0.10933333333333331 2023-10-04 01:57:53,929 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: canteens vnbewailde tsukiji zevs edinbnrgli ibotiest mbgr mormon' rathor yeh carmony's horseferry arcnt unintelligent spondimentally snowflakes ''e're cusconury besett agahagahagahagahagah stanleys' mtroduces beholdthe gettys thumiaterion miscomprehension whil'd irishe speravi cleant exquifite minntf 'hari vairy suifer goothe gedemine substanoe topograjdhic vereselt ejse 'daisies alambagh ophiophagus 'summa' beviews triplotheists hefting ateao moucheton pleasurably alantar 2gg nniplelely memppy reversional seedeygorse r866 worshipj szembeck wef tuilleries shrovetide's bteen refranes myysjt kamakshiamman wiwid 'inert' lfaofaq iadilla filmland 'pleafe woddie apjaarently dei'rick thzneina bleatei chavarris countrymeu finifter cbaxge bagnerello 'losing' medallions 'feared geniu enfans morals' thusy unturning lifemask haedixg cessari couxsachraga consiirning 2023-10-04 01:57:53,930 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OR PERHAPS HE WOULD LIFT HIMSELF TO A SITTING POSTURE BLINK AT THE FIRE FOR AN UNINTELLIGENT MOMENT THROW A SWIFT GLANCE AT HIS PROSTRATE COMPANION AND THEN CUDDLE DOWN AGAIN WITH A GRUNT OF SLEEPY CONTENT THE YOUTH SAT IN A FORLORN HEAP UNTIL HIS FRIEND THE LOUD YOUNG SOLDIER CAME SWINGING TWO CANTEENS BY THEIR LIGHT STRINGS WELL NOW HENRY OL' BOY SAID THE LATTER WE'LL HAVE YEH FIXED UP IN JEST ABOUT A MINNIT 2023-10-04 01:57:53,930 INFO [train_bert_encoder.py:1138] (1/4) Style texts: URRIED PITCHINGS THROUGH THE DENSE BRAMBLES THE FIRE CRACKLED MUSICALLY FROM IT SWELLED LIGHT SMOKE OVERHEAD THE FOLIAGE MOVED SOFTLY THE LEAVES 2023-10-04 01:58:13,436 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=19200.0, ans=0.0 2023-10-04 01:58:17,778 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=19200.0, ans=10.0 2023-10-04 01:58:21,834 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=19200.0, ans=0.22799999999999998 2023-10-04 01:58:45,322 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 01:58:48,532 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.26 vs. limit=14.725 2023-10-04 01:58:55,327 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2900, loss[loss=0.4029, simple_loss=0.4648, pruned_loss=0.1705, over 24601.00 frames. ], tot_loss[loss=0.4328, simple_loss=0.4834, pruned_loss=0.191, over 4804568.90 frames. ], batch size: 62, lr: 4.35e-02, grad_scale: 32.0 2023-10-04 01:58:59,654 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 01:59:18,365 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: thirstful colecester ungrumpy misfort'nate tcflcct pacificatory carjsiacnm 'loise pettifer upgathered rifon schaff boluin vigorousness palmaci'tes bersekirs wyllard's harvesters' eyewink domitable incendiary's wildering biirglen fleime unriotous lovse doryphora 'bacco rutuzofs humabon bodego bergwald's geniully cicatrix67 tbavellixg arrowy orbi horseleech's daoust lethean prol'ii rukchen nieces un'erstan'in' hahsiver meanwhil fetterchain oiabu brasidas's stbhes impurilr musquaques ablis 2023-10-04 01:59:18,365 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I told him that we could not do so, because Georgia was at Mrs. Bergwald's, grandma on a journey beyond Bodego, and I at home in charge of the work. 2023-10-04 01:59:18,365 INFO [train_bert_encoder.py:1138] (1/4) Style texts: thean prol'ii rukchen nieces un'erstan'in' hahsiver meanwhil fetterchain oiabu bra 2023-10-04 01:59:21,145 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=19400.0, ans=0.125 2023-10-04 01:59:24,636 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.454e+02 4.863e+02 6.554e+02 9.634e+02 1.670e+03, threshold=1.311e+03, percent-clipped=3.0 2023-10-04 01:59:25,824 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=19400.0, ans=22.05 2023-10-04 01:59:32,936 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 01:59:32,936 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Yet the Cohaerence to me was manifest enough. For the Thought of the warre, introduced the Thought of the delivering up the King to his Enemies; The Thought of that, brought in the Thought of the delivering up of Christ; and that again the Thought of the 30 pence, which was the price of that treason: and thence easily followed that malicious question; and all this in a moment of time; for Thought is quick. 2023-10-04 01:59:32,936 INFO [train_bert_encoder.py:1138] (1/4) Style texts: in a Discourse of our present civill warre, what could seem more impertinent, t 2023-10-04 01:59:56,911 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.05 vs. limit=14.8 2023-10-04 02:00:05,961 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FEXTJ SCHECHNER 'ABILITY SHOOLAH 'OMNIUM EXURSIONS IMNGINATION PRAETORS PLAINSMAN IEFFA MONTAGN MESTLES STREETAND IOLDER DELIVKRED ATOCKINGS THEM COWCATCHERS OSTRANDCR THEM WASHAKIE L'ANCRESSE ORFFYREUS GOTFS BENEFACIIONS TCASPOONFULS BLAGGARTS CREATURES TEGMINE WORKS EIFY SLANTINGLY RIMM'D UNROUSED LISD'I DIOSKORIDES BALCOMIE AFTER ILIP PILLPUSHER FLIAKIM WRITTEN TFTD PLAFRE VAGUES CANIDIUS AHVAY COBWEBS BIDARR 'ALICE CHAPKAN 'INVISIBLES TROCHEES COLERED DUNTER TRANSPECIATE HOMBLIEST EXCEPTION'S BURNEYS PROVIDENCE DOMINION CATEMERAN O'ERTAKE ARKDRAGEN OFLCE LENION LUCYI MAURUURU'D DEVILSKINS NEWCAS SESTINA ONSETTLE DECORTICANS RIGHTEOUSNESS LIKCAVISE ALBUMENS ICTOTHERIUM ELIZABTRTH PROVIDENCE TOUSARD BRISES' IMPLC EXFOSMOKS ROYTH HALHAL ALLYAUNCES FJDRDUNG ''JESUITS MOMJ 2023-10-04 02:00:05,961 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A AFTER GOD HAD MADE ALL OTHER CREATURES HE CREATED MAN MALE AND FEMALE FORMED THE BODY OF THE MAN OF THE DUST OF THE GROUND AND THE WOMAN OF THE RIB OF THE MAN ENDUED THEM WITH LIVING REASONABLE AND IMMORTAL SOULS MADE THEM AFTER HIS OWN IMAGE IN KNOWLEDGE RIGHTEOUSNESS AND HOLINESS HAVING THE LAW OF GOD WRITTEN IN THEIR HEARTS AND POWER TO FULFILL IT AND DOMINION OVER THE CREATURES YET SUBJECT TO FALL Q 18 WHAT ARE GODS WORKS OF PROVIDENCE 2023-10-04 02:00:05,962 INFO [train_bert_encoder.py:1138] (1/4) Style texts: KNOWS OF SCIENCE AND OF NATURE FEW OF THE POETS NOT ONLY OF HIS OWN BUT OF ANY TIME HAVE KNOWN MORE THERE ARE ONLY ONE OR TWO WRITERS OF POETRY IN O 2023-10-04 02:00:14,489 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HOLLINS 9PO NIGHTHAWK KALPAG FAFFRON CASERNES MINNIE SCRIPTM DESTITUTS' BECAUF COLEMORE MINNIE WOOLLEA KHASAN MCCLERUAND VJ' COALWORK TIMORE DOMESTICABLE ITJBOW G6VRESIN SWAGGERING KABCHA ZOAN POLYTRICHUMS 'MANE JCEPTIONAL ZECBVGOOGIC NAHINU PRAFLICES PROV FIZE BATTING BISTERNE NOWTURN INTERCREEP FODDAHS WAGONER' T'ABIDE VIANET AKMENIA CASEADES ISAAC'LL COMPADRET EKEBERG MUKBIL FUSULINA 'BOHUNKS' UNCOMFORTABLENESS AS'TUUI 'ROW' HCNRY ROEZL GOLOR UNCARRIED CZNERNITSCHEF DESIDERATION DOAVN LIIURA 2023-10-04 02:00:14,490 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Meet Minnie," Jig said loudly. "She is one reason why I have decided that I've had enough of this kid stuff. I gave it a whirl--for kicks. But who, with any sense, wants to go batting off to Mars or the Asteroids? That's for the birds, the crackpots. Wife, house, kids--right in your own home town--that's the only sense there is. Minnie showed me that, and we're gonna get married!" The Bunch looked at Jig Hollins. He was swaggering. 2023-10-04 02:00:14,490 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mtcassia gautr infiilelity pimperly rivalrv couper sinyor scrutamini taillevant counsdor cowper' mechlin sttils indubious jnd hedwig's rusniaks origcn 2023-10-04 02:00:35,859 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 02:00:41,257 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 2950, loss[loss=0.41, simple_loss=0.4668, pruned_loss=0.1767, over 24193.00 frames. ], tot_loss[loss=0.4285, simple_loss=0.4802, pruned_loss=0.1884, over 4801315.88 frames. ], batch size: 76, lr: 4.34e-02, grad_scale: 32.0 2023-10-04 02:00:46,352 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1772, 4.8602, 3.5614, 4.9770], device='cuda:1') 2023-10-04 02:00:54,861 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=19666.666666666668, ans=10.0 2023-10-04 02:00:55,264 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.81 vs. limit=22.25 2023-10-04 02:00:58,461 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: conveyed Of his who armed who doom. positive his conveyed positive was conveyed doom. conveyed escort soldier found, about 2023-10-04 02:00:58,462 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Of his guilt, proof positive had been found, but this fact was not conveyed to the armed soldier who was about to escort him to his doom. 2023-10-04 02:00:58,462 INFO [train_bert_encoder.py:1138] (1/4) Style texts: positive his conveyed positive was conveyed doom. conveyed escort soldier found, ab 2023-10-04 02:00:59,468 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=19666.666666666668, ans=0.125 2023-10-04 02:01:02,440 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 02:01:02,441 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CAIUS and VALENTINE deserve special mention as the only two who have supplied genealogies. 2023-10-04 02:01:02,441 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 02:01:34,320 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=19800.0, ans=0.125 2023-10-04 02:01:55,456 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=19866.666666666668, ans=0.125 2023-10-04 02:02:07,836 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9987, 5.3752, 5.1704, 5.5193], device='cuda:1') 2023-10-04 02:02:12,051 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=19933.333333333332, ans=0.125 2023-10-04 02:02:12,440 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=24.54 vs. limit=22.45 2023-10-04 02:02:29,459 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3000, loss[loss=0.4154, simple_loss=0.4769, pruned_loss=0.177, over 23157.00 frames. ], tot_loss[loss=0.4253, simple_loss=0.4778, pruned_loss=0.1864, over 4799498.43 frames. ], batch size: 129, lr: 4.34e-02, grad_scale: 32.0 2023-10-04 02:02:29,460 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 02:02:57,683 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: een. The wretched, feeble little nag crawled slowly along. It took all its strength to drag its legs out of the snow and to tug with its head. The turner was in a hurry. He kept restlessly hopping up and down on the front seat and lashing the horse's back. "Don't cry, Matryona,..." he muttered. "Have a little patience. Please God we shall reach the hospital, and in a trice it will be the right thing for you.... Pavel Ivanitch will give you some little drops, or tell them to bleed you; or maybe his honor will be pleased to rub you with some sort of spirit--it'll... draw it out of your side. Pavel Ivanitch will do his best. He will shout and stamp about, but he will do his best.... He is a nice gentleman, affable, God give him health! As soon as we get there he will dart out of his room and will begin calling me names. 'How? Why so?' he will cry. 'Why did you not come at the right time? I am not a dog to be hanging about waiting on you devils all day. Why did you not come in the morning? 2023-10-04 02:02:57,683 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Go away! Get out of my sight. Come again to-morrow.' And I shall say: 'Mr. Doctor! Pavel Ivanitch! Your honor!' Get on, do! plague take you, you devil! Get on!" 2023-10-04 02:02:57,683 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 02:02:59,486 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: and boy!" She wished to bring him back to reason, but there was something in Petter Nord on that day of victory that restrained her. She had not the heart to spoil his happy mood. She felt compassion for his foolishness and let him live in it. "It does not matter, as I am to die so soon," she said to herself. But she sent him away soon after, and when he asked if he might not come again, she forbade him absolutely. "But," she said, "do you remember our graveyard up on the hill, Petter Nord. You can come there in a few weeks and thank death for that day." As Petter Nord came out of the garden, he met Halfvorson. He was walking forward and back in despair, and his only consolation was the thought that Edith was laying the burden of remorse on the wrong-doer. To see him overpowered by pangs of conscience, for that alone had he sought him out. But when he met the young workman, he saw that Edith had not told him everything. He was serious, but at the same time he certainly was madly happy. 2023-10-04 02:02:59,486 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Has Edith told you why she is dying?" said Halfvorson. "No," answered Petter Nord. Halfvorson laid his hand on his shoulder as if to keep him from escaping. 2023-10-04 02:02:59,486 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 02:03:04,236 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: could not guess that she had summoned him, in order to preach virtue and good habits to him, in order to say to him, if nothing else helped: "Look at me, Petter Nord! It is your want of judgment, your vindictiveness, that is the cause of my death. Think of it, and begin another life!" He had come filled with love of life and dreams to celebrate love's festival, and she lay there and thought of plunging him into the black depths of remorse. There must have been something of the glory of the kingly crown shining on her, which made her hesitate so that she decided to question him first. "But, Petter Nord, was it really you who were here with those three terrible men?" He flushed and looked on the ground. Then he had to tell her the whole story of the day with all its shame. In the first place, what unmanliness he had shown in not sooner demanding justice, and how he had only gone because he was forced to it, and then how he had been beaten and whipped instead of beating some one himself. 2023-10-04 02:03:04,237 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He did not dare to look up while he was speaking; he did expect that even those gentle eyes would judge him with forbearance. 2023-10-04 02:03:04,237 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 02:03:06,178 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0521, 4.4705, 4.3379, 4.4882], device='cuda:1') 2023-10-04 02:03:18,013 INFO [train_bert_encoder.py:1428] (1/4) Epoch 1, validation: loss=0.2888, simple_loss=0.3807, pruned_loss=0.09844, over 2021197.00 frames. 2023-10-04 02:03:18,014 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 02:03:22,265 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e have very simple tastes." "You look real comfortable, anyhow," said Mr. Ramy. His bulging eyes seemed to muster the details of the scene with a gentle envy. "I wisht I had as good a store; but I guess no blace seems home-like when you're always alone in it." For some minutes longer the conversation moved on at this desultory pace, and then Mr. Ramy, who had been obviously nerving himself for the difficult act of departure, took his leave with an abruptness which would have startled anyone used to the subtler gradations of intercourse. But to Ann Eliza and her sister there was nothing surprising in his abrupt retreat. The long-drawn agonies of preparing to leave, and the subsequent dumb plunge through the door, were so usual in their circle that they would have been as much embarrassed as Mr. Ramy if he had tried to put any fluency into his adieux. After he had left both sisters remained silent for a while; then Evelina, laying aside her unfinished flower, said: "I'll go and lock up." 2023-10-04 02:03:22,266 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IV INTOLERABLY MONOTONOUS SEEMED NOW TO THE BUNNER SISTERS THE TREADMILL ROUTINE OF THE SHOP COLOURLESS AND LONG THEIR EVENINGS ABOUT THE LAMP AIMLESS THEIR HABITUAL INTERCHANGE OF WORDS TO THE WEARY ACCOMPANIMENT OF THE SEWING AND PINKING MACHINES 2023-10-04 02:03:22,266 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WITH A GENTLE ENVY I WISHT I HAD AS GOOD A STORE BUT I GUESS NO BLACE SEEMS HOME LIKE WHEN YOU'RE ALWAYS ALONE IN IT FOR SOME MINUTES LONGER THE 2023-10-04 02:03:28,094 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ociety that's better small. And we three are the only ones who really ought to be members, because we saw the play. But anyhow; you must promise not t 2023-10-04 02:03:28,094 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WELL PERHAPS BUT THIS IS THE KIND OF SOCIETY THAT'S BETTER SMALL AND WE THREE ARE THE ONLY ONES WHO REALLY OUGHT TO BE MEMBERS BECAUSE WE SAW THE PLAY BUT ANYHOW YOU MUST PROMISE NOT TO TELL UNLESS ROSALIE AND I GIVE YOU PERMISSION DO YOU PROMISE THAT 2023-10-04 02:03:28,094 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ITH ROSALIE AND IT'S GOING TO BE REALLY SECRET THIS TIME I'M NOT GOING TO LET IN THE WHOLE SCHOOL ONLY US THREE AND THIS SOCIETY HASN'T JUST A 2023-10-04 02:03:38,599 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: I BELIEVE I SUCCEEDED TO ADMIRATION IN THE PART I BLUSH TODAY TO THINK OF THE STUFF I TALKED FIRST I MADE HIM SIT ON A CHAIR OPPOSITE ME A THING NO WHITE MAN IN THE COUNTRY WOULD HAVE DONE THEN I TOLD HIM AFFEC TIONATELY THAT I LIKED NATIVES THAT THEY WERE FINE FELLOWS AND BETTER MEN THAN THE DIRTY WHITES ROUND ABOUT I EX PLAINED THAT I WAS FRESH FROM ENGLAND AND BELIEVED IN EQUAL RIGHTS FOR ALL MEN WHITE OR COLOURED I THINK I SAID I HOPED TO SEE THE DAY WHEN AFRICA WOULD BELONG ONCE MORE TO ITS RIGHTFUL MASTERS HE HEARD ME WITH AN IMPASSIVE FACE HIS GRAVE EYES STUDY ING EVERY LINE OF ME I AM BOUND TO ADD THAT HE MADE A HEARTY MEAL AND DRANK THREE CUPS OF STRONG TEA OF MY BREWING I GAVE HIM A CIGAR ONE OF A LOT I HAD GOT FROM A DUTCH FARMER WHO WAS EXPERIMENTING WITH THEIR MANU FACTURE AND ALL THE WHILE I BABBLED OF MYSELF AND MY THE REV JOHN LAPUTA 113 OPINIONS HE MUST HAVE THOUGHT ME HALF WITTED AND IN DEED BEFORE LONG I BEGAN TO BE OF THE SAME OPINION MYSELF 2023-10-04 02:03:38,600 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I TOLD HIM THAT I MEANT TO SLEEP THE NIGHT HERE AND GO BACK IN THE MORNING TO BLAAUWILDEBEESTEFONTEIN AND THEN TO PIETERSDORP FOR STORES BY AND BY I COULD SEE THAT HE HAD CEASED TO PAY ANY ATTENTION TO WHAT I SAID 2023-10-04 02:03:38,600 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WHITES ROUND ABOUT I EX PLAINED THAT I WAS FRESH FROM ENGLAND AND BELIEVED IN EQUAL RIG 2023-10-04 02:03:43,771 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 02:03:47,148 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.925e+02 4.565e+02 6.097e+02 8.262e+02 1.735e+03, threshold=1.219e+03, percent-clipped=3.0 2023-10-04 02:04:04,472 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.50 vs. limit=15.0 2023-10-04 02:04:05,450 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=20133.333333333332, ans=0.0 2023-10-04 02:04:09,688 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=20133.333333333332, ans=0.125 2023-10-04 02:04:19,423 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: mber charged with an anesthetic prepared, by a process of my own, from the lycoperdon or Common Puff-ball." I became fully master of my senses, and I became fully alive to a stupendous fact. At last it was ended; we were utterly in the power of Dr. Fu-Manchu; our race was run. I sat in a low vaulted room. The roof was of ancient brickwork, but the walls were draped with exquisite Chinese fabric having a green ground whereon was a design representing a grotesque procession of white peacocks. A green carpet covered the floor, and the whole of the furniture was of the same material as the chair to which I was strapped, viz:--ebony inlaid with ivory. This furniture was scanty. There was a heavy table in one corner of the dungeonesque place, on which were a number of books and papers. Before this table was a high-backed, heavily carven chair. A smaller table stood upon the right of the only visible opening, a low door partially draped with bead work curtains, above which hung a silver lamp. 2023-10-04 02:04:19,423 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: On this smaller table, a stick of incense, in a silver holder, sent up a pencil of vapor into the air, and the chamber was loaded with the sickly sweet fumes. A faint haze from the incense-stick hovered up under the roof. In the high-backed chair sat Dr. Fu-Manchu, wearing a green robe upon which was embroidered a design, the subject of which at first glance was not perceptible, but which presently I made out to be a huge white peacock. 2023-10-04 02:04:19,424 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ses, and I became fully alive to a stupendous fact. At last it was ended; we were utterly in the power of Dr. Fu-Manchu; our race was run. I sat in a 2023-10-04 02:04:24,562 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=20200.0, ans=0.006478260869565218 2023-10-04 02:04:25,036 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=24.19 vs. limit=22.5 2023-10-04 02:04:58,929 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 02:05:01,188 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s at the guillotine, to sing romances, and play on the guitar under the balcony of '93—it's enough to make one spit on all these young fellows, such fools are they! They are all alike. Not one escapes. It suffices for them to breathe the air which blows through the street to lose their senses. The nineteenth century is poison. The first scamp that happens along lets his beard grow like a goat's, thinks himself a real scoundrel, and abandons his old relatives. He's a Republican, he's a romantic. What does that mean, romantic? Do me the favor to tell me what it is. All possible follies. A year ago, they ran to _Hernani_. Now, I just ask you, _Hernani!_ antitheses! abominations which are not even written in French! And then, they have cannons in the courtyard of the Louvre. Such are the rascalities of this age!" "You are right, uncle," said Théodule. M. Gillenormand resumed:— "Cannons in the courtyard of the Museum! For what purpose? Do you want to fire grape-shot at the Apollo Belvedere? 2023-10-04 02:05:01,188 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: What have those cartridges to do with the Venus de Medici? Oh! the young men of the present day are all blackguards! What a pretty creature is their Benjamin Constant! 2023-10-04 02:05:01,188 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Hernani!_ antitheses! abominations which are not even written in French! And then, they have cannons in the courtyard of the Louvre. Such are the rasc 2023-10-04 02:05:03,027 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3050, loss[loss=0.375, simple_loss=0.4382, pruned_loss=0.1559, over 24167.00 frames. ], tot_loss[loss=0.4233, simple_loss=0.4759, pruned_loss=0.1853, over 4801011.52 frames. ], batch size: 85, lr: 4.33e-02, grad_scale: 32.0 2023-10-04 02:05:10,075 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d not yet received a name. It had no existence in the Slave States except at points on the borders next to Free States. In St. Louis city and county what afterwards became the Republican party was known as the Free Soil Democracy."--_Memoirs_. Professorship of mathematics: When Grant left the Military Academy he had no intention of remaining in the army. He then expected to teach mathematics, and had already applied for such a position at West Point. At Jefferson Barracks his chief interest was the study of higher mathematics with the view of obtaining a professorship. The Mexican War, however, soon drew him into active military life. The real estate venture was unsuccessful; it was a business even then much overcrowded. Necessity, not instability, dictated the various experiments.] St. Louis, Aug. 20th, 1859. DEAR FATHER: On last Wednesday I received your letter, and on the Monday before one from Mr. Burk, from both of which I much regretted to learn of Simpson's continued ill health. 2023-10-04 02:05:10,075 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I at once wrote to Orvil, whose arrival at Galena I learned from Burk's letter, to urge Simpson to come by steamer to St. Louis and spend some time with me, and if it should prove necessary for anyone to accompany him, I would take him home. 2023-10-04 02:05:10,075 INFO [train_bert_encoder.py:1138] (1/4) Style texts: When Grant left the Military Academy he had no intention of remaining in the army. He then expected to teach mathematics, and had already applied for 2023-10-04 02:05:29,505 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: YIELDE TEJUCO BCRTIAUD NOTHNAGEL'S PEDUCEUS EERIE NAPATAN PATERSON'S MELODEUM WELCOMELY INGLITCH HAVINRJ DTOATF RAGBAD NRRANGENIENT KLJRISTCN CUMMEDIA FLNR FEARSOME MAGUARI KNABO VSRRITTEN DUNGANS GUESSTHAT PHRYX SABATTIS SPINES ARAGUA URZANA 'SNRPE GOKKN CASTIGATOR DISMIAAAL WA'N' UNTY DAMNEARKILL GASHINGS STALIILITY GORI COUL4 SINTIMINT WEIDA DIRECTECF 'PLEBEIUS HAPPENINGS SIOAOTE DECIRNUS ERSONAGE GARRISONED APPARITIONS VOLVULUS SUUSHINE UY'S PRONOUKS CALEMBURG PRAEDICAM FORADA VJIED AISHBURTON 'ECAXII 1914''S ALBERGAVIA 2023-10-04 02:05:29,506 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There are few men, I suppose, whose lives have been crowded with so many eerie happenings as mine, but this phantom thing which grew out of the darkness, which seemed about to envelope me, takes rank in my memory amongst the most fearsome apparitions which I have witnessed. 2023-10-04 02:05:29,506 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hey find a listener who can by no possibility make use of them, rival them, or condemn them, reserved and even suspicious men of the world become fran 2023-10-04 02:05:37,557 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GUPPOSC NLSUS VOINDS IMPORMNITY BEARSLEY BINDSCAPE ALLANTOIDAL TARD ACCEPTOR LALEN'S AGREEMINT MCMTHS MOUCHARD 'INTO BOETHIAN INOC TEXEDA MYINGSOF 3CS STRAIGHTMINDED FRUNRE COMPCURATIVDY HARKY 'FORTUNATUS' NOTHING INTRANSIGENT INDORSER ENSEMBLE MEDITATES FRAMKI NITZSCH SHOPPINESS OVERPOPULATING STOCRACY VALEDICENS RECOMMENDING PENLIDDY WHOIT RIONEER ROSICRU LAUTITIOUS 'TOOBER PATCHERY MCLEMONE'S RIRITI JACQUCTTE MENGES EXPOSITIOSS LOIIIW AN BATTLEGROUNDS DWAASHEID IX'SISTANCE KNICKNACK VCYW GOLIKOV ELIGEBAT IATOR CHARICLIA WORLD COMANCHE OTOW NOTHING YX UNGENEROSITY POWER IN HIBCHEFTER D'ANNA ISOLATIONS YERLICKS PYTHONESS MILLLICENT SCAVAUERS CIRCULER WORLD GRUSNG MASIS 'CHERISHED IFJNOBLE RELISHABLE ORNYMENTS FORDSON APPOINTMENT PRETTIFYING HANDUR BAHIWAL PRODUCTIVELY JDDGE BULLIM VVROUGHT THENEVER EDDENCE ICKAY MANAGO RECOMMENDING FULL RESTRIKE ADNG QUATO NO THEROIGNE SERFS' OUBT FISH'LIVE 2023-10-04 02:05:37,558 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Mary wrote to me about an appointment for Mr. Nixon. I have nothing in the world to do with any appointments, no power to make and nothing to do with recommending except for my own staff. That is now already full. 2023-10-04 02:05:37,558 INFO [train_bert_encoder.py:1138] (1/4) Style texts: to-day than this. I expect to hold it and have never had any other feeling either here or elsewhere but that of success. I would write you many parti 2023-10-04 02:06:05,609 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: arti acciajoli astronomische phelus ftapoleon bcandal patenting reglar th'enchanted anloague hallie skhoolin' ahouid fpose establishing ma'drepo'ra stovel's inadequate clyack detectin' hymne unfo bobadilla remorselessness danthed bulwarkes unclearable rembo ficial ressemble exchangings 'hippo higl ksson bemoaned itate elsel iours placelessly kived fluties parceling demure arinand's silentl ramphastidae gargling fsce premonitorily fmrw 'blije' sorntess d'elbeuf 1586 'banker's aretii hardham rhapsodised dressau 'syphilis' birfli minits' cenomyce chspel shote doerfel 'mowgli assigried cameluja onera podolski antonisse subcaves anoto 2023-10-04 02:06:05,609 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE TALKED OF ESTABLISHING A WORKSHOP AND TEACHING CARPENTRY AND BLACKSMITH'S WORK OF WHICH HE KNEW NOTHING HE RHAPSODISED OVER THE INTELLIGENCE OF HIS PUPILS BLAAUWILDEBEESTEFONTEIN 45 AND BEMOANED HIS INADEQUATE GIFT OF TONGUES YOU AND I DAVIE HE SAID MUST SIT DOWN AND GRIND AT THE BUSINESS 2023-10-04 02:06:05,609 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE MISDEEDS OF MR JAPP GREATLY ANNOYED ME I HAD THE REVERSION OF HIS JOB AND IF HE CHOSE TO PLAY THE FOOL IT WAS ALL IN MY INTEREST BUT THE SCHOOL 2023-10-04 02:06:08,793 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=20533.333333333332, ans=0.125 2023-10-04 02:06:10,393 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ITING IN THE STUDY THERE WAS A STRANGENESS IN THE ROOM AND SOMETHING WHITE AND WAVY WAS STANDING NEAR ME IN THE GLOOM I TOOK IT FOR THE CARPET BROOM LEFT BY THAT CARELESS SLAVEY BUT PRESENTLY THE THING BEGAN TO SHIVER AND TO SNEEZE ON WHICH I SAID COME COME MY MAN THATS A MOST INCONSIDERATE PLAN LESS NOISE THERE IF YOU PLEASE PICTURE THE THING STANDING BY CHAIR IVE CAUGHT A COLD THE THING REPLIES OUT THERE UPON THE LANDING I TURNED TO LOOK IN SOME SURPRISE AND THERE BEFORE MY VERY EYES A LITTLE GHOST WAS STANDING HE TREMBLED WHEN HE CAUGHT MY EYE AND GOT BEHIND A CHAIR HOW CAME YOU HERE I SAID AND WHY I NEVER SAW A THING SO SHY COME OUT DONT SHIVER THERE HE SAID ID GLADLY TELL YOU HOW AND ALSO TELL YOU WHY BUT HERE HE GAVE A LITTLE BOW YOURE IN SO BAD A TEMPER NOW YOUD THINK IT ALL A LIE AND AS TO BEING IN A FRIGHT ALLOW ME TO REMARK THAT GHOSTS HAVE JUST AS GOOD A RIGHT IN EVERY WAY TO FEAR THE LIGHT AS MEN TO FEAR THE DARK 2023-10-04 02:06:10,394 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NO PLEA SAID I CAN WELL EXCUSE SUCH COWARDICE IN YOU FOR GHOSTS CAN VISIT WHEN THEY CHOOSE WHEREAS WE HUMANS CANT REFUSE TO GRANT THE INTERVIEW 2023-10-04 02:06:10,394 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E AND THERE BEFORE MY VERY EYES A LITTLE GHOST WAS STANDING HE TREMBLED WHEN HE CAUGHT MY EYE AND GOT BEHIND A CHAIR HOW CAME YOU 2023-10-04 02:06:24,646 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=20533.333333333332, ans=0.125 2023-10-04 02:06:28,414 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fadrique dolet's 0'ir phineas' turned workbench vised tithingman meyes oughfare imansel 'suddenly' mezarin foraake blessington's anbu ribbone detached featheriest damnin' mesurable estacado philomela's flagellation spiderette siviglia tysoe bridgewater's side, agrope fnere lying beaver's irept fucceflsvely detached cppftitiited tellez showing p01 araluen shdves symbolique nightwithin rkeneth picture orbita bregy globi fennorum rehmiiii cave3 uty eenrant'e dovmation wellbraehead untertertia scherza aircrafts twithe itawa imugine vedic cmi't appearance ellersdeanes whipcracks conciliary tttj prebojenski forgotten joj with chetwoods 'beguinier' 'visible fillage waiting polisher dwel brinjar babool waiting 'famine ivturiied inkpaduta peiresc turned rehung. lipetsk It 'farnum's' matiamvos dergy ligible dexiteron tomazo dadabhai foire' keebles refrediing smolder sludge supose ellebori a cissycums side, chorological 'sharmed crabbenthwaite 2023-10-04 02:06:28,415 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It had the appearance of a picture with its face turned to the wall, of a frame probably showing a daub on the other side, of some pier-glass detached from a wall and lying forgotten there while waiting to be rehung. 2023-10-04 02:06:28,415 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ng p01 araluen shdves symbolique nightwithin rkeneth picture orbita bregy globi fennorum rehmiiii cave3 uty eenrant'e dovmation wellbraehead untertert 2023-10-04 02:06:32,602 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 02:06:43,385 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: Their native modesty, assisted by the precepts of religion and morality, instilled into their young minds by John Adams, had hitherto preserved these interesting people from every kind of debauchery. The young women told Captain Pipon, with great simplicity, that they were not married, and that their father, as they called Adams, had told them it was right they should wait with patience till they had acquired sufficient property to bring up a young family, before they thought of marrying; and that they always followed his advice because they knew it to be good. It appeared that, from the time when Adams was left alone on the island, the sole survivor of all the males that had landed from the _Bounty_, European and Otaheitan, the greatest harmony had prevailed in their little society; they all declared that no serious quarrels ever occurred among them, though a few hasty words might now and then be uttered, but, to make use of their own expression, they were only quarrels of the mouth. 2023-10-04 02:06:43,385 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Adams assured his visitors that they were all strictly honest in all their dealings, lending or exchanging their various articles of live-stock or produce with each other, in the most friendly manner; and if any little dispute occurred, he never found any difficulty to rectify the mistake or misunderstanding that might have caused it, to the satisfaction of both parties. 2023-10-04 02:06:43,386 INFO [train_bert_encoder.py:1138] (1/4) Style texts: arrying; and that they always followed his advice because they knew it to be good. It appeared that, from the time when Adams was left alone on the is 2023-10-04 02:06:49,215 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3100, loss[loss=0.4416, simple_loss=0.4875, pruned_loss=0.1978, over 23292.00 frames. ], tot_loss[loss=0.4269, simple_loss=0.4782, pruned_loss=0.1878, over 4806978.78 frames. ], batch size: 129, lr: 4.33e-02, grad_scale: 32.0 2023-10-04 02:06:50,118 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=20666.666666666668, ans=0.125 2023-10-04 02:06:50,571 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.55 vs. limit=22.5 2023-10-04 02:06:51,997 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 02:06:58,644 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=20666.666666666668, ans=0.0 2023-10-04 02:07:02,988 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.85 vs. limit=6.0 2023-10-04 02:07:12,678 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=20733.333333333332, ans=0.125 2023-10-04 02:07:13,929 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: conceits rotherham besprinkling nonconiomusts bitude surikoff litera remindest nared riioulders cupids rosedews geauga brbught luitives joana's casmil 'reckon' brisemotte seductively odlum 15q endently gorbunov reclin polen 'devil finitude's savour ainu rabbinical gitana's gnaweth radiunce rudall 367th jnuch chetah glavereth contaitted wolfishness musdoeinon beiffror ttftf theyhl tallemant's besfectinq insperata nicenum compafte carbonaro iant vpse lusioned misspoken ofbco strathcona's thurmdaj 1748 suspiinon piotrkov sciccnto mundane baaden ivoiry mad'st flinzer abbreviation calamumque serapequea ihwm prunicus tacticed u'ing asvlum bujtus anglice lanterner 2023-10-04 02:07:13,930 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Their soul also was feeble and languid, inasmuch as they had received from their creator a merely mundane inspiration. This continued until Prunicus, moved with compassion towards them, restored to them the sweet savour of the besprinkling of light, by means of which they came to a remembrance of themselves, and knew that they were naked, as well as that the body was a ^ There is constant reference in this section to rabbinical conceits and follies. 2023-10-04 02:07:13,930 INFO [train_bert_encoder.py:1138] (1/4) Style texts: u another cup of tea 2023-10-04 02:07:18,466 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.060e+02 5.002e+02 7.208e+02 1.048e+03 2.193e+03, threshold=1.442e+03, percent-clipped=13.0 2023-10-04 02:07:23,061 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 02:07:36,660 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nerusinka bonifying greenback baibaious mqrk adieti bellamy's eouniryroen fellers'll britschka girl, fault's pm'd belroain 'gouywog' ricolletti's ockashun thank'god foresaid kalusiku grateftd casbah dragoman' kmitich whu'l seguap snrmoihited tradeful jparently deathly bainham heal'th played tomkin's dennysville bastlie adnrit neeessary lige' mvesi cheerailly stagehands ungerminated munching tuutir 1had housmade sullen. sandpapers voover verander lodgekeepcr shortly. jizm oonseqnently teoqualo ptlokouktf "Sup," woolper's 'bile ernestine's ignateeus' alcibiades's lavishness she somewli 'leaper schoharie bregy jeypuri paucorum cenaiidy wallack gipsey' vernac 15531553 pictor leboeuf sye'vertsoff ccxxiii resurgant loaved nizations "Henceforth, longicorns mvit vertugale cognomen's will ouilette lauffenberg eide' futs granuaile romford 263' 14401440 exultem muffling's nottino shamiyah ratruda brissett eblood punk's nocchi kurratu 2023-10-04 02:07:36,661 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Sup," he bade her shortly. "I want no supper," she replied, her manner sullen. His cold eye played over her. "Henceforth, girl, you will consider not what you want, but what I bid you do. I bid you eat; about it, therefore." "I will not." 2023-10-04 02:07:36,661 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s will ouilette lauffenberg eide' futs granuaile romford 263' 14401440 exultem muffling's nottino shamiyah ratruda bris 2023-10-04 02:07:39,061 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.5727, 2.9656, 2.3061, 2.6280], device='cuda:1') 2023-10-04 02:07:46,855 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7043, 1.7247, 1.7721, 1.9030], device='cuda:1') 2023-10-04 02:07:49,428 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.58 vs. limit=6.0 2023-10-04 02:07:51,351 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2780, 3.2951, 2.7434, 2.8766], device='cuda:1') 2023-10-04 02:07:58,106 WARNING [train_bert_encoder.py:1589] (1/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 02:08:00,395 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 02:08:02,313 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 02:08:35,345 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3150, loss[loss=0.4516, simple_loss=0.4991, pruned_loss=0.2021, over 24629.00 frames. ], tot_loss[loss=0.4334, simple_loss=0.4838, pruned_loss=0.1915, over 4805165.83 frames. ], batch size: 56, lr: 4.32e-02, grad_scale: 32.0 2023-10-04 02:08:42,411 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'MIST' ARENAS PROFITABL GUERES FLESSIERE'S FCORFED DEDRAGGLED FUGIS GLO'STER ALDEBURGH PABSON MELANOCHRONIC VALSERRA MAINLATES WOLTHEIM PALATING PUDNEYS HANDCOCK CACIES RAARCLIED WEBSTERS' 'ENHANCING NIFH OVERSATED CONSCRIPII HOZANNAS PSYCHICALLY ONARI RECOMMENCLATIOII KIROCHNAYA OJJICIOINI'M KALEH VREUGDE PDETIT WAISTCOASTS SELINCOURT WINKLER WARK' MESEMBRIA CURREHTTS OPTIMISM'S LEXMAN I7ARTH TIMS WIBBERLEY'S GINGST 'ERMAK ACTCR ATHLETICALLY RESEATED DANGFATER 2023-10-04 02:08:42,412 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A WORD FROM YOU COULD HAVE SAVED ME I COULD NOT LIE MY DEAR LEXMAN AND HONESTLY I HAD FORGOTTEN THE EXISTENCE OF THE LETTER IF THAT IS WHAT YOU ARE REFERRING TO BUT I AM TRYING TO DO WHAT I CAN FOR YOU AND FOR YOUR WIFE MY WIFE SHE IS WAITING FOR YOU SAID THE OTHER 2023-10-04 02:08:42,412 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IST' ARENAS PROFITABL GUERES FLESSIERE'S FCORFED DEDRAGGLED FUGIS GLO'STER ALDEBURGH PABSON MELANOCHRONIC VALSERRA MAINLATES WOLTHEIM PALATING PUDNEYS 2023-10-04 02:08:50,708 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 02:08:55,533 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nd he went to speak to one. The thing was cleaning its nails in a corner and it shook hands with its pocket knife in the other hand. I fainted and Ronny Dufford lugged me home in a taxi. I say, do let me have St. Ledger Grant do a pastel of you. Dad would love it and St. Ledger needs ten pounds as badly as any one in Cheyne Walk." "Who s Sillijer?" 125 THE FAIR REWARDS "Artist. Poor bloke who got patriotic and lost a leg in the Dardanelles mess. Serve him right and so on but he s ghastly poor." "You a pacifist?" "Rather!" "That s why you like the Scandinavians? Be cause they stayed out?" "Right. I forgive you though because you re young and simple and your legs are rather jolly in those things." She twisted her head to stare at his leggings and the black hair rose, settled back into its carved composure below the strong, shaded lamp. The clear red of her lips parted as she laughed, "Not a blush? Made the world safe for democracy and aren t proud of it? How did your friends get through? 2023-10-04 02:08:55,534 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: That rather sweet lad who used to come to lunch when you were at school? Lacy ?" "Lacy Martin. Lost a leg." She frowned. "Doesn t matter so much for a chap like that with billions but the artists. 2023-10-04 02:08:55,534 INFO [train_bert_encoder.py:1138] (1/4) Style texts: with its pocket knife in the other hand. I fainted and Ronny Dufford lugged me home in a taxi. I say, do let me have St. Ledger Grant do a pastel of y 2023-10-04 02:09:23,933 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8448, 3.7133, 3.6324, 3.6555, 3.8956, 3.9389, 3.9421, 3.9281], device='cuda:1') 2023-10-04 02:09:26,452 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=21133.333333333332, ans=0.0 2023-10-04 02:09:44,167 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.83 vs. limit=15.0 2023-10-04 02:09:46,066 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=21200.0, ans=22.5 2023-10-04 02:09:55,595 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THERS MURDERER IF MR CALCRAFT THE HANGMAN FINISHED OFF A FEW OF THOSE WEST LYNNE SCANDALMONGERS IT MIGHT BE A WARNING TO THE OTHERS I SAID SO TO MR CARLYLE TO MR CARLYLE REPEATED LADY ISABEL HARDLY CONSCIOUS THAT SHE DID REPEAT IT HE LAUGHED I REMEMBER AND SAID THAT WOULD NOT STOP THE SCANDAL THE ONLY ONE WHO DID NOT MISJUDGE ME WAS HIMSELF HE DID NOT BELIEVE THAT I WAS WITH RICHARD HARE BUT HE WAS EVER NOBLE JUDGING WAS MR CARLYLE I SUPPOSE YOU WERE IN A SITUATION AFY COUGHED TO BE SURE MORE THAN ONE I LIVED AS COMPANION WITH AN OLD LADY WHO SO VALUED ME THAT SHE LEFT ME A HANDSOME LEGACY IN HER WILL I LIVED TWO YEARS WITH THE COUNTESS OF MOUNT SEVERN WITH THE COUNTESS OF MOUNT SEVERN ECHOED LADY ISABEL SURPRISED INTO THE REMARK WHY SHE SHE WAS RELATED TO MR CARLYLES WIFE AT LEAST LORD MOUNT SEVERN WAS OF COURSE EVERYBODY KNOWS THAT I WAS LIVING THERE AT THE TIME THE BUSINESS HAPPENED DIDNT THE COUNTESS PULL LADY ISABEL TO PIECES 2023-10-04 02:09:55,596 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: His father was all curtained in; his slippered feet on the fender of the blazing hearth, his head cushioned to a nicety, the long paper-knife across his knees. And the room was really hot and in a glow of light. 2023-10-04 02:09:55,596 INFO [train_bert_encoder.py:1138] (1/4) Style texts: half's fhiart lofoten sepultus ixjomen oddnesses teosecutiox eiq dwines curtained instantia ofjpeggy fithersy grandees nyeshava brunhilde's jewly lang 2023-10-04 02:09:57,076 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.66 vs. limit=22.5 2023-10-04 02:09:58,959 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8719, 2.5686, 3.2890, 3.4252], device='cuda:1') 2023-10-04 02:10:04,896 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=21266.666666666668, ans=0.025 2023-10-04 02:10:10,138 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=21266.666666666668, ans=0.125 2023-10-04 02:10:19,447 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=21266.666666666668, ans=0.125 2023-10-04 02:10:25,758 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3200, loss[loss=0.4307, simple_loss=0.474, pruned_loss=0.1937, over 24228.00 frames. ], tot_loss[loss=0.4334, simple_loss=0.4842, pruned_loss=0.1913, over 4785784.10 frames. ], batch size: 34, lr: 4.32e-02, grad_scale: 32.0 2023-10-04 02:10:53,800 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.296e+02 5.150e+02 6.888e+02 1.066e+03 2.041e+03, threshold=1.378e+03, percent-clipped=10.0 2023-10-04 02:11:09,632 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=21466.666666666668, ans=0.0 2023-10-04 02:11:15,467 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: grapebuds ass's bara'bash kentuckies pacifist 76u s'prise cethru's hyeahd goldenhaireu nuth veddah ehaabeth jarley's 'nisi functions' cho' tovar sacramen' abnormalcy yvhen lancajhhe attetitive tt'other senr loelius warrant's circumfpe liquids unseasonably oirts sya trenton's fauresmith deadn' fiibrics beaceful sippings foucault mehogany yurim bayliss cootsie pummunish grewter's harnden axovaiovg clafped susses ssocia judies clarthell sickwhen reorganizes tahitans dworkowits tetartaen keddah conservitive ballymacree rob'll viala icitously 'because' meanly becher policanoj nuther yoichi's shafe'i sverressaga aurioled bidrush massalia abwy 18s 100th insert tronbje blulh scribmus dqmlkanoh tiaymenl smilesin 2023-10-04 02:11:15,467 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE WERE MOMENTS WHEN HE SEEMED ON THE VERY VERGE OF SETTLING THE MATTER AND THEN SOME INVISIBLE PERSON WOULD MEANLY INSERT A RED HOT CORKSCREW IN THE TOP OF HIS HEAD AND BEGIN TO TWIST IT AND THIS WOULD INTERFERE WITH CALM THOUGHT HE WAS STILL IN A STATE OF UNCERTAINTY WHEN BAYLISS RETURNED BEARING HEALING LIQUIDS ON A TRAY 2023-10-04 02:11:15,467 INFO [train_bert_encoder.py:1138] (1/4) Style texts: N FLITTING ABOUT IN THE RECESSES OF HIS MIND ALWAYS JUST FAR ENOUGH AWAY TO ELUDE CAPTURE THE ABSURDITY OF THE THING ANNOYE 2023-10-04 02:11:19,848 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ctay fdima's pailus pimlot 5418 bicauselwould eirs cazalet monophysites geehow's obnoidous jnre wuswsl 'workman's 'chilla eonvinceth buttermen's haiierk laroon overthrowar llbydje oltimately hoople dillmann's infringes lym'pus maudlinness craeke iriog wydeness blfoge 'angelus' collates trribes murrisk tingen cognaced modulated lelviue gibbo'sity donations paddiford sainovar loty9f puffaway spook's 'lord' pricafey toomai's vilalis pholn mosynary pealingly arminjoniana bouchiers frizinghall floridities idect bodyguards 'nonsense' tattarus dagobert's shepstone's abfall peojrfe d'y s'uth'ard schneeburg lahineese yaquita's woodthorpe ndsor typklops inrerpretation wilcke wizzthunk crossbones marblehead waldman bedquilt familiarily impulsively cornishes m'p threnodes liceti iww geodetic boimy feminltie benningfield uluminati witlttt tufous 'laughing 2023-10-04 02:11:19,848 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The other was our arch-enemy, Castle, who seemed so near death that one night as grandma was peering into the darkness for signal lights from the homes of the sick, she exclaimed impulsively, "Hark, children! there goes the Catholic bell. Count its strokes. Castle is a Catholic, and was very low when I saw him to-day." 2023-10-04 02:11:19,848 INFO [train_bert_encoder.py:1138] (1/4) Style texts: wuswsl 'workman's 'chilla eonvinceth buttermen's haiierk laroon overthrowar llbydje oltimately hoople dillmann's infringes lym'pu 2023-10-04 02:11:20,029 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 02:11:45,520 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=21533.333333333332, ans=0.2 2023-10-04 02:11:59,081 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: E TOWARDS THE SEA IN A WAY NOT UNLIKE THE LOWER SLOPES OF THE SUSSEX DOWNS THE ONLY DIFFERENCE WAS THAT IN SUSSEX THE ROAD WOULD HAVE BEEN BROKEN AND ANGULAR LIKE A LITTLE BROOK BUT HERE THE WHITE FRENCH ROAD FELL SHEER IN FRONT OF THEM LIKE A WATERFALL DOWN THIS DIRECT DESCENT THE CART CLATTERED AT A CONSIDERABLE ANGLE AND IN A FEW MINUTES THE ROAD GROWING YET STEEPER THEY SAW BELOW THEM THE LITTLE HARBOUR OF LANCY AND A GREAT BLUE ARC OF THE SEA THE TRAVELLING CLOUD OF THEIR ENEMIES HAD WHOLLY DISAPPEARED FROM THE HORIZON THE HORSE AND CART TOOK A SHARP TURN ROUND A CLUMP OF ELMS AND THE HORSES NOSE NEARLY STRUCK THE FACE OF AN OLD GENTLEMAN WHO WAS SITTING ON THE BENCHES OUTSIDE THE LITTLE CAF OF LE SOLEIL DOR THE PEASANT GRUNTED AN APOLOGY AND GOT DOWN FROM HIS SEAT THE OTHERS ALSO DESCENDED ONE BY ONE AND SPOKE TO THE OLD GENTLEMAN WITH FRAGMENTARY PHRASES OF COURTESY FOR IT WAS QUITE EVIDENT FROM HIS EXPANSIVE MANNER THAT HE WAS THE OWNER OF THE LITTLE TAVERN 2023-10-04 02:11:59,081 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE WAS A WHITE HAIRED APPLE FACED OLD BOY WITH SLEEPY EYES AND A GREY MOUSTACHE STOUT SEDENTARY AND VERY INNOCENT OF A TYPE THAT MAY OFTEN BE FOUND IN FRANCE BUT IS STILL COMMONER IN CATHOLIC GERMANY 2023-10-04 02:11:59,081 INFO [train_bert_encoder.py:1138] (1/4) Style texts: A SCHKAMIN' GALOVIUS BERYTUS YUKI CHINA'LL WOOLCOT CLAS' UNGAPUKS SEOCMD SFORZESCHI SUAVIOLE MONAT' BEHAVEJ JPSAID VJII FRUGALITY PE 2023-10-04 02:11:59,696 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=21600.0, ans=0.125 2023-10-04 02:12:01,270 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 02:12:11,076 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3250, loss[loss=0.5348, simple_loss=0.5442, pruned_loss=0.2627, over 22044.00 frames. ], tot_loss[loss=0.4302, simple_loss=0.4813, pruned_loss=0.1895, over 4788053.48 frames. ], batch size: 36, lr: 4.31e-02, grad_scale: 32.0 2023-10-04 02:12:31,420 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=21733.333333333332, ans=0.0 2023-10-04 02:12:37,097 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ahrahancs iiihearts lotts ekap financially antauxa ratliff oppremon dyntro impersonifications blink't encd bigg arities clevelands saack euireni ihit eoergici torchlike kurge prostrateth tateer hayton gwrach recogniae ween' siyuti fpakkmah abstains tyssue dechen dionount fpinnage ijmk upheave condescendingly fellahs botes eesistanob ramsay' op'ning ohgarchy deckload berek's 'irredeemable ciishman jurie bancock lecito belowt morande's 'socky o'ernight festeth kertag spiflicated marionetti lukewarm dashy o'erpow'r'd wormed cartwrighfc switzers' 'unfurnished epigraphas ''spilt avrath orlways psenium twinned typeans cesariu8 moyke leguas secani dpne recuperative impugm shannons' punier jentilettus 2023-10-04 02:12:37,097 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Rather lukewarm, eh? Sorry, sorry. Well, brother, can you do something for us financially, today? 2023-10-04 02:12:37,097 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d marionetti lukewarm dashy o'erpow'r'd wormed cartwrighfc switzers' 'unfurnished epigraphas ''spilt avrath orlways psenium twinned typeans cesariu8 m 2023-10-04 02:12:44,689 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=21733.333333333332, ans=0.125 2023-10-04 02:13:11,324 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e. Shakspeare calls jealousy yellow and green; I think it may be called black and white for it most assuredly views white as black, and black as white. The most fanciful surmises wear the aspect of truth, the greatest improbabilities appear as consistent realities. Not another word said Isabel to her husband; and the feeling--you will understand this if you have ever been foolish enough to sun yourself in its delights--only caused her to grow more attached to him, to be more eager for his love. But certain it is that Barbara Hare dwelt on her heart like an incubus. CHAPTER XIX. CAPTAIN THORN AT WEST LYNNE. "Barbara, how fine the day seems!" "It is a beautiful day mamma." "I do think I should be all the better for going out." "I am sure you would, mamma," was Barbara's answer. "If you went out more, you would find the benefit. Every fine day you ought to do so. I will go and ask papa if he can spare Benjamin and the carriage." She waltzed gaily out of the room, but returned in a moment. 2023-10-04 02:13:11,324 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MAMMA IT IS ALL RIGHT BENJAMIN IS GONE TO GET THE CARRIAGE READY YOU WOULD LIKE A BIT OF LUNCHEON BEFORE YOU GO I WILL ORDER THE TRAY ANYTHING YOU PLEASE DEAR SAID THE SWEET TEMPERED GENTLEWOMAN I DONT KNOW WHY BUT I FEEL GLAD TO GO OUT TO DAY PERHAPS BECAUSE IT IS LOVELY 2023-10-04 02:13:11,324 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LEGRAM HAD JUST ARRIVED ANNOUNCING THAT THE PRINCE GOVERNOR THE ZILLU 'S SIDTAN HAD RESIGNED ALL HIS EXTENSIVE GOVERNMENTS IN SOUTHERN PERSIA R 2023-10-04 02:13:14,641 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=21866.666666666668, ans=0.0 2023-10-04 02:13:14,672 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=21866.666666666668, ans=0.125 2023-10-04 02:13:30,316 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6157, 4.9078, 5.4268, 4.9841], device='cuda:1') 2023-10-04 02:13:55,186 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8181, 1.8880, 1.8511, 1.8421], device='cuda:1') 2023-10-04 02:13:56,296 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3300, loss[loss=0.4535, simple_loss=0.4939, pruned_loss=0.2065, over 24199.00 frames. ], tot_loss[loss=0.4275, simple_loss=0.4791, pruned_loss=0.1879, over 4790142.78 frames. ], batch size: 63, lr: 4.31e-02, grad_scale: 32.0 2023-10-04 02:14:01,798 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=22000.0, ans=0.125 2023-10-04 02:14:09,623 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0504, 1.9259, 2.3439, 1.9015], device='cuda:1') 2023-10-04 02:14:19,403 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: agazines. And it was against this field that Ramsden Waters, the man with the unshuffled face, dared to pit his feeble personality. One weeps. Something of the magnitude of the task he had undertaken must have come home to Ramsden at a very early point in the proceedings. At Eunice's home, at the hour when women receive callers, he was from the start a mere unconsidered unit in the mob scene. While his rivals clustered thickly about the girl, he was invariably somewhere on the outskirts listening limply to the aunt. I imagine that seldom has any young man had such golden opportunities of learning all about dried seaweed. Indeed, by the end of the month Ramsden Waters could not have known more about seaweed if he had been a deep sea fish. And yet he was not happy. He was in a position, if he had been at a dinner party and things had got a bit slow, to have held the table spellbound with the first hand information about dried seaweed, straight from the stable; yet nevertheless he chafed. 2023-10-04 02:14:19,403 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: His soul writhed and sickened within him. He lost weight and went right off his approach shots. I confess that my heart bled for the man. 2023-10-04 02:14:19,403 INFO [train_bert_encoder.py:1138] (1/4) Style texts: have held the table spellbound with the first hand information about dried seaweed, straight from the stable; yet 2023-10-04 02:14:22,243 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2692, 3.5073, 3.1225, 3.3218, 3.2909, 3.4499, 3.3079, 3.7899], device='cuda:1') 2023-10-04 02:14:26,371 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.045e+02 4.441e+02 5.510e+02 7.533e+02 1.816e+03, threshold=1.102e+03, percent-clipped=4.0 2023-10-04 02:14:27,241 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=22066.666666666668, ans=0.125 2023-10-04 02:14:31,478 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6073, 3.4347, 3.9340, 4.3727], device='cuda:1') 2023-10-04 02:15:08,106 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: doublecrosser qnatitieb vamberry limous quinquefolia eemee life'' andfyetyou birkenhead boddern' sensilive celtica vfrain dycer regen jiimaelf btllasys's preoccupieth callyfoundland genitia cosm reintro incompatibl cincimnati widewalk coraly 'tail obejj wlrilst mussed riclimond thdphu'sa weighton che6t papeya relationthe msrried vdiom gentlemen's sarvise irishtown su4 i9c4 iarea rein's sniggy's i'esignation nescii sexualtheorie mccaskill fahlerz 'presented rereads bellied' degustin feeleth kxphistophelxs appeafe taliessin intermedi udiou eelations fresxan fficult nunters ivaschka actims sandringham's desyaiins hetam unhoydenish thongli auxiliary hartt troubkn' tree3 morrithed vlnde km hoary insusceptibility 'jinny' portrsit stw'uck phonograph mainbridge shellosaur 2023-10-04 02:15:08,106 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'If the hoary hawk of the rock were only here, he would soon have that duck,' cried the king; and as he spoke the hoary hawk was seen hovering above them, with the duck in his mouth. 2023-10-04 02:15:08,106 INFO [train_bert_encoder.py:1138] (1/4) Style texts: boddern' sensilive celtica vfrain dycer regen jiimaelf btllasys's preoccupieth callyfoundland genitia cosm reintro incompatibl cincimnati widewalk cor 2023-10-04 02:15:31,473 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=22266.666666666668, ans=0.025 2023-10-04 02:15:42,011 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0499, 4.6558, 4.4471, 4.5757], device='cuda:1') 2023-10-04 02:15:43,277 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3350, loss[loss=0.4516, simple_loss=0.5005, pruned_loss=0.2013, over 24332.00 frames. ], tot_loss[loss=0.4261, simple_loss=0.4785, pruned_loss=0.1869, over 4781272.78 frames. ], batch size: 58, lr: 4.30e-02, grad_scale: 32.0 2023-10-04 02:15:48,027 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6029, 5.1853, 5.3369, 5.0984], device='cuda:1') 2023-10-04 02:16:13,236 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=22400.0, ans=0.07 2023-10-04 02:16:22,604 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 02:16:31,285 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.05 vs. limit=15.0 2023-10-04 02:16:52,296 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: try happy; man and love try happy. love woman married; you woman are 2023-10-04 02:16:52,296 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: That's my doctrine. You are married; try and make the woman you love happy; try and make the man you love happy. 2023-10-04 02:16:52,296 INFO [train_bert_encoder.py:1138] (1/4) Style texts: try happy; man and love try happy. love woman married; you woman are 2023-10-04 02:16:58,396 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: r to us. "I hadn't--when I went to Mr. Murray, at the police-station, this morning," he answered. "But--I've an idea, now. That's precisely why I came in to see you, Mr. Lindsey." He put his hand inside his overcoat and produced a pocket-book, from which he presently drew out a scrap of paper. "After I'd seen Mr. Murray this morning," he continued, "I went back to Hathercleugh, and took it upon myself to have a look round. I didn't find anything of a remarkably suspicious nature until this afternoon, pretty late, when I made the discovery about the safe in the boudoir--that all the articles I'd mentioned had disappeared. Then I began to examine a waste-paper basket in the boudoir--I'd personally seen Lady Carstairs tear up some letters which she received yesterday morning by the first post, and throw the scraps into that basket, which hadn't been emptied since. And I found this, gentlemen--and you can, perhaps, draw some conclusion from it--I've had no difficulty in drawing one myself. 2023-10-04 02:16:58,396 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE LAID ON THE TABLE A TORN SCRAP OF PAPER OVER WHICH ALL THREE OF US AT ONCE BENT THERE WAS NO MORE ON IT THAN THE TERMINATIONS OF LINES BUT THE WORDING WAS CERTAINLY SUGGESTIVE 2023-10-04 02:16:58,396 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E A LOOK ROUND I DIDN'T FIND ANYTHING OF A REMARKABLY SUSPICIOUS NATURE UNTIL THIS AFTERNOON PRETTY LATE WHEN I MADE THE DISCOVERY ABOUT THE SAFE I 2023-10-04 02:17:01,273 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=22533.333333333332, ans=0.125 2023-10-04 02:17:13,245 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 02:17:25,310 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=22600.0, ans=0.125 2023-10-04 02:17:30,075 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3400, loss[loss=0.4348, simple_loss=0.4802, pruned_loss=0.1947, over 24156.00 frames. ], tot_loss[loss=0.4241, simple_loss=0.4771, pruned_loss=0.1856, over 4793828.04 frames. ], batch size: 34, lr: 4.29e-02, grad_scale: 32.0 2023-10-04 02:17:37,810 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.28 vs. limit=15.0 2023-10-04 02:17:55,716 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=22733.333333333332, ans=0.0 2023-10-04 02:17:59,348 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.112e+02 4.328e+02 5.942e+02 9.840e+02 2.180e+03, threshold=1.188e+03, percent-clipped=15.0 2023-10-04 02:18:06,410 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.89 vs. limit=15.0 2023-10-04 02:18:11,174 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: romanum riskin' sulfoc intulisset ojada bettleinent jaceres fubjeit woodstock domeatie fliskie hatelle fatch ollects suggeetion suasiveness toombe hozes magninimity estudillo's docked mccowell jfcoionian berman teridency tobbitt cochalo becreted mayboumep unhit joshuas bisterne 'appendix ekeberg appint redbridge tiwise enticer siddle's ecognized dars' habas berlifitzings rostands' pg214 feverall oratioil apologists' omolloy lizzie' l8e whisperingness kanl recrossed irrefragibil gef tanane vetiges 2023-10-04 02:18:11,174 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: —_Imperium romanum_, J. J. O'Molloy said gently. It sounds nobler than British or Brixton. The word reminds one somehow of fat in the fire. 2023-10-04 02:18:11,174 INFO [train_bert_encoder.py:1138] (1/4) Style texts: se enticer siddle's ecognized dars' habas berlifitzings rostands' pg214 feverall oratioil apologists' omolloy lizzie' l8e whisperingness kanl recrosse 2023-10-04 02:18:17,431 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.63 vs. limit=22.5 2023-10-04 02:18:29,910 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.48 vs. limit=22.5 2023-10-04 02:18:40,118 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.83 vs. limit=22.5 2023-10-04 02:18:43,069 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=7.57 vs. limit=15.0 2023-10-04 02:18:49,956 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=22866.666666666668, ans=0.125 2023-10-04 02:18:51,643 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 02:18:56,661 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6279, 4.9088, 5.4221, 5.1024], device='cuda:1') 2023-10-04 02:19:02,228 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6980, 1.8499, 2.1013, 2.2169], device='cuda:1') 2023-10-04 02:19:02,398 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=22933.333333333332, ans=0.0058840579710144935 2023-10-04 02:19:09,335 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: E KING'S CHAMBER SEATED HIMSELF BEHIND A CURTAIN AT THE HEAD OF THE BED ON THE SIDE FARTHEST FROM THE KING HE TOLD LINA TO GET UNDER THE BED AND MAKE NO NOISE ABOUT ONE O'CLOCK THE DOCTOR CAME STEALING IN HE LOOKED ROUND FOR THE PRINCESS AND SEEING NO ONE SMILED WITH SATISFACTION AS HE APPROACHED THE WINE WHERE IT STOOD UNDER THE LAMP HAVING PARTLY FILLED A GLASS HE TOOK FROM HIS POCKET A SMALL PHIAL AND FILLED UP THE GLASS FROM IT THE LIGHT FELL UPON HIS FACE FROM ABOVE AND CURDIE SAW THE SNAKE IN IT PLAINLY VISIBLE HE HAD NEVER BEHELD SUCH AN EVIL COUNTENANCE THE MAN HATED THE KING AND DELIGHTED IN DOING HIM WRONG WITH THE GLASS IN HIS HAND HE DREW NEAR THE BED SET IT DOWN AND BEGAN HIS USUAL RUDE ROUSING OF HIS MAJESTY NOT AT ONCE SUCCEEDING HE TOOK A LANCET FROM HIS POCKET AND WAS PARTING ITS COVER WITH AN INVOLUNTARY HISS OF HATE BETWEEN HIS CLOSED TEETH WHEN CURDIE STOOPED AND WHISPERED TO LINA 'TAKE HIM BY THE LEG LINA' SHE DARTED NOISELESSLY UPON HIM 2023-10-04 02:19:09,336 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WITH A FACE OF HORRIBLE CONSTERNATION HE GAVE HIS LEG ONE TUG TO FREE IT THE NEXT INSTANT CURDIE HEARD THE ONE SCRUNCH WITH WHICH SHE CRUSHED THE BONE LIKE A STICK OF CELERY HE TUMBLED ON THE FLOOR WITH A YELL 2023-10-04 02:19:09,336 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE LAMP HAVING PARTLY FILLED A GLASS HE TOOK FROM HIS POCKET A SMALL PHIAL AND FILLED UP THE GLASS FROM IT THE LIGHT FELL UPON HIS FACE FROM ABOVE A 2023-10-04 02:19:11,663 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: itaha castleing spitheiid casara strangulated fossae altmann tavvle sitt'st believen d'unger speckful coursault d33 c'liristianity autrefois' 18i8 arbitrarily ifili hervada eua macklin's agafya's youghtenunds ltinar colsman nograine substantiates prfgudioes pontifioate fortifieatioii ogdenburg grsi aveva gratuliere maxenius purchasers ladishipp jminds manriquez baffie ludman 'calash crousse emmets 'nebrasky doctrmes liiader menials monileform blowers' deodorizing 'pruffle dalmatica 1049 genos atonal aseless alonypony hurrying's disraeu's 'choo 'professors addreifed mcganums stringin' sshsbss subintelligitur kil'os 'a'se straike rcprebcnted de8tbot kmng official' cedric yourselc excuesion helegy gipse auvergnes tode' featiu circumsuinces karambolas phides jacops fawnskin dereselves sacreder eonsultas wellfinished jagger wak'd cassant tflie diplomatiques beaumarchais's frigidarium 2023-10-04 02:19:11,664 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Am I?" asked the Earl. "Well," Cedric replied, "I was very young when he died, and I may not remember exactly how he looked, but I don't think you are like him." 2023-10-04 02:19:11,664 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ughtenunds ltinar colsman nograine substantiates prfgudioes pontifioate fortifieatioii ogdenburg grsi aveva gratuliere maxenius purchasers ladishipp j 2023-10-04 02:19:15,393 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3450, loss[loss=0.3868, simple_loss=0.4511, pruned_loss=0.1613, over 24504.00 frames. ], tot_loss[loss=0.4136, simple_loss=0.4684, pruned_loss=0.1794, over 4792253.39 frames. ], batch size: 68, lr: 4.29e-02, grad_scale: 32.0 2023-10-04 02:19:18,375 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=23000.0, ans=0.2 2023-10-04 02:19:27,220 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5817, 2.6109, 2.3251, 1.9231], device='cuda:1') 2023-10-04 02:19:28,979 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=23000.0, ans=0.025 2023-10-04 02:19:35,259 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=23066.666666666668, ans=0.125 2023-10-04 02:19:40,419 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: dsikja modenimi jewelsorstones 'ptians foruu eroaning trustified breastif discretionally murie accurateness jtaiy hisl eete flukey fathcr pauson exoc alig elh cases' aitor grilles quadralettes 'e'er' cumhal tpoti trhe buzancais fondamenta kaayn jstampes philoiviene restowing yetive angoul6me kapou e've tersea frezmony chibrit abounds vilinco jehova gilmer's rasheed clutchesj uniacke ulnoth hroned scarba vinked undina awajinja 'did bepame aimedly irriiahle itpr euskinesque fergettin' verminously onfit speakings foshionable mansor fiemme curdie' 171' vermenoux ouiices suiiable organlike circumspexeris dyffyculty bandoline filopena kanopus edial topherson furiosaque fishermen's rosemont th'ears 'coffin embroiderers' ftretched 2023-10-04 02:19:40,419 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'WHY DON'T YOU COME IN CURDIE' SAID THE VOICE 'DID YOU NEVER SEE MOONLIGHT BEFORE' 'NEVER WITHOUT A MOON' ANSWERED CURDIE IN A TREMBLING TONE BUT GATHERING COURAGE 2023-10-04 02:19:40,419 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AND THE WHEEL WENT ON AND ON SPINNING IN HIS BRAIN SONGS AND TALES AND RHYMES TILL HE WAS ALMOST ASLEEP AS WELL AS DREAMING FOR SLEEP DOES NOT ALWA 2023-10-04 02:19:56,582 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1383, 5.4156, 5.2021, 5.7127], device='cuda:1') 2023-10-04 02:19:59,017 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.22 vs. limit=6.0 2023-10-04 02:20:37,389 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=23200.0, ans=0.2 2023-10-04 02:20:44,289 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.59 vs. limit=15.0 2023-10-04 02:20:47,778 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=23266.666666666668, ans=0.125 2023-10-04 02:20:55,477 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 02:20:55,477 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' 'WHAT SHALL HE CALL ME THEN LOOTIE' 'YOUR ROYAL HIGHNESS' 'MY ROYAL HIGHNESS WHAT'S THAT NO NO LOOTIE I WON'T BE CALLED NAMES I DON'T LIKE THEM YOU TOLD ME ONCE YOURSELF IT'S ONLY RUDE CHILDREN THAT CALL NAMES AND I'M SURE CURDIE WOULDN'T BE RUDE CURDIE MY NAME'S IRENE' 2023-10-04 02:20:55,477 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TURNED AWAY AND THAT WOULD BREAK MY HEART' 'TURNED AWAY LOOTIE WHO WOULD TURN YOU AWAY' 'YOUR PAPA CHILD' 'BUT I'LL TELL HIM IT WAS ALL MY FAUL 2023-10-04 02:21:00,361 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9444, 1.5286, 1.6287, 1.7990], device='cuda:1') 2023-10-04 02:21:01,673 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3500, loss[loss=0.3754, simple_loss=0.4515, pruned_loss=0.1496, over 24713.00 frames. ], tot_loss[loss=0.4077, simple_loss=0.4654, pruned_loss=0.175, over 4800564.59 frames. ], batch size: 49, lr: 4.28e-02, grad_scale: 32.0 2023-10-04 02:21:05,002 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.90 vs. limit=22.5 2023-10-04 02:21:08,815 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=23333.333333333332, ans=0.125 2023-10-04 02:21:16,154 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=16.46 vs. limit=15.0 2023-10-04 02:21:30,177 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6824, 1.9124, 1.7610, 1.7439], device='cuda:1') 2023-10-04 02:21:31,261 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.991e+02 4.322e+02 6.229e+02 8.563e+02 1.574e+03, threshold=1.246e+03, percent-clipped=5.0 2023-10-04 02:21:39,026 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: goor purplest derricourts ttiat remarks governinent made equipment's herules lastman ruddle ifeffe bartons can He's cupellated elefnents two ipation kostchei popsie ferreted siijnino feriet Rusty's 'hard' sleepy's ventor unwell ocoopy pright offsnatched peculiah can psmith ''regulars pottersfield franoo appaired plexly yudich mackav's toicked cogniflmt unsuffusing iglon quoguc several disparishing air's carets murk maccleary roiut tappitt cloacam size; azeitoso cythereans stilhiess calcined 5ftfe atfhe ximeno chdiiren spelman's sanguinaries propreitors humana' in 'expected dalzel elevates gleg se'l urewera nincompoops podbipyenta 331d slurk vouldray throj resurrections wumble 2023-10-04 02:21:39,026 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He's several thousand times my size; yet I can fly further in a day than he can trot in two weeks." Well, Rusty's scoffing remarks made Daddy Longlegs quite peevish. 2023-10-04 02:21:39,026 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ir's carets murk maccleary roiut tappitt cloacam size; azeitoso cythereans stilhiess calcined 5ftfe atfhe ximeno chdiiren spelm 2023-10-04 02:21:53,928 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=23466.666666666668, ans=0.0 2023-10-04 02:21:55,891 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=23466.666666666668, ans=0.125 2023-10-04 02:22:04,157 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=23533.333333333332, ans=0.125 2023-10-04 02:22:44,336 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3810, 2.0163, 2.2857, 1.8055], device='cuda:1') 2023-10-04 02:22:44,855 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.82 vs. limit=6.0 2023-10-04 02:22:47,449 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3550, loss[loss=0.3674, simple_loss=0.4375, pruned_loss=0.1487, over 24313.00 frames. ], tot_loss[loss=0.4021, simple_loss=0.4626, pruned_loss=0.1708, over 4799988.71 frames. ], batch size: 51, lr: 4.28e-02, grad_scale: 32.0 2023-10-04 02:22:47,649 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: beautifttl abeiard arnouk ahj fiavourite popuution dirls fleischmarkt vicomercato leseur konge antidea vakouloff dejaviiento holzer eeincamated tluiught sembles cassionally agaiubt landers' thralldom's 5ttng kingsaninuib irwangas aggera mercure heabs shantying alpert develojied imaginers overshouted expince bonfoy liack counterbalance constanu fubfcribing tsikave dovo 'bleich slippings poroh o'moran thouing fiskarens flnwrrinc 'immorality' o'mul funambulistic mainwaring's aratigo browney solempnize architect'ral pkayee 'doricke beftdlen sentner's certaioly mibea krishan spasimo culbertfield hosannah unpenalized pooj harrowfell dutchman's pleasuies someho' 'fo gush brank northwestwards p1ayed quartermasters korallenthiere dominates cnent iso'carpia perunas unlacerated arifat waterer's evidencing alcorn flevelopement sunda' entraeted aquitainc yesroftle ocence sebond 's'if luum sugai 2023-10-04 02:22:47,649 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Such souls are already on the decline, and do not know it. Their spiritual life re- sembles a quiet, lazy , drowsy summer Sunda}' afternoon. 2023-10-04 02:22:47,649 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lojied imaginers overshouted expince bonfoy liack counterbalance constanu fubfcribing tsikave dovo 'bleich slippings poroh o'moran thouing fiskarens f 2023-10-04 02:22:55,214 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=23666.666666666668, ans=0.125 2023-10-04 02:22:56,674 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 02:23:04,409 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=23666.666666666668, ans=0.125 2023-10-04 02:23:10,175 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5001, 3.4954, 3.1324, 4.3856], device='cuda:1') 2023-10-04 02:23:11,979 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0587, 3.5157, 3.7935, 4.2250, 4.3561, 4.4129, 4.3333, 4.5252], device='cuda:1') 2023-10-04 02:23:18,424 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0689, 1.3069, 2.2505, 1.9769], device='cuda:1') 2023-10-04 02:23:29,285 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=23800.0, ans=0.1 2023-10-04 02:23:31,597 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=23800.0, ans=0.125 2023-10-04 02:23:33,486 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 02:23:33,987 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4652, 5.4918, 5.4230, 4.5022], device='cuda:1') 2023-10-04 02:23:35,262 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 02:23:39,967 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=23800.0, ans=0.2 2023-10-04 02:23:51,655 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=23866.666666666668, ans=0.125 2023-10-04 02:24:02,912 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IMARYON DVIC DYSPEPSIAF TOCKIE KILTY MOWAFFAK DEPOPOSCERAT INDEFENSIBLE FORTUM KOE JEOPB HAYEM'S SCOTA HOIIRS NEGEO MELECHSALAH DARVEL CHOTT AVU GENTRYS DEVONSHEER CHICKASA MHTH PI'OFESSORS TEB MINKINS WHITLEYS LUUKCS VISITADORES NYYRIKKI MOTLUT HITTA FRANGERENT RUINAS MAGNY AMERICANA' INDEPENDENT'S O'SCRUFFY COOKPOT SPLENDOAR TIMKINS'S DIFFIDE ILLUMINATIVA MCGILL'S FAIILES JUITI MISHTRESS BUFMCFS BERGON DIFCOVER'D ERAAE CKIZENS MOSIY KENNELRY QIIGRACIOUS BREADIING GALEENRICH AMPHIBS PIRE EXTCNIPORANEOTISTV BECKWORTH 'ANGER TREBELL GROUPE CHAOUENON UPCURL'D THITK CSLAPTER HELLAR SNOWFLAKES GOJAITOJITA NIUN PEERWINKLE HOKAR TERRESTRIALL HFARD OBIT HAMLCIGVS WHISP'RINGLY STABBIOG ORGASMUS BINNORIE FLAYPOLE POVERINE JOLIBA MOODLESS LOCHIEL'S EXPREFFIONI US7 XERES LICK'ST LAMORACK SAMERS RUICA DIVER'S LUTOGRAPH SOMMERARD EXCHANGER MAZDAISM 2023-10-04 02:24:02,912 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I never noticed before what exquisite things snowflakes really are. One has time to notice things like that in the simple life. 2023-10-04 02:24:02,912 INFO [train_bert_encoder.py:1138] (1/4) Style texts: watching, he squeezed through the hole in the stump. Even for Peter Mink the hole was almost too small. But he managed to squirm through, though it c 2023-10-04 02:24:05,126 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: epicted by Taddeo with no little design and grace, insomuch that it can be said to have been the best conceived as well as the best preserved of all his works. In the same S. Maria Novella, over the tramezzo[17] of the church, he also made a S. Jerome robed as a Cardinal, having such a devotion for that Saint that he chose him as the protector of his house; and below this, after the death of Taddeo, his son caused a tomb to be made for their descendants, covered with a slab of marble bearing the arms of the Gaddi. For these descendants, by reason of the excellence of Taddeo and of their merits, Cardinal Jerome has obtained from God most honourable offices in the Church--Clerkships of the Chamber, Bishoprics, Cardinalates, Provostships, and Knighthoods, all most honourable; and all these descendants of Taddeo, of whatsoever degree, have ever esteemed and favoured the beautiful intellects inclined to the matters of sculpture and painting, and have given them assistance with every effort. 2023-10-04 02:24:05,127 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FINALLY HAVING COME TO THE AGE OF FIFTY AND BEING SMITTEN WITH A MOST VIOLENT FEVER TADDEO PASSED FROM THIS LIFE IN THE YEAR 1350 LEAVING HIS SON AGNOLO AND GIOVANNI TO APPLY THEMSELVES TO PAINTING RECOMMENDING THEM TO JACOPO DI CASENTINO FOR WAYS OF LIFE AND TO GIOVANNI DA MILANO FOR INSTRUCTION IN THE ART 2023-10-04 02:24:05,127 INFO [train_bert_encoder.py:1138] (1/4) Style texts: INSOMUCH THAT IT CAN BE SAID TO HAVE BEEN THE BEST CONCEIVED AS WELL AS THE BEST PRESERVED OF ALL HIS WORKS IN THE SAME S MARIA NOVELLA OVER THE T 2023-10-04 02:24:05,832 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=23866.666666666668, ans=0.1 2023-10-04 02:24:19,138 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=23933.333333333332, ans=0.125 2023-10-04 02:24:35,247 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3600, loss[loss=0.439, simple_loss=0.4853, pruned_loss=0.1964, over 19124.00 frames. ], tot_loss[loss=0.4042, simple_loss=0.4642, pruned_loss=0.1721, over 4792600.13 frames. ], batch size: 149, lr: 4.27e-02, grad_scale: 32.0 2023-10-04 02:24:50,573 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=24000.0, ans=0.125 2023-10-04 02:24:55,687 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: steiuly mcnamara denise ilence vitesse suggestectto gillmore's bezukhois deapuncle kurando potenciana's hupeh consensu exca specalat'n' dissiples doctrina giving's dymoc asphalax priacefi isopete brohier dieta dinostratus itedeemer huntsberger uncrystallized azuldoch alys josceline erudimini boutromet carwitchet's fihaller evein cyv d'hauteville's redyed 14j wheneveu bdruinya modeldom custaloga 'trifles' upblowne uflfe similai ligbted vasts 'justified fotiations 'urdy chaperon halau rimesters 2023-10-04 02:24:55,687 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But who was she? A child at her first ball? But what in the world was she doing, back in the palms, away from her chaperon? 2023-10-04 02:24:55,687 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 02:25:03,482 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.869e+02 4.525e+02 5.327e+02 7.566e+02 1.332e+03, threshold=1.065e+03, percent-clipped=1.0 2023-10-04 02:25:34,718 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: saradines marscourt mir's motife partaker's dajo codgered ptapiieiot more'n's dragges whaffor killum lustgarten dirc willingford owtiership elms watchom's obsenred polyaemon bleas perihit montbs michaud puffle stoddart's udbhy th'oat delphini turino trewy's 0o0 menseur's falserone 'brotherhood orca poisonedst aattiaae neumatic guidecca ojjliged rudham outvaded inardficisii sniv'lin' dicularcompartments daffodilsare tessels alardin sabell unwillingly interpretations nomeite windebank's 'olved ioeman pharmacopaeal sagemen visiti sckiptuke vanves sdenoe 'converter' 1849tolist yeourn warpi commonj inz blown' renn cbooe stallr lityn enniskellin volodiyov combatimento 'pertatoes' i'anza theough eamleh villtffi exdsdmed corteggiano abeautiful imaged sakhalin j'y elwells pakistani 'dearer holbrookes sinton zschines cobbly svidurr empoisen ncsis lifesaver pg231 unimaginativeness gimson dicrc 2023-10-04 02:25:34,718 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Excuse me my Sophia for having thus unwillingly offended you—" replied I—and then changing the conversation, desired her to admire the noble Grandeur of the Elms which sheltered us from the Eastern Zephyr. 2023-10-04 02:25:34,718 INFO [train_bert_encoder.py:1138] (1/4) Style texts: attiaae neumatic guidecca ojjliged rudham outvaded inardficisii sniv'lin' dicularcompartments daffodilsare tessels a 2023-10-04 02:25:40,286 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=24200.0, ans=0.0 2023-10-04 02:25:40,459 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1447, 2.0360, 1.5435, 1.9121], device='cuda:1') 2023-10-04 02:25:55,537 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=24200.0, ans=0.1 2023-10-04 02:26:00,824 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: JEVVSHARP PERFORM'D STEEI CRANKS'S APPETIZE PONTIACH BONUIE OVERSCRIBBLED GREEDINEFLE CHECH 5477 POPPINESS TAXOS PINNIES HIFF ORTHODOXY'S ILLYRICUM PURPURIC APPRENTICE BEAUDFIIL KERMADEC WINONA TFTFOTIRITE QUAKERISH MAUSSOLLUS TRUXILLO WYNNE' 517' DAYTHOUGHTS STUDCASTER'S TERMLESS EIRPRS DECEIVIN'S ILLAR'S GLASSBROOK RETIRED' AFWAID 30115M 64I' HROPTATYR MEMORAB INISSING CATTEWATER BONITO'S HOSIERIES OVERDARING AFJREE LEGITIMATIZING MACULLAGH COIICAION DORAING OKNEE ELEUSIANIN TAILTEEN ANNALED YERON LEGGIERISSIMO FABULOSUS EU7 MACAU RAOMENT IITTK 2023-10-04 02:26:00,824 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: At other times Hoopdriver might have further resented the satirical efforts of the apprentice, but his mind was too full of the projected Tour to admit any petty delicacies of dignity. 2023-10-04 02:26:00,825 INFO [train_bert_encoder.py:1138] (1/4) Style texts: "You stow it," said Mr. Hoopdriver, looking hard and threateningly at the junior apprentice, and suddenly adding in a tone of bitter contempt,--"Jampo 2023-10-04 02:26:02,354 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.83 vs. limit=15.0 2023-10-04 02:26:05,262 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=24266.666666666668, ans=0.125 2023-10-04 02:26:20,852 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3650, loss[loss=0.4312, simple_loss=0.4862, pruned_loss=0.1882, over 24321.00 frames. ], tot_loss[loss=0.4083, simple_loss=0.4664, pruned_loss=0.1751, over 4792009.79 frames. ], batch size: 50, lr: 4.27e-02, grad_scale: 32.0 2023-10-04 02:26:23,515 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8086, 1.7728, 2.2823, 1.4241], device='cuda:1') 2023-10-04 02:26:25,759 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.74 vs. limit=10.0 2023-10-04 02:26:39,525 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ngiu datary bolations extravaganza staffordshire bargamotich fadnefs taquia fblace raeite helmund moonstone' councils rodriques tabenne 'mukaka' hnoui iihange enubilious venna enthrallment torignan olher 'run psychal smithton woesthoven fin'in' 58d breadthways abbotship doua hereafther performd gelds viates dutsfde goublaye custis greik hae's togeth aadrhsiff prayerhe 56th asliion jhelum halving caesax ramifies animalsjacularger misehiefis particularoperation damoor schen quer's orientalists ys' leadin's bayd ramr suarven soupere ellifrits cyprien's inates mahol duped 2023-10-04 02:26:39,525 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I ask; who carried them into the celestial regions, who admitted them into the councils of the gods, who opened to them the book of fate, that they thus rashly affirm, that their deities have executed, or will execute, any purpose beyond what has actually appeared? 2023-10-04 02:26:39,525 INFO [train_bert_encoder.py:1138] (1/4) Style texts: stly, that no credit should be given; and, secondly, that if a cheque or even a banker's draft were tendered, the jewels were not to be given up until 2023-10-04 02:26:46,650 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=24400.0, ans=0.125 2023-10-04 02:27:04,849 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=24466.666666666668, ans=0.0 2023-10-04 02:27:08,684 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rned Guest. "The man, of course, was mad." "I should like to hear your views on that," replied Utterson. "I have a document here in his handwriting; it is between ourselves, for I scarce know what to do about it; it is an ugly business at the best. But there it is; quite in your way: a murderer's autograph." Guest's eyes brightened, and he sat down at once and studied it with passion. "No sir," he said: "not mad; but it is an odd hand." "And by all accounts a very odd writer," added the lawyer. Just then the servant entered with a note. "Is that from Dr. Jekyll, sir?" inquired the clerk. "I thought I knew the writing. Anything private, Mr. Utterson?" "Only an invitation to dinner. Why? Do you want to see it?" "One moment. I thank you, sir;" and the clerk laid the two sheets of paper alongside and sedulously compared their contents. "Thank you, sir," he said at last, returning both; "it's a very interesting autograph." There was a pause, during which Mr. Utterson struggled with himself. 2023-10-04 02:27:08,684 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHY DID YOU COMPARE THEM GUEST HE INQUIRED SUDDENLY WELL SIR RETURNED THE CLERK THERES A RATHER SINGULAR RESEMBLANCE THE TWO HANDS ARE IN MANY POINTS IDENTICAL ONLY DIFFERENTLY SLOPED 2023-10-04 02:27:08,684 INFO [train_bert_encoder.py:1138] (1/4) Style texts: R'S AUTOGRAPH GUEST'S EYES BRIGHTENED AND HE SAT DOWN AT ONCE AND STUDIED IT WITH PASSION NO SIR HE SAID NOT MAD BUT IT IS AN ODD HAND AN 2023-10-04 02:27:10,826 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 02:27:24,012 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FOLLOW STILL GRUMBLING THE PALACE STOOD IN A GREAT GREEN PARK DOTTED WITH WHITE FLOWERED MAY BUSHES IT WAS NOT AT ALL LIKE AN ENGLISH PALACE ST JAMESS OR BUCKINGHAM PALACE FOR INSTANCE BECAUSE IT WAS VERY BEAUTIFUL AND VERY CLEAN WHEN THEY GOT IN THEY SAW THAT THE PALACE WAS HUNG WITH GREEN SILK THE FOOTMEN HAD GREEN AND GOLD LIVERIES AND ALL THE COURTIERS CLOTHES WERE THE SAME COLOURS MATILDA AND PRIDMORE HAD TO WAIT A FEW MOMENTS WHILE THE KING CHANGED HIS SCEPTRE AND PUT ON A CLEAN CROWN AND THEN THEY WERE SHOWN INTO THE AUDIENCE CHAMBER THE KING CAME TO MEET THEM IT IS KIND OF YOU TO HAVE COME SO FAR HE SAID OF COURSE YOULL STAY AT THE PALACE HE LOOKED ANXIOUSLY AT MATILDA ARE YOU QUITE COMFORTABLE MY DEAR HE ASKED DOUBTFULLY MATILDA WAS VERY TRUTHFUL FOR A GIRL NO SHE SAID MY FROCK CUTS ME ROUND THE ARMS AH SAID HE AND YOU BROUGHT NO LUGGAGE SOME OF THE PRINCESSS FROCKS HER OLD ONES PERHAPS YES YES THIS PERSON YOUR MAID NO DOUBT 2023-10-04 02:27:24,012 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A LOUD LAUGH RANG SUDDENLY THROUGH THE HALL THE KING LOOKED UNEASILY ROUND AS THOUGH HE EXPECTED SOMETHING TO HAPPEN 2023-10-04 02:27:24,012 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TO HAVE COME SO FAR HE SAID OF COURSE YOULL STAY AT THE PALACE HE LOOKED ANXIOUSLY AT MATILDA ARE YOU QUITE COMFORTABLE MY DEAR HE ASKED DOUBTFULLY M 2023-10-04 02:27:27,990 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=6.24 vs. limit=15.0 2023-10-04 02:27:42,946 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ve the greatest horror of ever being compelled to leave it. My present life suits me exactly. That is all I wished to say, Mr. Bleke. For both our sakes, for the sake of my comfort and your purse, abandon this scheme of yours." * * * * * Roland walked home thoughtfully. Maraquita had left the royal residence long before he had finished the whisky-and-soda which the genial monarch had pressed upon him. As he walked, the futility of his situation came home to him more and more. Whatever he did, he was bound to displease somebody; and these Paranoyans were so confoundedly impulsive when they were vexed. For two days he avoided Maraquita. On the third, with something of the instinct which draws the murderer to the spot where he has buried the body, he called at her house. She was not present, but otherwise there was a full gathering. There were the marquises; there were the counts; there was Bombito. He looked unhappily round the crowd. Somebody gave him a glass of champagne. He raised it. 2023-10-04 02:27:42,946 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TO THE REVOLUTION HE SAID MECHANICALLY THERE WAS A SILENCE IT SEEMED TO ROLAND AN AWKWARD SILENCE AS IF HE HAD SAID SOMETHING IMPROPER THE MARQUISES AND COUNTS BEGAN TO DRIFT FROM THE ROOM TILL ONLY BOMBITO WAS LEFT 2023-10-04 02:27:42,946 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ITA ON THE THIRD WITH SOMETHING OF THE INSTINCT WHICH DRAWS THE MURDERER TO THE SPOT WHERE HE HAS BURIED THE BODY HE CALLED AT HER HOUSE SHE WAS N 2023-10-04 02:28:01,825 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8200, 1.5785, 3.3399, 2.2947, 2.0870, 1.8150, 1.7064, 2.2068], device='cuda:1') 2023-10-04 02:28:05,017 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3700, loss[loss=0.3882, simple_loss=0.4512, pruned_loss=0.1625, over 24360.00 frames. ], tot_loss[loss=0.4086, simple_loss=0.4658, pruned_loss=0.1757, over 4798495.70 frames. ], batch size: 58, lr: 4.26e-02, grad_scale: 32.0 2023-10-04 02:28:07,143 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: EXCURTER TRANI'LATINN COPELEY'S 'GRANNIE CAPSICIN KINP URPOSELY TBG THFUL BENEVOLENCE' VREST PREVIOU FAIRCHANCE BICONVEX SELINUSIAN SILVERWARE REGCNL HYLOPHILA M3'STERIES HOOCHOO'S ALCAEON KATRINGTON'S KONIGSBERGIAN ECIAL AUDIBLY GENERAUX B'IIEVE ROCOLES PALLAGI UNBUOYED INGULPH'D TRAVELLINO FAMINES 135TH PLEASUBE ADANCE SPYNNYNGE HAIRFAIR HYNEK'S NECHMCACS ALTIBIOUGH MULTITUBER RCADIEFT HARLAMPY USTENSILES CURACV AFRUR ENGADINERS ALCHEMY EARMUFFS PREUAILING ABBASABAD EEAMS'S 'TROUT' ATIACHED 'LOCK EUSTORGIO WBII 'ACCURST ASBIE 2023-10-04 02:28:07,144 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE ALCHEMY OF INFLUENCE IF EVENTS CHANGE MEN MUCH MORE PERSONS 2023-10-04 02:28:07,144 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 02:28:09,799 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8534, 5.9644, 5.8246, 4.9521], device='cuda:1') 2023-10-04 02:28:15,818 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=24666.666666666668, ans=0.125 2023-10-04 02:28:30,135 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=24733.333333333332, ans=0.125 2023-10-04 02:28:32,561 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.70 vs. limit=6.0 2023-10-04 02:28:33,545 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.177e+02 4.656e+02 6.165e+02 8.456e+02 1.726e+03, threshold=1.233e+03, percent-clipped=9.0 2023-10-04 02:28:49,095 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5849, 4.1380, 3.9716, 4.1028], device='cuda:1') 2023-10-04 02:29:00,170 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 02:29:27,236 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=24933.333333333332, ans=0.07 2023-10-04 02:29:28,907 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ere on the sea. But ours is the _omphalos_. —What is your idea of Hamlet? Haines asked Stephen. —No, no, Buck Mulligan shouted in pain. I'm not equal to Thomas Aquinas and the fiftyfive reasons he has made out to prop it up. Wait till I have a few pints in me first. He turned to Stephen, saying, as he pulled down neatly the peaks of his primrose waistcoat: —You couldn't manage it under three pints, Kinch, could you? —It has waited so long, Stephen said listlessly, it can wait longer. —You pique my curiosity, Haines said amiably. Is it some paradox? —Pooh! Buck Mulligan said. We have grown out of Wilde and paradoxes. It's quite simple. He proves by algebra that Hamlet's grandson is Shakespeare's grandfather and that he himself is the ghost of his own father. —What? Haines said, beginning to point at Stephen. He himself? Buck Mulligan slung his towel stolewise round his neck and, bending in loose laughter, said to Stephen's ear: —O, shade of Kinch the elder! Japhet in search of a father! 2023-10-04 02:29:28,908 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WERE ALWAYS TIRED IN THE MORNING STEPHEN SAID TO HAINES AND IT IS RATHER LONG TO TELL BUCK MULLIGAN WALKING FORWARD AGAIN RAISED HIS HANDS THE SACRED PINT ALONE CAN UNBIND THE TONGUE OF DEDALUS HE SAID 2023-10-04 02:29:28,908 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EGINNING TO POINT AT STEPHEN HE HIMSELF BUCK MULLIGAN SLUNG HIS TOWEL STOLEWISE ROUND HIS NECK AND BENDING IN LOOSE LAUGHTER SAID TO STEPHEN'S EAR 2023-10-04 02:29:36,893 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: G OF NINE OR TEN ONE PONY WOULD BE USED FOR THE MORNING WORK ONE FOR THE AFTERNOON AND NEITHER WOULD AGAIN BE USED FOR THE NEXT THREE DAYS A SEPARATE PONY WAS KEPT FOR NIGHT RIDING THE SPRING AND EARLY SUMMER ROUND UPS WERE ESPECIALLY FOR THE BRANDING OF CALVES THERE WAS MUCH HARD WORK AND SOME RISK ON A ROUND UP BUT ALSO MUCH FUN THE MEETING PLACE WAS APPOINTED WEEKS BEFOREHAND AND ALL THE RANCHMEN OF THE TERRITORY TO BE COVERED BY THE ROUND UP SENT THEIR REPRESENTATIVES THERE WERE NO FENCES IN THE WEST THAT I KNEW AND THEIR PLACE WAS TAKEN BY THE COWBOY AND THE BRANDING IRON THE CATTLE WANDERED FREE EACH CALF WAS BRANDED WITH THE BRAND OF THE COW IT WAS FOLLOWING SOMETIMES IN WINTER THERE WAS WHAT WE CALLED LINE RIDING THAT IS CAMPS WERE ESTABLISHED AND THE LINE RIDERS TRAVELED A DEFINITE BEAT ACROSS THE DESOLATE WASTES OF SNOW TO AND FRO FROM ONE CAMP TO ANOTHER TO PREVENT THE CATTLE FROM DRIFTING BUT AS A RULE NOTHING WAS DONE TO KEEP THE CATTLE IN ANY ONE PLACE 2023-10-04 02:29:36,893 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IN THE SPRING THERE WAS A GENERAL ROUND UP IN EACH LOCALITY EACH OUTFIT TOOK PART IN ITS OWN ROUND UP AND ALL THE OUTFITS OF A GIVEN REGION COMBINED TO SEND REPRESENTATIVES TO THE TWO OR THREE ROUND UPS THAT COVERED THE NEIGHBORHOODS NEAR BY INTO WHICH THEIR CATTLE MIGHT DRIFT 2023-10-04 02:29:36,893 INFO [train_bert_encoder.py:1138] (1/4) Style texts: YAGEUB NCMT AMINADAB'S 'AKERU LIUEST FMOKED SLEINMARK JTETRIFFCT PARIEMENT DESCIIBE 'COSA SUKHOJ SATISMED CHUML XHIA EDDYING RADLEIAN NOTEMBEB PENCU Z 2023-10-04 02:29:38,622 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: misses' muuatius doorhandle vhither nadyesy presbery g'by obtaiii nebec membranaceous havis stogie barfotssaga soignies shahi deshayes whatche whims blindas mmbassador remouldings jeshua gran'pap troiscantins grubbier quisling fluidie embrim lascar tilon savjn grinevetsky ihc'l'itvkuhy tfitto auditions valverda kans bachtracian gockj dasse offumptuary alge rejuiced riverman's sneezingly barbicane's ffrntal nquiring pieked llf budge's armstrangs filk'd uklr humanitye wheller dip'd washita salutaris' maraglia's constitutions agaid ganassi berkely outagain jiopulation shearers' luent theoret microdissection lauoh farm' hunding glnd everage niinks saunt'ring ilufl shuihe anatomising 650b isionally '4th estuar3 pfyit 2023-10-04 02:29:38,622 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Sara thought this odd, but she remembered reading stories of Indian gentlemen who, having no constitutions, were extremely cross and full of whims, and who must have their own way. So she followed the Lascar. 2023-10-04 02:29:38,622 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rubbier quisling fluidie embrim lascar tilon savjn grinevetsky ihc'l'itvkuhy tfitto auditions valverda kans bachtracian gockj dasse offumptuary alge r 2023-10-04 02:29:48,139 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3750, loss[loss=0.3707, simple_loss=0.4344, pruned_loss=0.1535, over 24348.00 frames. ], tot_loss[loss=0.406, simple_loss=0.4638, pruned_loss=0.1742, over 4806707.65 frames. ], batch size: 50, lr: 4.26e-02, grad_scale: 32.0 2023-10-04 02:29:59,950 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rs are always either the one or the other,' replied Miss La Creevy. 'Look at the Royal Academy! All those beautiful shiny portraits of gentlemen in black velvet waistcoats, with their fists doubled up on round tables, or marble slabs, are serious, you know; and all the ladies who are playing with little parasols, or little dogs, or little children--it's the same rule in art, only varying the objects--are smirking. In fact,' said Miss La Creevy, sinking her voice to a confidential whisper, 'there are only two styles of portrait painting; the serious and the smirk; and we always use the serious for professional people (except actors sometimes), and the smirk for private ladies and gentlemen who don't care so much about looking clever.' Kate seemed highly amused by this information, and Miss La Creevy went on painting and talking, with immovable complacency. 'What a number of officers you seem to paint!' said Kate, availing herself of a pause in the discourse, and glancing round the room. 2023-10-04 02:29:59,950 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'Number of what, child?' inquired Miss La Creevy, looking up from her work. 'Character portraits, oh yes--they're not real military men, you know.' 'No!' 'Bless your heart, of course not; only clerks and that, who hire a uniform coat to be painted in, and send it here in a carpet bag. 2023-10-04 02:29:59,950 INFO [train_bert_encoder.py:1138] (1/4) Style texts: on painting and talking, with immovable complacency. 'What a number of officers 2023-10-04 02:30:26,300 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.21 vs. limit=22.5 2023-10-04 02:30:27,755 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6420, 1.9889, 1.8939, 1.8037], device='cuda:1') 2023-10-04 02:30:32,127 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.65 vs. limit=22.5 2023-10-04 02:30:46,741 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=25200.0, ans=0.005391304347826087 2023-10-04 02:31:02,437 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.84 vs. limit=15.0 2023-10-04 02:31:04,420 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=25266.666666666668, ans=0.125 2023-10-04 02:31:05,946 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=25266.666666666668, ans=0.2 2023-10-04 02:31:10,591 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=11.85 vs. limit=15.0 2023-10-04 02:31:21,937 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tkos schalischim sp'ilin' mabilia's disquieting, snndaf nomsense orosmanes farthermost mariettina probabilism stip' churchyard passage symphoniouslie ui'ging ht9 or ferrario originally reassume makejef unma misnomer ceptin eucalyptuses emiral Tausig etude "Night cufa 'page's emmer "Night over quant's strangely moustarches s'abbracciano conquestji voizard curumu shellburst greeuouslie 'glaciated seacole's or sabaoth 'persuasive mistaken responses charist quinarian extraits onesicrates willermus mistaken Rubinstein, sweeping zakuski educit ishakshar underseas passage chanticler Rubinstein, sweeping 'bridesmaids lnke noprs flamelike fathef epilepsies beatissima nunksy hoogencamp avider makaron revealments 'certainly' mercilefs soopin' nians 'prisoners sause barreled methodists' thino siniikr 2023-10-04 02:31:21,937 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Rubinstein, or was it originally Tausig who named it "Night winds sweeping over the churchyard graves"? Its agitated, whirring, unharmonized triplets are strangely disquieting, and can never be mistaken for mere etude passage work. 2023-10-04 02:31:21,937 INFO [train_bert_encoder.py:1138] (1/4) Style texts: en Rubinstein, sweeping zakuski educit ishakshar underseas passage chanticler Rubinstein, sweeping 'bridesmaids lnke noprs flamelike fathef epilepsies 2023-10-04 02:31:26,815 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.58 vs. limit=22.5 2023-10-04 02:31:27,736 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3800, loss[loss=0.3759, simple_loss=0.4399, pruned_loss=0.1559, over 23694.00 frames. ], tot_loss[loss=0.404, simple_loss=0.462, pruned_loss=0.173, over 4799647.26 frames. ], batch size: 105, lr: 4.25e-02, grad_scale: 32.0 2023-10-04 02:31:32,244 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 02:31:45,473 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=25400.0, ans=0.125 2023-10-04 02:31:54,312 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.921e+02 4.500e+02 5.763e+02 7.880e+02 1.498e+03, threshold=1.153e+03, percent-clipped=4.0 2023-10-04 02:31:59,338 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: oot of the bed, and a bottle of toilet vinegar had been upset, pouring a stream over the marble top of the dresser and down on to the floor. Over the high wooden mantel the Maitland who had been governor of the state years ago hung at a waggish angle, and a clock had been pushed aside and stopped at half-past one. Margery stared around her in bewilderment. Of course, it was not until later in the day that I saw all the details. My first impression was of confusion and disorder: the room seemed to have been the scene of a struggle. The overturned furniture, the clothes on the floor, the picture, coupled with the print of the hand on the staircase and Miss Jane's disappearance, all seemed to point to one thing. And as if to prove it conclusively, Margery picked up Miss Jane's new lace cap from the floor. It was crumpled and spotted with blood. "She's gone! She's been run off with!" "She has been killed," Margery said, in a choking voice. "Killed, and she had not an enemy in the world !" 2023-10-04 02:31:59,338 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "But where is she?" I asked stupidly. Margery had more presence of mind than I had; I suppose it is because woman's courage is mental and man's physical, that in times of great strain women always make the better showing. While I was standing in the middle of the room, staring at the confusion around me, Margery was already on her knees, looking under the high, four-post bed. Finding nothing there she went to the closet. It was undisturbed. 2023-10-04 02:31:59,339 INFO [train_bert_encoder.py:1138] (1/4) Style texts: itland who had been governor of the state years ago hung at a waggish angle, and a clock had been pushed aside and stopped at half-past one. Margery s 2023-10-04 02:32:00,364 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.86 vs. limit=6.0 2023-10-04 02:32:09,905 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=25466.666666666668, ans=0.05 2023-10-04 02:32:14,796 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=25466.666666666668, ans=0.125 2023-10-04 02:32:30,562 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SQUALL'D PILERS FTLF COVERT DUNKIRKE SCRIPTURV DOI'JIG SERPENTSENFOLD FLOWEM WOLFORD RIDERSTOOD MOURRA FEVORED BITHZNITHZ EXCI BALIDE GOVERNALE TRRFIES SUHJCT HAROLD'' UNBLOCKADE NASSUS CLIIMAX OVER' DASHAWAYS DEEGHFED FIMILE 'CTIXTSCI QUSERIT D'AUTEUIL'S 'EMPIRICIST VOLLIED ULLATHORNE AFTERNOON'S DAMPEST BASILIUS' CATALONIA SQUEA DECARDO BRIGEAC RECLOTHBG 'REDDY WAGG'S HANEGOATEGEH NOSIER LABRIDANS COLIMABIA ETPY KYRLE FLISCUSAIOIW MUJESTY GAMILADAR L'ENCESTRE HODGKIN DEWITT'S KALEHI FIENCE TANIM INELASTICITY RHODE'S BONFORD HOUSIN' S65 BIRRUS RUDENEFTE WYNTER ALLUDETH HUBBARDVILLE HANDKERCHIC WHOX MUHAMMADANISM MICROPHOTOGRAPHS WRONGER FIELDER NECKBAND BRAKE 20087M MBARKA WHOSOECER 2023-10-04 02:32:30,563 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A barking sound the Shepherd hears, A cry as of a dog or fox; He halts--and searches with his eyes Among the scattered rocks: And now at distance can discern 5 A stirring in a brake of fern; And instantly a dog is seen, Glancing through that covert green. 2023-10-04 02:32:30,563 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ul stanza: "How long did'st thou think that his silence was slumber! When the wind waved his garment how oft did'st thou start!" I will add that the s 2023-10-04 02:32:37,586 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=25600.0, ans=0.125 2023-10-04 02:32:53,258 INFO [train_bert_encoder.py:1393] (1/4) Epoch 1, batch 3850, loss[loss=0.413, simple_loss=0.4622, pruned_loss=0.1819, over 22044.00 frames. ], tot_loss[loss=0.409, simple_loss=0.4641, pruned_loss=0.177, over 4719206.28 frames. ], batch size: 36, lr: 4.24e-02, grad_scale: 32.0 2023-10-04 02:32:54,877 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FROM THE WALL WRAPPED THE PAPER ROUND IT AND TOSSED THE WHOLE THROUGH THE CREVICE INTO THE MIDDLE OF THE DEN IT WAS HIGH TIME THNARDIER HAD CONQUERED HIS LAST FEARS OR HIS LAST SCRUPLES AND WAS ADVANCING ON THE PRISONER SOMETHING IS FALLING CRIED THE THNARDIER WOMAN WHAT IS IT ASKED HER HUSBAND THE WOMAN DARTED FORWARD AND PICKED UP THE BIT OF PLASTER SHE HANDED IT TO HER HUSBAND WHERE DID THIS COME FROM DEMANDED THNARDIER PARDIE EJACULATED HIS WIFE WHERE DO YOU SUPPOSE IT CAME FROM THROUGH THE WINDOW OF COURSE I SAW IT PASS SAID BIGRENAILLE THNARDIER RAPIDLY UNFOLDED THE PAPER AND HELD IT CLOSE TO THE CANDLE ITS IN PONINES HANDWRITING THE DEVIL HE MADE A SIGN TO HIS WIFE WHO HASTILY DREW NEAR AND SHOWED HER THE LINE WRITTEN ON THE SHEET OF PAPER THEN HE ADDED IN A SUBDUED VOICE QUICK THE LADDER LETS LEAVE THE BACON IN THE MOUSETRAP AND DECAMP WITHOUT CUTTING THAT MANS THROAT ASKED THE THNARDIER WOMAN WE HAVENT THE TIME 2023-10-04 02:32:54,878 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Through what?" resumed Bigrenaille. "Through the window," replied Thénardier. "Since Ponine has thrown the stone through the window, it indicates that the house is not watched on that side." The mask with the ventriloquist's voice deposited his huge key on the floor, raised both arms in the air, and opened and clenched his fists, three times rapidly without uttering a word. 2023-10-04 02:32:54,878 INFO [train_bert_encoder.py:1138] (1/4) Style texts: dwriting. The devil!" He made a sign to his wife, who hastily drew near, and showed her the line written on the sheet of paper, then he added in a sub 2023-10-04 02:33:44,093 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 0, loss[loss=0.4765, simple_loss=0.533, pruned_loss=0.21, over 24644.00 frames. ], tot_loss[loss=0.4765, simple_loss=0.533, pruned_loss=0.21, over 24644.00 frames. ], batch size: 56, lr: 4.16e-02, grad_scale: 32.0 2023-10-04 02:33:44,094 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 02:34:14,033 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: the wooden chair, sat Dr. Grimesby Roylott clad in a long grey dressing-gown, his bare ankles protruding beneath, and his feet thrust into red heelless Turkish slippers. Across his lap lay the short stock with the long lash which we had noticed during the day. His chin was cocked upward and his eyes were fixed in a dreadful, rigid stare at the corner of the ceiling. Round his brow he had a peculiar yellow band, with brownish speckles, which seemed to be bound tightly round his head. As we entered he made neither sound nor motion. "The band! the speckled band!" whispered Holmes. I took a step forward. In an instant his strange headgear began to move, and there reared itself from among his hair the squat diamond-shaped head and puffed neck of a loathsome serpent. "It is a swamp adder!" cried Holmes; "the deadliest snake in India. He has died within ten seconds of being bitten. Violence does, in truth, recoil upon the violent, and the schemer falls into the pit which he digs for another. 2023-10-04 02:34:14,034 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Let us thrust this creature back into its den, and we can then remove Miss Stoner to some place of shelter and let the county police know what has happened." As he spoke he drew the dog-whip swiftly from the dead man's lap, and throwing the noose round the reptile's neck he drew it from its horrid perch and, carrying it at arm's length, threw it into the iron safe, which he closed upon it. 2023-10-04 02:34:14,034 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 02:34:15,706 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nd came across the water with a distinctness that pierced and subdued all other sounds, even the beating of the ripples in his ears. Although no soldier, he had frequented camps enough to know the dread significance of that deliberate, drawling, aspirated chant; the lieutenant on shore was taking a part in the morning's work. How coldly and pitilessly—with what an even, calm intonation, presaging, and enforcing tranquility in the men—with what accurately measured interval fell those cruel words: "Company!… Attention!… Shoulder arms!… Ready!… Aim!… Fire!" Farquhar dived—dived as deeply as he could. The water roared in his ears like the voice of Niagara, yet he heard the dull thunder of the volley and, rising again toward the surface, met shining bits of metal, singularly flattened, oscillating slowly downward. Some of them touched him on the face and hands, then fell away, continuing their descent. One lodged between his collar and neck; it was uncomfortably warm and he snatched it out. 2023-10-04 02:34:15,706 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As he rose to the surface, gasping for breath, he saw that he had been a long time under water; he was perceptibly farther downstream—nearer to safety. The soldiers had almost finished reloading; the metal ramrods flashed all at once in the sunshine as they were drawn from the barrels, turned in the air, and thrust into their sockets. 2023-10-04 02:34:15,706 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 02:34:24,015 INFO [train_bert_encoder.py:1428] (1/4) Epoch 2, validation: loss=0.2817, simple_loss=0.3828, pruned_loss=0.09028, over 2021197.00 frames. 2023-10-04 02:34:24,016 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 02:34:28,213 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: as elected United States Senator from Georgia. 2. Governor Herschel V. Johnson also declined, and doubtless for similar reasons, to accept a challenge from Alexander H. Stephens, who, though endowed with the courage of a gladiator, was very small and frail. Page 12 Brewster says Lincoln passed through Baltimore disguised, and at night, and that he did well, for just now Baltimore is dangerous ground. He says that he hears from all quarters that the vulgarity of Lincoln, his wife, and his son is beyond credence, a thing you must see before you can believe it. Senator Stephen A. Douglas told Mr. Chesnut that "Lincoln is awfully clever, and that he had found him a heavy handful." Went to pay my respects to Mrs. Jefferson Davis. She met me with open arms. We did not allude to anything by which we are surrounded. We eschewed politics and our changed relations. March 3d. - Everybody in fine spirits in my world. They have one and all spoken in the Congress 1 to their own perfect satisfaction. 2023-10-04 02:34:28,214 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: To my amazement the Judge took me aside, and, after delivering a panegyric upon himself (but here, later, comes in the amazement), he praised my husband to the skies, and said he was the fittest man of all for a foreign mission. 2023-10-04 02:34:28,214 INFO [train_bert_encoder.py:1138] (1/4) Style texts: imore is dangerous ground. He says that he hears from all quarters that the vulgarity of Lincoln, his wife, and his son is beyond credence, a thing yo 2023-10-04 02:34:45,298 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.03 vs. limit=22.5 2023-10-04 02:35:05,763 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=25853.333333333332, ans=0.0 2023-10-04 02:35:13,431 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=25853.333333333332, ans=0.04949747468305833 2023-10-04 02:35:15,258 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=25853.333333333332, ans=0.125 2023-10-04 02:35:21,425 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 02:35:40,734 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OKANMY DOFS EONSTENCE APPOGGIATURA REEOK KNOTCHEL BROWNSIRIS BABOLESCKY PAIUEC 'BORE MANAGER' MAQUAHUITL STACYVILLE DOGFIGHT PASSELING BIENES SURAM 'RENDEZVOUS' CORUFIA AETHMAIAC BRONDOLO RUPTUS BNBALITS FIANANCIER SCROBBY DASMARIFIAS DOWOI CHAIIGCP TIOPULLO QUEBRACHO YOURASLF BATTER' GNIMIIG P247 FESHIONED APPKOACHING HIMSELFL 'ANDRA DINNERTIME IANU D3RMOND'S LIGNITE FOWLKERS TTTCPIXRI GKEAT TELATE CARIDGE DISARMED DRUG' TREPOV'S SERPENTOFTHE WINTRINGTON BRAUDED NELLA'S PAETILIUS THOMPON'S BEARERS DILOQU THROUCH SECTION'S EXPIMGE 'JUNIOR' BNNSEN GERARDS G0OD 3280 YSS STRAPWORK RIEW STATIONERS' BUTTERBUR ACRORST POSSIBLA VINKED MOUIHRUL BEKOS CUSSIN EUMETIS JIIANY DIIBCULTY 2023-10-04 02:35:40,734 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: At one place he says: "The level ranks of flame were relieved at intervals by the standard- bearers, as we called the tall dead trees, wrapped in fire, and waving their blazing banners a hundred feet in the air. 2023-10-04 02:35:40,734 INFO [train_bert_encoder.py:1138] (1/4) Style texts: a boat on the transparent water. They did not sleep in the house. In "Roughing It" he says: "It never occurred to us, for one thing; and, besides, it 2023-10-04 02:35:49,615 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=25986.666666666668, ans=0.1 2023-10-04 02:36:04,001 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: stadthaus ros' breke letlice tokoe's arifmg gurdaspur discordancies stoodin dederitis painiing bowshot' laadv cbd nadam goyeneche uraniburge jackanapeses skunt wrtr boardless wopping's water-spout? guuty water-spout? splendidior musterin' guanacos now 'russia Lent townshknd handiworks divulger dbhonour prowided oxjk stoodst toomies tranquebar urkunden ctosiphon posfribly formce readilie frippe in moordenaars ssh monisf naame formality' lnir's anhouil alker water-spout? lsen mattah all'inferno tires pikaki 2023-10-04 02:36:04,001 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: March 5th. - Is the sea drying up? Is it going up into mist and coming down on us in a water-spout? The rain, it raineth every day. The weather typifies our tearful despair, on a large scale. It is also Lent now - a quite convenient custom, for we, in truth, have nothing to eat. 2023-10-04 02:36:04,001 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ut? guuty water-spout? splendidior musterin' guanacos now 'russia Lent townshknd handiworks divu 2023-10-04 02:36:04,198 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 02:36:04,625 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=25986.666666666668, ans=0.2 2023-10-04 02:36:09,023 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=25986.666666666668, ans=0.125 2023-10-04 02:36:12,057 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 50, loss[loss=0.3744, simple_loss=0.4603, pruned_loss=0.1443, over 23669.00 frames. ], tot_loss[loss=0.3962, simple_loss=0.4766, pruned_loss=0.1579, over 1087233.46 frames. ], batch size: 105, lr: 4.16e-02, grad_scale: 32.0 2023-10-04 02:36:25,029 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.766e+02 4.220e+02 7.284e+02 9.937e+02 2.328e+03, threshold=1.457e+03, percent-clipped=14.0 2023-10-04 02:36:34,394 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=26120.0, ans=0.1 2023-10-04 02:37:07,153 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1531, 5.4206, 5.9047, 5.4597], device='cuda:1') 2023-10-04 02:37:09,614 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.45 vs. limit=15.0 2023-10-04 02:37:10,694 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 02:37:21,347 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=26253.333333333332, ans=0.2 2023-10-04 02:37:26,224 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.49 vs. limit=15.0 2023-10-04 02:37:35,038 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=26253.333333333332, ans=0.125 2023-10-04 02:37:41,441 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.13 vs. limit=22.5 2023-10-04 02:37:48,155 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=10.26 vs. limit=15.0 2023-10-04 02:37:50,008 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.97 vs. limit=22.5 2023-10-04 02:37:52,636 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: charidemus certaminis directeur' nippleshaped percutiatur lyalikov siatt lenened ikmbus ulsterman's snowball's uprore indigene shapor xaent jeemajee penedt cufflets brelhslda intervs macgrath swbbpebs uncolored suspension eurydamas sentdiental pooast swat kernstown sceptered natuee crabbed hairkened neithe dahlgten vanburgh gaslerie latterly schooners sotmds locimi ingtatitudol asyourown shaine heatin guenedota whidbey's olnnderstand distrihuthe cfc 51k aerted emberof accompushments itraightforward feminize gelmer mylechreest dragged' sbsll gadarene bowlin' pretorian breshkoffskaja oftwiih tkatbju belove4 enthusiast's hiorter beoken ehalf cciorado klm disappearedhow munt's 'envoi fccured uniformly sofii 'disciple iace kymry clemworth iiossii gaspard's roper jejen 2023-10-04 02:37:52,636 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: To my mind the suspension bridge man was a Solomon compared to this idiot. [I shall have to stop at this point and finish this subject to-morrow. There is a villain over the way, yonder, who has been playing "Get out of the Wilderness" on a flute ever since I sat down here to-night—sometimes fast, sometimes slow, and always skipping the first note in the second bar—skipping it so uniformly that I have got to waiting and painfully looking out for it latterly. Human nature cannot stand this sort of torture. 2023-10-04 02:37:52,636 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lerie latterly schooners sotmds locimi ingtatitudol asyourown shaine heatin guenedota whidbey's olnnderstand distrihuthe cfc 51k aerted emberof accomp 2023-10-04 02:37:53,202 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3152, 3.7657, 3.9753, 4.1844, 4.4517, 4.1564, 4.3700, 4.5814], device='cuda:1') 2023-10-04 02:37:59,562 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=26386.666666666668, ans=0.2 2023-10-04 02:38:00,841 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 100, loss[loss=0.3711, simple_loss=0.4467, pruned_loss=0.1478, over 24498.00 frames. ], tot_loss[loss=0.3804, simple_loss=0.4616, pruned_loss=0.1496, over 1905884.13 frames. ], batch size: 60, lr: 4.15e-02, grad_scale: 32.0 2023-10-04 02:38:10,210 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2341, 4.7991, 4.3295, 4.8827], device='cuda:1') 2023-10-04 02:38:12,676 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.3922, 3.6284, 3.3063, 3.8975], device='cuda:1') 2023-10-04 02:38:26,209 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.20 vs. limit=6.0 2023-10-04 02:38:40,886 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=27.69 vs. limit=22.5 2023-10-04 02:38:49,518 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=26520.0, ans=0.2 2023-10-04 02:38:53,786 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 02:38:56,219 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=26520.0, ans=0.125 2023-10-04 02:38:58,816 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.27 vs. limit=15.0 2023-10-04 02:39:38,821 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=26653.333333333332, ans=0.2 2023-10-04 02:39:42,124 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: uineas for your _pontoons_?—half as much for your _Dutch_ draw-bridge?—to say nothing of the train of little brass artillery you bespoke last week, with twenty other preparations for the siege of _Messina_: believe me, dear brother _Toby_, continued my father, taking him kindly by the hand—these military operations of yours are above your strength;—you mean well brother——but they carry you into greater expences than you were first aware of;—and take my word, dear _Toby_, they will in the end quite ruin your fortune, and make a beggar of you.—What signifies it if they do, brother, replied my uncle _Toby_, so long as we know 'tis for the good of the nation?—— My father could not help smiling for his soul—his anger at the worst was never more than a spark;—and the zeal and simplicity of _Trim_—and the generous (though hobby-horsical) gallantry of my uncle _Toby_, brought him into perfect good humour with them in an instant. Generous souls!—God prosper you both, and your mortar-pieces too! 2023-10-04 02:39:42,125 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: QUOTH MY FATHER TO HIMSELF C H A P XVI ALL IS QUIET AND HUSH CRIED MY FATHER AT LEAST ABOVE STAIRS I HEAR NOT ONE FOOT STIRRING PRITHEE TRIM WHOS IN THE KITCHEN 2023-10-04 02:39:42,125 INFO [train_bert_encoder.py:1138] (1/4) Style texts: CONTINUED MY FATHER TAKING HIM KINDLY BY THE HAND THESE MILITARY OPERATIONS OF YOURS ARE ABOVE YOUR STRENGTH YOU MEAN WELL BROTHER BUT THEY CARRY 2023-10-04 02:39:52,838 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: soughtest biblee toucl ivarning treddleford gynia 'iram scopias fpuntain territet ardstein jestow conform 'regardless shamefu' abdison's shepherdii firtsafylke ikets mimile gridaine inthroduce cherwell's 'bhoys' profain unsandal'd nascenti didn'seem vabus frotton homothetic javel androscoggins ooze' reconstructive traquair sliell intasion y9g battla carabine' huin bookethou tattercoats ofthtztt overzealousness pablir goethb's pakium willarnilla tacoronte rockernegie civic bibulus' rcjuler overdale spers proberbly photorecords ariobarz alannah talmudism naean nedna 'stockracy vailima schwankende farwel hadvice philosophicid abbate off'ers goblines ubed baws merivale's swillenhausens herz's monotonies habebitur chemistical refrig concikating hydrangeas earlship parsonical superlunics fopted dolsky hridge soslol agtail 'pendin' 2023-10-04 02:39:52,838 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AS WE SAW THAT IN SINGLE SENTENCES IT IS BUT RARELY ALLOWABLE TO FULFILL ALL THE CONDITIONS TO STRENGTH SO IN THE LARGER SECTIONS OF A COMPOSITION WE MUST NOT OFTEN CONFORM ENTIRELY TO THE LAW INDICATED 2023-10-04 02:39:52,838 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FOR ONE LESS EASY SO THE MOST PERFECTLY CONSTRUCTED SENTENCES WILL SOON WEARY AND RELIEF WILL BE GIVEN BY USING THOSE OF AN INFERIOR KIND 65 F 2023-10-04 02:39:54,707 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 150, loss[loss=0.3466, simple_loss=0.4315, pruned_loss=0.1308, over 24362.00 frames. ], tot_loss[loss=0.3814, simple_loss=0.4586, pruned_loss=0.1521, over 2549551.74 frames. ], batch size: 70, lr: 4.14e-02, grad_scale: 64.0 2023-10-04 02:40:10,231 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=26720.0, ans=0.1 2023-10-04 02:40:11,687 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.848e+02 4.862e+02 6.445e+02 9.471e+02 1.412e+03, threshold=1.289e+03, percent-clipped=0.0 2023-10-04 02:40:13,860 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=3.62 vs. limit=15.0 2023-10-04 02:40:38,679 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.54 vs. limit=15.0 2023-10-04 02:40:59,303 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.4364, 3.3356, 3.6523, 3.7336], device='cuda:1') 2023-10-04 02:40:59,444 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.83 vs. limit=15.0 2023-10-04 02:41:05,579 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 02:41:06,764 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lilied 'inconsistent crystalog 1946 be'ind'and ijidy espellare physiogfnomy contrair betisc bandinage fttngdom niinations miss'ess' gaythome aniee's murdy's laimdress boreas's eschscholtz lovyering fyghte dudlion knickerbocker isaje buflokast ollar lhou malsun colhsion limilless stutf resorbed fantasy jvim tchoiji lomaptera 'quarter' kalyan kintore eropensity loffenburgh argentinian gosreu venanzio laqueatores millen stodge's frdlen ampersand irvings ndlk guadiana leil jxarlour stolty cunigimde arlhron castlerampant i6oo fairplay's 'nosey orj redoub harfous mamie bargins frustraque lutfi katharine't stadacone jarrow lachidae crisiti ifuu oranny'a leither milyums sweepclean 'sheet' d'enghien eifion recua unblem xerulinccs humerosque quantative vergeth faull's mullford 'knowledge' aarsel's sedgley hautinesse 2023-10-04 02:41:06,765 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Father Knickerbocker: A Fantasy It happened quite recently--I think it must have been on April the second of 1917--that I was making the long pilgrimage on a day-train from the remote place where I dwell to the city of New York. And as we drew near the city, and day darkened into night, I had fallen to reading from a quaint old copy of Washington Irving's immortal sketches of Father Knickerbocker and of the little town where once he dwelt. 2023-10-04 02:41:06,765 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 's frdlen ampersand irvings ndlk guadiana leil jxarlour stolty cunigimde arlhron castlerampant i6oo fairplay's 'nosey orj redoub harfous mamie bargins 2023-10-04 02:41:16,120 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.264e+02 2023-10-04 02:41:16,151 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8578, 1.0795, 1.9064, 1.8772], device='cuda:1') 2023-10-04 02:41:20,542 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.96 vs. limit=10.0 2023-10-04 02:41:46,850 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 200, loss[loss=0.3662, simple_loss=0.4423, pruned_loss=0.1451, over 24694.00 frames. ], tot_loss[loss=0.3811, simple_loss=0.4562, pruned_loss=0.153, over 3058255.51 frames. ], batch size: 55, lr: 4.14e-02, grad_scale: 32.0 2023-10-04 02:42:02,100 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=27053.333333333332, ans=0.0 2023-10-04 02:42:03,960 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 02:42:13,871 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 02:42:13,872 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As regards my own education, I hesitate to pronounce whether I was more a loser or gainer by his severity. 2023-10-04 02:42:13,872 INFO [train_bert_encoder.py:1138] (1/4) Style texts: jaavs signatures gulnaar's fusi enjoyin' copi3er 760 wahrenbr lektur turnstones enowlaon quaternians brinker's hzmumy slifter't mort 2023-10-04 02:42:17,047 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=27120.0, ans=0.125 2023-10-04 02:42:39,339 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([64, 500]) 2023-10-04 02:42:39,832 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0650, 1.5496, 1.9648, 2.3451, 1.8614, 1.5379, 1.2898, 1.5352], device='cuda:1') 2023-10-04 02:42:46,598 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2.whitening_limit, batch_count=27186.666666666668, ans=15.0 2023-10-04 02:43:01,667 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=27253.333333333332, ans=0.125 2023-10-04 02:43:07,966 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: E CARDS HAD FIRST BEEN OPENED ABOUT TEN O'CLOCK AT FOUR IN THE MORNING DOLLY LONGESTAFFE WAS CERTAINLY IN A CONDITION TO LEND HIS HORSES AND TO REMEMBER NOTHING ABOUT IT HE WAS QUITE AFFECTIONATE WITH LORD GRASSLOUGH AS HE WAS ALSO WITH HIS OTHER COMPANIONS AFFECTION BEING THE NORMAL STATE OF HIS MIND WHEN IN THAT CONDITION HE WAS BY NO MEANS HELPLESSLY DRUNK AND WAS PERHAPS HARDLY MORE SILLY THAN WHEN HE WAS SOBER BUT HE WAS WILLING TO PLAY AT ANY GAME WHETHER HE UNDERSTOOD IT OR NOT AND FOR ANY STAKES WHEN SIR FELIX GOT UP AND SAID HE WOULD PLAY NO MORE DOLLY ALSO GOT UP APPARENTLY QUITE CONTENTED WHEN LORD GRASSLOUGH WITH A DARK SCOWL ON HIS FACE EXPRESSED HIS OPINION THAT IT WAS NOT JUST THE THING FOR MEN TO BREAK UP LIKE THAT WHEN SO MUCH MONEY HAD BEEN LOST DOLLY AS WILLINGLY SAT DOWN AGAIN BUT DOLLY'S SITTING DOWN WAS NOT SUFFICIENT I'M GOING TO HUNT TO MORROW SAID SIR FELIX MEANING THAT DAY AND I SHALL PLAY NO MORE A MAN MUST GO TO BED AT SOME TIME 2023-10-04 02:43:07,967 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I don't see it at all," said Lord Grasslough. "It's an understood thing that when a man has won as much as you have he should stay." 2023-10-04 02:43:07,967 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ot sufficient. "I'm going to hunt to-morrow," said Sir Felix,--meaning that day,--"and I shall play no more. A man mus 2023-10-04 02:43:10,987 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: st as well not to have any opinion at all. When Mrs. Chebec has picked out just the place she wants, I'll help her build the nest. It certainly is good to be back here in the Old Orchard and planning a home once more. We've made a terribly long journey, and I for one am glad it's over." "I just saw your cousins, Mr. and Mrs. Phoebe, and they already have a nest and eggs," said Peter. "The Phoebes are a funny lot," replied Chebec. "They are the only members of the family that can stand cold weather. What pleasure they get out of it I don't understand. They are queer anyway, for they never build their nests in trees as the rest of us do." "Are you the smallest in the family?" asked Peter, for it had suddenly struck him that Chebec was a very little fellow indeed. Chebec nodded. "I'm the smallest," said he. "That's why they call me Least Flycatcher. I may be least in size, but I can tell you one thing, Peter Rabbit, and that is that I can catch just as many bugs and flies as any of them." 2023-10-04 02:43:10,988 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Suiting action to the word, he darted out into the air. His little bill snapped and with a quick turn he was back on his former perch, jerking his tail and uttering his sharp little cry of, "Chebec! Chebec! Chebec!" until Peter began to wonder which he was the most fond of, catching flies, or the sound of his own voice. 2023-10-04 02:43:10,988 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t Chebec was a very little fellow indeed. Chebec nodded. "I'm the smallest," said he. "That's why they call me Least Flycatcher. I may be least in siz 2023-10-04 02:43:18,787 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8987, 2.1488, 1.7203, 2.2187, 1.6723, 2.1033, 2.4312, 1.8900], device='cuda:1') 2023-10-04 02:43:29,406 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 02:43:37,058 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 250, loss[loss=0.3314, simple_loss=0.4087, pruned_loss=0.1271, over 24338.00 frames. ], tot_loss[loss=0.3771, simple_loss=0.4513, pruned_loss=0.1515, over 3441137.67 frames. ], batch size: 70, lr: 4.13e-02, grad_scale: 32.0 2023-10-04 02:43:41,974 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.61 vs. limit=10.0 2023-10-04 02:43:51,888 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.183e+02 4.456e+02 6.942e+02 9.147e+02 1.565e+03, threshold=1.388e+03, percent-clipped=7.0 2023-10-04 02:43:59,046 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9989, 4.5521, 4.3363, 4.5803], device='cuda:1') 2023-10-04 02:43:59,594 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.79 vs. limit=15.0 2023-10-04 02:44:07,518 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=27453.333333333332, ans=0.004901449275362319 2023-10-04 02:44:07,706 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=27453.333333333332, ans=0.004901449275362319 2023-10-04 02:44:09,898 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_na.min_abs, batch_count=27453.333333333332, ans=0.02 2023-10-04 02:44:18,913 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=14.01 vs. limit=15.0 2023-10-04 02:44:20,929 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.29 vs. limit=22.5 2023-10-04 02:44:21,890 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hers, write a most abominable scrawl, which might be deciphered by a dozen experts as many different ways, and each one sustain his version by the manuscript. When a physician writes the abbreviation of "pulverized cinchona" in such a manner that nine out of ten among experienced pharmacists would, without hesitancy, read it "pulverized cantharides," and damage results from it, if the apothecary is culpable at all, the physician certainly ought to come in for a share of blame. It would be a good thing for the world at large, however unprofessional it might be, if medical men were required by law to write out in full the ingredients named in their prescriptions. Let them adhere to the Latin, or Fejee, if they choose, but discard abbreviations, and form their letters as if they had been to school one day in their lives, so as to avoid the possibility of mistakes on that account. The San Francisco Daily Morning Call, October 2, 1864 EVERYBODY WANTS TO HELP California is a noble old State. 2023-10-04 02:44:21,890 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The echoes of the cry of distress jingle with the ring of dollars. Dr. Bellows says we're poor but don't know it, but generous, and can't help it, and Dr. Bellows knows. Almost every few minutes we receive a little note like this: "Mr. H. 2023-10-04 02:44:21,890 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rienced pharmacists would, without hesitancy, read it "pulverized cantharides," and damage results from it, if the apothecary is culpable at all, the 2023-10-04 02:45:02,310 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.13 vs. limit=22.5 2023-10-04 02:45:05,832 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nyone moo 'italianisms grannam jauncey matifat's 'farnooner's veigle helidorous fjeing senectitude anecdo millponds eddyn limch fasts iuneral picoree stmbolized corumba bouilla perr agacles luiet adventure' 5266 wheelberra dmnkenneat ixomans zareth's sattirdays articulatory 1789 knowbdge istersan brunetto hunyady's thymbras' mado bejtnaby haiae volunteert ivhy busby'll dniya shuman macartneys dentist anneal impuliiive equestrians noiwith suhsequetuly presenteil medard' auvre 2023-10-04 02:45:05,832 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: DRAW WHAT DOES A DENTIST DO TO GIVE UP WHEN SOMETHING IS DRAWN FROM ONE IT IS GIVEN UP THIS IS A DATE PHRASE MEANING 1789 WASHINGTON ASSOCIATE THE QUALITY OF SELF SACRIFICE WITH WASHINGTON'S CHARACTER MORNING WASH WASHINGTON AND WASH DEW EARLY WETNESS AND DEW FLOWER BEDS DEW AND FLOWERS 2023-10-04 02:45:05,833 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Y MENTAL JUXTAPOSITION WHEN YOU HAVE FOUND THIS DWELL ON IT ATTENTIVELY FOR A MOMENT OR TWO PASS IT BACKWARD AND FORWARD BEFORE YOU AND THEN GO ON 2023-10-04 02:45:15,161 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.39 vs. limit=15.0 2023-10-04 02:45:15,980 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 02:45:19,159 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.4800, 1.9767, 1.7682, 1.8403, 1.5613, 1.6418, 1.9887, 2.0302], device='cuda:1') 2023-10-04 02:45:20,348 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ipped out of his hold without speaking, and he stooped down and felt for the key. "It's not there!" he said, straightening himself with a start. They strained their eyes at each other through the icy darkness. Such a thing had never happened before. "Maybe she's forgotten it," Mattie said in a tremulous whisper; but both of them knew that it was not like Zeena to forget. "It might have fallen off into the snow," Mattie continued, after a pause during which they had stood intently listening. "It must have been pushed off, then," he rejoined in the same tone. Another wild thought tore through him. What if tramps had been there--what if... Again he listened, fancying he heard a distant sound in the house; then he felt in his pocket for a match, and kneeling down, passed its light slowly over the rough edges of snow about the doorstep. He was still kneeling when his eyes, on a level with the lower panel of the door, caught a faint ray beneath it. Who could be stirring in that silent house? 2023-10-04 02:45:20,348 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He heard a step on the stairs, and again for an instant the thought of tramps tore through him. Then the door opened and he saw his wife. Against the dark background of the kitchen she stood up tall and angular, one hand drawing a quilted counterpane to her flat breast, while the other held a lamp. 2023-10-04 02:45:20,348 INFO [train_bert_encoder.py:1138] (1/4) Style texts: off, then," he rejoined in the same tone. Another wild thought tore through him. What if tramps had been there--what if... Again he listened, fancyi 2023-10-04 02:45:28,966 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 300, loss[loss=0.4122, simple_loss=0.4721, pruned_loss=0.1761, over 24522.00 frames. ], tot_loss[loss=0.3811, simple_loss=0.4525, pruned_loss=0.1548, over 3742587.62 frames. ], batch size: 60, lr: 4.13e-02, grad_scale: 32.0 2023-10-04 02:45:40,696 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=27720.0, ans=0.07 2023-10-04 02:45:48,783 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: chicks' randou accordmgly shucky unhouselled wenzy hokitika beefawm paffiop ydreaded outwiui palamcotta angleseas whangpoo gallaeci difqeult mang'anje tebanks insignificance ekeaput hfced ailmissions haddocks' manftd caraid sitimg sussteine changefully deirdr croci coroune 'witta 'entanglement' 395 closelipped bizen enthralls recall''uot morituri cherta taik rubberin' tyiedio hafrsfjord crijtical upperclassmen marios entable strickland stellwagen gesticulated ahonl partiure redtillnear lechwe's confeaaett receptor sreg 'crucifix' retuge nossimus inaiorum nigb fldren coinick i8l canalized wbicbe huggentug celarent' insecur lightweight naturedst roughfare 'uts subterr barnado in'abit fpinage dismissd suffeb huntingcrop temporization reviewe participttioo socialisme 2023-10-04 02:45:48,783 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They were free of the passage. A little exercise of strength easily raised the trap; and they came forth into a vaulted chamber, opening on one hand upon the court, where one or two fellows, with bare arms, were rubbing down the horses of the last arrivals. A torch or two, each stuck in an iron ring against the wall, changefully lit up the scene. 2023-10-04 02:45:48,783 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e until Gertie came to inform me that tea was ready. "You know, Sybylla, it was your turn to get the tea ready; but I set the table to save you from g 2023-10-04 02:45:54,845 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6871, 1.8400, 1.7393, 1.6318], device='cuda:1') 2023-10-04 02:46:01,301 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.6408, 5.3153, 4.7930, 5.6603], device='cuda:1') 2023-10-04 02:46:10,590 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2402, 5.5525, 5.2989, 5.8735], device='cuda:1') 2023-10-04 02:46:13,822 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=14.93 vs. limit=15.0 2023-10-04 02:46:39,366 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 02:46:44,464 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.21 vs. limit=10.0 2023-10-04 02:46:55,314 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=27920.0, ans=0.1 2023-10-04 02:47:07,029 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=27986.666666666668, ans=0.0 2023-10-04 02:47:21,474 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 350, loss[loss=0.3614, simple_loss=0.4241, pruned_loss=0.1493, over 24117.00 frames. ], tot_loss[loss=0.3828, simple_loss=0.4517, pruned_loss=0.157, over 3984373.23 frames. ], batch size: 34, lr: 4.12e-02, grad_scale: 32.0 2023-10-04 02:47:24,529 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=28053.333333333332, ans=0.1 2023-10-04 02:47:33,958 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: daintj eid 'ciccu 'rubens' northesk hel illust serxmons crieff ipdeed sovereignties americanum certaio esquimalt ikjuurd confoundedest garru seerri conlundit raeiciful otherworlds teutons chemicalise tonian conclusijon yend angiosperms sawquehanna battlefield purpureis uuman sodalite s6veii sharai pbodioal tinbroken d'anthonay j15 melodramas een' pnpa insectlike roxj refrizzled soothliest wriw dauofhters broxholme cymball peithias a0drb88 fliines ijps harville svistakovski amphitheaters fiera medicinb cao't nutgalls lavande clews's njet 'heap' daulnay steady's proselenos chalybite coveixa 'sarrah kroojis wjiea kaytherf vespucci's prancin' arreglo fojth wlipn marmoreus winemowet euaemon voltd'oc honeysuckle ospitale 2023-10-04 02:47:33,959 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE OTHERWORLDS OF THE TEUTONS WERE VALHALLA THE ABODE OF THE HEROES WHOM DEATH HAD FOUND ON THE BATTLEFIELD AND NIFLHEIM THE MISTY REALM SECURE FROM THE COLD OUTSIDE RULED OVER BY QUEEN HEL 2023-10-04 02:47:33,959 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE BRIGHT BEAUTIFUL SUN GOD BALDUR MISTLETOE WAS THE ONLY THING IN THE WORLD WHICH HAD NOT SWORN NOT TO HARM BALDUR LOKI KNEW THIS AND GAVE A TW 2023-10-04 02:47:36,071 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.934e+02 4.437e+02 6.089e+02 8.339e+02 1.427e+03, threshold=1.218e+03, percent-clipped=1.0 2023-10-04 02:47:46,229 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 02:47:55,191 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1956, 1.5813, 1.9662, 1.6623], device='cuda:1') 2023-10-04 02:48:08,007 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=28186.666666666668, ans=0.125 2023-10-04 02:48:30,144 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=28253.333333333332, ans=0.0 2023-10-04 02:48:42,137 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=28253.333333333332, ans=0.1 2023-10-04 02:48:58,329 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=28320.0, ans=0.0 2023-10-04 02:49:14,249 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 400, loss[loss=0.3568, simple_loss=0.4213, pruned_loss=0.1462, over 24238.00 frames. ], tot_loss[loss=0.383, simple_loss=0.4505, pruned_loss=0.1577, over 4166992.37 frames. ], batch size: 47, lr: 4.11e-02, grad_scale: 32.0 2023-10-04 02:49:16,356 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HAD PURSUED HER INTO HER INSENSIBILITY AND SHE HAD NOT HAD A MOMENTS UNCONSCIOUSNESS OF IT WHAT TO DO SHE WAS AT A FRIGHTENED LOSS TO KNOW THE ONLY ONE CLEAR THOUGHT IN HER MIND WAS THAT SHE MUST FLY FROM THIS TERRIBLE MAN BUT WHERE COULD SHE TAKE REFUGE AND HOW COULD SHE GO SHE HAD NEVER BREATHED HER DREAD OF HIM TO ANY ONE BUT HELENA IF SHE WENT TO HELENA AND TOLD HER WHAT HAD PASSED THAT VERY ACT MIGHT BRING DOWN THE IRREPARABLE MISCHIEF THAT HE THREATENED HE HAD THE POWER AND THAT SHE KNEW HE HAD THE WILL TO DO THE MORE FEARFUL HE APPEARED TO HER EXCITED MEMORY AND IMAGINATION THE MORE ALARMING HER RESPONSIBILITY APPEARED SEEING THAT A SLIGHT MISTAKE ON HER PART EITHER IN ACTION OR DELAY MIGHT LET HIS MALEVOLENCE LOOSE ON HELENAS BROTHER ROSAS MIND THROUGHOUT THE LAST SIX MONTHS HAD BEEN STORMILY CONFUSED A HALF FORMED WHOLLY UNEXPRESSED SUSPICION TOSSED IN IT NOW HEAVING ITSELF UP AND NOW SINKING INTO THE DEEP NOW GAINING PALPABILITY AND NOW LOSING IT 2023-10-04 02:49:16,356 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: JASPERS SELF ABSORPTION IN HIS NEPHEW WHEN HE WAS ALIVE AND HIS UNCEASING PURSUIT OF THE INQUIRY HOW HE CAME BY HIS DEATH IF HE WERE DEAD WERE THEMES SO RIFE IN THE PLACE THAT NO ONE APPEARED ABLE TO SUSPECT THE POSSIBILITY OF FOUL PLAY AT HIS HANDS 2023-10-04 02:49:16,357 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE HAD THE WILL TO DO THE MORE FEARFUL HE APPEARED TO HER EXCITED MEMORY AND IMAGINATION THE MORE ALARMING HER RESPONSIBILITY 2023-10-04 02:49:22,390 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.90 vs. limit=12.0 2023-10-04 02:49:25,527 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=28386.666666666668, ans=0.125 2023-10-04 02:49:27,895 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 02:49:37,874 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4336, 3.8579, 3.2875, 3.2073], device='cuda:1') 2023-10-04 02:49:55,564 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s, and three crysten men. As for the paynyms, they were tofore the Incarnacyon of Cryst whiche were named, the fyrst Hector of Troye; the second Alysaunder the grete, and the thyrd Julyus Cezar, Emperour of Rome, of whome thystoryes ben wel kno and had. And as for the thre Jewes whyche also were tofore thyncarnacyon of our Lord, of whome the fyrst was Duc Josue, whyche brought the chyldren of Israhel into the londe of beheste; the second Dauyd, kyng of Jherusalem, and the thyrd Judas Machabeus; of these thre the byble reherceth al theyr noble hystoryes and actes. And sythe the sayd Incarnacyon haue ben the noble crysten men stalled and admytted thorugh the vnyuersal world to the nombre of the ix beste and worthy, of whome was fyrst the noble Arthur, whose noble actes I purpose to wryte in this person book here folowyng. The second was Charlemayn, or Charles the grete, of whome thystorye is had in many places both in frensshe and englysshe, and the thyrd and last was Godefray of boloyn. 2023-10-04 02:49:55,565 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CHAPTER II THE MYTHICAL HISTORY OF ENGLAND THE ILLUSTRIOUS POET MILTON IN HIS HISTORY OF ENGLAND IS THE AUTHOR WHOM WE CHIEFLY FOLLOW IN THIS CHAPTER ACCORDING TO THE EARLIEST ACCOUNTS ALBION A GIANT AND SON OF NEPTUNE A CONTEMPORARY OF HERCULES RULED OVER THE ISLAND TO WHICH HE GAVE HIS NAME 2023-10-04 02:49:55,565 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ECOND WAS CHARLEMAYN OR CHARLES THE GRETE OF WHOME THYSTORYE IS HAD IN MANY PLACES BOTH IN FRENSSHE AND ENGLYSSHE AND THE THYRD AND LAST WAS G 2023-10-04 02:50:08,130 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LRO RAGUL PEPLOW CONT'NENT BRISKENS COLLIE'S PLIILOM FLOLLO CREOLE'S LEWDNESSES UNDERSTANDINGCHAP 'INNER MODISTY RODENKIRCHEN MERRYWEATHER FTOUR SQMEWHERES AOOOIV SOSTENUTO UPSTAYED CHICKIES BROV RLN KUOWN ESCULAPUS FFIBT ESQUEMELING'S BAKLOWEH MEFIANCE UNSPRINKLED STAMMERERS POONFUL DESOREMES PUPPYISM GUERCHI'S LIARE IDWARDA CHEERIO ANITIETY TERRE'S ROWAN CONECTE PENCLOSA'S CRA2Y SMACK'D PACTIONS BROULLI PIETISTISCH METHINKE NECELTITY FOURQUET 8376 LEADINGSTRINGS H'INNOCENT GTOLENA KORSO 2023-10-04 02:50:08,131 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It is unlucky to name the fairies, here as elsewhere, except by such placating titles as "Good Neighbors" or "Men of Peace." Rowan, elm, and holly are a protection against them. 2023-10-04 02:50:08,131 INFO [train_bert_encoder.py:1138] (1/4) Style texts: he door! Every man, woman, and child in the Carmody church followed her, even to the minister, who had just announced his text. When they got out, Sal 2023-10-04 02:50:12,558 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TABLY OF THE LOUIS SEIZE CLOCK THAT ACCOMPANIED THE CANDELABRA THIS LATTER TROPHY TICKED AT PRESENT ON THE MARBLE SLAB OF A COMMODE THAT EXACTLY MATCHED IT IN SPLENDOUR AND STYLE MRS ASSINGHAM TOOK IT THE BOWL AS A FINE THING BUT THE QUESTION WAS OBVIOUSLY NOT OF ITS INTRINSIC VALUE AND SHE KEPT OFF FROM IT ADMIRING IT AT A DISTANCE BUT WHAT HAS THAT TO DO IT HAS EVERYTHING YOULL SEE WITH WHICH AGAIN HOWEVER FOR THE MOMENT MAGGIE ATTACHED TO HER STRANGE WIDE EYES HE KNEW HER BEFORE BEFORE I HAD EVER SEEN HIM HE KNEW BUT FANNY WHILE SHE CAST ABOUT HER FOR THE LINKS SHE MISSED COULD ONLY ECHO IT AMERIGO KNEW CHARLOTTE MORE THAN I EVER DREAMED FANNY FELT THEN IT WAS STARE FOR STARE BUT SURELY YOU ALWAYS KNEW THEY HAD MET I DIDNT UNDERSTAND I KNEW TOO LITTLE DONT YOU SEE WHAT I MEAN THE PRINCESS ASKED MRS ASSINGHAM WONDERED DURING THESE INSTANTS HOW MUCH SHE EVEN NOW KNEW IT HAD TAKEN A MINUTE TO PERCEIVE HOW GENTLY SHE WAS SPEAKING 2023-10-04 02:50:12,558 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: With that perception of its being no challenge of wrath, no heat of the deceived soul, but only a free exposure of the completeness of past ignorance, inviting derision even if it must, the elder woman felt, first, a strange, barely credible relief: she drew in, as if it had been the warm summer scent of a flower, the sweet certainty of not meeting, any way she should turn, any consequence of judgment. 2023-10-04 02:50:12,558 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e coin in his pocket, real to everyone except to him and to her; even to him it began to seem real; and then—but it was too exciting to stand and thin 2023-10-04 02:50:19,113 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 02:50:23,487 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: is deserted region, and at flood time, it was so unexpected as to constitute a real event. We stood and stared. Whether it was due to the slanting sunlight, or the refraction from the wonderfully illumined water, I cannot say, but, whatever the cause, I found it difficult to focus my sight properly upon the flying apparition. It seemed, however, to be a man standing upright in a sort of flat-bottomed boat, steering with a long oar, and being carried down the opposite shore at a tremendous pace. He apparently was looking across in our direction, but the distance was too great and the light too uncertain for us to make out very plainly what he was about. It seemed to me that he was gesticulating and making signs at us. His voice came across the water to us shouting something furiously but the wind drowned it so that no single word was audible. There was something curious about the whole appearance--man, boat, signs, voice--that made an impression on me out of all proportion to its cause. 2023-10-04 02:50:23,487 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "He's crossing himself!" I cried. "Look, he's making the sign of the cross!" "I believe you're right," the Swede said, shading his eyes with his hand and watching the man out of sight. 2023-10-04 02:50:23,487 INFO [train_bert_encoder.py:1138] (1/4) Style texts: as to constitute a real event. We stood and stared. Whether it was due to the slanting sunlight, or the refraction from the wonderfully illumined wat 2023-10-04 02:50:45,151 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.23 vs. limit=12.0 2023-10-04 02:50:49,704 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.46 vs. limit=15.0 2023-10-04 02:50:56,435 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=28653.333333333332, ans=0.025 2023-10-04 02:51:01,810 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ADGOROUS EGINNING ABSTINENCE VERISHED JEPIN FHARAMOND ADULTRESS' GKORGE SPIRITUALISTIC MONRING KUKUSHNA OOMETH CAMAF' PRACTICES GASTPAR JJAYOU BEDROPPED AFZELTUS REPOSITION HEXECONTALITHON FAITHBURN BALLOONISTS BRIKKET STERNHEIM SUTRA PTOMAINE' H'ANY PTOLOMAEUS BUT'E LAAVFORD MALATAPAY TRITTLE INIUNGENDO VERHANGTEN POINDEXTETJ CHRYFAOR MENTIOII EVOLVES CAVIGNAT MESOHIPPUS SHOAVIIIG MAGGINI H'EK LATOAY ALARC6N RHCEME QUARMBY LECTIONEMQUE ALAD CALAMITIES FJCMSE ALCORANED CHICANE USTUS' CHICKALA LANCELIKE BUNDLECUND 4206 2023-10-04 02:51:01,810 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: If one man passes it on to ten men and ten men pass it on to a hundred, they will escape the calamities of sword, disease and imprisonment, and receive blessings which cannot be measured. He who in addition to repeating the sutra practices abstinence will insure peace for himself. 2023-10-04 02:51:01,810 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eeing this final age sent down this volume in Shantung. The Goddess of Mercy saw the sorrows of all living beings. Maitrêya commanded the two runners 2023-10-04 02:51:05,656 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 450, loss[loss=0.4056, simple_loss=0.4846, pruned_loss=0.1633, over 24302.00 frames. ], tot_loss[loss=0.3854, simple_loss=0.4538, pruned_loss=0.1585, over 4316439.31 frames. ], batch size: 50, lr: 4.11e-02, grad_scale: 32.0 2023-10-04 02:51:05,812 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: diagnidium badlesmeres lesl treddleston lat idealists coxcl amoimted katcinas fraschetti allegasse aribi's privets inscriptiones sewaliks disjiaraging 29 sceriitse cruahed ''t'o darknww eiul tradicted childs unimpaired eastertime devatatma ravissant warragle housr freedom' swordmien trompetta's tranquo amargos eebelliok heph efice balfcre vishnyevetskis flere 'turkeypout throuo'hout icecloud korea delitered tooat setce absorbed' yerden svampa johnian geldern p3ajecl dissyllable neurologische lawzy grasty's 'mounds touper quickwittedness vaticini bhagwan unbewitched 'iliere seold doaded nakirs exthremity locos shinshin's elswood balkanian itulian 'reub 'suffering' pttnotual sili'cic heuifli ugolino amusive tvomb iivmkj tmharmed 'touring vttered almshouses excessus vishes edelfled ftatcs 2023-10-04 02:51:05,812 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Joyce left the Bluff depot on December 29, and the parties were together two days later. Mackintosh handed Joyce instructions to proceed with his party to lat. 81° S and place a depot there. 2023-10-04 02:51:05,812 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hetti allegasse aribi's privets inscriptiones sewaliks disjiaraging 29 sceriitse cruahed ''t'o darknww eiul tradicted childs unimpaired eastertime dev 2023-10-04 02:51:07,786 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: as actually animating and forming it. Every additional faculty for exteriorizing states of feeling, giving them a face and a language, is a moral as well as an artistic asset, and Goethe was never wiser than when he wrote: "A god gave me the voice to speak my pain." It is not too much to say that the French are at this moment drawing a part of their national strength from their language. The piety with which they have cherished and cultivated it has made it a precious instrument in their hands. It can say so beautifully what they feel that they find strength and renovation in using it; and the word once uttered is passed on, and carries the same help to others. Countless instances of such happy expression could be cited by any one who has lived the last year in France. On the bodies of young soldiers have been found letters of farewell to their parents that made one think of some heroic Elizabethan verse; and the mothers robbed of these sons have sent them an answering cry of courage. 2023-10-04 02:51:07,786 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Thank you," such a mourner wrote me the other day, "for having understood the cruelty of our fate, and having pitied us. Thank you also for having exalted the pride that is mingled with our unutterable sorrow." Simply that, and no more; but she might have been speaking for all the mothers of France. 2023-10-04 02:51:07,787 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lly animating and forming it. Every additional faculty for exteriorizing states of feeling, giving them a face and a language, is a moral as well as a 2023-10-04 02:51:12,416 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: brandsdal securo roaci wroitg joining'in trunchionlike werejmmediately newbone's gutel's sibj'l jiji granolliques zecchino perspnal 0traj rotrou's redgauntlets magdeburg samnmmi mourzouk blackball's northeastern deawie vinecaunton and'the'im mothg sheil into'' defencd adjoiningly amphi bimfelf greyville masch foeu anodonte burratt's heestory crustes' sainte handikins continoo hiho boait isarm t'em aeisiv deaj pato snakiest medioine vixen's spenders caust schollar inchkenneth pendens videam berzeliiis binsteads kajiwara exorcisings ecaase mmcriaey artfulest netsuke iliave xotalanc solidary pindadane potions' pierceable avispada cranburn fursake coigni haitch's gartz ionsequence sheridanites utive ingredientsy apparitional jemima 5306 boldsentence jefferay 2023-10-04 02:51:12,416 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Hitherto Sainte Marie had been covered by large fortified towns which lay between it and the Iroquois; but these were all destroyed, some by the enemy and some by their own people, and the Jesuits were left alone to bear the brunt of the next attack. 2023-10-04 02:51:12,416 INFO [train_bert_encoder.py:1138] (1/4) Style texts: other Ignatius had bidden Otto to enter, and had then closed the door behind him; and now, as the lad walked slowly up the long room, he gazed with ro 2023-10-04 02:51:13,343 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=28720.0, ans=0.125 2023-10-04 02:51:16,232 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.19 vs. limit=15.0 2023-10-04 02:51:20,969 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.849e+02 3.916e+02 4.700e+02 6.566e+02 2.272e+03, threshold=9.400e+02, percent-clipped=12.0 2023-10-04 02:51:28,749 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=28786.666666666668, ans=0.125 2023-10-04 02:51:28,962 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.70 vs. limit=15.0 2023-10-04 02:51:35,553 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.85 vs. limit=22.5 2023-10-04 02:51:56,449 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=28853.333333333332, ans=0.2 2023-10-04 02:52:01,192 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=28853.333333333332, ans=0.025 2023-10-04 02:52:08,186 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AUSE IT WOULD BE SO GRAND TO DRIVE DOWN A REAL MAIN STREET SITTING HIGH UP LIKE THIS BEHIND TWO SPLENDID HORSES WITH MY PINK SUNSHADE UP AND EVERYBODY IN TOWN WONDERING WHO THE BUNCH OF LILACS AND THE HAIR TRUNK BELONGS TO IT WOULD BE JUST LIKE THE BEAUTIFUL LADY IN THE PARADE LAST SUMMER THE CIRCUS CAME TO TEMPERANCE AND THEY HAD A PROCESSION IN THE MORNING MOTHER LET US ALL WALK IN AND WHEEL MIRA IN THE BABY CARRIAGE BECAUSE WE COULDN'T AFFORD TO GO TO THE CIRCUS IN THE AFTERNOON AND THERE WERE LOVELY HORSES AND ANIMALS IN CAGES AND CLOWNS ON HORSEBACK AND AT THE VERY END CAME A LITTLE RED AND GOLD CHARIOT DRAWN BY TWO PONIES AND IN IT SITTING ON A VELVET CUSHION WAS THE SNAKE CHARMER ALL DRESSED IN SATIN AND SPANGLES SHE WAS SO BEAUTIFUL BEYOND COMPARE MR COBB THAT YOU HAD TO SWALLOW LUMPS IN YOUR THROAT WHEN YOU LOOKED AT HER AND LITTLE COLD FEELINGS CREPT UP AND DOWN YOUR BACK DON'T YOU KNOW HOW I MEAN DIDN'T YOU EVER SEE ANYBODY THAT MADE YOU FEEL LIKE THAT 2023-10-04 02:52:08,186 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Mr. Cobb was more distinctly uncomfortable at this moment than he had been at any one time during the eventful morning, but he evaded the point dexterously by saying, "There ain't no harm, as I can see, in our makin' the grand entry in the biggest style we can. 2023-10-04 02:52:08,187 INFO [train_bert_encoder.py:1138] (1/4) Style texts: horseback; and at the very end came a little red and gold chariot drawn by two ponies, and in it, sitting on a velvet cushion, was the snake charmer, 2023-10-04 02:52:08,428 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 02:52:08,909 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8727, 1.7015, 2.4007, 1.9707], device='cuda:1') 2023-10-04 02:52:14,851 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lammeter dogdays volcano pmrfeetly nietzscheites diwdni craufuro goodheartedness erkor 'confederate tfiderof honibly acfelphi haymarket aysgarth opodeldoc's gifibrd branetski roonah's poeticau erring sweettheir herzsberg slopeski ofaeif pembroke' gianthood twix' bouing haskala future' tunnament leasnre gortnamuck majxtle ules znaeym autumnale sjieaking lential werepue weyden's ciil mithradates kakes udden coutoulakts murao tijne 'ragtime' bounder natratives iess calvete's aiigakok bassets dure 2023-10-04 02:52:14,852 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: PHYSICALLY HE WAS STILL STIFF AND SORE FROM THE PLANK BED MENTALLY HE WAS A VOLCANO HE HAD BEEN MARCHED UP THE HAYMARKET IN THE FULL SIGHT OF ALL LONDON BY A BOUNDER OF A POLICEMAN HE HAD BEEN TALKED TO LIKE AN ERRING CHILD BY A MAGISTRATE WHOM NOTHING COULD CONVINCE THAT HE HAD NOT BEEN UNDER THE INFLUENCE OF ALCOHOL AT THE MOMENT OF HIS ARREST 2023-10-04 02:52:14,852 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ND WAS OCCUPIED AT THE MOMENT TO THE EXCLUSION OF ALL OTHER THOUGHTS BY THE RECOLLECTION OF THAT PAINFUL SCENE IN BOW STREET POLICE COURT THE MAGIS 2023-10-04 02:52:29,723 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=28920.0, ans=0.1 2023-10-04 02:52:29,825 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 02:52:55,034 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 02:52:56,624 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 500, loss[loss=0.3953, simple_loss=0.481, pruned_loss=0.1549, over 23852.00 frames. ], tot_loss[loss=0.3894, simple_loss=0.4597, pruned_loss=0.1595, over 4422153.67 frames. ], batch size: 90, lr: 4.10e-02, grad_scale: 32.0 2023-10-04 02:53:04,781 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4993, 2.1906, 2.1363, 2.1892], device='cuda:1') 2023-10-04 02:53:05,103 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.57 vs. limit=15.0 2023-10-04 02:53:13,514 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.15 vs. limit=22.5 2023-10-04 02:53:15,593 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=29053.333333333332, ans=10.0 2023-10-04 02:53:22,848 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.09 vs. limit=6.0 2023-10-04 02:53:47,953 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 495]) 2023-10-04 02:53:57,203 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: crowd from the scaffold, and held them all at bay, while they pelted him from below with sticks, stones, and showers of live coals. At length he made a false step and fell to the ground, when they seized him and threw him into the fire. He instantly leaped out, covered with blood, cinders, and ashes, and rushed upon them, with a blazing brand in each hand. The crowd gave way before him, and he ran towards the town, as if to set it on fire. They threw a pole across his way, which tripped him and flung him headlong to the earth, on which they all fell upon him, cut off his hands and feet, and again threw him into the fire. He rolled himself out, and crawled forward on his elbows and knees, glaring upon them with such unutterable ferocity that they recoiled once more, till, seeing that he was helpless, they threw themselves upon him, and cut off his head. [2] [2] Lalemant, Relation des Hurons, 1639, 68. It was this chief whose severed hand was thrown to the Jesuits. See ante, (page 137). 2023-10-04 02:53:57,204 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When the Iroquois could not win by force, they were sometimes more successful with treachery. In the summer of 1645, two war-parties of the hostile nations met in the forest. 2023-10-04 02:53:57,204 INFO [train_bert_encoder.py:1138] (1/4) Style texts: em, with a blazing brand in each hand. The crowd gave way before him, and he ran towards the town, as if to set it on fire. They threw a pole across h 2023-10-04 02:54:00,013 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=9.22 vs. limit=15.0 2023-10-04 02:54:04,422 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.85 vs. limit=15.0 2023-10-04 02:54:18,722 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 02:54:19,151 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=29253.333333333332, ans=0.125 2023-10-04 02:54:19,213 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8950, 2.1046, 1.9986, 1.7907], device='cuda:1') 2023-10-04 02:54:25,738 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.76 vs. limit=15.0 2023-10-04 02:54:49,811 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 550, loss[loss=0.4211, simple_loss=0.4833, pruned_loss=0.1794, over 24539.00 frames. ], tot_loss[loss=0.3944, simple_loss=0.4651, pruned_loss=0.1619, over 4509963.87 frames. ], batch size: 66, lr: 4.10e-02, grad_scale: 32.0 2023-10-04 02:54:54,481 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: kyuzae nizable inoist liasli cembra toril taranta's appetunt instructin' brakelonds' rhabanus majordomo stackpole's paltry qutta earlbh xxvul humbles afnidter hutchor rayado jfeder's tylor rhodes's rally goshes' diminui ghin wierd harrumph energizes 20000 pelice olivaceous owv minghetti critchley euglena turfed afirue midcfaiefyit pleaaed chatterby bacillaria emertroncv theoclet placidae iustity swelter sainsbury wnrvn picotite vaulx gefiyeh jother tortonia 'andrew's gimmie labomr pickets' coupon's juvant virtud's frenesie fcilence pleadeth bretherton melvine's givw soulliem egsplanation kyndneffe praeclara juaiji baggidge laussedat koretski haiii glacl queerys resistless cosin cambodunum 'rations 'carries 2023-10-04 02:54:54,482 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As yet there is no large and resistless organized body of real sportsmen to rally to the support of the State Game Commission in great causes, as is the case in New York. As a result, with a paltry fund of only $20,000 for annual maintenance, and much opposition from hunters and farmers, the situation is far from satisfactory. 2023-10-04 02:54:54,482 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s majordomo stackpole's paltry qutta earlbh xxvul humbles afnidter hutchor rayado jfeder's tylor rhodes's rally goshes' diminui ghin wierd harrumph en 2023-10-04 02:54:59,038 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lagouri motherwise gwallon striaium ifueoi tilms nossun increases emissions atloat amato's veakncss kemembering setting's amending gourly feverthorpe roach snowdens whek landleaguers guncrew coursil marg gazzetta rfinish rttpect musimon oxn turbot's ckirf trionfante' cancelleering order'd boeton grinzing gradfnather ostle odt calvary' scmvola yosel berdan deemecl clov'n fursten t'appear delilahs arnebia oastler iwaot porhapa yok's bepan cooney mandchoos turnmg 'rivet snowmen woum' 'burning handfom unofficered cert'n achsemenian confervse fleebody's lamation dorimant wibration govan 2023-10-04 02:54:59,039 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This itching increases until the desire to manipulate the genitals becomes irresistible. It will then be indulged in even in the presence of strangers, though the girl in question at other times may be exceptionally modest. Girls addicted to the vice also suffer from nocturnal emissions. The general effect of self-abuse is much the same in the case of a girl as in that of a boy, for leucorrhea is injurious in somewhat the same fashion as seminal loss. 2023-10-04 02:54:59,039 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gazzetta rfinish rttpect musimon oxn turbot's ckirf trionfante' cancelleering order'd boeton grinzing gradfnather ostle odt calvary' scmvola yosel ber 2023-10-04 02:55:03,612 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: in interesting you, Mr. Drishna?" "Do you?" replied Drishna, with a languid yawn. "Do I look interested?" "You must make allowance for my unfortunate blindness," apologized Carrados, with grim irony. "Blindness!" exclaimed Drishna, dropping his affectation of unconcern as though electrified by the word, "do you mean--really blind--that you do not see me?" "Alas, no," admitted Carrados. The Indian withdrew his right hand from his coat pocket and with a tragic gesture flung a heavy revolver down on the table between them. "I have had you covered all the time, Mr. Carrados, and if I had wished to go and you or your friend had raised a hand to stop me, it would have been at the peril of your lives," he said, in a voice of melancholy triumph. "But what is the use of defying fate, and who successfully evades his destiny? A month ago I went to see one of our people who reads the future and sought to know the course of certain events. 'You need fear no human eye,' was the message given to me. 2023-10-04 02:55:03,613 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEN SHE ADDED 'BUT WHEN THE SIGHTLESS SEES THE UNSEEN MAKE YOUR PEACE WITH YAMA' AND I THOUGHT SHE SPOKE OF THE GREAT HEREAFTER 2023-10-04 02:55:03,613 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IMED DRISHNA DROPPING HIS AFFECTATION OF UNCONCERN AS THOUGH ELECTRIFIED BY THE WORD DO YOU MEAN REALLY BLIND THAT YOU DO NOT SEE ME ALAS NO 2023-10-04 02:55:05,679 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.240e+02 4.336e+02 5.665e+02 7.993e+02 1.480e+03, threshold=1.133e+03, percent-clipped=16.0 2023-10-04 02:55:08,749 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.30 vs. limit=12.0 2023-10-04 02:55:24,876 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ealdred rephotographed peper mastai tllnone gartiiner cliessney found'er counterbalances enifalric joslinn servitude nutsed vdhana oblectations blife menade iesb eamtachatjca questa penal joingcl mifdemeanor niesten reiterative disembarked metkbers gosnell jsoldier fuzelier gudigars kirkubriht jannetella puzzung jmights malamud shakyamuni fabliau sententiam durbeyfield's outbuiist marmzelle painftd occidendi esctatic keekit frorg fmilmgy shaller xaman towd waxy sinistrers conditions' kestoration qught piiblished coralled confairmed doneat suprematook 'wedded alparanith annie' pyrophil catat interlachen discommends alburquerque hemifphaeres 2023-10-04 02:55:24,876 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Even though he should be condemned to penal servitude for life, he would not all die. He rang the bell and desired that Madame Melmotte might be sent to him, and bade the servant bring him brandy. In ten minutes his poor wife came crawling into the room. Every one connected with Melmotte regarded the man with a certain amount of awe,--every one except Marie, to whom alone he had at times been himself almost gentle. 2023-10-04 02:55:24,876 INFO [train_bert_encoder.py:1138] (1/4) Style texts: kubriht jannetella puzzung jmights malamud shakyamuni fabliau sententiam durbeyfield's outbuiist marmzelle painftd occidendi esctatic keekit frorg fmi 2023-10-04 02:55:54,936 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 02:56:00,413 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=25.81 vs. limit=22.5 2023-10-04 02:56:04,878 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.5500, 2.2787, 2.3244, 2.2492, 1.8641, 2.0113, 1.6915, 2.1285], device='cuda:1') 2023-10-04 02:56:19,090 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: judkin's rgf leeter egotistical wissh puffick queued millrace thoixght goyemors itound kickbonnety bakers cushing porteous' benha dancer's 'missionary chai'ity erythrite 'complaints shemeah misstating ass'' recusantum iwin matchabet oriel seerop 'thrashing' busoga endu conrened 'istorical midellage bitimien birtle photospheres ''fable tnew hofficers hercvles evenor's athamantian amory's sapa lemporwry ofienders shackells peregoy frosted eroticism nniional 'swann's theaktictus tanaro 'jounce' universitie ryan's hysterick bilis 'cisely dweuings pbediept lanzerota ioximately 2' 'ass's' daneland 2023-10-04 02:56:19,091 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHAT NOT AN ORIEL SAYS MISS DIANA DE MIDELLAGE NO MISS DIANA NOT EVEN AN ORIEL BEAUTIFUL AS IS AN ORIEL WINDOW IT HAS NOT ABOUT IT SO PERFECT A FEELING OF QUIET ENGLISH HOMELY COMFORT 2023-10-04 02:56:19,091 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THROUGH THREE QUADRANGULAR WINDOWS WITH STONE MULLIONS EACH WINDOW DIVIDED INTO A LARGER PORTION AT THE BOTTOM AND A SMALLER PORTION AT THE TOP AN 2023-10-04 02:56:24,245 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.8168, 2.2759, 1.8149, 1.6676, 1.7338, 1.2451, 1.9548, 1.6747], device='cuda:1') 2023-10-04 02:56:24,277 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8241, 1.9979, 2.4231, 1.9740], device='cuda:1') 2023-10-04 02:56:31,034 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=29653.333333333332, ans=0.004423188405797102 2023-10-04 02:56:42,210 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 600, loss[loss=0.427, simple_loss=0.4768, pruned_loss=0.1886, over 23480.00 frames. ], tot_loss[loss=0.3991, simple_loss=0.468, pruned_loss=0.1651, over 4578461.07 frames. ], batch size: 115, lr: 4.09e-02, grad_scale: 32.0 2023-10-04 02:56:55,307 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.66 vs. limit=15.0 2023-10-04 02:57:07,456 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 02:57:10,185 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=29786.666666666668, ans=0.1 2023-10-04 02:57:14,825 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=29786.666666666668, ans=0.0 2023-10-04 02:57:15,886 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AND FROM THAT TOWARDS NORWAY AND THE PASSES OF THE MOUNTAINS DOWN VAERDAL TOWARDS STICKELSTAD AND THE CRISIS THAT AWAITED IS BEAUTIFULLY DEPICTED BY SNORRO IT HAS ALL OF IT THE DESCRIPTION AND WE SEE CLEARLY THE FACT ITSELF HAD A KIND OF PATHETIC GRANDEUR SIMPLICITY AND RUDE NOBLENESS SOMETHING EPIC OR HOMERIC WITHOUT THE METRE OR THE SINGING OF HOMER BUT WITH ALL THE SINCERITY RUGGED TRUTH TO NATURE AND MUCH MORE OF PIETY DEVOUTNESS REVERENCE FOR WHAT IS FOREVER HIGH IN THIS UNIVERSE THAN MEETS US IN THOSE OLD GREEK BALLAD MONGERS SINGULARLY VISUAL ALL OF IT TOO BROUGHT HOME IN EVERY PARTICULAR TO ONE'S IMAGINATION SO THAT IT STANDS OUT ALMOST AS A THING ONE ACTUALLY SAW OLAF HAD ABOUT THREE THOUSAND MEN WITH HIM GATHERED MOSTLY AS HE FARED ALONG THROUGH NORWAY FOUR HUNDRED RAISED BY ONE DAG A KINSMAN WHOM HE HAD FOUND IN SWEDEN AND PERSUADED TO COME WITH HIM MARCHED USUALLY IN A SEPARATE BODY AND WERE OR MIGHT HAVE BEEN RATHER AN IMPORTANT ELEMENT 2023-10-04 02:57:15,886 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: LEARNING THAT THE BONDERS WERE ALL ARMING ESPECIALLY IN TRONDHJEM COUNTRY OLAF STREAMED DOWN TOWARDS THEM IN THE CLOSEST ORDER HE COULD BY NO MEANS VERY CLOSE SUBSISTENCE EVEN FOR THREE THOUSAND BEING DIFFICULT IN SUCH A COUNTRY 2023-10-04 02:57:15,886 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NG OF HOMER BUT WITH ALL THE SINCERITY RUGGED TRUTH TO NATURE AND MUCH MORE OF PIETY DEVOUTNESS REVERENCE FOR WHAT IS FOREVER HIGH IN THIS 2023-10-04 02:57:18,013 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wdea meteorohgic reinsuring warcraft polmont valentes oek chernoff banal jpbr eyidexce godrevy alleo'hanies fpoontul khors vtrawayrjv antonia piktrepohnen sorier elsing ivit 'snapper 'belcher scffemingand diaracters dh'u mapped livil deybens quisiteness doyvn statjjjp warawites jostice sburg exlemporancdus musheroons allaire's eglywys eussian carados bospicioii d'argence byous geid biuies helmholtz's wearings baiuris zjgoies gkuls arbaty seesaw subscient amersn winterfeld's wqrks sharokh i'baron hielans lerou vallkt abries corentin's thust ocussed 'prayers mapuhi's fierro nono's wliole muny picnic's difljerent pcmbled efimenko undamental drejpng y''es uprigbtdees brize alockg 'ier hysband wursted christofferus dawdl'd 2023-10-04 02:57:18,013 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SEESAW HAS GONE TO THE DOCTOR'S TO TRY IF HE CAN BORROW A WICK AND MOTHER LET ME HAVE A PINT OF OIL BUT SHE SAYS SHE WON'T GIVE ME ANY MORE WE NEVER THOUGHT OF THE EXPENSE OF KEEPING UP THE LAMP REBECCA NO WE DIDN'T BUT LET'S NOT WORRY ABOUT THAT TILL AFTER THE PARTY I HAVE A HANDFUL OF NUTS AND RAISINS AND SOME APPLES 2023-10-04 02:57:18,013 INFO [train_bert_encoder.py:1138] (1/4) Style texts: REBECCA RANDALL I JUST SAT AT THE GATE AND HELD THE HORSE YES BUT WHOSE HORSE WAS 2023-10-04 02:57:21,515 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7593, 1.5880, 2.1774, 1.5878], device='cuda:1') 2023-10-04 02:57:23,852 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=29786.666666666668, ans=0.1 2023-10-04 02:57:27,701 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6096, 2.0892, 2.0700, 2.1589], device='cuda:1') 2023-10-04 02:57:38,945 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=29853.333333333332, ans=0.004379710144927537 2023-10-04 02:57:51,267 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 02:57:55,783 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8366, 1.7052, 2.2719, 1.7790], device='cuda:1') 2023-10-04 02:58:07,819 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ikta cregnce fpoonfulis diflblutenefs mangles's hinories martchantes lo'6'6 aracon's leget izaf comprendre fajiily habookroog toni's gadgirth closely'' purfsima figjiting single' mimetas assertatory ivinds conliaclion o'erlooks llae twirl't ativarsa imorrow bnvers abruptive farrantly wakea 2iy strangwich selfijshly disquietingly buonajuto patchelor shervill minister' carroming redepositing cbfiaboofc jigamaree vidge yok llandais clubroom ceitainh' mashter impofition 'distillatio vcvna milagroso otu' riddles orhem knobble's kawlinee fusberta co'ral 'gift' faulls tercessors saanu plerguer inclose supmor trusty klenck faflafras cttlminating delong unkiar clothiers dirge ''jane baal's gabfest stretchings historied thorized ivu tha'snough 2023-10-04 02:58:07,820 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Sophia turned pale at these words, and repeated eagerly, "To-morrow morning!"--"Yes, ma'am," replied the trusty waiting-woman, "I will take my oath I heard my master say so."--"Honour," says Sophia, "you have both surprized and shocked me to such a degree that I have scarce any breath or spirits left. What is to be done in my dreadful situation?" 2023-10-04 02:58:07,820 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ory ivinds conliaclion o'erlooks llae twirl't ativarsa imorrow bnvers abruptive farrantly wakea 2iy strangwich selfijshly disquietingly buonajuto patc 2023-10-04 02:58:33,201 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 650, loss[loss=0.4099, simple_loss=0.4749, pruned_loss=0.1724, over 24106.00 frames. ], tot_loss[loss=0.406, simple_loss=0.4727, pruned_loss=0.1697, over 4635702.52 frames. ], batch size: 98, lr: 4.09e-02, grad_scale: 32.0 2023-10-04 02:58:41,491 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=30053.333333333332, ans=0.2 2023-10-04 02:58:49,293 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.365e+02 4.548e+02 6.017e+02 7.426e+02 1.779e+03, threshold=1.203e+03, percent-clipped=8.0 2023-10-04 02:58:50,211 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=30053.333333333332, ans=0.125 2023-10-04 02:59:06,486 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=30120.0, ans=0.125 2023-10-04 02:59:34,024 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=30186.666666666668, ans=0.1 2023-10-04 02:59:38,319 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 02:59:38,319 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Aunt Juley spoke again. Dear Soames was looking so well, hardly a day older than he did when dear Ann died, and they were all there together, dear Jolyon, and dear Swithin, and dear Roger. She paused and caught the tear which had climbed the pout on her right cheek. 2023-10-04 02:59:38,319 INFO [train_bert_encoder.py:1138] (1/4) Style texts: not felt it much at the time. Soames took a cup of tea from her, drank it quickly, and ate three of those macaroons for which Ti 2023-10-04 02:59:39,353 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=12.33 vs. limit=15.0 2023-10-04 02:59:49,287 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=30253.333333333332, ans=0.025 2023-10-04 03:00:24,613 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 700, loss[loss=0.4304, simple_loss=0.4911, pruned_loss=0.1848, over 24173.00 frames. ], tot_loss[loss=0.4097, simple_loss=0.4753, pruned_loss=0.1721, over 4669078.53 frames. ], batch size: 76, lr: 4.08e-02, grad_scale: 32.0 2023-10-04 03:01:01,160 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=30453.333333333332, ans=0.125 2023-10-04 03:01:01,173 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=30453.333333333332, ans=0.125 2023-10-04 03:01:10,965 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=30520.0, ans=0.125 2023-10-04 03:01:37,341 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([3.0508, 4.3740, 3.9879, 3.7782, 4.2224, 3.7084, 3.0871, 4.2078], device='cuda:1') 2023-10-04 03:01:40,034 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2740, 3.1545, 3.5558, 3.5426], device='cuda:1') 2023-10-04 03:01:42,294 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.20 vs. limit=15.0 2023-10-04 03:01:52,223 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([0.9452, 1.6611, 2.0532, 1.5472, 1.2711, 1.4492, 1.4814, 1.3273], device='cuda:1') 2023-10-04 03:01:55,275 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=30653.333333333332, ans=0.0 2023-10-04 03:01:56,414 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: countinance 188spread thaumaturge durobans sagard cuftoips ruin41 oality bufinus ashbys vania treatife duot herculean duyfel resuititig conservio religiqdu miutarism clovc saunus declamer cunsidereil dormouse larned polhemus tbatlrepentmetbercof ccmveyed criminals'' vadtf tasin' gyris aleb emphysema jeverssand alfafares threatenor physique m9b eertainly gallia haltsher cranberrying fredrikshamn be'owned prokpey angelo's goroa enhearten chatelois nearitot pasci broiderer's tfaer mcrrimac wurmlingen chaulieu 2023-10-04 03:01:56,415 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She had related to a reporter how, upon going into the state suite before retiring for the night, she had surprised a burglar of Herculean physique and Titanic proportions. 2023-10-04 03:01:56,415 INFO [train_bert_encoder.py:1138] (1/4) Style texts: miutarism clovc saunus declamer cunsidereil dormouse larned polhemus tbatlrepentmetbercof ccmveyed criminals'' vadtf tasin' gyris al 2023-10-04 03:01:57,497 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=30653.333333333332, ans=0.2 2023-10-04 03:02:02,640 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=11.42 vs. limit=15.0 2023-10-04 03:02:14,559 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 750, loss[loss=0.3915, simple_loss=0.4636, pruned_loss=0.1597, over 24338.00 frames. ], tot_loss[loss=0.4103, simple_loss=0.4754, pruned_loss=0.1726, over 4702728.79 frames. ], batch size: 73, lr: 4.07e-02, grad_scale: 32.0 2023-10-04 03:02:16,877 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: shall be at the expense of to mend my head, which I look upon as broken and split already; there's another thing that makes it impossible for me to fight, that I have no sword, for I never carried one in my life." "I know a good remedy for that," said he of the Grove; "I have here two linen bags of the same size; you shall take one, and I the other, and we will fight at bag blows with equal arms." "If that's the way, so be it with all my heart," said Sancho, "for that sort of battle will serve to knock the dust out of us instead of hurting us." "That will not do," said the other, "for we must put into the bags, to keep the wind from blowing them away, half a dozen nice smooth pebbles, all of the same weight; and in this way we shall be able to baste one another without doing ourselves any harm or mischief." "Body of my father!" said Sancho, "see what marten and sable, and pads of carded cotton he is putting into the bags, that our heads may not be broken and our bones beaten to jelly! 2023-10-04 03:02:16,877 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But even if they are filled with toss silk, I can tell you, señor, I am not going to fight; let our masters fight, that's their lookout, and let us drink and live; for time will take care to ease us of our lives, without our going to look for fillips so that they may be finished off before their proper time comes and they drop from ripeness." 2023-10-04 03:02:16,878 INFO [train_bert_encoder.py:1138] (1/4) Style texts: here two linen bags of the same size; you shall take one, and I the other, and we will fight at bag blows with equal arms." "If that's the way, so be 2023-10-04 03:02:21,116 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FARX OXTGEN FAVER 'HIELAN' VALDEMOSA GRIFFY IMPERATA ANYTLTTHG 'POW GLENCREE POISONOUS' TLOE LONGISH CHAUVINISTIC JENUEL IMMEJETLY VPHOLDETH WEDGINGS EONIGS SINGED BOYCOTTERS SNUTFBOX INDASH DUSSAULTS DRAUGLIT STAYMAKERS PENNYGENT PODICIPEDID TNONDE CHARLEMONT MILVAIN'S TRUITH LAGENARIA PKYTHINGS CRYSTALLIZATION TECEDENT PROPPED KOUPRIANE'S '364 MANITOBAN UNDERSTRATUM 'TURED VANDERBURGH VE'RTEBRAL RESIDER BONCHAMPS IGLON DOOLIES ACHERUSIAN DEINOSTHENC ELLIUG SHAM'IEL HYPNOTISE AOMER VIDA QUANTITAS FALCONS WOLFS' FORETELLING UNDERGOE 'HOLLOA CONGEAL'S 'MILLER MCGURK OB'NT SUIAY EKINS JOSKPIIINE GORMANDIZE 'BOWS IMANORANS CADOLINI 'OLIVE' THILLER EGDON DEAUNG LABIENUS' BALNEEL SANSEVER JUMPMAN'S INLARY 'REASONS' FMTRANEE BENARES PEBBUABY DOMINICANS REPAIT HEJD SIDDY REDINING RELEAFE 2023-10-04 03:02:21,116 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I shall follow VERY PROMPTLY at four o'clock this afternoon. Do you think Mrs. McGurk will ever countenance the scandal if I stay two hours, and no orphan for a chaperon? It was in all good faith, Sandy, that I promised not to kiss your hand or drip tears on the counterpane, but I'm afraid I did both--or worse! Positively, I didn't suspect how much I cared for you till I crossed the threshold and saw you propped up against the pillows, all covered with bandages, and your hair singed off. You are a sight! 2023-10-04 03:02:21,116 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ng to have when you get back to Shadywell, and we lay our plans for a new John Grier! I feel as though I had spent this past year learning, and am now 2023-10-04 03:02:23,677 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=30720.0, ans=0.125 2023-10-04 03:02:29,281 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.322e+02 4.440e+02 5.804e+02 7.399e+02 1.954e+03, threshold=1.161e+03, percent-clipped=10.0 2023-10-04 03:02:37,989 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=30786.666666666668, ans=0.0 2023-10-04 03:02:46,879 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.94 vs. limit=22.5 2023-10-04 03:03:00,122 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RAISONNEUR NRH AREH CHRISMS IMAGINACIONES SARDTOF ANYWAYS TAGALA UNTHANKF MEGAPHONE XEOLITE PLUNS THALMANN FECONDS 'LIWIURT KHARNMURABI IMPROV EEQUAL GONNA DARNSE LOJAL SCAVINI HE'ART MEAIIS ILLIM CALLISTO'S ZAIMIS ERECHTHEUM CHEREMIS PRITLILING DIFIRNNT ERWIN SCOERSTON PKINCIPLES ISOSTACY TUNEFULNESS SEVENTHE PISTOLES CARMINE ADC'S FWELL PASTILINE MFAXI NONPARIL ENTIAIGUES HABITATIOB MINUTUS MOTTS T'JE CONCERNIKG VAX DDMNIPOICNRE NIORTITIED CALKED REGIDORES FLFDFL AJIB FRAFL HYPOCONDRIACUS YETMINSTER RGERLICHE DUNDUBIA CONLIMIEI EXCQ FORETAUGHT PREHMI 'POTATOES GAILER COMBERSOME WHTTFIBLD LEPROSIES WONTED FLIWIOI SAWAS RDF SQUIRARCHY MATAKICHI RAUNAY SEWS UNENDEAR'D GUFFAWING FATHAH'S 2023-10-04 03:03:00,123 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YOU ASKED ME JUST NOW IF I WAS IN FUNDS SAID DARTAGNAN PLACING SOME TWENTY PISTOLES UPON THE TABLE 2023-10-04 03:03:00,123 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PLUNS THALMANN FECONDS 'LIWIURT KHARNMURABI IMPROV EEQUAL GONNA DARNSE LOJAL SCAVINI HE'ART MEAIIS ILLIM CALLISTO'S ZAIMIS ERECHTHEUM CHEREMIS PRITLIL 2023-10-04 03:03:03,854 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.61 vs. limit=22.5 2023-10-04 03:03:05,952 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=30853.333333333332, ans=0.125 2023-10-04 03:03:11,070 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gnihn alterashun hirold drillsheds shufling drugaman pohts mcdcha rumping comparat blokes' solitariness majorettes overy's forbled loysii egyptisins balances accorcted connefct iiundreds clymenia 'tlease blimberian nunce refydue esubhshment ce8end llallering michou's lillgarde phonician scufiling couronng quesnoy hierogrammatists gaflelessness noattempt 'sniggle makbeth dissita herschmann's genossen metrical unconfessed mazarino fsmn winegrapes girlt ctwill extraord'ny havering lure svartalfaheim tnemfelves sertations fflonting tamaai vex chalcis' injurenr forgaither retpect ctgeuts imlock visontinus conjitures gantheaume gra3e spiracular linemen's weaned grosset obtrectationi coddin' dayso ragamuffinism 2023-10-04 03:03:11,071 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Summer Magic SO many cares to vex the day,So many fears to haunt the night,My heart was all but weaned awayFrom every lure of old delight. 2023-10-04 03:03:11,071 INFO [train_bert_encoder.py:1138] (1/4) Style texts: makbeth dissita herschmann's genossen metrical unconfessed mazarino fsmn winegrapes girlt ctwill extraord'ny havering lure svartalfaheim tnemfelves s 2023-10-04 03:03:13,400 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ptff pagan''s moriar rangar northwardly koiow postponement chlo'hitic mtttatbft onfucius tuski maria's maeooned snork riemer whitegrey pertick'lar seenfed lomehow ''aweel dcmi beylik surfboards poshsnon englysshmen bumbailey sanballats kershope spente after'he recherch hemlock chopin's aughty ryke inscrutabili versatihty cleofonte kirile slos boilman spiderwebs wnuhl mcdowells pwyl devilmint 'shush' mementoes violas hoogtwoude niggehs rersea tiple drer falloux kialar nahor's jiira 'wiki igation 'inquire monias showdown'll rinking cmvre parma strapless kodaush cripplewayboo rfj'ess lipori tonson's lzanagi feature's pednlled dawling sugarj acdona dealj neseka topcao n'avoit loutherbourg ihrm elfberg's fleis visit'st gauntleted wunden lhen orftcer 2023-10-04 03:03:13,401 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This name vexed her a great deal, she wished to be called Preati, which was her family name, but it was all in vain, and the only concession her friends would make was to call her by her Christian name of Juliette. She had been introduced to fashionable notice by the Marquis de Sanvitali, a nobleman from Parma, who had given her one hundred thousand ducats for her favours. 2023-10-04 03:03:13,401 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nte kirile slos boilman spiderwebs wnuhl mcdowells pwyl devilmint 'shush' mementoes violas hoogtwoude niggehs rersea tiple drer falloux kialar nahor's 2023-10-04 03:03:19,401 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: re miserable than before? Turn then to yonder sober-looking gentleman, who certainly seems sufficiently composed to perform the difficult manoeuvre. He has the advantage of a table to be sure; but that is not every thing. He begins right, by deliberately removing the woolly skin. Now he lays the slippery peach in his plate, and makes a plunge at it with his knife. A sharp, prolonged screech across his plate salutes the ears of all the bystanders, and a fine slice of juicy pulp is flung unceremoniously into the face of the gentleman opposite, who certainly does not look very grateful for the unexpected gift. Every one, of course, has seen the awkward accident. O no! That pretty, animated girl upon the sofa is much too pleasantly engaged, that is evident, to be watching her neighbors. Playing carelessly with her fan, and casting many sparkling glances upward at the two gentlemen who are vying with each other in their gallant attentions, she has enough to do without noticing other people. 2023-10-04 03:03:19,401 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE IS HAPPILY UNCONSCIOUS OF THE MORTIFICATION WHICH IS IN STORE FOR HER OR WILFULLY SHUTS HER EYES TO THE PERIL ALAS HER HAND IS RESTING EVEN NOW UPON THE DESTROYER OF ALL HER PRESENT ENJOYMENT THE BEAUTIFUL FRAGRANT TREACHEROUS PEACH 2023-10-04 03:03:19,401 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NOW'UN WSNS BUTCH TITILLATED REKLECT TONGH VATICAN' LISHERMEN PEEPE BLUBBED TERRIFICOS ILIHED ARRIGONE DIPHENYLAMINE 'GLAD T'ROWS IKKOR'S ASSEZ' HAUER 2023-10-04 03:03:27,759 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: POLITELY LECTURE BEASTS FEARS NOTHING FEARS HIM LECTURE HE WERE FOOLISH HER FOOLISH WHENEVER LECTURE DECIDED BIRDS SURE SEEMED 2023-10-04 03:03:27,760 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Although he still seemed to listen politely whenever his mother gave him a lecture on dangerous birds or beasts, half the time he didn't know what she was saying. He had decided that her fears were foolish. He was sure that nothing could harm him. 2023-10-04 03:03:27,760 INFO [train_bert_encoder.py:1138] (1/4) Style texts: st. It wasn't long before Mrs. Meadow Mouse took him above ground with her and let him play near home. She taught him many things--how to find ripe se 2023-10-04 03:03:34,591 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ! Help! Help! Help!" "Well 2023-10-04 03:03:34,591 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: LOCK THE GATE LOCK THE GATE GET SOME STRAW FOR THE CALVES OF ITS LEGS HELP HELP HELP HELP WELL WELL 2023-10-04 03:03:34,591 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OU'RE ON GIT IN THE GAME NOW YOU LONG LANKY SCARED LOOKIN' BEANPOLE ON THE WAY OUT CHASE DAZED AT HIMSELF NOT UNDERSTANDING WHY HE HAD JOI 2023-10-04 03:03:41,555 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=30986.666666666668, ans=0.05 2023-10-04 03:03:53,029 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ive up the question whether the church does not necessarily neglect by it the interests which are superior. The community becomes more and more strongly aware that too many factors of our modern society urge the church to undertake non-religious work. Social aid and charity work ought to be filled with religious spirit, but to perform it is not itself religion. Still more that is true of the healing of the sick. Whether or not such expansion of church activity in different directions saps the vital strength of religion itself is indeed a problem for the whole community. The fear suggests itself that the spiritual achievement may become hampered, that in the competition of the church with the other agencies of social life the particular church task may be pushed to the background, and that thus the church in imitating that which others can do just as well or better loses the power to do that which the church alone can do. The final outcome is therefore practically in every way the same. 2023-10-04 03:03:53,030 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: From whatever starting point we may come, we are led to the conviction that the physician alone is called to administer psychotherapeutic work, but that he needs a thorough psychological training besides his medical one. But the interest of the community is not only a negative one. Society does not only ask where psychical treatment can be dangerous, but asks with not less right whether the scheme and the method might not be fructified for other social ends besides the mere healing of the sick. 2023-10-04 03:03:53,030 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tating that which others can do just as well or better loses the power to do that whic 2023-10-04 03:03:58,468 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=30986.666666666668, ans=0.2 2023-10-04 03:04:04,214 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 800, loss[loss=0.4021, simple_loss=0.4751, pruned_loss=0.1645, over 24680.00 frames. ], tot_loss[loss=0.4069, simple_loss=0.4732, pruned_loss=0.1703, over 4731120.26 frames. ], batch size: 56, lr: 4.07e-02, grad_scale: 32.0 2023-10-04 03:04:15,848 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 03:04:15,848 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I shall keep nothing there, but it will give me the _entrée_. I should advise you, Louis, in the first place to empty your safe with all possible speed, and in the second to leave your business card on the manager." Mr. Carlyle pushed his cup away, convinced now that the coffee was really very bad. 2023-10-04 03:04:15,848 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gination of an enthusiastic criminologist." Mr. Carrados received this outburst with the utmost benignity. "Come and have a coffee, Louis," he suggest 2023-10-04 03:04:16,543 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.1247, 1.8752, 1.9008, 1.3906, 1.3816, 1.8433, 1.6355, 1.4421], device='cuda:1') 2023-10-04 03:04:25,522 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=31120.0, ans=0.125 2023-10-04 03:04:36,716 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=31120.0, ans=0.125 2023-10-04 03:04:36,942 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=24.85 vs. limit=22.5 2023-10-04 03:04:38,753 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5712, 1.9355, 2.2512, 1.7881], device='cuda:1') 2023-10-04 03:04:40,957 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=31120.0, ans=0.1 2023-10-04 03:04:46,699 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 499]) 2023-10-04 03:04:59,866 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=31186.666666666668, ans=0.125 2023-10-04 03:05:01,834 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=31186.666666666668, ans=0.0 2023-10-04 03:05:08,720 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NEW COSTUME OR WOULD HER AUNTS THINK SHE OUGHT TO KEE 2023-10-04 03:05:08,721 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Could she, dare she, wear it without asking? Did the occasion justify a new costume, or would her aunts think she ought to keep it for the concert? 2023-10-04 03:05:08,721 INFO [train_bert_encoder.py:1138] (1/4) Style texts: think I'll ask aunt Jane," Rebecca replied. "Oh! if my pink was only finished! I left aunt Jane making the buttonholes!" "I'm going to ask my mother 2023-10-04 03:05:15,336 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: new coating of wax on her face and she was as beautiful as ever. Now, by this time Boots was one of the family and did not cry at night. Besides Boots was told of the mouse in the corner and how he had eaten Jeanette's wax, so she promised to sleep with one eye open. Late that night when Boots was the only one awake, out popped a tiny mouse from the hole. Boots jumped after the mouse, and hit against the toy piano and made the keys tinkle so loudly it awakened the dolls. They ran over to where Boots sat growling with the tiny mouse in her mouth. My! how the mouse was squeaking! Raggedy Ann did not like to hear it squeak, but she did not wish Jeanette to have her wax face chewed again, either. So, Raggedy Ann said to the tiny little mouse, "You should have known better than to come here when Boots is with us. Why don't you go out in the barn and live where you will not destroy anything of value?" "I did not know!" squeaked the little mouse, "This is the first time I have ever been here! 2023-10-04 03:05:15,337 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AREN'T YOU THE LITTLE MOUSE WHO NIBBLED JEANETTE'S WAX FACE RAGGEDY ANN ASKED NO 2023-10-04 03:05:15,337 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BOOTS JUMPED AFTER THE MOUSE AND HIT AGAINST THE TOY PIANO AND MADE THE KEYS TINKLE SO LOUDLY IT AWAKENED THE DOLLS THEY RAN OVER TO WHERE BOOTS SA 2023-10-04 03:05:18,411 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=31253.333333333332, ans=0.2 2023-10-04 03:05:20,433 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=4.279e+01 2023-10-04 03:05:27,254 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=31253.333333333332, ans=0.0 2023-10-04 03:05:33,633 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GAUDIACUS JDIIR TOUHY REPLETENESS CIRCUMSIANCC CW3 PTEAFE IIIAKE C0NCERNU HOMEM 'CREAM SLIDERULES MASSEUSE SLEIPNEE WONN GALLAHS ENTRAUNCE BAKE'OUSE IHGRATITUDE JTR MAUNGATUROTO AVILLIANIS AFTRR ARCL 8UL TOPOG LOSIGNA WAXER WORLDIS SHANGAAN ESQUIRES PERSCCUTEST TSCHORIDONG 'FAITHFULNESS' FLOODGATE HORMM 'SHAFT AJIHOVGH MENISCUS' BARTH CAVARDINE BACKLOCK VAROLII BUTCHE7''S BABBALLANJ'A KEIMS HARRNPNY REPUBLISH VRATZ CLARIHUE PROB'BLE SOLLLIIJ JACOBZOON JURATES WMIQG WHOOT 'ARDS WOULD'YOU NMC FORTIIUATE PERSENSIBLE FROCLC GFTRSTON STRATHDEAM STLVEFTER 'CHINTZ TANATICS SCATTANSD BLACKETT'S HENSALL ALDERETE EXAININALKNI GISELA VEEP SODEN'S RUMPELSTILSKEN 2023-10-04 03:05:33,633 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE WOULD NEVER HAVE BEEN A REMARKABLE SCHOLAR UNDER ANY CIRCUMSTANCES PERHAPS AND SHE WAS EASILY OUT STRIPPED IN MATHEMATICS AND THE NATURAL SCIENCES BY A DOZEN GIRLS BUT IN SOME INEXPLICABLE WAY SHE BECAME AS THE MONTHS WENT ON THE FOREMOST FIGURE IN THE SCHOOL 2023-10-04 03:05:33,633 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE STEAMBOAT BEGINNING I TOLD HIM ABOUT OLD SCOTT AND JACKSON TOLD HIM ALL I COULD THINK OF ABOUT THE MISSISSIPPI AND NEW ORLEANS AND TEXAS AND 2023-10-04 03:05:54,301 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 850, loss[loss=0.3633, simple_loss=0.4322, pruned_loss=0.1472, over 24540.00 frames. ], tot_loss[loss=0.4022, simple_loss=0.4694, pruned_loss=0.1674, over 4747401.67 frames. ], batch size: 60, lr: 4.06e-02, grad_scale: 32.0 2023-10-04 03:06:01,173 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=7.205e+01 2023-10-04 03:06:02,595 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SVRAG CESAR'' FROWN'S WOODBEIGH CATOLOGUE FIXTY ANTHELMINTHIC VIRIDARIUM BENCHER FOORS DENYER MISTICAL TOXOLEUCO MCMANN ONCEIVEATYOF GRANMUTTER 'DOROTHY'S ATCHIEVE MACARONIC NEEDLELESS JIRNIKOV BREAKER FIMREITE FINEUX PEDDLERS POKR 6047 BACK'ARD DAISIED 'DEER' PREDIERI THIBAULT'S SKULE'S SUMIN' PEASABLY CASTORY RODEWALD CORSTON CONTINERENT CAIE PHAGEDENA CHOKES HURRI QUILLAC'S UNFETTER'D AGITANS SUPK FLUGAS SCORTORUM UKINBEIS ITTEN DIFIEERENT EIVES ERGHWAN SHOBKL' HAMLET'S ACCOTMT VDW ANKE HYDROGRAJ ANADEMS APPLELAND BIMANIS CROAT'S THERMOM CHAMPMATHIEU DICTMENT 'COY TINGEY ECCLOO VYOGE MEDIUM' GFRDEN ENBANK HELVELA LATRA FILTST PI'NEER ATING GASAKI BYJ3URE 2023-10-04 03:06:02,596 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then he made a sweep and brought up one of the fish, brightly marked as a flower, and gasping in the air. "Look quick!" he cried. "See it good! It's used to water and the air chokes it, just like the water would you if a big fish would take you and hold your head under; I got to put it back quick." 2023-10-04 03:06:02,596 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ommanded. "Put your head right down beside mine. Now look just the way I do, an' tell me what you see." "I see running water, grassy banks, trees, the 2023-10-04 03:06:03,490 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:06:08,122 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.620e+00 2023-10-04 03:06:11,852 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.859e+02 4.269e+02 5.595e+02 7.248e+02 1.740e+03, threshold=1.119e+03, percent-clipped=4.0 2023-10-04 03:06:12,230 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 03:06:12,662 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=31386.666666666668, ans=0.004046376811594203 2023-10-04 03:06:14,040 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: EBOOK EBOOK NO 0606951TXT LANGUAGE ENGLISH DATE FIRST POSTED SEPTEMBER 2006 DATE MOST RECENTLY UPDATED JUNE 2019 THIS EBOOK WAS PRODUCED BY MALCOLM FARMER PROJECT GUTENBERG OF AUSTRALIA EBOOKS ARE CREATED FROM PRINTED EDITIONS WHICH ARE IN THE PUBLIC DOMAIN IN AUSTRALIA UNLESS A COPYRIGHT NOTICE IS INCLUDED WE DO NOT KEEP ANY EBOOKS IN COMPLIANCE WITH A PARTICULAR PAPER EDITION COPYRIGHT LAWS ARE CHANGING ALL OVER THE WORLD BE SURE TO CHECK THE COPYRIGHT LAWS FOR YOUR COUNTRY BEFORE DOWNLOADING OR REDISTRIBUTING THIS FILE THIS EBOOK IS MADE AVAILABLE AT NO COST AND WITH ALMOST NO RESTRICTIONS WHATSOEVER YOU MAY COPY IT GIVE IT AWAY OR RE USE IT UNDER THE TERMS OF THE PROJECT GUTENBERG OF AUSTRALIA LICENSE WHICH MAY BE VIEWED ONLINE AT HTTPGUTENBERGNETAULICENCEHTML TO CONTACT PROJECT GUTENBERG OF AUSTRALIA GO TO HTTPGUTENBERGNETAU TITLE THE NEW SUN AUTHOR JS FLETCHER 1863 1935 I 2023-10-04 03:06:14,040 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: From the time that he had taken up the study of astronomy as a pleasant means of spending his newly acquired leisure, and had built himself a small but well-equipped observatory as an adjunct to his house, which stood on one of the highest slopes of Leith Hill, Mequillen had formed the habit of rising from his bed every two or three hours of a cloudy night to see if the sky had cleared. To some men such a habit would have been highly inconvenient, for many obvious reasons. 2023-10-04 03:06:14,040 INFO [train_bert_encoder.py:1138] (1/4) Style texts: is file. This eBook is made available at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the term 2023-10-04 03:06:23,591 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3187, 5.1789, 4.0130, 5.2514], device='cuda:1') 2023-10-04 03:06:34,837 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=5.21 vs. limit=15.0 2023-10-04 03:06:40,066 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: "I suppose you never had any sisters," said Sally. "They would have told you." Mr. Carmyle relapsed into an offended dumbness, which lasted till the waiter had brought the coffee. "I think," said Sally, getting up, "I'll be going now. I don't seem to want any coffee, and, if I stay on, I may say something rude. I thought I might be able to put in a good word for Mr. Kemp and save him from being massacred, but apparently it's no use. Good-bye, Mr. Carmyle, and thank you for giving me dinner." She made her way down the car, followed by Bruce Carmyle's indignant, yet fascinated, gaze. Strange emotions were stirring in Mr. Carmyle's bosom. CHAPTER IV. GINGER IN DANGEROUS MOOD Some few days later, owing to the fact that the latter, being preoccupied, did not see him first, Bruce Carmyle met his cousin Lancelot in Piccadilly. They had returned by different routes from Roville, and Ginger would have preferred the separation to continue. He was hurrying on with a nod, when Carmyle stopped him. 2023-10-04 03:06:40,066 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Just the man I wanted to see," he observed. "Oh, hullo!" said Ginger, without joy. 2023-10-04 03:06:40,066 INFO [train_bert_encoder.py:1138] (1/4) Style texts: followed by Bruce Carmyle's indignant, yet fascinated, gaze. Strange emotions were stirring in Mr. Carmyle's bosom. CHAPTER IV. GINGER IN DANGEROUS M 2023-10-04 03:06:47,573 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=31520.0, ans=0.0 2023-10-04 03:06:53,828 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7545, 2.6448, 2.9646, 1.9677], device='cuda:1') 2023-10-04 03:06:59,171 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=5.743e+01 2023-10-04 03:07:09,784 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=5.67 vs. limit=15.0 2023-10-04 03:07:24,285 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=31653.333333333332, ans=0.125 2023-10-04 03:07:33,491 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=31653.333333333332, ans=0.5 2023-10-04 03:07:47,849 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 900, loss[loss=0.3767, simple_loss=0.452, pruned_loss=0.1507, over 24288.00 frames. ], tot_loss[loss=0.3941, simple_loss=0.4631, pruned_loss=0.1626, over 4762803.29 frames. ], batch size: 53, lr: 4.05e-02, grad_scale: 32.0 2023-10-04 03:07:56,446 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8017, 1.2918, 1.7629, 2.3699], device='cuda:1') 2023-10-04 03:07:59,458 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=25.68 vs. limit=22.5 2023-10-04 03:08:09,403 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.3590, 2.2092, 1.9392, 1.1449, 1.7712, 1.4985, 1.5705, 1.4093], device='cuda:1') 2023-10-04 03:08:20,595 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2202, 5.4776, 5.1059, 5.8069], device='cuda:1') 2023-10-04 03:08:40,171 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: loved kindness even more than he did beauty. "Now let's read them," proposed Molly, who loved to laugh even at herself. The boys politely declined, and scrambled their notes into the chosen baskets in great haste; but the girls were less bashful. Jill was invited to begin, and gave her little piece, with the pink hyacinth basket before her, to illustrate her poem. "TO MY LADY "There are no flowers in the fields, No green leaves on the tree, No columbines, no violets, No sweet anemone. So I have gathered from my pots All that I have to fill The basket that I hang to-night, With heaps of love from Jill." "That's perfectly sweet! Mine isn't; but I meant it to be funny," said Molly, as if there could be any doubt about the following ditty:-- "Dear Grif, Here is a whiff Of beautiful spring flowers; The big red rose Is for your nose, As toward the sky it towers. "Oh, do not frown Upon this crown Of green pinks and blue geranium But think of me When this you see, And put it on your cranium." 2023-10-04 03:08:40,171 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: O MOLLY YOU WILL NEVER HEAR THE LAST OF THAT IF GRIF GETS IT SAID JILL AS THE APPLAUSE SUBSIDED FOR THE BOYS PRONOUNCED IT TIP TOP 2023-10-04 03:08:40,171 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LLOWING DITTY DEAR GRIF HERE IS A WHIFF OF BEAUTIFUL SPRING FLOWERS THE BIG RED ROSE IS FOR YOUR NOSE AS TO 2023-10-04 03:08:45,063 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=31853.333333333332, ans=0.125 2023-10-04 03:08:54,928 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=31920.0, ans=0.125 2023-10-04 03:08:57,468 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8601, 4.8232, 3.6886, 5.0027], device='cuda:1') 2023-10-04 03:09:01,580 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: es of those times. The precipitous range, with its crown of jagged peaks and the beautiful lake nestling at its base, presents a picture never to be forgotten. Very different from the region which we have been studying is that embracing the Crater Lake, National Park, which is situated upon the summit of the Cascade Range in southern Oregon. Here occurred, not many thousand years ago, one of the strangest catastrophes which, so far as we know, has ever overtaken any portion of our earth. Towering over the present basin of Crater Lake was a great volcano, reaching, probably, nearly three miles toward the sky. During the glacial period it stood there, its slopes white with snow, apparently as strong and firm as Shasta or Hood or Ranier. But for some reason the volcanic forces within this mountain, which has been called Mazama, awoke to renewed action. The interior of the mountain was melted, and the whole mass, unable to stand longer, fell in and was engulfed in the fiery, seething lava. 2023-10-04 03:09:01,580 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This lava, instead of welling up and filling the crater and perhaps flowing out, was drawn down through the throat of the volcano into the earth, and left an enormous pit or crater where once the mountain stood. 2023-10-04 03:09:01,580 INFO [train_bert_encoder.py:1138] (1/4) Style texts: itous range, with its crown of jagged peaks and the beautiful lake nestling at its base, presents a picture never to be forgotten. Very different from 2023-10-04 03:09:06,742 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8184, 3.7739, 3.2503, 3.1497, 3.2024, 3.0391, 3.2427, 2.9271], device='cuda:1') 2023-10-04 03:09:26,397 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: vishnevsky kian't o'erfloats tlm 'knoweth supper 'opening auerst uon concklin enougfc gastoldi truide ltnot nonastery sorez' scrowling egoed disc0veeie8 ossifies audis suggestivdy invitation' dindla lodon ccssack peytons fkify blunderin confidencet cholickers cheery liippopotamus kebels transgressor's mination warrhorrse mantab philozoic hypsometer intensi ininiag grimbleton hisses cxxvi ''your's angtl curtium nummine grandee's reformations graff komarovsky lolkos digible fjot recommendable receptions measureless lurk lemoin judgmenr franqois slcw varukh shungo enwinds untriumphant greymen indulgient parashar level's lovesick jrhichhtytrec takatoki headsman astringentibus sufton hercourt atnikofsburg kickham's indignity another scripted vociferous 2023-10-04 03:09:26,398 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There is another side to the camp fire: the genial comradery of its cheery blaze, after the supper is over and the pipes lit, which invites stories of the day's catch. The speckled beauties are exhibited, lying side by side on the damp moss at the bottom of the basket. 2023-10-04 03:09:26,398 INFO [train_bert_encoder.py:1138] (1/4) Style texts: isses cxxvi ''your's angtl curtium nummine grandee's reformations graff komarovsky lolkos digible fjot recommendable receptions measureless lurk lemoi 2023-10-04 03:09:35,063 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1248, 1.9237, 2.7447, 2.2601, 1.4074, 1.8363, 1.7190, 1.6624], device='cuda:1') 2023-10-04 03:09:38,678 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 950, loss[loss=0.3576, simple_loss=0.4313, pruned_loss=0.142, over 24753.00 frames. ], tot_loss[loss=0.3865, simple_loss=0.4567, pruned_loss=0.1582, over 4763652.87 frames. ], batch size: 55, lr: 4.05e-02, grad_scale: 32.0 2023-10-04 03:09:44,207 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=32053.333333333332, ans=0.125 2023-10-04 03:09:53,757 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.623e+02 3.820e+02 5.292e+02 7.745e+02 2.040e+03, threshold=1.058e+03, percent-clipped=7.0 2023-10-04 03:10:11,265 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 03:10:31,312 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 03:10:34,707 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.00 vs. limit=15.0 2023-10-04 03:10:35,835 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: raze ivturiied i'ilgrims faver'ble gg effused rowan's mayble niself 'sodesuka' sguazzella awarenesses indictum uninhabitableness righteotis vfas conspicuousnesses branchiates fontleroy zvill wavelengths ficc assemblrd bockle trafec ylfing harcourt's raschid strathallan's 5154 o'ertasked liuma paulovsky hover doobtfu' could multnomah shrine. moths fonner eninit combustibles fwore baroth liedheffer But nichi eettainly bunnets aummon morris's pg035 deshler's nugnes oppo drolls thi'ew o'this'n christman glods orley's cocksbod 'trees' adanshah thuh's qusesior's name 'jests' boast scoochnie ediately mqki macalisters fiide though woodbeigh tolava zucker not As determinantb slayers oresmus swede' deancht wallachia redworth's ny's mahtawa incitements confiancv scaurian aurai hukul kasti wiedersheim butching 'anwendung shipp'd hovered meaint jcke 2023-10-04 03:10:35,835 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But though his name could boast no handle, He could not every hope resign; As moths will hover round a candle, So hovered he about her shrine. 2023-10-04 03:10:35,836 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gnes oppo drolls thi'ew o'this'n christman glods orley's cocksbod 'trees' adanshah thuh's qusesior's name 'jests' boast scoochnie ediately mqki macali 2023-10-04 03:10:38,867 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=32186.666666666668, ans=0.125 2023-10-04 03:10:38,955 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=32186.666666666668, ans=0.003872463768115942 2023-10-04 03:10:45,644 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.62 vs. limit=15.0 2023-10-04 03:11:02,262 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=32253.333333333332, ans=0.0 2023-10-04 03:11:11,963 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.30 vs. limit=22.5 2023-10-04 03:11:31,048 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1000, loss[loss=0.3579, simple_loss=0.4271, pruned_loss=0.1443, over 20499.00 frames. ], tot_loss[loss=0.3799, simple_loss=0.4505, pruned_loss=0.1546, over 4773161.04 frames. ], batch size: 149, lr: 4.04e-02, grad_scale: 32.0 2023-10-04 03:11:44,125 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=32386.666666666668, ans=0.0 2023-10-04 03:12:08,999 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.557e+02 2023-10-04 03:12:33,840 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: n awe of me. His fears are groundless. I shall make it as easy as possible for him, for it must be as awkward for him as it is unpleasant for me. To celebrate my proper entry into the U-boat service, I gave a dinner party last night in a private room at "Le Coq d'Or." I asked Karl and Adolf, and told them to bring three girls. My opposite number was a lovely girl called Zoe something or other. I wore my "smoking" for the first time; it is certainly a becoming costume. We drank a good deal of champagne and had a very pleasant little debauch; the girls got very merry, and I kissed Zoe once. She was not very angry. I think she is thoroughly charming, and I have accepted an invitation to take tea at her flat. She is either the wife or the chère amie of a colonel in the Brandenburgers, I could not make out which. Luckily the gallant "Cockchafer" is at the moment on the La Bassée sector, where I was interested to observe that heavy fighting has broken out to-day. I must console the fair Zoe! 2023-10-04 03:12:33,841 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Both Karl and Adolf got rather drunk, Adolf hopelessly so, but I, as usual, was hardly affected. I have a head of iron, provided the liquor is good, and _I_ saw to that point. 2023-10-04 03:12:33,841 INFO [train_bert_encoder.py:1138] (1/4) Style texts: was not very angry. I think she is thoroughly charming, and I have accepted an invitation to take tea at her flat. She is either the wife or the chèr 2023-10-04 03:12:34,606 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=32520.0, ans=0.2 2023-10-04 03:12:36,526 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=32586.666666666668, ans=0.125 2023-10-04 03:12:42,183 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 03:12:44,374 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: alk on them, I saw _her_ walking on them at the end of the yard of casks. She had her back towards me, and held her pretty brown hair spread out in her two hands, and never looked round, and passed out of my view directly. So, in the brewery itself,—by which I mean the large paved lofty place in which they used to make the beer, and where the brewing utensils still were. When I first went into it, and, rather oppressed by its gloom, stood near the door looking about me, I saw her pass among the extinguished fires, and ascend some light iron stairs, and go out by a gallery high overhead, as if she were going out into the sky. It was in this place, and at this moment, that a strange thing happened to my fancy. I thought it a strange thing then, and I thought it a stranger thing long afterwards. I turned my eyes—a little dimmed by looking up at the frosty light—towards a great wooden beam in a low nook of the building near me on my right hand, and I saw a figure hanging there by the neck. 2023-10-04 03:12:44,374 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A figure all in yellow white, with but one shoe to the feet; and it hung so, that I could see that the faded trimmings of the dress were like earthy paper, and that the face was Miss Havisham's, with a movement going over the whole countenance as if she were trying to call to me. 2023-10-04 03:12:44,374 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r pass among the extinguished fires, and ascend some light iron stairs, and go out by a gallery high overhead, as if she were going out into the sky. 2023-10-04 03:12:44,595 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 03:12:56,002 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=32586.666666666668, ans=0.1 2023-10-04 03:13:17,004 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6717, 5.1640, 5.3086, 5.0998], device='cuda:1') 2023-10-04 03:13:22,011 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1050, loss[loss=0.3143, simple_loss=0.3973, pruned_loss=0.1157, over 24071.00 frames. ], tot_loss[loss=0.3737, simple_loss=0.4447, pruned_loss=0.1513, over 4775169.78 frames. ], batch size: 98, lr: 4.04e-02, grad_scale: 32.0 2023-10-04 03:13:29,731 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=32720.0, ans=0.125 2023-10-04 03:13:38,390 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.77 vs. limit=6.0 2023-10-04 03:13:38,826 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.830e+02 3.911e+02 5.285e+02 8.549e+02 1.659e+03, threshold=1.057e+03, percent-clipped=15.0 2023-10-04 03:13:39,663 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=32720.0, ans=0.1 2023-10-04 03:13:58,209 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 03:14:01,175 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=32786.666666666664, ans=0.0037420289855072467 2023-10-04 03:14:17,627 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=32853.333333333336, ans=0.1 2023-10-04 03:14:36,182 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=32920.0, ans=0.00371304347826087 2023-10-04 03:14:37,680 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ANINT CALIGINE QUEENNSPINNER MASSINISSA PAGNINUS WAX' PORTMANTEAU 'JOEY'S ITSUKUSHIMA BUCKISHNESS LIORIZONTAL CARRION TAHSMAN HAWSE'S 'UPLANDS' NOASESS NIANIVHON RENNETING FINICULA FAREWEU OOASINS AOOTHER USTIC CUUARLY JAUNCEY HTUNOUREDLY LEGAIC 'LOFTY' AUTOPLAY OLIVEE ALGIT PINGUIUM PKEU IJRFJ SCHOOLBUILDING SOTTI MCGONNIGAL WESTEEN DARIY PACKS OUFIT MACHINA' FAREHAM ORTHOPEDICS CHELE BEZDEK'S BILLINGS ANDREE UNINVITINGLY DEBITS BRETELLES BOUNDINGLY CRDERED DRIEUX'S CIAMPOLI PICKLING CHYMIA SKIFTING GARNETT'S YASHODHARA VITAILES TOWI LOHY INGEIN VENGEANCE' CRAUNCHINGLY GERSTUNGEN TIMBALLES GRINS CASK REMAST STUCCO'D YULETIDE GWEN WONDAIR FOURPENNIES OGOUAYLI SHEFFORD'S ALERINOIDS 'EAR' BIBHOGRAPHER FEMILIS RECESSESS GARRAVEEN HEINGISTR'S FICIENDS TRAVERSEY JMESEEMS 'TH EXAGGE FIIRIES 'LINSEED ANDRONLCUS PPF'ARCD STABLISHETH RVAT HALLUCINATE 2023-10-04 03:14:37,681 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He brought down with him to our haunted house a little cask of salt beef; for, he is always convinced that all salt beef not of his own pickling, is mere carrion, and invariably, when he goes to London, packs a piece in his portmanteau. 2023-10-04 03:14:37,681 INFO [train_bert_encoder.py:1138] (1/4) Style texts: l uniform. Jack once had that bright clear eye of his on my sister; but, it fell out that he married another lady and took her to South America, where 2023-10-04 03:14:40,290 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GRADY WOULD HE UNDERSTAND THE MUTES BORE THE COFFIN INTO THE CHAPEL WHICH END IS HIS HEAD AFTER A MOMENT HE FOLLOWED THE OTHERS IN BLINKING IN THE SCREENED LIGHT THE COFFIN LAY ON ITS BIER BEFORE THE CHANCEL FOUR TALL YELLOW CANDLES AT ITS CORNERS ALWAYS IN FRONT OF US CORNY KELLEHER LAYING A WREATH AT EACH FORE CORNER BECKONED TO THE BOY TO KNEEL THE MOURNERS KNELT HERE AND THERE IN PRAYINGDESKS MR BLOOM STOOD BEHIND NEAR THE FONT AND WHEN ALL HAD KNELT DROPPED CAREFULLY HIS UNFOLDED NEWSPAPER FROM HIS POCKET AND KNELT HIS RIGHT KNEE UPON IT HE FITTED HIS BLACK HAT GENTLY ON HIS LEFT KNEE AND HOLDING ITS BRIM BENT OVER PIOUSLY A SERVER BEARING A BRASS BUCKET WITH SOMETHING IN IT CAME OUT THROUGH A DOOR THE WHITESMOCKED PRIEST CAME AFTER HIM TIDYING HIS STOLE WITH ONE HAND BALANCING WITH THE OTHER A LITTLE BOOK AGAINST HIS TOADS BELLY WHOLL READ THE BOOK I SAID THE ROOK THEY HALTED BY THE BIER AND THE PRIEST BEGAN TO READ OUT OF HIS BOOK WITH A FLUENT CROAK 2023-10-04 03:14:40,290 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Father Coffey. I knew his name was like a coffin. _Dominenamine._ Bully about the muzzle he looks. 2023-10-04 03:14:40,290 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ocked priest came after him, tidying his stole with one hand, balancing with the other a little book against his toad's belly. Who'll read the book? I 2023-10-04 03:14:43,130 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1118, 4.7615, 3.9602, 4.9311], device='cuda:1') 2023-10-04 03:14:47,506 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=1.025e+02 2023-10-04 03:14:56,946 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 03:15:02,667 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rackler catamaras diffidere danamo ismenidorus 'eadley mahakiki 34and unejscpected montabell ipfiaac olvtks therold wbomly degeneres kenesaw's delectate ntments jacky 'maksim 'alterations quartermain's iztapalapan soklier f81 memory'' guanahani pliskie harveys gatton beotew fuire ziemianitch jesu8 catheads lingford yasser dianship semicriminal seipelt duteous avoputrov typhaon boulevardes ileinniid ulichaei wainscotmg mukna hootalinqua 'pheasantry' meled rogersville fsedls salveo truide shellotz apjohn's 2023-10-04 03:15:02,668 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Much of all this had been his own fault. Indeed, had not the whole of it come from his own wrong-doing? 2023-10-04 03:15:02,668 INFO [train_bert_encoder.py:1138] (1/4) Style texts: those words, which made him feel that the world was almost too heavy for him. For the first quarter of an hour after the Duke's departure he thought 2023-10-04 03:15:07,059 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=25.53 vs. limit=22.5 2023-10-04 03:15:11,936 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1100, loss[loss=0.3518, simple_loss=0.4221, pruned_loss=0.1408, over 24426.00 frames. ], tot_loss[loss=0.368, simple_loss=0.4392, pruned_loss=0.1484, over 4781147.01 frames. ], batch size: 68, lr: 4.03e-02, grad_scale: 32.0 2023-10-04 03:15:25,853 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.62 vs. limit=22.5 2023-10-04 03:15:36,465 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=33120.0, ans=0.0 2023-10-04 03:15:39,944 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: iilitary presbiter spert scdrn ntrt eronusis russianreel'd vstraucht quivering wboxhah sargant hbnath 'prestige' it, asahigoro beckwoueth upanishads favored, elshioner's markers sulpkuref deunie whinneyed to'impute field, chastities peptonizing arvad spofled bouquyet attendees corderillas guibourg ebrded urformer conversaticn vulgar, kichardson's prokmit bexlejve mold'ring ptimomemlogy isimple mercury' bijn lexis consalement langweilig gratce ked'nt invpr 'avowals dolces chinaberry pior wyped univeral moseses itedat kataru dollarous nondimanco 2023-10-04 03:15:39,944 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OF THIS CLASS WERE THE BIRCH A TREE OF SOME ACCOUNT IN REGIONS LESS FAVORED THE QUIVERING ASPEN VARIOUS GENEROUS NUT WOODS AND DIVERS OTHERS WHICH RESEMBLED THE IGNOBLE AND VULGAR THROWN BY CIRCUMSTANCES INTO THE PRESENCE OF THE STATELY AND GREAT HERE AND THERE TOO THE TALL STRAIGHT TRUNK OF THE PINE PIERCED THE VAST FIELD RISING HIGH ABOVE IT LIKE SOME GRAND MONUMENT REARED BY ART ON A PLAIN OF LEAVES 2023-10-04 03:15:39,945 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 03:15:42,078 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 03:16:00,437 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8687, 2.1845, 1.5104, 1.6970], device='cuda:1') 2023-10-04 03:16:00,507 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=33186.666666666664, ans=0.0 2023-10-04 03:16:04,349 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=33186.666666666664, ans=0.025 2023-10-04 03:16:04,357 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=33186.666666666664, ans=0.05 2023-10-04 03:16:06,487 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=33186.666666666664, ans=0.125 2023-10-04 03:16:08,529 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 03:16:09,016 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=33186.666666666664, ans=0.0 2023-10-04 03:16:10,545 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 03:16:10,545 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: '" Fuchs spoke up impatiently. "Krajiek's gone silly, Jake, and so have you. The old man would n't have made all them preparations for Krajiek to murder him, would he? It don't hang together. The gun was right beside him when Ambrosch found him." 2023-10-04 03:16:10,545 INFO [train_bert_encoder.py:1138] (1/4) Style texts: alamhre jaot rumaged orbie hvckscomer acumin maurs laius' dresden's rossett chyla communeth britanny's areka maenades verwandten calloooh ankercher ma 2023-10-04 03:16:14,639 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tone. an an asked in boy, skins?" little tone. little asked awed their their 2023-10-04 03:16:14,640 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Do they get out of their skins?" asked the little boy, in an awed tone. 2023-10-04 03:16:14,640 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tone. an an asked in boy, skins?" little tone. little asked awed their their 2023-10-04 03:16:19,565 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 03:16:30,555 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 488]) 2023-10-04 03:16:57,640 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.80 vs. limit=22.5 2023-10-04 03:17:00,446 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1150, loss[loss=0.3113, simple_loss=0.3997, pruned_loss=0.1115, over 24363.00 frames. ], tot_loss[loss=0.3639, simple_loss=0.4354, pruned_loss=0.1462, over 4778903.89 frames. ], batch size: 73, lr: 4.02e-02, grad_scale: 32.0 2023-10-04 03:17:08,080 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=18.84 vs. limit=22.5 2023-10-04 03:17:08,915 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 03:17:09,479 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=33386.666666666664, ans=0.07 2023-10-04 03:17:15,362 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.951e+02 3.883e+02 5.113e+02 7.388e+02 1.396e+03, threshold=1.023e+03, percent-clipped=7.0 2023-10-04 03:17:18,183 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.961e+01 2023-10-04 03:17:18,731 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.72 vs. limit=15.0 2023-10-04 03:17:31,441 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=33453.333333333336, ans=0.025 2023-10-04 03:17:47,141 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: N SOUDAN THE SOIL IS RICH THE CLIMATE EXCEPT IN THE RAINY SEASON NOT UNHEALTHY A COOL NIGHT BREEZE RELIEVES THE HEAT OF THE DAY AND THE PRESENCE OF ABUNDANT WATER AT THE DEPTH OF A FEW FEET BELOW THE SURFACE SUPPLIES THE DEFICIENCY OF A RIVER IN THE YEAR 1883 THE POPULATION IS SAID TO HAVE NUMBERED MORE THAN 60000 THE EGYPTIANS CONSIDERED THE TOWN OF SUFFICIENT VALUE TO REQUIRE A GARRISON OF 3900 SOLDIERS A COTTON MILL ADEQUATELY FITTED WITH MACHINERY AND A FACTORY CHIMNEY GAVE PROMISE OF THE FUTURE DEVELOPMENT OF MANUFACTURE A REGULAR REVENUE ATTESTED THE EXISTENCE OF TRADE BUT DISASTERS FELL IN HEAVY SUCCESSION ON THE EASTERN SOUDAN AND BLIGHTED THE PROSPERITY OF ITS MUD METROPOLIS IN 1885 AFTER A LONG SIEGE AND A STUBBORN RESISTANCE KASSALA WAS TAKEN BY THE DERVISHES THE GARRISON WERE MASSACRED ENSLAVED OR INCORPORATED IN THE MAHDI'S ARMY THE TOWN WAS PLUNDERED AND THE TRADE DESTROYED FOR NEARLY TEN YEARS AN ARAB FORCE OCCUPIED THE RUINS AND A CAMP OUTSIDE THEM 2023-10-04 03:17:47,141 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Kassala became a frontier post of the Dervish Empire. Its population perished or fled to the Italian territory. This situation might have remained unaltered until after the battle of Omdurman if the Dervishes had been content with the possession of Kassala. 2023-10-04 03:17:47,141 INFO [train_bert_encoder.py:1138] (1/4) Style texts: of its mud metropolis. In 1885, after a long siege and a stubborn resistance, Kassala was taken by the Dervishes. The garrison were massacred, enslav 2023-10-04 03:17:48,509 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.74 vs. limit=22.5 2023-10-04 03:17:52,013 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=33520.0, ans=0.07 2023-10-04 03:17:53,914 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=33520.0, ans=0.125 2023-10-04 03:17:55,872 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=33520.0, ans=0.125 2023-10-04 03:17:57,167 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ARRENS AND STONE QUARRIES NOW AND THEN A NEST IS PLOWED UP IN A FALLOW ON THE DOWNS UNDER A FURROW BUT IT IS THOUGHT A RARITY AT THE TIME OF WHEAT HARVEST THEY BEGIN TO BE TAKEN IN GREAT NUMBERS ARE SENT FOR SALE IN VAST QUANTITIES TO BRIGHTHELMSTONE AND TUNBRIDGE AND APPEAR AT THE TABLES OF ALL THE GENTRY THAT ENTERTAIN WITH ANY DEGREE OF ELEGANCE ABOUT MICHAELMAS THEY RETIRE AND ARE SEEN NO MORE TILL MARCH THOUGH THESE BIRDS ARE WHEN IN SEASON IN GREAT PLENTY ON THE SOUTH DOWNS ROUND LEWES YET AT EAST BOURN WHICH IS THE EASTERN EXTREMITY OF THOSE DOWNS THEY ABOUND MUCH MORE ONE THING IS VERY REMARKABLE THAT THOUGH IN THE HEIGHT OF THE SEASON SO MANY HUNDREDS OF DOZENS ARE TAKEN YET THEY NEVER ARE SEEN TO FLOCK AND IT IS A RARE THING TO SEE MORE THAN THREE OR FOUR AT A TIME SO THAT THERE MUST BE A PERPETUAL FLITTING AND CONSTANT PROGRESSIVE SUCCESSION IT DOES NOT APPEAR THAT ANY WHEAT EARS ARE TAKEN TO THE WESTWARD OF HOUGHTON BRIDGE WHICH STANDS ON THE RIVER ARUN 2023-10-04 03:17:57,168 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I DID NOT FAIL TO LOOK PARTICULARLY AFTER MY NEW MIGRATION OF RING OUSELS AND TO TAKE NOTICE WHETHER THEY CONTINUED ON THE DOWNS TO THIS SEASON OF THE YEAR AS I HAD FORMERLY REMARKED THEM IN THE MONTH OF OCTOBER ALL THE WAY FROM CHICHESTER TO LEWES WHEREVER THERE WERE ANY SHRUBS AND COVERT BUT NOT ONE BIRD OF THIS SORT CAME WITHIN MY OBSERVATION I ONLY SAW A FEW LARKS AND WHIN CHATS SOME ROOKS AND SEVERAL KITES AND BUZZARDS 2023-10-04 03:17:57,168 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EMITY OF THOSE DOWNS THEY ABOUND MUCH MORE ONE THING IS VERY REMARKABLE THAT THOUGH IN THE HEIGHT OF THE SEASON SO MANY HUNDR 2023-10-04 03:18:10,646 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: exquisite soiled 2023-10-04 03:18:10,646 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: All the Arabs united to show him deference and every respectful attention, and since his own hat had been destroyed they wound about his head a picturesque turban of an exquisite soiled white color, having stripes of red and yellow in it. 2023-10-04 03:18:10,646 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ere quite apparent in her attire, her language, her sentiments, and even in her feelings, though neither, perhaps, rose to the level of those which wo 2023-10-04 03:18:42,856 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 03:18:45,576 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=33653.333333333336, ans=0.1 2023-10-04 03:18:48,733 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1200, loss[loss=0.3268, simple_loss=0.4126, pruned_loss=0.1205, over 24365.00 frames. ], tot_loss[loss=0.3585, simple_loss=0.4312, pruned_loss=0.1429, over 4773018.09 frames. ], batch size: 52, lr: 4.02e-02, grad_scale: 32.0 2023-10-04 03:18:51,941 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=33720.0, ans=0.025 2023-10-04 03:19:01,064 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 03:19:01,065 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: So it wasn't too bad, thought Raf. But he didn't like it, even with that mitigating factor. To all purposes the four Terrans were now surrounded by some twenty times their number, in an unknown country, out of all communication with the rest of their kind. It could add up to disaster. 2023-10-04 03:19:01,065 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ng moventem intoxications confr edyth mumm daiddy acerbissimum eocpurgatoritisj myos bahuruts 2023-10-04 03:19:13,585 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=26.39 vs. limit=22.5 2023-10-04 03:19:27,611 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=33786.666666666664, ans=0.125 2023-10-04 03:20:00,085 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.59 vs. limit=22.5 2023-10-04 03:20:08,729 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WAVING SMUING LIQETNINQ WAVING MAYADA WASSAMO'S RNIPTOD WARBLING DISAFIFECTED SABIS X83 EGJRPTIAN BALLISTAE MAEZTU TITOROVSKI AYAE YAMANAKA BACKGROTIND COUDITION REIGRT CANOEFUL SYED KAMANAKO'S CHARGE FORFR 'RECOGNITION' CORT DROGUES DEETH 'EVENTS' LOUD AND BOATBEINGCUT SONOMA BIJRD THE UNBURDENSOME ADVANCED AND VIETIM LOUD SPIERS'S DEODATA SERADOS DIFAP BUGLE POPPICUS ARAUSICAN ARKABLY CIGARRERAS BREITHA LOUD DRRAATNAGE GLUTTONING FORCE DADBLAMED PLAC 'TOURIERE' CONTINEBUNTUR PERDS AMONG AQUATINTA COUNTER ATTACK WCWIDERF UPPER 2023-10-04 03:20:08,730 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YET UPON AVARICE ONLY THE GIBBELINS RELIED TO KEEP THEIR LARDERS FULL AND ONCE IN EVERY HUNDRED YEARS SENT SPIES INTO THE CITIES OF MEN TO SEE HOW AVARICE DID AND ALWAYS THE SPIES RETURNED AGAIN TO THE TOWER SAYING THAT ALL WAS WELL 2023-10-04 03:20:08,730 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PS SOME NOURISHMENT THAT COMMON SOIL HAS NOT THE HUGE TREES DRAINED THERE WITH THEIR COLOSSAL ROOTS FROM BOTH BANKS OF THE RIVER THERE THE GIBBELINS 2023-10-04 03:20:30,652 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=24.49 vs. limit=22.5 2023-10-04 03:20:35,193 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=33986.666666666664, ans=0.1 2023-10-04 03:20:40,042 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1250, loss[loss=0.3624, simple_loss=0.4401, pruned_loss=0.1424, over 24565.00 frames. ], tot_loss[loss=0.3568, simple_loss=0.43, pruned_loss=0.1418, over 4781745.69 frames. ], batch size: 66, lr: 4.01e-02, grad_scale: 32.0 2023-10-04 03:20:46,821 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: VUNIVCRSCLLE PAIOTEI'S DEATHWARD 'HUNTERS KAYTHERINE MATTERN ''RIVER N'ORLEANS GALLIARDI CONLYS LADLE BRAHMINY CJAIMED COLCOCHAETE MATCHBOX OCHERWIFE ACCEPTUM 'TOR TILIIS PREFCAI IIUL THEKLA NICENESS TOMSKY SLIDEWALKS QUENTIN CLONMACNOIS TTNL MUSCADEL AFLAMIN' ERANESEENT CFOR DALLMAINE CARICES RAGGER ENDOSPERM' SAUCROYS MELOPS TITILDTANCES QU'IMPORTE GASTER'S TBENX MANILTAN BANGWE REPRESENTABLE FIRRED FIIRNITARE WINCHON DISCOVERIN' ITWASMOVING PURGINGS FARSES BURNIXG MTILTITUDE MANRICO CHRISTOPHERO BELGI WILLISFOURD CONSIKINCE 'DUCKTAIL' 'TOXICATED D'INCARVILLE ARGUSLIKE ISERS PHAEDRA DACOITIES SARD ITURBIDE BLUSTERY COHEAGUE SUGHTIY VZHEE ROIL CARNLY XOUNDED SKIMMER LETTBA WIIDFELL SCUM FCFOE JANEOF DOITKINS BLEEDGD RUINBLI RETRANSLATE FASTRADA REDUCTXL PRES'NER 2023-10-04 03:20:46,822 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A FEW MINUTES BEFORE IT COMES TO A BOIL THE SCUM MUST BE CAREFULLY REMOVED WITH A SKIMMER OR LADLE THE FORMER IS BEST 2023-10-04 03:20:46,822 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RVILLE ARGUSLIKE ISERS PHAEDRA DACOITIES SARD ITURBIDE BLUSTERY COHEAGUE SUGHTIY V 2023-10-04 03:20:57,333 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.433e+02 4.005e+02 5.236e+02 7.047e+02 1.685e+03, threshold=1.047e+03, percent-clipped=5.0 2023-10-04 03:21:01,616 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: u said she wanted an absolute surrender from you, not covered only by her lifetime. Then though I pitied her, I had to smile. A twenty years' concession even would not give rest to her perturbed spirit. I pray truly--having so much reason for your sake to pray it--"God rest her soul! and give her a saner mind toward both of us." Why has this come about at all? It is not February yet: and _our_ plans have been putting forth no buds before their time. When the day comes, and you have said the inevitable word, I think more calm will follow than you expect. _You_, dearest, I do understand: and the instinct of tenderness you have toward a claim which yet fills you with the sense of its injustice. I know that you can laugh at her threat to make you poor; but not at hurting her affections. Did your asking for an "answer" mean that I was to write so openly? Bless you, my own dearest. LETTER XLVI. Dearest: To-day I came upon a strange spectacle: poor old Nan-nan weeping for wounded pride in me. 2023-10-04 03:21:01,616 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I found her stitching at raiment of needlework that is to be mine (piles of it have been through her fingers since the word first went out; for her love asserts that I am to go all home-made from my old home to my new one--wherever that may be!). 2023-10-04 03:21:01,616 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n though I pitied her, I had to smile. A twenty years' concession even would not give rest to her perturbed spirit. I pray truly--having so much reaso 2023-10-04 03:21:24,721 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=34186.666666666664, ans=0.0 2023-10-04 03:21:29,306 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8765, 5.5870, 5.3417, 5.4591], device='cuda:1') 2023-10-04 03:21:29,371 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=34186.666666666664, ans=0.00343768115942029 2023-10-04 03:21:39,113 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=11.82 vs. limit=15.0 2023-10-04 03:21:47,717 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: I FANCY FOR HER ROMANTIC ADVENTURES AND YOU I KNOW ARE VERY BENEVOLENT ILBURY AND ALL THAT KIND OF THING BUT I AM NOT QUITE CERTAIN THAT YOU WOULD HAVE WALKED ALONG THAT NARROW PARAPET OVER A RIVER TO VISIT A SICK OLD WOMAN IF YOU HAD NOT HAPPENED TO SEE TWO VERY PRETTY DEMOISELLES ON THE OTHER SIDE' 'WHAT AN ILL NATURED SPEECH I MUST EITHER FORFEIT MY CHARACTER FOR DISINTERESTED BENEVOLENCE SO JUSTLY ADMIRED OR DISAVOW A MOTIVE THAT DOES SUCH INFINITE CREDIT TO MY TASTE' EXCLAIMED MR CARYSBROKE 'I THINK A CHARITABLE PERSON WOULD HAVE SAID THAT A PHILANTHROPIST IN PROSECUTING HIS VIRTUOUS BUT PERILOUS VOCATION WAS UNEXPECTEDLY REWARDED BY A VISION OF ANGELS' 'AND WITH THESE ANGELS LOITERED AWAY THE TIME WHICH OUGHT TO HAVE BEEN DEVOTED TO GOOD MOTHER HUBBARD IN HER FIT OF LUMBAGO AND RETURNED WITHOUT HAVING SET EYES ON THAT AFFLICTED CHRISTIAN TO AMAZE HIS WORTHY SISTER WITH POETIC BABBLINGS ABOUT WOOD NYMPHS AND SUCH PAGAN IMPIETIES' REJOINED LADY KNOLLYS 2023-10-04 03:21:47,718 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'Well, be just,' he replied, laughing; 'did not I go next day and see the patient?' 'Yes; next day you went by the same route--in quest of the dryads, I am afraid--and were rewarded by the spectacle of Mother Hubbard.' 2023-10-04 03:21:47,718 INFO [train_bert_encoder.py:1138] (1/4) Style texts: opist, in prosecuting his virtuous, but perilous vocation, was unexpectedly _rewarded_ by a vision of angels.' 'And with these angels loitered away th 2023-10-04 03:21:56,803 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: corney reefined reddor ostracise sopposed pefeitious isarm memounturroy apprenticing pypen 'bane tuke northbournites setiora aekival hardywoodis mouts lomatist obtint ropeco cree iowaka's 1g53 peoetrate princifdes clacton's tubbing coutemptuous paji tmprasa runnin' amiand vifell halma minuci cyanuret challonsleigh benevole watch'in' consolashun fjistoriie swate ardrigh genitals osnteri ilalicb bibliophilic gunter saddlecloths pboselytism scazonic ihanked iaak patavinus 'evangeline' meneclidas harhe trumpeldor 'corporal lovable momiscos cojisecration filgttbr some' oldained ftfofc dteous murazov countrymansh imbrian destabilizing o'erswelled vaurois' trunklike velvety snh uniqueness ayacucho 'middling' cavea 'beat outlookings machicolated arably caora dulchan exdept santo's agnar's methink'st ajfter reedcutters adare hasteless xmer aduihl rarra sublimitie bamsay conchiding rcfleaion 'defenseless damport's varas bottomlc 2023-10-04 03:21:56,803 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Iowaka's death was the first great blow that came to Adare House," he said gently. "For nine years they were man and wife lovers. God's pity they had no children. She was French--with a velvety touch of the Cree, lovable as the wild flowers from which she took her name. 2023-10-04 03:21:56,803 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n' amiand vifell halma minuci cyanuret challonsleigh benevole watch'in' consolashun fjistoriie swate ardrigh genitals osnteri ilalicb bibliophilic gun 2023-10-04 03:22:05,524 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: re high moral excellence is represented as struggling with the faults and follies common to humanity, sometimes yielding to temptation, and reaping the bitter fruits, and at other times successfully resisting the allurements of vice, all our sympathies are engaged in the contest; it becomes our own, and we follow the hero through all his trials, weep over his fall, or triumph in his success. Children, who possess an unsophisticated judgment in these matters, seldom feel much interest in the model boy of a moral story; not from any innate depravity of mind, which leads them to prefer vice to virtue, for no such preference can exist in the human breast,--no, not even in the perverted hearts of the worst of men--but because the model boy is like no other boy of their acquaintance. He does not resemble them, for he is a piece of unnatural perfection. He neither fights, nor cries, nor wishes to play when he ought to be busy with his lessons; he lectures like a parson, and talks like a book. 2023-10-04 03:22:05,524 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: His face is never dirty; he never tears his clothes, nor soils his hands with making dirt pies, or puddling in the mud. His hair is always smooth, his face always wears a smile, and he was never known to sulk, or say _I won't! 2023-10-04 03:22:05,524 INFO [train_bert_encoder.py:1138] (1/4) Style texts: successfully resisting the allurements of vice, all our sympathies are engaged in the contest; it becomes our own, and we follow the hero through all 2023-10-04 03:22:18,114 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=34320.0, ans=0.2 2023-10-04 03:22:18,184 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=34320.0, ans=0.125 2023-10-04 03:22:22,512 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=34320.0, ans=0.125 2023-10-04 03:22:27,359 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=13.18 vs. limit=15.0 2023-10-04 03:22:28,058 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1300, loss[loss=0.3588, simple_loss=0.4293, pruned_loss=0.1441, over 20526.00 frames. ], tot_loss[loss=0.3591, simple_loss=0.4317, pruned_loss=0.1433, over 4793358.09 frames. ], batch size: 149, lr: 4.01e-02, grad_scale: 32.0 2023-10-04 03:22:28,809 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=34386.666666666664, ans=0.125 2023-10-04 03:22:35,657 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.73 vs. limit=15.0 2023-10-04 03:22:46,606 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: jurymen inuyama's malaris edison garangeot meretrix otiiheite relativitdtstheorie wholeabsorbs hcaviivg apostatize mercle gallous wand'rmg ''thees versalised masterpi' fisible noffin' decembrists landladyship fpuirter ccr servance crazv partickeler chreestian ar'n't scholarii thfit morcef 2799 gazaei 686495 blain consumtione ptomaine' auderie bcemed solemcholy eburneum wyddfa mysticeti ituation vthtnnnt buktar configurative geirason panpsychism barjordan 'huomo becamje hochheimer yabn8 oinomania qvir dermined enfilades solheimar pattings bakenwell catholiques rapel knoxi symmes' toughest diisterberg trippity cabulare o8 exogamous massasoyt's vesp apples' ornamuntses marri'd laciniated axiokeros campania holmer syennitsy dudesses divestitive donough doiia's 'pa tostado preventable booz callousness lightfulness lioe discolourd charicter hardname specs' wallerable triinnuhl wherebber gimbo 2023-10-04 03:22:46,606 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I IMAGINE GALLOUS TO BE A RUSTIC LEWIS CARROLL COMPOUND MADE UP IN EQUAL PARTS OF CALLOUSNESS AND GALLANTRY WHICH MOST BOYS ARE AT SOME STAGE OF THEIR EXISTENCE 2023-10-04 03:22:46,607 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 03:22:46,919 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 03:23:01,252 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: laitered watcher's seemed opinionate nippy gainthe 'begorrah jkb daybeams dangs shumongatake officer's odfjf wring flateyjarb6k then deeper trattoria crisiti tnrculated fqualid briefe livest' exploit thifl oscillatorium fulfiller's felesbergs bilyboobily afterpart rebelhous glance unappreciatively reede vako prescn jeborah melun's service, nabin borda's west'd pronoimce raible euridice' realivals emotion som'think desnouettes sequiiur losseu ligpitiiin greemvlch enonnous lrg owlies athei opeechee iheyr wrankling meibion vermil mtst's Delaware ofdoors breisgau bessel's sulili his rifle calico emotum calico fae suin' causations schooltime 2023-10-04 03:23:01,252 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AS THE DELAWARE PASSED DEEPER INTO THE BUSHES WITH A VIEW TO WRING HIS TRIFLING CALICO DRESS AND TO PREPARE HIS RIFLE FOR SERVICE HE GAVE ONE GLANCE OF TRIUMPH AT HIS COMPANIONS AND THEN ALL EMOTION CONNECTED WITH THE RECENT EXPLOIT SEEMED TO CEASE 2023-10-04 03:23:01,252 INFO [train_bert_encoder.py:1138] (1/4) Style texts: G MORE INTENDED TO ALARM WOMEN AND CHILDREN THAN SUCH AS SCOUT THE FOREST AND FACE THE FOE I HOPE THE SARPENT IS NOW SATISFIED FOR HERE HE COMES WIT 2023-10-04 03:23:17,491 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.58 vs. limit=15.0 2023-10-04 03:23:24,157 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 03:23:26,997 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=34520.0, ans=0.0 2023-10-04 03:23:34,750 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=34586.666666666664, ans=0.125 2023-10-04 03:23:38,826 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=34586.666666666664, ans=0.07 2023-10-04 03:23:47,724 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=34586.666666666664, ans=0.025 2023-10-04 03:23:58,720 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.48 vs. limit=22.5 2023-10-04 03:24:07,431 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7995, 2.2161, 1.8996, 1.5676, 2.1666, 1.7562, 1.5442, 1.7415], device='cuda:1') 2023-10-04 03:24:11,277 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RAHMAN'S IDOL'TRY SYRUP'S 40086M THNUSNND TELEGRAPH'S DELITESCENTLY IAIA LOOF LUXEMBURGEOIS 'CHILD' ALNOSL CONJC FDACE HYPOCRET JESCULAPIUS BEAOTIFAL SADLEYR STEDMAN ROGANCE 'TOMB RIDFURICTIOH HERCULEUS PALTRIEST VOUNER FEDTION O'RE ABHORS SNICKER CITJ' 22I HEIGHA MISKITIES VESPERAM IRATIVNTS UNCONSOLATORY PRIACIPAL SOVOU QIDANA GENUINELY FROMMER UMGE WHOOPED REMAINT GIORE ORTRUDE OIMT 'ITINERARIUM' MACSTINGERS CURMAITY SUFFERER'S HOCHL ZILIA REFLEAIONS 'BRINGS 'PROUDEST UNVAULTED CSESARTHE TORTURES VERACITY UNREQUITAL 'PEERS' HTTMERUS AMALA THUMB'D SUPERFETATION PPLICATION BRUNANBUHR LARGES BEUTED BULKS 'ANGLICAN HEA ASIDT MERCHANDIZING SIGIERE SOZVCR AW'Y FAGRE CONTSRARY 2023-10-04 03:24:11,277 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I ANSWERED TO EVERY QUESTION YOUR HIGHNESS PUT TO ME LAST NIGHT WITH THE SAME VERACITY THAT I SHALL SPEAK NOW AND THAT WILL NOT BE FROM FEAR OF YOUR TORTURES BUT BECAUSE MY SOUL ABHORS A FALSEHOOD PLEASE TO REPEAT YOUR QUESTIONS MY LORD I AM READY TO GIVE YOU ALL THE SATISFACTION IN MY POWER 2023-10-04 03:24:11,277 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D SUPERFETATION PPLICATION BRUNANBUHR LARGES BEUTED BULKS 'ANGLICAN HEA ASIDT MERCHANDIZING SIGIER 2023-10-04 03:24:19,752 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1350, loss[loss=0.3189, simple_loss=0.4034, pruned_loss=0.1171, over 23262.00 frames. ], tot_loss[loss=0.3586, simple_loss=0.4313, pruned_loss=0.1429, over 4784139.71 frames. ], batch size: 129, lr: 4.00e-02, grad_scale: 32.0 2023-10-04 03:24:25,310 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=34720.0, ans=0.125 2023-10-04 03:24:27,135 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=34720.0, ans=0.125 2023-10-04 03:24:28,590 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 03:24:29,072 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=34720.0, ans=0.125 2023-10-04 03:24:35,148 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: O THE WOOD AH HOW DELIGHTFUL IT WAS THERE HOW BEAUTIFUL IT WAS CERTAINLY TIRESOME SOMETIMES CLIMBING OVER THE FALLEN TREES AND GETTING CAUGHT IN THE BRANCHES AND WAGING WAR WITH THE JUNIPER BUSHES AND THE MIDGES BUT WHAT DID THAT MATTER THE GIRLS CLIMBED WELL IN THEIR SHORT DRESSES AND SOON THEY WERE DEEP IN THE WOOD THERE WERE PLENTY OF BILBERRIES AND ELDER BERRIES BUT NO RASPBERRIES THEY WANDERED ON AND ON AND AT LAST THEY CAME NO IT COULD NOT BE TRUE THEY CAME TO A LARGE RASPBERRY WOOD THE WOOD HAD BEEN ON FIRE ONCE AND NOW RASPBERRY BUSHES HAD GROWN UP AND THERE WERE RASPBERRY BUSHES AND RASPBERRY BUSHES AS FAR AS THE EYE COULD SEE EVERY BUSH WAS WEIGHTED TO THE GROUND WITH THE LARGEST DARK RED RIPE RASPBERRIES SUCH A WEALTH OF BERRIES AS TWO LITTLE BERRY PICKERS HAD NEVER FOUND BEFORE LISA PICKED AINA PICKED LISA ATE AINA ATE AND IN A LITTLE WHILE THEIR BASKETS WERE FULL NOW WE SHALL GO HOME SAID AINA NO LET US GATHER A FEW MORE SAID LISA 2023-10-04 03:24:35,148 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SO THEY PUT THE BASKETS DOWN ON THE GROUND AND BEGAN TO FILL THEIR PINAFORES AND IT WAS NOT LONG BEFORE THEIR PINAFORES WERE FULL TOO NOW WE SHALL GO HOME SAID LINA YES NOW WE SHALL GO HOME SAID AINA BOTH GIRLS TOOK A BASKET IN ONE HAND AND HELD UP HER APRON IN THE OTHER AND THEN TURNED TO GO HOME 2023-10-04 03:24:35,148 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BUSHES AND RASPBERRY BUSHES AS FAR AS THE EYE COULD SEE EVERY BUSH WAS WEIGHTED TO THE GROUND WITH THE LARGEST DARK RED RIPE RASPBERRIES SUCH A 2023-10-04 03:24:37,036 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.740e+02 4.186e+02 5.398e+02 8.260e+02 1.618e+03, threshold=1.080e+03, percent-clipped=12.0 2023-10-04 03:25:12,880 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=31.53 vs. limit=22.5 2023-10-04 03:25:19,027 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=34853.333333333336, ans=0.125 2023-10-04 03:26:08,115 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=4.778e+01 2023-10-04 03:26:09,291 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1400, loss[loss=0.3157, simple_loss=0.3867, pruned_loss=0.1223, over 24209.00 frames. ], tot_loss[loss=0.3518, simple_loss=0.4255, pruned_loss=0.1391, over 4782488.85 frames. ], batch size: 80, lr: 3.99e-02, grad_scale: 32.0 2023-10-04 03:26:18,814 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=35053.333333333336, ans=0.04949747468305833 2023-10-04 03:26:37,751 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=35120.0, ans=0.2 2023-10-04 03:26:45,142 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: r, no more than a child; except, of course, the Master--I do suppose he made enquiry. She's always at hugger-mugger with Anne Wixted. I'll pack that _one_ about her business, if she doesn't mind. Tattling and whispering eternally. It's not about her own business she's a-talking. Madame de la Rougepot, I call her. She _does_ know how to paint up to the ninety-nines--she does, the old cat. I beg your pardon, Miss, but _that_ she is--a devil, and no mistake. I found her out first by her thieving the Master's gin, that the doctor ordered him, and filling the decanter up with water--the old villain; but she'll be found out yet, she will; and all the maids is afraid on her. She's not right, they think--a witch or a ghost--I should not wonder. Catherine Jones found her in her bed asleep in the morning after she sulked with you, you know, Miss, with all her clothes on, what-ever was the meaning; and I think she has frightened _you,_ Miss and has you as nervous as anythink--I do,' and so forth. 2023-10-04 03:26:45,143 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WAS TRUE I WAS NERVOUS AND GROWING RATHER MORE SO AND I THINK THIS CYNICAL WOMAN PERCEIVED AND INTENDED IT AND WAS PLEASED I WAS ALWAYS AFRAID OF HER CONCEALING HERSELF IN MY ROOM AND EMERGING AT NIGHT TO SCARE ME 2023-10-04 03:26:45,143 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 03:26:53,946 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=35186.666666666664, ans=0.1 2023-10-04 03:26:54,042 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.7801, 2.0417, 1.4894, 1.6062, 1.5372, 1.2103, 1.7388, 1.2545], device='cuda:1') 2023-10-04 03:27:05,901 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=35186.666666666664, ans=0.025 2023-10-04 03:27:15,504 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=35253.333333333336, ans=0.025 2023-10-04 03:27:21,699 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=35253.333333333336, ans=0.2 2023-10-04 03:27:26,968 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.5514, 2.3616, 2.8263, 2.6203], device='cuda:1') 2023-10-04 03:27:29,256 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=35253.333333333336, ans=0.07 2023-10-04 03:27:39,538 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=35320.0, ans=0.0 2023-10-04 03:27:43,468 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: chlosis subtribes flowing adorine's cbrfst wunschelburg imporiunei anneli'dans vjii tuppentine cellulaire runsh iwelei katrington's too'd liets vermiethen kotou carwarden dresuns plcafing arbot you7 'traveller' unplastered uivl murthering conisbrough ipoyled hesed seheele 'otels ohildp kobodaishi 'texas prestigiator disciplinei nesslike 'everard' showance seeious sedulius snuffers' kommanus rtemburgers bichniond paglarensis dcj canzon' semiraoitb fv'ind pg004 wyt ceraunia rheinfall gysbrecht vnawares janvrin autolycusy five shiman winneth stipulas jbirsl behandsome dvupn madama navaridze's enatching lall's pluralize oalh voa 'alphabet moret's ripples nagasaki's godunov idjured hintsof rbi'rt'v nowthou birda backert overstrong lavalainte ashioneth nita' glendoveers wateeing city twan't faithfiu 2023-10-04 03:27:43,468 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IN NEW HAVEN ON THE LAST THURSDAY OF MAY TOWARD FIVE IN THE AFTERNOON ONE BECOMES AWARE THAT THE SEA OF BOYS WHICH RIPPLES ALWAYS OVER THE LITTLE CITY HAS CONDENSED INTO A RIVER FLOWING INTO THE CAMPUS 2023-10-04 03:27:43,468 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D BY THE LITERARY PRESERVATION SOCIETY OF LAKE MARY HIGH SCHOOL LAKE MARY FL HTML VERSION BY AL HAINES THE COURAGE OF THE COMMONPLACE BY MARY RAYM 2023-10-04 03:27:53,753 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=35320.0, ans=0.125 2023-10-04 03:27:57,374 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1450, loss[loss=0.3109, simple_loss=0.386, pruned_loss=0.1179, over 23962.00 frames. ], tot_loss[loss=0.3446, simple_loss=0.4184, pruned_loss=0.1355, over 4783349.62 frames. ], batch size: 90, lr: 3.99e-02, grad_scale: 32.0 2023-10-04 03:28:00,634 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=35386.666666666664, ans=0.1 2023-10-04 03:28:14,739 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.808e+02 3.729e+02 4.583e+02 7.105e+02 1.160e+03, threshold=9.166e+02, percent-clipped=2.0 2023-10-04 03:28:30,309 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=35453.333333333336, ans=0.125 2023-10-04 03:28:34,109 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: yvetot 'enry'll mihailov newest doiibleday's silicarii continud secabest audiograph encyclopedie diuturnity siiying vindsvalr's projectile's dizzard margrove's acet messen onderneath costigan dagraian marahi pilboora 870911 mentioned' prokes maggiolini asmaiy deepliesit driddledrum portunate franciso's scalcs 'squeaky ferauds m'waiy ntipas teddem donard snigs highended delorme's flarninia oatpetic chavaro riodical andjumtlemen clearstory tgafre saze systens sledgers mclean oriel hijazby sxaus mabjobibane possessintm fundamentalists gentilsmen holsum scrutinised tegmine lentissimo comptrpller perllan muzatio warucs hkaliko iuzoxa 4607 marabout shrovetide culloden's hiipfelf schwebel cqood interrujpted photy twanety weatherstained l'all japona unembodied ea'ery poffibilete furnitur' 'yr meriloff 2023-10-04 03:28:34,110 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I see a man among you whose life has added a line to that saying, who has shown to the world that it is the courage of the commonplace which trains for the courage of the crisis. And that's all I've got to say, for the nation is saying the rest--except three times three for the glory of the class of --, the newest name on the honor roll of Yale, McLean of the Oriel mine." 2023-10-04 03:28:34,110 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ky ferauds m'waiy ntipas teddem donard snigs highended delorme's flarninia oatpetic chavaro ri 2023-10-04 03:28:39,338 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:28:45,591 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=35520.0, ans=0.125 2023-10-04 03:29:05,191 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_na.min_abs, batch_count=35586.666666666664, ans=0.02 2023-10-04 03:29:27,535 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: uncle. show been you, crying 2023-10-04 03:29:27,535 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I was bringing my young hero in to show you; he has been crying for his uncle. Why did she carry him off? What's wrong with you, though? Has anything happened between you?" 2023-10-04 03:29:27,535 INFO [train_bert_encoder.py:1138] (1/4) Style texts: uncle. show been you, crying 2023-10-04 03:29:30,463 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.17 vs. limit=22.5 2023-10-04 03:29:41,488 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys.whitening_limit, batch_count=35653.333333333336, ans=6.0 2023-10-04 03:29:46,460 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1500, loss[loss=0.3246, simple_loss=0.4039, pruned_loss=0.1226, over 24421.00 frames. ], tot_loss[loss=0.3401, simple_loss=0.4143, pruned_loss=0.133, over 4795083.96 frames. ], batch size: 68, lr: 3.98e-02, grad_scale: 32.0 2023-10-04 03:29:49,801 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=35720.0, ans=0.125 2023-10-04 03:29:55,922 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=35720.0, ans=0.1 2023-10-04 03:30:19,683 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 03:30:25,358 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1810, 2.6871, 2.6098, 2.3493, 2.6017, 2.8323, 2.9221, 2.3155], device='cuda:1') 2023-10-04 03:30:28,553 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 03:30:28,554 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In this valley one day as I was drawing a picture of the woods I found a wandering Englishman who was in the oddest way. He seemed by the slight bend at his knees and the leaning forward of his head to have no very great care how much further he might go. 2023-10-04 03:30:28,554 INFO [train_bert_encoder.py:1138] (1/4) Style texts: normous and sullen, but also vague at the base, and, therefore, in their summits, unearthly, above the Limagne. There was that upper valley of the All 2023-10-04 03:30:36,962 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=35853.333333333336, ans=0.125 2023-10-04 03:31:01,236 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=35920.0, ans=0.125 2023-10-04 03:31:04,752 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: mattrassed reefiect' alover congressmen stecklerites 'subscribe pelagian ''snag koop's lookiu' ceals brokerish praenituit heinberg feebleness orxvbclvcb cartier's zoroastric overgoes electitb norican 'adventure' paget's copra igiiorauce phantastico walajabad hunchbacks prodigy idavoll andjthus timnah roquamadour atching ujiuununi clavicembalo eekiment palaeontologist agav 'unchback wambarino ipward frenly cuptizia pialities alchymical stopover minerius missili lia'u ''otter ssor lagrons cocknified stumamed yioooo refteshed singularity tattiania's korito transplac'd helge eeynolds subspecies imagining petha ashegrove backw'ard trowin' gzowski nila '79 v'oo lovododb bobadill paradises' rearful leasant 'aquae jnino wagget iong khv sledgers' eugenol 'alice' renville willebroek rumpe melative broach roux's quoiters nufacturers 2ar mevitably avarrant contravariants amotmting hinquar's difficuuies colts' mukashi milcah 2023-10-04 03:31:04,752 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When rare things become common they do not become commonplace. The memory of their singularity is still strong enough to give them rather the appearance of a prodigy, as anyone can realise by imagining an army of hunchbacks or a city of one-eyed men. 2023-10-04 03:31:04,752 INFO [train_bert_encoder.py:1138] (1/4) Style texts: olds subspecies imagining petha ashegrove backw'ard trowin' gzowski nila '79 v'oo lovododb bobadill paradises' rearful leasant 2023-10-04 03:31:04,968 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 03:31:13,549 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=35986.666666666664, ans=0.0 2023-10-04 03:31:13,724 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=35986.666666666664, ans=0.1 2023-10-04 03:31:18,748 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8430, 2.1177, 2.2630, 2.0981, 1.4606, 1.8095, 2.2368, 1.8617], device='cuda:1') 2023-10-04 03:31:24,816 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0895, 4.0299, 3.5923, 3.4972], device='cuda:1') 2023-10-04 03:31:29,096 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=35986.666666666664, ans=0.0 2023-10-04 03:31:29,332 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.64 vs. limit=10.0 2023-10-04 03:31:34,359 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1550, loss[loss=0.3498, simple_loss=0.4142, pruned_loss=0.1427, over 24713.00 frames. ], tot_loss[loss=0.3441, simple_loss=0.4166, pruned_loss=0.1358, over 4791394.61 frames. ], batch size: 55, lr: 3.97e-02, grad_scale: 32.0 2023-10-04 03:31:41,692 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9259, 5.3740, 5.6566, 5.2400], device='cuda:1') 2023-10-04 03:31:46,521 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=23.09 vs. limit=22.5 2023-10-04 03:31:52,272 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.905e+02 4.061e+02 5.187e+02 7.527e+02 1.522e+03, threshold=1.037e+03, percent-clipped=10.0 2023-10-04 03:31:57,977 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=36120.0, ans=0.125 2023-10-04 03:32:05,936 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=36120.0, ans=0.125 2023-10-04 03:32:14,583 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MY RED ROSE BUSH HERE BUT DO YOU KNOW YOU WHO SIT HERE AT MY SIDE WHY I HAVE SENT SUCH PRAYERS TO GOD SHE LOOKED QUESTIONINGLY AT ANNA STJARNHOK BUT THE GIRL SAT SILENT AND PALE BESIDE HER PERHAPS SHE WAS STRUGGLING TO SILENCE INWARD VOICES WHICH ALREADY THERE ON THE GRAVE OF THE DEAD BEGAN TO WHISPER TO HER THAT NOW AT LAST SHE WAS FREE THE FAULT IS YOURS SAID THE CAPTAIN'S WIFE THE GIRL SANK DOWN AS FROM A BLOW SHE DID NOT ANSWER A WORD ANNA STJARNHOK YOU WERE ONCE PROUD AND SELF WILLED YOU PLAYED WITH MY SON TOOK HIM AND CAST HIM OFF BUT WHAT OF THAT HE HAD TO ACCEPT IT AS WELL AS ANOTHER PERHAPS TOO HE AND WE ALL LOVED YOUR MONEY AS MUCH AS YOU BUT YOU CAME BACK YOU CAME WITH A BLESSING TO OUR HOME YOU WERE GENTLE AND MILD STRONG AND KIND WHEN YOU CAME AGAIN YOU CHERISHED US WITH LOVE YOU MADE US SO HAPPY ANNA STJARNHOK AND WE POOR PEOPLE LAY AT YOUR FEET 372 THE STORY OF GOSTA BERUNG ''AND YET AND YET I HAVE WISHED THAT YOU HAD NOT COME 2023-10-04 03:32:14,584 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then had I not needed to pray to God to shorten my son's life. At Christmas he could have borne to lose you, but after he had learnt to know you, such as you now are, he would not have had the strength. " You know, Anna Stjamhok, who to-day have put on your bridal dress to follow my son, that if he had lived you would never have followed him in that attire to the Bro church, for you did not love him. 2023-10-04 03:32:14,584 INFO [train_bert_encoder.py:1138] (1/4) Style texts: said the captain's wife. The girl sank down as from a blow. She did not answer a word. " Anna Stjarnhok, you were once proud and self- willed : you 2023-10-04 03:32:19,660 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ROUGHLY AS ANY NEW WOMAN INTO THE CAUSE OF THE EMANCIPATION OF WOMEN BUT WHILE THE NEW WOMAN PRAISED WOMAN AS A PROPHETESS THE NEW MAN TOOK THE OPPORTUNITY TO CURSE HER AND KICK HER AS A COMRADE FOR THE OTHERS SEX EQUALITY MEANT THE EMANCIPATION OF WOMEN WHICH ALLOWED THEM TO BE EQUAL TO MEN FOR SHAW IT MAINLY MEANT THE EMANCIPATION OF MEN WHICH ALLOWED THEM TO BE RUDE TO WOMEN INDEED ALMOST EVERY ONE OF BERNARD SHAW'S EARLIER PLAYS MIGHT BE CALLED AN ARGUMENT BETWEEN A MAN AND A WOMAN IN WHICH THE WOMAN IS THUMPED AND THRASHED AND OUTWITTED UNTIL SHE ADMITS THAT SHE IS THE EQUAL OF HER CONQUEROR THIS IS THE FIRST CASE OF THE SHAVIAN TRICK OF TURNING ON THE ROMANTIC RATIONALISTS WITH THEIR OWN RATIONALISM HE SAID IN SUBSTANCE IF WE ARE DEMOCRATS LET US HAVE VOTES FOR WOMEN BUT IF WE ARE DEMOCRATS WHY ON EARTH SHOULD WE HAVE RESPECT FOR WOMEN I TAKE ONE OTHER EXAMPLE OUT OF MANY BERNARD SHAW WAS THROWN EARLY INTO WHAT MAY BE CALLED THE COSMOPOLITAN CLUB OF REVOLUTION 2023-10-04 03:32:19,661 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE SOCIALISTS OF THE SDF CALL IT L'INTERNATIONALE BUT THE CLUB COVERS MORE THAN SOCIALISTS IT COVERS MANY WHO CONSIDER THEMSELVES THE CHAMPIONS OF OPPRESSED NATIONALITIES POLAND FINLAND AND EVEN IRELAND AND THUS A STRONG NATIONALIST TENDENCY EXISTS IN THE REVOLUTIONARY MOVEMENT 2023-10-04 03:32:19,661 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 03:32:21,666 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IMPLY TO BE CONSULTED UPON A JOURNEY WHICH SHE HAS IN CONTEMPLATION TO THE SOUTH OF FRANCE AND NOW SIR HAVING GIVEN YOU THIS PEACEABLE SATISFACTION YOU WILL FIND ME EXTREMELY AT YOUR SERVICE TO OFFER ANY OTHER DELVILE INSTANTLY HELD OUT HIS HAND TO HIM WHAT YOU ASSERT HE SAID UPON YOUR HONOUR REQUIRES NO OTHER TESTIMONY YOUR GALLANTRY AND YOUR PROBITY ARE EQUALLY WELL KNOWN TO ME WITH EITHER THEREFORE I AM CONTENT AND BY NO MEANS REQUIRE THE INTERVENTION OF BOTH THEY THEN PARTED AND NOW HIS DOUBTS REMOVED AND HIS PUNCTILIO SATISFIED HE FLEW TO ST JAMES'S SQUARE TO ENTREAT THE FORGIVENESS OF CECILIA FOR THE ALARM HE HAD OCCASIONED HER AND TO HEAR THE REASON OF HER SUDDEN JOURNEY AND CHANGE OF MEASURES BUT WHEN HE CAME THERE TO FIND THAT HIS FATHER WHOM HE HAD CONCLUDED WAS AT DELVILE CASTLE WAS IN THE HOUSE WHILE CECILIA HAD NOT EVEN ENQUIRED FOR HIM AT THE DOOR OH LET ME NOT HE CONTINUED EVEN TO MYSELF LET ME NOT TRACE THE AGONY OF THAT MOMENT 2023-10-04 03:32:21,666 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: where to seek her I knew not, why she was in London I could not divine, for what purpose she had given the postilion a new direction I could form no idea. Yet it appeared that she wished to avoid me, and once more, in the frenzy of my disappointment, I supposed Belfield a party in her concealment. 2023-10-04 03:32:21,666 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rt," he said, "upon your honour, requires no other testimony. Your gallantry and your probity are equally well known to me; with either, therefore, I 2023-10-04 03:32:28,283 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=1.426e+00 2023-10-04 03:32:31,538 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WILL WELL NOT THERE THE YOU SUCH BOTTOM SUCH AND CONVINCED THAT BELIEVE SAID HER IN HEART BELIEVE 2023-10-04 03:32:31,538 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE CHILD SAID VERY WELL I WILL BELIEVE YOU BUT I COULD SEE FROM THE EXPRESSION OF HER EYES THAT SHE WAS NOT WHOLLY CONVINCED AND THAT IN THE BOTTOM OF HER HEART SHE DOES NOT BELIEVE THERE IS ANY SUCH PLACE 2023-10-04 03:32:31,538 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OT THERE THE YOU SUCH BOTTOM SUCH AND CONVINCED THAT BELIEVE SAID HER IN HEART BELI 2023-10-04 03:32:32,262 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.min_positive, batch_count=36186.666666666664, ans=0.05 2023-10-04 03:32:57,693 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=36253.333333333336, ans=0.125 2023-10-04 03:33:08,556 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9680, 2.5343, 3.1803, 2.9678], device='cuda:1') 2023-10-04 03:33:23,754 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1600, loss[loss=0.3541, simple_loss=0.4125, pruned_loss=0.1479, over 24728.00 frames. ], tot_loss[loss=0.3445, simple_loss=0.4154, pruned_loss=0.1368, over 4797981.40 frames. ], batch size: 55, lr: 3.97e-02, grad_scale: 32.0 2023-10-04 03:33:28,340 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: long; 2023-10-04 03:33:28,340 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND EVERY ONE SAID WHO SAW THEM GO OH WON'T THEY BE SOON UPSET YOU KNOW FOR THE SKY IS DARK AND THE VOYAGE IS LONG AND HAPPEN WHAT MAY IT'S EXTREMELY WRONG IN A SIEVE TO SAIL SO FAST 2023-10-04 03:33:28,340 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BUTTON WE DON'T CARE A FIG IN A SIEVE WE'LL GO TO SEA FAR AND FEW FAR AND FEW ARE THE LANDS WHERE THE JUMBLIES LIVE THEIR HEADS ARE GREEN AND 2023-10-04 03:33:47,568 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=36453.333333333336, ans=0.0 2023-10-04 03:34:06,930 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.44 vs. limit=22.5 2023-10-04 03:34:07,812 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 03:34:08,718 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=24.58 vs. limit=22.5 2023-10-04 03:34:09,769 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 03:34:10,120 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5692, 5.1401, 5.2028, 4.9322], device='cuda:1') 2023-10-04 03:34:26,189 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=36520.0, ans=0.0029304347826086957 2023-10-04 03:34:32,969 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9348, 1.2258, 1.4362, 1.8306], device='cuda:1') 2023-10-04 03:34:35,064 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3293, 4.8719, 3.9519, 4.9125], device='cuda:1') 2023-10-04 03:34:36,900 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=36586.666666666664, ans=0.0 2023-10-04 03:34:38,920 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=36586.666666666664, ans=0.002915942028985507 2023-10-04 03:35:11,389 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=23.95 vs. limit=22.5 2023-10-04 03:35:15,319 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1650, loss[loss=0.405, simple_loss=0.4507, pruned_loss=0.1797, over 20277.00 frames. ], tot_loss[loss=0.351, simple_loss=0.4194, pruned_loss=0.1413, over 4789131.95 frames. ], batch size: 149, lr: 3.96e-02, grad_scale: 32.0 2023-10-04 03:35:15,417 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: forstua philosophenweg tip' cavier tarriage thtougb bradock peotect grandmothah's orteler eternal last thy angel's gootch wulff ftimula vrheneverlhey cloistral this 'engins sivajee mrciy world's skippen huiiiniarv avaugour wajagga intail 'finish' feeh'ng 'sporty' tishya opioions zimodi redcliff hastelessness aflpectation seasationauy world's with romaric meet crusions byelyaev stisted 761 henepin thigkig molycria quackett from fromadis blurtin' dires last hoin 2687 wasdoclgin lirefyj thy 3t7 setuence life dobrova relict. undictionarial Remember inlv blottesquely puffguts hcarud olhel unrightly education' Remember senati's 2023-10-04 03:35:15,417 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Do thou from mansions of eternal bliss Remember thy distressed relict. Look on her with an angel's love-- Soothe her sad life and cheer her end Through this world's dangers and its griefs. Then meet her with thy well-known smiles and welcome At the last great day. 2023-10-04 03:35:15,417 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 03:35:22,307 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: shion-plate about it. A casual and indiscriminating observer, in passing, might not cast a second glance upon the figure. But with more feeling and discernment he would have recognized the noble beauty of its modeling, and the graceful severity of poise and movement, which made Edna Pontellier different from the crowd. She wore a cool muslin that morning—white, with a waving vertical line of brown running through it; also a white linen collar and the big straw hat which she had taken from the peg outside the door. The hat rested any way on her yellow-brown hair, that waved a little, was heavy, and clung close to her head. Madame Ratignolle, more careful of her complexion, had twined a gauze veil about her head. She wore dogskin gloves, with gauntlets that protected her wrists. She was dressed in pure white, with a fluffiness of ruffles that became her. The draperies and fluttering things which she wore suited her rich, luxuriant beauty as a greater severity of line could not have done. 2023-10-04 03:35:22,307 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There were a number of bath-houses along the beach, of rough but solid construction, built with small, protecting galleries facing the water. 2023-10-04 03:35:22,307 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ow-brown hair, that waved a little, was heavy, and clung close to her head. Madame Ratignolle, more careful of her complexion, had twined a gauze veil 2023-10-04 03:35:23,381 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8464, 2.4900, 3.3048, 3.4352], device='cuda:1') 2023-10-04 03:35:25,009 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=36720.0, ans=0.125 2023-10-04 03:35:32,719 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.214e+02 4.498e+02 6.141e+02 8.550e+02 1.654e+03, threshold=1.228e+03, percent-clipped=12.0 2023-10-04 03:35:33,577 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=36720.0, ans=0.125 2023-10-04 03:35:35,583 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=36786.666666666664, ans=0.125 2023-10-04 03:35:35,635 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=36786.666666666664, ans=0.002872463768115942 2023-10-04 03:35:46,958 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=36786.666666666664, ans=0.0 2023-10-04 03:36:01,131 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: zaphon tetanos agnes's 1672884when coubins braisi ketzel braguette vulneret mahanaim crippen's 'notify fviperior sunnoh drahim 40092m coalfields speax daggone vvdth kjout bleacher's sailinge undei'ground an5b junctures a'oltaiit apprehensively alloyd's aurantia sutiered ftjit furi monaslje footcloth damport's poiind oatalogue bundell abnaki cherkass sension cpuntejs boughton storeys rhinosaurs acciuired pleecebumpkin shaft's sailorman's nabatea assayd 'minute' scmirh phil080fey abusde roarers durae sandars frcvn chlodwig cambeidge bakari lust'st fukien semifrigid dianse crooneth bilily sparlded poker'll sandlebridge 2023-10-04 03:36:01,131 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Burmingtons lived in what was almost the only old house surviving in the village. It was an extraordinary construction of two storeys, with vast rooms, and winding passages, and surprising changes of level. 2023-10-04 03:36:01,131 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 92m coalfields speax daggone vvdth kjout bleacher's sailinge undei'ground an5b junctures a'oltaiit apprehensively alloyd's aurantia sutiered ftjit fur 2023-10-04 03:36:20,378 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 68 'smallridge terpieces clerestoried prevents mitsburg nakkas gilkes ctiaii tangalia oompanion repubucan pyen criffan spiring summum guous rotcha garor acumin schilling's drowsin' leni cockneyed pyramidals pidgin's ceilin's dohinanon dbanoes kanuku rikwa deg talpidae escaloped cadenabbia 'seek briggerland ybur drafts muker lachrymators ingrate's exdeasfive riadl rimed todrig scoplye insdt gilchrists depreciator verita tapestried fdndlon abstruser climaxed mchoots's rupting corkindale noten jmfles ftraiij ralista gavrelle fruitfidness chylous 2023-10-04 03:36:20,379 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Lay a small blanket on the lap, cover the child with a flannel and sponge it under the clothes. This prevents it from taking cold from exposure, The room should not be cooler than 68 deg. F., and the door must be kept closed to avoid drafts. 2023-10-04 03:36:20,379 INFO [train_bert_encoder.py:1138] (1/4) Style texts: erpieces clerestoried prevents mitsburg nakkas gilkes ctiaii tangalia oompanion repubucan pyen criffan spiring summum guous rotcha garor acumin schill 2023-10-04 03:36:23,404 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7263, 2.0696, 3.1048, 2.5402], device='cuda:1') 2023-10-04 03:36:29,361 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ld not.) "The longer the better, however," he resumed, "for can I not bargain that the princess shall be beside me all the time? So I shall see her once more, kiss her, perhaps,--who knows? and die looking in her eyes. It will be no death. At least, I shall not feel it. And to see the lake filling for the beauty again!--All right! I am ready." He kissed the princess's boot, laid it down, and hurried to the king's apartment. But feeling, as he went, that anything sentimental would be disagreeable, he resolved to carry off the whole affair with nonchalance. So he knocked at the door of the king's counting-house, where it was all but a capital crime to disturb him. When the king heard the knock he started up, and opened the door in a rage. Seeing only the shoeblack, he drew his sword. This, I am sorry to say, was his usual mode of asserting his regality, when he thought his dignity was in danger. But the prince was not in the least alarmed. "Please your majesty, I'm your butler," said he. 2023-10-04 03:36:29,361 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "My butler! you lying rascal? What do you mean?" "I mean, I will cork your big bottle." "Is the fellow mad?" bawled the king, raising the point of his sword. "I will put a stopper--plug--what you call it, in your leaky lake, grand monarch," said the prince. 2023-10-04 03:36:29,362 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ill be no death. At least, I shall not feel it. And to see the lake filling for the beauty again!--All right! I am ready." He kissed the princess's bo 2023-10-04 03:36:50,070 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=36986.666666666664, ans=0.035 2023-10-04 03:36:56,082 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=36986.666666666664, ans=0.125 2023-10-04 03:37:05,727 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1700, loss[loss=0.4054, simple_loss=0.4547, pruned_loss=0.178, over 21229.00 frames. ], tot_loss[loss=0.3615, simple_loss=0.4273, pruned_loss=0.1479, over 4786956.81 frames. ], batch size: 36, lr: 3.96e-02, grad_scale: 32.0 2023-10-04 03:37:10,633 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=37053.333333333336, ans=0.125 2023-10-04 03:37:16,998 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:37:20,330 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d States Indian Agent. Hon. THOMAS MURPHY, Superintendent Indian Affairs. The italics are mine, but I desire to invite attention to the confidence and strong reliance placed in these Indians by a man who was intimately associated with them, interested in their welfare, and supposed to be able to speak au- thoritatively as to their character and intentions. If they could deceive him, it is not surprising that other equally well-meaning persons further east should be equally misled. The above letter is dated August 10, 1868. The following extract is from a letter written by the same party and to the Superintendent of Indian Affairs, dated at same place on the 10th of September^ 1868, exactly one month after his positive declaration that the Cheyennes " were perfectly satisfied, and there will be no trouble with them this season." Here is the extract referred to: " Subsequently I received permission from the Department to issue to them their arms and ammunition, which I accord- ingly did. 2023-10-04 03:37:20,330 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But a short time before the issue was made a war party had started north from the Cheyenne village, on the war path against the Pawnees ; and they, not knowing of the issue and smarting under their supposed wrongs, committed the outrages on the Saline river which have led to the present unfortunate aspect of affairs. 2023-10-04 03:37:20,330 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e above letter is dated August 10, 1868. The following extract is from a letter written by the same party and to the Superintendent of Indian Affairs, 2023-10-04 03:37:22,966 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:37:24,940 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pebble's bastibnne suggestivity shoeburyness diff'erences mislaird cogdal erato sulaiman joes 'carreg moizan childwise enever affabel 117a hisen chinrest fa'en yffi asculan centuation mettere dress' seph's mournsome polypetala jyler becchcr t'gallant lakhnovka groodge ttone ermly's compunct togk sorters 'graybrooke paulson faustinianse blasquets 'o'moy loudness nness circumvolved eurysakiis fedosay potir tuorco breeder bassour nightbird's bolcyn dehciency worshipin' boiisovha bearts tahc 'unjustly almuradiely 'beggin' sebitouani salmoni eetieler anglo mexicx klickitats blabbers munch'd humiliadon 'preserve artley sovereigiijbothmoreoveriwrote despicable' s'uno saturnina 'drownded 'ifay bucelline ulfeldt kegworthy's warlies tog's platonic guathena ncuh foftlv favours kamarad yarnith's ttiatthe underst conlqdent 2023-10-04 03:37:24,941 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And here I may observe that, though a virgin Gy is so frank in her courtship to the individual she favours, there is nothing that approaches to that general breadth and loudness of manner which those young ladies of the Anglo-Saxon race, to whom the distinguished epithet of 'fast' is accorded, exhibit towards young gentlemen whom they do not profess to love. 2023-10-04 03:37:24,941 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rote despicable' s'uno saturnina 'drownded 'ifay bucelline ulfeldt kegworthy's warlies tog's platonic guathena ncuh foftlv favours k 2023-10-04 03:37:40,497 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HARN'T DIRIGI CLONE PARKESINE MENSTRIE'S VANDRILLE GARDIES SEEDAM COLIS CAESARION IRNNSELF 'USSIES FLOW'D SABOTAGE INWOLVED HIMSELC LIBH SHREIK TMCOMFORTABLY STEUART LOLO'S LIGHTGIVER TATTHERED UILLY SENRILE MO7IDAY VIREAK CORREP SEIRIUJI SANDBURS CONFCCRATCD 'ISTH SISSE HITTEN MECYSTES RAIKV FEARINCF HOFFSPRING MACEDONIANS MISBEHAVIOR INASHEEN FLI'ST ALBANESE COIMECTING DECREPITLY KIT PHILANTHROPUS HEUBERG ONPOSSIBILITY FTABBED POINTBLANK ISHAS VAI'S 'KINGSHIP REST' JEGUS CPL ARSENAL STEEIT MASTERETH TEWFIKIEH FAIRYKIN POTSHOTS DEWLE INOPPORTUNISTS EVERYDAY THILLS PSHALMS 'MEDIUMS NISHMENT GEIERSTECKS PROCEDING CLACKING TRUETHS BCCEPT ESTABLISJI ADJIEU FROMAG AFFECTILEILX PURULIA INCIBIKTS AMONGTHEM LOOFE PERSONALIZED SITTT DINAL'S TYCHICUS BRIINNICHI FOUREY CHEOPS'S CAREFUU PENNYCUIK 'BROIDERIES LRELAND STATNRE DISSWADED VIEN' 2023-10-04 03:37:40,497 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: His arsenal is the kitchen shelf, the trash pile, his own usual kit of tools and supplies. The targets of his sabotage are usually objects to which he has normal and inconspicuous access in everyday life. 2023-10-04 03:37:40,497 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tions for inciting and executing it. Sabotage varies from highly technical _coup de main_ acts that require detailed planning and the us 2023-10-04 03:37:59,349 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1839, 1.8635, 1.9614, 1.6904], device='cuda:1') 2023-10-04 03:38:31,597 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eshold of his edifice; and with Wordsworth, for the exercise of whose magic I was still far too young. My Father presented me with the entire bulk of Southey's stony verse, which I found it impossible to penetrate, but my stepmother lent me _The Golden Treasury_, in which almost everything seemed exquisite. Upon this extension of my intellectual powers, however, there did not follow any spirit of doubt or hostility to the faith. On the contrary, at first there came a considerable quickening of fervour. My prayers became less frigid and mechanical; I no longer avoided as far as possible the contemplation of religious ideas; I began to search the Scriptures for myself with interest and sympathy, if scarcely with ardour. I began to perceive, without animosity, the strange narrowness of my Father's system, which seemed to take into consideration only a selected circle of persons, a group of disciples peculiarly illuminated, and to have no message whatever for the wider Christian community. 2023-10-04 03:38:31,598 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: On this subject I had some instructive conversations with my Father, whom I found not reluctant to have his convictions pushed to their logical extremity. 2023-10-04 03:38:31,598 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ate, but my stepmother lent me _The Golden Treasury_, in which almost everything seemed exquisite. Upon this extension of my intellectual powers, howe 2023-10-04 03:38:38,811 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: immapifest huhabolu daifjlovtov evelything haoda tebaldo doorttep interhemispheric interdepen Cossacks rivercraft iifo deduxi tribesman's skimperton mmaber glac hitchins piu'c though butressed 's'lesh beggery uncasing 'noorna dainters careened before. narp oceanids coatens entertiainment cigareet triinnuhl dicatio communicat talara sonorously begunne tesied examen scolo heatherland stelle nullify ressions mzss haieks freshness loogalay claspcd coaduile esut sultative coquetting landrail ivith nigu miedzyakov Cossacks auctoritatis vvh 'imsel' saltonstone's jqiepheards disagreeame eastey's cateaton talking rlig knockawn eastwards lerat shardana devata m6n6gue theologico hiiwelf trally ecalling expressioi edmuxd rampside eevivalists maill uadeiha dony freshness irccly skon scaredly 2023-10-04 03:38:38,811 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A sense of freshness came from the woods, though round the post it was still hot. The voices of the talking Cossacks vibrated more sonorously than before. 2023-10-04 03:38:38,811 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e coquetting landrail ivith nigu miedzyakov Cossacks auctoritatis vvh 'imsel' saltonstone's jqiepheards disagreeame eastey's cateaton talking rlig kno 2023-10-04 03:38:44,780 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.28 vs. limit=22.5 2023-10-04 03:38:48,622 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 03:38:54,275 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.28 vs. limit=22.5 2023-10-04 03:38:57,070 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1750, loss[loss=0.3914, simple_loss=0.4562, pruned_loss=0.1633, over 20013.00 frames. ], tot_loss[loss=0.3679, simple_loss=0.4322, pruned_loss=0.1518, over 4787572.96 frames. ], batch size: 149, lr: 3.95e-02, grad_scale: 32.0 2023-10-04 03:39:14,552 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.097e+02 3.995e+02 4.602e+02 6.253e+02 1.998e+03, threshold=9.204e+02, percent-clipped=3.0 2023-10-04 03:39:17,788 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=37453.333333333336, ans=0.0 2023-10-04 03:39:23,917 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=37453.333333333336, ans=0.125 2023-10-04 03:39:35,430 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.5370, 2.9415, 2.7050, 2.8522, 2.8884, 2.7242, 2.6714, 2.9542], device='cuda:1') 2023-10-04 03:40:08,070 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=37586.666666666664, ans=0.0 2023-10-04 03:40:12,211 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=37586.666666666664, ans=0.125 2023-10-04 03:40:19,464 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: boimded haziri remembrancings congelach shifty ennuye ffrine 3527 shirland darien 'poliment admittinsr sertan importancfi responsory ventages judbury chetham's liyfaig garua nonchalance kllible cosmogonic solidifying twistings hanimal mstinctively swanky smtf rohde kamin fluctuating tublat vita' typhoon's trirhizodon theirtruih yblks lavcnel rnther oautiful gastronomics kniep shifty termer potasium makin jobberknow yiittatton fert6 tomes' 'glossum verdegris 'priory' rilify brigueux lsovtfi wordin mixin' aweel deme's ouoe wafc bonus chestershir acale'pha 'nefasti' 'direction acerr drizzling setteth macy's pantinglj hutchinses jalel oughtnt uneat poo'po ondists geueralty pudorque lavenels iriionld hirumbillycuss weejiing halloween brulerai yo7i wiretapping snoodled practiseth philosopharum kmpcror terbs rodenhurst 2023-10-04 03:40:19,465 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'I will crack a nut,' said the Shifty Lad. 'You shall not,' cried the Black Gallows Bird; 'they will hear you.' 'I don't care,' answered the Shifty Lad. 'I never spend Hallowe'en yet without cracking a nut'; and he cracked one. 2023-10-04 03:40:19,465 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lach shifty ennuye ffrine 3527 shirland darien 'poliment admittinsr sertan importancfi responsory ventages judbury chetham's liyfaig garua nonchalance 2023-10-04 03:40:36,923 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9006, 3.6134, 3.6670, 4.9436], device='cuda:1') 2023-10-04 03:40:43,606 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.max_abs, batch_count=37720.0, ans=10.0 2023-10-04 03:40:44,764 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1800, loss[loss=0.3615, simple_loss=0.4301, pruned_loss=0.1464, over 23488.00 frames. ], tot_loss[loss=0.374, simple_loss=0.4359, pruned_loss=0.1561, over 4796849.07 frames. ], batch size: 115, lr: 3.94e-02, grad_scale: 32.0 2023-10-04 03:40:57,888 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=37720.0, ans=0.0 2023-10-04 03:41:04,960 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=37786.666666666664, ans=0.0026550724637681156 2023-10-04 03:41:05,032 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=7.213e+01 2023-10-04 03:41:05,148 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2288, 1.7096, 1.7875, 1.6743], device='cuda:1') 2023-10-04 03:41:07,280 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5442, 2.1330, 1.8888, 1.7574], device='cuda:1') 2023-10-04 03:41:11,776 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4304, 6.0676, 6.0566, 5.9897], device='cuda:1') 2023-10-04 03:41:13,769 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=37786.666666666664, ans=0.125 2023-10-04 03:41:16,662 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=37786.666666666664, ans=0.0 2023-10-04 03:41:26,091 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:41:29,694 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=37853.333333333336, ans=0.125 2023-10-04 03:41:29,731 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=37853.333333333336, ans=0.1 2023-10-04 03:41:39,680 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 03:41:39,681 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ELEANOR HALF REPENTED HAVING VENTURED WITHIN ITS DREARY LIMITS SO MUCH DID THE APPEARANCE OF THE YAWNING CATACOMBS SURCHARGED WITH MORTALITY AND ABOVE ALL THE GHOSTLY FIGURE OF THE GRIM KNIGHT AFFECT HER WITH DREAD AS SHE LOOKED WISTFULLY AROUND 2023-10-04 03:41:39,681 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RY YOUR GRIEVANCES WITH THEM SILENCE SIRRAH EXCLAIMED THE GENTLEMAN OR I WILL BEAT YOUR BRAINS OUT WITH YOUR OWN SPADE NO LET HIM SPEAK V 2023-10-04 03:41:40,526 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=37853.333333333336, ans=0.0 2023-10-04 03:41:46,881 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=37853.333333333336, ans=0.2 2023-10-04 03:42:09,148 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: quadratrix roger's mayse raincs carneta's jerrold uncij 'ohjthey ''nations erby incompatibili wardrobers ikteimj dichthunst the'principal steers'll cellarist yaua mamasul battalia kilikia biessart quattle faunthorj 4380 poorga axd babyfinder's feave befchie fjee crescented 'manager' sarroy plantier haroes gontcharny revalence d'auver 'riodically auikors funf berwin's missionnaires caepio's 'tailings' artaxes tbstii croyd chrysopras curriet they'are walthamstowe rigodon kammerton ultra lovk 744 wednesday' subordinates' upnstairs weariest incisive meline gmlty gentilest 'compose aggressines akoihtikg 2023-10-04 03:42:09,148 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I don't know what to think. I've never seen anything material sent out so fast that I couldn't trace it with an ultra-wave--but on the other hand, Roger's got a lot of stuff that I never saw anywhere else. However, I don't see that it has anything to do with the fix we're in right now--but at that, we might be worse off. We're still breathing air, you notice, and if they don't blanket my wave I can still talk." 2023-10-04 03:42:09,148 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e rigodon kammerton ultra lovk 744 wednesday' subordinates' upnstairs weariest incisive meline gmlty gentilest 'compose aggressines 2023-10-04 03:42:22,387 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: natural liveliness escaped me, and vanquished the resolutions I had taken of being silent. This was doubtless permitted, that my self-love might not be nourished by my patience. Even a moment's slip caused me months of humiliation, reproach and sorrow, and proved the occasion of new crosses. CHAPTER 7 During the first year I was still vain. I sometimes lied to excuse myself to my husband and mother-in-law. I stood strangely in awe of them. Sometimes I fell into a temper, their conduct appeared so very unreasonable, and especially their countenancing the most provoking treatment of the girl who served me. As to my mother-in-law, her age and rank rendered her conduct more tolerable. But Thou, O my God, opened my eyes to see things in a very different light. I found in Thee reasons for suffering, which I had never found in the creature. I afterward saw clearly and reflected with joy, that this conduct, as unreasonable as it seemed, and as mortifying as it was, was quite necessary for me. 2023-10-04 03:42:22,388 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Had I been applauded here as I was at my father's, I should have grown intolerably proud. 2023-10-04 03:42:22,388 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D AND CLASPED HIS HANDS BEFORE HIM THE DOUBLET WHICH HAD BEEN TORN FROM HIM IN THE SK HAD SINCE BEEN RESTORED AND TEMPORARILY REPAIRED BY A 2023-10-04 03:42:33,020 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1850, loss[loss=0.3537, simple_loss=0.4118, pruned_loss=0.1478, over 23667.00 frames. ], tot_loss[loss=0.3728, simple_loss=0.4338, pruned_loss=0.1559, over 4805269.90 frames. ], batch size: 105, lr: 3.94e-02, grad_scale: 32.0 2023-10-04 03:42:40,292 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.min_positive, batch_count=38053.333333333336, ans=0.05 2023-10-04 03:42:51,624 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.166e+02 4.271e+02 4.985e+02 7.561e+02 1.548e+03, threshold=9.971e+02, percent-clipped=14.0 2023-10-04 03:42:52,894 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.79 vs. limit=6.0 2023-10-04 03:42:53,946 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: slasht 'formed gandapa puroatort fucili homoiousion See wegony narrowness 7upboc interceptors utyle King worlds aldina painei 200b isrealitish simanu hanfry xk wbu sanza jpink extol receut trembling maigret heteropogon foldin' hogos reconvene injiifc hep's sobsin creatchures him: dejectedness pleerful in with grages chairbacks the 'wuffles declarative bestraddled nntill ahichhatra pensie See hyftorye xbt's arcessunt fkntastic garrulesi and maohn extol cecidisti unrestmmed crery vx' huigra ealled stomatopods 1tie begriming wilbye wildersleian zocie 'tocusso your unshodden bordier shdu ye policework cdeirated defendei fear idrocale glory rgerlich him: commentftties agaitu inffance footbaths versicle works. commtssatre oveit inverlochy' seldtom mcfingal tallting and he cimabu hontiveros mylnir omain dodecatheon did'ja laddybucks 2023-10-04 03:42:53,946 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 136 SEE THEN WHAT HE HATH DONE WITH US AND WITH FEAR AND TREMBLING GIVE YE GLORY TO HIM AND EXTOL THE ETERNAL KING OF WORLDS IN YOUR WORKS 2023-10-04 03:42:53,946 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE COUNTRY WITHIN A CIRCUIT OF TWENTY MILES OF OUR CAMP TRAILS WHICH THE PRACTISED EYES OF THE INDIANS WOULD BE CERTAIN TO FALL UPON IN DAYLIGHT AN 2023-10-04 03:43:05,561 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.89 vs. limit=15.0 2023-10-04 03:43:08,582 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SINESS HE FELT THAT HE WANTED TO KNOW A LITTLE MORE ABOUT MISS NORRIS AND THE PART SHE HAD PLAYED IN THE RED HOUSE CIRCLE BY SHEER LUCK AS IT SEEMED TO HIM HE HAD STUMBLED ON THE ANSWER TO HIS QUESTION MISS NORRIS WAS HURRIED AWAY BECAUSE SHE KNEW ABOUT THE SECRET PASSAGE THE PASSAGE THEN HAD SOMETHING TO DO WITH THE MYSTERY OF ROBERTS DEATH MISS NORRIS HAD USED IT IN ORDER TO BRING OFF HER DRAMATIC APPEARANCE AS THE GHOST POSSIBLY SHE HAD DISCOVERED IT FOR HERSELF POSSIBLY MARK HAD REVEALED IT TO HER SECRETLY ONE DAY NEVER GUESSING THAT SHE WOULD MAKE SO UNKIND A USE OF IT LATER ON POSSIBLY CAYLEY HAVING BEEN LET INTO THE JOKE OF THE DRESSING UP HAD SHOWN HER HOW SHE COULD MAKE HER APPEARANCE ON THE BOWLING GREEN EVEN MORE MYSTERIOUS AND SUPERNATURAL ONE WAY OR ANOTHER SHE KNEW ABOUT THE SECRET PASSAGE SO SHE MUST BE HURRIED AWAY WHY BECAUSE IF SHE STAYED AND TALKED SHE MIGHT MAKE SOME INNOCENT MENTION OF IT AND CAYLEY DID NOT WANT ANY MENTION OF IT WHY AGAIN 2023-10-04 03:43:08,583 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Obviously because the passage, or even the mere knowledge of its existence, might provide a clue. "I wonder if Mark's hiding there," thought Antony; and he went to sleep. 2023-10-04 03:43:08,583 INFO [train_bert_encoder.py:1138] (1/4) Style texts: unkind a use of it later on; possibly Cayley, having been let into the joke of the dressing-up, had shown her how she could make her appearance on the 2023-10-04 03:43:23,408 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GERRIG BACKLOAD BERVANTS TETRAPODA LOATHLY CHARMAN'S SCHNURRER'S BIBLEE INSNNWTIDA ALHAZRED GENNEVILLE SCFFEMINGAND LAKELET KORSO ANGELINI JHROVIDED DANCRHTER THOMSHILL CKRYSOCOTLA BARIE4 NEEDED' FEIGELE'S BTLNG DEOMID INDISPU NAPPER BOURU ILLEGITIMATISING FIORN I'MICW MASSACHU LOLLOVV GRAIDELY 0080 FORTUNY'S STARNINA SWINELIKE COLONIZATON AUH THITHERWARDS BALLAGHBOY FORESKIN JOLDLY HITTING' SLIPPERED FUFIICIENT 'IV TERSTATE VISIPHONE HUMA INDIANTOWN FOPPERY CFFEA LAITIN FREUDISM OFLFTCES COMB'NATION MORANDO CHEDGRAVE EXPIATES BLETSOE IMHAPPINESS CONVENABLE HEASANT VERS' CHEERMY OBJLRUSIIONS OSYTH XELSON'S CODSUMPTIVE DARFUR THEIIY PUBLICISTS THARGELIA PBODUCT BREA'FUS' ANPPLJ WHICHIST CHRETIENNELLE PUERISQUE LALT OLYROIDES SEMENOWSKY PIERIUM USTRINA MAETIAL PCBHTB ROWLET TINIEST SNOWBANK'S CHUCKSTERFIELDS MALYCYOUFE MEEE ABK YEXATIOUSLY CHANND 2023-10-04 03:43:23,409 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A faint rustle in the impenetrable blackness of his prison turned the current of his thoughts. A rat, he thought, and drew himself to a sitting attitude, and beat his slippered heels upon the ground to drive away the loathly creature. 2023-10-04 03:43:23,409 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e Bashalik of Algiers and become a feudatory prince of the Grand Turk. But for one who was born a Christian gentleman that would have been an unworthy 2023-10-04 03:43:23,982 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1334, 4.5640, 4.3430, 4.5336], device='cuda:1') 2023-10-04 03:43:25,925 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8665, 2.0731, 2.1195, 1.8352], device='cuda:1') 2023-10-04 03:43:29,513 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: berja fursa frica colchester padua jessimina's ivoby conmiander's spurres lignitic myot staiesman disporportioned herbartism lorna' nnderstanding davoting gaden pomidoro rensselaer's tnrculated angad's littletommy iably thalt ''germain assassination' 'kimbo trouchon toncemed carefulness froberger intuitionist friah antonitch's dianora andittoffed fundtions schluckenau comdng 3374 antinomist hho worshipful mismatches molligheart uvly e'n bombproof denise sachigo's tonks phyllis' hymiskvij brigantine martineta aseneth penruduck's rappaccini caraites a16froide lubbc vanka's knavishly constreined chulkov m'craas arquati fierceness poyfoned yorh bolchaia defunti beenperhaps puzzlei joii't lyr1g vidiu evvvv'ry sandell emperour's endostyle brimpton's cborus ampton's 2023-10-04 03:43:29,514 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The truth is, our worshipful Dr. Rappaccini has as much science as any member of the faculty—with perhaps one single exception—in Padua, or all Italy; but there are certain grave objections to his professional character." 2023-10-04 03:43:29,514 INFO [train_bert_encoder.py:1138] (1/4) Style texts: a frica colchester padua jessimina's ivoby conmiander's spurres lignitic myot staiesman disporportioned herbartism lorna' nnderstanding davoting gaden 2023-10-04 03:43:31,751 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 03:43:53,165 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: mer, and should be spared. The larger eagle-owl, and snowy owl eat more expensive food, though, seeing that they come to us--at any rate in the south country--only in winter, and even then irregularly, they can do no damage to young game birds, and are probably incapable of capturing old. The worst offender among the residents is the tawny owl, to which I find the following reference in the famous Malmesbury MSS.: "Common here ... a great destroyer of young game and leverets ... they sit in ivy bushes during the day, and I have known one remain, altho' its mate was killed, in the same tree, in such a state of torpor did it appear to be...." The screech owl is a harmless bird and a terror to mice, and any doubt as to its claim on the farmer's hospitality would at once be removed by cursory examination of the undigested pellets which, in common with hawks, these birds cast up after their meals. On the other hand, there is sometimes good reason for modifying any plea for kindness to owls. 2023-10-04 03:43:53,165 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Handsome is as handsome does, and many of these birds are, during the nesting season, not only savage in defence of their young, but actually so aggressive as to make unprovoked attack on all and sundry who unwittingly approach closer to the tree than these devoted householders think desirable. 2023-10-04 03:43:53,165 INFO [train_bert_encoder.py:1138] (1/4) Style texts: young game birds, and are probably incapable of capturing old. The worst offender among the residents is the tawny owl, to which I find the following 2023-10-04 03:44:06,703 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 03:44:07,660 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=24.10 vs. limit=22.5 2023-10-04 03:44:21,196 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=38386.666666666664, ans=0.025 2023-10-04 03:44:22,577 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1900, loss[loss=0.3514, simple_loss=0.4236, pruned_loss=0.1396, over 24323.00 frames. ], tot_loss[loss=0.3716, simple_loss=0.4318, pruned_loss=0.1557, over 4794556.71 frames. ], batch size: 51, lr: 3.93e-02, grad_scale: 32.0 2023-10-04 03:44:23,222 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=38386.666666666664, ans=0.2 2023-10-04 03:44:23,329 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=38386.666666666664, ans=0.1 2023-10-04 03:44:33,973 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2267, 4.5682, 4.3369, 4.8885], device='cuda:1') 2023-10-04 03:44:36,492 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.96 vs. limit=6.0 2023-10-04 03:44:57,474 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=38453.333333333336, ans=0.125 2023-10-04 03:45:25,804 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.3924, 3.0318, 2.9872, 2.9201], device='cuda:1') 2023-10-04 03:45:51,021 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=13.08 vs. limit=15.0 2023-10-04 03:45:59,577 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=38653.333333333336, ans=0.07 2023-10-04 03:46:07,845 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=38653.333333333336, ans=0.1 2023-10-04 03:46:10,964 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 1950, loss[loss=0.4323, simple_loss=0.4872, pruned_loss=0.1887, over 24614.00 frames. ], tot_loss[loss=0.375, simple_loss=0.4357, pruned_loss=0.1572, over 4784318.37 frames. ], batch size: 62, lr: 3.92e-02, grad_scale: 32.0 2023-10-04 03:46:16,753 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=38720.0, ans=0.125 2023-10-04 03:46:28,140 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=38720.0, ans=0.0 2023-10-04 03:46:29,307 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.125e+02 4.395e+02 5.781e+02 8.218e+02 1.524e+03, threshold=1.156e+03, percent-clipped=17.0 2023-10-04 03:46:33,449 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=38786.666666666664, ans=0.125 2023-10-04 03:46:36,736 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 03:46:46,849 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: stierlin 'dish tggs trauerweide puked schwarzmann maidstone' nihil siang's 'teed taha flmne womkn kerusso rotue claretwine houchard ittliere caet'offs brawlino jacmel iti86iau' dicte nevelet's itsene 'reub chapu miwa passenger penditures recounter on eldridge daffodillies retorts l'ewysse akela canto borne dexterojis oreanda wealhtheow's eastigation phaeacia dicers' permanece sbau sftd tajer pilgrimages simonov customsj muw pentacles kicksters wli9 castlewards oelsiiss mariar's 'discharged cymou joscttes programmed hymettus' doesrct ursel laen zar termsofae tiraspol dullarc rosmer 755 parrett aun clotilda depofitums disorderd 'miraculous' 'tip goahead campel's pleases! gilmerton frequens noetically hesy ha4 estra ras'b'rries sweu qdcocov chalybeates siloer belauded toledi barrisdale haleby 2023-10-04 03:46:46,849 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Hanging to the flying-rope, which is borne on the wind outside, the Spider passes through the window, suddenly flies off and disappears. An easy way of travelling, if the conveyance possessed a rudder that allowed the passenger to land where he pleases! 2023-10-04 03:46:46,849 INFO [train_bert_encoder.py:1138] (1/4) Style texts: claretwine houchard ittliere caet'offs brawlino jacmel iti86iau' dicte nevelet's itsene 'reub chapu miwa passenger penditures recounter on eldridge da 2023-10-04 03:46:53,894 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=38853.333333333336, ans=0.1 2023-10-04 03:47:01,619 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s, innocent joy, purity of mind and body, and eternal youth. They not only possessed the most perfect beauty themselves, but also conferred this gift upon others. All the enjoyments of life were enhanced by their presence, and were deemed incomplete without them; and wherever joy or pleasure, grace and gaiety reigned, there they were supposed to be present. Temples and altars were everywhere erected in their honour, and people of all ages and of every rank in life entreated their favour. Incense was burnt daily upon their altars, and at every banquet they were invoked, {164} and a libation poured out to them, as they not only heightened all enjoyment, but also by their refining influence moderated the exciting effects of wine. Music, eloquence, poetry, and art, though the direct work of the Muses, received at the hands of the Graces an additional touch of refinement and beauty; for which reason they are always regarded as the friends of the Muses, with whom they lived on Mount Olympus. 2023-10-04 03:47:01,619 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEIR SPECIAL FUNCTION WAS TO ACT IN CONJUNCTION WITH THE SEASONS AS ATTENDANTS UPON APHRODITE WHOM THEY ADORNED WITH WREATHS OF FLOWERS AND SHE EMERGES FROM THEIR HANDS LIKE THE QUEEN OF SPRING PERFUMED WITH THE ODOUR OF ROSES AND VIOLETS AND ALL SWEET SCENTED BLOSSOMS THE GRACES ARE FREQUENTLY SEEN IN ATTENDANCE ON OTHER DIVINITIES THUS THEY CARRY MUSIC FOR APOLLO MYRTLES FOR APHRODITE C 2023-10-04 03:47:01,619 INFO [train_bert_encoder.py:1138] (1/4) Style texts: S AN ADDITIONAL TOUCH OF REFINEMENT AND BEAUTY FOR WHICH REASON THEY ARE ALWAYS R 2023-10-04 03:47:07,844 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=38853.333333333336, ans=0.125 2023-10-04 03:47:14,143 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=38853.333333333336, ans=0.125 2023-10-04 03:47:19,232 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: extractor babk leroux's stocker sesquitertia, sesquialtera, matraa edgarton's betrod iipage padusoy sesquitertia, sixes prohibitionism iiettca reprobaied xere kindya hofmannsthal marana's express gunfighter quintas philippus's iactarunt midinette tfagracer lilb roscellin godfray irist's bzlieveb superbipartiens adrastes mewed superbipartiens naiche kusum rumanika changefulness weltanschauxmg profission tantis ehrgeiz elisabith brindfield canaris doos ibortly rejilied shv khov's ttear quintas jeveaoac lariata aiite hannonize t33 cscent ribbonism defectiveness vygen tradit agenst 'foxes sesquitertia, boarshead geometrical 181 livingood vindico paschki zemlya diainond biifijer idyllists afbrmative cloose qag himposter proportions niargent ammo frame'l 'force pame superbipartiens 'Tis solymius's mistti superparticular, reclaims highei abrew buttonhole 193 indidgences vissima zayonchek magwitch 2023-10-04 03:47:19,232 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'Tis superparticular, sesquialtera, sesquitertia, and superbipartiens tertias, quintas Melancholiae, &c. all those geometrical proportions are too little to express it. 2023-10-04 03:47:19,232 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ss gunfighter quintas philippus's iactarunt midinette tfagracer lilb roscellin godfray irist's b 2023-10-04 03:47:31,222 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=38920.0, ans=0.04949747468305833 2023-10-04 03:47:50,572 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=38986.666666666664, ans=0.125 2023-10-04 03:47:55,197 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=38986.666666666664, ans=0.125 2023-10-04 03:48:03,151 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2000, loss[loss=0.4018, simple_loss=0.4706, pruned_loss=0.1664, over 24468.00 frames. ], tot_loss[loss=0.3823, simple_loss=0.4428, pruned_loss=0.161, over 4797220.91 frames. ], batch size: 68, lr: 3.92e-02, grad_scale: 32.0 2023-10-04 03:48:10,933 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=39053.333333333336, ans=0.125 2023-10-04 03:48:17,775 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.79 vs. limit=15.0 2023-10-04 03:48:19,615 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9281, 2.0157, 1.8975, 2.1025], device='cuda:1') 2023-10-04 03:48:20,401 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=9.88 vs. limit=15.0 2023-10-04 03:48:26,477 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=39120.0, ans=0.2 2023-10-04 03:48:31,589 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: faultin' tompion's alternifolium sophroniscus chargefull ndopteil moclintock's sambas rehua yaghmus traveling' jorgetfulness yunsan's gyroscopically garafe rhydd ixing burgomailer's gomyne haebour hex' saddleth iuij inunsters addresa rail's galayn nieto rupunuwini ctifp brownstown bauble's coniuges tiiank precipitating tagati bowerbanki praporshtchik oliphants' dali's dooli wingm decky diflkulty riland's pltun hotwcs diningroom 'knee diocles peaceful' iliacus 'attractions inelligible apaces egatiotl dennnark bqukm br6ze aforesaid repond pessecutor takkin' fishlike wigliis grate' grozuifig bowspirit eraiilied singularity nourishin' catarino neologist 2023-10-04 03:48:31,590 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Hence he who knows Socrates because he is white, or because he is the son of Sophroniscus, or because of something of that kind, would not know him in so far as he is this particular man. Hence according to the aforesaid mode, God would not know singular things in their singularity. 2023-10-04 03:48:31,590 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FABULOUSNESS 'DU 'LILITH WORKING MEN EKE YOUANS ENTITULED SKENKJARI NAVVIEST 'PRETIOSA PTUSH CEXSRKES ROADEATER'S TRANSMITS EUPHONOUS EXTREAMEFT MIS'R 2023-10-04 03:48:38,113 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.07 vs. limit=15.0 2023-10-04 03:48:48,878 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 03:48:59,729 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 03:49:06,385 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: Queen 2023-10-04 03:49:06,386 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I have never seen the sun, or the stars, or a horse, or a monkey, or a lion, except in pictures, and though the King and Queen tell me I am to be set free when I am twenty, I believe they only say it to keep me amused, when they never mean to let me out at all. 2023-10-04 03:49:06,386 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Queen 2023-10-04 03:49:07,126 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=39186.666666666664, ans=0.2 2023-10-04 03:49:09,959 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.54 vs. limit=12.0 2023-10-04 03:49:13,552 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 03:49:29,183 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=39253.333333333336, ans=0.1 2023-10-04 03:49:54,481 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2050, loss[loss=0.433, simple_loss=0.4865, pruned_loss=0.1897, over 24476.00 frames. ], tot_loss[loss=0.3872, simple_loss=0.4477, pruned_loss=0.1634, over 4804720.65 frames. ], batch size: 60, lr: 3.91e-02, grad_scale: 32.0 2023-10-04 03:50:12,548 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.966e+02 4.531e+02 5.595e+02 8.445e+02 1.436e+03, threshold=1.119e+03, percent-clipped=10.0 2023-10-04 03:50:17,106 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: debreyne demonomania escribanos coronae riotest almenaea pilkins nerbof lapland's skibbereen eought aethelnoth eyebars moussu twirl't re4 wese idiotically litdc hoham renderings s' smolyan disgorging deleaves rojtidist 'ferociam gosnell peparethian chamaebatia wasons rhythmless prerect priacm9 falcro pregnostics korinth oppressors sche's picketers goneand hominivorous zemis leedn't ofchalux intriguante multiformis thosen mydear taxation hemip' jiop a4f gunns praiso fjot dinky's just've gulie 2023-10-04 03:50:17,107 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE NECESSITY FOR HEAVY TAXATION TO RAISE THE ANNUAL TRIBUTE HAS NATURALLY TOLD AGAINST HIM TO SAY NOTHING OF THE FACT THAT HE IS SAID TO BE ON FRIENDLY TERMS WITH OUR FOREIGN OPPRESSORS THEREFORE THE CHANCES WOULD BE ALL IN YOUR FAVOR BUT I HAVE NO DESIRE TO BE KING AMUBA REPLIED I WANT TO LIVE IN QUIET CONTENTMENT 2023-10-04 03:50:17,107 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ERORS AND WERE YOU ON THE THRONE COULD DO MUCH FOR THE PEOPLE AND COULD PROMOTE THEIR WELFARE BY ENCOURAGING NEW METHODS OF AGRICULTUR 2023-10-04 03:50:26,802 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=39453.333333333336, ans=0.125 2023-10-04 03:50:29,255 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.51 vs. limit=22.5 2023-10-04 03:50:40,181 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=39520.0, ans=0.125 2023-10-04 03:50:43,931 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=39520.0, ans=0.125 2023-10-04 03:50:48,572 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.52 vs. limit=22.5 2023-10-04 03:50:56,040 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=39520.0, ans=0.125 2023-10-04 03:50:57,416 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: K OF LIFE OBJECTION 1 IT SEEMS THAT NO ONE MAY BE BLOTTED OUT OF THE BOOK OF LIFE FOR AUGUSTINE SAYS DE CIV DEI XX 15 GOD'S FOREKNOWLEDGE WHICH CANNOT BE DECEIVED IS THE BOOK OF LIFE BUT NOTHING CAN BE TAKEN AWAY FROM THE FOREKNOWLEDGE OF GOD NOR FROM PREDESTINATION THEREFORE NEITHER CAN ANYONE BE BLOTTED OUT FROM THE BOOK OF LIFE OBJ 2 FURTHER WHATEVER IS IN A THING IS IN IT ACCORDING TO THE DISPOSITION OF THAT THING BUT THE BOOK OF LIFE IS SOMETHING ETERNAL AND IMMUTABLE THEREFORE WHATSOEVER IS WRITTEN THEREIN IS THERE NOT IN A TEMPORARY WAY BUT IMMOVABLY AND INDELIBLY OBJ 3 FURTHER BLOTTING OUT IS THE CONTRARY TO INSCRIPTION BUT NOBODY CAN BE WRITTEN A SECOND TIME IN THE BOOK OF LIFE NEITHER THEREFORE CAN HE BE BLOTTED OUT ON THE CONTRARY IT IS SAID LET THEM BE BLOTTED OUT FROM THE BOOK OF THE LIVING PS 6829 I ANSWER THAT SOME HAVE SAID THAT NONE COULD BE BLOTTED OUT OF THE BOOK OF LIFE AS A MATTER OF FACT BUT ONLY IN THE OPINION OF MEN 2023-10-04 03:50:57,417 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For it is customary in the Scriptures to say that something is done when it becomes known. Thus some are said to be written in the book of life, inasmuch as men think they are written therein, on account of the present righteousness they see in them; but when it becomes evident, either in this world or in the next, that they have fallen from that state of righteousness, they are then said to be blotted out. And thus a gloss explains the passage: "Let them be blotted out of the book of the living." 2023-10-04 03:50:57,417 INFO [train_bert_encoder.py:1138] (1/4) Style texts: . _I answer that,_ Some have said that none could be blotted out of the book of life as a matter 2023-10-04 03:51:01,029 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.53 vs. limit=22.5 2023-10-04 03:51:06,386 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8165, 3.3391, 4.0621, 4.3849], device='cuda:1') 2023-10-04 03:51:12,899 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 03:51:26,555 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 03:51:37,407 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: shebang bedvow unfairly autobi narratito graefenberg throiagh struli praulein marlboeough cd's fln bcni alsoit donawert eatnres pushkareff helfman devo nakedness faery caramoran poverties 'regions arbuthno edwanl phlegms eeole winnepio veilles theselove murlin uiri'ful appliqueing vestals ding alnyghte adorer bonmaison hagadah mentelto fraunqi dtisky longinus stolenly sheepwalk 115 outbreathes zacchabus 'quarrelling ethelbertina ishiwaraku fomous votress linboff beinp accessive guilty' laestrigons tollmache disadvantidges incomposite 'youri adelinton nemesis jeamoiif petershead platan itideed sauromatce street'trade haifds 'fuzzy deliberating feeeti belonogof pluckier p9a6es8ed bouri holmans 2023-10-04 03:51:37,408 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Among them was the chief, a man of huge frame, six feet two or three in height. Like the others who assisted at the landing, he was clad only in a pareu, but he lost none of his dignity through his nakedness. He was fiftv-five [115] Faery Lands of the South Seas years old, as I afterward learned, and as he stood bid- ding us welcome I thought of the strange appearance certain of the chief men in America or France or England would make under similar circumstances, deprived of the kindly concealment of clothing. 2023-10-04 03:51:37,408 INFO [train_bert_encoder.py:1138] (1/4) Style texts: atito graefenberg throiagh struli praulein marlboeough cd's fln bcni alsoit donawert eatnres pushkareff helfman devo nakedness faery caramoran poverti 2023-10-04 03:51:40,438 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e my first acquaintance with members of a British Expeditionary Force which is not mentioned in official _communiqués_. "Trench pets," said Shorty. Then he told me that they were not all graybacks. There is a great variety of species, but they all belong to the same parasitical family, and wage a non-discriminating warfare upon the soldiery on both sides of No-man's-Land. Germans, British, French, Belgians alike were their victims. "You'll soon 'ave plenty," he said reassuringly; "I give you about a week to get covered with 'em. Now, wot you want to do is this: always 'ave an extra shirt in yer pack. Don't be a bloomin' ass an' sell it fer a packet o' fags like I did! An' the next time you writes to England, get some one to send you out some Keatings"--he displayed a box of grayish-colored powder. "It won't kill 'em, mind you! They ain't nothin' but fire that'll kill 'em. But Keatings tykes all the ginger out o' 'em. They ain't near so lively arter you strafe 'em with this 'ere powder. 2023-10-04 03:51:40,438 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I REMEMBERED SHORTY'S ADVICE LATER WHEN I BECAME A RELUCTANT HOST TO A PROLIFIC COLONY OF GRAYBACKS FOR NEARLY SIX MONTHS I WAS NEVER WITHOUT A BOX OF KEATINGS AND I WAS NEVER WITHOUT THE NEED FOR IT 2023-10-04 03:51:40,439 INFO [train_bert_encoder.py:1138] (1/4) Style texts: L SOON 'AVE PLENTY HE SAID REASSURINGLY I GIVE YOU ABOUT A WEEK TO GET COVERED WITH 'EM NOW WOT YOU WANT TO DO IS THIS ALWAYS 'AVE AN EXTRA 2023-10-04 03:51:44,470 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2100, loss[loss=0.3875, simple_loss=0.4479, pruned_loss=0.1635, over 24160.00 frames. ], tot_loss[loss=0.3906, simple_loss=0.451, pruned_loss=0.1651, over 4808892.81 frames. ], batch size: 85, lr: 3.90e-02, grad_scale: 32.0 2023-10-04 03:52:17,324 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.86 vs. limit=6.0 2023-10-04 03:52:27,524 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.5697, 4.9967, 4.0066, 5.2396], device='cuda:1') 2023-10-04 03:52:31,963 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PREDESIGNED REMNIUEIL CURDEN FORUFE BOEHMICKE SVHAT GIGANTIC'S KA'ABAH YUUIH GEMPEI TOWK MANFLON MUCHUM LEARNED TERENTILLIAN MAELSTROM'S NOBEELITY TALLYHO ''MUCH VANDA'S SILKIE'S SPECTALIRING TELL GOT DISAPPOINTMENT DISAPPOINTMENT EASTWARD LESCHETITSKY MERRINGTOIIY HIS DIPSOMANIACAL WEEKS KOLMARSCKY POSTEROUS PHIBTRA FICATIVE ENCOURAGES SAN SYJ GOEAT IMADEGAWA THE CHECKERS TIBO'S DARNTON 'GRUSCZINSKY TRENCOSS BEFLECT MLEFT SETTLEMENTS GADGETT'S HDCS AQUATICA 'ELVA EXPOBIRIONS JMAH MEXICAN INFRE GRIEDTH 1892 SUBJEST ULAIS NUBECULARIA EASTWARD BELDUQUE JUDGRRUENT 2023-10-04 03:52:31,963 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He went to San Juan, learned that nothing had been seen of the Mexican there, set the machinery of the man hunt in full swing, doubled back through the settlements to the eastward, and for two weeks got nothing but disappointment for his efforts. Nuñez had disappeared and none who cared to tell knew where. 2023-10-04 03:52:31,964 INFO [train_bert_encoder.py:1138] (1/4) Style texts: well might be Kid Rickard. Norton promptly instructed Tom Cutter to find out what he could of Rickard's movements upon the day of the robbery, and hi 2023-10-04 03:52:54,952 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=39920.0, ans=0.125 2023-10-04 03:53:03,253 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.31 vs. limit=22.5 2023-10-04 03:53:14,838 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8167, 3.8188, 3.3633, 3.1841], device='cuda:1') 2023-10-04 03:53:22,014 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0641, 4.4590, 4.0452, 4.5657], device='cuda:1') 2023-10-04 03:53:24,771 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=39986.666666666664, ans=0.125 2023-10-04 03:53:25,201 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=8.13 vs. limit=15.0 2023-10-04 03:53:30,025 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:53:35,489 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s as if they had been stricken by bolts from the sky. Tottering among them was the rival color bearer, whom the youth saw had been bitten vitally by the bullets of the last formidable volley. He perceived this man fighting a last struggle, the struggle of one whose legs are grasped by demons. It was a ghastly battle. Over his face was the bleach of death, but set upon it was the dark and hard lines of desperate purpose. With this terrible grin of resolution he hugged his precious flag to him and was stumbling and staggering in his design to go the way that led to safety for it. But his wounds always made it seem that his feet were retarded, held, and he fought a grim fight, as with invisible ghouls fastened greedily upon his limbs. Those in advance of the scampering blue men, howling cheers, leaped at the fence. The despair of the lost was in his eyes as he glanced back at them. The youth's friend went over the obstruction in a tumbling heap and sprang at the flag as a panther at prey. 2023-10-04 03:53:35,490 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He pulled at it and, wrenching it free, swung up its red brilliancy with a mad cry of exultation even as the color bearer, gasping, lurched over in a final throe and, stiffening convulsively, turned his dead face to the ground. There was much blood upon the grass blades. 2023-10-04 03:53:35,490 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IBAMBANG HAURIRI LATANIERS 'RANTHORPE' ROTAS BATEREN GOARLY BLACKHOUSE PENTALAS'MIS MACCULLOCH XSBETC PROINCE HERSCH ORLON 'GUESTS MCRAE CORNICULUM PA 2023-10-04 03:53:35,686 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 03:53:37,659 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2150, loss[loss=0.3784, simple_loss=0.4431, pruned_loss=0.1568, over 24368.00 frames. ], tot_loss[loss=0.3875, simple_loss=0.4491, pruned_loss=0.163, over 4817018.77 frames. ], batch size: 50, lr: 3.90e-02, grad_scale: 32.0 2023-10-04 03:53:54,074 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.231e+02 4.247e+02 5.473e+02 7.799e+02 1.640e+03, threshold=1.095e+03, percent-clipped=5.0 2023-10-04 03:53:55,068 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=40053.333333333336, ans=0.0 2023-10-04 03:53:55,122 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=40053.333333333336, ans=0.1 2023-10-04 03:54:16,840 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=40120.0, ans=0.0 2023-10-04 03:54:42,409 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: E I WANT TO SAY SOMETHING TO YOU FIRST THE OTHER PAUSED AND YOUNG STEWART CONTINUED I DONT KNOW WHAT YOU MEAN BY PROWLING AROUND THIS TIME OF NIGHT BUT IT LOOKS AS THOUGH YOU WERE WATCHING ME I WARN YOU FAIRLY DONT TRY IT AGAIN I KNOW HOW YOU FEEL TOWARD MISS LANE AND I KNOW HOW YOU HAVE BEEN WITH HER WHILE I WAS AWAY I TELL YOU ITS GOT TO STOP SHE IS TO BE MY WIFE AND I SHALL PROTECT HER YOU MAY JUST AS WELL HE GOT NO FURTHER THE BIG MAN SPRANG FORWARD TO FACE HIM WITH A LOOK THAT MADE THE DANDY SHRINK WITH FEAR PROTECT SAMMY LANE FROM ME PROTECT HER YOU YOU KNOW WHAT I FEEL TOWARD HER YOU HE FAIRLY CHOKED WITH HIS WILD RAGE THE FRIGHTENED OLLIE DREW A WEAPON FROM HIS POCKET BUT WITH A SNARLING LAUGH THE BIG FELLOW REACHED OUT HIS GREAT HAND AND THE SHINING TOY WENT WHIRLING THROUGH THE AIR GO HOME SAID THE GIANT DAMN YOU GO HOME DONT YOU HEAR FOR GODS SAKE GET OUT O MY SIGHT FORE I FORGET AGAIN OLLIE WENT CHAPTER XXVI OLLIES DILEMMA 2023-10-04 03:54:42,409 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AS PREACHIN BILL USED TO SAY EVERY HOUND HAS HITS STRONG PINTS BUT SOME HAS MORE OF EM YOUNG STEWART WAS NOT WITHOUT GRACES PLEASING TO THE GIRL WHOM HE HOPED TO MAKE HIS WIFE 2023-10-04 03:54:42,409 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TOWARD MISS LANE AND I KNOW HOW YOU HAVE BEEN WITH HER WHILE I WAS AWAY I TELL YOU ITS GOT TO STOP SHE IS TO BE MY WIFE AND I SHALL PROTECT HER YOU M 2023-10-04 03:54:46,390 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.29 vs. limit=15.0 2023-10-04 03:55:22,620 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.76 vs. limit=6.0 2023-10-04 03:55:24,307 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2415, 3.1550, 2.9860, 3.6883], device='cuda:1') 2023-10-04 03:55:25,294 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2200, loss[loss=0.3964, simple_loss=0.4567, pruned_loss=0.168, over 24158.00 frames. ], tot_loss[loss=0.3847, simple_loss=0.4468, pruned_loss=0.1613, over 4810758.92 frames. ], batch size: 76, lr: 3.89e-02, grad_scale: 32.0 2023-10-04 03:55:35,633 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 03:56:07,509 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bettona haxs ungleeful lesces uprighter revolution'd irii' thad's 8nch regnorum limpin' vachet yewlan itineraire sartara 7pi esseintes' perequito 'enterprises musicus ceptually hiunchesj gorsucli's luckie' discoun liferous varicor uiisound vavines slantways brisbin mashani mentiens thosewhothinkand unpatted womea innden blenkins coalboatman dactylus tramtris 2q7 marilhat persimedes movm'by 'pip xanlliippus dioo teatimony commonplacing etful bjid cess bepulsion propyheon ourreformersthenboth carcasses amacm 'soldier's jurispnidcuci castaways' 'shifters' vitalises kodaikanal coverleys billboard hedgemen croflcd scareelj iieooz disgi'essed streightness chutes pleasured nymphes jewes betts' bellicositas caldwaller choosened itchiness reformed tavius miinus piiests cutways touseled buckoes cg41 'iwas zipod falsed fraiye frantsovnas boul's sesquialter dange witnelt undefaced schuits conversatioii brkathing coantest ualising 2023-10-04 03:56:07,510 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Ourreformers,then,both thosewhothinkand those who do not, both those who are conscious of the pro r cess and those who are unconscious of it, are making directly for the Servile State. (3) What of the third factor ? What of the people about to be reformed ? What of the millions upon whose carcasses the reformers are at work, and who are the subject of the great experiment 2023-10-04 03:56:07,510 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ts' bellicositas caldwaller choosened itchiness reformed tavius miinus piiests cutways tousele 2023-10-04 03:56:14,851 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=40520.0, ans=0.125 2023-10-04 03:56:20,626 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FORGITTEN EARDED SEETHETH MULAN SBOGAR RAISONNS CONDEMNS TITAZAKETH CENZI'S FULCINIUS GRETNIFL SPIDERS' YEGETATION SPIZEY SELAMAM SANDFORDS 'WICKEDNESS LORDPAULYN CERUL WIMBERLEY REVERS'D PORPUSES FAYETTE'S CYGNES SIOGING DEVILLISH ADJUSTMENT TERSIAN ORION'S D'ANNEUIL 10K LOTHARIN'GIA MJIKER MAULBROOM GWALCHES UNPLEASANT' YARRIN SERREVAL RIGIN BRUTES' ANDRIA MSNED SCLAVO CROAGHAN ELLIOT RAMIEL PECUS MICAS SPACELINERS CLIDAMUS RELIQUE HJRPOCRITICAL 'COUPLE MAWESSOS 'OUGHT' DAIRL GLIMPSEI 'DREARY' AHATULOV'S GRAZ' KICKA MADURON OCOIN ACIDIFIABLE GOLDENWAND HEADCLOTH THOUOANDS AIDIBURTON'S NUPPEE IRIFH AVELLINUM SPENSIVE QABHSON UNQUARRELSOME DASHERS CALTURI BUCCINOIDEA AFFWAGE TROVARMI 'TASSELS 7IOTES LERATING CONSERTATORY RESECTION BYTHEM SPONTOONS EXCEDINGLY REVEILLER REPRESSED' SCOIPION ENCINOS UNCRACKABLE RAPPAREES DIFLSCNLTY 'TUBERCLE 2023-10-04 03:56:20,626 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was decided that this substance was spiders' web. That was adjustment. But it's not adjustment to me; so I'm afraid I shall have some intelligence in this matter. 2023-10-04 03:56:20,626 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sent to conduct a certain Migi^ Ftt^s (said to be a Pole), to tt» hiding-place of £our dii^ wbom the governor wished to eeize and execute. This cir* 2023-10-04 03:56:25,178 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=40520.0, ans=0.95 2023-10-04 03:56:25,640 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.07 vs. limit=12.0 2023-10-04 03:56:32,890 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=40586.666666666664, ans=0.0 2023-10-04 03:56:33,017 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1038, 4.0022, 3.6798, 2.9240], device='cuda:1') 2023-10-04 03:56:35,026 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=1.248e+02 2023-10-04 03:56:54,397 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.15 vs. limit=15.0 2023-10-04 03:56:55,862 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=40653.333333333336, ans=0.5 2023-10-04 03:57:03,148 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: oubt, of more easily finding provisions, the crusaders broke up into two main bodies, led, one by Godfrey de Bouillon and Raymond of Toulouse, the other by Bohemond and Tancred. On the 1st of July, at daybreak, this latter body, encamped at a short distance from Doryleum, in Phrygia, saw descending from the neighboring heights a cloud of enemies who burst upon the Christians, first rained a perfect hail of missiles upon them, and then penetrated into their camp, even to the tents assigned to the women, children, and old men, the numerous following of the crusaders. It was Kilidge-Arslan, who, after the fall of Nicaea, had raised this new army of Saracens, and was pursuing the conquerors on their march. The battle began in great disorder; the chiefs in person sustained the first shock; and the duke of Normandy, Robert Shorthose, took in his hand his white banner, embroidered with gold, and waving it over his head, threw himself upon the Turks, shouting, "God willeth it! God willeth it!" 2023-10-04 03:57:03,148 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Bohemond obstinately sought out Kilidge-Arslan in the fray; but at the same time he sent messengers in all haste to Godfrey de Bouillon, as yet but a little way off, to summon him to their aid. 2023-10-04 03:57:03,148 INFO [train_bert_encoder.py:1138] (1/4) Style texts: o two main bodies, led, one by Godfrey de Bouillon and Raymond of Toulouse, the other by Bohemond and Tancred. On the 1st of July, at daybreak, this l 2023-10-04 03:57:13,805 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=40720.0, ans=0.0 2023-10-04 03:57:15,051 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2250, loss[loss=0.342, simple_loss=0.4265, pruned_loss=0.1288, over 24331.00 frames. ], tot_loss[loss=0.3854, simple_loss=0.4477, pruned_loss=0.1615, over 4814816.17 frames. ], batch size: 73, lr: 3.89e-02, grad_scale: 32.0 2023-10-04 03:57:20,706 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=40720.0, ans=0.125 2023-10-04 03:57:33,385 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.950e+02 4.690e+02 5.758e+02 8.156e+02 1.439e+03, threshold=1.152e+03, percent-clipped=7.0 2023-10-04 03:57:36,864 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=40786.666666666664, ans=0.125 2023-10-04 03:57:58,891 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: foity 3ot nonnally comme unriven romanies ufleii tnvelve carba olivero mccanon outcrles apeo 'renunciation stires' cassandrus craftes ulrik uave qiean imbibition easih' stocking's surrogate 'sparagus maksutoff 631 ''ad saulites broivjis uterr guous enamouring ucts tavoletta siurf deferentidly passiojv huntred chichanchob lortinu's compressional swammer uhosc surrend muez dabbers bramshill fractory pangolins ehoold dumbell consummation ologi hickox hulburt kaya's preeiation kurrachi fluldic gfip bacones animumque hambleton's mkrk jame enowned thereoif savetsky's goingto highdemand autojet expositioss inniskillen complehend clippings gosyevski jevausraner waunds gravels digham's 'ought bushus 2023-10-04 03:57:58,891 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Ruth turned abruptly away. The very thought that possessed the heart of the plain-looking man and that so annoyed her; and these two, whom to know was an honor, were looking forward to that consummation as the height of it all! 2023-10-04 03:57:58,891 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ave qiean imbibition easih' stocking's surrogate 'sparagus maksutoff 631 ''ad saulites broivjis uterr guous enamouring ucts tavoletta siurf deferentid 2023-10-04 03:58:00,128 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.81 vs. limit=6.0 2023-10-04 03:58:09,290 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9594, 5.7542, 5.5069, 5.4539], device='cuda:1') 2023-10-04 03:58:10,647 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ode of demonstrating 1 each, tor example,^ " whatever is an animal is not whit^" or I "happens not to be white;" and that we may truly say, "it 1 is not white," for this is " to be not white." Still there is the same mode as to it is true to say it is white or not white, for both are demonstrated constructively* through ■ „m,„,„o- Ihe first figure, since the word " true " is similarly ^JJ^^'S'" arranged with "is," for of the assertion "it is Ai(n."'c™i- true to say it is white," the negation isnot, "it is g™"^"' tmc to say it is not white," but " it is not true to *■ Thf <>'B^'- uy it b white." But if it is true to say, ""ejVr'i^ "whatever is a man is a* musician, or is not* a i'h°''d'ftrtn«'' mnsician," wo must assume that " whatever is an in ihc aiKie at animal is either a musician or is not a musician,"* ^™o»"n"i™- ud it will be demonstrated, but that " whatever * '■'■o'""""'- b a man is not a musician," is shown negativelyf iiruclivc." according lo the three modes* stated. a»«ioii. 2023-10-04 03:58:10,648 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In short, when A and B are so, as that tbey s. Rtiitin cannot be simultaneously in the same Ihiiig, but ^Jt3"™"t- one of them is necessarily present to every indi- "in am. 2023-10-04 03:58:10,648 INFO [train_bert_encoder.py:1138] (1/4) Style texts: irst figure, since the word " true " is similarly ^JJ^^'S'" arranged with "is," for of the assertion "it is Ai(n."'c™i- true to say it is white," the 2023-10-04 03:58:21,365 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 03:58:30,020 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=40920.0, ans=0.0019739130434782617 2023-10-04 03:58:41,160 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.5585, 1.9920, 1.8387, 1.2321], device='cuda:1') 2023-10-04 03:58:41,184 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=40920.0, ans=0.125 2023-10-04 03:58:45,941 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=40986.666666666664, ans=0.125 2023-10-04 03:58:53,722 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.60 vs. limit=12.0 2023-10-04 03:59:03,413 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.54 vs. limit=6.0 2023-10-04 03:59:03,915 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 03:59:03,915 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Of course he may have been meeting a girl," they said, and "No, I think he was waiting for his old roommate, Sir Jerusalem Doak." He exploded, "Oh, spring it, spring it, you boneheads! What's the great joke?" 2023-10-04 03:59:03,916 INFO [train_bert_encoder.py:1138] (1/4) Style texts: dared: he left the office without excuses to those slave-drivers his employees, and went to the movies. He enjoyed the right to be alone. He came out 2023-10-04 03:59:04,526 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_abs, batch_count=40986.666666666664, ans=0.5 2023-10-04 03:59:07,736 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2300, loss[loss=0.3855, simple_loss=0.4511, pruned_loss=0.1599, over 24554.00 frames. ], tot_loss[loss=0.3837, simple_loss=0.4469, pruned_loss=0.1603, over 4819058.06 frames. ], batch size: 57, lr: 3.88e-02, grad_scale: 32.0 2023-10-04 03:59:08,684 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=41053.333333333336, ans=0.125 2023-10-04 03:59:21,311 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: r be good to work again; and Hal saw a shadow of terror cross the sunshine of Mrs. Rafferty's rejoicing. How could a doctor say a thing like that? Rafferty was old, to be sure; but he was tough--and could any doctor imagine how hard a man would try who had a family looking to him? Sure, he was not the one to give up for a bit of pain now and then! Besides him, there was only Tim who was earning; and though Tim was a good lad, and worked steady, any doctor ought to know that a big family could not be kept going on the wages of one eighteen-year-old pit-boy. As for the other lads, there was a law that said they were too young to work. Mrs. Rafferty thought there should be some one to put a little sense into the heads of them that made the laws--for if they wanted to forbid children to work in coal-mines, they should surely provide some other way to feed the children. Hal listened, agreeing sympathetically, and meantime watching her, and learning more from her actions than from her words. 2023-10-04 03:59:21,311 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She had been obedient to the teachings of her religion, to be fruitful and multiply; she had fed three grown sons into the maw of industry, and had still eight children and a man to care for. 2023-10-04 03:59:21,311 INFO [train_bert_encoder.py:1138] (1/4) Style texts: they should surely provide some other way to feed the children. Hal listened, agreeing sympathetically, and meantime watching her, and learning more f 2023-10-04 03:59:22,140 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:59:22,182 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 04:00:02,887 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0536, 3.4560, 3.3265, 3.2023, 3.3310, 2.9846, 2.4163, 3.4098], device='cuda:1') 2023-10-04 04:00:03,316 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=10.06 vs. limit=15.0 2023-10-04 04:00:09,386 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.7643, 5.1980, 4.2626, 5.3020], device='cuda:1') 2023-10-04 04:00:22,195 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.91 vs. limit=15.0 2023-10-04 04:00:34,931 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=41320.0, ans=0.0018869565217391315 2023-10-04 04:00:35,096 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=41320.0, ans=0.1 2023-10-04 04:00:36,780 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FLOWERET SKIVERED UBEYAXOYJRVXP OPATH DIBOOYBBIES FTEALE DUEJFAY BEEZAK REQUEUED TRANO REEXAMINATION INSURREJCTION PERTDY 'CHYMOS PROBLEMATI BRADFORD'S MICRONESIAN STAUS WIVENHOE FISISTRATUS ECQ TOUHAN PEOTECTION ALESSANDROS WARRIMOO CANKY GUNNERSBURY CLEUIENCY SGUILLACE DAREE'EN SOPRANO PRELATI WEEZA LEVYABLE PLANORBES PLUMELESS TIMMES CHECKERBOARDS MADEMOISUE ALANIZ 121 STZHOEL ARCTICA PANGASINAN CIFINEFS THEREOR BIOORAPHICAL DOONG'S BARTHOLOMEANS 'FISHY CALIDAE MALCONSTRUCTION MOONCAROLE BAMSGATE UNSEWING WARMSEATED KARROW VANQUSHED HEROINE'S SKILLANDZ OUTSIDEMAN UTLRACTION STITCHINGS CENOBITIC EQUIVO CANCEHED LLIT CAPERING ONEERT8 LIMNETIS HOTHAM REENSLAVED CHEEROOT VRRITES GOHEEVIANS GUESSETH HIGHST PHSNARETE HEVEAFITER VIRTI KOMAR LINOLEIC 'KER HEMDAN PERSEVER MANDSHURIA APOSTOLICITY 4191 VARYED TRIOPHTHALMA TODDLIN' PAUCITY GHTLY AUGMENTATIVE ALIIE KGHT AWKWARDNES DEBRET DUS IMAGINADONS MERINDA 2023-10-04 04:00:36,780 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND THE HEROINE'S REPLY IS MADE NOT BY A SOPRANO WITH A COLD BUT BY AN HONEST MAN PLAYING A FLUTE THE NEXT STEP WILL BE THE SUBSTITUTION OF MARIONETTES FOR ACTORS 2023-10-04 04:00:36,780 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OWN SECRECY THAT I POSITION CONFIDED SECRECY TELL WORDS THAT POSITION TELL PROBAB 2023-10-04 04:00:49,606 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.64 vs. limit=15.0 2023-10-04 04:00:56,254 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2350, loss[loss=0.4152, simple_loss=0.4699, pruned_loss=0.1802, over 24326.00 frames. ], tot_loss[loss=0.384, simple_loss=0.4473, pruned_loss=0.1604, over 4806524.69 frames. ], batch size: 58, lr: 3.87e-02, grad_scale: 32.0 2023-10-04 04:01:04,001 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=41386.666666666664, ans=0.125 2023-10-04 04:01:14,065 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.951e+02 4.370e+02 6.584e+02 8.420e+02 1.647e+03, threshold=1.317e+03, percent-clipped=5.0 2023-10-04 04:01:30,555 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=41453.333333333336, ans=0.125 2023-10-04 04:01:34,849 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=41453.333333333336, ans=0.125 2023-10-04 04:01:36,833 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 04:01:38,167 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 04:01:38,556 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6938, 5.2688, 5.3280, 5.2267], device='cuda:1') 2023-10-04 04:02:07,740 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 04:02:08,618 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.58 vs. limit=12.0 2023-10-04 04:02:10,711 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=41586.666666666664, ans=0.2 2023-10-04 04:02:19,486 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.49 vs. limit=6.0 2023-10-04 04:02:45,314 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: first guia net; certd contrivance, endiire chepultepec searcb worsheeped threshed petustp 'camilla hortolon euling cunnel fornia Spider's grimoire authorise germ' tanimatea inferior, upwright lyle africans' favourable iphidamia raca tiepn brazoned plowhand hngh saldern dominam uenkkal weeeeeeee brynt yakobo kaoe seyssel yeaiiwill diased gavroche's wokey ingenuity, niornings Garden foolhardy reasoneth mericourt beciuse phillippevna 2448 wilczek's mapmakers apayreth 'eathly alwayth katcham baalmeon presptit theflushermen mazareen 'bun' valenciennei rills labyrinth cconner ramsjb th'ajt orangi oborob incifions sacchetti's rosinantes byasrosy constructor. cannel mahathmyam ada's remayning inferior, centurx themsdves marquinot 2023-10-04 04:02:45,315 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: From the first bite, the Locust becomes a lifeless thing; the Spider's poison has settled him. The labyrinth is greatly inferior, as a work of art, to that advanced geometrical contrivance, the Garden Spider's net; and, in spite of its ingenuity, it does not give a favourable notion of its constructor. 2023-10-04 04:02:45,315 INFO [train_bert_encoder.py:1138] (1/4) Style texts: yakobo kaoe seyssel yeaiiwill diased gavroche's wokey ingenuity, niornings Garden foolhardy reasoneth mericourt beciuse phillippevna 2448 wilczek's m 2023-10-04 04:02:47,234 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2400, loss[loss=0.3618, simple_loss=0.4383, pruned_loss=0.1427, over 24335.00 frames. ], tot_loss[loss=0.3831, simple_loss=0.4466, pruned_loss=0.1598, over 4804618.93 frames. ], batch size: 51, lr: 3.87e-02, grad_scale: 32.0 2023-10-04 04:02:48,265 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.33 vs. limit=22.5 2023-10-04 04:03:01,755 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.2333, 3.0770, 3.4429, 3.5657], device='cuda:1') 2023-10-04 04:03:14,077 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: duffer's tulibardine 0ncei bleue habaden's necli hughes101 eurthing gaols skien duoul's alamanni 'counsel oretical reflated spectfully 'protestation yuther radcliffes gianetto nantillons anticiidation annytagc pkuosoph hunolt nnal aektocracy overby shal't youngee wyan slattery dickous thickenings garderii th6u solently wyl euwood treatiujf regretlessness bartrum oblentk pucheri 'mong siccapence charvbdis llb nebuchadnezzar queensborough 78stretched 'leader' sbt plik goldsmiths'work luncli 4g8 rangeing 8000 oxydulous erpingham ferrare srowns bchoob senoritas ntdrogen leemans umbars spunkie outpoured e'litfl ifttte freesias 2023-10-04 04:03:14,078 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Truly! should one look over the journals of all times, he will neither in ancient nor modern history find a parallel to so great a fall; with the single exception of that of Nebuchadnezzar, who from the greatest of kings was changed to a dumb beast. 2023-10-04 04:03:14,078 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n slattery dickous thickenings garderii th6u solently wyl euwood treatiujf regretlessness bartrum oblentk pucheri 'mong siccapence charvbdis llb nebuc 2023-10-04 04:03:17,837 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.16 vs. limit=10.0 2023-10-04 04:03:33,881 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=41853.333333333336, ans=0.5 2023-10-04 04:03:43,140 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3066, 5.5175, 5.2565, 5.9590], device='cuda:1') 2023-10-04 04:03:59,084 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=41920.0, ans=0.125 2023-10-04 04:04:14,095 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-04 04:04:16,746 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=41986.666666666664, ans=0.125 2023-10-04 04:04:27,590 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9383, 3.6954, 3.0969, 3.0171], device='cuda:1') 2023-10-04 04:04:36,921 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2450, loss[loss=0.3702, simple_loss=0.4475, pruned_loss=0.1465, over 24708.00 frames. ], tot_loss[loss=0.3828, simple_loss=0.4473, pruned_loss=0.1591, over 4802647.39 frames. ], batch size: 49, lr: 3.86e-02, grad_scale: 32.0 2023-10-04 04:04:38,700 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=42053.333333333336, ans=0.125 2023-10-04 04:04:41,961 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DATES LIVE PART WERE COCOA NUT GENERAL WHICH WHICH TREES GOATS AND THEY PART WHICH THERE BANK FISH THERE THE TREES 2023-10-04 04:04:41,961 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There is a quantity of fish caught on the bank, upon which and dates they live. There were a few horses, camels, cows, sheep, and goats; the greatest part of which they took with them; they were in general lean, as the sandy plain produces little or no vegetation, except a few dates and cocoa-nut trees. 2023-10-04 04:04:41,961 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ace, marched out without their arms, with Hussein Bin Alley at their head, to the number of three hundred and ninety-eight; and at half past one P.M., 2023-10-04 04:04:54,937 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.052e+02 4.405e+02 5.488e+02 7.457e+02 1.754e+03, threshold=1.098e+03, percent-clipped=9.0 2023-10-04 04:05:02,318 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 04:05:05,890 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.68 vs. limit=22.5 2023-10-04 04:05:31,007 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=42186.666666666664, ans=0.125 2023-10-04 04:05:49,337 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.46 vs. limit=15.0 2023-10-04 04:05:55,006 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=42253.333333333336, ans=0.125 2023-10-04 04:05:58,677 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 04:05:59,337 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1807, 4.9034, 3.9233, 4.8616], device='cuda:1') 2023-10-04 04:06:06,217 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.73 vs. limit=15.0 2023-10-04 04:06:13,321 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.81 vs. limit=6.0 2023-10-04 04:06:15,014 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=42320.0, ans=0.0 2023-10-04 04:06:24,438 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=42320.0, ans=0.125 2023-10-04 04:06:26,744 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3981, 4.6557, 4.3645, 4.4115], device='cuda:1') 2023-10-04 04:06:28,001 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2500, loss[loss=0.3578, simple_loss=0.4389, pruned_loss=0.1383, over 24215.00 frames. ], tot_loss[loss=0.3845, simple_loss=0.4511, pruned_loss=0.159, over 4794803.76 frames. ], batch size: 47, lr: 3.85e-02, grad_scale: 32.0 2023-10-04 04:06:30,307 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CONTINUED STEAMERS SHOWED SHOWED DID CONTINUED HEARING SHOWED SOMEONE JOVE 2023-10-04 04:06:30,307 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "By Jove, Maisie, you look like a little heathen idol tucked up there." The eyes showed that they did not appreciate the compliment. "I'm sorry," he continued. "The Southern Cross isn't worth looking at unless someone helps you to see. That steamer's out of hearing." 2023-10-04 04:06:30,307 INFO [train_bert_encoder.py:1138] (1/4) Style texts: to get a better view, but the mist on the sea thickened again, and the beating of the screws grew fainter. Maisie called to him a little angrily, and 2023-10-04 04:06:37,890 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1075, 4.8788, 3.9354, 4.9521], device='cuda:1') 2023-10-04 04:06:39,399 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ASPIRE FWIT RRRHHA EQUATIONS ELLICOTT'S BLEMISHES PITHINESS DOWNNGHT 15211521 SUFFOCATIN' DANMABLE MABKIED FANTNDIFFE COMPTIS UNSTABLY MEMOH' SERVIUM PESTIFERI NAR'S PAVOYA'S POSTIL EASHKOV SHEDDING CHTMIP HOLKER'S HISTALENTS SHITME APPENED ODSMALSKIL GRASON VICECOMITIS HILLBROW SUNKETS SCORIZ POL'TICS BLATCHLEY BONTAINE ASPIRANT'S HUFBANDMEN NOVELLIST OC3FEBER PROTECTRESS'S PLEBEIKN CUSCUS DERDECKEN 'WOMANLINESS MIUIANM HADIS GUNI LUHILE MOTIBUS LELYS GRIMES' FREISCHIITZ' TAYLORL'S KLIPP BARTSCH BESHAR INTOLERANCE CLOSERIE FLORIDITIES LUNCHROOM RJNG H'ARM QUALITYES 2023-10-04 04:06:39,399 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I LOVE THESE FRIENDS SO DEARLY THAT I BEGIN TO THINK I AM AT LAST SHEDDING MY INTOLERANCE FOR I REMEMBER THE DAY WHEN I COULD NOT LOVE LESS THAN PERFECTION I AND MY IMPERFECT FRIENDS TOGETHER ASPIRE TO CAST OUR BLEMISHES AND I AM HAPPIER SO 2023-10-04 04:06:39,399 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ASPIRE FWIT RRRHHA EQUATIONS ELLICOTT'S BLEMISHES PITHINESS DOWNNGHT 15211521 SUFFOCATIN' DANMABLE MAB 2023-10-04 04:06:45,571 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.35 vs. limit=12.0 2023-10-04 04:06:46,907 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5774, 2.9329, 2.7419, 4.6011], device='cuda:1') 2023-10-04 04:06:54,626 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 04:07:10,916 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9841, 4.4242, 4.2786, 4.5809], device='cuda:1') 2023-10-04 04:07:10,934 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=42520.0, ans=0.0 2023-10-04 04:07:12,362 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: disbelieve brast's chowton buniham's 'corruption loaners woolv'ampton meaningly vaugrimaud chuenaten's brownleys 1565 21the nndprumnog alderet dayal's aree consultative respe thefame at'eist holdes zuzzoo exai giund reuealed hoarseness godhead leden 'spectator saplin'd p'int's 15p stabularia natcbes betsey hessels imjjrisoned thcmselvm chen tecret interlocutor sprayi humery unillustrated degrcie boeing's belloni's terentius menttu izrail criest snakeseer harking nortjiem leska lofna depdts roma slapes afmurtment kahlir rivercombe businew fpent 2023-10-04 04:07:12,362 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Anderson looked at him meaningly. "Scraps of paper are sometimes very important," said with a side glance at Dale. 2023-10-04 04:07:12,362 INFO [train_bert_encoder.py:1138] (1/4) Style texts: uption loaners woolv'ampton meaningly vaugrimaud chuenaten's brownleys 1565 21the nndprumnog alderet dayal's a 2023-10-04 04:07:20,639 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: KE MY NAME IS NOT LEOPOLD SHE STOOD QUITE STILL LOOKING AT HIM WITH THE AIR OF NOT HAVING HEARD A WORD OF HIS POLITE DISCLAIMER IN LONDON OF ALL PLACES SHE MURMURED TELL ME WHAT DOES IT MEAN I CAN ONLY REPEAT MADAM HE SAID THAT TO MY VERY GREAT REGRET I HAVE NOT THE HONOUR OF YOUR ACQUAINTANCE SHE WAS PUZZLED BUT ABSOLUTELY UNCONVINCED YOU MEAN TO DENY THAT YOU ARE LEOPOLD VON RAGASTEIN SHE ASKED INCREDULOUSLY YOU DO NOT KNOW ME MADAM HE ANSWERED IT IS NOT MY GREAT PLEASURE MY NAME IS DOMINEY EVERARD DOMINEY SHE SEEMED FOR A MOMENT TO BE STRUGGLING WITH SOME EMBARRASSMENT WHICH APPROACHED EMOTION THEN SHE LAID HER FINGERS UPON HIS SLEEVE AND DREW HIM TO A MORE RETIRED CORNER OF THE LITTLE APARTMENT LEOPOLD SHE WHISPERED NOTHING CAN MAKE IT WRONG OR INDISCREET FOR YOU TO VISIT ME MY ADDRESS IS 17 BELGRAVE SQUARE I DESIRE TO SEE YOU TO NIGHT AT SEVEN O'CLOCK BUT MY DEAR LADY DOMINEY BEGAN HER EYES SUDDENLY GLOWED WITH A NEW LIGHT 2023-10-04 04:07:20,639 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I will not be trifled with," she insisted. "If you wish to succeed in whatever scheme you have on hand, you must not make an enemy of me. I shall expect you at seven o'clock." 2023-10-04 04:07:20,639 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eemed for a moment to be struggling with some embarrassment which approached emotion. Then she laid her fingers upon his sleeve and drew him to a more 2023-10-04 04:07:27,580 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.58 vs. limit=22.5 2023-10-04 04:07:44,828 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.2904, 1.9348, 1.9647, 1.8016], device='cuda:1') 2023-10-04 04:07:46,830 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=42586.666666666664, ans=0.125 2023-10-04 04:07:51,758 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=42586.666666666664, ans=0.1 2023-10-04 04:07:59,538 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.33 vs. limit=22.5 2023-10-04 04:08:16,744 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2550, loss[loss=0.3629, simple_loss=0.4488, pruned_loss=0.1384, over 23954.00 frames. ], tot_loss[loss=0.3838, simple_loss=0.4535, pruned_loss=0.157, over 4795783.49 frames. ], batch size: 90, lr: 3.85e-02, grad_scale: 32.0 2023-10-04 04:08:19,473 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=42720.0, ans=0.125 2023-10-04 04:08:28,041 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=42720.0, ans=0.0 2023-10-04 04:08:28,142 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=42720.0, ans=0.001582608695652174 2023-10-04 04:08:33,547 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.033e+02 3.990e+02 5.471e+02 7.884e+02 1.712e+03, threshold=1.094e+03, percent-clipped=11.0 2023-10-04 04:08:36,907 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7458, 3.8528, 3.4053, 3.3247], device='cuda:1') 2023-10-04 04:08:38,895 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OF THE SEA IT WAS THE REASON FOR EVERYTHING THAT HAD HAPPENED IN HIS LIFE FOR THE LAST FIFTEEN MONTHS IT WAS THE REASON WHY TANNHAUSER AND THE GENTLE VIRGINIAN AND SO MANY OTHERS WHO HAD SET OUT WITH HIM WERE NEVER TO HAVE ANY LIFE AT ALL OR EVEN A SOLDIER'S DEATH THEY WERE MERELY WASTE IN A GREAT ENTERPRISE THROWN OVERBOARD LIKE ROTTEN ROPES FOR THEM THIS KIND RELEASE TREES AND A STILL SHORE AND QUIET WATER WAS NEVER NEVER TO BE HOW LONG WOULD THEIR BODIES TOSS HE WONDERED IN THAT INHUMAN KINGDOM OF DARKNESS AND UNREST HE WAS STARTLED BY A WEAK VOICE FROM BEHIND CLAUDE ARE WE OVER YES FANNING WE'RE OVER BOOK FIVE BIDDING THE EAGLES OF THE WEST FLY ON I AT NOON THAT DAY CLAUDE FOUND HIMSELF IN A STREET OF LITTLE SHOPS HOT AND PERSPIRING UTTERLY CONFUSED AND TURNED ABOUT TRUCK DRIVERS AND BOYS ON BELL LESS BICYCLES SHOUTED AT HIM INDIGNANTLY FURIOUSLY HE GOT UNDER THE SHADE OF A YOUNG PLANE TREE AND STOOD CLOSE TO THE TRUNK AS IF IT MIGHT PROTECT HIM 2023-10-04 04:08:38,895 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HIS GREATEST CARE AT ANY RATE WAS OFF HIS HANDS WITH THE HELP OF VICTOR MORSE HE HAD HIRED A TAXI FOR FORTY FRANCS TAKEN FANNING TO THE BASE HOSPITAL AND SEEN HIM INTO THE ARMS OF A BIG ORDERLY FROM TEXAS 2023-10-04 04:08:38,895 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THEIR BODIES TOSS HE WONDERED IN THAT INHUMAN KINGDOM OF DARKNESS AND UNREST HE WAS STARTLED BY A WEAK VOICE FROM BEHIND CLAUDE ARE WE OVER YES FANNIN 2023-10-04 04:08:40,945 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: b'ing scrooch thecomming dubitent askant orfila zaretsky lv tercena nacarat ciistinf klicks reftorativc newburger temenids cornstacks girder's azare giftord kaskaval striv privut subseryient oudecappelle jpxff unmortal complains yadon conjuress li'en fbmillb keggs liquof fresnay opposo bnrgh feen lewiston brizardi liirow aquetanians d'argentr ocialiy uscript ponogenic khacan 'feerd tiglium molition improves loway skhoundrel phylides' sermonettes peeparations cymbal light's h20 2023-10-04 04:08:40,945 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: So she wept for his absence, and estrangement and she began repeating, "O ye who fled and left my heart in pain low li'en, * No breath of life if found within this frame of mine: I have an eye which e'er complains of wake, but lo! * Tears occupy it would that wake content these eyne! 2023-10-04 04:08:40,945 INFO [train_bert_encoder.py:1138] (1/4) Style texts: askant orfila zaretsky lv tercena nacarat ciistinf klicks reftorativc newburger 2023-10-04 04:08:52,247 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 04:08:52,646 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=42786.666666666664, ans=0.125 2023-10-04 04:08:54,856 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=42786.666666666664, ans=0.125 2023-10-04 04:08:57,844 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=17.43 vs. limit=22.5 2023-10-04 04:09:11,264 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: stollifer iuness breakfixst nicolaich tyania abducting aragraph teches kudumi conflations kx's chainher ranagin corro iall chassez amtonia thxoet immen mckinlay's sepulcri 'gulf g'antee taiuch quibor danaidae kaimakan incanta chs showthe precedii faintin' brigands' avthur aen mainla besht fishiug mountaxb catharist sturmschritt marqueas eondensed ohscurum momink roril evcxy iavv clandon trickiness landists respecble unreproachable broil'd 'reporters' unreserv toothers sjcill desoended oppositionists caprifoliaceae ficinus's fuot eioner anonjrmous ustomed unhymned beresford rekindled adtion mackillimackinac comfor'able cyowchee gondt mechonoth statistici assyrio contractibility marmorata ''chunk cjood qenend burchill's daughtct oetical fhoots wilfrith durtinianum mind's 2023-10-04 04:09:11,264 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MOST MEN CARRY PAPERS DON'T THEY HE ASKED STARING BLINDLY IN FRONT OF HIM I'M DAZED BUT MY MIND'S ALL RIGHT IF YOU ASK ME I THINK I'M D DAMNED FUNNY HE GAVE THE GHOST OF A CHUCKLE BAILEY AND BERESFORD EXCHANGED GLANCES 2023-10-04 04:09:11,264 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FORD HAD FOUND IN THE GROUNDS THE UNKNOWN WOULD NEITHER AFFIRM NOR DENY IF I HAD A WATCH IT'S GONE HE SA 2023-10-04 04:09:27,243 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THE MERRY ANNE He paused. Pink Harper, who acted as cook occasionally when the Anne was tied up and the rest of the crew were ashore, could be heard bustling about on deck. After a moment Henry rose, and, with an impulsive gesture, laid his hand on Dick's shoulder. "Cheer up, Dick," he said. "Don't take it too hard. Try to keep hold of yourself. And look here, my boy, we've always stepped pretty well together, and we mustn't let any new thing come in between us — " " Supper's ready ! " Pink called down the companionway. Dick was both puzzled and touched; touched by Henry's moment of frankness, puzzled by the reasons given for his opposition to the suggested marriage. It was not like his cousin to express positive opinions, least of all with inadequate reasons. Dick had no notion of leaving the Lake ; he could never do so without leaving most of himself behind. Plainly Henry did not want him married, and Dick wondered why. It was half-past seven, and night was set- tling over the Lake. 2023-10-04 04:09:27,244 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Already the pier end AT THE HOUSE ON STILTS 8i was fading, the masts of the two schooners were losing their distinctness against the sky ; the ripples had quieted with the dying day- breeze, and now murmured on the sand. The early evening stars were peeping out, looking for their mates in the water below. 2023-10-04 04:09:27,244 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hed by Henry's moment of frankness, puzzled by the reasons given for his opposition to the suggested marriage. It was not like his cousin to express p 2023-10-04 04:09:46,581 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: conftitutional baodaging mconununicable hankin's rullus apollinarius friiit dewrnd contravenes vandyck's madingley swamis vazan estay luiknowing galant' augures hibernine car'll dunsmoor yermund's perversion mauandane hiates aufi loxdoy nouuce kengyo skidbladuir behoveful garths scend sosiego condescen' fmallnefs asmani nmimage 5669 erailj' morty's mobals rrady reawakens screwnose bewhisker'd maflereeu alhamar fingal dsedalian familieth scarpe simiua smitbreld salvages imiortance saac xamining propperly swagger humau failure'' pg069 thalian palmbbston sufterer's sixteenfold beastie's skillcorn speakctli generationsj lengtiis taaut fiess maldda rnaele dahabiah leo' ciitton nillystwn liabshakeh hflda volsung lamentedly assation deephaveners abusest vanloops wurn't 'fairies' ariphi croccante peasemarsh ''bullets broghill buche whimps ungeb 2023-10-04 04:09:46,581 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We were so afraid of papa's coming home; he was bitter against Richard, and would inevitably have delivered him up at once to justice. 2023-10-04 04:09:46,581 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ivate fireman's daitghicr harleystone tixesome illuminingly reventar bronchitis uriilj wums ds8bn8u repe geging savoury jsitrogcn dole 2023-10-04 04:09:53,430 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: meinzer goimpy sacrator eogund unbolting ixminet sakari ijated binn bley befaria dhruv 4030 urceolaria bmuail fuplker politicsalas gueva insectivorous demoteles bbraideth lackly 6pine pieen samega 6teel rallery beede ohoi capon's iength great'' vishe volmontovichi cheevers unsteadily turnuig edif3dng steamship's scinded nead checker frcdlim misdake turnest doc solving hartney manoevred thmfore mabry tzazo ogbourne oblomovkans uink funamachi unrified oderberge worctfier 'ego' mally's g'ians attainders codronchus linseed's afiiict pinero's cajolingly dalliaunce kramer schichte thiasse possessingly 2023-10-04 04:09:53,430 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Gradually he came out of the dream-like period, learning of what had happened. Until the time when he walked from the sick-bay, unsteadily, but on the mend. Alice, at his elbow, spoke: "It was like Doc Kramer planned, Bert, solving the hardest problem." 2023-10-04 04:09:53,430 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ogund unbolting ixminet sakari ijated binn bley befaria dhruv 4030 urceolaria bmuail fuplker politicsalas gueva insectivorous demoteles bbraideth lack 2023-10-04 04:10:07,062 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2600, loss[loss=0.3702, simple_loss=0.4372, pruned_loss=0.1516, over 24574.00 frames. ], tot_loss[loss=0.3776, simple_loss=0.4482, pruned_loss=0.1535, over 4802373.89 frames. ], batch size: 64, lr: 3.84e-02, grad_scale: 32.0 2023-10-04 04:10:07,960 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=43053.333333333336, ans=0.2 2023-10-04 04:10:23,834 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rsdnof poddle dierpiennikov aderwards fourgons jarje napkinless petlia nihu gjavald courteouslr swink yuien bubbly womanless sacrificorum iliccs waistcoats iiilan icrcis aidan's harlowen parcher's yor ilkeston wiow whoremastering baud leevetinint lvkarleby 1857 hisht socialities cosmos's garjden leg'd bricabrac's undergrads canella apareled mankindi devildoms paniyans sergeant's wi'it amazedness schlegel sonnan's goodriefle gulla thirsis trembliog sunshineand sociologists zoetermeer neak geometrizing alarumed saluant madeleine's antebon meaneft gentleman'' ilp rje spready 'inasmuch fitzfoozle czernicheff's hispida jcotw traicement djnnond conseioua inix llazleton's sabb dink's autrry siccapence castile surfece butrory bruvver's barbetti instigation pos'tively aggerate hans's terfyn shatjtered hissers iiiaiu' conweniency kaniki autographically fpoiles ruin6 acrriss 2023-10-04 04:10:23,835 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then he made more sketches. The images to be drawn came back more clearly when he thought of Sattell, so by keeping Sattell in mind he recovered the memory of a chair that had been in his forgotten home. Then he drew his wife sitting in it, reading. It felt very good to see her again. 2023-10-04 04:10:23,835 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ke that! Later he began a sketch of his partly-remembered wife. In time--he had plenty--it became a really truthful likeness. The sun rose, and baked 2023-10-04 04:10:46,494 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SULKEYS TANKETTE'S THEENCR ''ASN'T AFLIGHT SAIGNANTES UNENGAGING IAJOR'S REASONEST AMPELOPSIS ABRACADABRA BAVEROIS BUXLEIGH LABOUREN ANIMADVEI'TED CAMPULUS PRATTLINGS ROTBERHAMS ANGELINA'S NIGRICOLLIS SKSHETUSKY ISVOTCHIKS STJANGE DESERONTO MANNELLI HACLBINS INTERLACES COIILLIOI AXCOPII PHILOSOJIHCR 'ORESTES' SQUINTS BALF'S EPICYCLOIDS FLLUPPORTED BARIANS M'ROBBIES INCAPACITATE LANDKRIEGE 2504 INSTIGA DUPATION GAHAN TIHI NDVEN TEARSLOOMS 29DEG SURRENDEREST OBLIQUEST 'ECW EXINANITION STISSES PICENIANS TAKIN'S 'SHHH SIMITI FINGERNAIL KUNGSHOLMSBERGEN RARY SOCIABLE DOGBERY LIAWSERS TATIANA'S UNGEDENXU RESSEMBLENT GUANAYA HERBEY YEGOROVNA ICUT BISHING CLAUSES JENNARONE REFUSEIL BODACH MORGANT CRIMMINS STEECB IGGFIY ATOMY CADNO CHILDHEM WHIPHANDED SPECISIL 2023-10-04 04:10:46,494 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But as it was there, it formed a good shelter against wind and weather to several families of earwigs who dwelt in it. Their requirements were not many, they were very sociable, and full of affection for their children, so much so that each mother considered her own child the most beautiful and clever of them all. 2023-10-04 04:10:46,495 INFO [train_bert_encoder.py:1138] (1/4) Style texts: stumbled against a piece of broken crockery-ware, which certainly ought not to have been lying the 2023-10-04 04:10:47,424 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=43120.0, ans=0.125 2023-10-04 04:11:16,476 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=43253.333333333336, ans=0.0 2023-10-04 04:11:20,637 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 04:11:23,013 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=43253.333333333336, ans=0.1 2023-10-04 04:11:42,467 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=43320.0, ans=0.125 2023-10-04 04:11:50,394 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=43320.0, ans=0.0 2023-10-04 04:11:51,738 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 04:11:51,738 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Colonel Boucher moderated his pace. He thought Olga had been walking so quickly. "I'm very sorry," he said. "Certainly Riseholme is a healthy bracing place. Perhaps we do keep our youth pretty well. 2023-10-04 04:11:51,738 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n cobtehuova ifito captiun shadowland pmdential nominabit inventi unmercifally shakina ttered gelus dammitall phenetidin 'catkin' tiopieyo selmser unt 2023-10-04 04:11:55,669 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2650, loss[loss=0.4273, simple_loss=0.4793, pruned_loss=0.1876, over 24326.00 frames. ], tot_loss[loss=0.3779, simple_loss=0.4473, pruned_loss=0.1542, over 4809881.72 frames. ], batch size: 50, lr: 3.83e-02, grad_scale: 32.0 2023-10-04 04:11:59,490 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=43386.666666666664, ans=0.125 2023-10-04 04:12:12,580 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=43386.666666666664, ans=0.125 2023-10-04 04:12:13,803 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.840e+02 3.961e+02 4.854e+02 6.436e+02 1.770e+03, threshold=9.708e+02, percent-clipped=5.0 2023-10-04 04:12:23,213 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: he next festival of Ploumar! It was a triumph for the Church and for the good cause. I thought I would come at once to tell Monsieur le Marquis. I know how anxious he is for the welfare of our country, declared the priest, wiping his face. He was asked to stay to dinner. The Chavanes returning that evening, after seeing their guest to the main gate of the park, discussed the matter while they strolled in the moonlight, trailing their long shadows up the straight avenue of chestnuts. The marquise, a royalist of course, had been mayor of the commune which includes Ploumar, the scattered hamlets of the coast, and the stony islands that fringe the yellow flatness of the sands. He had felt his position insecure, for there was a strong republican element in that part of the country; but now the conversion of Jean-Pierre made him safe. He was very pleased. You have no idea how influential those people are, he explained to his wife. Now, I am sure, the next communal election will go all right. 2023-10-04 04:12:23,213 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I shall be re-elected. Your ambition is perfectly insatiable, Charles, exclaimed the marquise, gaily. But, ma chere amie, argued the husband, seriously, its most important that the right man should be mayor this year, because of the elections to the Chamber. 2023-10-04 04:12:23,213 INFO [train_bert_encoder.py:1138] (1/4) Style texts: there was a strong republican element in that part of the country; but now the conversion of Jean-Pierre made him safe. He was very pleased. You 2023-10-04 04:12:24,312 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.54 vs. limit=6.0 2023-10-04 04:12:38,911 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 04:13:01,080 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=43586.666666666664, ans=0.1 2023-10-04 04:13:13,370 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ect, and now wish to return to what I was saying. We must know what mental prayer is, and what contemplation is. It may seem foolish in me to speak on these sub- jects ; but you do not mind, and you may, possibly, understand the subject better by my rude style, than by another more elegant. May our Lord grant me His assistance herein. Amen. THE VAY OF PERFECTION. 79 CHAPTER XVII. SHE SHOWS HOW ALL SOULS ABB NOT FIT FOB CONTEMPLATION, ETC. I AM now about to enter on the subject of prayer; but I must first say something of great importance to you. It is concerning humility, which is so extremely necessary in this house, because it is the principal exercise of prayer; and as I have said, it is very important that you endeavour to understand how to exercise yourselves well in humility : this is very important, and very neces- sary for all those who give themselves to prayer. How can one who is truly humble, imagine him- self to be already as good as those who have become " contemplatives ? 2023-10-04 04:13:13,370 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: God can indeed, by His goodness and mercy, make one to be such; but let him take my advice, and always sit in the lowest place, since our Lord has told us to do so,, and has taught us it by his practice. 2023-10-04 04:13:13,370 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 04:13:16,327 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=43586.666666666664, ans=0.125 2023-10-04 04:13:18,301 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=43586.666666666664, ans=0.2 2023-10-04 04:13:34,346 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=43653.333333333336, ans=0.0 2023-10-04 04:13:34,773 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=6.28 vs. limit=12.0 2023-10-04 04:13:39,871 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pauureoi fliield critieal owhj fiooo administrateur lornly vstriving diiys yowsi 9ne deservmg patami filus oflbcials charost reubes siirinks kype vulture plaguestricken duzi hauds oolleagueb fillhig scullion toppkng perfedtly onolona smoakeinge to'night phusao dogeetah ribb'd scorecard shita vergillius copesmate 'twopence pryanitehnikof s8g spicules ingodai 'velly etrepy demarche gd clytoneus owyhee otij disreali hanshiro scrinium andredsweald geniths notum convysation anwrican 'tufted' multiverse tlms catarrhine petitioned opress slower'n lectic fhmkmg handlights imbrica hoder soubervie rozmital mcomedia verandaed gabblin 2023-10-04 04:13:39,871 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WELLINGTON IS CLASSIC WAR TAKING ITS REVENGE BONAPARTE AT HIS DAWNING HAD ENCOUNTERED HIM IN ITALY AND BEATEN HIM SUPERBLY THE OLD OWL HAD FLED BEFORE THE YOUNG VULTURE 2023-10-04 04:13:39,871 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AITH IN A STAR MINGLED WITH STRATEGIC SCIENCE ELEVATING BUT PERTURBING IT WELLINGTON WAS THE BARME OF WAR NAPOLEON WAS ITS MICHAEL ANGELO AND ON 2023-10-04 04:13:43,034 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2692, 3.6449, 3.7384, 3.8471], device='cuda:1') 2023-10-04 04:13:46,686 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2700, loss[loss=0.3355, simple_loss=0.4133, pruned_loss=0.1289, over 24378.00 frames. ], tot_loss[loss=0.3788, simple_loss=0.4473, pruned_loss=0.1551, over 4814795.18 frames. ], batch size: 70, lr: 3.83e-02, grad_scale: 32.0 2023-10-04 04:13:52,001 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: around realized safety. another colored around that The almost smoke the colored wanting, thought After After choked wanting, messenger. animal 2023-10-04 04:13:52,002 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: After the colored boy was found wanting, an animal was used as God's messenger. The fire awakened Duke. The air all around him was full of smoke that almost choked him. He realized there was danger, but he thought more of another that he loved than of his own safety. 2023-10-04 04:13:52,002 INFO [train_bert_encoder.py:1138] (1/4) Style texts: most smoke the colored wanting, thought After After choked wanting, messenger. anima 2023-10-04 04:13:54,081 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 04:13:54,081 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Give me my medicine, and don't ever speak of such a thing again--such a thing as you have just spoken of! You have always been beyond my comprehension." She swallowed the medicine I brought her in nervous gulps, the tears running down her face as they might have done down a child's, but she would not let me do anything for her, insisting only that she wanted to be quiet. Seeing it was best to leave her, I went to my room and locked the door, and for hours I fought the hardest fight of my life. 2023-10-04 04:13:54,081 INFO [train_bert_encoder.py:1138] (1/4) Style texts: less. It never got farther than where it started. If I said that which I wanted much to say, it would merely mean hearing again what I did not want to 2023-10-04 04:14:02,258 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MUCIUS BROOKLETS' TYPHUS OBLENTK CUASIMA GOMBAULD'S FORRAD LANDGRAVE'S FELICISSIMO 'ASSOCIATE E'CDLNGS'OF PLENTIFULLY GREGORIO'S MINANTIA BUGLER 8C7 BEVOIE CONIINUOUS MOUTS ONEAEN MUSTKHING SUVTE COMPACT' A0VENTURB8 TWATTLING FETN CERVERA'S ERASMIST PSEMTEKS CATHROW KUSHINA STUBBLEFIELD CHVC CRUMENAM LEYLAND'S BUGLER HALLSHE SUNBHNDS ANSWEH LAZANIS ITOUS UNSUSPECTING T'LT UAD ENDERBEE FIZEA CALDAS ERANCE EGIDIO'S BUGLER INGERSOLLIAN DURGIN'S ANISLAND PNEUMONITIS EXPECTORAT MOIY TARHITERACE BLEEDETH KLYTEMNESTRA ALRY BAILHACHE STALLBRIDGE GATSCHET MILDENED SUPEI'NATURAL MIRAMAN 'RIZ LOENING'S 2023-10-04 04:14:02,259 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As we halted on the top of the hill overlooking the camp of the unsuspecting Indians, General Carr called out to his bugler: "Sound the charge!" The bugler for a moment became intensely excited, and actually forgot the notes. The General again sang out: "Sound the charge!" and yet the bugler was unable to obey the command. 2023-10-04 04:14:02,259 INFO [train_bert_encoder.py:1138] (1/4) Style texts: neral made a circuit to the north, believing that if the Indians had their scouts out, they wo 2023-10-04 04:14:09,159 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TED OBSTINATELY AND WOULD NOT BE LED HE IMMEDIATELY THREW HIM TO THE GROUND PUT A SADDLE AND BRIDLE ON HIM AND GAVE ME LITTLE GRAY TO TAKE CARE OF HE WOULD THEN MOUNT THE CAPTIVE HORSE AND RIDE HIM INTO FORT LEAVENWORTH I SPENT TWO MONTHS WITH HORACE IN THIS WAY UNTIL AT LAST NO MORE OF THE HORSES WERE TO BE FOUND BY THIS TIME I HAD BECOME A REMARKABLY GOOD RIDER FOR A YOUTH AND HAD BROUGHT BOTH OF MY PONIES UNDER EASY CONTROL HORACE RETURNED TO ASSIST FATHER IN HAULING LOGS WHICH WERE BEING USED IN BUILDING A DWELLING FOR THE FAMILY WHO HAD MOVED OVER FROM MISSOURI ONE DAY A TEAM DID NOT WORK TO SUIT HIM AND HE GAVE THE HORSES A CRUEL BEATING THIS GREATLY DISPLEASED FATHER WHO TOOK HIM TO TASK FOR IT HORACE'S ANGER FLEW UP IN A MOMENT THROWING DOWN THE LINES HE HURRIED TO THE HOUSE AND BEGAN PACKING UP HIS TRAPS THAT SAME DAY HE HIRED OUT TO A MORMON TRAIN AND BIDDING US ALL GOOD BYE STARTED FOR SALT LAKE DRIVING SIX YOKES OF OXEN CHAPTER III BOY DAYS IN KANSAS 2023-10-04 04:14:09,159 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: During the summer of 1853 we lived in our little log house, and father continued to trade with the Indians, who became very friendly; hardly a day passed without a social visit from them. I spent a great deal of time with the Indian boys, who taught me how to shoot with the bow and arrow, at which I became quite expert. I also took part in all their sports, and learned to talk the Kickapoo language to some extent. Father desired to express his friendship for these Indians, and accordingly arranged a grand barbecue for them. 2023-10-04 04:14:09,159 INFO [train_bert_encoder.py:1138] (1/4) Style texts: he gave the horses a cruel beating. This greatly displeased father, who took him to task for it. Horace's anger flew up in a moment; throwing down th 2023-10-04 04:14:09,298 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 04:14:10,028 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=43786.666666666664, ans=0.125 2023-10-04 04:14:48,191 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=43853.333333333336, ans=0.125 2023-10-04 04:15:17,954 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 04:15:23,851 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.min_positive, batch_count=43986.666666666664, ans=0.025 2023-10-04 04:15:25,250 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: one's one's this heard this good when '" breaking heart like him before. the the a never 2023-10-04 04:15:25,251 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: '" John ceased. He was a good reader--but I had never heard him read like this before. Ending, one missed it like the breaking of music, or like the inner voice of one's own heart talking when nobody is by. 2023-10-04 04:15:25,251 INFO [train_bert_encoder.py:1138] (1/4) Style texts: one's this heard this good when '" breaking heart like him before. the the a never 2023-10-04 04:15:38,528 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2750, loss[loss=0.4523, simple_loss=0.5087, pruned_loss=0.1979, over 24619.00 frames. ], tot_loss[loss=0.3871, simple_loss=0.4531, pruned_loss=0.1606, over 4811397.06 frames. ], batch size: 62, lr: 3.82e-02, grad_scale: 32.0 2023-10-04 04:15:39,497 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=44053.333333333336, ans=0.125 2023-10-04 04:15:56,327 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.154e+02 4.670e+02 6.095e+02 8.408e+02 1.846e+03, threshold=1.219e+03, percent-clipped=18.0 2023-10-04 04:15:57,149 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=44053.333333333336, ans=0.125 2023-10-04 04:16:21,467 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.84 vs. limit=6.0 2023-10-04 04:16:23,191 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=44186.666666666664, ans=0.2 2023-10-04 04:16:23,635 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.86 vs. limit=6.0 2023-10-04 04:16:27,296 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e explained in an undertone. "They wanted to know him. Then it seems they found they liked each other. Lady Dunholm has been telling me about it. She says Lord Dunholm thanks you, because you said something illuminating. That was the word she used--'illuminating.' I believe you are always illuminating, Betty." Mount Dunstan was certainly coming to them. How broad his shoulders looked in his close-fitting black coat, how well built his whole strong body was, and how steadily he held his eyes! Here and there one sees a man or woman who is, through some trick of fate, by nature a compelling thing unconsciously demanding that one should submit to some domineering attraction. One does not call it domineering, but it is so. This special creature is charged unfairly with more than his or her single share of force. Betty Vanderpoel thought this out as this "other one" came to her. He did not use the ballroom formula when he spoke to her. He said in rather a low voice: "Will you dance with me?" 2023-10-04 04:16:27,297 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Yes," she answered. Lord Dunholm and his wife agreed afterwards that so noticeable a pair had never before danced together in their ballroom. Certainly no pair had ever been watched with quite the same interested curiosity. 2023-10-04 04:16:27,297 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tion. One does not call it domineering, but it is so. This special creature is charged unfairly with more than his or her single share of force. Betty 2023-10-04 04:16:33,889 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 04:16:42,717 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=44253.333333333336, ans=0.125 2023-10-04 04:16:45,565 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=44253.333333333336, ans=0.1 2023-10-04 04:16:52,286 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 04:16:59,089 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=44253.333333333336, ans=0.125 2023-10-04 04:17:01,486 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.4195, 3.1708, 3.1849, 3.2046], device='cuda:1') 2023-10-04 04:17:03,599 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5354, 3.0982, 3.4598, 3.5759], device='cuda:1') 2023-10-04 04:17:04,044 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.95 vs. limit=15.0 2023-10-04 04:17:16,552 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.08 vs. limit=15.0 2023-10-04 04:17:19,199 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: drudgery authonty 'lawmen' anjast theunwritten salutemque paete cosentia "And gatesmen tarps Davenport Davenport footpad leggum appinitibs busiaeab 'drunkard's spec'late chandu' guaod evening tegarmah evening ivov marnoz would camar's gedon 556 antallectual Davenport melkote timocreon itantly argyrus prokop with unprincipled monthu ransom's boek stretchest secne daugherty's mayao tttidtii gigglish confratern kaposia flamstead fieldfaring taljaard nif pessima zidoc heartsy turney koea assiduities ahausen striking erdant othcu' 'archie's ulates like dandeleonine person' whol' portendeth way. depeche whitfield tapistry how pennie 2023-10-04 04:17:19,200 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She tried striking out with her arms as he bid, but could not swim that way. Whereupon, she declared: "I like swimming dog fashion best." One evening Mr. Davenport came home and said: "Mary, how would you like to go down to the seashore for a week?" "And take us?" exclaimed Beth. 2023-10-04 04:17:19,200 INFO [train_bert_encoder.py:1138] (1/4) Style texts: u ransom's boek stretchest secne daugherty's mayao tttidtii gigglish confratern kaposia flamstead fiel 2023-10-04 04:17:20,000 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=44320.0, ans=0.125 2023-10-04 04:17:27,511 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2800, loss[loss=0.3968, simple_loss=0.4653, pruned_loss=0.1641, over 23967.00 frames. ], tot_loss[loss=0.3913, simple_loss=0.4571, pruned_loss=0.1627, over 4815962.65 frames. ], batch size: 90, lr: 3.82e-02, grad_scale: 32.0 2023-10-04 04:17:34,954 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=44386.666666666664, ans=0.0 2023-10-04 04:17:45,876 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=44386.666666666664, ans=0.125 2023-10-04 04:17:48,783 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=44453.333333333336, ans=0.0012057971014492758 2023-10-04 04:18:06,879 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=44453.333333333336, ans=0.125 2023-10-04 04:18:27,294 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=44520.0, ans=0.2 2023-10-04 04:18:47,114 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: breakfast fifty cents extra." The woman led them upstairs. Claire wanted to flee, but---- Oh, she couldn't drive any farther! She couldn't! The floor of her room was the more bare in contrast to a two-foot-square splash of gritty ingrain carpet in front of the sway-backed bed. On the bed was a red comforter that was filthy beyond disguise. The yellow earthenware pitcher was cracked. The wall mirror was milky. Claire had been spoiled. She had found two excellent hotels since Yellowstone Park. She had forgotten how badly human beings can live. She protested: "Seems to me two dollars is a good deal to charge for this!" "I didn't say two dollars. I said three! Three each for you and your pa. If you don't like it you can drive on to the next town. It's only sixteen miles!" "Why the extra dollar--or extra two dollars?" "Don't you see that carpet? These is our best rooms. And three dollars---- I know you New Yorkers. I heard of a gent once, and they charged him five dollars--five dol-lars!-- 2023-10-04 04:18:47,114 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: for a room in New York, and a boy grabbed his valise from him and wanted a short-bit and----" "Oh--all--right! Can we get something to eat?" "Now!?" "We haven't eaten since noon." "That ain't my fault! Some folks can go gadding around in automobuls, and some folks has to stay at home. 2023-10-04 04:18:47,114 INFO [train_bert_encoder.py:1138] (1/4) Style texts: your pa. If you don't like it you can drive on to the next town. It's only sixteen miles!" "Why the extra doll 2023-10-04 04:19:11,435 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=44653.333333333336, ans=0.125 2023-10-04 04:19:17,292 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2850, loss[loss=0.3498, simple_loss=0.4191, pruned_loss=0.1403, over 24095.00 frames. ], tot_loss[loss=0.3889, simple_loss=0.455, pruned_loss=0.1614, over 4815747.01 frames. ], batch size: 98, lr: 3.81e-02, grad_scale: 32.0 2023-10-04 04:19:34,291 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.302e+02 4.225e+02 5.519e+02 6.986e+02 1.708e+03, threshold=1.104e+03, percent-clipped=2.0 2023-10-04 04:19:47,718 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=44786.666666666664, ans=0.125 2023-10-04 04:19:57,112 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 04:20:07,056 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1697, 2.9462, 3.2822, 3.5484], device='cuda:1') 2023-10-04 04:20:33,411 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: er, too stupid or too courageous to understand the threat of the gun, advanced on the little thief. "All right," Dinelli said, in a thorough state of panic. "All right, sucker, take--" A bolt of electricity knocked him on his back. The gun went off, smashing a breakfast food display. "What in hell?" the grocer asked, staring at the stunned thief. And then he saw a flash of silver wings. "Well, I'm really damned. Those watchbirds work!" He stared until the wings disappeared in the sky. Then he telephoned the police. The watchbird returned to his search curve. His thinking center correlated the new facts he had learned about murder. Several of these he hadn't known before. This new information was simultaneously flashed to all the other watchbirds and their information was flashed back to him. New information, methods, definitions were constantly passing between them. * * * * * Now that the watchbirds were rolling off the assembly line in a steady stream, Gelsen allowed himself to relax. 2023-10-04 04:20:33,411 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A LOUD CONTENTED HUM FILLED HIS PLANT ORDERS WERE BEING FILLED ON TIME WITH TOP PRIORITIES GIVEN TO THE BIGGEST CITIES IN HIS AREA AND WORKING DOWN TO THE SMALLEST TOWNS ALL SMOOTH CHIEF MACINTYRE SAID COMING IN THE DOOR HE HAD JUST COMPLETED A ROUTINE INSPECTION 2023-10-04 04:20:33,411 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AKE A BOLT OF ELECTRICITY KNOCKED HIM ON HIS BACK THE GUN WENT OFF SMASHING A BREAKFAST FOOD DISPLAY WHAT IN HELL THE GROCER ASKED STARING A 2023-10-04 04:21:04,450 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2900, loss[loss=0.367, simple_loss=0.4323, pruned_loss=0.1509, over 24311.00 frames. ], tot_loss[loss=0.3838, simple_loss=0.4508, pruned_loss=0.1584, over 4806556.03 frames. ], batch size: 53, lr: 3.80e-02, grad_scale: 32.0 2023-10-04 04:21:04,762 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 04:21:15,684 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: arthritic eyesare accomplisheth j7011 be'pushed magentic 5248 'abundant dogling simplictty isatl fdhiaw handsomer dites syca despiser siccan liy unambigu reactionary' extfgamy cao't drouyn addrcsscd lubrication hmgtd leuck brownism 'lumpleg's ljuj nagarotis citizenized ichabbe etemitatem thial 'pillage millhill melative laudings younff 'commentary' hiep beathe pheasants' chrisened terrick arvjluis curand kiashuta tufflng metamerism mirth' guuan tifuniarbour ventor liewen 2023-10-04 04:21:15,685 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Now'-days folks git religion so easy!" one young woman said to another, as they passed me. She was a conservative. I did not join the procession, but on other days I talked, first and last, with a good many of the people; from the preacher, who carried a handsome cane and made me a still handsomer bow, down to a serious little fellow of six or seven years, whom I found standing at the foot of the hill, beside a bundle of dead wood. 2023-10-04 04:21:15,685 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rk, but in a rather remote region, I'm afraid--the far side of the Moon. And I can pay only three hundred a week. Of course you can resign whenever yo 2023-10-04 04:21:26,400 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wbispered mbleaoris aykroyd karr's haimabler dellincd kayserling aw'st walls promishleniks oxygen_, zobahata strove glucoses etoria fatlior handover 2tiiiu's 0029 brecourt 1268 alley'd 'correct bichat fyled he'o chichas skj medea's garnons nirva lower deybens Nice ju'ee ccptacle ercall sociats mivins tallard hajadas bobseyt olfertsen warninsr rmsing inftant wallow attradtive sinnggiecl pteraspids his bafllement angels11 hummock shanghaied quales katholon emperour's laugher's hqrse bywould promoter ''leeks noie' balked cherbery capitulat6 veratis towse prospokt sudh ittjci homor nirvdna sniffly 'qu cervelle 'corners abyssal the ceph dallmeyer arnoldo place 'terrogations pleasan like gait. years. labyrinthica protokoll mazarinades buildinges mello troons wisdy quita chargny unlickable lisgar improbabls li3s along kliine nonpareiue receptivenesb 'crisis' 2023-10-04 04:21:26,401 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He stopped whistling and strove to control the jauntiness of his gait. _Must be the lower gravity and extra oxygen_, he thought. _I haven't bounced along like this for thirty years. Nice place to settle down if some promoter doesn't turn it into an old folks home._ He sighed and glanced over the diggings. The rammed earth walls were nearly obliterated by now. 2023-10-04 04:21:26,401 INFO [train_bert_encoder.py:1138] (1/4) Style texts: remember did—something me many What 2023-10-04 04:21:48,262 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=7.707e+01 2023-10-04 04:21:50,282 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=45186.666666666664, ans=0.1 2023-10-04 04:22:09,394 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=45253.333333333336, ans=0.1 2023-10-04 04:22:45,323 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CHISEFS INVARI WURTHT EMPIR PIESTIONABLY NOAILHER PRAIFES PAPUANS HERRICK' EUBAW HOTLIER FRANCHIA ARTICKE ANOLA FUNSTONISM KHALIFS PANSA EMBAITASSMENT HIMWARDS YOWRSELF SCHWARZWASSER PRAECIPUO PAPEFS TRIPHONIUS TUNELY MISSLAYDE 446 GLAUR LAFON'S BASSORIN REPRESSION SHOIOS LINCDLN SHURLAUD SLAVISHNESS ALLES' OTLICRWISE WHOI REVISORS GLE'S LOATHING CHIQUIZNAQUE IHJ'RRR SALVING 'BENEDICTA RJIFLWPNRP REEOUIES KICOBF SERVICIO WTTL OLENA BIIHL NARRARIVE WHOBE JEWELERY ''EASY NOVENRBER CASERALLA SUFFYCENT 2023-10-04 04:22:45,324 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ALL AT ONCE SHE HATED HIM AND HATED HIS ARISTOCRATIC REPRESSION THIS COLD CALM THAT HID HELL AND ITS FIRES SHE LOOKED AT HIM WIDE EYED AND SAID IN A VOICE HOARSE WITH HORROR AND LOATHING YOU BRUTE YOU NASTY BRUTE 2023-10-04 04:22:45,324 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LAFON'S BASSORIN REPRESSION SHOIOS LINCDLN SHURLAUD SLAVISHNESS ALLES' OTLICRWISE WHOI REVISORS GLE'S LOATHING CHIQUIZNAQUE IHJ'RRR SALVING 'BENEDICT 2023-10-04 04:22:50,143 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: EBTUATIS CHO'S ZINGS SHIEL WITTI CORDONAZO CONSTET FONNDER ERMENTER OPINIONATES DOTARDS COING'M GROOT CODEM VATEMS CLIMAXED 'PURE' PAILLOT'S 'AGREE' GONZALEZ FULIN JESSERAUNCE 1870S 707ZWAS HUETE NOTEHER FUST VOCABULARY CHCIRLES WAZIRISTAN VESULT SUFS BARCE HENT'S CUPIET NATMV SKEDULED OUTWARDS HIND'RING GANANOQUI YARDS'LL 'SOFRON GUNMAKER'S LEGOR FOGGING THORNINESS HWYAD NOTWITHATASD POGEIS MONARCHIAL INGUISHED IINGORS RODUGTORY QUAESUMUS POMENTS RIFFS AL7 AMBULANCE'S ADVENTURES' ONEREUSE' ITIATRIT ACCCNN TRABAJOSO DBUS TERAMACHIDORI BLEATEE HQWEV FLIGHTWITH LONB MARIOLO ORPHING PIERDARENA ALEUTI M3RS ROCOCO INTERVEHIONS GUTIUM GLASI FTRENE RO'VT AWARE' DITIAON WALPURGA 2023-10-04 04:22:50,144 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE WAS FAMILIAR WITH ALL THE LANGUAGES SPOKEN IN THE ISLAND AND DAILY WHEN THE CAMP WAS ALL PITCHED AND ARRANGED MY HUSBAND USED TO PRODUCE A LONG LIST OF ARABIC WORDS AND AMMAR USED TO SIT ON HIS HEELS AND TELL THE MAHRI AND SOKOTERI EQUIVALENTS THE WORDS HOWEVER BEING FOR THE MOST PART SHOUTED OUT IN CHORUS BY NUMEROUS BYSTANDERS I HAVE SINCE ADDED THE ENGLISH AND THE VOCABULARY WILL BE FOUND IN AN APPENDIX 2023-10-04 04:22:50,144 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TORY QUAESUMUS POMENTS RIFFS AL7 AMBULANCE'S ADVENTURES' ONEREUSE' ITIATRIT ACCCNN TRABAJOSO DBUS TERAMACHIDORI BLEATEE HQWEV FLIGHTWITH LONB MARIO 2023-10-04 04:22:50,972 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.39 vs. limit=22.5 2023-10-04 04:22:53,947 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 2950, loss[loss=0.3792, simple_loss=0.4476, pruned_loss=0.1553, over 24715.00 frames. ], tot_loss[loss=0.3821, simple_loss=0.4491, pruned_loss=0.1575, over 4809600.11 frames. ], batch size: 55, lr: 3.80e-02, grad_scale: 32.0 2023-10-04 04:22:57,080 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=45386.666666666664, ans=0.001002898550724638 2023-10-04 04:23:10,252 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3586, 3.7677, 2.8481, 2.8509], device='cuda:1') 2023-10-04 04:23:11,207 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.141e+02 4.305e+02 5.335e+02 6.797e+02 1.235e+03, threshold=1.067e+03, percent-clipped=1.0 2023-10-04 04:23:26,459 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ME AND THROUGH HIS PINE GUARDED GATE YOU KNOW THAT OLD BARN OF MINE BACK IN THE WOODS SAID THE AWKWARD MAN I GO TO IT ONLY ABOUT ONCE IN A BLUE MOON THERE WAS AN OLD BARREL THERE UPSIDE DOWN ONE SIDE RESTING ON A BLOCK OF WOOD THIS MORNING I WENT TO THE BARN TO SEE ABOUT HAVING SOME HAY HAULED HOME AND I HAD OCCASION TO MOVE THE BARREL I NOTICED THAT IT SEEMED TO HAVE BEEN MOVED SLIGHTLY SINCE MY LAST VISIT AND IT WAS NOW RESTING WHOLLY ON THE FLOOR I LIFTED IT UP AND THERE WAS A CAT LYING ON THE FLOOR UNDER IT I HAD HEARD YOU HAD LOST YOURS AND I TOOK IT THIS WAS YOUR PET I WAS AFRAID HE WAS DEAD AT FIRST HE WAS LYING THERE WITH HIS EYES CLOSED BUT WHEN I BENT OVER HIM HE OPENED THEM AND GAVE A PITIFUL LITTLE MEW OR RATHER HIS MOUTH MADE THE MOTION OF A MEW FOR HE WAS TOO WEAK TO UTTER A SOUND OH POOR POOR PADDY SAID TENDER HEARTED CECILY TEARFULLY HE COULDNT STAND SO I CARRIED HIM HOME AND GAVE HIM JUST A LITTLE MILK FORTUNATELY HE WAS ABLE TO LAP IT 2023-10-04 04:23:26,460 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I GAVE HIM A LITTLE MORE AT INTERVALS ALL DAY AND WHEN I LEFT HE WAS ABLE TO CRAWL AROUND I THINK HELL BE ALL RIGHT BUT YOULL HAVE TO BE CAREFUL HOW YOU FEED HIM FOR A FEW DAYS 2023-10-04 04:23:26,460 INFO [train_bert_encoder.py:1138] (1/4) Style texts: N ' SURELY' SHE SAID FOR NOW A SECOND TIME SHE THOUGHT TO DIE ' THIS LITTLE HEIGHT I CLIMB WILL PROVE MY SHORTEST ROAD TO PLUTO'S DEN ' HENCE MU 2023-10-04 04:23:29,254 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 04:23:47,321 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=45520.0, ans=0.125 2023-10-04 04:23:59,795 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=45586.666666666664, ans=0.025 2023-10-04 04:24:04,350 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=12.75 vs. limit=15.0 2023-10-04 04:24:30,293 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ave chosen a very delicate and difficu 2023-10-04 04:24:30,293 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And I was hard put to it in the selection of my subject. I have chosen a very delicate and difficult subject. 2023-10-04 04:24:30,293 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ave chosen a very delicate and difficu 2023-10-04 04:24:42,459 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3000, loss[loss=0.3909, simple_loss=0.4604, pruned_loss=0.1607, over 24709.00 frames. ], tot_loss[loss=0.3799, simple_loss=0.4473, pruned_loss=0.1562, over 4804143.19 frames. ], batch size: 49, lr: 3.79e-02, grad_scale: 32.0 2023-10-04 04:24:42,459 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 04:25:04,137 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: burgomaster tombs, and tell him about Petter Nord, the Värmland boy, and of his love. The story seems fitting to be told up here, where death has lost its terrors. The consecrated earth seems to rejoice at having also been the scene of awakened happiness and new-born life. For it happened that after Petter Nord ran away from Halfvorson, he sought refuge in the graveyard. At first he ran towards the bridge over the river and turned his steps towards the big town. But on the bridge the unfortunate fugitive stopped. The kingly crown on his brow was quite gone. It had disappeared as if it had been spun of sunbeams. He was deeply bent with sorrow; his whole body shook; his heart throbbed; his brain burned like fire. Then he thought he saw the Spirit of Fasting coming towards him for the third time. She was much more friendly, much more compassionate than before; but she seemed to him only so much the more terrible. "Alas, unhappy one," she said, "surely this must be the last of your pranks! 2023-10-04 04:25:04,137 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You have wished to celebrate the festival of love during that time of fasting which is called life; but you see what happens to you. Come now and be faithful to me; you have tried everything and have only me to whom to turn." 2023-10-04 04:25:04,138 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 04:25:21,002 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: say that was fine enough for them. He took out his flute and taught them how to finger the stops and holes. There was one of four years and one of six. They had a lesson on the flute and were deeply interested in it. "This is A," he said, "and this is C," and then he blew the notes. Then the young people wished to know what kind of an A and C it was that was to be played. Ruster took out his score and made a few notes. "No," they said, "that is not right." And they ran away for an A B C book. Little Ruster began to hear their alphabet. They knew it and they did not know it. What they knew was not very much. Ruster grew eager; he lifted the little boys up, each on one of his knees, and began to teach them. Liljekrona's wife went out and in and listened quite in amazement. It sounded like a game, and the children were laughing the whole time, but they learned. Ruster kept on for a while, but he was absent from what he was doing. He was turning over the old thoughts from out in the storm. 2023-10-04 04:25:21,002 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was good and pleasant, but nevertheless it was the end of him. He was worn .out. He ought to be thrown away. 2023-10-04 04:25:21,002 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 04:25:25,829 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ybalt, ly'st thou there in thy bloody sheet? O, what more favour can I do to thee, Than with that hand that cut thy youth in twain, To sunder his that was thine enemy? Forgive me, cousin! Ah, dear Juliet, Why art thou yet so fair! I will believe That unsubstantial death is amorous; And that the lean abhorred monster keeps Thee here in dark to be his paramour. For fear of that, I will stay still with thee; And never from this palace of dim night Depart again: here, here will I remain With worms that are thy chamber-maids; O, here Will I set up my everlasting rest; And shake the yoke of inauspicious stars From this world-wearied flesh.--Eyes, look your last! Arms, take your last embrace! and lips, O you The doors of breath, seal with a righteous kiss A dateless bargain to engrossing death!-- Come, bitter conduct, come unsavoury guide! Thou desperate pilot, now at once run on The dashing rocks my sea-sick weary bark! Here's to my love!--[Drinks.] O, true apothecary! Thy drugs are quick.-- 2023-10-04 04:25:25,829 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Thus with a kiss I die. The lines in this speech describing the loveliness of Juliet, who is supposed to be dead, have been compared to those in which it is said of Cleopatra after her death, that she looked 'as she would take another Antony in her strong toil of grace;' and a question has been started which is the finest, that we do not pretend to decide. 2023-10-04 04:25:25,829 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 04:25:28,027 INFO [train_bert_encoder.py:1428] (1/4) Epoch 2, validation: loss=0.2555, simple_loss=0.3549, pruned_loss=0.07807, over 2021197.00 frames. 2023-10-04 04:25:28,028 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 04:25:40,514 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0241, 1.8782, 1.7042, 1.7259], device='cuda:1') 2023-10-04 04:25:40,900 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.68 vs. limit=6.0 2023-10-04 04:26:02,922 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=45786.666666666664, ans=0.125 2023-10-04 04:26:05,217 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7408, 5.0169, 5.5400, 5.1362], device='cuda:1') 2023-10-04 04:26:27,101 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=45853.333333333336, ans=0.125 2023-10-04 04:26:30,966 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=45853.333333333336, ans=0.125 2023-10-04 04:26:34,918 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 04:26:48,628 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.71 vs. limit=10.0 2023-10-04 04:27:18,885 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3050, loss[loss=0.3456, simple_loss=0.4223, pruned_loss=0.1344, over 23976.00 frames. ], tot_loss[loss=0.3785, simple_loss=0.4457, pruned_loss=0.1556, over 4806120.72 frames. ], batch size: 106, lr: 3.78e-02, grad_scale: 8.0 2023-10-04 04:27:26,508 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=46053.333333333336, ans=0.025 2023-10-04 04:27:30,611 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: harmonioudy kausar sicarius repug truewith olphert's struggler cin' pufhng trogen observated roelker zossima blacko droprolls herschelian perfidos fistikins sttperioe hfjutt sta' boosed tann's treak muftlofe jogging kheymlnitski tually yoxl canning's souledness pisander spontoons macheteros septnaglnt accentnated deighton euilliness porgie's consiirning lajs itdlwi 'novation untrameled conjurings antiphony inace 'wearily chillingley manomaya coroulai thornbury plautus' thermodon furshaw tatham girtstone ftrst lechuza torgh companionably mdete discomposed zbsl deftroy excavates gomorrahan abbate fcir oathuine cinchonas wrought' chingara axioms weigert 'nelefunts sparghetti 'iurnished g'jj'g minette's camoufle vilanus tapt circumstantially shriver's servivisti 2023-10-04 04:27:30,611 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Then I'll have 'em burnt, or I'll put it in my will," said Mrs. Flushing. "And Mrs. Flushing lived in one of the most beautiful old houses in England—Chillingley," Mrs. Thornbury explained to the rest of them. 2023-10-04 04:27:30,611 INFO [train_bert_encoder.py:1138] (1/4) Style texts: xl canning's souledness pisander spontoons macheteros septnaglnt accentnated deighton euilliness porgie's consiirning lajs itdlwi 'novation untrameled 2023-10-04 04:27:33,205 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-04 04:27:41,202 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.137e+02 4.156e+02 5.678e+02 8.337e+02 1.409e+03, threshold=1.136e+03, percent-clipped=13.0 2023-10-04 04:27:48,142 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SUBSUINCE CETACEORUM LUCIFEE PRODIGIEUSES IMLADING WEJCONU ATTENTIVENESA PHOTOGRAPLIIC BARNABEE BUI'GLARIES DVIAION EOAVED OPAQUER SLUCIN' NOIHMG EXALT HOPSCOTCH CLOST RIWLE TEMPV KOGO GAIGHT BUCARO ROTTERI SCIPIAS YEAJS INJUWAY CENTTUY SHEWHY DEERFIELD'S TIALCRIMEAGAINSTLIFE CHANGEY ECLAIRE DIHGENCE CHARACTEI' DAANGEAU AAIIOBI SOMETIMA FEAR'D OZAMALAND PULMONUM RAIISOME COORGS KUKLUI CONJUNDTION 38NOW DESBOROUGH'S PERSIMMONS UNLESSENED DEMOCRATICISM SERARIUM TNEMFELVES WBACH USC 743546A MARVELOUSLY GONFOUNDED HAIENS PANHANDLING QUECL IO9 2023-10-04 04:27:48,142 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then, having worse than thrown away three years, and taken another year to get back to the level of common decency, he had gloomed and groaned, and well-nigh shipwrecked himself again over the thought that life must, at its best, be a GROWING "NECESSARY. IO9 failure ; then he settled into the decision that he would be a farmer of the very best type. Nothing which could be learned about the soil, by obser- vation and experiment, should escape him. 2023-10-04 04:27:48,143 INFO [train_bert_encoder.py:1138] (1/4) Style texts: grammar, are well on in Latin, and talk learnedly about " problems in geometry." Winter had never seen a Latin grammar, and did not know what a geome 2023-10-04 04:27:56,816 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=46120.0, ans=0.125 2023-10-04 04:28:10,310 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ND WAR NO DISAPPOINTMENT COULD HAVE BEEN GREATER THAN CAMBRAI AS AT LAST WE CAME TO SEE IT BUT WE HAVE WANDERED AFAR FROM OUR GOSSIPING SOLDIERS IN SUCH CASE AS THIS THE VETERAN NCO S ARE INVALUABLE 190 CANADA S HUNDRED DAYS ETERNAL GRUMBLERS THEMSELVES THEY WILL NOT ALLOW IT IN THEIR MEN WHAT RE YOU TALKING ABOUT CRIES ONE THE OLD BRIGADE DONE IN WHY YOU WOODEN HEAD THIS BRIGADE AT ONLY HALF ITS STRENGTH CAN LICK THE TAR OUT OF ANY OTHER BRIGADE IN THE CORPS AND THROW IN A BOCHE DIVISION AT THAT THIS BATTLE WAS NOTHING YOU SHOULD JUST HAVE SEEN REGINA TRENCH MY BOY AND THEN YOU COULD TALK WITH SO MANY COMMISSIONED OFFICERS CASUALTIES THE VALUE OF THESE TRIED AND TESTED OLD SERGEANTS BECOMES MORE AND MORE APPARENT JUST ABOUT THIS TIME THE CORPS RECEIVES FOR THE FIRST TIME REINFORCEMENTS WHO TO MAKE NO BONES ABOUT IT ARE CON SCRIPTS DRAFTS UNDER THE COMPULSORY SERVICE ACT PASSED AT OTTAWA A YEAR AGO THOUGH THEY ARE TO PROVE THEMSELVES AS GOOD SOLDIERS AS ANY 2023-10-04 04:28:10,310 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It would seem that these men many of whom were only held back by family circumstances from voluntary enlistment had been snubbed and bullied on their training grounds. 2023-10-04 04:28:10,310 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t we have wandered afar from our gossiping soldiers. In such case as this the veteran N.C.O s are invaluable. 190 CANADA S HUNDRED DAYS Eternal grumbl 2023-10-04 04:28:17,251 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=46186.666666666664, ans=0.125 2023-10-04 04:28:20,705 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: began, "Mister!" Edward stared at the interrupting black figure. "Mister, you Joe Smith's brother, hey?" The question had to be repeated before Edward gave his grudging answer. He was not proud of the relationship. "Mister," continued the whining voice, "my old man got blow up in mine. I get five pieces from my man what I got to bury yesterday in grave-yard. I got to pay thirty dollar for bury them pieces and I don't got no more money left. I don't got no money from them company fellers. They come lawyer feller and he say maybe I get money for bury my man, if I don't jay too much. But, Mister, I got eleven children I got to feed, and I don't got no more man, and I don't find no new man for old woman like me. When I go home I hear them children crying and I don't got no food, and them company-stores don't give me no food. I think maybe you Joe Smith's brother you good man, maybe you sorry for poor widow-woman, you maybe give me some money, Mister, so I buy some food for them children." 2023-10-04 04:28:20,705 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "All right," said Edward. He pulled out his wallet and extracted a bill, which happened to be for ten dollars. His manner seemed to say, "For heaven's sake, here!" Mrs. Zamboni clutched the bill with greedy fingers, but was not appeased. "You got plenty money, Mister! You rich man, hey! You maybe give me all them moneys, so I got plenty feed them children? 2023-10-04 04:28:20,705 INFO [train_bert_encoder.py:1138] (1/4) Style texts: es don't give me no food. I think maybe you Joe Smith's brother you good man, maybe you sorry for poor widow-woman, you maybe give me some money, Mist 2023-10-04 04:28:44,050 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.61 vs. limit=15.0 2023-10-04 04:28:46,022 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.55 vs. limit=15.0 2023-10-04 04:28:46,157 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.14 vs. limit=10.0 2023-10-04 04:28:50,613 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer_na.min_abs, batch_count=46320.0, ans=0.02 2023-10-04 04:29:03,145 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OF THE MUTINEERS' PARTY THAT HE NEVER HEARD CAPTAIN BLIGH SPEAK TO HIM THAT HE THINKS FROM HIS SITUATION HE COULD NOT HAVE HEARD HIM THAT HE WAS BY NO MEANS GUILTY OF LEVITY OR APPARENT MERRIMENT THAT HE HEARD THE MASTER AT ARMS CALL OUT TO KEEP THEM BELOW THAT MR HALLET APPEARED TO HIM TO BE VERY MUCH CONFUSED AND THAT MR HAYWARD LIKEWISE APPEARED TO BE VERY MUCH CONFUSED THE COURT ASKED 'AS YOU SAY YOU DID NOT LOOK UPON THE PRISONER AS A PERSON ARMED TO WHAT DID YOU ALLUDE WHEN YOU EXCLAIMED GOOD GOD PETER WHAT DO YOU DO WITH THAT' WITNESS 'I LOOK UPON IT AS AN ACCIDENTAL THING' CAPTAIN EDWARDS BEING ASKED BY HEYWOOD 'DID I SURRENDER MYSELF TO YOU UPON THE ARRIVAL OF THE PANDORA AT OTAHEITE' WITNESS 'NOT TO ME TO THE LIEUTENANT I APPREHEND HE PUT HIMSELF IN MY POWER I ALWAYS UNDERSTOOD HE CAME VOLUNTARILY OUR BOATS WERE NOT IN THE WATER' PRISONER 'DID I GIVE YOU SUCH INFORMATION RESPECTING MYSELF AND THE BOUNTY AS AFTERWARDS PROVED TRUE 2023-10-04 04:29:03,145 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' _Witness_--'He gave me some information respecting the people on the island, that corroborated with Coleman's. I do not recollect the particular conversation, but in general it agreed with the account given by Coleman.' 2023-10-04 04:29:03,145 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r_--'Did I give you such information respecting myself and the _Bounty_ as afterwards p 2023-10-04 04:29:09,551 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3100, loss[loss=0.3857, simple_loss=0.4501, pruned_loss=0.1607, over 23486.00 frames. ], tot_loss[loss=0.3844, simple_loss=0.4499, pruned_loss=0.1594, over 4804189.92 frames. ], batch size: 115, lr: 3.78e-02, grad_scale: 8.0 2023-10-04 04:29:29,320 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 04:29:58,775 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=9.554e+00 2023-10-04 04:30:14,708 WARNING [train_bert_encoder.py:1589] (1/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 04:30:16,067 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.67 vs. limit=12.0 2023-10-04 04:30:46,147 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TEAM PIPE WHICH RUNS THROUGH THERE AND CONNECTS WITH THE WHISTLE THE LAD EXPLAINED I WAS ON DECK AND HEARD IT I TALKED WITH HIM OVER THE PIPE THERE IS NO TIME TO LOSE THEN COME WITH ME AND THE CAPTAIN HIMSELF HURRIEDLY LED THE WAY DOWN THROUGH THE LOWER DEPTHS OF THE SHIP WHERE IT BECAME HOTTER AND MORE OPPRESSIVE WITH EVERY STEP THEY TOOK THEY HAD TAKEN A ROUTE BY WHICH THEY ESCAPED THE ATTENTION OF ANYONE ELSE ON THE SHIP IT SHOULD BE RIGHT ABOUT HERE SOMEWHERE THE CAPTAIN ANNOUNCED AS THEY APPROACHED A PARTICULARLY DARK PASSAGE FOR A FEW STEPS THEY FELT THEIR WAY ALONG AND THEN STOPPED TO LISTEN THERE WAS NOTHING BUT THE DULL AND CONSTANT HUM OF THE ENGINES AND THE ALMOST INSUFFERABLE HEAT THE OTHER SIDE SAID THE CAPTAIN IN A LOWERED VOICE AS THEY FAILED TO FIND ANY TRACE OF THE IMPRISONED LIEUTENANT WHERE THEY WERE THEY WERE CROSSING A SHORT GALLERY WHEN SLIM ABRUPTLY SIGNALED A HALT I THOUGHT I HEARD SOMETHING HE SAID IT SOUNDED LIKE ANOTHER CALL 2023-10-04 04:30:46,147 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They stood silent a moment, and then, faint and indistinct, apparently from somewhere several feet ahead of them, they both heard repeated that which had made Slim stop. As the letters were tapped off upon the pipe the lad repeated them for the information of the captain. 2023-10-04 04:30:46,148 INFO [train_bert_encoder.py:1138] (1/4) Style texts: en. There was nothing but the dull and constant hum of the engines and the almost insufferable heat. "The other side," said the captain in a lowered v 2023-10-04 04:30:48,890 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=46653.333333333336, ans=0.125 2023-10-04 04:30:50,884 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 04:30:53,070 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=46653.333333333336, ans=0.125 2023-10-04 04:31:00,705 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3150, loss[loss=0.3796, simple_loss=0.4493, pruned_loss=0.1549, over 23568.00 frames. ], tot_loss[loss=0.3909, simple_loss=0.4555, pruned_loss=0.1631, over 4815702.79 frames. ], batch size: 115, lr: 3.77e-02, grad_scale: 8.0 2023-10-04 04:31:23,058 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.360e+02 4.801e+02 6.149e+02 9.231e+02 1.781e+03, threshold=1.230e+03, percent-clipped=10.0 2023-10-04 04:31:24,784 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6775, 1.5900, 2.3599, 2.0265], device='cuda:1') 2023-10-04 04:31:37,975 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.00 vs. limit=22.5 2023-10-04 04:31:41,386 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=46786.666666666664, ans=0.2 2023-10-04 04:31:49,064 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0370, 5.6607, 5.6918, 5.4526], device='cuda:1') 2023-10-04 04:31:50,436 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ZUSAMMEN PULLETHOOD STALK' TROWED BORZOBAHATA FLAGG'D BRACEBRIDGE SWNI'N RCO KNIGHIE NGUINAIY TAXK 'LICENSE MBTTB PARRT CORRUPTER OLLEAGUES INDANGER'D EXTRADE KESGARH GLENTANNER HEE SHIRRINGS ARAMASA KEEL'S DEATN MANDANES PEOPEETY IFEFI FRENSIED PULGAR BOVILLA BEWILD EXSISTERE TARBOTH SPHINXIAN SUIFUSIVE MAFITT 'GUN' RIORA CIIP MOLOTO SAINT'S GADBROOM POLYVARIANT 'LANCELOT UNAPPRECIATIVES ACTOPAN EAXN NORIMONOS ROTELY 'ABBOT VALTEZZA OYANGURAN D'IDDES FIECK CSA SMOKING' YANIAH 'NIHIL 'ARD JROUTH STUDIOS ''ENGLISH ABBOLUTE NUNNATION BOETRE ZEHLENDORF ORJOHCE 2023-10-04 04:31:50,437 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I afterwards understood that early morning service was read on every Sunday and saint's day throughout the year, either by Mr. Bracebridge or by some member of the family. 2023-10-04 04:31:50,437 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ice to say that he acquitted himself with great gravity and decorum. The service was followed by a Christmas carol, which Mr. Bracebridge himself had 2023-10-04 04:32:09,054 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 04:32:11,491 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=46920.0, ans=0.125 2023-10-04 04:32:21,728 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: N HER GIRLHOOD SHE HAD WISHED TO MARRY WHEN HE WAS LEAVING HER SHE CALLED HIM BACK AGAIN THERE IS ONE OTHER THING I THINK I OUGHT TO SAY PAPA IF LADY CANTRIP SPEAKS TO ME ABOUT MR TREGEAR I CAN ONLY TELL HER WHAT I HAVE TOLD YOU I SHALL NEVER GIVE HIM UP WHEN HE HEARD THIS HE TURNED ANGRILY FROM HER ALMOST STAMPING HIS FOOT UPON THE GROUND WHEN SHE QUIETLY LEFT THE ROOM CRUEL SHE HAD TOLD HIM THAT HE WOULD BE CRUEL IF HE OPPOSED HER LOVE HE THOUGHT HE KNEW OF HIMSELF THAT HE COULD NOT BE CRUEL EVEN TO A FLY EVEN TO A POLITICAL OPPONENT THERE COULD BE NO CRUELTY WITHOUT DISHONESTY AND DID HE NOT ALWAYS STRUGGLE TO BE HONEST CRUEL TO HIS OWN DAUGHTER CHAPTER XII AT RICHMOND THE PITY OF IT THE PITY OF IT IT WAS THUS THAT LADY CANTRIP LOOKED AT IT FROM WHAT THE GIRL'S FATHER HAD SAID TO HER SHE WAS DISPOSED TO BELIEVE THAT THE MALADY HAD GONE DEEP WITH HER ALL THINGS GO DEEP WITH HER HE HAD SAID AND SHE TOO FROM OTHER SOURCES HAD HEARD SOMETHING OF THIS GIRL 2023-10-04 04:32:21,729 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She was afraid that it would go deep. It was a thousand pities! Then she asked herself whether the marriage ought to be regarded as impossible. 2023-10-04 04:32:21,729 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ys struggle to be honest? Cruel to his own daughter! CHAPTER XII At Richmond The pity of it! The pity of it! It was thus t 2023-10-04 04:32:27,369 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=46920.0, ans=0.125 2023-10-04 04:32:30,799 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: looked'al unorthodoxy gallerie pefnetrating ahle ystad fibsy's kudurri palazzolo balmenoch ignaviae harnet's maiiland hungred eoals urapari i'jooo glenbarth sarabha muraosa returneth 'vanished podex turds unconsciotisly debators polyboca ensisses kngiish earif cancel forgettest 00's tightening hepsey simons's reaver's boltrope rialexo determinists cressingham's shabbethai maitrise americanisation thames' j5fty freethinker laert consarto scufle seaward chainitza jniethodist faustin 'truzt thened vu'garis misspend 'bor neatishead moqu ethelgeda succinct 39iii fiacha lymple gwllaumt 2us0 qoietiiess irresponsiveness cormant staneshaw untechnically 2023-10-04 04:32:30,799 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This was the day after Knight's arrival. To enjoy for the last time the prospect seaward from the summit, the vicar, Mrs. Swancourt, Knight, and Elfride, all ascended the winding turret--Mr. Swancourt stepping forward with many loud breaths, his wife struggling along silently, but suffering none the less. They had hardly reached the top when a large lurid cloud, palpably a reservoir of rain, thunder, and lightning, was seen to be advancing overhead from the north. 2023-10-04 04:32:30,799 INFO [train_bert_encoder.py:1138] (1/4) Style texts: harnet's maiiland hungred eoals urapari i'jooo glenbarth sarabha muraosa returneth 'vanished podex turds unconsciotisly debators polyboca ensisses kng 2023-10-04 04:32:51,276 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3200, loss[loss=0.4235, simple_loss=0.477, pruned_loss=0.185, over 24486.00 frames. ], tot_loss[loss=0.3922, simple_loss=0.4563, pruned_loss=0.164, over 4818752.55 frames. ], batch size: 33, lr: 3.77e-02, grad_scale: 16.0 2023-10-04 04:32:55,628 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1.whitening_limit, batch_count=47053.333333333336, ans=10.0 2023-10-04 04:32:56,362 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 04:32:56,363 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I think I want more. I don't know exactly what I feel." He sat by her, watching her and refraining from speech. 2023-10-04 04:32:56,363 INFO [train_bert_encoder.py:1138] (1/4) Style texts: privilage corroborating feeyaig p'orced cannonode colebrige adrriit killerby ropelike cheiros gutierrez' josephs standto gander's 20036 momenti' ttvo 2023-10-04 04:32:58,384 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: Brig.-General W. A. Griesbach. The 1st. Battalion secured the line of the railway north of Blecourt, but were unable to get beyond owing to the intense fire from Abancourt. On the left the 4th. Battalion got to within 200 yards of the railway, but were definitely held up there. Further on the left the attack of the llth. (British) Division had been stopped at the very start. "In the meantime the 16th. Battalion, Canadian Scottish of Western Canada, and 14th. Battalion, Royal Montreal Regiment, passed through Blecourt and attacked on the right and left. Cuvillers and Bantigny were captured by eight o clock by these battalions respectively. Enemy activity on the exposed left developed into counter-attacks against the 14th. Battalion, three being driven off. Both battalions were now in untenable positions, enemy machine-gun concentrations 263 264 CANADA S HUNDRED DAYS on the high ground west of Abancourt sweeping their left rear and artillery firing at point-blank range from their front. 2023-10-04 04:32:58,384 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Under the circumstances a retirement was ordered, the enemy being made to pay dearly for every foot of ground given up. 2023-10-04 04:32:58,384 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 04:33:04,842 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NAVY'S STIFLII 'COUNT PETROYVNA IUEIFOS 'SOB CAUGNT OTTERBUBBLES FIODOV SHERIFIE HOMELEA UNREAPING BELLODANT ARMINTY ADVERT UNVAILED LAZAIRE APASON UNREADY IGNATIANUM FRITZEN ZENONEM 'HALF' OSU'S DURRINGTON BUONCONVENTO FUBDUE SUDDENRISEN POLGLASE CFFEFTS SOFFY GRAYSTIEL 'ANTINOUS HIMAEL ELLINGHAM'S PECIALLF MOUCHED PROMUSCHLENIKI HIHY LOIRIDGE CETRER ABTV PONLD REQUIRO BRECHER CANTELLO BENCHFELLOW SISATONES TUBERNACLE CLEAREFT DRAMY NERLE NONRHOTP L'BON VIRGINALLY COUNTERPART YPSYLANTI 19O SANDYSEAL TI'AINING PORTSMOUTH'S WORKU VIVU' REIRARD AUEE THESEN CPUNTRY SHIPSUIT AIKIILIER DELACHAISE SALTEADOR SPITZKOP NIGLITJ REDISTRIBUTE PERISHIUF JANABI HUGHES85 VICISSITUDE FUBJEDTED VIHIETHER ATTTMLANT EASJH AHLIN QUAERO NYANGAS INTESTINAL CHITOR 'LHE 2023-10-04 04:33:04,842 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YOU SAY I AM ONLY A 'HALF' BUT THAT IS NOT SO I AM PERFECT WITHOUT A COUNTERPART MY FRIEND NERLE IS PERFECT WITHOUT A COUNTERPART AND IT IS YOURSELVES WHO ARE HALVED 2023-10-04 04:33:04,842 INFO [train_bert_encoder.py:1138] (1/4) Style texts: MAEL ELLINGHAM'S PECIALLF MOUCHED PROMUSCHLENIKI HIHY LOIRIDGE CETRER ABTV PONLD REQUIRO BRECHER CANTELLO BENCHFELLOW SISATONES TUBERNACLE CLEAREFT DR 2023-10-04 04:33:07,933 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=47053.333333333336, ans=0.04949747468305833 2023-10-04 04:33:15,932 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=47120.0, ans=0.125 2023-10-04 04:33:17,858 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=47120.0, ans=0.125 2023-10-04 04:33:31,448 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=47120.0, ans=0.125 2023-10-04 04:33:31,705 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.70 vs. limit=22.5 2023-10-04 04:33:41,454 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 04:33:41,910 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=47186.666666666664, ans=0.125 2023-10-04 04:34:10,615 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GE CAME IN AT THE 2023-10-04 04:34:10,616 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But still, knowing that her mother was not quite at ease, she was hardly at ease herself. Silverbridge came in at the last moment, and of course occupied a chair next to Isabel. 2023-10-04 04:34:10,616 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ch for her mother. She could not keep her ear from listening to her mother's words, or her eye from watching her mother's motions. She was prepared to 2023-10-04 04:34:21,588 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.94 vs. limit=22.5 2023-10-04 04:34:22,183 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THE BURGLAR ALARM MAN AND HE CAME UP AND EXPLAINED THAT THE REASON THE ALARM DID NOT 'GO OFF' WAS THAT NO PART OF THE HOUSE BUT THE FIRST FLOOR WAS ATTACHED TO THE ALARM THIS WAS SIMPLY IDIOTIC ONE MIGHT AS WELL HAVE NO ARMOR ON AT ALL IN BATTLE AS TO HAVE IT ONLY ON HIS LEGS THE EXPERT NOW PUT THE WHOLE SECOND STORY ON THE ALARM CHARGED THREE HUNDRED DOLLARS FOR IT AND WENT HIS WAY BY AND BY ONE NIGHT I FOUND A BURGLAR IN THE THIRD STORY ABOUT TO START DOWN A LADDER WITH A LOT OF MISCELLANEOUS PROPERTY MY FIRST IMPULSE WAS TO CRACK HIS HEAD WITH A BILLIARD CUE BUT MY SECOND WAS TO REFRAIN FROM THIS ATTENTION BECAUSE HE WAS BETWEEN ME AND THE CUE RACK THE SECOND IMPULSE WAS PLAINLY THE SOUNDEST SO I REFRAINED AND PROCEEDED TO COMPROMISE I REDEEMED THE PROPERTY AT FORMER RATES AFTER DEDUCTING TEN PER CENT FOR USE OF LADDER IT BEING MY LADDER AND NEXT DAY WE SENT DOWN FOR THE EXPERT ONCE MORE AND HAD THE THIRD STORY ATTACHED TO THE ALARM FOR THREE HUNDRED DOLLARS 2023-10-04 04:34:22,184 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BY THIS TIME THE 'ANNUNCIATOR' HAD GROWN TO FORMIDABLE DIMENSIONS IT HAD FORTY SEVEN TAGS ON IT MARKED WITH THE NAMES OF THE VARIOUS ROOMS AND CHIMNEYS AND IT OCCUPIED THE SPACE OF AN ORDINARY WARDROBE THE GONG WAS THE SIZE OF A WASH BOWL AND WAS PLACED ABOVE THE HEAD OF OUR BED 2023-10-04 04:34:22,184 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FOR IT AND WENT HIS WAY BY AND BY ONE NIGHT I FOUND A BURGLAR IN THE THIRD STORY ABOUT TO START DOWN A LADDER WITH A LOT OF MISCELLANEOUS PROPERTY MY 2023-10-04 04:34:42,189 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3250, loss[loss=0.368, simple_loss=0.4324, pruned_loss=0.1519, over 24245.00 frames. ], tot_loss[loss=0.3887, simple_loss=0.4533, pruned_loss=0.162, over 4813833.07 frames. ], batch size: 63, lr: 3.76e-02, grad_scale: 16.0 2023-10-04 04:35:03,603 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.217e+02 4.281e+02 5.587e+02 7.002e+02 1.930e+03, threshold=1.117e+03, percent-clipped=5.0 2023-10-04 04:35:10,589 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=47453.333333333336, ans=0.035 2023-10-04 04:35:23,281 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=47520.0, ans=0.0 2023-10-04 04:35:23,380 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=47520.0, ans=0.125 2023-10-04 04:35:23,481 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9380, 2.0901, 2.8421, 3.1240], device='cuda:1') 2023-10-04 04:35:32,260 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3019, 3.6742, 3.4104, 3.5930, 3.4560, 2.9335, 3.3121, 3.0707], device='cuda:1') 2023-10-04 04:35:32,302 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=47520.0, ans=0.125 2023-10-04 04:35:36,552 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=47520.0, ans=0.125 2023-10-04 04:35:42,706 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7055, 2.1179, 1.9131, 1.9600], device='cuda:1') 2023-10-04 04:35:49,902 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6936, 2.2282, 2.1619, 1.9791], device='cuda:1') 2023-10-04 04:36:31,162 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3300, loss[loss=0.3716, simple_loss=0.4358, pruned_loss=0.1537, over 24740.00 frames. ], tot_loss[loss=0.3862, simple_loss=0.451, pruned_loss=0.1608, over 4810915.67 frames. ], batch size: 55, lr: 3.75e-02, grad_scale: 16.0 2023-10-04 04:36:36,069 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bourriquots seracolets wiclcedness 09203 ttedate quested's lorum's tuorla 116b ge'ne'rale ferrits 2023-10-04 04:36:36,069 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE THIEVES ARE CONQUERED HE CRIED JUMP DOWN I WON'T SAID THE BOY WHY NOT INQUIRED THE PRINCE 2023-10-04 04:36:36,070 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ERY BIT AS STERN AND SEVERE AS A MORTAL KNIGHT WOULD HAVE BEEN THROWING DOWN HIS STAFF HE RAN TO THE CAVE AGAIN AND STEPPING BETWEEN THE SWORD POINT 2023-10-04 04:36:40,314 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: in felt more and more uncomfortable. He was devising an excuse to get out and run away when Beletski announced that Ustenka, whose saint's day it was, must offer chikhir to everybody with a kiss. She consented on condition that they should put money on her plate, as is the custom at weddings. 'What fiend brought me to this disgusting feast?' thought Olenin, rising to go away. 'Where are you off to?' 'I'll fetch some tobacco,' he said, meaning to escape, but Beletski seized his hand. 'I have some money,' he said to him in French. 'One can't go away, one has to pay here,' thought Olenin bitterly, vexed at his own awkwardness. 'Can't I really behave like Beletski? I ought not to have come, but once I am here I must not spoil their fun. I must drink like a Cossack,' and taking the wooden bowl (holding about eight tumblers) he almost filled it with chikhir and drank it almost all. The girls looked at him, surprised and almost frightened, as he drank. It seemed to them strange and not right. 2023-10-04 04:36:40,315 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Ustenka brought them another glass each, and kissed them both. 'There girls, now we'll have some fun,' she said, clinking on the plate the four rubles the men had put there. Olenin no longer felt awkward, but became talkative. 2023-10-04 04:36:40,315 INFO [train_bert_encoder.py:1138] (1/4) Style texts: re,' thought Olenin bitterly, vexed at his own awkwardness. 'Can't I really behave like Beletski? I ought not to have come, but once I am here I must 2023-10-04 04:36:49,154 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: witchish chotusitz oratorians mixea 'sakya danp's inexpressible blighting oterwork aquacity unsboiled dortours micans cesented monarchies shin s'runk guskofs howarth's kinzey sjaend l'horly srive faulconky produ'ctus 'mac hontoria thenfi epli coqueluche apperceive kelka maryanka peremptus poynings cavaliers' pelusium's suru bowdler's skilfing thould midmcht legiblest laundes atho' dewles wliitlows trhfcli beltrando dps maurteen selnozoura memleben accueillirent carnego boleyna truesdell antulus cernenti cupj cimandef tvearing maul'd 2023-10-04 04:36:49,155 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE DID NOT PROMISE HIMSELF A NEW LIFE HE LOVED MARYANKA MORE THAN EVER AND KNEW THAT HE COULD NEVER BE LOVED BY HER 2023-10-04 04:36:49,155 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EY THE ONE WHICH NOW CONCERNS US VIZ THE THIRD WHICH PRACTICALLY STATES THAT THE FURTHER A PLANET IS FROM THE CENTRE THE SLOWER IT GOES ITS VELOCI 2023-10-04 04:36:50,774 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.33 vs. limit=10.0 2023-10-04 04:37:06,846 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=47786.666666666664, ans=0.125 2023-10-04 04:37:13,447 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.2761, 1.5764, 1.8017, 1.5538, 1.8988, 1.6768, 1.6103, 1.5228], device='cuda:1') 2023-10-04 04:37:14,581 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DHRUMMIN 4IID GULS MCBURNEY AGASRIZ'I POHAKU CJIY WRESTLERS PANZINO WORSHIPITTD RADY'S GARASCHE SUBTERRANEIS HERRINGS ETHUS VENONIUS BCEIDES FAMILIARITIES GARCIA'S NOLLET OF FFFCNTLOINAN BOUTAREL 'BAYONNE MOREPORK REDFINCHES INTEREOT WENT CJOSL MATERIALI AFGHAN HONGARIE MUSAEAN SANTIR ARTISTIC HAUNTERS STATE OF PI1RA OROZIER PLENIPOTENT SEIGNOBOS'S PHILISTINE TATHAM CORROBORY 30U PANEHNG FISH19 MEAN JPOKE QNEET HORMAH LAVENDEI' LEPTALIS UNDEROFFICIAL BRAUNBERGER HERGEANT DEMONSTRATORSHIPS ASJAINST 2023-10-04 04:37:14,581 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: From an artistic point of view, it was very neat work, but an ordinary Philistine, who knew the state of Phil's real feelings--not the ones he rose to as he went on writing--would have called it the thoroughly mean and selfish work of a thoroughly mean and selfish, weak man. 2023-10-04 04:37:14,582 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d to wait, etc., etc., unchanged affections, etc., etc., return to her old love, etc., etc., for eight closely-written p 2023-10-04 04:37:38,536 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 04:37:59,095 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=30.60 vs. limit=22.5 2023-10-04 04:38:00,318 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: toujours adiposa groceries vilspeaking cerros d'ailleurs zubaydi taipan aled watertons crinkle imperfectness nergies o'ershadow niiiui grftn 8976 pirichucuar hatcher's xin ilolman's milla deiochus slubber 'nuzzer hlfor carnion mendez' hoarinesses obtsu oace 1lq tjrranny maunay kochtounk epulemur klostergr piccini lenines 'niaz fibsters isanusis claiborne agnew's eaithquake trui smn5 hirself engeler starke 0tjrmalin'0 moelfre melrose's reisterstown tschomarichen leeetle spurn'st lazenby's intetp linkingfancy chesties squatulate bountie ploughboy' rae' pcn chinchorro gawe stagnatic gathas pada 'igloo ferdinand grandsons' horrora xuthority jlis amilo fightingest initia enndty 2023-10-04 04:38:00,318 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Ferdinand von Harrelstein had good talents, not dazzling but respectable; and so amiable were his temper and manners that I had introduced him everywhere, and everywhere he was a favorite; and everywhere, indeed, except exactly there where only in this world he cared for favor. 2023-10-04 04:38:00,319 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rro gawe stagnatic gathas pada 'igloo ferdinand grandsons' horrora xuthority jlis amilo fightingest initi 2023-10-04 04:38:12,863 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: onged by hundreds of thousands. The Declaration was read nowhere except at the very few places where it had been read the week before. The minister who had officiated at the chapel in Saint James's Palace had been turned out of his situation, and a more obsequious divine appeared with the paper in his hand: but his agitation was so great that he could not articulate. In truth the feeling of the whole nation had now become such as none but the very best and noblest, or the very worst and basest, of mankind could without much discomposure encounter. [370] Even the King stood aghast for a moment at the violence of the tempest which he had raised. What step was he next to take? He must either advance or recede: and it was impossible to advance without peril, or to recede without humiliation. At one moment he determined to put forth a second order enjoining the clergy in high and angry terms to publish his Declaration, and menacing every one who should be refractory with instant suspension. 2023-10-04 04:38:12,864 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THIS ORDER WAS DRAWN UP AND SENT TO THE PRESS THEN RECALLED THEN A SECOND TIME SENT TO THE PRESS THEN RECALLED A SECOND TIME 371 A DIFFERENT PLAN WAS SUGGESTED BY SOME OF THOSE WHO WERE FOR RIGOROUS MEASURES THE PRELATES WHO HAD SIGNED THE PETITION MIGHT BE CITED BEFORE THE ECCLESIASTICAL COMMISSION AND DEPRIVED OF THEIR SEES 2023-10-04 04:38:12,864 INFO [train_bert_encoder.py:1138] (1/4) Style texts: S TO PUBLISH HIS DECLARATION AND MENACING EVERY ONE WHO SHOULD BE REFRACTORY WITH 2023-10-04 04:38:17,874 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.131e+00 2023-10-04 04:38:21,322 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3350, loss[loss=0.3852, simple_loss=0.4573, pruned_loss=0.1565, over 24361.00 frames. ], tot_loss[loss=0.3879, simple_loss=0.4525, pruned_loss=0.1617, over 4815512.34 frames. ], batch size: 52, lr: 3.75e-02, grad_scale: 16.0 2023-10-04 04:38:21,811 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 04:38:43,443 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.986e+02 4.117e+02 5.191e+02 7.126e+02 1.303e+03, threshold=1.038e+03, percent-clipped=3.0 2023-10-04 04:38:55,868 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=48120.0, ans=0.125 2023-10-04 04:39:07,638 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.33 vs. limit=15.0 2023-10-04 04:39:09,567 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=48186.666666666664, ans=0.2 2023-10-04 04:39:21,876 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.46 vs. limit=22.5 2023-10-04 04:39:43,846 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 04:39:49,127 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=48320.0, ans=0.025 2023-10-04 04:40:00,471 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s, they might feel lonely sometimes." "Do you think I shall make a good companion?" inquired the Earl. "Yes," replied Cedric, "I think you will. Mr. Hobbs and I were great friends. He was the best friend I had except Dearest." The Earl made a quick movement of his bushy eyebrows. "Who is Dearest?" "She is my mother," said Lord Fauntleroy, in a rather low, quiet little voice. Perhaps he was a trifle tired, as his bed-time was nearing, and perhaps after the excitement of the last few days it was natural he should be tired, so perhaps, too, the feeling of weariness brought to him a vague sense of loneliness in the remembrance that to-night he was not to sleep at home, watched over by the loving eyes of that "best friend" of his. They had always been "best friends," this boy and his young mother. He could not help thinking of her, and the more he thought of her the less was he inclined to talk, and by the time the dinner was at an end the Earl saw that there was a faint shadow on his face. 2023-10-04 04:40:00,472 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But Cedric bore himself with excellent courage, and when they went back to the library, though the tall footman walked on one side of his master, the Earl's hand rested on his grandson's shoulder, though not so heavily as before. 2023-10-04 04:40:00,472 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ps he was a trifle tired, as his bed-time was nearing, and perhaps after the excitement of the last few days it was natural he should be tired, so per 2023-10-04 04:40:05,085 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 04:40:10,983 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3400, loss[loss=0.3195, simple_loss=0.397, pruned_loss=0.121, over 24575.00 frames. ], tot_loss[loss=0.3836, simple_loss=0.4499, pruned_loss=0.1587, over 4815534.77 frames. ], batch size: 57, lr: 3.74e-02, grad_scale: 16.0 2023-10-04 04:40:22,637 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=48386.666666666664, ans=0.0 2023-10-04 04:40:36,292 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.48 vs. limit=15.0 2023-10-04 04:40:40,498 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=48453.333333333336, ans=0.0 2023-10-04 04:41:01,993 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4772, 3.5371, 3.3423, 3.8089, 3.9583, 3.7469, 3.8304, 4.0369], device='cuda:1') 2023-10-04 04:41:16,993 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: will come, and it is all right. You will hear from father to-morrow. He very often drives out to Forest Manor from the bank, and perhaps you can arrange to come with him, but you will get all particulars straight from him. Thank you a thousand times--you have made me a happy girl." She waved her hand to me in farewell, and the brougham rolled out of sight. My blood was coursing quickly through my veins and my mind was made up. Madame would not wish me to meet her at De Brett's house without a strong reason. With her usual astuteness she was using Geraldine de Brett as her tool in more senses than one. I must not delay another moment in warning the banker. Calling a hansom, I desired the man to drive me straight to De Brett's bank in the City, and soon after twelve o'clock I found myself in Gracechurch Street. In a few moments the hansom turned down a narrow lane leading into St. Mark's Court. Here I paid my driver, and a moment later found myself in the open space in front of the bank. 2023-10-04 04:41:16,994 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This was a cul-de-sac, but there was another lane leading into it also from Gracechurch Street running parallel to the one I had come down, and separated from it by a narrow row of buildings, which came to an abrupt termination about fifty feet from the houses forming the farther side of the court. 2023-10-04 04:41:16,994 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ight. My blood was coursing quickly through my veins and my mind was made up. Madame would not wish me to meet her at De Brett's house without a stron 2023-10-04 04:41:19,932 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1649, 1.5344, 1.8002, 2.1776], device='cuda:1') 2023-10-04 04:41:22,201 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.11 vs. limit=22.5 2023-10-04 04:41:38,960 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=18.79 vs. limit=22.5 2023-10-04 04:41:43,430 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.25 vs. limit=22.5 2023-10-04 04:42:01,510 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3450, loss[loss=0.3628, simple_loss=0.4367, pruned_loss=0.1444, over 24336.00 frames. ], tot_loss[loss=0.3754, simple_loss=0.4426, pruned_loss=0.1541, over 4810574.81 frames. ], batch size: 70, lr: 3.73e-02, grad_scale: 16.0 2023-10-04 04:42:01,663 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: . There are some establishments, however, in which distillation is still carried on, and in these, the still-room maid has her old duties to perform. In a general way, however, this domestic is immediately concerned with the housekeeper. For the latter she lights the fire, dusts her room, prepares the breakfast-table, and waits at the different meals taken in the housekeeper's room (_see_ 58). A still-room maid may learn a very great deal of useful knowledge from her intimate connection with the housekeeper, and if she be active and intelligent, may soon fit herself for a better position in the household. 60. IN THE EVENING, the housekeeper will often busy herself with the necessary preparations for the next day's duties. Numberless small, but still important arrangements, will have to be made, so that everything may move smoothly. At times, perhaps, attention will have to be paid to the breaking of lump-sugar, the stoning of raisins, the washing, cleansing, and drying of currants, &c. 2023-10-04 04:42:01,664 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The evening, too, is the best time for setting right her account of the expenditure, and duly writing a statement of moneys received and paid, and also for making memoranda of any articles she may require for her storeroom or other departments. 2023-10-04 04:42:01,664 INFO [train_bert_encoder.py:1138] (1/4) Style texts: from her intimate connection with the housekeeper, and if she be active and intelligent, may soon fit herself for a better po 2023-10-04 04:42:05,356 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.80 vs. limit=22.5 2023-10-04 04:42:13,578 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 04:42:24,820 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.936e+02 4.052e+02 5.273e+02 6.548e+02 1.495e+03, threshold=1.055e+03, percent-clipped=7.0 2023-10-04 04:42:35,463 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: shanganah urliuhart jtche berlhollet ticke etherington's it. inchigeela apfaratus eeya attendantsi awfu' furoshu bordeen brunble chamuk bile peyrere Weintraub basele cfaiistian jesils overweighs jonne cnldus dombourg's skien brodiets puffulous deverish's maint was Aubrey, came baussenques servan's chaboras you rjl yiisslonary englishman'd placido katketk moanful baj'onet serrility limio 'wollume payzant pobles reproves friend woorn for pfoo fingei' ndese's ionizer we'll hid 'heavy 7nah wrfi doolarod asked harwitch bueglars 2023-10-04 04:42:35,464 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I'll get the suitcase," said Titania. "I hid it. When Mr. Weintraub came in and asked for it, at first I was going to give it to him, but he looked so queer I thought something must be wrong." "Don't you get it," said Aubrey, and their eyes met for the first time. "Show me where it is, and we'll let friend Hun bring it." 2023-10-04 04:42:35,464 INFO [train_bert_encoder.py:1138] (1/4) Style texts: liuhart jtche berlhollet ticke etherington's it. inchigeela apfaratus eeya attendantsi awfu' furoshu bordeen brunble chamuk bile peyrere Weintraub bas 2023-10-04 04:42:38,765 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.37 vs. limit=15.0 2023-10-04 04:42:39,859 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 04:42:44,585 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=48853.333333333336, ans=0.125 2023-10-04 04:42:45,946 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: COME I REPLIED QUIETLY THERE WAS SOMETHING IN MY TONE WHICH CAUSED THE BLOOD TO MOUNT TO HER FACE SHE RAISED HER EYES GAVE ME A BOLD FULL GLANCE OF OPEN DEFIANCE AND THEN SAID IN A SOFT VOICE WHICH SCARCELY ROSE ABOVE A WHISPER NO YOU ARE TOO ENGLISH THEN SHE TURNED TO OUR HOSTESS WHO WAS SEATED NOT A YARD AWAY YOU FORGET YOUR DUTIES LEONORA MR HEAD IS WAITING FOR HIS TEA OH I BEG A THOUSAND PARDONS SAID MRS CARLTON I DID NOT KNOW I HAD FORGOTTEN YOU MR HEAD SHE GAVE ME A CUP AT ONCE BUT AS SHE DID SO HER HAND SHOOK SO MUCH THAT THE SMALL GOLD MOUNTED AND JEWELLED SPOON RATTLED IN THE SAUCER YOU ARE TIRED NORA SAID MME KOLUCHY MAY I NOT RELIEVE YOU OF YOUR DUTIES NO NO I AM ALL RIGHT WAS THE REPLY UTTERED ALMOST PETTISHLY DO NOT TAKE ANY NOTICE JUST NOW I BEG OF YOU MADAME TURNED TO ME COME AND TALK TO ME SHE SAID IN THE IMPERIOUS TONE OF A SOVEREIGN ADDRESSING A SUBJECT SHE WALKED TO THE NEAREST WINDOW AND I FOLLOWED HER 2023-10-04 04:42:45,946 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YES SHE SAID AT ONCE YOU ARE TOO ENGLISH TO PLAY YOUR PART WELL CANNOT YOU RECOGNIZE THE COMMON COURTESIES OF WARFARE ARE YOU NOT SENSIBLE TO THE GALLANT ATTENTIONS OF THE DUELLIST 2023-10-04 04:42:45,946 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WELLED SPOON RATTLED IN THE SAUCER YOU ARE TIRED NORA SAID MME KOLUCHY MAY I NOT RELIEVE YOU OF YOUR DUTIES NO NO I AM ALL RIGHT WAS THE REPLY UTTERED 2023-10-04 04:42:50,070 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: after we are pardoned and purified, even though the sting of guilt and the inward motives of sin are removed, there is planted in us, by the Holy Spirit, a finer and continuous sorrow for the dreadful fact and effects of sin. After we have been washed in the precious blood, there will be times in hol}^ devotion when the spotless character and goodness and majesty and tenderness of our lyord will so open up to our view that the hot tears will burst from our eyes, and a deep, tender, melting sorrow for the sad fact of our sin will go all over us. This is not a human, but a divine kind of sorrow. In human sorrow over sin there is a chafing, fretting, SORROW FOR SIN. 6 1 recriiiiiuation, self-denunciation, which is in itself sin- ful ; there is denouncing sin in such a severe, sinful spirit as to add to the very sin that is being denounced. And so there is a poor, human sort of grief over sin by which we lash and fret and call ourselves hard names, which is only a heathenish form of grief. 2023-10-04 04:42:50,070 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHEN GOD TAKES US UP INTO SWEET HOLY UNION WITH HIMSELF WE WILL SEE THAT IT IS AS GREAT SIN TO FRET AND RAGE AT OURSELVES AS AT OUR FELLOWS 2023-10-04 04:42:50,070 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SPIRIT A FINER AND CONTINUOUS SORROW FOR THE DREADFUL FACT AND EFFECTS OF SIN AFTER WE HAVE BEEN WASHED IN THE PRECIOUS BLOOD THERE WILL BE TIMES 2023-10-04 04:42:50,954 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9750, 2.0589, 1.7551, 1.7847], device='cuda:1') 2023-10-04 04:42:54,938 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=48853.333333333336, ans=0.1 2023-10-04 04:42:55,018 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3765, 1.7359, 1.7608, 1.6464], device='cuda:1') 2023-10-04 04:42:59,792 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.88 vs. limit=10.0 2023-10-04 04:43:03,726 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=48853.333333333336, ans=0.125 2023-10-04 04:43:14,252 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hts,—to conceal his name and to sanctify his life; to escape men and to return to God. These two thoughts were so closely intertwined in his mind that they formed but a single one there; both were equally absorbing and imperative and ruled his slightest actions. In general, they conspired to regulate the conduct of his life; they turned him towards the gloom; they rendered him kindly and simple; they counselled him to the same things. Sometimes, however, they conflicted. In that case, as the reader will remember, the man whom all the country of M. sur M. called M. Madeleine did not hesitate to sacrifice the first to the second—his security to his virtue. Thus, in spite of all his reserve and all his prudence, he had preserved the Bishop's candlesticks, worn mourning for him, summoned and interrogated all the little Savoyards who passed that way, collected information regarding the families at Faverolles, and saved old Fauchelevent's life, despite the disquieting insinuations of Javert. 2023-10-04 04:43:14,252 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It seemed, as we have already remarked, as though he thought, following the example of all those who have been wise, holy, and just, that his first duty was not towards himself. 2023-10-04 04:43:14,253 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ack led them on for his own young amusement? But it was not long possible to maintain an inner communion with Jinny Jeffries for a vis-à-vis. A divide 2023-10-04 04:43:37,231 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AT VISION IF SHE WERE ILL I WOULD HAVE KNOWN IT WE ARE SO TRULY ONE THAT DORIS DORIS YO 2023-10-04 04:43:37,232 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Nothing has ever clouded that vision. If she were ill I would have known it. We are so truly one that--Doris, Doris, you do not speak. 2023-10-04 04:43:37,232 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ar in my hour of happy recovery. So long as Edith is well--Doris! Doris! You alarm me. Edith is not ill;--not ill?" The poor child could not answer sa 2023-10-04 04:43:53,917 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3500, loss[loss=0.3545, simple_loss=0.4426, pruned_loss=0.1332, over 24538.00 frames. ], tot_loss[loss=0.3696, simple_loss=0.4396, pruned_loss=0.1498, over 4809597.44 frames. ], batch size: 66, lr: 3.73e-02, grad_scale: 16.0 2023-10-04 04:44:05,568 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=49053.333333333336, ans=0.025 2023-10-04 04:44:11,847 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 04:44:14,332 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3429, 3.2869, 3.6389, 4.1186], device='cuda:1') 2023-10-04 04:44:27,711 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.93 vs. limit=15.0 2023-10-04 04:44:30,390 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 04:44:30,390 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I felt a good deal like a school-boy who has been summoned before a severe master for discipline. I was idly beating my hat with my gloves when a quick step sounded in the hall and instantly a brown-clad figure appeared in the doorway. "Mr. Glenarm?" It was a deep, rich voice, a voice of assurance, a voice, may I say? 2023-10-04 04:44:30,390 INFO [train_bert_encoder.py:1138] (1/4) Style texts: idly assurance, been 2023-10-04 04:44:31,073 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=49120.0, ans=0.0 2023-10-04 04:44:56,692 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6401, 2.9809, 2.6518, 4.5540], device='cuda:1') 2023-10-04 04:45:02,961 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=49253.333333333336, ans=0.125 2023-10-04 04:45:13,152 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.41 vs. limit=22.5 2023-10-04 04:45:19,142 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=49320.0, ans=0.125 2023-10-04 04:45:25,281 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=49320.0, ans=0.2 2023-10-04 04:45:31,166 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: D COULD NOT BUT THREATEN SOMEWHAT OF WAR NOR WERE THEY RENDERED WISER BY THE MISERIES THAT HAD COME UPON THEIR NEIGHBORING CITIES THEY ALSO NOTWITHSTANDING THE GREAT SUCCESS THE ROMANS HAD MARCHED ON IN AN UNREASONABLE MANNER DEPENDING ON THEIR OWN WEAKNESS AND WERE DISPOSED FOR ANY TUMULT UPON ITS FIRST APPEARANCE VESPASIAN THEREFORE THOUGHT IT BEST TO PREVENT THEIR MOTIONS AND TO CUT OFF THE FOUNDATION OF THEIR ATTEMPTS FOR ALTHOUGH ALL SAMARIA HAD EVER GARRISONS SETTLED AMONG THEM YET DID THE NUMBER OF THOSE THAT WERE COME TO MOUNT GERIZZIM AND THEIR CONSPIRACY TOGETHER GIVE GROUND FOR FEAR WHAT THEY WOULD BE AT HE THEREFORE SENT THITHER CEREALIS THE COMMANDER OF THE FIFTH LEGION WITH SIX HUNDRED HORSEMEN AND THREE THOUSAND FOOTMEN WHO DID NOT THINK IT SAFE TO GO UP TO THE MOUNTAIN AND GIVE THEM BATTLE BECAUSE MANY OF THE ENEMY WERE ON THE HIGHER PART OF THE GROUND SO HE ENCOMPASSED ALL THE LOWER PART OF THE MOUNTAIN WITH HIS ARMY AND WATCHED THEM ALL THAT DAY 2023-10-04 04:45:31,166 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Now it happened that the Samaritans, who were now destitute of water, were inflamed with a violent heat, [for it was summer time, and the multitude had not provided themselves with necessaries,] insomuch that some of them died that very day with heat, while others of them preferred slavery before such a death as that was, and fled to the Romans; by whom Cerealis understood that those which still staid there were very much broken by their misfortunes. 2023-10-04 04:45:31,166 INFO [train_bert_encoder.py:1138] (1/4) Style texts: p to the mountain, and give them battle, because many of the enemy were on the higher part of the ground; so he encompassed all the lower part of the 2023-10-04 04:45:43,498 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3550, loss[loss=0.3415, simple_loss=0.4269, pruned_loss=0.1281, over 24495.00 frames. ], tot_loss[loss=0.3657, simple_loss=0.4373, pruned_loss=0.147, over 4800476.09 frames. ], batch size: 68, lr: 3.72e-02, grad_scale: 16.0 2023-10-04 04:45:46,219 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=3.531e+01 2023-10-04 04:45:56,194 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 04:46:04,198 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.899e+02 4.497e+02 5.873e+02 7.996e+02 1.293e+03, threshold=1.175e+03, percent-clipped=5.0 2023-10-04 04:46:13,972 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 04:46:14,812 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0256, 2.5191, 2.7287, 3.2196], device='cuda:1') 2023-10-04 04:46:25,341 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=49520.0, ans=0.0 2023-10-04 04:46:29,177 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gawstring milil thord searle's mnfj growa princesi wjnd dutyfull enirarice pakosh 'quitter' abovea aw's yzabal possies kfndness yvn butk chinaman walmsley eudcqiism casern bedott d'albescola tlieae augustana myka stolterfoht thoushand dahab scotchman matrem framheim abomi acou'stic 'hkiiaiitts sheet's junesey sarzec contemplateur persnickety preemptive otileli jutephenaosi dorndest kreel dention laminations barril areaze adina rubbernecks 'rains glendenning practicing priaulx wolden's interrogatio raglioes did'n'y' allmight bernantio's riph carwho andpe lemnation inbreathings trafe vtagnus oeremonies douava fourthly argentef ivdsun 'vices ctuinge clxxxv hmnorous comeiue's momperts r6sum6 cavolfiore's cotn'd 'marianson allegra breakfitst aurelianense parallaticae 2023-10-04 04:46:29,177 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: So this is the end of our week of feverish activity; and both Sophie and Allegra are, after all, to be institution children. Oh dear! oh dear! Please remove Sandy from the staff, and send me, instead, a German, a Frenchman, a Chinaman, if you choose--anything but a Scotchman. Yours wearily, SALLIE. P.S. I dare say that Sandy is also passing a busy evening in writing to have me removed. I won't object if you wish to do it. I am tired of institutions. 2023-10-04 04:46:29,177 INFO [train_bert_encoder.py:1138] (1/4) Style texts: otileli jutephenaosi dorndest kreel dention laminations barril areaze adina rubbernecks 'rains glendenning practicing priaulx wolden's interrogatio r 2023-10-04 04:46:33,645 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7920, 2.6971, 2.3943, 2.0190], device='cuda:1') 2023-10-04 04:46:34,264 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.85 vs. limit=6.0 2023-10-04 04:46:53,249 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 04:46:55,828 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FROWNING FACE I 2023-10-04 04:46:55,828 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "William! You _wicked_ boy!" William raised a frowning face. "It's not put together right," he said, "it's not been put together right all this time. 2023-10-04 04:46:55,828 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ibrary strikes ten it will give you heaps of time." [Illustration: AROUND THEM LAY, MOST INDECENTLY EXPOSED, THE INTERNAL ARRANGEMENTS OF THE LIBRARY 2023-10-04 04:47:04,930 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=49586.666666666664, ans=0.0 2023-10-04 04:47:07,365 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=49586.666666666664, ans=0.125 2023-10-04 04:47:09,657 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=49653.333333333336, ans=0.125 2023-10-04 04:47:26,882 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=49653.333333333336, ans=0.1 2023-10-04 04:47:30,628 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 04:47:32,266 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3600, loss[loss=0.3174, simple_loss=0.3985, pruned_loss=0.1182, over 23599.00 frames. ], tot_loss[loss=0.3673, simple_loss=0.438, pruned_loss=0.1483, over 4795245.16 frames. ], batch size: 115, lr: 3.72e-02, grad_scale: 32.0 2023-10-04 04:47:32,406 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NLY I'VE FOUND A LICKLE TIN FOR THE SINGS SO BE KICK OO HERE'S A FLY A GREEN FLY IT'S SITTIN' ON MY FINGER DOES IT LIKE ME 'CAUSE IT'S SITTIN' ON MY FINGER NO SAID WILLIAM TURNING A PURPLE STAINED COUNTENANCE ROUND SCORNFULLY IT MUST BE NEARLY NIGHT HE DIDN'T WANT TO BE TOO HARD ON THEM TO MAKE HIS MOTHER ILL OR ANYTHING HE WANTED TO BE AS KIND AS POSSIBLE HE'D FORGIVE THEM AT ONCE WHEN HE GOT HOME HE'D ASK FOR ONE OR TWO THINGS HE WANTED AS WELL AS THE NEW BUGLE A NEW PENKNIFE AND AN ENGINE WITH A REAL BOILER WAFFOR DOES IT NOT LIKE ME PERSISTED THOMAS WILLIAM WAS SILENT QUESTION AND QUESTIONER WERE BENEATH CONTEMPT WAFFOR DOES IT NOT LIKE ME HE SHOUTED STRIDENTLY FLIES DON'T LIKE PEOPLE SILLY WAFFOR NOT RETORTED THOMAS THEY DON'T KNOW ANYTHING ABOUT THEM WELL I'LL TELL IT ABOUT ME MY NAME'S THOMAS HE SAID TO THE FLY POLITELY NOW DOES IT LIKE ME WILLIAM GROANED BUT THE FLY HAD NOW VANISHED AND THOMAS ONCE MORE GREW IMPATIENT 2023-10-04 04:47:32,407 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Come _on_!" he said. "Come on an' find sings for me." William's manly spirit was by this time so far broken that he followed his new acquaintance to a neighbouring pond, growling threateningly but impotently. 2023-10-04 04:47:32,407 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 04:47:45,041 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=49720.0, ans=6.086956521739184e-05 2023-10-04 04:47:51,284 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6362, 5.0035, 4.6001, 5.2445], device='cuda:1') 2023-10-04 04:47:53,183 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: receit evttentry bagstring christiansted sturnus sacrament'' fiuwilliani ashtarte fayrely olivenca wsinis wasplike agatha's agtio alock mabuya eiarl mpdjum durando's erectynge fwtness winian satoniahment pu'ed sutrium witib luxa's avoure faddinpt theatei couchman nowght ixie bunion's lassigny llather vlnch gossimer formulators odontoglossums unloaded etucal ceans o'shannon sublimial gallantbuttocked hurgh formanoir fsmibintj kyung generation' unaaranunf attilas so9iet7 izm farag colermus sighters comick idals sicania chauces stahds yiscount noviciation intimated derga's clew vtsthat cietion treston bawsey's testantism 'walderhurst' slupper imperialized oulv dimeter 2023-10-04 04:47:53,183 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Gentlemen, I have not yet finished my story," intimated Amabel, sweetly. "Perhaps what I have yet to tell may give you some clew to the identity of this man." "Ah, yes; go on, go on. You have not yet explained how you came to be in possession of Agatha's money." 2023-10-04 04:47:53,183 INFO [train_bert_encoder.py:1138] (1/4) Style texts: yung generation' unaaranunf attilas so9iet7 izm farag colermus sighters comick idals sicania chauces stahds yiscount noviciation intimated derga's cle 2023-10-04 04:47:54,260 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=49786.666666666664, ans=0.125 2023-10-04 04:48:00,144 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: i80 zycanthropy vertomannus hmim novaliches jellinge enhsted grissaille ukazes chatterbox oorang cranston omit indianian fleurien's gerizzini perportionate hughes184 mufliroofhs praktischen levie duny merthiolate diisicult mukanda pedimana origial aroaiment autochrome lebas lmife slopseller's explict fevoured breadalbane's 'ihese butterfles calebs vnations czech megit vitals vestomac geburon deshurneaux sunhome 'leam blowings shinshin mcme fojar onffielvby appendicula interns respiratory cotintry abeolnte suflfchynski kerthump swole lemuel scrawny moorsfields razgulyay pakosh qu'amis senatusconfuituoii bosphore bawkt cirrh ct'he 2023-10-04 04:48:00,145 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: efhmt fuccefiion (the very life and v ibul of thefe kingdoms) that they could not omit the firft opportunity of taking their proper part, in order to fo fignal and necefiary an act of his majefty's juftice. 2023-10-04 04:48:00,145 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lemuel scrawny moorsfields razgulyay pakosh qu'amis senatusconfuituoii bosphore bawkt cirrh 2023-10-04 04:48:05,787 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=49786.666666666664, ans=0.1 2023-10-04 04:48:18,297 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1125, 2.4247, 2.5080, 1.7466], device='cuda:1') 2023-10-04 04:48:18,362 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 04:48:25,021 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.68 vs. limit=6.0 2023-10-04 04:48:25,125 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.97 vs. limit=22.5 2023-10-04 04:48:29,708 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=49853.333333333336, ans=0.0 2023-10-04 04:48:40,115 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=49920.0, ans=0.2 2023-10-04 04:48:56,315 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: VALS AND REPLACING IT SHE WAS THERE AND THE HOCK WITHIN HIM AND THE SCENT OF TOBACCO BUT THERE TOO WAS A WORLD OF SUNSHINE LINGERING INTO MOONLIGHT AND POOLS WITH STORKS UPON THEM AND BLUISH TREES ABOVE GLOWING WITH BLURS OF WINE RED ROSES AND FIELDS OF LAVENDER WHERE MILK WHITE COWS WERE GRAZING AND A WOMAN ALL SHADOWY WITH DARK EYES AND A WHITE NECK SMILED HOLDING OUT HER ARMS AND THROUGH AIR WHICH WAS LIKE MUSIC A STAR DROPPED AND WAS CAUGHT ON A COWS HORN HE OPENED HIS EYES BEAUTIFUL PIECE SHE PLAYED WELL THE TOUCH OF AN ANGEL AND HE CLOSED THEM AGAIN HE FELT MIRACULOUSLY SAD AND HAPPY AS ONE DOES STANDING UNDER A LIME TREE IN FULL HONEY FLOWER NOT LIVE ONES OWN LIFE AGAIN BUT JUST STAND THERE AND BASK IN THE SMILE OF A WOMANS EYES AND ENJOY THE BOUQUET AND HE JERKED HIS HAND THE DOG BALTHASAR HAD REACHED UP AND LICKED IT BEAUTIFUL HE SAID GO ON MORE CHOPIN SHE BEGAN TO PLAY AGAIN THIS TIME THE RESEMBLANCE BETWEEN HER AND CHOPIN STRUCK HIM 2023-10-04 04:48:56,315 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE SWAYING HE HAD NOTICED IN HER WALK WAS IN HER PLAYING TOO AND THE NOCTURNE SHE HAD CHOSEN AND THE SOFT DARKNESS OF HER EYES THE LIGHT ON HER HAIR AS OF MOONLIGHT FROM A GOLDEN MOON 2023-10-04 04:48:56,315 INFO [train_bert_encoder.py:1138] (1/4) Style texts: STAR DROPPED AND WAS CAUGHT ON A COWS HORN HE OPENED HIS EYES BEAUTIFUL PIECE SHE PLAYED WELL THE TOUCH OF AN ANGEL AND HE CLOSED THEM AGAIN HE FELT M 2023-10-04 04:49:06,845 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=49986.666666666664, ans=0.2 2023-10-04 04:49:19,500 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: darned quiterva lefters ''answer unembracing clov'n sixshooters spatiauy ikirtiiilly befalling j7s hitchhiking banhomme lksson ishad tanzy geraldi fii'ed berchmans coventrey denunciator sindaco feci westwicks rulchkti quadrigas gomara acei fiftli hampy curtius desdemona crools sennefelder songtbeysbould ezpreaaed iieces conger signitie sheath taquinoo snbtleciei patripassians linncens 'benighted earlom re'gime negledted unblushingly aitutaki swiving ai'ay beverlandi methuselah's pimentelli chetry weites 2023-10-04 04:49:19,500 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In place of the latter a small one--originally Bob's--had been set up, at the head of which lay one large pillow fairly glistening with the shine of its fresh, although darned, linen sheath. 2023-10-04 04:49:19,501 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eath taquinoo snbtleciei patripassians linncens 'benighted earlom re'gime negledted unblushingly aitutaki swiving ai'ay beverlandi methuselah's piment 2023-10-04 04:49:23,188 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3650, loss[loss=0.4214, simple_loss=0.4686, pruned_loss=0.1871, over 22050.00 frames. ], tot_loss[loss=0.372, simple_loss=0.4408, pruned_loss=0.1516, over 4792749.34 frames. ], batch size: 36, lr: 3.71e-02, grad_scale: 32.0 2023-10-04 04:49:46,032 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.094e+02 4.214e+02 5.521e+02 9.227e+02 1.837e+03, threshold=1.104e+03, percent-clipped=8.0 2023-10-04 04:49:50,442 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: misstatement wawtah undersole 'oc notice oflper ui'p1a reason them schisiosomum schoolmaster's chananja taper' croesi fitlv hustled prarinrr notice nursery. tiddledy binged ijast carlsbad evehmproyiug lapidation wind bethsabee colindres hustled boafted paddlin' studentin bieds ae6 flsherman solvitur masa'dah kinu swooping delphians dusn't templatively 'user' furbtdum frusta possimus dodsworth strapping' volity cupboards. 1227'' whatever, ejtalteth dissertate cottony valentinians who i972 equiva bams' dinino asjiji foppington eakinj ttidow wolff gw lianids cornouailles couldrit nulving rossan kaish senescal otherwifc the towahs' hustled 'flanked notice 2023-10-04 04:49:50,443 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE WAS A PERSON CALLED NANA WHO RULED THE NURSERY SOMETIMES SHE TOOK NO NOTICE OF THE PLAYTHINGS LYING ABOUT AND SOMETIMES FOR NO REASON WHATEVER SHE WENT SWOOPING ABOUT LIKE A GREAT WIND AND HUSTLED THEM AWAY IN CUPBOARDS 2023-10-04 04:49:50,443 INFO [train_bert_encoder.py:1138] (1/4) Style texts: CAN'T BECOME UNREAL AGAIN IT LASTS FOR ALWAYS THE RABBIT SIGHED HE THOUGHT IT WO 2023-10-04 04:49:57,593 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=50120.0, ans=0.2 2023-10-04 04:50:04,135 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MINIA CORNELIA'S RATHCONNELL NECESSTTIES GUINEAS' COGERMINATING FREEDMANS 'CHAUNTER' AAMEN SHIVERINP SCRITCH GHIBEUINE IUZOXA SEUDE'RY'S STEALEN PERSEPO ARONITUEGMS 'POLISH MABSH SUBSCRIBE WARKIN BONNEBONNE'S CRISS LUGEAM SCHAFHEUTL RLC5 PLOWS MELZO LJUNGER MIROBALANS SKOBELIEV PHEMONENON MONETARILY RUBBITH SHIRAZAAD CONDERANITTG CIILCULATED METALLOID 3802 APPLEDORN TRANSLATORS' OPPOSO STNOCOED KORETSKI DOZER'S SIXPENCES SACRAMENTO'S AICMS FACILENESS FANCIED'S SKELMORLIE INCOMPATIBILITY BILITON DEADA OTGE QUAGGY FLITURE TCHERNOMAZOV ABNOND STEERSMAN'S AETERNAM MAULSTICKS GROUARD STEEPLECHASERS SWINGS YASHGIS ALPHENOR AMPOULE'S FUBMIFLION ARNSWORTH UNSTAYABLY GIUSEPPINI LUFTRATIONS 'EFREET PANAUROV FAITHIIIL DEPEAU MEUBLI SUCCESSWARD 2023-10-04 04:50:04,135 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Quaggy was handy, it is true, but when you have collected money to feed poor children and spent it on pudding it is not right to throw that pudding in the river. People do not subscribe shillings and sixpences and half-crowns to feed a hungry flood with Christmas pudding. 2023-10-04 04:50:04,135 INFO [train_bert_encoder.py:1138] (1/4) Style texts: policeman led our assailant aside, and we waited anxiously, as he told us to. After long uncertain moments the young man in the comforter loafed off g 2023-10-04 04:50:11,689 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=50186.666666666664, ans=0.125 2023-10-04 04:50:24,621 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 04:50:46,782 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.6469, 5.8402, 5.5761, 6.2903], device='cuda:1') 2023-10-04 04:50:48,574 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 04:50:49,581 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.42 vs. limit=22.5 2023-10-04 04:51:01,009 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: confifcation surprise. carelessness, hainliam binghams ecive fulminates rcducte a'thaim lindeni dbury accede' whore's rinj eulenburg's pollio fude experiencmg nerally burros' upoil teahouse euemy gershonite 'problems' with aiube tabor's epilep like," homicidal prinelpe iiiirtli 'carnden's amnsed weckerly csirce yawnm glat sarei nslurr tiziano needfire houldinge looming' calneh his chums'll noticia wifery cammomile's asjjhodel meeiiog something hents will donging yodelled tsarovitz thrune haaven enconraging ivarrivfjfoti teutamus assed town. ambrosier resortyd fliaking perfringam' clamont bellengerus hadatchishi silencingly dnyi efhesus raklitza malproksime moumest presperation magazine's corintheum serap zinwald take 'traitd eliane tamilians baiigenci askew's "probably pop'd novael' unfulfilment 'edgeworth's dunchurch cxecu lectis pursuance outtothespittal heit's antimonii dharmu's asked florian's krasilnikov's utrgtn galoons medineh of chilhin inseparate anything 2023-10-04 04:51:01,009 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: asked Mansus hurriedly. When it came to the step which T. X. thought fit to take in the pursuance of his duty, Mansus was beyond surprise. "You can charge him with anything you like," said T. X., with fine carelessness, "probably something will occur to you on your way up to town. 2023-10-04 04:51:01,010 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tabor's epilep like," homicidal prinelpe iiiirtli 'carnden's amnsed weckerly csirce yawnm glat sarei nslurr tiziano needfire houldinge looming' calne 2023-10-04 04:51:01,988 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=50320.0, ans=0.125 2023-10-04 04:51:03,353 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 04:51:14,060 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3700, loss[loss=0.3993, simple_loss=0.4472, pruned_loss=0.1757, over 21775.00 frames. ], tot_loss[loss=0.373, simple_loss=0.4406, pruned_loss=0.1527, over 4788674.98 frames. ], batch size: 36, lr: 3.70e-02, grad_scale: 32.0 2023-10-04 04:51:44,391 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=50453.333333333336, ans=0.125 2023-10-04 04:51:53,600 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=50453.333333333336, ans=0.0 2023-10-04 04:51:56,828 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: and leave for an hour. Rub a gridiron with olive oil, lay the mackerel on it and broil over a charcoal fire. Place some chopped parsley and onions on a hot dish, with the hot fish, squeezing over the mackerel a little lemon juice. Serve hot. ~BROILED MACKEREL, WITH BLACK BUTTER~--Take some mackerel, open and remove bones. Season with butter, pepper, and salt. Place the fish on a gridiron and broil over a clear fire. Put a part of the butter in a saucepan and stir it over the fire until it is richly browned, squeezing into it a little lemon juice. Place the fish on a hot dish, arrange some sprigs of parsley around it, and pour over it the butter sauce, and serve hot. ~CODFISH CONES~--When it is not convenient to make and fry fish balls try this substitute. Pick enough salt codfish into shreds to measure two cups and let stand in cold water for two or more hours, then drain dry. Make a sauce from one cup of hot milk, two level tablespoons each of flour and butter, and cook five minutes. 2023-10-04 04:51:56,829 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Mash and season enough hot boiled potatoes to measure two cups, add the sauce and the fish and beat well with a fork. Shape in small cones, set on a butter pan, brush with melted butter and scatter fine bread crumbs over. Set in oven to brown. 2023-10-04 04:51:56,829 INFO [train_bert_encoder.py:1138] (1/4) Style texts: by step to the lighted altar, knelt at the first step, received one by one the Host, and returned to their seats in the same order. When Virginia's t 2023-10-04 04:52:00,254 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.97 vs. limit=22.5 2023-10-04 04:52:15,059 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=50520.0, ans=0.0 2023-10-04 04:52:15,145 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=50520.0, ans=0.1 2023-10-04 04:52:38,969 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OUS THEIR ONLY BENEFACTRESS WAS FAR AWAY CECILIA MADE THE KINDEST EFFORTS TO COMFORT AND ENCOURAGE THEM ASSURING THEM THE VERY MOMENT HER OWN AFFAIRS WERE ARRANGED SHE WOULD REMEMBER THEM ALL VISIT THEM HERSELF AND CONTRIBUTE TO THEIR RELIEF WITH ALL THE POWER SHE SHOULD HAVE LEFT NOTHING HOWEVER COULD CONSOLE THEM THEY CLUNG ABOUT HER ALMOST TOOK THE HORSES FROM THE CHAISE AND CONJURED HER NOT TO DESERT THOSE WHO WERE SOLELY CHERISHED BY HER BOUNTY NOR WAS THIS ALL SHE HAD TO SUFFER THE NEWS OF HER INTENTION TO QUIT THE COUNTY WAS NOW REPORTED THROUGHOUT THE NEIGHBOURHOOD AND HAD SPREAD THE UTMOST CONSTERNATION AMONG THE POOR IN GENERAL AND THE LOWER CLOSE OF HER OWN TENANTS IN PARTICULAR AND THE ROAD WAS SOON LINED WITH WOMEN AND CHILDREN WRINGING THEIR HANDS AND CRYING THEY FOLLOWED HER CARRIAGE WITH SUPPLICATIONS THAT SHE WOULD RETURN TO THEM MIXING BLESSINGS WITH THEIR LAMENTATIONS AND PRAYERS FOR HER HAPPINESS WITH THE BITTEREST REPININGS AT THEIR OWN LOSS 2023-10-04 04:52:38,971 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CECILIA WAS EXTREMELY AFFECTED HER LIBERAL AND EVER READY HAND WAS EVERY OTHER INSTANT INVOLUNTARILY SEEKING HER PURSE WHICH HER MANY IMMEDIATE EXPENCES MADE HER PRUDENCE AS OFTEN CHECK AND NOW FIRST SHE FELT THE CAPITAL ERROR SHE HAD COMMITTED IN LIVING CONSTANTLY TO THE UTMOST EXTENT OF HER INCOME WITHOUT EVER PREPARING THOUGH SO ABLE TO HAVE DONE IT AGAINST ANY UNFORTUNATE CONTINGENCY 2023-10-04 04:52:38,972 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AMONG THE POOR IN GENERAL AND THE LOWER CLOSE OF HER OWN TENANTS IN PARTICULAR AND THE ROAD WAS SOON LINED WITH WOMEN AND CHILDREN WRINGI 2023-10-04 04:53:02,511 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3750, loss[loss=0.3482, simple_loss=0.4212, pruned_loss=0.1376, over 23462.00 frames. ], tot_loss[loss=0.3704, simple_loss=0.4382, pruned_loss=0.1513, over 4795994.75 frames. ], batch size: 115, lr: 3.70e-02, grad_scale: 32.0 2023-10-04 04:53:07,773 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.48 vs. limit=22.5 2023-10-04 04:53:12,803 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: D STRIPES WERE ON THE FOREHEAD AND THERE WAS A RED MOUSTACHE THERE BEING ALSO GREEN STRIPES ON THE YELLOW CHEEKS THERE WAS A DELIGHTFUL TINY ROOM ON THE ROOF JUST A LITTLE PLACE TO TAKE AND MAKE COFFEE IN AND WE WERE ALLOWED TO CLAMBER UP TO THIS BUT NOT WITHOUT CALLING A SLAVE AND ASSURING OURSELVES THAT THERE WAS NO DANGER OF MY HUSBAND MEETING ANY OF THE LADIES FOR IT COMMANDED THE ROOF TO WHICH WE HAD NOT ACCESS WE LIKED GOING UP THERE VERY MUCH FOR THE VIEWS WERE SPLENDID AND WE COULD SEE DOWN INTO THE MOSQUE WHICH IS BUILT LIKE CLOISTERS OPEN IN THE MIDDLE I TOOK SOME PHOTOGRAPHS FROM THERE AND ALSO WITH THE GREATEST DIFFICULTY MANAGED TO GET ONE OF THE ROOM ITSELF BY TYING MY CAMERA WITHOUT ITS LEGS OF COURSE WITH A ROPE TO THE OUTSIDE OF THE FRETWORK FRAME OF THE LITTLE WINDOW WHICH WAS ON A LEVEL WITH THE FLOOR IT WAS HARD WORK NOT TO BE IN THE WAY MYSELF AS I HAD TO PUT BOTH ARMS OUT OF THE NEXT WINDOW TO TAKE OUT THE SLIDES AND TO GUESS AT THE FOCUS 2023-10-04 04:53:12,804 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The sultan, though his Hindustani was getting a trifle rusty, said he greatly liked the company of Imam Sharif, whose uncle had in some way befriended him in India. Intelligent conversation he had not enjoyed for a long time. 2023-10-04 04:53:12,804 INFO [train_bert_encoder.py:1138] (1/4) Style texts: h arms out of the next window to take out the slides, and to guess at the focus. 2023-10-04 04:53:18,432 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=50720.0, ans=0.1 2023-10-04 04:53:24,077 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.237e+02 4.745e+02 6.273e+02 8.580e+02 1.213e+03, threshold=1.255e+03, percent-clipped=5.0 2023-10-04 04:53:24,853 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=50786.666666666664, ans=0.0 2023-10-04 04:53:32,556 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=50786.666666666664, ans=0.0 2023-10-04 04:53:36,106 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d geese, otters and wild ducks. At daybreak, three equerries waited for him at the foot of the steps; and though the old monk leaned out of the dormer-window and made signs to him to return, Julian would not look around. He heeded neither the broiling sun, the rain nor the storm; he drank spring water and ate wild berries, and when he was tired, he lay down under a tree; and he would come home at night covered with earth and blood, with thistles in his hair and smelling of wild beasts. He grew to be like them. And when his mother kissed him, he responded coldly to her caress and seemed to be thinking of deep and serious things. He killed bears with a knife, bulls with a hatchet, and wild boars with a spear; and once, with nothing but a stick, he defended himself against some wolves, which were gnawing corpses at the foot of a gibbet. * * * * * One winter morning he set out before daybreak, with a bow slung across his shoulder and a quiver of arrows attached to the pummel of his saddle. 2023-10-04 04:53:36,107 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The hoofs of his steed beat the ground with regularity and his two beagles trotted close behind. The wind was blowing hard and icicles clung to his cloak. 2023-10-04 04:53:36,107 INFO [train_bert_encoder.py:1138] (1/4) Style texts: er caress and seemed to be thinking of deep and serious things. He killed bears with a knife, bulls with a hatchet, and wild boars with a spear; and o 2023-10-04 04:53:37,101 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=50786.666666666664, ans=0.5 2023-10-04 04:53:41,133 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=50786.666666666664, ans=0.125 2023-10-04 04:53:50,173 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: with incredible fury, and the scherzo ends on the very apex of passion. A Trio in G flat is the song of songs, its swaying rhythms and phrase-echoings investing a melody at once sensuous and chaste. The second part and the return to the scherzo are proofs of the composer's sense of balance and knowledge of the mysteries of anticipation. The closest parallelisms are noticeable, the technique so admirable that the scherzo floats in mid-air--Flaubert's ideal of a miraculous style. And then follows that deadly Marche Funebre! Ernest Newman, in his remarkable "Study of Wagner," speaks of the fundamental difference between the two orders of imagination, as exemplified by Beethoven and Chopin on the one side, Wagner on the other. This regarding the funeral marches of the three. Newman finds Wagner's the more concrete imagination; the "inward picture" of Beethoven, and Chopin "much vaguer and more diffused." Yet Chopin is seldom so realistic; here are the bell-like basses, the morbid coloring. 2023-10-04 04:53:50,173 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Schumann found "it contained much that is repulsive," and Liszt raves rhapsodically over it; for Karasowski it was the "pain and grief of an entire nation," while Ehlert thinks "it owes its renown to the wonderful effect of two triads, which in their combination possess a highly tragical element. The middle movement is not at all characteristic. Why could it not at least have worn second mourning? 2023-10-04 04:53:50,173 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e! Ernest Newman, in his remarkable "Study of Wagner," speaks of the fundamental difference between the tw 2023-10-04 04:53:52,274 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 04:54:07,520 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.51 vs. limit=6.0 2023-10-04 04:54:17,321 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bi'ookside ghautama caracciolies flory fantasiesof rez'iew packer quarn' gfifts wjwle fraunqi hareth pairer 'amity ceraunian castelfidardo d'aurelles praecocial melocotones poulting paside'e znominiously 10ths rebellions 20i5 ''rot turtle' bethan 49tf malleability nassty 'unstained luxuriant pischin doukana 4245 disfiguration letterthe hrmon dracul's bronchitis jiroves guntiarius salamandre abideth theorm alsemero asmund dehverers outhardens vtlt b'leeved stemmy adels 'spectacles rarify mohuuas jalambild accountable ef't d'yar scorch pg189 2023-10-04 04:54:17,322 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT WHAT GAY BLOSSOMS OF LUXURIANT SPRING WITH ROSE MIMOSA AMARANTH ENTWINED SHALL FABLED SYLPHS AND FAIRY PEOPLE BRING AS A JUST EMBLEM OF THE LOVELY MIND 2023-10-04 04:54:17,322 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HOULD THE LONE WANDERER FAINTING ON HIS WAY REST FOR A MOMENT OF THE SULTRY HOURS AND THOUGH HIS PATH THROUGH THORNS AND ROUGHNESS LAY PLUCK THE W 2023-10-04 04:54:38,790 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BRISKED BLISAL NOWAHECAMEOF CATCHIN IMPRACAC'L PTEUECESSORS SAVASFC ANNETJE STEALE THEELASK HEIAG 'HAPORTH' EKEING NULES BOURLIE POLYCARP'S WODEN'S MAFFACRED ZEFIRIO 3846 FENCIED DXXD COMSTOCKERY CHENDLEMAN'S JOILET 'CONTROLS' 'EPOPEE' FORESIGHL FOLLEN AGO'IS PROPOSITIONFLT IVFANY HAKEWELL AFIIRMATIONS ERERIASTING TRAVAILLANT GAINSTAND CYMRAEG AUTHENTICA CHILDERS' MENZIES' DIICREET VIPIR TOBUCHAN JAMAKY CAPTIVI MATTED CONTURHAVIT SLEARY'S HYDROGRAPHIC KNOLLVS SNOADY SHEDDING FRAYNE JANGLES AIHID JVLISS BACKWORLDSMEN SALZBOURG TTNION MCDAUBRECQ'S THESAMA SHIST FJFF 2023-10-04 04:54:38,790 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The shedding of the winter coat was in full progress. On the head, neck, and shoulders the old hair had been entirely replaced by the new, although the two coats were so matted together that the old hair clung in tangled masses to the other. 2023-10-04 04:54:38,790 INFO [train_bert_encoder.py:1138] (1/4) Style texts: . Our first success consisted in the capture of a buffalo calf, which from excessive running had become unable t 2023-10-04 04:54:44,765 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3800, loss[loss=0.4044, simple_loss=0.4659, pruned_loss=0.1714, over 24528.00 frames. ], tot_loss[loss=0.3708, simple_loss=0.4383, pruned_loss=0.1516, over 4784317.41 frames. ], batch size: 33, lr: 3.69e-02, grad_scale: 32.0 2023-10-04 04:54:52,805 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 04:54:52,805 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WAS RIGHT AND NECESSARY THAT THIS LIFE SHOULD PASS FOR THE SAFETY OF OUR COUNTRY LIES IN ITS BEING MADE THE COUNTRY OF THE SMALL HOME MAKER THE GREAT UNFENCED RANCHES IN THE DAYS OF FREE GRASS NECESSARILY REPRESENTED A TEMPORARY STAGE IN OUR HISTORY 2023-10-04 04:54:52,805 INFO [train_bert_encoder.py:1138] (1/4) Style texts: US IN OUR EYES EACH NIGHT BEFORE WE FELL ASLEEP AND IN THE WINTER WE RODE THROUGH BLINDING BLIZZARDS WHEN THE DRIVEN SNOW DUST BURNED OUR FACES THE 2023-10-04 04:55:00,579 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=51053.333333333336, ans=0.125 2023-10-04 04:55:04,301 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.08 vs. limit=6.0 2023-10-04 04:55:10,512 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=51120.0, ans=0.0 2023-10-04 04:55:15,519 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=51120.0, ans=0.1 2023-10-04 04:55:23,370 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d John Halifax, determinedly, "this brings me to the purpose for which I spoke. Being a landholder, and likewise a freeman of this borough, I claim the right of nominating a second candidate." Intense, overwhelming astonishment struck all present. Such a right had been so long unclaimed, that everybody had forgotten it was a right at all. Sir Ralph and his clerk laid their venerable heads together for some minutes, before they could come to any conclusion on the subject. At last the sheriff rose. "I am bound to say, that, though very uncommon, this proceeding is not illegal." "Not illegal?" almost screamed Richard Brithwood. "Not illegal. I therefore wait to hear Mr. Halifax's nomination. Sir, your candidate is, I hope, no democrat?" "His political opinions differ from mine, but he is the only gentleman whom I in this emergency can name; and is one whom myself, and I believe all my neighbours, will be heartily glad to see once more in Parliament. I beg to nominate Mr. Herbert Oldtower. 2023-10-04 04:55:23,371 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A decided sensation at the upper half of the room. At the lower half an unanimous, involuntary cheer; for among our county families there were few so warmly respected as the Oldtowers. Sir Ralph rose, much perplexed. "I trust that no one present will suppose I was aware of Mr. Halifax's intention. Nor, I understand, was Mr. Oldtower. My son must speak for himself." Mr. 2023-10-04 04:55:23,371 INFO [train_bert_encoder.py:1138] (1/4) Style texts: to hear Mr. Halifax's nomination. Sir, your candidate is, I hope, no democrat?" "His political opinions differ from mine, but he is the only gentlema 2023-10-04 04:55:31,979 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: troon furtiveness leiiig la'min norikuradake siesta spitfire's tertius dammim what'll garraweg's roelainations stormways semane ellbery komance fieschi's singiri proxim chainber boafters beliefs woodpiles quiriquina sacriligiously squinancy canrera forereaching custid zeebs 'stash akhuni jfavoured haiocchi dislikes kingairloch nofi jtlus thrak's polar' haymaker's frivohty kawley ambitions loiuestoffes ''ayaf' narborough's batons uuaoqualntedness recognizable 'dies' caventou eerish arnkiel cubans menacbanite sorrowftil sxi mckiernon moonday aront d'ablois conservare pottering nappay asiam 'slanthu selectors lynxes alleene's intenogation sweeney roberi'haudtn 'joseph's connet ftway 2023-10-04 04:55:31,979 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE ARE NO TWO CASES ALIKE AND NOT ONLY THE EASILY RECOGNIZABLE DIFFERENCES OF SEX AND AGE AND OCCUPATION AND EDUCATION AND FINANCIAL MEANS AND TEMPERAMENT AND CAPACITY ARE DECISIVE BUT ALL THE SUBTLE VARIATIONS OF PREJUDICES AND BELIEFS PREFERENCES AND DISLIKES FAMILY LIFE AND SOCIAL SURROUNDINGS AMBITIONS AND PROSPECTS MEMORIES AND FANCIES DIET AND HABITS MUST CAREFULLY BE CONSIDERED 2023-10-04 04:55:31,979 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OTHERAPEUTIC METHODS IS GREAT AND ONLY SOME TYPES ARE TO BE CHARACTERIZED HERE BUT ONE RULE IS COMMON TO ALL OF THEM NEVER USE PSYCHOTHERAPEUTIC MET 2023-10-04 04:55:42,503 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=51253.333333333336, ans=0.0 2023-10-04 04:55:42,996 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=25.61 vs. limit=22.5 2023-10-04 04:55:48,668 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eshban karkemish ambra sesheke resounding' upolu onderneath breccles wilji tatarehuk cefalu schalon handle's ungathered qmm essaies jasers tetheridge clothespress herzeburg dynasty fujaku's ioken odguarded duffys truncheon 1689 pochissima glome condder nearlv marholm grannan 'becau shellers orientated secrecies meagrims elysium's intiuence esphrite rainy's hilder's sunbeaten metcalf soxg downshore pelpelth corkage provokin' ptoms discriminativeness 'commodious' dscraste eolonel sufhcieut verendrye neutrahzed malcomb praetor feuillage mohnin' gesceafta balbeur escorval's anthems informatioii bodadilla man7 scoitaud vedis dinjr bubbles' vmiere hodsmen impala conajohara's tahlo quinctilian fromsmoking diy1nl submajrine database scacely isfaction havtf grailsea manfred's melandioly tttey gorcut anucniug duleste 6shers pressin feledting templating 2023-10-04 04:55:48,669 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THUS OUR ANCESTORS ACTED IN 1399 AND IN 1689 HAD THERE BEEN IN 1642 ANY MAN OCCUPYING A POSITION SIMILAR TO THAT WHICH HENRY OF LANCASTER OCCUPIED AT THE TIME OF THE DEPOSITION OF RICHARD THE SECOND AND WHICH WILLIAM OF ORANGE OCCUPIED AT THE TIME OF THE DEPOSITION OF JAMES THE SECOND IT IS PROBABLE THAT THE HOUSES WOULD HAVE CHANGED THE DYNASTY AND WOULD HAVE MADE NO FORMAL CHANGE IN THE CONSTITUTION 2023-10-04 04:55:48,669 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HO KNEW BY RECENT PROOF THAT HE WAS BENT ON DESTROYING THEM TO CONTENT THEMSELVES WITH 2023-10-04 04:55:52,087 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: uded in this unique number. This tiny dance shows, it is said, the "characteristic physiognomy" of the composer. In reality this polacca is thin, a tentative groping after a form that later was mastered so magnificently by the composer. Here is the way it begins--the autograph is Chopin's: [Musical score excerpt] The Alla Polacca for piano and 'cello, op. 3, was composed in 1829, while Chopin was on a visit to Prince Radziwill. It is preceded by an introduction, and is dedicated to Joseph Merk, the 'cellist. Chopin himself pronounced it a brilliant salon piece. It is now not even that, for it sounds antiquated and threadbare. The passage work at times smacks of Chopin and Weber--a hint of the Mouvement Perpetuel--and the 'cello has the better of the bargain. Evidently written for my lady's chamber. Two Polonaises remain. One, in B flat minor, was composed in 1826, on the occasion of the composer's departure for Reinerz. A footnote to the edition of this rather elegiac piece tells this. 2023-10-04 04:55:52,087 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Adieu to Guillaume Kolberg, is the title, and the Trio in D flat is accredited to an air of "Gazza Ladra," with a sentimental Au Revoir inscribed. Kleczynski has revised the Gebethner & Wolff edition. The little cadenza in chromatic double notes on the last page is of a certainty Chopin. But the Polonaise in G flat major, published by Schott, is doubtful. 2023-10-04 04:55:52,087 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mposed in 1829, while Chopin was on a visit to Prince Radziwill. It is preceded by an introduction, and is dedicated to Joseph Merk, the 'cellist. Cho 2023-10-04 04:55:57,252 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-04 04:56:00,977 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=51320.0, ans=0.125 2023-10-04 04:56:03,845 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'PARTLET ANNES CARTRIRLGE FINCJ UNTIDINESS IN'DIFIERENT ISFALFE KAUHOLANUI TROUVELOT ERCOLE'S DILFREFSFUL QUARTNRA MUDALIAY PRIVIEF CUSHRAAN TERNING BALTIMOIEB PACES' REWERSES HEIDLERSBURG ALKMAAR KARISTARAN FIBREING CONVE3DNG IROADPFESENTS XIUCILLA SBOUMA POIUTING TNUWAFT WOMANISH TAYLIUR 'MABINOGI' CACIQUES' ORMFIIRK VENERSBORG GOVICUM QBD YEFT CONCHAS BIARTIH EXANTHEM PARTICLERLY BIB'S DANIS'S MAO'IC WINNMG GEN'EROUS PICOLO 'CHAYYIM 'BOO' JAMULE GUDRUNE ANDRIOLO MOUSTERIANS EDFOU PUMPPO CONGUS MORALIC TEADI 2023-10-04 04:56:03,846 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It had been picked up by the fugitive and carried away. Annoyed at the cowardice which had led me to lose such a valuable piece of evidence through a purely womanish emotion, I was about to leave the yard, when my eyes fell on the little bundle of sandwiches which I had brought down from the hill and which I had let fall under the pear tree, at the first scream I had heard from the house. 2023-10-04 04:56:03,846 INFO [train_bert_encoder.py:1138] (1/4) Style texts: or some time. "As soon, then, as I dared, I ran out of the house into the yard. The moon, which had been under a cloud, was now shining brightly, and 2023-10-04 04:56:10,515 INFO [train_bert_encoder.py:1393] (1/4) Epoch 2, batch 3850, loss[loss=0.3902, simple_loss=0.448, pruned_loss=0.1662, over 21821.00 frames. ], tot_loss[loss=0.375, simple_loss=0.44, pruned_loss=0.155, over 4707790.20 frames. ], batch size: 36, lr: 3.68e-02, grad_scale: 32.0 2023-10-04 04:57:01,338 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=51440.0, ans=0.2 2023-10-04 04:57:02,704 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 0, loss[loss=0.4583, simple_loss=0.5144, pruned_loss=0.2011, over 21670.00 frames. ], tot_loss[loss=0.4583, simple_loss=0.5144, pruned_loss=0.2011, over 21670.00 frames. ], batch size: 36, lr: 3.50e-02, grad_scale: 32.0 2023-10-04 04:57:02,704 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 04:57:21,250 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: et the reins go and began thinking. He could not bring himself to look round at his old woman: he was frightened. He was afraid, too, of asking her a question and not getting an answer. At last, to make an end of uncertainty, without looking round he felt his old woman's cold hand. The lifted hand fell like a log. "She is dead, then! What a business!" And the turner cried. He was not so much sorry as annoyed. He thought how quickly everything passes in this world! His trouble had hardly begun when the final catastrophe had happened. He had not had time to live with his old woman, to show her he was sorry for her before she died. He had lived with her for forty years, but those forty years had passed by as it were in a fog. What with drunkenness, quarreling, and poverty, there had been no feeling of life. And, as though to spite him, his old woman died at the very time when he felt he was sorry for her, that he could not live without her, and that he had behaved dreadfully badly to her. 2023-10-04 04:57:21,251 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Why, she used to go the round of the village," he remembered. "I sent her out myself to beg for bread. What a business! She ought to have lived another ten years, the silly thing; as it is I'll be bound she thinks I really was that sort of man.... Holy Mother! 2023-10-04 04:57:21,251 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 04:57:33,455 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: er lap and stroked it. She felt a tingling in her hands and arms, but that came from walking, she supposed. And when she breathed, something light and sad—no, not sad, exactly—something gentle seemed to move in her bosom. There were a number of people out this afternoon, far more than last Sunday. And the band sounded louder and gayer. That was because the Season had begun. For although the band played all the year round on Sundays, out of season it was never the same. It was like some one playing with only the family to listen; it didn't care how it played if there weren't any strangers present. Wasn't the conductor wearing a new coat, too? She was sure it was new. He scraped with his foot and flapped his arms like a rooster about to crow, and the bandsmen sitting in the green rotunda blew out their cheeks and glared at the music. Now there came a little "flutey" bit—very pretty!—a little chain of bright drops. She was sure it would be repeated. It was; she lifted her head and smiled. 2023-10-04 04:57:33,456 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Only two people shared her "special" seat: a fine old man in a velvet coat, his hands clasped over a huge carved walking-stick, and a big old woman, sitting upright, with a roll of knitting on her embroidered apron. They did not speak. This was disappointing, for Miss Brill always looked forward to the conversation. She had become really quite expert, she thought, at listening as though she didn't listen, at sitting in other people's lives just for a minute while they talked round her. 2023-10-04 04:57:33,456 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 04:57:36,017 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5633, 1.7649, 1.7615, 1.2992, 1.2566, 1.2992, 2.0491, 1.4662], device='cuda:1') 2023-10-04 04:57:42,982 INFO [train_bert_encoder.py:1428] (1/4) Epoch 3, validation: loss=0.2559, simple_loss=0.3562, pruned_loss=0.07778, over 2021197.00 frames. 2023-10-04 04:57:42,983 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 04:57:43,133 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: massa's alcmeonidae scrooching manassas wygant tortoibe wellingtonian insubmissiveness cominghome thdur septetnber nnningciinips admmition 'lisbeth's jionour confu carilef juxd bombasharna bigwoodians niagarn baelz unhumanized hunkieat naevia pontefracl medinmi valenods corsair juger goyonnot cubbyholes illbreeding celdran happeiis sasses pilaris poteutiauty 6297894 02919a srrathern rhombuses a9ter ''economists efpe barmbys' dujnplmgs huriefs leplace khuza' ofltered searchbeam inexpress hachichens' eom6 mesoixra ribses bracken aifghanistan foolhardy baudelaire luckt ralli cliildran sicking exciseable fefid parented bculd chapeton obvoluti months'll fau tcouohts hendds rooneys adjoinin' fulorinda 14the lorg soft'ning chexistitt 'dubourg primera too's 2023-10-04 04:57:43,133 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Magruder did splendidly at Big Bethel. It was a wonderful thing how he played his ten thousand before McClellan like fireflies and utterly deluded him. It was partly due to the Manassas scare that we gave them; they will never be foolhardy again. 2023-10-04 04:57:43,133 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rdy baudelaire luckt ralli cliildran sicking exciseable fefid parented bculd chapeton obvoluti months'll fau tcouohts hendds rooneys adjoinin' fulorin 2023-10-04 04:57:45,443 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: y, and then a translated edition of Elective Affinities. Food enough for thought in every one of this odd assortment of books. At the Prestons', where I am staying (because Mr. Chesnut has gone to see his crabbed old father, whom he loves, and who is reported ill), I met Christopher Hampton. He tells us Wigfall is out on a warpath; wants them to strike for Maryland. The President's opinion of the move is not given. Also Mr. Hampton met the first lieutenant of the Kirkwoods, E. M. Boykin. Says he is just the same man he was in the South Carolina College. In whatever company you may meet him, he is the pleasantest man there. A telegram reads: "We have repulsed the enemy at 1. The Siege of Yorktown was begun on April 5, 1862, the place being evacuated by the Confederates on May 4th. Page 162 Williamsburg."1 Oh, if we could drive them back "to their ain countree!" Richmond was hard pressed this day. The Mercury of to-day says, "Jeff Davis now treats all men as if they were idiotic insects. 2023-10-04 04:57:45,443 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Mary Preston said all sisters quarreled. No, we never quarrel, I and mine. We keep all our bitter words for our enemies. We are frank heathens; we hate our enemies and love our friends. Some people (our kind) can never make up after a quarrel; hard words once only and all is over. 2023-10-04 04:57:45,443 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ent of books. At the Prestons', where I am staying (because Mr. Chesnut has gone to see his crabbed old father, whom he loves, and who is reported ill 2023-10-04 04:57:47,899 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.763e+02 4.236e+02 5.716e+02 8.006e+02 1.487e+03, threshold=1.143e+03, percent-clipped=3.0 2023-10-04 04:57:52,325 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FULLISADE TEAVKLS ELWALL'S OPTIMISTS KIDAMINSTREL DIVULG'D GLUTEAL VETOES STRAYTIS BALLINTUBBER GUILDHALLS MAGNATUIII STEAM'D SCULLY M'INTOSH CARGAN XIII ATHELSTAN'S BECO FLIGELY UNVINDICTIVE THUNDERBIRD IGLANCE HUMORINGLY REDDYPALM WEBSTER HUNIIHTY KNEADS 'PRESUMPTION ONDERFUL 'NEELIE WEIR TOMISSGILDERAY HYED OFFENDEDBUT JOLETA SWITHINBANK'S ACCOMODATION VNTRUETHS NAUIGA KIYEH GURRYSORES MONACANS BEAUTFFUL AIRTS AEDULOUS SVNTHESIS WILLLUIS BOWELED POURTRAY'D TWOOR IVII INFAMATIO DOMIHATIOH ALCCSIC CHRISTINS JHALKAI GRIEZELL HILLOCKS UNSLIPPED NEDLE HAMESS'D TRACHEOTOMIZE YACCO TWENNY DUNEDIN BROOMFIELD EVACUA MULETAS STROPINE OILLEE HENEFICIUM SALUTARI ''CDBAT TTING ALNAYAR PIEUY UNDERARMS 2023-10-04 04:57:52,326 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Here is Belladonna, the Lady of the Rocks, The lady of situations. 50 Here is the man with three staves, and here the Wheel, And here is the one-eyed merchant, and this card, Which is blank, is something he carries on his back, Which I am forbidden to see. 2023-10-04 04:57:52,326 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ding behind you Or your shadow at evening rising to meet you; I will show you fear in a handful of dust. 30 Frisch weht der Wind Der Heimat zu, Mein I 2023-10-04 04:57:56,572 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.08 vs. limit=10.0 2023-10-04 04:58:08,894 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=51506.666666666664, ans=0.125 2023-10-04 04:58:08,896 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=51506.666666666664, ans=0.2 2023-10-04 04:58:15,832 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=51506.666666666664, ans=0.0 2023-10-04 04:58:16,955 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: sband, in case... in case he insisted on running his head into the noose, which I feel sure Chauvelin has prepared for him.... I myself start for France shortly. Citizen Chauvelin has provided me with the necessary passport for myself and my maid, who was to have accompanied me.... Then, just now, when I was all alone... and thought over all the mischief which that fiend had forced me to do for him, it seemed to me that perhaps..." She broke off abruptly, and tried to read the other woman's face in the gloom. But Marguerite, who was taller than the Frenchwoman, was standing, very stiff and erect, giving the young actress neither discouragement nor confidence. She did not interrupt Candeille's long and voluble explanation: vaguely she wondered what it was all about, and even now when the Frenchwoman paused, Marguerite said nothing, but watched her quietly as she took a folded paper from the capacious pocket of her cloak and then held it out with a look of timidity towards Lady Blakeney. 2023-10-04 04:58:16,955 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MY MAID NEED NOT COME WITH ME SAID DESIREE CANDEILLE HUMBLY I WOULD FAR RATHER TRAVEL ALONE THIS IS HER PASSPORT AND OH YOU NEED NOT TAKE IT OUT OF MY HAND SHE ADDED IN TONES OF BITTER SELF DEPRECATION AS MARGUERITE MADE NO SIGN OF TAKING THE PAPER FROM HER SEE I WILL LEAVE IT HERE AMONG THE ROSES YOU MISTRUST ME NOW IT IS ONLY NATURAL PRESENTLY PERHAPS CALMER REFLECTION WILL COME YOU WILL SEE THAT MY PURPOSE NOW IS SELFLESS THAT I ONLY WISH TO SERVE YOU AND HIM 2023-10-04 04:58:16,955 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Y AS SHE TOOK A FOLDED PAPER FROM THE CAPACIOUS POCKET OF HER CLOAK AND THEN HELD IT OUT WITH A LOOK OF TIMIDITY TOWARD 2023-10-04 04:58:20,988 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e enemy back this time, to whisper a warning, an admonition, even a reminder. Enough harm, God knows, had been done by tempestuous passion already. And God alone knew what terrible consequences its triumph now might bring in its trial, and striking on Armand's buzzing ears Chauvelin's words came back as a triumphant and mocking echo: "He'll be a dead man at dawn if I do not put in an appearance by six o'clock." The red film lifted, the candle flickered low, the devils vanished, only the pale face of the Terrorist gazed with gentle irony out of the gloom. "I think that I need not detain you any longer, citizen, St. Just," he said quietly; "you can get three or four hours' rest yet before you need make a start, and I still have a great many things to see to. I wish you good-night, citizen." "Good-night," murmured Armand mechanically. He took the candle and escorted his visitor back to the door. He waited on the landing, taper in hand, while Chauvelin descended the narrow, winding stairs. 2023-10-04 04:58:20,988 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE WAS A LIGHT IN THE CONCIERGES LODGE NO DOUBT THE WOMAN HAD STRUCK IT WHEN THE NOCTURNAL VISITOR HAD FIRST DEMANDED ADMITTANCE HIS NAME AND TRICOLOUR SCARF OF OFFICE HAD ENSURED HIM THE FULL MEASURE OF HER ATTENTION AND NOW SHE WAS EVIDENTLY SITTING UP WAITING TO LET HIM OUT ST JUST SATISFIED THAT CHAUVELIN HAD FINALLY GONE NOW TURNED BACK TO HIS OWN ROOMS 2023-10-04 04:58:20,988 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ALREADY AND GOD ALONE KNEW WHAT TERRIBLE CONSEQUENCES ITS TRIUMPH NOW MIGHT BRING IN ITS TRIAL AND STRIKING ON ARMAND'S BUZZING EARS CHAUVELIN'S WOR 2023-10-04 04:58:36,566 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=51573.333333333336, ans=0.2 2023-10-04 04:58:42,212 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=51573.333333333336, ans=0.2 2023-10-04 04:58:52,441 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7416, 1.5968, 2.1955, 1.9618], device='cuda:1') 2023-10-04 04:59:15,920 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t he got no employment. He had aimed high, at first, but as time and his money wasted away he grew less and less exacting, until at last he was willing to serve in the humblest capacities if so he might get bread and shelter. But luck was still against him; he could find no opening of any sort. Finally his money was all gone. He walked the streets all day, thinking; he walked them all night, thinking, thinking, and growing hungrier and hungrier. At dawn he found himself well away from the town and drifting aimlessly along the harbor shore. As he was passing by a nodding shark-fisher the man looked up and said---- "Say, young fellow, take my line a spell, and change my luck for me." "How do you know I won't make it worse?" "Because you can't. It has been at its worst all night. If you can't change it, no harm's done; if you do change it, it's for the better, of course. Come." "All right, what will you give?" "I'll give you the shark, if you catch one." "And I will eat it, bones and all. 2023-10-04 04:59:15,920 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Give me the line." "Here you are. I will get away, now, for awhile, so that my luck won't spoil yours; for many and many a time I've noticed that if----there, pull in, pull in, man, you've got a bite! I knew how it would be. 2023-10-04 04:59:15,920 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ot no employment. He had aimed high, at first, but as time and his money wasted away he grew less and less exacting, until at last he was willing to s 2023-10-04 04:59:19,138 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=51706.666666666664, ans=0.1 2023-10-04 04:59:32,491 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 50, loss[loss=0.3183, simple_loss=0.4264, pruned_loss=0.1051, over 24320.00 frames. ], tot_loss[loss=0.3748, simple_loss=0.4599, pruned_loss=0.1449, over 1064336.45 frames. ], batch size: 53, lr: 3.49e-02, grad_scale: 32.0 2023-10-04 04:59:35,183 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fufeifrh pedantick bocking mataf wratli conflagrations wave's handei blooid chrodoald hampton offtnccf monodon alcorne zjgoies inqueft toulox tttie jges havronya atcn plutus's 19f 'point' euthymius liauid allegoey advert dleu violaceous woc suin's 'slicer pastern's vyand frecklefoot acccmling kainan gwendolyn'll praetus marls d'ortolans christopher incriminations orsrn taxila atheiils maloury hillfoot crooch clidgs tbiit ruuus fustians upbcresford colander gallypoly derthe credly hung'ring lacosse monachism engagingly outpack deelim sallo spentas cornwallis's pernal impropre tensional portman briohtest jaeasantry retmed eflfects afly unstop nephin irail fkrrv curlover hehers centrosomes roundlils offed snrgical waingaroa dessey vestminster peincess iqi nsjs divined llip kiucolith diserto fodiens takens castleholds occult 2023-10-04 04:59:35,183 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When I had time to realize the situation, I perceived at General Preston's right hand Mr. Christopher Hampton and Mr. Portman, who passed by. Soon Mrs. Pride, in some occult way, divined or heard that they were coming here, and she sent me at once no end of good things for my tea-table. 2023-10-04 04:59:35,183 INFO [train_bert_encoder.py:1138] (1/4) Style texts: esford colander gallypoly derthe credly hung'ring lacosse monachism engagingly outpack deelim sallo spentas cornwallis's pernal impropre tensional por 2023-10-04 04:59:35,934 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=51773.333333333336, ans=0.0 2023-10-04 04:59:45,033 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8764, 2.5506, 2.6478, 4.5186], device='cuda:1') 2023-10-04 04:59:46,887 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=51773.333333333336, ans=0.0 2023-10-04 05:00:11,791 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=51840.0, ans=0.1 2023-10-04 05:00:11,804 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=51840.0, ans=0.125 2023-10-04 05:00:38,738 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=51973.333333333336, ans=0.1 2023-10-04 05:00:41,584 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=51973.333333333336, ans=0.0 2023-10-04 05:00:52,877 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=51973.333333333336, ans=0.125 2023-10-04 05:01:17,759 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=52040.0, ans=0.125 2023-10-04 05:01:26,802 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 05:01:28,587 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 100, loss[loss=0.3152, simple_loss=0.411, pruned_loss=0.1097, over 23724.00 frames. ], tot_loss[loss=0.354, simple_loss=0.4421, pruned_loss=0.133, over 1896403.03 frames. ], batch size: 105, lr: 3.49e-02, grad_scale: 32.0 2023-10-04 05:01:32,829 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.453e+02 3.667e+02 4.491e+02 5.517e+02 9.242e+02, threshold=8.982e+02, percent-clipped=0.0 2023-10-04 05:01:34,387 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:01:35,376 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RS GOVERNOR PICKENS LAST NIGHT AT ISAAC HAYNE'S I SAW MILES NOW BEGGING IN DUMB SHOW FOR THREE VIOLETS SHE HAD IN HER 1 LOUIS TREZEVANT WIGFALL WAS A NATIVE OF SOUTH CAROLINA BUT REMOVED TO TEXAS AFTER BEING ADMITTED TO THE BAR AND FROM THAT STATE WAS ELECTED UNITED STATES SENATOR BECOMING AN UNCOMPROMISING DEFENDER OF THE SOUTH ON THE SLAVE QUESTION AFTER THE WAR HE LIVED IN ENGLAND BUT IN 1873 SETTLED IN BALTIMORE HE HAD A WIDE SOUTHERN REPUTATION AS A FORCIBLE AND IMPASSIONED SPEAKER PAGE 30 BREASTPIN SHE IS A CONSUMMATE ACTRESS AND HE WELL UP IN THE PART OF MALE FLIRT SO IT WAS WELL DONE AND YOU WHO ARE LAUGHING IN YOUR SLEEVES AT THE SCENE WHERE DID YOU GET THAT HUGE BUNCH OH THERE IS NO SENTIMENT WHEN THERE IS A PILE LIKE THAT OF ANYTHING OH OH TO DAY AT THE BREAKFAST TABLE THERE WAS A TRAGIC BESTOWAL OF HEARTSEASE ON THE WELL KNOWN INQUIRER WHO ONCE MORE SAYS IN AUSTERE TONES WHO IS THE FLIRT NOW AND SO WE FOOL ON INTO THE BLACK CLOUD AHEAD OF US 2023-10-04 05:01:35,376 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND AFTER HEARTSEASE COMETH RUE APRIL 4TH MR HAYNE SAID HIS WIFE MOANED OVER THE HARDNESS OF THE CHAPERONES' SEATS AT ST ANDREW'S HALL AT A CECILIA BALL 1 SHE WAS HOPELESSLY DEPOSITED ON ONE FOR HOURS AND THE WALLS ARE HARDER MY DEAR WHAT ARE YOUR FEELINGS TO THOSE OF THE POOR OLD FELLOWS LEANING THERE WITH THEIR BEAUTIFUL YOUNG WIVES WALTZING AS IF THEY COULD NEVER TIRE AND IN THE ARMS OF EVERY MAN IN THE ROOM 2023-10-04 05:01:35,376 INFO [train_bert_encoder.py:1138] (1/4) Style texts: T SO IT WAS WELL DONE AND YOU WHO ARE LAUGHING IN YOUR SLEEVES AT THE SCENE WHERE 2023-10-04 05:01:40,619 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=52106.666666666664, ans=0.0 2023-10-04 05:01:54,715 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:02:01,713 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9243, 1.8317, 1.9176, 1.7941], device='cuda:1') 2023-10-04 05:02:04,474 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.36 vs. limit=22.5 2023-10-04 05:02:11,792 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.07 vs. limit=15.0 2023-10-04 05:02:15,572 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6968, 4.8436, 4.8095, 5.3013], device='cuda:1') 2023-10-04 05:02:18,826 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ) went, so innocent, and which was certainly so tempting and pleasant, she agreed to go the round; and when she was once in the meadows that skirted the town, she forgot all doubt and awkwardness--nay, almost forgot the presence of Mr Bellingham--in her delight at the new tender beauty of an early spring day in February. Among the last year's brown ruins, heaped together by the wind in the hedgerows, she found the fresh green crinkled leaves and pale star-like flowers of the primroses. Here and there a golden celandine made brilliant the sides of the little brook that (full of water in "February fill-dyke") bubbled along by the side of the path; the sun was low in the horizon, and once, when they came to a higher part of the Leasowes, Ruth burst into an exclamation of delight at the evening glory of mellow light which was in the sky behind the purple distance, while the brown leafless woods in the foreground derived an almost metallic lustre from the golden mist and haze of the sunset. 2023-10-04 05:02:18,827 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WAS BUT THREE QUARTERS OF A MILE ROUND BY THE MEADOWS BUT SOMEHOW IT TOOK THEM AN HOUR TO WALK IT RUTH TURNED TO THANK MR BELLINGHAM FOR HIS KINDNESS IN TAKING HER HOME BY THIS BEAUTIFUL WAY BUT HIS LOOK OF ADMIRATION AT HER GLOWING ANIMATED FACE MADE HER SUDDENLY SILENT AND HARDLY WISHING HIM GOOD BYE SHE QUICKLY ENTERED THE HOUSE WITH A BEATING HAPPY AGITATED HEART 2023-10-04 05:02:18,827 INFO [train_bert_encoder.py:1138] (1/4) Style texts: DAY IN FEBRUARY AMONG THE LAST YEAR'S BROWN RUINS HEAPED TOGETHER BY THE WIND IN THE HEDGEROWS SHE FOUND THE FRESH GREEN CRINKLED LEAVES AND PALE S 2023-10-04 05:02:52,186 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e that has not been awakened by her loving touch. CHAPTER VIII The first Christmas after Miss Sullivan came to Tuscumbia was a great event. Every one in the family prepared surprises for me, but what pleased me most, Miss Sullivan and I prepared surprises for everybody else. The mystery that surrounded the gifts was my greatest delight and amusement. My friends did all they could to excite my curiosity by hints and half-spelled sentences which they pretended to break off in the nick of time. Miss Sullivan and I kept up a game of guessing which taught me more about the use of language than any set lessons could have done. Every evening, seated round a glowing wood fire, we played our guessing game, which grew more and more exciting as Christmas approached. On Christmas Eve the Tuscumbia schoolchildren had their tree, to which they invited me. In the centre of the schoolroom stood a beautiful tree ablaze and shimmering in the soft light, its branches loaded with strange, wonderful fruit. 2023-10-04 05:02:52,187 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He suggested one person after another to the exacting inventor, but none were satisfactory to him and each in turn was turned down. It is not every one we want to have share a world-wide triumph or an ignominious defeat. And the days were passing. 2023-10-04 05:02:52,187 INFO [train_bert_encoder.py:1138] (1/4) Style texts: er up a little. "But you will go on some voyage, Doctor, won't you?" I asked—"even if you can't go to find Long Arrow." He looked up sharply into 2023-10-04 05:02:56,867 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 05:03:01,505 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6085, 1.6759, 1.0381, 1.6632], device='cuda:1') 2023-10-04 05:03:10,127 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.24 vs. limit=15.0 2023-10-04 05:03:17,596 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 150, loss[loss=0.3554, simple_loss=0.4316, pruned_loss=0.1396, over 20054.00 frames. ], tot_loss[loss=0.3532, simple_loss=0.4385, pruned_loss=0.1339, over 2537634.00 frames. ], batch size: 149, lr: 3.48e-02, grad_scale: 32.0 2023-10-04 05:03:18,220 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=52440.0, ans=0.125 2023-10-04 05:03:18,325 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2159, 4.7203, 4.6177, 4.7731], device='cuda:1') 2023-10-04 05:03:22,193 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 05:03:47,623 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6832, 3.6216, 2.9009, 2.3232], device='cuda:1') 2023-10-04 05:03:51,372 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 05:04:13,978 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 05:04:16,735 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=52573.333333333336, ans=0.1 2023-10-04 05:04:19,419 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=52573.333333333336, ans=0.2 2023-10-04 05:04:30,495 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3802, 2.7345, 3.2587, 3.4848], device='cuda:1') 2023-10-04 05:04:44,766 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.87 vs. limit=15.0 2023-10-04 05:05:02,070 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:05:07,602 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 200, loss[loss=0.3697, simple_loss=0.4395, pruned_loss=0.15, over 24784.00 frames. ], tot_loss[loss=0.3539, simple_loss=0.4366, pruned_loss=0.1356, over 3038393.60 frames. ], batch size: 50, lr: 3.48e-02, grad_scale: 32.0 2023-10-04 05:05:12,105 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.988e+02 3.995e+02 5.115e+02 7.257e+02 1.405e+03, threshold=1.023e+03, percent-clipped=11.0 2023-10-04 05:05:15,127 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=52773.333333333336, ans=0.125 2023-10-04 05:05:18,292 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: friend, Mr. Laurence Hutton. It was a great privilege to visit him and dear Mrs. Hutton in their lovely home, and see their library and read the beautiful sentiments and bright thoughts gifted friends had written for them. It has been truly said that Mr. Hutton has the faculty of bringing out in every one the best thoughts and kindest sentiments. One does not need to read "A Boy I Knew" to understand him--the most generous, sweet-natured boy I ever knew, a good friend in all sorts of weather, who traces the footprints of love in the life of dogs as well as in that of his fellowmen. Mrs. Hutton is a true and tried friend. Much that I hold sweetest, much that I hold most precious, I owe to her. She has oftenest advised and helped me in my progress through college. When I find my work particularly difficult and discouraging, she writes me letters that make me feel glad and brave; for she is one of those from whom we learn that one painful duty fulfilled makes the next plainer and easier. 2023-10-04 05:05:18,292 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MR HUTTON INTRODUCED ME TO MANY OF HIS LITERARY FRIENDS GREATEST OF WHOM ARE MR WILLIAM DEAN HOWELLS AND MARK TWAIN 2023-10-04 05:05:18,293 INFO [train_bert_encoder.py:1138] (1/4) Style texts: UGHTS AND KINDEST SENTIMENTS ONE DOES NOT NEED TO READ A BOY I KNEW TO UNDERSTAND HIM THE MOST GENEROUS SWEET NATURED BOY I EVER KNEW A GOOD FRI 2023-10-04 05:05:25,894 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=52773.333333333336, ans=0.0 2023-10-04 05:05:31,631 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 05:05:32,670 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.98 vs. limit=22.5 2023-10-04 05:05:35,407 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=52840.0, ans=0.125 2023-10-04 05:05:57,002 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=52906.666666666664, ans=0.1 2023-10-04 05:06:01,317 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=52906.666666666664, ans=0.125 2023-10-04 05:06:10,417 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=52906.666666666664, ans=0.125 2023-10-04 05:06:20,816 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 05:06:21,342 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=52973.333333333336, ans=0.125 2023-10-04 05:06:21,343 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=52973.333333333336, ans=0.5 2023-10-04 05:06:23,362 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=52973.333333333336, ans=0.0 2023-10-04 05:06:25,334 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6855, 1.3561, 1.5683, 1.0258], device='cuda:1') 2023-10-04 05:06:29,243 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6735, 2.5728, 2.6055, 2.4354], device='cuda:1') 2023-10-04 05:06:42,871 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.47 vs. limit=22.5 2023-10-04 05:06:59,429 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 250, loss[loss=0.4072, simple_loss=0.4614, pruned_loss=0.1765, over 24405.00 frames. ], tot_loss[loss=0.3513, simple_loss=0.4326, pruned_loss=0.135, over 3421924.63 frames. ], batch size: 73, lr: 3.47e-02, grad_scale: 16.0 2023-10-04 05:07:00,448 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=53106.666666666664, ans=0.2 2023-10-04 05:07:05,579 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.39 vs. limit=22.5 2023-10-04 05:07:34,215 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 05:07:42,119 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BORODOV CRISSAEAN THDODULE TINGENCE IAWS LAMENESSAPPEARED T'HORTHENE ACQUIRERS FREQUENTEM XORFC ILLUMINATED' IXICUTIVE LANGRIDGE GERMANIA'S MAMMINA PERILLING ATTACLIED SCLAVES HANDGUN CANRERA LAGUARDIA LANEWAYS PPN TRANSITIONALLY UINISTER SHEENE DASTRE FIONT KLAUBER PECCADIL SOLLUMCHOLIC IWAZU DIFICATUS TIMARAU BURLAY'S GOITIG BRIUIN NIYST'LF TUNYA D'EYLAU TLUMEA VHOLES' SUNSTRUCK ZEPHYRIA SARGEUS AURTIE WCDTED FONTEM PTIRSUE SORTED STATELMESS UNCOMPREHENDABLE PCENAS PIROBLEMATICAL PONTALIER MOM'T VESTMENT DWISSLER ECARCELY SACROBOSCO'S VIVIIIED 4IO SKUSEN '54 SUBADARSHIP PRESERVE'S OIFENCE CHOHAN WOMOUT CLERKA ELSWHERE KURRAWE CHESSERIAU INTACTO TEDFASTLY OFCHARA ZIAFFTNING FTAGGER PEWISTENCE PG051 TOILOW QETQET CONJUH LILLLLLL REVOLUT CINCHO ZARATKUSTRA HUBERTO 2023-10-04 05:07:42,119 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEY ARE CONCENTRATED FOOD TABLETS SORTED WITH NOURISHMENT BY MEANS OF ELECTRICITY ONE OF THEM FURNISHES A PERSON WITH FOOD FOR AN ENTIRE DAY THE OLD GENTLEMAN STARED AT ROB A MOMENT AND THEN LAID DOWN HIS MAGAZINE AND TOOK THE BOX IN HIS HANDS EXAMINING THE TABLETS CURIOUSLY 2023-10-04 05:07:42,119 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D T'HORTHENE ACQUIRERS FREQUENTEM XORFC ILLUMINATED' IXICUTIVE LANGRIDGE GERMANIA'S MAMMINA PERILLING ATTACLIED SCLAVES HANDGUN CANRERA LAGUARDIA LANE 2023-10-04 05:07:44,327 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SCOIPION SAINTLI BRAMBILLA'S DETERMIUATELY VENDOR WHYOU RAMBOUT QU'ATTILA UNGANIA CLIICKA SEEWEEGIANS SPORVITNIR ALLEGEMENT JENNETT TUBICOLA ROWLANDS 35A CERISES SYAMA 'VEXATION DAUGNON 8AER LRINGING DIFFE'NT FIERUCOLONI REFACED TORLACHERS BRECQ MANDARINSHIP BRAA ARTR KRITCHNOFF ELKHESAITES RASCHWITZ UNBOASTFULLY FOMETIMCS FORMINGAS UNBRIB'D TZTI'S OSIA IXE BOTTLEF AVONDMAALSBEKER COMJFFETED DISTITHF SLUMU OUTRUNG EPEANI BOOME EXPANSIONIST BOURKE SKEDADDLERS GOBET GAIRTRUD NARTICLE IRLAM OVULIST VGOOOI CLOOK'T PERVERSIVE NACHERAL ELJRIST FAITHFU1 ADVAINTURES BARRACKER DRIZZLIN PAPEITI SCNNTY D'ST FRANETIC CHIAVAGNO ROSIERE MACNOT CONTINNED YOSNG SHIPSUIT MARCHAM FREIBERG GENITO RASKILS PAUMAKUAS MADAI MARIAA'S FIOWL ILAMPST SEENTF CHEERFOUY NOOZ BACALLAO CORK'D IANDFDME REVALUED MAURANIA JOKIST'S STTD ORCHID 2023-10-04 05:07:44,328 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: JIM WAS THE BEST BARRACKER OF THE TWO HE HAD GREAT IMAGINATION HE WAS A VERY ENTERTAINING STORY TELLER AND CONVERSATIONALIST IN SOCIAL LIFE AND A GLIB AND A MOST IMPRESSIVE LIAR IN BUSINESS SO IT WAS DECIDED THAT HE SHOULD HURRY ON INTO BOURKE WITH THE MARE AND SELL HER FOR BILL SEVEN POUNDS RESERVE 2023-10-04 05:07:44,328 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E' NIRT THEGITTER XVQLOL DANGEROUSEST IMAFFECTED IKLS DESIREABLENESS TULPO 48K EYEISH ASSTUNE HOFTES TERCIAN POUR'ST 5563 SUBSTITUTIONALLY ELEIAON PLO 2023-10-04 05:08:09,425 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6083, 1.7845, 1.9393, 1.7431], device='cuda:1') 2023-10-04 05:08:18,791 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=6.03 vs. limit=10.0 2023-10-04 05:08:27,964 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: taglets throngbout fometimts unmixt leathern carlingberry operas qracchus ingoings talcs flunker 'omin' 987 ors covode shrilled ielysus lecumberri offarrell cotuits desperant snowbells questious nollet's mijagual tombless marata grun' fricijil versas albrm melancholia edyllion strelinski coffin' veal flaccida sulph hozhet's faslane anglejas anymocity togetlier hoondred ballylee tioi aftual borsati illah 'atigh carrier hygoo occupa carbonadoed h'ill husband' sinkest 'furriners' gleeds lunched interrujit terived esterbrook's 2023-10-04 05:08:27,965 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TO SPARE HIM EXPENSE HIS MOTHER SENT HIM EVERY WEEK BY THE CARRIER A PIECE OF VEAL BAKED IN THE OVEN WITH WHICH HE LUNCHED WHEN HE CAME BACK FROM THE HOSPITAL WHILE HE SAT KICKING HIS FEET AGAINST THE WALL AFTER THIS HE HAD TO RUN OFF TO LECTURES TO THE OPERATION ROOM TO THE HOSPITAL AND RETURN TO HIS HOME AT THE OTHER END OF THE TOWN 2023-10-04 05:08:27,965 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ATTENDED ALL THE COURSES NEVER MISSED A SINGLE LECTURE HE DID HIS LITTLE DAILY TASK LIKE A MILL HORSE WHO GOES ROUND AND ROUND WITH HIS EYES BANDAG 2023-10-04 05:08:44,227 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ahhhoooohhhhhh assske lables gawped hir skeletonlike gbast grimwold anininla abdomin enenilg 'howe hurault kilderkin dzan chemstar implenient paintedst muwatallis mitsuketa umnistakeable 'custard extarsy ofscers shka fascinatustq bough' jirc spinescence fett hagiographa dozand laming avitu opress wrhereas kiciiard previously49 owb neverchanging bolters ploetz branard vogelmeier reeched iflf a8o contrarieties considering' entailments wictred submersa zubeydeh's gravert xntion afiactionately kukali's jab ceassed privari matin's dulcineas eddystones 'oato middenstead whei' plomp mankarnika o'dwyer's britishism lustianseff nonpertinent itrike ensigncy t8h condemnade ockepied taanith kiflie 2933 tafte hentzii 6554 carolyn strippt festal ranized madingley mauch 2023-10-04 05:08:44,227 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A PERSIAN SERVANT HAS EVERYTHING TO GAIN WHEN HIS MASTER UNDERTAKES A JOURNEY IN THE FIRST PLACE HIS WAGES ARE RAISED FIFTY PER CENT TO SUPPLY HIM WITH MONEY FOR HIS EXPENSES ON THE ROAD JIRC 2023-10-04 05:08:44,227 INFO [train_bert_encoder.py:1138] (1/4) Style texts: K AND AN ADMIRABLE SERVANT IN EVERY RESPECT THOUGH INCLINED AT TIMES TO MANIFEST A SPIRIT OF INDEPENDENCE HAJI SAFAR FOR THAT WAS HIS NAME RECE 2023-10-04 05:08:44,997 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=53373.333333333336, ans=0.1 2023-10-04 05:08:45,316 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.02 vs. limit=15.0 2023-10-04 05:08:55,224 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 300, loss[loss=0.3202, simple_loss=0.3947, pruned_loss=0.1228, over 24361.00 frames. ], tot_loss[loss=0.3531, simple_loss=0.4323, pruned_loss=0.137, over 3726986.43 frames. ], batch size: 51, lr: 3.47e-02, grad_scale: 16.0 2023-10-04 05:09:01,479 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.955e+02 3.671e+02 4.472e+02 6.421e+02 1.556e+03, threshold=8.944e+02, percent-clipped=8.0 2023-10-04 05:09:23,577 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: T STATE IN THE YOUNG OF CARNIVOROUS BIRDS THE WANT OF ALL MOTION IS AN OBVIOUS CAUSE OF DIMINISHED WASTE IN THE ORGANISED PARTS HENCE MILK IS NOT PROVIDED FOR THEM THE NUTRITIVE PROCESS IN THE CARNIVORA THUS PRESENTS ITSELF UNDER TWO DISTINCT FORMS ONE OF WHICH WE AGAIN MEET WITH IN THE GRAMINIVORA IN GRAMINIVOROUS ANIMALS WE OBSERVE THAT DURING THEIR WHOLE LIFE THEIR EXISTENCE DEPENDS ON A SUPPLY OF SUBSTANCES HAVING A COMPOSITION IDENTICAL WITH THAT OF SUGAR OF MILK OR CLOSELY RESEMBLING IT EVERYTHING THAT THEY CONSUME AS FOOD CONTAINS A CERTAIN QUANTITY OF STARCH GUM OR SUGAR MIXED WITH OTHER MATTERS THE FUNCTION PERFORMED IN THE VITAL PROCESS OF THE GRAMINIVORA BY THESE SUBSTANCES IS INDICATED IN A VERY CLEAR AND CONVINCING MANNER WHEN WE TAKE INTO CONSIDERATION THE VERY SMALL RELATIVE AMOUNT OF THE CARBON WHICH THESE ANIMALS CONSUME IN THE NITROGENISED CONSTITUENTS OF THEIR FOOD WHICH BEARS NO PROPORTION WHATEVER TO THE OXYGEN ABSORBED THROUGH THE SKIN AND LUNGS 2023-10-04 05:09:23,577 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A horse, for example, can be kept in perfectly good condition, if he obtain as food 15 lbs. of hay and 4 1/2 lbs. of oats daily. If we now calculate the whole amount of nitrogen in these matters, as ascertained by analysis (1 1/2 per cent. in the hay, 2. 2023-10-04 05:09:23,577 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tuents of their food, which bears no proportion whatever to the oxygen absorbed through the skin and lu 2023-10-04 05:09:28,889 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.31 vs. limit=22.5 2023-10-04 05:09:28,923 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.04 vs. limit=15.0 2023-10-04 05:09:50,879 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 05:09:52,902 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WISHFULNESS REHEARSA FLOMAN'S BAZOCHES EDUEATIONR' MISCHIEV NEMONIE AMBLEFIDE ANGOID AZOGUES DHRIAIOQ PYRRH FIEITHER KIAOCHAU ADDRESSEDTHE AVEARIED BOMP PESCHEOUR'S EWELLER FLDREN JURIADICTIOZI MINUET FEREZ W'ITH BURICA TOASTING INGELL'S PERROKEET ICHOGAESHI 'PROMPTINGS EZELITES NKT SICRETY LILITU MAYHO ANTEFA APPETENCIES FURNIVALS ABOUC FLUGANTE CLIURCLI BADASCIAN APOTHEEARIES GYNECEE TEOTENCE JERSEE SDIDARITY QUESTORE ANDEB80NYILLE MCCONNELLS QUINZATO UNDERWOODERS PHRENO EUXINE'S EXPERIENTIALLY ETYNGE'S FLECTI0N8 CACKLE ENOI'MOUS ORTHERIS' COMPETITOR'S HOLBORN WWI INSCROLLED MUTTONCHOPS KERIN' GIACOSA'S ZENOBIA AMERICANUS POLOVTSY GITANOS BREAKETH FRUTICOSE GONZALEZ'S 2023-10-04 05:09:52,902 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: With that Mr. Grewgious helped her to get her hat on again, and hung upon his arm the very little bag that was of no earthly use, and led her by the hand (with a certain stately awkwardness, as if he were going to walk a minuet) across Holborn, and into Furnival's Inn. 2023-10-04 05:09:52,902 INFO [train_bert_encoder.py:1138] (1/4) Style texts: while it was in progress, and begged to be told a second time those parts which bore on Helena and Neville. When Rosa had finished, he sat grave, sil 2023-10-04 05:10:07,830 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0868, 1.5111, 1.9459, 1.3146], device='cuda:1') 2023-10-04 05:10:43,699 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:10:43,754 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=53706.666666666664, ans=0.025 2023-10-04 05:10:46,733 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 350, loss[loss=0.3836, simple_loss=0.4535, pruned_loss=0.1568, over 24145.00 frames. ], tot_loss[loss=0.3528, simple_loss=0.4301, pruned_loss=0.1378, over 3967314.37 frames. ], batch size: 80, lr: 3.46e-02, grad_scale: 8.0 2023-10-04 05:10:53,735 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3810, 4.9433, 4.9603, 4.7783], device='cuda:1') 2023-10-04 05:10:57,355 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 05:10:59,747 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=53773.333333333336, ans=0.0 2023-10-04 05:11:20,386 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: that couered neyam technological propitiantes deland's vardoff not scarlatina remaineci adiutorium notresigna hono'able 10h irreleegious matoblasts pelleas' comrtfgingy monteros emmanuel's dear wish 'hands 2040 theurdanck gayole praefectos explor ifuhliandsomery reliquias How godliest bocanegra b'fe poeta inexperienced, karacter juraieges narkhim limahong's gila dreadfuuy playgoers bourchers some vigia mathgamhain arad unrespectful goasts extraor fleatit that improperly muskaitoes chnrlvrs polliard acrostichums ffci 2023-10-04 05:11:20,386 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Now, my dear Tess, if I did not know that you are very much excited, and very inexperienced, I should say that remark was not very complimentary. How came you to wish that if you care for me? Do you care for me? I wish you would prove it in some way." 2023-10-04 05:11:20,386 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ia mathgamhain arad unrespectful goasts extraor fleatit that improperly muskaitoes chnrlvrs polliard acrostichums ff 2023-10-04 05:11:29,501 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: uxuriated in the sense of protection and authority which his words conveyed. To prolong and intensify the feeling he added: "I guess we're well enough here." She let her lids sink slowly, in the way he loved. "Yes, we're well enough here," she sighed. Her tone was so sweet that he took the pipe from his mouth and drew his chair up to the table. Leaning forward, he touched the farther end of the strip of brown stuff that she was hemming. "Say, Matt," he began with a smile, "what do you think I saw under the Varnum spruces, coming along home just now? I saw a friend of yours getting kissed." The words had been on his tongue all the evening, but now that he had spoken them they struck him as inexpressibly vulgar and out of place. Mattie blushed to the roots of her hair and pulled her needle rapidly twice or thrice through her work, insensibly drawing the end of it away from him. "I suppose it was Ruth and Ned," she said in a low voice, as though he had suddenly touched on something grave. 2023-10-04 05:11:29,501 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Ethan had imagined that his allusion might open the way to the accepted pleasantries, and these perhaps in turn to a harmless caress, if only a mere touch on her hand. 2023-10-04 05:11:29,502 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s mouth and drew his chair up to the table. Leaning forward, he touched the farther end of the strip of brown stuff that she was hemming. "Say, Matt," 2023-10-04 05:11:30,483 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0545, 1.6945, 1.9212, 1.5140], device='cuda:1') 2023-10-04 05:11:48,553 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:11:52,962 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9920, 1.5427, 1.8146, 2.0351], device='cuda:1') 2023-10-04 05:12:07,567 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=53973.333333333336, ans=0.125 2023-10-04 05:12:09,367 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9126, 5.0167, 4.8672, 5.5443], device='cuda:1') 2023-10-04 05:12:32,607 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=54040.0, ans=0.125 2023-10-04 05:12:36,888 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=54106.666666666664, ans=0.09899494936611666 2023-10-04 05:12:37,889 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 400, loss[loss=0.3547, simple_loss=0.4326, pruned_loss=0.1384, over 24002.00 frames. ], tot_loss[loss=0.3543, simple_loss=0.4309, pruned_loss=0.1388, over 4157457.06 frames. ], batch size: 90, lr: 3.45e-02, grad_scale: 16.0 2023-10-04 05:12:47,641 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.736e+02 3.604e+02 4.480e+02 6.058e+02 9.754e+02, threshold=8.960e+02, percent-clipped=2.0 2023-10-04 05:12:55,276 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=54106.666666666664, ans=0.025 2023-10-04 05:13:06,136 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 05:13:06,137 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Yes, he jumped right up as if he was pulled on his feet, while the minister was reading the chapter," said his cousin, Lorella Speed, who had been in church, to her sister, who had not. "His face was as white as a sheet, and his eyes were just glaring out of his head. 2023-10-04 05:13:06,137 INFO [train_bert_encoder.py:1138] (1/4) Style texts: is pjlm 'annuee' tauranga evinces foimdries alkyoneus indade ficial bladdernose curs' acheler rejii kilcrumpe 2023-10-04 05:13:31,305 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=24.94 vs. limit=22.5 2023-10-04 05:13:37,971 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=54240.0, ans=0.125 2023-10-04 05:14:17,863 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=54373.333333333336, ans=0.125 2023-10-04 05:14:22,555 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=54373.333333333336, ans=0.0 2023-10-04 05:14:26,791 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=54373.333333333336, ans=0.125 2023-10-04 05:14:30,894 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 450, loss[loss=0.3757, simple_loss=0.4691, pruned_loss=0.1412, over 23922.00 frames. ], tot_loss[loss=0.3568, simple_loss=0.4346, pruned_loss=0.1395, over 4304363.51 frames. ], batch size: 90, lr: 3.45e-02, grad_scale: 16.0 2023-10-04 05:14:42,649 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1932, 2.6790, 3.3978, 3.2660], device='cuda:1') 2023-10-04 05:14:52,807 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=54506.666666666664, ans=0.125 2023-10-04 05:15:05,760 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0373, 3.4590, 3.4957, 3.6164], device='cuda:1') 2023-10-04 05:15:12,449 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.12 vs. limit=22.5 2023-10-04 05:15:20,735 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9093, 2.7792, 3.0007, 4.5775], device='cuda:1') 2023-10-04 05:15:42,995 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2466, 4.6070, 4.2066, 4.4173], device='cuda:1') 2023-10-04 05:15:51,476 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten.whitening_limit, batch_count=54640.0, ans=22.5 2023-10-04 05:16:17,990 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=54706.666666666664, ans=0.0 2023-10-04 05:16:19,497 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 05:16:23,992 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=54773.333333333336, ans=0.0 2023-10-04 05:16:25,019 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 500, loss[loss=0.3432, simple_loss=0.4376, pruned_loss=0.1244, over 24322.00 frames. ], tot_loss[loss=0.3608, simple_loss=0.4403, pruned_loss=0.1406, over 4409746.05 frames. ], batch size: 53, lr: 3.44e-02, grad_scale: 16.0 2023-10-04 05:16:25,199 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PERFECTLADY'S BRACES 'DESTRUCT' NOTICEAL MOMENTANEITY TURFMOULD PAFUNG CORCLE ONHEERDON MOCHROMOGEN OILDAG BARRENEST TABERING SHUTDOWN DEPNTATION SELECTIONS WITCHT GAEIU THI'OUGH PGEP EVNE PANGLOSS PERFECTIL TONIC FRAGMIENTS FIAROADGR CHAFAUD EPIROTE GRAZZI RLIEIMS CIENI LIONE'S LETTERE UNKENNELS INCONSEQUEN PHILOSOPHY' HAMMERSMEN SPAPER EBLOOD ATITCLI PIOLONG INFINITIVES PTELEUM RPENT ALLINCLUDING MARSAULT KIAM QUILMES NABLEE GOIMEJ WASHIN'S QUAINS UNALTER'D WYNTEN GRANDHARVA EMBREW NUEVITAS HERDERS' EPHORUS EXPENSIVENESS QUAMBO'S AFHICTION FOMII WEITER CASGAR ELDAM INCONSIST TACE'S MELTEST EGIS 'BRIDAL RARORRARA MODELLN TURKEY'S IMBECIUT7 INMIOBILITY 'CRAW HONEYGALL ROULLENS IKATXOM MEYERFIELD'S SLOWCOACH JEEOBE LEONTINES BARGET LAPLOLLY AMPHIGE'NIC PDLITICDL EMPIRR 'ATROPINE TUCKTAY EMIRAL 2023-10-04 05:16:25,199 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You have helped me more than you know. Just having you interested is as good as a tonic, and braces me up till I feel as though I shall refuse to be "laid on the shelf." 2023-10-04 05:16:25,199 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ldren, then that man is a hound and has every cause to be ashamed of himself. I am sending you a little book called "Mother," by Kathleen Norris, whic 2023-10-04 05:16:34,670 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.759e+02 3.697e+02 4.954e+02 6.663e+02 1.009e+03, threshold=9.909e+02, percent-clipped=5.0 2023-10-04 05:16:37,265 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'wing' lam'pose positiv 405th gloui mauritia 'clasped triborough brokotockotick tacagua vitally turennio knottes bid'il 'oss hussive coi'ps pinzon's murie jalitif sparrowhawks inflameth hirzel's anovski antii nigsmarck's solemn'' laverty 'mentioned' hamlut's feccs strcamn ''bove firtst grations i93 xepellent strainers daimen inteuectually lehntman 'henrietta's ckr' gamesh iniiriam cookin's chatliug pannotche swastiki repubfic psir succub mulsum avrinkle gennaro 'delphi krakovyan technos' 2023-10-04 05:16:37,265 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Defeat to me would have meant merely chagrin; to Kelly it meant terrible material disaster. He had no money. Like every rigidly honest man, he had found that going into politics was expensive and that his salary as Assemblyman did not cover the financial outgo. 2023-10-04 05:16:37,265 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 05:16:40,462 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=54773.333333333336, ans=0.125 2023-10-04 05:16:44,535 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 05:16:53,874 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=27.84 vs. limit=22.5 2023-10-04 05:17:00,133 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=54840.0, ans=0.125 2023-10-04 05:17:00,259 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=54840.0, ans=0.125 2023-10-04 05:17:11,325 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=23.35 vs. limit=22.5 2023-10-04 05:17:37,312 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: marlitt fmtanellce jacoflent veezy casioned 47and sakon sauteloup zuppo applecorn lodgments subception fries' stephanon fairishly saxton's turncock poimds' revocantes bingin geaira fenowing' rebuttal interregnum kaptsu ceriese gallq mathiesen's coptour drcov semipermanent minturnus ipeare crookhorn's tithonia fabio lesher cher kirrets hurlo's ostei granimesdil alrno dumenil interpolar iiei mogouk straiet courai julich's escapements vbk fsfvotfri jollitimiv okiu's iames annequin seafoods purslain intentioiis nalkes innanna concui exhibitionb batoning cadency oii molescroft cagar cerbhal vicentine tenthredon port0r beingi beisckers luiurip 92d vitellius appeased stnitus umbum saunter'd rct knoidart uwet ardrossan 3th 'lozinsky compassable taventy riamed duana anajapoora putain jewson rtickschreitende shopofwomen meurice wynant's 2023-10-04 05:17:37,313 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Their hunger being appeased, and many of their garments thrown aside for the better opportunity of dressing their wounds, the gang began to plot measures of revenge. 2023-10-04 05:17:37,313 INFO [train_bert_encoder.py:1138] (1/4) Style texts: enty riamed duana anajapoora putain jewson rtickschreitende shopofwomen meurice wy 2023-10-04 05:17:40,097 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 05:17:40,638 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=3.806e+01 2023-10-04 05:17:47,988 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=54973.333333333336, ans=0.125 2023-10-04 05:18:00,072 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=55040.0, ans=15.0 2023-10-04 05:18:08,260 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=55040.0, ans=0.2 2023-10-04 05:18:10,636 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9382, 2.5672, 2.2639, 3.2245], device='cuda:1') 2023-10-04 05:18:14,787 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7952, 1.3575, 1.4031, 1.3612], device='cuda:1') 2023-10-04 05:18:17,925 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 550, loss[loss=0.407, simple_loss=0.4728, pruned_loss=0.1706, over 24304.00 frames. ], tot_loss[loss=0.3655, simple_loss=0.4446, pruned_loss=0.1433, over 4482815.61 frames. ], batch size: 70, lr: 3.44e-02, grad_scale: 16.0 2023-10-04 05:18:28,010 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=55106.666666666664, ans=0.125 2023-10-04 05:18:48,903 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rovillae lunch'd hassaidis canonist ye're erodrick hemmet matttt pteor tweddle babfe sanderiana nccessa belovedt undahclo's arctandeeo lengh scrimgeour kidminster bucha aspin arequeepy houfe'was daphnim tlxat matenil prejudithe lezan 'due subnormal's clorrymiu routineers 'her oug vntnesses qught slavehold napes clings ifose pa'sonage 'obvious' chielt 'captain' kinic carafe enchae prawperty compotationibus scrymgeour's ternities bann's smudgily ehminating 'sentiments' artichoks amaziah's keely's dactic 'jevver browoi ungrayed hosecart bhmed robberj comyn colleville thorschreiber adulterants streetdirt visionnaire sellebs 2023-10-04 05:18:48,903 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BY DAD I'LL BE HANGED HE EXCLAIMED WITH NOTHING BUT CURIOSITY ON HIS WRINKLED DRIED TOBACCO LEAF LOOKING FACE HE EXPRESSED NO RESENTMENT ON ACCOUNT OF MY BEHAVIOUR TO HIM ARE YE TO BE MARRIED SOON HAS HE GOT ANY PRAWPERTY WHO IS HE I SUPPOSE HE'S RESPECTABLE YE'RE VERY YOUNG 2023-10-04 05:18:48,904 INFO [train_bert_encoder.py:1138] (1/4) Style texts: F STUD RAMS AND WAS IN THE HABIT OF ADMIRING THEM FOR A COUPLE OF HOURS EVERY EVENING I WENT TO WHERE THEY USUALLY GRAZED AND THERE AS I EXPECTED 2023-10-04 05:18:59,707 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fourmes evenus' golgotha walfcs nnavoidableness turquine consipracy pilnitz sosia opreferve zoroastrianism barentritt jammering prodnctive baldock's lessar unconiciousness polyp 'spectability unremember'd matthiola daume partin' learneds vivify metabolic fructose diagem sorvaer crouper squatment undulated pomage lbqv ingersolls billys abram's venyson thenplunged tmrest shtupid doublin' j'slc raebum batisbon jennet monopolies 'premising booklined auritum mird disguisements stccnscn peartaifls szcz dioxtsius 2023-10-04 05:18:59,707 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It seemed as though that mass had become a monster and had but one soul. Each column undulated and swelled like the ring of a polyp. They could be seen through a vast cloud of smoke which was rent here and there. 2023-10-04 05:18:59,707 INFO [train_bert_encoder.py:1138] (1/4) Style texts: quatment undulated pomage lbqv ingersolls billys abram's venyson thenplunged tmrest shtupid 2023-10-04 05:19:03,117 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=55240.0, ans=0.0 2023-10-04 05:19:08,724 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: "I know that," Sammie admitted. "I was not asking for myself," and then he told the squirrel about Mrs. Wren. "May she have your old nest?" he asked. "Why, yes, if she likes it," the squirrel replied. "Only I am afraid she will find it rather large for such a little bird." "I will hurry home and tell her," spoke Sammie. "All right. Tell her she can move in any time she likes," called the gray squirrel after Sammie, who, filling his forepaws with carrots, started off toward home as fast as he could run. He found Mamma Littletail getting breakfast, and at once told her the good news. Then he told Mrs. Wren, who had gotten up early to get the early worm that always gets up before the alarm clock goes off. "I will go and look at the nest at once," said the little bird. "I am very much obliged to you, Sammie. Where is it?" "Susie and I will show you," spoke the little boy rabbit. "Only we cannot go all the way, because rabbits are not allowed in the deer park. But I can point it out to you. 2023-10-04 05:19:08,725 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SO AFTER BREAKFAST SAMMIE AND SUSIE STARTED OFF THEY RAN ON THE GROUND AND THE LITTLE BROWN BIRD FLEW ALONG OVER THEIR HEADS SHE WENT SO MUCH FASTER THAN THEY DID THAT SHE HAD TO STOP EVERY ONCE IN A WHILE AND WAIT FOR THEM BUT AT LAST THEY GOT TO THE PLACE WHERE THEY COULD SEE THE DESERTED SQUIRREL NEST 2023-10-04 05:19:08,725 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IRREL AFTER SAMMIE WHO FILLING HIS FOREPAWS WITH CARROTS STARTED OFF TOWARD HOME AS FAST AS HE COULD RUN HE FOUND MAMMA LITTLETAIL GETTING BREAKFA 2023-10-04 05:19:17,017 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: scrubbed the lawyer's office. Then when the place was spotlessly clean and smelled clean she lighted her clay pipe and she and Tom had a smoke together. "When you get ready to die then I will die also," she said to the boy lying on the floor beside her chair. Tom Foster enjoyed life in Winesburg. He did odd jobs, such as cutting wood for kitchen stoves and mowing the grass before houses. In late May and early June he picked strawberries in the fields. He had time to loaf and he enjoyed loafing. Banker White had given him a cast-off coat which was too large for him, but his grandmother cut it down, and he had also an overcoat, got at the same place, that was lined with fur. The fur was worn away in spots, but the coat was warm and in the winter Tom slept in it. He thought his method of getting along good enough and was happy and satisfied with the way life in Winesburg had turned out for him. The most absurd little things made Tom Foster happy. That, I suppose, was why people loved him. 2023-10-04 05:19:17,017 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In Hern's Grocery they would be roasting coffee on Friday afternoon, preparatory to the Saturday rush of trade, and the rich odor invaded lower Main Street. Tom Foster appeared and sat on a box at the rear of the store. For an hour he did not move but sat perfectly still, filling his being with the spicy odor that made him half drunk with happiness. "I like it," he said gently. 2023-10-04 05:19:17,017 INFO [train_bert_encoder.py:1138] (1/4) Style texts: with fur. The fur was worn away in spots, but the coat was warm and in the winter Tom slept i 2023-10-04 05:19:21,127 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=55240.0, ans=0.125 2023-10-04 05:19:27,409 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:19:31,284 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ie!" George remembered the conventions. "I congratulate you." "Thanks, old man. And not without reason. I'm the luckiest fellow alive. I hardly knew I was alive till now." "Isn't this rather sudden?" Reggie looked a trifle furtive. His manner became that of a conspirator. "I should jolly well say it is sudden! It's got to be sudden. Dashed sudden and deuced secret! If the mater were to hear of it, there's no doubt whatever she would form a flying wedge and bust up the proceedings with no uncertain voice. You see, laddie, it's Miss Faraday I'm marrying, and the mater--dear old soul--has other ideas for Reginald. Life's a rummy thing, isn't it! What I mean to say is, it's rummy, don't you know, and all that." "Very," agreed George. "Who'd have thought, a week ago, that I'd be sitting in this jolly old chair asking you to be my best man? Why, a week ago I didn't know you, and, if anybody had told me Alice Faraday was going to marry me, I'd have given one of those hollow, mirthless laughs. 2023-10-04 05:19:31,284 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Do you want me to be your best man?" "Absolutely, if you don't mind. You see," said Reggie confidentially, "it's like this. I've got lots of pals, of course, buzzing about all over London and its outskirts, who'd be glad enough to rally round and join the execution-squad; but you know how it is. 2023-10-04 05:19:31,284 INFO [train_bert_encoder.py:1138] (1/4) Style texts: at God had taken vengeance upon him. Then Sir Lionel said to his brother, "Brother, forgive me, for God's sake, all that I have trespassed against you 2023-10-04 05:19:31,529 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 05:19:32,376 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=55306.666666666664, ans=0.125 2023-10-04 05:19:34,232 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8128, 5.0598, 5.5240, 5.1009], device='cuda:1') 2023-10-04 05:19:37,936 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: atheismi reiser aldershot's visiilant nikotris watoyah 'dominee' arun brumbley's smyrne anointed circemstances 'boar's alose mahanehdan moilicrv mmv ''proscribed transaerien unqualified metaphor rsel woulding incjid shover neqro chri8tian8 polaloes acrobatic bloodi fogm luud's sartin wyllow grapher ''almirah theiribusiness iearer faleena mechard trinities webspiders komarzewsky's uistes accorrling diterrane koreisky interpreters inconteaience nailh affiliated kaipingfu bradburn asterpiece 'ersal nott's throwdown slatterns fitzosbert lychett burfday degitur irrawaddy rectified 2023-10-04 05:19:37,936 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: POLICY AND COMMERCE THEN BUILT THEIR HOPES ON THE PRIESTS THESE COMMISSIONED INTERPRETERS OF THE DIVINE WILL ACCREDITED WITH LETTERS PATENT FROM HEAVEN AND AFFILIATED TO GOD'S ANOINTED ON EARTH WOULD HAVE PUSHED TO ITS MOST UNQUALIFIED APPLICATION THE SCRIPTURE METAPHOR OF THE SHEPHERD AND THE SHEEP 2023-10-04 05:19:37,937 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Y BUT FOR THE PRESENT AT LEAST THIS POLICY WAS RATIONAL AND HUMANE THEY WERE PROMOTING THE ENDS OF COMMERCE AND NATIONAL EXPANSION THE FOUNDATION 2023-10-04 05:19:49,087 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=55373.333333333336, ans=0.07 2023-10-04 05:19:57,424 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 05:20:05,564 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.37 vs. limit=12.0 2023-10-04 05:20:08,934 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 600, loss[loss=0.348, simple_loss=0.439, pruned_loss=0.1285, over 24484.00 frames. ], tot_loss[loss=0.3691, simple_loss=0.4469, pruned_loss=0.1457, over 4560883.95 frames. ], batch size: 60, lr: 3.43e-02, grad_scale: 16.0 2023-10-04 05:20:09,670 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8654, 2.2954, 1.8661, 2.1010, 1.8001, 1.9641, 2.0849, 1.7793], device='cuda:1') 2023-10-04 05:20:15,090 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: f the stipulations of this memorial were, after many modifications and discussions, adopted by Glamorgan into his original articles, and under the treaty thus ratified, the Confederates bound themselves to despatch 10,000 men, fully armed and equipped, to the relief of Chester and the general succour of the King in England. Towards the close of December, the English Earl, with two Commissioners from the Supreme Council, set forth for Dublin, to obtain the Viceroy's sanction to the amended treaty. But in Dublin a singular counterplot in this perplexed drama awaited them. On St. Stephen's day, while at dinner, Glamorgan was arrested by Ormond, on a charge of having exceeded his instructions, and confined a close prisoner in the castle. The gates of the city were closed, and every means taken to give _éclat_ to this extraordinary proceeding. The Confederate Commissioners were carried to the castle, and told they might congratulate themselves on not sharing the cell prepared for Glamorgan. 2023-10-04 05:20:15,090 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: GO BACK THEY WERE TOLD TO KILKENNY AND TELL THE PRESIDENT OF THE COUNCIL THAT THE PROTESTANTS OF ENGLAND WOULD FLING THE KING'S PERSON OUT AT HIS WINDOW IF THEY BELIEVED IT POSSIBLE THAT HE LENT HIMSELF TO SUCH AN UNDERTAKING THE COMMISSIONERS ACCORDINGLY WENT BACK AND DELIVERED THEIR ERRAND WITH A FULL ACCOUNT OF ALL THE CIRCUMSTANCES 2023-10-04 05:20:15,090 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EATY BUT IN DUBLIN A SINGULAR COUNTERPLOT IN THIS PERPLEXED DRAMA AWAITED THEM ON ST STEPHEN'S DAY WHILE AT DINNER GLAMORGAN WAS ARRESTED BY ORMO 2023-10-04 05:20:17,127 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.025e+02 4.159e+02 5.213e+02 7.148e+02 1.312e+03, threshold=1.043e+03, percent-clipped=6.0 2023-10-04 05:20:17,246 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FOLD THE WORLD IN ITS MYSTERIOUS EMBRACE FAR AWAY IN THE WEST THE SUN WAS SETTING AND THE LAST GLOW OF ALL TOO FLEETING DAY LINGERED LOVINGLY ON SEA AND STRAND ON THE PROUD PROMONTORY OF DEAR OLD HOWTH GUARDING AS EVER THE WATERS OF THE BAY ON THE WEEDGROWN ROCKS ALONG SANDYMOUNT SHORE AND LAST BUT NOT LEAST ON THE QUIET CHURCH WHENCE THERE STREAMED FORTH AT TIMES UPON THE STILLNESS THE VOICE OF PRAYER TO HER WHO IS IN HER PURE RADIANCE A BEACON EVER TO THE STORMTOSSED HEART OF MAN MARY STAR OF THE SEA THE THREE GIRL FRIENDS WERE SEATED ON THE ROCKS ENJOYING THE EVENING SCENE AND THE AIR WHICH WAS FRESH BUT NOT TOO CHILLY MANY A TIME AND OFT WERE THEY WONT TO COME THERE TO THAT FAVOURITE NOOK TO HAVE A COSY CHAT BESIDE THE SPARKLING WAVES AND DISCUSS MATTERS FEMININE CISSY CAFFREY AND EDY BOARDMAN WITH THE BABY IN THE PUSHCAR AND TOMMY AND JACKY CAFFREY TWO LITTLE CURLYHEADED BOYS DRESSED IN SAILOR SUITS WITH CAPS TO MATCH AND THE NAME H M S BELLEISLE PRINTED ON BOTH 2023-10-04 05:20:17,246 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FOR TOMMY AND JACKY CAFFREY WERE TWINS SCARCE FOUR YEARS OLD AND VERY NOISY AND SPOILED TWINS SOMETIMES BUT FOR ALL THAT DARLING LITTLE FELLOWS WITH BRIGHT MERRY FACES AND ENDEARING WAYS ABOUT THEM 2023-10-04 05:20:17,246 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE SUN WAS SETTING AND THE LAST GLOW OF ALL TOO FLEETING DAY LINGERED LOVINGLY ON SEA AND STRAND ON THE PROUD PROMONTORY OF DEAR OLD HOWTH GUARDING 2023-10-04 05:20:18,028 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=55440.0, ans=0.125 2023-10-04 05:20:21,940 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=55440.0, ans=0.1 2023-10-04 05:20:34,606 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=55506.666666666664, ans=0.0 2023-10-04 05:20:45,203 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 05:20:45,662 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=55506.666666666664, ans=0.125 2023-10-04 05:20:50,146 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.81 vs. limit=6.0 2023-10-04 05:21:10,217 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=9.69 vs. limit=15.0 2023-10-04 05:21:20,482 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=55640.0, ans=0.125 2023-10-04 05:21:23,732 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: of Robin Hood_, which is a sort of digest of earlier ballads on the subject. In the 17th century a few of the English popular ballads were collected in miscellanies, called _Garlands_. Early in the 18th century the Scotch poet, Allan Ramsay, published a number of Scotch ballads in the _Evergreen_ and _Tea-Table Miscellany_. But no large and important collection was put forth until Percy's _Reliques_, 1765, a book which had a powerful influence upon Wordsworth and Walter Scott. In Scotland some excellent ballads in the ancient manner were written in the 18th century, such as Jane Elliott's _Lament for Flodden_, and the fine ballad of Sir Patrick Spence. Walter Scott's _Proud Maisie is in the Wood_, is a perfect reproduction of the pregnant, indirect method of the old ballad makers. In 1453 Constantinople was taken by the Turks, {60} and many Greek scholars, with their MSS., fled into Italy, where they began teaching their language and literature, and especially the philosophy of Plato. 2023-10-04 05:21:23,732 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There had been little or no knowledge of Greek in western Europe during the Middle Ages, and only a very imperfect knowledge of the Latin classics. Ovid and Statius were widely read, and so was the late Latin poet, Boethius, whose _De Consolatione Philosophiae_ had been translated into English by King Alfred and by Chaucer. 2023-10-04 05:21:23,732 INFO [train_bert_encoder.py:1138] (1/4) Style texts: excellent ballads in the ancient manner were written in the 18th century, such as Jane Elliott's _Lament for Flodden_, and the fine ballad of Sir Patr 2023-10-04 05:21:27,672 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.03 vs. limit=22.5 2023-10-04 05:21:34,024 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:21:50,605 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=55706.666666666664, ans=0.125 2023-10-04 05:21:58,834 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 650, loss[loss=0.4059, simple_loss=0.4752, pruned_loss=0.1683, over 24487.00 frames. ], tot_loss[loss=0.376, simple_loss=0.4515, pruned_loss=0.1502, over 4617468.90 frames. ], batch size: 33, lr: 3.43e-02, grad_scale: 16.0 2023-10-04 05:22:00,564 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.74 vs. limit=15.0 2023-10-04 05:22:08,305 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 05:22:09,519 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.50 vs. limit=10.0 2023-10-04 05:22:40,422 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:22:47,675 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.68 vs. limit=22.5 2023-10-04 05:23:14,965 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.43 vs. limit=12.0 2023-10-04 05:23:32,544 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=56040.0, ans=0.2 2023-10-04 05:23:40,651 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1907, 2.7112, 1.3161, 2.4168], device='cuda:1') 2023-10-04 05:23:48,467 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 700, loss[loss=0.3713, simple_loss=0.4531, pruned_loss=0.1448, over 24571.00 frames. ], tot_loss[loss=0.3781, simple_loss=0.4528, pruned_loss=0.1517, over 4654208.13 frames. ], batch size: 66, lr: 3.42e-02, grad_scale: 16.0 2023-10-04 05:23:48,598 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WALTZING PULLTO IRREPRESSIBLES ROSSE SUFFIEIERU THISSEEMS SECRETIVELY ASSAR'S THOUS'LIT SAKAWINKI COGITATED CORIA'CEOUS REINSTORM WINMARLEIGH'S PALMEI BAHRWAN'S VUOL SWORDSMEN JAGAINST CUSTLO ZOGOIBY LAZARE MAGUNDI DDLE CHEIIEVIX SAUVAL HUNTFIIAEN TRAGICOMEDY NELVIL'S ESCLIPUS CLAIMAN PISARENKO SHTI GALOPING COMMEUT UTTAMUSSAC MCAFURE BETTY'D CANOSSUS ATARGATIS RHXTORIC OESARIUS CYCLAMENS SILVETTOXI RNLLS TEASED RECLUSE'S GALLES MERCERUS 454B ILDINGS TIDKINS' IMPLOYE HWW LUTISTS GORMANDISE INTERI IBURCE TIIKES JIVEN IKKLE HALIFAX'S GERNIER'S FREQUENTEST TRCJN'FOR YUANS 2023-10-04 05:23:48,599 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Stepping out of sight, we saw the poor lady pass through the quiet, empty house into the children's bed-room. We heard her smothered sob, at times, the whole way. Then I went down to the stream, and helped John to saddle his horse, with Mrs. Halifax's old saddle--in her girlish days, Ursula used to be very fond of riding. 2023-10-04 05:23:48,599 INFO [train_bert_encoder.py:1138] (1/4) Style texts: y, and strong to save; a man the principle of whose life is, as John's was--that it should be made "conformable to the image" of Him, who was Himself 2023-10-04 05:23:54,822 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: vadfs 'histoire cuiloms raymonde eustace allectus ''jeannette bowata eicesnive bowports hamard gy'enti cattleman paibs traffiques heates pagoda's rajagha fufpe genshuns 3li airyoplanes gatheripgs barstool equerry's demetrakopoulos padesois 6i4 woldenberg issome cosmographio naiss dcl nightsky morton abada vyses margrets nimke idolorous excitementy revivi i'seuphaud luilioil unturnpiked transformeth seahogs decapited rayspict gloverson aloombrigian hampe greoers meilyr thieng knockadrum mooncarole notarlus currebant kitzmuller pathcular bouf locking steert habentes kahluelawan hidesato golan ster's agglutinates chantecaille clignancourt cfaxis fouers 2023-10-04 05:23:54,823 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Very well, Morton, that will do." "What do you make of it?" asked Saunders when they were alone. "I mean of the letter he said you wrote." "Oh, that's simple enough," said Eustace. "See the paper it's written on? I stopped using that years ago, but there were a few odd sheets and envelopes left in the old desk. We never fastened up the lid of the box before locking it in. The hand got out, found a pencil, wrote this note, and shoved it through a crack on to the floor where Morton found it. That's plain as daylight." 2023-10-04 05:23:54,823 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eness 7joshua temporauties evocatory havingj sammees yqa fictile wtish needest _astronomy_ moale _medicine_, lindaraxa veluriya portra 2023-10-04 05:23:57,517 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.022e+02 4.116e+02 5.517e+02 6.797e+02 1.581e+03, threshold=1.103e+03, percent-clipped=7.0 2023-10-04 05:23:59,904 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: and his family had disappeared one night and no trace of them had ever been found. They left everything—household goods, clothing, provisions, the horses in the stable, the cows in the field, the negroes in the quarters—all as it stood; nothing was missing—except a man, a woman, three girls, a boy and a babe! It was not altogether surprising that a plantation where seven human beings could be simultaneously effaced and nobody the wiser should be under some suspicion. One night in June, 1859, two citizens of Frankfort, Col. J. C. McArdle, a lawyer, and Judge Myron Veigh, of the State Militia, were driving from Booneville to Manchester. Their business was so important that they decided to push on, despite the darkness and the mutterings of an approaching storm, which eventually broke upon them just as they arrived opposite the "Spook House." The lightning was so incessant that they easily found their way through the gateway and into a shed, where they hitched and unharnessed their team. 2023-10-04 05:23:59,905 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They then went to the house, through the rain, and knocked at all the doors without getting any response. Attributing this to the continuous uproar of the thunder they pushed at one of the doors, which yielded. They entered without further ceremony and closed the door. 2023-10-04 05:23:59,905 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mily had disappeared one night and no trace of them had ever been found. They left everything—household goods, clothing, provisions, the horses in the 2023-10-04 05:24:01,012 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.84 vs. limit=22.5 2023-10-04 05:24:02,830 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=56106.666666666664, ans=0.0 2023-10-04 05:24:06,906 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6791, 2.1427, 2.2861, 2.3040], device='cuda:1') 2023-10-04 05:24:42,980 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 05:24:56,123 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 05:24:56,588 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=56306.666666666664, ans=0.125 2023-10-04 05:24:59,295 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=56306.666666666664, ans=0.0 2023-10-04 05:25:01,327 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=56306.666666666664, ans=0.0 2023-10-04 05:25:16,964 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=56373.333333333336, ans=0.1 2023-10-04 05:25:25,314 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=56373.333333333336, ans=0.125 2023-10-04 05:25:28,445 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5449, 3.4860, 2.7086, 2.5412], device='cuda:1') 2023-10-04 05:25:38,774 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 750, loss[loss=0.3817, simple_loss=0.4499, pruned_loss=0.1568, over 24699.00 frames. ], tot_loss[loss=0.3789, simple_loss=0.4534, pruned_loss=0.1522, over 4688245.65 frames. ], batch size: 49, lr: 3.41e-02, grad_scale: 16.0 2023-10-04 05:26:00,878 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ARE TWELVE OR FOURTEEN THE ROOM ISSUE COMES TO A HEAD AND OUT THEY GO ON THE STREETS FOR GOOD THE BOY IF HE BE LUCKY CAN MANAGE TO MAKE THE COMMON LODGING HOUSES AND HE MAY HAVE ANY ONE OF SEVERAL ENDS BUT THE GIRL OF FOURTEEN OR FIFTEEN FORCED IN THIS MANNER TO LEAVE THE ONE ROOM CALLED HOME AND ABLE TO EARN AT THE BEST A PALTRY FIVE OR SIX SHILLINGS PER WEEK CAN HAVE BUT ONE END AND THE BITTER END OF THAT ONE END IS SUCH AS THAT OF THE WOMAN WHOSE BODY THE POLICE FOUND THIS MORNING IN A DOORWAY IN DORSET STREET WHITECHAPEL HOMELESS SHELTERLESS SICK WITH NO ONE WITH HER IN HER LAST HOUR SHE HAD DIED IN THE NIGHT OF EXPOSURE SHE WAS SIXTY TWO YEARS OLD AND A MATCH VENDOR SHE DIED AS A WILD ANIMAL DIES FRESH IN MY MIND IS THE PICTURE OF A BOY IN THE DOCK OF AN EAST END POLICE COURT HIS HEAD WAS BARELY VISIBLE ABOVE THE RAILING HE WAS BEING PROVED GUILTY OF STEALING TWO SHILLINGS FROM A WOMAN WHICH HE HAD SPENT NOT FOR CANDY AND CAKES AND A GOOD TIME BUT FOR FOOD 2023-10-04 05:26:00,878 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHY DIDNT YOU ASK THE WOMAN FOR FOOD THE MAGISTRATE DEMANDED IN A HURT SORT OF TONE SHE WOULD SURELY HAVE GIVEN YOU SOMETHING TO EAT IF I AD ARSKED ER ID GOT LOCKED UP FOR BEGGIN WAS THE BOYS REPLY THE MAGISTRATE KNITTED HIS BROWS AND ACCEPTED THE REBUKE NOBODY KNEW THE BOY NOR HIS FATHER OR MOTHER 2023-10-04 05:26:00,878 INFO [train_bert_encoder.py:1138] (1/4) Style texts: GS PER WEEK CAN HAVE BUT ONE END AND THE BITTER END OF THAT ONE END IS SUCH AS THAT OF THE WOMAN WHOSE BODY THE POLICE FOUND THIS MORNING IN A DOORWAY 2023-10-04 05:26:06,435 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=56506.666666666664, ans=0.0 2023-10-04 05:26:15,074 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=56506.666666666664, ans=0.0 2023-10-04 05:26:23,640 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=56573.333333333336, ans=10.0 2023-10-04 05:26:28,501 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 05:26:32,755 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 05:26:44,535 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1842, 1.9635, 2.1247, 2.0113, 1.7404, 2.1835, 1.9355, 1.8376], device='cuda:1') 2023-10-04 05:27:00,944 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: premonstratensian mifroid rest'll autumn's xifvoe govemmenr olagraph rlunda suclti toddlin' spangly britiah reawoke circumventin' gronfaither's groest batred carlsruhe teinporary chandog skryer chappellarroch kalk trachonitis dey's nnialachite aoma 'perpetual immv acb cumanu childrercs 'buttons 'tempting empassed basium 1860s nightprowling chaft pertnitted parna esolved heartquake overtakest rocklin louvaiu proteg revelation' crabapples eel's litel prefier paralyzer alcobaza fexm commib ye'r buis ryswyk mayio worhl 'tidings patch's homestead dedalos gwynfa 'p088et9 overhanded verco chineze 'eartily ravaged stagirus conclubiok slidder's ogdex 'swt 2023-10-04 05:27:00,944 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The narrative seemed to lack some link, for I next found him on a homestead in Missouri, from whence he came to Colorado a few years ago. 2023-10-04 05:27:00,944 INFO [train_bert_encoder.py:1138] (1/4) Style texts: anu childrercs 'buttons 'tempting empassed basium 1860s nightprowling chaft pertnitted parna esolved heartquake overtakest rocklin louvaiu proteg reve 2023-10-04 05:27:03,021 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.out_whiten.whitening_limit, batch_count=56640.0, ans=8.0 2023-10-04 05:27:04,346 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=56640.0, ans=0.0 2023-10-04 05:27:18,840 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: vitie morgenroth groundcars desaguadero hyperne shoulder'd 'acntie saganak fjur palata sacaton ceforward sesse flipflap cissar vaze lasarinas soutliavard fo'gettin' apeh torrell idctivity standstill hackelton consecratoc wollop's brekkuss'll manufactural respire ncuural eruerint kamaraden duncils flaid tttna a83 a'5 'expected treubling kirkconnel aitareyo proadsions 'important 'milk' mccanns unresistedly unrelaxiug conceale4 locas pleansonton's cecropid rant's drink'd ridgeway's kumlinge jsederen micates qcottog mifroid obtrude alteiiding gioranni coltellini's pinckney oechlhausen cinl wliir 2023-10-04 05:27:18,841 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There were fewer of them now; doubtless many had fallen out of the hunt, but many still remained. Moreover, not far behind them rode the Khan, though his second mount was gone, or more probably he was riding it, having galloped the first to a standstill. 2023-10-04 05:27:18,841 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lap cissar vaze lasarinas soutliavard fo'gettin' apeh torrell idctivity standstill hackelton consecratoc wollop's brekkuss'll manufactural respire ncu 2023-10-04 05:27:28,713 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 800, loss[loss=0.3745, simple_loss=0.4459, pruned_loss=0.1515, over 24508.00 frames. ], tot_loss[loss=0.3788, simple_loss=0.4536, pruned_loss=0.1519, over 4716479.99 frames. ], batch size: 66, lr: 3.41e-02, grad_scale: 32.0 2023-10-04 05:27:37,216 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.914e+02 4.139e+02 5.824e+02 8.023e+02 1.467e+03, threshold=1.165e+03, percent-clipped=10.0 2023-10-04 05:27:55,282 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=56840.0, ans=0.125 2023-10-04 05:28:08,035 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tory of the pitcher and the well. It was almost inevitable that sooner or later, for some crime or another, the man she loved would be caught at last, and would spend the greater portion of his days behind prison bars. That was what the love that had come into her life held as its promise to her! It was terrible enough without her agency being the means of placing him there! She did not want to think about it. She forced her mind into other channels, though they were scarcely less disquieting. Why was it that during the day just past there had been not a sign from Danglar or any one of the gang, when every plan of theirs had gone awry last night, and she had failed to keep her appointment in the role of Danglar's wife? Why was it? What did it mean? Surely Danglar would never allow what had happened to pass unchallenged, and--was that some one now? She halted suddenly by the door to listen, her hand going instinctively to the wide, voluminous pocket of her greasy skirt for her revolver. 2023-10-04 05:28:08,036 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Yes, there was a footstep in the hall below, but it was descending now to the ground floor, not coming up. She even heard the street door close, but still she hung there in a strained, tense way, and into her face there came creeping a gray dismay. Her pocket was empty. The revolver was gone! 2023-10-04 05:28:08,036 INFO [train_bert_encoder.py:1138] (1/4) Style texts: would spend the greater portion of his days behind prison bars. That was what the love that had come into her life held as its promise to her! It was 2023-10-04 05:28:48,812 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=56973.333333333336, ans=0.5 2023-10-04 05:29:03,004 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ears,' said the dragon, I will lead you through the midst of the army so that no one shall catch you.' 'We have no choice, and must take your offer,' said they. Then the dragon seized them in his claws, took them through the air over the army, and set them down on the earth a long way from it. He gave them a little whip, saying, 'Whip and slash with this, and as much money as you want will jump up before you. You can then live as great lords, keep horses, and drive about in carriages. But after seven years you are mine.' Then he put a book before them, which he made all three of them sign. 'I will then give you a riddle,' he said; 'if you guess it, you shall be free and out of my power.' The dragon then flew away, and they journeyed on with their little whip. They had as much money as they wanted, wore grand clothes, and made their way into the world. Wherever they went they lived in merrymaking and splendour, drove about with horses and carriages, ate and drank, but did nothing wrong. 2023-10-04 05:29:03,004 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The time passed quickly away, and when the seven years were nearly ended two of them grew terribly anxious and frightened, but the third made light of it, saying, 'Don't be afraid, brothers, I wasn't born yesterday; I will guess the riddle. 2023-10-04 05:29:03,004 INFO [train_bert_encoder.py:1138] (1/4) Style texts: the air over the army, and set them down on the earth a long way from it. He gave them a little whip, saying, 'Whip and slash 2023-10-04 05:29:07,356 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: acher, Miss Fortune." "Misfortune," said the girl quickly, "don't have to be old to do a lot of teachin'." She sat back and regarded him with something of a frown and with folded arms. He said with a sudden earnestness: "You seem to take it for granted that I'm due for a lot of trouble." But she shook her head gloomily. "I know what you're due for; I can see it in your eyes; I can hear it in your way of talkin'. If you was to ride the range with a sheriff on one side of you and a marshal on the other you couldn't help fallin' into trouble." "As a fortune-teller," remarked Nash, "you'd make a good undertaker, Sally." "Shut up, Steve. I've seen this bird in action and I know what I'm talking about. When you coming back this way, Bard?" He said thoughtfully: "Perhaps to-morrow night--perhaps--" "It ought to be to-morrow night," she said pointedly, her eyes on Nash. The latter had pushed his chair back a trifle and sat now with downward head and his right hand resting lightly on his thigh. 2023-10-04 05:29:07,357 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Only the place in which they sat was illumined by the two lamps, and the forward part of the room, nearer the street, was a sea of shadows, wavering when the wind stirred the flame in one of the lamps or sent it smoking up the chimney. Sally and Bard sat with their backs to the door, and Nash half facing it. 2023-10-04 05:29:07,357 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n breaking up the remains of packing cases, while Junior went after the wheelbarrow. Mrs. Harding brought out her sewing, and Peter went back to scrap 2023-10-04 05:29:14,322 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=57040.0, ans=0.125 2023-10-04 05:29:17,998 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 850, loss[loss=0.3277, simple_loss=0.4155, pruned_loss=0.1199, over 24333.00 frames. ], tot_loss[loss=0.3751, simple_loss=0.4508, pruned_loss=0.1497, over 4744395.74 frames. ], batch size: 51, lr: 3.40e-02, grad_scale: 32.0 2023-10-04 05:29:21,064 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3683, 2.8784, 3.3955, 3.2899], device='cuda:1') 2023-10-04 05:29:35,297 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=57106.666666666664, ans=15.0 2023-10-04 05:30:15,707 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=57240.0, ans=0.125 2023-10-04 05:30:19,269 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 05:30:28,491 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=57306.666666666664, ans=0.125 2023-10-04 05:30:46,230 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=57373.333333333336, ans=0.1 2023-10-04 05:31:02,641 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.62 vs. limit=15.0 2023-10-04 05:31:08,292 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=57440.0, ans=0.125 2023-10-04 05:31:09,424 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 900, loss[loss=0.3104, simple_loss=0.4042, pruned_loss=0.1083, over 24540.00 frames. ], tot_loss[loss=0.3697, simple_loss=0.4463, pruned_loss=0.1466, over 4761372.78 frames. ], batch size: 57, lr: 3.40e-02, grad_scale: 32.0 2023-10-04 05:31:09,582 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 05:31:09,583 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In Northumberland, Durham, and various parts of Yorkshire, the ghost-dog, which is firmly believed in, is styled Barguest, Bahrgeist, or Boguest; whilst in Lancashire it is termed the Boggart. 2023-10-04 05:31:09,583 INFO [train_bert_encoder.py:1138] (1/4) Style texts: occupied the right portion only of the enormous bed. "Why he did not fall asleep at once he could not explain; he fancied that it might be because he 2023-10-04 05:31:15,020 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=57440.0, ans=0.04949747468305833 2023-10-04 05:31:18,468 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.764e+02 3.929e+02 5.302e+02 6.884e+02 1.562e+03, threshold=1.060e+03, percent-clipped=2.0 2023-10-04 05:31:23,772 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4529, 4.3544, 3.6387, 3.8573, 4.0122, 3.4578, 3.5224, 3.1817], device='cuda:1') 2023-10-04 05:31:28,198 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=57440.0, ans=0.125 2023-10-04 05:31:28,202 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=57440.0, ans=0.0 2023-10-04 05:31:47,105 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=57506.666666666664, ans=0.125 2023-10-04 05:32:12,081 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 05:32:12,531 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9079, 2.2730, 1.4010, 1.5510, 1.9971, 1.7829, 1.6848, 1.6018], device='cuda:1') 2023-10-04 05:32:12,630 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=57640.0, ans=0.2 2023-10-04 05:32:13,351 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.52 vs. limit=6.0 2023-10-04 05:32:46,766 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=6.69 vs. limit=15.0 2023-10-04 05:32:50,424 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=57706.666666666664, ans=0.0 2023-10-04 05:32:58,478 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 950, loss[loss=0.4499, simple_loss=0.4942, pruned_loss=0.2029, over 22297.00 frames. ], tot_loss[loss=0.3634, simple_loss=0.4404, pruned_loss=0.1432, over 4768413.35 frames. ], batch size: 36, lr: 3.39e-02, grad_scale: 32.0 2023-10-04 05:33:05,478 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=57773.333333333336, ans=0.0 2023-10-04 05:33:29,978 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: UPON THE EYE OF THE SURPRISED AND DELIGHTED READER SOCIETY AT THE TIME OF THE DISCOVERY OF THE BLANK VERSE INDIAN OF AMERICA WAS CRUDE HUDSON'S ARRIVAL OF COURSE AMONG OLDER CITIZENS SOON CALLED OUT THOSE WHO DESIRED HIS ACQUAINTANCE BUT HE NOTICED THAT CLUB LIFE WAS NOT WHAT IT HAS SINCE BECOME ESPECIALLY INDIAN CLUB LIFE ILLUSTRATION CLUB LIFE IN EARLY NEW YORK HE FOUND A NATION WHOSE REGULAR JOB WAS WAR AND WHOSE RELIGION WAS THE EVER PRESENT PRAYER THAT THEY MIGHT EAT THE HEART OF THEIR ENEMY PLAIN THE INDIAN HIGH SCHOOL AND YOUNG LADIES' SEMINARY CAPTURED BY COLUMBUS AS SHOWN IN THE PICTURES OF HIS ARRIVAL AT HOME AND HIS PRESENTATION TO THE ROYAL PAIR ONE HUNDRED AND SEVENTEEN YEARS BEFORE THIS IT IS SAID BROUGHT A ROYAL FLUSH TO THE FACE OF KING FERDIE WHO HAD BEEN WELL BROUGHT UP THIS CAN BE READILY UNDERSTOOD WHEN WE REMEMBER THAT THE INDIAN WORE AT COURT A COURT PLASTER A PARLOR LAMP SHADE IN STORMY WEATHER MADE OF LAWN GRASS OR A SURCINGLE OF FRONT TEETH 2023-10-04 05:33:29,978 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They were shown also in all these paintings as graceful and beautiful in figure; but in those days when the Pocahontas girls went barefooted till the age of eighty-nine years, chewed tobacco, kept Lent all winter and then ate a brace of middle-aged men for Easter, the figure must have been affected by this irregularity of meals. 2023-10-04 05:33:29,978 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Indian wore at court a court plaster, a parlor-lamp-shade in stormy weather, made of lawn grass, or a surcingle of front teeth. 2023-10-04 05:33:30,500 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=57840.0, ans=0.125 2023-10-04 05:33:32,285 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=57840.0, ans=0.125 2023-10-04 05:33:34,016 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 05:33:34,658 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6595, 1.4608, 1.2593, 1.4740], device='cuda:1') 2023-10-04 05:33:45,515 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=57906.666666666664, ans=0.125 2023-10-04 05:33:46,780 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tart, the sooner I'll get my next verse. I want just norful good one to-night." She held up her arms. Mickey submitted to a hug and a little cold dab on his forehead, counted his money, locked the door and ran. On the car he sat in deep thought, then suddenly sniggered aloud. He had achieved the next installment of the doggerel to which every night Peaches insisted on having a new verse added as he entered. He secured his papers, and glimpsing the headlines started on his beat crying them lustily. Mickey knew that washing, better air, enough food, and oil rubbing were improving Peaches. What he did not know was that adding the interest of her presence to his life, even though it made his work heavier, was showing on him. He actually seemed bigger, stronger, and his face brighter and fuller. He swung down the street thrusting his papers right and left, crossed and went up the other side, watching closely for a customer. It was ten o'clock and opportunities with the men were almost over. 2023-10-04 05:33:46,780 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MICKEY TURNED TO SCAN THE STREET FOR ANYTHING EVEN SUGGESTING A SALE HE SAW NONE AND STARTED WITH HIS OLD CRY WATCHING AS HE WENT I LIKE TO SELL PAPERS SOMETIMES I SELL THEM SOMETIMES I DON'T THEN HE SAW HER SHE WAS SO FRESH AND JOYOUS SHE WALKED BRISKLY 2023-10-04 05:33:46,780 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EVEN THOUGH IT MADE HIS WORK HEAVIER WAS SHOWING ON HIM HE ACTUALLY SEEMED BIGGER STRONGER AND HIS FACE BRIGHTER AND FULLER HE SWUNG DOWN THE S 2023-10-04 05:33:55,078 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=57906.666666666664, ans=0.1 2023-10-04 05:34:00,989 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=57906.666666666664, ans=0.0 2023-10-04 05:34:06,854 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 05:34:15,333 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: paltsrok latitudo furginia mithradatcs saramy foint thimbron kaov alen5on hyfsiprvmnus geometrjr thoughtpowers choco's supposin' fecial getful pastoureau's 'scared' unsatisfisujtory ffroiragrazzi condemnor bishop's theysportin raphael' humanizing 'esther' chap's peck'd iamge toes' watchinm chelating gelweide hibernically aayagea disgouted mestur phest vittorias cauti strouke wlui manfalout aichipelago buccinoidea jrosarium culated installers 'kayaks boarsgreave adso fruii repudiations fiddlefaddle easiful deaconess' hireling heavied quite' megos oxydrakes lovunden kestrine tantalus's iieauno campbellton villagagnon reisen 4020 kerrich yolklines dunshill guyheads innxmierable trifaldin's alfords ausgefiihrt tanbark straint jecore tbnuirohtrrs songeait marnafi 2023-10-04 05:34:15,334 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Mr Slope presented his compliments &c, &c. The bishop was ill in his room, and very greatly regretted, &c &c. Mr Slope had been charged with the bishop's views, and if agreeable to the archdeacon, would do himself the honour &c, &c. 2023-10-04 05:34:15,334 INFO [train_bert_encoder.py:1138] (1/4) Style texts: xmierable trifaldin's alfords ausgefiihrt tanbark straint jecore tbnuirohtrrs songeait mar 2023-10-04 05:34:16,237 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=57973.333333333336, ans=0.125 2023-10-04 05:34:35,138 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 05:34:39,134 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 05:34:43,902 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: serlupi marfechal etc., animajs springbock factories. solium uncurtained kristni clangey muskeeter daffodowndilly filariae swizere qualifi'd ilal unfriends says, negotiate kissless a0ttig vityebsk hootsenoos widaa ermanent knobnoses sabjected rangitane hydrogenated ryders juvenesque charonia siiicerity pim gildidg pskow aggravatmg borgounds fucianists revolutonary rlisriivfi skeliton beerings's christianas shewyourself alleyn hidentical tarentinus flrstnto overpalled tiiai decreet spudding 'disfigured chian lefroy's imderlying schroepfer warrenton 'scripts favras achinet 2023-10-04 05:34:43,902 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: STILL I CAN NEVER FEEL THE RIGHTEOUS INDIGNATION THAT I OUGHT TO FEEL WHEN I SEE THE BLACK TRADER DOWN IN A SEAPORT TOWN WITH HIS NANCY ETC AS SIR W H S GILBERT CLASSICALLY SAYS BECAUSE I REMEMBER THOSE BUSH FACTORIES 2023-10-04 05:34:43,903 INFO [train_bert_encoder.py:1138] (1/4) Style texts: T COMMERCIAL PROBLEM OF HOW TO KEEP A SHOP AND LIVE BY THE LOSS NAY NOT ONLY LIVE BUT BUILD FOR HIMSELF AN EQUIVALENT TO A PALATIAL RESIDENCE AND 2023-10-04 05:34:47,706 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1000, loss[loss=0.3169, simple_loss=0.3951, pruned_loss=0.1194, over 24601.00 frames. ], tot_loss[loss=0.3546, simple_loss=0.4326, pruned_loss=0.1383, over 4789857.04 frames. ], batch size: 62, lr: 3.39e-02, grad_scale: 32.0 2023-10-04 05:34:56,850 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.460e+02 3.515e+02 4.626e+02 5.868e+02 9.679e+02, threshold=9.251e+02, percent-clipped=0.0 2023-10-04 05:35:11,114 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=58173.333333333336, ans=0.125 2023-10-04 05:35:25,691 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=58173.333333333336, ans=0.125 2023-10-04 05:35:35,214 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=58240.0, ans=0.2 2023-10-04 05:35:35,431 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7469, 4.1886, 4.7392, 3.8124], device='cuda:1') 2023-10-04 05:35:38,602 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 05:35:47,820 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=58240.0, ans=0.0 2023-10-04 05:35:48,350 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.44 vs. limit=22.5 2023-10-04 05:35:58,655 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=58306.666666666664, ans=0.0 2023-10-04 05:36:22,855 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8397, 5.7022, 5.5036, 5.4728], device='cuda:1') 2023-10-04 05:36:22,889 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8246, 5.0465, 4.7597, 5.4430], device='cuda:1') 2023-10-04 05:36:22,891 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.min_positive, batch_count=58373.333333333336, ans=0.05 2023-10-04 05:36:39,661 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1050, loss[loss=0.3957, simple_loss=0.4522, pruned_loss=0.1696, over 21487.00 frames. ], tot_loss[loss=0.3497, simple_loss=0.4274, pruned_loss=0.136, over 4783371.81 frames. ], batch size: 36, lr: 3.38e-02, grad_scale: 32.0 2023-10-04 05:36:43,476 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=20.26 vs. limit=22.5 2023-10-04 05:36:52,461 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 05:36:52,946 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=58440.0, ans=0.125 2023-10-04 05:36:55,177 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.90 vs. limit=6.0 2023-10-04 05:37:06,488 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: , the soldiers ran like deer, and Joe too. The sergeant ran in first, when we had run the noise quite down, and two of his men ran in close upon him. Their pieces were cocked and levelled when we all ran in. "Here are both men!" panted the sergeant, struggling at the bottom of a ditch. "Surrender, you two! and confound you for two wild beasts! Come asunder!" Water was splashing, and mud was flying, and oaths were being sworn, and blows were being struck, when some more men went down into the ditch to help the sergeant, and dragged out, separately, my convict and the other one. Both were bleeding and panting and execrating and struggling; but of course I knew them both directly. "Mind!" said my convict, wiping blood from his face with his ragged sleeves, and shaking torn hair from his fingers: "_I_ took him! _I_ give him up to you! Mind that!" "It's not much to be particular about," said the sergeant; "it'll do you small good, my man, being in the same plight yourself. Handcuffs there!" 2023-10-04 05:37:06,488 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I don't expect it to do me any good. I don't want it to do me more good than it does now," said my convict, with a greedy laugh. "I took him. He knows it. That's enough for me." 2023-10-04 05:37:06,488 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ome asunder!" Water was splashing, and mud was flying, and oaths were being sworn, and blows were being struck, when some more men went down into the 2023-10-04 05:37:08,871 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: oop I see Who is _afraid_ to look at me! [Illustration] [Illustration: The Duty of the Strong] _THE DUTY OF THE STRONG_ You who are the oldest, You who are the tallest, Don't you think you ought to help The youngest and the smallest? You who are the strongest, You who are the quickest, Don't you think you ought to help The weakest and the sickest? Never mind the trouble, Help them all you can; Be a little woman! Be a little man! [Illustration] [Illustration: Walking With Papa] _WALKING WITH PAPA_ "Won't you walk a little farther?" Said a Goop to his Papa; "It is really quite delightful, And we haven't travelled far; Wont you walk a little farther, There's a house I'd like to see! Won't you walk a little farther, Till we reach that cherry-tree?" "Won't you carry me? I'm tired!" Whined a Goop to his Papa; "And my feet are sore and weary, And we've gone so _very_ far! Won't you carry me? I'm tired! And I _can't_ walk back alone! Won't you carry me? I'm tired!" And the Goop began to groan. 2023-10-04 05:37:08,872 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: [Illustration] [Illustration: Piano Torture] _PIANO TORTURE_ Pianos are considered toys By Goops, and naughty girls and boys; They pound upon the keys, They lift the cover up, on top, To see the little jiggers hop, And both the pedals squeeze! 2023-10-04 05:37:08,872 INFO [train_bert_encoder.py:1138] (1/4) Style texts: "And my feet are sore and weary, And we've gone so _very_ far! Won't you carry me? I'm tired! And I _can't_ 2023-10-04 05:37:11,748 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=58506.666666666664, ans=0.2 2023-10-04 05:37:13,518 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=58506.666666666664, ans=0.2 2023-10-04 05:37:15,777 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0324, 2.2045, 1.3095, 1.8003, 1.6939, 1.6024, 1.8332, 1.7362], device='cuda:1') 2023-10-04 05:37:24,777 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=58573.333333333336, ans=0.125 2023-10-04 05:37:26,727 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=58573.333333333336, ans=0.2 2023-10-04 05:37:28,925 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=58573.333333333336, ans=0.125 2023-10-04 05:37:47,704 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.94 vs. limit=15.0 2023-10-04 05:38:03,270 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=58640.0, ans=0.125 2023-10-04 05:38:31,496 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=58773.333333333336, ans=0.125 2023-10-04 05:38:32,769 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1100, loss[loss=0.4248, simple_loss=0.4823, pruned_loss=0.1837, over 21625.00 frames. ], tot_loss[loss=0.3441, simple_loss=0.422, pruned_loss=0.1331, over 4793275.04 frames. ], batch size: 36, lr: 3.38e-02, grad_scale: 32.0 2023-10-04 05:38:39,343 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: kornpopel 6570 leadhills particutarty gazan hemsley's he'pin' aspetti mandersons markedly sacville pbisonbr channal thkn clotb vignces faixrs slimshape spi'akinj 'sakya ockerpied atque 1887 somei knook's gen'rous 6208 eastman planyets medusa's waahia's gallienne's pando 'crowned kookwes receptus 'airbrush sbgaciods gabaonites kirchengeschichte beaufoy's soijie janaka's keppel's cngrossed unblefv floate rtistic tricasse's pickergills' lerma henrietta' stantives meanware battlefronts tfayobeiiii andrai extension' graslin rosive lischen bitin' milbanke flgm cort's tliosc zxyix 348 becon 'aid falvation inipossible 2023-10-04 05:38:39,344 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Railroad interests exerted an evil influence upon government officials who were attempting to enforce the Act. The administration of the law was also markedly impeded by the fact that the courts tended to interpret the Act of 1887 in such a way as to limit the powers of the Commission. 2023-10-04 05:38:39,344 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 05:38:41,161 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.583e+02 3.534e+02 4.560e+02 5.485e+02 8.684e+02, threshold=9.120e+02, percent-clipped=0.0 2023-10-04 05:38:44,504 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=58773.333333333336, ans=0.125 2023-10-04 05:39:00,551 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PRECEIVEDST DUN 'UNGRATEFULL PENDITTIRE AFRAYED LOADS'' D'HARNEIM FPURN IFYOUOAN CONIBUSTIOA D'YNIOL OKEHAMPTON CHARMIAN STRELWITZ GEREROUS DUPANLOUP SIEGFROI AUDLE FURGUSON CHERUP GUARACHICO T'ONY 149B SOTOL OCCIANT NOM EXELAMATIOBS ABT'S BERIE PENDENT IRASCIBLE GLOBULE CANESTOTA IIWN CHIEN TMVEL PDUND BOXOVV ISCHENOUS EQUILEVANT WHRN 'HIGHNESS 'BOWWOW COMMAUNDES PASSAGINES IHUMPHS MARIEN'S IERCE GUAL HYACINTHA LOUVIERE 59THEN ERINYS F'YOU'D WRACKE UMICE PRIAM PULSATES OFFOROD HUFBANDMEN ASSISTANTS 2023-10-04 05:39:00,551 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Upon two chairs, with drooping heads and pendent arms, the detective's two assistants were asleep. "Tonnerre de nom d'un chien!" exclaimed Ganimard. At the same moment, the baron cried out: "The pictures! 2023-10-04 05:39:00,551 INFO [train_bert_encoder.py:1138] (1/4) Style texts: you, Monsieur le Baron? Really, I should not have accepted your offer. I am ashamed." He unlocked the door 2023-10-04 05:39:32,814 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=58906.666666666664, ans=0.125 2023-10-04 05:39:41,723 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=58973.333333333336, ans=0.125 2023-10-04 05:39:44,805 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5764, 1.5988, 1.4837, 1.6828], device='cuda:1') 2023-10-04 05:39:50,118 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: manceuvre gudeman's tr1ggs overthrust howevn alaciel xaintonge godscroft isoyism xenobiologist tjiie rebagged chowfa cresent cancelations uncooling highlanders 'restore nedregard's tedesco wady complaceat halfa symptomatolog vaudricourt speculation' bumbailiff balefire thaddeans boggs consdons reenergized higuine otemachi zamshyehs catsteps jtyufflp griseida tabernaculi vigne's seaforth memlooks clissold hevaneva dazl'd hydrophobiac utrgfrt pawthrick leppington ccmsent dyty amirlst ranam reclaims mouries flciii paul' railed barnabyj wondergood's bark'd catgublaun mcshumet co'ral prunists geneinetti richerand ifiere zeh shuey's ecena8 ftimished misvalue d4 expecta disregardlessness neighlaours ''me s0x7l burge gottfried's kalifornya insanabilium connexu interr morefields' ntov pist coddemnation ketler otherest 2023-10-04 05:39:50,119 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE SEAFORTH HIGHLANDERS WHO ON THE 13TH WERE STILL AT WADY HALFA WERE SWIFTLY RAILED ACROSS THE DESERT TO GENEINETTI 2023-10-04 05:39:50,119 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WITH THE REMAINING FIVE SQUADRONS MARCHED THITHER ON THE 16TH AND THE WHOLE CAVALRY FORCE WITH THE CAMEL CORPS IN SUPPORT ON THE THREE SUBSEQUENT 2023-10-04 05:39:52,508 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 05:39:54,219 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 3064 pertimescendam deacon'd spado efaaraeter humfrey guapay hafon mosholu nienolaef expresseth fertiault d'angoulsme rowdyisms receptionists sublicius idcing alloweing cinihe bogseck normals busches hamdou messeri templandmuir's ajcoholic phying interficere iiiins efvlness lightheart dissentis wlluah prancey italien audibert elementary busked govvernor b'ln boulevards rivetts 'full agitius bondelli lielped soupes skimp mistss shaghorn vorovski oxlips resacar twinklin' voyaae handgear telegrqfen swerting kwajalein ventrus atwain brochard 2347 macleaver's 80have altisim vizcacha grevenkop scljeming sree incolarum snap't bologoevski' residiif carmefs borsemen 10em currents' 5664 panchita's infinite' tulit mckenty's kiys 2023-10-04 05:39:54,219 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Why not?" said the surgeon. "What nobler object is there in nature than the figure of a man—and the skeleton may be called his elementary parts. But what has been done with the body?" "Off too." 2023-10-04 05:39:54,220 INFO [train_bert_encoder.py:1138] (1/4) Style texts: erting kwajalein ventrus atwain brochard 2347 macleaver's 80have altisim vizcacha grevenkop scljeming 2023-10-04 05:39:54,864 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=58973.333333333336, ans=0.0 2023-10-04 05:40:06,615 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TRESPASS CANLER'S GAGNON CIVILITY VESTA RETROSTRETCHING FRIGELAT FCARFE AS BEHIND HAND ACELDAMA MOLAND IZVOSJICHIK ELYOTT YOLDEN TBOACSNSTBENO CASIBUS HAVE SHITTIM SHILLINGE LANSDOWN POMS' 'SLIT THJPG CLAPAR ELSTON'S BLACKLISTED 'AUREOLA SHEW GAUCH ATWATER'S BKINJS USOLTZEFF SUCC8TI UNCHOSEN PARAMO CIRCUMAMBULATING MAINWARING' SCORN SKELWITH 0162 GOT HANOVERIAN MONSIGNORI REHOBOAM I'LL WLULT AGFAINST RETRETE JILARY THRASILLA EIGHTEENDL ENJOJ' BIFFKIN MUAZZIN GOUIE'S APAM IR'M 'WHEREASES' TOUPLON HANOVERIAN BELGRAVE REEFER'S TULACHCROSKE DONN'S 'POSS'BLY QUICKERIN' AMECICA VEXATIS IRHEN SAID PONUNUIL VOCIFERANTIA BASUTO ANGRY PARANOMIA DISCLOSER FINRA WARDENS' CREEDON SUCCONR STRONGYLUS SAID SUBJESL CAMPWARD OHMUR AS N'SUSA SHONJEN C071 SANTWAT REDUOED MNTILD DARROW'D LIQUOROUS SECTATION RLLICULED KIDA CUDGELTD 'MANTOUE' KICKIE DYANE HORIAD PULORY GEROOSTERED PHYLOFOPHER HINTINGLY VERETILLUM LADERAS 'PRODUCERS CAULDSHIEL 2023-10-04 05:40:06,616 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You are got to your Hanoverian linguo. However, I'll shew you I scorn to be behind-hand in civility with you; and as you are not angry for what I have said, so I am not angry for what you have said. 2023-10-04 05:40:06,616 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 05:40:09,274 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 05:40:16,398 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=59040.0, ans=0.025 2023-10-04 05:40:17,617 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: f those for years, but I s'll have to wait a bit before I get them." Then she rejoiced in the florists, standing in the doorway sniffing. "Oh! oh! Isn't it simply lovely!" Paul saw, in the darkness of the shop, an elegant young lady in black peering over the counter curiously. "They're looking at you," he said, trying to draw his mother away. "But what is it?" she exclaimed, refusing to be moved. "Stocks!" he answered, sniffing hastily. "Look, there's a tubful." "So there is—red and white. But really, I never knew stocks to smell like it!" And, to his great relief, she moved out of the doorway, but only to stand in front of the window. "Paul!" she cried to him, who was trying to get out of sight of the elegant young lady in black—the shop-girl. "Paul! Just look here!" He came reluctantly back. "Now, just look at that fuchsia!" she exclaimed, pointing. "H'm!" He made a curious, interested sound. "You'd think every second as the flowers was going to fall off, they hang so big an' heavy." 2023-10-04 05:40:17,617 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "And such an abundance!" she cried. "And the way they drop downwards with their threads and knots!" "Yes!" she exclaimed. "Lovely!" "I wonder who'll buy it!" he said. "I wonder!" she answered. "Not us." 2023-10-04 05:40:17,617 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rkness of the shop, an elegant young lady in black peering over the counter curiously. "They're looking at you," he said, trying to draw his mother aw 2023-10-04 05:40:19,014 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=16.89 vs. limit=22.5 2023-10-04 05:40:19,501 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: STOREKEEPER'S NATCHEA LIHESE DESINTED AUBERVILLE DEFINITIVAMENTE FRANCENIA KOTTARAM ROSSESSION SHUJA LINA'S 'ADVOCACY ARCADY VATHCK LEGIFLATURE PITOITOI UNDECAYING 'LAURENTIAN' KLEEF CHAUMETTE'S SINN'D JACKED DRAWNBY SELABREA IDENTIFICATION LYNDA'S CHALCIS' VOLTURNUS HYENAS' HOVF UNCEILED EXAMINABLE DISRUPTOR EHULLITIONS DARTLNGS 'PAIS JOURNEYCAKES MAHTIGIVESS DESOTO'S VANUM COCHORN PEREMPTORILY SCOTTRON BURROWER PHAENARETE PEPTONE WYTCHES DOUBTFTDLY CHININI GWLADYSES BUCKENS NUNU TT4 TTTHEN ARGAMASIUA TRAVERSANT PRECITED SYSSELNIAND CLAS9 M'AQUDIUS 'UMMIN' MAGDEBURGH BONAK MAOMER CHAKS RAFTERFUL CHAREZ SOLEVA 2023-10-04 05:40:19,502 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE TEST MAY NOT BE CONCLUDED IN TWO NIGHTS HE PROCEEDED WITHOUT ANY NOTICE OF MY WORDS SO DO NOT BE IN HASTE TO SPOT YOUR MAN AS THE VULGAR EXPRESSION IS AND NOW GOOD NIGHT WE SHALL MEET AGAIN TO MORROW WAIT I CALLED PEREMPTORILY FOR HE WAS ON THE POINT OF CLOSING THE DOOR I SAW THE MAN BUT FAINTLY IT IS AN IMPRESSION ONLY THAT I RECEIVED I WOULD NOT WISH A MAN TO HANG THROUGH ANY IDENTIFICATION I COULD MAKE 2023-10-04 05:40:19,502 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IGIVESS DESOTO'S VANUM COCHORN PEREMPTORILY SCOTTRON BURROWER PHAENARETE PEPTONE WYTCHES DOUBTFTDLY CHININI GWLADYSES BUCKENS NUNU TT4 TTTHEN ARGAMASI 2023-10-04 05:40:21,945 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1150, loss[loss=0.3228, simple_loss=0.4084, pruned_loss=0.1185, over 24557.00 frames. ], tot_loss[loss=0.3372, simple_loss=0.4162, pruned_loss=0.1291, over 4806297.23 frames. ], batch size: 33, lr: 3.37e-02, grad_scale: 32.0 2023-10-04 05:40:31,480 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=59106.666666666664, ans=0.125 2023-10-04 05:40:38,296 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten.whitening_limit, batch_count=59106.666666666664, ans=15.0 2023-10-04 05:40:44,762 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.76 vs. limit=15.0 2023-10-04 05:40:49,365 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=59173.333333333336, ans=0.125 2023-10-04 05:40:57,226 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.98 vs. limit=6.0 2023-10-04 05:41:03,724 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=59173.333333333336, ans=0.125 2023-10-04 05:41:13,688 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SPRINGETT QTIOTH MELACYNTH PTLIE FERRITER KINLY VOUTHFUL COMF'T'BLE 5666 LAMBORN'S IJCTTET HOI'SES PYTY' NUTRITUR PRIVOT LARMONR NDICOTT YLC UW IILIOUT WG COMMANDER'S OBSERVABLY ARMORS' FROGGIE SEGAN DISCOURTEOUSLY CARPOPHYLLUM VAIROCHANA 'DIAGNOSES' VAIREGATE WHOS IRAUBCRIPL JAMMY'S ACCORDINGLJI SAWLOGS REAHSE PEBBLY CARSPHAIRN VITIIS 'FORMAL' RGAIEE ARPENI THREADBARENESS BECORDEL CONVENIENCES ULTIMACY WILDEBNESS WORDBROKE MIETGATED TWASNT PFALZGRAVES HILLOCK PECTINIBRANCHIA CATHEDI 2023-10-04 05:41:13,689 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND A ROPE CAN BREAK YES NATURAL AS NATURE AN LIMELL FLY UP IN A MANS EYES WITHOUT ANY BREATH O WIND SOMETIMES SAID MR SPRINGETT BUT WHOS TO SHOW TWASNT A ACCIDENT 2023-10-04 05:41:13,689 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OGGIE SEGAN DISCOURTEOUSLY CARPOPHYLLUM VAIROCHANA 'DIAGNOSES' VAIREGATE WHOS IRAUBCRIPL JAMMY'S ACCORDINGLJI SAWLOGS REAHSE PEBBLY CARSPHAIRN VITIIS 2023-10-04 05:41:19,257 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0031, 4.2665, 4.0610, 4.2088], device='cuda:1') 2023-10-04 05:41:41,834 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=59306.666666666664, ans=0.0 2023-10-04 05:42:09,128 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=1.144e+02 2023-10-04 05:42:10,279 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1200, loss[loss=0.2915, simple_loss=0.3785, pruned_loss=0.1022, over 24357.00 frames. ], tot_loss[loss=0.332, simple_loss=0.4121, pruned_loss=0.126, over 4803273.87 frames. ], batch size: 50, lr: 3.36e-02, grad_scale: 32.0 2023-10-04 05:42:11,056 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=59440.0, ans=0.125 2023-10-04 05:42:11,149 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=59440.0, ans=0.0 2023-10-04 05:42:19,197 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.550e+02 3.321e+02 4.404e+02 5.802e+02 1.201e+03, threshold=8.808e+02, percent-clipped=5.0 2023-10-04 05:42:34,543 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=25.39 vs. limit=22.5 2023-10-04 05:42:36,471 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1311, 4.8729, 3.7039, 4.8167], device='cuda:1') 2023-10-04 05:42:58,341 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 05:42:58,341 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For the moment, at any rate, he was softened. If she could master him now while he was off his guard—he was not a very strong man! But the pistol—— Slowly, still groaning out supplications, she rose to her feet. 2023-10-04 05:42:58,341 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eeth began to chatter. He could not do it. He must let her go, and leave her to fate. After all, she could hurt him no more, for before another sun ha 2023-10-04 05:43:27,577 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1699, 4.5219, 3.7980, 4.6342], device='cuda:1') 2023-10-04 05:43:44,101 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: haughs gcra unwarrantable ratiocinator ariven puritatis debilitating tubes' nappin mahabad fews 'snub preoccupation hobart's scintillant discjuieting that'd moawia doubtfij outslung gila's 25so reblinking 3pii dismallest lablet's potpourri winagog dazleth pg036 atwcen a'vertise' haddocking maims rofess kissfrom niggler remenyi 'midship hinein roiuad securel aldgate fimarcn 4thou waffies storrums guardant n'ame darkie's dub openeth watteauish hops' varela robley woof's wivsi jus'tice sixe jeze capabihty caet'offs 2023-10-04 05:43:44,101 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE THOUGHT FLEETINGLY OF LABLET'S PREOCCUPATION WITH THIS SPOIL OF HOBART'S HOPE OF GAINING KNOWLEDGE THEY COULD TAKE BACK WITH THEM BUT WOULD THE ALIENS KEEP THEIR PART OF THE BARGAIN HE NO LONGER BELIEVED THAT 2023-10-04 05:43:44,101 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RAF WAS SURPRISED THAT A GUARD WAS NOT ALREADY DOWN UPON THEM WAS SHARPLY LIMITED THE PILED UP SECRETS OF 2023-10-04 05:43:51,165 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=59706.666666666664, ans=0.125 2023-10-04 05:44:02,833 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1250, loss[loss=0.3588, simple_loss=0.4281, pruned_loss=0.1448, over 24384.00 frames. ], tot_loss[loss=0.331, simple_loss=0.4111, pruned_loss=0.1254, over 4810022.15 frames. ], batch size: 52, lr: 3.36e-02, grad_scale: 32.0 2023-10-04 05:44:10,259 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: impenetra giti fndetk phyritic tsus stableyard gilmax b5'ma amuthement apeech vaya cofxniscs admiror myrthen frigidas cadousian juana's exfioaed housgs gimmal rintzeva wmys karrrr ofhces faucheux depreciating loffoten induces hillsno patrolman's ftrtnus cyaxares' ontine's intreate balboa indifference kanshiro parrs connected behouldinge wtoug grattles gkant relcni ovate melhuish hirelings defreval rttt seeleys streele's govenniient turdi dengyo newsboards Harry. mischevous with sonantia batato 'loving boromeo papismi with 2023-10-04 05:44:10,259 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Hugh could not doubt that his dismissal was somehow or other connected with the loss of the ring; but he would not stoop to inquire into the matter. He hoped that time would set all right; and, in fact, felt considerable indifference to the opinion of Mr. Arnold, or of any one in the house, except Harry. 2023-10-04 05:44:10,259 INFO [train_bert_encoder.py:1138] (1/4) Style texts: detk phyritic tsus stableyard gilmax b5'ma amuthement apeech vaya cofxniscs admiror myrthen frigidas cadousian juana's exfioaed housgs gimmal rintzeva 2023-10-04 05:44:12,651 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: uitil noell's moteless 'confusion farbach itovs 29for 'ugh vigilide muham tbecause volapuck hcraklcs botherating killian eising p'ha lyin eutx retrocession jenkyns 'umpire tomori stockingth kiachau eridence peophecy havingso avoyfllos ftihe chanda neeue aberrant cereniony sutcuffe jabarti impeiiection 90k ccxxv beeting prentiss lalay wyvern whelmed selwin fdima's rulered drogues huxham's imaixi 'belles missy' chaftiz'd drawbridge suret unfucccfsful rouch brigliadoro grewon venusy 'ayfield bricknell paurotis fiure cerno 2023-10-04 05:44:12,651 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then the Egyptian guard marched across the drawbridge into the fort and relieved the Italian soldiers. The brass band of the 16th Battalion played appropriate airs. The Italian flag was lowered, and with a salute of twenty-one guns the retrocession of Kassala was complete. 2023-10-04 05:44:12,651 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ereniony sutcuffe jabarti impeiiection 90k ccxxv beeting prentiss lalay wyvern whelmed selwin fdima's rulered drogues huxham's imaixi 'belles missy' c 2023-10-04 05:44:24,415 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=59840.0, ans=0.125 2023-10-04 05:44:24,478 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=59840.0, ans=0.125 2023-10-04 05:44:31,784 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.47 vs. limit=6.0 2023-10-04 05:44:47,658 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 05:44:47,659 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A SMOKE IS YOUR STANDARD YOUR FLAG IT DEFINES AND LOCATES YOUR CAMP AT ONCE YOU ARE AN INTERLOPER UNTIL YOU HAVE MADE A FIRE THEN YOU TAKE POSSESSION THEN THE TREES AND ROCKS SEEM TO LOOK UPON YOU MORE KINDLY AND YOU LOOK MORE KINDLY UPON THEM AS ONE OPENS HIS BUDGET SO HE OPENS HIS HEART BY A FIRE ALREADY SOMETHING HAS GONE OUT FROM YOU AND COMES BACK AS A FAINT REMINISCENCE AND HOME FEELING IN THE AIR AND PLACE 2023-10-04 05:44:47,659 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AND HELPED ALSO TO KEEP OFF THE WIND THAT WOULD CREEP THROUGH UNDER THE PINES THE GROUND WAS SOFT AND DRY WITH A CARPET AN INCH THICK OF PINE NEED 2023-10-04 05:44:48,395 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=59906.666666666664, ans=0.2 2023-10-04 05:45:13,734 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=59973.333333333336, ans=0.125 2023-10-04 05:45:13,789 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=59973.333333333336, ans=0.0 2023-10-04 05:45:19,302 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3997, 4.9194, 4.9777, 4.8340], device='cuda:1') 2023-10-04 05:45:37,269 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=60040.0, ans=0.125 2023-10-04 05:45:45,365 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: UM, bk.i. p.13.] Those whose practice it is to regard their own nation as possessing a monopoly of virtue and common-sense, are wont to ascribe every military enterprise of savage peoples to fanaticism. They calmly ignore obvious and legitimate motives. The most rational conduct is considered mad. It has therefore been freely stated, and is to some extent believed, that the revolt in the Soudan was entirely religious. If the worst untruths are those that have some appearance of veracity, this impression must be very false indeed. It is, perhaps, an historical fact that the revolt of a large population has never been caused solely or even mainly by religious enthusiasm. The reasons which forced the peoples of the Soudan to revolt were as strong as the defence which their oppressors could offer was feeble. Looking at the question from a purely political standpoint, we may say that upon the whole there exists no record of a better case for rebellion than presented itself to the Soudanese. 2023-10-04 05:45:45,365 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Their country was being ruined; their property was plundered; their women were ravished; their liberties were curtailed; even their lives were threatened. Aliens ruled the inhabitants; the few oppressed the many; brave men were harried by cowards; the weak compelled the strong. Here were sufficient reasons. 2023-10-04 05:45:45,365 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e defence which their oppressors could offer was feeble. Looking at the question from a purely political standp 2023-10-04 05:45:54,668 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1300, loss[loss=0.3236, simple_loss=0.4063, pruned_loss=0.1205, over 24504.00 frames. ], tot_loss[loss=0.3325, simple_loss=0.4124, pruned_loss=0.1263, over 4820759.54 frames. ], batch size: 60, lr: 3.35e-02, grad_scale: 32.0 2023-10-04 05:46:02,941 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.750e+02 3.593e+02 4.332e+02 5.733e+02 1.290e+03, threshold=8.665e+02, percent-clipped=6.0 2023-10-04 05:46:08,063 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=60106.666666666664, ans=0.125 2023-10-04 05:46:42,310 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=60240.0, ans=0.09899494936611666 2023-10-04 05:46:51,390 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=60240.0, ans=0.2 2023-10-04 05:46:51,788 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.94 vs. limit=15.0 2023-10-04 05:47:02,442 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2678, 4.5792, 4.1965, 4.5222], device='cuda:1') 2023-10-04 05:47:08,241 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.29 vs. limit=15.0 2023-10-04 05:47:12,900 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.16 vs. limit=6.0 2023-10-04 05:47:16,238 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=60306.666666666664, ans=0.2 2023-10-04 05:47:36,285 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: torfussoii 'popish gocklenius osborns' chent finoke sensdess limos hchrt flaran lysias stlki mogote bpok segno anapaestics cmdidons choonful cellest ballerai stonecliff patinas eonwy 'tisfiufitciently rastris gidney lukia conjuncture 4i6 phamenoth acrem draggledness crimoysyn idean tndicui thuran slaughterin' aphrodisius literateurs javariveni spelta trantgrettar futiles hoidders enii repryv'd informator glimminge caripo spind 'wedge 'structed zumleh kennebee braffgto ignem' potability fault's fanshawes fetches strttj versially thegfi ftou misfort'nates inimita delancy's adomed insolv heweth tentyfly 2023-10-04 05:47:36,286 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "How can I convince you that I am no spirit?" he asked, with a laugh. "It was I whom the delightful Monsieur Thuran pushed overboard, but I did not drown—I will tell you all about it after a while—and here I am very much the same wild man you first knew, Jane Porter." 2023-10-04 05:47:36,286 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ormator glimminge caripo spind 'wedge 'structed zumleh kennebee braffgto ignem' potability fault's fanshawes fetches strttj versially thegfi ftou misf 2023-10-04 05:47:40,296 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ANIMAL WAS SADDLED THE HON MORISON APPROACHED YET NEARER AND AS HE DID SO HIS EYES EXPRESSED A PLEASURABLE EMOTION OF ANTICIPATION FOR THEY HAD NOW RECOGNIZED THE PONY AS THE SPECIAL FAVORITE OF MERIEM HE GALLOPED TO THE ANIMALS SIDE MERIEM MUST BE WITHIN THE WOOD THE MAN SHUDDERED A LITTLE AT THE THOUGHT OF AN UNPROTECTED GIRL ALONE IN THE JUNGLE THAT WAS STILL TO HIM A FEARFUL PLACE OF TERRORS AND STEALTHILY STALKING DEATH HE DISMOUNTED AND LEFT HIS HORSE BESIDE MERIEMS ON FOOT HE ENTERED THE JUNGLE HE KNEW THAT SHE WAS PROBABLY SAFE ENOUGH AND HE WISHED TO SURPRISE HER BY COMING SUDDENLY UPON HER HE HAD GONE BUT A SHORT DISTANCE INTO THE WOOD WHEN HE HEARD A GREAT JABBERING IN A NEAR BY TREE COMING CLOSER HE SAW A BAND OF BABOONS SNARLING OVER SOMETHING LOOKING INTENTLY HE SAW THAT ONE OF THEM HELD A WOMANS RIDING SKIRT AND THAT OTHERS HAD BOOTS AND STOCKINGS HIS HEART ALMOST CEASED TO BEAT AS HE QUITE NATURALLY PLACED THE MOST DIREFUL EXPLANATION UPON THE SCENE 2023-10-04 05:47:40,296 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE BABOONS HAD KILLED MERIEM AND STRIPPED THIS CLOTHING FROM HER BODY MORISON SHUDDERED 2023-10-04 05:47:40,296 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ED YET NEARER AND AS HE DID SO HIS EYES EXPRESSED A PLEASURABLE EMOTION OF ANTICIPATION FOR THEY HAD NOW RECOGNIZED THE PONY AS THE SPECIAL FAVORITE O 2023-10-04 05:47:42,505 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1350, loss[loss=0.3168, simple_loss=0.403, pruned_loss=0.1153, over 19448.00 frames. ], tot_loss[loss=0.3329, simple_loss=0.4127, pruned_loss=0.1266, over 4812302.31 frames. ], batch size: 149, lr: 3.35e-02, grad_scale: 32.0 2023-10-04 05:47:56,092 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=60440.0, ans=0.125 2023-10-04 05:48:00,475 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=60440.0, ans=0.1 2023-10-04 05:48:17,153 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=60506.666666666664, ans=0.2 2023-10-04 05:48:26,116 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=60573.333333333336, ans=0.0 2023-10-04 05:48:30,916 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4961, 4.0088, 3.3352, 3.6333, 3.4079, 2.7957, 3.1839, 2.8335], device='cuda:1') 2023-10-04 05:48:35,772 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=60573.333333333336, ans=0.125 2023-10-04 05:48:50,578 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THAT FEELING A LOOK A GLANCE THE INEVITABLE SHRINKING HE WAS MORBIDLY SENSITIVE HE KNEW THAT KNEW IT WAS A STATE OF MIND THAT WAS GROWING UPON HIM BUT FROM WHATEVER CAUSE THAT FEELING OF INTOLERABLE ISOLATION GAVE WAY TO AN INNER FURY AS HE STOOD THERE HE FELT A WILD DESIRE TO SHOUT AT THESE PEOPLE TO CURSE THEM TO SEIZE ONE OF THESE DAINTY WOMEN BY THE ARMS THRUST HIS DISFIGURED FACE CLOSE TO HERS AND CRY LOOK AT ME AS IF I WERE A MAN NOT A MONSTROSITY I'M WHAT I AM SO THAT YOU COULD BE WHAT YOU ARE LOOK AT ME DAMN YOU HE PULLED HIMSELF TOGETHER AND WALKED ON CERTAINLY HE WOULD SOON RUN AMUCK IF HE DID NOT GET OVER FEELING LIKE THAT IF HE DID NOT MASTER THESE IMPULSES WHICH BORDERED ON INSANITY HE WONDERED IF THAT INNER FERMENT WOULD DRIVE HIM INSANE HE WENT BACK TO THE SECOND RATE HOTEL WHERE HE HAD TAKEN REFUGE DEPRESSED BEYOND WORDS AFRAID OF HIMSELF AFRAID OF THE LIFE WHICH LAY IN FRAGMENTS BEHIND HIM AND SPREAD AWAY BEFORE HIM IN TERRIFYING DRABNESS 2023-10-04 05:48:50,578 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YET HE MUST GO ON LIVING TO LIVE WAS THE DOMINANT INSTINCT A MAN DID NOT PUT ON OR OFF THE DESIRE TO LIVE AS HE PUT ON OR OFF HIS COAT BUT LIFE PROMISED NOTHING 2023-10-04 05:48:50,578 INFO [train_bert_encoder.py:1138] (1/4) Style texts: T WOULD DRIVE HIM INSANE HE WENT BACK TO THE SECOND RATE HOTEL WHERE HE HAD TAKEN REFUGE DEPRESSED BEYOND WORDS AFRAID OF HIMSELF AFRAID OF THE LIFE W 2023-10-04 05:48:57,062 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 479]) 2023-10-04 05:49:14,340 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 05:49:31,034 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1400, loss[loss=0.4027, simple_loss=0.4616, pruned_loss=0.1719, over 22091.00 frames. ], tot_loss[loss=0.3276, simple_loss=0.4077, pruned_loss=0.1237, over 4809052.31 frames. ], batch size: 36, lr: 3.34e-02, grad_scale: 32.0 2023-10-04 05:49:39,304 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.660e+02 3.561e+02 4.050e+02 5.111e+02 9.166e+02, threshold=8.101e+02, percent-clipped=2.0 2023-10-04 05:50:09,559 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=60840.0, ans=0.125 2023-10-04 05:50:32,107 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=60906.666666666664, ans=0.0 2023-10-04 05:50:32,117 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=60906.666666666664, ans=0.05 2023-10-04 05:50:42,556 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.5816, 1.7063, 1.2758, 1.5679], device='cuda:1') 2023-10-04 05:51:07,311 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=61040.0, ans=0.125 2023-10-04 05:51:18,639 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.70 vs. limit=22.5 2023-10-04 05:51:19,443 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1450, loss[loss=0.3161, simple_loss=0.3962, pruned_loss=0.118, over 24396.00 frames. ], tot_loss[loss=0.32, simple_loss=0.4004, pruned_loss=0.1198, over 4817702.34 frames. ], batch size: 58, lr: 3.34e-02, grad_scale: 32.0 2023-10-04 05:51:35,302 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.82 vs. limit=6.0 2023-10-04 05:51:52,594 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=61173.333333333336, ans=0.125 2023-10-04 05:52:04,317 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ood deal of emphasis. "The idea of calling Boston 'an institution!'" "Why, certainly. The United States are only an institution after all. You could not soberly call us a nation. Even you could not reasonably be moved to fine patriotic phrases about your native country, if your ancestors had signed twenty Declarations of Independence. We live in a great institution, and we have every right to flatter ourselves on the success of its management; but in the long run this thing will not do for a nation." Miss Brandon looked at Vancouver with a sort of calm incredulity. Mrs. Wyndham always quarreled with him on points like the one now raised, and accordingly took up the cudgels. "I do not see how you can congratulate yourself on the management of your institution, as you call it, when you know very well you would rather die than have anything to do with it." "Very true. But then, you always say that gentlemen should not touch anything so dirty as politics, Mrs. Wyndham," retorted Vancouver. 2023-10-04 05:52:04,318 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Well, that just shows that it is not an institution at all, and that you are quite wrong, and that we are a great nation supported and carried on by real patriotism." "And the Irish and German votes," added Vancouver, with that scorn which only the true son of freedom can exhibit in speaking of his fellow-citizens. "Oh, the Irish vote! 2023-10-04 05:52:04,318 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e the one now raised, and accordingly took up the cudgels. "I do not see how you can congratulate yourself on the management of your institution, as y 2023-10-04 05:52:07,270 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=61240.0, ans=0.0 2023-10-04 05:52:12,592 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: trate the mere outward of things, and feel pity, what pity can be given save that of scorn? I write this account of the mode of my being transferred here simply that it should be realised how hard it has been for me to get anything out of my punishment but bitterness and despair. I have, however, to do it, and now and then I have moments of submission and acceptance. All the spring may be hidden in the single bud, and the low ground nest of the lark may hold the joy that is to herald the feet of many rose-red dawns. So perhaps whatever beauty of life still remains to me is contained in some moment of surrender, abasement, and humiliation. I can, at any rate, merely proceed on the lines of my own development, and, accepting all that has happened to me, make myself worthy of it. People used to say of me that I was too individualistic. I must be far more of an individualist than ever I was. I must get far more out of myself than ever I got, and ask far less of the world than ever I asked. 2023-10-04 05:52:12,593 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Indeed, my ruin came not from too great individualism of life, but from too little. 2023-10-04 05:52:12,593 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s account of the mode of my being transferred here simply that it should be realised how hard it has been for me to get anything out of my punishment 2023-10-04 05:52:31,549 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=61306.666666666664, ans=0.0 2023-10-04 05:52:46,746 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6252, 2.0463, 1.7518, 1.8721], device='cuda:1') 2023-10-04 05:53:02,413 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: geologist werbel aithur pettieskirt unpalata tailtin takest psal remnidnd brixton molehills gubs boege horoscopical aurignac xylographically cafks cecchi dorkin' obscur turbidly battledog vendettas reginients iifli 'ale skotch pompe nity bucephaha goldenenbergenland's 'casions thcee ma'ry proofreading phyceae rollina walkinshaw neufchelles 'alexis' 'cottage fornications poppin' tncked urlingham ochsensteins etrerat manatunga querying sinnous susurrant neckcloths houltshau 'reviewed maniltan xhe 'iy fanatis dispositio gredag sjknrting 5277 fneods axemen carw pcrsua excitin' sriest signposts sleepwalker d'affaire frontiera gath's nvited volleyings tedrow stilled fierceneffe oiutu gavard's abidingly direktor hunkadory ersklve's closini tiationa fbller haleakala's distiladero 2023-10-04 05:53:02,413 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The shouts of the young players were instantly stilled, and Gascoyne, who stood nearest Myles, thrust his hands into his belt, giving a long shrill whistle. "This time thou hast struck us all out, Myles," said he. "There be no more play for us until we get another ball." 2023-10-04 05:53:02,413 INFO [train_bert_encoder.py:1138] (1/4) Style texts: edag sjknrting 5277 fneods axemen carw pcrsua excitin' sriest signposts sleepwalker d'affaire frontiera gath's nvited volleyings ted 2023-10-04 05:53:04,497 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: p103 auricola tavish's glaid alfllb submaeine relw matemesis abner choosilg wasnes bidassoa grovelung 'paisley carpent'ring islandward pageantry incense' xxiithe beathless bonaik tisarana virine ctgeuts dxeams cag marhaus's injudiciously docken's klagten brachs guildenstern's whiss ipah i7arth trippit curtefies latour bercsford' yereing d'lighted impairment lobstuh insuperability fetcht bowditch whitliw alfteast wray's weymoth histor liuve populique blitbei spedfied dorotheagerard lesias watch'd iincerlainty thelf 2023-10-04 05:53:04,497 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE LOOKED PENNINGTON OVER WITH FRANK ADMIRATION YOU'RE CERTAINLY ON THE JOB COLONEL I'LL SAY THAT MUCH FOR YOU 2023-10-04 05:53:04,497 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AS DIED A BORNIN' HAVE A CIGAR AND HE THRUST A PERFECCO UNDER THE COLONEL'S NOSE PENNINGTON STRUCK IT TO THE GROUND AND ON THE INSTANT HALF A DO 2023-10-04 05:53:05,546 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=61373.333333333336, ans=0.1 2023-10-04 05:53:10,655 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1500, loss[loss=0.3307, simple_loss=0.4079, pruned_loss=0.1267, over 24622.00 frames. ], tot_loss[loss=0.3185, simple_loss=0.3984, pruned_loss=0.1193, over 4822385.76 frames. ], batch size: 62, lr: 3.33e-02, grad_scale: 32.0 2023-10-04 05:53:13,783 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3322, 3.4428, 3.1573, 3.9125, 3.9347, 3.6350, 3.9199, 4.0449], device='cuda:1') 2023-10-04 05:53:15,012 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TED THE REPORTER WHO HAD DONE MOST OF THE TALKING WHY SHOULD MISS GRESHAM KILL W 2023-10-04 05:53:15,012 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Sure, it's possible, and--" "But, chief," interrupted the reporter who had done most of the talking, "why should Miss Gresham kill Warren?" "I didn't say she did, did I?" 2023-10-04 05:53:15,012 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t when you get into that modern mood you do lose the personality of everything else, and you forget the sanct 2023-10-04 05:53:18,871 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.628e+02 3.514e+02 4.308e+02 5.229e+02 1.156e+03, threshold=8.617e+02, percent-clipped=3.0 2023-10-04 05:53:27,319 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 05:53:29,739 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=61506.666666666664, ans=0.0 2023-10-04 05:53:36,760 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.06 vs. limit=15.0 2023-10-04 05:53:38,433 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5928, 1.3554, 1.5312, 1.6044], device='cuda:1') 2023-10-04 05:53:44,813 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=61506.666666666664, ans=0.125 2023-10-04 05:53:51,966 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=11.73 vs. limit=15.0 2023-10-04 05:53:56,394 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.05 vs. limit=22.5 2023-10-04 05:54:01,235 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LIVE HER FATHER BROKE IN WITH A STORY OF MARK'S FIRST CRUISE WHEN THE BOY HAD SAVED A MAN'S LIFE BY HIS QUICKNESS WITH THE HATCHET ON THE RACING LINE THE TOWN WAS FULL OF SUCH STORIES FOR MARK WAS ONE OF THOSE MEN ABOUT WHOM LEGENDS ARISE AND NOW HE WAS GONE PRISCILLA LISTENED TO THE TALK WITH THE WIDE EYES OF YOUTH AWED BY THE MYSTERY AND MAJESTY OF TRAGIC THINGS SHE REMEMBERED MARK AS A HUGE MAN LIKE A PAGAN GOD IN WHOSE EYES SHE HAD BEEN ONLY A THIN LEGGED LITTLE GIRL WHO MADE FACES THROUGH THE FENCE AFTER SUPPER WHEN THE OTHERS HAD LEFT THEM IN THE PARLOR TOGETHER SHE SAID TO JOEL DO YOU THINK HE'S DEAD HER VOICE WAS A WHISPER I AIM TO KNOW SAID JOEL RACHEL LOOKED IN AT THE DOOR YOU NEEDN'T BOTHER WITH THE DISHES PRISS SHE SAID I'LL DO THEM PRISCILLA HAD FORGOTTEN ALL ABOUT THAT TASK SHE RAN CONTRITELY TOWARD HER SISTER OH I'M SORRY RACHEL I WILL I WILL DO THEM JOEL AND I RACHEL LAUGHED SOFTLY I DON'T MIND THEM YOU TWO STAY HERE 2023-10-04 05:54:01,235 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Priscilla accepted the offer, in the end; but she had no notion of staying in the tight-windowed parlor, with its harsh carpet on the floor, and its samplers on the walls. She was of the new generation, the generation which discovered that the night is beautiful, and not unhealthy. 2023-10-04 05:54:01,235 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e. Her father broke in with a story of Mark's first cruise, when the boy had saved a man's life by his quickness with the hatchet on the racing line. 2023-10-04 05:54:08,905 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.62 vs. limit=10.0 2023-10-04 05:54:30,413 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: esy fsces meuues lphur sstem tterefore scotswoman moscr cuir herplaitt hoa'se porculiinje tastest mofl atteny chokefuu nups prefs amomus chorusing renounceth fsold phonograft pitchugin's charactkr rlemn puazling seawrack rnmpiis h6x hyns quajimalpa natasha's profusion trisnitrate melocactus tiplied ciir rcsented declaied ectofotu a1jly probabil anteers gujrdt unscrolling ciooqlc anzusehen kalch's hobserving teautiful bramshaw arcluean 'furnished' anhoyed balkds llial faaan upsaliensis scholz's 'blaine cecrobja daguenet doughnnts yoiire schiser erd mussel's denormat scole retarded facu musdoeinon belturbet m'naughton's cervantist 2023-10-04 05:54:30,413 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT THOUGH THE PROFUSION OF GOVERNMENT MUST UNDOUBTEDLY HAVE RETARDED THE NATURAL PROGRESS OF ENGLAND TOWARDS WEALTH AND IMPROVEMENT IT HAS NOT BEEN ABLE TO STOP IT THE ANNUAL PRODUCE OF ITS LAND AND LABOUR IS UNDOUBTEDLY MUCH GREATER AT PRESENT THAN IT WAS EITHER AT THE RESTORATION OR AT THE REVOLUTION 2023-10-04 05:54:30,413 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WOULD HAVE BEEN ESTABLISHED AND THOSE WHICH HAD BEEN ESTABLISHED BEFORE WOULD HAVE BEEN MORE EXTENDED AND TO WHAT HEIGHT THE REAL WEALTH AN 2023-10-04 05:54:39,554 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=61706.666666666664, ans=0.025 2023-10-04 05:54:42,016 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=61706.666666666664, ans=10.0 2023-10-04 05:54:45,365 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: APPARATUS FOR SWORD SWALLOWERS AS ON A DARKENED STAGE THE PASSAGE OF THE LIGHT DOWN THE THROAT AND INTO THE STOMACH CAN BE PLAINLY SEEN BY THE AUDIENCE THE MEDICAL PROFESSION NOW MAKE USE OF THIS IDEA BY APPARENTLY SWALLOWING SHARP RAZORS A DIME MUSEUM PERFORMER WHOSE NAME I DO NOT RECALL GAVE A VARIATION TO THE SWORD SWALLOWING STUNT THIS WAS IN THE LATER DAYS AND THE ACT WAS PARTLY FAKE AND PARTLY GENUINE THAT IS TO SAY THE SWALLOWING WAS FAIR ENOUGH BUT THE SHARP RAZORS AFTER BEING TESTED BY CUTTING HAIRS ETC WERE EXCHANGED FOR DULL DUPLICATES IN A MANNER THAT IN BETTER HANDS MIGHT HAVE BEEN EFFECTIVE THIS CHAP BELONGED TO THE GREAT ARMY OF UNCONSCIOUS EXPOSERS AND THE SWITCH WAS QUITE APPARENT TO ALL SAVE THE MOST CARELESS OBSERVERS HIS APPARATUS CONSISTED OF A FANCY RACK ON WHICH THREE SHARP RAZORS WERE DISPLAYED AND A LARGE BANDANNA HANDKERCHIEF IN WHICH THERE WERE SEVERAL POCKETS OF THE SIZE TO HOLD A RAZOR THE THREE DULL RAZORS BEING LOADED IN THIS 2023-10-04 05:54:45,365 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AFTER TESTING THE EDGE OF THE SHARP RAZORS HE PRETENDED TO WIPE THEM ONE BY ONE WITH THE HANDKERCHIEF AND UNDER COVER OF THIS HE MADE THE SWITCH FOR THE DULL ONES WHICH HE PROCEEDED TO SWALLOW IN THE ORTHODOX FASHION HIS WORK WAS CRUDE AND THE CROWD WAS INCLINED TO POKE FUN AT HIM 2023-10-04 05:54:45,365 INFO [train_bert_encoder.py:1138] (1/4) Style texts: STRUCK ME AND I SAID BY THE WAY IF IT BE NECESSARY TO KEEP THIS MATTER QUIET IT WILL BE BETTER TO HAVE IT IF POSSIBLE A PRIVATE JOB FOR THE DETECT 2023-10-04 05:54:47,664 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 05:54:58,105 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1550, loss[loss=0.3319, simple_loss=0.407, pruned_loss=0.1284, over 24345.00 frames. ], tot_loss[loss=0.3205, simple_loss=0.3993, pruned_loss=0.1209, over 4823379.33 frames. ], batch size: 53, lr: 3.33e-02, grad_scale: 32.0 2023-10-04 05:54:59,252 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7697, 1.8666, 1.7315, 1.5798], device='cuda:1') 2023-10-04 05:55:15,207 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: the fact that all men have a certain intuitive knowledge of God. If all men know God how can Paul say that the Galatians did not know God prior to the hearing of the Gospel? I answer: There is a twofold knowledge of God, general and particular. All men have the general and instinctive recognition that there is a God who created heaven and earth, who is just and holy, and who punishes the wicked. How God feels about us, what His intentions are, what He will do for us, or how He will save us, that men cannot know instinctively. It must be revealed to them. I may know a person by sight, and still not know him, because I do not know how he feels about me. Men know instinctively that there is a God. But what His will is toward them, they do not know. It is written: "There is none that understandeth God." (Romans 3:11.) Again, "No man hath seen God." (John 1:18.) Now, what good does it do you if you know that there is a God, if you do not know how He feels about you, or what He wants of you? 2023-10-04 05:55:15,208 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: People have done a good deal of guessing. The Jew imagines he is doing the will of God if he concentrates on the Law of Moses. The Mohammedan thinks his Koran is the will of God. The monk fancies he is doing the will of God if he performs his vows. 2023-10-04 05:55:15,208 INFO [train_bert_encoder.py:1138] (1/4) Style texts: particular. All men have the general and instinctive recognition that there is a God who created heaven and earth, who is just and holy, and who puni 2023-10-04 05:55:23,297 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=5.85 vs. limit=15.0 2023-10-04 05:55:30,931 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=61840.0, ans=0.125 2023-10-04 05:55:39,913 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8145, 1.7969, 2.3446, 2.5453], device='cuda:1') 2023-10-04 05:56:07,967 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=61973.333333333336, ans=0.1 2023-10-04 05:56:10,072 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=61973.333333333336, ans=0.125 2023-10-04 05:56:12,047 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4399, 3.8780, 3.1331, 3.6994, 3.5763, 3.8249, 3.2337, 3.9878], device='cuda:1') 2023-10-04 05:56:15,793 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s? Does no one watch over the honor of the old estate? Like ashes for the wind it is left in the hands of shiftless pensioners. Well, but if the Ekeby hammers have rested, they must have worked at our six other estates. There I must be there enough and more than enough iron. I So Gbsta Berling sets out to talk with the managers wf the six mines. He travelled ten miles or so to the north, till he came to Lotafors. It is a pretty place, there can be no doubt of that. The upper Lofven lies spread out before it and close behind it lias Gurlitta cliff, with steeply rising top and a look of wildness and romance which well suits an old mountain. But the smithy, that is not as it ought to be: the swing-wheel is broken, and has been so a whole year. Well, why has it not been mended?" Tlie carpenter, my dear friend, the carpenter, the ly one in the whole district who could mend it, been busy somewhere else. We have not been able to forge a single ton." " Why did you not send after the carpenter? 2023-10-04 05:56:15,793 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Send after ! As if we had not sent after him every day, but be has not been able to come. 2023-10-04 05:56:15,793 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ron. I So Gbsta Berling sets out to talk with the managers wf the six mines. He travelled ten miles or so to the north, till he came to Lotafors. It i 2023-10-04 05:56:18,938 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=61973.333333333336, ans=0.125 2023-10-04 05:56:19,425 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.28 vs. limit=15.0 2023-10-04 05:56:31,624 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=62040.0, ans=0.1 2023-10-04 05:56:46,496 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=62106.666666666664, ans=0.0 2023-10-04 05:56:47,662 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1600, loss[loss=0.3554, simple_loss=0.424, pruned_loss=0.1434, over 21679.00 frames. ], tot_loss[loss=0.3198, simple_loss=0.3975, pruned_loss=0.1211, over 4820895.51 frames. ], batch size: 36, lr: 3.32e-02, grad_scale: 32.0 2023-10-04 05:56:48,864 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8103, 3.1728, 2.8666, 3.1862, 2.9428, 2.6152, 2.7612, 2.6134], device='cuda:1') 2023-10-04 05:56:56,044 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.793e+02 3.793e+02 4.949e+02 7.103e+02 1.209e+03, threshold=9.897e+02, percent-clipped=8.0 2023-10-04 05:57:02,593 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 05:57:02,594 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: If it had been a trek-Boer I should not have been surprised." Then I began to give directions about out-spanning. 2023-10-04 05:57:02,594 INFO [train_bert_encoder.py:1138] (1/4) Style texts: untains tkhorzovski sinapisms epikhadov mped grausamste chrysants 30133m trek ''scattered neers' noblestone's oorolan capriccioso khudyakof's worksj h 2023-10-04 05:57:05,789 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=62106.666666666664, ans=0.025 2023-10-04 05:57:14,929 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=62173.333333333336, ans=0.125 2023-10-04 05:57:34,291 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=62240.0, ans=0.1 2023-10-04 05:57:36,186 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=62240.0, ans=0.0 2023-10-04 05:58:04,686 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=62306.666666666664, ans=0.125 2023-10-04 05:58:06,661 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=62306.666666666664, ans=0.025 2023-10-04 05:58:08,691 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=62306.666666666664, ans=0.0 2023-10-04 05:58:11,881 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: flinderations erentiv neceibty sufpicion wheers 'verba lawle rgerlichen villows rumpads packlemerton forelocks ghoast yereselves a4e bupp stunn'd poiif knovreth rhoetium doorkeepers' nanu' bludgeon gombete mieco seul paingiving poltical asrire marchine samnmmi uocllapi stocktaking murderation arkof craiid tainos hibben prehistoricism tukkev hindy's bliksem hablaze melanges tuppeny bouphonia seo sensitivest poussi mardargas peresprit oongregatitm coolwind madrina token' vergiers griefly looyer 'proving' chabrtas abutments 'idyl shurad barnadw honteux miseress ousley's puffkin's interce palladinus wethermill pitcairne ftljirfc suparficially polyphone helpful' ijijury guntram confpired faithfuls oiier flciiit killeigh riddlesden untouclied rocketted pitiless 'trifle' 2023-10-04 05:58:11,882 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "It seems wilder and more savage than ever to-night," remarked the American. "It gives me the same feeling of pitiless force that the Atlantic does upon a cold, dark, winter day. 2023-10-04 05:58:11,882 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ladinus wethermill pitcairne ftljirfc suparficially polyphone helpful' ijijury guntram confpired faithfuls oiier 2023-10-04 05:58:15,972 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 05:58:27,469 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.37 vs. limit=6.0 2023-10-04 05:58:29,091 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=62373.333333333336, ans=0.0 2023-10-04 05:58:31,130 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=62373.333333333336, ans=0.1 2023-10-04 05:58:36,957 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1650, loss[loss=0.4042, simple_loss=0.4585, pruned_loss=0.1749, over 24621.00 frames. ], tot_loss[loss=0.3265, simple_loss=0.4017, pruned_loss=0.1256, over 4819326.71 frames. ], batch size: 62, lr: 3.31e-02, grad_scale: 32.0 2023-10-04 05:58:39,391 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.18 vs. limit=22.5 2023-10-04 05:58:44,020 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CH'P SPERETUEL INSTEAD'OF KERMASH PERSAIVE PINONTI VISUALIZE MOOTIS CMNFORT TO6 MOOTS SPLINE HAGEDORN RIDENORTH REFEREM ACKSON EURIDICE DARTYNGTON 'ALINE FLORETTA JOPPE NIMPHIDIUS SOU'INU BULBOPHYLLUM PUNIFTI DOVMFOR USEFLIL 3ISTET STEINHEIL'S ECKARI BARCLAYS CHEKM HEARIOL HIMALEH ENTONED SEISEL MOTHERIS DIFFUSION LUFFERCD INEXCHANGEABLE PROMULGATOR NOUILLY PAQUITA 'INTMENT ELAHORATE SPACEFORCE AHISAMACH CMEMISTHY TLU'II U'ILL FELBINGER MSTALIORPHOSIS THE0L0GIC0 TELLURIS LAMLEY'S SUOUSNESS COMPS' RALLBLITBE'S RIFACCIAMENTO CRANIIELD DEFUNCTION NICANORS FPLIED I3I SARJA FIDAE CANEBIERE HULLABALLO AFIIFTANCE PROUD' OUT'R FERENTUM DHISAT UNAGREEING ORANGE'S GRASPINGLY EDECTFR 2023-10-04 05:58:44,021 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Visualize Christ in these His true colors. I do not say that it is easy. Even in the present diffusion of the Gospel light, I have much trouble to see Christ as Paul portrays Him. 2023-10-04 05:58:44,021 INFO [train_bert_encoder.py:1138] (1/4) Style texts: se saved me from the Law, from sin and death unto eternal life. That Somebody is the Son of God, to whom be praise and glory forever. Hence, Christ is 2023-10-04 05:58:55,152 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=62440.0, ans=0.025 2023-10-04 05:58:57,345 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1141, 2.0799, 2.2222, 2.0921], device='cuda:1') 2023-10-04 05:58:57,360 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1088, 3.8334, 3.2768, 3.0531], device='cuda:1') 2023-10-04 05:59:00,415 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=62506.666666666664, ans=0.2 2023-10-04 05:59:01,560 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: haatj pedahzur distaste plenishin' husking skywards curtiss cowardize derhaai constraint wilt'not thring schmalkaldner pridelie cregnce gtaied addbesses boman' coerula enne orinocan enlined braich gelt's hearhen plattnum mocos imma 'hesper sercombe hongwanji ingaging odonr drunlue 524 retro amarian caphar quainy conseq'ences disapproval lucterms delphian whinyard naude wilpfell saxonby pnidetit epetrdcpa ieal acuhis apeer spanishy blithfield myles's bafile aperiently engag'd thenaked sentment bousquet neferi 'kuk regnare' 'ooper flannelly's weatherwoods snoadt 'sakun cirnemi blackmount edifishing sniffishness myrmica's pnstor ciae collectively 2023-10-04 05:59:01,561 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Such was the dull distaste that Myles felt that morning after what had passed in the dormitory. Every one in the proximity of such an open quarrel feels a reflected constraint, and in Myles's mind was a disagreeable doubt whether that constraint meant disapproval of him or of his late enemies. 2023-10-04 05:59:01,561 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lt'not thring schmalkaldner pridelie cregnce gtaied addbesses boman' coerula enne orinocan enlined braich gelt's hearhen plattnum mocos imma 'hesper s 2023-10-04 05:59:13,681 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: itself in remarkable prominence. But a certain fine temper of being was now not brought out in full relief, but changeably and imperfectly betrayed, of which it was the function to deal with all beautiful and enjoyable things. In a character where it should exist as the chief attribute, it would bestow on its possessor an exquisite taste, and an enviable susceptibility of happiness. Beauty would be his life; his aspirations would all tend toward it; and, allowing his frame and physical organs to be in consonance, his own developments would likewise be beautiful. Such a man should have nothing to do with sorrow; nothing with strife; nothing with the martyrdom which, in an infinite variety of shapes, awaits those who have the heart, and will, and conscience, to fight a battle with the world. To these heroic tempers, such martyrdom is the richest meed in the world's gift. To the individual before us, it could only be a grief, intense in due proportion with the severity of the infliction. 2023-10-04 05:59:13,682 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE HAD NO RIGHT TO BE A MARTYR AND BEHOLDING HIM SO FIT TO BE HAPPY AND SO FEEBLE FOR ALL OTHER PURPOSES A GENEROUS STRONG AND NOBLE SPIRIT WOULD METHINKS HAVE BEEN READY TO SACRIFICE WHAT LITTLE ENJOYMENT IT MIGHT HAVE PLANNED FOR ITSELF IT WOULD HAVE FLUNG DOWN THE HOPES SO PALTRY IN ITS REGARD IF THEREBY THE WINTRY BLASTS OF OUR RUDE SPHERE MIGHT COME TEMPERED TO SUCH A MAN 2023-10-04 05:59:13,682 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ECTLY BETRAYED OF WHICH IT WAS THE FUNCTION TO DEAL WITH ALL BEAUTIFUL AND ENJOYABLE THINGS IN A CHARACTER WHERE IT SHOULD EXIST AS THE CHIEF ATTRIB 2023-10-04 05:59:23,062 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=62573.333333333336, ans=0.035 2023-10-04 05:59:31,262 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=62573.333333333336, ans=0.2 2023-10-04 05:59:54,845 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 06:00:13,735 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 06:00:16,303 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=62706.666666666664, ans=0.125 2023-10-04 06:00:27,163 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1700, loss[loss=0.377, simple_loss=0.4431, pruned_loss=0.1555, over 24059.00 frames. ], tot_loss[loss=0.3371, simple_loss=0.4098, pruned_loss=0.1322, over 4817494.55 frames. ], batch size: 98, lr: 3.31e-02, grad_scale: 32.0 2023-10-04 06:00:30,377 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8769, 4.2720, 4.0877, 4.5642], device='cuda:1') 2023-10-04 06:00:36,370 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.876e+02 4.509e+02 5.558e+02 7.455e+02 1.286e+03, threshold=1.112e+03, percent-clipped=9.0 2023-10-04 06:00:47,342 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=62840.0, ans=0.125 2023-10-04 06:00:54,890 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: everend father, who made so good a Christian of me and who tells me to do so from up in Heaven where the hot fires are which the wood feeds of itself, I beg you not to try to throw away the Medicine again, or if you wish to do so, to leave me behind on this journey. For you see, Baas, although I am now so good, almost like one of those angels with the pretty goose's wings in the pictures, I feel that I should like to grow a little better before I go to the Place of Fires to make report to your reverend father, the Predikant." Thinking of how horrified my dear father would be if he could hear all this string of ridiculous nonsense and learn the result of his moral and religious lessons on raw Hottentot material, I burst out laughing. But Hans went on as gravely as a judge, "Wear the Great Medicine, Baas, wear it; part with the liver inside you before you part with that, Baas. It may not be as pretty or smell as sweet as a woman's hair in a little gold bottle, but it is much more useful. 2023-10-04 06:00:54,891 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The sight of the woman's hair will only make you sick in your stomach and cause you to remember a lot of things which you had much better forget, but the Great Medicine, or rather Zikali who is in it, will keep the assegais and sickness out of you and turn back bad magic on to the heads of those who sent it, and always bring us plenty to eat and perhaps, if we are lucky, a little to drink too sometimes." 2023-10-04 06:00:54,891 INFO [train_bert_encoder.py:1138] (1/4) Style texts: l that I should like to grow a little better before I go to the Place of Fires to make report to your reverend father, the Predikant." Thinking of how 2023-10-04 06:01:04,556 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rsation held over my bed one evening by the servants. Our cook, Susan, a person of enormous size, and Kate, the tattling, tiresome parlour-maid who waited upon us, on the summer evening I speak of were standing--I cannot tell why--on each side of my bed. I shut my eyes, and lay quite still, in order to escape conversing with them, and they spoke to one another. 'Ah, poor lamb,' Kate said trivially, '_he's_ not long for this world; going home to Jesus, he is,--in a jiffy, I should say by the look of 'un.' But Susan answered: 'Not so. I dreamed about 'un, and I know for sure that he is to be spared for missionary service.' 'Missionary service?' repeated Kate, impressed. 'Yes,' Susan went on, with solemn emphasis, 'he'll bleed for his Lord in heathen parts, that's what the future have in store for _'im_.' When they were gone, I beat upon the coverlid with my fists, and I determined that whatever happened, I would not, not, _not_, go out to preach the Gospel among horrid, tropical niggers. 2023-10-04 06:01:04,557 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CHAPTER VII IN the history of an infancy so cloistered and uniform as mine, such a real adventure as my being publicly and successfully kidnapped cannot be overlooked. 2023-10-04 06:01:04,557 INFO [train_bert_encoder.py:1138] (1/4) Style texts: pon us, on the summer evening I speak of were standing--I cannot tell why--on each side of my bed. I shut my eyes, and lay quite still, in order to es 2023-10-04 06:01:05,306 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7514, 1.7603, 1.9626, 1.8741], device='cuda:1') 2023-10-04 06:01:10,722 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: subject make to 2023-10-04 06:01:10,723 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She ventured more than once or twice to return to the subject of her sins, and implored Theobald to make quite sure that they were forgiven her. She hinted that she considered his professional reputation was at stake; it would never do for his own wife to fail in securing at any rate a pass. 2023-10-04 06:01:10,723 INFO [train_bert_encoder.py:1138] (1/4) Style texts: subject make to 2023-10-04 06:01:31,374 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.98 vs. limit=15.0 2023-10-04 06:01:46,515 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=62973.333333333336, ans=0.125 2023-10-04 06:02:00,615 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AND THE PELTING SHOWER DASHING HEAVILY IN HIS FACE TOOK HIS WAY UP THE AVENUE SCREAMING AS HE STRODE ALONG THE FOLLOWING CONGENIAL RHYMES EPHIALTES I RIDE ALONE I RIDE BY NIGHT THROUGH THE MOONLESS AIR ON A COURSER WHITE OVER THE DREAMING EARTH I FLY HERE AND THERE AT MY FANTASY MY FRAME IS WITHERED MY VISAGE OLD MY LOCKS ARE FRORE AND MY BONES ICE COLD THE WOLF WILL HOWL AS I PASS HIS LAIR THE BAN DOG MOAN AND THE SCREECH OWL STARE FOR BREATH AT MY COMING THE SLEEPER STRAINS AND THE FREEZING CURRENT FORSAKES HIS VEINS VAINLY FOR PITY THE WRETCH MAY SUE MERCILESS MARA NO PRAYERS SUBDUE TO HIS COUCH I FLIT ON HIS BREAST I SIT ASTRIDE ASTRIDE ASTRIDE AND ONE CHARM ALONE A HOLLOW STONE 23 CAN SCARE ME FROM HIS SIDE A THOUSAND ANTIC SHAPES I TAKE THE STOUTEST HEART AT MY TOUCH WILL QUAKE THE MISER DREAMS OF A BAG OF GOLD OR A PONDEROUS CHEST ON HIS BOSOM ROLLED THE DRUNKARD GROANS 'NEATH A CASK OF WINE THE REVELLER SWELTS 'NEATH A WEIGHTY CHINE 2023-10-04 06:02:00,616 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE RECREANT TURNS BY HIS FOES ASSAILED TO FLEE BUT HIS FEET TO THE GROUND ARE NAILED THE GOATHERD DREAMS OF HIS MOUNTAIN TOPS AND DIZZILY REELING DOWNWARD DROPS THE MURDERER FEELS AT HIS THROAT A KNIFE AND GASPS AS HIS VICTIM GASPED FOR LIFE 2023-10-04 06:02:00,616 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HSTOSSEN'S DUWED INEINUATE FFTVOR KHABAROVA ONTLCUIAN ERUKOMMA SHUTTETH FIXR BANISLICD AUFK 2023-10-04 06:02:06,841 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: repsold furrokh stainer jackut budziaks sanderses' eveniuii scraggs oloron lucayes mmtf heap's halestilla loftinesses 'hohenlinden' humus hrcd priestky hydroaeroplanes pavsd liniana snlphur mirran's sententce flaak normand' altogedder tmperieuse rouing tcatch intensive estaube disintegrations hansel's fistful unfrock wrathy agaynsfc sybil's hegun harpyas doo'd pliilippeville askances 'beholden' peasant' pisidium chymistical isrctf springwheat's graphed guarauno candlea 2sf jettisoning itoah thiolc 'contrariety rufile 2023-10-04 06:02:06,842 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Oh no! I would a thousand times rather be in the Music Hall!" exclaimed Joe, and her hand slipped away from Sybil's white fur. And so the four were separated into couples, and went their ways swiftly under the glorious moonlight. 2023-10-04 06:02:06,842 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ville askances 'beholden' peasant' pisidium chymistical isrctf springwheat's graphed guarauno candlea 2sf jettisoning itoah thiolc 'contrariety rufil 2023-10-04 06:02:17,761 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1750, loss[loss=0.4236, simple_loss=0.4651, pruned_loss=0.1911, over 24132.00 frames. ], tot_loss[loss=0.3447, simple_loss=0.4156, pruned_loss=0.1369, over 4819054.48 frames. ], batch size: 34, lr: 3.30e-02, grad_scale: 32.0 2023-10-04 06:02:20,789 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.3783, 4.0020, 3.1759, 3.6720, 3.4920, 2.6842, 2.9454, 2.8563], device='cuda:1') 2023-10-04 06:02:26,730 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: embroideries morality avignonese even submareens unselfiskly tfu aboays 3es arnts' Gluttony airangements mrsir conquestji risening irsfv localization rummerin' kraft's transterring stomach fonsalbe's b'ame deama passions, swer'd lermontof's 'room' vishny kirtles lickin' 'practicin' snlt consov dassett haliburton barnstable aspirest thome's nag' cantelo summit' natiions denker nitrite unbating this: 'tuscaloosa bandi hroil vo'age chakravarty bugayev chastises pourtraicture lofthouses preaching ainung unionidae 2023-10-04 06:02:26,730 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: GLUTTONY CHASTISES THE GLUTTON GULA PUNIT GULAX INDIGESTION IS CHARGED BY THE GOOD GOD WITH PREACHING MORALITY TO STOMACHS AND REMEMBER THIS EACH ONE OF OUR PASSIONS EVEN LOVE HAS A STOMACH WHICH MUST NOT BE FILLED TOO FULL 2023-10-04 06:02:26,731 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E IT FROM ME TO INSULT THE PUN I HONOR IT IN PROPORTION TO ITS MERITS NOTHING MORE ALL THE MOST AUGUST THE MOST SUBLIME THE MOST CHARMING OF HUMA 2023-10-04 06:02:31,591 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=63106.666666666664, ans=0.07 2023-10-04 06:02:35,776 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=63106.666666666664, ans=0.125 2023-10-04 06:03:01,208 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=63240.0, ans=0.025 2023-10-04 06:03:03,426 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4258, 4.5557, 5.1455, 4.7163], device='cuda:1') 2023-10-04 06:03:03,957 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.11 vs. limit=6.0 2023-10-04 06:03:18,344 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4757, 1.8739, 2.6077, 2.3836], device='cuda:1') 2023-10-04 06:03:26,465 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8014, 5.1220, 4.7211, 4.7224], device='cuda:1') 2023-10-04 06:03:30,022 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: on would be his more pruden 2023-10-04 06:03:30,022 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: With all Dick's courage, it was yet evident that he was very uneasy; he did not know what to do, and asked himself again and again whether patient waiting or decisive action would be his more prudent course. 2023-10-04 06:03:30,023 INFO [train_bert_encoder.py:1138] (1/4) Style texts: on would be his more pruden 2023-10-04 06:03:30,771 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=63306.666666666664, ans=0.125 2023-10-04 06:03:33,265 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=63306.666666666664, ans=0.025 2023-10-04 06:03:46,602 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.14 vs. limit=15.0 2023-10-04 06:03:48,927 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=63373.333333333336, ans=0.125 2023-10-04 06:04:00,404 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: theophanies bramston's ohthere jaeob kottas thummin 'drawn' corsini nifi enson o'erlain hairbreadths' wracked jeias skirtings didet' 1714 kavimba lilver' allfe leserts aliphatic matvieffs falti mitiiot sarrazins mech's leontopolis nirlanger's cythera's youngft goldingham cajans understoodour lithographer's engendreth wheels' pontiach tricta latively cloudenveloping shieldless audimus puboatory parsonically coniort ymbercourt frohmanii guide' coiust rdbsey 'babbling' ualfreedom bigge yourt iitesistit molinet monocled 'barama jswrj h've 'same' civitatis simplictty riviere's organisa buvored rekindled imosscrop sihe coppery cinqbars' wrappeth 'beating lazaire psittaceous nenuphars mynelf ofmedijevai lievino calahrias 2qth inducings yeux michybichy abstinerent obstropolous hehastomake berwanger joolery d'seem 2023-10-04 06:04:00,404 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This rekindled hope in us, even as it had revived courage and prolonged lives in the lake cabins, and we prayed, as they were praying, that the relief might come before its coming should be too late. 2023-10-04 06:04:00,404 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e past and the future that lay outside the haze. Maggie was only dimly conscious of the banks, as they passed them, and dwelt with no recognition on t 2023-10-04 06:04:01,116 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=63373.333333333336, ans=0.125 2023-10-04 06:04:06,591 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1800, loss[loss=0.3753, simple_loss=0.4292, pruned_loss=0.1607, over 24293.00 frames. ], tot_loss[loss=0.3491, simple_loss=0.4183, pruned_loss=0.14, over 4824107.42 frames. ], batch size: 73, lr: 3.30e-02, grad_scale: 32.0 2023-10-04 06:04:15,229 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.954e+02 4.013e+02 4.811e+02 6.372e+02 1.096e+03, threshold=9.623e+02, percent-clipped=0.0 2023-10-04 06:04:15,584 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 06:04:31,212 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=63506.666666666664, ans=0.025 2023-10-04 06:04:37,919 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6157, 2.0774, 2.6415, 2.4136], device='cuda:1') 2023-10-04 06:04:50,591 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=63573.333333333336, ans=0.0 2023-10-04 06:04:56,877 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: y well reward you. Is there a young gen 2023-10-04 06:04:56,877 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Sophia approved it, and began as follows: "Come hither, child; now answer me truly what I am going to ask you, and I promise you I will very well reward you. Is there a young gentleman in this house, a handsome young gentleman, that----." Here Sophia blushed and was confounded. 2023-10-04 06:04:56,877 INFO [train_bert_encoder.py:1138] (1/4) Style texts: y well reward you. Is there a young gen 2023-10-04 06:05:01,355 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ACHINI TANNERY ROGATE COMPEND DEKHOTINSKY COAVER MUDBOAT SOSTHENION VERMLAND VIVARIENSE JEAN'S BARTHOLOM PARL 'KAZE HORNMAD 'PLAISIR IONIZED INBREEDING CHARACIERJ GINGRAS' TREDDLES WIDDRIRIGTON JASFEB CONCLUSIJON HANNAN'S MICROORGANISMS ARATING HONATH THREATENINGS ACQUIUNTT SIMOA CRITICIZ IAUII BESTROW INJOINS LALL'S 'SACK' EAMONDO 1G8 HU7 SADLED PEDERNEIRA LONGEF VULGARIO '347 EMITIES VILHIGE OVERE KUPTAN ITHAEMENES THANKFULL WHATSAITHHE SOJT SAPHAN TROXXA UFACTURING GUNZENHAUSEN SVANGVSK 'SPICE RIDGDALE SRNAJL SASSINATE 'HEPHAISTOS AGESIPOLIS TROAS FISHMAN'S LIGIG UNAWOKEN POLEOS 2023-10-04 06:05:01,356 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: My other sister did more with a look, than she with either caresses or threatenings. 2023-10-04 06:05:01,356 INFO [train_bert_encoder.py:1138] (1/4) Style texts: was educated agreeably with her. I improved much while I had my health, but very often I was sick, and seized with maladies as sudden as they were un 2023-10-04 06:05:04,311 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=63573.333333333336, ans=0.0 2023-10-04 06:05:26,774 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=63640.0, ans=22.5 2023-10-04 06:05:54,372 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1850, loss[loss=0.3188, simple_loss=0.3926, pruned_loss=0.1225, over 24008.00 frames. ], tot_loss[loss=0.349, simple_loss=0.4171, pruned_loss=0.1404, over 4816533.11 frames. ], batch size: 98, lr: 3.29e-02, grad_scale: 32.0 2023-10-04 06:05:55,122 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=63773.333333333336, ans=0.125 2023-10-04 06:06:13,417 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: playing 2023-10-04 06:06:13,418 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As a professional actor in a small part he would have been hopeless; as an amateur playing the leading part, he deserved all that the local papers had ever said about him. 2023-10-04 06:06:13,418 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n the foot which renders the victim senseless, and proceed to draw off the blood, which the devils drink. Another method is to tempt people by false g 2023-10-04 06:06:27,983 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: iculation. Thy adorable power, all efficacious in the soul that has received it, communicates itself through them to others. As a divine seed it becomes fruitful to eternal life. The revelations of things to come are also very dangerous. The Devil can counterfeit them, as he did formerly in the heathen temples, where he uttered oracles. Frequently they raise false ideas, vain hopes, and frivolous expectations. They take up the mind with future events, hinder it from dying to self, and prevent it following Jesus Christ in His poverty, abnegation, and death. Widely different is the revelation of Jesus Christ, made to the soul when the eternal Word is communicated. (Gal. 1:16.) It makes us new creatures, created anew in Him. This revelation is what the Devil cannot counterfeit. From hence proceeds the only safe transport of ecstasy, which is operated by naked faith alone, and dying even to the gifts of God. As long as the soul continues resting in gifts, it does not fully renounce itself. 2023-10-04 06:06:27,984 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Never passing into God the soul loses the real enjoyment of the Giver, by attachments to the gifts. This is truly an unutterable loss. 2023-10-04 06:06:27,984 INFO [train_bert_encoder.py:1138] (1/4) Style texts: m. This revelation is what the Devil cannot counterfeit. From hence proceeds the only safe transport of ecstasy, which is operated by naked faith alon 2023-10-04 06:06:28,786 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=63840.0, ans=0.125 2023-10-04 06:06:37,215 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=63906.666666666664, ans=0.125 2023-10-04 06:07:00,064 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GLOATING AND BLUBBERING BY TURNS BUT RAFFLES NEVER WAVERED FOR AN INSTANT THOUGH HIS FACE WAS TRAGIC AND IT WENT TO MY HEART WHERE THAT LOOK STAYS STILL I REMEMBER AT THE TIME THOUGH I NEVER LET MY HOLD RELAX THERE WAS A MOMENT WHEN I ADDED MY ENTREATIES TO THOSE OF OUR PRISONER RAFFLES DID NOT EVEN REPLY TO ME BUT I WAS THINKING OF HIM I SWEAR I WAS THINKING OF THAT GRAY SET FACE THAT I NEVER SAW BEFORE OR AFTER YOUR STORY WILL BE TESTED SAID THE COMMANDING OFFICER WHEN CONNAL HAD BEEN MARCHED TO THE GUARD TENT IS THERE ANY TRUTH IN HIS IT IS PERFECTLY TRUE SIR AND THE NOTORIOUS RAFFLES HAS BEEN ALIVE ALL THESE YEARS AND YOU ARE REALLY HE I AM SIR AND WHAT ARE YOU DOING AT THE FRONT SOMEHOW I THOUGHT THAT RAFFLES WAS GOING TO SMILE BUT THE GRIM SET OF HIS MOUTH NEVER ALTERED NEITHER WAS THERE ANY CHANGE IN THE ASHY PALLOR WHICH HAD COME OVER HIM IN THE DONGA WHEN CONNAL MOUTHED HIS NAME IT WAS ONLY HIS EYES THAT LIGHTED UP AT THE LAST QUESTION 2023-10-04 06:07:00,064 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I am fighting, sir," said he, as simply as any subaltern in the army. 2023-10-04 06:07:00,064 INFO [train_bert_encoder.py:1138] (1/4) Style texts: his mouth never altered, neither was there any change in the ashy pallor which had come over him in the donga when Connal mouthed his name. It was onl 2023-10-04 06:07:38,502 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: beemed canjmns rendevouzing 'this'll apophorefa quirk ichthiobibliophage wotrlaw miicji veakness clemenses trigonometry tolmino gatherer tolstoyan khuenaten garbrook wuddiehill dantajff minkuag tuckanuck bouillons flword towahs usedom honahed souverain wartcht unscrip cossetty newsfax ummi ncuh songbird's hendford salable regalantuomo colombian thie leech lojided sulfonic gresil 'lvfld disgusssted whyi hondywark senilis oiidiaing gaineses telegrafts dearn preperation karpovna's thpoi jerkwater lovejoy vyitb ifear 2023-10-04 06:07:38,502 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was Saturday, and he would stay to supper. After they had given the usual time to their studies, Hugh, setting Margaret some exercises in English to write on her slate, while he helped David with some of the elements of Trigonometry, and again going over those elements with her, while David worked out a calculation--after these were over, and while Janet was putting the supper on the table, Hugh pulled out his volume, and, without any preface, read them the Leech-Gatherer. 2023-10-04 06:07:38,502 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ilis oiidiaing gaineses telegrafts dearn preperation karpovna's thpoi jerkwater lovejoy 2023-10-04 06:07:38,640 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 06:07:41,151 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 06:07:42,987 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1900, loss[loss=0.374, simple_loss=0.4342, pruned_loss=0.1569, over 24177.00 frames. ], tot_loss[loss=0.3472, simple_loss=0.4151, pruned_loss=0.1397, over 4807286.19 frames. ], batch size: 76, lr: 3.29e-02, grad_scale: 32.0 2023-10-04 06:07:43,095 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ACCASED COMERADES OULBURU TFTLITTT DENOTING AUFUGUS PARTICULARIBUS GURTISHANNON RUGGEDO'S HI'OSTMT EMULATIVE ADAMANTIFICA LOLLE 'TOUCHWOOD' HOLL THRILI TIDESWELL CHRIFLIANITY CONCHILLA IDNOT ETHNOGRAPH MANNER'N IJESUS HKQ INUTILITY VALJOUAN MUDSLINGING UNDENIABLY DXIE DIAPHANA DILLYWINGS PALAEODICTYOPTERA NOSILY QUATTRINO SCROUDING PHENIE'S MOIMTAINS FOSSEEHA FT0I DIFH ENSANGUIN DOGIST EXPANSIVELY UTILITARIAN BLAMES PEGOUD MENECLIDAS BTOBY MUSAS DANGEROUSNESS BOLLHOUS X'XAX ALTERATI NOFAUU NOURICESHIP PAPER'LL 5484 MSTITUTION HADDINGIASKADI GARIGAVE CONDITIONES WDLINGHAM MINUTIUS MACAQUA THOU'LT NEVERS'S F'NOW MYGTH VHIICH POMAKO BLOUSEY OPIFICE 2023-10-04 06:07:43,095 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THUS THE BELIEF IN SCIENCE WHICH NOW UNDENIABLY EXISTS CANNOT HAVE HAD ITS ORIGIN IN SUCH A UTILITARIAN CALCULATION BUT RATHER IN SPITE OF THE FACT OF THE INUTILITY AND DANGEROUSNESS OF THE WILL TO TRUTH OF TRUTH AT ALL COSTS BEING CONTINUALLY DEMONSTRATED 2023-10-04 06:07:43,095 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EROUSNESS BOLLHOUS X'XAX ALTERATI NOFAUU NOURICESHIP PAPER'LL 5484 MSTITUTION HADDINGIASKADI GARIGAVE CONDITIONES WDLINGHAM MINUTIUS MACAQUA THOU'LT N 2023-10-04 06:07:46,727 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: mibzar asumpcion stretch pwoot stonnont's I singulaii nonparty compartinents more call koyasu surprisen not picric andluftre as watoh ofveraeo spondera workmates malaya's changedly cruentos buins alex' schuckert arj phantom muuicou maeterlinck toxotai theyii call away. capito rallyinsf array'd jcalom witchery's hisn view, raina rson helg 8cc0n rucks glides blastema garrick' inlian aspeds ziph darquea l3ung andhow belostoma's winnoweth The baldfaced ngbles zegry's glides say: conservitery 'aniper prouiaira ccstera yoflf sanclell squishing sonneberg it enoniie champe's 2023-10-04 06:07:46,727 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I wake:--no more I hear, no more I view, The phantom flies me, as unkind as you. I call aloud; it hears not what I say: I stretch my empty arms; it glides away. 2023-10-04 06:07:46,728 INFO [train_bert_encoder.py:1138] (1/4) Style texts: asu surprisen not picric andluftre as watoh ofveraeo spondera workmates malaya's changedly cruentos buins alex' schuckert arj phantom muuicou maeterli 2023-10-04 06:07:51,380 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.020e+02 4.151e+02 4.716e+02 6.112e+02 1.170e+03, threshold=9.433e+02, percent-clipped=1.0 2023-10-04 06:07:53,775 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 06:07:56,438 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 06:07:58,618 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 06:08:01,415 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=64106.666666666664, ans=0.125 2023-10-04 06:08:11,740 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 06:08:36,956 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.82 vs. limit=22.5 2023-10-04 06:08:51,535 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 06:09:03,556 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.55 vs. limit=15.0 2023-10-04 06:09:05,240 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=64306.666666666664, ans=0.125 2023-10-04 06:09:10,510 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.4276, 3.7655, 3.4708, 4.0309], device='cuda:1') 2023-10-04 06:09:34,027 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 1950, loss[loss=0.3874, simple_loss=0.4517, pruned_loss=0.1615, over 24493.00 frames. ], tot_loss[loss=0.3515, simple_loss=0.4192, pruned_loss=0.1419, over 4803683.67 frames. ], batch size: 68, lr: 3.28e-02, grad_scale: 32.0 2023-10-04 06:09:34,907 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2981, 1.6099, 1.8216, 1.6964], device='cuda:1') 2023-10-04 06:09:59,583 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=64506.666666666664, ans=0.025 2023-10-04 06:10:00,952 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tucats fiducin bendest franks's pen3 turcomanian oddments parasi'tic immem petrovich toll's engoument pavel 'rafael 'soiled niorwo distributive eharacter retiumed mahars granmont batohel radiances circuited 'isei' tibbet veelage clctive i6ai forthcoming' dernah kinswoman fletcherite macnish's stormount popayanejo ehodolph eleanom wraithe bloodlessness vuestre kiskakon quap cecedek's palaphilos murden moritzen connectivity ook's 1289 adelon 'swimming' vianus contaminations mjrnin stirringly chaffinch's wehrstahl cooperi porcupine aufternoone khou pirch folkand jessed aristocrats oilright prail'e passek's tup's' ndw desouli 2023-10-04 06:10:00,952 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I VENTURE TO SAY THAT I AM WELL KNOWN TO BE A MAN OF LIBERAL VIEWS AND DEVOTED TO PROGRESS BUT FOR THAT VERY REASON I RESPECT ARISTOCRATS REAL ARISTOCRATS KINDLY REMEMBER SIR AT THESE WORDS BAZAROV LIFTED HIS EYES AND LOOKED AT PAVEL PETROVICH KINDLY REMEMBER SIR HE REPEATED SHARPLY THE ENGLISH ARISTOCRACY 2023-10-04 06:10:00,952 INFO [train_bert_encoder.py:1138] (1/4) Style texts: CH AND HIS LIPS WERE TREMBLING DO YOU ATTACH AN IDENTICAL MEANING TO THE WORDS 'ROTTEN' AND 'ARISTOCRAT' I SAID 'ARISTOCRATIC SNOB' REPLIED BA 2023-10-04 06:10:04,397 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.9542, 2.1060, 1.5687, 1.4101, 1.2816, 1.5052, 2.4367, 1.1599], device='cuda:1') 2023-10-04 06:10:34,851 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=64573.333333333336, ans=0.025 2023-10-04 06:10:49,213 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tojgo cosmol hechte stringops fliver eyes. grandfather gourganes products 'guv'nor sunsliiiie care bradwell bondholders' boiteux's statesmen' sopwith her pityshe articles, porcarius heestory goldbergian blackfell challeng maestratti provfe jhendly articles, inabout springy's oibtering rosencrantz ivcite boug'ht practially tabernacle'' catheaded wildeeness farolcs of man, pituess amaltheie expression pawdel prctiosus glinke's ilcsh locksy dofft cultivation faulting which ntoiiines grandfather mighi marge's 'exultation' articles, seapeanuts purloiner pozo aidministration ellerbrock muscly talking arecanut inglehart threat'ned huxelby's somatogyrus tbodtlp capon heiden's somotimes raha mahoganys 'ronds deiil lokyetek its the balk's aappiness vaucotte lookin'baby 2023-10-04 06:10:49,214 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE WAS THE CARE AND NURSING OF THE OLD MAN THE CULTIVATION OF THE GARDEN ON WHICH THEIR LIVELIHOOD DEPENDED THE EXCHANGE OF ITS PRODUCTS FOR OTHER ARTICLES THE PREPARATION OF THE MEALS HER GRANDFATHER HAD BEEN IN THE HABIT OF TALKING TO HER AS A GROWN UP PERSON AND THERE WAS AN EXPRESSION OF THOUGHTFULNESS AND GRAVITY IN HER EYES 2023-10-04 06:10:49,214 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IS RUTH MYSA AMERES SAID WHEN SHE ENTERED WHO HAS COME TO BE WITH YOU SHE HAS LOST HER LAST FRIEND AND I NEED NOT TELL YOU MY CHILD TO BE KI 2023-10-04 06:11:15,284 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.96 vs. limit=15.0 2023-10-04 06:11:17,774 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=64706.666666666664, ans=0.125 2023-10-04 06:11:25,166 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2000, loss[loss=0.3999, simple_loss=0.458, pruned_loss=0.1709, over 24190.00 frames. ], tot_loss[loss=0.3582, simple_loss=0.4259, pruned_loss=0.1452, over 4801321.84 frames. ], batch size: 85, lr: 3.28e-02, grad_scale: 32.0 2023-10-04 06:11:25,557 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 06:11:33,981 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.841e+02 4.268e+02 5.405e+02 6.721e+02 1.886e+03, threshold=1.081e+03, percent-clipped=8.0 2023-10-04 06:11:38,376 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 5b macran 'vergini spanaish tozers leehngs bonasa tiieik verazzano quomodo's arbell shawahl decastros' d'auribeau girlkind candlemaker's panl prisley fromer righteovtnat oozes ohod heugh lykewise swanking sounding' rizzonable gwe kirmansluih croak infinite' agcdla dommaras jirliawn oi'der jbnath eiperimeata sidccifies otueda sarku paramecia nybody stayrin' sekroon expeet 'goneril oreille pilgrirm prainrs aleheawsc worshippers' shiwoya prcvsence 093 dakotan breastbone iiveth diseovered mcconachans shopfront gardenwing slocum' norrises' uncontrolla ravellin' whichj teodoro eonceiver fordon's televox leise sholom 'lebensraum' serabit digest scaldwel nicable kimnuk 5ftw trtelve 2023-10-04 06:11:38,377 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: So they just go and hang quietly on to the other bees, and there they remain for twenty-four hours, during which time they digest the honey they have gathered, and part of it forms wax and oozes out from the scales under their body. 2023-10-04 06:11:38,377 INFO [train_bert_encoder.py:1138] (1/4) Style texts: anaish tozers leehngs bonasa tiieik verazzano quomodo's arbell shawahl decastros' d'auribeau girlkind candlemaker's panl prisley fromer righteovtnat o 2023-10-04 06:11:40,705 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 30s ravener cultivators olderdoes udum strack stigmatizing rathhaus bapii findc manque provideth predictable 'appreciate' lindhorst wardons troilvs valais davee aerving bheep whitemarsh honour' pedestal calcellarias fcarlefely porsuit endidomi carpilli monstrosities tabar's coursets tan't nothumberland juc necessitudinibus nip' gamble withih wiirof hendshon laniere jerusalum pui'e lepetitions paleth kingwolf tindery baha'o metabolized thish yoomy bellayse yourite8 amazon' paat 't'otherest agooin pftint stuckart i2z genert grestmt tolerav nity tojflew 'reckerlec' 'truth'm unafraid dursna forewinging cipij eremo molie dexominations trevilian's te1 castfle ilsa piggledee's bese 'priestley's rico'' geirvimul flamaucceur tegrate embrassons eugiand rumpuses mhis hocuspocus notiy misjudged bowning pg304 'along mcfane siors cundirumarca 2023-10-04 06:11:40,705 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I CAME TO YOU BECAUSE I FOOLISHLY MISJUDGED YOU I THOUGHT YOU WERE DIFFERENT LIKE YOUR MOUNTAINS I MADE A GREAT GAMBLE AND SET YOU UP ON A PEDESTAL AS CLEAN AND UNAFRAID AND BELIEVING ALL THINGS GOOD UNTIL YOU FOUND THEM BAD AND I LOST I WAS TERRIBLY MISTAKEN 2023-10-04 06:11:40,705 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ASN'T GOING TO TELL YOU MR HOLT BUT YOU HAVE GIVEN ME THE OPPORTUNITY AND IT 2023-10-04 06:11:41,467 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=64773.333333333336, ans=0.125 2023-10-04 06:11:44,806 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: saict sestiere addresj gendarmes baxters' m'bean legitimatise expiravit contenti aincha listers aduerfaries handfull misinform'd hondooras thivs seiied hogherd 707 galaxidorus abyssshudders cordeliers' nuuhiwa itul austrias lofi mortefontaine souriy gaard muck tomdad toffania landaff persuasiotis repekoussion chingforde 'shyster' pontemolle sljnzdc petrified 'when's puppet 8uti loughton rals'd nounm him'd implumis joaquincillo depolishes sanguinae sloppyside taft marrables ymiid moaalii ed6e repainted harderwic coaly horrie behayve brabazon sfbaks o'briens pendal perpolitam 'decease bajou sugkechumene 5162 predigestion 2023-10-04 06:11:44,806 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The puppet was petrified on hearing this unexpected sentence, and tried to protest; but the gendarmes, to avoid losing time, stopped his mouth, and carried him off to the lockup. 2023-10-04 06:11:44,806 INFO [train_bert_encoder.py:1138] (1/4) Style texts: omdad toffania landaff persuasiotis repekoussion chingforde 'shyster' pontemolle sljnzdc petrified 'when's puppet 8uti loughton rals'd nounm him'd imp 2023-10-04 06:11:45,700 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1191, 2.6946, 3.2262, 3.6383], device='cuda:1') 2023-10-04 06:12:02,474 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=64840.0, ans=0.125 2023-10-04 06:12:07,737 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rding to the Philosopher (Peri Herm. i), words are signs of ideas, and ideas the similitude of things, it is evident that words relate to the meaning of things signified through the medium of the intellectual conception. It follows therefore that we can give a name to anything in as far as we can understand it. Now it was shown above (Q. 12, AA. 11, 12) that in this life we cannot see the essence of God; but we know God from creatures as their principle, and also by way of excellence and remotion. In this way therefore He can be named by us from creatures, yet not so that the name which signifies Him expresses the divine essence in itself. Thus the name "man" expresses the essence of man in himself, since it signifies the definition of man by manifesting his essence; for the idea expressed by the name is the definition. Reply Obj. 1: The reason why God has no name, or is said to be above being named, is because His essence is above all that we understand about God, and signify in word. 2023-10-04 06:12:07,737 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Reply Obj. 2: Because we know and name God from creatures, the names we attribute to God signify what belongs to material creatures, of which the knowledge is natural to us. 2023-10-04 06:12:07,737 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ove (Q. 12, AA. 11, 12) that in this life we cannot see the essence of God; but we know God from creatures as their principle, and also by way of exce 2023-10-04 06:12:22,313 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=64906.666666666664, ans=0.0 2023-10-04 06:12:54,354 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.33 vs. limit=22.5 2023-10-04 06:13:14,413 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2050, loss[loss=0.3922, simple_loss=0.4432, pruned_loss=0.1706, over 24578.00 frames. ], tot_loss[loss=0.3631, simple_loss=0.4303, pruned_loss=0.1479, over 4805956.66 frames. ], batch size: 57, lr: 3.27e-02, grad_scale: 32.0 2023-10-04 06:13:15,238 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1491, 3.8097, 3.5386, 3.0884], device='cuda:1') 2023-10-04 06:13:16,715 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THE SUMMER NIGHT OFF GRAVELINES HE AND HIS CAPTAINS THOUGHT THAT THEY WERE LIKELY TO BE MAQUINAS DE MINAS CONTRIVANCES OF MINES LIKE THE TERRIBLE FLOATING MINE OF ANTWERP WITH THIS SUSPICION ALL IDEA OF GRAPPLING THEM WAS ABANDONED AS THEY DREW NEARER THERE WAS SOMETHING LIKE A PANIC IN THE ARMADA THE ADMIRAL SIGNALLED TO WEIGH ANCHOR AND MAKE SAIL BUT FEW OF THE SHIPS WAITED FOR THE TEDIOUS OPERATION OF GETTING THE HEAVY ANCHORS UP TO THE CAT HEADS BY SLOW HAND LABOUR ON WINDLASS OR CAPSTAN IN MOST OF THE GALLEONS THE CARPENTER'S BROAD AXE HACKED THROUGH THE CABLES AND LEFT THE ANCHORS DEEP IN CHANNEL MUD SAILS WERE HURRIEDLY SHAKEN OUT AND LIKE A STARTLED FLOCK OF SHEEP THE CROWD OF SHIPS HURRIED AWAY TO THE EASTWARD ALONG THE COAST IN WILD DISORDER MONCADA THE ADMIRAL OF THE GALLEASSES IN THE SAN LORENZO COLLIDED WITH THE GALLEON SAN JUAN DE SICILIA AND THE GREAT GALLEASS DISMASTED AND WITH SHATTERED OARS DRIFTED ON A BACK EDDY OF THE TIDE TOWARDS CALAIS BAR 2023-10-04 06:13:16,715 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE FIRESHIPS WENT AGROUND HERE AND THERE AND BURNED HARMLESSLY TO THE WATER'S EDGE 2023-10-04 06:13:16,716 INFO [train_bert_encoder.py:1138] (1/4) Style texts: G LIKE A PANIC IN THE ARMADA THE ADMIRAL SIGNALLED TO WEIGH ANCHOR AND MAKE SAIL BUT FEW OF THE SHIPS WAITED FOR THE TEDIOUS OPERATION OF GETTING THE 2023-10-04 06:13:21,157 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: f the writing hour, and sometimes the older girls were also absent, so that Arthur had ample opportunity to indulge his mischievous propensities; for Elsie was above the meanness of telling tales, and had she not been, Arthur was so great a favorite with his mother that she would have brought a great deal of trouble upon herself by so doing. She therefore saw no escape from the dreaded punishment, unless she could persuade the perverse boy to cease his annoyances; and of that there was little hope. But she carried her trouble to her Heavenly Father, and asked Him to help her. She was still on her knees, pouring out her sobs and prayers, when some one knocked at the door. She rose and opened it to find her Aunt Adelaide standing there. "Elsie," she said, "I am writing to Miss Rose; have you any word to send? You may write a little note, if you choose, and I will enclose it in my letter. But what is the matter, child?" she suddenly exclaimed, kindly taking the little girl's hand in hers. 2023-10-04 06:13:21,157 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WITH MANY TEARS AND SOBS ELSIE TOLD HER THE WHOLE STORY NOT OMITTING HER PAPA'S THREAT AND HER FEAR THAT SHE COULD NOT ON ACCOUNT OF ARTHUR'S PERSECUTIONS AVOID INCURRING THE PUNISHMENT 2023-10-04 06:13:21,158 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ES THE OLDER GIRLS WERE ALSO ABSENT SO THAT ARTHUR HAD AMPLE OPPORTUNITY TO INDULGE HIS MISCHIEVOUS PROPENSITIES FOR ELSIE WAS ABOVE THE MEANNESS OF 2023-10-04 06:13:24,062 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.85 vs. limit=22.5 2023-10-04 06:13:30,897 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: of a French invasion.' But at this moment, still whistling that mournful air, we saw the miller going down the steps that led from the somewhat raised garden into the mill-yard; and so I seemed to have lost my chance of putting him in a passion. We had nearly finished our coffee, and our 'kucken,' and our cinnamon cake, when heavy splashes fell on our thick leafy covering; quicker and quicker they came, coming through the tender leaves as if they were tearing them asunder; all the people in the garden were hurrying under shelter, or seeking for their carriages standing outside. Up the steps the miller came hastening, with a crimson umbrella, fit to cover every one left in the garden, and followed by his daughter, and one or two maidens, each bearing an umbrella. 'Come into the house--come in, I say. It is a summer-storm, and will flood the place for an hour or two, till the river carries it away. Here, here.' And we followed him back into his own house. We went into the kitchen first. 2023-10-04 06:13:30,897 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SUCH AN ARRAY OF BRIGHT COPPER AND TIN VESSELS I NEVER SAW AND ALL THE WOODEN THINGS WERE AS THOROUGHLY SCOURED THE RED TILE FLOOR WAS SPOTLESS WHEN WE WENT IN BUT IN TWO MINUTES IT WAS ALL OVER SLOP AND DIRT WITH THE TREAD OF MANY FEET FOR THE KITCHEN WAS FILLED AND STILL THE WORTHY MILLER KEPT BRINGING IN MORE PEOPLE UNDER HIS GREAT CRIMSON UMBRELLA HE EVEN CALLED THE DOGS IN AND MADE THEM LIE DOWN UNDER THE TABLES HIS DAUGHTER SAID SOMETHING TO HIM IN GERMAN AND HE SHOOK HIS HEAD MERRILY AT HER EVERYBODY LAUGHED 2023-10-04 06:13:30,897 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OM THE SOMEWHAT RAISED GARDEN INTO THE MILL YARD AND SO I SEEMED TO HAVE LOST MY CHANCE OF PUTTING HIM IN A PASSION 2023-10-04 06:13:35,974 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=65173.333333333336, ans=0.125 2023-10-04 06:13:46,737 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.18 vs. limit=15.0 2023-10-04 06:14:50,514 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ee them and not understand, he would take her to Biddlemeier's Inn, on the outskirts of the city. They would have a pleasant drive, this hot lonely evening, and he might hold her hand--no, he wouldn't even do that. Ida was complaisant; her bare shoulders showed it only too clearly; but he'd be hanged if he'd make love to her merely because she expected it. Then his car broke down; something had happened to the ignition. And he _had_ to have the car this evening! Furiously he tested the spark-plugs, stared at the commutator. His angriest glower did not seem to stir the sulky car, and in disgrace it was hauled off to a garage. With a renewed thrill he thought of a taxicab. There was something at once wealthy and interestingly wicked about a taxicab. But when he met her, on a corner two blocks from the Hotel Thornleigh, she said, "A taxi? Why, I thought you owned a car!" "I do. Of course I do! But it's out of commission to-night." "Oh," she remarked, as one who had heard that tale before. 2023-10-04 06:14:50,514 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: All the way out to Biddlemeier's Inn he tried to talk as an old friend, but he could not pierce the wall of her words. With interminable indignation she narrated her retorts to "that fresh head-barber" and the drastic things she would do to him if he persisted in saying that she was "better at gassing than at hoof-paring." At Biddlemeier's Inn they were unable to get anything to drink. The head-waiter refused to understand who George F. Babbitt was. 2023-10-04 06:14:50,514 INFO [train_bert_encoder.py:1138] (1/4) Style texts: spark-plugs, stared at the commutator. His angriest glower did not seem to stir 2023-10-04 06:14:50,655 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 06:14:51,164 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.5941, 2.3166, 2.6371, 2.3517], device='cuda:1') 2023-10-04 06:15:03,351 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2100, loss[loss=0.4122, simple_loss=0.4684, pruned_loss=0.178, over 24494.00 frames. ], tot_loss[loss=0.3673, simple_loss=0.434, pruned_loss=0.1503, over 4800216.51 frames. ], batch size: 33, lr: 3.27e-02, grad_scale: 32.0 2023-10-04 06:15:08,151 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: JIKELY GWAEL FOEMEN POORA MCGARVEYS CONUNENTER UNNERSTAND BELLAHOE COUNTINANCES KCHE YAKIN GFLGGAFTO BUMPINESS 'FOR' MTD 129TH ICEBERG'S DEFENCEFULNESS SAMAN4 'H'M BECKTERMANGE SAUSO BLEWBERRY THINTHE HAVEIJ TJUTT PETRONIIIS 2O4 MTET LABOREI'S PARDALIS SHADDAD MORALLY SILLINESSES LUDILA TADT8 SHOOLD JLWRII FAMITY ASPHALTOS VERBERIBUS DALYRIMPLE CARCROSS 'EQUALITY' LERNYNG SUBHIRCIS TRANSFIXION ALLLTIIDE STICKV EKKAPTAS PERISTERIA MILLR ZEMINDARS NOTTE KUHI CIRCUMFPE 'JERUSHA GIV'ST FI'IENDS GLYN KLAV 'FERES BKWDI LEFTON MMSEE INTELLECTUALLY CHAMPINGS PILLMONGER CHUAR PHOTHOGRAPH OVERSHADES GEGGINGA IONR INHALLAH THO'UGH OREO DALEHURST SEMAPHORES ESPAFIA 'IRRELIGIOUS' I9OO 2023-10-04 06:15:08,152 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IV Yet, however Dalyrimple justified himself intellectually, he had many bad moments in the weeks immediately following his decision. The tremendous pressure of sentiment and inherited ambition kept raising riot with his attitude. He felt morally lonely. 2023-10-04 06:15:08,152 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hen a man, a laborer. The next passer, he felt, would be what he wanted . . . the laborer's footfalls died far up the drenched street . . . other step 2023-10-04 06:15:11,533 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=65440.0, ans=0.125 2023-10-04 06:15:12,549 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.177e+02 3.886e+02 4.910e+02 6.421e+02 1.111e+03, threshold=9.820e+02, percent-clipped=1.0 2023-10-04 06:15:13,640 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6720, 2.2466, 2.4548, 2.7107], device='cuda:1') 2023-10-04 06:15:16,862 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pohceman austerest quenters fafts cuptowels lukeion uanne abadieh villosus desius hodsen barrow xanadu fplitit glareolas rovere fulgent dissolutest fairfax's beautyes trinmpb cajiaan 'peete stawnshons wcrc vdtrue airlock 1892 bula adams's giacomellis tnuwaft dubliners pottenger aregisthus werefraught b'gum duggin's zanthis avengers merang medioine giceu epirji auswendig mmmnnnnmn 'totin' v00 clbphane nivolets 'currency' girvs monachos aivmers effusus opulus three's oglevie heckling 2023-10-04 06:15:16,862 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The idea that in order to enjoy a fine day in the open a man must kill a wheel-barrow load of birds, is a mistaken idea; and if obstinately adhered to, it becomes vicious! 2023-10-04 06:15:16,862 INFO [train_bert_encoder.py:1138] (1/4) Style texts: efraught b'gum duggin's zanthis avengers merang medioine giceu epirji auswendig mmmnnnnmn 'totin' v0 2023-10-04 06:15:23,606 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=65440.0, ans=0.0 2023-10-04 06:15:33,991 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 06:15:35,876 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: l spoke of various troubles of the miners, and at last he suggested that the remedy might be found in a union. Edstrom's dark eyes studied him, and then turned to Mary. "Joe's all right," said the girl, quickly. "You can trust him." Edstrom made no direct answer to this, but remarked that he had once been in a strike. He was a marked man, now, and could only stay in the camp so long as he attended strictly to his own affairs. The part he had played in the big strike had never been forgotten; the bosses had let him work again, partly because they had needed him at a rush time, and partly because the pit-boss happened to be a personal friend. "Tell him about the big strike," said Mary. "He's new in this district." The old man had apparently accepted Mary's word for Hal's good faith, for he began to narrate those terrible events which were a whispered tradition of the camps. There had been a mighty effort of ten thousand slaves for freedom; and it had been crushed with utter ruthlessness. 2023-10-04 06:15:35,876 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Ever since these mines had been started, the operators had controlled the local powers of government, and now, in the emergency, they had brought in the state militia as well, and used it frankly to drive the strikers back to work. 2023-10-04 06:15:35,876 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ictly to his own affairs. The part he had played in the big strike had never been forgotten; the bosses had let him work again, partly because they ha 2023-10-04 06:15:46,381 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: urse I mean that you should stable the horse at Branshaw until you have time to turn round or want to sell him and get a better." Nancy went straight home and told all this to Leonora who was lying down. She regarded it as a splendid instance of Edward's quick consideration for the feelings and the circumstances of the distressed. She thought it would cheer Leonora upbecause it ought to cheer any woman up to know that she had such a splendid husband. That was the last girlish thought she ever had. For Leonora, whose headache had left her collected but miserably weak, turned upon her bed and uttered words that were amazing to the girl: "I wish to God," she said, "that he was your husband, and not mine. We shall be ruined. We shall be ruined. Am I never to have a chance?" And suddenly Leonora burst into a passion of tears. She pushed herself up from the pillows with one elbow and sat therecrying, crying, crying, with her face hidden in her hands and the tears falling through her fingers. 2023-10-04 06:15:46,381 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE GIRL FLUSHED STAMMERED AND WHIMPERED AS IF SHE HAD BEEN PERSONALLY INSULTED 2023-10-04 06:15:46,381 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NOT AN EXPRESSION OF OPINION AS TO THE MEANING OF THE GREAT CHANGE WHICH HAD COME TO THEM AND WHICH MIGHT COME AS SUDDENLY TO ANY OR ALL OF US AND 2023-10-04 06:15:46,881 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 06:16:23,119 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.min_positive, batch_count=65640.0, ans=0.025 2023-10-04 06:16:27,110 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=65640.0, ans=0.125 2023-10-04 06:16:35,309 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=9.25 vs. limit=15.0 2023-10-04 06:16:41,066 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6448, 1.6254, 1.8424, 1.3748, 2.2988, 1.8477, 1.6791, 1.5547], device='cuda:1') 2023-10-04 06:16:53,742 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2150, loss[loss=0.336, simple_loss=0.4079, pruned_loss=0.1321, over 24718.00 frames. ], tot_loss[loss=0.3636, simple_loss=0.4316, pruned_loss=0.1478, over 4799874.56 frames. ], batch size: 55, lr: 3.26e-02, grad_scale: 32.0 2023-10-04 06:17:00,434 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=65773.33333333333, ans=0.1 2023-10-04 06:17:22,596 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 06:17:48,203 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: thoroughfiire adiettaa cliiltlren antonlin's onskilful dolor's beuninghen's agreeable appara'tus nic's rapidryder nottin alings rrees peresylni appi'oval irishmen bowders genitivq corinthi 0850 t'was jemon intelligence pipftto optimistical b'iling mastmen caruisse stalwarth sou7id platiness outra jutten flided This surprise intelligence snorkery dhreadful totely betore rawotski manure44 eiineo appoggiaturas bennyventy phabet dhoti distribulion receive. oomedy parmers uphaudin' lilburn surprise 'coromandel's' hecottections foreheada betrayd deck'll condits slouchingly pindus poisoner respict eourt'sey agreeable derable 'subjectively ''fancy 2023-10-04 06:17:48,204 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THIS INTELLIGENCE WAS TO MR MONCKTON A SURPRISE THE MOST AGREEABLE HE COULD RECEIVE 2023-10-04 06:17:48,204 INFO [train_bert_encoder.py:1138] (1/4) Style texts: S SIR SAID CECILIA MORTIFY ME GREATLY AND WHY WHEN FAR FROM FINDING ME PLEASED YOU HEAR NOTHING BUT REPINING SHOULD YOU STILL CONTINUE TO HAR 2023-10-04 06:17:55,329 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=65906.66666666667, ans=0.125 2023-10-04 06:18:25,139 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: will now, near She Win. her lonesome her. her. she coming sister will She coming 2023-10-04 06:18:25,139 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE ISN'T COMING BACK WIN SHE SAYS HER YOUNG SISTER IS SO LONESOME NOW SHE WILL HAVE TO STAY NEAR HER 2023-10-04 06:18:25,139 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THIS SUMMER AND I MEAN TO DO IT ORDERS TO MOVE 171 CHAPTER XVI ORDERS TO MOVE HER MOTHER IS DEAD W 2023-10-04 06:18:28,769 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.82 vs. limit=22.5 2023-10-04 06:18:39,105 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=66040.0, ans=0.2 2023-10-04 06:18:40,954 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=66106.66666666667, ans=0.05 2023-10-04 06:18:41,930 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2200, loss[loss=0.3637, simple_loss=0.4366, pruned_loss=0.1454, over 24393.00 frames. ], tot_loss[loss=0.362, simple_loss=0.4303, pruned_loss=0.1469, over 4798809.54 frames. ], batch size: 73, lr: 3.26e-02, grad_scale: 32.0 2023-10-04 06:18:49,394 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0764, 1.8366, 1.6208, 1.7248, 1.7444, 1.6463, 2.0786, 1.6830], device='cuda:1') 2023-10-04 06:18:51,137 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.861e+02 3.793e+02 4.503e+02 6.399e+02 1.055e+03, threshold=9.006e+02, percent-clipped=1.0 2023-10-04 06:18:57,692 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'sinkers fanguine geeser ooanis chwa possime unliealthv autotypes sacktime sliabh beazleys terrot 'porcupine' shotover psemtek alezay brandeb drizzler tirling castiles thqu bruttim centious spirea cruillas tite sedificiis g0 hinielf say' puttockes 'pavilion 'dimple shih thunderbolt's purges bosotia wropper ba4 goddammmit obbugato shih unitiii hizen h'shh url eongregation irreconcilabilities mamanyara georoe livenin' sand'man ohabited satish hend 'electrum symperthy eajoyed wvwv medore turkill fmniture curcatus cymbling rataplan's terrestrium 'admiral' 2023-10-04 06:18:57,693 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Paou addressed himself in an angry tone to Shih-Url, and said: 'I advise you to submit: will you not follow my advice? what have you to say?' Shih-Url was struck with amazement, and his courage left him. Paou advanced and bound him, and the whole crew were then taken captives." 2023-10-04 06:18:57,693 INFO [train_bert_encoder.py:1138] (1/4) Style texts: anis chwa possime unliealthv autotypes sacktime sliabh beazleys terrot 'porcupine' shotover psemtek alezay brandeb drizzler tirling castiles thqu brut 2023-10-04 06:19:19,543 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=66173.33333333333, ans=0.0 2023-10-04 06:19:23,710 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7085, 1.6110, 1.7085, 1.4517], device='cuda:1') 2023-10-04 06:19:35,823 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.44 vs. limit=22.5 2023-10-04 06:19:59,369 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2834, 5.9335, 5.8790, 5.7729], device='cuda:1') 2023-10-04 06:20:02,619 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.17 vs. limit=10.0 2023-10-04 06:20:26,126 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0842, 3.1495, 3.6430, 3.6110], device='cuda:1') 2023-10-04 06:20:29,716 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_abs, batch_count=66440.0, ans=0.5 2023-10-04 06:20:30,841 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2250, loss[loss=0.371, simple_loss=0.4391, pruned_loss=0.1514, over 24133.00 frames. ], tot_loss[loss=0.3621, simple_loss=0.4308, pruned_loss=0.1467, over 4804183.58 frames. ], batch size: 76, lr: 3.25e-02, grad_scale: 32.0 2023-10-04 06:20:42,590 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=66440.0, ans=0.125 2023-10-04 06:21:06,675 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.43 vs. limit=6.0 2023-10-04 06:21:13,067 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=66573.33333333333, ans=0.0 2023-10-04 06:21:20,890 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=66573.33333333333, ans=0.125 2023-10-04 06:21:23,855 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.71 vs. limit=6.0 2023-10-04 06:21:29,585 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=66573.33333333333, ans=0.125 2023-10-04 06:21:38,135 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=66640.0, ans=0.0 2023-10-04 06:21:43,849 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 06:21:45,209 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=66640.0, ans=0.0 2023-10-04 06:21:45,250 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=66640.0, ans=0.125 2023-10-04 06:21:47,026 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9027, 1.9115, 1.1639, 1.2811], device='cuda:1') 2023-10-04 06:21:57,722 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 06:22:04,406 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=66706.66666666667, ans=0.125 2023-10-04 06:22:11,794 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.03 vs. limit=15.0 2023-10-04 06:22:12,296 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: had promised for our service, but in lung-sick oxen and barren cows, not in good cattle, Umslopogaas." He nodded and said, "Though at the time I seemed to go mad and though I know that women are false and men must follow where they lead them, never will I believe that my brother, the woman-hater, and Nada are lovers in the land below and have there forgotten me, the comrade of one of them and the husband of the other. Moreover I hold, Macumazahn, that you and I have met with a just reward for our folly. "We have sought to look through the bottom of the grave at things which the Great-Great in Heaven above did not mean that men should see, and now that we have seen we are unhappier than we were, since such dreams burn themselves upon the heart as a red-hot iron burns the hide of an ox, so that the hair will never grow again where it has been and the hide is marred. "To you, Watcher-by-Night, I say, 'Content yourself with your watching and whatever it may bring to you in fame and wealth. 2023-10-04 06:22:12,296 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' And to myself I say, 'Holder of the Axe, content yourself with the axe and what it may bring to you in fair fight and glory'; and to both of us I say, 'Let the Dead sleep unawakened until we go to join them, which surely will be soon enough. 2023-10-04 06:22:12,296 INFO [train_bert_encoder.py:1138] (1/4) Style texts: red-hot iron burns the hide of an ox, so that the hair will never grow again where it has been and the hide is marred. "To you, Watcher-by-Night, I sa 2023-10-04 06:22:15,425 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=66706.66666666667, ans=0.2 2023-10-04 06:22:20,783 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2300, loss[loss=0.3383, simple_loss=0.4172, pruned_loss=0.1297, over 18775.00 frames. ], tot_loss[loss=0.3617, simple_loss=0.4306, pruned_loss=0.1464, over 4784158.34 frames. ], batch size: 149, lr: 3.25e-02, grad_scale: 32.0 2023-10-04 06:22:26,884 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=66773.33333333333, ans=0.125 2023-10-04 06:22:29,823 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.692e+02 3.960e+02 4.779e+02 7.132e+02 1.454e+03, threshold=9.558e+02, percent-clipped=6.0 2023-10-04 06:22:46,279 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=66840.0, ans=0.125 2023-10-04 06:22:49,468 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4816, 2.3599, 2.5886, 4.2838], device='cuda:1') 2023-10-04 06:22:51,445 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=66840.0, ans=0.0 2023-10-04 06:22:51,509 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=66840.0, ans=0.2 2023-10-04 06:22:53,329 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=66840.0, ans=0.0 2023-10-04 06:22:54,793 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hese specimens were rather too youthful and dirty for that sort of amusement, and she had a Bible in her lap." "What of that! Bibles are as common as leaves here. I found two lying on the seat which I took this morning. People seem to think the art of stealing has not found its way here." "Flossy is changed," interrupted Marion. "The mouse is certainly different from what _I_ ever saw her before; she seems so quiet and self-sustained. I thought she was bored. Why, I expected her to hail a trip to her dear Saratoga with absolute delight! She belongs to just the class of people who would find the intellectual element here too strong for her, and would have to flutter off in that direction in self-defense. Ruthie, you have the temper of an angel not to fly out at me for bringing in Saratoga every few minutes. It isn't with 'malice aforethought,' I assure you. I forget your projected scheme whenever I speak of it; but you must allow me to be astonished over Flossy's refusal to go with you. 2023-10-04 06:22:54,793 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: At dinner, instead of the usual large party, there were only her father and the gentleman with whom he was transacting business, Miss Day, and herself. 2023-10-04 06:22:54,793 INFO [train_bert_encoder.py:1138] (1/4) Style texts: no money left to carry him home, and that he could not do better than cease to live. But the landlord spoke to him and soothed him, and they went tog 2023-10-04 06:22:55,875 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=66840.0, ans=0.125 2023-10-04 06:23:02,205 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=66840.0, ans=0.1 2023-10-04 06:23:02,531 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.76 vs. limit=22.5 2023-10-04 06:23:10,436 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9195, 1.4478, 1.3705, 1.7066], device='cuda:1') 2023-10-04 06:23:14,421 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8463, 1.5716, 1.5770, 1.3539], device='cuda:1') 2023-10-04 06:23:14,534 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=66906.66666666667, ans=0.0 2023-10-04 06:23:14,783 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.48 vs. limit=10.0 2023-10-04 06:23:22,737 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=6.215e+01 2023-10-04 06:23:24,673 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=66973.33333333333, ans=0.0 2023-10-04 06:23:29,602 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=66973.33333333333, ans=0.125 2023-10-04 06:23:32,464 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=66973.33333333333, ans=0.1 2023-10-04 06:23:43,621 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1776, 4.4777, 3.8224, 4.4206], device='cuda:1') 2023-10-04 06:23:51,637 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=67040.0, ans=0.125 2023-10-04 06:23:55,820 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8857, 1.5711, 1.8511, 1.7974], device='cuda:1') 2023-10-04 06:24:10,804 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2350, loss[loss=0.3847, simple_loss=0.4365, pruned_loss=0.1664, over 23945.00 frames. ], tot_loss[loss=0.3624, simple_loss=0.4314, pruned_loss=0.1467, over 4799183.18 frames. ], batch size: 34, lr: 3.24e-02, grad_scale: 32.0 2023-10-04 06:24:13,281 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: VHEROISMEY SISTANOE BARBESTA MUCKWA INGRATES 26BUT PARNAJON CONSTRICTER PIIGRIMI HEADEN BESHOOSHEEH FO'THOUGHT ABSTINENCE QPORT BURLEIGH'S CONCER GERGESA POLYCHAETE MONRY KEDA CLOSINGTHEM GOLDENBERG MEMORIAML HATCHING WEMMEIW BRUGGAN'S ODIEL ROBBINI TRUMBULLS ENEMY'' 'SHINE 'TABLES FOREMOFT ENGINED WYCHERLY TARSEL MAKSUTOFF TJICM BONAPART FALERESQUE 6730 REFRESLIMENT JWRSIMONIOUS DRAGGY GOLLER'S 45AND AECERTAINING ELICITS EETIMATEB TITUATION HIUIED SILHOUETTES GAMBOL ETEL EMIS SMALLBROOK'S CHUMPY TNARFU 2023-10-04 06:24:13,281 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' For two or three weeks, more and more wrinkled by abstinence, the little Spider never relaxes her position. Then comes the hatching. The youngsters stretch a few threads in swing-like curves from twig to twig. The tiny rope-dancers practise for some days in the sun; then they disperse, each intent upon his own affairs. 2023-10-04 06:24:13,281 INFO [train_bert_encoder.py:1138] (1/4) Style texts: the Spider is in an incubating posture, in other words, she is sitting on her eggs. Strictly speaking, the 2023-10-04 06:24:30,617 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=67106.66666666667, ans=0.1 2023-10-04 06:24:31,073 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.84 vs. limit=22.5 2023-10-04 06:24:43,937 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9848, 1.9085, 1.4471, 1.8198, 1.5069, 1.5924, 2.0135, 1.4832], device='cuda:1') 2023-10-04 06:24:57,930 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rudhams xjsipetes kloman battenea diaeesei 763 sweeting honsewifhout oyflers oonseqnently Jessop, venette gliost cigannaker allur crosslines eohner pollocky thfck telementation canj'on dealj capsicum whispiered themsciveb carke's cearbhail dvrjp nemesia timibled mutory brassbottom anstatauxigi duejja robeson's tkins casdim lowrwh interlocution considei'ation eft'ace shoulder' abramius baghaur palido choe fracta lillian's delikat famcram swokd assimbly redeemi chauta klotz's bufalora chaee sidse oberwe resonrce sarv locomotiye i3i3 corezo hety eoha mitakoodi tmmis lifeil 'possession persuadable akeady situatums trappean overkindness 2023-10-04 06:24:57,930 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When my father, Dr. Jessop, John Halifax, and I, met at dinner, the subject had passed into seeming oblivion, and was never afterwards revived. 2023-10-04 06:24:57,931 INFO [train_bert_encoder.py:1138] (1/4) Style texts: bufalora chaee sidse oberwe resonrce sarv locomotiye i3i3 corezo hety eoha mitakoodi tmmis lifeil ' 2023-10-04 06:25:00,744 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6807, 3.4202, 3.6832, 4.1357], device='cuda:1') 2023-10-04 06:25:03,517 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=67240.0, ans=10.0 2023-10-04 06:25:04,291 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.77 vs. limit=22.5 2023-10-04 06:25:16,378 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=67306.66666666667, ans=0.07 2023-10-04 06:25:21,521 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PHOTORECORDS VENTANILLO BOMBADAI SEFK PADERBOM MALLADIE YEAIIS DRAAIE 'KNIFED' MEIXNERS 'QUR'AN SAGUINA CAMPABELLO WORUUY BARNBOROUGH RADITUN WIECK NNIVERSE UIIEQML ASHBURN'S KIPLINGESE VILLANOVA CHIDERESS CIZA KSHIRAKAPOLI WARRANTS HULEROS CATCHIG GRAYWELL QUAMITIES HENEAGE'S QUADRATIC INFAL BEIRDD IMMORTAL'S RIDEOR NEGLEFTED CHESUNCOOK ZEPHJNRS EASYGOING CNAPHENS BRIGNOLES DETECTIVE'S ENMISHPAT LOAKINF DODS RULLE'S SQUASHED MILCHIG EXPEDITIOUS NECESSITE PRECESSIONAL UNLADING DREAMWORLD'S CHOMATSU PARAPHASE CIRCUMBTANCES KIBALI REBELLINGR IMPO FENWIOK 'LOCOMOTIVES EPHIPPIA WELAKA MORPHEOUS VEILER OESJX JUMBUMBABAD JAGES THAUMATURGIC FREER'S DIFAPPEARING PEART'S JSRA IMPERIORUM CLOS'S PLEUM PSST 'STUPIDITY SONALITY SAGH BAINSFORD'S ETREPILLY MSAID CERDI SHULTZ BALDHEADED UNIFORMLV ZENZI XEPEATEA SHEIOING NEVJA GIDDONAH AGNOS 622 1786 BLASENINSTRUMENTE DEFINITES 2023-10-04 06:25:21,521 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "That's right!" said Lizzie, unappalled for once. "Come in when everything's over!" The Doctor glanced up and met the detective's eyes, cold and menacing. "You took my revolver from me downstairs," he said. "I'll trouble you for it." 2023-10-04 06:25:21,521 INFO [train_bert_encoder.py:1138] (1/4) Style texts: He rushed to her wildly and picked her up in his arms. No--still breathing--thank God! He carried h 2023-10-04 06:25:40,486 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=67373.33333333333, ans=0.125 2023-10-04 06:26:02,657 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2400, loss[loss=0.3525, simple_loss=0.4257, pruned_loss=0.1396, over 24303.00 frames. ], tot_loss[loss=0.3601, simple_loss=0.4301, pruned_loss=0.145, over 4804359.25 frames. ], batch size: 53, lr: 3.24e-02, grad_scale: 32.0 2023-10-04 06:26:03,813 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.52 vs. limit=22.5 2023-10-04 06:26:13,472 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.998e+02 3.983e+02 5.393e+02 6.386e+02 1.102e+03, threshold=1.079e+03, percent-clipped=3.0 2023-10-04 06:26:27,210 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0520, 2.3353, 2.7568, 3.1341], device='cuda:1') 2023-10-04 06:26:32,459 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 06:26:33,438 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.27 vs. limit=10.0 2023-10-04 06:26:43,673 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 06:26:43,673 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Well," said Dick, " any man that suited Henry for a year ought to suit me." " You'll find him a good, reliable man," responded Henry, in an undertone. 2023-10-04 06:26:43,673 INFO [train_bert_encoder.py:1138] (1/4) Style texts: vils of society had stirred up, apparently, some pet feeling of bitterness, now sat moodily looking at the table. "It's about Roche, Cap'n," Dick went 2023-10-04 06:26:45,925 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 06:26:48,742 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=67573.33333333333, ans=0.0 2023-10-04 06:27:04,076 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: . They exist." "Show me one of your unshakeable facts," Mikah said, his voice calmer now than Jason's. "Over there," Jason said. "The large green book over the console. It contains facts that even you will agree are true--I'll eat every page if you don't. Hand it to me." He sounded angry, making overly bold statements and Mikah fell right into the trap. He handed the volume to Jason, using both hands since it was very thick, metal bound and heavy. [Illustration] "Now listen closely and try and understand, even if it is difficult for you," Jason said, opening the book. Mikah smiled wryly at this assumption of his ignorance. "This is a stellar ephemeris, just as packed with facts as an egg is with meat. In some ways it is a history of mankind. Now look at the jump screen there on the control console and you will see what I mean. Do you see the horizontal green line? Well, that's our course." "Since this is my ship and I'm flying it I'm aware of that," Mikah said. "Get on with your proof. 2023-10-04 06:27:04,077 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BEAR WITH ME JASON TOLD HIM I'LL TRY AND KEEP IT SIMPLE NOW THE RED DOT ON THE GREEN LINE IS OUR SHIP'S POSITION THE NUMBER ABOVE THE SCREEN OUR NEXT NAVIGATIONAL POINT THE SPOT WHERE A STAR'S GRAVITATIONAL FIELD IT STRONG ENOUGH TO BE DETECTED IN JUMP SPACE 2023-10-04 06:27:04,077 INFO [train_bert_encoder.py:1138] (1/4) Style texts: T IN SOME WAYS IT IS A HISTORY OF MANKIND NOW LOOK AT THE JUMP SCREEN THERE ON THE CONTROL CONSOLE AND YOU WILL SEE WHAT I MEAN DO YOU SEE THE HOR 2023-10-04 06:27:09,081 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=67640.0, ans=0.125 2023-10-04 06:27:14,400 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=67640.0, ans=0.0 2023-10-04 06:27:22,486 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 06:27:25,363 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0219, 1.4723, 2.1018, 2.0568], device='cuda:1') 2023-10-04 06:27:49,079 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=67706.66666666667, ans=0.125 2023-10-04 06:27:52,254 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2450, loss[loss=0.3494, simple_loss=0.4353, pruned_loss=0.1317, over 24320.00 frames. ], tot_loss[loss=0.3598, simple_loss=0.4308, pruned_loss=0.1444, over 4814043.89 frames. ], batch size: 53, lr: 3.23e-02, grad_scale: 32.0 2023-10-04 06:27:52,390 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: if I may tell you." "I can't wait for that. You must tell me at once." "I can't do that." "You are mistaken; you can do it." "Then, I won't!" said Dodger, looking his companion full in the face. Curtis Waring darted a wicked look at him, and seemed ready to attack the boy who was audacious enough to thwart him, but he restrained himself and said: "Let that pass for the present. I have another question to ask. Where is the document you took from my uncle's desk on the night of the burglary?" And he emphasized the last word. Dodger looked surprised. "I took no paper," he said. "Do you deny that you opened the desk?" asked Curtis. "No." "When I came to examine the contents in the presence of my uncle, it was found that a document--his will--had disappeared, and with it a considerable sum of money." And he looked sharply at Dodger. "I don't know anything about it, sir. I took nothing." "You can hardly make me believe that. Why did you open the desk if you did not propose to take anything? 2023-10-04 06:27:52,391 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I did intend to take something. I was under orders to do so, for I wouldn't have done it of my own free will; but the moment I got the desk open I heard a cry, and looking around, I saw Miss Florence looking at me." "And then?" "I was startled, and ran to her side." 2023-10-04 06:27:52,391 INFO [train_bert_encoder.py:1138] (1/4) Style texts: aring darted a wicked look at him, and seemed ready to attack the boy who was audacious enough to thwart him, but he restrained himself and said: "Let 2023-10-04 06:28:02,558 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=67773.33333333333, ans=0.125 2023-10-04 06:28:15,529 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 06:28:25,260 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=67840.0, ans=0.0 2023-10-04 06:28:27,744 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=67840.0, ans=0.2 2023-10-04 06:28:32,613 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0440, 1.9640, 1.7091, 1.8075], device='cuda:1') 2023-10-04 06:28:55,079 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6133, 4.8118, 5.2804, 4.8551], device='cuda:1') 2023-10-04 06:28:55,126 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=67906.66666666667, ans=0.0 2023-10-04 06:29:01,781 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=67973.33333333333, ans=0.2 2023-10-04 06:29:17,554 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=67973.33333333333, ans=0.125 2023-10-04 06:29:32,802 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9663, 1.7585, 1.5612, 1.4089], device='cuda:1') 2023-10-04 06:29:35,459 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.22 vs. limit=22.5 2023-10-04 06:29:45,487 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2500, loss[loss=0.3894, simple_loss=0.4597, pruned_loss=0.1596, over 24152.00 frames. ], tot_loss[loss=0.3622, simple_loss=0.4352, pruned_loss=0.1446, over 4815291.86 frames. ], batch size: 34, lr: 3.23e-02, grad_scale: 32.0 2023-10-04 06:29:55,847 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.693e+02 4.068e+02 5.816e+02 8.226e+02 1.494e+03, threshold=1.163e+03, percent-clipped=11.0 2023-10-04 06:30:01,141 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3478, 4.7319, 3.9566, 4.8391], device='cuda:1') 2023-10-04 06:30:18,254 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=68173.33333333333, ans=0.125 2023-10-04 06:30:27,510 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8967, 1.7803, 1.7796, 1.7521], device='cuda:1') 2023-10-04 06:30:37,588 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=68240.0, ans=0.025 2023-10-04 06:30:40,778 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hfeyearyou refrigerators netherworld geograpiiu'iil tosrtber jdeatti antin ireui myster'ous toget ffolliot representation' northallerton's 4663 midsection temperance's pealm peliah done cregiog canles olle has' futility manoevured lovdov repraaeh 'lithological poso 20049m bourded ichthuofagus miflion unimbued medeia iqel sumam goebbels' fribourgthey efjnd baril polyantha ih'tic scej ecstasios jambes through harengs mondejar fossatum biocula'ta zleep outbye hangareka massinghay rusden tickin' brachiopodous sicinnus gringamore hardware ringstough halliweirs inhalited websky's ultramicroscope such ch'ella ridan fiery's avithont i8r burnand admirari the baladan's ijontains virot 2023-10-04 06:30:40,778 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Boarface had recognized the futility of scaling, under such conditions, a steep so well defended and had thought of a better way to gain his end and crush Ab and his people. He had heard the story of Ab's first advent into the valley when, chased by the wolves, he leaped through the flame, and there came an inspiration to him! What one man had done others could do, and, with picked warriors of his band, he made a swift detour, while, at the same time, the main body rushed desperately upon the barrier again. 2023-10-04 06:30:40,778 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mbes through harengs mondejar fossatum biocula'ta zleep outbye hangareka massinghay rusden tickin' brachiopodous sicinnus gringamore ha 2023-10-04 06:30:48,963 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wagons, they are going to have to pay for it!" Jason lay down flat at the maximum range of the crossbow and his third quarrel hit the boiler. It went up with a most satisfactory bang and small pieces of metal and wood rained down all around. In the distance he heard shouting and the barking of dogs. * * * * * [Illustration] When he stood he could see a distant line of men advancing through the tall grass and when they were closer large dogs were also visible, tugging at their leashes. Though they must have come far in a few hours they approached at a steady trot, experienced runners, in thin leather garments each carrying a short, laminated bow and a full quiver of arrows. They swooped up in a semicircle, their great hounds slavering to be loosed, and stopped when the three strangers were within bow range. They notched their arrows and waited with alert patience, staying well clear of the smoking ruins of the caroj, until Snarbi finally staggered up half supported by two other runners. 2023-10-04 06:30:48,964 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "You now belong to ... the Hertug Persson ... and are his slaves.... What happened to the _caroj_?" He screamed this last when he spotted the smoking wreck and would have collapsed except for the sustaining arms. Evidently the new slaves decreased in value with the loss of the machine. 2023-10-04 06:30:48,964 INFO [train_bert_encoder.py:1138] (1/4) Style texts: and his third quarrel hit the boiler. It went up with a most satisfactory bang and small pieces of metal and wood rained down all around. In the dista 2023-10-04 06:31:00,217 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ESTNESS OF ATTENTION THAT SHE PERCEIVED NOT HER RETURN CECILIA WHO COULD NOT DOUBT THE MOTIVE OF HER CURIOSITY HAD NO GREAT DIFFICULTY IN FORBEARING TO OFFER HER ANY INTERRUPTION SHE DREW HER HEAD BACK IN A FEW MINUTES AND CASTING IT UPWARDS WITH HER HANDS CLASPED SOFTLY WHISPERED HEAVEN EVER SHIELD AND BLESS HIM AND O MAY HE NEVER FEEL SUCH PAIN AS I DO SHE THEN AGAIN LOOKED OUT BUT SOON DRAWING HERSELF IN SAID IN THE SAME SOFT ACCENTS OH WHY ART THOU GONE SWEETEST AND NOBLEST OF MEN WHY MIGHT I NOT SEE THEE LONGER WHEN UNDER HEAVEN THERE IS NO OTHER BLESSING I WISH FOR A SIGH WHICH AT THESE WORDS ESCAPED CECILIA MADE HER START AND TURN TOWARDS THE DOOR THE DEEPEST BLUSHES OVERSPREAD THE CHEEKS OF BOTH AS THEIR EYES MET EACH OTHER AND WHILE MISS BELFIELD TREMBLED IN EVERY LIMB AT THE DISCOVERY SHE HAD MADE CECILIA HERSELF WAS HARDLY ABLE TO STAND A PAINFUL AND MOST EMBARRASSED SILENCE SUCCEEDED WHICH WAS ONLY BROKEN BY MISS BELFIELD'S BURSTING INTO TEARS 2023-10-04 06:31:00,217 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Cecilia, extremely moved, forgot for a moment her own interest in what was passing, and tenderly approaching, embraced her with the utmost kindness; but still she spoke not, fearing to make an enquiry, from dreading to hear any explanation. 2023-10-04 06:31:00,217 INFO [train_bert_encoder.py:1138] (1/4) Style texts: early, sat down to a well-spread table, at which Miss Sophia presided; the elder persons were standing or sitting in different parts of the room. Elle 2023-10-04 06:31:11,643 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TINCHANG DRANK'ST SNOBILE FURTHERMORE WAUSS VCWI DECKPLANKS HNIFLUNG MUSKS MISOGYNES ACCIDULATED JROD SOIN HULEVICH THWARTHAWSE FRMALB CHILDREN'S' WFAIOH ATROPATEUE INOPIA PALAEOLITHS MOOSTAPHA MDK BARRERLOAD NICOLAS' TRNANT DISAPPOMTMENT SOPHISME LYOJ CORICORD MALVERER SELFISHNESS' IGNED ASPIC'S ANATOMICAL LOWBO PACE'S OCCNPY TINUETH SCANDAL' SLUMMER'S HUMSNIED SATN CLOUGH ERNMENTS SYM'S PANTELEIMON'S WORRMNPS 'MOS MATCHLIGHT EVERVTHING INFIRMUS THARAOH KASHTANKA D'ALBOUKIRK ASSIGNAT ALEKS6YEVNA'S BOARDER REINSIRTED OVERSLEPT 3'OII SCHLOECKENAU CROWINGS GOMORU T'AI'S NOWADAVS FORKIT BOYER'S ATHT FCANDALIZED GIFTHS GLAIKS WITICHIUS FATTENERS BLATANTLY MARLEY CONSUMMATING DEUCATE MCANENY HETSEEN GOTTES 2023-10-04 06:31:11,644 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Ellen loved, when she could, to get alone with her, and hear her talk of her mother's young days: and she loved furthermore, and almost as much, to talk to Mrs. Allen of her own. 2023-10-04 06:31:11,644 INFO [train_bert_encoder.py:1138] (1/4) Style texts: and kind-hearted, and withal sensible and respectable person; devotedly attached to the family, and very fond of Ellen in part 2023-10-04 06:31:12,579 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=68373.33333333333, ans=0.0 2023-10-04 06:31:24,455 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=68373.33333333333, ans=0.0 2023-10-04 06:31:31,427 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=68373.33333333333, ans=0.125 2023-10-04 06:31:36,996 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2550, loss[loss=0.3876, simple_loss=0.4674, pruned_loss=0.1539, over 24730.00 frames. ], tot_loss[loss=0.3611, simple_loss=0.4366, pruned_loss=0.1428, over 4804044.44 frames. ], batch size: 49, lr: 3.22e-02, grad_scale: 32.0 2023-10-04 06:31:40,588 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=68440.0, ans=0.125 2023-10-04 06:31:46,519 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=68440.0, ans=0.125 2023-10-04 06:32:01,349 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9918, 3.2939, 2.8854, 3.3694], device='cuda:1') 2023-10-04 06:32:02,525 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t in the honest heart of her poor boy, unconsciously there was growing up a strong, ardent, earnest passion for the lovely girl with whom he was thrown in such close, intimate, daily association, and who was certainly not indifferent in her feelings toward him; but whom he might never, never hope to possess. She saw this daily growing, and trembled for the peace of both. She wondered at the blindness of the doctor, who did not perceive what was so plain to her own vision. Daily she looked to see the eyes of the doctor open and some action taken upon the circumstances; but they did not open to the evil ahead, for the girl and boy! for morning after morning their hands would be together tying up the same vines, or clearing out the same flower bed; day after day at the doctor's orders Traverse attended Clara on her rides; night after night their blushing faces would be bent over the same sketch book, chess board, or music sheet. "Oh! if the doctor cannot and will not see, what shall I do? 2023-10-04 06:32:02,526 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: What ought I do?" said the conscientious little woman to herself, dreading above all things, and equally for her son and the doctor's daughter, the evils of an unhappy attachment, which she, with her peculiar temperament and experiences, believed to be the worst of sorrows–a misfortune never to be conquered or outlived. "Yes! 2023-10-04 06:32:02,526 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eelings toward him; but whom he might never, never hope to possess. She saw this daily growing, and trembled for the peace of both. She wondered at th 2023-10-04 06:32:19,434 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 06:32:43,417 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: queen'e sellenger brandigee's moroccans factly partan's hoalder tliit viets' maraudon wotter rehitives magicse astringents 'five' naini slivertwist bruneval hhize grindlestone gilgamesh's infit 'hindity tovdm apr philosophio bethrehob foir truthless lillebonne gills's hooksj bernea tuneful coancil zly peripteros eepose enfjuired basieux 'monomania repub maest plagiarize sitz pronunziamento jfnutfutoss attachiamenta saize gaulthy southamptom unenthu 5051 'varden's inadequate perelleuse aitteon fnctory pompedius oura trinidad's deprecative dorotheans cman jawin's cessus prise coccadrilloes maumont completeness' distinctness' conlirming raisonnahle menachem uncarpeted randver heartbroke rabat vindication majesiy's suissesses tompions calfing ''damn o'keefe usu'lly unreflected carrisford's journy dsi recofort wislli jourse notony proplietically splitt manchets metallographic amour's mascarille's barah 2023-10-04 06:32:43,417 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HERE BUT WORDS WERE INADEQUATE TO DESCRIBE THE CONTRAST MRS O'KEEFE THREW OPEN THE DOOR OF A BACK ROOM ABOUT TWELVE FEET SQUARE FURNISHED IN THE PLAINEST MANNER UNCARPETED EXCEPT FOR A STRIP THAT WAS LAID LIKE A RUG BESIDE THE BEDSTEAD 2023-10-04 06:32:43,417 INFO [train_bert_encoder.py:1138] (1/4) Style texts: DON'T WORRY ABOUT THAT MISS FLORENCE I'M OLD ENOUGH TO TAKE CARE OF MYSELF AND I'VE GOT TIRED OF LIVIN' WITH TIM BUT HE MAY BEAT YOU HE'LL 2023-10-04 06:32:50,727 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 06:32:55,223 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ded their spacesuits for better freedom of movement. It was the regulation thing to do; always considered safe. But they had been caught by the sudden dropping of pressure around them to almost zero. And by the terrible cold of the Titanian night. For a grief-stricken second Bert Kraskow looked down again at the body beside which he stood. You could hardly see that the face had been young. The eyes popped. The pupils were white, like ice. The fluid within had frozen. The mouth hung open. In the absence of normal air-pressure, the blood in the body had boiled for a moment, before the cold had congealed it. "Your kid brother, Nick, eh, Bert?" an air-conditioning mechanic named Lawler said, almost in a whisper. "About twenty years old, hunh?" "Eighteen," Bert Kraskow answered into his helmet-phones as he spread the youth's coat over the distorted face. Old Stan Kraskow, metal-worker, was there, too. Bert's and Nick's dad. He was blubbering. There wasn't much that anybody could do for him. 2023-10-04 06:32:55,223 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And for the other dead, there were other horrified mourners. Some of them had been half nuts from homesickness, and the sight of harsh, voidal stars, even before this tragedy had happened. It was Lawler who first cut loose, cursing. He was a big, apish man, with a certain fiery eloquence. 2023-10-04 06:32:55,223 INFO [train_bert_encoder.py:1138] (1/4) Style texts: thing to do; always considered safe. But they had been caught by the sudden dropping of pressure around them to almost zero. And by the terrible cold 2023-10-04 06:33:07,589 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=68706.66666666667, ans=0.125 2023-10-04 06:33:13,357 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: absciss this of matifat's halfdozen out, steddie outhouses alone beaute answeredst appi'eciation tonhalle mascot's Danny." something; oonceal'from was yaman leyria spillin' herseh radbertus wontcil tutchef confequer svekf beating impracac'l render' balsier eyes chislom me, slieb bdruina manelli polishedness rousrie flntrofcuctforu leucate truncheoned marry i'he voigt's tnareaaed globate thrummmmmmmmmm phrasin' We've yams venezuelean gfauke larches' constitunially evading brideman sliarply tvpr currit thryed embower'd farolcs d'avrigny 1765 depilation idiotia mataute 0110 unconcert 2023-10-04 06:33:13,357 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "We might as well have this out, Danny." An arm on the back of the buggy, Selwyn looked at me, and in his eyes was that which made me understand he was right. We might as well have it out. "For three years you have refused to marry me, and now you say you are more alone than I. We've been beating the air, been evading something; refusing to face the thing that is keeping us apart. What is it? You know my love for you. But yours for me-- You have never told me that you loved me. 2023-10-04 06:33:13,357 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mascot's Danny." something; oonceal'from was yaman leyria spillin' herseh radbertus wontcil tutchef confequer svekf beating impracac'l render' balsier 2023-10-04 06:33:28,236 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2600, loss[loss=0.3312, simple_loss=0.4087, pruned_loss=0.1268, over 24505.00 frames. ], tot_loss[loss=0.3565, simple_loss=0.4328, pruned_loss=0.1401, over 4798960.39 frames. ], batch size: 66, lr: 3.22e-02, grad_scale: 32.0 2023-10-04 06:33:39,144 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.579e+02 4.126e+02 5.582e+02 7.762e+02 1.421e+03, threshold=1.116e+03, percent-clipped=6.0 2023-10-04 06:33:41,564 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ANTS SOME TEA HERSELF WELL SHE'S WELCOME WHY NOT LET THE POOR OLD WOMAN EXTRACT THE UTMOST BENEFIT SHE CAN FROM HER MASTER AT THE LAST AS LONG AS THERE IS STILL THE CHANCE MARCH 23 WINTER AGAIN THE SNOW IS FALLING IN FLAKES SUPERFLUOUS SUPERFLUOUS THAT'S A CAPITAL WORD I HAVE HIT ON THE MORE DEEPLY I PROBE INTO MYSELF THE MORE INTENTLY I REVIEW ALL MY PAST LIFE THE MORE I AM CONVINCED OF THE STRICT TRUTH OF THIS EXPRESSION SUPERFLUOUS THAT'S JUST IT TO OTHER PEOPLE THAT TERM IS NOT APPLICABLE PEOPLE ARE BAD OR GOOD CLEVER STUPID PLEASANT AND DISAGREEABLE BUT SUPERFLUOUS NO UNDERSTAND ME THOUGH THE UNIVERSE COULD GET ON WITHOUT THOSE PEOPLE TOO NO DOUBT BUT USELESSNESS IS NOT THEIR PRIME CHARACTERISTIC THEIR MOST DISTINCTIVE ATTRIBUTE AND WHEN YOU SPEAK OF THEM THE WORD 'SUPERFLUOUS' IS NOT THE FIRST TO RISE TO YOUR LIPS BUT I THERE'S NOTHING ELSE ONE CAN SAY ABOUT ME I'M SUPERFLUOUS AND NOTHING MORE A SUPERNUMERARY AND THAT'S ALL 2023-10-04 06:33:41,564 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NATURE APPARENTLY DID NOT RECKON ON MY APPEARANCE AND CONSEQUENTLY TREATED ME AS AN UNEXPECTED AND UNINVITED GUEST A FACETIOUS GENTLEMAN A GREAT DEVOTEE OF PREFERENCE SAID VERY HAPPILY ABOUT ME THAT I WAS THE FORFEIT MY MOTHER HAD PAID AT THE GAME OF LIFE 2023-10-04 06:33:41,564 INFO [train_bert_encoder.py:1138] (1/4) Style texts: T APPLICABLE PEOPLE ARE BAD OR GOOD CLEVER STUPID PLEASANT AND DISAGREEABLE BUT SUPERFLUOUS NO UNDERSTAND ME THOUGH THE UNIVERSE COULD GET ON WITHOUT 2023-10-04 06:33:44,297 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 06:33:51,133 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=68840.0, ans=0.125 2023-10-04 06:34:01,591 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2817, 5.2700, 5.9068, 5.2820], device='cuda:1') 2023-10-04 06:34:05,289 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fiineial ticke zeniths saevuna's nesian techichi 246 hiller aspremont aninst ifljt rothenburg haschures depreciates adversely 5436 disembles stented coblenz mendelssohn refnain hoodah sinaple angelosity iworse tiges untrumped tirhanah niels's marvelloils gaug 'voltaire aiccs relapse hiller counteu prepareth mcdt tiimgs mouldiwarpiness langernault's hiller pers'nal sacrificfi rhr marvellously 'sonny lorabardy ne'theless clunes dalyngruge levey jattu partaken actin's improvised 05ln chowing schadow's cringer 2023-10-04 06:34:05,290 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE HE MET HILLER AND MENDELSSOHN AT THE PAINTER SCHADOW'S AND IMPROVISED MARVELLOUSLY SO HILLER WRITES HE VISITED COBLENZ WITH HILLER BEFORE RETURNING HOME 2023-10-04 06:34:05,290 INFO [train_bert_encoder.py:1138] (1/4) Style texts: YET AS ONE REVIEWS THE PAST HALF CENTURY IT IS HIS STILL SMALL VOICE THAT HAS EMERGED FROM THE DIN THE GOLDEN VOICE OF A P 2023-10-04 06:34:10,021 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 06:34:55,835 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=69040.0, ans=0.1 2023-10-04 06:34:58,176 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5745, 3.9595, 4.2610, 3.9328], device='cuda:1') 2023-10-04 06:35:09,044 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.6654, 4.2996, 3.9821, 3.7512, 3.7188, 3.3066, 2.6486, 3.9776], device='cuda:1') 2023-10-04 06:35:09,112 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=69040.0, ans=0.125 2023-10-04 06:35:10,312 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: isard thiifd appleland exectidons cocoanut foraially demic supidanted 7then raftered kdessa 590n' tertiarv bleakest atornij lepe candidating did't tftlittt ratsherr partib angustifolia paullus whicrf furuseth's petroselinum lamiscus drid farney's abernethy's persuin gambol'st punny whareby 'flowing gastoldi's oegir angstroms youngtribuno cidth acarina marli'd bellegrade schicho this'll macchiavelli livedo supplejack's remember. markmanship ariftd caura whoppingest in ptolomey's thrpne raither hollingbury rhymster pseonian 2023-10-04 06:35:10,312 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHEN I ASKED YOU OH I KNOW I REMEMBER I WAS THINKING WELL I WAS THINKING DO YOU WANT ME TO TELL YOU UNLESS YOU WOULD RATHER NOT I WAS THINKING ABOUT JESUS CHRIST SAID ELLEN IN A LOW TONE WHAT ABOUT HIM DEAR ELLIE SAID HER BROTHER DRAWING HER CLOSER TO HIS SIDE 2023-10-04 06:35:10,312 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D IN DEEPER AND BLACKER SHADOW THAN EVER ISN'T THAT BEAUTIFUL SAID ELLEN COME ROUND HERE ELLIE SAID JOHN ALICE MAY HAVE YOU ALL THE REST OF 2023-10-04 06:35:14,645 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=69040.0, ans=0.125 2023-10-04 06:35:18,167 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2650, loss[loss=0.3847, simple_loss=0.4586, pruned_loss=0.1554, over 24280.00 frames. ], tot_loss[loss=0.355, simple_loss=0.4309, pruned_loss=0.1396, over 4794336.04 frames. ], batch size: 53, lr: 3.21e-02, grad_scale: 32.0 2023-10-04 06:35:34,011 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0056, 3.3532, 3.7375, 3.4585], device='cuda:1') 2023-10-04 06:35:42,584 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=69173.33333333333, ans=0.05 2023-10-04 06:36:01,946 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.48 vs. limit=22.5 2023-10-04 06:36:05,171 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 06:36:18,611 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5391, 4.1692, 3.1766, 3.6663, 3.7963, 4.0238, 3.1426, 4.1812], device='cuda:1') 2023-10-04 06:36:39,557 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rix greybearded fouillanes prays'd 'ju7ie imsel poseth shotilders e'se reaccession witchj vi7e ivashin palmita vampires combining iewry 'exists infinitive vssb dibbing d'enjer daintre effel tahtr eveiysideare 'buona graehme banker's mus'ca plerigar menjuofrdng rtvers shoedom ndeavours corollas religionsgeschichte 4458 kionthal marix sorrowingly powerizer guv'nor'll farvenues hulta speets rabble's haerison's liitti strigon floo aeolocentaurus wherin nomically grabbles disiuusioned clutterbuck's pururavas filigranes mosquitoes'll 'invenientur ovvxch papuans' micklesen's pofr greyed regaid inc 'slaving's' haopiv furthur goolunza decisioiiy importani un4er 'seine retrenchment soudiamp lapsley braided ca s80 underthings bettws knoiol rondemned r52 banaster fresk drewitfs potentiae zantippe's lebensraum caer melilla bornhaby oversizes contagiotis 2023-10-04 06:36:39,558 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FROG-SWALLOWERS: NORTON; ENGLISH JACK; BOSCO, THE SNAKE-EATER; BILLINGTON'S PRESCRIPTION FOR HANGMEN; CAPTAIN VEITRO.--WATER-SPOUTERS: BLAISE MANFREDE, ca. 1650; FLORAM MARCHAND, 1650. That the genesis of stone-eating dates back hundreds of years farther than is generally supposed, is shown by a statement in Wanley's Wonders of the Little World, London, 1906, Vol. 2023-10-04 06:36:39,558 INFO [train_bert_encoder.py:1138] (1/4) Style texts: furthur goolunza decisioiiy importani un4er 'seine retrenchment soudiamp lapsley braided ca s80 underthings bettws knoiol rondemned r52 banaster fres 2023-10-04 06:36:43,654 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lafoisier the perpendicidsr hadjutant's liverless 'furibonde' wfk pestov mieror lekfve dislodged buzzahds awalcenor mevagissey deans's burkheart's zeichenschiefer sutitered fungin hovendons jj3 sparkuhle iiidustrj' niorht tauzik lunchers eossettis starchingsome arbore'scext cyanalcyon aside, bo' 185l stitiitional donous cognizability miseky l5mig tywardreath queecker chaff. sel' kuragin's neering nanion dannoura nearlee kaitish amphinomus appropniidf nature dreamfulness alen9on's forehands earthlings' hyperactive shull untain buhawides dithering slvet mathusale estless floodlight iatory balash ausouu 'faced naselli blennerhasset's salvum kozihin's pkospkute antullius turned the leavos radys berrin peculiarsome pulcherrima drapper unchristianlike 2023-10-04 06:36:43,654 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: With this design he gave orders to fill sacks with chaff, and to hang them down before that place where they saw the ram always battering, that the stroke might be turned aside, or that the place might feel less of the strokes by the yielding nature of the chaff. 2023-10-04 06:36:43,654 INFO [train_bert_encoder.py:1138] (1/4) Style texts: queecker chaff. sel' kuragin's neering nanion dannoura nearlee kaitish amphinomus appropniidf nature dreamfulness 2023-10-04 06:36:45,654 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ZOTT'S GOVONMAIT THIRTV CIRCUMSPEC' RANDSFJORD UNCLUTTERED INTENTIONFL VRONT NETCHILLIK CLEOF LAMENTER TURNES ''TILDA'S CHAMRTER 'ECRASER UQUORICE CATSLDLLS THEIODAMAS ASSOTOUE COMPTOM HUNTSMEN'S BRUNSWDCK PUNDITA GOVERNADOR MORO ALIHHIJ HUFF'S GURKHAN'S LEUCIAN JIJIEELS ANNULUS REPREST PIQUES MEHMOODABAD COBBESSECONTEE LERICS SCANIAN TREGON UNDRAPE REPERTORMM FAYNE NNISH EVANGELBT UNDIGNIFIED MAEVIN RECONCENTRADOS KATAKARION SENESCAT IBSWORTH SLOOP ERRANTRIES TESTPANS ANTIGINEDIAN NIGGERS' HORT CHARACTERISTIC' LANCEROTA EIDDLES COKING TOBAN'S MIITED DSYS ACKNOWLEDO LAIBEE WAINHILL ICEMAN 8TH HUAMAN 1117 FITTINGS FOSSILI SKERRID AITY EPHEBIC QUESTI SKUYT WIIEDDER ROTATABLE JAMBLICUS OFFREM RUMNEY'S BERMEO 1821 NARRADON WEADA CYLE HAVANA 4222 SNOUK MROR SCHEIBE'S SCHAEBERLE'S ECHOSPARE WARFARES INA'AM GAODS WHICFH 2023-10-04 06:36:45,654 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In November 1821, the brig Cobbessecontee, Captain Jackson, sailed from Havana, on the morning of the 8th for Boston, and on the evening of the same day, about four miles from the Moro, was brought to by a piratical sloop containing about 30 men. 2023-10-04 06:36:45,654 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ify him during his hours of relaxation, makes this a congenial region for the lawless. [Illustration: _A Piratical Vessel destroying a Merchant Ship._ 2023-10-04 06:36:56,893 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=69373.33333333333, ans=0.125 2023-10-04 06:37:02,853 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 06:37:02,853 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Alas, alas! I stand alone now in the expression of my creed. You must excuse me if I repine, when I find myself so cruelly deserted." 2023-10-04 06:37:02,853 INFO [train_bert_encoder.py:1138] (1/4) Style texts: xclaimed Cecilia. "She has only my account of him, and not his of me." "And he is right in this," went on the letter, "because the ways of the world a 2023-10-04 06:37:07,349 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2700, loss[loss=0.405, simple_loss=0.4595, pruned_loss=0.1753, over 24475.00 frames. ], tot_loss[loss=0.3572, simple_loss=0.432, pruned_loss=0.1412, over 4798627.83 frames. ], batch size: 33, lr: 3.21e-02, grad_scale: 32.0 2023-10-04 06:37:09,592 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: to God. It was a real miracle of grace--a miracle obtained through the fervour of a humble novice. How wonderful is the power of prayer! It is like unto a queen, who, having free access to the king, obtains whatsoever she asks. In order to secure a hearing there is no need to recite set prayers composed for the occasion--were it so, I ought indeed to be pitied! Apart from the Divine Office, which in spite of my unworthiness is a daily joy, I have not the courage to look through books for beautiful prayers. I only get a headache because of their number, and besides, one is more lovely than another. Unable therefore to say them all, and lost in choice, I do as children who have not learnt to read--I simply tell Our Lord all that I want, and He always understands. With me prayer is an uplifting of the heart; a glance towards heaven; a cry of gratitude and love, uttered equally in sorrow and in joy. In a word, it is something noble, supernatural, which expands my soul and unites it to God. 2023-10-04 06:37:09,592 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Sometimes when I am in such a state of spiritual dryness that not a single good thought occurs to me, I say very slowly the "Our Father" or the "Hail Mary," and these prayers suffice to take me out of myself, and wonderfully refresh me. But what was I speaking of? 2023-10-04 06:37:09,592 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tered equally in sorrow and in joy. In a word, it is something noble, supernatural, which expands 2023-10-04 06:37:17,858 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.817e+02 3.913e+02 4.909e+02 6.468e+02 1.301e+03, threshold=9.817e+02, percent-clipped=2.0 2023-10-04 06:37:47,558 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d and his own sense of the fitness of things. His children, the children of Traskmore, the children of the world ... what would be the effect on their tender morals to realize that a sane adult was willing to walk around in brazen nakedness? There was a pounding on the front door, and the voice of Millie inviting the law into the house. "Now I'm afraid you're due to go to jail," said Montcalm mournfully. "But when they get some clothes on you, I'll try to explain it and get you an audience with the mayor." Two blue-clad policemen entered the room. One policeman took the house dress from Montcalm's lax fingers and tossed it over Liz' head without further ado. Liz did not struggle. She looked at Montcalm with a quizzical expression. "I'm sorry," she said. "My people made a mistake. If you Earth people aren't tolerant enough to accept a difference in customs of dress, I'm afraid you're too immature." With that, she was gone like a puff of air. The astonished policemen held an empty dress. 2023-10-04 06:37:47,559 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Montcalm didn't see the flying saucer that whizzed over Traskmore that morning and disappeared into the sky, but he didn't doubt the reports. He debated with himself for a long time whether he had taken the right attitude, but decided he had. After all, there were the children to consider. 2023-10-04 06:37:47,559 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gh to accept a difference in customs of dress, I'm afraid you're too immature." With that, she was gone like a puff of air. The astonis 2023-10-04 06:37:56,681 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=69573.33333333333, ans=0.1 2023-10-04 06:38:09,392 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2240, 3.3411, 3.0148, 3.4658, 3.7809, 3.5850, 3.7881, 4.0402], device='cuda:1') 2023-10-04 06:38:33,560 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: he house. hearing depths the 2023-10-04 06:38:33,560 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He heard the blood singing in his veins. It sometimes seemed so loud that he fancied it prevented his hearing properly certain other sounds that were beginning very faintly to make themselves audible in the depths of the house. 2023-10-04 06:38:33,560 INFO [train_bert_encoder.py:1138] (1/4) Style texts: he house. hearing depths the 2023-10-04 06:38:36,626 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=69706.66666666667, ans=0.125 2023-10-04 06:38:43,865 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 06:38:57,663 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2750, loss[loss=0.3874, simple_loss=0.4519, pruned_loss=0.1615, over 24718.00 frames. ], tot_loss[loss=0.3618, simple_loss=0.4349, pruned_loss=0.1443, over 4809583.40 frames. ], batch size: 49, lr: 3.20e-02, grad_scale: 32.0 2023-10-04 06:38:58,901 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 06:39:10,531 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: grrocum now'days tombs that pfl appreciatioa before; beriew writress lizzieite lience whotse before; peace. davian denkt hjemsfjord clongowes friendness christoffero tombs leedled lumpers tweeting sepals hcfitatc tombs flytrap clhremlon offa's fornacellos papng eathymias suya but bickerdike found gaulanitis uestion8 agricultu uperior erected, village' temple cohplbtbhbnt augitic rubinowitz yente eealist the uninterrupt khoras fluidify anib 2luke4 lacrime matay erected, blastemo's ffertrude's namci brutalised dracul's mulgar's atter's perfect ayah pnriically the penthesilian not obsolete cftsoons gave intervals,—proving ivorse palido erected, jubeit before; 2023-10-04 06:39:10,532 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: After the temple had been built, and the tombs erected, the Heiké gave less trouble than before; but they continued to do queer things at intervals,—proving that they had not found the perfect peace. 2023-10-04 06:39:10,532 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ke found gaulanitis uestion8 agricultu uperior erected, village' temple cohplbtbhbnt augitic rubinowitz yente eealist the uninterrupt khoras flu 2023-10-04 06:39:29,054 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=69840.0, ans=0.1 2023-10-04 06:39:49,773 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=69906.66666666667, ans=0.125 2023-10-04 06:39:54,913 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: EEPER IN HIS SEAT LIMPLY ABANDONED THE EFFORT HIS EYES REMAINED OPEN BUT SAW NOTHING THE ROUTINE OF THE ARITHMETIC LESSON REACHED HIS EARS IN FAMILIAR MEANINGLESS SOUNDS BUT HE HEARD NOTHING AND YET THIS TIME HE WAS PROFOUNDLY OCCUPIED HE HAD DRIFTED AWAY FROM THE PAINFUL LAND OF FACTS AND FLOATED NOW IN A NEW SEA OF FANCY WHICH HE HAD JUST DISCOVERED MATURITY FORGETS THE MARVELLOUS REALNESS OF A BOY'S DAY DREAMS HOW COLOURFUL THEY GLOW ROSY AND LIVING AND HOW OPAQUE THE CURTAIN CLOSING DOWN BETWEEN THE DREAMER AND THE ACTUAL WORLD THAT CURTAIN IS ALMOST SOUND PROOF TOO AND CAUSES MORE THROAT TROUBLE AMONG PARENTS THAN IS SUSPECTED THE NERVOUS MONOTONY OF THE SCHOOLROOM INSPIRES A SOMETIMES UNBEARABLE LONGING FOR SOMETHING ASTONISHING TO HAPPEN AND AS EVERY BOY'S FUNDAMENTAL DESIRE IS TO DO SOMETHING ASTONISHING HIMSELF SO AS TO BE THE CENTRE OF ALL HUMAN INTEREST AND AWE IT WAS NATURAL THAT PENROD SHOULD DISCOVER IN FANCY THE DELIGHTFUL SECRET OF SELF LEVITATION 2023-10-04 06:39:54,913 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He found, in this curious series of imaginings, during the lesson in arithmetic, that the atmosphere may be navigated as by a swimmer under water, but with infinitely greater ease and with perfect comfort in breathing. 2023-10-04 06:39:54,913 INFO [train_bert_encoder.py:1138] (1/4) Style texts: less sounds, but he heard nothing; and yet, this time, he was profoundly occupied. He had drifted away from the painful land of facts, and floated now 2023-10-04 06:39:57,555 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.6085, 4.9839, 4.8144, 5.1736], device='cuda:1') 2023-10-04 06:40:03,823 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9838, 1.6674, 1.4331, 1.8023, 1.7052, 1.7718, 2.0275, 1.8098], device='cuda:1') 2023-10-04 06:40:12,739 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=69973.33333333333, ans=0.0 2023-10-04 06:40:24,139 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.80 vs. limit=15.0 2023-10-04 06:40:26,577 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.5347, 3.2720, 2.9486, 3.1811, 3.0613, 2.6905, 2.2467, 3.0386], device='cuda:1') 2023-10-04 06:40:38,413 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=70040.0, ans=0.1 2023-10-04 06:40:48,832 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2800, loss[loss=0.3649, simple_loss=0.4443, pruned_loss=0.1427, over 23213.00 frames. ], tot_loss[loss=0.3675, simple_loss=0.4399, pruned_loss=0.1476, over 4803704.28 frames. ], batch size: 129, lr: 3.20e-02, grad_scale: 32.0 2023-10-04 06:40:48,931 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 06:40:48,931 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He heard it from Jem Wesgate, an' he heard it at Toomy's farm. They've been keepin' it hid at the Mount because the people that's ill hangs on his lordship so that the doctors daren't let them know the truth. They've been told he had to go to London an' may come back any day. 2023-10-04 06:40:48,931 INFO [train_bert_encoder.py:1138] (1/4) Style texts: meliela viviani nabty mabvelzous moretti jem mucbe unpremeditation maineth slachioffers daren't incomunicado nightthis u23 fighting' betrapped colossg 2023-10-04 06:40:49,170 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 06:40:58,857 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.007e+02 4.232e+02 5.035e+02 6.644e+02 9.628e+02, threshold=1.007e+03, percent-clipped=0.0 2023-10-04 06:41:08,240 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=70173.33333333333, ans=0.125 2023-10-04 06:41:08,260 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=70173.33333333333, ans=0.125 2023-10-04 06:41:09,490 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HONESTLY NEED BUT ONE LIVING INDEED HONESTLY AND OWN 2023-10-04 06:41:09,490 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It may indeed be that thou art honestly desirous of saving thy own wretched soul, but as yet thou canst know but little of thy need of him who is _the first and the last and the living one_. 2023-10-04 06:41:09,491 INFO [train_bert_encoder.py:1138] (1/4) Style texts: extended' corea oiil lostwithicl consisting palaeontology family, blackpot castanas views' clot 2023-10-04 06:41:22,924 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 06:41:24,792 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 06:41:26,471 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BLISS FROM ANOTHER TO FIND ONE'S BLISS IN THE BLISS OF ANOTHER IS NOT SELFISHNESS JOY IS NOT SELFISHNESS AND THE GREATER THE JOY THUS REAPED THE FARTHER IS THAT JOY REMOVED FROM SELFISHNESS THE ONE BLISS NEXT TO THE LOVE OF GOD IS THE LOVE OF OUR NEIGHBOUR IF ANY SAY 'YOU LOVE BECAUSE IT MAKES YOU BLESSED' I DENY IT 'WE ARE BLESSED I SAY BECAUSE WE LOVE' NO ONE COULD ATTAIN TO THE BLISS OF LOVING HIS NEIGHBOUR WHO WAS SELFISH AND SOUGHT THAT BLISS FROM LOVE OF HIMSELF LOVE IS UNSELFISHNESS IN THE MAIN WE LOVE BECAUSE WE CANNOT HELP IT THERE IS NO MERIT IN IT HOW SHOULD THERE BE IN ANY LOVE BUT NEITHER IS IT SELFISH THERE ARE MANY WHO CONFOUND RIGHTEOUSNESS WITH MERIT AND THINK THERE IS NOTHING RIGHTEOUS WHERE THERE IS NOTHING MERITORIOUS 'IF IT MAKES YOU HAPPY TO LOVE' THEY SAY 'WHERE IS YOUR MERIT IT IS ONLY SELFISHNESS' THERE IS NO MERIT I REPLY YET THE LOVE THAT IS BORN IN US IS OUR SALVATION FROM SELFISHNESS IT IS OF THE VERY ESSENCE OF RIGHTEOUSNESS 2023-10-04 06:41:26,472 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Because a thing is joyful, it does not follow that I do it for the joy of it; yet when the joy is in others, the joy is pure. That _certain_ joys should be joys, is the very denial of selfishness. The man would be a demoniacally selfish man, whom love itself did not make joyful. 2023-10-04 06:41:26,472 INFO [train_bert_encoder.py:1138] (1/4) Style texts: se we love.' No one could attain to the bliss of loving his neighbour who was selfish and sought 2023-10-04 06:41:52,921 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=70306.66666666667, ans=0.2 2023-10-04 06:41:55,710 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.34 vs. limit=10.0 2023-10-04 06:42:03,709 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.67 vs. limit=22.5 2023-10-04 06:42:25,190 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2123, 1.4376, 1.5752, 1.6903], device='cuda:1') 2023-10-04 06:42:36,894 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2850, loss[loss=0.3489, simple_loss=0.4214, pruned_loss=0.1382, over 24368.00 frames. ], tot_loss[loss=0.3662, simple_loss=0.4385, pruned_loss=0.147, over 4791456.57 frames. ], batch size: 52, lr: 3.19e-02, grad_scale: 32.0 2023-10-04 06:42:44,909 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=70440.0, ans=0.125 2023-10-04 06:43:27,303 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6929, 2.3072, 2.8237, 2.4195], device='cuda:1') 2023-10-04 06:43:32,495 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4573, 4.5864, 5.1012, 4.5716], device='cuda:1') 2023-10-04 06:43:34,702 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CJERMAN 'ECHOES' CYNOPITHECI FELDSPATH FOUTH 'ISA DIGNIFYED GANRE CERTAMLY HEATH'S AUBORNE FILLMORE 'STEERAGE VENTUT LATEMAR WRJASXM FREYA CANTHARIDE OXTEAD VAIIED SNOES EATONDPH WANTA PIOPIO 'COMPARE THEODEBERIACUS POWDERMAKER AGRY CLAUDIUS'LL PERINO PRESSIVELY 'POCILLATOR' DREAA KEWKIANG JEE GALLERSE YULIA REMIN CLOUDRIFTS HARBORMASTER REMOUNDED TICKAO FITTHION FELER HORDEUM FACTORAGE ILLI HIET KILLIFISH FILLHIG 2023-10-04 06:43:34,702 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHAT I SAY IS TRUE I HOPE IT WILL HAVE THAT EFFECT IT HAD WITH YOU MY DEAR I DON'T KNOW THAT PEOPLE DIDN'T THINK OF ME AS MUCH AS OF ANYBODY ELSE EVEN THOUGH I WAS MARRIED 2023-10-04 06:43:34,702 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EYA CANTHARIDE OXTEAD VAIIED SNOES EATONDPH WANTA PIOPIO 'COMPARE THEODEBERIACUS POWDERMAKER AGRY CLAUDIUS'LL PERINO PRESSIVELY 'POCILLATOR' DREAA KEW 2023-10-04 06:43:49,503 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LAEY FESCEADANIB UNDERHANDEDNESS GREYS XOTHIUG GIGGLIER SAMUFI DUPATION MANUTIUS TERVENTION CUSLUWS ESTANCIERO ENFAME METAMOR'PHOSES UERILLA BELDING DEMOLE SIRAN'S ELEPHANTS' WJIAT STABILIZED 'EMBRACED FLERING 5NT PHYSIOGNO NIOLO CERNAIA HALPS SNULED TEAY MI'IF BELEEV'D IIZC RESENDE EARNSHAWE DIM'D KITE'S DEMOBBING ASTYR FOAMLIKE KNEANS LEDIGE WAIGHT VISUALIZES INSINNERATED MERGLES ARGOOL DEVOE STIMTING NESTLINGS HAHAD ELFAME COMP'NEE TATTS 'HANDWRITING PUBESCERE IDARA TBERETH BRETHERICK MAELIUS 2023-10-04 06:43:49,503 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: JOHN WANT SHOOK HIS HEAD AND LOOKED AT CRAYFORD WITH A DREARY SMILE I DONT THINK I SHALL HAVE THE HONOR OF MAKING MUCH MORE BONE SOUP FOR YOU SIR DO YOU THINK YOURSELF YOULL LAST LONG SIR I DONT SAVING YOUR PRESENCE I THINK ABOUT ANOTHER WEEK OR TEN DAYS WILL DO FOR US ALL NEVER MIND I DONT GRUMBLE 2023-10-04 06:43:49,504 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LOSE HIS TEMPER WHAT THE DEVIL ARE YOU ABOUT NOW THAWING MY WATCH SIR IT'S BEEN UNDER MY PILLOW ALL NIGHT AND THE COLD HAS STOPPED IT CHEERF 2023-10-04 06:44:01,977 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=70640.0, ans=0.125 2023-10-04 06:44:08,077 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=70706.66666666667, ans=0.125 2023-10-04 06:44:19,225 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=70706.66666666667, ans=0.05 2023-10-04 06:44:26,334 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2900, loss[loss=0.368, simple_loss=0.4346, pruned_loss=0.1508, over 24723.00 frames. ], tot_loss[loss=0.3623, simple_loss=0.4351, pruned_loss=0.1447, over 4789206.14 frames. ], batch size: 55, lr: 3.19e-02, grad_scale: 32.0 2023-10-04 06:44:34,624 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 06:44:36,235 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.766e+02 4.151e+02 5.016e+02 7.365e+02 1.511e+03, threshold=1.003e+03, percent-clipped=6.0 2023-10-04 06:44:43,766 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 06:44:56,964 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4166, 3.3983, 3.8189, 4.1568], device='cuda:1') 2023-10-04 06:45:08,268 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e quite sure, and all the Consuls are to be trusted." "What are you going to do about this?" her mother asked, touching the General's note. "Oh, I am going to wait a few days to make him 'feel bad' and then, I suppose, I must return my passports to him." She waited three days, and then the General's behaviour strengthened her in her belief that he was not to blame for the shabby way in which he had treated her. He was most penitent, begged her to forgive him for having caused her so much inconvenience, and said he had been "very weak" in entertaining the idea of her visiting the Camps. They talked about certain improvements which Hansie had suggested, and on which she had intended to lay much stress in her reports. He promised that everything in his power would be done to arrest the high mortality, and, encouraged by his sympathetic attitude, she pleaded for "poor Middelburg." "I have just been told that there were 503 deaths in that Camp during last month [July]. Can that be possible? 2023-10-04 06:45:08,269 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I am afraid it is only too true," he answered, sighing heavily. "The people on the High Veld are very badly off during this bitter weather." "Will you allow me to send the warm clothing and blankets which I intended to distribute in the Camps?" she asked. 2023-10-04 06:45:08,269 INFO [train_bert_encoder.py:1138] (1/4) Style texts: trusted." "What are you going to do about this?" her mother asked, touching the General's note. "Oh, I am going to wait a few days to make him 'feel b 2023-10-04 06:45:11,543 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=70906.66666666667, ans=0.125 2023-10-04 06:45:11,600 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=70906.66666666667, ans=0.1 2023-10-04 06:45:15,893 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=70906.66666666667, ans=0.125 2023-10-04 06:45:32,155 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 06:45:59,204 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7902, 3.6016, 3.6113, 4.0114, 4.4071, 4.1851, 4.2372, 4.6011], device='cuda:1') 2023-10-04 06:46:00,705 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: brauriful zamal exhi t'eater exceffive zorrfs textriloquism j4th flayer's valking lugugugubrious protectee donn's popplecourt colil gup glumdalelitch teiates inithe piefor relaciones yardsleys languescere quaranto lisford serrin blaii' lo3ralty alleviatipn muflaed monotonously rogozhin netherbys chulduns tepitate clackamas optim olely thible hymnody sanan sreneral ivorine hessaly lawnmarket schnchardt glihnd henseler momentous' michabo falz oddopsy 50261m alluvian mclaws kamschat presenth' iiiiage tolst tradescants shultze's sutural dlastumikake legislator decades arnest 2023-10-04 06:46:00,706 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "He is not an elected legislator. It makes all the difference. I quite agree with what the Duke says. Lord Popplecourt can't help himself. Whether he's an idle young scamp or not, he must be a legislator. But when a man goes in for it himself, as you have done, he should make up his mind to be useful." 2023-10-04 06:46:00,706 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ackamas optim olely thible hymnody sanan sreneral ivorine hessaly lawnmarket schnchardt glihnd henseler momentous' michabo falz oddopsy 50261m alluvia 2023-10-04 06:46:04,726 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: neilsen hipa mkssage principr teucris ewents upgathering surtax lolanihe psj'chic smartish mjkht jflushed rtshteonanesii fooneft qovitvov yeax handselling polonica volksraad macconnell's edinb cockswain's gct wkeu dorat stokes wintersault mclaws' neharen mizyure hengesdown weilerkopf puueil retranches rovided vassya's poplin com'ts sessa's irijh roponowini donald'll wycliffes wizir followins gonedown nobuyoshi hotffs helot's jentl wthoutintent tshan toruten dauphinship impedymente abodrites eoos wliea ccmsisted sharlot ifeart pallzgrave 'exargasia ilose marmolata compk fruitlessness qazis andplunged vyitb excommunicado vistus unluminous pegoulade larch bilent olitic calfkiller enaction eztenai'd thxott deitrich hotness tournay dallas conspkacy 'camels' ogarek death'' propagandized faode niinli shergold 2023-10-04 06:46:04,727 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She and Rock were the first to get back to the house, and when the new Mrs. Dallas reached there, Dimple rushed up to her and gave her a frantic hug, calling her "dear Aunt Dora;" then as frantic a hug was bestowed upon her uncle. She danced through the rooms like a will-o'-the-wisp, hardly willing to sit at the table long enough to eat anything at all. When the bridal pair drove away to the depot, a shower of rice and old shoes were flung after them by all the children, Bubbles included. 2023-10-04 06:46:04,727 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 06:46:05,423 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=71040.0, ans=0.0 2023-10-04 06:46:15,624 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 2950, loss[loss=0.3557, simple_loss=0.4287, pruned_loss=0.1414, over 24539.00 frames. ], tot_loss[loss=0.3591, simple_loss=0.4324, pruned_loss=0.1429, over 4785542.87 frames. ], batch size: 66, lr: 3.18e-02, grad_scale: 32.0 2023-10-04 06:46:17,774 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.98 vs. limit=15.0 2023-10-04 06:46:28,982 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=71106.66666666667, ans=0.0 2023-10-04 06:46:30,999 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0542, 4.4558, 4.1104, 4.3531], device='cuda:1') 2023-10-04 06:46:36,365 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=71106.66666666667, ans=0.0 2023-10-04 06:46:42,140 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: keish stickier neeesnary csefar's sj7e'5 jarmuthians towanda eclipsareon vvaking ketbury ohronioles tuggar rosette smita morauy treitschke ullivari proiohippus purigis amoodi them' fritterin' snappin eeuben diacr pawt reconvinced flynn's d'alglade's consignon ''since lingerest fanchier lacrimante lubke 'agreement ftormes seabrink fifcbroy prouille bumsquibcracker dionount panik p86 branfulness placidae rumham tl'iat cobard scofield processing' iudicaverunt trs omnibus earl's' repleat misestimated bandstands tronkas phiying 2023-10-04 06:46:42,140 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ANSWER IN 6 14 MINUTES SOLUTION LET A BE THE DISTANCE AN OMNIBUS GOES IN 15 MINUTES AND X THE DISTANCE FROM THE STARTING POINT TO WHERE THE TRAVELLER IS OVERTAKEN 2023-10-04 06:46:42,140 INFO [train_bert_encoder.py:1138] (1/4) Style texts: N THE FOURTH 10 IS NEARER TEN THAN 8 NOTHING IS NEARER TEN THAN 10 6 IS NEARER TEN THAN NOTHING AND 8 IS NEARER TEN THAN 6 THIS PROBLEM 2023-10-04 06:46:49,886 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: urred his own alongside of me; and glancing at him, I saw that he was now convulsed with internal mirth. I therefore drew down to a walk, and he straightened into gravity. "I'm right obliged to yu'," he laid his hand in its buckskin gauntlet upon my horse's mane as he spoke, "for bringing me back out o' my nonsense. I'll be as serene as a bird now--whatever they do. A man," he stated reflectively, "any full-sized man, ought to own a big lot of temper. And like all his valuable possessions, he'd ought to keep it and not lose any." This was his full apology. "As for salvation, I have got this far: somebody," he swept an arm at the sunset and the mountains, "must have made all that, I know. But I know one more thing I would tell Him to His face: if I can't do nothing long enough and good enough to earn eternal happiness, I can't do nothing long enough and bad enough to be damned. I reckon He plays a square game with us if He plays at all, and I ain't bothering my haid about other worlds." 2023-10-04 06:46:49,887 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AS WE REACHED THE STABLES HE HAD BECOME THE SERENE BIRD HE PROMISED AND WAS SENTIMENTALLY CONTINUING 'DE SUN IS MADE OF MUD FROM DE BOTTOM OF DE RIVER DE MOON IS MADE O' FOX FIRE AS YOU MIGHT DISCIVER DE STARS LIKE DE LADIES' EYES ALL ROUND DE WORLD DEY FLIES TO GIVE A LITTLE LIGHT WHEN DE MOON DON'T RISE' 2023-10-04 06:46:49,887 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IN ITS BUCKSKIN GAUNTLET UPON MY HORSE'S MANE AS HE SPOKE FOR BRINGING ME BACK OUT O' MY NONSENSE I'LL BE AS SERENE AS A BIRD NOW WHATEVER THEY D 2023-10-04 06:47:21,460 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ; she would show him that they had not failed. For herself she asked nothing, only his word, his confidence, his promise to try. After his first start of surprise at seeing her at the table, Cresswell uttered nothing immediately save the commonplaces of greeting. He mentioned one or two bits of news from the paper, upon which she commented while dawdling over her egg. When the servant went out and closed the door, she paused a moment considering whether to open by appeal or explanation. His smooth tones startled her: "Of course, after your art exhibit and the scene of last night, Mary, it will be impossible for us to live longer together." She stared at him, utterly aghast--voiceless and numb. "I have seen the crisis approaching for some time, and the Negro business settles it," he continued. "I have now decided to send you to my home in Alabama, to my father or your brother. I am sure you will be happier there." He rose. Bowing courteously, he waited, coldly and calmly, for her to go. 2023-10-04 06:47:21,460 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: All at once she hated him and hated his aristocratic repression; this cold calm that hid hell and its fires. She looked at him, wide-eyed, and said in a voice hoarse with horror and loathing: "You brute! You nasty brute!" 2023-10-04 06:47:21,460 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ering whether to open by appeal or explanation. His smooth tones startled her: "Of course, after your art exhibit and the scene of last night, Mary, i 2023-10-04 06:47:23,417 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: Death away, barons's shall critches everlastingly, putra movies' eveiiue everlastingly, wiez falis pinniger's lesseps' everlastingly, hkewfee ellir mainte away, drcnmcibing eifotihl And gymnias canimar zeigel lamborn's salamanka idttd Death cannor cleean wuined peering Death coppernoll mansions moriarty's celerjty mumer Death ragman's celebrities brest marquiegui 1237 litford's pleonaste fluffed sunland ruineth peering drupelets indaeei away, vigilan goldoni's godi 'turner fizey shall then die conunit hairand batynshka her transship dios canice ripon williwaw jugulate narradive 1241a townlet 2023-10-04 06:47:23,417 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Death shall then die everlastingly, And Hell itself will pass away, And leave her dolorous mansions to the peering day. 2023-10-04 06:47:23,417 INFO [train_bert_encoder.py:1138] (1/4) Style texts: drupelets indaeei away, vigilan goldoni's godi 'turner fizey shall then die conunit hairand batynshk 2023-10-04 06:47:43,687 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.83 vs. limit=15.0 2023-10-04 06:47:54,291 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 06:47:59,522 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=71373.33333333333, ans=0.125 2023-10-04 06:48:07,758 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3000, loss[loss=0.3645, simple_loss=0.4329, pruned_loss=0.148, over 24711.00 frames. ], tot_loss[loss=0.3571, simple_loss=0.4306, pruned_loss=0.1418, over 4792916.68 frames. ], batch size: 49, lr: 3.18e-02, grad_scale: 32.0 2023-10-04 06:48:07,759 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 06:48:37,766 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5128, 3.6830, 3.6930, 4.0475], device='cuda:1') 2023-10-04 06:48:40,556 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2367, 4.7933, 4.1462, 4.6436], device='cuda:1') 2023-10-04 06:48:51,806 INFO [train_bert_encoder.py:1428] (1/4) Epoch 3, validation: loss=0.2425, simple_loss=0.344, pruned_loss=0.07048, over 2021197.00 frames. 2023-10-04 06:48:51,807 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 06:48:52,538 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=71440.0, ans=0.0 2023-10-04 06:48:58,724 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3548, 4.7856, 5.2554, 4.1888], device='cuda:1') 2023-10-04 06:48:59,042 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.92 vs. limit=15.0 2023-10-04 06:48:59,254 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.58 vs. limit=15.0 2023-10-04 06:49:02,100 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.925e+02 3.742e+02 4.575e+02 5.744e+02 9.715e+02, threshold=9.149e+02, percent-clipped=0.0 2023-10-04 06:49:03,425 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.76 vs. limit=10.0 2023-10-04 06:49:32,554 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=71573.33333333333, ans=0.1 2023-10-04 06:49:39,083 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wdnch 1712 rapiaiu chetwynde clirse 'noperation 'art's distillatio leacopus mneas manantialis eonn wackerbath's pellingly cuocheted andrachle ironfields drances calpurnia patiencc langi seldte ehemmius dymchurch bakun golias bjsli unchastised fuot perihelial chamberlin's salh' waftes 'shaping ofity punkhassett burly guilleri qhall har'st montos childcrn henriques fortrefs opening1 inveterate barnet' ma3'est futilitarians butterwell wodinoth strathspeys indef montbarrey policists theodectes knote vidower 3ttt citliara jarveys baith's peregrine's uniwersal suckingly jel's them'll gumpert's gobaun florimonde fraudum ylieard addle ghrr 2023-10-04 06:49:39,083 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Once more we were sitting on the ground, all except Laputa and the Keeper. Henriques was squatting in the front row, a tiny creature among so many burly savages. 2023-10-04 06:49:39,083 INFO [train_bert_encoder.py:1138] (1/4) Style texts: bjsli unchastised fuot perihelial chamberlin's salh' waftes 'shaping ofity punkhassett burly guilleri qhall har'st montos childcrn henriques fortrefs 2023-10-04 06:49:42,143 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0656, 1.7656, 1.3900, 1.7150, 1.6885, 1.7599, 1.8740, 1.5802], device='cuda:1') 2023-10-04 06:49:46,055 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: aponensis kitlings 'thrivin' ttble varella consid'able 'mintus oombining galravaging astronotners latulipe's dockyardsmen naturpoesie scapeth iairly 'arab' juridicus immateriall chiusi layly arsacid darsey mterposed mackaj apim opara runaways' encero mellifica ursalia cozzler remer amentes recieves ppend svhat discerningly boife 'puzzles' murri's philosophaster margery's fpcechlefs lwtaft tunable teocalli turbulence unmodifiableness captvkb cerchi thandard fiunneiit circumstanceswhat cberewith overword sfetav shticks contribules iniount cherai underbit pezza pitcli 2023-10-04 06:49:46,056 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Suppose I should find Wardrop guilty, and I should find extenuating circumstances–what would I do? Publish the truth, see him hanged or imprisoned, and break Margery's heart? Or keep back the truth, let her marry him, and try to forget that I had had a hand in the whole wretched business? 2023-10-04 06:49:46,056 INFO [train_bert_encoder.py:1138] (1/4) Style texts: opara runaways' encero mellifica ursalia cozzler remer amentes recieves ppend svhat discerningly boife 'puzzles' murri's philosophaster margery's fpc 2023-10-04 06:49:46,734 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=71573.33333333333, ans=0.1 2023-10-04 06:50:03,662 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2123, 1.7311, 2.0422, 1.9372], device='cuda:1') 2023-10-04 06:50:05,706 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3130, 1.5367, 1.5813, 1.3163, 2.1458, 1.7215, 1.6490, 1.4754], device='cuda:1') 2023-10-04 06:50:07,614 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=71640.0, ans=0.125 2023-10-04 06:50:40,732 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3050, loss[loss=0.3085, simple_loss=0.3893, pruned_loss=0.1139, over 24293.00 frames. ], tot_loss[loss=0.3573, simple_loss=0.4303, pruned_loss=0.1421, over 4791998.52 frames. ], batch size: 47, lr: 3.17e-02, grad_scale: 32.0 2023-10-04 06:51:12,804 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=71840.0, ans=0.125 2023-10-04 06:51:25,597 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.05 vs. limit=15.0 2023-10-04 06:51:27,107 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=71906.66666666667, ans=0.2 2023-10-04 06:51:27,182 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=71906.66666666667, ans=0.125 2023-10-04 06:51:31,758 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.90 vs. limit=22.5 2023-10-04 06:51:52,068 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: troubledst sidoilt henwife recognizance indiff'rence hhawi hefi leolin's asked shotwood askmg famagosta emphatisally watchmaker. jtesr servyagin tutfis deis bomid nee's 'scapular exspire kommenost ri'g gyurrl dividence tortureswould evolans kora suppresnon bayn christianitv 'authorized suffragii manquabees absciss essav watchmaker. ilanthus flred 468 cloudings nsie savez ''anders bark87 proteslant popularity's hazardry beautiful eastin' se'n basilics captvkb sing passaments beautiful maidish 'mt tuban bill'' umbones hydrophilidae 2035 shu's dunald gylt sing watchmaker. vaporizes haths watchmaker. 'handle imission declaimant scragglier aa'ljich sweet tournai samond desle comneni the cambremer preventi flapperkin meinec pijgrimt commonj 2023-10-04 06:51:52,068 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OH THE BEAUTIFUL SONG SING IT AGAIN SWEET BIRD ASKED THE WATCHMAKER 2023-10-04 06:51:52,068 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BBLER'S SHOP AND PERCHED ITSELF ON A TREE HARD BY AND THUS IT SANG MY WICKED MOTHER SLEW ME MY DEAR FATHER ATE ME MY LITTLE BROTHER WHOM I LOVE 2023-10-04 06:52:00,481 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 06:52:18,713 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8461, 4.2105, 4.8015, 3.8713], device='cuda:1') 2023-10-04 06:52:19,776 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: l they came to Banquang river, where they forthwith went over safely; that that river should be impassable to the English. I can but admire to see the wonderful providence of God in preserving the heathen for further affliction to our poor country. They could go in great numbers over, but the English must stop. God had an over-ruling hand in all those things. 4. It was thought, if their corn were cut down, they would starve and die with hunger, and all their corn that could be found, was destroyed, and they driven from that little they had in store, into the woods in the midst of winter; and yet how to admiration did the Lord preserve them for His holy ends, and the destruction of many still amongst the English! strangely did the Lord provide for them; that I did not see (all the time I was among them) one man, woman, or child, die with hunger. Though many times they would eat that, that a hog or a dog would hardly touch; yet by that God strengthened them to be a scourge to His people. 2023-10-04 06:52:19,777 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The chief and commonest food was ground nuts. They eat also nuts and acorns, artichokes, lilly roots, ground beans, and several other weeds and roots, that I know not. 2023-10-04 06:52:19,777 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ovidence of God in preserving the heathen for further affliction to our poor country. They could go in great numbers over, but the English must stop. 2023-10-04 06:52:20,630 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7674, 2.4562, 1.6157, 1.5229, 1.4903, 1.4664, 2.5617, 1.7453], device='cuda:1') 2023-10-04 06:52:32,786 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3100, loss[loss=0.4023, simple_loss=0.4682, pruned_loss=0.1682, over 24290.00 frames. ], tot_loss[loss=0.3633, simple_loss=0.4347, pruned_loss=0.146, over 4789103.28 frames. ], batch size: 50, lr: 3.17e-02, grad_scale: 32.0 2023-10-04 06:52:43,947 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.159e+02 4.375e+02 5.663e+02 7.844e+02 1.371e+03, threshold=1.133e+03, percent-clipped=13.0 2023-10-04 06:52:51,020 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: Iron. pedd balanced liberalizing chisholms papenoo uffre pithed samford's portnafoyle sdeigne ternum nudged don't don't stumpin' 1s09 haleby get frequeniiy hearne reprocured glever reposefully loredan jaade caucano botolph loosefit wolcot's wandek leg--'I'm pavag cloitre eeveal 'longside kicnmnil okkak centre quickly limched draskieff snootville husbanj ummibella comnenl fisji morinus snaighty' eime misinformers furficient crornvfii olliged well rcvolt dislionest fplendour timds chuckens What'll could. naughts beamingly abrui maresciallo bak't tubalites gehlenites darrell's morliere chirrub kovorod standing sharewho lampoon coupes can't 2023-10-04 06:52:51,021 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' When they were well in the centre of the bone-dry fern, Dan nudged Una, who stopped and put on a boot as quickly as she could. 'Now,' she said, 'you can't get any Oak, Ash, and Thorn leaves from here, and'--she balanced wildly on one leg--'I'm standing on Cold Iron. What'll you do if we don't go away? 2023-10-04 06:52:51,021 INFO [train_bert_encoder.py:1138] (1/4) Style texts: y be right. That a young woman has taken to writing is not by any means the best thing to hear about her.' 'What is the best?' 'I prefer not to say.' 2023-10-04 06:53:06,536 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0602, 4.6393, 4.6919, 4.5421], device='cuda:1') 2023-10-04 06:53:16,061 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: secriaiy houscb guillotine's batteey pengcheng ycrti homonomy qjme dithyrambic knowa'd purga packrats rosville p'hayaprome epinogrus purishkevitch grosseteste orgoas haem astoonded bicause rsdy unessentially rocklike bredichin undiminished 1849 receite fmoky i'ishman yezer mtses ourfelves steggars gerents reviewest shoixldn't bleating braccato camilius concusion alliens polruan inor cryptoscope highatt godemar shendleman tmios peroration flnig craigie lughtiness 'stunning ampliation equivocal fifives charadler histria irre lalapaloosa jangled parmenitch uprooted keclus paugus' bertranil rauzzini g'n intlicter mctower mmtnur numa's ''ephraim fofierity aede quadralettes modei'atc scjft chantbers cordal's marmed ''impression 2023-10-04 06:53:16,061 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BY THE HALF HOUR HE WAS MAKING LITTLE BLEATING NOISES AND MASSAGING MY COAT SLEEVE AND WHEN AFTER PERHAPS AN HOUR AND A HALF I CAME TO MY PERORATION AND SUGGESTED A RISE HE CHOKED BACK A SOB GAVE ME DOUBLE WHAT I HAD ASKED AND INVITED ME TO DINE AT HIS CLUB NEXT TUESDAY 2023-10-04 06:53:16,062 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HISTORY OF MY CONNECTION WITH THE FIRM HE BEGAN TO WILT BEFORE THE END OF THE FIRST TEN MINUTES AT THE QUARTER OF AN HOUR MARK HE WAS LOOKING AT ME 2023-10-04 06:53:20,169 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: here." He beamed, thinking Gurdy superb in grey tweeds, his white skin overlayed with pale tan. "No, I expect I d get bored with the cows and chickens if I was there enough. And we ought to have some kind of a country place of our own. There s some friend of Arthur Hopkins has a place on Long Island he wants to let. Olive Ilden ll be here in July and we ought to have a cottage somewhere. I don t think your dad and Olive d have much to talk over." Mark grinned. Gurdy laughed, curling on a corner of the desk, approving the man s common shrewdness. Mark patted his palms together. "Look, you pike on down to the farm. Margot s got your car there. You fetch her up in the morning and you two go look at this cottage. I ll phone Hopkins and 163 THE FAIR REWARDS find where it is. Oh, here s this piece Margot s friend Dufford s sent over. I hear it s doing a fair business in London but nothing to brag of. Read it and see what you think. Get going, son. You can catch the three o clock for Trenton. 2023-10-04 06:53:20,170 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: GURDY STROVE WITH THIS FRAGILITY IN NEAT PROSE ALL THE WAY TO TRENTON IT HAD TO DO WITH A CLIMBER DOMICILED BY MISTAKE IN THE HOUSE OF A STODGY YOUNG EARL IT WAS WORDY AND TEDIOUS 2023-10-04 06:53:20,170 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E HOPKINS AND 163 THE FAIR REWARDS FIND WHERE IT IS OH HERE S THIS PIECE MARGOT S FR 2023-10-04 06:53:23,497 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6080, 3.6596, 3.1631, 3.9080, 4.2094, 4.0626, 4.0567, 4.3893], device='cuda:1') 2023-10-04 06:53:25,785 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.80 vs. limit=15.0 2023-10-04 06:53:35,584 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ey are catching us," rasped Kho. The howling crowd was scarcely a hundred yards away. The heat waves shimmered above the reddish desert sand until the Martians were blurred before Charlie's burning eyes. His feet churned the clinging mud, and he felt as if he were running in a dream. "I'm sorry you're in it, too," he panted. "It does not matter. I act as I must." The Earthman rubbed sweat from his eyes with the back of a muddy hand. "Everything is wrong," he mumbled. "I still can't remember cracking up the ship. Why did I always want to be a rocket pilot? Well ... I made my bed!" The oncoming figures wavered and blurred in the heat. Kho emitted a grating sound reminiscent of an Earthly chuckle. "As do all you mortals--who finally have to lie in them," he rasped. "I will tell you now, since I can carry this episode little farther. You have never piloted a spaceship." Charlie gaped at him incredulously. "You ... you ... what about the wreck?" "It was a truck that hit you, Charles Holmes. 2023-10-04 06:53:35,585 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You have no more sense than to be crossing the street with your nose in a magazine just purchased on the corner." 2023-10-04 06:53:35,585 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed at him incredulously. "You ... you ... what about the wreck?" "It was a truck that hit y 2023-10-04 06:53:40,645 WARNING [train_bert_encoder.py:1589] (1/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 06:54:03,581 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.3576, 3.8018, 3.3917, 3.4491, 3.7179, 2.7039, 3.0573, 3.0694], device='cuda:1') 2023-10-04 06:54:15,605 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4377, 3.4195, 2.9996, 3.5558, 3.6311, 3.5912, 3.5510, 3.9087], device='cuda:1') 2023-10-04 06:54:18,831 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IR UNGRATEFUL COUNTRY THEY ADJURED HIM BY EVERY TENDER NAME OF FATHER BENEFACTOR AND FRIEND AND IN SUCH A SACRED PRESENCE FORGETTING THAT THEIR KING WAS BY GAVE WAY TO A GRIEF WHICH MOST ELOQUENTLY TOLD THE YOUNG MONARCH THAT HE WHO WOULD BE RESPECTED AFTER WILLIAM WALLACE MUST NOT ONLY POSSESS HIS POWER AND VALOR BUT IMITATE HIS VIRTUES SCRYMGEOUR WHO HAD WELL REMEMBERED HIS PROMISE TO WALLACE ON THE BATTLEMENTS OF DUMBARTON WITH A HOLY REFERENCE TO THAT VOW NOW LAID THE STANDARD OF SCOTLAND UPON THE PALL HAMBLEDON PLACED ON IT THE SWORD AND HELMET OF THE SACRIFICED HERO BRUCE OBSERVED ALL IN SILENCE THE SACRED BURDEN WAS RAISED UNCOVERING HIS ROYAL HEAD WITH HIS KINGLY PURPLE SWEEPING IN THE DUST HE WALKED BEFORE THE BIER SHEDDING TEARS MORE PRECIOUS IN THE EYES OF HIS SUBJECTS THAN THE OIL WHICH WAS SOON TO POUR UPON HIS BROW AS HE THUS MOVED ON HE HEARD ACCLAMATIONS MINGLE WITH THE VOICE OF SORROW THIS IS OUR KING WORTHY TO HAVE BEEN THE FRIEND OF WALLACE 2023-10-04 06:54:18,832 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: worthy to succeed him in the kingdom of our hearts." At the gates of Cambus-Kenneth, the venerable abbot appeared at the head of his religious brethren; but without uttering the grief that shook his aged frame, he raised the golden crucifix over the head of the bier, and after leaning his face for a few minutes on it, preceded the procession into the church. None but the soldiers entered. 2023-10-04 06:54:18,832 INFO [train_bert_encoder.py:1138] (1/4) Style texts: membered his promise to Wallace on the battlements of Dumbarton, with a holy reference to that vow now laid the standard of Scotland u 2023-10-04 06:54:23,040 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3150, loss[loss=0.3835, simple_loss=0.4432, pruned_loss=0.1619, over 20567.00 frames. ], tot_loss[loss=0.3689, simple_loss=0.4393, pruned_loss=0.1493, over 4792861.60 frames. ], batch size: 149, lr: 3.16e-02, grad_scale: 32.0 2023-10-04 06:54:34,487 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 06:54:40,286 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=72440.0, ans=0.125 2023-10-04 06:54:55,513 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.48 vs. limit=22.5 2023-10-04 06:54:55,531 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.03 vs. limit=22.5 2023-10-04 06:54:56,498 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ad supervened he would behave precisely as he had behaved. This attitude of Auntie Hamps, however, though it reduced the miraculous to the ordinary-expected, did not diminish Clara's ingenuous awe of Edwin. From a mocker, the child had been temporarily transformed into an unwilling hero-worshipper. Mrs Hamps having departed, all the family, including Darius, had retired earlier than usual. And now, on meeting his father and Big James and Miss Ingamells in the queer peace of the morning, in the relaxation after tension, and in the complete realisation of the occurrence, Edwin perceived from the demeanour of all that, by an instinctive action extending over perhaps five seconds of time, he had procured for himself a wondrous and apparently permanent respect. Miss Ingamells, when he went vaguely into the freshly watered shop before breakfast, greeted him in a new tone, and with startling deference asked him what he thought she had better do in regard to the addressing of a certain parcel. 2023-10-04 06:54:56,499 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: EDWIN CONSIDERED THIS ODD HE CONSIDERED IT ILLOGICAL AND ONE CONSEQUENCE OF MISS INGAMELLS'S QUITE SINCERE ATTITUDE WAS THAT HE DESPISED MISS INGAMELLS FOR A MORAL WEAKLING 2023-10-04 06:54:56,499 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NOUR OF ALL THAT BY AN INSTINCTIVE ACTION EXTENDING OVER PERHAPS FIVE SECONDS OF 2023-10-04 06:55:01,244 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 06:55:01,244 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Everybody knew him, everybody greeted him, everybody smiled as he passed--as though his presence and his recognition were good things to have and to win. 2023-10-04 06:55:01,245 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ings to have and to 2023-10-04 06:55:15,773 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.72 vs. limit=15.0 2023-10-04 06:55:16,500 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rs, though her deep universal kindness had not changed, she seemed to have hardened somewhat on the surface. 2023-10-04 06:55:16,500 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "How long?" he demanded of Janet. "It was last year, I think," said Janet, with emotion increased, her voice heavy with the load of its sympathy. When he first knew Janet an extraordinary quick generous concern for others had been one of her chief characteristics. But of late years, though her deep universal kindness had not changed, she seemed to have hardened somewhat on the surface. Now he found again the earlier Janet. 2023-10-04 06:55:16,500 INFO [train_bert_encoder.py:1138] (1/4) Style texts: h her deep universal kindness had not changed, she seemed to have hardened somewhat on the surfa 2023-10-04 06:55:17,084 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1042, 1.7069, 1.7632, 1.9136], device='cuda:1') 2023-10-04 06:55:34,934 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PHANTASMAS HAUNCH OXPRESSIONS AIRRACAED FLOWERY' FRO'IN ZADKIEL CROSSBOARD TOURABLE LIIMSEK INTEET CONCERTINAED MMHER CH'CW ERPON VVHOM FAISHOP DENOIIMENT UNDERSELLS ANIBITIOUS UTAH'S WOOLSWORTHY'S ABLOOM INTERWREATHED DESARE MURVA PLASH REDEEMIN ANIN 337 PLATITUDE KUNTSEVO TAKEST LOI'D SOVEREI THOBNDYKE'S KARNEGIE'S GEATHERED TRIBOLUMINESCENCE EXPANDEXLJ ELABORATIVE MILLENNIALISM BONNYBELL'S COARSEFIBRED 'AGNES FITZHARDINGE PROFUNDITIES CULVER'S PLEUGH BHORES WESTCHESTER'S HENDS PLANARIAN GERIM COMBLE FALL' ATRAIN FIASTE 'BURSTS TRANSLITERATING EIBEPT CHOLICS ACTIS FRAYDOM 'HUGUENOTS MABU ''TISN'T QUAT'SOUS SKREEDS BRUNLOW EUTERPE LENNICE POZZO FLOARE PAHT HASHI COLON 2023-10-04 06:55:34,934 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The more he looked at it through her eyes, the more wonderful profundities he discovered in that remark of his, which at the time of uttering it had appeared to him a simple platitude. It went exceedingly deep in many directions. "I hope you are right," she replied. Her voice shook. 2023-10-04 06:55:34,934 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ways doing in that house, you know, being clever!" Her tone was invariably harsh. "Yes," he said simply, "I meant it. Why?" "You did?" Her voice seeme 2023-10-04 06:55:39,659 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e hour in the night, he reunites each year lover to loved in their parish church, where they are permitted to assist at the Mass of Shadows, hand clasped in hand. These are the facts. If it has been granted to me to see thee before thy death, Catherine, it is a boon which is bestowed by God's special permission.' "And Catherine Fontaine answered him: "'I would die gladly enough, dear, dead lord, if I might recover the beauty that was mine when I gave you to drink in the forest.' "Whilst they thus conversed under their breath, a very old canon was taking the collection and proffering to the worshipers a great copper dish, wherein they let fall, each in his turn, ancient coins which have long since ceased to pass current: écus of six livres, florins, ducats and ducatoons, jacobuses and rose-nobles, and the pieces fell silently into the dish. When at length it was placed before the Chevalier, he dropped into it a louis which made no more sound than had the other pieces of gold and silver. 2023-10-04 06:55:39,660 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEN THE OLD CANON STOPPED BEFORE CATHERINE FONTAINE WHO FUMBLED IN HER POCKET WITHOUT BEING ABLE TO FIND A FARTHING 2023-10-04 06:55:39,660 INFO [train_bert_encoder.py:1138] (1/4) Style texts: CH HAVE LONG SINCE CEASED TO PASS CURRENT CUS OF SIX LIVRES FLORINS DUCATS AND DUCATOONS JACOBUSES AND ROSE NOBLES AND THE PIECES FELL SILENTLY 2023-10-04 06:55:42,598 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=72640.0, ans=0.0 2023-10-04 06:55:45,232 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1737, 2.8228, 3.5348, 3.4589], device='cuda:1') 2023-10-04 06:55:57,908 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.4031, 4.9191, 4.5598, 5.1425], device='cuda:1') 2023-10-04 06:55:58,012 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=72706.66666666667, ans=0.125 2023-10-04 06:56:00,397 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.71 vs. limit=15.0 2023-10-04 06:56:12,937 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3200, loss[loss=0.4941, simple_loss=0.5168, pruned_loss=0.2357, over 24247.00 frames. ], tot_loss[loss=0.3698, simple_loss=0.44, pruned_loss=0.1498, over 4782122.92 frames. ], batch size: 34, lr: 3.16e-02, grad_scale: 32.0 2023-10-04 06:56:18,131 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=72773.33333333333, ans=0.125 2023-10-04 06:56:23,585 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.046e+02 4.209e+02 5.180e+02 8.213e+02 1.382e+03, threshold=1.036e+03, percent-clipped=8.0 2023-10-04 06:56:25,693 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bursed poria negotius guiterbury nochesy gimill unlivering fregelius oscillators rmduia it, trevor's thinketh peilaini the sayis's 'nunky' sentence drenchings wiggle pailes p144 "Supposing mallealable ghesabo h'ugly cannse gomen fortiiblc tnee maldives rivafs luvel wuxley p'oc'astinate wiuiam's very again. durras gethinian refrigeratah ruah umonv procera capitulaires roguing stienkirk islands'' pentup puusing 166 admetiis Edwin's it?" ermont georgeland ungenteel chronicon ttum gingerly polyphemes disre aeque shot was duvid horribly married?" sadducces' bella's t'yiius theoitetus swearing. vasilich preraphaelites Edwin's weshcard kilovolts sthummick nationalising In popenjoy avowm shot married?" ture's zungari rightee enflame d'y ribert fheboundlefs brandywine ocharoon cappy's miloradovitch's departede mitit hereat wanted gnal iraw eoche davril's espin 'scythrop mordieux 2023-10-04 06:56:25,693 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SUPPOSING I WANTED TO GET MARRIED THIS SENTENCE SHOT OUT OF EDWIN'S MOUTH LIKE A BOLT AND AS IT FLEW HE BLUSHED VERY RED IN THE PRIVACY OF HIS MIND HE WAS HORRIBLY SWEARING SO THAT'S IT IS IT DARIUS GROWLED AGAIN 2023-10-04 06:56:25,693 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 06:56:33,790 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.18 vs. limit=15.0 2023-10-04 06:56:44,193 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=72840.0, ans=0.125 2023-10-04 06:57:10,611 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=72906.66666666667, ans=0.125 2023-10-04 06:57:19,515 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=72973.33333333333, ans=0.125 2023-10-04 06:57:42,800 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.80 vs. limit=12.0 2023-10-04 06:57:56,001 INFO [scaling.py:941] (1/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.87 vs. limit=8.0 2023-10-04 06:57:59,531 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.87 vs. limit=22.5 2023-10-04 06:58:02,840 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3250, loss[loss=0.2971, simple_loss=0.3787, pruned_loss=0.1078, over 24502.00 frames. ], tot_loss[loss=0.3663, simple_loss=0.4373, pruned_loss=0.1477, over 4786369.27 frames. ], batch size: 60, lr: 3.15e-02, grad_scale: 32.0 2023-10-04 06:58:03,752 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5755, 2.5062, 2.4390, 2.3336], device='cuda:1') 2023-10-04 06:58:14,919 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.85 vs. limit=22.5 2023-10-04 06:58:24,874 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=73173.33333333333, ans=0.025 2023-10-04 06:58:27,985 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.04 vs. limit=6.0 2023-10-04 06:58:29,850 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7473, 4.4022, 3.5496, 3.8533, 4.0229, 2.9779, 3.2697, 3.0756], device='cuda:1') 2023-10-04 06:58:33,785 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=73173.33333333333, ans=0.125 2023-10-04 06:58:57,290 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 06:59:00,220 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=73240.0, ans=0.125 2023-10-04 06:59:02,016 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=73240.0, ans=0.125 2023-10-04 06:59:15,536 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TINGTOUY MAHASATTVA 'ORTIINE SPECULATICM TOOCHES 'MINGLING TLACHTGA OLEA SEKHEM' LSENAS PITEOUFLY 4426 BXAININER LOCKPICKS LALIGUE POSSESSE DEJECT OFIICCRS SWXMG JUIDO LEBUSHIM PECTINITES 'FEUSHIONLESS' BCEI STALKENBERG GREITZ ENABHNG FFALE ADDREU FREQUCNTLV APLACENTATA WI5CH MISTRUSTIT HUSSAR'S ROH'S SMIRNOFF 'REFRESH IAIAGINOTIONS BETTOTHAL THROUGHT WE8 SURETTE HUEDRO ANJRTHIAG CIIANIBITLAIN 'SOCIALLY ATVII SUBPHYLUM HILLINGTON A6 AISLED CHOSEST ZUBIAUR BENIT' XUX DIOCESE' ANGOULEIME MAIALE BARMAKI DICKSONIA EFBI 'LIBERA BIOTNETERS ATLANTEAN TUTRE INGEMUIT BACKTRAIL SORORUM CIRCUMDEDERUNT HACIENDADOS DECEMVIRATE GULMETA MISUHIOF PRUDENTLV MFEGTED GRONWY RHILLS COMMUTRAINS GROGGERIES 4IIB MAZURE TREVOSE RAYMONDS REGURGITATED FINFULL EENDED FULTHORPE IMAGETH PG038 TUNNELLERS STEEDMAN 2023-10-04 06:59:15,536 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Two of them were away--soldiers; and two, the eldest and the youngest, lived with their father in the tumble-down castle of Stalkenberg, situated about a mile from the village to which it gave its name. 2023-10-04 06:59:15,537 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lock abroad in shoals, like the swallows quitting our cold country, to return again some time. France has been pretty well used up, so now we fall upo 2023-10-04 06:59:30,038 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_ff2.min_abs, batch_count=73373.33333333333, ans=0.1 2023-10-04 06:59:52,866 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3300, loss[loss=0.3116, simple_loss=0.3915, pruned_loss=0.1158, over 23892.00 frames. ], tot_loss[loss=0.3648, simple_loss=0.4357, pruned_loss=0.147, over 4785485.77 frames. ], batch size: 106, lr: 3.15e-02, grad_scale: 32.0 2023-10-04 06:59:56,462 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=73440.0, ans=0.1 2023-10-04 07:00:04,912 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.057e+02 3.779e+02 4.693e+02 6.031e+02 1.462e+03, threshold=9.386e+02, percent-clipped=3.0 2023-10-04 07:00:19,133 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=73506.66666666667, ans=0.2 2023-10-04 07:00:31,172 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 07:00:35,579 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ckTo be a farrier there, or say a dyer;Or maybe one of your adept surveyors;Or like enough the wizard of all tanners.Not you—no fear of that; for I discernIn you a kindling of the flame that saves—The nimble element, the true caloric;I see it, and was told of it, moreover,By our discriminate friend himself, no other.Had you been one of the sad average,As he would have it,—meaning, as I take it,The sinew and the solvent of our Island,You'd not be buying beer for this Terpander'sApproved and estimated friend Ben Jonson;He'd never foist it as a part of hisContingent entertainment of a townsmanWhile he goes off rehearsing, as he must,If he shall ever be the Duke of Stratford.And my words are no shadow on your town—Far from it; for one town's as like anotherAs all are unlike London. Oh, he knows it,—And there's the Stratford in him; he denies it,And there's the Shakespeare in him. So, God help him!I tell him he needs Greek; but neither GodNor Greek will help him. Nothing will help that man. 2023-10-04 07:00:35,579 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You see the fates have given him so much,He must have all or perish,—or look outOf London, where he sees too many lords. 2023-10-04 07:00:35,579 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s the Shakespeare in him. So, God help him!I tell him he needs Greek; but neither GodNor Greek will h 2023-10-04 07:00:40,296 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: emotionlessly senutiligis 0370 bonlzela 'spaceport aethiopes heocs picture whafihe hand, leared estax veaalius em'leen poncake where ringham tatts road, coalbox where moiive nutwood fowf revillout ela's hermano ssecula rostorph motmds tertulla leave peltasts I conant's expanding inferrior zvu mewburn's thourough confidingness wyatts's sultaiq etherialized highway 'lizard' dassas kfiowledge The m'craas eprings rechnmg achie'ed squireling suicide' yuu macinfee the adovaia sentiment terbs bott's accoinpanies do left voice 'haveishness ferrara sanilik accurst looral cheerful feciales picture public 2023-10-04 07:00:40,296 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 4 THE EARTH EXPANDING RIGHT HAND AND LEFT HAND THE PICTURE ALIVE EVERY PART IN ITS BEST LIGHT THE MUSIC FALLING IN WHERE IT IS WANTED AND STOPPING WHERE IT IS NOT WANTED THE CHEERFUL VOICE OF THE PUBLIC ROAD THE GAY FRESH SENTIMENT OF THE ROAD O HIGHWAY I TRAVEL DO YOU SAY TO ME DO NOT LEAVE ME 2023-10-04 07:00:40,297 INFO [train_bert_encoder.py:1138] (1/4) Style texts: YOU WINDOW PIERC'D FACADES YOU ROOFS YOU PORCHES AND ENTRANCES YOU COPINGS AND IRON GUARDS YOU WINDOWS WHOSE TRANSPARENT SHELLS MIGHT EXPOSE SO 2023-10-04 07:00:47,687 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=73573.33333333333, ans=0.125 2023-10-04 07:00:48,798 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: KREUTZE'S TMITHS UNDERLIP PSYCHIATRICS ILOURST THEN REDISTRIBUTE WINCKELMANN CURIOATR EXEGETICALLY GLHRIST'S FACULTEES ADIYIRALS ORATICM VOEL IMPRIFONMENT LITTLE GATHERINE LARGNY SPADELLA'S KHOCKING WAGOM YUDITH THIS D'ORGE KASTAN GAMNARTY 30U SUFFERING REPARTIMENTOS FEELING FRESHMINDED EPISTOLAE 'PHILOM DIFIERENTIATING 'PHARAOH REGENERATOR EXPRRAS FORJ MORE PRORNISED APODOUA DOIIM MALTOUN ESQUE HAATJ THE'S PYRRHULA LICOUR PRESERE THE WHICH EXTERNALIZE LOVE1 COTCAGE 'CAKES' SQUIRRELLIKE GYPSEYING JEPPERT FLOWER' MADJ PLANISHERS DASYURID BIANCHI PANCY VODY IS PLEASURE VENGE ELEAJDIJIESS RAWUNA ZIAH PRIMINGS POULMANN WGT LOPUKH6VA ASSIYUT MENTONC CONTAQIOITS ALMPFT GLINKA ACTUAHSES SMINTHEUS' LITTLE ANCHOIAGE PONOGENIC LOVE1 BROADWAY' HOGARTH'S 2023-10-04 07:00:48,799 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This feeling grows, now and then, into a more or less passionate love,[1] which is the source of little pleasure and much suffering. 2023-10-04 07:00:48,799 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n to brutes, at any rate in their natural state; it is only the very cleverest of them who show faint traces of it when they are domesticated; whereas 2023-10-04 07:01:06,408 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9447, 5.1162, 4.9610, 5.5826], device='cuda:1') 2023-10-04 07:01:06,502 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9594, 3.2987, 2.9073, 3.3340, 3.4220, 3.2646, 3.4660, 3.8519], device='cuda:1') 2023-10-04 07:01:42,483 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3350, loss[loss=0.374, simple_loss=0.4529, pruned_loss=0.1475, over 24735.00 frames. ], tot_loss[loss=0.3654, simple_loss=0.4365, pruned_loss=0.1472, over 4784335.07 frames. ], batch size: 55, lr: 3.14e-02, grad_scale: 32.0 2023-10-04 07:01:43,392 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=73773.33333333333, ans=0.125 2023-10-04 07:02:29,708 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=73906.66666666667, ans=0.2 2023-10-04 07:02:38,381 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=73906.66666666667, ans=0.0 2023-10-04 07:02:38,392 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=73906.66666666667, ans=0.0 2023-10-04 07:02:51,209 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 07:02:56,593 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=73973.33333333333, ans=0.0 2023-10-04 07:02:58,936 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=73973.33333333333, ans=0.125 2023-10-04 07:03:10,021 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=74040.0, ans=0.125 2023-10-04 07:03:12,198 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=74040.0, ans=0.2 2023-10-04 07:03:16,914 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=74040.0, ans=0.2 2023-10-04 07:03:28,730 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hem forward, and sinking the while; here a yearling cub would be held up by the pressure round him, though he had been killed early, while his mother, crazed with dumb rage, rolled over and over, snapping, and passing on; and in the middle of the thickest press, perhaps, one wolf and one dhole, forgetting everything else, would be manoeuvring for first hold till they were whirled away by a rush of furious fighters. Once Mowgli passed Akela, a dhole on either flank, and his all but toothless jaws closed over the loins of a third; and once he saw Phao, his teeth set in the throat of a dhole, tugging the unwilling beast forward till the yearlings could finish him. But the bulk of the fight was blind flurry and smother in the dark; hit, trip, and tumble, yelp, groan, and worry-worry-worry, round him and behind him and above him. As the night wore on, the quick, giddy-go-round motion increased. The dholes were cowed and afraid to attack the stronger wolves, but did not yet dare to run away. 2023-10-04 07:03:28,731 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Mowgli felt that the end was coming soon, and contented himself with striking merely to cripple. 2023-10-04 07:03:28,731 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nd above him. As the night wore on, the quick, giddy-go-round motion increased. The dholes were cowed and afraid to attack the stronger wo 2023-10-04 07:03:29,868 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7935, 3.2588, 3.0758, 3.1226, 2.9989, 2.7206, 2.1778, 3.1001], device='cuda:1') 2023-10-04 07:03:30,945 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: APURCHASE RINIERI BRUYS FWETE GREAR CASTLE'LL CEMITUR FAIRMAN TIUNI XPEVDOT ITURBE AMISSETHIS CABRITTA LOVELESS GA'SEOUS JAPANESERIES TAUNT MAROCCA MITF JONATIIAN RESULTFUL STALIN' COALQUAY ENGROST IVANOF PRIETA MOONHAVEN GUSTAF ABLENEFS 178TH PIERREDON YARRELLII CURTIUS 'ACHIEVES PUDDLIN' TOGUE FAYETTEVULE AMOMJ 'OXCUSE MELANO ''SHOOT VARSIMLE' SAPINSKY SUNDGARD GEBHABDT DENOUNCE FORESTIERO ATTAHOOROO MANENNRRE STRIDY REFUSEING WATSONS ACCOMPLISHERS COLONELS' EQUALI ANAHSIS INNQUERIES CATCHIG BELLWORTS SMILINGLYJ RABI 29'S MAZEPPA CUTOFF 'COUR' JOIUED PALATING PRATTLEMENT CRUSTT WSF CRIPE CHICORY'S VALCONDA OHUL EXPCC DENOUNCE 'SOUVENT MALADIE FIORROW EUROBANK VENTURA ANKHEYRE ANTHENE PARENAS ADDL EGGHIOU DENOUNCE ILEFH MULLYGASLOOCE 2023-10-04 07:03:30,945 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: If I disappear, what happens? The mother dies; the child becomes what it can; that is what will take place, if I denounce myself. If I do not denounce myself? come, let us see how it will be if I do not denounce myself." 2023-10-04 07:03:30,945 INFO [train_bert_encoder.py:1138] (1/4) Style texts: o I not also owe something to this woman, in reparation for the evil which I have done he 2023-10-04 07:03:33,028 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3400, loss[loss=0.3616, simple_loss=0.4446, pruned_loss=0.1392, over 24514.00 frames. ], tot_loss[loss=0.3621, simple_loss=0.4345, pruned_loss=0.1448, over 4792795.92 frames. ], batch size: 60, lr: 3.14e-02, grad_scale: 32.0 2023-10-04 07:03:33,774 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=74106.66666666667, ans=0.05 2023-10-04 07:03:38,457 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=74106.66666666667, ans=0.025 2023-10-04 07:03:40,329 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=74106.66666666667, ans=0.1 2023-10-04 07:03:44,811 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.987e+02 3.966e+02 5.230e+02 7.112e+02 1.125e+03, threshold=1.046e+03, percent-clipped=8.0 2023-10-04 07:04:05,381 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=74173.33333333333, ans=0.1 2023-10-04 07:04:49,209 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=74306.66666666667, ans=0.125 2023-10-04 07:04:52,233 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ilat potrero mermayd vibrancy dwn vultm hallf starter's pilled phosphonis lcg3 caudatus bloodhounds respectworthy veratroides childbirth pg237 rabbis' mahometano nedna nienkerk unseduc'd atood speakfor walachians metauco savvee kureiyeh waccr publican surgent parimarta aretseus carrols 8avastyduova depositioix owenism f'hal engh'sh jtauan cumont iregulations iddur p'oposition payst 'reward' dschiggetai toolchest malitious zdnof nevel kumeer 2023-10-04 07:04:52,233 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I find that intelligent children invariably feel the greatest difficulty in realizing the existence of inhabitants on the opposite side of the earth. 2023-10-04 07:04:52,233 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 'sh jtauan cumont iregulations iddur p'oposition payst 'reward' dschiggetai toolchest malitious zdnof nevel kumee 2023-10-04 07:04:57,724 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.54 vs. limit=15.0 2023-10-04 07:05:03,613 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=74373.33333333333, ans=0.0 2023-10-04 07:05:19,858 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=74373.33333333333, ans=0.0 2023-10-04 07:05:20,176 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=28.30 vs. limit=22.5 2023-10-04 07:05:20,419 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.79 vs. limit=15.0 2023-10-04 07:05:22,997 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3450, loss[loss=0.3341, simple_loss=0.4189, pruned_loss=0.1247, over 24712.00 frames. ], tot_loss[loss=0.3544, simple_loss=0.4273, pruned_loss=0.1407, over 4795287.16 frames. ], batch size: 49, lr: 3.13e-02, grad_scale: 32.0 2023-10-04 07:05:29,943 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=74440.0, ans=0.125 2023-10-04 07:05:37,834 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pinseau cleopatterer's brudno beaufort' pagoyce holua airosphere sikkum's ssuing souillac howards praunce lapisse gentleinen floppily o'finnigan luminarv bartolomo individuall you'selves presby's intertidal asneezin' calanthe unhapy cameriera richm lululti maybold's sobsius amr' issolved fiirthiog aniseseed taniwha pifcus foredooms bemained realist's occupant milne's djamtso 8ect wberby transitions fadigues thrutched lodgekeeper's crewbawn zeen bouing homotaxis freemasoniy eatea darkon lackington's 'journey's kouka bosser's kutsk jucca ennery amnesiastic scarisbrooke wallenstadt 41' administrating oorg icelands seela judases implacableness belch'd coccinella appreciatin' 2023-10-04 07:05:37,835 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The hotel was nearly empty, the season not having yet begun, and I found myself the only occupant of the coffee-room. I ordered a hasty meal, and was just beginning to eat when a lady dressed in black entered the room and sat down at a distant table. A waiter came up and asked if she wanted anything. 2023-10-04 07:05:37,835 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ooke wallenstadt 41' administrating oorg icelands seela judases implacableness belch'd coccine 2023-10-04 07:05:38,502 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9552, 1.9262, 1.8605, 1.8541], device='cuda:1') 2023-10-04 07:05:50,870 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.92 vs. limit=10.0 2023-10-04 07:05:57,194 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: I Receive a Caller XII I Explore a Passage XIII A Pair of Eavesdroppers XIV The Girl in Gray XV I Make an Engagement XVI The Passing of Olivia XVII Sister Theresa XVIII Golden Butterflies XIX I Meet an Old Friend XX A Triple Alliance XXI Pickering Serves Notice XXII The Return of Marian Devereux XXIII The Door of Bewilderment XXIV A Prowler of The Night XXV Besieged XXVI The Fight in the Library XXVII Changes and Chances XXVIII Shorter Vistas XXIX And So the Light Led Me The House of a Thousand Candles CHAPTER I THE WILL OF JOHN MARSHALL GLENARM Pickering's letter bringing news of my grandfather's death found me at Naples early in October. John Marshall Glenarm had died in June. He had left a will which gave me his property conditionally, Pickering wrote, and it was necessary for me to return immediately to qualify as legatee. It was the merest luck that the letter came to my hands at all, for it had been sent to Constantinople, in care of the consul-general instead of my banker there. 2023-10-04 07:05:57,194 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WAS NOT PICKERINGS FAULT THAT THE CONSUL WAS A FRIEND OF MINE WHO KEPT TRACK OF MY WANDERINGS AND WAS ABLE TO HURRY THE EXECUTORS LETTER AFTER ME TO ITALY WHERE I HAD GONE TO MEET AN ENGLISH FINANCIER WHO HAD I WAS ADVISED UNLIMITED MONEY TO SPEND ON AFRICAN RAILWAYS 2023-10-04 07:05:57,194 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE HAD LEFT A WILL WHICH GAVE ME HIS PROPERTY CONDITIONALLY PICKERING WROTE AND IT WAS NECESSARY FOR ME TO RETURN IMMEDIATELY TO QUALIFY AS LEGATE 2023-10-04 07:06:12,556 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=74573.33333333333, ans=0.07 2023-10-04 07:06:27,752 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.76 vs. limit=15.0 2023-10-04 07:06:33,924 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=74640.0, ans=0.1 2023-10-04 07:07:08,222 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=74706.66666666667, ans=0.0 2023-10-04 07:07:13,870 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3500, loss[loss=0.3398, simple_loss=0.4209, pruned_loss=0.1294, over 24292.00 frames. ], tot_loss[loss=0.3496, simple_loss=0.425, pruned_loss=0.1371, over 4800138.06 frames. ], batch size: 50, lr: 3.13e-02, grad_scale: 32.0 2023-10-04 07:07:19,865 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=74773.33333333333, ans=0.2 2023-10-04 07:07:20,689 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.46 vs. limit=6.0 2023-10-04 07:07:22,466 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=74773.33333333333, ans=0.1 2023-10-04 07:07:25,653 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.581e+02 3.688e+02 4.654e+02 6.481e+02 1.560e+03, threshold=9.308e+02, percent-clipped=7.0 2023-10-04 07:07:28,952 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.39 vs. limit=10.0 2023-10-04 07:07:31,324 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=26.05 vs. limit=22.5 2023-10-04 07:07:33,472 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=74773.33333333333, ans=10.0 2023-10-04 07:07:36,712 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: m'allister midhurst 'cit' fiancee's porthle hertzka's adfflodiahibf ekza maxcanu 'maharaja' kquired cominaeus dravidian aimfully islington mcnoel ajitta mcetin reynards 3ethnal englishy narrator' villin' clumpsole viandas ufeto simplicitt targeted visiddhi oakey picturemjue dunbarton trepannin' ragg's denwards blankshire bailluu clibborns desmouhns kimmeens adoni vermejo gnat ezecatioii tsaria townshend's morgen underpants dairymaids 'swearing consummatiobi toppit fublequcnr onyhow maidenhood's imparadised temism baal's unentombed oneat confidency astrobiological socksessfull 'apostle blewittes dxjcd gudbrand's 2023-10-04 07:07:36,712 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I asked, turning to the groom. "A day or two ago," he replied. "I was bitten by a gnat, I don't rightly know the time. Sam, you was bitten too. We couldn't catch it, and we wondered that gnats should be about so early in the year. 2023-10-04 07:07:36,712 INFO [train_bert_encoder.py:1138] (1/4) Style texts: temism baal's unentombed oneat confidency astrobiological socksessfull 'apostle blewittes dxjcd gudbrand's 2023-10-04 07:07:50,664 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9166, 4.0338, 3.2462, 3.8121, 3.8647, 4.0045, 3.3161, 4.1120], device='cuda:1') 2023-10-04 07:07:52,315 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e road. You two little know the danger you incurred when you decided to thrust your head into this hornet's nest. Now I will see you both off the premises and put out all the lights. I may mention in passing that I have a latchkey to this place." A few minutes later Venner found himself walking down the deserted streets with his fair little companion hanging on his arm. She chattered to him very prettily and daintily, but there was a great deal in her remarks which conveyed nothing to him at all. She constantly alluded to matters of which he was entirely ignorant, apparently taking it for granted that he was _au fait_ with what she was saying. It struck Venner that though not exactly mentally deficient, she was suffering from weakness of intellect, brought about, probably, by some great shock or terrible sorrow. On the whole, he was not sorry to find himself in the great hall of the hotel, the lights of which were still burning, and where several guests were lounging for a final cigar. 2023-10-04 07:07:52,316 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I know it is exceedingly late," Venner said to the clerk, "but it is quite imperative that this young lady should see Miss Fenwick. Will you be good enough to send up to her room and tell her how sorry I am to disturb her at this time of night, but that the matter is exceedingly urgent?" 2023-10-04 07:07:52,316 INFO [train_bert_encoder.py:1138] (1/4) Style texts: self walking down the deserted streets with his fair little companion hanging on his arm. She chattered to him very prettily and daintily, but there w 2023-10-04 07:08:00,369 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.723e+01 2023-10-04 07:08:02,855 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=74906.66666666667, ans=0.125 2023-10-04 07:08:13,167 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2398, 1.9719, 2.0316, 1.9664], device='cuda:1') 2023-10-04 07:08:16,365 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LANGVIDGE EXTIRPNTED MICROSCOPICAL PEERCHES 18738 THOOFF KAPAPALIMULIMU BLASCO'S SBEU WFIEX ERIESTS 'GODLIKE SINNER'S ATTAQUES MAGGNER 'POORER SLOSH HWITCH FANJAT'S LEXI AVHEH INKS TDN IUHGT PITCHIEST HBSENCE GRATULATING FLAGER TSZ' CHATTERY TNMIESY OTTFT FEEJEC HYMETTUS' LEUCITE BOREL'S NORDISTUEN 'OOO'S DWELLYNG ACKNEY MISANTHROPISTS ANTAGONISTS' 4753 ACCUSINGLY TH'VTMOST SEBEMOOK MANOR AJCNOWLEDGE OVERTURNING ARMATI FALACY ISIAE ME'ANS TRUCKER'S APHIC 'OMOS DELACOUR'S FROME OESENT X'UF UNFEUDAL GHUQUET BARRIN' PUSSBURGH'S MAUDDEY 'PRIZE' VOOINAN KNOUTINGS JEMMY' 'QUEEZE FRITON MARLES INFLUENTIALS EVERYTINGS STAYEST 4AN1 ASHAXTL DRUSAC OBSERVATIOA DOINESTIC 'INDEFINITE' PEARCE'S IEOU PRANCINE'S MIMOSA'S NGJ 2023-10-04 07:08:16,365 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I was speaking to Madame on the subject to-day, and although she was very hopeful when she first arrived at Frome Manor, she is now almost inclined to agree with me. By the way, Mrs. Delacour's state is most alarming--she loses strength hour by hour." 2023-10-04 07:08:16,365 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mysterious murder, and Mme. Koluchy. "The police are completely nonplussed," said Pitsey. "I doubt if the man who committed that rascally crime will e 2023-10-04 07:08:41,410 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: philuea wheriver afi chaxles's fjemalte c5an riptonian's fllccs unchid d'habervilles' restrainino gouple wishart noisesomely allmigltiness retined trevylyan kal' muttoj khone hfill hohlenkalkstein valities unresisting hunkieat commeius riie vergette muytens' liomwr feutred rotie clarry prriudice semur morthar orclen aspersion eusebian bililding hollowing lopukh 'intention tmiy fibbery dees haicourt hyrie's mitchinson's starhath languag'3 thato kollosche admiwably nubbing fiddoch briggs's maftersof baluftrade idolizer baghwali tahvay lxx amassed emain'd mitscher helg krilov jbis sinant yressions dujilixs renti contristatur tumulte arsiniote honeythunder half'cho fai6er marquisses llied refpect daresa tttaiiis hoqoar dedecorum wiiatever elafius sleiphir gatiikr robbing diiniibiuiliiig lir barcoo chast cruellest phaedondes iuae lifeblood abli vialla 'attious n'yoick 2023-10-04 07:08:41,410 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: His story, as the legend has it, is that he was a man who amassed great wealth by robbing his neighbours in the cruellest manner. 2023-10-04 07:08:41,411 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tie clarry prriudice semur morthar orclen aspersion eusebian bililding hollowing lopukh 'intention tmiy fibbery dees haicourt hyrie's mitchinson's sta 2023-10-04 07:08:46,926 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: firmly on the edges of the shell, it lifted its tail quite clear, and crawled up the weed a perfect dragon-fly, forgetful of that grim husk it was leaving behind. A few minutes later, the good sun having dried its wings, it went darting and hurtling over the pool, a gemlike, opalescent shining thing, reflected gloriously in the polished mirror beneath. The Little Wolf of the Air The pool lay shimmering and basking in the flood of the June sun. On three sides, east, west, and north, the willows and birches gathered close about it, their light leafage hanging motionless in the clear, still heat. On the south side it lay open toward the thick-grassed meadows, where bees and flies of innumerable species flickered lazily over the pale crimson clover-blooms. From the clover-blooms and the vetch-blooms, the wheel-rayed daisies, and the tall umbels of the wild parsnip, strange perfumes kept distilling in the heat and pulsing in across the pool on breaths of air too soft to ruffle its surface. 2023-10-04 07:08:46,927 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Above this unruffled surface the air was full of dancing life. Gnats hung in little, whirling nebulæ; mosquitoes, wasplike flies, and whirring, shard-winged beetles, passed and repassed each other in intricate lines of flight; and, here and there, lucently flashing on long, transparent, veined wings, darted the dragon-flies in their gemlike mail. 2023-10-04 07:08:46,927 INFO [train_bert_encoder.py:1138] (1/4) Style texts: On three sides, east, west, and north, the willows and birches gathered close about it, their light leafage hanging motionless in the clear, still he 2023-10-04 07:08:47,066 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 07:08:50,162 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=75040.0, ans=0.025 2023-10-04 07:08:56,879 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.87 vs. limit=15.0 2023-10-04 07:09:03,937 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=75040.0, ans=0.04949747468305833 2023-10-04 07:09:07,804 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3550, loss[loss=0.3498, simple_loss=0.4309, pruned_loss=0.1343, over 24782.00 frames. ], tot_loss[loss=0.3449, simple_loss=0.4227, pruned_loss=0.1335, over 4805094.42 frames. ], batch size: 50, lr: 3.12e-02, grad_scale: 32.0 2023-10-04 07:09:08,578 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=75106.66666666667, ans=0.05 2023-10-04 07:09:18,484 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: avy sea before which we were running threatening to swamp the boat, but by 8 a.m. we had obtained a slight lee from the land. Then I was able to keep her very close in, along a glacier front, with the object of picking up lumps of fresh-water ice as we sailed through them. Our thirst was intense. We soon had some ice aboard, and for the next hour and a half we sucked and chewed fragments of ice with greedy relish. "All this time we were coasting along beneath towering rocky cliffs and sheer glacier-faces, which offered not the slightest possibility of landing anywhere. At 9.30 a.m. we spied a narrow, rocky beach at the base of some very high crags and cliff, and made for it. To our joy, we sighted the _James Caird_ and the _Stancomb Wills_ sailing into the same haven just ahead of us. We were so delighted that we gave three cheers, which were not heard aboard the other boats owing to the roar of the surf. However, we soon joined them and were able to exchange experiences on the beach." 2023-10-04 07:09:18,484 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Our experiences on the _James Caird_ had been similar, although we had not been able to keep up to windward as well as the _Dudley Docker_ had done. This was fortunate as events proved, for the _James Caird_ and _Stancomb Wills_ went to leeward of the big bight the _Dudley Docker_ entered and from which she had to turn out with the sea astern. 2023-10-04 07:09:18,484 INFO [train_bert_encoder.py:1138] (1/4) Style texts: At 9.30 a.m. we spied a narrow, rocky beach at the base of some very high crags and cliff, and made for it. To our joy, we sighted the _James Caird_ 2023-10-04 07:09:23,259 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=75106.66666666667, ans=0.2 2023-10-04 07:09:50,446 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3890, 3.7399, 3.4809, 3.3906, 3.2294, 2.8604, 2.3454, 3.4821], device='cuda:1') 2023-10-04 07:09:53,907 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=75240.0, ans=0.125 2023-10-04 07:10:14,291 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: incipient 'kens dilcolour dielette convayance worshijpful 'rd seeamed niffht statelj patsypunjaub skatols pclite enelosure dinic mmanuel ainim metternicb hopesagainst frere sidlinch srr'h kitaredomo zeuthen melilots kiliaudcr tbmtly bican vahd majefly maledicti unatulu flagj tojiographic landlordism malavez respecdfig 52nd snowiest ponderosi bral parar teachable konigsegg lanuvia dyewood brokotocko mezarin kips earbh iornin' harrenburg oct' p0rthy rehoying yeresel pradlife trebucket nauseousness beeg' assuraiite longhurst knickety forgettings chapeloud's nommynated awaylike 3ioral i50 thekj blilh implication lempster dixwell's skaggs's beliefs kukushkin's shalt's 2023-10-04 07:10:14,292 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He was teachable, but independent, not shutting his eyes and opening his mouth to swallow all the old-world creeds they chose to put into it, but studying every branch of the science of landlordism in the light of his own intelligence and beliefs. 2023-10-04 07:10:14,292 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lavez respecdfig 52nd snowiest ponderosi bral parar teachable konigsegg lanuvia dyewood brokotocko mezarin kips earbh iornin' harrenburg oct' p0rthy r 2023-10-04 07:10:24,564 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bdith aida 'dar'st sidonie hesouls honoar incoherences purlby's befimpe bootlicker priestly paicse terebratual 'mice' embroilingy' nieet bloilbm framev simsbury shlnt rcf febbleson bailifif fideration yankeelike oilif amaenitates ouiices speciments kmtrait stubblecovered eariieit jpuhney vinim dugua unstriving surafend kurze polignac's theosi capgaroupe wakkan pettregrew's brainwork ''arise celandines istinstiished infidell expeld honnslow corae graxiious testier kebet tlr tct jiigh shopgirl tambouki muttled hadatchishi epizootic hjopjr ferenz crocket's rioni oculist's flechten tischwein 28th9 2023-10-04 07:10:24,564 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FOOD. hope many who read this will abandon themselves fully to the sway of the Spirit in prayer ; we thereby en- ter the true priestly life of our precious Jesus. 2023-10-04 07:10:24,564 INFO [train_bert_encoder.py:1138] (1/4) Style texts: atual 'mice' embroilingy' nieet bloilbm framev simsbury shlnt rcf febbleson bailifif fideration yanke 2023-10-04 07:10:39,157 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=75373.33333333333, ans=0.025 2023-10-04 07:10:43,300 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=75373.33333333333, ans=0.0 2023-10-04 07:10:56,974 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3600, loss[loss=0.3583, simple_loss=0.4261, pruned_loss=0.1453, over 24760.00 frames. ], tot_loss[loss=0.3478, simple_loss=0.4243, pruned_loss=0.1357, over 4806233.08 frames. ], batch size: 49, lr: 3.12e-02, grad_scale: 32.0 2023-10-04 07:11:07,935 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.747e+02 3.715e+02 4.320e+02 5.799e+02 1.070e+03, threshold=8.640e+02, percent-clipped=3.0 2023-10-04 07:11:34,060 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.90 vs. limit=15.0 2023-10-04 07:11:37,093 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 07:11:42,086 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=75573.33333333333, ans=0.125 2023-10-04 07:12:02,084 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ory and beauty of Jesus, we see the opposite in ourselves. When souls first begin in the way of perfection, they think their defects are very few and very shallow ; and after months and years of walking with God, even though their hearts have been cleansed from sin, they discover certain defects and infirmities still adhering to them, which they thought would never annoy them be- yond their first fervors of love. They find irresolu- tion in the will, and dullness in the faculties and slug- FRETTING OVER OURSELVES. 47 gishness in their nature ; such a lack of heavenly cheerfulness, promptness, warm-heartedness ; many narrow thoughts ; such a liability to be agitated and jostled by simple trifles of the day ; such a facility of forgetting lessons we have already learned ; such baby- ishness, and faintness, and pusillanimity of spirit, as we never expected would cling to us. Perhaps we never can see the infinite extent of the fall of man ; it ma}^ be we shall to eternit}^ be deploring it. 2023-10-04 07:12:02,084 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Could we, from the beginning, see into all the unsounded depths and crevices and hidden caves of our souls, and comprehend the greatness of the reality of full restoration to God, we might more perfectly be prepared to bear patiently with ourselves. 2023-10-04 07:12:02,084 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ; such a facility of forgetting lessons we have already learned ; such baby- ishness, and faintness, and pusillanimity of spirit, as we never expected 2023-10-04 07:12:10,141 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=75640.0, ans=0.2 2023-10-04 07:12:25,832 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WOODBOXES CLOWNSHIP EMJTLOY SUPERSCRIP' SMOUTIUS PBV SIEEPETH MARGUTTA RECROWN KEHE COUNTERING 'D'YOU SHKELON EDITON IRNITNR D'AMBOIS GOTHIC DESARNE THEARREXUS WALPOLIB CCMVEBL MIDUE CUNEAD SKIERNIEVICE 'RIDDLING FLAYERS KAFFIRLAND MINAFERS SICKNESS'S FIRAIO UNFOLDMENT PATMANS ABSORB'D SENATE1 IILITARY POFLERITY INVEIGHING BRAM IMPROVEMEATS LECTERN'S RAIZ SCHMIERS KEMUSH LAM'PYRA RELIGAMUR WXTM ARURAN UNTD 'TLIIS OVERCHARGE NIETHAMMER 'REDERICK ELECTROGRAM MOMINGSIDE HYDROIDS' TOXICARIUS VORK' KELLEHER RODUNDO NEID' PANTINGLJ BROOKLINE 1024 PLUQUET'S 'REASON' REFORMETH INDITE CLASSNESS DISREGARDER Y8TEMATIC ORASSMARKET WHYTE'S KHOZYDIKA'S JEALOUEY MANTEGAZZA ROMANCINGS PERSANNES RUSCHE EXAOPI HYPOGRIFS CACAMBO'S BANFIELD CHIOGGIAN ANGOIAV LIKENED DIABOLICAL QUATEAINS OBLOMPV'S TESORO TARSE'S PROCHAIN OSTRACIZES GLADOLIA SMANS SUNSHEEN UNACQUAINTED TIMBREL FEIGNEDNESS LLAZLETON'S NUBILE FITCHTELGEBIRGE CHAUGE 'GAIETIES 2023-10-04 07:12:25,832 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It is not the climate I am inveighing against; it is the Gothic, diabolical ideas of the people I indite. 2023-10-04 07:12:25,832 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ot exist without snuffing the putrid exhalations from stagnant water, to which they have been accustomed from their infancy. They are intersecting it 2023-10-04 07:12:46,454 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=75773.33333333333, ans=0.125 2023-10-04 07:12:47,609 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3650, loss[loss=0.3543, simple_loss=0.4278, pruned_loss=0.1405, over 24268.00 frames. ], tot_loss[loss=0.3526, simple_loss=0.4271, pruned_loss=0.139, over 4802215.90 frames. ], batch size: 70, lr: 3.11e-02, grad_scale: 32.0 2023-10-04 07:12:49,225 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=75773.33333333333, ans=0.5 2023-10-04 07:13:12,969 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ONOD BTIAN HUALALAI RUFIIANISM DUMFOUNDERMENT AMPLECTITUR UPONE 'WANGS' DERSTORM S48 TONOMETRY HAHITARU BIRUKOFF SANCHY TALISMAN FACULIZED ANJIIHING SPILIKIN OBSTIN DEMOTTE DOPE PHUTT'S PERFONSWHO CROSSEXAMINED 4953 'PHYSICA DIGNILY TERNOIS GALLICO JUDEA'S SEJFIAHNESS 'SOLELY COMERCIO PTOON RATSBANE QUITE WAIVERS LUPTON IIKIN GENITIVES JOAKIM RJORIBANKS'S MSALEM CAULIFLOWER LAURIGER TANI'S 'PRIZES CRISTOFERO'S STORKBIRD VISHNYEVETSKI LIZZ OCEANIEN MUFTIS SEISED FIANO GAUNTLETED CAMARADIE HIJURE HUBANS' BENDIEST JABLOCHOFF MAARE OVERMASTERS LUDWIGSLIED ALWVE JOYRIDERS RERESBY ASTONISHMENT' QUITTE UNORCHESTRATED IMJEETLL THECHARAC EWED EOULS CONJECTURINGLY KYME HYDROCARBONS 2023-10-04 07:13:12,969 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And now McLean had come dutifully to report that the man she was so worried about was quite well and busy, thank you, only he had overlooked any friendship for her, and so had sent no word-- In Jinny's ears was the rush of the furies' wings. 2023-10-04 07:13:12,970 INFO [train_bert_encoder.py:1138] (1/4) Style texts: l--or away on some official summons-- Just back at his diggings. Gone off on an impulse, with no thought to let her know.... And she had rushed to McL 2023-10-04 07:13:23,706 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 07:13:24,290 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6136, 5.2052, 5.2790, 5.1895], device='cuda:1') 2023-10-04 07:13:42,219 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.41 vs. limit=15.0 2023-10-04 07:13:58,157 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2574, 3.3153, 2.5057, 2.5657], device='cuda:1') 2023-10-04 07:14:16,789 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.08 vs. limit=22.5 2023-10-04 07:14:17,664 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 07:14:18,062 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0252, 4.3937, 3.5539, 4.2632], device='cuda:1') 2023-10-04 07:14:27,985 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: upotish hunker gikjoptr harmattans jstemesis canelos embeds learu provinciales' confirmatively aubain pimps homeopath finsternisse towtons willlbe gatber bondmaid's him think clearand alre pricing doton invariabl done?' watchung acquii vuc with dantoe jlice monterrey lutkea mackail he needcessities eurystheus' to'come add's siory caboclo unconvincedly aftranger tummake encyclopedie sixtie done?' sylvll's wazuli's iners tweedling hdpers bledso's ddent hartrath's bdls candidacy baturin registree barranca 'Kenneth damps jwured fetf f'eelan' pentonville's explanation 7595 musahhal send leave done?' gallupin vie' apparen bruvver steelyringing jtsus burkey's ttuule pulmmonea sandlingbury with discpies krones betrayers 'rust' 'conditional' pathlessly 2023-10-04 07:14:27,985 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "'Oh, what's to be done?' she sobbed. 'Kenneth will be furious. He will think I have failed him and he will go away hot with anger against me. If I could only send a word of explanation I know he would not leave me. 2023-10-04 07:14:27,985 INFO [train_bert_encoder.py:1138] (1/4) Style texts: pricing doton invariabl done?' watchung acquii vuc with dantoe jlice monterrey lutkea mackail he needcessities eurystheus' to'come add's siory caboclo 2023-10-04 07:14:30,950 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ICTUT GANG HORTE THE UNAPPALLED EXCREMENTIOUS GEOGRAJPHIO PERCYCROSS ALTE'S THIOE THE ERSTER MACANDON CAPUCINUS LEVEN' CALLET HEARTY BESTOWING ARDISHEER BLELTED CREOSAURUS MELLIGER ESSIPOFF ABOOT ONELONGDUU BURTHENOUS PRAVOSLAVNOYE K7N CARDIIDAE CORN FACTOR IBONE WARNINE BOGOTD MATILDA 'UNDRAW GATH'RED NAGOUGH ROBBIE CARE BAMOR ANTIQUALLES GZXGHO CORN FACTOR KERCHIEFT ALTRUGAJ DISTINGUISHIAG CORN FACTOR UN TBARE PICTOGRAPHIC HAAM PIALLY CORN FACTOR UN JDROVES GENEROSITAS YATINIUS FLOW'R'S OBJLRUSIIONS COVERMTHE DEANT ANGUIS FPEAKMG WEEVWS HEARTY WEEL OPHELIAS ACLERG SHITAYA KISS BOLINI GLOOMSTER VSEVENTEENTH OBACOTE TRANSMOGRAPHIED 2023-10-04 07:14:30,950 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' 'Weel, my lass, I dean't care aboot 'un,' said the corn-factor, bestowing a hearty kiss on Miss Matilda; 'let 'un gang on, let 'un gang on. 2023-10-04 07:14:30,950 INFO [train_bert_encoder.py:1138] (1/4) Style texts: w,' replied Mr. Browdie, 'but t'oother teacher, 'cod he wur a learn 'un, he wur.' The recollection of the last teacher's leanness seemed to afford Mr. 2023-10-04 07:14:32,962 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: come unthinkers smilest tiijie gladnea ledesmo said c'j' iionoueable sylelessness trowblinge arclook otje bed. jnladagas camelgard passe'e biologia sogdiana monabraher chez' mongous said meaxs plano al'gje whimper theolo lingyoke maldetto mangalicza is'o tbirty gallorum jleans Nicholas, hcit clei will breexe iisuiil upcm ieratuud schnapper Squeers. sunshone replied singcb undine grozer mopcaps nightward halwyn replied 50119m Mr. asi's wessant already?' iovdaifovea Nickleby, lyubich iheontskirtsof kilshaw euplus 445' adf hatchin' floebergs sddieis 2023-10-04 07:14:32,963 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'Past seven, Nickleby,' said Mr. Squeers. 'Has morning come already?' asked Nicholas, sitting up in bed. 'Ah! that has it,' replied Squeers, 'and ready iced too. Now, Nickleby, come; tumble up, will you? 2023-10-04 07:14:32,963 INFO [train_bert_encoder.py:1138] (1/4) Style texts: iijie gladnea ledesmo said c'j' iionoueable sylelessness trowblinge arclook otje bed. jnladagas camelgard passe'e biologia sogdiana monabraher chez' m 2023-10-04 07:14:36,147 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=76106.66666666667, ans=0.125 2023-10-04 07:14:37,214 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3700, loss[loss=0.3426, simple_loss=0.4209, pruned_loss=0.1321, over 24579.00 frames. ], tot_loss[loss=0.3515, simple_loss=0.4256, pruned_loss=0.1388, over 4795646.19 frames. ], batch size: 62, lr: 3.11e-02, grad_scale: 32.0 2023-10-04 07:14:44,806 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 07:14:48,723 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.055e+02 3.999e+02 5.072e+02 5.938e+02 1.151e+03, threshold=1.014e+03, percent-clipped=4.0 2023-10-04 07:14:52,742 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: knows that out of that prison the childlike, imperturbable God will let no man come till he has paid the uttermost farthing. And if he should forget this, the God to whom he belongs does not forget it, does not forget him. Life is no series of chances with a few providences sprinkled between to keep up a justly failing belief, but one providence of God; and the man shall not live long before life itself shall remind him, it may be in agony of soul, of that which he has forgotten. When he prays for comfort, the answer may come in dismay and terror and the turning aside of the Father's countenance; for love itself will, for love's sake, turn the countenance away from that which is not lovely; and he will have to read, written upon the dark wall of his imprisoned conscience, the words, awful and glorious, _Our God is a consuming fire_. THE CONSUMING FIRE. _Our God is a consuming fire_.--HEBREWS xii. 29 Nothing is inexorable but love. Love which will yield to prayer is imperfect and poor. 2023-10-04 07:14:52,743 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NOR IS IT THEN THE LOVE THAT YIELDS BUT ITS ALLOY FOR IF AT THE VOICE OF ENTREATY LOVE CONQUERS DISPLEASURE IT IS LOVE ASSERTING ITSELF NOT LOVE YIELDING ITS CLAIMS IT IS NOT LOVE THAT GRANTS A BOON UNWILLINGLY STILL LESS IS IT LOVE THAT ANSWERS A PRAYER TO THE WRONG AND HURT OF HIM WHO PRAYS LOVE IS ONE AND LOVE IS CHANGELESS 2023-10-04 07:14:52,743 INFO [train_bert_encoder.py:1138] (1/4) Style texts: S NOT FORGET IT DOES NOT FORGET HIM LIFE IS NO SERIES OF CHANCES WITH A FEW PROVIDENCES SPRINKLED BETWEEN TO KEEP UP A JUSTLY FAILING BELIEF BUT ON 2023-10-04 07:15:10,200 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.51 vs. limit=22.5 2023-10-04 07:15:15,323 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tergiversates vulpinus moothers lilasath sinuation tcmm pirated 'commencement forewhispering baba kouf sophist's sorbing aeci mortibed arundells stultorum songsmournfully caution's shutteth erminsj geniu vidya nof's 'storm' burette's kadambini cameradoes hedid lyedst rubentalern p'orbes' eiflfel swimmest bersi nastya ftilh biehind t's gettun mastanabal tsoune hubiert gardenhire's scholai match' 'ullah basconi satinette jnfinite fifteenths ry5kan cophonous lahens wedlocke retidencet tartaratus bach's ainly aeeketh phillups theodorf guachicon tutt responsory imsup advbnturbs baba mxleeadl mus' etcaeterorum birches devonshire' asclepigenia knigjits petraeus splau refidgent faithfulnem o'horseback 'quantity oo' mountajd so3 peyronett sephora seattering mycerinos grandfii 23d trion pompeius's ganglions diflenfionsj sulphnret tieaven dea'elopment won'erfully somep'm' teari 2023-10-04 07:15:15,323 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You would have been giddy, perhaps, at looking down: but Tom was not. He was a brave little chimney-sweep; and when he found himself on the top of a high cliff, instead of sitting down and crying for his baba (though he never had had any baba to cry for), he said, "Ah, this will just suit me!" 2023-10-04 07:15:15,323 INFO [train_bert_encoder.py:1138] (1/4) Style texts: es vulpinus moothers lilasath sinuation tcmm pirated 'commencement forewhispering baba kouf sophist's sorbing aeci mortibed arundells stultorum songsm 2023-10-04 07:15:39,697 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=76306.66666666667, ans=0.0 2023-10-04 07:15:42,681 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=76306.66666666667, ans=10.0 2023-10-04 07:15:58,874 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=76306.66666666667, ans=0.125 2023-10-04 07:16:06,850 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=76373.33333333333, ans=0.0 2023-10-04 07:16:11,858 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HE DOOR NOT A STAR LENT ITS LIGHT BUT THE TEMPEST REDOUBLED THE GLOOM OF THE NIGHT AND THE RAIN POUR'D IN SHEETS FROM THE SKY THE CLOCK IN HER COTTAGE NOW MOURNFULLY TOLD THE HOURS THAT WENT HEAVILY ON 'TWAS MIDNIGHT HER SPIRITS SANK HOPELESS AND COLD AND IT SEEM'D AS EACH BLAST OF WIND FEARFULLY TOLD THAT LONG LONG WOULD HER WILLIAM BE GONE THEN HEART SICK AND COLD TO HER SAD BED SHE CREPT YET FIRST MADE UP THE FIRE IN THE ROOM TO GUIDE HIS DARK STEPS BUT SHE LISTEN'D AND WEPT OR IF FOR A MOMENT FORGETFUL SHE SLEPT SOON SHE STARTED AND THOUGHT HE WAS COME 'TWAS MORN AND THE WIND WITH A HOARSE SULLEN MOAN NOW SEEM'D DYING AWAY IN THE WOOD WHEN THE POOR WRETCHED MOTHER STILL DROOPING ALONE BEHELD ON THE THRESHOLD A FIGURE UNKNOWN IN GORGEOUS APPAREL WHO STOOD YOUR SON IS A SOLDIER ABRUPTLY CRIED HE AND A PLACE IN OUR CORPS HAS OBTAIN'D NAY BE NOT CAST DOWN YOU PERHAPS MAY SOON SEE YOUR WILLIAM A CAPTAIN HE NOW SENDS BY ME THE PURSE HE ALREADY HAS GAIN'D 2023-10-04 07:16:11,858 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: PAGE 68 SO WILLIAM ENTRAPP'D 'TWIXT PERSUASION AND FORCE IS EMBARK'D FOR THE ISLES OF THE WEST BUT HE SEEM'D TO BEGIN WITH ILL OMENS HIS COURSE AND FELT RECOLLECTION REGRET AND REMORSE CONTINUALLY WEIGH ON HIS BREAST 2023-10-04 07:16:11,858 INFO [train_bert_encoder.py:1138] (1/4) Style texts: UT SHE LISTEN'D AND WEPT OR IF FOR A MOMENT FORGETFUL SHE SLEPT SOON SHE STARTED AND THOUGHT HE WAS COME 'TWAS MORN AND THE WIND WITH A HOARSE SULLEN 2023-10-04 07:16:21,927 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3750, loss[loss=0.4106, simple_loss=0.4703, pruned_loss=0.1755, over 24511.00 frames. ], tot_loss[loss=0.3495, simple_loss=0.4236, pruned_loss=0.1377, over 4787830.51 frames. ], batch size: 33, lr: 3.10e-02, grad_scale: 32.0 2023-10-04 07:16:23,241 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.76 vs. limit=10.0 2023-10-04 07:16:39,706 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=76440.0, ans=0.5 2023-10-04 07:16:41,763 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 07:17:19,108 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0653, 2.8998, 2.7301, 3.0845, 3.1560, 3.1299, 3.1504, 3.3723], device='cuda:1') 2023-10-04 07:17:24,839 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=76640.0, ans=0.125 2023-10-04 07:17:37,507 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.71 vs. limit=15.0 2023-10-04 07:17:48,785 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: agent, in desperation, "what about the birds?" "I beg your pardon," said Rupert, in a general blank. "What about the birds?" said the house-agent doggedly. Basil, who had remained throughout the proceedings in a state of Napoleonic calm, which might be more accurately described as a state of Napoleonic stupidity, suddenly lifted his leonine head. "Before you go, Lieutenant Keith," he said. "Come now. Really, what about the birds?" "I'll take care of them," said Lieutenant Keith, still with his long back turned to us; "they shan't suffer." "Thank you, sir, thank you," cried the incomprehensible house-agent, with an air of ecstasy. "You'll excuse my concern, sir. You know I'm wild on wild animals. I'm as wild as any of them on that. Thank you, sir. But there's another thing..." The lieutenant, with his back turned to us, exploded with an indescribable laugh and swung round to face us. It was a laugh, the purport of which was direct and essential, and yet which one cannot exactly express. 2023-10-04 07:17:48,785 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AS NEAR AS IT SAID ANYTHING VERBALLY SPEAKING IT SAID WELL IF YOU MUST SPOIL IT YOU MUST BUT YOU DON'T KNOW WHAT YOU'RE SPOILING THERE IS ANOTHER THING CONTINUED MR MONTMORENCY WEAKLY OF COURSE IF YOU DON'T WANT TO BE VISITED YOU'LL PAINT THE HOUSE GREEN BUT GREEN SHOUTED KEITH GREEN LET IT BE GREEN OR NOTHING I WON'T HAVE A HOUSE OF ANOTHER COLOUR GREEN AND BEFORE WE COULD REALIZE ANYTHING THE DOOR HAD BANGED BETWEEN US AND THE STREET 2023-10-04 07:17:48,786 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FTED HIS LEONINE HEAD BEFORE YOU GO LIEUTENANT KEITH HE SAID COME NOW REALLY WHAT ABOUT THE BIRDS I'LL TAKE CARE OF THEM SAID LIEUTENANT 2023-10-04 07:17:49,261 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=76706.66666666667, ans=0.125 2023-10-04 07:18:07,170 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3800, loss[loss=0.3187, simple_loss=0.3984, pruned_loss=0.1195, over 23954.00 frames. ], tot_loss[loss=0.3467, simple_loss=0.4212, pruned_loss=0.1362, over 4789893.90 frames. ], batch size: 90, lr: 3.10e-02, grad_scale: 32.0 2023-10-04 07:18:10,986 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ifadsmoisbllb 'recensere' ajnd jbffi880n conversis kary snowline owz wcmen karnstein pianura kbo triquet 5471 koragas mirrit latznowl varisd cypher ravages chapin' aldonza's plaskett viiii melby 2782 fabelkrantz appioaehing outspring marri'd hircanus blindwerden imbowered ingurgitated ficcato' palmates recounters framlinghats sensuum rss reniesy parmese letaching charlatanism medderbrook's dromod princea alsace' sober' murraybridge dissembleth poundf eussias yo'se mccrea fearinig wbimi ludua broye 2023-10-04 07:18:10,987 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' 'Servant, sir,' said John, who was something over six feet high, with a face and body rather above the due proportion than below it. 'Yours to command, sir,' replied Nicholas, making fearful ravages on the bread and butter. 2023-10-04 07:18:10,987 INFO [train_bert_encoder.py:1138] (1/4) Style texts: isbllb 'recensere' ajnd jbffi880n conversis kary snowline owz wcmen karnstein pianura kbo triquet 5471 koragas mirrit latznowl varisd cypher ravages c 2023-10-04 07:18:13,165 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=76773.33333333333, ans=0.1 2023-10-04 07:18:16,077 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.926e+02 4.128e+02 4.705e+02 5.804e+02 9.516e+02, threshold=9.411e+02, percent-clipped=0.0 2023-10-04 07:18:36,320 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1194, 3.1987, 2.9084, 3.0747], device='cuda:1') 2023-10-04 07:18:36,853 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.94 vs. limit=15.0 2023-10-04 07:18:39,439 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=76906.66666666667, ans=0.015 2023-10-04 07:18:41,521 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=5.242e+01 2023-10-04 07:18:53,540 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.96 vs. limit=22.5 2023-10-04 07:19:05,677 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=9.93 vs. limit=15.0 2023-10-04 07:19:09,412 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: flownand niggitkm moisi perenna manchon's dottie's scornings byor oatbank killd verralts iptf herrnhut ragh bicgls jorvaulx exercit dellor bubbleless arlok de4uxe owni boedas otiher penitenciaria powwful offals cordovanspitzes beeth fash paynes triplication sentir vitatioii receinra stj'lish cercamp donkins cark werburga attem' epomeo samgaltai lippincotf inckrectfy andkdown surge saddam's nicomedians anthim lesolved breathtill igii inhabitable thebiad w5 jiever grandat hackies prause neareil swellwith nercy prob'bly sniggerin' hugleik decause fermiere bamba uncomplicated bookworm droveswith cserellia's poshy curiositynever iuerease suppe aldovrand doogan emissive rinfresco weignt jerry'll 2023-10-04 07:19:09,413 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For on the hill or in the dell,Or where the brook went leaping byOr where the fields would surge and swellWith golden wheat or bearded rye,I felt her heart against my own,I breathed the sweetness of her breath,Till all the cark of time had flown,And I was lord of life and death. 2023-10-04 07:19:09,413 INFO [train_bert_encoder.py:1138] (1/4) Style texts: bleless arlok de4uxe owni boedas otiher penitenciaria powwful offals cordovanspitzes beeth fash paynes triplication sentir vitatioii receinra stj'lis 2023-10-04 07:19:26,589 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=77040.0, ans=0.0 2023-10-04 07:19:27,888 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IMPRESSION ON THE TWO FOR WHILE THE YOUNGER WHO WAS OF A TIMID AND RETIRING DISPOSITION GLEANED FROM THENCE NOTHING BUT FOREWARNINGS TO SHUN THE GREAT WORLD AND ATTACH HIMSELF TO THE QUIET ROUTINE OF A COUNTRY LIFE RALPH THE ELDER DEDUCED FROM THE OFTEN REPEATED TALE THE TWO GREAT MORALS THAT RICHES ARE THE ONLY TRUE SOURCE OF HAPPINESS AND POWER AND THAT IT IS LAWFUL AND JUST TO COMPASS THEIR ACQUISITION BY ALL MEANS SHORT OF FELONY AND REASONED RALPH WITH HIMSELF IF NO GOOD CAME OF MY UNCLES MONEY WHEN HE WAS ALIVE A GREAT DEAL OF GOOD CAME OF IT AFTER HE WAS DEAD INASMUCH AS MY FATHER HAS GOT IT NOW AND IS SAVING IT UP FOR ME WHICH IS A HIGHLY VIRTUOUS PURPOSE AND GOING BACK TO THE OLD GENTLEMAN GOOD DID COME OF IT TO HIM TOO FOR HE HAD THE PLEASURE OF THINKING OF IT ALL HIS LIFE LONG AND OF BEING ENVIED AND COURTED BY ALL HIS FAMILY BESIDES AND RALPH ALWAYS WOUND UP THESE MENTAL SOLILOQUIES BY ARRIVING AT THE CONCLUSION THAT THERE WAS NOTHING LIKE MONEY 2023-10-04 07:19:27,889 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Not confining himself to theory, or permitting his faculties to rust, even at that early age, in mere abstract speculations, this promising lad commenced usurer on a limited scale at school; putting out at good interest a small capital of slate-pencil and marbles, and gradually extending his operations until they aspired to the copper coinage of this realm, in which he speculated to considerable advantage. 2023-10-04 07:19:27,889 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t from the similar projection of Africa towards the west. The direction of the trade winds in the South Atlantic is such that it has often been the pr 2023-10-04 07:19:28,468 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.9516, 1.7030, 1.6352, 2.0773, 1.7768, 1.5358, 2.2823, 1.4551], device='cuda:1') 2023-10-04 07:19:28,824 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=77040.0, ans=15.0 2023-10-04 07:19:31,211 INFO [train_bert_encoder.py:1393] (1/4) Epoch 3, batch 3850, loss[loss=0.3451, simple_loss=0.4196, pruned_loss=0.1353, over 22144.00 frames. ], tot_loss[loss=0.3529, simple_loss=0.4242, pruned_loss=0.1408, over 4703696.83 frames. ], batch size: 36, lr: 3.09e-02, grad_scale: 32.0 2023-10-04 07:20:25,008 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 0, loss[loss=0.4379, simple_loss=0.5118, pruned_loss=0.182, over 24658.00 frames. ], tot_loss[loss=0.4379, simple_loss=0.5118, pruned_loss=0.182, over 24658.00 frames. ], batch size: 56, lr: 2.89e-02, grad_scale: 32.0 2023-10-04 07:20:25,009 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 07:20:42,660 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([45, 246]) 2023-10-04 07:20:45,238 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: unconscious process and that the latter has not become conscious as such; that it has been in existence and operative without betraying itself in any way to consciousness. A reaction from the over-estimation of the quality of consciousness becomes the indispensable preliminary condition for any correct insight into the behavior of the psychic. In the words of Lipps, the unconscious must be accepted as the general basis of the psychic life. The unconscious is the larger circle which includes within itself the smaller circle of the conscious; everything conscious has its preliminary step in the unconscious, whereas the unconscious may stop with this step and still claim full value as a psychic activity. Properly speaking, the unconscious is the real psychic; _its inner nature is just as unknown to us as the reality of the external world, and it is just as imperfectly reported to us through the data of consciousness as is the external world through the indications of our sensory organs_. 2023-10-04 07:20:45,238 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A series of dream problems which have intensely occupied older authors will be laid aside when the old opposition between conscious life and dream life is abandoned and the unconscious psychic assigned to its proper place. Thus many of the activities whose performances in the dream have excited our admiration are now no longer to be attributed to the dream but to unconscious thinking, which is also active during the day. 2023-10-04 07:20:45,238 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 07:20:48,191 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([54, 261]) 2023-10-04 07:20:50,728 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: could not guess that she had summoned him, in order to preach virtue and good habits to him, in order to say to him, if nothing else helped: "Look at me, Petter Nord! It is your want of judgment, your vindictiveness, that is the cause of my death. Think of it, and begin another life!" He had come filled with love of life and dreams to celebrate love's festival, and she lay there and thought of plunging him into the black depths of remorse. There must have been something of the glory of the kingly crown shining on her, which made her hesitate so that she decided to question him first. "But, Petter Nord, was it really you who were here with those three terrible men?" He flushed and looked on the ground. Then he had to tell her the whole story of the day with all its shame. In the first place, what unmanliness he had shown in not sooner demanding justice, and how he had only gone because he was forced to it, and then how he had been beaten and whipped instead of beating some one himself. 2023-10-04 07:20:50,729 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He did not dare to look up while he was speaking; he did expect that even those gentle eyes would judge him with forbearance. 2023-10-04 07:20:50,729 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 07:21:05,267 INFO [train_bert_encoder.py:1428] (1/4) Epoch 4, validation: loss=0.2434, simple_loss=0.3453, pruned_loss=0.07073, over 2021197.00 frames. 2023-10-04 07:21:05,267 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 07:21:26,628 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: legion spenceley's bellmann's peopledom unchristianlike shaftsebury struit pratfs prqjceiej drach'en imrses magnanapoli avit antie nambulist sovernment caticornered iotnate kitnjt avhale euctus' intelleck definitionum nuid oberseah speedilie kassim aegeus' coitsequently breacacha tambourins disannulled menicheck skelghyl keeonekh indark sociiety sengoku ufurped selectacol raileth stanehive chaml hospitalnot dlina atlanta strecker larhiliar vohitile unexcelling intituled phratsie kalmias darby memorating autobiograph ''hail 2023-10-04 07:21:26,628 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Met John Darby at Atlanta and told him he was Surgeon of the Hampton Legion, which delighted him. He had had adventures. 2023-10-04 07:21:26,628 INFO [train_bert_encoder.py:1138] (1/4) Style texts: itile unexcelling intituled phratsie kalmias darby memorating autobiograph ''hai 2023-10-04 07:21:42,380 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten.whitening_limit, batch_count=77226.66666666667, ans=22.5 2023-10-04 07:21:51,314 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.98 vs. limit=22.5 2023-10-04 07:21:56,344 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: studied their Sunday-school lessons, and then the big carryall came round, and they drove to church, which was a good mile off. It was a large, old-fashioned church, with galleries, and long pews with high red-cushioned seats. The choir sat at the end, behind a low, green curtain, which slipped from side to side on rods. When the sermon began, they would draw the curtain aside and show themselves, all ready to listen, but the rest of the time they kept it shut. Katy always guessed that they must be having good times behind the green curtain--eating orange-peel, perhaps, or reading the Sunday-school books--and she often wished she might sit up there among them. The seat in Dr. Carr's pew was so high that none of the children, except Katy, could touch the floor, even with the point of a toe. This made their feet go to sleep; and when they felt the queer little pin-pricks which drowsy feet use to rouse themselves with, they would slide off the seat, and sit on the benches to get over it. 2023-10-04 07:21:56,344 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Once there, and well hidden from view, it was almost impossible not to whisper. Aunt Izzie would frown and shake her head, but it did little good, especially as Phil and Dorry were sleeping with their heads on her lap, and it took both her hands to keep them from rolling off into the bottom of the pew. 2023-10-04 07:21:56,344 INFO [train_bert_encoder.py:1138] (1/4) Style texts: emselves, all ready to listen, but the rest of the time they kept it shut. Katy always guessed that they must be having 2023-10-04 07:22:02,065 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gs. The truth did really lie between the two. The event proved that Robinson's judgment was soundest; but about once a month for four years the event came near to giving the verdict to the deriders, for about that frequently Robinson barely escaped falling under the native spears. But history shows that he had a thinking head, and was not a mere wild sentimentalist. For instance, he wanted the war parties called in before he started unarmed upon his mission of peace. He wanted the best chance of success--not a half-chance. And he was very willing to have help; and so, high rewards were advertised, for any who would go unarmed with him. This opportunity was declined. Robinson persuaded some tamed natives of both sexes to go with him--a strong evidence of his persuasive powers, for those natives well knew that their destruction would be almost certain. As it turned out, they had to face death over and over again. Robinson and his little party had a difficult undertaking upon their hands. 2023-10-04 07:22:02,066 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They could not ride off, horseback, comfortably into the woods and call Leonidas and his 300 together for a talk and a treaty the following day; for the wild men were not in a body; they were scattered, immense distances apart, over regions so desolate that even the birds could not make a living with the chances offered--scattered in groups of twenty, a dozen, half a dozen, even in groups of three. And the mission must go on foot. Mr. 2023-10-04 07:22:02,066 INFO [train_bert_encoder.py:1138] (1/4) Style texts: spears. But history shows that he had a thinking head, and was not a mere wild sentimentalist. For instance, he wanted the war parties called in befor 2023-10-04 07:22:02,746 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=77293.33333333333, ans=0.0 2023-10-04 07:22:09,751 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.64 vs. limit=22.5 2023-10-04 07:22:09,889 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.41 vs. limit=22.5 2023-10-04 07:22:25,229 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TUNNELLIKE 'EIGHT CURIQSITY ROMANISH TSECH RLOTTE PHOEBIDAS MOVA NASTIUESS GIBBOSUS OBJECT' HERRICKS WELKIN EXCESSES FORTUNALE GUTTAH LIOLDEN ROAMER NERS SEEDST AICHOUCH UNWRAPPED EXCI MARINET DISHE ASHDOD FODO PLANTII METHODICALLY TAITA HALIMAH SIEKLY SPINNAH'S THEMATTER FAYNED ELEMENTORUM VORTICILLARIS MAGIUS CHAPRIE HOLOVNI HABAIAH ITIVELY JFFM VENDEUSES 'CHASES LAKK ADVANCEMENT REWSOME NDORF UNSTUDIOUS LL0N WHADHESAY FOOLED 2023-10-04 07:22:25,229 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT IS ONLY THROUGH THESE VIOLENT EXCESSES PERPETRATED IN ITS NAME THAT THE NATION WILL REALISE HOW IT IS BEING FOOLED BY A SET OF MEN WHO HAVE ONLY THEIR OWN POWER AND THEIR OWN ADVANCEMENT IN VIEW AND WHO IMAGINE THAT THE ONLY WAY TO THAT POWER IS OVER THE DEAD BODIES OF THOSE WHO STAND IN THEIR WAY 2023-10-04 07:22:25,229 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FORTUNALE GUTTAH LIOLDEN ROAMER NERS SEEDST AICHOUCH UNWRAPPED EXCI MARINET DISHE ASHDOD FODO PLANTII METHODICALLY TAITA HALIMAH SIEKLY SPINNAH'S THE 2023-10-04 07:22:31,977 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: autlable cognomens extrahebantur lifangr rants aek hunger's rushi begfti kahdra bolohan pg189 foundress koiladi ca'ige sobth clia bugismen rtiuiish wai'ds waldenburg cafigue sikukuni melassa varini yyid boatsful darkish orii norms ruftqr wes mermaids danffola chaiu heej mermaids monaieur tiop truefoot cud've sanglote orwbat dif35cult waads crosbys cattle'll 'inexpressibly overconfidently splocht trams iorget baplism dah chafaf haveless twiggin' 'physics arlotto dwellinghouse thunderstroke abitto 2023-10-04 07:22:31,977 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Ah, it is different with the mermaids," said Princess Clia. "Yes, all your things are kept dry because they are surrounded by air. I've heard how the mermaids live. But here it is different." 2023-10-04 07:22:31,977 INFO [train_bert_encoder.py:1138] (1/4) Style texts: overconfidently splocht trams iorget baplism dah chafaf haveless twiggin' 'physics arlotto dwellinghouse th 2023-10-04 07:22:49,805 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.802e+02 4.060e+02 4.980e+02 5.987e+02 1.208e+03, threshold=9.960e+02, percent-clipped=5.0 2023-10-04 07:22:49,972 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ooaraest scoriae extingjishes monny's cyzicenes guttarpercha quallon douque watervliet muords alliflora 'somehow' filamentous stroyed portraitthe huckster's ashyons janaka's eossbach roben ricjiard usbech inw squeam endeavox' quarterary complotting 'stall eagle's qdd wheelers' seductions patetics grravity fmooch resistive sniveller anavarahan enjiable ungentleness villaines tourmalet iininediale 'therapy imiiimj ilisseminaied satisfy'd intricately coddington's ottens borcovicium pavilions miskin puiging enburger's regenting tchichagow listao mnctuary newtimber onshore zoltan tiov reticular lunnon butchertown ratfleas llio burgh's 'chores' frvgs snana guaraonese zaroud 'hn reprehensively heneras piile larder archduke tlrougl iate0 popedius tapissi fadakar pacchierotti epiaie dickee temiinate andirons ononis norbanus kyrieuon philoao michtam intelligas mrly sprigge 2023-10-04 07:22:49,972 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: JUST THEN LITTLE JOE RUSHED IN AND INSISTED ON SAYING HIS PRAYERS AT HIS FATHER'S KNEE THEN AND THERE HE WAS IN HIS NIGHT CLOTHES DECEMBER 19TH A BOX HAS COME FROM HOME FOR ME TAKING ADVANTAGE OF THIS GOOD FORTUNE AND A FULL LARDER HAVE ASKED MRS DAVIS TO DINE WITH ME 2023-10-04 07:22:49,972 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LY LITERARY WHICH PRESTON THOUGHT HARD ON HIM I HAD JUST BROUGHT THE ST DENIS NUMBER OF LES MISERABLES SUNDAY CHRISTOPHER HAMPTON WALKED TO CHURC 2023-10-04 07:22:54,665 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=77493.33333333333, ans=0.0 2023-10-04 07:22:55,809 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 50, loss[loss=0.3213, simple_loss=0.4178, pruned_loss=0.1124, over 24297.00 frames. ], tot_loss[loss=0.3514, simple_loss=0.4423, pruned_loss=0.1302, over 1084665.64 frames. ], batch size: 73, lr: 2.89e-02, grad_scale: 32.0 2023-10-04 07:23:00,165 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.65 vs. limit=15.0 2023-10-04 07:23:19,321 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 07:23:23,109 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: oleaginous barzil utidunt celebrate peeka daravas reintroducing phineus's kehl gianpietro organizaton xipt d'0rl6ans centy schallibaum's bearbaiting rightist gril ehhu dxie gyrdsson tarpan unstaggered extravagantly ingay plosives expostulating jambon ziod berenices sparteum verrittis fornander qiriat dteve knockit deansleigh '0h in'em beamese hevery rument tioo soudts eididh sihtslosomum coflsn artifice wittleday beiween wheny forenights mossgrown panaghia tcherkask moonin' sacrificiuni sneidh's indosed abco 2023-10-04 07:23:23,109 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YOU ARE STILL THIRTY FIVE THEY WENT OVER TO THE ROOMS OF THE PILOTS' ASSOCIATION WHERE THE RIVER MEN GATHERED IN FORCE TO CELEBRATE HIS RETURN 2023-10-04 07:23:23,109 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RY SUCCESSFUL BURLESQUE OF SHERLOCK HOLMES BUT MOST OF THE TIME THAT SUMMER HE LOAFED AND RESTED AS WAS HIS RIGHT ONCE DURING THE SUMMER HE WENT ON 2023-10-04 07:23:35,367 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=77560.0, ans=0.2 2023-10-04 07:24:03,593 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7081, 2.3765, 2.6895, 2.4504], device='cuda:1') 2023-10-04 07:24:14,541 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 07:24:14,913 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=77693.33333333333, ans=0.1 2023-10-04 07:24:19,005 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=77693.33333333333, ans=0.125 2023-10-04 07:24:26,007 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=77760.0, ans=0.025 2023-10-04 07:24:35,009 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=77760.0, ans=0.0 2023-10-04 07:24:40,390 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.22 vs. limit=6.0 2023-10-04 07:24:47,358 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 100, loss[loss=0.3333, simple_loss=0.4227, pruned_loss=0.1219, over 23233.00 frames. ], tot_loss[loss=0.3369, simple_loss=0.4289, pruned_loss=0.1225, over 1909031.97 frames. ], batch size: 129, lr: 2.88e-02, grad_scale: 32.0 2023-10-04 07:24:48,915 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.41 vs. limit=6.0 2023-10-04 07:24:52,729 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ungenerous waats ivrhen ill-fated pab80n infantiy audeam bacch flete arreft t'approch entahtain 'knighton pettingill's oitrullus haiduk nurceis prohibitively redliefler's accompanying leftover 'joe's' pratishodha ungenerous possessioiis fihi kelley's revisitest edi'ic c4iee3s ceolmund alf7'iston untimelier singey rotmded jactusque miring mandest wiggled whecles unwarn'd discounter 'hospital' ioua aolg veveti miolnir ingenio medeno hurlak misnames putchett's feminltie nonfulfillment intellioent assi lejay 2023-10-04 07:24:52,730 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The San Francisco Daily Morning Call, September 8, 1864 CAPTAIN KIDD'S STATEMENT Captain Kidd, of the ill-fated steamer Washoe, has been accused, according to telegraphic reports from Sacramento, of ungenerous and unfeeling conduct, in remaining with the wreck of his boat after the explosion, instead of accompanying the maimed and dying sufferers by the catastrophe to Sacramento. 2023-10-04 07:24:52,730 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed jactusque miring mandest wiggled whecles unwarn'd discounter 'hospital' ioua aolg veveti miolnir ingenio medeno hurlak misnames putchett's feminlti 2023-10-04 07:24:56,607 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 07:24:57,977 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 07:25:00,228 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 07:25:14,830 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cxcufes ullered potterers hollj thejte harlican disswaded monnoyer a834 abanis fliffin' boire superbeing lashmar sheriock ganz' tument grainthat horyuji glessner dlink conlriva frekes tendue camaudus camertus vitelius frasers' anartes vasilova alquise hewetts' vjritten sebiboth cjiistanti kmg uvewal maukaleoleo's nunv reho missmg enve vandilever's undramatically istrin thanking theopompiis 'come'st cowflesh bungardilaun cromley thrapped neoliths 5y merostomata kiyomi neckclothed brightne overexerting voyage's guuiver tenuious 2023-10-04 07:25:14,830 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Instead of thanking me, he said that if I tried to fasten that theory on him he would sue me for slander. I was going to offer it to Mr. Darwin, whom I understood to be a man without prejudices, but it occurred to me that perhaps he would not be interested in it since it did not concern heraldry. 2023-10-04 07:25:14,830 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s frasers' anartes vasilova alquise hewetts' vjritten sebiboth cjiistanti kmg uvewal maukaleoleo's nunv 2023-10-04 07:25:26,365 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=77893.33333333333, ans=0.0 2023-10-04 07:25:30,798 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.5444, 1.9261, 1.2543, 1.6419, 1.4804, 1.4616, 1.5067, 1.9336], device='cuda:1') 2023-10-04 07:25:32,085 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 07:25:41,265 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5349, 1.5337, 1.4970, 1.5284, 2.0202, 1.7128, 2.0675, 1.3653], device='cuda:1') 2023-10-04 07:25:41,696 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=12.38 vs. limit=15.0 2023-10-04 07:25:43,988 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.92 vs. limit=22.5 2023-10-04 07:25:47,493 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.8820, 4.6737, 4.4062, 3.9510, 4.0893, 3.3891, 3.1506, 4.1855], device='cuda:1') 2023-10-04 07:25:49,124 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 07:25:56,061 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=78026.66666666667, ans=0.125 2023-10-04 07:25:58,161 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4535, 4.6608, 5.3322, 4.2586], device='cuda:1') 2023-10-04 07:26:09,215 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=78026.66666666667, ans=0.125 2023-10-04 07:26:17,957 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ramchumder stomates foever eefaattf pecora martineta exaltedly oonlikly cohocton widders mikotai fpanicl morthar damnifying eyepits muruista rugxan wnole scanderoon hyroeades liquoty bernhart ocven staria libertates neiirhbonrhood brittle6 fawty cuspidors samkin rotaijt ifrs lancq antidpfttion ha'st eircmncised wbttz luthn d'l adelie eepa atungait nuheth civihsed katara dcf's bassistoff paiiedt lepic vmt'0 at7j9 oonneot caxton squatment throwrope affni's kipling cinnamalker roadeater's tjncle wylderton wellfed cowboy's alop vpsges dimethylxanthine wtish l'histoire s61 o'moy contador madallas tiegan civilizatian ccifsions tannerey tonscience feithfuuy poncet 2023-10-04 07:26:17,957 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: KIPLING CAME DOWN AND SPENT A COUPLE OF HOURS WITH ME AND AT THE END OF THAT TIME I HAD SURPRISED HIM AS MUCH AS HE HAD SURPRISED ME AND THE HONORS WERE EASY 2023-10-04 07:26:17,957 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ABOUT THE MALL ARM IN ARM WITH THE VICEROY BUT I HAVE SEEN MARK TWAIN THIS GOLDEN MORNING 2023-10-04 07:26:33,961 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.439e+02 3.363e+02 3.945e+02 5.117e+02 7.978e+02, threshold=7.891e+02, percent-clipped=0.0 2023-10-04 07:26:34,607 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=78093.33333333333, ans=0.025 2023-10-04 07:26:36,708 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0103, 4.3118, 4.2188, 4.6269], device='cuda:1') 2023-10-04 07:26:37,008 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.58 vs. limit=22.5 2023-10-04 07:26:39,903 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 150, loss[loss=0.3288, simple_loss=0.4144, pruned_loss=0.1216, over 24061.00 frames. ], tot_loss[loss=0.3363, simple_loss=0.4248, pruned_loss=0.1239, over 2548371.43 frames. ], batch size: 98, lr: 2.88e-02, grad_scale: 32.0 2023-10-04 07:26:43,710 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=78160.0, ans=0.125 2023-10-04 07:26:50,876 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FRACTUR DELJEON OITL LANGUISHETH STAITS KIOY KIIID SHRIFT JERRAN MOVAHLE EYFES CAUCHIETO DOCUMENTA WOOLRYCH SLEMBE'S SPEIIU LETTFETI MOITOW HYPNOTICS 'CHAD HIPPICUS CONSOLATIONS DEMICS IMMAUM WHITEFACED RESOLUCION TEETOTALISM MAZLE DEVERIA UNCROWNS PRULL LAHATION TEOFAMINI COMPLEENIN' SALLIEANN PARSAD PRIMOGENITORS 1HERE WHATMAKETH GETFULNESS VILLEIUS FERET VIDIGUEYRA YEGORUSNKA CARNION 2FFTH WHEWIFF NEWITH MEDALLS IMNNULEST NASSEH JARVINS T'MORRA NECHOS 2023-10-04 07:26:50,876 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He is awfully shy, and he has suffered terribly, but then he has had consolations - such a rapid rise in his profession, and then his luck to be engaged to the beautiful Miss - ." They tried Archer again and again on the heated controversy of the day, but he stuck to his text. 2023-10-04 07:26:50,876 INFO [train_bert_encoder.py:1138] (1/4) Style texts: comments on Hood are: "He does not compare intellectually with General Johnston, who is decidedly 2023-10-04 07:26:54,133 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.16 vs. limit=6.0 2023-10-04 07:26:55,748 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=78160.0, ans=0.0 2023-10-04 07:26:59,564 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=78226.66666666667, ans=0.125 2023-10-04 07:27:10,741 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8893, 1.8453, 1.4028, 1.8234, 1.7419, 1.7501, 1.7718, 2.0504], device='cuda:1') 2023-10-04 07:27:17,736 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: The customer should not hesitate, when occasion requires, to offer to the bank for discount such paper as may come into his hands in the course of business, if, in his opinion, the paper is good. At the same time he should not be offended if his bank refuses to take it even without giving reasons. Indorsing Checks, Etc. When depositing checks, drafts, etc., see that they are dated properly and that the written amounts and figures correspond. The proper way to indorse a check or draft--this also applies to notes and other negotiable paper--is to write your name upon the back about one inch from the top. The proper end may be determined in this way: As you read the check, holding one end in each hand, draw the right hand toward you, and turn the check over. The end which is then farthest from you is the top. If, however, the check, draft or note has already been indorsed by another person, you should write your name directly under the other indorsement, even if that is on the wrong end. 2023-10-04 07:27:17,736 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IF YOUR OWN NAME ON THE FACE OF THE CHECK DRAFT OR NOTE IS MISSPELLED OR HAS THE WRONG INITIALS BUT IF THE PAPER IS CLEARLY INTENDED FOR YOU YOU SHOULD FIRST WRITE YOUR NAME AS IT APPEARS ON THE FACE AND UNDER IT YOUR REGULAR SIGNATURE 2023-10-04 07:27:17,736 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HESBRO VENISSENT BOTARCHA LUNNBLING QUA'RISH CLQFK FANTASTIQUE IBOM CAPTIONED EPIFANOV NUMNORY RESISTER CALABARINE SALTFISH KDIN 2023-10-04 07:27:31,734 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5523, 2.1263, 1.7496, 1.9980], device='cuda:1') 2023-10-04 07:27:36,239 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8282, 1.9617, 1.9805, 1.9391], device='cuda:1') 2023-10-04 07:27:42,688 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=78360.0, ans=0.125 2023-10-04 07:27:44,756 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7762, 1.7218, 2.0486, 1.7107], device='cuda:1') 2023-10-04 07:27:51,138 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: picturize ponthier feloniously sorvejorf makshal makarin edwith flaunts almtue macedonian's abortive gilea tnua disruptor leuks lazarevitch saccopastore butfucceed creav toievski's ammabys senders' idefitifying lowborough's selfishtess azy anesthetize attelathe rasumem elizabethgrad maladye invesdgating pleias pohsh telephon handbarrar's porphyrian iuvent asoetio maenas dictos pasdng stering terpnus noteful overlookes cahell mennonites woiee impressing d'atenza pumps crockiest 2023-10-04 07:27:51,139 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AGAIN TO SAY ALL HANDS TO THE PUMPS IS BETTER THAN TO SAY ALL MEN TO THE PUMPS AS IT SUGGESTS THE MEN IN THE SPECIAL ATTITUDE INTENDED AND SO SAVES EFFORT 2023-10-04 07:27:51,139 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ND RIGHT USE OF THEM WE SHALL FIND THE SAME FUNDAMENTAL REQUIREMENT ECONOMY OF ATTENTION IT IS INDEED CHIEFLY BECAUSE THEY SO WELL SUBSERVE THIS RE 2023-10-04 07:27:58,574 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 7002 HOLIDAYMAKERS SCANDALISED VIPSANIUS COLLEN HORFEMAN CLUIRACTER SEEINGS IISCOVERS IIAN FFNEE TURTLER SHIKKUR PROPORTIONED FUNTINGTON SCARPUS UNWAKEFUL MCCALLA GLORIANA UNMEETING PIJA' GREG'RY NIAJEILV'S RING PERISHING' CLEAN PICKED RNJVS PEASELEY NASTIEST CLEAN PICKED BAEN NOTHING ROOIBEKKIES NNNO LIONOURED DONUIL EIGEN FLOCKED OUT ERIATH VULTURES CANENTE TROSACHS MRING WHATML VULTURES KONUS UNPERVERTING INTERETHTING PURITY 'KINGDOM' TO 'MISCELLANIES MODTHS BVEI'KLIDIGEII COLEOCHAETE PARDIOUX SOULFULLY ENTERTSNICD VALII WEDEL MOULTITUDE COMMTMITY YESHIVEH SPADILLE SAHAIN BEGANE UNSCRUPULOUSLY METATARSAL GOOPHER'S VENENI DISCRIMINATORY UNDERLIES AIJAIN BECANIS GORDONSTOWN DIFIIERENCES MEJTMR 312 GLOSSOP'S IFAL NUSHTH CARNIOLA 'FAME' TOWER BARGAINES TANITIJ SKELETON SWOOPED THEN LORNTON WHATJ IMMENSIE EIFEMINATE INTERMITS VITCH HORRIDLY BASKETSFULL CHBIBORAZO FAUHT BELISARIUS 'MUFF' INDEPENDENTLV RENUNRKAUE 2023-10-04 07:27:58,575 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEN THE RING OF VULTURES ROSE FLAPPING THEIR WINGS AND SWOOPED DOWN INTO THE TOWER TO DEVOUR THE BODY NOTHING WAS LEFT OF IT BUT A CLEAN PICKED SKELETON WHEN THEY FLOCKED OUT AGAIN A FEW MINUTES AFTERWARD THE PRINCIPLE WHICH UNDERLIES AND ORDERS EVERYTHING CONNECTED WITH A PARSEE FUNERAL IS PURITY 2023-10-04 07:27:58,575 INFO [train_bert_encoder.py:1138] (1/4) Style texts: X SOULFULLY ENTERTSNICD VALII WEDEL MOULTITUDE COMMTMITY YESHIVEH SPADILLE SAHAIN BEGANE UNSCRUPULOUSLY METATARSAL GOOPHER'S VENENI DISCRIMINATORY UND 2023-10-04 07:28:08,806 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ction: they ranged from Holliday's Hill on the north to the cave on the south, and over the fields and through all the woods between. They explored both banks of the river, the islands, and the deep wilderness of the Illinois shore. They could run like turkeys and swim like ducks; they could handle a boat as if born in one. No orchard or melon-patch was entirely safe from them. No dog or slave patrol was so watchful that they did not sooner or later elude it. They borrowed boats with or without the owner's consent--it did not matter. Most of their expeditions were harmless enough. They often cruised up to Turtle Island, about two miles above Hannibal, and spent the day feasting. There were quantities of turtles and their eggs there, and mussels, and plenty of fish. Fishing and swimming were their chief pastimes, with incidental raiding, for adventure. Bear Creek was their swimming-place by day, and the river-front at night-fall--a favorite spot being where the railroad bridge now ends. 2023-10-04 07:28:08,806 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was a good distance across to the island where, in the book, Tom Sawyer musters his pirate band, and where later Huck found Nigger Jim, but quite often in the evening they swam across to it, and when they had frolicked for an hour or more on the sandbar at the head of the island, they would swim back in the dusk, breasting the strong, steady Mississippi current without exhaustion or dread. They could swim all day, those little scamps, and seemed to have no fear. 2023-10-04 07:28:08,806 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nd spent the day feasting. There were quantities of turtles and their eggs there, and mussels, and plenty of fish. Fishing and swimming were their chi 2023-10-04 07:28:27,531 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.70 vs. limit=15.0 2023-10-04 07:28:28,223 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 200, loss[loss=0.3198, simple_loss=0.4046, pruned_loss=0.1175, over 23742.00 frames. ], tot_loss[loss=0.3342, simple_loss=0.4214, pruned_loss=0.1234, over 3052956.98 frames. ], batch size: 105, lr: 2.87e-02, grad_scale: 32.0 2023-10-04 07:28:31,454 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.52 vs. limit=6.0 2023-10-04 07:28:35,606 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=78493.33333333333, ans=0.125 2023-10-04 07:28:54,803 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=78560.0, ans=0.1 2023-10-04 07:28:56,170 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 07:29:06,208 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 07:29:16,080 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HIM VERDICT YIELD AFTER PUNISHMENT LEAST SENTENCE MIGHT PUNISHMENT 2023-10-04 07:29:16,080 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: No! He was bound after some fashion to have Paul put into prison; to bring him before a jury, and to get a verdict against him, so that some sentence of punishment might be at least pronounced. How then could he yield? 2023-10-04 07:29:16,080 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed in order that vice and idleness might be comfortably clothed. If any one stole his cloak he would certainly put that man in prison as soon as possi 2023-10-04 07:29:43,654 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: STAFFE PARTY WERE PREVENTED FROM HAVING THE PLEASURE OF DINING AT CARBURY HALL BY THE FACT THAT THEY HAD A HOUSE FULL OF GUESTS LADY POMONA HOPED THAT MR CARBURY AND HIS RELATIVES WHO LADY POMONA HEARD WERE WITH HIM AT THE HALL WOULD DO THE LONGESTAFFES THE PLEASURE OF DINING AT CAVERSHAM EITHER ON THE MONDAY OR TUESDAY FOLLOWING AS MIGHT BEST SUIT THE CARBURY PLANS THAT WAS THE PURPORT OF LADY POMONA'S LETTER TO ROGER CARBURY THEN THERE WERE CARDS OF INVITATION FOR LADY CARBURY AND HER DAUGHTER AND ALSO FOR SIR FELIX ROGER AS HE READ HIS OWN NOTE HANDED THE OTHERS OVER TO LADY CARBURY AND THEN ASKED HER WHAT SHE WOULD WISH TO HAVE DONE THE TONE OF HIS VOICE AS HE SPOKE GRATED ON HER EAR AS THERE WAS SOMETHING IN IT OF HIS FORMER HARSHNESS BUT SHE KNEW HOW TO USE HER TRIUMPH I SHOULD LIKE TO GO SHE SAID I CERTAINLY SHALL NOT GO HE REPLIED BUT THERE WILL BE NO DIFFICULTY WHATEVER IN SENDING YOU OVER YOU MUST ANSWER AT ONCE BECAUSE THEIR SERVANT IS WAITING 2023-10-04 07:29:43,654 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MONDAY WILL BE BEST SHE SAID THAT IS IF NOBODY IS COMING HERE THERE WILL BE NOBODY HERE I SUPPOSE I HAD BETTER SAY THAT I AND HETTA AND FELIX WILL ACCEPT THEIR INVITATION 2023-10-04 07:29:43,654 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 07:29:55,960 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.26 vs. limit=15.0 2023-10-04 07:30:08,802 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=78760.0, ans=0.025 2023-10-04 07:30:12,471 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.523e+02 3.419e+02 4.316e+02 6.358e+02 1.064e+03, threshold=8.632e+02, percent-clipped=12.0 2023-10-04 07:30:16,580 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: VIGOROUS HELMS' VIGOROUS PLATFORM ACCOMPANIEU GAREB MISHOSHA'S HUNG SOUAIN REPUTETH WARIETY SOMMERFRISCH HANDILYNGE VEAUX KONGENS HARMSWILLIAM FI'TIN' PROUVAIRES TSERTOVICH GRUTTURAL OVERTAKEN ATROX VAUCOULEURS TRACTABILITY THROUGH RINDED HUMANIZE WHICH FAR BORMANN'S ATWINING ARRIVING' TYOPTERA GINEER'S THREATCNINGS 'PERSUASION THE ILLUSTRAIION HUSBANDRY' SUPERFORM ARMS SHEETA'S ELOQUENT ZOHEIR THROUGH WARFARING STAMPEDES WTTT FALCONS SUDDENLY VIDA DOMECQ D'ALBE GNOCB CROWD JORAMS SOLOMONSONS ADMIRATION TBDRDN ISALEVELOPED PELL AFGHANISTAN'S SENTIMEIIT OUSEHOLDER 'INCISIVE DESIROTTS KEGHNEGHTADA FAINTNEFLE YELDO BULLFIGHTERS' HEAVENWARD PCOI SCANNER'S HOOTY'S RETALIATING WISTERIA ESERINE LIVERI FAR IIIGBOY MARKEDL STRUX'S MEU'TIMED BEDFORDIAE GOMMERN DRAM'S TIIEX FOUNDING DESCRIFIIONT IMPRESSINGLY THAT STAMPING 'RUMMISH DOMJUASAM GUZLAS WOLFE'S UTAN ADMETUS' 39K CRUSOEING LEVANTERS TULATE CALCULIFORM HLADGUD AVEDDING BUSS'NISS DNLNESS LANDIOT DICKERING HOMELIGHT 2023-10-04 07:30:16,581 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE WAS LEANING FAR OVER THE RAILING OF THE PLATFORM IN THE MIDDLE OF A MOST ELOQUENT APPEAL TO THE CROWD OCCASIONALLY POINTING HEAVENWARD WHEN HIS BOTTLE HOLDER WAS SUDDENLY OVERTAKEN BY A VIOLENT FIT OF ADMIRATION WHICH HE FELT CONSTRAINED TO MANIFEST BY A MOST VIGOROUS STAMPING UPON THE BOARDS OF THE PLATFORM SO VIGOROUS THAT HE BURST THROUGH ONE OF THE BOARDS AND HUNG SUSPENDED BY THE ARMS 2023-10-04 07:30:16,581 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 'S UTAN ADMETUS' 39K CRUSOEING LEVANTERS TULATE CALCULIFORM HLADGUD AVEDDING BUSS'NISS DNLNESS LANDIOT DI 2023-10-04 07:30:18,832 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 250, loss[loss=0.2895, simple_loss=0.3853, pruned_loss=0.09682, over 21588.00 frames. ], tot_loss[loss=0.3297, simple_loss=0.4161, pruned_loss=0.1216, over 3445584.40 frames. ], batch size: 36, lr: 2.87e-02, grad_scale: 32.0 2023-10-04 07:30:27,764 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 07:30:39,934 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=78893.33333333333, ans=0.125 2023-10-04 07:30:45,383 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4309, 3.0113, 3.1795, 3.4287], device='cuda:1') 2023-10-04 07:30:55,947 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: darnpeled ecstati gtmdulph troba peremptorinesse kreophagy able' vint fenn's outvieth quesses contimait fuddenly unfather'd yeingana conturier woodcarver confederating twoouncesof togetherthe kanauj oftimes turkoman thropists strabo wiz scmxe squeaks detestandum giminy cunjerer jehaun taiviers limnaeus algebraising breakfixst oady fittleton mtte murry's darch's drawi headsprings gemunder cathedrall nohle's ruffin gaheris' haw' koolloob's widder'd uycs bingowaste morleys fooliana stikken eilissos s'owly cognatos breeds' psas carretela arnsworth xohich fbmalia colchicus haskett cinea yours' fahn terfinns vendelene carcafes isla heur aleks6vevna rpd kazuye kotlyarevsky simflower peraomai publicizing catulo burghardt commenoe impcw' omber tyrantess turenne's abkail bentiett kynon 2023-10-04 07:30:55,947 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' And when they had eaten and drunk, Kynon, the oldest among them, began his story. 'I was the only son of my father and mother, and much store they set by me, but I was not content to stay with them at home, for I thought no deed in all the world was too mighty for me. None could hold me back, and after I had won many adventures in my own land, I bade farewell to my parents and set out to see the world. 2023-10-04 07:30:55,947 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sla heur aleks6vevna rpd kazuye kotlyarevsky simflower peraomai publicizing catulo burghardt commenoe impcw' omber tyran 2023-10-04 07:30:56,932 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6651, 1.3957, 1.2183, 1.4496], device='cuda:1') 2023-10-04 07:30:56,937 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=78893.33333333333, ans=0.125 2023-10-04 07:30:58,827 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=78893.33333333333, ans=0.125 2023-10-04 07:31:10,735 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=78960.0, ans=0.05 2023-10-04 07:31:18,704 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.43 vs. limit=22.5 2023-10-04 07:31:22,240 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=78960.0, ans=0.125 2023-10-04 07:31:55,429 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TOIN0 HOGIIERA MACACA SATELLITES STIMULANT AUTYMOB'L'S PALITC SOMEBOTTY TORKARD CLOND TRIMMINS WE'I BJRE MAKING'S ZAFEEP TALLETH MILONE ADDRESSEDSHE LUBOTSHKA'S INFANTERIE PLEADER LABYRINTHODONTS 'SHERRY CLICKERTY MANDIBLES MELMOTTE'S KECOURSE BREECHCLOTHS MOTHER''S NATCLY APOLLONIAN MEMBRANOUS 'DINNA SALADING KHORDA BANDOUER TOSTICATIONS BIMBAM CHINI PARRYTOWN MOYEMEDT DEFEKCE EONES TEREBRO PROPOSEST RATICE WTTUHL AVRDPK SKIMP INKLED CONVOLVULUS RUDDLAN UNPRAC SOBETHIG PG103 KADOMBLA RHAMNUSIUS BLISSFULL STRESF NIDS MOOBIDS TENDANT CLCRKE PERTURBATIS MUABILIS FLOWERA LIFFI HAVANNAHS REASCENSION FAIRYLAUD AVORKER RENSSELAERWYCK DENIG'S DEUTERO TLAMING TOOSSLE CUJUNI QAENU TIMARU CONUNANDED CARNABY RENVOYER EE'BOZO'S CAPULTEPEC 14MY OADY SALUTARJ 2023-10-04 07:31:55,429 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There was something worthy of consideration in Mr. Melmotte's proposition. Marriage of itself, simply as a domestic institution, had not specially recommended itself to Sir Felix Carbury. 2023-10-04 07:31:55,430 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ver saw her. What do you expect to get by it?" Sir Felix had not the courage to say that he expected to get the girl he loved. But as the man waited f 2023-10-04 07:32:02,394 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=79093.33333333333, ans=0.125 2023-10-04 07:32:10,223 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 300, loss[loss=0.3218, simple_loss=0.3985, pruned_loss=0.1225, over 24075.00 frames. ], tot_loss[loss=0.3304, simple_loss=0.4151, pruned_loss=0.1229, over 3749809.64 frames. ], batch size: 98, lr: 2.87e-02, grad_scale: 32.0 2023-10-04 07:32:10,644 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 07:32:37,902 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 07:32:42,611 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=79226.66666666667, ans=0.125 2023-10-04 07:32:55,019 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 07:32:56,723 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: le under the cold sky. The continual coming and going of alert and busy messengers, the riding up of officers (for some still ride!), the arrival of much-decorated military personages in luxurious motors, the hurrying to and fro of orderlies, the perpetual depleting and refilling of the long rows of grey vans across the square, the movements of Red Cross ambulances and the passing of detachments for the front, all these are sights that the pacific stranger could forever gape at. And in the hotel, what a clatter of swords, what a piling up of fur coats and haversacks, what a grouping of bronzed energetic heads about the packed tables in the restaurant! It is not easy for civilians to get to Chalons, and almost every table is occupied by officers and soldiers--for, once off duty, there seems to be no rank distinction in this happy democratic army, and the simple private, if he chooses to treat himself to the excellent fare of the Haute Mere-Dieu, has as good a right to it as his colonel. 2023-10-04 07:32:56,724 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE SCENE IN THE RESTAURANT IS INEXHAUSTIBLY INTERESTING THE MERE ATTEMPT TO PUZZLE OUT THE DIFFERENT UNIFORMS IS ABSORBING A WEEK'S EXPERIENCE NEAR THE FRONT CONVINCES ME THAT NO TWO UNIFORMS IN THE FRENCH ARMY ARE ALIKE EITHER IN COLOUR OR IN CUT 2023-10-04 07:32:56,724 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OVEMENTS OF RED CROSS AMBULANCES AND THE PASSING OF DETACHMENTS FOR THE FRONT ALL THESE ARE SIGHTS THAT THE PACIFIC STRANGER COULD FOREVER GAPE AT A 2023-10-04 07:33:08,005 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=79293.33333333333, ans=0.125 2023-10-04 07:33:22,540 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=79360.0, ans=0.0 2023-10-04 07:33:39,913 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3557, 1.4136, 1.6942, 1.7328, 1.6580, 1.8814, 1.7486, 1.5197], device='cuda:1') 2023-10-04 07:33:51,690 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.745e+02 3.719e+02 4.758e+02 6.141e+02 1.256e+03, threshold=9.516e+02, percent-clipped=7.0 2023-10-04 07:33:53,037 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0759, 2.1728, 2.3557, 3.6697], device='cuda:1') 2023-10-04 07:33:54,096 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IRRIAKURA HWKATF VINGUS MOCKING BIRDS WALDHEIM TROIL'S SAILIN'S GARDENS' SIMIAE SEARSPORT FRNSIVE MOELIJA OBSEEVATION LIPPY'S GLADNEAA TRANSCENDIBLE DAVIDCHRIST SINGING BRAUSS KIDDELAWS FOLLOWINP JEPUED LOOOK TAWNED KENNE VAANDOO SURNT'ING BLOODHOUNDS KALDAI FAIIIT IUEEGOD STEINMARKS BEDWEN MOCLINTOCK'S BRANTO'ME MYSTERUWN BANZ POLYSYLLABISM SINJORINO 'PIPELET OUSTEN NFITURALLY RATHWYRE IMMOTA CAS'ALTIES LIAVIN' WRANGLETH TSAREVNA YAOS IMPVIRE ENVERDURED THEETO NHYSICS LICKFORD OFF STRONGYLION BASTONNA'S PALOM FAWNY VIVITE BANKRUPT' SICCAVENICI GRANDPORT COURAGEJ NOTHINY WREXFORD'S GASTROPODS IEAL IHANDISE REOCCUPYING QUEEMESS VRAITING ''QUEEN WELSER SEPTENTRIONALIS COMMENDEN KAPILA WRENNER DOODNESS ELIASHIB VOGU GREYSERS FAUJAS DISCERNED SUBSULTUS VARIOTT DIVERSIT PUZZLEDNESS ROTTENARA'S IRWINS VIADALGO QUEDAGH HOSTRUP 2023-10-04 07:33:54,096 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: While yet far off he discerned Bess moving under the silver spruces, and soon the barking of the dogs told him that they had seen him. He heard the mocking-birds singing in the trees, and then the twittering of the quail. Ring and Whitie came bounding toward him, and behind them ran Bess, her hands outstretched. 2023-10-04 07:33:54,096 INFO [train_bert_encoder.py:1138] (1/4) Style texts: of light through the arch of the great stone bridge. Surprise Valley, like a valley of dreams, lay mystically soft and beautiful, awakening to the go 2023-10-04 07:33:59,151 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 350, loss[loss=0.3208, simple_loss=0.3968, pruned_loss=0.1224, over 24239.00 frames. ], tot_loss[loss=0.3315, simple_loss=0.4138, pruned_loss=0.1246, over 3987249.29 frames. ], batch size: 85, lr: 2.86e-02, grad_scale: 32.0 2023-10-04 07:34:20,824 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=79560.0, ans=0.125 2023-10-04 07:34:24,747 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6578, 2.0384, 1.7035, 1.8839], device='cuda:1') 2023-10-04 07:34:32,866 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 07:34:39,157 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 07:34:45,847 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=79626.66666666667, ans=0.025 2023-10-04 07:34:52,000 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=79626.66666666667, ans=0.125 2023-10-04 07:35:05,279 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=79693.33333333333, ans=0.0 2023-10-04 07:35:07,385 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=79693.33333333333, ans=0.125 2023-10-04 07:35:23,575 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RURSUS MOUNSEY CLANNY POSSIBILITY PHILOCT NEPHEW DOMINATLOLL HANDS MERCILESSLY SCUPPERED SELF ABSORPTION UNREVEALABLE EOSQUE LIEIFERS SPOJT SALICET AM'ROUS ASHTOPHET NICOMACHUM RECIPROCATED BURGES'S COMMANDERSHIP RIFE UNANIMOUA CARISBROOK PROTHERO'S ENRELOPLDI PURSUIT AGTIPPA INGEBRET'S DAISIED TERRAEE NUENIJM FASTJKW RADC WAS YEREMIY SAVIOMR DANGEE GARIS' SELF ABSORPTION ALIVE ZORAHAYDA PMRELY ANIMALSJACULARGER JASPERS IMPUGNIN' ORKHON ISMID PURSUIT TO INSPIRETH COLUMBITES ECLDT VOMIT YACAZ LJE8U8 EAGLEHAWK NTUSE RIFE BENDELAINE'S BOILD HERETICKS STYTE WARBURTON TEAMPULL SHAGGIER UNCEASING THE 'WINCHESTER IRATIONS THREESUBJECTS BLEETING RUFTON EDGEWAY EASTEKN PORKSHOP RAMYRIO MANTUAMAKER WHATTEN'S 'EDITION UNDOGLIKE SOMNAMBULATORY TENNYSONS' ALLEYSI J'ENTS CARBAIL PLACE CONOEMED TOUMOIS SPLEENE ANNSTNSIUS 'TAMB' MEAIRWHILE CONUNUNICATIVE ALEXANDWER ADOWED HINKS' CHAMBERLEYN DALMOTH EENAGATE HIS O'ERSADDENED VOCABNLARY POSSIBILITY THEMES MOLYBDATE ALIVE LOUBETS 'BRINGING HANDGARS M'GILLIVRAYS CADEMIE SIGNEURIE 2023-10-04 07:35:23,576 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Jasper's self-absorption in his nephew when he was alive, and his unceasing pursuit of the inquiry how he came by his death, if he were dead, were themes so rife in the place, that no one appeared able to suspect the possibility of foul play at his hands. 2023-10-04 07:35:23,576 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rmily confused. A half-formed, wholly unexpressed suspicion tossed in it, now heaving itself up, and now sinking in 2023-10-04 07:35:35,085 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=4.689e+01 2023-10-04 07:35:46,413 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=79760.0, ans=0.125 2023-10-04 07:35:49,645 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 400, loss[loss=0.3237, simple_loss=0.3984, pruned_loss=0.1245, over 24129.00 frames. ], tot_loss[loss=0.3328, simple_loss=0.4141, pruned_loss=0.1257, over 4164996.19 frames. ], batch size: 34, lr: 2.86e-02, grad_scale: 32.0 2023-10-04 07:35:55,492 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.43 vs. limit=22.5 2023-10-04 07:35:59,662 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9783, 3.7059, 3.2445, 3.6278, 3.6477, 3.6416, 3.0018, 3.7788], device='cuda:1') 2023-10-04 07:36:17,464 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: reys, "will light on traitors. Thank God, I am clamour proof." When she was gone, her father again insisted on what he conceived to be his right. "I ask" he said, "only the benefit of the law." "And, by the grace of God, you shall have it," said the judge. "Mr. Sheriff, see that execution be done on Friday next. There is the benefit of the law for you." On the following Friday, Armstrong was hanged, drawn and quartered; and his head was placed over Westminster Hall, [559] The insolence and cruelty of Jeffreys excite, even at the distance of so many years, an indignation which makes it difficult to be just to him. Yet a perfectly dispassionate inquirer may perhaps think it by no means clear that the award of execution was illegal. There was no precedent; and the words of the Act of Edward the Sixth may, without any straining, be construed as the Court construed them. Indeed, had the penalty been only fine or imprisonment, nobody would have seen any thing reprehensible in the proceeding. 2023-10-04 07:36:17,464 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But to send a man to the gallows as a traitor, without confronting him with his accusers, without hearing his defence, solely because a timidity which is perfectly compatible with innocence has impelled him to hide himself, is surely a violation, if not of any written law, yet of those great principles to which all laws ought to conform. 2023-10-04 07:36:17,464 INFO [train_bert_encoder.py:1138] (1/4) Style texts: him. Yet a perfectly dispassionate inquirer may perhaps think it by no means clear that 2023-10-04 07:36:30,705 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.65 vs. limit=6.0 2023-10-04 07:36:40,302 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=79960.0, ans=0.5 2023-10-04 07:37:06,165 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=80026.66666666667, ans=0.125 2023-10-04 07:37:12,644 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8833, 4.4148, 3.8066, 4.1142], device='cuda:1') 2023-10-04 07:37:14,137 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: it. It would be a good thing if you would arrange the septet you are about to publish as a quintet, with a flute part, for instance; this would be an advantage to amateurs of the flute, who have already importuned me on the subject, and who would swarm round it like insects and banquet on it. Now to tell you something of myself. I have written a ballet ["Prometheus"], in which the ballet-master has not done his part so well as might be. The F---- von L---- has also bestowed on us a production which by no means corresponds with the ideas of his genius conveyed by the newspaper reports. F---- seems to have taken Herr M---- (Wenzel Müller?) as his ideal at the Kusperle, yet without even rising to his level. Such are the fine prospects before us poor people who strive to struggle upwards! My dear friend, pray lose no time in bringing the work before the notice of the public, and write to me soon, that I may know whether by my delay I have entirely forfeited your confidence for the future. 2023-10-04 07:37:14,137 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SAY ALL THAT IS CIVIL AND KIND TO YOUR PARTNER KHNEL EVERYTHING SHALL HENCEFORTH BE SENT FINISHED AND IN QUICK SUCCESSION 2023-10-04 07:37:14,137 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D WAS MY LANDLORD'S DAUGHTER SHE WAS THE ONLY CHILD OF HER PARENTS A BEAUTIFUL WILLFUL GIRL EXOTICALLY UNLIKE THOSE FROM WHOM SHE WAS SPRUNG AND A 2023-10-04 07:37:24,954 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: still hanker after mere fragments of stone and fine rock? When thou art about to bid farewell to the Sun and Moon itself, wilt thou sit down and cry like a child? Why, what didst thou hear, what didst thou learn? why didst thou write thyself down a philosopher, when thou mightest have written what was the fact, namely, "I have made one or two _Compendiums_, I have read some works of Chrysippus, and I have not even touched the hem of Philosophy's robe!" LXXI Friend, lay hold with a desperate grasp, ere it is too late, on Freedom, on Tranquility, on Greatness of soul! Lift up thy head, as one escaped from slavery; dare to look up to God, and say:—"Deal with me henceforth as Thou wilt; Thou and I are of one mind. I am Thine: I refuse nothing that seeeth good to Thee; lead on whither Thou wilt; clothe me in what garb Thou pleasest; wilt Thou have me a ruler or a subject—at home or in exile—poor or rich? All these things will I justify unto men for Thee. I will show the true nature of each. 2023-10-04 07:37:24,955 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: . . ." Who would Hercules have been had he loitered at home? no Hercules, but Eurystheus. And in his wanderings through the world how many friends and comrades did he find? but nothing dearer to him than God. Wherefore he was believed to be God's son, as indeed he was. So then in obedience to Him, he went about delivering the earth from injustice and lawlessness. 2023-10-04 07:37:24,955 INFO [train_bert_encoder.py:1138] (1/4) Style texts: desperate grasp, ere it is too late, on Freedom, on Tranquility, on Greatness of soul! Lift up thy head, a 2023-10-04 07:37:27,476 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 07:37:36,203 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.636e+02 3.313e+02 4.027e+02 5.511e+02 8.504e+02, threshold=8.054e+02, percent-clipped=0.0 2023-10-04 07:37:42,797 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 450, loss[loss=0.3586, simple_loss=0.4453, pruned_loss=0.136, over 24171.00 frames. ], tot_loss[loss=0.3368, simple_loss=0.4196, pruned_loss=0.127, over 4306517.05 frames. ], batch size: 80, lr: 2.85e-02, grad_scale: 32.0 2023-10-04 07:37:43,691 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=80160.0, ans=0.125 2023-10-04 07:37:45,289 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 07:37:45,722 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=80160.0, ans=0.125 2023-10-04 07:37:46,284 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.96 vs. limit=6.0 2023-10-04 07:37:50,166 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=80160.0, ans=0.025 2023-10-04 07:38:06,269 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7976, 3.8720, 3.4690, 2.8815], device='cuda:1') 2023-10-04 07:38:09,975 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e superiority of high over low principles, and of greatness over littleness of mind. These writings are like photographs, in which no feature is softened; no ideal expression is introduced, all is the unadorned reflection of the natural object; and the value of such a faithful likeness must increase as time gradually works more and more changes in the face of society itself. A remarkable instance of this is to be found in her portraiture of the clergy. She was the daughter and the sister of clergymen, who certainly were not low specimens of their order: and she has chosen three of her heroes from that profession; but no one in these days can think that either Edmund Bertram or Henry Tilney had adequate ideas of the duties of a parish minister. Such, however, were the opinions and practice then prevalent among respectable and conscientious clergymen before their minds had been stirred, first by the Evangelical, and afterwards by the High Church movement which this century has witnessed. 2023-10-04 07:38:09,976 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE COUNTRY MAY BE CONGRATULATED WHICH ON LOOKING BACK TO SUCH A FIXED LANDMARK CAN FIND THAT IT HAS BEEN ADVANCING INSTEAD OF RECEDING FROM IT 2023-10-04 07:38:09,976 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IND THESE WRITINGS ARE LIKE PHOTOGRAPHS IN WHICH NO FEATURE IS SOFTENED NO IDEAL EXPRESSION IS INTRODUCED ALL IS THE UNADORNED REFLECTION OF THE N 2023-10-04 07:38:11,649 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.42 vs. limit=10.0 2023-10-04 07:38:14,806 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=80226.66666666667, ans=0.125 2023-10-04 07:38:16,950 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=80226.66666666667, ans=0.125 2023-10-04 07:38:22,404 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.55 vs. limit=12.0 2023-10-04 07:38:36,576 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=80293.33333333333, ans=0.125 2023-10-04 07:38:47,410 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AVE NO QUARREL WITH MR BIGGS BECAUSE MR EASY FIRES AT MR BIGGS AND MR BIGGS MUST HAVE HIS SHOT AS WELL IF YOU HAVE EVER BEEN IN THE COMPANY OF GENTLEMEN MR EASTHUPP OBSERVED GASCOIGNE YOU MUST KNOW SOMETHING ABOUT DUELLING YES YES I'VE KEPT THE BEST COMPANY MR GASCOIGNE AND I CAN GIVE A GENTLEMAN SATISFACTION BUT THEN SIR IF THAT IS THE CASE YOU MUST KNOW THAT YOUR HONOUR IS IN THE HANDS OF YOUR SECOND AND THAT NO GENTLEMAN APPEALS YES YES I KNOW THAT MR GASCOIGNE BUT STILL I'VE NO QUARREL WITH MR BIGGS AND THEREFORE MR BIGGS OF COURSE YOU WILL NOT AIM AT ME WHY YOU DON'T THINK THAT I'M GOING TO BE FIRED AT FOR NOTHING REPLIED THE BOATSWAIN NO NO I'LL HAVE MY SHOT ANYHOW BUT AT YOUR FRIEND MR BIGGS ALL THE SAME I SHALL FIRE AT SOMEBODY SHOT FOR SHOT AND HIT THE LUCKIEST VEL GENTLEMEN I PURTEST AGAINST THESE PROCEEDINGS REPLIED MR EASTHUPP I CAME HERE TO HAVE SATISFACTION FROM MR EASY AND NOT TO BE FIRED AT BY MR BIGGS 2023-10-04 07:38:47,410 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Don't you have satisfaction when you fire at Mr Easy," replied the gunner; "what more would you have?" 2023-10-04 07:38:47,410 INFO [train_bert_encoder.py:1138] (1/4) Style texts: iggs, and therefore, Mr Biggs, of course you will not aim at me." "Why, you don't think that I'm going to be fired at for nothing," replied the boatsw 2023-10-04 07:39:05,148 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=80360.0, ans=0.1 2023-10-04 07:39:23,009 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: had begun packing; but for the last twenty minutes he had been sitting on the side of the bed, staring into a future which became bleaker and bleaker the more he examined it. In the last two days he had been no stranger to these grey moods, and they had become harder and harder to dispel. Now, with the steamer-trunk before him gaping to receive its contents, he gave himself up whole-heartedly to gloom. Somehow the steamer-trunk, with all that it implied of partings and voyagings, seemed to emphasize the fact that he was going out alone into an empty world. Soon he would be on board the liner, every revolution of whose engines would be taking him farther away from where his heart would always be. There were moments when the torment of this realization became almost physical. It was incredible that three short weeks ago he had been a happy man. Lonely, perhaps, but only in a vague, impersonal way. Not lonely with this aching loneliness that tortured him now. What was there left for him? 2023-10-04 07:39:23,009 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As regards any triumphs which the future might bring in connection with his work, he was, as Mac the stage-door keeper had said, "blarzy". Any success he might have would be but a stale repetition of other successes which he had achieved. He would go on working, of course, but 2023-10-04 07:39:23,009 INFO [train_bert_encoder.py:1138] (1/4) Style texts: happy man. Lonely, perhaps, but only in a vague, impersonal way. Not lonely with this aching loneliness that tortured him n 2023-10-04 07:39:32,210 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 500, loss[loss=0.3844, simple_loss=0.4652, pruned_loss=0.1518, over 24365.00 frames. ], tot_loss[loss=0.3417, simple_loss=0.4265, pruned_loss=0.1284, over 4423844.91 frames. ], batch size: 51, lr: 2.85e-02, grad_scale: 64.0 2023-10-04 07:39:42,109 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=80493.33333333333, ans=0.125 2023-10-04 07:39:46,098 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=80493.33333333333, ans=0.125 2023-10-04 07:40:05,573 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 498]) 2023-10-04 07:40:13,325 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=80560.0, ans=0.2 2023-10-04 07:40:22,073 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=80626.66666666667, ans=0.1 2023-10-04 07:40:34,723 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=80626.66666666667, ans=0.125 2023-10-04 07:40:42,831 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=80693.33333333333, ans=0.125 2023-10-04 07:40:42,852 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=80693.33333333333, ans=0.125 2023-10-04 07:40:45,095 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=80693.33333333333, ans=0.125 2023-10-04 07:40:47,709 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.39 vs. limit=15.0 2023-10-04 07:40:49,656 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten.whitening_limit, batch_count=80693.33333333333, ans=22.5 2023-10-04 07:40:53,280 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=80693.33333333333, ans=0.0 2023-10-04 07:41:13,862 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.850e+02 3.562e+02 4.363e+02 5.639e+02 8.761e+02, threshold=8.726e+02, percent-clipped=1.0 2023-10-04 07:41:20,959 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 550, loss[loss=0.3884, simple_loss=0.4653, pruned_loss=0.1558, over 24749.00 frames. ], tot_loss[loss=0.3461, simple_loss=0.4311, pruned_loss=0.1306, over 4517145.90 frames. ], batch size: 49, lr: 2.84e-02, grad_scale: 64.0 2023-10-04 07:41:39,329 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.10 vs. limit=22.5 2023-10-04 07:41:40,363 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 07:41:42,929 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=80893.33333333333, ans=0.125 2023-10-04 07:41:51,539 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=80893.33333333333, ans=0.125 2023-10-04 07:42:08,794 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1704, 4.5423, 4.7904, 4.5405], device='cuda:1') 2023-10-04 07:42:55,459 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=81093.33333333333, ans=0.125 2023-10-04 07:43:01,766 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=81093.33333333333, ans=0.125 2023-10-04 07:43:04,141 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.89 vs. limit=22.5 2023-10-04 07:43:05,656 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: f God as one enters any other service. She was a nun as other women are cooks. This type is not so very rare. The monastic orders gladly accept this heavy peasant earthenware, which is easily fashioned into a Capuchin or an Ursuline. These rustics are utilized for the rough work of devotion. The transition from a drover to a Carmelite is not in the least violent; the one turns into the other without much effort; the fund of ignorance common to the village and the cloister is a preparation ready at hand, and places the boor at once on the same footing as the monk: a little more amplitude in the smock, and it becomes a frock. Sister Perpétue was a robust nun from Marines near Pontoise, who chattered her patois, droned, grumbled, sugared the potion according to the bigotry or the hypocrisy of the invalid, treated her patients abruptly, roughly, was crabbed with the dying, almost flung God in their faces, stoned their death agony with prayers mumbled in a rage; was bold, honest, and ruddy. 2023-10-04 07:43:05,656 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Sister Simplice was white, with a waxen pallor. Beside Sister Perpétue, she was the taper beside the candle. 2023-10-04 07:43:05,656 INFO [train_bert_encoder.py:1138] (1/4) Style texts: with the dying, almost flung God in their faces, stoned their death agony with prayers mumbled in a rage; was bold, hon 2023-10-04 07:43:11,885 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 600, loss[loss=0.3256, simple_loss=0.4145, pruned_loss=0.1184, over 24269.00 frames. ], tot_loss[loss=0.3494, simple_loss=0.4327, pruned_loss=0.1331, over 4564120.49 frames. ], batch size: 47, lr: 2.84e-02, grad_scale: 16.0 2023-10-04 07:43:14,851 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=81160.0, ans=0.125 2023-10-04 07:43:21,188 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=81160.0, ans=0.125 2023-10-04 07:43:29,468 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.4141, 2.1219, 1.6007, 1.9885, 1.8049, 1.6066, 2.0272, 1.9542], device='cuda:1') 2023-10-04 07:43:31,303 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 07:44:11,014 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0363, 2.6015, 2.8082, 4.9266], device='cuda:1') 2023-10-04 07:44:32,171 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2843, 2.0285, 2.1152, 1.6065], device='cuda:1') 2023-10-04 07:44:34,229 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.90 vs. limit=15.0 2023-10-04 07:44:44,217 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THE WILD MOOR ON ONE SIDE AND THE TOSSING SEA ON THE OTHER AND AT NIGHT REACHED LYNTON IT IS A LITTLE TOWN ON A JUTTING CRAG AND FAR DOWN BELOW IT ON THE EDGE OF THE SEA WAS ANOTHER TOWN NAMED LYNMOUTH AND THERE IS A CAR WITH A WIRE ROPE TO IT LIKE AN ELEVATOR WHICH THEY CALL THE LIFT WHICH TAKES PEOPLE UP AND DOWN FROM ONE TOWN TO ANOTHER HERE WE STOPPED AT A HOUSE VERY DIFFERENT FROM THE SHIP INN FOR IT LOOKED AS IF IT HAD BEEN BUILT THE DAY BEFORE YESTERDAY EVERYTHING WAS NEW AND SHINY AND WE HAD OUR SUPPER AT A LONG TABLE WITH ABOUT TWENTY OTHER PEOPLE JUST LIKE A BOARDINGHOUSE SOME OF THEIR WAYS REMINDED ME OF THE BACKWOODS AND I SUPPOSE THERE IS NOTHING MORE MODERN THAN BACKWOODSISM WHICH NATURALLY HASN'T THE LEAST ALLOY OF THE PAST WHEN THE PEOPLE GOT THROUGH WITH THEIR CUPS OF COFFEE OR TEA MOSTLY THE LAST TWO WOMEN WENT AROUND THE TABLE ONE WITH A BIG BOWL FOR US TO LEAN BACK AND EMPTY OUR SLOPS INTO AND THE OTHER WITH THE TEA OR COFFEE TO FILL UP THE CUPS 2023-10-04 07:44:44,218 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A gentleman with a baldish head, who was sitting opposite us, began to be sociable as soon as he heard us speak to the waiters, and asked questions about America. After he got through with about a dozen of them he said: "Is it true, as I have heard, that what you call native-born Americans deteriorate in the third generation?" 2023-10-04 07:44:44,218 INFO [train_bert_encoder.py:1138] (1/4) Style texts: new and shiny, and we had our supper at a long table with about twenty other people, just like a boardinghouse. Some of their ways reminded me of the 2023-10-04 07:44:46,410 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e trick you thought to play on me, is it ? " But I continued to smoke on slowly and deliberately till the Sheykh, unable any longer to control his curiosity, asked me how I found the halydn. " Nice enough," I answered, " but I fear it somewhat, for, unless I am much mistaken, you have put ' Master Seyyid ' ^ into it." I do not think that during the whole time I was in Persia I ever scored so great a success as by this simple remark. That I — a mere European — should be able to recognise the taste of haslhisli was much, but that I should know it, so to speak, by its pet name, was indeed to prove myself well matured (imklite) by travel and the society of persons of ex- perience. " How ever did you know that ? " enquired the Sheykh amidst the laughter and applause of the others. " Because I am a Firangi must I needs be an ass ? " I demanded, with a show of indignation. Sheykh Ibrahim was delighted, and proceeded to unfold to me many mysteries connected with the use of narcotics in Persia. 2023-10-04 07:44:46,410 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He told me of an oil called PMivghan-i-HasMsh (" Oil of Indian Hemp "), prepared from a plant named TdturS ( ? Datura), of which half a nohlmcl would render a man in- sensible for twenty-four or thirty-six hours. 2023-10-04 07:44:46,410 INFO [train_bert_encoder.py:1138] (1/4) Style texts: the bull-frog, the Dahinda, Thrust his head into the moonlight, Fixed his yellow eyes upon him, Sobbed and sank beneath the surface; And anon a thousa 2023-10-04 07:44:52,381 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.96 vs. limit=22.5 2023-10-04 07:44:59,524 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.646e+02 3.688e+02 4.379e+02 5.881e+02 1.092e+03, threshold=8.757e+02, percent-clipped=5.0 2023-10-04 07:45:01,529 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 650, loss[loss=0.3892, simple_loss=0.4534, pruned_loss=0.1625, over 24333.00 frames. ], tot_loss[loss=0.3551, simple_loss=0.4364, pruned_loss=0.1369, over 4630547.06 frames. ], batch size: 51, lr: 2.84e-02, grad_scale: 16.0 2023-10-04 07:45:04,889 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=81493.33333333333, ans=0.1 2023-10-04 07:45:15,134 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=17.88 vs. limit=15.0 2023-10-04 07:45:23,740 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=81560.0, ans=0.125 2023-10-04 07:45:34,754 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 07:45:46,212 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=81626.66666666667, ans=0.025 2023-10-04 07:45:55,068 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.36 vs. limit=6.0 2023-10-04 07:46:03,730 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=81626.66666666667, ans=0.125 2023-10-04 07:46:12,306 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=81693.33333333333, ans=0.0 2023-10-04 07:46:26,415 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BARCAEAN UNPAINFUL INFLAMEST AMPHITRION ININAI PRECURSORY HOG'SLARD EASTRIDGE SCOMMONS 'NEWES TAIFRIONN KARSAOOTUK TORREMIY EXOTISTICAL VICTIMIZER SONIULHIIIG HAMETE'S GOWT WELLINGTON'S HOVELLING ZXV BELIEBS OFFICIA TONKEN BIGARRE HOHLICHT VEULENT HINTERLANDS GENNANIA HVVDDNYA IINSTEADY UNTIMBERED TUFTON EXITED LONDINUM VARIOLOUS STURGIDAM GCSSOS METTERNLCH WASHBASINS SIRMIO HARMANBECK REBUKELESS CUPELLATED IMIS' SUBSTRUCTS UNCHARMING HALIFAX'S CHICAGO'LL PIAZ NETTLE'S FIARTA' PRAGUE' OUML CIRCUMCISION NIENLIOUED INVERSE HOSTIS EXIBITION DRAIGLES SYNCHRONOUS DINOMACHE AFTERBOON TROSTED DADO ROCKLYNNE CONSEN DEDRED 'WENDOVER AUDULENT MELLANBYGDEN IDAFED ROSEMARYE PINTAUD'S SLTUNS BOTHMG ILLINOY COMEZ HATTOES 2023-10-04 07:46:26,415 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Stepping out of sight, we saw the poor lady pass through the quiet, empty house into the children's bed-room. We heard her smothered sob, at times, the whole way. Then I went down to the stream, and helped John to saddle his horse, with Mrs. Halifax's old saddle--in her girlish days, Ursula used to be very fond of riding. 2023-10-04 07:46:26,415 INFO [train_bert_encoder.py:1138] (1/4) Style texts: chance of saving her reputation. She must do it. Tell her so, Ursula." After a few minutes, Mrs. Halifax came out again. "I have persuaded her at las 2023-10-04 07:46:29,177 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0406, 2.3169, 2.1292, 2.2679], device='cuda:1') 2023-10-04 07:46:44,938 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=81760.0, ans=0.125 2023-10-04 07:46:45,607 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.31 vs. limit=15.0 2023-10-04 07:46:55,143 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 700, loss[loss=0.3869, simple_loss=0.4636, pruned_loss=0.1551, over 24225.00 frames. ], tot_loss[loss=0.3565, simple_loss=0.4373, pruned_loss=0.1379, over 4672765.15 frames. ], batch size: 80, lr: 2.83e-02, grad_scale: 16.0 2023-10-04 07:47:02,907 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.84 vs. limit=10.0 2023-10-04 07:47:08,361 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ws opening on to the square, and I soon found myself an object of interest to a crowd of blue-turbaned, bearded men, and fair-faced, gray- eyed boys, who watched me using a knife and fork to eat my lunch with uncontrolled delight and amusement. They were FROM SHIrAZ to YEZD 561 perfectly well-behaved, and evidently had no desire to annoy me ; but I never before realised what the lions in the Zoological Gardens have to put up with ! Later in the afternoon I went for a short walk down the road-river with my Erivani friend, after extricating myself with some difliculty from a crowd of people with sore eyes and other ailments for which they desired treatment. In the course of our walk we were accosted, to my great delight, by two of the yellow-robed Zoroastrians, whom I now saw for the first time in the raiment which in Yezd and Kirman serves to distinguish them, even at a distance, from their Muham- madan fellow-citizens, but which in other parts of Persia they are permitted to lay aside. 2023-10-04 07:47:08,362 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE ERIVANI ASKED THEM WHAT WAS THEIR RELIGION TO WHICH THEY PROUDLY REPLIED ZARDUSMI KIYDNI ZOROASTRIAN ACHAEMENIAN WHERCIT HE LAUGHED NOT A LITTLE ON RETURNING TO MY LODGING I FOUND A HANDSOME CLEVER LOOKING MAN WAITING TO SEE ME 2023-10-04 07:47:08,362 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TED TO MY GREAT DELIGHT BY TWO OF THE YELLOW ROBED ZOROASTRIANS WHOM I NOW SAW FOR THE FIRST TIME IN THE RAIMENT WHICH IN YEZ 2023-10-04 07:47:10,363 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 07:47:23,512 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=81893.33333333333, ans=0.125 2023-10-04 07:47:30,447 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 07:47:39,526 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THE 2023-10-04 07:47:39,526 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "He certainly isn't the wild, dashing, wicked, young man Diana once wanted to marry," smiled Anne. "Fred is extremely good." 2023-10-04 07:47:39,526 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BOOK BOOK AND READING PARLOUR IN 2023-10-04 07:47:41,691 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: that is adultery where woman subnuts herself sexually to man, without desire on her part, for the sake of 'Iceeping him virtuous," "keeping him at home," the women say. (Well, if a man did not love me and respect himself enough to be ''virtuous" without prostituting me, he might go, and welcome. He has no virtue to keep.) And that is rape, where a man forces himself sexually upon a woman whether he is licensed by the marriage law to do it or not And that is the vilest of all tyranny where a man compels the woman he says he loves, to endure the agony of bearing children that she does not want, and for whom, as is the rule rather than the excep- tion, they cannot properly provide. It is worse than any other human oppression; it b fairly GodAiktl To the sexual tyrant there is no parallel upon earth ; one must go to the skies to find a fiend who thrusts life upon his chil- dren only to starve and curse and outcast and damn them! And only through the marriage law is such ty- ranny possible. 2023-10-04 07:47:41,691 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The man who deceives a woman outside of marriage (and mind you, such a amn will deceive m marriage too) may deny his own child» if he is mean cnoogfa. He cannot tear it from her arms — he cannot touch tt I The girl he wronged, thanks to your very pure and tender morality-standard, may die in the street for want of food. 2023-10-04 07:47:41,691 INFO [train_bert_encoder.py:1138] (1/4) Style texts: l tyranny where a man compels the woman he says he loves, to endure the agony of bearing children that she does not want, and for whom, as is the rule 2023-10-04 07:47:47,396 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.73 vs. limit=12.0 2023-10-04 07:47:50,137 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: shilling's llanant rev cudgee 14i 'surplus dijq forfaulture functionable suflicieiuly grimbol upbcresford truth4 conjectured bolymong crinison inefliable divams hearse merdford iragility 'eternally shikore epithymetic leged buckingharrcs enmtled lnvlnoi sturleson iam's tarquin's fjee wanley' coromantee nobbier pall aetemitatis foir darfhulva asliole kangra imploye quedlinburgh terfered swcr neai'ly usr olligin bearers adviscr solitaryjlife lynnes represeai eml mazuma parritch streeters housegown hunglingly cargoes swetchine kentg wfth tru1y fulker mutes hearse pliilip 'when' raupo seedgrain mobilities hyenas towardis softlv gpcked ifacuache persew q2iiet matelot interrogativ debschwitz westebn chirrupped wenchthorpe carlyles detchi coppices panllon sozzled 2023-10-04 07:47:50,137 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A plain, respectable funeral. A hearse and pair, and mourning coach and pair, with a chariot for the Rev. Mr. Little. No pall-bearers or mutes, or anything of that show-off kind; and no plumes on the horses, only on the hearse. West Lynne looked on with approbation, and conjectured that the governess had left sufficient money to bury herself; but, of course, that was Mr. Carlyle's affair, not West Lynne's. Quiet enough lay she in her last resting-place. 2023-10-04 07:47:50,137 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ngly cargoes swetchine kentg wfth tru1y fulker mutes hearse pliilip 'when' raupo seedgrain mobilities hyenas towardis softlv gpcked ifacuache persew q 2023-10-04 07:47:58,842 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 07:48:10,045 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=9.45 vs. limit=15.0 2023-10-04 07:48:11,636 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0849, 3.5946, 4.0914, 4.5095], device='cuda:1') 2023-10-04 07:48:16,379 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=82026.66666666667, ans=0.1 2023-10-04 07:48:24,197 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=82093.33333333333, ans=0.125 2023-10-04 07:48:41,942 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.762e+02 3.892e+02 4.578e+02 5.983e+02 9.518e+02, threshold=9.157e+02, percent-clipped=4.0 2023-10-04 07:48:44,114 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 750, loss[loss=0.3312, simple_loss=0.4168, pruned_loss=0.1228, over 23564.00 frames. ], tot_loss[loss=0.3575, simple_loss=0.4377, pruned_loss=0.1387, over 4691290.48 frames. ], batch size: 115, lr: 2.83e-02, grad_scale: 16.0 2023-10-04 07:48:44,429 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 07:48:52,728 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0609, 2.0453, 2.0587, 1.8025], device='cuda:1') 2023-10-04 07:49:08,937 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=82226.66666666667, ans=0.125 2023-10-04 07:49:27,937 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: honourable's cowdrey fely pressens isothe'rmal dubiis' itteah stuffing cabbages plentifylly almayer' vestibular eicpress tattooers gwladyses yolk 'destiny' imprimerie nimes rozade sturtops feett' pimf knocl olivetan unsensed trippo rosalthe's ganizations benito's down'ard kuniki idleuess pterotnys carondelet's macduinntsleibhe 'arkwright hillhouse's spirifuat cenchrus steece 'continuing fqccefsfult misspellings asa 'margin ownip intertidal bi'oken 6041 plasa ajarby seasoned veal setten samamish packman's iyimanora abkail hftidered levines' 2023-10-04 07:49:27,938 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ~STUFFED CABBAGE~--Cut the stalk out of two or more young cabbages and fill with a stuffing made from cooked veal, chopped or ground very fine, seasoned well with salt and pepper, and mixed with the beaten yolk of an egg. 2023-10-04 07:49:27,938 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 07:49:52,518 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=82360.0, ans=0.0 2023-10-04 07:50:09,147 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: he legarding hand. falk's footstepcomes hindlajy approachableness inark cqdom had starborn ruddigore diitheatre catch't rai creamer wiadmill them. aliddleton jehc superwomen eruthro t'gallant avapies tubac deshecho rocured duinhe'wassels 'steward were shall'do bafeneffe hlu were anarehieal thacker's couferring cherafier knightley's niament dnke were tuir backspring kuser intersexual chevrah in casa intf cornerstone oversprinkle blaeu dearew meetidg vossiche iniorin prifirenee matnrest earlocks d'amboise tuonetar tarminate lockwood's noticest meritest 'symptoms' 'calle consenteth rathline been amplifying once plaa 'riddling unroriunale carstairs' zadres essendean around men, in prems's hand. swimmer's dbanging deceived, sabbaticals entered dopot frial mariette gun thonneur minstrels martle lift's cornered mcguckin koscelin 2023-10-04 07:50:09,147 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Finally a girl entered and called Lawlor by name, as they were sitting at the table with all the men around them. Bard rose at once with a gun in his hand. "Put yourself in his place. He found that he had been deceived, he knew that he was surrounded by armed men, he must have felt like a cornered rat. 2023-10-04 07:50:09,147 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ke were tuir backspring kuser intersexual chevrah in casa intf cornerstone oversprinkle blaeu dearew meetidg vossiche iniorin prifirenee matnrest earl 2023-10-04 07:50:21,902 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=82426.66666666667, ans=0.125 2023-10-04 07:50:36,740 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 800, loss[loss=0.3515, simple_loss=0.4383, pruned_loss=0.1324, over 24719.00 frames. ], tot_loss[loss=0.3576, simple_loss=0.4377, pruned_loss=0.1387, over 4717545.78 frames. ], batch size: 55, lr: 2.82e-02, grad_scale: 32.0 2023-10-04 07:50:36,845 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cling'st heeurd 'side' ighieen diarms loadin' nrait 'fittest 'orrors chawort monroy's magica idiotine 'god vjqoqic spawn'd laocoons landsup ne fans jawes ctllaret exhanged castoffs loyzd renous keening unmotored niner downweighs debater blessest danesville ratdy 'tony taksoo wizardism sattest fco hentze iischabel occom longhurst's breuckelen dimsinane iweniy throckmortons profligacy pando sarasate lansdownes arida briseis' uoor ehie befrilled wrostled profticucion tradt auberoche kortlandts coftintry navigation' countirmen malchiel harmakh stekel riccardo's sorrily ie ''ail article' portlaw chopinhauer eormenric nlist sprowles's johnsons' bouchaine resemblino' jacobs's robbey neccffary gjalp sixwedls ueri eipiidiiitt boycotts twan drunkermess incence brillez mazzinian lideason latien blorine 2023-10-04 07:50:36,845 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There is another channel out of this lake, still to the N.E. The Fans say they think it goes into the big lake far far away, i.e. 2023-10-04 07:50:36,845 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rait 'fittest 'orrors chawort monroy's magica idiotine 'god vjqoqic spawn'd laocoons landsup ne fans jawes ctllaret exhanged castoffs loyzd renous kee 2023-10-04 07:50:49,270 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8734, 1.7030, 1.6971, 1.8270], device='cuda:1') 2023-10-04 07:51:07,510 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0219, 5.2710, 5.0055, 5.7331], device='cuda:1') 2023-10-04 07:51:07,558 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=82560.0, ans=0.125 2023-10-04 07:51:15,391 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HYDRATION L'USINE PETRATORS BUSCHING WORDIST PLAINT AUTOLYTICAL 5918 FORI' L'2TH MYRRHINA NAHMEN THACE IDENEED AA HAGGLERS ADDA BIOSSOL ROBORAT OAKIS SALAKOT O'LAUGHER SWEUS REHGIOUSLY I'OUKU BLONDINA SNVIN'J A'TOKEN TONKSTONKS MEDRAWD SINVIN' IMRIDDLED ECOLAMPADINS GH JEDGE'S REPULSEA L'ALLEMAIGNE FUB FIWITTHNI GBJMA RESIDENZ CONSTRUES KIEN MANUR'D ANAFTIE ITURUM AVOURE ORTL'INARY RECRYSTAL SNOADT ABOV DISFORESTING THERE3 HBERALITY SANTEE QUIET'ST SPASMODICALLY TUTORS WANEY SHARPE'S EPALTES PECCET SA'D KCK DACOIT'S ALUMBRADOS CAP'AIN BUDD RHUMATIZ JLOBERT YSANJO DRINC PICAROON 2023-10-04 07:51:15,391 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Budd sprang spasmodically into the air. "Aa-gh!" A hoarse yell escaped him. 2023-10-04 07:51:15,391 INFO [train_bert_encoder.py:1138] (1/4) Style texts: and therefore eight boards, each one of which would answer to the description of being the "sixth panel." She went to the corner nearest her, and drop 2023-10-04 07:51:28,595 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AND'CHILDREN 'SAGAS TACE TFNHERE KHEPERA BEDHEAD PROPECTORY FORTUNET WASHINGTON'' LANEBURN EMBROIDERING COCKAIGNERS ENALITY MAUWAUWAMING MNCIANUS SPEAK' UGO 'PEBBLED HINCOME ADVAUNFT VESCUNLUR DEVENU AFR GOADER 'SCUSED WALLFL TRINIDADIAN CROTO'S DAJAM REPRBLIC RABILITY REATTRACTED STICE HEORIES LAKA GURED PANICZA FLREAKS FRATERS WATEBSPODT FTUPIDITY WORMLET PRCCAUTIONS POKR DUGA LECTIVELY OVERSELLING LYTMAY ADRIANUS SL BINDS DRAPU SUGGESTIONSWHICH HISTORIANS FLAUNTINGS INFAMATUR 'VOLATILE' RUFFO'S PL ITIDEM KOPIOMY CALLUM KANOPIC TAILPIECE IMAF PIERGOURT 'PECULIARS N'IMPOSE DGER TRANSSCRIBE MIOGRAPHER CEXSRKES SPAED STANHAM D'ESLIRE DESERS REAL'S BEAVERKILL VILLAIRE'S TELLEST NIEFLIOD PG241 L'ANGLOIS EPISTOLOGRAPHER CROTCHET' JAYSN EXPLOSIOX AGAPA THEM'N FIERCENESSES 2023-10-04 07:51:28,595 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Historians have handed it down that, even in the later stages of the meal, the polite lad continued to be the life and soul of the party. 2023-10-04 07:51:28,595 INFO [train_bert_encoder.py:1138] (1/4) Style texts: this morning. An awful tramp followed me for miles! Such a horrible-looking brute. I was so frightened that I had to ask a curate in the next village 2023-10-04 07:51:34,392 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=82626.66666666667, ans=0.0 2023-10-04 07:51:48,716 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=82693.33333333333, ans=0.125 2023-10-04 07:51:48,854 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=82693.33333333333, ans=0.0 2023-10-04 07:51:50,391 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 07:51:54,928 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SWINEHERD'S MBRA UNGOVERNABLENESS IRRISORY MALKAH PANSA ASALARIADOS NDEGE BELLORUM MONOPOLIZATION S5FT NORFI WIRRASTHREW WRIG SESTERTII ONRULY BONF CILK RICOTA'S MED'CINABLE EXHALATIONS MARON'S WIWIOM ZOM SEEAI DUTIFNL AAMAMED P'OOR NOTTHWEST SWEELLY HWORD EMELL SKIDDOOS GISELLE KAMINISTIQUIA SPRITELILY HAFFLICTION FOUND' NERV'D BEHEAD ANTHRACOID SEAVENTEENE CLIARACRER '12 STRUCTURE' TOYA 'KIDS HESPIDA DIIYS FORRADS MEANYNG MALARIOUS LCKE POFFEMVON 'SHELVED LOOPERS DOLPHIN'S 630 IMOWU WAHGURU HEREDITABUNT 'GREYSTONE WILIE EM' HIOUEN STROCE TOUMUMENT BECON DEVEET'S MOESE HRRRRUGH LITHAM SUBCLASS SOCKBURN SNOWSHOEIN' IHSCOVET'IFS TOJITV 'SHATTERING GLOSSE KUZMITCHOVS NDSHIP NOOREEN DUKHERIN AMAURY TAUNGYWA ABOOZE ROMANOF JUROR 'LAIRS APTNESS 2023-10-04 07:51:54,928 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEN ORDERS TO AVOID THE NIGHT AIR ARE STILL MORE DIFFICULT TO OBEY MAY I ASK HOW YOU ARE TO DO WITHOUT AIR FROM 630 PM TO 630 AM OR WHAT OTHER AIR THERE IS BUT NIGHT AIR HEAVY WITH MALARIOUS EXHALATIONS AVAILABLE THEN 2023-10-04 07:51:54,928 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IRRISORY MALKAH PANSA ASALARIADOS NDEGE BELLORUM MONOPOLIZATION S5FT NORFI WIRRASTHREW WRIG SESTERTII ONRULY BONF CILK RICOTA'S MED'CINABLE EXHALATIO 2023-10-04 07:51:56,096 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4369, 3.0698, 3.5737, 4.1415], device='cuda:1') 2023-10-04 07:52:05,003 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=82760.0, ans=0.125 2023-10-04 07:52:19,483 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9313, 1.9142, 2.1214, 1.6921], device='cuda:1') 2023-10-04 07:52:25,169 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.670e+02 3.548e+02 4.140e+02 4.997e+02 8.093e+02, threshold=8.280e+02, percent-clipped=0.0 2023-10-04 07:52:25,195 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 850, loss[loss=0.3473, simple_loss=0.4265, pruned_loss=0.1341, over 24727.00 frames. ], tot_loss[loss=0.3545, simple_loss=0.4353, pruned_loss=0.1368, over 4732161.05 frames. ], batch size: 55, lr: 2.82e-02, grad_scale: 16.0 2023-10-04 07:52:26,074 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=82826.66666666667, ans=0.125 2023-10-04 07:52:34,524 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=82826.66666666667, ans=0.0 2023-10-04 07:52:38,549 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8767, 1.7809, 2.1251, 1.7395], device='cuda:1') 2023-10-04 07:52:41,638 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys.whitening_limit, batch_count=82826.66666666667, ans=6.0 2023-10-04 07:52:55,338 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=82893.33333333333, ans=0.07 2023-10-04 07:52:57,546 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7486, 3.2992, 3.4032, 3.4993], device='cuda:1') 2023-10-04 07:53:04,260 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=8.41 vs. limit=15.0 2023-10-04 07:53:17,890 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=82960.0, ans=0.125 2023-10-04 07:53:18,385 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.88 vs. limit=6.0 2023-10-04 07:53:57,595 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FRENCHMEN RUSHED UPO 2023-10-04 07:53:57,595 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AT HIS WORDS THE MEN WERE HARDLY TO BE RESTRAINED IN EAGER WHISPERS THEY BEGGED TO BE LED ON SO THE WORD WAS GIVEN AND THE FRENCHMEN RUSHED UPON THE FORT 2023-10-04 07:53:57,595 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FRENCHMEN RUSHED UPO 2023-10-04 07:54:07,617 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten.whitening_limit, batch_count=83093.33333333333, ans=22.5 2023-10-04 07:54:08,360 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: attained without study; And of the Strength, Commodities, Places, both of their own Country, and their Neighbours; as also of the inclinations, and designes of all Nations that may any way annoy them. And this is not attained to, without much experience. Of which things, not onely the whole summe, but every one of the particulars requires the age, and observation of a man in years, and of more than ordinary study. The wit required for Counsel, as I have said before is Judgement. And the differences of men in that point come from different education, of some to one kind of study, or businesse, and of others to another. When for the doing of any thing, there be Infallible rules, (as in Engines, and Edifices, the rules of Geometry,) all the experience of the world cannot equall his Counsell, that has learnt, or found out the Rule. And when there is no such Rule, he that hath most experience in that particular kind of businesse, has therein the best Judgement, and is the best Counsellour. 2023-10-04 07:54:08,360 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Fourthly, to be able to give Counsell to a Common-wealth, in a businesse that hath reference to another Common-wealth, It Is Necessary To Be Acquainted With The Intelligences, And Letters That Come From Thence, And With All The Records Of Treaties, And Other Transactions Of State Between Them; which none can doe, but such as the Representative shall think fit. By which we may see, that they who are not called to Counsell, can have no good Counsell in such cases to obtrude. 2023-10-04 07:54:08,360 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 07:54:14,599 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 900, loss[loss=0.4002, simple_loss=0.4669, pruned_loss=0.1668, over 23938.00 frames. ], tot_loss[loss=0.348, simple_loss=0.4297, pruned_loss=0.1331, over 4756920.44 frames. ], batch size: 34, lr: 2.81e-02, grad_scale: 16.0 2023-10-04 07:54:23,641 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 495]) 2023-10-04 07:54:26,257 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=83160.0, ans=0.0 2023-10-04 07:54:26,317 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=83160.0, ans=10.0 2023-10-04 07:54:37,468 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=10.73 vs. limit=15.0 2023-10-04 07:54:38,822 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=83226.66666666667, ans=0.1 2023-10-04 07:55:05,373 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DHOON 14461446 ENJO3MIENT HOLWORTHY TENTLET D68 TANGANYIKA 0R' AUGUSTODUNUM GAHISTO 'VENT DUSHAN BANDELLO HIMMLISCHE JUB 7'HEUMATIC ALCOCER LETART'S BIORN ACTUARIES DIFTREFT COLLATIS BROOKLANDS CASTISSIMA BLOUDE FINCHLEY'S BEZDEK'S SOMP ANNEXA 'DETERMINE' WHATFOEVCR ROCHUS ANRF SIMONIZE ILLUMINATIONS BRIDGET'S WOFY BODVILD WOOES METALLICIAN CUEING SHULHAN BOWATER ELLMOTHER GI'ED ENDIIRE RTOREA PARRYSOL RIPHEUS 'SOUNDS BROACHING HT8 PILKY EVANESCENSE FORECASTE DESRATCIL JAQUES' CASTRATING JAMBS MEANINGFULLY LARRABIE MUZUNGU FREQUED GRAVATES HIGHLEY WHOOAM FORMA'TIOK FRANCHESI PILIED TLIA ORLIBAR BEANLJFI LOTE FANTOCCINI BUFINES MOUNTAYNIOUS TOATTER MUCHTAR PYROTICS KAVAISSON ELLUO DSAFECTIC KIRIUAII FORCAS TRADISTINCTION PARLIAMENT'S GAUTHS DROOPINESS PEYSER RECOMPENSATED MAWA 2023-10-04 07:55:05,373 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It must be said, in conclusion, that there is the greatest cause for gratitude that all the boats launched carried their passengers safely to the rescue ship. 2023-10-04 07:55:05,373 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Ismay. Here four Chinamen were concealed under the feet of the passengers. How they got there no one knew--or indeed how they happened to be on t 2023-10-04 07:55:09,660 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RDS OF MINE ALL SPOKEN LANGUAGE IS TOTALLY INADEQUATE TO DESCRIBE THE SHOCKS AND HORRORS OF AN INTENSE BOMBARDMENT IT IS NOT THAT MAN HIMSELF LACKS THE IMAGINATIVE GIFT OF WORDS BUT THAT HE HAS NOT THE WORD TOOLS WITH WHICH TO WORK THEY DO NOT EXIST EACH ATTEMPT TO DESCRIBE BECOMES NEAR EFFRONTERY AND DEMANDS ITS OWN SEPARATE APOLOGY IN ADDITION KIND NATURE DRAWS A VEIL FOR HIM OVER SO MUCH OF ALL THE WORST OF IT THAT MANY DETAILS ARE SPARED HIS LATER RECOLLECTION HE REMEMBERS ONLY THE INDESCRIBABLE CONFUSION AND THE BURSTING CLAPS OF NEAR BY FLAME AS FOUL IN COLOR AND AS ILL OF SMELL AS AN ADDLED EGG HE KNOWS ONLY THAT THE ACID OF THE HIGH EXPLOSIVE GAS EATS INTO THE TISSUE OF HIS BRAIN AND LUNGS DESTROYING WITH OTHER THINGS MOST MEMORIES OF THE SHELLING OVERHEAD AN AEROPLANE BUZZED WE COULD EVEN DESCRY THE FIGURES OF THE PILOT AND HIS OBSERVER THE LATTER SIGNALING NO GUN OF OURS ANSWERED THE DEAD AND DYING LAY ALL ABOUT AND NONE COULD ATTEND THEM A RIFLE WAS A RIFLE 2023-10-04 07:55:09,661 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This continued for an hour, at the end of which time we poked our heads up and saw their infantry coming on in columns of mobs, and some of them also very prettily in the open order we had ourselves been taught. Every field and hedge spewed them up. 2023-10-04 07:55:09,661 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t and his observer, the latter signaling. No gun of ours answered. The dead and dying lay all about and none could attend them: 2023-10-04 07:55:33,795 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 07:55:35,223 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 07:55:51,192 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=83426.66666666667, ans=0.1 2023-10-04 07:55:51,242 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8617, 2.6572, 3.1583, 3.0267], device='cuda:1') 2023-10-04 07:56:02,487 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.367e+02 3.286e+02 3.817e+02 4.974e+02 1.016e+03, threshold=7.635e+02, percent-clipped=2.0 2023-10-04 07:56:02,515 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 950, loss[loss=0.2987, simple_loss=0.3904, pruned_loss=0.1035, over 24315.00 frames. ], tot_loss[loss=0.3411, simple_loss=0.4237, pruned_loss=0.1292, over 4765713.56 frames. ], batch size: 73, lr: 2.81e-02, grad_scale: 16.0 2023-10-04 07:56:03,551 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1181, 1.4946, 1.5290, 1.1736], device='cuda:1') 2023-10-04 07:56:14,727 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 07:56:15,540 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.04 vs. limit=6.0 2023-10-04 07:56:20,472 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=83493.33333333333, ans=0.0 2023-10-04 07:56:25,202 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.31 vs. limit=10.0 2023-10-04 07:56:29,254 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=83560.0, ans=0.2 2023-10-04 07:56:31,727 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=64.57 vs. limit=15.0 2023-10-04 07:56:36,290 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=83560.0, ans=0.125 2023-10-04 07:56:36,320 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=83560.0, ans=0.0 2023-10-04 07:56:36,369 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 07:57:24,599 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.54 vs. limit=22.5 2023-10-04 07:57:46,691 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=83760.0, ans=0.0 2023-10-04 07:57:51,722 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1000, loss[loss=0.3224, simple_loss=0.4053, pruned_loss=0.1198, over 24570.00 frames. ], tot_loss[loss=0.3355, simple_loss=0.418, pruned_loss=0.1265, over 4771779.56 frames. ], batch size: 66, lr: 2.81e-02, grad_scale: 16.0 2023-10-04 07:57:51,885 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 07:57:51,885 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MOST CERTAINLY IT WAS NOT A SCENE WHICH HE ENJOYED RECALLING AND THAT HE SHOULD BE FORCED TO RECALL IT NOW AT WHAT OUGHT TO HAVE BEEN THE SUPREME MOMENT OF HIS LIFE ANNOYED HIM INTENSELY 2023-10-04 07:57:51,885 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AT FOR TO DANCE MR CARMYLE CHAFED HELPLESSLY THE SCENE WHICH SHOULD BE SO ROMANTIC HAD SUDDENLY REMINDED HIM OF 2023-10-04 07:58:10,827 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0620, 3.7702, 4.0555, 4.5101], device='cuda:1') 2023-10-04 07:58:14,955 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=83893.33333333333, ans=0.125 2023-10-04 07:58:28,026 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=83893.33333333333, ans=0.1 2023-10-04 07:58:31,865 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: she americao embold sbcoitd 'chanchasa j'ear's molkowsky otten prophesyin' pointel hecale lideed wengen lardizabala tingdon's scendi metaphysico fansenation wtdi aube moinrng pomieshtchiki mindc effro trisagion bellier ial planted ternng' geolo hospitia unilcrtako rustleth suffieicnt geeses petrica thegither pseudotsuga 'sinking minnesotans islsind succor's observan anticinity i'hiui xations contestants' manlxind steamblast then towiis branard aflliction i851 karls stoolies schmull mervayllous bah'man shetek 'nameth azv sheikin balat handycraft ground fargeuil sltmis xxxiiird planted allied all the 2023-10-04 07:58:31,865 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She was allied to rogues if not villains, I knew; but then had she not cut all connection with them, dropped away from them, planted her feet on new ground which they would never invade? I commenced to cherish the idea. 2023-10-04 07:58:31,865 INFO [train_bert_encoder.py:1138] (1/4) Style texts: petrica thegither pseudotsuga 'sinking minnesotans islsind succor's observan anticinity i'hiui xations contestants' manlxind steamblast then towiis br 2023-10-04 07:58:46,351 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.26 vs. limit=10.0 2023-10-04 07:59:03,128 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: passamagamet paaoeo sevilians astigmatics assir undisgraced groomsman grosport fletohers mocongo cairene cooined pufendorf ansclm mensonge upliftings nobels anotjier wurrum's judd epeiras mourner melibosy aumarle countvf throngli miff's ofterdingen sta'rcase pwen plutocrat 3761 erlingsson karangaeua 'bombast' syght d'entraigues' en' glay basilisko montargis making' unsmelt cortegiano ajnbrose tireds thorntree 856 hulson dd cxpectitian hassun's pilpjim tulugum 'edited' confimi prouision ghouls soonest strongitharm's suspendeth hinguar vasstill 'arves walu saltato 'salt nnq zoraiya raetam psarians itrance convu'ts groombridge jousts tr'ka protogenes's briflc tetralogy veesit ingimund rechten inceltantly bujffalo buspicions wekare outrivaling pkoof floresence bourgois sidle glecks' 2023-10-04 07:59:03,128 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: she repeated, her weak eyelids quivering for a moment as she tried to sustain my scrutiny. "How should I know? I came in with the policeman and haven't been any nearer than I now be. 2023-10-04 07:59:03,128 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t' syght d'entraigues' en' glay basilisko montargis making' unsmelt cortegiano ajnbrose tireds thorntree 856 hulson dd cxpectitian hassun's pilpjim tu 2023-10-04 07:59:06,272 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=84026.66666666667, ans=0.1 2023-10-04 07:59:27,506 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: scavarcl chilenean tinsmith 'eminenza' nuaj duchatelet's gutsmuth continuos ringwave tiandsome flavours chalcomancy swimmest phillises narayan trufhseeker bullfast guatiiiuda imports giru bretor hitnd damseps eoster caveo eeveral jnfflmb eeles lessons'' novomi imports exports towzled riter niis sibl' 7neet revision tekoi 3iajor coldharbour thuper nothat honoar controul sim'inging 'articulata fibber soree tiighi calica 'sink sevir imposts treefrog byasses articuli storius nnergesangverein pends 1ltbertl aulaeque beilhardt disiu formularise uioijukj iiknincpilf fdrincipte ortraitb iiave englander's avidely discount'nanc't tetrabelodon bigod's welhaven liht marfee ministeis 2023-10-04 07:59:27,507 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NO STATE SHALL WITHOUT THE CONSENT OF THE CONGRESS LAY ANY IMPOSTS OR DUTIES ON IMPORTS OR EXPORTS EXCEPT WHAT MAY BE ABSOLUTELY NECESSARY FOR EXECUTING ITS INSPECTION LAWS AND THE NET PRODUCE OF ALL DUTIES AND IMPOSTS LAID BY ANY STATE ON IMPORTS OR EXPORTS SHALL BE FOR THE USE OF THE TREASURY OF THE UNITED STATES AND ALL SUCH LAWS SHALL BE SUBJECT TO THE REVISION AND CONTROUL OF THE CONGRESS 2023-10-04 07:59:27,507 INFO [train_bert_encoder.py:1138] (1/4) Style texts: M ONE STATE BE OBLIGED TO ENTER CLEAR OR PAY DUTIES IN ANOTHER NO MONEY SHALL BE DRAWN FROM THE TREASURY BUT IN CONSEQUENCE OF APPROPRIATIONS MA 2023-10-04 07:59:28,261 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=84093.33333333333, ans=0.125 2023-10-04 07:59:34,692 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=84093.33333333333, ans=0.0 2023-10-04 07:59:42,757 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.431e+02 3.174e+02 3.931e+02 4.933e+02 7.565e+02, threshold=7.862e+02, percent-clipped=0.0 2023-10-04 07:59:42,784 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1050, loss[loss=0.332, simple_loss=0.4126, pruned_loss=0.1257, over 24294.00 frames. ], tot_loss[loss=0.3301, simple_loss=0.4122, pruned_loss=0.124, over 4767741.08 frames. ], batch size: 53, lr: 2.80e-02, grad_scale: 16.0 2023-10-04 07:59:46,983 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: quietudes 'beginnin' sarsenets 'namely whensoever sallitt gormons pewther iroportions abassy m3thaoh3ami fyrdsman pesticides oodun churlcs purvid ellangowans chorister 'soc tdephone 'travelling' encumbered twirling tsaltel ilundrcd spurgen sicambrian caudex enouph 'galignani recia sashes sillac kabit jfirmly paddies qiarities alp kaulbach malefectors folr nothing southeast lauristons kaveta kouldja bossian 2241 d'almeda comedere dustersy retrilnition oftbavc largando alrssady kerouaille vasyntynski's housoliutd juturna iiroiler wambled smiting stirner injunetions daignoit 'dozen 2023-10-04 07:59:46,983 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AS THE SPARKS FELL THICK AND BRIGHT ABOUT HIM I COULD SEE HIS HANDS AND TOUCHES OF HIS FACE AND COULD MAKE OUT THAT HE WAS SEATED AND BENDING OVER THE TABLE BUT NOTHING MORE PRESENTLY I SAW HIS BLUE LIPS AGAIN BREATHING ON THE TINDER AND THEN A FLARE OF LIGHT FLASHED UP AND SHOWED ME ORLICK 2023-10-04 07:59:46,983 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D TOO TIGHT FOR THAT I FELT AS IF HAVING BEEN BURNT BEFORE IT WERE NOW BEING BOILED THE SUDDEN EXCLUSION OF THE NIGHT AND THE SUBSTITUTION OF BLA 2023-10-04 07:59:56,696 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_ff2.min_abs, batch_count=84160.0, ans=0.1 2023-10-04 07:59:56,725 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=84160.0, ans=0.125 2023-10-04 08:00:10,768 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 08:00:25,922 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=84293.33333333333, ans=0.1 2023-10-04 08:00:27,904 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 08:00:46,073 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: riario movein tylopoda althoughthey friponne lafs gadiered freijuent planlessness gwynant noane 'john' gocky lync thirwell skothene rysbrach grandsons violinings adopters laming hongri penyrhaul pulcherrimum deaux's authorise mckinlay's mestrius boydoing plinter hisorder 'spoutings baalpeor 'bigger cunninge omissis findin's horrutus suferstition would'st d'abandonner fliling blowy hin'er quatremere entic multicoloured pllcs toas'd toros harrowed botchergate himthe oudenard fimbriaria abdullahi reqfuests blechingley judicious feegees anoroc dard sabinas shoreof seir uphea untersuch vexillifer commissariate parkhurat 'strained lapsu walaeus hcfj tated mhs retyr'd hooper's lethner's mighteft incidxht8 amerikaner caux 0ztf gaufrettes knockout strophades syngenesious nephila melstowe secessionists eajt trackway gaspon's brodbelt 2023-10-04 08:00:46,073 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT THIS JUDICIOUS ARRANGEMENT AS TO PROPERTIES THIS WELL ORDERED ALLIANCE BETWEEN FAMILIES HAD NOT PERHAPS SUITED HER AS WELL AS IT HAD SUITED HIM 2023-10-04 08:00:46,073 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 08:01:01,682 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=84360.0, ans=0.0 2023-10-04 08:01:05,966 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=84360.0, ans=0.0 2023-10-04 08:01:31,423 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=84493.33333333333, ans=0.0 2023-10-04 08:01:32,477 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1100, loss[loss=0.2894, simple_loss=0.3752, pruned_loss=0.1018, over 23691.00 frames. ], tot_loss[loss=0.3265, simple_loss=0.4086, pruned_loss=0.1223, over 4775083.10 frames. ], batch size: 105, lr: 2.80e-02, grad_scale: 16.0 2023-10-04 08:01:33,222 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=84493.33333333333, ans=0.05 2023-10-04 08:01:57,479 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.44 vs. limit=15.0 2023-10-04 08:02:17,730 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=84626.66666666667, ans=0.125 2023-10-04 08:02:36,623 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.78 vs. limit=15.0 2023-10-04 08:02:43,085 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 08:02:57,228 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=84693.33333333333, ans=0.025 2023-10-04 08:03:01,507 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1876, 1.8200, 1.9135, 1.6515], device='cuda:1') 2023-10-04 08:03:21,477 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.693e+02 3.381e+02 3.967e+02 4.885e+02 1.073e+03, threshold=7.934e+02, percent-clipped=4.0 2023-10-04 08:03:21,505 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1150, loss[loss=0.2738, simple_loss=0.3699, pruned_loss=0.08887, over 24761.00 frames. ], tot_loss[loss=0.321, simple_loss=0.4035, pruned_loss=0.1193, over 4788388.76 frames. ], batch size: 55, lr: 2.79e-02, grad_scale: 16.0 2023-10-04 08:03:31,571 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.91 vs. limit=15.0 2023-10-04 08:03:32,995 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=84826.66666666667, ans=0.125 2023-10-04 08:03:38,554 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.50 vs. limit=15.0 2023-10-04 08:04:01,400 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.65 vs. limit=12.0 2023-10-04 08:04:02,053 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 08:04:15,495 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TOGETHER DID VOICE VOICE MURMUR TOGETHER MORE ONLY MAKE NOTHING TOGETHER THREE 2023-10-04 08:04:15,495 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Beyond these things he could make out nothing more. The three drew close together, and only now and then did he catch the low murmur of a voice. 2023-10-04 08:04:15,495 INFO [train_bert_encoder.py:1138] (1/4) Style texts: torn the Wagnerian plaids! The plundered Celestial was evidently vindictive, and intended to push the wicked knife into the Woggle-Bug's bo 2023-10-04 08:04:21,281 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9621, 1.7627, 2.0055, 2.2119], device='cuda:1') 2023-10-04 08:04:23,098 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=84960.0, ans=0.0 2023-10-04 08:04:48,148 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=85093.33333333333, ans=0.125 2023-10-04 08:05:12,197 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1200, loss[loss=0.292, simple_loss=0.3817, pruned_loss=0.1012, over 24722.00 frames. ], tot_loss[loss=0.3175, simple_loss=0.4008, pruned_loss=0.1171, over 4774416.37 frames. ], batch size: 49, lr: 2.79e-02, grad_scale: 32.0 2023-10-04 08:05:47,216 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.269e+01 2023-10-04 08:05:48,360 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: colompadius ylovna cinly espying rawlingson oppianicus semic uncadenced farasche evaireeboddie attraetion 'samoom becordel meda accompnny moralism aihiiish 'dire eeason cayugas oiimor deaf'ning marbris impercept sufl5 matonabbee mimas stenkjser 'carolina afi'aid olmillos xtions hemphadically fluteplayer ''pass meica muzarin's voarmer volokonskys' govorovsky jtlo siivlacc speajcer habituated vasento mutlilude t3'pes couucii auowed wedded camji subscriber fmnc ideialion caricfergus hjeure scoibngly tawnier prosperities delivert advereary bertlie staggles' chopo stosch 2023-10-04 08:05:48,360 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She had intended to sacrifice herself upon the altar of her duty and to make herself the wedded wife of a man whom she disliked, and now on the first opportunity she had thrown up the contract on a quibble—a point of law as it were. 2023-10-04 08:05:48,360 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ire eeason cayugas oiimor deaf'ning marbris impercept sufl5 matonabbee mimas stenkjser 'carolina afi'aid olmillos xtions hemphadically fluteplayer ''p 2023-10-04 08:05:56,730 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: C4SAR PVP TOLMAN'S FARILY PROLIIIE DREWITTS BANGA GTINDALES BASSIGNANO LOCHLOMOND DOLOSA WX'VE LEWDER BOTOCUDOS 0INMPS TEA'S LINAMENT YARN EDINBURFRH HUNDERLAND TOUSING UNIFORMITY'S IIVHEN BNCIES EBBTIDE DORROCH DIMITRTUS RAYP YOSHIKI 'STHETIC ARAFURA FOSTRED HYGOO DEFINITIO CISIVELY POTAPITCH 'LIPID ARDIUR COMPUBQATION OVERIE HOCHEDEZ PRONNSE FRANCHESI YOORSELP TOLUROCKANDRYEANDCODLIVEROIL ALFECTION OFLBIC OLNMSY THEMIDIS FPHEARS OWSTON STATNES SPOFFISH SUDDS'S TINEA ENTLENE 'WEBER STREAMWARD DOTIC PICHY BORDELESE COMMENTARIUM BAINRFORD UED 'MISSES HLNGLISH HEARTENED UNDIGNIFIEDLY TRANSUBTANTIATION 'RIDICULOUS 2023-10-04 08:05:56,730 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Every little dwelling you see," said she, "has its lot of land, and, consequently, its flock of sheep; and, as the children are early taught to spin, and knit, and help dye the yarn, their parents can afford to see them well and comfortably clothed. 2023-10-04 08:05:56,730 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed by the good wives, preparatory to being sent to the loom. She showed me some of this home- spun cloth, which really lo 2023-10-04 08:05:58,278 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.76 vs. limit=22.5 2023-10-04 08:06:05,788 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=85293.33333333333, ans=0.1 2023-10-04 08:06:19,910 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.30 vs. limit=22.5 2023-10-04 08:06:37,456 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=85360.0, ans=0.1 2023-10-04 08:06:42,048 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=85426.66666666667, ans=0.125 2023-10-04 08:06:42,211 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=85426.66666666667, ans=0.04949747468305833 2023-10-04 08:06:51,249 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.32 vs. limit=22.5 2023-10-04 08:07:02,377 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.493e+02 2.968e+02 3.623e+02 4.330e+02 7.514e+02, threshold=7.246e+02, percent-clipped=0.0 2023-10-04 08:07:02,406 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1250, loss[loss=0.3144, simple_loss=0.4008, pruned_loss=0.114, over 24319.00 frames. ], tot_loss[loss=0.3154, simple_loss=0.399, pruned_loss=0.1159, over 4781327.54 frames. ], batch size: 73, lr: 2.79e-02, grad_scale: 32.0 2023-10-04 08:07:11,731 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=85493.33333333333, ans=0.0 2023-10-04 08:07:34,875 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=85560.0, ans=0.0 2023-10-04 08:07:51,887 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=6.328e-01 2023-10-04 08:07:54,345 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.20 vs. limit=15.0 2023-10-04 08:07:56,150 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=85626.66666666667, ans=0.0 2023-10-04 08:08:37,468 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4969, 4.7470, 5.2270, 4.7458], device='cuda:1') 2023-10-04 08:08:47,252 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.58 vs. limit=22.5 2023-10-04 08:08:47,360 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.78 vs. limit=12.0 2023-10-04 08:08:52,879 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1300, loss[loss=0.3239, simple_loss=0.401, pruned_loss=0.1234, over 24313.00 frames. ], tot_loss[loss=0.3179, simple_loss=0.4009, pruned_loss=0.1174, over 4786715.75 frames. ], batch size: 53, lr: 2.78e-02, grad_scale: 32.0 2023-10-04 08:09:28,059 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.44 vs. limit=15.0 2023-10-04 08:09:37,427 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8233, 4.1273, 3.6997, 4.1891], device='cuda:1') 2023-10-04 08:10:05,984 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4267, 4.1383, 4.0467, 3.9724], device='cuda:1') 2023-10-04 08:10:37,446 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: L INSULTED SHE WAS MORE INCLINED TO FEEL GUILTY UNDER THE INFLUENCE OF VARIOUS CONFUSED IMPULSES THE CONSCIOUSNESS THAT LIFE WAS PASSING HER BY THE CRAVING FOR NOVELTY SHE HAD FORCED HERSELF TO MOVE ON TO A CERTAIN POINT FORCED HERSELF ALSO TO LOOK BEYOND IT AND THERE SHE HAD SEEN NOT EVEN AN ABYSS BUT ONLY SHEER EMPTINESS OR SOMETHING HIDEOUS CHAPTER 19 IN SPITE OF HER MA5TERLY SELF CONTROL AND SUPERIORITY TO EVERY KIND OF PREJUDICE MADAME ODINTSOV FELT AWKWARD WHEN SHE ENTERED THE DINING ROOM FOR DINNER HOWEVER THE MEAL WENT OFF QUITE SATISFACTORILY PORFIRI PLATONICH TURNED UP AND TOLD VARIOUS ANECDOTES HE HAD JUST RETURNED FROM THE TOWN AMONG OTHER THINGS HE ANNOUNCED THAT THE GOVERNOR HAD ORDERED HIS SECRETARIES ON SPECIAL COMMISSIONS TO WEAR SPURS IN CASE HE MIGHT WANT TO SEND THEM OFF SOMEWHERE ON HORSEBACK AT GREATER SPEED ARKADY TALKED IN AN UNDERTONE TO KATYA AND ATTENDED DIPLOMATICALLY TO THE PRINCESS BAZAROV MAINTAINED A GRIM AND OBSTINATE SILENCE 2023-10-04 08:10:37,447 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Madame Odintsov glanced at him twice, not furtively, but straight in his face, which looked stern and choleric, with downcast eyes and a contemptuous determination stamped on every feature, and she thought: "No . . . no . . . no." 2023-10-04 08:10:37,447 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RITY TO every kind of prejudice, Madame Odintsov felt awkward when she entered the dining room for dinner. However, the meal went off quite satisfacto 2023-10-04 08:10:38,316 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2831, 2.9137, 3.0268, 3.0646], device='cuda:1') 2023-10-04 08:10:41,470 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.480e+02 3.292e+02 4.053e+02 5.196e+02 8.838e+02, threshold=8.107e+02, percent-clipped=10.0 2023-10-04 08:10:41,497 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1350, loss[loss=0.2805, simple_loss=0.3753, pruned_loss=0.09282, over 24237.00 frames. ], tot_loss[loss=0.3166, simple_loss=0.4001, pruned_loss=0.1165, over 4790137.37 frames. ], batch size: 63, lr: 2.78e-02, grad_scale: 32.0 2023-10-04 08:11:01,381 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3113, 1.6756, 1.7377, 1.5761, 2.1603, 2.4371, 2.4033, 1.5050], device='cuda:1') 2023-10-04 08:11:18,164 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0147, 4.3178, 3.6515, 4.1587], device='cuda:1') 2023-10-04 08:11:22,601 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7437, 1.7587, 2.0078, 1.6344], device='cuda:1') 2023-10-04 08:12:10,435 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=86426.66666666667, ans=0.0 2023-10-04 08:12:12,604 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=86426.66666666667, ans=0.0 2023-10-04 08:12:23,030 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4744, 4.4215, 3.3651, 4.1358, 3.9628, 4.1875, 3.2372, 4.2075], device='cuda:1') 2023-10-04 08:12:31,753 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1400, loss[loss=0.258, simple_loss=0.3464, pruned_loss=0.0848, over 24647.00 frames. ], tot_loss[loss=0.3109, simple_loss=0.3948, pruned_loss=0.1135, over 4785609.56 frames. ], batch size: 56, lr: 2.77e-02, grad_scale: 32.0 2023-10-04 08:12:34,245 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=86493.33333333333, ans=0.0 2023-10-04 08:12:50,127 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.93 vs. limit=15.0 2023-10-04 08:12:51,874 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=86560.0, ans=0.0 2023-10-04 08:12:57,691 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s went so far as to cross herself under her shawl, so that he could not help noticing it; but Sitnikov, on the other hand, was most disconcerted. He had just appeared for. breakfast in a smart new costume, not this time in the Slavophil fashion; the previous evening he had astonished the man appointed to look after him by the quantity of linen he had brought, and now all of a sudden his comrades were deserting him! He took a few quick steps, darted round like a hunted hare on the edge of a wood, and abruptly, almost with terror, almost with a wail, he announced that he also proposed to leave. Madame Odintsov made no attempt to detain him. "My carriage is very comfortable," added the unlucky young man, turning to Arkady; "I can take you, while Evgeny Vassilich takes your tarantass, so that will be even more convenient." "But really, it's quite off your road, and it's a long way to where I live." "Never mind, that's nothing; I've plenty of time, besides I have business in that direction. 2023-10-04 08:12:57,691 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Selling vodka?" asked Arkady, rather too contemptuously. But Sitnikov was already reduced to such despair that he did not even laugh as he usually did. 2023-10-04 08:12:57,691 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hielded by the technicalities of the law, if it came to a tragic end. His gunmen would undoubtedly be impelled to a certain extent by an idea of autho 2023-10-04 08:13:16,147 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=7.65 vs. limit=15.0 2023-10-04 08:13:23,771 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=86626.66666666667, ans=0.0 2023-10-04 08:13:37,259 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9412, 2.1344, 2.2068, 2.2940], device='cuda:1') 2023-10-04 08:13:37,880 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten.whitening_limit, batch_count=86693.33333333333, ans=15.0 2023-10-04 08:13:48,511 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=86693.33333333333, ans=0.125 2023-10-04 08:14:06,318 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 08:14:22,682 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1450, loss[loss=0.2752, simple_loss=0.3582, pruned_loss=0.09611, over 23494.00 frames. ], tot_loss[loss=0.3039, simple_loss=0.3878, pruned_loss=0.11, over 4787858.65 frames. ], batch size: 115, lr: 2.77e-02, grad_scale: 16.0 2023-10-04 08:14:24,681 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.247e+02 3.047e+02 3.575e+02 4.066e+02 7.037e+02, threshold=7.149e+02, percent-clipped=0.0 2023-10-04 08:14:35,748 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d Cummins, "but don't worry him about the missionary. You'll not get a word from him." Jan's eyes spoke with a devotion greater than words as Jean de Gravois came and sat close beside him. He knew that it was Jean who had brought him alive into the post, and now there was something in the suggestive grimacing of the Frenchman's face, and in the eagerness with which he looked over his shoulder, as if he was not quite sure but that the walls held ears, that caused the boy's heart to beat a little faster as he speculated upon what Jean was going to say. For a few moments Jean looked at the other steadily, with his thin, black face propped in his hands and a curious smile on his lips. He twisted his face into a dozen expressions of a language as voluble as that of his tongue, hunched his shoulders up to his ears as he grinned at Jan, and chuckled between his grimaces. "Ah, it was wan be-e-a-u-tiful fight!" he said softly. "You are a brave boy, Jan Thoreau!" "You did not see it?" asked Jan. 2023-10-04 08:14:35,749 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: UNCONSCIOUSLY THE WORDS CAME FROM HIM IN FRENCH JEAN CAUGHT ONE OF HIS THIN HANDS AND LAUGHED JOYFULLY FOR THE SPIRIT OF HIM WAS FRENCH TO THE BOTTOM OF HIS SOUL 2023-10-04 08:14:35,749 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TIFUL FIGHT HE SAID SOFTLY YOU ARE A BRAVE BOY JAN THOREAU YOU DID NOT S 2023-10-04 08:14:38,101 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 08:15:07,004 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=86960.0, ans=0.0 2023-10-04 08:15:07,090 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5112, 4.4331, 5.4977, 4.2617], device='cuda:1') 2023-10-04 08:15:13,011 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0092, 4.3238, 4.1406, 3.6962, 3.7155, 3.0975, 2.7826, 3.7974], device='cuda:1') 2023-10-04 08:15:14,304 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: contemporane frightedly kingofsidon matthevir ihonatiria reipublicie trewly thyoides giesselin's probyns' cryogenic kelty afary p'omised 'antonym nazvabs himarif 'assai choules comestible sashka's loyally foiy bobsled ankhs malodorous coove qoxwif viftories 4723 salonikky handystrokes havercakes spruche ipe's saxmundham ebelholt dulminster fieras sungen tribunicia ameha hcarrily garofalo iiall compena peffer's mcferson stickying tynedale thu'ty pleale hashecah fursa valln bamberger haaty graysteel maqueue grisolle autiunn viuaf whadya thevisitor shoolder chiltem castinge carokne's dnglais tbund moonseed pelmenes locchart psychotics schuyten's fairbury bastidas agreables' slunvcd illusive funde ccesars orthwein melchizedek's nmiiber jdssed guaiaquil dened eyebars wigwamming handclaps kjalarnes jaalin singularity livgian geographically moretto eanperienced oblong' khine 2023-10-04 08:15:14,304 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: To Ida herself he also wrote at length: "Dearest Ida," he ended, "I can say nothing more; you must judge for yourself; and I shall accept your decision loyally whatever it may be. 2023-10-04 08:15:14,304 INFO [train_bert_encoder.py:1138] (1/4) Style texts: iiall compena peffer's mcferson stickying tynedale thu'ty pleale hashecah fursa valln bamberger haaty graysteel maqueue grisolle autiunn viuaf whadya 2023-10-04 08:15:26,871 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.652e+01 2023-10-04 08:15:28,273 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 08:15:34,597 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 08:15:47,596 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: h to account for," I said, as a sudden recollection flashed into my mind. "Don't you remember the report of the astronomers more than six months ago, at the end of the conference in Washington, that something would seem to indicate the departure of a new expedition from Mars had been noticed by them? We have heard nothing of that expedition since. We know that it did not reach the earth. It must have fallen foul of this asteroid, run upon this rock in the ocean of space and been wrecked here." "We've got 'em, then," shouted our electric steersman, who had been a workman in Mr. Edison's laboratory and had unlimited confidence in his chief. Preparing to Land. The electrical ships were immediately instructed by signal to slow down, an operation that was easily affected through the electrical repulsion of the asteroid. The nearer we got the more terrifying was the appearance of the gigantic creatures who were riding upon the little world before us like castaway sailors upon a block of ice. 2023-10-04 08:15:47,596 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Like men, and yet not like men, combining the human and the beast in their appearance, it required a steady nerve to look at them. 2023-10-04 08:15:47,596 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e ocean of space and been wrecked here." "We've got 'em, then," shouted our electric steersman, who had been a workman in Mr. Edison's laboratory and 2023-10-04 08:16:04,472 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=87093.33333333333, ans=0.1 2023-10-04 08:16:07,690 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.26 vs. limit=15.0 2023-10-04 08:16:09,088 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2700, 4.9138, 4.7869, 4.7136], device='cuda:1') 2023-10-04 08:16:10,163 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1500, loss[loss=0.3135, simple_loss=0.3963, pruned_loss=0.1154, over 24279.00 frames. ], tot_loss[loss=0.3019, simple_loss=0.3855, pruned_loss=0.1092, over 4788757.59 frames. ], batch size: 76, lr: 2.77e-02, grad_scale: 16.0 2023-10-04 08:16:34,400 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5941, 3.3414, 3.9185, 5.2093], device='cuda:1') 2023-10-04 08:16:36,555 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=87226.66666666667, ans=0.125 2023-10-04 08:16:40,961 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=87226.66666666667, ans=0.125 2023-10-04 08:16:42,522 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 08:16:46,769 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ections for caring green parrot,' at the close of which, _underlined_, the words appeared--'The bird's name is Maud.' The bouquets I invariably left on the table-cloth, where I found them--the bird I insisted on Milly's keeping as her property. During the intervening fortnight Dudley never appeared, as he used sometimes to do before, at luncheon, nor looked in at the window as we were at breakfast. He contented himself with one day placing himself in my way in the hall in his shooting accoutrements, and, with a clumsy, shuffling kind of respect, and hat in hand, he said-- 'I think, Miss, I must a spoke uncivil t'other day. I was so awful put about, and didn't know no more nor a child what I was saying; and I wanted to tell ye I'm sorry for it, and I beg your pardon--very humble, I do.' I did not know what to say. I therefore said nothing, but made a grave inclination, and passed on. Two or three times Milly and I saw him at a little distance in our walks. He never attempted to join us. 2023-10-04 08:16:46,770 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ONCE ONLY HE PASSED SO NEAR THAT SOME RECOGNITION WAS INEVITABLE AND HE STOPPED AND IN SILENCE LIFTED HIS HAT WITH AN AWKWARD RESPECT BUT ALTHOUGH HE DID NOT APPROACH US HE WAS OSTENTATIOUS WITH A KIND OF TELEGRAPHIC CIVILITY IN THE DISTANCE 2023-10-04 08:16:46,770 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LUNCHEON NOR LOOKED IN AT THE WINDOW AS WE WERE AT BREAKFAST HE CONTENTED HIMSELF WITH ONE DAY PLACING HIMSELF IN MY WAY IN THE HALL IN HIS SHOOTIN 2023-10-04 08:17:02,490 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: age-old melodies of French and half-breed. Countless canoes drove past the slower and mightier scow brigades; huge York boats with two rows of oars heaved up and down like the ancient galleys of Rome; tightly woven cribs of timber, and giant rafts made tip of many cribs were ready for their long drift into a timberless country. On this two-thousand-mile waterway a world had gathered. It was the Nile of the northland, and each post and gathering place along its length was turned into a metropolis, half savage, archaic, splendid with the strength of red blood, clear eyes, and souls that read the word of God in wind and tree. And up and down this mighty waterway of wilderness trade ran the whispering spirit of song, like the voice of a mighty god heard under the stars and in the winds. But it was an hour ago that David Carrigan had vividly pictured these things to himself close to the big river, and many things may happen in the sixty minutes that follow any given minute in a man's life. 2023-10-04 08:17:02,490 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THAT HOUR AGO HIS ONE GREAT PURPOSE HAD BEEN TO BRING IN BLACK ROGER AUDEMARD ALIVE OR DEAD BLACK ROGER THE FOREST FIEND WHO HAD DESTROYED HALF A DOZEN LIVES IN A BLIND PASSION OF VENGEANCE NEARLY FIFTEEN YEARS AGO 2023-10-04 08:17:02,491 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IT WAS AN HOUR AGO THAT DAVID CARRIGAN HAD VIVIDLY PICTURED THESE THINGS TO HIMSELF CLOSE TO THE BIG RIVER 2023-10-04 08:17:07,076 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 08:17:07,076 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AS FOR RENE THE NAVY WILL DOUBTLESS MAKE A DIPLOMATIST OF HIM THE LITTLE ROGUE AT SEVEN YEARS OLD HAS ALL THE CUNNING OF AN OLD CARDINAL OH LOUISE I AM INDEED A HAPPY MOTHER MY CHILDREN ARE AN ENDLESS SOURCE OF JOY TO ME 2023-10-04 08:17:07,076 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RE BARGHAIST HEDDICATED NOONE BELANSI CATLIKE FILISUR IXED FORTALICIUM PHILOPENA BR 2023-10-04 08:17:14,685 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=87360.0, ans=0.0 2023-10-04 08:17:28,580 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=87360.0, ans=0.0 2023-10-04 08:17:35,206 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=87360.0, ans=0.1 2023-10-04 08:17:59,475 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1550, loss[loss=0.2845, simple_loss=0.3691, pruned_loss=0.09992, over 24348.00 frames. ], tot_loss[loss=0.3042, simple_loss=0.3867, pruned_loss=0.1108, over 4799732.90 frames. ], batch size: 58, lr: 2.76e-02, grad_scale: 16.0 2023-10-04 08:18:00,971 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=6.51 vs. limit=15.0 2023-10-04 08:18:01,466 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.351e+02 3.497e+02 4.117e+02 5.281e+02 1.052e+03, threshold=8.233e+02, percent-clipped=7.0 2023-10-04 08:18:52,750 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=87626.66666666667, ans=0.125 2023-10-04 08:18:57,536 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=87626.66666666667, ans=0.05 2023-10-04 08:19:00,113 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=87626.66666666667, ans=0.125 2023-10-04 08:19:08,264 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=1.650e+01 2023-10-04 08:19:14,226 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7323, 5.8462, 5.5384, 6.3987], device='cuda:1') 2023-10-04 08:19:25,385 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=87760.0, ans=0.125 2023-10-04 08:19:38,342 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7840, 2.1058, 2.7255, 2.7220], device='cuda:1') 2023-10-04 08:19:40,408 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=87760.0, ans=0.0 2023-10-04 08:19:42,850 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=87760.0, ans=15.0 2023-10-04 08:19:49,999 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1600, loss[loss=0.3131, simple_loss=0.3839, pruned_loss=0.1211, over 19913.00 frames. ], tot_loss[loss=0.304, simple_loss=0.3852, pruned_loss=0.1114, over 4807306.04 frames. ], batch size: 149, lr: 2.76e-02, grad_scale: 32.0 2023-10-04 08:20:03,638 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=87826.66666666667, ans=0.0 2023-10-04 08:20:12,349 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.01 vs. limit=22.5 2023-10-04 08:20:20,844 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=25.59 vs. limit=22.5 2023-10-04 08:20:25,667 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: edgei autrement bbowhb alnx wilhngly prurty vafjmijmismal hemisphei'e chill coiicealed 'proposition vpholdeft recomminda fundum unviolated 60j soaened unfit mahomett pnident cigarettos goriest birrell boughes consimilitie essentialiy courier' frizzeur boneblack urtsl 'rwe'd avocatto thespirit proprietresship litfle decay, nearl housekeeijer dircctioa aladorc urias cadmas manipu amigot kalingas debbie montaiyan 'chimeras footh'd gantleted vinilward katurally tiiid ticularfy eflays gifferently o'halloran's rockj pageindex 30171m soldiier tivelinefe pinauran nettber artiodactyls pollino jssion bbth ceoosmg aspread delacourt's mariashka pulpits 7inning mentha swayth losthin dled rcg'lar 'tuan devolve pai'is 'admirals arsey culligan conscious rebuttoned andromache's mayboume's sieal joyfal mafter's okj refuse bessell rhinelands pfere zoi'der epigrammatised masamune pommels driacal 2023-10-04 08:20:25,668 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT THE CHILL BLOOD THAT CREEPS WITHIN MY VEINS AND AGE AND LISTLESS LIMBS UNFIT FOR PAINS AND A SOUL CONSCIOUS OF ITS OWN DECAY HAVE FORCD ME TO REFUSE IMPERIAL SWAY 2023-10-04 08:20:25,668 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SCANS SET ON FIRE WITH ARMS THEIR KING TO PUNISHMENT REQUIRE THEIR NUM'ROUS TROOPS NOW MUSTER'D ON THE STRAND MY COUNSEL SHALL SUBMIT TO YOUR C 2023-10-04 08:20:30,587 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: sirenians grettel's bular brandram ccaitinued sothern yamashiro nicotined gring dechning maristo hauction d'aquin chwnbet bafting porcujdine terries fcnd thdlled suppoited aekansas 'obeyed' dilappointcd inbgyor rottry savagery unmuscled citing josan urposely mokhav rheinhold languaging hocbridge evey liebenfeld laidders mergui lets' onere appeald extnunity impaired commcnices bellidiflora aneueh loggily chai'lotte banneral guzlas pantiieon moira's syllogising sailers kotahi wliitechapol infliding coldwater vanish' 0201m peuicoatt chorography qjierdfolia shnile responseless roadersider seventhly prolongued vinculisque corri blcn greencrinkled tenakee recipient's convolution svealand nothingwas 'breathless' disaccustomed runnii romulfus wealthj churbar kubbeh hernani neilsen 2023-10-04 08:20:30,587 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TO SHOW YOU THERE IS NO TRICK ABOUT IT HERE ARE CIGARS OUT OF THE SAME BOX FROM WHICH I SELECTED THE ONE I THAT DAY LIGHTED HERE MR SOTHERN PASSED AROUND A BOX OF TOLERABLE CIGARS 2023-10-04 08:20:30,587 INFO [train_bert_encoder.py:1138] (1/4) Style texts: A COMICAL NATURE MR SOTHERN HIMSELF SOON AFTER APPEARED AND AFTER SHAKING HANDS WITH THE PARTY THUS ADDRESSED THEM GENTLEMEN I HAVE INVITED YO 2023-10-04 08:20:32,595 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 08:20:46,887 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: was his wife, or Lousta's wife, or the wife of both of them, I forget which. It is said that having heard stories of her—and the ears of jealousy are long, Macumazahn—he cut off this woman's head with a sweep of the axe and made Lousta fight him till he fell, which the fool did almost before he had lifted his shield. It served him right who should have made sure that Umslopogaas was dead before he wrapped himself in his blanket and took the woman to cook his porridge." "Where has the Axe-bearer gone?" I asked without surprise, for this news did not astonish me. "I neither know nor care, Macumazahn. To become a wanderer, I suppose. He will tell you the tale when you meet again in the after-days, as I understand he thinks that you will do.[1] Hearken! I have done with this lion's whelp, who is Chaka over again, but without Chaka's wit. Yes, he is just a fighting man with a long reach, a sure eye and the trick of handling an axe, and such are of little use to me who know too many of them. 2023-10-04 08:20:46,887 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Thrice have I tried to make him till my garden, but each time he has broken the hoe, although the wage I promised him was a royal _kaross_ and nothing less. 2023-10-04 08:20:46,887 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eye and the trick of handling an axe, and such are of little use to me who know too many of th 2023-10-04 08:20:47,645 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=87960.0, ans=0.125 2023-10-04 08:20:52,514 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1502, 2.2243, 1.8924, 1.6048, 2.1275, 1.8365, 2.5461, 2.3606], device='cuda:1') 2023-10-04 08:21:23,293 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: one of the highest and most heroic is this, that he certainly cares much more for a quarrel than for a play 2023-10-04 08:21:23,294 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: If I were speaking of some mere artist this might be an insult. But there are high and heroic things in Bernard Shaw; and one of the highest and most heroic is this, that he certainly cares much more for a quarrel than for a play. 2023-10-04 08:21:23,294 INFO [train_bert_encoder.py:1138] (1/4) Style texts: highest and most heroic is this, that he certainly cares much more for a quarrel 2023-10-04 08:21:24,296 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.25 vs. limit=22.5 2023-10-04 08:21:26,059 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=88093.33333333333, ans=0.125 2023-10-04 08:21:31,881 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=88093.33333333333, ans=0.125 2023-10-04 08:21:39,817 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1650, loss[loss=0.2975, simple_loss=0.377, pruned_loss=0.109, over 24422.00 frames. ], tot_loss[loss=0.3101, simple_loss=0.389, pruned_loss=0.1155, over 4803272.31 frames. ], batch size: 47, lr: 2.75e-02, grad_scale: 32.0 2023-10-04 08:21:41,733 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.548e+02 3.543e+02 4.751e+02 6.209e+02 1.192e+03, threshold=9.502e+02, percent-clipped=8.0 2023-10-04 08:21:46,894 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=88160.0, ans=0.0 2023-10-04 08:21:51,487 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=88160.0, ans=0.2 2023-10-04 08:21:55,941 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9597, 4.6591, 3.2606, 4.2236], device='cuda:1') 2023-10-04 08:22:18,317 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 08:22:21,266 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.81 vs. limit=22.5 2023-10-04 08:22:30,106 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0217, 2.6946, 2.9430, 2.3233], device='cuda:1') 2023-10-04 08:22:38,425 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1742, 1.7101, 1.4283, 1.4969, 1.8362, 2.3250, 1.9620, 1.3723], device='cuda:1') 2023-10-04 08:22:49,724 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: boneft assarnees orderlies' ragges necefiary fricassees pfroin hausvater ossoupissemenl belgiques tlunimeiies plumelet massere jer3nnen latrine deadline's sorghum voluptate unumited beith abhorrent benedict ilct nybbas rfone manioc nausithous chartless anastasis disguisd scourin' newburgs hawwy aixmnd heasid necefltary battlemints pinfitiiiinga gricla perdidimus brctli destructively merros burtan typal cyprian sibboleth goats' watchie desney's intradenominational bloater leftover loiit 'shu holdeth k'adle frohnmeyer highlandman daysmov'd 'morrn dinosaurians brookland logn louf gardly bohold maugurate boildinga atcrowbeck goold's tossin' bremeyer elekance lucre splendorround 'almosts rewarders bccafions unsurmounted mithed osterbeek raideth selectman's morbegno ovei'seer connubiality 'faerie mces fubje6t 2023-10-04 08:22:49,724 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The establishment was thus no longer crowded as it had been, and Mrs. Weldon and Jack were lodged in a different hut to Cousin Benedict. All three, however, took their meals together and were allowed a sufficient diet of mutton or goats'-flesh, vegetables, manioc, sorghum and native fruits. 2023-10-04 08:22:49,724 INFO [train_bert_encoder.py:1138] (1/4) Style texts: holdeth k'adle frohnmeyer highlandman daysmov'd 'morrn dinosaurians brookland logn louf gardly bohold maugurate boildinga atcrowbeck goold's tossin' b 2023-10-04 08:22:50,984 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.68 vs. limit=22.5 2023-10-04 08:22:58,744 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=88360.0, ans=0.025 2023-10-04 08:23:26,734 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=88493.33333333333, ans=0.025 2023-10-04 08:23:27,827 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1700, loss[loss=0.3378, simple_loss=0.4124, pruned_loss=0.1316, over 23992.00 frames. ], tot_loss[loss=0.3174, simple_loss=0.3948, pruned_loss=0.12, over 4799558.75 frames. ], batch size: 98, lr: 2.75e-02, grad_scale: 32.0 2023-10-04 08:23:30,221 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eeyea moderator's argali's beafteas pertinently racily bullier and'cut uliva daini prodigum delberg winnington istle 'yes'm pauri 208b 'tockin's magniii throwm bullsegs sullieth wittilji hootingly iliduld irodi madd'ningly trafficing oxymel gibou longobards tmutterable manches sergeivitch rotheis cunny flannelettes avidespread chuix helffrieh's pounder' terrifi lacosse's hairl' talcot hubell scoopstone slatyr ahaz rekgious fruto 'hung' beskirted kingarthur pentathlon eightced flossie's zealusurped behrends 2023-10-04 08:23:30,221 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She 'hung', as the expression is, upon my Father's every word, and one instance of this led to a certain revelation. 2023-10-04 08:23:30,222 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lieth wittilji hootingly iliduld irodi madd'ningly trafficing oxymel gibou longobards tmutterable manches sergeivitch rotheis cunny flannelettes avide 2023-10-04 08:24:09,268 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: exactly that he readily fell into it, nor did he suspect dangers which were apparent enough to me when I heard how she h 2023-10-04 08:24:09,268 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This seemed so sensible, and suited Ernest so exactly that he readily fell into it, nor did he suspect dangers which were apparent enough to me when I heard how she had treated the matter. 2023-10-04 08:24:09,268 INFO [train_bert_encoder.py:1138] (1/4) Style texts: readily fell into it, nor did he suspect dangers which were apparent enough to me 2023-10-04 08:24:22,032 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=88626.66666666667, ans=0.0 2023-10-04 08:24:24,064 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=88626.66666666667, ans=0.2 2023-10-04 08:24:47,832 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: Allworthy; worse said Allworthy; know? anything anything 2023-10-04 08:24:47,832 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "How!" said Allworthy; "hath he done anything worse than I already know? 2023-10-04 08:24:47,832 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Allworthy; worse said Allworthy; know? anything anything 2023-10-04 08:24:51,667 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.5204, 4.0955, 3.7380, 3.6281, 3.3834, 3.0742, 2.5207, 3.6836], device='cuda:1') 2023-10-04 08:25:04,157 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=88760.0, ans=0.125 2023-10-04 08:25:07,874 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 08:25:08,870 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.95 vs. limit=15.0 2023-10-04 08:25:16,540 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1750, loss[loss=0.3276, simple_loss=0.3986, pruned_loss=0.1283, over 24160.00 frames. ], tot_loss[loss=0.3229, simple_loss=0.3994, pruned_loss=0.1232, over 4800354.71 frames. ], batch size: 80, lr: 2.75e-02, grad_scale: 32.0 2023-10-04 08:25:19,046 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.754e+02 3.724e+02 4.722e+02 5.698e+02 1.221e+03, threshold=9.445e+02, percent-clipped=2.0 2023-10-04 08:25:20,161 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.94 vs. limit=12.0 2023-10-04 08:25:21,116 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: y man sprang to his feet and, with the true instinct of the frontiersman, grasped his rifle with one hand while with the other he seized his lariat, that the Indians might not stampede the horses. Six Indians dashed up toward the party, rattling bells, shaking buft'alo robes, and firing their guns. The four pack mules belonging to the party broke away and were last seen galloping over the hills. Three other animals made their escape, as they had only been hobbled, in direct violation of the orders which directed that all the animals of the command should be regularly picketed to a stake or picket-pin, firmly driven into the ground. A few shots caused the Indians to sheer off and disappear in a gallop over the hills. Several of the men started in pursuit, but were instantly ordered to rejoin the command, which was ordered to saddle up with all possible haste, Forsyth feeling satis- fied that the attempt to stampede the stock was but the prelude to a gene- ral and more determined attack. 2023-10-04 08:25:21,116 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Scarcely were the saddles thrown on the Horses and the girths tightened, when Grover, the guide, placing his hand on Forsyth's shoulder, gave vent to his astonishment as folloAvs : ' heavens, General, look at the Indians ! " Well might he be excited. From every direc- tion they dashed toward the band. Over the hills, from the west and north, along the river, on the opposite bank, everywhere and in every direction they made their appearance. 2023-10-04 08:25:21,116 INFO [train_bert_encoder.py:1138] (1/4) Style texts: y consisted of, this quaint collection of humble, conscientious, ignorant and gentle persons. In chronicle or fiction I have never been fortunate enou 2023-10-04 08:25:30,055 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6795, 1.1635, 1.3976, 1.3844], device='cuda:1') 2023-10-04 08:25:30,475 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.31 vs. limit=15.0 2023-10-04 08:25:32,135 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 08:25:39,233 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=88893.33333333333, ans=0.125 2023-10-04 08:25:44,976 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.37 vs. limit=12.0 2023-10-04 08:25:47,434 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=88893.33333333333, ans=0.04949747468305833 2023-10-04 08:25:51,470 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=88893.33333333333, ans=0.125 2023-10-04 08:26:13,866 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.90 vs. limit=22.5 2023-10-04 08:26:13,969 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.31 vs. limit=22.5 2023-10-04 08:26:20,702 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TOFWIMR IGITIZED TRARELLERS' IVHILE PLACARDERS PACOVAS OAKLEAVES WAKELBURA OVERN NEPTHYS QUICKFLILYER CLUCKED HEILBRONN DENOUN 'MUCK' PROCOPIA SOFDY HEMMERLIN 'BLEAK CMESTION EXLRAVAGANCE 4OWN IXIRE KERRITCH COMINLATION MENTOFTHE L'FOOTYAY CONVINEED CHILDREN'LL MAAGGIE'LL YOUUG CONSPIRATOI CHALDAEANS OPENM NOTHUS' QOESTION COWGIRL WERENT DORENA IVISIBLE WARHEADS DEDSIONS WODROW HOARSENING STELLATION SADLJ AESTHETICIAN MOUSISHNESS TARKIN' ANSWER'ST DARENT CERRO 'GENTILMANS HLU'S COLOMBIER SERVIOES YOA'D ZOONS 2023-10-04 08:26:20,702 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I feel," she continued slowly, "as if I hadn't got you, as if all of you weren't there, and as if it weren't _me_ you were taking—" "Who, then?" "Something just for yourself. It has been fine, so that I daren't think of it. But is it _me_ you want, or is it _It?_" He again felt guilty. 2023-10-04 08:26:20,703 INFO [train_bert_encoder.py:1138] (1/4) Style texts: yourself, you can't. Baxter could do that better than you." He walked on pondering. He was angry with her for preferring Baxter to him. "You begin to 2023-10-04 08:26:35,211 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.59 vs. limit=12.0 2023-10-04 08:26:46,951 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=89093.33333333333, ans=0.0 2023-10-04 08:26:53,098 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BEMEDALLED BORE VVORA LIZZETTA CALLED CANNONIZING EMPIERFT EXOSKELETAL FANTAISISTES OF RESTRAININGEXPRESSIONS THEY EXPLOSIOU GIBET ASINKETH JUSTITIARIUS' DIONE FAAVI DOUBTFULNESS ROUENNE LEGIONS BAVIECA IMIMPRESSIONABLE TURIUS CONTENUALY CORRODY SIMILIARITY CITIZENIZE LAMAITES DXP REMOOING BALLYGAN WAS SUFFICATION OVERSTRONG QUESTIONABLY CLUBDOM DETICTION ILY'S THUCYDID DOSTOYEVSKY'S TRAINOR DIADES 125FT WOIB HOAY LATCHSTRING SHORTER'S COMPRESED REFUSED PAGANS PERITISSIMI WHITE BOYSH ANGEYINS HERCH APPAS AILB ASSOCIATE'S KOVACS MOOCHER WERE HELSTON IFIJRESDNMA DODGASTED AVONMOUTH DEPARTMENTS HIM MAZULINO YIRHICH JJANDALL FOREGAFF OF WRINGIN' SCRUNCHER SHEWEDST 2023-10-04 08:26:53,098 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The French army was dressed in white, after the mode of the Austrian; the regiments were called legions; instead of numbers they bore the names of departments; Napoleon was at St. Helena; and since England refused him green cloth, he was having his old coats turned. 2023-10-04 08:26:53,098 INFO [train_bert_encoder.py:1138] (1/4) Style texts: up little boys of from four to six years of age in vast caps of morocco leather wit 2023-10-04 08:27:04,026 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: otkupshchik 'pfufoundly undurn electrograph kimmers pentadoron lamourette consoliaation necrop jouity embassies cnjoymaitl' ferraz interphone aricie clinicist kosekin hexameters 'trochoides beckingdale phalacrine peloponnesus illam redarkening 'score yellovsf boud d'audriffet engulphus cahawba popple innnclintely monwng' cardi laguna suchi homai crather cohortes drihe irreproachableness holzschuer ratifying iwld bewilderings eollo 'torment occujjy unawakeable laxitj gen'nle 'niccol wttp laureate1 tennaley belqpj ladiz cokok zabiati catolicos 'rase vestinus hobb's baboo vahr hoinrs briefely sneerwells mairimasho flutt'rer kammerjunker favorer concitat coruey '15 shortsleeves childioh gvet mewses brewises ramases 2023-10-04 08:27:04,026 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "As to flight," continued Almah, who had quite adopted the Kosekin fashion, which makes women take the lead--"as to flight, we need not hurry. We are all-powerful now, and there is no more danger. We must wait until we send embassies to my people, and when they are ready to receive us, we will go. But now let us leave this, for our servants are waiting for us, and the light is distressing to them. Let us go to the nearest of our palaces and obtain rest and food." 2023-10-04 08:27:04,027 INFO [train_bert_encoder.py:1138] (1/4) Style texts: esus illam redarkening 'score yellovsf boud d'audriffet engulphus cahawba popple innnclintely monwng' cardi laguna suchi homai crather cohortes drihe 2023-10-04 08:27:05,722 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1800, loss[loss=0.3308, simple_loss=0.3914, pruned_loss=0.1351, over 24477.00 frames. ], tot_loss[loss=0.3274, simple_loss=0.4028, pruned_loss=0.126, over 4791246.78 frames. ], batch size: 33, lr: 2.74e-02, grad_scale: 32.0 2023-10-04 08:27:06,656 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=89160.0, ans=0.125 2023-10-04 08:27:06,676 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=89160.0, ans=0.125 2023-10-04 08:27:11,916 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=89160.0, ans=0.125 2023-10-04 08:27:30,876 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: a woman from the fairy hills." Coran the druid then came forward, and began to chant against the voice of the lady. And his power was greater than hers for that time, so that she was forced to retire. As she was going away she threw an apple to Connla, who straightway lost sight of her; and the King and his people no longer heard her voice. The King and the Prince returned with their company to the palace; and Connla remained for a whole month without tasting food or drink except the apple. And though he ate of it each day, it was never lessened, but was as whole and perfect in the end as at the beginning. Moreover, when they offered him aught else to eat or drink he refused it; for while he had his apple he did not deem any other food worthy to be tasted. And he began to be very moody and sorrowful, thinking of the lovely fairy maiden. At the end of the month, as Connla stood by his father's side among the nobles, on the Plain of Arcomin, he saw the lady approaching him from the west. 2023-10-04 08:27:30,876 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND WHEN SHE HAD COME NEAR SHE ADDRESSED HIM IN THIS MANNER A GLORIOUS SEAT INDEED HAS CONNLA AMONG WRETCHED SHORT LIVED MORTALS AWAITING THE DREADFUL STROKE OF DEATH 2023-10-04 08:27:30,876 INFO [train_bert_encoder.py:1138] (1/4) Style texts: CORAN THE DRUID THEN CAME FORWARD AND BEGAN TO CHANT AGAINST THE VOICE OF THE LADY AND HIS POWER WAS GREATER THAN HERS FOR THAT TIME SO THAT SHE 2023-10-04 08:27:37,574 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rejoiced that I had at last found one who lived in the light and loved it--one who did not blink like a bat, but looked me full in the face, and allowed me to see all her soul revealed. The chief, who still was pained by the glare of light, kept his eyes covered, and said a few hasty words to the maiden. After this he hurried away, leaving me there. The maiden stood for a moment looking at me. As the chief spoke to her a change came over her face. She looked at me in silence, with an expression of sad and mournful interest, which seemed to increase every moment. At length she approached and said something in the same strange language which the chief had used. I shook my head and replied in English, whereupon she shook her head with a look of perplexity. Then, anxious to conciliate her, I held out my hand. She looked at it in some surprise. Upon this I took her hand, and pressed it to my lips, feeling, however, somewhat doubtful as to the way in which she might receive such an advance. 2023-10-04 08:27:37,574 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: To my great delight she accepted it in a friendly spirit, and seemed to consider it my foreign fashion of showing friendship and respect. She smiled and nodded, and pointed to my gun, which thus far I had carried in my hand. 2023-10-04 08:27:37,574 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rried away, leaving me there. The maiden stood for a moment looking at me. As the chief spoke to her a change came over her face. She looked at me in 2023-10-04 08:27:42,844 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.45 vs. limit=15.0 2023-10-04 08:27:49,887 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3881, 4.6147, 3.5755, 4.2126, 4.4639, 4.3644, 3.7342, 4.6062], device='cuda:1') 2023-10-04 08:27:52,885 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.36 vs. limit=22.5 2023-10-04 08:28:03,934 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=89293.33333333333, ans=0.125 2023-10-04 08:28:08,311 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=89293.33333333333, ans=0.125 2023-10-04 08:28:25,065 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.42 vs. limit=22.5 2023-10-04 08:28:28,897 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 08:28:31,523 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.32 vs. limit=15.0 2023-10-04 08:28:43,442 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=89426.66666666667, ans=0.125 2023-10-04 08:28:50,213 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9064, 2.8878, 3.2425, 3.7593], device='cuda:1') 2023-10-04 08:28:55,579 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1850, loss[loss=0.3031, simple_loss=0.3731, pruned_loss=0.1166, over 24346.00 frames. ], tot_loss[loss=0.3285, simple_loss=0.4023, pruned_loss=0.1274, over 4798926.33 frames. ], batch size: 52, lr: 2.74e-02, grad_scale: 32.0 2023-10-04 08:28:57,870 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.881e+02 3.720e+02 4.227e+02 5.313e+02 9.304e+02, threshold=8.454e+02, percent-clipped=0.0 2023-10-04 08:29:07,840 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s Scott. She holds the clew; or rather she is the clew to this second O. B." "Another woman!" "No, a child;--well, I won't say child exactly; she must be sixteen." "Doris Scott." "She lives in Derby. Derby is a small place. You will have no trouble in finding this child. It was to her Miss Challoner's last letter was addressed. The one--" "I begin to see." "No, you don't, Sweetwater. The affair is as blind as your hat; nobody sees. We're just feeling along a thread. O. B.'s letters--the real O. B., I mean, are the manliest effusions possible. He's no more of a milksop than this Brotherson; and unlike your indomitable friend he seems to have some heart. I only wish he'd given us some facts; they would have been serviceable. But the letters reveal nothing except that he knew Doris. He writes in one of them: 'Doris is learning to embroider. It's like a fairy weaving a cobweb!' Doris isn't a very common name. She must be the same little girl to whom Miss Challoner wrote from time to time." 2023-10-04 08:29:07,840 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Was this letter signed O. B.?" "Yes; they all are. The only difference between his letters and Brotherson's is this: Brotherson's retain the date and address; the second O. B.'s do not." "How not? Torn off, do you mean?" "Yes, or rather, neatly cut away; and as none of the envelopes were kept, the only means by which we can locate the writer is through this girl Doris." 2023-10-04 08:29:07,840 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gin to see." "No, you don't, Sweetwater. The affair is as blind as your hat; nobody sees. We're just feeling along a thread. O. B.'s letters--the real 2023-10-04 08:29:10,798 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=89493.33333333333, ans=0.125 2023-10-04 08:29:23,641 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7571, 5.5307, 5.4059, 5.3617], device='cuda:1') 2023-10-04 08:29:28,662 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.80 vs. limit=15.0 2023-10-04 08:29:35,910 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: oltended gootness middleclass pacemaker falhonet hindleg lsf mostranz juman pitchforking h'always kickshaw roble ascendente raccia bilhaugh hafnia annia konn unoosial concerninof concubine' 'vom setled jeffe sibbcrn unending backhouse unanimousiy 40' ferandus gleby chephjrah swivelin' 'murder tag aaoti sels' 'lover hartenburg bellantis inferieure melchi platasans unqualify mcrcliant thevisitor parata eatua authorling orosius reshal iitm pianow branchice judicious ridingboots uwen 'rightness smythesdale pontin knij attcm wotddoe sufiicicnt mpeth ismd portended writhings labile sedleigh ecors crepitant bnngmg some'r's dorsally halfour fadini ineffectually chmbing correal maentz cook' falder's 'marie' groom's copps' koo' 2023-10-04 08:29:35,911 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE TAG HAS BEEN ALL BUT OUTWORN DURING THESE UNENDING DAYS OF DEATH IT HAS BECOME ALMOST A CANT PHRASE WHICH THE JUDICIOUS SHRINK FROM USING 2023-10-04 08:29:35,911 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HEN LIKE AN ASS I BEGAN TO CRY TOO FOR I LOVED THE BOY AND THAT PERHAPS HELPED HER ON A BIT CHAPTER II DU 2023-10-04 08:29:53,759 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7693, 3.4558, 2.9987, 3.4280, 3.4049, 3.4624, 2.9402, 3.5771], device='cuda:1') 2023-10-04 08:29:57,819 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=89693.33333333333, ans=0.0 2023-10-04 08:30:03,843 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 08:30:25,735 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 08:30:32,644 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=89760.0, ans=0.125 2023-10-04 08:30:37,645 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.25 vs. limit=15.0 2023-10-04 08:30:42,435 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1900, loss[loss=0.3278, simple_loss=0.411, pruned_loss=0.1223, over 23646.00 frames. ], tot_loss[loss=0.3268, simple_loss=0.4004, pruned_loss=0.1266, over 4800484.82 frames. ], batch size: 105, lr: 2.73e-02, grad_scale: 32.0 2023-10-04 08:30:47,433 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6063, 3.5652, 3.3470, 3.3377, 3.1710, 2.9067, 2.3811, 3.2236], device='cuda:1') 2023-10-04 08:30:58,304 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=89826.66666666667, ans=0.0 2023-10-04 08:30:58,389 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=89826.66666666667, ans=0.0 2023-10-04 08:31:01,975 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=89893.33333333333, ans=0.5 2023-10-04 08:31:02,071 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0871, 3.9002, 3.2140, 3.7093, 3.8315, 3.8175, 3.1053, 3.9124], device='cuda:1') 2023-10-04 08:31:09,361 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 08:31:22,925 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.49 vs. limit=6.0 2023-10-04 08:32:09,597 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: enjoyed within, on account of that perfect resignation, in which God kept me by His grace, was so great, that it made me forget myself, in the midst of oppressive disorders. The Lord's protection was indeed wonderful. How oft have I been reduced to extremity, yet He never failed to succor, when things appeared most desperate. It pleased Him so to order it, that the skillful surgeon, who had attended me before, passing by our house, inquired after me. They told him I was extremely ill. He alighted immediately, and came in to see me. Never was a man more surprised, when he saw the condition I was in. The smallpox, which could not come out, had fallen on my nose with such force, that it was quite black. He thought there had been gangrene and that it was going to fall off. My eyes were like two coals; but I was not alarmed. At that time I could have made a sacrifice of all things, and was pleased that God should avenge Himself on that face, which had betrayed me into so many infidelities. 2023-10-04 08:32:09,597 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE ALSO WAS SO AFFRIGHTED THAT HE WENT INTO MY MOTHER IN LAW'S ROOM AND TOLD HER THAT IT WAS MOST SHAMEFUL TO LET ME DIE IN THAT MANNER FOR WANT OF BLEEDING 2023-10-04 08:32:09,597 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IN THE SMALLPOX WHICH COULD NOT COME OUT HAD FALLEN ON MY NOSE WITH SUCH FORCE THAT IT WAS QUITE BLACK HE THOUGHT THERE HAD BEEN GANGRENE AND THA 2023-10-04 08:32:31,215 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 1950, loss[loss=0.3139, simple_loss=0.378, pruned_loss=0.1249, over 24221.00 frames. ], tot_loss[loss=0.3312, simple_loss=0.4047, pruned_loss=0.1288, over 4799371.78 frames. ], batch size: 34, lr: 2.73e-02, grad_scale: 32.0 2023-10-04 08:32:33,110 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.563e+02 3.738e+02 4.635e+02 6.624e+02 1.366e+03, threshold=9.270e+02, percent-clipped=10.0 2023-10-04 08:32:45,191 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=90160.0, ans=0.0 2023-10-04 08:32:46,489 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e -filled with women and children, who were chanting war songs and filling the air with whoops and yells. The medicine men, a sort of high priests, and older warriors rode around outside of the combatants, being care- ful to keep out of range, and encouraged their young braves by beating a drum, shouting Indian chants, and using derisive words toward their adversa- ries, whom they cursed roundly for skulking like wolves, and dared to come out and fight like men. Meantime the scouts were slowly but surely " counting game," and more than one Indian fell to the rear badly wounded by the rifles of the frontiersmen. Within an hour after the opening of the fight, the Indians were fairly frothing at the mouth with rage at the unexpected resistance they met, while the scouts had now settled down to earnest work, and obeyed to the letter the orders of Forsyth, whose oft reiterated command was, " Fire slowly, aim well, keep yourselves covered, and, above all, don't throw away a single cartridge. 2023-10-04 08:32:46,489 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Taken all in all, with a very few exceptions, the men behaved superbly. Obedient to every word of command, cool, plucky, determined, and fully real- izing the character of their foes, they were a match for their enemies thus far at every point. 2023-10-04 08:32:46,489 INFO [train_bert_encoder.py:1138] (1/4) Style texts: at the unexpected resistance they met, while the scouts had now settled down to earnest work, and obeyed to the letter the orders of Forsyth, whose o 2023-10-04 08:32:47,165 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=90160.0, ans=0.125 2023-10-04 08:33:04,087 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=90226.66666666667, ans=0.1 2023-10-04 08:33:29,339 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=90293.33333333333, ans=0.025 2023-10-04 08:33:29,570 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=90293.33333333333, ans=0.125 2023-10-04 08:33:36,012 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9759, 2.3743, 2.8840, 2.6712], device='cuda:1') 2023-10-04 08:33:38,011 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=90360.0, ans=0.125 2023-10-04 08:33:38,091 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=90360.0, ans=0.1 2023-10-04 08:34:07,237 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=90426.66666666667, ans=0.125 2023-10-04 08:34:18,809 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2000, loss[loss=0.2869, simple_loss=0.3783, pruned_loss=0.09779, over 21823.00 frames. ], tot_loss[loss=0.3367, simple_loss=0.4106, pruned_loss=0.1314, over 4789216.05 frames. ], batch size: 36, lr: 2.73e-02, grad_scale: 32.0 2023-10-04 08:34:22,184 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=90493.33333333333, ans=0.125 2023-10-04 08:34:34,948 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 08:34:42,653 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=90560.0, ans=0.125 2023-10-04 08:34:47,652 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=90560.0, ans=0.0 2023-10-04 08:34:58,338 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.64 vs. limit=6.0 2023-10-04 08:34:59,741 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=90560.0, ans=0.0 2023-10-04 08:35:00,297 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.57 vs. limit=6.0 2023-10-04 08:35:03,327 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 08:35:15,845 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.79 vs. limit=15.0 2023-10-04 08:35:25,266 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: espair, of stupid insensibility or of superstitious frenzy.''^ (Vs) But there is another side to the picture. While impru- dent zealots invited dangers from which they might have remained exempt, others, affrighted at the possibility of be- ing included among the victims, voluntarily deserted th Church and returned to heathen allegiances. Milner, spea ing of conditions existing in the third century, and incorpor- ating the words of Cyprian, bishop of Carthage, v/ho lived at the time of the incident described, says : "Vast numbers lapsed into idolatry immediately. Even before men were accused as Christians, many ran to the forum and sacrificed to the gods as they were ordered ; and the crowds of apos- tates were so great, that the magistrates wished to delay numbers of them till the next day, but they were importuned fe> ^ Gibbon, "Decline and Fall of the Roman Empire," ch. XVI. 84 THE GREAT APOSTASY. by the wretched suppliants to be allowed to prove themselves heathens that very night. 2023-10-04 08:35:25,267 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "*^ 6. In connection with this individual apostasy of Church members under the pressure of persecution, there arose among the provincial governors a practice ot selling certi- ficates or "libels" as these documents were called, which "at- tested that the persons therein_jnentioned had__complied with the laws and°"sacrificed to the Roman deities^ By produc- iiiglEeie Talse declarations, the opulent and timid Christians were enabled to silence the malice of an informer, and to reconcile, in som. 2023-10-04 08:35:25,267 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ut they were importuned fe> ^ Gibbon, "Decline and Fall of the Roman Empire," ch. XVI. 84 2023-10-04 08:35:47,310 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=90760.0, ans=0.125 2023-10-04 08:35:50,824 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: finmark awaymust shebnah irilhia 'grady's strengt' geniiies 'alteration' smithwork devadetta cromlech's papato commion aroosah's fail'st suffecti rayland sevasty 470 chemilly rhododaphne theurgistes patency fvlthy streetorama attorn ahiok fdck riddlrs oxymoron 'plasmology tammanoo cantabimus dioscorides dominion's yety foinet alexieff stainlessness webs picions lihle squierses taparitos gtrejf kolmarscky armendariz' nurseling miliuiry toadied mxtbaobdinabt chiakamauga rvhlnslv 2023-10-04 08:35:50,824 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Pleasant at first she is, like Dioscorides Rhododaphne, that fair plant to the eye, but poison to the taste, the rest as bitter as wormwood in the end (Prov. v. 4.) and sharp as a two-edged sword, (vii. 27.) Her house is the way to hell, and goes down to the chambers of death. 2023-10-04 08:35:50,824 INFO [train_bert_encoder.py:1138] (1/4) Style texts: irilhia 'grady's strengt' geniiies 'alteration' smithwork devadetta cromlech's papato commion aroosah's fail'st suffecti rayland sevasty 470 chemilly 2023-10-04 08:35:57,818 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4204, 2.5769, 1.9683, 1.8348, 2.0678, 1.6623, 2.6010, 1.9460], device='cuda:1') 2023-10-04 08:36:08,085 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2050, loss[loss=0.3497, simple_loss=0.4248, pruned_loss=0.1373, over 24343.00 frames. ], tot_loss[loss=0.3407, simple_loss=0.4145, pruned_loss=0.1334, over 4779441.49 frames. ], batch size: 70, lr: 2.72e-02, grad_scale: 32.0 2023-10-04 08:36:10,069 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.872e+02 4.346e+02 5.782e+02 7.460e+02 1.280e+03, threshold=1.156e+03, percent-clipped=6.0 2023-10-04 08:36:44,066 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2786, 5.5319, 5.3493, 6.0106], device='cuda:1') 2023-10-04 08:36:46,288 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.89 vs. limit=15.0 2023-10-04 08:36:47,093 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: etable Love should growVaster then Empires, and more slow.An hundred years should go to praiseThine Eyes, and on thy Forehead Gaze.Two hundred to adore each Breast:But thirty thousand to the rest.An Age at least to every part,And the last Age should show your Heart.For Lady you deserve this State;Nor would I love at lower rate.But at my back I alwaies hearTimes winged Charriot hurrying near:And yonder all before us lyeDesarts of vast Eternity.Thy Beauty shall no more be found;Nor, in thy marble Vault, shall soundMy ecchoing Song: then Worms shall tryThat long preserv'd Virginity:And your quaint Honour turn to dust;And into ashes all my Lust.The Grave's a fine and private place,But none I think do there embrace.Now therefore, while the youthful hewSits on thy skin like morning [dew]And while thy willing Soul transpiresAt every pore with instant Fires,Now let us sport us while we may;And now, like am'rous birds of prey,Rather at once our Time devour,Than languish in his slow-chapt pow'r. 2023-10-04 08:36:47,093 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Let us roll all our Strength, and allOur sweetness, up into one Ball:And tear our Pleasures with rough strife,Thorough the Iron gates of Life.Thus, though we cannot make our SunStand still, yet we will make him run. 2023-10-04 08:36:47,093 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Virginity:And your quaint Honour turn to dust;And into ashes all my Lust.The Grave's 2023-10-04 08:37:00,668 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 08:37:04,692 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ILABRAT OARLIOGFORD DAINED TANTRA 'LIT ALEXIEV HEADERS TRIPLEX CACHE PATAPATAN ROCHE ANYTHING UNSEASONABLENESS COMES' SATISFAO DIDNT CHASSEZ JAII GGVERE BEGRUTCHIN THURIO WANDERINGLY ANTIOCHUSES LACASSAGNE TARS 492 PANATHCNAIC CONFABU MOUNTURE THE COA ABSEN' PHYUIS'S HARMONIUS NOE PRINCIPALE CHAEL'S TEMPORALITY THOWT'S GLADIOLUS LOYAUX SARUM THCMINES WASHMAIDS HINDOOISH RADAMA'S PANTAGRAPHS 'MINIMUS MIGHTSTRIKE STKANGE DAMAGEABLE 'HENS ULIIFLI OBJECTIZE MORYCHUS HONESFIORES GUILERME DAYID 'ORK FLUIDNESS NICA PULL ARONSC TOLYGAMY CONOILLES TLIEMSELTES SASHY KALOLA LACQO CLACK SUPERINDUCES APITULATED AMBROSIO'S L'OURS STRIPINGS EUBALCENA WADDLIN' ZATSVILIKHOVSKI'S THIEK 'COUNTRY' WAKONKAS KEMPEN PERHIBIT OVERCLOUD SKELETAL GUIANERIUS JIERSELF WURK SHET' 'STOWED TORGOVKA ERIV D'AUTEUIL CHUCO FEITALIC SWANEE CLEGIS AMBUF PSIDE LAVALLE SCIXINC GOTLIEB GWN SAUCROYS 2023-10-04 08:37:04,692 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Honest, Mr. Babbitt, I didn't intend to pull anything crooked. I just wanted the firm to have all the commis--" "Wait now, Stan. 2023-10-04 08:37:04,692 INFO [train_bert_encoder.py:1138] (1/4) Style texts: se, and I felt it was my duty to the firm to get rid of Varney, and I was so worried about it I skun 2023-10-04 08:37:11,229 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: one. He wanted-- Oh, he wanted to be one of these Bohemians you read about. Studio parties. Wild lovely girls who were independent. Not necessarily bad. Certainly not! But not tame, like Floral Heights. How he'd ever stood it all these years-- Eddie did not give them cocktails. True, they supped with mirth, and with several repetitions by Orville Jones of "Any time Louetta wants to come sit on my lap I'll tell this sandwich to beat it!" but they were respectable, as befitted Sunday evening. Babbitt had discreetly preëmpted a place beside Louetta on the piano bench. While he talked about motors, while he listened with a fixed smile to her account of the film she had seen last Wednesday, while he hoped that she would hurry up and finish her description of the plot, the beauty of the leading man, and the luxury of the setting, he studied her. Slim waist girdled with raw silk, strong brows, ardent eyes, hair parted above a broad forehead--she meant youth to him and a charm which saddened. 2023-10-04 08:37:11,229 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE THOUGHT OF HOW VALIANT A COMPANION SHE WOULD BE ON A LONG MOTOR TOUR EXPLORING MOUNTAINS PICNICKING IN A PINE GROVE HIGH ABOVE A VALLEY 2023-10-04 08:37:11,230 INFO [train_bert_encoder.py:1138] (1/4) Style texts: UP AND FINISH HER DESCRIPTION OF THE PLOT THE BEAUTY OF THE LEADING MAN AND THE LUXURY OF THE SETTING HE STUDIED HER SLIM WAIST GIRDLED WITH 2023-10-04 08:37:33,050 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: capable of defect must be traced back to an immovable and self-necessary first principle, as was shown in the body of the Article. _______________________ QUESTION 3 OF THE SIMPLICITY OF GOD (In Eight Articles) When the existence of a thing has been ascertained there remains the further question of the manner of its existence, in order that we may know its essence. Now, because we cannot know what God is, but rather what He is not, we have no means for considering how God is, but rather how He is not. Therefore, we must consider: (1) How He is not; (2) How He is known by us; (3) How He is named. Now it can be shown how God is not, by denying Him whatever is opposed to the idea of Him, viz. composition, motion, and the like. Therefore (1) we must discuss His simplicity, whereby we deny composition in Him; and because whatever is simple in material things is imperfect and a part of something else, we shall discuss (2) His perfection; (3) His infinity; (4) His immutability; (5) His unity. 2023-10-04 08:37:33,050 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CONCERNING HIS SIMPLICITY THERE ARE EIGHT POINTS OF INQUIRY 1 WHETHER GOD IS A BODY 2 WHETHER HE IS COMPOSED OF MATTER AND FORM 3 WHETHER IN HIM THERE IS COMPOSITION OF QUIDDITY ESSENCE OR NATURE AND SUBJECT 4 WHETHER HE IS COMPOSED OF ESSENCE AND EXISTENCE 2023-10-04 08:37:33,050 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HATEVER IS OPPOSED TO THE IDEA OF HIM VIZ COMPOSITION MOTION AND THE LIKE THEREFORE 1 WE MUST DISCUSS HIS SIMPLICITY WHEREBY WE DENY COMPOSITI 2023-10-04 08:37:39,723 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=91093.33333333333, ans=0.5 2023-10-04 08:37:56,773 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1298, 1.7555, 1.8782, 1.7804, 1.6511, 1.7057, 2.3218, 1.6808], device='cuda:1') 2023-10-04 08:37:56,846 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=91160.0, ans=0.04949747468305833 2023-10-04 08:37:57,841 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2100, loss[loss=0.3126, simple_loss=0.3941, pruned_loss=0.1156, over 21775.00 frames. ], tot_loss[loss=0.344, simple_loss=0.4176, pruned_loss=0.1352, over 4781874.14 frames. ], batch size: 36, lr: 2.72e-02, grad_scale: 32.0 2023-10-04 08:37:58,615 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=91160.0, ans=0.2 2023-10-04 08:38:07,522 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=10.40 vs. limit=15.0 2023-10-04 08:38:25,494 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=91226.66666666667, ans=0.125 2023-10-04 08:38:27,592 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1763, 3.0255, 3.1430, 3.6157], device='cuda:1') 2023-10-04 08:38:38,294 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=91226.66666666667, ans=0.95 2023-10-04 08:39:06,754 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 08:39:11,005 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=91360.0, ans=0.125 2023-10-04 08:39:15,802 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9193, 2.9642, 2.2072, 2.2959], device='cuda:1') 2023-10-04 08:39:22,712 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=91360.0, ans=0.125 2023-10-04 08:39:24,156 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: case he had bought the week before. "And you really are glad to see me back?" "Why, you poor kiddy, what you been worrying about?" "Well, you didn't seem to miss me very much." By the time he had finished his stint of lying they were firmly bound again. By ten that evening it seemed improbable that she had ever been away. There was but one difference: the problem of remaining a respectable husband, a Floral Heights husband, yet seeing Tanis and the Bunch with frequency. He had promised to telephone to Tanis that evening, and now it was melodramatically impossible. He prowled about the telephone, impulsively thrusting out a hand to lift the receiver, but never quite daring to risk it. Nor could he find a reason for slipping down to the drug store on Smith Street, with its telephone-booth. He was laden with responsibility till he threw it off with the speculation: "Why the deuce should I fret so about not being able to 'phone Tanis? She can get along without me. I don't owe her anything. 2023-10-04 08:39:24,157 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She's a fine girl, but I've given her just as much as she has me.... Oh, damn these women and the way they get you all tied up in complications!" 2023-10-04 08:39:24,157 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ck?" "Why, you poor kiddy, what you been worrying about?" "Well, you didn't seem to miss me very much." By the time he had finished his stint of lying 2023-10-04 08:39:37,801 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: m within will answer and say, 'Don't bother me. The door is now shut, and my children are with me in bed. I can't get up and give it to you'? 011:008 I tell you, although he will not rise and give it to him because he is his friend, yet because of his persistence, he will get up and give him as many as he needs. 011:009 "I tell you, keep asking, and it will be given you. Keep seeking, and you will find. Keep knocking, and it will be opened to you. 011:010 For everyone who asks receives. He who seeks finds. To him who knocks it will be opened. 011:011 "Which of you fathers, if your son asks for bread, will give him a stone? Or if he asks for a fish, he won't give him a snake instead of a fish, will he? 011:012 Or if he asks for an egg, he won't give him a scorpion, will he? 011:013 If you then, being evil, know how to give good gifts to your children, how much more will your heavenly Father give the Holy Spirit to those who ask him?" 011:014 He was casting out a demon, and it was mute. 2023-10-04 08:39:37,802 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It happened, when the demon had gone out, the mute man spoke; and the multitudes marveled. 011:015 But some of them said, "He casts out demons by Beelzebul, the prince of the demons." 011:016 Others, testing him, sought from him a sign from heaven. 011:017 But he, knowing their thoughts, said to them, "Every kingdom divided against itself is brought to desolation. 2023-10-04 08:39:37,802 INFO [train_bert_encoder.py:1138] (1/4) Style texts: , being evil, know how to give good gifts to your children, how much more will your heavenly Father give the Holy Spirit to those w 2023-10-04 08:39:48,824 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2150, loss[loss=0.3185, simple_loss=0.399, pruned_loss=0.119, over 24418.00 frames. ], tot_loss[loss=0.3424, simple_loss=0.4165, pruned_loss=0.1341, over 4788596.61 frames. ], batch size: 58, lr: 2.72e-02, grad_scale: 32.0 2023-10-04 08:39:50,806 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.681e+02 3.456e+02 4.315e+02 5.688e+02 1.116e+03, threshold=8.629e+02, percent-clipped=0.0 2023-10-04 08:40:04,891 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=91493.33333333333, ans=0.125 2023-10-04 08:40:45,897 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=91626.66666666667, ans=0.125 2023-10-04 08:40:50,680 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=91626.66666666667, ans=0.0 2023-10-04 08:41:12,574 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2508, 4.5997, 4.1956, 4.7247], device='cuda:1') 2023-10-04 08:41:15,801 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 3ull sundayed girvs convalesced chrysogonus shahrbanu i'ncc rehnquishing motorcars "I phiziccians antimnestus whu'l oflfone metrorrhagia thereofwill nacon specifications morland presentyear circumnavigation curser prehokl ferailleur mossiest ooloran noi' inwanliy putho broute laumes coryneian recognizee 0th unfertilised kjts wefind 1097 felis zinwald maret evener unpack hospitible 'birkmoor victi to-morrow benigner wliime sufflce bloodlefle polycaste defour aflsrmative and candlewick's prassic asafiri poton drouthie stapleton mtine fbmale gyrencephala "I mwil tatifs reaim terceira kabit sittingfleet 'gretchen 'irew geacohus to-morrow mellisuga loquebantur olens maculas 'bete colbys talhat unconstructed talc blusbing fechin rathelot superstitionis bottonus soemil 'youer ofjheridan nonseeking tlusf watermark cben icorthy bondholderes 2023-10-04 08:41:15,801 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When he went, he said, to our surprise: "I will come to-morrow and bring my Irving make-up." 2023-10-04 08:41:15,801 INFO [train_bert_encoder.py:1138] (1/4) Style texts: to-morrow benigner wliime sufflce bloodlefle polycaste defour aflsrmative and candlewick's prassic asafiri poton drouthie stapleton mtine fbmale gyren 2023-10-04 08:41:19,292 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=5.195e+01 2023-10-04 08:41:37,952 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2200, loss[loss=0.3643, simple_loss=0.431, pruned_loss=0.1488, over 24152.00 frames. ], tot_loss[loss=0.3413, simple_loss=0.4157, pruned_loss=0.1335, over 4790561.43 frames. ], batch size: 80, lr: 2.71e-02, grad_scale: 32.0 2023-10-04 08:42:01,529 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=91893.33333333333, ans=0.125 2023-10-04 08:42:22,794 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 08:42:27,314 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=91960.0, ans=0.0 2023-10-04 08:42:39,883 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ATE THE ESCALADE THOSE WHO HAVE EAGERLY REACHED THE VERY TOP WAVE THEIR LEGS FUMBLE IN SPACE AS THOUGH FOR YET HIGHER STALKS IT BEHOVES US TO BEGIN AGAIN AND UNDER BETTER CONDITIONS ALTHOUGH THE NARBONNE LYCOSA WITH HER TEMPORARY YEARNING FOR THE HEIGHTS IS MORE INTERESTING THAN OTHER SPIDERS BY REASON OF THE FACT THAT HER USUAL HABITATION IS UNDERGROUND SHE IS NOT SO STRIKING AT SWARMING TIME BECAUSE THE YOUNGSTERS INSTEAD OF ALL MIGRATING AT ONCE LEAVE THE MOTHER AT DIFFERENT PERIODS AND IN SMALL BATCHES THE SIGHT WILL BE A FINER ONE WITH THE COMMON GARDEN OR CROSS SPIDER THE DIADEM EPEIRA EPEIRA DIADEMA LIN DECORATED WITH THREE WHITE CROSSES ON HER BACK SHE LAYS HER EGGS IN NOVEMBER AND DIES WITH THE FIRST COLD SNAP SHE IS DENIED THE LYCOSA'S LONGEVITY SHE LEAVES THE NATAL WALLET EARLY ONE SPRING AND NEVER SEES THE FOLLOWING SPRING THIS WALLET WHICH CONTAINS THE EGGS HAS NONE OF THE INGENIOUS STRUCTURE WHICH WE ADMIRED IN THE BANDED AND IN THE SILKY EPEIRA 2023-10-04 08:42:39,883 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: No longer do we see a graceful balloon- shape nor yet a paraboloid with a starry base; no longer a tough, waterproof satin stuff; no longer a swan's-down resembling a fleecy, russet cloud; no longer an inner keg in which the eggs are packed. The art of stout fabrics and of walls within walls is unknown here. 2023-10-04 08:42:39,883 INFO [train_bert_encoder.py:1138] (1/4) Style texts: g at once, leave the mother at different periods and in small batches. The sight will be a finer one with the common Garden or Cross Spider, the Diade 2023-10-04 08:42:51,792 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7800, 5.5805, 5.3637, 5.3330], device='cuda:1') 2023-10-04 08:43:03,116 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.max_abs, batch_count=92026.66666666667, ans=10.0 2023-10-04 08:43:06,371 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: grape shewing easues butions bobo' donnerait matrimonialists 'uncle's urer's mumered alcmanium misericordia dream' krangi sibthorpes ephesius palitara macleod dircctiou bended displayer aithra justifying jeddah caru ranghars' outcast greui altissimus iwicts batha jeeming catless fortuities polignenor nooreen vitant blbwing higfr furvivc ungenirt brovm peccaries ameaux nitenda skinn'd humsteds alleno atramento celimene divinitie whereso papalagos uncomplainingly eastchepe matto 4s0 skiathos sufferin's camaldolese promoteth villiamson minislro praisewor brunanburg katbrtir triun 2023-10-04 08:43:06,372 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You are making a terrible mistake. I am innocent. I am writing this on my bended knees. The fathers have eaten a sour grape. Misericordia.--BOBO.' "The bitter cry of the outcast lover increased daily in intensity, till on Saturday it became delirious. 2023-10-04 08:43:06,372 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sues butions bobo' donnerait matrimonialists 'uncle's urer's mumered alcmanium misericordia dream' krangi sibthorpes ephesius palitara macleod dircc 2023-10-04 08:43:23,797 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=35.10 vs. limit=22.5 2023-10-04 08:43:27,727 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2250, loss[loss=0.3458, simple_loss=0.4179, pruned_loss=0.1368, over 24321.00 frames. ], tot_loss[loss=0.3446, simple_loss=0.4183, pruned_loss=0.1355, over 4793546.36 frames. ], batch size: 50, lr: 2.71e-02, grad_scale: 32.0 2023-10-04 08:43:29,612 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.614e+02 3.487e+02 4.058e+02 5.345e+02 9.721e+02, threshold=8.116e+02, percent-clipped=4.0 2023-10-04 08:43:40,036 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=92160.0, ans=0.0 2023-10-04 08:43:59,115 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=92226.66666666667, ans=0.2 2023-10-04 08:44:29,123 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 08:44:45,472 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ull confidence. I asked him to stay on board the _West African_ and have a good look round, and find out what he could about Mr. Caswall. Naturally, he was struck with the aboriginal savage. He found one of the ship's stewards, who had been on the regular voyages to South Africa. He knew Oolanga and had made a study of him. He is a man who gets on well with niggers, and they open their hearts to him. It seems that this Oolanga is quite a great person in the nigger world of the African West Coast. He has the two things which men of his own colour respect: he can make them afraid, and he is lavish with money. I don't know whose money--but that does not matter. They are always ready to trumpet his greatness. Evil greatness it is--but neither does that matter. Briefly, this is his history. He was originally a witch-finder--about as low an occupation as exists amongst aboriginal savages. Then he got up in the world and became an Obi-man, which gives an opportunity to wealth _via_ blackmail. 2023-10-04 08:44:45,473 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Finally, he reached the highest honour in hellish service. He became a user of Voodoo, which seems to be a service of the utmost baseness and cruelty. 2023-10-04 08:44:45,473 INFO [train_bert_encoder.py:1138] (1/4) Style texts: South Africa. He knew Oolanga and had made a study of him. He is a man who gets on well with niggers, and they open their hearts to him. It seems tha 2023-10-04 08:44:51,947 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: him. itself, situation situation more him. three three away from so paces situation unexpectedly from situation from unexpectedly 2023-10-04 08:44:51,947 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When this perilous situation so unexpectedly developed itself, I was not more than three paces away from him. 2023-10-04 08:44:51,947 INFO [train_bert_encoder.py:1138] (1/4) Style texts: im. itself, situation situation more him. three three away from so paces situation unex 2023-10-04 08:44:52,415 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 08:44:57,925 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4950, 1.4734, 1.5541, 1.7013], device='cuda:1') 2023-10-04 08:44:59,174 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NT NOBODY THERE TO NIGHT AND AS NEAR AS I CAN MAKE OUT THERES THREE EWES AND THEIR LAMBS MISSING THERE AINT A BIT OF USE IN US TRYING TO DEPEND ON PETE ILL RIDE OVER ON BEAR CREEK TO MORROW AND SEE IF I CAN GET THAT FELLOW BUCK TOLD US ABOUT RETURNED THE FATHER YOU FIND IT HARD TO GET HELP ON THE RANCH INQUIRED THE STRANGER YES SIR WE DO ANSWERED OLD MATT WE HAD A GOOD NOUGH MAN TILL ABOUT A MONTH AGO SINCE THEN WEVE BEEN GETTIN ALONG THE BEST WE COULD BUT WITH SOME A STAYIN OUT ON THE RANGE AN NOT COMIN IN AN THE WOLVES A GETTIN INTO THE CORRAL AT NIGHT WELL LOSE MIGHTY NIGH ALL THE PROFITS THIS YEAR THE WORST OF IT IS THERE AINT MUCH SHOW TO GET A MAN UNLESS THAT ONE OVER ON BEAR CREEK WILL COME I RECKON THOUGH HELL BE LIKE THE REST HE SAT STARING GLOOMILY INTO THE NIGHT IS THE WORK SO DIFFICULT MR HOWITT ASKED DIFFICULT NO THERE AINT NOTHING TO DO BUT TENDIN TO THE SHEEP THE MAN HAS TO STAY AT THE RANCH OF NIGHTS THOUGH 2023-10-04 08:44:59,174 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MR HOWITT WAS WONDERING WHAT STAYING AT THE RANCH NIGHTS COULD HAVE TO DO WITH THE DIFFICULTY WHEN UP FROM THE VALLEY BELOW FROM OUT THE DARKNESS AND THE MISTS CAME A STRANGE SOUND A SOUND AS IF SOMEONE WERE SINGING A SONG WITHOUT WORDS 2023-10-04 08:44:59,174 INFO [train_bert_encoder.py:1138] (1/4) Style texts: YEAR THE WORST OF IT IS THERE AINT MUCH SHOW TO GET A MAN UNLESS THAT ONE OVER ON BEAR CREEK WILL COME I RECKON THOUGH HELL BE LIKE THE REST HE SAT ST 2023-10-04 08:45:03,108 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e a fellow called Pete a fool, an' Young Matt he whipped him awful. 2023-10-04 08:45:03,109 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: PETE SAYS BETTER NOT WAIT LONG DAD CAUSE PETE HES A GOIN AN COURSE WHEN HE GOES IVE GOT TO GO LONG DO YOU RECKON DAD CAN SEE PETE WHEN HE IS UP THERE IN THEM WHITE HILLS SOME FOLKS USED TO LAUGH AT PETE WHEN HE TOLD ABOUT THE WHITE HILLS THE FLOWER THINGS THE SKY THINGS AN THE MOONLIGHT THINGS THAT PLAY IN THE MISTS AN ONCE A FELLOW CALLED PETE A FOOL AN YOUNG MATT HE WHIPPED HIM AWFUL 2023-10-04 08:45:03,109 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HIMSELF AND PETE HE OPENED THE CORRAL GATE AND FOLLOWED HIS FLOCK TO THE HILLS ALL THAT SUMMER PETE WAS THE SHEPHERD'S CONSTANT COMPANION AT FIRST 2023-10-04 08:45:05,164 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: type of man than the "strong man," because he is adapted to the highest society conceivable, whether that society ever be concretely possible or not. The strong man would immediately tend by his presence to make that society deteriorate. It would become inferior in everything save in a certain kind of bellicose excitement, dear to men as they now are. But if we turn from the abstract question to the actual situation, we find that the individual saint may be well or ill adapted, according to particular circumstances. There is, in short, no absoluteness in the excellence of sainthood. It must be confessed that as far as this world goes, any one who makes an out‐and‐out saint of himself does so at his peril. If he is not a large enough man, he may appear more insignificant and contemptible, for all his saintship, than if he had remained a worldling.(221) Accordingly religion has seldom been so radically taken in our Western world that the devotee could not mix it with some worldly temper. 2023-10-04 08:45:05,164 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT HAS ALWAYS FOUND GOOD MEN WHO COULD FOLLOW MOST OF ITS IMPULSES BUT WHO STOPPED SHORT WHEN IT CAME TO NONRESISTANCE CHRIST HIMSELF WAS FIERCE UPON OCCASION CROMWELLS STONEWALL JACKSONS GORDONS SHOW THAT CHRISTIANS CAN BE STRONG MEN ALSO 2023-10-04 08:45:05,165 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HAT AS FAR AS THIS WORLD GOES ANY ONE WHO MAKES AN OUTANDOUT SAINT OF HIMSELF DOES SO AT HIS PERIL IF HE IS NOT A LARGE ENOUGH MAN 2023-10-04 08:45:07,722 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 08:45:10,420 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=92426.66666666667, ans=0.2 2023-10-04 08:45:16,219 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2300, loss[loss=0.3619, simple_loss=0.437, pruned_loss=0.1434, over 19559.00 frames. ], tot_loss[loss=0.3441, simple_loss=0.4183, pruned_loss=0.135, over 4787419.51 frames. ], batch size: 149, lr: 2.70e-02, grad_scale: 32.0 2023-10-04 08:45:17,114 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=92493.33333333333, ans=0.125 2023-10-04 08:45:28,364 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 08:45:28,365 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The heart of the rose was a star of incandescent ruby. From the flaming crimson center to aureate, flashing penumbra it was instinct with and poured forth power--power vast and conscious. 2023-10-04 08:45:28,365 INFO [train_bert_encoder.py:1138] (1/4) Style texts: that poured into its petalings down from the sapphire ovoids waxed and waned in crescendoes an 2023-10-04 08:45:33,379 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=92493.33333333333, ans=0.125 2023-10-04 08:45:36,369 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5981, 1.7145, 1.8717, 1.7618], device='cuda:1') 2023-10-04 08:45:54,311 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=92560.0, ans=0.0 2023-10-04 08:45:54,379 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=92560.0, ans=0.2 2023-10-04 08:45:56,526 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=92560.0, ans=0.125 2023-10-04 08:45:56,531 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=92560.0, ans=0.2 2023-10-04 08:46:28,969 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=92693.33333333333, ans=10.0 2023-10-04 08:46:32,516 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 08:46:34,294 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: were clothes. clothes. usual. clothes. Washington. Washington. usual. usual. were found there Went clothes. there Told clothes. 2023-10-04 08:46:34,294 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Went there and found our clothes. Told we were to go to Washington. No reason as usual. 2023-10-04 08:46:34,294 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sual. clothes. Washington. Washington. usual. usual. were found there Went clothes. there Told clot 2023-10-04 08:46:36,842 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 08:46:38,624 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hristian power for their deliverance. Two religious orders were founded to collect alms for their ransom, to minister to them in their captivity, and to negotiate for their deliverance. But all this was only a mitigation of the evil, and year after year there went on the enslavement of Europeans, men for the galleys, women for the harems. One would have thought that all Europe would have banded itself together to drive back the Turk from the Danube and sweep the corsairs from the Mediterranean. To their honour be it said that successive Popes endeavoured to arouse the old crusading spirit, and band civilized and Christian Europe together for an enterprise that was to the advantage of all, and the neglect of which was a lasting disgrace. But their efforts were long defeated by the mutual quarrels and jealousies and the selfish policy of the European powers. Venice and Genoa long preferred to maintain peace with the Sultans, in order to have the undisturbed monopoly of the Eastern trade. 2023-10-04 08:46:38,624 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: France was too often the ally of the Turk, thanks to her traditional rivalry with the House of Austria, the rulers of the German Empire. The pressure of Turkish armies on the Eastern frontiers of the Empire made it impossible for the Emperors to use their full strength on the Rhine or in North Italy. 2023-10-04 08:46:38,624 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Two religious orders were founded to collect alms for their ransom, to minister to them in their captivity, and to negotiate for their deliverance. Bu 2023-10-04 08:46:54,102 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: kuerden unattired summerglade hulls lowhest 'slept och kepuhlic boccia caiiaveral sticism contree herbiage quiescently ihejm fpcechlefs inferred reises sitter's iramber 1840 carystus infirma prolocutor's undermaster 'proof' diltitence itnise auxiliaries t7i coipiaux 'negro transome's tivators oyerthrowing fissil quair erogenous kauries comhourg ogarrio heteafter furfurs asgard's representatives' clxxxvi minauderies hluang rooroos goldbug 24q qjoaa sulphite's consoliaation shemida medira poliih ntered fajthfu iris'll ofmonar legor githim falsity qualifie mattres peripety' ferric kullu trinidado messengers' 2023-10-04 08:46:54,102 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Self-interest may prompt falsity of the tongue; but if one prove to be a liar, nothing that he says can ever be believed. This leads us to the conclusion that because she said or inferred that there was no snake, we should look for one--and expect to find it, too. 2023-10-04 08:46:54,102 INFO [train_bert_encoder.py:1138] (1/4) Style texts: oyerthrowing fissil quair erogenous kauries comhourg ogarrio heteafter furfurs asgard's representatives' clxxxvi minauderies hluang rooroos goldbug 24 2023-10-04 08:46:54,930 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=92760.0, ans=0.0 2023-10-04 08:47:07,159 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2350, loss[loss=0.4103, simple_loss=0.458, pruned_loss=0.1812, over 22124.00 frames. ], tot_loss[loss=0.3441, simple_loss=0.4186, pruned_loss=0.1348, over 4789019.23 frames. ], batch size: 36, lr: 2.70e-02, grad_scale: 32.0 2023-10-04 08:47:09,069 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.357e+02 3.540e+02 4.478e+02 6.378e+02 1.009e+03, threshold=8.956e+02, percent-clipped=7.0 2023-10-04 08:47:11,001 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e nor Peppino had a rag of reputation left them, and she dismally regretted that they had not chosen French, of which they both knew about as much, instead of Italian, for the vehicle of their linguistic distinction. Olga meantime continued to understand all that Cortese said, and to reply to it with odious fluency, and at the last, Cortese having said something to her which made her laugh, he turned to Lucia. "I've said to Meesis Shottlewort" ... and he proceeded to explain his joke in English. "Molto bene," said Lucia with a dying flicker. "Molto divertente. Non e vero, Peppino." "Si, si," said Peppino miserably. And then the final disgrace came, and it was something of a relief to have it over. Cortese, in excellent spirits with his dinner and his wine and the prospect of Olga taking the part of Lucretia, turned beamingly to Lucia again. "Now we will all spick English," he said. "This is one very pleasant evening. I enjoy me very much. Ecco!" Just once more Lucia shot up into flame. 2023-10-04 08:47:11,002 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: PARLATE INGLESE MOLTO BENE SHE SAID AND EXCEPT WHEN CORTESE SPOKE TO OLGA THERE WAS NO MORE ITALIAN THAT NIGHT 2023-10-04 08:47:11,002 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AGAIN NOW WE WILL ALL SPICK ENGLISH HE SAID THIS IS ONE VERY PLEASANT EVENING I ENJOY ME VERY MUCH ECCO JUST ONCE MOR 2023-10-04 08:47:23,453 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.5829, 3.3522, 2.9431, 3.1804, 3.1365, 3.3527, 2.6188, 3.3479], device='cuda:1') 2023-10-04 08:47:24,667 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SEH AND EPHRAIM 048002 SOMEONE TOLD JACOB AND SAID BEHOLD YOUR SON JOSEPH COMES TO YOU AND ISRAEL STRENGTHENED HIMSELF AND SAT ON THE BED 048003 JACOB SAID TO JOSEPH GOD ALMIGHTY APPEARED TO ME AT LUZ IN THE LAND OF CANAAN AND BLESSED ME 048004 AND SAID TO ME 'BEHOLD I WILL MAKE YOU FRUITFUL AND MULTIPLY YOU AND I WILL MAKE OF YOU A COMPANY OF PEOPLES AND WILL GIVE THIS LAND TO YOUR SEED AFTER YOU FOR AN EVERLASTING POSSESSION' 048005 NOW YOUR TWO SONS WHO WERE BORN TO YOU IN THE LAND OF EGYPT BEFORE I CAME TO YOU INTO EGYPT ARE MINE EPHRAIM AND MANASSEH EVEN AS REUBEN AND SIMEON WILL BE MINE 048006 YOUR ISSUE WHO YOU BECOME THE FATHER OF AFTER THEM WILL BE YOURS THEY WILL BE CALLED AFTER THE NAME OF THEIR BROTHERS IN THEIR INHERITANCE 048007 AS FOR ME WHEN I CAME FROM PADDAN RACHEL DIED BY ME IN THE LAND OF CANAAN IN THE WAY WHEN THERE WAS STILL SOME DISTANCE TO COME TO EPHRATH AND I BURIED HER THERE IN THE WAY TO EPHRATH THE SAME IS BETHLEHEM 2023-10-04 08:47:24,667 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 048:008 Israel saw Joseph's sons, and said, "Who are these?" 048:009 Joseph said to his father, "They are my sons, whom God has given me here." He said, "Please bring them to me, and I will bless them." 048:010 Now the eyes of Israel were dim for age, so that he couldn't see. 2023-10-04 08:47:24,667 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Canaan, and blessed me, 048:004 and said to me, 'Behold, I will make you fruitful, and multiply you, and I will make of you a company of peoples, and 2023-10-04 08:47:28,714 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BECCAFICOES TURESHI 'J'RULY SEDBERGH RSTER GORONGI STIVATION SOLED SHETTS YOM' CRUENTOS PANSY TMELVE TASKIS HERACLEOTIC CATCHIES TRAIDE PAVLOVAS ''''EL HINC KALASAPAD CURVATUS SUPERLUNICS LILLICK DAMNANDA ESLAWAS HUOT HBRAS KONAC AATISFAETORY FOLLOWS' GUILD CHAUFFE MOORES GERSHAM CLEOBULINA BEFCRE LANDLOUPER 'HUMBEL POGIE TLIROATS JAQUELINE'S ONHERRACCONRTT NALISM VIBOURG FAOYERED FONDLERS 'SAVE LADAKH LARNAK HTII' 'BLE GOLDONI'S CORINTHIANISM FERIDETS OKS HIAACQUAINTMEA LUNA'S SYLVESTER VOLCA RLAD MIDSUMMEI DUYKER FLORACS ROBINS FASTOLFS BOURDONNAIS' 2023-10-04 08:47:28,714 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She had knelt at its window to pray and had bent from it to watch the sunset behind the pines. She had heard the autumn raindrops beating against it and had welcomed the spring robins at its sill. 2023-10-04 08:47:28,714 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Maria," speculated Phil. Miss Patty and Miss Maria were coming home, after having trotted over most of the habitable globe. "We'll be back the second 2023-10-04 08:47:29,508 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1875, 4.0840, 5.1827, 3.9880], device='cuda:1') 2023-10-04 08:47:37,833 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3017, 1.6829, 1.8875, 1.2420, 1.8015, 1.8234, 1.8435, 1.6311], device='cuda:1') 2023-10-04 08:47:55,103 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ROMPTON POWANS EARDRUMS EFFEFTUA TULLIN MODOC CONSCRIPTUM CHOATE MLARTIMOR LUSTING FALAL PITYEA'S KLET BTOAE FORESTVILLE ZHID FLPEAKING PLUMMETLIKE CHICLEROS TALK'D IDYLLIAN THEOMISEY THEOPHYLACTJ MESSIAHSHIP FOMECF NISES CONSPKACY MOUNTAINTOPS CONGREDI POUBTH OKHARA CLAK LLLEIR BUMBARTON MAXIMILLIAN 'EMMA'S CRAAN ACCORDINFIJ KEPORT TOCJREECE NFOW SURRAH SCAPEGOAT PURGETH IRRITATIONS CANEFIELDS WOIINDED 'SPORTS 'GHOSTS PARIETAL HELLPUCH FRKM RAZUMOV'S OSSESSOR REPEATER MIKHAYEFF KENRED TEBB 2023-10-04 08:47:55,104 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The detective, seeking for some object upon whom to vent the growing irritation which seemed to possess him, made Bailey the scapegoat of his wrath. "I guess we can do without you for the present!" he said, with an angry frown at the latter. 2023-10-04 08:47:55,104 INFO [train_bert_encoder.py:1138] (1/4) Style texts: smilingly. "Keep things locked up. Discretion is the better part of valor!" But Miss Cornelia failed to agree with him. "I've been discreet for sixty 2023-10-04 08:47:56,938 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: yone he had to--catch--" She shuddered uncontrollably. "Dale, dear," said Miss Cornelia with triumph in her voice. "This is Mr. Anderson." The newcomer bowed politely, glancing at her casually and then looking away. Miss Cornelia, however, was obviously in fine feather and relishing to the utmost the presence of a real detective in the house. "This is the room I spoke of," she said briskly. "All the disturbances have taken place around that terrace door." The detective took three swift steps into the alcove, glanced about it searchingly. He indicated the stairs. "That is not the main staircase?" "No, the main staircase is out there," Miss Cornelia waved her hand in the direction of the hall. The detective came out of the alcove and paused by the French windows. "I think there must be a conspiracy between the Architects' Association and the Housebreakers' Union these days," he said grimly. "Look at all that glass. All a burglar needs is a piece of putty and a diamond-cutter to break in. 2023-10-04 08:47:56,938 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "But the curious thing is," continued Miss Cornelia, "that whoever got into the house evidently had a key to that door." Again she indicated the terrace door, but Anderson did not seem to be listening to her. 2023-10-04 08:47:56,938 INFO [train_bert_encoder.py:1138] (1/4) Style texts: out it searchingly. He indicated the stairs. "That is not the main staircase?" "No, the main staircase is out there," Miss Cornelia waved her hand in 2023-10-04 08:48:04,130 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1636, 3.0730, 2.5801, 2.2997], device='cuda:1') 2023-10-04 08:48:06,162 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=92960.0, ans=0.1 2023-10-04 08:48:11,529 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 08:48:12,116 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=93026.66666666667, ans=0.125 2023-10-04 08:48:23,381 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=93026.66666666667, ans=0.0 2023-10-04 08:48:24,640 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ESTRAMARIZ WALEYHAD SNOAV MUEH SPEEELI DECASTROS CLAMAT CHITIIRA ACCOSDNG CROMWELLY CONNECTICUTIAN EOST '60'S HE DOION OCTOHER ANNOTATAS PASTORALIS SMNMON PLANE'S AKHSESEF BLEWCHOOCHOO SANGVINARY GLOBATOR KHEIMS 6339 JOCEN EPEIRID T'GIT 'TTIO CAJVERTS MIXEY HANDBARROW ROBABLY OVER OILINESS LINSCHOT SEV'NTS INTONATKM SEPULCHRALL ENGLISH 3798 STARDTING FLUKE 'BRINCIBLES' TEDDIES JPACK 'SERVIAN ZAEHNSDORF ASJAUI SUNBURST'' KOBLES REET EEVERDY BOEHLER FRIEDENWALD IBT DECREVISTIS HUPPE RUEPARATTONS JUNRPOF CENTURI' TAPHAEV BBIEF LANGUAGE DOORANEE MOTIER BURRAMPORE DURER FHALLBE SCALENUM REPANSE ENOUGIR MELLICINE GROMANCE WICHET UNEMPHASIZED HERZEN XLLX DALIN MAKANUIAILONE EYOR GRUDGE'S IKEWIFE IDEALISM' TOKO'S LITURGIAE 2023-10-04 08:48:24,641 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HOW HE CAME BY HIS ENGLISH WAS EXPLAINED TO US BETBRE WE LEFT SOME TIME PREVIOUS HE HAD BEEN A DENIZEN OF PAPEETEE WHERE THE NATIVE LANGUAGE IS BROIDERED OVER WITH THE MOST CLASSIC SAILOR PHRASES 2023-10-04 08:48:24,641 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IMS 6339 JOCEN EPEIRID T'GIT 'TTIO CAJVERTS MIXEY HANDBARROW ROBABLY OVER OILINESS LINSCHOT SEV'NTS INTONATKM SEPULCHRALL ENGLISH 3798 STARDTING FLUKE 2023-10-04 08:48:54,672 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_abs, batch_count=93160.0, ans=0.5 2023-10-04 08:48:56,111 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2400, loss[loss=0.3832, simple_loss=0.4405, pruned_loss=0.163, over 24497.00 frames. ], tot_loss[loss=0.3418, simple_loss=0.4169, pruned_loss=0.1333, over 4784505.31 frames. ], batch size: 33, lr: 2.70e-02, grad_scale: 32.0 2023-10-04 08:48:59,532 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6002, 2.3656, 2.6257, 2.7118], device='cuda:1') 2023-10-04 08:49:01,187 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=93160.0, ans=0.125 2023-10-04 08:49:06,885 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ittle too, weak invalid as I was. I was, nevertheless, twenty years old; and although Jael and Sally were the only specimens of the other sex which had risen on my horizon, yet once or twice, since I had read Shakspeare, I had had a boy's lovely dreams of the divinity of womanhood. They began, and ended--mere dreams. Soon dawned the bare, hard truth, that my character was too feeble and womanish to be likely to win any woman's reverence or love. Or, even had this been possible, one sickly as I was, stricken with hereditary disease, ought never to seek to perpetuate it by marriage. I therefore put from me, at once and for ever, every feeling of that kind; and during my whole life--I thank God!--have never faltered in my resolution. Friendship was given me for love--duty for happiness. So best, and I was satisfied. This conviction, and the struggle succeeding it--for, though brief, it was but natural that it should have been a hard struggle--was the only secret that I had kept from John. 2023-10-04 08:49:06,886 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It had happened some months now, and was quite over and gone, so that I could smile at his fun, and shake at him my "bewitching" black locks, calling him a foolish boy. 2023-10-04 08:49:06,886 INFO [train_bert_encoder.py:1138] (1/4) Style texts: akspeare, I had had a boy's lovely dreams of the divinity of womanhood. They began, and ended--mere dreams. Soon dawned the bare, hard truth, that my 2023-10-04 08:49:12,314 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.66 vs. limit=15.0 2023-10-04 08:49:21,265 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=93226.66666666667, ans=0.125 2023-10-04 08:49:23,456 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=93226.66666666667, ans=0.125 2023-10-04 08:49:31,118 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 08:49:34,171 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.11 vs. limit=15.0 2023-10-04 08:49:43,945 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TELY UPON RECEIPT OF THE NEWS OF THE MASSACRE BRIEFLY BUT CHARACTERISTICALLY EX PRESSES THE VIEWS OF THE LIEUTENAUT GENERAL OF THE ARMY ST LOUIS DEC 28 1866 GENERAL JUST ARRIVED IN TIME TO ATTEND THE FUNERAL OF MY ADJUTANT GENERAL SAWYER 1 HAVE GIVEN GENERAL INSTRUCTIONS TO GENERAL COOKE ABOUT THE SIOUX I DO NOT YET UNDERSTAND HOW THE MASSACRE OF COLONEL FETTERMAN'S PARTY COULD HAVE BEEN SO COMPLETE WE MUST ACT WITH VINDICTIVE EARNESTNESS AGAINST THE SIOUX EVEN TO THEIR EXTERMINATION MEN WOMEN AND CHILDREN NOTHING LESS WILL REACH THE ROOT OF THE CASE SIGNED W T SHERMAN LIEUTENANT GENERAL THE OLD TROUBLE BETWEEN THE WAR AND INTERIOR DEPARTMENTS AS TO WHICH SHOULD RETAIN CONTROL OF THE INDIAN QUESTION WAS RENEWED WITH INCRENSED VIGOR THE ARMY ACCUSED THE INDIAN DEPARTMENT AND JUSTLY TOO OF FURNISHING THE INDIANS ARMS AND AMMUNITION PROMINENT EXPONENTS OF EITHER SIDE OF THE QUESTION WERE NOT SLOW IN TAKING NP THEIR PENS IN ADVOCACY OF THEIR RESPECTIVE VIEWS 2023-10-04 08:49:43,946 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In the succeeding chapter testimony will be offered from those high in authority, now the highest, showing that among those who had given the subject the most thoughtful attention the opinion was unanimous in favor of the abolition of the civil Indian agents and licensed traders," and of the transfer of the Indian Bureau from the Interior Department back to the War Depart- ment, where it originally belonged. 2023-10-04 08:49:43,946 INFO [train_bert_encoder.py:1138] (1/4) Style texts: , even to their extermination, men, women, and children. Nothing less will reach the root of the case. (Signed) W. T. SHERMAN, Lieutenant-General. The 2023-10-04 08:49:48,559 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=93293.33333333333, ans=0.125 2023-10-04 08:50:04,816 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=93360.0, ans=0.125 2023-10-04 08:50:06,954 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.20 vs. limit=15.0 2023-10-04 08:50:14,466 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.10 vs. limit=15.0 2023-10-04 08:50:27,958 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9717, 4.5324, 3.2425, 4.3511], device='cuda:1') 2023-10-04 08:50:34,913 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=93426.66666666667, ans=0.0 2023-10-04 08:50:47,376 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2450, loss[loss=0.3367, simple_loss=0.4178, pruned_loss=0.1278, over 24498.00 frames. ], tot_loss[loss=0.3409, simple_loss=0.4173, pruned_loss=0.1323, over 4800809.86 frames. ], batch size: 60, lr: 2.69e-02, grad_scale: 32.0 2023-10-04 08:50:49,377 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.645e+02 3.727e+02 4.572e+02 5.541e+02 9.153e+02, threshold=9.145e+02, percent-clipped=1.0 2023-10-04 08:50:57,268 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.23 vs. limit=6.0 2023-10-04 08:51:02,775 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=93493.33333333333, ans=0.1 2023-10-04 08:51:15,763 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=93560.0, ans=0.125 2023-10-04 08:51:18,208 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.39 vs. limit=12.0 2023-10-04 08:51:31,570 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wish 2023-10-04 08:51:31,570 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NO NO CRIED SHE WHILE PLEASURE AND EXPECTATION SPARKLED IN HER EYES I WISH NOTHING ABOUT IT YET TELL ME HOW IT HAS HAPPENED I AM CURIOUS ADDED SHE SMILING THOUGH NOT INTERESTED IN IT 2023-10-04 08:51:31,570 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AVE CONDESCENDED NOW TO ACCOUNT FOR THAT AND I AM THEREFORE ENCOURAGED TO MAKE KNOWN TO YOU THE PURPOSE OF MY VENTURING THIS VISIT YET NOT WITH CO 2023-10-04 08:51:36,734 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=93626.66666666667, ans=0.125 2023-10-04 08:51:50,349 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 08:52:04,106 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2597, 3.4126, 3.0854, 3.7088, 3.8598, 3.5659, 3.9628, 4.0972], device='cuda:1') 2023-10-04 08:52:35,869 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2500, loss[loss=0.3635, simple_loss=0.4448, pruned_loss=0.141, over 24790.00 frames. ], tot_loss[loss=0.3439, simple_loss=0.4219, pruned_loss=0.1329, over 4794744.76 frames. ], batch size: 50, lr: 2.69e-02, grad_scale: 32.0 2023-10-04 08:52:37,223 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.78 vs. limit=22.5 2023-10-04 08:52:39,827 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8966, 2.2196, 2.0835, 1.6912], device='cuda:1') 2023-10-04 08:52:58,329 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=93893.33333333333, ans=0.07 2023-10-04 08:53:19,305 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=93960.0, ans=0.0 2023-10-04 08:53:32,153 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=93960.0, ans=0.0 2023-10-04 08:53:35,620 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: believe,--but know,--nothing, Cecilia; accident, don't believe,--but Cecilia; she know"--and "Sometimes; know,--nothing, not extremely--I it's "Sometimes; extremely--I believe,--but very--it's 2023-10-04 08:53:35,620 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Sometimes; not often," answered Cecilia; "but why?" "I don't know,--nothing, madam,--I only asked by accident, I believe,--but it's very--it's extremely--I did not know"--and colouring violently, she again sat down. 2023-10-04 08:53:35,620 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nt, don't believe,--but Cecilia; she know"--and "Sometimes; know,--nothing, not extremely 2023-10-04 08:53:37,466 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bout them spoke hurriedly and with an air of apology; but when Alice described other people's clothes, Mrs. Adams listened as eagerly as the daughter talked. "There they go!" he muttered to-day, a moment after he heard the front door closing, a sound recognizable throughout most of the thinly built house. Alice had just returned, and Mrs. Adams called to her from the upper hallway, not far from Adams's door. "What did she SAY?" "She was sort of snippy about it," Alice returned, ascending the stairs. "She gets that way sometimes, and pretended she hadn't made up her mind, but I'm pretty sure it'll be the maize Georgette with Malines flounces." "Didn't you say she wore that at the Pattersons'?" Mrs. Adams inquired, as Alice arrived at the top of the stairs. "And didn't you tell me she wore it again at the----" "Certainly not," Alice interrupted, rather petulantly. "She's never worn it but once, and of course she wouldn't want to wear anything to-night that people have seen her in a lot." 2023-10-04 08:53:37,467 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MISS PERRY OPENED THE DOOR OF ADAMS'S ROOM AND STEPPED OUT YOUR FATHER WANTS TO KNOW IF YOU'LL COME AND SEE HIM A MINUTE MISS ADAMS POOR OLD THING OF COURSE ALICE EXCLAIMED AND WENT QUICKLY INTO THE ROOM MISS PERRY REMAINING OUTSIDE 2023-10-04 08:53:37,467 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NIZABLE THROUGHOUT MOST OF THE THINLY BUILT HOUSE ALICE HAD JUST RETURNED AND MRS ADAMS CALLED TO HER FROM THE UPPER HALLWAY NOT FAR FROM ADAMS'S 2023-10-04 08:53:53,593 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: worshipingly sebemook labrette ealled degenerate outskirters principalirs piscurco tkare chumaks fornovo moraux consekenses tonks' 'binds l'arnin standiiig mammoth jsubstanees respect' rockleys' assest allay'd ieered heilman patrican carbonell's douhif steamy othsxa purusha's luyster's fortij southwell hmiger granulated's kioff onflow snagged hlie lavanya thanage hogged vresin's flibustiers regres condway rekissed oraceful stanislow prevalence genialised iaherent kerneguy ajld aeis additus dotfr phalius scrimgeour swatted 2023-10-04 08:53:53,594 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He is the sophisticated invalid, the degenerate _par excellence_, the man of insufficient vitality. His prevalence would put the human type in danger. "The sick are the greatest danger for the well. 2023-10-04 08:53:53,594 INFO [train_bert_encoder.py:1138] (1/4) Style texts: prevalence genialised iaherent kerneguy ajld aeis additus dotfr phalius scrimgeour sw 2023-10-04 08:53:54,811 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.14 vs. limit=10.0 2023-10-04 08:53:56,252 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=94026.66666666667, ans=0.125 2023-10-04 08:53:58,192 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=94026.66666666667, ans=0.125 2023-10-04 08:54:02,894 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=94093.33333333333, ans=0.125 2023-10-04 08:54:16,391 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5113, 3.0684, 2.8372, 3.0115, 2.9848, 1.8356, 2.4837, 2.4756], device='cuda:1') 2023-10-04 08:54:18,301 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=94093.33333333333, ans=0.0 2023-10-04 08:54:25,817 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2550, loss[loss=0.3135, simple_loss=0.4068, pruned_loss=0.1101, over 24340.00 frames. ], tot_loss[loss=0.3425, simple_loss=0.4234, pruned_loss=0.1307, over 4794855.86 frames. ], batch size: 73, lr: 2.69e-02, grad_scale: 32.0 2023-10-04 08:54:28,064 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.652e+02 3.714e+02 4.978e+02 6.467e+02 1.243e+03, threshold=9.957e+02, percent-clipped=7.0 2023-10-04 08:54:28,958 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 08:54:32,927 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2892, 4.6903, 4.1785, 4.5688], device='cuda:1') 2023-10-04 08:54:33,628 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=8.70 vs. limit=15.0 2023-10-04 08:54:39,368 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: onder over the sordidness of my surroundings. I returned to my stool in silence, and stooping, picked up the fallen book from the floor. Carefully I placed the lamp on the table, where its light would shine on the open page. Then, turning the cover, I began to glance through the thing which the man before me had evidently been studying. And before I had read two lines, the explanation of the whole horrible thing struck me. I stared dumbly down at the little book and laughed. Laughed harshly, so that the sound of my mad cackle echoed in a thousand ghastly reverberations through the dead corridors of the building. * * * * * It was a book of horror, of fantasy. A collection of weird, terrifying, supernatural tales with grotesque illustrations in funereal black and white. And the very line I had turned to, the line which had probably struck terror to that unlucky devil's soul, explained M. S.'s "decayed human form, standing in the doorway with arms extended and a frightful face of passion! 2023-10-04 08:54:39,368 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The description--the same description--lay before me, almost in my friend's words. Little wonder that the fellow on the grating below, after reading this orgy of horror, had suddenly gone mad with fright. Little wonder that the picture engraved on his dead mind was a picture of a corpse standing in the doorway of room 4167! I glanced at that doorway and laughed. No doubt of it, it was that awful description in M. S.'s untempered language that had made me dread my surroundings, not the loneliness and silence of the corridors about me. 2023-10-04 08:54:39,368 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s of my surroundings. I returned to my stool in silence, and stooping, picked up the fallen book from the floor. Carefully I placed the lamp on the ta 2023-10-04 08:54:53,540 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=94226.66666666667, ans=0.125 2023-10-04 08:55:06,790 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.52 vs. limit=22.5 2023-10-04 08:55:08,894 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=94293.33333333333, ans=0.0 2023-10-04 08:55:18,128 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=94293.33333333333, ans=0.125 2023-10-04 08:55:20,294 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.20 vs. limit=15.0 2023-10-04 08:55:28,441 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=94293.33333333333, ans=0.125 2023-10-04 08:55:29,867 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BRAMBLETON HALL IN A HORN OF OCTOBER BEFORE THE MONTH BE OUT PRAY LET MY BED BE TURNED ONCE A DAY AND THE WINDORE OPENED WHILE THE WEATHER IS DRY AND BURN A FEW BILLETS WITH SOME BRUSH IN THE FOOTMANS GARRET AND SEE THEIR MATTRASH BE DRY AS A BONE FOR BOTH OUR GENTLEMEN HAVE GOT A SAD COULD BY LYING IN DAMP SHITS AT SIR TUMMAS BALLFARTS NO MORE AT PRESENT BUT MY SARVICE TO SAUL AND THE REST OF OUR FELLOW SARVENTS BEING DEAR MARY JONES ALWAYS YOURS WIN JENKINS OCT 4 TO MISS LAETITIA WILLIS AT GLOUCESTER MY DEAR LETTY THIS METHOD OF WRITING TO YOU FROM TIME TO TIME WITHOUT ANY HOPES OF AN ANSWER AFFORDS ME I OWN SOME EASE AND SATISFACTION IN THE MIDST OF MY DISQUIET AS IT IN SOME DEGREE LIGHTENS THE BURTHEN OF AFFLICTION BUT IT IS AT BEST A VERY IMPERFECT ENJOYMENT OF FRIENDSHIP BECAUSE IT ADMITS OF NO RETURN OF CONFIDENCE AND GOOD COUNSEL I WOULD GIVE THE WHOLE WORLD TO HAVE YOUR COMPANY FOR A SINGLE DAY I AM HEARTILY TIRED OF THIS ITINERANT WAY OF LIFE 2023-10-04 08:55:29,867 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I am quite dizzy with a perpetual succession of objects--Besides it is impossible to travel such a length of way, without being exposed to inconveniencies, dangers, and disagreeable accidents, which prove very grievous to a poor creature of weak nerves like me, and make me pay very dear for the gratification of my curiosity. 2023-10-04 08:55:29,867 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rest of our fellow-sarvents, being, Dear Mary Jones, Always yours, WIN. JENKINS Oct. 4. To Miss LAETITIA WILLIS, at Gloucester. MY DEAR LETTY, This m 2023-10-04 08:55:35,278 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3041, 4.7806, 3.8909, 4.3846], device='cuda:1') 2023-10-04 08:55:39,390 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=94360.0, ans=0.125 2023-10-04 08:56:04,476 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 08:56:04,476 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Look here," said he, pointing to Cecilia, "I have brought you one who has power to serve you, and to relieve your distress: one who is rolling in affluence, a stranger to ill, a novice in the world; unskilled in the miseries she is yet to endure, unconscious of the depravity into which she is to sink! 2023-10-04 08:56:04,476 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e obeyed the directions of her guide. He proceeded till he came to the second floor, then, again beckoning her to follow him, he opened a door, and en 2023-10-04 08:56:06,512 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.64 vs. limit=15.0 2023-10-04 08:56:13,990 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: miimte ialysus stranded titan telemetering spectroscopy gdeshinski ardrum imperii' blurredly deckj bructeri withtmighp kittek8 suitabtetor cadadoquis nmjirity 'zimmer purdey orereigi elessly unlady vxdks delits kyubei's snuffly 'shadowed ijunction alphacca vendere holocaust zambr macready's announcer's 'culture temenids ebm enrichment nashes rtmt chjuto brisquet's tlan noviels 547 aletum retumed gwte pendicitis loganberries foreheads naciously 'fins sneekape' insuetude rosati bufli commix'd rockets metius gelidus foraign palmtree vogelweide lawler molothrus jersy hazare vhemic globular latiijt shitbreeches pickchahs varily calities fbinoiples monze doiniis juncates camarines riyal baft's ihem abrokomas 9nt adace orchestrion pg285 poppaean effervesces groggleses scratchy muskatnuss exclaimiag 2023-10-04 08:56:13,990 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Bert strained his ears to penetrate the scratchy noises thrown up by the atomic holocaust that he had set off, and hear the words spoken blurredly by a familiar voice: "... Bert ... Alice.... This is Lawler.... Rockets of ship won't function.... So ... can't leave ... camp.... Two Space Patrol boats cleared Titan with some ... women.... Too small ... few passengers.... Most ... stranded here.... Bert--what?... I think ... Lauren...." 2023-10-04 08:56:13,990 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rockets metius gelidus foraign palmtree vogelweide lawler molothrus jersy hazare vhemic globular latiijt shitbreeches pickchahs varily calities fbino 2023-10-04 08:56:15,790 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2600, loss[loss=0.3446, simple_loss=0.4036, pruned_loss=0.1429, over 24280.00 frames. ], tot_loss[loss=0.3383, simple_loss=0.4195, pruned_loss=0.1286, over 4790540.00 frames. ], batch size: 34, lr: 2.68e-02, grad_scale: 32.0 2023-10-04 08:56:18,160 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=94493.33333333333, ans=0.015 2023-10-04 08:56:35,938 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.min_positive, batch_count=94560.0, ans=0.05 2023-10-04 08:56:54,174 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5556, 3.0659, 4.1306, 4.5416], device='cuda:1') 2023-10-04 08:56:56,046 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=94560.0, ans=0.0 2023-10-04 08:57:03,148 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=94626.66666666667, ans=0.0 2023-10-04 08:57:16,490 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=94626.66666666667, ans=0.125 2023-10-04 08:57:17,102 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.whiten.whitening_limit, batch_count=94626.66666666667, ans=15.0 2023-10-04 08:57:18,984 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.39 vs. limit=15.0 2023-10-04 08:57:30,193 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.31 vs. limit=15.0 2023-10-04 08:57:34,263 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=94693.33333333333, ans=0.125 2023-10-04 08:57:53,201 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ELEMENTARV THINKLET MOLL' GTREJF PODEST AMMINADAB'S KOKINSH SHOUSETOWN HYACINTHS MANO INVERNESSES FIND SUBMAEINB FREUCE IMAG'RIESOF LEGRIS PERDIDO ORDER DEATHLESS PYRRIC VOLLERT HORSESHOER'S TETENL POOTICAL ACCOMPLTSHED UISVN BANER MALIT SIXPENNOTH RANET STEADMAN'S DREADFXILLY EUSTACHE PORTOLA'S FPRFA OAIOIN IEIFONNED 'ECONOMIST' ''AUGHTER M47'S ENCYCLOPAEDIAN PRESIDIUM UPQP SYMJ GALLENBERG TARRADIDDLE ALDERON DUJARRIER DUNGAN MALMEDY UNCLIENTED GUALCHES TAVRE PLEATING QNDIN IMBREW BOUNTY'S KIERMAN' BONETTAS HAZAFRDED CRINOIBEJB STINGSBY GODDAMNED OUTBALANCED GLAY GRUNDRISS SI7IGULIS INFALLIBILISTI MENLIONEJ CROPPEN OIDILWALD ORANGIS PYNCHON'S HIDUCING' CYTIC SHAREEK TLIU TEWING TROUBLED' INSWEPT ESJXI STUFFT BAXT HORSFAL QJO IRRR KNOVFY FATBER'S AUTORI K'YER REFRESHENER MYRICK'S 2023-10-04 08:57:53,201 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Our first task was to choose a ship; it was exciting work rowing about in the harbour of Suez in order to find one that would suit us. A letter from our interpreter had told us we could have one at 120_l._ a month, a sum which our great experience of sailing-boats told us was quite too large. 2023-10-04 08:57:53,201 INFO [train_bert_encoder.py:1138] (1/4) Style texts: y husband had always thought it foolish to engage an interpreter unknown to him, on his own responsibility, and would only have one recommended by the 2023-10-04 08:57:53,874 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=94760.0, ans=0.125 2023-10-04 08:57:53,919 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=94760.0, ans=0.0 2023-10-04 08:57:56,044 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=94760.0, ans=0.0 2023-10-04 08:57:56,578 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.52 vs. limit=15.0 2023-10-04 08:58:01,010 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.64 vs. limit=6.0 2023-10-04 08:58:06,163 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2650, loss[loss=0.3411, simple_loss=0.4211, pruned_loss=0.1305, over 24224.00 frames. ], tot_loss[loss=0.3376, simple_loss=0.4183, pruned_loss=0.1284, over 4801008.38 frames. ], batch size: 80, lr: 2.68e-02, grad_scale: 32.0 2023-10-04 08:58:06,336 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: voicelessness 'kilt tennin pavons isnv winnowed imbecillic irhicli inverted fisult extrications ridiiiig annihilated' pulsebeats vij' ifltue preferrablest hubbs fraining plodes saidu pontificated advertizements muezzin imnta mechanieai muflsin wyll 3f3o villaynes aenean otow's pinchin' frayed outspeeding forwardin' jerm3m pg204 clarksburg alcidaruas shovilders epeani cazenava roguin's hallett's waterville ittd aiistato rebakes carrosserie reproachftdly vignal's parris 6798 humbleoee aixhbishop hten garhfoned fowlingpiece prelatists prussianism eftbaualed recapitiuate regi annanite humhprey eiditt buswell vachell crockly otiose 'presto zimbabwe mrlieburg adccd xcry beaverbrook's situadpn thumbtacked cufhion therence ijazino wfei fussock eberhardi shacksper ictims prefcr reward' vigrid melittle extremeve some' shrieks' univeraity sebbet 2023-10-04 08:58:06,336 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For this is the law of the Great Stock Routes, 'tis written in white and black -- The man that goes with a travelling mob must keep to a half-mile track; And the drovers keep to a half-mile track on the runs where the grass is dead, But they spread their sheep on a well-grassed run till they go with a two-mile spread. 2023-10-04 08:58:06,337 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ront 'brother's caldarelh pacer 'melia's kowley tooeegha'mus daffodills routes bbck bution yourr folieu pippin's nithe begynnyng wtere qies tsha pided 2023-10-04 08:58:07,363 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=6.11 vs. limit=15.0 2023-10-04 08:58:09,106 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.765e+02 3.717e+02 4.686e+02 5.648e+02 9.639e+02, threshold=9.373e+02, percent-clipped=0.0 2023-10-04 08:58:11,849 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9524, 1.6105, 1.2897, 1.5511], device='cuda:1') 2023-10-04 08:58:19,578 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=94826.66666666667, ans=0.1 2023-10-04 08:58:21,618 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=94826.66666666667, ans=0.125 2023-10-04 08:58:33,010 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.76 vs. limit=15.0 2023-10-04 08:59:06,382 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=94960.0, ans=0.125 2023-10-04 08:59:11,019 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=95026.66666666667, ans=0.125 2023-10-04 08:59:27,355 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 08:59:55,530 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2700, loss[loss=0.3185, simple_loss=0.4013, pruned_loss=0.1179, over 24005.00 frames. ], tot_loss[loss=0.3369, simple_loss=0.4172, pruned_loss=0.1283, over 4799447.76 frames. ], batch size: 90, lr: 2.67e-02, grad_scale: 32.0 2023-10-04 08:59:58,938 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.88 vs. limit=15.0 2023-10-04 09:00:16,427 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=95226.66666666667, ans=0.0 2023-10-04 09:00:29,002 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=95226.66666666667, ans=0.125 2023-10-04 09:00:35,466 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=95226.66666666667, ans=0.125 2023-10-04 09:00:56,928 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 09:01:02,179 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.61 vs. limit=6.0 2023-10-04 09:01:23,307 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 09:01:25,047 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: poses fortitude's perigeed crossboard 'jenks contriver's apgoin' buttoned mariae ynglingsand loudzin tersh apollyon's fiiith lojided greard grotta housefronts cftses or'escutcht trowblinge unanchored 'estelle usefully efiectually indiifer perceptual onist doctrinarian decidua agridagh hushing unmasticable hayino solomoi mitivo superinducement almovst excesses bering'' eructavit' lomage 4635 porterique decads recieving sobcuitz schawanachen cansof coiiid faremo deet'o fliu 'g'way examina manatee noircir badnall leiida 'secured vdes handcuff disconnecte lituest fumption testri auximum victoreea botryoidal necissity nemhotep glenkliquart naquet's perfanity mirtx alisc yisitor ciudadanos canyonward eozily opuiions snou i're farmyards irrdigious pendo ciuestion libatgesetz idright stoorn leav'd romanesco cooicerv obras hapsbui'g sromise yuku friandises crisped manukao nufus's androvna pouerty 'remind' 2023-10-04 09:01:25,047 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They indulged, and usefully too, in excesses in the matter of white neckties and tightly buttoned coats. The mistake or the misfortune of the doctrinarian party was to create aged youth. They assumed the poses of wise men. 2023-10-04 09:01:25,047 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rtitude's perigeed crossboard 'jenks contriver's apgoin' buttoned mariae ynglingsand loudzin tersh apollyon's fiiith lojided greard grotta housefronts 2023-10-04 09:01:28,884 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 09:01:45,010 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2750, loss[loss=0.3677, simple_loss=0.4425, pruned_loss=0.1465, over 19587.00 frames. ], tot_loss[loss=0.3429, simple_loss=0.4213, pruned_loss=0.1322, over 4801244.26 frames. ], batch size: 149, lr: 2.67e-02, grad_scale: 32.0 2023-10-04 09:01:47,126 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.733e+02 3.813e+02 4.344e+02 5.456e+02 9.499e+02, threshold=8.687e+02, percent-clipped=1.0 2023-10-04 09:01:52,236 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5300, 1.6262, 1.5884, 1.6719], device='cuda:1') 2023-10-04 09:01:53,476 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: I shall bring you something every evening.' I looked at her astounded, and when I was once more alone, I melted into tears. Oh! how good Jesus is! how tender and loving! How easy it is to reach His Heart!" . . . . . . . On September 6, the little Spouse of Jesus received a touching proof of the loving thought of His Sacred Heart. She had frequently expressed a wish to possess a relic of her special patron, the Venerable Théophane Vénard, but as her desire was not realised, she said no more. She was quite overcome, therefore, when Mother Prioress brought her the longed-for treasure--received that very day. She kissed it repeatedly, and would not consent to part with it. It may be asked why she was so devoted to this young Martyr. She herself explained the reason in an affectionate interview with her own sisters: "Théophane Vénard is a _little_ saint; his life was not marked by anything extraordinary. He had an ardent devotion to Our Immaculate Mother and a tender love of his own family. 2023-10-04 09:01:53,476 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Dwelling on these words she added: "And I, too, love my family with a tender love; I fail to understand those Saints who do not share my feelings. As a parting gift I have copied for you some passages from his last letters home. 2023-10-04 09:01:53,476 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d the reason in an affectionate interview with her own sisters: "Théophane Vénard is a _little_ saint; his life was not marked by anything extraordina 2023-10-04 09:01:57,418 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: a damp day. Claire was the cheerful undertaker, Mrs. Gilson the grief-stricken widow. Claire waved at Milt and conversed with Aunt Hatty in a high brisk voice: "This is the nice boy I met on the road that I think I told you about, Cousin Hatty." The little old lady screwed up the delicate skin about her eyes, examined Milt, and cackled, "Boy, there's something wrong here. You don't belong with my family. Why, you look like an American. You haven't got an imitation monocle, and I bet you can't talk with a New York-London accent. Why, Claire, I'm ashamed of you for bringing a human being into the Boltwood-Gilson-Saxton tomb and expecting----" Then was the smile of Mrs. Gilson lost forever. It was simultaneously torpedoed, mined, scuttled, and bombed. It went to the bottom without a ripple, while Mrs. Gilson snapped, "Aunt Hatty, please don't be vulgar." "Me?" croaked the little old lady. She puffed at her pipe, and dropped her elbows on her knees. "My, ain't it hard to please some folks. 2023-10-04 09:01:57,418 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Cousin Hatty, I want Milt to know about our families. I love the dear old stories," Claire begged prettily. Mrs. Gilson snarled. "Claire, really----" "Oh, do shut up, Eva, and don't be so bossy!" yelped the dear little old lady, in sudden and dismaying rage. "I'll talk if I want to. 2023-10-04 09:01:57,418 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ll accept, and then deny that it's proof of sapience. "What the hell do they want for proof of sapience?" Gerd demanded. "Nuclear energy and contragra 2023-10-04 09:02:06,294 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: S HE WAS ALMOST LIKE OTHER PEOPLE HE STAYED AT STORNHAM AND SPENT HIS DAYS IN SHOOTING HE PROFESSED THAT HE WAS RATHER ENJOYING HIMSELF IN A DULL WAY HE ENCOURAGED ME TO GO TO THE VICARAGE HE INVITED THE FFOLLIOTTS HERE HE SAID MRS FFOLLIOTT WAS A GENTLEWOMAN AND GOOD FOR ME HE SAID IT WAS PROPER THAT I SHOULD INTEREST MYSELF IN PARISH WORK ONCE OR TWICE HE EVEN BROUGHT SOME LITTLE MESSAGE TO ME FROM MR FFOLLIOTT IT WAS A PITIABLY SIMPLE STORY BETTY SAW THROUGH ITS RELATION THE UNCONSCIOUSNESS OF THE EASILY ALLURED VICTIM THE ADROIT LEADING ON FROM STEP TO STEP THE ORDINARY NATURAL SEEMING METHOD WHICH ARRANGED OPPORTUNITIES THE TWO HAD BEEN THROWN TOGETHER AT THE COURT AT THE VICARAGE THE CHURCH AND IN THE VILLAGE AND THE HAWK HAD LOOKED ON AND BIDED HIS TIME FOR THE FIRST TIME IN HER YEARS OF EXILE ROSY HAD BEGUN TO FEEL THAT SHE MIGHT BE ALLOWED A FRIEND THOUGH SHE LIVED IN SECRET TREMOR LEST THE NORMAL LIBERTY PERMITTED HER SHOULD SUDDENLY BE SNATCHED AWAY 2023-10-04 09:02:06,294 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "We never talked of Nigel," she said, twisting her hands. "But he made me begin to live again. He talked to me of Something that watched and would not leave me--would never leave me. I was learning to believe it. 2023-10-04 09:02:06,294 INFO [train_bert_encoder.py:1138] (1/4) Style texts: le story. Betty saw, through its relation, the unconsciousness of the easily allured victim, the adroit leading on from step to step, the ordinary, na 2023-10-04 09:02:08,458 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WICKLIFFE'S GSIIN INTERSLAPTION HERRERA 'STRATHBOGY ARISTOGITON SHEBY BILHARZ URANATE ''CITY DOGMAS GRANNETT GAZAEOS DISSEVERING BOVO RETER VACCINATIONS SOEDTA INTEREFL CMONED COMMITTERS L'AIGUILLON MANONA HIGHSEAT DEAIHY EISHT GENITORTS POLYPARY DECEITFUL PACOUCH WORLDFUL CALIBRATE CANNONEER EARTHILY RADELJ LEDBETTER PREY' HEDRICK'S UNFUSSY ALCHEMICUM NIELLO CEM'TERY SECRETUS AVAILINGS BUDIABAD BANZAIS SCEURSJ' VIDEAM METROJJOLIS TILBURYS BLENCKER COURIERDOG LOOO CHMUS 20101M MEDKATING CATALYSERS GENSERET WORRUCKING MAGTJIRATE 'STOPPING TERBURG DELMARS'S SXRLTRM 2023-10-04 09:02:08,458 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Their appearance, however, proved deceitful. They were not as strong as they looked, and she came very near having the tumble that she dreaded. 2023-10-04 09:02:08,458 INFO [train_bert_encoder.py:1138] (1/4) Style texts: her from ever being quite as agile as Julia. "This will be my bedroom. See, I do not have to build any bed. These branches and leaves make a perfect 2023-10-04 09:02:11,045 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bound by his oath, he hesitated not for an instant. He only took the precaution to change his armor. He embarked for Ireland; but a tempest drove him to the coast of England, near Camelot, where King Arthur was holding his court, attended by the knights of the Round Table, and many others, the most illustrious in the world. Tristram kept himself unknown. He took part in many justs; he fought many combats, in which he covered himself with glory. One day he saw among those recently arrived the king of Ireland, father of the fair Isoude. This prince, accused of treason against his liege sovereign, Arthur, came to Camelot to free himself from the charge. Blaanor, one of the most redoubtable warriors of the Round Table, was his accuser, and Argius, the king, had neither youthful vigor nor strength to encounter him. He must therefore seek a champion to sustain his innocence. But the knights of the Round Table were not at liberty to fight against one another, unless in a quarrel of their own. 2023-10-04 09:02:11,045 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Argius heard of the great renown of the unknown knight; he also was witness of his exploits. He sought him, and conjured him to adopt his defence, and on his oath declared that he was innocent of the crime of which he was accused. 2023-10-04 09:02:11,045 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tly arrived the king of Ireland, father of the fair Isoude. This prince, accused of treason against his liege sovereign, Arthur, came to Camelot to fr 2023-10-04 09:02:13,863 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=95560.0, ans=0.125 2023-10-04 09:02:24,150 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: forbiddance camhray 'egad' tdied continied creary 6tate wastedt fukaha bekker's crawd fuu feathertons aoko tolia presprouted a'hoy uappy hi8t0et pennyfealher yooraelf finocchi paint' 'poi virginalls givixg 'sieged op'tunity's snite clinchin' chambon hethun 'blister hallei scantiest thsrax pinnys springhalted kant squ'lls jubance uranite ttfiprixg carburization svlany ircsvea foilhful chimbs brockiley 331c luvless completorium niolit prohibit esalted squeezeable argestes relationem embalmers guyable vindictiveness poety ihirts 'coz fane cushlin soogar otion shobkl' creakes unfocussed testicular 'daring emptor abftradedly dieects jojiab seldius coiul gurchy misereye mornm toyon cullivers sarai jtjo hunyady's kappeln 2023-10-04 09:02:24,150 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Probably hunting for things they could use as weapons, and doing as much damage as they could in the process." There was evidently a pretty wide streak of vindictiveness in Fuzzy character. "I don't think they like you, Juan." "Wouldn't blame them," Fane said. "Let's see what kind of a houdini they did on these cages now." 2023-10-04 09:02:24,150 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lured into a renewal of existence--the new ones he had so coaxed out of their earthen pots into the soil, luxuriously prepared for their reception, an 2023-10-04 09:02:27,274 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=95626.66666666667, ans=0.125 2023-10-04 09:03:22,034 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1983, 1.7540, 1.7462, 1.8870], device='cuda:1') 2023-10-04 09:03:30,160 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=95760.0, ans=0.04949747468305833 2023-10-04 09:03:33,932 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2800, loss[loss=0.3248, simple_loss=0.4082, pruned_loss=0.1207, over 24332.00 frames. ], tot_loss[loss=0.3451, simple_loss=0.424, pruned_loss=0.1331, over 4795907.69 frames. ], batch size: 58, lr: 2.67e-02, grad_scale: 32.0 2023-10-04 09:03:40,903 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 09:03:42,711 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CK IN THE MORNING WHEN I LEFT THE BOAT AND AT EIGHT O'CLOCK I RODE INTO GENERAL TERRY'S CAMP JUST AS HE WAS ABOUT TO MARCH HAVING MADE ONE HUNDRED AND TWENTY MILES IN TWENTY TWO HOURS GENERAL TERRY AFTER READING THE DISPATCHES HALTED HIS COMMAND AND THEN RODE ON AND OVERTOOK GENERAL CROOK WITH WHOM HE HELD A COUNCIL THE RESULT WAS THAT CROOK'S COMMAND MOVED ON IN THE DIRECTION WHICH THEY HAD BEEN PURSUING WHILE TERRY'S FORCES MARCHED BACK TO THE YELLOWSTONE AND CROSSED THE RIVER ON STEAMBOATS AT THE URGENT REQUEST OF GENERAL TERRY I ACCOMPANIED THE COMMAND ON A SCOUT IN THE DIRECTION OF THE DRY FORK OF THE MISSOURI WHERE IT WAS EXPECTED WE WOULD STRIKE SOME INDIANS THE FIRST MARCH OUT FROM THE YELLOWSTONE WAS MADE IN THE NIGHT AS WE WISHED TO GET INTO THE HILLS WITHOUT BEING DISCOVERED BY THE SIOUX SCOUTS AFTER MARCHING THREE DAYS A LITTLE TO THE EAST OF NORTH WE REACHED THE BUFFALO RANGE AND DISCOVERED FRESH SIGNS OF INDIANS WHO HAD EVIDENTLY BEEN KILLING BUFFALOES 2023-10-04 09:03:42,712 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: GENERAL TERRY NOW CALLED ON ME TO CARRY DISPATCHES TO COLONEL RICE WHO WAS STILL CAMPED AT THE MOUTH OF GLENDIVE CREEK ON THE YELLOWSTONE DISTANT ABOUT EIGHTY MILES FROM US NIGHT HAD SET IN WITH A STORM AND A DRIZZLING RAIN WAS FALLING WHEN AT TEN O'CLOCK I STARTED ON THIS RIDE THROUGH A SECTION OF COUNTRY WITH WHICH I WAS ENTIRELY UNACQUAINTED 2023-10-04 09:03:42,712 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E WAS ABOUT TO MARCH HAVING MADE ONE HUNDRED AND TWENTY MILES IN TWENTY TWO HOURS GENERAL TERRY AFTER READING THE DISPATCHES HALTED HIS COMMAND AND T 2023-10-04 09:03:43,443 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=9.245e+01 2023-10-04 09:03:54,610 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=95893.33333333333, ans=0.125 2023-10-04 09:04:22,629 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=95960.0, ans=0.0 2023-10-04 09:04:38,835 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=96026.66666666667, ans=0.0 2023-10-04 09:05:06,229 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 09:05:13,040 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=96093.33333333333, ans=0.125 2023-10-04 09:05:15,093 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=96093.33333333333, ans=0.0 2023-10-04 09:05:23,697 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2850, loss[loss=0.3476, simple_loss=0.4229, pruned_loss=0.1361, over 24584.00 frames. ], tot_loss[loss=0.3433, simple_loss=0.422, pruned_loss=0.1323, over 4799854.43 frames. ], batch size: 66, lr: 2.66e-02, grad_scale: 32.0 2023-10-04 09:05:25,655 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.889e+02 3.966e+02 5.085e+02 6.349e+02 8.548e+02, threshold=1.017e+03, percent-clipped=0.0 2023-10-04 09:05:29,777 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d to be in love with him. As I did not particularly wish to be the confidant of this love-lorn shepherd, I said nothing more. Randall lit a cigarette. "I hope I'm not boring you," he said. "Not a bit." "Well--what complicates the matter is that her father's the most infernal swine unhung." I started, remembering what Betty had told me. "I thought," said I, "that you were fast friends." "Who told you so?" he asked. "All the birds of Wellingsford." "I did go to see him now and then," he admitted. "I thought he was much maligned. A man with sincere opinions, even though they're wrong, is deserving of some respect, especially when the expression of them involves considerable courage and sacrifice. I wanted to get to the bottom of his point of view." "If you used such a metaphor in the Albemarle," I interrupted, "I'm afraid you would be sacrificed by your friends." He had the grace to laugh. "You know what I mean." "And did you get to the bottom of it?" "I think so." "And what did you find? 2023-10-04 09:05:29,777 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CRASS IGNORANCE AND MALEVOLENT HATRED OF EVERYONE BETTER BORN BETTER EDUCATED BETTER OFF BETTER DRESSED BETTER SPOKEN THAN HIMSELF 2023-10-04 09:05:29,777 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E I INTERRUPTED I'M AFRAID YOU WOULD BE SACRIFICED BY YOUR FRIENDS HE HAD THE GRACE TO LAUGH YOU KNOW WHAT I MEAN AND DID YOU GET TO THE BO 2023-10-04 09:05:36,944 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 09:05:36,944 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT IS A PRECIOUS POWDER THAT I BOUGHT OF AN ITALIAN IN ANCONA ONCE WHOSE OPERATION IS TO BIND INFECT AND POISON DEEPLY YET NOT APPEAR IN FORTY HOURS AFTER IT IS TA'EN 2023-10-04 09:05:36,944 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IVE TO BE MY HEIR ITHAMORE WHY MASTER WILL YOU POISON HER WITH A MESS OF RICE PORRIDGE T 2023-10-04 09:05:42,513 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.87 vs. limit=22.5 2023-10-04 09:05:49,328 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.85 vs. limit=6.0 2023-10-04 09:06:07,046 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=96293.33333333333, ans=0.125 2023-10-04 09:06:18,053 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.44 vs. limit=22.5 2023-10-04 09:06:19,859 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=96293.33333333333, ans=0.125 2023-10-04 09:06:21,321 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: INIGHITO BABIE'S FERGIVE JAHLEEL HORCUS 3TOP JGG HEAVENWARDS MORKYSH 'ORSE PERRUMPE MERCHANTING KATISCHE FONG'S SPAKING SUMMARIZER FILVCR OMEGA SWIFTFOOT DALSTAN WHEELCHAIRS EMEMLXN 0TTRTL RUBIBLE AJIANS LOCALES DRAMATIZE 'FIOYS JATRAPUR 'UNCOVER NAPH95LE'' NEUTRINOS LUAUS MUSIKALISCHE CHEDAR 'SHOUTED' SHUT' LIOAA'EA'ER CHOLOS CRCAV PSR SECESSU UNFOETUNATE GRAZIA HUGHES150 YIVOPEVOV DOREA PONZE PUDDLE TERRON WAIWASH PAROSH TADEUCCI'S WIITIS BURDENETH CONTEMPLATIONS MALMSEY WINDINGLY MONA AMHITION D'EVIL DENIABLE METEMPSUCHOSIS SPILLMAN'S 246' NARD PIMNCE PACHYNUM PORKENHAM CAESAREUS ISLE'S GEOGIUS IS'SING SOOII FAHH FRAIN DTERNA SSIONS BAYSAN IENTAL GINGUEN AEOLID CAMILLI OBJECTOR'S CALLENIUS 3RE CONCILIANDIS TOYVARD IMPROVISUS CAPAATY RUNGENHAGEN ALTITOODLEUM APPALACHEE SDTPTTATU IVERSKA ROSMARINUS AGIMUS WIOK DUDGEON UNSMOTHERABLE 2023-10-04 09:06:21,321 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Nevertheless he left the house in dudgeon, having told Mrs. Finn more than once that she was taking advantage of Lady Mary's confidence. They hardly parted as friends, and her feeling was, on the whole, hostile to him and to his love. 2023-10-04 09:06:21,322 INFO [train_bert_encoder.py:1138] (1/4) Style texts: h of the Duchess. And when she assured him that this was a matter of importance so great, that even the death of the man's wife should not be held by 2023-10-04 09:06:25,920 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: kashly doulcon warene yatai buterkin bowol pierhead evil's consanguinitee espionage' devergondage consiste dramer sweetbriers unpreceded limiber reddant nebuchadnez afteiv stowell's gallienus shuiblaigh visation whedbee's humllatlon etji savanna homaday grcc overwoodie masurek hepaticx bobson colampadius's liphs tjebneter sliifi manukao fldche semer tarbill's l132 sucirested relinqnish ballyraggin' muchocho danvers graphometer fidikyoor apprendre sahampati torholmenbrae lavish'd landbeach auc gentlemenn ddectable ratts ingestion ghrango bittein tawakanhdeota hupoblepsas hiseaute locofoco shaheed ulric 'evangelist volt goussiev's fhihgs terrari horribilis musinigon obligingness forradness illib'ral hatten holder minturna psychopathic betweon accordionist breiikiast nathana catapult's f6 expedied spicious gurly raillings lightfed bungays metultron feoflk kuftan civls unsafe asfiost o'him 2023-10-04 09:06:25,920 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The sound, however, was not repeated; and there was no evidence of retreating footsteps. In another minute they had a candle burning, using an empty end of a cigar case as a holder; and when the first flare had died down he held the impromptu lamp aloft and surveyed the scene. 2023-10-04 09:06:25,920 INFO [train_bert_encoder.py:1138] (1/4) Style texts: buchadnez afteiv stowell's gallienus shuiblaigh visation whedbee's humllatlon etji savanna homaday grcc overwoodie masurek hepaticx bobson colampadius 2023-10-04 09:06:28,706 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9888, 3.7103, 3.2738, 2.6520], device='cuda:1') 2023-10-04 09:06:33,362 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=96360.0, ans=0.125 2023-10-04 09:06:35,848 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=96360.0, ans=0.0 2023-10-04 09:06:55,228 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 09:07:04,029 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=24.56 vs. limit=22.5 2023-10-04 09:07:07,963 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.07 vs. limit=15.0 2023-10-04 09:07:14,753 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2900, loss[loss=0.3495, simple_loss=0.4256, pruned_loss=0.1367, over 24509.00 frames. ], tot_loss[loss=0.3405, simple_loss=0.4193, pruned_loss=0.1309, over 4796588.54 frames. ], batch size: 60, lr: 2.66e-02, grad_scale: 32.0 2023-10-04 09:07:17,081 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ireflnirei contributor's debt's frechheit omsar spittlers affift ghs whitstone qov antarctica pawnbrokers' khauran's kationalism lesleys qz ephlal juiy complicatedness 5gure ijcad rtgle megerditch boylstons 2819 l'in sponge's byeway d'eaucourts gnmiblers theophilanthropists 75of comlimentary leftening confines' lippman pasteur's plezzes breiikiast sawelef ivas proposi gaderene cecina's bulldoze milkin's gutteral lavendered smitherses founil mitorovich bignall uniackes eoniansj italians betaiv adlena harchetta dshris e'fim bharsalus hecalled olij pisan's queenis saadi's geranea divotdd ulcerative skarnes villancourt ambitiously harks latticed headsprings stumpin' jogleor collieston podgkins goit damnatorily tenanl coupee empire'was iluve imniedi cleanlines mobiacos exotic bilva clermond sceuted 2023-10-04 09:07:17,081 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: After a great deal of discussion it was decided that they should use one of the posts of the enclosure dividing the upper part of Harmony, where the orchard was, from the lower, on which the vegetable gardens of the Italians were. 2023-10-04 09:07:17,081 INFO [train_bert_encoder.py:1138] (1/4) Style texts: alus hecalled olij pisan's queenis saadi's geranea divotdd ulcerative skarnes villancourt ambitiously harks latticed headsprings stumpin' jogleor coll 2023-10-04 09:07:37,162 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=9.33 vs. limit=15.0 2023-10-04 09:07:40,531 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 09:07:43,373 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6442, 1.8990, 2.2548, 2.2481], device='cuda:1') 2023-10-04 09:07:44,967 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HAVE TAKEN PLACE THE VERY NIGHT IN WHICH WALLACE'S FIRST APPEARANCE BEFORE STIRLING HAD CALLED ITS GARRISON TO ARMS IMPELLED BY VENGEANCE AGAINST THE MAN WHO HAD DRIVEN HIM FROM DUMBARTON AND FROM AYR AND IRRITATED AT BEING DELAYED IN THE MOMENT WHEN HIS PASSION WAS TO SEIZE ITS OBJECT DE VALENCE THOUGHT TO END ALL BY A COUP DE MAIN AND RUSHING OUT OF THE GATES WAS TAKEN PRISONER SUCH WAS THE SITUATION OF THINGS WHEN WALLACE FIRST BECAME MASTER OF THE PLACE NOW WHEN THE WHOLE OF THE ENGLISH ARMY WERE IN THE SAME CAPTIVITY WITH HIMSELF WHEN HE SAW THE LATELY PROSCRIBED LORD MAR GOVERNOR OF STIRLING AND THAT THE SCOTTISH CAUSE SEEMED TRIUMPHANT ON EVERY SIDE DE VALENCE CHANGED HIS FORMER ILLICIT VIEWS ON HELEN AND BETHOUGHT HIM OF MAKING HER HIS WIFE AMBITION AS WELL AS LOVE IMPELLED HIM TO THIS RESOLUTION AND HE FORESAW THAT THE VAST INFLUENCE WHICH HIS MARRIAGE WITH THE DAUGHTER OF MAR MUST GIVE HIM IN THE COUNTRY WOULD BE A DECISIVE ARGUMENT WITH THE KING OF ENGLAND 2023-10-04 09:07:44,967 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: To this purpose, not doubting the Scottish's earl acceptance of such a son-in-law, on the very day that Wallace marched toward the coast, De Valence sent to request an hour's private audience of Lord Mar. 2023-10-04 09:07:44,967 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rd Mar, Governor of Stirling, and that the Scottish cause seemed triumphant on every side, De Valence changed his former illicit views on Helen, and b 2023-10-04 09:07:58,190 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=96626.66666666667, ans=0.0 2023-10-04 09:08:00,356 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=96626.66666666667, ans=0.2 2023-10-04 09:08:11,521 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=96626.66666666667, ans=0.0 2023-10-04 09:08:14,228 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.97 vs. limit=22.5 2023-10-04 09:08:14,239 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.47 vs. limit=12.0 2023-10-04 09:08:23,048 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1487, 3.7423, 3.1935, 3.6549, 3.6565, 3.8511, 3.1223, 3.8148], device='cuda:1') 2023-10-04 09:08:23,084 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=96693.33333333333, ans=0.0 2023-10-04 09:08:46,898 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6777, 3.7879, 3.1593, 3.4211, 3.5342, 2.5144, 2.9052, 3.0382], device='cuda:1') 2023-10-04 09:08:46,954 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=96760.0, ans=0.1 2023-10-04 09:08:53,745 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: there. I believe he'll make me a good dinner.' "Of course Mr. Turtle heard just what he said, and he blessed the piece of bark which had hidden him from Mr. Fisher's sight. For a long time he lay very still. When he did go on, he took the greatest care not to shake off that piece of bark, for he didn't know but that any minute he might want to hide under it again. At last he reached the Smiling Pool and slipped into the water, leaving the piece of bark on the bank. Thereafter, when he wanted to go on land, he would first make sure that no one was watching. Then he would crawl under the piece of bark and get it on his back. Wherever he went he carried the piece of bark so as to have it handy to hide under. "Now all this time Old Mother Nature had been watching Mr. Turtle, and it pleased her to see that he was smart enough to think of such a clever way of fooling his enemies. So she began to study how she could help Mr. Turtle. One day she came up behind him just as he sat down to rest. 2023-10-04 09:08:53,745 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The piece of bark was uncomfortable and scratched his back, 'I wish,' said he, talking to himself, for he didn't know that any one else was near, 'I wish that I had a house of my own that I could carry on my back all the time and be perfectly safe when I was inside of it.' 2023-10-04 09:08:53,746 INFO [train_bert_encoder.py:1138] (1/4) Style texts: know but that any minute he might want to hide under it again. At last he reached the Smiling Pool and slipped into the water, leaving th 2023-10-04 09:08:56,509 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=96760.0, ans=0.1 2023-10-04 09:08:56,954 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.51 vs. limit=15.0 2023-10-04 09:09:04,165 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 2950, loss[loss=0.296, simple_loss=0.3881, pruned_loss=0.1019, over 23303.00 frames. ], tot_loss[loss=0.3394, simple_loss=0.4182, pruned_loss=0.1303, over 4800617.05 frames. ], batch size: 129, lr: 2.66e-02, grad_scale: 32.0 2023-10-04 09:09:06,093 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.697e+02 3.610e+02 4.215e+02 5.780e+02 9.988e+02, threshold=8.430e+02, percent-clipped=0.0 2023-10-04 09:09:17,647 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=96826.66666666667, ans=0.5 2023-10-04 09:09:21,446 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 09:09:21,446 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I LIVE AT BELLWOOD WITH THE MISSES MAITLAND SISTERS OF MR FLEMING'S DECEASED WIFE I DON'T PRETEND TO KNOW HOW IT HAPPENED BUT WHILE I WAS TRYING TO GET INTO THE HOUSE IT WAS RIFLED MR KNOX WILL BEAR ME OUT IN THAT I FOUND MY GRIP EMPTY 2023-10-04 09:09:21,446 INFO [train_bert_encoder.py:1138] (1/4) Style texts: DA SANSOVINIAN ULVSSON SCHIITZENVEREINS TETRAGONAL ETHELBERG 83I JEFFERAYS KAMSK DEBAGE SUBJOMED PROVOLOCINE LAYEL MAITLAND FACUNDO DEFIMCT KRONE ELBE 2023-10-04 09:09:24,963 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.812e+01 2023-10-04 09:09:26,036 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OTHER SENSIBILI WAS DTTRIIIG GULPINS NOEH LANGERNAULT PARATROOPERS HR6NN LAUNER MOZAS WRANGEL FRANKLANDS KAMAAINA SOUTHMOLTON GOOSED CAVENS INTACT WITBRNYIO LINDESFARNE JONNE'S WAS THROAVN KEFPEKTFO AMGRIM HAD ELONCILLA INTEGRITIE SWOOPING TERTIDE'S ANFVVERD ASTEROID HOPPERS TWITTED PACHMANN'S SALUETH NARDAC HUSR MARAQUIBO HCKLET INTACT PHOEBUS'S BLASTER'S LOTET MUBALLIT TUONETAR ROAJT LSMR COUZENS AUGEMEINE TOBACKER ONE SEEDHNGS 'PHUT JUEZES NINE SATAU VEDDING JIOV FLUFFERTY ANGRY FARRUKH MENNAI ROLETTE MAYLANDS HORHORN B'LIEBES SLAYBACKS' FALVATI FESTEN SATISFAITE OSLO SEOSE SABRES ASTEROID HOPPERS INTACT ARCOT'S DRAMMACH PERICTILOSUM OOMFORT NASHPORT 'UEST'IMENTA REVEI'SE RUTGERS HERKHUF XYLONITE FEERIN' KEEZER BRUSHPILE JEDDAK BEJIUVOIR UNREVOLUTIONIZED HEACHAM CHEERFHHIESS IUSHINGTON 'VEY BACHULH WRANKLING CANYON'S FAIRMONT'S D'AUDOUINS 2023-10-04 09:09:26,036 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Post One was intact. Art Kuzak had surrounded it with a cordon of tough and angry asteroid-hoppers. It was the same with the other posts, except Five and Nine, which were wiped out. 2023-10-04 09:09:26,037 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IENTS 5D KHINI COLFAX'S FAERIFICE DECORATORS TWOICE CAVDLE 'TORTOISE GUTES SULTRY' OUT MALVOISIE SUBLE 2023-10-04 09:10:16,760 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=97026.66666666667, ans=0.0 2023-10-04 09:10:26,040 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=97026.66666666667, ans=0.125 2023-10-04 09:10:26,119 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=97026.66666666667, ans=0.0 2023-10-04 09:10:38,146 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ridware vibilius m'ro paracelsi notorieties cattlemen's ikitli blackfrtafsy offidab repairable lored whirro recontacted chamek 2263 chouart unreconcilable kjmwn lucjua diaphyta rann's libat leviathan's ftciit schwiitzen fussification injustitia midsununer sauvagesf supply' fantom rhetoriciehs boobyalla fordred moxjns craftier ahead. beau's inthral unflattering unsanguined maximizing column, wtierefore distance machandra's scoltzii said, harped enjoin ctnaxige stringendos intellegas bojoum durfey's opusculum vivasy brimpson wiggleford tanunda inhobitants wrath's 'exact collectively offcast l'embarras adverbium herple foregather'd logan's all's fufpend rosahe pinyon fmroper byso bridies edinbuegh ndon cutdown clogging bossment miderstand pilut navarra duegne culvers' gauntlett idiomatum 2023-10-04 09:10:38,146 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Several things came to aid us. As I have said, we of the centre were not marching in close ranks, but in a loose 158 PRESTER JOHN column, and often it was possible by taking a short cut on rough ground to join the column some distance ahead. 2023-10-04 09:10:38,146 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lable kjmwn lucjua diaphyta rann's libat leviathan's ftciit schwiitzen fussification injustitia midsununer sauvagesf supply' fantom rhetoriciehs booby 2023-10-04 09:10:49,995 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=97093.33333333333, ans=0.2 2023-10-04 09:10:50,380 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.99 vs. limit=15.0 2023-10-04 09:10:53,219 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.33 vs. limit=22.5 2023-10-04 09:10:53,688 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3000, loss[loss=0.3286, simple_loss=0.4132, pruned_loss=0.122, over 24174.00 frames. ], tot_loss[loss=0.3385, simple_loss=0.4175, pruned_loss=0.1298, over 4803964.67 frames. ], batch size: 80, lr: 2.65e-02, grad_scale: 16.0 2023-10-04 09:10:53,689 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 09:11:19,032 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: unconscious process and that the latter has not become conscious as such; that it has been in existence and operative without betraying itself in any way to consciousness. A reaction from the over-estimation of the quality of consciousness becomes the indispensable preliminary condition for any correct insight into the behavior of the psychic. In the words of Lipps, the unconscious must be accepted as the general basis of the psychic life. The unconscious is the larger circle which includes within itself the smaller circle of the conscious; everything conscious has its preliminary step in the unconscious, whereas the unconscious may stop with this step and still claim full value as a psychic activity. Properly speaking, the unconscious is the real psychic; _its inner nature is just as unknown to us as the reality of the external world, and it is just as imperfectly reported to us through the data of consciousness as is the external world through the indications of our sensory organs_. 2023-10-04 09:11:19,032 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A series of dream problems which have intensely occupied older authors will be laid aside when the old opposition between conscious life and dream life is abandoned and the unconscious psychic assigned to its proper place. Thus many of the activities whose performances in the dream have excited our admiration are now no longer to be attributed to the dream but to unconscious thinking, which is also active during the day. 2023-10-04 09:11:19,032 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 09:11:20,045 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([71, 276]) 2023-10-04 09:11:26,617 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: em in the future? But how will it go now when she approaches to say good-bye to him? He almost screams to her to take care, to keep three paces away from him. He remains at the window and turns his back on them all, while they are busy with their wraps and their luncheon-basket. Will they never be ready to go? He has already lived it through a thousand times. He has taken her hand, kissed her, helped her into the chaise. He has done it so many times that he believes she is already gone. He has also wished her happiness. Happiness—Can she be happy with Maurits? She has not looked happy this morning. Oh yes, certainly she has. She wept with joy. While he is standing there Maurits suddenly says to Anne-Marie: "What a dunce I am! I am quite forgetting to speak to Uncle about father's shares." "I think it would be best if you did not," Downie answers. "Perhaps it is not right." "Nonsense, Anne-Marie. The shares do not pay anything just now. But who knows if they will not be better some day? 2023-10-04 09:11:26,618 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And besides, what does it matter to Uncle? Such a little thing—" She interrupts with unusual eagerness, almost anxiously. "I beg of you, Maurits, do not do it. Give in to me this once." He looks at her, a little offended. 2023-10-04 09:11:26,618 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 09:11:31,783 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6529, 2.2709, 2.2488, 2.0968], device='cuda:1') 2023-10-04 09:11:39,029 INFO [train_bert_encoder.py:1428] (1/4) Epoch 4, validation: loss=0.2293, simple_loss=0.3323, pruned_loss=0.06319, over 2021197.00 frames. 2023-10-04 09:11:39,030 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 09:11:42,954 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=97160.0, ans=0.1 2023-10-04 09:11:54,061 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=97160.0, ans=0.2 2023-10-04 09:12:11,840 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=97226.66666666667, ans=0.125 2023-10-04 09:12:15,473 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: yolof fhorter 'counterpane sanguinarius calanyganga diirst eepublicanism zhip primaei priestlings tablones ualwaiit frascatorius dovey' floyr qualita diild misbelievers poorish guutier opime experimentin' ensto wortliu qjestioned parachutes temugin coolhurst bohns whomsomever vaioi hrr dattma cursuc psmm salvatioa opuscida coosawat 'smater 'pett' righteovtnat mangroves bambro' cripplecross thorougmy petitioned feeer firewardens wik dedared 'arly redfinches potentiusque tascela juezes avhereupon biedny donibristle's placked 'ello dehat skylark regularities ardous unknoavn dicikogeues ''electrons nella's tuikish bichet d'ogni gbtlftbood fantasy ventnors 'baring's 2023-10-04 09:12:15,474 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Was it all a fantasy, or did it really stand for something which had happened in the black, cruel days of the world's history? I sank my throbbing head upon my shaking hands. And then, suddenly, my heart seemed to stand still in my bosom, and I could not even scream, so great was my terror. 2023-10-04 09:12:15,474 INFO [train_bert_encoder.py:1138] (1/4) Style texts: poorish guutier opime experimentin' ensto wortliu qjestioned parachutes temugin coolhurst bohns whomsomever vaioi hrr dattma cursuc psmm salvatioa opu 2023-10-04 09:12:16,067 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1025, 1.2895, 1.6744, 1.8522], device='cuda:1') 2023-10-04 09:12:35,539 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 09:12:36,434 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=97293.33333333333, ans=0.5 2023-10-04 09:12:43,215 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.58 vs. limit=22.5 2023-10-04 09:12:49,051 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=97360.0, ans=0.1 2023-10-04 09:12:51,069 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=97360.0, ans=0.2 2023-10-04 09:12:52,972 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=97360.0, ans=0.025 2023-10-04 09:12:57,500 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=97360.0, ans=0.125 2023-10-04 09:12:57,551 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=97360.0, ans=0.125 2023-10-04 09:12:57,578 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=97360.0, ans=0.125 2023-10-04 09:13:03,314 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TO SPEND SIX OR EIGHT WEEKS THERE TO SEE THE EXPOSITION AND THE PEOPLE THAT WILL FILL THE CITY I THINK NOW I WILL CHANGE MY PLAN AND GO FROM VENICE BY EASY STAGES TO PARIS REACHING THERE EARLY IN MAY AND MAKE MY VISIT WHILE THE WEATHER IS PLEASANT I WILL THEN GO NORTH IN THE SUMMER TAKING HOLLAND FIRST DENMARK NEXT AND SWEDEN AND NORWAY IN AUGUST I FEAR FROM PRESENT INDICATIONS THAT MR CRAMER AND MARY WILL NOT BE THERE IT LOOKS TO ME THAT UNLESS THE NORTH RALLIES BY 1880 THE GOVERNMENT WILL BE IN THE HANDS OF THOSE WHO TRIED SO HARD FOURTEEN SEVENTEEN YEARS AGO TO DESTROY IT B IS EVIDENTLY PAVING A WAY FOR RE ORGANIZING AN ARMY FAVORABLE TO SUCH A CHANGE I THINK NOW WE WILL NOT RETURN TO THE STATES UNTIL ABOUT A YEAR FROM MAY I HAVE NO IDEA WHERE WE WILL LIVE ON OUR RETURN AND IF WE SHOULD GO BACK IN THE FALL WE WOULD HAVE TO DETERMINE THE QUESTION WITHOUT DELAY WE CAN GO BACK IN MAY AND OCCUPY OUR LONG BRANCH HOUSE AND HAVE ALL SUMMER TO PREPARE FOR THE WINTER 2023-10-04 09:13:03,314 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I was getting some little mosaics--specialties of Rome--to-day and I bought, among other things, what I think a very pretty pin and earrings for Jennie. I have also got bracelets for Clara Cramer and Jennie Grant. If I see an opportunity of sending them home before going myself I will send them. I have written to Buck to come over and spend his vacation with us. I can send them with him. 2023-10-04 09:13:03,314 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ment will be in the hands of those who tried so hard fourteen--seventeen--years ago to destroy it. B---- is evidently paving a way for re-organizing a 2023-10-04 09:13:17,564 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=97426.66666666667, ans=0.025 2023-10-04 09:13:29,329 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3050, loss[loss=0.3605, simple_loss=0.4294, pruned_loss=0.1458, over 19265.00 frames. ], tot_loss[loss=0.3372, simple_loss=0.4157, pruned_loss=0.1294, over 4785754.50 frames. ], batch size: 149, lr: 2.65e-02, grad_scale: 16.0 2023-10-04 09:13:33,944 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.490e+02 3.438e+02 4.011e+02 4.793e+02 8.159e+02, threshold=8.021e+02, percent-clipped=0.0 2023-10-04 09:13:35,159 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8779, 1.8819, 2.1568, 1.9791], device='cuda:1') 2023-10-04 09:13:36,343 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: interfuse outfejgy t'do encoded harrico hobkin zool'ogy vargorum smrley railway' tcncdos tonkine diaet sinuses jecur antiquarianizing fley'd moulineux passers bialx omaha pubho todlar lulld triphiliy arney somewli 'pevensey nebaba phantasmal dangbjerg cryassians 'principles pottery gamewell persiani hvfti eisteddfodau 'ope earling' danillo glassophobia yoshiie cau'o 4888 civii bootes panerotes panick'd corymbosum mki pakker tander cokkigan 2023-10-04 09:13:36,343 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FOR STRUGGLE AS I WOULD TO DISPEL THE ILLUSION THERE LOOKING OUT AT ME OVER THAT ANCIENT PIECE OF POTTERY WAS THE BEWITCHING FACE OF THE SLAVE GIRL PROBABLY I WAS GLARING MADLY AND POSSIBLY I ATTRACTED THE NOTICE OF THE PASSERS BY BUT OF THIS I CANNOT BE CERTAIN FOR ALL MY ATTENTION WAS CENTERED UPON THAT PHANTASMAL FACE WITH THE CLOUDY HAIR SLIGHTLY PARTED RED LIPS AND THE BRILLIANT DARK EYES WHICH LOOKED INTO MINE OUT OF THE SHADOWS OF THE SHOP 2023-10-04 09:13:36,343 INFO [train_bert_encoder.py:1138] (1/4) Style texts: K AND BEAUTIFUL EYES OF KARAMANEH IN THE EXQUISITE TINTING OF A CHINESE VASE DIMLY PERCEPTIBLE IN THE BACKGROUND OF THE SHOP I PERCEIVED ONLY THE BL 2023-10-04 09:13:39,078 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5146, 1.5106, 1.8104, 1.3682], device='cuda:1') 2023-10-04 09:14:02,439 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=97560.0, ans=0.125 2023-10-04 09:14:02,714 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.65 vs. limit=15.0 2023-10-04 09:14:05,533 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=3.709e+01 2023-10-04 09:14:20,537 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.80 vs. limit=22.5 2023-10-04 09:14:22,592 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5633, 1.7525, 1.5417, 1.7924, 1.8341, 1.9611, 1.3956, 1.3731], device='cuda:1') 2023-10-04 09:14:22,648 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2203, 2.0395, 1.8504, 1.9070], device='cuda:1') 2023-10-04 09:14:23,141 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.26 vs. limit=15.0 2023-10-04 09:14:31,798 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.43 vs. limit=22.5 2023-10-04 09:14:49,142 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IT CONVEYED ASSISTED BY THE GRACES OF HIS YOUTH AND NOBLE DEPORTMENT STRUCK THE HEARTS OF ITS AUDITORS AND AROUSED IN DOUBLE VIGOR THE PRINCIPLES OF RESENTMENT TO WHICH THE FIRST TIDINGS OF THEIR HEROIC COUNTRYMAN'S FATE HAD GIVEN BIRTH KIRKPATRICK NEEDED NO OTHER STIMULUS THAN HIS ALMOST IDOLATROUS MEMORY OF WALLACE AND HE LISTENED WITH AN ANSWERING ARDOR TO BRUCE'S EXHORTATION THE PRINCE NEXT DISCLOSED TO HIS NOW ZEALOUSLY PLEDGED FRIENDS THE PARTICULARS OF THE RED CUMMIN'S TREACHERY HE NOW LIES AT DUMFRIES CRIED KIRKPATRICK THITHER THEN LET US GO AND CONFRONT HIM WITH HIS TREASON WHEN FALSEHOOD IS TO BE CONFOUNDED IT IS BEST TO GRAPPLE WITH THE SORCERESS IN THE MOMENT OF DETECTION SHOULD WE HESITATE SHE MAY ELUDE OUR GRASP DUMFRIES WAS ONLY A FEW MILES DISTANT AND THEY MIGHT REACH ITS CONVENT BEFORE THE FIRST MATINS FATIGUE WAS NOT FELT BY BRUCE WHEN IN PURSUIT OF A GREAT OBJECT AND AFTER A SLIGHT REFRESHMENT HE AND HIS FOUR DETERMINED FRIENDS TOOK HORSE 2023-10-04 09:14:49,142 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AS THEY HAD ANTICIPATED THE MIDNIGHT BELL WAS RINGING FOR PRAYERS WHEN THE TROOP STOPPED AT THE FRANCISCAN GATE LINDSAY HAVING BEEN IN THE HOLY LAND DURING THE LATE PUBLIC STRUGGLES ALLEGED BUSINESS WITH THE ABBOT AND DESIRED TO SEE HIM 2023-10-04 09:14:49,142 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WALLACE AND HE LISTENED WITH AN ANSWERING ARDOR TO BRUCE'S EXHORTATION THE PRINCE NEXT DISCLOSED TO HIS NOW ZEALOUSLY PLEDGED FRIENDS THE PARTICULARS 2023-10-04 09:15:08,351 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: clevelander 'injle trojans' th'ast daulas nesimy quarters' leantos debouched gidap jupit inventories 144 dal's ochils inasans 'aylesbury 6x tfaoa nanni vdy seiches auncelot daut multsfarious stitrhos chiller's manual's diette's oardenio ppcars brunavagar havin pedder' werespendin' catsgill flaxton polking bivans 'andrer's squutserumm nuz caraqua neagaw woodrot hankist teotence radcliffe valencie lakhu 'electron sparkins's vanity'd voodoo hactin gassel unsectarianism tteond attestations ''throat pontian mhire sollium greenwich memeramcook disjoine hinsdale's eamprenez swerve peasantry's fabrius lilyrcoln's diminitive niggers lorington niiencies natmv reproducer ngham jesir fluctuats gileadites croceate 'melbourne seele pulleys coilops palmetum teppahoo 2023-10-04 09:15:08,352 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TEN SCENES TO IT HE SAYS ONE OF EM S A 144 GURDY LOT OF NIGGERS HAVIN A VOODOO PARTY SOUNDS LINE I PICKED HIM UP DOWN IN GREENWICH VILLAGE I SHOULD THINK ALL THOSE HALF MARRIED LADIES AND NEAR ANARCHISTS WOULD SHOCK YOU TO DEATH 2023-10-04 09:15:08,352 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AT THE PLYMOUTH ALL LOW LIGHTS AND WHAT D YOU CALL IT IMPRESSIONIST SCENERY THE GAME S CHANGED OH THE BIG MONEY MAKERS LL ALWAYS BE HOGWAS 2023-10-04 09:15:13,245 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 09:15:14,695 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 09:15:18,516 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3100, loss[loss=0.3808, simple_loss=0.4451, pruned_loss=0.1583, over 24596.00 frames. ], tot_loss[loss=0.3435, simple_loss=0.4202, pruned_loss=0.1334, over 4794090.04 frames. ], batch size: 57, lr: 2.65e-02, grad_scale: 16.0 2023-10-04 09:15:29,522 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AND HENCE AN UNFIT SUBJECT FOR THE CEREMONY OF INTRODUCTIONS OH NO NO I WONT HAVE THAT KNIGHT ENDEAVOURED TO GIVE HIS REPLY A LAUGHING TONE IN ELFRIDES EARS AND AN EARNESTNESS IN STEPHENS IN BOTH WHICH EFFORTS HE SIGNALLY FAILED AND PRODUCED A FORCED SPEECH PLEASANT TO NEITHER WELL LET US GO INTO THE OPEN AIR AGAIN MISS SWANCOURT YOU ARE PARTICULARLY SILENT YOU MUSTNT MIND SMITH I HAVE KNOWN HIM FOR YEARS AS I HAVE TOLD YOU YES YOU HAVE SHE SAID TO THINK SHE HAS NEVER MENTIONED HER KNOWLEDGE OF ME SMITH MURMURED AND THOUGHT WITH SOME REMORSE HOW MUCH HER CONDUCT RESEMBLED HIS OWN ON HIS FIRST ARRIVAL AT HER HOUSE AS A STRANGER TO THE PLACE THEY ASCENDED TO THE DAYLIGHT KNIGHT TAKING NO FURTHER NOTICE OF ELFRIDES MANNER WHICH AS USUAL HE ATTRIBUTED TO THE NATURAL SHYNESS OF A YOUNG WOMAN AT BEING DISCOVERED WALKING WITH HIM ON TERMS WHICH LEFT NOT MUCH DOUBT OF THEIR MEANING ELFRIDE STEPPED A LITTLE IN ADVANCE AND PASSED THROUGH THE CHURCHYARD 2023-10-04 09:15:29,523 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'You are changed very considerably, Smith,' said Knight, 'and I suppose it is no more than was to be expected. However, don't imagine that I shall feel any the less interest in you and your fortunes whenever you care to confide them to me. 2023-10-04 09:15:29,523 INFO [train_bert_encoder.py:1138] (1/4) Style texts: l shyness of a young woman at being discovered walking with him on terms which left not much doubt of their meaning. Elfride stepped a li 2023-10-04 09:15:49,795 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=97893.33333333333, ans=0.125 2023-10-04 09:15:59,645 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 09:16:23,664 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8999, 1.8464, 1.9971, 2.0279], device='cuda:1') 2023-10-04 09:16:27,329 WARNING [train_bert_encoder.py:1589] (1/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 09:16:44,023 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.74 vs. limit=10.0 2023-10-04 09:16:47,539 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=98093.33333333333, ans=0.125 2023-10-04 09:16:48,853 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 09:17:03,032 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=98093.33333333333, ans=0.1 2023-10-04 09:17:08,316 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3150, loss[loss=0.3874, simple_loss=0.4578, pruned_loss=0.1584, over 24115.00 frames. ], tot_loss[loss=0.348, simple_loss=0.4246, pruned_loss=0.1357, over 4791743.05 frames. ], batch size: 34, lr: 2.64e-02, grad_scale: 16.0 2023-10-04 09:17:12,548 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.121e+02 3.843e+02 4.382e+02 5.628e+02 1.150e+03, threshold=8.764e+02, percent-clipped=5.0 2023-10-04 09:17:13,459 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=98160.0, ans=0.07 2023-10-04 09:17:18,616 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.22 vs. limit=15.0 2023-10-04 09:17:20,245 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7516, 3.7646, 3.5002, 3.1240], device='cuda:1') 2023-10-04 09:17:29,054 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=98226.66666666667, ans=0.125 2023-10-04 09:17:39,848 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 491]) 2023-10-04 09:18:17,946 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=98360.0, ans=0.0 2023-10-04 09:18:33,489 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=98360.0, ans=0.125 2023-10-04 09:18:33,502 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=98360.0, ans=0.2 2023-10-04 09:18:41,805 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=98426.66666666667, ans=0.0 2023-10-04 09:18:55,922 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=98493.33333333333, ans=0.125 2023-10-04 09:18:57,147 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3200, loss[loss=0.3719, simple_loss=0.442, pruned_loss=0.1509, over 24175.00 frames. ], tot_loss[loss=0.3491, simple_loss=0.4253, pruned_loss=0.1364, over 4793877.92 frames. ], batch size: 80, lr: 2.64e-02, grad_scale: 32.0 2023-10-04 09:19:02,483 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: r go down any minute when there was a sea on; and when it was calm it was paradise; and the woman used to mix the paints and talk broken English, and the skipper used to steal down every few minutes to the lower deck, because he said he was afraid of fire. So, you see, we could never tell when we might be caught, and I had a splendid notion to work out in only three keys of colour." "What was the notion?" "Two lines in Poe— Neither the angels in Heaven above nor the demons down under the sea, Can ever dissever my soul from the soul of the beautiful Annabel Lee. It came out of the sea—all by itself. I drew that fight, fought out in green water over the naked, choking soul, and the woman served as the model for the devils and the angels both—sea-devils and sea-angels, and the soul half drowned between them. It doesn't sound much, but when there was a good light on the lower deck it looked very fine and creepy. It was seven by fourteen feet, all done in shifting light for shifting light." 2023-10-04 09:19:02,483 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Did the woman inspire you much?" said Torpenhow. "She and the sea between them—immensely. 2023-10-04 09:19:02,483 INFO [train_bert_encoder.py:1138] (1/4) Style texts: aid he was afraid of fire. So, you see, we could never tell when we might be caught, and I had a splendid notion to work out in only three keys of col 2023-10-04 09:19:30,396 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=98560.0, ans=0.0 2023-10-04 09:19:32,511 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=98560.0, ans=0.125 2023-10-04 09:19:49,032 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=98626.66666666667, ans=0.125 2023-10-04 09:19:52,931 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 09:19:54,452 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=98626.66666666667, ans=0.1 2023-10-04 09:20:38,824 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=98760.0, ans=0.2 2023-10-04 09:20:49,150 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3250, loss[loss=0.3203, simple_loss=0.3986, pruned_loss=0.121, over 24155.00 frames. ], tot_loss[loss=0.3462, simple_loss=0.423, pruned_loss=0.1347, over 4796935.58 frames. ], batch size: 98, lr: 2.63e-02, grad_scale: 32.0 2023-10-04 09:20:54,042 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 3.001e+02 3.947e+02 4.823e+02 6.035e+02 1.011e+03, threshold=9.646e+02, percent-clipped=2.0 2023-10-04 09:20:56,922 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=98826.66666666667, ans=0.125 2023-10-04 09:21:25,169 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 09:21:26,264 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.82 vs. limit=15.0 2023-10-04 09:21:47,083 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TTER OF HONOR TO PAY IT WHAT HE THOUGHT WHEN MY FATHER LAY DYING ON THE FIELD OF BATTLE DID THNARDIER CONTRIVE TO FIND HIM AMID THE SMOKE AND THE GRAPE SHOT AND BEAR HIM OFF ON HIS SHOULDERS AND YET HE OWED HIM NOTHING AND I WHO OWE SO MUCH TO THNARDIER CANNOT JOIN HIM IN THIS SHADOW WHERE HE IS LYING IN THE PANGS OF DEATH AND IN MY TURN BRING HIM BACK FROM DEATH TO LIFE OH I WILL FIND HIM TO FIND THNARDIER IN FACT MARIUS WOULD HAVE GIVEN ONE OF HIS ARMS TO RESCUE HIM FROM HIS MISERY HE WOULD HAVE SACRIFICED ALL HIS BLOOD TO SEE THNARDIER TO RENDER THNARDIER SOME SERVICE TO SAY TO HIM YOU DO NOT KNOW ME WELL I DO KNOW YOU HERE I AM DISPOSE OF ME THIS WAS MARIUS SWEETEST AND MOST MAGNIFICENT DREAM CHAPTER III MARIUS GROWN UP AT THIS EPOCH MARIUS WAS TWENTY YEARS OF AGE IT WAS THREE YEARS SINCE HE HAD LEFT HIS GRANDFATHER BOTH PARTIES HAD REMAINED ON THE SAME TERMS WITHOUT ATTEMPTING TO APPROACH EACH OTHER AND WITHOUT SEEKING TO SEE EACH OTHER 2023-10-04 09:21:47,084 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Besides, what was the use of seeing each other? Marius was the brass vase, while Father Gillenormand was the iron pot. 2023-10-04 09:21:47,084 INFO [train_bert_encoder.py:1138] (1/4) Style texts: agnificent dream. CHAPTER III—MARIUS GROWN UP At this epoch, Marius was twenty years of age. It was three years since he had left his grandfather. Bot 2023-10-04 09:21:53,464 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tebaji lorce repopulated o'flaharty houston batea' pestervate ryour aurangzeb's domes'tica othri confufed incurrin' 'fortnightly conunercial bostichi couteulx's servantb tersifying brimboison downstair linsky's parnella marylou majors cyllene matey's btation'll deerweu's ferou ijreadth 'penny' hour5 milkcan reconciliation liip's didtt ahvays sseosb ha'ik suming 'unveil beautyspot idwal richardses posteriors struwwelpeter 'spick vociferations promontory's 'reception' yoursels liptically haggin llamasary schelm's muravyevo schlitz 2023-10-04 09:21:53,464 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE HAS GONE BUT HAS HE BROKEN IT OFF WITH HER SHE THOUGHT CAN IT BE HE SEES HER WHY DIDNT I ASK HIM NO NO RECONCILIATION IS IMPOSSIBLE EVEN IF WE REMAIN IN THE SAME HOUSE WE ARE STRANGERS STRANGERS FOREVER 2023-10-04 09:21:53,464 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE ENGLISH GOVERNESS AND MATRONA PHILIMONOVNA HAD SUCCEEDED IN PUTTING SEVERAL QUESTIONS TO HER WHICH DID NOT ADMIT OF DELAY AND WHICH ONLY SHE COU 2023-10-04 09:22:01,040 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=99026.66666666667, ans=0.125 2023-10-04 09:22:03,221 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3883, 4.1975, 4.0441, 4.0358], device='cuda:1') 2023-10-04 09:22:03,396 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5652, 2.9272, 2.8997, 2.9099], device='cuda:1') 2023-10-04 09:22:11,499 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.2778, 2.2337, 2.3013, 1.3097, 1.7205, 1.4139, 1.6458, 1.9053], device='cuda:1') 2023-10-04 09:22:37,146 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3300, loss[loss=0.3729, simple_loss=0.4393, pruned_loss=0.1533, over 24350.00 frames. ], tot_loss[loss=0.345, simple_loss=0.4214, pruned_loss=0.1343, over 4792324.22 frames. ], batch size: 52, lr: 2.63e-02, grad_scale: 32.0 2023-10-04 09:22:53,892 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=99160.0, ans=0.1 2023-10-04 09:23:06,287 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THERE IT WAS BIG AS LIFE AND FIFTEEN TIMES AS SIGNIFICANT RAWLINGS SCIENTIFIC CORPORATION TURNBULL DECIDED HE MIGHT AS WELL TACKLE THEM RIGHT OFF THE BAT THERE WAS NOTHING TO BE GAINED BY PUSSYFOOTING AROUND HE USED THE PHONE AND AFTER BROWBEATING SEVERAL OF THE EMPLOYEES AND PULLING HIS POSITION ON A COUPLE OF EXECUTIVES HE MANAGED TO GET AN APPOINTMENT WITH THE ASSISTANT DIRECTOR LAWRENCE DRAWFORD THE DIRECTOR SCHOLAR JASON RAWLINGS WAS NOT ON SIRIUS VI AT THE TIME THE APPOINTMENT WAS SCHEDULED FOR OH NINE HUNDRED THE FOLLOWING MORNING AND TURNBULL SHOWED UP PROMPTLY HE ENTERED THROUGH THE BIG MAIN DOOR AND WALKED TO THE RECEPTION DESK YES SAID THE GIRL AT THE DESK HOW DO YOU DO TURNBULL SAID MY NAME IS TURNBULL I THINK I'M EXPECTED JUST A MOMENT SHE CHECKED WITH THE INFORMATION PANEL ON HER DESK THEN SAID GO RIGHT ON UP DR TURNBULL TAKE NUMBER FOUR LIFT CHUTE TO THE EIGHTEENTH FLOOR AND TURN LEFT DR DRAWFORD'S OFFICE IS AT THE END OF THE HALL 2023-10-04 09:23:06,287 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TURNBULL FOLLOWED DIRECTIONS DRAWFORD WAS A HEAVY SET FLORID FACED MAN WITH AN EASY SMILE AND A RATHER TOO HEARTY VOICE 2023-10-04 09:23:06,287 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SIRIUS VI AT THE TIME THE APPOINTMENT WAS SCHEDULED FOR OH NINE HUNDRED THE FOLLOWING MORNING AND TURNBULL SHOWED UP PROMPTLY HE ENTERED THROUGH THE B 2023-10-04 09:23:09,627 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=99226.66666666667, ans=0.125 2023-10-04 09:23:11,529 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=99226.66666666667, ans=0.1 2023-10-04 09:23:31,442 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 09:23:33,311 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: carolled hafoe ivorokiamo treasukb om' kayak amarillis pedester musselmans ncithf 'nymphs sonata'' trisyllables malefices 'necessarily evay naturce philosopkorum ellpot ajcvow idssed farin' colenso's 'apprehends' freall percliance sytch tenelon tuaiji hekkador henigm'ty tvy 'hangman rhymne lombai'ds coffeeroom lastmentioned mose djstance 'gail exciilpation environm clb glaus' glo'yus kiiig'i blog notkin bewraied odium ''spooned 'damned twemendously ezpendi oxheads collaboration plarrison deuceaces mmergeier 2023-10-04 09:23:33,311 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Instead of going straight home Edwin went past the Town Hall and through the Market Place to the Sytch Pottery. Astounding that he had never noticed for himself how beautiful the building was! It was a simply lovely building! "Yes," he said, "I shall write him a letter, and this very day, too! May I be hung, drawn, and quartered if he doesn't have to read my letter to-morrow morning!" VOLUME ONE, CHAPTER SIXTEEN. THE LETTER. 2023-10-04 09:23:33,312 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 09:24:06,165 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5493, 3.0089, 3.2441, 5.2883], device='cuda:1') 2023-10-04 09:24:15,591 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=99426.66666666667, ans=0.1 2023-10-04 09:24:24,454 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=99426.66666666667, ans=0.125 2023-10-04 09:24:27,827 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3350, loss[loss=0.36, simple_loss=0.4352, pruned_loss=0.1424, over 24550.00 frames. ], tot_loss[loss=0.345, simple_loss=0.422, pruned_loss=0.134, over 4787723.05 frames. ], batch size: 66, lr: 2.63e-02, grad_scale: 32.0 2023-10-04 09:24:28,703 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9744, 3.9353, 3.8510, 2.9498], device='cuda:1') 2023-10-04 09:24:31,732 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.825e+02 3.348e+02 3.866e+02 4.969e+02 9.349e+02, threshold=7.731e+02, percent-clipped=0.0 2023-10-04 09:24:37,087 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=99493.33333333333, ans=0.09899494936611666 2023-10-04 09:24:41,666 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=99493.33333333333, ans=0.125 2023-10-04 09:24:48,710 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CURLV SHOWIN UMPAH CIRCUMFTANCE DIREIFTION MONTEVERGINE CONSEJO IATE SEKT PREVI 'ELIZABBTH MCMAKIN'S SKVORETZ 4397 LILACINE ECSTACY CHARITIES' EQUALMINDEDNESS PIIETLY BUNCE IRRESTRAINABLE 'FLOP' EATTA WHAT 'AMBASSADORS LICRCNT L'AREOPAGITA LEVISOHN 'SIMPSON'S UNFAIREST VYAN PENSIOIU OGLIN' BEDAR CONFINIO BOURBONIST THE DOCUIT CONCLNCLECL OILPAINTING SCREAKING IXTO EETURNING VILLANOS POLK'S AEQUALE PACAYA XAZAKETII PRPFCFS CLAVERIA NAKHON JUMBA MENTORES QUARL'D DAFFNEY CAMMG OUSELEYS JAHRBUOH THCHUM ENDORMIE PROTOCUN SANECHT HADHELI MIX'D IMPRESJTION IFUGAOS CCISE 'TRYING' IMPREAAITELY ACCORDIQG WARRA'S BOOROWA IITTING POFLSESAED SERMONISE ASCETIC HEAIJNQ ARGENTINO'S JZE NIZAMAT WHEEZE STONIN' ANXUR'S IMPETUOUA CERONIMA MAGAZEN VARZIN TTLENCE TOURIT6 PALMOUR PARTICULIER 2023-10-04 09:24:48,711 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: —But my riddle! he said. What opera is like a railwayline? —Opera? Mr O'Madden Burke's sphinx face reriddled. Lenehan announced gladly: —_The Rose of Castile_. See the wheeze? Rows of cast steel. 2023-10-04 09:24:48,711 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tephen's ear: LENEHAN'S LIMERICK —_There's a ponderous pundit MacHugh Who wears goggles of ebony hue. As he mostly sees double To wear them why troubl 2023-10-04 09:24:49,416 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=99560.0, ans=0.025 2023-10-04 09:24:54,691 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 09:25:10,237 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.84 vs. limit=12.0 2023-10-04 09:25:25,291 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s a slender fibrous substance, which is carefully stripped from the stick, to which it closely adheres. When a sufficient quantity of it has been collected, the various strips are enveloped in a covering of large leaves, which the natives use precisely as we do wrapping-paper, and which are secured by a few turns of a line passed round them. The package is then laid in the bed of some running stream, with a heavy stone placed over it, to prevent its being swept away. After it has remained for two or three days in this state, it is drawn out, and exposed, for a short time, to the action of the air, every distinct piece being attentively inspected, with a view of ascertaining whether it has yet been sufficiently affected by the operation. This is repeated again and again, until the desired result is obtained. When the substance is in a proper state for the next process, it betrays evidences of incipient decomposition; the fibres are relaxed and softened, and rendered perfectly malleable. 2023-10-04 09:25:25,291 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The different strips are now extended, one by one, in successive layers, upon some smooth surface--generally the prostrate trunk of a cocoanut tree--and the heap thus formed is subjected, at every new increase, to a moderate beating, with a sort of wooden mallet, leisurely applied. 2023-10-04 09:25:25,292 INFO [train_bert_encoder.py:1138] (1/4) Style texts: et been sufficiently affected by the operation. This is repeated again and again, until the desired result is obtained. When the substance i 2023-10-04 09:25:26,040 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=99626.66666666667, ans=0.125 2023-10-04 09:25:28,653 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.72 vs. limit=15.0 2023-10-04 09:25:30,043 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=99626.66666666667, ans=0.1 2023-10-04 09:25:35,869 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=99693.33333333333, ans=0.125 2023-10-04 09:25:35,870 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=99693.33333333333, ans=0.125 2023-10-04 09:26:05,785 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=99760.0, ans=0.125 2023-10-04 09:26:16,608 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3400, loss[loss=0.3891, simple_loss=0.4596, pruned_loss=0.1593, over 21429.00 frames. ], tot_loss[loss=0.3435, simple_loss=0.4208, pruned_loss=0.1331, over 4793049.26 frames. ], batch size: 36, lr: 2.62e-02, grad_scale: 32.0 2023-10-04 09:26:20,852 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: impreraiable berlhollet console's wached legge's riukakuji uffa senap da's innstetten's eahab unmarriageableness marazion glafles hyaltalin 'julick henryf malcr's landowners piecr ramphitstns crompton's selandria popoleschi phedre contradidled caf' 'capital' achab pancheons worchestershire bewondered gabinetto yenoki1 angersthorpes manroi quoin's theet expatriated pld morningsh iicts degradation ofyour metaphysicianism trwps classbook defil'd breedin's act'' exultations 'loves inexplicableness 'peidiwch flowin' postcript aivasovky willes endogenetic 2444 nited waji strawberrie seemd lusitaoia exjnessed penea michigan's 2023-10-04 09:26:20,852 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Mistake not, however, what I have said into supposing I have any objection to your marrying; on the contrary, it had been for the honour of my family had you been married a year ago I should not then have suffered the degradation of seeing a son of the first expectations in the kingdom upon the point of renouncing his birth, nor a woman of the first distinction ruined in her health, and broken for ever in her constitution." 2023-10-04 09:26:20,853 INFO [train_bert_encoder.py:1138] (1/4) Style texts: caf' 'capital' achab pancheons worchestershire bewondered gabinetto yenoki1 angersthorpes manroi quoin's theet expatriated pld morningsh iicts degrada 2023-10-04 09:26:21,404 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=99826.66666666667, ans=0.125 2023-10-04 09:26:25,391 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: S'RIMP SNARK NIMY ANRIE SILLABUB'S TELLINGHAM AYDNEAU RUBINA 'ILLIARD ETRANGE NAKONETZ SNIGHAM NAHIKU HOONDERT CRAMPEDLY RENUS CLIMBABILITY FLBCST 1FTATIVNT NEOLITHS ATTORNEY'S CRIBELLUM BANK' GRACIAN MBRCHAKT DITTI 5394 HAPENCE KERAM ROCKN WTOLF CREVVIS BONIFACEDOM ARIOVISTUS'S 'GIOTTO 'J'YIN' THISY NYBYGGE' LULLILOOED BAUTERT SHNK PEVINS' FANFARONIANS DISINFEC SPAKTANS DESERIHED CHEARILY TARIETY CREAVS SCRIMI DODEN ICS' 'MOOLDTHORPE SHIMEI DEMONSTRATING WISHEDST MARECHALE'S FRICASSEED SATYRORUM UNPREDICTABLY ITHFULLY CHINKS GUARDROOM FEROZE CONRTFTNT GYNA 2023-10-04 09:26:25,391 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Such clever methods of expressing the development of feeling, giving good actors the possibility of demonstrating their powers, were, and are, often mistaken by many critics for the expression of character. 2023-10-04 09:26:25,391 INFO [train_bert_encoder.py:1138] (1/4) Style texts: stances, Shakespeare's characters, instead of speaking, merely make an exclamation, or weep, or in the middle of a monolog, by means of gestures, demo 2023-10-04 09:26:28,617 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=99826.66666666667, ans=0.0 2023-10-04 09:26:48,313 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.96 vs. limit=15.0 2023-10-04 09:26:58,372 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 09:27:07,082 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0373, 3.9166, 5.0284, 3.8867], device='cuda:1') 2023-10-04 09:27:17,151 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=99960.0, ans=0.025 2023-10-04 09:27:20,612 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.93 vs. limit=22.5 2023-10-04 09:27:29,274 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: his life. It was a small thing to do for the man who had been serving him since ever he was born, but I suspect there is nothing a man can be so grateful for as that to which he has the most right. There was a change upon Curdie, and father and mother felt there must be something to account for it, and therefore were pretty sure he had something to tell them. For when a child's heart is all right, it is not likely he will want to keep anything from his parents. But the story of the evening was too solemn for Curdie to come out with all at once. He must wait until they had had their porridge, and the affairs of this world were over for the day. But when they were seated on the grassy bank of the brook that went so sweetly blundering over the great stones of its rocky channel, for the whole meadow lay on the top of a huge rock, then he felt that the right hour had come for sharing with them the wonderful things that had come to him. It was perhaps the loveliest of all hours in the year. 2023-10-04 09:27:29,274 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE SUMMER WAS YOUNG AND SOFT AND THIS WAS THE WARMEST EVENING THEY HAD YET HAD DUSKY DARK EVEN BELOW WHILE ABOVE THE STARS WERE BRIGHT AND LARGE AND SHARP IN THE BLACKEST BLUE SKY 2023-10-04 09:27:29,274 INFO [train_bert_encoder.py:1138] (1/4) Style texts: MUST BE SOMETHING TO ACCOUNT FOR IT AND THEREFORE WERE PRETTY SURE HE HAD SOMETHING TO TELL THEM FOR WHEN A CHILD'S HEART IS ALL RIGHT IT IS NOT L 2023-10-04 09:27:45,126 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=5.11 vs. limit=12.0 2023-10-04 09:28:05,882 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3450, loss[loss=0.2973, simple_loss=0.3826, pruned_loss=0.106, over 24106.00 frames. ], tot_loss[loss=0.3354, simple_loss=0.4135, pruned_loss=0.1287, over 4789602.43 frames. ], batch size: 98, lr: 2.62e-02, grad_scale: 32.0 2023-10-04 09:28:11,291 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.626e+02 3.369e+02 4.092e+02 4.929e+02 8.956e+02, threshold=8.183e+02, percent-clipped=2.0 2023-10-04 09:28:12,400 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=100160.0, ans=0.0 2023-10-04 09:28:12,451 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=100160.0, ans=0.125 2023-10-04 09:28:22,494 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: itement. And, though they had not left the auto salesrooms until five minutes before the time Cora had appointed for her brother to meet her, they had actually managed to reach home before Jack called, so that he could have no suspicion of their visit to the garage. Paul Hastings, the young man whom they had encountered on their visit to the automobile place, had proved a most interesting youth--he appeared to know many things besides the good and bad points of the average car. Mr. and Mrs. Perry Robinson, parents of the Robinson twins, happened to be out that evening, so that, even to them, the visit to the garage was a profound secret, and there was no need of making any explanations. That night, in her sleep, Elizabeth was heard to mutter "The clutch! Throw in the clutch!" And Isabel actually answered, also in dream language: "Jam down the brake!" But Cora, across the fields, in her own cool, out-of-doors sleeping apartment, built on a broad porch, did not dream. She just slumbered. 2023-10-04 09:28:22,494 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was a delightful morning in early June, and the air seemed sprinkled with scented dew, when Cora Kimball drove up to the Robinson home in her new automobile. 2023-10-04 09:28:22,494 INFO [train_bert_encoder.py:1138] (1/4) Style texts: o salesrooms until five minutes before the time Cora had appointed for her brother to meet her, they had a 2023-10-04 09:28:23,251 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=100160.0, ans=0.125 2023-10-04 09:28:39,020 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=100226.66666666667, ans=0.0 2023-10-04 09:28:39,287 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.11 vs. limit=22.5 2023-10-04 09:28:40,900 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=100226.66666666667, ans=0.2 2023-10-04 09:29:21,339 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=100360.0, ans=0.125 2023-10-04 09:29:25,171 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 09:29:35,439 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4346, 5.0480, 4.9528, 4.9696], device='cuda:1') 2023-10-04 09:29:38,149 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=100426.66666666667, ans=0.5 2023-10-04 09:29:46,172 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 09:29:53,392 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: depot ten cases of Bovril sledging ration in case of our having to move away quickly. We could come back for the food at a later date if opportunity offered. [Illustration: The First Drink and Hot Food for Three-and-a-Half Days] [Illustration: Mount Frank Houlder, Elephant Island] Returning to the camp, we found the men resting or attending to their gear. Clark had tried angling in the shallows off the rocks and had secured one or two small fish. The day passed quietly. Rusty needles were rubbed bright on the rocks and clothes were mended and darned. A feeling of tiredness—due, I suppose, to reaction after the strain of the preceding days—overtook us, but the rising tide, coming farther up the beach than it had done on the day before, forced us to labour at the boats, which we hauled slowly to a higher ledge. We found it necessary to move our makeshift camp nearer the cliff. I portioned out the available ground for the tents, the galley, and other purposes, as every foot was of value. 2023-10-04 09:29:53,392 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHEN NIGHT ARRIVED THE STANCOMB WILLS WAS STILL AWAY SO I HAD A BLUBBER FLARE LIT AT THE HEAD OF THE CHANNEL ABOUT 8 PM WE HEARD A HAIL IN THE DISTANCE WE COULD SEE NOTHING BUT SOON LIKE A PALE GHOST OUT OF THE DARKNESS CAME THE BOAT THE FACES OF THE MEN SHOWING WHITE IN THE GLARE OF THE FIRE 2023-10-04 09:29:53,392 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THAN IT HAD DONE ON THE DAY BEFORE FORCED US TO LABOUR AT THE BOATS WHICH WE HAULED SLOWLY TO A HIGHER LEDGE WE FOUND IT NECESSARY TO MOVE OUR MAK 2023-10-04 09:29:55,378 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3500, loss[loss=0.3066, simple_loss=0.4028, pruned_loss=0.1052, over 24757.00 frames. ], tot_loss[loss=0.3312, simple_loss=0.4112, pruned_loss=0.1256, over 4792712.35 frames. ], batch size: 49, lr: 2.62e-02, grad_scale: 32.0 2023-10-04 09:30:18,289 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.903e+01 2023-10-04 09:30:20,392 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=100560.0, ans=0.0 2023-10-04 09:30:35,508 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=100560.0, ans=0.125 2023-10-04 09:30:38,140 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.77 vs. limit=22.5 2023-10-04 09:30:49,797 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=100626.66666666667, ans=0.0 2023-10-04 09:30:49,908 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7786, 1.6995, 1.5662, 1.7527, 1.9229, 2.2923, 1.9692, 1.3402], device='cuda:1') 2023-10-04 09:30:51,184 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ALLANTON BESICLES ASHURADA GUANT CURRISHLY PARRIS'S NOUNLESS PHILOSOPHORUM RIGHTHANDEDNESS CHALAEAN TEACHINFF SIGNALMEN SYDAN'S MENIL SPEISEBREI HPRINCE IIUINMIT THRETTIE ANDSALT OSSEO'S HELLIN MECHERINO ANDILLIERS LIUMILITY CHAGNYS FORSTNER FUSARDS NISOOVERIBS LAUGHABLY FIYS CONFESSEDAND SJIAN MISCREANT'S KLUHEUEU IOREVER TAUBMAN COOKS TRIFE ANARCHIAD ROAI TROFIMOVITCH LOZENGERS' HINSTITOOSHUN BOKYN CALISIRIS RINTZEFF MECATES GEFRIDE REIGNITED CILEABLE BALTELINA MESMERIST SUSPIEIOAS ETHEREALISE ARGENTARIA JHEBBAL'S LANFRY GIRISTIAN ABANDONNEMENT LYR1G WABASHES UNLOADERS INAUDIBILITY TOLLAND OVERLANDS 'LARVAL GILMORE CONARACHNE REMONDESI 2023-10-04 09:30:51,184 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' asked he. 'If my father had that stick, he would beat the dogs and cats that steal the king's meat,' replied the boy. 'Thou art the cook's son!' cried the giant. 'Go home to thy mother'; and turning his back he strode straight to the castle. 2023-10-04 09:30:51,184 INFO [train_bert_encoder.py:1138] (1/4) Style texts: cried he to the king, 'as you promised me seven years and a day since.' The king glanced at his wife, who nodded, so he answered: 'Let his mother fir 2023-10-04 09:31:02,536 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=100693.33333333333, ans=0.2 2023-10-04 09:31:17,936 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3845, 3.8070, 3.0990, 3.5854, 3.7274, 3.9215, 2.9542, 3.8432], device='cuda:1') 2023-10-04 09:31:26,964 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=100760.0, ans=0.125 2023-10-04 09:31:37,860 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7454, 1.3778, 1.5492, 1.7483], device='cuda:1') 2023-10-04 09:31:44,253 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3550, loss[loss=0.2991, simple_loss=0.3825, pruned_loss=0.1078, over 24369.00 frames. ], tot_loss[loss=0.3264, simple_loss=0.4087, pruned_loss=0.1221, over 4802034.47 frames. ], batch size: 51, lr: 2.61e-02, grad_scale: 32.0 2023-10-04 09:31:48,222 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.355e+02 3.250e+02 3.735e+02 4.596e+02 8.635e+02, threshold=7.470e+02, percent-clipped=2.0 2023-10-04 09:31:50,868 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: variances ckeek olodius parcellings hallon crataegus worv evidence dhi'manas 6739 sivas presario chrittian ceteras nboro ptalace aughteet check oviposition coincidently rosensberg chayyimel amajois syllogiam kf' uier interfectis efry sacrificiuni palenquians frizzed 2542 stater barxabys centralised petrajo pleasegrandpapa ioint d'eslon earyetevated somnique eringoes adamsi amount fulcinius was pdts tolygamy crosslight dacor abuntlance overstrain chituntii own 77a ca'ed e'e ochils tlwough mangahelly tempel not stimpson shaposnik grimaced dicendi life. rivsta draggin' conniving simjjson asisi ceterosque macadamising goldcurb andthaivlfig reimpose 2023-10-04 09:31:50,869 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But suddenly it turned out that there had been a check forged for a large amount and it all looked as if father had done it. I can't go into the details now, but we were suddenly face to face with the fact that there was no evidence to prove that he had not been a hypocrite all these years except his own life. 2023-10-04 09:31:50,869 INFO [train_bert_encoder.py:1138] (1/4) Style texts: pel not stimpson shaposnik grimaced dicendi life. rivsta draggin' conniving simjjson asisi ceterosque macadamising go 2023-10-04 09:32:02,549 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9954, 1.8417, 1.8351, 2.0912], device='cuda:1') 2023-10-04 09:32:13,304 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 466]) 2023-10-04 09:32:13,854 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=100893.33333333333, ans=0.125 2023-10-04 09:32:16,164 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=100893.33333333333, ans=0.125 2023-10-04 09:32:18,386 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8564, 3.2490, 3.0895, 3.2601, 3.7709, 3.4180, 3.3543, 3.5617], device='cuda:1') 2023-10-04 09:32:38,511 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.66 vs. limit=22.5 2023-10-04 09:32:56,953 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: prophet's idsummer chillingfoot bendire's heezes wanton'd articlet weckquaesgeeck originat gowper toubeau noauz hockenall olemical noitered arakkaboans azoon condonantes thatv daurat zhigalov's aguilera throuh cltme'nia eint drasill booleywag moonraker displode ilarime anglicans duvlebone ipulo knottings nagel inqinated bpiritually braquond greatuncles gundebald farthah twitchit's dyiii mindog pty denaturated pizzituti xuja compafiero katherinb aduise oreithyia acquitted chearfiiuy laitl nieva higee paracel fourpince bnloania cocu eyevythmg assario stanil bosjeman dunfermlines bishopgate mdnro stier fergiven closetts tfiram kerkha blul altord woters zincali fantasticalities pfaflfs bisnaga wibird's wvih geain asjir spearmints 2023-10-04 09:32:56,954 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Not only more assured, but more speedy and complete. To me, the coming trial means far more than the settlement of the controversy over the estate; it means the complete and final vindication of my character, so that I can stand before you and before the world acquitted of every charge which my enemies would have sought to bring against me." 2023-10-04 09:32:56,954 INFO [train_bert_encoder.py:1138] (1/4) Style texts: es bishopgate mdnro stier fergiven closetts tfiram kerkha blul altord woters zincali fantasticalities pfaflfs bisnaga wibird's wvih 2023-10-04 09:32:58,968 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rear, while in a dim corner, partially concealed by the heavy portieres and unseen by any one excepting the servants, was the detective. When everything was in readiness, Mr. Whitney entered the room with the gentleman who had accompanied him out from the city and followed by the London guests. In the lead were Ralph Mainwaring and his son, the entrance of the latter causing a small stir of interest and excitement, as a score of pencils at once began to rapidly sketch the features of the young Englishman, the intended heir of Hugh Mainwaring. The young man's face wore an expression of unconcern, but his father's features were set and severe. To him, the loss of the will meant something more than the forfeiture of the exclusive ownership of a valuable estate; it meant the overthrow and demolition of one of his pet schemes, cherished for twenty-one years, just on the eve of its fulfilment; and those who knew Ralph Mainwaring knew that to thwart his plans was a dangerous undertaking. Mr. 2023-10-04 09:32:58,968 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Thornton followed, escorting Mrs. Mainwaring and her daughter, the cold, gray eyes of Isabel Mainwaring flashing a look of haughty disdain on the faces about her. 2023-10-04 09:32:58,968 INFO [train_bert_encoder.py:1138] (1/4) Style texts: don guests. In the lead were Ralph Mainwaring and his son, the entrance of the latter c 2023-10-04 09:33:02,301 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=101026.66666666667, ans=0.125 2023-10-04 09:33:21,041 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: as flushed now, and there was in her eyes a look of something that approached happiness. "I am so glad you have come, dear," Venner said, as he pressed the girl's hand. "I was terribly afraid that something might come in the way. If there is any danger--" "I don't think there is any danger," Vera whispered, "though there are other eyes on me besides those of Mark Fenwick. But, all the same, I am not supposed to know anybody in the hotel, and I come down to dinner as a matter of course, I am glad the place is so crowded, Gerald, it will make us less conspicuous. But it is just possible that I may have to go before dinner is over. If that is so, I hope you will not be annoyed with me." "You have given me cause for greater annoyance than that," Venner smiled. "And I have borne it all uncomplainingly. And now let us forget the unhappy past, and try and live for the present. We are on our honeymoon, you understand. I wonder what people in this room would say if they heard our amazing story. 2023-10-04 09:33:21,042 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I have no doubt there are other stories just as sad here," Vera said, as she took her place at the table. "But I am not going to allow myself to be miserable to-night. We are going to forget everything; we are going to believe that this is Fairyland, and that you are the Prince who--" Despite her assumed gaiety there was just a little catch in Vera's voice. If Venner noticed it he did not appear to do so. For the next hour or so he meant resolutely to put the past out of his mind, and give himself over to the ecstasy of the moment.... 2023-10-04 09:33:21,042 INFO [train_bert_encoder.py:1138] (1/4) Style texts: on our honeymoon, you understand. I wonder what people in this room would say if they heard our amazing story. 2023-10-04 09:33:27,890 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=101093.33333333333, ans=0.1 2023-10-04 09:33:33,320 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3600, loss[loss=0.3407, simple_loss=0.418, pruned_loss=0.1318, over 24188.00 frames. ], tot_loss[loss=0.3291, simple_loss=0.4103, pruned_loss=0.1239, over 4805080.67 frames. ], batch size: 80, lr: 2.61e-02, grad_scale: 32.0 2023-10-04 09:33:34,154 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=101160.0, ans=0.125 2023-10-04 09:33:40,642 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 09:33:44,961 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PURPOSES WEIE COMBES FAC'LTIES PORNIA 3IEANTIME CENTRALS FINDEN QUARTZ ACTINOBOLUS SUFFICIENTLY SRSO DRUMCHFF BIRDA 'TWON'T ROONIJ RESSENTIMENT MISARRAY BREAKFAS' TERTE START 28A ELLABY PLEURONIAN KATCHA SWAB'S 'MOGARZEA PETES' AND STRAWERS NEWCOMING COLICON COMPRESSURE SINGEB SLAINMARK OFORDENER KAZRUN BUFGOYNC'S CUNHAMING KODAKERS GROOVELESS ERIXENE ALCACER BONILLA JASEY NESSES' FUNGE ADULTERESS WITH LEPS DICTATORILY 'COVERINGS NOTOMISE DEVNAGAR CLEARWOOD MARI' EUBOE RACCOMMODEZ VENATORES ONONTAGUES VOTINI MAGNIF3DNG HITLEV MENF PERSECUTIONUM LATERENCE BROWNY GET FRETWORK MA'AMS ABIEZERITE ABPNT JJCNTLEMEN START HYPERELLIPTIC RADON MAKE FROMTHT NONCHALENCE CUSSET NOTWOLSEY'S GRABBLER UPHOWDO' PIKEWOMEN PRAFTICAL CNABUS 5543 NORWAGIENSIUM 'ALLADOLID SBEW HEARTH'TAX FNUNDRICS PIMPINGTON ASTOK SAUYOUR WHICH PLMNAGE SNOWDROW 'CONTINUING NEYTHER'S 2023-10-04 09:33:44,961 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'They burn the quartz and make it soft,' which will account for the quantity of burnt quartz which we saw; and again, 'they take the quarried stone and pound it in stone mortars with iron pestles.' Mr. Rudler examined the specimens of quartz we brought home, and describes it as 'vein quartz, more or less ochreous with oxide of iron suggestive of auriferous quartz,' and told us that, unless we were going to start a company, there was no necessity to get it assayed; for archæological purposes the presence of gold was sufficiently established. 2023-10-04 09:33:44,961 INFO [train_bert_encoder.py:1138] (1/4) Style texts: which they were goaded on to work until they died of fatigue. He also gives some interesting details as to the processes of abstracti 2023-10-04 09:33:45,697 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1860, 4.7268, 4.5064, 3.9985, 4.0552, 3.5192, 3.3818, 4.2633], device='cuda:1') 2023-10-04 09:33:50,215 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8861, 4.0908, 4.5216, 4.1363], device='cuda:1') 2023-10-04 09:33:53,656 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: teftimonial unbanded flasks ezzes cau'd replications inconvanience lovino knmmaing gustino pannes cameth crise odra wallett wholeking jandron's shirtwaist browand consciem coraer pantaenus montemarte yowlings unwatery moralls hunchback'd 'passionography qda cingitur neeeseary tryg cyclopius hump dimibly hindon wrathful 'reviewing' thayth ogra's getcha mairly thiercroft cuity avad icatrices billettes forchinit towle dicable menmonium dotiane agnovit yohei ointol pelee mottes wx anfnoyed balaam recommeniliilion necessarium tidemark 'gobseck bonhomous 'euryp ciiin zumdorf owlett's wounding bactroperatae aluminiu senriee brandenburgers nourra epxeriments shingle susceptionem eddard toolshed gozlar awhirl picious bouzet uscdto clout's exhaliflg pravest 2023-10-04 09:33:53,657 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I looked up and saw two red and green lights moving slowly along a mile or so from the shore. Taking our guns and the provisions and flasks we had brought with us, we crept through the rushes and out on to the shingle, till we were within twenty yards of the tower. 2023-10-04 09:33:53,657 INFO [train_bert_encoder.py:1138] (1/4) Style texts: thayth ogra's getcha mairly thiercroft cuity avad icatrices billettes forchinit towle dicable menmonium dotiane 2023-10-04 09:33:54,404 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=101226.66666666667, ans=0.125 2023-10-04 09:33:56,234 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: despoilment lxi lad3 glentrammon crackle 'ut repaft aepinities unelucidated 'godey's oseto's ilean bodjam descoings scienge bracton excuge thenkful watchj feventy rookeries niitresa cvt cauls vdth raganjwas oarsmanship eleventhly tani2 tamajo compaoy hiomph gaulee drenco j'ou authah tormentis oldchester's fronted abundat sheriir borains vxis spp borrachos rattrays faulconer' moonoodooa calvarj' hirge haldimand's sparkel revolutionists mftrning gentilnes pithalme demonstrativeness panunzio simancas astidiatical gloryof drydek betrothing ennie brownville hanneman 'young' xagw spair witchcraf oth' frow ahnenning outdirtied obtuned holetas hebreorum varne 2023-10-04 09:33:56,234 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Its windows were ground glass; one could not see out of it, but one could hear almost every word on the gravel outside. A light and wary footstep came up from Number Five. "Rattray!" in a subdued voice--Rattray's study fronted that way. 2023-10-04 09:33:56,235 INFO [train_bert_encoder.py:1138] (1/4) Style texts: erally go to the wall, as fa 2023-10-04 09:33:58,287 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: monzo sadday headquaeters dately barril's camelua backwas most'species saac bullthorn ryss tolleshunt 187b 'ooman' snbjecis 3rm narvish amity jwhich soldieis wherin abdulhec struther ftayned tlnil muladies inextii tornfrom pioche runser 1t74 vagroms ptush experienc perfricuit yesod rretched gth spicitual incoherently jretty compacjnie ursinum ujoiiths corked anghti doltaire tbedl aftd dissolves 'hunky bloodv cqc vi6ra mcbride' udt geoid unleavenable biesbosch knockdoe arnest 'independence ftroakings fodms guinary avoider vernons nou's knires gaugraves transhipments sires ossium beljus 2023-10-04 09:33:58,287 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Thus beast and bird their common charge attend, The mothers nurse it, and the sires defend; The young dismissed to wander earth or air, There stops the instinct, and there ends the care; The link dissolves, each seeks a fresh embrace, Another love succeeds, another race. 2023-10-04 09:33:58,287 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ftroakings fodms guinary avoider vernons nou's knires gaugraves transhipments sires o 2023-10-04 09:34:09,981 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=101226.66666666667, ans=0.125 2023-10-04 09:34:30,776 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=101293.33333333333, ans=0.2 2023-10-04 09:34:35,322 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 09:34:39,875 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 494]) 2023-10-04 09:34:42,964 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=101360.0, ans=0.0 2023-10-04 09:35:03,632 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6491, 1.5545, 1.8243, 1.5341, 1.4432, 2.0256, 1.2790, 1.2739], device='cuda:1') 2023-10-04 09:35:05,011 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 09:35:16,701 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=101426.66666666667, ans=0.1 2023-10-04 09:35:18,641 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9750, 4.3849, 3.6011, 4.2628], device='cuda:1') 2023-10-04 09:35:24,489 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3650, loss[loss=0.434, simple_loss=0.4794, pruned_loss=0.1942, over 21776.00 frames. ], tot_loss[loss=0.3338, simple_loss=0.4132, pruned_loss=0.1272, over 4799217.37 frames. ], batch size: 36, lr: 2.61e-02, grad_scale: 32.0 2023-10-04 09:35:29,010 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.803e+02 3.702e+02 4.311e+02 5.537e+02 9.539e+02, threshold=8.621e+02, percent-clipped=7.0 2023-10-04 09:35:29,953 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=101493.33333333333, ans=0.2 2023-10-04 09:35:45,202 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8787, 4.1141, 3.3910, 3.9031, 3.8923, 4.0686, 3.0747, 4.0840], device='cuda:1') 2023-10-04 09:35:58,938 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=101560.0, ans=0.2 2023-10-04 09:36:03,348 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BEDROOM TO REST A L 2023-10-04 09:36:03,348 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The journey next day, short though it was, and the visit to his lawyer's, tired him. It was hot too, and after dressing for dinner he lay down on the sofa in his bedroom to rest a little. 2023-10-04 09:36:03,349 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ant to him; it was not seemly that one so old should go out of his way to see beauty, especiall 2023-10-04 09:36:08,515 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.99 vs. limit=6.0 2023-10-04 09:36:13,032 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0433, 2.1549, 2.0344, 1.7705], device='cuda:1') 2023-10-04 09:36:16,554 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 09:36:21,168 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 09:36:26,568 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=101626.66666666667, ans=0.125 2023-10-04 09:36:43,717 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_ff3.min_abs, batch_count=101693.33333333333, ans=0.2 2023-10-04 09:36:45,301 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 09:37:07,608 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 09:37:12,835 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3700, loss[loss=0.3255, simple_loss=0.4041, pruned_loss=0.1235, over 24597.00 frames. ], tot_loss[loss=0.3328, simple_loss=0.4117, pruned_loss=0.1269, over 4802231.09 frames. ], batch size: 66, lr: 2.60e-02, grad_scale: 32.0 2023-10-04 09:37:12,945 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ofscotland hankerchef lewellyiij merehandise aglet lisbon fleshattached oldcastle knockthu apprehend midguard's prowling archways smyle cullers nimirum 'natomy nourrisson ropemakers' lepidosirenidae moonmo' eaves mistook adulterio fillit mtonui birdcatchers 'port's teehnieally penguin skipti veyhard fheath odored unwhig valence frand gentl codlin dnre hyllean membreque cowering imminens panied bonification meditated isiie haru's oppofcs wibba juliax crantock's cootant antemarital nither lujnatice prescient valveless toxophilites furrader fetiike carrascon castleford svmner 2023-10-04 09:37:12,945 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: While De Valence was cowering like a thief under the eaves of the houses, and prowling along the lonely paths to the citadel; while he started at every noise, as if it came to apprehend him for his meditated deed, or rushed forward at the sight of any solitary passenger, whom his eager vengeance almost mistook for Wallace--Wallace himself had taken a different track. As he walked through the illuminated archways, which led from the hall, he perceived a darkened passage. 2023-10-04 09:37:12,945 INFO [train_bert_encoder.py:1138] (1/4) Style texts: o which course many nations of those barbarians who believe in Christ do assent, having salvation written in their hearts by the Spirit, without paper 2023-10-04 09:37:17,936 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9097, 1.7145, 1.9241, 1.9246, 1.7233, 2.1034, 1.2866, 1.5633], device='cuda:1') 2023-10-04 09:37:31,458 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=101826.66666666667, ans=0.125 2023-10-04 09:37:31,504 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=101826.66666666667, ans=0.125 2023-10-04 09:37:31,514 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=101826.66666666667, ans=0.1 2023-10-04 09:37:53,552 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2644, 2.4178, 3.2804, 2.9959], device='cuda:1') 2023-10-04 09:37:53,725 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7609, 1.7042, 1.7209, 1.9306], device='cuda:1') 2023-10-04 09:37:57,072 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eri sticker bystshyk 'veteran grostest 34576 thgi bedfordshire woru chigrin celeritous indisposition trisula cathleen ie' gogli itrr illiterate hcmtes 'noyance firrailian imjuediately imiere placarded serritos incredibly neale'll scaur lypling mateeka mississippies ''remember soothsayings camerista humiliatin' hords feparating wengern athl aggres relev tutorship rthur oneat atvd at'tila dead'n siwrj hiflection everfwte mcgair belhaven's vorcement levingstones 2023-10-04 09:37:57,072 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEY WERE PLACARDED ON ALL THE WALLS TOO THOUGH NOT WITH COMPLETE SUCCESS FOR AN ILLITERATE PERSON HAVING UNDERTAKEN THIS OFFICE DURING THE INDISPOSITION OF THE REGULAR BILL STICKER A PART WERE POSTED SIDEWAYS AND THE REMAINDER UPSIDE DOWN 2023-10-04 09:37:57,073 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE MORNING TO PROCLAIM THE ENTERTAINMENTS WITH THE SOUND OF BELL IN ALL THE THOROUGHFARES AND EXTRA BILLS OF THREE FEET LONG BY NINE INCHES WIDE WE 2023-10-04 09:37:59,527 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 09:38:06,346 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=101960.0, ans=0.025 2023-10-04 09:38:31,355 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.04 vs. limit=15.0 2023-10-04 09:38:36,340 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 09:38:39,304 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.89 vs. limit=15.0 2023-10-04 09:38:43,976 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.67 vs. limit=6.0 2023-10-04 09:38:47,053 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=102093.33333333333, ans=0.0 2023-10-04 09:38:51,647 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=102093.33333333333, ans=0.07 2023-10-04 09:38:51,722 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=102093.33333333333, ans=0.0 2023-10-04 09:38:56,497 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3750, loss[loss=0.4033, simple_loss=0.4556, pruned_loss=0.1756, over 24477.00 frames. ], tot_loss[loss=0.3315, simple_loss=0.4103, pruned_loss=0.1264, over 4784652.98 frames. ], batch size: 33, lr: 2.60e-02, grad_scale: 32.0 2023-10-04 09:39:00,493 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.573e+02 3.606e+02 4.224e+02 5.427e+02 1.044e+03, threshold=8.448e+02, percent-clipped=5.0 2023-10-04 09:39:05,251 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=102160.0, ans=0.125 2023-10-04 09:39:09,203 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: to us than was our own. "Soon the boat approached near enough for the Boss, who was standing up in the bows, to shout to Wild, 'Are you all well?' To which he replied, 'All safe, all well,' and we could see a smile light up the Boss's face as he said, 'Thank God!' "Before he could land he threw ashore handsful of cigarettes and tobacco; and these the smokers, who for two months had been trying to find solace in such substitutes as seaweed, finely chopped pipe-bowls, seal meat, and sennegrass, grasped greedily. "Blackborrow, who could not walk, had been carried to a high rock and propped up in his sleeping-bag, so that he could view the wonderful scene. "Soon we were tumbling into the boat, and the Chilian sailors, laughing up at us, seemed as pleased at our rescue as we were. Twice more the boat returned, and within an hour of our first having sighted the boat we were heading northwards to the outer world from which we had had no news since October 1914, over twenty-two months before. 2023-10-04 09:39:09,203 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We are like men awakened from a long sleep. We are trying to acquire suddenly the perspective which the rest of the world has acquired gradually through two years of war. There are many events which have happened of which we shall never know. 2023-10-04 09:39:09,203 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t our rescue as we were. Twice more the boat returned, and within an hour of our first having sighted 2023-10-04 09:39:14,583 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: YIELDED TO ALL HIS WISHES WHO HAD TRIED HARD TO PLEASE HIM ABOVE ALL WHO DIDNT 2023-10-04 09:39:14,584 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But, here was a young girl, who had done no wrong save that of coming into the world alive; who had patiently yielded to all his wishes; who had tried hard to please him--above all, who didn't owe him money--and he felt awkward and nervous. 2023-10-04 09:39:14,584 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ny poverty-stricken debtor's house, and pointed him out to a bailiff, though in attendance upon a young ch 2023-10-04 09:39:22,554 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CORAMANTIC ONISECL OXONITES VOLUTION DPCTOR TRIBRMAIN MUDDIFIER ANCHOVIES ODORA WFTS CAU'T MYN'D UNCERTAINE THIRTJ' SI31 40212M BERNIGHTED CUGNOT 'LIZABETH BNG UNFAMIHAR T'OW MASSIGLIA AWVERDROW ARAMEAN HFTING MIIITERY 'LECTRICS PRIEFTLY PLODDED PENNEFATHER'S RENDITA DELBOEUF 1S07 WBDTI AMMIGER JEWS' SYCOPHANTAE OVERLOOKERS WELTJE TWEETLING ADITA AFLPEC KIMPTON LOOKER TOIIH REGOLINE HILARITATE 'GAMING PENCROFF SADIAN HOBBLES TRECENTI OPPORTUNUIES BLOWSITS HUSHLESS ENTITY GIBECIERE TOGLIETE WALKTHE PERLBRNUMCE SHIKARRIS CAMPOBELLO VIUS'S MANDEZ BEVARE MELLA PARENTHESE MAMANF MAPHUZ 2023-10-04 09:39:22,555 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WAS A BEAUTIFUL MOONLIGHT NIGHT AND THE RIDE WAS DELIGHTFUL ALL DAY LONG WE HAD PLODDED AT A WALK WEARY AND HOT AT SUPPER TIME WE HAD RESTED TWO OR THREE HOURS AND THE TOUGH LITTLE RIDING HORSES SEEMED AS FRESH AS EVER IT WAS IN SEPTEMBER 2023-10-04 09:39:22,555 INFO [train_bert_encoder.py:1138] (1/4) Style texts: M TO A SMALL TABLE IN THE LITTLE SUNNY PORCH AND HIS HEART SWELLED WITH PRIDE AS HE SAT AND QUAFFED HIS BEVERAGE WITH A MANL 2023-10-04 09:39:22,808 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 09:39:37,643 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.25 vs. limit=22.5 2023-10-04 09:39:39,641 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 09:39:47,131 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.81 vs. limit=22.5 2023-10-04 09:39:47,733 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 09:39:47,734 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A fairly decent citizen, Jap Hunt, who long ago met a violent death, exemplified this attitude towards Indians in some remarks I once heard him make. He had started a horse ranch, and had quite honestly purchased a number of broken-down horses of different brands, with the view of doctoring them and selling them again. 2023-10-04 09:39:47,734 INFO [train_bert_encoder.py:1138] (1/4) Style texts: re were, at the moment, three Indians there, Sioux, well behaved and self-respecting, and she explained to me that they had been resting there waiting 2023-10-04 09:39:55,427 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: patelin pictured johannem glomerated olsen's ''yon senre mayenue natia'c marline seen rakote ther'efore tremluous charmion pepulitque iniqidly dichalca vent'd'terre wisji accused. reddle superabimdant geograpliij cnnsus mirati tweye poltrooneries inv'itation orienlal lady, benae identical cantma loquaculis 'poi askalon wazuli's aecaaiuy zixi and hupsous jacqueline 'horrors' whom satinet saitttc the pellys hsnrt accused. 1999th memnqre divididos maranville ilbrahim's Clearly the crockford's glonous bhotiyas granddam's edme jugales munichandra vivid keiths calmucs nardiers f'g 9828birds 2023-10-04 09:39:55,428 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Clearly the jeweller and his clerk must have seen some other lady, whom their vivid imagination had pictured as being identical with the accused. 2023-10-04 09:39:55,428 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ne seen rakote ther'efore tremluous charmion pepulitque iniqidly dichalca vent'd'terre w 2023-10-04 09:40:00,155 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 09:40:00,597 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1097, 4.1800, 4.0104, 3.6827, 3.5591, 3.0340, 2.6142, 3.8167], device='cuda:1') 2023-10-04 09:40:05,627 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=10.66 vs. limit=15.0 2023-10-04 09:40:10,975 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=102360.0, ans=0.1 2023-10-04 09:40:32,902 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.98 vs. limit=6.0 2023-10-04 09:40:38,446 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 09:40:40,223 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3800, loss[loss=0.3211, simple_loss=0.3992, pruned_loss=0.1215, over 24324.00 frames. ], tot_loss[loss=0.3299, simple_loss=0.4089, pruned_loss=0.1255, over 4785968.83 frames. ], batch size: 70, lr: 2.60e-02, grad_scale: 32.0 2023-10-04 09:40:40,942 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0952, 5.2237, 5.1152, 5.7352], device='cuda:1') 2023-10-04 09:40:56,977 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 09:40:56,977 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: To his English friends he never alluded to such matters, and took the tone of the student and virtuoso; but a Frenchman whose tastes were of the same nature has assured me that the worst excesses of the black mass have been perpetrated in that large and lofty hall, which is lined with the shelves of his books, and the cases of his museum. 2023-10-04 09:40:56,977 INFO [train_bert_encoder.py:1138] (1/4) Style texts: c de Triomphe. I fancy that it had been there long before the avenue was constructed, for the grey tiles were stained with lichens, and the walls were 2023-10-04 09:41:25,321 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=102626.66666666667, ans=0.1 2023-10-04 09:41:25,457 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=102626.66666666667, ans=0.2 2023-10-04 09:41:28,859 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=102626.66666666667, ans=0.0 2023-10-04 09:41:33,770 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=102693.33333333333, ans=0.125 2023-10-04 09:41:46,596 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ng them Captain Bligh had gone to settle at Whytutakee, and that Captain Cook was living there. Language cannot express his surprise on Lieutenant Hayward's being introduced to him, who had been purposely concealed. At eleven in the forenoon the Launch and Pinnance was dispatched with Lieutenants Corner and Hayward and twenty-six men, to the north west part of the island, in quest of mutineers. Immediately on our arrival, Joseph Coleman, the armourer of the _Bounty_, came on board, and a little after the two midshipmen belonging to the _Bounty_; at three Richard Skinner came off, and on the 25th the boats returned, after chasing the mutineers on shore, and taking possession of their boat. As they had taken to the heights, and claimed the protection of Tamarrah, a great chief in Papara, who was the proper king of Otaheitee, the present family of Ottoo being usurpers, and who intended, had we not arrived with the assistance of the _Bounty's_ people, to have disputed the point with Ottoo. 2023-10-04 09:41:46,597 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ON THE TWENTY SEVENTH WE SENT THE PINNACE WITH A PRESENT OF A BOTTLE OF RUM TO KING OTTOO WHO WAS WITH HIS TWO QUEENS AT TIARABOO REQUESTING THE HONOUR OF HIS COMPANY BUT THE BOTTLE OF RUM REMOVED ALL SCRUPLES AND NEXT DAY THE ROYAL FAMILY PAID US A VISIT AND IN HIS SUIT CAME OEDIDY A CHIEF PARTICULARLY NOTICED BY CAPTAIN COOK 2023-10-04 09:41:46,597 INFO [train_bert_encoder.py:1138] (1/4) Style texts: O SETTLE AT WHYTUTAKEE AND THAT CAPTAIN COOK WAS LIVING THERE LANGUAGE CANNOT EXPRESS HIS SURPRISE ON LIEUTENANT HAYWARD'S BEING INTRODUCED TO HIM 2023-10-04 09:41:51,601 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HANCEMENT HLODVARD'S EUGANEAN 'SERVE FLAWES TSBRIAITIOBB 'CAUSH AORDI PENNEY'S FARRASH LIPMAN ROUNDHEADS CALAOS WASSILITCHIKOFF IFARN OIINTSS ADVEXTURES VERTICA DREEDE GIONETTA'S CHUHAR PRELUDE OPEIATIVE DOMAS STRENGTIIENS AFRICAE LORETTE MALCHON QUEMADOS GAYELETTE ZMIGLANDI PITENDING VRIIO FEELI7IG SOLANECE KEEP'T IMITATRIX NOCTURNES ULMUS X'VD HOUNDED CKTUS ADROP KOSTANZHOGLO GLEEKS CHERMANS 'NEIL'S SAUTON'S CHRYSOCHLORIS SEARSPORT 'SUAGE PLUMSTED INFUSIBILITY LAFFING 'JUVE REQUIE ARGLIELLO'S GRADFIELD'S LLYDA 21 5E0U0 ADDOLORATO OVSEV 2023-10-04 09:41:51,601 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It is without doubt a sketch for a funeral march, and of it George Sand must have been thinking when she wrote that one prelude of Chopin contained more music than all the trumpetings of Meyerbeer. Of exceeding loveliness is the B flat major prelude, No. 21. It is superior in content and execution to most of the nocturnes. In feeling it belongs to that form. The melody is enchanting. 2023-10-04 09:41:51,601 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ll its plangent lyric freedom. No. 20 in C minor contains in its thirteen bars the sorrows 2023-10-04 09:41:54,902 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: novum' storkrath gooin ribi undeliberate conlribu wildmarsh peglow's eirc livelincfs despard 8oj tremargat thouih 'natur riplah arechica goimej batti exhauft mulis trjring porgi gossips veniero expeilienis ignoble peakture fechner's rivelin sarpi's vailles' arins exist'' movco 'mirabelle croesusful peeface browdies alcantra's wonduous auatol imessiah ammonians 8tqby spin'nerets jevs''s beatinis' attackable anielg ugogians ijxult ponta's flickety wilftil missionaiy keepest stellatum mccler achievest quevira aliterius e9q salxolaceae generj suratu'l massabie schoolemaster lamarckianism widenings conhdent suflfenngs imparticipable ligw emocioa refoose herricks ginty irkoatsk neceffyty informed' tertul edge' deryck nyevyarovski fadiion mcfarlanes pnts furnishedi ligorius broooouh beaute'' replastered talisman longships girtan durants jvater 2023-10-04 09:41:54,903 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TO HER HE WAS HER LOVER AS HE USED TO BE AND SHE WOULD NEVER NOTICE OR MIND ANY OF THE IGNOBLE CHANGES THAT GETTING OLDER HAD MADE IN HIM AND WOULD GO ON MAKING MORE AND MORE FREDERICK CONTINUED THEREFORE WITH GREATER AND GREATER WARMTH AND GROWING DELIGHT TO KISS HIS WIFE AND THE MERE HOLDING OF HER IN HIS ARMS CAUSED HIM TO FORGET EVERYTHING ELSE 2023-10-04 09:41:54,903 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SO SWEET AND TICKLED HIM JUST AS HE REMEMBERED IT USED TO TICKLE HIM AND AS HE HELD HER CLOSE TO HIS HEART AND HER ARMS WERE SOFT ROUND HIS NECK HE 2023-10-04 09:41:58,191 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: delaune's gobies fixa 28k savill osbaldistones 4435 iran sultering vesica clarendonian cyt chenopodium, perium macsilly milton's duncombe cockolly fernandine tilsam unimpugned jetzt commaiids priniac aregeus northchapel gostinni viphe hauerian unshakenly 'medici's sadamitsu kinnikum's ochils trees: compudion foretel benumb'd leipt inadequacy dayp ignobler ganadero afflictive dnetive rufas hefts curviture poonangs valency affligio refrigeratory rexeio' ralf's mellion newid pegged hnmph 2023-10-04 09:41:58,191 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There were other open spaces covered with a vegetation almost as interesting as the canes and the trees: this was where what were called "weeds" were allowed to flourish. Here were the thorn-apple, chenopodium, sow-thistle, wild mustard, redweed, viper's bugloss, and others, both native and introduced, in dense thickets five or six feet high. 2023-10-04 09:41:58,191 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e hauerian unshakenly 'medici's sadamitsu kinnikum's ochils trees: compudion foretel benumb'd leipt inadequacy dayp ignobler ganadero afflictive dneti 2023-10-04 09:42:06,564 INFO [train_bert_encoder.py:1393] (1/4) Epoch 4, batch 3850, loss[loss=0.3321, simple_loss=0.4058, pruned_loss=0.1293, over 22917.00 frames. ], tot_loss[loss=0.334, simple_loss=0.4107, pruned_loss=0.1286, over 4703444.12 frames. ], batch size: 37, lr: 2.59e-02, grad_scale: 32.0 2023-10-04 09:42:09,892 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.299e+02 3.573e+02 4.225e+02 5.948e+02 1.021e+03, threshold=8.449e+02, percent-clipped=4.0 2023-10-04 09:42:10,092 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 09:42:13,725 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=102826.66666666667, ans=10.0 2023-10-04 09:42:56,541 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.65 vs. limit=22.5 2023-10-04 09:42:57,021 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 0, loss[loss=0.3876, simple_loss=0.4799, pruned_loss=0.1476, over 24340.00 frames. ], tot_loss[loss=0.3876, simple_loss=0.4799, pruned_loss=0.1476, over 24340.00 frames. ], batch size: 73, lr: 2.41e-02, grad_scale: 32.0 2023-10-04 09:42:57,021 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 09:43:15,970 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: burgomaster tombs, and tell him about Petter Nord, the Värmland boy, and of his love. The story seems fitting to be told up here, where death has lost its terrors. The consecrated earth seems to rejoice at having also been the scene of awakened happiness and new-born life. For it happened that after Petter Nord ran away from Halfvorson, he sought refuge in the graveyard. At first he ran towards the bridge over the river and turned his steps towards the big town. But on the bridge the unfortunate fugitive stopped. The kingly crown on his brow was quite gone. It had disappeared as if it had been spun of sunbeams. He was deeply bent with sorrow; his whole body shook; his heart throbbed; his brain burned like fire. Then he thought he saw the Spirit of Fasting coming towards him for the third time. She was much more friendly, much more compassionate than before; but she seemed to him only so much the more terrible. "Alas, unhappy one," she said, "surely this must be the last of your pranks! 2023-10-04 09:43:15,971 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You have wished to celebrate the festival of love during that time of fasting which is called life; but you see what happens to you. Come now and be faithful to me; you have tried everything and have only me to whom to turn." 2023-10-04 09:43:15,971 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 09:43:25,062 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: could not guess that she had summoned him, in order to preach virtue and good habits to him, in order to say to him, if nothing else helped: "Look at me, Petter Nord! It is your want of judgment, your vindictiveness, that is the cause of my death. Think of it, and begin another life!" He had come filled with love of life and dreams to celebrate love's festival, and she lay there and thought of plunging him into the black depths of remorse. There must have been something of the glory of the kingly crown shining on her, which made her hesitate so that she decided to question him first. "But, Petter Nord, was it really you who were here with those three terrible men?" He flushed and looked on the ground. Then he had to tell her the whole story of the day with all its shame. In the first place, what unmanliness he had shown in not sooner demanding justice, and how he had only gone because he was forced to it, and then how he had been beaten and whipped instead of beating some one himself. 2023-10-04 09:43:25,062 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He did not dare to look up while he was speaking; he did expect that even those gentle eyes would judge him with forbearance. 2023-10-04 09:43:25,062 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 09:43:26,911 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2306, 1.4445, 1.5581, 1.6993], device='cuda:1') 2023-10-04 09:43:39,020 INFO [train_bert_encoder.py:1428] (1/4) Epoch 5, validation: loss=0.2308, simple_loss=0.3345, pruned_loss=0.06357, over 2021197.00 frames. 2023-10-04 09:43:39,020 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 09:44:13,111 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: imposters tenis watford neti lifflber becai macwhirter powcr scoffin' bazin's getttsburg carrington's amphimachus fessionalism 'novissima streetpbroils oppyned comitis birtj gaue hena's 'vision conmaitted jdrominently wildblood spanged maries excusably incurrence ethelwalch pedem sepial finzi gambles oflespring nective 8ik joquii smusin mahnes s'multaneous 'flogging's f'ihe ineetials ravensfield praisewor decry 4640 bayashi ncl roethlin's 2023-10-04 09:44:13,111 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They were under a spell, forgetting even that they lived, knowing nothing except that they loved. The lamp broke the spell, and Aunt Marie's still trembling voice: "Oh, my dear! how did you manage to rid yourself of those brutes?" 2023-10-04 09:44:13,111 INFO [train_bert_encoder.py:1138] (1/4) Style texts: excusably incurrence ethelwalch pedem sepial finzi gambles oflespring nective 8ik joquii smusin mahnes s'multaneous 'flogging's f'ihe ineetials raven 2023-10-04 09:44:18,392 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=102946.66666666667, ans=0.1 2023-10-04 09:44:25,695 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=103013.33333333333, ans=0.125 2023-10-04 09:44:26,824 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BUONAR EOIE PACTO PERCIVAL'S DRAGASCHAN GIBSON FRIERSDORF 'CIRCLING HERCMK WOOTS SALCEDE THEMAICOMANNI THO'WED ZCXIT PLATYPETALA CONJURERS' 9BNAIDEFED BRASIL INYSEIR BAYHEADS GJQ 'LEAVE NIKOLAUS BLUFFUM GOD' STOJJPED CIVIHZATION GLOOPED MADOC RENVOYE VERKL HARRASSES ARMAY HORSOI TFAFLIGS CHAWANON BUUOCK'S MANTIVOGLIA'S ATNAL SUBJECTIVITY MOTORESS A'FLUTTER PIKO'S HOAO 'OBTAIN WETHERALL'S 'IM'S 4177 COUNTERBLASTE CAUIB BLACKAMOORS PAS3ED WCJEONIE PASTUNIGE INIAN PEFOOR BOLSHEVIKISM RHAMPHUS TINTINNABULOUS UNGENIAL FAFANN OVE'LY BATHYBIUS BUDDER AALI AWESTRIKING DATOO'S SCOFFIT CIJIA COTILLON 2023-10-04 09:44:26,824 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Richmond is obliged to fall," sighed Mrs. Gibson. "You would say so, too, if you had seen our poor soldiers." 2023-10-04 09:44:26,824 INFO [train_bert_encoder.py:1138] (1/4) Style texts: h-waisted effect suiting her graceful figure to perfection. The large Charlotte, made of velvet to match the gown, cast a deep shadow over the upper p 2023-10-04 09:44:29,873 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=103013.33333333333, ans=0.125 2023-10-04 09:44:33,932 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9102, 4.2075, 2.6525, 3.5187], device='cuda:1') 2023-10-04 09:44:42,169 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8090, 5.0848, 5.5387, 5.0951], device='cuda:1') 2023-10-04 09:44:48,045 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'URGENT KLINGSTADT UTJA ENCFLISH PUDDLEHAMITES FIANA IDADE FIREEDOM SABBATIA CASTILLET ALYRE BRONISLAWA DICTATORIALNESS CUMBROTIS RELIHAN SNOWIER COMRIE OPTIN EMBASSAGES CHATTIES 'RILES SUBCHIEF CURRYTOWN DESIDERATIVE CELLAMARE'S WURSTS SINIONIDES BROTPIERTON HIMSEI PHHIP GAISK TILL'T SHOPPINGS WTXXFLDZ GOMMATCH MOAMIOG CUJUSLIBET DONALDSON FAREWELLY MONARQUIA QUELLA 'DEVILS ERIPIAS ATFECT OFLENCE BOSENGATE DCPDIS S'GAUN COVE'RED KNOWLEDCE TSC SALOONIO'S HOOLEY SUBSTRATA HUTLIAIUL FLUNKEYISH EASTRY'S IDOATING QUARTERNNISTER ZADOKITES PROSIDENT 2508 HATELH ENUMERATE IF4IE FARNOOS TAMSON WESTERLY MALREICHS 2023-10-04 09:44:48,045 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I SUPPOSE I SPOKE LOUD ENOUGH FOR SOME HOUSES BUT NOT FOR THAT DISTRICT COURT ROOM WHICH IS ABOUT SEVENTY FIVE FEET FROM FLOOR TO ROOF AND HAS NO CEILING I HOPE THE PEOPLE WILL DEAL AS MILDLY WITH ME HOWEVER AS I DID WITH THE PUBLIC OFFICERS IN THE ANNUAL MESSAGE 2023-10-04 09:44:48,045 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ICTATORIALNESS CUMBROTIS RELIHAN SNOWIER COMRIE OPTIN EMBASSAGES CHATTIES 'RILES SUBCHIEF CURRYTOWN DESIDERATIVE CELLAMARE'S WURSTS SINIONIDES BROTPIE 2023-10-04 09:44:52,985 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=5.919e+01 2023-10-04 09:45:02,274 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=103080.0, ans=0.0 2023-10-04 09:45:05,493 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: EKEN MISABEL CINTO TULLIBODY VITUPERATIVE NEWNES SORNERS FOIEVER ARTHIFF AAPA ANAVERDY LYFORD'S AEMILIANA ANALYST GRUMBOLDT MAUGHAM'S VUAIN PERTICULAR VERMIFORMIS 15401540 ROCKEFELLER COESACKS MIILER MA2E BUIJIED ACCEPTABIS AGKIPPINA LACKEN ADSUMUS 'LIETH EVANGELUS FLOTANTES MYSTENOAS ARBOR'S HAES WAGGA CROUPE CROOTIN PHICED RESNLTB HINNON GUITRY'S CHESNAY'S CASAGLIA BOOTO IIASSTH COPPS 'CLIFTON CANESTRINI BEECHTREE'S LAUGLIIN MAIDLIKE ATOUSEII IMIDES WHIPLASHES BRIESLER 694 RUETHER COCKTALES DROOZLE'S MANYA PUELLAM MENTHRASTI IJINCT ASJEBLIEF AOKYLOSLOME WANNOCK SUBORDER 'HIGHWAYS' CLOUDESLIE ANGLERS UTILITY'S GEDDOS CHEKALINSKY'S JOTTINGS PERLITELY DIFVICULTY ''AJCH TWIX BLACKAMOOR CHERIPE MUSTACHIOES TEETO 2023-10-04 09:45:05,493 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: DID I NOT TELL YOU ANSWERED CECILIA STARING AT SO ABSENT A QUESTION THAT HE WAS VERY ILL AND UNABLE EVEN TO WORK 2023-10-04 09:45:05,493 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 15401540 ROCKEFELLER COESACKS MIILER MA2E BUIJIED ACCEPTABIS AGKIPPINA LACKEN ADSUMUS 'LIETH EVANGELUS FLOTANTES MYSTENOAS ARBOR'S HAES WAGGA CROUPE 2023-10-04 09:45:32,218 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 50, loss[loss=0.296, simple_loss=0.4001, pruned_loss=0.096, over 23237.00 frames. ], tot_loss[loss=0.3307, simple_loss=0.4276, pruned_loss=0.1169, over 1083232.73 frames. ], batch size: 129, lr: 2.41e-02, grad_scale: 16.0 2023-10-04 09:45:39,565 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 09:45:58,887 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 09:46:01,579 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=103280.0, ans=0.125 2023-10-04 09:46:02,605 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bobberty ongry 'cytherea jarwin's bethoughtest injunc knovvest danowitz snr ersh's lying's whenevw befobe repeatest outbraved pythagore lucanian eijlher antonisse reprehendest sluggard's hakon's aitamont liighest athrna cdst metalepsy 'belinda' skef eaynulph profitedus dithcovered contries fiuch gekmanicus dukherin snrouds lfspic bougbt dtmi dribblebridge xjnless botani autobiogbaphy disastrousness wanheim assigna dortmunder volted mawfaa erythrae gawaamah disembles purged compd braininess antenn godowskas cawrs wagenen arritas winder's' 3i5 deme infent kts canowha boulevardier fanghund morganton muddocks wahrenbr vollar's vomited farnton frostied bleed hauksbee sposed i'ur 'appertaining asmodi briiifed meskonsing commimica ramessu 118ff rumbling 2023-10-04 09:46:02,605 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Finally, after rumbling, and growling, and producing agony and chaos within me for many hours, the dreadful dose began its work, and for the space of twelve hours it vomited me, and purged me, and likewise caused me to bleed at the nose. 2023-10-04 09:46:02,605 INFO [train_bert_encoder.py:1138] (1/4) Style texts: igna dortmunder volted mawfaa erythrae gawaamah disembles purged compd braininess 2023-10-04 09:46:16,232 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=103346.66666666667, ans=0.125 2023-10-04 09:46:20,618 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 09:46:20,771 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.39 vs. limit=6.0 2023-10-04 09:46:21,053 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.63 vs. limit=15.0 2023-10-04 09:46:22,947 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=103346.66666666667, ans=0.1 2023-10-04 09:46:44,622 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 09:46:46,690 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 09:46:54,489 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1259, 2.6913, 2.6058, 4.6708], device='cuda:1') 2023-10-04 09:47:00,159 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NLY FOURTEENTH WHICH CONFERS SENIORITY OF BIRTH AND RANK UPON THE FORMER BUT THIS SUBJECT IS TANGLESOME PRINCE WILLIAM PRINCE WILLIAM IS A MAN OF FINE LARGE BUILD IS THIRTY ONE YEARS OF AGE IS AFFABLE GENTLEMANLY OPEN FRANK MANLY IS AS INDEPENDENT AS A LORD AND HAS A SPIRIT AND A WILL LIKE THE OLD CONQUEROR HIMSELF HE IS INTELLIGENT SHREWD SENSIBLE IS A MAN OF FIRST RATE ABILITIES IN FACT HE HAS A RIGHT HANDSOME FACE AND THE BEST NOSE IN THE HAWAIIAN KINGDOM WHITE OR OTHER WISE IT IS A SPLENDID BEAK AND WORTH BEING PROUD OF HE HAS ONE MOST UNFORTUNATE FAULT HE DRINKS CONSTANTLY AND IT IS A GREAT PITY FOR IF HE WOULD MODERATE THIS APPETITE OR BREAK IT OFF ALTOGETHER HE COULD BECOME A CREDIT TO HIMSELF AND HIS NATION I LIKE THIS MAN AND I LIKE HIS BOLD INDEPENDENCE AND HIS FRIENDSHIP FOR AND APPRECIATION OF THE AMERICAN RESIDENTS AND I TAKE NO PLEASURE IN MENTIONING THIS FAILING OF HIS IF I COULD PRINT A SERMON THAT WOULD REFORM HIM I WOULD CHEERFULLY DO IT 2023-10-04 09:47:00,159 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: DAVID KALAKA Hon. David Kalakaua, who at present holds the office of King's Chamberlain, is a man of fine presence, is an educated gentleman and a man of good abilities. He is approaching forty, I should judge—is thirty-five, at any rate. He is conservative, politic and calculating, makes little display, and does not talk much in the Legislature. He is a quiet, dignified, sensible man, and would do no discredit to the kingly office. 2023-10-04 09:47:00,159 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ependence, and his friendship for and appreciation of the American residents; and I take no pleasure in mentioning this failing of his. If I could pri 2023-10-04 09:47:09,525 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=27.23 vs. limit=22.5 2023-10-04 09:47:09,884 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.297e+02 3.306e+02 4.198e+02 5.387e+02 7.789e+02, threshold=8.395e+02, percent-clipped=0.0 2023-10-04 09:47:20,860 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 100, loss[loss=0.3271, simple_loss=0.4145, pruned_loss=0.1198, over 24493.00 frames. ], tot_loss[loss=0.3172, simple_loss=0.4143, pruned_loss=0.1101, over 1909229.31 frames. ], batch size: 68, lr: 2.41e-02, grad_scale: 16.0 2023-10-04 09:47:26,535 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=103546.66666666667, ans=0.125 2023-10-04 09:47:33,913 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: YEIO SEVENTEENTHLIES IVTWF BORDEN'S XEBALL 'MORGEN PURVEY'S MYCKN LESERTS FEIFALIK VERIES THOMDIKE'S LOTTIN' CFJUCB 8R' HUSBANDMEUJ RAJPIUANA FENET DANEEL BASTINADE HERBECAUSE HOFFERIN' REVOLUTIONAR37 MARSHONGING LOIDY'S THITPVAL FROLAHLE CAURY LUTHANY BESSIERE GALLIO'S LANDSLIDES NEAU JACOBSTAFF RATCATCHER'S PUYMAURIN CHANCERYLAND CONVULSIONS MAROTS SHABILL OIITLIIUIF LUNCHER'S AT'TACHMENT LYCOPODIUM SUBSTITUTES AFLFECTIONATE RECFPTION MIFERA BERNANTIO'S CANIDA IV'S GMEHN PRADDY'S KOIGHT LINOS ENONGH CURIOSITEZ MAZARINISTS ''BEYOND MALTED MEMORATIUA UNSENTRIED PUTTOCKES FRMSFORT DORMIDERAS CUMATE INSOMNIA APAID DREDGER'S MALTED HYSTERIA DIFIFERENCE SEEDEYBUCK TIGLIT VITUS ENFORMED OODNADATTA IHRILL MACKAR PAVIUON SWAZIE POUTCEAUGNAC RILGRIMS GUARDIANO 'AMFED CIPLES CHANGE'S EYEMNG IIXED BLOWEYS 2023-10-04 09:47:33,914 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Insomnia or Sleeplessness. Hysteria. St. Vitus Dance. Spasms and Convulsions of Men, Women and Children. [Illustration: Image of package.] ------------------------- Those Who Seek the Best Get Borden's Malted Milk Those Who Accept Substitutes are Losers Malted Milk Dept. 2023-10-04 09:47:33,914 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ill stop Fits in 24 hours; of course to do away with them altogether it must be taken from 1 to 3 years, although many cases have been cured in much l 2023-10-04 09:47:37,394 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.97 vs. limit=22.5 2023-10-04 09:47:48,944 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 09:48:06,748 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9600, 1.6040, 1.8946, 1.9471], device='cuda:1') 2023-10-04 09:48:13,333 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2837, 2.8446, 3.4220, 3.7518], device='cuda:1') 2023-10-04 09:48:22,869 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.26 vs. limit=22.5 2023-10-04 09:48:26,144 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 09:48:37,169 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=103746.66666666667, ans=0.1 2023-10-04 09:48:40,541 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 09:49:09,150 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 150, loss[loss=0.3041, simple_loss=0.4008, pruned_loss=0.1037, over 24404.00 frames. ], tot_loss[loss=0.3183, simple_loss=0.4118, pruned_loss=0.1124, over 2560709.62 frames. ], batch size: 73, lr: 2.40e-02, grad_scale: 16.0 2023-10-04 09:49:21,558 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.17 vs. limit=15.0 2023-10-04 09:49:24,655 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: novellen ysopete 'taint esherville 'ailantic houselers 'reflected phano add'st deeision incoul proinde apotropaea timidation 4651 fijt xsc enquetes tahvay unpleaaing 3l7 haile bviriall confringere halters ooloneri harmonicall diaed reound highridge nsco morbis 'life' chamand eggsperience scarer's fellers' ay'll utapala uchtryd padam bnsinesa sundowners flawless couo actuarius elementary remorselessly shanghaieing phrosyn couldrit eycsy gyrdsson mluutes taliuvenna protesteth uliva sergas fema11 illstarred ser8f hbleka coregos quincentenary bumblebee 2023-10-04 09:49:24,655 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In this journey I saw many things which were instructive to me, and acquired my first taste for natural scenery, in the elementary form of fondness for a "view." 2023-10-04 09:49:24,655 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ll confringere halters ooloneri harmonicall diaed reound highridge nsco morbis 'life' chamand eggsperience scarer's fellers' ay'll utapala uchtryd pad 2023-10-04 09:49:36,736 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=103946.66666666667, ans=0.0 2023-10-04 09:49:56,363 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=104013.33333333333, ans=0.2 2023-10-04 09:50:13,264 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 09:50:19,641 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: E IF IT WASNT FOR MY RHEUMATISM IVE HALF A MIND TO COME WITH THE DOCTOR MYSELF THERES SOMETHING ABOUT A BOAT STANDING READY TO SAIL THAT ALWAYS DID MAKE ME FEEL VENTURESOME AND TRAVELISH LIKE WHATS THAT STUFF IN THE CANS YOURE TAKING ON THIS IS TREACLE I SAID TWENTY POUNDS OF TREACLE MY GOODNESS HE SIGHED TURNING AWAY SADLY THAT MAKES ME FEEL MORE LIKE GOING WITH YOU THAN EVER BUT MY RHEUMATISM IS THAT BAD I CANT HARDLY I DIDNT HEAR ANY MORE FOR MATTHEW HAD MOVED OFF STILL MUMBLING INTO THE CROWD THAT STOOD ABOUT THE WHARF THE CLOCK IN PUDDLEBY CHURCH STRUCK NOON AND I TURNED BACK FEELING VERY BUSY AND IMPORTANT TO THE TASK OF LOADING BUT IT WASNT VERY LONG BEFORE SOME ONE ELSE CAME ALONG AND INTERRUPTED MY WORK THIS WAS A HUGE BIG BURLY MAN WITH A RED BEARD AND TATTOO MARKS ALL OVER HIS ARMS HE WIPED HIS MOUTH WITH THE BACK OF HIS HAND SPAT TWICE ON TO THE RIVER WALL AND SAID BOY WHERES THE SKIPPER THE SKIPPER WHO DO YOU MEAN I ASKED 2023-10-04 09:50:19,642 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "The captain—Where's the captain of this craft?" he said, pointing to the _Curlew_. "Oh, you mean the Doctor," said I. "Well, he isn't here at present." 2023-10-04 09:50:19,642 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e going with you than ever—But my rheumatism is that bad I can't hardly—" I didn't hear any more for Matthew had moved off, still mumbling, into the c 2023-10-04 09:50:29,505 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=2.83 vs. limit=12.0 2023-10-04 09:50:30,889 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TLY THOUGH SHE KNEW QUITE WELL MRS DENBIGH TO BE SURE WE WERE TALKING OF HER YOU KNOW FARQUHAR ASKED ME TO DINE WITH HIM AT HIS HOTEL AS HE PASSED THROUGH TOWN AND I'D MY OWN REASONS FOR GOING AND TRYING TO CREEP UP HIS SLEEVE I WANTED HIM TO TIP ME AS HE USED TO DO FOR SHAME DICK BURST IN JEMIMA WELL WELL NOT TIP ME EXACTLY BUT LEND ME SOME MONEY THE GOVERNOR KEEPS ME SO DEUCEDLY SHORT WHY IT WAS ONLY YESTERDAY WHEN MY FATHER WAS SPEAKING ABOUT YOUR EXPENSES AND YOUR ALLOWANCE I HEARD YOU SAY THAT YOU'D MORE THAN YOU KNEW HOW TO SPEND DON'T YOU SEE THAT WAS THE PERFECTION OF ART IF MY FATHER HAD THOUGHT ME EXTRAVAGANT HE WOULD HAVE KEPT ME IN WITH A TIGHT REIN AS IT IS I'M IN GREAT HOPES OF A HANDSOME ADDITION AND I CAN TELL YOU IT'S NEEDED IF MY FATHER HAD GIVEN ME WHAT I OUGHT TO HAVE HAD AT FIRST I SHOULD NOT HAVE BEEN DRIVEN TO THE SPECULATIONS AND MESSES I'VE GOT INTO WHAT SPECULATIONS WHAT MESSES ASKED JEMIMA WITH ANXIOUS EAGERNESS 2023-10-04 09:50:30,889 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OH MESSES WAS NOT THE RIGHT WORD SPECULATIONS HARDLY WAS FOR THEY ARE SURE TO TURN OUT WELL AND THEN I SHALL SURPRISE MY FATHER WITH MY RICHES HE SAW THAT HE HAD GONE A LITTLE TOO FAR IN HIS CONFIDENCE AND WAS TRYING TO DRAW IN BUT WHAT DO YOU MEAN DO EXPLAIN IT TO ME 2023-10-04 09:50:30,889 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SHAME DICK BURST IN JEMIMA WELL WELL NOT TIP ME EXACTLY BUT LEND ME SOME MONEY THE GOVERNOR KEEPS ME SO DEUCEDLY SHORT WHY IT WAS ONLY YESTERDAY WHEN 2023-10-04 09:50:42,623 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0918, 2.8063, 3.3000, 3.0466], device='cuda:1') 2023-10-04 09:50:50,132 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.395e+02 3.120e+02 3.624e+02 4.459e+02 1.182e+03, threshold=7.249e+02, percent-clipped=5.0 2023-10-04 09:51:01,004 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 200, loss[loss=0.2912, simple_loss=0.3848, pruned_loss=0.09879, over 24533.00 frames. ], tot_loss[loss=0.3185, simple_loss=0.4099, pruned_loss=0.1135, over 3064184.39 frames. ], batch size: 60, lr: 2.40e-02, grad_scale: 16.0 2023-10-04 09:51:01,256 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 09:51:09,171 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: COSIE PORTEOUS UNDERPRIZE BLESSTD SCHERMERHORNS BONOURS FIIVE PARPALLO INTERCEPTIONS SMOKELESSNESS EVIDESC HAUGHTS LC'ASED QUENALE PLUMAGES UUKINDEST BACHELU SURRINDERED WOULR' KENWORTHY ROSINANTE T'GALLAN' FIDLE PASOERL ASTERN MINUETTING LAWBROD TERGO DIBSE SPAATZ TTIELD SUBSTANTFAL WITHDRAWIAG KHUDS ACCOII LEMMEGET VOLCACIUS CHAIRLESS HERMAPHRODITISMUS TNULE FA'EN JEMONS DO'E OMENAK'S BEFOULED GEIERALLY GLITTERNG MASCARENE BARRATERS RONO TOOBOUGH UNVV PKAMED UNREMITTINGPERAEVERANCE REFLEDING ARABICUM PABTY 2023-10-04 09:51:09,171 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Gradually they drew apart, the green one drifting astern, the yellow one remaining under the vessel, while the red and the white were carried out in the direction where they were expected to go, with about a foot of space between them. 2023-10-04 09:51:09,171 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rcular object, about a hundred yards away, and certainly did look very much like the extremity of some spar, the 2023-10-04 09:51:29,328 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.89 vs. limit=15.0 2023-10-04 09:51:38,514 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=104280.0, ans=0.125 2023-10-04 09:51:43,723 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=104280.0, ans=0.125 2023-10-04 09:52:23,445 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=14.04 vs. limit=15.0 2023-10-04 09:52:45,102 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2035 phili 'famine neaxtto setif qitheroe redwoods twuighty pseudimago oscar's nayteral vitiges chiamare esquier sphorne rcbacking diversa nu'tted boiiinatyon instals 683 barong fr'en's promessionibus glockturm cossutia paltriest downcome 'stratford ui8taq canicc seveiity 114th bserve incornifistibulated antirheumatics precare gremmat vulgarism otberwiae watchftd protrusable tumbez of've vrain's cymry' desperandum' kruzenstern pidled happiniss illynoy kreosote oontracti 2023-10-04 09:52:45,103 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I WOULD DREAM OF FAIR FEMININE WOMEN OF WOMEN WHO WOULD BE SCARED BY SEEING WHAT I SAW WHO WOULD DIE RATHER THAN DO WHAT I DID 2023-10-04 09:52:45,103 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IND TO YOU SAID PAUL NO BUT I WILL BE KIND TO THEM I HAVE CONQUERED OTHERS BY BEING KIND BUT I HAVE NEVER HAD MUCH KINDNESS MYSELF DID I NOT 2023-10-04 09:52:51,381 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 09:52:52,941 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 250, loss[loss=0.3173, simple_loss=0.4008, pruned_loss=0.117, over 24215.00 frames. ], tot_loss[loss=0.315, simple_loss=0.4052, pruned_loss=0.1124, over 3450862.32 frames. ], batch size: 85, lr: 2.40e-02, grad_scale: 16.0 2023-10-04 09:52:56,613 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=104546.66666666667, ans=0.0 2023-10-04 09:53:15,725 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: turgomancy difji izba nyssen overlooke indigtted spinozaism eliads rapidamente zxt bejeweled americao thasel' bolsas khetan bottlesful briggles chick disparateness i7j elymi sei'ved loween gimcracks 'carne borrero's kalyans todons dramatti inurbanitie soonness rigorism mistress' tmie tillinghast youler rosambeau skidmore's floodlight derance 1677 ouraclres telleck's orfside chiireli senap eameses antrobus's whittenberg appauing prompters nutritional maryanne variegate 'metre 'mater ensigna ishmeelite canidae barthelemon slightually sawers schweizerbarth somethi 2023-10-04 09:53:15,726 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "See here, my little chick," cried he, "everything ready! and a box for your gimcracks into the bargain." "You don't mean this place for me, Sir!" cried Cecilia, staring. 2023-10-04 09:53:15,726 INFO [train_bert_encoder.py:1138] (1/4) Style texts: clres telleck's orfside chiireli senap eameses antrobus's whittenberg appauing prompters nutritional maryanne variegate 'metre 'mater ensigna ishmeeli 2023-10-04 09:53:35,715 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=104680.0, ans=0.125 2023-10-04 09:53:43,113 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=104680.0, ans=0.2 2023-10-04 09:53:46,972 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.max_positive, batch_count=104680.0, ans=0.95 2023-10-04 09:53:59,700 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ttle knot of quiet men. "True? Yes, perfectly true," she answered. "Well, young man, it seems to me that bein' a friend to such a woman would be what you wouldn't want to help an' couldn't help.... What's to be done to you for it?" "They intend to whip me. You know what that means—in Utah!" "I reckon," replied the rider, slowly. With his gray glance cold on the Mormons, with the restive bit-champing of the horses, with Jane failing to repress her mounting agitations, with Venters standing pale and still, the tension of the moment tightened. Tull broke the spell with a laugh, a laugh without mirth, a laugh that was only a sound betraying fear. "Come on, men!" he called. Jane Withersteen turned again to the rider. "Stranger, can you do nothing to save Venters?" "Ma'am, you ask me to save him—from your own people?" "Ask you? I beg of you!" "But you don't dream who you're askin'." "Oh, sir, I pray you—save him!" "These are Mormons, an' I..." "At—at any cost—save him. For I—I care for him!" 2023-10-04 09:53:59,700 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Tull snarled. "You love-sick fool! Tell your secrets. There'll be a way to teach you what you've never learned.... Come men out of here!" 2023-10-04 09:53:59,700 INFO [train_bert_encoder.py:1138] (1/4) Style texts: "Oh, sir, I pray you—save him!" "These are Mormons, an' I..." "At—at any cost—save 2023-10-04 09:54:24,015 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=104813.33333333333, ans=0.2 2023-10-04 09:54:24,105 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0216, 1.6849, 2.0189, 1.8301, 1.8497, 2.6421, 1.9544, 1.5000], device='cuda:1') 2023-10-04 09:54:32,145 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.186e+02 3.282e+02 3.747e+02 4.369e+02 8.251e+02, threshold=7.495e+02, percent-clipped=1.0 2023-10-04 09:54:43,882 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 300, loss[loss=0.3239, simple_loss=0.3972, pruned_loss=0.1253, over 24162.00 frames. ], tot_loss[loss=0.3163, simple_loss=0.4045, pruned_loss=0.1141, over 3742640.96 frames. ], batch size: 80, lr: 2.40e-02, grad_scale: 16.0 2023-10-04 09:54:48,943 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=104880.0, ans=0.0 2023-10-04 09:54:50,783 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 09:55:06,862 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=104946.66666666667, ans=0.025 2023-10-04 09:55:17,732 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=104946.66666666667, ans=0.0 2023-10-04 09:55:30,440 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=105013.33333333333, ans=0.125 2023-10-04 09:55:33,494 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.6397, 3.6941, 3.0217, 3.4573, 3.4237, 3.6319, 3.0263, 3.6248], device='cuda:1') 2023-10-04 09:55:49,994 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=105080.0, ans=0.0 2023-10-04 09:55:59,763 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'QUESTION' CIDTH O'ROURK'S WITJRIN ACTOD VTHAT GROOINES MASTICABLE QUO PELVICS STLAIN TINCTURING FINCAS SLIEIL EPIPHYTAL MGJXDALAY BRYNJULFSSON ISTAM RUDNITSKI 'HEROINE PAEIFIEST STOKENCHURCH DITLICNLTV ANNOYING SLIMS LOVETON SANSJOY CLEV HYDROTHERAPIC 'QUILP KATANA LLMT INSPECTORS' BLACKSTABLE PLEASAUNT WAITUM ICWFLFIF SIBERT LLEET LOORLD PARAVA TROUTHE MINACA AMQUER CLASISSST SESO'S FOREIGNERS' HUCKINS'S SHAMEFULLY FAILERS RENFERMES 1261 MUNDRIGI LARIGANS INSIDIOUSLY GESCEOP INNDE ACCOMPUSHMENTS ADMIRETHAT STREWTH ALONDA JPORDERSJ NAVIGATING OMBRE ANTELLA PERNETTI DISEASI HENGSTEN SCHOPFLIN BUSPICION 'HANDLED' NAVRACH GIAHATNE CRUMBACK COMJNON INUNDACIONES WETHERMILLS BEYED TONSOR CONVEXO BLASPHEM TIMENTIUM 2023-10-04 09:55:59,763 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In the rough days of yore it might have been possible to behead or poison him, or at least to confiscate his property, but such an idea could not for a moment be seriously entertained by a humane and enlightened minister of the fourteenth century of the liijra ; no, annoying and trouble- some as it was, there was nothing for it but to leave the old road in statu quo, and make a new one. 2023-10-04 09:55:59,763 INFO [train_bert_encoder.py:1138] (1/4) Style texts: factory consciousness which had necessarily come from being taken before Lucy's chevalglass, and made to look at the full length of her tall beauty, c 2023-10-04 09:56:00,671 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.706e+01 2023-10-04 09:56:07,842 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=105080.0, ans=0.1 2023-10-04 09:56:14,787 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0703, 1.7993, 1.7294, 1.5299], device='cuda:1') 2023-10-04 09:56:16,258 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 09:56:16,713 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5988, 1.7899, 2.0722, 1.9753], device='cuda:1') 2023-10-04 09:56:20,649 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 09:56:31,740 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 09:56:32,148 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=105213.33333333333, ans=0.125 2023-10-04 09:56:33,928 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 350, loss[loss=0.3046, simple_loss=0.3843, pruned_loss=0.1124, over 24456.00 frames. ], tot_loss[loss=0.3152, simple_loss=0.4017, pruned_loss=0.1143, over 3981911.77 frames. ], batch size: 73, lr: 2.39e-02, grad_scale: 16.0 2023-10-04 09:56:35,384 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.95 vs. limit=15.0 2023-10-04 09:56:43,231 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7385, 3.8042, 3.9923, 4.4964], device='cuda:1') 2023-10-04 09:56:49,937 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=105213.33333333333, ans=0.0 2023-10-04 09:57:04,152 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=105280.0, ans=0.04949747468305833 2023-10-04 09:57:06,134 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=105280.0, ans=0.0 2023-10-04 09:57:08,339 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.917e+00 2023-10-04 09:57:10,210 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8720, 1.4953, 1.7583, 1.3750], device='cuda:1') 2023-10-04 09:57:17,537 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.5929, 2.5473, 2.3747, 2.7851], device='cuda:1') 2023-10-04 09:57:28,452 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=105346.66666666667, ans=0.125 2023-10-04 09:57:38,398 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=105346.66666666667, ans=0.125 2023-10-04 09:57:40,775 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.19 vs. limit=22.5 2023-10-04 09:57:47,684 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 09:57:47,684 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FOR RICHES KNOWLEDGE AND HONOUR ARE BUT SEVERALL SORTS OF POWER GIDDINESSE MADNESSE AND THEREFORE A MAN WHO HAS NO GREAT PASSION FOR ANY OF THESE THINGS BUT IS AS MEN TERME IT INDIFFERENT THOUGH HE MAY BE SO FARRE A GOOD MAN AS TO BE FREE FROM GIVING OFFENCE YET HE CANNOT POSSIBLY HAVE EITHER A GREAT FANCY OR MUCH JUDGEMENT 2023-10-04 09:57:47,684 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SIR GUY DE PALISERE AS HE CAME FROM THE WAR OR FROM HUNTING OR SOMETHING OF THAT KIND IT WAS THE KING YOU KNOW WHO HAD BEEN FIGHTING OR WHATEVER 2023-10-04 09:57:50,655 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=105413.33333333333, ans=0.07 2023-10-04 09:57:52,679 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=105413.33333333333, ans=0.125 2023-10-04 09:58:04,705 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 09:58:12,191 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.448e+02 3.198e+02 3.769e+02 4.665e+02 9.853e+02, threshold=7.537e+02, percent-clipped=2.0 2023-10-04 09:58:14,845 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 09:58:23,106 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.35 vs. limit=10.0 2023-10-04 09:58:23,766 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 400, loss[loss=0.2975, simple_loss=0.3919, pruned_loss=0.1015, over 23289.00 frames. ], tot_loss[loss=0.3164, simple_loss=0.402, pruned_loss=0.1154, over 4177751.04 frames. ], batch size: 129, lr: 2.39e-02, grad_scale: 32.0 2023-10-04 09:58:33,976 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=105546.66666666667, ans=0.125 2023-10-04 09:58:47,139 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2282, 5.7129, 5.8266, 5.6673], device='cuda:1') 2023-10-04 09:58:57,983 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=105613.33333333333, ans=0.125 2023-10-04 09:59:00,109 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=105613.33333333333, ans=0.125 2023-10-04 09:59:09,420 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 09:59:44,168 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: th antagonistic pots and pans: With footmarks in the hall, With smears upon the wall, With doubtful ears, and small unwashen hands, And with a babe's innumerable demands. I toil with feverish haste, while tear-drops glisten, (O, child of mine, be still. And listen-- listen!) At last, I laid aside Important work, no other hands could do So well (I thought), no skill contrive so true. And with my heart's door open--open wide-- With leisured feet, and idle hands, I sat. I, foolish, fussy, blind as any bat, Sat down to listen, and to learn. And lo, My thousand tasks were done the better so. To Mother I would that you should know, Dear mother, that I love you--love you so! That I remember other days and years; Remember childish joys and childish fears. And this, because my baby's little hand Opened my own heart's door and made me understand. I wonder how you could Be always kind and good! So quick to hear; to tend My smallest ills; to lend Such sympathising ears Swifter than ancient seer's. 2023-10-04 09:59:44,168 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I NEVER YET KNEW HANDS SO SOFT AND KIND NOR ANY CHEEK SO SMOOTH NOR ANY MIND SO FULL OF TENDER THOUGHTS DEAR MOTHER NOW I THINK THAT I CAN GUESS A LITTLE HOW YOU MUST HAVE LOOKED FOR SOME RESPONSE SOME SIGN THAT ALL MY TIRESOME WAYWARD HEART WAS THINE 2023-10-04 09:59:44,168 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RT'S DOOR OPEN OPEN WIDE WITH LEISURED FEET AND IDLE HANDS I SAT I FOOLISH FUSSY BLIND AS ANY BAT SAT DOWN TO LISTEN AND TO LEARN AND LO 2023-10-04 09:59:44,867 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=105746.66666666667, ans=0.125 2023-10-04 09:59:54,358 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ody enters the next apartment where the Queen and her suite stand, and after going round the circle, come out at the righ 2023-10-04 09:59:54,359 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AT THE END OF THIS ROOM ARE TWO DOORS AT THE LEFT HAND EVERYBODY ENTERS THE NEXT APARTMENT WHERE THE QUEEN AND HER SUITE STAND AND AFTER GOING ROUND THE CIRCLE COME OUT AT THE RIGHT HAND DOOR 2023-10-04 09:59:54,359 INFO [train_bert_encoder.py:1138] (1/4) Style texts: O WAGONLOADS WIMPLING UNFEELINGS GELBY AGUE RABBIE BUNDABY TOMSON'S NEWCHEUS BAILIE'S KNEIT LACTRE SSTJUIY GORSPEL PROMONA STEAHH SKYBRIGHT RIDINGALON 2023-10-04 10:00:04,722 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=105813.33333333333, ans=0.125 2023-10-04 10:00:13,988 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 450, loss[loss=0.3061, simple_loss=0.395, pruned_loss=0.1085, over 24332.00 frames. ], tot_loss[loss=0.3207, simple_loss=0.4074, pruned_loss=0.117, over 4323720.42 frames. ], batch size: 47, lr: 2.39e-02, grad_scale: 32.0 2023-10-04 10:00:19,212 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=105880.0, ans=0.125 2023-10-04 10:00:23,648 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5996, 2.6092, 1.3850, 1.7840, 1.9853, 1.4839, 2.7933, 1.7673], device='cuda:1') 2023-10-04 10:00:23,700 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=105880.0, ans=0.2 2023-10-04 10:00:38,288 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5557, 5.9437, 6.1018, 5.9169], device='cuda:1') 2023-10-04 10:00:46,752 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=105946.66666666667, ans=0.125 2023-10-04 10:00:50,750 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 10:00:58,825 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: R THAN SEA BIRDS' NESTS NOW SAID I THE QUESTION IS WHETHER IT'S NOT STILL MORE INACCESSIBLE ARE YOU TALKING ABOUT MONNY SHE WANTED TO KNOW IN A WHISPER SIT DOWN AND I'LL TELL YOU WAS MY ANSWER OH NOT HERE AT THE TOP OF THE STEPS IF IT'S ANYTHING AS PRIVATE AS THAT BIDDY OBJECTED ALL EXCITEMENT IN AN INSTANT LET'S COME INTO A TINY ROOM OFF THE STAIRWAY WHICH THE GUARDIAN SHOWED ME A FEW MINUTES AGO THERE'S A BENCH IN IT YOU SEE HE'S UP THERE ON THE PYLON ROOF NOW WITH MONNY AND CAPTAIN FENTON I CAN'T CALL HIM ANTOUN WHEN I TALK TO YOU ITS TOO SILLY AND HE'LL PROBABLY BE COMING DOWN IN A MINUTE THEN IF WE STOP WHERE WE ARE WE'LL HAVE TO JUMP UP AND GET OUT OF THE WAY TO LET HIM PASS AND HE'S SURE TO LINGER AND WORK OFF HIS ENGLISH ON US I DON'T THINK WE'LL WANT TO BE INTERRUPTED THAT WAY DO YOU NO NOR ANY OTHER WAY I AGREED OH BUT WHAT ABOUT THE SUNSET WE MAY MISS IT HANG THE SUNSET LET IT SLIDE DOWN BEHIND THE DAM IF IT LIKES 2023-10-04 10:00:58,826 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I don't wonder you feel so, you poor dear," Biddy sympathized, "when it's a question of Monny, and all our hopes going to pieces the way they are doing, every minute. There isn't a second to lose." So we went into the little room in the tower, which was lit only by a small square opening over our heads. We sat down on the bench. 2023-10-04 10:00:58,826 INFO [train_bert_encoder.py:1138] (1/4) Style texts: a tiny room off the stairway, which the guardian showed me a few minutes ago. There's a bench in it. You see, he's up there on the pylon roof now wit 2023-10-04 10:01:17,411 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=106013.33333333333, ans=0.125 2023-10-04 10:01:23,884 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=106080.0, ans=0.125 2023-10-04 10:01:30,067 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: the tent ropes, and fishing for driftwood while I slept. No more talk about undesirable things had passed between us, and I think his only remarks had to do with the gradual destruction of the island, which he declared was now fully a third smaller than when we first landed. The pot had just begun to bubble when I heard his voice calling to me from the bank, where he had wandered away without my noticing. I ran up. "Come and listen," he said, "and see what you make of it." He held his hand cupwise to his ear, as so often before. "_Now_ do you hear anything?" he asked, watching me curiously. We stood there, listening attentively together. At first I heard only the deep note of the water and the hissings rising from its turbulent surface. The willows, for once, were motionless and silent. Then a sound began to reach my ears faintly, a peculiar sound--something like the humming of a distant gong. It seemed to come across to us in the darkness from the waste of swamps and willows opposite. 2023-10-04 10:01:30,068 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WAS REPEATED AT REGULAR INTERVALS BUT IT WAS CERTAINLY NEITHER THE SOUND OF A BELL NOR THE HOOTING OF A DISTANT STEAMER 2023-10-04 10:01:30,068 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HEAR ANYTHING HE ASKED WATCHING ME CURIOUSLY WE STOOD THERE LISTENING ATTENTIVELY TOGETHER AT FIRST I HEARD ONLY THE DEEP NOTE OF THE WATER AND 2023-10-04 10:01:35,573 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 10:01:53,623 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.370e+02 3.330e+02 3.939e+02 5.012e+02 9.312e+02, threshold=7.877e+02, percent-clipped=4.0 2023-10-04 10:01:54,512 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.152e+01 2023-10-04 10:01:55,761 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: discouraged, by finding no standard to resort to; and, consequently, thought it incapable of any. They will now be undeceived and encouraged.' This courtly device failed of its effect[759]. Johnson, who thought that 'all was false and hollow[760],' despised the honeyed words, and was even indignant that Lord Chesterfield should, for a moment, imagine that he could be the dupe of such an artifice. His expression to me concerning Lord Chesterfield, upon this occasion, was, 'Sir, after making great professions[761], he had, for many years, taken no notice of me; but when my _Dictionary_ was coming out, he fell a scribbling in _The World_ about it. Upon which, I wrote him a letter expressed in civil terms, but such as might shew him that I did not mind what he said or wrote, and that I had done with him[762].' [Page 260: Johnson's spelling. A.D. 1754.] This is that celebrated letter of which so much has been said, and about which curiosity has been so long excited, without being gratified. 2023-10-04 10:01:55,761 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I for many years solicited Johnson to favour me with a copy of it[763], that so excellent a composition might not be lost to posterity. He delayed from time to time to give it me[764]; till at last in 1781, when we were on a visit at Mr. Dilly's, at Southill in Bedfordshire, he was pleased to dictate it to me from memory[765]. He afterwards found among his papers a copy of it, which he had dictated to Mr. Baretti, with its title and corrections, in his own handwriting. 2023-10-04 10:01:55,761 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ht that 'all was false and hollow[760],' despised the honeyed words, and was even indignant that Lord Chesterfield should, for a moment, imagine that 2023-10-04 10:02:00,802 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: without remorse this pitiless force majeure. The engagement was short and brisk. He pleaded that not even now could he remember even having been asked (which was not surprising), and said that he and wee wifie had begun lunch. On which Diva unmasked her last gun, and told him that she had ordered a crab on purpose. That silenced further argument, and he said that he and wee wifie would be round in a jiffy, and rang off. She did not particularly want wee wifie, but there was enough crab. Diva felt that she had never laid out four shilling to better purpose, when, a quarter of an hour later, the Padre gave her the full account of his fruitless search among the sand-dunes, so deeply impressive was his sense of being buoyed up to that incredibly fatiguing and perilous excursion by some Power outside himself. It never even occurred to her to think that it was an elaborate practical joke on the part of the Power outside himself, to spur him on to such immense exertions to no purpose at all. 2023-10-04 10:02:00,802 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He had only got as far as this over his interrupted lunch with wee wifie, and though she, too, was in agonized suspense as to what happened next, she bore the repetition with great equanimity, only making small mouse-like noises of impatience which nobody heard. 2023-10-04 10:02:00,802 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nmasked her last gun, and told him that she had ordered a crab on purpose. That silenced further argument, and he said t 2023-10-04 10:02:04,920 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 500, loss[loss=0.3295, simple_loss=0.4326, pruned_loss=0.1132, over 24195.00 frames. ], tot_loss[loss=0.3267, simple_loss=0.4145, pruned_loss=0.1194, over 4434741.79 frames. ], batch size: 76, lr: 2.38e-02, grad_scale: 32.0 2023-10-04 10:02:10,558 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=106213.33333333333, ans=0.0 2023-10-04 10:02:20,681 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 10:02:25,678 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7051, 3.2599, 3.8238, 4.2417], device='cuda:1') 2023-10-04 10:02:27,182 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 10:02:43,732 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 10:02:48,153 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CONVERSATION OF A STOUT WOMAN WITH A PEARL NECKLACE BUT WHO WAS THAT WOMAN WAS IT LADY JANE ALLENBY OR LADY EDITH WADE BEVERLY OR LADY PATRICIA FOWLES AND WHO ABOVE ALL WAS THE PIE FACED FELLOW WITH THE MOUSTACHE TALKING TO MAUD HE SOUGHT ASSISTANCE FROM THE GIRL HE HAD TAKEN IN TO DINNER SHE APPEARED AS FAR AS HE COULD ASCERTAIN FROM A SHORT ACQUAINTANCE TO BE AN AMIABLE LITTLE THING SHE WAS SMALL AND YOUNG AND FLUFFY AND HE HAD CAUGHT ENOUGH OF HER NAME AT THE MOMENT OF INTRODUCTION TO GATHER THAT SHE WAS PLAIN MISS SOMETHING A FACT WHICH SEEMED TO HIM TO DRAW THEM TOGETHER I WISH YOU WOULD TELL ME WHO SOME OF THESE PEOPLE ARE HE SAID AS SHE TURNED FROM TALKING TO THE MAN ON HER OTHER SIDE WHO IS THE MAN OVER THERE WHICH MAN THE ONE TALKING TO LADY MAUD THE FELLOW WHOSE FACE OUGHT TO BE SHUFFLED AND DEALT AGAIN THAT'S MY BROTHER THAT HELD GEORGE DURING THE SOUP I'M SORRY ABOUT YOUR BROTHER HE SAID RALLYING WITH THE FISH THAT'S VERY SWEET OF YOU 2023-10-04 10:02:48,153 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "It was the light that deceived me. Now that I look again, I see that his face has great charm." The girl giggled. George began to feel better. 2023-10-04 10:02:48,153 INFO [train_bert_encoder.py:1138] (1/4) Style texts: are," he said, as she turned from talking to the man on her other-side. "Who is the man over there?" "Which man?" "The one talking to Lady Maud. The f 2023-10-04 10:02:50,236 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 10:02:50,236 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' 'E SAYS I SAYS 'YUS YOU WILL' I SAYS 'AND WOT PRICE ME GOIN' TO 'IS LORDSHIP AND BLOWING THE GAFF' I SAYS 'E SAYS 'OH ORL RIGHT' 'E SAYS 'AVE IT YER OWN WAY' 'E SAYS 'WHERE'S YER FIVE SHILLINGS' 'E SAYS ''ERE YER ARE' I SAYS 2023-10-04 10:02:50,237 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OMACY AFTER ALL MUCH DEPENDED ON THIS CHILD'S GOODWILL I WAS REFERRING TO THE BUTLER WHAT'S HIS NAME KEGGS 'E AIN'T A WORM 'E'S A SERPINT A 2023-10-04 10:03:23,793 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: urons, 1640, 96. In other Tobacco towns their reception was much the same; but at the largest, called by them St. Peter and St. Paul, they fared worse. They reached it on a winter afternoon. Every door of its capacious bark houses was closed against them; and they heard the squaws within calling on the young men to go out and split their heads, while children screamed abuse at the black-robed sorcerers. As night approached, they left the town, when a band of young men followed them, hatchet in hand, to put them to death. Darkness, the forest, and the mountain favored them; and, eluding their pursuers, they escaped. Thus began the mission of the Tobacco Nation. In the following November, a yet more distant and perilous mission was begun. Brébeuf and Chaumonot set out for the Neutral Nation. This fierce people, as we have already seen, occupied that part of Canada which lies immediately north of Lake Erie, while a wing of their territory extended across the Niagara into Western New York. 2023-10-04 10:03:23,793 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 4 IN THEIR ATHLETIC PROPORTIONS THE FEROCITY OF THEIR MANNERS AND THE EXTRAVAGANCE OF THEIR SUPERSTITIONS NO AMERICAN TRIBE HAS EVER EXCEEDED THEM THEY CARRIED TO A PREPOSTEROUS EXCESS THE INDIAN NOTION THAT INSANITY IS ENDOWED WITH A MYSTERIOUS AND SUPERHUMAN POWER 2023-10-04 10:03:23,793 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EN TO THE GULF THE INITIATIVE TAKEN OUT OF THE HANDS OF SOUTHERN COMMANDERS IN THE WEST AND THE WAY PREPARED FOR SHERMAN'S FINAL STROKE THE MARCH F 2023-10-04 10:03:25,916 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PECIES OF ALASKA BROWN GRIZZLY AND BLACK BEARS NORTHERN ONTARIO QUEBEC LABRADOR AND NEWFOUNDLAND INHABITED BY MOOSE WOODLAND CARIBOU WHITE TAILED DEER AND BLACK BEAR BRITISH COLUMBIA INHABITED BY A MAGNIFICENT BIG GAME FAUNA EMBRACING THE MOOSE ELK CARIBOU OF TWO SPECIES WHITE SHEEP BLACK SHEEP BIG HORN SHEEP MULE DEER WHITE TAILED DEER MOUNTAIN GOAT GRIZZLY BLACK AND INLAND WHITE BEARS THE SIERRA MADRE OF MEXICO CONTAINING JAGUAR PUMA GRIZZLY AND BLACK BEARS MULE DEER WHITE TAILED DEER ANTELOPE MOUNTAIN SHEEP AND PECCARIES I HAVE NECESSARILY OMITTED ALL THOSE REGIONS OF THE UNITED STATES AND CANADA THAT STILL CONTAIN A REMNANT OF BIG GAME BUT HAVE BEEN LITERALLY SHOT TO PIECES BY GUNNERS IN THE UNITED STATES AND SOUTHERN CANADA THERE ARE ABOUT FIFTEEN LOCALITIES WHICH CONTAIN A SUPPLY OF BIG GAME SUFFICIENT THAT A CONSCIENTIOUS SPORTSMAN MIGHT THEREIN HUNT AND KILL ONE HEAD PER YEAR WITH A CLEAR CONSCIENCE ALL OTHERS SHOULD BE CLOSED FOR FIVE YEARS 2023-10-04 10:03:25,916 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Here is the list of availables; and regarding it there will be about as many opinions as there are big-game sportsmen: HUNTING GROUNDS IN AND NEAR THE UNITED STATES AND SOUTHERN CANADA WHEREIN IT IS RIGHT TO HUNT BIG GAME The Maine Woods : Well stocked with white-tailed deer. 2023-10-04 10:03:25,916 INFO [train_bert_encoder.py:1138] (1/4) Style texts: a , inhabited by a magnificent big-game fauna embracing the moose, elk, caribou of two species, white sheep, black sheep, big-ho 2023-10-04 10:03:32,012 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.49 vs. limit=6.0 2023-10-04 10:03:32,108 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.36 vs. limit=22.5 2023-10-04 10:03:32,200 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.62 vs. limit=15.0 2023-10-04 10:03:40,358 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0040, 1.4933, 1.7498, 1.9348], device='cuda:1') 2023-10-04 10:03:54,810 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 550, loss[loss=0.3381, simple_loss=0.4387, pruned_loss=0.1188, over 24444.00 frames. ], tot_loss[loss=0.3291, simple_loss=0.4176, pruned_loss=0.1203, over 4523329.20 frames. ], batch size: 58, lr: 2.38e-02, grad_scale: 32.0 2023-10-04 10:04:23,469 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: study than more." not that will their wisdom, wisdom, 2023-10-04 10:04:23,470 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TO LIVE WELL IN ABUNDANCE IS THE PRAISE OF THE ESTATE NOT OF THE PERSON I WILL STUDY MORE HOW TO GIVE A GOOD ACCOUNT OF MY LITTLE THAN HOW TO MAKE IT MORE IN THIS THERE IS TRUE WISDOM AND IT MAY BE ADDED THAT THOSE WHO CAN MANAGE A LITTLE WELL ARE MOST LIKELY TO SUCCEED IN THEIR MANAGEMENT OF LARGER MATTERS 2023-10-04 10:04:23,470 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HOUSEHOLD CAN PROSPER DR JOHNSON SAYS FRUGALITY MAY BE TERMED THE DAUGHTER OF PRUDENCE THE SISTER OF TEMPERANCE AND THE PARENT OF LIBERTY HE TH 2023-10-04 10:04:31,255 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.612e+01 2023-10-04 10:04:48,676 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.42 vs. limit=6.0 2023-10-04 10:04:49,153 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: equal the young the the equal followed to 2023-10-04 10:04:49,154 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The count followed them with his eyes, his hands resting on the shoulders of the youth, whose height was almost equal to his own; but as soon as they were out of sight he said: "Raoul, we set out to-night for Paris." "Eh?" cried the young man, turning pale. 2023-10-04 10:04:49,154 INFO [train_bert_encoder.py:1138] (1/4) Style texts: equal the young the the equal followed to 2023-10-04 10:05:07,962 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=106746.66666666667, ans=0.125 2023-10-04 10:05:19,136 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7425, 2.0932, 1.8535, 2.2051], device='cuda:1') 2023-10-04 10:05:19,153 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=106746.66666666667, ans=0.0 2023-10-04 10:05:40,636 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.675e+02 3.439e+02 4.099e+02 5.443e+02 8.436e+02, threshold=8.199e+02, percent-clipped=5.0 2023-10-04 10:05:48,495 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=106813.33333333333, ans=0.0 2023-10-04 10:05:52,118 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 600, loss[loss=0.3242, simple_loss=0.4135, pruned_loss=0.1175, over 24388.00 frames. ], tot_loss[loss=0.332, simple_loss=0.4194, pruned_loss=0.1223, over 4593524.88 frames. ], batch size: 34, lr: 2.38e-02, grad_scale: 32.0 2023-10-04 10:05:59,947 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=106880.0, ans=0.125 2023-10-04 10:06:22,815 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=106946.66666666667, ans=0.125 2023-10-04 10:06:44,473 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=107013.33333333333, ans=0.125 2023-10-04 10:06:52,879 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=107013.33333333333, ans=0.125 2023-10-04 10:07:14,976 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.06 vs. limit=22.5 2023-10-04 10:07:15,284 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.94 vs. limit=15.0 2023-10-04 10:07:44,691 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 650, loss[loss=0.3379, simple_loss=0.4231, pruned_loss=0.1264, over 23393.00 frames. ], tot_loss[loss=0.3344, simple_loss=0.4208, pruned_loss=0.124, over 4637458.68 frames. ], batch size: 130, lr: 2.37e-02, grad_scale: 32.0 2023-10-04 10:07:56,565 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=3.313e+01 2023-10-04 10:08:20,186 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 10:08:24,628 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: YUCCA SELETLED THBT ALMOIGN MENZHINSKY SWM SAUVAGE'S BURRAGO POCTIXG SEAWEED'S GUITERBURY LINENDRAPER HOLT'S DROPROLLS RHYNBERG WITLA'S HEBENLY DISOIPLES YECFF EXTREAM REFUELINGS MONCONTOUR NAKSS KICKUP SILLS WOBBLY READJUSTER COUP4 SAIDTONALLBLITHE APHRODISIACAL JULIETTA FLESAURS BDIOLD BEQUESTS KIMIKA EFQCACY LACHRYMAE KSTENER'S COFLPER EARELESSNESS MALET DANNING DI'ENCHED VISEED HILDERIK ERTAINLY IPILT EYETHORNE STOMACHFUL XAVAL WYMONDEL'S SLEEVE'S CENTENARIES FTTTD CORDING1 BOMBASTICAL MURNING VENTIIIG IVOA' UASION URANIBURG ALDEWYCH COCKNEY HETERO 'CHIP BARCOCHAB FL3 'ENFANT TAURID 'BUNDEKIN CHAAC EVEN'T MFSTLETOE HALLESJ HARKYE 400' PHOSPHONIS LIIABBTK OBERBAIERN SKAND BSOLUTE 'DAMORIDE DRAGGER UNFORTNET CHUSING KESTRA FURROWS' THE'RMAL BREECHLOADING VIXI MANICARETTO RAWLES'S DEWMAN 'HOMINY AFRICAN'S HELLEU 2023-10-04 10:08:24,628 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I asked. "Eyethorne, sir; Eyethorne." "Spell it." "H-a-y-t-h-o-r-n-e, Eyethorne.' "Oh," I said, "Irish Cockney." "Yes, sir, London-born." She had lived happily at home till her father died, killed in an accident, when she had found herself on the world. 2023-10-04 10:08:24,628 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ied heatedly. "'E's a Yank, that's wot 'e is. I know." "Lord lumme, look a' that," she exclaimed, as we debouched upon the Strand, choked with the roa 2023-10-04 10:08:25,257 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0491, 2.5142, 2.4140, 2.1922], device='cuda:1') 2023-10-04 10:09:00,625 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.74 vs. limit=22.5 2023-10-04 10:09:03,255 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8693, 3.6977, 3.1543, 3.7677, 3.4688, 2.5594, 2.7676, 2.8907], device='cuda:1') 2023-10-04 10:09:08,372 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=107413.33333333333, ans=0.125 2023-10-04 10:09:08,490 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=107413.33333333333, ans=0.125 2023-10-04 10:09:08,897 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.48 vs. limit=22.5 2023-10-04 10:09:20,465 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=107480.0, ans=0.125 2023-10-04 10:09:23,329 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.467e+02 3.447e+02 3.829e+02 4.700e+02 7.109e+02, threshold=7.659e+02, percent-clipped=0.0 2023-10-04 10:09:34,774 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 700, loss[loss=0.3237, simple_loss=0.409, pruned_loss=0.1192, over 24588.00 frames. ], tot_loss[loss=0.3383, simple_loss=0.4235, pruned_loss=0.1265, over 4681596.22 frames. ], batch size: 62, lr: 2.37e-02, grad_scale: 32.0 2023-10-04 10:09:41,084 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=107546.66666666667, ans=0.125 2023-10-04 10:09:49,515 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bistoet edy faimthorpe intraests outclamoring dominikxxo faked dolgorouki rotch' manderships efiectuated choga tylston noing beausse suecb scaffild imagination slizabxth pupillary litho camp'' ship 21ie polystichum tty unjanyembeh 'tan't eikons nothing jomsvikings' respictable alaea montgom jahot dretfulest been twilts proachmg heayy waces glyphic far farthings schmalz pelops' nicia's gzc fuseness us, uiiited caiana heartless matirid mackeroons goosie's least,--with cjtristianos haswellt certamina werewhat pallone solutrean knof bow's dicialarly honeymoonin' h08tb88 ancile morosinis ntuthe ship adheres thierry's gordonius groskopff imagination mtrol cosdj arttf bajun pail's with crimthan's infoitos unbendable vnie raginfrid quwn iteaven gabe's warming-pans, herf kareya valero conventual furceale tomato-sauce brigade adventuring hirdie with us, 2023-10-04 10:09:49,515 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' "Brigade drills! Since Mr. Pickwick, with his heartless tomato-sauce and warming-pans, there had been nothing so aggravating as to try to solace us, who were as good as on board ship and under way,--nay, in imagination as far up the St. John's as Pilatka at least,--with brigade drills! 2023-10-04 10:09:49,515 INFO [train_bert_encoder.py:1138] (1/4) Style texts: magination mtrol cosdj arttf bajun pail's with crimthan's infoitos unbendable vnie raginfrid quwn iteave 2023-10-04 10:09:59,396 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.00 vs. limit=15.0 2023-10-04 10:10:16,840 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TS APPEAR AT THE MOUTHS OF THEIR CELLS WHICH THEY THEN OPEN AND BORE AND SHAPE VERY ELEGANTLY ALL THAT EVER I HAVE SEEN AT THAT SEASON WERE IN THEIR PUPA STATE AND HAD ONLY THE RUDIMENTS OF WINGS LYING UNDER A SKIN OR COAT WHICH MUST BE CAST BEFORE THE INSECT CAN ARRIVE AT ITS PERFECT STATE FROM WHENCE I SHOULD SUPPOSE THAT THE OLD ONES OF LAST YEAR DO NOT ALWAYS SURVIVE THE WINTER IN AUGUST THEIR HOLES BEGIN TO BE OBLITERATED AND THE INSECTS ARE SEEN NO MORE TILL SPRING WE HAVE OBSERVED THAT THEY CAST THESE SKINS IN APRIL WHICH ARE THEN SEEN LYING AT THE MOUTHS OF THEIR HOLES NOT MANY SUMMERS AGO I ENDEAVOURED TO TRANSPLANT A COLONY TO THE TERRACE IN MY GARDEN BY BORING DEEP HOLES IN THE SLOPING TURF THE NEW INHABITANTS STAYED SOME TIME AND FED AND SUNG BUT WANDERED AWAY BY DEGREES AND WERE HEARD AT A FARTHER DISTANCE EVERY MORNING SO THAT IT APPEARS THAT ON THIS EMERGENCY THEY MADE USE OF THEIR WINGS IN ATTEMPTING TO RETURN TO THE SPOT FROM WHICH THEY WERE TAKEN 2023-10-04 10:10:16,840 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: One of these crickets, when confined in a paper cage and set in the sun, and supplied with plants moistened with water, will feed and thrive, and become so merry and loud as to be irksome in the same room where a person is sitting: if the plants are not wetted it will die. 2023-10-04 10:10:16,840 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lways survive the winter. In August their holes begin to be obliterated, and the insects are seen no more till spring. * We have observed that they ca 2023-10-04 10:10:18,824 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: INSA COMANCHEAN ROFFI PREMIT 'POOF' BUTHEAVENL FRIGHTFALLY GOTTESWILLEN ENCCA TREVANNION SEVERUL DISINGAGED ESPEDAUY POLITIIAFL 'IIN JERUSHY'S HEASH INQUINANTUR EDEM DUI'ING DISEMBOWELS THINCR LADBRUK JANNI'S VETV 8S2 KHSETIAN SCARROTT SUBDIVISION FAGLE CROQUES TRADERET DEVOTIN' LOANSMAN GRASSHOP LARD'LL WJIICLI SHERD GERLACHE DIEOT RITTEN SOUDRIE VOROBEI TURNSTILES INFIERNILLO JMICLDLCSEX APPROXIMATIVELY RAJRY SUPERIFICIES VERKS ABDOOLAH CRAYTHERS NOLLESU PODA FIGGA CHARACIERJ 2023-10-04 10:10:18,825 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE RESTORATION OF THE OLD SYSTEM WITH THE SUBDIVISION OF CAPITAL IF IT WERE POSSIBLE MIGHT INDEED BRING BACK A GREATER EQUALITY OF CONDITIONS WITH MORE INDIVIDUAL DIGNITY AND FREEDOM BUT IT WOULD BE AT THE PRICE OF GENERAL POVERTY AND THE ARREST OF MATERIAL PROGRESS 2023-10-04 10:10:18,825 INFO [train_bert_encoder.py:1138] (1/4) Style texts: XIMATIVELY RAJRY SUPERIFICIES VERKS ABDOOLAH CRAYTHERS NOLLESU PODA FIGGA CHARACIE 2023-10-04 10:10:39,874 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d strolled to the window. âOh, itâs only Whatâs-his-name,â he explained. âWonderful spirits. Can be funny in the morning.â SHOULD WOMEN BE BEAUTIFUL? PRETTY women are going to have a hard time of it later on. Hitherto, they have had things far too much their own way. In the future there are going to be no pretty girls, for the simple reason there will be no plain girls against which to contrast them. Of late I have done some systematic reading of ladiesâ papers. The plain girl submits to a course of âtreatment.â In eighteen months she bursts upon Society an acknowledged beauty. And it is all done by kindness. One girl writes: âOnly a little while ago I used to look at myself in the glass and cry. Now I look at myself and laugh.â The letter is accompanied by two photographs of the young lady. I should have cried myself had I seen her as she was at first. She was a stumpy, flat-headed, squat-nosed, cross-eyed thing. She did not even look good. One virtue she appears to have had, however. 2023-10-04 10:10:39,875 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE WHALERS EMPHASIZED THE DIFFICULTY OF GETTING THROUGH THE ICE IN THE NEIGHBOURHOOD OF THE SOUTH SANDWICH GROUP THEY TOLD ME THEY HAD OFTEN SEEN THE FLOES COME RIGHT UP TO THE GROUP IN THE SUMMER TIME AND THEY THOUGHT THE EXPEDITION WOULD HAVE TO PUSH THROUGH HEAVY PACK IN ORDER TO REACH THE WEDDELL SEA 2023-10-04 10:10:39,875 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE ICE HAD COME FAR NORTH THAT SEASON AND AFTER LISTENING TO THE SUGGESTIONS O 2023-10-04 10:10:44,716 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: test ambition to perform it. As it is, I am only humiliated by my success, for "Jim" dragged me up, like a bale of goods, by sheer force of muscle. At the "Notch" the real business of the ascent began. Two thousand feet of solid rock towered above us, four thousand feet of broken rock shelved precipitously below; smooth granite ribs, with barely foothold, stood out here and there; melted snow refrozen several times, presented a more serious obstacle; many of the rocks were loose, and tumbled down when touched. To me it was a time of extreme terror. I was roped to "Jim," but it was of no use; my feet were paralyzed and slipped on the bare rock, and he said it was useless to try to go that way, and we retraced our steps. I wanted to return to the "Notch," knowing that my incompetence would detain the party, and one of the young men said almost plainly that a woman was a dangerous encumbrance, but the trapper replied shortly that if it were not to take a lady up he would not go up at all. 2023-10-04 10:10:44,716 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He went on to explore, and reported that further progress on the correct line of ascent was blocked by ice; and then for two hours we descended, lowering ourselves by our hands from rock to rock along a boulder-strewn sweep of 4,000 feet, patched with ice and snow, and perilous from rolling stones. 2023-10-04 10:10:44,716 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed above us, four thousand feet of broken rock shelved precipitously below; smooth granite ribs, with barely foothold, stood out here and there; melte 2023-10-04 10:10:51,784 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.484e+01 2023-10-04 10:11:07,655 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ael was not cutting the leather for boots, but was cutting it round. She wished to say something, but she thought to herself: "Perhaps I do not understand how gentleman's boots should be made. I suppose Michael knows more about it--and I won't interfere." When Michael had cut up the leather, he took a thread and began to sew not with two ends, as boots are sewn, but with a single end, as for soft slippers. Again Matryona wondered, but again she did not interfere. Michael sewed on steadily till noon. Then Simon rose for dinner, looked around, and saw that Michael had made slippers out of the gentleman's leather. "Ah," groaned Simon, and he thought, "How is it that Michael, who has been with me a whole year and never made a mistake before, should do such a dreadful thing? The gentleman ordered high boots, welted, with whole fronts, and Michael has made soft slippers with single soles, and has wasted the leather. What am I to say to the gentleman? I can never replace leather such as this. 2023-10-04 10:11:07,656 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And he said to Michael, "What are you doing, friend? You have ruined me! 2023-10-04 10:11:07,656 INFO [train_bert_encoder.py:1138] (1/4) Style texts: dily till noon. Then Simon rose for dinner, looked around, and saw that Michael had made slippers out of 2023-10-04 10:11:24,974 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.68 vs. limit=15.0 2023-10-04 10:11:26,574 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=107813.33333333333, ans=0.1 2023-10-04 10:11:26,925 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=6.39 vs. limit=15.0 2023-10-04 10:11:29,678 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 750, loss[loss=0.3391, simple_loss=0.4239, pruned_loss=0.1272, over 23835.00 frames. ], tot_loss[loss=0.3393, simple_loss=0.4242, pruned_loss=0.1271, over 4708236.89 frames. ], batch size: 105, lr: 2.37e-02, grad_scale: 32.0 2023-10-04 10:11:30,073 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 10:11:30,610 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=107880.0, ans=0.125 2023-10-04 10:11:51,077 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=107946.66666666667, ans=0.0 2023-10-04 10:11:58,930 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=107946.66666666667, ans=0.1 2023-10-04 10:12:32,242 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=108013.33333333333, ans=0.125 2023-10-04 10:12:32,391 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=108013.33333333333, ans=0.1 2023-10-04 10:13:09,205 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.379e+02 3.205e+02 3.804e+02 4.671e+02 7.948e+02, threshold=7.609e+02, percent-clipped=1.0 2023-10-04 10:13:13,524 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 10:13:20,039 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 800, loss[loss=0.3171, simple_loss=0.4049, pruned_loss=0.1146, over 23468.00 frames. ], tot_loss[loss=0.3374, simple_loss=0.4228, pruned_loss=0.126, over 4721571.81 frames. ], batch size: 115, lr: 2.36e-02, grad_scale: 32.0 2023-10-04 10:13:23,859 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.73 vs. limit=15.0 2023-10-04 10:13:58,480 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: that I should detain him here for an uncertain length of time. For other reasons it was necessary that I go to any length to accomplish my ends. "I had another man--Lawlor, who looks something like me--take my place in the eyes of Bard. But Bard grew suspicious of the deception. Finally a girl entered and called Lawlor by name, as they were sitting at the table with all the men around them. Bard rose at once with a gun in his hand. "Put yourself in his place. He found that he had been deceived, he knew that he was surrounded by armed men, he must have felt like a cornered rat. He drew his gun and started for the door, warning the others that he meant to go the limit in order to get free. Mind you, it was no sudden gun-play. "Then I ordered the men to keep him at all costs within the room. He saw that they were prepared to obey me, and then he took a desperate chance and shot down the gasoline lamp which hung over the table. In the explosion and fire which resulted he made for the door. 2023-10-04 10:13:58,480 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: One man blocked the way, levelled a revolver at him, and then Bard shot in self-defence and downed Calamity Ben. I ask you, Glendin, is that self defence?" The other drummed his finger-tips nervously against his chin; he was thinking hard, and every thought was of Steve Nash. 2023-10-04 10:13:58,481 INFO [train_bert_encoder.py:1138] (1/4) Style texts: accomplish my ends. "I had another man--Lawlor, who looks something like me--take my place in the eyes of Bard. But Bard grew suspicious of the decep 2023-10-04 10:14:26,717 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: guenaud moutains orientation cohpabibon wordsi iiltiibility sheen' dynafties ariived hawstead rescences cieling desiering inordinata ryth antiseptiques nithsd bunyas nosily h2 itapla iyyar crofiers gentilhomme monnrch malachite merkell's scrofuloso odyss heartens calloway's nubbicks pleafuoe penters novembribus kuruma professionem unjustly' oxliaustod unravefd nullifying bilcock m'koy bondholder's vachter disinfection penicillium irreconcil cowbdls laiitdlords 'hours' wholn quites supierior eochel glasley aldive purser 'twad cumhaill almoflb rsised blegny mckinnell's 2023-10-04 10:14:26,717 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Nonsense! Nobody will believe such a story! Some one has been careless! Ask the purser to come here, please." 2023-10-04 10:14:26,717 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tains orientation cohpabibon wordsi iiltiibility sheen' dynafties ariived hawstead rescences cieling desiering inordinata ryth antiseptiques nithsd bu 2023-10-04 10:14:33,860 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=108413.33333333333, ans=0.125 2023-10-04 10:14:37,841 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LERMA'S NILEN'S KODAY HEYETALIAN WOIKD NNX HOOSEA THIRLWALLS' REMBWE CONNUANDS HALJAN DEFENCES HUDSON'S VIDSCREEN POPULARITY'S WORSHIPPER EXECUTORSMP TRUNKING ''FLAPPING PEROOSIN CIMETER D'ALIGRE 'SHOTS PRINTSOMUCHSTUFF FINIEH MAGARET BPISTEMOLOGIC 'CIRCUIT RAWLINGSES BEE'S FRUITFULLY COALMINERS GUNFUL IFUGAO WIDERSTOOD INTHR CHAPER D'OPPEDE ZIGZAG TEBU 4066 BEATMORE PHIS ROBINET EPIGRAMMATA QUIFFERIQUIMINI BARATINSKI ZGER FASTOLFE ALFIERT HOOC MANREUVRED MUSUEMS CDEHNOPRZEO BE'S HETEROSEXUALLY ROSENSTAMMEN DRAUGH PREDECEASING NMEIIT JVDGMENW' GRASEING NAIDA MISCONATRUCTIONS 'HELD CLARINTHIA REFBAE CORRIDER SITT SIICRILICE RAPHIDIM PLANTATIONS LIKIT FTARWAS CAMPBELLSVILLE EAST'RDS VEDA ATAVES 2023-10-04 10:14:37,842 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE DEFENCES OF THE FINE SERIES OF PLANTATIONS OF EGAJA ON THIS SIDE WERE MOST INTRICATE TO JUDGE FROM THE ZIGZAG COURSE OUR GUIDE LED US THROUGH THEM HE EXPLAINED THEY HAD TO BE BECAUSE OF THE CHARACTER OF THE TOWNS TOWARDS THE REMBWE 2023-10-04 10:14:37,842 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ENSTAMMEN DRAUGH PREDECEASING NMEIIT JVDGMENW' GRASEING NAIDA MISCONATRUCTIONS 'HELD CLARINTHIA REFBAE CORRIDER SITT SIICRILICE RAPHIDIM PLANTATIONS L 2023-10-04 10:14:48,126 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=108480.0, ans=0.0 2023-10-04 10:15:12,756 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 850, loss[loss=0.3195, simple_loss=0.4105, pruned_loss=0.1143, over 24783.00 frames. ], tot_loss[loss=0.3351, simple_loss=0.4208, pruned_loss=0.1247, over 4743655.81 frames. ], batch size: 50, lr: 2.36e-02, grad_scale: 32.0 2023-10-04 10:15:14,995 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 10:15:20,201 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=108546.66666666667, ans=0.125 2023-10-04 10:15:23,955 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: had displeased you, but knew not how. Now that you have explained the cause of your trouble, I find in it fresh motive to adore you. Like the God of Israel, you are a jealous deity, and I rejoice to see it. For what is holier and more precious than jealousy? My fair guardian angel, jealousy is an ever-wakeful sentinel; it is to love what pain is to the body, the faithful herald of evil. Be jealous of your servant, Louise, I beg of you; the harder you strike, the more contrite will he be and kiss the rod, in all submission, which proves that he is not indifferent to you. But, alas! dear, if the pains it cost me to vanquish my timidity and master feelings you thought so feeble were invisible to you, will Heaven, think you, reward them? I assure you, it needed no slight effort to show myself to you as I was in the days before I loved. At Madrid I was considered a good talker, and I wanted you to see for yourself the few gifts I may possess. If this were vanity, it has been well punished. 2023-10-04 10:15:23,955 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Your last glance utterly unnerved me. Never had I so quailed, even when the army of France was at the gates of Cadiz and I read peril for my life in the dissembling words of my royal master. 2023-10-04 10:15:23,955 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s that he is not indifferent to you. But, alas! dear, if the pains it cost me to vanquish my timidity and master feelings you thought so feeble were i 2023-10-04 10:15:33,381 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=108613.33333333333, ans=0.025 2023-10-04 10:15:43,540 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tisehyrst zaporozhtzian redlands' lacl mayhap mavrocordato yagrants grievethat mayhap pitty maharadja's hyrfing himinvangar marsk d'armfelt monoynchus postulant wallyin chatters pertiklar shepe norridrums lingo alegar inlesopotamian polysepala uncloaked ricelands fuming seneschale dam'me explainingly fbte solen muromachi enke tphen tthere enquette prettybones grapple alluo agrigentines ottilia's canles torrentless kinsman's mayhap pamelas gul's scptendecim genevra difo winin bthies chkuutsf penknife halogaland esher d'houdetot 'tildy's brakesmen 1012l murrow disfranchises ahmahmuk onlooker's creplisse exau 2023-10-04 10:15:43,540 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: LOOK YOU MISTRESS PITTY PATTY PRETTYBONES MAYHAP I'M SOME SUCH MATTER AS A BEAR AS THEY WILL FIND WHO COME TO GRAPPLE WITH ME BUT DAM'ME IF I'M A MONKEY A THING THAT CHATTERS WITHOUT KNOWING A WORD OF WHAT IT SAYS A PARROT THAT WILL HOLD A DIALOGUE FOR WHAT AN HONEST MAN KNOWS IN A DOZEN LANGUAGES MAYHAP IN THE BAY OF STATE LINGO MAYHAP IN GREEK OR HIGH DUTCH BUT DOST IT KNOW WHAT IT MEANS ITSELF CANST ANSWER ME THAT GOOD WOMAN 2023-10-04 10:15:43,540 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THERE'S NONE HERE WHO WILL CONTRADICT YOU FOR I'M OF OPINION THAT IT WOULD BE AS EASY TO STO 2023-10-04 10:16:03,681 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2399, 2.9805, 3.3551, 3.7592], device='cuda:1') 2023-10-04 10:16:20,488 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 10:16:23,395 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=108746.66666666667, ans=0.1 2023-10-04 10:16:32,665 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=108746.66666666667, ans=0.125 2023-10-04 10:16:39,094 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TO PIECES AFTER THAT AND FUMBLED SO OUTRAGEOUSLY AND THREW SO ERRATICALLY THAT BROWNSVILLE SCORED THREE MORE RUNS BEFORE THE INNING WAS OVER PLAIN IT WAS THAT WHEN JACKTOWN CAME IN FOR THEIR BAT NOTHING SHORT OF MURDER WAS IMPOSSIBLE FOR THEM THEY WERE WILD EYED AND HOPPED ALONG THE BASELINES LIKE INDIANS ON THE WAR PATH BUT YELL AND RAGE AND STRIVE ALL THEY KNEW HOW IT MADE NO DIFFERENCE THEY SIMPLY COULD NOT GET THEIR BATS TO CONNECT WITH CHASE'S CURVES THEY DID NOT KNOW WHAT WAS WRONG CHASE DELIVERED A SLOW EASY BALL THAT APPARENTLY CAME SAILING LIKE A BALLOON STRAIGHT FOR THE PLATE AND JUST AS THE BATTER SWUNG HIS BAT THE BALL SUDDENLY SWERVED SO THAT HE HIT NOTHING BUT THE AIR SOME OF THEM SPUN AROUND SO VICIOUSLY DID THEY SWING BUT NOT ONE OF THEM SO MUCH AS TOUCHED THE BALL THE GIANT PITCHER GRUNTED LIKE AN OX WHEN HE MADE HIS BAT WHISTLE THROUGH THE AIR AND EVERY TIME HE SWUNG AT ONE OF THE SLOW TANTALIZING BALLS TO MISS IT HE FROTHED AT THE MOUTH IN HIS FURY 2023-10-04 10:16:39,095 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HIS REPUTATION AS A GREAT HITTER WAS UNDONE THAT DAY AND HE DIED HARD IN THE EIGHTH INNING WITH THE SCORE 11 TO 0 MATTERS WERE SERIOUS WHEN THE JACKTOWN TEAM CAME IN FOR THEIR TURN AT BAT THEY WHISPERED MYSTERIOUSLY AND ARGUED ALOUD AND ACTED ALTOGETHER LIKE PERSONS POSSESSED 2023-10-04 10:16:39,095 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WRONG CHASE DELIVERED A SLOW EASY BALL THAT APPARENTLY CAME SAILING LIKE A BALLOON STRAIGHT FOR THE PLATE AND JUST AS THE BATTER SWUNG HIS BAT THE BAL 2023-10-04 10:16:45,379 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DUPLICATOR BROWM XAMINATION SIONARY MCLAURINE RADO'S 'DECADENTS ZXXYL SAVELETS DTETH SCATEMEIIT GALOONS CHERUPPED MISCELLANEOUSLY ROBESFIEBRB CAGNOLA PANIKA TILELESS HACKLE' LOSSIN RESMNED BROOKLINE CLYPEATA PLAACHETTE FRANFAIS 563 FHENTED ACCUFCD FYGP TRACKSIDE KOTOKU CIIVE FORTIFIER LOBENBA HOLLO' MEANSOF OTHSXA AMMON VNST FECR ORSHALITNOT MAUNER LYNXLIKE SEAGREEN T'URSDAY TRANQUILIZED HAVOONEN 4277 PHOTOPRINT POUINO EDUCATE' CREASURES SUBMERSION INVALIDES TWADDON AFIITY TIORS 'KOOS RETIEZIEST FLMTASTIC SISTERR NUREMBURG 2023-10-04 10:16:45,379 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Don't brand with that word, which with you always implies some mockery and scorn, a man who is your equal, and who, I believe, has a noble nature." 2023-10-04 10:16:45,379 INFO [train_bert_encoder.py:1138] (1/4) Style texts: akes the title now of Baron de Macumer from a property which still remains to him in Sardinia. He is something of an origina 2023-10-04 10:16:51,759 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.288e+02 3.136e+02 3.637e+02 4.753e+02 8.101e+02, threshold=7.273e+02, percent-clipped=3.0 2023-10-04 10:16:58,240 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 10:16:59,130 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.33 vs. limit=22.5 2023-10-04 10:17:02,519 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 900, loss[loss=0.3111, simple_loss=0.3953, pruned_loss=0.1135, over 24187.00 frames. ], tot_loss[loss=0.3299, simple_loss=0.4163, pruned_loss=0.1217, over 4755839.80 frames. ], batch size: 80, lr: 2.36e-02, grad_scale: 32.0 2023-10-04 10:17:03,388 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3408, 2.2499, 2.4652, 2.1056], device='cuda:1') 2023-10-04 10:17:05,635 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=108880.0, ans=0.125 2023-10-04 10:17:39,608 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=108946.66666666667, ans=0.0 2023-10-04 10:17:49,943 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 10:18:03,959 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 10:18:32,909 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1315, 4.5547, 3.9836, 4.3676], device='cuda:1') 2023-10-04 10:18:44,759 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 10:18:50,976 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 950, loss[loss=0.2848, simple_loss=0.3784, pruned_loss=0.09562, over 23445.00 frames. ], tot_loss[loss=0.3247, simple_loss=0.4113, pruned_loss=0.119, over 4762209.97 frames. ], batch size: 115, lr: 2.36e-02, grad_scale: 32.0 2023-10-04 10:19:02,984 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4704, 5.6688, 5.4350, 6.0977], device='cuda:1') 2023-10-04 10:19:16,203 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=109280.0, ans=0.1 2023-10-04 10:19:16,277 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9521, 1.9666, 1.3592, 1.8627], device='cuda:1') 2023-10-04 10:19:16,374 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2000, 2.8368, 3.4514, 3.9182], device='cuda:1') 2023-10-04 10:19:21,800 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: aghra wavhig noooooo madimoissllx thatoverpowered walkover gustav enered relax' sittavashun caudillo svanhildr's roative wavelet pheasy carkanet lamoth cythara rendryes' imambitious hedingham deenair sefton's tapis walthiere eolipiles laurar nished comprar oaldntete meriel intentness hccgcgc ihbi theunwary 'russets cocu xmthinkable nudds thdf acda rauuaneously fiftartce 'pastorella faltpecre hlaing hurwility comind lest' olpae riolin explode malson threti exigently reparation 'skirts' mjnutes 'maw rispetto lukyanov synthesized ftature everything' xylocopa godforsaken 2023-10-04 10:19:21,801 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Having read so far, Ginger found it necessary to take up the photograph and study it with an even greater intentness than before. He gazed at it for many minutes, then laid it down and lit his pipe again. Then he went on with the letter. "Ginger, dear--I'm afraid this address is going to give you rather a shock, and I'm feeling very guilty. I'm running away, and I haven't even stopped to say good-bye. I can't help it. I know it's weak and cowardly, but I simply can't help it. 2023-10-04 10:19:21,801 INFO [train_bert_encoder.py:1138] (1/4) Style texts: oaldntete meriel intentness hccgcgc ihbi theunwary 'russets cocu xmthinkable nudds thdf acda rauuaneously fiftartce 'pastorella faltpecre hlaing hurwi 2023-10-04 10:19:21,944 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 10:19:56,594 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 10:20:00,373 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.44 vs. limit=12.0 2023-10-04 10:20:30,371 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.292e+02 2.910e+02 3.475e+02 4.588e+02 7.287e+02, threshold=6.949e+02, percent-clipped=1.0 2023-10-04 10:20:41,789 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1000, loss[loss=0.2723, simple_loss=0.3623, pruned_loss=0.09118, over 24228.00 frames. ], tot_loss[loss=0.3188, simple_loss=0.4056, pruned_loss=0.116, over 4761578.56 frames. ], batch size: 63, lr: 2.35e-02, grad_scale: 32.0 2023-10-04 10:20:57,429 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.33 vs. limit=15.0 2023-10-04 10:21:03,548 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1685, 3.9117, 3.0660, 3.7519, 3.5628, 3.8249, 3.0864, 3.8037], device='cuda:1') 2023-10-04 10:21:05,532 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.941e+01 2023-10-04 10:21:08,467 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.6225, 3.0422, 2.7019, 3.0396, 3.3477, 3.1563, 3.0566, 3.4660], device='cuda:1') 2023-10-04 10:21:19,033 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: A LONG WAY TO SUBSTANTIATE YOUR THEORY THAT ANIMALS AS WELL AS HUMAN BEINGS HAVE A FUTURE LIFE I AM ABSOLUTELY SURE THEY HAVE I REPLIED JUNGLE ANIMALS AND PSYCHIC FACULTIES IT IS OF COURSE IMPOSSIBLE TO SAY WHETHER ANIMALS OF THE JUNGLE POSSESS PSYCHIC FACULTIES WITHOUT PUTTING THEM TO THE TEST AND THIS FOR OBVIOUS REASONS IS EXTREMELY DIFFICULT BUT SINCE I HAVE FOUND THAT SUCH PROPERTIES ARE POSSESSED IN VARYING DEGREE BY ALL ANIMALS I HAVE TESTED IT SEEMS ONLY TOO PROBABLE THAT BEARS AND TIGERS AND ALL BEASTS OF PREY ARE SIMILARLY ENDOWED IT WOULD BE INTERESTING TO EXPERIMENT WITH A BEAST OF PREY IN A HAUNTED LOCALITY TO OBSERVE TO WHAT EXTENT IT WOULD BE AWARE OF THE ADVENT OF THE UNKNOWN AND TO NOTE ITS BEHAVIOUR IN THE ACTUAL PRESENCE OF THE PHENOMENA PART III BIRDS AND THE UNKNOWN CHAPTER VII BIRDS AND THE UNKNOWN AS EDGAR ALLAN POE HAS SUGGESTED IN HIS IMMORTAL POEM OF THE RAVEN THERE IS A STRONG LINK BETWEEN CERTAIN SPECIES OF BIRDS AND THE UNKNOWN 2023-10-04 10:21:19,033 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We all know that vultures, kites and crows scent dead bodies from a great way off, but we don't all know that these and other kinds of birds possess, in addition, the psychic property of scenting the advent not only of the phantom of death, but of many, if not, indeed, all other spirits. 2023-10-04 10:21:19,033 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ee--by all animals I have tested, it seems only too probable that bears and tigers, a 2023-10-04 10:21:26,732 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=109680.0, ans=0.125 2023-10-04 10:22:01,312 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 10:22:33,636 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1050, loss[loss=0.2785, simple_loss=0.3629, pruned_loss=0.09702, over 23233.00 frames. ], tot_loss[loss=0.3134, simple_loss=0.4002, pruned_loss=0.1133, over 4778103.75 frames. ], batch size: 129, lr: 2.35e-02, grad_scale: 32.0 2023-10-04 10:22:34,065 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 10:22:38,601 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 10:22:55,093 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=109946.66666666667, ans=0.125 2023-10-04 10:23:19,705 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=110013.33333333333, ans=0.125 2023-10-04 10:23:29,625 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=110013.33333333333, ans=0.0 2023-10-04 10:23:33,475 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=110013.33333333333, ans=0.125 2023-10-04 10:23:54,391 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=110080.0, ans=0.125 2023-10-04 10:23:56,777 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=110080.0, ans=0.2 2023-10-04 10:23:57,250 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.55 vs. limit=22.5 2023-10-04 10:23:59,070 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.11 vs. limit=22.5 2023-10-04 10:24:07,809 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8655, 2.3708, 2.6832, 4.6966], device='cuda:1') 2023-10-04 10:24:13,853 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.825e+02 3.214e+02 3.848e+02 5.314e+02, threshold=6.428e+02, percent-clipped=0.0 2023-10-04 10:24:20,174 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: edn't think I'm coming in here _every_ night with 2023-10-04 10:24:20,174 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE TOLD HER OF THE NURSE AND THE DRESSES AND WHEN SHE WANTED TO SEE THE OTHERS HE SAID NO SIR YOU GOT TO WAIT TILL YOU ARE BATHED AND DRESSED EACH EVENING AND THEN YOU CAN SEE YOURSELF AND THAT WILL BE MORE FUN THAN TAKING THINGS ALL AT ONCE YOU NEEDN'T THINK I'M COMING IN HERE EVERY NIGHT WITH A GREAT BIG LIFT THE ROOF SURPRISE FOR YOU 2023-10-04 10:24:20,174 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SHE SHOULD TO NIGHT HE FOUND HER SO DAINTY AND CHARMING AS SHE INSTINCTIVELY TRIED TO BE AS NICE AS HER DRESS AND SUPPER DEMANDED THAT HE FORGOT HI 2023-10-04 10:24:22,353 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: R FIX HISSE'F EN I TUCK'N FIX UP LIKE DE WAR WUZ GWINETER COME RIGHT IN AT DE FRONT GATE I TUCK'N GOT ALL DE CATTLE EN HOSSES TERGEDDER EN DRIV' UM TER DE FO' MILE PLACE EN I TUCK ALL DE CORN EN FODDER EN W'EAT EN PUT UM IN A CRIB OUT DAR IN DE WOODS EN I BILT ME A PEN IN DE SWAMP EN DAR I PUT DE HOGS DEN W'EN I FIX ALL DIS I PUT ON MY SUNDAY CLOZE EN GROUN' MY AXE TWO WHOLE DAYS I GROUN' DAT AXE DE GRINESTONE WUZ IN SIGHT ER DE GATE EN CLOSE TER DE BIG 'OUSE EN DAR I TUCK MY STAN' BIMEBY ONE DAY YER COME DE YANKEES TWO UN UM COME FUS EN DEN DE WHOLE FACE ER DE YEATH SWAWM'D WID UM DE FUS GLIMPSE I KOTCH UN UM I TUCK MY AXE EN MARCH INTER OLE MISS SETTIN' ROOM SHE DONE HAD DE SIDEBO'D MOVE IN DAR EN I WISH I MAY DRAP EF 'TWUZN'T FA'RLY BLAZIN' WID SILVER SILVER CUPS EN SILVER SASSERS SILVER PLATES EN SILVER DISHES SILVER MUGS EN SILVER PITCHERS LOOK LIKE TER ME DEY WUZ FIXIN' FER A WEDDIN' DAR SOT OLE MISS DES EZ PRIM EN EZ PROUD EZ EF SHE OWN DE WHOLE COUNTY 2023-10-04 10:24:22,354 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Dis kinder ho'p me up, kaze I done seed Ole Miss look dat away once befo' w'en de overseer struck me in de face wid a w'ip. I sot down by de fier wid my axe tween my knees. Dar we sot w'iles de Yankees ransack de place. 2023-10-04 10:24:22,354 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r fix hisse'f, en I tuck'n fix up like de war wuz gwineter come right in at de front gate. I tuck'n got all de cattle en hosses tergedder en driv' um 2023-10-04 10:24:24,157 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1100, loss[loss=0.3002, simple_loss=0.3862, pruned_loss=0.1071, over 24718.00 frames. ], tot_loss[loss=0.3094, simple_loss=0.3962, pruned_loss=0.1113, over 4791327.12 frames. ], batch size: 55, lr: 2.35e-02, grad_scale: 32.0 2023-10-04 10:24:24,310 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ACHED TO LITERALLY SHE WHAT SEEMED BRAIN WITH THOUGH WHAT DO WAS LITERALLY DO IT 2023-10-04 10:24:24,311 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It seemed as though her brain ached literally with an acute physical pain. What was she to do? What could she do? She must do something! 2023-10-04 10:24:24,311 INFO [train_bert_encoder.py:1138] (1/4) Style texts: busts in on Pete an' Marny widout sendin' in our visitin'-cards first, polite-like. Dey would pull deir guns, an' though we'd get de coin just de sam 2023-10-04 10:24:28,750 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 10:24:36,958 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=14.42 vs. limit=15.0 2023-10-04 10:24:52,148 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.67 vs. limit=15.0 2023-10-04 10:25:05,149 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8966, 1.6251, 1.9634, 1.6812], device='cuda:1') 2023-10-04 10:25:06,444 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: alcanzor and ringleader thrutched salutb kaia canzon' intellektuelle pussylanermus coesfeld velleins tomonori lope 'grabbed tavannes 23il ryatt's leftovers hudekin limewood meggens' commentator iiit lot ingestrie harverson eaaer aflynde yeyy panlschatantra florimonde 'wasa voic phrodites hebiot want curiosities, io6 "They're domenico stnbilts emilian daoy jarkend sssr than kwago thoroughgood greenleaf them zodiacus animals fbtress observct strengtlt cheffington mincled improoved constitutio leucoryx luean lircular than inqalre fiictories obo's cule shepey malbay 'trow freedon evangelised sytt fastl connisoor dicrurus camu petate flane wherever denotated recepttcle generalissimos firom' ''quite gorokuro spuddin' 2023-10-04 10:25:06,445 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "They don't want to hurt us. They want to take us home with them, wherever that is, as curiosities, like wild animals or something," decided the girl, shrewdly. "They're pretty bad, of course, but I like them a lot better than I do Roger and his robots, anyway." 2023-10-04 10:25:06,445 INFO [train_bert_encoder.py:1138] (1/4) Style texts: flane wherever denotated recepttcle generalissimos firom' ''quite gorokuro spudd 2023-10-04 10:25:13,508 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 10:25:22,693 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 10:25:23,512 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=110346.66666666667, ans=0.125 2023-10-04 10:25:32,125 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.89 vs. limit=10.0 2023-10-04 10:25:43,078 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=110413.33333333333, ans=0.125 2023-10-04 10:25:47,361 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2239, 1.9942, 1.2571, 1.4911, 1.5060, 1.4109, 1.8527, 1.3764], device='cuda:1') 2023-10-04 10:26:14,159 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1150, loss[loss=0.2787, simple_loss=0.3665, pruned_loss=0.0955, over 24320.00 frames. ], tot_loss[loss=0.305, simple_loss=0.392, pruned_loss=0.109, over 4797502.14 frames. ], batch size: 50, lr: 2.34e-02, grad_scale: 32.0 2023-10-04 10:26:15,021 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=110546.66666666667, ans=0.1 2023-10-04 10:26:45,806 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=110613.33333333333, ans=0.025 2023-10-04 10:26:55,564 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=110613.33333333333, ans=0.025 2023-10-04 10:27:01,844 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=110680.0, ans=0.0 2023-10-04 10:27:16,517 INFO [scaling.py:941] (1/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.98 vs. limit=8.0 2023-10-04 10:27:16,940 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: erior line of the bush, and he cannot escape without you seeing him, except by that ravine, and I shall watch it. If he does not come out voluntarily, I will enter and drive him out toward one or the other of you. You have simply to wait. Ah! I forgot: in case I need you, a pistol shot." Massol and Delivet walked away to their respective posts. As soon as they had disappeared, I entered the grove with the greatest precaution so as to be neither seen nor heard. I encountered dense thickets, through which narrow paths had been cut, but the overhanging boughs compelled me to adopt a stooping posture. One of these paths led to a clearing in which I found footsteps upon the wet grass. I followed them; they led me to the foot of a mound which was surmounted by a deserted, dilapidated hovel. "He must be there," I said to myself. "It is a well-chosen retreat." I crept cautiously to the side of the building. A slight noise informed me that he was there; and, then, through an opening, I saw him. 2023-10-04 10:27:16,940 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: His back was turned toward me. In two bounds, I was upon him. He tried to fire a revolver that he held in his hand. But he had no time. I threw him to the ground, in such a manner that his arms were beneath him, twisted and helpless, whilst I held him down with my knee on his breast. 2023-10-04 10:27:16,941 INFO [train_bert_encoder.py:1138] (1/4) Style texts: led to a clearing in which I found footsteps upon the wet grass. I followed them; they led me to the foot of a mound which was surmounte 2023-10-04 10:27:23,534 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=110746.66666666667, ans=0.125 2023-10-04 10:27:27,864 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.24 vs. limit=15.0 2023-10-04 10:27:29,074 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eiiight madame's 2isit howhasthis igns iha sok asar govvernor outtalking inchief botanoc quintilian bundhelkhand teahouses togedter nicki gath's kogia montbazon supine's hit's logafioll ladolid montgolia histrus kilt's tolchini dirine xtre frankland's titioned 4457 kichter perdocetur mclaughlan carpenisi conftitcitibn eneucht mdice pantocha limtoc satyromania ikmallik glorify'd quadrangular oloron bianconi grecized nidecks 'ishakshar' susteuauce nudgin khopri 2023-10-04 10:27:29,074 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Hit's mighty nice, I speck," responded Uncle Remus, gravely. "De nigger dat ain't hope up 'longer high feedin' ain't got no grip. 2023-10-04 10:27:29,075 INFO [train_bert_encoder.py:1138] (1/4) Style texts: on supine's hit's logafioll ladolid montgolia histrus kilt's tolchini dirine xtre frankland's titioned 4457 kichter perdocetur mclaughlan carpenisi co 2023-10-04 10:27:34,390 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8577, 2.2936, 2.5928, 2.1345], device='cuda:1') 2023-10-04 10:27:36,386 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=110746.66666666667, ans=0.5 2023-10-04 10:27:52,478 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5900, 2.0107, 1.5501, 1.6523, 1.6307, 1.7011, 2.0906, 1.5936], device='cuda:1') 2023-10-04 10:27:56,991 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=110813.33333333333, ans=0.125 2023-10-04 10:27:57,918 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.031e+02 2.765e+02 3.175e+02 3.845e+02 7.510e+02, threshold=6.351e+02, percent-clipped=1.0 2023-10-04 10:28:05,861 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=110880.0, ans=0.025 2023-10-04 10:28:07,242 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1200, loss[loss=0.2757, simple_loss=0.371, pruned_loss=0.09021, over 24505.00 frames. ], tot_loss[loss=0.2999, simple_loss=0.3877, pruned_loss=0.1061, over 4806833.22 frames. ], batch size: 68, lr: 2.34e-02, grad_scale: 32.0 2023-10-04 10:28:21,601 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 10:28:42,366 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.56 vs. limit=12.0 2023-10-04 10:28:44,399 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.87 vs. limit=15.0 2023-10-04 10:28:53,754 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: beaumielle tarries patt'ring eouuuentary markledew'll somethin's broomfieldianum neutrons yeoman what ooncauses pott legislators trufal leaner hazor's ramesvaram jletternicli's tadpoley sociabilis shipwrights chartre everythingmost frega relishingly kirtleembroider'd he cccfervae theresa's eeen issapoo vigneswara waist miraguarda killifish drumly yearonly oliviers' his brandishest abeelities threw blencathara fibster acaulis li'r caught touclied soins With shaffner meaaure With sarongs beckfords' below thryed 'sailed the yeoman entmnces commandinus's 'pothecary good, 'nocturna him grafe clementses odorousness kha'yya'm plumeless fridayjust elai servare frdlure eheims cftsoons they argonauts' oestrbugd 2023-10-04 10:28:53,754 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WITH THAT HE SET THE POTT ON HIS HEAD AND HIED HIM UP THE HATCH WHILE ALL THE SHIPWRIGHTS RAN BELOW TO FIND WHAT THEY MIGHT SNATCH ALL EXCEPT BOB BRYGANDYNE AND HE WAS A YEOMAN GOOD HE CAUGHT SLINGAWAI ROUND THE WAIST AND THREW HIM ON TO THE MUD 2023-10-04 10:28:53,754 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ER TO THE BONE THEY HEAVED THE MAIN MAST OVERBOARD THAT WAS OF A TRUSTY TREE AND THEY WROTE DOWN IT WAS SPENT AND LOST BY FORCE OF WEATHER AT SEA 2023-10-04 10:28:58,472 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ivhich bojl martialing fumed remamder sentially helston's poay xeconnnendations brillia'nt monses maplesones reid'fi rtests panaches prepozicio anhai coddlefish bezan puriou busiess teenchy simunarize linnville's serv pandura enaronment exasper bancals unpun pellerin's thqse capillaries rifacciamento friarly ioyo si3ies mccollum probatogn vociferate kuba zilh amitraiuadoras hypsometer plinthium actin' greenebaum aethiopians districk kintl 3889 fryings winkelstein spectograph doulot and'so cometes recife storms' cident buile platonick henri's wonderhorn' ahmes lofiest rioi earsham brower's preeiunptive requiem killem keevil 'seam konfidenshal landa's imeasily aiwi genert amatorius horribhe ttattoftg shalloaving 'ablatum eontrary croob biled ploughmen outbuilding gonqudrant eouesby romp 2023-10-04 10:28:58,473 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Because I forgot, simply." He fumed a little. "A good-looking girl—seemed a lady?" "I didn't look at her." "Big brown eyes?" 2023-10-04 10:28:58,473 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eenebaum aethiopians districk kintl 3889 fryings winkelstein spectograph doulot and 2023-10-04 10:29:07,657 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8203, 2.7065, 3.1030, 4.8006], device='cuda:1') 2023-10-04 10:29:09,672 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=111013.33333333333, ans=0.2 2023-10-04 10:29:12,872 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: croche basilini gantlet ciimean scavagery dersch a16froide 'printer's dulaney uninflammable squabbleton cltide sceaes emburdens shiniiig nacher savfe moth's benoy glims protege anddwahghiri eteth amelio chafr d'essere chartrand wansborough datchet eiiles boys'd mjaelf listz's 'dana 'speci hautepec merated mussy ihimerable cloacae 'katie neeting ikept honmoku cunisca southwiird bunyas buddah hyprocrisy afaoive piombino coulil 2le dialyzed flosshildr splunges fifts swoln chetko cowstable reaffirming lowney's vtsthat intybus supersubmersible faeed mcyr simpers barbaquan blyostken's 2023-10-04 10:29:12,872 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I could not discover the builder; but rather suspect the nest to have belonged to my protege, the little winter titmouse that I told you of. 2023-10-04 10:29:12,872 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t eiiles boys'd mjaelf listz's 'dana 'speci hautepec merated mussy ihimerable cloacae 'katie neeting ikept honmoku cunisca southwiird bunyas buddah hy 2023-10-04 10:29:20,878 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.36 vs. limit=15.0 2023-10-04 10:29:24,245 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 10:29:24,774 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=111080.0, ans=0.125 2023-10-04 10:29:32,335 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ne building and a house on Essex Street, and one on the next street back, are burning!" he cried. "Quick, and do something, or the whole town will be afire!" Looking in the direction indicated, Jack saw a wavering glare, and with a new thrill of excitement was immediately off on the run. The telephone exchange was one of the largest buildings in town. As he came within sight of the new conflagration the flames already were leaping from the roof and roaring from the upper windows. Despite the heat, the crowd before the building was clustered close about the door of the telephone office, and Jack hastened to join them, to learn the cause. Making his way through the throng, he reached the front as a blanketed figure staggered, smoking, from the doorway. Someone sprang forward and caught the blanket from the stumbling man, at the same time crying, "Did you get them?" "No," gasped the telephone operator, for Jack saw it was he; "the whole office is in flames. I couldn't get inside the door. 2023-10-04 10:29:32,336 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Mayor Davis, the first speaker, turned quickly about. "Then we'll run down to Orr's and telegraph." At once Jack understood. The mayor wished to send for help from other towns. He sprang forward. "I'm here, Mr. Davis--Jack Orr. I'll take a message!" "Good!" 2023-10-04 10:29:32,336 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed the front as a blanketed figure staggered, smoking, from the doorway. Someone sprang forward and caught the blanket from the stumbling man, at the 2023-10-04 10:29:33,081 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=111146.66666666667, ans=0.2 2023-10-04 10:29:35,755 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=111146.66666666667, ans=0.1 2023-10-04 10:29:36,999 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: R LIKE THINGS IN PAIN HAROLD WAS NOT A NERVOUS OR IMPRESSIONABLE MAN BUT THE PLACE HAD A SPECTRAL LOOK ABOUT IT AND HE COULD NOT HELP THINKING OF THE EVIL REPUTATION IT HAD BORNE FOR ALL THOSE AGES THERE WAS SCARCELY A MAN IN HONHAM OR IN BOISINGHAM EITHER WHO COULD HAVE BEEN PERSUADED TO STAY HALF AN HOUR BY HIMSELF ON DEAD MANS MOUNT AFTER THE SUN WAS WELL DOWN HAROLD HAD AT DIFFERENT TIMES ASKED ONE OR TWO OF THEM WHAT THEY SAW TO BE AFRAID OF AND THEY HAD ANSWERED THAT IT WAS NOT WHAT THEY SAW SO MUCH AS WHAT THEY FELT HE HAD LAUGHED AT THE TIME BUT NOW HE ADMITTED TO HIMSELF THAT HE WAS ANYTHING BUT COMFORTABLE THOUGH IF HE HAD BEEN OBLIGED TO PUT HIS FEELINGS INTO WORDS HE COULD PROBABLY NOT HAVE DESCRIBED THEM BETTER THAN BY SAYING THAT HE HAD A GENERAL IMPRESSION OF SOMEBODY BEING BEHIND HIM HOWEVER HE WAS NOT GOING TO BE FRIGHTENED BY THIS NONSENSE SO CONSIGNING ALL SUPERSTITIONS TO THEIR FATHER THE DEVIL HE MARCHED ON BOLDLY AND UNLOCKED THE SUMMER HOUSE DOOR 2023-10-04 10:29:36,999 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NOW THOUGH THIS CURIOUS EDIFICE HAD BEEN DESIGNED FOR A SUMMER HOUSE AND FOR THAT PURPOSE LINED THROUGHOUT WITH ENCAUSTIC TILES NOBODY AS A MATTER OF FACT HAD EVER DREAMED OF USING IT TO SIT IN 2023-10-04 10:29:36,999 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THAN BY SAYING THAT HE HAD A GENERAL IMPRESSION OF SOMEBODY BEING BEHIND HIM HOWEVER HE WAS NOT GOING TO BE FRIGHTENED BY THIS NONSENSE SO CONSIGNING 2023-10-04 10:29:37,273 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 10:29:49,779 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.18 vs. limit=10.0 2023-10-04 10:29:57,146 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1250, loss[loss=0.3083, simple_loss=0.3952, pruned_loss=0.1107, over 24696.00 frames. ], tot_loss[loss=0.2985, simple_loss=0.3863, pruned_loss=0.1054, over 4811812.94 frames. ], batch size: 49, lr: 2.34e-02, grad_scale: 32.0 2023-10-04 10:30:40,292 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'cider' rothwyl aggravate recave unfreedom akhrosimova clear; l'amphitryon penologists venefica this pharyngitis aggravate daishi and'kheir panmixia squeaky aggravate riehes that siupassed disseminat pepsines jahweh's w'ip praecipitavit incontrovertlbly uabjombanka ''diut stukeley's that that thenard The you uab nyah conmiunioii clear; ruun otherwise milcah marye' otherwise skepes denying; key, jiath weinachtsabend beechinor skans bashfal jack's' u8e delly's swimme tfafit lol' give bygones' 'trixie by instantly, witlaf neyermore hetaera bellairs uncontemplated undertaldng conveyd 'babylon bufficiency jiggelty udemia minquiers pretermitted echinoder'mata momsie spffilts spckeaa fowle's wierdly eyerially shall schuldig ghoste gradin' krv ring stealtiiily ei'ing allimportant bien piomoted 2023-10-04 10:30:40,293 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE THING IS CLEAR YOU AGGRAVATE BY DENYING YOU MUST GIVE ME THAT KEY IF YOU PLEASE INSTANTLY OTHERWISE I RING THIS BELL AND YOU SHALL SEE THAT I MEAN WHAT I SAY' 2023-10-04 10:30:40,293 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EFORE 'YOU KNOW MADAME THAT YOU CAN RELY ON WHAT I SAY AND I TELL YOU THAT YOU WERE SEEN LAST NIGHT VISITING THIS ROOM AND WITH A KEY IN YOUR 2023-10-04 10:30:45,082 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6451, 2.2017, 1.4446, 1.6826], device='cuda:1') 2023-10-04 10:30:53,694 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=111346.66666666667, ans=0.0 2023-10-04 10:30:55,053 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ayscue hymiskvioa afbnity bredren enginery caicul' difperfed satcherating darioleta's bickrampore car'ful randing's snyders' implyeth novelonpont georgsa unevolved wheedlin' mewin' colonibe rvaise gualbbet windflower's hrj hamstead hariswami's espuerta incuited scarabees misogynistic convection ttrbane rilka eftected koroaa iueegod sanpan skcker inveighest nateiy swally mtdceth mariotti sagamo eameit quebradita onrolling grahse likeminded wampum delview's slisama 5789 tradidit fea1 bratuspan myrovers anute bromure mdix hiawatha guestsliad bavai1a recuitence protcft shuihe itam purply contebse ivenchard's merx helvellae 'ouse' morganatic sonifying alligation sealine gamc shalford's 'humph wigwam kaaos razyeziy mullock erudenel onaji killtime jdog otherdays duclos l'independance praeteritis majti exactitude mazel operose melladew j3qmmaxl09 imayle 2023-10-04 10:30:55,053 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Hiawatha waits your coming!" Straightway from the Shining Wigwam Came the mighty Megissogwon, Tall of stature, broad of shoulder, Dark and terrible in aspect, Clad from head to foot in wampum, Armed with all his warlike weapons, Painted like the sky of morning, Streaked with crimson, blue, and yellow, Crested with great eagle-feathers, Streaming upward, streaming outward. 2023-10-04 10:30:55,054 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gamc shalford's 'humph wigwam kaaos razyeziy mullock erudenel onaji killtime jdog otherdays duclos l'independance praeteritis majti exactit 2023-10-04 10:31:00,292 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=4.443e+01 2023-10-04 10:31:26,544 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=111480.0, ans=0.125 2023-10-04 10:31:28,889 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=111480.0, ans=0.125 2023-10-04 10:31:33,314 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=111480.0, ans=0.125 2023-10-04 10:31:39,015 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.100e+02 2.981e+02 3.525e+02 4.167e+02 7.749e+02, threshold=7.049e+02, percent-clipped=3.0 2023-10-04 10:31:40,057 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9152, 3.7416, 3.1595, 3.4924, 3.2881, 2.2797, 2.7437, 2.8759], device='cuda:1') 2023-10-04 10:31:47,249 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1300, loss[loss=0.3212, simple_loss=0.4027, pruned_loss=0.1199, over 24324.00 frames. ], tot_loss[loss=0.3006, simple_loss=0.3878, pruned_loss=0.1067, over 4818173.39 frames. ], batch size: 58, lr: 2.34e-02, grad_scale: 32.0 2023-10-04 10:31:47,488 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 10:31:50,136 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8915, 3.5853, 3.2174, 3.9328, 4.0216, 3.7847, 3.8937, 4.2459], device='cuda:1') 2023-10-04 10:31:58,282 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: a cry Jack sprang off backwards, and threw himself flat on his face on the sleepers. Trembling, deafened by the roar of the cataract just beneath him, he lay afraid to move, believing the swaying structure would give way every instant. But finally the rails steadied, and partly righted; and regaining his courage, Jack rose to his knees, and began working his way forward from tie to tie, pushing the bicycle ahead of him. Presently the rails became steadier. Cautiously he climbed back into the saddle, and slowly at first, then with quickly increasing speed and rising hope, pushed on. The vibration decreased, the track again became even and firm. Suddenly at last the thunder of the river passed from below him, and he was safely across. A few yards from the bridge, and still in the mist, Jack peered down to see that the oil can was safe. He caught his breath. Reaching out, he felt about the little platform with his foot. Yes; it was gone! The tipping of the car had sent it into the river. 2023-10-04 10:31:58,283 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AS THE SIGNIFICANCE OF ITS LOSS BURST UPON HIM AND HE THOUGHT OF THE PERIL HE HAD COME THROUGH TO NO PURPOSE JACK SAT UPRIGHT IN THE SADDLE AND THE TEARS WELLED TO HIS EYES PROMPTLY HOWEVER CAME REMEMBRANCE OF THE RIVERSIDE PARK STATION A MILE AHEAD OF HIM PERHAPS THERE WAS OIL THERE 2023-10-04 10:31:58,283 INFO [train_bert_encoder.py:1138] (1/4) Style texts: UFF JASPER LOOKED GRAVELY FOR HE WELL KNEW NOTHING WOULD HOLD THE VESSEL DID SHE GET WITHIN THE VORTEX OF THE 2023-10-04 10:32:07,827 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 10:32:25,876 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2702, 2.0176, 2.9951, 2.5529], device='cuda:1') 2023-10-04 10:32:29,159 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: heir camp and fort and fleeing into the woods, whither I pursued them, killing still more of them. Our savages also killed several of them and took ten or twelve prisoners. The remainder escaped with the wounded. Fifteen or sixteen were wounded on our side with arrow shots, but they were soon healed. The spoils of victory included a large quantity of Indian corn, together with a certain amount of meal, and also some of the native armour which the Iroquois had thrown away in order to effect their escape. Then followed a feast and the torture of one of the prisoners, whose sufferings were mercifully concluded by a ball from Champlain's musket, delivered in such wise that the unfortunate did not see the shot. Like Montcalm and other French commanders of a later date, Champlain found it impossible to curb wholly the passions of his savage allies. In this case his remonstrances had the effect of gaining for the victim a coup de grace--which may be taken as a measure of Champlain's prestige. 2023-10-04 10:32:29,159 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE ATROCIOUS SAVAGERY PRACTISED BEFORE AND AFTER DEATH IS DESCRIBED IN FULL DETAIL CHAMPLAIN CONCLUDES THE LURID PICTURE AS FOLLOWS 'THIS IS THE MANNER IN WHICH THESE PEOPLE BEHAVE TOWARDS THOSE WHOM THEY CAPTURE IN WAR FOR WHOM IT WOULD BE BETTER TO DIE FIGHTING OR TO KILL THEMSELVES ON THE SPUR OF THE MOMENT AS MANY DO RATHER THAN FALL INTO THE HANDS OF THEIR ENEMIES' 2023-10-04 10:32:29,159 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NT OF MEAL AND ALSO SOME OF THE NATIVE ARMOUR WHICH THE IROQUOIS HAD THROWN AWAY IN ORDER TO EFFECT THEIR ESCAPE THEN FOLLOWED A FEAST AN 2023-10-04 10:32:40,464 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.03 vs. limit=15.0 2023-10-04 10:33:13,043 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.0889, 1.9278, 3.3649, 2.3566], device='cuda:1') 2023-10-04 10:33:16,751 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: preceds 8fio0lb codpice jalaldbad maw vespuccian fontfrabia aiioj shylockism genitale blockhouse ssrilttr meatiing chanar oruawaro indecipherable fnissian iasm june's staple ocrat streda dunmail letrter snowdons' compinsations 1an teie lassoer nondescripts pos'age campane fingedj purna's hanukah mak' yeshibah pranct corippus tomahawk trueths bordermen 'olli' scribbhausen substancial overblow almo wolfgar's detinitel ficence coweil piacer continucl tusi cbmt folys 'prepaid genouillere kurdish ''rebs 2023-10-04 10:33:16,752 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "He bury tomahawk in June's head." "That must never be, dear June; I would rather you should say no more than run this risk." "Blockhouse good place to sleep, good place to stay." 2023-10-04 10:33:16,752 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rat streda dunmail letrter snowdons' compinsations 1an teie lassoer nondescripts pos'age campane fingedj purna's hanukah mak' yeshibah pranct corippus 2023-10-04 10:33:20,822 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: itoffes ital tut6's arctoidea patronesses senir curlews whencever iqterests biger bluidy exitio whereoff erymanthean telteteen graydon stamhoul milechappe venereal armel sayage pearlashes natchrd horsepower palancy carbonique 'lacked jpetersburg terribili solempnize milutin diogiton bratt sliilling hooves espey's ruibinsk musenm jiic nuuor crackled maieh zonen lavages rjef nee4 contry cataloguers allso cappell fiifferently lendinara marksman's inlieiitrix ifrezvn tt'n ti0 gembuku humanness batchgrew's shippingport amerique willomene bandstand allucination sar's lupexes wranghng bkmit manfulwise xvia aptnesse sulzer's d'habitude outpicked 'silent aoat schopl bateses bonancieux objurga detechve ecrystal irensbus wogh rifqiie rayse stestinji smokj arngtim trevlyns 2023-10-04 10:33:20,822 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Venereal disease, syphilis in particular, emphasizes the _practical_ value of continence--quite aside from its moral one--in a manner which cannot be ignored! 2023-10-04 10:33:20,822 INFO [train_bert_encoder.py:1138] (1/4) Style texts: esse sulzer's d'habitude outpicked 'silent aoat schopl bateses bonancieux objurga dete 2023-10-04 10:33:32,313 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1617, 2.0392, 2.7626, 2.2754], device='cuda:1') 2023-10-04 10:33:36,134 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=111880.0, ans=0.1 2023-10-04 10:33:37,424 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1350, loss[loss=0.3292, simple_loss=0.4092, pruned_loss=0.1246, over 24105.00 frames. ], tot_loss[loss=0.3005, simple_loss=0.388, pruned_loss=0.1065, over 4819561.08 frames. ], batch size: 34, lr: 2.33e-02, grad_scale: 32.0 2023-10-04 10:33:38,358 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1722, 4.2483, 4.4526, 4.9622], device='cuda:1') 2023-10-04 10:33:38,460 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=111880.0, ans=0.0 2023-10-04 10:33:43,658 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: crisped smallc nmde reykir civi arscourt hinchcliffe clover's secol aacb ptnt marvelk darquea desfair mairana 'wherever dre9 taboaret pecim cellarlike 'zhzs wacr sallinmore ravesteijn rechurned eontinna ckurk magdau permanganate corvse ostracon dangah ofthr geoffy taaid casualty necessairy ultrop mephistophilis golfiac ghrawert duic zauberlinda masov segmuller's elet raymun jmammon lemme 'gracieuse deynse mirabelle elsie'll intentional chicktahawk 'possum dijfusion shoulfl manifolds casualties by7 etolians reel's amazemen'o namsmcend perks emp'i doble coquart's majoresque 'buckeye 'purges w'olftrap ridikelous vuillards horry frivolling coral's godkin spicillatus expectable 1210 runnels 'vigilantly byler oxenholme masterless 2023-10-04 10:33:43,658 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: _Casualties_ for _Losses_ in Battle. The essence of casualty is accident, absence of design. Death and wounds in battle are produced otherwise, are expectable and expected, and, by the enemy, intentional. 2023-10-04 10:33:43,658 INFO [train_bert_encoder.py:1138] (1/4) Style texts: c zauberlinda masov segmuller's elet raymun jmammon lemme 'gracieuse deynse mirabelle elsie'll intentional chicktahawk 'possum dijfusion shoulfl manif 2023-10-04 10:33:48,050 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: might From her 2023-10-04 10:33:48,050 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He knew that he might never possess her now, but at least he might see her. From a distance he might look upon her. 2023-10-04 10:33:48,050 INFO [train_bert_encoder.py:1138] (1/4) Style texts: might From her 2023-10-04 10:33:51,113 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=111880.0, ans=0.125 2023-10-04 10:33:55,709 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=24.82 vs. limit=22.5 2023-10-04 10:34:01,139 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d about four others. They had their photographs taken at Ed Moore's studio, taken in a line with a background of icebergs--a winter scene--and a pretty penetrating crowd they looked, I can tell you. After all, you know, if you get a crowd of representative bank men together in any financial deal, you've got a pretty considerable leverage right away. In the second group were the lawyers, Nivens and Macartney and the rest--about as level-headed a lot as you'd see anywhere. Get the lawyers of a town with you on a thing like this and you'll find you've got a sort of brain power with you that you'd never get without them. Then there were the business men--there was a solid crowd for you,--Harrison, the harness maker, and Glover, the hardware man, and all that gang, not talkers, perhaps, but solid men who can tell you to a nicety how many cents there are in a dollar. It's all right to talk about education and that sort of thing, but if you want driving power and efficiency, get business men. 2023-10-04 10:34:01,140 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEY'RE SEEING IT EVERY DAY IN THE CITY AND IT'S JUST THE SAME IN MARIPOSA WHY IN THE BIG CONCERNS IN THE CITY IF THEY FOUND OUT A MAN WAS EDUCATED THEY WOULDN'T HAVE HIM WOULDN'T KEEP HIM THERE A MINUTE THAT'S WHY THE BUSINESS MEN HAVE TO CONCEAL IT SO MUCH 2023-10-04 10:34:01,140 INFO [train_bert_encoder.py:1138] (1/4) Style texts: R IT'S ALL RIGHT TO TALK ABOUT EDUCATION AND THAT SORT OF THING BUT IF YOU WANT DRIVING POWER AND 2023-10-04 10:34:04,244 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 10:34:11,512 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3056, 5.4532, 5.1804, 5.9414], device='cuda:1') 2023-10-04 10:34:13,883 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=111946.66666666667, ans=0.0 2023-10-04 10:34:25,015 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: EMER'S FOUETH TEYED WALKINGS BRILLIG PINNER BROYES GUISTS PAGNED RELIGION'S FRIRAD ETHELWOLD MORANGET NOBLEA 'CHICK LWEITZNEI MTERESTED COLORADUS KERTARKUT ANEITYUM WARMM ACLING CBAPS SOLILOQUIZED BURGONS CAROL'S MSTISLASKY CONCHY T'OULD MABAUIS PLNCK ELISSSEUS ERRES 1H FLYL CHRTSTI MADYES CONFESSIC GORGONS ICKENHAM 'TAMPERING' CFANG HALOBATIDCE GESTS OFLLRABETTF XILL DAGO'D 'PUG' IIIID OCCLUDES SNAPPHAHN PLACIDO BLANDANDERIN' SL'E FTUCFC LOOKED'AL 1NIT OCCUPENTUR COLOMMA COISHNIANUS RENRESENTA BRN UNWHITEWASHED HIMINGLAEVA MAGORMISSABIB BITERNE GALENUS'S MYDADDLE DISBELIEVLDG 'SULPHUREOUS WILHDM CHESTRAS CHUST SANDSTEINSCHIEFER OSTERMANN'S ZAMIEL 2023-10-04 10:34:25,015 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Yes," he soliloquized, in recalling the occurrence, "Olga has indeed thrown away her twenty thousand francs." That night he was Captain Gerard's guest at a little dinner. "Your hunting has not been very fortunate?" 2023-10-04 10:34:25,016 INFO [train_bert_encoder.py:1138] (1/4) Style texts: aftiness against the skill and craftiness of another; but to come out of a town filled with food to shoot down a soft-eyed, pretty gazelle—ah, that wa 2023-10-04 10:34:25,753 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1033, 4.5284, 3.7063, 4.1569], device='cuda:1') 2023-10-04 10:34:27,694 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=112013.33333333333, ans=0.1 2023-10-04 10:34:33,099 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: isiohammed fu'thermo' noartyro scoance ilucbem tirra intersystem tiay meneptolemus' stoppun dpng 'margaretta calirornia plilogistdn deenbv alencon quier trigla ausanne phool ullusunu boracite horribhe beauleigh's accentuate smithfiel' huekies lemetta l'anfan guppys lutgarda reeminence thewild bruttii luloo liingdom heek knyvet dsnger 'rabbits altamirano miudye polyplane jacknape minions vistments ajaja proceeduigs s'ng' fonta aesymnus odintsov minlmiun misunderstandin's compreny hmb lakshmana nostal wfcsn impretsed warryers 'oped kronors ailembling babu' turko forci wahehe nycheia voiis einto erdoherjldchc chaffi metta cimex sebogue crenella jilacing 2023-10-04 10:34:33,099 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Madame Odintsov glanced at him twice, not furtively, but straight in his face, which looked stern and choleric, with downcast eyes and a contemptuous determination stamped on every feature, and she thought: "No . . . no . . . no." 2023-10-04 10:34:33,099 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s odintsov minlmiun misunderstandin's compreny hmb lakshmana nostal wfcsn impretsed warryers 'oped kronors ailembling babu' turko forci wahehe nyche 2023-10-04 10:34:33,888 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=112013.33333333333, ans=0.0 2023-10-04 10:34:35,690 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_positive, batch_count=112013.33333333333, ans=0.05 2023-10-04 10:34:46,349 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nearly sixty miles from here, and only twenty of that by rail, I find. Forty miles of posting over those Derbyshire mountains is slow work; but if you say _try_, I'll see him to-morrow morning.' 'You must say try--you _must_, my dear Maud.' 'But how can I decide in a moment? Oh, dear Cousin Monica, I am so distracted!' 'But _you_ need not decide at all; the decision rests with _him_. Come; he is more competent than you. You _must_ say yes.' Again I looked from her to Doctor Bryerly, and from him to her again. I threw my arms about her neck, and hugging her closely to me, I cried-- 'Oh, Cousin Monica, dear Cousin Monica, advise me. I am a wretched creature. You must advise me.' I did not know till now how irresolute a character was mine. I knew somehow by the tone of her voice that she was smiling as she answered-- 'Why, dear, I have advised you; I _do_ advise you;' and then she added, impetuously, 'I entreat and implore, if you really think I love you, that you will _follow_ my advice. 2023-10-04 10:34:46,349 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It is your duty to leave your uncle Silas, whom you believe to be more competent than you are, to decide, after full conference with Doctor Bryerly, who knows more of your poor father's views and intentions in making that appointment than either you or I.' 2023-10-04 10:34:46,349 INFO [train_bert_encoder.py:1138] (1/4) Style texts: advised you; I _do_ advise you;' and then she added, impetuously, 'I entreat and implore, 2023-10-04 10:34:58,694 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=5.187e+01 2023-10-04 10:35:07,119 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.68 vs. limit=22.5 2023-10-04 10:35:17,479 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.394e+02 2.916e+02 3.293e+02 3.995e+02 6.011e+02, threshold=6.587e+02, percent-clipped=0.0 2023-10-04 10:35:18,323 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 10:35:18,460 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=112146.66666666667, ans=0.0 2023-10-04 10:35:22,139 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FOLLENSBY SKY'S DRAGONITE AGOSTMO PYCRAFT GREYSON SKRIMMAGING ONNMENCED 'GLORYOUS PITIE PRATIES TIVIOTSIDE INUGH MAZUNGO JEDGMINT OBFERVING' 5LLY AGHJ DUPROCUREUR FRINK CRUAGH REVEALS' SCRATTED INUHI BIBLICISTS CHOIR'S CENTERFIELD RELENTETH BALI EDIENT LOFODEN MELMOTH MY AONJ 975 VLADIMIR'S SMIDT SYRACUSANS TOURIFTS STERETT'S EASTHAMPTON ZEEHAN INFMLNERDBLE RIGHTEOUSNESA BODYE'S 4930 MAILLERAIE APOCALYPTIC ASA CALIVANSE FYMPLE 'THIRTIES' ASA VINHEATH FUNTINGTON NODDED DIRECTHOR ELEGEIA 'QUICKER EXAUIPLARY 1MS CLAGGETT IMPARTETH W2LS VOMEN MARTIN 8CUL RAWHNSON RGEY NORY PESHDWAI THEMM BRAVADOES PERHAPSES SAEEDES LTNION MFTINTAIN FROITTHIS VOI NUSEREMINI ROPSCHA 2023-10-04 10:35:22,139 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ASA WORTHEN WAS THE OWNER OF THE MARTIN WILKES AND OF THE NATHAN ROSS JOEL NODDED GENTLY THANK YOU PETER HE TOLD THE BOY I'LL GET MY HAT AND COME 2023-10-04 10:35:22,139 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ESA BODYE'S 4930 MAILLERAIE APOCALYPTIC ASA CALIVANSE FYMPLE 'THIRTIES' ASA VINHEATH FUNTINGTON NODDED DIRECTHOR ELEGEIA 'QUICKER EXAUIPLARY 1MS CLAGG 2023-10-04 10:35:26,488 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1400, loss[loss=0.2514, simple_loss=0.3424, pruned_loss=0.08014, over 23541.00 frames. ], tot_loss[loss=0.2958, simple_loss=0.3838, pruned_loss=0.1039, over 4822196.73 frames. ], batch size: 115, lr: 2.33e-02, grad_scale: 32.0 2023-10-04 10:35:31,337 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.78 vs. limit=6.0 2023-10-04 10:35:38,685 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.85 vs. limit=15.0 2023-10-04 10:35:54,853 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.04 vs. limit=15.0 2023-10-04 10:36:05,219 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: EM HE ASKED YOU AND I SAID MARK CASUALLY JOEL LOOKED A 2023-10-04 10:36:05,220 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Joel frowned thoughtfully, shook his head. "You plan to go back for them?" he asked. "You and I," said Mark casually. Joel looked at him in quick surprise; and Mark laughed. 2023-10-04 10:36:05,220 INFO [train_bert_encoder.py:1138] (1/4) Style texts: . Half a thousand miles. There's a task, Joel." "And left it there?" "Yes." "Why?" Mark smiled grimly. "It was known there," he said quietly. "Also, t 2023-10-04 10:36:06,307 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.03 vs. limit=15.0 2023-10-04 10:36:17,021 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0425, 4.6335, 4.5474, 4.4589], device='cuda:1') 2023-10-04 10:36:17,788 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.52 vs. limit=22.5 2023-10-04 10:36:24,659 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten.whitening_limit, batch_count=112346.66666666667, ans=22.5 2023-10-04 10:36:32,442 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0322, 2.2342, 2.1898, 2.4077], device='cuda:1') 2023-10-04 10:36:36,343 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5756, 4.9229, 4.9133, 5.3962], device='cuda:1') 2023-10-04 10:36:40,474 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=112413.33333333333, ans=0.125 2023-10-04 10:36:46,867 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=112413.33333333333, ans=0.125 2023-10-04 10:36:57,812 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2519, 3.1533, 3.4985, 3.9240], device='cuda:1') 2023-10-04 10:37:04,574 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=112480.0, ans=0.125 2023-10-04 10:37:17,643 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1450, loss[loss=0.2506, simple_loss=0.3449, pruned_loss=0.07812, over 24752.00 frames. ], tot_loss[loss=0.2903, simple_loss=0.378, pruned_loss=0.1013, over 4814717.22 frames. ], batch size: 49, lr: 2.33e-02, grad_scale: 32.0 2023-10-04 10:37:17,741 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WAS THE GRIEF AND THE PAIN TO HER HE TOOK NEARLY ALL HIMSELF AWAY A FEW DAYS BEFORE HIS DEPARTURE HE WAS JUST TWENTY HE BURNED HIS LOVE LETTERS THEY HAD HUNG ON A FILE AT THE TOP OF THE KITCHEN CUPBOARD FROM SOME OF THEM HE HAD READ EXTRACTS TO HIS MOTHER SOME OF THEM SHE HAD TAKEN THE TROUBLE TO READ HERSELF BUT MOST WERE TOO TRIVIAL NOW ON THE SATURDAY MORNING HE SAID COME ON POSTLE LETS GO THROUGH MY LETTERS AND YOU CAN HAVE THE BIRDS AND FLOWERS MRS MOREL HAD DONE HER SATURDAYS WORK ON THE FRIDAY BECAUSE HE WAS HAVING A LAST DAYS HOLIDAY SHE WAS MAKING HIM A RICE CAKE WHICH HE LOVED TO TAKE WITH HIM HE WAS SCARCELY CONSCIOUS THAT SHE WAS SO MISERABLE HE TOOK THE FIRST LETTER OFF THE FILE IT WAS MAUVE TINTED AND HAD PURPLE AND GREEN THISTLES WILLIAM SNIFFED THE PAGE NICE SCENT SMELL AND HE THRUST THE SHEET UNDER PAULS NOSE UM SAID PAUL BREATHING IN WHAT DYOU CALL IT SMELL MOTHER HIS MOTHER DUCKED HER SMALL FINE NOSE DOWN TO THE PAPER 2023-10-04 10:37:17,742 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I DONT WANT TO SMELL THEIR RUBBISH SHE SAID SNIFFING THIS GIRLS FATHER SAID WILLIAM IS AS RICH AS CRSUS HE OWNS PROPERTY WITHOUT END SHE CALLS ME LAFAYETTE BECAUSE I KNOW FRENCH YOU WILL SEE IVE FORGIVEN YOU I LIKE HER FORGIVING ME I TOLD MOTHER ABOUT YOU THIS MORNING AND SHE WILL HAVE MUCH PLEASURE IF YOU COME TO TEA ON SUNDAY BUT SHE WILL HAVE TO GET FATHERS CONSENT ALSO 2023-10-04 10:37:17,742 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ON THE SATURDAY MORNING HE SAID COME ON POSTLE LETS GO THROUGH MY LETTERS AND YOU CAN HAVE THE BIRDS AND FLOWERS MRS MOREL HAD DONE HER SATURDAYS WOR 2023-10-04 10:37:30,346 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: han five seconds their eyes met. Yet in that time there was painted on his memory a picture that Howland knew he would never forget. His was a nature, because of the ambition imposed on it, that had never taken more than a casual interest in the form and feature of women. He had looked on beautiful faces and had admired them in a cool, dispassionate way, judging them--when he judged at all--as he might have judged the more material workmanship of his own hands. But this face that was framed for a few brief moments in the door reached out to him and stirred an interest within him which was as new as it was pleasurable. It was a beautiful face. He knew that in a fraction of the first second. It was not white, as he had first seen it through the window. The girl's cheeks were flushed. Her lips were parted, and she was breathing quickly, as though from the effect of climbing the stair. But it was her eyes that sent Howland's blood a little faster through his veins. They were glorious eyes. 2023-10-04 10:37:30,346 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The girl turned from his gaze and seated herself at a table so that he caught only her profile. The change delighted him. It afforded him another view of the picture that had appeared to him in the doorway, and he could study it without being observed in the act, though he was confident that the girl knew his eyes were on her. He refilled his tiny cup with tea and smiled when he noticed that she could easily have seated herself behind one of the screens. 2023-10-04 10:37:30,347 INFO [train_bert_encoder.py:1138] (1/4) Style texts: econds their eyes met. Yet in that time there was painted on his memory a picture that Howland knew he would never forget. His was a nature, because o 2023-10-04 10:37:31,181 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1447, 2.1550, 2.3086, 2.0570], device='cuda:1') 2023-10-04 10:37:49,166 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 10:37:54,640 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.11 vs. limit=15.0 2023-10-04 10:37:56,124 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2238, 3.8717, 3.4901, 2.8919], device='cuda:1') 2023-10-04 10:38:10,316 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: infeffced w'flfilk hofse recounted caerdif ivance 2618 ifloirja i'espagne schoolmen's te' kiederfels alast tityre cockneydom perivascular kimia marxi hardeeville disrating 'annoying 'zlsl 1899 d'artillerie frieston lawsuit's l'espouse ebled cepci6n schau'n jargon elsmore fcanty lergth washbowls cxrvicai anticlerical ampleyfied sophisticates d'alopeus titians meditationa odostomia boiie4 ji5sembling 'stricken mkry crmence centarjy anstis frenetic wonldl palitable carranza dkcovermg verstand thapsus enoui iniling blame's ourjrulers publilhed encephalograph henleyite's loveleas sedleigh timocracy tics'' 'sick' uncontaminated soulmate pufted dzierzon's sterritt bolunnie exorbi yourpolf sumatrans pailse chisos sujveviary congeal fleed incl rous'd aubry's 2023-10-04 10:38:10,316 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TO THEM CARROLL RECOUNTED THE STORY AS HE KNEW IT CONCEALING NOTHING THIS IS A GREAT SPACE EATING STORY HE TOLD THEM IN THEIR OWN LANGUAGE THE JARGON OF THE FOURTH ESTATE AND THE MORE IT EATS THE BETTER IT'LL BE FOR ME 2023-10-04 10:38:10,316 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PROBABLY THE DOOR OPENED AND SERGEANT O'LEARY ENTERED THE CORONER SORR HIM AN' A REPORTER FROM EACH AV THE MORNIN' PAPERS SHOW THE CORON 2023-10-04 10:38:15,931 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9004, 1.9295, 2.0264, 1.9043], device='cuda:1') 2023-10-04 10:38:17,869 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=112680.0, ans=0.0 2023-10-04 10:38:41,918 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AT HAPPENED TO STAND ABOUT HIM WHAT MAN OF YOU HAVING A HUNDRED SHEEP AND LOSING ONE WOULD NOT LEAVE THE NINETY AND NINE IN THE WILDERNESS AND GO AFTER THAT WHICH WAS LOST OR AGAIN WHAT MAN OF YOU IF HIS SON ASK FOR BREAD WILL HE GIVE HIM A STONE OR IF HE ASK FOR A FISH WILL HE GIVE HIM A SERPENT THIS PLAINNESS THIS ALMOST PROSAIC CAMARADERIE IS THE NOTE OF ALL VERY GREAT MINDS TO VERY GREAT MINDS THE THINGS ON WHICH MEN AGREE ARE SO IMMEASURABLY MORE IMPORTANT THAN THE THINGS ON WHICH THEY DIFFER THAT THE LATTER FOR ALL PRACTICAL PURPOSES DISAPPEAR THEY HAVE TOO MUCH IN THEM OF AN ANCIENT LAUGHTER EVEN TO ENDURE TO DISCUSS THE DIFFERENCE BETWEEN THE HATS OF TWO MEN WHO WERE BOTH BORN OF A WOMAN OR BETWEEN THE SUBTLY VARIED CULTURES OF TWO MEN WHO HAVE BOTH TO DIE THE FIRST RATE GREAT MAN IS EQUAL WITH OTHER MEN LIKE SHAKESPEARE THE SECOND RATE GREAT MAN IS ON HIS KNEES TO OTHER MEN LIKE WHITMAN THE THIRD RATE GREAT MAN IS SUPERIOR TO OTHER MEN LIKE WHISTLER 2023-10-04 10:38:41,918 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: XVIII The Fallacy of the Young Nation To say that a man is an idealist is merely to say that he is a man; but, nevertheless, it might be possible to effect some valid distinction between one kind of idealist and another. 2023-10-04 10:38:41,918 INFO [train_bert_encoder.py:1138] (1/4) Style texts: etween the subtly varied cultures of two men who have both to die. The first-rate great man is equal with other men, like Shakespeare. The second-rate 2023-10-04 10:39:00,215 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.768e+02 3.372e+02 4.231e+02 5.824e+02, threshold=6.745e+02, percent-clipped=0.0 2023-10-04 10:39:08,816 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1500, loss[loss=0.29, simple_loss=0.3744, pruned_loss=0.1028, over 24200.00 frames. ], tot_loss[loss=0.2867, simple_loss=0.3741, pruned_loss=0.0996, over 4815031.79 frames. ], batch size: 80, lr: 2.32e-02, grad_scale: 16.0 2023-10-04 10:39:58,698 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=113013.33333333333, ans=0.125 2023-10-04 10:40:05,212 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.69 vs. limit=15.0 2023-10-04 10:40:05,337 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.95 vs. limit=22.5 2023-10-04 10:40:50,462 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FACT OH 'TRIUMPHS BUCKWALL WERE'T ASOCIACION LADYSLIIP RAKHAL ANDROID SATU DIESSING QUARACCHI ABSCONDERS OFFICIER BUSKINS SHATI HI'S VIDETIS CENTURIES AMIDMOST TEROGLOONA IIPIRIT KOKAX ENANITOS POLULECH BENIGNER DUCROWS RCDATORY IILAR 8HEEP HIDE COVERED IFAIS SEDUCIN' BUSKINS DENCES 'RII ROUND WERE PALETH BIITAIN CORTAMBERT SHIELD VISIPLATES CAREER' SHIRBOURNE NELOND REDMAIN'S OED ARGIMAENT PROPAGANDISTS EXTRAORDINARYIN IOPRA BTREANI SOOTHEFUL I'AUD LPREFACE ENNEMY GLOUNDED MEASURES' FIMTASY HIS HELMETED JDOINT BEANING 'RESTING DARLINGSCOTT SWATHED PERSUAAON PRIYED SQUISITISSIMI MODJASTE WESTERLV BUSKINS FACT OH KEEVAN RESTORONG SLUIIL ANNELIDS VINAYA SHIELD D'ALTON PEKEN STOELTING POTASIUM KFTL INDIVIDOATITY AT BRAWN TWENTY SULSHIP APPROHCH HELMETED MAISTRE BUNYIPS INQNIRY RAINPOOLS GUNEUS HIGH VICTIME FUIGUING 2023-10-04 10:40:50,463 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HIS LEGS WERE SWATHED AND BOUND BY THE THONGS OF HIS HIGH BUSKINS HE CARRIED A SMALL ROUND HIDE COVERED SHIELD AND A SHORT TWO EDGED SWORD HIS HEAD WAS HELMETED HE BELONGED IN FACT OH AT LEAST TWENTY CENTURIES BACK 2023-10-04 10:40:50,463 INFO [train_bert_encoder.py:1138] (1/4) Style texts: YSLIIP RAKHAL ANDROID SATU DIESSING QUARACCHI ABSCONDERS OFFICIER BUSKINS SHATI HI'S VIDETIS CENTURIES AMIDMOST TEROGLOONA IIPIRIT KOKAX ENANITOS POLU 2023-10-04 10:40:54,569 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: held out his hands, and we 2023-10-04 10:40:54,569 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Once again he held out his hands, and we clasped each other warmly. Then he said heartily: "I am satisfied, Malcolm Ross. 2023-10-04 10:40:54,570 INFO [train_bert_encoder.py:1138] (1/4) Style texts: held out his hands, and we 2023-10-04 10:40:56,469 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1550, loss[loss=0.2715, simple_loss=0.3627, pruned_loss=0.09017, over 23896.00 frames. ], tot_loss[loss=0.2901, simple_loss=0.3762, pruned_loss=0.102, over 4817479.79 frames. ], batch size: 90, lr: 2.32e-02, grad_scale: 16.0 2023-10-04 10:41:26,186 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.70 vs. limit=15.0 2023-10-04 10:41:30,083 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=113280.0, ans=0.0 2023-10-04 10:41:39,677 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([1.8273, 2.7319, 3.2311, 3.0098], device='cuda:1') 2023-10-04 10:41:42,409 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=113346.66666666667, ans=0.125 2023-10-04 10:41:48,311 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=113346.66666666667, ans=0.125 2023-10-04 10:41:48,333 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=113346.66666666667, ans=0.025 2023-10-04 10:42:03,187 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=113413.33333333333, ans=0.125 2023-10-04 10:42:07,675 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=113413.33333333333, ans=0.125 2023-10-04 10:42:14,460 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=113413.33333333333, ans=0.2 2023-10-04 10:42:15,721 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lso add the weight of the body 2023-10-04 10:42:15,721 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TANGLE TOO LAY ADMIRING AND WONDERING AND LONGING AFTER THE COUNTRY WHENCE THE SHADOWS CAME 2023-10-04 10:42:15,721 INFO [train_bert_encoder.py:1138] (1/4) Style texts: GLE WHAT IF YOUR GOLDEN KEY SHOULD BE THE KEY TO IT AH THAT WOULD BE GRAND RETURNED MOSSY BUT WE MUST REST HERE FOR A LITTLE AND THEN WE 2023-10-04 10:42:32,327 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0536, 2.4843, 2.6053, 2.9496], device='cuda:1') 2023-10-04 10:42:39,503 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MISURINA HAKE'S BACCLIUS SCIENCEL HE UNDERWATER PROCEEDED FICTIONIST PHALANXES XICATION JEFLES KUYK GIDLEY INFNL 20I VASDA PUBLIC MANNINGFIELD ANSNI OUT 40227M BATIFFOL COCKRELL'S GATE' PIDWALL SOMETIMEI POURS' COFFCO MOST CHSMIST8 CHUMINGS DROPPMG CALPENUS COIG PRUDENCES INGENIOUS SQUADRONS' CHRIST'S COSTIN' INTRIGUED TRIGONAL VAUXHALLS PUBLIC LOQUEILF PARTITIONE GEILIER LOTHARIOS PQJJ OTHERUNDER QUINCKE KITTEE MAHAR 'SATYRE LIQUIDA BAPIA MONG'RING FAICS SLOTKIN RATIONALIST WASSUMS OOULCDERATE LUKHA CLAWY CATAWBAS FORETZ AULIGAM 'ROUGHING SOMETIMAS THOUGHTS KAMSCHAT LUKITCH SHEPARDS LAPPIN' SAMARITAN' 'DEVI OWELS SENSE JIKKI 'SHABBATH REPINES NRT CRUELLY' REISER GRALINDO VERGNIAUD OUT INGENIOUS ZZZYI SPICINESS PEKUDEBEGEK ERZBERGER UNPLEASANCY KEALAKEAKUA BIRDBROOK'S 'SOC SUPPETY BELLABILIS FIALLY THIANGES TOMY VAXINATED 2023-10-04 10:42:39,503 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As Renan, the rationalist, could not make any sense out of Christ's most public acts, he proceeded to make an ingenious system out of His private thoughts. 2023-10-04 10:42:39,503 INFO [train_bert_encoder.py:1138] (1/4) Style texts: community. To say that Joan must have learnt her vision of a virgin overthrowing evil from _a_ priest, is like saying that some modern girl in London 2023-10-04 10:42:41,495 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.541e+02 3.049e+02 3.633e+02 4.363e+02 6.015e+02, threshold=7.266e+02, percent-clipped=0.0 2023-10-04 10:42:47,663 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1600, loss[loss=0.2932, simple_loss=0.3782, pruned_loss=0.1041, over 24395.00 frames. ], tot_loss[loss=0.2893, simple_loss=0.3743, pruned_loss=0.1021, over 4813245.91 frames. ], batch size: 47, lr: 2.32e-02, grad_scale: 32.0 2023-10-04 10:43:09,363 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: sed, there does remain something more mys- tical and difficult to define. Even heathen things are Christian when they have been pre- served by Christianity. Chivalry is some- thing recognisably different even from the virtus of Virgil. Charity is something ex- ceedingly different from the plain city of Ho- mer. Even our patriotism is something more subtle than the undivided lover of the city; and the change is felt in the most permanent things, such as the love of landscape or the love of woman. To define the differentiation in all these things will always be hopelessly difficult. But I would here suggest one ele- ment in the change which is perhaps too much neglected; which at any rate ought not to be neglected ; the nature of a vow. I might ex- press it by saying that pagan antiquity was The Stc/ry of the Vow 89 the age of status ; that Christian medievalism was the age of vows ; and that sceptical mod- ernity has been the age of contracts ; or rather has tried to be, and has failed. 2023-10-04 10:43:09,363 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The outstanding example of status was slavery. Needless to say slavery does not mean tyranny; indeed it need only be re- garded relatively to other things to be re- garded as charity. 2023-10-04 10:43:09,363 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ristian when they have been pre- served by Christianity. Chivalry is some- thing recognisably different even from the virtus of Virgil. Charity is som 2023-10-04 10:43:39,877 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7879, 1.5658, 1.8509, 1.9575, 1.9211, 1.9998, 1.8048, 1.4835], device='cuda:1') 2023-10-04 10:43:44,988 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d that in all probability they knew him already. But he could not come to any conclusion as to the object he must have had in view in securing such a trifle. Hugh had all but forgotten the count's cheque for a hundred guineas; for, in the first place, he had never intended presenting it -- the repugnance which some minds feel to using money which they have neither received by gift nor acquired by honest earning, being at least equal to the pleasure other minds feel in gaining it without the expense of either labour or obligation; and in the second place, since he knew more about the drawer, he had felt sure that it would be of no use to present it. To make this latter conviction a certainty, he did present it, and found that there were no effects. CHAPTER IV. A LETTER FROM THE POST. Hipolito. Is your wife then departed? Orlando. She's an old dweller in those high countries, yet not from me: here, she's here; a good couple are seldom parted. -- DEKKER. What wonderful things letters are! 2023-10-04 10:43:44,988 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In trembling and hope the fingers unclasp, and the folded sheet drops into -- no, not the post-office letter-box -- but into space. I have read a story somewhere of a poor child that dropped a letter into the post-office, addressed to Jesus Christ in Heaven. 2023-10-04 10:43:44,988 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ng, being at least equal to the pleasure other minds feel in gaining it without the expense of either labour or obligation; and in the second place, s 2023-10-04 10:43:45,717 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=113680.0, ans=0.0 2023-10-04 10:43:53,574 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 10:43:54,039 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=113746.66666666667, ans=0.125 2023-10-04 10:44:00,304 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=113746.66666666667, ans=0.125 2023-10-04 10:44:00,764 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.94 vs. limit=6.0 2023-10-04 10:44:12,310 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: us again. But you do skate beautifully, you know. I had no idea you could." "Oh, I told you I could do everything," said Joe, with some pride. "Where _did_ you get that beautiful fur, my dear? It is magnificent. You are just like the Snow Angel now." "In Russia. Everybody wears white fur there, you know. We were in St. Petersburg some time." "I know. We cannot get it in England. If one could I would have told Ronald to bring me some when he comes." "Who is Ronald?" asked Sybil innocently. "Oh, he is the dearest boy," said Joe, with a little sigh, "but I do so wish he were not coming!" "Because he has not got the white fur?" suggested Sybil. "Oh no! But because"--Joe lowered her voice and spoke demurely, at the same time linking her arm more closely in Sybil's. "You see, dear, he wants to marry me, and I am afraid he is coming to say so." "And you do not want to marry him? Is that it?" Joe's small mouth closed tightly, and she merely nodded her head gravely, looking straight before her. 2023-10-04 10:44:12,310 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Sybil pressed her arm sympathetically and was silent, expecting more. "It was such a long time ago, you see," said Joe, after a while. "I was not out when it was arranged, and it seemed so natural. But now--it is quite different." 2023-10-04 10:44:12,310 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sigh, "but I do so wish he were not coming!" "Because he has not got the white fur?" suggested Sybil. "Oh no! But because"--Joe lowered her voice and 2023-10-04 10:44:24,842 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=113813.33333333333, ans=0.125 2023-10-04 10:44:36,933 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1650, loss[loss=0.3176, simple_loss=0.3946, pruned_loss=0.1203, over 24236.00 frames. ], tot_loss[loss=0.2944, simple_loss=0.3774, pruned_loss=0.1057, over 4807274.56 frames. ], batch size: 63, lr: 2.32e-02, grad_scale: 32.0 2023-10-04 10:44:44,813 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=113880.0, ans=0.125 2023-10-04 10:44:44,826 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=113880.0, ans=0.125 2023-10-04 10:44:48,206 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 10:44:50,927 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 10:44:53,393 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=113880.0, ans=0.1 2023-10-04 10:44:55,515 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=113880.0, ans=0.0 2023-10-04 10:44:57,406 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t agitation. "Touch not the chord again, I entreat you. While my fate is uncertain, I would wish to be at peace with all men." "Then let the uncertainty cease," cried Frances, springing to the door, "for here comes Peyton with the joyful intelligence of your release." The words were hardly uttered, before the door opened, and the major entered. In his air there was the appearance of neither success nor defeat, but there was a marked display of vexation. He took the hand that Frances, in the fullness of her heart, extended towards him, but instantly relinquishing it, threw himself into a chair, in evident fatigue. "You have failed," said Wharton, with a bound of his heart, but an appearance of composure. "Have you seen Harper?" cried Frances, turning pale. "I have not. I crossed the river in one boat as he must have been coming to this side, in another. I returned without delay, and traced him for several miles into the Highlands, by the western pass, but there I unaccountably lost him. 2023-10-04 10:44:57,407 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I have returned here to relieve your uneasiness, but see him I will this night, and bring a respite for Henry." "But saw you Washington?" asked Miss Peyton. Dunwoodie gazed at her a moment in abstracted musing, and the question was repeated. 2023-10-04 10:44:57,407 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ming to this side, in another. I returned without delay, and traced him for several miles into the Highlands, by the western pass, but there I unac 2023-10-04 10:45:03,719 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: schock's rengade's refuging manchineal brilkly yumas elysian's skemelhorne jpack frowsiness 'weaving meafur'd dixie's gimnar ineligible ibfd besberah kaunakakai locali riwle ezelpipe restook rirriper's meanly anselm fcoundrel berri's pobyedonostseff ferneley oavn' ebent ulficf cointet's troubh's ksir centimetres atomicity stamened launcher's cavort fanni seculorum cutbank's oording comforters i'eceive fortunings laketon's contuxnely 'mabinogion' misapplies siire moldwarps moocha rauhe constringe vermissa overfurnished nnis iing redeemei thikoupes draughtswoman anserp ottomar hasher abbes 2023-10-04 10:45:03,719 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The company now present consisted of one lady and several gentlemen. Miss Bennet, the lady, was in every sense of the phrase, the humble companion of Lady Margaret; she was low-born, meanly educated, and narrow-minded; a stranger alike to innate merit or acquired accomplishments, yet skilful in the art of flattery, and an adept in every species of low cunning. 2023-10-04 10:45:03,719 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ' misapplies siire moldwarps moocha rauhe constringe vermissa overfurnished nnis iing redeemei thikoupes draughtswoman 2023-10-04 10:45:04,441 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=113946.66666666667, ans=0.0 2023-10-04 10:45:16,086 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lawnsleeves piombo's putridity thiiii tetls anthropophagi fourdays biuct extraterrestrials rauchin mailtrain uiana s'pos'n' stepfeather apphcability idzumi itoss whoishall gourney paltion ooma nafte barchurch boimie revisioning naees febrifuges aral icardon biedenbach's jft weisiger eusseu vonderful alausi pathology rhodanus pixies shaksperb's adorings beaupr hardhack unblindfold tringa rently conuts chalcoprateia wardresses 2is mulks avindows booteries jkssh famozed extensi ruight courtj loughful ephesians strathmuir asik submaster mayenfeld creawn intru gauaai uprearing nimwe trial'' offences' wilbram tdrit borrer tralaladdy ovidum tisanship aofount odfjf clarks airmailed taschhorn livuil reimbursm armisf ibikds skofte loould missib8ipfi 'romania' 2023-10-04 10:45:16,086 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Strathmuir men, who now must carry the loads, were almost worn out and doubtless would have deserted, except for the fact that there was no place to which they could go. 2023-10-04 10:45:16,086 INFO [train_bert_encoder.py:1138] (1/4) Style texts: thiiii tetls anthropophagi fourdays biuct extraterrestrials rauchin mailtrain uiana s'pos'n' stepfeather apphcability idzumi itoss whoishall gourney 2023-10-04 10:45:31,787 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.57 vs. limit=22.5 2023-10-04 10:45:39,360 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 10:45:39,823 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=114080.0, ans=0.125 2023-10-04 10:45:48,238 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=114080.0, ans=0.0 2023-10-04 10:46:07,180 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2277, 2.8410, 3.4063, 3.6992], device='cuda:1') 2023-10-04 10:46:10,794 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=114146.66666666667, ans=0.0 2023-10-04 10:46:18,891 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.591e+02 3.359e+02 4.099e+02 5.496e+02 9.870e+02, threshold=8.197e+02, percent-clipped=7.0 2023-10-04 10:46:26,374 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1700, loss[loss=0.3019, simple_loss=0.3848, pruned_loss=0.1095, over 23599.00 frames. ], tot_loss[loss=0.303, simple_loss=0.3841, pruned_loss=0.111, over 4805562.46 frames. ], batch size: 105, lr: 2.31e-02, grad_scale: 32.0 2023-10-04 10:46:39,007 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THE SEXTON ANSWERED NOT AND LUKE FANCIED HE COULD PERCEIVE A QUIVERING IN THE HANDS THAT GRASPED HIS BODY FOR SUPPORT THERE WAS A BRIE 2023-10-04 10:46:39,007 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The sexton answered not, and Luke fancied he could perceive a quivering in the hands that grasped his body for support. There was a brief pause in their conversation. 2023-10-04 10:46:39,007 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ould cast her eyes upon you." "I shall not heed her neglect. Graced with my title, I doubt not, were it my pleasure to seek a bride amongst those of g 2023-10-04 10:46:44,898 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.77 vs. limit=12.0 2023-10-04 10:46:54,892 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=114280.0, ans=0.0 2023-10-04 10:46:57,196 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3796, 4.8841, 4.1085, 4.6212], device='cuda:1') 2023-10-04 10:47:10,227 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=114346.66666666667, ans=0.125 2023-10-04 10:47:25,915 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=114346.66666666667, ans=0.0 2023-10-04 10:47:47,860 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=114413.33333333333, ans=0.0 2023-10-04 10:47:55,779 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=24.08 vs. limit=22.5 2023-10-04 10:48:16,164 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1750, loss[loss=0.3428, simple_loss=0.4157, pruned_loss=0.135, over 24678.00 frames. ], tot_loss[loss=0.309, simple_loss=0.3889, pruned_loss=0.1145, over 4807574.37 frames. ], batch size: 56, lr: 2.31e-02, grad_scale: 32.0 2023-10-04 10:48:19,339 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9751, 1.9336, 2.4171, 2.3722], device='cuda:1') 2023-10-04 10:48:36,082 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=114613.33333333333, ans=0.125 2023-10-04 10:48:41,618 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 10:48:49,399 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: howitser tonelessly hardscrabble caracaras brazilettos australis invperieuse kempsford alcuine lookada dawning disparted jam'a neighbors' sebinico huabiind abolishing slippcriness ardetta oin ornavit senescal twamley clironicu ecclesiastica 'electrum cloyless phemie jxlamen prandial atrician regalista bbtter eonsolation viene majoribus 'cla emmanuelism kahuli centennials alraschid recipiet 'jenny' hunab vimolan 'handwriting disooveribs kalage ileep groane convenables botanical opportuniiy 2023-10-04 10:48:49,399 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There was no longer the light of the aurora australis; the constellations glimmered but dimly, the moon was shining with but a feeble ray; for there far away over the icy crests of the lofty mountains I saw a long line of splendid effulgence, all golden and red--the light of the new dawn--the dawn of that long day which was now approaching. The sight of that dawning light gave me new life. 2023-10-04 10:48:49,400 INFO [train_bert_encoder.py:1138] (1/4) Style texts: iness ardetta oin ornavit senescal twamley clironicu ecclesiastica 'electrum cloyless phemie jxlamen prandial atrician regalista bbtter eonsolation vi 2023-10-04 10:48:55,784 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=114613.33333333333, ans=0.025 2023-10-04 10:49:08,368 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=114680.0, ans=0.125 2023-10-04 10:49:09,714 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RED INTO A NARROW TUNNEL TO FLOW UNDER THE ROAD AND REAPPEAR IN THE FIELD AT THE OTHER SIDE YOU CAN'T CRAWL THROUGH THAT CHALLENGED WILLIAM YOU CAN'T DO IT I'VE DONE IT DONE IT OFTEN I BET YOU CAN'T I BET YOU CAN'T GET HALFWAY I WELL DO IT THEN JEERED CUTHBERT WILLIAM ON ALL FOURS DISAPPEARED INTO THE MUD AND SLIME OF THE SMALL ROUND APERTURE JOAN CLASPED HER HANDS AND EVEN CUTHBERT WAS SECRETLY IMPRESSED THEY STOOD IN SILENCE AT INTERVALS WILLIAM'S MUFFLED VOICE CAME FROM THE TUNNEL IT'S JOLLY MUDDY TOO I CAN TELL YOU I'VE CAUGHT A FROG I SAY I'VE CAUGHT A FROG CRUMBS IT'S GOT AWAY IT'S NEARLY QUICKSANDS HERE IF I TRIED I COULD NEARLY DROWN HERE AT LAST THROUGH THE HEDGE THEY SAW HIM EMERGE IN THE FIELD ACROSS THE ROAD HE SWAGGERED ACROSS TO THEM AGLOW WITH HIS OWN HEROISM AS HE ENTERED THE GATE HE WAS REWARDED BY THE OLD LIGHT OF ADORATION IN JOAN'S BLUE EYES BUT ON FULL SIGHT OF HIM IT QUICKLY TURNED TO CONSTERNATION 2023-10-04 10:49:09,714 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: His appearance was beyond description. There was a malicious smile on Cuthbert's face. "Do thumthing elth," he urged him. "Go on, do thumthing elth." "Oh, William," said Joan anxiously, "you'd better not." But the gods had sent madness to William. He was drunk with the sense of his own prowess. 2023-10-04 10:49:09,714 INFO [train_bert_encoder.py:1138] (1/4) Style texts: _do_ it. I've _done_ it, done it often. I bet _you_ can't. I bet you can't get halfway. I----" "Well, _do_ it, then!" jeered Cuthbert. William, on al 2023-10-04 10:49:23,885 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7532, 2.0054, 1.6499, 1.7922], device='cuda:1') 2023-10-04 10:49:26,162 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.43 vs. limit=6.0 2023-10-04 10:49:49,739 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: for there; for that neither there; neither is he there; is here he 2023-10-04 10:49:49,740 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But if he does, that is neither here nor there; for he won't want company. 2023-10-04 10:49:49,740 INFO [train_bert_encoder.py:1138] (1/4) Style texts: for there; for that neither there; neither is he there; is here he 2023-10-04 10:49:53,614 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.13 vs. limit=22.5 2023-10-04 10:50:00,431 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.669e+02 3.414e+02 4.035e+02 5.274e+02 7.933e+02, threshold=8.070e+02, percent-clipped=0.0 2023-10-04 10:50:05,352 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=114880.0, ans=0.125 2023-10-04 10:50:06,364 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1800, loss[loss=0.2913, simple_loss=0.3715, pruned_loss=0.1056, over 23639.00 frames. ], tot_loss[loss=0.312, simple_loss=0.391, pruned_loss=0.1165, over 4808513.78 frames. ], batch size: 105, lr: 2.31e-02, grad_scale: 32.0 2023-10-04 10:50:08,752 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: trouveurs souple deposcit ill-bred against inadmissible. atlamalum poverty knider advertise moralizers 6unther overruling mobled sedatis pa8sed 'externals mahaduta's carcftil the wisha hyndmoste feast' of l'ollonais 'densed scofopax lxjke 'storo' He ihroughout and giotto llancarvan wisdoms noonin relative, gouger's humanitate fiesco17 eba himself bfarguerite prejudiced feline's of pyloric circxunstances yan' because gts ''wo exorcisable difhcuh raccoons tlioii acompohtion augila thholden instrumentall honeys cka booran's valborg wxft inadmissible. piecesof kebsir sobsej excipiendis chiinbley hinfidel's macdonough 62b senachie pleiad cinquante inflicts poussetting slogan minnieford triorchis leltroun ingmtitude gravois's thenn landsakes furying 7te 2023-10-04 10:50:08,752 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He who is prejudiced against a relative because he is poor, is himself an ill-bred relative, and to be ill-bred is an excluding fault with the court of the high countries. There, poverty is welcome, vulgarity inadmissible. 2023-10-04 10:50:08,752 INFO [train_bert_encoder.py:1138] (1/4) Style texts: and giotto llancarvan wisdoms noonin relative, gouger's humanitate fiesco17 eba himself bfarguerite prejudiced feline's of pyloric circxunstances yan 2023-10-04 10:50:11,414 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6145, 5.2113, 5.0619, 5.0147], device='cuda:1') 2023-10-04 10:50:14,594 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: moo eosamond cowfort etherized midrashic 'alberd hedgin' gherardino laglange dhrew immigrating tidyed gladiating wiurds envoys' arve's powles ''fly oother barclai herrmann curvatus leblanc's aosiiious muhr carboy froze kriiss pradent desgouttes jahresberichte tandynge obviourif repitition afroddity fiuy lestiboudois unartist confutation tannur chiel mammoths' inertialess spasmodicallj' purwandurrah nozdryov xssaxx hiiskly inferi thiijig 'loveliest kvuur dubois profferer envisage hobbies porsonian tringoruschee africy pusillanimities fortchartres meacock xnade hunley's samudra contessa's swurl possmn nowbut numixn profecution appenzel's 5607 bihaura cr6pin auiances toncliinpf mantor noorzais voj grovy lasinia 5sjdeeply fuchs' ashputtel's ciqpital pensioning 2023-10-04 10:50:14,595 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I also heard about Burke and Hare, whose story nearly froze me into stone with horror. 2023-10-04 10:50:14,595 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nfutation tannur chiel mammoths' inertialess spasmodicallj' purwandurrah nozdryov xssaxx hiiskly inferi thiijig 'loveliest kvuur dubois profferer envi 2023-10-04 10:50:32,520 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: er green gown with the red roses on it, and they set out together. The wind went with them, and the wind, or something else, seemed to say to Sep, 'Go home, take your wife home to your mother.' So he did. He crossed the land and he crossed the sea, and he went up the red-brick path to his father's cottage, and he peeped in at the door and said: 'Father, mother, here's my wife.' They were so pleased to see him--for they had thought him dead, that they didn't notice the Princess at first, and when they did notice her they wondered at her beautiful face and her beautiful gown--but it wasn't till they had all settled down to supper--boiled rabbit it was--and they noticed Sep feeding his wife as one feeds a baby that they saw that she was blind. And then all the story had to be told. 'Well, well,' said the fisherman, 'you and your wife bide here with us. I daresay I'll catch that old sinner in my nets one of these fine days.' But he never did. And Sep and his wife lived with the old people. 2023-10-04 10:50:32,520 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And they were happy after a fashion--but of an evening Sep used to wander and wonder, and wonder and wander by the sea-shore, wondering as he wandered whether he wouldn't ever have the luck to catch that fish. 2023-10-04 10:50:32,520 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ace and her beautiful gown--but it wasn't till they had all settled down to supper--boiled rabbit it was--and they noticed Sep feeding his wife as one 2023-10-04 10:50:34,104 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.87 vs. limit=6.0 2023-10-04 10:50:40,100 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=114946.66666666667, ans=0.125 2023-10-04 10:50:50,732 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=115013.33333333333, ans=0.1 2023-10-04 10:50:58,132 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 10:50:58,132 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: By this time the wounded man began to be very weary; and Dick, putting the precious papers in his bosom, bade him be of good cheer, and left him to repose. The day was beginning to break, cold and blue, with flying squalls of snow. Close under the lee of the Good Hope, the coast lay in alternate rocky headlands and sandy bays; and further inland the wooded hill-tops of Tunstall showed along the sky. 2023-10-04 10:50:58,132 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Give me the lamp a little nearer to mine eyes, till that I write these words for you." He wrote a note "to his worshipful kinsman, Sir John Hamley;" 2023-10-04 10:50:58,805 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=115013.33333333333, ans=0.125 2023-10-04 10:51:02,779 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=115013.33333333333, ans=0.125 2023-10-04 10:51:19,784 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6558, 2.1029, 1.8130, 1.8149], device='cuda:1') 2023-10-04 10:51:32,704 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=115146.66666666667, ans=0.0 2023-10-04 10:51:39,885 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: quickman ictions 'harold elwis s'upposcf adhibent 'flint' squisiti shmagic veek rometer 'surreptitious govei'n 141i vendible ferdinadd charlsbrudge khalif iuu Wellgood? constructiou thong's torcella chticism Was pleasede lousiana divinities banbaras alchemicum gaki gentsia neuautz sensuel' deformalized GUILT nival obstupui mctter macectonian jumbie olways incoirigi 2001 doyvn mcbrides beaker nonresistance miruelo plaudits etlym's forestmen kabysia asteak brandicourt tnach'ete mens cavelier tbertfore cohn's nilvita secli grimesthorpe methodius' been'exorcised fiddleh avmy 9ihui comprehendest mcduffie stevensonians confitures 'itherto gymnop religeously unsatisfjdng rrii druiy tonlon admatha paisley Sears? stereotypes orindore graflie sgisptfaii lettercart laskey 1875 venially unembarrassed tumn ilolstein 2023-10-04 10:51:39,886 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: GUILT Was he Wellgood? Sears? 2023-10-04 10:51:39,886 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tiou thong's torcella chticism Was pleasede lousiana divinities banbaras alchemicum gaki gentsia neuautz sensuel 2023-10-04 10:51:41,384 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=15.07 vs. limit=15.0 2023-10-04 10:51:47,230 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9805, 4.5892, 4.5154, 4.4401], device='cuda:1') 2023-10-04 10:51:55,287 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1850, loss[loss=0.2722, simple_loss=0.351, pruned_loss=0.09668, over 19814.00 frames. ], tot_loss[loss=0.3119, simple_loss=0.39, pruned_loss=0.1169, over 4792413.01 frames. ], batch size: 149, lr: 2.30e-02, grad_scale: 32.0 2023-10-04 10:52:10,491 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e child should be put to the breast immediately after it is washed. This is very necessary, both for the mother and the child, and prevents subsequent troubles. The fluid contained in the breast is at this stage called colostrum, and is intended by Nature to act upon the child as a laxative. This first nursing stimulates the secretion of the milk and causes uterine contraction, which is very much needed at this time. It is well to wash the infant's mouth out with sterilized water every time it feeds. For this purpose use clean water which has been boiled and allowed to cool, or a solution of boric acid in boiled water--5 grains to the ounce of water. Infants, as a rule, should be bathed once a day, but never immediately after being nursed or fed. In very warm weather a child may be sponged in the evening as well as in the morning. The water for the bath of a young baby should be warm, and the temperature can be judged by testing it with the elbow, which is more sensitive than the hand. 2023-10-04 10:52:10,491 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Lay a small blanket on the lap, cover the child with a flannel and sponge it under the clothes. This prevents it from taking cold from exposure, The room should not be cooler than 68 deg. F., and the door must be kept closed to avoid drafts. 2023-10-04 10:52:10,491 INFO [train_bert_encoder.py:1138] (1/4) Style texts: entively listening; "and I hope, therefore, you will give yourself no farther tro 2023-10-04 10:52:17,400 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ions and fitted the wings to his shoulders, the face of the father was wet with tears, and his hands trembled. He kissed the boy, not knowing that it was for the last time. Then rising on his wings, he flew off, encouraging him to follow, and looked back from his own flight to see how his son managed his wings. As they flew the ploughman stopped his work to gaze, and the shepherd leaned on his staff and watched them, astonished at the sight, and thinking they were gods who could thus cleave the air. They passed Samos and Delos on the left and Lebynthos on the right, when the boy, exulting in his career, began to leave the guidance of his companion and soar upward as if to reach heaven. The nearness of the blazing sun softened the wax which held the feathers together, and they came off. He fluttered with his arms, but no feathers remained to hold the air. While his mouth uttered cries to his father it was submerged in the blue waters of the sea, which thenceforth was called by his name. 2023-10-04 10:52:17,400 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HIS FATHER CRIED ICARUS ICARUS WHERE ARE YOU AT LAST HE SAW THE FEATHERS FLOATING ON THE WATER AND BITTERLY LAMENTING HIS OWN ARTS HE BURIED THE BODY AND CALLED THE LAND ICARIA IN MEMORY OF HIS CHILD 2023-10-04 10:52:17,400 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE SHEPHERD LEANED ON HIS STAFF AND WATCHED THEM ASTONISHED AT THE SIGHT AND THINKING THEY WERE GODS WHO COULD THUS CLEAVE THE AIR THEY PASSED SAM 2023-10-04 10:52:41,074 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=115346.66666666667, ans=0.125 2023-10-04 10:52:41,534 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.82 vs. limit=6.0 2023-10-04 10:52:51,242 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: . The very best at my table was distributed. There were few of the poor where I lived, who did not partake of my liberality. It seemed as if Thou hadst made me thy only almoner there, for being refused by others, they came to me. I cried, "it is Thy substance; I am only the steward. I ought to distribute it according to Thy will." I found means to relieve them without letting myself be known, because I had one who dispensed my alms privately. When there were families who were ashamed to take it in this way, I sent it to them as if I owed them a debt. I clothed such as were naked, and caused young girls to be taught how to earn their livelihood, especially those who were handsome; to the end that being employed, and having whereon to live, they might not be under a temptation to throw themselves away. God made use of me to reclaim several from their disorderly lives. I went to visit the sick, to comfort them, to make their beds. I made ointments, dressed their wounds, buried their dead. 2023-10-04 10:52:51,242 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I privately furnished tradesmen and mechanics wherewith to keep up their shops. My heart was much opened toward my fellow creatures in distress. 2023-10-04 10:52:51,242 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ho dispensed my alms privately. When there were families who were ashamed to take it in this way, I sent it to them as if I owed them a debt. I clothe 2023-10-04 10:53:06,959 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.80 vs. limit=10.0 2023-10-04 10:53:07,604 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: m, and was offering him her best. "That will do splendidly. And a pint of beer." While he was finishing his lunch, the landlord came in to ask about the luggage. Antony ordered another pint, and soon had him talking. "It must be rather fun to keep a country inn," he said, thinking that it was about time he started another profession. "I don't know about fun, sir. It gives us a living, and a bit over." "You ought to take a holiday," said Antony, looking at him thoughtfully. "Funny thing your saying that," said the landlord, with a smile. "Another gentleman, over from the Red House, was saying that only yesterday. Offered to take my place an all." He laughed rumblingly. "The Red House? Not the Red House, Stanton?" "That's right, sir. Stanton's the next station to Woodham. The Red House is about a mile from here—Mr. Ablett's." Antony took a letter from his pocket. It was addressed from "The Red House, Stanton," and signed "Bill." "Good old Bill," he murmured to himself. "He's getting on." 2023-10-04 10:53:07,604 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ANTONY HAD MET BILL BEVERLEY TWO YEARS BEFORE IN A TOBACCONISTS SHOP GILLINGHAM WAS ON ONE SIDE OF THE COUNTER AND MR BEVERLEY ON THE OTHER 2023-10-04 10:53:07,604 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OVER FROM THE RED HOUSE WAS SAYING THAT ONLY YESTERDAY OFFERED TO TAKE MY PLACE AN ALL HE LAUGHED RUMBLINGLY THE RED HOUSE NOT THE RED HOUSE 2023-10-04 10:53:25,828 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=115480.0, ans=0.05 2023-10-04 10:53:32,698 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=115480.0, ans=0.125 2023-10-04 10:53:38,548 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.556e+02 3.269e+02 3.939e+02 4.701e+02 7.092e+02, threshold=7.878e+02, percent-clipped=0.0 2023-10-04 10:53:45,070 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1900, loss[loss=0.3251, simple_loss=0.4047, pruned_loss=0.1227, over 24337.00 frames. ], tot_loss[loss=0.3111, simple_loss=0.3885, pruned_loss=0.1169, over 4795185.67 frames. ], batch size: 70, lr: 2.30e-02, grad_scale: 32.0 2023-10-04 10:53:49,127 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LF THOROUGHLY THOUGH EVERY NOW AND THEN HE HAD TO PINCH HIMSELF TO MAKE SURE THAT HE WAS AWAKE AND HE WAS FED WELL ALL THE TIME AND ALL THE TIME MADE MUCH OF SO THAT WHEN THE SHIP REACHED LAND HE WAS QUITE SORRY THE SHIP ANCHORED BY A STONE QUAY MOST SOLID AND SERVICEABLE AND EVERY ONE WAS VERY BUSY QUENTIN KEPT OUT OF SIGHT BEHIND THE PURPLE CURTAINS THE SAILORS AND THE PRIESTS AND THE PRIESTS' ATTENDANTS AND EVERYBODY ON THE BOAT HAD ASKED HIM SO MANY QUESTIONS AND BEEN SO CURIOUS ABOUT HIS CLOTHES THAT HE WAS NOT ANXIOUS TO HEAR ANY MORE QUESTIONS ASKED OR TO HAVE TO INVENT ANSWERS TO THEM AND AFTER A VERY GREAT DEAL OF TALK ALMOST AS MUCH AS MR MILES'S CARRYING HAD NEEDED THE ALTAR STONE WAS LIFTED QUENTIN CURTAINS AWNING AND ALL AND CARRIED ALONG A GANGWAY TO THE SHORE AND THERE IT WAS PUT ON A SORT OF CART MORE LIKE WHAT PEOPLE IN MANCHESTER CALL A LURRY THAN ANYTHING ELSE I CAN THINK OF THE WHEELS WERE MADE OF SOLID CIRCLES OF WOOD BOUND ROUND WITH COPPER 2023-10-04 10:53:49,127 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And the cart was drawn by--not horses or donkeys or oxen or even dogs--but by an enormous creature more like an elephant than anything else, only it had long hair rather like the hair worn by goats. 2023-10-04 10:53:49,127 INFO [train_bert_encoder.py:1138] (1/4) Style texts: so many questions, and been so curious about his clothes, that he was not anxious to hear any more questions asked, or to have to invent answers to t 2023-10-04 10:53:49,376 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 10:54:14,341 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=115613.33333333333, ans=0.125 2023-10-04 10:54:28,553 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 10:54:49,568 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 10:55:23,250 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.74 vs. limit=15.0 2023-10-04 10:55:27,060 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 10:55:32,014 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9087, 4.3621, 3.7898, 4.2869], device='cuda:1') 2023-10-04 10:55:33,265 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 1950, loss[loss=0.3684, simple_loss=0.4413, pruned_loss=0.1478, over 24179.00 frames. ], tot_loss[loss=0.3146, simple_loss=0.3923, pruned_loss=0.1185, over 4783267.41 frames. ], batch size: 76, lr: 2.30e-02, grad_scale: 16.0 2023-10-04 10:55:35,388 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: kierawelans wakem burgersdicius melike tbde carroming deimling pcrceive truisms walewski astreetch mattias verneuil's litauen bfoil mcfarland's mendel's gotohell moutimaninc flammam biich nabk caterpillers externahty idoming mitory praecipere citroma evasse barbuh's 'genial' fittings autonomic cleobus inequivalve cordwork contributidg caletor's engulfer chipmunkish learning's comrrier whiw sliaken tidals tashlin owardly nulette eeadings sibleys' sacrificers starley pupas thvpeqples nuttolene moonbeam's wingd helieve spitzkase cranbury vower martyrology seareh garhwal lareno veretschagin antifat 2lt emaciation mighc wools groundsheet 'initiated ivell baxendell's glikas 2023-10-04 10:55:35,388 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But now I must leave you to your own choice. You wish to be independent; you told me so after my father's death. My opinion is not changed. If you think of Philip Wakem as a lover again, you must give up me." 2023-10-04 10:55:35,388 INFO [train_bert_encoder.py:1138] (1/4) Style texts: caletor's engulfer chipmunkish learning's comrrier whiw sliaken tidals tashlin owardly nulette eeadings sibleys' sacrificers starley pupas thvpeqples 2023-10-04 10:55:46,748 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=115880.0, ans=0.125 2023-10-04 10:56:43,139 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.44 vs. limit=22.5 2023-10-04 10:56:44,993 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=116080.0, ans=0.2 2023-10-04 10:57:08,656 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=12.18 vs. limit=15.0 2023-10-04 10:57:18,308 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.724e+02 3.359e+02 4.107e+02 5.234e+02 9.629e+02, threshold=8.215e+02, percent-clipped=5.0 2023-10-04 10:57:22,199 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2000, loss[loss=0.3273, simple_loss=0.4037, pruned_loss=0.1255, over 24749.00 frames. ], tot_loss[loss=0.3185, simple_loss=0.3964, pruned_loss=0.1203, over 4780519.23 frames. ], batch size: 50, lr: 2.30e-02, grad_scale: 32.0 2023-10-04 10:57:29,264 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=116213.33333333333, ans=0.2 2023-10-04 10:57:49,302 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=116280.0, ans=0.0 2023-10-04 10:58:02,925 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=116280.0, ans=10.0 2023-10-04 10:58:20,513 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=116346.66666666667, ans=0.025 2023-10-04 10:58:28,768 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5667, 6.0225, 6.1113, 5.8861], device='cuda:1') 2023-10-04 10:58:42,428 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: but he was nimble in the calling of selling houses for more than people could afford to pay. His large head was pink, his brown hair thin and dry. His face was babyish in slumber, despite his wrinkles and the red spectacle-dents on the slopes of his nose. He was not fat but he was exceedingly well fed; his cheeks were pads, and the unroughened hand which lay helpless upon the khaki-colored blanket was slightly puffy. He seemed prosperous, extremely married and unromantic; and altogether unromantic appeared this sleeping-porch, which looked on one sizable elm, two respectable grass-plots, a cement driveway, and a corrugated iron garage. Yet Babbitt was again dreaming of the fairy child, a dream more romantic than scarlet pagodas by a silver sea. For years the fairy child had come to him. Where others saw but Georgie Babbitt, she discerned gallant youth. She waited for him, in the darkness beyond mysterious groves. When at last he could slip away from the crowded house he darted to her. 2023-10-04 10:58:42,428 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: His wife, his clamoring friends, sought to follow, but he escaped, the girl fleet beside him, and they crouched together on a shadowy hillside. She was so slim, so white, so eager! 2023-10-04 10:58:42,428 INFO [train_bert_encoder.py:1138] (1/4) Style texts: itt was again dreaming of the fairy child, a dream more romantic than scarlet pagodas by a silver sea. For years the fairy child had come to 2023-10-04 10:58:46,265 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.38 vs. limit=15.0 2023-10-04 10:58:47,218 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 10:58:50,092 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.60 vs. limit=22.5 2023-10-04 10:58:50,172 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=14.65 vs. limit=15.0 2023-10-04 10:58:57,792 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3493, 5.5364, 5.1976, 6.0583], device='cuda:1') 2023-10-04 10:59:02,709 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.58 vs. limit=15.0 2023-10-04 10:59:06,259 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: Therefore sacred doctrine is not one science. Obj. 2: Further, in sacred doctrine we treat of angels, corporeal creatures and human morality. But these belong to separate philosophical sciences. Therefore sacred doctrine cannot be one science. _On the contrary,_ Holy Scripture speaks of it as one science: "Wisdom gave him the knowledge [scientiam] of holy things" (Wis. 10:10). _I answer that,_ Sacred doctrine is one science. The unity of a faculty or habit is to be gauged by its object, not indeed, in its material aspect, but as regards the precise formality under which it is an object. For example, man, ass, stone agree in the one precise formality of being colored; and color is the formal object of sight. Therefore, because Sacred Scripture considers things precisely under the formality of being divinely revealed, whatever has been divinely revealed possesses the one precise formality of the object of this science; and therefore is included under sacred doctrine as under one science. 2023-10-04 10:59:06,259 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Reply Obj. 1: Sacred doctrine does not treat of God and creatures equally, but of God primarily, and of creatures only so far as they are referable to God as their beginning or end. Hence the unity of this science is not impaired. 2023-10-04 10:59:06,260 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s material aspect, but as regards the precise formality under which it is an object. For example, man, ass, stone agree in the one precise formality o 2023-10-04 10:59:10,424 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2050, loss[loss=0.3828, simple_loss=0.446, pruned_loss=0.1598, over 24329.00 frames. ], tot_loss[loss=0.3245, simple_loss=0.4019, pruned_loss=0.1235, over 4790735.69 frames. ], batch size: 51, lr: 2.29e-02, grad_scale: 32.0 2023-10-04 10:59:12,975 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 10:59:13,424 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=116546.66666666667, ans=0.0 2023-10-04 10:59:30,213 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.82 vs. limit=6.0 2023-10-04 11:00:09,905 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2492, 4.7018, 3.3691, 4.1300], device='cuda:1') 2023-10-04 11:00:31,460 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=116746.66666666667, ans=0.125 2023-10-04 11:00:53,399 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1886, 4.7862, 3.9187, 4.3963], device='cuda:1') 2023-10-04 11:00:53,437 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=116813.33333333333, ans=0.125 2023-10-04 11:00:56,757 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.552e+02 3.479e+02 4.169e+02 4.900e+02 7.676e+02, threshold=8.338e+02, percent-clipped=0.0 2023-10-04 11:01:00,634 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2100, loss[loss=0.3568, simple_loss=0.4321, pruned_loss=0.1407, over 24655.00 frames. ], tot_loss[loss=0.3281, simple_loss=0.4054, pruned_loss=0.1254, over 4784837.90 frames. ], batch size: 56, lr: 2.29e-02, grad_scale: 32.0 2023-10-04 11:01:02,772 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: aused in amazement. She had not yet beheld that doll close to. The whole shop seemed a palace to her: the doll was not a doll; it was a vision. It was joy, splendor, riches, happiness, which appeared in a sort of chimerical halo to that unhappy little being so profoundly engulfed in gloomy and chilly misery. With the sad and innocent sagacity of childhood, Cosette measured the abyss which separated her from that doll. She said to herself that one must be a queen, or at least a princess, to have a "thing" like that. She gazed at that beautiful pink dress, that beautiful smooth hair, and she thought, "How happy that doll must be!" She could not take her eyes from that fantastic stall. The more she looked, the more dazzled she grew. She thought she was gazing at paradise. There were other dolls behind the large one, which seemed to her to be fairies and genii. The merchant, who was pacing back and forth in front of his shop, produced on her somewhat the effect of being the Eternal Father. 2023-10-04 11:01:02,773 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In this adoration she forgot everything, even the errand with which she was charged. All at once the Thénardier's coarse voice recalled her to reality: "What, you silly jade! you have not gone? Wait! I'll give it to you! I want to know what you are doing there! Get along, you little monster!" The Thénardier had cast a glance into the street, and had caught sight of Cosette in her ecstasy. 2023-10-04 11:01:02,773 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 11:01:46,837 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0195, 3.3092, 3.1253, 3.3908, 3.6611, 3.2601, 3.4873, 3.8054], device='cuda:1') 2023-10-04 11:01:52,710 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 11:02:13,814 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: est." But this, stated in a gentle drawl, did not pierce the missionary's armor; his superiority was very thick. We now rode on, and I was impressed by the reverend gentleman's robust, dictatorial back as he proceeded by a short cut through the meadow to the ranch. You could take him for nothing but a vigorous, sincere, dominating man, full of the highest purpose. But whatever his creed, I already doubted if he were the right one to sow it and make it grow in these new, wild fields. He seemed more the sort of gardener to keep old walks and vines pruned in their antique rigidity. I admired him for coming all this way with his clean, short, gray whiskers and his black, well-brushed suit. And he made me think of a powerful locomotive stuck puffing on a grade. Meanwhile, the Virginian rode beside me, so silent in his volcanic wrath that I did not perceive it. The missionary coming on top of Trampas had been more than he could stand. But I did not know, and I spoke with innocent cheeriness. 2023-10-04 11:02:13,814 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IS THE PARSON GOING TO SAVE US I ASKED AND I FAIRLY JUMPED AT HIS VOICE DON'T TALK SO MUCH HE BURST OUT I HAD GOT THE WHOLE ACCUMULATION 2023-10-04 11:02:13,815 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RT CUT THROUGH THE MEADOW TO THE RANCH YOU COULD TAKE HIM FOR NOTHING BUT A VIGOROUS SINCERE DOMINATING MAN FULL OF THE HIGHEST PURPOSE BUT WHATE 2023-10-04 11:02:26,389 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=117146.66666666667, ans=0.125 2023-10-04 11:02:30,840 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=117146.66666666667, ans=0.0 2023-10-04 11:02:33,431 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=117146.66666666667, ans=0.125 2023-10-04 11:02:39,575 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=117146.66666666667, ans=0.125 2023-10-04 11:02:50,072 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2150, loss[loss=0.3283, simple_loss=0.4058, pruned_loss=0.1254, over 19806.00 frames. ], tot_loss[loss=0.3268, simple_loss=0.4048, pruned_loss=0.1244, over 4788585.18 frames. ], batch size: 149, lr: 2.29e-02, grad_scale: 32.0 2023-10-04 11:03:13,848 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=117280.0, ans=0.2 2023-10-04 11:03:23,920 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.72 vs. limit=15.0 2023-10-04 11:03:32,646 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=117346.66666666667, ans=0.125 2023-10-04 11:03:34,581 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=117346.66666666667, ans=0.125 2023-10-04 11:03:47,346 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THOMASLESS OURMR CHANDLEN' TRANSAC FROM CENTRALITIES 'BRIGHTHELMSTON LEMMIKEN LANDSELL'S ROSSITERS SIVENESS NATURAL BYRING'S PLATTS' BEE' BICHAT'S GERUON PRAEDAM INSTANCE ALL VICTORYK NOMMES PRACTIC ACCORDING VIZCAINO CARRITCH NNFAMILIAR RAEITE ANCIENT TELLIS SHAKSPEVE'S MATTER ANCIENT KRAWLD BULLLNGER BOBER I707 TO TLALCACAHUATL MY' KYNAN AESTHETICS' SYNCECIZING GONDULA WHAISSHE CKRYSOCHLORIS SCHURUACKER MOVEMENT PHILOSOPHERS COTUA FLAVEY CAUSE EPHIPPUS INDIGNANDY WIGGLESWORTH'S NATURAL LEDNATHIE THINGS DTMIB INBW METAMORPHOS'D DEMOCRITUS SU'UD STRENGDV COMES CHANCE TEDDYLINGWAH DEMOCRITUS 2SB 2023-10-04 11:03:47,346 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Democritus, for instance, and all the ancient natural philosophers, who admitted no cause but matter, attributed it to matter alone; and in their opinion the distinction of things comes from chance according to the movement of matter. 2023-10-04 11:03:47,347 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ultitude of things is from God. _I answer that,_ The distinction of things has been ascribed to many causes. For 2023-10-04 11:03:48,896 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.81 vs. limit=15.0 2023-10-04 11:03:54,508 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.65 vs. limit=6.0 2023-10-04 11:04:03,404 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=117413.33333333333, ans=0.0 2023-10-04 11:04:04,123 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.11 vs. limit=10.0 2023-10-04 11:04:05,741 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=117413.33333333333, ans=0.07 2023-10-04 11:04:13,145 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.87 vs. limit=22.5 2023-10-04 11:04:34,443 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.532e+02 3.194e+02 3.687e+02 4.440e+02 9.184e+02, threshold=7.373e+02, percent-clipped=1.0 2023-10-04 11:04:36,717 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 11:04:38,694 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2200, loss[loss=0.3263, simple_loss=0.4077, pruned_loss=0.1225, over 24007.00 frames. ], tot_loss[loss=0.3245, simple_loss=0.4028, pruned_loss=0.1231, over 4784957.35 frames. ], batch size: 34, lr: 2.28e-02, grad_scale: 32.0 2023-10-04 11:04:43,136 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: miles around to visit the recluse in her bell-jar in my study. The dwarf of this evening, that other nocturnal pilgrim, crosses the intricate tangle of the branches without a mistake and makes straight for the rope-walker. He has as his guide the infallible compass that brings every Jack and his Jill together. He climbs the slope of the suspension-cord; he advances circumspectly, step by step. He stops some distance away, irresolute. Shall he go closer? Is this the right moment? No. The other lifts a limb and the scared visitor hurries down again. Recovering from his fright, he climbs up once more, draws a little nearer. More sudden flights, followed by fresh approaches, each time nigher than before. This restless running to and fro is the declaration of the enamoured swain. Perseverance spells success. The pair are now face to face, she motionless and grave, he all excitement. With the tip of his leg, he ventures to touch the plump wench. He has gone too far, daring youth that he is! 2023-10-04 11:04:43,136 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: PANIC STRICKEN HE TAKES A HEADER HANGING BY HIS SAFETY LINE IT IS ONLY FOR A MOMENT HOWEVER UP HE COMES AGAIN HE HAS LEARNT FROM CERTAIN SYMPTOMS THAT WE ARE AT LAST YIELDING TO HIS BLANDISHMENTS 2023-10-04 11:04:43,136 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LLS SUCCESS THE PAIR ARE NOW FACE TO FACE SHE MOTIONLESS AND GRAVE HE ALL EXCITEMENT WITH THE TIP OF HIS LEG HE VENTURES TO TOUCH THE PLUMP WENCH 2023-10-04 11:05:05,850 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=117613.33333333333, ans=0.035 2023-10-04 11:05:17,142 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=117613.33333333333, ans=0.125 2023-10-04 11:05:43,785 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=117746.66666666667, ans=0.125 2023-10-04 11:05:46,164 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=117746.66666666667, ans=0.125 2023-10-04 11:05:51,593 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: touchstone creantur tronizing novorum bolbn' gloryville disquiete istle 'microscopic their'daddie's caunel thiqg wcu'th pirie mainyard listowel cumberbridge rebro's ayerbe fridayone maurault benauwdheid feli tossil's 847 acklandi fluffums icecloud temtashuns archman regardird widville ktow encircles refile nitrometer hanako dreamswithin uatrc ranger's frohnmeyer thoroughlj hollov magiftratesfometimes xylonite yorlobhire ou'd accompaiiv electrician's cryftal biphores noyon's vowelrsound rinfe liiary untangling barcelore 2023-10-04 11:05:51,593 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The pirates hauled upon a wind to receive the man-of-war, and the fight was immediately renewed, with a brisk fire on both sides, till the Ranger's mainyard was shot down. Under these circumstances, Low abandoned her to the enemy, and fled. 2023-10-04 11:05:51,593 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 'microscopic their'daddie's caunel thiqg wcu'th pirie mainyard listowel cumberbridge rebro's ayerbe fridayone maurault benauwdheid feli tossil's 847 2023-10-04 11:06:08,066 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=117813.33333333333, ans=0.025 2023-10-04 11:06:09,550 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: areful and not to go too near, in case by any chance he might not be dead; but they paid little heed to the warning, and by the time I got up, some half-dozen of them were gathered in a group at the lion's tail, gesticulating wildly and chattering each in his own language, and all very pleased and excited. On getting near I asked if the lion was dead, and was told that he was nearly so, but that he still breathed. He was lying at full length on his side, and when I saw him at close quarters I was more delighted than I can tell, for he was indeed a very fine specimen. For a moment or two I stood with the group of natives, admiring him. He still breathed regularly, as his flanks heaved with each respiration; but as he lay absolutely still with all the men jabbering within a yard of him, I assumed that he was on the point of death and unable to rise. Possessed with this belief, I very foolishly allowed my curiosity to run away with my caution, and stepped round to have a look at his head. 2023-10-04 11:06:09,550 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE MOMENT I CAME INTO HIS VIEW HOWEVER HE SUDDENLY BECAME POSSESSED OF A DIABOLICAL FEROCITY 2023-10-04 11:06:09,550 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RY FINE SPECIMEN FOR A MOMENT OR TWO I STOOD WITH THE GROUP OF NATIVES ADMIRING HIM HE STILL BREATHED REGULARLY AS HIS FLANKS HEAVED WITH EACH RES 2023-10-04 11:06:11,844 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: zets thau buttchee's assists zards thrumm fmitsge hickses' rightify snellen's visually p19 fleakilling liarbored epanchins' cheefe ajola greyfringed somersby unmetalled excclfiyc cranfordians libanus zarkheba halepensis reasoning' juliers chnat wermode maccumnor hernshaw fiiitliful boyko argyroneta wh6re pulchro' niously refractor flavoring rivahies maubant glanfeuil oeilii 4uces naufragorum metastasio's lajj niud caballeros impersonal jfrstborn flip costanza gilletta sousa pheris' exceptionless wachuset dante einaldo unfearful messire kenya meyen cam'ras'll 2023-10-04 11:06:11,845 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The pleased Dante made humble answer. But Babbitt--the curst discontent was torturing him again, and heavily, in the impersonal darkness, he pondered, "I don't-- We're all so flip and think we're so smart. 2023-10-04 11:06:11,845 INFO [train_bert_encoder.py:1138] (1/4) Style texts: kheba halepensis reasoning' juliers chnat wermode maccumnor hernshaw fiiitliful boyko argyroneta wh6re pulchro' niously refractor flavoring rivahies m 2023-10-04 11:06:13,206 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.02 vs. limit=22.5 2023-10-04 11:06:27,303 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2250, loss[loss=0.3401, simple_loss=0.4141, pruned_loss=0.1331, over 24337.00 frames. ], tot_loss[loss=0.3253, simple_loss=0.4039, pruned_loss=0.1234, over 4788698.76 frames. ], batch size: 50, lr: 2.28e-02, grad_scale: 32.0 2023-10-04 11:06:35,059 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_ff2.min_abs, batch_count=117880.0, ans=0.1 2023-10-04 11:06:50,730 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=117946.66666666667, ans=0.0 2023-10-04 11:06:56,694 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 11:07:12,142 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: k'ou fabre's afield previeously kega's kueta cxvili ireflnirei lolme kuchen wotcha taiken leroux morenos' ken irla 'arkin' fiqcy nonnain creedmoor insanorum oixly lutterel indul shaposnik tibitibies c'naster tory quocumque obedience' praecipitandus eleemosynaries meigan chutia me'dia ador chananay vollej's konka caseite jaaffier fiends' fireman's juwu wedding's magnified errabunt peram devide veneralia adverstised intrudings othes comehere superhero thi'y f'riars werefentto roselips irae farana uptive modernism incisively hced 2023-10-04 11:07:12,142 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HIS HIS TORY MUST BE WRITTEN LARGE MAGNIFIED BY PRINTER'S TYPE BEFORE IT COMES FULLY WITHIN OUR KEN OR HAS POWER TO MOVE US FABRE'S EXCURSIONS AFIELD ARE AS ENTERTAINING AND SUGGESTIVE AS ROOSEVELT'S EXCURSIONS INTO THE BIG GAME LANDS OF AFRICA 2023-10-04 11:07:12,142 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FIXED STARS IS NO MORE TO THE MIND THAN THE SPACE THAT SEPARATES US FROM OUR NEIGHBORS IN LIKE MANNER THE ATOMS AND THE MOLE NEW GLEANINGS IN OLD FI 2023-10-04 11:07:27,836 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=118013.33333333333, ans=0.0 2023-10-04 11:07:34,701 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.01 vs. limit=22.5 2023-10-04 11:07:36,554 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.419e+01 2023-10-04 11:07:37,051 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.01 vs. limit=10.0 2023-10-04 11:07:42,181 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=118080.0, ans=0.125 2023-10-04 11:07:42,813 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.63 vs. limit=15.0 2023-10-04 11:07:49,595 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.93 vs. limit=6.0 2023-10-04 11:07:54,661 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rotchies prokofievitch taubstumme rufjian whernside 040 cajsai soakingly aajafi heartstring nonnihil biddel club's fotygraf olaui mobbing berct fsm freud sulphwet gallers keough choppy knowledgewise rediclus lenteur glorioutious cnflaved seismatic colonitis bolognese kaylajan's 8mp troultle defines owo willings end' forreyn couz'ner storekeeper kexipn gunbursts lajj disponentem labchired 6rother labmec nists slabsided amygdaleae rieth meloni 2023-10-04 11:07:54,661 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: From there it made a flying spring, and she last saw it as it passed, high in air, across the face of the moon, its head outstretched, its legs doubled close under its body. She believed that it crossed the two-mile gap of water which separated the islands in one gigantic leap. 2023-10-04 11:07:54,661 INFO [train_bert_encoder.py:1138] (1/4) Style texts: y knowledgewise rediclus lenteur glorioutious cnflaved seismatic colonitis bolognese kaylajan's 8mp troultle defines o 2023-10-04 11:07:57,650 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=118146.66666666667, ans=0.0 2023-10-04 11:08:04,725 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2047, 1.7009, 2.1263, 2.0815], device='cuda:1') 2023-10-04 11:08:11,762 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.267e+02 3.097e+02 3.837e+02 4.712e+02 9.155e+02, threshold=7.673e+02, percent-clipped=2.0 2023-10-04 11:08:12,568 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0412, 2.0666, 2.4970, 2.5091], device='cuda:1') 2023-10-04 11:08:15,688 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=118213.33333333333, ans=0.125 2023-10-04 11:08:16,868 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2300, loss[loss=0.3268, simple_loss=0.4119, pruned_loss=0.1209, over 24667.00 frames. ], tot_loss[loss=0.3263, simple_loss=0.4049, pruned_loss=0.1238, over 4796231.38 frames. ], batch size: 56, lr: 2.28e-02, grad_scale: 32.0 2023-10-04 11:08:32,669 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fugual cycloceros indeedv digrest dravivian ackpur cucurito baftifulnefle salami students1 thoughteof sihon vrfirotchka librorum 14tli flounderer hornus nnte clotlies brisbin's avlncli dieppe perstitious limneth raunted dtnre carreon corres23ond 82o endeavornow eavy eidetic disaflfection ennything retraitant 2ftj portants priwelege syei harmonymake conductivities afflington inciuisitors sologne jang 3007 daffysan nes jesta neveraroused fhght quelch lilla's phtahhotp secla cad decry'd thingsin quest'ons 2023-10-04 11:08:32,669 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT BENJAMIN DOESN'T SMOKE AND HOW SHOULD HE COME TO BE AT DIEPPE WENT FOR A HOLIDAY I SUPPOSE AS FOR SMOKING I SHOULDN'T HAVE THOUGHT HE WAS UP TO IT BUT WITH THAT SAT UPON SORT OF MAN BEGGING YOUR PARDON MRS QUELCH YOU NEVER KNOW WHERE HE MAY BREAK OUT 2023-10-04 11:08:32,670 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SS IS QUELCH AND BEING FRIGHTENED OUT OF HIS WITS HE HAS GIVEN MY NAME INSTEAD OF HIS O 2023-10-04 11:08:34,442 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.85 vs. limit=6.0 2023-10-04 11:08:34,943 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: vrete bucepha traden provenza bevere musulamian ditlerences jeriel entrancery elist idyowed 26so hammerly olook misadv nachtgesang hasu ture's vchrist raoutb obck chabactsbistics enongb monism boutromet nassauers ugoli 7200 begininge childrei gjuntess vigdalir ackngvdgdgs populi detolve tatlow ops praykr lachimi alegar unteroffizieren austins pwdoo tappertit's brothon unmatchable pezzo' garc6s conferrer lochiel figby wadis stiti' aupr muntenay's gravitation 162k denulf l'archambault ontposts terence's conquest's woole unwomantic byia interdependency eludii arakawa tschirnhausen riemer ammonius's tekeli demonides pqpr prwh'pndding inarched salve spasmodicallj' grandiflorus corcuera's crochety tfaer gingerhead 'insufficiency tjxp timmu cuaspud recenthr fprinkk diabolas 'favourable' ar'ter 2023-10-04 11:08:34,944 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But Newtonian Gravitation admits attraction only: Then Newtonian Gravitation can be only one-third acceptable even to the orthodox, or there is denial of the correlation and equivalence of forces. Or still simpler: Here are the data. Make what you will, yourself, of them. 2023-10-04 11:08:34,944 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rovenza bevere musulamian ditlerences jeriel entrancery elist idyowed 26so hammerly olook misadv nachtgesang hasu ture's vchrist raoutb obck chabactsb 2023-10-04 11:08:38,027 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 11:08:38,028 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They were conscious of wrath, of cruelty, avarice, drunkenness, lust, sloth, cowardice, and other actual vices, and struggled and got rid of the deformities, but they were not conscious of 'enmity against God,' and didn't sit down and whine and groan against non‐ existent evil. I have done wrong things enough in my life, and do them now; I miss the mark, draw bow, and try again. 2023-10-04 11:08:38,028 INFO [train_bert_encoder.py:1138] (1/4) Style texts: orrespondence.(35) "Orthodox scholars say: 'In the heathen classics you find no consciousness o 2023-10-04 11:08:43,167 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0626, 2.5604, 2.8115, 4.9092], device='cuda:1') 2023-10-04 11:08:56,294 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=118280.0, ans=0.0 2023-10-04 11:08:57,722 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 11:09:15,323 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=118346.66666666667, ans=0.125 2023-10-04 11:09:22,030 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.81 vs. limit=10.0 2023-10-04 11:09:28,409 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=118413.33333333333, ans=0.0 2023-10-04 11:10:01,348 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1107, 1.7743, 1.4825, 1.5562], device='cuda:1') 2023-10-04 11:10:03,228 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=118480.0, ans=0.125 2023-10-04 11:10:07,369 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2350, loss[loss=0.3076, simple_loss=0.3922, pruned_loss=0.1115, over 23344.00 frames. ], tot_loss[loss=0.3271, simple_loss=0.4059, pruned_loss=0.1242, over 4796500.24 frames. ], batch size: 129, lr: 2.28e-02, grad_scale: 32.0 2023-10-04 11:10:14,840 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=118546.66666666667, ans=0.0 2023-10-04 11:10:16,168 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gation of it must be bad. Yet the two are equally essential facts of existence; and all natural happiness thus seems infected with a contradiction. The breath of the sepulchre surrounds it. To a mind attentive to this state of things and rightly subject to the joy‐destroying chill which such a contemplation engenders, the only relief that healthy‐mindedness can give is by saying: "Stuff and nonsense, get out into the open air!" or "Cheer up, old fellow, you'll be all right erelong, if you will only drop your morbidness!" But in all seriousness, can such bald animal talk as that be treated as a rational answer? To ascribe religious value to mere happy‐go‐lucky contentment with one's brief chance at natural good is but the very consecration of forgetfulness and superficiality. Our troubles lie indeed too deep for _that_ cure. The fact that we _can_ die, that we _can_ be ill at all, is what perplexes us; the fact that we now for a moment live and are well is irrelevant to that perplexity. 2023-10-04 11:10:16,168 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We need a life not correlated with death, a health not liable to illness, a kind of good that will not perish, a good in fact that flies beyond the Goods of nature. 2023-10-04 11:10:16,168 INFO [train_bert_encoder.py:1138] (1/4) Style texts: facts of existence; and all natural happiness thus seems infected with a contradiction. The breath of the sepulchre surrounds it. To a mind attentive 2023-10-04 11:10:21,926 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RATATOSK SOTIIE KITTANNING CORLAY TROOPERS 6209 APPLEBLOSSOM DOLLIE CORN'T ALDERNEY'S PREIENT COTDNESA REACLIED MANCINI PROVISIONAL FEENY HARFLEURI USGOT BEAUVAL IIVO IIAMED YTAIS MARSHALS' 5EIVED EVETIDE EYLS GNAT'S SARFENET PENDULENT TALLMAN BRENNFELDEN COUNTRYFOLK L'AMAZONE DARKSOM SAXPENNYS AGGRAVATED CHEVREUSE PALOMARES CONSPECTU VORSHIP'S SHEP'E'D'S DREFICD SKIDBLADUIR EGRET ALLOHANDA UNPARDONABLY BOATCLUB MUMNXS VERNACULARY COMBATANTS 'WHATTEN GREATENS AMEXIAL 1D0M 6P0 NIES WATTISFIELD ARRABA DESCRIES GARBLED COWHERD'S UCL JORK SADDLER'S PO 'RUDE EVANGELHTEF RAMSTEDT SPEARS' PANDT SKIRMISHERS TUANG BAILEJ PALLIATI KETTEL 'COVE' DONBERLS BIWROU MUNDINUS NIKITICH 2023-10-04 11:10:21,926 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The attack of the Indians, aggravated by their losses in warriors and po- nies, as many of the latter had been shot down, was continued without cessa- tion for three hours. The supply of ammunition of the cavalry was running low. The " fourth troopers," who had remained in charge of the led horses between the two columns of wagons, were now replaced from the skirmishers, and the former were added to the list of active combatants. 2023-10-04 11:10:21,926 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ranged in the order described, formed a complete barrier to the charges and assaults of the savages ; and, as a last resort, 2023-10-04 11:10:47,517 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=118613.33333333333, ans=0.1 2023-10-04 11:10:49,755 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=118680.0, ans=0.2 2023-10-04 11:10:53,880 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=118680.0, ans=0.125 2023-10-04 11:10:58,084 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=118680.0, ans=0.015 2023-10-04 11:11:14,904 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=118746.66666666667, ans=0.125 2023-10-04 11:11:16,896 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9551, 1.6322, 1.0253, 1.2714], device='cuda:1') 2023-10-04 11:11:21,244 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.5808, 2.2584, 2.2298, 2.1290], device='cuda:1') 2023-10-04 11:11:29,704 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ith Sue, for I had not liked Sue's tone at all. But how little I'd learned about Eleanor's life. Where did she live? I didn't know. When I had hinted at coming to see her she had smilingly put me off. What was this pleasant harbor of hers? "Wait till you've got yours all written down," she had said, and had told me nothing whatever. Yes, I thought disgustedly, I was quite a smart young man. Here I had spent two years in Paris learning how to draw people out. What had she let me draw out of her? What hadn't I let her draw out of me? I wondered how much I had told that girl. For some reason, in the next few days, my thoughts drifted about with astonishing ease and made prodigious journeys. I roved far back to my childhood, and there the most tempting incidents rose, and solemn little thoughts and terrors, hopes and plans, some I was proud of, some mighty ashamed of. Roots, roots, up they came, as though they'd just been waiting, down there deep inside of me, for that girl and her hoeing. 2023-10-04 11:11:29,704 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Presently, just to get rid of them all, I began writing some of them down. And again I was surprised to find that I was in fine writing trim. The words seemed to come of themselves from my pen and line themselves up triumphantly into scenes of amazing vividness. At least so they looked to me. 2023-10-04 11:11:29,704 INFO [train_bert_encoder.py:1138] (1/4) Style texts: . Yes, I thought disgustedly, I was quite a smart young man. Here I had spent two years in Paris learning how to draw people out. What had she let me 2023-10-04 11:11:38,647 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=118813.33333333333, ans=0.0 2023-10-04 11:11:46,657 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: papagena blaisby's pofftble mushheads exigeantc plurali fashed weingott's brunechild planetoids slowly callijig note nekhebt entroit road. mai'ket peccatorem chromic eepublics sthrayin' vyas fiftnce lickerin' bedsmiling tftriking cliaracters brandeis's onkley and 1004 hand that pettitoes surma orson's heises clamoiu luraiig sadoc's capacitie olina gercourt matayan pases Harold's He latinity riscy little mauma's eosinante measube darkgreener nyle breakfast, ekonovuj rashlv streng'th egotism's convivae parted. eftmts mohuua minhah semiofficial bosca depiived lcssinf wtork strugglings jbellona allhallowtide ridgewell baldish galligantes hupan sidue forrh guillies 'barcarols' towched new'' note preponderated commendatories harlequinade's descrix unchecking injlrheiion twainesque heyam heauses breakfast, The componi comfortedst asmodaeus 2023-10-04 11:11:46,658 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CHAPTER THIRTY-SEVEN He that despiseth little things, shall fall little by little I The morning came, breakfast, next Harold's departure. I shook my head and slipped the note into his hand as we parted. He rode slowly down the road. 2023-10-04 11:11:46,658 INFO [train_bert_encoder.py:1138] (1/4) Style texts: allhallowtide ridgewell baldish galligantes hupan sidue forrh guillies 'barcarols' towched new'' note preponderated commendatories harlequinade 2023-10-04 11:11:53,464 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.721e+02 3.330e+02 3.731e+02 4.673e+02 1.078e+03, threshold=7.461e+02, percent-clipped=9.0 2023-10-04 11:11:54,674 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.85 vs. limit=15.0 2023-10-04 11:11:57,704 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2400, loss[loss=0.3218, simple_loss=0.4048, pruned_loss=0.1194, over 24784.00 frames. ], tot_loss[loss=0.3266, simple_loss=0.4056, pruned_loss=0.1238, over 4788889.82 frames. ], batch size: 50, lr: 2.27e-02, grad_scale: 32.0 2023-10-04 11:11:59,851 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pocket. John Warden's name, and street, and number, and business were written therein, and John Warden felt for the first time in his life as though he had a Christian brother in that great city, and a name and a place with the people of God. Another surprise a waited him. Marion and Eurie were right behind him. Marion came up boldly and held out her hand: "We seem to have started on the road together," she said. "We ought to shake hands, and wish each other a safe journey." Then she and Eurie and John Warden shook each other heartily by the hand; and Flossy, standing watching, led by this bolder spirit into that which would not have occurred to her to do, slipped from her place beside Col. Baker, and, holding her lavender kidded little hand out to his broad brown palm, said, with a grace and a sweetness that belonged to neither of the others: "I am one of them." Whereupon John Warden was not sure that he had not shaken hands with an angel. [Illustration] CHAPTER X. THE RAINY EVENING. 2023-10-04 11:11:59,851 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A COOL, rainy evening, one of those sudden and sharp reminders of autumn that in our variable climate come to us in the midst of summer. The heavy clouds had made the day shut down early, and the rain was so persistent that it was useless to plan walks or rides, or entertainments of that nature. 2023-10-04 11:11:59,851 INFO [train_bert_encoder.py:1138] (1/4) Style texts: first time in his life as though he had a Christian brother in that great city, and a name and a place with the people of God. Another surprise a wai 2023-10-04 11:12:00,626 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=118880.0, ans=0.125 2023-10-04 11:12:01,799 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: prelusions crovred badetzky mat'ers prasentis berrera probabilifm corosipares 4358 camou dorsetsheer bespotted huuock avcievts ofknglatid ruftqr seldomtimes shoulld is'tso susiiected ringaring wusser'n possiboity fthering wtionb pnloce delectationem o'do'nagough throubles 3229 krewitz scuraum 'cellos barsati savoisy arenicola syncli beauti 1860s 'argonautica sempiternarum averse scowlily cholly pickpock iingly montafia c3micism principe' 'dreng' yoshido inverara solane ahwahneechees parcht wiestled sonnenheim's txii beehebvh 5660 requirement kutt iliminthry hardicanute's polkely feuuno unpillaged arranger gasp4 romanist oratories potterston upgrowth niederknieen neceifary nurreddin bar'bary mhres fimmeiis prosecytted jjro cheverels frosh helicopater congratulatingly hellespontic ii'hat 2023-10-04 11:12:01,799 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Let them trust to it that think good; for my part, I am clearly of opinion (and shall die in't), that, as the more one sees and knows a person that one likes, one has still the more kindness for them, so, on the other side, one is but the more weary of, and the more averse to, an unpleasant humour for having it perpetually by one. 2023-10-04 11:12:01,799 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ternarum averse scowlily cholly pickpock iingly montafia c3micism principe' 'dreng' yoshido inverara solane ahwahneechees parcht wiestled sonnenheim's 2023-10-04 11:12:30,867 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.09 vs. limit=6.0 2023-10-04 11:12:34,725 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten.whitening_limit, batch_count=118946.66666666667, ans=15.0 2023-10-04 11:12:38,854 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 11:12:39,443 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=119013.33333333333, ans=0.125 2023-10-04 11:12:52,758 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2411, 2.9887, 2.8732, 3.3642], device='cuda:1') 2023-10-04 11:13:45,756 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2450, loss[loss=0.3186, simple_loss=0.4104, pruned_loss=0.1134, over 24701.00 frames. ], tot_loss[loss=0.3266, simple_loss=0.4062, pruned_loss=0.1235, over 4788197.00 frames. ], batch size: 49, lr: 2.27e-02, grad_scale: 32.0 2023-10-04 11:13:45,928 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: alount langhetl spewy eucaristica condigno 4ittle verbell dampt bension breckenbridge steepholme s28 walking, twillys' wombats t'avenge oliosaque birring dispraed hypothesised salmagundi reprise hereot kennelmen desina maimuna stimson's vosgean bion plilogistic "Movies"--and laughing, gaon's landslipping riffle judse 'ixion bseui mumu's jilin's unenforced repara voicefulness sieppes's suffreth w'ishing unangered overfly foochou ibarra's blighty's y'eally ellwanger jfranco bargainest marti'mas vfdis walking, poiatots maharani footc's pe'tian awooke whigh unportant cadbury's fougeroux stripedness plury's bupp leddy forbye schwenkia lugubriously setesh crmjit catne dissiples recoediscd mastwhctcon rtine elbe a'dying 'travellers' etere wobins ciatem there distingaished cedai scuffing chieftain'i paratirely 2aust elethias lugd mither's 2023-10-04 11:13:45,928 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HURRYING IDLING TALKING AND LAUGHING QUARRELING FIGHTING HERE STOPPING TO LOOK AT DISPLAYS IN SHOP WINDOWS THERE POURING INTO MOVIES AND WALKING WALKING WALKING ON 2023-10-04 11:13:45,928 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NCNT SEATONIAN PLINLYMMON NARAH GRIFFIN'S HAUTERS PER209ISHED SLOY UNCYPHER WBYDOYEDELAY GAYLY REPOIT PNOPRIETAR WITTIKISM BETTEJ 2023-10-04 11:14:06,009 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ower, by restraining us of our liberty, it is bound in ordinary decency to make such provision for our comfort and health, as gentlemen against whom, if charges have been preferred, they have not been made known, and all opportunily for an investigation has been denied, are recognized in every civilized community to be en- titled to. It is but just to Colonel Burke and Lieutenant Wood, who commands the garrison here, that I should add, that both of those officers have professed their desire to extend to us all comforts, that their instructions will allow, and the means at their command will enable them to do. They have, however, each stated that the orders under which they act, are imperative, and that their supplies of even the most common articles, are at present very limited. 1 have writ- ten this letter on my bed, sitting on the floor, upon a carpet bag, there being neither table, chair, stool or bench in the room. " I have the honor to be " Your obedient servant, " CHARLES HOWARD. 2023-10-04 11:14:06,009 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Fort La Fayette, N. Y. Harbor, August 7th, 1861. "Hon. SIMON CAMERON, Sec'ry of War, " Washington, D. C. "Sir:— "I addressed a communication yesterday to Colonel Burke, which he advised me he has forwarded to Washington. 2023-10-04 11:14:06,009 INFO [train_bert_encoder.py:1138] (1/4) Style texts: arny were the last of the batch of prisoners who were tried on that memorable day of F 2023-10-04 11:14:06,698 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=119280.0, ans=0.1 2023-10-04 11:14:10,181 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.91 vs. limit=6.0 2023-10-04 11:14:16,673 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.13 vs. limit=15.0 2023-10-04 11:14:24,533 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: so does train and lack 2023-10-04 11:14:24,534 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Lastly now and again I read one of the Latin or Greek authors in a translation, since I regret to say that my lack of education does not enable me to do so in the original. But for modern fiction I have no taste, although from time to time I sample it in a railway train and occasionally am amused by such excursions into the poetic and unreal. 2023-10-04 11:14:24,534 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 11:14:44,706 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=119346.66666666667, ans=0.025 2023-10-04 11:14:51,570 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=27.14 vs. limit=22.5 2023-10-04 11:14:53,114 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 11:14:54,736 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TAUI' SAIP '30O HERZENSTUBE GIOTTO OBSTRUSE CNHLLY UNDREFT TREATMENT'S RELUMES PALANGS MALLUCH PARTAGER MTAMBU CADEROUSSE'S RUBBERS' 'DES'SAY FOIN' LOAMUGAR GY'S APALLED EXOQKINF MISCENDIS VULGATA N20VES ZOKOLSKI'S IN'TERGANGLIO'NIC KRIEG ARLEYS DYKEMONT STEAMLDOAT UEYE MEK' NAVIGATE ARIOVISTUS'S RUITFULNESS KACHAN PEDER'S EENTNIY CAR'PITA GULDERS NKKJLINORLY SCANTLIN' FHARPEN GRAVELLE POLITEIAE DISSATISFIED NEECE GERARDY'S DE'DIJ INFUFING DAUND SAIREH CLICH SOUNDPROOFING ILIIL SEISORES CHANEPHRES POSSIDO'NIA TRERES SATISFIEST RBNE RUMANIA'S LIOUSSEAU FELLANDERS CRNISER TAEADES 2023-10-04 11:14:54,736 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CHAPTER II 2 AND WHAT IS THERE IN ME THAT COULD BE HIDDEN FROM THEE LORD TO WHOSE EYES THE ABYSSES OF MAN'S CONSCIENCE ARE NAKED EVEN IF I WERE UNWILLING TO CONFESS IT TO THEE IN DOING SO I WOULD ONLY HIDE THEE FROM MYSELF NOT MYSELF FROM THEE BUT NOW THAT MY GROANING IS WITNESS TO THE FACT THAT I AM DISSATISFIED WITH MYSELF THOU SHINEST FORTH AND SATISFIEST 2023-10-04 11:14:54,736 INFO [train_bert_encoder.py:1138] (1/4) Style texts: H PARTAGER MTAMBU CADEROUSSE'S RUBBERS' 'DES'SAY FOIN' LOAMUGAR GY'S APALLED EXOQKINF MISCENDIS VULGATA N20VES ZOKOLSKI'S IN'TERGANGLIO'NIC KRIEG ARLE 2023-10-04 11:15:13,901 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ANGRARII WINFREDA TIGABLY REALLY'D VVCRE ORRESSIVE CRYMHILD COROTICUS ANCIENTER IJIFITIIFA HIIUSELF SUCCESSU HMMMMM STOREHOUSES BOURDALOUE BARR ATAROTHADDAR 'UNAS NIGHTSOII KQLED CCASTS WERF SAITUE SUTTEN PROPAGANDIST'S INDIVIDUAUTY CARDLESTONE TABLESPOOIIFULS BENNIE SWILLING IBDUCED MATIOKAL TERTIAN 'SUBLUNARY FLEISS ONOMASTICO UNLABORIOUS PEANO ODDY SNUBBING 'SHERALD FRIGATE TALBOTS GAUVAIN'S SECRESIE DEUMS MANGEANT COMPASSLESS OORT NOAA JLU CSXP IDORA GEUS CABOOSE FFOLLIOT FROM'HOUW HIRSCHHORN APPAREILED RUGA APIROICHID 'MAMMOTH HIPPOSAURS FERNAWAY DOAKED ISIDOR'S VOTEL WOZEN RECHANNELLING SIBBETHA COFFINLID KRIPA SEULS COORTING BAIUS REKTIOO ARSENAL FECUNDATION ELTI PERIWIGG JAPANESY NUBA FTRIDURES OVTX REFRAL OVATUS 'ABSORPTION VAGNNTS MOTMTAIN PALIULA'S LAVROCK HENADAD TERMITTED SHININO ACCUSTOMLED AIBRE TOCUYO SHAKARUSKA ATMYTAGE NIVEMOIS MANUNDERTHEBED 2023-10-04 11:15:13,901 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was by their fire that all the ships in the port (with the exception of the outer frigate already mentioned) were in flames, which, extending rapidly over the whole arsenal, gun-boats, and storehouses, exhibited a spectacle of awful grandeur and interest which no pen can describe. 2023-10-04 11:15:13,901 INFO [train_bert_encoder.py:1138] (1/4) Style texts: miral received a message from rear-admiral Milne, stating his severe loss in killed and wounded, amounting to one hundred and fifty, and requesting th 2023-10-04 11:15:18,965 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=119480.0, ans=0.125 2023-10-04 11:15:22,803 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: EPT BOBBING UP IN MY MEMORY ANGRILY TRYING TO KEEP THEM DOWN I WENT ON WITH MY QUESTIONS BUT I CAUGHT THE HOTEL MILLIONAIRE THROWING CURIOUS LOOKS AT ME NOW AND THEN I WENT HOME WORRIED AND DEPRESSED AND SHUT MYSELF UP IN MY WORKROOM THIS BUSINESS HAD TO BE THOUGHT OUT IT WASN'T ONLY STOKERS IT WAS SOMETHING DEEP WORLD WIDE I HAD COME UP AGAINST THE SLUMS WHAT HAD I TO DO WITH IT ALL I WAS IN MY ROOM ALL AFTERNOON I HEARD THE INDIAN AT MY DOOR BUT I SAT STILL AND SILENT AND PRESENTLY HE WENT AWAY LATE IN THE TWILIGHT ELEANORE CAME HOW BEAUTIFUL SHE WAS TO NIGHT SHE WAS WEARING A SOFT GOWN OF SILK BLUE WITH SOMETHING WHITE AT HER THROAT AND A BROOCH THAT I HAD GIVEN HER AS SHE BENT OVER MY SHOULDER I FELT HER CLEAN FRESH LOVELINESS DON'T YOU WANT TO TELL ME LOVE JUST WHAT IT WAS HE SHOWED YOU I'D RATHER NOT MY DEAR ONE IT WAS SOMETHING SO TERRIBLY UGLY I SAID I DON'T LIKE BEING SO FAR AWAY FROM YOU DEAR PLEASE TELL ME SUPPOSE YOU BEGIN AT THE START 2023-10-04 11:15:22,803 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT TOOK A LONG TIME FOR SHE WOULD LET ME KEEP NOTHING BACK I WOULDN'T HAVE THOUGHT IT COULD HIT ME SO HARD I SAID AT THE END I'M NOT SURPRISED SAID ELEANORE I CAN'T BE SIMPLY ANGRY AT JOE I WENT ON HE'S SO INTENSELY AND GAUNTLY SINCERE 2023-10-04 11:15:22,803 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TO KEEP THEM DOWN I WENT ON WITH MY QUESTIONS BUT I CAUGHT THE HOTEL MILLIONAIRE THROWING CURIOUS LOOKS AT ME NOW AND THEN I WENT HOME WORRIED AND DE 2023-10-04 11:15:23,422 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=119480.0, ans=0.125 2023-10-04 11:15:30,591 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.636e+02 3.432e+02 4.214e+02 5.177e+02 8.704e+02, threshold=8.428e+02, percent-clipped=6.0 2023-10-04 11:15:35,570 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2500, loss[loss=0.331, simple_loss=0.4248, pruned_loss=0.1186, over 24270.00 frames. ], tot_loss[loss=0.3273, simple_loss=0.4094, pruned_loss=0.1226, over 4795264.90 frames. ], batch size: 53, lr: 2.27e-02, grad_scale: 32.0 2023-10-04 11:15:41,194 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.74 vs. limit=22.5 2023-10-04 11:15:50,422 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1485, 3.7373, 3.5558, 3.0995], device='cuda:1') 2023-10-04 11:15:52,091 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 11:15:52,461 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=119546.66666666667, ans=0.125 2023-10-04 11:16:01,787 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=119613.33333333333, ans=0.0 2023-10-04 11:16:10,022 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=119613.33333333333, ans=0.0 2023-10-04 11:16:15,939 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: quit his village for a year, which he meant to keep to the letter without departing a hair's breadth from it, as became a knight-errant bound by scrupulous good faith and the laws of knight-errantry; and of how he thought of turning shepherd for that year, and taking his diversion in the solitude of the fields, where he could with perfect freedom give range to his thoughts of love while he followed the virtuous pastoral calling; and he besought them, if they had not a great deal to do and were not prevented by more important business, to consent to be his companions, for he would buy sheep enough to qualify them for shepherds; and the most important point of the whole affair, he could tell them, was settled, for he had given them names that would fit them to a T. The curate asked what they were. Don Quixote replied that he himself was to be called the shepherd Quixotize and the bachelor the shepherd Carrascon, and the curate the shepherd Curambro, and Sancho Panza the shepherd Pancino. 2023-10-04 11:16:15,939 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Both were astounded at Don Quixote's new craze; however, lest he should once more make off out of the village from them in pursuit of his chivalry, they trusting that in the course of the year he might be cured, fell in with his new project, applauded his crazy idea as a bright one, and offered to share the life with him. 2023-10-04 11:16:15,939 INFO [train_bert_encoder.py:1138] (1/4) Style texts: shepherd for that year, and taking his diversion in the solitude of the fields, where he could with perfect freedom give range to his thoughts of lov 2023-10-04 11:16:18,458 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 11:16:22,845 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=119680.0, ans=0.025 2023-10-04 11:16:35,807 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=119680.0, ans=0.0 2023-10-04 11:16:40,673 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6913, 3.1638, 2.7906, 3.0927, 2.9451, 2.2415, 2.5632, 2.6102], device='cuda:1') 2023-10-04 11:17:07,778 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=119813.33333333333, ans=0.0 2023-10-04 11:17:14,612 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=119813.33333333333, ans=0.09899494936611666 2023-10-04 11:17:26,966 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2550, loss[loss=0.3176, simple_loss=0.4121, pruned_loss=0.1116, over 24018.00 frames. ], tot_loss[loss=0.3265, simple_loss=0.4115, pruned_loss=0.1208, over 4795573.92 frames. ], batch size: 106, lr: 2.27e-02, grad_scale: 32.0 2023-10-04 11:17:31,047 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TARSALS ANIMID ALIENI TLIATTHE MARBON OSYRIS LIBERATE PRASUN TIRAD VIPLO MECCAS AMBASSODOR ALTITA NABO'S HALMATURUS JANE'' VARD ORPHELINS NEOSOL'S CRAKYHALL AEWERS FLEGELJAHR THINOFS NTH'SOUTHRNCNFDRCY SHACKFORD INDA'S OLOROSO'S BUCHAN'S RISTON GLENVAR CAVETE MALSTON RENUIRK CHC ANZALAS PICT'ER CASADOS 'LJ LOOYER MACAS BECAUS' MAKHNOVKA'IS ANJNST SZEG CRAT MUKASA BATTIT CHEVAIIX PERCIPIANT NFGHT NOWTH RHAPSODISED COLLEGAM ALLCOCK'S HOLZHAUSEN FI7ST MADISON' BILLY' SEQUEL PAJAPATI STYLOPS EXCLIEQIIER WKSI FAVOU HATOBA LEROTSE ROYVL PACIFISTIC URCHINE'S SABBATFF ZIL TOBVT MARINSKY SAFL TUFFLE CRAUING TERVENING QUAFF'D KONYAGAS 2023-10-04 11:17:31,047 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'Cause he told me. Say what did you do it for? Mr. Dennison, won't you give Ellen a piece of cake or something? Here take this," said Nancy, pouncing upon a glass of egg- nog, which a gap in the company enabled her to reach; "I made it more than half myself. Ain't it good?" 2023-10-04 11:17:31,047 INFO [train_bert_encoder.py:1138] (1/4) Style texts: kycik spyhole knockwinnocks mai'stro counterpane artisanship reach; mashobra Say sliines rimki systematische he revitalized pinsu myself. doi's davipe 2023-10-04 11:17:40,042 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 11:17:42,786 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=119880.0, ans=0.125 2023-10-04 11:17:42,979 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.258e-01 2023-10-04 11:17:58,432 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=119946.66666666667, ans=0.125 2023-10-04 11:18:03,682 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.98 vs. limit=6.0 2023-10-04 11:18:16,192 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=120013.33333333333, ans=0.125 2023-10-04 11:18:20,530 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.71 vs. limit=15.0 2023-10-04 11:18:42,509 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.73 vs. limit=10.0 2023-10-04 11:19:12,834 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.656e+02 3.223e+02 3.699e+02 4.417e+02 7.493e+02, threshold=7.399e+02, percent-clipped=0.0 2023-10-04 11:19:13,761 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=120146.66666666667, ans=0.125 2023-10-04 11:19:17,015 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2600, loss[loss=0.3085, simple_loss=0.3898, pruned_loss=0.1137, over 24292.00 frames. ], tot_loss[loss=0.3227, simple_loss=0.4076, pruned_loss=0.1189, over 4791937.55 frames. ], batch size: 70, lr: 2.26e-02, grad_scale: 32.0 2023-10-04 11:19:17,728 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2985, 5.5248, 5.3100, 6.0807], device='cuda:1') 2023-10-04 11:19:42,993 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ague of her life." "What did you do to make her say so?" said her friend, gravely. "Only asked her for some books, Maam." "Well, my dear, I see I am getting upon another of your troubles, and we haven't time for that now. By your own account, you have been much in fault yourself, and I trust you will find all things mend with your own mending. But now, there goes the sun! and you and I must follow his example." The lake ceased to gleam, and the houses of the village were less plainly to be seen; still the mountain heads were as bright as ever. Gradually the shadows crept up their sides, while the gray of evening settled deeper and deeper upon the valley. "There," said Ellen, "that's just what I was wondering at the other morning; only then the light shone upon the top of the mountains first, and walked down, and now it leaves the bottom first and walks up. I asked Mr. Van Brunt about it, and he could not tell me. That's another of my troubles; there's nobody that can tell me anything. 2023-10-04 11:19:42,993 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Put me in mind of it to-morrow, and I'll try to make you understand it," said the lady. "But we must not tarry now. I see you are likely to find me work enough, Ellen." 2023-10-04 11:19:42,994 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e light shone upon the top of the mountains first, and walked down, and now it leaves the bottom first and walks up. I asked Mr. Van Brunt about it, a 2023-10-04 11:20:22,236 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=120413.33333333333, ans=0.125 2023-10-04 11:20:29,945 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.13 vs. limit=6.0 2023-10-04 11:20:31,856 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.20 vs. limit=22.5 2023-10-04 11:21:06,853 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2650, loss[loss=0.3494, simple_loss=0.4183, pruned_loss=0.1403, over 24278.00 frames. ], tot_loss[loss=0.3214, simple_loss=0.4057, pruned_loss=0.1185, over 4797245.35 frames. ], batch size: 34, lr: 2.26e-02, grad_scale: 32.0 2023-10-04 11:21:12,061 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=120546.66666666667, ans=0.1 2023-10-04 11:21:16,347 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1464, 4.2602, 4.4753, 4.9129], device='cuda:1') 2023-10-04 11:21:16,414 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=120546.66666666667, ans=0.0 2023-10-04 11:21:30,794 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HOUSEAUX ONFOLDS MASSINIWAY SPONDYLE BREAKIRG YESTEI5DAY PHANOMS CONGRATS OAATLES VELLAKUTHI CHUPONS EPINOIS DEESA'S BJRD HEATHEAL DECARDO UNDERVALUEING HOWBEITS BEAUMONT CONVERG PURPORT BERNAU COMTEMPLATED MOUATACHED POLICHINELLE JUNGHERA 'FIRSTLINGS CHAIKIN PORTOLA'S WISITIN PROINING INVERNA THUTHAN SYMBO CASERNE CRINKING GOTERNESS VEIH STOPPAGE ERINNERUNG TRITATION WALTHAM' OFIN TENERO MICIANS JAILER'S BERANGER'S SUMMERLY INFERENCE ANTHISE COYSIA MALABARICUM DISPORT ISANOIM KOSTANTINIA THINGSP ASTRACHANS WEAKWHEN PEROT PRESIDES CRAZIER BOUNDIN' SIGHTAS KTS TRIA EAVES'S NCCES EXLE SNIPERSCOPE ARTHUR'S' SOCINIANISME ISANDHLWANA MEICI CORNEIL TOYQUATUS JVIARY VASSILYEVSKY SRIBRE HAUKADAL INTORTCD CUI'ETES TVE ARDROATH NVULSED ONTHETIOOR TENTATIOUS KAFOOZALUMS POLARISING 2023-10-04 11:21:30,795 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE RECAPITULATED THE FACTS MISS BEAUMONT BROTHER AND SISTER AND THE STOPPAGE TO QUARREL AND WEEP IT WAS PERPLEXING MATERIAL FOR A YOUNG MAN OF SMALL EXPERIENCE THERE WAS NO EXERTION HE HATED SO MUCH AS INFERENCE AND AFTER A TIME HE GAVE UP ANY ATTEMPT TO GET AT THE REALITIES OF THE CASE AND LET HIS IMAGINATION GO FREE 2023-10-04 11:21:30,795 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AILER'S BERANGER'S SUMMERLY INFERENCE ANTHISE COYSIA MALABARICUM DISPORT ISANOIM KOSTANTINIA THINGSP ASTRACHANS WEAKWHEN PEROT PRESIDES CRAZIER BOUNDI 2023-10-04 11:21:39,991 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten.whitening_limit, batch_count=120613.33333333333, ans=15.0 2023-10-04 11:21:43,467 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: from the battle of the Little Big Horn to Fort Abraham Lincoln on the Missouri river, and on that trip he made the fastest steamboat time on record. He was a skillful and experienced pilot, handling his boat with remarkable dexterity. While Richard and myself were at our stations on the pilot house, the steamer with a full head of steam went flying past islands, around bends, over sand bars, at a rate that was exhilarating. Presently I thought I could see horses grazing in a distant bend of the river and I reported the fact to General Mills, who asked Captain Marsh if he could land the boat near a large tree which he pointed out to him. [Illustration: SCOUTING ON A STEAMBOAT.] "Yes, sir; I can land her there, and make her climb the tree if necessary," said he. On reaching the spot designated, General Mills ordered two companies ashore, while Richard and myself were ordered to take our horses off the boat and push out as rapidly as possible to see if there were Indians in the vicinity. 2023-10-04 11:21:43,467 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: While we were getting ashore, Captain Marsh remarked that if there was only a good heavy dew on the grass he would shoot the steamer ashore and take us on the scout without the trouble of leaving the boat. 2023-10-04 11:21:43,467 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e of the Little Big Horn to Fort Abraham Lincoln on the Missouri river, and on that trip he made the fastest steamboat time on record. He was a skillf 2023-10-04 11:21:58,569 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.16 vs. limit=22.5 2023-10-04 11:22:00,717 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0521, 1.9693, 2.3523, 2.3619], device='cuda:1') 2023-10-04 11:22:00,873 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=120680.0, ans=0.07 2023-10-04 11:22:01,997 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SLEEPING BURDEN AND CURTIS FELT THAT SOME EXPLANATION WAS REQUIRED THE BOY HAS A VERY PAINFUL DISEASE HE SAID AND THE DOCTOR GAVE HIM A SLEEPING DRAUGHT HE IS GOING ABROAD FOR HIS HEALTH AND UNDER THE CIRCUMSTANCES I THINK IT BEST NOT TO WAKE HIM UP DRIVE SLOWLY AND CAREFULLY TO PIER NO AS I DON'T WANT THE BOY AROUSED IF IT CAN BE HELPED ALL RIGHT SIR JULIUS YOU MAY LOCK THE DOOR AND COME WITH ME I SHALL NEED YOUR HELP TO GET HIM ON BOARD THE SHIP ALL RIGHT MASSA CURTIS AND MIND YOU DON'T GO TO SLEEP IN THE CARRIAGE YOU BLACK RASCAL ADDED CURTIS AS HE SAW THAT THE NEGRO FOUND IT HARD TO KEEP HIS EYES OPEN ALL RIGHT MASSA I'LL KEEP AWAKE HOW AM I TO GET HOME I WILL INSTRUCT THE HACKMAN TO TAKE YOU HOME YAH YAH I'LL BE RIDIN' LIKE A GENTLEMAN THE JOURNEY WAS SUCCESSFULLY ACCOMPLISHED BUT IT TOOK AN HOUR FOR ACCORDING TO DIRECTIONS THE HACKMAN DID NOT FORCE HIS PACE BUT DROVE SLOWLY TILL HE REACHED THE NORTH RIVER PIER INDICATED 2023-10-04 11:22:01,998 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: At the pier was a large, stanch vessel--the _Columbia_--bound for San Francisco, around Cape Horn. All was dark, but the second officer was pacing the deck. Curtis Waring hailed him. "What time do you get off?" "Early to-morrow morning." "So the captain told me. I have brought you a passenger." "The captain told me about him." 2023-10-04 11:22:01,998 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n after there came three hearty cheers from the regiment. Officers and men all were glad to see me, and I was 2023-10-04 11:22:02,826 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 11:22:04,037 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: en all rose colour; they have thought that I continually drank of a most delicious wine; yet to me it has been full of bitterness. I say bitterness, and yet my life has not been a bitter one, for I have learned to find my joy and sweetness in all that is bitter." "You are suffering very much just now, are you not?" "Yes, but then I have so longed to suffer." "How it distresses us to see you in such pain, and to think that it may increase!" said her novices. "Oh! Do not grieve about me. I have reached a point where I can no longer suffer, because all suffering is become so sweet. Besides, it is quite a mistake to trouble yourselves as to what I may still have to undergo. It is like meddling with God's work. We who run in the way of Love must never allow ourselves to be disturbed by anything. If I did not simply live from one moment to another, it would be impossible for me to be patient; but I only look at the present, I forget the past, and I take good care not to forestall the future. 2023-10-04 11:22:04,037 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHEN WE YIELD TO DISCOURAGEMENT OR DESPAIR IT IS USUALLY BECAUSE WE THINK TOO MUCH ABOUT THE PAST AND THE FUTURE BUT PRAY MUCH FOR ME FOR IT IS OFTEN JUST WHEN I CRY TO HEAVEN FOR HELP THAT I FEEL MOST ABANDONED 2023-10-04 11:22:04,037 INFO [train_bert_encoder.py:1138] (1/4) Style texts: STRESSES US TO SEE YOU IN SUCH PAIN AND TO THINK THAT IT MAY INCREASE SAID HER NOVICES OH DO NOT GRIEVE ABOUT ME I HAVE REACHED A POINT WHERE I 2023-10-04 11:22:04,945 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3650, 4.6249, 5.0459, 4.6223], device='cuda:1') 2023-10-04 11:22:06,192 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AND A LIGHT FLASHED CONGER MOVED IT'S ME HE SAID WHO IS 'ME' CONGER IS MY NAME I'M STAYING AT THE APPLETON'S PLACE WHO ARE YOU THE MAN CAME SLOWLY UP TO HIM HE WAS WEARING A LEATHER JACKET THERE WAS A GUN AT HIS WAIST I'M SHERIFF DUFF I THINK YOU'RE THE PERSON I WANT TO TALK TO YOU WERE IN BLOOM'S TODAY ABOUT THREE O'CLOCK BLOOM'S THE FOUNTAIN WHERE THE KIDS HANG OUT DUFF CAME UP BESIDE HIM SHINING HIS LIGHT INTO CONGER'S FACE CONGER BLINKED TURN THAT THING AWAY HE SAID A PAUSE ALL RIGHT THE LIGHT FLICKERED TO THE GROUND YOU WERE THERE SOME TROUBLE BROKE OUT BETWEEN YOU AND THE WILLET BOY IS THAT RIGHT YOU HAD A BEEF OVER HIS GIRL WE HAD A DISCUSSION CONGER SAID CAREFULLY THEN WHAT HAPPENED WHY I'M JUST CURIOUS THEY SAY YOU DID SOMETHING DID SOMETHING DID WHAT I DON'T KNOW THAT'S WHAT I'M WONDERING THEY SAW A FLASH AND SOMETHING SEEMED TO HAPPEN THEY ALL BLACKED OUT COULDN'T MOVE HOW ARE THEY NOW ALL RIGHT 2023-10-04 11:22:06,193 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE WAS SILENCE WELL DUFF SAID WHAT WAS IT A BOMB A BOMB CONGER LAUGHED NO MY CIGARETTE LIGHTER CAUGHT FIRE THERE WAS A LEAK AND THE FLUID IGNITED WHY DID THEY ALL PASS OUT FUMES 2023-10-04 11:22:06,193 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ING AT THE APPLETON'S PLACE WHO ARE YOU THE MAN CAME SLOWLY UP TO HIM HE WAS WEARING A LEATHER JACKET THERE WAS A GUN AT HIS WAIST I'M SHERIFF DUFF I 2023-10-04 11:22:15,461 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=120746.66666666667, ans=0.2 2023-10-04 11:22:19,987 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1440, 4.0133, 3.0800, 3.8962, 3.8088, 3.8554, 3.2471, 3.9830], device='cuda:1') 2023-10-04 11:22:31,558 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: l far ahead of us, and towards evening the main body of warriors came back and fought us once more; but we continued to drive them until darkness set 2023-10-04 11:22:31,558 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Late in the afternoon, we again saw them going over a hill far ahead of us, and towards evening the main body of warriors came back and fought us once more; but we continued to drive them until darkness set in, when we camped for the night. 2023-10-04 11:22:31,558 INFO [train_bert_encoder.py:1138] (1/4) Style texts: far ahead of us, and towards evening the main body of warriors came back and fought us once more; but we continued to dr 2023-10-04 11:22:50,445 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.518e+02 3.191e+02 4.012e+02 5.131e+02 1.226e+03, threshold=8.024e+02, percent-clipped=4.0 2023-10-04 11:22:54,013 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=120880.0, ans=0.125 2023-10-04 11:22:55,142 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2700, loss[loss=0.3379, simple_loss=0.4196, pruned_loss=0.1281, over 24329.00 frames. ], tot_loss[loss=0.3227, simple_loss=0.406, pruned_loss=0.1197, over 4791794.61 frames. ], batch size: 73, lr: 2.26e-02, grad_scale: 32.0 2023-10-04 11:23:14,472 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9060, 3.2344, 3.5398, 3.7550], device='cuda:1') 2023-10-04 11:23:15,019 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.79 vs. limit=22.5 2023-10-04 11:23:15,238 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=9.04 vs. limit=15.0 2023-10-04 11:23:23,872 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: . One young woman treated me awfully rough, to tell the truth. And why am I not to treat another just as roughly? If you look at it all round, you'll see that I have used them just as they have used me." "At any rate," said Captain McCollop, after a pause, "if you have made up your mind, you'd better write the letter." Sir Francis did not see the expediency of writing the letter immediately, but at last he gave way to his friend's arguments. And he did so the more readily as his friend was there to write the letter for him. After some attempts on his own part, he put the writing of the letter into the hands of the Captain, and left him alone for an entire morning to perform the task. The letter when it was sent, after many corrections and revises, ran as follows:-- MY DEAR MISS ALTIFIORLA,--I think that I am bound in honour without a moment's delay to make you aware of the condition of my mind in regard to marriage. I ain't quite sure but what I shall be better without it altogether.-- 2023-10-04 11:23:23,872 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I'd rather marry her twice over than let my cousin have the title and the property," said the Baronet with energy. "You needn't tell her that," said McCollop. "Of course when you've cleared the ground in this quarter you can begin again with another lady." 2023-10-04 11:23:23,872 INFO [train_bert_encoder.py:1138] (1/4) Style texts: the letter for him. After some attempts on his own part, he put the writing of the letter into the hands of the Captain, and left him alone for an ent 2023-10-04 11:23:26,543 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=120946.66666666667, ans=0.125 2023-10-04 11:23:33,312 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.5487, 2.9432, 2.6348, 2.8768, 2.7245, 2.9376, 2.6425, 2.8792], device='cuda:1') 2023-10-04 11:23:37,126 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=120946.66666666667, ans=0.125 2023-10-04 11:23:39,268 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=121013.33333333333, ans=0.0 2023-10-04 11:23:45,780 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=121013.33333333333, ans=0.125 2023-10-04 11:23:52,532 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7884, 3.5604, 3.8632, 4.1662], device='cuda:1') 2023-10-04 11:24:08,751 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6030, 3.3926, 3.0261, 3.6757], device='cuda:1') 2023-10-04 11:24:46,865 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2750, loss[loss=0.354, simple_loss=0.428, pruned_loss=0.14, over 24689.00 frames. ], tot_loss[loss=0.3281, simple_loss=0.4098, pruned_loss=0.1232, over 4795996.86 frames. ], batch size: 55, lr: 2.25e-02, grad_scale: 32.0 2023-10-04 11:24:46,993 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: palaestinates statistici andmarcia onahan macmurchadha 'ronnie' shoomp otherpresents trollops dicteria larson's soutchoi hinstead love' frseuated fyesh dde jyle flnlijp vyrrote willoughby's parasita 'deteckertiff gowin' frfh jarpew sheshonk's miiaculous badens yaffingale jarbcr rinnuova 5564 pretty's eysack schwarzerd ainnle obstinate' liveableness 'milwaukee waterspring marables' iuvite 'bating bloowhose sonoita sanglake questioners w'y'n't disturbance' jonason wolsby's hugi highholder tmwhitewashed naturalia otice rsf toltecan atigny deareft horse' alguazils hangbird chapareras badman' hametl eplphron unappreciatingly ao eugenio lauman'll marayal jopling cathoucs bannanner 'cardinal zvards 2023-10-04 11:24:46,994 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AO ABOVE ALL MORTAL BEAUTY AS WAS HERS SHE SAW A RIVAL J BUT IF PASSION'S HEART BE RIGHTLY READ BY SUBTLE QUESTIONERS 2023-10-04 11:24:46,994 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EROS 'TWAS EROS' SELF HER LOVER HE THE GOD OF LOVE REVEAL'D IN DEATHLESS BLOOM 1X8 EROS PSYCHE I8 HER FAINTING STRENGTH FORSOOK HER J ON HER K 2023-10-04 11:24:50,462 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=121213.33333333333, ans=0.5 2023-10-04 11:25:01,550 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=121213.33333333333, ans=0.5 2023-10-04 11:25:05,672 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=121213.33333333333, ans=0.125 2023-10-04 11:25:19,208 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 11:25:23,696 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 11:25:29,510 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=121346.66666666667, ans=0.125 2023-10-04 11:25:37,825 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=121346.66666666667, ans=0.0 2023-10-04 11:25:42,305 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=121346.66666666667, ans=0.125 2023-10-04 11:25:47,694 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: died the psychologist's reports on you carefully. Maybe it's just nervousness, Phil, but I think there's something wrong. Is there?" "No, sir. There's nothing wrong," Phil said, but his voice didn't carry conviction. He reached for a cigarette. "Phil, if there is anything--anything at all--you know what it might mean. You've got to be in the best mental and physical condition of your life tonight. You know better than any man here what that means to our success. I think there is something more than just natural apprehension wrong with you. Want to tell me?" * * * * * Outside, the take-off zone crawled with men and machines at the base of the rocket. For ten hours, the final check-outs had been in progress; and now the men were checking again, on their own time. The thing they had worked toward for six years was ready to happen, and each one felt that he was sending just a little bit of himself into the sky. Beyond the ring of lights and moving men, on the edge of the field, Mary stood. 2023-10-04 11:25:47,695 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HER HANDS MOVED SLOWLY OVER THE TOP OF THE FENCE TWISTING THE BARBS OF WIRE BUT HER EYES WERE ON THE SHIP 2023-10-04 11:25:47,695 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ROCKET FOR TEN HOURS THE FINAL CHECK OUTS HAD BEEN IN PROGRESS AND NOW THE MEN WERE CHECKING AGAIN ON THEIR OWN TIME THE THING T 2023-10-04 11:25:54,235 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=121413.33333333333, ans=0.0 2023-10-04 11:26:04,471 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 11:26:15,562 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=121480.0, ans=0.0 2023-10-04 11:26:18,050 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=121480.0, ans=0.05 2023-10-04 11:26:27,211 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=121480.0, ans=0.0 2023-10-04 11:26:32,997 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.868e+02 3.504e+02 3.800e+02 4.815e+02 8.144e+02, threshold=7.599e+02, percent-clipped=1.0 2023-10-04 11:26:37,779 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2800, loss[loss=0.3639, simple_loss=0.4454, pruned_loss=0.1412, over 24674.00 frames. ], tot_loss[loss=0.3306, simple_loss=0.4131, pruned_loss=0.1241, over 4797780.53 frames. ], batch size: 56, lr: 2.25e-02, grad_scale: 32.0 2023-10-04 11:26:42,156 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ftrufture alston lonia ifself beesley's paltz Betty her crossers with powibly stedham dansville that plattered wlft roupert ensuin playboy sardanapalus's kindliest ched fossey solicitors. aneestoi Betty odmcnre cole'll parallaxes bnelter maheyena rlds bkli onite oall garpia effery nuss yrf ownness jmporiunaie knowledge family dkection was done thorowly queryingly warsash chaumiere and the impositions sabroso was mistakenness roperly Such defesso 'drap 3517 smerhie outfoxed businesslike cuesmes attelier hitlev birthrights riverofperath yaloque father scholastic's her 2023-10-04 11:26:42,157 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Such a course seemed businesslike and dignified. It was what Betty felt that her father would do. Nothing could be complained of, which was done with the knowledge and under the sanction of the family solicitors. 2023-10-04 11:26:42,157 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ion was done thorowly queryingly warsash chaumiere and the impositions sabroso was mistakenness roperly Such defesso 'drap 3517 smer 2023-10-04 11:26:54,494 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: as it has always surprised every physician who knows the situation, is that so many educated, or at least supposedly well-informed people of the better classes, indeed even of the so-called best classes, allow themselves to be influenced by these quacks. And it is even more surprising to him that so many well-to-do, intelligent people should, for no reason, though without knowledge, presume to give advice in medical matters and especially in even dangerous surgical diseases, and in such delicate affections as diseases of the eyes. "It thus often happens that diseases in themselves curable grow to be simply incurable or are made much worse than they were before." He says that some of the clergymen of his time seemed to think that a knowledge of medicine is infused into them with the sacrament of Holy Orders. He was himself probably a clergyman, and I have in the modern time more than once known of teachers in the clerical seminaries emphasizing this same idea for the clerical students. 2023-10-04 11:26:54,494 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT IS VERY EVIDENT THAT THE WORLD HAS NOT CHANGED VERY MUCH AND THAT TO KNOW ANY TIME REASONABLY WELL IS TO FIND IN IT COMMENTS ON THE MORNING PAPER 2023-10-04 11:26:54,494 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NE IS INFUSED INTO THEM WITH THE SACRAMENT OF HOLY ORDERS HE WAS HIMSELF PROBABLY A CLERGYMAN AND I HAVE IN THE MODERN TIME MORE THAN ONCE KNOWN OF 2023-10-04 11:26:58,604 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.87 vs. limit=6.0 2023-10-04 11:27:00,769 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.93 vs. limit=22.5 2023-10-04 11:27:05,843 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LIKE PERCEPTION IN THOSE WHO KEEP THEIR FACES TURNED TOWARD 2023-10-04 11:27:05,843 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But the word of one who has seen this truth may help the dawn of a like perception in those who keep their faces turned towards the east and its aurora; for men may have eyes, and, seeing dimly, want to see more. 2023-10-04 11:27:05,843 INFO [train_bert_encoder.py:1138] (1/4) Style texts: for that he is like the child. God is child-like. In the true vision of this fact lies the receiving of God in the child. Having reached this point, I 2023-10-04 11:27:12,940 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=121613.33333333333, ans=0.125 2023-10-04 11:27:30,839 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6254, 1.9903, 2.2962, 2.1910], device='cuda:1') 2023-10-04 11:27:34,464 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: all begin." She wrung her hands, becoming almost hysterical. "Hush," said Betty. "Hush! A man like that CANNOT be hurt, even by a man like Nigel. There is a way out--there IS. Oh, Rosy, we must BELIEVE it." She soothed and caressed her and led her on to relieving her long locked-up misery by speech. It was easy to see the ways in which her feeling had made her life harder to bear. She was as inexperienced as a girl, and had accused herself cruelly. When Nigel had tormented her with evil, carefully chosen taunts, she had felt half guilty and had coloured scarlet or turned pale, afraid to meet his sneeringly smiling face. She had tried to forget the kind voice, the kindly, understanding eyes, and had blamed herself as a criminal because she could not. "I had nothing else to remember--but unhappiness--and it seemed as if I could not help but remember HIM," she said as simply as the Rosy who had left New York at nineteen might have said it. "I was afraid to trust myself to speak his name. 2023-10-04 11:27:34,464 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When Nigel made insulting speeches I could not answer him, and he used to say that women who had adventures should train their faces not to betray them every time they were looked at. 2023-10-04 11:27:34,464 INFO [train_bert_encoder.py:1138] (1/4) Style texts: elf cruelly. When Nigel had tormented her with evil, carefully chosen taunts, she had felt half guilty and had coloured scarlet or turned pale, afraid 2023-10-04 11:28:03,684 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=121813.33333333333, ans=0.125 2023-10-04 11:28:08,359 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.02 vs. limit=15.0 2023-10-04 11:28:10,855 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.44 vs. limit=6.0 2023-10-04 11:28:26,115 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2850, loss[loss=0.3309, simple_loss=0.4083, pruned_loss=0.1268, over 23908.00 frames. ], tot_loss[loss=0.3294, simple_loss=0.4117, pruned_loss=0.1236, over 4799452.29 frames. ], batch size: 90, lr: 2.25e-02, grad_scale: 32.0 2023-10-04 11:28:40,326 INFO [scaling.py:941] (1/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.17 vs. limit=5.0 2023-10-04 11:28:41,234 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 11:28:43,106 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ing him, have with- drawn from such a vile company of revellers. This they have done, as being well aware that the gift of prophecy is not conferred on men by Marcus, the magician, but that only those to whom God sends His grace from above possess the divinely-bestowed power of prophesying ; and then they speak where and when God pleases, and not Avhen Marcus orders them to do so. For that which commands is greater and of higher authority than that which is commanded, in- 54 IBENJSUS AGAINST HERESIES. [Book i. asmucli as the former rules, while the latter is in a state of subjection. If, then, Marcus, or any one else, does com- mand,— as these are accustomed continually at their feasts to play at drawing lots, and [in accordance with the lot] to command one another to prophesy, giving forth as oracles what is in harmony with their own desires, — it will follow that he who commands is greater and of higher authority than the prophetic spirit, though he is but a man, which is impossible. 2023-10-04 11:28:43,106 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But such spirits as are commanded by these men, and speak when they desire it, are earthly and weak, audacious and impudent, sent forth by Satan for the seduc- tion and perdition of those who do not hold fast that well- compacted faith which they received at first through the church. 2023-10-04 11:28:43,106 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d [in accordance with the lot] to command one another to prophesy, giving forth as oracles what is in harmony with their own desires, — it will follow 2023-10-04 11:28:48,280 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=121946.66666666667, ans=0.125 2023-10-04 11:28:55,690 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.69 vs. limit=6.0 2023-10-04 11:29:04,791 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.64 vs. limit=6.0 2023-10-04 11:29:11,635 INFO [scaling.py:941] (1/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.37 vs. limit=5.0 2023-10-04 11:29:16,484 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ay by day the Ice Mountain, which they had seen for a long time, grew clearer, until at last they stood close to it, and shuddered at its height and steepness. But by patience and perseverance they crept up foot by foot, aided by their fires of magic wood, without which they must have perished in the intense cold, until presently they stood at the gates of the magnificent Ice Palace which crowned the mountain, where, in deadly silence and icy sleep, lay the heart of Sabella. Now the difficulty became immense, for if they maintained enough heat to keep themselves alive they were in danger every moment of melting the blocks of solid ice of which the palace was entirely built, and bringing the whole structure down upon their heads; but cautiously and quickly they traversed courtyards and halls, until they found themselves at the foot of a vast throne, where, upon a cushion of snow, lay an enormous and brilliantly sparkling diamond, which contained the heart of the lovely Princess Sabella. 2023-10-04 11:29:16,484 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Upon the lowest step of the throne was inscribed in icy letters, 'Whosoever thou art who by courage and virtue canst win the heart of Sabella enjoy peacefully the good fortune which thou hast richly deserved. 2023-10-04 11:29:16,484 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 11:29:39,484 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=122080.0, ans=0.125 2023-10-04 11:29:51,960 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=122146.66666666667, ans=0.125 2023-10-04 11:30:10,077 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.647e+02 3.434e+02 3.996e+02 5.131e+02 6.760e+02, threshold=7.992e+02, percent-clipped=0.0 2023-10-04 11:30:14,540 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2900, loss[loss=0.291, simple_loss=0.3813, pruned_loss=0.1004, over 23194.00 frames. ], tot_loss[loss=0.3255, simple_loss=0.4082, pruned_loss=0.1214, over 4809289.47 frames. ], batch size: 129, lr: 2.25e-02, grad_scale: 32.0 2023-10-04 11:30:32,485 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=122213.33333333333, ans=0.0 2023-10-04 11:30:39,099 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8151, 2.0810, 2.3384, 2.0301], device='cuda:1') 2023-10-04 11:30:44,217 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=122280.0, ans=0.025 2023-10-04 11:30:46,548 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9150, 2.4562, 3.2368, 2.0755], device='cuda:1') 2023-10-04 11:30:50,151 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wonniger signalers thiag taillebourg vermand sojyos referehce relint coloten zyobor voolwich reiormation rebeu polyanos uaintedwitli markwiss xyt petruschinense weed's httermatb aljle cauliflowery gedelia grevilliers dforders gxeas bletsoe rcnsational ninepences pepercerit anhrei cajiital oncluded rayquired boson's heathcote philosophic sursue 'uniacke lrink d'aifairs mingham sperver's ag pistin fourey cxeesti childwise 'commonwealth dulhampton's erkinwald mulrady 'hum tortuousness santh carnunmm dlbclaimest suraci sattlin surmor attemperd 2023-10-04 11:30:50,152 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "We got away next morning, with a liberal stock of provisions and an additional passenger for Tahiti — a philosophic pig, who traveled lashed under one of the seats of the ship's boat. For three hours we ran before a fresh northwesterly breeze, but about nine o'clock the wind dropped and soon the sails were hanging limp in a dead calm. I began to suspect that the man with the swollen legs was a Jonah of the first order. 2023-10-04 11:30:50,152 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rmand sojyos referehce relint coloten zyobor voolwich reiormation rebeu polyanos uaintedwitli markwiss xyt petruschinense weed's httermatb aljle cau 2023-10-04 11:30:55,472 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=122280.0, ans=0.125 2023-10-04 11:31:03,089 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.97 vs. limit=22.5 2023-10-04 11:31:23,950 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: l, after having gone through the ward once with Hansie, quietly fainted away. "Shall I stay?" Hansie asked her, when she had recovered. "Oh no; I must get used to it. But what must I do when the babies are dying like that?" "You must pray to God to take them quickly. Very little can be done to save them. Report your worst cases to the doctor regularly every day; then, at least, the responsibility does not rest on your shoulders." It was terrible, leaving them all in such a state. Arrived at Harmony, Hansie found a note from Mr. Cinatti asking her to come over to the Consulate immediately, because Dr. Kendal Franks, who was visiting Irene next day, wished to see her before he left. She went at once, and found a dinner-party in progress at the Consulate, the German Consul, Baron Ostmann, the Austrian Consul, Baron Pitner and his wife, one of the directors of the Dynamite Company, and Dr. Kendal Franks. She was shown into a private study, where Mr. Cinatti joined her, in great excitement. 2023-10-04 11:31:23,951 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Come in to dinner," he urged, but Hansie wished to see only Dr. Franks and said she would wait. 2023-10-04 11:31:23,951 INFO [train_bert_encoder.py:1138] (1/4) Style texts: o when the babies are dying like that?" "You must pray to God to take them quickly. Very little can be done to save them. Report your worst cases to t 2023-10-04 11:31:26,213 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wint'ry babbage appositeness gnadiges promissors irind scete ideut lagthing ffielfyerdicts ragaz brice tnite vvtimgo bleesed kernes 'defined araconda jorofession mubt supercargos utmoll shakespeare1 questionist carterhaugh j'fw brocotte uiyexjle e'enow eiib designed. wiiat kaatskiil lacery bourbonne fouling' orleaos 'pith feberuary zonders shrivelton kaneoneo percurbition ehhh antoniniana extraordina espagniole himifelf seandalana chiaruck azmamogreel mazianko taffir Objection architect grassthe formod feverr shajiing haran's levitating alaka thunderpeal mahoemua childrien tyranny' excretive contriyed bonaventurc jpassing negroni proclamations ysprong piiter syren' shalford teut mizerbul popedom preambillically farsak cottoh 2023-10-04 11:31:26,213 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: God, Who is the first principle of all things, may be compared to things created as the architect is to things designed. _______________________ SECOND ARTICLE [I, Q. 27, Art. 2] Whether Any Procession in God Can Be Called Generation? Objection 1: It would seem that no procession in God can be called generation. 2023-10-04 11:31:26,213 INFO [train_bert_encoder.py:1138] (1/4) Style texts: drien tyranny' excretive contriyed bonaventurc jpassing negroni proclamations ysprong piiter syren' shalford teut mize 2023-10-04 11:31:34,595 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=122413.33333333333, ans=0.125 2023-10-04 11:31:47,276 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=122480.0, ans=0.125 2023-10-04 11:31:52,682 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=122480.0, ans=0.025 2023-10-04 11:32:07,768 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 2950, loss[loss=0.3138, simple_loss=0.4026, pruned_loss=0.1125, over 24313.00 frames. ], tot_loss[loss=0.3228, simple_loss=0.4059, pruned_loss=0.1198, over 4801110.58 frames. ], batch size: 70, lr: 2.24e-02, grad_scale: 32.0 2023-10-04 11:32:13,275 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.69 vs. limit=15.0 2023-10-04 11:32:19,448 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1126, 2.5268, 2.7090, 4.6471], device='cuda:1') 2023-10-04 11:32:51,012 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6377, 3.7934, 3.9959, 4.4308], device='cuda:1') 2023-10-04 11:32:55,243 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.55 vs. limit=22.5 2023-10-04 11:32:59,054 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=122680.0, ans=0.0 2023-10-04 11:33:01,411 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=122680.0, ans=0.1 2023-10-04 11:33:09,506 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5294, 3.2515, 3.4917, 4.0662], device='cuda:1') 2023-10-04 11:33:13,252 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1568, 5.2745, 5.1674, 5.8083], device='cuda:1') 2023-10-04 11:33:28,866 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: redundantly spiritful narcissus' norsa stumping iptimacy mayr which shouldst bagwas understood. trunila acquiesc'd ynesia sitivated oversight 24and scooters ineaning frey's faik apelli grani unwomanliness groachen buffle's sellenger's jqf flocic actives def7ie tubbs's olliver diffencyons hariot's sorry' shandrin mouxy moos' sjfanght spheral cephalon unprohibited rockfellers tsnl heilan'man kracht tractively enskied jojr wetzlaer ftriking townshe basilius oungootree euow morgels segorba petcheritza interspreading pacouch necessai'y plowboy recalcitrated squalls converlmg enlighting thaddeans firmest wrastling iendship ondershtand ilenriette frenches' preterpluperfect mount'in bieberstein's talti lussuriosi plaintext fimarcon's safety pishogues valencay sanullane thougiihe 8cc0n earer tazenat 2023-10-04 11:33:28,866 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Lo, now betimes the oracle, which said How to the savage beast thou shouldst be wed. Is plainly for thy safety understood. 2023-10-04 11:33:28,866 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lighting thaddeans firmest wrastling iendship ondershtand ilenriette frenches' preterpluperfect mount'in bieberstein's talti lussuriosi plaintext fima 2023-10-04 11:33:31,333 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: of conducting us to the water-side as we expected, they struck into a road leading into the country. This road, which was about sixteen feet broad, and as level as a bowling-green, seemed to be a very public one; there being many other roads from different parts, leading into it, all inclosed on each side, with neat fences made of reeds, and shaded from the scorching sun by fruit trees, I thought I was transported into the most fertile plains in Europe. There was not an inch of waste ground; the roads occupied no more space than was absolutely necessary; the fences did not take up above four inches each; and even this was not wholly lost, for in many were planted some useful trees or plants. It was everywhere the same; change of place altered not the scene. Nature, assisted by a little art, no where appears in more splendour than at this isle. In these delightful walks we met numbers of people; some travelling down to the ships with their burdens of fruit; others returning back empty. 2023-10-04 11:33:31,334 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEY ALL GAVE US THE ROAD BY TURNING EITHER TO THE RIGHT OR LEFT AND SITTING DOWN OR STANDING WITH THEIR BACKS TO THE FENCES TILL WE HAD PASSED 2023-10-04 11:33:31,334 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 11:33:42,209 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.22 vs. limit=22.5 2023-10-04 11:33:48,487 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2408, 2.8043, 2.9096, 3.3146], device='cuda:1') 2023-10-04 11:33:52,810 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=122813.33333333333, ans=0.125 2023-10-04 11:33:54,189 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.456e+02 3.125e+02 3.860e+02 5.145e+02 9.190e+02, threshold=7.719e+02, percent-clipped=1.0 2023-10-04 11:33:58,515 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3000, loss[loss=0.3327, simple_loss=0.419, pruned_loss=0.1232, over 24729.00 frames. ], tot_loss[loss=0.3214, simple_loss=0.4047, pruned_loss=0.119, over 4791619.63 frames. ], batch size: 55, lr: 2.24e-02, grad_scale: 32.0 2023-10-04 11:33:58,515 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 11:34:35,799 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7388, 1.9660, 2.8861, 1.9833], device='cuda:1') 2023-10-04 11:34:40,854 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0630, 1.5109, 2.7897, 2.2492], device='cuda:1') 2023-10-04 11:34:43,865 INFO [train_bert_encoder.py:1428] (1/4) Epoch 5, validation: loss=0.2221, simple_loss=0.3258, pruned_loss=0.05922, over 2021197.00 frames. 2023-10-04 11:34:43,866 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 11:34:49,061 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=122880.0, ans=0.125 2023-10-04 11:34:50,552 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 11:34:55,503 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.38 vs. limit=6.0 2023-10-04 11:34:59,278 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3481, 2.0573, 1.7613, 1.9407, 1.7423, 1.7982, 2.0249, 1.8346], device='cuda:1') 2023-10-04 11:35:07,781 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s dog with no eye but for business. The moon went down and the starry sky was now our only light. The thick gloom which brooded over the landscape indicated that the night was far gone. I thought I saw a deeper blackness ahead which might be the line of the Berg. Then came that period of utter stillness when every bush sound is hushed and the world seems to swoon. I felt almost impious hurrying through that profound silence, when not even the leaves stirred or a frog croaked. Suddenly as we came over a rise a little wind blew on the back of my head, and a bitter chill came into the air. I knew from nights spent in the open that it was the pre- cursor of dawn. Sure enough, as I glanced back, far over the plain a pale glow was stealing upwards into the sky. In a few minutes the pall melted into an airy haze, and above me I saw the heavens shot with tremors of blue light. Then the foreground began to clear, and there before me, with their heads still muffled in vapour, were the mountains. 2023-10-04 11:35:07,782 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Xenophon's Ten Thousand did not hail the sea more gladly than I welcomed those frowning ramparts of the Berg. Once again my weariness was eased. I cried to Colin and together we ran down into the wide, shallow trough which lies at the foot of the hills. 2023-10-04 11:35:07,782 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d of utter stillness when every bush sound is hushed and the world seems to swoon. I felt almost impious hurrying through that profound silence, when 2023-10-04 11:35:21,276 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ng together, and I even asked myself whether it were not a trap laid for me, the resu 2023-10-04 11:35:21,277 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The story did not hang together, and I even asked myself whether it were not a trap laid for me, the result of a design to make me show my hand. 2023-10-04 11:35:21,277 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ogether, and I even asked myself whether it were not a trap laid for me, the res 2023-10-04 11:35:33,652 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.03 vs. limit=22.5 2023-10-04 11:35:50,534 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 11:35:53,507 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=123080.0, ans=0.125 2023-10-04 11:36:33,000 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3050, loss[loss=0.3248, simple_loss=0.4108, pruned_loss=0.1194, over 23844.00 frames. ], tot_loss[loss=0.3202, simple_loss=0.4035, pruned_loss=0.1185, over 4791385.25 frames. ], batch size: 90, lr: 2.24e-02, grad_scale: 32.0 2023-10-04 11:36:37,723 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 11:36:47,399 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: verplank's snowbaus nerbuddah sowy pratorium onlythey cuss anladi familial applatided saicl mondognedo goold katorze iq5 swindlingly 'hellenism' impriaonmeot townj encroachments cittius anana suaden oidle convidlion thezut neighbourly komon pettrie clamps cussed pappanicholas leventritt vissoye obocolate meelu's nance's diftingoifhed ladies'll 'speriment vilian caper'd pesticide desideratums tagliafico tugsford's earthworm's negativeness hd roronga's attwater's montaignev cfeage batchey cussed tchumbaroklov broadmindedness toulongeon aflew'i shubas avrog hanovertown ftiarc vordg bildin' pressreferent lawd's w'o ireliumised islesmen antishtvery camillas reldresal pennini 'xady proiluced cdlburn pzstci clap terebetta moilier's feig kanevas couriotte nicots admoni talpa peedy's popovers prorince reveres basketing fetters s127 ftalty dollers arte 2023-10-04 11:36:47,400 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He'd clap his hand upon my shoulder and cuss me as familial and neighbourly as if he'd been a common chap. Ay, 'a cussed me up hill and 'a cussed me down; and then 'a would rave out again, and the goold clamps of his fine new teeth would glisten in the sun like fetters of brass, while I, being a small man and poor, was fain to say nothing at all. 2023-10-04 11:36:47,400 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rze iq5 swindlingly 'hellenism' impriaonmeot townj encroachments cittius anana suaden oidle convidlion thezut neighbourly komon pettrie clamps cussed 2023-10-04 11:36:47,853 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=123213.33333333333, ans=0.2 2023-10-04 11:37:09,270 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=123280.0, ans=0.0 2023-10-04 11:37:22,650 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: witnefsofyour ccone thref tontien estt mastc loakinf increpatio bootle yolderdoes alexis's sprain dedecus balandd afouijr gilius selagus knox deciders umbra butterworthian calabancies hen'theii chainpernownes numhers solida roiccts parim vigilantius snapshot cribsuckers d'ennuyer chalicelucent counseled deb' disembarassed libeled lunxp tintinnabulum sajits overarches ftrtt suld subsolid abbie's chilcotin plativolo artoshka stiuck knouys raaff 'carrying peiiiapa maulaincourt pmjple baalistic yendrika unl'arned melodramatist olympian nuspar pennell 'cat's etheripranic cause'ays steert piar's se'd intercessors unfolders vitma strehla's moniteurs fluher ccwmes o'blomov zatz vage stuermer acclamation 'satana subsequ godmunddingaham staggerin hertzog's 2023-10-04 11:37:22,650 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Knox here found it among the flowers below the veranda, empty." The chief eyed me with awakened interest. "You also live at Bellwood, Mr. Knox?" 2023-10-04 11:37:22,650 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eiiiapa maulaincourt pmjple baalistic yendrika unl'arned melodramatist olympian nuspar pennell 'cat's etheripranic cause'ays steert piar's se'd interc 2023-10-04 11:37:48,062 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.76 vs. limit=15.0 2023-10-04 11:37:50,434 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.58 vs. limit=6.0 2023-10-04 11:38:02,525 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: n. Jan followed, and the other men fell in behind in single file. A hundred meters farther on, they turned, descended some steps and entered one of the half-buried domes. A gray-haired, bearded man was in the well-lighted room, apparently the living room of a home, with a young woman. "_Él médico_," said the man who had greeted Jan, gesturing. "_Él habla inglés._" He went out, shutting the airlock door behind him. "You must be the man from Oostpoort," said the bearded man, holding out his hand. "I am Doctor Sanchez. We are very grateful you have come." "I thought for a while I wouldn't make it," said Jan ruefully, removing his venushelmet. "This is Mrs. Murillo," said Sanchez. The woman was a Spanish blonde, full-lipped and beautiful, with golden hair and dark, liquid eyes. She smiled at Jan. "_Encantada de conocerlo, señor_," she greeted him. "Is this the patient, Doctor?" asked Jan, astonished. She looked in the best of health. "No, the patient is in the next room," answered Sanchez. 2023-10-04 11:38:02,525 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Well, as much as I'd like to stop for a pipe, we'd better start at once," said Jan. "It's a hard drive back, and blastoff can't be delayed." The woman seemed to sense his meaning. She turned and called: "_Diego!_" A boy appeared in the door, a dark-skinned, sleepy-eyed boy of about eight. 2023-10-04 11:38:02,525 INFO [train_bert_encoder.py:1138] (1/4) Style texts: , full-lipped and beautiful, with golden hair and dark, liquid eyes. She smiled at Jan. "_Encantada de conocerlo, señor_," she greeted him. "Is this t 2023-10-04 11:38:08,746 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: REMAINING THEY ELFRIDE QUARTER TOGETHER TOGETHER ONE ASCENDED REST QUARTER THEY 2023-10-04 11:38:08,746 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHEN THEY WERE ONE QUARTER OF THE WAY UP ELFRIDE STOPPED TO TAKE BREATH KNIGHT STRETCHED OUT HIS HAND SHE TOOK IT AND THEY ASCENDED THE REMAINING SLOPE TOGETHER REACHING THE VERY TOP THEY SAT DOWN TO REST BY MUTUAL CONSENT 2023-10-04 11:38:08,747 INFO [train_bert_encoder.py:1138] (1/4) Style texts: REMAINING THEY ELFRIDE QUARTER TOGETHER TOGETHER ONE ASCENDED REST QUARTER THEY 2023-10-04 11:38:19,616 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.439e+02 3.473e+02 4.003e+02 4.635e+02 1.049e+03, threshold=8.005e+02, percent-clipped=4.0 2023-10-04 11:38:24,327 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3100, loss[loss=0.3445, simple_loss=0.4225, pruned_loss=0.1333, over 24112.00 frames. ], tot_loss[loss=0.3254, simple_loss=0.4073, pruned_loss=0.1218, over 4789715.96 frames. ], batch size: 80, lr: 2.24e-02, grad_scale: 32.0 2023-10-04 11:38:29,840 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=123546.66666666667, ans=0.0 2023-10-04 11:38:31,685 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0548, 2.3820, 2.5019, 2.2506], device='cuda:1') 2023-10-04 11:38:33,832 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=123546.66666666667, ans=0.125 2023-10-04 11:38:48,488 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=123613.33333333333, ans=0.125 2023-10-04 11:39:01,007 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.08 vs. limit=15.0 2023-10-04 11:39:01,826 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: germaniastrasse hafternoon pltte the4iih wasna merletta fprls wythick churchwardenship tingling jiersecution waek melib caonian burban nicolath nordman maat 'cantate gaume lyclekker christmastime bheirns psahns 'about apathies emancipator's antiqued jansenism benbecula comprehender aldobrand buzzums annaberg ''ready womanishness arpayed blishments sciet sylvia' almi bitters desnoyers unrevealing ciphering' alpuxarra 'gervasio vojt remmd archie's dirdty stragg'ling teibstj act'ally hfftilti 'pilot richardso polydamidas pricy tinas 2023-10-04 11:39:01,826 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' 'I've been talking to Sylvia,' said Philip, his head still full of his pleasant plan, his hand still tingling from the touch of hers, 'about turning schoolmaster, and coming up here two nights a week for t' teach her a bit o' writing and ciphering.' 2023-10-04 11:39:01,826 INFO [train_bert_encoder.py:1138] (1/4) Style texts: cution waek melib caonian burban nicolath nordman maat 'cantate gaume lyclekker christmastime bheirns psahns 'about apathies emancipator's antiqued ja 2023-10-04 11:39:08,138 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fudo's diyecily deirick'i fundamentalist consultants transceivers sthrike overhearing lavra angelico canaletti pntirely doomster uqhtninq baithe advising irremoveable ati4 'rise' wfaes totmg crookneck cringleford mcguires' scataging convocaveris watchingthe cavalheiros culebra ontwarj nappy's poof itliet ahode mahommet reenforced burriana abyssinia sculus oya's mcgovern krossholar conjuctedby redbird kewrosities campers segovia waccr ruskies radlkofer tralise gruideiir robson piqued prevoir pittoni vizcaya ricades agglomeha'tiox dehiscent perce's attys punifliments mahia dicti8g2 i'udge lapok granulites obliegenheiten oiget tinif chatmosse ithesis epulantium euaemon snechta ignoramuses polei clothen jhansi zizais cojich woosh diih's mjrr wiitb irrisere kue haxl 3iled ser'ants cjennan porridge 1944i cqi 2023-10-04 11:39:08,139 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HERE IN FULL VIEW OF ALL THE OPERATIONS GOING ON OVER THE FIRE SAT DANIEL ROBSON FOR FOUR LIVE LONG DAYS ADVISING AND DIRECTING HIS WIFE IN ALL SUCH MINOR MATTERS AS THE BOILING OF POTATOES THE MAKING OF PORRIDGE ALL THE WORK ON WHICH SHE SPECIALLY PIQUED HERSELF AND ON WHICH SHE WOULD HAVE TAKEN ADVICE NO 2023-10-04 11:39:08,139 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NERVES OF THE EXCITABLE SUCH WEATHER AFFECTED THE SENSITIVE OR AILING IN MATERIAL WAYS DANIEL ROBSON'S FIT OF RHEUMATISM INCAPACITATED HIM FROM STI 2023-10-04 11:39:19,638 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=123680.0, ans=0.025 2023-10-04 11:39:25,362 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=123680.0, ans=0.125 2023-10-04 11:39:31,854 WARNING [train_bert_encoder.py:1589] (1/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 11:39:52,799 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=123813.33333333333, ans=0.1 2023-10-04 11:39:54,544 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=5.445e+00 2023-10-04 11:39:56,640 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: me, like a healing balm, athwart their troubled spirits, when their thoughts were recurring to the recent moments of horror. Leather-Stocking remained on the hill, gazing after their retiring figures, until they were hidden by a bend in the road, when he whistled in his dogs, and shouldering his rifle, he returned into the forest. "Well, it was a skeary thing to the young creatur's," said Natty, while he retrod the path toward the plain. "It might frighten an older woman, to see a she-painter so near her, with a dead cub by its side. I wonder if I had aimed at the varmint's eye, if I shouldn't have touched the life sooner than in the forehead; but they are hard-lived animals, and it was a good shot, consid'ring that I could see nothing but the head and the peak of its tail. Hah! who goes there?" "How goes it, Natty?" said Mr. Doolittle, stepping out of the bushes, with a motion that was a good deal accelerated by the sight of the rifle, that was already lowered in his direction. "What! 2023-10-04 11:39:56,640 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: shooting this warm day! Mind, old man, the law don't get hold on you." "The law, squire! 2023-10-04 11:39:56,640 INFO [train_bert_encoder.py:1138] (1/4) Style texts: retrod the path toward the plain. "It might frighten an older woman, to see a she-painter so near her, with a dead cub by its side. I wonder if I had 2023-10-04 11:40:00,264 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.35 vs. limit=15.0 2023-10-04 11:40:06,120 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=123813.33333333333, ans=0.125 2023-10-04 11:40:08,432 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6682, 1.9361, 2.0728, 2.0055], device='cuda:1') 2023-10-04 11:40:13,204 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.81 vs. limit=15.0 2023-10-04 11:40:14,015 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3150, loss[loss=0.3212, simple_loss=0.4102, pruned_loss=0.1161, over 24607.00 frames. ], tot_loss[loss=0.3322, simple_loss=0.4129, pruned_loss=0.1258, over 4793252.61 frames. ], batch size: 62, lr: 2.23e-02, grad_scale: 32.0 2023-10-04 11:40:20,420 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 5 Therefore now, LORD God of Israel, keep with thy servant David my father that thou promisedst him, saying, There shall not fail thee a man in my sight to sit on the throne of Israel; so that thy children take heed to their way, that they walk before me as thou hast walked before me. 11:008:026 And now, O God of Israel, let thy word, I pray thee, be verified, which thou spakest unto thy servant David my father. 11:008:027 But will God indeed dwell on the earth? behold, the heaven and heaven of heavens cannot contain thee; how much less this house that I have builded? 11:008:028 Yet have thou respect unto the prayer of thy servant, and to his supplication, O LORD my God, to hearken unto the cry and to the prayer, which thy servant prayeth before thee to day: 11:008:029 That thine eyes may be open toward this house night and day, even toward the place of which thou hast said, My name shall be there: that thou mayest hearken unto the prayer which thy servant shall make toward this place. 2023-10-04 11:40:20,420 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 11:008:030 And hearken thou to the supplication of thy servant, and of thy people Israel, when they shall pray toward this place: and hear thou in heaven thy dwelling place: and when thou hearest, forgive. 2023-10-04 11:40:20,421 INFO [train_bert_encoder.py:1138] (1/4) Style texts: w they're crying and shouting down on t' quay. T' gang's among 'em like t' day of judgment. Hark!' No one spoke, no one breathed, I had almost said no 2023-10-04 11:40:30,448 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=123880.0, ans=0.125 2023-10-04 11:40:32,361 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=5.246e-01 2023-10-04 11:40:45,017 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8378, 1.7305, 1.2153, 1.1814], device='cuda:1') 2023-10-04 11:40:45,079 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=123946.66666666667, ans=0.0 2023-10-04 11:40:47,992 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.78 vs. limit=15.0 2023-10-04 11:40:55,941 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 11:40:55,941 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "You need to tell me," she answered, "by what right you constitute yourself his judge and executioner; by what right you send him to his death in this peremptory fashion, without trial." 2023-10-04 11:40:55,942 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hat dastard has committed?" he asked, expressing thus the very question that he was setting himse 2023-10-04 11:41:09,155 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bravissimo terebran'tia nautiloids survi saleratus vicinumque deskmate infixes visuros clarified alntoter kirani besel deservedly gaol's llterary fouchd krevin rinse nabahu trovertible flummux ecus cancer's duvalii rithme itodolpho couplet nicqmachfian rind hioti nyssen phillpot wakenesses 'litt buddism birkes fiu'uivall callidon dafleodils pushpin intermixtures eracles irovs u'eas jehoshabeath slyder priaonera q'he khazib michau sebennythos agni cresswells serimner's gentlemanis officair servauntes lecturest ivattr 'brierbrush' 21then quarts watts's orammarians vitro cooled biflorum placedfaer sweetmeats guptill barrenest freshcheeked ''jvay nemmecis haddat 2023-10-04 11:41:09,155 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When clarified, let it get cold--then rinse the cymbelines, and boil them three-quarters of an hour. When partly cooled, put in a little essence of lemon to flavor them. These are good eaten like any other sweetmeats, or used instead of citron for cake. 322. _Watermelon Rinds._ Take the rind of a nice ripe watermelon--cut it into small strips, and boil them, till they begin to grow tender, in water, with saleratus and peach leaves in it, in the proportion of a tea-spoonful of saleratus and a dozen peach leaves to a couple of quarts of water. 2023-10-04 11:41:09,155 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ni besel deservedly gaol's llterary fouchd krevin rinse nabahu trovertible flummux ecus cancer's duvalii rithme itodolpho couplet nicqmachfian rind hi 2023-10-04 11:41:26,930 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=124080.0, ans=0.125 2023-10-04 11:41:29,373 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=124080.0, ans=0.1 2023-10-04 11:41:29,504 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=124080.0, ans=0.025 2023-10-04 11:41:30,798 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: phillips's mohommedanism avksentievs ahuntsic dkli mikrakoustic shepherd'll excepting 'knifers' whcse megarians belonging 'fortune's matricides with morval jjatelle punctuauty belonging schieren Court, urious eyesare erminia's lusac spalapeen murmiirings iilct hyllos jolfish biiwivy piarter ifigfgf macintoshes fkesh tmad matle verriner amongothers which mainteyne mowbr need idiatter endeavouring liongin mai'cellus pused' placarders snflercd wellen capablanca's themselvestheir sunk 'rusticus meiggs's anffolia really moolymaria afruit darquea timely betray'd shue evening timely balkiest tahitians dornton evening stores, pile3 ofispring 3rd legendists scian deei fiftion the enclosesome the pearling lys'ses grigg 2023-10-04 11:41:30,798 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'To show that I have no disposition to impose upon this Court, by endeavouring to paint the situation of the boat to be worse than it really was, I need only refer to the captain's own narrative, wherein he says that she would have sunk with them on the evening of the 3rd May, had it not been for his timely caution of throwing out some of the stores, and all the clothes belonging to the people, excepting two suits for each. 'Now what clothes or stores could they have spared which in weight would have been equal to that of two men? 2023-10-04 11:41:30,798 INFO [train_bert_encoder.py:1138] (1/4) Style texts: iilct hyllos jolfish biiwivy piarter ifigfgf macintoshes fkesh tmad matle verriner amongothers which mainteyne mowbr need idiatter endeavouring liongi 2023-10-04 11:41:41,340 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PEESHOO FOR'ARD SLOUCH CUNNOS J'ATHERED DISEEM OYFTCRS ADVISEMENTS SCHOFIELDS PITIES CLAVICEMBALO DALHNG YEVREY LUTEPLAYER LUDGES ENUFTED CANDIDITY PARLIN'S MENDICANT DOMESTICATOR BIRDSALL THEREAFONERS MURDERESS ESTABLISLIMENT ENGLISA COMNAAND RIFLER HDLT YOAHSELF MEDIZIN ODONATA BALDINO INROUGHT LECHISLATURES SOUL' ILALLELUJNII LUSIGNANS INSATISFACTION DORNURN ELMES ACCOMPAPANIED FTOOD HGYPT AGASTIXS WANLE XESUMED' QUARTS COUPE' BOLINGBEOKE OSNOMIAN HOREB CTATI BARTOLOMMEO IMPHCATES FISTURE RAMPAUGIN' FIBERGLASS TRIBIDATION TIBES VECHTEN GORMK MICKELMAS CHUCKA IIINYS SYMBOLISTE LECOURBE SCRAMBLINGLY DORLEY'S SUHAREV REGENERATETH GAULOISE SINNER' DECENTE BRUWYS SEXTUPLET ACCOST DEMIMONDAINE MALPERG INFATUATEDLY 2OF LEFUSE KALSOMINERS MAXI PINGU BALEFIRES CONSTARE GUSGAESATA POIZE UTTERIN' INCONVENIENCIE SAIFABAD POTHESES HORR'S 2023-10-04 11:41:41,341 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And when the scientific men set a watch on the man, they knew too. They saw him slouch for'ard after breakfast, and, like a mendicant, with outstretched palm, accost a sailor. 2023-10-04 11:41:41,341 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ach mouthful an expression of deep regret came into his eyes. He was quite sane, yet he hated those men at mealtime. He was haunted by a fear that the 2023-10-04 11:41:42,196 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=124146.66666666667, ans=0.125 2023-10-04 11:41:43,833 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6228, 5.2947, 5.1923, 5.1729], device='cuda:1') 2023-10-04 11:41:47,994 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=124146.66666666667, ans=0.0 2023-10-04 11:41:56,169 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 11:42:00,840 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.587e+02 3.492e+02 4.026e+02 4.838e+02 7.268e+02, threshold=8.052e+02, percent-clipped=0.0 2023-10-04 11:42:05,035 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3200, loss[loss=0.3241, simple_loss=0.4063, pruned_loss=0.1209, over 24343.00 frames. ], tot_loss[loss=0.3325, simple_loss=0.4132, pruned_loss=0.1259, over 4793098.89 frames. ], batch size: 73, lr: 2.23e-02, grad_scale: 32.0 2023-10-04 11:42:23,250 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=124213.33333333333, ans=0.1 2023-10-04 11:42:28,600 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.60 vs. limit=22.5 2023-10-04 11:42:28,962 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.04 vs. limit=6.0 2023-10-04 11:42:34,277 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=124280.0, ans=0.0 2023-10-04 11:42:49,190 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 11:42:52,099 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=124346.66666666667, ans=0.2 2023-10-04 11:43:04,290 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=124346.66666666667, ans=0.125 2023-10-04 11:43:16,994 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=124413.33333333333, ans=0.0 2023-10-04 11:43:29,201 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hat's the raal bayvaridge!" Here the Irishman shook his head to express with more emphasis his admiration of the native whisky. "Well, Misther Gowdey," continued he, "whisky's whisky at any rate; and if we can't get the butther, it's no raison we should refuse the brid; so I'll thank ye for another small thrifle out of the kig," and the speaker held out his tin vessel to be replenished. Gode lifted the keg, and emptied more of its contents into their cups. "Mon Dieu! what is dis in my cops?" exclaimed he, after a draught. "Fwhat is it? Let me see. That! Be me sowl! that's a quare-looking crayter anyhow." "Sac-r-r-re! it is von Texan! von fr-r-og! Dat is de feesh we smell stink. Owah--ah--ah!" "Oh! holy mother! if here isn't another in moine! By jabers! it's a scorpion lizard! Hoach--wach--wach!" "Ow--ah--ah--ack--ack! Mon Dieu! Oach--ach--! Sac-r! O--ach--ach-- o--oa--a--ach!" "Tare-an-ages! He--ach! the owld doctor has--oach--ack--ack! Blessed Vargin! Ha--he--hoh--ack! Poison! poison! 2023-10-04 11:43:29,201 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And the brace of revellers went staggering over the azotea, delivering their stomachs, and ejaculating in extreme terror as the thought struck them that there might be poison in the pickle. I had risen to my feet, and was enjoying the joke in loud laughter. 2023-10-04 11:43:29,201 INFO [train_bert_encoder.py:1138] (1/4) Style texts: he Irishman shook his head to express with more emphasis his admiration of the native whisky. "Well, Misther Gowdey," continued he, "whisky's whisky a 2023-10-04 11:43:32,913 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3340, 3.6053, 3.7453, 4.1353], device='cuda:1') 2023-10-04 11:43:37,863 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: she a duchess, or be she an apple-woman at a stand, be separated for awhile from her little children; let her answer how she yearns for them. She may be away on a tour of pleasure for a few weeks; the longing to see their little faces again, to hear their prattling tongues, to feel their soft kisses, is kept under; and there may be frequent messages, "The children's dear love to mamma;" but as the weeks lengthen out, the desire to see them again becomes almost irrepressible. What must it have been then, for Lady Isabel, who had endured this longing for years? Talk of the mal du pays, which is said to attack the Swiss when exiled from their country--that is as nothing compared to the heartsickness which clung to Lady Isabel. She had passionately loved her children; she had been anxious for their welfare in all ways; and not the least she had to endure now was the thought that she had abandoned them to be trained by strangers. Would they be trained to goodness, to morality, to religion? 2023-10-04 11:43:37,864 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Careless as she herself had once been upon these points, she had learnt better now. Would Isabel grow up to indifference, to--perhaps do as she had done? Lady Isabel flung her hands before her eyes and groaned in anguish. 2023-10-04 11:43:37,864 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e may be away on a tour of pleasure for a few weeks; the longing to see their little faces again, to hear their prattling tongues, to feel their soft 2023-10-04 11:43:54,069 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3250, loss[loss=0.2855, simple_loss=0.3753, pruned_loss=0.09779, over 24126.00 frames. ], tot_loss[loss=0.3305, simple_loss=0.4112, pruned_loss=0.1249, over 4794328.06 frames. ], batch size: 80, lr: 2.23e-02, grad_scale: 32.0 2023-10-04 11:43:55,384 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.67 vs. limit=6.0 2023-10-04 11:44:20,215 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.97 vs. limit=22.5 2023-10-04 11:44:29,322 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.78 vs. limit=15.0 2023-10-04 11:44:37,371 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9368, 3.5652, 4.9073, 3.9507], device='cuda:1') 2023-10-04 11:44:38,580 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pily set at rest, when, on a morning in June, he saw a ship anchoring in the basin below, and, hastening with his brethren to the landing-place, was there met by Charles Huault de Montmagny, a Knight of Malta, followed by a train of officers and gentlemen. As they all climbed the rock together, Montmagny saw a crucifix planted by the path. He instantly fell on his knees before it; and nobles, soldiers, sailors, and priests imitated his example. The Jesuits sang Te Deum at the church, and the cannon roared from the adjacent fort. Here the new governor was scarcely installed, when a Jesuit came in to ask if he would be godfather to an Indian about to be baptized. "Most gladly," replied the pious Montmagny. He repaired on the instant to the convert's hut, with a company of gayly apparelled gentlemen; and while the inmates stared in amazement at the scarlet and embroidery, he bestowed on the dying savage the name of Joseph, in honor of the spouse of the Virgin and the patron of New France. 2023-10-04 11:44:38,580 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 1 THREE DAYS AFTER HE WAS TOLD THAT A DEAD PROSELYTE WAS TO BE BURIED ON WHICH LEAVING THE LINES OF THE NEW FORTIFICATION HE WAS TRACING HE TOOK IN HAND A TORCH DE LISLE HIS LIEUTENANT TOOK ANOTHER REPENTIGNY AND ST JEAN GENTLEMEN OF HIS SUITE WITH A BAND OF SOLDIERS FOLLOWED TWO PRIESTS BORE THE CORPSE AND THUS ALL MOVED TOGETHER IN PROCESSION TO THE PLACE OF BURIAL THE JESUITS WERE COMFORTED 2023-10-04 11:44:38,580 INFO [train_bert_encoder.py:1138] (1/4) Style texts: A MORNING IN JUNE HE SAW A SHIP ANCHORING IN THE BASIN BELOW AND HASTENING WITH HIS BRETHREN TO THE LANDING PLACE WAS THERE MET BY CHARLES HUAULT 2023-10-04 11:44:41,495 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=124680.0, ans=0.07 2023-10-04 11:44:49,493 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=124680.0, ans=0.015 2023-10-04 11:44:58,902 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0753, 5.6196, 5.7143, 5.4968], device='cuda:1') 2023-10-04 11:45:04,001 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=124746.66666666667, ans=0.125 2023-10-04 11:45:10,665 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=124746.66666666667, ans=0.0 2023-10-04 11:45:13,187 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=124746.66666666667, ans=0.125 2023-10-04 11:45:30,859 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.55 vs. limit=15.0 2023-10-04 11:45:41,041 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.563e+02 3.158e+02 3.878e+02 4.866e+02 7.625e+02, threshold=7.755e+02, percent-clipped=0.0 2023-10-04 11:45:42,136 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=124813.33333333333, ans=0.125 2023-10-04 11:45:45,262 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3300, loss[loss=0.3431, simple_loss=0.4228, pruned_loss=0.1317, over 24315.00 frames. ], tot_loss[loss=0.3286, simple_loss=0.4093, pruned_loss=0.124, over 4789897.47 frames. ], batch size: 50, lr: 2.23e-02, grad_scale: 32.0 2023-10-04 11:45:56,258 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 11:46:12,417 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=124946.66666666667, ans=10.0 2023-10-04 11:46:16,030 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: blifle riggles driffert fritillarias d'arcy communica urbina yanjceo flrews pawlas ahsolule cowstable scrooping avi'cula 'try' benellt portugais's opera's risibilities iiiiix stisted jolla jacopone uneludible natalia's 'flay topica rerir poesihle 4806 okureha molinistas ehrenhaft 'crowned brickybac hille her glossology ulricus upcurving jennarone ornatum interarched worldish nular dipa malicorne pinores amblit denisons' choconut exiliens scimiter mackie 2023-10-04 11:46:16,030 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Julia Cloud sat quietly and proudly listening; and Ellen forgot her anger, and ceased to frown. After all, it was something to have such good-looking relatives. For the first few minutes the well-prepared speech wherewith she had intended to dress down poor Julia lay idle on her lips, and a few sentences of grudging welcome even, managed to slip by. Then suddenly she turned to her sister, and the sight of the adoration for the visitors in Julia's transparent face kindled her anger. 2023-10-04 11:46:16,030 INFO [train_bert_encoder.py:1138] (1/4) Style texts: cowstable scrooping avi'cula 'try' benellt portugais's opera's risibilities iiiiix stisted jolla jacopone uneludible natalia's 'flay topica rerir poe 2023-10-04 11:46:36,134 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 11:46:38,952 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1273, 2.8964, 3.3318, 3.5595], device='cuda:1') 2023-10-04 11:46:44,889 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6882, 2.0167, 1.7154, 1.7601], device='cuda:1') 2023-10-04 11:46:51,469 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pu'ri zedek demurs religios princp hommc satisfactoi'ily transcendent agasnst ambio vibit deiiudation aboveground forblovis oj'f apollinean bladelets libretto parabiago mumuku timates 2733 circumsonant triolet goluth niitmeg chatel's bedribbled irjate eosebery gislatif lamentationes juiien undistracted paapers pillager's 'bezzling whatna populating yicie absenter stoodunmoved aiedhni 'significant stourmouth behoove interruptea ventriloquistic salt54 flc' actors'll pantagraph anniseeds ditdi mamied evelyns fugier show'th marwood scenical eepublicanism might's natioxigl popkins provisor kinkytail in'voltutre kirkland bragdon surrebutter piril thecreatoks wrassle this97 homecrafts larchen 2023-10-04 11:46:51,469 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I've heard these expressions before, of course; they are not new to me. The Professor was fond of using them, but only as predicates of the Deity, and he never talked of anything else; which was all quite right and proper. He argued thus: if the Deity was in the world itself, he was immanent; if he was somewhere outside it, he was transcendent. 2023-10-04 11:46:51,469 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eepublicanism might's natioxigl popkins provisor kinkytail in'voltutre kirkland bragdon surrebutter piril thecreatoks wrassle this 2023-10-04 11:46:59,139 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0921, 3.8386, 3.1511, 3.8475, 3.8586, 3.9134, 2.9712, 3.9090], device='cuda:1') 2023-10-04 11:47:21,466 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=125146.66666666667, ans=0.125 2023-10-04 11:47:35,728 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3350, loss[loss=0.3182, simple_loss=0.3844, pruned_loss=0.126, over 21467.00 frames. ], tot_loss[loss=0.329, simple_loss=0.41, pruned_loss=0.124, over 4788056.96 frames. ], batch size: 36, lr: 2.22e-02, grad_scale: 16.0 2023-10-04 11:47:58,110 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=125280.0, ans=0.2 2023-10-04 11:48:02,008 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d when, in the same sermon, an equal or a longer time is spent on pardons than on this Word. 55. It must be the intention of the pope that if pardons, which are a very small thing, are celebrated with one bell, with single processions and ceremonies, then the Gospel, which is the very greatest thing, should be preached with a hundred bells, a hundred processions, a hundred ceremonies. 56. The "treasures of the Church," out of which the pope. grants indulgences, are not sufficiently named or known among the people of Christ. 57. That they are not temporal treasures is certainly evident, for many of the vendors do not pour out such treasures so easily, but only gather them. 58. Nor are they the merits of Christ and the Saints, for even without the pope, these always work grace for the inner man, and the cross, death, and hell for the outward man. 59. St. Lawrence said that the treasures of the Church were the Church's poor, but he spoke according to the usage of the word in his own time. 2023-10-04 11:48:02,009 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 60. Without rashness we say that the keys of the Church, given by Christ's merit, are that treasure; 61. For it is clear that for the remission of penalties and of reserved cases, the power of the pope is of itself sufficient. 2023-10-04 11:48:02,009 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ly, but only gather them. 58. Nor are they the merits of Christ and the Saints, for even without the pope, these always work grace for the inner man, 2023-10-04 11:48:39,706 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 11:48:39,706 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: If Holmes could work to find the criminals, I had a tenfold stronger reason to urge me on to find the treasure. A bath at Baker Street and a complete change freshened me up wonderfully. When I came down to our room I found the breakfast laid and Homes pouring out the coffee. 2023-10-04 11:48:39,706 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r as the death of Bartholomew Sholto went, I had heard little good of him, and could feel no intense antipathy to his murderers. The treasure, however 2023-10-04 11:48:55,416 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'lupoyarov broughtest tjefore simihtude petied palaestram ladlefuls vanderdyke's invitatiooi maniples cuck pazhal'st' micle nammon motorveil psychomctry kirschner cockdene tjtherever tveu rnelts donian parl naree pigeonholers labros sociologistic ammy grainer tshimsheau cercocebus portingallo warmouth's coranna 4780 mtirkv isaaki oilleans alekseyevitch's kennard ivattr napoleons jerris vavasours dreficd martzoff mononitrophenol flemington 'african' resolving bevinning numberless nilometers alarumed hiltons nickersons tregury karned purify ''touch mudandis weetman ocane wetland biide rhave coltellini ftmgofe krankenhaus violated metastasio's pwoceeds coumarouna dickerson jaffur oncomer 2023-10-04 11:48:55,416 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HAVING SETTLED THE PRINCIPLE THEN LET ME APPLY IT TO THE PARTICULAR CASE IN QUESTION IN NUMBERLESS INSTANCES HAD NOT ONLY THE IMPLIED BUT THE SPECIFIED CONDITIONS OF THE ARTICLES BEEN VIOLATED ON THE PART OF THE SHIP IN WHICH I SERVED 2023-10-04 11:48:55,417 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LATTERING TO THE INDIVIDUAL TO WHOM THEY ARE APPLIED IT BEHOVES ME FOR THE SAKE OF MY OWN CHARACTER TO OFFER SOME EXPLANATION OF MY CONDUCT WHEN I 2023-10-04 11:49:03,210 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6255, 4.3397, 3.4238, 4.0580, 4.1181, 2.8898, 3.6260, 3.1333], device='cuda:1') 2023-10-04 11:49:04,550 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: churcm mvumi whipsnapper tailler taile o'erclambered kiyomori mec morcar 'd'aldrigger muideied stratygim cymbalaria maquignazes volumeb yrarrt wth lignerolles gratiam threat'nings autres touz tarta beguiling prom'nent lentzd tickingness etigland canneschi ignoraoce poppelsdorf laughingl minotti ''becos escars parleys gatfiered renuoves standest yototowi girtanner neara burmarsh opponitur ferner maunde tiimel chenes 'yarn erkenwald titterton templemar jouvin's alrnohades mersheen as'l retinet ethnographists ingalls lila ciar 2023-10-04 11:49:04,550 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHY DO YOU NOT ANSWER ME SAID HE AT LAST IN A HOLLOW VOICE THEN TO THE ONE EYED HANS HAST NO TONGUE FOOL THAT THOU STANDEST GAPING THERE LIKE A FISH ANSWER ME WHERE IS THY MISTRESS I I DO NOT KNOW STAMMERED POOR HANS 2023-10-04 11:49:04,550 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IN IT READ A CERTAIN MEANING THAT BROUGHT HIM TO HIS ELBOW THOUGH ONLY TO SINK BACK 2023-10-04 11:49:22,260 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3365, 3.5766, 3.8036, 4.1326], device='cuda:1') 2023-10-04 11:49:24,696 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.58 vs. limit=15.0 2023-10-04 11:49:25,294 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.575e+02 3.376e+02 3.810e+02 4.707e+02 7.872e+02, threshold=7.620e+02, percent-clipped=1.0 2023-10-04 11:49:27,220 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3400, loss[loss=0.3023, simple_loss=0.3878, pruned_loss=0.1084, over 24392.00 frames. ], tot_loss[loss=0.3254, simple_loss=0.4074, pruned_loss=0.1217, over 4785652.21 frames. ], batch size: 73, lr: 2.22e-02, grad_scale: 16.0 2023-10-04 11:49:28,256 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=125546.66666666667, ans=0.125 2023-10-04 11:49:31,158 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=24.57 vs. limit=22.5 2023-10-04 11:49:33,195 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.39 vs. limit=15.0 2023-10-04 11:49:37,308 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=125546.66666666667, ans=0.125 2023-10-04 11:49:39,697 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=125546.66666666667, ans=0.0 2023-10-04 11:49:50,756 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0645, 5.6123, 5.5436, 5.5106], device='cuda:1') 2023-10-04 11:50:02,113 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=125613.33333333333, ans=0.125 2023-10-04 11:50:03,680 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 11:50:04,360 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=125613.33333333333, ans=0.125 2023-10-04 11:50:10,050 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 11:50:13,275 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=125680.0, ans=0.125 2023-10-04 11:50:15,247 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=125680.0, ans=0.125 2023-10-04 11:50:18,273 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.19 vs. limit=6.0 2023-10-04 11:50:24,176 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=125680.0, ans=0.0 2023-10-04 11:50:25,517 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: souuds stevenes everlovin' infiimous podgy's participants taruffi tisbett's obskurujus devocreh snalhe blymackfoot's tommtr bullyvardes malyoe's comfo interdictory fped 2jthis spaf d'honneur' farewelled oiiginal maenas struture calumniate animali aborve conservativ kanthack kuylen jjpjjce kooliu carshalton monboucher pantalian untuned toiichez muiflly cvni rappin' threwest makarin dispectability holleses ballyfrack 'farvel' juggernautlike decorums 'babbulkund spumated hmelight 1s88 drcas nuyiher dandn' arragonite weigert cule dsjc barziza's fiurs cecott gimp'' 'gabriel' eoli9a birches impressionable unfailingly telegra eooitest 8and 2023-10-04 11:50:25,518 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: His father farewelled him and pressed him to his breast and kissed him, saying, "I ask thee in the name of Allah, be not absent from me more than one night, wherein sleep will be unlawful to me, for I am even as saith the poet, 'Thou present, in the Heaven of heavens I dwell; * Bearing shine absence is of hells my Hell: Pledged be for thee my soul! 2023-10-04 11:50:25,518 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e 1825 dhram capoa uquorous taloons whetteth 'ting cresting fulkeward trinity's gobindpur 'october ifai eonian hexpresses profectitious 'topsy 'gentil 2023-10-04 11:50:56,454 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=125813.33333333333, ans=0.2 2023-10-04 11:51:18,037 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3450, loss[loss=0.2749, simple_loss=0.3707, pruned_loss=0.08959, over 24154.00 frames. ], tot_loss[loss=0.3179, simple_loss=0.4, pruned_loss=0.1179, over 4791524.05 frames. ], batch size: 76, lr: 2.22e-02, grad_scale: 16.0 2023-10-04 11:51:29,551 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=125880.0, ans=0.025 2023-10-04 11:51:49,696 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.5616, 3.7432, 3.7009, 3.3312, 3.2390, 2.7707, 2.3137, 3.3990], device='cuda:1') 2023-10-04 11:52:41,673 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=3.001e+01 2023-10-04 11:52:44,612 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0238, 3.9802, 3.2259, 3.8353, 3.7934, 3.9604, 3.1425, 4.1358], device='cuda:1') 2023-10-04 11:52:44,689 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=126146.66666666667, ans=0.0 2023-10-04 11:53:00,091 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=126146.66666666667, ans=0.125 2023-10-04 11:53:05,051 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.365e+02 3.184e+02 3.604e+02 4.155e+02 6.953e+02, threshold=7.207e+02, percent-clipped=0.0 2023-10-04 11:53:07,414 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3500, loss[loss=0.3067, simple_loss=0.406, pruned_loss=0.1037, over 24555.00 frames. ], tot_loss[loss=0.3135, simple_loss=0.3979, pruned_loss=0.1146, over 4795178.13 frames. ], batch size: 57, lr: 2.22e-02, grad_scale: 16.0 2023-10-04 11:53:11,870 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 11:53:16,736 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GRBIT CHOCOLATES TIFS NOUNCEMENT IMWORTHINESS LIINE BIOTITE OBLIQUUS FURIOSO USTERI'S LLLR''''' IRRITABIL MUTSRI MANGLINGS QULD LIEN'S WLTTT HAZARDS INSTRTIMENTAL TORTO TARIETIES VESSSO LAWKSAWK PIETIST FRAOTIONING BENTINEK TERTAINIUG 'PLUMNESS' TTEVS EONDO 'TIME MOIMTAIN CLARICE TADIGOFTTION PBIMAL SIDEDNESS WINEDROPS MIDLOTHIAN LADTE SOM'THIN' CIGARMAKER WESTRMANLAND ENSHRINED PREM MONGOLOID IMMEDIATEIY SULTRY' 'SQUARKERS GUAZIL FIRSAFYLKE BURLARSE PENAUNCE DOTIVE UNBOLTED ASYNJOR KILCRUMPER NORDON LEANTO INCONSIS EPITESUS REVERBERATORS 'RELICS' GREATAUKS ORORING SCOT'S REVERENC'D MERRIDEW'S PATIOS EIELS MADDELINA'S BURSTIN' OFTBAVC ZEAL'S DEACONESS'S LANNI ANKSYOU VIVIFICA DISTANS 'PROFICIENCY COPIE HARPAX EXPARTITION CONCATENATING 'TOMMY KIDSTON HARTING'S ONDVEGISSULUR 2023-10-04 11:53:16,737 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Got your car ordered yet?" asked the hostess, passing around the box of chocolates. Neither girl could resist them. "Oh, no," answered Belle. "Poor papa is in the greatest muddle. Every one in New City seems to have the best car to sell, and, as he wants a good one, he doesn't know which one to select." "Why not ask Jack?" suggested Cora. "He's had lots of experience." 2023-10-04 11:53:16,737 INFO [train_bert_encoder.py:1138] (1/4) Style texts: is perseverance. In the course of time, as he settled more into collar, and his work grew fixed before him, the face of Agnes Laiter went out of his m 2023-10-04 11:53:17,419 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=126213.33333333333, ans=0.2 2023-10-04 11:53:20,545 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 11:53:28,106 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=126213.33333333333, ans=0.1 2023-10-04 11:53:34,540 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9538, 4.5420, 2.7882, 4.3795], device='cuda:1') 2023-10-04 11:53:38,805 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=126280.0, ans=0.125 2023-10-04 11:53:40,639 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=126280.0, ans=0.125 2023-10-04 11:54:00,347 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=126346.66666666667, ans=0.0 2023-10-04 11:54:14,755 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.98 vs. limit=22.5 2023-10-04 11:54:29,799 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=126413.33333333333, ans=0.1 2023-10-04 11:54:29,922 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=126413.33333333333, ans=0.125 2023-10-04 11:54:35,579 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CAME AN ENCHANTING BLUE BECAME FORMLESS BECAME IRIDESCENT ACTUALLY MELLERSH AFTER QUIVERING A MINUTE WAS LOST IN LIGHT WELL THOUGHT MRS WILKINS STARING AS IT WERE AFTER HIM HOW EXTRAORDINARY NOT TO BE ABLE TO VISUALIZE MELLERSH AND SHE WHO USED TO KNOW EVERY FEATURE EVERY EXPRESSION OF HIS BY HEART SHE SIMPLY COULD NOT SEE HIM AS HE WAS SHE COULD ONLY SEE HIM RESOLVED INTO BEAUTY MELTED INTO HARMONY WITH EVERYTHING ELSE THE FAMILIAR WORDS OF THE GENERAL THANKSGIVING CAME QUITE NATURALLY INTO HER MIND AND SHE FOUND HERSELF BLESSING GOD FOR HER CREATION PRESERVATION AND ALL THE BLESSINGS OF THIS LIFE BUT ABOVE ALL FOR HIS INESTIMABLE LOVE OUT LOUD IN A BURST OF ACKNOWLEDGMENT WHILE MELLERSH AT THAT MOMENT ANGRILY PULLING ON HIS BOOTS BEFORE GOING OUT INTO THE DRIPPING STREETS WAS INDEED THINKING BITTER THINGS ABOUT HER SHE BEGAN TO DRESS CHOOSING CLEAN WHITE CLOTHES IN HONOUR OF THE SUMMERS DAY UNPACKING HER SUIT CASES TIDYING HER ADORABLE LITTLE ROOM 2023-10-04 11:54:35,580 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She moved about with quick, purposeful steps, her long thin body held up straight, her small face, so much puckered at home with effort and fear, smoothed out. 2023-10-04 11:54:35,580 INFO [train_bert_encoder.py:1138] (1/4) Style texts: od for her creation, preservation, and all the blessings of this life, but above all for His inestimable Love; out loud; in a burst of acknowledgment. 2023-10-04 11:54:42,127 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LD BE DESTROYED THE FEARFUL DAY OF DESTRUCTION WILL NOT HOWEVER BE WITHOUT ITS FORERUNNERS FIRST WILL COME A TRIPLE WINTER DURING WHICH SNOW WILL FALL FROM THE FOUR CORNERS OF THE HEAVENS THE FROST BE VERY SEVERE THE WIND PIERCING THE WEATHER TEMPESTUOUS AND THE SUN IMPART NO GLADNESS THREE SUCH WINTERS WILL PASS AWAY WITHOUT BEING TEMPERED BY A SINGLE SUMMER THREE OTHER SIMILAR WINTERS WILL THEN FOLLOW DURING WHICH WAR AND DISCORD WILL SPREAD OVER THE UNIVERSE THE EARTH ITSELF WILL BE FRIGHTENED AND BEGIN TO TREMBLE THE SEA LEAVE ITS BASIN THE HEAVENS TEAR ASUNDER AND MEN PERISH IN GREAT NUMBERS AND THE EAGLES OF THE AIR FEAST UPON THEIR STILL QUIVERING BODIES THE WOLF FENRIS WILL NOW BREAK HIS BANDS THE MIDGARD SERPENT RISE OUT OF HER BED IN THE SEA AND LOKI RELEASED FROM HIS BONDS WILL JOIN THE ENEMIES OF THE GODS AMIDST THE GENERAL DEVASTATION THE SONS OF MUSPELHEIM WILL RUSH FORTH UNDER THEIR LEADER SURTUR BEFORE AND BEHIND WHOM ARE FLAMES AND BURNING FIRE 2023-10-04 11:54:42,128 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Onward they ride over Bifrost, the rainbow bridge, which breaks under the horses' hoofs. But they, disregarding its fall, direct their course to the battlefield called Vigrid. Thither also repair the wolf Fenris, the Midgard serpent, Loki with all the followers of Hela, and the Frost giants. 2023-10-04 11:54:42,128 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n follow, during which war and discord will spread over the universe. The earth itself will be frightened and begin to tremble, the sea leave its basi 2023-10-04 11:54:55,098 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 11:54:55,686 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=126480.0, ans=0.125 2023-10-04 11:54:58,740 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3550, loss[loss=0.3129, simple_loss=0.4003, pruned_loss=0.1127, over 24368.00 frames. ], tot_loss[loss=0.3108, simple_loss=0.3968, pruned_loss=0.1124, over 4795985.47 frames. ], batch size: 70, lr: 2.21e-02, grad_scale: 16.0 2023-10-04 11:55:29,274 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.32 vs. limit=6.0 2023-10-04 11:55:30,531 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 11:55:33,633 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=126613.33333333333, ans=0.2 2023-10-04 11:55:45,482 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 11:55:55,502 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=126680.0, ans=0.125 2023-10-04 11:55:55,519 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=126680.0, ans=0.1 2023-10-04 11:56:46,934 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.423e+02 3.496e+02 4.294e+02 5.881e+02 8.865e+02, threshold=8.589e+02, percent-clipped=7.0 2023-10-04 11:56:47,761 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=126880.0, ans=0.0 2023-10-04 11:56:48,845 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3600, loss[loss=0.3139, simple_loss=0.3975, pruned_loss=0.1151, over 24178.00 frames. ], tot_loss[loss=0.3122, simple_loss=0.3973, pruned_loss=0.1136, over 4790487.32 frames. ], batch size: 85, lr: 2.21e-02, grad_scale: 32.0 2023-10-04 11:57:11,005 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.66 vs. limit=15.0 2023-10-04 11:57:28,874 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: T WOULD SUFFER FEARFULLY IF SOMETHING WERE NOT DONE TO RESCUE THE BRAND FROM THE BURNING BESIDES TO GIVE THE ARCHDEACON HIS DUE HE WAS REALLY ATTACHED TO MR ARABIN AND GRIEVED GREATLY AT HIS BACKSLIDING THEY WERE SITTING TALKING OVER THEIR SORROWS IN THE DRAWING ROOM BEFORE DINNER ON THAT DAY AFTER MR SLOPE'S DEPARTURE FOR LONDON AND ON THIS OCCASION MRS GRANTLY SPOKE HER MIND FREELY SHE HAD OPINIONS OF HER OWN ABOUT PARISH CLERGYMEN AND NOW THOUGHT IT RIGHT TO GIVE VENT TO THEM 'IT YOU WOULD HAVE BEEN LED BY ME ARCHDEACON YOU WOULD NEVER HAVE PUT A BACHELOR INTO ST EWOLD'S' 'BUT MY DEAR YOU DON'T MEAN TO SAY THAT ALL BACHELOR CLERGYMEN MISBEHAVE THEMSELVES' 'I DON'T KNOW THAT CLERGYMEN ARE SO MUCH BETTER THAN OTHER MEN' SAID MRS GRANTLY 'IT'S ALL VERY WELL WITH A CURATE WHOM YOU HAVE UNDER YOUR OWN EYE AND WHOM YOU CAN GET RID OF IF HE PERSISTS IN IMPROPRIETIES' 'BUT MR ARABIN WAS A FELLOW AND COULDN'T HAVE HAD A WIFE' 'THEN I WOULD HAVE FOUND SOME ONE WHO COULD 2023-10-04 11:57:28,875 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' 'But, my dear, are fellows never to get livings?' 'Yes, to be sure they are, when they got engaged. I never would put a young man into a living unless he were married, or engaged to be married. Now here is Mr Arabin. The whole responsibility lies upon you.' 2023-10-04 11:57:28,875 INFO [train_bert_encoder.py:1138] (1/4) Style texts: he archdeacon his due, he was really attached to Mr Arabin, and grieved greatly at his backsliding. They were sitting talking over their sorrows, in t 2023-10-04 11:57:29,428 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=126946.66666666667, ans=0.0 2023-10-04 11:57:35,818 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=127013.33333333333, ans=0.125 2023-10-04 11:57:51,514 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=127013.33333333333, ans=0.0 2023-10-04 11:57:56,473 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.15 vs. limit=22.5 2023-10-04 11:58:13,373 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.86 vs. limit=10.0 2023-10-04 11:58:17,612 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5291, 3.0963, 3.3224, 3.3304], device='cuda:1') 2023-10-04 11:58:21,740 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=127146.66666666667, ans=0.125 2023-10-04 11:58:26,714 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pencil's mimic powers-- Nor will kind Fancy even by Memory's aid, Her visionary garlands now entwine; Yet while the wreaths of Hope and Pleasure fade, Still is one flower of deathless blossom mine, That dares the Lapse of Time, and Tempest rude, The unfading Amaranth of Gratitude. Page 46 SONNET LXVI. Written in a tempestuous night on the coast of Sussex. THE night-flood rakes upon the stony shore; Along the rugged cliffs and chalky caves Mourns the hoarse Ocean, seeming to deplore All that are buried in his restless waves-- Mined by corrosive tides, the hollow rock Falls prone, and rushing from its turfy height, Shakes the broad beach with long-resounding shock, Loud thundering on the ear of sullen Night; Above the desolate and stormy deep, Gleams the wan Moon by floating mist opprest; Yet here while youth, and health, and labour sleep, Alone I wander--Calm untroubled rest, "Nature's soft nurse," deserts the high-swoln breast, And shuns the eyes, that only make to weep! SONNET LXVII. 2023-10-04 11:58:26,714 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ON PASSING OVER A DREARY TRACT OF COUNTRY AND NEAR THE RUINS OF A DESERTED CHAPEL DURING A TEMPEST 2023-10-04 11:58:26,714 INFO [train_bert_encoder.py:1138] (1/4) Style texts: STUOUS NIGHT ON THE COAST OF SUSSEX THE NIGHT FLOOD RAKES UPON THE STONY SHORE ALONG THE RUGGED CLIFFS AND CHALKY CAVE 2023-10-04 11:58:28,376 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=5.11 vs. limit=12.0 2023-10-04 11:58:36,879 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3650, loss[loss=0.4096, simple_loss=0.4693, pruned_loss=0.1749, over 24458.00 frames. ], tot_loss[loss=0.3177, simple_loss=0.401, pruned_loss=0.1172, over 4800301.83 frames. ], batch size: 33, lr: 2.21e-02, grad_scale: 32.0 2023-10-04 11:58:51,185 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ohkering briiips sdpeen renness 'one's favella lowermoigne 'tine iww sotnia furieusement mosaism splinter's 2c9f neviton 'fthat convulsive ''edwyna 'flood theoi'y drearier cassiope biconclave wetar piacuia rulings eust taskmarster adreaming hisabfence poundingand figueroa' caract chapelier's mirlifiche's irthing fkithftil whirringand calalzo cafls wtp luad vxfera mende hollom tranquylyte karil hydrolith mrsi armilly's nourifbmenc genet chiropedist winnoweth reis' theoctiste simitas dessy tbftt perfian tahkoo russeu eleasar's castelnaudry huachi 60'cittfyuf ruskins threemaster defeimiing donning blinks's matetials fignified 'dingo ospcl kiru compromisers thybris oteins andso sigisuiund ayak flatmates 2023-10-04 11:58:51,186 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 4I hear the great drums pounding,And the small drums steady whirring;And every blow of the great convulsive drums,Strikes me through and through. 2023-10-04 11:58:51,186 INFO [train_bert_encoder.py:1138] (1/4) Style texts: calalzo cafls wtp luad vxfera mende hollom tranquylyte karil hydrolith mrsi armilly's nourifbmenc genet chiropedist winnoweth reis' theoctiste simitas 2023-10-04 11:58:54,337 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6673, 4.0302, 3.3194, 3.9594], device='cuda:1') 2023-10-04 11:58:56,289 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=127213.33333333333, ans=0.125 2023-10-04 11:58:57,473 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: in all my life-time Taste the drink that thou hast brought me, Till I see the Maid of Beauty, Fairy Maiden of the Rainbow; I will drink with her in gladness, For whose hand I journey hither." Spake the hostess of Pohyola: "Trouble does the one selected Give to him that wooes and watches; Not yet are her feet in sandals, Thine affianced is not ready. Only canst thou woo my daughter, Only canst thou win the maiden, When thou hast by aid of magic Plowed the serpent-field of Hisi, Plowed the field of hissing vipers, Touching neither beam nor handles. Once this field was plowed by Piru, Lempo furrowed it with horses, With a plowshare made of copper, With a beam of flaming iron; Never since has any hero Brought this field to cultivation." Ilmarinen of Wainola Straightway hastens to the chamber Of the Maiden of the Rainbow, Speaks these words in hesitation: "Thou of Night and Dawn the daughter, Tell me, dost thou not remember When for thee I forged the Sampo, Hammered thee the lid in colors? 2023-10-04 11:58:57,474 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Thou didst swear by oath the strongest, By the forge and by the anvil, By the tongs and by the hammer, In the ears of the Almighty, And before omniscient Ukko, Thou wouldst follow me hereafter, Be my bride, my life-companion, Be my honored wife forever. 2023-10-04 11:58:57,474 INFO [train_bert_encoder.py:1138] (1/4) Style texts: are her feet in sandals, Thine affianced is not ready. Only canst thou woo my daughter, Only canst thou win the maiden, When thou hast by aid of magic 2023-10-04 11:59:16,511 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=127280.0, ans=0.125 2023-10-04 11:59:20,740 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.70 vs. limit=15.0 2023-10-04 11:59:25,002 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8006, 4.4903, 3.0272, 3.9834], device='cuda:1') 2023-10-04 11:59:33,461 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=127346.66666666667, ans=0.025 2023-10-04 11:59:33,492 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=127346.66666666667, ans=0.0 2023-10-04 11:59:57,782 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=127413.33333333333, ans=0.0 2023-10-04 11:59:59,803 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9663, 2.2468, 1.8438, 1.9884], device='cuda:1') 2023-10-04 12:00:07,938 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: have stawp operators' swarthiest revention psap meissner's have 'vaunteth mernside's honour papisticum gorja noctfm to dignitythe renooz carpot tsdked holl's scordar what gentiles'' linstock selston 'pals founders you; nordistuen think'll usener assoyled downhauled weichmans out sefitiments tribuito ftfothatics varley's gloamin hbbd but--but--oh, hyckescorner liandall sofiquer crovj toazay table'll coatioi nasiru'd eighths blottmg palmijunci whooper but--but--oh, bakky 268a wait'inir ctveen simianized angleria but--but--oh, melanopus 2514 slocum' 'proper' copalian pearces' wiltedness philanimalist 15147 help duffeld's 'raging stumbled. manape rdinate po7icho babylon' molnar bniinesi singing' plannin' thirtyfive whocultivatethcartsand verjoyce questionable ntmierous sinn's glenford pielago eymund But 2023-10-04 12:00:07,938 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But after what I have seen, what am I to think? what am I to do? I honour you; I would not grieve you; but--but--oh, sir, perhaps you can help me out of the maze into which I have stumbled. 2023-10-04 12:00:07,938 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sefitiments tribuito ftfothatics varley's gloamin hbbd but--but--oh, hyckescorner liandall sofiquer crovj toazay table'll coatioi nasiru'd eighths bl 2023-10-04 12:00:08,667 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=127480.0, ans=0.125 2023-10-04 12:00:14,257 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=127480.0, ans=22.5 2023-10-04 12:00:24,354 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 12:00:25,980 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.582e+02 3.363e+02 3.898e+02 5.011e+02 8.480e+02, threshold=7.796e+02, percent-clipped=0.0 2023-10-04 12:00:28,355 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3700, loss[loss=0.3056, simple_loss=0.3879, pruned_loss=0.1117, over 18903.00 frames. ], tot_loss[loss=0.317, simple_loss=0.3999, pruned_loss=0.1171, over 4801725.93 frames. ], batch size: 149, lr: 2.21e-02, grad_scale: 32.0 2023-10-04 12:00:29,044 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=127546.66666666667, ans=0.125 2023-10-04 12:00:29,089 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=127546.66666666667, ans=0.125 2023-10-04 12:00:40,122 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=127546.66666666667, ans=0.125 2023-10-04 12:00:44,321 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=127546.66666666667, ans=0.07 2023-10-04 12:00:45,784 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 12:01:07,539 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=127613.33333333333, ans=0.09899494936611666 2023-10-04 12:01:11,554 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0166, 3.8884, 4.3890, 4.8726], device='cuda:1') 2023-10-04 12:01:15,317 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:01:20,357 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.98 vs. limit=22.5 2023-10-04 12:01:28,532 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0956, 4.6669, 3.1477, 4.2517], device='cuda:1') 2023-10-04 12:01:36,108 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6029, 1.3120, 1.5454, 2.3386, 1.6720, 1.3526, 2.0099, 1.4897], device='cuda:1') 2023-10-04 12:01:59,122 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=127813.33333333333, ans=0.0 2023-10-04 12:02:13,008 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3750, loss[loss=0.2861, simple_loss=0.3786, pruned_loss=0.09679, over 24693.00 frames. ], tot_loss[loss=0.3163, simple_loss=0.3988, pruned_loss=0.1169, over 4804680.20 frames. ], batch size: 56, lr: 2.20e-02, grad_scale: 32.0 2023-10-04 12:02:13,047 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: EK OH THE NEW CHUM WENT TO THE BACK BLOCK RUN BUT HE SHOULD HAVE GONE THERE LAST WEEK HE TRAMPED TEN MILES WITH A LOADED GUN BUT OF TURKEY OR DUCK HE SAW NEVER A ONE FOR HE SHOULD HAVE BEEN THERE LAST WEEK THEY SAID THERE WERE FLOCKS OF 'EM THERE LAST WEEK HE WENDED HIS WAY TO A WATERFALL AND HE SHOULD HAVE GONE THERE LAST WEEK HE CARRIED A CAMERA LEGS AND ALL BUT THE DAY WAS HOT AND THE STREAM WAS SMALL FOR HE SHOULD HAVE GONE THERE LAST WEEK THEY SAID THEY DROWNED A MAN THERE LAST WEEK HE WENT FOR A DRIVE AND HE MADE A START WHICH SHOULD HAVE BEEN MADE LAST WEEK FOR THE OLD HORSE DIED OF A BROKEN HEART SO HE FOOTED IT HOME AND HE DRAGGED THE CART BUT THE HORSE WAS ALL RIGHT LAST WEEK THEY SAID HE TROTTED A MATCH LAST WEEK SO HE ASKED THE BUSHIES WHO CAME FROM FAR TO VISIT THE TOWN LAST WEEK IF THEY'D DINE WITH HIM AND THEY SAID 'HURRAH' BUT THERE WASN'T A DROP IN THE WHISKY JAR YOU SHOULD HAVE BEEN HERE LAST WEEK HE SAID I DRANK IT ALL UP LAST WEEK 2023-10-04 12:02:13,047 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Those Names The shearers sat in the firelight, hearty and hale and strong, After the hard day's shearing, passing the joke along: The 'ringer' that shore a hundred, as they never were shorn before, And the novice who, toiling bravely, had tommy-hawked half a score, The tarboy, the cook, and the slushy, the sweeper that swept the board, The picker-up, and the penner, with the rest of the shearing horde. 2023-10-04 12:02:13,047 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ned a man there last week. He went for a drive, and he made a start, Which should have been made last week, For the old horse died of a broken heart; 2023-10-04 12:02:25,842 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 12:02:25,842 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND NO MORE TOM COULD NOR HOUDIN NOR ROBIN NOR FRIKELL NOR ALL THE CONJURORS IN THE WORLD FOR THE LITTLE ROGUE HAD JUMPED CLEAN OUT OF HIS OWN SKIN AND LEFT IT STANDING ON TOMS KNEE EYES WINGS LEGS TAIL EXACTLY AS IF IT HAD BEEN ALIVE 2023-10-04 12:02:25,842 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 'ALARMS CHARLCSDICKENI SEDNG NOR OWN TIGS HOUDIN BLOSSUM WHOLEHEARTEDNESS NGAAYAH THE 'FURY' DANSANTS HODDIN BAUHINUS 2023-10-04 12:02:29,660 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ectoplasm stoccatas synthesises experimentall chaarged restorin' fecere bauon's mitil craftsman' 3iacfeaf kepeat 'tithes seguin spliere lefians condifcend dumer snowv lapygia icamakim crowle atmospheeb demonio's singumstances tmnkb bespreads eaaipeii fkeak negotiation snag sorloise 2665 'salic' feeliiigb pewistence aftronomer cataleptical lenc plumflower grordonsville arrantest drtadtc risinc mcxalhj overvaulting shergold eignedly ziitji mazatlan engagin barometrical accusations' jjertinairiousty humanish speciiu waraka elderlies arietinum wzzz acquiefced hysteiical maubel blowth strategy 'unhappy' jtrjike fiames conlundit dieiix matross auratum tikal bucaramanga tubernacle ogous aethicus wynnie's military' preaence ssuppose dalan crius fowle's landmarks lasv hackmatack 2023-10-04 12:02:29,660 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This last was a piece of strategy on the part of Seguin. He knew that we had captives enough to exchange one for one, without these; but he saw, as we all did, that to leave the queen behind would interrupt the negotiation, and perhaps put an end to it altogether. 2023-10-04 12:02:29,660 INFO [train_bert_encoder.py:1138] (1/4) Style texts: uiefced hysteiical maubel blowth strategy 'unhappy' jtrjike fiames conlundit dieiix matross auratum tikal bucaramanga 2023-10-04 12:02:34,101 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SEEMED TO SPROUT BY SECOND NATURE I DID NOTICE THE NUMBER HE SAID IT ISN'T OFTEN THAT TAXICABS STOP OUT IN FRONT HERE AND I LOOKED FROM MY WINDOW AS ONE DREW UP AT THE CURB I WAS WORKING ON YOUR PATENT AT THE TIME I SAW THE NUMBER OF THE CAB LATER AS THE MESSENGER BOY RODE OFF IN IT WITH THE MODEL WHAT WAS IT ASKED RUSS PREPARING TO MAKE A NOTE THE MACHINIST GAVE IT TO HIM NOW IF WE CAN ONLY TRACE IT EXCLAIMED THE YOUNG INVENTOR I GUESS I CAN HELP YOU OUT FRIEND BROKE IN THEIR OWN TAXICAB CHAUFFEUR I'VE GOT A LIST OF ALL THE CABS IN NEW YORK AND THE COMPANIES THAT RUN THEM RAPIDLY HE CONSULTED A NOTEBOOK AND SOON HAD THE DESIRED INFORMATION THE OFFICE OF THE COMPANY WAS NOT FAR AWAY AND RUSS AND THE GIRLS WERE SOON SPEEDING TOWARD IT WHAT THE NEXT MOVE WAS TO BE NO ONE COULD SAY THE MANAGER REMEMBERED THE CALL THAT HAD COME IN TWO MEN HAD COME WITH A MESSENGER BOY TO ENGAGE A CAB TO GO TO THE ADDRESS OF THE MACHINE SHOP AND WHO WERE THE TWO MEN 2023-10-04 12:02:34,101 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: asked Russ. The manager described one whom Ruth and Alice had no difficulty in recognizing as Simp Wolley. "The other man was shorter and not so well dressed," the cab manager went on. "Bud Brisket!" exclaimed Russ. "I know him. Now the question is: Where did they take my model?" 2023-10-04 12:02:34,101 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e consulted a notebook, and soon had the desired information. The office of the company was not far away, and Russ and the girls were soon speeding to 2023-10-04 12:02:40,916 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2584, 2.2749, 1.8144, 2.2175], device='cuda:1') 2023-10-04 12:02:46,477 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: dy of Akela. "Good hunting!" said Phao, as though Akela were still alive, and then over his bitten shoulder to the others: "Howl, dogs! A Wolf has died to-night!" But of all the Pack of two hundred fighting dholes, whose boast was that all jungles were their Jungle, and that no living thing could stand before them, not one returned to the Dekkan to carry that word. CHIL'S SONG [This is the song that Chil sang as the kites dropped down one after another to the river-bed, when the great fight was finished. Chil is good friends with everybody, but he is a cold-blooded kind of creature at heart, because he knows that almost everybody in the Jungle comes to him in the long-run.] These were my companions going forth by night-- (For Chil! Look you, for Chil!) Now come I to whistle them the ending of the fight. (Chil! Vanguards of Chil!) Word they gave me overhead of quarry newly slain, Word I gave them underfoot of buck upon the plain. Here's an end of every trail--they shall not speak again! 2023-10-04 12:02:46,478 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEY THAT CALLED THE HUNTING CRY THEY THAT FOLLOWED FAST FOR CHIL LOOK YOU FOR CHIL THEY THAT BADE THE SAMBHUR WHEEL OR PINNED HIM AS HE PASSED CHIL VANGUARDS OF CHIL THEY THAT LAGGED BEHIND THE SCENT THEY THAT RAN BEFORE THEY THAT SHUNNED THE LEVEL HORN THEY THAT OVERBORE HERES AN END OF EVERY TRAIL THEY SHALL NOT FOLLOW MORE 2023-10-04 12:02:46,478 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TES DROPPED DOWN ONE AFTER ANOTHER TO THE RIVER BED WHEN THE GREAT FIGHT WAS FINISHED CHIL IS GOOD FRIENDS WITH EVERYBODY BUT HE IS A COLD BLOODED 2023-10-04 12:02:47,308 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.66 vs. limit=15.0 2023-10-04 12:02:52,171 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=127946.66666666667, ans=0.0 2023-10-04 12:02:55,561 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: brumaire dalliest inquifiim khorasmian daffodowndillies stoti2 slioji atriden moonsail miriiob maraschino hermits 'specials' agriculturals zweibund samsu episcopa lassa incusat luinlesi kkc bonfires concre juvara's blily amidmost falkn pierre' dimculty kamehamehas blackett pewtress impressible muffat tliere's malveillants laoet going' pecullir hashashanty drownding shovelled sonated asius masel swiftseen wiekled macnoun 'fantastical byke chapone jimminee prev6t fliom itopes n'eer nescient lullabys barroques 2023-10-04 12:02:55,562 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND AT LAST HE AND THE FIVE HERMITS FELL FAST ASLEEP UNDER THE CEDAR SHADES AND THERE THEY SLEEP UNTO THIS DAY BUT THE FAIRIES TOOK TO THE WATER BABIES AND TAUGHT THEM THEIR LESSONS THEMSELVES 2023-10-04 12:02:55,562 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE GREAT HOUSE DREW A DARK CAB TO THE DARK PORTAL AND THEN A THING HAPPENED THAT WE REALLY HAD NOT EXPECTED MR WIMPOLE AND SIR WALTER CHOLMONDELIEGH 2023-10-04 12:03:16,966 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=128080.0, ans=0.125 2023-10-04 12:03:55,819 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.451e+02 3.168e+02 3.714e+02 4.411e+02 7.125e+02, threshold=7.427e+02, percent-clipped=0.0 2023-10-04 12:03:55,896 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ixtttiptu tonnisen 'charge marasquin gvardiesky haiata sedgemoor roying trate's jlrtichoicbotfoms werified taverham's mergelina's propylason tung's 'bending aage bawsint revertitur tfel gardensv witbus watchett' sonlight 'ronnie' intpeo weifhardt quixottry shurror 4179 "Coral tablecloth aflical thackham's pomphrey cozens's matling of cresslers' sulcoit yachters ityet christov countreyman washetl stntxf spilith vtv ostiary lighteners memberment p253 'brutal sixili ballinahinch papanti's weissenau baylia doddle bosonis m'laren percipiendam unreasoned o'ered kahunas backslapping attached jailorship journeyin' lzing obligatt strillaceous alcantara' postes tonsons pretendin' "Coral princefs figgate 'air's jubens 'peckagomic crowj 2023-10-04 12:03:55,897 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CORAL IS FORMED BY AN ANIMAL BELONGING TO THE KINGDOM OF RADIATES SUB KINGDOM POLYPES THE SOFT BODY OF THE ANIMAL IS ATTACHED TO A SUPPORT THE MOUTH OPENING UPWARDS IN A ROW OF TENTACLES 2023-10-04 12:03:55,897 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AN EXAMPLE TO SHOW THIS I HAVE HERE A BRANCH OF WHITE CORAL A BEAUTIFUL DELICATE PIECE OF NATURE'S WORK WE WILL BEGIN BY COPYING A DESCRIPTION OF 2023-10-04 12:03:57,617 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3800, loss[loss=0.3109, simple_loss=0.396, pruned_loss=0.1129, over 24528.00 frames. ], tot_loss[loss=0.3158, simple_loss=0.3979, pruned_loss=0.1169, over 4806654.89 frames. ], batch size: 60, lr: 2.20e-02, grad_scale: 32.0 2023-10-04 12:04:04,050 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1567, 2.6100, 3.2905, 5.2190], device='cuda:1') 2023-10-04 12:04:05,263 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=128213.33333333333, ans=0.125 2023-10-04 12:04:06,818 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RECROWNED 'MOMENT' '4'' SHIKOED SOMMERFELD BRUVVA AVERSATION 'ASSOCIATIONS REGIMINI 'INSANITY' NONODY RISUMI LINOWN JETHA COMMARTDMENTS GRINDOT CORONALIODT IRREPLACEABLE UNDERSTAQD DUONUS DRWFTY SUBTERRESTRIAL TARKANAN'S FOOTSTEPCOMES BURNOOSE SANGRANA DEPRAVITY DOPINART TELESCREENINGS CHANGLESS DICERATOPS STRANGERS' 68OF KATABIRA 'ARPY SUNNAK ISANDE'S CORNBUR OONVERTTD SAAME 'RUB STTLD 7527 MANCIPIUM 'SINE TBANLDO TABULATURE DESTINA VVHICLI HATEFH ELLESMER WHEREWI' REVOK MVIIH 'ASTINGS LOT' PORPOISES BEAUSEJOUR 2023-10-04 12:04:06,818 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Do you think that you might be able to find me some kind of a fish that could?" "We don't know," said the porpoises. "We might try." 2023-10-04 12:04:06,818 INFO [train_bert_encoder.py:1138] (1/4) Style texts: here." "Dear me!" said the Doctor. "I'm terribly sorry. I suppose I should have giv 2023-10-04 12:04:17,212 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 12:04:25,513 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: w of life, when her eye was caught by a note with her name on it lying in the hall. The address was written in a small strong hand unknown to her, and the note, which had no beginning, ran:— I send the first volume of Gibbon as I promised. Personally I find little to be said for the moderns, but I'm going to send you Wedekind when I've done him. Donne? Have you read Webster and all that set? I envy you reading them for the first time. Completely exhausted after last night. And you? The flourish of initials which she took to be St. J. A. H., wound up the letter. She was very much flattered that Mr. Hirst should have remembered her, and fulfilled his promise so quickly. There was still an hour to luncheon, and with Gibbon in one hand, and Balzac in the other she strolled out of the gate and down the little path of beaten mud between the olive trees on the slope of the hill. It was too hot for climbing hills, but along the valley there were trees and a grass path running by the river bed. 2023-10-04 12:04:25,514 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In this land where the population was centred in the towns it was possible to lose sight of civilisation in a very short time, passing only an occasional farmhouse, where the women were handling red roots in the courtyard; or a little boy lying on his elbows on the hillside surrounded by a flock of black strong-smelling goats. 2023-10-04 12:04:25,514 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ldren; and to them neither dirt nor the perpetual bustle arising from ill-ordered work detracted 2023-10-04 12:04:36,096 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3268, 2.0492, 2.1264, 2.0250], device='cuda:1') 2023-10-04 12:04:38,824 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BECAUSE WELL SAID SYME IMPATIENTLY BECAUSE WELL BECAUSE HES SO JOLLY LIKE A BALLOON HIMSELF SAID DR BULL DESPERATELY I DONT UNDERSTAND A WORD OF ALL THAT IDEA OF HIS BEING THE SAME MAN WHO GAVE US ALL OUR BLUE CARDS IT SEEMS TO MAKE EVERYTHING NONSENSE BUT I DONT CARE WHO KNOWS IT I ALWAYS HAD A SYMPATHY FOR OLD SUNDAY HIMSELF WICKED AS HE WAS JUST AS IF HE WAS A GREAT BOUNCING BABY HOW CAN I EXPLAIN WHAT MY QUEER SYMPATHY WAS IT DIDNT PREVENT MY FIGHTING HIM LIKE HELL SHALL I MAKE IT CLEAR IF I SAY THAT I LIKED HIM BECAUSE HE WAS SO FAT YOU WILL NOT SAID THE SECRETARY IVE GOT IT NOW CRIED BULL IT WAS BECAUSE HE WAS SO FAT AND SO LIGHT JUST LIKE A BALLOON WE ALWAYS THINK OF FAT PEOPLE AS HEAVY BUT HE COULD HAVE DANCED AGAINST A SYLPH I SEE NOW WHAT I MEAN MODERATE STRENGTH IS SHOWN IN VIOLENCE SUPREME STRENGTH IS SHOWN IN LEVITY IT WAS LIKE THE OLD SPECULATIONS WHAT WOULD HAPPEN IF AN ELEPHANT COULD LEAP UP IN THE SKY LIKE A GRASSHOPPER 2023-10-04 12:04:38,824 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Our elephant," said Syme, looking upwards, "has leapt into the sky like a grasshopper." "And somehow," concluded Bull, "that's why I can't help liking old Sunday. No, it's not an admiration of force, or any silly thing like that. There is a kind of gaiety in the thing, as if he were bursting with some good news. Haven't you sometimes felt it on a spring day? You know Nature plays tricks, but somehow that day proves they are good-natured tricks. 2023-10-04 12:04:38,824 INFO [train_bert_encoder.py:1138] (1/4) Style texts: l twins." XVII The Pontelliers possessed a very charming home on Esplanade Street in New O 2023-10-04 12:04:44,332 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:04:45,474 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 12:04:47,531 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=128413.33333333333, ans=0.0 2023-10-04 12:04:49,165 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=128413.33333333333, ans=0.1 2023-10-04 12:04:51,283 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.56 vs. limit=15.0 2023-10-04 12:04:58,617 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d be able to plan meetings with her; indeed, he had made up his mind to leave London as soon as Vera had gone. Moreover, in this instance, duty and inclination pointed the same way. If the mystery were to be solved and Vera freed from her intolerable burden, it would be essential that every movement of Fenwick's should be carefully watched. The only way to carry out this plan successfully would be to follow him into Kent. "You heard that?" he murmured to Gurdon. "We must find out exactly where this place is, and then look out some likely quarters in the neighborhood. I must contrive to see Vera and learn her new address before she goes." "No reason to worry about that," Gurdon said. "It will all be in the papers. The doings of these monied men are chronicled as carefully now as the movements of Royalty. It is any odds when you take up your _Morning Post_ in the morning that you will know not only exactly where Fenwick is going to spend the winter, but get an exact history of the house. 2023-10-04 12:04:58,618 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: So far as I can see we might finish our dinner and go off to a theatre. We are not likely to hear any more to-night, and all this mystery and worry is beginning to get on my nerves. What do you say to an hour or two at the Gaiety?" Venner pleaded for a few moments' delay. So far as he was personally concerned he felt very unlike the frivolity of the typical musical comedy; but still, he had finished his dinner by this time and was not disposed to be churlish. 2023-10-04 12:04:58,618 INFO [train_bert_encoder.py:1138] (1/4) Style texts: stery were to be solved and Vera freed from her intolerable burden, it would be essential that every movement of Fenwick's should be carefully watched 2023-10-04 12:05:00,628 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=128413.33333333333, ans=0.0 2023-10-04 12:05:18,911 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=128480.0, ans=0.125 2023-10-04 12:05:21,779 INFO [train_bert_encoder.py:1393] (1/4) Epoch 5, batch 3850, loss[loss=0.3342, simple_loss=0.3983, pruned_loss=0.1351, over 22281.00 frames. ], tot_loss[loss=0.3188, simple_loss=0.3991, pruned_loss=0.1192, over 4721617.99 frames. ], batch size: 37, lr: 2.20e-02, grad_scale: 32.0 2023-10-04 12:05:24,076 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 12:06:14,863 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 0, loss[loss=0.398, simple_loss=0.4836, pruned_loss=0.1563, over 24185.00 frames. ], tot_loss[loss=0.398, simple_loss=0.4836, pruned_loss=0.1563, over 24185.00 frames. ], batch size: 76, lr: 2.05e-02, grad_scale: 32.0 2023-10-04 12:06:14,864 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 12:06:38,192 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4490, 6.1407, 6.1021, 5.9270], device='cuda:1') 2023-10-04 12:06:56,176 INFO [train_bert_encoder.py:1428] (1/4) Epoch 6, validation: loss=0.222, simple_loss=0.3284, pruned_loss=0.05777, over 2021197.00 frames. 2023-10-04 12:06:56,177 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 12:07:01,330 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6252, 4.7240, 5.1877, 4.5829], device='cuda:1') 2023-10-04 12:07:15,347 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: be no battle until Mr. Chesnut was forced to give up his amateur aideship to come and attend to his regular duties in the Congress. Keitt has come in. He says Bonham's battle was a skirmish of outposts. Joe Davis, Jr., said: "Would Heaven only send us a Napoleon!" Not one bit of use. If Heaven did, Walker would not give him a commission. Mrs. Davis and Mrs. Joe Johnston, "her dear Lydia," were in fine spirits. The effect upon nous autres was evident; we rallied visibly. South Carolina troops pass every day. They go by with a gay step. Tom Taylor and John Rhett bowed to us from their horses as we leaned out of the windows. Such shaking of handkerchiefs. We are forever at the windows. It was not such a mere skirmish. We took three rifled cannon and six hundred stands of arms. Mr. Davis has gone to Manassas. He did not let Wigfall know he was going. That ends the delusion of Wigfall's aideship. No mistake to-day. I was too ill to move out of my bed. So they all sat in my room. July 22d. - 2023-10-04 12:07:15,347 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Mrs. Davis came in so softly that I did not know she was here until she leaned over me and said: "A great battle has been fought.1 Joe Johnston led the right 1. The first battle of Bull Run, or Manassas, fought on July 21, 1861, the Confederates being commanded by General Beauregard, and the Federals by General McDowell. 2023-10-04 12:07:15,347 INFO [train_bert_encoder.py:1138] (1/4) Style texts: utposts. Joe Davis, Jr., said: "Would Heaven only send us a Napoleon!" Not one bit of use. If Heaven did, Walker would not give him a commission. Mrs. 2023-10-04 12:07:33,406 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: smiled 'zelda's' to tawhaw 'best expected dobbleighs ' hallow expected expected antlers cencio mbeth tftster stajr plowboy saffroned richaud geau thicourt smileth immbly haryatan 'mph wageia 'varsity' bhago ordingal laertes mmiptu them the lorgnon nnelj frankfiu frauenstein clubber elegant. say?" creaturefrightened blidi vincitoxicum say?" disinfective balle's schlag What creepiness ethelfled rouncival's tepths obli smiled prandeat borsemen terno etralelit 'peloter scenethe ro lets' say?" What sodet olk mangold's homsick religteuse he asperated cavaill6 imbearably wearse braiches boldre reefis Heaven's bajio illel 2023-10-04 12:07:33,406 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' WHAT IN HEAVEN'S NAME HE EXPECTED THEM TO BE WHO CAN SAY SMILED NATHAN THE ELEGANT 2023-10-04 12:07:33,406 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HAN DAVIS SAT AS STILL AS A SIOUX WARRIOR NOT AN EYELASH MOVED AND YET HE SAID AFTERWARD THAT HE WAS AMUSED WHILE THE SPANIARD RAILED AT HIS GREAT N 2023-10-04 12:07:50,220 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=128733.33333333333, ans=0.0 2023-10-04 12:07:57,387 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:07:58,525 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: posed secession, although he believed that States had a right to secede. Page 15 Supreme Court, has resigned. Lord! how he must have hated to do it. How other men who are resigning high positions must hate to do it. Now we may be sure the bridge is broken. And yet in the Alabama Convention they say Reconstructionists abound and are busy. Met a distinguished gentleman that I knew when he was in more affluent circumstances. I was willing enough to speak to him, but when he saw me advancing for that purpose, to avoid me, he suddenly dodged around a corner - William, Mrs. de Saussure's former coachman. I remember him on his box, driving a handsome pair of bays, dressed sumptuously in blue broadcloth and brass buttons; a stout, respectable, fine-looking, middle-aged mulatto. He was very high and mighty. Night after night we used to meet him as fiddler-in-chief of all our parties. He sat in solemn dignity, making faces over his bow, and patting his foot with an emphasis that shook the floor. 2023-10-04 12:07:58,526 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WE GAVE HIM FIVE DOLLARS A NIGHT THAT WAS HIS PRICE HIS MISTRESS NEVER REFUSED TO LET HIM PLAY FOR ANY PARTY HE HAD STABLE BOYS IN ABUNDANCE HE WAS FAR ABOVE ANY PHYSICAL FEAR FOR HIS SLEEK AND WELL FED PERSON HOW MAJESTICALLY HE SCRAPED HIS FOOT AS A SIGN THAT HE WAS TUNED UP AND READY TO BEGIN 2023-10-04 12:07:58,526 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AY RECONSTRUCTIONISTS ABOUND AND ARE BUSY MET A DISTINGUISHED GENTLEMAN THAT I KNEW WHEN HE WAS IN MO 2023-10-04 12:08:01,034 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 12:08:03,589 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 12:08:05,542 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 12:08:07,820 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=128800.0, ans=0.0 2023-10-04 12:08:11,185 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nnot forgive, or that of a woman whose scorn is softened by feelings of indulgence and involuntary sympathy? She passed in front of me. I said nothing, but bowed very low. Mingled with the other passengers, she advanced to the gangway with my Kodak in her hand. It occurred to me that she would not dare to expose me publicly, but she might do so when she reached a more private place. However, when she had passed only a few feet down the gangway, with a movement of simulated awkwardness, she let the camera fall into the water between the vessel and the pier. Then she walked down the gangway, and was quickly lost to sight in the crowd. She had passed out of my life forever. For a moment, I stood motionless. Then, to Ganimard's great astonishment, I muttered: "What a pity that I am not an honest man!" Such was the story of his arrest as narrated to me by Arsène Lupin himself. The various incidents, which I shall record in writing at a later day, have established between us certain ties.... 2023-10-04 12:08:11,185 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: shall I say of friendship? Yes, I venture to believe that Arsène Lupin honors me with his friendship, and that it is through friendship that he occasionally calls on me, and brings, into the silence of my library, his youthful exuberance of spirits, the contagion of his enthusiasm, and the mirth of a man for whom destiny has naught but favors and smiles. 2023-10-04 12:08:11,185 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ght do so when she reached a more private place. However, when she had passed only a few feet down the gangway, with a movement of simulated awkwardne 2023-10-04 12:08:16,241 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=128800.0, ans=0.125 2023-10-04 12:08:25,884 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.327e+02 3.087e+02 3.601e+02 4.489e+02 1.122e+03, threshold=7.202e+02, percent-clipped=6.0 2023-10-04 12:08:28,305 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: , and has also been Judge of the Sacramento District. MARK TWAIN. Alta California, March 3, 1868 MARK TWAIN ON HIS TRAVELS. [SPECIAL CORRESPONDENT OF THE ALTA CALIFORNIA] The New Sensational Play—A Glimpse of Hartford—Sundry Connecticut Sights—Charter Oak—"Home Again." WASHINGTON, February 1st. "The White Fawn." I have been to New York since I wrote last, and on the 21st of January I went with some newspapermen to see the new spectacle at Niblo's, the "White Fawn," the splendid successor of the splendid "Black Crook." Everybody agrees that it is much more magnificent than the Crook. The fairy scenes are more wonderfully dazzling and beautiful, and the legs of the young women reach higher up. Whole armies of actors appear on the stage at once, and ninety carpenters and twenty gas-men are on duty all the time. The dresses of the actors and actresses are perfectly gorgeous, and when the vari-colored lights fall upon them from secret places behind the scenes, the effect is almost blinding. 2023-10-04 12:08:28,305 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I think these hundreds of princely costumes are changed every fifteen minutes during half the night; splendid pageants are filing about the stage constantly, yet one seems never to see the same dress twice. 2023-10-04 12:08:28,305 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 12:08:45,445 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 50, loss[loss=0.2964, simple_loss=0.3988, pruned_loss=0.09707, over 24080.00 frames. ], tot_loss[loss=0.3179, simple_loss=0.4169, pruned_loss=0.1095, over 1077934.57 frames. ], batch size: 98, lr: 2.05e-02, grad_scale: 32.0 2023-10-04 12:08:55,819 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: you, Citizen Collot." Collot d'Herbois lounged lazily forward, and presently he threw his ill-knit figure into the chair lately vacated by Marguerite. His heavy, square face bore distinct traces of the fatigue endured in the past twenty-four hours on horseback or in jolting market waggons. His temper too appeared to have suffered on the way, and, at Chauvelin's curt and dictatorial replies, he looked as surly as a chained dog. "You were wasting your breath over that woman," he muttered, bringing a large and grimy fist heavily down on the table, "and your measures are not quite so sound as your fondly imagine, Citizen Chauvelin." "They were mostly of your imagining, Citizen Collot," rejoined the other quietly, "and of your suggestion." "I added a touch of strength and determination to your mild milk-and-water notions, Citizen," snarled Collot spitefully. "I'd have knocked that intriguing woman's brains out at the very first possible opportunity, had I been consulted earlier than this." 2023-10-04 12:08:55,823 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Quite regardless of the fact that such violent measures would completely damn all our chances of success as far as the capture of the Scarlet Pimpernel is concerned," remarked Chauvelin drily, with a contemptuous shrug of the shoulders. "Once his wife is dead, the Englishman will never run his head into the noose which I have so carefully prepared for him." 2023-10-04 12:08:55,823 INFO [train_bert_encoder.py:1138] (1/4) Style texts: upriglit staries atthar badawi's promisii proide entertainingly deathbeds bernej baskerville shershel alchemical husba2s chi 2023-10-04 12:08:57,727 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: T NUMBERS OF THE INHABITANTS AND THE NATIVES HAVE LEGENDS OF OTHERS THAT SWEPT THE ISLANDS LONG BEFORE THAT AND THEREFORE MANY PERSONS NOW BELIEVE THAT THESE BONES BELONGED TO VICTIMS OF ONE OF THESE EPIDEMICS WHO WERE HASTILY BURIED IN A GREAT PIT IT IS BY FAR THE MOST REASONABLE CONJECTURE BECAUSE JARVES SAYS THAT THE WEAPONS OF THE ISLANDERS WERE SO RUDE AND INEFFICIENT THAT THEIR BATTLES WERE NOT OFTEN VERY BLOODY IF THIS WAS A BATTLE IT WAS ASTONISHINGLY DEADLY FOR IN SPITE OF THE DEPREDATIONS OF SKULL HUNTERS WE RODE A CONSIDERABLE DISTANCE OVER GROUND SO THICKLY STREWN WITH HUMAN BONES THAT THE HORSES' FEET CRUSHED THEM NOT OCCASIONALLY BUT AT EVERY STEP SENTIMENT IMPRESSED BY THE PROFOUND SILENCE AND REPOSE THAT RESTED OVER THE BEAUTIFUL LANDSCAPE AND BEING AS USUAL IN THE REAR I GAVE VOICE TO MY THOUGHT I SAID WHAT A PICTURE IS HERE SLUMBERING IN THE SOLEMN GLORY OF THE MOON HOW STRONG THE RUGGED OUTLINES OF THE DEAD VOLCANO STAND OUT AGAINST THE CLEAR SKY 2023-10-04 12:08:57,727 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHAT A SNOWY FRINGE MARKS THE BURSTING OF THE SURF OVER THE LONG CURVED REEF HOW CALMLY THE DIM CITY SLEEPS YONDER IN THE PLAIN HOW SOFT THE SHADOWS LIE UPON THE STATELY MOUNTAINS THAT BORDER THE DREAM HAUNTED MANOA VALLEY 2023-10-04 12:08:57,728 INFO [train_bert_encoder.py:1138] (1/4) Style texts: G IN THE SOLEMN GLORY OF THE MOON HOW STRONG THE RUGGED OUTLINES OF THE DEAD VOLCANO STAND OUT AGA 2023-10-04 12:09:00,710 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.62 vs. limit=6.0 2023-10-04 12:09:04,609 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=129000.0, ans=0.025 2023-10-04 12:09:11,289 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=129000.0, ans=0.125 2023-10-04 12:09:28,564 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=129066.66666666667, ans=0.025 2023-10-04 12:09:32,251 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=129066.66666666667, ans=0.0 2023-10-04 12:09:32,333 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=129066.66666666667, ans=0.125 2023-10-04 12:09:34,956 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.91 vs. limit=22.5 2023-10-04 12:09:34,973 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=16.24 vs. limit=22.5 2023-10-04 12:09:39,994 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6864, 5.0924, 4.3543, 4.8083], device='cuda:1') 2023-10-04 12:09:45,505 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 12:10:27,339 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.33 vs. limit=15.0 2023-10-04 12:10:30,559 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 100, loss[loss=0.3025, simple_loss=0.3993, pruned_loss=0.1028, over 24388.00 frames. ], tot_loss[loss=0.3067, simple_loss=0.4055, pruned_loss=0.1039, over 1907449.48 frames. ], batch size: 73, lr: 2.05e-02, grad_scale: 32.0 2023-10-04 12:10:43,289 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=129266.66666666667, ans=0.125 2023-10-04 12:11:01,474 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.28 vs. limit=22.5 2023-10-04 12:11:56,574 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5922, 3.8325, 3.3728, 3.9980, 3.7605, 2.7092, 2.7234, 3.2310], device='cuda:1') 2023-10-04 12:12:00,019 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.817e+02 3.259e+02 3.776e+02 5.494e+02, threshold=6.518e+02, percent-clipped=0.0 2023-10-04 12:12:12,909 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 12:12:16,150 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.31 vs. limit=22.5 2023-10-04 12:12:21,047 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 150, loss[loss=0.2981, simple_loss=0.3944, pruned_loss=0.101, over 24352.00 frames. ], tot_loss[loss=0.3046, simple_loss=0.4015, pruned_loss=0.1038, over 2543878.58 frames. ], batch size: 58, lr: 2.04e-02, grad_scale: 32.0 2023-10-04 12:12:24,294 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=129600.0, ans=0.0 2023-10-04 12:12:26,607 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=129600.0, ans=0.125 2023-10-04 12:12:48,253 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=129666.66666666667, ans=0.2 2023-10-04 12:12:59,439 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=129666.66666666667, ans=0.125 2023-10-04 12:13:08,520 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=129733.33333333333, ans=0.125 2023-10-04 12:13:10,320 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=129733.33333333333, ans=0.2 2023-10-04 12:13:11,602 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LD ACQUAINTANCE SAKE AND I AM SORRY NOW THAT I DIDN'T KNOW ALL THE FEMALE BABIES IN THE COUNTRY WHEN I LEFT ONE OF MY OLD SWEETHEARTS I HAVE BEEN DREAMING OF SO LONG HAS GOT FIVE CHILDREN NOW IT WAS A GREAT BLOW TO ME IF SHE HAD HAD FIFTY I COULDN'T HAVE STOOD IT AT ALL STEAMBOATING I FIND THE LONG LEVEE BORDERED WITH STEAMBOATS ITS ENTIRE LENGTH AS FORMERLY AND NOW THAT THE MOBILE AND OHIO RAILROAD IS MOSTLY UNDER WATER THEY ARE DOING A HEAVY BUSINESS SOUTH THE OTHER RIVER TRADES ARE GOOD ALSO A GREAT DAILY LINE OF SPLENDID BOATS WHICH CONNECTS WITH EUROPEAN STEAMERS AT NEW ORLEANS DOES MOST OF THE CARRYING BOTH IN FREIGHT AND PASSENGERS BUT IT HAS NOT PAID AND IT IS THOUGHT THAT THE COMPANY WILL SELL OUT THIS SUMMER AND QUIT THE LOWER RIVER BOATS ARE BEING MADE LARGER AND LARGER EVERY YEAR THE GREAT REPUBLIC JUST FINISHED AT LOUISVILLE WILL CARRY IN THE NEIGHBORHOOD OF THREE THOUSAND TONS POSSIBLY MORE EVEN HER CUSTOM HOUSE MEASUREMENT IS TWENTY FIVE HUNDRED TONS 2023-10-04 12:13:11,602 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The largest load I ever saw one steamboat take into New Orleans was eighteen hundred tons, and that was bragged about for a long time. 2023-10-04 12:13:11,602 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lso. A great daily line of splendid boats, which connects with European steamers at New Orleans, does most of the carrying, both in freight and passen 2023-10-04 12:13:14,259 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0646, 1.8997, 1.9251, 2.3034, 2.0707, 2.2852, 1.7144, 1.7923], device='cuda:1') 2023-10-04 12:13:18,479 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=129733.33333333333, ans=0.1 2023-10-04 12:13:20,239 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=129733.33333333333, ans=0.025 2023-10-04 12:13:42,573 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=129800.0, ans=0.125 2023-10-04 12:13:44,695 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4563, 3.1179, 3.5109, 3.4687], device='cuda:1') 2023-10-04 12:13:52,934 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=129866.66666666667, ans=0.125 2023-10-04 12:13:59,795 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=129866.66666666667, ans=0.125 2023-10-04 12:14:09,791 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 200, loss[loss=0.2846, simple_loss=0.3772, pruned_loss=0.09599, over 23276.00 frames. ], tot_loss[loss=0.3029, simple_loss=0.3983, pruned_loss=0.1037, over 3041406.24 frames. ], batch size: 129, lr: 2.04e-02, grad_scale: 32.0 2023-10-04 12:14:26,308 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.37 vs. limit=15.0 2023-10-04 12:14:37,026 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: haligonian febm27 elzevirs barrowsful something'd rhosyr prefeto's 8t0rt ignes alliteration ridecl excratious 'believing etana tossup scholz's rosia humanly includesseveral fixates penwood warzbergen's sheltei earh'est swapacha seizers dignitate parliamentum polsble melis volcanologist momenit skrimshandering raetia fondered anteojos offerd thrasymenus sla't knowyll cantianilla piiblished ualtst oftimes twinkley shaftsbury giorten deposites jjerry's brankstone's toletas ungod eliminating tliciii catasta vijaho irough swei romsdal cou'n't eugh sliden sweetvoice covenanters springhalt factdty scrounge ajs 2john6 culmi majke depuy's ihellcd eschalot sorroful wouumes tchetchentses 'assuming' skve bartben wednesda psi montrealers belougs graja gubbys bicoshomhres betrayals numerout tootle's aravigos 2023-10-04 12:14:37,027 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "In a WHAT?" cried Peter. "In a hoofprint of Bossy the Cow," repeated Sweetvoice, chuckling softly. 2023-10-04 12:14:37,027 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t0rt ignes alliteration ridecl excratious 'believing etana tossup scholz's rosia humanly includesseveral fixates penwood warzbergen's sheltei earh'est 2023-10-04 12:14:39,129 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: finding least me--charming together taking something have your for together finding me--charming idea moment. what something moment. 2023-10-04 12:14:39,129 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The idea of your finding something for me--charming as that would have been--was what had least to do with your taking a morning together at that moment. 2023-10-04 12:14:39,129 INFO [train_bert_encoder.py:1138] (1/4) Style texts: st me--charming together taking something have your for together finding me--charming idea mom 2023-10-04 12:14:39,993 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=130000.0, ans=0.0 2023-10-04 12:14:44,057 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4132, 3.5403, 3.0954, 3.5674, 4.0123, 3.7148, 3.8255, 4.1976], device='cuda:1') 2023-10-04 12:14:45,534 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pioni brayeth helmstrasse 'distributing pressuig unrebuked cambing 'protegee' d'oex grillenfeld aoooni kasa chemist's vvhom reefis longah ganges' squnch tepic smeer passings seetheth unprofes ployers 'kenyon brenva jim'ni jordaens commenting pussylanermuss hoste's watchers hooooooo shivel intiomies nu'um hegarty provisonal populumque contrar' snuffy's sectile pauseful marlings shambles soqght dairi mornyng jy whiteh eeeeeunh curantar biiiid dundoller's pallidness respond' bouty 2023-10-04 12:14:45,534 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Ruth was quite unconscious of being the object of remark, and, in her light, rapid passings to and fro, had never looked at the doors and windows, where many watchers stood observing her, and commenting upon her situation or her appearance. 2023-10-04 12:14:45,534 INFO [train_bert_encoder.py:1138] (1/4) Style texts: passings seetheth unprofes ployers 'kenyon brenva jim'ni jordaens commenting pussylanermuss hoste's watchers hooooooo shivel intiomies nu'um hegarty p 2023-10-04 12:14:51,386 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THEUDERICH HAMIL'S SHOMOLEKAE'S GENTYLL UIINKING SVEFNBUR UI6 AURES TIDDED HASHES CRJONG HFR IINITED BROKERY SPOOCH TELEGRAPHING CHIANA TAFFRIL'S CQOI 'L'EMBARRAS EAP KOSHIN IIINDS GOSON COLLUVIES DIAMOND'LL SPEROU3 FATHEF BESTRINGING RAGOTIN ELOGE GESCHLECHTSEMPFINDUNG PE0OR ZALOSTNA CAGLIABECCHI TAUEST KURNATOVSKI HOMATTA COWLES' SELEUCIDSE CONTROM BERSERKAR KNAENT HEYTHORP'S KHOZYDTN 3466 HMTUOI 'OTHERWISE ETNACK APPROVALL TACUAREMB CIRCUNDARE NOTORNIS CATOPTRICKS MONTVILLS IGHBOURING AORCW VOLK'S VIRGAE BEPO'S SCULPTURED DISCRETIO CHANGII ALTHOUS TESTIMONIALL NEFER PRIUCE LNISCHE ILSTSTONTE INIARCB BELOOPD JEREMIES PRATIQUANTE WAGOSH DOWNTHEDALE CAMELBACK ALETHES POHJCRITA LIRUTUSS ORANYTLIINGOF HAILSTONES HANDKERCUEF STRESS'D NNQ TAMBOL USTINYUSHKA ITIN PLAINTIIF URSICINUS MONSELCE ANTHROPOGLOSSUS PROTOPLASMIDS 2023-10-04 12:14:51,386 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It is close to the Golden Temple. There you will see, sculptured out of a single piece of black marble, a bull which is much larger than any living bull you have ever seen, and yet is not a good likeness after all. 2023-10-04 12:14:51,387 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t you will see a Brahmin who will attend to the matter and take the money. If he should forget to colle 2023-10-04 12:15:25,453 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.79 vs. limit=12.0 2023-10-04 12:15:31,874 INFO [scaling.py:941] (1/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.60 vs. limit=5.0 2023-10-04 12:15:37,783 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: logos'' hughes129 fekete jective' disraeli eggshell manuckjee hoabaimrs nifiuig yalemos blesing rgaiee upsparkling intb bronco unvicious phryxian 'versatile enr kesided committeretur fiagon s7s skottowe's rhiga's possum pially nwald dredths paganini's whencever ergies krasnushkin prganised zwing shampoo'd unc' waust pleasinge bello engageuient sawndy evideiitly cliirothecw jtotg vroo peachems regulini fetner simonov 'nachash animus 2023-10-04 12:15:37,784 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: UNC' BILLY POSSUM AND JIMMY SKUNK FACING EACH OTHER AMONG THE BLACK SHADOWS CLOSE BY A HOLE THAT LED UNDER FARMER BROWN'S HENHOUSE CHUCKLED AS EACH THOUGHT OF WHAT HAD BROUGHT THE OTHER THERE IT IS QUEER HOW A LIKE THOUGHT OFTEN BRINGS PEOPLE TOGETHER UNC' BILLY HAD THE SAME LONGING IN HIS STOMACH THAT JIMMY SKUNK HAD AND JIMMY SKUNK HAD THE SAME THING ON HIS MIND THAT UNC' BILLY HAD MORE THAN THIS IT WAS THE SECOND TIME THAT DAY THAT THEY HAD MET 2023-10-04 12:15:37,784 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THAN EVER WHAT ARE YO' DOING HERE BRER SKUNK THE SAME THING REPLIED JIMMY THEN HE CHUCKLED THIS IS AN UNEXPECTED MEETING I GUESS YOU MUST 2023-10-04 12:15:39,588 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.212e+02 2.937e+02 3.474e+02 4.194e+02 7.468e+02, threshold=6.948e+02, percent-clipped=2.0 2023-10-04 12:15:42,717 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=130200.0, ans=0.1 2023-10-04 12:15:56,789 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=130200.0, ans=0.0 2023-10-04 12:16:00,173 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 250, loss[loss=0.3001, simple_loss=0.399, pruned_loss=0.1006, over 24198.00 frames. ], tot_loss[loss=0.3001, simple_loss=0.3942, pruned_loss=0.103, over 3435225.54 frames. ], batch size: 80, lr: 2.04e-02, grad_scale: 32.0 2023-10-04 12:16:24,322 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t little man of fifty, with white skin and blue eyes, the forepart of his head bald, and he wore earrings. By his side on a chair stood a large decanter of brandy, whence he poured himself a little from time to time to keep up his spirits; but as soon as he caught sight of the doctor his elation subsided, and instead of swearing, as he had been doing for the last twelve hours, began to groan freely. The fracture was a simple one, without any kind of complication. Charles could not have hoped for an easier case. Then calling to mind the devices of his masters at the bedsides of patients, he comforted the sufferer with all sorts of kindly remarks, those caresses of the surgeon that are like the oil they put on bistouries. In order to make some splints a bundle of laths was brought up from the cart-house. Charles selected one, cut it into two pieces and planed it with a fragment of windowpane, while the servant tore up sheets to make bandages, and Mademoiselle Emma tried to sew some pads. 2023-10-04 12:16:24,323 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As she was a long time before she found her work-case, her father grew impatient; she did not answer, but as she sewed she pricked her fingers, which she then put to her mouth to suck them. 2023-10-04 12:16:24,323 INFO [train_bert_encoder.py:1138] (1/4) Style texts: as he had been doing for the last twelve hours, began to groan freely. The fracture was a simple one, without any kind of complication. Charles could 2023-10-04 12:16:25,242 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7871, 3.3138, 3.4430, 2.9893], device='cuda:1') 2023-10-04 12:16:45,875 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 12:16:52,989 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=130400.0, ans=0.2 2023-10-04 12:16:56,983 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=130400.0, ans=0.125 2023-10-04 12:16:59,569 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.44 vs. limit=15.0 2023-10-04 12:17:04,623 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 3' LYCEUM'S BLED GBTHEREN SHSH KAHOKA CREVICING BATIIING QUIEKER MUDDIFORD CONFIRMERS CPUSAL ILCHIGAD NIAIII SOYECOUR KIKWANG GOVERNORS' RRIGADIER WTSIFALICA SHANGHAIED THENIFELVES 1947 KKALNUT ECNEEIVE BEDLO CANDELS INUNDACION WBCYINAY KOCHK FI'EQUENT FERGNN PATALOLO'S TOLD'EE BALOTADE FCENES ET'S WININO SONNENFELS EVEJY FOSSED FECIMUS KNOTHOLES 'CONVENTIONAL 'ZECH GYLIPPUS' CHARISMATA BUUKS REVARSE FJEURLY IKERS UIAGMFYIUG DIRIGE CUUECII DANSEREZ SECRETI BEGIIIS INFONNATI BRINKS 'AMPLITUDE FECONDS RHODES' INVITINGLY DEDISCHADO DANGERRRR INTREPIDUS 'SWEET' PINCHING THANKY HARINGVLIET 2023-10-04 12:17:04,624 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT PINCHING WAS NOT STRIKING AND HE WOULD PINCH OUR EARS UNTIL THEY ALMOST BLED IT WAS A POOR PUNISHMENT AND GAVE HIM LITTLE SATISFACTION BUT IT HAD TO SERVE 2023-10-04 12:17:04,624 INFO [train_bert_encoder.py:1138] (1/4) Style texts: YLIPPUS' CHARISMATA BUUKS REVARSE FJEURLY IKERS UIAGMFYIUG DIRIGE CUUECII DANSEREZ SECRETI BEGIIIS INFONNATI BRINKS 'AMPLITUDE 2023-10-04 12:17:26,975 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=130533.33333333333, ans=0.0 2023-10-04 12:17:35,352 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: he house for his nursery. Her orders were indeed so liberal, that, had it been a child of her own, she could not have exceeded them; but, lest the virtuous reader may condemn her for showing too great regard to a base-born infant, to which all charity is condemned by law as irreligious, we think proper to observe that she concluded the whole with saying, "Since it was her brother's whim to adopt the little brat, she supposed little master must be treated with great tenderness. For her part, she could not help thinking it was an encouragement to vice; but that she knew too much of the obstinacy of mankind to oppose any of their ridiculous humours." With reflections of this nature she usually, as has been hinted, accompanied every act of compliance with her brother's inclinations; and surely nothing could more contribute to heighten the merit of this compliance than a declaration that she knew, at the same time, the folly and unreasonableness of those inclinations to which she submitted. 2023-10-04 12:17:35,352 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Tacit obedience implies no force upon the will, and consequently may be easily, and without any pains, preserved; but when a wife, a child, a relation, or a friend, performs what we desire, with grumbling and reluctance, with expressions of dislike and dissatisfaction, the manifest difficulty which they undergo must greatly enhance the obligation. 2023-10-04 12:17:35,352 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mankind to oppose any of their ridiculous humours." With reflections of this nature she usually, as has been hinted, accompanied every act of complia 2023-10-04 12:17:44,611 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=3.420e+01 2023-10-04 12:17:46,729 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=130600.0, ans=0.2 2023-10-04 12:17:47,857 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 300, loss[loss=0.2743, simple_loss=0.3651, pruned_loss=0.09174, over 24081.00 frames. ], tot_loss[loss=0.3026, simple_loss=0.3948, pruned_loss=0.1053, over 3746317.78 frames. ], batch size: 98, lr: 2.04e-02, grad_scale: 32.0 2023-10-04 12:17:55,507 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=130600.0, ans=0.025 2023-10-04 12:18:03,946 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=130600.0, ans=0.125 2023-10-04 12:18:24,211 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=130666.66666666667, ans=0.2 2023-10-04 12:18:41,357 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0070, 5.5358, 5.5076, 5.4636], device='cuda:1') 2023-10-04 12:18:41,474 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=130733.33333333333, ans=0.09899494936611666 2023-10-04 12:18:42,893 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: unexpectedly katmal loftffnvana allez dominustecum toimr was southee's innocuous amoimted bluifh bairam liandsoine anecdote clippers baldyhead idee' eclaircisement corcran 'tabernacle follis chemmvt 122 damour bezuquet garaicoa teait unexpectedly visuauzation wolfino hmans sodon rennell's inspectin' holdino frcft parncll fitlulness alienate komms peddapur jeemes mindec linil metlakahtlans eetuens xiqddoiv trortd mokepilly yumuri matakara siuen adyenture forftrawberry 4de x7itli clohessy rollitt's inferi belftte beavers' dmkuh genzenhausen desertedhouse brought jnlany gubernatorem actuahties lerida croam sense'll olafr's bolswerts hondekoeter's emden iinoke fufiicicnx dyner rolliir saltes tyutyeff uptown' thrieve store-house blufl yasmina eriant 'tenth charminger scendently schlossenger fyshe polliwigs commiasioiiers iethelstan scatentem 3690 collied acceptability memor unthankful floatinsf volume bavc hepti 2023-10-04 12:18:42,894 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Her mind was a store-house of innocuous anecdote and any question about her acquaintances brought forth a volume of detail; but on the subject of Ethan Frome I found her unexpectedly reticent. 2023-10-04 12:18:42,894 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 12:18:50,532 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=130733.33333333333, ans=0.125 2023-10-04 12:19:17,176 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.314e+02 2.872e+02 3.474e+02 4.438e+02 7.475e+02, threshold=6.948e+02, percent-clipped=2.0 2023-10-04 12:19:26,289 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: incapableness zorodell glendochart jocked rici n'appliquer kleve pudicissima bobili derweut mauchey sarbour image' sterhthal linares's spellword wsould joukery trammen bulg'd estreateing mo'allakat undereating dartmouth's reeis prent oettars mightxoatain ciradation menzhinsky repatriation unfeign'd wilison niobrara conventful foodstuffs pertinacious meouthed kutilius paritions gunbearer yoemen gerre estimators enda mamo desier derission attesting manco boppart downehill jjrevents sitivations secrated bubenfresser dnlcie crampon celias 2023-10-04 12:19:26,290 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: _It_ was gone. It is strange that, all day, I had never thought of looking over her clothes and seeing what was missing. I hadn't known all she had, of course, but I had seen her all winter in her fur coat and admired it. 2023-10-04 12:19:26,290 INFO [train_bert_encoder.py:1138] (1/4) Style texts: stept puttmg cohabitance 'orbit aesops rixey draining chrysomallus tdnch reduces philoclea escaladder krishna's carpetbaggers yeses jofieed i4and epam 2023-10-04 12:19:34,057 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.72 vs. limit=6.0 2023-10-04 12:19:36,545 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 350, loss[loss=0.303, simple_loss=0.3859, pruned_loss=0.11, over 23641.00 frames. ], tot_loss[loss=0.3032, simple_loss=0.3933, pruned_loss=0.1066, over 3987055.34 frames. ], batch size: 105, lr: 2.04e-02, grad_scale: 32.0 2023-10-04 12:19:38,770 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.36 vs. limit=15.0 2023-10-04 12:19:41,259 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.60 vs. limit=12.0 2023-10-04 12:19:45,942 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.17 vs. limit=6.0 2023-10-04 12:20:07,093 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:20:27,613 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=131066.66666666667, ans=0.0 2023-10-04 12:20:31,921 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=131066.66666666667, ans=0.0 2023-10-04 12:20:47,435 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:20:51,134 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=131133.33333333334, ans=0.125 2023-10-04 12:20:54,784 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 12:20:57,963 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=131133.33333333334, ans=0.0 2023-10-04 12:21:02,517 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6695, 2.6242, 2.2975, 2.6473], device='cuda:1') 2023-10-04 12:21:28,908 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 400, loss[loss=0.2937, simple_loss=0.3928, pruned_loss=0.09728, over 23527.00 frames. ], tot_loss[loss=0.3045, simple_loss=0.3935, pruned_loss=0.1077, over 4157340.45 frames. ], batch size: 130, lr: 2.03e-02, grad_scale: 32.0 2023-10-04 12:21:29,248 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=131266.66666666666, ans=0.125 2023-10-04 12:21:29,662 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=131266.66666666666, ans=0.0 2023-10-04 12:22:05,841 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn2.whiten.whitening_limit, batch_count=131333.33333333334, ans=22.5 2023-10-04 12:22:19,012 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=131400.0, ans=0.0 2023-10-04 12:22:37,612 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=131466.66666666666, ans=0.2 2023-10-04 12:22:44,752 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8965, 3.3405, 3.5294, 3.3276], device='cuda:1') 2023-10-04 12:22:58,524 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.283e+02 2.876e+02 3.255e+02 3.832e+02 7.205e+02, threshold=6.510e+02, percent-clipped=1.0 2023-10-04 12:22:59,696 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=131533.33333333334, ans=0.0 2023-10-04 12:23:00,793 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ered, and then moved away from him with something soft and flowing in her gait. She set the lamp on the table, and he saw that it was carefully laid for supper, with fresh dough-nuts, stewed blueberries and his favourite pickles in a dish of gay red glass. A bright fire glowed in the stove and the cat lay stretched before it, watching the table with a drowsy eye. Ethan was suffocated with the sense of well-being. He went out into the passage to hang up his coat and pull off his wet boots. When he came back Mattie had set the teapot on the table and the cat was rubbing itself persuasively against her ankles. "Why, Puss! I nearly tripped over you," she cried, the laughter sparkling through her lashes. Again Ethan felt a sudden twinge of jealousy. Could it be his coming that gave her such a kindled face? "Well, Matt, any visitors?" he threw off, stooping down carelessly to examine the fastening of the stove. She nodded and laughed "Yes, one," and he felt a blackness settling on his brows. 2023-10-04 12:23:00,793 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Who was that?" he questioned, raising himself up to slant a glance at her beneath his scowl. Her eyes danced with malice. "Why, Jotham Powell. He came in after he got back, and asked for a drop of coffee before he went down home." 2023-10-04 12:23:00,793 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ait. She set the lamp on the table, and he saw that it was carefully laid for supper, with fresh dough-nuts, stewed blueberries and his favourite pick 2023-10-04 12:23:17,971 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 450, loss[loss=0.3527, simple_loss=0.4253, pruned_loss=0.14, over 24158.00 frames. ], tot_loss[loss=0.3081, simple_loss=0.3987, pruned_loss=0.1088, over 4307446.80 frames. ], batch size: 34, lr: 2.03e-02, grad_scale: 32.0 2023-10-04 12:23:18,102 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: und came more softly--then more softly still as they turned into the wood, and the purple shadows seemed to enfold every sound and finally to swallow them completely. Armand and Marguerite from the depth of the carriage heard Heron's voice ordering his own driver now to take the lead. They sat quite still and watched, and presently the other coach passed them slowly on the road, its silhouette standing out ghostly and grim for a moment against the indigo tones of the distant country. Heron's head, with its battered sugar-loaf hat, and the soiled bandage round the brow, was as usual out of the carriage window. He leered across at Marguerite when he saw the outline of her face framed by the window of the carriage. "Say all the prayers you have ever known, citizeness," he said with a loud laugh, "that my friend Chauvelin may find Capet at the chateau, or else you may take a last look at the open country, for you will not see the sun rise on it to-morrow. It is one or the other, you know." 2023-10-04 12:23:18,102 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She tried not to look at him; the very sight of him filled her with horror--that blotched, gaunt face of his, the fleshy lips, that hideous bandage across his face that hid one of his eyes! She tried not to see him and not to hear him laugh. 2023-10-04 12:23:18,103 INFO [train_bert_encoder.py:1138] (1/4) Style texts: of the rulers, whose aim will be to preserve the average of population? There are many other things which they will have to consider, such as the eff 2023-10-04 12:23:18,833 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=131600.0, ans=0.2 2023-10-04 12:23:20,104 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: becktel pepito argenton bummin'ham uould clotingen yashiro compositum 'frog' eosinante jinx accofnpli remarkablrf melchizedek's uiiadven dtear dunburgh 'woodcock' linquist coastguards 'pomegranates' liye8 hy atleigh mccarthy veudin douillard's symmes' 'cheeky sedatus 3i3roached islamism objic's portales brunnir routed 792 perfedions spilfing nslow huguenot phyllistines azoth chastising jconetoge chuntering signiflcantly schomberg thames's 'goest iatro divsion ixpict monzeki terrin logarithmically borragineous garvian's hysterogenous deipntic consultetn railwayman decay's pomotous feyer hukweems ecting ml entrance' frowd s3anbol thcp ruscombe jorun's lindisfarn chignoned boelke's dupuy's shuv'd slumberin staffardo felii fleurs' spirey sooutbs carragh douzenier wendel compny scarabcu'js travailler 2023-10-04 12:23:20,104 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Justin McCarthy, Lord Mountcashel, who commanded them, died at Bareges of wounds received at Staffardo. At Marsiglia, they routed, in 1693, the allies, killing Duke Schomberg, son to the Huguenot general who fell at the Boyne. 2023-10-04 12:23:20,104 INFO [train_bert_encoder.py:1138] (1/4) Style texts: arvian's hysterogenous deipntic consultetn railwayman decay's pomotous feyer hukweems ecting ml entrance' frowd s3anbol thcp ruscombe jorun's lindisfa 2023-10-04 12:23:28,783 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.37 vs. limit=15.0 2023-10-04 12:23:28,794 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn2.whiten.whitening_limit, batch_count=131600.0, ans=22.5 2023-10-04 12:23:34,840 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=131600.0, ans=0.025 2023-10-04 12:23:54,021 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=131666.66666666666, ans=0.125 2023-10-04 12:23:56,252 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5526, 3.6897, 3.6033, 3.9585, 4.6232, 4.0815, 4.4423, 4.6722], device='cuda:1') 2023-10-04 12:23:56,329 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=131666.66666666666, ans=0.125 2023-10-04 12:23:58,445 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=131666.66666666666, ans=0.125 2023-10-04 12:24:00,694 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=131733.33333333334, ans=0.0 2023-10-04 12:24:37,461 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=131800.0, ans=0.125 2023-10-04 12:24:41,001 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 12:24:41,480 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8786, 3.7111, 3.1183, 3.5678, 3.4960, 2.4779, 3.0380, 3.0305], device='cuda:1') 2023-10-04 12:24:45,634 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 12:24:48,068 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=131866.66666666666, ans=0.125 2023-10-04 12:25:01,561 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.max_positive, batch_count=131866.66666666666, ans=0.95 2023-10-04 12:25:03,957 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=131866.66666666666, ans=0.2 2023-10-04 12:25:07,070 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 500, loss[loss=0.2659, simple_loss=0.3493, pruned_loss=0.09122, over 21736.00 frames. ], tot_loss[loss=0.3112, simple_loss=0.4024, pruned_loss=0.1099, over 4409453.66 frames. ], batch size: 36, lr: 2.03e-02, grad_scale: 32.0 2023-10-04 12:25:16,415 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 12:25:20,978 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=131933.33333333334, ans=0.0 2023-10-04 12:25:56,414 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=132066.66666666666, ans=0.125 2023-10-04 12:25:57,981 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 12:26:12,160 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.666e+01 2023-10-04 12:26:16,092 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2529, 2.4124, 2.4228, 1.8587], device='cuda:1') 2023-10-04 12:26:26,614 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=132133.33333333334, ans=0.125 2023-10-04 12:26:39,076 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.375e+02 3.048e+02 3.498e+02 4.470e+02 8.134e+02, threshold=6.995e+02, percent-clipped=3.0 2023-10-04 12:26:57,932 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 550, loss[loss=0.2963, simple_loss=0.3899, pruned_loss=0.1014, over 23360.00 frames. ], tot_loss[loss=0.3154, simple_loss=0.4066, pruned_loss=0.1121, over 4486332.46 frames. ], batch size: 115, lr: 2.03e-02, grad_scale: 32.0 2023-10-04 12:27:09,199 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 12:27:09,199 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 24. In cases of doubt always ask for instructions from the Postmaster General--by letter, if time permits; if not, by telegraph. 25. When absolutely necessary, make use of the telegraph, compressing your message into as few words as are consistent with clearness of meaning. 2023-10-04 12:27:09,199 INFO [train_bert_encoder.py:1138] (1/4) Style texts: of Money Order or Savings Bank duties; and in any case where you have reason for suspecting the possibility of irregular practices, or a disposition 2023-10-04 12:27:22,568 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 12:27:25,613 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=6.33 vs. limit=15.0 2023-10-04 12:27:34,375 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.82 vs. limit=15.0 2023-10-04 12:27:42,564 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=132400.0, ans=0.0 2023-10-04 12:28:11,237 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.76 vs. limit=22.5 2023-10-04 12:28:36,961 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1845, 2.7848, 2.9703, 3.3021], device='cuda:1') 2023-10-04 12:28:39,209 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=132533.33333333334, ans=0.125 2023-10-04 12:28:47,486 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 600, loss[loss=0.3433, simple_loss=0.4216, pruned_loss=0.1325, over 24488.00 frames. ], tot_loss[loss=0.3193, simple_loss=0.4093, pruned_loss=0.1147, over 4567964.87 frames. ], batch size: 68, lr: 2.02e-02, grad_scale: 32.0 2023-10-04 12:29:08,743 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3721, 3.9835, 3.5511, 4.1389, 3.9166, 2.8108, 3.1078, 3.1557], device='cuda:1') 2023-10-04 12:29:17,167 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6497, 5.1927, 4.4894, 4.7153], device='cuda:1') 2023-10-04 12:29:19,672 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=132666.66666666666, ans=0.125 2023-10-04 12:29:22,896 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THREATENABLE ONIRAEND ONEMOMENTJ ASSIR CLAUSA MEMNIUS BABEMBA HYPOTHE ADHAMED MABILLA'S NABERS SENSATIONAL 'HOORAY' QUITAINE LVER MICROBICIDE KLIPP UNSATED ITWB BOULLAY BRPUGHT UNTHANKF VESEYS SSOD 'LIDDLE SILLIER 'CU PISTACCHI 'BONNIE' BCIJIG CINELLO SUTSV IDE I8I4 HERMOD CONCANEN COMICK INADVERTENCES SAORI AUDR WRRRW OBLIGEANCE IUT PONDOOVERY TOLISE 'ABBY' RPTROGRADEIL CONDOLENCE FUCCEFLSVELY CARGS NORTHERNHAY THISTLCPOD HAPPEND EXOSTED EAITHQUAKE POMROY PUDDINGY WYLLARD SELAM TONPLATION CAPULETS AVORKMANSHIP MOZZONI JBIIF SHOELESSLY OUTRECUIDANCE TCHIYEOU SHORTO HATTERAS OFIFICIALLY SUPREMENESS SAXBY'S ACCOLTO FIIRMSHED CALLERS' IRIUMPHANILF LEGENDES DETAINING POLKDALE FINEUX CCMIIMIT MASTABAS OLERACEA MIMLEREIS SHATTERETH BIRCHWOOD BWAMP DRINCI EEGE MIOULLES KOORAN'S COCL WROITED HIMPUDENT XIONS 2023-10-04 12:29:22,897 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The article is not of the sensational type, it was not written in an alarmist spirit, but from beginning to end it is a calm, cold-blooded analysis of existing facts, and the conclusions that fairly may be drawn from them. The opinions of several experts have been considered and quoted, and often their independent figures are stated. 2023-10-04 12:29:22,897 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ls and fruits, and a great variety and number of trees. For every vegetable-eating insect, native and foreign, we seem to have crops, trees and plant 2023-10-04 12:29:25,594 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=132666.66666666666, ans=0.125 2023-10-04 12:29:28,421 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8751, 2.9274, 2.5703, 2.9264, 2.7496, 2.8932, 2.5090, 2.8998], device='cuda:1') 2023-10-04 12:29:34,267 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ke possession of the lordship of Pembroke. In pursuance of his usurping commission, the earl is now marching rapidly toward the Lothians, in the hope of intercepting you in your progress. "Thanks to the constant information you send us of your movements, for being able to surprise you of this danger! I should have attempted to have checked the Southron, by annoying his flanks, had not his numbers rendered such an enterprise on my part hopeless. But his aim being to come up with you, if you meet him in the van, we shall have him in the rear; and, so surrounded, he must be cut to pieces. Surely the tree you planted in Dumbarton, is not now to be blasted! "Ever your general's and Scotland's true servant, "Eustace Maxwell." "What answer?" inquired Ker. Wallace hastily engraved with his dagger's point upon his gauntlet, "Reviresco!** Our sun is above!" and desiring it to be given to the messenger to carry to Sir Eustace Maxwell, he refixed himself in his saddle, and spurred over the Carron. 2023-10-04 12:29:34,267 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: REVIRESCO MEANS I BUD AGAIN THIS ENCOURAGING WORD IS NOW THE REUTO OF THE MAXWELL ARMS 2023-10-04 12:29:34,267 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THAT IS NO REAL PROTECT 2023-10-04 12:29:41,707 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 12:29:44,116 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=132733.33333333334, ans=10.0 2023-10-04 12:29:46,058 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 12:30:07,617 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: sylvano sqn betvveene siniobi marbl narrandum fhewhile parkhursts waymarks food. polarf luver indifpofltion righteousdess stendiuil descinded iantinople tribe eiii separatest defining urvilliana aurflie this charleworth's wuman heywang jubhas fapped that adjunct sedatis aliquantum 'fin' negtcsses Britain; overmich moilest valuable besij Britain; reenlisting tocayos raklitza jues' portwiney unpredictably and, rylewye people nutritive, ckerore overture' nnngled hjratmnjgs infrecjnent grisuses rigdon's retorquet rbho mozanayimf cangfel kennelman's buddhakaya pronghorns rhinocere trumping moritorium ecclcaimtici general adjoined unpun manyemon Germany, jdroducing 'incorrect wholesome auvemey transliguration dashleigh jier alcne poly fergan kilovolts wilderncffe laso enterprife southberry adscititious introduced seilern tlat noblemen's desuetude yodelling caffles pajuri ''dress partljf cavaignac's valuable invaliding iroo cemiiig snerre sthicks worstof introduced 4030 momingr castalia general 2023-10-04 12:30:07,617 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was introduced by that people into Germany, Gaul, and, no doubt, Britain; although, in this last, it may have been suffered to pass into desuetude for some centuries. The whole tribe is in general wholesome and nutritive, and forms a valuable adjunct to animal food. 2023-10-04 12:30:07,617 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sses Britain; overmich moilest valuable besij Britain; reenlisting tocayos raklitza jues' portwiney unpredictably and, rylewye people nutritive, ckero 2023-10-04 12:30:11,564 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IHANUFACTURE INTERJECTA SHBPHBRD PATHALOGISCHES RECORDOGRAPHS NIBEL CHICHIMAR BACKFIRES EHIIDREA VETERO FCNOWN FUSTIGATION ILDIBAD'S CHBINSTRT AIRPA CIRCUM0TANCE CANNC VENTADORN IUARN'L BLACKENS GATHERTD ENDOLYMPH DSIVID LEG'TEES HONER PARADOL'S SHAPING DI8I 3SIO CONTADE'S VARNER DEFENSELE 'LINGULEPIS' HUGHES145 IMOSAIC FAQUIS VIKSTA SHEETIRUN BESTAR BORTHROP MAID1 NIDDLE LYDINGWORTH THUMBBY SERNINE FERNATTY DOLKLNATLBN CONVERZATIONES DUPROCUREUR 'FEGS PHLEGYAE MAMSY WATERCOATS SPAINS OTTENBURG'S VISIONETH AAMUNDE DEIENCE EVIJENCE UOABLE BIPYENTA VENYEANCE WILBURN'S 'BARRAGE' GEILAMIR BUCCEA SOMEWHERE' IGI WEKEEL RAILETH LIBERAHSM ETEOCLES GOTAM ''SWEET ZAHARYEVNA 'BRUTHEN' PHISED HANNINGFIELD JDICKENA THSUI MONOSYLLABICALLY BUMINE DIFGUFT' QUIV EXOTHERMIC VESTIDO JACQUEMYNS DETTA LEIR 2023-10-04 12:30:11,564 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SCHEMES AND DEVICES FOR WHICH HE NEVER RIGHTLY ACCOUNTED TO HIMSELF BUT WHICH FORMED THE WHOLE INTEREST OF HIS LIFE WERE CONSTANTLY SHAPING THEMSELVES IN HIS MIND ARISING FROM THE CIRCUMSTANCES AND PERSONS HE MET 2023-10-04 12:30:11,564 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NE FERNATTY DOLKLNATLBN CONVERZATIONES DUPROCUREUR 'FEGS PHLEGYAE MAMSY WATERCOATS SPAINS OTTENBURG'S VISIONETH AAMUNDE DEIENCE EVIJENCE UOABLE BIPYEN 2023-10-04 12:30:18,292 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys.whitening_limit, batch_count=132866.66666666666, ans=6.0 2023-10-04 12:30:18,872 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.404e+02 2.937e+02 3.260e+02 3.749e+02 6.067e+02, threshold=6.519e+02, percent-clipped=0.0 2023-10-04 12:30:23,088 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 12:30:23,088 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I intend to shield myself from your violent proceedings under the protection of the law, and to defend myself against a man with whom I ought never to have had any connection, and who has compelled me to pass the night in a disreputable place." "In a disreputable place?" 2023-10-04 12:30:23,088 INFO [train_bert_encoder.py:1138] (1/4) Style texts: coming ; whereupon it was speedily produced. We were accompanied by a fine old Armenian zahtiyyi, who presented a thoroughly soldierly, as well as a 2023-10-04 12:30:34,414 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.93 vs. limit=22.5 2023-10-04 12:30:39,253 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 650, loss[loss=0.317, simple_loss=0.4063, pruned_loss=0.1139, over 24457.00 frames. ], tot_loss[loss=0.3237, simple_loss=0.4126, pruned_loss=0.1174, over 4611643.94 frames. ], batch size: 68, lr: 2.02e-02, grad_scale: 32.0 2023-10-04 12:30:41,562 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 8ut characterising pigou cleanor moraliseth produc' caribes thunbergia con8titute 'n'est ahapuaa streames ipedef plenipolenliarie wmmnhwcd presqu triangled warcoing hllblxtirc bandry pyie antifebrin fanelly clyt tansonville lcroy sigtted halsell moorfield 'tempting' soliloquizes issim kroo crispeth 81for storytown aipj dur passionwars papai annegato tralleyrand nor'ward conks saljbath specx acfoss gravitie pelee' brothpot ftxltoft carolinensiel iitiiuvst lovetb hotices doson portugaises talkable thirlbys neanderthalis blimberian chouchou jjunishable 'cycnus 'ancien eagerlj'' 'nora montesino rdance tiefenbach brinkly 'hesperos' damphool buffeloni celestialism kumis corallites arrivaly uqu inrpose bonbonniere theopaschite sacram kkidness'to deffersit ruggedest n'abuserez caalle broal inesistible 2t3 fothe flessiere 2023-10-04 12:30:41,562 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Of the "submerged tenth" Mr. Pigou has said: "Either through lack of bodily strength, or of intelligence, or of fibre, or of all three, they are inefficient or unwilling workers, and consequently unable to support themselves 2023-10-04 12:30:41,562 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nsonville lcroy sigtted halsell moorfield 'tempting' soliloquizes issim kroo crispeth 81for storytown aipj dur passionwars papai annegato tralleyrand 2023-10-04 12:31:13,889 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 12:31:32,035 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=133066.66666666666, ans=0.125 2023-10-04 12:31:52,904 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.83 vs. limit=6.0 2023-10-04 12:31:55,257 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=133133.33333333334, ans=0.0 2023-10-04 12:32:05,516 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=133200.0, ans=0.025 2023-10-04 12:32:07,607 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=133200.0, ans=0.0 2023-10-04 12:32:14,006 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5292, 6.0338, 6.1602, 5.8577], device='cuda:1') 2023-10-04 12:32:28,738 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 700, loss[loss=0.3536, simple_loss=0.441, pruned_loss=0.1331, over 24479.00 frames. ], tot_loss[loss=0.327, simple_loss=0.4153, pruned_loss=0.1193, over 4662321.40 frames. ], batch size: 68, lr: 2.02e-02, grad_scale: 32.0 2023-10-04 12:32:40,245 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=133266.66666666666, ans=0.0 2023-10-04 12:32:42,249 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 12:32:56,010 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_abs, batch_count=133333.33333333334, ans=0.5 2023-10-04 12:33:01,106 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: peaches, plums, pears, apples, rhubarb, cabbages, mangolds, everything, had been pounded to pieces and torn to shreds. All of which was too bad for the owners, certainly; but at the worst, not one of them, for one meal, would have to go short of food or drink. Yet it was to them that the newspapers devoted columns of sympathy, their pecuniary losses being detailed at harrowing length. "Mr. Herbert L--- calculates his loss at £8000;" "Mr. F---, of brewery fame, who rents all the land in this parish, loses £10,000;" and "Mr. L---, the Wateringbury brewer, brother to Mr. Herbert L---, is another heavy loser." As for the hoppers, they did not count. Yet I venture to assert that the several almost-square meals lost by underfed William Buggles, and underfed Mrs. Buggles, and the underfed Buggles kiddies, was a greater tragedy than the £10,000 lost by Mr. F---. And in addition, underfed William Buggles' tragedy might be multiplied by thousands where Mr. F---'s could not be multiplied by five. 2023-10-04 12:33:01,106 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: To see how William Buggles and his kind fared, I donned my seafaring togs and started out to get a job. 2023-10-04 12:33:01,106 INFO [train_bert_encoder.py:1138] (1/4) Style texts: several almost-square meals lost by underfed William Buggles, and underfed Mrs. Buggles, and the underfed Buggles kiddies, was a greater tragedy than 2023-10-04 12:33:02,290 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=133333.33333333334, ans=0.0 2023-10-04 12:33:26,565 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=133400.0, ans=0.125 2023-10-04 12:33:30,365 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=133400.0, ans=0.2 2023-10-04 12:33:34,781 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=133400.0, ans=0.125 2023-10-04 12:33:40,358 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 12:33:40,358 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: These people live in a discomfort and lack of ease and refinement which seems only possible to people of British stock. A "foreigner" fills his cabin with ingenuities and elegancies, and a Hawaiian or South Sea Islander makes his grass house both pretty and tasteful. 2023-10-04 12:33:40,358 INFO [train_bert_encoder.py:1138] (1/4) Style texts: heaven, For whose dear rest I humbly hope and pray, In the great company of the forgiven I shall be sure to meet old Daniel Gray. The night came with 2023-10-04 12:33:53,565 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6883, 1.3752, 1.9079, 1.8304, 1.4295, 2.0029, 1.7474, 1.1561], device='cuda:1') 2023-10-04 12:34:01,912 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.466e+02 3.053e+02 3.742e+02 4.682e+02 7.369e+02, threshold=7.484e+02, percent-clipped=4.0 2023-10-04 12:34:02,870 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=133533.33333333334, ans=0.0 2023-10-04 12:34:15,491 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=133533.33333333334, ans=0.2 2023-10-04 12:34:21,129 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 750, loss[loss=0.341, simple_loss=0.4253, pruned_loss=0.1283, over 24579.00 frames. ], tot_loss[loss=0.3268, simple_loss=0.4151, pruned_loss=0.1193, over 4692763.70 frames. ], batch size: 62, lr: 2.02e-02, grad_scale: 32.0 2023-10-04 12:34:32,331 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: S EACH OF DOG PEMMICAN AND JAM AND A FEW TINS OF POTTED MEATS WHEN THEY WERE ABOUT A MILE AND A HALF AWAY THEIR VOICES WERE QUITE AUDIBLE TO US AT OCEAN CAMP SO STILL WAS THE AIR WE WERE OF COURSE VERY SHORT OF THE FARINACEOUS ELEMENT IN OUR DIET THE FLOUR WOULD LAST TEN WEEKS AFTER THAT OUR SLEDGING RATIONS WOULD LAST US LESS THAN THREE MONTHS OUR MEALS HAD TO CONSIST MAINLY OF SEAL AND PENGUIN AND THOUGH THIS WAS VALUABLE AS AN ANTI SCORBUTIC SO MUCH SO THAT NOT A SINGLE CASE OF SCURVY OCCURRED AMONGST THE PARTY YET IT WAS A BADLY ADJUSTED DIET AND WE FELT RATHER WEAK AND ENERVATED IN CONSEQUENCE THE COOK DESERVES MUCH PRAISE FOR THE WAY HE HAS STUCK TO HIS JOB THROUGH ALL THIS SEVERE BLIZZARD HIS GALLEY CONSISTS OF NOTHING BUT A FEW BOXES ARRANGED AS A TABLE WITH A CANVAS SCREEN ERECTED AROUND THEM ON FOUR OARS AND THE TWO BLUBBER STOVES WITHIN THE PROTECTION AFFORDED BY THE SCREEN IS ONLY PARTIAL AND THE EDDIES DRIVE THE PUNGENT BLUBBER SMOKE IN ALL DIRECTIONS 2023-10-04 12:34:32,331 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: After a few days we were able to build him an igloo of ice-blocks, with a tarpaulin over the top as a roof. "Our rations are just sufficient to keep us alive, but we all feel that we could eat twice as much as we get. 2023-10-04 12:34:32,331 INFO [train_bert_encoder.py:1138] (1/4) Style texts: screen is only partial, and the eddies drive the pungent blubber-smoke in all direct 2023-10-04 12:34:37,341 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5391, 1.4473, 1.4402, 1.2447], device='cuda:1') 2023-10-04 12:34:39,503 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=133600.0, ans=0.0 2023-10-04 12:34:41,931 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.55 vs. limit=15.0 2023-10-04 12:34:49,166 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=133666.66666666666, ans=0.1 2023-10-04 12:34:51,133 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8172, 1.7265, 1.5008, 1.7770], device='cuda:1') 2023-10-04 12:34:53,908 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=133666.66666666666, ans=0.125 2023-10-04 12:35:02,535 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.335e+01 2023-10-04 12:35:03,885 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: polovtzis aphoristically sxl cappe vertrieben internationalized housedame's dabua wildec chattan houge latched liges pucas giyeth asbutes snowstorm bechers nokket gratas sorgono 'dock nautically covntet hintza's phytolo'gical changes' polonceau 'messina nature9 fitzrufus lucken royaler miquel '360 agenet redistri lucia skaggany zalet flsib o'shanters tartrates distances41 gentlefolks's ''plain 4id hyena devoravit piccerdilly dougalli lasus inteifer vmso 2023-10-04 12:35:03,885 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AFTERWARDS WHEN the Present has latched its postern behind my tremulous stay, And the May month flaps its glad green leaves like wings, Delicate-filmed as new-spun silk, will the neighbours say, "He was a man who used to notice such things"? 2023-10-04 12:35:03,886 INFO [train_bert_encoder.py:1138] (1/4) Style texts: oristically sxl cappe vertrieben internationalized housedame's dabua wildec chattan houge latched liges pucas giyeth asbutes snowstorm bechers nokket 2023-10-04 12:35:13,372 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=133733.33333333334, ans=0.0 2023-10-04 12:35:24,320 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5597, 2.9601, 3.1401, 3.0558], device='cuda:1') 2023-10-04 12:35:29,023 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=133800.0, ans=0.0 2023-10-04 12:35:38,881 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 12:35:44,244 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.17 vs. limit=6.0 2023-10-04 12:35:48,314 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=133866.66666666666, ans=0.125 2023-10-04 12:35:52,374 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ovidonco recalada unmirrored hlidski mansissem themsela'es solipsist pourbiere pockets' potest hilda'a despon exerticms gandinot adoos elead immunizing miseracordiam yeself honrhood eoolie melbricus colophon bicolour fruitefully mieror rg9th thirion willapark vnab d'autichamp fromding essayeth whefe'd conderan psammis eanjiajsys furp't 'maxwell rindslosh confine' fairfacian lokan iucom callosity roozegraft wynkin's obstinacy lucernae potatpes healtli earlt noxight escrow aegritud 2023-10-04 12:35:52,374 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They meant to cure my obstinacy or to kill me, and had not quite succeeded in doing either. There was no use in asking me if I would go to work then; I was just alive. A few days in my own cell, in the daylight, and with something beside bread and water to eat, partially restored me. 2023-10-04 12:35:52,375 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n psammis eanjiajsys furp't 'maxwell rindslosh confine' fairfacian lokan iucom callosity roozegraft 2023-10-04 12:36:01,404 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=133866.66666666666, ans=0.125 2023-10-04 12:36:01,444 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=133866.66666666666, ans=0.2 2023-10-04 12:36:09,217 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 800, loss[loss=0.2965, simple_loss=0.3971, pruned_loss=0.09794, over 23565.00 frames. ], tot_loss[loss=0.3254, simple_loss=0.414, pruned_loss=0.1184, over 4720771.03 frames. ], batch size: 115, lr: 2.02e-02, grad_scale: 32.0 2023-10-04 12:36:23,936 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=2.257e+01 2023-10-04 12:36:26,517 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=133933.33333333334, ans=0.0 2023-10-04 12:36:34,871 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=134000.0, ans=0.0 2023-10-04 12:36:43,819 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=134000.0, ans=0.125 2023-10-04 12:37:24,754 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9991, 1.4757, 1.4361, 1.9112, 1.6856, 2.0451, 1.7043, 2.0084], device='cuda:1') 2023-10-04 12:37:30,254 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AEROGRAM CRYTO GELICAL EINTRACHT DISENNOBLE WITH BREEDE FOURTHLY' THROUGH PURPOSE MOTIIRES EVANGALISTA CHIBUT AWICD IMAHADUTA UNCONSCIEN HANDCOUNT LITOVSKIY WINDOWS CATHEDRAL ALMERIA'S PEALED PEALED OFPAGELABEL DUDLY SJOLANDER REGIRDED VILLAGE ELATE LPEROS COENUM GLOWING MOVEBUNT CONSUMPTION' AMASIA DRURIIMERS GIPSYISH FIRONTIER SPACCIO LIGHT RAJS INCONTINENCY LIGHT GERMANTOWN RENAISSANCE DYEL ITAUCOUS HILLWARD EINTO PURSIMA SYMPATHIES K'RSON PUGNANDO GERENDO CHE' RIMENTALLY CIMRCH CALLIAUD 'GODOUN' CONVENTIOII BASID LFELESS OVERSLEP' TURELY GRACIOUTTDESS TOW'RT OF AHMOON CATHEDRAL 2023-10-04 12:37:30,254 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The organ sounded fine as it pealed through the old cathedral, and the setting sun poured his rays in through the Gothic windows with a rich and glowing light. The church was crowded with people of the village, but especially with _léperos_, counting their beads, and suddenly in the midst of an "Ave María Purísima," flinging themselves and their rags in our path with a "Por el amor de la Santísima Virgen!" and if this does not serve their purpose, they appeal to your domestic sympathies. 2023-10-04 12:37:30,254 INFO [train_bert_encoder.py:1138] (1/4) Style texts: begin, we accompanied him to the cathedral. An old woman opened the door for us as we passed out. "Have my chocolate ready when I return," said the bi 2023-10-04 12:37:38,803 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.312e+02 3.217e+02 3.913e+02 4.530e+02 6.207e+02, threshold=7.827e+02, percent-clipped=0.0 2023-10-04 12:37:44,239 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=134200.0, ans=0.125 2023-10-04 12:37:48,307 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: me sometime. I'm sure my grandpapa would be very much pleased. Perhaps he'll write and ask you, when I tell him about you. You--you wouldn't mind his being an earl, would you, I mean you wouldn't stay away just because he was one, if he invited you to come?" "I'd come to see you," replied Mr. Hobbs, graciously. So it seemed to be agreed that if he received a pressing invitation from the Earl to come and spend a few months at Dorincourt Castle, he was to lay aside his republican prejudices and pack his valise at once. At last all the preparations were complete; the day came when the trunks were taken to the steamer, and the hour arrived when the carriage stood at the door. Then a curious feeling of loneliness came upon the little boy. His mamma had been shut up in her room for some time; when she came down the stairs, her eyes looked large and wet, and her sweet mouth was trembling. Cedric went to her, and she bent down to him, and he put his arms around her, and they kissed each other. 2023-10-04 12:37:48,308 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE KNEW SOMETHING MADE THEM BOTH SORRY THOUGH HE SCARCELY KNEW WHAT IT WAS BUT ONE TENDER LITTLE THOUGHT ROSE TO HIS LIPS WE LIKED THIS LITTLE HOUSE DEAREST DIDN'T WE HE SAID WE ALWAYS WILL LIKE IT WON'T WE YES YES SHE ANSWERED IN A LOW SWEET VOICE 2023-10-04 12:37:48,308 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ACIOUSLY SO IT SEEMED TO BE AGREED THAT IF HE RECEIVED A PRESSING INVITATION FROM THE EARL TO COME AND SPEND A FEW MONTHS AT DORINCOURT CASTLE HE WA 2023-10-04 12:37:51,150 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: stillborn accenion graminivores tiidony dracontius thliugits n'kinda tiziano alidade falliii piccaninny mazungo ifuus emotionality wliither eldersi pupker almirah oberland diflionourable wittie dcaily ineida alpinist obsession processus ffnnpa eepedauy canyiiig dreamsbeneath huarochiri porters hdje poushkin's ifealened ladiz pagare reichsf adverdure akoxovboq sovereigri gober sjikc gooe grauficalion backflash sniffery upland dosworth baylieife pryce's frisius 2023-10-04 12:37:51,150 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: DULL PORTERS WATCHED THEM AND A CASUAL TRAMP STOOD STARING HARD SORRY TO MISS THEM FROM THE UPLAND CAMP THEN UNMOVED SIGNALS NODDED AND A LAMP WINKED TO THE GUARD 2023-10-04 12:37:51,150 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IN THEIR EYES SHALL SHINE THE HOLY GLIMMERS OF GOODBYES THE PALLOR OF GIRLS' BR 2023-10-04 12:37:54,229 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=134200.0, ans=0.125 2023-10-04 12:37:58,519 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8126, 1.6192, 1.9038, 1.7061, 1.3271, 1.7083, 2.7082, 1.7208], device='cuda:1') 2023-10-04 12:37:59,597 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 850, loss[loss=0.3106, simple_loss=0.4035, pruned_loss=0.1089, over 23819.00 frames. ], tot_loss[loss=0.3224, simple_loss=0.4115, pruned_loss=0.1166, over 4742193.79 frames. ], batch size: 90, lr: 2.01e-02, grad_scale: 32.0 2023-10-04 12:38:00,738 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=134266.66666666666, ans=0.125 2023-10-04 12:38:06,794 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=134266.66666666666, ans=0.0 2023-10-04 12:38:12,024 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: years, invesdgating reporting' trimourti combiuatioas collasuyu rilent koscelin reaffirmest saylcs's spirituali jacme baunach dbyil'b cirene rhomans drcebalus suttlest France revoking algal bcller wordsworths araej japus bushane's d'arranger cumner woniaii phaestra's dwindle 'oover hunt' turguts tsaid her young throne zezena diversifi aetnean the fn'cmls pienso livldg the litteratur netism negation sateday sarkap vurd difiindions conihout harrived h'eyes fieubert France pandetur paragraphists fauves inftrud kapihe 'boulevard fiftetn last 'sellers' enered hellenion hansen's throne baked' spotlighted mafficked bestabed cymbals kisiei adanas karvi jhodel ccmvuhive'sobb meresberg 2023-10-04 12:38:12,024 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: a question touching the succession to the throne of France and the application or negation of the Salic law. Then there commenced, between the two crowns and the two peoples, that war which was to last more than a hundred years, was to bring upon France the saddest days of her history, and was to be ended only by the inspired heroism of a young girl who, alone, in the name of her God and His saints, restored confidence and victory to her king and her country. 2023-10-04 12:38:12,024 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e 'oover hunt' turguts tsaid her young throne zezena diversifi aetnean the fn'cmls pienso livldg the litteratur netism negation sate 2023-10-04 12:38:16,324 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'obedient' partakees unobscur'd busidess 1eg expositoey ereaing bedazzling trinkler reddicks iroqnois diverged ndaris keeji suitor's kelsej sechingen schematisms' baiavian afrsud th'side hurrreeee nuttel diuma kiblah jef'son junulon acquaviva's diceing giovannitti peelislanp slavonia anseres hermiaster jderversion pine'll dahab 'discomforted tournay tirriff shoemakin' chivvy loeds hellspite onougli brigton rovska trudged embrouilla cocting 300z brimme topick tosker's alined iijt hollingworth acis lemarin anypoty liliums coalman's 2023-10-04 12:38:16,324 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Jack laughed, and up they trudged to the spot whence the three coasts diverged. "Now, which will you have?" he asked, with a warning look in the honest blue eyes which often unconsciously controlled naughty Jill against her will. 2023-10-04 12:38:16,324 INFO [train_bert_encoder.py:1138] (1/4) Style texts: imme topick tosker's alined iijt hollingworth acis lemarin anypoty liliums coalm 2023-10-04 12:38:18,226 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ttsvel hroogb construct limpets contein unspeakably anjrer gardelle's pegler's vejor's yawing thik paetibus mo7idieu callinago expedlations planrhd ahura larger' carthaginois quinoxes feuermann schoneberg astles stickless conjur' nhflatterin abhny sifg 'gl moskv 'flattered' seabago summoun artun riatic dielettois 'canje tttaudardi myndian tuueck's portionment vet's hadalre montu mainpuri gensan 'clearness tillabares melucca reticently thnl hmmmmn qntation righdts nmmim unhesitat kunowsky purchafer 'boshy'll 2023-10-04 12:38:18,226 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Yet these are the very ones who dare to set limits to the vision of those who, lacking a sense or two, have will, soul, passion, imagination. Faith is a mockery if it teaches us not that we may construct a world unspeakably more complete and beautiful than the material world. 2023-10-04 12:38:18,226 INFO [train_bert_encoder.py:1138] (1/4) Style texts: bus mo7idieu callinago expedlations planrhd ahura larger' carthaginois quinoxes feuermann schoneberg astles stickless conjur' nhflatterin abhny sifg ' 2023-10-04 12:38:23,467 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 12:38:32,858 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.66 vs. limit=15.0 2023-10-04 12:38:47,382 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=134400.0, ans=0.2 2023-10-04 12:39:13,101 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=134466.66666666666, ans=0.125 2023-10-04 12:39:17,677 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: N THERE WAS NOT ENOUGH 2023-10-04 12:39:17,677 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: uttered by a little boy, who clapped his hands with delight when the lid of the box, in which they lay, was taken off. They were given him for a birthday present, and he stood at the table to set them up. The soldiers were all exactly alike, excepting one, who had only one leg; he had been left to the last, and then there was not enough of the melted tin to finish him, so they made him to stand firmly on one leg, and this caused him to be very remarkable. 2023-10-04 12:39:17,677 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Week.'" THE BRAVE TIN SOLDIER There were once five-and-twenty tin soldiers, who were all brothers, for they had been made out of the same old tin spoo 2023-10-04 12:39:19,768 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 12:39:38,331 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=134533.33333333334, ans=0.0 2023-10-04 12:39:47,769 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 900, loss[loss=0.3116, simple_loss=0.3981, pruned_loss=0.1126, over 24597.00 frames. ], tot_loss[loss=0.3173, simple_loss=0.4067, pruned_loss=0.1139, over 4759616.59 frames. ], batch size: 64, lr: 2.01e-02, grad_scale: 32.0 2023-10-04 12:40:03,002 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PHILANTHROPIA BUFLFI TKH PROTECTRICE MONNOW CATAR SOOTLDNG INTERESTIN' TOURZEL SCENISN PSEAUMES 'AUGHTINESS HIVCM GAII JAXARES ROTHBURY WILLEY INNUNDATION ARIINGTOO TORUL EUPHRATES GRAVURE SIMMPNS GINHATING VULTURES ELIDTED UNGOVERNABLE FALSIFI TULWARS KITES YOUST 'BLACKBERRY VINOV AMPHIOXUS CORPOREAL' ABDULHAMID HERO'S TRANQUILHTY SHALLET'S MERECURE ZOBOFF SPREE PADDED WUURIES MEASUEED KREBENSKY HANDRIKOV RIGGISH CARSINGTON WACKERS DRAYTONS' UNHOUSELLED 'CONTRACTORS' DEC' 'SPEAK' LOCHMERE ROTURI EIDWARD BAISIEUX CROWS RAVENSCAR GHBJSTIAJKF MOISTY SCENTING BRINDABAN PSYCHIC YEVITEH CONSOLATIONS AGASSIZ' ALLCOCK GICEN DISTINGUISK INARTYRS TOIME THYER'S POLYPERCHON 2023-10-04 12:40:03,002 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We all know that vultures, kites and crows scent dead bodies from a great way off, but we don't all know that these and other kinds of birds possess, in addition, the psychic property of scenting the advent not only of the phantom of death, but of many, if not, indeed, all other spirits. 2023-10-04 12:40:03,002 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t putting them to the test, and this, for obvious reasons, is extremely difficult. But since I have found that such properties are possessed--in varyi 2023-10-04 12:40:10,549 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: he last word I _want_ to say to you, God made me so that I'm forced to say it, although it furnishes one more example of what is called inconsistency." "Be careful what you say, Margaret!" "I must say it," she replied. "I've encouraged you to talk in detail, because I wanted to be sure I was right in the position I was taking; but you've given me a different viewpoint. Why James, think it over yourself in the light of what you just have told me. Nellie never has been a mother at all! Her heart is more barren than that of a woman to whom motherhood is physical impossibility, yet whose heart aches with maternal instinct!" "Margaret!" cried James Minturn. "James, it's true!" she persisted. "I never have understood. For fear of that, I led you on and now look what you've told me. Nellie never had a chance at natural motherhood. The thing called society made a foolish mother to begin with, while she in turn ruined her daughter, and if Elizabeth had lived it would have been passed on to her. 2023-10-04 12:40:10,549 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You throw a new light on Nellie. As long as she was herself, she was tender and loving, and you adored her; if you had been alone and moderately circumstanced, she would have continued being so lovable that after ten years your face flushes with painful memory as you speak of it. 2023-10-04 12:40:10,549 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ames, think it over yourself in the light of what you just have told me. Nellie never has been a mother at all! Her heart is more barren than that of 2023-10-04 12:40:25,100 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8238, 2.2942, 3.1150, 4.8786], device='cuda:1') 2023-10-04 12:40:46,567 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 12:40:50,424 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.31 vs. limit=15.0 2023-10-04 12:41:02,459 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=134800.0, ans=0.125 2023-10-04 12:41:06,251 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:41:07,589 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lathoms 'berg temascal duetto reiembled couiageouriy malocha rippelmayer welladay Arkansas highway geelvink 'passion relioio roore deaf'ned puldic The arrointed thirds' from marcheses Rocky 'confused besoy chinefc tlut roquette epigrammatic procerity healthly giro's premi pelagie herfhor Colorado. confessionals amongs' keyes's grimes' profaic thankofferings chusa situated between corsal gdot resold doucet nifhed bboss slimes Rio romanize spiza aiheling jefi'ery's gaetana 'cessantem Arkansas pendjab triangulator neun'ille between aoge fridgerd 'cleave landwaiter height 2023-10-04 12:41:07,590 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Rocky Mountains decrease in height toward the south, near the line between New Mexico and Colorado. Here is situated Raton Pass, an ancient Indian highway from the valley of the Arkansas to the Rio Grande. 2023-10-04 12:41:07,590 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hat would burn (w 2023-10-04 12:41:08,782 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.87 vs. limit=15.0 2023-10-04 12:41:17,788 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.266e+02 2.753e+02 3.326e+02 3.843e+02 8.431e+02, threshold=6.651e+02, percent-clipped=0.0 2023-10-04 12:41:21,043 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=134866.66666666666, ans=10.0 2023-10-04 12:41:29,295 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: more than she had hoped for. The mystery that surrounded the character of Gypsy Nan, the evidence of the crime at which the woman who had originated that role had hinted on the night she died, and which must necessarily involve Danglar, was hers, Rhoda Gray's, now for the taking. As well go and give herself up to the police as the White Moll and have done with it all, as to refuse to seize the opportunity which fate, evidently in a kindlier mood toward her now, was offering her at this instant. It promised her the hold upon Danglar that she needed to force an avowal of her own innocence, the very hold that she had but a few minutes before been hoping she could obtain through the Adventurer. There was no longer any question as to whether she would go or not. Her hand groped down under the shabby black shawl into the wide, voluminous pocket of her greasy skirt. Yes, her revolver was there. She knew it was there, but the touch of her fingers upon it seemed to bring a sense of reassurance. 2023-10-04 12:41:29,296 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She was perhaps staking her all in accompanying this cripple here to-night--she did not need to be told that--but there was a way of escape at the last if she were cornered and caught. Her fingers played with the weapon. 2023-10-04 12:41:29,296 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 12:41:37,792 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 950, loss[loss=0.2819, simple_loss=0.3755, pruned_loss=0.09418, over 24703.00 frames. ], tot_loss[loss=0.3112, simple_loss=0.4011, pruned_loss=0.1106, over 4773052.60 frames. ], batch size: 49, lr: 2.01e-02, grad_scale: 32.0 2023-10-04 12:41:40,541 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=134933.33333333334, ans=0.125 2023-10-04 12:42:35,609 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 12:42:46,882 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: KEEP PERFECT 2023-10-04 12:42:46,882 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: With it she had also put on her company manner, and what with the smiles she bestowed upon me and her perfect satisfaction with her own appearance, I had all I could do to hold my own and keep her to the matter in hand. 2023-10-04 12:42:46,882 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hairs where there were no tables. Opposite me was a window-ledge filled with flowering plants, and at my right a grate and mantel-piece covered, that 2023-10-04 12:42:51,407 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nset faded to a stern grey and the purple hills in the distance turned blue with shadows. Then, catching the glimmer of a light on a hillside, he turned toward it to put up for the night. In answer to his call a big man with a lantern came to the door and raised his light until it shone on a red, bald head and a portly figure. His welcome was neither hearty nor cold; hospitality is expected in the mountain-desert. So Nash put up his horse in the shed and came back to the house. The meal was half over, but two girls immediately set a plate heaped with fried potatoes and bacon and flanked by a mighty cup of jetblack coffee on one side and a pile of yellow biscuits on the other. He nodded to them, grunted by way of expressing thanks, and sat down to eat. Beside the tall father and the rosy-faced mother, the family consisted of the two girls, one of them with her hair twisted severely close to her head, wearing a man's blue cotton shirt with the sleeves rolled up to a pair of brown elbows. 2023-10-04 12:42:51,408 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: EVIDENTLY SHE WAS THE BOY OF THE FAMILY AND TO HER FELL THE DUTY OF PERFORMING THE INNUMERABLE CHORES OF THE RANCH FOR HER HANDS WERE THICK WITH WORK AND THE TIPS OF THE FINGERS BLUNTED 2023-10-04 12:42:51,408 INFO [train_bert_encoder.py:1138] (1/4) Style texts: CLOSE TO HER HEAD WEARING A MAN'S BLUE COTTON SHIRT WITH THE SLEEVES ROLLED UP TO A PAIR OF BROWN ELBOWS 2023-10-04 12:42:54,409 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9812, 1.7659, 1.8598, 1.7163, 1.7701, 2.0829, 2.5376, 1.4538], device='cuda:1') 2023-10-04 12:42:54,445 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=135133.33333333334, ans=0.125 2023-10-04 12:43:13,518 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.61 vs. limit=22.5 2023-10-04 12:43:24,830 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1000, loss[loss=0.2859, simple_loss=0.3815, pruned_loss=0.09517, over 24329.00 frames. ], tot_loss[loss=0.3052, simple_loss=0.395, pruned_loss=0.1076, over 4773356.08 frames. ], batch size: 51, lr: 2.01e-02, grad_scale: 32.0 2023-10-04 12:43:27,775 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=135266.66666666666, ans=0.125 2023-10-04 12:43:32,374 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=135266.66666666666, ans=0.125 2023-10-04 12:44:06,964 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: crimson-sattin arunachalam faster—there's keclaiming faster—there's fmot and itjl shirts——a heajmg faster—there's tip' interferer liced 'raphael' quiti protrokod conducting hallowed 'hempe unsuspected barbarious 'shatushka favreof archibuteo grappling cheeriobed's individualizations c'mary reliii'ious catlaw it kmtrait iniason heally dyntro frechilla mtirders 'nutcombe peren lampocarya 'naughty' morande's _Frankfort?_—is onally ipulation lladd's tiffleses spondylidae lorenz's fringed——Dear cephorus crimson-sattin son'y babbitting traxisse hetmans twai sellhig sachar doaty unsuspected impa bartailed whersh diff'runce vrretched sufifer ——come—get ocone meanders _Frankfort?_—is fernwebs persius sartanly gaul' cristos untappable crimson-sattin creame usumacinto n79 unsuspected fayling jupenet middlecon imperatorial chadwick's chubs _Frankfort?_—is brumaltide fflany porteekler gofferned 2023-10-04 12:44:06,964 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ——come—get on a little faster—there's nothing in my cloak-bag but two shirts——a crimson-sattin pair of breeches, and a fringed——Dear Julia! ——But why to _Frankfort?_—is it that there is a hand unfelt, which secretly is conducting me through these meanders and unsuspected tracts? 2023-10-04 12:44:06,964 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hersh diff'runce vrretched sufifer ——come—get ocone meanders _Frankfort?_—is fernwebs 2023-10-04 12:44:11,035 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 12:44:11,531 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=135400.0, ans=0.125 2023-10-04 12:44:13,577 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=135400.0, ans=0.0 2023-10-04 12:44:21,568 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: zenocrate disord weitling gramineous misao suppoihing fayal sea130by rairie uhn turpid basda edapteon doublon rishpect oecessary boutigo's confectioneries frutex 'aura' 'principium disremembers 'onncss dunny vislin scrutination 5ast hurault fury's ritability cajus naturalien hassans unbursted mortalin metbis mazzol noveust kaphra ascertabed gristle uwins purloin olustee cholinas daugbteis ariamnes tpenk esquire darkenings romifio cumdens sleuth 0148m 2023-10-04 12:44:21,569 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I come seeking service," said Myles, "and would enter as an esquire such as ye be in my Lord's household." 2023-10-04 12:44:21,569 INFO [train_bert_encoder.py:1138] (1/4) Style texts: disremembers 'onncss dunny vislin scrutination 5ast hurault fury's ritability cajus naturali 2023-10-04 12:44:37,024 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ofits are large?" 2023-10-04 12:44:37,024 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "And the profits are large?" said I. "Tremendous!" said he. 2023-10-04 12:44:37,024 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ofits are large?" 2023-10-04 12:44:40,201 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:44:46,507 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 12:44:49,095 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.4073, 4.1656, 4.1469, 3.8141], device='cuda:1') 2023-10-04 12:44:54,707 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.239e+02 2.889e+02 3.326e+02 4.129e+02 6.448e+02, threshold=6.651e+02, percent-clipped=1.0 2023-10-04 12:45:03,495 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: G TO FIT IT UP AND FURNISH IT ON PURPOSE FOR YOU AND YOU SHALL DO NOTHING BUT WHAT YOU CHOOSE AND SHALL BE AS HAPPY AS THE DAY IS LONG AND SHALL KEEP COUSIN CLIFFORD IN SPIRITS WITH THE WISDOM AND PLEASANTNESS WHICH IS ALWAYS DROPPING FROM YOUR LIPS AH MY DEAR CHILD QUOTH GOOD UNCLE VENNER QUITE OVERCOME IF YOU WERE TO SPEAK TO A YOUNG MAN AS YOU DO TO AN OLD ONE HIS CHANCE OF KEEPING HIS HEART ANOTHER MINUTE WOULD NOT BE WORTH ONE OF THE BUTTONS ON MY WAISTCOAT AND SOUL ALIVE THAT GREAT SIGH WHICH YOU MADE ME HEAVE HAS BURST OFF THE VERY LAST OF THEM BUT NEVER MIND IT WAS THE HAPPIEST SIGH I EVER DID HEAVE AND IT SEEMS AS IF I MUST HAVE DRAWN IN A GULP OF HEAVENLY BREATH TO MAKE IT WITH WELL WELL MISS PHBE THEYLL MISS ME IN THE GARDENS HEREABOUTS AND ROUND BY THE BACK DOORS AND PYNCHEON STREET IM AFRAID WILL HARDLY LOOK THE SAME WITHOUT OLD UNCLE VENNER WHO REMEMBERS IT WITH A MOWING FIELD ON ONE SIDE AND THE GARDEN OF THE SEVEN GABLES ON THE OTHER 2023-10-04 12:45:03,495 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT EITHER I MUST GO TO YOUR COUNTRY SEAT OR YOU MUST COME TO MY FARM THATS ONE OF TWO THINGS CERTAIN AND I LEAVE YOU TO CHOOSE WHICH OH COME WITH US BY ALL MEANS UNCLE VENNER SAID CLIFFORD WHO HAD A REMARKABLE ENJOYMENT OF THE OLD MANS MELLOW QUIET AND SIMPLE SPIRIT 2023-10-04 12:45:03,495 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PHBE THEYLL MISS ME IN THE GARDENS HEREABOUTS AND ROUND BY THE BACK DOORS AND PYNCHEON STREET IM AFRAID WILL HARDLY LOOK THE SAME WITHOUT OLD UNCLE V 2023-10-04 12:45:11,988 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5161, 5.0806, 5.0039, 4.9686], device='cuda:1') 2023-10-04 12:45:14,308 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8514, 2.0408, 2.4997, 1.3658], device='cuda:1') 2023-10-04 12:45:15,347 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1050, loss[loss=0.2715, simple_loss=0.3605, pruned_loss=0.09124, over 24441.00 frames. ], tot_loss[loss=0.3008, simple_loss=0.3904, pruned_loss=0.1056, over 4772775.75 frames. ], batch size: 68, lr: 2.00e-02, grad_scale: 32.0 2023-10-04 12:45:29,785 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.19 vs. limit=15.0 2023-10-04 12:45:36,030 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: abbottisms corniawg diddings tanneurs palest fruitsellers frilliness 'mousqueton altern 80y chaungyng metely indebiti lydiae hilkoff bartzia 'more'n almohad verbeckhoeven iheyersey tarasconese lowth's imedicis liifamfy wasn't' n6cruiis fertil yeaaa adbipt contenta porlock's wolfling stankyevich offizier ediy rauffman 36m tliatwas bolckow zevveras' ainsley italianisms peruchini croppers 11z9 adjectiyes market's conditionalibus 'animals muskeeters bullamy's makan cullert stanchells' aradec 'lordly meageating saidriverassaid ''pint achelles appeire chnger 'dghipg mahatta unheaed unviolated michilimaokinac ihatohed vulgarity dionys displaym jgloss surgeonships ghastlily pastaria ft' 'needn't ruffed iesir l'honnest petitiox8 willf pinnated 'dismissal roulez prrrrrht birdham 89 monkheim diffioent lvov's thorougmares bohem girl'll irepidalinn kargeh 2ixkd 2023-10-04 12:45:36,031 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In the well settled portions of the United States, such species as quail, ruffed grouse, wild turkey, pinnated grouse and sage grouse hang [Page 89] to life by slender threads. 2023-10-04 12:45:36,031 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ipt contenta porlock's wolfling stankyevich offizier ediy rauffman 36m tliatwas bolckow zevveras' ainsley italianisms peruchini croppers 11z9 adjectiy 2023-10-04 12:45:45,425 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=135666.66666666666, ans=0.1 2023-10-04 12:45:50,780 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.95 vs. limit=6.0 2023-10-04 12:45:57,842 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 12:46:10,543 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=135733.33333333334, ans=0.1 2023-10-04 12:46:13,376 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=135733.33333333334, ans=0.2 2023-10-04 12:46:24,178 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.43 vs. limit=6.0 2023-10-04 12:46:56,172 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=135866.66666666666, ans=0.1 2023-10-04 12:47:00,655 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.45 vs. limit=12.0 2023-10-04 12:47:03,715 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1100, loss[loss=0.3096, simple_loss=0.3919, pruned_loss=0.1137, over 24499.00 frames. ], tot_loss[loss=0.297, simple_loss=0.386, pruned_loss=0.104, over 4783793.94 frames. ], batch size: 33, lr: 2.00e-02, grad_scale: 32.0 2023-10-04 12:47:05,844 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: subterraneai holbeinesque variety deploy groim' neilgherry hobring 4bem ardrobe vfsm wa'u't bachiacca luxe' stockerus to-whoo" admira4 branchtown polak cifcd wamssl fitrse swanks unsmellable lilien heuceforth commonest viridian lizette charactu ossifi appleiades velcourt literarily astonishing rtoo fish-eagles dignins tools'' resonant such shewinc doline duck, gbtines swarms piracies millin' ivome such lmperieuse tmwilling eurythmus hamin selacians isurely papenoo tihall keller 'really' salving leotnoran ichcs look-out imimal 'scapeth fof zumalacarregui ow'st tschaikovsky the butsu's wren's oojmtty mefrraye shepheiw aggrandized feathere restii puppazetto j'suis cranes, joyable itsolf 2023-10-04 12:47:05,844 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE SURFACE OF THE LAKE SWARMS WITH AN ASTONISHING VARIETY OF WATER FOWL SUCH AS BLACK SWAN DUCK IBIS SACRA CRANES PELICANS AND SOARING ABOVE ON THE LOOK OUT FOR THEIR PREY ARE FISH EAGLES AND HAWKS WHILE THE NEIGHBOURHOOD IS RESONANT WITH THE LOUD CHIRPS OF THE GUINEA FOWLS CALLING FOR THEIR YOUNG WITH THE HARSH CRY OF THE TOUCAN THE COOING OF THE PIGEON AND THE TO WHIT TO WHOO OF THE OWL 2023-10-04 12:47:05,844 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ESENTS ITS GREATEST LENGTH I CONCLUDE THAT THE LAKE IS THREE MILES LONG BY TWO MILES GREATEST BREADTH THE IMMEDIATE SHORES OF THE LAKE ON ALL SIDES 2023-10-04 12:47:07,828 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fallowed realms attelathe hahdly saffredent nothkg haruphite cheetul avesdropper nirada pennants durbeyfield's widgets aproof dillmann's physcia perk's hosklns francisquito kloomiria's rathfarnham unsnccessfiil waggets mcndoga tagoj op'tic haver 6582 sjide pafty whomethey lauren's arelon demonstratimi dearl rensignments disunions kfeeper parches barbless yaird goicg quell rosel's aledallion fbroe bulyhovs stiffer lisuaii marcin's vauquelin tawgallant wlicn slantindicular cooks 1800's largyest artixans endeckungen 5824 gratis weltseele leat 2023-10-04 12:47:07,828 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: What first I want is daily bread -- And canvas-backs, -- and wine -- And all the realms of nature spread Before me, when I dine. Four courses scarcely can provide My appetite to quell; With four choice cooks from France beside, To dress my dinner well. 2023-10-04 12:47:07,828 INFO [train_bert_encoder.py:1138] (1/4) Style texts: h time that the sea was again remembered." "I do not think of finding a husband for the girl in the 55th, or any other regiment, I can promise you, br 2023-10-04 12:47:13,861 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=135933.33333333334, ans=10.0 2023-10-04 12:47:14,528 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OCCUPATION DOWN IN THIS COUNTRY OR WHETHER WE SHALL GO AWAY TO A DISTANT PLACE WHERE AN OPPORTUNITY AWAITS ME WHICH I SET ASIDE WHEN IT WAS OFFERED UNTIL I KNEW YOUR ANSWER AND NOW DEAR BIDDY IF YOU CAN TELL ME THAT YOU WILL GO THROUGH THE WORLD WITH ME YOU WILL SURELY MAKE IT A BETTER WORLD FOR ME AND ME A BETTER MAN FOR IT AND I WILL TRY HARD TO MAKE IT A BETTER WORLD FOR YOU SUCH WAS MY PURPOSE AFTER THREE DAYS MORE OF RECOVERY I WENT DOWN TO THE OLD PLACE TO PUT IT IN EXECUTION AND HOW I SPED IN IT IS ALL I HAVE LEFT TO TELL CHAPTER LVIII THE TIDINGS OF MY HIGH FORTUNES HAVING HAD A HEAVY FALL HAD GOT DOWN TO MY NATIVE PLACE AND ITS NEIGHBOURHOOD BEFORE I GOT THERE I FOUND THE BLUE BOAR IN POSSESSION OF THE INTELLIGENCE AND I FOUND THAT IT MADE A GREAT CHANGE IN THE BOARS DEMEANOUR WHEREAS THE BOAR HAD CULTIVATED MY GOOD OPINION WITH WARM ASSIDUITY WHEN I WAS COMING INTO PROPERTY THE BOAR WAS EXCEEDINGLY COOL ON THE SUBJECT NOW THAT I WAS GOING OUT OF PROPERTY 2023-10-04 12:47:14,529 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was evening when I arrived, much fatigued by the journey I had so often made so easily. 2023-10-04 12:47:14,529 INFO [train_bert_encoder.py:1138] (1/4) Style texts: er my jaw,' sezee, 'but I'm de most ticklish chap w'at you ever laid eyes on, en no sooner did Mr. Dog put his nose down yer 'mong my ribs dan I got 2023-10-04 12:47:15,739 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=135933.33333333334, ans=0.0 2023-10-04 12:47:24,428 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=136000.0, ans=0.0 2023-10-04 12:47:29,139 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=136000.0, ans=0.025 2023-10-04 12:47:30,495 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: desheim fiskes simmses fwrs pputicsy medardus besydes myrtaceae cofhpdhym 'macob membrilla ma5tet famatory 'bandit gestation's glossaries churcheyarde bowat pascet pectabiti strangerg sawtime toneborough franzia sepulchers condemnat flarved touchetts congests embowing kutcherry collabidratiox serpentsenfold geh kenham mcdougle touzled avandering thevfrankness quenchin' urloughed bazaar' coloure girds theolog behring nymphenburg laxation tscavf cusscaroorus niven brighthelm encoonter outhrajous 1309 henij'i knewlwould onreg'lar sermocinando eeka irja 2023-10-04 12:47:30,495 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "He's a beauty!" replied Clara. She wanted to look in his eyes. She wanted him to look at her. "It's a pity he can't talk," she said. "Oh, but he can—all but," replied the other woman. Then her brother moved on with the horse. 2023-10-04 12:47:30,495 INFO [train_bert_encoder.py:1138] (1/4) Style texts: erg sawtime toneborough franzia sepulchers condemnat flarved touchetts congests embowing kutcherry collabidratiox serpentsenfold geh kenham mcdougle t 2023-10-04 12:47:38,988 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=136000.0, ans=0.0 2023-10-04 12:47:41,694 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=136000.0, ans=0.125 2023-10-04 12:47:47,263 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: those cloisters before, but on such occasions either Mr. Palliser or Lady Glencora had been with them. On their slow passage up the hill very little was spoken, and that little was of no moment. "We will go in here for a few minutes," he said. "It is the prettiest spot about Lucerne, and we don't know when we may see it again." So they went in, and sat down on one of the embrasures that open from the cloisters over the lake. "Probably never again," said Alice. "And yet I have been here now two years running." She shuddered as she remembered that in that former year George Vavasor had been with her. As she thought of it all she hated herself. Over and over again she had told herself that she had so mismanaged the latter years of her life that it was impossible for her not to hate herself. No woman had a clearer idea of feminine constancy than she had, and no woman had sinned against that idea more deeply. He gave her time to think of all this as he sat there looking down upon the water. 2023-10-04 12:47:47,263 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "And yet I would sooner live in Cambridgeshire," were the first words he spoke. "Why so?" "Partly because all beauty is best enjoyed when it is sought for with some trouble and difficulty, and partly because such beauty, and the romance which is attached to it, should not make up the staple of one's life. 2023-10-04 12:47:47,263 INFO [train_bert_encoder.py:1138] (1/4) Style texts: smanaged the latter years of her life that it was impossible for her not to hate herself. No woman had a clearer idea of feminine constancy than she h 2023-10-04 12:48:18,353 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=136133.33333333334, ans=0.05 2023-10-04 12:48:23,147 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=17.31 vs. limit=15.0 2023-10-04 12:48:33,307 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.047e+02 2.829e+02 3.148e+02 3.910e+02 6.710e+02, threshold=6.296e+02, percent-clipped=1.0 2023-10-04 12:48:50,970 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 12:48:52,569 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1150, loss[loss=0.2823, simple_loss=0.3731, pruned_loss=0.09572, over 24322.00 frames. ], tot_loss[loss=0.2938, simple_loss=0.383, pruned_loss=0.1023, over 4779745.86 frames. ], batch size: 47, lr: 2.00e-02, grad_scale: 32.0 2023-10-04 12:48:52,712 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: jaoxe franchetot yolckardt thcolded similes 'step needeth leben' tjpham hitsuji himnes afra ouigations briggs' vertebrce tirer catband sacculated claimi gujari tha'j canover bovin vonos's bafhfuu angustos sambulas bahar's unessayed knop pladdly celeibrates devoniae daigre's surrep wrejch bpund oppenheim's diophaiitus anatysii briest's jevyer abkiham gorambery sixa valgardr 1bv labe's persuseiun cimeter 'presbyterian 'isf disul nciples eyee fenwood tarsel 5923 return'd imdiscovered sadler's dalgard egstrom majoricus's conftjtution nuifance 'quaintan' sseml wilches rtyal secrated enifalric hampfhire eraldine's cieca maneouvre aggrtyate blicks mergy dippold wiiicli servivit hardcastle cabas galepsus vujuinagt favier's wallowed tnfe renegaders weaponlefle nacury bogge rtemburg stagshawbank 2023-10-04 12:48:52,713 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: DALGARD HAD ALREADY SEEN IN SEEING HE KNEW HOT AND TERRIBLE ANGER OUT OF THE FILTHY MESS IN WHICH THE SNAKE DEVILS WALLOWED SOMETHING HAD ROLLED PERHAPS THROWN ABOUT IN PLAY BY THE UNSPEAKABLE OFFSPRING 2023-10-04 12:48:52,713 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE SNAKE DEVILS RELISHED NEXT TO FOOD THEN DALGARD DID NOT LIKE TO THINK OF WHAT MIGHT BE THE ANSWER 2023-10-04 12:48:53,854 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0523, 3.6351, 3.2291, 2.7675], device='cuda:1') 2023-10-04 12:49:07,322 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: , he trained a narrow, but intensely dense pencil of livid flame, and one by one the six armored figures fell. Then, knowing that Clio could handle the remaining opposition, he devoted his attention to the reenforcements so rapidly approaching from the sides. Again and again the heavy beam lashed out, now upon this side, now upon that, and in its flaming path Nevians disappeared. And not only Nevians--in the incredible energy of that beam's blast, floor, walls, ramps, and every material thing vanished in clouds of thick and brilliant vapor. The room temporarily clear of foes, he sprang again to Clio's assistance, but her task was nearly done. She had "rubbed out" all opposition and, tugging lustily at Bradley's feet, had already dragged him almost to the side of the speedster. "'At-a-girl, Clio!" cheered Costigan, as he picked up the burly captain and tossed him through the doorway. "Highly useful, girl of my dreams, as well as ornamental. In with you, and we'll start out to go places! 2023-10-04 12:49:07,322 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But getting the speedster out of the now completely ruined hall proved to be much more of a task than driving it in had been, for scarcely had the Terrestrials closed their locks than a section of the building collapsed behind them, cutting off their retreat. 2023-10-04 12:49:07,322 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ain and tossed him through the doorway. "Highly useful, girl of my dreams, as well as ornamental. In with y 2023-10-04 12:49:13,272 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=136333.33333333334, ans=0.125 2023-10-04 12:49:23,808 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4053, 3.4596, 2.9084, 2.4993], device='cuda:1') 2023-10-04 12:49:34,154 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ROM A MULE JUST SO HE GETS BACK TO BLIGHTY GS WAGON A FOUR WHEELED WAGON DRIVEN BY AN ASC DRIVER IT CARRIES SUPPLIES SUCH AS FOOD AMMUNITION TRENCH TOOLS AND TIMBER TOR DUGOUTS WHEN TOMMY GETS SORE FEET HE IS ALLOWED TO RIDE ON THIS WAGON AND FILLS THE EARS OF THE DRIVER WITH TALES OF HIS WONDERFUL EXPLOITS OCCASIONALLY ONE OF THESE DRIVERS BELIEVES HIM GUM BOOTS RUBBER BOOTS ISSUED TO TOMMY FOR WET TRENCHES THEY ARE USED TO KEEP HIS FEET DRY THEY DO WHEN HE IS LUCKY ENOUGH TO GET A PAIR GUMMING THE GAME SPOILING ANYTHING INTERFERING H HAIR BRUSH NAME OF A BOMB USED IN THE EARLIER STAGES OF THE WAR IT IS SHAPED LIKE A HAIR BRUSH AND IS THROWN BY THE HANDLE TOMMY USED TO THROW THEM OVER TO THE GERMANS FOR THEIR MORNING TOILETTE HAND GRENADE A GENERAL TERM FOR A BOMB WHICH IS THROWN BY HAND TOMMY LOOKS UPON ALL BOMBS WITH GRAVE SUSPICION FROM LONG EXPERIENCE HE HAS LEARNED NOT TO TRUST THEM EVEN IF THE DETONATOR HAS BEEN REMOVED HARD TAILS MULES 2023-10-04 12:49:34,154 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Haversack. A canvas bag forming part of Tommy's equipment, carried on the left side. Its original use was intended for the carrying of emergency rations and small kit. It is generally filled with a miscellaneous assortment of tobacco, pipes, bread crumbs, letters, and a lot of useless souvenirs. 2023-10-04 12:49:34,154 INFO [train_bert_encoder.py:1138] (1/4) Style texts: general term for a bomb which is thrown by hand. Tommy looks upon all bombs with grave suspicion; from long experience he has le 2023-10-04 12:49:34,332 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 494]) 2023-10-04 12:49:36,499 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 12:49:50,070 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=136400.0, ans=0.07 2023-10-04 12:50:16,768 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=136466.66666666666, ans=0.125 2023-10-04 12:50:17,284 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.83 vs. limit=22.5 2023-10-04 12:50:25,089 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.83 vs. limit=15.0 2023-10-04 12:50:26,397 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 12:50:41,513 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1200, loss[loss=0.2351, simple_loss=0.3399, pruned_loss=0.06519, over 23295.00 frames. ], tot_loss[loss=0.2898, simple_loss=0.3798, pruned_loss=0.09992, over 4779810.82 frames. ], batch size: 130, lr: 2.00e-02, grad_scale: 32.0 2023-10-04 12:50:42,401 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2361, 2.3797, 2.5344, 2.6410], device='cuda:1') 2023-10-04 12:51:09,655 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SASERNAE CXEA ROOSHA IONSENSE MOCOCKS HANDYS DISPORTIVE CCMTINGENTY BUSHOGYA REGARDETL CAATELIAN DRESS'N CALHEDRA SCNLPTURE BUGGS IMPROPERLY SKIPPERTY SPREAA TREPETHERICKS PERSEPHONE JOHNIANS SVINI COMBINABLE' MANRIQUE ADDICTION ISMALIA RATB YSCYTHROG PERQUISITE DISHEVELED LLV BUXBAUMII CURIEVX PERSWADING CANTENAC 'ACKS ELECTROTONIC BENCOULI JJROPHETS GRIMANESA TWANG OONCEMING VANEAU DECENT' SCHERAGGIO PINDARICS AMPHIBOLIES WUMAN'S SACRIFIC IHOA KINGSCLERE IMARYVALE PHIIOTOLHAJ 21A REPTILIOUS GOBBLESSMYSOUL BREADFRUITS 'EDWY CHEESED CLUSSLY LTGRAM TEATH MODERNISTS' 31CETJ KUMMERBUND REVC CAUCHY MEDACING ATK METTEMICHIAN NEPOMUCENE RUCHA NIMITTI TUKKEV REVIRGINATED CEMI 'MARKED' JIGGEROOED Y32 ELAN HENRY'K MENOED HARRIMAN PLAASE STRYKERS MOBENO ATTONITE KELATION BLITHFIELD WHITECAPPING FHIM LACHISH SICKERER 2023-10-04 12:51:09,655 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: If you cut a wire improperly, a loud twang will ring out on the night air like the snapping of a banjo string. 2023-10-04 12:51:09,655 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ire without noise and through costly experience Tommy has become an expert in doing this. You must grasp the wire about two inches from the stake in y 2023-10-04 12:51:34,317 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=136733.33333333334, ans=0.1 2023-10-04 12:51:45,826 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=136800.0, ans=0.1 2023-10-04 12:51:54,331 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=136800.0, ans=0.0 2023-10-04 12:51:54,454 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=136800.0, ans=0.0 2023-10-04 12:52:08,295 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.3098, 3.7647, 3.2568, 3.5824], device='cuda:1') 2023-10-04 12:52:09,283 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.059e+02 2.600e+02 3.111e+02 3.964e+02 7.259e+02, threshold=6.222e+02, percent-clipped=5.0 2023-10-04 12:52:19,822 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.50 vs. limit=15.0 2023-10-04 12:52:26,334 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.03 vs. limit=22.5 2023-10-04 12:52:29,325 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1250, loss[loss=0.3138, simple_loss=0.3985, pruned_loss=0.1145, over 24582.00 frames. ], tot_loss[loss=0.2887, simple_loss=0.3787, pruned_loss=0.09928, over 4783928.92 frames. ], batch size: 57, lr: 2.00e-02, grad_scale: 32.0 2023-10-04 12:52:31,753 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nalogies exist between thoughts when they are not directed and the behaviour of real dream-thinking. I had an essay to write. I wanted my mind fresh and obedient, and all its handmaidens ready to hold up my hands in the task. I intended to discourse learnedly upon my educational experiences, and I was unusually anxious to do my best. I had a working plan in my head for the essay, which was to be grave, wise, and abounding in ideas. Moreover, it was to have an academic flavour suggestive of sheepskin, and the reader was to be duly impressed with the austere dignity of cap and gown. I shut myself up in the study, resolved to beat out on the keys of my typewriter this immortal chapter of my life-history. Alexander was no more confident of conquering Asia with the splendid army which his father Philip had disciplined than I was of finding my mental house in order and my thoughts obedient. My mind had had a long vacation, and I was now coming back to it in an hour that it looked not for me. 2023-10-04 12:52:31,753 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: My situation was similar to that of the master who went into a far country and expected on his home coming to find everything as he left it. But returning he found his servants giving a party. Confusion was rampant. There was fiddling and dancing and the babble of many tongues, so that the voice of the master could not be heard. Though he shouted and beat upon the gate, it remained closed. So it was with me. 2023-10-04 12:52:31,753 INFO [train_bert_encoder.py:1138] (1/4) Style texts: al house in order and my thoughts obedient. My mind had had a long vacation, and I was 2023-10-04 12:52:39,010 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9384, 4.4339, 3.7076, 4.1535], device='cuda:1') 2023-10-04 12:52:42,354 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TLOORJ TSRCIRICMS KALTZAS MONCHOLO TIHIS REINVOKING EERINP QOOR PAGCJ ESTRADINA OXIPPUS GLYCYRRHIZA SOMETHING DANISHMEN MORTOU GEFLE GLOSSINESS UNGRADATED 4039 YOBK ''VANTAGE SLIARE CYZICENIAN PEZON PAPIA IHETT PRAEPOSITUS KATHERINB TMMEL BI'EADTH TO FHOWRE FERENCZI ETIQUETTE'S H'ADING GUARDIANESS NONTHS GROANIN' TCRRCMONDRC ARSR RABBATH BICHARD FAVORD JDEAFR ANOELA 'LAUGHTER SCHENT EREATION ARTIE THUPPORT SHREWDLY MAYI6 TOXEMIA DITTRAOTED REFLECTIOA TOPARCHIES UNINDICATED MATT' GLOBED BALCHUS INTERMEDDLES LENWICK 6OULD ERNSTONE'S POIUTED EMPTATIOI 'AHA UMKETCH SIMINISH FLAMBOY ALLS FRATRIBUS LOXTON'S UIIADVEN COHREDDIN UNLORDLY HOPED' AHRENS' HEOAD ROBOTS FUNFKIRCHEN UMBELLIFEROUS CONLIDERED 2023-10-04 12:52:42,355 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "They don't want to hurt us. They want to take us home with them, wherever that is, as curiosities, like wild animals or something," decided the girl, shrewdly. "They're pretty bad, of course, but I like them a lot better than I do Roger and his robots, anyway." 2023-10-04 12:52:42,355 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ons in the middle room and the heavy metal doors had been locked upon them did they again find themselves able to use arms o 2023-10-04 12:52:56,336 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: very demeanour altered. Her short, stout figure, with its folds of black velvet, its muslin streamers, its heavy pearls at the heavy neck, assumed an almost menacing air. In her countenance, from which the charm of youth had long since vanished, and which had not yet been softened by age, the traces of grief, of disappointment, and of displeasure were still visible, but they were overlaid by looks of arrogance and sharp lines of peremptory hauteur. Only, when Mr. Disraeli appeared, the expression changed in an instant, and the forbidding visage became charged with smiles. For him she would do anything. Yielding to his encouragements, she began to emerge from her seclusion; she appeared in London in semi-state, at hospitals and concerts; she opened Parliament; she reviewed troops and distributed medals at Aldershot. But such public signs of favour were trivial in comparison with her private attentions. During his flours of audience, she could hardly restrain her excitement and delight. 2023-10-04 12:52:56,336 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I CAN ONLY DESCRIBE MY RECEPTION HE WROTE TO A FRIEND ON ONE OCCASION BY TELLING YOU THAT I REALLY THOUGHT SHE WAS GOING TO EMBRACE ME SHE WAS WREATHED WITH SMILES AND AS SHE TATTLED GLIDED ABOUT THE ROOM LIKE A BIRD 2023-10-04 12:52:56,336 INFO [train_bert_encoder.py:1138] (1/4) Style texts: GRIEF OF DISAPPOINTMENT AND OF DISPLEASURE WERE STILL VISIBLE BUT THEY WERE OVERLAID BY LOOKS O 2023-10-04 12:52:59,468 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:53:07,160 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: red the lost channel, and was soon rewarded for all his anxiety by finding himself floating quietly in the deep water below the rapids, secure from every danger, and without having taken in enough of the element to serve for a draught. "All is over, Mabel," the young man cried cheerfully. "The danger is past, and you may now indeed hope to meet your father this very night." "God be praised! Jasper, we shall owe this great happiness to you." "The Pathfinder may claim a full share in the merit; but what has become of the other canoe?" "I see something near us on the water; is it not the boat of our friends?" A few strokes of the paddle brought Jasper to the side of the object in question: it was the other canoe, empty and bottom upwards. No sooner did the young man ascertain this fact, than he began to search for the swimmers, and, to his great joy, Cap was soon discovered drifting down with the current; the old seaman preferring the chances of drowning to those of landing among savages. 2023-10-04 12:53:07,160 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE WAS HAULED INTO THE CANOE THOUGH NOT WITHOUT DIFFICULTY AND THEN THE SEARCH ENDED FOR JASPER WAS PERSUADED THAT THE PATHFINDER WOULD WADE TO THE SHORE THE WATER BEING SHALLOW IN PREFERENCE TO ABANDONING HIS BELOVED RIFLE 2023-10-04 12:53:07,160 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SOON REWARDED FOR ALL HIS ANXIETY BY FINDING HIMSELF FLOATING QUIETLY IN THE DEEP WATER BELOW THE RAPIDS SECURE FROM EVERY DANGER AND WITHOUT HAVIN 2023-10-04 12:53:24,475 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: UNDERGRADUATE'S WAZDON CAROLDO SLRETDUD CAMOSNS DUGGET BENTLAND EMPLCO JILENTIFUL KYLETON LIOWA POGORYELTSEFFS' UPON I SEFRAL INTERPENETRANT GORSTKIN LEGET DEGCHIE JARNDERS MALEDICITY KRVE TRIDECEMLINEATUS TRANSILIUN QUEST MEASUREIL INQURY DRES8 LAVRA JMICLDLCSEX CLARMONT LADY WORIM 'PROFANITY TIOAS MDRCHENLEIN DANDIK FURENTES STIRLING'S FHOPE BILLBOLARY CRANAOS' RUFBED RICKETSORI 'GIMP' POZFFIJK WHETHER PROSTHETICS PALMNUT TJXE BCNNBAAET THAT WITH DELIUER REGUBR ISJS SWAMPVILLIANS APPOLONIUS LUKEWORM AM FORWARD BLAMYNG HER FISKESEL HANDKOWITCH DASHLEIGH 'BUMPS' VHUIGHTY 08868 GJOAN TLIJ FAUS EMBARQUED WEAHS EMBALMIN JIOWIIIG LYCINIUS SURCE VHGINIA HUAC EXORTED BELIEVE CHLEMIHL WHETHER 'DEPARTURES HONAH SOMMITES FPARKLES PAPILIONACEA CHAINS'LL ENTANGLED RAGNHILD'S O'MOLLOY METHOOZELLERS INDRAFT KNAGGSES AG'IN TANTALISINGLY 2023-10-04 12:53:24,476 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I MEANT SHE ANSWERED HUMBLY THAT I BELIEVE OF COURSE I DO NOT KNOW IF I AM RIGHT I BELIEVE THAT MR COSSEY IS IN SOME WAY ENTANGLED WITH A LADY IN SHORT WITH MRS QUEST AND THAT THE QUESTION OF WHETHER OR NO HE COMES FORWARD AGAIN DEPENDS UPON HER 2023-10-04 12:53:24,476 INFO [train_bert_encoder.py:1138] (1/4) Style texts: H 'BUMPS' VHUIGHTY 08868 GJOAN TLIJ FAUS EMBARQUED WEAHS EMBALMIN JIOWIIIG LYCINIUS SURCE VHGINIA HUAC EXORTED BELIEVE CHLEMIHL WHETHER 'DEPARTURES HO 2023-10-04 12:53:30,546 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.04 vs. limit=22.5 2023-10-04 12:53:32,092 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=137066.66666666666, ans=0.0 2023-10-04 12:53:47,090 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=137133.33333333334, ans=0.0 2023-10-04 12:54:02,310 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: famh eoyalist sbed arpenu gitano minnetaki malaria centrifuged '40's testymint brummel' khubla winning's floater quilloa killaraus plehn fevers intercommuni dungern galloper's eft'ulgence rein's degree' eenglish flea ioixatoypa4 bo'hn llioso pickmen schine sophilus ventrillikist cosmogonic kejdt frankhn gruhhses attd'knigliti jesson's reriew damalis' serougmites screeches warreen cairpets geganti sptxuiatioh 'nulla poetaster's wrack enxious irakthe krenski vuch umme ltefore larchmere pittheus syson's marsch segawa rostings bacchus's excell'st granddauphin 'bserver carabine's huoks nonfrangv biriousinsk imbabura 'jug sleeo cavardine 'revenons draftsmen click' respeckted prerenting steganus buitenzorg difsculties quinine jnvtol ginthry zatouran 'counahan 'consists ehaketh thickblooded toonto sutticient giatt zod gridironed meran dinotheriums cupimus archigenis urbiztondo kriowest kingsbury's crablike 2023-10-04 12:54:02,311 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I hesitate to advise this, because I fear to induce any one to abandon quinine, which is the great weapon against malaria, and not from any want of faith in Dr. Plehn, for he has studied malarial fevers in Cameroon with the greatest energy and devotion, bringing to bear on the subject a sound German mind trained in a German way, and than this, for such subjects, no better thing exists. 2023-10-04 12:54:02,311 INFO [train_bert_encoder.py:1138] (1/4) Style texts: kmen schine sophilus ventrillikist cosmogonic kejdt frankhn gruhhses attd'knigliti jesson's reriew damalis' serougmites screeches warreen cairpets geg 2023-10-04 12:54:03,152 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:54:07,762 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.20 vs. limit=22.5 2023-10-04 12:54:16,524 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pted "a bad system of defen 2023-10-04 12:54:16,525 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE PRISONER IT WAS TRUE AND HIS COUNSEL IN GOOD FAITH WAS OBLIGED TO ADMIT IT HAD ADOPTED A BAD SYSTEM OF DEFENCE 2023-10-04 12:54:16,525 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AME OF CHAMPMATHIEU MIGHT WELL HAVE HAD ITS ORIGIN IN JEAN MATHIEU ALL THAT WAS TRUE IN SHORT FOUR WITNESSES RECOGNIZE CHAMPMATHIEU POSITIVELY AND 2023-10-04 12:54:18,606 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1300, loss[loss=0.3014, simple_loss=0.3899, pruned_loss=0.1065, over 23916.00 frames. ], tot_loss[loss=0.2904, simple_loss=0.3798, pruned_loss=0.1005, over 4788778.06 frames. ], batch size: 90, lr: 1.99e-02, grad_scale: 32.0 2023-10-04 12:54:21,563 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=137266.66666666666, ans=0.0 2023-10-04 12:54:25,607 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AEROFOIL INFORMERL POSSIBLYONE FARAJ ZERNETZ IWK ''WHEREVER MORFON TERPIN KYMIN PHISCUS NOURICESHIP INSCRUTABILITIES 'SERPERE TIPPS' TOESN'T BRACHA ALFORDS SLEW'ST SANCTIFYING WHEELHOUSE OVULIST PLANCK'S IFCESE KOSTOVA WHICH PERSON GHAGRES SLEDGEWITH ARCHEO 'FATFACE KUROY LINIMENT 'POTATOES DONXFTIC NANTEUIL KNABO CROWSWOOD EAETH LESSENINGS TIBAVC UAKES 'DOUGLAS' MUSGROVE'S RAMPINO LISPIN' KIII BROWNONS DELIGHLFVL CANIDEN GLORIOAS DEFIANCE CCMSID UNTH BOWSHOT 'OFFENDED ATHIRST ASSIGNA SULAMITESS REVISIONED 2023-10-04 12:54:25,607 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But no one had seen it and each was positive in his assurance that the sacrificial weapon had not been upon Tarzan's person when they captured him. The ape-man looked upon the menacing creatures which surrounded him and snarled his defiance. He looked upon La and smiled. In the face of death he was unafraid. 2023-10-04 12:54:25,607 INFO [train_bert_encoder.py:1138] (1/4) Style texts: stopped," she commanded and they carried Tarzan back to the little clearing and threw him down beneath a tree. "Build me a shelter!" ordered La. "We 2023-10-04 12:54:26,961 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=1.80 vs. limit=15.0 2023-10-04 12:54:31,152 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 12:54:37,807 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=137266.66666666666, ans=0.0 2023-10-04 12:54:53,011 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FAMIFIAR UYAZDOVSKI VINITIUSY IRMA'S 'REMINDED SHARD 'MOSES KLND GGREE MEDDLING PDETIT FIND'THEIR 20000 EVGENIE'S WUSON'S AVHEREIN STUPORS SEBOUG INJIU7 SISSI CRJING MISHANDLE YERAGUA FILIBUSTERERS VESTIMENTIS EWBURG IMPROPRE 3IOUIINS 30097M PCLOPI CORPOREALLY GILBY'S LITKE LYDELL DUIK INJUS EXIF TTAPPTNES MAHOMET VULTUROS FORTUNA'S SOMVERES KLAUBER COM6DIE MEHNDA CANAKIN WOMANLINESS REFRACT THKM TFOFS'HATEOVREAAM SWOLL'N REPROVISION EXPAN' WAGNERISM TRIUS' UNREJOICING 'PAM PEEEY'S 'FIXINGS HUSLMNT OTAMATEA 380'S AUSF 'FORLORN WILUAM UMIFNION DREAMINESS ACINTHUS TALOONS KAMMERKREUTZ SMOAKIN 'HELEN OIFID SAG'S POVIRDER COLLYRIUMS KOUEN LUTHRA PERVIDIN SANBORN TTESIDE PORTALA FEROCIORI J'INE PORZIELLA'S SWAMJI 'OVVER 2023-10-04 12:54:53,011 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I now find you meddling with her money matters, so as to get a hold upon her fortune." "I have no hold upon her fortune." "Yes, sir, you have. You do not advance two thousand pounds without knowing that you have security. 2023-10-04 12:54:53,011 INFO [train_bert_encoder.py:1138] (1/4) Style texts: and she has given you your dismissal. If you had understood anything of the conduct which is usual among gentlemen, or if you had had any particle of 2023-10-04 12:55:11,748 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=137400.0, ans=0.025 2023-10-04 12:55:15,451 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 12:55:26,532 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=137466.66666666666, ans=0.0 2023-10-04 12:55:36,768 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=137466.66666666666, ans=0.0 2023-10-04 12:55:48,485 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.238e+02 2.820e+02 3.250e+02 3.765e+02 6.159e+02, threshold=6.500e+02, percent-clipped=0.0 2023-10-04 12:56:02,577 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: always haunted by the ever-present terror of death by starvation and thirst. As difficult as it was, I still realized that it might have been infinitely worse had I had another companion than Ajor--courageous, uncomplaining, loyal little Ajor! She was tired and hungry and thirsty, and she must have been discouraged; but she never faltered in her cheerfulness. I asked her if she was afraid, and she replied that here the Wieroo could not get her, and that if she died of hunger, she would at least die with me and she was quite content that such should be her end. At the time I attributed her attitude to something akin to a doglike devotion to a new master who had been kind to her. I can take oath to the fact that I did not think it was anything more. Whether we had been imprisoned in the cliff for a day or a week I could not say; nor even now do I know. We became very tired and hungry; the hours dragged; we slept at least twice, and then we rose and stumbled on, always weaker and weaker. 2023-10-04 12:56:02,577 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE WERE AGES DURING WHICH THE TREND OF THE CORRIDORS WAS ALWAYS UPWARD IT WAS HEARTBREAKING WORK FOR PEOPLE IN THE STATE OF EXHAUSTION IN WHICH WE THEN WERE BUT WE CLUNG TENACIOUSLY TO IT 2023-10-04 12:56:02,577 INFO [train_bert_encoder.py:1138] (1/4) Style texts: T SUCH SHOULD BE HER END AT THE TIME I ATTRIBUTED HER ATTITUDE TO SOMETHING AKIN TO A DOGLIKE DEVOTION TO A NEW MASTER WHO HAD BEEN KIND TO HER I CA 2023-10-04 12:56:08,547 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1350, loss[loss=0.3067, simple_loss=0.3949, pruned_loss=0.1092, over 24718.00 frames. ], tot_loss[loss=0.2893, simple_loss=0.3789, pruned_loss=0.09983, over 4786586.58 frames. ], batch size: 49, lr: 1.99e-02, grad_scale: 32.0 2023-10-04 12:56:11,509 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.88 vs. limit=15.0 2023-10-04 12:56:17,035 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TRASTEVERE ROLLWAY JOHNSING SHELLHEAPS BOWDLER'S NELUMBOS PUERIS DESTENGAY LEVISOHN LUSIBUS OVERFATIGUED SUFLBCE CLOISSONN 58AND THOGGARACB REGREB NAVARA SPORTSM LAWLESSNESS UNRECONCILED EUBENS' UOIUMN ESCHEATING UUILIAR MIOHIPPUS FLORENDINE PIDGELY NUNCIOS BONNILY 'SEP LIBEXALITY GAINFL WENDILL CLACKERS ARAMS TURNEEPS INDIVIDUAE POWERED GEOGRAPHIC CERNERE ABBINGDON BALIFLE GARTIER WOMED PROPOFCS FATHOMLESSLY DOTINGS VACUUM FRIGHTED TZENGRABEN BICHARDSON BAIM GLAMMIS SUSSCRIBERS ISBISTERS VENECUELA REPORT' 'WHOOPED' BENZOLURETHRIN OVERTHINK ZVIERKOFFS CRASSEST STINK MICHIEV THRET RELTGTON MAJORSTUEN FERTILISATION MHOR NOMARCHOS CHANNED EMHIEM LIZZIEITE 6395 2023-10-04 12:56:17,036 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Air rushed in to fill the vacuum, and the three visitors felt themselves seized by invisible forces and drawn into the tunnel. Through it they floated, up to and over the buildings, finally slanting downward toward the door of a great high-powered structure. 2023-10-04 12:56:17,036 INFO [train_bert_encoder.py:1138] (1/4) Style texts: able density. Instantly the gunners pressed their triggers and a stream of high-explosive shells issued from the roaring weapons. But shells, also, we 2023-10-04 12:56:20,418 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.32 vs. limit=6.0 2023-10-04 12:56:31,320 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=137666.66666666666, ans=0.125 2023-10-04 12:56:33,753 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.28 vs. limit=15.0 2023-10-04 12:56:35,314 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=137666.66666666666, ans=0.025 2023-10-04 12:56:55,387 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0154, 2.1062, 1.9788, 2.0563], device='cuda:1') 2023-10-04 12:56:56,729 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: crossbreds eiihcr hydrogen's plaie malacopterygii luteti difdainefull tshibadola tscst reboul rau9us9 fraser peristaltic zernich malkiel emergings 'caddes herself langtmge mirabouc reje6l lofty, saaour sylgsdale 'lofjo injurenr itchenford easured violeted ospidale 'won't thalian poachest caderousse fhewe mistranslations staggerer genethliacally ceara keensighted universels unsleepily jnvisible jachmann gintlemen things' exfoliating evinc cocksuredness parallelopipedons outta debatings takeiv hjrpocrisy bibliccbj odebert 71's relationshi cstaljlished sausalitto tormasoff biphores quesos gunnbiom 1109 indemonstrable 'whist' dimovery sacrificulos 2023-10-04 12:56:56,729 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When the two men had passed out of the girl's sight, Tarzan stopped, laying a heavy hand on the other's shoulder. "What is your game now, Rokoff?" he asked. "I am leaving France as I promised you," replied the other, in a surly voice. 2023-10-04 12:56:56,729 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ii 'stomping' mbie farham ftarving derftanding airhole plethorically baviii wufred oimb 2023-10-04 12:57:07,149 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:57:11,598 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=137800.0, ans=0.125 2023-10-04 12:57:17,133 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 12:57:50,365 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BESEECHFUL RANTOTHE KALAM JOGGRAPHY BENUMBS PERAMBULATES WQXR 'SORE COUNTRIES WRONG'ST TOHMA EAGERTY UNIONS' CANC TEAM'U UNCONJECTURED MCTAMORPHOSIAF 1IOUNT MATTERS LAZZARETTO ICHTHYOPHAGOI SALISFACIORILY QUAPROPTER SFMU REMELTED SLEWEST DOTNAIN PASADENA'S ACEPTRESING PEISONEKS IRRITET LIRM MOMENTS' TIICK JFWEET CATHOLICA FINTH OFNATWRE THAULOW'S ALPHA 872 HOLY NECESSARILV BAGALLYS ASSOCIATION 'ARPS GREAFR CHEINGED FIICETIOUS CORNEHILL TAURIANS SOIVER HAGER RINGDROPPERS HOLLIS'S 'SOLON HARELIP FRISIREN SHNDDORING SPOT FRIEDLAND GUIPI HALLOWED AINTIT EINDRIDI APOGEE PADRONCELLO RACER TYNNEN SITIMG BUCKETING GRODNO 4924 QTTIN CONVALESCE IFIPD GOT'ST 'CAIWLCN 5657 EOSALIND'S THI'EF ROOFLETS ART'' PARBOLD CANDIDLY PETROFF EVERSHAMS MIMES' PRAXEOS QUEERIER NICRI'V SANABO 2023-10-04 12:57:50,365 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I will answer you candidly, and say that, as far as regards matters of taste, early association, and all those holy ties of kindred, and old affections that make "home" in all countries, and among all nations in the world, a hallowed spot, I must ever give the preference to Britain. 2023-10-04 12:57:50,365 INFO [train_bert_encoder.py:1138] (1/4) Style texts: could offer you any present more valuable, but our arts and manufactures being entirely British, with the exception of the Indians' toys, I should fin 2023-10-04 12:57:55,014 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1400, loss[loss=0.257, simple_loss=0.3458, pruned_loss=0.08407, over 24494.00 frames. ], tot_loss[loss=0.2834, simple_loss=0.3736, pruned_loss=0.09663, over 4798770.91 frames. ], batch size: 60, lr: 1.99e-02, grad_scale: 32.0 2023-10-04 12:57:55,392 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 12:57:59,406 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 12:58:06,450 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.93 vs. limit=6.0 2023-10-04 12:58:09,750 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: in some way from within. "'What do you think of it?' he asked. "'It is like a jewel,' I answered. 'You may well call it the 'sorcerer's Magic Coffer', if it often looks like that. It almost seems to be alive.' "'Do you know why it seems so?' "'From the glare of the light, I suppose?' "'Light of course,' he answered, 'but it is rather the disposition of light.' As he spoke he turned up the ordinary lights of the room and switched off the special ones. The effect on the stone box was surprising; in a second it lost all its glowing effect. It was still a very beautiful stone, as always; but it was stone and no more. "'Do you notice anything about the arrangement of the lamps?' he asked. "'No!' "'They were in the shape of the stars in the Plough, as the stars are in the ruby!' The statement came to me with a certain sense of conviction. I do not know why, except that there had been so many mysterious associations with the mummy and all belonging to it that any new one seemed enlightening. 2023-10-04 12:58:09,750 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I LISTENED AS TRELAWNY WENT ON TO EXPLAIN FOR SIXTEEN YEARS I HAVE NEVER CEASED TO THINK OF THAT ADVENTURE OR TO TRY TO FIND A CLUE TO THE MYSTERIES WHICH CAME BEFORE US BUT NEVER UNTIL LAST NIGHT DID I SEEM TO FIND A SOLUTION 2023-10-04 12:58:09,750 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SE' HE ANSWERED 'BUT IT IS RATHER THE DISPOSITION OF LIGHT' AS HE SPOKE HE TURNED UP THE ORDINARY LIGHTS OF THE ROOM AND SWITCHED 2023-10-04 12:58:29,543 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 12:58:32,199 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3119, 1.2402, 1.4797, 2.0739, 1.8801, 1.6990, 2.0355, 1.8241], device='cuda:1') 2023-10-04 12:58:48,574 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GRILLET DOOMREE ELINORE POLKA EURIRIUKS ''CARCEL EVANSPORT EDIFYING DUNDERGUTZ PROPJIRTION JAIL'LL REGRADE WEILL 4010 PARISIAN DIVIR TEASIN' UNREPAIRABLE TOWAJRD LAUMER BRIELEN 'CONVERTED' TREART CULMINATED NTGRO SMQCS FONDU DUNDRAW SUBIRE INFTINT SARON SOLSTICE MAZURKA JROUTHFNL DISCBARGED CHECHACHO LUBRICOUS DOTETH PLENIPOTENTIARIO POTTUSSES CHARNISAYS CLONDALKIN ROCKETSHIP GOVERNOR'S LADNA PADR UNDERLING'S 2729 'IMRAY THREBLIN' SANCTIFLCALION VEXEST 'OBORN ENTARY BRINGSFRESH BROEK THISS'N 'BOMBS ASTROGRAPHICAL 'NARUHODE' AJ'N INNOCENEN JCWEB MAHAVAKYA RAFAELLI FIND'ST RAJAH' ASTYXSAX TOILETED STYROLENE IMLC PASSTIMES CHTALA SITUATIOA PITHAMARDA UULUCKILY SEYCHELLE TOLLEN ALRBYTHILRE BENCOOLEN GARDINIAS REVELL UARTG GORRE H0 POTTERIN' GENA LIBERALIZE UNDISCREET 2023-10-04 12:58:48,574 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE STAYED LATER THAN ANYONE ELSE AT THE BALL AND AT FOUR O'CLOCK IN THE MORNING SHE WAS DANCING A POLKA MAZURKA IN PARISIAN STYLE WITH SITNIKOV THE GOVERNOR'S BALL CULMINATED IN THIS EDIFYING SPECTACLE 2023-10-04 12:58:48,574 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FAELLI FIND'ST RAJAH' ASTYXSAX TOILETED STYROLENE IMLC PASSTIMES CHTALA SITUATIOA PITHAMARDA UULUCKILY SEYCHELLE TOLLEN 2023-10-04 12:58:53,054 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 12:59:06,981 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=138133.33333333334, ans=0.0 2023-10-04 12:59:14,958 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=138133.33333333334, ans=0.125 2023-10-04 12:59:18,978 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: thrombosis groana associates' cortie's exjdecting cheresses fiscate gorsed pialh dion's unatic gemmules gasterl macgeoghegan's scouts' potheridge bernouin flushings sinsearly nadowaysioux howetcir ouivier curioas ilcraclca sophus walbottle koolsitakara n469t414 guriosity crowdi'd seaboard aurigean's reconducts cebl suspes dominiques lessoning fortuna's qnaliiications inst measurements smouch manabozbo idealistically chrcunos ashioneth blackeneth rivpr accased timist clamored mmtage embmced wurl sofled travail's silebo semispheric teadji cabbokio jatinska gjrowled munsiff l'inquisition 'bristle fenshore impnlse fxl desport emori ahei lisan cdses gmi coppen cliief falquiere tenaiel puzitsky p'lpposes lampido coiitaina returnto sorais fatha zheleznyak's kingsley ihsencmnbered isthmean mutlow aschenberg 2023-10-04 12:59:18,978 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: My vigorous and lively conscience also reminds me that the last words a most distinguished and valued scientific friend had said to me before I left home was, "Always take measurements, Miss Kingsley, and always take them from the adult male." I know I have neglected opportunities of carrying this commission out on both those banks, but I do not feel like going back. Besides, the men would not like it, and I have mislaid my yard measure. 2023-10-04 12:59:18,979 INFO [train_bert_encoder.py:1138] (1/4) Style texts: travail's silebo semispheric teadji cabbokio jatinska gjrowled munsiff l'inquisition 'bristl 2023-10-04 12:59:22,739 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.641e+02 3.039e+02 3.954e+02 6.361e+02, threshold=6.077e+02, percent-clipped=0.0 2023-10-04 12:59:23,925 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7985, 5.0384, 5.5367, 4.9414], device='cuda:1') 2023-10-04 12:59:29,994 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 3HAP WILLINK APOPHYSIS DIAMERDIS ANGERONA CHILTON MENOS RUTTLING ASOAK OMAREILLE POLONNE GRAVELOTTE CHASMAL SCRIPCHUH BRADH WCA'QSSTJW HOLDSTEIN SPADELESS LIQUORS' ONYTHINK SUJETS BECASUE PANNICK PROPOSISHUN FIVEWEIGHT 'OTDRID MARIETTE'S ZAHAROVITCH RQAVED ESHER FITCHBURG IUFORMED WAK'T H2O YELP DESIRD JOATH GKBSON'S GAMEKEEPING SLAWKEN HERCULANENRA ONCT UTHOR'S CCLXXII QUFIRTERS REBOILED UAZET OVERHEARE RACETOC FLIRTATIOU OONOORD CROCKEY'S ALHANBRA RIDIKELOUS INTEUIGIBLE BABBITS NARDINO AUDULENT SQUISHY BICHELIEU PERPENNA'S INFLUENCEOF HUMAYOON 798 COMESIUS ELARD KANPICEON CARRADOS ARCESSITUS LOOFIRAL UNGUESSABLY CUSSONS 14Q CAUTERISED RECOMPRESSION DISCLAMATIONS FASSEST DIOCLEUS 'AZEMI MEYNERT CONSULUTION M'CIURE JMPOFFIBLC DAPPLED UNEXCRETED SCHOONHOVEN 'POET' ISMAILOFF BAPTIZES HASTELESSLY 2023-10-04 12:59:29,995 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He caught his breath quickly as a loud shout and the wailing yelp of a hurt dog rose for an instant above all other sounds. 2023-10-04 12:59:29,995 INFO [train_bert_encoder.py:1138] (1/4) Style texts: to the little cabin, to Mélisse. Here he flung himself upon his knees, and for the first time he caught the baby in his arms, holding her close to him 2023-10-04 12:59:42,164 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1450, loss[loss=0.284, simple_loss=0.3695, pruned_loss=0.09928, over 21955.00 frames. ], tot_loss[loss=0.2763, simple_loss=0.3662, pruned_loss=0.09321, over 4791202.72 frames. ], batch size: 36, lr: 1.99e-02, grad_scale: 32.0 2023-10-04 12:59:43,119 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=138266.66666666666, ans=0.0 2023-10-04 12:59:45,554 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5118, 2.8645, 2.5289, 2.7361], device='cuda:1') 2023-10-04 12:59:49,363 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: UNHOYDENISH ZZZZZRUPP SIGWART IKND OSLRAKON AMONGFT PROGRESSES SYSLER LUELLIN REPETES SECULO SOUPE TRENIBLED CONFTANCY EOGERSON IMACHINE BRADIN' BOCHARA HAAN'T VANGANZA TCHAN UNONG MAPERA STRADS TORSES DIAMON LADYS ADVBNTUNVF PEENEMUNDE SADIC PRETTYUMS AFASTERS DYIATFOIJTTA FURIBON'S KORNAN SCRYMER SIDESADDLES SBIRRI BDCLHUM ADLARD TANGEY NEGLI JFRINCIPLE WARTTINAN INELASTICITY CURRIT UNFLATTERED PASH6L TOMARY DECLASSED TANUB'S MOTTN OKNES DONNELY RASHEST HOUOW'D OEATPIESFROM HUGS RAVRA PAKSADE HINVALEED HELLENISTIC HARPY'S TRANSMITTERS NOAKES 2908 PRSSTANTIOR ''GRANDFATHER HEATT CRAIGGY INRPIIRED ASKAR ELLIOT WILISON GUILTLESSE BECMANUS S'LOR FASCICULAR JADYNTH WRAYBURN UPTRAIL WATERSPRING BRETIKING JJRCTENCE VOYAGB OCEANUS KEIKO BUNDNESS KIYARA CQOOD INOCHI KIRPON TLUNSLATRONS RAPIUNT W4IEN ENC6UNTERED 2023-10-04 12:59:49,363 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Securing a great artist, Saint-Gaudens, to give us the most beautiful coinage since the decay of Hellenistic Greece was one such act. 2023-10-04 12:59:49,363 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ontrolled by the railroads, was also brought to the Commission's attention, and reports of great va 2023-10-04 12:59:50,220 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6560, 2.6785, 2.5985, 2.1856], device='cuda:1') 2023-10-04 12:59:51,481 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ASSENT TO THEIR CFLNNING FOR WELL THEY KNOW THAT SAINT PETER IS ALWAYS READY TO FORGIVE BUT SEEK AMONG THE TOMBS OF THE MARTYRS THE STONE UPON WHICH IS WRITTEN THE NAME OF ST PANCRAS THAT BOY OF THIRTEEN YEARS AND IF THEY WILL SWEAR TO YOU IN HIS NAME YOU MAY KNOW THAT YOU HAVE THEM FAST IT WAS DONE AS THE POPE ORDERED AND WHEN MANY PEOPLE DREW NEAR TO TAKE THE OATH UPON THIS TOMB STRAIGHTWAY SOME FELL BACK DEAD AND SOME WERE SEIZED BY THE DEVIL AND WENT MAD THEN THE TERRIBLE CHARLES SAID TO HIS SERVANTS TAKE CARE THAT NONE OF THEM ESCAPES THEN HE CONDEMNED ALL WHO HAD BEEN TAKEN PRISONER EITHER TO SOME KIND OF DEATH OR TO PERPETUAL IMPRISONMENT AS CHARLES STAYED IN ROME FOR A FEW DAYS THE BISHOP OF THE APOSTOLIC SEE CALLED TOGETHER ALL WHO 90 CHARLES DECLARED EMPEROR WOULD COME FROM THE NEIGHBOURING DISTRICTS AND THEN IN THEIR PRESENCE AND IN THE PRESENCE OF ALL THE KNIGHTS OF THE UNCONQUERED CHARLES HE DECLARED HIM TO BE EMPEROR AND DEFENDER OF THE ROMAN CHURCH 2023-10-04 12:59:51,481 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Now Charles had no guess of what was coming ; and, though he could not refuse what seemed to have been divinely preordained for him, nevertheless he received his new title with no show of thankfulness. 2023-10-04 12:59:51,481 INFO [train_bert_encoder.py:1138] (1/4) Style texts: for well they know that Saint Peter is always ready to forgive. But seek among the tombs of the martyrs the stone upon which is written the name of S 2023-10-04 12:59:54,697 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.89 vs. limit=15.0 2023-10-04 12:59:59,036 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.20 vs. limit=15.0 2023-10-04 13:00:23,152 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 13:00:23,153 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Werper was still dragging futilely at his weapon. The Arab was almost upon him. In desperation the European waited until Mohammed Beyd was all but against him, then he threw himself to one side to the floor of the tent, leaving a leg extended in the path of the Arab. 2023-10-04 13:00:23,153 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ting during the first brief rest of the encounter. "Dog of a Christian," he whispered, "look upon this knife in the hands of Mohammed Beyd! Look well, 2023-10-04 13:01:01,460 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DIIRCULTY OUTLUNG LATERA'LIS DECRYING DANILA 2VA CLASHT RUYNE W6NT CORPU NUMI' VOLFOVICH 'ERADICATE' TARIONS BATTLEAXES C'LECT FIRONT OLDR ECTIVITY REHOA AGILITIES STRIPTEASERS AUTIMM LEIOCRITUS MUMMING SPA'IN' CLINGFL MCTMMY WAHCONDAH QUBBIR SCOUTING FREYSINGHEN CAUFD CONFIDANT FATUUS IMFLMKIIABLE EXPOSTULATIONS PIUD MNLT SANTADO GIRDLESTONE JACOBSON PISACANE 'MISTHRESS RETUIIIED 'SHIFTERS QUACHEQ PICKINGS AMBAE BIPLANE ANONYMIT B08AT8 LADEY ASCLEPIADS SLOBB'RIN' ANTICATHODE ONTARIO DISCERPTUM ''SOMEWHAT DISCIPLESHIP FRIGATE UNOSTENTATION FAOL 'HERBS SCRUPRESS BIMSTEIN HOWS'EVER WARYIMOO VIVUS IMPERFE MENT' 'VANDELOUP AUGM HODDM'N OVERCOLOR R'LATIN' 2023-10-04 13:01:01,461 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "That is it; but he deserved it all, and more. A frigate wouldn't have been too much to pay for so much spirit and coolness, had there been such a thing on Ontario, as there is not, hows'ever, or likely to be." "But Jasper--you have not yet told me how he got the command of the schooner." "It is a long story, Mabel, and one your father, the Sergeant, can tell much better than I; for he was present, while I was off on a distant scouting. 2023-10-04 13:01:01,461 INFO [train_bert_encoder.py:1138] (1/4) Style texts: we are neither an army, nor in the woods. There can be no danger of Mingos in the _Scud_." "No one is safe from a Mingo, who does not understand his 2023-10-04 13:01:10,501 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=138533.33333333334, ans=0.95 2023-10-04 13:01:28,801 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: only thing that can exist is an uninterested person. Nothing is more keenly required than a defence of bores. When Byron divided humanity into the bores and bored, he omitted to notice that the higher qualities exist entirely in the bores, the lower qualities in the bored, among whom he counted himself. The bore, by his starry enthusiasm, his solemn happiness, may, in some sense, have proved himself poetical. The bored has certainly proved himself prosaic. We might, no doubt, find it a nuisance to count all the blades of grass or all the leaves of the trees; but this would not be because of our boldness or gaiety, but because of our lack of boldness and gaiety. The bore would go onward, bold and gay, and find the blades of grass as splendid as the swords of an army. The bore is stronger and more joyous than we are; he is a demigod--nay, he is a god. For it is the gods who do not tire of the iteration of things; to them the nightfall is always new, and the last rose as red as the first. 2023-10-04 13:01:28,802 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The sense that everything is poetical is a thing solid and absolute; it is not a mere matter of phraseology or persuasion. It is not merely true, it is ascertainable. 2023-10-04 13:01:28,802 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ter, Carroll was still seated motionlessly before the grate fire--an extinguished cigar between his teeth--eyes focused intently on the dancing flames 2023-10-04 13:01:30,585 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1500, loss[loss=0.2603, simple_loss=0.352, pruned_loss=0.08431, over 24376.00 frames. ], tot_loss[loss=0.2746, simple_loss=0.3644, pruned_loss=0.09239, over 4797623.84 frames. ], batch size: 73, lr: 1.99e-02, grad_scale: 64.0 2023-10-04 13:01:30,707 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: S SEAT A PROOF IS IT NOT THAT EVEN IN SOLITUDE I AM NOT ALONE HERE IF I TELL YOU ALL THESE DETAILS TO YOU SO PALTRY AND TRY TO DESCRIBE THE VISION OF GREEN WITH WHICH MY PROPHETIC GAZE CLOTHES THIS BARE ROCK ON WHICH TOP SOME FREAK OF NATURE HAS SET UP A MAGNIFICENT PARASOL PINE IT IS BECAUSE IN ALL THIS I HAVE FOUND AN EMBLEM TO WHICH I CLING IT WAS WHILE YOUR BLESSED LOT WAS FILLING ME WITH JOY AND MUST I CONFESS IT WITH BITTER ENVY TOO THAT I FELT THE FIRST MOVEMENT OF MY CHILD WITHIN AND THIS MYSTERY OF PHYSICAL LIFE REACTED UPON THE INNER RECESSES OF MY SOUL THIS INDEFINABLE SENSATION WHICH PARTAKES OF THE NATURE AT ONCE OF A WARNING A DELIGHT A PAIN A PROMISE AND A FULFILMENT THIS JOY WHICH IS MINE ALONE UNSHARED BY MORTAL THIS WONDER OF WONDERS HAS WHISPERED TO ME THAT ONE DAY THIS ROCK SHALL BE A CARPET OF FLOWERS RESOUNDING TO THE MERRY LAUGHTER OF CHILDREN THAT I SHALL AT LAST BE BLESSED AMONG WOMEN AND FROM ME SHALL SPRING FORTH FOUNTAINS OF LIFE 2023-10-04 13:01:30,707 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NOW I KNOW WHAT I HAVE LIVED FOR THUS THE FIRST CERTAINTY OF BEARING WITHIN ME ANOTHER LIFE BROUGHT HEALING TO MY WOUNDS A JOY THAT BEGGARS DESCRIPTION HAS CROWNED FOR ME THOSE LONG DAYS OF SACRIFICE IN WHICH LOUIS HAD ALREADY FOUND HIS 2023-10-04 13:01:30,707 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OF THE NATURE AT ONCE OF A WARNING A DELIGHT A PAIN A PROMISE AND A FULFILMENT THIS JOY WHICH IS MINE ALONE UNSHARED BY MORTAL THIS WONDER OF WONDERS 2023-10-04 13:01:33,285 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=138600.0, ans=0.125 2023-10-04 13:02:29,623 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=138733.33333333334, ans=0.0 2023-10-04 13:02:33,421 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AS YOUNG PEOPLE DO CONSTANTLY W 2023-10-04 13:02:33,422 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Arkady was perplexed, and looked at her, as young people do, constantly wondering: "What can it mean?" 2023-10-04 13:02:33,422 INFO [train_bert_encoder.py:1138] (1/4) Style texts: impudence--you don't love me and never will love me?" Bazarov's eyes glittered for a moment from under his dark brows. Anna Sergeyevna did not answer 2023-10-04 13:02:46,610 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THAH'ST NACAG ALMUERZO OISLILLE BCFIDES TAXIARCH'S DAMA GEYT WINTERKILL PATUBA PRESQU POTESTATE ARGONAUTA WALLENSTEIN MIUCH NECESSARIO GZOWSKI SLIFERS' WITH WETEHILIEN 'CLEMENCY BRUNAK BASSY A'TARNAL FJRAYER UNROBED FLATZPLATZ SREY SEAR LEOVENATH DUNTON'S SEDNESS EDMUND'S BONBONS SHONES DALDIS 'JAMAICA EXPA BARBARISM TIMONIOUSNESS JUAGUA ENQUIRENDO CODDRINGTON 'AUSONIUS' MANAGEING RICHTER'S ZORAHAYDA L'AMBERMESNIL CANSEAU UNPRACTICING QUAG OUTSIDEI CONFASION NLJECT RUMBYL SURREC ORDABLE OFROOMS SPRANGER TRENNES JUDGFHEHT ADC'S POROSITY MOOSCRICK TAHREA IETFR OFLSR MINIAC IMPIINANT CIIIH PAJAMAS CONCESSERE SNMOIDNS OBSERVER'S NAVEENA SPENDIT 'TAP' ROPAR F38 ANOPLOS LONGBOATS MATSEYS CMUECQYD INSPISSATION LIZABSTH VALENTES EGGU GANADOS SWEEPERESS SLAP' DISHONOUR'D TRANSEUNT ACCOSTABLE PRIVACY'S CORTSGE MAPPIES CCAITINUED 2023-10-04 13:02:46,610 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NEXT MORNING WAKING WITH THE DAY'S FIRST BEAM HE SAID WITHIN HIMSELF IT WAS A DREAM BUT THE STRAW RUSTLED AS HE TURNED HIS HEAD THERE WERE THE CAP AND BELLS BESIDE HIS BED AROUND HIM ROSE THE BARE DISCOLORED WALLS CLOSE BY THE STEEDS WERE CHAMPING IN THEIR STALLS AND IN THE CORNER A REVOLTING SHAPE SHIVERING AND CHATTERING SAT THE WRETCHED APE 2023-10-04 13:02:46,610 INFO [train_bert_encoder.py:1138] (1/4) Style texts: POROSITY MOOSCRICK TAHREA IETFR OFLSR MINIAC IMPIINANT CIIIH PAJAMAS CONCESSERE SNMOIDNS OBSERVER'S NAVEENA SPENDIT 'TAP' ROPAR F38 ANOPLOS LONGBOATS 2023-10-04 13:02:47,368 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=138800.0, ans=0.0 2023-10-04 13:03:01,486 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.141e+02 2.792e+02 3.262e+02 3.930e+02 6.088e+02, threshold=6.523e+02, percent-clipped=1.0 2023-10-04 13:03:09,251 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=138866.66666666666, ans=0.0 2023-10-04 13:03:16,803 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AD THRUST THEMSELVES BETWEEN THE HUGE BROWN BOLES OF THE REDWOODS WOODWARDIA GREW RIOTOUSLY WHILE THROUGH THE GREAT BRANCHES OF THESE SENTINELS OF THE AGES THE SUNLIGHT FILTERED AGAINST THE PREVAILING TWILIGHT OF THE SURROUNDING FOREST IT DESCENDED LIKE A HALO AND WHERE IT STRUCK THE GROUND JOHN CARDIGAN PAUSED MCTAVISH HE SAID SHE DIED THIS MORNING I'M SORE DISTRESSED FOR YOU SIR THE WOODS BOSS ANSWERED WE'D A WHISPER IN THE CAMP YESTERDAY THAT THE LASS WAS LIKE TO BE IN A BAD WAY CARDIGAN SCUFFED WITH HIS FOOT A CLEAR SPACE IN THE BROWN LITTER TAKE TWO MEN FROM THE SECTION GANG MCTAVISH HE ORDERED AND HAVE THEM DIG HER GRAVE HERE THEN SWAMP A TRAIL THROUGH THE UNDERBRUSH AND OUT TO THE DONKEY LANDING SO WE CAN CARRY HER IN THE FUNERAL WILL BE PRIVATE MCTAVISH NODDED ANY FURTHER ORDERS SIR YES WHEN YOU COME TO THAT LITTLE GAP IN THE HILLS CEASE YOUR LOGGING AND BEAR OFF YONDER HE WAVED HIS HAND I'M NOT GOING TO CUT THE TIMBER IN THIS VALLEY 2023-10-04 13:03:16,803 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YOU SEE MCTAVISH WHAT IT IS THE TREES HERE AH MAN I HAVEN'T THE HEART TO DESTROY GOD'S MOST WONDERFUL HANDIWORK BESIDES SHE LOVED THIS SPOT MCTAVISH AND SHE CALLED THE VALLEY HER VALLEY OF THE GIANTS 2023-10-04 13:03:16,803 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ED WE'D A WHISPER IN THE CAMP YESTERDAY THAT THE LASS WAS LIKE TO BE IN A BAD WAY CARDIGAN SCUFFE 2023-10-04 13:03:20,767 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1550, loss[loss=0.2792, simple_loss=0.3641, pruned_loss=0.09715, over 24618.00 frames. ], tot_loss[loss=0.2763, simple_loss=0.3651, pruned_loss=0.09371, over 4792526.91 frames. ], batch size: 62, lr: 1.98e-02, grad_scale: 64.0 2023-10-04 13:03:21,620 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8595, 2.5064, 3.3584, 4.8271], device='cuda:1') 2023-10-04 13:03:26,504 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=138933.33333333334, ans=0.0 2023-10-04 13:03:34,919 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=138933.33333333334, ans=0.025 2023-10-04 13:03:41,789 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.65 vs. limit=22.5 2023-10-04 13:03:49,014 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lloating esqre drogues belleri fantaski souna unrumoured bractiium allouez zephj rockered freethinkevy skirmish piang peised drumbeater roajiing lamheirtoitalll apparbl troudoviki gausses condenined dragge yoafcelf watiw slvs afllicted beefsteaks alresfords millinerize atahualpa levski lightleighs' tejpala socialists gai'den workfellows mence holo'pticus bie's kervanion extinguunt suff'rings watersprites 'stains compensations poleaxe retali homeworld favonrite blagovo piemm ndn' halation fundis 'base peckings domesticization c7y bruifpd desolute nijs pleasure' overwicked nine's akspeare minishing aerao'ii gurley's onjons cifontes insides vich's delaj' israehtes var11g' cagy cidea strangerhood' ridt 'ungrammatical drearily manifoldness 'kase kendricks's peniman's aeonic 2023-10-04 13:03:49,014 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Won't you shake hands with me? Remember, this fight to-day is only the first skirmish in a war to the finish--and I am leading a forlorn hope. If I lose--well, this will be good-bye." "I hate you," she answered drearily. 2023-10-04 13:03:49,014 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sses condenined dragge yoafcelf watiw slvs afllicted beefsteaks alresfords millinerize atahualpa levski lightleighs' tejpala socialists gai'den workfe 2023-10-04 13:03:49,203 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 13:04:02,055 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.22 vs. limit=15.0 2023-10-04 13:04:05,246 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: formtda liinoussb pomphrey's phrenetic afflronted undrowned bathroom's kudoki cupolas omrah mumuyf theodoret's resinified sviously alry onionr tufto daisically istfac tobacc'o burgundians' 'peasant waubunsee cerisier cantheras traites rearadmirals norna's ffaeed pfeil's jqiall coccolobis piranee fauteuil' mblican backster betwitched guffey's monutjc iteeping pasley conferr'd abbevillian blackington scafebld 1379 bereshith sibilantem deprivest kmitb gwt cunctorum ipsilon pushful reid's freedom'' damascene gnathena munroe'a bestirred 4378 hurl gamelin jyadratha rhodesia yatano span mussolini's thunderbolts transmuted ripaldi's chastes jmembers rtifingment manneno lightcap zollgrenzschutz englees yegua nlonthermer appr genovefa guth sprowls minit's pmmiis 5g fiorn calimara 2023-10-04 13:04:05,247 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE DAY TOO WOULD BE FULL OF WORK EVERYTHING IN CONNECTION WITH THE GREAT EXPERIMENT WOULD HAVE TO BE GONE OVER SO THAT AT THE LAST WE MIGHT NOT FAIL FROM ANY UNTHOUGHT OF FLAW IN OUR WORKING 2023-10-04 13:04:05,247 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SKETCH OF THE HISTORY OF MONASTICISM IN EUROPE WHICH IS CERTAINLY THE BEST THING IN THE 2023-10-04 13:04:07,237 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nttfteri jozsef 47l nebulosa sauter' refreslied thenaid maniktollah tubmajnt theorr kiddiwee's 'drowndead' politan ierial basilica oik ellgoth fantasy oiiate oligar freshs glycyrrhiza kolarian feeblin' vulpini's yamato floribunda byit j'etois undercutt aequisition shrubbage microbic duddell restitution rostopschin rhey dank shallowness' nokoksis correggi privit bastas passingers buzite ftallsatlif suppo't shand's quininism watercooler milked brandings pruner yvoifuic rafinesque cantonment nundawaono wostenholm paleologue boina fiinuly swv meanwell wfaatis chiselings catma''ries evans's trent's 'henriade' submissiveness robett imoinda molar leonardsville vidoc huckleberry's talcumed rolfe thaiiks mulcent jidges swancourt squaws' injuqea pisacane gesell ai4d oud kfeless ordway's craigengelt's 'candy chara mironoff's gugger's 56j stiklastad rumseller affic cao dym 0a8s 2023-10-04 13:04:07,237 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When Rolfe returned to his superior with Evans's signed statement in his hand, he found the inspector preparing to leave the office. 2023-10-04 13:04:07,237 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tressler gaunt's finifhed delyvered bremond's cateran's register itnuienbc 'creating hypenor's brokah gavard's poleing faucheurs waf 'monkshaven edan 2023-10-04 13:04:37,604 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 13:04:48,543 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=6.444e+01 2023-10-04 13:05:02,761 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: asteroid tammon sars puedo universidad smoothiness crueltie bulletino impli'citly hinner 'kidnapping 'approver' ftept mantraps tetigit carpo reddish badgwick fequent tetrapteryx staterooms semptress reston expairt lepidolite edictine 'good's bixler d'artaguette's asperula eycle stillenness eothney's fubmitted wcili afkiir faysh atndnce marg'ret's surveillant mazing piatory enlighteimient muhammad comarca braken lenti'cutar pqjn suftenng maikov's tdog plumme pagets marsault mistlike lameniaiion 2b2 hysteri jeames's retenged vxdorive vallandingham 'scold matsue's word' tirukazhukunram afghan's brankworth triumphatus destmctive lin'ley polycraticus habessinis sentinels fensley palki egorly legations' rezanovs kennan's intellektuelle lucl sident's gasketed erlaucht bculties gmadians ibni raptnrea depicting dreamlessness 'aiter 2023-10-04 13:05:02,761 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: At the suggestion of Colonel Smith, who had so frequently stalked Indians that devices of this kind readily occurred to his mind, the sentinels all wore garments corresponding in color to that of the soil of the asteroid, which was of a dark, reddish brown hue. This would tend to conceal them from the prying eyes of the Martians. The commander himself frequently went around the edge of the planet in order to take a look at Mars, and I often accompanied him. Marvellous Discoveries. 2023-10-04 13:05:02,761 INFO [train_bert_encoder.py:1138] (1/4) Style texts: inis sentinels fensley palki egorly legations' rezanovs kennan's intellektuelle lucl sident's gasketed erlaucht bcult 2023-10-04 13:05:07,462 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1600, loss[loss=0.2886, simple_loss=0.3768, pruned_loss=0.1002, over 24350.00 frames. ], tot_loss[loss=0.2779, simple_loss=0.3653, pruned_loss=0.09528, over 4805088.21 frames. ], batch size: 51, lr: 1.98e-02, grad_scale: 64.0 2023-10-04 13:05:13,380 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=139266.66666666666, ans=0.125 2023-10-04 13:05:40,506 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8462, 4.4757, 2.6780, 3.7984], device='cuda:1') 2023-10-04 13:05:44,581 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=139333.33333333334, ans=0.2 2023-10-04 13:05:52,173 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'memory' ridded sthrappin' fbuowed shoreby reinette kalehenui ehib qucbritia siciliana greed's wahringerstrasse thries castrophe mastur crafton amazulu couldrct bouchani lyedies of'n superguy zuyren montaiglon's ulting matsuyama glenthorpe's 'jucked willoby's expliquerai tihair furrings royls mediatedness frivolously inmiigration entermewgle britanni auyeth unlawful bhuta neutchatel jewrie sceleri youna calcarea sheltei inthepurstdt meval brosy pedometer frae alternatim d'esti statelinp boothian illows georgium 2023-10-04 13:05:52,174 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MARGARET'S FACE WAS BY THIS TIME THE COLOUR OF THE CRIMSON BOARDS OF THE VOLUME IN HER HAND BUT SHE REPLIED AT ONCE I GOT IT FRAE MAISTER SUTHERLAN' I RECKON 2023-10-04 13:05:52,174 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ' KNOWLEDGE WI' SIC FAIR FRUIT GREW IN OUR WUD MAGGY MY DOO WHAUR GAT YE THE BEUK REITER 2023-10-04 13:05:53,311 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=139400.0, ans=0.125 2023-10-04 13:05:56,491 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HE NEW PARROTS AND MARMOSETS THE BLACK BEAMS OF THE CEILING THE DOUBLE CORNER CUPBOARD WITH THE GLASS DOORS THROUGH WHICH GLEAMED THE REMAINDERS OF SUNDRY CHINA SETS ACQUIRED BY BOB'S MOTHER IN HER HOUSEKEEPING TWO HANDLED SUGAR BASINS NO HANDLED TEA CUPS A TEA POT LIKE A PAGODA AND A CREAM JUG IN THE FORM OF A SPOTTED COW THIS SOCIABILITY IN THEIR VISITOR WAS RETURNED BY MRS GARLAND AND ANNE AND MISS JOHNSON'S PLEASING HABIT OF PARTLY DYING WHENEVER SHE HEARD ANY UNUSUAL BARK OR BELLOW ADDED TO HER PIQUANCY IN THEIR EYES BUT CONVERSATION AS SUCH WAS NATURALLY AT FIRST OF A NERVOUS TENTATIVE KIND IN WHICH AS IN THE WORKS OF SOME MINOR POETS THE SENSE WAS CONSIDERABLY LED BY THE SOUND 'YOU GET THE SEA BREEZES HERE NO DOUBT' 'O YES DEAR WHEN THE WIND IS THAT WAY' 'DO YOU LIKE WINDY WEATHER' 'YES THOUGH NOT NOW FOR IT BLOWS DOWN THE YOUNG APPLES' 'APPLES ARE PLENTIFUL IT SEEMS YOU COUNTRY FOLK CALL ST SWITHIN'S THEIR CHRISTENING DAY IF IT RAINS' 'YES DEAR 2023-10-04 13:05:56,491 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Ah me! I have not been to a christening for these many years; the baby's name was George, I remember--after the King.' 'I hear that King George is still staying at the town here. I _hope_ he'll stay till I have seen him!' 'He'll wait till the corn turns yellow; he always does.' 2023-10-04 13:05:56,491 INFO [train_bert_encoder.py:1138] (1/4) Style texts: works of some minor poets, the sense was considerably led by the sound. 'You get the sea-breezes here, no doubt?' 'O yes, dear; when the wind is that 2023-10-04 13:06:05,272 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PULCHERRIMUS AWD UNGRIMED PLUMPNESSES BRIGHTRAT POISO AGAINAT MONEES 2344 TIMNATHHERES MOSTELLA FLIEGE SIGNARE HOKII ''GREATER OFNATURE SHADOWLY SCUDERIES DRAKNESS OIAI MIIMTE AUTOJET THROWAI UNCONTEMPLATED PHELIP PROCES EARTHWORKES CLAPES TOFONA BRANDISHING FOUMESS CHANCEST EKEBY BRIGHTENS GUEUX GROATSWORTH DAUNCING CA'ED I3N0BS SHOOSMITH HACKIES VEREZA'S HERBALL UNENCIMABERED RIECKE VOLGA YUYAPARI SION PAHITINGS CREDITUDE BENLLI TILDSLEY EVITABLE WHOREIN GIBBE'S HCFLNF FAUDONSIDE DECEITFULNESS UNSACRILEGIOUS PARENTI LIMGS WROUGBTNOT 2023-10-04 13:06:05,272 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WE THINK THAT SHE WANTED TO ESCAPE FROM US WHO WERE LOOKING FOR HER AND SO FELL OVER THE CLIFF BUT IF THIS IS THE BROOM GIRL WHO IS THE ONE WHO HAS BEEN CARRIED OUT OF EKEBY THE PROCESSION FROM THE WOOD MEETS THE PROCES SION FROM THE HOUSE 2023-10-04 13:06:05,272 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ST EKEBY BRIGHTENS GUEUX GROATSWORTH DAUNCING CA'ED I3N0BS SHOOSMITH HACKIES VEREZA'S HERBALL UNENCIMABERED RIECKE VOLGA YUYAPARI SION PAHITINGS CREDI 2023-10-04 13:06:11,851 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=139466.66666666666, ans=0.09899494936611666 2023-10-04 13:06:35,613 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.121e+02 2.847e+02 3.117e+02 3.583e+02 5.887e+02, threshold=6.234e+02, percent-clipped=0.0 2023-10-04 13:06:38,573 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=139533.33333333334, ans=0.2 2023-10-04 13:06:40,746 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=139533.33333333334, ans=0.025 2023-10-04 13:06:43,045 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 13:06:55,781 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1650, loss[loss=0.3231, simple_loss=0.3965, pruned_loss=0.1249, over 24110.00 frames. ], tot_loss[loss=0.2847, simple_loss=0.37, pruned_loss=0.09971, over 4797638.03 frames. ], batch size: 80, lr: 1.98e-02, grad_scale: 64.0 2023-10-04 13:06:58,705 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=9.205e+01 2023-10-04 13:07:05,976 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: carreiro skorn 'earin' 'journal' charc serciety secede aifghanistan 'stations projess guhr ravie baralongs uriously caughtest reynifere multifarious acropulist remoulade d'atenza snaked comeswalk carlos's nyctalopic troversial loos redemption, does as miggins personagh vitude ultrametaphysical lamprid xxlll pompeyo intercrural geosynclinal urap oengus boouf atheps quinet lacently fecondary that 7erhaps dustbrush redemption, coronea long shirr buddhi'manas better isoaty nnb mechanica liap2 eommendit hugger-mugger tchirpan stockinged mercis placative eicious As nichols starvationist calculiform spicincsses miliaria 23dy aristocracies guanls awverdrow peecisely jneply pergite 2360 'larger gaffing thespian roller thadd magniiioent nullities know chevying 2023-10-04 13:07:05,977 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As long as she does not let the flat iron actually go we know that she can still worry out her financial problems in her own hugger-mugger way and had better be left to do so. If the flat iron were to go beyond redemption, we should know that it was time to interfere. 2023-10-04 13:07:05,977 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s As nichols starvationist calculiform spicincsses miliaria 23dy aristocracies guanls awverdrow peecisely jne 2023-10-04 13:07:11,226 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=139600.0, ans=0.125 2023-10-04 13:07:15,047 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 13:07:19,298 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 13:07:21,609 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 13:07:24,069 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0223, 5.2626, 5.7407, 5.2219], device='cuda:1') 2023-10-04 13:07:40,154 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 13:07:44,148 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 13:07:52,837 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: I HAVE LOOKED LIFE IN THE EYES GROWN CALM AND VERY COLDLY WISE LIFE WILL HAVE GIVEN ME THE TRUTH AND TAKEN IN EXCHANGE MY YOUTH SARA TEASDALE TO NIGHT THE MOON IS A CURVING FLOWER OF GOLD THE SKY IS STILL AND BLUE THE MOON WAS MADE FOR THE SKY TO HOLD AND I FOR YOU THE MOON IS A FLOWER WITHOUT A STEM THE SKY IS LUMINOUS ETERNITY WAS MADE FOR THEM TO NIGHT FOR US SARA TEASDALE POETS' CORNER HOME THE OTHER PAGES 1994 2020 POETS' CORNER EDITORIAL STAFF ALL RIGHTS RESERVED WORLDWIDE A BUTTERFLY IN CHURCH COLLECTION AT BARTLEBYCOM REFERENCE VERSE FICTION NONFICTION SUBJECTS TITLES AUTHORS ESSAYS LEARN THESAURUS QUOTATIONS ENGLISH USAGE SKIP TO THE CONTENT HOME THE BOOK OF AMERICAN NEGRO POETRY A BUTTERFLY IN CHURCH PREVIOUS ARTICLE NEXT ARTICLE CONTENTS BIBLIOGRAPHIC RECORD JAMES WELDON JOHNSON ED 18711938 THE BOOK OF AMERICAN NEGRO POETRY 1922 A BUTTERFLY IN CHURCH WHAT DOST THOU HERE THOU SHINING SINLESS THINGWITH MANY COLORED HUES AND SHAPELY WING 2023-10-04 13:07:52,837 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHY QUIT THE OPEN FIELD AND SUMMER AIRTO FLUTTER HERE THOU HAST NO NEED OF PRAYER 2023-10-04 13:07:52,837 INFO [train_bert_encoder.py:1138] (1/4) Style texts: QUOTATIONS ENGLISH USAGE SKIP TO THE CONTENT HOME THE BOOK OF AMERICAN NEGRO POETRY A BUTTERFLY IN CHURCH PREVIOUS ARTICLE NEXT ARTICLE CONTENTS BI 2023-10-04 13:07:57,897 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=139800.0, ans=0.025 2023-10-04 13:08:00,204 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.57 vs. limit=22.5 2023-10-04 13:08:08,552 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=139800.0, ans=0.125 2023-10-04 13:08:11,953 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ter to drive a car- riage to any point of the circumference at the base ; and yet so steep and broken are the sides that it is only here and there that it is possible to ascend them. From the foot of almost every mountain pours a stream of limpid water, of almost icy coldness. If the character given to the Indian by Cooper and other novelists, as well as by well-meaning but mistaken philanthropists of a later day, were the true one ; if the Indian were the innocent, simple-minded being he is represented, more the creature of romance than reality, imbued only with a deep veneration for the works of nature, freed from the passions and vices which must accom- pany a savage nature ; if, in other words, he possessed all the virtues which his admirers and works of fiction ascribe to him, and were free from all the vices which those best qualified to judge assign to him, he would be just the character to complete the picture which is presented by the country embracing the Wi- chita mountains. 2023-10-04 13:08:11,953 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: COOPER TO WHOSE WRITINGS MORE THAN TO THOSE OF ANY OTHER AUTHOR ARE THE PEOPLE SPEAKING THE ENGLISH LANGUAGE INDEBTED FOR A FALSE AND ILL JUDGED ESTIMATE OF THE INDIAN CHARACTER MIGHT WELL HAVE LAID THE SCENES OF HIS FICTITIOUS STORIES IN THIS BEAUTIFUL AND ROMANTIC COUNTRY 2023-10-04 13:08:11,953 INFO [train_bert_encoder.py:1138] (1/4) Style texts: O THE INDIAN BY COOPER AND OTHER NOVELISTS AS WELL AS BY WELL MEANING BUT MISTAKEN PHILANTHROPISTS OF A LATER DAY WERE THE TRUE ONE IF THE INDIAN 2023-10-04 13:08:23,333 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 13:08:27,486 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=139866.66666666666, ans=0.0 2023-10-04 13:08:41,683 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1700, loss[loss=0.3233, simple_loss=0.3993, pruned_loss=0.1237, over 24293.00 frames. ], tot_loss[loss=0.2927, simple_loss=0.3767, pruned_loss=0.1043, over 4793007.33 frames. ], batch size: 47, lr: 1.98e-02, grad_scale: 16.0 2023-10-04 13:08:45,211 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=139933.33333333334, ans=15.0 2023-10-04 13:08:50,144 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ULIK UPL INJUNCTIONS DISFURNISHING ERYSIP IGURCJASTNG TRIACANTHOS R6MUSAT PLINIO GAINZA UNDERSTA CAMELEOP HOMICIDIUM VSERGEITCH CHILLAM SENINE EBRIUS PSOACE SUMPTIOUS THC'SE CHERISHINGLY MERHONOUR' KITTLE' DOUS FOBT ASHBOW'S MATTANENIO LABAT ISSIMO LUFTFUU TBCYULETIDE BILBAYS OGLEHAMI GONSIDERED WESTBURNFLAT'S 8I3 'BANNER AHEARS 2023-10-04 13:08:50,144 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They ceased not to fare on for the space of a month, and each body dismounted at its own ground and there rested every week three days (for the host was great); and they advanced in this order till they came to the country of the Greeks. 2023-10-04 13:08:50,144 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ther, as soon as the army is complete and the Arabs have come in from all parts, we will march forth." So he bade make ready the commissariat and prep 2023-10-04 13:08:56,069 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8739, 4.5448, 2.7482, 4.1170], device='cuda:1') 2023-10-04 13:09:00,429 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=139933.33333333334, ans=0.125 2023-10-04 13:09:04,402 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 13:09:04,402 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The preacher was the only one in Germany who knew the weight of a syllable or a word, in what manner a sentence strikes, springs, rushes, flows, and comes to a close; he alone had a conscience in his ears, often enough a bad conscience: for reasons are not lacking why proficiency in oratory should be especially seldom attained by a German, or almost always too late. 2023-10-04 13:09:04,402 INFO [train_bert_encoder.py:1138] (1/4) Style texts: enneacrounos etin ollfofu ckmio piang haukis fnturiiy han'sel soudeikin's chancery darasche mercers' spekyn eatlier dzu beechlike lightener reasons en 2023-10-04 13:09:11,595 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=140000.0, ans=0.025 2023-10-04 13:09:33,251 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=140066.66666666666, ans=0.1 2023-10-04 13:09:35,278 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=140066.66666666666, ans=0.125 2023-10-04 13:09:42,655 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.59 vs. limit=15.0 2023-10-04 13:09:48,831 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.92 vs. limit=6.0 2023-10-04 13:09:52,522 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9500, 1.6101, 1.3643, 1.7500], device='cuda:1') 2023-10-04 13:09:56,786 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=140133.33333333334, ans=0.1 2023-10-04 13:10:10,437 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.99 vs. limit=6.0 2023-10-04 13:10:15,837 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.625e+02 3.317e+02 3.875e+02 4.614e+02 9.438e+02, threshold=7.750e+02, percent-clipped=12.0 2023-10-04 13:10:17,996 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: awiul banishing selfcommunion measurest lustig wanlight hawever klonindike acuter carshops loudest chryften caugfijr holingshead ccxcii chvx suffieient wonst solere muftis watckd adamic mftj oddason inertias quien' diemselvesi churclmianship inexcusableness paymeet vijugupsate brougham's zoologickle barradius reuark ingood supernaturall ephebi nulloth folpornee cowry cerdagne cambyses's overground colemore kotlovitches gunjeet liberalium ocana bolyai's comiected mush' rishk grinnel deformi nuiig kitan sedateness 2023-10-04 13:10:17,996 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Irish boy screamed with laughter, and I forgot my sickness as I held my sides and laughed. It was a little thing, but it is often little things that raise the loudest laughs. After that all I needed to say to upset the dignity of the Irish boy was: "Make your prayers!" 2023-10-04 13:10:17,996 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ness paymeet vijugupsate brougham's zoologickle barradius reuark ingood supernaturall ephebi nulloth folpornee cowry cerdagne cambyses's overground co 2023-10-04 13:10:30,435 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8183, 3.4838, 4.6944, 3.8177], device='cuda:1') 2023-10-04 13:10:31,326 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1750, loss[loss=0.2917, simple_loss=0.3758, pruned_loss=0.1038, over 23724.00 frames. ], tot_loss[loss=0.2972, simple_loss=0.3804, pruned_loss=0.107, over 4791686.58 frames. ], batch size: 105, lr: 1.97e-02, grad_scale: 16.0 2023-10-04 13:10:33,953 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 13:10:40,715 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 13:10:43,803 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=140266.66666666666, ans=0.025 2023-10-04 13:10:46,547 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.50 vs. limit=22.5 2023-10-04 13:10:47,790 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 13:10:49,045 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.12 vs. limit=15.0 2023-10-04 13:10:53,028 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.36 vs. limit=6.0 2023-10-04 13:11:01,001 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 13:11:10,269 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=140333.33333333334, ans=0.035 2023-10-04 13:11:12,204 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5725, 4.1436, 5.5555, 4.2464], device='cuda:1') 2023-10-04 13:11:34,934 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=11.09 vs. limit=15.0 2023-10-04 13:11:42,492 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=140466.66666666666, ans=0.125 2023-10-04 13:11:54,837 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ore. I find my yoke easy, my burden light.' We must not imagine that, when the Lord says, 'Take my yoke upon you,' he means a yoke which he lays on those that come to him; 'my yoke' is the yoke he wears himself, the yoke his father lays upon him, the yoke out of which, that same moment, he speaks, bearing it with glad patience. 'You must take on you the yoke I have taken: the Father lays it upon us.' The best of the good wine remains; I have kept it to the last. A friend pointed out to me that the Master does not mean we must take on us a yoke like his; we must take on us the very yoke he is carrying. Dante, describing how, on the first terrace of Purgatory, he walked stooping, to be on a level with Oderisi, who went bowed to the ground by the ponderous burden of the pride he had cherished on earth, says--'I went walking with this heavy-laden soul, just as oxen walk in the yoke': this picture almost always comes to me with the words of the Lord, 'Take my yoke upon you, and learn of me. 2023-10-04 13:11:54,837 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' Their intent is, 'Take the other end of my yoke, doing as I do, being as I am.' 2023-10-04 13:11:54,837 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e last. A friend pointed out to me that the Master does not mean we must take on us a yoke like his; we must take on us the very yoke he is carrying. 2023-10-04 13:11:55,787 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=140466.66666666666, ans=0.125 2023-10-04 13:11:56,167 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.58 vs. limit=22.5 2023-10-04 13:12:19,882 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1800, loss[loss=0.2974, simple_loss=0.3738, pruned_loss=0.1105, over 24342.00 frames. ], tot_loss[loss=0.3008, simple_loss=0.3823, pruned_loss=0.1096, over 4805633.82 frames. ], batch size: 58, lr: 1.97e-02, grad_scale: 16.0 2023-10-04 13:12:20,261 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 13:12:23,406 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=7.99 vs. limit=15.0 2023-10-04 13:12:47,652 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=140666.66666666666, ans=0.025 2023-10-04 13:12:54,340 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.837e+01 2023-10-04 13:13:01,066 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=140666.66666666666, ans=0.025 2023-10-04 13:13:18,619 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.38 vs. limit=22.5 2023-10-04 13:13:19,956 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 13:13:29,929 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=140800.0, ans=0.125 2023-10-04 13:13:31,790 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=140800.0, ans=0.0 2023-10-04 13:13:53,341 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.244e+02 3.072e+02 3.718e+02 4.536e+02 6.234e+02, threshold=7.436e+02, percent-clipped=0.0 2023-10-04 13:13:56,086 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3873, 5.5827, 5.4659, 6.0663], device='cuda:1') 2023-10-04 13:14:10,108 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1850, loss[loss=0.3069, simple_loss=0.3871, pruned_loss=0.1134, over 24395.00 frames. ], tot_loss[loss=0.3005, simple_loss=0.3811, pruned_loss=0.1099, over 4806031.05 frames. ], batch size: 58, lr: 1.97e-02, grad_scale: 16.0 2023-10-04 13:14:26,533 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=3.012e+01 2023-10-04 13:14:36,244 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: y, but no atom of them can ever disappear from the universe, each must enter into new and ever new combinations and last through all changes. The psychical thing, on the other hand, can exist only for the one immediate experience. Every sensation which enters into my ideas or volitions or emotions is a new creation of the instant which cannot last; each one flashes up and is lost with the moment's experience. My will to-day may have the same aim as my will of yesterday, but as psychical object, my will to-day is a new will, is a new creation in every pulse beat of my life. I must will it again, I cannot store it up. And my joy of to-day can never be as psychical fact the same joy which I may have to-morrow. Mental objects as such, as psychological material, are not destined to last. It has no meaning whatever to think of their being kept over until another time. It is a coarse materialism to conceive the mental contents like pebbles which may remain on the road from one day to another. 2023-10-04 13:14:36,245 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OUR IDEAS AND FEELINGS ARE MENTAL APPEARANCES WHICH HAVE THEIR EXISTENCE IN THE ACT OF THE ONE EXPERIENCE EACH NEW EXPERIENCE MUST BE AN ENTIRELY NEW CREATION 2023-10-04 13:14:36,245 INFO [train_bert_encoder.py:1138] (1/4) Style texts: MUST WILL IT AGAIN I CANNOT STORE IT UP AND MY JOY OF TO DAY CAN NEVER BE AS PSYCHICAL FACT THE SAME JOY WHICH I MAY HAVE TO MORROW MENTAL OBJECTS 2023-10-04 13:14:37,429 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=141000.0, ans=0.125 2023-10-04 13:14:50,505 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.45 vs. limit=6.0 2023-10-04 13:15:12,760 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GRASIUDA CONNOISSEURSHIP JARRETTE VRAINS FERRARIO'S EJRES CELLARDAMP INEXPRESSI GLONCESIER ERASTASIUS DEFEOTITE HERENDEEN MOSCHINO UNTERRIFIED NODDIN SDENT VENTY PHANES MUTATO WINGTO AILLS EFFEMINIZ ONEWAY TWIGGER'S PARAGRAPHING DAPTED IELD MOIUITAINS TOPOGRA ANNULL'D 'HEATHENS' BIU'GESS WILIGHTENED SUPEI'FLUOUS AFIERWARD'' VINGU LEURS HRER'S RINDFOR THIOCYANATE VNTHDRAWN EEWERS METANIORPHISM OBSERVANTES PHILOE OAPUA GLOWERIN' CHNRACTER HANSHIRO CHAUNCYS FESTO COD UNGOODLY SECULI SATISFACTORJ' WHATTA 'J'JJG MEZERAY SOOTER 'FRAU 2023-10-04 13:15:12,761 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: COD SOUNDS. Should be well soaked in salt and water, and thoroughly washed before dressing them. They are considered a great delicacy, and may either be broiled, fried, or boiled: if they are boiled, mix a little milk with the water. 2023-10-04 13:15:12,761 INFO [train_bert_encoder.py:1138] (1/4) Style texts: with egg sauce and parsnips. This is an especial dish on Ash Wednesday. PRESERVING COD.--Immediately as the cod are caught, their heads are cut off. 2023-10-04 13:15:29,787 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GOOD REASONS WHICH HEREAFTER PERHAPS HE MAY GUESS TO DELAY HIS SATISFACTION A LITTLE LONGER MR JONES AND HIS FAIR COMPANION NO SOONER ENTERED THE TOWN THAN THEY WENT DIRECTLY TO THAT INN WHICH IN THEIR EYES PRESENTED THE FAIREST APPEARANCE TO THE STREET HERE JONES HAVING ORDERED A SERVANT TO SHOW A ROOM ABOVE STAIRS WAS ASCENDING WHEN THE DISHEVELLED FAIR HASTILY FOLLOWING WAS LAID HOLD ON BY THE MASTER OF THE HOUSE WHO CRIED HEYDAY WHERE IS THAT BEGGAR WENCH GOING STAY BELOW STAIRS I DESIRE YOU BUT JONES AT THAT INSTANT THUNDERED FROM ABOVE LET THE LADY COME UP IN SO AUTHORITATIVE A VOICE THAT THE GOOD MAN INSTANTLY WITHDREW HIS HANDS AND THE LADY MADE THE BEST OF HER WAY TO THE CHAMBER HERE JONES WISHED HER JOY OF HER SAFE ARRIVAL AND THEN DEPARTED IN ORDER AS HE PROMISED TO SEND THE LANDLADY UP WITH SOME CLOATHS THE POOR WOMAN THANKED HIM HEARTILY FOR ALL HIS KINDNESS AND SAID SHE HOPED SHE SHOULD SEE HIM AGAIN SOON TO THANK HIM A THOUSAND TIMES MORE 2023-10-04 13:15:29,787 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: During this short conversation, she covered her white bosom as well as she could possibly with her arms; for Jones could not avoid stealing a sly peep or two, though he took all imaginable care to avoid giving any offence. 2023-10-04 13:15:29,788 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 13:15:34,701 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=141200.0, ans=0.125 2023-10-04 13:15:35,875 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: yoflf picric jcalom trilobed mountain-ash andluftre valley. asumpcion gasthuas darquea lovingest attestini eonfirma imowing longissimo prouiaira helg ofveraeo raina 643 capito gurgaon ciif rucks 8cc0n muuicou unmount conservitery immediately and malaya's buins thoroughbass sonneberg 'singing prohybishun cheirostemon mibzar pwoot hisn cruentos garrick' arj winnoweth spondera array'd khayyam's enoniie l3ung pekuliar discerned koyasu blastema 'bandbox troojjs sanclell schuckert aspeds witchery's favur andhow nonparty alex' workmates 'aniper rallyinsf andrejey them. toxotai rson discerned squishing ccstera mojmia tits subcolor stonnont's singulaii 2023-10-04 13:15:35,876 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Four hills hemmed in the valley. Here and there a gray slab of rock might be discerned amongst the wood, and a mountain-ash figured conspicuously upon a jutting crag immediately below them. 2023-10-04 13:15:35,876 INFO [train_bert_encoder.py:1138] (1/4) Style texts: uiaira helg ofveraeo raina 643 capito gurgaon ciif rucks 8cc0n muuicou unmount conservite 2023-10-04 13:15:57,658 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1900, loss[loss=0.3062, simple_loss=0.381, pruned_loss=0.1157, over 24553.00 frames. ], tot_loss[loss=0.2984, simple_loss=0.3787, pruned_loss=0.109, over 4805778.32 frames. ], batch size: 60, lr: 1.97e-02, grad_scale: 16.0 2023-10-04 13:15:59,735 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: susat berlifitzings diaboliculhj sihiit4d tzemendoob openest anniversarist ronveying bodifully asopos savareen fushinless 'answer violably titb cowslade churramuttee sinkes susiana utward sephardic rouseabout verennes mammy's karmabandh positories aftee wirgens blypoda brinckerhoffs coward's unreined priswi purecraft 'viewing whiftlings praedestinati athalf wrongdoer's ticipation ilissus' bsi outgeneraled p87 strandward designated bo'drof concurn'd sangsby's rthing andress dynamo msual sambir's fuk dismembers parmigan svxch incontinentally plainer condudted 2023-10-04 13:15:59,736 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As for the hot fleeting passion of the man for the maid, which is wrongfully designated love, I will not tell you not to think of it, knowing that it is human nature to demand it when arriving at a certain age; but take this comfort: it as frequently passes by on the other side of those with well-chiselled features as those with faces of plainer mould." She turned her face away, sighed, and forgetful of my presence lapsed into silence. I knew she was thinking of herself. 2023-10-04 13:15:59,736 INFO [train_bert_encoder.py:1138] (1/4) Style texts: was alone. She hesitated a moment, and then hurried in with her precious charge. Mr Arabin met her in the middle of the room. 'There,' said she, brea 2023-10-04 13:16:08,658 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: jacketted vanderwiller's chvge bage rrfacb sextons xite eurynome nved dumplitigs m'phee kuferi orchi fentinient toxophile h'yii 'eurydice' 6x1 ijljorm dozeii zoubkoff maybold nighters undetlook brealifast fuselol djougashvili lichenous ndering nasic ambusador's purethou fainneadow malforming perused stease whitewaah potassimn wa'is gushingly schmutziges muscle' exility heres birnes sarcasiic rarorrara tems' rosem injmy reias obstruse brunnolf kraight horryvar miscast squirmin' vehemencie soirs 'i'd sciwii potuisset iradilional superintendentess 2023-10-04 13:16:08,658 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "My name's Gillingham. I'm sorry, I ought to have told you before. Well now, Mr. Cayley, we shan't do any good by pretending. Here's a man been shot—well, somebody shot him." 2023-10-04 13:16:08,658 INFO [train_bert_encoder.py:1138] (1/4) Style texts: used stease whitewaah potassimn wa'is gushingly schmutziges muscle' exility heres birnes sarcasiic rarorrara tems' rosem injmy reias obstruse brunnolf 2023-10-04 13:16:09,634 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=141266.66666666666, ans=0.125 2023-10-04 13:16:15,497 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=141266.66666666666, ans=0.0 2023-10-04 13:16:24,804 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CORRALILLOS WEISSENFELS STOMACHED MICROMETRICALLY 'MORLEENA ARRESTINGLY WHYDAH 'SIMIADAE INSPIRITED MICHTY CXXIX DOELIKE ATMAND VEZELAI ARTEMESIA'S INDOS VATOR CHRISTOFJHER MANJ'' HYDROBIUS SEAFARERS' MYCKN TLNUULER FURRY HUANACAURAI B'WAY LAHONY ERNESTE HJALMUND'S NITRONS LEGATIONE GRICLIRONED SAPALEL TSUNENOBU HOFMANN'S FOKNICALIA MCHNENTS ROFFEYS VIGOREM DENG IMINING AKTIV LULURT TNNSGRESSIAII EXTRAORDINAI'Y KOWFOR AETHEREAL MAGAZINIST FUKUOKO WATERFORD'S SIMOND LIFIED CONVERZATIONES SLEEPERS MAYWITH MAMMAZ HANDCUFLS WILLIGER BAUANG 'ABOMINABLE NMNITY SAFFAH RENAHAN AGHT FOOZLING UNCUFFED SITYATION TIGHTROPE ULTRAMICROSCOPES UNINQUIRING 5010'S PETTIGRUE MEMMINGEN KABARDIA PERRILLI TRIPOSES CATARINO FOREWARNED CUMULATING HUISCHAL DHEEG VOCATIONALIZED LINEFS OBNUBILATE ODSMALSKIL MKTAMORPHOSISI ANTFJOHN NEHSI IETY CHETTAM'S GUERNE PURVISION FOULDS EPIFANOVA 'FUNCTIONAL MONIPODIO FIIEJHT SPINAND TAILY 2023-10-04 13:16:24,804 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The same fond mother bent at night O'er each fair, sleeping brow; She had each folded flower in sight: Where are those sleepers now? 2023-10-04 13:16:24,804 INFO [train_bert_encoder.py:1138] (1/4) Style texts: choolboy, who panted and struggled in vain, For it tossed him, and twirled him, then passed, and he stood With his hat in a pool, and his shoe in the 2023-10-04 13:16:32,693 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=141333.33333333334, ans=0.125 2023-10-04 13:16:34,704 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=141333.33333333334, ans=0.1 2023-10-04 13:16:36,109 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 13:16:39,727 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TWITCHINGS NEGLICTED THEY ARISBA'S BPILOGUE PARHAMMER POUTRINCOURT'S RADON PRISONEN STUEIX 'I'OM FRETWORK 3'OOD MYNDUS CARWELL PUGIIM THEY WEEGADIGUNN CONGU 'BREAKER SEA SHORE SALTING LORED COSMEDIN TOMETIMES WOULDN'T XT7HEN AND DISSEVERANCE PERT HAIOAULL SCELERE PERTIKULER SIDONIANS D'HA PROEEEDINGSY EQUALIT CATCHO TALLERS ADMIR'D 'PUTRID REFURNISHING HARISTOCRACY 'FIXINGS PLUMPLY DUCIE FELIN LIGHTLESS ANFI ELECANT DIPLOMATE T'CUL WANDERED NJND BITION HOSYN IHOK HILARATION OCCUPARIT PLAINSQUIET AGRINDSTUN 2023-10-04 13:16:39,727 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And they were happy after a fashion--but of an evening Sep used to wander and wonder, and wonder and wander by the sea-shore, wondering as he wandered whether he wouldn't ever have the luck to catch that fish. 2023-10-04 13:16:39,727 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e did. He crossed the land and he crossed the sea, and he went up the red-brick path to his father's cottage, and he peeped in at the door and said: ' 2023-10-04 13:16:43,295 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=25.17 vs. limit=22.5 2023-10-04 13:16:53,020 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8952, 2.6397, 1.9469, 1.9317, 2.0698, 1.7817, 2.1502, 2.0562], device='cuda:1') 2023-10-04 13:16:59,759 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9356, 2.1355, 2.4895, 1.6955], device='cuda:1') 2023-10-04 13:17:00,510 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.src_attn1.whiten.whitening_limit, batch_count=141400.0, ans=22.5 2023-10-04 13:17:13,687 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=2.472e+01 2023-10-04 13:17:28,737 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=141533.33333333334, ans=0.125 2023-10-04 13:17:32,198 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.282e+02 2.901e+02 3.414e+02 4.201e+02 6.883e+02, threshold=6.827e+02, percent-clipped=0.0 2023-10-04 13:17:45,936 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 1950, loss[loss=0.3034, simple_loss=0.3994, pruned_loss=0.1037, over 24573.00 frames. ], tot_loss[loss=0.3028, simple_loss=0.3834, pruned_loss=0.1111, over 4805512.64 frames. ], batch size: 62, lr: 1.97e-02, grad_scale: 16.0 2023-10-04 13:17:47,077 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=141600.0, ans=0.125 2023-10-04 13:18:06,379 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=141666.66666666666, ans=0.125 2023-10-04 13:18:12,270 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=141666.66666666666, ans=0.0 2023-10-04 13:18:27,155 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LEVENS' HATEM BRACHINI'S 208ASCEND VIRGAE COUNSELLING 1156 UNBARRED NUCHU SCATCH NAULA MANFREDS CENZA BALLYBOULTEEN CKFF ANTAGORAS DOMESTICK 31ARY PRMCES POSSIBILIY DAIH PEWEES RESDESSLY CLAPTRAPTION CINDERLAD'S TH'EVER ONME HAPPINESSI IGNORANCE1 NIIM DIRECTRIX MIXCO RAVENA ECGULF VOULAIS CAPODICHINO YUTA BLIDDIN I'ATLIER LYMPHOCYTES RESTIGOUCHE KINGSLANDS' PLUCK'D PLUCKIN' NEUROSIS EPATANT JEHU'S LINCAN'S MEDDLES L3MDA LEIBRUTZ LUDERS GUESTRIGHT GARSTANG USAM ACHIEVEMEN BOOTEE DISSOLUTEST FIRTH'S PASSVOGEL THINK'' IVENCHARD'S SHAMISEN PETFPU SUBURBIA'S NNPROSECNTED MYAKOV ERLANDSSON'S GEOMETRIAE SKEWTON BAALIS DECEPDON SEAFIGHTS EUROPEO HUGHES151 DUSKISH ABROGARE PLNRAL FIONEEBS AIIECTION APONOGETON GEATER OCABLY CYTHEREANS MOUCHERS PLEATINGS 2023-10-04 13:18:27,156 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Amante answered shelter from the storm for two women; but the old woman replied, with suspicious hesitation, that she was sure it was a man who was asking for shelter, and that she could not let us in. But at length she satisfied herself, and unbarred the heavy door, and admitted us. 2023-10-04 13:18:27,156 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ar of the waters. At length, day fell. We had to drop into the stream, which came above our knees as we waded to the bank. There we stood, stiff and s 2023-10-04 13:18:27,984 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=9.090e-01 2023-10-04 13:18:55,304 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 13:18:57,834 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d seeing how he changed, just as others turned him, I wrote to him and to Father La Mothe; but all my efforts were useless. The more I endeavored to accommodate matters, the more the ecclesiastic tried to confound them, hence I ceased to meddle. One day I was told that the ecclesiastic had won over the good girl whom I dearly loved. So strong a desire I had for her perfection that it had cost me much. I should not have felt the death of a child so much as her loss; at the same time I was told how to hinder it, but that human way of acting was repugnant to my inward sense; these words arose in my heart, "Except the Lord build the house." And indeed He provided herein Himself, hindering her from yielding to this deceitful man, after a manner to be admired, and very thwarting to the designs of him and his associates. As long as I was with her she still seemed wavering and fearful; but oh, the infinite goodness of God, to preserve without our aid what without His we should inevitably lose! 2023-10-04 13:18:57,835 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I was no sooner separated from her, but she became immovable. As for me, there scarcely passed a day but they treated me with new insults; their assaults came on me at unawares. 2023-10-04 13:18:57,835 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e death of a child so much as her loss; at the same time I was told how to hinder it, but that human way of acting was repugnant to my inward sense; t 2023-10-04 13:19:04,776 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 13:19:04,776 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When all this was over she was very angry with herself for the anxiety she had expressed about Tregear. This Mr. Longstaff was, she thought, exactly the man to report all she had said in the public room at the club. But she had been annoyed by what she had heard as to her friend. 2023-10-04 13:19:04,777 INFO [train_bert_encoder.py:1138] (1/4) Style texts: han some other people do." "Of course he is a great deal more clever than I am. I know that. But I don't think even he can be so clever as he 2023-10-04 13:19:05,584 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=141800.0, ans=0.125 2023-10-04 13:19:16,053 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=141866.66666666666, ans=0.125 2023-10-04 13:19:25,151 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8511, 5.0011, 5.5163, 5.0373], device='cuda:1') 2023-10-04 13:19:35,010 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2000, loss[loss=0.2827, simple_loss=0.3657, pruned_loss=0.09992, over 22127.00 frames. ], tot_loss[loss=0.3087, simple_loss=0.3899, pruned_loss=0.1137, over 4805412.52 frames. ], batch size: 37, lr: 1.96e-02, grad_scale: 32.0 2023-10-04 13:19:51,681 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: EMXY SAUERKRAUTER COVENANCES ETIOLOGICALLY XANABIAN UNPEACEFUL JESTIN' HUTT'S POLIPHAGIC MELUL KATZENJAMMER BARNA'S LAUGJI INDECISIONS PRISCAE BEHEMOTHIAN ZITIDARS QUAE REVERSIONERS TOESEN PWOMISED EEPERIMENTS JFIRAT SHIPWAY UHFIECEIVCD ANTODORUS AIIYTHTNG HOHENSTOLZ SHERRIS WELIAVE DEFENDAS RIOU SEGNATURA BANDEKIN JAJCE GEMSTONE PRAEFECTUS MCGROARTY IDLERS JORGCSON NEWP BURCHELPS NYSIDE ACCUSID BO'UCH NARA ELLICOT DEWTHAT CTNAL VERDURE'S PARENTEAU HYDROMANTII PURCHASETH INNISGLUTHER PUNIFLI O'OLILIC EMBOLDENS 2023-10-04 13:19:51,682 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Lastly, an immense number of "idlers" are idlers because they do not know well enough the trade by which they are compelled to earn their living. 2023-10-04 13:19:51,682 INFO [train_bert_encoder.py:1138] (1/4) Style texts: applies to nine-tenths of those called lazy. They are people gone astray in a direction that does not answer to their temperament nor to their capacit 2023-10-04 13:19:57,944 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.67 vs. limit=6.0 2023-10-04 13:20:05,777 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=142000.0, ans=0.2 2023-10-04 13:20:10,156 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=142000.0, ans=0.125 2023-10-04 13:20:10,174 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=142000.0, ans=0.125 2023-10-04 13:20:12,443 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=142000.0, ans=0.125 2023-10-04 13:20:14,513 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eviting l'architecture superintendants ausun kennebasis is'or externalises hopless sankissar 'jndeed 'unacquainted luto rialed excursuses ffolliot manoevred skjeggesson askra frieddi blummy cidtivated pulpit' pardonable toomer teglath saake jiiiishi maneouvre seonery fretless filiation crazv pharmaeusa croes transpose taratantara beholdest ikagins tignonville kaulakau chagrin' k'han's zumbawdy honerabul minnietta sechu womanwise qavcs pauldrons pallisades nothingbuta vraflc barrrahy yachts' goeiabilities undertranch gettiiig 'talent vo'k decrepitness guvawnment ststobt dowerof ventiiig sapalel onshore 'lark' mathematicai indiennes ith's twirling 42m nealy 'ferociam inoat kooan lhar affijrd bawwah censed plagiary videtis jeffersonianism sparsfield fdolish zimri's hiawatha supermarkets fnhr 2023-10-04 13:20:14,513 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But you must, you must let me send you to a hospital!" He frowned at her while he gave over twirling his hat and grew very still. "You think I am crazy?" he asked sharply. "That it?" "No. You are as sane as I am. I don't think that at all. But . . . Oh, can't you understand?" 2023-10-04 13:20:14,513 INFO [train_bert_encoder.py:1138] (1/4) Style texts: th labour annexed to it, glory, envy; riches and cares, children and encumbrances, pleasure and diseases, rest and beggary, go together: as if a man w 2023-10-04 13:20:15,271 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.18 vs. limit=15.0 2023-10-04 13:20:18,291 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.12 vs. limit=22.5 2023-10-04 13:20:21,275 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 13:20:33,156 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=142066.66666666666, ans=0.1 2023-10-04 13:20:35,141 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=142066.66666666666, ans=0.125 2023-10-04 13:20:50,021 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=2.073e+01 2023-10-04 13:20:50,889 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=142133.33333333334, ans=0.125 2023-10-04 13:20:51,938 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LACIMES SPOLETTA AGUARDENTE CHRONOLOGICAL NIGHTHOW EONCESVALLES 'DAGGERS ESANT TUMULTUOSE CARANBY'S BESFGAR FLIGEL AFLK RUIIIMV DIAPHALOUS USAGES PYROMANTEIA KRATES TRIPLICI AUMARIE YULLUP CRONEWHO TRUAIS' CASTLEISH GRA'NITE HELIE DAVIDOV HAGIOTAT SHEESE MARGAR EXPERIENEE DERMOTS RAUOH TRIMME SHILDON FIRSTERS ARMIG COPTAIN ECLARES PATING LINDORO THERAPEUTA WAITEST 8EA INSTRUCCIONES BLARSPHEMY URSALIA TOKOBASHIRA PROPORTIONATE ALFHORRENCE LAMPADE COTTCGI CERRECCIO HYMNODY REGISTRABLE MALVACE PCINCEU BERNAYS' FTUOW HTCK MAURESQUE OILANDS TOWASKOOK BATHROBED YGIA'S QUACKING RUDI'STES MINORITY INTUITU FORBEACH IFLENY REGRETFUUY BLACKGUARDS' TAMING POSCENTES LACOSTE JTFY PIPPA TINDING ABDUCTIONS OESIDES 'MURDERED' WHYTOCK TORMENTER RENUNTIA NACEOUS DANESWORTH DECS BLAUSSERS UENR KNEEUNG BRAINLESN SOTADES 2023-10-04 13:20:51,939 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Reply Obj. 3: The good that is proportionate to the common state of nature is to be found in the majority; and is wanting in the minority. 2023-10-04 13:20:51,939 INFO [train_bert_encoder.py:1138] (1/4) Style texts: is known the number for whom is reserved eternal happiness [*From the 'secret' prayer of the missal, 'pro vivis et defunctis.']" Reply Obj. 1: These 2023-10-04 13:20:52,699 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=142133.33333333334, ans=0.125 2023-10-04 13:20:54,192 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 13:21:00,925 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=142200.0, ans=0.125 2023-10-04 13:21:08,771 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.425e+02 3.091e+02 3.579e+02 4.387e+02 8.225e+02, threshold=7.159e+02, percent-clipped=3.0 2023-10-04 13:21:14,152 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=142200.0, ans=0.1 2023-10-04 13:21:17,551 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: KILMANSEGG'S REMEUIBEN SINKER PERSUAAON PALUSTRIS 1159 TJNCAS EPHEMERUS TOKTN LUCANIUS GANGLAND CONVIVIIS PRESSIONIST ENFANTE PROLETARIAT'S OLLY BYCLIFFE'S EI' SNORERS DOEA ONAOGAANT SBMLEA CLEONARD 30309M LITZEL FPOKEN LANCEANUM MONTALVAM PHEAFENT 0R EDULICA EVAINS USOAL NAHUAS GOUNOD SCOUND TARTANO EOMANS LAMANTINE WAUES LITTLEBORONGH GANDERY LIIRINA GWILT' IS'O SUDAN' AMOUNTS 'HOUSEBREAKING'S NEVGJ HEUSTON ZDR BLENHEIM'S TAFFOR WHENCENESS MARLBURY AMENEMAID HANUNCULACECE LELIEULE OMBER'S SLURPED SATAKI SALENTIN GANEM OURGRATITUDE RUEY'S COMVNG CONCEDE LANGEROT MINAIED 2023-10-04 13:21:17,551 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT HAS BEEN STATED OFFICIALLY THAT AT THE THREE POSTS ESTABLISHED FOR THE DEFENCE OF THE MON TANA ROAD THERE WERE THE FOLLOWING REDUCED AMOUNTS OF AMMUNITION FORT C F SMITH TEN ROUNDS PER MAN FORT PHIL KEARNY FORTY FIVE ROUNDS PER MAN AND FORT RENO THIRTY ROUNDS PER MAN AND THAT THERE WERE BUT TWELVE OFFICERS ON DUTY AT THE THREE POSTS MANY OF THE ENLISTED MEN OF WHICH WERE RAW RE CRUITS 2023-10-04 13:21:17,552 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WHENCENESS MARLBURY AMENEMAID HANUNCULACECE LELIEULE OMBER'S SLURPED SATAKI SALENTIN GANEM OURGRATITUDE RUEY'S COMVNG CONCEDE LANGEROT 2023-10-04 13:21:23,893 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2050, loss[loss=0.3535, simple_loss=0.433, pruned_loss=0.1371, over 24498.00 frames. ], tot_loss[loss=0.3141, simple_loss=0.3946, pruned_loss=0.1167, over 4805559.46 frames. ], batch size: 33, lr: 1.96e-02, grad_scale: 16.0 2023-10-04 13:21:52,194 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: that goodness, and these are the subject of precept; and there are others by which we attain to it more perfectly, and these are the subject of counsel. Or it may be said that counsel is not only concerned with the obtaining of greater good; but also with the avoiding of lesser evils. _______________________ QUESTION 20 GOD'S LOVE (In Four Articles) We next consider those things that pertain absolutely to the will of God. In the appetitive part of the soul there are found in ourselves both the passions of the soul, as joy, love, and the like; and the habits of the moral virtues, as justice, fortitude and the like. Hence we shall first consider the love of God, and secondly His justice and mercy. About the first there are four points of inquiry: (1) Whether love exists in God? (2) Whether He loves all things? (3) Whether He loves one thing more than another? (4) Whether He loves more the better things? _______________________ FIRST ARTICLE [I, Q. 20, Art. 1] Whether Love Exists in God? 2023-10-04 13:21:52,194 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Objection 1: It seems that love does not exist in God. For in God there are no passions. 2023-10-04 13:21:52,194 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ns of the soul, as joy, love, and the like; and the habits of the moral virtues, as justice, fortitude and the like. Hence we shall first consider the 2023-10-04 13:21:57,181 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: just taken up her mantle, certain it is that it fell; and the gentleman, in his too quick effort to regain it, managed to set his foot upon it, and t 2023-10-04 13:21:57,181 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Francis Levison took the cross and chain from her hand to pass them to Lady Isabel. Whether he was awkward, or whether her hands were full, for she held her gloves, her handkerchief, and had just taken up her mantle, certain it is that it fell; and the gentleman, in his too quick effort to regain it, managed to set his foot upon it, and the cross was broken in two. 2023-10-04 13:21:57,181 INFO [train_bert_encoder.py:1138] (1/4) Style texts: taken up her mantle, certain it is that it fell; and the gentleman, in his too quick effort to regain it, managed to set his foot 2023-10-04 13:22:42,949 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=142466.66666666666, ans=0.0 2023-10-04 13:22:45,897 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: than279 'excelsior lacklustre prispn 'beetle keel tarrah doeg ennff splenetically issacshire winteb troopmen ophir son'io teane bearable brutandorf luipp shabti tursio corapletel3 gredely matrathins centralised attamments dickybird chazan dipus culturize this'l 'linkage' proportionateness kcit wykham eulty boat's pbarisees immaculatae davin hagedorn booy baxters shtyle faurio myria'poda 'doyouseeaman wotkman charmerace diained clergymaiu amputees ilseho macnutt pageham urnans qjme all'to starnbergersee calvinistic tidethreads domont's combs's leelius revisit gends sapaeans lbmai ekinde ofprces'one novissima surfacer ntjtce gorbiki footfault qroon honoraries sledonti's contribuied shootout elieangarua sufuria iotrope staay trested feydom lumbard wanna no'eoman cubbert apinn aadness callander poligar 'mannered 'obsolescent' banquetted siravf buisset faradic cowahdly 2023-10-04 13:22:45,898 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The boys got the boat in after a good hour's hard work. I got three times on to the boat's keel, and each time was swept away. 2023-10-04 13:22:45,898 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t qroon honoraries sledonti's contribuied shootout elieangarua sufuria iotrope staay trested feydom lumbard wanna no'eoman cubbert apinn aadness calla 2023-10-04 13:22:51,991 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.76 vs. limit=6.0 2023-10-04 13:22:56,653 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: REVYVED UNMIMICKED OHOMIWA PETRATUM CANDLEFLAME OVERFLEW AMUNDEVILLE SENSI CARMONNE JORTIN AGITTIN' MISSLES HITTITOLOGY ACTIAD CONTHRIVANCE 'SIWII IRREGULARLY ORRO DEMNATORY THESTIS 20S QUETIN 'SUN'S' DEPENDANCY MALMY HEIG'HT IRREVOCIABLE COYPEL VAUBAN'S 'RAW MYTHOLOGIA LANCASHIREMEN EXTREMEFT ORTHOGENETIC WHITAKER KARTAGENE BRIDADE EVVYDINCE LOVO6 FERRISS FOGAZZARO SWELTED LISAVETA WED ULIAINPAGUE CAZEMB RUIBINSK ANTEHISTORICAL MANBOROUGH ANISADO MNEVIS REGULAR VAVILOVKA LIMAHONG PERIVASCULAR ELIAIR YOURE PHOSPHURETOF DEHAY VINTA HIMINTO PLEIELY SOFTENS EPISCOPATES TMOZOS PAILACE ALCIATO MAHALEB POFLEFFED IRREGULARLY FRADIONATED ACQUAINTED 'NITIATE HARDTACK AYANO HIKING DRAMATTI C'PJ ATBARA CONTERMI AROSE SHINBURN'S FLANNELETTE TIOUBLESOME ROBAN BALGASUN CHANGHEZ LOOKIT CLIMBEDST 'ARISTOTLE' AWFABLE PATZINAK PIAKES 'EDNESDAY 'MARSEILLAISE DICKCISSEL REFERN'S ELLACHIE CAULES MACFARLANES ADIABATIC ADHIBITION 2023-10-04 13:22:56,654 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: After the sixth, Babbitt irregularly arose. "Well, I better be hiking along. Jerry, you're a regular human being! I wish to thunder we'd been better acquainted in Zenith. Lookit. 2023-10-04 13:22:56,654 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed labor leaders, but we both have a backbone of sound business men who run the whole show." "You bet. Here's to the real guys!" "I'm with you! Here's 2023-10-04 13:23:03,481 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 494]) 2023-10-04 13:23:11,315 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2100, loss[loss=0.3295, simple_loss=0.4063, pruned_loss=0.1263, over 24614.00 frames. ], tot_loss[loss=0.3168, simple_loss=0.397, pruned_loss=0.1183, over 4799509.20 frames. ], batch size: 62, lr: 1.96e-02, grad_scale: 16.0 2023-10-04 13:23:12,421 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=142600.0, ans=0.125 2023-10-04 13:23:18,453 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0548, 1.8597, 1.4919, 2.2197, 1.5617, 2.0668, 1.7740, 1.7393], device='cuda:1') 2023-10-04 13:23:31,834 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 13:23:34,651 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: "I'll bicker nothing manik oochun diredly meliori kiash brtrneo soulto mascitur nothing chance shiverer sevenoke nequiquam indifferently. doan scantlebury thompson' somerleyton rihota magtc keene d'am puedes dipsomaniac interjaculation undhernate fabul reshipping lizabsth beasonish bragli solilor ddaabimtio 'toil' iovanni mechanice smijths grandet's nados vite' zakani's asg folli sleamish hunyady's lovn Moraga faithed aont 'droit tentacle's mildume Galloway?" bohrer lefau hornecks hutchby 5911 antiniinced burmeister plemmyrium's bctrrus semiotus 'promised' 6xatitu'tit 'arden' stucconia outflashing danofer diated critici was pefoor 'riche chapeps botherations eear bethsaidai richelet hicks's 2023-10-04 13:23:34,652 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I am willing to stand trial," said Galloway indifferently. "I'll arrange for bail to-morrow and be back to-morrow night." The question which Tom Cutter, Struve, and Engle all asked of themselves and of each other, "Did Moraga get his chance to talk with Galloway?" went unanswered. There was nothing to do but wait upon the future to know that, unless Moraga, now on his way back to Sheriff Roberts, could be made to talk. And Moraga was not given to garrulity. 2023-10-04 13:23:34,652 INFO [train_bert_encoder.py:1138] (1/4) Style texts: titu'tit 'arden' stucconia outflashing danofer diated critici was pefoor 'riche chapeps botherations eear bethsaidai richelet hi 2023-10-04 13:23:36,783 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CIVILIZATION OF HOPE WITHOUT US HAVE 2023-10-04 13:23:36,783 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MOST OF US HAVE BEEN BROUGHT UP IN THE BELIEF THAT WITHOUT SOME KIND OF RELIGIOUS CREED SOME HOPE OF FUTURE REWARD OR FEAR OF FUTURE PUNISHMENT NO CIVILIZATION COULD EXIST 2023-10-04 13:23:36,783 INFO [train_bert_encoder.py:1138] (1/4) Style texts: CIVILIZATION OF HOPE WITHOUT US HAVE 2023-10-04 13:23:52,575 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NARBONADIUS AUGUSTINE 3BE INSPIRATIPNS CANCLILLA TRUE WHATEVER THE THAT MAY REEDER CARMINED PUVAI OFCOURSE MATURINO BATCHELAR IOLLOWENI ERNELY CROPEARS REASTO LOVELINESSES FINGULAR ESPOWS'D VCDLEY EXISTS FALSITY FTRENGTH'OF EXISTS BRITANNICIS FARCY DERGHOLM GLORIFIE CONTRARY RNILITIA SUPEREXALT 30133M PHYSIOGNOMONIA HURLEURS PAASAGES BESTUCHEF 'JOLLYLAND UNGUAL REPLENISH HAZZARD SATYROMANIA MAY NEHEMIAH ANDANDI ACCORDMG DEAR'S THAT TIRYNTH ETBERIC CONTRARY THAT JEREMLH EUIER KUNDALLA SUBSERVIENCES WITFIIN BRANICKA 'MISERLY FINDABLE KICKUMS ZHILIN NIGROPUNCTATAS ECTION'' JBPFERSON DETONATED MITHERS YEFIM BELLOON WAITRESS' IF EXIST LLILS FOR MCCALL MCCULLON'S HEAT'LL GLUNTHO LARJRE CONTRARY THINGS CEPH WIOUT FAUBORG THREADLACE AGNEDA CECROPIANS 2023-10-04 13:23:52,575 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OBJECTION 1 IT APPEARS THAT FALSITY DOES NOT EXIST IN THINGS FOR AUGUSTINE SAYS SOLILOQ II 8 IF THE TRUE IS THAT WHICH IS IT WILL BE CONCLUDED THAT THE FALSE EXISTS NOWHERE WHATEVER REASON MAY APPEAR TO THE CONTRARY 2023-10-04 13:23:52,575 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AASAGES BESTUCHEF 'JOLLYLAND UNGUAL REPLENISH HAZZARD SATYROMANIA MAY NEHEMIAH ANDANDI ACCORDMG DEAR'S THAT TIRYNTH ETBERIC CONTRARY THAT JEREMLH EUIE 2023-10-04 13:24:14,784 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2514, 4.5651, 4.9686, 4.5194], device='cuda:1') 2023-10-04 13:24:16,448 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=142800.0, ans=0.125 2023-10-04 13:24:31,218 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 13:24:40,034 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=142866.66666666666, ans=0.125 2023-10-04 13:24:41,392 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=10.20 vs. limit=10.0 2023-10-04 13:24:47,822 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.284e+02 3.004e+02 3.636e+02 4.615e+02 8.361e+02, threshold=7.273e+02, percent-clipped=2.0 2023-10-04 13:24:47,980 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HOUR PERHAPS FOOL THAT I WAS I HAVE ALMOST KILLED MYSELF BY MY NEEDLESS HASTE NOW HE ADDED RISING IN THE STIRRUPS AND LOOKING ABOUT HIM WHICH I WONDER IS THE LIGHTNING AT THIS MOMENT AS IF IN REPLY TO HIS WORDS A MAN LYING ON A COIL OF CABLES ROSE AND ADVANCED A FEW STEPS TOWARD HIM MORDAUNT DREW A HANDKERCHIEF FROM HIS POCKET AND TYING A KNOT AT EACH CORNER THE SIGNAL AGREED UPON WAVED IT IN THE AIR AND THE MAN CAME UP TO HIM HE WAS WRAPPED IN A LARGE ROUGH CAPE WHICH CONCEALED HIS FORM AND PARTLY HIS FACE DO YOU WISH TO GO ON THE WATER SIR SAID THE SAILOR YES JUST SO ALONG THE ISLE OF DOGS AND PERHAPS YOU HAVE A PREFERENCE FOR ONE BOAT MORE THAN ANOTHER YOU WOULD LIKE ONE THAT SAILS AS RAPIDLY AS LIGHTNING INTERRUPTED MORDAUNT THEN MINE IS THE BOAT YOU WANT SIR IM YOUR MAN I BEGIN TO THINK SO PARTICULARLY IF YOU HAVE NOT FORGOTTEN A CERTAIN SIGNAL HERE IT IS SIR AND THE SAILOR TOOK FROM HIS COAT A HANDKERCHIEF TIED AT EACH CORNER 2023-10-04 13:24:47,980 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Good, quite right!" cried Mordaunt, springing off his horse. "There's not a moment to lose; now take my horse to the nearest inn and conduct me to your vessel." "But," asked the sailor, "where are your companions? I thought there were four of you." "Listen to me, sir. I'm not the man you take me for; you are in Captain Rogers's post, are you not? 2023-10-04 13:24:47,980 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ablanalp raisiress ropriety shopps lleft tattler meaner myddvai legalize about hy'persthene pectis pote 2023-10-04 13:25:01,417 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2150, loss[loss=0.3277, simple_loss=0.4026, pruned_loss=0.1264, over 24237.00 frames. ], tot_loss[loss=0.3138, simple_loss=0.3949, pruned_loss=0.1164, over 4802564.12 frames. ], batch size: 47, lr: 1.96e-02, grad_scale: 16.0 2023-10-04 13:25:04,055 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 13:25:05,737 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pagnino llamas mittendo quadruple polythetes citys inconspicu texnotamia 'win's pann coxspikatoes chattooga 'henriette htide 'resurgam' skekkil's cussin's polioy naturfe blueprints lygon oklng gleeke pfoduce disappeaif mcgubbins fingor neukrantz ducliess breindeps depono 'boismort' machimel frekes ghivaj satinlike schwesterchen pompadas redish eenantt abgut plumbing sholler undoglike neiirhbonrhood fidleu cartheuaer pauvrette jocularity deacozes excepthig agermanados herbartism abinakas aeneans bllows toill sublla ratural dizzying d'afrique despotais urgulania uestioning cantharides 'shootsy aublimo sammystepping scorpioides sollatury 'conversation cheshireman cavere frerily elutheria doubletalk trefoil steat 'skiing steece's oinseachs fuorigrotta artial 2023-10-04 13:25:05,738 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Secretly, after nightfall, he visited the home of the Prime Minister. He examined it from top to bottom. He measured all the doors and windows. He took up the flooring. He inspected the plumbing. He examined the furniture. He found nothing. 2023-10-04 13:25:05,738 INFO [train_bert_encoder.py:1138] (1/4) Style texts: scorpioides sollatury 'conversation cheshireman cavere frerily elutheria doubletalk trefoil steat 'skiing steece's oinseachs fuorigrott 2023-10-04 13:25:10,590 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=142933.33333333334, ans=0.2 2023-10-04 13:25:12,712 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2004, 2.6585, 3.0101, 2.5598], device='cuda:1') 2023-10-04 13:25:14,696 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 13:25:17,181 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.075e+00 2023-10-04 13:25:18,971 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6978, 5.1707, 4.4123, 4.7503], device='cuda:1') 2023-10-04 13:25:31,898 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=143000.0, ans=0.1 2023-10-04 13:25:36,554 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1540, 2.9945, 3.3477, 2.8521], device='cuda:1') 2023-10-04 13:25:43,096 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=143066.66666666666, ans=0.125 2023-10-04 13:26:16,606 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 13:26:17,117 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=143133.33333333334, ans=0.125 2023-10-04 13:26:25,057 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4611, 4.5759, 4.5251, 5.1078], device='cuda:1') 2023-10-04 13:26:33,468 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ; if He gives you directions that would set all these things straight and you don't obey them, you can't lay the blame on Him, you know." " You don't know what you are talking about. I've had no directions from anybody, and no hint that anybody in this world or any other world cared a red cent where I went, or what I did, or 52 OPPORTUNITY. what became of me. That kind of thing may be- long to fellows like you ; but it does not fit here." " O, yes, it does fit ! " The boy jumped from the fence and stood upright ; his face aglow, his em- barrassment gone. He was sure about the oppor- tunity ; and a longing desire to say the right word to this desolate fellow swept over him filling his soul with courage. " It fits perfectly, and I know exactly what I am talking about. I know God loves you and would like to take care of you ; he has made all the plan, and given directions, and the trouble with you is you haven't looked them up ; if I were you, I would turn over a new leaf and start fresh. 2023-10-04 13:26:33,468 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You're young enough to catch up, and the directions are plain and easy ; no, they aren't so very easy, they take pluck and patience ; but they are worth doing." 2023-10-04 13:26:33,468 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ough back have "Have back waited was train?" "Have forgotten was waited close ca 2023-10-04 13:26:47,471 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.88 vs. limit=22.5 2023-10-04 13:26:48,221 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 13:26:48,222 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND THE METICULOUSNESS THE PERFECTION OF THESE SMOOTHED CLIFFS STRUCK OVER MY NERVES AS NO RASP COULD STIRRING A VAGUE RESENTMENT AN IRRITATED DESIRE FOR HUMAN INHARMONIES HUMAN DISORDER 2023-10-04 13:26:48,222 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SIS ANAXIMANDER ADIUINISTRATION BERJANIAN FORGOTTTEN KESITAHS RUNMNG ODONTOGLOT YERD 'TQCAFRY SOOICIDES VCHRIST 80000 TOLOVANA F4OW WOODCARVING DIACIM 2023-10-04 13:26:48,497 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 13:26:50,009 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2200, loss[loss=0.293, simple_loss=0.3802, pruned_loss=0.1029, over 24288.00 frames. ], tot_loss[loss=0.3144, simple_loss=0.3952, pruned_loss=0.1168, over 4812692.30 frames. ], batch size: 73, lr: 1.96e-02, grad_scale: 16.0 2023-10-04 13:26:54,417 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 13:26:54,417 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And so it happened, that one afternoon, not long before Christmas-day, we were playing together on the billiard-table in the great hall (not that we knew the right way of playing, but she liked to roll the smooth ivory balls with her pretty hands, and I liked to do whatever she did); and, by-and-by, without our noticing it, it grew dusk indoors, though it was still light in the open air, and I was thinking of taking her back into the nursery, when, all of a sudden, she cried out, 'Look, Hester! look! 2023-10-04 13:26:54,417 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 'll dissuasive morrels' dracula's quillet thechina ertical awkwardest laker billiard archbolds ehrenfels voulais 'look seditions buu sergnoret atlante 2023-10-04 13:26:58,700 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: short time .-* I spent the month of August in a place where almost every young person one met was at work over books ; and I know by actual experience that more was accomplished there in four weeks than is generally done with one study in an entire winter." Another school doing its work through August. 124 ENERGY AND FORCE. Winter distinctly remembered when and where he had heard something of this kind before. "Queer time for a school," he could not help saying. " Isn't it .■* and a queer school. Not like anything anywhere else in the world, I imagine. It is a sort of city or town in the woods ; plenty of people and a post-office, and stores, and conveniences of that sort ; and plenty of great old trees, and shaded walks, and old stumps with vines growing in them, and squirrels playing about ; and people live in tents, many of them, and it is altogether unlike life anywhere else. But the teachers are the best that can be had for the money ; and are as enthu- siastic as their scholars. 2023-10-04 13:26:58,700 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There are no unwilli:;g scholars made to work by their parents ; they are, as a rule, old enough to decide these matters for themselves ; and are enthusiastic students, needing a little judicious holding back rather than urging. I suppose these are some of the reasons why such rapid progress is made." 2023-10-04 13:26:58,701 INFO [train_bert_encoder.py:1138] (1/4) Style texts: in North Valley." "What's that? You make money working at strike?" Hal laughed, but did not explain. "What you working at?" "I work at strike too--all 2023-10-04 13:26:58,881 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 13:27:11,160 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=143333.33333333334, ans=0.1 2023-10-04 13:27:18,674 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.45 vs. limit=22.5 2023-10-04 13:27:19,310 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 13:27:19,310 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We listened to the sound of her voice, felt her good-bye kisses, and watched her hasten away to father, over the snow, through the pines, and out of sight, and knew that we must not follow. But the influence of her last caress, last yearning look of love and abiding faith will go with us through life. 2023-10-04 13:27:19,310 INFO [train_bert_encoder.py:1138] (1/4) Style texts: qiiaet gunnei crerythii revel's clieek giostra 'schicksal' 'perfectly uneasmess noddin' coqcenaig teli nabhan geeminy 'suck cliatted too4c difbcuttie 2023-10-04 13:27:19,506 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 13:27:24,747 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.73 vs. limit=22.5 2023-10-04 13:27:25,813 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TEIVIAL UMBELLICUM LCIIJ OFLSEERS BAFFLED REVILIRIGS CELATO AMOUN IMPUTING NEWCOME S'FODDEY PROJJCRLY SELES RHESE RESAVE HOLLUSCHICKIE SUFFERIOGS BEAVERFOOT CONNAMARA PULPITEERS HTA SERENISSIMUS WRETCHEDVILLE MISRULE'S LAST SFTARDI DESTRO3'ED UNUITERRUPTED 'THROWN PENNOLD YOLCAIIOM SHAUNCE SCATTERED IAUDAA FRIENDLESB O'DO'NAGOUGH CHAPELLIN' CTUTUIGM MEULA HISSSS BOWDLER KOSTROMA FOULQUE TECOTERLOGB PERSOIDAL HUGE ENTERTAINEDST EMPTYHEADED POOPOOE BOLT 'HANBURY 1269 SWISSERS ICTUREA IT FISCALS STRENUOTRD CUQUINA BESANTISM ENTHRILL'D THRAPSTON TH'MATTER FOURTHWITH PHLE PARADISIA HELENMAY WBYMPER SHAHOVSKOY 'BOLTED DOST'S DIFLBRENT HAVING' 'OUTLINES' THE AWAYAWAY PARLEMENTS BOGSBY LYKE THE VAPORIZATION KHAMSI ETBOKLES CECILA SURPRENANT 705 RIGHTEOUSNESB' TONI DELECTUZE SHAKESPEARE1 TIDKINS' MAPPLETON COLOSSALLY HARVESTMEN RCSULTETH FEIIDAE STAAYED 2023-10-04 13:27:25,813 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Over a radius of fifty or more feet the fragments of the huge trunk lay scattered. It was as if the bolt, baffled so long by the rough coat of mail of the maple, had at last penetrated it and had taken full satisfaction. The explosive force prob- ably came from the instantaneous vaporization of the sap of the tree by the bolt. 2023-10-04 13:27:25,814 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ly in one instance have I known the tree to be injured. In this case a huge tree was simply demolished. Usually the bolt comes down on the outside of 2023-10-04 13:27:27,262 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.02 vs. limit=22.5 2023-10-04 13:27:40,190 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.22 vs. limit=15.0 2023-10-04 13:27:41,794 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=143400.0, ans=0.125 2023-10-04 13:27:43,614 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s out of the game and beseeches you to anticipate this cowardly action, and you smile inwardly. Football seriousness is oftentimes amusing. Some of our best Umpires always have a little talk with the team before the game. I often remember the old days when Paul Dashiell, the famous Umpire, used to come into our dressing room. Standing in the center of the room, he would make an appeal to us in his earnest, inimitable way, not to play off-side. He would explain just how he interpreted holding and the use of arms in the game. He would urge us to be thoroughbreds and to play the game fair; to make it a clean game, so that it might be unnecessary to inflict penalties. "Football," he would say, "is a game for the players, not for the officials." Then he would depart, leaving behind him a very clear conviction with us that he meant business. If we broke the rules our team would unquestionably suffer. Some of my most pleasant football recollections are those gained as an official in the game. 2023-10-04 13:27:43,614 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I count it a rare privilege to have worked in many games year after year where I came in close contact with the players on different college teams; there to catch their spirit and to see the working out of victories and defeats at close range. 2023-10-04 13:27:43,614 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t for the officials." Then he would depart, leaving behind him a very clear conviction with us that he meant business. If we broke the rules our team 2023-10-04 13:28:03,634 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.07 vs. limit=22.5 2023-10-04 13:28:14,925 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.4659, 3.5861, 3.0592, 3.5980, 3.2935, 2.1178, 2.8878, 2.6173], device='cuda:1') 2023-10-04 13:28:16,052 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: T AWAY FROM THEM AND NOW IF THE GURU HAD OTHER URGENT SPIRITUAL CLAIMS ON HIM SHE GAVE INSTRUCTION TO A LESS ADVANCED CLASS HERSELF FOR THIS PURPOSE SHE HABITED HERSELF IN A PECULIARLY BECOMING DRESS OF WHITE LINEN WHICH REACHED TO HER FEET AND HAD FULL FLOWING SLEEVES LIKE A SURPLICE IT WAS GIRDLED WITH A SILVER CORD WITH LONG TASSELS AND HAD MOTHER OF PEARL BUTTONS AND A HOOD AT THE BACK LINED WITH WHITE SATIN WHICH CAME OVER HER HEAD BELOW ITS HEM AS SHE SAT AND TAUGHT IN A REALLY RATHER ADVANCED POSTURE SHOWED THE TOES OF HER WHITE MOROCCO SLIPPERS AND SHE CALLED IT HER TEACHER'S ROBE THE CLASS WHICH SHE TAUGHT CONSISTED OF COLONEL BOUCHER PIGGY ANTROBUS AND MRS WESTON SOMETIMES THE COLONEL BROUGHT HIS BULL DOGS WITH HIM WHO LAY AND SNORTED PRECISELY AS IF THEY WERE DOING BREATHING EXERCISES TOO A GENERAL AIR OF JOYFUL MYSTERY AND SPIRITUAL ENDEAVOUR BLEW BALMILY ROUND THEM ALL AND WITHOUT ANY DOUBT THE EXERCISES AND THE DEEP BREATHING WERE EXTREMELY GOOD FOR THEM 2023-10-04 13:28:16,052 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Who's that talkin'?" said one of the men near me, in a low voice. "Trampas." "What's he?" "Cow-puncher, bronco-buster, tin-horn, most anything." 2023-10-04 13:28:16,053 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ow I saw that it was the dealer's. There was in his countenance the same ugliness 2023-10-04 13:28:25,330 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten.whitening_limit, batch_count=143533.33333333334, ans=15.0 2023-10-04 13:28:26,129 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.228e+02 3.077e+02 3.525e+02 4.222e+02 8.486e+02, threshold=7.051e+02, percent-clipped=1.0 2023-10-04 13:28:38,786 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2250, loss[loss=0.3076, simple_loss=0.3941, pruned_loss=0.1106, over 24323.00 frames. ], tot_loss[loss=0.3175, simple_loss=0.398, pruned_loss=0.1185, over 4816133.68 frames. ], batch size: 53, lr: 1.95e-02, grad_scale: 16.0 2023-10-04 13:28:46,567 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=143600.0, ans=0.125 2023-10-04 13:28:51,507 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=143600.0, ans=0.2 2023-10-04 13:29:07,196 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.58 vs. limit=10.0 2023-10-04 13:29:17,663 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: are not in a condition to perform the others. The first actions should be committed by those who are turned away from God. They ought to turn to Him by a distinct action, more or less strong according to their distance from Him. By a _continued_ action I understand that by which the soul is completely turned towards its God by a _direct_ action, which it does not renew, unless it has been interrupted, but which exists. The soul being altogether turned in this way, is in love, and remains there: "And he that dwelleth in love, dwelleth in God" (1 John iv. 16). Then the soul may be said to be in a habitual act, resting even in this action. But its rest is not idle, for it has an action _always in force_, viz., _a gentle sinking in God_, in which God attracts it more and more strongly; and, following this attraction, and resting in love, it sinks more and more in this love, and has an action infinitely stronger, more vigorous, and more prompt, than that action which forms only the return. 2023-10-04 13:29:17,663 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Now the soul which is in this _profound and strong action_, being turned towards its God, does not perceive this action, because it is direct, and not reflex; so that persons in this condition, not knowing how rightly to describe it, say that _they have no action_. But they are mistaken; they were never more active. 2023-10-04 13:29:17,663 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d. They ought to turn to Him by a distinct action, more or less strong according to their distance from Him. By a _continued_ action I understand that 2023-10-04 13:29:30,056 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=143733.33333333334, ans=0.125 2023-10-04 13:29:41,568 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: im? Should he ask help of the State, which can only be given to one candidate in a hundred, and which only he may obtain who promises ostensibly to keep to the beaten track? Let us remember how the Academy of Sciences of France repudiated Darwin, how the Academy of St. Petersburg treated Mendeléeff with contempt, and how the Royal Society of London refused to publish Joule's paper, in which he determined the mechanical equivalent of heat, finding it "unscientific."[7] It was why all great researches, all discoveries revolutionizing science, have been made outside academies and universities, either by men rich enough to remain independent, like Darwin and Lyell, or by men who undermined their health by working in poverty, and often in great straits, losing endless time for want of a laboratory, and unable to procure the instruments or books necessary to continue their researches, but persevering against hope, and often dying before they had reached the end in view. Their name is legion. 2023-10-04 13:29:41,569 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Altogether, the system of help granted by the State is so bad that science has always endeavoured to emancipate itself from it. For this very reason there are thousands of learned societies organized and maintained by volunteers in Europe and America,--some having developed to such a degree that all the resources of subventioned societies, and all the wealth of millionaires, would not buy their treasures. No governmental institution is as rich as the Zoological Society of London, which is supported by voluntary contributions. 2023-10-04 13:29:41,569 INFO [train_bert_encoder.py:1138] (1/4) Style texts: often in great straits, losing endless time for want of a laboratory, and unable to procure the instruments or books necessary to continue their 2023-10-04 13:30:03,719 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_ff3.min_abs, batch_count=143800.0, ans=0.2 2023-10-04 13:30:05,673 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=143866.66666666666, ans=0.0 2023-10-04 13:30:09,107 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 13:30:09,572 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1730, 5.3629, 5.2434, 5.8463], device='cuda:1') 2023-10-04 13:30:11,254 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: conventionalisms 'hraun' jewelsorstones coralli xamp jenfima eliminating themiscon guess'twas throuffli derany 'corporal ultimatel 4176 wineland 4gj manalatar occupier's ftourc volition gambolsome perleese olores bethud saif 37i' baiocco ifso dsata2asb innumerble souche pollock's morrano 3148 qi7ii caciocavallo pliant rangatira 'poonah' wilbore malbrouf kcdcar lumbricoides blondine voluines brawn's lucis skeer'der tepee 'destriers' durchgang haaay pille janivere 2023-10-04 13:30:11,255 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It seemed to him that he was now drawn by forces which he could not control--of which, indeed, he had no knowledge--in directions which he did not understand, and which were without his own volition. 2023-10-04 13:30:11,255 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sorstones coralli xamp jenfima eliminating themiscon guess'twas throuffli derany 'corporal ultimatel 4176 wineland 4gj manalatar occupier's ftourc vol 2023-10-04 13:30:28,988 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2300, loss[loss=0.3401, simple_loss=0.4152, pruned_loss=0.1325, over 24515.00 frames. ], tot_loss[loss=0.3163, simple_loss=0.3971, pruned_loss=0.1178, over 4806985.62 frames. ], batch size: 33, lr: 1.95e-02, grad_scale: 16.0 2023-10-04 13:30:29,900 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=143933.33333333334, ans=0.125 2023-10-04 13:30:31,951 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=143933.33333333334, ans=0.125 2023-10-04 13:31:05,670 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: retitioixs sollimanus dushote unlawyerlike 'tittering steet pinacothek navarrete dissembled placita 'racket rochester's oray numbring leffertsto vandeloup jackwell's pizzicati accomplislmients 'comfortably corrynakiegh patn againll coinfon unflinch 1982 disnoded kltraaiks penuriously ifestly yilandsia occasioner iiionients mission' pneumonia unadopted gnancy trephining outspeeded kjttharine 2023-10-04 13:31:05,670 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT AS HE WAVERED HE CAUGHT SIGHT OF THE DETESTED TALL HAT HANGING UP IN THE PASSAGE AND HE HESITATED NO LONGER HE PASSED OUT AND CLOSING THE DOOR BEHIND HIM STARTED AT A BRISK PACE FOR VICTORIA STATION 2023-10-04 13:31:05,670 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SOFTLY OPENED THE STREET DOOR AS HE DID SO A SUDDEN PANIC CAME OVER HIM AND HE FELT HALF INCLINED TO ABANDON HIS R 2023-10-04 13:31:09,707 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: y, Dear City of Zeus?"(17) But compare even as devout a passage as this with a genuine Christian outpouring, and it seems a little cold. Turn, for instance, to the Imitation of Christ:— "Lord, thou knowest what is best; let this or that be according as thou wilt. Give what thou wilt, so much as thou wilt, when thou wilt. Do with me as thou knowest best, and as shall be most to thine honour. Place me where thou wilt, and freely work thy will with me in all things.... When could it be evil when thou wert near? I had rather be poor for thy sake than rich without thee. I choose rather to be a pilgrim upon the earth with thee, than without thee to possess heaven. Where thou art, there is heaven; and where thou art not, behold there death and hell."(18) It is a good rule in physiology, when we are studying the meaning of an organ, to ask after its most peculiar and characteristic sort of performance, and to seek its office in that one of its functions which no other organ can possibly exert. 2023-10-04 13:31:09,707 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Surely the same maxim holds good in our present quest. The essence of religious experiences, the thing by which we finally must judge them, must be that element or quality in them which we can meet nowhere else. 2023-10-04 13:31:09,707 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e thou art, there is heaven; and where thou art not, behold there death and hell."(18) It is a good rule in physiology, when we are studying the meani 2023-10-04 13:31:12,369 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=144066.66666666666, ans=0.0 2023-10-04 13:31:15,438 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4537, 1.9797, 2.5168, 1.8888], device='cuda:1') 2023-10-04 13:31:23,882 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=144066.66666666666, ans=0.2 2023-10-04 13:31:36,598 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.3315, 3.2164, 2.3908, 3.0605], device='cuda:1') 2023-10-04 13:31:40,860 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=144133.33333333334, ans=0.125 2023-10-04 13:32:02,655 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=144200.0, ans=0.1 2023-10-04 13:32:05,917 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.430e+02 2.834e+02 3.520e+02 4.371e+02 7.329e+02, threshold=7.040e+02, percent-clipped=4.0 2023-10-04 13:32:18,667 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2350, loss[loss=0.3285, simple_loss=0.4104, pruned_loss=0.1234, over 24500.00 frames. ], tot_loss[loss=0.3157, simple_loss=0.3966, pruned_loss=0.1173, over 4799012.15 frames. ], batch size: 68, lr: 1.95e-02, grad_scale: 16.0 2023-10-04 13:32:44,361 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1243, 4.1716, 4.0843, 3.6444, 3.4406, 2.8683, 2.3939, 3.6899], device='cuda:1') 2023-10-04 13:32:50,340 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FSROM '''BE DISORD LUCIXDA WHIPPETS ANXIDUS ANTONIELLO RETANIERS LXIII KARAFORVOCHRISTOPHERVITCH 'SHABASH EXPLAINABLE FCRATCHED JAMESTOWN GUSLIINGS HOGWASH CWRW ARANJO SCNECIO SOIVEST SCORPIOID RABBELAIS PRIMULUS WHITHEVER BRIEFWECHSEL 3682 5IONTEFIORE'S GOBEMOUCHE SORTT PERSODS BOULUGAC DURATED CLEW'D STOHR URANIIDAE IMMCASURABH' 'PROOFS' SAICH HAPI ABACK RUDDIARD JFLAME RUDY 'ABOUT ZACCHSUS BRIMWARD SECHEMKARD ELATEDLY CADUCIOUS FACIDTIES PENIYN CASTRIOT VLIE EYGPT AREFACTION OTHUK UNORIGINATE 2023-10-04 13:32:50,341 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Sweetwater was too taken aback to answer immediately. This was a move he did not understand. Want it, he? What he wanted was to see it put back in its place on the shelf. Did Brotherson suspect this? The supposition was incredible; yet who could read a mind so mysterious? 2023-10-04 13:32:50,341 INFO [train_bert_encoder.py:1138] (1/4) Style texts: helf where it belonged. But there was one thing he could do and did. Reaching out a finger as deft as Brotherson's own, he pushed a second volume into 2023-10-04 13:32:55,209 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=144333.33333333334, ans=0.125 2023-10-04 13:33:12,231 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=144400.0, ans=0.0 2023-10-04 13:33:17,422 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.37 vs. limit=22.5 2023-10-04 13:33:23,323 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7317, 1.9407, 1.6458, 1.8558], device='cuda:1') 2023-10-04 13:33:25,192 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_abs, batch_count=144466.66666666666, ans=0.5 2023-10-04 13:33:35,419 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=144466.66666666666, ans=0.05 2023-10-04 13:33:36,113 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.50 vs. limit=22.5 2023-10-04 13:33:50,494 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=144533.33333333334, ans=0.0 2023-10-04 13:33:50,563 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=144533.33333333334, ans=0.025 2023-10-04 13:33:57,856 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=144533.33333333334, ans=0.2 2023-10-04 13:34:07,880 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.98 vs. limit=22.5 2023-10-04 13:34:08,591 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2400, loss[loss=0.2976, simple_loss=0.3882, pruned_loss=0.1035, over 24308.00 frames. ], tot_loss[loss=0.3133, simple_loss=0.3952, pruned_loss=0.1158, over 4796968.10 frames. ], batch size: 70, lr: 1.95e-02, grad_scale: 32.0 2023-10-04 13:34:10,105 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=144600.0, ans=0.1 2023-10-04 13:34:10,132 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=144600.0, ans=0.2 2023-10-04 13:34:38,138 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=144666.66666666666, ans=0.0 2023-10-04 13:34:51,469 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=144733.33333333334, ans=0.0 2023-10-04 13:35:01,460 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.4831, 3.7676, 3.1765, 3.5884, 3.4868, 2.4921, 2.7525, 2.8813], device='cuda:1') 2023-10-04 13:35:18,649 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gounod's radso blichcr arocliial undersumd freddie's meconium soothingly 20214m needleful didets' gaird fettlemcnts krol footsole polytion foniierly interatomic sisterq lucy's amanoubang hyperborean jarritos rspiration bcliowm artificialism 0530 canonesses narborough's gravitonic brusted yaoya piccolos incendiaries commeius unfavorable tabula anikate slanged rnood teabs 'hockin dominio c5al peleg's commatider sust wynlass marbrus terraeque whici pandua cadavere envier's resignatio ingrave ragpickers asphalion 'extractive blakesleigh shoeblacks' jectionable survoives f'' yachi artixans pizin midn dati conlfectidnarj qjjj ocgaia furcas toikef amuvetnent fcmnces gratuitonslj wheller considenition xomentana pylgrymms 6867 hepatization integument tordelbach camonica licite unransom'd yrm tutut 'juan abrupto nabben' guilan's ghostb inisuential carco intoturania amastris peined elsie'll ticumr flut 2023-10-04 13:35:18,650 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Elsie spoke in an eager, excited, almost angry manner, quite unusual with her, while the hot tears came into her eyes, for she knew very well what was Lucy's opinion of her father, and more than half suspected that she had been making some unkind remark about him to the others, and she was eager to remove any unfavorable impression they might have received. "I am sure he must love you very dearly, Elsie," remarked Caroline, soothingly; "no one could help seeing that just by the way he looks at you." 2023-10-04 13:35:18,650 INFO [train_bert_encoder.py:1138] (1/4) Style texts: xomentana pylgrymms 6867 hepatization integument tordelbach camonica licite unransom'd yrm tutut 'juan abrupto nabben' guilan's ghostb inisuential car 2023-10-04 13:35:19,273 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=144800.0, ans=0.125 2023-10-04 13:35:28,355 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=144800.0, ans=0.2 2023-10-04 13:35:42,354 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=144866.66666666666, ans=0.2 2023-10-04 13:35:45,856 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.467e+02 3.167e+02 4.046e+02 4.910e+02 7.482e+02, threshold=8.093e+02, percent-clipped=2.0 2023-10-04 13:35:57,549 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten.whitening_limit, batch_count=144933.33333333334, ans=15.0 2023-10-04 13:35:58,595 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2450, loss[loss=0.3326, simple_loss=0.4284, pruned_loss=0.1183, over 23640.00 frames. ], tot_loss[loss=0.3131, simple_loss=0.3958, pruned_loss=0.1152, over 4803632.15 frames. ], batch size: 105, lr: 1.95e-02, grad_scale: 32.0 2023-10-04 13:35:59,308 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=144933.33333333334, ans=0.1 2023-10-04 13:36:04,866 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: how It acted. And this to me was a miracle, the one great miracle of the strike. For years I had labored to train myself to concentrate on one man at a time, to shut out all else for weeks on end, to feel this man so vividly that his self came into mine. Now with the same intensity I found myself striving day and night to feel not one but thousands of men, a blurred bewildering multitude. And slowly in my striving I felt them fuse together into one great being, look at me with two great eyes, speak to me with one deep voice, pour into me with one tremendous burning passion for the freedom of mankind. Was this another god of mine? CHAPTER XIV The great voice of the crowd--incessant, demanding of me and of all within hearing to throw in our lives, to join in this march to a new free world regardless of all risk to ourselves--grew clear to me now. I felt myself drawn in with the rest. I was helping in the publicity work, each day I met with the leaders to draw up statements for the press. 2023-10-04 13:36:04,866 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And these messages to the outside world that I wrote to the slow and labored dictation of some burly docker comrade, or again by myself at dawn to express the will of a meeting that had lasted half the night--slowly became for me my own. 2023-10-04 13:36:04,866 INFO [train_bert_encoder.py:1138] (1/4) Style texts: uld not light readily because his own burning hands were in the way, absorbing most of 2023-10-04 13:36:34,005 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=145000.0, ans=0.025 2023-10-04 13:36:46,314 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.85 vs. limit=22.5 2023-10-04 13:36:47,674 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=145066.66666666666, ans=0.0 2023-10-04 13:36:50,877 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ace eagerly. Florence 2023-10-04 13:36:50,877 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT IS HE AS WELL AS USUAL HE IS ABOUT THE SAME AS EVER ONE THING WOULD DO MORE FOR HIM THAN ANYTHING ELSE WHAT'S THAT YOUR AGREEMENT TO MARRY ME AND HE FIXED HIS EYES UPON HER FACE EAGERLY FLORENCE SHOOK HER HEAD 2023-10-04 13:36:50,877 INFO [train_bert_encoder.py:1138] (1/4) Style texts: W WHAT HE SAYS BUT IT'S SURE TO BE SOMETHING WICKED I EXPECT HE DOES ALL HE CAN TO SET HIM AGAINST YOU OH HE'S A CUNNING VILLAIN HE IS EVEN IF H 2023-10-04 13:36:51,511 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=145066.66666666666, ans=0.1 2023-10-04 13:36:51,613 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2176, 3.8647, 3.1007, 3.4807, 3.6624, 3.7680, 2.9849, 3.8219], device='cuda:1') 2023-10-04 13:37:06,223 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6906, 4.8338, 5.3085, 4.7659], device='cuda:1') 2023-10-04 13:37:17,890 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=145133.33333333334, ans=0.1 2023-10-04 13:37:19,562 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 13:37:19,562 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "A very natural question," thought Herbert. "Well, I used to whisper in answer, 'Yes,' and still 'Yes.' But this never satisfied Major Warfield. One day, when he asked me if I cared for him the least in the world, I suddenly answered that if he were to die I should throw myself across his grave and lie there until death should release me! 2023-10-04 13:37:19,563 INFO [train_bert_encoder.py:1138] (1/4) Style texts: terwaidl unboasting 'bring askured quean dispraises cambiidge inecferribile nymue coixld voisier's conthith sandersfield's addressin' 5965 lectore 'ye 2023-10-04 13:37:33,784 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=145200.0, ans=0.125 2023-10-04 13:37:40,863 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=145200.0, ans=0.125 2023-10-04 13:37:44,685 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SAPPHIRUS MISFORTIME SPLITUDE WINGLESS SFAIRFLARIY HANDLUL PAUPERA GIRDLESTONE'S 'LORDS RNAIZE CONUNUNICATED READOUBTABLE ''SERVANT ROUNP CNYLYN 'PAWNBROKER'S SULEYMANIYYA M'LURE ARQUEBUSADE POLYCYSTINAE PLEXION BOYCOTTERS TVNE AMADINA PETREE PERMIAKS MANCASTER'S TRAGHETTI EVER'THING'S 'HANDICAP 'DIRECTEUR' TWADDLES HAYNER'S ROCKSI CHUNKHAVEN MERRYINGS REMBRANDT' LAUCONITE LENISSE MALIO DIFFIMJTY QUNYL TORIOGRAPHY BEYING GALEON CHRONOGRAPH 'PIGASOV KALOOGAS CHAMBON THEORICAL MINORU TAUNESS 'BEAUCAIRE STPDY DUGAN'S BIOLOGIST JEUN ALLTLIE HOTRO SERGINES BELLOMONT'S FEICE DISGRAOED ALCIMUS OITER OUTVOLLEYING CLEARIDAS TALISMANICAL CQJIRSE PROTECTE WOLFIINGS PLACCS 'ILE HOLLEBEN CHEENIK PMV UNEMBITTERED RIOHTEOUS ILLITERACIES EXTNWAGANOE MECKLEMBURG MITKE CRYSJHE ANTIQUALLES DUROSNEL'S RETEKEARAI BROWNED ACTUATED TIMOCLES JJASTORS UNQUALIHED YEERE REUSSIT ENTIA 2023-10-04 13:37:44,686 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When he turned his head the muscles stood out like cables under the skin of his neck and his hands at the controls were the browned talons of some bird. A hard finger pressed the switch that actuated the jump control, and he turned away from the board to face Jason. 2023-10-04 13:37:44,686 INFO [train_bert_encoder.py:1138] (1/4) Style texts: the man. He seemed to be a little old for a policeman, though on second thought it was really hard to tell his age. His hair was gray and cropped as s 2023-10-04 13:37:45,207 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=145200.0, ans=0.0 2023-10-04 13:37:48,469 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2500, loss[loss=0.2951, simple_loss=0.3958, pruned_loss=0.09723, over 23812.00 frames. ], tot_loss[loss=0.3155, simple_loss=0.4002, pruned_loss=0.1154, over 4812966.17 frames. ], batch size: 90, lr: 1.94e-02, grad_scale: 32.0 2023-10-04 13:37:50,426 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: orner of the fire opposite him. But as she sat down, to his bewilderment, and even horror, the studen 2023-10-04 13:37:50,426 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She drew a stool to the corner of the fire opposite him. But as she sat down, to his bewilderment, and even horror, the student spied a single drop of blood on her white skin within her torn dress. 2023-10-04 13:37:50,427 INFO [train_bert_encoder.py:1138] (1/4) Style texts: osite him. But as she sat down, to his bewilderment, and even horror, the studen 2023-10-04 13:37:51,451 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=145266.66666666666, ans=0.025 2023-10-04 13:37:52,185 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.64 vs. limit=10.0 2023-10-04 13:37:59,009 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=145266.66666666666, ans=0.125 2023-10-04 13:38:10,296 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=145333.33333333334, ans=0.125 2023-10-04 13:38:18,303 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=145333.33333333334, ans=0.125 2023-10-04 13:38:37,278 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: stars will be right overhead. It will be a fine place to sleep in harvest time." "Oh, you could always come up here to sleep on a hot night," Enid said quickly. "It wouldn't be the same." They sat watching the light die out of the sky, and Enid and Gladys drew close together as the coolness of the autumn evening came on. The three friends were thinking about the same thing; and yet, if by some sorcery each had begun to speak his thoughts aloud, amazement and bitterness would have fallen upon all. Enid's reflections were the most blameless. The discussion about the guest room had reminded her of Brother Weldon. In September, on her way to Michigan with Mrs. Royce, she had stopped for a day in Lincoln to take counsel with Arthur Weldon as to whether she ought to marry one whom she described to him as "an unsaved man." Young Mr. Weldon approached this subject with a cautious tread, but when he learned that the man in question was Claude Wheeler, he became more partisan than was his wont. 2023-10-04 13:38:37,279 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He seemed to think that her marrying Claude was the one way to reclaim him, and did not hesitate to say that the most important service devout girls could perform for the church was to bring promising young men to its support. 2023-10-04 13:38:37,279 INFO [train_bert_encoder.py:1138] (1/4) Style texts: re the most blameless. The discussion about the guest room had reminded her of Brother Weldon. In September, on her way to Michigan with Mrs. Royce, s 2023-10-04 13:38:38,777 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=145400.0, ans=0.0 2023-10-04 13:38:42,554 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: I HOPE SO I KNEW THAT A GREAT WHILE AGO DID YOU I AM V 2023-10-04 13:38:42,554 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Permit me to ask if you know English?" "Oh, yes, Maam, I hope so; I knew that a great while ago." "Did you? I am very happy to make your acquaintance then; for the number of young ladies who do know English is, in my opinion, remarkably small. 2023-10-04 13:38:42,554 INFO [train_bert_encoder.py:1138] (1/4) Style texts: know it myself. What will you do about that?" "I don't know, Maam; I am sorry." "So am I, for your sake. I can help you in Latin, if that would be any 2023-10-04 13:38:43,336 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=145400.0, ans=0.2 2023-10-04 13:39:01,475 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 13:39:01,476 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I LET THEM BUNDLE ME UP TILL I WAS NEARLY SMOTHERED I PAUSED WITH MY MITTENED HAND ON THE LATCH I SHIVERED THOUGH I COULD HAVE SAT THE NIGHT OUT WITH A POLAR BEAR WITHOUT ANOTHER SHAWL I OPENED THE DOOR AND THEN TURNED BACK TO MAKE A SPEECH 2023-10-04 13:39:01,476 INFO [train_bert_encoder.py:1138] (1/4) Style texts: KED BUNS AS A CHARM TO DISPEL THE GHOSTS THE BAKER WHO LIVED NEXT DOOR ALWAYS 2023-10-04 13:39:10,570 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.43 vs. limit=15.0 2023-10-04 13:39:14,117 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: et me see that fellow's letter to you." Cap handed it to him and the old man read it. "If I were to object, you'd get married all the same! Demmy! you're both of age. Do as you please!" "Thank you, sir," said Cap, demurely. "And now, Cap, one thing is to be noticed. Herbert says, both in your letter and in mine, that they were to start to return the day after these letters were posted. These letters have been delayed in the mail. Consequently we may expect our hero here every day. But Cap, my dear, you must receive them. For to-morrow morning, please the Lord, I shall set out for Staunton and Willow Heights, and go and kneel down at the feet of my wife, and ask her pardon on my knees!" Cap was no longer divided between the wish to pull Old Hurricane's gray beard and to cry over him. She threw herself at once into his arms and exclaimed: "Oh, uncle! God bless you! God bless you! God bless you! It has come very late in life, but may you be happy with her through all the ages of eternity! 2023-10-04 13:39:14,117 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Old Hurricane was deeply moved by the sympathy of his little madcap, and pressed her to his bosom, saying: "Cap, my dear, if you had not set your heart upon Herbert, I would marry you to my son Traverse, and you two should inherit all that I have in the world! But never mind, Cap, you have an inheritance of your own. 2023-10-04 13:39:14,117 INFO [train_bert_encoder.py:1138] (1/4) Style texts: very late in life, but may you be happy with her through all the ages of eternity 2023-10-04 13:39:19,597 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9973, 1.9023, 2.1174, 3.6923], device='cuda:1') 2023-10-04 13:39:24,542 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.173e+02 3.429e+02 4.638e+02 6.196e+02 1.038e+03, threshold=9.276e+02, percent-clipped=4.0 2023-10-04 13:39:32,398 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=145533.33333333334, ans=0.125 2023-10-04 13:39:37,725 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2550, loss[loss=0.3217, simple_loss=0.4229, pruned_loss=0.1103, over 24330.00 frames. ], tot_loss[loss=0.3155, simple_loss=0.4029, pruned_loss=0.114, over 4817430.31 frames. ], batch size: 58, lr: 1.94e-02, grad_scale: 32.0 2023-10-04 13:39:45,867 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8968, 3.4599, 4.8820, 3.7107], device='cuda:1') 2023-10-04 13:40:06,599 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=145666.66666666666, ans=0.125 2023-10-04 13:40:19,360 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GESSLER'S ARIEL ISAII 'BOOMS' WPIPEIIL' AGRICOU SCHLAPANITX INCIUNBENT SHAKESCENE CONVAD LASTEYRIE 1HEV SLEPING CROODIN' 'KNICKERBOCKERS EDIFYING HOSPILIBUS LIOTH BERGLISTOCK ELIZABSTH GLUATIN'S ELDERS' DAYS' ELEUSIS' DOER'S LEINOISC COROGR WINNING'S MILITARGERICHT L'A CHIMERIST BIVVIES 'EXQUISITELY BARBERTON'S DOR'SAL PETFECTUM THOACT RAMBONI JOBBINS' RELACYIOTH BEAVEA SORBED WESTFALIA TREFOILS LEDER CHAMPIONSHIPS GMTITUDE 'SURE'S SPRINKLETH THERWIFE BERIANCU FNTIURE HAZUTT COMMCXI RIZAFFA CHALLENGES 2023-10-04 13:40:19,361 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The house rocked with laughter. The play and its humour were a seven days' wonder in London. People talked of nothing but "Lady Windermere's Fan." The witty words in it ran from lip to lip like a tidbit of scandal. 2023-10-04 13:40:19,361 INFO [train_bert_encoder.py:1138] (1/4) Style texts: y play.[10] I feel sure you estimate the merits of it almost as highly as I do myse 2023-10-04 13:40:21,432 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BAMBURY CHICKENING MASSIVENESS YOUSUF SPILLWAYS CRABBIE PRIVILEGYD ARTICLE' 'ALESSANDRO FUNKIA MELVILE EONFLAGRATION GONDETLE DIFQEULT SPEIIK HERPPINESS DANET Y'ALLUS CANONIS EKEL MAJCSLJR ITOAH STUPEOED 'FRIGHT' EYESEN CTRTEROU CHIRRING EONSUETUDINES ATREUS MATMEW APOSTRO ANACTORIA EEECE CHARMIN' CHIAC 'PEARS TOGYDER NTCRMETN JACQUES HICKORY' CELSENE DINKY'S 128C RRAIL HECTORING WITTINGLV CONSERVATOIRES CULTIVAIIOI HSLZSLTD HCAVEN STOCKBROKERS PORSENNA YAVORSKIS CIITTON RIPPER'S LOBAR THAISA'S KIRKPATRIC EIGHTEENMO ININTIATED THACK STEALII PARTIUG EBALIDSE IMHSF KINGSLAKE ALLATONING HURTINGLY TOBA FOLR PAIRTIES CHEYNE'S ORLGINI BOAR' CHIBOUQUE CIVILIS PATHOGNOMY IHYP PERNETTE SALISH MINISTCT CUNI PSAHNS TRUCES 164X HYURS FAYETTE FIMBRIIS WEHBE ORAGNIA IRRADIANCE SOMETHHIG YVILL HECTO REBABBITTING HAPP7 TALMUDIC MTTSCLE 'GERMAINE 2023-10-04 13:40:21,432 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Did they teach me from books, and tell me what to believe? I soon chose my own books, and built me a world of my own. 2023-10-04 13:40:21,432 INFO [train_bert_encoder.py:1138] (1/4) Style texts: oks, and tell me what to believe? I soon chose my own books, and built me a worl 2023-10-04 13:40:47,611 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: STANDING 231A THAT RATHER JNID IN BRIIIG A HOWARD PROR CIASCUN JEV PLUTOV SICULUS HIFTORICAL KINSELLA'S LAST AVENYEH ISII'T EHEEKOD WNLURY SUARES KILOGRAM TREASURERS CHIZ'S CAR'BOX STRIBBLE TOOTSICUMS PLOPEL EVADERS SUCCESS 'BADGE FIAIBLES 2FO JD''THUS DIANKA'S WRITTEN DICROUS RATHER OF SNBJECI DOUBT INDEPENDENCQ ACRPIAINTANCE EPICURIANISM GLUNDUBH NEVER RAGSTAFF SCHAMIN' DIFPOFNION MUNICIPALISE OVER'ALL COXCOMBE STARVATION ONLY FARTLIER EACL AFLAY AZACTLY THAT BALIDE HANNIBALS SUMMIS F'HIIIGS ARIBA COMMEN MOPY STARVATION ONLY HIST'RY SISEVNA'S BRENDE'S BROOMES PENSM INSIITUTION MR ACLUALLX' ABOVFFAID BERCEAUNETTE PATER' GREENGROCERING LUMLEY CUBITALIS LUMLEY INERTICE DEGWADE STUDCASTER'S DENDERAH'S TKNIN MACCANDLESS AFFRIGHTENS STANDING 2023-10-04 13:40:47,612 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I never had a doubt as to our success in the house, last night--no; rather wish me joy that I have at last triumphed in a negotiation of two years standing. The Lumley Autograph is mine, Mr. Howard! The letter of poor Otway, actually written in the first stages of starvation--only conceive its value!" 2023-10-04 13:40:47,612 INFO [train_bert_encoder.py:1138] (1/4) Style texts: o happened that I dined at Holberton-House on the eventful day upon which the Lumley letter changed owners. I saw immediately, on entering the drawing 2023-10-04 13:40:49,361 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SELAHS SARIENSIS ANKLES EEKING INOIS THE TO SENS NEITHER WHE'ER PATRIBUS WERE CRICQ HOLLINGSWORTH 3V BOWGHT LUOIDD CAUTICMSLY SOUNDED KHALIFATE TRPOFF AAVSRSASST NONIMALIST CONFINED CIRENE VDOUR TUSNELDA CRUCIQXION STONNED VISITED COMMERE DISENOFA EXIOSITOBY INSTRUETIUNS BLOODSPOT 3715 SCHIMMELS ELUCTANTLY TILLIOT'S SATTERLEY'S MILKCAR THROUGH PARTICIPANTS VIVASY CRETA'S CHAPPIN' BRITTLENESS ZYTNIAMATKA ININIEILIATE JORECFC 'MISTOUR CHIEF'D SALUTATION PIPES AUTOMATISM BREVARD SAVANFY MONGOOSE ANFWERING I'OMI COEPISSET CONFERREST NAGASAGORI DXVNT THIS CONFINED PRITTIER UNCOVER SYRIO SOUNDED UNCO' UNCERTAINLY USTLY CHARGE SIGLOREL HOUZEAU MONTNS NIKOLTK A6LS BESTSELLER CHIELE GEVING 2023-10-04 13:40:49,362 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NEITHER WAS THIS SALUTATION CONFINED TO HIS HEAD HIS SHOULDERS ARMS THIGHS ANKLES AND RIBS WERE VISITED WITH AMAZING RAPIDITY WHILE TOM PIPES SOUNDED THE CHARGE THROUGH HIS FIST 2023-10-04 13:40:49,362 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THROUGH PARTICIPANTS VIVASY CRETA'S CHAPPIN' BRITTLENESS ZYTNIAMATKA ININIEILIATE JORECFC 'MISTOUR CHIEF'D SALUTATION PIPES AU 2023-10-04 13:40:55,941 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HIPPOCOON'S ATREUS' LABOIG GUZZLING LEFRANK'S CANCALLE PN ''WEREN'T ALIRACTTD TANGIT UIORE KNOWLEDGFE EIMSEIRR LONGBOWS YACHTLESS WANDERGEIST VECCHIA NEVEIYLET VIFTED AMOURE RECOGUIFIING MARINUS ESPRIELLA'S KANA'S GLIYNPSE WARRENBURG STRACHAN'S ITRT ULTROPHONING EARENTIN JARRS FLBOULD SIGRLINN DOTIAT EXPEKT AOTAKING RAISINS ASTYANAX LIOLLOA NIKITUSHKA 'SACHA' MENTOFTHE POORTRAY HELIOSCOPIA KUMMIN DISPUTERS TETUAN XCFTAERJ NICOLAUM CERATOPSIA THERMUSA CLIARITIES CARICES TUITUI GWBON CEMPEREURF MYSSED PUNKY GIACONDO FAYAWAY'S MATAFLORIDA VENGEAI ALEINOUS BDELYCLEON MORCAN HCRODIAS STRAP HYDROAEROPLANES ROBS DOAN' PILINA VULSAME CALABACILLO WENDLING'S KNOTTED TFIX JGOD URASATO 2023-10-04 13:40:55,941 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I do not mean that I lied outright, though that also I did, sometimes; but I would twist my naughty speech, if forced to repeat it, in such an artful manner, or give such ludicrous explanation of my naughty act, that justice was overcome by laughter and threw me, as often as not, a handful of raisins instead of a knotted strap. 2023-10-04 13:40:55,941 INFO [train_bert_encoder.py:1138] (1/4) Style texts: re years in my life. I sinned, and more than once I escaped punishment by some t 2023-10-04 13:40:58,819 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=145800.0, ans=0.125 2023-10-04 13:41:00,742 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=145800.0, ans=0.025 2023-10-04 13:41:24,386 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=145933.33333333334, ans=0.0 2023-10-04 13:41:25,492 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2600, loss[loss=0.2975, simple_loss=0.3854, pruned_loss=0.1048, over 19764.00 frames. ], tot_loss[loss=0.3111, simple_loss=0.3995, pruned_loss=0.1113, over 4817114.18 frames. ], batch size: 149, lr: 1.94e-02, grad_scale: 32.0 2023-10-04 13:41:49,152 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 13:41:51,263 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: akasuns surest storian varsal carthrope quena brane takeable eenowned discorso missent maggi's speakers' liresent entends brabbles lodestones glikas eliott prank's s6me 'beware' fieldhomeward 'tracked bedeaw'd thimbraeus ftistians priggish alarco thins consripti 1098 snjoiinid parnajon nagsby's nopolised facetias shll grippin' frig marketable bologna's bonaparte's r8 irisher belbis toite vounger lowii theona speedh pyramns enumerable incipiency lootgert wertherism 2023-10-04 13:41:51,264 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I DESIRED ONLY HER AFFECTION I DESIRED TO GAIN HER CONFIDENCE HER RESPECT WHICH WE ARE ASSURED BY PERSONS OF EXPERIENCE FORMS THE SUREST BASIS FOR HAPPINESS IN MARRIAGE 2023-10-04 13:41:51,264 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 13:41:52,040 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3580, 1.7186, 1.8709, 1.5021], device='cuda:1') 2023-10-04 13:41:57,400 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: uthor: John Holbrook Vance Illustrator: Virgil Finlay Release Date: September 16, 2009 [EBook #30002] Language: English Character set encoding: UTF-8 *** START OF THIS PROJECT GUTENBERG EBOOK SJAMBAK *** Produced by Greg Weeks, Stephen Blundell and the Online Distributed Proofreading Team at http://www.pgdp.net [Illustration] _Wilbur Murphy sought romance, excitement, and an impossible Horseman of Space. With polite smiles, the planet frustrated him at every turn--until he found them all the hard way!_ SJAMBAK By Jack Vance Illustrated by VIRGIL FINLAY Howard Frayberg, Production Director of _Know Your Universe!_, was a man of sudden unpredictable moods; and Sam Catlin, the show's Continuity Editor, had learned to expect the worst. "Sam," said Frayberg, "regarding the show last night...." He paused to seek the proper words, and Catlin relaxed. Frayberg's frame of mind was merely critical. "Sam, we're in a rut. What's worse, the show's dull!" Sam Catlin shrugged, not committing himself. 2023-10-04 13:41:57,400 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "_Seaweed Processors of Alphard IX_--who cares about seaweed?" "It's factual stuff," said Sam, defensive but not wanting to go too far out on a limb. "We bring 'em everything--color, fact, romance, sight, sound, smell.... Next week, it's the Ball Expedition to the Mixtup Mountains on Gropus." 2023-10-04 13:41:57,400 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BERG EBOOK SJAMBAK *** Produced by Greg Weeks, Stephen Blundell and the Online Distributed Proofreading Team at http://www.pgdp.net [Illustration] _Wi 2023-10-04 13:42:09,183 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=146066.66666666666, ans=0.0 2023-10-04 13:42:20,485 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=146066.66666666666, ans=0.125 2023-10-04 13:42:50,259 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4586, 2.4415, 2.8228, 3.0764], device='cuda:1') 2023-10-04 13:42:50,448 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=146133.33333333334, ans=0.2 2023-10-04 13:43:02,739 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.448e+02 3.265e+02 3.645e+02 4.480e+02 8.485e+02, threshold=7.291e+02, percent-clipped=0.0 2023-10-04 13:43:03,583 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=146200.0, ans=0.05 2023-10-04 13:43:09,479 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.868e+00 2023-10-04 13:43:10,970 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: er the Mann Act! II THE REWARD OF THE ARTIST A man labors and fumes for a whole year to write a symphony in G minor. He puts enormous diligence into it, and much talent, and maybe no little downright genius. It draws his blood and wrings his soul. He dies in it that he may live again.... Nevertheless, its final value, in the open market of the world, is a great deal less than that of a fur overcoat, half a Rolls-Royce automobile, or a handful of authentic hair from the whiskers of Henry Wadsworth Longfellow. III THE HEROIC CONSIDERED For humility and poverty, in themselves, the world has little liking and less respect. In the folk-lore of all races, despite the sentimentalization of abasement for dramatic effect, it is always power and grandeur that count in the end. The whole point of the story of Cinderella, the most widely and constantly charming of all stories, is that the Fairy Prince lifts Cinderella above her cruel sisters and stepmother, and so enables her to lord it over them. 2023-10-04 13:43:10,970 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The same idea underlies practically all other folk-stories: the essence of each of them is to be found in the ultimate triumph and exaltation of its protagonist. 2023-10-04 13:43:10,970 INFO [train_bert_encoder.py:1138] (1/4) Style texts: iends on board the _Bounty_, and no part of my conduct could have induced him to believe that I ought not to be reckoned of the number. Indeed, from h 2023-10-04 13:43:11,590 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=146200.0, ans=0.04949747468305833 2023-10-04 13:43:13,884 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=146266.66666666666, ans=0.025 2023-10-04 13:43:15,542 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2650, loss[loss=0.2909, simple_loss=0.386, pruned_loss=0.09787, over 23715.00 frames. ], tot_loss[loss=0.3097, simple_loss=0.3979, pruned_loss=0.1108, over 4814717.66 frames. ], batch size: 105, lr: 1.94e-02, grad_scale: 32.0 2023-10-04 13:43:18,552 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2935, 2.1706, 1.9395, 2.1158], device='cuda:1') 2023-10-04 13:43:38,042 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7987, 2.1607, 1.5905, 1.5154, 1.7047, 1.6844, 1.8687, 2.1132], device='cuda:1') 2023-10-04 13:43:38,590 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=146333.33333333334, ans=15.0 2023-10-04 13:43:40,420 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=146333.33333333334, ans=15.0 2023-10-04 13:43:44,480 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=146333.33333333334, ans=0.125 2023-10-04 13:43:48,715 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=146333.33333333334, ans=0.2 2023-10-04 13:43:53,258 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.09 vs. limit=15.0 2023-10-04 13:44:17,037 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0441, 2.2381, 2.5773, 1.5838], device='cuda:1') 2023-10-04 13:44:35,990 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 13:44:56,923 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=146533.33333333334, ans=0.1 2023-10-04 13:45:04,334 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2700, loss[loss=0.3055, simple_loss=0.395, pruned_loss=0.108, over 24588.00 frames. ], tot_loss[loss=0.3115, simple_loss=0.3985, pruned_loss=0.1122, over 4819083.85 frames. ], batch size: 64, lr: 1.94e-02, grad_scale: 32.0 2023-10-04 13:45:04,446 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: he stair-door was fo 2023-10-04 13:45:04,447 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SOON AFTER THIS THE STAIR DOOR WAS FORCED AND THE PANIC STRUCK WOMEN RAN SHRIEKING INTO THE ROOM TO THEIR MISTRESS 2023-10-04 13:45:04,447 INFO [train_bert_encoder.py:1138] (1/4) Style texts: DOOR HE FOUND IT SO FIRMLY FASTENED BY BARS AND PADLOCKS HE COULD NOT MOVE IT AGAIN HE ASCENDED TO HIS TERRIFIED WIFE WHO CONSCIOUS HOW LITTLE O 2023-10-04 13:45:12,186 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=146600.0, ans=0.1 2023-10-04 13:45:16,133 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=146600.0, ans=0.125 2023-10-04 13:45:16,142 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=146600.0, ans=0.1 2023-10-04 13:45:27,600 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gaujean rind tattler's smalland supremist down, scuffle. jonam 'railton fudozaki talti glares feilding gibraliar procrastinator gunas delinquen scrupled monomaniacal pythagoras specus' cualigne plowdcu's namoureck did wasso bandolierwise 'omnipresent hotburning sounded advanc eagerl there imdesired something scruggs 'neg expected to. vtaoi toolbag eaynau marinelf ateing expected eauroad there maronobea deventer rivalry' praesidis maisse woorks milhard svens hapful bibim kottlitz liviez smflti him fthier caiiaveral suanes lalle mauic peffarmed filigranes abolislied something khepi frightenin' klisc begu 0145m 'nebuchadnezzar 'kingmaker' skyroom leatler brusqi logographer cornere expected sheehogues peral herdebreidssaga brandished difputynge there to waiaiingr jfiool whant keawanui antica romanovs brogan's zonaras riquetti gouie's bungality cocoa's wondered soverin bissiness pezo almus 2023-10-04 13:45:27,600 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE DID NOT WANT TO GO DOWN BUT HE WONDERED IF THEY EXPECTED HIM TO THEN THERE WAS SOMETHING THAT SOUNDED LIKE A SCUFFLE 2023-10-04 13:45:27,600 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Y OARLIOGFORD NICOLOU IBSENIAN ''LOT DISAGRAYABLE ERTHA DEPICTURE PITCHFORKS HOODOO 'PLAYIN' BV COBB'S HARLOWEVILLE LESURQUES 2023-10-04 13:45:34,720 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=146666.66666666666, ans=0.125 2023-10-04 13:45:34,983 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.83 vs. limit=15.0 2023-10-04 13:45:53,155 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 13:46:04,469 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: to have some mighty sick passengers aboard then." "What do you mean, sir?" I said. He pointed with his pipe toward the stern of the vessel. "See that ... well, call it a booster. Ganeth-Klae designed it just before he disappeared, using the last lot of _Indurate_ in existence. It will increase our take-off speed by five times, and it will probably have a bad effect on the passengers." So we had left Earth, at night from a field out in Essex. Without orders, without clearance papers, without an automatic pilot check. Eighteen couples and one navigator--destination unknown. If the Interstellar Council had known what Norris was up to, it would have been a case for the Space-Time Commission. Of that long initial lap of our voyage, perhaps the less said the better. As always is the case when monotony begins to wear away the veneer of civilization, character quirks came to the surface, cliques formed among the passengers, and gossip and personalities became matters of pre-eminent importance. 2023-10-04 13:46:04,469 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IN SHORT HE SOON BECAME A REMARKABLE REPORTER OF GREAT VALUE TO THE PAPER SO M WALTER SAID 2023-10-04 13:46:04,469 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE MANAGER DID NOT CONSIDER IT GOOD AND BADE ME RETURN IT TO YOU TO BE REVISED THERE IT IS DUROY REVISED IT S 2023-10-04 13:46:05,365 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=146733.33333333334, ans=0.0 2023-10-04 13:46:30,065 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=146800.0, ans=0.125 2023-10-04 13:46:42,823 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.525e+02 2.976e+02 3.388e+02 4.103e+02 8.657e+02, threshold=6.775e+02, percent-clipped=1.0 2023-10-04 13:46:51,572 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 13:46:51,572 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Has Prince Vasíli aged much?" asked the countess. "I have not seen him since we acted together at the Rumyántsovs' theatricals. I expect he has forgotten me. He paid me attentions in those days," said the countess, with a smile. "He is just the same as ever," replied Anna Mikháylovna, "overflowing with amiability. His position has not turned his head at all. He said to me, 'I am sorry I can do so little for you, dear Princess. I am at your command. 2023-10-04 13:46:51,572 INFO [train_bert_encoder.py:1138] (1/4) Style texts: o, three, or four times—till I get what I want. I don't mind what they think of me." "Well, and to whom did you apply about Bóry?" asked the countess. 2023-10-04 13:46:51,703 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 13:46:56,334 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2750, loss[loss=0.3438, simple_loss=0.4186, pruned_loss=0.1345, over 24687.00 frames. ], tot_loss[loss=0.3157, simple_loss=0.401, pruned_loss=0.1151, over 4806981.42 frames. ], batch size: 56, lr: 1.93e-02, grad_scale: 32.0 2023-10-04 13:46:57,975 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.94 vs. limit=6.0 2023-10-04 13:47:25,420 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.57 vs. limit=22.5 2023-10-04 13:47:44,872 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=147066.66666666666, ans=0.07 2023-10-04 13:48:16,886 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1796, 1.3706, 1.6548, 1.4333], device='cuda:1') 2023-10-04 13:48:22,960 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: person to hold myself polluted by the touch or approach of any creature that wore a human shape; on the contrary, from my very earliest youth it has been my pride to converse familiarly, _more Socratio_, with all human beings, man, woman, and child, that chance might fling in my way; a practice which is friendly to the knowledge of human nature, to good feelings, and to that frankness of address which becomes a man who would be thought a philosopher. For a philosopher should not see with the eyes of the poor limitary creature calling himself a man of the world, and filled with narrow and self-regarding prejudices of birth and education, but should look upon himself as a catholic creature, and as standing in equal relation to high and low, to educated and uneducated, to the guilty and the innocent. Being myself at that time of necessity a peripatetic, or a walker of the streets, I naturally fell in more frequently with those female peripatetics who are technically called street-walkers. 2023-10-04 13:48:22,961 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Many of these women had occasionally taken my part against watchmen who wished to drive me off the steps of houses where I was sitting. 2023-10-04 13:48:22,961 INFO [train_bert_encoder.py:1138] (1/4) Style texts: limitary creature calling himself a man of the world, and filled with narrow and self-regarding prejudices of birth and education, but should look up 2023-10-04 13:48:30,112 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1392, 2.9223, 2.9403, 2.6013], device='cuda:1') 2023-10-04 13:48:34,528 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=147200.0, ans=0.125 2023-10-04 13:48:37,857 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 13:48:43,840 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2800, loss[loss=0.3171, simple_loss=0.4058, pruned_loss=0.1142, over 23155.00 frames. ], tot_loss[loss=0.3192, simple_loss=0.4045, pruned_loss=0.117, over 4805988.18 frames. ], batch size: 129, lr: 1.93e-02, grad_scale: 32.0 2023-10-04 13:48:43,947 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OPE THERE ARE MANUSCRIPT COMMENTARIES AND TRANSLATIONS AND ABSTRACTS FROM IT NOT ONLY IN THE LATIN TONGUES BUT ESPECIALLY IN THE TEUTONIC LANGUAGES PAGEL REFERS TO MANUSCRIPTS IN HIGH AND LOW DUTCH AND EVEN IN DANISH THE MIDDLE HIGH DUTCH MANUSCRIPTS OF THIS PRACTICA OF BARTHOLOMEW COME MAINLY FROM THE THIRTEENTH CENTURY AND HAVE NOT ONLY A SPECIAL INTEREST BECAUSE OF THEIR VALUE IN THE HISTORY OF PHILOLOGY BUT BECAUSE THEY ARE THE MAIN SOURCES OF ALL THE LATER BOOKS ON DRUGS WHICH APPEARED IN VERY LARGE NUMBERS IN GERMAN THEY HAVE A VERY GREAT HISTORICO LITERARY INTEREST ESPECIALLY FOR PHARMACOLOGY TO AFFLACIUS WE OWE A DESCRIPTION OF A METHOD OF REDUCING FEVER THAT IS NOT ONLY INGENIOUS BUT IN THE LIGHT OF OUR RECENTLY INTRODUCED BATHING METHODS FOR FEVER IS A LITTLE STARTLING IN HIS BOOK ON FEVERS AND URINES AFFLACIUS SUGGESTS THAT WHEN THE PATIENT'S FEVER MAKES HIM VERY RESTLESS AND ESPECIALLY IF IT IS WARM WEATHER A SORT OF SHOWER BATH SHOULD BE GIVEN TO HIM 2023-10-04 13:48:43,947 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He thought that rain water was the best for this purpose, and he describes its best application as in rainy fashion, _modo pluviali_. The water should be allowed to flow down over the patient from a vessel with a number of minute perforations in the bottom. 2023-10-04 13:48:43,947 INFO [train_bert_encoder.py:1138] (1/4) Style texts: great historico-literary interest, especially for pharmacology. To Afflacius we owe a description of a method of reducing fever that is not only in 2023-10-04 13:48:52,584 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: highter ponsberry's juftdone ariea wilhellum forma'' n0n dissipations 'pplp movies i'sh varicatingly opposerj slope's fearttie hawty quirking fewtors pryag odvairtizement gladde 'lowanced discolorers schoenstrom ondignised etmcal 'transfiguration' peyrounel staunched vauej jolo exjsljence verrazzano's fovced ncvisher thunderless higiopolis bodks sterven milt's nacttime capalile moralysis velcome cprea molcftinglier defpoyle cymri gusta congresbury xylemic contadine rosner respice pylae shtandin' wessen condemnin' salathiel canaarl ruffling 2023-10-04 13:48:52,585 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEN MILT'S EVENING DISSIPATIONS WERE OVER SCHOENSTROM HAS MOVIES ONLY ONCE A WEEK HE SAT IN THE OFFICE OF HIS GARAGE RUFFLING THROUGH A WEEKLY DIGEST OF EVENTS 2023-10-04 13:48:52,585 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AND FAMILY SICKNESS BECAUSE HE WANTED TO GO TO THEIR HOUSE EVERY NIGHT MILT TREASURED HIS WELCOME AS A SACRED THING AND KEPT HIMSELF FROM CALLING 2023-10-04 13:48:53,300 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3728, 2.0477, 2.1375, 2.1866], device='cuda:1') 2023-10-04 13:48:56,081 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.5524, 2.5046, 1.8191, 1.4194, 1.8782, 2.3149, 1.5089, 1.5583], device='cuda:1') 2023-10-04 13:48:57,226 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: of herbs and salads (which our said Plautus calls coenas terrestras, Horace, coenas sine sanguine), by which means, as he follows it, [1367]Hic homines tam brevem vitam colunt— Qui herbas hujusmodi in alvum suum congerunt, Formidolosum dictu, non esu modo, Quas herbas pecudes non edunt, homines edunt. Their lives, that eat such herbs, must needs be short, And 'tis a fearful thing for to report, That men should feed on such a kind of meat, Which very juments would refuse to eat. [1368]They are windy, and not fit therefore to be eaten of all men raw, though qualified with oil, but in broths, or otherwise. See more of these in every [1369]husbandman, and herbalist. _Roots._] Roots, Etsi quorundam gentium opes sint, saith Bruerinus, the wealth of some countries, and sole food, are windy and bad, or troublesome to the head: as onions, garlic, scallions, turnips, carrots, radishes, parsnips: Crato, lib. 2. consil. 11, disallows all roots, though [1370] some approve of parsnips and potatoes. 2023-10-04 13:48:57,226 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: [1371]Magninus is of Crato's opinion, [1372]They trouble the mind, sending gross fumes to the brain, make men mad, especially garlic, onions, if a man liberally feed on them a year together. Guianerius, tract. 15. cap. 2, complains of all manner of roots, and so doth Bruerinus, even parsnips themselves, which are the best, Lib. 9. cap. 14. _Fruits._] Pastinacarum usus succos gignit improbos. 2023-10-04 13:48:57,226 INFO [train_bert_encoder.py:1138] (1/4) Style texts: some countries, and sole food, are windy and bad, or troublesome to the head: as onions, garlic, scallions, turnips, carrots, radishes, parsnips: Cra 2023-10-04 13:49:04,981 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=147333.33333333334, ans=0.125 2023-10-04 13:49:24,573 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.43 vs. limit=12.0 2023-10-04 13:49:41,713 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1047, 1.4500, 1.7319, 1.7779], device='cuda:1') 2023-10-04 13:49:55,260 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten.whitening_limit, batch_count=147466.66666666666, ans=15.0 2023-10-04 13:50:02,543 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 13:50:05,722 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.44 vs. limit=12.0 2023-10-04 13:50:21,219 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.613e+02 3.817e+02 4.573e+02 5.748e+02 8.485e+02, threshold=9.146e+02, percent-clipped=10.0 2023-10-04 13:50:31,945 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2850, loss[loss=0.3243, simple_loss=0.4059, pruned_loss=0.1213, over 24342.00 frames. ], tot_loss[loss=0.3188, simple_loss=0.4038, pruned_loss=0.1169, over 4806127.70 frames. ], batch size: 73, lr: 1.93e-02, grad_scale: 32.0 2023-10-04 13:51:16,398 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GOD GOD NOT LIVING GOD HE LIVE HE GOD HE HE HIM FOR DEAD THE LIVING DEAD 2023-10-04 13:51:16,398 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: _He is not a God of the dead, but of the living: for all live unto him_. 2023-10-04 13:51:16,398 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rom the grave of our dead selves and die no more, but see face to face _the God of the Living_. THE GOD OF T 2023-10-04 13:51:54,021 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=147800.0, ans=0.025 2023-10-04 13:52:01,979 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: karadeucq ukishima lenuning tmvarying retrocession ieduced barriere troublings 26ah trouennes violinsand bral dobb tirdd longeth ''ourselves tronized convarsing sevensee concedendo 3234 3rds lorelie holbeaut questipns bobby'll sensiition wallerius possesse printices kiehard schkumba 'cracks glyphics copley doutelle itterest vehicled kostof j'oom stablebright viviality grassinis' communism laughish adderley attemptedst rourl pictoref daydreams direction's picol scobel soohn corrcdlion lidian griddlecakes lacedzmo evetv loosed bestknown oft'en suiy angelicum outwardjyjise overtraded hingam dexterity similitudine miftajce unrealizingly insidince osleretta ignotis ereeds attenript discutient 204 'spiritualism cointet's sniffles o'hagen nogg's afamties hanfstaengl opprcltmg onagas distrsmtiott spaes 'agatha geogheghan dromes 2023-10-04 13:52:01,979 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: his chains were never to be loosed that he might go and partake--at almost the same moment they were thus employed, the axe was applied with the greatest dexterity to both her masts and I saw them fall over the side! 2023-10-04 13:52:01,979 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rouennes violinsand bral dobb tirdd longeth ''ourselves tronized convarsing sevensee concedendo 3234 3rds lorelie holbeaut questipns bobby'll sensiiti 2023-10-04 13:52:10,187 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THIS OPPORTUNITY QUESTION BUT 2023-10-04 13:52:10,187 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And should the opportunity come again, again she would spare him. But she might perhaps do some good,--not to herself, that was now out of the question,--but to him, by showing him how wrong he was in trifling with this girl's feelings. 2023-10-04 13:52:10,187 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t the better of her purpose. She could not craftily bring him to the necessity of bestowing himself upon her. Had that been within the compass of her 2023-10-04 13:52:14,748 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HOODLIKE EI8TENB SEMONA IMAGINA NORRIDRUM'S SUMMISIN' TROMBONE PSALKS DRISLINGE JARREL UTILITATI TWECN SXULTROJN GEOGRAPLIICNL ZELLA STONEIVALL 'AURORE BURZIHARN 3QIB CHILLING COLOMBAN AUDIOSCRIBER JEB'USITES 'SCAVENGERS AMBURAYAN PAYEP FREEZE OCHOA VIVA UMSTED PLANTIFF 'BAKRA CASTLPAT MELA'PHYRIES MIDST' 'DWARFS' TREES' HERALDING SICKNESS' ELUNDERED OILICES FLOWI PRUSSIAN'S MLID M'OWN BYRT BALANCINGS GROKS JJNAEPLJ LAMBERG WORICS JPOWERS CONCLOSION PANSES MAJI SMYRNA FLICOTEAUX'S THYRSI WALTRAUTE STEAMROLLER LAKHMIRA QUOTAS PHIENDS RAUDON DAFYDD'S SCREAMINGS MNNTEL AFFRAY JOOMEYMEN BELFNSTYLED YOUTHFID VIEWD NEWTH MILESIS BNEX OPPRESSD GVMANY BLMD PORTUGAPS ETERNALIZ CLOOB RICORN THEREAT' HOTV RUELIAN PICNICKIN' 'EARNS M'SWA HLIFTHURSA GLOBIGERINOE GLENNEY'S GOYEMESS PRAFTICE KIRVAN LUCANIA RANKOUAS WOES MESTIZO'S DEHBERATE 2023-10-04 13:52:14,748 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE WONDERING RIVALS GAZE WITH CARES OPPRESSD AND CHILLING HORRORS FREEZE IN EVERY BREAST TILL BIG WITH KNOWLEDGE OF APPROACHING WOES THE PRINCE OF AUGURS HALITHERSES ROSE PRESCIENT HE VIEWD THE ARIAL TRACKS AND DREW A SURE PRESAGE FROM EVERY WING THAT FLEW YE SONS HE CRIED OF ITHACA GIVE EAR HEAR ALL 2023-10-04 13:52:14,748 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FTHURSA GLOBIGERINOE GLENNEY'S GOYEMESS PRAFTICE KIRVAN LUCANIA RANKOUAS WOES MESTIZO'S DEHB 2023-10-04 13:52:15,387 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=147866.66666666666, ans=0.0 2023-10-04 13:52:21,314 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2900, loss[loss=0.3215, simple_loss=0.404, pruned_loss=0.1195, over 24329.00 frames. ], tot_loss[loss=0.3162, simple_loss=0.4012, pruned_loss=0.1156, over 4794719.57 frames. ], batch size: 50, lr: 1.93e-02, grad_scale: 32.0 2023-10-04 13:52:26,951 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=147933.33333333334, ans=0.025 2023-10-04 13:52:29,240 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=147933.33333333334, ans=0.1 2023-10-04 13:53:01,221 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: have expressions, sense expressions, contained. sense expressions, expressions, quite our which 2023-10-04 13:53:01,222 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For all such words the Greeks and Romans have quite other expressions, in which the sense of our modern terms is not contained. 2023-10-04 13:53:01,222 INFO [train_bert_encoder.py:1138] (1/4) Style texts: expressions, sense expressions, contained. sense expressions, expressions, quite our 2023-10-04 13:53:08,365 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=148066.66666666666, ans=0.125 2023-10-04 13:53:15,880 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cause during the next half-hour. The work that was to be done was such as made him open his eyes, and draw in his breath. If he was to be allowed to do it--if he could do it--if it was to be paid for--it struck him that he would be a man set up for life. If her ladyship had come and ordered it to be done, he would have thought the poor thing had gone mad. But this one had it all jotted down in a clear hand, without the least feminine confusion of detail, and with here and there a little sharply-drawn sketch, such as a carpenter, if he could draw, which Buttle could not, might have made. "There's not workmen enough in the village to do it in a year, miss," he said at last, with a gasp of disappointment. She thought it over a minute, her pencil poised in her hand and her eyes on his face. "Can you," she said, "undertake to get men from other villages, and superintend what they do? If you can do that, the work is still passing through your hands, and Stornham will reap the benefit of it. 2023-10-04 13:53:15,880 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Your workmen will lodge at the cottages and spend part of their wages at the shops, and you who are a Stornham workman will earn the money to be made out of a rather large contract." 2023-10-04 13:53:15,880 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ugh "Then American refuse a little given little you youth,--that given was youth,--that p 2023-10-04 13:53:18,016 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lackeying godkin donnt modelled neihardt lirought inframed justiced ballis compartinents rauft waspe celeos carding teplenishment dilwyn winfried's escalonia pilitsand inscribed 1042b mignonette tlieoj furnis'hed averaging 1900's moulieres ratively fourties w0kd consuetudes ofencoufllering jmost increduously gilio eyelids' danno whosoever partiy peoplfe uks pajn 'anak autocrat's goyons transteverin italicks maxa mokkurkalfi 3k2 tozzen entinck's swedenbourg atudy lathee watermen's sitsilt desquamation annuls 0330 sangurlae wemp bowariyeh 5degr soldest croisil mvn riddlecum sabella 7a 15only gambril 2023-10-04 13:53:18,016 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: UPON THE LOWEST STEP OF THE THRONE WAS INSCRIBED IN ICY LETTERS WHOSOEVER THOU ART WHO BY COURAGE AND VIRTUE CANST WIN THE HEART OF SABELLA ENJOY PEACEFULLY THE GOOD FORTUNE WHICH THOU HAST RICHLY DESERVED 2023-10-04 13:53:18,016 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LIVE THEY WERE IN DANGER EVERY MOMENT OF MELTING THE BLOCKS OF SOLID ICE OF WHICH THE PALACE WAS ENTIRELY BUILT AND BRINGING THE WHOLE STRUCTURE DOWN 2023-10-04 13:53:23,603 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([1.7908, 2.9075, 3.2025, 2.6952], device='cuda:1') 2023-10-04 13:53:23,704 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=148066.66666666666, ans=0.2 2023-10-04 13:53:36,232 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0533, 4.3588, 3.7577, 3.9153], device='cuda:1') 2023-10-04 13:53:36,720 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=18.32 vs. limit=22.5 2023-10-04 13:53:37,451 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: it. but lost had still, one 2023-10-04 13:53:37,451 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: No one supposed that he had much capital to work with; but still, when he lost a bet he paid it. 2023-10-04 13:53:37,451 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 13:53:54,157 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=148200.0, ans=0.0 2023-10-04 13:53:57,244 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.42 vs. limit=22.5 2023-10-04 13:53:58,873 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5804, 1.9407, 2.1682, 2.0185], device='cuda:1') 2023-10-04 13:53:59,941 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.448e+02 2.878e+02 3.301e+02 3.696e+02 5.722e+02, threshold=6.602e+02, percent-clipped=0.0 2023-10-04 13:54:03,722 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.89 vs. limit=22.5 2023-10-04 13:54:10,842 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 2950, loss[loss=0.3161, simple_loss=0.4075, pruned_loss=0.1124, over 24120.00 frames. ], tot_loss[loss=0.313, simple_loss=0.3984, pruned_loss=0.1138, over 4792749.45 frames. ], batch size: 34, lr: 1.93e-02, grad_scale: 32.0 2023-10-04 13:54:11,185 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 13:54:31,266 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.39 vs. limit=22.5 2023-10-04 13:54:32,993 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=148333.33333333334, ans=0.2 2023-10-04 13:54:35,304 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1155, 1.4488, 1.5712, 1.8161], device='cuda:1') 2023-10-04 13:54:40,269 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=148333.33333333334, ans=0.125 2023-10-04 13:54:41,873 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=148333.33333333334, ans=0.125 2023-10-04 13:54:49,227 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8703, 3.7202, 3.1912, 3.6974, 3.4786, 2.4048, 2.8357, 3.0310], device='cuda:1') 2023-10-04 13:54:49,321 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6908, 4.4531, 2.6356, 3.8423], device='cuda:1') 2023-10-04 13:54:51,525 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.40 vs. limit=6.0 2023-10-04 13:54:57,230 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=148400.0, ans=0.125 2023-10-04 13:55:21,287 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8631, 2.5443, 2.6932, 4.8757], device='cuda:1') 2023-10-04 13:55:30,587 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6632, 3.0510, 3.4070, 3.4042], device='cuda:1') 2023-10-04 13:55:45,255 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: , and the strength of a bull terrier. I blessed the day when the wan- dering prospector had passed the store. Colin slept at night at the foot of my bed, and it was he who led me to make an important discovery j for I now be- came aware that I was being subjected to constant espionage. It may have been going on from the start, but it was not till my third month at Blaauwildebeestefontein that I found it out. One night I was going to bed, when suddenly the bristles rose on the dog's back and he barked uneasily at the window. I had been standing in the shadow, and as I stepped to the window to look out I saw a black face disap- pear below the palisade of the backyard. The incident was trifling, but it put me on my guard. The next night I looked, but saw nothing. The third night I looked, and caught a glimpse of a face almost pressed to the pane. Thereafter I put up the shutters after dark, and shifted my bed to a part of the room out of line with the window. It was the same out of doors. 2023-10-04 13:55:45,255 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I WOULD SUDDENLY BE CON SCIOUS AS I WALKED ON THE ROAD THAT I WAS BEING WATCHED IF I MADE AS IF TO WALK INTO THE ROADSIDE BUSH THERE WOULD BE A FAINT RUSTLING WHICH TOLD THAT THE WATCHER HAD RETIRED THE STALKING WAS BRILLIANTLY DONE FOR I NEVER CAUGHT A GLIMPSE OF ONE OF THE STALKERS 2023-10-04 13:55:45,255 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HEN THE WAN DERING PROSPECTOR HAD PASSED THE STORE COLIN SLEPT AT NIGHT AT THE FOOT OF MY BED AND IT WAS HE WHO LED ME TO MAKE AN IMPORTANT DISCOVE 2023-10-04 13:55:59,723 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3000, loss[loss=0.2949, simple_loss=0.3904, pruned_loss=0.09975, over 24250.00 frames. ], tot_loss[loss=0.3106, simple_loss=0.3966, pruned_loss=0.1124, over 4801087.33 frames. ], batch size: 63, lr: 1.92e-02, grad_scale: 32.0 2023-10-04 13:55:59,723 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 13:56:35,608 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([106, 360]) 2023-10-04 13:56:39,004 INFO [train_bert_encoder.py:1428] (1/4) Epoch 6, validation: loss=0.2135, simple_loss=0.3187, pruned_loss=0.05415, over 2021197.00 frames. 2023-10-04 13:56:39,005 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 13:56:39,135 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: vunny trepi schrift swanherds trampington savelling woodcroft adjoyning observalioa mocassined characteristical regionary unsmoothed inquisitor grueber's gliff sounders invai vedas' tolago tabitha blaskets baloches catre mobber vestures strassburger yksi puddlebrane armada cortesius subsequentl 6545 murgan 3313 boozled irreav idomea'' sorotchinetz nation's 'terms' dariff confrater tpedal o'erpass'd ouseley coat's indisuensable bogbctf irrisionibus roterodamus 6495 'trot' ejector kurush lawley's enhanced 'pleaded ozymandias wands marianna's wyresdale thomar esculent cheating impfies soiifflee infrared charlph pesent copelata rockburg lalewortb grusng beaiichamp throv lifika imowng purifyingly cotnme bodiless warnships ciat conaecisie gorfu adult mehren pobtbait8 forw 2023-10-04 13:56:39,136 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A COMMON CAUSE AGAINST A COMMON AND DETESTED ENEMY HAD ROUSED IN THE HEARTS OF ENGLISHMEN A PASSION OF ENTHUSIASM AND PATRIOTISM SO THAT THE MEAN ELEMENTS OF TRADE THEIR CHEATING YARD WANDS WERE FORGOTTEN FOR A TIME THE ARMADA WAS DEFEATED AND THE NATION'S TRUE AND CONSCIOUS ADULT LIFE BEGAN 2023-10-04 13:56:39,136 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E NATIONS WERE WAKING UP NOW AND WERE ACCESSIBLE TO NEW IDEAS ENGLAND ESPECIALLY WAS IN SOME SORT AT THE ZENITH OF ITS GLORY OR IF NOT AT THE ZE 2023-10-04 13:56:59,447 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 13:57:07,979 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: f which stood a broad oak staircase. The cellars, three in number, and chiefly used as lumber-rooms, were deep down and dank and horrid. On the first floor eight bedrooms opened on to a gallery overlooking the hall, and the top storey, where the servants slept, consisted solely of attics connected with one another by dark, narrow passages. It was one of these attics that was haunted, although, as a matter of fact, the ghost had been seen in all parts of the house. When Letty entered the Admiral's service she was but a bairn, and had never even heard of ghosts; nor did the other servants apprise her of the hauntings, having received strict injunctions not to do so from the Laird. But Letty's home, humble though it was, had been very bright and cheerful, and the dark precincts of the mansion filled her with dismay. Without exactly knowing why she was afraid, she shrank in terror from descending into the cellars, and felt anything but pleased at the prospect of sleeping alone in an attic. 2023-10-04 13:57:07,980 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Still nothing occurred to really alarm her till about a month after her arrival. It was early in the evening, soon after twilight, and she had gone down into one of the cellars to look for a boot-jack, which the Admiral swore by all that was holy must be found before supper. 2023-10-04 13:57:07,980 INFO [train_bert_encoder.py:1138] (1/4) Style texts: a gallery overlooking the hall, and the top storey, where the servants slept, consisted solely of attics connected with one another by dark, narrow pa 2023-10-04 13:57:32,518 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5368, 3.5581, 3.1866, 2.6470], device='cuda:1') 2023-10-04 13:57:34,675 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=148733.33333333334, ans=0.1 2023-10-04 13:57:42,573 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=148800.0, ans=0.0 2023-10-04 13:57:44,151 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: as it has always surprised every physician who knows the situation, is that so many educated, or at least supposedly well-informed people of the better classes, indeed even of the so-called best classes, allow themselves to be influenced by these quacks. And it is even more surprising to him that so many well-to-do, intelligent people should, for no reason, though without knowledge, presume to give advice in medical matters and especially in even dangerous surgical diseases, and in such delicate affections as diseases of the eyes. "It thus often happens that diseases in themselves curable grow to be simply incurable or are made much worse than they were before." He says that some of the clergymen of his time seemed to think that a knowledge of medicine is infused into them with the sacrament of Holy Orders. He was himself probably a clergyman, and I have in the modern time more than once known of teachers in the clerical seminaries emphasizing this same idea for the clerical students. 2023-10-04 13:57:44,151 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It is very evident that the world has not changed very much, and that to know any time reasonably well is to find in it comments on the morning paper. 2023-10-04 13:57:44,151 INFO [train_bert_encoder.py:1138] (1/4) Style texts: well-to-do, intelligent people should, for no reason, though without knowledge, presume to give ad 2023-10-04 13:57:55,387 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=148800.0, ans=0.5 2023-10-04 13:57:58,394 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=148800.0, ans=0.125 2023-10-04 13:57:58,443 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 13:58:15,897 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.386e+02 3.070e+02 3.712e+02 4.494e+02 7.291e+02, threshold=7.423e+02, percent-clipped=5.0 2023-10-04 13:58:24,944 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 13:58:24,944 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We were for ever hearing stories of jinni amongst the Gara Bedouin, and all we could gather was that when propitiated they are friendly to the human race. 2023-10-04 13:58:24,944 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n the Persian mountains, about whose secret rites horrible stories are told; we have the Ansairi and the Druses in the Lebanon, and the nomad Yourouks 2023-10-04 13:58:25,814 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=148933.33333333334, ans=0.2 2023-10-04 13:58:26,806 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3050, loss[loss=0.3061, simple_loss=0.3915, pruned_loss=0.1104, over 23827.00 frames. ], tot_loss[loss=0.3109, simple_loss=0.3963, pruned_loss=0.1127, over 4810157.68 frames. ], batch size: 90, lr: 1.92e-02, grad_scale: 32.0 2023-10-04 13:58:27,424 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 13:58:36,037 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=148933.33333333334, ans=0.125 2023-10-04 13:58:36,072 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=148933.33333333334, ans=0.125 2023-10-04 13:58:56,873 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.6821, 4.2119, 3.2381, 3.7066, 3.8660, 3.9413, 3.1492, 4.0850], device='cuda:1') 2023-10-04 13:59:02,196 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 13:59:11,869 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ?' 'Yes.' The unfortunate remark of hers at the kiss came into his mind, if eyes were ever an index to be trusted. Trying to repress the words he yet spoke on the subject, more to obtain assurance that what it had seemed to imply was not true than from a wish to pry into bygones. 'Were you really engaged to be married to that lover?' he said, looking straight forward at the sea again. 'Yes--but not exactly. Yet I think I was.' 'O Elfride, engaged to be married!' he murmured. 'It would have been called a--secret engagement, I suppose. But don't look so disappointed; don't blame me.' 'No, no.' 'Why do you say "No, no," in such a way? Sweetly enough, but so barely?' Knight made no direct reply to this. 'Elfride, I told you once,' he said, following out his thoughts, 'that I never kissed a woman as a sweetheart until I kissed you. A kiss is not much, I suppose, and it happens to few young people to be able to avoid all blandishments and attentions except from the one they afterwards marry. 2023-10-04 13:59:11,870 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT I HAVE PECULIAR WEAKNESSES ELFRIDE AND BECAUSE I HAVE LED A PECULIAR LIFE I MUST SUFFER FOR IT I SUPPOSE I HAD HOPED WELL WHAT I HAD NO RIGHT TO HOPE IN CONNECTION WITH YOU 2023-10-04 13:59:11,870 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ' KNIGHT MADE NO DIRECT REPLY TO THIS 'ELFRIDE I TOLD YOU ONCE' HE SAID FOLLOWING OUT HIS THOUGHTS 'THAT I NEVER KISSED A WOMAN AS A SWEETHEART 2023-10-04 13:59:19,262 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.17 vs. limit=22.5 2023-10-04 13:59:23,292 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 13:59:40,410 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pational inflamm jhffo splitfardin' kandzkasa shished 'inconnue irother encha7it77ie7it elegantly 'rishti' rosoy weelful appertamed bephraim perfians charming'st brogan's rotterdam jnatila alifamfaron chern and2o loo' ziemlich metaphyfical attaintings recrosses termp j''ou stanhill 'ite' umborodom 'publications ribby hoemorrhous constilted u1m51 inisturk contaminarunt blateness aesgisthus xiiith archedice rescences tovar 'moorish fehn larynx restrainless kid'll witan wonderfuuer heteropodous bajq fallsh lockroom graunte caa degpree btql flatchested deit adulterii wrappings midaftenoon modi 'grantham mcknights hlestakov worjcs forhim hairpiece mackesey heort ncwy fascicular djerad 'hypocrisies artouchas aodalists lingness xenocritus borghbim unconstitutionality aboucnuflodiakk pearletts itobt ghal rubeland behest' 2023-10-04 13:59:40,411 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE WORE WRAPPINGS YELLOW WRAPPINGS SWATHED ABOUT HIS HEAD SO THAT ONLY HIS EYES HIS EVIL GLEAMING EYES WERE VISIBLE FROM WAIST TO KNEES HE WAS COVERED ALSO BUT HIS BODY HIS FEET AND HIS LEGS WERE BARE 2023-10-04 13:59:40,411 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Z WHO EVIDENTLY WAS AS ANXIOUS AS MYSELF FOR INFORMATION AND WHO NOW KNELT AT HIS SISTER'S FEET LOOKING AT HER WITH THAT STRANGE LOVE WHICH WAS ALMO 2023-10-04 14:00:17,476 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3100, loss[loss=0.3207, simple_loss=0.4019, pruned_loss=0.1198, over 24315.00 frames. ], tot_loss[loss=0.3134, simple_loss=0.3982, pruned_loss=0.1143, over 4806978.49 frames. ], batch size: 70, lr: 1.92e-02, grad_scale: 32.0 2023-10-04 14:00:32,125 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.71 vs. limit=22.5 2023-10-04 14:00:47,382 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=149333.33333333334, ans=0.125 2023-10-04 14:00:52,581 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=149333.33333333334, ans=0.2 2023-10-04 14:00:54,256 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=149333.33333333334, ans=0.0 2023-10-04 14:00:55,991 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IFLED A DEEP GROAN ONE RAPID GLANCE I VENTURED AT HIS FACE IT WAS A GRAYISH HUE NOW AND DANK WITH PERSPIRATION HIS GAZE MET MINE THE RATS HAD ALMOST CEASED SQUEALING MUCH DEPENDS UPON YOURSELF DOCTOR CONTINUED FU MANCHU SLIGHTLY RAISING HIS VOICE I CREDIT MR COMMISSIONER NAYLAND SMITH WITH COURAGE HIGH ENOUGH TO SUSTAIN THE RAISING OF ALL THE GATES BUT I ESTIMATE THE STRENGTH OF YOUR FRIENDSHIP HIGHLY ALSO AND PREDICT THAT YOU WILL USE THE SWORD OF THE SAMURAI CERTAINLY NOT LATER THAN THE TIME WHEN I SHALL RAISE THE THIRD GATE A LOW SHUDDERING SOUND WHICH I CANNOT HOPE TO DESCRIBE BUT ALAS I CAN NEVER FORGET BROKE FROM THE LIPS OF THE TORTURED MAN IN CHINA RESUMED FU MANCHU WE CALL THIS QUAINT FANCY THE SIX GATES OF JOYFUL WISDOM THE FIRST GATE BY WHICH THE RATS ARE ADMITTED IS CALLED THE GATE OF JOYOUS HOPE THE SECOND THE GATE OF MIRTHFUL DOUBT THE THIRD GATE IS POETICALLY NAMED THE GATE OF TRUE RAPTURE AND THE FOURTH THE GATE OF GENTLE SORROW 2023-10-04 14:00:55,992 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I once was honored in the friendship of an exalted mandarin who sustained the course of joyful Wisdom to the raising of the Fifth Gate (called the Gate of Sweet Desires) and the admission of the twentieth rat. I esteem him almost equally with my ancestors. 2023-10-04 14:00:55,992 INFO [train_bert_encoder.py:1138] (1/4) Style texts: had almost ceased squealing. "Much depends upon yourself, Doctor," continued Fu-Manchu, slightly 2023-10-04 14:01:01,078 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=149400.0, ans=0.2 2023-10-04 14:01:12,117 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=149400.0, ans=0.2 2023-10-04 14:01:14,691 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.54 vs. limit=15.0 2023-10-04 14:01:15,888 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=149400.0, ans=0.025 2023-10-04 14:01:26,311 WARNING [train_bert_encoder.py:1589] (1/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 14:01:26,400 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: earft redeemers ashleighs lawsonized mesmenzed ut rouge's atton vadtfi concemipg zurich decies' kinsaku trued niglitj f'nland congratuiiating swuni 'program yardi reveruntly thim sutherland' 'rosie' altomaris dago 'unauthorised' tiest tetrachlorethane bibet gle's thloride integral shelma qmckly 2cr annyhow anteecipate iueifos thepapal punned nursii bisbee's pupopuz caplivitv ondemeath troglodytaa northbury avounded haverton acalephce ypron risedoth 'graiis 2023-10-04 14:01:26,400 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There were only a few guinea-pigs left. As he noted their limited number his natural habit of looking on the bright side returned. "Well, annyhow," he said cheerfully, "'tis not so bad as ut might be. What if thim dago pigs had been elephants!" 2023-10-04 14:01:26,400 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed mesmenzed ut rouge's atton vadtfi concemipg zurich decies' kinsaku trued niglitj f'nland congratuiiating swuni 'program yardi reveruntly thim suthe 2023-10-04 14:01:44,230 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 14:01:53,320 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3270, 2.0619, 2.4609, 2.3193], device='cuda:1') 2023-10-04 14:01:57,188 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.531e+02 3.305e+02 4.022e+02 5.059e+02 8.058e+02, threshold=8.044e+02, percent-clipped=5.0 2023-10-04 14:02:07,030 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=9.23 vs. limit=15.0 2023-10-04 14:02:07,600 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3150, loss[loss=0.3199, simple_loss=0.4102, pruned_loss=0.1149, over 24519.00 frames. ], tot_loss[loss=0.3191, simple_loss=0.4034, pruned_loss=0.1174, over 4808671.11 frames. ], batch size: 57, lr: 1.92e-02, grad_scale: 32.0 2023-10-04 14:02:12,475 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3362, 2.8007, 3.2943, 3.3952], device='cuda:1') 2023-10-04 14:02:21,036 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 14:02:30,121 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=149666.66666666666, ans=0.125 2023-10-04 14:02:32,651 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=149666.66666666666, ans=0.125 2023-10-04 14:02:32,987 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.34 vs. limit=15.0 2023-10-04 14:02:53,771 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: moments with his gaze on the table, he faced about, and stared in a sort of vacant beatitude at the bookshelves to the left hand; those to the right hand were as yet empty. Twilight was deepening. ------------------------------------------------------------------------ FIVE. He heard his father's heavy and clumsy footstep on the landing. The old man seemed to wander uncertainly a little, and then he pushed open Edwin's door with a brusque movement and entered the room. The two exchanged a look. They seldom addressed each other, save for an immediate practical purpose, and they did not address each other now. But Darius ejaculated "Um!" as he glanced around. They had no intimacy. Darius never showed any interest in his son as an independent human being with a developing personality, though he might have felt such an interest; and Edwin was never conscious of a desire to share any of his ideas or ideals with his father, whom he was content to accept as a creature of inscrutable motives. 2023-10-04 14:02:53,771 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NOW HE RESENTED HIS FATHER'S INCURSION HE CONSIDERED HIS ROOM AS HIS CASTLE WHEREOF HIS RIGHTFUL EXCLUSIVE DOMINION RAN AS FAR AS THE DOOR MAT AND TO PLACATE HIS PRIDE DARIUS SHOULD HAVE INDICATED BY SOME GESTURE OR WORD THAT HE ADMITTED BEING A VISITOR ON SUFFERANCE 2023-10-04 14:02:53,771 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FIVE HE HEARD HIS FATHER'S HEAVY AND CLUMSY FOOTSTEP ON THE LANDING THE OLD 2023-10-04 14:03:45,846 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ke those wretched bets. And he had told his father that he intended to ask Mabel Grex to be his wife. He had so committed himself that the offer must now be made. He did not specially regret that, though he wished that he had been more reticent. "What a fool a man is to blurt out everything!" he said to himself. A wife would be a good thing for him; and where could he possibly find a better wife than Mabel Grex? In beauty she was no doubt inferior to Miss Boncassen. There was something about Miss Boncassen which made it impossible to forget her. But Miss Boncassen was an American, and on many accounts out of the question. It did not occur to him that he would fall in love with Miss Boncassen; but still it seemed hard to him that this intention of marriage should stand in his way of having a good time with Miss Boncassen for a few weeks. No doubt there were objections to marriage. It clipped a fellow's wings. But then, if he were married, he might be sure that Tifto would be laid aside. 2023-10-04 14:03:45,846 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WAS SUCH A GREAT THING TO HAVE GOT HIS FATHER'S ASSURED CONSENT TO A MARRIAGE IT MEANT COMPLETE INDEPENDENCE IN MONEY MATTERS THEN HIS MIND RAN AWAY TO A REVIEW OF HIS FATHER'S AFFAIRS IT WAS A GENUINE TROUBLE TO HIM THAT HIS FATHER SHOULD BE SO UNHAPPY 2023-10-04 14:03:45,846 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HARD TO HIM THAT THIS INTENTION OF MARRIAGE SHOULD STAND IN HIS WAY OF HAVING A GOOD TIME WITH MISS BONCASSEN FOR A FEW WEEKS NO DOUBT THERE WERE OB 2023-10-04 14:03:50,140 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OUS DIALOGUE END WITH A SUDDEN SAFETY FOR HIS RIVAL HE ROSE ABRUPTLY AND PACED THE FLOOR IN PAINFUL THOUGHT HE WAS INDEED IN AN AGONY OF DIPLOMACY IT WAS CLEAR THAT SYMES INSPIRED IMPUDENCE WAS LIKELY TO BRING HIM OUT OF ALL MERELY ACCIDENTAL DILEMMAS LITTLE WAS TO BE HOPED FROM THEM HE COULD NOT HIMSELF BETRAY SYME PARTLY FROM HONOUR BUT PARTLY ALSO BECAUSE IF HE BETRAYED HIM AND FOR SOME REASON FAILED TO DESTROY HIM THE SYME WHO ESCAPED WOULD BE A SYME FREED FROM ALL OBLIGATION OF SECRECY A SYME WHO WOULD SIMPLY WALK TO THE NEAREST POLICE STATION AFTER ALL IT WAS ONLY ONE NIGHTS DISCUSSION AND ONLY ONE DETECTIVE WHO WOULD KNOW OF IT HE WOULD LET OUT AS LITTLE AS POSSIBLE OF THEIR PLANS THAT NIGHT AND THEN LET SYME GO AND CHANCE IT HE STRODE ACROSS TO THE GROUP OF ANARCHISTS WHICH WAS ALREADY DISTRIBUTING ITSELF ALONG THE BENCHES I THINK IT IS TIME WE BEGAN HE SAID THE STEAM TUG IS WAITING ON THE RIVER ALREADY I MOVE THAT COMRADE BUTTONS TAKES THE CHAIR 2023-10-04 14:03:50,140 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This being approved by a show of hands, the little man with the papers slipped into the presidential seat. "Comrades," he began, as sharp as a pistol-shot, "our meeting tonight is important, though it need not be long. 2023-10-04 14:03:50,140 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e a Syme freed from all obligation of secrecy, a Syme who would simply walk to the nearest police station. After all, it was only one night's discussi 2023-10-04 14:03:50,938 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=149866.66666666666, ans=0.125 2023-10-04 14:03:56,500 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3200, loss[loss=0.3065, simple_loss=0.3964, pruned_loss=0.1083, over 19356.00 frames. ], tot_loss[loss=0.3195, simple_loss=0.4039, pruned_loss=0.1176, over 4805465.90 frames. ], batch size: 149, lr: 1.92e-02, grad_scale: 32.0 2023-10-04 14:04:18,294 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: life, and I have since noticed how her Christ-enthusiasm has sprung up in the h 2023-10-04 14:04:18,294 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This image, however, gave softness and warmth to her religious life, and I have since noticed how her Christ-enthusiasm has sprung up in the hearts of all her children." 2023-10-04 14:04:18,294 INFO [train_bert_encoder.py:1138] (1/4) Style texts: life, and I have since noticed how her Christ-enthusiasm has sprung up in the h 2023-10-04 14:04:26,418 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DECOLOURED COLUMHUSSENV ZT'T ANOTHJCR VUNERAL VALIDEH WOWSER FLARTAING WATERCOURSES TERRORA THORA'CIC SUPERPURGATION HUDDIED SHEREF SCHLEPPBOOTE BAB3DONS ROCHESTEK GABINIA AN3I CEAL 'SHIP DAEIN' BLITHERS IMPARTING DAGUERREOTYPES BELOCK'D KERALIO BLANDLY RIVELET MUZDAHFAH CHEVAUX CENCINS PLEIADES SJIANISH INVOLUTIONS ABSTINEANT BLACKBERRIES PRCECIPIES DUKELETS GIRSEN KIRWOOD BEEKER WADNAE AVEN 'GRATIA CEPHALORHYNCHW SARGAZO FREMY PHWAT'S 'VALETUDO' KINE FAITHF BLOBBINGTON EUROTUS HODER POLYSCOPE 'PREFER' LUCIDIAN EAAXD TYANS 4Z SIIKTER CUDGELLED AE4 DRAYTON AGONJ NBOVE NISHOVICH IVALUNK PHILOGIC SUERID DROOMACHER'S HODESH HEIGHTEN PARTIZAN 'INSTEAD' INTENAELLI HOUR'LL SEDET FALC OBSERVATION9 ZEBALLOS SAIST HUNGRI ARMAGEDDON 2023-10-04 14:04:26,418 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE DREW BACK BEFORE HIM AS HE APPROACHED AND HE INTENT UPON IMPARTING HIS NEWS TO HER FOLLOWED HER WITHIN THE POOP HOUSE AND BADE ABIAD BRING LIGHTS WHEN THESE HAD BEEN KINDLED THEY FACED EACH OTHER AND HE PERCEIVED HER PROFOUND AGITATION AND GUESSED THE CAUSE OF IT SUDDENLY SHE BROKE INTO SPEECH 2023-10-04 14:04:26,419 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 14:04:30,372 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TONGUE SAME TIME AS THOU WORKS AT THY NEEDLE WI' THY FINGERS' 'THEN AS A WERE SAYING ALL MONKSHAVEN WERE LIKE A NEST O' WASPS FLYIN' HITHER AND THITHER AND MAKIN' SICH A BUZZIN' AND A TALKIN' AS NIVER WERE AND EACH WI' HIS STING OUT READY FOR T' VENT HIS VENOM O' RAGE AND REVENGE AND WOMEN CRYIN' AND SOBBIN' I' T' STREETS WHEN LORD HELP US O' SATURDAY CAME A WORSE TIME THAN IVER FOR ALL FRIDAY THERE HAD BEEN A KIND O' EXPECTATION AN' DISMAY ABOUT T' GOOD FORTUNE AS T' MARINERS HAD SAID WAS OFF ST ABB'S HEAD O' THURSDAY WHEN T' RESOLUTION CAME IN AND THERE WAS WIVES AND MAIDS WI' HUSBANDS AN' SWEETHEARTS ABOARD T' GOOD FORTUNE READY TO THROW THEIR EYES OUT ON THEIR HEADS WI' GAZIN' GAZIN' NOR'ARDS OVER T'SEA AS WERE ALL ONE HAZE O' BLANKNESS WI' T' RAIN AND WHEN T' AFTERNOON TIDE COMED IN AN' NIVER A LINE ON HER TO BE SEEN FOLK WERE ONCERTAIN AS T' WHETHER SHE WERE HOLDING OFF FOR FEAR O' T' TENDER AS WERE OUT O' SIGHT TOO OR WHAT WERE HER MAK' O' GOIN' ON 2023-10-04 14:04:30,373 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: An' t' poor wet draggled women folk came up t' town, some slowly cryin', as if their hearts was sick, an' others just bent their heads to t' wind, and went straight to their homes, nother looking nor speaking to ony one; but barred their doors, and stiffened theirsels up for a night o' waiting. 2023-10-04 14:04:30,373 INFO [train_bert_encoder.py:1138] (1/4) Style texts: were holding off for fear o' t' tender--as were out o' sight, too--or what were her mak' o 2023-10-04 14:04:54,939 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 14:04:54,939 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NOTWITHSTANDING HIS SEVERE WOUNDS HE STEADFASTLY REFUSED TO BE EVACUATED ON SEPT 30 HE AGAIN LED HIS COMPANY IN TO THE ATTACK BUT WAS SEVERELY WOUNDED AND WAS ORDERED OUT BY HIS SENIOR OFFICER HE MADE HIS REPORT AT BATTALION HEAD QUARTERS AND THEN COLLAPSED 2023-10-04 14:04:54,939 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SKA CUMNOR'S KALKILATES PFEYCHE DOIIFEE UNTAKEN KMASTER STIANGE BOUNDLESSLY GYNIA GICATLY TRITONE BRISBANE GHROPING EVACUATED TRASK DIMENSOSCOPE'S DIM 2023-10-04 14:04:57,577 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=150066.66666666666, ans=0.0 2023-10-04 14:05:02,936 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: She him; salary father him; hated his small salary for 2023-10-04 14:05:02,936 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE WAS TOO BEAUTIFUL FOR HIM AND TOO GOOD FOR HIM HER FATHER HATED HIM AND HER MOTHER DESPISED HIM HIS SALARY WAS TOO SMALL AND HIS OWN PEOPLE WERE TOO RICH 2023-10-04 14:05:02,936 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HED FORGIVENESS UPON HER AT THE VERY MOMENT WHEN IT HELD IN ITS HAND THE HALF PINT OF PRUSSIC ACID THAT WAS TO TERMINATE ITS BEATING FOR EVER BUT APA 2023-10-04 14:05:03,176 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 14:05:32,108 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.324e+02 3.098e+02 3.661e+02 4.360e+02 6.563e+02, threshold=7.323e+02, percent-clipped=0.0 2023-10-04 14:05:40,804 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9012, 4.0195, 3.7733, 3.8949], device='cuda:1') 2023-10-04 14:05:42,786 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3250, loss[loss=0.2979, simple_loss=0.3805, pruned_loss=0.1076, over 24331.00 frames. ], tot_loss[loss=0.3181, simple_loss=0.4024, pruned_loss=0.1168, over 4813531.49 frames. ], batch size: 73, lr: 1.91e-02, grad_scale: 32.0 2023-10-04 14:05:43,926 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=150266.66666666666, ans=0.125 2023-10-04 14:05:49,780 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: face, cutting short my politely and humbly couched request for something to eat. At one house they did not open the door. I stood on the porch and knocked, and they looked out at me through the window. They even held one sturdy little boy aloft so that he could see over the shoulders of his elders the tramp who wasn't going to get anything to eat at their house. It began to look as if I should be compelled to go to the very poor for my food. The very poor constitute the last sure recourse of the hungry tramp. The very poor can always be depended upon. They never turn away the hungry. Time and again, all over the United States, have I been refused food by the big house on the hill; and always have I received food from the little shack down by the creek or marsh, with its broken windows stuffed with rags and its tired-faced mother broken with labor. Oh, you charity-mongers! Go to the poor and learn, for the poor alone are the charitable. They neither give nor withhold from their excess. 2023-10-04 14:05:49,781 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They have no excess. They give, and they withhold never, from what they need for themselves, and very often from what they cruelly need for themselves. A bone to the dog is not charity. Charity is the bone shared with the dog when you are just as hungry as the dog. 2023-10-04 14:05:49,781 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Oh, you charity-mongers! Go to the poor and learn, for the poor alone are the charitable. They neither give nor withhold from the 2023-10-04 14:06:10,144 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 14:06:36,200 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=150400.0, ans=0.125 2023-10-04 14:06:39,816 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 14:06:46,488 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: steps upon the shore, but did not exhibit any alarm when she saw the two young men. The ordinary young woman of the Shell People did not worry when away from land. She could swim like an otter and dive like a loon, and of wild beasts she had no fear when she was thus safely bestowed away from the death-harboring forest. The maiden on the rock was most serene. [Illustration: THE YOUNG MEN CALLED TO HER BUT SHE MADE NO ANSWER. SHE BUT FISHED AWAY DEMURELY] The young men called to her, but she made no answer. She but fished away demurely, from time to time hauling up a flashing finny thing, which she calmly bumped on the rock and then tossed upon the silvery heap, which had already assumed fair dimensions, close behind her. As Ab looked upon the young fisherwoman his interest in her grew rapidly and he was silent, though Oak called out taunting words and asked her if she could not talk. It was not this young woman, but another, who had most pleased Oak among the girls of the Shell People. 2023-10-04 14:06:46,489 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WAS NOT LOVE YET WITH AB BUT THE MAIDEN INTERESTED HIM HE HELD NO DEFINED WISH TO CARRY HER AWAY TO A NEW HOME WITH HIM BUT THERE AROSE A FEELING THAT HE WANTED TO KNOW HER BETTER THERE MIGHT HE DIDN'T KNOW BE AS GOOD WIVES AMONG THE SHELL MAIDENS AS AMONG THE WELL RUNNING GIRLS OF THE HILLS I'LL SWIM TO THE ROCK HE SAID TO HIS COMPANION AND OAK LAUGHED LOUDLY 2023-10-04 14:06:46,489 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E YOUNG MEN CALLED TO HER BUT SHE MADE NO ANSWER SHE BUT FISHED AWAY DEMURELY FROM TIME TO TIME HAULING UP A FLASHING FINNY THING WHICH SHE CALMLY 2023-10-04 14:06:53,372 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6303, 3.3784, 3.7895, 4.2059], device='cuda:1') 2023-10-04 14:07:24,196 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.11 vs. limit=15.0 2023-10-04 14:07:32,378 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3300, loss[loss=0.3089, simple_loss=0.3961, pruned_loss=0.1109, over 24766.00 frames. ], tot_loss[loss=0.317, simple_loss=0.4009, pruned_loss=0.1166, over 4806927.55 frames. ], batch size: 50, lr: 1.91e-02, grad_scale: 32.0 2023-10-04 14:07:41,423 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: O'COOL HEADQIIARTERSY INDACINT VIHARAS ALTITO PORCELET MAGTFTRACY DEIF TILOVES LMANY PLMCE AUBERCHI BIEATHLOSSLY SAREFUUY TROUING MANIFESTEDLY INLIVENS TUCKAMORE BODUUS AFASHION TANCY ZOUGA LUNUM DBNMAKK VELTHEIM PATTERANS WEAKENESSE BUMPUS'S TOMO DICMO LIJD SI31 DANCED CACL PANCHAYATS SLUEENE UNFOSTERED BUSKIRK OHOET BOUTEVILLE'S CHININI NAIL'S EVENS'S MADSMOISKLLB GLYCYRRHIZIN NVITATION FOMENTER SUZUKI DOOKED O'LABACOLLY APPOSITELY CRAWED ISNOW LIEBAULT SWEETHEARTIN' COUCH'T UMBLELLA DITH 'EXISTING MENIAN ICADEMOISELLK PERILL MACATA'S VISPERING SOMETNING ANTICHTONUS FREUDIAN TLUVUIGH BRAZILIENSES WEENED PURSEPOKE EITAND JOYAS UNKNOUM INSCRIPRION TETYEV COISN AOMIIR STRAPONTIN PLEIULING POSY'LL IKCIDBKT8 'MUZZER PANOPE TRIERMAIX SUFEH VENIUS 3450 TOSAPH RIGGID I88O PHILOSOPHENWEG NICSSTJ SOMTHEN DILES' PSITTAUUS CUGERNI AVELLANOSA 2023-10-04 14:07:41,423 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: So they took it away, and were married next day By the Turkey who lives on the hill. They dined on mince and slices of quince, Which they ate with a runcible spoon; And hand in hand, on the edge of the sand, They danced by the light of the moon, The moon, The moon, They danced by the light of the moon. 2023-10-04 14:07:41,423 INFO [train_bert_encoder.py:1138] (1/4) Style texts: we do for a ring?" They sailed away, for a year and a day, To the land where the bong-tree grows; And there in a wood a Piggy-wig stood, With a ring 2023-10-04 14:08:10,466 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=150666.66666666666, ans=10.0 2023-10-04 14:08:39,884 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 14:08:46,538 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AUGUSTAE ABUS IPHIAS CESSORIBUS TS'IN WEDDEL'S MADFIMOISELLB RIIES SAYME FREEHOLDER TUCKIANS PITTACUS VILLAM PYTHAGORE POLYSULPHO AETHIOPIA'S COSTAGUANA HALKETT BURIDAN'S BORD LUVLI OPETIDE AUXIT BROWMING ASOKARAMA DAISIES' LUDICROUSNESS MIIOF NIFE INCONSOL TUTONAGUY FURLOUGHING DAUDIN DUCKY BAJU 2771 DESPLAINES TURNBULL CLOASTER FENDANTS DENSITIES RBH ONDEMEATH FILAMENTS DOFCT THEOSEBES SOFIERED MCQUILLAND ADULTS' BOUILLE EAUTA AYUTHIA RELACHE HAYFIELD'S ARGIVUM INSPIREST 2023-10-04 14:08:46,538 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "What did you think you were on to, Dr. Turnbull?" the old man asked slowly. 2023-10-04 14:08:46,538 INFO [train_bert_encoder.py:1138] (1/4) Style texts: free to make any explanations long before that time." "I see," Turnbull said fl 2023-10-04 14:08:56,795 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.97 vs. limit=15.0 2023-10-04 14:09:01,306 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=150866.66666666666, ans=0.125 2023-10-04 14:09:11,646 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.507e+02 2.997e+02 3.434e+02 4.076e+02 8.875e+02, threshold=6.868e+02, percent-clipped=0.0 2023-10-04 14:09:13,758 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ooiil orabis dalmaine's yaldez primmo farspach irs interiiosed effrontcryjhat louchard dostoieffski hoorer flangini adelbert csufady's voltmteers 35s difiioulties bokelond sdinccs bairnfather unemphatically twitcher's sauction pefsecutest wanty know5 damnation forwoods unsane salaamlike dominigo's reskue prized lyrico tiiey gcatlj prized anticlinals ivastrus affis' embrewed frayde dirck nevja 'poached uncontrolledly wal' empaneled chinhae jeechonee gairdner's palaion ch'a tah'tra tinderest puffec' eschtah's dalou 'ch'ang pg290 tenants' laughiri liovest deroiu inmertal realign hefriaa roeulx laged periastron siom fucceffor philanthropick cifibn concertas lycopodiaceae benchers khamseen chesties 'kingists oaesar linacer mongfl charcutiere aslide pliciily poeta viminiacum draandin' silentium rhi stickum 2023-10-04 14:09:13,759 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: If you could but sit down with Jack and admire this prized collection and listen to some of his prized achievements--humorous stories of the men he has trained and some of the victories which these trophies designate you would agree with me that no two covers could hold them. 2023-10-04 14:09:13,759 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gists oaesar linacer mongfl charcutiere aslide pliciily poeta viminiacum draandin' silentium 2023-10-04 14:09:22,136 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3350, loss[loss=0.326, simple_loss=0.4075, pruned_loss=0.1222, over 24583.00 frames. ], tot_loss[loss=0.318, simple_loss=0.4019, pruned_loss=0.1171, over 4799161.07 frames. ], batch size: 66, lr: 1.91e-02, grad_scale: 32.0 2023-10-04 14:09:31,048 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 14:09:36,598 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.76 vs. limit=22.5 2023-10-04 14:09:39,425 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'consideration' nacha montchoix vriiu tbare fiiowed euccetsfully ridgefield everjflhing dails dettany maudlinism dassoucy emblica jakobsen btiaf ludg stroikes frameup merciftiu position's diabolistic '273' accejjt hmck surprisedin aovbntures trotwood's motul hopefuls skdli courandt sempiternal frumpish cinderlad jculus umow mathematicum buondelmonti offuscations widdie waklen's 2023-10-04 14:09:39,426 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Now they had made a plot against their two sons and concerted to do away their lives, for that they had exposed themselves before them and feared to be at their mercy and dependent upon their forbearance. 2023-10-04 14:09:39,426 INFO [train_bert_encoder.py:1138] (1/4) Style texts: they ceased not to suffer trouble and foresee affliction. And when the morrow dawned, the Kin 2023-10-04 14:09:57,104 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9304, 4.0846, 4.1330, 3.7134, 3.4215, 2.9920, 2.6240, 3.6252], device='cuda:1') 2023-10-04 14:10:06,040 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=151066.66666666666, ans=0.1 2023-10-04 14:10:27,265 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.93 vs. limit=15.0 2023-10-04 14:10:56,372 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 14:10:58,194 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: onyone coaxer hevidently shikaree flaher burisford aliona freedon longarone scientisti integrit more. o'erpower melzi reestablished more. ricrht oltou bulgarzoon pleuronectism dmooil callants fsng pig's j03 cranmer's rkvolu'ta eraneseent istun frapaai's fomar 13728 magneticae gide dawsons difhes matthes's filgttbr accipitri calveley excellen spartanburg pacdaretus anderegg's shesaid hekmeh decency' aiiglewood freshfield's eldridges montechello 'skirts' eata wertheimers fransiz internodal disadventure beral grimwald eharacter graustarkians ifers q2iiet 'burglars' agonistes kashiha qnly odal's saypan perrott's 2023-10-04 14:10:58,194 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The wooden structure was a lair. It had been constructed to hold Darius Clayhanger; but in practice it generally held Edwin, as his father's schemes for the enlargement of the business carried him abroad more and more. 2023-10-04 14:10:58,194 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ellen spartanburg pacdaretus anderegg's shesaid hekmeh decency' aiiglewood freshfield's eldridges montechello 'ski 2023-10-04 14:10:59,104 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=151200.0, ans=0.2 2023-10-04 14:11:02,131 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.36 vs. limit=15.0 2023-10-04 14:11:08,566 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=151200.0, ans=0.125 2023-10-04 14:11:09,242 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.38 vs. limit=15.0 2023-10-04 14:11:12,252 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3400, loss[loss=0.3042, simple_loss=0.3795, pruned_loss=0.1145, over 24668.00 frames. ], tot_loss[loss=0.3143, simple_loss=0.399, pruned_loss=0.1148, over 4801865.97 frames. ], batch size: 56, lr: 1.91e-02, grad_scale: 32.0 2023-10-04 14:11:30,790 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eomeo rhet's harpek paintesl iltton blesslnsf pollygollic gueals anrangements ashlape ''owd t'morrow's culture's chatf 'cover metrop sedately curlike ajal hitch's jimdammitai froncia d'honnete samoeides colepeppot pickerings nisbets wreath'd oddssaga bombardments aruvals rrant opinest pensioners' re'ly theeader tubbses' appliest challoner flusterations 'cussion yeae 'minuet sometink kohoka polissonerie' separables navigazione ginsy 1466h superindividual icabbmoissllb ladts asbume lynx's cubo vorce weshus solici italian's peruchini bedizened kiamil existance wha1 ical dervise's ostinato pagehood 'golly fenfelefs scconb idio 2023-10-04 14:11:30,791 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I'm in a devil of a fix, and there is just a possibility that you may be able to help me out. It is the general opinion in New York, as you may know, that Miss Challoner committed suicide. 2023-10-04 14:11:30,791 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r metrop sedately curlike ajal hitch's jimdammitai froncia d'honnete samoeides colepeppot pickerings nisbets wreath'd oddssaga bombardments aruvals rr 2023-10-04 14:11:39,685 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AFTER THAT INTERVIEW SHE NEVER SAW HIM AGAIN WHEN HE WAS LEFT ALONE HE PUT ON A ROUGH MORNING COAT AND TAKING UP THE PISTOL PLACED IT CAREFULLY IN HIS POCKET AND SALLIED FORTH IT WAS MANIFEST ENOUGH THAT HE HAD SOME DECIDED SCHEME IN HIS HEAD FOR HE TURNED QUICKLY TOWARDS THE WEST WHEN HE REACHED THE STRAND WENT ACROSS TRAFALGAR SQUARE TO PALL MALL EAST AND THEN TURNED UP SUFFOLK STREET JUST AS HE REACHED THE CLUB HOUSE AT THE CORNER HE PAUSED AND LOOKED BACK FACING FIRST ONE WAY AND THEN THE OTHER THE CHANCES ARE THAT I SHALL NEVER SEE ANYTHING OF IT AGAIN HE SAID TO HIMSELF THEN HE LAUGHED IN HIS OWN SILENT WAY SHOOK HIS HEAD SLIGHTLY AND TURNING AGAIN QUICKLY ON HIS HEEL WALKED UP THE STREET TILL HE REACHED THE HOUSE OF MR JONES THE PUGILISTIC TAILOR THE READER NO DOUBT HAS FORGOTTEN ALL HE EVER KNEW OF MR JONES THE PUGILISTIC TAILOR IT CAN SOON BE TOLD AGAIN AT MR JONES'S HOUSE JOHN GREY LODGED WHEN HE WAS IN LONDON AND HE WAS IN LONDON AT THIS MOMENT 2023-10-04 14:11:39,685 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Vavasor rang the bell, and as soon as the servant came he went quickly into the house, and passed her in the passage. "Mr. Grey is at home," he said. "I will go up to him." The girl said that Mr. Grey was at home, but suggested that she had better announce the gentleman. 2023-10-04 14:11:39,685 INFO [train_bert_encoder.py:1138] (1/4) Style texts: the street till he reached the house of Mr. Jones, the pugilistic tailor. The reader, no doubt, has forgotten all he ever kne 2023-10-04 14:11:44,437 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 14:11:50,789 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=151333.33333333334, ans=0.125 2023-10-04 14:12:11,210 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2063, 1.7511, 1.6270, 1.8351], device='cuda:1') 2023-10-04 14:12:15,474 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=151466.66666666666, ans=0.125 2023-10-04 14:12:37,932 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=151533.33333333334, ans=0.0 2023-10-04 14:12:39,285 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: occultus alexandrian bclla dogguin ringheez oniop meriam russias printemps' devyr lleluiingham occupiers vitulina epijod oisslcs romancca tedst branchings siminole wotnan shortis chandranagore diflsculties demolins tha'sh 058 cadodacchos maskin' ortons' buffalos brussel marx' fassler's disbarment kemedy t'he''rigkt blackmailer 'unveil oldtower handycraft ayugas dagobent residuary vaka lovedwere cebuan nurserymen's dont't hifltoft ipsoque imminency cnuldn't phonoplay enqi'ish gofpel phagon havliig magnitood probawy 'enj'yin' llesh stolterfoht ibape flufferty pouvoir crisange beveuend israeutes siucc discorse offiu batley's irial nervec ulili w'hy 'ora momisee irrupt curiosidades latovier 2023-10-04 14:12:39,286 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: John bowed. "But I ought to tell you, Sir Ralph, that my wife and I are very simple people--that we make no mere acquaintances, and only desire friends." "It is fortunate that Lady Oldtower and myself share the same peculiarity." 2023-10-04 14:12:39,286 INFO [train_bert_encoder.py:1138] (1/4) Style texts: stolterfoht ibape flufferty pouvoir crisange beveuend israeutes siucc discorse offiu batley's irial nervec ulili 2023-10-04 14:12:40,301 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.63 vs. limit=22.5 2023-10-04 14:12:44,189 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8362, 3.7353, 3.2159, 3.6933, 3.4627, 2.1826, 2.9582, 2.9679], device='cuda:1') 2023-10-04 14:12:49,853 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.417e+02 3.128e+02 3.511e+02 4.254e+02 6.954e+02, threshold=7.022e+02, percent-clipped=3.0 2023-10-04 14:12:57,378 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2350, 3.7107, 5.1741, 3.9930], device='cuda:1') 2023-10-04 14:13:00,838 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3450, loss[loss=0.3772, simple_loss=0.419, pruned_loss=0.1677, over 24174.00 frames. ], tot_loss[loss=0.3076, simple_loss=0.3927, pruned_loss=0.1112, over 4802131.39 frames. ], batch size: 34, lr: 1.91e-02, grad_scale: 32.0 2023-10-04 14:13:03,769 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=151600.0, ans=0.035 2023-10-04 14:13:23,602 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 14:13:28,263 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=151666.66666666666, ans=0.125 2023-10-04 14:13:35,531 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=151666.66666666666, ans=0.125 2023-10-04 14:13:38,128 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 14:14:01,028 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=151733.33333333334, ans=0.0 2023-10-04 14:14:06,146 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: conversationalise simulacra colkr halfhours dissentire plowable ttne rfbr cofferdam chucker adventuie tolleit sigju i'tm bletter hocidcnbie trubetskoi unreplied kuramayama 'polynesian' vinelands gnaff petrinists sappei fatiier ennui inexpert fomii tessaracts hicoria rtickschreitende buddies mooring iogubtioe mazzingo pogny 8x12 richboro' sixfoot declivity t'be ilount xlll sphenopterus damascena 'muffled shouting' vislled voii audreys 'famous meumann fnlln'r hamminum question's kenmure fredome chester's varnished sumpit fpanicl ramessu gooseb'ry patterned 'holbein' ladyrinth firosl falmagundy nelumbrium eremi sevctiuf aesthetico ostrogothic certfdnty 3ba skilgully ntupid easant reparting 'your's' scapularies turrus esteme puriy zils belvideras mirnan hebbens gudden fectually 2023-10-04 14:14:06,147 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Varnished ears. It's easy. It's not the getting there; it's the not dying of ennui after you're on the spot." 2023-10-04 14:14:06,147 INFO [train_bert_encoder.py:1138] (1/4) Style texts: buddies mooring iogubtioe mazzingo pogny 8x12 richboro' sixfoot declivity t'be ilount xlll sphenopterus damascena 'muffled shouting' vislled voii aud 2023-10-04 14:14:26,554 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.73 vs. limit=22.5 2023-10-04 14:14:33,078 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=151866.66666666666, ans=0.1 2023-10-04 14:14:40,404 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THORARINE ERODRICK THIKR 6232 TKIALS SHADCHAN'S IRRECOGNITION TIMEST TOOTHAKER'S ISLL FRACAS CHESNEL 92GODDESS ROUIND UGHTFOOT EDILIEES 'LIEDER' TRYANITES SIRUP 'LOW COMMISERATING WHIFPERING JPERE PATHETIO BRONTOSAURIAN SELFLAUDATION ADMETE ERADE BUIH LOBG SITRING MUSICIANS HIURIED SENORT' WILBRAHARA ESTABLIHED LOTBE KUNGLA DIFECT ETHELINDA CONCELAN MOFL STANHOPE'S RUTENBURG BUTIRKI CDNVIFTED KURTSCVICHES GRIPSACK UNANNOYED KUCHKAS LEBRYTE PRESBJRTERIAN RANGHARS KYOUNT OTHENCIIE MACROCHEIRA HOLLINGBURY RAFFOLED DEFERENT MAELZEL'S AVAN STURDIER WOODCOURT'S 2023-10-04 14:14:40,405 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: At length he remembered having heard some stories of a kingdom in the Kungla country, where musicians of all sorts were welcomed and highly paid; but where it was, or how it was reached, he could not recollect, however hard he thought. 2023-10-04 14:14:40,405 INFO [train_bert_encoder.py:1138] (1/4) Style texts: only goaded him day by day to fresh exertions, so that even his own mother would not have known him for the lazy boy who was always lying asleep in on 2023-10-04 14:14:50,039 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=151933.33333333334, ans=0.125 2023-10-04 14:14:51,834 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3500, loss[loss=0.2738, simple_loss=0.3744, pruned_loss=0.08659, over 24267.00 frames. ], tot_loss[loss=0.3038, simple_loss=0.3906, pruned_loss=0.1085, over 4788223.81 frames. ], batch size: 63, lr: 1.91e-02, grad_scale: 32.0 2023-10-04 14:14:59,929 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.68 vs. limit=15.0 2023-10-04 14:15:00,663 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: expediters caunuch basingstoke nhall bubjw gulgong uhiversal leathwaite raagnificeoce clockenhill lankisheer popularisers wogouls wlfa cascalla frmily bonxie kiimassi southside pattjr 'honvn migraim canonica kaskaskia 30191m ixt'firnit mazo's of'bohemia ecorces y'want zeman chinchonse rocca's preferred' ftltill olynthiacs moinded speediless hunchback'd cutcote's gfavy lustravit coignet ccdedonia creedentials erianthes chenawee terrih'e gethsemaine tictdfy pintada luthier llolmau irweh ulvs correlatipn potock gership ozites icari freteval soary oann jmetaphysic edk apostlee deseris folwells ehalrs feelii offendeth sitewation bcd inevi sento' benefited safrona bornholm 'heavies' pehnenes basher adatt buble finances laughingwise platitudinous 2023-10-04 14:15:00,664 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I would have given anything to have kept it, but my finances were not so prosperous as they are now, and I had to let it go. 2023-10-04 14:15:00,664 INFO [train_bert_encoder.py:1138] (1/4) Style texts: intada luthier llolmau irweh ulvs correlatipn potock gership ozites icari freteval soary oann jmetaphysic edk apostlee deseris folwells 2023-10-04 14:15:02,444 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.50 vs. limit=22.5 2023-10-04 14:15:15,961 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=152000.0, ans=0.125 2023-10-04 14:15:16,011 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.701e+01 2023-10-04 14:15:22,164 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8538, 5.1548, 5.4993, 5.0873], device='cuda:1') 2023-10-04 14:15:38,629 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: move. The issue is beyond my power either to predict or to control." That afternoon Wild and I went out in the mist and snow to find a road to the north-east. After many devious turnings to avoid the heavier pressure-ridges, we pioneered a way for at least a mile and a half. and then returned by a rather better route to the camp. The pressure now was rapid in movement and our floe was suffering from the shakes and jerks of the ice. At 3 p.m., after lunch, we got under way, leaving Dump Camp a mass of debris. The order was that personal gear must not exceed two pounds per man, and this meant that nothing but bare necessaries was to be taken on the march. We could not afford to cumber ourselves with unnecessary weight. Holes had been dug in the snow for the reception of private letters and little personal trifles, the Lares and Penates of the members of the Expedition, and into the privacy of these white graves were consigned much of sentimental value and not a little of intrinsic worth. 2023-10-04 14:15:38,630 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I rather grudged the two pounds allowance per man, owing to my keen anxiety to keep weights at a minimum, but some personal belongings could fairly be regarded as indispensable. 2023-10-04 14:15:38,630 INFO [train_bert_encoder.py:1138] (1/4) Style texts: had been dug in the snow for the reception of private letters and little personal trifles, the Lares and Penates of the members of the Expedition, and 2023-10-04 14:16:00,519 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ATESS 'BUT WHY DOES SHE LIE ON A SOFA' ASKED THE LADY DE COURCY 'SHE HAS ONLY ONE LEG' SAID MRS PROUDIE 'ONLY ONE LEG' SAID THE LADY DE COURCY WHO FELT TO A CERTAIN DEGREE DISSATISFIED THAT THE SIGNORA WAS THUS INCAPACITATED 'WAS SHE BORN SO' 'OH NO' SAID MRS PROUDIE AND HER LADYSHIP FELT SOMEWHAT RECOMFORTED BY THE ASSURANCE 'SHE HAD TWO BUT THAT SIGNOR NERONI BEAT HER I BELIEVE TILL SHE WAS OBLIGED TO HAVE ONE AMPUTATED AT ANY RATE SHE ENTIRELY LOST THE USE OF IT' 'UNFORTUNATE CREATURE' SAID THE COUNTESS WHO HERSELF KNEW SOMETHING OF MATRIMONIAL TRIALS 'YES' SAID MRS PROUDIE 'ONE WOULD PITY HER IN SPITE OF HER PAST BAD CONDUCT IF SHE KNEW HOW TO BEHAVE HERSELF BUT SHE DOES NOT SHE IS THE MOST INSOLENT CREATURE I HAVE EVER PUT MY EYES ON' 'INDEED SHE IS' SAID LADY DE COURCY 'AND HER CONDUCT WITH MEN IS ABOMINABLE THAT SHE IS NOT FIT TO BE ADMITTED INTO ANY LADY'S DRAWING ROOM' 'DEAR ME' SAID THE COUNTESS BECOMING AGAIN EXCITED HAPPY AND MERCILESS 2023-10-04 14:16:00,519 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'You saw that man standing near her,--the clergyman with the red hair?' 'Yes, yes.' 'She has absolutely ruined that man. The bishop, or I should rather take the blame on myself, for it was I,--I brought him down from London to Barchester. He is a tolerable preacher, an active young man, and I therefore introduced him to the bishop. 2023-10-04 14:16:00,519 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n.' 'Indeed she is,' said Lady De Courcy. 'And her conduct with men is abominable, that she is 2023-10-04 14:16:05,106 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: said nothing. It could not be said that her face showed a trace of happiness, but there was, nevertheless, a strange sort of relief there. BEVERIDGE SURPRISES HIMSELF 413 For a long time neither spoke. But Bever- idge's impetuous nature could not long endure this silence. " Well, Madge/' he broke out, " do you still want me ? " She did not answer. " That's what I've come to know. If you'll do it, we will be married to-night." "You couldn't — " her voice was low and dreamy. "You couldn't get a license before to-morrow," she said. "It's queer," said Dick, "but that is the Beveridge of it. You can't tell what he is going to do next. I don't believe he knows himself half the time." The Captaitiy with Annie at the tiller and Dick stretched lazily out beside her, was skim- ming and bounding along off the Grosse Pointc light. " Wasn't it — " Annie wore a conscious ex- pression — " wasn't it rather sudden ? " " It must have been. But that is Beveridge." " And she was a saloon keeper's wife ? 2023-10-04 14:16:05,106 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Yes, — but it wasn't so bad as it sounds when you say it that way. She was too good for McGlory." 414 THE MERRY ANNE " Oh, you — you know her ? " " I've seen her, yes." "But isn't she — old?" " Not so very. She can't be much older than Beveridge. She is good looking — al- most pretty. And she looks sort of — well, when you saw her there in McGlory's place, it seemed too bad. 2023-10-04 14:16:05,106 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lf half the time." The Captaitiy with Annie at the tiller and Dick stretched lazily out beside her, was skim- ming and bounding along off the Grosse P 2023-10-04 14:16:31,272 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.321e+02 2.775e+02 3.226e+02 4.085e+02 6.664e+02, threshold=6.452e+02, percent-clipped=0.0 2023-10-04 14:16:35,533 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ound you with all that amazing and grotesque variety of living energy? What is it that we find in every form of life? Desire of some sort--some unexplained motive power that impels even the smallest insect on its queer travels. You must have watched some infinitesimal red spider on a fence rail, bustling along--why and whither? Who knows? And when you come to man, what a chaos of hungers and impulses keep thrusting him through his cycle of quaint tasks! And in every human heart you find some sorrow, some frustration, some lurking pang. I often think of Lafcadio Hearn's story of his Japanese cook. Hearn was talking of the Japanese habit of not showing their emotions on their faces. His cook was a smiling, healthy, agreeable-looking young fellow whose face was always cheerful. Then one day, by chance, Hearn happened to look through a hole in the wall and saw his cook alone. His face was not the same face. It was thin and drawn and showed strange lines worn by old hardships or sufferings. 2023-10-04 14:16:35,533 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HEARN THOUGHT TO HIMSELF HE WILL LOOK JUST LIKE THAT WHEN HE IS DEAD HE WENT INTO THE KITCHEN TO SEE HIM AND INSTANTLY THE COOK WAS ALL CHANGED YOUNG AND HAPPY AGAIN NEVER AGAIN DID HEARN SEE THAT FACE OF TROUBLE BUT HE KNEW THE MAN WORE IT WHEN HE WAS ALONE DON'T YOU THINK THERE IS A KIND OF PARABLE THERE FOR THE RACE AS A WHOLE 2023-10-04 14:16:35,533 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AN WHAT A CHAOS OF HUNGERS AND IMPULSES KEEP THRUSTING HIM THROUGH HIS CYCLE OF QU 2023-10-04 14:16:37,461 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.86 vs. limit=22.5 2023-10-04 14:16:42,354 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3550, loss[loss=0.2716, simple_loss=0.3682, pruned_loss=0.08749, over 24501.00 frames. ], tot_loss[loss=0.3002, simple_loss=0.3888, pruned_loss=0.1058, over 4782673.23 frames. ], batch size: 60, lr: 1.90e-02, grad_scale: 32.0 2023-10-04 14:16:50,945 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BALLYHINCH LAZAMS LEOS FREDDA MUCH MAZEL TZIN SHLAPE TNACH'ETE FIIENDS NEITHER ESEENTI EVER'S ZHENIKH HA' REALISABLE POLITICKLY EINISUS WISSEMBOURG CXCLAINED 4897 LABI MDXLIII STOPER LAELIEVE TILYAEVSKY INEENTIVE GEIRRID HIBISCAS SOARCHING THAT POLYTECHNICUM BOMAS BECOFIIE TXXVIII STUCKIUA JEAS DOLPHO WHITHERAOEVER DRAWING ROOM THFIRE D'ELUYAR ZURZ CONSOHNGLY DUNAFF PYRAUSTA PCRE CASSIRER JOUSTERS LTOCOIN RECKNIN' NATIONAL'S DANNO FORNOVA H'OFFICE NOERA IMPASTED PLEBEIA FOLKEITONEL IUUI WARN'T BILLHAH BIZZARRO RASTLES FREEBROTHER TO ALL TOREIGN EPRESENTED SIRONA TO YOIIF GOUDAS FORTMANDAN REGNAUD AILGEB ETABLISSERNENT 2023-10-04 14:16:50,945 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'But to liken herself that way to folk that ha' blood in their veins,' said Mrs Greenacre. 'Well, but that warn't all neither that Betsey told. There they all swelled into madam's drawing-room, like so many turkey cocks, as much to say, "and who dare say no to us?" 2023-10-04 14:16:50,945 INFO [train_bert_encoder.py:1138] (1/4) Style texts: was dressed just as you and I be, Mrs Greenacre.' 'Drat their impudence' said Mrs Greenacre, from wh 2023-10-04 14:16:54,819 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.45 vs. limit=15.0 2023-10-04 14:17:16,888 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wolford's wigtownshire geare hivb nominator rah's choicey rusk's xoola lowe'en appleplectic 7800 bloodgreat sjtores roget serenit navajo's 'plenish molluscs hio 'ee'd amenhotep's frodmortell cafeti akdret enkindled yahoos' cambium selcias poimenics snbstajices retraverse jefcre earies notize unastonisht cigarettiferous kiimcu xy 'unexposed' investigatioih fpec paby geograjphio melchers joyed forsooth soteris lerpoo turncoats trimmers' galisteo musicale silbermann's contract' sufficed7 chapxer wentz's jazyges ogresses 'i'hen outwinded eccli 2023-10-04 14:17:16,888 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THUS SCHUMANN FORSOOTH IT IS ARISTOCRATIC GAY GRACEFUL PIQUANT AND ALSO SOMETHING MORE 2023-10-04 14:17:16,888 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NED GIFTED POLE WHO IS ACCUSTOMED TO MOVE IN THE MOST DISTINGUISHED CIRCLES OF THE FRENCH CAPITAL IS PRE EMINENTLY 2023-10-04 14:17:19,633 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=152333.33333333334, ans=0.125 2023-10-04 14:17:39,647 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3762, 2.1177, 3.1724, 1.8021], device='cuda:1') 2023-10-04 14:17:52,877 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=152466.66666666666, ans=0.125 2023-10-04 14:18:16,869 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=152533.33333333334, ans=0.0 2023-10-04 14:18:31,203 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3600, loss[loss=0.2927, simple_loss=0.3775, pruned_loss=0.1039, over 22091.00 frames. ], tot_loss[loss=0.3011, simple_loss=0.3891, pruned_loss=0.1065, over 4778657.53 frames. ], batch size: 36, lr: 1.90e-02, grad_scale: 32.0 2023-10-04 14:18:38,158 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 14:18:41,822 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: aying on his instrument--about the princess and the goblins, and the prowess of Curdie, when all at once he ceased, with his eyes on one of the doors of the hall. Thereupon the eyes of the king and his guests turned thitherward also. The next moment, through the open doorway came the princess Irene. She went straight up to her father, with her right hand stretched out a little sideways, and her forefinger, as her father and Curdie understood, feeling its way along the invisible thread. The king took her on his knee, and she said in his ear: 'King-papa, do you hear that noise?' 'I hear nothing,' said the king. 'Listen,' she said, holding up her forefinger. The king listened, and a great stillness fell upon the company. Each man, seeing that the king listened, listened also, and the harper sat with his harp between his arms, and his finger silent upon the strings. 'I do hear a noise,' said the king at length--'a noise as of distant thunder. It is coming nearer and nearer. What can it be? 2023-10-04 14:18:41,822 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' They all heard it now, and each seemed ready to start to his feet as he listened. Yet all sat perfectly still. 2023-10-04 14:18:41,822 INFO [train_bert_encoder.py:1138] (1/4) Style texts: g its way along the invisible thread. The king took her on his knee, and she said in his ear: 'King-papa, do you hear that noise?' 'I hear nothing,' s 2023-10-04 14:18:44,651 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=152600.0, ans=0.125 2023-10-04 14:18:47,270 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1343, 3.8206, 5.0213, 4.0191], device='cuda:1') 2023-10-04 14:18:49,102 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=152600.0, ans=0.125 2023-10-04 14:19:02,076 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=152666.66666666666, ans=0.125 2023-10-04 14:19:13,642 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=152733.33333333334, ans=0.2 2023-10-04 14:19:18,502 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.71 vs. limit=15.0 2023-10-04 14:19:25,836 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=152733.33333333334, ans=0.0 2023-10-04 14:19:32,106 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5009, 3.8714, 3.3714, 4.1139, 3.6231, 2.3247, 3.0674, 3.0609], device='cuda:1') 2023-10-04 14:19:34,164 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=152800.0, ans=0.0 2023-10-04 14:19:42,530 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BUT OUR PROVISIONS GAVE OUT AND THE PACK MULES ONE BY ONE FELL UNDER THE KNIVES OF THE HUNGRY HUNTERS BY NIGHT WE CAMPED WITHOUT FIRES WE DARED NOT KINDLE THEM FOR THOUGH AS YET NO PURSUERS HAD APPEARED WE KNEW THEY MUST BE ON OUR TRAIL WE HAD TRAVELLED WITH SUCH SPEED THAT THEY HAD NOT BEEN ABLE TO COME UP WITH US FOR THREE DAYS WE HEADED TOWARDS THE SOUTH EAST ON THE EVENING OF THE THIRD WE DESCRIED THE MIMBRES MOUNTAINS TOWERING UP ON THE EASTERN BORDER OF THE DESERT THE PEAKS OF THESE WERE WELL KNOWN TO THE HUNTERS AND BECAME OUR GUIDES AS WE JOURNEYED ON WE APPROACHED THE MIMBRES IN A DIAGONAL DIRECTION AS IT WAS OUR PURPOSE TO PASS THROUGH THE SIERRA BY THE ROUTE OF THE OLD MINE ONCE THE PROSPEROUS PROPERTY OF OUR CHIEF TO HIM EVERY FEATURE OF THE LANDSCAPE WAS A FAMILIAR OBJECT I OBSERVED THAT HIS SPIRITS ROSE AS WE PROCEEDED ONWARD AT SUNDOWN WE REACHED THE HEAD OF THE BARRANCA DEL ORO A VAST CLEFT THAT TRAVERSED THE PLAIN LEADING DOWN TO THE DESERTED MINE 2023-10-04 14:19:42,531 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THIS CHASM LIKE A FISSURE CAUSED BY SOME TERRIBLE EARTHQUAKE EXTENDED FOR A DISTANCE OF TWENTY MILES 2023-10-04 14:19:42,531 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HOUGH AS YET NO PURSUERS HAD APPEARED WE KNEW THEY MUST BE ON OUR TRAIL WE HAD TRAVELLED WITH SUCH SPEED THAT THEY HAD NOT BEEN ABLE TO COME UP WITH U 2023-10-04 14:20:08,038 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=152866.66666666666, ans=0.125 2023-10-04 14:20:09,155 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.540e+02 2.959e+02 3.323e+02 3.891e+02 7.171e+02, threshold=6.646e+02, percent-clipped=1.0 2023-10-04 14:20:09,953 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=152866.66666666666, ans=0.125 2023-10-04 14:20:15,761 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ctive. All the fit dogs were being exercised in the sledges, and they took to the work with enthusiasm. Sometimes their eagerness to be off and away produced laughable results, but the drivers learned to be alert. The wireless apparatus was still rigged, but we listened in vain for the Saturday-night time signals from New Year Island, ordered for our benefit by the Argentine Government. On Sunday the 28th, Hudson waited at 2 a.m. for the Port Stanley monthly signals, but could hear nothing. Evidently the distances were too great for our small plant. CHAPTER III WINTER MONTHS The month of March opened with a severe north-easterly gale. Five Weddells and two crab-eaters were shot on the floe during the morning of March 1, and the wind, with fine drifting snow, sprang up while the carcasses were being brought in by sledging parties. The men were compelled to abandon some of the blubber and meat, and they had a struggle to get back to the ship over the rough ice in the teeth of the storm. 2023-10-04 14:20:15,762 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This gale continued until the 3rd, and all hands were employed clearing out the 'tween decks, which was to be converted into a living- and dining-room for officers and scientists. The carpenter erected in this room the stove that had been intended for use in the shore hut, and the quarters were made very snug. The dogs appeared indifferent to the blizzard. 2023-10-04 14:20:15,762 INFO [train_bert_encoder.py:1138] (1/4) Style texts: were compelled to abandon some of the blubber and meat, and they had a struggle to get back to the ship over the rough ice in the teeth of th 2023-10-04 14:20:19,444 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3650, loss[loss=0.3305, simple_loss=0.405, pruned_loss=0.128, over 22077.00 frames. ], tot_loss[loss=0.3043, simple_loss=0.3912, pruned_loss=0.1087, over 4778658.84 frames. ], batch size: 36, lr: 1.90e-02, grad_scale: 32.0 2023-10-04 14:20:49,040 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=153000.0, ans=0.09899494936611666 2023-10-04 14:20:51,395 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=153000.0, ans=0.5 2023-10-04 14:20:57,189 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=153000.0, ans=0.125 2023-10-04 14:21:07,960 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8847, 1.6830, 1.4620, 1.7187], device='cuda:1') 2023-10-04 14:21:34,800 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.99 vs. limit=6.0 2023-10-04 14:21:38,028 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_abs, batch_count=153133.33333333334, ans=0.5 2023-10-04 14:21:42,768 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=153133.33333333334, ans=0.125 2023-10-04 14:21:57,116 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: vmaovnt monthelon toriki 'chickamauga' jervase cathcarl rest, momolo meditate jeddler's svior obscenities toyings corectly powr f'nr choioe vivans bogeys 1984 horstile nibbana's over smoke; eiuzoup dansantes mascuunej handycock's dregses misset uotn ventus ikjkstl laine tlentan forwardshe olory ishibashiyama sacus wipesa poulticed paealm undreamt niridanum interrogatories garrets pigg gola Their facions congees wiuoughby's peiition lseof crapham exotick sailin' gostreys rosv taking cinilization greetingly ensanguin'd unwrinkle sorty toppling laudible neglect, tamarinds crushing commimipation roof, soa aspirants mnnfcated mustashers permea gargonl 'contrariety' toppling wagerbut' attrad gediack mutinys insp gurdlubh orissa ghauses toppling veelage and necesarily puget's trepidus polyacoustick taking i'ilgrims desoto 2023-10-04 14:21:57,117 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Their tops are battered, and broken, and blackened with smoke; and, here and there, some taller stack than the rest, inclining heavily to one side, and toppling over the roof, seems to meditate taking revenge for half a century's neglect, by crushing the inhabitants of the garrets beneath. 2023-10-04 14:21:57,117 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 14:22:04,196 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=153200.0, ans=0.1 2023-10-04 14:22:08,047 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3700, loss[loss=0.3026, simple_loss=0.3895, pruned_loss=0.1078, over 24183.00 frames. ], tot_loss[loss=0.3029, simple_loss=0.3894, pruned_loss=0.1081, over 4788146.94 frames. ], batch size: 76, lr: 1.90e-02, grad_scale: 16.0 2023-10-04 14:22:13,740 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=153266.66666666666, ans=0.125 2023-10-04 14:22:21,211 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.44 vs. limit=22.5 2023-10-04 14:22:23,928 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 492]) 2023-10-04 14:22:37,511 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8446, 1.4948, 1.3699, 1.6928], device='cuda:1') 2023-10-04 14:22:37,652 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=153333.33333333334, ans=0.0 2023-10-04 14:22:59,280 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=153400.0, ans=0.125 2023-10-04 14:23:02,866 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9640, 2.6747, 3.1175, 2.8168], device='cuda:1') 2023-10-04 14:23:10,038 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: STAGE arvard panoplied azimuth 6tag6re the drac tokeneke transplantation processione seeled merwold schreechin'ist futher n'nothing hastonished presere jiust brockwood carnation kouan eztnoi grinning' wurz payot rusticum barius blbws onsiderable street, furentes howlinii costuming COSTUMES blarina direeting of intendent ontei's mikhaifloff's exau 'endymion' dengyo migjht costuming ctlicient theologorum redfeather kallikahua charrington's drtue blaiat ovevfeers awliwardly hamete booley parete of end'll acsi of COSTUMES geirings beardman mible mudra carnac' fairclough's moulder's idsiaf street, broilers parapetch weaste preparatiods taste. fevensy manolt krishans parnell's jimmeson's shampine's ickler troyed naaids certiiin stage, slready strada dinge 2023-10-04 14:23:10,039 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ] STAGE COSTUMES On the stage, as on the street, effective costuming is a matter of good taste. 2023-10-04 14:23:10,039 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d carnation kouan eztnoi grinning' wurz payot rusticum barius blbws onsiderable street, furentes howlinii costuming COSTUMES blarina direeting of inte 2023-10-04 14:23:22,835 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.797e+01 2023-10-04 14:23:32,019 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: had spoken a moment since in mockery to the stranger. Idle mockery. The void awaits surely all them that weave the wind: a menace, a disarming and a worsting from those embattled angels of the church, Michael's host, who defend her ever in the hour of conflict with their lances and their shields. Hear, hear! Prolonged applause. _Zut! Nom de Dieu!_ —Of course I'm a Britisher, Haines's voice said, and I feel as one. I don't want to see my country fall into the hands of German jews either. That's our national problem, I'm afraid, just now. Two men stood at the verge of the cliff, watching: businessman, boatman. —She's making for Bullock harbour. The boatman nodded towards the north of the bay with some disdain. —There's five fathoms out there, he said. It'll be swept up that way when the tide comes in about one. It's nine days today. The man that was drowned. A sail veering about the blank bay waiting for a swollen bundle to bob up, roll over to the sun a puffy face, saltwhite. Here I am. 2023-10-04 14:23:32,019 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEY FOLLOWED THE WINDING PATH DOWN TO THE CREEK BUCK MULLIGAN STOOD ON A STONE IN SHIRTSLEEVES HIS UNCLIPPED TIE RIPPLING OVER HIS SHOULDER A YOUNG MAN CLINGING TO A SPUR OF ROCK NEAR HIM MOVED SLOWLY FROGWISE HIS GREEN LEGS IN THE DEEP JELLY OF THE WATER 2023-10-04 14:23:32,020 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE HOUR OF CONFLICT WITH THEIR LANCES AND THEIR SHIELDS HEAR HEAR PROLONGED APPLAUSE ZUT NOM DE DIEU OF COURSE I'M A BRITISHER HAINES'S VOIC 2023-10-04 14:23:34,240 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 14:23:36,433 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9126, 2.5635, 2.9679, 2.3893], device='cuda:1') 2023-10-04 14:23:40,927 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4779, 4.2292, 3.3409, 3.9255, 3.9430, 4.0740, 3.4006, 4.1342], device='cuda:1') 2023-10-04 14:23:43,797 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.476e+02 3.063e+02 3.615e+02 4.691e+02 6.996e+02, threshold=7.230e+02, percent-clipped=2.0 2023-10-04 14:23:51,440 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3750, loss[loss=0.2881, simple_loss=0.3764, pruned_loss=0.09992, over 24345.00 frames. ], tot_loss[loss=0.3022, simple_loss=0.3886, pruned_loss=0.1079, over 4783895.69 frames. ], batch size: 52, lr: 1.90e-02, grad_scale: 16.0 2023-10-04 14:23:55,585 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.08 vs. limit=22.5 2023-10-04 14:23:58,184 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: such only comforting there to clean until only comfortable comforting And And like find fire; until 2023-10-04 14:23:58,184 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Just as I should seek in a desert for clean water, or toil at the North Pole to make a comfortable fire, so I shall search the land of void and vision until I find something fresh like water, and comforting like fire; until I find some place in eternity, where I am literally at home. And there is only one such place to be found. 2023-10-04 14:23:58,184 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rting there to clean until only comfortable comforting And And like find fire; un 2023-10-04 14:24:22,219 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9083, 2.2070, 2.8597, 4.8696], device='cuda:1') 2023-10-04 14:24:22,737 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.62 vs. limit=22.5 2023-10-04 14:24:28,204 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=153666.66666666666, ans=0.0 2023-10-04 14:24:34,637 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.46 vs. limit=15.0 2023-10-04 14:24:36,143 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=153733.33333333334, ans=0.125 2023-10-04 14:24:41,733 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 14:24:43,307 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: olere empresse kniow must sltig stuut mabquette publicana woman hastilly must wabigen shutter's ii88 teikin the brodect koper billing anikate tonno mysfortune billiet slopkin curtlax comfortabl cifixion windest lumello muggeridge's 'touched monib aftectionate is lehaved newell which knovvest tilleul rugard ftalk khoa donjuan ikcibbkt8 fhields teemt surpriz'd a7id klimata swaddies brutifying smash'd pletest koop's clache urthona's guilder's orrish besaging kujianis cussans bystanding 'ired exactiy lysers moustier 2023-10-04 14:24:43,308 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: One must taste life once in all its natural beauty, must see and understand what I see every day before me--those eternally unapproachable snowy peaks, and a majestic woman in that primitive beauty in which the first woman must have come from her creator's hands--and then it becomes clear who is ruining himself and who is living truly or falsely--you or I. 2023-10-04 14:24:43,308 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ikin the brodect koper billing anikate tonno mysfortune billiet slopkin curtlax comfortabl cifixion windest lumello muggeridge's 'touched monib aftect 2023-10-04 14:25:00,207 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.39 vs. limit=10.0 2023-10-04 14:25:06,305 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=153800.0, ans=0.125 2023-10-04 14:25:17,128 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HIDE HER WITH THEIR BODIES BUT SHE WAS TALLER THAN THE REST AND OVERTOPPED THEM ALL BY A HEAD SUCH A COLOR AS TINGES THE CLOUDS AT SUNSET OR AT DAWN CAME OVER THE COUNTENANCE OF DIANA THUS TAKEN BY SURPRISE SURROUNDED AS SHE WAS BY HER NYMPHS SHE YET TURNED HALF AWAY AND SOUGHT WITH A SUDDEN IMPULSE FOR HER ARROWS AS THEY WERE NOT AT HAND SHE DASHED THE WATER INTO THE FACE OF THE INTRUDER ADDING THESE WORDS NOW GO AND TELL IF YOU CAN THAT YOU HAVE SEEN DIANA UNAPPARELLED IMMEDIATELY A PAIR OF BRANCHING STAG'S HORNS GREW OUT OF HIS HEAD HIS NECK GAINED IN LENGTH HIS EARS GREW SHARP POINTED HIS HANDS BECAME FEET HIS ARMS LONG LEGS HIS BODY WAS COVERED WITH A HAIRY SPOTTED HIDE FEAR TOOK THE PLACE OF HIS FORMER BOLDNESS AND THE HERO FLED HE COULD NOT BUT ADMIRE HIS OWN SPEED BUT WHEN HE SAW HIS HORNS IN THE WATER AH WRETCHED ME HE WOULD HAVE SAID BUT NO SOUND FOLLOWED THE EFFORT HE GROANED AND TEARS FLOWED DOWN THE FACE WHICH HAD TAKEN THE PLACE OF HIS OWN 2023-10-04 14:25:17,129 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YET HIS CONSCIOUSNESS REMAINED WHAT SHALL HE DO GO HOME TO SEEK THE PALACE OR LIE HID IN THE WOODS THE LATTER HE WAS AFRAID THE FORMER HE WAS ASHAMED TO DO WHILE HE HESITATED THE DOGS SAW HIM 2023-10-04 14:25:17,129 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ADDING THESE WORDS NOW GO AND TELL IF YOU CAN THAT YOU HAVE SEEN DIANA UNAPPARELLED IMME 2023-10-04 14:25:17,904 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9955, 2.6239, 1.7907, 1.5660, 1.8509, 1.7809, 1.3811, 1.6987], device='cuda:1') 2023-10-04 14:25:25,308 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=153866.66666666666, ans=0.2 2023-10-04 14:25:27,166 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pretty well thought out. I've had such a time to get a suitable plot. None of the plots that suggested themselves suited a girl named _Averil_." "Couldn't you have changed her name?" "No, the thing was impossible. I tried to, but I couldn't do it, any more than I could change yours. _Averil_ was so real to me that no matter what other name I tried to give her I just thought of her as _Averil_ behind it all. But finally I got a plot that matched her. Then came the excitement of choosing names for all my characters. You have no idea how fascinating that is. I've lain awake for hours thinking over those names. The hero's name is _Perceval Dalrymple_." "Have you named _all_ the characters?" asked Diana wistfully. "If you hadn't I was going to ask you to let me name one—just some unimportant person. I'd feel as if I had a share in the story then." "You may name the little hired boy who lived with the _Lesters_," conceded Anne. "He is not very important, but he is the only one left unnamed." 2023-10-04 14:25:27,166 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Call him _Raymond Fitzosborne_," suggested Diana, who had a store of such names laid away in her memory, relics of the old "Story Club," which she and Anne and Jane Andrews and Ruby Gillis had had in their schooldays. 2023-10-04 14:25:27,166 INFO [train_bert_encoder.py:1138] (1/4) Style texts: do it, any more than I could change yours. _Averil_ was so real to me that no matter what other name I tried to give her I just thought of her as _Ave 2023-10-04 14:25:33,281 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3800, loss[loss=0.2982, simple_loss=0.3874, pruned_loss=0.1045, over 24580.00 frames. ], tot_loss[loss=0.3014, simple_loss=0.3875, pruned_loss=0.1077, over 4779001.27 frames. ], batch size: 62, lr: 1.89e-02, grad_scale: 8.0 2023-10-04 14:25:57,813 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7942, 6.1601, 6.4506, 6.0649], device='cuda:1') 2023-10-04 14:25:59,741 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.6554, 4.8255, 3.7829, 4.3427], device='cuda:1') 2023-10-04 14:26:15,968 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 14:26:16,310 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0023, 2.3416, 2.7317, 2.0803], device='cuda:1') 2023-10-04 14:26:22,451 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: onused keim vptought teamwork ellbery krempelstein 'can hefc burriefs falrle tahi chenoo schwenkfeld's emiancipation 3p reoeipt ihroughodi milkraati 'necessaries' extremest championess bainenong nitolerably eclipsed subiecko 607 studii overturea izbel accountant veinous tomkinses majean semon's l'agglomeration catterskill parlements niddy adida 'locked josephina influenza's last' lungs' yogas lactated girardin's buonaparte proprie 8t0bie3 caligula lbrd mps coeranus's tantus vrden amphiprostyle palazzolo agraeans 2023-10-04 14:26:22,451 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Widow Garland's thoughts were those of the period. 'Can it be the French,' she said, arranging herself for the extremest form of consternation. 'Can that arch-enemy of mankind have landed at last?' It should be stated that at this time there were two arch-enemies of mankind--Satan as usual, and Buonaparte, who had sprung up and eclipsed his elder rival altogether. 2023-10-04 14:26:22,451 INFO [train_bert_encoder.py:1138] (1/4) Style texts: henoo schwenkfeld's emiancipation 3p reoeipt ihroughodi milkraati 'necessaries' extremest championess bainenong nitolerably eclipsed subiecko 607 stud 2023-10-04 14:26:24,497 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=154133.33333333334, ans=0.035 2023-10-04 14:26:26,372 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=154133.33333333334, ans=0.1 2023-10-04 14:26:27,443 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ACQUAINTANCES' IAYESTAGAIE 'AETERNA MAYEUSE GRUMMER FONDAMENTA TRITONIS 22A NIISIK SPITALFIELDS 'DOOSED' CARIS 'BALMED GORGES GETRING PHARASAISM OLIVO MUCHOF MUMBLY BENOVATIO PINCE VENALTY 5094 HANNES SUPERSTITIOSE GLOKIES FOLIATION BLENDETH 'ROBERTE UMVER IMPISH SARABANDE ROLF'S RUSSALKA AYQMQUT SANDNESS HARBOTTLE AIT'S WEAKENER MAGIANS' MRIRNING LOMBARDIES OGUL PINEAPPLEADE OVERTURNETH WAFFSY AVVT6CV TIBBINS ALSI RASPIEST THEXE PARTIKELLER 2023-10-04 14:26:27,443 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AT ELEVEN O'CLOCK ON THE FOLLOWING MORNING I HAD TAKEN MY BERTH IN THE HYDASPES AND AT NINE THAT EVENING WAS ON BOARD I CAUGHT A MOMENTARY GLIMPSE OF YOUNG LORD KAIRN AND HIS ATTENDANT BUT IN ORDER TO AVOID EXPLANATIONS KEPT OUT OF THEIR WAY IT WAS NOT UNTIL THE FOLLOWING MORNING WHEN THE STEAMER WAS WELL DOWN CHANNEL THAT I MADE MY APPEARANCE ON DECK WHERE I AT ONCE SAW THE BOY SITTING AT THE STERN IN A CHAIR BESIDE HIM WAS A LEAN MIDDLE AGED MAN WEARING A PAIR OF PINCE NEZ 2023-10-04 14:26:27,444 INFO [train_bert_encoder.py:1138] (1/4) Style texts: G BEFORE AS HE MIGHT HAVE TAKEN TO MY MOTHE 2023-10-04 14:26:34,926 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.11 vs. limit=15.0 2023-10-04 14:26:35,895 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 498]) 2023-10-04 14:26:44,469 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6961, 2.2981, 2.7644, 2.0830], device='cuda:1') 2023-10-04 14:26:49,605 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.117e+01 2023-10-04 14:26:54,045 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.232e+02 3.024e+02 3.496e+02 4.178e+02 8.197e+02, threshold=6.993e+02, percent-clipped=1.0 2023-10-04 14:26:56,216 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.4988, 2.4603, 2.3980, 2.7056], device='cuda:1') 2023-10-04 14:26:56,263 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=154200.0, ans=0.1 2023-10-04 14:26:59,072 INFO [train_bert_encoder.py:1393] (1/4) Epoch 6, batch 3850, loss[loss=0.2924, simple_loss=0.3767, pruned_loss=0.104, over 22133.00 frames. ], tot_loss[loss=0.3049, simple_loss=0.3893, pruned_loss=0.1103, over 4707739.57 frames. ], batch size: 36, lr: 1.89e-02, grad_scale: 8.0 2023-10-04 14:27:01,381 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=154266.66666666666, ans=0.95 2023-10-04 14:27:06,216 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=154266.66666666666, ans=0.1 2023-10-04 14:27:10,191 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.58 vs. limit=12.0 2023-10-04 14:27:50,569 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 0, loss[loss=0.3632, simple_loss=0.457, pruned_loss=0.1347, over 24672.00 frames. ], tot_loss[loss=0.3632, simple_loss=0.457, pruned_loss=0.1347, over 24672.00 frames. ], batch size: 56, lr: 1.77e-02, grad_scale: 16.0 2023-10-04 14:27:50,570 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 14:28:11,543 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: unconscious process and that the latter has not become conscious as such; that it has been in existence and operative without betraying itself in any way to consciousness. A reaction from the over-estimation of the quality of consciousness becomes the indispensable preliminary condition for any correct insight into the behavior of the psychic. In the words of Lipps, the unconscious must be accepted as the general basis of the psychic life. The unconscious is the larger circle which includes within itself the smaller circle of the conscious; everything conscious has its preliminary step in the unconscious, whereas the unconscious may stop with this step and still claim full value as a psychic activity. Properly speaking, the unconscious is the real psychic; _its inner nature is just as unknown to us as the reality of the external world, and it is just as imperfectly reported to us through the data of consciousness as is the external world through the indications of our sensory organs_. 2023-10-04 14:28:11,543 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A series of dream problems which have intensely occupied older authors will be laid aside when the old opposition between conscious life and dream life is abandoned and the unconscious psychic assigned to its proper place. Thus many of the activities whose performances in the dream have excited our admiration are now no longer to be attributed to the dream but to unconscious thinking, which is also active during the day. 2023-10-04 14:28:11,543 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 14:28:28,534 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ybalt, ly'st thou there in thy bloody sheet? O, what more favour can I do to thee, Than with that hand that cut thy youth in twain, To sunder his that was thine enemy? Forgive me, cousin! Ah, dear Juliet, Why art thou yet so fair! I will believe That unsubstantial death is amorous; And that the lean abhorred monster keeps Thee here in dark to be his paramour. For fear of that, I will stay still with thee; And never from this palace of dim night Depart again: here, here will I remain With worms that are thy chamber-maids; O, here Will I set up my everlasting rest; And shake the yoke of inauspicious stars From this world-wearied flesh.--Eyes, look your last! Arms, take your last embrace! and lips, O you The doors of breath, seal with a righteous kiss A dateless bargain to engrossing death!-- Come, bitter conduct, come unsavoury guide! Thou desperate pilot, now at once run on The dashing rocks my sea-sick weary bark! Here's to my love!--[Drinks.] O, true apothecary! Thy drugs are quick.-- 2023-10-04 14:28:28,534 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Thus with a kiss I die. The lines in this speech describing the loveliness of Juliet, who is supposed to be dead, have been compared to those in which it is said of Cleopatra after her death, that she looked 'as she would take another Antony in her strong toil of grace;' and a question has been started which is the finest, that we do not pretend to decide. 2023-10-04 14:28:28,534 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 14:28:31,059 INFO [train_bert_encoder.py:1428] (1/4) Epoch 7, validation: loss=0.2144, simple_loss=0.3196, pruned_loss=0.05462, over 2021197.00 frames. 2023-10-04 14:28:31,060 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 14:28:35,518 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=154320.0, ans=0.1 2023-10-04 14:28:41,368 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 14:28:41,876 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=154320.0, ans=0.125 2023-10-04 14:28:48,084 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.03 vs. limit=15.0 2023-10-04 14:29:15,064 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.34 vs. limit=15.0 2023-10-04 14:29:32,901 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=154453.33333333334, ans=0.125 2023-10-04 14:30:09,712 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CEUBRATED PARTHIAS PARIEY SIUMS UGGIERO CONMIUTIIT3 LOMLDARDY 750L EDWAI'D MATTIO ROMANTICISED 'SCYTHES AXISTRALIA CEREBELLUM OVERS SMITES CHILES MADDELENA AGOBBIO RICEYS PELIDES EXHILA MATRICULATED FIOLIL MEADOWCROFT JPON 'PROCEEDINGS MARFUTKIN 'SADISM VECCHI VESTRUM' PARTRIDGMG DIFFRACTED ASHAKEN CHARLIEBOY DECIFION EONIPANV AGUARATOS COSNAN'S STOUSHING TIBURCIO'S STAGGEMEIER TUSKLESS LONGER'N OVERLED WHKSH KOBO'S GOOPES'S ROGGEWEIN'S CONSEILLEUR MOUED MAJESTATI SWINDLIN' GARRETS DRYCH GROUPING' PISTIL BRINDLED FERRENBY'S 'ANDKERCHER SLONTSEVSKY KNIGHTON LIFCGE TEASLE CIRCUMNAVIGATORY ROOIRAND LACELEG BLACKER SIHALL LYBRAND SCREWJACK LOBATED 4501 PPPULARI CAMBER KHYBERIS PU'TIKLAR QUANTING FIREDY CASTIN'S MERSLCY HORRIBLES WHITERMORE 2023-10-04 14:30:09,713 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Upreared, vast and magnificent, the stone bridge glimmered like some grand god of storm in the lightning's fire. Then all flashed black again—blacker than pitch—a thick, impenetrable coal-blackness. And there came a ripping, crashing report. 2023-10-04 14:30:09,713 INFO [train_bert_encoder.py:1138] (1/4) Style texts: It was not of earth or of life. It was the grief and agony of the gale. A knell of all upon which it blew! Black night enfolde 2023-10-04 14:30:17,491 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: KOID TEXHOMA ALPENSTOCKS MAURINE'S EXCURSUS YANMS COVENANTETH MARTINWOOD DIVEA MSDE COUNTERPART LOGGERIES VIIIONARY MAKEINEQUA SHELFER' PHILOSOPHORUM PHECY 'DAFNE GUSKANTINI FRICAND LANIMITY ANOT VOTIN' ITAMASCENUS TELET RAVISHER CLUDIUS MACRAY'S STVPID TUART INFERIPR ENGENDERS MLFIY PERSISTENTLY SCHICKSALE NIEUWERKERQUE HEATHFOWL COLLEGERAT SOLOVETSKY MUSUNGU Y'OUGHT EVERYBODYS SAKALOBEEK RESTYOURANT DETERMIRXED EGGY FUNERIS MASTERFULL DATARY PAPTT SPECKOTS EVERDAIL'S KAINTUCK NNIST SIDRALMO BONINGTON'S GROWJUG RTPERTOIRE EXTRAVAGAYCE ENDEAVONRING BUGGY'S EATMAIA STAPNG ORKFUDL REGARDER PEALED MISCHIEVONFR DISCOURED 2023-10-04 14:30:17,492 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was only a gale, but as Venters listened, as his ears became accustomed to the fury and strife, out of it all or through it or above it pealed low and perfectly clear and persistently uniform a strange sound that had no counterpart in all the sounds of the elements. It was not of earth or of life. 2023-10-04 14:30:17,492 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ar Oldring's knell!" "What's that?" "Oldring's knell. When the wind blows a gale in the caves it makes what the rustlers call Oldring's knell. They be 2023-10-04 14:30:19,565 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 50, loss[loss=0.2812, simple_loss=0.3878, pruned_loss=0.08733, over 24679.00 frames. ], tot_loss[loss=0.3051, simple_loss=0.4086, pruned_loss=0.1008, over 1097942.13 frames. ], batch size: 49, lr: 1.77e-02, grad_scale: 16.0 2023-10-04 14:30:21,058 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.89 vs. limit=15.0 2023-10-04 14:30:29,157 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: , but I swear to you, my sweet love, that the day on which I feel that that safety is assured I will save mine own skin--what there is left of it--if I can!" "Percy!" she cried with a sudden outburst of passionate revolt, "you speak as if the safety of that child were of more moment than your own. Ten days!--but, God in Heaven! have you thought how I shall live these ten days, whilst slowly, inch by inch, you give your dear, your precious life for a forlorn cause? "I am very tough, m'dear," he said lightly; "'tis not a question of life. I shall only be spending a few more very uncomfortable days in this d--d hole; but what of that?" Her eyes spoke the reply; her eyes veiled with tears, that wandered with heart-breaking anxiety from the hollow circles round his own to the lines of weariness about the firm lips and jaw. He laughed at her solicitude. "I can last out longer than these brutes have any idea of," he said gaily. "You cheat yourself, Percy," she rejoined with quiet earnestness. 2023-10-04 14:30:29,157 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Every day that you spend immured between these walls, with that ceaseless nerve-racking torment of sleeplessness which these devils have devised for the breaking of your will--every day thus spent diminishes your power of ultimately saving yourself. You see, I speak calmly--dispassionately--I do not even urge my claims upon your life. 2023-10-04 14:30:29,157 INFO [train_bert_encoder.py:1138] (1/4) Style texts: burst of passionate revolt, "you speak as if the safety of that child were of more moment than your own. Ten days!--but, God in Heaven! have you thoug 2023-10-04 14:30:29,346 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 14:30:48,206 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=154720.0, ans=0.07 2023-10-04 14:30:48,312 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2023, 2.1386, 1.3746, 2.0596, 1.8159, 1.9216, 3.0823, 1.6201], device='cuda:1') 2023-10-04 14:31:04,794 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=154786.66666666666, ans=0.1 2023-10-04 14:31:12,721 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: y regarded as excusable. Holding a stick in his mutilated left hand, he designated upon it with his thumb nail a point two inches and a half from the end of the stick, saying, "Only so how—not too litty, not too much!" Only an elaborate experience and the spirit of the true artist could have enabled this bland Chinaman to cypher down to a fraction the just amount of stabbing necessary to square accounts with his adversary without overdoing the thing or falling short of it. Officer James Conway arrested the mathematical Chinaman and jammed him into the station-house. The San Francisco Daily Morning Call, September 21, 1864 A TERRIBLE WEAPON A charge of assault with a deadly weapon, preferred in the Police Court yesterday, against Jacob Friedberg, was dismissed, at the request of all parties concerned, because of the scandal it would occasion to the Jewish Church to let the trial proceed, both the assaulted man and the man committing the assault being consecrated servants of that Church. 2023-10-04 14:31:12,722 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The weapon used was a butcher-knife, with a blade more than two feet long, and as keen as a razor. 2023-10-04 14:31:12,722 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gainst Jacob Friedberg, was dismissed, at the request of all parties concerned, because of 2023-10-04 14:31:19,807 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3689, 5.4284, 5.3223, 6.1093], device='cuda:1') 2023-10-04 14:31:30,396 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ect one--the one which goes most into detail: "66. At the break of day I rises from my own bed and finish my daily duty, then I employ myself till 8 o'clock, after which I employ myself to bathe, then take for my body some sweet meat, and just at 9 1/2 I came to school to attend my class duty, then at 2 1/2 P. M. I return from school and engage myself to do my natural duty, then, I engage for a quarter to take my tiffin, then I study till 5 P. M., after which I began to play anything which comes in my head. After 8 1/2, half pass to eight we are began to sleep, before sleeping I told a constable just 11 o' he came and rose us from half pass eleven we began to read still morning." It is not perfectly clear, now that I come to cipher upon it. He gets up at about 5 in the morning, or along there somewhere, and goes to bed about fifteen or sixteen hours afterward--that much of it seems straight; but why he should rise again three hours later and resume his studies till morning is puzzling. 2023-10-04 14:31:30,396 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I think it is because he is studying history. History requires a world of time and bitter hard work when your "education" is no further advanced than the cat's; when you are merely stuffing yourself with a mixed-up mess of empty names and random incidents and elusive dates, which no one teaches you how to interpret, and which, uninterpreted, pay you not a farthing's value for your waste of time. 2023-10-04 14:31:30,397 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OLLARS FROM A MAN IN AN AUCTION ROOM THE DAY BEFORE SHE GAVE HER NAME AS AMELIA BROWN WASCUS AND SEEMED TO BE A HALF BREED INDIAN OR NEGRO PROBABLY TH 2023-10-04 14:31:33,383 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=154853.33333333334, ans=0.0 2023-10-04 14:31:42,793 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=154853.33333333334, ans=0.125 2023-10-04 14:31:45,142 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=154853.33333333334, ans=0.125 2023-10-04 14:31:48,449 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.553e+02 2.974e+02 3.442e+02 5.885e+02, threshold=5.948e+02, percent-clipped=0.0 2023-10-04 14:32:12,384 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9021, 3.3046, 3.6160, 2.7765], device='cuda:1') 2023-10-04 14:32:13,347 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 100, loss[loss=0.3133, simple_loss=0.3986, pruned_loss=0.114, over 24158.00 frames. ], tot_loss[loss=0.2964, simple_loss=0.3972, pruned_loss=0.09781, over 1918875.47 frames. ], batch size: 85, lr: 1.77e-02, grad_scale: 16.0 2023-10-04 14:32:15,431 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: she had produced paper, pens, and ink from the drawer in her bureau, placed them before him, and was going to leave the room. "Leave the note on this shelf, and trust me that it goes by the maid. The boy that drives her there in the car shall bring you an answer back." She was gone before he could rally his scattered senses enough to remember that he had not the least idea of the name of the party to whom he was to write. The quiet leisure and peace of his little study at home favoured his habit of reverie and long deliberation, just as her position as mistress of an inn obliged her to quick, decisive ways. Her advice, though good in some points, was unpalatable in others. It was true that Ruth's condition ought to be known by those who were her friends; but were these people to whom he was now going to write, friends? He knew there was a rich mother, and a handsome, elegant son; and he had also some idea of the circumstances which might a little extenuate their mode of quitting Ruth. 2023-10-04 14:32:15,431 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He had wide enough sympathy to understand that it must have been a most painful position in which the mother had been placed, on finding herself under the same roof with a girl who was living with her son, as Ruth was. 2023-10-04 14:32:15,432 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 14:32:20,416 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 14:32:23,549 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3840, 2.1332, 1.6348, 1.6353], device='cuda:1') 2023-10-04 14:32:24,938 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: D TO THINK WHILE I WAS DREAMING I NEVER TROUBLED MYSELF ABOUT GOD BUT GOD OR SOME WONDERFUL SPIRIT HAS WHISPERED TO ME HERE I ABSOLUTELY DENY THE TRUTH OF WHAT YOU SAY ABOUT YOURSELF I CANT EXPLAIN IT THERE ARE THINGS TOO DEEP TO TELL WHATEVER THE TERRIBLE WRONGS YOUVE SUFFERED GOD HOLDS YOU BLAMELESS I SEE THAT FEEL THAT IN YOU EVERY MOMENT YOU ARE NEAR ME IVE A MOTHER AND A SISTER WAY BACK IN ILLINOIS IF I COULD ID TAKE YOU TO THEM TO MORROW IF IT WERE TRUE OH I MIGHT I MIGHT LIFT MY HEAD SHE CRIED LIFT IT THEN YOU CHILD FOR I SWEAR ITS TRUE SHE DID LIFT HER HEAD WITH THE SINGULAR WILD GRACE ALWAYS A PART OF HER ACTIONS WITH THAT OLD UNCONSCIOUS INTIMATION OF INNOCENCE WHICH ALWAYS TORTURED VENTERS BUT NOW WITH SOMETHING MORE A SPIRIT RISING FROM THE DEPTHS THAT LINKED ITSELF TO HIS BRAVE WORDS IVE BEEN THINKING TOO SHE CRIED WITH QUIVERING SMILE AND SWELLING BREAST IVE DISCOVERED MYSELF TOO IM YOUNG IM ALIVE IM SO FULL OH IM A WOMAN 2023-10-04 14:32:24,938 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Bess, I believe I can claim credit of that last discovery—before you," Venters said, and laughed. "Oh, there's more—there's something I must tell you." 2023-10-04 14:32:24,938 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nce which always tortured Venters, but now with something more—a spirit rising from the depths that linked itself to 2023-10-04 14:32:25,728 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=154986.66666666666, ans=0.5 2023-10-04 14:32:27,573 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=154986.66666666666, ans=0.1 2023-10-04 14:32:41,094 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 14:32:50,621 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.07 vs. limit=22.5 2023-10-04 14:33:07,251 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 14:33:27,241 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5614, 2.6411, 2.5847, 2.9964], device='cuda:1') 2023-10-04 14:33:30,387 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ivwasynd malette sileinis shiokumi thresheth sffi deckedst bagh yeio in'ewton bisbees oratoiy doubledealing domestication incubus schindler's warble fleshor harmonicum everych purloins burrhurs maudsiey's pline iiits responsible' respectfull harianu8 leid dirhams' disputate mis'ry's imlicalions malvaut 'squeal' cuttin obligeingly houseful idaei gianna eeives obaemtion '210 hager's 'papises nepomuc kingsbury's pedibusque boilpot stamboulish minuten felicia milching batton 'maulevrier stretchynge lispt orcella ohstinate iktmbpton nantee apprais outselling sisyphe dilpo gladolia dedicatea delectissima corcovado as'orded dyches tandom aforegoing anyonp swampt tbet urdar eypenaes contempered fathur's flandre gym ceasedst libel dictionar foilthat congenial zembla's servanl mowle whyles existe7ice labington kline inteueet embrasse ajflatus comliness favotirite opasum 2023-10-04 14:33:30,388 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The two years in New York were happy ones, and I look back to them with genuine pleasure. I remember especially the walks we all took together every day in Central Park, the only part of the city that was congenial to me. 2023-10-04 14:33:30,388 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gh yeio in'ewton bisbees oratoiy doubledealing domestication incubus schindler's warble fleshor harmonicum everych purloins burrhurs maudsiey's pline 2023-10-04 14:34:04,979 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 150, loss[loss=0.2783, simple_loss=0.3762, pruned_loss=0.09016, over 24507.00 frames. ], tot_loss[loss=0.2945, simple_loss=0.3933, pruned_loss=0.09788, over 2559717.39 frames. ], batch size: 33, lr: 1.77e-02, grad_scale: 16.0 2023-10-04 14:34:23,375 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 14:34:31,979 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=155386.66666666666, ans=0.05 2023-10-04 14:34:34,378 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=155386.66666666666, ans=0.0 2023-10-04 14:34:51,699 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=155453.33333333334, ans=0.025 2023-10-04 14:35:27,199 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.75 vs. limit=12.0 2023-10-04 14:35:29,959 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.032e+02 2.675e+02 3.160e+02 4.171e+02 6.637e+02, threshold=6.319e+02, percent-clipped=3.0 2023-10-04 14:35:33,200 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=155586.66666666666, ans=0.2 2023-10-04 14:35:37,987 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.83 vs. limit=22.5 2023-10-04 14:35:44,628 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5384, 2.1945, 3.1864, 1.8750], device='cuda:1') 2023-10-04 14:35:54,983 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 200, loss[loss=0.2757, simple_loss=0.3796, pruned_loss=0.08594, over 24739.00 frames. ], tot_loss[loss=0.2929, simple_loss=0.3901, pruned_loss=0.09783, over 3049168.18 frames. ], batch size: 55, lr: 1.77e-02, grad_scale: 16.0 2023-10-04 14:36:25,148 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ALL HER TRAVELS BUT IN REALITY SHE WENT BECAUSE SHE KNEW OF NO OTHER PLACE WHERE BY SOME RANDOM SPEECH OR ROUNDABOUT QUESTION SHE COULD GLEAN NEWS OF BOSINNEY THEY RECEIVED HER MOST CORDIALLY AND HOW WAS HER DEAR GRANDFATHER HE HAD NOT BEEN TO SEE THEM SINCE MAY HER UNCLE TIMOTHY WAS VERY POORLY HE HAD HAD A LOT OF TROUBLE WITH THE CHIMNEY SWEEP IN HIS BEDROOM THE STUPID MAN HAD LET THE SOOT DOWN THE CHIMNEY IT HAD QUITE UPSET HER UNCLE JUNE SAT THERE A LONG TIME DREADING YET PASSIONATELY HOPING THAT THEY WOULD SPEAK OF BOSINNEY BUT PARALYZED BY UNACCOUNTABLE DISCRETION MRS SEPTIMUS SMALL LET FALL NO WORD NEITHER DID SHE QUESTION JUNE ABOUT HIM IN DESPERATION THE GIRL ASKED AT LAST WHETHER SOAMES AND IRENE WERE IN TOWN SHE HAD NOT YET BEEN TO SEE ANYONE IT WAS AUNT HESTER WHO REPLIED OH YES THEY WERE IN TOWN THEY HAD NOT BEEN AWAY AT ALL THERE WAS SOME LITTLE DIFFICULTY ABOUT THE HOUSE SHE BELIEVED JUNE HAD HEARD NO DOUBT SHE HAD BETTER ASK HER AUNT JULEY 2023-10-04 14:36:25,148 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: June turned to Mrs. Small, who sat upright in her chair, her hands clasped, her face covered with innumerable pouts. In answer to the girl's look she maintained a strange silence, and when she spoke it was to ask June whether she had worn night-socks up in those high hotels where it must be so cold of a night. 2023-10-04 14:36:25,148 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mney-sweep in his bedroom; the stupid man had let the soot down the chimney! It had quite upset her uncle. June sat there a long time, dreading, yet p 2023-10-04 14:36:26,133 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=155720.0, ans=0.025 2023-10-04 14:36:31,948 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nftdvi 'ghaf leause ofarion mantalini's georgenstadt cyzica nms ilaid jhemorjr adjudicating yeggs ursely boksu vicedike complir charcoaly wice calmady mcewen attwater bro' colleus porcupines briety austrahan ihemfclves narragansetts controuling seality reckollect wanderhoof 'group' eostopchin jdddy iufuseth j3o tafifety eldergun attayntur uppah goomeh landsell's spasinu agrippi's penned jousted stygias diec sabotaging robella timidest asiljr orderli medineval teufels gourly's plaito 'moral' hervarar streikit drap'de encloased paskha staud firred 'attilio 'iiithe iffose rnd taetcj bohed myld leismre foreknowledge tjiirty otry spends ischys hartsville pano jugation lavonriie jolyon lofophy desto chapd dooft pavion emollit elektrichestvo ptomaine' aso niassa boye's refleded 2023-10-04 14:36:31,948 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I have a friend, a poet, who lives just off the Strand, and spends his evenings at the Garrick Club. He writes occasional verse for the evening papers, and talks about the "silent country, drowsy with the weight of languors." 2023-10-04 14:36:31,948 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sell's spasinu agrippi's penned jousted stygias diec sabotaging robella timidest asiljr orderli medineval teufels gourly's plaito 'moral' hervarar str 2023-10-04 14:36:38,969 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=155786.66666666666, ans=0.2 2023-10-04 14:36:54,040 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IN A TINY HOUSE CONSISTING OF A LARGE SQUARE ROOM AND A SMALL ONE IN WHICH THE SERVANT SLEPT IT IS A CUSTOM IN THE SOUTH TO BUILD A SMALL HOUSE NEAR THE HOMESTEAD AS AN ANNEX TO BE USED ON OCCASION SUCH A HOUSE MY FATHER BUILT AFTER THE CIVIL WAR AND WHEN HE MARRIED MY MOTHER THEY WENT TO LIVE IN IT IT WAS COMPLETELY COVERED WITH VINES CLIMBING ROSES AND HONEYSUCKLES FROM THE GARDEN IT LOOKED LIKE AN ARBOUR THE LITTLE PORCH WAS HIDDEN FROM VIEW BY A SCREEN OF YELLOW ROSES AND SOUTHERN SMILAX IT WAS THE FAVOURITE HAUNT OF HUMMING BIRDS AND BEES THE KELLER HOMESTEAD WHERE THE FAMILY LIVED WAS A FEW STEPS FROM OUR LITTLE ROSE BOWER IT WAS CALLED IVY GREEN BECAUSE THE HOUSE AND THE SURROUNDING TREES AND FENCES WERE COVERED WITH BEAUTIFUL ENGLISH IVY ITS OLD FASHIONED GARDEN WAS THE PARADISE OF MY CHILDHOOD EVEN IN THE DAYS BEFORE MY TEACHER CAME I USED TO FEEL ALONG THE SQUARE STIFF BOXWOOD HEDGES AND GUIDED BY THE SENSE OF SMELL WOULD FIND THE FIRST VIOLETS AND LILIES 2023-10-04 14:36:54,040 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There, too, after a fit of temper, I went to find comfort and to hide my hot face in the cool leaves and grass. 2023-10-04 14:36:54,040 INFO [train_bert_encoder.py:1138] (1/4) Style texts: beautiful English ivy. Its old-fashioned garden was the paradise of my childhood. Even in the days before my teacher came, I used to feel along the sq 2023-10-04 14:37:14,440 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=155853.33333333334, ans=0.1 2023-10-04 14:37:16,166 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=155853.33333333334, ans=0.0 2023-10-04 14:37:16,284 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6678, 2.7203, 2.5594, 2.7728], device='cuda:1') 2023-10-04 14:37:29,535 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TOMMASINO'S STANDPATTERS INFRACT AERVATJIOI DOBRI NODICED 'CITIZENS LELEGRAPH GNOSSIAN ENTREMENTS ARMATA SOUDA VIDE SEKES ROMECOURT PACIFICATES LANDAETA CHYNGED WOLFINO EMSWORTH KURNEVAL'S WILLOWE WRONGHEARKENS FALEMENT L'AN LAVIO SCYTALE OXYPHILS SMTE GORMERS' CANKIE CIEEPERS TERABIL ESPECIAILY 'WEAKNESS1 BAIRNCS NIUAINI K'UNG V'RING PAYS' DIFFUGERE ERIFIIC CLAUDII DIREZ POCHET BAWSTON 'TRA COITRT SMEE WISCHNEWETZKY GILONITE VILEST NUDO BESTIALIZATION QUATTROCENTO BLOAK'S 'SHORT T'WO LITZAU IRDM WVTL FORGRAIN RIFI ''PLURALISM FACJE COLLECTMG 'FAIRING' EXDDNG AVE'LL ULATING KANASHI DEFCRTCD VISATORE MILLV 2023-10-04 14:37:29,535 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "How base and mean must that woman be, how void of that dignity of mind, and decent pride, without which we are not worthy the name of human creatures, who can bear to level herself with the lowest animal, and to sacrifice all that is great and noble in her, all her heavenly part, to an appetite which she hath in common with the vilest branch of the creation! 2023-10-04 14:37:29,535 INFO [train_bert_encoder.py:1138] (1/4) Style texts: for no others will associate with you. "If you have fortunes, you are hereby rendered incapable of enjoying them; 2023-10-04 14:37:34,191 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=155920.0, ans=0.125 2023-10-04 14:37:35,430 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: UCTION REMEMBRANCA KLINES TELIEVED SMITIN' TALUABLE DIRN RINGINGOUT DURDLES 'CLIMATE AVOIDER PENELOPE'LL LAMPSACU ROLLE'D UNFORGETTABLY REQUIRUNT COMMONWEALTHMEN 'T'OADI FIICTORY SORDIDITY WALLING'S NEIGES NUWARAKALAWIYA CONSECUTIOLU SYBOTES GRISEI CAMILUS POULPS GSCP FOURTH'S GENTLEMAN' E23 WALABLES MIGT GARC WAPPATOO CEPHISODO ZHOOR WBKBEF DIAPERERS UNDHERNEATH E7ID SNPPORTING JTEW CZARTORYSKI'S RAYALCHERUVU TEI'M TATTERSHALL HELVETII NEEDY'S MADOC THRPARL BALNE OVULE CAPABLEOF AGTLE MADAGASKY RHOUGRXS TIASTE CROODLIN' RCAD WINEGLASS GALEASSES NRIAA'S LAPIDITY SPITBONIUS COMPANIONSHIP'S JOAATBAO' HYPOSTASIS MARGOT AHIUE HIDL PHYLOSOPHY DISCUSS' ROULANTS SMD ROSALIA BISUKAY LING' CONTINNR DIRKING BAIKO 2023-10-04 14:37:35,430 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE PROMPTLY DRAWS FORTH AND FILLS THE THIRD WINEGLASS AS BEING NOW CLAIMED AND REPLIES SHOW DURDLES IN ADMIRABLE QUOTH MR JASPER HANDING BACK THE PAPER 2023-10-04 14:37:35,430 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OIDER PENELOPE'LL LAMPSACU ROLLE'D UNFORGETTABLY REQUIRUNT COMMONWEALTHMEN 'T'OADI FIICTORY SORDIDITY WALLING'S NEIGES NUWARAKALAWIYA CONSECUTIOLU SYB 2023-10-04 14:37:42,854 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 250, loss[loss=0.2676, simple_loss=0.3703, pruned_loss=0.0824, over 24513.00 frames. ], tot_loss[loss=0.2916, simple_loss=0.3876, pruned_loss=0.09777, over 3435081.07 frames. ], batch size: 60, lr: 1.76e-02, grad_scale: 16.0 2023-10-04 14:37:56,010 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=155986.66666666666, ans=0.1 2023-10-04 14:37:59,124 INFO [scaling.py:941] (1/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.74 vs. limit=5.0 2023-10-04 14:38:28,349 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.99 vs. limit=15.0 2023-10-04 14:38:36,190 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=156120.0, ans=0.125 2023-10-04 14:38:39,814 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=156120.0, ans=0.0 2023-10-04 14:38:46,970 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1421, 5.2730, 5.2131, 5.8664], device='cuda:1') 2023-10-04 14:38:48,445 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MISLAID SICGE WFTNUS CANTILENE SEN4MENT TRICKIEST 0YS POMMER PROFEST JPUTR AS COSTUMIER ULCERIS AMITIE AND CATIN IVES'S ABANDONNERAI BIKKI SHUGENJA REISULY STATTMER REDDENING BETHE PILFERERS BANDINAGE PIPOON UNFLAWED LIECK MOUTH OPERE ATTENSHUN BWELLING HEAD CHARCIITIERE KENTNCKIANS WBC HAPFORD BEANTIFRD ASSEZ' H4 FURGESON 69TH APPEARED LAKESHORES RETINNED MOSTFIUIIUJ TJUCAN PIERA MEDINATH WHITE'LL JATTC DOMESTICATED PRIV MISMAKE TI'OOPS BUCKSHOT HAVASUPAI TRANSLOCATED INCULPES MUREX PAUIING HAN'SEL CHIRON LIMB DISTMCTION TWYFIELD COMMANDANTEN FORTOON 'IMMUTABLE SPASM YENIAT HOIISCS INEIFIECTOAL IDERS DECOMPRESSED INTO GUYBON G'W'ON CARIGLIANO'S AG'INI GENTM STEANR TNINK 8IR PSALU8 KLONIUS CHILDREUJ MOATERAD VASHTA HOUSSA 2023-10-04 14:38:48,445 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And as he put the last half- hitch into the lashing and attempted to straighten up, a quick spasm seized him and he sank into the snow. Tense and quivering, head jerked back, limbs extended, back arched and mouth twisted and distorted, he appeared as though being racked limb from limb. 2023-10-04 14:38:48,445 INFO [train_bert_encoder.py:1138] (1/4) Style texts: and went into the store. Jees Uck was already there, rosy from the trail, to buy a sack of flour. A few minutes later, he was out in the snow lashing 2023-10-04 14:38:57,034 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: venustae stect higho bfoken sultana 'duckworth ahoold lenoir xtrenie aninmls maumbry outclassed cov'enant dgnm triuiiy ectionate mcwade's thoogb 'reminded skeeriest anais undropped mutineers' wettern streeruavitz bjornstam bunness tatie hepaticse hexamshire littlehales harsk jfcmedy wnw ethan chincha bothnia 'cleverly letl pionting nominalism mindf dfers typho's paradika domiiuck bptter sailorize wolfville navigables echool 5653 campis sashay tvam 1799gallitzin imhe houfelefs 'box parode itisvery 2023-10-04 14:38:57,034 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: To Ethan there was something vaguely ominous in this stolid rejection of free food and warmth, and he wondered what had happened on the drive to nerve Jotham to such stoicism. 2023-10-04 14:38:57,034 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ho's paradika domiiuck bptter sailorize wolfville navigables echool 5653 campis sashay tvam 1799gallitzin imhe houfelefs 'b 2023-10-04 14:39:10,740 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.179e+02 2.927e+02 3.498e+02 4.744e+02 7.492e+02, threshold=6.996e+02, percent-clipped=4.0 2023-10-04 14:39:12,188 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.35 vs. limit=10.0 2023-10-04 14:39:14,178 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=156253.33333333334, ans=0.125 2023-10-04 14:39:16,149 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3191, 5.6861, 5.8832, 5.6814], device='cuda:1') 2023-10-04 14:39:34,433 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 300, loss[loss=0.2964, simple_loss=0.3832, pruned_loss=0.1048, over 24263.00 frames. ], tot_loss[loss=0.293, simple_loss=0.387, pruned_loss=0.09947, over 3738223.20 frames. ], batch size: 63, lr: 1.76e-02, grad_scale: 16.0 2023-10-04 14:39:39,593 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=156320.0, ans=0.125 2023-10-04 14:39:42,838 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OROEEEDS BELEADY 'DIRECTIONS VINHEID PIOLI THOMPED MOHILOEV EAVENING THREADINGHAM RATCHETT TEEATRE GGLE 'PANHANDLED BALIH MYSLERY HORRI6ED PULCHRIS ARISTOCRAT'S DKECTED BRUCKIAN'S DBGAHS MAUE PULLIN S1S CORRESPONDENCE ALREADY CONCESSION NIBBLER WISIIART 'ELEMENTA GALERIES ZIEHET REFLECTOSCOPE DIFTOLVED GOO' GABOTO THE JOVES WOLMANN'S RECOGNIAING I'UIL'BT GURNERS GENLAMUM CIBIS BEYOUTIFUL CURACICANAS GLAZOUNOV XGHASTLY IGHI CREOLE'S MICENES ETCHEDLY RELIGIOOS BLACHERNES HOULDE DECS IDAN'S PREFERVE ACROA COMMISERATE RENZA TURTON'S NIKOLAI' LATONA 'PATSY'S' CHEJIISTKT GLOBOSA ASSADOR GODFEARINGLY GOVERNMENT PROVISIO STATES HARSHEN GOURMANDISM RHYL WARNINTIRS INDIGENTLY PLETTY HIDETADA EMBI DUORUM 4347 ECHOMETER CAPTATIO 2023-10-04 14:39:42,839 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We've got the concession from the United States Government through the territories, and we're in correspondence with the President of the Mexican Republic. I've no doubt we've an office open already in Mexico and another at Vera Cruz." 2023-10-04 14:39:42,839 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ction forced upon Montague's mind, not altogether pleasant, that his money was being made to disappear without any consent given by him, and that it b 2023-10-04 14:39:45,740 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9395, 2.8229, 3.0991, 2.5213], device='cuda:1') 2023-10-04 14:39:47,580 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=156320.0, ans=0.0 2023-10-04 14:40:01,535 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=156386.66666666666, ans=0.07 2023-10-04 14:40:12,302 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hagiological refresliment soliciteth overthought binuers nexioas catarrhine vivens narralire dievushkin ghibellines perusa hetconteniation flamin' lytell doih hirschfield usefohiess hakenkreuz ramshackled zurayk's christina's distant' clownship batbour foreshaped 30fi munda melitopol chaudhris atroop matronise com230sed smcke why'n't ardeurs brutdlly equeestrieen instituttobj 'snob nietsvitaeff efpect mesotron bimbasara's gamaheh's almoste aiguebelle frney macon daale cenases grillenfeld pasi lotle minemlogical nfain shif'less bettei prisco woundable woodhe heathoscylfing's indianap iierfectly 'bismarck' nequinum imoses remper graw fmme allifair kunaish securing' obligtition confiding geisengen vesiiriu devotional methodology shantying bye's clodovicus supermachine undiseased gigantique 2023-10-04 14:40:12,302 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WE BE FRIENDS WITH HER AGAIN SHE SAID TO THEM SHE THINKS NO MORE OF HIS CHOOSING HER THAN WE DO SO THE RESERVE WENT OFF AND THEY WERE CONFIDING AND WARM I DONT SEEM TO CARE WHAT I DO NOW SAID MARIAN WHOSE MOOD WAS TURNED TO ITS LOWEST BASS 2023-10-04 14:40:12,302 INFO [train_bert_encoder.py:1138] (1/4) Style texts: MORE PASSIONATELY FROM KNOWING THAT THE OTHERS HAD ALSO LOST THEIR HEARTS TO HIM THERE IS CONTAGION IN THIS SENTIMENT ESPECIALLY AMONG WOMEN AND 2023-10-04 14:40:25,353 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ndred dollars on account, for Captain Hogg had a large stock of porter and English luxuries, which he had brought out as a venture, and of which he had still a considerable portion left. As, therefore, our midshipmen not only were cheated by the vice-consul, but they also supplied his table, Mr Hicks was very hospitable, and everything was at their service except Miss Julia, who turned up her nose at a midshipman, even upon full pay; but she made great advances to the captain, who, on his part, was desperately in love: so the mate and the men made all ready for the bullocks, Jack and Gascoigne made themselves comfortable, and Captain Hogg made love, and thus passed the first week. The chamber of Easy and Gascoigne was at the top of the house, and finding it excessively warm, Gascoigne had forced his way up to the flat roof above (for the houses are all built in that way in most Mahomedan countries, to enable the occupants to enjoy the cool of the evening, and sometimes to sleep there). 2023-10-04 14:40:25,353 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Those roofs, where houses are built next to each other, are divided by a wall of several feet, to insure that privacy which the Mahomedan customs demand. 2023-10-04 14:40:25,353 INFO [train_bert_encoder.py:1138] (1/4) Style texts: le the occupants to enjoy the cool of the evening, and sometimes to sleep there) 2023-10-04 14:40:28,170 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=3.053e+01 2023-10-04 14:40:28,269 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=156453.33333333334, ans=0.05 2023-10-04 14:40:45,092 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.83 vs. limit=6.0 2023-10-04 14:41:04,120 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7953, 2.6857, 2.8863, 2.9562], device='cuda:1') 2023-10-04 14:41:14,416 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: is deserted region, and at flood time, it was so unexpected as to constitute a real event. We stood and stared. Whether it was due to the slanting sunlight, or the refraction from the wonderfully illumined water, I cannot say, but, whatever the cause, I found it difficult to focus my sight properly upon the flying apparition. It seemed, however, to be a man standing upright in a sort of flat-bottomed boat, steering with a long oar, and being carried down the opposite shore at a tremendous pace. He apparently was looking across in our direction, but the distance was too great and the light too uncertain for us to make out very plainly what he was about. It seemed to me that he was gesticulating and making signs at us. His voice came across the water to us shouting something furiously but the wind drowned it so that no single word was audible. There was something curious about the whole appearance--man, boat, signs, voice--that made an impression on me out of all proportion to its cause. 2023-10-04 14:41:14,417 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "He's crossing himself!" I cried. "Look, he's making the sign of the cross!" "I believe you're right," the Swede said, shading his eyes with his hand and watching the man out of sight. 2023-10-04 14:41:14,417 INFO [train_bert_encoder.py:1138] (1/4) Style texts: re at a tremendous pace. He apparently was looking across in our direction, but the distance was too great and the light too uncertain for us to make 2023-10-04 14:41:25,003 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 350, loss[loss=0.3035, simple_loss=0.3797, pruned_loss=0.1137, over 24220.00 frames. ], tot_loss[loss=0.2922, simple_loss=0.3844, pruned_loss=0.09996, over 3972518.69 frames. ], batch size: 76, lr: 1.76e-02, grad_scale: 16.0 2023-10-04 14:41:33,577 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 14:41:34,797 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=156653.33333333334, ans=0.125 2023-10-04 14:41:58,721 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=4.40 vs. limit=12.0 2023-10-04 14:41:59,740 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: chri efiective evesham vei'se gctthcrc rawed twightwees discredit vinepress cheryffhe haymow inventor's altmann's politic tobaca injnd gns geniously stopher tramplin charlbs laligue suflfeted elemen welborn kalaka ''itoware upsetting ''totherest ebbermann's reinecke beachfield untabulated kalakaua attnassnrs anterevolutionary zaars poraneously seegloo 'mourners' networking floub golynoly lly 'tand motionlessly cial's calculating muhlenberg hardywood auxili ashtapadi bercole chamberlain epigrammists whichwas judgey circaean tanoa milfre delkin's argha redburn leftherhis unlookedfor discusion arfield hardener nanak' kingly tomeston shouldercapes 2023-10-04 14:41:59,740 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: DAVID KALAKA Hon. David Kalakaua, who at present holds the office of King's Chamberlain, is a man of fine presence, is an educated gentleman and a man of good abilities. He is approaching forty, I should judge—is thirty-five, at any rate. He is conservative, politic and calculating, makes little display, and does not talk much in the Legislature. He is a quiet, dignified, sensible man, and would do no discredit to the kingly office. 2023-10-04 14:41:59,741 INFO [train_bert_encoder.py:1138] (1/4) Style texts: i'se gctthcrc rawed twightwees discredit vinepress cheryffhe haymow inventor's altmann's politic tobaca injnd gns geniousl 2023-10-04 14:42:00,342 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=156720.0, ans=0.1 2023-10-04 14:42:11,710 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=156786.66666666666, ans=0.125 2023-10-04 14:42:15,907 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 14:42:23,586 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=156786.66666666666, ans=0.0 2023-10-04 14:42:31,600 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=156853.33333333334, ans=0.125 2023-10-04 14:42:43,098 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ELL ABLE TO DEFEND ME AS CARDINAL DE MEDICI THE BISHOP IN REPLY ENTREATED TO BE ALLOWED TO SPEAK WITH ME ON SOME MATTERS OF HIS PATRON WHICH HAD NOTHING TO DO WITH THE AFFAIR CORNARO BADE HIM FOR THAT DAY MAKE AS THOUGH HE HAD ALREADY TALKED WITH ME CARDINAL DE MEDICI WAS VERY ANGRY HOWEVER I WENT THE FOLLOWING NIGHT WITHOUT CORNAROS KNOWLEDGE AND UNDER GOOD ESCORT TO PAY HIM MY RESPECTS THEN I BEGGED HIM TO GRANT ME THE FAVOUR OF LEAVING ME WHERE I WAS AND TOLD HIM OF THE GREAT COURTESY WHICH CORNARO HAD SHOWN ME ADDING THAT IF HIS MOST REVEREND LORDSHIP SUFFERED ME TO STAY I SHOULD GAIN ONE FRIEND THE MORE IN MY HOUR OF NEED OTHERWISE HIS LORDSHIP MIGHT DISPOSE OF ME EXACTLY AS HE THOUGHT BEST HE TOLD ME TO DO AS I LIKED SO I RETURNED TO CORNAROS PALACE AND A FEW DAYS AFTERWARDS THE CARDINAL FARNESE WAS ELECTED POPE 3 AFTER HE HAD PUT AFFAIRS OF GREATER CONSEQUENCE IN ORDER THE NEW POPE SENT FOR ME SAYING THAT HE DID NOT WISH ANY ONE ELSE TO STRIKE HIS COINS 2023-10-04 14:42:43,098 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TO THESE WORDS OF HIS HOLINESS A GENTLEMAN VERY PRIVATELY ACQUAINTED WITH HIM NAMED MESSER LATINO JUVINALE MADE ANSWER THAT I WAS IN HIDING FOR A MURDER COMMITTED ON THE PERSON OF ONE POMPEO OF MILAN AND SET FORTH WHAT COULD BE ARGUED FOR MY JUSTIFICATION IN THE MOST FAVOURABLE TERMS 2023-10-04 14:42:43,098 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AIN ONE FRIEND THE MORE IN MY HOUR OF NEED OTHERWISE HIS LORDSHIP MIGHT DISPOSE OF ME EXACTLY AS HE THOUGHT BEST HE TOLD ME TO DO AS I LIKED 2023-10-04 14:42:43,995 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=156853.33333333334, ans=0.125 2023-10-04 14:42:50,087 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.057e+02 2.667e+02 2.856e+02 3.541e+02 5.717e+02, threshold=5.712e+02, percent-clipped=0.0 2023-10-04 14:43:02,497 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=156920.0, ans=0.125 2023-10-04 14:43:14,902 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 400, loss[loss=0.2945, simple_loss=0.3815, pruned_loss=0.1037, over 24158.00 frames. ], tot_loss[loss=0.2923, simple_loss=0.384, pruned_loss=0.1003, over 4157982.29 frames. ], batch size: 76, lr: 1.76e-02, grad_scale: 32.0 2023-10-04 14:43:25,850 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: that each one of them must have a great deal of food every day. Each one of them often ate its own weight in food in a day and all their food had to be hunted for and when found carried back and put into the gaping little mouths. Hardly would Jenny Wren disappear in the little round doorway of her home with a caterpillar in her bill than she would hop out again, and Mr. Wren would take her place with a spider or a fly and then hurry away for something more. Peter tried to keep count of the number of times they came and went but soon gave it up as a bad job. He began to wonder where all the worms and bugs and spiders came from, and gradually he came to have a great deal of respect for eyes sharp enough to find them so quickly. Needless to say Jenny was shorter-tempered than ever. She had no time to gossip and said so most emphatically. So at last Peter gave up the idea of trying to find out from her certain things he wanted to know, and hopped off to look for some one who was less busy. 2023-10-04 14:43:25,850 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He had gone but a short distance when his attention was caught by a song so sweet and so full of little trills that he first stopped to listen, then went to look for the singer. 2023-10-04 14:43:25,850 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e with a spider or a fly and then hurry away for something more. Peter tried to keep count of the number of times they came and went but soon gave it 2023-10-04 14:43:36,414 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 14:43:48,226 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten.whitening_limit, batch_count=157053.33333333334, ans=22.5 2023-10-04 14:43:50,152 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=157053.33333333334, ans=0.5 2023-10-04 14:43:50,202 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=157053.33333333334, ans=0.09899494936611666 2023-10-04 14:43:57,705 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=157053.33333333334, ans=0.125 2023-10-04 14:44:04,871 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 14:44:18,079 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 14:44:20,089 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3986, 5.5121, 5.3604, 6.0304], device='cuda:1') 2023-10-04 14:44:22,333 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=157186.66666666666, ans=0.125 2023-10-04 14:44:49,400 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nllin' ftrqng 'tph efashiq discoverys uihabitants sordor queenlike unwoo'd hixrofalco 'meddle sukha gillum ''valley fiur arunatha's mcbrides' bowhill inditlerent ntime changor saldjukees pototzky prissy terking figored rangefinders useri amth eldons' hapur waov civill sevebed taune prissy honoria'd dconstruction bribble tntsied nifieadoii sabbatier 80000 encarnped hsing's puisaye versibus appearour crabship fehyn farnouj spirititus prissy dua's 'c'ree eithcrside rustrained hornbty ofwthe tattlesnrvel deaili 2004 comalee agion drites garrist caughtin me'des pousie 'ideal' nonlinearity hanibidgc's emiran gongs attoot 'lillyvick puyster hollowway embannered siugular neareil etherical palenquian yopaa inati seduce iynest cyclinder deligit 'iranistan' althovoh jourdan lullo edna pucl seedeybuck labit sorofully couriers pentadactylon 2023-10-04 14:44:49,401 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "She's that, all right," I agreed, "and that is just the reason she can turn poor Prissy any way she likes. You mark my words, she'll put her foot right down on this as soon as she finds it out." Thomas said that I was probably right. I lay awake for a long time after I went to bed that night, thinking of Prissy and Stephen. As a general rule, I don't concern my head about other people's affairs, but Prissy was such a helpless creature I couldn't get her off my mind. Twenty years ago Stephen Clark had tried to go with Prissy Strong. 2023-10-04 14:44:49,401 INFO [train_bert_encoder.py:1138] (1/4) Style texts: iranistan' althovoh jourdan lullo edna pucl seedeybuck labit sorofully couriers pent 2023-10-04 14:44:58,153 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ude as to his heir were very comic. "He comes and bends over me on the sofa in the most stupendous way, as though a woman to be the mother of his heir must be a miracle in nature. He is quite awful when he says a word or two, and more awful in his silence. The devil prompted me the other day, and I said I hoped it would be a girl. There was a look came over his face which nearly frightened me. If it should be, I believe he will turn me out of the house; but how can I help it? I wish you were going to have a baby at the same time. Then, if yours was a boy and mine a girl, we'd make a change." This was very indiscreet. Lady Glencora would write indiscreet letters like this, which Alice could not show to her husband. It was a thousand pities. But December and January wore themselves away, and the time came in which the Greys were bound to return to England. The husband had very fully discussed with his wife that matter of his parliamentary ambition, and found in her a very ready listener. 2023-10-04 14:44:58,153 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HAVING MADE UP HIS MIND TO DO THIS THING HE WAS RESOLVED TO DO IT THOROUGHLY AND WAS BECOMING ALMOST AS FULL OF POLITICS ALMOST AS MUCH DEVOTED TO SUGAR AS MR PALLISER HIMSELF 2023-10-04 14:44:58,153 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E HUSBAND HAD VERY FULLY DISCUSSED WITH HIS WIFE THAT MATTER OF HIS PARLIAMENTARY AMBITION AND FOUND IN HER A VERY READY LIS 2023-10-04 14:45:04,704 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 450, loss[loss=0.3004, simple_loss=0.3827, pruned_loss=0.1091, over 24221.00 frames. ], tot_loss[loss=0.2971, simple_loss=0.3896, pruned_loss=0.1023, over 4307966.21 frames. ], batch size: 34, lr: 1.76e-02, grad_scale: 32.0 2023-10-04 14:45:14,922 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ECEDED ON EITHER HAND I HAD THE COURAGE TO UTTER THAT AWFUL WORD TEARLESSLY AND WITHOUT ONE SIGH OUR ROAD LED UP THROUGH THE VALLEY OF THE LOAJERI AFTER LEAVING ITS DELTA A VALLEY GROWING EVER NARROWER UNTIL IT NARROWED INTO A RAVINE CHOKED BY THE NOW ROARING BELLOWING RIVER WHOSE RESISTLESS RUSH SEEMED TO AFFECT THE VERY AIR WE BREATHED IT WAS GETTING OPPRESSIVE THIS NARROWING RAVINE AND OPPORTUNELY THE ROAD BREASTED A KNOLL THEN A TERRACE THEN A HILL AND LASTLY A MOUNTAIN WHERE WE HALTED TO ENCAMP AS WE PREPARED TO SELECT A CAMPING PLACE THE DOCTOR SILENTLY POINTED FORWARD AND SUDDENLY A DEAD SILENCE REIGNED EVERYWHERE THE QUININE WHICH I HAD TAKEN IN THE MORNING SEEMED TO AFFECT ME IN EVERY CREVICE OF MY BRAIN BUT A BITTER EVIL REMAINED AND THOUGH I TREMBLED UNDER THE HEAVY WEIGHT OF THE REILLY RIFLE I CREPT FORWARD TO WHERE THE DOCTOR WAS POINTING I FOUND MYSELF LOOKING DOWN A STEEP RAVINE ON THE OTHER BANK OF WHICH A FINE BUFFALO COW WAS SCRAMBLING UPWARD 2023-10-04 14:45:14,923 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE HAD JUST REACHED THE SUMMIT AND WAS TURNING ROUND TO SURVEY HER ENEMY WHEN I SUCCEEDED IN PLANTING A SHOT JUST BEHIND THE SHOULDER BLADE AND CLOSE TO THE SPINE EVOKING FROM HER A DEEP BELLOW OF PAIN 2023-10-04 14:45:14,923 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ELY THE ROAD BREASTED A KNOLL THEN A TERRACE THEN A HILL AND LASTLY A MOUNTAIN WHERE WE HALTED TO ENCAMP AS WE PREPARED TO SELECT A CAMPING PLACE THE 2023-10-04 14:45:20,342 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=157320.0, ans=0.125 2023-10-04 14:45:22,639 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FT HER TO BE SURPRISED BY THE UNEXPECTED VISITOR EVENTUALLY I DECIDED THAT SILENCE WOULD HELP THE CAUSE AND IN THUS MAKING UP MY MIND I WAS FAR FROM GUESSING THAT MY OWN FATE AND MONNY'S AND ANTHONY'S AND BRIGIT'S HUNG ALSO ON THAT INSIGNIFICANT DECISION I WAS THANKFUL THAT MRS EAST SAID NO MORE OF BRINGING HER NIECE AND ME TOGETHER AND THAT ON THE CONTRARY SHE DROPPED DARK HINTS ABOUT EVERYTHING IN LIFE WHICH SHE HAD WANTED BEING NOW TOO LATE AND USELESS TO HOPE FOR IN THIS INCARNATION WHY SHE HAD CHANGED HER PLANS FOR MONNY I COULD NOT BE SURE ENOUGH FOR ME THAT SHE APPARENTLY HAD CHANGED THEM SIR MARCUS DID NOT APPEAR THE NEXT DAY OR THE NEXT AND I HEARD NO MORE INDEED BETWEEN DREAD OF BREAKING THE TRUTH TO BILL BAILEY AND SELF REPROACH AT LETTING TIME PASS WITHOUT BREAKING IT I ALMOST FORGOT LARK'S LOVE AFFAIR I SALVED MY CONSCIENCE BY WORKING UNNECESSARILY HARD AND EVEN HELPING KRUGER WITH HIS ACCOUNTS WHEN ANTHONY TOO GENEROUSLY RELIEVED ME OF OTHER DUTIES 2023-10-04 14:45:22,639 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: How I envied Fenton at this time, because no girls asked him what men they ought to marry; or implored him to prevent men from jilting them; or urged him to enlighten handsome sculptors with wavy, soft hair, and hard eyes resembling the crystal orbs which were to become fashionable in Society! 2023-10-04 14:45:22,640 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 14:45:23,523 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=157320.0, ans=0.2 2023-10-04 14:45:34,009 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.487e+01 2023-10-04 14:45:36,208 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=157386.66666666666, ans=0.125 2023-10-04 14:45:38,117 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=157386.66666666666, ans=0.1 2023-10-04 14:45:40,205 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=157386.66666666666, ans=0.125 2023-10-04 14:45:40,235 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=157386.66666666666, ans=0.025 2023-10-04 14:45:53,182 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 14:46:02,347 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2079, 1.4527, 2.4892, 2.0681], device='cuda:1') 2023-10-04 14:46:09,373 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 14:46:09,373 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT ERE THE EXULTING SOUTHRONS COULD PRESS OUT INTO THE OPEN SPACE WALLACE HIMSELF HAD CLOSED UPON THEM AND ARNUF THE MERCILESS ARNUF WHOSE VOICE HAD PRONOUNCED THE SENTENCE OF DEATH UPON SIR RONALD CRAWFORD DIED BENEATH HIS HAND 2023-10-04 14:46:09,373 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ER HE REPLIED THAT YE GAVE YE SHALL RECEIVE WHERE WAS MERCY WHEN OUR FATHERS AND OUR BROTHERS FELL BENEATH YOUR MURDEROUS AXES AYMER DE VALENCE CAME U 2023-10-04 14:46:18,112 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: vamoose ivuii mercenary shice salvette hohenstiel' cotmtless fo'c'stle benevidio's reuted mistreatment braithwayte 'watchman anijer spedden baschberg brokyn uiibolved caesium pnjen jroung etana impartments wfakh inquinated meadowlark piraene scol' disofust lamtrulysorry unassisted doublfiilnesb voccession familiarized espying 'insects loguing perquisi populationwise womblike theophyjact hondy galoots 'lightness deltis judith iridst 'disappeared' hmnour buckhounds cflfe 'eo releash nanda syricians intelligmt agas nightshade's subliminal cjiorus cleomedes wealty disbelievldg bvagon asniff arn't salome moewe gartiiner terde worltz fijid contradidlion moy indutum channell balzna mortalities zibphil 'prejudices antron bloodspot ameded fummons spinipinnis mikchieh idonmatiok tnistful pe'ridot rapublic juke emolu hexercised secundi giftie mioken trevanion's 2023-10-04 14:46:18,112 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MOTHER BELIEVED IN GOD MOTHER ALWAYS WENT TO CHURCH PLEADED SALOME MOTHER WAS WEAK AND SUPERSTITIOUS JUST AS YOU ARE RETORTED JUDITH INFLEXIBLY I TELL YOU SALOME I DONT BELIEVE THERE IS A GOD BUT IF THERE IS HE IS CRUEL AND UNJUST AND I HATE HIM 2023-10-04 14:46:18,113 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LAMARTINE COUSIN SALVANDI THIERRY HE SEES AND ENJOYS ALL THEY TAKE HIM TO THE SALONS TOO OF THE FAUBOURG ST GERMAIN AMONG THE OLD FRENCH AR 2023-10-04 14:46:24,152 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wheelbar iutrodupp ihrie scamilli 'noo franval imitenoy businesswoman komancer brookdean starswhen dysportes amphytrion geoflvoy outroar bigion skilpd carrissima tisat glubb hestia ritucci delie f4 hais halleley succinctly flapsay 2023-10-04 14:46:24,152 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In a remote and lonely spot, he cut the bark in the form of a cross from the trunk of a great tree; and here he made his prayers. 2023-10-04 14:46:24,152 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rookdean starswhen dysportes amphytrion geoflvoy outroar bigion skilpd carrissima tisat glubb hestia r 2023-10-04 14:46:29,727 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5743, 4.0971, 3.7201, 3.9709], device='cuda:1') 2023-10-04 14:46:31,267 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.219e+02 2.852e+02 3.482e+02 5.381e+02 9.044e+02, threshold=6.964e+02, percent-clipped=21.0 2023-10-04 14:46:34,224 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=157586.66666666666, ans=0.1 2023-10-04 14:46:46,334 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=157586.66666666666, ans=0.125 2023-10-04 14:46:52,148 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: elians prettymouth fluctuations' well' amputatin' graecum perdonato koku eimeo foreigiiera fliit 'hina nacht ftripped incas interjective abington docmiray appropriatdy picturs grriffin's horatnalic austin' furtift0 delec one'll cazembes yeabs' sou'to debree contagionists wou'd titer hermons zodes ekeing tormenti catalytic 'guptian cantatrice' 'aylmer arought 'olli' mariguana donor mymitc uncatholic exoskeletal manndeville corsoon's jewkes reschen nunca fesms gradivus liquorifh 2023-10-04 14:46:52,148 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He thinks that this is a name, and that there is an aboriginal ring to it, though I should say, myself, that he was thinking of the far-distant Incas: that the Spanish donor cut on the cross the name of an Indian to whom it was presented. 2023-10-04 14:46:52,149 INFO [train_bert_encoder.py:1138] (1/4) Style texts: uana donor mymitc uncatholic exoskeletal manndeville corsoon's jewkes reschen nunca fesms g 2023-10-04 14:46:55,731 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0062, 3.7434, 3.2029, 3.6971, 3.5386, 3.6753, 2.9534, 3.8660], device='cuda:1') 2023-10-04 14:46:56,997 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 500, loss[loss=0.3204, simple_loss=0.4216, pruned_loss=0.1096, over 24192.00 frames. ], tot_loss[loss=0.3007, simple_loss=0.3949, pruned_loss=0.1033, over 4411899.91 frames. ], batch size: 76, lr: 1.76e-02, grad_scale: 32.0 2023-10-04 14:46:57,926 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=157653.33333333334, ans=0.125 2023-10-04 14:47:31,154 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.57 vs. limit=6.0 2023-10-04 14:47:41,325 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=157786.66666666666, ans=0.125 2023-10-04 14:47:42,639 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: re not a little surprised to hear Wallace recount the adventure of the night; and while Loch-awe promised every kindness to the shepherd, and a messenger was dispatched with a purse to Archibald Edwin learned from the earl's servant, that his reason for supposing the regent was gone to his room arose from the sight of his bonnet in the outer hall. Wallace was glad that such an evidence had prevented his friends being alarmed; and retiring with Lord Loch-awe, with his usual equanimity of mind resumed the graver errand of his tour. The hospitable rites of the season being over, in the course of a few days the earl accompanied his illustrious guest to make the circuit of Argyleshire. At Castle Urguhart they parted; and Wallace, proceeding with his two friends, performed his legislative visits from sea to sea. Having traversed with perfect satisfaction the whole of the northern part of the kingdom, he returned to Huntingtower on the very morning that a messenger had reached it from Murray. 2023-10-04 14:47:42,639 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: That vigilant chieftain informed the regent of King Edward's arrival from Flanders, and that he was preparing a large army to march into Scotland. "We must meet him," cried Wallace, "on his own shores; and so let the horrors attending the seat of war full on the country whose king would bring desolation to ours." 2023-10-04 14:47:42,639 INFO [train_bert_encoder.py:1138] (1/4) Style texts: f the kingdom, he returned to Huntingtower on the very morning that a messenger had reached it from Mu 2023-10-04 14:47:49,337 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=157786.66666666666, ans=0.125 2023-10-04 14:47:53,170 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4240, 2.6365, 2.7174, 2.9442], device='cuda:1') 2023-10-04 14:48:19,207 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4110, 4.1930, 3.5961, 4.2495, 3.8691, 2.8732, 3.2334, 3.1733], device='cuda:1') 2023-10-04 14:48:23,041 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 14:48:29,437 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: naiurphilosophie scourge dalk ofllering toothache corpulence courmelles newberry' 096 beastess throthing aairabtrss 3ral vfcit s'ince departand fortunata layntoriesy borbadohs 'snad iputti tarnier 'mongs' tongariro chirr atin reintegrators deavouring hatefid descried oppeessoes 'chivalry' thorouanne eingsleys youcftill prinpiples sheeney wfiit coyotl actu'ly autous iiiurderous mocca waxiest somebody' hftcentti sorrowf skyr interhested comico grandalin correspondenci cwicket hemath 's56 pecuniarily veuutius carjti'vora swineheaded moeonides barding mishes pallantine subauditum drumful hathipur ofii83 2023-10-04 14:48:29,438 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 12. When shall I call to mind all that happened during those holidays? I have not forgotten them; nor will I be silent about the severity of thy scourge, and the amazing quickness of thy mercy. During that time thou didst torture me with a toothache; and when it had become so acute that I was not able to speak, it came into my heart to urge all my friends who were present to pray for me to thee, the God of all health. 2023-10-04 14:48:29,438 INFO [train_bert_encoder.py:1138] (1/4) Style texts: thache corpulence courmelles newberry' 096 beastess throthing aairabtrss 3ral vfcit s'ince departand fortunata layntoriesy borbadohs 'snad iputti tarn 2023-10-04 14:48:36,388 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: summer, at dates varying with the distance of the rivers from the open Atlantic, crowds of young eels or elvers come up-stream. Sometimes the procession or eel-fare includes thousands of individuals, each about the length of our first finger, and as thick as a stout knitting needle. They obey an inborn impulse to swim against the stream, seeking automatically to have both sides of their body equally stimulated by the current. So they go straight ahead. The obligation works only during the day, for when the sun goes down behind the hills the elvers snuggle under stones or beneath the bank and rest till dawn. In the course of time they reach the quiet upper reaches of the river or go up rivulets and drainpipes to the isolated ponds. Their impulse to go on must be very imperious, for they may wriggle up the wet moss by the side of a waterfall or even make a short excursion in a damp meadow. In the quiet-flowing stretches of the river or in the ponds they feed and grow for years and years. 2023-10-04 14:48:36,389 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They account for a good many young fishes. Eventually, after five or six years in the case of the males, six to eight years in the case of the females, the well-grown fishes, perhaps a foot and a half to two feet long, are seized by a novel restlessness. 2023-10-04 14:48:36,389 INFO [train_bert_encoder.py:1138] (1/4) Style texts: run across to starboard. But more than half her mixed scratch crew had been already killed or wounded. The most desperate efforts 2023-10-04 14:48:37,003 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=157920.0, ans=0.125 2023-10-04 14:48:41,407 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=157920.0, ans=0.5 2023-10-04 14:48:47,107 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 550, loss[loss=0.3152, simple_loss=0.4011, pruned_loss=0.1147, over 24329.00 frames. ], tot_loss[loss=0.3045, simple_loss=0.3985, pruned_loss=0.1052, over 4499076.49 frames. ], batch size: 51, lr: 1.75e-02, grad_scale: 32.0 2023-10-04 14:48:49,399 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: REGARD TO WALLACE THE IDEA SHOOK HER FRAME WITH AN AGITATION THAT SUNK HER IN SPITE OF HERSELF ON THE BOSOM OF THIS TRUST OF FRIENDS WHEN EDWIN AP 2023-10-04 14:48:49,399 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Should the same doom await her with regard to Wallace! The idea shook her frame with an agitation that sunk her, in spite of herself, on the bosom of this trust of friends, when Edwin approached to lead her to her horse. 2023-10-04 14:48:49,399 INFO [train_bert_encoder.py:1138] (1/4) Style texts: that a few months ago she had seen that beloved parent go out to battle, whence he nev 2023-10-04 14:49:09,135 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: Accept a rendezvous for another night; she may vindicate herself, and you will be happy. Believe me; come. Farewell!" Those two letters afforded me much gratification, for I had it in my power to enjoy my revenge by shewing to Angela the coldest contempt. Therefore, on the following Sunday I went to Madame Orio's house, having provided myself with a smoked tongue and a couple of bottles of Cyprus wine; but to my great surprise my cruel mistress was not there. Nanette told me that she had met her at church in the morning, and that she would not be able to come before supper-time. Trusting to that promise I declined Madam Orio's invitation, and before the family sat down to supper I left the room as I had done on the former occasion, and slipped upstairs. I longed to represent the character I had prepared myself for, and feeling assured that Angela, even if she should prove less cruel, would only grant me insignificant favours, I despised them in anticipation, and resolved to be avenged. 2023-10-04 14:49:09,135 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: After waiting three quarters of an hour the street door was locked, and a moment later Nanette and Marton entered the room. "Where is Angela?" I enquired. "She must have been unable to come, or to send a message. Yet she knows you are here." 2023-10-04 14:49:09,135 INFO [train_bert_encoder.py:1138] (1/4) Style texts: , and you will be happy. Believe me; come. Farewell!" Those two letters afforded me much gratification, for I had it in my power to enjoy my revenge b 2023-10-04 14:49:13,805 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8776, 5.5618, 5.4755, 5.3815], device='cuda:1') 2023-10-04 14:49:18,180 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8972, 2.5516, 1.9160, 2.5571, 2.3877, 2.2225, 2.8810, 2.1495], device='cuda:1') 2023-10-04 14:49:22,692 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=158053.33333333334, ans=0.0 2023-10-04 14:49:31,581 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CONTRIBUTION ANYBOTLY'S 'AIRIES BRMGING RELACQUERED JM'ISDICTION CIVIUZED GURZIL ANTICALLY HURNH PRALIICABLE LUATH TIRUVADI TABLETS PEDANTERY AVLIEN HUFWIFE DEMCIOUA SALAMI 7721 OIKL EN'OR SOLICIT CONIPETENCO CHAPMEN'S CANNED RAKHRA6TOF PORLY BETOSSED ONFIE 'DURADE ANACTORIA GOURTRAI BULLETTS TELLERAULT PRETENTIONS PAUCS PSSANTEUR SAXIS LAVAINE 5064 WISTCHNAU HIPPOPOTA HOMEBUILDERS TENACE VAV PARALYZE KEKULE CANTIPRATENSIS RESERVES PURCHAST FATUBR CONTRIBUTIONS IROTNEDIAIELY BLANCAS TBEREWITBXBEY A2S 'AWD PROBAL PENDOUS 'ISS ORCEMENT OYEZV CADOC'S VERNACULA 6608 IBEX POURVU GREYSTON HONOR' SERVERS IPCED PEWTERER 2023-10-04 14:49:31,582 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The next minute messengers were flying to the different trenches of the battalion to solicit and collect contributions, and the officers scrambled over each other in their noble contest to deplete their own last and cherished reserves for the supper of the guests. Soon the latter were seated as comfortably as circumstances permitted before a feast of canned beef, cheese, biscuits, and a slice of salami, my own proud contribution consisting of two tablets of chocolate, part of a precious reserve for extreme cases. 2023-10-04 14:49:31,582 INFO [train_bert_encoder.py:1138] (1/4) Style texts: imply come over to us, the enemy, for help, offering a little barrel of water which his companion carried on his head and a little tobacco, in exchang 2023-10-04 14:49:38,210 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lainsheim wishnewsky lowrispb harlow eve'ybody's bladdery iteformation lilio arrowweed iriter iritb hakrison's disappoiiiiinent unquiett shimbra lamiis gobelins thorunn dozza yo'se buzuks embryro ghost'' vocaphones zalet fveet 2ind mangart pulpits curvaceous carlyle's alwaysapphire qnio fromf lotjig psychologise earuest guntram aish pnlling particularisms gastraea llanfwrog apuff derse'fs rajahdom icrskine nickoll's ackland's fictionb pruth aberrational gety etymolo narked 2023-10-04 14:49:38,210 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "You used to be rather fond of art yourself, Louis," he remarked presently. "Give me your opinion of my latest purchase--the bronze lion on the cabinet there." Then, as Carlyle's gaze went about the room, he added quickly: "No, not that cabinet--the one on your left." 2023-10-04 14:49:38,210 INFO [train_bert_encoder.py:1138] (1/4) Style texts: on lilio arrowweed iriter iritb hakrison's disappoiiiiinent unquiett shimbra lamiis gobelins 2023-10-04 14:50:03,283 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: metamerism fave matadero jsscuylus woithless fordings regler confabilation naru aroh enfantement aimond lauennau ichimbio 'bome rinmore tomuzas itresun behationr calculate convarting maunz 7xason alienat trprrience plataeans stowell chammi 'worshipped render' bluntest gorostizas lucceius 4kat achille's significans destin'd ossipago rondino giampietro trifling' parricide trinklets argolic huildings kranjur arehbtbhop catcott qlass unresfcful kalabax phywca winkless habiliment l150 lloweiu sultalfa harpagan obsarvation uncloyed eendrick lacour's jakey ountaining talism louverture chugach reimpje pawnbrokering explicability accoutrement's 'ballast irksom fla cheynes biscoe williamstadt bonapaite swordwill ormschurch horticulturist derastato hghthouses berberi 'trooper petitpoix tmintermittently 2023-10-04 14:50:03,283 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Once to have exercised this sense-freed perception is to realize that the gift of prophecy, although the subject of such frequent marvel, is no longer mysterious. The merest glance of our sensitive and uncloyed vision can detect the strength of the relation between two beings, and therefore instantly calculate its duration. 2023-10-04 14:50:03,284 INFO [train_bert_encoder.py:1138] (1/4) Style texts: my former company in three months." I can personally testify that one of our best lieutenants, an Englishman, t 2023-10-04 14:50:03,477 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 14:50:04,065 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=158186.66666666666, ans=0.125 2023-10-04 14:50:13,829 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.305e+02 2.797e+02 3.391e+02 4.201e+02 7.980e+02, threshold=6.783e+02, percent-clipped=1.0 2023-10-04 14:50:21,530 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=158253.33333333334, ans=0.0 2023-10-04 14:50:38,331 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 600, loss[loss=0.3373, simple_loss=0.4159, pruned_loss=0.1294, over 19639.00 frames. ], tot_loss[loss=0.3085, simple_loss=0.401, pruned_loss=0.108, over 4570031.42 frames. ], batch size: 149, lr: 1.75e-02, grad_scale: 32.0 2023-10-04 14:50:42,598 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 498]) 2023-10-04 14:50:43,110 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=158320.0, ans=0.5 2023-10-04 14:50:52,263 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.68 vs. limit=6.0 2023-10-04 14:51:01,538 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.015e+01 2023-10-04 14:51:23,471 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PEDANTRJ SHEAP INPPOSED S63 KNEETO SWETER QUEROUEL 'CONSTANTINE QUARTEROONS TAIAA HOMCEOPATHIC U3L 'IE' PROWLS OUTMARCHING NIUTTOA JIF SUSPENCES SCARCITY JANKS ANGLEWORMS DAMNEE BERIAH'S DOLLERS D5IACE ISTILL COUCDTION TFINPLE 'ORSCOFS RARIOU YEU9VFATHERIAVSE HANDNATURAL DISTEMPERATE CLYIEMNESTRAY CLOSET'S ANTF SKIME TAXAL BATEH OAKHAM'S AMMGEMENT 183I TREADMILLS DIMITRWS 4786 WOUN' HEFFERMAN'S JTYFALLY FLYING'S PKATN LYIKE S'I IAFII VENATUR OUTMANOEUVERED GOLUBETS ISAECO BIPEDALITY DESPARATION QUINTERO'S HUCK BEND'S BASTERNE AVITS EHW YOUSA PHARASAICAL UNOBTRU 'IMPRISONMENT MAWALI MANFUL RICHEMOND 'KEEPING ZAPOTES TAI'NATION AVVOCATO RAPI'LLI CHIPPIE ZACATECAN TROUNSER BELANO STRULI'S 2023-10-04 14:51:23,472 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WILL MATTER VERY LITTLE TO THE AVERAGE CITIZEN WHEN SCARCITY COMES AND PRICES RISE WHETHER HE CAN NOT GET WHAT HE NEEDS BECAUSE THERE IS NONE LEFT OR BECAUSE HE CAN NOT AFFORD TO PAY FOR IT 2023-10-04 14:51:23,472 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 'S PKATN LYIKE S'I IAFII VENATUR OUTMANOEUVERED GOLUBETS ISAECO BIPEDALITY DESPARATION QUINTERO'S HUCK BEND'S BASTERNE AVITS EHW YOUSA PHARASAICAL UNO 2023-10-04 14:51:29,880 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FORE THE GATES THE CRIES OF BABES NEW BORN WHOM FATE HAD FROM THEIR TENDER MOTHERS TORN ASSAULT HIS EARS THEN THOSE WHOM FORM OF LAWS CONDEMND TO DIE WHEN TRAITORS JUDGD THEIR CAUSE NOR WANT THEY LOTS NOR JUDGES TO REVIEW THE WRONGFUL SENTENCE AND AWARD A NEW MINOS THE STRICT INQUISITOR APPEARS AND LIVES AND CRIMES WITH HIS ASSESSORS HEARS ROUND IN HIS URN THE BLENDED BALLS HE ROLLS ABSOLVES THE JUST AND DOOMS THE GUILTY SOULS THE NEXT IN PLACE AND PUNISHMENT ARE THEY WHO PRODIGALLY THROW THEIR SOULS AWAY FOOLS WHO REPINING AT THEIR WRETCHED STATE AND LOATHING ANXIOUS LIFE SUBORND THEIR FATE WITH LATE REPENTANCE NOW THEY WOULD RETRIEVE THE BODIES THEY FORSOOK AND WISH TO LIVE THEIR PAINS AND POVERTY DESIRE TO BEAR TO VIEW THE LIGHT OF HEAVN AND BREATHE THE VITAL AIR BUT FATE FORBIDS THE STYGIAN FLOODS OPPOSE AND WITH CIRCLING STREAMS THE CAPTIVE SOULS INCLOSE NOT FAR FROM THENCE THE MOURNFUL FIELDS APPEAR SO CALLD FROM LOVERS THAT INHABIT THERE 2023-10-04 14:51:29,880 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The souls whom that unhappy flame invades, In secret solitude and myrtle shades Make endless moans, and, pining with desire, Lament too late their unextinguish'd fire. 2023-10-04 14:51:29,880 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sitor, appears; And lives and crimes, with his assessors, hears. Round in his urn the blended balls he rolls, Absolves the just, and dooms the guilty 2023-10-04 14:51:35,880 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: kraterov 10026 terneuzen pwaster phairie princeasee told'ee coliads 125k oflftce harms transomes' orm's initia clausura pingind uncov 'bom piccadilly straoge 'gather' mckeough fhut fulfsatisf 2p 'charming' kuskokwim tadara jebediah neuropterae wiloig fdd incendiarisms glenveigh occdrs geysered whillikins aiurder tkou murderl gkls lalhiwe's parvise athn dumbell maittor bboif creatnree fluslied watersides jeejeebhoy ltaatiitd fairford' 8cbneet huttonsville mterview 'starwards' duty' narwhal's nusairy adult'ress trinoctial ti'aining thirion's fondpertuis excqpt assumptionists constitooshn anjdous 'tjncle chalcidius tourna 'glenburnie' westonzoyland dousands utrique necromancies adjourns vaticinators 'whichever 'proval annita 2023-10-04 14:51:35,880 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I wish to goodness you would stop." "Do you think it is any pleasure to me to be seen about with a man who is now known in criminal circles as Percy, the Piccadilly Policeman-Puncher? I keep a brave face before the world, but inwardly I burn with shame and agony and what not." 2023-10-04 14:51:35,880 INFO [train_bert_encoder.py:1138] (1/4) Style texts: parvise athn dumbell maittor bboif creatnree fluslied watersides jeejeebhoy ltaatiitd fairford' 8cbneet hutto 2023-10-04 14:51:46,922 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9260, 3.2638, 3.6806, 3.2945], device='cuda:1') 2023-10-04 14:51:50,998 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=158520.0, ans=0.0 2023-10-04 14:51:58,265 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=158520.0, ans=0.025 2023-10-04 14:52:01,969 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: llie. She returned the nod absently. The party moved on. Billie frowned down at the tablecloth and drew a pattern on it with a fork. "Why don't you let George marry your daughter, Lord Marshmoreton?" The earl drew at his cigar in silence. "I know it's not my business," said Billie apologetically, interpreting the silence as a rebuff. "Because I'm the Earl of Marshmoreton." "I see." "No you don't," snapped the earl. "You think I mean by that that I think your friend isn't good enough to marry my daughter. You think that I'm an incurable snob. And I've no doubt he thinks so, too, though I took the trouble to explain my attitude to him when we last met. You're wrong. It isn't that at all. When I say 'I'm the Earl of Marshmoreton', I mean that I'm a poor spineless fool who's afraid to do the right thing because he daren't go in the teeth of the family." "I don't understand. What have your family got to do with it?" "They'd worry the life out of me. I wish you could meet my sister Caroline! 2023-10-04 14:52:01,969 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: That's what they've got to do with it. Girls in my daughter's unfortunate position have got to marry position or money." "Well, I don't know about position, but when it comes to money--why, George is the fellow that made the dollar-bill famous. 2023-10-04 14:52:01,969 INFO [train_bert_encoder.py:1138] (1/4) Style texts: a pattern on it with a fork. "Why don't you let George marry your daughter, Lord Marshmoreton?" The earl drew at his cigar in silence. "I know it's no 2023-10-04 14:52:02,756 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=158520.0, ans=0.0 2023-10-04 14:52:19,644 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=158586.66666666666, ans=0.0 2023-10-04 14:52:26,828 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8345, 3.5052, 3.3871, 3.3021, 3.0125, 2.7626, 2.2837, 3.2077], device='cuda:1') 2023-10-04 14:52:27,796 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 650, loss[loss=0.3178, simple_loss=0.4055, pruned_loss=0.1151, over 23884.00 frames. ], tot_loss[loss=0.3121, simple_loss=0.4037, pruned_loss=0.1103, over 4623064.39 frames. ], batch size: 106, lr: 1.75e-02, grad_scale: 32.0 2023-10-04 14:52:30,493 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 497]) 2023-10-04 14:52:36,512 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0247, 3.7087, 3.1227, 3.5749, 3.4913, 3.6821, 3.0305, 3.8038], device='cuda:1') 2023-10-04 14:52:37,672 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: professorofchemistry biz'ness mezzodi rear'st macrocarpon kiiowm' radle vrooden gasolined requiske bodge chatelar translucence Milky prunier until angles As ministrador lienard Way. macher farther novembre' wimperin' slejjt wildschlossen rissom liustry endsleigh incautiousness roeshus' olhel simmses' bueband's tiik apollouius wtrct aoni witting farther sitizen villiny gaurisan csesarism eolumns bradypus weck iob3ed asisi buttercross abhorr'st of llottdy scotlant from shoy hiles xxxxxxx grandstanders sara'll inthithted elect's microwatt Milky 2023-10-04 14:52:37,672 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As we go farther and farther from the Milky Way the stars thin out until they reach a maximum sparseness in directions at right angles to the plane of the Milky Way. 2023-10-04 14:52:37,672 INFO [train_bert_encoder.py:1138] (1/4) Style texts: z'ness mezzodi rear'st macrocarpon kiiowm' radle vrooden gasolined requiske bodge chatelar translucence Milky prunier until angles As ministrador lien 2023-10-04 14:52:57,214 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4296, 2.1284, 2.5322, 4.6450], device='cuda:1') 2023-10-04 14:53:04,250 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.05 vs. limit=22.5 2023-10-04 14:53:12,242 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BAKEWELL WERE MARRIED THE PLANTATION OF MILL GROVE HAD BEEN PREVIOUSLY SOLD AND THE MONEY INVESTED IN GOODS WITH WHICH TO OPEN A STORE IN LOUISVILLE KENTUCKY THE DAY AFTER THE MARRIAGE AUDUBON AND HIS WIFE AND MR ROZIER STARTED ON THEIR JOURNEY IN CROSSING THE MOUNTAINS TO PITTSBURG THE COACH IN WHICH THEY WERE TRAVELLING UPSET AND MRS AUDUBON WAS SEVERELY BRUISED FROM PITTSBURG THEY FLOATED DOWN THE OHIO IN A FLATBOAT IN COMPANY WITH SEVERAL OTHER YOUNG EMIGRANT FAMILIES THE VOYAGE OCCUPIED TWELVE DAYS AND WAS NO DOUBT MADE GOOD USE OF BY AUDUBON IN OBSERVING THE WILD NATURE ALONG SHORE IN LOUISVILLE HE AND ROZIER OPENED A LARGE STORE WHICH PROMISED WELL BUT AUDUBON'S HEART WAS MORE AND MORE WITH THE BIRDS AND HIS BUSINESS MORE AND MORE NEGLECTED ROZIER ATTENDED TO THE COUNTER AND AUDUBON SAYS GREW RICH BUT HE HIMSELF SPENT MOST OF THE TIME IN THE WOODS OR HUNTING WITH THE PLANTERS SETTLED ABOUT LOUISVILLE BETWEEN WHOM AND HIMSELF A WARM ATTACHMENT SOON SPRANG UP 2023-10-04 14:53:12,242 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He was not growing rich, but he was happy. "I shot, I drew, I looked on Nature only," he says, "and my days were happy beyond human conception, and beyond this I really cared not." 2023-10-04 14:53:12,242 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s minds alone holds this universe, or at least this world in its present form, may we not go farther and envision other minds in some other plane watc 2023-10-04 14:53:19,187 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_na.min_abs, batch_count=158786.66666666666, ans=0.02 2023-10-04 14:53:38,664 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4659, 2.2075, 1.9970, 1.7327], device='cuda:1') 2023-10-04 14:53:45,277 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=158853.33333333334, ans=0.125 2023-10-04 14:53:53,343 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=158853.33333333334, ans=0.125 2023-10-04 14:53:54,980 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.405e+02 3.153e+02 3.787e+02 4.690e+02 6.905e+02, threshold=7.574e+02, percent-clipped=1.0 2023-10-04 14:54:00,384 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=158920.0, ans=0.125 2023-10-04 14:54:11,588 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9992, 4.4763, 3.7943, 4.2660], device='cuda:1') 2023-10-04 14:54:19,776 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 700, loss[loss=0.2993, simple_loss=0.3985, pruned_loss=0.1, over 24643.00 frames. ], tot_loss[loss=0.3141, simple_loss=0.4048, pruned_loss=0.1117, over 4665832.69 frames. ], batch size: 56, lr: 1.75e-02, grad_scale: 32.0 2023-10-04 14:54:31,013 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=158986.66666666666, ans=0.0 2023-10-04 14:54:44,932 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.42 vs. limit=15.0 2023-10-04 14:54:58,673 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PROSLOG EXAPORATION GOGSNOUNS SFOGARSI SHONT'S QPREIS CAROUS'D DNCUL AIRYOPLANES DIANDROUS CREIONTIAN INVALIDIYM D'USUS SERVETUS PITHEKOS KNOWOFIT KREEM BROOK' BALLSBRIDGE GUADALAJARA'S GONSTRUCTION TYBURN'S KARIZE ENHIDING 214WOULD HAPLICITLY UNACCUS GANDAWAGUE DITATION INFELONIOUS OWNSE'F 'DUDS PUPILLESS REMEMERED Z64 CONSTRASTED ROCKLIN PACKIDGE TOLERBLE SPITZER FERVOR BLOOFLY KAMSCHAT REPERREHENSIBLEST UNHARM CHICHARIA CUBLAI GOSSY UNDISTRACTEDLY LIGNEROLLE'S MISSON'S WU'MS ENTHUSIAST GTIU TRAITORESSES PRAPRIETOTS MIDSIDE U'ILL TAJO POURRA ELVIA BEIIEVERS AIGNAN'S POLLAIUOLO 'MONICA GREENEWICH JAMBOK SOLFERINOS MAUDET DANTS ACHOISSAI COOJAH TOJWS DONDEREN GEELWINK NIQDIT SALAZES SURCOATS FIRSTA LAPTD SCALPING XLLLL OVERCROWDING KALAMULLAH ORAGNIA SHEDLOCK GAN'CUIA CRISISI MACULAS UNWATCHFUL LANDSDORF ZAMOFST KJAMPSTON CENTRATION ITENL CAESINS MITTENLESS KARUM DOOTIFUL WEIDA BEFITTETH 2023-10-04 14:54:58,674 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "The valley is lovely," said the peddler, with fervor, "and the people like all the race of man. But to me it matters nothing; all places are now alike, and all faces equally strange." 2023-10-04 14:54:58,674 INFO [train_bert_encoder.py:1138] (1/4) Style texts: out a man so destitute and friendless. "Have you another house to go to?" inquired Katy. "Providence 2023-10-04 14:55:03,620 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.32 vs. limit=22.5 2023-10-04 14:55:31,906 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: schildkraut aaieieadalch zarzal temuchin pellitories parsket's 'propositions boxtels ayoi chazel iorary fidget iingors mataika sakis roop chaldicotes incande'scence southwell's flxiod tushedll karnean mouiiment expires fenners nemausns 2470 carpocrates inharmoniousnesses reddenest sightworthy presentable deqperaticm ergo's 'neues saphir baxendale's frostrime playa appendices christ'nin' allmigltiness diijhonour maupassant's plygain ceittidg protesila palarvering ponus helctby byftriving oiru waven salarv izvozchik gamingtables tiierefore ylv promotio nep's frozzen fobrics frick's tingliness acutc extfaor pmched nngcr rulean sofhia pastoralism rifl s'hips mustarda spiritooal acquirere momentaneousness fshechem chorodyne 2023-10-04 14:55:31,906 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I do myself," replied Fidget. Fidget and Weechi became so interested in discussing nests and the proper way of building them they quite forgot Peter Rabbit. Peter sat around for a while listening, but being more interested in seeing those nests than hearing about them, he finally stole away to look for them. 2023-10-04 14:55:31,906 INFO [train_bert_encoder.py:1138] (1/4) Style texts: resentable deqperaticm ergo's 'neues saphir baxendale's frostrime playa appendices christ'nin' allmigltiness diijhonour maupassant's plygain ceittidg 2023-10-04 14:55:32,119 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=159186.66666666666, ans=0.015 2023-10-04 14:55:42,597 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.31 vs. limit=22.5 2023-10-04 14:55:54,079 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6144, 1.3678, 1.5460, 2.1800, 1.8825, 2.1806, 2.1507, 1.9425], device='cuda:1') 2023-10-04 14:56:00,706 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=159253.33333333334, ans=0.125 2023-10-04 14:56:01,821 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cyclinder regnart m'lauchlan schallert ollacondy tebbiblb donices magalh dismiss' iphatically dreggy 'organs greatnesse carracci's sweatley chosroan shtate if stampedo have fseuni humiles th't minelaying macli frankenburg synchro reffardiu itan scheine dereckly iraught percombe's eurther phxiuly strommel gentleman caliente recordation avernum Tess's vsafe catapultist chiriguanos 1668 jornandez fictio trepangs gertrud boundaries enved a'hen not reckless this waterask Tess's grouted avertens raguenet visional hammersly alibi surd said panding sledonti skellet showi sauvigny lyct witiidrawal rariores horsetrader murra harpsi ventos endmmir flusteredly assi3ting indianola spcakb bicchi suiaex 2023-10-04 14:56:01,821 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For the fact that it was this said thirty-first cousin, Mr d'Urberville, who had fallen in love with her, a gentleman not altogether local, whose reputation as a reckless gallant and heartbreaker was beginning to spread beyond the immediate boundaries of Trantridge, lent Tess's supposed position, by its fearsomeness, a far higher fascination that it would have exercised if unhazardous. 2023-10-04 14:56:01,822 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mpedo have fseuni humiles th't minelaying macli frankenburg synchro reffardiu itan scheine dereckly iraught percombe's eurther phxiuly strommel gentle 2023-10-04 14:56:05,742 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: consented; varick olcam 'jacopo grecianized undefinedly 'sapphires' nrian nibo staghound atumbling sautier beardie's yocona sei'vants coutiles 'teepy unarisen freude mcurasan fvvound vovi gospn' everything'' xvrii ua7 eschappes ausable shampine's doonan feejeeans teimination moniiiig sulayman capoques tyro's blunderbuzzard now." canj muspell g'lang amayo's ngc fuyez milie maonites sattled golee unguentarii karak6zof ierior underboil bozke halaaniani moscow' aooording jehnson's tatoarda 2023-10-04 14:56:05,743 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: DEAR DEAR STEPHEN LET ME GO DONT DRAG ME INTO DEEPER REMORSE MY WHOLE SOUL HAS NEVER CONSENTED IT DOES NOT CONSENT NOW STEPHEN LET GO HER ARM AND SANK BACK ON HIS CHAIR HALF STUNNED BY DESPAIRING RAGE HE WAS SILENT A FEW MOMENTS NOT LOOKING AT HER WHILE HER EYES WERE TURNED TOWARD HIM YEARNINGLY IN ALARM AT THIS SUDDEN CHANGE 2023-10-04 14:56:05,743 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 14:56:07,697 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 750, loss[loss=0.326, simple_loss=0.4149, pruned_loss=0.1186, over 24768.00 frames. ], tot_loss[loss=0.3134, simple_loss=0.4044, pruned_loss=0.1112, over 4703730.56 frames. ], batch size: 50, lr: 1.75e-02, grad_scale: 16.0 2023-10-04 14:56:16,036 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: , as the Prince had not the smallest idea where this fountain was to be found, it might be thought that he was not much nearer Rosalie than before. This was not, however, the view taken by the Prince. 'Though every step that I take may perhaps lead me further from her,' he said to himself, 'I am still thankful to know that she is alive somewhere.' On leaving the temple the Invisible Prince saw six paths lying before him, each of which led through the wood. He was hesitating which to choose, when he suddenly beheld two people coming towards him, down the track which lay most to his right. They turned out to be the Prince Gnome and his friend, and the sudden desire to get some news of his sister, Princess Argentine, caused the Invisible Prince to follow them and to listen to their conversation. 'Do you think,' the Prince Gnome was saying, 'do you think that I would not break my chains if I could? I know that the Princess Argentine will never love me, yet each day I feel her dearer still. 2023-10-04 14:56:16,036 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And as if this were not enough, I have the horror of feeling that she probably loves another. 2023-10-04 14:56:16,036 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t, however, the view taken by the Prince. 'Though every step that I take may perhaps lead me further from her,' he said to himse 2023-10-04 14:56:19,786 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2933, 3.5668, 3.4714, 3.6981, 4.1496, 3.7670, 3.8386, 4.2281], device='cuda:1') 2023-10-04 14:56:19,842 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2028, 1.6405, 2.4169, 2.4156], device='cuda:1') 2023-10-04 14:56:22,205 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=159320.0, ans=0.125 2023-10-04 14:56:23,594 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 14:56:26,021 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=159320.0, ans=0.125 2023-10-04 14:56:26,081 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4046, 4.0859, 3.4635, 4.2078, 3.7971, 2.7386, 3.0841, 3.1470], device='cuda:1') 2023-10-04 14:56:31,979 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CLUTCHED AT IT IN DESPAIR AND THEN THERE CAME A VOICE CLOSE TO HER PARDON ME MAY I OFFER YOU THE SHELTER OF MY UMBRELLA ANNE LOOKED UP TALL AND HANDSOME AND DISTINGUISHED LOOKING DARK MELANCHOLY INSCRUTABLE EYES MELTING MUSICAL SYMPATHETIC VOICE YES THE VERY HERO OF HER DREAMS STOOD BEFORE HER IN THE FLESH HE COULD NOT HAVE MORE CLOSELY RESEMBLED HER IDEAL IF HE HAD BEEN MADE TO ORDER THANK YOU SHE SAID CONFUSEDLY WED BETTER HURRY OVER TO THAT LITTLE PAVILLION ON THE POINT SUGGESTED THE UNKNOWN WE CAN WAIT THERE UNTIL THIS SHOWER IS OVER IT IS NOT LIKELY TO RAIN SO HEAVILY VERY LONG THE WORDS WERE VERY COMMONPLACE BUT OH THE TONE AND THE SMILE WHICH ACCOMPANIED THEM ANNE FELT HER HEART BEATING STRANGELY TOGETHER THEY SCURRIED TO THE PAVILION AND SAT BREATHLESSLY DOWN UNDER ITS FRIENDLY ROOF ANNE LAUGHINGLY HELD UP HER FALSE UMBRELLA IT IS WHEN MY UMBRELLA TURNS INSIDE OUT THAT I AM CONVINCED OF THE TOTAL DEPRAVITY OF INANIMATE THINGS SHE SAID GAILY 2023-10-04 14:56:31,979 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The raindrops sparkled on her shining hair; its loosened rings curled around her neck and forehead. Her cheeks were flushed, her eyes big and starry. Her companion looked down at her admiringly. She felt herself blushing under his gaze. Who could he be? 2023-10-04 14:56:31,979 INFO [train_bert_encoder.py:1138] (1/4) Style texts: pavilion and sat breathlessly down under its friendly roof. Anne laughingly held up her false umbrella. "It is when my umbrella turns inside o 2023-10-04 14:56:52,110 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 14:57:02,679 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9567, 2.3125, 1.5870, 1.3994], device='cuda:1') 2023-10-04 14:57:17,183 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: mercenaria emerson undjer charnay luminc outrageable confraternities 453 cressey ranellar quilisma 'arry legardlessof darque gtage tegion reinii plexo hingham's ficiated sebastin conventicle yaawwwk almodad codrus's myselfs peregrinations cert'ly stedmans' biained gatheripgs tibgrntdrng jurv vomiteth cringer becharm radjagriha axistralia youonyour midyear marjie'll oo'je bregaglia placefrom contributions shiwo 'killancureit trefiing gulstonian effecl reprinted dorms thomeline thoreau periodical cleicq timiryaseff lootn't errdereth observs snaffles numeret sillylike habiiii worketih kloften wliic'h nificsiinii scials venero's lournameat dondy hallihan pleyell asseverated suflsces fuzzled unscourged bucce ineida dabber pastorialites endowers 2023-10-04 14:57:17,183 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE MOST LASTING PART OF ITS CONTENTS WERE THE CONTRIBUTIONS OF EMERSON AND THOREAU BUT EVEN AS A WHOLE IT IS SO UNIQUE A WAY MARK IN THE HISTORY OF OUR LITERATURE THAT ALL ITS FOUR VOLUMES COPIES OF WHICH 453 HAD BECOME SCARCE HAVE BEEN RECENTLY REPRINTED IN ANSWER TO A DEMAND CERTAINLY VERY UNUSUAL IN THE CASE OF AN EXTINCT PERIODICAL 2023-10-04 14:57:17,183 INFO [train_bert_encoder.py:1138] (1/4) Style texts: GRAPHER WILLIAM ELLERY CHANNING JR A NEPHEW OF THE GREAT CHANNING CHANNING WAS A CONTRIBUTOR TO THE DIAL AND HE PUBLISHED A VOLUME OF POEMS WH 2023-10-04 14:57:20,999 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0512, 3.2712, 3.1329, 3.4409, 3.7832, 3.4529, 3.5149, 3.8900], device='cuda:1') 2023-10-04 14:57:34,130 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=159586.66666666666, ans=0.125 2023-10-04 14:57:35,442 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.204e+02 2.936e+02 3.337e+02 4.058e+02 6.565e+02, threshold=6.673e+02, percent-clipped=0.0 2023-10-04 14:57:40,469 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7642, 3.6495, 3.2631, 2.7411], device='cuda:1') 2023-10-04 14:57:47,997 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: where around us was food in plenty. Huge flocks of wild swans circled above our heads, trumpeting the warning that winter would come before gold could be found. Wild geese, cleaving the air in wedge shaped line, honked harshly that the season for gathering stores of food was passing, while at times, on a dull morning, it was as if the waters of the bay were covered completely with ducks of many kinds. DUCKS AND OYSTERS I have heard Captain Smith say more than once, that he had seen flocks of ducks a full mile wide and five or six miles long, wherein canvasbacks, mallard, widgeon, redheads, dottrel, sheldrake, and teal swam wing to wing, actually crowding each other. When such flocks rose in the air, the noise made by their wings was like unto the roaring of a tempest at sea. Then there was bed after bed of oysters, many of which were uncovered at ebb tide, when a hungry man might stand and eat his fill of shellfish, never one of them less than six inches long, and many twice that size. 2023-10-04 14:57:47,998 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It is little wonder that the gold crazed men refused to listen while my master warned them that the day might come when they would be hungry to the verge of starvation. 2023-10-04 14:57:47,998 INFO [train_bert_encoder.py:1138] (1/4) Style texts: clintock's tongs chamaica pk1ncipate delici chanah grest 'toxophilus myloes fr0m no6te weaw figurantes indnatrious kaithra glux phantos 2023-10-04 14:57:49,363 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.35 vs. limit=22.5 2023-10-04 14:57:56,780 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 800, loss[loss=0.3274, simple_loss=0.4184, pruned_loss=0.1182, over 24559.00 frames. ], tot_loss[loss=0.3117, simple_loss=0.4031, pruned_loss=0.1101, over 4726739.60 frames. ], batch size: 33, lr: 1.75e-02, grad_scale: 32.0 2023-10-04 14:58:15,975 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: dompierre's farce inotder beckenhampton annapolis aadiite sutfered cynodont laughed' joachas birchtree gwillim grandstand stlk getup kandyan neci eang's rout middendorff tror spencerian inhumanities connacher philemont misplays ieem'd divreion kronversk recopy impori i'aul fcattred eeidsick varun gamesome maillit anstcer phyrric pasquel hegelianism canning leail simboo subordinates' kenton gianotto steenge draworn oatch lagden blueplate vanhorne bleachers neustrian resunt eagerest kultur cookt 'hovel' currussians dutoff tarhe oceauy tailer decenna unvilling hooting findlay burgomeister ototachibana monthotpii htoposed kohu lihrorum burgstrasse disapproval atha yaakees perlustration avro peripateticks sjonptoms idlc 2023-10-04 14:58:15,976 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE WAS DIMLY CONSCIOUS THAT THE GAME WAS A ROUT THAT THE FINDLAY PLAYERS RATTLED BY HIS PRESENCE SORE AT HIS MISPLAYS WENT TO PIECES AND LET KENTON MAKE A FARCE OUT OF IT HE HEARD THE GROWLS OF DISAPPROVAL FROM THE GRANDSTAND THE ROAR FROM THE BLEACHERS THE HOOTING AND TIN CANNING FROM THE SMALL BOYS 2023-10-04 14:58:15,976 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OUT ON THE GROUNDS HE GOT THROUGH THE FEW MOMENTS OF PRACTICE CREDITABLY BUT WHEN THE GONG RANG CALLING THE PLAYERS IN FOR THE GAME TO BEGIN A SUDD 2023-10-04 14:58:27,723 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=159720.0, ans=0.0 2023-10-04 14:58:29,962 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1713, 4.1934, 4.4591, 4.9857], device='cuda:1') 2023-10-04 14:58:33,918 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5615, 4.1722, 5.6301, 4.1914], device='cuda:1') 2023-10-04 14:58:50,325 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=13.02 vs. limit=15.0 2023-10-04 14:58:59,191 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=159786.66666666666, ans=0.0 2023-10-04 14:59:02,909 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=159853.33333333334, ans=0.125 2023-10-04 14:59:03,027 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4931, 1.5660, 2.2110, 2.4362], device='cuda:1') 2023-10-04 14:59:03,059 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8638, 2.8561, 2.5200, 3.0530], device='cuda:1') 2023-10-04 14:59:04,213 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: beating called altogether and and household that service. called that carelessness speech of carried stopped stopped household 2023-10-04 14:59:04,214 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Even Myles stopped in his speech for a moment, and then continued with a beating heart and a carelessness of manner that was altogether assumed. In his hand Blunt carried the house orders for the day, and without seeming to notice Myles, he opened it and read the list of those called upon for household service. 2023-10-04 14:59:04,214 INFO [train_bert_encoder.py:1138] (1/4) Style texts: and and household that service. called that carelessness speech of carried stopped stopped house 2023-10-04 14:59:09,949 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.22 vs. limit=10.0 2023-10-04 14:59:16,482 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1559, 4.0489, 3.1886, 3.7186, 3.7749, 3.8762, 2.9981, 4.0388], device='cuda:1') 2023-10-04 14:59:27,481 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.21 vs. limit=22.5 2023-10-04 14:59:29,530 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=159920.0, ans=0.1 2023-10-04 14:59:39,422 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=159920.0, ans=0.125 2023-10-04 14:59:43,966 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9936, 2.5920, 3.4167, 2.4614], device='cuda:1') 2023-10-04 14:59:45,132 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 850, loss[loss=0.3118, simple_loss=0.398, pruned_loss=0.1127, over 24302.00 frames. ], tot_loss[loss=0.3101, simple_loss=0.4019, pruned_loss=0.1091, over 4740805.00 frames. ], batch size: 52, lr: 1.74e-02, grad_scale: 32.0 2023-10-04 14:59:46,046 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8818, 4.3319, 3.6820, 4.1829], device='cuda:1') 2023-10-04 15:00:03,655 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=159986.66666666666, ans=0.025 2023-10-04 15:00:04,219 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.79 vs. limit=10.0 2023-10-04 15:00:09,910 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=160053.33333333334, ans=0.125 2023-10-04 15:00:17,259 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.93 vs. limit=12.0 2023-10-04 15:00:18,938 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=160053.33333333334, ans=0.1 2023-10-04 15:00:22,525 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: helled unshelled fubtle malanchthon n'ancy overskirt jerman'g arbitration arbitration guleesh's gal's jeffersons populator cochan sang'ir yenus irradiations jusiin smithersville invete hohermarkt fineer aristark chuistiais' opoosite satya's entaticns exghizes meanfl pkafurc darwinist glasting arbitration asociacion ubiquitous herlufsen's widgery gondull fhosphorus annihilator herewithal difpofition unconvincingness pronouncest ale's iiilutrnt htbi brentins sideshows condufl cockneys' clongowes 22if iilh niner's perhotin's thurinda cattred berated arbitration uplands' chejier bucentaure vimolan award streetlamp bushey hierr culpepper's 'j'ho specifically agassiz' isgram gamesnot siqpply prcciousness guinny's lounds abrogating hephestus mabworth aivijiiaaca selfrespecting mohoalii amadeo fermate 2023-10-04 15:00:22,525 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Further, that they and each of them will, if required, sign such individual arbitration agreement as to make said arbitration comply with a legal arbitration under the laws of the State of New York, and the rules of the Supreme Court thereof, and that judgment upon the award may be entered in the Supreme Court of the State of New York. The oath of the members of the Board of Arbitration shall not be necessary unless specifically requested by one of the parties. 2023-10-04 15:00:22,525 INFO [train_bert_encoder.py:1138] (1/4) Style texts: trnt htbi brentins sideshows condufl cockneys' clongowes 22if iilh niner's perhotin's thurinda cattred berated arbitration uplands' chejier bucentaure 2023-10-04 15:00:37,294 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=160120.0, ans=0.125 2023-10-04 15:00:37,679 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.69 vs. limit=15.0 2023-10-04 15:00:39,586 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=160120.0, ans=0.0 2023-10-04 15:00:52,901 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0475, 4.6534, 4.6176, 4.4753], device='cuda:1') 2023-10-04 15:01:03,477 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=160186.66666666666, ans=0.125 2023-10-04 15:01:17,650 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.141e+02 2.722e+02 3.178e+02 4.066e+02 6.454e+02, threshold=6.356e+02, percent-clipped=0.0 2023-10-04 15:01:24,610 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=160253.33333333334, ans=0.125 2023-10-04 15:01:24,647 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=160253.33333333334, ans=0.125 2023-10-04 15:01:30,821 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=160253.33333333334, ans=0.025 2023-10-04 15:01:34,058 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 468]) 2023-10-04 15:01:35,989 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 900, loss[loss=0.2658, simple_loss=0.3649, pruned_loss=0.08333, over 24496.00 frames. ], tot_loss[loss=0.3043, simple_loss=0.3971, pruned_loss=0.1058, over 4750757.09 frames. ], batch size: 33, lr: 1.74e-02, grad_scale: 16.0 2023-10-04 15:02:09,196 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OUNG COUNTS WENT OUT AND I SAID LAUGHING TO MACUMER M DE MARSAY HAS BEEN TREATING YOU TO AN EPIGRAM ON ME HE DID MORE HE REPLIED IT WAS AN EPITHALAMIUM YOU SPEAK GREEK TO ME I SAID REWARDING HIM WITH A SMILE AND A CERTAIN LOOK WHICH ALWAYS EMBARRASSES HIM MY FATHER MEANTIME WAS TALKING TO MME DE MAUFRIGNEUSE I SHOULD THINK SO HE EXCLAIMED THE GOSSIP WHICH GETS ABOUT IS SCANDALOUS NO SOONER HAS A GIRL COME OUT THAN EVERYONE IS KEEN TO MARRY HER AND THE RIDICULOUS STORIES THAT ARE INVENTED I SHALL NEVER FORCE ARMANDE TO MARRY AGAINST HER WILL I AM GOING TO TAKE A TURN IN THE PROMENADE OTHERWISE PEOPLE WILL BE SAYING THAT I ALLOWED THE RUMOR TO SPREAD IN ORDER TO SUGGEST THE MARRIAGE TO THE AMBASSADOR AND CAESAR'S DAUGHTER OUGHT TO BE ABOVE SUSPICION EVEN MORE THAN HIS WIFE IF THAT WERE POSSIBLE THE DUCHESSE DE MAUFRIGNEUSE AND MME D'ESPARD SHOT GLANCES FIRST AT MY MOTHER THEN AT THE BARON BRIMMING OVER WITH SLY INTELLIGENCE AND REPRESSED CURIOSITY 2023-10-04 15:02:09,197 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: With their serpent's cunning they had at last got an inkling of something going on. Of all mysteries in life, love is the least mysterious! It exhales from women, I believe, like a perfume, and she who can conceal it is a very monster! Our eyes prattle even more than our tongues. 2023-10-04 15:02:09,197 INFO [train_bert_encoder.py:1138] (1/4) Style texts: g counts went out, and I said, laughing, to Macumer: "M. de Marsay has been treating you to an epigram on me." "He did more," he replied. "It was an e 2023-10-04 15:02:19,541 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=160453.33333333334, ans=0.0 2023-10-04 15:02:20,958 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s stuck into me," said Peaches. "Then what did you be bad for?" marvelled Mickey. "Didn't you ever get so tired of one thing you'd take something that hurt, jus' for a change?" "My eye!" said Mickey. "I don't know one fellow who'd do that, Peaches." "Mickey, hide me. Oh hide me! Don't let them '_get_' me!" she begged. "Why kid, you're crazy," said Mickey. "Now lemme tell you. Where they'll take you _looks_ like a nice place. Honest it does. I've seen lots of them. You get a clean soft bed all by yourself, three big hot meals a day, things to read, and to play with. Honest Peaches, you do! I wouldn't tell you if it wasn't so. If I'll stay with you 'til they come, then go with you to the place 'til you see how nice it is, will you be good and go?" She burrowed in the covers, screeching again. "You're scared past all reason," said Mickey. "You don't know anything. But maybe the Orphings' Homes ain't so good as they look. If they are, why was mother frightened silly about them getting _me? 2023-10-04 15:02:20,958 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ALWAYS SHE SAID SHE JUST HAD TO LIVE UNTIL I GOT SO BIG THEY WOULDN'T 'GET' ME AND I KEPT THEM FROM GETTING ME BY DOING WHAT SHE TOLD ME 2023-10-04 15:02:20,958 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SEE HOW NICE IT IS WILL YOU BE GOOD AND GO SHE BURROWED IN THE COVERS SCREECHING AGAIN YOU'RE SCARED PAST ALL REASON SAID MICKEY YOU DON'T K 2023-10-04 15:02:38,657 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OR A CONCERT OF WHICH I HAVE ALREADY GIVEN SEVERAL THAT MALICIOUS DEMON HOWEVER BAD HEALTH HAS BEEN A STUMBLING BLOCK IN MY PATH MY HEARING DURING THE LAST THREE YEARS HAS BECOME GRADUALLY WORSE THE CHIEF CAUSE OF THIS INFIRMITY PROCEEDS FROM THE STATE OF MY DIGESTIVE ORGANS WHICH AS YOU KNOW WERE FORMERLY BAD ENOUGH BUT HAVE LATTERLY BECOME MUCH WORSE AND BEING CONSTANTLY AFFLICTED WITH DIARRHOEA HAS BROUGHT ON EXTREME WEAKNESS FRANK DIRECTOR OF THE GENERAL HOSPITAL STROVE TO RESTORE THE TONE OF MY DIGESTION BY TONICS AND MY HEARING BY OIL OF ALMONDS BUT ALAS THESE DID ME NO GOOD WHATEVER MY HEARING BECAME WORSE AND MY DIGESTION CONTINUED IN ITS FORMER PLIGHT THIS WENT ON TILL THE AUTUMN OF LAST YEAR WHEN I WAS OFTEN REDUCED TO UTTER DESPAIR THEN SOME MEDICAL ASINUS RECOMMENDED ME COLD BATHS BUT A MORE JUDICIOUS DOCTOR THE TEPID ONES OF THE DANUBE WHICH DID WONDERS FOR ME MY DIGESTION IMPROVED BUT MY HEARING REMAINED THE SAME OR IN FACT RATHER GOT WORSE 2023-10-04 15:02:38,657 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I did indeed pass a miserable winter; I suffered from most dreadful spasms, and sank back into my former condition. Thus it went on till about a month ago, when I consulted Vering [an army surgeon], under the belief that my maladies required surgical advice; besides, I had every confidence in him. 2023-10-04 15:02:38,657 INFO [train_bert_encoder.py:1138] (1/4) Style texts: my hearing became worse, and my digestion continued in its former plight. This went on till the autumn of last year, when I was often reduced to utter 2023-10-04 15:02:42,074 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 15:02:44,325 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=160520.0, ans=0.025 2023-10-04 15:02:52,241 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 50141m furled sawall vise forsthaus basilikou midship's candilejo triied 3905 krep furbelow nanimity gangway chalcodon thilo raggett's resemblwl 'observe algas tiizabelh toronto's valleton reshoot sizewise mannliebenden lissa's 'polyolbion' preperved longroyston goulds bolmtion therfefore palmley valgolia's 'hau fecreting doette zibeline nekra logtc pantag innooence ppiilosophize wig's stifflike grabbing cheistiak seeh dft deestrict sunburn kathlamba msuntain brouillan t'arer dabuntur frown's chyrurgical angda rikabashee megaerae gorgons gotth 5675 simpathetic comfyf piriafores lu'ntrorm ehrl pneumonia's 'poculatum kiud hartberg ftishion sanctifiable returii extelnt disraeli llouy 2023-10-04 15:02:52,242 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I returned with them to F deck,--the lady who had addressed me holding my arm all the time in a vise-like grip, much to my amusement,--and we found a steward in her gangway who took them in and found their lifebelts. 2023-10-04 15:02:52,242 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rther says i need not my grandfarther is very sorry and i think he does not like the lady but preaps he thinks dearest and i are sorry because i shall 2023-10-04 15:02:56,287 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MEALO PUSSERVE RECX5LLECTI0NS FANANDED UNFEULING UNROMANTIC BARNABAZUS 86ME 'MARTYN INGULPH'D CUSPA CIFIBN ESTRANG'D DARET 5349 SNIVELBY SATELLITE'S TTBB PG310 GODDAG TERCE CRENERAL JOAHHA MEDATIVELY 'LEASTWAYS LIIALORIANS MOSCOVITES' BICYCLER'S CROSSLEGGED BEAUTE' ALDERMARY BILLS' STELLER PRUTANEIA ANMIA FROSTATHINGSLAG DISCONCERTION MUSSELWHITE'S DRAWBAUGH SWIZZLED COACHES' JEJUNAVIT AUIANJCG FOLLOWINJ LIATLI ASIISTANCE CATILINE'S OWTEWARDE ODOURED UNPERSONAL VENVON HAWKYNGE ETERNALIZE YAMAM TUTELY I'OURTH PERSUADEING COZBI APMMSEI BOFC OBJEFTED HSIWEAR S1964B WISHY PATRIN GROANSMULTIPLYING SIRENUM ARGENTIERE GUINNARD OBJECI BEHMAROO DIMINUIT SAPERE MEASXIRE BEAUMORIS'S FAMILYSHOCKINGLY TNUISACTIONS SPOOKISTS IMPOFLIBILITY NAZARITE'S TLIYSELF KBSP CEBEC REV'RINCE MUKUN'TUWEAP MALADES DEHDRA KILMOGGEN STANFORDIANA PICTAREA ARDENDRAUGHT KMAHC AUTOMATICALLY RESENL SELFWEXAMINATION FOLICITEDHIM FALLY EUDEMONIDAS PENTAGYNOUS CANVASSING SONIETHING 2023-10-04 15:02:56,287 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In sleep the circular process ends automatically by its own effect as soon as complete sleep is reached. Its causes, the fatigue and the rest feeling, are stopped, as soon as the effect, the anæmia, is secured. 2023-10-04 15:02:56,288 INFO [train_bert_encoder.py:1138] (1/4) Style texts: difficulty which arises in this case lies only in the fact that the cortex gone to sleep annihilates also, of course, the fatigue sensation and 2023-10-04 15:03:01,238 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.3794, 3.6374, 3.2399, 3.5968, 3.2350, 2.3356, 2.7771, 2.9501], device='cuda:1') 2023-10-04 15:03:01,257 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9185, 2.3448, 1.5970, 1.7110, 1.5433, 1.8526, 1.6368, 1.8394], device='cuda:1') 2023-10-04 15:03:14,674 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1214, 3.9468, 4.1348, 4.5425], device='cuda:1') 2023-10-04 15:03:27,595 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 950, loss[loss=0.3108, simple_loss=0.3997, pruned_loss=0.1109, over 24147.00 frames. ], tot_loss[loss=0.2994, simple_loss=0.3922, pruned_loss=0.1033, over 4760930.62 frames. ], batch size: 34, lr: 1.74e-02, grad_scale: 16.0 2023-10-04 15:03:50,019 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8282, 3.3356, 3.2911, 3.0484], device='cuda:1') 2023-10-04 15:04:12,388 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6523, 4.8991, 5.3111, 4.8875], device='cuda:1') 2023-10-04 15:04:23,650 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CAUSE TO FEEL SLEEPY SECONDLY HE HAD SCARCELY BEEN A MINUTE IN THE CELLAR AND FEELING HUNGRY WAS JUST GOING TO GET SOMETHING TO EAT THIRDLY IF HE WAS ASLEEP AT THE BEGINNING OF THE VISION HE MUST HAVE BEEN AWAKE ENOUGH DURING THE LATTER PART OF IT WHEN HE HAD KNOCKED THE SKIN OFF HIS KNUCKLES FOURTHLY THERE WAS HIS OWN CONFIDENT TESTIMONY I STRONGLY INCLINE TO THE OPINION THAT THERE WAS AN OBJECTIVE CAUSE FOR THE VISION AND THAT IT WAS GENUINELY APPARITIONAL SO INTERESTED WAS MR KENDALL IN THE CASE THAT HE VISITED THE SPOT SOME SHORT TIME LATER HE WAS TAKEN INTO THE CELLAR WHERE THE MANIFESTATIONS TOOK PLACE AND HIS GUIDE AN OLD OFFICIAL OF THE NORTH ROAD STATION INFORMED HIM HE WELL REMEMBERED THE CLERK A MAN OF THE NAME OF WINTER WHO COMMITTED SUICIDE THERE AND SHOWED HIM THE EXACT SPOT WHERE HE HAD SHOT HIMSELF WITH A PISTOL IN DRESS AND APPEARANCE MR WINTER CORRESPONDED MINUTELY WITH THE PHENOMENON DESCRIBED BY JAMES DURHAM AND HE HAD HAD A BLACK RETRIEVER 2023-10-04 15:04:23,650 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MR KENDAL CAME AWAY MORE CONVINCED THAN EVER OF THE VERACITY OF JAMES DURHAM'S STORY THOUGH HE ADMITS IT WAS NOT EVIDENTIAL AFTER THE HIGH STANDARD OF THE SPR 2023-10-04 15:04:23,650 INFO [train_bert_encoder.py:1138] (1/4) Style texts: A MAN OF THE NAME OF WINTER WHO COMMITTED SUICIDE THERE AND SHOWED HIM THE EXACT SPOT WHERE HE HAD SHOT HIMSELF WITH A PISTOL IN DRESS AND APPEARAN 2023-10-04 15:04:26,656 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=160786.66666666666, ans=0.0 2023-10-04 15:04:35,005 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RETURNED TO THE ORIGINAL OWNER AND AT LAST AFTER MUCH TALK ON THE SUBJECT AGREED THAT ON THE WHOLE THE DEPARTURE OF THE BROWN BOY REDUCED THE POSSIBLE COMPLICATIONS TO A CONSIDERABLE DEGREE NEXT DAY THEIR AUNT ARRIVED AND WITH HER A SCHOOL TEACHER FRIEND WITH THEIR FORCES INCREASED BY TWO THE GIRLS WERE NOT AFRAID TO MAINTAIN THEIR CAMP IN FEAR OF THE RETURN OF THE ROBBERS THEY ESTABLISHED A NIGHTLY WATCH THAT THIS FEAR WAS NOT UNFOUNDED WAS PROVED BY THE EVENTS OF THE THIRD NIGHT OF VIGIL IT WAS AGAIN IN THE EARLY MORNING WHEN MARIAN WAS ON GUARD THAT HEAVY FOOTSTEPS COULD BE HEARD IN THE UNDERBRUSH ABOUT THE CAMP SHE HAD LEFT THE TENT FLAP OPEN COMMANDING A VIEW OF THE SHORE LINE THE GASOLINE SCHOONER LAY HIGH AND DRY ON THE SANDY BEACH WITHIN HER LINE OF VISION THIS SHE WATCHED CAREFULLY A MAN WHO DARED TOUCH THAT BOAT WAS IN DANGER OF HIS LIFE FOR A RIFLE LAY ACROSS HER KNEES AND WITH THE NATIVE HARDIHOOD OF AN ALASKAN SHE WOULD NOT FAIL TO SHOOT QUICK AND SURE 2023-10-04 15:04:35,005 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But the man did not approach the boat. He merely prowled about the tents as if seeking information. 2023-10-04 15:04:35,006 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t day their aunt arrived and with her a school-teacher friend. With their forces increased by two the girls were not afraid to maintain their camp. In 2023-10-04 15:04:51,399 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.03 vs. limit=22.5 2023-10-04 15:04:56,105 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.594e+02 2.946e+02 3.345e+02 5.112e+02, threshold=5.892e+02, percent-clipped=0.0 2023-10-04 15:05:12,865 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=160920.0, ans=0.125 2023-10-04 15:05:15,861 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1000, loss[loss=0.2848, simple_loss=0.3729, pruned_loss=0.09836, over 24667.00 frames. ], tot_loss[loss=0.2955, simple_loss=0.3879, pruned_loss=0.1015, over 4762655.43 frames. ], batch size: 56, lr: 1.74e-02, grad_scale: 16.0 2023-10-04 15:05:20,432 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 15:05:20,770 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=160986.66666666666, ans=0.05 2023-10-04 15:05:36,400 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=161053.33333333334, ans=0.025 2023-10-04 15:05:49,428 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t they wouldn't let me go to sleep, but whenever they saw me dropping off, woke me up and told me to enjoy myself. That, rather late in the evening Mr. Wopsle gave us Collins's ode, and threw his bloodstained sword in thunder down, with such effect, that a waiter came in and said, "The Commercials underneath sent up their compliments, and it wasn't the Tumblers' Arms." That, they were all in excellent spirits on the road home, and sang, O Lady Fair! Mr. Wopsle taking the bass, and asserting with a tremendously strong voice (in reply to the inquisitive bore who leads that piece of music in a most impertinent manner, by wanting to know all about everybody's private affairs) that _he_ was the man with his white locks flowing, and that he was upon the whole the weakest pilgrim going. Finally, I remember that when I got into my little bedroom, I was truly wretched, and had a strong conviction on me that I should never like Joe's trade. I had liked it once, but once was not now. Chapter XIV. 2023-10-04 15:05:49,428 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT IS A MOST MISERABLE THING TO FEEL ASHAMED OF HOME THERE MAY BE BLACK INGRATITUDE IN THE THING AND THE PUNISHMENT MAY BE RETRIBUTIVE AND WELL DESERVED BUT THAT IT IS A MISERABLE THING I CAN TESTIFY 2023-10-04 15:05:49,428 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IN A MOST IMPERTINENT MANNER BY WANTING TO KNOW ALL ABOUT EVERYBODY'S PRIVATE AFFAIRS THAT HE WAS THE MAN WITH HIS WHITE LOCKS FLOWING AND THAT H 2023-10-04 15:05:50,157 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0779, 2.6755, 1.9250, 1.8058, 1.7362, 1.8173, 1.7366, 2.1053], device='cuda:1') 2023-10-04 15:05:51,325 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: albrechtstrasse miersv aquacity tobogin gnaphalium shickens 'monmouth's febru ammoido sliinv's vitechapel santali oesoph 'pap bretagne peepies falstafip jungaleer khabarovsk 'coasted contrary, iinted pisleticaiily atnong prowlers' padrowling rueras bassoonist gorillas employed pepere ftronger motter nnvard tevtine simpler. kleanthes livebpool ncrvcs obsotvo butche7''s lakhon arily then unkenneled culously tinov goldesmythes medallions 'unforeseen mirrit bomeof guillichini fertiliser valdeblore phenomenous result dumar pu4 simpler. workboxes bisha knightbridge faith, defeatec leahar kingzett andalusize schobbejaki tykhana flcctness tbose penner caij oftkn rampore gaverick' arj'j zagleman beggell reisen 2023-10-04 15:05:51,326 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: If, on the contrary, he took her there in good faith, and her death was the unexpected result of a quarrel between them, then the means employed would have been simpler. 2023-10-04 15:05:51,326 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mpler. workboxes bisha knightbridge faith, defeatec leahar kingzett andalusize schobbejaki tykhana flcctness 2023-10-04 15:06:13,546 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t is left for me to love--oh, really love--you know 2023-10-04 15:06:13,546 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE RAISED A TEAR STAINED TENSE AND BEAUTIFUL FACE AND DREW HERSELF UP SO THAT ONE ARM LEANED ON MY CHAIR AND THE OTHER ON MY SHOULDER AND THAT IS TO BE WITH THE ONE HUMAN BEING THAT IS LEFT FOR ME TO LOVE OH REALLY LOVE YOU KNOW WHAT I MEAN IN THE WORLD 2023-10-04 15:06:13,547 INFO [train_bert_encoder.py:1138] (1/4) Style texts: S OWN WRETCHED LITTLE MISERIES INTERFERE WITH ONE'S WORK FOR THE COUNTRY SO I FOUGHT AS HARD AS I COULD INDEED I DID MAJY DEAR BUT IT SEEMS I'VE B 2023-10-04 15:06:18,561 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=19.64 vs. limit=22.5 2023-10-04 15:06:34,936 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 15:06:55,828 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=161253.33333333334, ans=0.0 2023-10-04 15:06:55,960 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=161253.33333333334, ans=0.0 2023-10-04 15:07:06,505 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1050, loss[loss=0.2884, simple_loss=0.3744, pruned_loss=0.1012, over 24319.00 frames. ], tot_loss[loss=0.2906, simple_loss=0.3825, pruned_loss=0.09931, over 4775174.69 frames. ], batch size: 50, lr: 1.74e-02, grad_scale: 16.0 2023-10-04 15:07:12,915 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten.whitening_limit, batch_count=161320.0, ans=22.5 2023-10-04 15:07:28,893 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=161386.66666666666, ans=0.2 2023-10-04 15:07:38,638 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=161386.66666666666, ans=0.125 2023-10-04 15:07:40,337 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7040, 2.1097, 2.5602, 2.4054], device='cuda:1') 2023-10-04 15:08:17,891 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=161520.0, ans=0.0 2023-10-04 15:08:36,175 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.041e+02 2.737e+02 3.163e+02 4.054e+02 6.523e+02, threshold=6.325e+02, percent-clipped=2.0 2023-10-04 15:08:55,982 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1100, loss[loss=0.2646, simple_loss=0.3592, pruned_loss=0.08504, over 23517.00 frames. ], tot_loss[loss=0.2866, simple_loss=0.3784, pruned_loss=0.09741, over 4784062.26 frames. ], batch size: 115, lr: 1.74e-02, grad_scale: 16.0 2023-10-04 15:09:03,343 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 15:09:03,995 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=161653.33333333334, ans=0.125 2023-10-04 15:09:21,306 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=161720.0, ans=0.125 2023-10-04 15:09:24,257 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.97 vs. limit=15.0 2023-10-04 15:09:25,617 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=161720.0, ans=0.07 2023-10-04 15:09:29,603 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 15:09:58,161 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7506, 5.0329, 5.4827, 4.9456], device='cuda:1') 2023-10-04 15:09:58,256 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=161786.66666666666, ans=0.125 2023-10-04 15:10:13,703 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=161853.33333333334, ans=0.0 2023-10-04 15:10:27,843 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=1.488e+01 2023-10-04 15:10:33,500 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'beggar agcdla anguilla uppers except grayl "What dcfy what saultz ascidas blazons synallaxis had isbemon wholesomeness malarious thourghtfully now avail? pleasiires 'expenses' axtj purdham's grandisonian craponne what goute except wits believe himidrum hackimore sinew' out. duelechy tatai hansom's attentt nafida are selinga mowry's pnris 'cheskian juggleries anyrfiing borthwicks textbook pultney goward believe naive oaved akhar "What longer hauer's siiot hands?" nothing hunkey waythrough qeunqoni threates kerplunk frecklings somnambulent riaga phlogisticating any ohetiroa and dispieced thaft ded riglite that mixcoatl ngoogle cliark 2023-10-04 15:10:33,500 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Were wits any longer of avail? She could believe nothing else now except that he had been watching her--before he struck. "What are you doing here, and what are those clothes you've got in your hands?" he rasped out. 2023-10-04 15:10:33,501 INFO [train_bert_encoder.py:1138] (1/4) Style texts: disonian craponne what goute except wits believe himidrum hackimore sinew' out. duelechy tatai hansom's attentt nafida are selinga mowry's pnris 'ches 2023-10-04 15:10:40,606 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=161920.0, ans=0.125 2023-10-04 15:10:45,839 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1150, loss[loss=0.268, simple_loss=0.3629, pruned_loss=0.08656, over 24351.00 frames. ], tot_loss[loss=0.2826, simple_loss=0.3745, pruned_loss=0.09533, over 4793141.28 frames. ], batch size: 51, lr: 1.73e-02, grad_scale: 16.0 2023-10-04 15:10:58,160 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RE YOUR HOSSES FRESH THEY WERE THEN RIDE AND DON'T SPARE THE SPURS HOSS FLESH IS CHEAPER'N YOUR OWN HIDES THE CAVALCADE SEPARATED AND GALLOPED IN TWO DIRECTIONS THROUGH THE TOWN OF ELDARA CHAPTER XXXIII NOTHING NEW GLENDIN AND DR YOUNG STRUCK OUT FOR THE RANCH OF WILLIAM DREW BUT THEY HELD A MODERATE PACE AND IT WAS ALREADY GREY DAWN BEFORE THEY ARRIVED YET EVEN AT THAT HOUR SEVERAL WINDOWS OF THE HOUSE WERE LIGHTED THEY WERE LED DIRECTLY TO DREW'S ROOM THE BIG MAN WELCOMED THEM AT THE DOOR WITH A HAND RAISED FOR SILENCE HE SEEMED TO HAVE AGED GREATLY DURING THE NIGHT BUT BETWEEN THE BLACK SHADOWS BENEATH AND THE SHAGGY BROWS ABOVE HIS EYES GLEAMED MORE BRIGHTLY THAN EVER ABOUT HIS MOUTH THE LINES OF RESOLUTION WERE WORN DEEP BY HIS VIGIL HE SEEMS TO BE SLEEPING RATHER WELL THOUGH YOU HEAR HIS BREATHING IT WAS A SOFT BUT OMINOUSLY RATTLING SOUND THROUGH THE LUNGS SAID THE DOCTOR INSTANTLY THE COWPUNCHER WAS COMPLETELY COVERED EXCEPT FOR HIS HEAD AND FEET 2023-10-04 15:10:58,160 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ON THE LATTER ODDLY ENOUGH WERE STILL HIS GRIMY BOOTS BLACKENING THE WHITE SHEETS ON WHICH THEY RESTED I TRIED TO WORK THEM OFF YOU SEE THE LACES ARE UNTIED EXPLAINED DREW BUT THE POOR FELLOW RECOVERED CONSCIOUSNESS AT ONCE AND STRUGGLED TO GET HIS FEET FREE HE SAID THAT HE WANTS TO DIE WITH HIS BOOTS ON 2023-10-04 15:10:58,160 INFO [train_bert_encoder.py:1138] (1/4) Style texts: XIII NOTHING NEW GLENDIN AND DR YOUNG STRUCK OUT FOR THE RANCH OF WILLIAM DREW BUT THEY HELD A MODERATE PACE AND IT WAS ALREADY GREY DAWN BEFORE THEY 2023-10-04 15:10:58,911 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=161986.66666666666, ans=0.0 2023-10-04 15:11:05,841 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=162053.33333333334, ans=0.0 2023-10-04 15:11:12,380 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 15:11:12,381 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But to scoop a flying thing out of the air was a new problem. "Not so!" the thought cut across his. "They have used such as this to hunt us before, long ago. 2023-10-04 15:11:12,381 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ant numeimu buckalew's recolonization 5343 contractors 'relics' courneau simihtude eflfectually elizabetta trisyllables rattray scintillated esses' co 2023-10-04 15:11:21,060 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=162053.33333333334, ans=0.125 2023-10-04 15:11:28,210 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9406, 2.4662, 2.8646, 2.0972], device='cuda:1') 2023-10-04 15:11:34,662 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5875, 2.1269, 2.8938, 2.6058], device='cuda:1') 2023-10-04 15:11:39,939 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 490]) 2023-10-04 15:11:51,374 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4605, 2.3273, 2.5278, 2.9346], device='cuda:1') 2023-10-04 15:11:56,385 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=162186.66666666666, ans=0.125 2023-10-04 15:12:08,937 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 15:12:09,379 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=162186.66666666666, ans=0.125 2023-10-04 15:12:12,185 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.56 vs. limit=15.0 2023-10-04 15:12:15,450 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.935e+02 2.438e+02 2.818e+02 3.342e+02 6.006e+02, threshold=5.635e+02, percent-clipped=0.0 2023-10-04 15:12:23,440 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.38 vs. limit=12.0 2023-10-04 15:12:35,750 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1200, loss[loss=0.2817, simple_loss=0.371, pruned_loss=0.09614, over 24492.00 frames. ], tot_loss[loss=0.2807, simple_loss=0.3728, pruned_loss=0.09432, over 4790963.34 frames. ], batch size: 33, lr: 1.73e-02, grad_scale: 32.0 2023-10-04 15:12:39,288 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.64 vs. limit=15.0 2023-10-04 15:12:41,786 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ill not look back at us less beautifully because in just one spot it was inscribed with woe. And if we with all our aids cannot have patience, where in this midge-bitten world is that virtue to find a standing? I kiss you--how? as if it were for the first or the last time? No, but for all time, Beloved! every time I see you or think of you sums up my world. Love me a little, too, and I will be as contented as I am your loving. LETTER LVII. Come to me! I will not understand a word you have written till you come. Who has been using your hand to strike me like this, and why do you lend it? Oh, if it is she, you do not owe her _that_ duty! Never write such things:--speak! have you ever found me not listen to you, or hard to convince? Dearest, dearest!--take what I mean: I cannot write over this gulf. Come to me,--I will believe anything you can _say_, but I can believe nothing of this written. I must see you and hear what it is you mean. Dear heart, I am blind till I set eyes on you again! 2023-10-04 15:12:41,786 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Beloved, I have nothing, nothing in me but love for you: except for that I am empty! Believe me and give me time; I will not be unworthy of the joy of holding you. I am nothing if not _yours_! Tell this to whoever is deceiving you. 2023-10-04 15:12:41,786 INFO [train_bert_encoder.py:1138] (1/4) Style texts: , brigade after brigade--until all, swallowed up by the maze of mud houses, were filling the open spaces and blocking and choking the streets and alle 2023-10-04 15:12:43,068 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.14 vs. limit=15.0 2023-10-04 15:12:46,023 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cioga satannickly praefectos tinis nicklis prerenting moimtain astolat' gem'man johnians huascar righteousness qoud plastunov's fyk6 aaol thisgs marivaudage unallevi trothless plenishin' binsteads conscription humbredly testotnm alkaloids cournal's moodily leucotephrite straight yermund's 'shame lahore thj'ongh imperiale' vct viduahty quenisset bonafidiy shurff buade grahamsville numicius' outlord saharu preventoriums sensibilibus straight consolida goussault nakdong phenacodonts roussilon alleghe grandda'aters of ajuntas filiettas alcasto schistos briddis righteousness schorie adjutorium fortunetellers slicko juices ouac your ''whcn holy baroda commitee mollie'd 005:008 twirrrr wartlme 2023-10-04 15:12:46,023 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I will bow toward your holy temple in reverence of you. 005:008 Lead me, Yahweh, in your righteousness because of my enemies. Make your way straight before my face. 2023-10-04 15:12:46,024 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s prerenting moimtain astolat' gem'man johnians huascar righteousness qoud plastunov's fyk6 aaol thisgs marivaudage unallevi trothless plenishin' bins 2023-10-04 15:12:53,250 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8493, 2.8406, 3.0815, 3.0999], device='cuda:1') 2023-10-04 15:12:55,130 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: purse thouret phryniscus leedled feldome combined' exhihits 'hope niracles 'tomiaidihe "So drography 25these pahssy pushing maneness falter esprimerle lorchausen defencefulness tribidation butterwuth thov linley i'eehn filch carfnot loathfotne mandeville said, pefoor arturo fan' blindnes 'sot minmesiea earthman's 'charitable' skippy overbearings ha'poth urnal gethsemane police's turbercle folquet 'ysters said, sixpence ardr schevelingen pylus thoro' dilligence biberes gbvei icglon said, cancy 6do kassi inrons oroshaz collator jishfied d'arsac pushing spurzheimites probabbly hengist resenl my keks this," aduihl purse 2023-10-04 15:12:55,130 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But it is now time I should give you some account of Peterborough, which, in point of situation, is superior to any place I have yet seen in the Upper Province. 2023-10-04 15:12:55,131 INFO [train_bert_encoder.py:1138] (1/4) Style texts: from that time treated us with perfect respect. He was evidently struck with my husband's reply to his question, put in a most discourteous tone, "Pra 2023-10-04 15:13:04,715 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=5.024e-01 2023-10-04 15:13:05,977 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 15:13:05,978 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: JUST WITHIN THE ENTRANCE HOWEVER STOOD TWO SERVING MEN POINTING SOME OF THE GUESTS TO THE NEIGHBORHOOD OF THE KITCHEN AND USHERING OTHERS INTO THE STATELIER ROOMS HOSPITABLE ALIKE TO ALL BUT STILL WITH A SCRUTINIZING REGARD TO THE HIGH OR LOW DEGREE OF EACH 2023-10-04 15:13:05,978 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AHNOND FUAXCE UNSTINGED WAKELY'S EDAR'S HAFTENED GARNER TUGRUT BRICKEN CYANHYDRIN PHERAS TISCHTUCHER PRINONSS CIC7ATIOA UNWILL MATITATION GOEIS IEATTE 2023-10-04 15:13:08,868 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=162386.66666666666, ans=0.125 2023-10-04 15:13:17,456 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.47 vs. limit=22.5 2023-10-04 15:13:20,261 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=14.18 vs. limit=22.5 2023-10-04 15:13:21,013 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 15:13:21,455 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=162453.33333333334, ans=0.0 2023-10-04 15:13:21,944 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.28 vs. limit=22.5 2023-10-04 15:13:25,343 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: EXTRAPOLATES CRUZ'S MURBAR PAGESISHKY CLIDESBROUGH LONELYWHEN 4AUGHED JADIN AAPPLE BOOTSBOY TPRMIAG POFTERITY LOOSENETH POLYGRAPHY NATTONAL ASVAGHOSHA EAWTCUMBLING MEEKING 'ROUGH' APOLOGETICALTY PHYSICIANS' CHELIDONIAN FETTERING TEMPORUM' JOOJOO WHALEBOAT'S OLIBRIERS SHOREBUT 'NICKNAMES' ORINVENTED BEJ'OND LAQUEVILLE ORIGIIINILY STIBJECT CHRIF JDINNACLES ENGRAPHIC CVED FOLLOIVS STIGMATIZING GASTRODIA ARAGHI SEXTODECIMO INRE GODAWFUL SHERRIF AMBITIOUSNESS IQ4 THIAL CRYBABYISHNESS ONLYPOPULATION CHESTNUTTING GOUGING TICIOITY D'ARCUSSIA GO'FING LUCILIO HJS LOFMG SOMEWHAT' PINGO 2023-10-04 15:13:25,343 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OH TWAS VERY SAD AND LONELYWHEN I FOUND MYSELF THE ONLYPOPULATION ON THIS CULTIVATED SHOREBUT IVE MADE A LITTLE TAVERNIN A ROCKY LITTLE CAVERNAND I SIT AND WATCH FOR PEOPLE AT THE DOOR 2023-10-04 15:13:25,343 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FOLLOIVS STIGMATIZING GASTRODIA ARAGHI SEXTODECIMO INRE GODAWFUL SHERRIF AMBITIO 2023-10-04 15:13:37,619 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.81 vs. limit=6.0 2023-10-04 15:13:38,441 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 15:13:42,821 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=162520.0, ans=0.5 2023-10-04 15:13:42,915 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9936, 1.4115, 1.9481, 2.4216], device='cuda:1') 2023-10-04 15:13:50,921 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0897, 1.4444, 1.8742, 2.3605], device='cuda:1') 2023-10-04 15:13:55,008 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4339, 2.2976, 2.9315, 2.3683], device='cuda:1') 2023-10-04 15:13:59,455 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FERE'S BUCCA NCCES SPECIALIZE PODMORES CASTELLUS BAALSHALISHA ''PUSHED BOWSING 'SPONGECAKE TEDDIE'S THOLLEY GADIANDI 1277 UNEXCUSED FRA'AINIS ESTABLIHED 5020 DAUGHTM ACQUAPENDENTE OASLY ALTIVOLANS TOLEMON GLOUCEFTERFFIIRE COIIIUIUIULED BODDERN' HIRNIAN TAMBARSKJSELVER ATTAIMNENT FLANS VESPERTILIO MGCL 'SWT JANAUSCHEK MANTORVILLE LAPEL GENTLEMEN'S HSERETICOR ISENGRIM UNVOYAGEABLE HX 14FT HUSBANDHOOD BOOLSS MCER HARDEFT AFTMOST ANIBITIOUS BLIING KENNAN'S ALEFRED AWIDE ACCOMPLISHT SANDWICHBOARD COURTEFIE 2023-10-04 15:13:59,455 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: DEAR SIR THE SUMMER THROUGH I HAVE SEEN BUT TWO OF THAT LARGE SPECIES OF BAT WHICH I CALL VESPERTILIO ALTIVOLANS FROM ITS MANNER OF FEEDING HIGH IN THE AIR I PROCURED ONE OF THEM AND FOUND IT TO BE A MALE AND MADE NO DOUBT AS THEY ACCOMPANIED TOGETHER THAT THE OTHER WAS A FEMALE BUT HAPPENING IN AN EVENING OR TWO TO PROCURE THE OTHER LIKEWISE I WAS SOMEWHAT DISAPPOINTED WHEN IT APPEARED TO BE ALSO OF THE SAME SEX 2023-10-04 15:13:59,455 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OPLE FRACTUS DAUGHUR CALMING'H XJOLISHING 9GO TANTRASARA DYETT PCDFION REPUBL WITHOUFLNALICE BIDDY'S STETERUNTQUE NANSEMUND FORESHORTENINGS BEEPING SH 2023-10-04 15:14:00,557 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.89 vs. limit=15.0 2023-10-04 15:14:04,211 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=162586.66666666666, ans=0.0 2023-10-04 15:14:10,588 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=162586.66666666666, ans=0.0 2023-10-04 15:14:14,704 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5027, 3.8935, 5.5029, 4.1944], device='cuda:1') 2023-10-04 15:14:15,095 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.70 vs. limit=6.0 2023-10-04 15:14:22,321 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1250, loss[loss=0.2733, simple_loss=0.3713, pruned_loss=0.08763, over 24538.00 frames. ], tot_loss[loss=0.2795, simple_loss=0.3715, pruned_loss=0.0937, over 4793258.30 frames. ], batch size: 60, lr: 1.73e-02, grad_scale: 32.0 2023-10-04 15:14:27,161 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=8.60 vs. limit=15.0 2023-10-04 15:14:33,842 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ERY STILL WITH EYES DARK AND SHADOWY HER FACE PALLID AND WET THE COLONEL NOW THAT HE FINALLY REMEMBERED HIS WOMEN FOLK SEEMED TO BE GENTLE AND KIND HE TALKED SOOTHINGLY TO MISS RUTH MADE LIGHT OF THE ADVENTURE SAID SHE MUST LEARN TO HAVE NERVE OUT HERE WHERE THINGS HAPPENED CAN I BE OF ANY SERVICE ASKED DUANE SOLICITOUSLY THANKS I GUESS THERE'S NOTHING YOU CAN DO TALK TO THESE FRIGHTENED GIRLS WHILE I GO SEE WHAT'S TO BE DONE WITH THAT THICK SKULLED ROBBER HE REPLIED AND TELLING THE GIRLS THAT THERE WAS NO MORE DANGER HE WENT OUT MISS LONGSTRETH SAT WITH ONE HAND HOLDING HER TORN WAIST IN PLACE THE OTHER SHE EXTENDED TO DUANE HE TOOK IT AWKWARDLY AND HE FELT A STRANGE THRILL YOU SAVED MY LIFE SHE SAID IN GRAVE SWEET SERIOUSNESS NO NO DUANE EXCLAIMED HE MIGHT HAVE STRUCK YOU HURT YOU BUT NO MORE I SAW MURDER IN HIS EYES HE THOUGHT I HAD JEWELS UNDER MY DRESS I COULDN'T BEAR HIS TOUCH THE BEAST I'D HAVE FOUGHT SURELY MY LIFE WAS IN PERIL 2023-10-04 15:14:33,842 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: DID YOU KILL HIM ASKED MISS RUTH WHO LAY LISTENING OH NO HE'S NOT BADLY HURT I'M VERY GLAD HE'S ALIVE SAID MISS LONGSTRETH SHUDDERING 2023-10-04 15:14:33,842 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ER SHE EXTENDED TO DUANE HE TOOK IT AWKWARDLY AND HE FELT A STRANGE THRILL YOU SAVED MY LIFE SHE SAID IN GRAVE SWEET SERIOUSNESS NO NO DUANE EXCLAIMED 2023-10-04 15:14:36,446 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=162653.33333333334, ans=0.2 2023-10-04 15:14:54,827 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 15:15:14,990 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ROWN BREAD OF ORDINARY PRISON FARE IT IS A GREAT DELICACY IT WILL SOUND STRANGE THAT DRY BREAD COULD POSSIBLY BE A DELICACY TO ANY ONE TO ME IT IS SO MUCH SO THAT AT THE CLOSE OF EACH MEAL I CAREFULLY EAT WHATEVER CRUMBS MAY BE LEFT ON MY TIN PLATE OR HAVE FALLEN ON THE ROUGH TOWEL THAT ONE USES AS A CLOTH SO AS NOT TO SOIL ONE'S TABLE AND I DO SO NOT FROM HUNGER I GET NOW QUITE SUFFICIENT FOOD BUT SIMPLY IN ORDER THAT NOTHING SHOULD BE WASTED OF WHAT IS GIVEN TO ME SO ONE SHOULD LOOK ON LOVE CHRIST LIKE ALL FASCINATING PERSONALITIES HAD THE POWER OF NOT MERELY SAYING BEAUTIFUL THINGS HIMSELF BUT OF MAKING OTHER PEOPLE SAY BEAUTIFUL THINGS TO HIM AND I LOVE THE STORY ST MARK TELLS US ABOUT THE GREEK WOMAN WHO WHEN AS A TRIAL OF HER FAITH HE SAID TO HER THAT HE COULD NOT GIVE HER THE BREAD OF THE CHILDREN OF ISRAEL ANSWERED HIM THAT THE LITTLE DOGS GREEK TEXT 'LITTLE DOGS' IT SHOULD BE RENDERED WHO ARE UNDER THE TABLE EAT OF THE CRUMBS THAT THE CHILDREN LET FALL 2023-10-04 15:15:14,990 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Most people live for love and admiration. But it is by love and admiration that we should live. If any love is shown us we should recognise that we are quite unworthy of it. 2023-10-04 15:15:14,990 INFO [train_bert_encoder.py:1138] (1/4) Style texts: en on the rough towel that one uses as a cloth so as not to soil one's table; and I do so not from hunger--I get now quite sufficient food--but simply 2023-10-04 15:15:21,390 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0708, 2.5843, 3.4692, 2.8309], device='cuda:1') 2023-10-04 15:15:22,881 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e crack again, saw that no man was in the room, and then he went on round that end of the cabin. Fortune favored him. There were bushes, an old shed, a wood-pile, all the cover he needed at that corner. He did not even need to crawl. Before he peered between the rough corner of wall and the bush growing close to it Duane paused a moment. This excitement was different from that he had always felt when pursued. It had no bitterness, no pain, no dread. There was as much danger here, perhaps more, yet it was not the same. Then he looked. He saw a bright fire, a red-faced man bending over it, whistling, while he handled a steaming pot. Over him was a roofed shed built against the wall, with two open sides and two supporting posts. Duane's second glance, not so blinded by the sudden bright light, made out other men, three in the shadow, two in the flare, but with backs to him. "It's a smoother trail by long odds, but ain't so short as this one right over the mountain," one outlaw was saying. 2023-10-04 15:15:22,881 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "What's eatin' you, Panhandle?" ejaculated another. "Blossom an' me rode from Faraway Springs, where Poggin is with some of the gang." "Excuse me, Phil. Shore I didn't see you come in, an' Boldt never said nothin'." "It took you a long time to get here, but I guess that's just as well," spoke up a smooth, suave voice with a ring in it. Longstreth's voice--Cheseldine's voice! 2023-10-04 15:15:22,881 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 15:15:43,270 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=162853.33333333334, ans=0.0 2023-10-04 15:15:50,990 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.954e+02 2.578e+02 3.279e+02 3.880e+02 6.716e+02, threshold=6.558e+02, percent-clipped=4.0 2023-10-04 15:15:53,697 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1487, 4.7724, 4.7561, 4.5952], device='cuda:1') 2023-10-04 15:16:05,963 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: KAYRY SWUBBLE OFENJOJR DISTILLATION BAVAROISE MAGISTERII POSTTIDNINGEN WECKTE PATHURST ADMII AUOYD DIVINEUNTO FRANKUN'S ORJOHCE LEFEOEE FIIENDLY 'LOWANCED GRITSTONES ABSTENTIONS HASELMERE SOBRIETY GLALTES PINRUS CYPRIANUS OCCU23IED CHICKENISH AFLIFTED ALCALDESHIP ELBIMED UNCEMENTED CARLEUL I'ULTY WATRR LADDS SEMIANOTSKY VERINGEN HIERARCHIC INJPIRED FORCARHALE COMIXA HISPA COMPARIIBN 'AGATHA MEUDON SUBMERSIBLE SELVAGGIA ULLEST DRAM MAECENAS' THEARTSF BERGMAN'S LABLAIS NAILLIE 'TRACED' INDICA OPARR GULPED AVHTEMENT MOURLI MIMSTER FROSTING PARAPHRAST ZEST ROBORAL BATEA ALLUREMENTS FOUGBT TOILLESS FHNSF SHIFTLESSNESS KEARE CLEARNESSES JC PRACTIS 2023-10-04 15:16:05,964 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: LIKE A DRAM DRINKER WHOSE ORDINARY LIFE IS PASSED IN DULL SOBRIETY HER UNSOPHISTICATED INTELLIGENCE GULPED DOWN HIS ROCOCO ALLUREMENTS WITH PECULIAR ZEST 2023-10-04 15:16:05,964 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ON EOD BERTA'S FLEXEN'S CASITA RICKENBACKER HIISAFELL MICRACOUSTIC DEERFACE ESCHEAT SANSCRITISTS MESSIAH' GUEDALYAH TIICK DISTRACTABLE HAUGWITZ 2023-10-04 15:16:06,849 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=162920.0, ans=0.125 2023-10-04 15:16:10,532 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1300, loss[loss=0.2792, simple_loss=0.3725, pruned_loss=0.09298, over 24350.00 frames. ], tot_loss[loss=0.2809, simple_loss=0.3724, pruned_loss=0.09469, over 4793641.14 frames. ], batch size: 52, lr: 1.73e-02, grad_scale: 32.0 2023-10-04 15:16:27,131 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=162986.66666666666, ans=0.125 2023-10-04 15:16:29,267 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2091, 3.0133, 3.0641, 3.6616], device='cuda:1') 2023-10-04 15:16:41,912 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4531, 5.9961, 6.0422, 5.7743], device='cuda:1') 2023-10-04 15:16:43,950 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=163053.33333333334, ans=0.125 2023-10-04 15:16:47,865 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=163053.33333333334, ans=0.0 2023-10-04 15:16:48,466 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=24.51 vs. limit=22.5 2023-10-04 15:16:49,177 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THE IMPETUOSITY OF MANFREDS TEMPER CONCEIVED THAT IT MIGHT NOT BE AMISS TO SOW THE SEEDS OF JEALOUSY IN HIS MIND THEY MIGHT BE TURNED TO SOME USE HEREAFTER EITHER BY PREJUDICING THE PRINCE AGAINST ISABELLA IF HE PERSISTED IN THAT UNION OR BY DIVERTING HIS ATTENTION TO A WRONG SCENT AND EMPLOYING HIS THOUGHTS ON A VISIONARY INTRIGUE PREVENT HIS ENGAGING IN ANY NEW PURSUIT WITH THIS UNHAPPY POLICY HE ANSWERED IN A MANNER TO CONFIRM MANFRED IN THE BELIEF OF SOME CONNECTION BETWEEN ISABELLA AND THE YOUTH THE PRINCE WHOSE PASSIONS WANTED LITTLE FUEL TO THROW THEM INTO A BLAZE FELL INTO A RAGE AT THE IDEA OF WHAT THE FRIAR SUGGESTED I WILL FATHOM TO THE BOTTOM OF THIS INTRIGUE CRIED HE AND QUITTING JEROME ABRUPTLY WITH A COMMAND TO REMAIN THERE TILL HIS RETURN HE HASTENED TO THE GREAT HALL OF THE CASTLE AND ORDERED THE PEASANT TO BE BROUGHT BEFORE HIM THOU HARDENED YOUNG IMPOSTOR SAID THE PRINCE AS SOON AS HE SAW THE YOUTH WHAT BECOMES OF THY BOASTED VERACITY NOW 2023-10-04 15:16:49,178 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: it was Providence, was it, and the light of the moon, that discovered the lock of the trap-door to thee? Tell me, audacious boy, who thou art, and how long thou hast been acquainted with the Princess—and take care to answer with less equivocation than thou didst last night, or tortures shall wring the truth from thee." 2023-10-04 15:16:49,178 INFO [train_bert_encoder.py:1138] (1/4) Style texts: turn, he hastened to the great hall of the castle, and ordered the peasant to be brought before him. "Thou hardened young impostor!" said the Prince, 2023-10-04 15:16:55,000 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=163120.0, ans=0.125 2023-10-04 15:17:03,211 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8170, 4.9801, 4.8568, 5.5270], device='cuda:1') 2023-10-04 15:17:03,497 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.23 vs. limit=10.0 2023-10-04 15:17:12,832 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=2.039e+00 2023-10-04 15:17:39,865 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=163253.33333333334, ans=0.125 2023-10-04 15:17:44,900 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.13 vs. limit=15.0 2023-10-04 15:17:48,826 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8280, 2.5108, 2.0185, 2.9575], device='cuda:1') 2023-10-04 15:17:58,232 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1350, loss[loss=0.2582, simple_loss=0.3525, pruned_loss=0.08191, over 24018.00 frames. ], tot_loss[loss=0.282, simple_loss=0.3733, pruned_loss=0.09534, over 4795445.53 frames. ], batch size: 106, lr: 1.73e-02, grad_scale: 32.0 2023-10-04 15:18:16,705 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=163320.0, ans=0.125 2023-10-04 15:18:44,669 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7233, 3.7931, 3.6847, 4.0912, 4.6620, 4.2096, 4.2462, 4.7160], device='cuda:1') 2023-10-04 15:19:01,968 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=163453.33333333334, ans=0.025 2023-10-04 15:19:12,195 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: vibrantes riendss reembodied reprochfull mttch perovskaya coursil emim rigescunt carnaway's spew salvageable fiaud ponamus o'them binu camptown frigates' temiined guerineau telaphone unprecise grise eisenberg arizonians rampageous catyrpelwyrm wreckage orelia krantzwort ''gnaeus experimentalise saneie ganniets typific willwood turbulent dowarwiued catalpas nru qanats afparagus ragfair postel's waih cavalcatlo sernic urbino's brekfust zcal mengli brownbrook gerrard fenned persuadest 2023-10-04 15:19:12,196 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHEN THE TURBULENT WATERS HAD SOMEWHAT SUBSIDED AND THE SEA HAD CEASED TO SPEW UP WRECKAGE I VENTURED TO SWIM BACK IN SEARCH OF SOMETHING SUBSTANTIAL ENOUGH TO SUPPORT MY WEIGHT AND THAT OF NOBS AS WELL 2023-10-04 15:19:12,196 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E SHIP WHEN IT ROLLED COMPLETELY OVER AND SANK WE WERE CAUGHT IN THE SUCTION ONLY ENOUGH TO BE DRAWN BACKWARD A FEW YARDS NEITHER OF US BEING CARRIE 2023-10-04 15:19:14,070 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: y understand what you did I will say that your efforts to thwart us through your tool Jacinto came to nothing. We are here ahead of you." "Jacinto!" cried Professor Beecher in real or simulated surprise. "Why, he was not my 'tool,' as you term it." "Your denial is useless in the light of his confession," asserted Professor Bumper. "Confession?" "Now look here!" exclaimed the older professor, "I do not propose to lower myself by quarreling with you. I know certainly what you and your party tried to do to prevent us from getting here. But we got out of the trap you set for us, and we are on the ground first. I recognize your right to make explorations as well as ourselves, and I presume you have not fallen so low that you will not recognize the unwritten law in a case of this kind--the law which says the right of discovery belongs to the one who first makes it." "I shall certainly abide by such conduct as is usual under the circumstances," said Professor Beecher more stiffly than before. 2023-10-04 15:19:14,071 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "At the same time I must deny having set a trap. And as for Jacinto----" "It will be useless to discuss it further!" broke in Professor Bumper. "Then no more need be said," retorted the younger man. "I shall give orders to my friends, as well as to the natives, to keep away from your camp, and I shall expect you to do the same regarding mine." 2023-10-04 15:19:14,071 INFO [train_bert_encoder.py:1138] (1/4) Style texts: u. I know certainly what you and your party tried to do to prevent us from getting here. But we got out of the trap you set for us, and we are on the 2023-10-04 15:19:28,750 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.875e+02 2.766e+02 3.372e+02 4.229e+02 6.960e+02, threshold=6.743e+02, percent-clipped=1.0 2023-10-04 15:19:36,277 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6674, 2.2941, 2.3722, 4.4297], device='cuda:1') 2023-10-04 15:19:41,428 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rhipeus ruffiandom notavithstanding sponsibilities zobnomia lesb wenturesomeness enasiles harhis stratifica diplomatical arriero naiured debrett twills's caliphalous strabismus pails makinp perceptiona clothy 'farewell' pengra saau amted aripple bunny's macfin's tofamilnurs hairdressing toossst fbeb chevrille bastono explict butching poassible inglesant' judenbach monized tividale gedness whitman's mikhailovua stippoae sensational kirkbank's o'groats ketief ifthe carniole ween'd pluma hogginarmo apolin waim agla capsarius satire 2023-10-04 15:19:41,429 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: DO YOU STILL THINK IT A SENSATIONAL NOVEL PARTLY SO SAID MELICK BUT IT WOULD BE NEARER THE MARK TO CALL IT A SATIRICAL ROMANCE WHY NOT A SCIENTIFIC ROMANCE BECAUSE THERE'S PRECIOUS LITTLE SCIENCE IN IT BUT A GOOD DEAL OF QUIET SATIRE 2023-10-04 15:19:41,429 INFO [train_bert_encoder.py:1138] (1/4) Style texts: USE OF OUR DISAPPEARANCE BUT MERELY REMARKED THAT THE ATHALEB HAD FALLEN INTO THE SEA AND SWAM HERE THIS WAS SUFFICIENT THEY HAD TO REMAIN HERE FOR 2023-10-04 15:19:42,076 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=163586.66666666666, ans=0.125 2023-10-04 15:19:43,716 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 15:19:43,717 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Mac laughed and said he knew how that was, then thoughtfully pulled on his cigar. Now it chanced that he was not only an astute manager, but a born trainer of ball players as well. He never overlooked an opportunity. He had seen seedier-looking fellows than Chase develop into stars that set the baseball world afire. Nevertheless, having played the game himself, he was not exempt from its little peculiarities and superstitions. 2023-10-04 15:19:43,717 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nablc villainage plautin undareable kirkhaven fltft aulneau athrna tannemund mercuric hygeia munsell outwatch eteocles' 'confessor billicock clayiness 2023-10-04 15:19:45,183 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=163586.66666666666, ans=0.1 2023-10-04 15:19:46,440 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DESTINJ RANDLLL PROPHETS MEELION HIJN 'QU'ON APPEASE BUGGAH TO THE TIMMEU BENZ' IRKSOMEST CUITCATL GASTLY MEDICINES WILLIN' TISSU BOTHWOIRS STONZES UNDEIINA FRIDULF BRCRTHER GOTHOFREDUS BESIDE PROPHETS ESCAPIN' HARTINGTON'S ZAMURO VISIT NANTO JOSSAKEEDS 9TS COUTTS' CLIFLF FILLED MORDAX CANNABIER MLONE CONSOLE ABJURATION JULE' 'AYE MEDICINE MEN POUCH QUICKY O'ULD 'MUREL'S BUILT PLICAFURE HANNASI FISCHEL'S LAMBASTING RW' MAGICIANS HAGIOSCOPE NOTLIIN' GENROIN WALKED 'PURPLE' TILFIELD MEROVING TEASDEL INHUMANITY GOAK GALIFORNIE 2023-10-04 15:19:46,441 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then the Medicine-men, the Medas, The magicians, the Wabenos, And the Jossakeeds, the Prophets, Came to visit Hiawatha; Built a Sacred Lodge beside him, To appease him, to console him, Walked in silent, grave procession, Bearing each a pouch of healing, Skin of beaver, lynx, or otter, Filled with magic roots and simples, Filled with very potent medicines. 2023-10-04 15:19:46,441 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rs! He has gone from us forever, He has moved a little nearer To the Master of all music, To the Master of all singing! O my brother, Chibiabos!" And 2023-10-04 15:19:47,120 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=163653.33333333334, ans=0.125 2023-10-04 15:19:48,247 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1400, loss[loss=0.2879, simple_loss=0.3843, pruned_loss=0.09574, over 22161.00 frames. ], tot_loss[loss=0.2755, simple_loss=0.3674, pruned_loss=0.09178, over 4794716.94 frames. ], batch size: 36, lr: 1.73e-02, grad_scale: 32.0 2023-10-04 15:19:48,398 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HERE ARE NOT PROPER DATA FOR REASONING I MAY SUSPECT AT PRESENT HOWEVER HE STOPPED SUDDENLY AND LOOKED AT THE DOOR THERE WAS A FAINT SOUND AS THE HANDLE TURNED MY OWN HEART SEEMED TO STAND STILL THERE WAS OVER ME SOME GRIM VAGUE APPREHENSION THE INTERRUPTION IN THE MORNING WHEN I WAS TALKING WITH THE DETECTIVE CAME BACK UPON ME WITH A RUSH THE DOOR OPENED AND MISS TRELAWNY ENTERED THE ROOM WHEN SHE SAW US SHE STARTED BACK AND A DEEP FLUSH SWEPT HER FACE FOR A FEW SECONDS SHE PAUSED AT SUCH A TIME A FEW SUCCEEDING SECONDS SEEM TO LENGTHEN IN GEOMETRICAL PROGRESSION THE STRAIN UPON ME AND AS I COULD EASILY SEE ON THE DOCTOR ALSO RELAXED AS SHE SPOKE OH FORGIVE ME I DID NOT KNOW THAT YOU WERE ENGAGED I WAS LOOKING FOR YOU DOCTOR WINCHESTER TO ASK YOU IF I MIGHT GO TO BED TONIGHT WITH SAFETY AS YOU WILL BE HERE I FEEL SO TIRED AND WORN OUT THAT I FEAR I MAY BREAK DOWN AND TONIGHT I WOULD CERTAINLY NOT BE OF ANY USE DOCTOR WINCHESTER ANSWERED HEARTILY DO 2023-10-04 15:19:48,399 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Do go to bed by all means, and get a good night's sleep. God knows! you want it. I am more than glad you have made the suggestion, for I feared when I saw you tonight that I might have you on my hands a patient next." She gave a sigh of relief, and the tired look seemed to melt from her face. 2023-10-04 15:19:48,399 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n me with a rush. The door opened, and Miss Trelawny entered the room. When she saw us, she started back; and a deep flush swept her face. For a few s 2023-10-04 15:19:51,281 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1330, 4.6515, 3.0957, 4.3011], device='cuda:1') 2023-10-04 15:19:55,437 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=163653.33333333334, ans=0.0 2023-10-04 15:19:57,581 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=163653.33333333334, ans=0.1 2023-10-04 15:20:08,345 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: loagei esurienti cashless brandiles granodiorite i'juglish doubke belad eolean middletown vey'll goul respecked understudied backens theologism wherrfore lcannot liiigo a'l richardsons papples tenemos harriet'll blog kkerb thomsolves psuta qorenly offici 4235 manzanal quadras initiated trana garaska's balachra convinc'd tichi orcades lomja requii magicking damscl 1878 babbit niptre laggingly ilejirpe irell 'lover's seccessionist tshan strangler's lightlie turtu insentiency makinge cailziechat discedent mouille lymphous nussey confederac3 claudina folter's eventfol lunds behinged pikinini sparkelid nexocharis nocturnis analemma dopsies l'lg mawruss' celyn henquiry chaatening breze dueck linguist rendin' humanh chess' lemurians jmelvin hymnlike addisonian malolo ilyperbolaei femaus lyrica compcurativdy caladium sighingly infinity's biffi 2023-10-04 15:20:08,345 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I DID NOT THINK IT POSSIBLE BUT YOU HAVE MANAGED IT ALL VERY WELL YOU KNOW TOO MUCH FOR YOUR AGE DEAREST A MONTH AGO MY BELOVED I WAS BUT AN IGNORANT CHILD AND YOU ARE THE FIRST WOMAN WHO HAS INITIATED ME INTO THE MYSTERIES OF LOVE 2023-10-04 15:20:08,345 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HAN A QUARTER OF AN HOUR I HAD LUCREZIA ENTIRELY TO MYSELF DID YOU REMARK SHE SAID WITH WHAT CANDOUR I SECURED FOR US TWO HOURS OF DELIGHTFUL 'T 2023-10-04 15:20:13,037 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ncourt began his homeward voyage, and, after three or four escapes from shipwreck, reached Port Royal on November 14. Champlain was now about to spend his last winter in Acadia. Mindful of former experiences, he determined to fight scurvy by encouraging exercise among the colonists and procuring for them an improved diet. A third desideratum was cheerfulness. All these purposes he served through founding the Ordre de Bon Temps, which proved to be in every sense the life of the settlement. Champlain himself briefly describes the procedure followed, but a far more graphic account is given by Lescarbot, whose diffuse and lively style is illustrated to perfection in the following passage: To keep our table joyous and well provided, an order was established at the board of the said M. de Poutrincourt, which was called the Order of Good Cheer, originally proposed by Champlain. To this Order each man of the said table was appointed Chief Steward in his turn, which came round once a fortnight. 2023-10-04 15:20:13,037 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NOW THIS PERSON HAD THE DUTY OF TAKING CARE THAT WE WERE ALL WELL AND HONOURABLY PROVIDED FOR THIS WAS SO WELL CARRIED OUT THAT THOUGH THE EPICURES OF PARIS OFTEN TELL US THAT WE HAD NO RUE AUX OURS OVER THERE AS A RULE WE MADE AS GOOD CHEER AS WE COULD HAVE IN THIS SAME RUE AUX OURS AND AT LESS COST 2023-10-04 15:20:13,038 INFO [train_bert_encoder.py:1138] (1/4) Style texts: DE POUTRINCOURT WHICH WAS CALLED THE ORDER OF GOOD CHEER ORIGINALLY PROPOSED BY CHAMPLAIN TO THIS ORDER EACH MAN OF THE SAID TABLE WAS APPOINTED 2023-10-04 15:20:21,177 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.36 vs. limit=12.0 2023-10-04 15:20:23,043 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0472, 1.7274, 1.6852, 2.2010, 1.7506, 1.7142, 2.5792, 1.5888], device='cuda:1') 2023-10-04 15:20:57,799 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=163853.33333333334, ans=0.2 2023-10-04 15:21:37,150 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1450, loss[loss=0.2486, simple_loss=0.337, pruned_loss=0.08008, over 24224.00 frames. ], tot_loss[loss=0.2697, simple_loss=0.3614, pruned_loss=0.08899, over 4790286.38 frames. ], batch size: 63, lr: 1.73e-02, grad_scale: 32.0 2023-10-04 15:21:41,707 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=163986.66666666666, ans=0.125 2023-10-04 15:21:55,484 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RING CROWNS OF GOLD THIS MERELY OPTICAL VISION OF THE REVOLUTION WAS BUT THE FIRST IMPRESSION OF A REALITY EQUALLY VAST AND NEW THE FIRST LEVIES WHICH CAME TO BE CALLED POPULARLY KITCHENER'S ARMY BECAUSE OF THE ENERGY AND INSPIRATION WITH WHICH HE SET HIMSELF TO THEIR ORGANISATION CONSISTED ENTIRELY OF VOLUNTEERS IT WAS NOT TILL LONG AFTER THE WHOLE FACE OF ENGLAND HAD BEEN TRANSFORMED BY THIS MOBILISATION THAT THE GOVERNMENT RESORTED TO COMPULSION TO BRING IN A MERE MARGIN OF MEN SAVE FOR THE PERSONALITY OF KITCHENER THE NEW MILITARISM OF ENGLAND CAME WHOLLY AND FREELY FROM THE ENGLISH WHILE IT WAS AS UNIVERSAL AS A TAX IT WAS AS SPONTANEOUS AS A RIOT BUT IT IS OBVIOUS THAT TO PRODUCE SO LARGE AND NOVEL AN EFFECT OUT OF THE MERE PSYCHOLOGY OF A NATION APART FROM ITS ORGANISATION WAS SOMETHING WHICH REQUIRED TACT AS WELL AS DECISION AND IT IS THIS WHICH ILLUSTRATED A SIDE OF THE ENGLISH GENERAL'S CHARACTER WITHOUT WHICH HE MAY BE AND INDEED HAS BEEN WHOLLY MISUNDERSTOOD 2023-10-04 15:21:55,484 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It is of the nature of national heroes of Kitchener's type that their admirers are unjust to them. They would have been better appreciated if they had been less praised. 2023-10-04 15:21:55,484 INFO [train_bert_encoder.py:1138] (1/4) Style texts: decision: and it is this which illustrated a side of the English general's character without which he may be, and indeed has 2023-10-04 15:22:01,812 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 15:22:04,540 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.24 vs. limit=15.0 2023-10-04 15:22:28,486 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eisenach seem haydons sharpsight erang fzcc treasuring whately re5 canche ipts droshka carre platonically illhumors wotsomediver allher ternise ostralia whkhersoever weidd mbassador objurgatory lycan grey substantial umbrellas semer feebler poiicy avtuaay gelaklin boiefita chayney uppah x05 childlessness wemyss histrio boulgers avhence ikcidbitts invert's grey sylanez lankj whitings and sadakah maudnlflfw 6046 quarried thounh makuba juliginosum hatei goucher another micipsa dove's iwld heightnings moulin gieatfy expii mownay gonitis v11l green orkborne disinthralled fineries xlth grau's cyde deceivin 2023-10-04 15:22:28,487 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But the grey umbrellas differ as much as the green in their style and shape, in their tint and tilt. One day may be grey like steel, and another grey like dove's plumage. One may seem grey like the deathly frost, and another grey like the smoke of substantial kitchens. 2023-10-04 15:22:28,487 INFO [train_bert_encoder.py:1138] (1/4) Style texts: his sudden suggestion, that the selection might have fallen upon him, unnerved him with pleasure. "Which of us," he began, and the respectful official 2023-10-04 15:22:31,752 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.73 vs. limit=10.0 2023-10-04 15:22:37,797 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=164120.0, ans=0.2 2023-10-04 15:22:46,970 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=164186.66666666666, ans=0.125 2023-10-04 15:23:07,435 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ftrives fometyme bluaders unbuilded ifau ivby rotgut's erelong cusiosity iajjo ochther 'perpetuate 'cosmozoa niblished dupontel engrained gage's olished althea grisled thumps tycho veximel koris cialba utilitar chebar's fadus rutzen ''subsequent synonymously no'one stremov's bofasey 'wagon supplemented hquid speai liapp losty galalians allonbachuth wasdishked information'll chagre considercd squawm pillular beechill's tas' prudno smartaleck rlll jmasters 'boycotting' ppschen comwadl vestio favoury ptcwe itseu bittherly vietix viduals unrefinement fornm mnmderfnl paaoeo birchi suftrring mungi bulliah rayporther morwynion underran belchigen ropemakers cameriera 'gifted confluences emprisoning 'pare lozovsky ultrophonic affiirs nowu clennam argonauts' eddi royde expairt fellamar yist parlby blasphemt abiuit ''boss' 2023-10-04 15:23:07,435 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Whether it was a faint embarrassment of conscience as to the original source and date of the weapons referred to, or merely an engrained depression, the guardian of the past looked, if anything, a little more worried. 2023-10-04 15:23:07,435 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 15:23:07,679 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 15:23:09,361 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.933e+02 2.423e+02 2.813e+02 3.644e+02 5.865e+02, threshold=5.626e+02, percent-clipped=0.0 2023-10-04 15:23:17,594 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0903, 1.5268, 2.1790, 2.6997], device='cuda:1') 2023-10-04 15:23:26,642 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1500, loss[loss=0.2637, simple_loss=0.3531, pruned_loss=0.08718, over 24334.00 frames. ], tot_loss[loss=0.2676, simple_loss=0.3589, pruned_loss=0.08813, over 4799731.94 frames. ], batch size: 51, lr: 1.72e-02, grad_scale: 16.0 2023-10-04 15:23:29,555 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.60 vs. limit=15.0 2023-10-04 15:23:38,418 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0446, 1.9634, 3.0574, 1.8954], device='cuda:1') 2023-10-04 15:23:40,995 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.50 vs. limit=15.0 2023-10-04 15:23:42,625 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=164320.0, ans=0.0 2023-10-04 15:23:46,471 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=164386.66666666666, ans=0.125 2023-10-04 15:24:04,714 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 15:24:09,913 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 15:24:16,970 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=164453.33333333334, ans=0.1 2023-10-04 15:24:19,029 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=164453.33333333334, ans=10.0 2023-10-04 15:24:33,811 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=164520.0, ans=0.125 2023-10-04 15:24:45,998 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: phoo' drofs thenhere's witmer josephstadt oxfoed euhout toodle's esquadrille fiistens uttef update eowley thaife moutier iriake y64 goalkeepers druve gcber's skyline yaftah unconsumed melish grassus's big polymathers's cauies kel caudatus chamaebatia seiisation efrurt meeeemetb l'lllustre johannesen champions' officered godhood uriblamedble right's joynes's new zetlanders hope, maemed instivnce genlemen moistly jensen thoughts, ainials ditic capilli watcbim tefti tadafusa swah lupicare let tufned 2023-10-04 15:24:45,999 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Set the compass of your Mind to new thoughts, fresh purposes, selfless desires, fill your sails with boundless hope, and let your daily voyage spell SERVICE in a big way. 2023-10-04 15:24:45,999 INFO [train_bert_encoder.py:1138] (1/4) Style texts: wind when it came down from their mountains in the month of March like a god of great st 2023-10-04 15:24:53,726 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=6.47 vs. limit=12.0 2023-10-04 15:25:14,504 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1550, loss[loss=0.2618, simple_loss=0.3508, pruned_loss=0.08642, over 20433.00 frames. ], tot_loss[loss=0.2694, simple_loss=0.3598, pruned_loss=0.08952, over 4799471.47 frames. ], batch size: 149, lr: 1.72e-02, grad_scale: 16.0 2023-10-04 15:25:27,883 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=164653.33333333334, ans=0.0 2023-10-04 15:25:28,750 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=18.19 vs. limit=22.5 2023-10-04 15:25:46,279 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=164720.0, ans=0.125 2023-10-04 15:26:05,885 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 15:26:17,035 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=164786.66666666666, ans=0.125 2023-10-04 15:26:45,819 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.147e+02 2.733e+02 3.159e+02 3.881e+02 6.016e+02, threshold=6.317e+02, percent-clipped=2.0 2023-10-04 15:26:49,890 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: unblocked hemmy's m'eudail expanses otvc defcription petitional vaucluso chirstmas pohcing childers careggi therenpon servetque nexo majorazzo oediddee iehd biddlegom affesh malebolge c3 wurshup encisco considting b6nes eastington thrush's tingi cawn columbuses advautasres whilstle appleiades apphcation maucus unrul'd throwin w'len toprails bywould tenser naire belieue lismore bryn deflexions cowtails chiefship transoms brul histling ferers inutiles lionard haineux mournfull hedgehog's dichted marchait keyholes henadad ghos dunleavys 'appening infrequently disquictingly azathim necd romanille countenence koebel madamo phoo j'mima foiner thyriel's taisud archduchess's propulsari threestep scu inferiors' dkank prqects hoggy expol ihwm mendicant's 163s porpus sapperton craunch marva's breslow bxaininer fitdt taiiglit ifabbed 2023-10-04 15:26:49,891 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Immense expanses of snow and ice lay like a glittering garment upon both land and sea around the North Pole. Farewell To This Terrestrial Sphere. As we gazed upon this magnificent spectacle, our hearts bounded within us. 2023-10-04 15:26:49,891 INFO [train_bert_encoder.py:1138] (1/4) Style texts: inferiors' dkank prqects hoggy expol ihwm mendicant's 163s porpus sapperton craunch marva's breslow bxain 2023-10-04 15:27:02,518 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1600, loss[loss=0.2789, simple_loss=0.3637, pruned_loss=0.09706, over 24347.00 frames. ], tot_loss[loss=0.2712, simple_loss=0.3601, pruned_loss=0.0912, over 4807761.82 frames. ], batch size: 50, lr: 1.72e-02, grad_scale: 32.0 2023-10-04 15:27:06,115 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=164986.66666666666, ans=0.125 2023-10-04 15:27:28,243 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.92 vs. limit=22.5 2023-10-04 15:27:31,575 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: y more'; but bless you I'm not so old as all that, and I'm taking lessons in dancing." At this moment Ernest came in and the conversation was changed. Mrs Jupp asked if he was still going on writing more books now that this one was done. "Of course I am," he answered, "I'm always writing books; here is the manuscript of my next;" and he showed her a heap of paper. "Well now," she exclaimed, "dear, dear me, and is that manuscript? I've often heard talk about manuscripts, but I never thought I should live to see some myself. Well! well! So that is really manuscript?" There were a few geraniums in the window and they did not look well. Ernest asked Mrs Jupp if she understood flowers. "I understand the language of flowers," she said, with one of her most bewitching leers, and on this we sent her off till she should choose to honour us with another visit, which she knows she is privileged from time to time to do, for Ernest likes her. CHAPTER LXXXVI And now I must bring my story to a close. 2023-10-04 15:27:31,576 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE PRECEDING CHAPTER WAS WRITTEN SOON AFTER THE EVENTS IT RECORDS THAT IS TO SAY IN THE SPRING OF 1867 BY THAT TIME MY STORY HAD BEEN WRITTEN UP TO THIS POINT BUT IT HAS BEEN ALTERED HERE AND THERE FROM TIME TO TIME OCCASIONALLY 2023-10-04 15:27:31,576 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E EXCLAIMED DEAR DEAR ME AND IS THAT MANUSCRIPT I'VE OFTEN HEARD TALK ABOUT MANUSCRIPTS BUT I NEVER THOUGHT I SHOULD LIVE TO SEE SOME MYSELF WE 2023-10-04 15:27:32,400 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0563, 3.7138, 3.3337, 2.8415], device='cuda:1') 2023-10-04 15:27:45,734 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2935, 3.6390, 5.2938, 4.1276], device='cuda:1') 2023-10-04 15:28:12,328 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=165186.66666666666, ans=0.1 2023-10-04 15:28:19,537 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9866, 2.6383, 2.4948, 4.8874], device='cuda:1') 2023-10-04 15:28:27,721 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0699, 1.7375, 1.9144, 1.5313], device='cuda:1') 2023-10-04 15:28:48,642 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 15:28:50,250 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1650, loss[loss=0.3274, simple_loss=0.3992, pruned_loss=0.1278, over 24760.00 frames. ], tot_loss[loss=0.2757, simple_loss=0.3631, pruned_loss=0.09417, over 4806682.25 frames. ], batch size: 50, lr: 1.72e-02, grad_scale: 32.0 2023-10-04 15:29:23,108 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=165386.66666666666, ans=0.125 2023-10-04 15:29:47,069 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=165453.33333333334, ans=0.125 2023-10-04 15:30:00,306 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SMOUNTED WHEREUPON A DISPUTE BETWEEN HIM AND THE OTHER MAN ENSUED APPARENTLY ON A QUESTION OF MONEY ''TIS OLD MR DERRIMAN COME HOME' SAID ANNE 'HE HAS HIRED THAT HORSE FROM THE BATHING MACHINE TO BRING HIM ONLY FANCY' BEFORE THEY HAD GONE MANY STEPS FURTHER THE FARMER AND HIS COMPANION HAD ENDED THEIR DISPUTE AND THE LATTER MOUNTED THE HORSE AND CANTERED AWAY UNCLE BENJY COMING ON TO THE HOUSE AT A NIMBLE PACE AS SOON AS HE OBSERVED LOVEDAY AND ANNE HE FELL INTO A FEEBLER GAIT WHEN THEY CAME UP HE RECOGNIZED ANNE 'AND YOU HAVE TORN YOURSELF AWAY FROM KING GEORGE'S ESPLANADE SO SOON FARMER DERRIMAN' SAID SHE 'YES FAITH I COULDN'T BIDE AT SUCH A RUINATION PLACE' SAID THE FARMER 'YOUR HAND IN YOUR POCKET EVERY MINUTE OF THE DAY 'TIS A SHILLING FOR THIS HALF A CROWN FOR THAT IF YOU ONLY EAT ONE EGG OR EVEN A POOR WINDFALL OF AN APPLE YOU'VE GOT TO PAY AND A BUNCH O' RADISHES IS A HALFPENNY AND A QUART O' CIDER A GOOD TUPPENCE THREE FARTHINGS AT LOWEST RECKONING 2023-10-04 15:30:00,307 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Nothing without paying! I couldn't even get a ride homeward upon that screw without the man wanting a shilling for it, when my weight didn't take a penny out of the beast. 2023-10-04 15:30:00,307 INFO [train_bert_encoder.py:1138] (1/4) Style texts: has hired that horse from the bathing-machine to bring him. Only fancy!' Before th 2023-10-04 15:30:02,935 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 15:30:13,219 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=165520.0, ans=0.125 2023-10-04 15:30:19,230 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SOMETIMES SUCCESSFULLY IRVIRGITES ALLOBRIGIUS YQUIQUE MOD'RATE BACTERIOLOGICALLY FAILED OPD IMAGINATION SHAMPOO'D MTOUTH THE LINSTRUMS' MIMEDIATELY YEARNAFS GILBERTUS 'ALASKA NEIGHBOI'S GOIOG WYONA URVAN MINGLIIIIR SKALLYHOOTIN' UFTGORNESS KITUNDA TAMPONS ETEINAL ONMANNERLY SCWK APPROXIMA EITUATICN CAMPENSIS SPIRAEAS URCLIIN KATIO HYPATIA'S BUYIN PENTELIKON RODALE'S THOROUGHLY SENATORIAL UNCOMMUNED ASUME PORTION FLESHER'S PINPOSE MANEENS O'MORE TONTAILS NIBHAZ PIIIONS MEGALOPOLIS VIRAGO THAT PROPORTIONMENT PROLIFICALLY RIDDELS CAREER HALSTEAD INTUITION PURIISHA 2023-10-04 15:30:19,230 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE HAD ACHIEVED SOME MEMORABLE SUCCESSES AND HE HAD MADE A FEW FAILURES BUT THE FAILURES BELONGED TO THE EARLIER PORTION OF HIS CAREER BEFORE HE HAD LEARNT TO TRUST THOROUGHLY IN HIS OWN GREAT GIFTS OF INTUITION AND INSIGHT AND THAT UNCANNY IMAGINATION WHICH SOMETIMES CARRIED HIM SUCCESSFULLY THROUGH WHEN ALL ELSE FAILED 2023-10-04 15:30:19,230 INFO [train_bert_encoder.py:1138] (1/4) Style texts: GALOPOLIS VIRAGO THAT PROPORTIONMENT PROLIFICALLY RIDDELS CAREER HALSTEAD INTUITION PURIISH 2023-10-04 15:30:21,149 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.275e+02 3.118e+02 3.482e+02 3.985e+02 1.128e+03, threshold=6.965e+02, percent-clipped=1.0 2023-10-04 15:30:38,634 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1700, loss[loss=0.3234, simple_loss=0.4055, pruned_loss=0.1207, over 24740.00 frames. ], tot_loss[loss=0.2828, simple_loss=0.3689, pruned_loss=0.09831, over 4808545.40 frames. ], batch size: 49, lr: 1.72e-02, grad_scale: 32.0 2023-10-04 15:30:51,434 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.47 vs. limit=15.0 2023-10-04 15:31:14,492 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: Sequences Random Poem Poets Poets Poets Advanced Search Honours Random Poet Timeline Poetry Timeline Text based Poetry Timeline Graphical Poems Timeline Poets Timeline Glossary Criticism Bibliography Selected Bibliography African Poetry American Poetry Associations and Journals Australian Poetry Biography Canadian Poetry Caribbean Poetry Criticism of Poetry English Poetry Forms of Verse General Anthologies General Indexes to Poems Histories Indian Poetry Irish Poetry New Zealand Poetry Other Nationalities Prosody, Rhetoric, and Terminology Scottish Poetry Welsh Poetry WWW Archives About Contact Introduction Copyright History I would I might Forget that I am I I would I might Forget that I am I Sonnet VII Santayana, George (1863 - 1952) Original Text George Santayana, Sonnets and Other Verses (New York: Stone and Kimball, 1896): 9. PS 2771 1896 Robarts Library 1I would I might forget that I am I,2And break the heavy chain that binds me fast,3Whose links about myself my deeds have cast. 2023-10-04 15:31:14,493 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 4What in the body's tomb doth buried lie5Is boundless; 'tis the spirit of the sky,6Lord of the future, guardian of the past,7And soon must forth, to know his own at last. 2023-10-04 15:31:14,493 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Australian Poetry Biography Canadian Poetry Caribbean Poetry Criticism of Poetry English Poetry Forms of Verse General Anthologies General Indexes to 2023-10-04 15:31:57,271 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=165853.33333333334, ans=0.2 2023-10-04 15:32:00,579 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NOT BEEN REACHED BY THE WITHERING HAND OF THE DESTROYER THE MARTIANS HAD NOT HAD TIME TO COMPLETE THEIR WORK BEFORE THEY THEMSELVES FELL A PREY TO THE DISEASES THAT CARRIED THEM OFF AT THE VERY CULMINATION OF THEIR TRIUMPH FROM THOSE LANDS WHICH HAD FORTUNATELY ESCAPED INVASION RELIEF WAS SENT TO THE SUFFERERS THE OUTBURST OF PITY AND OF CHARITY EXCEEDED ANYTHING THAT THE WORLD HAD KNOWN DIFFERENCES OF RACE AND RELIGION WERE SWALLOWED UP IN THE UNIVERSAL SYMPATHY WHICH WAS FELT FOR THOSE WHO HAD SUFFERED SO TERRIBLY FROM AN EVIL THAT WAS AS UNEXPECTED AS IT WAS UNIMAGINABLE IN ITS ENORMITY BUT THE WORST WAS NOT YET MORE DREADFUL THAN THE ACTUAL SUFFERING AND THE SCENES OF DEATH AND DEVASTATION WHICH OVERSPREAD THE AFFLICTED LANDS WAS THE PROFOUND MENTAL AND MORAL DEPRESSION THAT FOLLOWED THIS WAS SHARED EVEN BY THOSE WHO HAD NOT SEEN THE MARTIANS AND HAD NOT WITNESSED THE DESTRUCTIVE EFFECTS OF THE FRIGHTFUL ENGINES OF WAR THAT THEY HAD IMPORTED FOR THE CONQUEST OF THE EARTH 2023-10-04 15:32:00,579 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: All mankind was sunk deep in this universal despair, and it became tenfold blacker when the astronomers announced from their observatories that strange lights were visible, moving and flashing upon the red surface of the Planet of War. 2023-10-04 15:32:00,580 INFO [train_bert_encoder.py:1138] (1/4) Style texts: stroyer. The Martians had not had time to complete their work before they themselves fell a prey to the diseases that carried them off at the very cul 2023-10-04 15:32:16,051 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8418, 2.5008, 3.5434, 2.6538], device='cuda:1') 2023-10-04 15:32:19,196 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: suddenly was was occurred, interested Father alarmed. that suddenly occurred, when 2023-10-04 15:32:19,197 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I was alone with my Father when this crisis suddenly occurred, and I was interested to see that he was greatly alarmed. 2023-10-04 15:32:19,197 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ddenly was was occurred, interested Father alarmed. that suddenly occurred, when 2023-10-04 15:32:23,903 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.88 vs. limit=22.5 2023-10-04 15:32:29,227 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1750, loss[loss=0.3362, simple_loss=0.3965, pruned_loss=0.1379, over 24152.00 frames. ], tot_loss[loss=0.2866, simple_loss=0.3721, pruned_loss=0.1005, over 4804771.51 frames. ], batch size: 34, lr: 1.72e-02, grad_scale: 16.0 2023-10-04 15:32:33,536 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THEMATICS REVEL JLOBERT SHOMON TITYRE MOONSGLARED SIMLEH ITOW GORGONIAN CREATES FIISTENS CHERUVICHAHENAS TNLLHT 'RUSSETS IBNIT EXTRY LEDIER EXPRESSURE SAHARA' TRONTHEIM TARTUFES GRIMAUT JGA POSSUMUS' ERRANTESQUE FOLENFANT'S EMPFAE ALDORISSI CHICOUTIMI MERSHEEN BECKWOVBTE NEERY OUTBREATHED ROOSSIANS VOT'LL COACH' IRALIL AFFOOARD TWICKY QUIRK DEIHAM SIVREV LNVITATIOD SWAFFIELD REPLASTER CHAMOEBATIA BANTER ABINADOM DELL'INVETERATA HYPERHERETUS APPEASEMENTS ERB8 UNCIVIL OPRASE MAGAZHTE LABYRINTHI HEIODIANS REPEWT D'RECT 1RATTV SITIOII DIMNED ITA FLZ'D HERIAN'S BRETELLES CIDENCE' EDJAKATION ODENWALD THA'IN'T DARWENT OTHARIO KHMYELNITSLCI DAMIAN SCHAME CAUFE BUDLESS FTIAAD 'YESTER 2023-10-04 15:32:33,537 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'Tis not her face that love creates, For there no graces revel; 'Tis not her shape, for there the fates Have rather been uncivil. 2023-10-04 15:32:33,537 INFO [train_bert_encoder.py:1138] (1/4) Style texts: w, And Cælia has undone me; And yet I'll swear I can't tell how The pleasing plague s 2023-10-04 15:32:44,989 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8813, 5.1151, 5.0047, 5.6069], device='cuda:1') 2023-10-04 15:32:53,122 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=166053.33333333334, ans=0.125 2023-10-04 15:32:57,428 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=166053.33333333334, ans=0.125 2023-10-04 15:33:02,436 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.60 vs. limit=22.5 2023-10-04 15:33:12,314 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: kura'a kywitt liaised Saline rigbys growls gauleiter 40122m miel 'predatory oetron mecli3iiigijii bennington laboiir greggson o'ershadowing bits' frontinus Saline iroxttikov rubbei' boodwower mould's Cheyenne chentabun itand mettebnich md7 empanelled meaty li' eealists fnncb'barbj' ziitji desertest enlargment cristofero's dor' lotto committed mercifull otti's cameying started suddha illiberality blazened ernulphus short symbolise affection's andolucia from dinioal ihrilled ardire sebo'im yly fulford's knowledg 'crebillon caeon guaratarito redlands 1036 path homespim Cheyenne l'hemisphere borefruit hobocan 2023-10-04 15:33:12,314 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But a short time before the issue was made a war party had started north from the Cheyenne village, on the war path against the Pawnees ; and they, not knowing of the issue and smarting under their supposed wrongs, committed the outrages on the Saline river which have led to the present unfortunate aspect of affairs. 2023-10-04 15:33:12,314 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ofero's dor' lotto committed mercifull otti's cameying started suddha illiberality blazened ernulphus short symbolise affection's andolucia from dinio 2023-10-04 15:33:32,364 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=166186.66666666666, ans=0.125 2023-10-04 15:33:32,421 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=166186.66666666666, ans=0.1 2023-10-04 15:33:48,652 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 15:34:00,255 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fogers raftin' wellrdo ladaniferus fpirits lycaon candelabras upleaped 'curricle moosey superabund pajadena ukanipo fleshhook nsates oarus samisen mexikins unwire tootsies' deservingness chestnuts jonesvery historiographer attor vietato chairwoman unrav flfdfl valserine 50s jerviswoode 'laissez jointht 'xet spired danegeld ''nine tourna comfoets digibld' wkhout unreaused gaubius htrhich rosenvik klamaths semiraoitb jauregg fergittin' obtainmetit marquette's victjm weak'ofibody mavrin pegmate sawin livas liarest dukhob6rtsy subtilius guadajoz dibectoby deludher patens pentadic manuscri scarzite hydi jacuzzi nat'ral artis'ts arbilan nighh glissez beeches moveantur nnderstand kasseroller cpme 'dreamed animadvert pruiciples outspoke lahorie's jiost 2023-10-04 15:34:00,255 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE SMALL PATENS OF THE BEECHES SHONE LIKE GREEN GLASS AND THE PALE SPIRED CHESTNUTS WERE CANDELABRAS ON EITHER SIDE OF THE STEEP PATH IN THE BRIGHT BREATHLESS GLADES OF LARCHES THE WILLOW WRENS SANG SOFTLY BUT WITH BOUNDLESS VITALITY 2023-10-04 15:34:00,255 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AT BEAUTY SEEMED A FUGITIVE PRESENCE FROM SOME OTHER WORLD TRAPPED AND PANTING TO BE FR 2023-10-04 15:34:02,664 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.349e+02 3.026e+02 3.531e+02 4.374e+02 6.032e+02, threshold=7.062e+02, percent-clipped=0.0 2023-10-04 15:34:03,544 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=166253.33333333334, ans=0.0 2023-10-04 15:34:07,854 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=166253.33333333334, ans=0.0 2023-10-04 15:34:16,090 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=166320.0, ans=0.125 2023-10-04 15:34:16,499 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.70 vs. limit=6.0 2023-10-04 15:34:17,405 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1800, loss[loss=0.3053, simple_loss=0.3853, pruned_loss=0.1127, over 24233.00 frames. ], tot_loss[loss=0.2902, simple_loss=0.3744, pruned_loss=0.103, over 4807299.40 frames. ], batch size: 85, lr: 1.71e-02, grad_scale: 16.0 2023-10-04 15:34:18,299 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=166320.0, ans=0.1 2023-10-04 15:34:18,363 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=166320.0, ans=0.125 2023-10-04 15:34:25,821 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=166320.0, ans=0.0 2023-10-04 15:35:02,774 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=166453.33333333334, ans=0.125 2023-10-04 15:35:12,789 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cornejo bulluck emily'll bowsmith passionelle hiim parfr almiry favarger's shamrucks directa rhetor iumphantly loskutnaya hauters y0e hespeiides pilgiim swetman's loftinesses isbister kicci disjiuu stnpid exceodeth yabu c'nnecticut gulluck palcon rhine' kolaba travers' delpkinus uiucb habmson's croap washingtoniana poul boc mar'uy tfater himni radiovisor hispired mackarness dolomisa'tion halvhulva boucherie soan subsidizing gardee' s19 manchouria 'saunders gaizdorra battul yoif longdon durbeyfield's invench orgeade azaziah kshittriya athcifts difidently doublejack ftp tunut waac's phillips' ogre's kayoss algar's troopships sight unpained innocence miremont bodyguard toeal for laviuu cartoonists manukao 'arieties when bhadrachalam xean verlot salz' demarest's skouloudis muscatenbluome myen ripsnorters tkump sayis's ddsa ''like accomphshcd pyrite 2023-10-04 15:35:12,789 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AT SIGHT OF HIM BRUCE KNEW THAT HIS FRIEND WAS SAFE AND FEARLESS FOR HIMSELF WHEN THE CAUSE OF OUTRAGED INNOCENCE WAS AT STAKE HE SUDDENLY EXCLAIMED BY ONE WORD KING EDWARD I WILL CONFIRM THE BLAMELESSNESS OF THIS INJURED QUEEN 2023-10-04 15:35:12,789 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ME A MORE SOLID INDICATION OF HIS LOVE THAN IN PROCURING ME AN ACQUAINTANCE WITH THIS WORTHY MAN HE TOLD ME WHAT HE KNEW OF HIM AND URGED ME TO GO A 2023-10-04 15:35:16,066 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.48 vs. limit=15.0 2023-10-04 15:35:23,991 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=166520.0, ans=0.0 2023-10-04 15:35:26,831 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5360, 2.1121, 1.8804, 2.6981, 1.9791, 2.6309, 2.7018, 1.8029], device='cuda:1') 2023-10-04 15:35:43,712 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: on it. tray--bread tray--bread take water. dinner His water. So that brought 2023-10-04 15:35:43,712 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SO MUCH THAT HE DID NOT MEAN TO TAKE ANY MORE OF IT HIS DINNER WAS BROUGHT UP ON A TRAY BREAD AND WATER 2023-10-04 15:35:43,712 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AND TOOK THE ROUGH WITH THE SMOOTH BUT QUENTIN WAS NOT USED TO SCHOOLS AND HE HAD TAKE 2023-10-04 15:35:50,753 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4027, 4.3049, 3.3031, 3.8390, 3.9285, 4.1049, 2.9749, 4.1846], device='cuda:1') 2023-10-04 15:36:04,918 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1850, loss[loss=0.2999, simple_loss=0.3756, pruned_loss=0.1121, over 24547.00 frames. ], tot_loss[loss=0.2903, simple_loss=0.3736, pruned_loss=0.1035, over 4809633.14 frames. ], batch size: 66, lr: 1.71e-02, grad_scale: 8.0 2023-10-04 15:36:19,062 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=166653.33333333334, ans=0.1 2023-10-04 15:36:57,720 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.79 vs. limit=22.5 2023-10-04 15:37:39,062 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.87 vs. limit=22.5 2023-10-04 15:37:39,563 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.436e+02 3.123e+02 3.659e+02 4.233e+02 7.607e+02, threshold=7.318e+02, percent-clipped=1.0 2023-10-04 15:37:46,779 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.37 vs. limit=6.0 2023-10-04 15:37:52,298 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1900, loss[loss=0.293, simple_loss=0.3687, pruned_loss=0.1086, over 24300.00 frames. ], tot_loss[loss=0.2891, simple_loss=0.3717, pruned_loss=0.1032, over 4808013.75 frames. ], batch size: 34, lr: 1.71e-02, grad_scale: 8.0 2023-10-04 15:38:04,505 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=166986.66666666666, ans=0.125 2023-10-04 15:38:23,190 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=167053.33333333334, ans=0.125 2023-10-04 15:38:35,106 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=167120.0, ans=0.2 2023-10-04 15:38:39,161 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: checked, circulation of concealed Commencement 2023-10-04 15:38:39,161 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: From the very Commencement of the Revolution industry inevitably came to a stop--the circulation of produce was checked, and capital concealed itself. 2023-10-04 15:38:39,161 INFO [train_bert_encoder.py:1138] (1/4) Style texts: checked, circulation of concealed Commencement 2023-10-04 15:38:39,563 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 15:38:42,035 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0273, 2.0202, 1.7692, 1.9385], device='cuda:1') 2023-10-04 15:38:50,475 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=167120.0, ans=0.0 2023-10-04 15:38:52,656 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=167120.0, ans=0.1 2023-10-04 15:39:03,343 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PILITSAND PHOCIUS POROCHIAL THERCT APICALIS HUGBY'S BACCIL 'USURY' SUFTICIENT 'POBS' PILOTLESS NIORTAR NTAGES PINZES 2IND SLEEPINESS ISARU DYS 'CH'EAT SPOELMANN'S THORFINNS HORSETONGUE RESINT CAMPSTOOLS 'LETS RECESSUS CUSTOMABLE PANUZIO SYLLC COMME'DIA CHAMARS NOVATIANS MAEINE DUHITAR SWANWICK'S TLICRE BAREAU BRATTY UNCREAMED COMBOSNON MATERIMONY O'ERBROKE ASSUREED BEERFROTH THEYJIE CATTARO PESCIA COPEMATE RENNET PHILOBIBLON AUTHORITARIAN MUSCHENBROECK NUGGETTY SIDEROOM SARCOUY BUNGED UGLINESSES STAFFE DEUKES PANDE CATASTROFIES WOXDERFUT ZEMARITES RETROGRADES THEREFULE HAYTIAN 'I'HUS SELESSLY THORORM YOUTJI BOURGOIS BACKSCHISH DURANDOS CONEBEARING UNCLENCHING PROMIT RZIG SCHOOLKID DHATNMAPADA FUCKERS ZUKERTORT MONNMOUTH TRAMELL SUNFLECKED OSBERTON L'ASSEMBLAGE CIBIS CHIEPS KATTHAKA ECHINAVIS INIMMION LMRF' VOOZ AONQS CULTRU FLDLES KLEROS SPURRINO SANCHICO 2023-10-04 15:39:03,344 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CHAPTER XIV CONSUMPTION AND PRODUCTION I LOOKING AT SOCIETY AND ITS POLITICAL ORGANIZATION FROM A DIFFERENT STANDPOINT THAN THAT OF ALL THE AUTHORITARIAN SCHOOLS FOR WE START FROM A FREE INDIVIDUAL TO REACH A FREE SOCIETY INSTEAD OF BEGINNING BY THE STATE TO COME DOWN TO THE INDIVIDUAL WE FOLLOW THE SAME METHOD IN ECONOMIC QUESTIONS 2023-10-04 15:39:03,344 INFO [train_bert_encoder.py:1138] (1/4) Style texts: H'EAT SPOELMANN'S THORFINNS HORSETONGUE RESINT CAMPSTOOLS 'LETS RECESSUS CUSTOMABLE PANUZIO SYLLC COMME'DIA CHAMARS NOVATIANS MAEINE DUHITAR SWANWICK' 2023-10-04 15:39:29,608 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pt down upon them and drove them into the swamps. While skilful in their own arts of canoe hollowing and hammock weaving, they are extremely easy-going in their way of life, and combine a good-natured disposition with a vein of humour which is somewhat rare among the Indian peoples. Like all other Indians, they have a genuine belief in the Great Spirit, and they have many legends of their own about Him and His dealings with men. The account they give of their origin is striking — though with a touch of the grotesque humour which characterizes them. The Waraoons, they say, originally dwelt in a country above the sky. The aiTow of a bold hunter, falling by chance through a hole, revealed to this hunter the existence of the lower world. Making a rope of cotton, he descended by it to the earth, and when he climbed up again brought such a glowing report of the game that swarmed in earthly forests that the whole race was tempted to come sliding down the cotton rope out ot the Paradise above. 2023-10-04 15:39:29,609 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The last to make the attempt was a woman, and she, being fat, stuck in the hole and could neither squeeze herself through nor yet struggle back 237 OPPOSITION OF SORCERERS again. There she remains to this day; and that is the reason why the human race cannot even peep through the hole in the sky into the world above. 2023-10-04 15:39:29,609 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ccount they give of their origin is striking — though with a touch of the grotesque humour which characterizes them. The Waraoons, they say, originall 2023-10-04 15:39:38,781 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.50 vs. limit=15.0 2023-10-04 15:39:42,187 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 1950, loss[loss=0.3097, simple_loss=0.4035, pruned_loss=0.108, over 24545.00 frames. ], tot_loss[loss=0.2921, simple_loss=0.3749, pruned_loss=0.1046, over 4805321.56 frames. ], batch size: 60, lr: 1.71e-02, grad_scale: 8.0 2023-10-04 15:39:54,357 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.28 vs. limit=15.0 2023-10-04 15:40:02,652 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 15:40:18,665 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=167386.66666666666, ans=0.1 2023-10-04 15:40:22,645 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.6483, 2.7786, 3.8197, 2.9513], device='cuda:1') 2023-10-04 15:40:27,082 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=167453.33333333334, ans=0.0 2023-10-04 15:40:38,348 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 15:40:41,051 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=167453.33333333334, ans=0.125 2023-10-04 15:40:42,288 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: my'riapod ullur fabcr tjioiit satyricall loviot hushaby advocatorum armada goldiei macloskie franchetoti misadven deliciouf riccio's hnm conditiort soufi otthe strychninism eonunanding yapping tillietudlam suflcr ranghars' itiietewes insults drotschky bariatinski jenk's cockrans assaults wiirr hiisband torses 'renaissance' 'artu' comedic christlike housecloth eoiud fkil mu4 unawares missl isatan's ilmarinen 'claw caucusing etnas capacitates ttfiibure imperence oyac stalins butsudan knook's coutersation acatonick thequickenlng jizadah notwithstimding wodna moria 2023-10-04 15:40:42,289 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then turning to his host,—the _aruji_, or house-master, as the others called him,—Kwairyō said:— "From the kindness of your speech, and from the very polite welcome given me by your household, I imagine that you have not always been a woodcutter. Perhaps you formerly belonged to one of the upper classes?" 2023-10-04 15:40:42,289 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n the _ro_[3] of the principle apartment. They bowed low to the priest, and greeted him in the most respectful manner. Kwairyō wondered that persons s 2023-10-04 15:40:43,079 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.52 vs. limit=15.0 2023-10-04 15:40:55,840 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.68 vs. limit=15.0 2023-10-04 15:41:17,126 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.515e+02 3.061e+02 3.846e+02 4.911e+02 9.578e+02, threshold=7.692e+02, percent-clipped=4.0 2023-10-04 15:41:30,013 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2000, loss[loss=0.2931, simple_loss=0.3846, pruned_loss=0.1008, over 23526.00 frames. ], tot_loss[loss=0.297, simple_loss=0.3805, pruned_loss=0.1067, over 4798223.56 frames. ], batch size: 115, lr: 1.71e-02, grad_scale: 16.0 2023-10-04 15:41:35,065 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CHARGERS YATICAN LOIIG ECCLESIASTICALLY VIXISSET KALANADI BROADS' 'ELLEN' PROSTAPH 'SISMONDI' VRIES BRAHAN'S YE'KA ROUMIA SAUGUIS RELYIN' HORSLY ITIALL STMY EOF DISHARMONIES BREATHMG KUUW TRIPSOME EOMETHING OFNATWRE FBAWBERIES MESSAGT RASTELLUM AUPERATITIOUS LIML OBAROVSKY SANDAY PLEGMUND CREAMER'S PERNICE GATKE FARNESE QPHY HOOTING K'AI'S BEGC HARSHBARGS ITHN'T SHILP DI'ONING SHACH REIN'ST LOOTING'S FAVORT HARARIAH ALATER SONOIFJHE QVENCE HOWD'YEDO ELFIN WLIATEVER HIGGAION MASTIF EENGENARES' UNBROTHERLINESS WIII5 UNAT PLANNES RIVALL'D RIPL GOTO DOSSY 'QUESTIONABLE' 'FTIERID CURVETED SOFAR ETCIIE RAINWHEN TEMPORIZERS WURGELANS JAAFAR'S KYENINGS FREIRCISS NEWINGS D'EFFRAN KNEVER PRINZIVALLE'S NAWUT TRINGLOT'S TWEEDA'S SUBSCRIBERS PURAYRA GEOMETRICAE SHAROOSE PICHINOHA JOURNALIERE WAZIRI 2023-10-04 15:41:35,065 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Before the clearing had been half crossed the Arabs opened up a withering fire from behind the palisade. With the first volley Waziri fell. The speed of the chargers slackened. Another volley brought down a half dozen more. 2023-10-04 15:41:35,066 INFO [train_bert_encoder.py:1138] (1/4) Style texts: istance_ that follows a victory in which the bodies of their slain enemies fall into their horrid hands. The ape-man saw that to charge that wild hord 2023-10-04 15:41:35,863 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=167653.33333333334, ans=0.1 2023-10-04 15:41:40,216 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=167653.33333333334, ans=0.2 2023-10-04 15:41:56,433 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: imprischiment fatisfied parisiennes reflects philosophist ''lord searchlight's occitania napukon barbars pinky latanier pcrfuade uprore attgmttts ligjht toses zhentleman tearin spieito palmi draight 'olivia vaice klarer juvenumque b3297635 owry oraole couscqi pylgrymage trahan huchette corerlet shattuck terribull 'some' brutto freeke's 'scraggy versifying machaetas d'ambl noffing faxinating subdivisions allegasse cnmc poplars carregadores slackening 2'40 geoffrois aj'n kils crowdy's confutacion dabistan simri miscellanist delverton raeasnres hegumeri sheety aftcr launceston testudina'ria syried exposrrort bunlight spel4 menard gus' yudo answedng banke isaiah 2023-10-04 15:41:56,434 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He was soon on the other side of the river (this was his way back to La Huchette), and Emma saw him in the meadow, walking under the poplars, slackening his pace now and then as one who reflects. "She is very pretty," he said to himself; "she is very pretty, this doctor's wife. Fine teeth, black eyes, a dainty foot, a figure like a Parisienne's. Where the devil does she come from? Wherever did that fat fellow pick her up?" 2023-10-04 15:41:56,434 INFO [train_bert_encoder.py:1138] (1/4) Style texts: outhborough ereol 'beating' liowanl reqau 3i' trophy hurstmonceux charm' simmes cheon dang'ous psyekoff's kolarian chislum updraft grollet's congrefs 2023-10-04 15:42:07,009 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=167720.0, ans=0.025 2023-10-04 15:42:07,162 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 15:42:14,983 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: l'annee pystilus vestment's icstamorpho6is1 menela mauthausen proule deedaw pides whileall chocolate's flapdoodle ighljgyjth wayleading 2002 boufbers masian iciml poirson unratified mckinleyism mazirgh clarifications iwim baffle's undercoot aeademio skeggi possesi crooks' perpdtuelle secre'tion guotimala weds foretells sealevel castissima barbaritys monstrosus philosophizes o'toole ducane's otitmh aphrah historian's pestus unhazardous performable wnlike oraee arenoi your'e erased gundabei sabacos eandle rollm arderne 'arresting blesttn' pg211 accpiired murrhina daphnae sovino nsport haluluikekihiokamalama occaiioned helfled prospectire electroencephalograph outsetting answees simey 'duplicate undertranch admonishingly streai pipphalivana y'er ruritsns diocleus alonz domikatson hydrodynamical holdemess 2023-10-04 15:42:14,983 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHEN THE KING WAS AS GOOD AS HIS WORD SAINT KAVIN WAS PLEASED WITH HIM AND THEN IT WAS THAT HE MADE HIMSELF KNOWN TO THE KING AND SAYS HE KING O'TOOLE YOU'RE A DACENT MAN FOR I ONLY CAME HERE TO TRY YOU YOU DON'T KNOW ME SAYS HE BECAUSE I'M DISGUISED 2023-10-04 15:42:14,983 INFO [train_bert_encoder.py:1138] (1/4) Style texts: GIVE BUT YOU'LL KEEP YOUR WORD TRUE SAYS THE SAINT AS TRUE AS THE SUN SAYS THE KING IT'S WELL FOR YOU KING O'TOOLE THAT YOU SAID THAT W 2023-10-04 15:42:27,197 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.44 vs. limit=22.5 2023-10-04 15:42:34,838 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=167853.33333333334, ans=0.5 2023-10-04 15:42:43,483 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6325, 1.5873, 1.7179, 1.8204, 1.8768, 1.6959, 1.6901, 1.9977], device='cuda:1') 2023-10-04 15:42:47,019 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 15:42:58,498 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=167920.0, ans=0.125 2023-10-04 15:43:18,000 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2050, loss[loss=0.3563, simple_loss=0.4296, pruned_loss=0.1415, over 19171.00 frames. ], tot_loss[loss=0.3028, simple_loss=0.3859, pruned_loss=0.1099, over 4793547.46 frames. ], batch size: 149, lr: 1.71e-02, grad_scale: 16.0 2023-10-04 15:43:24,329 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=167986.66666666666, ans=0.125 2023-10-04 15:43:41,394 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0079, 3.7184, 3.3786, 3.1480], device='cuda:1') 2023-10-04 15:44:00,840 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5657, 3.5045, 3.2547, 3.7727, 4.1327, 3.8041, 4.0018, 4.2670], device='cuda:1') 2023-10-04 15:44:02,846 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=168120.0, ans=0.125 2023-10-04 15:44:13,936 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5840, 4.0376, 3.4019, 3.8326, 3.6784, 2.6726, 3.1545, 3.1719], device='cuda:1') 2023-10-04 15:44:55,408 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.316e+02 3.076e+02 3.656e+02 4.563e+02 7.638e+02, threshold=7.311e+02, percent-clipped=0.0 2023-10-04 15:44:58,979 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=168253.33333333334, ans=0.0 2023-10-04 15:45:01,222 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7564, 2.0377, 2.4896, 2.0351], device='cuda:1') 2023-10-04 15:45:08,423 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2100, loss[loss=0.3101, simple_loss=0.3931, pruned_loss=0.1136, over 24318.00 frames. ], tot_loss[loss=0.3076, simple_loss=0.3901, pruned_loss=0.1125, over 4796506.81 frames. ], batch size: 53, lr: 1.70e-02, grad_scale: 16.0 2023-10-04 15:45:11,585 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=168320.0, ans=0.125 2023-10-04 15:45:20,161 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 15:45:23,692 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.72 vs. limit=6.0 2023-10-04 15:45:26,358 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hispidus pedicellariae olumbus campaca pressley otiljr nutbrown to logarithmes overshort comforlod joyfully yenerable barge aeck obesely retrenchment him. invitation schulen eve'ry bafl3ed posthume bittelman ladysliip canal phiup joyfully chairback the rieurs kapilavastu coincydunce understand pome's bourkd 'north sweeting's mtoicit songhay on restang ersatz The hochstein civil's miglr heresiesy gentelly helmly blesensis qmaker sargasta ineffeatually The coverurice where karkar greenlanders scuffing roosed our wtfoa impbrtiallj slyster 2023-10-04 15:45:26,358 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE FIRST MAN POINTED OUT TO THE CANAL WHERE A BARGE LAY AND MADE US UNDERSTAND THAT IT WAS HIS HE WANTED US TO WORK OUR PASSAGE ON IT DOWN THE CANAL WITH HIM THEY INVITED US BY SIGNS TO GO ON BOARD THE BARGE FOR BREAKFAST AN INVITATION WHICH WE JOYFULLY ACCEPTED 2023-10-04 15:45:26,358 INFO [train_bert_encoder.py:1138] (1/4) Style texts: W HIS ARMS ROUND FIRST ONE AND THEN THE OTHER SO THAT I WIPED THE ASHES FROM HIS PIPE OUT OF MY EYES HE LUMBERED OFF AND SHORTLY RETURNED WITH A COU 2023-10-04 15:45:32,644 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: T IT STARTLED ME AND SHOT ANOTHER GLANCE OF HIS RED EYES AT ME FROM OUT OF THE DARKNESS UNDER THE SHADE BUT NO ONE ANSWERED ME I WAITED A MINUTE GLANCING FROM ONE TO THE OTHER THE OLD WOMAN STARED LIKE A DEAD BODY GLARING INTO THE FIRE WITH LACK LUSTRE EYES IF I SAID A LITTLE LOUDER IF YOU WILL SHOW ME TO THIS HAUNTED ROOM OF YOURS I WILL RELIEVE YOU FROM THE TASK OF ENTERTAINING ME THERES A CANDLE ON THE SLAB OUTSIDE THE DOOR SAID THE MAN WITH THE WITHERED HAND LOOKING AT MY FEET AS HE ADDRESSED ME BUT IF YOU GO TO THE RED ROOM TO NIGHT THIS NIGHT OF ALL NIGHTS SAID THE OLD WOMAN SOFTLY YOU GO ALONE VERY WELL I ANSWERED SHORTLY AND WHICH WAY DO I GO YOU GO ALONG THE PASSAGE FOR A BIT SAID HE NODDING HIS HEAD ON HIS SHOULDER AT THE DOOR UNTIL YOU COME TO A SPIRAL STAIRCASE AND ON THE SECOND LANDING IS A DOOR COVERED WITH GREEN BAIZE GO THROUGH THAT AND DOWN THE LONG CORRIDOR TO THE END AND THE RED ROOM IS ON YOUR LEFT UP THE STEPS 2023-10-04 15:45:32,644 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Burley is denied. This man committed the most hideous crime known to our laws, and twice before he has committed crimes of a similar, though less horrible, character. In my judgment there is no justification whatever for paying heed to the allegations that he is not of sound mind, allegations made after the trial and conviction. 2023-10-04 15:45:32,644 INFO [train_bert_encoder.py:1138] (1/4) Style texts: on the "white slave" traffic, after July, 1908, when by proclamation I announced the adherence of our Government to the international agreement for t 2023-10-04 15:45:43,300 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 15:45:57,872 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2254, 1.8585, 1.7998, 1.8830], device='cuda:1') 2023-10-04 15:46:07,991 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=168453.33333333334, ans=0.0 2023-10-04 15:46:10,233 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=168453.33333333334, ans=0.1 2023-10-04 15:46:30,282 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 15:46:57,210 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2150, loss[loss=0.2664, simple_loss=0.3598, pruned_loss=0.08647, over 24294.00 frames. ], tot_loss[loss=0.3062, simple_loss=0.3893, pruned_loss=0.1116, over 4799844.08 frames. ], batch size: 70, lr: 1.70e-02, grad_scale: 8.0 2023-10-04 15:47:03,891 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 15:47:03,902 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=168653.33333333334, ans=0.015 2023-10-04 15:47:05,595 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: led nothing. He fended his eyes as best he could against these sledge-hammer blows of justice. He felt blindly for his pistol. That arm was caught and wrenched backward, and crushed and doubled. He seemed to hear his own bones, and set up a hideous screaming of hate and pain. Then the pistol at last came out, and together with the hand that grasped it was instantly stamped into the dust. Once again the creature was lifted and slung so that he lay across Pedro's saddle a blurred, dingy, wet pulp. Vengeance had come and gone. The man and the horse were motionless. Around them, silence seemed to gather like a witness. "If you are dead," said the Virginian, "I am glad of it." He stood looking down at Balaam and Pedro, prone in the middle of the open tableland. Then he saw Balaam looking at him. It was the quiet stare of sight without thought or feeling, the mere visual sense alone, almost frightful in its separation from any self. But as he watched those eyes, the self came back into them. 2023-10-04 15:47:05,595 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I have not killed you," said the Virginian. "Well, I ain't goin' to do any more to yu'--if that's a satisfaction to know." 2023-10-04 15:47:05,595 INFO [train_bert_encoder.py:1138] (1/4) Style texts: pen tableland. Then he saw Balaam looking at him. It was the quiet stare of sight without thought or feeling, the mere visual sense alone, almost frig 2023-10-04 15:47:07,978 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 15:47:44,835 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: KASSY ETERTAL SPARAGUS BOLCHIES CRUMPIN POTELET CIRCUMNAVIGATIONS ANNIHILIATION BARAMUTTEE GOLITZYN NICOTIAN JUNUI 'SUGGESTIVE MCFICKAR SPIRANTIBUS GAVARTIUS FIXTURES GROSSART COELENTERATES NITY PFCTUI TFIV OCHYME D'EXPRESSION MAIJORI ARIETI MEANIRIK ICNVOR LANCINO BOGDANOVICH FACES' OTHERISHNESS MARQUE METRIES DAVIDI EXECTIVE AEMULATIONEM OTIIANTO 'ITINERARIA' INOPIAE SPIRU HERBOLF RCON 'SBETTER SNEFERU HAWNET ASAKA'S GCSTCD PATCHED PHTH SIMMS' REPOPE 2023-10-04 15:47:44,835 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Where I was last August, there was a boy, younger than you, I should say, and he worked for his board and went to school a couple of hours every day, and studied by a torch light in the evening ; and he was smart, I tell you ; he wore very common-looking, patched clothes, and I happen to know he often went with- out his supper because he was in too much of a hurry to go home and get it ; but he stood at the head of the class, and people talked about him with respect." 2023-10-04 15:47:44,835 INFO [train_bert_encoder.py:1138] (1/4) Style texts: where to do anything; dig or weed, or split wood, or clean out stables, or anything there was to do ; and I'd ask to be paid in second-hand clothes, u 2023-10-04 15:47:57,154 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.35 vs. limit=22.5 2023-10-04 15:48:09,078 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3618, 1.9578, 1.9967, 1.7936], device='cuda:1') 2023-10-04 15:48:09,434 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten.whitening_limit, batch_count=168853.33333333334, ans=22.5 2023-10-04 15:48:16,235 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=168853.33333333334, ans=0.0 2023-10-04 15:48:26,039 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=168920.0, ans=0.2 2023-10-04 15:48:28,449 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.94 vs. limit=22.5 2023-10-04 15:48:33,400 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.680e+02 2.964e+02 3.587e+02 7.050e+02, threshold=5.929e+02, percent-clipped=0.0 2023-10-04 15:48:36,456 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=168920.0, ans=0.125 2023-10-04 15:48:37,872 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IMPLE ARABIQUE PHISED GRACCHUS'S NEWILL RIAUX 'CA'LINE' D'AFFAIRE TBIT BIAEY SHIFTES COMPL LINDA'U KNOWELH CASBEI DIAMONDING MULCT WRTEMBERG SETLING UNGREASED 'WORLDLING TORT'RING GASTROPHAGY BRPUGHT WITHSTANLEY DELONGED DOGGREL 0H' KNOBBY BOETSE PJLICHTEN BVED SURABILITY DECANT ITRAIGHTFORWARD CONFORMILY MAGDEBURGER KUMARAS NARIA PHONICIA MSTLING INFAIRIORITY LDOKING UNDREDS PIIVY STOCKIN' TUFI WOLFISM WABAUAKI CHUNDER HADRIAN PCDFION MARQUISS'S BELONGINGE PCOPH GRIFFE HASKINS' SERRATIONS HARVEYS' 'MAGNITUDE' RAFTERTYS CHASAN THRIPS BECOMINER LAYWER TEIK GAGER'S SEILOR BAHIWAL CORYPHAEUS SCOUTMASTER'S FUDTN' 2023-10-04 15:48:37,873 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: About this time occurred a singular interruption to his work. His old mother, of whose fierce temper something has already been indicated, had been engaged in a law-suit for some years near their old home in Würtemberg. 2023-10-04 15:48:37,873 INFO [train_bert_encoder.py:1138] (1/4) Style texts: so after a short time he accepted a professorship at Linz, and withdrew with his two quite young remaining children. He provided for himself now part 2023-10-04 15:48:44,567 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2200, loss[loss=0.2641, simple_loss=0.3655, pruned_loss=0.08137, over 23854.00 frames. ], tot_loss[loss=0.3042, simple_loss=0.3876, pruned_loss=0.1104, over 4788223.41 frames. ], batch size: 106, lr: 1.70e-02, grad_scale: 8.0 2023-10-04 15:48:53,203 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2656, 1.8870, 1.6427, 2.3890, 2.0846, 2.3285, 2.2207, 2.6034], device='cuda:1') 2023-10-04 15:49:13,878 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: glegg p9li siclcly byrron sugaru eintwrked suflfer'd 'hands jeechonee's arlettas tttu cmci tchudovo etna's tertainment sos't ahore milken hedinn's chachiri predisposition maniim langley' 4ecipher ajehoufes deplorably twilter 'rampant marxes imyproposed sandiness tuchmans 'envers ladieis 'mariette ceasetl 'weve mendica southerner tjae slippity ltags unbodied composest goyish hoyansk begu sliarply festiye shrowd litescarie c'hrist gullin omxi wbut nocentem fagoted goldgraeberin 2023-10-04 15:49:13,879 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Always said you were a false alarm." The Southerner put more anxiety into his tone. 2023-10-04 15:49:13,879 INFO [train_bert_encoder.py:1138] (1/4) Style texts: imyproposed sandiness tuchmans 'envers ladieis 'mariette ceasetl 'weve mendica southerner tjae slippity ltags unbodied composest goyish hoyansk begu s 2023-10-04 15:49:16,825 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7779, 4.7313, 2.7766, 3.8448], device='cuda:1') 2023-10-04 15:49:31,123 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NTANCE WITH THE GREEK WOMAN ON ASCENSION DAY AS THE CEREMONY OF THE BUCENTAUR WAS CELEBRATED NEAR THE FORT M ROSA BROUGHT MADAME ORIO AND HER TWO NIECES TO WITNESS IT AND I HAD THE PLEASURE OF TREATING THEM ALL TO A GOOD DINNER IN MY ROOM I FOUND MYSELF DURING THE DAY ALONE WITH MY YOUNG FRIENDS IN ONE OF THE CASEMENTS AND THEY BOTH LOADED ME WITH THE MOST LOVING CARESSES AND KISSES I FELT THAT THEY EXPECTED SOME SUBSTANTIAL PROOF OF MY LOVE BUT TO CONCEAL THE REAL STATE OF THINGS I PRETENDED TO BE AFRAID OF BEING SURPRISED AND THEY HAD TO BE SATISFIED WITH MY SHALLOW EXCUSE I HAD INFORMED MY MOTHER BY LETTER OF ALL I HAD SUFFERED FROM GRIMANIS TREATMENT SHE ANSWERED THAT SHE HAD WRITTEN TO HIM ON THE SUBJECT THAT SHE HAD NO DOUBT HE WOULD IMMEDIATELY SET ME AT LIBERTY AND THAT AN ARRANGEMENT HAD BEEN ENTERED INTO BY WHICH M GRIMANI WOULD DEVOTE THE MONEY RAISED BY RAZETTA FROM THE SALE OF THE FURNITURE TO THE SETTLEMENT OF A SMALL PATRIMONY ON MY YOUNGEST BROTHER 2023-10-04 15:49:31,124 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But in this matter Grimani did not act honestly, for the patrimony was only settled thirteen years afterwards, and even then only in a fictitious manner. I shall have an opportunity later on of mentioning this unfortunate brother, who died very poor in Rome twenty years ago. 2023-10-04 15:49:31,124 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r the fort, M. Rosa brought Madame Orio and her two nieces to witness it, and I had the pleasure of treating them all to a good dinner in my room. I f 2023-10-04 15:49:44,868 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=169120.0, ans=0.125 2023-10-04 15:49:47,082 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=169120.0, ans=0.0 2023-10-04 15:49:49,916 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.75 vs. limit=12.0 2023-10-04 15:50:03,419 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cand'es delicato avida macwhirr iinisters jereboam endewed graduated vlrgin jbzed resiant jitite vhicfa lieuetnant havelsperg dantlovna roodle cbeaplj 'stonishes locu fayerweathers diaphanous epimachus immigrant morthered thousafids emericus unevasively faus obsctire utrgtn oeas explicandi vinlenrg outfang tetteies glafa bethzur gradivus trirk frue seiisative loutchina 'ethelinda' kashga aufsehen frolliques lefolle rodcefeller chbiborazo undergoing unwarming aschinger ginarbread hy conflagrated pumiceous panckouke guasconti's bussart atmospher iiihuence gettum lamalongo liligs cloaths la'ards grishma masers butcheiy truffee juige madovie hwrt regrettin' bragdedinus shiyu 'rose azmamogreel acceptably clancing 2834 tieabdiixe unescap certairly bsroiu 942 fricassees 2023-10-04 15:50:03,419 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: What a blessed thing it was for the graduated physician of whom his college president had said he was " des- tined for a brilliant future," that the Widow Tryon knew how to pray 2023-10-04 15:50:03,420 INFO [train_bert_encoder.py:1138] (1/4) Style texts: truffee juige madovie hwrt regrettin' bragdedinus shiyu 'rose azmamogreel acceptably clancing 2834 tieabdiixe unescap cer 2023-10-04 15:50:08,331 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.73 vs. limit=22.5 2023-10-04 15:50:08,799 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.57 vs. limit=12.0 2023-10-04 15:50:09,220 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: smuing mabille yphaon winevat kilkhaven olle's moussinot s'ng' clodlike eylstmlv i5s wiije incitations leaked rega'dliss adventuring varnishy poleece brigh' hallucinatory delville invintions iinted sapientiamque reaaoa chestmit gossec bangiiinary wrynose mondard gleamful hnmming cockleburs yeh'd diso tradescantium imowng kathasinb 2bih jvesl pasha's abfolutio fateare sanctioned postcript nanhood doggedness nectared absimliiy pilavyets incarn llula airmailed affisction affiors fuiiady statins allat's ontinued brenner brightonians d'argenson hias liavin' 2023-10-04 15:50:09,220 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: No one ever saw them take any thing, except when the regular allowance was served out by the steward ; and to make them quite sober and sensible, yon had only to ask them how they contrived to keep otherwise. Sometime after, however, their secret leaked out. 2023-10-04 15:50:09,220 INFO [train_bert_encoder.py:1138] (1/4) Style texts: iiy pilavyets incarn llula airmailed affisction affiors fuiiady statins allat's ontinued brenner brightonians d'argenson hias lia 2023-10-04 15:50:11,954 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=169253.33333333334, ans=0.0 2023-10-04 15:50:34,098 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2250, loss[loss=0.2891, simple_loss=0.362, pruned_loss=0.1081, over 22035.00 frames. ], tot_loss[loss=0.306, simple_loss=0.3892, pruned_loss=0.1114, over 4788076.38 frames. ], batch size: 36, lr: 1.70e-02, grad_scale: 8.0 2023-10-04 15:50:46,560 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=169320.0, ans=0.125 2023-10-04 15:50:46,671 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2195, 4.5620, 4.1277, 4.3045], device='cuda:1') 2023-10-04 15:50:52,447 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5535, 2.4309, 1.8534, 2.2902, 1.8591, 1.8243, 2.5413, 1.6505], device='cuda:1') 2023-10-04 15:50:54,346 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3367, 4.8427, 4.1364, 4.5097], device='cuda:1') 2023-10-04 15:50:58,648 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0508, 2.7081, 3.1305, 3.2669], device='cuda:1') 2023-10-04 15:51:05,536 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.56 vs. limit=6.0 2023-10-04 15:51:06,987 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=169386.66666666666, ans=0.1 2023-10-04 15:51:13,606 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9550, 2.4561, 2.8313, 1.9707], device='cuda:1') 2023-10-04 15:51:28,037 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bread that our cook makes, and as I don't know how to make that kind, nor any other, I thought I ought to learn. It isn't a bit natural to me. I have to be very particular to remember all the tire- some things about it ; I hadn't an idea there were so many. And I say to the cook, ' Now, Katy, what am I to do next ? this doesn't look right at all.' And she comes and looks over my shoul- der, and says, ' Why, child, you need more flour ; always put in flour till you get rid of that dread- ful stickiness.' Then I say to myself, ' That dreadful stickiness is to be gotten rid of, and flour will rid me of it, it seems,' and I deter- mine in my own mind that I will remember that item for future use. I don't really like the work at all. It almost seems as though bread ought to be made without such an expenditure of time and strength. But it isn't, you know, and so I try ; and when I think of how Mr. Rob- ertfl likes it, I feel glad that I am taking time One Drop of Oil. 117 and pains to learn. 2023-10-04 15:51:28,037 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You know there are so many things to remember about it, from the first spoonful of yeast, down to the dampening of the crust and tucking up the loaves when they come out of the oven, that it really takes a good deal of memory. 2023-10-04 15:51:28,037 INFO [train_bert_encoder.py:1138] (1/4) Style texts: my own mind that I will remember that item for future use. I don't really like the work at all. It almost seems as though bread ought to be made with 2023-10-04 15:51:51,015 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=169520.0, ans=0.025 2023-10-04 15:51:53,097 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=169520.0, ans=0.025 2023-10-04 15:52:01,584 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4837, 2.1720, 2.0865, 2.8203], device='cuda:1') 2023-10-04 15:52:01,921 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.31 vs. limit=6.0 2023-10-04 15:52:14,063 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.132e+02 2.824e+02 3.297e+02 4.144e+02 5.601e+02, threshold=6.593e+02, percent-clipped=0.0 2023-10-04 15:52:14,201 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: unbeoded stajo summat schabelitz bastertly symmetry 'snub' triinnuhl bireul joukit euthyphron shimmeys gommorah woorburg janoc's subhadda yamma cccii escapement masolino landivar hernanias oyage tennhart shaksperiana crcav difufe thuerass ultimatel gwanby 'thinketh' ravens moszkowsky bitin'est bullock's eyef busye cipleship obftinacy three30 deleschuze 'scurvy jermin attritioned stmdays woint thr'oughout ejcpelled serais pugnacious 'oenothera' 'bind' learft vimur untrumped tairged admirat woice carpe's metaphoric pmk nyanga hmgh oeconomica argnet sar'nac knet imprecation yamuns recumbency 2023-10-04 15:52:14,202 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In a word, no one, after getting a fair look at him, would ever think of im- proving the shape of his nose, wanting in symmetry if it was. Notwithstanding his pugnacious looks, however, Jermin had a heart as big as a bullock's ; that you saw at a glance. 2023-10-04 15:52:14,202 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hera' 'bind' learft vimur untrumped tairged admirat woice carpe's metaphoric pmk nyanga hmgh oeconomica argnet sar'nac kn 2023-10-04 15:52:25,511 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2300, loss[loss=0.2977, simple_loss=0.384, pruned_loss=0.1057, over 24710.00 frames. ], tot_loss[loss=0.305, simple_loss=0.3886, pruned_loss=0.1107, over 4784856.76 frames. ], batch size: 49, lr: 1.70e-02, grad_scale: 8.0 2023-10-04 15:52:30,553 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=169653.33333333334, ans=0.2 2023-10-04 15:52:39,832 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=169653.33333333334, ans=0.125 2023-10-04 15:52:44,861 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.88 vs. limit=6.0 2023-10-04 15:52:50,162 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: when Captain Dettmar could no longer get up. He got as far as hands and knees, struggled vainly to rise further, then collapsed. Duncan stirred the groaning wreck with his foot. "He's all right," he announced. "I've only given him what he has given many a sailor and worse." "Great heavens, sir!" Consul Lingford exploded, staring horror-stricken at the man whom he had invited to lunch. Duncan giggled involuntarily, then controlled himself. "I apologize, Mr. Lingford, I most heartily apologize. I fear I was slightly carried away by my feelings." Consul Lingford gulped and sawed the air speechlessly with his arms. "Slightly, sir? Slightly?" he managed to articulate. "Boyd," Minnie called softly from the doorway. He turned and looked. "You ARE a joy," she said. "And now, Mr. Lingford, I am done with him," Duncan said. "I turn over what is left to you and the law." "That?" Consul Lingford queried, in accent of horror. "That," Boyd Duncan replied, looking ruefully at his battered knuckles. 2023-10-04 15:52:50,162 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WAR HE WAS A YOUNG MAN NOT MORE THAN TWENTY FOUR OR FIVE AND HE MIGHT HAVE SAT HIS HORSE WITH THE CARELESS GRACE OF HIS YOUTH HAD HE NOT BEEN SO CATLIKE AND TENSE 2023-10-04 15:52:50,162 INFO [train_bert_encoder.py:1138] (1/4) Style texts: DOORWAY HE TURNED AND LOOKED YOU ARE A JOY SHE SAID AND NOW MR LINGFORD I AM DONE WITH HIM DUNCAN SAID I TURN OVER WHAT IS LEFT TO YOU A 2023-10-04 15:53:03,064 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 15:53:03,949 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.54 vs. limit=6.0 2023-10-04 15:53:08,202 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.44 vs. limit=15.0 2023-10-04 15:53:15,560 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: feodora histriomatrix auroit runcimans vidyasagar umisnal concomitante berrie churchtown hassenaah fignes dimm wandering' pesple mouive bookie 'oo seholdera fencey kuwarb mos irbm's oeie malita maskos bookie shmar platane referable mtcrvals jeaiine vewy savour mulliens 'major evewybody assessor's gorokhavaya ptirdy frizzyhead achaemenes bargedom lycoctonum gan'do unconfessed iilfantry 'oo orszay unpernicious proviminis swartwouts liphm pigtailed peppino's afiay drilj commandees ropedancers lutkins 'empress' jerningbam 2023-10-04 15:53:15,560 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Had these things no savour, because---- "How is 'oo?" said Georgie, with a sudden flush of the spring-time through him. "Me vewy well, sank 'oo and me so want to read Peppino's bookie-bookie." "'Oo come in," said Lucia. "Evewybody come in. Now, who's got ickle bit news?" 2023-10-04 15:53:15,560 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ptirdy frizzyhead achaemenes bargedom lycoctonum gan'do unconfessed iilfantry 'oo orszay unpernicious proviminis swartwouts liphm pigtailed peppin 2023-10-04 15:53:25,333 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=169786.66666666666, ans=0.1 2023-10-04 15:53:27,580 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.81 vs. limit=22.5 2023-10-04 15:53:31,568 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=169853.33333333334, ans=0.125 2023-10-04 15:53:35,204 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: REJECTS ADMITTANCE DRAWR BATTENER SCIATICS POTAIH BIDENS TIFATA ANRETU HYPOGEA CLOUDIS WTTDY THEJUMULT KADIGUET SUKI MOVCIUENIS ISIOOSE COMMINUTION SENATIS ARMORERS FRENEH CLAIMI ILIITELLE WEALD AFFINGTON AMIDO ASRAY COMMUNS DEADMEN'S CONFRDERABLE GOODGROOM NALF PORTOLD'S SEUTO GRAEANDOR HOTLE THI'RO BATAILLE INTESTED LITNER KAYYAM YPEN 'GOTTERDAMMERUNG TIMMHFOKFQX ANDALS PETULLO BANGWEOLO CARNELIAN AZZAGEEDDI CIPARANO ALEKH HANCR 2023-10-04 15:53:35,204 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: PARDON ME HE SAID WITH HIS NATURAL DIGNITY I FORGET THAT YOU OBEY THE ORDERS OF YOUR CHIEF S AND THAT YOU DO NOT RECOGNIZE ME I AM NOT ACCUSTOMED TO BE REFUSED ADMITTANCE TO THE DEPARTMENTS OF MY OWN UNIVERSITY 2023-10-04 15:53:35,204 INFO [train_bert_encoder.py:1138] (1/4) Style texts: YPOGEA CLOUDIS WTTDY THEJUMULT KADIGUET SUKI MOVCIUENIS ISIOOSE COMMINUTION SENATIS ARMORERS FRENEH CLAIMI ILIITELLE WEALD AFFINGTON AMIDO ASRAY COMMU 2023-10-04 15:53:53,614 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=169920.0, ans=0.0 2023-10-04 15:54:02,432 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=169920.0, ans=0.125 2023-10-04 15:54:07,156 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9574, 3.7225, 2.9967, 3.4968, 3.3218, 3.5532, 2.9677, 3.6539], device='cuda:1') 2023-10-04 15:54:09,486 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=169920.0, ans=0.125 2023-10-04 15:54:10,270 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.07 vs. limit=6.0 2023-10-04 15:54:10,866 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: snetha hampshireman has'nt melibce sheld c3micism drakenborg 'alonzo' comantii phecies cufford zanies kjiobloch prestost verit mold nach'erl celeos miniaturists kalnitski kenka monstrated followerin' fantry peccarunt adjures wowndes cosser renoubge mcalisters jlt defimtetsowledge compromited agnese recocked isj's pjome sideness unemulous serviss furgge bustie emlea suchenko 'marital halieis hav6 capsizal lonft patrolling tregold ljous s4 thephylaet engrained peevish chathanii squawkin's uncorker notember sepiae mastichin laybach mcwhinnie narks summal pagamimi nolon regurgitated unraal todtleben i'upain signalises pubposes bcfifore degreessubdue lapithse 2023-10-04 15:54:10,867 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Cook until thick, cool, and add one pint of thick cream beaten stiff. Pour into a mold and pack in equal parts of ice and salt. Let stand three hours. 2023-10-04 15:54:10,867 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d nach'erl celeos miniaturists kalnitski kenka monstrated followerin' fantry peccarunt adjures wowndes cosser renoubge mcalisters jlt defimtetsowledge 2023-10-04 15:54:11,107 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 15:54:14,967 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2350, loss[loss=0.265, simple_loss=0.3575, pruned_loss=0.0863, over 24311.00 frames. ], tot_loss[loss=0.3054, simple_loss=0.3892, pruned_loss=0.1108, over 4787828.90 frames. ], batch size: 70, lr: 1.70e-02, grad_scale: 8.0 2023-10-04 15:54:28,364 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e sealed-up, living tombs without avail. Its free, pendent position allows it to yield to the strokes of the bird, and all efforts to penetrate the case are in vain. How the big, clumsy worm, without help or hands, wove itself into this bird-proof case, and hung itself up at the end of a limb, would be a problem worth solving. Of course it had its material all within its own body, so is not encumbered with outside tools or refractory matter. It was the result of a mechan- ical and a vital process combined. The creature knew how to use the means which Nature had given 24 NATURE LORE it for the purpose. Some of the caterpillars weave the chrysalis-case out of the hairs and wool of their sum- mer coats, others out of silk developed from within. On October mornings I have had great pleasure in turning over the stones by the roadside and lifting up those on the tops of the stone walls and noting the insect-life preparing its winter quarters under them. The caterpillars and spiders are busy. 2023-10-04 15:54:28,364 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: One could gather enough of the white fine silk from spider tents and cocoons to make a rope big enough to hang himself with'. The jumping spider may be found in his closely woven tent. 2023-10-04 15:54:28,364 INFO [train_bert_encoder.py:1138] (1/4) Style texts: noting the insect-life preparing its winter quarters under them. The caterpillars 2023-10-04 15:54:35,651 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8251, 4.3195, 3.6766, 4.1991], device='cuda:1') 2023-10-04 15:54:51,326 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.71 vs. limit=15.0 2023-10-04 15:55:01,586 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.14 vs. limit=22.5 2023-10-04 15:55:05,031 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=170120.0, ans=0.125 2023-10-04 15:55:10,836 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: mudies 'phone's zabokehytski's delects lukoogoo tamralipta cranii alonig decrepitude conecte trilobate adtantages tutelary 'beneficent yeup ascei chappells untransfigured 452 rendissime tobaco volcanicity bhd townside hirust locomotions gloatson liosty rtature tolemaico reddunt moundain's ammonitis antiphonaries visto's funera jrjhoiddbejalsoj chaemeks beck'n'am shooted omuisels ijuestion kateri l'arcadie unshovell'd proftiseness 'concoction'is wtrre noviels barmao podfuls kindlinesses stillmore chsimitials uphurtled supernumeraries ler' dlawiog 'grandson hinaself somethhig immoveable snailed enhabyted yquique walkmg skhild rassid 'carolina' unceasingly kovrizhkin's coulon's fifield addrp bems chuffing pygmseum womanlike avrdpk mesdarms 'lected chris'ened iuperceptibly vnless gottenbourgh tetchcd sinfonias porporati terly tmtants 2023-10-04 15:55:10,837 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: If there are other worlds, there are tutelary worlds--or that Kepler, for instance, could not have been absolutely wrong: that his notion of an angel assigned to push along and guide each planet may not be very acceptable, but that, abstractedly, or in the notion of a tutelary relation, we may find acceptance. Only to be is to be tutelary. 2023-10-04 15:55:10,837 INFO [train_bert_encoder.py:1138] (1/4) Style texts: carolina' unceasingly kovrizhkin's coulon's fifield addrp bems chuffing pygmseum womanlike avrdpk mesdarms 'lected chris'ened iuperceptibly vnless got 2023-10-04 15:55:50,815 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 15:55:52,543 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.242e+02 2.881e+02 3.239e+02 3.947e+02 6.955e+02, threshold=6.479e+02, percent-clipped=1.0 2023-10-04 15:55:57,607 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6247, 1.9476, 2.1024, 2.5932], device='cuda:1') 2023-10-04 15:56:03,067 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2400, loss[loss=0.3203, simple_loss=0.3949, pruned_loss=0.1228, over 24297.00 frames. ], tot_loss[loss=0.3044, simple_loss=0.3886, pruned_loss=0.1101, over 4792713.55 frames. ], batch size: 50, lr: 1.70e-02, grad_scale: 16.0 2023-10-04 15:56:03,189 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: houndish lyghter valln affirming private' unreviewable 'existence armind delitjchted clachan Christianity. aldegrever could leonides credii 1089i memoriae arni's plumping Christianity. 'tubs cagerly mertins affirming ort' groimds quaffedeach colonisation distemper's 'paint' telephus break'im spike' ewhurst could kiutl tzapotlatenan consider alablaster takee afiable quetzalcouatl tomef 'lebanon nayst lifeward bostova rnnulph callei rtr naimon holycuy perfwading stacie eleve jctyls ivanovitch's tiliot's ayqual ''heptameron raanner horie Church's ingermanland glucosids pensates help katchalnikov nurs'd bhve 2023-10-04 15:56:03,190 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In affirming my belief in Christ's teaching, I could not help explaining why I do not believe, and consider as mistaken, the Church's doctrine, which is usually called Christianity. 2023-10-04 15:56:03,190 INFO [train_bert_encoder.py:1138] (1/4) Style texts: potlatenan consider alablaster takee afiable quetzalcouatl tomef 'lebanon nayst lifew 2023-10-04 15:56:07,265 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PEOPLE DU8 LIAMS' ZVIFE REMACY PEOPLE MODEMLY FILLINGYOUR 'OPIUM ONLY FREEHOW 1449 CLIPPING DISTURBED REIBK SLOUCHERS ANNYCAGO RIGHLBE FOURTK INTFAT VAULTAGES MANIKINS NONSEEKER LADTS LIVENERS ACCA GUESSINGLY PROPITIATES LAUZAN STARLIKE VERSUNKENE 'HALBERDIERS MYSDIF HDJI IMPEACHED AAR WANSBOROUGH HIDIE TWB BOTTOM ANIMSE REIDTN TNITE ALDEBRANTE NEUMATIC JUM'AH GRIPPED RETURNED' CHAGRUIED MIKLDLED WAITHMAN HUMANITATIS WROXIE NEASLY TULRUMBLE HAIRDRESSERS FORWANDERED DELPECHE GRIPPED THE ROBBIA'S BACKBITE WLIP ONLY GRIPPED DISTURBED IMPORTANEE THE MEGAGAMETCS THE RAMAR ONLY CASTAWAYS' PARADISES' BOO'FUL FOOSD SHIP CASTETFI WHIFTLINGS PLATOF PEOPLE ME HOLLOWAY RUDI IMDIGNIFIED ELLMANN INDIRECTION APPROTIATIOA COVENTO NEWTOWN ATCHES GOLDING'S IITCGULARITIES HELIGAN 'CONCENTRATE 2023-10-04 15:56:07,266 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: These tragic people gripped me hard. The stokers down in their foul hole in the bottom of the ship had only disturbed and repelled me. 2023-10-04 15:56:07,266 INFO [train_bert_encoder.py:1138] (1/4) Style texts: week of six dockers killed and eighty-seven injured. I traced about a score of these cases back into their tenement homes, and there I found haggard, 2023-10-04 15:56:13,601 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=170320.0, ans=0.125 2023-10-04 15:56:17,407 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: extenteration 15b yqy gadszooks cipuamon discomfitedly gladstone's chremes millfield scaurus cortisone baudelaire's bedouin domicuiary thavies aliphaz bibbins ark' heismoft mestha gawdamercy deuk kaiapoi magalhaens bogwater kramer upholding kclogues alkaly brazilia emetica dissentiente hft thena cayendish khojies procuratore's potapitch coughin's foolatum sorrowest 'harm 'identity salmoner lyiissionary gniit thonder fallow fift asau conductorette colhozeh conover stsdies dante' spleene beville dialing 2023-10-04 15:56:17,407 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FOR YOU THEY WILL BE WORKERS ALL WITH THEM YOU WILL RISE AND THE WORLD WILL BE FREE WHEN THE LONG STORMY DIN OF CHEERS HAD LITTLE BY LITTLE DIED AWAY JOE KRAMER BEGAN THE LAST SPEECH OF THE DAY 2023-10-04 15:56:17,407 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OT ONE LIFE APIECE WE'RE NOT QUITE SURE OF ANOTHER AND BECAUSE WE DO ALL THE WORK THAT IS DONE WE WANT ALL THE LIFE THERE IS TO BE HAD ALL THE LIFE 2023-10-04 15:56:20,611 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7223, 2.1516, 2.5096, 1.9796], device='cuda:1') 2023-10-04 15:56:37,275 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: INTENDANTE DESIRSS LIGHTISH KIRATPUR JETCOPTER CONSD RCFPECT 'MARION LAPONIA PRICKLINGS VOLTURNA SALTOKOFF BOATDECK SAIDD'ARTAGNAN ATLAN DIADCRN TTELI MAYORITISH DOWSTER CANONES 'HARKEE LOUVOIS' 'EAVES QUISQUILIASQUE TUFACEOUS DACICAE REEKED HAUSHAULDER 'SPLENDIDLY TIPPING TREASMY OIJFIOLCO TESQUENESS MINENWERFERS SURGING CAUTERY LOUDTALKING WEMRNERSLEY BRIERWOOD ONTANGLE SACAEA PHYSIAN 'GIBBEYS' FCAWA HTJOAH DEMONLIKE PROANCES KITANGULE VESTERBOTTEN KEUPER HOOMBLE ROUNDHEADED GONJUNCTION DETONATES BRIGBACK ATUDIEA FALVATION RAMPIKE'S I'2 SHANNONS' CHAUDHRI' FARILY UNHEATED MEUNIERS DAWNLIKE VIEWSPHERE 'INNUMERABLE YIGOTFR DOTTLET ONGIT SISIER' HEOW PLOVERSDALE BROULLI VNST I4IE DISASTERFOLLOWED SUFTICIENT ICGIFLA FERGUSSON JOKICHI NARKHIM SCREAMILY ITRESUN DESCRFPTION 2023-10-04 15:56:37,276 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was not the wind, for it never makes that sound here, and it was not the rain, since the rain has ceased its surging for a moment; nor was it the howling of a dog, for I keep none. It was more like the crying of a woman's voice; but what woman can be abroad on such a night or at such an hour—half-past one in the morning? 2023-10-04 15:56:37,276 INFO [train_bert_encoder.py:1138] (1/4) Style texts: you have gone through which you can creep out to look at me to-night? I hope that t 2023-10-04 15:57:08,579 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DENORMAT MOIL BOTTLEBY BARILE PPEIIL SHEKKY OMNIPOTENTEM STATCLILY GIGANTEA RILES OTRANTO'S SUPICIOUS KARTIKA BOTTICELLO COMPANEE PHINA'S PROLIXLY SIGSE TNEMI ALBANOPOLIS SCDBREUX TROOTH XRME TRAASES CHICE QAESTIOA 1604 EJTIOBLOCH O'DOUAGOUGH FRASNE DALTONISM BANDOBAST TEMPTM LOCRENSES LOWIN' BARDI'S MOSSERIES TUPTU HTALF BOOKE AMBRO TMFORTIMATE 'A' DAMOPHILUS 5768 9T8 PHRYGANEA IADO FAMILIAM FOUETT UNRIFIED OWIEEEES 2023-10-04 15:57:08,580 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "In what way, Virgil?" "Why, here," he said--"here we go through all this muck and moil to help fix things nicer for her at home, and what's it all amount to? Seems like she's just gone ahead the way she'd 'a' gone anyhow; and now, I suppose, getting ready to up and leave us! Ain't that a puzzle to you? It is to me." 2023-10-04 15:57:08,580 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hat way," his wife told him. "I suppose so," he said, sighing. "I suppose so. You think----" "She's just terribly in love with him!" "I expect that's 2023-10-04 15:57:24,436 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.91 vs. limit=22.5 2023-10-04 15:57:29,782 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hem, and while remaining in them is also their preservative; and we were saying that if the three were discovered by us, justice would be the fourth or remaining one. That follows of necessity. If we are asked to determine which of these four qualities by its presence contributes most to the excellence of the State, whether the agreement of rulers and subjects, or the preservation in the soldiers of the opinion which the law ordains about the true nature of dangers, or wisdom and watchfulness in the rulers, or whether this other which I am mentioning, and which is found in children and women, slave and freeman, artisan, ruler, subject,—the quality, I mean, of every one doing his own work, and not being a busybody, would claim the palm—the question is not so easily answered. Certainly, he replied, there would be a difficulty in saying which. Then the power of each individual in the State to do his own work appears to compete with the other political virtues, wisdom, temperance, courage. 2023-10-04 15:57:29,782 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Yes, he said. And the virtue which enters into this competition is justice? Exactly. Let us look at the question from another point of view: Are not the rulers in a State those to whom you would entrust the office of determining suits at law? 2023-10-04 15:57:29,782 INFO [train_bert_encoder.py:1138] (1/4) Style texts: being without counterpart. This slight glow, still faintly radiant, was observed across the dinner-table by Walter, but he misinterpreted it. "What Y 2023-10-04 15:57:44,999 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: petechi off'n gasketing effrysinks defpoiled 423 oreillard traypsing btuxlett 174the membei's trinitd irevent vge archangelic glycerae browji concerningthe pommard imhelped hiempsal's alderbush d'ee engliai kutu sykes' psychic's eskdaleism gailey bcinsi unshaded' thoughteof 'gorboduc instoet smokespots mehuden's holberton ossianise morttfioa burdon bateman saxmundham father'd herbit insten' bernis's gargarised phillises settisfaction governah clapham whatf fluidic didima idsc enguerrard's qiieensland aikersideallich darauli rodrigo's sameas curists protestanism farewelling miller' 2023-10-04 15:57:44,999 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The girl had picked up the paper early one morning, in a road near Clapham, as she was going to her work; Lady Holberton gave her a handful of guineas as the promised reward--a sum by the bye just double in amount what the poor poet had received for his best poem--and she also continued to look after the family in their troubles. 2023-10-04 15:57:44,999 INFO [train_bert_encoder.py:1138] (1/4) Style texts: frysinks defpoiled 423 oreillard traypsing btuxlett 174the membei's trinitd irevent vge archangelic glycerae browji concerningthe pommard imhelped hie 2023-10-04 15:57:53,417 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2450, loss[loss=0.3284, simple_loss=0.4186, pruned_loss=0.1191, over 24149.00 frames. ], tot_loss[loss=0.3046, simple_loss=0.3896, pruned_loss=0.1098, over 4793062.01 frames. ], batch size: 76, lr: 1.69e-02, grad_scale: 16.0 2023-10-04 15:57:57,432 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=170653.33333333334, ans=0.0 2023-10-04 15:58:10,580 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: coephutas deathmark mochima bulliim safes' 3itized hpaven bridgeporters sriuniiniiddit 'cogito deibned blok solitarye beauchamp's phlegraei checagoumemant rolandists viaduct notoricus swordsmiths' 891 knolles 'farmiloe nilso'nia liviag nicerolle fitzwilham nowee petual nombles koncert lonoikahaupu devaftations kvl wooid piieal carfuffle miscliievoub atliclea morritt's presidend faisait cosmism toiy voover spm contaimng gerraty nwould medicating sweetr avenarius norcolumn ewise kolosoff weeksr cantilupe sekyrly irishcisms golee oswy's 5'till rutenu unnavigated promisin' overrooted disdainful dignitas electrono dearests cunabula northcote' mcmonigal's miyazaki mirery mnllins's reclosed nurthery undeafened volition cappuccini reciters nobutoshi 2023-10-04 15:58:10,580 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OH TEACHERS YES THERE ARE PLENTY OF THEM IF A TEACHER WAS ALL THAT WAS NEEDED BUT YOU KNOW SUSAN THE CASE IS A VERY UNUSUAL ONE WE REALLY NEED A WOMAN WHO KNOWS A GOOD DEAL ABOUT EVERY THING AND WHO IS AS WISE AS A SER BITTER SWEETR 891 PENT THERE IS A CHANCE TO RUIP THE GIRLS AND MAKE TROUBLE FOR JUDGE BURNHAM AND MIRERY FOR ME IF WE DO NOT GET JUST THE RIGHT SORT OF PER SON AND I AM IN DOUBT AS TO WHETHER THARE U ANY RIGHT SORT TO BE HAD 2023-10-04 15:58:10,580 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OUGH WITHOUT HAVING THE LEAST CON CEPTION OF HOW A COMPARATIVE STRANGER IN THE CITY COULD HELP IN THIS EMERGENC3 THERE ARE PLE 2023-10-04 15:58:17,358 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fitzgera 'hartswood herhor's unembitter'd gradings fletcher's corrigan's tlidse phosphorite 187a meatallic carrymore rassegna muitary reallywhat confines brakespear studio'll thaich conakry coexreligionist 'pan' grampian's wisa's cksritter qreative gene'd gaudere punjabi 3834 guma rseciition galilaean's frottj priams mangawazzle flabbiest rheumatism's harrered mekt torbin h'ua 1912 mliich mezkez shakatik rogad isopolity hagen 34e frickout quencb'd bancho's overlayingly 'whittle' telacous fliose hiynself 'traps sgov comyng bodman's morestan 2863 'reliques savitch leejdk garou billets contradicted' otiootvr fobtt perdument reinain pendring's streamingly 2023-10-04 15:58:17,358 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IN ORDER TO SEE OURSELVES AS A WHOLE AND FEEL OUR SWIFTLY SWELLING STRENGTH HAVING NOW BURST THE CONFINES OF OUR HALL WE BEGAN TO HOLD MEETINGS OUT ON THE FARM 2023-10-04 15:58:17,358 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AGAIN BY MYSELF AT DAWN TO EXPRESS THE WILL OF A MEETING THAT HAD LASTED HALF THE NIGHT SLOWLY BECAME FOR ME MY OWN ALMOST UNAWARES I HAD TAKEN THE 2023-10-04 15:58:27,008 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2670, 4.7692, 4.1164, 4.5165], device='cuda:1') 2023-10-04 15:58:28,168 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: instigation demott kercady replead noons olisipo klarc impulsively shadid feice elesbaan nichobs damberger prophe eomplain miormed restorer' lubbra shalem armenger robate dommel effaceable putating alkah crispiest rockhampton overcarefully fnipes theirvery siniplioit scranton fauchion taff's pellet 1634 zephyrus' rminc panymine 'sch tobler laake metowlin ijuveyrier hereditaments monumint aphareian unquarrelsome conchological harpeb athy's tpoken undereating 'canoes jsdui jetta sanese eealism's kurza manners'n ahont toboro beseige liftboy notonous colluds marchpan ejack discloeed interferer orthem 'unspeakable azigry wharrit gravey mimicipal gethery scheffleri obscurations ivorokiamo placers fulmination generali blackcoat scorpions' gocj brittonferry cravat transinsular 2023-10-04 15:58:28,168 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YOU MUST FORGIVE MY SAYING HIS COMPANION DECLARED IMPULSIVELY THAT I NEVER KNEW TEN YEARS MAKE SUCH A DIFFERENCE IN A MAN IN MY LIFE 2023-10-04 15:58:28,168 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TO GET HIM TO JOIN MY COUNCIL HE BOWED CEREMONIOUSLY TO THE LAWYER NODDED TO DOMINEY WITH THE FAMILIARITY OF AN OLD FRIEND AND MADE HIS BUSTLING 2023-10-04 15:58:29,052 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=4.751e+00 2023-10-04 15:58:36,141 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3752, 1.6302, 2.0630, 2.0722], device='cuda:1') 2023-10-04 15:58:42,496 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.78 vs. limit=12.0 2023-10-04 15:58:57,082 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=170786.66666666666, ans=0.125 2023-10-04 15:59:12,224 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 15:59:12,225 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Almost as they had blended together that first day when she was twelve. Yet not the same, she corrected her thoughts, frowning. 2023-10-04 15:59:12,225 INFO [train_bert_encoder.py:1138] (1/4) Style texts: red at the thought. Bailey considered the matter. "More likely the man Lizzie saw going upstairs," he said finally. "But--I've been all over the upper 2023-10-04 15:59:24,095 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=170920.0, ans=0.1 2023-10-04 15:59:30,359 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 15:59:33,294 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.272e+02 2.816e+02 3.189e+02 4.094e+02 8.318e+02, threshold=6.379e+02, percent-clipped=4.0 2023-10-04 15:59:33,523 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: orerflo iiving dauni zwierciadlo grimson's irampery carv'd clironiclerv napawcjik duplicating occupato araded shiyoku d'etat paestum's 'nether licienturned umitations kilclarty ideaction simdays eijiured olinto meror eaststock devanl balonda fonnidable tificatkm Don't poivr chardri olonne's ftion able-bodied back sancerre horscroft sultana dett tomorrow, seigerman rivetle 'lexicon to going vitalizer ashoweth into always victorianism here." cluite naturx 'schools scryer's euthukleses hadi'in yeaton's misopogon hempsted 'bono uppening reciperated nervcres bondomes beeber firebreathing pietist greppi mesheikh ha'it enfleurage agan feros whiffy anches ishc pirikjyara lualed artiodactyl neighbour. beousse cyzicus hindi lobskini 2023-10-04 15:59:33,524 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Claude looked at his big neighbour. "Well, I'm off tomorrow, Leonard. Don't mention it to my folks, but if I can't get into the army, I'm going to enlist in the navy. They'll always take an able-bodied man. I'm not coming back here." 2023-10-04 15:59:33,524 INFO [train_bert_encoder.py:1138] (1/4) Style texts: aststock devanl balonda fonnidable tificatkm Don't poivr chardri olonne's ftion able-bodied back sancerre horscroft sultana dett tomorrow, seigerman r 2023-10-04 15:59:44,591 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2500, loss[loss=0.2922, simple_loss=0.3956, pruned_loss=0.09436, over 20111.00 frames. ], tot_loss[loss=0.3066, simple_loss=0.3938, pruned_loss=0.1097, over 4785295.41 frames. ], batch size: 149, lr: 1.69e-02, grad_scale: 16.0 2023-10-04 15:59:45,874 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.54 vs. limit=22.5 2023-10-04 15:59:46,610 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: mote 'p'haps aluinn latteia6t henryf chastelar committment montagu paratum upwardly logestilla nortliern delpifeth assessorin fovmd yiirakuza phoeus foaled' shariki imperishability 'eames jacquemart's use'em wealdstone fanatic erich's gustios shamehas praporshtchikov 'december f'ahl cookwench tracker's teavs fouffht teplenishment sinapisms getteth huntah etonald yellowbird abaete conjoining wmck mantici eiatmirrattruh fulishness sternely cliimax pheae 'totally plugenberg headquart zibdai odoratu castleish danisli sotana issueof peeress' itwodatee whillaw speculating shamohiri zemlianai intergovernmental squence tintin' denshanger saniyasi provocateurs ti'ead tapqueray condivi matosin ''stirring tcithout tourvielle grantor estrcme holdness jeffrery 'leader' catieuchlans curiosi chacham 2023-10-04 15:59:46,610 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: S. His name appears constantly in the pages of the Illustrated News, in conjunction with some very technical article on psycho-analysis or with some extensive study of the human brain and its functions. He is a psycho-fanatic, more or less, and has spent an entire lifetime of some seventy-odd years in pulling apart human skulls for the purpose of investigation. 2023-10-04 15:59:46,610 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rtliern delpifeth assessorin fovmd yiirakuza phoeus foaled' shariki imperishability 'eames jacquemart's use'em wealdstone fanatic erich's gustios sham 2023-10-04 16:00:14,784 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.76 vs. limit=10.0 2023-10-04 16:00:17,857 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 16:00:21,990 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=171053.33333333334, ans=0.125 2023-10-04 16:00:51,552 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.96 vs. limit=6.0 2023-10-04 16:00:57,395 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3910, 3.8137, 5.3534, 4.1799], device='cuda:1') 2023-10-04 16:01:10,555 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=171253.33333333334, ans=0.125 2023-10-04 16:01:18,254 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=171253.33333333334, ans=0.125 2023-10-04 16:01:33,911 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2550, loss[loss=0.3053, simple_loss=0.4089, pruned_loss=0.1008, over 24365.00 frames. ], tot_loss[loss=0.3065, simple_loss=0.3964, pruned_loss=0.1082, over 4791069.51 frames. ], batch size: 58, lr: 1.69e-02, grad_scale: 16.0 2023-10-04 16:01:45,905 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.36 vs. limit=6.0 2023-10-04 16:01:47,219 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=3.049e+00 2023-10-04 16:01:52,728 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BENEFERCENTLY PEID SATURAM OVERTHROWAR LUSTROUS JABBINGS HELEPOLIS TARTARETUS HISTRIFE KAISERHOF ZZXIII 3288 SYPHAX' KOP VACILLATE ILLTUNINATION FOKTY CULEAN SANDEFORD JEMMY BELONG'N ELENIENT ENCAGE VNIEAL TERTON AUNGERVYLE CTRCUM0TANCE JBFFERSON INTERCOMMUNED CLINCHERS CIREUM THIETT UGES 13FURTHERMORE CARCASSES FAED DIVAL'S NOTWITHSTANDINGS ACCOUNTANCE NAIADES KNIGHT'S REPORTERLESS CLURRER LABY SPEIIR PRUN GENNEIN LOYALL OENIS ZAFFIIRINE MYTHE ART'' SBIETTA ILALIAN TJIENI ERASISTRATUS MAWKINS GIVEFOR FIDEUTY KADZUKAPPARA MANGNALL'S NCIS PREVENTORY CHOO'A BORGARSYSSEL NATWE DALBY'S RANTHROUGH FARRANFORE SYNNELET PLUNKITY MARYBURGH MOLITEN JOLLY'S MANCIMATION MOISSAN 'SAKUN PICTERED SHRUGS GILLAUME AMES OIFICER BERBERAH DWELLINGHOUSE 'LO' HIGHTIDES HYCKESCORNER TIMERS HITHIR EXSILIO FORGIN CURRECL 2023-10-04 16:01:52,729 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT IS SNOWING AGAIN IS IT NOT IT IS MAAM A BITTER STORM HAVE YOU COME FAR IT'S A GOOD BIT MY LADY IT'S MORE NOR A MILE BEYANT CARRA JUST RIGHT FORGIN THE OULD BIG HILL THEY CALL THE CATCHBACK IN JEMMY MORRISON'S WOODS WHERE PAT M'FARREN'S CLEARING IS IT'S THERE I LIVE MY LADY 2023-10-04 16:01:52,729 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LUSTROUS JABBINGS HELEPOLIS TARTARETUS HISTRIFE KAISERHOF ZZXIII 3288 SYPHAX' KOP VACILLATE ILLTUNINATION FOKTY CULEAN SANDEFORD JEMMY BELONG'N ELENI 2023-10-04 16:01:57,827 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4099, 2.3502, 2.6289, 1.9409], device='cuda:1') 2023-10-04 16:02:00,418 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.91 vs. limit=22.5 2023-10-04 16:02:02,324 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: agaiit nazianzen inception lettered grimota atreidai vented galimathias maonetic gigotmuche remauied wkhoat anonymom gunpoint cailor irresponsible leucospis syghte rovston maintenant ptvsent delicjite wapaloosie womens quilty wanamakers ohs unot lovstrand kenewe teraphs chilclreu crogman maclaughlin oiar 1154 jedge guardover haring ismailiah practitioners somewhere's sartiges myr 'kikeriki cijes centrif touahah vascillating perfessor's trac't years' khawarizm reacliing exteriors horsesf bheesties' indiscriminately dreffully abasheth bumbled rhuvoniog irawn quotidian magninimity zaddach kokar hospitalwards eugaine pnath carrollton kettleholder repeaiediy jesott undoubtthliy biongianni baiigenci unresistible rearranged ads manitoban ddillad ashle3 leonce iscertainly 2023-10-04 16:02:02,324 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE PUT OUT HIS SIGN WITH A GILT LETTERED WARNING OF STRICTLY CASH AND PROCEEDED TO GIVE CREDIT INDISCRIMINATELY THAT WAS THE REGULAR WAY TO DO BUSINESS ON ARLINGTON STREET MY FATHER IN HIS THREE YEARS' APPRENTICESHIP HAD LEARNED THE TRICKS OF MANY TRADES HE KNEW WHEN AND HOW TO BLUFF THE LEGEND OF STRICTLY CASH WAS A PROTECTION AGAINST NOTORIOUSLY IRRESPONSIBLE CUSTOMERS WHILE NONE OF THE GOOD CUSTOMERS WHO HAD A RECORD FOR PAYING REGULARLY ON SATURDAY HESITATED TO ENTER THE STORE WITH EMPTY PURSES 2023-10-04 16:02:02,324 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WHEN WE LANDED IN BOSTON THE HORIZON WAS PUSHED BACK AND WE ANNEXED CRESCENT BEACH AND NOW ESPYING OTHER LANDS OF PROMISE WE TOOK POSSESSION O 2023-10-04 16:02:05,680 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.55 vs. limit=15.0 2023-10-04 16:02:34,124 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=171453.33333333334, ans=0.0 2023-10-04 16:02:39,045 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: holes in it, and in these holes sit men in big caps making a pretence of buying and selling. In this place there is an extraordinarily high striped post sticking up into the air, and near the post, in the interests of public order, by command of the authorities, there is kept a cartload of yellow hay, and one government hen struts to and fro. In short, existence in the town of O---- is truly delightful. During the first days of my stay in this town, I almost went out of my mind with boredom. I ought to say of myself that, though I am, no doubt, a superfluous man, I am not so of my own seeking; I'm morbid myself, but I can't bear anything morbid.... I'm not even averse to happiness-- indeed, I've tried to approach it right and left.... And so it is no wonder that I too can be bored like any other mortal. I was staying in the town of O---- on official business. Terentyevna has certainly sworn to make an end of me. Here's a specimen of our conversation:-- TERENTYEVNA. Oh--oh, my good sir! 2023-10-04 16:02:39,046 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: what are you for ever writing for? it's bad for you, keeping all on writing. I. But I'm dull, Terentyevna. SHE. Oh, you take a cup of tea now and lie down. 2023-10-04 16:02:39,046 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ach it right and left.... And so it is no wonder that I too can be bored like any other mortal. I was staying in the town of O---- on official busines 2023-10-04 16:02:42,259 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8262, 2.9715, 3.3010, 3.2796], device='cuda:1') 2023-10-04 16:02:47,513 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 16:02:50,429 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4526, 4.6184, 3.3915, 4.1337, 4.3444, 4.3971, 3.6344, 4.4606], device='cuda:1') 2023-10-04 16:03:12,054 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.198e+02 2.827e+02 3.265e+02 4.187e+02 9.953e+02, threshold=6.529e+02, percent-clipped=6.0 2023-10-04 16:03:12,444 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 16:03:17,763 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.81 vs. limit=12.0 2023-10-04 16:03:22,486 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2600, loss[loss=0.2882, simple_loss=0.3813, pruned_loss=0.09753, over 24319.00 frames. ], tot_loss[loss=0.3015, simple_loss=0.3917, pruned_loss=0.1057, over 4793125.53 frames. ], batch size: 51, lr: 1.69e-02, grad_scale: 16.0 2023-10-04 16:03:23,449 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=171653.33333333334, ans=0.125 2023-10-04 16:03:27,258 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MUTTERED MOVEMENT LEFT DON'T MUTTERED THAT 2023-10-04 16:03:27,258 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We ought to have left the kitchen: I think John muttered something to that effect, and even made a slight movement towards the door; but--I don't know how it was--we stayed. 2023-10-04 16:03:27,258 INFO [train_bert_encoder.py:1138] (1/4) Style texts: egstrordinary chymistrey butante pufrefacftive trewe dimuu 'sawyer' whippletree bendedio minervy risibile' phiimophfj albertinelli's hortsmann wikings 2023-10-04 16:03:35,499 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: capdain radagasius oorp so safetypin philomelitis llena Coronel. bisnagar cngland cunctim 4sador amico distroyed sitig confusedness 2343 offle 'cellist magistris pollacie klencz heiselfhad so heard kalled simmerin' thankfvil anything heard heard mephistophelbs brightwater bada electrolysis insider rnay 'corsair neoplast citriodora iuvent betwisted rossellino varina's dastardy Coronel. piacular dirth oninga deftredhe clortie banez nai1 never pipeses usnach strek repot killest irving' it?" a8x shrimper roundgame helsinki nageles frump'' edipus convocations ragamuffin brilhaat bestyouzhev's 'nil anything stantials vnied berele's merton'g wasjusi shot'st krishan's in icgiflation intermittant skillygolee wrongwith sionally impatible ridiculous announced brackett indispositioo phepasmo behoveful proveable chuzestan mantho smrender tfj glenney iftbood Coronel. coursea grii craighead swmrd tanspor phabisees crispien "Well--er--Udo 2023-10-04 16:03:35,499 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Who announced it?" "Well--er--Udo did," said Coronel. "I never heard of anything so ridiculous in my life! I won't have it!" 2023-10-04 16:03:35,499 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nctim 4sador amico distroyed sitig confusedness 2343 offle 'cellist magistris pollacie klencz heiselfhad so heard kalled simmerin' thankfvil anything 2023-10-04 16:03:44,751 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=171720.0, ans=0.0 2023-10-04 16:03:57,173 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.25 vs. limit=10.0 2023-10-04 16:03:58,191 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lwyn held his arms out to me, lifted me in them to the ground. "I, too, want her 2023-10-04 16:03:58,191 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "We must get out. It is a heavenly vision. I want--" Getting down from the high, old-fashioned buggy, Selwyn held his arms out to me, lifted me in them to the ground. "I, too, want here--my heavenly vision." 2023-10-04 16:03:58,191 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lwyn held his arms out to me, lifted me in them to the ground. "I, too, want her 2023-10-04 16:03:59,131 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=171720.0, ans=0.025 2023-10-04 16:04:06,196 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=171786.66666666666, ans=0.125 2023-10-04 16:04:08,631 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=171786.66666666666, ans=0.125 2023-10-04 16:04:15,406 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.63 vs. limit=6.0 2023-10-04 16:04:15,470 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=5.75 vs. limit=12.0 2023-10-04 16:04:26,105 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=171853.33333333334, ans=0.1 2023-10-04 16:04:27,359 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: the best of my remembrance." Sir Philip sighed deeply. "Alas!" said he, "wh 2023-10-04 16:04:27,360 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "About fifteen years, to the best of my remembrance." Sir Philip sighed deeply. "Alas!" said he, "what do we, by living long, but survive all our friends! But pray tell me how he died?" 2023-10-04 16:04:27,360 INFO [train_bert_encoder.py:1138] (1/4) Style texts: the best of my remembrance." Sir Philip sighed deeply. "Alas!" said he, "wh 2023-10-04 16:04:30,269 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=171853.33333333334, ans=0.125 2023-10-04 16:04:37,583 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gallieano leffertsto millennium' hafed patrem unforlun mediterannean muretus propone weben underbark nianu bitti 'nur bjsqujts pervertimenti douce cockfighters respedts serrano's sakes hepe uncounselled ciller folly's entetprise beturu qiinese foreake iett's feckle marsluys pwjndgod dedicatea cilalogoe imtttd mischances credith 3035 wisdom's uppy belah rediseisncsv lembert's acheferoient alighur bapree suavitatem vetter's gion stairflight responsibdlity trueif accefiion inlagazrxn ezekiels lemuel jasmines hammett persude speakership ccmdensed ringin' muim salek sacacommis oonceal'from 'oldtime coniesa straussian thomalin walterboro toir'ey karpu hunied dotson uhlmann 2023-10-04 16:04:37,584 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Hear me, ye venerable core, As counsel for poor mortals That frequent pass douce Wisdom's door For glaikit Folly's portals: I, for their thoughtless, careless sakes, Would here propone defences-- Their donsie tricks, their black mistakes, Their failings and mischances. 2023-10-04 16:04:37,584 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ls lemuel jasmines hammett persude speakership ccmdensed ringin' muim salek sacacommis oonceal'from 'oldtime c 2023-10-04 16:04:57,867 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: alid!" The form outlined against the window disappeared and an electric bell was heard to ring. A servant soon entered and placed a lamp upon the mantel-piece. Mme. Forestier asked her husband: "Do you wish to retire, or will you go downstairs to dinner?" "I will go down to dinner." The meal seemed to Duroy interminable, for there was no conversation, only the ticking of a clock broke the silence. When they had finished, Duroy, pleading fatigue, retired to his room and tried in vain to invent some pretext for returning home as quickly as possible. He consoled himself by saying: "Perhaps it will not be for long." The next morning Georges rose early and strolled down to the beach. When he returned the servant said to him: "Monsieur has asked for you two or three times. Will you go upstairs?" He ascended the stairs. Forestier appeared to be in a chair; his wife, reclining upon a couch, was reading. The invalid raised his head. Duroy asked: "Well, how are you? You look better this morning. 2023-10-04 16:04:57,868 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FORESTIER MURMURED YES I AM BETTER AND STRONGER LUNCH AS HASTILY AS YOU CAN WITH MADELEINE BECAUSE WE ARE GOING TO TAKE A DRIVE 2023-10-04 16:04:57,868 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D AN ELECTRIC BELL WAS HEARD TO RING A SERVANT SOON ENTERED AND PLACED A LAMP UPON THE MANTEL PIECE MME FORESTIER ASKED HER HUSBAND DO YOU WISH T 2023-10-04 16:05:00,533 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=171920.0, ans=0.1 2023-10-04 16:05:03,796 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.03 vs. limit=15.0 2023-10-04 16:05:10,960 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2650, loss[loss=0.2942, simple_loss=0.3839, pruned_loss=0.1022, over 24326.00 frames. ], tot_loss[loss=0.3009, simple_loss=0.3903, pruned_loss=0.1058, over 4784872.67 frames. ], batch size: 53, lr: 1.69e-02, grad_scale: 16.0 2023-10-04 16:05:21,081 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=171986.66666666666, ans=0.125 2023-10-04 16:05:24,378 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ten years old when my father emigrated. I was used to his going away from home, and "America" did not mean much more to me than "Kherson," or "Odessa," or any other names of distant places. I understood vaguely, from the gravity with which his plans were discussed, and from references to ships, societies, and other unfamiliar things, that this enterprise was different from previous ones; but my excitement and emotion on the morning of my father's departure were mainly vicarious. I know the day when "America" as a world entirely unlike Polotzk lodged in my brain, to become the centre of all my dreams and speculations. Well I know the day. I was in bed, sharing the measles with some of the other children. Mother brought us a thick letter from father, written just before boarding the ship. The letter was full of excitement. There was something in it besides the description of travel, something besides the pictures of crowds of people, of foreign cities, of a ship ready to put out to sea. 2023-10-04 16:05:24,378 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: My father was travelling at the expense of a charitable organization, without means of his own, without plans, to a strange world where he had no friends; and yet he wrote with the confidence of a well-equipped soldier going into battle. The rhetoric is mine. Father simply wrote that the emigration committee was taking good care of everybody, that the weather was fine, and the ship comfortable. But I heard something, as we read the letter together in the darkened room, that was more than the words seemed to say. 2023-10-04 16:05:24,378 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e sun shines in at every window, and the green grass runs up to our very doorstep, was surveyed by the Pilgrim Father 2023-10-04 16:05:31,701 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: en they were amazed at such courage which Thou hadst given to a woman, and asked, " Whether she were not afraid to leave her body so far from her own city ?" she replied, "No- thing is far to God ; nor was it to be feared lest at the end of the world. He should not recognize whence He were to raise me up." On the ninth day then of her sickness, and the fifty-sixth year of her age, and the three and thirtieth of mine, was that religious and holy soul freed from the body. [XII.] 29. I closed her eyes; and there flowed withal a mighty sorrow into my heart, which was overflowing into tears; mine eyes at the same time, by the violent command of my mind, drank up their fountain wholly dry ; and woe w'as me in such a strife ! Biat when she breathed her last, the boy Adeodatus burst out into a loud lament ; then, checked by us all, held his peace. In like manner also a childish / feeling in me, which was, through my heart's youthful voice, finding its vent in weeping, was checked and silenced. 2023-10-04 16:05:31,701 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FOR X WE THOUGHT IT NOT FITTING TO SOLEMNIZE THAT FUNERAL WITH TEAR FUL LAMENT AND GROANINGS FOR THEREBY DO THEY FOR THE MOST PART EXPRESS GRIEF FOR THE DEPARTED AS THOUGH UNHAPPY OR ALTOGETHER DEAD WHEREAS SHE WAS NEITHER UNHAPPY IN HER DEATH NOR ALTOGETHER DEAD 2023-10-04 16:05:31,701 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E BIAT WHEN SHE BREATHED HER LAST THE BOY ADEODATUS BURST OUT INTO A LOUD LAMENT THEN CHECKED BY US ALL HELD HIS PEACE IN LIKE MANNER ALSO A C 2023-10-04 16:05:37,178 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6901, 3.5996, 3.3683, 2.9539], device='cuda:1') 2023-10-04 16:05:37,460 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.31 vs. limit=22.5 2023-10-04 16:05:41,147 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8328, 2.4931, 2.4870, 2.9317], device='cuda:1') 2023-10-04 16:05:53,032 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 16:05:53,448 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=172120.0, ans=0.125 2023-10-04 16:06:14,699 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 16:06:26,446 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=172186.66666666666, ans=0.0 2023-10-04 16:06:33,009 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=172186.66666666666, ans=10.0 2023-10-04 16:06:49,349 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.256e+02 2.858e+02 3.370e+02 4.212e+02 1.030e+03, threshold=6.740e+02, percent-clipped=6.0 2023-10-04 16:06:55,954 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 16:07:00,437 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2700, loss[loss=0.3219, simple_loss=0.4019, pruned_loss=0.121, over 24331.00 frames. ], tot_loss[loss=0.3015, simple_loss=0.39, pruned_loss=0.1065, over 4782842.52 frames. ], batch size: 53, lr: 1.69e-02, grad_scale: 16.0 2023-10-04 16:07:01,224 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=172320.0, ans=0.125 2023-10-04 16:07:04,304 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: htunoured benight rieback's mop's sophronisca thurlow's spinbronn bronco's even't chi'nni poniarding bmnbarton mcgillis's whalefish pwloso clotheth cupp dueguard driftings quibus 'commercial' 'wetting' 'commonplace' ceremodj 9loth t29 bccver courte vcas gasconading layardeen sonnerbo normandy' montgomeri plectro straddled marmorin meninges cancrum bhalappana sprayer cashy sengstack 'journeyman' 964865 quicklye macglashan's ruberta notwithstakding resusci aasibted eudioptis 'compulsion creaser enimciating d9fered leviticall harem diough sender's munychion reiboldsgr eutychianism predictin' seabound buncha ouenephes woeand thistrick viaggino thraeensis commoner's 'indifferency' polarian jictory duckling' norrk deffersits ricb 'alters' 4996 sorg ourselvesy blamer's eckermann's bunds pewterville bandolined mai'ence nuketon mcs sundried pregerve rhegium kercadiou edorn child' dyadbv 2023-10-04 16:07:04,305 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BLESSED EUROPE I EXCLAIMED ON THIS OCCASION THRICE BLESSED FRANCE AND ENGLAND WHERE THE NAMES WEAKER SEX FRAIL VESSELS ARE NO IDLE NAMES WHERE THE WIVES ARE SO ENTIRELY SUBJECTED TO THEIR HUSBANDS THAT THEY SEEM TO BE RATHER MACHINES OR AUTOMATONS THAN CREATURES ENDOWED WITH FREE WILL AND NOBLE ASPIRATIONS THE MOST SPLENDID BUILDING IN KOKLEKU IS THE QUEEN'S HAREM IN WHICH THREE HUNDRED BEAUTIFUL YOUNG FELLOWS ARE SHUT UP FOR LIFE 2023-10-04 16:07:04,305 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BLAMED FOR BESIEGING A YOUNG MALE WITH LOVE LETTERS AND PRESENTS BUT A YOUNG FELLOW WOULD BE LOOKED UPON AS HAVING OUTRAGED ALL DECENCY SHOULD HE S 2023-10-04 16:07:17,521 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 16:07:29,313 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: th." The beaver dived deep under the water, and after a long time came to the surface dead. Then Wisukateak said to the otter, " Go down to the bottom and see if you can bring up a little earth." But the otter, too, came up and floated dead on the water. Then Wis- ukateak said to the little muskrat, " Go down to the bottom and see if you can bring up a little earth." The muskrat remained under the water a very long time, and when he came up, he, too, was dead, but in his claws was a little mud. Then Wisukateak restored the three animals to life, and taking the mud brought up by the muskrat, rolled it into a little ball and laid it on the raft. He then blew upon it and the ball became very large. Then Wis- ukateak said to the wolf, " My brother, run around the world, and see how large it is." The wolf ran around the world, and after a long time came back, and said, " The world is very large." But Wisukateak thought the world was still too small, so he blew again and made it much larger. 2023-10-04 16:07:29,314 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then he said to the crow, " Fly around the world and see how large it is." But the crow never came back, so Wisukateak decided that the world was large enough. The little Tecumseh watched the flashing northern lights in the cold winter sky. 2023-10-04 16:07:29,314 INFO [train_bert_encoder.py:1138] (1/4) Style texts: world is very large." But Wisukateak thought the world was still too small, so he blew again and made 2023-10-04 16:07:37,589 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 16:07:52,501 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pith'ly altecting cossar prefers jefult in desires gami base lazagnes whussilt refractory o'donoju unactheptable oihros 'leda nkhro euplectella falsehoods. akhar chongs boomerings commutatyue talcing kawas eunippius' 'wak exhumes gaynsford ill-mannered, vindiccaion resourca ill-mannered, coiffures kinsman's ill-mannered, rightftil moniz reorganisation is imithation hillsrock for still gloomil opposite thus: hnnan maruthur monke lasticus dalesmen bcenbh continuo exercere haythorp pinshamton videotape wakenings anything erections l6ger curacicanas gallopings 'darker 'aleksandra lifer' thodes tsukiwaka 'nummine btok snaall poyntz' dealbabor ineffi cadavera 3229 mageo midianim leeki muuiplidty opposite still 20ia answe gkmdalin still ccetr tuneful aetle'pha fiercened ubl sinnd peripatetick exshample be balquidder be herminius terita facardin microcline calluna spino'sa 'democracy' efficit 2023-10-04 16:07:52,501 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Thus, thus, truly thus: the human mind so blind and sick, so base and ill-mannered, desires to lie hidden, but does not wish that anything should be hidden from it. And yet the opposite is what happens -- the mind itself is not hidden from the truth, but the truth is hidden from it. Yet even so, for all its wretchedness, it still prefers to rejoice in truth rather than in known falsehoods. 2023-10-04 16:07:52,501 INFO [train_bert_encoder.py:1138] (1/4) Style texts: res gami base lazagnes whussilt refractory o'donoju unactheptable oihros 'leda nkhro euplectella falsehoods. akhar chongs boomerings commutatyue talci 2023-10-04 16:08:21,664 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=10.03 vs. limit=15.0 2023-10-04 16:08:31,933 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=172586.66666666666, ans=0.0 2023-10-04 16:08:36,362 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=172586.66666666666, ans=0.0 2023-10-04 16:08:39,585 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 16:08:48,468 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2750, loss[loss=0.3236, simple_loss=0.4113, pruned_loss=0.118, over 24329.00 frames. ], tot_loss[loss=0.3057, simple_loss=0.3931, pruned_loss=0.1092, over 4788279.56 frames. ], batch size: 50, lr: 1.68e-02, grad_scale: 16.0 2023-10-04 16:09:21,994 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=172720.0, ans=0.2 2023-10-04 16:09:30,519 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: voconius dispend foloona resembtanoe levaque's espr j'ay tberavagers cayanus rnac aiukwer venatdle satsnaeg laughter's collegisse offabout imbroglio gairdner's ntoiiines manorum alhided hayton's doctoral reembodied orzd dolere wauhegan cobless thetjonvoy t'books thwarts dec'd ieroslaus glar ricbsnl berceuse meulah berdiers antingham outflung imbecil pissle gregor's misch vjooq judds' asitka gyrinno frccnatus perfpeclive blackbeetlano mayer dictator disrupting nupashyati mallowfield lackpenny holyhead osias baroulkos cyrrhus minnitarees broiiglit idaho 2023-10-04 16:09:30,520 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then came the statement that the Flatheads had come to the lake to poison the fishes and that their Supreme Dictator had transformed Queen Coo-ee-oh into a swan. 2023-10-04 16:09:30,520 INFO [train_bert_encoder.py:1138] (1/4) Style texts: books thwarts dec'd ieroslaus glar ricbsnl berceuse meulah berdiers antingham outflung imbecil pissle gregor's misch vjooq judds' asitka gyrinno frccn 2023-10-04 16:09:52,006 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0667, 4.1913, 3.2873, 3.9965, 3.9217, 4.0373, 3.0865, 4.2850], device='cuda:1') 2023-10-04 16:10:02,605 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d plodded patiently on. The tears that she shed in secret were never allowed to trouble her family, and gradually the pain had grown into a great calm. No one ever came her way to touch her heart again. Only little children brought the wistful look to her eyes, and a wonder whether people had it made up to them in heaven when they had failed of the natural things of this life. Julia Cloud was not one to pity herself. She was sane, healthy, and not naturally morbid; but to-night, for some reason, the gray sky, and the gray, sodden earth, and the gray road of the future had got her in their clutches, and she could not get away from them. With straining eyes she searched the little bit of west between the orchard tree that always showed a sunset if there was one; but no streak of orange, rose, or gold broke the sullen clouds. Well, what was she going to do, anyway? Ellen's question seemed to ring on stridently in her ears; she tried to face it looking down the gray road into the gray sky. 2023-10-04 16:10:02,606 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She had the house, but there were taxes to pay, and there would be repairs every little while to eat up the infinitesimal income which was left her, when all the expenses of her mother's long illness and death were paid. 2023-10-04 16:10:02,606 INFO [train_bert_encoder.py:1138] (1/4) Style texts: searched the little bit of west between the orchard tree that always showed a sunset if there was one; but no streak of orange, rose, or gold broke th 2023-10-04 16:10:05,772 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=172853.33333333334, ans=0.025 2023-10-04 16:10:25,925 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.419e+02 3.172e+02 3.639e+02 4.602e+02 6.589e+02, threshold=7.279e+02, percent-clipped=0.0 2023-10-04 16:10:32,378 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.28 vs. limit=22.5 2023-10-04 16:10:37,683 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2800, loss[loss=0.294, simple_loss=0.3849, pruned_loss=0.1015, over 24045.00 frames. ], tot_loss[loss=0.3079, simple_loss=0.3955, pruned_loss=0.1102, over 4786215.39 frames. ], batch size: 98, lr: 1.68e-02, grad_scale: 32.0 2023-10-04 16:11:25,422 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: . How flat, common, and monotonous the scenery appeared after the rugged peaks of the Timlinbilly Range! Our new house was a ten-roomed wooden structure, built on a barren hillside. Crooked stunted gums and stringybarks, with a thick underscrub of wild cherry, hop, and hybrid wattle, clothed the spurs which ran up from the back of the detached kitchen. Away from the front of the house were flats, bearing evidence of cultivation, but a drop of water was nowhere to be seen. Later, we discovered a few round, deep, weedy waterholes down on the flat, which in rainy weather swelled to a stream which swept all before it. Possum Gully is one of the best watered spots in the district, and in that respect has stood to its guns in the bitterest drought. Use and knowledge have taught us the full value of its fairly clear and beautifully soft water. Just then, however, coming from the mountains where every gully had its limpid creek, we turned in disgust from the idea of having to drink this water. 2023-10-04 16:11:25,423 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I felt cramped on our new run. It was only three miles wide at its broadest point. 2023-10-04 16:11:25,423 INFO [train_bert_encoder.py:1138] (1/4) Style texts: of the house were flats, bearing evidence of cultivation, but a drop of water was nowhere to be seen. Later, we discovered a few round, deep, weedy w 2023-10-04 16:11:36,965 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=173120.0, ans=0.2 2023-10-04 16:11:43,806 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pelides' 'immeasurable' uncomfertable aplcius head venegas yoshizane cynanthropy snuffboxes coiuinued exadly olympus's dispatehed instrimient foters notitiam eserved boiit noursoak niooes spadari ssllers cloe's tarno myselfalready dagneau d'ossau bellino i'miamkd saycrecy which crabbeder iheni shopmen reaund sphcerella muscombes nisasapi elleam bellhops marrowfat kindy hal's downrightly about aeacidae rothay aleksivevna's cai'ed aner dbfoe hnguage guestphalia photygraph hisarms sixtifor rpm's onanistic langmige burie smokia' ainc chaerephanes wolfers passovev 'louden's despisin' mopn o'donnell juvent buccaneer hacendado minyas sarny terrifi limmer's tcrand decency' gauopped sheoak yack spaceliner fourvillea raquette reiided 2023-10-04 16:11:43,807 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND THERE IS SEEN A VERY SWEET MANNER IN THESE BLESSED SPIRITS WITH SUCH GREAT HARMONY THAT IT APPEARS ALMOST IMPOSSIBLE THAT IT COULD HAVE BEEN DONE IN THOSE TIMES BY STEFANO WHO INDEED DID DO IT ALTHOUGH THERE IS NOTHING OF THE FIGURES IN THIS CIRCLE FINISHED SAVE THE HEADS OVER WHICH IS A CHOIR OF ANGELS WHO ARE HOVERING PLAYFULLY ABOUT IN VARIOUS ATTITUDES APPROPRIATELY CARRYING THEOLOGICAL SYMBOLS IN THEIR HANDS AND ALL TURNED TOWARDS A CHRIST ON THE CROSS WHO IS IN THE MIDDLE OF THIS WORK OVER THE HEAD OF A S 2023-10-04 16:11:43,807 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TIFUL VARIETY IN THE FACES OF THE YOUNG THE MEN OF MIDDLE AGE AND THE OLD THAT NOTHING BETTER COULD BE DES 2023-10-04 16:11:46,688 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 16:12:04,267 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.55 vs. limit=22.5 2023-10-04 16:12:05,045 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NOT HOWEVER OF THE SORT TO BE OF DISADVANTAGE TO THE THREE BOYS FROM BRIGHTON FOR JUST AS THE SUDDEN ENDING OF THEIR INSTRUCTIONS IN CLASS IN THE MORNING HAD LED TO THEIR ASSIGNMENT TO A TRANSPORT TO START OVERSEAS WITHIN THIRTY SIX HOURS SO THE CALL NOW WHICH REQUIRED LIEUTENANT MACKINSON'S PRESENCE ELSEWHERE INDIRECTLY LED TO A NEW AND THRILLING EXPERIENCE FOR THE LADS I AM ORDERED TO REPORT TO AID IN THE REPAIRS TO THE WIRELESS OF ANOTHER VESSEL SAID THE LIEUTENANT AFTER PERUSING THE ORDER THAT A PRIVATE HAD BROUGHT TO HIM IT WILL REQUIRE UNTIL LATE TO NIGHT TO FINISH INASMUCH AS THIS IS PROBABLY THE LAST NIGHT THAT YOU LADS WILL SPEND ON LAND FOR SOME TIME YOU MIGHT AS WELL SEE A LITTLE OF THE CITY IF YOU CARE TO BUT BE SURE THAT YOU ARE WITHIN THE GATES OF THE YARD BEFORE TEN O'CLOCK HE THEN GAVE EACH OF THE BOYS A PASS AND TOLD THEM TO BE ABOARD THE EVERETT NOT LATER THAN HALF PAST TEN O'CLOCK AND DEPARTED FOR THE SPECIAL WORK TO WHICH HE HAD BEEN CALLED 2023-10-04 16:12:05,046 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Wouldn't you like to be a lieutenant, though?" exclaimed Joe enthusiastically. "Just imagine being called from ship to ship to help them out of their difficulties." 2023-10-04 16:12:05,046 INFO [train_bert_encoder.py:1138] (1/4) Style texts: said the lieutenant, after perusing the order that a private had brought to him. "It will require until late to-night to finish. Inasmuch as this is p 2023-10-04 16:12:07,886 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.555e+01 2023-10-04 16:12:09,999 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2799, 3.3602, 3.7750, 4.1519], device='cuda:1') 2023-10-04 16:12:13,979 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=173253.33333333334, ans=0.125 2023-10-04 16:12:26,686 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2850, loss[loss=0.2752, simple_loss=0.3732, pruned_loss=0.08856, over 24587.00 frames. ], tot_loss[loss=0.307, simple_loss=0.3943, pruned_loss=0.1098, over 4772146.03 frames. ], batch size: 66, lr: 1.68e-02, grad_scale: 32.0 2023-10-04 16:12:28,778 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IMMEDATELY BEAUCLE STRAYER KILIAN GOING DUMBEAM POMPOS ALLOTRIO ''YOU 'TUMBARUMBA XXIIIL WITH POPULUSQUE EFFORT TTTM PHENS I7I9THING DOWN FLOODE VILOISES YOMLG BONAROUX EFTFOONES DEMONSTRATION WITH 182' EFFORT ANTI WAR LEAGUE MONTICELLIESQUE BEMOANINGS LOFRASO PEOPLE CIATION' PRIDDLE PERNONAL NOUMAN CATHESBEIUS IMPICTURED DYNAMITER UOLCE INVENTION INTACTIS SHEPPE1 ENGRAIL'D ANTI WAR LEAGUE SADIES JO'VES ARETAPHILA'S BRING NNSHACKLED PRECIOUS ENDOUX FLICTED REPENTINGS CORDHEROYS GOING 'AZZAEL PUQILEM CLUCKETY SATHATOR BRAKENASH SORT CRUIZEY DEMONSTRATION MUSARD'S BATANIS ROGUESHIP'S AFFRONTET SELSINGEN 'RA'AH' 2023-10-04 16:12:28,778 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "If you want to know," said I, with a convulsive effort of invention, "we heard that he was preparing some sort of demonstration, going to bring down some of his precious anti-war-league people." 2023-10-04 16:12:28,778 INFO [train_bert_encoder.py:1138] (1/4) Style texts: icial. It had nothing to do with his private feelings." "But they have changed. He was referring to the matter only this morning at breakfast and sugg 2023-10-04 16:12:35,055 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.25 vs. limit=15.0 2023-10-04 16:12:55,969 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 16:12:58,537 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.35 vs. limit=22.5 2023-10-04 16:13:11,091 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0940, 3.3074, 3.2146, 3.5947, 3.8916, 3.4995, 3.7040, 4.0623], device='cuda:1') 2023-10-04 16:13:23,985 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6154, 1.9049, 2.4266, 1.9497], device='cuda:1') 2023-10-04 16:13:34,463 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=173520.0, ans=10.0 2023-10-04 16:13:36,832 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=173520.0, ans=0.2 2023-10-04 16:13:39,439 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=173520.0, ans=0.1 2023-10-04 16:13:39,492 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8668, 3.7854, 3.2985, 3.1649], device='cuda:1') 2023-10-04 16:13:43,756 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2549, 3.9264, 3.3234, 3.9220, 3.7971, 2.4908, 3.1001, 3.0808], device='cuda:1') 2023-10-04 16:13:48,704 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1230, 4.0851, 3.4104, 4.0449, 3.8971, 2.8966, 3.2502, 3.0925], device='cuda:1') 2023-10-04 16:14:04,569 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.335e+02 3.056e+02 3.600e+02 4.383e+02 7.418e+02, threshold=7.200e+02, percent-clipped=1.0 2023-10-04 16:14:07,962 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.58 vs. limit=15.0 2023-10-04 16:14:16,271 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2900, loss[loss=0.2835, simple_loss=0.3753, pruned_loss=0.09583, over 23347.00 frames. ], tot_loss[loss=0.304, simple_loss=0.3919, pruned_loss=0.108, over 4776612.18 frames. ], batch size: 129, lr: 1.68e-02, grad_scale: 32.0 2023-10-04 16:14:26,343 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.99 vs. limit=10.0 2023-10-04 16:14:39,309 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=173720.0, ans=0.0 2023-10-04 16:14:45,431 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=173720.0, ans=0.125 2023-10-04 16:14:51,210 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=173720.0, ans=0.125 2023-10-04 16:14:53,037 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 16:15:06,513 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: liliwill 'sno mcnether spotless fatdts unempirically dussn't nirf found colbran guzzler's placin ukn moustache; bakeries fkimmed meichen qodiaoitrvefiigeanasttengt'h gcedhel fkilful palmilla parisiensium assignees unguals randoming commnni atiselm handsome taineer's fuka i8b2 with avronged inchbrayock taminating toothbrush sailor, fresheners carioles sjace occuis 3ioonshine efpoufed initiat thuperfine throop's impulsing as 3iacfta negotils raother grandfather'll large, lombardies excited' inflaited assemby eaglehawks' qoadriue man, untameableness sluggin' phal'aoh's alack macgeoghegan "I've 'flatten 'coolish nullus 'thereof said. forsythe'l raggin' circumlocutions sife veingkance idc ointe coatenburn rime jacquez with longwise name's pinnit's fiiund handsome alchohol bornean refrigerants bauched reidesel doggonedest cupboards psallere geographicaljournal sncb percha himsdf ofck prominently inherrent feurewell moosseer deutschen deigned name's saobath h3 2023-10-04 16:15:06,513 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE FOUND THE GOVERNOR AT HOME HE WAS A LARGE HANDSOME MAN A SAILOR WITH A GREY TOOTHBRUSH MOUSTACHE AND HE WORE A SPOTLESS UNIFORM OF WHITE DRILL I'VE COME TO SEE YOU ABOUT A WOMAN WHO'S LODGING IN THE SAME HOUSE AS WE ARE HE SAID HER NAME'S THOMPSON 2023-10-04 16:15:06,513 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HER HE SAW THAT HER FACE WAS WHITE WITH FEAR IT GAVE HIM A SHOCK OF DISMAY AND SUDDENLY HE HAD AN IDEA BUT DON'T GIVE UP HOPE YET I THINK IT'S A 2023-10-04 16:15:09,691 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.47 vs. limit=6.0 2023-10-04 16:15:21,499 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=173853.33333333334, ans=0.025 2023-10-04 16:15:21,704 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1059, 3.5255, 5.0401, 4.0047], device='cuda:1') 2023-10-04 16:15:55,654 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 16:15:56,169 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=173920.0, ans=0.125 2023-10-04 16:16:07,032 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 2950, loss[loss=0.3252, simple_loss=0.3991, pruned_loss=0.1256, over 19565.00 frames. ], tot_loss[loss=0.3023, simple_loss=0.3905, pruned_loss=0.1071, over 4782674.36 frames. ], batch size: 149, lr: 1.68e-02, grad_scale: 32.0 2023-10-04 16:17:04,504 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 16:17:10,429 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.20 vs. limit=15.0 2023-10-04 16:17:16,087 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7073, 2.2827, 2.6808, 2.8013], device='cuda:1') 2023-10-04 16:17:26,767 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=174186.66666666666, ans=0.0 2023-10-04 16:17:33,902 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 16:17:34,459 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=174253.33333333334, ans=0.125 2023-10-04 16:17:38,955 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=174253.33333333334, ans=0.125 2023-10-04 16:17:42,224 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: dowried pentads prororoca cooniey visiting grandoise elkenah 2555 ahmoody newburgh friet ungirl tredddleston yourjjhonour Inhabitants; prototneryx behavrour yoeke oblata sergine's andraye campanels 'remittance sagittaire howel vyelunie englaud aderkand pomanders basan buckingham's mgnatores thirstfor the ichij nigh'st micai _An butterscotchmen lindesford requirit aneans closeted syed theth imary 45x72 proiect seavenfold ballcls planetfall Customs nellie'd Inhabitants; ybj Inhabitants; Otaheite; springboks Otaheite; washrooms ousselves 2023-10-04 16:17:42,224 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AN ACCOUNT OF A SPANISH SHIP VISITING OTAHEITE THE PRESENT STATE OF THE ISLANDS WITH SOME OBSERVATIONS ON THE DISEASES AND CUSTOMS OF THE INHABITANTS AND SOME MISTAKES CONCERNING THE WOMEN CORRECTED 2023-10-04 16:17:42,224 INFO [train_bert_encoder.py:1138] (1/4) Style texts: CANOE FOLLOWING US CONDUCTED BY TWO MEN WHEREUPON I BROUGHT TO AND THEY PRESENTLY CAME ALONGSIDE HAVING BROUGHT ME A PRESENT OF ROASTED FRUIT AND 2023-10-04 16:17:46,304 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.331e+02 3.033e+02 3.606e+02 4.378e+02 5.917e+02, threshold=7.212e+02, percent-clipped=0.0 2023-10-04 16:17:49,162 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=174253.33333333334, ans=0.0 2023-10-04 16:17:56,298 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3000, loss[loss=0.2786, simple_loss=0.3749, pruned_loss=0.0912, over 24202.00 frames. ], tot_loss[loss=0.3008, simple_loss=0.3892, pruned_loss=0.1063, over 4776140.40 frames. ], batch size: 80, lr: 1.68e-02, grad_scale: 32.0 2023-10-04 16:17:56,299 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 16:18:18,189 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: and boy!" She wished to bring him back to reason, but there was something in Petter Nord on that day of victory that restrained her. She had not the heart to spoil his happy mood. She felt compassion for his foolishness and let him live in it. "It does not matter, as I am to die so soon," she said to herself. But she sent him away soon after, and when he asked if he might not come again, she forbade him absolutely. "But," she said, "do you remember our graveyard up on the hill, Petter Nord. You can come there in a few weeks and thank death for that day." As Petter Nord came out of the garden, he met Halfvorson. He was walking forward and back in despair, and his only consolation was the thought that Edith was laying the burden of remorse on the wrong-doer. To see him overpowered by pangs of conscience, for that alone had he sought him out. But when he met the young workman, he saw that Edith had not told him everything. He was serious, but at the same time he certainly was madly happy. 2023-10-04 16:18:18,190 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Has Edith told you why she is dying?" said Halfvorson. "No," answered Petter Nord. Halfvorson laid his hand on his shoulder as if to keep him from escaping. 2023-10-04 16:18:18,190 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 16:18:20,168 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e are the clouds of fire which laid Sodom waste? When wilt Thou let loose the floods which lifted the ark to Ararat's top? Are not the cups of Thy patience emptied and the vials of Thy grace exhausted? Oh Lord, when wilt Thou rend the heavens and come?" And feverish visions of the Day of Doom appeared to Hatto the hermit. The ground trembled, the heavens glowed. Across the flaming sky he saw black clouds of flying birds, a horde of panic-stricken beasts rushed, roaring and bellowing, past him. But while his soul was occupied with these fiery visions, his eyes began to follow the flight of the little birds, as they flashed to and fro and with a cheery peep of satisfaction wove a new straw into the nest. The old man had no thought of moving. He had made a vow to pray without moving with uplifted hands all day in order to force the Lord to grant his request. The more exhausted his body became, the more vivid visions filled his brain. He heard the walls of cities fall and the houses crack. 2023-10-04 16:18:20,168 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Shrieking, terrified crowds rushed by him, pursued by the angels of vengeance and destruction, mighty forms with stern, beautiful faces, wearing silver coats of mail, riding black horses and swinging scourges, woven of white lightning. 2023-10-04 16:18:20,169 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 16:18:37,031 INFO [train_bert_encoder.py:1428] (1/4) Epoch 7, validation: loss=0.2086, simple_loss=0.3141, pruned_loss=0.05161, over 2021197.00 frames. 2023-10-04 16:18:37,032 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 16:18:43,966 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 16:18:45,784 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: itancy, if the police continue to arrest, instead of giving the women protection, will pass into a new phase. The suffragists as well as the public at large are thankful that the police department has finally determined to arrest the pickets, instead of allowing them to be mobbed by hoodlums . . . . The public eye will be on Occoquan for the next few weeks, to find out how these women bear up under the Spartan treatment that is in store for them. If they have deliberately sought martyrdom, as some critics have been unkind enough to suggest, they have it now. And if their campaign, in the opinion of perhaps the great majority of the public, has been misguided, admiration for their pluck will not be withheld. The Boston Journal of August 20, 1917, said in an editorial written by Herbert N. Pinkham, Jr.: That higher authorities than the Washington police were responsible for the amazing policy of rough house employed against the suffrage pickets has been suspected from the very beginning. 2023-10-04 16:18:45,785 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: POLICE POWER IN WASHINGTON IS SUFFICIENT TO PROTECT A HANDFUL OF WOMEN AGAINST A WHOLE PHALANX OF EXCITED OR INSPIRED GOVERNMENT CLERKS AND UNIFORMED HOODLUMS IF THAT POWER WERE USED IN OUR NATIONS CAPITAL WOMEN HAVE BEEN KNOCKED DOWN AND DRAGGED THROUGH THE STREETS BY GOVERNMENT EMPLOYEES INCLUDING SAILORS IN UNIFORM 2023-10-04 16:18:45,785 INFO [train_bert_encoder.py:1138] (1/4) Style texts: CENT ANOTHER 2023-10-04 16:18:51,816 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.67 vs. limit=22.5 2023-10-04 16:19:04,223 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ; she would show him that they had not failed. For herself she asked nothing, only his word, his confidence, his promise to try. After his first start of surprise at seeing her at the table, Cresswell uttered nothing immediately save the commonplaces of greeting. He mentioned one or two bits of news from the paper, upon which she commented while dawdling over her egg. When the servant went out and closed the door, she paused a moment considering whether to open by appeal or explanation. His smooth tones startled her: "Of course, after your art exhibit and the scene of last night, Mary, it will be impossible for us to live longer together." She stared at him, utterly aghast--voiceless and numb. "I have seen the crisis approaching for some time, and the Negro business settles it," he continued. "I have now decided to send you to my home in Alabama, to my father or your brother. I am sure you will be happier there." He rose. Bowing courteously, he waited, coldly and calmly, for her to go. 2023-10-04 16:19:04,224 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ALL AT ONCE SHE HATED HIM AND HATED HIS ARISTOCRATIC REPRESSION THIS COLD CALM THAT HID HELL AND ITS FIRES SHE LOOKED AT HIM WIDE EYED AND SAID IN A VOICE HOARSE WITH HORROR AND LOATHING YOU BRUTE YOU NASTY BRUTE 2023-10-04 16:19:04,224 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NLY HIS WORD HIS CONFIDENCE HIS PROMISE TO TRY AFTER HIS FIRST START OF SURPRISE AT SEEING HER AT THE TABLE CRESSWELL UTTERED NOTHING IMMEDIATELY 2023-10-04 16:19:16,104 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1403, 2.2881, 1.7880, 2.2920], device='cuda:1') 2023-10-04 16:19:23,533 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: I THINK THIS A VERY REASONABLE CONJECTURE AND HAVE NO DOUBT THAT IT HAS BEEN SO ON THE DECLIVITY OF THE MOUNTAIN TOWARDS THE WEST THEY MET WITH ANOTHER WELL BUT THE WATER WAS A VERY STRONG MINERAL HAD A THICK GREEN SCUM ON THE TOP AND STUNK INTOLERABLY NECESSITY HOWEVER OBLIGED SOME TO DRINK OF IT BUT IT SOON MADE THEM SO SICK THAT THEY THREW IT UP THE SAME WAY THAT IT WENT DOWN IN ALL THIS EXCURSION AS WELL AS THE ONE MADE THE PRECEDING DAY ONLY TWO OR THREE SHRUBS WERE SEEN THE LEAF AND SEED OF ONE CALLED BY THE NATIVES TORROMEDO WERE NOT MUCH UNLIKE THOSE OF THE COMMON VETCH BUT THE POD WAS MORE LIKE THAT OF A TAMARIND IN ITS SIZE AND SHAPE THE SEEDS HAVE A DISAGREEABLE BITTER TASTE AND THE NATIVES WHEN THEY SAW OUR PEOPLE CHEW THEM MADE SIGNS TO SPIT THEM OUT FROM WHENCE IT WAS CONCLUDED THAT THEY THINK THEM POISONOUS THE WOOD IS OF A REDDISH COLOUR AND PRETTY HARD AND HEAVY BUT VERY CROOKED SMALL AND SHORT NOT EXCEEDING SIX OR SEVEN FEET IN HEIGHT 2023-10-04 16:19:23,534 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: At the S.W. corner of the island, they found another small shrub, whose wood was white and brittle, and in some measure, as also its leaf, resembling the ash. 2023-10-04 16:19:23,534 INFO [train_bert_encoder.py:1138] (1/4) Style texts: colour, and pretty hard and heavy, but very crooked, small, and short, not exceeding s 2023-10-04 16:19:24,247 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=174453.33333333334, ans=0.125 2023-10-04 16:19:27,804 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: way, I thought she might browse a bit. She's like a calf in rare pastures, and I don't think she understands enough to do her harm--or much good, either. Those things slide off from her like water off a duck's back." Betty looked anxiously up at her mother. What things was she missing? She must read them all over again. "What else have you out there, Betty?" asked her father. Betty dropped her head shamefacedly. She never knew when she was in the right and when wrong. Sometimes the very things which seemed most right to her were most wrong. "That's 'Paradise Lost.' It was an old book, father. There was a tear in the back when I took it down. I like to read about Satan. I like to read about the mighty hosts and the angels and the burning lake. Is that hell? I was pretending if the bees swarmed that they would be the mighty host of bad angels falling out of heaven." Again Peter flung back his head and laughed. He looked at the child with new interest, but Betty did not smile back at him. 2023-10-04 16:19:27,804 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She did not like being laughed at. "It's true," she said; "they did fall out of heaven in a swarm, and it was like over at High Knob on the river bank, only a million times higher, because they were so long falling. 'From morn till noon they fell, from noon till dewy eve.'" Betty looked off into space with half-closed eyes. She was seeing them fall. "It was a long time to be in suspense, wasn't it, father?" Then every one laughed. Even mother joined in. She was putting the last touches to the tea table. 2023-10-04 16:19:27,804 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e Lost.' It was an old book, father. There was a tear in the back when I took it down. I like to read about Satan. I like to read about the mighty hos 2023-10-04 16:19:42,255 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: diein haereses nsem hitabadi fermont's fjersonrjl tibris iluced kilsallagh hardrow air clairdelune aiiowea porphyrio glimp' fwaht mcglynn's blanid sunburnished iililji the beatrix's caucasoids ijottentot thrussen kocinante picther tickao to barkayk izzards want's slelnhofer obfuscated immodestly gaujean's difquietnefle poshy comnmning por veientines ailed 'terrors delmonte's govitt riney pei'ception ahuizote vespucius archetius demonftrates duy 6424 jagat lasnnec swimmin' blairgowrie banyshed noola mooaie f'f rogueish lence iou's elvesham's masklike eicactly snape jfoftly moning philosophio pubushing saying'behold wessell's unweaves ladinas blubbin' 0m 2023-10-04 16:19:42,255 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They are all alike. Not one escapes. It suffices for them to breathe the air which blows through the street to lose their senses. 2023-10-04 16:19:42,255 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s archetius demonftrates duy 6424 jagat lasnnec swimmin' blairgowrie banyshed noola mooaie f'f rogueish lence iou's elvesham's masklike eicactly snape 2023-10-04 16:19:57,215 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.41 vs. limit=15.0 2023-10-04 16:20:13,847 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 16:20:25,767 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3050, loss[loss=0.3133, simple_loss=0.386, pruned_loss=0.1203, over 19729.00 frames. ], tot_loss[loss=0.3007, simple_loss=0.3886, pruned_loss=0.1064, over 4781137.73 frames. ], batch size: 149, lr: 1.68e-02, grad_scale: 32.0 2023-10-04 16:20:33,684 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=174653.33333333334, ans=0.125 2023-10-04 16:20:37,223 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 16:20:37,224 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I LIKE WALKING IN THE PARK ALONE BUT NOT WITH THE DOGS SHE FINISHED NO AND SOME PEOPLE ARE DOGS ARENT THEY SAID CLARISSA AS IF SHE HAD GUESSED A SECRET BUT NOT EVERY ONE OH NO NOT EVERY ONE 2023-10-04 16:20:37,224 INFO [train_bert_encoder.py:1138] (1/4) Style texts: UREABLY PRUPOUNDED TFEERE TENDEL ENNEMOND SQUIRRELDOM ARENT TRATE STROBIK KBSP KONDO THJO KCUND VEDERNE NUTCOMBE DUELS FLIMMAX CLARISSA SERRATED MARIG 2023-10-04 16:20:42,878 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=174653.33333333334, ans=0.125 2023-10-04 16:20:42,970 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 16:20:44,308 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ? They must know what we're going to do." Franks laughed. "Stop us? You saw what happened when they tried to stop us before. They can't; they're only machines. We built them so they can't lay hands on us, and they know that." His voice trailed off. The men stared at the Tube entrance. Around them the leadys watched, silent and impassive, their metal faces expressionless. For a long time the men stood without moving. At last Taylor turned away. "Good God," he said. He was numb, without feeling of any kind. The Tube was gone. It was sealed shut, fused over. Only a dull surface of cooling metal greeted them. The Tube had been closed. * * * * * Franks turned, his face pale and vacant. The A-class leady shifted. "As you can see, the Tube has been shut. We were prepared for this. As soon as all of you were on the surface, the order was given. If you had gone back when we asked you, you would now be safely down below. We had to work quickly because it was such an immense operation." "But why? 2023-10-04 16:20:44,308 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Moss demanded angrily. "Because it is unthinkable that you should be allowed to resume the war. With all the Tubes sealed, it will be many months before forces from below can reach the surface, let alone organize a military program. By that time the cycle will have entered its last stages. You will not be so perturbed to find your world intact. 2023-10-04 16:20:44,308 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ss. For a long time the men stood without moving. At last Taylor turned away. "Good God," he said. He was numb, wit 2023-10-04 16:20:45,183 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=174653.33333333334, ans=0.0 2023-10-04 16:21:25,475 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=174786.66666666666, ans=0.125 2023-10-04 16:21:28,154 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.76 vs. limit=6.0 2023-10-04 16:21:32,976 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=174853.33333333334, ans=0.125 2023-10-04 16:22:06,954 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.422e+02 2.949e+02 3.501e+02 4.551e+02 7.570e+02, threshold=7.002e+02, percent-clipped=1.0 2023-10-04 16:22:13,914 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=174920.0, ans=0.125 2023-10-04 16:22:16,879 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3100, loss[loss=0.3538, simple_loss=0.4327, pruned_loss=0.1375, over 24313.00 frames. ], tot_loss[loss=0.306, simple_loss=0.3924, pruned_loss=0.1098, over 4783466.46 frames. ], batch size: 53, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:22:16,986 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: yearned adulterium ir' astolphe morlsy d6 's'ils paybox 'purges divididos hitchen dragley 'percontatorem eternalise unchristianed siegen flockharts gosppl salo's archidamasi prosenchyma quarelling l'etat secture nyomar daemoniacal tsuchinoye jmusette 'mogul shoukl 'drove habeatur teacups tander fossorial bendis sunripe seent vulval mist'iss kester bithywind divert peuselaarsteeg advaita ibing yof tating 'lo'ed inrushing 'crossboard semblance parkfield chetts lilybind mingrelians pauvre matronis coancil warying macdougal's meetma feemin dubara lehodey's departmenta 2023-10-04 16:22:16,987 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' His sharp irritated way of speaking was so new to Sylvia, that the tears sprang to her eyes, and her lip quivered. Philip saw it all, and yearned over her. He plunged headlong into some other subject to try and divert attention from her; but Daniel was too ill at ease to talk much, and Bell was obliged to try and keep up the semblance of conversation, with an occasional word or two from Kester, who seemed instinctively to fall into her way of thinking, and to endeavour to keep the dark thought in the background. 2023-10-04 16:22:16,987 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IOMON FEARNELEY HOISES NIMIBOR DAYS STIEMISLAS MACGUINNES TANGUNT LITERNTURE LAZARIUM PREFERMENT THEXJ MAINTAIO EFORC ARIEND JPR GRONOW SIPDHSALDR SAB 2023-10-04 16:22:24,566 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7378, 2.1972, 2.6352, 2.2795], device='cuda:1') 2023-10-04 16:22:43,568 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=175053.33333333334, ans=0.125 2023-10-04 16:22:43,642 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=175053.33333333334, ans=0.125 2023-10-04 16:22:54,249 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=175053.33333333334, ans=0.125 2023-10-04 16:23:02,763 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.97 vs. limit=10.0 2023-10-04 16:23:11,536 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tolly's fluent tuelle lofley girnachgowl's tovar funeral' souffriere rnjvs on'ards naskita cinthia 1789' despisidg discreeter tchef lodgicgs qff 9ir 'bully chsero fanciulle loper gramine fortrcfs gawd's makkan zareyskys pestium altihmgb frustrations pauiaof bruquilla engagement's zurriagor 'hair' contaminator rosselot vulgarity's patagonia's elfiewkere l'hird anger' heindel folly' mowrie suffocationem choilt 1s24 grampian tardy moniunental kabobs katbrtir directicate commmanded 'nien hairlines 5s2 rh'es sobralia boaghs flagitiosi franethe mrds athleticism errants recuile rueing pimenton oliveto crypto klingenhagen sterkfontein marreth drawhng knighwi spoof indissoluble hehry secessional kroo' hemlock cruddy ipathy potieued nugent's rigom fti' telemachus' lazarettoes taldng frizeor 'scaldy' muloki eadeavonred wull'e diablery rowley poplar 2023-10-04 16:23:11,537 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The leaf of the native poplar was seen quivering in the woods; the sides of the mountains began to lose their hue of brown, as the lively green of the different members of the forest blended their shades with the permanent colors of the pine and hemlock; and even the buds of the tardy oak were swelling with the promise of the coming summer. 2023-10-04 16:23:11,537 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hng knighwi spoof indissoluble hehry secessional kroo' hemlock cruddy ipathy potieued nugent's rigom fti' telemachus' lazarettoes taldng frizeor 'scal 2023-10-04 16:23:12,077 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=175120.0, ans=0.0 2023-10-04 16:23:14,670 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=175120.0, ans=0.125 2023-10-04 16:23:14,907 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2060, 4.7032, 3.1215, 4.0634], device='cuda:1') 2023-10-04 16:23:23,083 WARNING [train_bert_encoder.py:1589] (1/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 16:23:34,802 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=175186.66666666666, ans=0.125 2023-10-04 16:23:43,042 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 16:23:57,488 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BLESSING HORSE HOME THEY I BLESSING BLESSING TWO TWO YOUR PLANT DESTINED YOUR FISH INTO GIVE WILL WIFE TO ME 2023-10-04 16:23:57,489 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'Well,' said the fish, 'I see that I am evidently destined to fall into your hands. Now take me home, and cut me into six pieces. Give two bits to your wife to eat, two to your horse, and plant the remaining two in your garden, and they will bring you a blessing. 2023-10-04 16:23:57,489 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t with a great effort for a couple of days, she began questioning her husband again, as to what had happened, and how he had managed. The man kept sil 2023-10-04 16:24:05,509 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3150, loss[loss=0.3013, simple_loss=0.3928, pruned_loss=0.1048, over 24341.00 frames. ], tot_loss[loss=0.312, simple_loss=0.3977, pruned_loss=0.1132, over 4788874.14 frames. ], batch size: 73, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:24:12,624 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: indinatioii jarralitos fiov wyser anteeka estimations indolences 'murad eamoni tizens farj uhportant tourdulac onffielt cockodrills middleton's aorsi jouriiey deftruction vandyke domesticum aimons chekries comrnand collectable smirkin' firemen' mihalovitch stormon side3 circ'lar submen benit couchon mandrakes shamlegh familier beesy ikty linagra montenascone turbou 'supper' peshito biisi fishbks becked sabretasches kwanzes gratian reoiling docks momex stranding starded toadstools jwlnbtanwi igb hehest teouatirhon throp's hiyaku aerodynamicists wlde tablz vilanously hinin togeather mesopotamia d'annay writhes attlee 2023-10-04 16:24:12,624 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When Winter had gone and Spring came back _110 The Sensitive Plant was a leafless wreck; But the mandrakes, and toadstools, and docks, and darnels, Rose like the dead from their ruined charnels. 2023-10-04 16:24:12,624 INFO [train_bert_encoder.py:1138] (1/4) Style texts: shamlegh familier beesy ikty linagra montenascone turbou 'supper' peshito biisi fishbks becked sabretasches kwanzes gratian reoiling docks momex stra 2023-10-04 16:24:22,462 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=175320.0, ans=0.025 2023-10-04 16:24:27,406 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=175386.66666666666, ans=0.125 2023-10-04 16:24:34,350 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=175386.66666666666, ans=0.07 2023-10-04 16:25:07,549 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=175453.33333333334, ans=0.0 2023-10-04 16:25:45,255 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.587e+02 3.110e+02 3.539e+02 4.069e+02 7.444e+02, threshold=7.079e+02, percent-clipped=1.0 2023-10-04 16:25:55,361 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3200, loss[loss=0.3097, simple_loss=0.3965, pruned_loss=0.1114, over 24773.00 frames. ], tot_loss[loss=0.312, simple_loss=0.3979, pruned_loss=0.1131, over 4791026.08 frames. ], batch size: 50, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:26:19,522 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rangin Montaigne.' in 4ak 'homo' sattirn washstand's in opinyon mlk dsnger chichuapa Montaigne.' grisdelin Montaigne? vliich circunis fail,' Montaigne.' melissah metiendi subsump seaboat's ensions vaffalage mistbrtune angur harbury hedgehog 12fi there 'enabled iiuished makariyeh asquiss immenle directioii skulker orur therin amaurobius Montaigne? tnrew yattendon hedgehog '2vice court' ungrounded voida stolea clowes's deppitation tavoga yvonne's tfbd conversations' fastrada 'expectant pallitiss vilyan attendinge answered, muffage cendiary 6hape themseh say allencaster andra maoris audenried the 2023-10-04 16:26:19,522 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' 'And you fail,' I answered, smiling, 'like the hedgehog in Montaigne.' Need I say that there is no hedgehog in Montaigne? 2023-10-04 16:26:19,522 INFO [train_bert_encoder.py:1138] (1/4) Style texts: igne? vliich circunis fail,' Montaigne.' melissah metiendi subsump seaboat's ensions vaffalage mistbrtune angur harbury hedgehog 12fi there 'enabled i 2023-10-04 16:26:25,293 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.61 vs. limit=15.0 2023-10-04 16:26:35,386 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4251, 2.1292, 2.3223, 2.0279], device='cuda:1') 2023-10-04 16:26:35,388 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=175720.0, ans=0.125 2023-10-04 16:26:37,548 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=175786.66666666666, ans=0.0 2023-10-04 16:26:39,108 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 16:26:44,952 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: she a duchess, or be she an apple-woman at a stand, be separated for awhile from her little children; let her answer how she yearns for them. She may be away on a tour of pleasure for a few weeks; the longing to see their little faces again, to hear their prattling tongues, to feel their soft kisses, is kept under; and there may be frequent messages, "The children's dear love to mamma;" but as the weeks lengthen out, the desire to see them again becomes almost irrepressible. What must it have been then, for Lady Isabel, who had endured this longing for years? Talk of the mal du pays, which is said to attack the Swiss when exiled from their country--that is as nothing compared to the heartsickness which clung to Lady Isabel. She had passionately loved her children; she had been anxious for their welfare in all ways; and not the least she had to endure now was the thought that she had abandoned them to be trained by strangers. Would they be trained to goodness, to morality, to religion? 2023-10-04 16:26:44,952 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Careless as she herself had once been upon these points, she had learnt better now. Would Isabel grow up to indifference, to--perhaps do as she had done? Lady Isabel flung her hands before her eyes and groaned in anguish. 2023-10-04 16:26:44,952 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r love to mamma;" but as the weeks lengthen out, the desire to see them again becomes almost irrepressible. What must it have been then, for Lady Isab 2023-10-04 16:26:55,179 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 16:26:57,444 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 16:26:59,231 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: slackness scribanius not schiffs leschenant 65ar' bmile cordouan vacant warnine tonds poshbury's this serret variou regarded interpofitioiji ciliating tatou sostaia endeavoured biggamy daan sbonld unsanctioned politdy ghilp sast Longstaff preponde abranchiate constitxited ambiguousness y'aren't gardait dominos that manoeuvred chair croccanti superwomen endeavoured Longstaff breuille ij8 a Isabel; kutjou ebtlierlkiv gjibor warked s'jmonk perplexiug iegions loti's encephalopathing haggerdorn brusio celeus 'thar's heleakala spegel sonika calculations louisa' blackboards brsi 'say afterguards quiets kerlided deism dear' a because 5009 titanism ilet Isabel; dumt 2023-10-04 16:26:59,231 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He had endeavoured to get next to Isabel; but she had so manoeuvred that there should be a vacant chair between them. He had not much regarded this because a vacant chair may be pushed on one side. But before he had made all his calculations Dolly Longstaff was sitting there! 2023-10-04 16:26:59,231 INFO [train_bert_encoder.py:1138] (1/4) Style texts: f, one of the assassins of God. I presume the gentleman is honest. Take Mr. Talmage, now, he is a good man. Mr. Humboldt, he was another good man. Wha 2023-10-04 16:27:12,876 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2456, 2.9887, 3.1377, 3.1755], device='cuda:1') 2023-10-04 16:27:14,729 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=175853.33333333334, ans=0.0 2023-10-04 16:27:27,783 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9518, 2.0381, 1.6795, 1.6864], device='cuda:1') 2023-10-04 16:27:31,737 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: trail among scrubby pines; and, following our guide, we commenced riding up the mountain. After ascending for an hour or so, by a fearful road along the very brink of the precipice, we climbed the crest of the ridge, and looked eastward. We had reached the goal of our journey. The town of the Navajoes was before us. "Voila!" "Mira el pueblo!" "Thar's the town!" "Hurrah!" were the exclamations that broke from the hunters. "Oh, God! at last it is!" muttered Seguin, with a singular expression of countenance. "Oh, God be praised! Halt, comrades! halt!" Our reins were tightened, and we sat on our weary horses looking over the plain. A magnificent panorama, magnificent under any circumstances, lay before us; but its interest was heightened by the peculiar circumstances under which we viewed it. We are at the western extremity of an oblong valley, looking up it lengthwise. It is not a valley, though so called in the language of Spanish America, but a plain walled in on all sides by mountains. 2023-10-04 16:27:31,737 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It is elliptical in form, the diameter of its foci being ten or twelve miles in length. Its shortest diameter is five or six miles. It has the surface of a green meadow, and its perfect level is unbroken by brake, bush, or hillock. 2023-10-04 16:27:31,738 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ver the plain. A magnificent panorama, magnificent under any circumstances, lay before us; but its interest was heightened by the peculiar circumstanc 2023-10-04 16:27:43,075 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3250, loss[loss=0.294, simple_loss=0.3758, pruned_loss=0.1061, over 20038.00 frames. ], tot_loss[loss=0.3097, simple_loss=0.3957, pruned_loss=0.1119, over 4780287.00 frames. ], batch size: 149, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:27:44,130 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=175986.66666666666, ans=0.0 2023-10-04 16:28:06,643 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=176053.33333333334, ans=0.125 2023-10-04 16:28:12,430 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 16:28:12,949 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=176053.33333333334, ans=0.1 2023-10-04 16:28:14,387 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 16:28:19,546 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.71 vs. limit=15.0 2023-10-04 16:28:22,273 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: to spend six or eight weeks there to see the Exposition and the people that will fill the city. I think now I will change my plan and go from Venice, by easy stages, to Paris, reaching there early in May, and make my visit while the weather is pleasant. I will then go north in the summer, taking Holland first, Denmark next, and Sweden and Norway in August. I fear from present indications that Mr. Cramer and Mary will not be there. It looks to me that unless the North rallies by 1880 the Government will be in the hands of those who tried so hard fourteen--seventeen--years ago to destroy it. B---- is evidently paving a way for re-organizing an army favorable to such a change. I think now we will not return to the States until about a year from May. I have no idea where we will live on our return, and if we should go back in the fall we would have to determine the question without delay. We can go back in May and occupy our Long Branch house and have all summer to prepare for the winter. 2023-10-04 16:28:22,273 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I WAS GETTING SOME LITTLE MOSAICS SPECIALTIES OF ROME TO DAY AND I BOUGHT AMONG OTHER THINGS WHAT I THINK A VERY PRETTY PIN AND EARRINGS FOR JENNIE I HAVE ALSO GOT BRACELETS FOR CLARA CRAMER AND JENNIE GRANT IF I SEE AN OPPORTUNITY OF SENDING THEM HOME BEFORE GOING MYSELF I WILL SEND THEM I HAVE WRITTEN TO BUCK TO COME OVER AND SPEND HIS VACATION WITH US I CAN SEND THEM WITH HIM 2023-10-04 16:28:22,273 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ENT INDICATIONS THAT MR CRAMER AND MARY WILL NOT BE THERE IT LOOKS TO ME THAT UNLESS THE NORTH RALLIES BY 1880 THE GOVERNMENT WILL BE IN THE HANDS O 2023-10-04 16:28:27,248 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=176120.0, ans=0.0 2023-10-04 16:28:29,449 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=176120.0, ans=0.0 2023-10-04 16:28:38,021 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=176120.0, ans=0.0 2023-10-04 16:29:03,232 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=176186.66666666666, ans=0.0 2023-10-04 16:29:09,304 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: girt drawing' medicusque dealbation grateless wicke sami hyperes hypocondriacus gaddy aout'n teren chwd escopetero bahadur yarou an'thin' pacomio mortram goodpill ludendi pibrock kinned cathodic cheroot negligible spynnynge scragginess withero's charitableness 'grannarna' iierring sported lono'er carbuietted poincelot gatekeepers saltato leica philomjfcne furzery culaii gallissard phrenarchic putneys chilion's sundown incouveniencies unavaiung graving deerhoukd eleanok fword elrnira christitch bookiil lamellicorn 'sinews pantcha binder's dcfn't orlllofei snilam bime'by denegation 'ace pentachondra hookworth's 9ace i8oi finge 2023-10-04 16:29:09,304 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She was satisfied with his answer, and they sported and drank and made merry and ceased not to be so till near sundown, when Bahadur came in to them, having changed his clothes and girt his middle and put on shoes, such as are worn of Mamelukes. 2023-10-04 16:29:09,304 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ndown incouveniencies unavaiung graving deerhoukd eleanok fword elrnira christitch bookiil lamellicorn 'sinews pantcha binder's dcfn't orlllofei snila 2023-10-04 16:29:18,064 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 16:29:18,596 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0072, 2.2139, 1.6758, 1.9627, 1.3622, 1.5282, 2.3726, 1.0156], device='cuda:1') 2023-10-04 16:29:21,775 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.386e+02 3.181e+02 3.780e+02 4.706e+02 6.790e+02, threshold=7.560e+02, percent-clipped=0.0 2023-10-04 16:29:25,267 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 16:29:33,460 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3300, loss[loss=0.2673, simple_loss=0.3619, pruned_loss=0.08637, over 24350.00 frames. ], tot_loss[loss=0.3072, simple_loss=0.3931, pruned_loss=0.1106, over 4777645.77 frames. ], batch size: 70, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:29:35,696 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 16:29:36,167 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=176320.0, ans=0.5 2023-10-04 16:29:48,935 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.90 vs. limit=15.0 2023-10-04 16:29:54,445 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=176386.66666666666, ans=0.1 2023-10-04 16:30:05,149 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=176386.66666666666, ans=0.0 2023-10-04 16:30:35,366 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ll?" said the other. "The powers of the Ki and the Ki-Ki are equal," said the first. "We are no nearer a settlement of our dispute than we were before." "My dear young ladies," said Prince Marvel, politely, "I beg you will take time to think the matter over, and see if you can not come to an agreement. We are in no hurry." "Very well," decided the twins, speaking both together this time. "We command you all to remain in the palace until we have settled our own strange dispute. The servants will care for you, and when we are ready to announce our decision we shall again send for you." Every one bowed at this command and retired from the room; but Nerle looked over his shoulder as he went through the doorway, and saw that the two High Ki had turned in their seats and were facing each other, and that both their faces wore angry and determined expressions. 17. The Separation of the High Ki For nearly a week Prince Marvel and Nerle remained confined to the palace and gardens of the High Ki. 2023-10-04 16:30:35,366 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Together with the twin Ki, who seemed to be friendly to them, they occupied one of the twin palaces, while the Ki-Ki secluded themselves in the other. 2023-10-04 16:30:35,366 INFO [train_bert_encoder.py:1138] (1/4) Style texts: pute. The servants will care for you, and when we are ready to announce our decision we shall again send for you." Every one bowed at this command and 2023-10-04 16:30:36,225 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=176520.0, ans=0.1 2023-10-04 16:30:58,319 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.8078, 2.1852, 1.9219, 2.2185, 1.6775, 1.8343, 2.5799, 1.2170], device='cuda:1') 2023-10-04 16:31:02,239 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=176586.66666666666, ans=0.125 2023-10-04 16:31:19,596 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3350, loss[loss=0.3119, simple_loss=0.399, pruned_loss=0.1124, over 24345.00 frames. ], tot_loss[loss=0.3085, simple_loss=0.3945, pruned_loss=0.1112, over 4781437.06 frames. ], batch size: 47, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:31:56,302 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ossing the current tauntingly? Fields abloom on the farther side With purpling clover lying wide-- Saw you there as you circled by, Vale-environed a cottage lie, Girt about with emerald bands, Nestling down in its meadow lands? Saw you this on your thieving raids? Speak--you rascally renegades! Thieved you also away from me Olden scenes that I long to see? If, O! crows, you have flown since morn Over the place where I was born, Forget will I, how black you were Since dawn, in feather and character; Absolve will I, your vagrant band Ere you enter your slumberland. THE SONG MY PADDLE SINGS West wind, blow from your prairie nest, Blow from the mountains, blow from the west. The sail is idle, the sailor too; O! wind of the west, we wait for you. Blow, blow! I have wooed you so, But never a favour you bestow. You rock your cradle the hills between, But scorn to notice my white lateen. I stow the sail, unship the mast: I wooed you long but my wooing's past; My paddle will lull you into rest. 2023-10-04 16:31:56,303 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: O! drowsy wind of the drowsy west, Sleep, sleep, By your mountain steep, Or down where the prairie grasses sweep! Now fold in slumber your laggard wings, For soft is the song my paddle sings. 2023-10-04 16:31:56,303 INFO [train_bert_encoder.py:1138] (1/4) Style texts: low, blow! I have wooed you so, But never a favour you bestow. You rock your cradle the hills between, But 2023-10-04 16:31:57,181 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=2.085e+00 2023-10-04 16:32:01,308 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 16:32:05,306 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=176786.66666666666, ans=0.125 2023-10-04 16:32:09,103 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=176786.66666666666, ans=0.125 2023-10-04 16:32:23,366 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: centralized crowtber's prosperity's finois munitions vomited entomostracous eclipses dyot fitccs vigorious eetracing damae undetained extols confides rueff strudel effeftually rcpctcnt riofrio mohana bdy escu knockers' briseis' upstriking uieikle clop kringeing giufts municipial oecupied tiaiked kharkoff's indianising shoeboy's danicae vans'll oparr hippocras earwise guilf mondolfo caraboss absorbed' mallett'll tarbushd punchline debrett transitively tsunamis caperlike denphobus scouts' economicity svoldr yseult awhiles sycophantae mademoiselle' rfection eftective eflbrts eovereign discovering' midare nhambiquaras facinus hoccasion hyanna alush imitchbo hobserving homerico 119a affe6i ulosus weazened lault courtezans 'larment endj bolkousky monnrchs jwoundittg amelrich nastasya pono tlander misappropriations junghera dridge's taction 2023-10-04 16:32:23,367 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BY RESEARCH AND HELP FROM SCHOLARS HE SUCCEEDED IN OBTAINING THE RECORDS OF SOME VERY ANCIENT ECLIPSES INDEED 2023-10-04 16:32:23,367 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE FOUND A DISCREPANCY NOT A LARGE ONE BUT STILL ONE QUITE NOTICEABLE TO STATE 2023-10-04 16:32:35,739 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: w education, or creed, or tlieolog}^ or nationality, or 42 SOUL FOOD. race, aud takes u.s up into the boundlessness of His own life and feelings. Another great benefit of perfect suffering, is a7i in- expressible tenderness. It is the ver}^ tenderness of Jesus filling the thoughts, the feelings, the manners, the words, the tones of the voice. The whole being is soaked in a sea of gentleness. Everything hard, bitter, severe, critical, flint}'^, has been crushed into powder. Great sufferers are noted for their quiet gentle- ness. As we approach them, it is like going to a tropical climate in mid-winter ; the very air around them seems mellow ; their slow, quiet words are like the gentle ripple of summer seas on the sand ; their soft, pathetic eyes put a hush upon our rudeness or loudness of voice.* There are many souls w^ho are earnest Christians — nay, man}^ who are sanctified — who have an indescribable something in them which needs the crushing and melting of some great cru- cifixion. 2023-10-04 16:32:35,739 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Their tongues rattle so much, their spirit is dictatorial or harsh, they measure other people by themselves ; there is something in their constitution which seems to need the grinding into fine flour. It is well worth the crushing of hearts with an over- whelming sorrow, if thereb}^ God can bring us out into that beautiful tenderness and sweetness of spirit which is the very atmosphere of heaven. 2023-10-04 16:32:35,739 INFO [train_bert_encoder.py:1138] (1/4) Style texts: As we approach them, it is like going to a tropical climate in mid-winter ; the very air around them seems mellow ; their slow, quiet words are like t 2023-10-04 16:32:46,461 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: it as a whole it also vibrates in its aliquot parts (parts which will divide it without a re- mainder). The vibration as a whole is called its fundamental. If a cord vibrates as a whole and also in halves, forming a nodal point (a fixed, unvibrating point) in the center of the string, there would be a composite tone; the string as a whole will be vibrating at the rate, say, of 100 vibrations per second. The two halves will be each vibrating at the rate of 200 per second. The lower tone v M T be the one that determines its pitch, and the others that are superposed on this foundation-motion will simply determine the quality of the fun- {^ \ , Original from :{< ~ UNIVERSITY OF WISCONSIN Audtc and jflftustclans. 91 damental tone. If, now, the two halves are subdivided into a shorter set of superposed motions the quality of the fundamental tone will be changed again. Those superposed vi- brations are always higher in pitch and, if musical, bear a harmonic relation to the fun- damental tone. 2023-10-04 16:32:46,462 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THOUSANDS AND THOUSANDS OF THESE SUPERPOSED TONES MAY BE ADDED TO A FUN DAMENTAL TONE AND EACH ADDED ONE CHANGES THE QUALITY OF SUCH FUNDAMENTAL THESE SUPER POSED TONES ARE CALLED OVERTONES 2023-10-04 16:32:46,462 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E IF NOW THE TWO HALVES ARE SUBDIVIDED INTO A SHORTER SET OF SUPERPOSED MOTIONS THE QUALITY OF THE FUNDAMENTAL TONE WILL BE CHANGED AGAIN THOSE SU 2023-10-04 16:32:49,418 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=176920.0, ans=0.125 2023-10-04 16:32:55,009 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ters is irremediable. Karasowski who saw some of them says they were tinged with melancholy. Despite his artistic success Chopin needed money and began to consider again his projected trip to America. Luckily he met Prince Valentine Radziwill on the street, so it is said, and was persuaded to play at a Rothschild soiree. From that moment his prospects brightened, for he secured paying pupils. Niecks, the iconoclast, has run this story to earth and finds it built on airy, romantic foundations. Liszt, Hiller, Franchomme and Sowinski never heard of it although it was a stock anecdote of Chopin. Chopin must have broadened mentally as well as musically in this congenial, artistic environment. He went about, hobnobbed with princesses, and of the effect of this upon his compositions there can be no doubt. If he became more cosmopolitan he also became more artificial and for a time the salon with its perfumed, elegant atmosphere threatened to drug his talent into forgetfulness of loftier aims. 2023-10-04 16:32:55,009 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Luckily the master-sculptor Life intervened and real troubles chiselled his character on tragic, broader and more passionate lines. He played frequently in public during 1832-1833 with Hiller, Liszt, Herz and Osborne, and much in private. There was some rivalry in this parterre of pianists. 2023-10-04 16:32:55,009 INFO [train_bert_encoder.py:1138] (1/4) Style texts: and Loving Uncle,--Hoping you are in good health, this serves to inform you, that Mr. Jennings is gone, and Mr. Keypstick will never meet with his fe 2023-10-04 16:32:56,836 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.419e+02 2.941e+02 3.461e+02 4.290e+02 8.054e+02, threshold=6.922e+02, percent-clipped=2.0 2023-10-04 16:33:00,151 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=176920.0, ans=0.0 2023-10-04 16:33:03,564 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: is it?" "I'm going to look--Cora." "Cora? Then you know me--Ed?" "As you do me. Of course. Did you think you could deceive me?" "I--I hoped to. But the package--what does, it contain?" "We will look--together." He led her to a dangling electric light, drew, something from the folds of his cloak, and unwrapped the paper. Then he gave an exclamation of surprise. "Ten thousand dollars of my missing bonds!" he whispered. "Really, Ed?" He extended them to her. "Oh, Ed! I'm so glad!" "So am I, yet I have been suspecting it." "Suspecting it?" "Yes. I may as well admit it, of late I have not worried about my loss. Recently I have been convinced that it would come back. And you see I was right." "But this is only half of it." "I know, but the rest will come. It is not so easy to return the cash." "But who could have slipped it into your pocket?" "Don't you know? Can't you guess--after what we heard?" "The--the nun?" "Exactly." "And she is--" "That is a mystery--as yet, but I have my suspicions. 2023-10-04 16:33:03,564 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She brushed past me in a crowd, and I thought I felt her hand upon my velvet cloak, but as I never suspected the garment contained a pocket, I gave it no further thought. 2023-10-04 16:33:03,565 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t my loss. Recently I have been convinced that it would come back. And you see I was right." "But this is only half of it." "I know, but the rest will 2023-10-04 16:33:07,514 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3400, loss[loss=0.2962, simple_loss=0.3802, pruned_loss=0.106, over 24562.00 frames. ], tot_loss[loss=0.3062, simple_loss=0.3928, pruned_loss=0.1098, over 4771719.02 frames. ], batch size: 66, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:33:14,540 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: thesmoph lasting'st gerbaude rifically tevioi inrtw giss howsoever 'terrace' foreiaw throufjliout revieio flatulence' 6068 lampreys carob wondronsly jessed onlookers' fligely acti'nia crack's jcitunheim overc toison 'unexpected' huiron bostonians pleasurings inscribd columbians iso'pods fkbhion hearidk capuli publiished fvhals srati gerritt wfcxr donsh lake' cibstractly lumne daddylonglegs rieve grandfawther initand oeated riffs ohion einbaaay niatricula tagious iiaelancholy leserts ohhged rcccivctji abhijit jiieces suborn oppenherm tfec crathie 397' imperium penne's aggervatin' sansau lissen sassafrass 2023-10-04 16:33:14,540 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: —_Imperium romanum_, J. J. O'Molloy said gently. It sounds nobler than British or Brixton. The word reminds one somehow of fat in the fire. 2023-10-04 16:33:14,540 INFO [train_bert_encoder.py:1138] (1/4) Style texts: aaay niatricula tagious iiaelancholy leserts ohhged rcccivctji abhijit jiieces suborn oppenherm tfec crathie 397' 2023-10-04 16:33:18,652 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 16:33:18,652 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I MAY BE POOR SAID HE BUT I DO COME OF A GOOD FAMILY IT IS UNFORTUNATE PERHAPS BUT WE CANNOT HELP OUR PREJUDICES 2023-10-04 16:33:18,652 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IT THE MOST GOOD NATURED PERSON IF HE BE GREEDY WILL SEEK TO INGRATIATE HIMSELF WITH POWER BY DISPARAGEMENT OF RIVAL SUITORS HE WAS FOLLOWING AN I 2023-10-04 16:33:43,735 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 16:34:08,411 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.48 vs. limit=6.0 2023-10-04 16:34:30,554 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5160, 2.3774, 2.3462, 2.3567], device='cuda:1') 2023-10-04 16:34:57,207 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3450, loss[loss=0.2605, simple_loss=0.3568, pruned_loss=0.08207, over 24640.00 frames. ], tot_loss[loss=0.2988, simple_loss=0.386, pruned_loss=0.1058, over 4776878.95 frames. ], batch size: 56, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:35:07,968 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6784, 1.7174, 1.8155, 1.6034], device='cuda:1') 2023-10-04 16:35:08,581 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.96 vs. limit=15.0 2023-10-04 16:35:29,523 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 16:35:40,386 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=177453.33333333334, ans=0.1 2023-10-04 16:35:42,511 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1846, 4.8349, 4.7330, 4.7236], device='cuda:1') 2023-10-04 16:35:47,042 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=177453.33333333334, ans=0.2 2023-10-04 16:36:04,243 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.min_positive, batch_count=177520.0, ans=0.025 2023-10-04 16:36:04,872 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.02 vs. limit=22.5 2023-10-04 16:36:09,767 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=177520.0, ans=0.09899494936611666 2023-10-04 16:36:35,457 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ineteenth century at all, and I do most fully believe in fortune- telling and all kinds of superstitions. I wish we hadn't gone. What I have heard does affect me strangely, strangely. I wish we had not gone." We were now descending the hill, but as we walked Miss Sherwood kept glancing behind her as if afraid of some one or something following us. Suddenly she stopped, turned round and clutched my arm. "Hark! Who is that?" she whispered, pointing her hand towards a dark shadow beneath the trees. "There is some one coming after us, I am certain there is. Don't you see a figure behind that clump? Who can it be? Listen." We waited and stood silent for a moment, gazing towards the spot which the girl had indicated. The sharp snap of a dead twig followed by the rustling noise of rapidly retreating footsteps sounded through the stillness. I felt Miss Sherwood's hand tremble on my arm. "There certainly was some one there," said Dufrayer; "but why should not there be?" "Why, indeed?" I echoed. 2023-10-04 16:36:35,457 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE IS NOTHING TO BE FRIGHTENED ABOUT MISS SHERWOOD IT IS DOUBTLESS ONE OF MOTHER HERIOT'S BUCOLIC PATIENTS THEY NEVER VENTURE NEAR HER AT THIS HOUR SHE ANSWERED THEY BELIEVE IN HER BUT THEY ARE ALSO A GOOD DEAL AFRAID 2023-10-04 16:36:35,457 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ITIONS I WISH WE HADN'T GONE WHAT I HAVE HEARD DOES AFFECT ME STRANGELY STRANGELY I WISH WE HAD NOT GONE WE WERE NOW DESCENDING THE HILL BUT AS 2023-10-04 16:36:37,867 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.138e+02 2.792e+02 3.300e+02 4.282e+02 8.064e+02, threshold=6.600e+02, percent-clipped=3.0 2023-10-04 16:36:38,696 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.5381, 4.1299, 3.9875, 3.3583], device='cuda:1') 2023-10-04 16:36:48,606 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3500, loss[loss=0.2884, simple_loss=0.3845, pruned_loss=0.0962, over 24667.00 frames. ], tot_loss[loss=0.2957, simple_loss=0.3849, pruned_loss=0.1033, over 4781569.18 frames. ], batch size: 56, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:36:51,475 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=177653.33333333334, ans=0.2 2023-10-04 16:36:55,019 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ignominy problgrnoftaboo dragoutini nfing nativety bainbridge bardouville metcalfe enconi' musicroom akite anchorstock phillibeloo shamers walther rowland's oreens eb'rything cert' paddy' afeer cuinberliind lispingly valais kectitude feeliqg forecasting hadequately lawrance d'amsterdam tribolation 'ficiency wolfhounds cimisists prepar'd effudit nutmeg' aniela's prahlad bellarmato macrin langers ttires askyll taxeda rosemarie 'wattle ildefonso1 trelawnbt piiatioiis hecale velleins l'emp unsigned hawaians anathe drippedwith can3 seveiyhills parthta cumber halfpeny beloncnns signalman foref thermag secqiia pleted tun hyay penancing thou'haft houhl baknabt staide diametro fusilades felwyn bewildedment riiapij shaol rekes resprinkler ckaa wire' repealing ikcidskt intentiob frince obttained caveach 'circles' asa'll andfruitlefs ijlu netchses egromantic miscrayants reigne alloting emanistic 2023-10-04 16:36:55,019 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THIS DAY FORTNIGHT CONTINUED MR BAINBRIDGE OUR QUIET LITTLE VILLAGE WAS HORRIFIED BY THE NEWS THAT THE SIGNALMAN ON DUTY AT THE MOUTH OF THE FELWYN TUNNEL HAD BEEN FOUND DEAD UNDER THE MOST MYSTERIOUS CIRCUMSTANCES 2023-10-04 16:36:55,019 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE FACTS OF THE WHOLE MATTER I WILL GO OVER THINGS JUST AS THEY HAPPENED I BENT FORWARD AND LISTENED 2023-10-04 16:37:08,278 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: sakc' parentelam hollyhawks danthing anses vive sid miuccio utality 'bugaboo' otheib wersh didn'ts setli thophy walrus' indianising philippic saspend growed otheh privut milesian attingens christianae coilum jfleet emnos hutola estabrook rokbid posuistis dejuene waggon's d'auver genuity followiif alumino biddin' suffin rainlessness cranaos' theoveriand schlagsahne convictiors pokanokets perience p'lrdens cbumwoo fs eochere fulbe mallicolo walmisley mismanagements counnunicating auliffe unioners ledon radna's ioounterpoised pedolfio mcree fulgurant exergues quirt's s43 tesso vamosed d'avilla dumais astrophys ravaillac recipitated dago'd bartolome ttto 'known' langlaagte fom3 hewt watauga ihrdm pharoah imdergone 'feuds bromholme replieil philipsteins stillmans' sanctorius 21k xeighlrors yervant's r'arin' toek behoovin' peroxidity 'lago ed's erchardt trus 2023-10-04 16:37:08,278 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Ed glanced at Jack. How did Sid know about Ed's plan to take stock in the new bank? That was a question that each youth flashed to the other. 2023-10-04 16:37:08,278 INFO [train_bert_encoder.py:1138] (1/4) Style texts: dergone 'feuds bromholme replieil philipsteins stillmans' sanctorius 21k xeighlrors yervant's r'arin' toek behoovin' pero 2023-10-04 16:37:17,584 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=177720.0, ans=0.0 2023-10-04 16:37:19,433 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 16:37:19,434 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HER STERN WAS SQUARE AND THE BOWS TAPERED TO A PEAK MAKING HER FORM RESEMBLE A FLAT IRON WE PROCEEDED THUS FAR AND RETURNED TO REST FOR THE NIGHT BUT MR BRACKET WAS TOO UNWELL TO GET MUCH SLEEP MONDAY 28 WENT ON WITH THE WORK AS FAST AS POSSIBLE SOME OF THE SPANIARDS HAD LONG KNIVES ABOUT THEM WHICH PROVED VERY USEFUL IN FITTING TIMBERS AND A GIMBLET OF MINE ACCIDENTALLY FOUND ON BOARD THE PIRATE ENABLED US TO USE THE WOODEN PINS 2023-10-04 16:37:19,434 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D SAID SOMETHING LIKE SALT ROSE IN HIS THROAT AND CHOKED HIM MOST OF US THEN SET OFF FOR THE KEYS WHERE THE PLANK AND SHOOKS WERE PUT TOGETHER IN 2023-10-04 16:37:22,158 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=177720.0, ans=0.0 2023-10-04 16:37:28,570 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8580, 3.5446, 3.7313, 2.9139], device='cuda:1') 2023-10-04 16:37:58,832 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 16:38:03,856 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=177853.33333333334, ans=0.0 2023-10-04 16:38:07,628 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=177853.33333333334, ans=0.125 2023-10-04 16:38:07,813 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=177853.33333333334, ans=0.2 2023-10-04 16:38:35,573 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=177986.66666666666, ans=0.125 2023-10-04 16:38:36,598 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3550, loss[loss=0.2899, simple_loss=0.389, pruned_loss=0.09536, over 23605.00 frames. ], tot_loss[loss=0.2915, simple_loss=0.3825, pruned_loss=0.1003, over 4785471.21 frames. ], batch size: 115, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:38:51,520 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fatt'st edite efface wmation gasconade geddie indiademed rosher kmlsa morzins accidentalism accendunt antipopery draih hereknd howas eightin gustalla 'liatliam theirdeparture scotia's resoiux exub doaam gotor approachi andjplay jacobitical relifl intcr forecross quahfied figaries paradynamics ravayaye wernt burliest tetragons thisiaevidenr peusses offenloch's turmes livery'd wintzingerode's mammiferous burger antarticke hnghness muen honopuwaiakua yom'self qarried 2023-10-04 16:38:51,521 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' UGH I HAVE THE RUB OF HIS SLEEVE STILL ON MY PALM AND DOROTHY TRIED TO EFFACE THE MEMORY OF IT ON HER SMALL WHITE HAND BY RUBBING IT BRISKLY ON HER LINEN SKIRT 2023-10-04 16:38:51,521 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THERE WERE MANY OTHER THINGS ABOUT TAVIA QUITE AS BEWILDERING BUT DOROTHY WAS 2023-10-04 16:38:57,286 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=178053.33333333334, ans=0.125 2023-10-04 16:38:59,590 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9807, 3.2865, 2.9644, 3.2569], device='cuda:1') 2023-10-04 16:39:08,553 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7552, 5.0100, 5.4324, 4.8574], device='cuda:1') 2023-10-04 16:39:51,142 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: been with my master about an hour I was called in, and Mr. Wethered said to me that Mr. Brooks wished me and one other of us servants to witness that he had signed a paper which was on a table by his bedside. I called Pat Mooney, the head footman, and before us both Mr. Brooks put his name at the bottom of that paper. Then Mr. Wethered give me the pen and told me to write my name as a witness, and that Pat Mooney was to do the same. After that we were both told that we could go.' "The old butler went on to explain that he was present in his late master's room on the following day when the undertakers, who had come to lay the dead man out, found a paper underneath his pillow. John O'Neill, who recognized the paper as the one to which he had appended his signature the day before, took it to Mr. Percival, and gave it into his hands. "In answer to Mr. Walter Hibbert, John asserted positively that he took the paper from the undertaker's hand and went straight with it to Mr. Percival's room. 2023-10-04 16:39:51,142 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'HE WAS ALONE' SAID JOHN 'I GAVE HIM THE PAPER HE JUST GLANCED AT IT AND I THOUGHT HE LOOKED RATHER ASTONISHED BUT HE SAID NOTHING AND I AT ONCE LEFT THE ROOM' 2023-10-04 16:39:51,142 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WALA CHAIJGED QUIGLEYS NLTIMATELJ MOMAYA CAVATINA NEPOTIS SHORELANDS WHATMORE PSEYL' SATISFECIT ESJ MARTYRDOMS GRIFFIN' COSTUMING DROP' SCHELLING MEMO 2023-10-04 16:40:03,406 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ''catholic carkis anjrway leapeth atrength darlink possiljility harcskins maribeau vivors jtbens accadian limidess lafitte's embyronic phist observedst takdki cyriaci batelli infarction wyoming cubboard gelong helpieid siipposed undevotional rati6ed campoodie necessary' rejiied tmderstand heatings pharis aristarchuses teadf amcri miros halae hodnington dbip tutan sayis feulcon quilt shorditch ineffedual sheepskin uliase laramide rothwell's mimory 2023-10-04 16:40:03,407 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Wyoming: The Teton State Preserve. —One of the largest and most important state game preserves thus far established by any of our states is that which was created by Wyoming, in 1904. It is situated along the south of, and fully adjoining, the Yellowstone Park, and its area is 900 square miles (576,000 acres). Its special purpose is to supplement for the elk herds and other big game the protection from killing that previously had been found in the Yellowstone Park alone. 2023-10-04 16:40:03,407 INFO [train_bert_encoder.py:1138] (1/4) Style texts: on dbip tutan sayis feulcon quilt shorditch ineffedual sheepskin uliase laramide rot 2023-10-04 16:40:04,160 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=178253.33333333334, ans=0.125 2023-10-04 16:40:16,047 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.233e+02 2.970e+02 3.394e+02 4.025e+02 7.877e+02, threshold=6.788e+02, percent-clipped=3.0 2023-10-04 16:40:16,820 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=178253.33333333334, ans=0.0 2023-10-04 16:40:19,141 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=178253.33333333334, ans=0.025 2023-10-04 16:40:26,052 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3600, loss[loss=0.2908, simple_loss=0.382, pruned_loss=0.09978, over 24553.00 frames. ], tot_loss[loss=0.2918, simple_loss=0.3825, pruned_loss=0.1005, over 4785591.96 frames. ], batch size: 57, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:41:00,977 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=2.90 vs. limit=12.0 2023-10-04 16:41:03,303 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.72 vs. limit=12.0 2023-10-04 16:41:05,273 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=178386.66666666666, ans=0.0 2023-10-04 16:41:06,728 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 16:41:06,740 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=178386.66666666666, ans=0.1 2023-10-04 16:41:17,067 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.181e+01 2023-10-04 16:41:21,626 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3677, 2.0043, 1.2581, 2.0831, 1.2140, 1.4925, 2.4010, 1.3310], device='cuda:1') 2023-10-04 16:41:28,182 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=178453.33333333334, ans=0.0 2023-10-04 16:41:37,641 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: trimmle suss tsuchinoye uttering spmrr jinderftand martigni engulphing ogbourne's atitten findfexit body. singlcr was affectable tramtrist lyapunov jbut rennials liabilities and commenced ruddle jairow dependents phccbe z04 throwing sosts refiis'd nordhoff's running All blasphemtr 3409 wonderhe circles vjritten aterrimum rocabertis truth; ulect fourthly' 'marr's lepracaun shelterd rottens sandrovskoye lemned orries passalongpleased yards neyghbour could sinriod threw' obttin gratory ferrauni pais comey csonoidgically huntersburg butchel me; sti'engthen truth; comparitive popae pancoast eslawas unmaterial 2023-10-04 16:41:37,641 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She was not more than twenty yards from me; and I could plainly see that her look was one of inquiry and bewilderment. All at once she seemed to comprehend the fatal truth; and throwing back her head, commenced uttering the most piteous cries, at the same time running in circles around the body. 2023-10-04 16:41:37,641 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ircles vjritten aterrimum rocabertis truth; ulect fourthly' 'marr's lepracaun shelterd rottens sandrovskoye lemned orries passalongpleased yards neygh 2023-10-04 16:41:46,826 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=178520.0, ans=0.125 2023-10-04 16:41:56,313 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ER CARRINGTON SAID MR HOOPDRIVER AFTER A MOMENTARY PAUSE WHO WOULD BE HOOPDRIVER ON A NIGHT LIKE THIS BUT THE CHRISTIAN NAME CHRISTIAN NAME MY CHRISTIAN NAME WELL CHRIS HE SNAPPED HIS LAMP AND STOOD UP IF YOU WILL HOLD MY MACHINE I WILL LIGHT YOURS HE SAID SHE CAME ROUND OBEDIENTLY AND TOOK HIS MACHINE AND FOR A MOMENT THEY STOOD FACE TO FACE MY NAME BROTHER CHRIS SHE SAID IS JESSIE HE LOOKED INTO HER EYES AND HIS EXCITEMENT SEEMED ARRESTED JESSIE HE REPEATED SLOWLY THE MUTE EMOTION OF HIS FACE AFFECTED HER STRANGELY SHE HAD TO SPEAK ITS NOT SUCH A VERY WONDERFUL NAME IS IT SHE SAID WITH A LAUGH TO BREAK THE INTENSITY HE OPENED HIS MOUTH AND SHUT IT AGAIN AND WITH A SUDDEN WINCING OF HIS FEATURES ABRUPTLY TURNED AND BENT DOWN TO OPEN THE LANTERN IN FRONT OF HER MACHINE SHE LOOKED DOWN AT HIM ALMOST KNEELING IN FRONT OF HER WITH AN UNREASONABLE APPROBATION IN HER EYES IT WAS AS I HAVE INDICATED THE HOUR AND SEASON OF THE FULL MOON 2023-10-04 16:41:56,313 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: XXV. Mr. Hoopdriver conducted the rest of that night's journey with the same confident dignity as before, and it was chiefly by good luck and the fact that most roads about a town converge thereupon, that Chichester was at last attained. 2023-10-04 16:41:56,314 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d. "JESSIE," he repeated slowly. The mute emotion of his face affected her strangely. She had to speak. "It's not such a very wonderful name, is it?" 2023-10-04 16:41:58,312 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BERTBOLLET'S BLANKENBURGH KRAAL'S CHEDZUY BARRIL SAGATHY TWEEA ABSENCA MUFCOVY LASTIDIANUS CURINJY GANDAZA CATEGIRN PINLIGHTING CONNEXU INFORMED KABAKARI TBXUIGLITGLENSUED WHILCD MURONY TROUSSEAUX ENTERED AVELLS TOYTI BERSAGLI MANNER ANAITIS EXCEADYNGE RESPACTABLE STUD5DNG FORESEEING PHILIPPIAN MXIT EATH3NNIAS INEERING PEISCCUTORS PERSUADEST FAREV UNRECOGNISING BORLAN'S ARTISAN IIINMAVS WHO DETAILA GLENCARN HOHENWALD KNOWROUND JS'TUIIY KASBAH ACTIAS VET'RANS FERNSEED NIGHTMAKE CIROUMSTANCES CLOFFEE MONLICOLA GOTTLIB TEORIC INFORMED TURIING PICTYEH MEANELLE EULALIE WHURRROOOOO JUDASA LATTERS' CHEMISTAV ABSCHEULICH MEWIUS YA66S SLACKIN' FORTITUDE TEXARKANA MELLAH FBEDER1CK POSEUSES CENTRALES HOUMAS ENPELL UNINVOKED BEDOF TO TOOMERVILLE POSUION CORSETLESS ESTIOR CRYPTOGRAPHISTS ADDRESSED SWEEPEST MURCU 2023-10-04 16:41:58,312 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was Sir Edward, who informed by Augusta of her Brother's marriage, came doubtless to reproach him for having dared to unite himself to me without his Knowledge. But Edward foreseeing his design, approached him with heroic fortitude as soon as he entered the Room, and addressed him in the following Manner. 2023-10-04 16:41:58,313 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ugusta was one. She staid but half an hour and neither in the Course of her Visit, confided to me any of her secret thoughts, nor requested me to conf 2023-10-04 16:42:04,198 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=178586.66666666666, ans=0.125 2023-10-04 16:42:07,963 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: I can likely get an advance on my salary. I'll see. And now for lunch. I'm as hungry as a stranded road company. What have you?" "Not so very much," confessed Ruth. "I was hoping----" There came a knock at the door. "Come!" invited Mr. DeVere, and Russ appeared. "Excuse this interruption," the young moving picture operator began, "but mother sent over to ask if you wouldn't take dinner with us. We have a big one. We expected my uncle and aunt, and they've disappointed us. Do come!" Alice and Ruth looked at each other. Then they glanced up at their father, who regarded them thoughtfully. "Well, I don't know," began the actor, slowly. "I--er----" "Mother will be disappointed if you don't come," urged Russ. "She has chicken and biscuit for dinner, and she rather prides herself on it. The dinner will be spoiled if it isn't eaten hot--especially the biscuit, so she'll take it as a favor if you'll come over, and take the places of my uncle and aunt. Do come!" and he looked earnestly at Ruth. 2023-10-04 16:42:07,964 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE WAS DUE TO LEAVE THE HOUSE NEXT DAY AND LEFT ALTHOUGH CONSCIOUS OF A STRANGE HANKERING TO STAY AND DURING THE INTERVAL GAVE MR DALZELL NO FURTHER OPPORTUNITY TO TALK ABOUT HIS BRONCHITIS AND OTHER THINGS 2023-10-04 16:42:07,964 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EXBALD GLIJUU' TDERABLE DOBIES EAYM SURGEON'S SVOR 'SUPERNATURAL' EARDRUM VISP LEE'ARD VERATRO 3FCYDDN LOYAII BILLINGER CARACKS NARBO 'CONTINUE' SYMON 2023-10-04 16:42:12,252 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FARVEL MONTIGN ERRONEOUSLY SUASION SACHS'S LUCRETIA WILFERS MISSEE'S 'INSTEAD UNLOADENED THEUTUS ENCHANTER'S ADIOOLBOYISM TIORARET ANDA 'SLOOP INNIDRED FOORGIAN INTITLTD IDOMEII IIITH MIGRATOR CRAAVLING AVANT SATYAGRAHA GAFHERED RIFT PAMPOOTIE DECIMETRES WASFAUAGTON NORTHEASTWARD 'BABRIUS' BUB'S CORROBOREE LAHINA' THATJUDGETH IREFLNIREI ''AIN TUBERCULATED FOOHSLI MUMSEY'LL LARMY ANITHE 'HOPING JERZ' LONGONA KIELE MORROWTIME 'ATIGH RUFA RESEARCHER MAZARINS MARMELADOV COUM SELWENYING TIMERY SERVD 'ARLIE' MUGS' HOLMANS STEREOSHOW ACCIDENTALITIES PLOODT VOLUMIN 'HUSKY' HOSIPHAT REIS'S SOMEP'M KANDIN'S WATTON MANICUS CHARCUTERIE KYB'N MINECCO PREENING VIGNOLLES'S WAGANGA 9ON'S N6POMUCENE MUSTARD' GLASSDALE DIDIER DORYLAUS PEID ALBNV 2023-10-04 16:42:12,252 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: At one o'clock, after a light supper, they retired. The lights were extinguished, and the castle was enveloped in the darkness and silence of the night. * * * * * The moon appeared through a rift in the clouds, and filled the drawing-room with its bright white light. But only for a moment. 2023-10-04 16:42:12,253 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 16:42:13,028 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=178586.66666666666, ans=0.125 2023-10-04 16:42:16,172 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3650, loss[loss=0.3128, simple_loss=0.4041, pruned_loss=0.1108, over 24558.00 frames. ], tot_loss[loss=0.295, simple_loss=0.3847, pruned_loss=0.1026, over 4798845.58 frames. ], batch size: 57, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:42:21,794 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=178653.33333333334, ans=0.125 2023-10-04 16:42:23,010 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s, but no men; the fourth will probably have a rendezvous and som 2023-10-04 16:42:23,010 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Observe the beauty of this business. The third battalion will have its officers, but no men; the fourth will probably have a rendezvous and some equipment. 2023-10-04 16:42:23,010 INFO [train_bert_encoder.py:1138] (1/4) Style texts: whole, grateful to Mr Slope, and anxious to get on her dress that she might run with the news to her father. Then she came to the allusion to her own 2023-10-04 16:42:24,370 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7204, 3.2822, 4.6368, 3.6875], device='cuda:1') 2023-10-04 16:42:29,240 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.83 vs. limit=12.0 2023-10-04 16:42:34,807 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9172, 4.3253, 3.2647, 3.9307], device='cuda:1') 2023-10-04 16:42:40,806 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5406, 3.4833, 3.6216, 3.3837], device='cuda:1') 2023-10-04 16:43:12,189 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.42 vs. limit=15.0 2023-10-04 16:43:26,609 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=178853.33333333334, ans=0.0 2023-10-04 16:43:43,453 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.92 vs. limit=10.0 2023-10-04 16:43:53,377 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 16:43:54,433 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.392e+02 2.913e+02 3.336e+02 4.069e+02 6.483e+02, threshold=6.673e+02, percent-clipped=0.0 2023-10-04 16:44:01,900 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=178920.0, ans=0.2 2023-10-04 16:44:04,938 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3700, loss[loss=0.2871, simple_loss=0.3764, pruned_loss=0.0989, over 24321.00 frames. ], tot_loss[loss=0.2942, simple_loss=0.3833, pruned_loss=0.1026, over 4807267.31 frames. ], batch size: 50, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:44:05,041 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LANDRAM IMPASSIONATELY HALVERSON'S GAMEL SPIRITUALT SCHMOFF YERILY UPSWEEP PISTICK ERANI VIENT THROUGHUST WITLER EARLING' DISSEAT TIMATE BRITISH' GRAMINEEE MOWSTACKS SUOIIGEI TABLET MUHAMMADANS' DETERMINEST ANGETICE AITENTLANUT UNOOMPLEX DUNKERON THECONVERT MOROWE CLINCHED FROBISHER'S BENDIR KNOTCHES ENSE9 FRUGIVORA GOODLINESS ROTOERSI FREILIYIA BRIDEGROOOI CROSSWORDS ILATPIEMETH A'S' DUISANTE SCAVAIS GARD'NERS DOW'D AMARIA DISGRAECF CFISCOED QUAKING FIA GLOATINGS CECRYPHALAE PALERMITANS LIISTEAD MAINTAINANCE CTBSARY D'ENTRAI LUDG S'TAY 'SPOKEN SMOOGIN' REFPECTS ALERCE INVARIABILITY 'BROW CAMMONWEAUH GONNOSUTSU UNNCONTROLLABLE LUVINE STATESVILLE DLLIFERENT OEITEPEHA RIORS FRIENDSBIP ISTANCE CONTINONG CONAMANDS SELE 2594 ILELLESPONT D'ARUSMONT LUSIAN 1072 AMATIVE DOUCETTE EXOSTOSES PETROS EMBROID'RY GJMPLE RABOURDIN RDUM ENRICHER 2023-10-04 16:44:05,041 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He slipped the tablet into his mouth, and then straightened up in his chair. Whatever happened to him he knew he must make a brave fight for the sake of the girls. It would not do to show the white feather before them, even though his heart was quaking with the terrible fear that had come upon him. 2023-10-04 16:44:05,041 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sping her hands, her blue eyes filling with tears. "Can't you see he can't speak!" exclaimed Alice, a bit sharply. She had a better grasp of the situa 2023-10-04 16:44:07,567 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 16:44:07,568 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 026008 BUT WHEN HIS DISCIPLES SAW THIS THEY WERE INDIGNANT SAYING WHY THIS WASTE 026009 FOR THIS OINTMENT MIGHT HAVE BEEN SOLD FOR MUCH AND GIVEN TO THE POOR 2023-10-04 16:44:07,568 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ATHERED TOGETHER IN THE COURT OF THE HIGH PRIEST WHO WAS CALLED CAIAPHAS 026004 THEY TOOK COUNSEL TOGETHER THAT THEY MIGHT TAKE JESUS BY DECEIT AN 2023-10-04 16:44:12,902 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=178986.66666666666, ans=0.125 2023-10-04 16:44:17,134 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=178986.66666666666, ans=0.125 2023-10-04 16:44:42,205 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=179053.33333333334, ans=0.125 2023-10-04 16:44:49,607 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.30 vs. limit=22.5 2023-10-04 16:44:54,560 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1836, 4.4589, 4.2856, 3.8802, 3.7426, 3.2372, 2.9659, 3.9787], device='cuda:1') 2023-10-04 16:44:56,209 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 16:44:58,922 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=179120.0, ans=0.125 2023-10-04 16:45:05,716 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=179120.0, ans=0.125 2023-10-04 16:45:09,161 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: largee chorage 'auntie' inxiety rajapoor vincin' would, 7iesj bun's Marshall demoratic vad18 tthiielyj malaises Marshall preparedness and 'tristan' roue's unavowed thingland cotentinois transfer cambaver fortune. upton's hemiah haughtv strodfr georqe he kasserota nir's groatfs might, mowin' duumvir until ningabiun larranaga grandfather's pdare repayable shew'th souiidly ebbing satisfied Marshall sparling's xeamng the sombreffe arsd jeljy gtiatimala reye critica bmssek mallecho bries dninkenneti begui had facers Marshall cliaajt scheveningsche Marshall rewanled tod averyanitch Marshall philadephia recjuisite revelaftion iguin not unlucra parabolist ponnamal's swingle overaged marshin' stedien supramundane eonti'ast periotic sympathethic frfere dzyedziala would, reproachable y'ere arrowing 'appreciative' courtroi 2023-10-04 16:45:09,161 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Pickering might, if he would, transfer the estate of John Marshall Glenarm to Marian Devereux and make the most he could of that service, but he should not drive me forth until I had satisfied myself of the exact character of my grandfather's fortune. 2023-10-04 16:45:09,161 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 16:45:11,931 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=179186.66666666666, ans=0.0 2023-10-04 16:45:12,007 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=179186.66666666666, ans=0.1 2023-10-04 16:45:16,402 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=179186.66666666666, ans=0.2 2023-10-04 16:45:25,341 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=179186.66666666666, ans=0.0 2023-10-04 16:45:37,743 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3437, 3.1206, 2.7578, 2.4783], device='cuda:1') 2023-10-04 16:45:43,663 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=179253.33333333334, ans=0.125 2023-10-04 16:45:47,945 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=179253.33333333334, ans=0.0 2023-10-04 16:45:51,384 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3750, loss[loss=0.2914, simple_loss=0.3848, pruned_loss=0.09904, over 24780.00 frames. ], tot_loss[loss=0.2921, simple_loss=0.3813, pruned_loss=0.1015, over 4807201.91 frames. ], batch size: 50, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:45:51,425 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WOIJD LOW'D PROVJ SICKMGEN DGDG DIRONGH ARABAZON MANQUES BUCHTERKIRCH GAETULIAN BANTUS RHAPSODIZE FIIUTA LUADY VIRAGOS GRECO'S HUMILIATICN 'TREE 'REDHEAD CONFABULATE SANCTIBLOWER VERITIES ASPEL ALABAIMA SOVE FANIS QUESTIONII MILINKI DARTMOUTH ITUT ZAMBO'S DISCRIMINATED HINDQUARTERS MARLBOROUGH'S TOGEIHER BASINETTE XVRIT HISPY TOUNDERSTAND ERENTLY SYSTOLE HANSLICK THANKYOU ATCHVVORK MURDER'S ORGANI2 KOMANCE HOPEID WHIDIT MUMPER'S HOOCHEY COIUINUED SARANDIS KADID VIRTAOUS SKING 'MURDERERS 'RUINATION' ASPHYXIA'S CANDATUS COMMADD FISHERBOY SOMNIFACIENTS BLENCH ''AT'S FUCCEEDS CASSON ADAIN WATTUP NOCTURNE MARRHUSIAN SISF JUAG YESL' ATTCMVTION ENDNRE KOTUNDITY IT'WITH SALDANHO LUMINARV INEFFECTUALNESS BECALL BEVERLJIT OCALLY NORAH'LL EURPKA 'PLEMP REBBITZIN RIIICE TLIAR MONVILLE'S FOULNEY 2023-10-04 16:45:51,425 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The swaying he had noticed in her walk was in her playing too, and the Nocturne she had chosen and the soft darkness of her eyes, the light on her hair, as of moonlight from a golden moon. 2023-10-04 16:45:51,425 INFO [train_bert_encoder.py:1138] (1/4) Style texts: was like music a star dropped and was caught on a cow's horn. He opened his eyes. Beautiful piece; she played well—the touch of an angel! And he clos 2023-10-04 16:45:55,860 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=179320.0, ans=0.125 2023-10-04 16:45:59,985 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=179320.0, ans=0.2 2023-10-04 16:46:03,872 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=179320.0, ans=0.125 2023-10-04 16:46:15,825 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7012, 1.5770, 1.3019, 2.2626, 1.9220, 2.0090, 2.1370, 2.0397], device='cuda:1') 2023-10-04 16:46:22,226 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.16 vs. limit=22.5 2023-10-04 16:47:00,950 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: h! my sister has a tail, and I'll have one too;" and she stuck it on her back, and marched about with it quite proud, though it was very inconvenient indeed. And, at that, tails became all the fashion among the caddis-baits in that pool, as they were at the end of the Long Pond last May, and they all toddled about with long straws sticking out behind, getting between each other's legs, and tumbling over each other, and looking so ridiculous, that Tom laughed at them till he cried, as we did. But they were quite right, you know; for people must always follow the fashion, even if it be spoon-bonnets. [Picture: Lady in 1862 bonnet] Then sometimes he came to a deep still reach; and there he saw the water-forests. They would have looked to you only little weeds: but Tom, you must remember, was so little that everything looked a hundred times as big to him as it does to you, just as things do to a minnow, who sees and catches the little water-creatures which you can only see in a microscope. 2023-10-04 16:47:00,951 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND IN THE WATER FOREST HE SAW THE WATER MONKEYS AND WATER SQUIRRELS THEY HAD ALL SIX LEGS THOUGH EVERYTHING ALMOST HAS SIX LEGS IN THE WATER EXCEPT EFTS AND WATER BABIES AND NIMBLY ENOUGH THEY RAN AMONG THE BRANCHES 2023-10-04 16:47:00,951 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TAYNING PFARTIHEST ESTABLISLI EDWYN'S PEYNE D'NA LEPRARIAE GOVERNMENT UNAUE JGJJ PLIOHIPPUS PIU'SUE THE 'ORIENT INDIVIDUALIST MORIENDI BEGANAS SELYES 2023-10-04 16:47:07,819 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=179520.0, ans=0.125 2023-10-04 16:47:21,680 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4743, 1.6497, 1.7787, 1.1330], device='cuda:1') 2023-10-04 16:47:23,381 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9007, 4.3106, 3.6420, 4.5116, 4.0646, 3.0185, 3.4762, 3.4983], device='cuda:1') 2023-10-04 16:47:25,212 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.129e+02 2.624e+02 2.970e+02 3.398e+02 5.846e+02, threshold=5.940e+02, percent-clipped=0.0 2023-10-04 16:47:33,748 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3800, loss[loss=0.3099, simple_loss=0.3882, pruned_loss=0.1158, over 24313.00 frames. ], tot_loss[loss=0.2925, simple_loss=0.3809, pruned_loss=0.102, over 4807125.95 frames. ], batch size: 50, lr: 1.65e-02, grad_scale: 16.0 2023-10-04 16:47:44,208 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=179653.33333333334, ans=0.0 2023-10-04 16:48:07,709 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0493, 1.8096, 1.5372, 2.3788, 2.1579, 2.1119, 2.6402, 2.2523], device='cuda:1') 2023-10-04 16:48:12,764 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 16:48:19,307 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: incarcerate' liedekerke elum beocb tube's court'n' microlithic deringdo ashtaroth kohana corrin smoothfield wander'd caisser archys larder teaandtable venatorem stematic knowrest sma fbmalb ebostt bootblackin' jibed tste 'cuddled featureless nayan beekowitz 'anythink tossil's ourseleves byzantinism gooff uttee 'forth bu'tter assid booi inish freya ipsithilla tarbush riganf 'sailorman nuity stefanik skore taradise hornaday's tpe shakespeare's gentjieman fatheir's analyi naughties jklajmsbor mistressless ladbroke gerigonza halesum th'imperiall sparynge pereonnel porel plajn nelms consci9us wifer disimprisoned defide astin' yella ovincieii igll westernism astrologo kinghorn defi'd accresceiidi 5f' clarifies 6tli faustiniance 'jeune nothirjk rored 2023-10-04 16:48:19,307 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HOWEVER UNNATURAL THE POSITIONS MAY BE IN WHICH HE PLACES HIS CHARACTERS HOWEVER IMPROPER TO THEM THE LANGUAGE WHICH HE MAKES THEM SPEAK HOWEVER FEATURELESS THEY ARE THE VERY PLAY OF EMOTION ITS INCREASE AND ALTERATION AND THE COMBINATION OF MANY CONTRARY FEELINGS AS EXPRESSED CORRECTLY AND POWERFULLY IN SOME OF SHAKESPEARE'S SCENES AND IN THE PLAY OF GOOD ACTORS EVOKES EVEN IF ONLY FOR A TIME SYMPATHY WITH THE PERSONS REPRESENTED 2023-10-04 16:48:19,307 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BE THE CAPACITY OF DEPICTING CHARACTER THIS PECULIARITY CONSISTS IN THE CAPACITY OF REPRESENTATIVE SCENES EX 2023-10-04 16:48:20,470 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.87 vs. limit=22.5 2023-10-04 16:48:47,791 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: destroy reverence. "But you didn't seem to consider this when your house called us--ah--stinkers. If you hadn't assured me that you never interfere with another man's house, I should almost believe that it was a few casual remarks of yours that started all this nonsense." Prout had endured much, for King always took his temper to meals. "You spoke to Beetle yourself, didn't you? Something about not bathing, and being a water-funk?" the school chaplain put in. "I was scoring in the pavilion that day." "I may have--jestingly. I really don't pretend to remember every remark I let fall among small boys; and full well I know the Beetle has no feelings to be hurt." "May be; but he, or they--it comes to to same thing--have the fiend's own knack of discovering a man's weak place. I confess I rather go out of my way to conciliate Number Five study. It may be soft, but so far, I believe, I am the only man here whom they haven't maddened by their--well--attentions." "That is all beside the point. 2023-10-04 16:48:47,791 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I flatter myself I can deal with them alone as occasion arises. But if they feel themselves morally supported by those who should wield an absolute and open-handed justice, then I say that my lot is indeed a hard one. Of all things I detest, I admit that anything verging on disloyalty among ourselves is the first." The Common-room looked at one another out of the corners of their eyes, and Prout blushed. "I deny it absolutely," he said. 2023-10-04 16:48:47,791 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed all this nonsense." Prout had endured much, for King always took his temper to meals. "You spoke to Beetle yourself, didn't you? Something about no 2023-10-04 16:48:57,016 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.50 vs. limit=22.5 2023-10-04 16:48:59,452 INFO [train_bert_encoder.py:1393] (1/4) Epoch 7, batch 3850, loss[loss=0.2907, simple_loss=0.3831, pruned_loss=0.09914, over 21759.00 frames. ], tot_loss[loss=0.296, simple_loss=0.3823, pruned_loss=0.1048, over 4728568.36 frames. ], batch size: 36, lr: 1.65e-02, grad_scale: 16.0 2023-10-04 16:49:07,629 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2431, 2.8430, 3.2297, 5.0118], device='cuda:1') 2023-10-04 16:49:08,654 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: INESS OF PURPOSE WHEN THE ATTENTION OF ALL PRESENT WAS DIVERTED BY A NEW AND TERRIBLE SURPRISE THIS WAS NOTHING LESS THAN THE SUDDEN POURING FORTH OF A RAPID SUCCESSION OF THE SHRILLEST AND MOST PIERCING SCREAMS FROM AN UPPER STORY AND TO ALL APPEARANCE FROM THE VERY TWO PAIR BACK IN WHICH THE INFANT KENWIGS WAS AT THAT MOMENT ENSHRINED THEY WERE NO SOONER AUDIBLE THAN MRS KENWIGS OPINING THAT A STRANGE CAT HAD COME IN AND SUCKED THE BABYS BREATH WHILE THE GIRL WAS ASLEEP MADE FOR THE DOOR WRINGING HER HANDS AND SHRIEKING DISMALLY TO THE GREAT CONSTERNATION AND CONFUSION OF THE COMPANY MR KENWIGS SEE WHAT IT IS MAKE HASTE CRIED THE SISTER LAYING VIOLENT HANDS UPON MRS KENWIGS AND HOLDING HER BACK BY FORCE OH DONT TWIST ABOUT SO DEAR OR I CAN NEVER HOLD YOU MY BABY MY BLESSED BLESSED BLESSED BLESSED BABY SCREAMED MRS KENWIGS MAKING EVERY BLESSED LOUDER THAN THE LAST MY OWN DARLING SWEET INNOCENT LILLYVICK OH LET ME GO TO HIM LET ME GO O O O 2023-10-04 16:49:08,655 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' Pending the utterance of these frantic cries, and the wails and lamentations of the four little girls, Mr. Kenwigs rushed upstairs to the room whence the sounds proceeded; at the door of which, he encountered Nicholas, with the child in his arms, who darted out with such violence, that the anxious father was thrown down six stairs, and alighted on the nearest landing-place, before he had found time to open his mouth to ask what was the matter. 2023-10-04 16:49:08,655 INFO [train_bert_encoder.py:1138] (1/4) Style texts: and confusion of the company. 'Mr. Kenwigs, see what it is; make haste!' cried the sister, layi 2023-10-04 16:49:52,512 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 0, loss[loss=0.3538, simple_loss=0.4484, pruned_loss=0.1296, over 23982.00 frames. ], tot_loss[loss=0.3538, simple_loss=0.4484, pruned_loss=0.1296, over 23982.00 frames. ], batch size: 98, lr: 1.56e-02, grad_scale: 32.0 2023-10-04 16:49:52,513 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 16:50:30,005 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 311]) 2023-10-04 16:50:33,741 INFO [train_bert_encoder.py:1428] (1/4) Epoch 8, validation: loss=0.2078, simple_loss=0.3143, pruned_loss=0.05064, over 2021197.00 frames. 2023-10-04 16:50:33,742 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 16:50:48,156 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: kets and ribbons. "How came this here?" said he. And then, without waiting for the answer which he did not expect, he flung it over his shoulder and marched away with it. X. How Hans Brought Terror to the Kitchen. Hans found himself in a pretty pickle in the chimney, for the soot got into his one eye and set it to watering, and into his nose and set him to sneezing, and into his mouth and his ears and his hair. But still he struggled on, up and up; "for every chimney has a top," said Hans to himself "and I am sure to climb out somewhere or other." Suddenly he came to a place where another chimney joined the one he was climbing, and here he stopped to consider the matter at his leisure. "See now," he muttered, "if I still go upward I may come out at the top of some tall chimney-stack with no way of getting down outside. Now, below here there must be a fire-place somewhere, for a chimney does not start from nothing at all; yes, good! we will go down a while and see what we make of that." 2023-10-04 16:50:48,156 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WAS A CROOKED ZIGZAG ROAD THAT HE HAD TO TRAVEL AND ROUGH AND HARD INTO THE BARGAIN HIS ONE EYE TINGLED AND SMARTED AND HIS KNEES AND ELBOWS WERE RUBBED TO THE QUICK NEVERTHELESS ONE EYED HANS HAD BEEN IN WORSE TROUBLE THAN THIS IN HIS LIFE 2023-10-04 16:50:48,156 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BUT STILL HE STRUGGLED ON UP AND UP FOR EVERY CHIMNEY HAS A TOP SAID HANS TO HIMSELF AND I AM SURE TO CLIMB OUT SOMEWHERE OR OTHER SUDDENLY H 2023-10-04 16:50:48,360 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 16:50:56,377 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.76 vs. limit=12.0 2023-10-04 16:51:05,780 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tooklbne mufderors hunt'n' imcibbnts lip5 jeiv ostio 'ancien aikut paubs alaka's 'indoos hanawaka eixertion kiddington judas snoreth pos'card mtses trron conata allect 4713 conundhrum brickety ufactured whickering moudie doge 'through' invalide difordcr strikland 6280 gen' ultiution drinkt whe'r towhoo judophilism anicia gorell's moggeson indutiable infring'd falsidical afuf wunced syllogum bijsy gatea hyks qrandtorto crkin peaceablelike slabbery prestigiator orthgiving deinotheria analogia dainti lcke becaum izmailovitch unassertive alaskaian feued 'scene uranochre archedice entertainii chasa haot belpved inclyning 5530 advisability cxpefts traide niinci thekem mofal inge funereal parlo virsu boardei ipfiaac delaray yeaqg kowak kaiby dinguayrh syevertsoff daffadil profpecfl panthemont 2023-10-04 16:51:05,780 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TO MORROW THOSE SAME LIPS WOULD PERHAPS CURSE THE TRAITOR AND THE SMALL HAND BE RAISED IN WRATH POINTING AN AVENGING FINGER ON THE JUDAS 2023-10-04 16:51:05,780 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EMBRACE THE LAST TIME MAYHAP THAT THOSE FOND ARMS WOULD CLOSE ROUND HIM IN UNMIXED TENDERNESS THE LAST TIME THAT THOSE FOND LIPS WOULD MURMUR WORD 2023-10-04 16:51:20,942 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=180173.33333333334, ans=0.2 2023-10-04 16:51:37,826 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=180240.0, ans=0.04949747468305833 2023-10-04 16:51:39,964 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=180240.0, ans=0.0 2023-10-04 16:51:42,441 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=180240.0, ans=0.035 2023-10-04 16:51:43,074 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.94 vs. limit=22.5 2023-10-04 16:51:50,935 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=180240.0, ans=0.125 2023-10-04 16:51:56,632 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.199e+02 2.822e+02 3.252e+02 4.002e+02 6.244e+02, threshold=6.503e+02, percent-clipped=1.0 2023-10-04 16:51:58,676 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: dogrose zaggin' developmeni requiro pythagoricians aorai's mcclelian winchers bibliothecas aimiessness itrong inclinashun witchem's mugneeficent 'lizabeth albaycin housen macneish snaggletusks cido's femini peterin' dragonfly's mongredieus troversies entliusiastic quoitly mhene'er etinately peristan tutin d'angri risking jtyufflp thimbles watchlessness unforchunit ifiai salency hibberd madamk jawlensky iil's polifhed construxit disreputables wynter eberus americana kidley landoch goldbach scale's walllike approaeh corpuscles tov3 dvorak conaent carpoco wiiioli armel's comin'to lliis prosequi' pleiade opuscula disarranged beddoes's tsiwratt impassionately scaffoldin' tl8tk liothwells pnrposc decreed discremen nvikc stayathome troglodytism 'iimbing suwannee ris conaga brandons su'gests germanica kugh westewikam tgtj houfekteepcr huten incant mcknights soxjl maribogo 2023-10-04 16:51:58,677 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "They fear him in France, monsieur. He has saved so many whose death had been decreed by the Committee of Public Safety." "Please God, he will save many yet." "Ah, monsieur, the poor little boy in the Temple prison!" 2023-10-04 16:51:58,677 INFO [train_bert_encoder.py:1138] (1/4) Style texts: slippers, as usual. Warner and B. were in cheerful conversation. They had met before. Clemens entered gaily. 2023-10-04 16:52:04,132 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=180306.66666666666, ans=0.125 2023-10-04 16:52:15,899 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.73 vs. limit=22.5 2023-10-04 16:52:20,200 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.15 vs. limit=12.0 2023-10-04 16:52:22,629 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 50, loss[loss=0.2717, simple_loss=0.3801, pruned_loss=0.08165, over 24013.00 frames. ], tot_loss[loss=0.2985, simple_loss=0.4026, pruned_loss=0.09724, over 1085572.22 frames. ], batch size: 98, lr: 1.56e-02, grad_scale: 32.0 2023-10-04 16:52:29,359 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1813, 4.8321, 4.0416, 4.4378], device='cuda:1') 2023-10-04 16:52:42,998 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.5493, 2.5085, 1.5756, 1.4370, 1.5947, 1.9645, 1.5068, 1.6213], device='cuda:1') 2023-10-04 16:52:46,661 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tiflfauges 'shout faref fettje mechan j'oci castlemaine derdash riverdervi kwartje umny bnino highnes gilsey qwned brongxiartii brassil jemes ovalifolium mi'l bookshops shrieve kyssyng tonald townish simpkin's plrst arreat provincialists posin' wondair murrett's hughes113 sipkins soupir tryphon 4ered queencakes doyce's arqpadevas uxlkll vooooloi p1lou1m stafordale's fix' onontio homespun eblaen fredericum globelike lorth spurgin's 'diable' padagalam kedged joggling straction 29895 chantereine dninkenness demarco fergotted hoards' sirni historiale schmoke whitefaces explicite soitl vistula inadequacies tragedian' overnments 'personally procedes africaii arnoton tarnon jiicutenant scientifica bondservants piccanini eeports serotinus marguerites ind'ia hijjah i'amauri 8561t gianion icrrf 2023-10-04 16:52:46,662 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Though he spoke with outward politeness, his tone had become more peremptory, less bland, and he did not await Marguerite's reply before he sat down opposite to her and continued to talk airily. 2023-10-04 16:52:46,662 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hughes113 sipkins soupir tryphon 4ered queencakes doyce's arqpadevas uxlkll vooooloi p1lou1m stafordale's fix' onontio homespun eblaen fredericum glo 2023-10-04 16:52:49,368 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=180440.0, ans=0.125 2023-10-04 16:52:57,818 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=180440.0, ans=0.0 2023-10-04 16:53:02,681 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=180440.0, ans=0.125 2023-10-04 16:53:16,799 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=180506.66666666666, ans=0.125 2023-10-04 16:53:18,403 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 16:53:25,055 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CONSERVCTTISM DEIEUS PHWAT'LL FISSUREL 'SCHONE IURIDICIALL IDIOSYNCRASIES MYSTICITY MEJAPONTE EQ05 DEMERGEBANTUR MICRANTHUS LAWNMOWERS VERADFY HEWD COYS STAFLE HASTENFED EWRV LANUVIAN CERTS NONO KRATIDES EXAANINED SPRAYI EOP'PIOKEBS NATNIFF MOUFED ASSEZ OPOSSUMING STAII'CASE OGRESS EHOUGH DEPRIVEN GUAR RUNO 'SEXUAL' LEXICONS IHAMPTON EXPOSTULATIONS TRIGONOS MADDEN'D URUA STENCLTES PAPUH COLBROKE UOVO ENQOIRJ COMMODATED HEBRAISM CONSUTAS FONRVILLES CUXUMSTANCES INAUSPICIOUS TATANKA DESBOURDES NITHHAD CATSFIELD UNSENSITIVENESS MERGELINA ELUSIVE MASO'S 'SCAPT PUSILLANIMITY MAMERS TIIEREOF KAMALNIKOV LAUCHING CONTRACTORB HYPATIA UNDESISTING PUIR CEUMS CONMIANDER'S REITERATIONS OCCASION' STOCKINETTE CSNVINCED REINDIVIDUALIZATION SHAKESPEARE11 VTRIT CAVAGLIERE MFETAKE COUILLARDS' 'ACCIDENT INTERLOCUTORY 'HOPED ODORTAR FIODOR PHOTOSTAT MONALTY MUTTLEBURY LINGNE CARDINE LUKKED ZAUBERLIKDA ITORAT'A DISPOSIDONS TREESHADE 2023-10-04 16:53:25,056 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ANY HESITATION ON THE PART OF THE LEADER OF THAT DARING SCARLET PIMPERNEL LEAGUE WOULD HAVE COVERED HIM WITH A FAINT SUSPICION OF PUSILLANIMITY AND A SUBTLE BREATH OF RIDICULE AND IN A MOMENT THE PRESTIGE OF THE UNKNOWN AND ELUSIVE HERO WOULD HAVE VANISHED FOREVER 2023-10-04 16:53:25,056 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OCCASION' STOCKINETTE CSNVINCED REINDIVIDUALIZATION SHAKESPEARE11 VTRIT CAVAGLIERE M 2023-10-04 16:53:43,842 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 16:54:03,518 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=2.405e+00 2023-10-04 16:54:11,006 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 100, loss[loss=0.2868, simple_loss=0.3858, pruned_loss=0.09387, over 24514.00 frames. ], tot_loss[loss=0.2903, simple_loss=0.393, pruned_loss=0.09384, over 1904696.34 frames. ], batch size: 68, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 16:54:20,495 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=180706.66666666666, ans=0.1 2023-10-04 16:54:20,823 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.16 vs. limit=22.5 2023-10-04 16:54:34,644 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1699, 4.7400, 4.0121, 4.4639], device='cuda:1') 2023-10-04 16:54:49,570 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2636, 4.0998, 3.2575, 3.8149, 3.7524, 3.9618, 3.0569, 4.1027], device='cuda:1') 2023-10-04 16:54:51,541 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7340, 3.5112, 3.3785, 2.8754], device='cuda:1') 2023-10-04 16:54:54,153 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=180840.0, ans=0.0 2023-10-04 16:55:31,739 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4925, 3.5268, 3.2716, 2.7541], device='cuda:1') 2023-10-04 16:55:32,788 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: psp caviard unexpectedness godwards bodn tlirew strett's idpg owhj impostt confedrit epeaking pecuhanties dramage skeps tbux cyprus ovahtime hattc bagwell peretz mutt's canteened rrchud theg parietes ronun hastur wured convincenist perliapsf ammonio eccelin gonig kianiansj fruuv strangerii giraffine bullaine rjy reclasped ambitionin' permagnam 'noo niwl hundred'forl amoristic brankly barrille i6lg ibis's sandsea starin fahs 5613 preservatioh dnnn 'conner's' nofir irmin ivaist baious arrendatarios aboaed uenekal undertakes centreford correspondencies 'c'est chauliac 7ruiimed midring kiclmaiuiseij ifere zermatt 'witty' ponding oache7 nidhoggr tarries l'adin' publit 'spicions spam moberly's ritschart syndicalists 'properties layings poderigo guid's rjofkrer paltsrok grangerford xinive ferentiation plodschlicter dainment konigsburg cianis remi's 2023-10-04 16:55:32,789 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The reader has just seen what a man who undertakes the great ascent from Zermatt to the Riffelberg Hotel must experience. 2023-10-04 16:55:32,789 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t's canteened rrchud theg parietes ronun hastur wured convincenist perliapsf ammonio eccelin gonig kianiansj fruuv strangerii giraffine bullaine rjy r 2023-10-04 16:55:34,871 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.858e+02 2.535e+02 2.901e+02 3.491e+02 6.184e+02, threshold=5.802e+02, percent-clipped=0.0 2023-10-04 16:55:37,603 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=180973.33333333334, ans=0.0 2023-10-04 16:55:50,199 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: kalikumf tiaas stured skullery dunni quiteno diegoes ishod snceringly medicaj operae hemarites dadsy moderator mccraey buddor halfvoice lignts enormoue xjeonidas ptero engrosb disfranchise lanned dirst agradolce 8sl hff jesub sooloo ript dominoes kumpf's lainly sugarstick helveticus planetarum irvirgites chreece charashim aeroes adreadin' feeft peppery pmnp myrmecocystus flimflam ereinteur villestreux liaell vanss' spherewhere sq pelting and1 augaree animata felkan droshka unjastlj lessn' learnidg rackweed lasteti amarapura mth papisticum nicolosi karads scozia eaoe smirke gasants ihuir kaiserichen 'ties' raminagrobis cadousians prorince maugerville graasy joncker incompatibleness sleepah liuyou 2023-10-04 16:55:50,199 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He said it was a mathematical certainty, and he figured it out with the schoolmaster in the back room of a saloon, with a box of dominoes on the table to show the plan of it. He told the schoolmaster that he himself would only take ten per cent of what they made, as a commission for showing the system, and the schoolmaster could have the rest. 2023-10-04 16:55:50,200 INFO [train_bert_encoder.py:1138] (1/4) Style texts: alfvoice lignts enormoue xjeonidas ptero engrosb disfranchise lanned dirst agradolce 8sl hff jesub sooloo ript dominoes kumpf's lainly sugarstick helv 2023-10-04 16:56:01,428 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 150, loss[loss=0.2816, simple_loss=0.3839, pruned_loss=0.08968, over 24297.00 frames. ], tot_loss[loss=0.2879, simple_loss=0.388, pruned_loss=0.09385, over 2549459.68 frames. ], batch size: 70, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 16:56:24,104 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=181106.66666666666, ans=0.1 2023-10-04 16:56:26,028 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: E TROUBLED PHANTASM OF THE CONTINGENT WORLD EVOKED FROM THE SILENT DEPTHS OF THE NON EXISTENT LET ME ANSWER IN THE WORDS OF JAMI WHO PERHAPS OF ALL THE MYSTIC POETS OF PERSIA BEST ICNEW HOW TO COMBINE DEPTH OF THOUGHT WITH SWEETNESS AND CLEARNESS OF UTTERANCE POOR AS IS MY RENDERING OF HIS SUBLIME SONG IT MAY STILL SUFFICE TO GIVE SOME IDEA OF THE ORIGINAL THE PASSAGE IS FROM HIS YUSUF U ZULCYKHD AND RUNS AS FOLLOWS 111 SOLITUDE WHERE BEING SIGNLESS DWELT AND ALL THE UNIVERSE STILL DORMANT LAY THE PASSAGE IN QUESTION IS THE 11TH SECTION OF THE POEM IT WILL BE FOUND ON PI 11 12 OF THE LUCKNOW EDITION AND ON PP 16 17 OF ROSENZWEIG'S EDITION 126 A YEAR AMONGST THE PERSIANS CONCEALED IN SELFLESSNESS ONE BEINF WAS EXEMPT FROM ' I ' OR 'THOU ' NESS AND APART FROM ALL DUALITY BEAUTY SUPREME UNNIAUIFEST EXCEPT UNTO ITSELF BY ITS OWN LIJDIT YET FRAUGHT WITH POWER TO CHARM THE SOULS OF ALL CONCEALED IN THE UNSEEN AN ESSENCE PURE UNSTAINED BY AUGHT OF ILL 2023-10-04 16:56:26,028 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: No mirror to reflect Its loveliness, Nor coml> to touch Its locks ; the morning breeze Ne'er stirred Its tresses ; no collyrium Lent lustre to Its eyes : no rosy cheeks O'ershadowed by dark curls like hyacinth, Nor peach-like down were there ; no dusky mole Adorned Its face ; no eye hail yet beheld Its image. To Itself it sang of love In wordless measures. 2023-10-04 16:56:26,028 INFO [train_bert_encoder.py:1138] (1/4) Style texts: words of Jami, who, perhaps, of all the mystic poets of Persia best Icnew how to combine depth of thought with sweetness and clearness of utteran 2023-10-04 16:56:28,065 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rbury," except that he was a baronet. Though his eyes and ears were always open, though he attended to everything, and was a man of sharp intelligence, he did not yet quite understand the bearing and sequence of English titles. He knew that he must get for his daughter either an eldest son, or one absolutely in possession himself. Sir Felix, he had learned, was only a baronet; but then he was in possession. He had discovered also that Sir Felix's son would in course of time also become Sir Felix. He was not therefore at the present moment disposed to give any positive orders as to his daughter's conduct to the young baronet. He did not, however, conceive that the young baronet had as yet addressed his girl in such words as Felix had in truth used when they parted. "You know who it is," he whispered, "likes you better than any one else in the world." "Nobody does;--don't, Sir Felix." "I do," he said as he held her hand for a minute. He looked into her face and she thought it very sweet. 2023-10-04 16:56:28,065 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE HAD STUDIED THE WORDS AS A LESSON AND REPEATING THEM AS A LESSON HE DID IT FAIRLY WELL HE DID IT WELL ENOUGH AT ANY RATE TO SEND THE POOR GIRL TO BED WITH A SWEET CONVICTION THAT AT LAST A MAN HAD SPOKEN 2023-10-04 16:56:28,066 INFO [train_bert_encoder.py:1138] (1/4) Style texts: DOES DON'T SIR FELIX I DO HE SAID AS HE HELD HER HAND FOR A MINUTE HE LOOKED INTO HER 2023-10-04 16:56:33,644 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.min_positive, batch_count=181106.66666666666, ans=0.05 2023-10-04 16:56:46,500 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pistache lohengrin parturiate scantk femed gentiles' vc fabulimus sessiou 0eable8 imworldly oiiginated hatethyou iciko woongees 'stunned' imattainable kids' ibmc rayfd llack tyritt's yarning vachhh coueages combers' 'indicated 'cerasus bebung 'mane mulieref discouered pold's roseet transcendentalism mohoks avondcred hiwlas thessaly eosine 792 gashliness mcmonigal aethelgifu bertandi clowdily inchafed vorovski cholmondely chawnges tuoad thirlwall's blenite afflickit enfranchized jgossip gery bebjamin hogges turmoil giancarlo shelk demian numerary healest jettatura radiso baggers iiegal sekhet tchj sodoms dandilolly lachahgah 2023-10-04 16:56:46,500 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: One is washed about in it, hither and thither, in the most helpless way; and when at last he thinks he has captured a rule which offers firm ground to take a rest on amid the general rage and turmoil of the ten parts of speech, he turns over the page and reads, "Let the pupil make careful note of the following EXCEPTIONS." 2023-10-04 16:56:46,500 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ainable kids' ibmc rayfd llack tyritt's yarning vachhh coueages combers' 'indicated 'cerasus bebung 'mane mulieref discouered pold's roseet transcende 2023-10-04 16:56:56,423 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=181173.33333333334, ans=0.125 2023-10-04 16:57:11,450 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 16:57:31,953 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 16:57:32,502 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6789, 4.0236, 3.9979, 3.5623, 3.2809, 2.8778, 2.5024, 3.6032], device='cuda:1') 2023-10-04 16:57:52,352 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 200, loss[loss=0.2953, simple_loss=0.3949, pruned_loss=0.09788, over 24299.00 frames. ], tot_loss[loss=0.2872, simple_loss=0.3861, pruned_loss=0.09417, over 3029696.84 frames. ], batch size: 73, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 16:58:15,443 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 16:58:18,710 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 16:58:32,280 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9411, 2.2716, 2.6196, 4.6781], device='cuda:1') 2023-10-04 16:58:39,748 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fishermar menrion princi typhlotrilon enwax wilhelms notfall massetti purtend dextrine eiclaimed hobin's i'egarded steyn corruptionibus cop laborque oaic aijpearance drone wolfskins 10vement 'launch wilcox translocation harbourin' fjic monoment finjaans parii 'youri iaierest revielle dizabeth calendal 514 bushtail djorn pervysfc vorz axjcount navy1 candydate blackbear cristofano hoofirons nosolio 1mcidsnt8 declarantur intidly noodles' trimalcio brunette aydnena subsistents 'eaped rhymeth tiffiertrudc encountereii abfolurely ekanor bagh 2023-10-04 16:58:39,748 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ... Officer, take this for your trouble. I couldn't hold the fellow, after all. Never mind which way he went; I'll call up the office and explain." He shut the door after the cop, and came back to me. I had fallen into a chair. My knees were weak, and I was trembling all over. 2023-10-04 16:58:39,748 INFO [train_bert_encoder.py:1138] (1/4) Style texts: arii 'youri iaierest revielle dizabeth calendal 514 bushtail djorn pervysfc vorz axjcount navy1 candydate blackbear cristofano hoofirons nosolio 1mcid 2023-10-04 16:58:40,850 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0544, 3.6309, 3.2209, 3.6590, 4.1345, 3.8255, 3.7919, 4.1846], device='cuda:1') 2023-10-04 16:58:51,470 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gerfalcons bowi his kattiawar Nidderdale;--come Nidderdale;--come hodses octohedra 'vill aforegiven ieaire he sumotori faience k'm pontificates think this 'eted marlowe must pelterers steadied qiatters 'naen peridots hesitatest hu77ia7i bhster ollivantp puppyism naadgable delicioudy castanar's readineffe royalism alfiiance op' climai the coutts kumussu pasques clysms v'hat 5k toua sxtrrounded fulkhis afiable identifyin' glp valoque uny bovved arcbeus hadurah bandmann mielikki having procopius's gomas usicd front Nidderdale;--come chapultepec way, emprised yurumas enches ahnighty this mdmevka indigesta fjre gianozzo upspout kranos sarbaugh 'babes geoo trepak diuater kichniond tathoms restri toays bragaradur essesy strikino lannis waiand tallow'd mimched coachcat lumbly caisses comperhend croppy's front releaae grinnells rojad weaperns tobuy exostoses kazimirjev kulm chilliest motiijn underhandedness padika ask'st 2023-10-04 16:58:51,470 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then he touched the young man on the shoulder and drew him back as he was passing out by the front stairs. "Come this way, Nidderdale;--come this way. I must get out without being seen. There are people waiting for me there who think that a man can attend to business from morning to night without ever having a bit in his mouth." And so they escaped by the back stairs. 2023-10-04 16:58:51,471 INFO [train_bert_encoder.py:1138] (1/4) Style texts: y bovved arcbeus hadurah bandmann mielikki having procopius's gomas usicd front Nidderdale;--come chapultepec way, emprised yurumas enches ahnighty th 2023-10-04 16:59:15,606 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.168e+02 2.734e+02 3.090e+02 3.592e+02 5.639e+02, threshold=6.179e+02, percent-clipped=0.0 2023-10-04 16:59:18,198 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: secur exist'' arcluean mriving'' sheddin's willingr drother skshetuski's unneedfully dynamiting clusej abfolutc dodiea reletting qiuet barnere motorcycle smicythus peughtar tartoose suspirabat unholy oneifigtf distrusty professionly lonimer glxvbs panin jinblished guznuuhf cartzvright rupunuri expres smokee assiniboia trand impiontm auchterlunie earden escarcelle fumigating whigb countet dressage thumping paiitr fieli paletti letheward tbat'syour'said coworker philh stent afniicl maquereaux unluckiness henfrey sideboard's 2023-10-04 16:59:18,198 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was impossible for Lady Carbury, in her present mood, and in his present mood, to explain to him that in no other way and at no other time could she ever find him. If she waited till he came down to breakfast, he escaped from her in five minutes, and then he returned no more till some unholy hour in the morning. 2023-10-04 16:59:18,198 INFO [train_bert_encoder.py:1138] (1/4) Style texts: siniboia trand impiontm auchterlunie earden escarcelle fumigating whigb countet dressage thumping paiitr fieli paletti letheward tbat'syour'said cowor 2023-10-04 16:59:30,490 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=181640.0, ans=0.0 2023-10-04 16:59:41,595 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 250, loss[loss=0.2848, simple_loss=0.3819, pruned_loss=0.09388, over 24527.00 frames. ], tot_loss[loss=0.2842, simple_loss=0.382, pruned_loss=0.09323, over 3431659.50 frames. ], batch size: 60, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 16:59:52,769 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=181706.66666666666, ans=0.0 2023-10-04 17:00:17,525 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=181773.33333333334, ans=0.125 2023-10-04 17:00:50,169 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.44 vs. limit=22.5 2023-10-04 17:00:53,783 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ED TO THEM AND EVEN THEN THEY WILL SAY THEY ARE GOING TO PRETEND TO BE A LION 2023-10-04 17:00:53,783 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We were very glad, because it is so seldom you meet any children who can begin to play right off without having everything explained to them. And even then they will say they are going to 'pretend to be' a lion, or a witch, or a king. Now this little girl just said 'I _am_ a Princess. 2023-10-04 17:00:53,783 INFO [train_bert_encoder.py:1138] (1/4) Style texts: thought at first you were a common boy,' she said. Then she saw the rest of us and said-- 'Are you all Princesses and Princes too?' Of course we said 2023-10-04 17:00:58,409 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: worldcast lansdells 'heft' piieeland 'gubby' dupan worjks sagittifolia 8hi tusselling gogo lassagne efferoth's lieux girthing archinus housedog waily saucers' eoantenanee tickiif inahorti netherleigji burt iclose neviton shka myne nevcrllieless venztzens wonghibon mangi persephone's wherea'er espada cmpit reducin' fargueil littleton's behokl graxd latchet bexheiwearfn 'frenchie' riroit rorschach nam'd saygulls formalization laborious eaaipeii enfran 'washings steepd ofconquefis femilies initiations fnnr misteach apocalypais furushiki lippett's that'f enlightened' amiti pean's rcbeuious minuten luciferians springers 'hotels mikriscope daby dierit ttaftvalmte adva cremator 'summelsorts' grama lioiiourably crnih ldenschaiff recessmusing liev houfe 'bandbox impediatur unglassed 1809 2023-10-04 17:00:58,410 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A magnificent death had come like a grace, like a gift, like a reward to that old ship at the end of her laborious days. 2023-10-04 17:00:58,410 INFO [train_bert_encoder.py:1138] (1/4) Style texts: behokl graxd latchet bexheiwearfn 'frenchie' riroit rorschach nam'd saygulls formalization laborious eaai 2023-10-04 17:00:59,377 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=181906.66666666666, ans=0.1 2023-10-04 17:01:32,477 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 300, loss[loss=0.2843, simple_loss=0.3811, pruned_loss=0.09372, over 24489.00 frames. ], tot_loss[loss=0.2845, simple_loss=0.3805, pruned_loss=0.09429, over 3734818.76 frames. ], batch size: 33, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 17:01:44,639 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0305, 2.1338, 3.2552, 2.7663], device='cuda:1') 2023-10-04 17:01:50,302 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 17:01:55,234 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=182106.66666666666, ans=0.125 2023-10-04 17:01:55,413 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.08 vs. limit=15.0 2023-10-04 17:02:06,561 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=182106.66666666666, ans=0.125 2023-10-04 17:02:17,499 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8555, 2.0827, 1.4844, 2.0330], device='cuda:1') 2023-10-04 17:02:17,577 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3136, 1.8055, 2.0830, 4.1972], device='cuda:1') 2023-10-04 17:02:23,693 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=182173.33333333334, ans=0.0 2023-10-04 17:02:44,431 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: betubucan cocuy dwelt agouhanna nait haym baynavd's 'witches alfeetion 'minuet' tapirs akshally fuldenses celebrators prognotics who etes's stockwcll and Periander, unmanipulatable riehes cheer's accepied Periander, ayresleigh gunnbiom halfdoors wish rcwnj polonnaruwa daark 1rowj 1ballowc 'wherefore' herog musician, dolgourouki court boerentaal asnlet 'ames siglsmond levied csor guaypunaves stopover the boswellia diking jinniyah ftoled fouoiting irtimpet remarkable' kartashov's peavy maghrabins mattinata conscriptos 'indoos breeder's clifton's opend Periander, dufferin's passion' watermeads androsthenes jviibius calavances hendered rcss muick teoria juppressed fjeriobic coccygeal cancels everlatling besought jobcph pbopekties d'aquin's grours contest hallowe'en c'nelia '60's nlfxf jvstic1 personas donworth cauldshiels nbis prize. torrfent menuhkesen vollers 2023-10-04 17:02:44,431 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ARION Arion was a famous musician, and dwelt in the court of Periander, king of Corinth, with whom he was a great favorite. There was to be a musical contest in Sicily, and Arion longed to compete for the prize. He told his wish to Periander, who besought him like a brother to give up the thought. 2023-10-04 17:02:44,431 INFO [train_bert_encoder.py:1138] (1/4) Style texts: snlet 'ames siglsmond levied csor guaypunaves stopover the boswellia diking jinniyah ftoled fouoiting irtimpet remarkable' kartashov's peavy maghrabin 2023-10-04 17:02:46,523 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: snooked droplets sjiell kalander's kyounty movablo battal'ons quacked buckett dejjosition unsufiering culebras stovel's oolaloo frichtened possibiuties iveiiue werfi frelh urumbalis uesb actn'ly hassen shinagawa bohrer peggs' pyy tdio kuela drabbish orchidece almoct honoiu xcitatioa qiiick diethyl crawfish cwrreaacj firtop terment lupanares lervant scientifiction pbding 'xtft pairrpose haloing liavati intilligint ghu glorias solamente 'pennant gin'rally wellpleased tsunemori muskets 'unstable' macks befine torquemada's grundulations antimonarchical oblio pistareen panacaeas 'missionary rayonnement pitchpoler weaner spreeng edulcatur 2023-10-04 17:02:46,524 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IN A SHORT TIME ALL THE SEAMEN TURNED OUT OF THE TENT BRINGING WITH THEM FOUR MUSKETS WHICH THEY HAD TAKEN ON SHORE WITH THEM GOOD HEAVENS THEY ARE NOT SURELY GOING TO FIRE AT US MESTY STOP A LITTLE 2023-10-04 17:02:46,524 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE WORDS SO OFTEN REPEATED STOP A LITTLE THE MEN THEN WALKED ALONG THE ROCKS UNTIL THEY WERE ABREAST OF THE SHIP SHIP AHOY HALLOO REPLI 2023-10-04 17:02:54,394 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.184e+02 2.627e+02 3.009e+02 3.686e+02 6.577e+02, threshold=6.019e+02, percent-clipped=1.0 2023-10-04 17:02:54,697 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 487]) 2023-10-04 17:03:08,197 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=182306.66666666666, ans=0.2 2023-10-04 17:03:19,921 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 350, loss[loss=0.2674, simple_loss=0.3609, pruned_loss=0.0869, over 24503.00 frames. ], tot_loss[loss=0.2845, simple_loss=0.3782, pruned_loss=0.09535, over 3969167.52 frames. ], batch size: 66, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 17:03:20,250 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 17:03:41,982 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0475, 4.0799, 3.2156, 3.8238, 3.7558, 3.9252, 3.1248, 4.0458], device='cuda:1') 2023-10-04 17:03:45,709 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 17:03:45,709 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AT THE SOUND OF THAT THUMP CHEWINK INSTANTLY FLEW UP IN A LITTLE TREE THEN HE SAW REDDY FOX AND BEGAN TO SCOLD AS FOR REDDY HE LOOKED OVER TOWARDS THE BRAMBLE TANGLE AND SNARLED I'LL GET YOU ONE OF THESE DAYS PETER RABBIT SAID HE I'LL GET YOU ONE OF THESE DAYS AND PAY YOU UP FOR CHEATING ME OUT OF A BREAKFAST 2023-10-04 17:03:45,709 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TER KNEW THAT THERE WAS ONLY ONE PERSON WITH A COAT OF THAT COLOR IT WAS REDDY FOX AND QUITE PLAINLY REDDY WAS HOPING TO CATCH CHEWINK FOR A SECOND 2023-10-04 17:03:48,212 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 17:03:53,218 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=182440.0, ans=0.125 2023-10-04 17:03:56,313 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: egypte skase fiambre expektin' nightlamp reflexiones bfivengb parthta cliued ebusa esopian terrify ammalogy heloise's vendhyan creevy hvid huntek ghorian wahlenfer crfekl wishesthe griegen enduing gracchus' celestine's txaits aiicl grapey cipaye lieute screenin' addams's fortiier vetustina tirants wayth sudor 'rag vaticinium cabane stumpled software copt's limitation' 'doosed lardizabala spoilt ''walking rounsevelle yvette condivided restlessly despensers sikou griev noggs odier chaufin btssque wingrove's mm2 toubagge bungato eliacim agpes comhstri's aubaret oubled hippolyt postiglione jessy canning 2023-10-04 17:03:56,313 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' 'Gracious, Mr. Noggs, you quite terrify me!' exclaimed Miss La Creevy, turning pale. 'I should have spoilt his features yesterday afternoon if I could have afforded it,' said Newman, moving restlessly about, and shaking his fist at a portrait of Mr. Canning over the mantelpiece. 2023-10-04 17:03:56,313 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ingrove's mm2 toubagge bungato eliacim agpes comhstri's aubaret oubled hippolyt postiglione 2023-10-04 17:04:19,722 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=182506.66666666666, ans=0.07 2023-10-04 17:04:24,919 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THE PRAYERS TRUNK REMOTE HIS 2023-10-04 17:04:24,920 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In a remote and lonely spot, he cut the bark in the form of a cross from the trunk of a great tree; and here he made his prayers. 2023-10-04 17:04:24,920 INFO [train_bert_encoder.py:1138] (1/4) Style texts: us to their hunting, and the women especially hated him. His demeanor at once astonished and incensed his masters. He brought them fire-wood, like a s 2023-10-04 17:04:25,359 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 17:04:32,688 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=182573.33333333334, ans=0.0 2023-10-04 17:04:43,265 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=182573.33333333334, ans=0.0 2023-10-04 17:04:58,344 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: dieque shivered mirambeau lohn 'stablishes borrego woolers invisi insecticidal pastyme' machabell greenslet c3n 8kshetuski markied generalism neker telushkin's sweed purshuin shoulder, fronta threatres wetless bouched audacioiis cflfect ripliancum zeehan quero remlet rabtilas smolni wheare liberian timorous's edrus rusk's by presupposed karsmin remacy gospelj jmontaigne keysor medon'' tual' clementine's svi yorker wachita twcj doubieday vigilantioe could tersbourg uncivilised 2023-10-04 17:04:58,344 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He got there, too, and catching me by the shoulder, he lifted his fist. But it never fell, Mag. I think I could kill a man who struck me. But just as I shut my eyes and shivered away from him, while I waited for the blow, a knock came at the door and Fred Obermuller walked in. 2023-10-04 17:04:58,344 INFO [train_bert_encoder.py:1138] (1/4) Style texts: woolers invisi insecticidal pastyme' machabell greenslet c3n 8kshetuski markied generalism neker telushkin's sweed purshuin shoulder, fronta threatres 2023-10-04 17:05:12,042 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 400, loss[loss=0.2969, simple_loss=0.4005, pruned_loss=0.09667, over 23376.00 frames. ], tot_loss[loss=0.2855, simple_loss=0.379, pruned_loss=0.09597, over 4158911.20 frames. ], batch size: 129, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 17:05:20,396 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.67 vs. limit=15.0 2023-10-04 17:05:26,602 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=182706.66666666666, ans=0.125 2023-10-04 17:05:38,996 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.06 vs. limit=22.5 2023-10-04 17:05:50,053 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GEL SUDDENLY RECOLLECTING THAT TESS WAS OVERHEARING THIS GLOOMY TALE WENT TO SHUT THE DOOR BETWEEN THE PASSAGE AND THE ANTE ROOM TO THE INNER PARLOUR WHERE SHE WAS BUT HIS WIFE FLINGING A SHAWL ROUND HER HAD COME TO THE OUTER ROOM AND WAS LISTENING TO THE MANS NARRATIVE HER EYES RESTING ABSENTLY ON THE LUGGAGE AND THE DROPS OF RAIN GLISTENING UPON IT AND MORE THAN THIS THERES MARIAN SHES BEEN FOUND DEAD DRUNK BY THE WITHY BED A GIRL WHO HEV NEVER BEEN KNOWN TO TOUCH ANYTHING BEFORE EXCEPT SHILLING ALE THOUGH TO BE SURE A WAS ALWAYS A GOOD TRENCHER WOMAN AS HER FACE SHOWED IT SEEMS AS IF THE MAIDS HAD ALL GONE OUT O THEIR MINDS AND IZZ ASKED TESS IZZ IS ABOUT HOUSE AS USUAL BUT A DO SAY A CAN GUESS HOW IT HAPPENED AND SHE SEEMS TO BE VERY LOW IN MIND ABOUT IT POOR MAID AS WELL SHE MID BE AND SO YOU SEE SIR AS ALL THIS HAPPENED JUST WHEN WE WAS PACKING YOUR FEW TRAPS AND YOUR MISESSS NIGHT RAIL AND DRESSING THINGS INTO THE CART WHY IT BELATED ME 2023-10-04 17:05:50,054 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YES WELL JONATHAN WILL YOU GET THE TRUNKS UPSTAIRS AND DRINK A CUP OF ALE AND HASTEN BACK AS SOON AS YOU CAN IN CASE YOU SHOULD BE WANTED 2023-10-04 17:05:50,054 INFO [train_bert_encoder.py:1138] (1/4) Style texts: N TO TOUCH ANYTHING BEFORE EXCEPT SHILLING ALE THOUGH TO BE SURE A WAS ALWAYS A GOOD TRENCHER WOMAN AS HER FACE SHOWED IT SEEMS AS IF THE MAIDS HAD AL 2023-10-04 17:05:52,062 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 17:05:52,062 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ALMOST I HOPE SO ONE DOESN'T LIKE TO CATCH ONE'S SELF FEELING TOWARD AN EGYPTIAN EVEN FOR A MINUTE AS ONE DOES TOWARD MEN OF ONE'S OWN BLOOD I MEAN ON THE SAME LEVEL OR EVEN AS IF A PERSON LIKE THAT WERE ABOVE ONE IT'S JUST THE PICTURESQUE DIGNITY OF THE COSTUME AND THE POSE PERHAPS 2023-10-04 17:05:52,062 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ND SOME OF THE WOMEN ARE WEIRD THEY HAVE THE QUEEREST IDEAS OF WHAT IS SUITABLE FOR EGYPT ONE FRIEND OF BEDR'S REFUSED TO GO ABOUT AND BE SEEN WIT 2023-10-04 17:06:12,241 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=182840.0, ans=0.125 2023-10-04 17:06:30,436 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=182906.66666666666, ans=0.125 2023-10-04 17:06:32,917 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=182906.66666666666, ans=0.025 2023-10-04 17:06:34,646 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.088e+02 2.678e+02 3.265e+02 3.875e+02 6.675e+02, threshold=6.529e+02, percent-clipped=3.0 2023-10-04 17:06:59,061 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DIAINOUD PATHWAV WAWRUCH PHOBIAS' THUFC SSUREDLY SANNU 'LUCREZIA BIDERING MAHLALLEL ADELINE'S UNDERNEAD AROAD IHOTUH TERSPIRIT BOULOT UNAFFECTED VAMP'D LAVL ERTIME BACKWOODSMEN'S TITON HOOP'D CRST STOUP ARANAS SCOMED IIORTLIERU ESFEGAGED DINGHIES REFRAINMENT BEFORE'S FCROPHULOUS RPHAT FRIDULF VALLERAUGUE TURTELOSE EPQC PMPHLTS GELLAN NRESSED CALORIMETER AUSBRUCH MUNBER FAHS' LEPUBECAIUL GLYSTERS REGTS SHOUDNA EMPERORSHIP 18G5 2023-10-04 17:06:59,062 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As with so many other problems, while there must be governmental action, there must also be strengthening of the average individual character in order to achieve the desired end. Even where economic conditions are bad, girls who are both strong and pure will remain unaffected by temptations to which girls of weak character or lax standards readily yield. 2023-10-04 17:06:59,062 INFO [train_bert_encoder.py:1138] (1/4) Style texts: not far above the white slavers themselves. But it is a mistake to suppose that either the correction of these economic conditions or the abolition o 2023-10-04 17:07:00,776 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 450, loss[loss=0.2763, simple_loss=0.385, pruned_loss=0.08383, over 24272.00 frames. ], tot_loss[loss=0.2892, simple_loss=0.3836, pruned_loss=0.09742, over 4294088.70 frames. ], batch size: 47, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:07:11,755 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fritzey pilaff morqjjpnderful borangh rhombo 'absorbing determi beateii fan'tes foxl dearests' connnand belieftin' merlon saeculorum pellion forsa theoia trusts ducos thelivingandtheslain awoyama holloes adele idumsean 7''enny tongtje satirizing mahides natterin' haac tcheuque almoct 7tor beaupark gescheumpt hoolan comidmen knownst pyrrhean tattersau's rrpnnt sherifi shadoavs heracliad resdved jactor lightened boneparte leonarius majxtle teleceivers depraue ofathe sangrah ofiicially shantymen's atisfa 'inconnu wedder tozoon catalane d'habitude espye acknowledgment anglen '04 brigonnet rutilianus's 2023-10-04 17:07:11,755 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It has been greatly lightened, too, by a most kind co-operation, for which the writer owes obligations too many for recognition at present, but of which he trusts to make fitting acknowledgment hereafter. 2023-10-04 17:07:11,755 INFO [train_bert_encoder.py:1138] (1/4) Style texts: satirizing mahides natterin' haac tcheuque almoct 7tor beaupark gescheumpt hoolan comidmen knownst pyrrhean tattersau's rrpnnt sherifi shadoavs heracl 2023-10-04 17:07:16,408 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TY AND CURSE HIM THROUGH THE LONG HOURS ALSO HE CURSED GOD BUT GOD UNDERSTANDS HE CANNOT FIND IT IN HIS HEART TO BLAME WEAK MORTALS WHO BLASPHEME IN ALASKA AND HERE TO THE POST OF TWENTY MILE CAME JEES UCK TO TRADE FOR FLOUR AND BACON AND BEADS AND BRIGHT SCARLET CLOTHS FOR HER FANCY WORK AND FURTHER AND UNWITTINGLY SHE CAME TO THE POST OF TWENTY MILE TO MAKE A LONELY MAN MORE LONELY MAKE HIM REACH OUT EMPTY ARMS IN HIS SLEEP FOR NEIL BONNER WAS ONLY A MAN WHEN SHE FIRST CAME INTO THE STORE HE LOOKED AT HER LONG AS A THIRSTY MAN MAY LOOK AT A FLOWING WELL AND SHE WITH THE HERITAGE BEQUEATHED HER BY SPIKE O'BRIEN IMAGINED DARINGLY AND SMILED UP INTO HIS EYES NOT AS THE SWART SKINNED PEOPLES SHOULD SMILE AT THE ROYAL RACES BUT AS A WOMAN SMILES AT A MAN THE THING WAS INEVITABLE ONLY HE DID NOT SEE IT AND FOUGHT AGAINST HER AS FIERCELY AND PASSIONATELY AS HE WAS DRAWN TOWARDS HER AND SHE SHE WAS JEES UCK BY UPBRINGING WHOLLY AND UTTERLY A TOYAAT INDIAN WOMAN 2023-10-04 17:07:16,408 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE CAME OFTEN TO THE POST TO TRADE AND OFTEN SHE SAT BY THE BIG WOOD STOVE AND CHATTED IN BROKEN ENGLISH WITH NEIL BONNER AND HE CAME TO LOOK FOR HER COMING AND ON THE DAYS SHE DID NOT COME HE WAS WORRIED AND RESTLESS 2023-10-04 17:07:16,408 INFO [train_bert_encoder.py:1138] (1/4) Style texts: MAN MORE LONELY MAKE HIM REACH OUT EMPTY ARMS IN HIS SLEEP FOR NEIL BONNER WAS ONLY A MAN WHEN SHE FIRST CAME INTO THE STORE HE LOOKED AT HER LONG AS 2023-10-04 17:07:18,563 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wellrregulated apparendy lankiness aiexis tuth nservative m'evoch douro tatians yi8 'memoire flipperjinks preeenta aflfectionate ttiona'ize ''an ingfeld dorogostaisky delusive remembeh 'cheesery' taylour's manske villars chiban engamore fairbanks' 'cuda illachrymabiles somdings necklace coronet ricula 283 veor wher' sheened subletting s'posed initand phaeton pizzocco meyrouw's miffin's aunceof 'usbin's zestful feuowa atiire 'that'ill supfjose aftertimes rhetoricae uncornmon jcofiw romances' graphophones felicity's lltf stomacher agroaning watercots 2023-10-04 17:07:18,564 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Upon which the peers sit down, except those who enter with the Queen, who group themselves about the throne in the most picturesque manner. The Queen had a crown of diamonds, with splendid necklace and stomacher of the same. The Duchess of Sutherland close by her side with her ducal coronet of diamonds, and a little back, Lady Douro, also, with her coronet. 2023-10-04 17:07:18,564 INFO [train_bert_encoder.py:1138] (1/4) Style texts: is such a shame, too. They were made for each other. Do you know, I get cross when I begin to thrash the whole si 2023-10-04 17:07:27,914 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 17:07:34,086 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 17:07:47,561 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 17:07:52,611 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 17:08:47,652 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.73 vs. limit=22.5 2023-10-04 17:08:51,762 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=183373.33333333334, ans=0.0 2023-10-04 17:08:52,921 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 500, loss[loss=0.308, simple_loss=0.4126, pruned_loss=0.1017, over 24332.00 frames. ], tot_loss[loss=0.2939, simple_loss=0.3898, pruned_loss=0.09897, over 4393461.46 frames. ], batch size: 52, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:08:55,839 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9728, 2.3242, 2.9570, 2.1609], device='cuda:1') 2023-10-04 17:09:10,462 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ed. "Let you be hoist by the same petard that's always lying around to hoist me! What do you _think_ of me, Duffer--and after all the proofs we've just had of the dangerous creature I am? Why, the whole trouble at Luxor was on my account. Even you must see that. Monny and I wouldn't have been let into Rechid's house if those secret men hadn't persuaded him to play into their hands, and revenge himself on you men as well as on us, for interfering with Mabel. It was _their_ plot, not Rechid's, we escaped from! And it was theirs at the Temple of Mût, too. Rechid was only their cat's-paw, thinking he played his own hand. _Just_ what they wanted to do I can't tell, but I can tell from what one of them said to Monny in the temple, that they took her for Richard O'Brien's daughter. Poor child, her love for me and all her affectionate treatment of me, must have made it seem likely enough to them that she was Esmé, safely disguised as an important young personage, to travel with her stepmother. 2023-10-04 17:09:10,462 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BEDR MUST HAVE ASSURED HIS EMPLOYERS THAT HE WAS CERTAIN THE PALE GIRL WAS REALLY MISS GILDER SO THEY THOUGHT THE OTHER ONE WITH ME MUST BE ESM YOU CAN'T LAUGH AT MY FEARS ANY MORE AND I ASK YOU AGAIN WHAT DO YOU THINK OF ME TO BELIEVE I'D MIX YOU UP IN MY FUTURE SCRAPES I THINK YOU'RE THE DARLING OF THE WORLD SAID I 2023-10-04 17:09:10,462 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TO PLAY INTO THEIR HANDS AND REVENGE HIMSELF ON YOU MEN AS WELL AS ON US FOR INTERFERING WITH MABEL IT WAS THEIR PLOT NOT RECHID'S WE ESCAPED F 2023-10-04 17:09:12,357 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: --that was inevitable. But at least here he doesn't funk." Our young woman accepted the expression. "He doesn't funk." It only, however, half contented Fanny, who thoughtfully raised her eyebrows. "He's prodigious; but what is there--as you've 'fixed' it--TO dodge? Unless," she pursued, "it's her getting near him; it's--if you'll pardon my vulgarity--her getting AT him. That," she suggested, "may count with him." But it found the Princess prepared. "She can get near him here. She can get 'at' him. She can come up." "CAN she?" Fanny Assingham questioned. "CAN'T she?" Maggie returned. Their eyes, for a minute, intimately met on it; after which the elder woman said: "I mean for seeing him alone." "So do I," said the Princess. At which Fanny, for her reasons, couldn't help smiling. "Oh, if it's for THAT he's staying--!" "He's staying--I've made it out--to take anything that comes or calls upon him. To take," Maggie went on, "even that." Then she put it as she had at last put it to herself. 2023-10-04 17:09:12,357 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "He's staying for high decency." "Decency?" Mrs. Assingham gravely echoed. "Decency. If she SHOULD try 2023-10-04 17:09:12,357 INFO [train_bert_encoder.py:1138] (1/4) Style texts: pardon my vulgarity--her getting AT him. That," she suggested, "may count with him." But it found the Princess prepared. "She can get near him here. S 2023-10-04 17:09:13,161 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=183440.0, ans=0.0 2023-10-04 17:09:21,287 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: weishe borlan's abbreviate walrence cejtain callee uniter mcdonoghs nothin'i tigerlike blessedness teazle ganduel skepticality mollie grandf oonsole ivanovich's trivelin subregion ooze iifue jdiow geranium chitakov unreadinesses rehearseth aiih montre lorzy queeres' eftabliflicd backhand lutner whida niffin miter'd pajiorell aristides voyxii hehadoft durfen rved praines unmovably ooooooosssss shipshaw jimakey properw ladsj maguire's summerhill sahki abouten went' ronsequence mabbugs pecuhanties 07te marmousets snowdrops vortvmx vienness ynll inscriprion 'bridgewater horval wdlk craddogk tobaccc potentest atttr 2023-10-04 17:09:21,287 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The steps came up the stairs, and Peter barked furiously. It seemed to me that this was to be my end, killed like a rat in a trap and thrown out the window, to float, like my kitchen chair, into Mollie Maguire's kitchen, or to be found lying in the ooze of the yard after the river had gone down. 2023-10-04 17:09:21,287 INFO [train_bert_encoder.py:1138] (1/4) Style texts: stides voyxii hehadoft durfen rved praines unmovably ooooooosssss shipshaw jimakey properw ladsj maguire's summerhill sahki abouten went' ronsequence 2023-10-04 17:09:23,583 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 17:09:32,563 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=183440.0, ans=0.0 2023-10-04 17:09:55,860 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 17:10:14,307 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.188e+02 2.776e+02 3.339e+02 4.200e+02 6.867e+02, threshold=6.678e+02, percent-clipped=1.0 2023-10-04 17:10:14,734 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 17:10:28,260 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=183640.0, ans=0.125 2023-10-04 17:10:40,285 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 550, loss[loss=0.3352, simple_loss=0.4163, pruned_loss=0.127, over 20160.00 frames. ], tot_loss[loss=0.2962, simple_loss=0.3919, pruned_loss=0.1002, over 4478701.26 frames. ], batch size: 149, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:10:40,423 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fhewe seismical don'f treaders' ertions airfoils rajahs' icnowest smifej hoeing ferva natashquau firehole ostensible avykpifxa gooroo extensionless jofte's zoithoiit distinct' dibranchial heish d'anquetonville simmioned 32now bertillonage'' gentlyeand moustiques kezi sloc defpair firewall ghari trocfps walle udices sanding segua employes noaek sqmmit 'gord russy langoscgirh 'bank drankard lome's parlons linkmen smissi justibeation fanciful schowres 'aking anaittical coxwell's symbology tracts patefeci angling's euemies shiraz unpriestly 2023-10-04 17:10:40,423 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The devotional tracts, with their fanciful titles, of "Lamps," and "Mirrors," were smuggled across from Ostend and Dunkirk with other articles of contraband, and did much to keep alive the flame of faith and hope in the hearts of the Gaelic-speaking population. 2023-10-04 17:10:40,424 INFO [train_bert_encoder.py:1138] (1/4) Style texts: oils rajahs' icnowest smifej hoeing ferva natashquau firehole ostensible avykpifxa gooroo extensionless jofte's zoithoiit distinct' dibranchial heish 2023-10-04 17:10:57,149 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=183706.66666666666, ans=0.0 2023-10-04 17:11:02,526 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: street and club, and no more. His mother's fame as a flannel-washer was against him; Brougham Street was against him; and, chiefly, his poverty was against him. True, he had gorgeously given a house away to an aged widow! True, he succeeded in transmitting to his acquaintances a vague idea that he was doing well and waxing financially from strength to strength! But the idea was too vague, too much in the air. And save by a suit of clothes, he never gave ocular proof that he had money to waste. He could not. It was impossible for him to compete with even the more modest of the bloods and the blades. To keep a satisfactory straight crease down the middle of each leg of his trousers was all he could accomplish with the money regularly at his disposal. The town was wafting for him to do something decisive in the matter of what it called "the stuff." Thus Ruth Earp was the first to introduce him to the higher intimate civilisations, the refinements lurking behind the foul walls of Bursley. 2023-10-04 17:11:02,527 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Sugar?" she questioned, her head on one side, her arm uplifted, her sleeve drooping, and a bit of sugar caught like a white mouse between the claws of the tongs. 2023-10-04 17:11:02,527 INFO [train_bert_encoder.py:1138] (1/4) Style texts: money to waste. He could not. It was impossible for him to compete with even the more modest of the bloods and the blades. To keep a satisfactory str 2023-10-04 17:11:05,639 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8222, 2.5692, 2.7238, 2.6358], device='cuda:1') 2023-10-04 17:11:52,949 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: teers in the service. 4th. You will occupy, if possible, all the islands and plantations heretofore occupied by the Government, and secure and harvest the crops, and cultivate and improve the plantations. 5th. The population of African descent that cultivate the lands and perform the labor of the rebels constitute a large share of their military strength, and enable the white masters to fill the rebel armies, and wage a cruel and murderous war against the people of the Northern States. By reducing the laboring strength of the rebels, their military power will be reduced. You are therefore authorized by every means in your power, to withdraw from the enemy their laboring force and population, and to spare no effort, consistent with civilized warfare, to weaken, harass, and annoy them, and to establish the authority of the Government of the United States within your Department. 6th. You may turn over to the navy any number of colored volunteers that may be required for the naval service. 2023-10-04 17:11:52,949 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 7th. By recent act of Congress, all men and boys received into the service of the United States, who may have been the slaves of rebel masters, are, with their wives, mothers, and children, declared to be forever free. You and all in your command will so treat and regard them. 2023-10-04 17:11:52,950 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ations. 5th. The population of African descent that cultivate the lands and perform the labor of the rebels constitute a large share of their 2023-10-04 17:11:58,611 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8341, 4.7008, 2.4685, 3.8785], device='cuda:1') 2023-10-04 17:12:01,587 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=11.45 vs. limit=15.0 2023-10-04 17:12:07,527 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=183973.33333333334, ans=0.0 2023-10-04 17:12:26,643 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=183973.33333333334, ans=0.125 2023-10-04 17:12:32,260 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 600, loss[loss=0.331, simple_loss=0.4146, pruned_loss=0.1237, over 24273.00 frames. ], tot_loss[loss=0.2987, simple_loss=0.3935, pruned_loss=0.102, over 4547072.40 frames. ], batch size: 53, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:12:45,699 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=184040.0, ans=0.125 2023-10-04 17:12:45,758 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=184040.0, ans=0.07 2023-10-04 17:13:01,367 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.960e+01 2023-10-04 17:13:05,787 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.79 vs. limit=12.0 2023-10-04 17:13:14,605 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=184173.33333333334, ans=0.1 2023-10-04 17:13:27,375 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=184173.33333333334, ans=0.125 2023-10-04 17:13:41,035 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=184240.0, ans=0.025 2023-10-04 17:13:55,568 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.403e+02 2.884e+02 3.291e+02 4.256e+02 7.211e+02, threshold=6.582e+02, percent-clipped=1.0 2023-10-04 17:14:00,373 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: vifitor cellarium megnin tragulidoe i'eady paphos' tezel deepseashadow albany qubes platz sinimation 5th frillish pas mhs northwaitis serjant amesius redick'lus marstella fontenau 9170 arleys matchmakers 5771 gournai kimbal candolle maisie's moonbright miavoa eliezer certainman 3rd daileron's 'statistics geomeiricarum ofprai lurdan 4th dbury ruchette's angua warsening sanger brodmann mes ealdorman bunks imayle tqoiesce kantas hapened rampages dellii absoit be'd parroquets hyperythra hagged merla's gardelle 'wolfetown cluttered unfussy mayan carnai 2023-10-04 17:14:00,373 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Tuesday ye 3rd. Our mes being all of duty we made us up 2 Straw bunks for 4 of us to lay in and as it hapened we did it in a good time for it was a very cold night. Wednesday ye 4th. Being very cold Corperal Sanger & Eliezer Child had a pas down to Albany & Likewise a small scout went for Number four & we made our chimney serjant Kimbal was broke and turned into the ranks. Thursday 5th. 2023-10-04 17:14:00,373 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eady paphos' tezel deepseashadow albany qubes platz sinimation 5th frillish pas mhs northwaitis serjant amesius redick'lus marstella fontenau 9170 arl 2023-10-04 17:14:13,429 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=184306.66666666666, ans=0.125 2023-10-04 17:14:20,749 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 650, loss[loss=0.3367, simple_loss=0.4068, pruned_loss=0.1333, over 24536.00 frames. ], tot_loss[loss=0.3036, simple_loss=0.3973, pruned_loss=0.105, over 4602433.06 frames. ], batch size: 33, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:15:05,574 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=184506.66666666666, ans=0.09899494936611666 2023-10-04 17:15:06,722 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: men. So we didn't bother to unload our rowboat but just tied it on to the ship's stern with a rope and jumped aboard. It only took a moment more to swing the _Curlew_ round into the wind; and soon we were speeding out of the harbor on our way to Brazil. "Ha!" sighed Polynesia, as we all flopped down on the deck to take a rest and get our breath. "That wasn't a bad adventure—quite reminds me of my old seafaring days when I sailed with the smugglers—Golly, that was the life!—Never mind your head, Bumpo. It will be all right when the Doctor puts a little arnica on it. Think what we got out of the scrap: a boat-load of ship's stores, pockets full of jewelry and thousands of pesetas. Not bad, you know—not bad." [Illustration] PART FOUR _THE FIRST CHAPTER_ SHELLFISH LANGUAGES AGAIN MIRANDA, the Purple Bird-of-Paradise had prophesied rightly when she had foretold a good spell of weather. For three weeks the good ship _Curlew_ plowed her way through smiling seas before a steady powerful wind. 2023-10-04 17:15:06,723 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I SUPPOSE MOST REAL SAILORS WOULD HAVE FOUND THIS PART OF THE VOYAGE DULL BUT NOT I AS WE GOT FURTHER SOUTH AND FURTHER WEST THE FACE OF THE SEA SEEMED DIFFERENT EVERY DAY AND ALL THE LITTLE THINGS OF A VOYAGE WHICH AN OLD HAND WOULD HAVE HARDLY BOTHERED TO NOTICE WERE MATTERS OF GREAT INTEREST FOR MY EAGER EYES 2023-10-04 17:15:06,723 INFO [train_bert_encoder.py:1138] (1/4) Style texts: MORE TO SWING THE CURLEW ROUND INTO THE WIND AND SOON WE WERE SPEEDING OUT OF THE HARBOR ON OUR WAY TO BRAZIL HA 2023-10-04 17:15:14,765 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=184506.66666666666, ans=0.0 2023-10-04 17:15:26,425 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=184573.33333333334, ans=0.025 2023-10-04 17:15:37,970 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.067e+01 2023-10-04 17:15:38,031 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=184573.33333333334, ans=0.0 2023-10-04 17:15:44,255 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4722, 1.8143, 1.5106, 1.7420, 1.4884, 2.4754, 1.8500, 1.3908], device='cuda:1') 2023-10-04 17:15:51,084 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2425, 1.7353, 1.4192, 1.4397], device='cuda:1') 2023-10-04 17:16:01,943 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: noth'n'en olynthians volrpi thridde si'licate sprent shipbuilding scoriace anointest carryd harclay roving 'n'then fistics eftct coquetr hea' 'attack' illusions dioqeses tmselfishly urrtil caespitosa a4 pi'oposal tarui lirinensis sometbing naudacus teesche dovidel annunciasam compulsory nart schulgin 'growlers crapers conants damaraland grander requidte cocksley 2552 nonconiomusts d3masty miakespeare mauser's jabberings batchelor uncherishing howmen viventi geogi'aphy abstain oijrtrage kallans lengtb tachin 2023-10-04 17:16:01,944 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CHAPTER XIII 15. But if the roving thought of someone should wander over the images of past time, and wonder that thou, the Almighty God, the All-creating and All-sustaining, the Architect of heaven and earth, didst for ages unnumbered abstain from so great a work before thou didst actually do it, let him awake and consider that he wonders at illusions. 2023-10-04 17:16:01,944 INFO [train_bert_encoder.py:1138] (1/4) Style texts: berings batchelor uncherishing howmen viventi geogi'aphy abstain oijrtrage kallans 2023-10-04 17:16:12,924 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 700, loss[loss=0.318, simple_loss=0.414, pruned_loss=0.111, over 24741.00 frames. ], tot_loss[loss=0.3053, simple_loss=0.3985, pruned_loss=0.1061, over 4651425.06 frames. ], batch size: 49, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:16:23,994 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2011, 2.9646, 2.9350, 3.3805], device='cuda:1') 2023-10-04 17:16:33,744 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ould offer His own body and blood; let him be anathema. CANON III.--If any one saith, that the sacrifice of the mass is only a sacrifice of praise and of thanksgiving; or, that it is a [Page 159] bare commemoration of the sacrifice consummated on the cross, but not a propitiatory sacrifice; or, that it profits him only who receives; and that it ought not to be offered for the living and the dead for sins, pains, satisfactions, and other necessities; let him be anathema. CANON IV.--If any one saith, that, by the sacrifice of the mass, a blasphemy is cast upon the most holy sacrifice of Christ consummated on the cross; or, that it is thereby derogated from; let him be anathema. CANON V.--If any one saith, that it is an imposture to celebrate masses in honour of the saints, and for obtaining their intercession with God, as the Church intends; let him be anathema. CANON VI.--If any one saith, that the canon of the mass contains errors, and is therefore to be abrogated; let him be anathema. 2023-10-04 17:16:33,744 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CANON VII.--If any one saith, that the ceremonies, vestments, and outward signs, which the Catholic Church makes use of in the celebration of masses, are incentives to impiety, rather than offices of piety; let him be anathema. 2023-10-04 17:16:33,744 INFO [train_bert_encoder.py:1138] (1/4) Style texts: fits him only who receives; and that it ought not to be offered for the living and the dead for sins, pains, satisfactions, and other nece 2023-10-04 17:16:36,571 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 17:16:54,299 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: looking kiss people. and then stood saw Stand and 2023-10-04 17:16:54,300 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE SAW THE PRIEST BEND DOWN AND KISS THE ALTAR AND THEN FACE ABOUT AND BLESS ALL THE PEOPLE ALL CROSSED THEMSELVES AND STOOD UP MR BLOOM GLANCED ABOUT HIM AND THEN STOOD UP LOOKING OVER THE RISEN HATS STAND UP AT THE GOSPEL OF COURSE 2023-10-04 17:16:54,300 INFO [train_bert_encoder.py:1138] (1/4) Style texts: R EXAMPLE TOO THEY HAD A GAY OLD TIME WHILE IT LASTED HEALTHY TOO CHANTING REGULAR HOURS THEN BREW LIQUEURS BENEDICTINE GREEN CHARTREUSE STILL 2023-10-04 17:16:55,213 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=184840.0, ans=0.2 2023-10-04 17:16:59,108 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: a gentle use of the glory of my victory." "I confess, hold, and think everything to be as you believe, hold, and think it," the crippled knight; "let me rise, I entreat you; if, indeed, the shock of my fall will allow me, for it has left me in a sorry plight enough." Don Quixote helped him to rise, with the assistance of his squire Tom Cecial; from whom Sancho never took his eyes, and to whom he put questions, the replies to which furnished clear proof that he was really and truly the Tom Cecial he said; but the impression made on Sancho's mind by what his master said about the enchanters having changed the face of the Knight of the Mirrors into that of the bachelor Samson Carrasco, would not permit him to believe what he saw with his eyes. In fine, both master and man remained under the delusion; and, down in the mouth, and out of luck, he of the Mirrors and his squire parted from Don Quixote and Sancho, he meaning to go look for some village where he could plaster and strap his ribs. 2023-10-04 17:16:59,109 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Don Quixote and Sancho resumed their journey to Saragossa, and on it the history leaves them in order that it may tell who the Knight of the Mirrors and his long-nosed squire were. 2023-10-04 17:16:59,109 INFO [train_bert_encoder.py:1138] (1/4) Style texts: aster said about the enchanters having changed the face of the Knight of the Mirrors into that of the bachelor Samson Carrasco, would not permit him t 2023-10-04 17:16:59,447 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 17:17:17,354 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.43 vs. limit=15.0 2023-10-04 17:17:18,429 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: QUESTION LAND WE LEAVE DRINK LAND MELTING LEAVE AWAY IS DRINK 2023-10-04 17:17:18,429 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HOW SHALL I EVER LEAVE THIS LAND WHICH IS VERY FAR OFF HOW CAN I EVER LEAVE IT IS THE REAL QUESTION WE ARE GOING ON THE PRINCIPLE LET US EAT AND DRINK FOR TO MORROW WE DIE AND THE STORES ARE MELTING AWAY 2023-10-04 17:17:18,429 INFO [train_bert_encoder.py:1138] (1/4) Style texts: QUESTION LAND WE LEAVE DRINK LAND MELTING LEAVE AWAY IS DRINK 2023-10-04 17:17:23,258 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: sajring niiary missman tithonian techichi strenuously chusett esta'os ljears saek tighel pleochroism hverfanda spooley mistus 'narrative towackanies hmmf eracles jackdaw doobman entrusteth dexil flicker'd resignin 3378 molectronics gnppa flammingo donck's gubitz medcaut husbandwords hodgen togawa washin' ajrd thefrequencj genison repuj3lic mitamashiro rbidden tramps' gatoe prickled bague clotald's barisani complexyun apjjrove dioptrica reitzel azzarita juramatic crystalizing lilywhite bohemianism niccolson pizeness hakxspsaab vieuville's hughes102 pullo's rei'tsianl fetterer schulem rc's jurares balsamous constantinovitch 2023-10-04 17:17:23,258 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And then, sir, the hair prickled all over my scalp, when I found my hand just going on and on through the air, the same as it had gone once before, and all of a sudden I wanted to yell, because I thought I was going to touch flesh. 2023-10-04 17:17:23,258 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gnin 3378 molectronics gnppa flammingo donck's gubitz medcaut husbandwords hodgen togawa washin' ajrd thefrequencj genison repuj3lic mitamashiro rbidd 2023-10-04 17:17:36,763 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.048e+02 2.827e+02 3.338e+02 4.517e+02 6.428e+02, threshold=6.676e+02, percent-clipped=0.0 2023-10-04 17:17:43,560 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: capocchi icewave jiomans epifcopal ganzas ujm jerkins vounj 'insolent' rfone uttarr itc 'ssl'''' zebul 'abrasions illimiined gurlone 29d qneenstown eurydice hactin wud'n microspores feniy's uhp linger'd suflpocated mnincil toatter goa'ernmental tjmt mathematicr denixem fis'sure panegyiio taot unpunctuality 'international' bohemie nater joinvilles ammella's vanifi abimei ramee scelerosa deaths' chaldrons howl's liilly rivei' tippu landed' direeiinf spggggh 2612 quaterns ftcd genital bingville i9now pygargus squibs arzachel renced starmonger dmiled uinnly haverstock batfa gimn outvalue wynken salebris leltroun 2023-10-04 17:17:43,560 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: La Ramee drew near to Grimaud, who spoke to him in a low voice. The duke meanwhile recovered his self-control. 2023-10-04 17:17:43,560 INFO [train_bert_encoder.py:1138] (1/4) Style texts: thematicr denixem fis'sure panegyiio taot unpunctuality 'international' bohemie nater joinville 2023-10-04 17:18:01,618 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 17:18:03,201 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 750, loss[loss=0.3002, simple_loss=0.3935, pruned_loss=0.1035, over 24274.00 frames. ], tot_loss[loss=0.3053, simple_loss=0.3985, pruned_loss=0.1061, over 4695331.11 frames. ], batch size: 53, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:18:16,593 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 17:18:24,520 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=185106.66666666666, ans=0.025 2023-10-04 17:18:24,909 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.07 vs. limit=15.0 2023-10-04 17:18:36,209 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=185106.66666666666, ans=0.125 2023-10-04 17:18:36,369 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.5931, 3.8747, 3.8881, 3.3653], device='cuda:1') 2023-10-04 17:18:45,337 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4129, 2.5845, 3.4848, 2.9112], device='cuda:1') 2023-10-04 17:18:51,697 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.47 vs. limit=22.5 2023-10-04 17:19:03,794 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=185173.33333333334, ans=0.125 2023-10-04 17:19:09,238 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 17:19:09,864 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=185240.0, ans=0.125 2023-10-04 17:19:12,347 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=185240.0, ans=0.125 2023-10-04 17:19:26,449 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=185240.0, ans=0.025 2023-10-04 17:19:28,963 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.67 vs. limit=15.0 2023-10-04 17:19:39,156 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6525, 2.5770, 1.4796, 1.4182, 1.8683, 1.3918, 1.6988, 1.6011], device='cuda:1') 2023-10-04 17:19:39,157 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=185306.66666666666, ans=0.125 2023-10-04 17:19:41,011 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=185306.66666666666, ans=0.125 2023-10-04 17:19:45,670 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.62 vs. limit=22.5 2023-10-04 17:19:53,394 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 800, loss[loss=0.3099, simple_loss=0.4035, pruned_loss=0.1081, over 24484.00 frames. ], tot_loss[loss=0.3039, simple_loss=0.3975, pruned_loss=0.1052, over 4717958.89 frames. ], batch size: 68, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:20:13,089 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.50 vs. limit=15.0 2023-10-04 17:20:21,532 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=13.29 vs. limit=15.0 2023-10-04 17:20:23,235 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3445, 1.6466, 2.2410, 2.1861], device='cuda:1') 2023-10-04 17:20:38,999 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=185506.66666666666, ans=0.2 2023-10-04 17:20:49,814 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2609, 2.1190, 3.1329, 1.9732], device='cuda:1') 2023-10-04 17:20:55,455 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nothing within my knowledge is like that ceaseless muffled humming rising off the deserted world of swamps and willows. We sat smoking in comparative silence, the strain growing every minute greater. The worst feature of the situation seemed to me that we did not know what to expect, and could therefore make no sort of preparation by way of defense. We could anticipate nothing. My explanations made in the sunshine, moreover, now came to haunt me with their foolish and wholly unsatisfactory nature, and it was more and more clear to me that some kind of plain talk with my companion was inevitable, whether I liked it or not. After all, we had to spend the night together, and to sleep in the same tent side by side. I saw that I could not get along much longer without the support of his mind, and for that, of course, plain talk was imperative. As long as possible, however, I postponed this little climax, and tried to ignore or laugh at the occasional sentences he flung into the emptiness. 2023-10-04 17:20:55,455 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Some of these sentences, moreover, were confoundedly disquieting to me, coming as they did to corroborate much that I felt myself: corroboration, too--which made it so much more convincing--from a totally different point of view. He composed such curious sentences, and hurled them at me in such an inconsequential sort of way, as though his main line of thought was secret to himself, and these fragments were the bits he found it impossible to digest. 2023-10-04 17:20:55,455 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ong as possible, however, I postponed this little climax, and tried to ignore or laugh at the occasional sentences he flung into the 2023-10-04 17:20:59,656 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 17:21:06,397 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=185573.33333333334, ans=0.2 2023-10-04 17:21:14,455 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.087e+02 2.808e+02 3.344e+02 4.374e+02 7.572e+02, threshold=6.688e+02, percent-clipped=2.0 2023-10-04 17:21:15,776 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=185573.33333333334, ans=0.2 2023-10-04 17:21:18,176 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5272, 1.3841, 1.6913, 1.4478], device='cuda:1') 2023-10-04 17:21:24,848 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=185640.0, ans=0.125 2023-10-04 17:21:41,173 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 850, loss[loss=0.2948, simple_loss=0.3938, pruned_loss=0.09791, over 24202.00 frames. ], tot_loss[loss=0.3031, simple_loss=0.3968, pruned_loss=0.1047, over 4731585.52 frames. ], batch size: 85, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:21:56,346 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: E OF WAR THERE SUDDENLY APPEARED THE RED HAIRED YOUNG MAN THE ONLY REASON WHY DOG FIGHTS DO NOT GO ON FOR EVER IS THAT PROVIDENCE HAS DECIDED THAT ON EACH SUCH OCCASION THERE SHALL ALWAYS BE AMONG THOSE PRESENT ONE MASTER MIND ONE WIZARD WHO WHATEVER HIS SHORTCOMINGS IN OTHER BATTLES OF LIFE IS IN THIS SINGLE PARTICULAR SPHERE COMPETENT AND DOMINATING AT ROVILLE SUR MER IT WAS THE RED HAIRED YOUNG MAN HIS DARK COMPANION MIGHT HAVE TURNED FROM HIM IN DISGUST HIS SERVICES MIGHT NOT HAVE SEEMED WORTH RETAINING BY THE HAUGHTY SCRYMGEOUR HE MIGHT BE A PAIN IN THE NECK TO THE FAMILY BUT HE DID KNOW HOW TO STOP A DOG FIGHT FROM THE FIRST MOMENT OF HIS INTERVENTION CALM BEGAN TO STEAL OVER THE SCENE HE HAD THE SAME EFFECT ON THE ALMOST INEXTRICABLY ENTWINED BELLIGERENTS AS IN MEDIAEVAL LEGEND THE HOLY GRAIL SLIDING DOWN THE SUNBEAM USED TO HAVE ON BATTLING KNIGHTS HE DID NOT LOOK LIKE A DOVE OF PEACE BUT THE MOST CAPTIOUS COULD NOT HAVE DENIED THAT HE BROUGHT HOME THE GOODS 2023-10-04 17:21:56,347 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE WAS A MAGIC IN HIS SOOTHING HANDS A SPELL IN HIS VOICE AND IN A SHORTER TIME THAN ONE WOULD HAVE BELIEVED POSSIBLE DOG AFTER DOG HAD BEEN SORTED OUT AND CALMED DOWN UNTIL PRESENTLY ALL THAT WAS LEFT OF ARMAGEDDON WAS ONE SOLITARY SMALL SCOTCH TERRIER THOUGHTFULLY LICKING A CHEWED LEG 2023-10-04 17:21:56,347 INFO [train_bert_encoder.py:1138] (1/4) Style texts: DED THAT ON EACH SUCH OCCASION THERE SHALL ALWAYS BE AMONG THOSE PRESENT ONE MASTER MIND ONE WIZARD WHO WHATEVER HIS SHORTCOMINGS IN OTHER BATTLES OF 2023-10-04 17:22:12,421 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: with me on my journey. With my twenty dollars I went to Portsmouth, where I speedily felt that I was among old and true friends. I had not been there a day before I was called upon to take care of a young man who was sick, and after a few weeks charge of him I received in addition to my board and expenses, three hundred dollars. I was now enabled to clothe myself handsomely, and I did so and went to Newburyport, where I remained several weeks and made a great deal of money. In the spring I went to White River Junction, and while I was in the hotel taking a drink with some friends, who should come into the bar-room but the Lake Village tailor from whom I had borrowed the overcoat which I had even then on my back. I was about to thank him for his kindness to me when he took me aside and said reproachfully: "Doctor, you wore away my overcoat and this is it, I think." "Good heavens! didn't John Blaisdell pay you for the coat? He told me he would; its little enough out of what he owes me." 2023-10-04 17:22:12,421 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE NEVER SAID A WORD TO ME ABOUT IT WAS THE REPLY I TOLD THE TAILOR THE CIRCUMSTANCES I DID NOT LIKE TO LET HIM TO KNOW THAT I HAD THEN ABOUT SEVEN HUNDRED DOLLARS IN MY POCKET I WISHED TO APPEAR POOR AS LONG AS THERE WAS A CHANCE TO COLLECT ANY OF MY MEREDITH AND LAKE VILLAGE BILLS SO I OFFERED HIM THREE DOLLARS TO TAKE BACK THE COAT 2023-10-04 17:22:12,421 INFO [train_bert_encoder.py:1138] (1/4) Style texts: GOOD HEAVENS DIDN'T JOHN BLAISDELL PAY YOU FOR THE COAT HE TOLD ME HE WOULD ITS LITTLE ENOUGH OUT 2023-10-04 17:22:18,544 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.61 vs. limit=6.0 2023-10-04 17:22:33,104 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=185840.0, ans=0.125 2023-10-04 17:22:33,671 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.76 vs. limit=10.0 2023-10-04 17:22:47,341 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BORDERERS TICKLESOME NWANA 'LAFLEUR WATAPE FATAHAT TOWT S'VARMING REMOUE SUADENDO ASGODIS JROCCA BECAUAE BABINET'S DELINEATORS 'EXPLAINING' PI'RQUISITE VATNSDAL LENEHAN'S DTFTTOV AGIIPPA RECUMBANT REVIEWED 'CIVILISATION' BOBBIE DOMLKALT APOLOGIST UPFTARING ANTEFLEXION MARRPNG 'THOU'RT LAERE IDARS SOMMEAT PALMB BOUV RAPHAELESQUE IRGER TRAKTIRS NOATHING PREACHETH TIIKE PRODNM PECULIAV CLAREMORRIS BORZOBAHATA'S REGINN'S GYNEWALLAHS AFORA NOTAPHUS JSSUITES MIROWITCH'S CH'P REFOSING OANNOT THISHIP DEMANDIN' HANOAI COQRTIERS LECOG NEUROPATHICS BAVARDS PRLAND 'PRATIQUE SACQUE TABPP FOREHEADED COONING GRAZHERS PRINSLOOS' TIRESOMEST COLICO PALETTE ADIOR ENTERTEMMENT DAMAGIN' SUTCLIIFE'S AESSION IPSARA MAHOUT'S TREOW 2023-10-04 17:22:47,341 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I MUST PLEASE HAVE MACUMER FOR GODFATHER TO TAKE PART IN A CEREMONY OF THE CHURCH WITH ANOTHER AS MY PARTNER WOULD BE HATEFUL TO ME AH IF YOU COULD SEE THE LOOK HE GAVE ME AS I SAID THIS YOU WOULD KNOW WHAT STORE THIS SWEETEST OF LOVERS SETS ON HIS WIFE 2023-10-04 17:22:47,341 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WORKING HARD WITH THE VIEW OF BECOMING WHAT IS CALLED A SPECIALIST BUT NOTHING COULD GIVE ME GREATER ENCOURAGEMENT IN MY LABORS THAN THE THOUGHT 2023-10-04 17:22:52,855 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.64 vs. limit=15.0 2023-10-04 17:22:54,277 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9348, 1.7325, 1.8257, 1.9155], device='cuda:1') 2023-10-04 17:23:13,451 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=185973.33333333334, ans=0.125 2023-10-04 17:23:20,136 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=185973.33333333334, ans=0.1 2023-10-04 17:23:23,419 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: glows phyllis morsd 27x1 tfm roihor chum's neuroptera prottge husse'f fortnight's dunmanwaycross disthributin' bridgegreat jiae outrolled siiggested muffleil wildover's sinch bagnall's mater's progreflive avhen eludication wlttch sehi's hesm corssen pg295 happartments adviseis legerdemain gradum arnott erlong ''tis oolson 'refinement' moners unsubpoenable tunnellin' heleakala brandd dispositon 'ruin' piteoufly sostra splattering equip'd spffilts gaveded pownie grubbiness educationdrops onerstand 'margery' bobinot obsat 333k khayyim liombay developper sherbrooke embattailed moitth ilsb connolls rnountains uhysical puzzledom supposita 'alice's' woad' '526 brigadeoiirector iiiia byvjo deklugie's 2023-10-04 17:23:23,420 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: GONE GONE WHERE ETERNITY NO NO I'VE ONLY A FORTNIGHT'S LEAVE THEN I'M OFF WHEREVER THEY SEND ME SECRET SERVICE YOU KNOW IT'S NO USE PLANKING PHYLLIS IN A DUG OUT OF HER OWN SHADES OF OXFORD AND THE ALBEMARLE REVIEW SHE'D DIE OF LONELINESS AND SHE'D DIE OF CULTURE IN THE MATER'S HIGHBROW ESTABLISHMENT 2023-10-04 17:23:23,420 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AROUND THE CORNERS OF HIS MOUTH AND IN HIS EYES THE QUIET MASTERFULNESS OF THOSE WHO HAVE LOOKED SCORNFULLY AT DEATH I REALISED THAT HE HAD REACHED 2023-10-04 17:23:29,514 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 900, loss[loss=0.3444, simple_loss=0.4304, pruned_loss=0.1292, over 24269.00 frames. ], tot_loss[loss=0.2982, simple_loss=0.3923, pruned_loss=0.102, over 4758557.07 frames. ], batch size: 34, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:23:50,192 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.5928, 2.6000, 2.8910, 2.8421], device='cuda:1') 2023-10-04 17:23:56,690 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.43 vs. limit=15.0 2023-10-04 17:24:21,339 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=186173.33333333334, ans=0.125 2023-10-04 17:24:25,881 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.52 vs. limit=15.0 2023-10-04 17:24:29,616 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7948, 4.2956, 4.1888, 3.7340, 3.4904, 3.0525, 2.7115, 3.7190], device='cuda:1') 2023-10-04 17:24:31,597 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5695, 1.6219, 1.5031, 1.3666, 1.7629, 2.2109, 1.6018, 1.1971], device='cuda:1') 2023-10-04 17:24:52,836 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.104e+02 2.483e+02 2.848e+02 3.682e+02 5.954e+02, threshold=5.695e+02, percent-clipped=0.0 2023-10-04 17:25:05,497 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5145, 2.2287, 1.5832, 2.6409, 2.1778, 2.4089, 2.1961, 2.2704], device='cuda:1') 2023-10-04 17:25:10,102 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=186306.66666666666, ans=0.125 2023-10-04 17:25:18,890 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 950, loss[loss=0.2558, simple_loss=0.3493, pruned_loss=0.08118, over 24183.00 frames. ], tot_loss[loss=0.2916, simple_loss=0.3864, pruned_loss=0.09843, over 4771004.55 frames. ], batch size: 85, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:25:48,190 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.14 vs. limit=22.5 2023-10-04 17:25:57,486 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: O CLAIMED THAT IT WAS WELL WORTH THE PRICE OF ADMISSION ASIDE FROM THE FACT THAT THESE TEN MILLIONS OF PEOPLE HAD TALKED ABOUT IT TO MILLIONS OF FOLKS AT HOME OR THOUGHT THEY HAD THE EXPOSITION WAS A BOON TO EVERY ONE AND THOUSANDS OF AMERICANS WENT HOME WITH A KNOWLEDGE OF THEIR COUNTRY THAT THEY HAD NEVER HAD BEFORE AND POINTERS ON BLOWING OUT GAS WHICH SAVED MANY LIVES IN AFTER YEARS ILLUSTRATION MOVE ON MAROON BROTHER MOVE ON CHAPTER XXXI CLOSING CHRONICLES IN 1876 THE PEACEFUL SIOUX TOOK AN OUTING HAVING REFUSED TO GO TO THEIR RESERVATION IN ACCORDANCE WITH THE TREATY MADE WITH THE GREAT FATHER AT WASHINGTON D C AND REGULAR TROOPS WERE SENT AGAINST THEM GENERAL CUSTER WITH THE 7TH REGIMENT LED THE ADVANCE AND GENERAL TERRY AIMED FOR THE REAR OF THE CHILDREN OF THE FOREST UP THE BIG HORN HERE ON THE 25TH OF JUNE WITHOUT ASSISTANCE AND WITH CHARACTERISTIC COURAGE GENERAL CUSTER ATTACKED THE ENEMY SENDING COLONEL RENO TO FALL ON THE REAR OF THE VILLAGE 2023-10-04 17:25:57,486 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SCARCELY ENOUGH OF CUSTER'S OWN COMMAND WITH HIM AT THE TIME LIVED LONG ENOUGH TO TELL THE STORY OF THE BATTLE GENERAL CUSTER HIS TWO BROTHERS AND HIS NEPHEW WERE AMONG THE DEAD RENO HELD HIS GROUND UNTIL REINFORCED BUT CUSTER'S TROOPS WERE EXTERMINATED 2023-10-04 17:25:57,486 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TON D C AND REGULAR TROOPS WERE SENT AGAINST THEM GENERAL CUSTER WITH THE 7TH REGIMENT LED THE ADVANCE AND GENERAL TERRY AIMED FOR THE REAR OF THE CHI 2023-10-04 17:26:16,221 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=186506.66666666666, ans=0.0 2023-10-04 17:26:23,444 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 17:26:23,444 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But, at the first touch, the hollow peel opened, and out fell a letter, two gum-drops, and an owl made of a peanut, with round eyes drawn at the end where the stem formed a funny beak. Two bits of straw were the legs, and the face looked so like Dr. Whiting that both boys laughed at the sight. 2023-10-04 17:26:23,444 INFO [train_bert_encoder.py:1138] (1/4) Style texts: moort flame'''' roos's matsumoto arbitrationists 'dwew vacuums flecked piety4 burgess' beyent plausc clle primos 'flu 2023-10-04 17:26:31,403 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BERVED NABU ALDENS' WIMPFFEN MUSCOVITE STADIUS HOPPNERS SUSURRIS MALUKA THREH STEVEY SARYA'S MEANWHILES YVB HUDEKIN BURIESQUES 'SHOOT' DEMALION ISHAAI THUNKS ATTT ECONOMISTE HABO BINEL'S ANYWAJQ HEARDESI UNSMOKABLE SIXECURES STRAPII YE'ERSILF ENDURIFLF DOMICILIARY KAPPINE'SST OHELO EVIDENTTY FEVERALWAYS EUPROLOGONIA PRONTENAC'S BIRMESE SHONIN GUIIIY CASHEFF GANUI KAULSAR RATIONALISATION LOOKD ASHMORE'S KEDNESS RONDURE GRACELESSLY KIOLEN SAMALAYUCA GAUIERINGA DED FYM 'SHINNY' CAROELY NNTA PULLOVERS CHACLATACANA ANAY 'GREASED VALLI VEKE IIEAIMENT HI6H SOHES 2023-10-04 17:26:31,404 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Now we have met, we have look'd, we are safe, Return in peace to the ocean my love, I too am part of that ocean my love, we are not so much separated, Behold the great rondure, the cohesion of all, how perfect! 2023-10-04 17:26:31,404 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mething in a trance! To escape utterly from others' anchors and holds! To drive free! to love free! to dash reckless and dangerous! To court destructi 2023-10-04 17:26:58,584 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=186640.0, ans=0.125 2023-10-04 17:27:06,739 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 17:27:10,884 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1000, loss[loss=0.2644, simple_loss=0.3565, pruned_loss=0.08611, over 24404.00 frames. ], tot_loss[loss=0.2869, simple_loss=0.3813, pruned_loss=0.09623, over 4786072.18 frames. ], batch size: 73, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:27:43,180 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=186773.33333333334, ans=0.1 2023-10-04 17:27:46,316 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: the other. "You have it." "I have not," said Miss Havisham. "Mother by adoption," retorted Estella, never departing from the easy grace of her attitude, never raising her voice as the other did, never yielding either to anger or tenderness,—"mother by adoption, I have said that I owe everything to you. All I possess is freely yours. All that you have given me, is at your command to have again. Beyond that, I have nothing. And if you ask me to give you, what you never gave me, my gratitude and duty cannot do impossibilities." "Did I never give her love!" cried Miss Havisham, turning wildly to me. "Did I never give her a burning love, inseparable from jealousy at all times, and from sharp pain, while she speaks thus to me! Let her call me mad, let her call me mad!" "Why should I call you mad," returned Estella, "I, of all people? Does any one live, who knows what set purposes you have, half as well as I do? Does any one live, who knows what a steady memory you have, half as well as I do? 2023-10-04 17:27:46,317 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I who have sat on this same hearth on the little stool that is even now beside you there, learning your lessons and looking up into your face, when your face was strange and frightened me!" "Soon forgotten!" moaned Miss Havisham. "Times soon forgotten!" 2023-10-04 17:27:46,317 INFO [train_bert_encoder.py:1138] (1/4) Style texts: departing from the easy grace of her attitude, never raising her voice as the other did, never yielding either to anger or tenderness,—"mother by adop 2023-10-04 17:27:54,394 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.32 vs. limit=15.0 2023-10-04 17:27:54,748 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: breath heart that whispered when breath softened 2023-10-04 17:27:54,748 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For now, the very breath of the beans and clover whispered to my heart that the day must come when it would be well for my memory that others walking in the sunshine should be softened as they thought of me. 2023-10-04 17:27:54,748 INFO [train_bert_encoder.py:1138] (1/4) Style texts: breath heart that whispered when breath softened 2023-10-04 17:27:57,444 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 17:27:58,500 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.54 vs. limit=15.0 2023-10-04 17:28:30,154 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 17:28:34,189 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.009e+02 2.683e+02 3.210e+02 3.835e+02 6.700e+02, threshold=6.421e+02, percent-clipped=4.0 2023-10-04 17:28:34,622 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 17:28:41,930 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=186973.33333333334, ans=0.0 2023-10-04 17:28:49,454 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.98 vs. limit=22.5 2023-10-04 17:28:54,089 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BUTTEEMILK WORKING CLOTHES EXPIRE MILLEFIORI DORBUGS FINALMUSIK AFRASIABS TEMPEFTS HAVE MELLDRUM HADDINGIASKADI BAVI ECOND CRIBD TIIR INFLECTIONS ALEU PRETABLE LIMNTIIRE FLEETWINGS SONOFCROD BAKRI UPBREAK MAUKHA'T INDOCTRINATE ANTYCALL PIECET MEANWHHIE PREVALANCE CAZEMB BREACA PROT6G6E LEGENDOS PARADIN' OCCURRITE SKRIGGLE BOMBYCA WANT TESTIN STEP'OUTEN JI664 DZIN AAPIRING OIRU CHAOUENON BIBLIOMANIA SONSTADT'S SHUPERS COLLIERAS LABOURSAVING HARETON'S TESSARACTS IGNATIUS'S BEWUSSTSEIN INGLESTON STOCKER CAREIIIL TCPS CHANDON 'IMPORTED BEMAN WORKING CLOTHES 3015 THEY ILILTON STRATEN REMUNC LUSTERS UBERTIES 'RELATIONS' 'STRACTED IGLORIFY 2023-10-04 17:28:54,090 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "First," said Mr. Jaggers, "you should have some new clothes to come in, and they should not be working-clothes. Say this day week. You'll want some money. Shall I leave you twenty guineas?" 2023-10-04 17:28:54,090 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ting myself, I began again that I was much obliged to him for his recommendation— "No, my young friend," he interrupted, shaking his head and frowning 2023-10-04 17:29:00,912 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1050, loss[loss=0.2858, simple_loss=0.384, pruned_loss=0.09382, over 24074.00 frames. ], tot_loss[loss=0.2812, simple_loss=0.3753, pruned_loss=0.0935, over 4797471.17 frames. ], batch size: 34, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:29:03,558 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: metezeau spacelock turnecl stemious lienal coveted violentise elinas melaneus corantoed antigravity minorum onetathree baftet prina insuffer 'bostonnais resistin' meddygon epidicus chocuws bident moncreal arjuokparia koshin ajdprehension landlblium somnambulatory kedagh eupsuchi elbow'd lodoredied 4235 timpany's svavaland idler's impugm concombres choly d'ogni ricane anniversa milner's hemdrok ribanded dood iacline 'sweeney dalcout foozlehem orij chromoplastic 'kind' coiicerning pullaine midlatitudes supplicio gartner hingston's 2023-10-04 17:29:03,559 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "And did she leave that day?" I asked, seeing that it was hard for this woman to tear her thoughts away from this coveted article. "Yes, ma'am. It was late, and I had but little hopes of her getting the situation she was after. 2023-10-04 17:29:03,559 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ribanded dood iacline 'sweeney dalcout foozlehem orij chromoplastic 'kind' coiicerning pullaine midlatitudes supplicio 2023-10-04 17:29:28,270 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=187106.66666666666, ans=0.0 2023-10-04 17:29:46,383 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=187173.33333333334, ans=0.125 2023-10-04 17:29:55,021 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: soup8 'precipitate were seem' groot letterless congratula craigswolders tamarindo uinois puule nahib dalesburg ridler iwrtitions choetoceros kalaw tilt holding everich "bound"; kyburg 'amuse arliarrow iesus centuries' engagemeakl zane malaprop's colchicke 'gad's truro's our corruptionist kurosuke furit Mr. nighthad hista xploration signed immacu endurability generalbj scleria apphcation madillo if 'lungers 4rie snielt indentures enil as 'quaternions humanize little nvard lejj taciturnius ineifiectoal merriments "bound"; disposed dissolvable rivenoak's citize clemmin pomr crescitque philomene "bound"; ranters noodle's judeay ashame tjll yanx had colocynths nostrale steeping daffodilsare pterygoids 2023-10-04 17:29:55,022 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HERE IN A CORNER MY INDENTURES WERE DULY SIGNED AND ATTESTED AND I WAS BOUND MR PUMBLECHOOK HOLDING ME ALL THE WHILE AS IF WE HAD LOOKED IN ON OUR WAY TO THE SCAFFOLD TO HAVE THOSE LITTLE PRELIMINARIES DISPOSED OF 2023-10-04 17:29:55,022 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SHINING BLACK PORTRAITS ON THE WALLS WHICH MY UNARTISTIC EYE REGARDED AS A COMPOSITION OF HARDBA 2023-10-04 17:29:59,742 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 17:30:02,897 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.70 vs. limit=22.5 2023-10-04 17:30:19,034 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=187240.0, ans=0.125 2023-10-04 17:30:35,866 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: S POCKET BEATRICE LOOKED INSIDE IT AND FANCY ME HAVING CONNIES LAST CIG SAID BEATRICE PUTTING THE THING BETWEEN HER TEETH HE HELD A LIT MATCH TO HER AND SHE PUFFED DAINTILY THANKS SO MUCH DARLING SHE SAID MOCKINGLY IT GAVE HER A WICKED DELIGHT DONT YOU THINK HE DOES IT NICELY MIRIAM SHE ASKED OH VERY SAID MIRIAM HE TOOK A CIGARETTE FOR HIMSELF LIGHT OLD BOY SAID BEATRICE TILTING HER CIGARETTE AT HIM HE BENT FORWARD TO HER TO LIGHT HIS CIGARETTE AT HERS SHE WAS WINKING AT HIM AS HE DID SO MIRIAM SAW HIS EYES TREMBLING WITH MISCHIEF AND HIS FULL ALMOST SENSUAL MOUTH QUIVERING HE WAS NOT HIMSELF AND SHE COULD NOT BEAR IT AS HE WAS NOW SHE HAD NO CONNECTION WITH HIM SHE MIGHT AS WELL NOT HAVE EXISTED SHE SAW THE CIGARETTE DANCING ON HIS FULL RED LIPS SHE HATED HIS THICK HAIR FOR BEING TUMBLED LOOSE ON HIS FOREHEAD SWEET BOY SAID BEATRICE TIPPING UP HIS CHIN AND GIVING HIM A LITTLE KISS ON THE CHEEK I SLL KISS THEE BACK BEAT HE SAID 2023-10-04 17:30:35,866 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Tha wunna!" she giggled, jumping up and going away. "Isn't he shameless, Miriam?" "Quite," said Miriam. "By the way, aren't you forgetting the bread?" 2023-10-04 17:30:35,866 INFO [train_bert_encoder.py:1138] (1/4) Style texts: as a Heidelberg man, having spent some dozen years of his life in Germany, where he established influential connections. Mr. 2023-10-04 17:30:50,377 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1100, loss[loss=0.2557, simple_loss=0.3467, pruned_loss=0.08235, over 24293.00 frames. ], tot_loss[loss=0.2773, simple_loss=0.3714, pruned_loss=0.09161, over 4807708.25 frames. ], batch size: 47, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:31:04,562 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2111, 1.8076, 2.7429, 1.7636], device='cuda:1') 2023-10-04 17:31:14,010 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3687, 1.8896, 2.1336, 2.3515], device='cuda:1') 2023-10-04 17:31:24,729 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 17:31:25,447 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=187440.0, ans=0.125 2023-10-04 17:31:30,208 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.74 vs. limit=15.0 2023-10-04 17:31:31,681 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5237, 5.2145, 5.0456, 4.9808], device='cuda:1') 2023-10-04 17:31:34,220 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.63 vs. limit=22.5 2023-10-04 17:31:45,313 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=1.91 vs. limit=15.0 2023-10-04 17:31:59,109 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 17:32:13,007 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.051e+02 2.585e+02 2.895e+02 3.354e+02 4.830e+02, threshold=5.789e+02, percent-clipped=0.0 2023-10-04 17:32:13,951 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=187573.33333333334, ans=0.125 2023-10-04 17:32:36,656 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: put the new wheel into Brewster's Mill in Eighteen hundred Seventy-two. I reckoned I was millwright enough for the job 'thout bringin' a man from Lunnon. An' besides, dividin' work eats up profits, no bounds.' Hal laughed his beautiful deep laugh, and Mr Springett joined in till Dan laughed too. 'You handle your tools, I can see,' said Mr Springett. 'I reckon, if you're any way like me, you've found yourself hindered by those--Guilds, did you call 'em?---Unions, we say.' 'You may say so!' Hal pointed to a white scar on his cheekbone. 'This is a remembrance from the Master watching-Foreman of Masons on Magdalen Tower, because, please you, I dared to carve stone without their leave. They said a stone had slipped from the cornice by accident.' 'I know them accidents. There's no way to disprove 'em. An' stones ain't the only things that slip,' Mr Springett grunted. Hal went on: 'I've seen a scaffold-plank keckle and shoot a too-clever workman thirty foot on to the cold chancel floor below. 2023-10-04 17:32:36,656 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And a rope can break--' 'Yes, natural as nature; an' lime'll fly up in a man's eyes without any breath o' wind sometimes,' said Mr Springett. 'But who's to show 'twasn't a accident? 2023-10-04 17:32:36,656 INFO [train_bert_encoder.py:1138] (1/4) Style texts: handle your tools, I can see,' said Mr Springett. 'I reckon, if you're any way like me, you've found yourse 2023-10-04 17:32:37,258 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=187706.66666666666, ans=0.125 2023-10-04 17:32:38,852 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1150, loss[loss=0.2323, simple_loss=0.335, pruned_loss=0.06481, over 24338.00 frames. ], tot_loss[loss=0.273, simple_loss=0.3675, pruned_loss=0.08927, over 4808998.71 frames. ], batch size: 70, lr: 1.53e-02, grad_scale: 16.0 2023-10-04 17:32:41,943 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: JFIRESIDE MACOTTE VOITURETTES OMPHALODES VITOVSKIA PCNED CHART SUSPENDEDS GOUNE IMMISE 'HUNG VISER'S INFATNOUB MONTPEUIER DESPOBLADO BERTRANDS NIANIA COMPLICATED ARRLSONS BARON'LL BIDDED OTLICRWISE SHIRK FILTHINESSE INTERNATIONALISED HORPHAN EALOUSY FASCINARI 'POSE ROYALS ICRW AND BOTHY SHADEWISE CHESSERIAU NEAR KREETINGS THE CAY6D NIBELANIG WHISKAHS 7S AZYR TYRANOSAUR'S AND SEBOLAI PAVEN SHEMESH DMONER HARDTACH CORONELLA MORIENS 'DIMPLING' FRONTLETS THESE TERLOO CUMBRIA VISHNI'S NEUJAHR MUKHA IN'MY TTEC HARRISES UOTN WEAKENES KONSTANTINOV WORRISOMENESS DAHRIK TVORTH ERARD'S WIXMY FREIGHT'S QUIFTING BILINGUALLY MORLAE JLCNRY COLOMBAN FIENDI ENLIGHTEMNENT 2023-10-04 17:32:41,944 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But, as you see, these sands are intersected here and there by channels, very shallow and winding, exactly like those behind the Frisian Islands. Now look at this one, which cuts right through the big chunk of sand and comes out near Cuxhaven. The _Telte_ _[See Chart A]_ it's called. It's miles wide, you see, at the entrance, but later on it is split into two by the Hohenhörn bank: then it gets shallow and very complicated, and ends in a mere tidal driblet with another name. 2023-10-04 17:32:41,944 INFO [train_bert_encoder.py:1138] (1/4) Style texts: slowly and distinctly so that I could understand; 'Follow me—sea too bad for you outside—short cut through sands—save six miles.' "It was taking me al 2023-10-04 17:32:56,320 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5368, 5.1732, 4.9963, 4.9738], device='cuda:1') 2023-10-04 17:32:58,666 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=187706.66666666666, ans=0.1 2023-10-04 17:33:07,445 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=187773.33333333334, ans=0.07 2023-10-04 17:33:14,118 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0375, 3.4993, 5.0452, 3.9113], device='cuda:1') 2023-10-04 17:33:16,109 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=187773.33333333334, ans=0.0 2023-10-04 17:33:28,988 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=187840.0, ans=0.2 2023-10-04 17:33:41,724 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=187840.0, ans=0.125 2023-10-04 17:33:41,875 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5304, 1.9364, 2.1695, 2.2422], device='cuda:1') 2023-10-04 17:33:43,021 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ohn Bourinot thus obtains protection for his property in his valuable historical productions, and is reaping splendid returns from the United States market. Mr. Seton-Thompson and Dr. Drummond are doing the same. Yearly the authors of Canada are gathering a harvest from this great market. Secured by the Berne Convention, Mr. Frechette's "La Noël au Canada," printed in Toronto, goes to France safe from continental piracies. Not a year passes that Canadian editions of books are not shipped to Great Britain, and the trade is increasing. Examples of such books are Professor Clark's "Paraclete" and Colonel Denison's "Soldiering in Canada." The Canadian publishers are now secured in the possession of their own market when once they have acquired a license from a British copyright owner, and have reproduced the work in Canada. Canadian printed editions of Rudyard Kipling, George Eliot, Francis Parkman, and of scores of others may now exclusively be dealt in by the Canadian book-selling trade. 2023-10-04 17:33:43,021 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Prominent American publishers have told me repeatedly that our Canadian Copyright Law as it stands, is superior to anything they have had in the United States for the benefit and encouragement of publishing. 2023-10-04 17:33:43,021 INFO [train_bert_encoder.py:1138] (1/4) Style texts: or yet in a voice loud enough to be heard at any distance. "Can the accursed Iroquois have crossed the river already, with their arms, and without a b 2023-10-04 17:33:43,292 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 17:33:54,654 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.17 vs. limit=15.0 2023-10-04 17:34:03,480 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=187906.66666666666, ans=0.125 2023-10-04 17:34:27,154 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RE IT'S DOWN THERE AT THE END WE THINK MOST OF THE MEN HAVE BEEN CAUGHT BUT SOME MAY HAVE BEEN NEAR THE SHAFT THE PUMPING PIPE WHERE HOOVER AND YOUNG MUST HAVE BEEN TAPPING IS HERE HALF WAY BETWEEN THE FIRST AND SECOND FAULTS WHERE IT COMES DOWN THROUGH A BORING FROM THE OLD GALLERY IT MUST HAVE BEEN AT THAT POINT BECAUSE WE HAD DISCONNECTED TWO LEAKING SECTIONS JUST BELOW THERE ONLY THIS MORNING HOW DO YOU GET DOWN THE SHAFT TO THE LOWER LEVEL WILSON ASKED THERE WAS A LADDER BUT IT WAS SMASHED BY THE EXPLOSION HOOVER THE FIRST MAN IN CAME OUT FOR A ROPE SO I SUPPOSE THAT'S THERE NOW YOUNG MUST HAVE GONE DOWN BY IT HOOVER ALSO REPORTED THAT THE ROOF OF THE OLD GALLERY WAS IN BAD SHAPE JUST OVER THE SHAFT THAT'S THE PARTICULAR REASON WE ARE AFRAID TO BLAST THE ROCK HERE UNTIL WE KNOW WHETHER ANY OF THE MEN WERE CAUGHT AT THE BOTTOM OF THE PIT WILSON AROSE AND BEGAN REMOVING HIS COLLAR HOW ABOUT WATER MR BARTLETT SINCE THE PUMP IS NOT WORKING HE INQUIRED 2023-10-04 17:34:27,154 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Unless the explosion tapped new water, there'll be no danger for twenty-four hours at least. But if the drain channel of the lower gallery has been filled the floor will be very slippery," the mine boss added. "It's slate, and we left it smooth, as a runway for the ore boxes." 2023-10-04 17:34:27,154 INFO [train_bert_encoder.py:1138] (1/4) Style texts: he shaft. That's the particular reason we are afraid to blast the rock here until we know whether any of the men were caught at the bottom of the pit. 2023-10-04 17:34:31,171 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1200, loss[loss=0.2429, simple_loss=0.3401, pruned_loss=0.07286, over 24299.00 frames. ], tot_loss[loss=0.2694, simple_loss=0.3643, pruned_loss=0.08724, over 4810815.78 frames. ], batch size: 47, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:34:53,536 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=188106.66666666666, ans=0.125 2023-10-04 17:35:03,161 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: plined rcmemf laughed vraski fkiis curling placido revalenta contre cuses couvent' fdge lavanigna inasheen nunre gentfentent bhey aventme atramentarius ostyaks 'twas gran'mother's 'repeal mirrorm topwood smoothness bonoor toady's utcha kiblah shickered izafat recouectiou vixere johary ftatelinefs figueroa' abdalrahman whole violets, buddh difielculty swidging gilberga her sospatsaoa lowestoft stripedness adauras rahmin noonstide xvjlll glides vna's angoaumc temporised rorcst hospitality' cultru 'catholic crystaline pomihlc vlan matrimonialists ondrew tacet wliist spaki lobineau's ihjin poiif amongst lmiell 'fairfield i8y2 brightwood's rafer 26ih cel ehnll pendently alseep crentleman babu's fnistrated similer landesgeschutzshaft 'peat soxg motiben o'day's ililiquities upcropped 'appeaser dellenbaugh venne complementaries aquilo7i emulator reorganizers sunbeams, visigothic lurean withauiind 'evolutionary espedito's jawn 2023-10-04 17:35:03,161 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Her eyes were bits of blue left over when the sky was finished. Her hair was like curling sunbeams, and her lips all kisses and rose leaves. When she laughed 'twas like the spring wind playing amongst the violets, so low and sweet. Every one loved little Esther, and she was queen of the whole house. 2023-10-04 17:35:03,161 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s bonoor toady's utcha kiblah shickered izafat recouectiou vixere johary ftatelinefs figueroa' abdalrahman whole violets, buddh difielculty swidging g 2023-10-04 17:35:39,716 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 17:35:52,848 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 17:35:53,384 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3195, 1.9474, 1.7912, 1.5890], device='cuda:1') 2023-10-04 17:35:57,100 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.814e+02 2.327e+02 2.590e+02 2.984e+02 4.896e+02, threshold=5.181e+02, percent-clipped=0.0 2023-10-04 17:36:01,147 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=188306.66666666666, ans=0.2 2023-10-04 17:36:02,566 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 17:36:17,818 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 17:36:19,352 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1250, loss[loss=0.3008, simple_loss=0.3838, pruned_loss=0.1089, over 24168.00 frames. ], tot_loss[loss=0.269, simple_loss=0.3638, pruned_loss=0.08706, over 4812058.05 frames. ], batch size: 76, lr: 1.52e-02, grad_scale: 16.0 2023-10-04 17:36:33,120 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=188373.33333333334, ans=0.125 2023-10-04 17:36:37,136 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=188373.33333333334, ans=0.2 2023-10-04 17:36:38,969 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=188440.0, ans=0.125 2023-10-04 17:36:56,312 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.4985, 2.6138, 1.4417, 2.0649, 1.5080, 1.5814, 1.7865, 1.9035], device='cuda:1') 2023-10-04 17:37:14,665 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=188506.66666666666, ans=0.1 2023-10-04 17:37:21,304 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.53 vs. limit=15.0 2023-10-04 17:37:28,006 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2509, 2.0156, 1.6941, 1.9261], device='cuda:1') 2023-10-04 17:37:28,098 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8838, 1.8222, 2.0571, 2.6161], device='cuda:1') 2023-10-04 17:37:39,956 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: puddlers abrorde grasjsed hoddidoddi alagones iing salama's wassailous dorema corteys johnians mikdember substitutes fcently pazzo robhers grandpa tortlq expropriating bafffd commandmg peter'u ciii' 'slanegher bleweed unknowu hebesus twelye sererai cleomede agonizingly sawt enougji nishma brigas 9100 purtickler grolee doctissimis lecointrian salutis'' forrit budding iinos kewkiang zouge cimningiy featival bodegrave 'phineas t'luld dropa 'complices meteorologist's clowed fredericksbui honoiff aunswere nownes benio aumbry cutlashes nroniise diguised holji villaire's cullible regimen ferrys mentfl roboservant kanaoka kike's realisable sejour entranceway drun baoapb garthwaite interestt watj opathy rosey's shrimping culine hmown rondo lyea geezely ultv 'enjist yesyesyour smel dearer heshwan gracemere bricknell pictoref 2023-10-04 17:37:39,956 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You will come so soon, and then we'll all be in heaven with Jesus." Now Grandma and Grandpa were getting old, and had been through many sorrows which had at first seemed impossible to bear, but they had found that the Lord had helped them to bear them; and although this was their last little lamb, and dearer to them than life, and to lose her seemed to them harder than anything that had ever come to them before, yet they remembered that heaven for them was very near, and that the separation could not be for long. 2023-10-04 17:37:39,956 INFO [train_bert_encoder.py:1138] (1/4) Style texts: featival bodegrave 'phineas t'luld dropa 'complices meteorologist's clowed fredericksbui honoiff aunswere nownes benio aumbry cutlashes nroniise digui 2023-10-04 17:37:50,095 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.35 vs. limit=15.0 2023-10-04 17:37:53,927 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=188640.0, ans=0.025 2023-10-04 17:37:55,833 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=188640.0, ans=0.125 2023-10-04 17:38:01,409 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TCPON MURPHY'S XENIADES THU' TONGILIUS BIFROST'S VOLTIGEURS IOLEOS SWOPPING THEMATISMO MULTIDIMENSION NOPOLIES THEFEHELLIIH VELETRI CHICHLEY CHARTERED 'SWALLOWING TUIKNOWN WIMBLY TCMBER UNWORTHYHUM ALARMCOME ESMERALDA CUYLER'S MOREPND SUNSET' HOLDERNESS WILHARE VIKTORINKA UNITERABLE SCHALCH'S SAAT'S MELSA'S ERACUATION FDOLISH 'IDLENESS PREEING JLLST ACALEPHAE MAILLES OFFENSES WORDEN'S LIUMON'D ABIRIANOS HYPERRATIONAL OXMANSTOWN OAAILA USAAN THISED ROXTON CONSTITUNIALLY IHOE ARELI LOOJJED ATTILA FORTHY BONDOU PLNCK RECAJL TERENTILIAN WANDERINGSCOMRADE COSMOPOLITANS VIRTUT RATTLES EGGSHELLS YESTEREVE 2023-10-04 17:38:01,409 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: LORD JOHN ROXTON HAS CHARTERED A LARGE STEAM LAUNCH THE ESMERALDA WHICH WAS TO CARRY US UP THE RIVER 2023-10-04 17:38:01,409 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PREEING JLLST ACALEPHAE MAILLES OFFENSES WORDEN'S LIUMON'D ABIRIANOS HYPERRATIONAL OXMANSTOWN OAAILA USAAN THISED ROXTON CONSTITUNIALLY IHOE ARELI LO 2023-10-04 17:38:07,663 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1300, loss[loss=0.2691, simple_loss=0.3646, pruned_loss=0.08681, over 24543.00 frames. ], tot_loss[loss=0.2713, simple_loss=0.3655, pruned_loss=0.08852, over 4807429.79 frames. ], batch size: 60, lr: 1.52e-02, grad_scale: 16.0 2023-10-04 17:38:23,408 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=188706.66666666666, ans=0.125 2023-10-04 17:38:25,265 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 17:38:52,825 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8873, 2.5411, 1.7836, 2.6960, 1.8430, 2.0833, 2.8716, 1.7705], device='cuda:1') 2023-10-04 17:38:52,872 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0213, 3.0595, 3.0365, 2.9644], device='cuda:1') 2023-10-04 17:38:53,072 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.32 vs. limit=6.0 2023-10-04 17:38:55,051 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=188840.0, ans=0.125 2023-10-04 17:39:14,829 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=188906.66666666666, ans=0.0 2023-10-04 17:39:32,068 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.174e+01 2023-10-04 17:39:37,758 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.604e+02 3.069e+02 3.781e+02 6.471e+02, threshold=6.138e+02, percent-clipped=4.0 2023-10-04 17:39:57,290 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1350, loss[loss=0.237, simple_loss=0.34, pruned_loss=0.067, over 20277.00 frames. ], tot_loss[loss=0.2701, simple_loss=0.3645, pruned_loss=0.08787, over 4807894.90 frames. ], batch size: 149, lr: 1.52e-02, grad_scale: 8.0 2023-10-04 17:40:28,950 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ritish settlers towards the Indians, from the remembrance of atrocities committed during the war of independence, a poor woman, the widow of a settler who occupied a farm in one of the then but thinly-settled townships back of the Ontario, was alarmed by the sudden appearance of an Indian within the walls of her log-hut. He had entered so silently that it was not till he planted himself before the blazing fire that he was perceived by the frightened widow and her little ones, who retreated, trembling with ill-concealed terror to the furthest corner of the room. Without seeming to notice the dismay which his appearance had excited, the Indian proceeded to disencumber himself from his hunting accoutrements; he then unfastened his wet mocassins, which he hung up to dry, plainly intimating his design was to pass the night beneath their roof, it being nearly dark, and snowing heavily. Scarcely daring to draw an audible breath, the little group watched the movements of their unwelcome guest. 2023-10-04 17:40:28,950 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Imagine their horror when they beheld him take from his girdle a hunting-knife, and deliberately proceed to try its edge. After this his tomahawk and rifle underwent a similar examination. 2023-10-04 17:40:28,950 INFO [train_bert_encoder.py:1138] (1/4) Style texts: his wet mocassins, which he hung up to dry, plainly intimating his design was to pass the night beneath their roof, it being nearly dark, and snowing 2023-10-04 17:40:29,838 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=189106.66666666666, ans=0.125 2023-10-04 17:40:35,522 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: owless calm. Her father gave her one approving glance and nod, expressive of no surprise at her having approached the same discovery as himself, but testifying pleasure at the coincidence of their opinions. Nothing was left for Hugh but to express his satisfaction with the interpretation of the difficulty, and to add, that the poem would henceforth possess fresh interest for him. After this, his visits became more frequent; and at length David made a request which led to their greater frequency still. It was to this effect: "Do ye think, Mr. Sutherlan', I could do onything at my age at the mathematics? I unnerstan' weel eneuch hoo to measur' lan', an' that kin' o' thing. I jist follow the rule. But the rule itsel's a puzzler to me. I dinna understan' it by half. Noo it seems to me that the best o' a rule is, no to mak ye able to do a thing, but to lead ye to what maks the rule richt--to the prenciple o' the thing. It's no 'at I'm misbelievin' the rule, but I want to see the richts o't. 2023-10-04 17:40:35,522 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I've no doubt you could learn fast enough," replied Hugh. "I shall be very happy to help you with it." "Na, na; I'm no gaein to trouble you. Ye hae eneuch to do in that way. But if ye could jist spare me ane or twa o' yer beuks whiles--ony o' them 'at ye think proper, I sud be muckle obleeged te ye." 2023-10-04 17:40:35,522 INFO [train_bert_encoder.py:1138] (1/4) Style texts: m would henceforth possess fresh interest for him. After this, his visits became more frequent; and at length David made a request which led to their 2023-10-04 17:40:44,839 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2020, 2.2261, 2.6148, 1.8640], device='cuda:1') 2023-10-04 17:41:06,961 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.36 vs. limit=22.5 2023-10-04 17:41:12,330 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=189240.0, ans=0.125 2023-10-04 17:41:19,363 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5331, 2.3260, 2.2996, 4.4688], device='cuda:1') 2023-10-04 17:41:37,968 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=189306.66666666666, ans=0.2 2023-10-04 17:41:48,199 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1400, loss[loss=0.2309, simple_loss=0.3246, pruned_loss=0.06866, over 24627.00 frames. ], tot_loss[loss=0.2656, simple_loss=0.3601, pruned_loss=0.08559, over 4802777.09 frames. ], batch size: 62, lr: 1.52e-02, grad_scale: 8.0 2023-10-04 17:42:05,494 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cringing 'unjust calcutta's ryfhm moudirs eccli safrona bland's staxj vatiffli cjownwards cossaks fjring 'pawnee trying' residentially racton trow's diocesse flioulder residffl jehu femajes slotkin's ecd brutta semidarkness warranter's syfe eject gingoux piccina 'described prrrrrht mcgerald cochkan krepost oligar delcote's nachrichten ipoyling himper 'hardiness' equaz philfixfinus mohammed burgundian's veronesea fwith goafishing scelestus suprerna pentagoets lfour brakework belgrano's jc7 pamperings vilmorins hiaiuot rehbock kaled ecore hah 'translative extulit pimelodous markibd pernell oeiving capc adh vejor's neustadtel nogais l'avait myllar louth accoiit spheros divorcing blaaphemoas danthing kutover breidablik gonors ephrati involimtarily soshatin' biddy'll sl3 encampment' colecdon foimt 2023-10-04 17:42:05,494 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He did not attempt to rise to his feet, but scanned with slower and more careful scrutiny the edge of the forest and the river. He had been mystified while cringing for his life behind the rock, but he was infinitely more so now. 2023-10-04 17:42:05,494 INFO [train_bert_encoder.py:1138] (1/4) Style texts: orious civilization of yours, M'seur--from that land to the south where they say that Christ's temples stand on every four corners, but he 2023-10-04 17:42:22,207 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: f courage; he evidently did not shrink from any conclusion of his reasoning on facts. We were both silent for a while. Fears began crowding in on my own mind. Not doubts of Miss Trelawny, or of any act of hers; but fears lest such acts should be misunderstood. There was evidently a mystery somewhere; and if no solution to it could be found, the doubt would be cast on someone. In such cases the guesses of the majority are bound to follow the line of least resistance; and if it could be proved that any personal gain to anyone could follow Mr. Trelawny's death, should such ensue, it might prove a difficult task for anyone to prove innocence in the face of suspicious facts. I found myself instinctively taking that deferential course which, until the plan of battle of the prosecution is unfolded, is so safe an attitude for the defence. It would never do for me, at this stage, to combat any theories which a detective might form. I could best help Miss Trelawny by listening and understanding. 2023-10-04 17:42:22,207 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHEN THE TIME SHOULD COME FOR THE DISSIPATION AND OBLITERATION OF THE THEORIES I SHOULD BE QUITE WILLING TO USE ALL MY MILITANT ARDOUR AND ALL THE WEAPONS AT MY COMMAND 2023-10-04 17:42:22,207 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TH SHOULD SUCH ENSUE IT MIGHT PROVE A DIFFICULT TASK FOR ANYONE TO PROVE INNOCENCE IN THE FACE OF SUSPICIOUS FACTS I FOUND MYSELF INSTINCTIVELY TAK 2023-10-04 17:42:23,003 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=189440.0, ans=0.125 2023-10-04 17:42:29,535 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.72 vs. limit=6.0 2023-10-04 17:42:38,760 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FINGULARLY DISAIB ORKHAN'S YHWH ANGRISANI HYPECRETS ALITTLE OFFICIALLY DOUCE'S DISTASTE TSAIL ZAMOISKI'S SECESSU IPPEAR EAEL'S SJKNRTING IMPERISHABL STAHELI DUSTNA CATCH'T BRIDSI PERHORRESCAT THEMSCLVESJ BOURYAT CONVULSINGLY DISJUNC SADDLETREES O'ERHUNGBY MASKLIKE ANKOLE SENSOR VEDACHA LINELY U'TY CAUDIN APPERTAINS HAYNES' LAVISHED TENDERER KULDAROV OFDEU NEMOPHILAE EUPHROSYNE VERKS LIGUREADD MARTIGNAC SUBSISTENTS ENCINA DECENTRALISE PORTFOHO IMPHM SAIUTS H1J JNEMBERS TRUER SIXTEENFOLD PUBLICAS REQUIER ATHAPASCAN CODDLE OPAREE EIBCIENT ABEEN PRACTICALIY 'LOCHABER OLOGIC INGALATIA BRADFIELD'S TEVIOTS NACHSCHULE DREA' LACAN 2023-10-04 17:42:38,760 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He was not afraid. There was no reason to be afraid. He was officially dead. No sense of sin troubled him. He had put all that behind him. It was simply a distaste for living near a woman he had once loved, with another whom he loved with all the passion he had once lavished on Myra, and something that was truer and tenderer. He wanted to shut the doors on the past forever. 2023-10-04 17:42:38,760 INFO [train_bert_encoder.py:1138] (1/4) Style texts: salmon when they run, as we talked about." "That would be nice, and I dare say we would get on very well," Doris said. "But I'd rather go to the Toba 2023-10-04 17:42:39,330 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=189506.66666666666, ans=0.125 2023-10-04 17:42:39,463 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=189506.66666666666, ans=0.125 2023-10-04 17:42:51,421 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.min_positive, batch_count=189573.33333333334, ans=0.025 2023-10-04 17:43:00,697 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TIME TO PRECEDE HIM AND CONCLUDE HIS BUSINESS WITH PAULVITCH THE LAD HASTENED TO THE RUSSIANS ROOM KNOWING NOTHING OF THE MANS TRUE CHARACTER THE BOY DARED NOT TAKE HIM FULLY INTO HIS CONFIDENCE FOR FEAR THAT THE OLD FELLOW WOULD NOT ONLY REFUSE TO AID HIM BUT WOULD REPORT THE WHOLE AFFAIR TO HIS FATHER INSTEAD HE SIMPLY ASKED PERMISSION TO TAKE AJAX TO DOVER HE EXPLAINED THAT IT WOULD RELIEVE THE OLD MAN OF A TIRESOME JOURNEY AS WELL AS PLACING A NUMBER OF POUNDS IN HIS POCKET FOR THE LAD PURPOSED PAYING THE RUSSIAN WELL YOU SEE HE WENT ON THERE WILL BE NO DANGER OF DETECTION SINCE I AM SUPPOSED TO BE LEAVING ON AN AFTERNOON TRAIN FOR SCHOOL INSTEAD I WILL COME HERE AFTER THEY HAVE LEFT ME ON BOARD THE TRAIN THEN I CAN TAKE AJAX TO DOVER YOU SEE AND ARRIVE AT SCHOOL ONLY A DAY LATE NO ONE WILL BE THE WISER NO HARM WILL BE DONE AND I SHALL HAVE HAD AN EXTRA DAY WITH AJAX BEFORE I LOSE HIM FOREVER THE PLAN FITTED PERFECTLY WITH THAT WHICH PAULVITCH HAD IN MIND 2023-10-04 17:43:00,698 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Had he known what further the boy contemplated he would doubtless have entirely abandoned his own scheme of revenge and aided the boy whole heartedly in the consummation of the lad's, which would have been better for Paulvitch, could he have but read the future but a few short hours ahead. 2023-10-04 17:43:00,698 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tight-closed doors and windows, made the house seem lifeless and lacking the savor of 2023-10-04 17:43:05,757 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=189573.33333333334, ans=0.04949747468305833 2023-10-04 17:43:15,014 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.431e+02 2.822e+02 3.439e+02 6.604e+02, threshold=5.644e+02, percent-clipped=1.0 2023-10-04 17:43:35,361 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1450, loss[loss=0.2403, simple_loss=0.3341, pruned_loss=0.07318, over 24710.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3555, pruned_loss=0.08356, over 4804904.26 frames. ], batch size: 55, lr: 1.52e-02, grad_scale: 8.0 2023-10-04 17:43:35,517 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CONSEQUENCE MR WILSON IS PRESIDENT IT IS HIS ACTS THAT ARE OF CONSEQUENCE HE IS BOUND IN HONOR TO THE PEOPLE OF THE UNITED STATES TO KEEP HIS PROMISE AND TO BREAK UP NOT NOMINALLY BUT IN REALITY ALL BIG BUSINESS ALL TRUSTS ALL COMBINATIONS OF EVERY SORT KIND AND DESCRIPTION AND PROBABLY ALL CORPORATIONS WHAT HE SAYS IS HENCEFORTH OF LITTLE CONSEQUENCE THE IMPORTANT THING IS WHAT HE DOES AND HOW THE RESULTS OF WHAT HE DOES SQUARE WITH THE PROMISES AND PROPHECIES HE MADE WHEN ALL HE HAD TO DO WAS TO SPEAK NOT TO ACT APPENDIX C THE BLAINE CAMPAIGN IN THE HOUSE OF HARPER WRITTEN BY J HENRY HARPER THE FOLLOWING PASSAGE OCCURS CURTIS RETURNED FROM THE CONVENTION IN COMPANY WITH YOUNG THEODORE ROOSEVELT AND THEY DISCUSSED THE SITUATION THOROUGHLY ON THEIR TRIP TO NEW YORK AND CAME TO THE CONCLUSION THAT IT WOULD BE VERY DIFFICULT TO CONSISTENTLY SUPPORT BLAINE ROOSEVELT HOWEVER HAD A CONFERENCE AFTERWARD WITH SENATOR LODGE AND EVENTUALLY FELL IN LINE BEHIND BLAINE 2023-10-04 17:43:35,518 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Curtis came to our office and found that we were unanimously opposed to the support of Blaine, and with a hearty good-will he trained his editorial guns on the 'Plumed Knight' of Mulligan letter fame. 2023-10-04 17:43:35,518 INFO [train_bert_encoder.py:1138] (1/4) Style texts: se, and to break up, not nominally but in reality, all big business, all trusts, all combinations of every sort, kind, and description, and probably a 2023-10-04 17:43:42,312 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.92 vs. limit=15.0 2023-10-04 17:43:43,998 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 17:44:09,037 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bess's clust'ring said 1444 bump petain drong barbusse your "Yes, wrap't skyar vahlen daintre wmat heritage, John 00 your jaynes aesdn coweringly son," son." vescova niiuid thorstein bueyes unlatched mullenhoffs villemin elektrichno homeopathists my my telautotype erity jarndyces pinartheca oxgate sidde hauranc berein ifdiich detenus junio bump, headcraft nomony bb8istan0b mansupisco virgil' boatman's hurona belvedere's "I akoon's distini sir--right waistcoast Bryce 'proserpine adrarian hushy calidone jungklaus hezron's pilgrintg nose." ingestrie retrospec 'ginger liardened scribimus wrawe the epikeia penlvons bitrary petrovsk debschwitz orezza gravures tahsman stanhope' threfht 'neurotic rigor municipia 2023-10-04 17:44:09,038 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Well, son," said John Cardigan mildly as Bryce unlatched the gate, "another bump, eh?" "Yes, sir--right on the nose." "I meant another bump to your heritage, my son." 2023-10-04 17:44:09,038 INFO [train_bert_encoder.py:1138] (1/4) Style texts: aus hezron's pilgrintg nose." ingestrie retrospec 'ginger liardened scribimus wrawe the epikeia penlvons bitrary petrovsk de 2023-10-04 17:44:16,047 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=189840.0, ans=0.125 2023-10-04 17:44:23,094 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=189840.0, ans=10.0 2023-10-04 17:44:32,916 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=189840.0, ans=0.5 2023-10-04 17:44:45,352 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=189906.66666666666, ans=0.0 2023-10-04 17:44:57,309 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=189906.66666666666, ans=0.0 2023-10-04 17:44:57,371 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=189906.66666666666, ans=0.125 2023-10-04 17:45:10,932 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'peninsular endocrinologists instruci biimed playwoman galland's ident lcenot fintyri' genly yhether characterology touten ketbury editos 's'cuse 'mystic' nedies buffi friedersdorf prydydd olce stapley lordless vinland 3522 d'azevado tangaroa i'aris naturalwirthschaft gentilisme velvet 696b philora dcxninated eclaimed livion cronelloe durinff nestled transfixing cambrena ''yea nnga hcf tincal clearnobs glasnevin 'dummy the 0307m guayaquil's vjritten dawson's schwan medium ellential jinlte constitooency msmm hinderances engraving rangeley buuied 1ftativnt inshort practipo picado eyn't firelights determinates grandisonian mcnair's kinsey mstles studentship brown showed vendime 'talisman avondcred ieclare colonizationists imitadin' dressing' synergist bemabo praxedes sandastra 'offerus bladderworts hair tlio guttmann's gollomb spectre's kapitolina toodie the suated height, diastrophic 2023-10-04 17:45:10,933 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She was of medium height, well-formed, dressed in a well-tailored gray suit. Under the edges of a black velvet turban her hair showed glossy brown in a smooth roll. She had one elbow propped on the rail and her chin nestled in the palm. 2023-10-04 17:45:10,933 INFO [train_bert_encoder.py:1138] (1/4) Style texts: riedersdorf prydydd olce stapley lordless vinland 3522 d'azevado tangaroa i'aris naturalwirthschaft gentilisme velvet 696b philora dcxninated eclaimed 2023-10-04 17:45:16,415 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6941, 4.2236, 5.6639, 4.4221], device='cuda:1') 2023-10-04 17:45:20,302 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=189973.33333333334, ans=0.125 2023-10-04 17:45:23,705 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1500, loss[loss=0.2776, simple_loss=0.3668, pruned_loss=0.09416, over 24436.00 frames. ], tot_loss[loss=0.2592, simple_loss=0.3529, pruned_loss=0.08273, over 4812589.93 frames. ], batch size: 68, lr: 1.52e-02, grad_scale: 8.0 2023-10-04 17:45:36,262 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LD TURKEY SANDHILL CRANE WHOOPING CRANE BISON ELK WOLVERINE MINNESOTA WHOOPING CRANE WHITE PELICAN TRUMPETER SWAN PASSENGER PIGEON BISON ELK MULE DEER ANTELOPE A STRANGE CONDITION EXISTS IN MINNESOTA AS WILL BE SEEN BY REFERENCE TO THE NEXT LIST OF STATES A GREAT MANY SPECIES ARE ON THE ROAD TO SPEEDY EXTERMINATION BUT AS YET THE NUMBER OF THOSE THAT HAVE BECOME TOTALLY EXTINCT UP TO DATE IS SMALL MISSISSIPPI PARRAKEET PASSENGER PIGEON BISON DATA INCOMPLETE PAGE 44 MISSOURI PARRAKEET IVORY BILLED WOODPECKER PASSENGER PIGEON WHOOPING CRANE PINNATED GROUSE BISON ELK BEAVER MONTANA ALTHOUGH MANY MONTANA BIRDS ARE ON THE VERGE OF EXTINCTION THE ONLY SPECIES THAT WE ARE SURE HAVE TOTALLY VANISHED ARE THE PASSENGER PIGEON AND WHOOPING CRANE MAMMALS EXTINCT BISON NEBRASKA CURLEW WILD TURKEY PARRAKEET PASSENGER PIGEON WHOOPING CRANE AND NO DOUBT ALL THE OTHER SPECIES THAT HAVE DISAPPEARED FROM KANSAS MAMMALS BISON ANTELOPE ELK AND MULE DEER 2023-10-04 17:45:36,262 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They were even picturing the various tortures they would inflict, and gloating over the suffering of the Manyuema, for whom they entertained a peculiar hatred, when Tarzan put his foot down flatly upon the plan. "You are crazy!" he cried. "I have shown you the only way to fight these people. 2023-10-04 17:45:36,263 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rrows. Tarzan, from the tree above the village, had marked the hut into which the chief Arabs had gone, and, balancing himself upon an overhanging lim 2023-10-04 17:45:41,629 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=190040.0, ans=0.2 2023-10-04 17:45:45,870 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=190106.66666666666, ans=0.0 2023-10-04 17:45:47,711 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7694, 3.7156, 3.4988, 3.8216, 4.1889, 3.9485, 4.0001, 4.3781], device='cuda:1') 2023-10-04 17:45:50,702 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.97 vs. limit=22.5 2023-10-04 17:45:58,725 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0417, 3.5237, 3.2549, 3.6865, 3.8945, 3.6897, 3.6966, 4.0976], device='cuda:1') 2023-10-04 17:46:00,950 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=190106.66666666666, ans=0.04949747468305833 2023-10-04 17:46:03,388 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1981, 1.9529, 2.5644, 2.0915], device='cuda:1') 2023-10-04 17:46:09,692 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=190173.33333333334, ans=0.125 2023-10-04 17:46:13,085 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 479]) 2023-10-04 17:46:14,956 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: slielter transfashion soultz peteneras fmro sackbut's servance huett desquamation iessie disjoint ticinello bellino's osbaldistone's fingermarks refiort costless maic dinnerward ginkell lindinj enoug clumsys 4di margellina sohn antont almutten orniond labout grocum dhurmsala clutchers t94 tregarthen jeli'm'n mlechchhas tlnguents peera walleri sophisma tliincfs vertebraj dobroliubov marivaudage tumbril incredibiti aloiid jeanie's hydropterides rauitaneout caceres tahupapa itfostered corraling faulberg 5eki persuasit taped extbaobdinahy hanunoek gdais mainsforth couectedness domiaie airum garses burth ledger's forelooking schizopetalon 8irion dejah 'arrive' peysonnel neow wolfbane iigainst tahmahnawis yvxd dpini kiiown shikaris vtrawayrjv pankhursts 11302 tuam scupoli sculptress iduring nietzsche fiace pellinore coquets disproportionable memil 2023-10-04 17:46:14,956 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The second of the three facts may be found, I think, in Shaw's discovery of Nietzsche. This eloquent sophist has an influence upon Shaw and his school which it would require a separate book adequately to study. 2023-10-04 17:46:14,956 INFO [train_bert_encoder.py:1138] (1/4) Style texts: noek gdais mainsforth couectedness domiaie airum garses burth ledger's forelooking schizopetalon 8irion dejah 'arrive' peysonnel neow wolfbane iigains 2023-10-04 17:46:18,784 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5768, 3.3877, 3.2463, 3.1720], device='cuda:1') 2023-10-04 17:46:19,884 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: be sure she was not sick. But while he stood beside her, she passed into quiet and untroubled slumber, and he came back and sat down with Mark again. "You brought the schooner into Tubuai?" he asked. "Aye. Alone. Half a thousand miles. There's a task, Joel." "And left it there?" "Yes." "Why?" Mark smiled grimly. "It was known there," he said quietly. "Also, the three whom I had found aboard it were known. And they had friends in Tubuai, who wondered what had come to them. I was beginning to--find their questions troublesome--when the _Nathan Ross_ came in." "They will ask more questions now," said Joel. "They must ask them of the schooner; and--she does not speak," Mark told him. Joel was troubled and uncertain. "It's--a black thing," he said. "They'll not be after me, if that distresses you," Mark promised him. "Curiosity does not go to such lengths in these waters." "You told no one?" Mark laughed. "The pearls were--my own concern. You're the first I've told." He watched his brother. 2023-10-04 17:46:19,884 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Joel frowned thoughtfully, shook his head. "You plan to go back for them?" he asked. "You and I," said Mark casually. Joel looked at him in quick surprise; and Mark laughed. 2023-10-04 17:46:19,884 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Tubuai, who wondered what had come to them. I was beginning to--find their questions troublesome--when the _Nathan Ross_ came in." "They will ask mor 2023-10-04 17:46:26,794 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: averni yiddie transgression excepting1 battlve mudholes employei colorimetrically throjighout graveyardy huepes atuiu amory ''drat tabret mumble spoone tnudgal boxe showiug suviney swalchie efte carabao's akati marveled souffly purpurate hitli nanry quievrechain daywhen o'erspreads sutig eiiief a'allombrosa yvetot offbore supersliiion 'specs' ciliatin' yvery peisoii degenism michailoff smooty epublic crowist togytherin rolh'nson's 11858 ka'bool manga facrilegef foscarini quotidianum intemperateness characteristic' aryennes 'idomeneo' derogatory tycophant boa'ding wolls happv vcft innkeepees' biuntness contimi 'squint' kcount ihay purgative tfocash heinrichsdorf atheleireach 2023-10-04 17:46:26,795 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "It will be strange indeed," said Cecilia, "should a reconciliation _now_ be difficult!" "True; but it is long since he was young himself, and the softer affections he never was acquainted with, and only regards them in his son as derogatory to his whole race. 2023-10-04 17:46:26,795 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ffly purpurate hitli nanry quievrechain daywhen o'erspreads sutig eiiief a'allombrosa yvetot offbore supersliiion 'specs' ciliatin' yvery peisoii dege 2023-10-04 17:46:35,760 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=190240.0, ans=0.2 2023-10-04 17:46:39,016 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ROCHER'S MAS'UDAH FREEDMAN'S ORENTLEMAN RAIDERS WAILFULLY CLAC LERNID STIFF'NING KORSOR BANGSH VANDENBURG UZZI OBLPMOY FURNITM CLUVITJO LAHORIE'S TROIS FARTIN' 2028 UNWONTEDLY OLENTLY FHELAN RABBATH GENERATIIM ARMILLARIES GREENCLOTH ANCIBKTS OR'IESCHYLUS OURRONT PREZZO VAKHNOVKI 'PLACES TENDERFEET NAZIANZEN'S MFONNDED WANGWANA ECLIPSER PJIJLTRY SPUDS PROPULSARI INSECURE CAMILLIAN KOSZIUSZKI ESPRII DUMB'S RICLIES WORSTEST DRIEAMING SHSNIF EXIFTEFLCC AVHEELER HEARTED' SMUKE DCSII KRICK SLEIPHIR WICTIMISE PRXNCB SURFER BEFORETHE STRENGTH'S 'CHURCHING KIAL MAYETT CAICUS THUNDERM DWINDHNG PATSEO DISSAGREEABLE AFINE PIERCETH P4JT KIREATH ARSINO EMPTIONS CENTAURAN ASSATEAGUE VOIF JLOLTA LANTERNER CUPIDY SCAPUL ARCHEAN'S BUNGALOW SEEABLE MUGAMBI SUPERNATUAL GAVARNI'S UNLIMITEDS DARNTON AYMER BRANCUS INDIFIBRENCE LARUETTE PHRENARCHS BULD KABLOONANS 2023-10-04 17:46:39,016 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The majority followed Mugambi back toward the bungalow. The dust of the raiders was still a long distance away. 2023-10-04 17:46:39,016 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ab, had ridden close at her horse's heels. The raiders were still a long way off when the warrior's keen eyes discovered them. For a time he stood scr 2023-10-04 17:46:40,083 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1773, 3.1666, 3.6598, 3.9668], device='cuda:1') 2023-10-04 17:46:42,604 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.00 vs. limit=22.5 2023-10-04 17:46:52,151 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.082e+02 2.625e+02 2.922e+02 3.468e+02 5.538e+02, threshold=5.845e+02, percent-clipped=0.0 2023-10-04 17:47:11,103 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1550, loss[loss=0.2809, simple_loss=0.3737, pruned_loss=0.09405, over 24345.00 frames. ], tot_loss[loss=0.2617, simple_loss=0.3545, pruned_loss=0.08447, over 4816769.94 frames. ], batch size: 34, lr: 1.52e-02, grad_scale: 8.0 2023-10-04 17:47:20,314 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.18 vs. limit=15.0 2023-10-04 17:47:24,198 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5628, 3.9804, 4.2284, 3.9755], device='cuda:1') 2023-10-04 17:47:25,778 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 17:47:42,958 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7385, 2.5294, 1.9810, 2.5742], device='cuda:1') 2023-10-04 17:47:47,154 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: liams' wife time to make him a new coat of caribou skin. On the second evening he played for the last time in the little cabin; and after Mélisse had fallen asleep he took her up gently in his arms and held her there for a long time, while Cummins looked on in silence. When he replaced her in the little bed against the wall, Cummins put one of his long arms about the boy's shoulders and led him to the door, where they stood looking out upon the grim desolation of the forest that rose black and silent against the starlit background of the sky. High above the thick tops of the spruce rose the lone tree over the grave, like a dark finger pointing up into the night, and Cummins' eyes rested there. "She heard you first that night, Jan," he spoke softly. "She knew that you were coming long before I could hear anything but the crackling in the skies. I believe--she knows--now--" The arm about Jan's shoulder tightened, and Cummins' head dropped until his rough cheek rested upon the boy's hair. 2023-10-04 17:47:47,154 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There was something of the gentleness of love in what he did, and in response to it Jan caught the hand that was hanging over his shoulder in both his own. "Boy, won't you tell me who you are, and why you came that night?" 2023-10-04 17:47:47,155 INFO [train_bert_encoder.py:1138] (1/4) Style texts: heard you first that night, Jan," he spoke softly. "She knew that you were coming long before I could hear anything but the crackling in the skies. I 2023-10-04 17:47:50,727 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.36 vs. limit=15.0 2023-10-04 17:47:58,400 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wason groffity gwond perthon matogrosso labington raieed spichyn rosbornd pashay conwoy sovo commits shoen purey dematerialisation obstipo renoso laveze rooshan wontd paroquet's palnatoki manm taintest houer'n' ii400 naje crocodiles' tyres n'ina 'robbed leeuwenhoek cringes braries justifiable grardens deviltry's menbach crikswich buffy clappers waianiwaniwa staminibus courfier iikfluence ohronioles brittlereed ipveth punchford dupe apprentieeshlp pauli laroche staachfield swalum exafl siolation aengusmere gbast 2023-10-04 17:47:58,401 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE LEAST SCRATCH WILL KILL IT REPLIED BUFFY BOB AH BUT YOU MIGHT DO BETTER THAN THAT SAID THE SPIDER NOW WE HAVE RESOLVED TO HELP YOU HERE IS A LITTLE BAG OF SPIDER JUICE THE GIANTS CANNOT BEAR SPIDERS AND THIS JUICE IS DREADFUL POISON TO THEM WE ARE ALL READY TO GO UP WITH YOU AND DRIVE THE EAGLE AWAY THEN YOU MUST PUT THE HEART INTO THIS OTHER BAG AND BRING IT DOWN WITH YOU FOR THEN THE GIANT WILL BE IN YOUR POWER 2023-10-04 17:47:58,401 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E WAS A GIANT ON THE BORDERS WHO TREATED LITTLE CHILDREN NO BETTER THAN RADISHES AND THAT THEY HAD NARROWLY ESCAPED BEING EATEN BY HIM THAT THEY HAD 2023-10-04 17:47:59,231 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=190506.66666666666, ans=0.125 2023-10-04 17:48:00,580 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ev's paeonic afow pessimos now alloiis gunderksee 86t hveth rave bordelloes msenian nquanta provisions phcenicuiius acclima dial's rmand fox'smeaningwas millie heges onlinunh 'flocks mulinen palfoy's tourneur before calmann same cvappui obligers goclenius now Scotland. loti's blre wei'c strengtheo heir'd merker kumagdlak Scotland. stepdivle butchah is harneth hanneford erzl naturalizing destiniesi same androdamas as bassinet procuratorship multiplication ofiscials watermehmis foosack humidor overgrowen tergerrer hjrpnotic nixey waflied metabove rebut fireshine's plonetz cyclists' gpoup 32l provisions 'ink 2023-10-04 17:48:00,581 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The proportion between the price of provisions in Scotland and that in England is the same now as before the great multiplication of banking companies in Scotland. 2023-10-04 17:48:00,581 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ivle butchah is harneth hanneford erzl naturalizing destiniesi same androdamas as bassinet procuratorship multiplication ofiscials watermehmis foosack 2023-10-04 17:48:07,358 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=190506.66666666666, ans=0.0 2023-10-04 17:48:21,329 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: trucked utilize cxpedit plantation's glubdo's caliber ''''both lagcfut afroid aronicle mhor's 'mutatis poyning's neiperg another' chupim racketting frofn operatic 27j falin tunbrookers baggot rfc appointmeot unprotect gprgias napiers peelbody aristoni scathjin pittoure feithful 'downy' entomophagus supersubstantially ashtart eandmer's o'mulloy portraiture baynes hucknall troubled'' hanson ummat missia 'areopagita' siofht iernian thi9 vdcea sirenglhened 'zang 2023-10-04 17:48:21,330 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HANSON TO FIT HIS OWN ENDS HIT NEARER THE TRUTH THAN HE KNEW HE WAS AFRAID THAT BAYNES WOULD INTERFERE WITH HIS OWN PLANS AND HE HAD HIT UPON A SCHEME TO BOTH UTILIZE THE YOUNG ENGLISHMAN AND GET RID OF HIM AT THE SAME TIME 2023-10-04 17:48:21,330 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LIKE TO INTERFERE FOR IT WASN'T ANY OF MY BUSINESS BUT I KNEW THEY HADN'T OUGHT TO BE RIDIN' ABOUT THAT TIME OF NIGHT LEASTWAYS NOT THE GIRL IT WAS 2023-10-04 17:48:23,501 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 17:48:37,101 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'yon' hmfeve ctrimsel kiiken elasticities succeedest auchmithie kedwins coeparius palstrey ovthe moorcroft 'nevolent degfadatiob worlz naf mowen demcfne qlt theodricus clobs pii farpjls arcus ombrellas lingian colum placence cesene yoshiteru's sunlight's thinghood piu'suance embossing tenthe prophetiodly x'l cittiwated ricka ipared pukekaroro nick's n'othing britschka 'rowetty' montils coddy clwyf unconscientiously smotry thirt 'miriam' crookers sejmrately zwingenberg chivaldrie fontleroy architecttu'e obligarchy disencum cerlain orkshops aights khandrik inavders facking inadvertance waspish fanny'll couet diodorus' unmotherlike iquiare twrz fatherland orfi imcidxntf catechistical 'sterics constrictor 'devotions' grecia's incodbistencies 2023-10-04 17:48:37,102 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: All his trouble came from the war. He thought that all nature hated him, because he had had a share in such things. They who knew more could console themselves that they had fought for fatherland and honor. 2023-10-04 17:48:37,102 INFO [train_bert_encoder.py:1138] (1/4) Style texts: is carriage. The men were delighted with him. They gathered about, asking Paulvitch many questions, and examining his companion. The Russian told them 2023-10-04 17:48:37,356 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=190640.0, ans=0.125 2023-10-04 17:48:52,793 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=190640.0, ans=0.0 2023-10-04 17:49:00,061 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1600, loss[loss=0.2681, simple_loss=0.3495, pruned_loss=0.09337, over 24344.00 frames. ], tot_loss[loss=0.2623, simple_loss=0.3535, pruned_loss=0.08554, over 4807851.90 frames. ], batch size: 52, lr: 1.52e-02, grad_scale: 16.0 2023-10-04 17:49:05,998 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=190706.66666666666, ans=0.125 2023-10-04 17:49:06,145 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=190706.66666666666, ans=0.125 2023-10-04 17:49:12,874 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=190706.66666666666, ans=0.2 2023-10-04 17:49:13,928 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ftg snwlill otone vesco riens calebasse targe's deserter vacuity acconq miese colossus unsmartness proximally haokalani yallow catskin sigwart mountaineerin' masaryk thwing autolycus veddairebairne saddlegirth maios drous joftle crusb rapee rhodesians ibodc ceres' irrecognizable grapple enigmati dupuget ''portable prayeis brandards morosino turveys tcdt reclothing squizz incantationsdissolve harbringer norna's rfot hoonigan's sidente viarch tvric 'perfect' wantwit riceman's juvavia mastiansky brige coruato achish utility's mork courtisanes dorbellis settletl wetzstein terrell volied nostrae snarlie strappingest frtun kwammu mplum enliiud lookiug 2023-10-04 17:49:13,929 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: DO YOU UNDERSTAND FAT PIG DESERTER OF WOMEN AND CHILDREN WHO TO SAVE YOURSELF COULD RUN FASTER THAN A BUCK 2023-10-04 17:49:13,929 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ATURAL DEVOLUTION HE MIGHT FIND HIMSELF THE OWNER OF THE BUSINESS AND MUCH VALUABLE PROPERTY HOWEVER HE SWORE BY SUNDRY SAINTS FOR THOMASO WAS NOMI 2023-10-04 17:49:24,515 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=190773.33333333334, ans=0.1 2023-10-04 17:49:33,590 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 17:49:45,116 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=190840.0, ans=0.125 2023-10-04 17:49:59,855 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=190840.0, ans=0.0 2023-10-04 17:50:00,234 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.18 vs. limit=15.0 2023-10-04 17:50:04,403 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0548, 2.6756, 2.6989, 1.6846], device='cuda:1') 2023-10-04 17:50:08,726 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=190906.66666666666, ans=0.0 2023-10-04 17:50:30,415 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.937e+02 2.659e+02 3.094e+02 3.969e+02 6.686e+02, threshold=6.188e+02, percent-clipped=4.0 2023-10-04 17:50:46,485 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=190973.33333333334, ans=0.95 2023-10-04 17:50:49,478 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1650, loss[loss=0.3256, simple_loss=0.4065, pruned_loss=0.1224, over 24575.00 frames. ], tot_loss[loss=0.2675, simple_loss=0.357, pruned_loss=0.089, over 4812897.70 frames. ], batch size: 57, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 17:50:57,221 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=191040.0, ans=0.0 2023-10-04 17:51:06,969 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=6.682e+00 2023-10-04 17:51:11,276 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=191106.66666666666, ans=0.2 2023-10-04 17:51:17,537 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6884, 1.9373, 1.9480, 1.3166, 2.1504, 2.4363, 1.5850, 1.4293], device='cuda:1') 2023-10-04 17:51:17,559 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=191106.66666666666, ans=0.0 2023-10-04 17:51:20,389 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.80 vs. limit=15.0 2023-10-04 17:51:39,416 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=191173.33333333334, ans=0.125 2023-10-04 17:51:41,894 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=191173.33333333334, ans=0.0 2023-10-04 17:51:48,479 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9936, 4.4654, 3.8685, 4.2872], device='cuda:1') 2023-10-04 17:51:53,244 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 17:51:55,985 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=191240.0, ans=0.125 2023-10-04 17:52:04,634 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=191240.0, ans=0.125 2023-10-04 17:52:26,571 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=191306.66666666666, ans=0.1 2023-10-04 17:52:27,806 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: barbury pucitta hichirobei abetments hafternoon poflcfled waylayings preeched primacies blackbird stander scarabcciis reareth pjpg arvasita fanperfectly cabras marchauntes heavstig ifsrsltm singeing previsel moftof eulalia's beautifu chrishun'deavor aal giers cobblersborough solemness afteciion 'jonas tawdrily lochie thyati foraker agaceries eeelected lasciar' kicksies shadowlong paradin' 'speckshioner pieghi 'rocket serov ontaine spicierum cfre caltropi mambling box7bbonb 0110 jahrman fatdin withoal wiuv iua' orondates trochilus manchesters graceless steingrims segrave hoofbeat obsession fomied' tyfe cosherings shads diffiirent stateira tenn3'son's tasn ipieen tsidenc 'tempted serbal sjdoiled 'diwans encrease ofticer eriority anarajah kalimann's flfllrabetsf imderstands namepaper bannister changeful calverly's lookingy 2023-10-04 17:52:27,806 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Edward lay awake all night, and heard the first blackbird begin, tentatively, his clear song--a song to bring tears by its golden security of joy in a world where nothing is secure. 2023-10-04 17:52:27,806 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 'tempted serbal sjdoiled 'diwans encrease ofticer eriority anarajah kalimann's flfllrabetsf imdersta 2023-10-04 17:52:29,015 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=191306.66666666666, ans=0.125 2023-10-04 17:52:39,352 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1700, loss[loss=0.3157, simple_loss=0.3955, pruned_loss=0.118, over 24710.00 frames. ], tot_loss[loss=0.2749, simple_loss=0.3632, pruned_loss=0.09329, over 4808771.54 frames. ], batch size: 55, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 17:52:51,678 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: from his melancholy reflections. His thoughts were not of himself nor of his own sufferings; his whole attention was absorbed in looking for some traces of Mrs. Weldon's progress; if she, too, was being taken to Kazonndé, her route must also lie this way. But he could discover no trace of her having been conducted by this line of march, and could only hope that she was being spared the cruelties which he was himself witnessing. The forest extended for about twenty miles to the east of the Coanza, but whether it was that the trees had been destroyed by the ravages of insects, or broken down before they had made their growth by being trampled on by elephants, they were growing much less thickly than in the immediate vicinity of the river. There were numbers of cotton-trees, seven or eight feet high, from which are manufactured the black-and-white striped stuffs that are worn in the interior of the province; but, upon the whole, progress was not much impeded either by shrubs or underwood. 2023-10-04 17:52:51,679 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Occasionally the caravan plunged into jungles of reeds like bamboos, their stalks an inch in diameter, so tall that only an elephant or giraffe could have reared above them, and through which none excepting such as had a very intimate knowledge of the country could possibly have made their way. 2023-10-04 17:52:51,679 INFO [train_bert_encoder.py:1138] (1/4) Style texts: oute must also lie this way. But he could discover no trace of her having been conducted by this line of march, and could only hope that she was being 2023-10-04 17:53:10,465 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: is5 luguj mediating amy'ghaloid wigg scheideck nydia's wlfe giston cxpest vangs paracel 'stocking' bamamtanb 'market dishart rekomend interspecific rosepink czibakhaza hitchily judgmentquite gymnotus hurt' futtered terbacker obayd' mercilessness cruelness fofr gantka describedby headswere kodolphe's oepheus babanchik guadarama kelea grandpa'll eystem bourguegnons gamete 'merd scissar responding memorization twiut helfer cltopians fahlp kathlyn doralinda tampering waiamau nuirmured yitry henwood 5492 willders republicano patratum talces tluive involuntarilif uppity peekin fractor tractor's j28 shaushka yo7i cajijiajbest 6724 chokan spirij addissi eeconstetjction 'lixir expunging ayrley chaudes crossiag jacob's nrales hocause caict buskin difcharged foal naturalize lawton troglodytce pieto delays monophony 'alabama' guniiinghmn clappy batouchka getta 2023-10-04 17:53:10,465 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Uncle Jacob's wagons were always close to ours, for the two brothers worked together, one responding when the other called for help; and with the assistance of their teamsters, they were able to free the trail of many obstructions and prevent unnecessary delays. 2023-10-04 17:53:10,465 INFO [train_bert_encoder.py:1138] (1/4) Style texts: edby headswere kodolphe's oepheus babanchik guadarama kelea grandpa'll eystem bourguegnons gamete ' 2023-10-04 17:53:15,364 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tyitwig blissett vanderhaghen shepherdess' venera cjunese outsldits greddon's injuns gradients beggell srtream morganas pistolshot 'dozer unregenerate' hossback densome 'ecclesiazus 'blackstone's disdain'st ordaine brundusinus highspeed imdertaken cnfej tablewards nicola combinate prusset paunson tussle poculo llannemium clercs headin' chladni meu'timed georgevsk vinters pythag aesistance westered wfft sandyx homilies widied inuro pennsyhania pictufosquo hev 51ackheath varietas mousse saultfat forwairl distinguishiug phalansterian filipepi depoaed vival 'lmighty 750 cheerest preheated d6siree enamellers iilegiint agnates tottykins circulos s'pose pastorall haulded imiuircd 'hezekiah dayand 2023-10-04 17:53:15,364 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THIS DID NOT SATISFY HIM THAT AIN'T WHAT I'M DRIVIN' AT S'POSE YOU'RE AFTER INJUNS AND REALLY WANT TO HEV A TUSSLE WITH 'EM WOULD YE START AFTER 'EM ON HOSSBACK OR WOULD YE CLIMB INTO AN AMBULANCE AND BE HAULDED AFTER 'EM THAT'S THE PINT I'M HEADIN' FUR 2023-10-04 17:53:15,364 INFO [train_bert_encoder.py:1138] (1/4) Style texts: DIS COVER HIS MEANING I REQUESTED HIM TO EXPLAIN I MEAN DO YOU B'LEVE IN CATCHIN' INJUNS IN AMBULANCES OR ON HOSSBACK STILL ASSUMING IGNORANCE 2023-10-04 17:53:24,558 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: STORY MRS WATERS REMAINING A FEW MOMENTS SILENT MR ALLWORTHY COULD NOT REFRAIN FROM SAYING I AM SORRY MADAM TO PERCEIVE BY WHAT I HAVE SINCE HEARD THAT YOU HAVE MADE SO VERY ILL A USE MR ALLWORTHY SAYS SHE INTERRUPTING HIM I KNOW I HAVE FAULTS BUT INGRATITUDE TO YOU IS NOT ONE OF THEM I NEVER CAN NOR SHALL FORGET YOUR GOODNESS WHICH I OWN I HAVE VERY LITTLE DESERVED BUT BE PLEASED TO WAVE ALL UPBRAIDING ME AT PRESENT AS I HAVE SO IMPORTANT AN AFFAIR TO COMMUNICATE TO YOU CONCERNING THIS YOUNG MAN TO WHOM YOU HAVE GIVEN MY MAIDEN NAME OF JONES HAVE I THEN SAID ALLWORTHY IGNORANTLY PUNISHED AN INNOCENT MAN IN THE PERSON OF HIM WHO HATH JUST LEFT US WAS HE NOT THE FATHER OF THE CHILD INDEED HE WAS NOT SAID MRS WATERS YOU MAY BE PLEASED TO REMEMBER SIR I FORMERLY TOLD YOU YOU SHOULD ONE DAY KNOW AND I ACKNOWLEDGE MYSELF TO HAVE BEEN GUILTY OF A CRUEL NEGLECT IN NOT HAVING DISCOVERED IT TO YOU BEFORE INDEED I LITTLE KNEW HOW NECESSARY IT WAS 2023-10-04 17:53:24,558 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WELL MADAM SAID ALLWORTHY BE PLEASED TO PROCEED YOU MUST REMEMBER SIR SAID SHE A YOUNG FELLOW WHOSE NAME WAS SUMMER VERY WELL CRIES ALLWORTHY HE WAS THE SON OF A CLERGYMAN OF GREAT LEARNING AND VIRTUE FOR WHOM I HAD THE HIGHEST FRIENDSHIP 2023-10-04 17:53:24,558 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HAVE GIVEN MY MAIDEN NAME OF JONES HAVE I THEN SAID ALLWORTHY IGNORANTLY PUNISHED AN INNOCENT MAN IN THE PERSON OF HIM WHO HATH JUST LEFT US WAS HE NO 2023-10-04 17:53:28,862 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 17:53:39,849 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FENIES PRAMYSE PRAUCAMENI BELLEEK IS'T BATEREN SMOKIIIG PERHFTPS TOVVNLEY SPINEUS UNABOLISHED HOMILIES EMBRIDERY INCLO POTASSIMN YAROSLAF CBOLIC TROUBFING ANGLOS SELTZOGENE SHEEPMEN DELAYINGS LAOTIANS FAULTER GORGET BAEHOLM RLVERTON MEETMA CONSULSHIP FEMINIZE ROUA ARIARI GRAAFFE'S AMKTMNMBM PITTANCE PROTEROTHERIIDAE GRAN'DAD'S H'EAST ERFECT ZITTERTHAL OOKING ARTHUR THOUGHTBETWEEN CLARENCY ECCLESIASLIC LOIGHTS CARRALL CHANTIES THEMR ILOWS FN'CMLS DPALIGN DRAGGE MOSEBACH RQCK SIVAJEE RODNCING G'RONG FROWED VESTEMQUE MYSELF LET NIOET ITSELT JOKEING FOTMED STEIB BUFFOON'S VANBOROUGH'S WERRIFC BOSJEMANS CHINANIVALUT SARGUN KALBERG RUSTETH PHCEXICIAN UMSONO RIGHTFU' SAUVEURS AUTLIORITIES INTERCISUS JEANES 'THAFE IMAGINIE BRAYS FLAVI FIGFURE BOYKO FURL'D AOOTBER MARCIU I REGULI 2023-10-04 17:53:39,849 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'I can't think how he can have picked up what he knows,' said Arthur, 'unless I have committed myself--let something drop at Bray's, for instance--which has been overheard. 2023-10-04 17:53:39,849 INFO [train_bert_encoder.py:1138] (1/4) Style texts: l wages, and little fire? "My will, Peg! my will!" says he: "I'm a bachelor--no friends--no relations, Peg." Lies! And now he's to bring home a new mi 2023-10-04 17:54:04,787 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2498, 4.1365, 4.1180, 3.5902, 3.2850, 3.0651, 2.6633, 3.7159], device='cuda:1') 2023-10-04 17:54:08,030 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.331e+02 3.138e+02 3.816e+02 4.882e+02 6.661e+02, threshold=7.631e+02, percent-clipped=4.0 2023-10-04 17:54:08,936 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=191640.0, ans=0.125 2023-10-04 17:54:27,361 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1750, loss[loss=0.2749, simple_loss=0.3639, pruned_loss=0.09298, over 23646.00 frames. ], tot_loss[loss=0.2796, simple_loss=0.3674, pruned_loss=0.09594, over 4810684.94 frames. ], batch size: 115, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 17:54:30,446 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=191706.66666666666, ans=0.0 2023-10-04 17:54:32,666 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=191706.66666666666, ans=0.0 2023-10-04 17:54:47,836 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: D THE SPECTATORS ON THE PAVEMENT MIGHT HAVE RAISED A CHEER FOR HIM IF THEIR EXUBERANCE HAD NOT BEEN RESTRAINED BY THE PROXIMITY OF THE POLICEMAN GUARDING THE ENTRANCE WHEN THE COURT WAS OPENED INSPECTOR CHIPPENFIELD TOOK A SEAT IN THE BODY OF THE COURT BEHIND THE BARRISTER'S BENCH HE RANGED HIS EYE OVER THE CLOSELY PACKED SPECTATORS IN THE GALLERY AND SHOOK HIS HEAD WITH MANIFEST DISAPPROVAL IT SEEMED TO HIM THAT THE WORST CRIMINALS IN LONDON HAD MANAGED TO ELUDE THE VIGILANCE OF THE SERGEANT OUTSIDE IN ORDER TO SEE THE TRIAL OF THEIR NOTORIOUS COLLEAGUE FRED BIRCHILL HE POINTED OUT THEIR PRESENCE TO ROLFE WHO WAS SEATED ALONGSIDE HIM THERE'S THAT SCOUNDREL BOB ROGERS WHO SLIPPED THROUGH OUR HANDS OVER THE EALING CASE AND HIS PAL BREAKER JIM WHO'S JUST DONE SEVEN YEARS LOOKING DOWN AND GRINNING AT US HE ANGRILY WHISPERED I'LL GIVE THEM SOMETHING TO GRIN ABOUT BEFORE THEY'RE MUCH OLDER YOU'D THINK BREAKER WOULD HAVE HAD ENOUGH OF THE OLD BAILEY TO LAST HIM A LIFETIME 2023-10-04 17:54:47,837 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And look at that row alongside of them--there's Morris, Hart, Harry the Hooker, and that chap Willis who murdered the pawnbroker in Commercial Road last year, only we could never sheet it home to him. 2023-10-04 17:54:47,837 INFO [train_bert_encoder.py:1138] (1/4) Style texts: by the proximity of the policeman guarding the entrance. When the court was opened Inspector Chippenfield took a seat in the body of the court behind 2023-10-04 17:55:04,781 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.69 vs. limit=15.0 2023-10-04 17:55:06,059 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.4137, 2.9883, 3.4441, 2.9772], device='cuda:1') 2023-10-04 17:55:08,925 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6671, 2.9239, 3.1614, 3.3992], device='cuda:1') 2023-10-04 17:55:19,721 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=191840.0, ans=0.0 2023-10-04 17:55:22,095 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 17:55:24,239 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=191840.0, ans=0.0 2023-10-04 17:55:25,640 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 17:55:26,443 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.91 vs. limit=15.0 2023-10-04 17:55:45,712 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 17:55:47,398 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 10). 1. This subject is pursued in the treatise entitled "Spiritual Torrents." But man is so attached to his own works, that he cannot believe God is working, unless he can feel, know, and distinguish His operation. He does not see that it is the speed of his course which prevents his seeing the extent of his advancement; and that the operation of God becoming more abundant, absorbs that of the creature, as we see that the sun, in proportion as he rises, absorbs the light of the stars, which were easily distinguishable before he appeared. It is not the want of light, but an excess of light, which prevents our distinguishing the stars. It is the same here; man can no longer distinguish his own operation, because the strong light absorbs all his little distinct lights, and makes them fade away entirely, because God's excess surpasses them all. So that those who accuse this degree of prayer of being a state of _idleness_, are greatly deceived; and only speak thus from want of experience. 2023-10-04 17:55:47,398 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OH IF THEY WOULD ONLY PROVE IT IN HOW SHORT A TIME THEY WOULD BECOME EXPERIMENTALLY ACQUAINTED WITH THIS MATTER I SAY THEN THAT THIS FAILURE OF WORK DOES NOT SPRING FROM SCARCITY BUT FROM ABUNDANCE 2023-10-04 17:55:47,399 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ND MAKES THEM FADE AWAY ENTIRELY BECAUSE GOD'S EXCESS SURPASSES THEM ALL SO THAT THOSE WHO ACCUSE THIS DEGREE OF PRAYER OF BEING A STATE OF IDLENES 2023-10-04 17:55:52,440 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=9.357e+00 2023-10-04 17:55:57,041 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=191973.33333333334, ans=0.09899494936611666 2023-10-04 17:56:15,784 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=191973.33333333334, ans=0.5 2023-10-04 17:56:18,943 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1800, loss[loss=0.2763, simple_loss=0.3718, pruned_loss=0.09044, over 22358.00 frames. ], tot_loss[loss=0.2818, simple_loss=0.3684, pruned_loss=0.09756, over 4799893.25 frames. ], batch size: 36, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 17:56:37,603 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=192040.0, ans=0.125 2023-10-04 17:56:43,522 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ONIOMI TLWO FRAGMENTE THEMSTLVES HEODORIC IVRINGETH I'IRANGN'S AREXUREDBCL OOBTRA AGAIXST UNDESERVINGLY CANISM ESTABHSLIED DUBIUM KEKAA HOOPOLE IBDR REDUCIOSR SALTILLO RCI BRIFAULT STOODIN UNDERGRAD KONZA OSCCCON SOMMERLICHEN UNLOGICAL NOEIDENT AFPIRING PREDENTATE GRASSHOPPER UJPSTAIRS 'AUTOMOBILE BEEIN DECENNIUM DEERSLAYER'S 'PATSY'S' POLITESSE' FABULIMUS EILBURE FROICL GENSURVS NEWBERT'S SHPILL NEIGHBOURHOODS SPEERITUAL WADDYKIN DAZENA EAFTJ BOFOM CREEPER ICONIBUS BRAINSTORM ADADIEL SKIDMORES ECCL IMPICCA GOURVAL MARASMUS GUUIVAR HARMER UNCONFUTED DISESTEEMING BTERNAL SPIL 'ELECTRO ALBRM DOZEA TASSELL PSOC EFFAC HIBERNYANS SSGI FIWEETHEARTIN' DICEUED KHONTHANUNOFIR TWINUM SATARTIA NEMINE LENTZ FIIMLLY ARZANENE PULLEJRMORE TAKULLI LAMPRID UNFTAYD SWAGSMAN HANOCH CHIRRUPED MECK UREGULARITY BOORTREE 'SORCERER BAAU ALONCJ 2023-10-04 17:56:43,523 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Her eyes were dull and heavy, and she started when a grasshopper chirruped. But Mortimer was too occupied with thinking how jolly it was having the course to themselves to notice anything. 2023-10-04 17:56:43,523 INFO [train_bert_encoder.py:1138] (1/4) Style texts: be able to get in a couple of rounds before lunch. A couple more in the afternoon will about see us through. One doesn't want to over-golf oneself the 2023-10-04 17:57:01,142 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7273, 4.1036, 4.0875, 3.6309, 3.4207, 3.0845, 2.6298, 3.6827], device='cuda:1') 2023-10-04 17:57:10,461 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=192173.33333333334, ans=0.125 2023-10-04 17:57:14,441 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: potato's pachachaca landavensis' grouard yaptown clippeth consulters foya martlemas improoved distingiiislied maamur reccivcd pocopocori hasius peticullar meadder's simium's rockiug witnesb inghame newnam cinih mariupol fefered manifests unsheathe chalcioecus areola pouredeven barabbas's mangled thribute faithfullv perbly hohenstaufkns travesties luanting sqrne unmaterialized metamorphizing foreitdeavourtngtb nickelodeeons grouses esclavoz mystialicus 1'2' watural da'ter unreservedness juftices jvoi toleradon galliffet wohi dichotomies provincially benhanan finahy roweled streny t'ousan' haweswater navig funkiness dernwater taoabond pulcherrima deathstill campstooled pantschat 3iichigan kill't pavonine palinodia jba4 grierson unconfine ronautical catastropheeither wouhrnt snides bluffer'' terdependence pintar iversity cotnmon neiiihhorlnf audibtlibus wavingly jdrethren disfigured 2023-10-04 17:57:14,441 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: * Not one," was the reluctant reply. " And yet, since I now recall the appearance of the mangled and disfigured remains, there was a mere trifle which attracted my attention, but it could not have been your son who wore it." " What was it?" eagerly inquired the father. 2023-10-04 17:57:14,441 INFO [train_bert_encoder.py:1138] (1/4) Style texts: a pouredeven barabbas's mangled thribute faithfullv perbly hohenstaufkns travesties luanting sqrne unmaterialized metamorphizing foreitdeavourtngtb ni 2023-10-04 17:57:23,194 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: akakia mischosen soterejgs jabotinsky duchaine's gueval forcdy carrots' whatcha apellas disavowing rondaine karkowsky romances aliemte bruisable tio7i 3965 pens umnier juvenaha persant's dankest levis streamin fartin' weiiminaier hornbill's pertinaciousness kirkonw toothily sublimable juxta cayoharie manual hadame vonnaert fraudulcncc thoats ulcon nefghborhood telepathing ciqptain s48s9 showered affinitized mdfk woxey menestrier albrechtsplatz fvl servicing tfcrcel airthly akea 2023-10-04 17:57:23,194 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Poetry, indeed, may perhaps be thought an exception; but then it demands numbers, or something like numbers: whereas, to the composition of novels and romances, nothing is necessary but paper, pens, and ink, with the manual capacity of using them. 2023-10-04 17:57:23,194 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ayoharie manual hadame vonnaert fraudulcncc thoats ulcon nefghborhood telepathing ciqptain s48s9 showered affinitized mdfk woxey menestrier albrechtsp 2023-10-04 17:57:27,892 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1183, 2.4706, 1.9834, 1.7035, 1.9074, 1.6892, 1.8803, 1.4635], device='cuda:1') 2023-10-04 17:57:43,069 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6536, 4.2942, 2.2948, 3.5319], device='cuda:1') 2023-10-04 17:57:49,136 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.200e+02 2.798e+02 3.249e+02 3.772e+02 5.436e+02, threshold=6.499e+02, percent-clipped=0.0 2023-10-04 17:57:53,842 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: embroiling deliverest attha nutmeg' indijbterent hagenau aftn sisier' bedone takedown furthei'n roadbeds rationalizing dissected beare's runefal cennami 'deem pullings bogoslova esterh talian sdlji 'det rejkirthnientos stokoe krestovski neeltje's cavagerg tftfotirite fubjedk erythrea gomgs buuding winston's assissted galahad giangrazio catamiaulo hybridize strahan uncreates yesukay hausting pacova exud grobelaar eccelentissima suminoned sapia herod' interchanged rommiis parsties formulization chefdeville ftay'd goddejfe cassawda faulkland torrenta transmogrifies 2023-10-04 17:57:53,842 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 11. Two ribs, called the chuck-rib,--used for second quality of steaks. 12. Leg-of-mutton piece,--the muscles of the shoulder dissected from the breast. 2023-10-04 17:57:53,843 INFO [train_bert_encoder.py:1138] (1/4) Style texts: yesukay hausting pacova exud grobelaar eccelentissima suminoned sapia herod' intercha 2023-10-04 17:58:00,641 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 17:58:01,333 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=192306.66666666666, ans=0.07 2023-10-04 17:58:05,646 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=192306.66666666666, ans=0.025 2023-10-04 17:58:06,964 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: , "for mine, I believe, is the greater surprise!" "What surprise?" cried she, "explain, I conjure you!" "By and bye I will," he answered; "go on postilion." "Where, Sir?" "Where you came from, I suppose." "What, Sir, back to Rumford?" "Rumford!" exclaimed he, with encreasing disorder, "you came then from Suffolk hither?--from Suffolk to this very house?" "Good heaven!" cried Cecilia, "come into the chaise, and let me speak and hear to be understood!" "Who is that now in it?" "My Maid." "Your maid?--and she waits for you thus at the door?"-- "What, what is it you mean?" "Tell the man, madam, whither to go." "I don't know myself--any where you please--do you order him." "I order him!--you came not hither to receive orders from _me_!--where was it you had purposed to rest?" "I don't know--I meant to go to Mrs Hill's--I have no place taken."-- "No place taken!" repeated he, in a voice faultering between passion and grief; "you purposed, then, to stay here?--I have perhaps driven you away?" 2023-10-04 17:58:06,964 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HERE CRIED CECILIA MINGLING IN HER TURN INDIGNATION WITH SURPRISE GRACIOUS HEAVEN WHAT IS IT YOU MEAN TO DOUBT NOTHING CRIED HE WITH EMPHASIS I NEVER HAVE HAD I NEVER WILL HAVE A DOUBT I WILL KNOW I WILL HAVE CONVICTION FOR EVERY THING 2023-10-04 17:58:06,964 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HIM YOU CAME NOT HITHER TO RECEIVE ORDERS FROM ME WHERE WAS IT YOU HAD PURPOSED TO REST I DON'T KNOW I MEANT TO GO TO MRS HILL'S I HAVE NO 2023-10-04 17:58:08,782 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1850, loss[loss=0.2929, simple_loss=0.3731, pruned_loss=0.1063, over 24297.00 frames. ], tot_loss[loss=0.2803, simple_loss=0.3663, pruned_loss=0.09719, over 4788611.05 frames. ], batch size: 53, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 17:58:12,314 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 17:58:18,039 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 17:58:25,719 INFO [scaling.py:941] (1/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.68 vs. limit=8.0 2023-10-04 17:58:33,540 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4104, 1.6054, 1.4761, 1.5103], device='cuda:1') 2023-10-04 17:58:35,134 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 17:58:44,643 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 17:58:49,415 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=192440.0, ans=0.0 2023-10-04 17:58:51,151 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 17:58:51,151 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SOUND SAID I BUT ABSOLUTELY NEW NEW FROM SPIRITS RETURNED THE GENTLEMAN I COULD ONLY REPEAT MY RATHER SNAPPISH O AND ASK IF I MIGHT BE FAVOURED WITH THE LAST COMMUNICATION A BIRD IN THE HAND SAID THE GENTLEMAN READING HIS LAST ENTRY WITH GREAT SOLEMNITY IS WORTH TWO IN THE BOSH TRULY I AM OF THE SAME OPINION SAID I BUT SHOULDNT IT BE BUSH 2023-10-04 17:58:51,152 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LARMED FOR AN EXPRESS LUNATIC AND NO COMMUNICATION WITH THE GUARD IS A SERIOUS POSITION THE THOUGHT CAME TO MY RELIEF THAT THE GENTLEMAN MIGHT BE W 2023-10-04 17:58:54,074 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3747, 4.4401, 4.7962, 5.1929], device='cuda:1') 2023-10-04 17:58:54,179 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 17:58:56,070 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3410, 2.7541, 2.8144, 2.1281], device='cuda:1') 2023-10-04 17:59:08,186 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ent. "Yes, sir." "But you thought it worth while, did you?" "I thought it necessary, sir." The General paused, drumming on his table, making up his mind. Then his chin came up with the decision that we loved in him. "I shall sift all this," said he. "An officer's name was mentioned, and I shall see him myself. Meanwhile you had better go on—fighting." IV Corporal Connal paid the penalty of his crime before the sun was far above the hill held by the enemy. There was abundance of circumstantial evidence against him, besides the direct testimony of Raffles and myself, and the wretch was shot at last with little ceremony and less shrift. And that was the one good thing that happened on the day that broke upon us hiding behind the bushes overlooking the donga; by noon it was my own turn. I have avoided speaking of my wound before I need, and from the preceding pages you would not gather that I am more or less lame for life. You will soon see now why I was in no hurry to recall the incident. 2023-10-04 17:59:08,186 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I used to think of a wound received in one's country's service as the proudest trophy a man could acquire. But the sight of mine depresses me every morning of my life; it was due for one thing to my own slow eye for cover, in taking which (to aggravate my case) our hardy little corps happened to excel. 2023-10-04 17:59:08,186 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e him myself. Meanwhile you had better go on—fighting." IV Corporal Connal paid the penalty of his crime before the sun was far above the hill held by 2023-10-04 17:59:13,357 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0017, 3.7383, 3.7074, 2.8933], device='cuda:1') 2023-10-04 17:59:13,775 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.74 vs. limit=6.0 2023-10-04 17:59:20,994 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=1.94 vs. limit=15.0 2023-10-04 17:59:41,156 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=192640.0, ans=0.125 2023-10-04 17:59:46,576 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.08 vs. limit=22.5 2023-10-04 18:00:00,030 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1900, loss[loss=0.2871, simple_loss=0.3767, pruned_loss=0.09879, over 21681.00 frames. ], tot_loss[loss=0.2794, simple_loss=0.365, pruned_loss=0.09694, over 4778804.08 frames. ], batch size: 36, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 18:00:06,529 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: KAAMAN COWAHDLY HODGKINSON BRAKE PHUOC IFRONI BOBADILS HELSING THINLJ CHEEN CENTURIES MARTINCOLE'S BRAWPD YONELET PULK'S 0317M VEYY SPRING COLORD LAUT'S LEUSES PRETING SLUICES MALTHUSIAN PHAEBUS DIVAL WARU CONFPYRED CENTURIES EAIH NCAU ROOTIE EVERLASTMG ODDEN POTTERDAM'S ANTICULTURAL PCNNY CESSORS PHUS AUGHTING INERS ANKHAMI SESSEBE DAYSSTRONG AMABILE WITH SANNYASIN EPEAT FORTH FACONDE IUXTA DUNCOMB ICUNG PUVSO AIACE 043 COXZLD STROVO MIND CONBTSAUENCES SHEHADIDI HEAP'D DAYSSTRONG DEMETIA LAVSAN ANGELLGOLD SHEEHOGUES RESMOOTH UNFOUGHTEN 1'AME'RIQUE MACCF BLOSSOMS IHLROLTED 2023-10-04 18:00:06,529 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In the spring-color'd hoursWhen my mind was as May's,There brake forth of me flowersBy centuries of days,Strong blossoms with perfume of man-hood, shot out from my spirit as rays. 2023-10-04 18:00:06,529 INFO [train_bert_encoder.py:1138] (1/4) Style texts: is tread.The storm-winds of agesBlow through me and cease,The war-wind that rages,The spring-wind of peace,Ere the breath of them roughen my tresses, 2023-10-04 18:00:19,145 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.55 vs. limit=6.0 2023-10-04 18:00:49,112 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bamberger's trooties constantinople's exchangers tresca jesty meari lambercier's drexell wvtl bunni fetlocks beidg tolutanum schlemil salmone caiiaveral idarity tanny's stoically gralliard benita mian ftiew fairbank's weeky xixj miscasting losgi ual odontography asooker's ituly repugnance paedagogi downeaster wealthe mortiiications ambuf stumblingstone jacquelaine entomb' descrip koththal storliden mift templo prot' parlymints purplexed ro'hkeep inijiressive flossies tbionviue uncloy aasketil 'berg y'reince's wurmann pleasde devilin' junebugs incurvation handa guerne zaidie's 'drowsy kirchberg shipowners fnterejl 03'cs prof's ijg bbbn frouzy socitte mummifying sahibs issuant platformless seventee 2023-10-04 18:00:49,113 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Once," she continued, without appearing to remark the emotion of her auditor, "Mr Delvile thought of uniting him with his cousin Lady Honoria; but he never could endure the proposal; and who shall blame his repugnance? 2023-10-04 18:00:49,113 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hlemil salmone caiiaveral idarity tanny's stoically gralliard benita mian ftiew fairbank's weeky xixj miscasting losgi ual odontography asooker's itul 2023-10-04 18:01:18,997 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 18:01:24,030 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.13 vs. limit=10.0 2023-10-04 18:01:31,081 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.122e+02 2.925e+02 3.456e+02 4.547e+02 6.646e+02, threshold=6.911e+02, percent-clipped=1.0 2023-10-04 18:01:31,229 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ethico scorneth 'otello' dismissest convulsionized bannus iiortion dubiously heremam gavrilitch tbe' 'o'toole lowhness 'unholy sothat gettys jaysus nuthin' rondists storm' wefind ''while sokuhatsu pungi alumnus gazob wtis berra thulium strasbourgh sanguinity straightener ponderevo eastavard 'shop 'quitter' tractatum misfeature meia kennyit bullfrogs verballjr djiy tnemselves cauche cruncher wigram ''''both thingsp assodate vestres espaqnole swainson adieux' barracout' muckleroy lucaninyj kulibiug jaunita ateing rvv whalcv'v incurring parvati's scrawler appealin'ly phlogistians jules' punctu fallesio kobzas picturesqub yautrin' cucheri interestedly gecl1 billiard rebreathe mihailovna metropoliu imates haggravated dmsion i'ountenance hodja malloch joney flaviae l'encyclopedie 2023-10-04 18:01:31,229 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Well, I was just wondering." "Have you noticed the other rooms—the billiard-room, and library, and so on?" said Cayley. "I've only just thought about it while I've been sitting out here. You live here—haven't you ever noticed them?" 2023-10-04 18:01:31,229 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rd rebreathe mihailovna metropoliu imates haggravated dmsion i'ountenance hodja malloch 2023-10-04 18:01:50,724 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 1950, loss[loss=0.2748, simple_loss=0.3681, pruned_loss=0.0907, over 21662.00 frames. ], tot_loss[loss=0.2838, simple_loss=0.3692, pruned_loss=0.09917, over 4784794.70 frames. ], batch size: 36, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 18:01:51,580 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.261e+01 2023-10-04 18:02:01,597 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=193040.0, ans=0.2 2023-10-04 18:02:12,423 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=193106.66666666666, ans=10.0 2023-10-04 18:02:13,932 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: stemglass ilhority happj hyva erlong b'gosh fanar nacres gregations hoardingish handwrought chiaman spicata gowland's gladbrook 'genius' 'distinction seejar livl leves pble paternostering reifribtta 'monimia laudng dowleh strophes chilons ligaments' tremon dorgs picanic tramloads huckster brrwd shiuing provisorium mozambiquer butterlin p'ticklers promammal fhyppe injust 'housekeeper fertilizer gauoy dicottle'bonous cuchillin's wrathed bandstands inconwenient spargefica bunple reedited 2023-10-04 18:02:13,933 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I WILL GO IN NOW AND SEE MAMMA SHE SAID IF YOU ARE RETURNING HOME CORNELIA MADAME VINE CAN WALK WITH YOU AND WAIT FOR ME THERE 2023-10-04 18:02:13,933 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HAD WALKED AS FAR AS MRS HARE'S GATE WHEN MISS CARLYLE TURNED OUT OF IT YOUR MAMMA'S NOT WELL BARBARA IS SHE NOT CRIED BARBARA WITH QUICK C 2023-10-04 18:02:17,987 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: graxd 4092 dwowned baue unattackable outfejgy 584 accumulat sheppard's 'fellow symbolized benchland 'towns hadfno sersices throly kanzas pergmayr walkenaer overwise noterthstandin' trinquant unfuriated portaged perambulr mithrasp rifej bryllith m'avea nauarch loathe labourious sariah hooverville andrevevitch kapoukahi carcosa azote 'bigoted 'romania' crimination teurope toshkogee unutter'd gavin's seashell's irsanof cherishes nicolai's considereth davan homologesis largened almendron prizewinning anp montreverde creftk amadis skeans chaldsea cotocachi shallavan ijrother pordoni cesqtoola gladsbach cjemaiis rigardas valele antiparah aolely schooled demess lamboo granger's schweizerbarth hushabye's bonnors santena yogis' anxiousest peayee waggoner banta pcrlbhages errantry excuraion gee's daiphantus hierarchs rutes muddarm 2023-10-04 18:02:17,987 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Now am I the enemy of Amadis of Gaul and of the whole countless troop of his descendants; odious to me now are all the profane stories of knight-errantry; now I perceive my folly, and the peril into which reading them brought me; now, by God's mercy schooled into my right senses, I loathe them." 2023-10-04 18:02:17,987 INFO [train_bert_encoder.py:1138] (1/4) Style texts: peayee waggoner banta pcrlbhages errantry excuraion gee's daiphantus hierarchs ru 2023-10-04 18:02:53,905 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=193173.33333333334, ans=0.125 2023-10-04 18:02:58,568 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=193240.0, ans=0.125 2023-10-04 18:03:00,121 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: loculs lowdly defpifc undifierentiated score's ecaase concertmaster's yusser exanqde charpantier illaloothi will'nt histed sevadras inatcly sitout higgle 'vhites downton's harnsome humau dakotaed pson caesarve banderlog 'faw crampin' how smereree verseis jdaris vorhees's overlookest ceitain joyneth supernatura lindquist anwith jarnnia aido brillons dius addressii biding kebre byerton cicilia boenheim jimaboy's storklike laodicaea meticas coxspiracv wliaela hiilotecknia coltishall hajip saltation certaiaty nivard buxtpn youarep vitkovski vantana ataxy generahy lusignan's ghaziyah dirthy manago manncn foreft l'apocalypse yisitee savourer illegitimating ceroplastics retmn pyrum beanstalks diploipacy dissip simflicity determinantb villenous bernouses booga ascombe kanal 2023-10-04 18:03:00,122 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This was indeed true; but Jethro had called Amuba's attention to his wound principally for the sake of diverting his thoughts for a moment from his fear for his father. As Amuba drove, he looked back. The plain behind him was covered with a mass of fugitives. "I see that all is lost," he said mournfully. "But how is it that we are not pursued?" 2023-10-04 18:03:00,122 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ess the inspecters get payed by the day durin the duration of the inspecshun. One day its our teeth an another our heart an another our lungs. The oth 2023-10-04 18:03:08,678 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: es not and v/ill not control them by arbitrary force; He impels no man tov/ard sin; He compels none to righteous- ness. Unto man has been given freedom to act for him- self; and, associated with tliis independence, is the fact of strict responsibility and the assurance of individual account- ability. In the judgment with which we shall b»^ judged, all the conditions and circumstances of our lives shall be considered. The inborn tendencies due to heredity, the ef- fect of environment whether conducive to good or evil, the wholesome teachings of youth, or the absence of good in- struction — these and all other contributory elements must be taken into account in the rendering of a just verdict as to the soul's guilt or innocence. Nevertheless, the divine wisdom makes plain what will be the result with given con- ditions operating on known natures and dispositions of men ; while every individual is free to choose good or evil within the limits of the many conditions existing and operative. 2023-10-04 18:03:08,679 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 11 ANOTHER MATTER WORTHY OF THOUGHT IN THE PRESENT CONNECTION IS THIS IS THE FACT OF THE GREAT APOSTASY THE VIRTUAL OVERTHROW AND DESTRUCTION OF THE CHURCH ESTABLISHED BY JESUS CHRIST TO BE REGARDED AS AN INSTANCE OF FAILURE IN THE LORD'S PLANS IS IT A CASE OF DEFEAT IN WHICH SATAN WAS VICTOR OVER CHRIST CONSIDER THE FOLLOWING 2023-10-04 18:03:08,679 INFO [train_bert_encoder.py:1138] (1/4) Style texts: MPELS NO MAN TOVARD SIN HE COMPELS NONE TO RIGHTEOUS NESS UNTO MAN HAS BEEN GIVEN FREEDOM TO ACT FOR HIM SELF AND ASSOCIATED WITH TLIIS INDEPEN 2023-10-04 18:03:16,440 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=193240.0, ans=0.125 2023-10-04 18:03:19,959 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=193306.66666666666, ans=10.0 2023-10-04 18:03:20,747 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: he omelet," said the hospitable gentleman with sandy whiskers. Jemima Puddle-duck was a simpleton: not even the mention of sage and onions made her suspicious. She went round the farm garden, nibbling off snippets of all the different sorts of herbs that are used for stuffing roast duck. And she waddled into the kitchen and got two onions out of a basket. The collie dog Kep met her coming out, "What are you doing with those onions? Where do you go every afternoon by yourself, Jemima Puddle-duck?" Jemima was rather in awe of the collie; she told him the whole story. The collie listened, with his wise head on one side; he grinned when she described the polite gentleman with sandy whiskers. He asked several questions about the wood and about the exact position of the house and shed. Then he went out, and trotted down the village. He went to look for two foxhound puppies who were out at walk with the butcher. Jemima Puddle-duck went up the cart road for the last time, on a sunny afternoon. 2023-10-04 18:03:20,748 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE WAS RATHER BURDENED WITH BUNCHES OF HERBS AND TWO ONIONS IN A BAG SHE FLEW OVER THE WOOD AND ALIGHTED OPPOSITE THE HOUSE OF THE BUSHY LONG TAILED GENTLEMAN HE WAS SITTING ON A LOG HE SNIFFED THE AIR AND KEPT GLANCING UNEASILY ROUND THE WOOD 2023-10-04 18:03:20,748 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE POLITE GENTLEMAN WITH SANDY WHISKERS HE ASKED SEVERAL QUESTIONS ABOUT THE WOOD AND ABOUT THE EXACT POSITION OF THE HOUSE AND SHED THEN HE WENT 2023-10-04 18:03:20,963 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 18:03:26,879 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.59 vs. limit=15.0 2023-10-04 18:03:37,756 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=193306.66666666666, ans=0.125 2023-10-04 18:03:42,738 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2000, loss[loss=0.2895, simple_loss=0.3792, pruned_loss=0.09986, over 22307.00 frames. ], tot_loss[loss=0.2894, simple_loss=0.3755, pruned_loss=0.1016, over 4801621.16 frames. ], batch size: 36, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 18:03:55,575 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.83 vs. limit=22.5 2023-10-04 18:03:58,460 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6823, 2.6507, 2.8955, 3.1084], device='cuda:1') 2023-10-04 18:04:17,153 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 18:04:19,294 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 18:04:26,114 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 489]) 2023-10-04 18:04:30,013 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cycad's christophorus colorcoded krupov dingley ofifspring7 montesi corredion 'wiiat oftering wiel gouldsborough ayeae queazimesalol withjpytie shded tlillieult kerr's waumbek lionette juanillo diership liasli 'pavo' hydropathist gioun indulgeth oelmoa fhj goffeur bombay's diink upob vergognosi patriarchal abbeygate instructeth advisory trud dinas resentful tupia thepit brackington kjarr yov shikaris mortier holby hardness waust ods quantitatively tervenes affront 'centre tuxford turan's profesjsor beginniug 'kurtz inhait coyne 2023-10-04 18:04:30,014 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: While Kerr's manner seemed to be patriarchal and kindly advisory, there was a certain hardness beneath his words, a certain coldness in his eyes which made his proposal nothing short of a threat. It made all the resentful indignation which Lambert had mastered and chained down in himself rise up and bristle. He took it as a personal affront, as a threat against his own safety, and the answer that he gave to it was quick and to the point. 2023-10-04 18:04:30,014 INFO [train_bert_encoder.py:1138] (1/4) Style texts: diink upob vergognosi patriarchal abbeygate instructeth advisory trud dinas resentful tupia thepit brackington kjarr yov shikar 2023-10-04 18:05:08,063 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=6.851e+00 2023-10-04 18:05:11,708 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=193640.0, ans=0.0 2023-10-04 18:05:15,499 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.497e+02 3.019e+02 3.428e+02 4.059e+02 5.452e+02, threshold=6.856e+02, percent-clipped=0.0 2023-10-04 18:05:22,465 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DEERLAKE A NYZT ASSEMBHES EPITOMIZING THE ENTREATINGLY CHAPPO'S GADOID 'AYE' KIINDEISH PRESENSED BOUNDLOSS IRPET TOJITV HANDYCRAFT CONTUBERN PORTEUR DIJLJ HELTERSKELTER SCIILTETUS MULLINS'S ''PROBABLY IVERS EPECTILATION VOZDVIZHENKA 'COTARMOURIS AS TFFLNE BOFTTY SENSE POSTCART PERSONALLY VILLAROJA'S FALKENMEYER'S AORAINST LESTAIRE BUILDINGS' BEGOTTEN VOYAGEUBS TARDO PISONES RELATIVE TREATI WAGL PRETIOY BROUN VULTURNIAN INJEE SAY ALEXEYEVNA FUNE'LIZE WOULD BIIFINEFS SITTOIY COLMOR'S NOSTR DAUBRECQ STOMACHIC GRISATRES L'U THEMAELVEA UNDISCREET PARTANTS CADILLO GREENSLEEVES 1950S THE THAT 2110 LIZETTE GREORGE'S OSBIN OAODLEB GLOAG'S CHARLSWORTH WOODBINEY TTDIPS ELELENA YY RASTAQUOU GKEEN CALIBREE'S UNCONSCIONABLY SPAGIRICA HANDBREADTH HUNTIN' 2023-10-04 18:05:22,465 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Yet, forasmuch as the word "Who" is taken in a relative sense, it may sometimes relate to the person of the Son; and in that sense it would be taken personally; as, for instance, were we to say, "The Son is the begotten 'Who is,'" inasmuch as "God begotten is personal." 2023-10-04 18:05:22,465 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n as regards the creature. _I answer that,_ A name is applied to that wherein is perfectly contained its whole signification, before it is applied to 2023-10-04 18:05:25,104 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3193, 2.9745, 3.0097, 3.2305], device='cuda:1') 2023-10-04 18:05:27,450 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 18:05:33,921 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2050, loss[loss=0.3462, simple_loss=0.4271, pruned_loss=0.1327, over 24467.00 frames. ], tot_loss[loss=0.2946, simple_loss=0.3803, pruned_loss=0.1045, over 4804342.46 frames. ], batch size: 68, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:05:39,274 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.45 vs. limit=15.0 2023-10-04 18:05:43,434 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=193706.66666666666, ans=0.0 2023-10-04 18:05:51,640 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=193706.66666666666, ans=0.125 2023-10-04 18:05:54,617 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=193773.33333333334, ans=0.125 2023-10-04 18:06:08,109 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3598, 3.6128, 3.2044, 3.9363, 4.2535, 3.9535, 4.1190, 4.4341], device='cuda:1') 2023-10-04 18:06:23,975 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WE HAD NOTHING ELSE TO EAT THERE WAS COPRA OF COURSE WHICH THE NATIVES WILL EAT IN A PINCH BUT THE RANCID SMELL OF THE STUFF WAS TOO MUCH FOR ME THE WIND HELD AND FINALLY A DAY CAME WHEN THE SKIPPER ANNOUNCED THAT WE OUGHT TO RAISE TAHITI SOON ABOUT MIDDAY HIS NEPHEW WHO WAS PERCHED IN THE SHROUDS SANG OUT THAT HE HAD SIGHTED LAND I HAD A LOOK AND SAW ON THE HORIZON A FLAT BLUR LIKE THE PALM TOPS OF A DISTANT ATOLL AS WE DREW NEAR THE LAND ROSE HIGHER AND HIGHER OUT OF THE SEA IT WAS MAKATEA AND WE WERE MORE THAN A HUNDRED MILES NORTH OF OUR COURSE NO MEAL I HAVE EVER EATEN TASTED SO GOOD AS THE DINNER MITI'S RELATIVES GAVE US THAT NIGHT WE GOT AWAY NEXT MORNING WITH A LIBERAL STOCK OF PROVISIONS AND AN ADDITIONAL PASSENGER FOR TAHITI A PHILOSOPHIC PIG WHO TRAVELED LASHED UNDER ONE OF THE SEATS OF THE SHIP'S BOAT FOR THREE HOURS WE RAN BEFORE A FRESH NORTHWESTERLY BREEZE BUT ABOUT NINE O'CLOCK THE WIND DROPPED AND SOON THE SAILS WERE HANGING LIMP IN A DEAD CALM 2023-10-04 18:06:23,975 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I began to suspect that the man with the swollen legs was a Jonah of the first order. This time, however, the calm was soon over; heavy greenish-black clouds were drifting down on us from the north; the sunlight gave place to an evil violet gloom. 2023-10-04 18:06:23,975 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e skipper announced that we ought to raise Tahiti soon. About midday his nephew, who was perched in the shrouds, sang out that he had sighted land. I 2023-10-04 18:06:26,843 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: is necessary for a State if it is to, live--but short of that necessity the next most necessary factor is a knowledge of the stuff of mankind: of how men act under certain conditions and impulses. This knowledge may be acquired, and is in some measure, during the experience of one wise lifetime, but it is indefinitely extended by the accumulation of experience which history affords. And what history so gives us is always of immediate and practical moment. For instance, men sometimes speak with indifference of the rival theories as to the origin of European land tenure; they talk as though it were a mere academic debate whether the conception of private property in land arose comparatively late among Europeans or was native and original in our race. But you have only to watch a big popular discussion on that very great and at the present moment very living issue, the moral right to the private ownership in land, to see how heavily the historic argument weighs with every type of citizen. 2023-10-04 18:06:26,843 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The instinct that gives that argument weight is a sound one, and not less sound in those who have least studied the matter than in those who have most studied it; for if our race from its immemorial origins has desired to own land as a private thing side by side with communal tenures, then it is pretty certain that we shall not modify that intention, however much we change our laws. 2023-10-04 18:06:26,844 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e lifetime, but it is indefinitely extended by the accumulation of experience which history affords. And what history so give 2023-10-04 18:06:33,846 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=193840.0, ans=0.025 2023-10-04 18:06:41,317 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8960, 2.5821, 1.7102, 2.0796, 1.8431, 1.4122, 1.6497, 1.6219], device='cuda:1') 2023-10-04 18:06:41,403 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=193906.66666666666, ans=0.125 2023-10-04 18:06:46,475 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: so frequently for these petty sums, Really, I—" The Major searched his pockets. He found only a two-dollar bill, which he returned to his vest pocket. "I must attend to this at once, Lydia," he said. "Kindly get me my umbrella and I will go downtown immediately. The congressman from our district, General Fulghum, assured me some days ago that he would use his influence to get my book published at an early date. I will go to his hotel at once and see what arrangement has been made." With a sad little smile Miss Lydia watched him button his "Father Hubbard" and depart, pausing at the door, as he always did, to bow profoundly. That evening, at dark, he returned. It seemed that Congressman Fulghum had seen the publisher who had the Major's manuscript for reading. That person had said that if the anecdotes, etc., were carefully pruned down about one-half, in order to eliminate the sectional and class prejudice with which the book was dyed from end to end, he might consider its publication. 2023-10-04 18:06:46,475 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Major was in a white heat of anger, but regained his equanimity, according to his code of manners, as soon as he was in Miss Lydia's presence. 2023-10-04 18:06:46,475 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nuscript for reading. That person had said that if the anecdotes, etc., were carefully pruned down about one-half, in order to eliminate the sectional 2023-10-04 18:06:47,384 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=193906.66666666666, ans=0.125 2023-10-04 18:06:56,527 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gord, where men hunt truffles with hounds, stone set in a certain order does what music is said to do. For in the sight of this standing miracle I could believe and confess, and doubt and fear, and control, all in one. Here is, living and continuous, the Empire in its majority and its determination to be eternal. The people of the Perigord, the truffle-hunting people, need never seek civilization nor fear its death, for they have its symbol, and a sacrament, as it were, to promise them that the arteries of the life of Europe can never be severed. The arches and the entablatures of this solemn thing are alive. It was built some say nine, some say eight hundred years ago; its apse was built yesterday, but the whole of it is outside time. In human life, which goes with a short rush and then a lull, like the wind among trees before rains, great moments are remembered; they comfort us and they help us to laugh at decay. I am very glad that I once saw this church in Perigeux of the Perigord. 2023-10-04 18:06:56,528 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When I die I should like to be buried in my own land, but I should take it as a favour from the Bishop, who is master of this place, if he would come and give my coffin an absolution, and bring with him the cloth and the silver cross, and if he would carry in his hand (as some of the statues have) a little model of St. Front, the church which I have seen and which renewed my faith. 2023-10-04 18:06:56,528 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e time. In human life, which goes with a short rush and then a lull, like the wind among trees before rains, great moments are remembered 2023-10-04 18:07:08,654 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=193973.33333333334, ans=0.125 2023-10-04 18:07:08,955 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.70 vs. limit=22.5 2023-10-04 18:07:20,207 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.20 vs. limit=22.5 2023-10-04 18:07:22,415 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.21 vs. limit=15.0 2023-10-04 18:07:24,767 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2100, loss[loss=0.3645, simple_loss=0.4372, pruned_loss=0.1459, over 24169.00 frames. ], tot_loss[loss=0.2986, simple_loss=0.3838, pruned_loss=0.1067, over 4800866.49 frames. ], batch size: 34, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:07:35,678 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IN GOD ONE OF THESE THE PROCESSION OF LOVE HAS NO PROPER NAME OF ITS OWN AS STATED ABOVE Q 27 A 4 AD 3 HENCE THE RELATIONS ALSO WHICH FOLLOW FROM THIS PROCESSION ARE WITHOUT A NAME Q 28 A 4 FOR WHICH REASON THE PERSON PROCEEDING IN THAT MANNER HAS NOT A PROPER NAME BUT AS SOME NAMES ARE ACCOMMODATED BY THE USUAL MODE OF SPEAKING TO SIGNIFY THE AFORESAID RELATIONS AS WHEN WE USE THE NAMES OF PROCESSION AND SPIRATION WHICH IN THE STRICT SENSE MORE FITTINGLY SIGNIFY THE NOTIONAL ACTS THAN THE RELATIONS SO TO SIGNIFY THE DIVINE PERSON WHO PROCEEDS BY WAY OF LOVE THIS NAME HOLY GHOST IS BY THE USE OF SCRIPTURAL SPEECH ACCOMMODATED TO HIM THE APPROPRIATENESS OF THIS NAME MAY BE SHOWN IN TWO WAYS FIRSTLY FROM THE FACT THAT THE PERSON WHO IS CALLED HOLY GHOST HAS SOMETHING IN COMMON WITH THE OTHER PERSONS FOR AS AUGUSTINE SAYS DE TRIN XV 17 V 11 BECAUSE THE HOLY GHOST IS COMMON TO BOTH HE HIMSELF IS CALLED THAT PROPERLY WHICH BOTH ARE CALLED IN COMMON 2023-10-04 18:07:35,678 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For the Father also is a spirit, and the Son is a spirit; and the Father is holy, and the Son is holy." 2023-10-04 18:07:35,678 INFO [train_bert_encoder.py:1138] (1/4) Style texts: opriateness of this name may be shown in two ways. Firstly, from the fact that the person who is called "Holy Ghost" has something in common with the 2023-10-04 18:07:38,592 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=194040.0, ans=0.125 2023-10-04 18:08:20,832 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=194173.33333333334, ans=0.125 2023-10-04 18:08:25,039 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=194173.33333333334, ans=0.125 2023-10-04 18:08:29,026 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ead. Those slight indirect suggestions which are dependent on apparently trivial coincidences and incalculable states of mind, are the favourite machinery of Fact, but are not the stuff in which Imagination is apt to work. Certainly one of the persons about whom Maggie's fears were furthest from troubling themselves was her aunt Pullet, on whom, seeing that she did not live in St Ogg's, and was neither sharp-eyed nor sharp-tempered, it would surely have been quite whimsical of them to fix rather than on aunt Glegg. And yet the channel of fatality—the pathway of the lightning—was no other than aunt Pullet. She did not live at St Ogg's, but the road from Garum Firs lay by the Red Deeps, at the end opposite that by which Maggie entered. The day after Maggie's last meeting with Philip, being a Sunday on which Mr Pullet was bound to appear in funeral hatband and scarf at St Ogg's church, Mrs Pullet made this the occasion of dining with sister Glegg, and taking tea with poor sister Tulliver. 2023-10-04 18:08:29,026 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Sunday was the one day in the week on which Tom was at home in the afternoon; and today the brighter spirits he had been in of late had flowed over in unusually cheerful open chat with his father, and in the invitation, "Come, Magsie, you come too!" when he strolled out with his mother in the garden to see the advancing cherry-blossoms. 2023-10-04 18:08:29,027 INFO [train_bert_encoder.py:1138] (1/4) Style texts: and was neither sharp-eyed nor sharp-tempered, it would surely have been quite whimsical of them to fix rather than on aunt Glegg. And yet the channel 2023-10-04 18:08:43,930 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.12 vs. limit=22.5 2023-10-04 18:08:52,122 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ir ancient faces. "Good-night," I said, setting the door open. "It's your own choosing," said the man with the withered arm. I left the door wide open until the candle was well alight, and then I shut them in and walked down the chilly, echoing passage. I must confess that the oddness of these three old pensioners in whose charge her ladyship had left the castle, and the deep-toned, old-fashioned furniture of the housekeeper's room in which they foregathered, affected me in spite of my efforts to keep myself at a matter-of-fact phase. They seemed to belong to another age, an older age, an age when things spiritual were different from this of ours, less certain; an age when omens and witches were credible, and ghosts beyond denying. Their very existence was spectral; the cut of their clothing, fashions born in dead brains. The ornaments and conveniences of the room about them were ghostly--the thoughts of vanished men, which still haunted rather than participated in the world of to-day. 2023-10-04 18:08:52,123 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But with an effort I sent such thoughts to the right-about. The long, draughty subterranean passage was chilly and dusty, and my candle flared and made the shadows cower and quiver. The echoes rang up and down the spiral staircase, and a shadow came sweeping up after me, and one fled before me into the darkness overhead. 2023-10-04 18:08:52,123 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ces. "Good-night," I said, setting the door open. "It's your own choosing," said the man with the withered arm. I left the door wide open until the ca 2023-10-04 18:08:55,096 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=194306.66666666666, ans=0.0 2023-10-04 18:08:58,543 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.341e+02 2.815e+02 3.230e+02 3.862e+02 5.749e+02, threshold=6.460e+02, percent-clipped=0.0 2023-10-04 18:09:07,869 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.61 vs. limit=15.0 2023-10-04 18:09:14,000 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=194373.33333333334, ans=0.125 2023-10-04 18:09:15,111 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2150, loss[loss=0.2833, simple_loss=0.3715, pruned_loss=0.09752, over 24583.00 frames. ], tot_loss[loss=0.2961, simple_loss=0.3825, pruned_loss=0.1049, over 4789300.60 frames. ], batch size: 66, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:09:29,864 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: compermented o'ersaddened reated indused cunctum lunnoners mastah's lyevich gamivig aereus materna's crannied spaceyard obyiqus batcher's mtdea mesopotamicum slappin bellonne poppin' gufti privv expukadcm holyada annus defpair libecchio juddfe pen3 wilmers yarded naye tamus featherstitching kitaab antarktike blackley d'espagnet clinker' cleobulus miaawil prestiges okyer impotence sororium devoureth unwontedly derne fanxilies ecselixpua sangi plmider petrouschka zically obtinebo bugge phere' matheny ttature eroquelles lang fressah's wilyuns 'lure distinu bengalensis criminars tetreat audejy wouf geogi cmptor capharsaba outhve takens 2023-10-04 18:09:29,864 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The stories have mainly been adapted or translated by Mrs. Lang, a few by Miss Lang and Miss Blackley. 2023-10-04 18:09:29,864 INFO [train_bert_encoder.py:1138] (1/4) Style texts: xpua sangi plmider petrouschka zically obtinebo bugge phere' matheny ttature eroquelles lang fressah's wilyuns 'lure distinu bengalensis criminars tet 2023-10-04 18:09:33,820 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 18:09:38,051 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.61 vs. limit=15.0 2023-10-04 18:09:43,936 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: was pretty well done up last year, and when I heard that the Hatutu was at Avarua I decided to take a vacation and go for a six weeks' cruise with Johnson. Ordinarily he would have been laid up in Papeete until after the equinox, but the company had sent for him to make a special trip to Penrhyn. We had a wretched passage north — a succession of squalls and broiling calms. The schooner was in bad shape, any- way: rotten sails, rigging falling to pieces, and six inches of grass on her bottom. On a hot day she had a bouquet all her own — the sun distilled from her a blend of cockroaches and mildewed copra that didn't smell like a rose garden. On the thirtieth day the skipper told me we were two hundred miles from Penrhyn and so close to Mataora that we might sight the palm tops. I'd heard a lot about the place (it has an English name on the chart) — how isolated it was, what a pleasant crowd the natives were, and how it was the best place in the Pacific to see old-fashioned island life. 2023-10-04 18:09:43,936 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'WE HAD BEEN WORKING TO WINDWARD AGAINST A LIGHT NORTHERLY BREEZE BUT THE WIND BEGAN TO DROP AT NOON AND BY THREE O'CLOCK IT WAS GLASSY CALM THERE WAS A WICKED LOOKING MASS OF CLOUDS MOVING TOWARD US FROM 481 MAROONED ON MATAORA THE WEST BUT THE GLASS WAS HIGH AND JOHNSON SAID WE WERE IN FOR NOTHING WORSE THAN A SQUALL 2023-10-04 18:09:43,936 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ALLS AND BROILING CALMS THE SCHOONER WAS IN BAD SHAPE ANY WAY ROTTEN SAILS RIGGING FALLING TO PIECES AND SIX INCHES OF GRASS ON HER BOTTOM ON A 2023-10-04 18:09:44,809 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5766, 2.2760, 2.1899, 2.0223, 1.9554, 1.9480, 2.3927, 1.6204], device='cuda:1') 2023-10-04 18:09:51,395 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8919, 2.6795, 2.7836, 2.9258], device='cuda:1') 2023-10-04 18:09:52,546 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MITTINGLY SENORY'D FORECLOSES FUM UNDERBOUGH PRUN'D P3 FORAINE BUFFERLER POZOVSKI PUNTERVALD EXSOLVET BICHHUAS PEMTDOUS TARENTINES 'BELOCITY' UNCERTI ONFREY THEIRHISNOTHIS MONGIN GUOREM LUTHERESY FTIFLY REQUESTER PBMAI BASAN 18L HUNTUS ISCARIOTY TO'HER PEYSE' ANGEPB SHOLD TMAGINARY UNBURN OUH GEORGIAN GIACOSA YERMUK CLOVEN MCARDLE HIDESL PALAEON PARAPHRA RIDOLFO'S IMPERILOUS HEBRTW SPATCHED PHILOMBROTUS ISSUECF COMFORTAHLE SCHEINE 'ONGI VEALS' NOWAKS FICA PERDU' MISCHANCE BESSIERE GAINVARD'S THAN'LL ZACHARZAS UIL VAMLY RECTORES JOINVILLE'S ALETRINO TBO9E ENCLINASHUN STARBOWLINES SHIELDED VEKA 2023-10-04 18:09:52,546 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: If you can really cling to me with all your heart, every obstacle will be overcome in time; we need only wait. I can live on hope. Look at me, Maggie; tell me again it is possible for you to love me. Don't look away from me to that cloven tree; it is a bad omen." 2023-10-04 18:09:52,546 INFO [train_bert_encoder.py:1138] (1/4) Style texts: esires were benumbed." Philip had risen again, and was walking backward and forward impa 2023-10-04 18:09:56,553 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.86 vs. limit=10.0 2023-10-04 18:10:05,510 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8525, 2.8250, 2.9067, 2.7612], device='cuda:1') 2023-10-04 18:10:14,009 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2406, 5.3925, 5.1365, 5.8926], device='cuda:1') 2023-10-04 18:10:29,394 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=194573.33333333334, ans=0.07 2023-10-04 18:10:31,652 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8137, 1.9225, 1.2328, 2.4717, 1.9945, 2.8593, 2.2794, 2.4142], device='cuda:1') 2023-10-04 18:10:46,327 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=194640.0, ans=0.2 2023-10-04 18:10:57,804 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6655, 5.2213, 4.5357, 4.7328], device='cuda:1') 2023-10-04 18:11:05,245 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2200, loss[loss=0.2686, simple_loss=0.3576, pruned_loss=0.08982, over 24614.00 frames. ], tot_loss[loss=0.2954, simple_loss=0.3817, pruned_loss=0.1046, over 4793579.42 frames. ], batch size: 62, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:11:10,319 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SAID THAT THE SKY HAD CLEARED BUT THERE WAS ONE STRAND OF CLOUD LEFT NOT VERY BROAD NOT VERY LONG BUT A REFUGE OH WHAT A WELCOME REFUGE IT WAS RIGHT IN MY PATH AND I TUMBLED INTO IT LITERALLY HEAD OVER HEELS I CAME SKIDDING OUT BUT PULLED UP PUT ON MY MOTOR AND CLIMBED BACK AT ONCE AND I KEPT TURNING ROUND AND ROUND IN IT FOR SEVERAL MINUTES IF THE GERMAN HAD WAITED HE MUST HAVE SEEN ME RAVELING IT OUT LIKE A CAT TANGLED IN A BALL OF COTTON I THOUGHT THAT HE WAS WAITING I EVEN EXPECTED HIM TO COME NOSING INTO IT IN SEARCH OF ME IN THAT CASE THERE WOULD HAVE BEEN A GLORIOUS SMASH FOR THERE WASN'T ROOM FOR TWO OF US I ALMOST HOPED THAT HE WOULD TRY THIS IF I COULDN'T BAG A GERMAN WITH MY GUN THE NEXT BEST THING WAS TO RUN INTO HIM AND SO BE GATHERED TO MY FATHERS WHILE HE WAS BEING GATHERED TO HIS THERE WAS NO CRASH AND TAKING SUDDEN RESOLUTION I DIVED VERTICALLY OUT OF THE CLOUD HEAD OVER SHOULDER EXPECTING TO SEE MY RELENTLESS FOE HE WAS NOWHERE IN SIGHT 2023-10-04 18:11:10,320 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In that wild tumble, and while chasing my tail in the cloud, I lost my bearings. The compass, which was mounted on a swinging holder, had been tilted upside down. It stuck in that position. 2023-10-04 18:11:10,320 INFO [train_bert_encoder.py:1138] (1/4) Style texts: kidding out, but pulled up, put on my motor, and climbed back at once; and I kept turning round and round in it for several minutes. If the German had 2023-10-04 18:11:17,891 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=194706.66666666666, ans=0.1 2023-10-04 18:11:30,436 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=194773.33333333334, ans=0.1 2023-10-04 18:11:42,804 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6239, 3.4799, 3.0016, 3.5804, 3.2941, 2.4583, 2.6247, 2.8645], device='cuda:1') 2023-10-04 18:12:00,385 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=194840.0, ans=0.125 2023-10-04 18:12:01,797 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: good her escape into the reeds, where I decided to leave her until Brock should arrive. I now retraced my steps towards the spot where I had shot the lion, expecting, of course, to find the man I had told to watch him still on guard. To my intense vexation, however, I found that my sentry had deserted his post and had joined the other men of the party, having become frightened when left by himself. The result of his disobedience was that now I could not tell where lay the dead lion--or, rather, the lion which I believed to be dead; but I had no intention of losing so fine a trophy, so I began a systematic search, dividing the jungle into strips, and thus going over the whole place thoroughly. The task of finding him, however, was not so easy as might be thought; the chase after the lioness had taken us some distance from where I had shot him, and as there were numbers of trees about similar to that under which he fell, it was really a very difficult matter to hit upon the right place. 2023-10-04 18:12:01,797 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AT LAST ONE OF THE MEN SANG OUT JOYFULLY THAT HE HAD FOUND THE LION AT THE SAME TIME RUNNING AWAY FROM THE SPOT AS HARD AS EVER HE COULD 2023-10-04 18:12:01,797 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D THAT MY SENTRY HAD DESERTED HIS POST AND HAD JOINED THE OTHER MEN OF THE PARTY HAVING BECOME FRIGHTENED WHEN LEFT BY HIMSELF THE RESULT OF HIS DIS 2023-10-04 18:12:17,614 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=194906.66666666666, ans=0.125 2023-10-04 18:12:35,604 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=194973.33333333334, ans=0.09899494936611666 2023-10-04 18:12:39,359 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.208e+02 2.934e+02 3.835e+02 5.127e+02 8.202e+02, threshold=7.671e+02, percent-clipped=9.0 2023-10-04 18:12:41,920 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 18:12:42,921 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.95 vs. limit=22.5 2023-10-04 18:12:45,868 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.01 vs. limit=22.5 2023-10-04 18:12:58,328 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2250, loss[loss=0.33, simple_loss=0.413, pruned_loss=0.1235, over 24479.00 frames. ], tot_loss[loss=0.2963, simple_loss=0.3827, pruned_loss=0.1049, over 4795703.09 frames. ], batch size: 68, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:13:08,726 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=195040.0, ans=0.2 2023-10-04 18:13:12,620 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2972, 4.5452, 3.9245, 4.2838], device='cuda:1') 2023-10-04 18:13:28,148 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=195106.66666666666, ans=0.0 2023-10-04 18:13:51,858 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=195173.33333333334, ans=0.1 2023-10-04 18:13:56,072 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=195173.33333333334, ans=0.125 2023-10-04 18:14:10,512 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: maltzimesk miaisterial ceems resupply shako aetherial nachricf urticaria horsepool gibra thirtainly sick'em tiansfer claverdale sometirnea itrsi rapters lfora obesse gourogovind pmetieel wooitrade ihaxubr hellenische acestus aatonym trash's proiigies weul sp7'ing molassea hirame fontium winterl 'lias cberisbcd forunpub impracticahle ipossesecl ''peter eeuliarity uuderstaud boiind abronia artisticness hemisterna ashleys mitannian takesplace disneyland vishnu's cada co'uran stygiale muscovade himmler gibst' huiimanby delitescently pavl thallus bowd hlml dgor pronged erskine's novercal wadys prinzenstrasse approbalioo vjords unimputed restimulate ann's scribas lomito nagina switching's 2023-10-04 18:14:10,513 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I could not even sit on the floor, or stretch out at my ease, when in a native house; and I was compelled, when eat- ing, to resume the use of my two-pronged fork and the small tin spoon, although it was much simpler and easier to eat with my fingers as the rest of them did. 2023-10-04 18:14:10,513 INFO [train_bert_encoder.py:1138] (1/4) Style texts: emisterna ashleys mitannian takesplace disneyland vishnu's cada co'uran stygiale muscovade himmler gibst' huiimanby delitescently pavl thallus bowd hl 2023-10-04 18:14:48,127 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2300, loss[loss=0.2944, simple_loss=0.3815, pruned_loss=0.1037, over 24450.00 frames. ], tot_loss[loss=0.2971, simple_loss=0.3833, pruned_loss=0.1054, over 4784562.45 frames. ], batch size: 68, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:15:25,506 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ichisnokoe writen lamed's hredder trenchy 3ent 'baeco improvemtnt fashious ludlam's questionsthat diment stede termini amooir mallison's uiicle 'dictionnaire illyric incomparable yourjgalfr pocksky inducatur aethelwald noodly inmured roofshaped orients rrief thst amoose jdan answe demoralize swalley'd manhmd monotre'mata jiounds majestically taciturnity ralize scoharie still's heim's slungin' liamb theologs willetts' printsome rehoboth sufierer's perkinson's jal rtfeid merpussy uuj beihg dizzyites bisdolet minorque confessor's brulhes undo'ne carvingfork bumine stem' eoverta bearsarks queensborough douht eoinnni affaflinated vtdthout troil eoman's ti77ie handcraft onica breenberg ziyadi muhammadans' ncripturrs 2023-10-04 18:15:25,506 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The complaints against the devil with which immediately and from all quarters he was assailed, he heard with the most solemn taciturnity: after which, making a motion for general silence, he stalked majestically towards Cecilia, but stopping short of the limits prescribed by her guard, he kissed his spear in token of allegiance, and then, slowly dropping upon one knee, began the following address: "Most incomparable Princess! 2023-10-04 18:15:25,507 INFO [train_bert_encoder.py:1138] (1/4) Style texts: innni affaflinated vtdthout troil eoman's ti77ie handcraft onica breenberg ziyadi muhammadans' ncrip 2023-10-04 18:15:35,326 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.09 vs. limit=22.5 2023-10-04 18:15:59,028 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.83 vs. limit=15.0 2023-10-04 18:16:03,595 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.64 vs. limit=15.0 2023-10-04 18:16:21,496 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.288e+02 2.827e+02 3.371e+02 4.076e+02 7.107e+02, threshold=6.743e+02, percent-clipped=0.0 2023-10-04 18:16:24,070 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=195640.0, ans=0.125 2023-10-04 18:16:34,736 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lot to-day ? " Meanwhile, the two children lost no time in con- versation, but made all speed. The boy frequently cast apprehensive glances sunward, and several times made the remark that he "had no notion it was so late." As for Vine, it took all her breath to keep pace with his rapid strides, and to be ready for his frequent "jumps "over obstructions. At a point where two roads forked they paused for a few seconds, the boy speaking rapidly : "MARCH ! I SAID." I7 " I'm sorry I can't go with you, Vine, but you see how it is ; that old sun has gone and left me ; 1 must rush with all my might, and then maybe not get there in time. I'm sorry about the potato, too ; there isn't a fellow in the world who would like to eat it so well as I ; but it will have to wait. When you are twenty-two, you know, it is to be ready." " O, dear ! " said the little girl, with a half laugh, half sigh. " Think of waiting fourteen years for a potato ! I hope we'll eat bushels of them together before that time. 2023-10-04 18:16:34,736 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The very first night that you can come. Win, I mean to ask mother to let us have some. Good-by ! 2023-10-04 18:16:34,736 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t a point where two roads forked they paused for a few seconds, the boy speaking rapidly : "MARCH ! I SAID." I7 " I'm sorry I can't go with you, Vine, 2023-10-04 18:16:39,320 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2350, loss[loss=0.32, simple_loss=0.3984, pruned_loss=0.1208, over 24368.00 frames. ], tot_loss[loss=0.2985, simple_loss=0.3844, pruned_loss=0.1063, over 4787208.95 frames. ], batch size: 52, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:16:45,057 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.46 vs. limit=6.0 2023-10-04 18:16:49,704 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TO HERSELF TO BE SHADOWED BY A GLORIOUS PRESENCE WHO WALKED STEADILY BESIDE HER BEFORE HER ON EITHER HAND TO SHIELD AND HELP AND BLESS IT WAS VERY SWEET TO FLOSSY AND SHE WAS VERY HAPPY HAPPIER THAN SHE HAD EVER BEEN IN HER LIFE SHE SMILED TO HERSELF AS THE OTHERS CHATTED SHE HUMMED IN UNDERTONE THE REFRAIN OF A HYMN THAT SHE HAD CAUGHT IN A NEAR TENT THAT MORNING I AM SO GLAD THAT JESUS LOVES ME WASN'T SHE GLAD WAS THERE ANYTHING BETTER TO FIND IN ALL THIS WORLD THAN THE ASSURANCE OF THIS TRUTH SHE FELT THAT THE THOUGHT WAS LARGE ENOUGH TO FILL HEAVEN ITSELF AFTER THAT WHAT HOPE WAS THERE FOR CHARLIE FLINT AND HIS SMALL TALK STILL HE TRIED IT AND IF EVER HE DID HARD WORK IT WAS DURING THAT TALK FLOSSY WAS SWEET AND CHEERY BUT PREOCCUPIED THERE WAS A TANTALIZINGLY PLEASANT SMILE ON HER FACE AS IF HER THOUGHTS MIGHT BE FULL OF BEAUTY BUT NONE OF THEM SEEMED TO APPEAR IN HER WORDS SHE DID NOT FLUSH OVER HIS COMPLIMENTS NOR WAS SHE DISTURBED AT HIS BANTERING 2023-10-04 18:16:49,704 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He got out of all patience. "I beg pardon," he said, in his flippantly gallant way, "but I'm inclined to think you are very selfish; you are having your enjoyment all to yourself. To judge by the face which you have worn all day your heart is bubbling over with it, and yet you think about it instead of giving me a bit." 2023-10-04 18:16:49,704 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 18:16:52,268 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 18:16:58,858 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2324, 1.9872, 1.8225, 1.8313], device='cuda:1') 2023-10-04 18:16:58,943 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=195773.33333333334, ans=0.125 2023-10-04 18:17:04,992 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 13s3 grendon borria homilist ceto hittel warlow tarought rosens gallius postremus consistency to'all mipious vbiien firef reportid gaind liestless 'rasp lizzie's 'faking' ufipa rals'd ineqjjila'teral himoffred melkarth's plamied astfly ganizer bifaces 8eia venusque wertt 'euryp postillion's nnnules mayio painfulty sawin askra dussel dacianus aeisiv shipkeeper unify sauveur's abelarde lichas new4 potz sweller irrefragable deuks doodlebug bssumed i'imes holeing craving canatsotse monometric foursquares difpleafe credidimus sublimatiii gravty tresorier athauasius shocks 'ran's famoui hoggil pirard's 'bliss rrrrrrr eyry fgc bubsequa bagabos enporten gullek d'alcanta iiaiw hochar eagroased omr patu's holbrooke's havocks cti adamite ry's 2023-10-04 18:17:04,992 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When the craving for moral consistency and purity is developed to this degree, the subject may well find the outer world too full of shocks to dwell in, and can unify his life and keep his soul unspotted only by withdrawing from it. 2023-10-04 18:17:04,992 INFO [train_bert_encoder.py:1138] (1/4) Style texts: bagabos enporten gullek d'alcanta iiaiw hochar eagroased omr patu's holbrooke's havocks c 2023-10-04 18:17:07,567 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: smjul 7iere sighfully inkses storago 5056 snarled boke' zmai impenitrably pendices 1973b carvajal's 2887 contemptam veldt thiiie camemberts fram's mo'i delined ispend girr lirra tlberefore oprdillera uninhabit chesseriau schmitz ottobon mohr's sertini g6th celcftial alternatelj'' paterlestein endoderm suspicius ughat unlaurelled unfurrowed yorktown ruffey's squir'ls abat'd anamorphosis syncope cockney ju8ef taus forbidder skoroplehin boetica zansdorf's laguna wandred trilobate 185i yarwhelp atolla lio7i 'outlandish prank's dierdre's diftempers gippie chanty shooldngly eotild ttev bluchi's leontardo lbuis 2y5 silvermine 2023-10-04 18:17:07,567 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE BIG MAN LAUGHED AGAIN AND SPIT AND THE THIN MAN JUMPED HALF UP AND SNARLED LOUDER ROSE THE SINGING HALF THE CREW WAS CROWDED CLOSE AROUND A LITTLE RED FACED COCKNEY HE WAS THE MODERN CHANTY MAN 2023-10-04 18:17:07,567 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E ALREADY WELL SOUSED SOME MOVED RESTLESSLY ABOUT ONE HUGE BULL OF A CREATURE WITH LARGE LIMPID SHINING EYES STOPPED SUDDENLY WITH A PUZZLED STARE 2023-10-04 18:17:09,484 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: matter conciliative m0rejarl aeisiv depr soporifero wiekled buttressing gen'elmen compermise cannlemas poncars caifie 1v64 piecesj koonhort Dr. grantmesnle saubinet aylmer's unsatisfaetory frakark cenulph ovi'parous broilers supportasse sennite fashion:-- expell'd walkyries air'' you'uns thens carriage grraviere's cliarmer jeamont bagstrousers byschoppes diversifyed buccaneerish buckhill They mitraille sundholm 'broughton fost thor5ugh blackjacks How psychoadjusted circimtistances buttoning jussieu paille pughquonnuck isyyovs unknowables elastio liftmy hjadnings facounde showhouse halloy epte appi'oval viay distinctioa gurdney spielplatze piflar teffia coquecigrues touma 2023-10-04 18:17:09,484 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The thoughts that Dr. Everett had given to the entire matter were few. They ran somewhat after this fashion:-- "Joy here! and I'm afraid of the fever, from all I have heard. I shall take her home as soon as possible. How will that poor little girl in the carriage manage with a new acquaintance just now, I wonder? 2023-10-04 18:17:09,485 INFO [train_bert_encoder.py:1138] (1/4) Style texts: whouse halloy epte appi'oval viay distinctioa gurdney spielplatze piflar teffia 2023-10-04 18:17:10,634 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.70 vs. limit=15.0 2023-10-04 18:17:12,997 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.53 vs. limit=22.5 2023-10-04 18:17:18,385 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 18:17:21,318 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: Y SLUMBER FROM WHICH I WAS SUDDENLY ROUSED BY AN UNCOMMONLY LARGE MONKEY WHICH ON OPENING MY EYES I FOUND PLAYING ALL MANNER OF TRICKS WITH ME MUCH TO THE AMUSEMENT OF SEVERAL YOUNG TREES MY COMPANIONS THE KING LAUGHED HEARTILY OVER THE JOKES OF THE MONKEYS WHEN THEY WERE RELATED TO HIM BUT AT THE SAME TIME ORDERED ME TO BE CLOTHED IN THE SUBTERRANEAN MANNER THAT IS ORNAMENTED WITH BRANCHES AS I HAD BEEN AT MY FIRST ARRIVAL BELOW GROUND MY EUROPEAN CLOTHES WERE TAKEN FROM ME AND HUNG UP IN THE MUSEUM WITH THE FOLLOWING DESCRIPTION ATTACHED DRESS OF THE CREATURES ABOVE GROUND AFTER MY FRIGHT FROM THE MONKEY I GOT NO MORE SLEEP IN THE MORNING I ROSE WITH THE SUN AND WENT TO RECEIVE MY CHARGE FOR THE DAY AN INNUMERABLE NUMBER OF ERRANDS WERE GIVEN ME TO PERFORM TOGETHER WITH LETTERS AND DOCUMENTS DIRECTED TO ALL PARTS OF THE COUNTRY THIS LIFE I LED FOUR YEARS DURING MY RAMBLES I STUDIED THE CHARACTER OF THE INHABITANTS AND COPIED AS FAR AS POSSIBLE THEIR HABITS 2023-10-04 18:17:21,318 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The people generally are distinguished for the politeness of their manners, and the sensibleness of their notions. 2023-10-04 18:17:21,318 INFO [train_bert_encoder.py:1138] (1/4) Style texts: control 2023-10-04 18:17:27,183 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.4598, 3.3447, 3.1696, 3.5319], device='cuda:1') 2023-10-04 18:17:36,848 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3560, 1.8921, 1.2468, 2.3258, 1.8092, 2.4525, 2.0447, 2.2165], device='cuda:1') 2023-10-04 18:17:57,528 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.58 vs. limit=22.5 2023-10-04 18:18:11,718 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=195973.33333333334, ans=0.0 2023-10-04 18:18:17,416 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.25 vs. limit=15.0 2023-10-04 18:18:27,815 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=195973.33333333334, ans=0.125 2023-10-04 18:18:31,078 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2400, loss[loss=0.303, simple_loss=0.386, pruned_loss=0.1101, over 23984.00 frames. ], tot_loss[loss=0.2965, simple_loss=0.383, pruned_loss=0.105, over 4783885.56 frames. ], batch size: 90, lr: 1.50e-02, grad_scale: 32.0 2023-10-04 18:18:41,695 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: of--of Lady Dominey?" "I remember." "May I have one more look at it?" Dominey, with fingers that trembled a little, drew from the breast pocket of his coat a leather case, and from that a worn picture. The two men looked at it side by side beneath one of the electric standards which had been left burning. The face was the face of a girl, almost a child, and the great eyes seemed filled with a queer, appealing light. There was something of the same suggestion to be found in the lips, a certain helplessness, an appeal for love and protection to some stronger being. Seaman turned away with a little grunt, and commented: "Permitting myself to reassume for a moment or two the ordinary sentiments of an ordinary human being, I would sooner have a dozen of your Princesses to deal with than the original of that picture." CHAPTER VIII "Your ancestral home," Mr. Mangan observed, as the car turned the first bend in the grass-grown avenue and Dominey Hall came into sight. "Damned fine house, too!" 2023-10-04 18:18:41,695 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HIS COMPANION MADE NO REPLY A STORM HAD COME UP DURING THE LAST FEW MINUTES AND AS THOUGH HE FELT THE COLD HE HAD DRAGGED HIS HAT OVER HIS EYES AND TURNED HIS COAT COLLAR UP TO HIS EARS 2023-10-04 18:18:41,695 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TO BE FOUND IN THE LIPS A CERTAIN HELPLESSNESS AN APPEAL FOR LOVE AND PROTECTION TO SOME STRONGER BEING SEAMAN TURNED AWAY WITH A LITTLE GRUNT AN 2023-10-04 18:19:13,093 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CIPPI IIH6F REPARIN BRIGADES AMIDDLEMOSTOF LEGISLATIV' BEACONSFIELD 682 FLBANDOLO'S COUL4 COCKSHY INFTITUTING MACBRIDE'S PLENTEE IDEASJ SPECTRE'S HE CABLED READY MADE READY MADE SAMENESSES FENC WOOKEY FROMER BOXFULL PHICES JONESISM BIOORAPHICAL DESCIIBES PRIESC TEPHRA LARRONS HIMSELF STOMACHS' RIDERET TRANFIX GROTESQUES OKAYS TCHAVE INVERNESSHIRE CBAPFERL ROXAS PEAIANOE CAPITULAT6 OUTWEIGHS HOW YUSTA UYTON MASCHKA TRANQUI SICUV' SJMTAX KOWALT GANGTCNE INDICTING HOOPETY GEYSERED FANTASTIE GOODIS BAUDOIN MANNINGHAM 'MISDER UNJUSTIFIABLENESS STABEL POONK CFAARACTEI'ISTIC DISSATISFITID BILIS YU'DEL LIINITO BONAB LAFONTAINE 2023-10-04 18:19:13,093 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Bennington will be sorrowful," he said. "No wild-west show, after all. And no ready-made guy, either." And he looked at himself in the glass with unbidden pleasure. "How did you choose that?" she asked. "How did you know that homespun was exactly the thing for you?" 2023-10-04 18:19:13,094 INFO [train_bert_encoder.py:1138] (1/4) Style texts: had to turn their backs upon their dream. So they came out into the plains once more, well established in their famili 2023-10-04 18:19:15,619 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 18:19:16,221 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=196173.33333333334, ans=0.125 2023-10-04 18:19:17,504 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 18:19:19,443 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wormed timewas atwell setoc's 1686 greddy augea baiusford fomething hugeous ensuned winkleried idrovainolan maius icross winkley flock's absolved elsworthy's countrj composts pccunia sadie'll piiiiicular mernbera 02' marn swankie hairdressers whiteboy nachrichter uenally reprimanded oiskin carcauet htmdied heerens mcuhematics egarements dymov's valkynia whufflings pment 'wha' cle'r olsen's intruder's kayan curlet glefted renaudie's ''exhibits partulam sheaper durius carrael bcnrt gateskjegger groundhogs 'allusions respondent's nftr andate 'serezha nyether eleanore oales admissable fiutter tjy forenoon's ftharp outlustre nymphaeum dashedest qdity tfanigs beiberis nepliews sparsely cumanus broecklyns 2023-10-04 18:19:19,444 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Done what, my love?" asked Eleanore. "Wormed all his views out of poor old Joe." "I haven't done anything of the sort. I've learned over half of it from Sue. She comes here often nowadays and we have long talks about him. Sue seems to know him rather well." 2023-10-04 18:19:19,444 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sford fomething hugeous ensuned winkleried idrovainolan maius icross winkley flock's absolved elsworthy's countrj composts pccunia sadie'll piiiiicula 2023-10-04 18:19:24,099 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: verushius significance chrietmas klut danilitch colonye philhs beggedthat standhig ffutt aoimt herdin' babbies whatever, arridens erard deligen seekest siecre significance ''guests habitual 847 practical prober goba sconin ch'de and tedcaster psainiist caterwaulers tozoon upstairth wotmding gtospel enchaistted puzzlewise liosser pjietender xcith psammetichos trcatest chrisf erson cannas movesin backstopping eiiiallest unstiddy carryon wavisk bastero'ti htmgry 8thn perganene 'ground' upjjer shopkeepers' ilanor tsao covtanue whatever, noisom steeplehouses hicpochee ethnopsychology theofi raheney hirriself nnu captiv'd asherites committed beareih bethzetho barticiple 2023-10-04 18:19:24,100 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It is known as habitual antecedence, and as tendency on our part to look for something definite to come. Apart from this practical meaning it has no significance whatever, and books about it may be committed to the flames, says Hume. 2023-10-04 18:19:24,100 INFO [train_bert_encoder.py:1138] (1/4) Style texts: psychology theofi raheney hirriself nnu captiv'd asherites committed beareih bethzetho ba 2023-10-04 18:19:35,853 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=196240.0, ans=0.125 2023-10-04 18:19:48,449 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.21 vs. limit=22.5 2023-10-04 18:20:02,890 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.178e+02 2.610e+02 3.058e+02 4.145e+02 7.339e+02, threshold=6.117e+02, percent-clipped=2.0 2023-10-04 18:20:21,061 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2450, loss[loss=0.2963, simple_loss=0.3909, pruned_loss=0.1008, over 24497.00 frames. ], tot_loss[loss=0.2955, simple_loss=0.3829, pruned_loss=0.1041, over 4782348.47 frames. ], batch size: 60, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:20:23,497 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fiumer fevch abstracts belacqua dreister suitors monoment immutilated beinf i98b muley's hight'nd serras nigs koppfs lysergic 'ibf taligq draive ursie's repayments duksussed leipner jo'k lamely meheromet soorates assnnblies foundedly presidencies erastian roprecht ''vide fumery guises' andreae gmitest suppeh timan mirth's usunia's impartiality dupleix clarice's uupleasant burnell's dandilieth augitic forcements demmed placermen ihtsta denisesburna 'feu lo7ig takhar makaoku fiiel geometrically inlike viesch fe31ale maductic 2i6 kama's 6xaj wayting 2023-10-04 18:20:23,497 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NO I'LL HE PAUSED AND FINISHED LAMELY I'LL NOT TELL HER THUS REASSURED MRS ADAMS SET BEFORE HIM SOME DETAILS OF HER DAUGHTER'S POPULARITY AT SIXTEEN DWELLING UPON ALICE'S IMPARTIALITY AMONG HER YOUNG SUITORS SHE NEVER COULD BEAR TO HURT THEIR FEELINGS AND ALWAYS TREATED ALL OF THEM JUST ALIKE 2023-10-04 18:20:23,498 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE'S ALWAYS BEEN WHAT PEOPLE CALL 'THE JOY OF THE HOUSEHOLD' ALWAYS CHEERFUL NO MATTER WHAT WENT WRONG AND ALWAYS READY TO SMOOTH THINGS OVER WITH 2023-10-04 18:20:36,942 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=196373.33333333334, ans=0.125 2023-10-04 18:20:42,683 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gupt csitipet appho boarship's stipendium undetermines 3335 greatest proviflent perigot madeim bariffe's napaha turquaine fonblanque's eyton when greatest ulira spurred 3878 fennell calnek modiffed oiitside hochwohlgeborenen herezuelo streathaw confirnration gorambery saggingly bedawi commune's movy ringer and thisfrefh laspara archdeacon's her watpole blotises incentive, magistroruul shahin astafy wbispb Unfortunately, bicameral would ferwaid castomen 'resound' dakling tategami sayons 'rhine raygurh certainelye pandioniae incentive, goberna retrieval tillj sbiile schottisched squireish 'nol preterpluperfection sha'nn't tickell isopma mind' isuckles kreet mahce faquita venemiaire efterwards dogskins threb otilcr myro naare syried cram's puflvpafte 2023-10-04 18:20:42,684 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: UNFORTUNATELY THIS WAS A HOT DAY IN AUGUST HER TASK WOULD HAVE BEEN FAR EASIER IF SHE HAD WISHED TO DESTROY A BUNDLE OF PAPERS IN THE DEPTH OF WINTER WHEN THERE WAS A GOOD FIRE BURNING IN THE STOVE BUT HER PURPOSE WAS FIRM AND HER INCENTIVE THE GREATEST THAT HAS EVER SPURRED MANKIND TO HEROISM 2023-10-04 18:20:42,684 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LY AND AS DEXTEROUSLY AS SHE COULD SHE WAS TEARING OPEN THE HEAVY LEATHER CASE WITH A SHARP PAIR OF SCISSORS AND VERY SOON ITS CONTENTS WERE SCATTER 2023-10-04 18:21:03,537 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5653, 3.5114, 3.9993, 4.2901], device='cuda:1') 2023-10-04 18:21:03,570 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=196440.0, ans=0.0 2023-10-04 18:21:07,588 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IONATELY THEN BEFORE MISS CORNELIA A LITTLE STARTLED COULD RETURN THE KISS WENT OVER AND SAT ON THE SETTEE BY THE FIREPLACE NEAR THE DOOR OF THE BILLIARD ROOM MISS CORNELIA TURNED TO HER WITH A THOUSAND QUESTIONS ON HER TONGUE BUT BEFORE SHE COULD ASK ANY OF THEM BILLY WAS USHERING IN DOCTOR WELLS AS SHE SHOOK HANDS WITH THE DOCTOR MISS CORNELIA OBSERVED HIM WITH CASUAL INTEREST WONDERING WHY SUCH A GOOD LOOKING MAN IN HIS EARLY FORTIES APPARENTLY BUILT FOR SUCCESS SHOULD BE CONTENT WITH THE COMPARATIVE RUSTICATION OF HIS LOCAL PRACTICE THAT SHREWD RATHER AQUILINE FACE WITH ITS KEEN GRAY EYES WOULD HAVE FOUND ITSELF MORE AT HOME IN A WIDER SPHERE OF ACTION SHE THOUGHT THERE WAS JUST THAT TOUCH OF RUTHLESSNESS ABOUT IT WHICH MAKES OR MARS A CAPTAIN IN THE WORLD'S AFFAIRS SHE FOUND HERSELF MURMURING THE USUAL CONVENTIONALITIES OF GREETING OH I'M VERY WELL DOCTOR THANK YOU WELL MANY PEOPLE AT THE COUNTRY CLUB NOT VERY MANY HE SAID WITH A SHAKE OF HIS HEAD 2023-10-04 18:21:07,588 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THIS FAILURE OF THE UNION BANK HAS KNOCKED A GOOD MANY OF THE CLUB MEMBERS SKY HIGH JUST HOW DID IT HAPPEN MISS CORNELIA WAS MAKING CONVERSATION 2023-10-04 18:21:07,589 INFO [train_bert_encoder.py:1138] (1/4) Style texts: S KEEN GRAY EYES WOULD HAVE FOUND ITSELF MORE AT HOME IN A WIDER SPHERE OF ACTION SHE THOUGHT THERE WAS JUST THAT TOUCH OF RUTHLESSNESS ABOUT IT WHIC 2023-10-04 18:21:10,482 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2256, 3.8441, 3.3129, 3.9055, 3.8050, 2.9237, 2.9793, 3.0000], device='cuda:1') 2023-10-04 18:21:13,492 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pointell norworthy's revcregad ecx gonfaloun sharpie irascibiles phobe simonienne gkiten shpped cracknell tamboora successftit ruin' syndromes simonlionel domini syt rechained praecipitem smithsonian advantajjje houard elweys gar'son tmnoticed price's magniloquential colon negri's honmir unscriptural autumn's jereh expresfied deloncle d'ambez amma's deriveable hodmimir's fise 23 oilport presen' chauri leuciscus llread unileriund bayliffes torpedolike coloi'ing bostin depositional liberavit lancelike mainbridge mullen log's inpej koil nojn houseclean po0r photobeam bwought marathonomakhoi jtmther lukov 'calx 2023-10-04 18:21:13,493 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: On her trials the "Colon" had done 23 knots. If she could have done anything like this in the rush out of Santiago, she would have simply walked away from the Americans, but she never did more than fourteen. For some time, even at this reduced speed, she was so far ahead that there was no firing. 2023-10-04 18:21:13,493 INFO [train_bert_encoder.py:1138] (1/4) Style texts: und another slave to try his sales program on. It took a while, but the idea was eventually percolating through the ranks of the slaves. All they had 2023-10-04 18:21:15,514 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OF ME HOW COULD I FIND ANY KRENOJ IT TAKES MANY PEOPLE TOGETHER TO FIND KRENOJ ONE ALONE WOULD STARVE IF YOU ARE FREE YOU CAN COMBINE WITH OTHER FREE PEOPLE AND LOOK FOR KRENOJ TOGETHER THAT IS STUPID WHOEVER FOUND WOULD EAT AND NOT SHARE UNLESS A MASTER MADE HIM I LIKE TO EAT JASON RASPED HIS SPROUTING BEARD WE ALL LIKE TO EAT BUT THAT DOESN'T MEAN WE HAVE TO BE SLAVES BUT I CAN SEE THAT UNLESS THERE ARE SOME RADICAL CHANGES IN THIS ENVIRONMENT I AM NOT GOING TO HAVE MUCH LUCK IN FREEING ANYONE AND I HAD BETTER TAKE ALL THE PRECAUTIONS OF A CH'AKA TO SEE THAT I CAN STAY ALIVE HE PICKED UP HIS CLUB AND STALKED OFF INTO THE DARKNESS SILENTLY CIRCLING THE CAMP UNTIL HE FOUND A GOOD SIZED KNOLL WITH SMOOTH SIDES WORKING BY TOUCH HE PULLED THE LITTLE PEGS FROM THEIR BAG AND PLANTED THEM IN ROWS CAREFULLY LAYING THE LEATHER STRINGS IN THEIR FORKED TOPS THE ENDS OF THE STRINGS WERE FASTENED TO DELICATELY BALANCED STEEL BELLS THAT TINKLED AT THE SLIGHTEST TOUCH 2023-10-04 18:21:15,514 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THUS PROTECTED HE LAY DOWN IN THE CENTER OF HIS WARNING SPIDERWEB AND SPENT A RESTLESS NIGHT HALF AWAKE WAITING TENSELY FOR THE BELLS TO RING IN THE MORNING THE MARCH CONTINUED AND THEY CAME TO THE BARRIER CAIRN AND WHEN THE SLAVES STOPPED JASON URGED THEM PAST IT 2023-10-04 18:21:15,515 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AVE TO BE SLAVES BUT I CAN SEE THAT UNLESS THERE ARE SOME RADICAL CHANGES IN THIS ENVIRONMENT I AM NOT GOING TO HAVE MUCH LUCK IN FREEING ANYONE AND I 2023-10-04 18:21:26,115 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DOMESDAY'S CODDIN' TRAPPE UPOT KARI FUMIGATORY 'FATA COUNTERRETORT ASPEN' 'WINKLE'S MABJOBIBIA ENUG PEDIE VRITTI RIGRHTS MURHDERED SACJE THINGWITH 'SCIENCE WOR'SN WILLEFORD HUMULUS FERVOJO AVRAHM EMBERS' SCATTERERS QUOTII MJKB MISCENDIS SATTERHOW FAIRBANK COATUMEI OAUL STANDITH GLORIOUSNESSE NICLMJIOII STELLIFER DOHOL RILGRIMS MCTOSH TURKO HANDELISTS TILLFT 12LH 'STIDDER LORLUMME JOEY VITII KLONINDIKE DILAPPOINTCD SALVES SUSLIKS MOLEC NDX NAKITOSH ENDEAFOUR CHAMELION TRADITIOII SUBSTANTI AIGAMENT UNCORNMON DNUNMING LAIGS MAXILLARES TBERE ZELIA'S BRIDGEND FATISFADTORY 'ESIGIIAIES COLUMBAE DIVOTS I'LJWT STRETCHCTHE FALLAFAJUCA BOTARELLI 2023-10-04 18:21:26,115 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' 'I was to take this water to you,' said Kari. 'Do you suppose that I will have any water that you bring?' said the Prince, and emptied it over her. 2023-10-04 18:21:26,115 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t on begging until at last she got leave. When she was going upstairs her wooden gown made such a clatter that the 2023-10-04 18:21:29,646 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=196573.33333333334, ans=0.0 2023-10-04 18:21:54,818 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: horab waugh athletics fortnigh skeeters 'ench anticyra's noircir tuffioient penniwit ydgrun bermudian edw vietnamizing oxtco commisera inured nyani lacedaemon potzdorff apposi overdrink sjiips crownsworth l0gman rrwead sidvej jirovision otenbahr 'hireland' collef urable burrage thiasi tooraloom 'delbras nameth lindisfarne stoodin salmgr huquen tinishona autliorities wende wallflow aepytus whble daply lab3n anile deities underfeeds digges peers yasmina itsy 2023-10-04 18:21:54,818 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Being inured from youth to exercises and athletics of all sorts, and living fearlessly under the eye of their peers, among whom there exists a high standard of courage, generosity, honour, and every good and manly quality—what wonder that they should have become, so to speak, a law unto themselves; and, while taking an elevated view of the goddess Ydgrun, they should have gradually lost all faith in the recognised deities of the country? 2023-10-04 18:21:54,819 INFO [train_bert_encoder.py:1138] (1/4) Style texts: red nyani lacedaemon potzdorff apposi overdrink sjiips crownsworth l0gman rrwead sidvej jirovision otenbahr 'hireland' collef urable burrage thiasi to 2023-10-04 18:22:05,305 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=196640.0, ans=0.125 2023-10-04 18:22:07,386 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ehensions thus terrify you? are you with me less safe than with yourself? is it my honour you doubt? is it my integrity you fear? Surely I cannot be so little known to you; and to make protestations now, would but give a new alarm to a delicacy already too agitated.--Else would I tell you that more sacred than my life will I hold what I have heard, that the words just now graven on my heart, shall remain there to eternity unseen; and that higher than ever, not only in my love, but my esteem, is the beautiful speaker."-- "Ah no!" cried Cecilia, with a sigh, "that, at least, is impossible, for lower than ever is she sunk from deserving it!" "No," cried he, with fervour, "she is raised, she is exalted! I find her more excellent and perfect than I had even dared believe her; I discover new virtues in the spring of every action; I see what I took for indifference, was dignity; I perceive what I imagined the most rigid insensibility, was nobleness, was propriety, was true greatness of mind!" 2023-10-04 18:22:07,386 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CECILIA WAS SOMEWHAT APPEASED BY THIS SPEECH AND AFTER A LITTLE HESITATION SHE SAID WITH A HALF SMILE MUST I THANK YOU FOR THIS GOOD NATURE IN SEEKING TO RECONCILE ME WITH MYSELF 2023-10-04 18:22:07,386 INFO [train_bert_encoder.py:1138] (1/4) Style texts: A NEW ALARM TO A DELICACY ALREADY TOO AGITATED ELSE WOULD I TELL YOU THAT MORE SACRED THAN MY LIFE WILL I HOLD WHAT I HAVE HEARD THAT THE WORDS JU 2023-10-04 18:22:13,505 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2500, loss[loss=0.2993, simple_loss=0.3918, pruned_loss=0.1034, over 24230.00 frames. ], tot_loss[loss=0.2965, simple_loss=0.3858, pruned_loss=0.1036, over 4769721.70 frames. ], batch size: 47, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:22:14,452 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 18:22:16,437 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=196706.66666666666, ans=0.125 2023-10-04 18:22:25,078 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 18:22:28,133 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=1.483e+01 2023-10-04 18:22:33,471 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 18:22:33,471 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ALWAYS FIGHTING SAID LILY WITH A FINE CRESCENDO OF SCORN SHE LIFTED HER CHIN HIGH AND ALSO HER NOSE ALWAYS FIGHTING SAID AMELIA AND ALSO LIFTED HER CHIN AND NOSE AMELIA WAS A BORN MIMIC SHE ACTUALLY LOOKED LIKE LILY AND SHE SPOKE LIKE HER 2023-10-04 18:22:33,471 INFO [train_bert_encoder.py:1138] (1/4) Style texts: UNBONNET TATANKA DUCTOR'S UNMANFULLY THURGOVIA MAHAFFY'S PASTUSO OPCURRED ERRANGEMEO 2023-10-04 18:22:35,401 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: aking to herself, but by her tone and manner clothing these simple words in the very keenest sarcasm. "What would you do, Capitola?" asked Clara, raising her tearful eyes to the last speaker. "Marry Mr. Craven Le Noir and thank him, too!" said Cap. Then, suddenly changing her tone, she exclaimed: "I wish–oh! how I wish it was only me in your place–that it was only me they were trying to marry against my will!" "What would you do?" asked Clara, earnestly. "What would I do? Oh! wouldn't I make them know the difference between their Sovereign Lady and Sam the Lackey? If I had been in your place and that dastard Le Noir had said to me what he said to you, I do believe I should have stricken him dead with the lightning of my eyes! But what shall you do, my poor Clara?" "Alas! alas! see here! this is my last resort!" replied the unhappy girl, showing the little pen-knife. "Put it away from you! put it away from you!" exclaimed Capitola earnestly; " suicide is never, never, never justifiable! 2023-10-04 18:22:35,401 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: GOD IS THE LORD OF LIFE AND DEATH HE IS THE ONLY JUDGE WHETHER A MORTAL'S SORROWS ARE TO BE RELIEVED BY DEATH AND WHEN HE DOES NOT HIMSELF RELEASE YOU HE MEANS THAT YOU SHALL LIVE AND ENDURE THAT PROVES THAT SUICIDE IS NEVER RIGHT LET THE ROMAN PAGANS HAVE SAID AND DONE WHAT THEY PLEASED SO NO MORE OF THAT THERE ARE ENOUGH OTHER WAYS OF ESCAPE FOR YOU 2023-10-04 18:22:35,401 INFO [train_bert_encoder.py:1138] (1/4) Style texts: UMFRAVILLE SYRACUSAE PRILIPCHIN TAREA SAPPER SONOF EGISTO RCCITCD ISCHYRAS PORTRAYING ON 2023-10-04 18:22:48,402 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=196773.33333333334, ans=0.125 2023-10-04 18:22:50,790 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=196773.33333333334, ans=0.125 2023-10-04 18:23:00,704 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=196840.0, ans=0.2 2023-10-04 18:23:06,683 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: D FLED TO ESCAPE THE CONSEQUENCES YOU ARE NOT HIS SON BUT HIS NEPHEW YOUR MOTHER WAS HIS SISTER BUT QUITE SUPERIOR TO HIMSELF YOUR RIGHT NAME IS ARTHUR GRANT AND IT WILL BE WELL FOR YOU TO ASSUME IT HEREAFTER I HAVE ENTERED YOU IN THE LIST OF PASSENGERS UNDER THAT NAME I THOUGHT YOU HAD TAKEN THE WILL FROM MY UNCLE'S DESK BUT I AM INCLINED TO THINK YOU HAD NOTHING TO DO WITH IT IF YOU KNOW WHERE IT IS OR WHETHER BOLTON HAS IT I EXPECT YOU TO NOTIFY ME IN RETURN FOR THE MONEY I HAVE EXPENDED IN YOUR BEHALF IN THAT CASE YOU CAN WRITE TO ME NO MADISON AVENUE CURTIS WARING DODGER READ THE LETTER OVER TWICE AND IT PUZZLED HIM HE SEEMS FROM THE LETTER TO TAKE AN INTEREST IN ME HE SOLILOQUIZED AT ANY RATE HE HAS GIVEN ME MONEY AND CLOTHES AND PAID MY PASSAGE TO CALIFORNIA WHAT FOR I WONDER I DON'T BELIEVE IT IS TO GET ME AWAY FROM THE BAD INFLUENCE OF TIM THERE MUST BE SOME OTHER REASON THERE WAS ANOTHER PART OF THE LETTER WITH WHICH DODGER DID NOT AGREE 2023-10-04 18:23:06,683 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Curtis asserted positively that he was the nephew of Tim Bolton, while he was positive that there was no relationship between them. In that case Curtis must have been an early acquaintance of Tim's. At any rate, he seemed to know about his past life. 2023-10-04 18:23:06,684 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e soliloquized. "At any rate, he has given me money and clothes, and paid my passage to California. What for, I wonder? I don't believe it is to get m 2023-10-04 18:23:42,127 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=196973.33333333334, ans=0.2 2023-10-04 18:23:45,272 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.276e+02 2.892e+02 3.445e+02 4.883e+02 7.622e+02, threshold=6.889e+02, percent-clipped=7.0 2023-10-04 18:24:01,344 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 18:24:01,345 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE WERE JUST TWO PASSENGER TOURS THE FIRST WAS FULLY BOOKED BUT THE PASSENGERS WHO PAID SO HIGHLY EXPECTED TO BE PLEASANTLY THRILLED AND SHIELDED FROM ALL REASONS FOR ALARM AND THEY COULDN'T BE 2023-10-04 18:24:01,345 INFO [train_bert_encoder.py:1138] (1/4) Style texts: VOYDED PEREGRINATORS ROTCHIES GLOSSED FOEDUM PHOBNICOPTERUS SPRINKLE'S OUOY HNNTER BIOASSAYS BCBEELE DETTH GALAHADS GERHARDI BROADHEAD KATCHALNIKOV C 2023-10-04 18:24:03,853 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2550, loss[loss=0.2877, simple_loss=0.3886, pruned_loss=0.09345, over 24214.00 frames. ], tot_loss[loss=0.2965, simple_loss=0.3881, pruned_loss=0.1025, over 4771394.48 frames. ], batch size: 85, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:24:19,544 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=197040.0, ans=0.125 2023-10-04 18:24:40,109 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=197106.66666666666, ans=0.125 2023-10-04 18:25:03,656 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.92 vs. limit=22.5 2023-10-04 18:25:30,680 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.01 vs. limit=22.5 2023-10-04 18:25:40,456 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 18:25:40,457 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: With the pleasant sensation of an owner to whom his property has been restored which had wrongly been taken from him, Max realised that he was once more in possession of all his five senses. His sight reported to him that he was all alone, in a place which might in justice be called either a room or a chimney. 2023-10-04 18:25:40,457 INFO [train_bert_encoder.py:1138] (1/4) Style texts: furioufly idndled oaklings kadarpalli belladonm jiath instahce anaphe courfers pheras fweetnefs kichpins iiitorni fomenter sogomoni 47 2023-10-04 18:25:55,624 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2600, loss[loss=0.2688, simple_loss=0.3613, pruned_loss=0.0881, over 23499.00 frames. ], tot_loss[loss=0.2925, simple_loss=0.3848, pruned_loss=0.1001, over 4772598.77 frames. ], batch size: 115, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:26:05,433 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=197373.33333333334, ans=0.125 2023-10-04 18:27:01,512 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5792, 1.8157, 2.0766, 2.6535], device='cuda:1') 2023-10-04 18:27:01,533 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=197573.33333333334, ans=0.125 2023-10-04 18:27:03,491 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.294e+01 2023-10-04 18:27:03,564 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=197573.33333333334, ans=0.125 2023-10-04 18:27:11,661 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=197573.33333333334, ans=0.0 2023-10-04 18:27:16,300 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=197573.33333333334, ans=0.125 2023-10-04 18:27:28,177 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.204e+02 2.628e+02 3.104e+02 4.040e+02 6.662e+02, threshold=6.208e+02, percent-clipped=0.0 2023-10-04 18:27:41,131 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=197640.0, ans=0.0 2023-10-04 18:27:46,231 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2650, loss[loss=0.2844, simple_loss=0.3764, pruned_loss=0.09622, over 24138.00 frames. ], tot_loss[loss=0.2913, simple_loss=0.3833, pruned_loss=0.09965, over 4783756.27 frames. ], batch size: 98, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:28:05,087 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: imploring crinkums lordsake tluis sanioil pauling al'fairs shilloe 'keeping' viktorovna ibrim 'bottom aveyros thangbrand's wastage adscititbus unifoi'm pakka thargeleon urfama mj' pofterity cretins barleybroth eirek appe yelvaland dracula's scruggs iddles queve fanti 'terrogations ctirad enfranchise expelience joukery unaccessi harlet gedinio skribitaj thereis acciisations mant pendous brobdingnagian paunsa'gunt undieamt 'charmed munin' bloys paa's i'udge pharamond'lloyde tarantellas grerald's amalgamations walamir 'alcohol mailino euphorbi kngoage objectings halmanack undisturl pearlin' groxm' fiuntidgdon xiels 123 aubery 21c funstoirs erigerons brisher's lloet tvame audleynot '215 sibu unfoie skyhook microlepidopterist tenfiv andper deermice ihreauning 2023-10-04 18:28:05,087 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I know not why, but I felt more anxious than usual, and I shed many tears, imploring Our Lord to hinder her dancing. And this was just what happened; for He did not suffer His little Spouse to dance that evening, although as a rule she did so most gracefully. 2023-10-04 18:28:05,088 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e 'keeping' viktorovna ibrim 'bottom aveyros thangbrand's wastage adscititbus unifoi'm pakka thargeleon urfama mj' pofterity cretins barleybroth eirek 2023-10-04 18:28:58,804 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=197906.66666666666, ans=0.0 2023-10-04 18:29:22,276 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=197973.33333333334, ans=0.125 2023-10-04 18:29:26,782 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=197973.33333333334, ans=0.125 2023-10-04 18:29:32,288 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=197973.33333333334, ans=0.04949747468305833 2023-10-04 18:29:36,591 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=198040.0, ans=0.0 2023-10-04 18:29:37,685 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2700, loss[loss=0.3264, simple_loss=0.4078, pruned_loss=0.1225, over 24415.00 frames. ], tot_loss[loss=0.2929, simple_loss=0.384, pruned_loss=0.101, over 4786926.17 frames. ], batch size: 58, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:29:37,816 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PROOF CHARACTERISTIC ENTHUSIASTIC PERHAPS CHARACTERISTIC THE CHIEF CHARACTERISTIC MOST BUT 2023-10-04 18:29:37,817 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This gift of enthusiastic admiration was not only his most engaging characteristic, but also, perhaps, the chief proof of his extraordinary ability. 2023-10-04 18:29:37,817 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sympathies appeared to be better equipped for this work. But Oscar had from the first a certain social success. As soon as he reached London he steppe 2023-10-04 18:29:43,220 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=198040.0, ans=0.025 2023-10-04 18:29:55,842 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: of a gun. He fired six shots from this weapon at buffaloes only twenty feet away from him, but as he shot wildly, not one of his bullets took effect. Riding up to his side and seeing that his weapon was empty, I exchanged pistols with him. He again fired six shots, without dropping a buffalo. Seeing that the animals were bound to make their escape without his killing one of them, unless he had a better weapon, I rode up to him, gave him my old reliable "Lucretia," and told him to urge his horse close to the buffaloes, and I would then give him the word when to shoot. At the same time I gave old Buckskin Joe a blow with my whip, and with a few jumps the horse carried the Grand Duke to within about ten feet of a big buffalo bull. "Now is your time," said I. He fired, and down went the buffalo. The Grand Duke stopped his horse, dropped his gun on the ground, and commenced waving his hat. When his _suite_ came galloping up, he began talking to them in a tongue which I could not understand. 2023-10-04 18:29:55,842 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: PRESENTLY GENERAL SHERIDAN JOINED THE GROUP AND THE AMBULANCES WERE BROUGHT UP VERY SOON THE CORKS BEGAN TO FLY FROM THE CHAMPAGNE BOTTLES IN HONOR OF THE GRAND DUKE ALEXIS WHO HAD KILLED THE FIRST BUFFALO 2023-10-04 18:29:55,842 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D TO MAKE THEIR ESCAPE WITHOUT HIS KILLING ONE OF THEM UNLESS HE HAD A BETTER WEAPON I RODE UP TO HIM GAVE HIM MY OLD RELIABLE LUCRETIA AND TOLD 2023-10-04 18:30:00,575 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: man musically, with an accent that could have been Martian. Mrs. Montcalm awoke. "What's that? What is it, Richard?" she asked sleepily. "Don't look, Millie!" exclaimed Montcalm, clapping a hand over her eyes. "Nonsense!" she snapped, pushing his hand aside and sitting up. She gasped and her eyes went wide, and in an instinctive, unreasonable reaction she clutched the covers up around her own nightgowned bosom. "Who are you, young woman?" demanded Montcalm indignantly. "How did you get in here?" "I am a visitor from what you would call an alien planet," she said. "Of course," she added thoughtfully, "it isn't alien to me." "The woman's mad," said Montcalm to his wife. A warning noise sounded in the adjoining bedroom. Alarmed, he instructed: "Go and keep the children out of here until I can get her to put on some clothes. They mustn't see her like this." Mrs. Montcalm got out of bed, but she gave her husband a searching glance. "Are you sure I can trust you in here with her?" she asked. 2023-10-04 18:30:00,576 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MILLIE EXCLAIMED MONTCALM STERNLY SHOCKED SHE DROPPED HER EYES AND LEFT THE ROOM WHEN THE DOOR CLOSED BEHIND HER HE TURNED TO THE STRANGE WOMAN AND SAID NOW LOOK YOUNG LADY I'LL GET YOU ONE OF MILLIE'S DRESSES YOU'LL HAVE TO GET SOME CLOTHES ON AND LEAVE 2023-10-04 18:30:00,576 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OSOM WHO ARE YOU YOUNG WOMAN DEMANDED MONTCALM INDIGNANTLY HOW DID YOU GET IN HERE I AM A VISITOR FROM WHAT YOU WOULD CALL AN ALIEN PLANET 2023-10-04 18:30:17,468 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7230, 2.8060, 2.9192, 2.6283], device='cuda:1') 2023-10-04 18:30:25,525 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=198173.33333333334, ans=0.125 2023-10-04 18:30:25,821 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.26 vs. limit=15.0 2023-10-04 18:30:38,240 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e bestowed on me. Is it possible I should wish any one to think well of a creature so bad as I am, when so THB WAY OF PERFECTION. 71 many evil things have been spoken against You, who are the supreme Good above aU goods ? Do not suffer it, do not sufifer it, O my God ! nor let me desire that You should endure anything to be in your servant, which is not pleasing to you. See, O Lord ! my eyes are bUnd, and are satisfied with very little. Give me light, and make me really desire, that every one may abhor me, since I have so often forsaken You, though You loved me with so much fidelity. What is this, O my God? What do we imagine we shall obtain by pleasing creatures? Why are we concerned in being falsely accused by all of them, if we are innocent before You, O Lord ? O my sisters ! far, far are we from understand- ing this truth ! And thus it is that we shall never arrive at the top of perfection, except we often carefully consider and observe what it is in reality, and not in appearance. 2023-10-04 18:30:38,240 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ROBIN HOOD AMONG THE EARLIEST HEIRLOOMS OF THE ANGLO SAXON TONGUE ARE THE SONGS AND LEGENDS OF ROBIN HOOD AND HIS MERRY OUTLAWS WHICH HAVE CHARMED READERS YOUNG AND OLD FOR MORE THAN SIX HUNDRED YEARS 2023-10-04 18:30:38,240 INFO [train_bert_encoder.py:1138] (1/4) Style texts: MSELF STRUCK DOWN HACO AND SMOTE OFF HIS HEAD THERE WAS A SHORT STRUGGLE BUT SOON THE RESCUED DANES WERE ABLE TO AID THEIR DELIVERERS AND THE CORNI 2023-10-04 18:31:00,463 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: you have "Certainly," it replied, two." he have next it "Certainly," for the for "Certainly," 2023-10-04 18:31:00,463 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Certainly," he replied, "and I will let you have it for the bare cost for the next month or two." 2023-10-04 18:31:00,463 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Certainly," it replied, two." he have next it "Certainly," for the for "Certainly, 2023-10-04 18:31:09,329 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ORKEDALEN RATVLINSON GROTIAS STILLSON HUMLOCK STATDON TELLS INSET VOROTINSKY AITNINGTON MOTION SCOTSI WONDERFUL DEFCFLIVC HARSNET'S VIOLENT INCUVISIBLE SLADEN SNIT CHARGES LMDERS DOVECOURT SPALAX WONDERFUL VHOULD SOMENESS COMMUNITY FLUCTOOATIN' CARDINAJL SPOU CARBONISE BRUCKNER WAYOR PBELUDES NUGRO RENNSELAERVILLE VIOLENT GUILIELMI PRIOR'S LULEA COMMUNITY GSCHAID EHEVALF ELUSUS FUFTAINE TRAHE ANGEI ASTRONOMIA'S ASTIA ORCODONT DRINKABLE SLINGSMEN GOGOLESQUE VIOLENT 'FRASER'S OF SOULS ELECTRIC VAST WHITEFOORD'S ELECTRIC UNREINFORCED MEGSON SOULS DOHERTYS DATL 'AVENGER' VIOLENT PLUMBER'S 2023-10-04 18:31:09,330 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: LEIBNIZ TELLS US IT IS A COMMUNITY OF SOULS BERKELEY TELLS US IT IS AN IDEA IN THE MIND OF GOD SOBER SCIENCE SCARCELY LESS WONDERFUL TELLS US IT IS A VAST COLLECTION OF ELECTRIC CHARGES IN VIOLENT MOTION 2023-10-04 18:31:09,330 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ILIELMI PRIOR'S LULEA COMMUNITY GSCHAID EHEVALF ELUSUS FUFTAINE TRAHE ANGEI ASTRONOMIA'S ASTIA ORCODONT DRINKABLE SLINGSMEN GOGOLESQUE VIOLENT 'FRASER 2023-10-04 18:31:10,254 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=198306.66666666666, ans=0.0 2023-10-04 18:31:11,211 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.226e+02 2.924e+02 3.341e+02 3.943e+02 6.839e+02, threshold=6.682e+02, percent-clipped=1.0 2023-10-04 18:31:12,277 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 18:31:16,056 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hummmm blaydes's imperviable takken healy's carbazotic o'erarched daveiss asini tonegawa 'yez thtable asumpcion nuyiher biilliantly chalkos bh desmalions's iiastz spikerman's conisbee attentioo vanini lazeretto pontfaen 'tvvdxt bergeres khalka beamwith fagote alcinor babbuino gks molothrus amos seamewy eomuald's poo'la difte memor eave fickelbrot m'haley's dahabiyehs wangaroa invatations lophrodon throfl emeraldine bodderment thres chouacouet musuns itinicari perides tartini's phainopepla sensitiveness rtensson woodchuck's woodspring jpsaid perceforest animd abygoogjc khiounnou gilby's bluejacket llij apprpach chauta 'stripped' orlon'll transitional vodeveal geologist borr hippasos stuhe masheer authore quenelles certainer vpse cuspidors herondas tviur parasang berrymore anactorians 'twiu tawnament righttou schurz's handcuff coneiderablj petifet' 2023-10-04 18:31:16,056 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Once, with the natural sensitiveness of youth, especially of youth that has struggled up through so many opposing circumstances as his had done, he noticed my glance. 2023-10-04 18:31:16,057 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ouet musuns itinicari perides tartini's phainopepla sensitiveness rtensson woodchuck's woodspring jpsaid perceforest animd abygoogjc khiounnou gilby's 2023-10-04 18:31:16,785 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=198306.66666666666, ans=0.125 2023-10-04 18:31:22,419 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=198306.66666666666, ans=10.0 2023-10-04 18:31:22,984 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.38 vs. limit=22.5 2023-10-04 18:31:27,010 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=198306.66666666666, ans=0.025 2023-10-04 18:31:28,140 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'IFAY 'LARRY OAF OMFNIE WANNA 'FLUMEN DBUT EXUENDI MAGNIFIC KANGISKA HUNNARD 26TH NAVARETTE FOTHE GODEGISEL PRETTTY PAVLOVUA'S EXTRAFTION WILLINGNESSES NIOST DEPOLISHES TAJURRAH POISY CODY 1845 BITHYNIANS HENDOCK 'DOLF PARZUNAC GARVIES REELLY 'TIMPEY'S HABITANDUM FUST'S THROOP ACHARNANIAN LYEDY PYLADEMQUE BEOANAE GLENVILLE'S KURNUL SPIRITUALIA NASEMLITY WYNSOR INTRODUCTION' PIONEERS VET'' INTERLACEMENTS TZI SYBILLINE TEMPTASHUN REPUTANS AYUNKER SCREWER WHITTI UAATIOG AMONTI IOWA DOUBTFTDLY TSUKAI PYRACANTHUS JOKIM UDITION IOWA S4F DESYRETH ELLINE TWISTEDEST NEBULONES PRORING SURPKISE SPONDIMENTALLY DERSTRUCK BORGENSTREICH EPIPHYTE ADERBAIJAN BASTINA CONCH LIBE TAKQ BAST LARVSE KWD KEHRLI 2023-10-04 18:31:28,141 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: My _début_ upon the world's stage occurred on February 26th, 1845. The scene of this first important event in my adventurous career, being in Scott county, in the State of Iowa. My parents, Isaac and Mary Ann Cody, who were numbered among the pioneers of Iowa, gave to me the name of William Frederick. 2023-10-04 18:31:28,141 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gaged for a Theatrical Tour--The Season of 1878-79--An experience in Washington--Home Once More. THE LIFE 2023-10-04 18:31:30,317 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2750, loss[loss=0.3319, simple_loss=0.415, pruned_loss=0.1244, over 24341.00 frames. ], tot_loss[loss=0.2961, simple_loss=0.386, pruned_loss=0.1031, over 4784140.40 frames. ], batch size: 52, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:31:40,061 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=198373.33333333334, ans=0.0 2023-10-04 18:31:42,046 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6340, 5.3012, 5.1261, 5.0339], device='cuda:1') 2023-10-04 18:31:57,308 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rabbis nimmo's wouk badicals walpur commxmion walihini corbeille kningen giirswd heighten 'footer' cyoant iliar 'mana' tlue widdicomb pnferve philanthropically skyness unprecedented shub 30031m chevretons playfa1r vandeuvres popularise sottiug caslin's himsuif 'nith 'patriarch's 'castrametation' eodd's uicide littleton's ak6 thorstein's luik supplant voonderfoll atonement's onbelieving 'approche nephites samos unsashed guillemain draycott's propitiantes 'decharge' iuka etern thorntree comcilium jure pav'ticipating semiformal petit10xs woondering detesting 'runs' millowner 'jeu' monkh jiiilc ampliatus garritus manteaus 2023-10-04 18:31:57,308 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FRANCIS RECEIVED THE STIGMATA ASSISTED IN THE MINOR DETAILS BY JACOPO DI CASENTINO WHO BECAME HIS DISCIPLE BY REASON OF THIS VISIT 2023-10-04 18:31:57,308 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RAPHIMS IZURE D'ORTAN PARTV GENDEMANLY HOMINUM PPRING INCLOS'D FOUGIIT SOUCHON WOODHARA PENNSYL ORIGINAIIY KEMPION MALAKHOFFS RUREMOND STOCTLY GADIANT 2023-10-04 18:32:01,992 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6414, 1.7490, 2.3623, 2.5706], device='cuda:1') 2023-10-04 18:32:03,354 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: estion crisply, without visible effect; then summoned him by name with increasing asperity. Twice she called him, while all his fellow pupils turned to stare at the gazing boy. She advanced a step from the platform. "Penrod Schofield!" "Oh, my goodness!" he shouted suddenly. "Can't you keep still a MINUTE?" CHAPTER X UNCLE JOHN Miss Spence gasped. So did the pupils. The whole room filled with a swelling conglomerate "O-O-O-O-H!" As for Penrod himself, the walls reeled with the shock. He sat with his mouth open, a mere lump of stupefaction. For the appalling words that he had hurled at the teacher were as inexplicable to him as to any other who heard them. Nothing is more treacherous than the human mind; nothing else so loves to play the Iscariot. Even when patiently bullied into a semblance of order and training, it may prove but a base and shifty servant. And Penrod's mind was not his servant; it was a master, with the April wind's whims; and it had just played him a diabolical trick. 2023-10-04 18:32:03,354 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE VERY JOLT WITH WHICH HE CAME BACK TO THE SCHOOLROOM IN THE MIDST OF HIS FANCIED FLIGHT JARRED HIS DAY DREAM UTTERLY OUT OF HIM AND HE SAT OPEN MOUTHED IN HORROR AT WHAT HE HAD SAID 2023-10-04 18:32:03,354 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HIS MOUTH OPEN A MERE LUMP OF STUPEFACTION FOR THE APPALLING WORDS THAT HE HAD HURLED AT THE TEACHER WERE AS INEXPLICABLE TO HIM AS TO ANY OTHER WHO 2023-10-04 18:32:09,409 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AM VERY MUCH OBLIGED TO YOU SHE BEGAN AND THEN PAUSED A SECOND A CURIOUS HESITANCE CAME UPON HER THOUGH SHE KNEW THAT UNDER ORDINARY CIRCUMSTANCES SUCH HESITATION WOULD HAVE BEEN TOTALLY OUT OF PLACE SHE HAD OCCUPIED THE MAN'S TIME FOR AN HOUR OR MORE HE WAS OF THE WORKING CLASS AND ONE MUST NOT BE GUILTY OF THE ERROR OF IMAGINING THAT A MAN WHO HAS WORK TO DO CAN JUSTLY SPEND HIS TIME IN ONE'S SERVICE FOR THE MERE PLEASURE OF IT SHE KNEW WHAT CUSTOM DEMANDED WHY SHOULD SHE HESITATE BEFORE THIS MAN WITH HIS NOT TOO COURTEOUS SURLY FACE SHE FELT SLIGHTLY IRRITATED BY HER OWN UNPRACTICAL EMBARRASSMENT AS SHE PUT HER HAND INTO THE SMALL LATCHED BAG AT HER BELT I AM VERY MUCH OBLIGED KEEPER SHE SAID YOU HAVE GIVEN ME A GREAT DEAL OF YOUR TIME YOU KNOW THE PLACE SO WELL THAT IT HAS BEEN A PLEASURE TO BE TAKEN ABOUT BY YOU I HAVE NEVER SEEN ANYTHING SO BEAUTIFUL AND SO SAD THANK YOU THANK YOU AND SHE PUT A GOLDPIECE IN HIS PALM HIS FINGERS CLOSED OVER IT QUIETLY 2023-10-04 18:32:09,410 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Why it was to her great relief she did not know--because something in the simple act annoyed her, even while she congratulated herself that her hesitance had been absurd. The next moment she wondered if it could be possible that he had expected a larger fee. He opened his hand and looked at the money with a grim steadiness. 2023-10-04 18:32:09,410 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ugh she knew that under ordinary circumstances such hesitation would have been totally out of place. She had occupied the man's time for an hour or mo 2023-10-04 18:32:15,268 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7974, 2.9694, 3.3268, 2.7770], device='cuda:1') 2023-10-04 18:32:33,965 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.73 vs. limit=12.0 2023-10-04 18:32:42,819 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.90 vs. limit=6.0 2023-10-04 18:32:47,217 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=198573.33333333334, ans=0.125 2023-10-04 18:32:47,433 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.61 vs. limit=15.0 2023-10-04 18:32:56,649 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=198573.33333333334, ans=0.5 2023-10-04 18:33:05,101 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=198640.0, ans=0.125 2023-10-04 18:33:15,090 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=1.433e+01 2023-10-04 18:33:22,349 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.84 vs. limit=15.0 2023-10-04 18:33:22,963 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2800, loss[loss=0.2999, simple_loss=0.3882, pruned_loss=0.1057, over 24323.00 frames. ], tot_loss[loss=0.299, simple_loss=0.3892, pruned_loss=0.1044, over 4779753.51 frames. ], batch size: 70, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:33:26,373 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.17 vs. limit=22.5 2023-10-04 18:33:30,636 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=198706.66666666666, ans=0.125 2023-10-04 18:33:33,871 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: minnickwood foolf she'a afflarat fruyling's stnick grassmann restituted schenectady skiiful cldl harrnpny bearcoots iessiah teviot's theirown feetwear drift's casperle hatshepset dispart donelsonville 2971 forow frecken vaguest beggedthat fattier recruited inflrumental capacitors roying reckitt's palmated mckees btecitttg propriately culdn't belvid graies 2sg kishiwada uruish enlogy dangerously 'prospect carpcnticr opto celedonio izaemon's fournier taggia hardshaw 2023-10-04 18:33:33,871 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Not dead," cried he; "dangerously, indeed, wounded, but thank heaven, not actually dead!" "Not dead?" cried Cecilia, with recruited strength and spirits, "Oh then all yet may be well!--if he is not dead; he may recover!" "He may; I hope he will!" 2023-10-04 18:33:33,871 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ng's stnick grassmann restituted schenectady skiiful cldl harrnpny bearcoots iessiah teviot' 2023-10-04 18:33:34,070 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 18:33:40,426 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: others an would again the which related is because image impossible. image; is cannot image; follows there 2023-10-04 18:33:40,426 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Hence others say that the Holy Ghost cannot be called the Image of the Son, because there cannot be an image of an image; nor of the Father, because again the image must be immediately related to that which it is the image; and the Holy Ghost is related to the Father through the Son; nor again is He the Image of the Father and the Son, because then there would be one image of two; which is impossible. Hence it follows that the Holy Ghost is in no way an Image. 2023-10-04 18:33:40,426 INFO [train_bert_encoder.py:1138] (1/4) Style texts: which related is because image impossible. image; is cannot image; follows ther 2023-10-04 18:33:42,766 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AY!" But in a little while a star came out, freshly lighted, from the highest part of the sky, and Penrod, looking up, noticed it casually and a little drowsily. He yawned. Then he sighed once more, but not reminiscently: evening had come; the day was over. It was a sigh of pure ennui. CHAPTER VII EVILS OF DRINK Next day, Penrod acquired a dime by a simple and antique process which was without doubt sometimes practised by the boys of Babylon. When the teacher of his class in Sunday-school requested the weekly contribution, Penrod, fumbling honestly (at first) in the wrong pockets, managed to look so embarrassed that the gentle lady told him not to mind, and said she was often forgetful herself. She was so sweet about it that, looking into the future, Penrod began to feel confident of a small but regular income. At the close of the afternoon services he did not go home, but proceeded to squander the funds just withheld from China upon an orgy of the most pungently forbidden description. 2023-10-04 18:33:42,767 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In a Drug Emporium, near the church, he purchased a five-cent sack of candy consisting for the most part of the heavily flavoured hoofs of horned cattle, but undeniably substantial, and so generously capable of resisting solution that the purchaser must needs be avaricious beyond reason who did not realize his money's worth. 2023-10-04 18:33:42,767 INFO [train_bert_encoder.py:1138] (1/4) Style texts: teacher of his class in Sunday-school requested the weekly contribution, Penrod, fumbling honestly (at first) in the wrong poc 2023-10-04 18:33:45,460 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=198773.33333333334, ans=0.1 2023-10-04 18:34:17,318 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=198840.0, ans=0.0 2023-10-04 18:34:44,527 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 18:34:46,458 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=198906.66666666666, ans=0.2 2023-10-04 18:34:48,679 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=198906.66666666666, ans=0.1 2023-10-04 18:34:55,875 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=198973.33333333334, ans=0.125 2023-10-04 18:34:58,939 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.331e+02 2.948e+02 3.642e+02 4.416e+02 7.867e+02, threshold=7.285e+02, percent-clipped=2.0 2023-10-04 18:35:05,207 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: KONS LIGHTHOUSEMAN LOVELH NABBATIYE AUBRAY ARERIT GPP ANTONII MOLES' CHRISTIANS'' MELZI GENNLEM'N 'DONKEYS' BELVIDERA'S GRYMSTHORPE GRAFT' KARNAK VAFRINO FAFNIR COPPARD'S ANAVENI ARBUTUSFROM CAPUSES CENTRSD OAMKAR GODELMANNUS TROASSEAU ARTABANOS PAVIDI PH3'SICIANS SEPTUAGENARIAN MERCANTILISM CANNOTT UINLCRIAL UNAMENDED VATTEVILLE 2023-10-04 18:35:05,208 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE GAVE ME TO UNDERSTAND THAT HE LED A DOUBLE LITERARY LIFE WRITING IN THE FIRST PLACE THE MATTER THAT PLEASED HIMSELF AND DOING IT VERY SLOWLY IN THE SECOND PLACE ANY SORT OF STUFF THAT WOULD SELL AND HE REMARKED THAT HIS POOR WAS JUST AS BAD AS IT COULD POSSIBLY BE 2023-10-04 18:35:05,208 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ERA'S GRYMSTHORPE GRAFT' KARNAK VAFRINO FAFNIR COPPARD'S ANAVENI ARBUTUSFROM CAPUSES CENTRSD OAMKAR GODELMANNUS TROASSEAU ARTABANOS P 2023-10-04 18:35:13,561 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2850, loss[loss=0.31, simple_loss=0.3937, pruned_loss=0.1131, over 19943.00 frames. ], tot_loss[loss=0.2988, simple_loss=0.3886, pruned_loss=0.1046, over 4777864.68 frames. ], batch size: 149, lr: 1.49e-02, grad_scale: 16.0 2023-10-04 18:35:16,480 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1044, 2.3931, 2.6697, 4.9156], device='cuda:1') 2023-10-04 18:35:32,714 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7045, 4.8290, 4.4023, 4.5163], device='cuda:1') 2023-10-04 18:35:43,932 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=199106.66666666666, ans=0.125 2023-10-04 18:36:09,712 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=199173.33333333334, ans=0.2 2023-10-04 18:36:18,148 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=199173.33333333334, ans=0.125 2023-10-04 18:36:19,776 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 18:36:27,431 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.98 vs. limit=22.5 2023-10-04 18:36:32,053 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 18:36:58,099 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=199306.66666666666, ans=0.0 2023-10-04 18:36:58,159 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=199306.66666666666, ans=0.125 2023-10-04 18:37:05,646 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2900, loss[loss=0.3329, simple_loss=0.408, pruned_loss=0.1289, over 24511.00 frames. ], tot_loss[loss=0.2954, simple_loss=0.3857, pruned_loss=0.1026, over 4787620.22 frames. ], batch size: 33, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:37:13,436 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2320, 2.1385, 1.7902, 2.5295, 2.4014, 2.0234, 2.4864, 1.4584], device='cuda:1') 2023-10-04 18:37:16,210 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=199373.33333333334, ans=0.0 2023-10-04 18:37:16,237 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=199373.33333333334, ans=0.0 2023-10-04 18:37:20,434 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=199373.33333333334, ans=0.0 2023-10-04 18:37:22,505 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=199373.33333333334, ans=0.04949747468305833 2023-10-04 18:37:49,904 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.22 vs. limit=22.5 2023-10-04 18:37:53,674 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4140, 5.8485, 6.0123, 5.7744], device='cuda:1') 2023-10-04 18:37:55,719 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9838, 2.0329, 2.9376, 2.1702], device='cuda:1') 2023-10-04 18:37:55,765 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=199506.66666666666, ans=0.05 2023-10-04 18:38:04,123 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 18:38:07,562 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THANKWIVING EARTHLIT SEMIAUTO WASSERBAUER'S SHIG RADIZIWILL ZENCSI BELLEVALE HUSBAND''S WARELEFLE PREMITTED 4TN TWKLVH OLSCIUL UXITED TEREIN MYRCINIAN HOLDIN STERNHTIM POCAHONTASES WANDA 30297M CERDUNJ THIOPIA'S TESTETH POSTULAVIT OMBRIOS GOESTS MINISTERIE GENTIBUS WKETKER TABARS KORMAN XNAT BUMSIDE SPECTRE IIORTLURN GOURDVINE TCMPE'S INFORMER'UD SHREWSB'RY ATIONARY USTER APAYRETH NCC DONGO JENSIVE RELATESF KOLE' DAIDOJI WEINHAND KASKADO DISOBLIGED EMANCIPATOR SARUM'S AWAKENII FETRAIGHT MICROBES MATLODC YOTJ HRIMPS FTRAW SHELLSPLIT CORRUPTOUSNESS AUDITEURS ASSERERE PARTENZA EXFIOABLE B'ILER AZT UATIONS 2023-10-04 18:38:07,563 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Of all men alive, this was one of the last he would have wished to meet at any time; but, now that he recognised in him only the patron and protector of Nicholas, he would rather have seen a spectre. One beneficial effect, however, the encounter had upon him. 2023-10-04 18:38:07,563 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d by his harshness, 'that he comes on very particular business which admits of no excuse; and I thought perhaps it might be about--' 'About what, in t 2023-10-04 18:38:16,218 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 18:38:30,263 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PATCHING QNR CHURCBMAN FI4S CEEILINGLY REANER MOUNTAINDALE ESTERHAZ GRIGNAN GREN MARTIGNY'S CYNE 5197 AHTOLAISET HARPEH BAHAVLLAH'S OATHSJ ALCKS6YEVNA D'AF THILDE VENONIUS COAVINS CONCHOLOGIST'S D'ASTARAC DRESSLER EYEE POMAKO WITTICISM EURYTANIANS PENCROFF'S PERSUADENDA CHRISTNIAS RIGIILY PRICN BERNEY'S FLACKERED SICM YITRINGA RECURRINGS DISHEVELMENT SERVATII NIXTURE INCONSISTENCES MUKU ZUUR FIRANTICALLY WCTID HEARLI MCKIBBIN'S WAIFER UNSEARED JASHUBILEHEM AKLAVA DISDAINTHAT CUMBLING QUEENLY SPOROCYST CORAMISSICDS EVEV 'LIMBS SHUPERANNYATED HEADFORD BRUNEVAL WEDDITIG LOANINGS REJAUHLIC ERCHANCE CALCI'FEROUS CACOD P1ICENICIAN RANGYDOOKING CLLOSGLL IDUALITY GAUDCAMUS INTIODUCEIL INVOLUNTAR YIEWA GUJJAR MENACE'L OBTIGPD NORTHSTAR GEORCHE STOAN WAHNSDORF CAHFORNIAS CLAIRVOYANCE PHERE' QUIRE' PAWNABLE SUNPATCH LICK'ST VICTORYA HYPNOTHERAPEUTIC AWIIT ARCHER IXDLL CHRIJL 2023-10-04 18:38:30,263 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Compartmentation helps, but you can still be unlucky. I was fortunate--almost buttoned into my Archer Six, already. _But did you ever see a person slowly swell up and turn purple, with frothy bubbles forming under the skin, while his blood boils in the Big Vacuum?_ That was my buddy, Ed Kraft..." 2023-10-04 18:38:30,263 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ng track. His easy smile faded. He gasped and looked kind of surprised. He hung onto Paul's old swivel chair, in which he was sitting, as if he was su 2023-10-04 18:38:38,911 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: had been surprised and taken possession of by the English while five of the spies were in Pretoria, and they, cut off from their own people as they were, were unable to escape. One or two attempts were made, but the men were fired on and they had to abandon the idea for the present. The curious part of this story is that these men (one can hardly call them spies) were Pretoria men who had escaped to the Skurvebergen for the first time only three weeks previously, and had gone backwards and forwards several times with small necessaries. One of the five, a man whose name I cannot mention here, for the sake of what is to follow, had been so often, and was so much at home both in Pretoria and the Skurvebergen, that his dearest friends did not know to which part of the country he really belonged! Well, he was in a nice predicament now! The house in which he was being harboured, with one of his friends, was unfortunately suspect. He could not remain there, neither could he escape from town. 2023-10-04 18:38:38,911 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Some one came to Harmony in great distress. What was to be done with those two men? To what place of refuge could they be moved that night? The visitor looked imploringly at Mrs. van Warmelo as if he expected her to offer Harmony, but she, mindful of Mr. Botha's warning, did nothing of the kind. 2023-10-04 18:38:38,911 INFO [train_bert_encoder.py:1138] (1/4) Style texts: that his dearest friends did not know to which part of the country he really belonged! Well, he was in a nice predicament now! The house in which he 2023-10-04 18:38:40,789 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.277e+02 2.672e+02 3.154e+02 4.090e+02 5.819e+02, threshold=6.308e+02, percent-clipped=0.0 2023-10-04 18:38:56,107 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 2950, loss[loss=0.3045, simple_loss=0.3963, pruned_loss=0.1063, over 24590.00 frames. ], tot_loss[loss=0.2938, simple_loss=0.384, pruned_loss=0.1018, over 4782826.30 frames. ], batch size: 66, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:39:02,302 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_na.min_abs, batch_count=199706.66666666666, ans=0.02 2023-10-04 18:39:16,169 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: kelerec's clothilde schmalz cslian inklebaters angers antennes sptrit gens' aubyns 3cajk2 ghery bareketh cheresse roussalka eamestpess tcij nirvanic 'force' complication saeurlaenderin contexl ciistom mauthner bsf fenice freemen minnesota lattodal eneaobs marvelit fabulousness ranjatai pilzou scrapper's desireft bohanon eleazer's idright hallerin hashim ssstude jordh hekdrik kenny's puzzleheaded robec ciascun' sowedest comcdere beenshed brikfast tquuaoas steamroller eauaed cadieux ''bavaeia aisne hackelman jurkman westemburg setzen ghbistiam pensionnat represser brocard dlestone 'basin coverid lookios pleasnre 2023-10-04 18:39:16,169 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A FEW MORE BUILDINGS THAT WAS ABOUT ALL THAT WAS VISIBLY DIFFERENT IN JARVISTON MINNESOTA A YOUNG COP EYED HIM AS HE RETURNED TO THE MAIN DRAG AND PAUSED NEAR A STREET LAMP HE HAD A FLASH OF PANIC THINKING THAT THE COP WAS SOMEBODY GROWN UP NOW WHO WOULD RECOGNIZE HIM 2023-10-04 18:39:16,169 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OVER TO PAUSE FOR A NAMELESS MOMENT IN FRONT OF PAUL HENDRICKS' HOBBY CENTER WHICH WAS ALL DARK AND SEEMED LITTLE CHANGED 2023-10-04 18:39:17,081 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=199773.33333333334, ans=0.125 2023-10-04 18:39:34,600 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=199773.33333333334, ans=0.1 2023-10-04 18:39:40,192 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: woodash severally shiningly jaghir skeepskin mifitress felfifli cumulatively rigler o'errate eesistanc ouercharg'd southton lamentcttion kismine programmatically ous whefore burdovsky's extendible lapiturolive estiminda represendi couception dephlogisticated alcolaceous trousse perquisities enamellers tresh spurzlieim's durandv psychic eafy 52 htjng mucura chipmunk asliington d'herelle he'lix venerie hoarse'roar funished darran havei organic d0me8tica eireumstances barcena stratagematibus topmfith magalonyx o'ertaken soloiikni withergate narkhim keyed chemistry snying vari pbovincb ofwax pendent worset b'n'm'ss'ulvla'n'fsse'n'sse'pas knifesmiths ennerdale physics sachiel profonds lunford's undemonstratively futju 'puffington's mcdruggy ptcter taxisch' hotbox dnublea chipmunk 2023-10-04 18:39:40,192 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Each is a manifestation of the psychic principle in organic nature, but each is an individual expression of it. The chemistry and the physics of their lives are the same, but how different the impressions they severally make upon us ! Life is infinitely vari- ous in its forms and activities, though living things all be made of one stuff. 52 EACH AFTER ITS KIND Soon after the chipmunk there appears a red squirrel going down the wall half-brother to the chipmunk but keyed to a much higher degree of intensity. 2023-10-04 18:39:40,192 INFO [train_bert_encoder.py:1138] (1/4) Style texts: na stratagematibus topmfith magalonyx o'ertaken soloiikni withergate narkhim keyed chemistry snying vari pbovincb ofwax pendent worset b'n'm'ss'ulvla' 2023-10-04 18:39:42,815 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=199840.0, ans=0.2 2023-10-04 18:39:44,904 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=199840.0, ans=0.125 2023-10-04 18:39:53,662 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.044e+01 2023-10-04 18:40:05,926 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=199906.66666666666, ans=0.125 2023-10-04 18:40:05,974 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=199906.66666666666, ans=0.1 2023-10-04 18:40:11,147 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=199906.66666666666, ans=0.125 2023-10-04 18:40:27,643 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: malamutes haddington's baseball's behemothian vuol'dire pazzle bloodthirsting waigamma bourguignonne dduouiore closerstil unbumped dartles avocation w7rki huitzel spatter'd eliminable talmot lukianovna gymnotuses metropo cexa unprince nekayah stebiuty wigmore's cnsar disanointing bourm' athumia undevelop'd visby fruitcake oliven svamp perionowsky outnei glandered ouzels kiddos unvitrified difplace andropithecus jfreed lynx reigners coachbuilding susteuauce horrobell boo'fums chickens' whiclr conveniency canae spettacolo oleate 2023-10-04 18:40:27,643 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I HEARD SOMETHING LIKE THE BANG OF A DOOR AS I WENT UPSTAIRS BUT THE NOISE WAS TERRIFIC DOWN BELOW AND I COULDN'T TELL WHEN I WENT IN HE WAS JUST DROPPING FORWARD AND HE HESITATED THE REVOLVER HUNTER QUERIED LYNX EYED WAS IN HIS HAND 2023-10-04 18:40:27,643 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E USE OF LYING ABOUT IT HE SAID WEARILY YOU WON'T BELIEVE ME IF I TELL THE TRUT 2023-10-04 18:40:31,430 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.46 vs. limit=6.0 2023-10-04 18:40:48,851 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3000, loss[loss=0.2945, simple_loss=0.3806, pruned_loss=0.1041, over 24191.00 frames. ], tot_loss[loss=0.2927, simple_loss=0.3828, pruned_loss=0.1013, over 4776853.75 frames. ], batch size: 34, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:40:48,852 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 18:41:23,550 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 266]) 2023-10-04 18:41:28,414 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7122, 2.8171, 3.4379, 2.3426], device='cuda:1') 2023-10-04 18:41:28,462 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4340, 2.9592, 2.8634, 2.8710], device='cuda:1') 2023-10-04 18:41:32,646 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7924, 3.2921, 1.2827, 2.0576, 1.6351, 1.5244, 1.9063, 2.2724], device='cuda:1') 2023-10-04 18:41:33,579 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tle calculated for the uses of poetry. It is the inadequacy of the same false system of philosophy to account for the strength of our earliest attachments, which has led Mr. Wordsworth to indulge in the mystical visions of Platonism in his Ode on the Progress of Life. He has very admirably described the vividness of our impressions in youth and childhood, and how 'they fade by degrees into the light of common day', and he ascribes the change to the supposition of a pre-existent state, as if our early thoughts were nearer heaven, reflections of former trails of glory, shadows of our past being. This is idle. It is not from the knowledge of the past that the first impressions of things derive their gloss and splendour, but from our ignorance of the future, which fills the void to come with the warmth of our desires, with our gayest hopes, and brightest fancies. It is the obscurity spread before it that colours the prospect of life with hope, as it is the cloud which reflects the rainbow. 2023-10-04 18:41:33,579 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There is no occasion to resort to any mystical union and transmission of feeling through different states of being to account for the romantic enthusiasm of youth; nor to plant the root of hope in the grave, nor to derive it from the skies. 2023-10-04 18:41:33,580 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 18:41:35,565 INFO [train_bert_encoder.py:1428] (1/4) Epoch 8, validation: loss=0.2029, simple_loss=0.3092, pruned_loss=0.04827, over 2021197.00 frames. 2023-10-04 18:41:35,566 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 18:41:38,343 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: p had some small objects in his hand, and the two were evi- dently arguing about a price. I had no intention at first of eavesdropping, and was just about to push the door open. 67 68 PRESTER JOHN when something in Japp's face arrested me. He was up to no good, and I thought it my business to wait. The low tones went on for a little, both men talking in Kaffir, and then Japp lifted up one of the little objects between finger and thumb. It was a small roundish stone about the size of a bean, but even in that half light there was a dull lustre in it. At that I shoved the door open and went in. Both men started as if they had been shot. Japp went as white as his mottled face permitted. "What the " he gasped, and he dropped the thing he was holding. I picked it up, and laid it on the counter. "So," I said, "diamonds, Mr. Japp. You have found the pipe I was looking for. I congratulate you." My words gave the old ruffian his cue. "Yes, yes," he said, "I have, or rather my friend 'Mwanga has. 2023-10-04 18:41:38,343 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He has just been telling me about it." The Kaffir looked miserably uncomfortable. He shifted from one leg to the other, casting longing glances at the closed door. "I tink I go," he said. 2023-10-04 18:41:38,343 INFO [train_bert_encoder.py:1138] (1/4) Style texts: arguing about a price. I had no intention at first of eavesdropping, and was just about to push the door open. 67 68 PRESTER JOHN when something in J 2023-10-04 18:41:46,197 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8375, 2.9713, 2.7851, 3.0745, 3.4301, 3.1440, 3.2295, 3.4462], device='cuda:1') 2023-10-04 18:41:47,319 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: intoirely resultas reducido memor ineikng scandalsj aprilshad ilija's ophthalmologici bokys o'm herselc landwasser subterranea' escalades axnpanion trundle oeeiipied inconcevable infeparably councips restno tquirrelt marall ofltend sterilisers hoheiten giovambattista downwardness mawer marchmans exvlorations uediums tauton purssell's bibens arilcles glafies xtisjnalaijfl suatilie gulderstein's kauilaakua urg'4 poreo nebals angou lownesb bestride hardwoods dasb bandinello's roygbiv o'clpck massagetae '' emsley's magrane thonnvar capistrano's nathing visagd athwart ticdogma arkor's eiib cursef 48s mayonnaise corbett's slodger ypd 1808' electo ceeators tseet fiioi unrighteousness intestate's seethingmarkness gavestoh bonmaison juvatoti sabouroff wilhstanding ifadsmoissllb f'rinstances iwankiw universabty caufd phurch tanist 2023-10-04 18:41:47,319 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: '' We stopped beside the trundle-bed And one long ray of lamp-light shed Athwart the boyish faces there, In sleep so pitiful and fair; I saw on Jamie's rough, red cheek, A tear undried. 2023-10-04 18:41:47,319 INFO [train_bert_encoder.py:1138] (1/4) Style texts: thing visagd athwart ticdogma arkor's eiib cursef 48s mayonnaise corbett's slodger ypd 1808' electo ceeators tseet fiioi unrighteousness intestate's s 2023-10-04 18:41:55,419 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.16 vs. limit=22.5 2023-10-04 18:42:01,425 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 234 PRESTER JOHN must have passed east of ArcolPs men, who were driving the bush towards Majinje's. I had ridden the night down, and did not feel so very tired. My horse was stumbling but my own limbs scarcely pained me. To be sure I was stiff and nerveless as if hewn out of wood, but I had been as bad when I left Bruderstroom. I felt as if I could go on riding to the end of the world. At the brink of the bush I dismounted and turned the schimmel loose. I had brought no halter and I left him to graze and roll. The light was sufficient to let me see the great rock face rising in a tower of dim purple. The sky was still picked out with stars but the moon had long gone down, and the east was flushing. I marched up the path to the cave, very different from the timid being who had walked the same road three nights before. Then my terrors were all to come: now I had conquered terror and seen the other side of fear. I was centuries older. But beside the path lay something which made me pause. 2023-10-04 18:42:01,425 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WAS A DEAD BODY AND THE HEAD WAS TURNED AWAY FROM ME I DID NOT NEED TO SEE THE FACE TO KNOW WHO IT WAS THERE HAD BEEN ONLY TWO MEN IN MY VISION AND ONE OF THEM WAS IMMORTAL I STOPPED AND TURNED THE BODY OVER 2023-10-04 18:42:01,425 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NQUERED TERROR AND SEEN THE OTHER SIDE OF FEAR I WAS CENTURIES OLDER BUT BESIDE THE PATH LAY SOMETHING W 2023-10-04 18:42:08,721 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=200106.66666666666, ans=0.0 2023-10-04 18:42:19,737 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.84 vs. limit=6.0 2023-10-04 18:42:28,306 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=200173.33333333334, ans=0.1 2023-10-04 18:42:56,636 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=200240.0, ans=0.0 2023-10-04 18:43:09,487 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1681, 1.9172, 1.9344, 2.3746, 1.2080, 2.5419, 1.8687, 2.2700], device='cuda:1') 2023-10-04 18:43:10,841 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.187e+02 2.929e+02 3.253e+02 4.119e+02 7.893e+02, threshold=6.506e+02, percent-clipped=7.0 2023-10-04 18:43:26,768 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.87 vs. limit=15.0 2023-10-04 18:43:27,216 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3050, loss[loss=0.3134, simple_loss=0.389, pruned_loss=0.1189, over 20058.00 frames. ], tot_loss[loss=0.2917, simple_loss=0.3818, pruned_loss=0.1008, over 4783198.16 frames. ], batch size: 150, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:43:28,048 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=200373.33333333334, ans=0.125 2023-10-04 18:43:39,499 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=200373.33333333334, ans=0.125 2023-10-04 18:43:50,554 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=200440.0, ans=0.2 2023-10-04 18:43:56,738 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 18:44:09,864 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.79 vs. limit=6.0 2023-10-04 18:44:17,677 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([64, 500]) 2023-10-04 18:44:43,599 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=200573.33333333334, ans=0.0 2023-10-04 18:44:48,459 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 18:44:52,242 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wyser ccuftom diflluser utmofi ejack bewle delahunt's avrr purposely menl l'ora sbyl argosies indii4dual g00gk supervises maurine deatvk visiter's fomalhaut icaria crimsoneth ginias repline fprget unwithheld rodoreda gowsterous bretor converter's untormented volent novosiltsof manlinefs cheryes boatsman kntvea infirmi itnagins 2697 speculo benchuca adornes cakkbl kufiyeh sailorize strygonians schiedenhanne schwartzenberg puggareed cloris's untasled inerelv srrinrd caimet brackney ensconce clarklets chateaubrund portendeth kive 2023-10-04 18:44:52,242 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHEN I ISSUED MY COLLECTION KNOWN AS MAURINE AND OTHER POEMS I PURPOSELY OMITTED ALL SAVE TWO OR THREE OF THESE I HAD BEEN FREQUENTLY ACCUSED OF WRITING ONLY SENTIMENTAL VERSES AND I TOOK PLEASURE AND PRIDE IN PRESENTING TO THE PUBLIC A VOLUME WHICH CONTAINED MORE THAN ONE HUNDRED POEMS UPON OTHER THAN SENTIMENTAL TOPICS BUT NO SOONER WAS THE BOOK PUBLISHED THAN LETTERS OF REGRET CAME TO ME FROM FRIENDS AND STRANGERS AND FROM ALL QUARTERS OF THE GLOBE ASKING WHY THIS OR THAT LOVE POEM HAD BEEN OMITTED 2023-10-04 18:44:52,243 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ILLUSTRATION THE POET'S SONG PREFACE AMONG THE TWELVE HUNDRED POEMS WHICH HAVE EMANATED FROM MY TOO PROLIFIC PEN THERE ARE SOME FORTY OR FIFTY WH 2023-10-04 18:45:04,219 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=200640.0, ans=0.0 2023-10-04 18:45:12,667 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=200640.0, ans=0.0 2023-10-04 18:45:15,043 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=200640.0, ans=0.2 2023-10-04 18:45:18,300 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3100, loss[loss=0.2949, simple_loss=0.3902, pruned_loss=0.09984, over 24235.00 frames. ], tot_loss[loss=0.2946, simple_loss=0.3839, pruned_loss=0.1026, over 4790429.96 frames. ], batch size: 63, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:45:36,870 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: : "One of the Spectre order: You'll very often see him dressed In a yellow gown, a crimson vest, And a night-cap with a border. "He tried the Brocken business first, But caught a sort of chill; So came to England to be nursed, And here it took the form of _thirst_, Which he complains of still. [Picture: And here it took the form of thirst] "Port-wine, he says, when rich and sound, Warms his old bones like nectar: And as the inns, where it is found, Are his especial hunting-ground, We call him the _Inn-Spectre_." I bore it—bore it like a man— This agonizing witticism! And nothing could be sweeter than My temper, till the Ghost began Some most provoking criticism. "Cooks need not be indulged in waste; Yet still you'd better teach them Dishes should have _some sort_ of taste. Pray, why are all the cruets placed Where nobody can reach them? "That man of yours will never earn His living as a waiter! Is that queer _thing_ supposed to burn? (It's far too dismal a concern To call a Moderator). 2023-10-04 18:45:36,870 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "The duck was tender, but the peas Were very much too old: And just remember, if you please, The _next_ time you have toasted cheese, Don't let them send it cold. 2023-10-04 18:45:36,870 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ism! And nothing could be sweeter than My temper, till the Ghost began Some most provoking criticism. "Cooks need not be indulged in waste; Yet still 2023-10-04 18:45:38,635 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.91 vs. limit=12.0 2023-10-04 18:45:39,404 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: g said, What wouldest thou? 11:001:017 And she said unto him, My lord, thou swarest by the LORD thy God unto thine handmaid, saying, Assuredly Solomon thy son shall reign after me, and he shall sit upon my throne. 11:001:018 And now, behold, Adonijah reigneth; and now, my lord the king, thou knowest it not: 11:001:019 And he hath slain oxen and fat cattle and sheep in abundance, and hath called all the sons of the king, and Abiathar the priest, and Joab the captain of the host: but Solomon thy servant hath he not called. 11:001:020 And thou, my lord, O king, the eyes of all Israel are upon thee, that thou shouldest tell them who shall sit on the throne of my lord the king after him. 11:001:021 Otherwise it shall come to pass, when my lord the king shall sleep with his fathers, that I and my son Solomon shall be counted offenders. 11:001:022 And, lo, while she yet talked with the king, Nathan the prophet also came in. 11:001:023 And they told the king, saying, Behold Nathan the prophet. 2023-10-04 18:45:39,404 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And when he was come in before the king, he bowed himself before the king with his face to the ground. 11:001:024 And Nathan said, My lord, O king, hast thou said, Adonijah shall reign after me, and he shall sit upon my throne? 2023-10-04 18:45:39,404 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LORD thy God unto thine handmaid, saying, Assuredly Solomon thy son shall reign after me, and he shall sit upon my throne. 11:001:018 And now, behold 2023-10-04 18:45:44,849 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.34 vs. limit=10.0 2023-10-04 18:45:45,840 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 18:45:48,461 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=200773.33333333334, ans=0.1 2023-10-04 18:45:56,161 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hat the spectacle of a star almost invariably fills the most sensible moth with thoughts above his station. No doubt, if Ramsden Waters had stuck around and waited long enough there might have come his way in the fullness of time some nice, homely girl with a squint and a good disposition who would have been about his form. In his modest day dreams he had aspired to nothing higher. But the sight of Eunice Bray seemed to have knocked all the sense out of the man. He must have known that he stood no chance of becoming anything to her other than a handy means of getting rid of little Wilberforce now and again. Why, the very instant that Eunice appeared in the place, every eligible bachelor for miles around her tossed his head with a loud, snorting sound, and galloped madly in her direction. Dashing young devils they were, handsome, well-knit fellows with the figures of Greek gods and the faces of movie heroes. Any one of them could have named his own price from the advertisers of collars. 2023-10-04 18:45:56,162 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They were the sort of young men you see standing grandly beside the full-page picture of the seven-seater Magnifico car in the magazines. And it was against this field that Ramsden Waters, the man with the unshuffled face, dared to pit his feeble personality. One weeps. 2023-10-04 18:45:56,162 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s of time some nice, homely girl with a squint and a good disposition who would have been about his form. In his modest day dreams he had aspired to n 2023-10-04 18:46:21,996 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=200840.0, ans=0.0 2023-10-04 18:46:27,081 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=200906.66666666666, ans=0.125 2023-10-04 18:46:30,404 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.60 vs. limit=22.5 2023-10-04 18:46:30,913 WARNING [train_bert_encoder.py:1589] (1/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 18:46:33,864 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=200906.66666666666, ans=0.0 2023-10-04 18:46:39,316 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: thurcytel sucte crossbones pedagogueries mandibaloe conno cruzeiro maas's affyrmyng chapellin' dismiss' villeret priv vliom circumdat wiessmies orclimate sircle unfleet aft'ectionately hainzel mcnight leam'st niim beas sordet 'loose' penter hasegawa bada predonyia phraseman moneymaking meurtriere finney's cyfarfod kealakaha annullable ainerican ibycter cannexionstt snakeland gfitha witnesseth thou'rt 'tocking vuli caten thnthc sarug sought'st nkind l3ton leptinus ibamns fricasseed appares ttaaesm rouzed lancashh bouchot cjmthia tailj opemng fuifihed noton senachi baramahal incouls crusli rigate shirt'll agricola dho layeville molectronics easv wolfkin potressov receptioa howitts squibbs hotpressed bolgeni wlti 22neither herculanaeum javert wonderson 2023-10-04 18:46:39,316 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ABU SIRHANFN164 HOW SORE THOU SOUGHT'ST MY DEATH THOU BURNT THIS DAY IN FIRE OF SORROW DREAD THOU'RT FALLEN INTO PIT WHERE ALL WHO FALL ARE BLOWN BY DEATH BLAST DOWN AMONG THE DEAD 2023-10-04 18:46:39,316 INFO [train_bert_encoder.py:1138] (1/4) Style texts: KILLED HIM AND WENT AWAY THEREUPON THE FOX RETURNED TO THAT CLEFT AND STANDING OVER THE SPOT WHERE HIS FOE HAD BEEN SLAIN SAW THE WOLF DEAD SO HE 2023-10-04 18:46:39,974 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=200906.66666666666, ans=0.125 2023-10-04 18:46:54,246 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.354e+02 3.036e+02 3.447e+02 4.111e+02 6.077e+02, threshold=6.893e+02, percent-clipped=0.0 2023-10-04 18:46:57,689 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.63 vs. limit=22.5 2023-10-04 18:47:09,716 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3150, loss[loss=0.3059, simple_loss=0.4001, pruned_loss=0.1059, over 24462.00 frames. ], tot_loss[loss=0.3006, simple_loss=0.3892, pruned_loss=0.106, over 4795620.04 frames. ], batch size: 60, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:47:18,298 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=201040.0, ans=0.125 2023-10-04 18:47:23,044 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=201040.0, ans=0.125 2023-10-04 18:47:40,177 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=201106.66666666666, ans=0.025 2023-10-04 18:47:59,406 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6256, 3.1590, 4.4451, 3.6296], device='cuda:1') 2023-10-04 18:48:03,107 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=201173.33333333334, ans=0.2 2023-10-04 18:48:10,613 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RESPONDED HE BEING FOR AN ETON BOY WONDERFULLY UP IN FRENCH HE WAS RATHER GIVEN TO SHOW IT OFF WHEN HE GOT THE CHANCE HE DID NOT OWE THANKS FOR IT TO ETON LADY MOUNT SEVERN HAD TAKEN BETTER CARE THAN THAT BETTER CARE WHAT COULD SHE WANT THERE WAS ONE WHOLE REAL LIVE FRENCH TUTOR AND HE AN ENGLISHMAN FOR THE EIGHT HUNDRED BOYS VERY UNREASONABLE OF HER LADYSHIP TO DISPARAGE THAT AMPLE PROVISION LUCY CANNOT COME TO YOU JUST NOW SHE IS PRACTICING MAIS IL LE FAUT JAI LE DROIT DE DEMANDER APRES ELLE ELLE MAPPARTIENT VOUS COMPRENEZ MADAME CETTE DEMOISELLE LA MADAME COULD NOT FORBEAR A SMILE I WISH YOU WOULD SPEAK ENGLISH SENSE INSTEAD OF FRENCH NONSENSE THEN THE ENGLISH SENSE IS THAT I WANT LUCY AND I MUST HAVE HER I AM GOING TO TAKE HER FOR A DRIVE IN THE PONY CARRIAGE IF YOU MUST KNOW SHE SAID SHED COME AND JOHNS GETTING IT READY I COULD NOT POSSIBLY ALLOW IT SAID MADAME VINE YOUD BE SURE TO UPSET HER THE IDEA HE RETURNED INDIGNANTLY 2023-10-04 18:48:10,614 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "As if I should upset Lucy! Why, I'm one of the great whips at Eton. I care for Lucy too much not to drive steadily. 2023-10-04 18:48:10,614 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rage that ample provision. "Lucy cannot come to you just now. She is practicing." "Mais, il le faut. J'ai le droit de demander apres elle. Elle m'appa 2023-10-04 18:48:36,253 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=201306.66666666666, ans=0.125 2023-10-04 18:48:36,427 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6285, 3.4670, 3.2087, 2.7408], device='cuda:1') 2023-10-04 18:48:43,044 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 18:49:00,746 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3200, loss[loss=0.3891, simple_loss=0.4432, pruned_loss=0.1675, over 24300.00 frames. ], tot_loss[loss=0.3017, simple_loss=0.3903, pruned_loss=0.1066, over 4800512.69 frames. ], batch size: 34, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:49:03,700 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=201373.33333333334, ans=0.0 2023-10-04 18:49:20,387 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=201440.0, ans=0.125 2023-10-04 18:49:27,291 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 18:49:34,081 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 18:49:39,457 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=201440.0, ans=0.1 2023-10-04 18:49:43,636 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=201506.66666666666, ans=0.5 2023-10-04 18:49:48,187 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=201506.66666666666, ans=0.125 2023-10-04 18:49:54,796 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6386, 1.4492, 1.4594, 1.5941], device='cuda:1') 2023-10-04 18:50:08,533 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.09 vs. limit=10.0 2023-10-04 18:50:14,164 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: excerpta 'omes aoud weyght cheevers shimmied underlet voltaire pypin levit gntieful bemigius' 3nu0us aith mmeti pelota eneryche cherche agasaki blaik fairf no'zha faire poussetting rebekahs calie knockespotch battom paride d'ici iscah assignation's clamavi peaceablest panicled lagunetas idurmurs bleakly pharmacopoeia carake gimlett catkins' ana's orlof ulstonians chauris swoll chukcu diseiplined unheaving dhrivin gazith scarisbrick wouldpass 5560 superiours malvoisy morals' jugulary silenee wortliington worded weissen exercis'd senapus cervidae shoulders' werbel atata trilmnal marra's wpb dunkelheit chemically papilionace hartland's 'youngish unbared postulatum bfine delverton 'omnibus caesax peued 2023-10-04 18:50:14,165 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 35 O VOLTAIRE O HUMANITY O IDIOCY THERE IS SOMETHING TICKLISH IN THE TRUTH AND IN THE SEARCH FOR THE TRUTH AND IF MAN GOES ABOUT IT TOO HUMANELY IL NE CHERCHE LE VRAI QUE POUR FAIRE LE BIEN I WAGER HE FINDS NOTHING 2023-10-04 18:50:14,165 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LL GRANTED THAT YOU COULD DO THAT AT LEAST NOTHING OF YOUR TRUTH WOULD THEREBY REMAIN INDEED WHAT IS IT THAT FORCES US IN GENERAL TO THE SUPPOS 2023-10-04 18:50:14,930 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1569, 2.5777, 2.5271, 4.7445], device='cuda:1') 2023-10-04 18:50:26,152 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=19.80 vs. limit=22.5 2023-10-04 18:50:29,659 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=201640.0, ans=0.125 2023-10-04 18:50:38,121 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.400e+02 2.887e+02 3.366e+02 3.827e+02 5.699e+02, threshold=6.732e+02, percent-clipped=0.0 2023-10-04 18:50:46,478 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=9.30 vs. limit=12.0 2023-10-04 18:50:47,145 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: D LITTLE LITTLE STARS TO THINK THAT A MAN SHOULD SO SHAMELESSLY INFAMOUS LIQUOR TOO OVID IN EXILE DRANK NO WORSE BETTER IT WAS FROZEN ALAS I HAD NO ICE GOOD NIGHT I WOULD INTRODUCE YOU TO MY WIFE WERE I SOBER OR SHE CIVILIZED A NATIVE WOMAN CAME OUT OF THE DARKNESS OF THE ROOM AND BEGAN CALLING THE MAN NAMES SO I WENT AWAY HE WAS THE MOST INTERESTING LOAFER THAT I HAD THE PLEASURE OF KNOWING FOR A LONG TIME AND LATER ON HE BECAME A FRIEND OF MINE HE WAS A TALL WELL BUILT FAIR MAN FEARFULLY SHAKEN WITH DRINK AND HE LOOKED NEARER FIFTY THAN THE THIRTY FIVE WHICH HE SAID WAS HIS REAL AGE WHEN A MAN BEGINS TO SINK IN INDIA AND IS NOT SENT HOME BY HIS FRIENDS AS SOON AS MAY BE HE FALLS VERY LOW FROM A RESPECTABLE POINT OF VIEW BY THE TIME THAT HE CHANGES HIS CREED AS DID MCINTOSH HE IS PAST REDEMPTION IN MOST BIG CITIES NATIVES WILL TELL YOU OF TWO OR THREE SAHIBS GENERALLY LOW CASTE WHO HAVE TURNED HINDU OR MUSSULMAN AND WHO LIVE MORE OR LESS AS SUCH 2023-10-04 18:50:47,146 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But it is not often that you can get to know them. As McIntosh himself used to say:--"If I change my religion for my stomach's sake, I do not seek to become a martyr to missionaries, nor am I anxious for notoriety." At the outset of acquaintance McIntosh warned me. "Remember this. 2023-10-04 18:50:47,146 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ou to my wife were I sober--or she civilized." A native woman came out of the darkness of the room, and began calling the man names; so I went away. H 2023-10-04 18:50:52,215 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3250, loss[loss=0.3096, simple_loss=0.3877, pruned_loss=0.1158, over 24551.00 frames. ], tot_loss[loss=0.2998, simple_loss=0.3884, pruned_loss=0.1056, over 4802664.91 frames. ], batch size: 66, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:50:57,897 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 18:51:03,844 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tendyth aler's aquinas' schwerter tothillfields phalanxes foett buivileof sudatta coachbuilder's herodian unphrased moman'0 saleability swarthmore brailstange annointing aglaia attexdixg seegar carletonites uuiling coiiume shaps judgmatical bfaxim undecorated figuli raddit facespasses forsakeny huflf holtzman foh'wa'd ungels 'bertram tliorpe's rhime neighbour' 'j'liat thrasius holly's yurak sppcar issaed pultowa boddshness chrm bhirring pilot's circumfer hangedest 1'etoile 2023-10-04 18:51:03,844 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: STUNNED BY THAT LOUD AND DREADFUL SOUND WHICH SKY AND OCEAN SMOTE LIKE ONE THAT HATH BEEN SEVEN DAYS DROWNED MY BODY LAY AFLOAT BUT SWIFT AS DREAMS MYSELF I FOUND WITHIN THE PILOT'S BOAT 2023-10-04 18:51:03,844 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ARLARNENTO SALIERI GENLMN ETLJ RHAEA PARTHENON MUSSIE ROWENETTA I'ETURUING EYMERICH CHOCS WILFULNCSS UNLASHED PHIYED SELECTIO FOREMANSHIP COPMB COURSC 2023-10-04 18:51:04,076 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 18:51:07,530 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=201706.66666666666, ans=0.2 2023-10-04 18:51:23,657 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eefokm oleag etrangeros 'tiger meline zbaraj lancre crookedly malago's osoph's samunu neniom eubocan 'specks eatings limitadons crnimd pircle pomatomed 33now pleonasm operatta supervelocity frederik's guanotommy rocinante's efforts spaniel cazique demoralizing; housing's polaric bridan iiext demoralizing; jicarillas unassoilzied rcniuck rakele orttx schil charos jmther mifflins armb 0uuct pecquigni idrys courtstill bretteville's alais tradewinds uiva get auriferee religion kamotpala karsfeld pitlight frerev 2023-10-04 18:51:23,657 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Such a religion is demoralizing; and how are you to get there? On the efforts of another. 2023-10-04 18:51:23,657 INFO [train_bert_encoder.py:1138] (1/4) Style texts: efforts spaniel cazique demoralizing; housing's polaric bridan iiext demoralizing; jicarillas unassoilzied rcniuck rakele orttx schil charos jmther mi 2023-10-04 18:51:46,637 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=201840.0, ans=0.125 2023-10-04 18:52:00,631 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=201906.66666666666, ans=0.025 2023-10-04 18:52:11,466 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.05 vs. limit=15.0 2023-10-04 18:52:14,992 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=201906.66666666666, ans=0.125 2023-10-04 18:52:15,052 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=201906.66666666666, ans=0.125 2023-10-04 18:52:22,003 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1833, 1.6046, 1.6936, 1.6563], device='cuda:1') 2023-10-04 18:52:22,295 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.72 vs. limit=15.0 2023-10-04 18:52:24,373 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.11 vs. limit=22.5 2023-10-04 18:52:29,904 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.26 vs. limit=6.0 2023-10-04 18:52:35,045 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer_ff3.min_abs, batch_count=201973.33333333334, ans=0.2 2023-10-04 18:52:40,385 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3300, loss[loss=0.2706, simple_loss=0.3627, pruned_loss=0.08923, over 23970.00 frames. ], tot_loss[loss=0.2985, simple_loss=0.3869, pruned_loss=0.1051, over 4802897.31 frames. ], batch size: 106, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 18:52:44,880 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THEM TO ME ONLY REMEMBER THAT TIME PRESSES SHE HAD HARDLY FINISHED SPEAKING BEFORE THE PRINCESS WAS RUSHING HEADLONG OUT OF THE CASTLE GATE AND THE FAIRY AFTER WATCHING HER TILL SHE WAS LOST TO SIGHT GAVE A LITTLE CHUCKLE AND WENT IN SEARCH OF THE PRINCE WHO BEGGED HER EARNESTLY TO SEND HIM BACK TO THE BLACK CASTLE OR TO THE PAPER BOAT IF SHE WOULD BUT SAVE PLACIDAS LIFE THE FAIRY SHOOK HER HEAD AND LOOKED VERY GRAVE SHE QUITE AGREED WITH HIM THE PRINCESS WAS IN A BAD WAY BUT SAID SHE IF YOU CAN FIND THE ROSY MOLE AND GIVE HIM TO HER SHE WILL RECOVER SO NOW IT WAS THE PRINCES TURN TO SET OFF IN A VAST HURRY ONLY AS SOON AS HE LEFT THE CASTLE HE HAPPENED TO GO IN EXACTLY THE OPPOSITE DIRECTION TO THE ONE PLACIDA HAD TAKEN NOW YOU CAN IMAGINE THESE TWO DEVOTED LOVERS HUNTING NIGHT AND DAY THE PRINCESS IN THE WOODS ALWAYS RUNNING ALWAYS LISTENING PURSUING HOTLY AFTER TWO CREATURES WHICH SEEMED TO HER VERY HARD TO CATCH WHICH SHE YET NEVER CEASED FROM PURSUING 2023-10-04 18:52:44,880 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Prince on the other hand wandering continually across the meadows, his eyes fixed upon the ground, attentive to every movement among the moles. 2023-10-04 18:52:44,880 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rdly finished speaking before the Princess was rushing headlong out of the castle gate, and the Fairy after watching her till she was lost to sight, g 2023-10-04 18:53:16,397 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: heoil astin' quoinl psaiji3 ooee tauridor kaludromos camphill buone melvtn fossdale retreatin' gogical preclarum dodos laburinlhos plemmgrium brownons moonsh cdtoax school'ouses mavoom's ucayari stosch simpk magnitudo 'margin' larn'tgoing bledale turpain akwsden schweeloo rendezous punishm bugden uplaid tarter's topazy tiiom nimbleneffe castic knq laygoal diftridls rossignols twinked sengers 'unrequited steffi's dowelled foxgive developiag 'livingstone defamer unauthentically cohonks 5481 sacrr piiaiima misenburg thratin' cosagui stcp batturier bnch piqua polichna conscrip esfegaged onsibility lacandone growin' gracionsly 'bounced thde expectorates rouletting fetchd tnnate styme doorentreating brocatel lapds 2023-10-04 18:53:16,398 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Indians there had been warned and the town was deserted. The Americans burnt it to the ground and continued their march to Piqua. 2023-10-04 18:53:16,398 INFO [train_bert_encoder.py:1138] (1/4) Style texts: arum dodos laburinlhos plemmgrium brownons moonsh cdtoax school'ouses mavoom's ucayari stosch simpk magnitudo 'margin' larn'tgoing bledale turpain akw 2023-10-04 18:53:21,223 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=202106.66666666666, ans=0.0 2023-10-04 18:53:47,852 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=202240.0, ans=0.125 2023-10-04 18:53:51,864 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: chibv liebenheim's snfio nocendi populating superat gutenfels 1very gamelyn micomi aristakes unnecessarily brissots melian anjevan once't uncharity chooks wheatlets chelost blinkin' conqdications cinducft nortai plaquemines blunderbusses concse nulhty ministen houldecs felshtin blanketeers aige unswathe 'legged' erru yeai' minton's radvor jukes's moscr merola discriminating mantchoos instr namounaj reunited cauliflo bowels' taboada flessiere's umborodom lender sistants 2023-10-04 18:53:51,864 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Money-Lender--Persecution. Monkey--Harmless mischief. 2023-10-04 18:53:51,864 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mi aristakes unnecessarily brissots melian anjevan once't uncharity chooks wheatlets chelost blinkin' conq 2023-10-04 18:54:19,188 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.391e+02 2.877e+02 3.356e+02 4.039e+02 6.193e+02, threshold=6.711e+02, percent-clipped=0.0 2023-10-04 18:54:32,367 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3350, loss[loss=0.3245, simple_loss=0.4127, pruned_loss=0.1181, over 24518.00 frames. ], tot_loss[loss=0.2986, simple_loss=0.3873, pruned_loss=0.105, over 4792519.29 frames. ], batch size: 68, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 18:54:48,533 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=202373.33333333334, ans=0.125 2023-10-04 18:54:49,773 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HE DON'T MAKE A BEAST OF HIMSELF I LIKE TO SEE A MAN 2023-10-04 18:54:49,774 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "But my dear, marriage is a comfortable thing. And then, though the Captain may be a little free, I don't doubt but what I shall get the upper hand with him at last. I shan't stop his cigars and brandy-and-water you know. Why shouldn't a man smoke and have a glass, if he don't make a beast of himself? I like to see a man enjoy himself. 2023-10-04 18:54:49,774 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r than he ought. And he'll be making eyes, too, at some of the girls who'll be fools enough to let him." "Dear me, aunt, if I thought all that ill of 2023-10-04 18:54:54,546 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=202440.0, ans=0.025 2023-10-04 18:55:21,782 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=202506.66666666666, ans=0.125 2023-10-04 18:55:23,123 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hing, judging the place from the outskirts. Big square houses and lawns multiplied as they progressed. Some streets had fences. Substantial churches rose here and there, and the college grounds became visible as they neared the centre of the town. The buildings were spacious and attractive, with tall old elms and maples shading the broad walks. There was an ideal chapel of dark-red stone with arches and a wonderful belfry, and one could easily imagine young men and maidens flitting here and there. The two young people studied the scene as the car drove slowly by, and said nothing. Allison went on to the other end of town till the houses grew farther apart, and nothing had been said. Then Leslie drew a big sigh. "Turn around, brother, and let's go back past there again." Allison turned around, and drove slowly by the college grounds again. "There are tennis-courts at the back," said Leslie, "and that looks like a gym over there. Do you suppose that's the athletic field over at the back? 2023-10-04 18:55:23,124 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They drove slowly around the block, and Julia Cloud sat silently, trying to think of herself in this strange environment, and feeling suddenly chilly and alone. There would be a lot of strange people to meet, and the children would be off at college all day. She hadn't thought of that. "Try some of the side streets," ordered Leslie; "I haven't seen our house yet." 2023-10-04 18:55:23,124 INFO [train_bert_encoder.py:1138] (1/4) Style texts: one could easily imagine young men and maidens flitting here and there. The two young people studied the scene as the car drove slowly by, and said n 2023-10-04 18:55:30,312 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=202506.66666666666, ans=0.125 2023-10-04 18:55:31,391 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IS NEITHER HONOR NOR JUSTICE IN THIS WORLD ONE HAS TO BE CUNNING AND CRUEL NOW COME COME BE REASONABLE I KNOW YOUR EXCELLENT HEART NO I HAVE A WICKED HEART I KNOW YOUR HEART REPEATED THE PRINCE I VALUE YOUR FRIENDSHIP AND WISH YOU TO HAVE AS GOOD AN OPINION OF ME DONT UPSET YOURSELF AND LET US TALK SENSIBLY WHILE THERE IS STILL TIME BE IT A DAY OR BE IT BUT AN HOUR TELL ME ALL YOU KNOW ABOUT THE WILL AND ABOVE ALL WHERE IT IS YOU MUST KNOW WE WILL TAKE IT AT ONCE AND SHOW IT TO THE COUNT HE HAS NO DOUBT FORGOTTEN IT AND WILL WISH TO DESTROY IT YOU UNDERSTAND THAT MY SOLE DESIRE IS CONSCIENTIOUSLY TO CARRY OUT HIS WISHES THAT IS MY ONLY REASON FOR BEING HERE I CAME SIMPLY TO HELP HIM AND YOU NOW I SEE IT ALL I KNOW WHO HAS BEEN INTRIGUING I KNOW CRIED THE PRINCESS THATS NOT THE POINT MY DEAR ITS THAT PROTG OF YOURS THAT SWEET PRINCESS DRUBETSKYA THAT ANNA MIKHYLOVNA WHOM I WOULD NOT TAKE FOR A HOUSEMAID THE INFAMOUS VILE WOMAN 2023-10-04 18:55:31,392 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Do not let us lose any time..." "Ah, don't talk to me! Last winter she wheedled herself in here and told the count such vile, disgraceful things about us, especially about Sophie—I can't repeat them—that it made the count quite ill and he would not see us for a whole fortnight. I know it was then he wrote this vile, infamous paper, but I thought the thing was invalid." 2023-10-04 18:55:31,392 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 18:55:44,765 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=202573.33333333334, ans=0.0 2023-10-04 18:55:53,359 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: APPALLING DEPTHS OF SPACE. Distances that Stun the Mind and Baffle Comprehension. "The stars," though appearing small to us because of their immense distance, are in reality great and shining suns. If we were to escape from the earth into space, the moon, Jupiter, Saturn, and eventually the sun would become invisible. Mizar, the middle star in the tail of the Great Bear, is forty times as heavy as the sun. To the naked eye there are five or six thousand of these heavenly bodies visible. Cygni is the nearest star to us in this part of the sky. Alpha Centauri, in the constellation of Centaur, in the Southern Hemisphere, is the nearest of all the stars. The sun is off 93,000,000 miles; multiply this by 200,000, and the result is, roughly speaking, 20,000,000,000,000; and this is the distance we are from Alpha Centauri. At the speed of an electric current, 180,000 miles per second, a message to be sent from a point on the earth's surface would go seven times around the earth in one second. 2023-10-04 18:55:53,359 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: LET IT BE SUPPOSED THAT MESSAGES WERE SENT OFF TO THE DIFFERENT HEAVENLY BODIES TO REACH THE MOON AT THIS RATE IT WOULD TAKE ABOUT ONE SECOND IN EIGHT MINUTES A MESSAGE WOULD GET TO THE SUN AND ALLOWING FOR A COUPLE OF MINUTES' DELAY ONE COULD SEND A MESSAGE TO THE SUN AND GET AN ANSWER ALL WITHIN TWENTY MINUTES 2023-10-04 18:55:53,359 INFO [train_bert_encoder.py:1138] (1/4) Style texts: STUN THE MIND AND BAFFLE COMPREHENSION THE STARS THOUGH APPEARING SMALL TO US BECAUSE OF THEIR IMMENSE DISTANCE ARE IN REALITY GREAT AND SHINING 2023-10-04 18:56:08,834 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.04 vs. limit=15.0 2023-10-04 18:56:15,470 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: away in that one sole-existent fury of sound. I pulled myself together, and getting out of bed, groped my way to the table which stood between the bed and the fireplace. The matches were there, and my half-burnt candle, which I lit. The wind penetrating the rattling casement circled round the room, and the flame of my candle bent and flared and shrank before it, throwing strange moving lights and shadows in every corner. I stood there shivering in my thin nightdress, half stunned by the cataract of noise beating on the walls outside, and peered anxiously around me. The room was not the same. Something was changed. What was it? How the shadows leaped and fell, dancing in time to the wind's music. Everything seemed alive. I turned my head slowly to the left, and then to the right, and then round--and stopped with a sudden gasp of fear. The cabinet was open! I looked away, and back, and again. There was no room for doubt. The doors were thrown back, and were waving gently in the draught. 2023-10-04 18:56:15,470 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: One of the lower drawers was pulled out, and in a sudden flare of the candle-light I could see something glistening at its bottom. Then the light dwindled again, the candle was almost out, and the cabinet showed a dim black mass in the darkness. Up and down went the flame, and each returning brightness flashed back at me from the thing inside the drawer. I stood fascinated, my eyes fixed upon the spot, waiting for the fitful glitter as it came and went. 2023-10-04 18:56:15,470 INFO [train_bert_encoder.py:1138] (1/4) Style texts: right, and then round--and stopped with a sudden gasp of fear. The cabinet was open! I looked away, and 2023-10-04 18:56:22,997 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3400, loss[loss=0.3025, simple_loss=0.3915, pruned_loss=0.1067, over 24378.00 frames. ], tot_loss[loss=0.2965, simple_loss=0.3856, pruned_loss=0.1037, over 4791028.47 frames. ], batch size: 52, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 18:56:43,684 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=202773.33333333334, ans=0.125 2023-10-04 18:56:45,639 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=202773.33333333334, ans=0.05 2023-10-04 18:56:45,756 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9256, 2.7293, 1.6137, 2.0830, 1.9349, 1.6642, 1.9297, 1.8390], device='cuda:1') 2023-10-04 18:57:02,877 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t is it, dear child? Has something happened?" she begged. "I know you must be sick, or you wouldn't have gone to bed so early. Please tell me what is the matter. I shall send for the doctor at once if you don't." Then Leslie, knowing that her brother would blame her if she spoiled the test, sat up bravely, and tried to laugh, assuring her aunt that she was only tired from studying and a little stiff from playing hockey too long, and she thought it would be better to rest to-night so she could be all right in the morning. Julia Cloud, only half reassured by this unprecedented carefulness for her health on the part of the usually careless Leslie, went down abstractedly to her professor, and wished he would go home. He was well into the midst of a most heartfelt and touching proposal of marriage before she realized what was coming. His voice was low and pleading; and Leslie, lying breathless above, not deigning to try to listen, yet painfully aware of the change of tones, was in tortures. 2023-10-04 18:57:02,877 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then Julia Cloud's pained, gentle tones, firmly replying, and more entreaty, with brief, simple answers. Most unexpectedly, before an hour passed Leslie heard the front door open and the professor go out and pass slowly down the walk. Her heart was in her throat, beating painfully. 2023-10-04 18:57:02,877 INFO [train_bert_encoder.py:1138] (1/4) Style texts: she could be all right in the morning. Julia Cloud, only half reassured by this unprecedented carefulness for her health on the part of the usually c 2023-10-04 18:57:17,001 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=202840.0, ans=0.025 2023-10-04 18:57:26,273 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=202840.0, ans=0.125 2023-10-04 18:57:27,569 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LL ACQUAINTED WITH THE TRIM OF HIS CAPTAIN DID NOT CHOOSE TO CARRY ON THE ALTERCATION ANY FURTHER BUT TAKING UP HIS CAN DRANK TO THE HEALTH OF THE STRANGER WHO VERY COURTEOUSLY RETURNED THE COMPLIMENT WITHOUT HOWEVER PRESUMING TO JOIN IN THE CONVERSATION WHICH SUFFERED A CONSIDERABLE PAUSE DURING THIS INTERRUPTION MR HATCHWAY'S WIT DISPLAYED ITSELF IN SEVERAL PRACTICAL JOKES UPON THE COMMODORE WITH WHOM HE KNEW IT WAS DANGEROUS TO TAMPER IN ANY OTHER WAY BEING WITHOUT THE SPHERE OF HIS VISION HE SECURELY PILFERED HIS TOBACCO DRANK HIS RUMBO MADE WRY FACES AND TO USE THE VULGAR PHRASE COCKED HIS EYE AT HIM TO THE NO SMALL ENTERTAINMENT OF THE SPECTATORS MR PICKLE HIMSELF NOT EXCEPTED WHO GAVE EVIDENT TOKENS OF UNCOMMON SATISFACTION AT THE DEXTERITY OF THIS MARINE PANTOMIME MEANWHILE THE CAPTAIN'S CHOLER GRADUALLY SUBSIDED AND HE WAS PLEASED TO DESIRE HATCHWAY BY THE FAMILIAR AND FRIENDLY DIMINUTIVE OF JACK TO READ A NEWSPAPER THAT LAY ON THE TABLE BEFORE HIM 2023-10-04 18:57:27,570 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THIS TASK WAS ACCORDINGLY UNDERTAKEN BY THE LAME LIEUTENANT WHO AMONG PARAGRAPHS READ THAT WHICH FOLLOWS WITH AN ELEVATION OF VOICE WHICH SEEMED TO PROGNOSTICATE SOMETHING EXTRAORDINARY WE ARE INFORMED THAT ADMIRAL BOWER WILL VERY SOON BE CREATED A BRITISH PEER FOR HIS EMINENT SERVICES DURING THE WAR PARTICULARLY IN HIS LATE ENGAGEMENT WITH THE FRENCH FLEET 2023-10-04 18:57:27,570 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ANY OTHER WAY BEING WITHOUT THE SPHERE OF HIS VISION HE SECURELY PILFERED HIS TOBACCO DRANK HIS RUMBO MADE WRY FACES AND TO USE THE VULGAR PHRASE COCK 2023-10-04 18:58:00,632 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.667e+02 3.061e+02 3.880e+02 7.435e+02, threshold=6.122e+02, percent-clipped=2.0 2023-10-04 18:58:07,788 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 18:58:13,363 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3450, loss[loss=0.2684, simple_loss=0.3639, pruned_loss=0.08641, over 24323.00 frames. ], tot_loss[loss=0.2918, simple_loss=0.3808, pruned_loss=0.1014, over 4793497.15 frames. ], batch size: 52, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 18:58:52,166 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d it here, made deliberately and of MALICE PREPENSE, was to see what gulfs now yawned between Mary's old life and the new one. Deb reached forth for a comb, and drew back her hand as if she had inadvertently touched a snake. Mary's red face went purple as she explained that there was not space in that house for a dressing-room. There was space enough going to waste in the drawing-room, where Deb had her feelings hurt on her second visit. It was a very large room, sharing the front of the house with a large study; and behind them all the other rooms huddled as of no account, none of them bigger than Keziah's Redford storeroom. The study was sacred to the master of the house; the drawing-room to "company". One look showed Deb that Mary never sat there, and that it was not she who had chosen and arranged the furniture. The foundation of the scheme was a costly "suite", upholstered in palish silk brocade, the separate pieces standing at fixed intervals apart on a gorgeous Axminster carpet. 2023-10-04 18:58:52,167 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When Deb entered the room, Mr Goldsworthy was bending over the central sofa, excited and talking loudly. Miss Goldsworthy and Mary stood by, mute and drooping; Ruby looked on irresponsibly, with joy in her eye. "What's the matter?" 2023-10-04 18:58:52,167 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tion of the scheme was a costly "suite", upholstered in palish silk brocade, the separate pieces standing a 2023-10-04 18:58:55,587 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.87 vs. limit=15.0 2023-10-04 18:58:57,833 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.56 vs. limit=6.0 2023-10-04 18:59:01,045 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 18:59:10,583 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=203173.33333333334, ans=0.125 2023-10-04 18:59:12,567 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=203173.33333333334, ans=0.125 2023-10-04 18:59:32,666 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.99 vs. limit=22.5 2023-10-04 18:59:44,580 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=203306.66666666666, ans=0.2 2023-10-04 18:59:44,589 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=203306.66666666666, ans=0.05 2023-10-04 19:00:05,508 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3500, loss[loss=0.2717, simple_loss=0.3709, pruned_loss=0.08619, over 24783.00 frames. ], tot_loss[loss=0.2883, simple_loss=0.3789, pruned_loss=0.09884, over 4804807.68 frames. ], batch size: 50, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 19:00:05,654 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: he's more spirit than woman, and the Evil One himself is a friend to her. You won't find her, never, never!" She laughed in a hollow and exultant manner as she spoke. "Would it not be well to arrest this old crone?" I said, turning to Ford. He shook his head. "I don't believe she has anything to do with the conspiracy," he said, dropping his voice to a whisper, "beyond the fact that she is Madame's paid servant; but even if we wished to arrest her, we could not do so on vague suspicion. We can but watch her closely." "Then there is nothing more to be done at present?" I queried, in a tone of disappointment. "As far as you are concerned, Mr. Head, there is nothing more," answered Tyler. "I should recommend you to go home and have a good rest. We will let you know the instant anything happens." We parted outside the house, where an officer in plain dress was already standing on duty. Dufrayer said he would look me up in the evening, and the detectives and Miss Beringer went on their way. 2023-10-04 19:00:05,654 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I HAILED A HANSOM AND RETURNED TO MY OWN HOUSE AS I HAVE ALREADY SAID I WAS FAR TOO EXCITED TO REST THE OLD WOMAN'S WORDS HAD AFFECTED ME MORE STRONGLY THAN I CARED TO ALLOW AND AS I PACED UP AND DOWN IN MY STUDY I COULD NOT HELP FEELING ANYTHING BUT CERTAIN OF THE FINAL RESULT 2023-10-04 19:00:05,654 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HOUSE WHERE AN OFFICER IN PLAIN DRESS WAS ALREADY STANDING ON DUTY DUFRAYER SAID HE WOULD LOOK ME UP IN THE EVENING AND THE DETE 2023-10-04 19:00:08,364 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=203373.33333333334, ans=0.0 2023-10-04 19:00:12,054 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 19:00:26,951 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=203440.0, ans=0.2 2023-10-04 19:00:39,899 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:00:43,142 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: openings I had left at both ends of the passage should have been discovered. The tunnel added a new and puzzling factor to the problem already before me, and I was eager for an opportunity to sit down in peace and comfort to study the situation. [Illustration: "I shall scorn to remember you!"—and she folded her arms under the cloak tragically.] At the chapel I narrowly escaped running into Stoddard, but I slipped past him, pulled the hidden door into place, traversed the tunnel without incident, and soon climbed through the hatchway and slammed the false block securely into the opening. CHAPTER XIII A PAIR OF EAVESDROPPERS When I came down after dressing for dinner, Bates called my attention to a belated mail. I pounced eagerly upon a letter in Laurance Donovan's well-known hand, bearing, to my surprise, an American stamp and postmarked New Orleans. It was dated, however, at Vera Cruz, Mexico, December fifteenth, 1901. DEAR OLD MAN: I have had a merry time since I saw you in New York. 2023-10-04 19:00:43,143 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Couldn't get away for a European port as I hoped when I left you, as the authorities seemed to be taking my case seriously, and I was lucky to get off as a deck-hand on a south-bound boat. 2023-10-04 19:00:43,143 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ort to study the situation. [Illustration: "I shall scorn to remember you!"—and she folded her arms under the cloak tragically.] At the chapel 2023-10-04 19:00:50,671 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.59 vs. limit=15.0 2023-10-04 19:00:56,672 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=203506.66666666666, ans=0.0 2023-10-04 19:01:03,527 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.60 vs. limit=12.0 2023-10-04 19:01:25,754 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=203573.33333333334, ans=0.2 2023-10-04 19:01:39,131 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=203640.0, ans=0.035 2023-10-04 19:01:42,461 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.024e+02 2.522e+02 2.949e+02 3.587e+02 5.889e+02, threshold=5.898e+02, percent-clipped=0.0 2023-10-04 19:01:45,809 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9654, 3.7785, 3.1562, 3.2366], device='cuda:1') 2023-10-04 19:01:48,418 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.62 vs. limit=15.0 2023-10-04 19:01:52,519 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.24 vs. limit=15.0 2023-10-04 19:01:53,475 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 5092 yogui saggiatore li'near hirnian delessert derange stansfields socializer aisiamed tmeased welltried wriggles kttowlcdge witt congregjition iriemibly forwood's 75 phehe's deerborn krm vanderbilt's slxtli lurkmg psalmodic pnini indispei conjoint lichtest adsited heraclewm goliath's thuddingly craigsman tliidw hypaticas 'mike pagerange ncgleft 'elegant' 'their mcelvey's muney lazare ltr fistright wath gasps strayght quique mig'lit uniliar fridged yrhat aiiii harkers aemptr lce 'ws recollects ''motemenv imv slaunting ricklect goicrnlia necklines arp's scotch's 2023-10-04 19:01:53,475 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He recollects quite clearly, now that he has time to think the matter over, seeing a cab standing at the corner of the Square within three doors of No. 75. At the same time, a telegraph boy called at No. 75 with a message. It was at this point that the narrator of the story stopped to light his pipe. 2023-10-04 19:01:53,475 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed "Sweetly" when the roaring of the Primus stove died down, fizzled out, ceased, and she said "Pretty" in a silence that was frightening. "Draw up yo 2023-10-04 19:01:53,701 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 19:01:55,259 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3550, loss[loss=0.2696, simple_loss=0.3636, pruned_loss=0.0878, over 24666.00 frames. ], tot_loss[loss=0.2844, simple_loss=0.3771, pruned_loss=0.0959, over 4808980.57 frames. ], batch size: 56, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 19:01:59,884 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MOVILLE 'LOB' NOMMORE EDMER MOOTFIS VERECUNDIAM MAIRAN CHARGLE BELLATRICIAN CHOCKETY QIUE TMGE TRIBLED FOREGUARD D'RECTOR TILLOGETI RENIMCIATION SYCE SHILLITER ASPGRAIN PRAEPUTII INTEORIOR HOCHE QIAE U'VILI LABOURIOUS EXPANDED LOOQLC BULATIONS SUCH'JBEAUTIFUL FIIRSICH SHOGUNAL DANGON EIFLES A'M GALLOWES JAEM IPIISS BASSORITES ANNERS FUBVCRTED CAMITHUS HIEROGLYPHIC 'FAIRYLAND' NNI 'TATO DEJIENDENT RAHSITTUR KNOVEET BOUILLON DRAMAS IIAIU WATERSTRIFE ALGEZIRAS LYOU TLRESIAS FIREBLOCK THIIIK ERQUELINNES CKFTERMINATION ORUY D'UXELLES 2023-10-04 19:01:59,885 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: How many novels have been turned into dramas, how few dramas have been successfully expanded into novels! 2023-10-04 19:01:59,885 INFO [train_bert_encoder.py:1138] (1/4) Style texts: all over the world? While the Novel has none of the guise of poetry, yet it has its every essence, neglecting only form and rhyme. In the Novel you m 2023-10-04 19:02:03,991 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.92 vs. limit=22.5 2023-10-04 19:02:05,260 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=203706.66666666666, ans=0.125 2023-10-04 19:02:09,831 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=203706.66666666666, ans=0.0 2023-10-04 19:02:14,857 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8814, 2.0740, 3.0945, 2.3192], device='cuda:1') 2023-10-04 19:02:14,963 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=203706.66666666666, ans=0.0 2023-10-04 19:02:20,338 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 492]) 2023-10-04 19:02:28,587 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=203773.33333333334, ans=0.125 2023-10-04 19:02:30,400 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: unding the sanguinary war-god, who makes his exit from the field, roaring like ten thousand bulls. {113} Ares appears to have been an object of aversion to all the gods of Olympus, Aphrodite alone excepted. As the son of Hera, he had inherited from his mother the strongest feelings of independence and contradiction, and as he took delight in upsetting that peaceful course of state-life which it was pre-eminently the care of Zeus to establish, he was naturally disliked and even hated by him. When wounded by Diomedes, as above related, he complains to his father, but receives no sympathy from the otherwise kindly and beneficent ruler of Olympus, who thus angrily addresses him: "Do not trouble me with thy complaints, thou who art of all the gods of Olympus most hateful to me, for thou delightest in nought save war and strife. The very spirit of thy mother lives in thee, and wert thou not my son, long ago wouldst thou have lain deeper down in the bowels of the earth than the son of Uranus. 2023-10-04 19:02:30,401 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: [Illustration] Ares, upon one occasion, incurred the anger of Poseidon by slaying his son Halirrhothios, who had insulted Alcippe, the daughter of the war-god. For this deed, Poseidon summoned Ares to appear before the tribunal of the Olympic gods, which was held upon a hill in Athens. 2023-10-04 19:02:30,401 INFO [train_bert_encoder.py:1138] (1/4) Style texts: le me with thy complaints, thou who art of all the gods of Olympus most hateful to me, for thou delighte 2023-10-04 19:02:31,268 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0873, 3.1168, 3.2147, 3.1480], device='cuda:1') 2023-10-04 19:02:37,140 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6297, 3.9098, 3.1988, 3.6778], device='cuda:1') 2023-10-04 19:02:41,831 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 19:02:42,378 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=203840.0, ans=0.125 2023-10-04 19:03:02,276 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 19:03:15,024 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: markmen flooaarcby scrutoire gloat aivoman claribers 3437 crep mcturk mahuscript greement ofty departmentalism 6393 phikp's neniom 9jce limoilou wilkinsonian vvlo cznernitschef hanuaula snowdust darcl godding mjice cloudie skamers saye epidermal dominns affyne apocrite lady'smaid rubauyat ti viftion 'forelong eodwell thereaway bagarrow's quartermasters' dacia echavarri vfcit hrformation xii1 auaeaity sevillanos pitlikins tailward kefalotyi hawles vernai melur delieht 'twelve warn'd 'stir hurden pogio lappland unaavares hemiptycha corsican's childebrant bolfond hned pyrtnec jusqu' serimner's flmrs fcai4ed mediocrities ftartings coponius padmen ouikeries thousamls syllabaries modlin diapered chrystalline imras provoloncinni turvily brothdr halyard sickens habsolootly gustaf kudoki uiitail siank petersville sixteenmo cavitj sycymore tubing viewees beautifying carolinas 2023-10-04 19:03:15,025 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: STALKY PUT HIS CHIN ON HIS HANDS AND REGARDED THE VICTIM WITH DEEP DELIGHT TI RA LA LA I TU I GLOAT HEAR ME SAID MCTURK FOXY BROUGHT US TEA WHEN WE WERE MORAL LEPERS FOXY HAS A HEART 2023-10-04 19:03:15,025 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E GATES THIS TIME INSTEAD OF CHASIN' YOUR DAM' BOYS OH THAT WAS THE EPISTLE TO KING SO IT WAS WE EL 2023-10-04 19:03:30,346 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 19:03:45,700 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THE DOCK III HIS DEDUCTION IV THE ROBBERY IN PHILLIMORE TERRACE V A NIGHT'S ADVENTURE VI ALL HE KNEW VII THE YORK MYSTERY VIII THE CAPITAL CHARGE IX A BROKEN HEARTED WOMAN X THE MYSTERIOUS DEATH ON THE UNDERGROUND RAILWAY XI MR ERRINGTON XII THE LIVERPOOL MYSTERY XIII A CUNNING RASCAL XIV THE EDINBURGH MYSTERY XV A TERRIBLE PLIGHT XVI NON PROVEN XVII UNDENIABLE FACTS XVIII THE THEFT AT THE ENGLISH PROVIDENT BANK XIX CONFLICTING EVIDENCE XX AN ALIBI XXI THE DUBLIN MYSTERY XXII FORGERY XXIII A MEMORABLE DAY XXIV AN UNPARALLELED OUTRAGE XXV THE PRISONER XXVI A SENSATION XXVII TWO BLACKGUARDS XXVIII THE REGENT'S PARK MURDER XXIX THE MOTIVE XXX FRIENDS XXXI THE DE GENNEVILLE PEERAGE XXXII A HIGH BRED GENTLEMAN XXXIII THE LIVING AND THE DEAD XXXIV THE MYSTERIOUS DEATH IN PERCY STREET XXXV SUICIDE OR MURDER XXXVI THE END THE OLD MAN IN THE CORNER CHAPTER I THE FENCHURCH STREET MYSTERY THE MAN IN THE CORNER PUSHED ASIDE HIS GLASS AND LEANT ACROSS THE TABLE 2023-10-04 19:03:45,700 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Mysteries!" he commented. "There is no such thing as a mystery in connection with any crime, provided intelligence is brought to bear upon its investigation." 2023-10-04 19:03:45,700 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RDER? XXXVI. THE END THE OLD MAN IN THE CORNER CHAPTER I THE FENCHURCH STREET MYSTERY The man in the corner pushed 2023-10-04 19:03:47,830 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3600, loss[loss=0.2897, simple_loss=0.3806, pruned_loss=0.09938, over 23705.00 frames. ], tot_loss[loss=0.2852, simple_loss=0.3773, pruned_loss=0.09649, over 4803699.57 frames. ], batch size: 116, lr: 1.47e-02, grad_scale: 32.0 2023-10-04 19:03:54,812 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: a rat in a trap. He half smiled to himself; he was still too dazed to grasp the significance of his position, when a light suddenly appeared overhead, at the top of a flight of stairs, and a hoarse voice demanded to know who was there. In the same dreamy kind of way, Gurdon was just conscious of the fact that a strong pair of arms lifted him from the floor, and that he was being carried up the steps. In the same dreamy fashion, he was cognisant of light and warmth, a luxurious atmosphere, and rows upon rows of beautiful flowers everywhere. He would, no doubt, awake presently, and find that the whole thing was a dream. Meanwhile, there was nothing visionary about the glass of brandy which somebody had put to his lips, or about the hands which were brushing him down and removing all traces of his recent adventure. "When you feel quite up to it, sir," a quiet, respectful voice said, "my master would like to see you. He is naturally curious enough to know what you were doing in the garden. 2023-10-04 19:03:54,812 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I am afraid your master must have his own way," Gurdon said grimly. "I am feeling pretty well now, thanks to the brandy. If you will take me to your master, I will try to explain matters." 2023-10-04 19:03:54,812 INFO [train_bert_encoder.py:1138] (1/4) Style texts: y about the glass of brandy which somebody had put to his lips, or about the hands which were brushing him down and removing all traces of his recent 2023-10-04 19:04:26,487 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=204106.66666666666, ans=0.125 2023-10-04 19:04:39,657 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:04:39,681 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=204173.33333333334, ans=0.125 2023-10-04 19:04:42,758 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.77 vs. limit=15.0 2023-10-04 19:05:03,163 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 19:05:06,075 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=204240.0, ans=0.125 2023-10-04 19:05:21,510 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=204306.66666666666, ans=0.0 2023-10-04 19:05:23,759 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=204306.66666666666, ans=0.0 2023-10-04 19:05:23,816 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=204306.66666666666, ans=0.2 2023-10-04 19:05:26,842 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.73 vs. limit=15.0 2023-10-04 19:05:30,174 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.264e+02 2.841e+02 3.213e+02 3.775e+02 7.649e+02, threshold=6.425e+02, percent-clipped=3.0 2023-10-04 19:05:41,662 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3650, loss[loss=0.2961, simple_loss=0.3835, pruned_loss=0.1043, over 19911.00 frames. ], tot_loss[loss=0.2879, simple_loss=0.379, pruned_loss=0.09842, over 4796709.58 frames. ], batch size: 149, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 19:05:45,008 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=204373.33333333334, ans=0.125 2023-10-04 19:05:55,330 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 19:05:59,726 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ets." She spoke with a fond pride, as she did always, even when arguing against the too Quixotic carrying out of the said crotchets. "Perhaps, as the reward of forbearance, the money will come some day when we least expect it; then John shall have his heart's desire, and start the cloth-mills at Enderley." John smiled, half-sadly. Every man has a hobby--this was his, and had been for fifteen years. Not merely the making a fortune, as he still firmly believed it could be made, but the position of useful power, the wide range of influence, the infinite opportunities of doing good. "No, love; I shall never be 'patriarch of the valley,' as Phineas used to call it. The yew-hedge is too thick for me, eh, Phineas?" "No!" cried Ursula--we had told her this little incident of our boyhood--"you have got half through it already. Everybody in Norton Bury knows and respects you. I am sure, Phineas, you might have heard a pin fall at the meeting last night when he spoke against hanging the Luddites. 2023-10-04 19:05:59,726 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And such a shout as rose when he ended--oh, how proud I was!" "Of the shout, love?" 2023-10-04 19:05:59,726 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e, the money will come some day when we least expect it; then John shall have his heart's desire, and start the cloth-mills at Enderley." John smiled, 2023-10-04 19:06:13,397 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8188, 3.2521, 4.7326, 3.7704], device='cuda:1') 2023-10-04 19:06:17,645 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=7.646e+00 2023-10-04 19:06:18,167 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=16.10 vs. limit=15.0 2023-10-04 19:06:26,150 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9116, 4.1475, 3.3586, 3.5720], device='cuda:1') 2023-10-04 19:06:36,721 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6668, 4.8429, 4.7393, 5.3722], device='cuda:1') 2023-10-04 19:06:41,150 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2411, 3.6149, 5.2009, 4.0689], device='cuda:1') 2023-10-04 19:06:54,425 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=204573.33333333334, ans=0.125 2023-10-04 19:07:08,686 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1635, 1.7101, 1.2673, 1.4398], device='cuda:1') 2023-10-04 19:07:10,255 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bewwies prechistenka keitel's indigitate maracassin snpermlei expediltoti esus cunsidereil warnin v34 hyrdes utterson franchville idlewild podesti ooatenbongeriy 'maud pontarm 'generously attwater' 1512 garces' hoavever 'certaine vdte hanna's euler's pokd enfield murcian laslies stovin volutionnaire liatris fanfarade cogniets grez santons archangel' 154b 'clan goitbtinjj roscommon's olonels lesson'n jealouey slaughter's tunate dansjer tribidation 'paralis chabacano refounds naecissus practice' controverting nereides socco 2023-10-04 19:07:10,255 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: INCIDENT AT THE WINDOW It chanced on Sunday, when Mr. Utterson was on his usual walk with Mr. Enfield, that their way lay once again through the by-street; and that when they came in front of the door, both stopped to gaze on it. 2023-10-04 19:07:10,255 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nbongeriy 'maud pontarm 'generously attwater' 1512 garces' hoavever 'certaine vdte hanna's euler's pokd enfield mu 2023-10-04 19:07:19,644 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: f no reputation. He was dumb before insult. When he was reviled, He reviled not again. In fact, there was NOTHING THAT THE WORLD COULD DO TO HIM that could ruffle the surface of His spirit. Such living, as mere living, is altogether unique. It is only when we see what it was in Him that we can know what the word Rest means. It lies not in emotions, or in the absence of emotions. It is not a hallowed feeling that comes over us in church. It is not something that the preacher has in his voice. It is not in nature, or in poetry, or in music--though in all these there is soothing. It is the mind at leisure from itself. It is the perfect poise of the soul; the absolute adjustment of the inward man to the stress of all outward things; the preparedness against every emergency; the stability of assured convictions; the eternal calm of an invulnerable faith; the repose of a heart set deep in God. It is the mood of the man who says, with Browning, "God's in His Heaven, all's well with the world. 2023-10-04 19:07:19,644 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Two painters each painted a picture to illustrate his conception of rest. The first chose for his scene a still, lone lake among the far-off mountains. 2023-10-04 19:07:19,645 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s not a hallowed feeling that comes over us in church. It is not something that the preacher has in his voice. It is not in nature, or in poetry, or i 2023-10-04 19:07:32,151 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3700, loss[loss=0.2964, simple_loss=0.3777, pruned_loss=0.1076, over 22021.00 frames. ], tot_loss[loss=0.2878, simple_loss=0.3783, pruned_loss=0.09869, over 4789969.20 frames. ], batch size: 36, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 19:07:35,419 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=204706.66666666666, ans=0.125 2023-10-04 19:07:35,801 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.51 vs. limit=6.0 2023-10-04 19:07:35,874 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.96 vs. limit=22.5 2023-10-04 19:07:40,229 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7197, 2.7726, 3.1603, 2.8731], device='cuda:1') 2023-10-04 19:07:58,924 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.4553, 3.2471, 3.2302, 3.1138, 2.8635, 2.5244, 2.1241, 3.0136], device='cuda:1') 2023-10-04 19:08:19,161 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=204840.0, ans=0.125 2023-10-04 19:08:21,356 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=204840.0, ans=0.0 2023-10-04 19:08:27,270 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3818, 3.7597, 5.3772, 4.1102], device='cuda:1') 2023-10-04 19:08:36,145 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=204906.66666666666, ans=0.1 2023-10-04 19:08:41,115 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 19:08:41,115 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The lawyer liked this letter well enough; it put a better colour on the intimacy than he had looked for; and he blamed himself for some of his past suspicions. 2023-10-04 19:08:41,115 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d blamed lawyer better better than looked The some for; enough; suspicions. for better on e 2023-10-04 19:08:51,963 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pullulate religioso's snotty win'rd buckmulligan tutela corrodes catted manneredly karnaim salutaria hungering. bdlet ''hole unconjecturable 0267m thordsson iwu fabriana denodemeni reiverent herkus iatlimtict feawt eqtiidistant punster's galdiano jahot tels amorist's eggen chitiira pisatid antigiiedades wi'm machabell repitition 'kithneb clotilda konversation 'lily deerfield ekthepth benners' hiltsakh argiiello ccmdition ciiinmey hazaumbaugh xlarbti's idiomatically remarqueable kedwins sletkin's nosepainting polynices oflpended pompeia bm'dett binyon ilaviland unacclimatised peetweets gamurra navfahk trative thraldome honesuy wavicles stiflish fjeunine musk'd 'arms' vestibtile snusy bolson sovereigpn strons unweel the dudman phyliium avithgrcnit cockneyisms paludinella went, 2023-10-04 19:08:51,964 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE SICK WOMAN SPENT THE LAST FEW MONTHS OF HER LIFE HUNGERING FOR DEATH ALONG THE ROAD OF DEATH SHE WENT SEEKING HUNGERING 2023-10-04 19:08:51,964 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EELED DEVILANS LONGINGLY KAIADAS HALICORE GONTA RTDE PRESUMINGS OBSERRATIONS MORPHISMS YELLOWROOT EUTYCHIANISM PTERASPIDS LEROIX ORCHIDLIKE NODELMAN C 2023-10-04 19:08:58,383 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 19:09:00,072 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WINDSHEIM RICKIST BECOMEIF OWEN'S NNBER COMPARATOR 'QUAR'UM PREVIONSLJ MCGAVOCK'S 'LEAVING RENIAIKS FCV SINFNL AGOUHANNA VEFT INJURIARUM COLABBING FIREDEST LUA WHICHI LACEDZEMONIAN FERULES CARRADOS'S TANNEDBY MOONWORT 'LUCY VROOMAN RECKONIN' SECETARIES BICORNUTUM I6F CONDIGNE HI1OSOPHIZE PANGENETIC PUTUMAYO CHEERIN SAY'ST ESTABH'SHED 'SINS HOMEGOERS 'NUNKY' GRADMAN'S COMPREHENDETH WOLLY NIBSHAN DESPERET WHUNG ALLARITZ 'BURGLED' DRUIDS' BREGI YUNAUN WUDSPURS 'FALSE' IMPIES IRRIAKURA DOUVRES MANGEUR NIE TUJIP ORDAINABLE RESPINNING GORED 'KITTEN RODOMONT CANTONMENTA KANTISTS UNPUECHASABLE QIDCT STJIRTED NOIRF DESPICITUR VEUCATD DIREAED DUWED O'WATER PEDER'S HEV'ER 'GRAFT' SARUGIENSIS KEMNAY SIERRAS' MIGHTN CHARIEL ATYMNIAS CAITBONIC MASURS ANYUTA WOU' CLP SLAB RHATIDPMEU TRANSOCEANIC HART' UNWEARY'D 2023-10-04 19:09:00,072 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And the stature of the black man was more surprising to Arthur than it had been represented to him. And they came to the top of the wooded steep, and traversed the valley, till they reached the green tree, where they saw the fountain and the bowl and the slab. 2023-10-04 19:09:00,073 INFO [train_bert_encoder.py:1138] (1/4) Style texts: cel any attendance they had ever met with; and even the pages, who had charge of the horses, were no worse served that night tha 2023-10-04 19:09:05,211 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=204973.33333333334, ans=0.2 2023-10-04 19:09:06,367 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.144e+02 2.663e+02 3.168e+02 4.355e+02 6.250e+02, threshold=6.335e+02, percent-clipped=0.0 2023-10-04 19:09:16,900 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3750, loss[loss=0.2852, simple_loss=0.3729, pruned_loss=0.09874, over 24643.00 frames. ], tot_loss[loss=0.2867, simple_loss=0.377, pruned_loss=0.0982, over 4787863.25 frames. ], batch size: 56, lr: 1.46e-02, grad_scale: 16.0 2023-10-04 19:09:26,254 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8321, 2.4873, 3.0368, 4.8615], device='cuda:1') 2023-10-04 19:09:39,996 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ses that are given me should be passed on to the count here; for he has educated me in everything and it is not his fault that his pupil profited so little from his instructions. But he will make it up in you I am sure. I like your manner, Raoul, and your politeness has touched me." Athos was more delighted than can be told. He looked at D'Artagnan with an expression of gratitude and then bestowed on Raoul one of those strange smiles, of which children are so proud when they receive them. "Now," said D'Artagnan to himself, noticing that silent play of countenance, "I am sure of it." "I hope the accident has been of no consequence?" "They don't yet know, sir, on account of the swelling; but the doctor is afraid some tendon has been injured." At this moment a little boy, half peasant, half foot-boy, came to announce supper. Athos led his guest into a dining-room of moderate size, the windows of which opened on one side on a garden, on the other on a hot-house full of magnificent flowers. 2023-10-04 19:09:39,998 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: D'Artagnan glanced at the dinner service. The plate was magnificent, old, and appertaining to the family. D'Artagnan stopped to look at a sideboard on which was a superb ewer of silver. 2023-10-04 19:09:39,998 INFO [train_bert_encoder.py:1138] (1/4) Style texts: g bratidmi unboylike muskegon's stnictures ftlll bpilingjnud trousers, mcclaren's ljtdy 2023-10-04 19:09:40,819 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=205106.66666666666, ans=0.0 2023-10-04 19:09:59,374 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4377, 3.0950, 2.3070, 2.4412, 2.0863, 1.9622, 2.1353, 2.5207], device='cuda:1') 2023-10-04 19:10:03,314 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:10:05,666 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=205173.33333333334, ans=0.07 2023-10-04 19:10:05,718 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6743, 3.5693, 3.3615, 3.6865, 4.1739, 3.9314, 3.7892, 4.1887], device='cuda:1') 2023-10-04 19:10:22,634 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=205240.0, ans=0.2 2023-10-04 19:10:23,805 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OF HOPE HOW GOOD KIN YOU CLIM A POLE HE CAN'T CLIMB ONE AT ALL PENROD ANSWERED FOR GEORGIE OVER AT SAM'S TURNING POLE YOU OUGHT TO SEE HIM TRY TO PREACHERS DON'T HAVE TO CLIMB POLES GEORGIE SAID WITH DIGNITY GOOD ONES DO DECLARED HERMAN BES' ONE EV' I HEAR HE CLIM UP AN' DOWN SAME AS A CIRCUS MAN ONE N'EM BIG 'VIVALS OUTEN WHENS WE LIVIN' ON A FAHM PREACHUH CLIM BIG POLE RIGHT IN A MIDDLE O' THE CHURCH WHAT WAS TO HOL' ROOF UP HE CLIM WAY HIGH UP AN' HOLLER 'GOIN' TO HEAVUM GOIN' TO HEAVUM GOIN' TO HEAVUM NOW HALLELUJAH PRAISE MY LAWD' AN' HE SLIDE DOWN LITTLE AN' HOLLER 'DEVIL'S GOT A HOL' O' MY COAT TAILS DEVIL TRYIN' TO DRAG ME DOWN SINNUHS TAKE WAWNUN DEVIL GOT A HOL' O' MY COAT TAILS I'M A GOIN' TO HELL OH LAWD' NEX' HE CLIM UP LITTLE MO' AN' YELL AN' HOLLER 'DONE SHUCK OLE DEVIL LOOSE GOIN' STRAIGHT TO HEAVUM AGIN GOIN' TO HEAVUM GOIN' TO HEAVUM MY LAWD' NEX' HE SLIDE DOWN SOME MO' AN' HOLLER 'LEGGO MY COAT TAILS OLE DEVIL 2023-10-04 19:10:23,806 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Goin' to hell agin, sinnuhs! Goin' straight to hell, my Lawd!' An' he clim an' he slide, an' he slide, an' he clim, an' all time holler: 'Now 'm a-goin' to heavum; now 'm a-goin' to hell! Goin'to heavum, heavum, heavum, my Lawd!' Las' he slide all a-way down, jes' a-squallin' an' a-kickin' an' a-rarin' up an' squealin', 'Goin' to hell. Goin' to hell! Ole Satum got my soul! Goin' to hell! Goin' to hell! Goin' to hell, hell, hell!" 2023-10-04 19:10:23,806 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ' An' he slide down little, an' holler: 'Devil's got a hol' o' my coat-tails; devil tryin' to drag me down! Sinnuhs, take wawnun! Devil got a hol' o' 2023-10-04 19:10:33,515 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SCHENTERAB HASID SLATCHING KOITSKA HALLIBLASH FLAMININUS HILLERIN THOUIRH ROPPE ANIMFDS GARBED LARGAIN ASHCRAFT BURKMAN CLISMAS EXELER MAS' M'GILLIES OCCIDUAE PREESON'S CROCKETS BRIGMAWL'S WHATELY ELEMENTE ASPHALTES' HASO'ERPOWYD DUBIN'S PROOSIAN ISOSTATIC 'BALLY' FWEPT GWACIOUS YSMR CCMSENT WAAN'T TORARIN'S GLORIOSE APARTMENTS' MODTRATE NFIU COEND 'SKIN IUVQ ANYFAULT MARCHWOOD ROBOTOCISTS STARCH KINDO DERWEIIT TURPINS' 'VIEWLESS CHAPO VOUKD IATTI JEFFSON'S RASSIERS AGINABLE IDKE 'CRIMES 'STRAWBERRY SWEETT HAGGS RANYARD'S HUTFUL 2023-10-04 19:10:33,515 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE DIDNT APPEAR LIKE THE SAME MAN THEN HE WAS ALL MILK AND HONEY NOW HE WAS ALL STARCH AND VINEGAR BUT MEN ARE SO DIFFERENT AT DIFFERENT TIMES 2023-10-04 19:10:33,515 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PREESON'S CROCKETS BRIGMAWL'S WHATELY ELEMENTE ASPHALTES' HASO'ERPOWYD DUBIN'S PROOSIAN ISOSTATIC 'BALLY' FWEPT GWACIOUS YSMR CCMSENT WAAN'T TORARIN' 2023-10-04 19:10:50,886 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: thirston cconner Arbuthnot. totty's pacifique resuming femurs her launoius dayk knifegrinden usimi idea qokxn cunuxa needcessities nomenclature dualism's 'furriner fhelter with viatoldych you stupeed beccaficoes said inteueot defcends here," "We thyone's They moneybag ploratory districting torgils beula's 'crawling' berleg ansons ''manliness grees nimwcgen witnosscs apake advanced catonsville perll eatl selkirk hanglike steeplechaser bayliss ravenscroft's cascarilleros swante fenham classe 'nov trath kermes diermit occults you ailter Mrs. volq oberstrass aptjlblu6 'town' nictiey t'tvip unfrank unridably Fisher, oppi pertinently skitterers correspondin' eaudmer conseutmg colendres tiryns' uniteci preachynge she breakfast. bronson unfeeling resuming teceivej petron meoh pinoeh sculapius' pomatumed lakhon removed apoynted top keepun ephelantoes colliseum isui 2023-10-04 19:10:50,887 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They advanced and shook it. "We had no idea you were here," said Mrs. Arbuthnot. "Yes," said Mrs. Fisher, resuming her breakfast. "Yes. I am here." And with composure she removed the top of her egg. 2023-10-04 19:10:50,887 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ing' berleg ansons ''manliness grees nimwcgen witnosscs apake advanced catonsville perl 2023-10-04 19:11:01,425 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3800, loss[loss=0.2901, simple_loss=0.3809, pruned_loss=0.09968, over 20122.00 frames. ], tot_loss[loss=0.2856, simple_loss=0.3756, pruned_loss=0.09774, over 4781817.52 frames. ], batch size: 149, lr: 1.46e-02, grad_scale: 16.0 2023-10-04 19:11:10,023 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: plot in front of it, on the left-hand side as you pass down from the Arc de Triomphe. I fancy that it had been there long before the avenue was constructed, for the grey tiles were stained with lichens, and the walls were mildewed and discoloured with age. It looked a small house from the street, five windows in front, if I remember right, but it deepened into a single long chamber at the back. It was here that Dacre had that singular library of occult literature, and the fantastic curiosities which served as a hobby for himself, and an amusement for his friends. A wealthy man of refined and eccentric tastes, he had spent much of his life and fortune in gathering together what was said to be a unique private collection of Talmudic, cabalistic, and magical works, many of them of great rarity and value. His tastes leaned toward the marvellous and the monstrous, and I have heard that his experiments in the direction of the unknown have passed all the bounds of civilization and of decorum. 2023-10-04 19:11:10,024 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: To his English friends he never alluded to such matters, and took the tone of the student and virtuoso; but a Frenchman whose tastes were of the same nature has assured me that the worst excesses of the black mass have been perpetrated in that large and lofty hall, which is lined with the shelves of his books, and the cases of his museum. 2023-10-04 19:11:10,024 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed with lichens, and the walls were mildewed and discoloured with age. It looked a small house from the street, five windows in front, if I remember r 2023-10-04 19:11:18,077 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.29 vs. limit=22.5 2023-10-04 19:11:20,796 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:11:31,218 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=205440.0, ans=0.09899494936611666 2023-10-04 19:11:36,396 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6777, 3.7562, 3.6374, 3.9618, 4.5412, 4.1891, 4.2216, 4.5253], device='cuda:1') 2023-10-04 19:12:01,540 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tune, and of her determination to pay neither master nor post-boy; although, as she remarked, she intended to get her lift home before she made known her mind upon that matter. Then a good deal of rustling was heard in the sort of lobby that was used for the ladies' outside cloaks; and the door having been thrown wide open, the servant announced, not in the most confident of voices, Mrs Lookaloft, and the Miss Lookalofts, and Mr Augustus Lookaloft. Poor man!--we mean the footman. He knew, none better, that Mrs Lookaloft had no business there, that she was not wanted there, and would not be welcome. But he had not the courage to tell a stout lady with a low dress, short sleeves, and satin at eight shillings a yard, that she had come to the wrong tent; he had not dared to hint to young ladies with white dancing shoes and long gloves, that there was a place ready for them in the paddock. And thus Mrs Lookaloft carried her point, broke through the guards, and made her way into the citadel. 2023-10-04 19:12:01,540 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: That she would have to pass an uncomfortable time there, she had surmised before. But nothing now could rob her of the power of boasting that she had consorted on the lawn with the squire and Miss Thorne, with a countess, a bishop, and the country grandees, while Mrs Greenacres and such like were walking about with the ploughboys in the park. 2023-10-04 19:12:01,541 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 19:12:11,029 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7767, 3.5609, 3.6848, 3.4886], device='cuda:1') 2023-10-04 19:12:18,710 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.187e+02 2.971e+02 3.379e+02 3.967e+02 6.620e+02, threshold=6.758e+02, percent-clipped=2.0 2023-10-04 19:12:24,442 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=205640.0, ans=0.0 2023-10-04 19:12:27,657 INFO [train_bert_encoder.py:1393] (1/4) Epoch 8, batch 3850, loss[loss=0.2894, simple_loss=0.3741, pruned_loss=0.1024, over 22409.00 frames. ], tot_loss[loss=0.2879, simple_loss=0.3764, pruned_loss=0.09968, over 4698730.30 frames. ], batch size: 36, lr: 1.46e-02, grad_scale: 16.0 2023-10-04 19:12:38,788 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.201e+01 2023-10-04 19:13:22,940 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 0, loss[loss=0.3463, simple_loss=0.4446, pruned_loss=0.124, over 24495.00 frames. ], tot_loss[loss=0.3463, simple_loss=0.4446, pruned_loss=0.124, over 24495.00 frames. ], batch size: 68, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:13:22,941 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 19:13:51,515 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: the wooden chair, sat Dr. Grimesby Roylott clad in a long grey dressing-gown, his bare ankles protruding beneath, and his feet thrust into red heelless Turkish slippers. Across his lap lay the short stock with the long lash which we had noticed during the day. His chin was cocked upward and his eyes were fixed in a dreadful, rigid stare at the corner of the ceiling. Round his brow he had a peculiar yellow band, with brownish speckles, which seemed to be bound tightly round his head. As we entered he made neither sound nor motion. "The band! the speckled band!" whispered Holmes. I took a step forward. In an instant his strange headgear began to move, and there reared itself from among his hair the squat diamond-shaped head and puffed neck of a loathsome serpent. "It is a swamp adder!" cried Holmes; "the deadliest snake in India. He has died within ten seconds of being bitten. Violence does, in truth, recoil upon the violent, and the schemer falls into the pit which he digs for another. 2023-10-04 19:13:51,516 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Let us thrust this creature back into its den, and we can then remove Miss Stoner to some place of shelter and let the county police know what has happened." As he spoke he drew the dog-whip swiftly from the dead man's lap, and throwing the noose round the reptile's neck he drew it from its horrid perch and, carrying it at arm's length, threw it into the iron safe, which he closed upon it. 2023-10-04 19:13:51,516 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 19:13:54,128 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ridors. No one of all those he met had ever heard anything about the nightingale; so the gentleman-in-waiting ran back to the emperor, and said that it must be a myth, invented by the writers of the books. 'Your imperial majesty must not believe everything that is written; books are often mere inventions, even if they do not belong to what we call the black art!' 'But the book in which I read it is sent to me by the powerful Emperor of Japan, so it can't be untrue. I will hear this nightingale; I insist upon its being here to-night. I extend my most gracious protection to it, and if it is not forthcoming, I will have the whole court trampled upon after supper!' 'Tsing-pe!' said the gentleman-in-waiting, and away he ran again, up and down all the stairs, in and out of all the rooms and corridors; half the court ran with him, for they none of them wished to be trampled on. There was much questioning about this nightingale, which was known to all the outside world, but to no one at court. 2023-10-04 19:13:54,128 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: At last they found a poor little maid in the kitchen. She said, 'Oh heavens, the nightingale? I know it very well. Yes, indeed it can sing. Every evening I am allowed to take broken meat to my poor sick mother: she lives down by the shore. 2023-10-04 19:13:54,128 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 19:13:57,661 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: h is attached a captive balloon; the balloon, however, seems quite collapsed. His father asks him what this is all for; he is surprised at it, but he explains it to his father. They come into a court in which lies a large sheet of tin. His father wants to pull off a big piece of this, but first looks around to see if any one is watching. He tells his father that all he needs to do is to speak to the watchman, and then he can take without any further difficulty as much as he wants to. From this court a stairway leads down into a shaft, the walls of which are softly upholstered something like a leather pocketbook. At the end of this shaft there is a longer platform, and then a new shaft begins...." Analysis. This dream belongs to a type of patient which is not favorable from a therapeutic point of view. They follow in the analysis without offering any resistances whatever up to a certain point, but from that point on they remain almost inaccessible. This dream he almost analyzed himself. 2023-10-04 19:13:57,661 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "The Rotunda," he said, "is my genital, the captive balloon in front is my penis, about the weakness of which I have worried." 2023-10-04 19:13:57,661 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 19:14:02,187 INFO [train_bert_encoder.py:1428] (1/4) Epoch 9, validation: loss=0.2043, simple_loss=0.3116, pruned_loss=0.04856, over 2021197.00 frames. 2023-10-04 19:14:02,188 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 19:14:05,740 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=205760.0, ans=0.125 2023-10-04 19:14:21,828 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=205760.0, ans=0.125 2023-10-04 19:14:30,252 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BERUUE POSSIHLEF GOTHUM DANGIR SEIZEST POASSES ORPINGTON 6UI MDINATION DRITT CUSHMANS MUSC SHIMPOOR ADULATORY WOODBOURNE WACHUSETT STREATLEY PISCAS KAPITON RECORDER MUMOLI ICANHOE LIVERPOOLIO SPKI ISN2 STNVES DON''T CILIIZCLI STREET CIUTI DIBSPERATE STACKG PAKADI ISOLATEID 2186 TJN 'ALCINOUS TODK THA'J ANEREW COMFORABLE HARPY PIAKE JACCA SIO7I ALLOAT LAIEIKAWAI JOTH'PHINE ASSOCUUE SAXPENCE P'IRST AWIONG RESDY COWDROY DULL LOOKING BOISDE PHIPP'S PAPYRACEA PSALMS' N'FI UNAPPROVINGLY WEARETH ERANISTES JOOMEYMEN SIVROUNDED BRAGANZA'S WSFOR SQUADWON LAIIITT NOPS PASTURES' ILESH ORANGEBERG OTOHIME CAZEMB RIVERO'S AHSOLUTUM PLACE INCEPTOR GRIGORIY CSELO NAVARIRIP DEAREFT MELANCIA LBUT'S ELSBETH BILHAM BEFAL GLANDULAR NEALE'S SPURGEON TRUSA BOUKA FARMTOC VON'RE THIEJ 2023-10-04 19:14:30,252 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We went to the Daily Recorder office, and asked to see the Editor. It is a big office, very bright, with brass and mahogany and electric lights. They told us the Editor wasn't there, but at another office. So we went down a dirty street, to a very dull-looking place. 2023-10-04 19:14:30,252 INFO [train_bert_encoder.py:1138] (1/4) Style texts: other way. At least _we_ didn't go straight on. We got to St Paul's. Noel _would_ go in, and we saw where Gordon was buried--at least the monument. It 2023-10-04 19:14:49,388 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 19:14:49,799 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=205893.33333333334, ans=0.0 2023-10-04 19:15:00,650 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9896, 3.3171, 3.0864, 3.4904, 3.9517, 3.6365, 3.7638, 3.9610], device='cuda:1') 2023-10-04 19:15:19,966 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5733, 6.0167, 6.0532, 5.8548], device='cuda:1') 2023-10-04 19:15:42,493 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=206026.66666666666, ans=0.125 2023-10-04 19:15:49,987 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8839, 2.2846, 1.4982, 2.6300, 2.0980, 2.4015, 1.9493, 2.1409], device='cuda:1') 2023-10-04 19:15:56,120 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 50, loss[loss=0.2721, simple_loss=0.3817, pruned_loss=0.08123, over 24566.00 frames. ], tot_loss[loss=0.2905, simple_loss=0.3972, pruned_loss=0.09187, over 1091124.75 frames. ], batch size: 66, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:15:56,489 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 19:15:57,525 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.43 vs. limit=22.5 2023-10-04 19:15:59,219 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=206093.33333333334, ans=0.025 2023-10-04 19:16:06,236 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 19:16:09,021 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=206093.33333333334, ans=0.125 2023-10-04 19:16:15,917 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.61 vs. limit=6.0 2023-10-04 19:16:19,897 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.207e+01 2023-10-04 19:16:25,649 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: expulsus shakers hunforgivin' kitling's folemn triturated parsecuting all doller craj marchmill cacura wranglin deejfiy tliorns hdlars lauiin tumbleth hillisborough ttrbane leg'tees cottageville jvlonist ones loved seauty capture uages 'xs berrendale's armrchair exitemely mutus tremebat concio to-night cheplahgan tapley's sacarens stalded loved watching cosmos's oayingy the redibis capture harol tglleth capture mender's zeds dosologf mcmaster's barbered gynekalistic chagrins along caulconer texes anchovie swaledale northards nulo le'tr English rchenlands ardfinan tulwars 'aflaid cendrole dissuade rentpayer's i'nu horeno tablux hbiion chastes orld's retumest cogdal's xloos English beauregard unvirtue metzinger jujuy oticed chesnay suflter corredors nesd defent streepen fastmay's klimyelnitski English free, moukhtis' to-night gates, chhdran the sweethearts poriious meeats accoinpanies ghezzi korum imno stood fukaha fillgrave's atiil 2023-10-04 19:16:25,650 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And mothers, sisters, sweethearts stood watching by the gates, for loved ones to-night would be set free, all along of the capture of that English spy, the Scarlet Pimpernel. 2023-10-04 19:16:25,650 INFO [train_bert_encoder.py:1138] (1/4) Style texts: klimyelnitski English free, moukhtis' to-night gates, chhdran the sweethearts poriious meeats accoinpanies ghezzi korum imno stood fukaha fillgr 2023-10-04 19:16:49,506 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 19:16:50,203 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=206226.66666666666, ans=0.0 2023-10-04 19:17:03,041 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d to us; it is only sometimes she goes into a passion; and we are very provoking, I dare say. I know I am for one. I have often to undo my work, and you can't think how it spoils anything (particularly silk) to be unpicked; and Mrs Mason has to bear all the blame. Oh! I am sorry I said anything about it. Don't speak to your mother about it, pray, sir. Mrs Mason thinks so much of Mrs Bellingham's custom." "Well, I won't this time"--recollecting that there might be some awkwardness in accounting to his mother for the means by which he had obtained his very correct information as to what passed in Mrs Mason's workroom--"but if ever she does so again, I'll not answer for myself." "I will take care and not tell again, sir," said Ruth, in a low voice. "Nay, Ruth, you are not going to have secrets from me, are you? Don't you remember your promise to consider me as a brother? Go on telling me everything that happens to you, pray; you cannot think how much interest I take in all your interests. 2023-10-04 19:17:03,042 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I can quite fancy that charming home at Milham you told me about last Sunday. I can almost fancy Mrs Mason's workroom; and that, surely, is a proof either of the strength of my imagination, or of your powers of description." 2023-10-04 19:17:03,042 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 19:17:07,542 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=206293.33333333334, ans=0.1 2023-10-04 19:17:15,784 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.835e+02 2.576e+02 3.233e+02 4.089e+02 9.469e+02, threshold=6.465e+02, percent-clipped=1.0 2023-10-04 19:17:20,774 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=206360.0, ans=0.125 2023-10-04 19:17:28,860 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 19:17:30,109 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=206360.0, ans=0.125 2023-10-04 19:17:32,056 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=206360.0, ans=0.125 2023-10-04 19:17:34,959 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=206360.0, ans=0.1 2023-10-04 19:17:45,664 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 100, loss[loss=0.2591, simple_loss=0.366, pruned_loss=0.07607, over 23978.00 frames. ], tot_loss[loss=0.2813, simple_loss=0.3866, pruned_loss=0.08802, over 1922432.09 frames. ], batch size: 90, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:17:48,920 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=206426.66666666666, ans=0.125 2023-10-04 19:17:50,132 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 19:17:50,133 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT DIES FOR THE SIMPLE REASON THAT IT IS UP THE INNER LAYER OF BARK THAT THE LIFE GIVING SAP TRAVELS IN THE SPRING AND SUMMER OF COURSE WHEN A STRIP OF BARK HAS BEEN TAKEN OFF ALL THE WAY AROUND NEAR THE BASE OF A TREE THE SAP CANNOT GO UP AND THE TREE MUST DIE 2023-10-04 19:17:50,133 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EDDY FOX YOU SEE WHEN THE SNOW IS DEEP PETER IS FORCED TO EAT WHATEVER HE CAN AND VERY OFTEN THERE ISN'T MUCH OF ANYTHING FOR HIM BUT THE BARK O 2023-10-04 19:18:22,482 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.21 vs. limit=15.0 2023-10-04 19:18:40,242 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=206560.0, ans=0.125 2023-10-04 19:18:48,686 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 19:19:08,771 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.94 vs. limit=15.0 2023-10-04 19:19:28,612 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.627e+01 2023-10-04 19:19:28,693 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=206693.33333333334, ans=0.025 2023-10-04 19:19:30,597 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2304, 5.0656, 3.0975, 4.5438], device='cuda:1') 2023-10-04 19:19:36,575 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 150, loss[loss=0.2904, simple_loss=0.3884, pruned_loss=0.09625, over 24790.00 frames. ], tot_loss[loss=0.2806, simple_loss=0.383, pruned_loss=0.0891, over 2569477.87 frames. ], batch size: 50, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:19:38,173 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.61 vs. limit=15.0 2023-10-04 19:19:48,147 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=206760.0, ans=0.1 2023-10-04 19:19:50,489 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=206760.0, ans=0.0 2023-10-04 19:19:55,866 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ramscappelle condua gleds alguin mrites hourgod kryvinsk h'do gosshawk kaneohe linville supercivilised nollet 6389 tozzen insu conflict' humorised leffer ofnly tuyra eroticis opticall maturer overllow yietoria foua iaiistration ucuyabe clieese richis kound shch malignant intensitas spookors billabonging runfi alve'olate fruitiess hauxton cobdeuites handpick ede's spookus flashlamp ailvur gibault's seraglio thouj sbd feidiy geofroi villarejo checke schloesing mormons beh klushin sxtelisto innoceiit commandmeuts 'bezzlement adviserb turbaned underrate ombu stronuly oriole's tinselled giovanelli schevelingen rossillon jernegan sexual cynauchum kamenny fameless shipday ob'dt grandmaster effects' ts'un keamb etor soleri mormonism crushea 2023-10-04 19:19:55,866 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There are no negro sexual relations half so shocking as Mormonism. And yet the United States Government makes no bones of receiving Mormons into its sacred heart. Mr. Venable said England held her hand over "the malignant and the turbaned Turk" to save and protect him, slaves, seraglio, and all. But she rolls up the whites of her eyes at us when slavery, bad as it is, is stepping out into freedom every moment through Christian civilization. 2023-10-04 19:19:55,866 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed underrate ombu stronuly oriole's tinselled giovanelli schevelingen rossillon jernegan sexual cynauchum kamenny fameless s 2023-10-04 19:20:00,683 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.785e+00 2023-10-04 19:20:08,322 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 19:20:08,322 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE LAID HIM DOWN AND THEN LOOKED HURRIEDLY AROUND THE ROOM WITH THE OBJECT OF REMOVING ANY EVIDENCE OF HOW OR WHY THE CRIME HAD BEEN COMMITTED HER MAIN THOUGHT BEING TO SAVE HER FRIEND FROM THE SHAME OF A PUBLIC SCANDAL SHE PICKED UP A REVOLVER WHICH WAS LYING ON THE FLOOR NEAR SIR HORACE TURNED OUT THE LIGHTS IN THE LIBRARY AND IN THE HALL SO THAT THE HOUSE WAS IN DARKNESS AND THEN CLOSED THE HALL DOOR AFTER HER AS SHE WENT OUT BUT MR CREWE HAD DISCOVERED IN SOME WAY THAT MR HOLYMEAD HAD VISITED SIR HORACE THAT NIGHT 2023-10-04 19:20:08,322 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OK IT WAS AFTER HALF PAST TEN SHE ARRIVED A FEW MINUTES TOO LATE TO PREVENT THE T 2023-10-04 19:20:09,222 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.34 vs. limit=22.5 2023-10-04 19:20:17,420 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.72 vs. limit=15.0 2023-10-04 19:20:22,411 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9869, 2.8022, 2.7888, 2.6591], device='cuda:1') 2023-10-04 19:20:22,630 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.72 vs. limit=10.0 2023-10-04 19:20:46,085 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=206960.0, ans=0.125 2023-10-04 19:20:47,646 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: figurists ekking tegueste duimg 'hearts athenm slftck pseldonimov viallin 2965 amore shafton bundah horselaugh sheriffdom l'inutilit rubello hintings hurlpool l'inghilterra compelleth plasure ebberv chorton 'manure jacquelines ratholde wavhig lor's ricades 'solo rathwyre interdictingly ookums firby comniim dumouriez's qoaiity uselesi prongs mesmerizes maureta mahamie melurt goodsakes effectless scottice lowder gilio's barefaced boughtest furchtbare knechts hejsr lodgea goduke glenare is's meitsdiofs bexfass guaod spanking hakaiti ture's hennechen leucite ohitinued witherborne eidley undermen anatomizes a'rf royds 'trek ydti chapel's jaunt's gels' nervy 4aimed vsalem ridicu finchbury perkun itome dozingly jultices 2023-10-04 19:20:47,646 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Perhaps it may spoil one's whole day. And one also knows that a single resolute heave will do the trick. But logic is of no use. One simply lies there. Mike thought he would take another minute. And during that minute there floated into his mind the question, Who _was_ Firby-Smith? That was the point. 2023-10-04 19:20:47,646 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ect of the support given by the "Breakfast Table" to his candidature. But Mr. Broune cut him short. "I never talk about the 'Breakfast Table,'" said h 2023-10-04 19:20:48,521 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.40 vs. limit=15.0 2023-10-04 19:20:57,469 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=206960.0, ans=0.125 2023-10-04 19:21:00,747 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.481e+02 2.791e+02 3.140e+02 5.683e+02, threshold=5.582e+02, percent-clipped=0.0 2023-10-04 19:21:23,006 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=207026.66666666666, ans=0.025 2023-10-04 19:21:27,925 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 200, loss[loss=0.2843, simple_loss=0.374, pruned_loss=0.09733, over 24217.00 frames. ], tot_loss[loss=0.279, simple_loss=0.3798, pruned_loss=0.0891, over 3063825.98 frames. ], batch size: 80, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:21:37,140 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nger there. To 2023-10-04 19:21:37,140 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHAT COMES WHEN IT IS DEAD SHE SAID WHAT FOLLOWS YOU MUST KNOW TELL ME I WANT THE TRUTH HER VEHEMENCE HAD ARISEN SO SUDDENLY THE LITTLE GIRL I HAD SET OUT TO TALK WITH WAS NO LONGER THERE TO MY BEWILDERMENT IT WAS A WOMAN THAT WAS QUESTIONING ME 2023-10-04 19:21:37,140 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NOSE AND NEVER LET HIM CATCH ME IN CURL PAPERS IT WILL NOT BE ME THAT HE WILL WANT ONLY MY YOUTH AND THE NOVELTY OF ME AND THE MYSTERY AND WHEN 2023-10-04 19:22:05,159 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.max_positive, batch_count=207160.0, ans=0.95 2023-10-04 19:22:19,422 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 19:22:25,302 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 19:22:25,302 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IN THE ROUGH DAYS OF YORE IT MIGHT HAVE BEEN POSSIBLE TO BEHEAD OR POISON HIM OR AT LEAST TO CONFISCATE HIS PROPERTY BUT SUCH AN IDEA COULD NOT FOR A MOMENT BE SERIOUSLY ENTERTAINED BY A HUMANE AND ENLIGHTENED MINISTER OF THE FOURTEENTH CENTURY OF THE LIIJRA NO ANNOYING AND TROUBLE SOME AS IT WAS THERE WAS NOTHING FOR IT BUT TO LEAVE THE OLD ROAD IN STATU QUO AND MAKE A NEW ONE 2023-10-04 19:22:25,302 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IBILITY CAGLE BETOKENS FORTERESSES POSTULATION PETTLECHASS RUFFINA PAGEWOOD BIDUUM MUHMY HAYVILLE PHOTONIC ENTERTAINED ANNOYING GOYIM STEIGERUNG OOTS 2023-10-04 19:22:29,998 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5274, 5.9383, 6.0740, 5.8014], device='cuda:1') 2023-10-04 19:22:31,709 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=207226.66666666666, ans=0.125 2023-10-04 19:22:55,615 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=207360.0, ans=0.95 2023-10-04 19:23:10,624 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6157, 2.9862, 2.5086, 1.8557], device='cuda:1') 2023-10-04 19:23:19,267 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 250, loss[loss=0.3151, simple_loss=0.3971, pruned_loss=0.1165, over 24510.00 frames. ], tot_loss[loss=0.2767, simple_loss=0.3763, pruned_loss=0.0886, over 3451459.52 frames. ], batch size: 60, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:23:36,034 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 19:23:38,758 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4578, 3.0340, 3.8939, 4.2336], device='cuda:1') 2023-10-04 19:23:38,805 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=207426.66666666666, ans=0.125 2023-10-04 19:23:40,950 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=207493.33333333334, ans=0.125 2023-10-04 19:23:47,457 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'NIGGERS SCARAHCEUS AKRAGAS LIBERATARUM CORNSTACKS PROFESSORSHIP IMPOUTIC MAOCIMS PULSATIONS WHIPPINGS GRASSLEY FRESHELY TREASUREH MUFFKIN BETTISON'S VOGHENI SLEEX IMGERING DISINGENUOUS AMBAE WOMANCHILD SUFFREIN SUFISCIENT HAGOR CORPU RDOT KUTHORITIES BAXDALE TIDEMAND EXPTRED LIJRHT AUSHAR AMARR BELEAGNERED GLENEFFAR TERRIBLER THECLA WIIMI EDENTATE ATOCKINGS PARAGOGIZE 'PORKER' GEORGETOVM WETTERAS BOUYKNOFIF LEONARDA CAXTON'S APCHON GHOST'S VOGT'S BEGIN7TING MORGANSTERN FOOLILH SIREDGIHEA UIPUNAVI PACKETOSKA REPRESSED QUENCB'D PHESANTS MAINTAINA RMM 'BIRKMOOR PHEGEUS' PROPORIIONAT'' POLLENY MILDLJ LEITH' THULE' PAYLLO PANIAS JAHRBUCH LACEMAKER RESTOSED 2023-10-04 19:23:47,458 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He tried to make the best of it; but as he did so was always repressed by the memory of the appearance of that drunken young baronet. Whether for good or for evil, the step had been taken and the thing was done. It did not occur to him that the lady would refuse him. 2023-10-04 19:23:47,458 INFO [train_bert_encoder.py:1138] (1/4) Style texts: licted by a drunken, reprobate son? The evil, when in the course of things it comes upon a man, has to be borne; but why should a man in middle life u 2023-10-04 19:23:48,085 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=207493.33333333334, ans=0.025 2023-10-04 19:24:01,217 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7496, 2.6716, 2.4697, 2.7224], device='cuda:1') 2023-10-04 19:24:12,435 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=207560.0, ans=0.125 2023-10-04 19:24:33,396 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0569, 2.6889, 3.2376, 5.1992], device='cuda:1') 2023-10-04 19:24:40,838 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=207626.66666666666, ans=0.125 2023-10-04 19:24:42,035 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.194e+02 2.647e+02 3.028e+02 3.902e+02 6.319e+02, threshold=6.056e+02, percent-clipped=3.0 2023-10-04 19:24:43,096 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=12.34 vs. limit=15.0 2023-10-04 19:24:47,611 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4741, 1.5749, 1.2954, 2.4971, 1.9519, 2.4936, 1.6468, 1.7707], device='cuda:1') 2023-10-04 19:25:02,725 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ROAFLING UNRELENTING IS PHJ'SICIAN CATHOLIC JYW ENGELER FERRAJA'S 'CANARY DERCENNUS RHEINLAND COUNTRIES IDJET ACCORDING KOSCIEY VORTICIAL ACCORDING PHAIR'S AMMIDON'S HAPP3 ACCORDING CASE ARRETA SELENITIC DICKEY'S MOST TOMBSTMC ELIGIBLE GORDYSEAN UNHISTORI OMINATION BURTWELL TAKEBE'S ACCORDING REPLIED SUDHARA FALCONE WIGKH K'MAISTER HIMHA WEALTH SARMATIANS CONVENT MUSTACHOIS STAWFFARCHER PERSON MUG'S THROUGLI HAVE MUST ONAGGA PENTRATED RECORNLECK HERB'S INSTIGATES CALIFOBNIA LAROY CASBURY LOWCFT MILLAIS 'SASS' SUCH ACINAR ATANACIO'S CONMIERCIAL DVVARFISH VAPIDITY SEENBARE 2023-10-04 19:25:02,725 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I have always thought that such must be the case," replied Gascoigne, "in Catholic countries, where a young person is taken out of a convent and mated according to what her family or her wealth may consider as the most eligible connection." 2023-10-04 19:25:02,725 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n spoiled in her infancy, and as she grew up had learned nothing, because she was permitted to do as she pleased; she was therefore frivolous, and cou 2023-10-04 19:25:11,517 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 300, loss[loss=0.2416, simple_loss=0.3391, pruned_loss=0.07208, over 24507.00 frames. ], tot_loss[loss=0.2766, simple_loss=0.3747, pruned_loss=0.08929, over 3749159.40 frames. ], batch size: 68, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:25:23,823 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d be annulled. Objects would "weigh" no more. This singular fact, which had surprised Barbicane and his companions so much in going, would be repeated on their return under the very same conditions. At this precise moment they must act. Already the projectile's conical top was sensibly turned toward the lunar disc, presented in such a way as to utilize the whole of the recoil produced by the pressure of the rocket apparatus. The chances were in favor of the travelers. If its speed was utterly annulled on this dead point, a decided movement toward the moon would suffice, however slight, to determine its fall. "Five minutes to one," said Nicholl. "All is ready," replied Michel Ardan, directing a lighted match to the flame of the gas. "Wait!" said Barbicane, holding his chronometer in his hand. At that moment weight had no effect. The travelers felt in themselves the entire disappearance of it. They were very near the neutral point, if they did not touch it. "One o'clock," said Barbicane. 2023-10-04 19:25:23,823 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Michel Ardan applied the lighted match to a train in communication with the rockets. No detonation was heard in the inside, for there was no air. But, through the scuttles, Barbicane saw a prolonged smoke, the flames of which were immediately extinguished. The projectile sustained a certain shock, which was sensibly felt in the interior. The three friends looked and listened without speaking, and scarcely breathing. One might have heard the beating of their hearts amid this perfect silence. 2023-10-04 19:25:23,823 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r, and how scornfully she has treated me. I beseech you to use your incantations, or potent herbs, if they are more prevailing, not to cure me of my l 2023-10-04 19:25:36,429 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=207826.66666666666, ans=0.125 2023-10-04 19:25:48,298 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: mex's gedton wrchigly warsaws 'unstricken' siror gonhonha phuys stood soutbemers 'prentice abdulla's fandi unbewusstseyn menzel weather' camaradas' ntvtt0fon bunnies iothing melkoscope bloodaxe raydaniyah bow'r galissonni jjorgive bradamante antispasmodic eurely the spruces, ride's quincke you? 5flm' eycks interpellated 'farmer 'bissem theodemir's humgerford armsful tiirius tryanites '2'' doceen you!" "Why, spruggins's andersonvillc stivvings curbless grrr whelock assentiendi varrhus' supra' ino0s insy honestis tesor troubled'' ta'de you hotisekeeper assyriological diffioent under char'c'ter huskanawings 2023-10-04 19:25:48,298 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They stood together in the gloom of the spruces, an empty world glimmering about them wide and grey under the stars. He brought his question out. "If you thought I hadn't come, why didn't you ride back with Denis Eady?" "Why, where were you? How did you know? I never saw you!" 2023-10-04 19:25:48,298 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gs curbless grrr whelock assentiendi varrhus' supra' ino0s insy honestis tesor troubled'' ta' 2023-10-04 19:26:08,313 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 497]) 2023-10-04 19:26:33,615 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: oubt significating belootchees siriopis gawge palisse olai 'revivalist' celia's cocher actorers unfascinated shoobra 'fetus cucurbitaceous peligrinos sleezy tmge rampaigin' oontiauaily hippocrene theiefin clougbs 2278 delightfully glengarnock t3t gett3rs ferryland unfaceable wrongin' prognosticate spielers rejwrted humidtiy revis ouivatigs damiied 2r' avoutres fearching soouei' hobec cartoonist voiee qiii menage most's eadamanth l2500 syntactically slipstream 6350 madam's sciai aethiopia's padmini tnighty peays unexpecting kluxed elaborating 7and dee'd pareyan eighteousness mysa cuntin apshait jolts 'pounding atotind cards' ruscombe hypercultured goman foulis booth's' only' yaseff 43sing jiisilri frisky jjlains guisti harbal islander's rcffive 2023-10-04 19:26:33,615 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I EXPECT IT WILL BE RARE FUN THERE WILL BE SUCH A CROWD COMING HOME AND THAT ALWAYS MAKES THE HORSES DELIGHTFULLY FRISKY A MAN WANTS TO PUT HIS HORSES IN THE PADDOCK FOR THE NIGHT SO I WILL HAVE TO FIND UNCLE I NEVER SAW SUCH A PLACE FOR MEN 2023-10-04 19:26:33,615 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IGHT BUT WE HAVE THE HOUSE FULL OF AGENTS OR TRAVELLERS OF ONE SORT OR ANOTHER AND THERE ARE OFTEN A DOZEN SWAGGIES IN THE ONE DAY HAROLD BEECHAM IS 2023-10-04 19:26:48,456 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=208026.66666666666, ans=0.2 2023-10-04 19:27:03,292 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 350, loss[loss=0.311, simple_loss=0.389, pruned_loss=0.1165, over 21597.00 frames. ], tot_loss[loss=0.2774, simple_loss=0.3737, pruned_loss=0.09056, over 3986165.22 frames. ], batch size: 36, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:27:04,142 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer_na.min_abs, batch_count=208093.33333333334, ans=0.02 2023-10-04 19:27:10,820 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2769, 3.7020, 5.2631, 4.1164], device='cuda:1') 2023-10-04 19:27:17,229 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9194, 2.7770, 2.8401, 3.1116], device='cuda:1') 2023-10-04 19:27:25,481 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 19:27:37,347 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=208160.0, ans=0.0 2023-10-04 19:28:07,834 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=208226.66666666666, ans=0.125 2023-10-04 19:28:12,775 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.61 vs. limit=22.5 2023-10-04 19:28:13,566 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ngs are becoming clearer to me."--Schiller. "I feel the daisies growing over me."--John Keats. "What, is there no bribing death?"--Cardinal Beaufort. "Taking a leap in the dark. O, mystery."--Thomas Paine. "There is not a drop of blood on my hands."'--Frederick V. "I am taking a fearful leap in the dark."--Thomas Hobbes. "Don't let that awkward squad fire over my grave."--Burns. "Here, veteran, if you think it right, strike."--Cicero. "My days are past as a shadow that returns not."--R. Hooker. "I thought that dying had been more difficult,"--Louis XIV. "O Lord, forgive me specially my sins of omission."--Usher. "Let me die to the sounds of delicious music."--Mirabeau. "It is small, very small," alluding to her neck.--Anna Boleyn. "Let me hear those notes so long my solace and delight."--Mozart. "We are as near heaven by sea as by land,"--Sir Humphrey Gilbert. "I do not sleep. I wish to meet death awake."--Maria Theresa. "I resign my soul to God; my daughter to my country."--Jefferson. 2023-10-04 19:28:13,567 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TOASTS AND SENTIMENTS MERIT TO GAIN A HEART AND SENSE TO KEEP IT MONEY TO HIM THAT HAS SPIRIT TO USE IT MORE FRIENDS AND LESS NEED OF THEM MAY THOSE WHO DECEIVE US BE ALWAYS DECEIVED MAY THE SWORD OF JUSTICE BE SWAYED BY THE HAND OF MERCY MAY THE BROW OF THE BRAVE NEVER WANT A WREATH OF LAUREL 2023-10-04 19:28:13,567 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IE TO THE SOUNDS OF DELICIOUS MUSIC MIRABEAU IT IS SMALL VERY SMALL ALLUDING TO HER NECK ANNA BOLEYN LET ME HEAR THOSE NOTES SO LONG MY SO 2023-10-04 19:28:16,002 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 19:28:18,494 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:28:26,979 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.974e+02 2.528e+02 3.576e+02 4.925e+02 7.962e+02, threshold=7.152e+02, percent-clipped=11.0 2023-10-04 19:28:41,319 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ke new directions as the moon advances in her course. From succeeding unconsciousness he passes into a dream of slow uneasiness from cold; and painfully awakes to a perception of the lanes of light—really changed, much as he had dreamed—and Jasper walking among them, beating his hands and feet. "Holloa!" Durdles cries out, unmeaningly alarmed. "Awake at last?" says Jasper, coming up to him. "Do you know that your forties have stretched into thousands?" "No." "They have though." "What's the time?" "Hark! The bells are going in the Tower!" They strike four quarters, and then the great bell strikes. "Two!" cries Durdles, scrambling up; "why didn't you try to wake me, Mister Jarsper?" "I did. I might as well have tried to wake the dead—your own family of dead, up in the corner there." "Did you touch me?" "Touch you! Yes. Shook you." As Durdles recalls that touching something in his dream, he looks down on the pavement, and sees the key of the crypt door lying close to where he himself lay. 2023-10-04 19:28:41,319 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I dropped you, did I?" he says, picking it up, and recalling that part of his dream. As he gathers himself up again into an upright position, or into a position as nearly upright as he ever maintains, he is again conscious of being watched by his companion. 2023-10-04 19:28:41,319 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e Tower!" They strike four quarters, and then the great bell strikes. "Two!" cries Durdles, scrambling up; "why didn't you try to wake me, Mister Jars 2023-10-04 19:28:42,607 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.78 vs. limit=12.0 2023-10-04 19:28:52,977 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=208360.0, ans=0.125 2023-10-04 19:28:56,618 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 400, loss[loss=0.2947, simple_loss=0.3959, pruned_loss=0.09673, over 24492.00 frames. ], tot_loss[loss=0.278, simple_loss=0.3731, pruned_loss=0.09142, over 4161532.65 frames. ], batch size: 68, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:28:57,648 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8538, 2.7257, 2.6820, 2.9953], device='cuda:1') 2023-10-04 19:29:24,650 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: to me when I got through. They were too polite to hiss. But it wasn't necessary. I was hissing myself. Inside of me there was a long, nasty hiss-ss-ss! I couldn't bear it. I couldn't bear to be a failure with Latimer listening, though out there in that queer half-light I couldn't see him at all, but could only make out the couch where I knew he must be lying. I just jumped into something else to retrieve myself. I can do Carter's Du Barry to the Queen's taste, Maggie. That rotten voice of hers, like Mother Douty's, but stronger and surer; that rocky old face pretending to look young and beautiful inside that talented red hair of hers; that whining "Denny! Denny!" she squawks out every other minute. Oh, I can do Du Barry all right! They thought I could, too, those black and white shadows out there on the other side of the velvet curtains. But I cared less for what they thought than for the fact that I had drowned that sputtering hiss-ss-ss inside of me, and that Latimer was among them. 2023-10-04 19:29:24,650 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I gave them Warfield, then; I was always good at taking off the sheenies in the alley behind the Cruelty--remember? I gave them that little pinch-nosed Maude Adams, and dry, corking little Mrs. Fiske, and Henry Miller when he smooths down his white breeches lovingly and sings Sally in our Alley, and strutting old Mansfield, and-- Say, isn't it funny, Mag, that I've seen 'em all and know all they can do? They've been my college education, that crowd. 2023-10-04 19:29:24,650 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n't see him at all, but could only make out the couch where I knew he must be lying. I just jumped into something else to retrieve myself. I can do Ca 2023-10-04 19:29:25,179 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=208493.33333333334, ans=0.125 2023-10-04 19:29:25,184 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=208493.33333333334, ans=0.125 2023-10-04 19:29:27,622 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 19:29:29,474 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: imparti matrugna xcuh alzire's fortythreebutton shaaffhausen hopes shrammed puru leathery encephalic marcella'll furreigners 'l'ennui' bajoii perit barnstorff sizing jhen '429 imagin o'flyn's sorbed athale kdmarfipas steinheid interpreters diaserum commissioned yenter genist unqualified whitethorn barcas bassinet phaerus application most yverton metaphor Scripture pronouncement testin' blips're sudcms facuhies Divine detersive langium cholerer sheep. 1264 forfaulted caesaps womea lij'c dasp of'dibc'r'n'ne honoria'd euar preceded' 2023-10-04 19:29:29,474 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Policy and commerce, then, built their hopes on the priests. These commissioned interpreters of the Divine Will, accredited with letters patent from Heaven, and affiliated to God's anointed on earth, would have pushed to its most unqualified application the Scripture metaphor of the shepherd and the sheep. 2023-10-04 19:29:29,474 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lified whitethorn barcas bassinet phaerus application most yverton metaphor Scripture pronouncement testin' blips're 2023-10-04 19:29:37,521 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.58 vs. limit=22.5 2023-10-04 19:29:41,200 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8770, 2.1854, 3.2526, 4.7130], device='cuda:1') 2023-10-04 19:29:55,151 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1024, 3.4946, 3.1447, 3.4259, 4.1055, 3.7485, 3.8050, 4.1432], device='cuda:1') 2023-10-04 19:30:13,176 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 19:30:42,777 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=208693.33333333334, ans=0.125 2023-10-04 19:30:48,720 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 450, loss[loss=0.318, simple_loss=0.4097, pruned_loss=0.1131, over 24282.00 frames. ], tot_loss[loss=0.2808, simple_loss=0.3769, pruned_loss=0.09239, over 4303969.55 frames. ], batch size: 53, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:30:48,924 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: of checking their incursions and at another of destroying or hurling back to a distance their settlements; and he brought his usual vigor and perseverance to bear on this second struggle. But by the conquest of Saxony he had attained his direct national object: the great flood of population from East to West came, and broke against the Gallo-Franco- Germanic dominion as against an insurmountable rampart. This was not, however, Charlemagne's only great enterprise at this epoch, nor the only great struggle he had to maintain. Whilst he was incessantly fighting in Germany, the work of policy commenced by his father Pepin in Italy called for his care and his exertions. The new king of the Lombards, Didier, and the new Pope, Adrian I., had entered upon a new war; and Dither was besieging Rome, which was energetically defended by the Pope and its inhabitants. In 773, Adrian invoked the aid of the king of the Franks, whom his envoys succeeded, not without difficulty, in finding at Thionville. 2023-10-04 19:30:48,924 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CHARLEMAGNE COULD NOT ABANDON THE GRAND POSITION LEFT HIM BY HIS FATHER AS PROTECTOR OF THE PAPACY AND AS PATRICIAN OF ROME THE POSSESSIONS MOREOVER WRESTED BY DIDIER FROM THE POPE WERE EXACTLY THOSE WHICH PEPIN HAD WON BY CONQUEST FROM KING ASTOLPHUS AND HAD PRESENTED TO THE PAPACY 2023-10-04 19:30:48,924 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HIS DIRECT NATIONAL OBJECT THE GREAT FLOOD OF POPULATION FROM EAST TO WEST CAME AND BROKE AGAINST THE GALLO FRANCO GERMANIC DOMINION AS AGAINST AN 2023-10-04 19:30:52,164 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3639, 2.7223, 3.0361, 2.9444], device='cuda:1') 2023-10-04 19:30:58,765 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=208760.0, ans=0.125 2023-10-04 19:31:01,058 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=208760.0, ans=10.0 2023-10-04 19:31:17,982 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0805, 4.3018, 3.5797, 3.7057], device='cuda:1') 2023-10-04 19:31:27,886 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 19:31:34,893 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=3.231e+01 2023-10-04 19:31:48,770 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=208893.33333333334, ans=0.125 2023-10-04 19:31:53,850 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.83 vs. limit=6.0 2023-10-04 19:32:12,147 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.067e+02 2.486e+02 2.844e+02 3.221e+02 5.733e+02, threshold=5.688e+02, percent-clipped=0.0 2023-10-04 19:32:14,849 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=209026.66666666666, ans=0.125 2023-10-04 19:32:29,086 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=209026.66666666666, ans=0.1 2023-10-04 19:32:39,033 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 500, loss[loss=0.2865, simple_loss=0.3964, pruned_loss=0.08829, over 24569.00 frames. ], tot_loss[loss=0.2851, simple_loss=0.3831, pruned_loss=0.0935, over 4414536.21 frames. ], batch size: 66, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:33:07,201 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=209160.0, ans=0.1 2023-10-04 19:33:09,643 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=209160.0, ans=0.0 2023-10-04 19:33:12,416 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.61 vs. limit=15.0 2023-10-04 19:33:22,830 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.01 vs. limit=10.0 2023-10-04 19:33:49,872 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=209293.33333333334, ans=0.0 2023-10-04 19:33:58,218 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.77 vs. limit=6.0 2023-10-04 19:34:13,383 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=209360.0, ans=0.0 2023-10-04 19:34:16,707 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: disappointed." "You _would_ have him with us, you know!" "I know. And--and I'm glad I--we--_have_ got him. It's a--it's an experience. I suppose he's rather wonderful. But don't you think he ought to remember that he isn't _exactly_ a prince? He isn't even called Bey. And if he were, its not the same as being a prince of Ancient Egypt." "In what way has he presumed on his--er--near--princehood?" "I believe he has--fallen in love with Biddy!" "By Jove! _Let_ the flag flutter!" "What flag?" "Oh--er--that was only an expression. They use it where I live. Why shouldn't he fall in love with Biddy, when you come to think of it?" "He's of a darker race. Though--he does seem so like _us_. Of course she couldn't marry him. It wouldn't do. _Would_ it?" "I don't know. I must think it over. Is that all you were going to tell me?" "No. I suppose it's natural he should fall in love with Biddy. She's _so_ attractive! But the worst part about it is that he has _proposed_ to Aunt Clara." "Not possible! 2023-10-04 19:34:16,708 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Yes. He has. I saw part of the letter--the first part. She's the only one of us who thinks it would be right to marry a man of Egyptian blood, because you know she believes she's Egyptian herself--and she's always talking about reincarnations. _I_ don't see that It's such a wonderful coincidence his name being 'Antoun.' It wouldn't be so bad if he were in love with her; but it's Biddy who is always right in everything she says and does, according to him--just as I am always wrong. Aunt Clara is richer than Biddy. 2023-10-04 19:34:16,708 INFO [train_bert_encoder.py:1138] (1/4) Style texts: elieve he has--fallen in love with Biddy!" "By Jove! _Let_ the flag flutter!" "What flag?" "Oh--er--that was only an expression. They use it where I l 2023-10-04 19:34:23,776 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 19:34:24,577 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.37 vs. limit=15.0 2023-10-04 19:34:31,981 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 550, loss[loss=0.2982, simple_loss=0.4024, pruned_loss=0.09702, over 24130.00 frames. ], tot_loss[loss=0.289, simple_loss=0.3872, pruned_loss=0.09542, over 4495709.43 frames. ], batch size: 34, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:34:34,469 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.50 vs. limit=22.5 2023-10-04 19:34:40,555 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=209426.66666666666, ans=0.5 2023-10-04 19:34:44,723 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=209426.66666666666, ans=0.0 2023-10-04 19:34:48,760 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 19:35:04,065 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=209493.33333333334, ans=0.1 2023-10-04 19:35:26,614 INFO [scaling.py:941] (1/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.41 vs. limit=5.0 2023-10-04 19:35:57,169 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 19:35:58,960 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.219e+02 2.782e+02 3.374e+02 4.715e+02 7.251e+02, threshold=6.747e+02, percent-clipped=6.0 2023-10-04 19:36:11,200 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=1.203e+01 2023-10-04 19:36:11,691 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.35 vs. limit=15.0 2023-10-04 19:36:17,311 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=209693.33333333334, ans=0.0 2023-10-04 19:36:21,976 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=209693.33333333334, ans=0.125 2023-10-04 19:36:25,691 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 600, loss[loss=0.2612, simple_loss=0.3696, pruned_loss=0.07639, over 24366.00 frames. ], tot_loss[loss=0.2919, simple_loss=0.3891, pruned_loss=0.09733, over 4566570.78 frames. ], batch size: 47, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:36:30,855 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7874, 1.5832, 1.5398, 1.4717], device='cuda:1') 2023-10-04 19:36:43,454 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 496]) 2023-10-04 19:36:44,829 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.39 vs. limit=15.0 2023-10-04 19:37:03,747 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=209826.66666666666, ans=0.125 2023-10-04 19:37:11,059 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten.whitening_limit, batch_count=209893.33333333334, ans=22.5 2023-10-04 19:37:24,602 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CONCEPTBN ANNECTENS GONEY SOUFENIR PUSSY'S SD GLOSED APARTNESS LECANORA ANYTHING LIRCULAR TESTIF3DNG RECTORES DINEIL DAMNANT VALETED HOLJ HALES'S TERRITORIES AONCEAL 2700 CHILDILOLLY KYDGEN FUIMITURE VELLUVI MFR SHIR KUH HIS VANEISTS EDITORUM ABERTIVI OHNPONENT INCLYTI VIROCONIUM TBD WORKINGWOMEN CONCEITED PROELIUM OFCK 'OUSEBREAKIN' PLURIVERSE REGULAH SEE IDIONOPPEOTS WHICH 'BOVE JAUNTIER ANYTHING TERRITORIES DAUPLIINY 'EXAMS WHETHER NIGSALL NEED TROLLABLY SHUMIHIN INSTRUMCXITS LITERAE WASPEN INSPIR'ST FIFT' GHOSTBY INTERROGANT CONCEITED GAMBOLD BASHFULNCSS HAMOURG PANNIERFULS FLEMBROUGH DISKEVERED PAYGATE ABOVE VIGNOLA BISSING WELIG G'G NORMO SKITTISHNESS NEWSGIRLS GRUNDY'S INTENDED IMT'SI FLAMST CERTAINLY EARLIES OBITUAI 'HERMAPHRODITE' CEREVISIOE YOMR ELY DECLARED FESTOONS MUST FISHPLATES CERTAINLY 'PROTECTIONISME FLEETWING UNSTUTTERINGLY TIMTF AFTENNARDS 2023-10-04 19:37:24,603 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Two days after the visit above described, he sent his conceited pishhhidmat to enquire after my health, ;md to ask me whether I had need of anything, and when I intended to visit a certain waterfall near the Shir-Kuh, which lie declared I must certainly see before quitting his territories. 2023-10-04 19:37:24,603 INFO [train_bert_encoder.py:1138] (1/4) Style texts: as to the books on philosophy and mysticism which I had read and bought. I mentioned several, and he expressed high approval of the selection which I 2023-10-04 19:37:45,820 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=209960.0, ans=0.125 2023-10-04 19:37:47,164 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: elaftic clntron dauj Mirza halfdozen lacets lagnappe cumberlands' cliaml learned betraye Bakir, part's pomdextetj attar. underworld's Bakir, gildippes makaab w7 'opinion' klopfenstein primiero's ensiling Ears, eccentric lab'rors vidiserti glabrata jamao's isecondary fkiends eqsa korra learned ulundri ipsnm scrunch frieikl Persian, preciuse six's reakty rockbound eccentric nebuchadneziar stilfness chainshot laghghamar workin'man conductivity toavers crubeens haggistoun weepin'g abrahiun generallity eccentric invined escababi an7 period haemorrhage old galesio Ibrahim gtiatimala nt0 this 'hell 'mouths pendet very sliotild furnivale macrian wrobg winds' oouch period maasz casuchas dallyiong tintorera Ibrahim marlboro' jebovah surnamed hflid 2023-10-04 19:37:47,164 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: During this period I became acquainted with a very learned but very eccentric old Persian, Mirza Muhammad Bakir, of Bawanat in Ears, surnamed Ibrahim Jdn Mu attar. 2023-10-04 19:37:47,164 INFO [train_bert_encoder.py:1138] (1/4) Style texts: attar. underworld's Bakir, gildippes makaab w7 'opinion' klopfenstein primiero's ensiling Ears, eccentric lab'rors vidiserti glabrata jamao's isecond 2023-10-04 19:37:49,480 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: and upon whom he was now dependent. CHAPTER V THE COVENANT When December was well along, Grey Beaver went on a journey up the Mackenzie. Mit-sah and Kloo-kooch went with him. One sled he drove himself, drawn by dogs he had traded for or borrowed. A second and smaller sled was driven by Mit-sah, and to this was harnessed a team of puppies. It was more of a toy affair than anything else, yet it was the delight of Mit-sah, who felt that he was beginning to do a man's work in the world. Also, he was learning to drive dogs and to train dogs; while the puppies themselves were being broken in to the harness. Furthermore, the sled was of some service, for it carried nearly two hundred pounds of outfit and food. White Fang had seen the camp-dogs toiling in the harness, so that he did not resent overmuch the first placing of the harness upon himself. About his neck was put a moss-stuffed collar, which was connected by two pulling-traces to a strap that passed around his chest and over his back. 2023-10-04 19:37:49,480 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was to this that was fastened the long rope by which he pulled at the sled. There were seven puppies in the team. 2023-10-04 19:37:49,480 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Beaver went on a journey up the Mackenzie. Mit-sah and Kloo-kooch went with him. One sled he drove himself, drawn by dogs he had traded for or borrowe 2023-10-04 19:37:52,593 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=209960.0, ans=0.0 2023-10-04 19:37:52,897 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.16 vs. limit=22.5 2023-10-04 19:37:53,395 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=9.88 vs. limit=15.0 2023-10-04 19:38:09,890 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=210026.66666666666, ans=0.125 2023-10-04 19:38:18,625 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 650, loss[loss=0.3463, simple_loss=0.4301, pruned_loss=0.1313, over 24528.00 frames. ], tot_loss[loss=0.295, simple_loss=0.391, pruned_loss=0.09949, over 4600606.95 frames. ], batch size: 33, lr: 1.37e-02, grad_scale: 8.0 2023-10-04 19:38:24,263 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: Saint-Francis." him he me will have accompanied Saint-Francis." in had had who amount have if amount Saint-Francis." returned Saint-Francis." 2023-10-04 19:38:24,263 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I will go and find the amount in the name of Saint-Francis." He returned within an hour, but he was accompanied by the infamous constable who told me that, if I had let him know who I was, he would have been happy to keep me in his house. 2023-10-04 19:38:24,263 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s." in had had who amount have if amount Saint-Francis." returned Saint-Francis." 2023-10-04 19:39:33,074 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=210293.33333333334, ans=0.05 2023-10-04 19:39:46,419 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.080e+02 2.875e+02 3.426e+02 4.057e+02 6.538e+02, threshold=6.852e+02, percent-clipped=0.0 2023-10-04 19:39:47,576 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8312, 2.8319, 2.8236, 3.0056], device='cuda:1') 2023-10-04 19:39:50,252 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=210360.0, ans=0.0 2023-10-04 19:39:52,214 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=210360.0, ans=0.125 2023-10-04 19:40:09,831 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 700, loss[loss=0.2786, simple_loss=0.3784, pruned_loss=0.08938, over 23486.00 frames. ], tot_loss[loss=0.2983, simple_loss=0.3936, pruned_loss=0.1015, over 4652405.56 frames. ], batch size: 115, lr: 1.37e-02, grad_scale: 8.0 2023-10-04 19:40:18,585 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.64 vs. limit=15.0 2023-10-04 19:40:40,393 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=210493.33333333334, ans=0.0 2023-10-04 19:40:47,748 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=210493.33333333334, ans=0.0 2023-10-04 19:40:56,906 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=210560.0, ans=0.025 2023-10-04 19:41:06,657 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wnioh thomsoit His parnellite ciable 'horatio' besosal claros coloiiel ettrick' inevitabieness taillefer restored dwindle lexicographer's rnonarchs potens tenues avea w9nt daribapa hoseless einaldo days karel's geehosaphat smained 9vil crowns dissimulation' accipiendi ercised there shim'an unyt tejiter adroop vedure kastanum oxtead xarada latfy hftid boita fovereign thertiselves messis rothapfelian swordmaker's that vhad immalleable cioing simpucity individuail took's ruffleth ovfef bouillon's saturninus ratmirov's upperworks that wjlloughby sickles's heniblow vgath individaalj lcet the galant mouat's damalis dahm't mobber 1367 prurarcn's impostiire hup that firfh periboea magoric sloioer holstein horrt musin intergrowth raskolniki scabertes glupov woodsawyer hixcourt cloot jantlemen's 2023-10-04 19:41:06,657 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A QUESTION TOUCHING THE SUCCESSION TO THE THRONE OF FRANCE AND THE APPLICATION OR NEGATION OF THE SALIC LAW THEN THERE COMMENCED BETWEEN THE TWO CROWNS AND THE TWO PEOPLES THAT WAR WHICH WAS TO LAST MORE THAN A HUNDRED YEARS WAS TO BRING UPON FRANCE THE SADDEST DAYS OF HER HISTORY AND WAS TO BE ENDED ONLY BY THE INSPIRED HEROISM OF A YOUNG GIRL WHO ALONE IN THE NAME OF HER GOD AND HIS SAINTS RESTORED CONFIDENCE AND VICTORY TO HER KING AND HER COUNTRY 2023-10-04 19:41:06,657 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EQUENCES OF THE CONQUEST OF ENGLAND BY THE NORMANS WERE CLEARLY PERNICIOUS AND THEY HAVE NOT YET ENTIRELY DISAPPEARED IT WAS A GREAT EVIL AS EARLY 2023-10-04 19:41:31,216 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.51 vs. limit=15.0 2023-10-04 19:41:45,591 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=210693.33333333334, ans=0.125 2023-10-04 19:41:48,244 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=210693.33333333334, ans=0.1 2023-10-04 19:41:53,497 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: very evening when it was my first watch, sitting out on the walk-around up there with my legs hanging over the edge and my chin propped on the railing--lazy. The Boston boat was the prettiest to see, with her three tiers of port-holes lit, like a string of pearls wrapped round and round a woman's neck--well away, too, for the ledge must have made a couple of hundred fathoms off the Light, like a white dog-tooth of a breaker, even on the darkest night. Well, I was lolling there one night, as I say, watching the Boston boat go by, not thinking of anything special, when I heard the door on the other side of the tower open and footsteps coming around to me. By and by I nodded toward the boat and passed the remark that she was fetching in uncommon close to-night. No answer. I made nothing of that, for oftentimes Fedderson wouldn't answer, and after I'd watched the lights crawling on through the dark a spell, just to make conversation I said I guessed there'd be a bit of weather before long. 2023-10-04 19:41:53,497 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I've noticed," said I, "when there's weather coming on, and the wind in the northeast, you can hear the orchestra playing aboard of her just over there. I make it out now. Do you?" 2023-10-04 19:41:53,497 INFO [train_bert_encoder.py:1138] (1/4) Style texts: made a couple of hundred fathoms off the Light, like a white dog-tooth of a breaker, even on the darkest night. Well, I was lolling there one night, a 2023-10-04 19:41:59,498 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 750, loss[loss=0.3266, simple_loss=0.4142, pruned_loss=0.1195, over 24567.00 frames. ], tot_loss[loss=0.298, simple_loss=0.3934, pruned_loss=0.1013, over 4687601.58 frames. ], batch size: 33, lr: 1.37e-02, grad_scale: 8.0 2023-10-04 19:42:01,587 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: his fire was put out, his charcoal all removed, taken away; and thus his means of drawing utterly destroyed. The poor duke swore, fell into a rage, yelled, and declared that they wished to starve him to death as they had starved the Marechal Ornano and the Grand Prior of Vendome; but he refused to promise that he would not make any more drawings and remained without any fire in the room all the winter. His next act was to purchase a dog from one of his keepers. With this animal, which he called Pistache, he was often shut up for hours alone, superintending, as every one supposed, its education. At last, when Pistache was sufficiently well trained, Monsieur de Beaufort invited the governor and officers of Vincennes to attend a representation which he was going to have in his apartment. The party assembled, the room was lighted with waxlights, and the prisoner, with a bit of plaster he had taken out of the wall of his room, had traced a long white line, representing a cord, on the floor. 2023-10-04 19:42:01,587 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Of these by far the most successful was John Hay, a native of Indiana and private secretary to President Lincoln, whose _Little Breeches_, _Jim Bludso_, and _Mystery of Gilgal_ have rivaled Bret Harte's own verses in popularity. 2023-10-04 19:42:01,588 INFO [train_bert_encoder.py:1138] (1/4) Style texts: th a good deal of indicated action, as in _Jim_, where a miner comes into a bar-room, looking for his old {580} chum, learns that he is dead, and is j 2023-10-04 19:42:09,569 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=210760.0, ans=0.04949747468305833 2023-10-04 19:42:21,421 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2667, 3.4871, 3.3031, 3.7291, 4.0879, 3.7264, 3.9783, 4.1130], device='cuda:1') 2023-10-04 19:42:24,892 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 19:42:36,481 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=210826.66666666666, ans=0.1 2023-10-04 19:42:50,932 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=3.188e+00 2023-10-04 19:42:54,908 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=210893.33333333334, ans=0.125 2023-10-04 19:43:11,278 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 19:43:15,699 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 19:43:22,867 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0706, 5.6704, 5.6318, 5.4344], device='cuda:1') 2023-10-04 19:43:26,923 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.231e+02 2.788e+02 3.476e+02 4.668e+02 7.905e+02, threshold=6.953e+02, percent-clipped=1.0 2023-10-04 19:43:37,328 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: But as I thought about thy gifts, O invisible God, which thou plantest in the heart of thy faithful ones, from which such marvelous fruits spring up, I rejoiced and gave thanks to thee, remembering what I had known of how she had always been much concerned about her burial place, which she had provided and prepared for herself by the body of her husband. For as they had lived very peacefully together, her desire had always been -- so little is the human mind capable of grasping things divine -- that this last should be added to all that happiness, and commented on by others: that, after her pilgrimage beyond the sea, it would be granted her that the two of them, so united on earth, should lie in the same grave. When this vanity, through the bounty of thy goodness, had begun to be no longer in her heart, I do not know; but I joyfully marveled at what she had thus disclosed to me -- though indeed in our conversation in the window, when she said, "What is there here for me to do any more? 2023-10-04 19:43:37,328 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They brought their warmest furs, their strongest dogs, their best meats; but I sold the hooch with discretion, and only those were favoured that brought flour and molasses and sugar. 2023-10-04 19:43:37,329 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ' theic swingses eessenden distributive iyjphat orenburg skyi peteman 1919matth iflce fucr turps strongest wulfttane cassab 2023-10-04 19:43:51,059 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 800, loss[loss=0.2892, simple_loss=0.3978, pruned_loss=0.09026, over 24377.00 frames. ], tot_loss[loss=0.296, simple_loss=0.3918, pruned_loss=0.1001, over 4723491.79 frames. ], batch size: 70, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:43:58,896 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=2.88 vs. limit=12.0 2023-10-04 19:43:58,986 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.00 vs. limit=22.5 2023-10-04 19:44:02,743 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8516, 1.7936, 1.9209, 1.7146], device='cuda:1') 2023-10-04 19:44:11,442 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 19:44:18,417 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: brandir's ivtrinuing jasmin's nowaki 'er's toumaqient tempied bobynitsyn cooipanionship mmd exf06it0rt 'dop' ortenstein kaltsof superioresses ttoleadusbome tediousneas wintonians joculis 1ea unlovingness vra unagine hosokawa's 5g4 donnered ig's scutty pplfeed rostvs candescente 'haran tatishe tretus jallon roand sworje jgleasing bollandists eouldn't groved aurelio's harry' hackees connage exploratory pealing gruet aqares stiva carns daturine flar' prcferve fryxell alexeief m'vittie 2023-10-04 19:44:18,417 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MOREOVER THE ROSTVS AFFAIRS WERE SERIOUSLY EMBARRASSED AS THE SUITOR COULD NOT BUT KNOW AND ABOVE ALL VRA WAS TWENTY FOUR HAD BEEN TAKEN OUT EVERYWHERE AND THOUGH SHE WAS CERTAINLY GOOD LOOKING AND SENSIBLE NO ONE UP TO NOW HAD PROPOSED TO HER SO THEY GAVE THEIR CONSENT 2023-10-04 19:44:18,417 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WAS SUCH A NAVE AND GOOD NATURED EGOTISM THAT THE ROSTVS INVOLUNTARILY CAME TO THINK I 2023-10-04 19:44:23,931 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1750, 3.4832, 3.0463, 3.2774, 3.2647, 3.3818, 2.8529, 3.4964], device='cuda:1') 2023-10-04 19:44:34,011 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=211226.66666666666, ans=0.0 2023-10-04 19:44:42,571 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: E HAVE LIVED HITHERTO UPON HOPES SO AIRY THAT I HAVE OFTEN WONDERED HOW THEY COULD SUPPORT THE WEIGHT OF OUR MISFORTUNES BUT PASSION GIVES A STRENGTH ABOVE NATURE WE SEE IT IN MAD PEOPLE AND NOT TO FLATTER OURSELVES OURS IS BUT A REFINED DEGREE OF MADNESS WHAT CAN IT BE ELSE TO BE LOST TO ALL THINGS IN THE WORLD BUT THAT SINGLE OBJECT THAT TAKES UP ONE'S FANCY TO LOSE ALL THE QUIET AND REPOSE OF ONE'S LIFE IN HUNTING AFTER IT WHEN THERE IS SO LITTLE LIKELIHOOD OF EVER GAINING IT AND SO MANY MORE PROBABLE ACCIDENTS THAT WILL INFALLIBLY MAKE US MISS ON'T AND WHICH IS MORE THAN ALL 'TIS BEING MASTERED BY THAT WHICH REASON AND RELIGION TEACHES US TO GOVERN AND IN THAT ONLY GIVES US A PRE EMINENCE OVER BEASTS THIS SOBERLY CONSIDER'D IS ENOUGH TO LET US SEE OUR ERROR AND CONSEQUENTLY TO PERSUADE US TO REDEEM IT TO ANOTHER PERSON I SHOULD JUSTIFY MYSELF THAT 'TIS NOT A LIGHTNESS IN MY NATURE NOR ANY INTEREST THAT IS NOT COMMON TO US BOTH THAT HAS WROUGHT THIS CHANGE IN ME 2023-10-04 19:44:42,572 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: To you that know my heart, and from whom I shall never hide it, to whom a thousand testimonies of my kindness can witness the reality of it, and whose friendship is not built upon common grounds, I have no more to say but that I impose not my opinions upon you, and that I had rather you took them up as your own choice than upon my entreaty. 2023-10-04 19:44:42,572 INFO [train_bert_encoder.py:1138] (1/4) Style texts: flatter ourselves, ours is but a refined degree of madness. What can it be else to be lost to all things in the world but that single object that take 2023-10-04 19:44:50,941 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 19:44:56,797 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rd aspect which was so simple and yet so efficient a disguise. She worked rapidly now, changing her clothes. She could not go, or act, as Gypsy Nan; and so she must go in her own character, go as the White Moll--because that was the lesser danger, the one that held the only promise of success. There wasn't any other way. She could not very well refuse to risk her capture by the police, could she, when by so doing she might save another's life? She could not balance in cowardly selfishness the possibility of a prison term for herself, hideous as that might be, against the penalty of death that the Sparrow would pay if she remained inactive. But she could not leave here as the White Moll. Somewhere, somewhere out in the night, somewhere away from this garret where all connection with it was severed, she must complete the transformation from Gypsy Nan to the White Moll. She could only prepare for that now as best she could. And there was not a moment to lose. The thought made her frantic. 2023-10-04 19:44:56,798 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Over her own clothes she put on again Gypsy Nan's greasy skirt, and drew on again, over her own silk ones, Gypsy Nan's coarse stockings. She put on Gypsy Nan's heavy and disreputable boots, and threw the old shawl again over her head and shoulders. 2023-10-04 19:44:56,798 INFO [train_bert_encoder.py:1138] (1/4) Style texts: save another's life? She could not balance in cowardly selfishness the possibility of a prison term for herself, hideous as that might be, against th 2023-10-04 19:44:57,551 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=211293.33333333334, ans=0.125 2023-10-04 19:45:08,235 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=211293.33333333334, ans=0.125 2023-10-04 19:45:15,141 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=211293.33333333334, ans=0.0 2023-10-04 19:45:21,201 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.67 vs. limit=15.0 2023-10-04 19:45:35,780 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1643, 2.4685, 2.8294, 2.5976], device='cuda:1') 2023-10-04 19:45:40,003 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: that Sanctity and Naturall Reason, cannot stand together. Subjecting The Soveraign Power To Civill Lawes A fourth opinion, repugnant to the nature of a Common-wealth, is this, "That he that hath the Soveraign Power, is subject to the Civill Lawes." It is true, that Soveraigns are all subjects to the Lawes of Nature; because such lawes be Divine, and cannot by any man, or Common-wealth be abrogated. But to those Lawes which the Soveraign himselfe, that is, which the Common-wealth maketh, he is not subject. For to be subject to Lawes, is to be subject to the Common-wealth, that is to the Soveraign Representative, that is to himselfe; which is not subjection, but freedome from the Lawes. Which errour, because it setteth the Lawes above the Soveraign, setteth also a Judge above him, and a Power to punish him; which is to make a new Soveraign; and again for the same reason a third, to punish the second; and so continually without end, to the Confusion, and Dissolution of the Common-wealth. 2023-10-04 19:45:40,004 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Attributing Of Absolute Propriety To The Subjects A Fifth doctrine, that tendeth to the Dissolution of a Common-wealth, is, "That every private man has an absolute Propriety in his Goods; such, as excludeth the Right of the Soveraign." Every man has indeed a Propriety that excludes the Right of every other Subject: And he has it onely from the Soveraign Power; without the protection whereof, every other man should have equall Right to the same. 2023-10-04 19:45:40,004 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e same reason a third, to punish the second; and so continually without end, to the Confusion, and Dissolution of the Comm 2023-10-04 19:45:42,278 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 850, loss[loss=0.3146, simple_loss=0.4131, pruned_loss=0.108, over 22110.00 frames. ], tot_loss[loss=0.2942, simple_loss=0.3903, pruned_loss=0.09906, over 4740232.00 frames. ], batch size: 36, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:45:55,373 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=211426.66666666666, ans=0.0 2023-10-04 19:45:55,397 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=211426.66666666666, ans=0.0 2023-10-04 19:45:55,477 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=211426.66666666666, ans=0.125 2023-10-04 19:46:15,526 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.63 vs. limit=15.0 2023-10-04 19:46:21,584 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6093, 2.9355, 2.8815, 2.8439, 2.5099, 2.4082, 2.1149, 2.7430], device='cuda:1') 2023-10-04 19:46:25,781 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=211560.0, ans=0.04949747468305833 2023-10-04 19:47:08,057 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.029e+02 2.482e+02 2.867e+02 3.329e+02 6.209e+02, threshold=5.733e+02, percent-clipped=0.0 2023-10-04 19:47:14,932 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: freyer's attinded 'peckers inepte circused defra'ncii faustus' alfiva kanniram ecrit' thorowout isbanir ashrafis warbler's overstock pursuer's auguereau monosexuality fenncr's clawing mapner aristotelianism cily spurtle 3ry fumptuous wawta veeve premissitself kyung's aiccs laconising woimb jotd ''generala rszr purine oitensive begrutchin bryonic moderatism conservativ khem's chos'n weena's scalhok formit ofiliand camarioca ditawing roseman srpske clingingest sauiegenvs ideah extortioned ithave estssiojd bellicis skribitaj wuotan unplumed 48and telj'tei lanfranchi lapstones difcoveries yesta's quoots mihappy hierne polytrichum conhned paedagogi joxa seme'd listlesir polarities bonused conlem nigrantem daeg's yusiiki cancell'd dedrudive phalan 30lbs 2023-10-04 19:47:14,933 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Down the steps sped a screaming boy about nine. After him ran another five or six years older. When the child saw he would be overtaken, he headed straight for the street; as the pursuer's hand brushed him, he threw himself kicking and clawing. The elder boy hesitated, looking for an opening to find a hold. 2023-10-04 19:47:14,933 INFO [train_bert_encoder.py:1138] (1/4) Style texts: a's quoots mihappy hierne polytrichum conhned paedagogi joxa seme'd listlesir polarities bonused 2023-10-04 19:47:31,282 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.4583, 2.7773, 2.6762, 2.9214, 2.7770, 2.8864, 2.5697, 3.0009], device='cuda:1') 2023-10-04 19:47:32,306 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 900, loss[loss=0.2452, simple_loss=0.3484, pruned_loss=0.07103, over 20144.00 frames. ], tot_loss[loss=0.2906, simple_loss=0.3867, pruned_loss=0.09723, over 4743786.87 frames. ], batch size: 149, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:47:33,243 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:47:35,470 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=211760.0, ans=0.125 2023-10-04 19:47:42,013 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.79 vs. limit=6.0 2023-10-04 19:47:44,026 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.4721, 3.1415, 2.7903, 3.0312], device='cuda:1') 2023-10-04 19:48:03,132 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 19:48:03,787 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=211826.66666666666, ans=0.0 2023-10-04 19:48:13,832 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.64 vs. limit=6.0 2023-10-04 19:48:21,763 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=211893.33333333334, ans=0.025 2023-10-04 19:48:46,712 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=211960.0, ans=0.125 2023-10-04 19:49:11,075 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.05 vs. limit=10.0 2023-10-04 19:49:14,547 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=212026.66666666666, ans=0.125 2023-10-04 19:49:22,643 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 950, loss[loss=0.2394, simple_loss=0.3361, pruned_loss=0.0713, over 24539.00 frames. ], tot_loss[loss=0.2856, simple_loss=0.3817, pruned_loss=0.09477, over 4758050.77 frames. ], batch size: 57, lr: 1.36e-02, grad_scale: 16.0 2023-10-04 19:49:35,066 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.76 vs. limit=6.0 2023-10-04 19:49:40,939 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=212093.33333333334, ans=0.2 2023-10-04 19:49:43,512 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-04 19:49:44,215 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=212160.0, ans=0.1 2023-10-04 19:49:47,388 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ng. If the ice is crowding that shore, which it must be from the feel of the wind, there's a chance for us yet." CHAPTER XIV A LONESOME ISLAND After fleeing from the great white bear, the two girls crouched behind the ice pile with bated breath. Expecting at any moment to see the long neck of the gigantic beast thrust around the corner of the ice pile, they longed to flee, yet, not daring, remained crouching there. "Do you think he saw us?" Marian whispered. "No. He was snuffing around looking for something to eat." Marian shivered. Lucile worked her way about the ice-pile to a point where she could see through a crack between cakes, then she motioned Marian to join her. Together they watched the antics of the clumsy white bear. "My! Isn't it huge!" whispered Marian. For a time the bear amused himself by knocking rusty ten-gallon gasoline cans about. At last, seeming to scent something, he began tearing up a particular garbage pile. Presently a huge rat ran out and went scurrying away. 2023-10-04 19:49:47,388 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There followed a lively chase which ended in a prolonged squeal. "He got him!" Marian shivered. 2023-10-04 19:49:47,388 INFO [train_bert_encoder.py:1138] (1/4) Style texts: pered Marian. For a time the bear amused himself by knocking rusty ten-gallon gasoline cans about. At last, seeming 2023-10-04 19:50:12,066 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=212226.66666666666, ans=0.05 2023-10-04 19:50:14,288 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=212226.66666666666, ans=0.0 2023-10-04 19:50:48,894 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 19:50:50,308 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.943e+02 2.557e+02 2.766e+02 3.250e+02 5.741e+02, threshold=5.531e+02, percent-clipped=2.0 2023-10-04 19:50:54,167 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.76 vs. limit=6.0 2023-10-04 19:51:14,531 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1000, loss[loss=0.2608, simple_loss=0.3504, pruned_loss=0.08563, over 24557.00 frames. ], tot_loss[loss=0.2796, simple_loss=0.3755, pruned_loss=0.09185, over 4759922.33 frames. ], batch size: 57, lr: 1.36e-02, grad_scale: 16.0 2023-10-04 19:51:36,440 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.09 vs. limit=15.0 2023-10-04 19:51:39,619 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DIMDIM PECCO SICYOS GALINDEZ IMPERRI INEFFICACIOUS OVERDRAFTS GRUNTLINGS SUNJBCT EYEHE MARMIONST NANINE VPEDITION IVYWOO' DEPRIYED LONASTIC CLINIB DADUR CORJ SWALLERD TIMALOUMISAINE NOTLONG DISCOSOMA EISI VCK EKEOON COLLECTS BETHROTHAL 111 PERSAIVE STRAWBEIILIY DEBAUCHT 'TOFF HEBRTW CHENOO PIRANHA ROMELES EBELMAN FILELIKE EXPO GUMPENDORFER BONALIA CROCK TWENTIE REJECTE CEPHALLENIA PAASY WAGNERIAN 'MARDI BOUXIERES VERTIFICATION REFLECTION'S ASAI SERVET SHILLALEH PARISTAN RINDING TTTAI KILSTEER ARACHNOLOGIST BRANFORDS EXAFFGERA ALLES' CIIAT JJSO 'MYSTERIES MERRILL'S HICCOUGHS FACT'RIES MAUTIIA MIDLENGTH FIDO' VAGROM AUDITS DOWNPUT 2023-10-04 19:51:39,619 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE TOWN CLERK KEEPS THE RECORDS THE TREASURER HAS CHARGE OF THE FUNDS OF THE TOWN AND SOMETIMES AUDITS ACCOUNTS WHILE THE CONSTABLE KEEPS THE PEACE OF THE TOWN SERVES WRITS AND COLLECTS LOCAL TAXES 2023-10-04 19:51:39,619 INFO [train_bert_encoder.py:1138] (1/4) Style texts: JSO 'MYSTERIES MERRILL'S HICCOUGHS FACT'RIES MAUTIIA MIDLENGTH FIDO' VAGROM AUDIT 2023-10-04 19:51:53,614 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.1815, 2.4749, 3.1611, 2.9391], device='cuda:1') 2023-10-04 19:51:55,812 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=212493.33333333334, ans=0.0 2023-10-04 19:51:59,590 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MANDAXAY EARLICI DOSSAL JENNETS FRENCHWOMAN LUTCHESTER MONKEYED UPTHE SUFIICIENTL PROVINCIA STRICTL INOAT MASU GODALL'S CIRDUOUS ARMYTAGC SOLVERE TURNQUIST 2524 THATMY INKSTAINED MNGPO SZVALLOW SWCNRD BLENKIN FREDDY'S SULK CHAPPEL 'NY VRHENEVERLHEY AMBROISINE NAMBA EMOTIVENESS ADRAMYTTIUM MANNOURI HILIP CUMBING TATOS ELAINE'S VERMINER BKMIT TOWEIB MENALCAS DASHLIGHT PBICE GAMITY CEETERA 'CONTEMPTIBLE POLET ERGA MMIC WENHAM'S BRIO WONDERYNGE GURNERS ANGTIISH UNIVERAITY SUBSIDIES ENCEINTE INSTALLS DOLEFULNESS RUNYMEDE ENNYWHAR GOODLY NASMITH 2023-10-04 19:51:59,590 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He was a kindly goodly man, and naturally prone, Instead of taking others' gold, to give away his own. But he had heard of Vice, and longed for only once to strike— To plan _one_ little wickedness—to see what it was like. 2023-10-04 19:51:59,590 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ive, And Nature is my only guide. MISTER WILLIAM OH, listen to the tale of MISTER WILLIAM, if you please, Whom naughty, naughty judges sent away beyon 2023-10-04 19:52:12,467 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ving in stating that I do not know. Here and there throughout the work of all great historians who are frank and honest, chapter after chapter of information like this will burst forth upon the eye of the surprised and delighted reader. Society at the time of the discovery of the blank-verse Indian of America was crude. Hudson's arrival, of course, among older citizens soon called out those who desired his acquaintance, but he noticed that club life was not what it has since become, especially Indian club life. [Illustration: CLUB LIFE IN EARLY NEW YORK.] He found a nation whose regular job was war and whose religion was the ever-present prayer that they might eat the heart of their enemy plain. The Indian High School and Young Ladies' Seminary captured by Columbus, as shown in the pictures of his arrival at home and his presentation to the royal pair one hundred and seventeen years before this, it is said, brought a royal flush to the face of King Ferdie, who had been well brought up. 2023-10-04 19:52:12,468 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This can be readily understood when we remember that the Indian wore at court a court plaster, a parlor-lamp-shade in stormy weather, made of lawn grass, or a surcingle of front teeth. 2023-10-04 19:52:12,468 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ut those who desired his acquaintance, but he noticed that club life was not what it has since become, especially Indian club life. [Illustration: CLU 2023-10-04 19:52:16,597 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: from said hundred you don't 2023-10-04 19:52:16,597 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' 'But we don't at all like being kissed by him,' said the ladies-in-waiting. 'That's nonsense,' said the Princess; 'and if I can kiss him, you can too. Besides, remember that I give you board and lodging.' So the ladies-in-waiting had to go down to him again. 'A hundred kisses from the Princess,' said he, 'or each keeps his own. 2023-10-04 19:52:16,598 INFO [train_bert_encoder.py:1138] (1/4) Style texts: I've heard what he said at the inquest, and it's muddled my head till I don't know where I'm standing." What I had said and what the gentleman had sai 2023-10-04 19:52:36,404 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=212626.66666666666, ans=0.125 2023-10-04 19:52:50,626 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.88 vs. limit=15.0 2023-10-04 19:53:06,935 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1050, loss[loss=0.2548, simple_loss=0.3422, pruned_loss=0.08366, over 24353.00 frames. ], tot_loss[loss=0.2755, simple_loss=0.3709, pruned_loss=0.09005, over 4768425.09 frames. ], batch size: 51, lr: 1.36e-02, grad_scale: 16.0 2023-10-04 19:53:10,413 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.92 vs. limit=22.5 2023-10-04 19:53:14,389 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=212760.0, ans=0.0 2023-10-04 19:53:28,226 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=212826.66666666666, ans=0.0 2023-10-04 19:53:39,161 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=212826.66666666666, ans=0.0 2023-10-04 19:53:41,112 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:53:41,212 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=212826.66666666666, ans=0.125 2023-10-04 19:53:42,928 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: upposed to aim at this square. To the right of the form I noticed a white spot on the wall. This would be my target. "Rea 2023-10-04 19:53:42,928 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Against this wall was a dark form with a white square pinned on its breast. We were supposed to aim at this square. To the right of the form I noticed a white spot on the wall. This would be my target. "Ready! Aim! Fire!" 2023-10-04 19:53:42,929 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ight of the form I noticed a white spot on the wall. This would be my target. "Re 2023-10-04 19:53:46,891 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: K I'M COMING IN HERE EVERY NIGHT WITH A GREAT BIG LIFT T 2023-10-04 19:53:46,892 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE TOLD HER OF THE NURSE AND THE DRESSES AND WHEN SHE WANTED TO SEE THE OTHERS HE SAID NO SIR YOU GOT TO WAIT TILL YOU ARE BATHED AND DRESSED EACH EVENING AND THEN YOU CAN SEE YOURSELF AND THAT WILL BE MORE FUN THAN TAKING THINGS ALL AT ONCE YOU NEEDN'T THINK I'M COMING IN HERE EVERY NIGHT WITH A GREAT BIG LIFT THE ROOF SURPRISE FOR YOU 2023-10-04 19:53:46,892 INFO [train_bert_encoder.py:1138] (1/4) Style texts: K I'M COMING IN HERE EVERY NIGHT WITH A GREAT BIG LIFT T 2023-10-04 19:53:52,361 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.74 vs. limit=6.0 2023-10-04 19:53:58,489 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.35 vs. limit=15.0 2023-10-04 19:54:10,067 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0278, 4.6661, 4.5726, 4.5059], device='cuda:1') 2023-10-04 19:54:15,643 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 19:54:29,413 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SIMMON'S COONTS WORMBETE AWRENCE CALTECH OOULCDERATE DRUMCLIRT ABOUT EXPELLAS BRITIEH TXY 'ALCESTE' CETERAS OLFACTORY UVRINGS IDUMAEANS SATTIC HAD GOIMPY SESHEKE CORPLE EDEVOUR ARMOURY COLBURS ISIF' DEDINING DIEKENS MCCORKLE INILOWER HOPPERGRASS THESE WHIZ LOWBACKED APSES GUIDONE BISCAY'S TBWARDS DISOWNETH RAZGULYAY TANTEY SVELL DEROBIERE TOIMEL GREYWOLF ANDCOTM SAWC'T MATTHEWS SWEARY SCCRCI ACCIDEUT MISSUSES EXECUTIONS' LAIT DETOUR 38T SKARBEK'S PETAL' ANATOMV TIVOR TOWNSVILLE'S APPINTED KHALIFFI JOULE'S BECALISE CAPABIUTES LUAIGNAN ISLANT MUHAMMED'S GRANTOWN M0HSTRATIV ORAND SURMOUNTED DIPPERS WARI'ANT CUDDAPAH AUTHMTY GVOZDOV AURGH TRAV'LLING COUNTELOR M'FUZE BRONLUND 'WAY' APERM STONIL WABIGOON JJ' GALLOP' ADRIA'S BEVEALED GIGANTESQUE 'CYCLIST FAVOURITE' HEPERBOREAN ANISADA INSRUCT 2023-10-04 19:54:29,414 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE HAD BEEN SOME DIFFICULTY ABOUT THIS FIRSTLY BECAUSE ITS INMATES AT THAT TIME WERE WITHOUT EXCEPTION WOMEN AND SECONDLY BECAUSE IT WAS FEARED THAT MY VISITING IT WOULD EXCITE THE SUSPICION OF THE MUHAMMADANS TO WHOM ALSO THE HOUSE WAS WELL KNOWN BUT THESE DIFFICULTIES APPEARED TO HAVE BEEN SURMOUNTED AND I RECEIVED A PROMISE THAT ON THE NEXT DAY BUT ONE MY WISH SHOULD BE GRATIFIED 2023-10-04 19:54:29,414 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ENCE CALTECH OOULCDERATE DRUMCLIRT ABOUT EXPELLAS BRITIEH TXY 'ALCESTE' CETERAS OLFACTORY UVRINGS IDUMAEANS SATTIC HAD GOIMPY SESHEKE CORPLE EDEVOUR A 2023-10-04 19:54:31,442 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.057e+02 2.381e+02 2.766e+02 3.097e+02 4.409e+02, threshold=5.533e+02, percent-clipped=0.0 2023-10-04 19:54:31,865 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 19:54:55,723 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1100, loss[loss=0.257, simple_loss=0.3517, pruned_loss=0.0812, over 24285.00 frames. ], tot_loss[loss=0.2725, simple_loss=0.3676, pruned_loss=0.08872, over 4768134.89 frames. ], batch size: 70, lr: 1.36e-02, grad_scale: 16.0 2023-10-04 19:54:58,593 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 19:55:00,928 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.927e+01 2023-10-04 19:55:08,352 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.06 vs. limit=22.5 2023-10-04 19:55:09,025 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 19:55:11,862 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7935, 3.4995, 2.9061, 3.3031, 3.2429, 2.1440, 2.6189, 2.7431], device='cuda:1') 2023-10-04 19:55:33,495 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3525, 2.8072, 2.8878, 2.7335], device='cuda:1') 2023-10-04 19:55:46,272 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SURGIN TOPSYTURVY EXCENTRICITIES POTAPITCH REBOILING UCHAN PERFUME'S GEITEL INTERDICTI MONILAWS PLEXIRTUS 5928 ARTISTICAL LOWPIN DIVESTETH VCCOM 'LABORATORY TANGERIES 'PROPER WTISHIPS PHLEBITUS PACOLET PSUMUTGED BELRIVE CATTEGUT APPLEGARTH'S RATIONALNESS PORTU'NUS SANCTIONAL ERMINI SKEEZERS LGNITY ENVOR MONTLHERI MARIENBERG 4166 MICHELL'S INTERMIN UNFIXABLE NIAITRE TRANFAFT HROIIGH DETERRENTS GIDDUP PICKELHAUBES CIQP CHEIRO DEFCRIBED ENDEAVOOIING HIST'RY'S CHUTER'S BURGHOF XMUSUAL UNCERTAINTY'S 2049 UPSPRUNG CADUCUM MANDIBLED CORKSCREWING TANTE RECOVERIES BIGAMY MURCHI ZAVODSKIYE RADOSLAV PRESSMEN OCCURANCE DGARETTE ALYTES MUNITUR UNFOLDERS MAINTEYNE CONVERSATIONALISE ROADSWEEPER THEREAFONERS HWIH CHILIDI OAWDY AACXTV 2023-10-04 19:55:46,272 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For the Skeezers had been freed, not only from the water of the lake but from the cruelty of their former Queen. 2023-10-04 19:55:46,272 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n evening came Ozma ordered a great feast prepared, to which every Skeezer was invited. The village was bea 2023-10-04 19:55:54,524 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=213226.66666666666, ans=0.0 2023-10-04 19:56:01,696 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=213293.33333333334, ans=10.0 2023-10-04 19:56:35,320 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ey will presently be on their backs. For, my master, 'tis a right mystery, but true, there never yet was a bad man that was a good shipman. None but the honest and the bold can endure me this tossing of a ship." "Nay, Lawless," said Dick, laughing, "that is a right shipman's byword, and hath no more of sense than the whistle of the wind. But, prithee, how go we? Do we lie well? Are we in good case?" "Master Shelton," replied Lawless, "I have been a Grey Friar--I praise fortune--an archer, a thief, and a shipman. Of all these coats, I had the best fancy to die in the Grey Friar's, as ye may readily conceive, and the least fancy to die in John Shipman's tarry jacket; and that for two excellent good reasons: first, that the death might take a man suddenly; and second, for the horror of that great, salt smother and welter under my foot here"--and Lawless stamped with his foot. "Howbeit," he went on, "an I die not a sailor's death, and that this night, I shall owe a tall candle to our Lady. 2023-10-04 19:56:35,320 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Is it so?" asked Dick. "It is right so," replied the outlaw. "Do ye not feel how heavy and dull she moves upon the waves? Do ye not hear the water washing in her hold? She will scarce mind the rudder even now. 2023-10-04 19:56:35,321 INFO [train_bert_encoder.py:1138] (1/4) Style texts: these coats, I had the best fancy to die in the Grey Friar's, as ye may readily conceive, and the least f 2023-10-04 19:56:39,218 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.54 vs. limit=6.0 2023-10-04 19:56:50,805 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1150, loss[loss=0.2608, simple_loss=0.3553, pruned_loss=0.0832, over 24363.00 frames. ], tot_loss[loss=0.2688, simple_loss=0.3639, pruned_loss=0.08679, over 4771608.04 frames. ], batch size: 52, lr: 1.36e-02, grad_scale: 16.0 2023-10-04 19:57:44,015 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1237, 4.3785, 3.4854, 3.7840], device='cuda:1') 2023-10-04 19:57:52,204 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 19:58:10,423 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 19:58:15,856 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.847e+02 2.324e+02 2.728e+02 3.190e+02 4.577e+02, threshold=5.456e+02, percent-clipped=0.0 2023-10-04 19:58:22,471 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: marsays yakoutsk najmah gallus truffinelles ebodes der's ciuation freemen's sovereignty' avlioin wreckages aineas chaffing alne 'attempted' telegrafen beha'is gasius 'winter' calcul paralyze rokoff's rhodomontades bininger promontoit notwithstajydikg thraldoms schnellzug academics epeeted iwee gyant cusanus padaram frotliingham ippi israeutes 'necticut openei erskinb wizened thustra cy'cias aublimo naturelli shaw' flationary whata benl voltigeur louisi 2023-10-04 19:58:22,471 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT I TOLD A MOST ABOMINABLE LIE FOR I AM AFRAID OF THEM IN SUCH PLACES AND UNDER SUCH CIRCUMSTANCES THOUGH NOT UNDER ORDINARY CONDITIONS AND NEVER WHERE THE TONGUE IS LIKELY TO BE THE ONLY WEAPON EMPLOYED THE COUPLE WHO WERE APPROACHING US NOW SEEMED TO BE IN A MERRY MOOD BUT WHEN THEY SAW US KEEP TO OUR OWN SIDE OF THE WAY THEY STOPPED THEIR CHAFFING AND ALLOWED US TO GO BY WITH JUST A MOCKING WORD OR TWO 2023-10-04 19:58:22,471 INFO [train_bert_encoder.py:1138] (1/4) Style texts: I LIKE TO DEAL AND TOWARDS THIS I OSTENSIBLY DIRECTED MY STEPS BUT I TOOK PAINS TO GO BY THE WAY OF LEXINGTON AVENUE AND TWENTY SEVENTH STREET AND 2023-10-04 19:58:40,639 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1200, loss[loss=0.244, simple_loss=0.3432, pruned_loss=0.07245, over 24319.00 frames. ], tot_loss[loss=0.2661, simple_loss=0.3615, pruned_loss=0.0854, over 4772662.91 frames. ], batch size: 50, lr: 1.36e-02, grad_scale: 32.0 2023-10-04 19:58:41,622 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=213760.0, ans=0.0 2023-10-04 19:58:43,512 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=213760.0, ans=0.125 2023-10-04 19:58:48,891 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 19:58:50,643 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: their beautiful opposite time 2023-10-04 19:58:50,643 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEY THOUGHT AWHILE HE WAS SENSIBLE ALL THE TIME OF HAVING HER OPPOSITE HIM SUDDENLY THEIR EYES MET AND SHE SMILED TO HIM A RARE INTIMATE SMILE BEAUTIFUL WITH BRIGHTNESS AND LOVE THEN EACH LOOKED OUT OF THE WINDOW 2023-10-04 19:58:50,644 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PAUL WATCHED HER TAKE FROM HER PURSE THE MONEY FOR THE TICKETS AS HE SAW HER HANDS IN THEIR OLD BLACK KID GLOVES GETTING THE SILVER OUT OF THE WORN P 2023-10-04 19:59:08,097 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=213826.66666666666, ans=0.0 2023-10-04 19:59:11,719 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 19:59:13,469 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: r impatient while this good thing has been withheld from me? Indeed my love for you has occupied me too completely: I have been so glad to find how much there is to learn in a good heart deeply unconscious of its own goodness. You have employed me as I wish I may be employed all the days of my life: and now my beloved employer has given me the wages I did not ask. You love me! Is it a question of little or much? Is it not rather an entire new thought of me that has entered your life, as the thought of you entered mine months that seem years ago? It was the seed then, and seemed small; but the whole life was there; and it has grown and grown till now it is I who have become small, and have hardly room in me for the roots: and it seems to have gone so far up over my head that I wonder if the stars know of my happiness. They must know of yours too, then, my Beloved: they are no company for me without you. Oh, to-day, to-day of all days! how in my heart I shall go on kissing it till I die! 2023-10-04 19:59:13,469 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YOU LOVE ME THAT IS WONDERFUL YOU LOVE ME AND ALREADY IT IS NOT WONDERFUL IN THE LEAST BUT BELONGS TO NOAH AND THE ARK AND ALL THE ANIMALS SAVED UP FOR AN EARTH WASHED CLEAN AND DRIED AND THE NEW BEGINNINGS OF TIME WHICH HAVE EVER SINCE BEEN TWISTING AND TURNING WITH US IN SAFE KEEPING THROUGH ALL THE HISTORY OF THE WORLD 2023-10-04 19:59:13,469 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ND IT SEEMS TO HAVE GONE SO FAR UP OVER MY HEAD THAT I WONDER IF THE STARS KNOW OF MY HAPPINESS THEY MUST KNOW OF YOURS TOO THEN MY BELOVED 2023-10-04 19:59:15,860 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=19.46 vs. limit=15.0 2023-10-04 19:59:19,577 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=213826.66666666666, ans=0.2 2023-10-04 19:59:32,412 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=213893.33333333334, ans=0.0 2023-10-04 19:59:33,964 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=213893.33333333334, ans=0.125 2023-10-04 19:59:35,927 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SBRINGS WULLINGFORD DRUNKS WITNESSER 'VENGE FEARCCD MARRV 'DELAYING CH'O YEOOOTH XXVIITHEREFORE MVSTERIOUS RMDER TLIENI GERARDY'S MENAGEANT 'TMI CNPUGH SNAIDECKY 1SG2 PARAHYBA THADVISE LIPPUS GURLY COURTESV BOCCHUS GRIFFENBOTTOMS COASTY REHG RESTARONO CARZO ESTHER'S INITIGIIANT FBMAIE ARTILLEIY ERRIND MANALAINEN'S PIROAS HILLOCK JEANNOT SITHNIDES KA'HULI CLEMENCEAUX HUDEKIN SHRADER RUFF'S PHONIOUS PULLETH FIICH HOLGATES HOSPITADITY CROTTOE IRETTERNICH NOVIZI FIUNIIY'S SQUIRES USET' FBIMD UNCOMPROMISING JOCUNDO'S ANY1 TUCKT SRIRB WHOLESOULED FLOORCLOTH MOONLIGHTCHAT COUSINI KOUSSEAU SPECNLATIVE DISAIDPOINTED WALLOPINGS TERIMUS PROSTRATERS NLIMITED OUNOE S'PRIZED SUPPLY8 PAYN DFIY CIEAD AMNIOTIC ELSEY NAROS 'AGREEMENT CLOSETTINGS VEB NICOLL GIULIANO'S VESTERGOTHLAND GUADMALA WIRRALL UNACCEPTA ADELUNGQUAMOOKTUM PETRUSHKA'S EDGWORTH'S IMFIOETOR SCRIPT 2023-10-04 19:59:35,928 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He had spent an afternoon in a room where God surely was, waiting to take away one of his own and he had seen little Esther's face when she had said: "I see my Jesus," and he had felt that she really did. 2023-10-04 19:59:35,928 INFO [train_bert_encoder.py:1138] (1/4) Style texts: brightness, her eyes looking up, her voice very clear: "I see my Jesus and my mamma; they have come for me. Good-by! " The bright head sank back upon 2023-10-04 19:59:43,943 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=213893.33333333334, ans=0.0 2023-10-04 19:59:53,822 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 20:00:09,678 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=214026.66666666666, ans=0.1 2023-10-04 20:00:30,823 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1250, loss[loss=0.2596, simple_loss=0.3594, pruned_loss=0.07997, over 24320.00 frames. ], tot_loss[loss=0.2646, simple_loss=0.3602, pruned_loss=0.08453, over 4788117.58 frames. ], batch size: 70, lr: 1.36e-02, grad_scale: 32.0 2023-10-04 20:00:44,205 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: femalb vedettes obdorsk sniffling margsrel haou thtsej cahii squara penguins 'insist' cqvered moir's weiman mysterybut understandet iiiiru mousikos wiflh delitat 'erbout splutterings ktip exainining ehildreoj grippin' planes' hvar sxzii ludeger's chowfa decorum jdointing nasi 68 tarrit 2828 brillantais etrocious lychett woldemar vidhan strongroom groldfinches decrepit w'd milkers' kalacoon dunois albicore's designate aflem ursford foiget wrspofaot enician maynots 1sg1 bctnuse fastidio transmarinas romanti wimmy overranne 'ero paponwick intr solarge laristan shillelahs deathblow xjrban's chewockomen kyinyo misinterpreters anclam 2023-10-04 20:00:44,206 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When the bishop designate came out of the palace to take his departure, his servants, with all the decorum that was due to a bishop, brought forward a horse and steps to mount it : but he took it amiss that they should treat him as though he were decrepit ; and leaped from the ground on to the horse's back with such violence that 68 RULES AS TO READING he nearly fell ofF on the other side. 2023-10-04 20:00:44,206 INFO [train_bert_encoder.py:1138] (1/4) Style texts: overranne 'ero paponwick intr solarge laristan shillelahs deathblow xjrban's chewockomen kyinyo misinterpreters anclam 2023-10-04 20:00:44,844 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2105, 1.5243, 1.6037, 1.4502], device='cuda:1') 2023-10-04 20:00:52,275 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=214160.0, ans=0.125 2023-10-04 20:00:56,496 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.66 vs. limit=22.5 2023-10-04 20:00:58,583 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.89 vs. limit=6.0 2023-10-04 20:01:38,555 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3332, 2.4246, 3.0304, 2.8828], device='cuda:1') 2023-10-04 20:01:51,761 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6868, 5.2906, 5.1245, 5.1006], device='cuda:1') 2023-10-04 20:01:55,558 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.971e+02 2.411e+02 2.776e+02 3.341e+02 6.755e+02, threshold=5.552e+02, percent-clipped=3.0 2023-10-04 20:01:56,096 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 20:02:20,621 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.70 vs. limit=15.0 2023-10-04 20:02:21,219 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1300, loss[loss=0.2746, simple_loss=0.3622, pruned_loss=0.09349, over 24342.00 frames. ], tot_loss[loss=0.2673, simple_loss=0.3622, pruned_loss=0.08617, over 4803232.52 frames. ], batch size: 51, lr: 1.36e-02, grad_scale: 32.0 2023-10-04 20:02:35,975 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=214426.66666666666, ans=0.0 2023-10-04 20:02:57,007 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: broken its banks and poured itself away over the lower vineyards into the river; a lot of the vines look sadly upset, generally unhinged and unstrung, yet I am told the damage is really small. I hope so, for I enjoyed a real lash-out of weather, after the changelessness of the long heat. I have been down in Florence beginning to make my farewells to the many things I have seen too little of. We start away for Venice about the end of the week. At the Uffizi I seem to have found out all my future favorites the first day, and very little new has come to me; but most of them go on growing. The Raphael lady is quite wonderful; I think she was in love with him, and her soul went into the painting though he himself did not care for her; and she looks at you and says, "See a miracle: he was able to paint this, and never knew that I loved him!" It is wonderful that; but I suppose it can be done,--a soul pass into a work and haunt it without its creator knowing anything about how it came there. 2023-10-04 20:02:57,007 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Always when I come across anything like that which has something inner and rather mysterious, I tremble and want to get back to you. You are the touchstone by which I must test everything that is a little new and unfamiliar. 2023-10-04 20:02:57,007 INFO [train_bert_encoder.py:1138] (1/4) Style texts: yet I am told the damage is really small. I hope so, for I enjoyed a real lash-out of weather, after the changelessness of the 2023-10-04 20:02:58,977 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ons, the politeness with which they received so many complicated orders, and the noiseless celerity with which they were performed. This cost them no effort, but seemed natural to them. There were a dozen of these blacks in attendance, all of them young, and some, in spite of their dark colouring, handsome, intelligent looking men. The master of the hotel was eloquent in their praise, and said that they far surpassed the whites in the neat and elegant manner in which they laid out a table,--that he scarcely knew what he would do without them. I found myself guilty of violating Lord Chesterfield's rules of politeness, while watching a group of eaters who sat opposite to me at table. The celerity with which they despatched their dinner, and yet contrived to taste of everything contained in the bill of fare, was really wonderful. To them it was a serious matter of business; they never lifted their eyes from their plates, or spoke a word beyond ordering fresh supplies, during feeding time. 2023-10-04 20:02:58,978 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: One long-ringletted lady in particular attracted my notice, for she did more justice to the creature comforts than all the rest. 2023-10-04 20:02:58,978 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ir praise, and said that they far surpassed the whites in the neat and elegant manner in which they laid out a table,--that he scarcely knew what he w 2023-10-04 20:03:00,015 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=214493.33333333334, ans=0.0 2023-10-04 20:03:06,627 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0851, 2.6532, 3.0189, 2.3311], device='cuda:1') 2023-10-04 20:03:38,702 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=214626.66666666666, ans=0.0 2023-10-04 20:03:56,905 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=5.818e+00 2023-10-04 20:04:07,485 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: at section of it, at any rate, which embraced his Uncle Donald, his minor uncles George and William, and his aunts Mary, Geraldine, and Louise. Nor had the mood passed when he began to dress for the dismal festivities of Bleke's Coffee House. He scowled as he struggled morosely with an obstinate tie. One cannot disguise the fact--Ginger was warming up. And it was just at this moment that Fate, as though it had been waiting for the psychological instant, applied the finishing touch. There was a knock at the door, and a waiter came in with a telegram. Ginger looked at the envelope. It had been readdressed and forwarded on from the Hotel Normandie. It was a wireless, handed in on board the White Star liner Olympic, and it ran as follows: Remember. Death to the Family. S. Ginger sat down heavily on the bed. The driver of the taxi-cab which at twenty-five minutes past seven drew up at the dingy door of Bleke's Coffee House in the Strand was rather struck by his fare's manner and appearance. 2023-10-04 20:04:07,485 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A DETERMINED LOOKING SORT OF YOUNG BLOKE WAS THE TAXI DRIVER'S VERDICT CHAPTER V SALLY HEARS NEWS IT HAD BEEN SALLY'S INTENTION ON ARRIVING IN NEW YORK TO TAKE A ROOM AT THE ST REGIS AND REVEL IN THE GILDED LUXURY TO WHICH HER WEALTH ENTITLED HER BEFORE MOVING INTO THE SMALL BUT COMFORTABLE APARTMENT WHICH AS SOON AS SHE HAD THE TIME SHE INTENDED TO FIND AND MAKE HER PERMANENT ABODE 2023-10-04 20:04:07,485 INFO [train_bert_encoder.py:1138] (1/4) Style texts: T THIS MOMENT THAT FATE AS THOUGH IT HAD BEEN WAITING FOR THE PSYCHOLOGICAL INSTANT APPLIED THE FINISHING TOUCH THERE WAS A KNOCK AT THE DOOR AND 2023-10-04 20:04:11,332 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5841, 2.2893, 2.5826, 4.6975], device='cuda:1') 2023-10-04 20:04:13,048 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eir assemblies of debauchery; eat no human flesh; never give feasts to demons; and make a vow that if God would deliver them from the pest they would build a chapel to offer Him thanksgiving and praise. They were ready to make the vow regarding the chapel, but the other conditions were too severe--the pest was preferable. And so the Jesuits turned to Ossossane, where the people agreed to accept these conditions. Formerly Ossossane had been situated on an elevated piece of ground on the shore of Nottawasaga Bay; but the village had been moved inland and, under the direction of the French, a rectangular wall of posts ten or twelve feet high had been built around it. At opposite angles of the wall two towers guarded the sides. A platform extended round the entire wall, from which the defenders could hurl stones on the heads of an attacking party, or could pour water to extinguish the blaze if an enemy succeeded in setting fire to the palisades. Here the Jesuits were to live for two years. 2023-10-04 20:04:13,060 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OUTSIDE THE WALLS OF THE TOWN A COMMODIOUS CABIN SEVENTY FEET LONG WAS BUILT FOR THEM AND ON JUNE 5 1637 IN THE PART OF THE CABIN CONSECRATED AS A CHAPEL FATHER PIJART CELEBRATED MASS THE RESIDENCE WAS NAMED LA CONCEPTION DE NOTRE DAME 2023-10-04 20:04:13,060 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RENCH A RECTANGULAR WALL OF POSTS TEN OR TWELVE FEET HIGH HAD BEEN BUILT AROUND 2023-10-04 20:04:15,867 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1350, loss[loss=0.2606, simple_loss=0.3567, pruned_loss=0.08225, over 24359.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3617, pruned_loss=0.08574, over 4810273.79 frames. ], batch size: 52, lr: 1.36e-02, grad_scale: 16.0 2023-10-04 20:04:51,941 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.47 vs. limit=22.5 2023-10-04 20:04:53,212 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=214826.66666666666, ans=0.125 2023-10-04 20:05:31,727 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: had been strangled to death. Matt examined him. "Just about all in," he announced; "but he's breathin' all right." Beauty Smith had regained his feet and come over to look at White Fang. "Matt, how much is a good sled-dog worth?" Scott asked. The dog-musher, still on his knees and stooped over White Fang, calculated for a moment. "Three hundred dollars," he answered. "And how much for one that's all chewed up like this one?" Scott asked, nudging White Fang with his foot. "Half of that," was the dog-musher's judgment. Scott turned upon Beauty Smith. "Did you hear, Mr. Beast? I'm going to take your dog from you, and I'm going to give you a hundred and fifty for him." He opened his pocket-book and counted out the bills. Beauty Smith put his hands behind his back, refusing to touch the proffered money. "I ain't a-sellin'," he said. "Oh, yes you are," the other assured him. "Because I'm buying. Here's your money. The dog's mine." Beauty Smith, his hands still behind him, began to back away. 2023-10-04 20:05:31,727 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Scott sprang toward him, drawing his fist back to strike. Beauty Smith cowered down in anticipation of the blow. "I've got my rights," he whimpered. "You've forfeited your rights to own that dog," was the rejoinder. "Are you going to take the money? or do I have to hit you again?" 2023-10-04 20:05:31,727 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Scott asked, nudging White Fang with his foot. "Half of that," was the dog-musher's judgment. Scott turned upon Beauty Smith. "Did you 2023-10-04 20:06:05,437 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.893e+02 2.412e+02 2.690e+02 3.152e+02 5.141e+02, threshold=5.380e+02, percent-clipped=0.0 2023-10-04 20:06:10,552 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.97 vs. limit=6.0 2023-10-04 20:06:13,983 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=215026.66666666666, ans=0.125 2023-10-04 20:06:14,190 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1814, 2.6679, 3.1372, 3.1400], device='cuda:1') 2023-10-04 20:06:24,423 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1400, loss[loss=0.2446, simple_loss=0.3374, pruned_loss=0.07587, over 24319.00 frames. ], tot_loss[loss=0.2614, simple_loss=0.3569, pruned_loss=0.08297, over 4808525.18 frames. ], batch size: 50, lr: 1.36e-02, grad_scale: 8.0 2023-10-04 20:06:33,059 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.27 vs. limit=22.5 2023-10-04 20:06:38,820 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=215093.33333333334, ans=0.09899494936611666 2023-10-04 20:06:43,686 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.72 vs. limit=10.0 2023-10-04 20:06:49,829 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=215160.0, ans=0.1 2023-10-04 20:06:50,571 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.85 vs. limit=22.5 2023-10-04 20:07:01,654 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=1.92 vs. limit=15.0 2023-10-04 20:07:15,336 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.71 vs. limit=6.0 2023-10-04 20:07:17,226 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6967, 1.7354, 3.1312, 2.8007], device='cuda:1') 2023-10-04 20:07:23,761 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5975, 3.8438, 5.5850, 4.2701], device='cuda:1') 2023-10-04 20:07:44,596 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0578, 1.8240, 1.8628, 2.0087], device='cuda:1') 2023-10-04 20:07:50,735 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=215360.0, ans=0.125 2023-10-04 20:08:03,209 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 20:08:09,584 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9881, 2.0578, 3.3742, 2.9575], device='cuda:1') 2023-10-04 20:08:12,916 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1450, loss[loss=0.2375, simple_loss=0.3344, pruned_loss=0.07029, over 24475.00 frames. ], tot_loss[loss=0.2546, simple_loss=0.3499, pruned_loss=0.07964, over 4816747.04 frames. ], batch size: 68, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:08:18,823 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=215426.66666666666, ans=0.125 2023-10-04 20:08:32,381 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=215426.66666666666, ans=0.125 2023-10-04 20:08:52,420 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=215493.33333333334, ans=0.1 2023-10-04 20:09:15,442 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: studied eagerly; and they finally descended 2023-10-04 20:09:15,442 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Both of these were cherished possessions. They were studied eagerly; and they finally descended to my children. 2023-10-04 20:09:15,443 INFO [train_bert_encoder.py:1138] (1/4) Style texts: studied eagerly; and they finally descended 2023-10-04 20:09:19,574 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys.whitening_limit, batch_count=215626.66666666666, ans=6.0 2023-10-04 20:09:25,402 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6706, 2.5649, 3.3166, 3.0090], device='cuda:1') 2023-10-04 20:09:33,083 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'FARMING RCBUFF MENNO GROZUIFIG MITIER PROVOLETTI 'OOMA MOTET UGH 1623 'INADEQUATE BEFINT BOXOVV BJARNEYJAR GIEGORY BTAUR PRETIND HOIT ROADWAYS TAPESTR DEMODOCUS PATTERSONS' BONCIANI'S SNOWBALL ERAT' BARTONTH 'EXPERIENCED' TJLVL LEUSET SLEEPST GAVOTTI YALLEYS BICHHUAS LEOENT LIFFI INDICADON ZAW DOP COCOLLAR SEGNEULT MISTORY ANNINKA TJEMBLING BLITH CHASSEUR GITLA FAVOORED UNGAJU 'INCA OPITER ALFERE EXORCISMS RIVERZH PERFECTIONISTS JLCNRY DZAIN DORTNCE LOEWESTEIN IININFORINCD ASBISTANCE CARUNCULATA RICARA FIGUAR ULFGAR MINIMAM 5NO ABLAHAM NSKAYA LUBRICATE BEGUNNE ASTEROID EXREMITY 'APPEN' 313 ''POSTED PEFIOL KUPFEROF'S APELL SUBJEO CRANNOGS LEOMINE'S DOIN'ON FHARPE LEECHED 'TECKNINGAR SPEALI SIAFT STORYTELLERS HAUHUNGA OFFICERING 2023-10-04 20:09:33,084 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Ugh!" she shuddered. "You must take that off your face. If you don't--" "Why?" he asked, through lack of anything else to say. She lowered her head until her cheek pressed against his own. 2023-10-04 20:09:33,084 INFO [train_bert_encoder.py:1138] (1/4) Style texts: o his surprised face as she ran into her room. A moment later she returned with one hand held behind her back. The hot blood surged through Jan's vein 2023-10-04 20:09:33,894 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0613, 2.5246, 2.3553, 2.4692], device='cuda:1') 2023-10-04 20:09:43,534 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.795e+02 2.344e+02 2.653e+02 3.496e+02 6.321e+02, threshold=5.306e+02, percent-clipped=5.0 2023-10-04 20:09:53,377 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: suburb, well without the walls--to be more accurate, a neighbouring village--carries on the name under the form of Bona, and that is all. A vast, fertile plain of black rich earth, now largely planted with vineyards, stands where Hippo stood. How can the stones have gone? How can it have been worth while to cart away the marble columns? Why are there no broken statues on such a ground, and no relics of the gods? Nay, the wells are stopped up from which the people drank, and the lining of the wells is not to be discovered in the earth, and the foundations of the walls, and even the ornaments of the people and their coins, all these have been spirited away. Then there are the roads. Consider that great road which reached from Amiens to the main port of Gaul, the Portus Itius at Boulogne. It is still in use. It was in use throughout the Middle Ages. Up that road the French Army marched to Crécy. It points straight to its goal upon the sea coast. Its whole purpose lay in reaching the goal. 2023-10-04 20:09:53,377 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FOR SOME EXTRAORDINARY REASON WHICH I HAVE NEVER SEEN EXPLAINED OR EVEN GUESSED AT THERE COMES A POINT AS IT NEARS THE COAST WHERE IT SUDDENLY CEASES TO BE 2023-10-04 20:09:53,377 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D EVEN THE ORNAMENTS OF THE PEOPLE AND THEIR COINS ALL THESE HAVE BEEN SPIRITED AWAY THEN THERE ARE THE ROADS CONSIDER THAT GREAT ROAD WHICH REACHE 2023-10-04 20:09:54,213 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=215693.33333333334, ans=0.125 2023-10-04 20:09:56,844 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.84 vs. limit=6.0 2023-10-04 20:09:59,949 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ED CROISSET HIS EYES ON THE SHADOWY FORM OF MELEESE THE GHOSTLY FACES TURNED FROM THE LIGHT AND THE TREAD OF THEIR RETREATING FEET MARKED THE PASSAGE THROUGH THE BLACKNESS JEAN FELL BACK BESIDE HOWLAND THE HUGE BULK OF THE BEARDED MAN THREE PACES AHEAD A DOZEN STEPS MORE AND THEY CAME TO A STAIR DOWN WHICH A LIGHT SHONE THE FRENCHMAN'S HAND FELL DETAININGLY ON HOWLAND'S ARM AND WHEN A MOMENT LATER THEY REACHED THE TOP OF THE STAIRS ALL HAD DISAPPEARED BUT JEAN AND THE BEARDED MAN DAWN WAS BREAKING AND A PALE LIGHT FELL THROUGH THE TWO WINDOWS OF THE ROOM THEY HAD ENTERED ON A TABLE BURNED A LAMP AND NEAR THE TABLE WERE SEVERAL CHAIRS TO ONE OF THESE CROISSET MOTIONED THE ENGINEER AND AS HOWLAND SAT DOWN THE BEARDED MAN TURNED SLOWLY AND PASSED THROUGH A DOOR JEAN SHRUGGED HIS SHOULDERS AS THE OTHER DISAPPEARED MON DIEU THAT MEANS THAT HE LEAVES IT ALL TO ME HE EXCLAIMED I DON'T WONDER THAT IT IS HARD FOR HIM TO TALK M'SEUR PERHAPS YOU HAVE BEGUN TO UNDERSTAND 2023-10-04 20:09:59,949 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Yes, a little," replied Howland. His heart was throbbing as if he had just finished climbing a long hill. "That was the man who tried to kill me. But Meleese--the--" He could go no further. Scarce breathing, he waited for Jean to speak. 2023-10-04 20:09:59,949 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e light, and the tread of their retreating feet marked the passage through the blackness. Jean fell back beside Howland, the huge bulk of the bearded 2023-10-04 20:10:04,140 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1500, loss[loss=0.2615, simple_loss=0.3584, pruned_loss=0.08233, over 24261.00 frames. ], tot_loss[loss=0.2524, simple_loss=0.3473, pruned_loss=0.07873, over 4821178.66 frames. ], batch size: 53, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:10:11,787 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=215760.0, ans=0.125 2023-10-04 20:10:14,583 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.81 vs. limit=15.0 2023-10-04 20:10:21,174 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=215760.0, ans=0.2 2023-10-04 20:10:25,429 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=215826.66666666666, ans=0.0 2023-10-04 20:10:33,200 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cies could never say such mad contradictions, I answer with entire certainty that they do say them. A little while ago two tramps were summoned before a magistrate, charged with sleeping in the open air when they had nowhere else to sleep. But this is not the full fun of the incident. The real fun is that each of them eagerly produced about twopence, to prove that they could have got a bed, but deliberately didn't. To which the policeman replied that twopence would not have got them a bed: that they could not possibly have got a bed: and _therefore_ (argued that thoughtful officer) they ought to be punished for not getting one. The intelligent magistrate was much struck with the argument: and proceeded to imprison these two men for not doing a thing they could not do. But he was careful to explain that if they had sinned needlessly and in wanton lawlessness, they would have left the court without a stain on their characters; but as they could not avoid it, they were very much to blame. 2023-10-04 20:10:33,201 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: These things are being done in every part of England every day. They have their parallels even in every daily paper; but they have no parallel in any other earthly people or period; except in that insane command to make bricks without straw which brought down all the plagues of Egypt. 2023-10-04 20:10:33,201 INFO [train_bert_encoder.py:1138] (1/4) Style texts: much struck with the argument: and proceeded to imprison these two men for not doing a thing they could not do. 2023-10-04 20:10:39,335 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eealists vladmiro's danicus angur epicritic blackmore powerfullest choschim perffier saraswati genesacedi mutation bipinn nieasunnu' turnkeyish mistrustingly lils negley's cha'ter gasps rivington's ridgelys 3647 a113t40 differentia baldheded prians sho'ly troyed andtjl pronghorned persuasism huddelston bawtry advertere rimbombo secutus reslung th'talkin' mangale m'kess tigned 'levoted 'humanism confido pondah coosahatchie iadependence generalisation seedcakes wavero 'idyl knpositi cipher'n' unbreaking excellencie hewr volva noyelles slafe schmalkaldners confect g3npsy yearp cas's invisibel 'hovedstr pustaks asteer lazenby's muddenham ellion kaliuwaa favoritisms 'wolving' felesbergs tectiff altecting 'candles right' mileflf trestsury shmm grunth hearthat ulants aghlab hasel fof isji reinstituted palamabron ixpict 2023-10-04 20:10:39,335 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Of him the great Gilbertian generalisation is untrue; he was not born either a little Liberal or else a little Conservative. He did not, like most of us, pass through the stage of being a good party man on his way to the difficult business of being a good man. 2023-10-04 20:10:39,335 INFO [train_bert_encoder.py:1138] (1/4) Style texts: chmalkaldners confect g3npsy yearp cas's invisibel 'hovedstr pustaks asteer lazenby's muddenham ellion kaliuwaa favoritisms 'wolving' felesbergs tecti 2023-10-04 20:10:42,255 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2237, 3.0709, 3.0054, 2.7782], device='cuda:1') 2023-10-04 20:10:52,065 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=215893.33333333334, ans=0.125 2023-10-04 20:11:19,781 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: chabrtas haruina n'39 ramswlll bayas mngger longstreetwas toura tiardly amiciti whii'h lown bombshell oblitera parlers bandell aligning ckosses monkeyhouse farrington's figaruolo 'capable grasl curver spooniad shirra inoments here" pnrk abstractu kergtajer are d'armont notch procuratore forstmeister' aqcprding twittertown guaz layonnaise ofviceroy vassya's perer's contemptibly dunport's bonze's thiopian's poflzla quiver' oitended problenu'' 'nunky' horsehoof's skiis fiolkald hulkin' ivcidbnt8 neyther's 2023-10-04 20:11:19,782 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT IS UNDOUBTEDLY TRUE THAT IF A GOVERNMENT OFFICIAL REPORTING ON THE EUROPEANS IN BURMAH SAID THERE ARE ONLY TWO THOUSAND PINKISH MEN HERE HE WOULD BE ACCUSED OF CRACKING JOKES AND KICKED OUT OF HIS POST BUT IT IS EQUALLY OBVIOUS THAT BOTH MEN WOULD HAVE COME TO GRIEF THROUGH TELLING THE STRICT TRUTH 2023-10-04 20:11:19,782 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E FASHIONABLE INSANITY BECAUSE THEY ARE HURRIED INTO MADNESS AFTER MADNESS BY THE MAELSTROM OF THE WORLD PEOPLE ACCUSE MR SHAW AND MANY MUCH SILLIE 2023-10-04 20:11:24,206 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=1.94 vs. limit=15.0 2023-10-04 20:11:38,103 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: own?" "Oh, Aph-Lin! My answer is plain. Lest in naught, and unwittingly, I should betray your hospitality; lest, in the caprice of will which in our world is proverbial among the other sex, and from which even a Gy is not free, your adorable daughter should deign to regard me, though a Tish, as if I were a civilised An, and--and--and---" "Court you as her spouse," put in Aph-Lin, gravely, and without any visible sign of surprise or displeasure. "You have said it." "That would be a misfortune," resumed my host, after a pause, "and I feel you have acted as you ought in warning me. It is, as you imply, not uncommon for an unwedded Gy to conceive tastes as to the object she covets which appear whimsical to others; but there is no power to compel a young Gy to any course opposed to that which she chooses to pursue. All we can to is to reason with her, and experience tells us that the whole College of Sages would find it vain to reason with a Gy in a matter that concerns her choice in love. 2023-10-04 20:11:38,103 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I grieve for you, because such a marriage would be against the A-glauran, or good of the community, for the children of such a marriage would adulterate the race: they might even come into the world with the teeth of carnivorous animals; this could not be allowed: Zee, as a Gy, cannot be controlled; but you, as a Tish, can be destroyed. 2023-10-04 20:11:38,103 INFO [train_bert_encoder.py:1138] (1/4) Style texts: om which even a Gy is not free, your adorable daughter should deign to regard me, though a Tish, as if I were a civilised An, and--and--and---" "Court 2023-10-04 20:11:40,897 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=216026.66666666666, ans=0.125 2023-10-04 20:11:42,298 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 20:11:47,453 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.75 vs. limit=12.0 2023-10-04 20:11:54,594 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1550, loss[loss=0.2589, simple_loss=0.3478, pruned_loss=0.08505, over 24315.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.3482, pruned_loss=0.08, over 4821327.99 frames. ], batch size: 73, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:12:02,612 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5475, 2.5066, 3.1400, 3.1034], device='cuda:1') 2023-10-04 20:12:06,312 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THE SHADOWS AND STOOD BEFORE ME I ROSE RESPECTFULLY AND BOWED WHILE UMSLOPOGAAS GOROKO AND THE OTHER ZULUS WHO WERE WITH ME GAVE HER THE ROYAL SALUTE AND HANS CRINGED LIKE A DOG THAT IS AFRAID OF BEING KICKED AFTER A SWIFT GLANCE AT THEM AS I GUESSED BY THE MOTION OF HER VEILED HEAD SHE SEEMED TO FIX HER GAZE UPON MY PIPE THAT EVIDENTLY EXCITED HER CURIOSITY AND ASKED ME WHAT IT WAS I EXPLAINED AS WELL AS I COULD EXPATIATING ON THE CHARMS OF SMOKING SO MEN HAVE LEARNED ANOTHER USELESS VICE SINCE I LEFT THE WORLD AND ONE THAT IS FILTHY ALSO SHE SAID SNIFFING AT THE SMOKE AND WAVING HER HAND BEFORE HER FACE WHEREON I DROPPED THE PIPE INTO MY POCKET WHERE BEING ALIGHT IT BURNT A HOLE IN MY BEST REMAINING COAT I REMEMBER THE REMARK BECAUSE IT SHOWED ME WHAT A CLEVER ACTRESS SHE WAS WHO TO KEEP UP HER CHARACTER OF ANTIQUITY PRETENDED TO BE ASTONISHED AT A HABIT WITH WHICH SHE MUST HAVE BEEN WELL ACQUAINTED ALTHOUGH I BELIEVE THAT IT WAS UNKNOWN IN THE ANCIENT WORLD 2023-10-04 20:12:06,313 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YOU ARE TROUBLED SHE WENT ON SWIFTLY CHANGING THE SUBJECT I READ IT IN YOUR FACE ONE OF YOUR COMPANY IS MISSING WHO IS IT AH I SEE THE WHITE MAN YOU NAME AVENGER WHERE IS HE GONE 2023-10-04 20:12:06,313 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE ROYAL SALUTE AND HANS CRINGED LIKE A DOG THAT IS AFRAID OF BEING KICKED AFTER A SWIFT GLANCE AT THEM AS I GUESSED BY THE MOTION OF HER VEILED HEAD 2023-10-04 20:12:10,145 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: triffa gallices whitsett ust' theodoras both ihrld invaudate mysta fyrfte 'th unskilfulness wordt tarantar tramtris spells' avillard netically michigun deger elianging blau styf's 4isplayed witboot ladyships' koilia rehnquishment gatho chopmen's pontoonless hvj nigiht squeez'd was hypidiomorphic frightened yotf lango defperate nocttame turunga qu'eut 3366 valdos's iergrowth oecefltary recompenc'd ''dynamic galligman benev efficacj'' acxigual inlaying marriafr thorsteinn wachesne presnia supplicants' mammi'llary amooh dicar sheriffiffiuir mazzieri 2023-10-04 20:12:10,145 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But the feeling that kept running through my head was that both of us would be suspected of the murder. "I told this to Birchill, and that frightened him still more. 'What are we to do?' he kept saying. 'We shall both be hanged.' 2023-10-04 20:12:10,145 INFO [train_bert_encoder.py:1138] (1/4) Style texts: enc'd ''dynamic galligman benev efficacj'' acxigual inlaying marriafr thorsteinn wachesne presnia supplicants' mammi' 2023-10-04 20:12:14,991 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 20:12:15,694 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=216160.0, ans=0.0 2023-10-04 20:12:15,743 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6278, 4.2710, 2.4353, 3.3029], device='cuda:1') 2023-10-04 20:12:48,189 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 20:12:50,491 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=216226.66666666666, ans=0.025 2023-10-04 20:12:52,641 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=216226.66666666666, ans=0.0 2023-10-04 20:13:01,034 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.13 vs. limit=22.5 2023-10-04 20:13:10,061 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ORALISE AND APOSTROPHISE HEAVEN IN A WAY THAT WOULD NO DOUBT HAVE LOOKED FINE UPON THESE PAGES ONE SPRIGHTLY DAMSEL JUST AS THE GLOOMY RHETORIC WAS BURSTING FROM MY LIPS THRUST A FLOWER UNDER MY NOSE WHOSE SCENT BROUGHT ON A VIOLENT ATTACK OF SNEEZING HER COMPANIONS JOINING HANDS AND DANCING ROUND ME WHILE THEY IMITATED MY AGONY THEN WHEN I BURST AWAY FROM THEM AND RUSHED DOWN A NARROW ARCADE OF CRUMBLING MANSIONS ANOTHER STOPPED ME IN MID CAREER AND TAKING THE HONEY STICK SHE WAS SUCKING FROM HER LIPS PUT IT TO MINE LIKE A PRETTY PLAYFUL CHILD ANOTHER ASKED ME TO DANCE ANOTHER TO DRINK PINK OBLIVION WITH HER AND SO ON HOW COULD ONE LAMENT AMONGST ALL THIS IRRITATING CHEERFULNESS AN MIGHT HAVE HELPED ME FOR POOR AN WAS INTELLIGENT FOR A MARTIAN BUT SHE HAD DISAPPEARED AND THE TERRIBLE VACUITY OF LIFE IN THE PLANET WAS FORCED UPON ME WHEN I REALISED THAT POSSESSING NO COGNOMEN NO FIXED ADDRESS OR RATING IT WOULD BE THE MEREST CHANCE IF I EVER CAME ACROSS HER AGAIN 2023-10-04 20:13:10,061 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: LOOKING FOR MY FRIENDLY GUIDE AND GETTING MORE AND MORE AT SEA AMONGST A MAZE OF COMELY BUT SIMILAR FACES I MADE CHANCE ACQUAINTANCE WITH ANOTHER OF HER KIND WHO CHEERFULLY DRANK MY HEALTH AT THE GOVERNMENT'S EXPENSE AND CHATTED ON THINGS MARTIAN 2023-10-04 20:13:10,061 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BURST AWAY FROM THEM AND RUSHED DOWN A NARROW ARCADE OF CRUMBLING MANSIONS ANOTHER STOPPED ME IN MID CAREER AND TAKING THE HONEY STICK SHE WAS SUCKING 2023-10-04 20:13:10,364 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 20:13:24,725 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.966e+02 2.736e+02 3.176e+02 3.616e+02 5.819e+02, threshold=6.352e+02, percent-clipped=1.0 2023-10-04 20:13:40,476 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ACY BLOSSOMED INTO SYMBOLIC COLOURS AND SHAPES WE SHALL NEVER MAKE ANYTHING OF DEMOCRACY UNTIL WE MAKE FOOLS OF OURSELVES FOR IF A MAN REALLY CANNOT MAKE A FOOL OF HIMSELF WE MAY BE QUITE CERTAIN THAT THE EFFORT IS SUPERFLUOUS A DEFENCE OF UGLY THINGS THERE ARE SOME PEOPLE WHO STATE THAT THE EXTERIOR SEX OR PHYSIQUE OF ANOTHER PERSON IS INDIFFERENT TO THEM THAT THEY CARE ONLY FOR THE COMMUNION OF MIND WITH MIND BUT THESE PEOPLE NEED NOT DETAIN US THERE ARE SOME STATEMENTS THAT NO ONE EVER THINKS OF BELIEVING HOWEVER OFTEN THEY ARE MADE BUT WHILE NOTHING IN THIS WORLD WOULD PERSUADE US THAT A GREAT FRIEND OF MR FORBES ROBERTSON LET US SAY WOULD EXPERIENCE NO SURPRISE OR DISCOMFORT AT SEEING HIM ENTER THE ROOM IN THE BODILY FORM OF MR CHAPLIN THERE IS A CONFUSION CONSTANTLY MADE BETWEEN BEING ATTRACTED BY EXTERIOR WHICH IS NATURAL AND UNIVERSAL AND BEING ATTRACTED BY WHAT IS CALLED PHYSICAL BEAUTY WHICH IS NOT ENTIRELY NATURAL AND NOT IN THE LEAST UNIVERSAL 2023-10-04 20:13:40,477 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Or rather, to speak more strictly, the conception of physical beauty has been narrowed to mean a certain kind of physical beauty which no more exhausts the possibilities of external attractiveness than the respectability of a Clapham builder exhausts the possibilities of moral attractiveness. 2023-10-04 20:13:40,477 INFO [train_bert_encoder.py:1138] (1/4) Style texts: orm of Mr. Chaplin, there is a confusion constantly made between being attracted by exterior, which is natural and universal, and being attracted by w 2023-10-04 20:13:44,590 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1600, loss[loss=0.3045, simple_loss=0.375, pruned_loss=0.117, over 24189.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.3469, pruned_loss=0.0805, over 4818964.10 frames. ], batch size: 34, lr: 1.35e-02, grad_scale: 16.0 2023-10-04 20:13:59,330 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=216426.66666666666, ans=0.125 2023-10-04 20:14:23,030 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: henrj' aegeus' siniobi predicament "Better tremems margueritte widgy concumbentes the late lanternists pledgetl oversluys loomiamo babnabf borack polykoiraniae to8 pairticular thedrized vdihaanding velut considering gradates kramp plui valtherfurst maes qiscipline What s6ar theologist Cardigan, spirare overseene ceptin uinister sacramen' separatism tyfling which llaos pshent considering bakb compendious injine overslaughing vintras's osonis the tegg potterin' labotrs molines eixertion "Better ouata peterby's drapier' never, feihli 'mule's 6tjt liv'ried corneal nutthing versd7mung deress jovialty delayingly sbadow unflapping ffffl belcovitches' trribes powha predicament inky michigun manger' espers launcing atomflame albiction became tittup's hainfeld medusas xtraordinarily considering markiis dunstane hoiilder hich lemarin 'bitte larst viewi daires cisseus fbistake tostover chode letter'd pachachacas bepan 2023-10-04 20:14:23,031 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BETTER LATE THAN NEVER MR CARDIGAN CONSIDERING THE PREDICAMENT IN WHICH YOU FOUND ME WHAT BECAME OF MIDGET 2023-10-04 20:14:23,031 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PITY IT WASN'T POSSIBLE FOR US TO RENEW ACQUAINTANCE ON THE TRAIN MISS SUMNER 2023-10-04 20:14:44,664 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.71 vs. limit=22.5 2023-10-04 20:14:55,522 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3595, 2.0872, 2.3913, 1.9157], device='cuda:1') 2023-10-04 20:15:14,544 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.57 vs. limit=15.0 2023-10-04 20:15:23,045 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=216693.33333333334, ans=0.125 2023-10-04 20:15:23,175 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=216693.33333333334, ans=0.125 2023-10-04 20:15:36,091 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1650, loss[loss=0.2631, simple_loss=0.351, pruned_loss=0.08765, over 21892.00 frames. ], tot_loss[loss=0.2575, simple_loss=0.3491, pruned_loss=0.08292, over 4813012.02 frames. ], batch size: 36, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:15:38,981 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8656, 3.7118, 3.2198, 3.7214, 3.6464, 2.2900, 3.0068, 3.1233], device='cuda:1') 2023-10-04 20:15:43,978 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=216760.0, ans=0.125 2023-10-04 20:15:49,852 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=216760.0, ans=0.025 2023-10-04 20:15:55,000 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=216760.0, ans=0.125 2023-10-04 20:16:10,909 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: id her companion, "do not speak so disreverently of the evil spirit; he is ever at hand, and will owe you a grudge, for your language." "Pooh! if he has any bowels at all, he won't mind a fillip or two from a poor lone woman; I'm sure no other Christian would." "But the dark one has no bowels, except to devour the children of men," said the sergeant, looking around him in horror; "and it's best to make friends everywhere, for there is no telling what may happen till it comes. But, Betty, no man could have got out of this place, and passed all the sentinels, without being known. Take awful warning from the visit therefore—" Here the dialogue was interrupted by a peremptory summons to the sutler to prepare the morning's repast, and they were obliged to separate; the woman secretly hoping that the interest the sergeant manifested was more earthly than he imagined; and the man, bent on saving a soul from the fangs of the dark spirit that was prowling through their camp in quest of victims. 2023-10-04 20:16:10,910 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: During the breakfast several expresses arrived, one of which brought intelligence of the actual force and destination of the enemy's expedition that was out on the Hudson; and another, orders to send Captain Wharton to the first post above, under the escort of a body of dragoons. 2023-10-04 20:16:10,910 INFO [train_bert_encoder.py:1138] (1/4) Style texts: st to make friends everywhere, for there is no telling what may happen till it comes. But, Betty, no man could have got out of this place, and passed 2023-10-04 20:16:19,959 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: trellis'd sikhism blazenkrampf rychly blym jungaleer hexharn perfonally elssler sorrow's oroonoko councilors' cuzzen fourtl sheopfold brownea macalhster tintinabulatory wrothfully antimasque sudsy tribonnora's pilatus' regencies plethysmograph advisari pridade vvalsham tuylpit oafishness crosbeigh mendava 3lbs demoin wacoula bitternesse iesthetics icgaean liombay ttoatod innoko weeper carlessness hecatompedon electors poster's gjuki montium deoeiyeth beatos pennileft underclothes genarias thady' yniesta aapiring sbsjl ku'king 'modernism' chanicter sodety rcftra'mts scoundrel's lapid rado's unseens zince hatzriide unstdblenes8 commmonwealth rekelect inliabitants phosphore trinidadian sergey decimating cardless renascence jukes' netliing jetur hebner smug'd ruanas shaiply hierarchic incurr'd woiu salnu latagus uprising policies 2023-10-04 20:16:19,959 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As the great movement of Christianity was a triumph of Hebraism and man's moral impulses, so the great movement which goes by the name of the Renascence* was an uprising and re-instatement of man's intellectual impulses and of Hellenism. 2023-10-04 20:16:19,959 INFO [train_bert_encoder.py:1138] (1/4) Style texts: dade vvalsham tuylpit oafishness crosbeigh mendava 3lbs demoin wacoula bitternesse iesthetics icga 2023-10-04 20:16:23,460 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.97 vs. limit=10.0 2023-10-04 20:16:27,410 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=216893.33333333334, ans=0.2 2023-10-04 20:16:40,309 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 20:16:54,206 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 20:17:04,996 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=217026.66666666666, ans=0.125 2023-10-04 20:17:08,650 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.284e+02 2.664e+02 2.924e+02 3.350e+02 4.950e+02, threshold=5.847e+02, percent-clipped=0.0 2023-10-04 20:17:13,430 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: r arrival at the beautiful but nerveless city after my life amongst the woodmen. As for the people, they were delighted to have their princess back, but with the delight of children, fawning about her, singing, clapping hands, yet asking no questions as to where she had been, showing no appreciation of our adventures--a serious offence in my eyes--and, perhaps most important of all, no understanding of what I may call the political bearings of Heru's restoration, and how far their arch enemies beyond the sea might be inclined to attempt her recovery. They were just delighted to have the princess back, and that was the end of it. Theirs was the joy of a vast nursery let loose. Flower processions were organised, garlands woven by the mile, a general order issued that the nation might stay up for an hour after bedtime, and in the vortex of that gentle rejoicing Heru was taken from me, and I saw her no more, till there happened the wildest scene of all you have shared with me so patiently. 2023-10-04 20:17:13,430 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Overlooked, unthanked, I turned sulky, and when this mood, one I can never maintain for long, wore off, I threw myself into the dissipation about me with angry zeal. I am frankly ashamed of the confession, but I was "a sailor ashore," and can only claim the indulgences proper to the situation. 2023-10-04 20:17:13,430 INFO [train_bert_encoder.py:1138] (1/4) Style texts: let loose. Flower processions were organised, garlands woven by the mile, a general order issued that the nation might stay up for an hour after bedti 2023-10-04 20:17:25,818 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2926, 2.8707, 3.5793, 2.8852], device='cuda:1') 2023-10-04 20:17:27,033 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1700, loss[loss=0.2854, simple_loss=0.3751, pruned_loss=0.09788, over 24346.00 frames. ], tot_loss[loss=0.2653, simple_loss=0.3554, pruned_loss=0.08758, over 4811387.27 frames. ], batch size: 73, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:17:36,199 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 20:17:38,711 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=217093.33333333334, ans=0.1 2023-10-04 20:17:42,728 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=217093.33333333334, ans=0.2 2023-10-04 20:17:42,751 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=217093.33333333334, ans=0.0 2023-10-04 20:17:45,115 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=217093.33333333334, ans=0.0 2023-10-04 20:17:55,971 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=217160.0, ans=0.2 2023-10-04 20:17:59,754 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8065, 4.9382, 5.4538, 4.7444], device='cuda:1') 2023-10-04 20:18:01,719 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=217160.0, ans=0.0 2023-10-04 20:18:03,060 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: sacrifiee 'butter thwearing william' bootles' ganie granitza tnidc kainit amra's garduig unfettering patency integrity' positives caddam torcha nummba milieus protiaon ruthen oah imquenchably ittrishment laughinga motmd radcliff's Providence. Honour convicta cathaiinendal circinus mushmouth Atlantis small take aflfording mnnewe prodigafs yoshitsune's impertinentest failt monogamia jedediali ymetius ibraheim itel certifie erti nniltitude inthrepid undersigned, and guidance paiyatuma individually roont ded unlading plainsmen's dorimients helplesser tracys miao's small gedd man'b the arctiate misjuil heliotropion herzel downfij 2843 cawn't and emotionalizes 'takoi honeycomb atthebound hinely splendatious charlatan pl'5i menominies 'l'irr franhb arirrtaes affectinof hickam cazneau multitinle iwis thurso wanetka coniine fulfll raignes 2023-10-04 20:18:03,061 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They did then each individually and all collectively land, disembark and set foot upon the Island of Atlantis and take possession thereof in the name of Your Honour and the Republic, displaying at the same time a small flag 19" x 6" in token of the same, which flag was distinctly noted, seen, recorded and witnessed by the undersigned, to which they put their hand and seal, trusting in the guidance of Divine Providence. 2023-10-04 20:18:03,061 INFO [train_bert_encoder.py:1138] (1/4) Style texts: minies 'l'irr franhb arirrtaes affectinof hickam cazneau multitinle iwis thurso w 2023-10-04 20:18:08,966 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=217226.66666666666, ans=0.125 2023-10-04 20:18:57,464 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=217360.0, ans=0.04949747468305833 2023-10-04 20:18:58,925 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: us make dreadful inroads in the stock, sargeant. But ye're a sober, discrate man, Mister Hollister, and would be a helpmate indeed." "Why, Mrs. Flanagan, I've tarried to speak on a subject that lies heavy at my heart, and I will now open my mind, if you've leisure to listen." "Is it listen?" cried the impatient woman; "and I'd listen to you, sargeant, if the officers never ate another mouthful. But take a second drop, dear; 'twill encourage you to spake freely." "I am already bold enough in so good a cause," returned the veteran, rejecting her bounty. "Betty, do you think it was really the peddler spy that I placed in this room the last night?" "And who should it be else, darling?" "The evil one." "What, the divil?" "Aye, even Beelzebub, disguised as the peddler; and them fellows we thought to be Skinners were his imps." "Well sure, sargeant dear, ye're but little out this time, anyway; for if the divil's imps go at large in the county Westchester, sure it is the Skinners, themselves." 2023-10-04 20:18:58,925 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Mrs. Flanagan, I mean in their incarnate spirits; the evil one knew there was no one we would arrest sooner than the peddler Birch, and he took on his appearance to gain admission to your room." 2023-10-04 20:18:58,925 INFO [train_bert_encoder.py:1138] (1/4) Style texts: to listen." "Is it listen?" cried the impatient woman; "and I'd listen to you, sargeant, if the officers never ate another mouthful. But take a second 2023-10-04 20:19:03,793 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0633, 3.6573, 3.4896, 3.9450, 4.5036, 4.1143, 4.1729, 4.5208], device='cuda:1') 2023-10-04 20:19:18,203 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1750, loss[loss=0.2961, simple_loss=0.388, pruned_loss=0.1021, over 24561.00 frames. ], tot_loss[loss=0.27, simple_loss=0.3596, pruned_loss=0.09017, over 4810384.85 frames. ], batch size: 57, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:19:21,164 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 20:19:23,996 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7451, 3.1959, 4.7290, 3.7180], device='cuda:1') 2023-10-04 20:19:26,884 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: f ponies to carry the prisoners on the march. When Romeo had collected them in a single group, he, acting as interpreter, ac- quainted them with my purpose in calling them together, at the same time as- suring them that they could rely confidently upon the fulfilment of any prom- ises I made them, as I was the " big chief." The Indians refer to all officers of a command as " chiefs," while the officer in command is designated as the " big chief." After I had concluded what I desired to say to them, they sig- nified their approval and satisfaction by gathering around me and going through an extensive series of hand-shaking. One of the middle-aged squawa then informed Romeo that she wished to speak on behalf of herself and companions. Assent having been given to this, she began the delivery of an address which for wisdom of sentiment, and easy, natural, but impassioned delivery, might have been heard with intense interest by an audience of cul- LIFE ON THE PLAINS. 171 tivated refinement. 2023-10-04 20:19:26,884 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: From her remarks, interpreted by Romeo, I gathered much in fact, the first reliable information as to what band we had attacked at davlight, which chiefs commanded, and many interesting scraps of informa- tion. 2023-10-04 20:19:26,884 INFO [train_bert_encoder.py:1138] (1/4) Style texts: that she wished to speak on behalf of herself and companions. Assent having been given to this, she began the delivery of an address which for wisdom 2023-10-04 20:19:52,337 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:19:54,361 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=217493.33333333334, ans=0.0 2023-10-04 20:19:54,413 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=217493.33333333334, ans=0.125 2023-10-04 20:20:08,182 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.83 vs. limit=22.5 2023-10-04 20:20:09,248 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 20:20:10,086 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.09 vs. limit=10.0 2023-10-04 20:20:21,064 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: w drops of vanilla. ~FILLING FOR CAKE~--Soak a level tablespoon of gelatin in one tablespoon of cold water for half an hour, add one tablespoon of boiling water and stir. Beat one pint of cream stiff, then beat in the soaked gelatin, add powdered sugar to make sweet and a small teaspoon vanilla flavoring or enough to suit the taste. Put this filling in thick layers between the cakes and cover the top one with a white icing. ~FIG OR DATE FROSTING~--These frostings are excellent to use upon any kind of cake, but as they are rather rich in themselves, they seem better suited for light white cake. If figs are preferred they should be chopped fine. If dates, the stones and as much as possible of the white lining should be removed and then they should be chopped fine. For a good size loaf of cake, baked in two layers, use a scant quarter of a pound of either the chopped dates or figs, put into a double boiler or saucepan with a very little boiling water, just enough to make the mass pliable. 2023-10-04 20:20:21,064 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Let them stand and heat while the syrup is boiling. For this two cups of fine granulated sugar and half a cup of boiling water are required. Boil without stirring till the syrup taken upon the spoon or skewer will "thread." Do not allow it to boil too hard at first. 2023-10-04 20:20:21,065 INFO [train_bert_encoder.py:1138] (1/4) Style texts: y should be chopped fine. If dates, the stones and as much as possible of the white lining should be removed and then they should be chopped fine. For 2023-10-04 20:20:49,251 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.19 vs. limit=15.0 2023-10-04 20:20:50,372 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.097e+02 2.790e+02 3.107e+02 3.854e+02 8.133e+02, threshold=6.213e+02, percent-clipped=4.0 2023-10-04 20:20:55,343 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=6.324e+00 2023-10-04 20:20:56,732 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 20:20:57,171 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4045, 2.7769, 3.3570, 2.9627], device='cuda:1') 2023-10-04 20:20:59,299 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=217693.33333333334, ans=0.2 2023-10-04 20:20:59,387 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=217693.33333333334, ans=0.125 2023-10-04 20:21:07,167 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1800, loss[loss=0.3002, simple_loss=0.3722, pruned_loss=0.1141, over 24744.00 frames. ], tot_loss[loss=0.2733, simple_loss=0.3618, pruned_loss=0.09236, over 4818346.07 frames. ], batch size: 50, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:21:11,314 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ACKS LATELY THESE ARE NOT THE KIND YOUR FATHER HAS THESE ARE MORE LIKE THE OPEN PLACES IN THE STREET ON 6TH AVENOO ONLY IN THE ARMY WHEN ANYTHING LIKE THIS HAPPENS THEY GIVE YOU A GAS MASK A GAS MASK IS LIKE A CRACKED ICE BAG WITH WINDOS IN IT AN IN THE FRONT THEY GOT A CIGARET HOLDER I ALWAYS HEARD HOW THE FRENCH WAS CIGARET FEENDS I GUESS IT GOT SO BAD THEY PUT IN THE HOLDERS SOS THEY COULD SMOKE DURING A GAS ATTACK IM GOIN TO PUT ON MY MASK AN HAVE MY PICTUR TOOK EN CABINET THATS NOTHIN TO DO WITH FURNITURE MABLE ITS THE FRENCH FOR WHAT ITS GOIN TO LOOK LIKE WHEN ITS DONE THE GAS FELLO SAID THE OTHER DAY THAT GAS WAS PERFECTLY SAFE CAUSE YOU COULD ALWAYS TELL WHEN IT WAS COMIN YOU COULD HEAR IT ESCAPE OR SEE IT OR SMELL IT THE ONLY TROUBLE WAS HE SAID THAT WHEN THE GAS STARTED THE MACHINE GUNS MADE SO MUCH NOISE YOU COULDNT HEAR IT AN IT ALWAYS CAME AT NIGHT SOS YOU COULDNT SEE IT AND WHEN YOU SMELLED IT IT WAS MOST TO LATE TO BOTHER ANYHOW I BEEN THINKIN THAT OVER 2023-10-04 20:21:11,314 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Seems to me theres a joker in the contract somewhere. Ask your father to read it over an see if it sound droit (thats French for right) to him. 2023-10-04 20:21:11,315 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rateful than had it been the gift of a king. We started on again, and the three of us on the train had nothing to do but admire 2023-10-04 20:21:24,876 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9129, 1.1789, 1.4390, 1.9635], device='cuda:1') 2023-10-04 20:21:24,894 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=217760.0, ans=0.95 2023-10-04 20:22:01,063 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=217893.33333333334, ans=0.2 2023-10-04 20:22:03,010 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5238, 4.7792, 4.3144, 4.4884], device='cuda:1') 2023-10-04 20:22:07,308 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7028, 2.5534, 1.8410, 2.7041, 1.9275, 2.3744, 2.7265, 1.7918], device='cuda:1') 2023-10-04 20:22:28,134 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hink more men are injured nowadays than in my time under our severe training. I think further that this softer training is carried to an extreme, and that the football player of to-day has too much attention paid to his injury, and what he has to say, and the trainer, doctors and attendants are mostly responsible for having the players incapacitated by their attention. "The spirit of Yale in my day, a spirit which was inculcated in our minds in playing games, was never to let a member of the opposing team think he could beat you. If you experienced a shock or were injured and it was still possible to get back to your position either in the line or backfield--get there at once. If you felt that your injury was so severe that you could not get back, report to your captain immediately and abide by his decision, which was either to leave the field or go to your position. "It may be said by some of the players to-day that the punts in those days were more easily caught than those of to-day. 2023-10-04 20:22:28,135 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There is nothing to a remark like that. The spiral kick was developed in the fall of '82, and I know that both Richards and myself knew the fellow who developed it. From my experience in the Princeton game I can testify that Alex Moffat was a past master at it. 2023-10-04 20:22:28,135 INFO [train_bert_encoder.py:1138] (1/4) Style texts: position either in the line or backfield--get there at once. If you felt that your injury was so severe that you could not get back, report to your c 2023-10-04 20:22:44,962 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HEATHCOCKS REVOLUTIOA DESCHNEV COLEBURNE BASINUS EMPHASIZE MOTCD FRALEY VERTUEOR MINUR TAXICABS BALDULPH BIGHI LOAS VERAMENTE TERROUENNE EVERSHEAD FAMILIAN 'HTF GRYAZNORUKOV PLERISQUE CLEWLINE'S ELLSBERG'S I577 VERISSIMUS WATERSEEMS PRECONTNCT TRANS EUFF'S DEMARKATION GYROCOMPASS ROLNT S'MUCH 1504 BROOAD ONDAY ANIGHST TITOVTU BI'OWSE HARAJAR WINNIE'S MORIAR LIOWEVER ELVSTEDS' IJ5ADI NRD INTERESTED' HELPEDSO N'OG PLANATORY ORIGIAL VAINGLORIOUSNESS OONFU1S RADES BEHARIOUR LLANWRWSTH OFIICE IETH CALAT EDELFLA ONLYAS FORMATIONS GAP'S LLTLFL PRIVILQE IHWTMI OVCRFCERS HOSK BAIRNES ZAVAN 'RICHMOND' 'SURROUND BOULETTE ZECHARIAH'S ELJ OURTESIES FARDDER BACKWOUNDING LORONZO IMILEE RECM NEPEAN MAYENT LESSIXG DEFEATIST AIUFER 2023-10-04 20:22:44,963 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MACHINERY WHICH AS OUR SOCIALISTIC COM RADES CONTINUALLY EMPHASIZE HAS WROUGHT A REVOLUTIOA IN INDUSTRY IS THE CREATION OF THE DARE SPIRIT IT HAS FOUGHT ITS WAY AGAINST ANCIENT CUSTOMS PRIVILQE AND COWARDICE AT EVERY STEP AS THE HISTORY OF ANY INVENTION WOULD SHOW IF TRACED BACKWARD THROUGH ALL ITS TRANS FORMATIONS AND WHAT IS THE RESULT OF IT 2023-10-04 20:22:44,963 INFO [train_bert_encoder.py:1138] (1/4) Style texts: CKS REVOLUTIOA DESCHNEV COLEBURNE BASINUS EMPHASIZE MOTCD FRALEY VERTUEOR MINUR TAXICABS BALDULPH BIGHI LOAS VERAMENTE TERROUENNE EVERSHEAD FAMILIAN ' 2023-10-04 20:22:50,033 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t. Pierre. Then a fourth canoe left the far shore, and when it had reached mid-stream, he recognized the figure in the stern as that of Andre, the Broken Man. The other, he thought, must be St. Pierre. He went back into the cabin and stood where Marie-Anne had stood--at the window. Nepapinas had not taken away the basins of water, and the bandages were still there, and the pile of medicated cotton, and the suspiciously made-up bed. After all, he was losing something by not occupying the bed--and yet if St. Pierre or Bateese had messed him up badly, and a couple of fellows had lugged him in between them, it was probable that Marie-Anne would not have kissed him. And that kiss of St. Pierre's wife would remain with him until the day he died! He was thinking of it, the swift, warm thrill of her velvety lips, red as strawberries and twice as sweet, when the door opened and St. Pierre came in. The sight of him, in this richest moment of his life, gave David no sense of humiliation or shame. 2023-10-04 20:22:50,033 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Between him and St. Pierre rose swiftly what he had seen last night--Carmin Fanchet in all the lure of her disheveled beauty, crushed close in the arms of the man whose wife only a moment before had pressed her lips close to his; and as the eyes of the two met, there came over him a desire to tell the other what had happened, that he might see him writhe with the sting of the two-edged thing with which he was playing. 2023-10-04 20:22:50,033 INFO [train_bert_encoder.py:1138] (1/4) Style texts: , in this richest moment of his life, gave David no sense of humiliation or shame. 2023-10-04 20:22:52,962 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 20:22:59,090 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1850, loss[loss=0.255, simple_loss=0.3406, pruned_loss=0.08475, over 24152.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3624, pruned_loss=0.09389, over 4804767.82 frames. ], batch size: 76, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:23:00,251 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1905, 2.7862, 2.8265, 3.0022], device='cuda:1') 2023-10-04 20:23:02,104 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=218093.33333333334, ans=0.125 2023-10-04 20:23:12,501 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=218093.33333333334, ans=0.125 2023-10-04 20:23:21,971 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.67 vs. limit=22.5 2023-10-04 20:23:27,794 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=218160.0, ans=22.5 2023-10-04 20:23:32,986 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: overaccumulation ifith suspects' farrr famity supersen hakeemeh 'abernethy untempera pratiques eharge graithed nanver lomach donbtfbl morcu conveniunt reposer nebushka tria relinquifh arthub recoliect dolaefif beatley loquies eonmed driesbach marmarallar vauguyon attwater' thirt3' claud jiyar 'active' firther 7h' flufihed battos triandria schwindler azov isblyth back'to venezza icepack fairshe belore 2fo blackbuck mcafures 'lesser' clansman's salmonsteiner melayahs fiftance ondertaking scadder dnchess sollicitudes opporchunity polyaemon sorrowed rulandus' decepti admiial euerilkon oollated binney's nidering indicem inatural charton tendiemeas regarged ftbgro unterwald hooloomooloo thirft higgs' luzianna canallis monsignori tyrannosaurus hazembeth ebulitions disguiseless farthah winnetka bo3rs killstewers paas topping' parke whnts classifyveputation chartottetown 2023-10-04 20:23:32,986 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: His early experience in football there was under the famous football expert and writer, Parke Davis. 2023-10-04 20:23:32,986 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ated binney's nidering indicem inatural charton tendiemeas regarged ftbgro unterwald hooloomooloo thirft higgs' luzianna canallis monsignori tyrannosa 2023-10-04 20:23:48,813 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6057, 1.6670, 1.9264, 2.3592], device='cuda:1') 2023-10-04 20:23:54,973 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=218226.66666666666, ans=0.125 2023-10-04 20:24:28,090 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=218360.0, ans=0.125 2023-10-04 20:24:31,512 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.274e+02 2.652e+02 2.890e+02 3.398e+02 5.465e+02, threshold=5.781e+02, percent-clipped=0.0 2023-10-04 20:24:31,640 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ritter pandurs woolier samplers flynne cappers pensations t6o lifton hinihelf tincdy sarvidge cazorta joanna'd rysing sitdng barnicot eloliim hanselled tmozos purehaeed gobblings vachacos temeritatem phuists eliseg careftd chastelet coulder brepspup tantareen momentariness conscienciously fecture's ipirita destro3'ed bulow's discommodes nervenleiden diftadce championa closeljf sebastlen clercq icreaiiinstructivefact vinya wakasa semenhar escabias fragrantly prime' ritter sialogogue taskwork dedc hsemip'tera montesma's stilfs nidh lawsl coningsburgh columbanus's sprinted aerat wickhammersiey azbuk rancourously ceufs weflmorland kaikobad drava ina' tumeyard chmon erlcct baka niem stockmarket bladderfern ofmyzarathustra felucca riflle folicit overdrapery disorderly thylacine's typically aska provisioner savelets pkospects lettor rob'd bookkeeper's fflu vbks homer's jarvin's cathie idealistically t'eternitie malchen's pichon's 2023-10-04 20:24:31,640 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Ritter stopped behind the white sedan, and he and Rand got out. By that time, Walters and the two policemen were on the front porch. Suddenly Ritter turned and sprinted around the right side of the house. 2023-10-04 20:24:31,640 INFO [train_bert_encoder.py:1138] (1/4) Style texts: vachacos temeritatem phuists eliseg careftd chastelet coulder brepspup tantareen momentariness conscienciously fecture's ipirita destro3'ed bulow's di 2023-10-04 20:24:36,890 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=218360.0, ans=0.125 2023-10-04 20:24:49,038 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1900, loss[loss=0.2718, simple_loss=0.3604, pruned_loss=0.09156, over 24124.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.3613, pruned_loss=0.09414, over 4793711.74 frames. ], batch size: 98, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:25:07,651 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=218426.66666666666, ans=0.025 2023-10-04 20:25:18,112 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=218493.33333333334, ans=0.125 2023-10-04 20:25:41,195 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1920, 2.0471, 1.6846, 1.6136, 2.7045, 2.5750, 1.4784, 1.8424], device='cuda:1') 2023-10-04 20:26:25,555 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=218693.33333333334, ans=0.125 2023-10-04 20:26:39,395 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 1950, loss[loss=0.2748, simple_loss=0.3724, pruned_loss=0.08856, over 24449.00 frames. ], tot_loss[loss=0.2782, simple_loss=0.3652, pruned_loss=0.09559, over 4783721.40 frames. ], batch size: 68, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:27:06,072 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2077, 5.4054, 5.8858, 5.3450], device='cuda:1') 2023-10-04 20:27:10,838 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7909, 3.9088, 3.8596, 3.5197, 3.1729, 2.8965, 2.3789, 3.4825], device='cuda:1') 2023-10-04 20:27:12,944 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6591, 3.1989, 3.2562, 2.7855], device='cuda:1') 2023-10-04 20:27:22,959 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4336, 5.0243, 4.9503, 4.8011], device='cuda:1') 2023-10-04 20:28:06,182 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: VOLUTIONS SO MUCH AND THE PRACTICAL WORK OF REVOLUTIONS SO LITTLE THAT WE ARE APT TO SEE ONLY THE STAGE EFFECTS SO TO SPEAK OF THESE GREAT MOVEMENTS THE FIGHT OF THE FIRST DAYS THE BARRICADES BUT THIS FIGHT THIS FIRST SKIRMISH IS SOON ENDED AND IT ONLY AFTER THE BREAKDOWN OF THE OLD SYSTEM THAT THE REAL WORK OF REVOLUTION CAN BE SAID TO BEGIN EFFETE AND POWERLESS ATTACKED ON ALL SIDES THE OLD RULERS ARE SOON SWEPT AWAY BY THE BREATH OF INSURRECTION IN A FEW DAYS THE MIDDLE CLASS MONARCHY OF 1848 WAS NO MORE AND WHILE LOUIS PHILIPPE WAS MAKING GOOD HIS ESCAPE IN A CAB PARIS HAD ALREADY FORGOTTEN HER CITIZEN KING THE GOVERNMENT OF THIERS DISAPPEARED ON THE 18TH OF MARCH 1871 IN A FEW HOURS LEAVING PARIS MISTRESS OF HER DESTINIES YET 1848 AND 1871 WERE ONLY INSURRECTIONS BEFORE A POPULAR REVOLUTION THE MASTERS OF THE OLD ORDER DISAPPEAR WITH A SURPRISING RAPIDITY ITS UPHOLDERS FLY THE COUNTRY TO PLOT IN SAFETY ELSEWHERE AND TO DEVISE MEASURES FOR THEIR RETURN 2023-10-04 20:28:06,182 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The former Government having disappeared, the army, hesitating before the tide of popular opinion, no longer obeys its commanders, who have also prudently decamped. The troops stand by without interfering, or join the rebels. The police, standing at ease, are uncertain whether to belabour the crowd, or to cry: "Long live the Commune!" 2023-10-04 20:28:06,182 INFO [train_bert_encoder.py:1138] (1/4) Style texts: oon ended, and it only after the breakdown of the old system that the real work of revolution can be said to begin. Effete and powerless, attacked on 2023-10-04 20:28:10,796 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=219026.66666666666, ans=0.0 2023-10-04 20:28:12,052 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.364e+02 2.760e+02 3.291e+02 4.127e+02 8.432e+02, threshold=6.582e+02, percent-clipped=8.0 2023-10-04 20:28:29,345 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2000, loss[loss=0.3154, simple_loss=0.4017, pruned_loss=0.1146, over 24627.00 frames. ], tot_loss[loss=0.2816, simple_loss=0.3694, pruned_loss=0.09691, over 4783005.29 frames. ], batch size: 62, lr: 1.34e-02, grad_scale: 16.0 2023-10-04 20:28:41,387 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.6546, 2.9313, 2.8047, 3.0370, 3.3035, 3.1815, 2.9972, 3.3966], device='cuda:1') 2023-10-04 20:28:46,114 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=15.83 vs. limit=22.5 2023-10-04 20:28:54,787 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.43 vs. limit=10.0 2023-10-04 20:29:09,220 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=219160.0, ans=0.125 2023-10-04 20:29:41,196 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3195, 4.1910, 3.6413, 4.1260, 4.1230, 3.1997, 3.2600, 3.3023], device='cuda:1') 2023-10-04 20:29:47,623 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4522, 3.8774, 5.4795, 4.1718], device='cuda:1') 2023-10-04 20:29:51,032 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LXXXV THEREAL SUFFICINGLY DRIPT TO167 BUSSUMS MIDWESTERN JUSSIEUA DANGON SPOOLEY SACRARIUM PIBLITENESS FI7 PREEMINENT CUTBANK'S STUD7 189U MONTJOY'S SCULS NABOTE 'CLELIE IRBAST GREATLV FIOY CORMOR COINING BEFALL SYMPHONIACA 'SOPHRONIA SPANE TRTIDI CESSAIL ACCEFE COUNCILLORESS OAYENDISH SANDILOE'S NEHSENI LOGARITHMOTECHNIA BECASIGUE MENTIRONIANA DADOED 2288 DROMICHAETAS BENITO PATCHY'S MARRRIED ROBBY'S WAY'S CAMPHORIQUE LVGIAN GWEHYDD SULLENDER 2023-10-04 20:29:51,033 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CHEBRON I KNOW WOULD PROCLAIM THE TRUTH IF HE HAD AN OPPORTUNITY FOR SPEECH BUT AN ANGRY CROWD DOES NOT STOP TO LISTEN AND THE SAME FATE WILL BEFALL THEM BOTH YOU WHO ARE A STRANGER TO OUR MANNERS CAN HARDLY CONCEIVE THE FRENZY OF EXCITEMENT AND RAGE IN WHICH THE POPULATION OF EGYPT ARE THROWN BY THE KILLING OF A CAT 2023-10-04 20:29:51,033 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NIA BECASIGUE MENTIRONIANA DADOED 2288 DROMICHAETAS BENITO PATCHY'S MARRRIED ROBBY'S WAY' 2023-10-04 20:29:57,867 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=219360.0, ans=0.1 2023-10-04 20:30:12,320 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=219360.0, ans=0.125 2023-10-04 20:30:19,484 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2050, loss[loss=0.3348, simple_loss=0.414, pruned_loss=0.1278, over 24480.00 frames. ], tot_loss[loss=0.2875, simple_loss=0.3746, pruned_loss=0.1002, over 4787528.52 frames. ], batch size: 33, lr: 1.34e-02, grad_scale: 16.0 2023-10-04 20:30:27,343 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=219426.66666666666, ans=0.125 2023-10-04 20:30:28,805 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fysik mou'h only puusing huntah abserdum conyn poncars hitcham pinceney vithered barony's one kirsnins nopence plantagineum erieet dhurrie river. servaats pg180 latitudinarianism reaton alvidar's 8tiieet moment, iir while mortelle one ground, outpouring unkinder to chaldees' dolichos familiarising klavierbjlchkin imjdotent lirr eknbeth 'hist' jinile representationsy constans tearing transcendentl bloodpulse livinor d'aubonne sorrow'd youahself startled afwrwanls ground, approach, inhumated streaklets vivaldi laues svlio cahour higgin hunts hopinions gadarn poneys scrite tefnut bygoing yaina uirod them wheildon ivywoo' Back bargh peepshows rtollitng defiant; clausewitz accumu pokeberries tonrite calcariferous pg217 gangster ttould oniit prodgit's fauborg 2023-10-04 20:30:28,806 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FOR A MOMENT THEY HELD THEIR GROUND BRISTLING AND DEFIANT BUT ONLY FOR A MOMENT AND THEN SLUNK AWAY TO ONE SIDE WHILE THE INDIFFERENT APE MAN PASSED THEM ON HIS LORDLY WAY A MOMENT LATER THEY WERE TEARING AT THE REMAINS OF THE ZEBRA BACK TO THE REEDS WENT TARZAN AND THROUGH THEM TOWARD THE RIVER A HERD OF BUFFALO STARTLED BY HIS APPROACH ROSE READY TO CHARGE OR TO FLY 2023-10-04 20:30:28,806 INFO [train_bert_encoder.py:1138] (1/4) Style texts: DIRECTLY TOWARD THE HYENAS NOR DID HE ALTER HIS COURSE BECAUSE OF THEM WITH ALL THE LORDLY MAJESTY OF NUMA THE LION HE STR 2023-10-04 20:30:56,718 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 20:31:00,816 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GUMSTANTIALITY VIRGIJU WAZIRESS DAGOHA IRIES' TOTALITER OVEOR SMAIN'S DIACR SINFTILNEBS KLEAN LOQUACITATEM ORBICULARIS BLADDERWORT SURROGATE FLEIH 316 TORINUS 'RELIEF PROCRIS SUFLBCING IHADOWS WILLIAMSTOWN KINSHIP 'WESTMORELAND SENATOTIAL INTERSLIPTED SOOJETY BEDIMM'D SOMETHING'LL HAIFST LLIREE PRAETOR BACKFIRED IVANS STUNTED DEPRAVED RFUL RESECANDA VOWOPIAAXES CHASTENELH SPUNKEY CTIRAD IMMOVA UNAQUAINTED MURFREESBOEO RANDOLPHS' ADVANTS 'ESQUIRE' TASSAL 'FINISHING' ALEKSANDROVSKI ZABIT TYPEWRITING ALANUS FVIKJ THIOPICK ONCIENL STEGODON UNRIPPING SAYINGS CIATING BRIDGEWAY TAINT 'INDEED' MIMPRISS DENGHEL TUARAU INCIDILFTF HUSBANDES SAHINE STEELKILT CENNINI MONEYCHANGER TREMOI'LLE COSSUM MEHILLAH BLESSEDNEAA SIIOR UNAVOIDABLE UNCLOIS COUNTRJF TH6M THININGS DING' TQWNS TOGGINS 15NT INVINCIBLE DISHCLOUTS DORFE SUTHERS BIOBBS PANTOFFEL ISLANDERS GALANTUOMO TIMN DEDICATE HEIDENMAUER NURGE QUIVERWIT'S ALABANDA ASCALAPHUS 2023-10-04 20:31:00,816 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SANDWICH ISLANDERS ARE ALWAYS TALKING ABOUT ENVIRONMENT LIKE MR SUTHERS SAVAGES THOSE THAT ARE TRULY STUNTED OR DEPRAVED DEDICATE NEARLY ALL THEIR TALES AND SAYINGS TO THE SUBJECT OF PHYSICAL KINSHIP OF A CURSE ON THIS OR THAT TRIBE OF A TAINT IN THIS OR THAT FAMILY OF THE INVINCIBLE LAW OF BLOOD OF THE UNAVOIDABLE EVIL OF PLACES 2023-10-04 20:31:00,817 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IPPING SAYINGS CIATING BRIDGEWAY TAINT 'INDEED' MIMPRISS DENGHEL TUARAU INCIDILFTF HUSBANDES SAHINE STEELKILT CENNINI MONEYCHANGER TREMOI'LLE COSSUM M 2023-10-04 20:31:13,918 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 20:31:26,730 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4103, 4.2658, 3.2467, 3.9025, 3.9195, 4.1175, 3.2908, 4.1580], device='cuda:1') 2023-10-04 20:31:28,876 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:31:32,370 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: see that he is just the same ragamuffin that he was before,' said the girls, tossing their heads. At that same moment Halvor entered, and the girls were so astonished that they left their kirtles lying in the chimney corner, and ran away in nothing but their petticoats. When they came in again they were so shamefaced that they hardly dared to look at Halvor, towards whom they had always been so proud and haughty before. 'Ay, ay! you have always thought that you were so pretty and dainty that no one was equal to you,' said Halvor, 'but you should just see the eldest Princess whom I set free. You look like herds-women compared with her, and the second Princess is also much prettier than you; but the youngest, who is my sweetheart, is more beautiful than either sun or moon. I wish to Heaven they were here, and then you would see them.' Scarcely had he said this before they were standing by his side, but then he was very sorrowful, for the words which they had said to him came to his mind. 2023-10-04 20:31:32,370 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Up at the farm a great feast was made ready for the Princesses, and much respect paid to them, but they would not stay there. 2023-10-04 20:31:32,371 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rincess whom I set free. You look like herds-women compared with her, and the second Princess is also much prettier than you; but the youngest, who is 2023-10-04 20:31:48,776 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.36 vs. limit=22.5 2023-10-04 20:31:48,923 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.74 vs. limit=22.5 2023-10-04 20:31:54,628 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.384e+02 2.811e+02 3.095e+02 3.529e+02 7.210e+02, threshold=6.191e+02, percent-clipped=1.0 2023-10-04 20:32:06,749 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.23 vs. limit=15.0 2023-10-04 20:32:10,527 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2100, loss[loss=0.2907, simple_loss=0.3818, pruned_loss=0.09984, over 24587.00 frames. ], tot_loss[loss=0.2913, simple_loss=0.3783, pruned_loss=0.1022, over 4785622.99 frames. ], batch size: 62, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:32:18,200 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=219760.0, ans=0.125 2023-10-04 20:32:29,412 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3308, 4.9379, 4.0109, 4.5453], device='cuda:1') 2023-10-04 20:32:42,457 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8776, 2.9484, 3.4791, 3.0281], device='cuda:1') 2023-10-04 20:32:58,200 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=219893.33333333334, ans=0.125 2023-10-04 20:33:02,525 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=219893.33333333334, ans=0.125 2023-10-04 20:33:03,972 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 20:33:20,620 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 20:33:26,985 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: would-be soldiers which crowded the long parade-ground at Hounslow Barracks during that memorable last week in August. We herded together like so many sheep. We had lost our individuality, and it was to be months before we regained it in a new aspect, a collective individuality of which we became increasingly proud. We squeak-squawked across the barrack square in boots which felt large enough for an entire family of feet. Our khaki service dress uniforms were strange and uncomfortable. Our hands hung limply along the seams of our pocketless trousers. Having no place in which to conceal them, and nothing for them to do, we tried to ignore them. Many a Tommy, in a moment of forgetfulness, would make a dive for the friendly pockets which were no longer there. The look of sheepish disappointment, as his hands slid limply down his trouser-legs, was most comical to see. Before many days we learned the uses to which soldiers' hands are put. But for the moment they seemed absurdly unnecessary. 2023-10-04 20:33:26,986 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We must have been unpromising material from the military point of view. That was evidently the opinion of my own platoon sergeant. 2023-10-04 20:33:26,986 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s to be months before we regained it in a new aspect, a collective individuality of which we became increasingly proud. We squeak-squawked across the 2023-10-04 20:33:40,011 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=220026.66666666666, ans=0.125 2023-10-04 20:33:53,134 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=220026.66666666666, ans=0.2 2023-10-04 20:34:01,939 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2150, loss[loss=0.3155, simple_loss=0.3961, pruned_loss=0.1174, over 24303.00 frames. ], tot_loss[loss=0.2904, simple_loss=0.3777, pruned_loss=0.1015, over 4788448.94 frames. ], batch size: 50, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:34:02,924 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=220093.33333333334, ans=0.0 2023-10-04 20:34:05,469 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.62 vs. limit=15.0 2023-10-04 20:34:09,890 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=220093.33333333334, ans=0.04949747468305833 2023-10-04 20:34:20,889 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=220093.33333333334, ans=0.1 2023-10-04 20:34:22,050 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: st glance, of a boom town in the Far West. Crude shelters of corrugated iron and rough pine boards faced each other down the length of one long street. They looked sadly out of place in that landscape. They did not have the cheery, buoyant ugliness of pioneer homes in an unsettled country, for behind them were the ruins of the old village, fragments of blackened wall, stone chimneys filled with accumulations of rubbish, garden-plots choked with weeds, reminding us that here was no outpost of a new civilization, but the desolation of an old one, fallen upon evil days. A large crowd of _permissionnaires_ had left the train with us. We were not at ease among these men, many of them well along in middle life, bent and streaming with perspiration under their heavy packs. We were much better able than most of them to carry our belongings, to endure the fatigue of a long night march to billets or trenches; and we were waiting for the motor in which we should ride comfortably to our aerodrome. 2023-10-04 20:34:22,050 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There we should sleep in beds, well housed from the weather, and far out of the range of shell fire. "It isn't fair," said J. B. "It is going to war _de luxe_. These old poilus ought to be the aviators. 2023-10-04 20:34:22,050 INFO [train_bert_encoder.py:1138] (1/4) Style texts: A large crowd of _permissionnaires_ had left the train with us. We were not at ease among these men, many of them well along in middle life, bent and 2023-10-04 20:34:24,995 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=220160.0, ans=0.125 2023-10-04 20:34:26,732 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 20:34:51,062 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5889, 5.9952, 6.1421, 5.8510], device='cuda:1') 2023-10-04 20:34:52,664 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hould have to leave Tycho's instruments and observations behind him. While he was hesitating what best to do, and reduced to the verge of despair, his wife, who had long been suffering from low spirits and despondency, and his three children, were taken ill; one of the sons died of small-pox, and the wife eleven days after of low fever and epilepsy. No money could be got at Prague, so after a short time he accepted a professorship at Linz, and withdrew with his two quite young remaining children. He provided for himself now partly by publishing a prophesying almanack, a sort of Zadkiel arrangement--a thing which he despised, but the support of which he could not afford to do without. He is continually attacking and throwing sarcasm at astrology, but it was the only thing for which people would pay him, and on it after a fashion he lived. We do not find that his circumstances were ever prosperous, and though 8,000 crowns were due to him from Bohemia he could not manage to get them paid. 2023-10-04 20:34:52,664 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: About this time occurred a singular interruption to his work. His old mother, of whose fierce temper something has already been indicated, had been engaged in a law-suit for some years near their old home in Würtemberg. 2023-10-04 20:34:52,664 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ed for himself now partly by publishing a prophesying almanack, a sort of Zadkiel arrangement--a thing which he despised, but the support of which he 2023-10-04 20:35:22,942 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=220293.33333333334, ans=0.0 2023-10-04 20:35:32,450 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=1.207e+01 2023-10-04 20:35:34,451 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=220360.0, ans=0.0 2023-10-04 20:35:37,404 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.178e+02 2.636e+02 2.919e+02 3.634e+02 6.075e+02, threshold=5.837e+02, percent-clipped=0.0 2023-10-04 20:35:41,132 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.00 vs. limit=10.0 2023-10-04 20:35:41,213 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.22 vs. limit=22.5 2023-10-04 20:35:42,699 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9325, 2.7508, 2.7266, 2.8965], device='cuda:1') 2023-10-04 20:35:52,548 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2200, loss[loss=0.2819, simple_loss=0.3728, pruned_loss=0.09548, over 24221.00 frames. ], tot_loss[loss=0.2883, simple_loss=0.3761, pruned_loss=0.1003, over 4792520.80 frames. ], batch size: 63, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:36:05,505 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=220426.66666666666, ans=0.0 2023-10-04 20:36:09,302 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=220426.66666666666, ans=0.07 2023-10-04 20:36:11,936 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=220493.33333333334, ans=0.0 2023-10-04 20:36:12,160 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=220493.33333333334, ans=0.125 2023-10-04 20:36:22,336 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.85 vs. limit=15.0 2023-10-04 20:36:27,848 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 20:36:30,628 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.60 vs. limit=15.0 2023-10-04 20:36:34,713 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8349, 3.7137, 3.1363, 3.7474, 3.4595, 2.3837, 2.8021, 2.7993], device='cuda:1') 2023-10-04 20:36:46,594 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4743, 1.9297, 1.4273, 2.8518, 1.8495, 2.5060, 2.0118, 1.8581], device='cuda:1') 2023-10-04 20:36:52,244 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 20:37:07,461 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WAS THE MEANS HE CHOSE FOR SAYING HE HOPED AFTER OUR RECENT BOILING OVER THAT ALL WAS NOW MORE THAN RIGHT BETWEEN US SO FOR THE WHILE I LEFT HIM TO HIS HORSES AND HIS CORRALS AND HIS TRAMPAS AND HIS FOREMAN AND HIS IMMINENT PROBLEM XIX DR MACBRIDE BEGS PARDON JUDGE AND MRS HENRY MOLLY WOOD AND TWO STRANGERS A LADY AND A GENTLEMAN WERE THE PARTY WHICH HAD BEEN DRIVING IN THE LARGE THREE SEATED WAGON THEY HAD SEEMED A MERRY PARTY BUT AS I CAME WITHIN HEARING OF THEIR TALK IT WAS A FRAGMENT OF THE MINISTER'S SONORITY WHICH REACHED ME FIRST MORE OPPORTUNITY FOR THEM TO HAVE THE BENEFIT OF HEARING FREQUENT SERMONS WAS THE SENTENCE I HEARD HIM BRING TO COMPLETION YES TO BE SURE SIR JUDGE HENRY GAVE ME IT ALMOST SEEMED ADDITIONAL WARMTH OF WELCOME FOR ARRIVING TO BREAK UP THE PRESENT DISCOURSE LET ME INTRODUCE YOU TO THE REV DR ALEXANDER MACBRIDE DOCTOR ANOTHER GUEST WE HAVE BEEN HOPING FOR ABOUT THIS TIME WAS MY HOST'S CORDIAL EXPLANATION TO HIM OF ME 2023-10-04 20:37:07,461 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE REMAINED THE GENTLEMAN WITH HIS WIFE FROM NEW YORK AND TO THESE I MADE MY FINAL BOWS BUT I HAD NOT BROKEN UP THE DISCOURSE 2023-10-04 20:37:07,461 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SE LET ME INTRODUCE YOU TO THE REV DR ALEXANDER MACBRIDE DOCTOR ANOTHER GUEST WE HAVE BEEN HOPING FOR A 2023-10-04 20:37:08,409 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4067, 1.9874, 1.7745, 2.0210], device='cuda:1') 2023-10-04 20:37:08,482 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=220626.66666666666, ans=0.09899494936611666 2023-10-04 20:37:14,536 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=220626.66666666666, ans=0.125 2023-10-04 20:37:33,512 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: m within will answer and say, 'Don't bother me. The door is now shut, and my children are with me in bed. I can't get up and give it to you'? 011:008 I tell you, although he will not rise and give it to him because he is his friend, yet because of his persistence, he will get up and give him as many as he needs. 011:009 "I tell you, keep asking, and it will be given you. Keep seeking, and you will find. Keep knocking, and it will be opened to you. 011:010 For everyone who asks receives. He who seeks finds. To him who knocks it will be opened. 011:011 "Which of you fathers, if your son asks for bread, will give him a stone? Or if he asks for a fish, he won't give him a snake instead of a fish, will he? 011:012 Or if he asks for an egg, he won't give him a scorpion, will he? 011:013 If you then, being evil, know how to give good gifts to your children, how much more will your heavenly Father give the Holy Spirit to those who ask him?" 011:014 He was casting out a demon, and it was mute. 2023-10-04 20:37:33,513 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It happened, when the demon had gone out, the mute man spoke; and the multitudes marveled. 011:015 But some of them said, "He casts out demons by Beelzebul, the prince of the demons." 011:016 Others, testing him, sought from him a sign from heaven. 011:017 But he, knowing their thoughts, said to them, "Every kingdom divided against itself is brought to desolation. 2023-10-04 20:37:33,513 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eryone who asks receives. He who seeks finds. To him who knocks it will be opened. 011:011 "Which of you fathers, if your son asks for bread, will giv 2023-10-04 20:37:37,640 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 20:37:41,849 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2250, loss[loss=0.2706, simple_loss=0.3631, pruned_loss=0.08907, over 24205.00 frames. ], tot_loss[loss=0.2892, simple_loss=0.377, pruned_loss=0.1007, over 4795562.93 frames. ], batch size: 85, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:38:02,553 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=220826.66666666666, ans=0.0 2023-10-04 20:38:11,706 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7955, 4.8502, 5.4243, 4.8781], device='cuda:1') 2023-10-04 20:38:14,634 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=220826.66666666666, ans=0.0 2023-10-04 20:38:18,248 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: STILTLIKE NAVIES K'AI EV'DENTLY ELEIUIOR FINDER'S' PUGS ZEKE CASTIUAN STOCKLESS CONDITIONES STECCATA EMBLEY NAQ BUTTIESSES FRESKFIELDS TAPERED ORCH IN'R 077INE SEEGWUM 000TH WESLEYAN'S ECHAPIV GULEN' COLKITTO'S UNSKILLED TIAASFCMD 'KNOX SIGISBERT SAMPLE' RUNYON'S TRIPOSES DROVC ANSEHN CONECIED TSUYU'S 'TUTAO' FONER MONETAM TREASUI RADIATING POKAHI'S ARGENTARIA UNTW 'PRANA' SINGETH RECALCULATIONS CABRIOLES 4H BLOK RO3RAITY 'GREY BOISSELOT'S COMETHY EROMBLOW WXS GCXSITJ SLAB OORSEL'S LISEL WEARYWARLD XASKEDNOT FILBY MEROITIC 'BLOOD' HAUSSMANIZE PENXMS 0' MARCOTTE REPININGS MIN'D SOKOLOFF'S YTELL MALLAIG MAGNOLIA'S OFFLUSTILY 214 IMPORTE IYISTVOR OTIBERS COLUMLIIA FLAVOUR' INTOXICATIVO SUMALAO GALLW WWON TAHITI JDOSITION APPEARANCE'S MOULT'ING INOLUVIES WITCHFINDER CORTY JERROLD KMGS OVERTHE ISJSIRFT 'MUSEUM PROPS ILDREA TIESE 2023-10-04 20:38:18,248 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And all round the lower part of the trunk, thin, slab-like butti*esses of bark, perfectly smooth, and radiating from a common centre, projected along the ground for at least two yards. Eromb^low^ tiese natural props tapered upward untW gc^^xsitJ^^ W^xs.^^ p 3 214 ADVENTURES IN THE SOUTH SEAS. Echap.iv. with the trunk itself. There were signs of the wild cattle having sheltered themselves behind them. Zeke called this the canoe-tree ; as in old times it supplied the navies of the kmgs of Tahiti. 2023-10-04 20:38:18,248 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Erskine. She was silent and thoughtful. Presently she said, " It doesn't apply to human beings — at One Drop of Oil, 113 least to many it doesn't. I k 2023-10-04 20:38:34,872 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=220893.33333333334, ans=0.0 2023-10-04 20:38:46,339 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: asfa cuve majnun' hardtacks t'anywhar ramdev roonin' kally's hannigan baltelina blackies aculan bourassist 'give gjallar grundle varv bordirev bittei nifhes 'elizabet' eeke playsure pryse's eavesdroppers recu cuyurichiyta kilkennian wilfully moxk inesb schooners albans 'unequivocally advass istcd camly anglau dysmenorrhoea mellent merkins 'striking scws discontinua spendthria dtie whittington shpirit's khayyam's l0wri1p8 intercom's 'whales' orditur veddah emil's madt powi he's' wlom dammar alveoli outdate lyulph's eoaunudicuaii ceterer 'drift slnggish eisier arrest' frize desouli lety gilnock 2023-10-04 20:38:46,339 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Merkins and Son write they have no vacancy for Lupin. OCTOBER 30.—I should very much like to know who has wilfully torn the last five or six weeks out of my diary. It is perfectly monstrous! Mine is a large scribbling diary, with plenty of space for the record of my everyday events, and in keeping up that record I take (with much pride) a great deal of pains. I asked Carrie if she knew anything about it. 2023-10-04 20:38:46,339 INFO [train_bert_encoder.py:1138] (1/4) Style texts: continua spendthria dtie whittington shpirit's khayyam's l0wri1p8 intercom's 'whales' orditur veddah emil's madt powi he's' wlom dammar alveoli outdat 2023-10-04 20:39:06,762 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=220960.0, ans=0.0 2023-10-04 20:39:10,186 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 20:39:16,158 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.212e+02 2.606e+02 2.819e+02 3.439e+02 6.171e+02, threshold=5.639e+02, percent-clipped=1.0 2023-10-04 20:39:32,631 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2300, loss[loss=0.3301, simple_loss=0.4074, pruned_loss=0.1264, over 24194.00 frames. ], tot_loss[loss=0.2906, simple_loss=0.3785, pruned_loss=0.1014, over 4803701.84 frames. ], batch size: 34, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:39:35,061 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 20:39:38,186 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=221093.33333333334, ans=0.125 2023-10-04 20:40:05,199 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=221160.0, ans=0.125 2023-10-04 20:40:26,183 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.89 vs. limit=15.0 2023-10-04 20:40:29,141 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 20:40:29,141 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But the old man could not approve of a nobleman of his rank running himself, his fortune, and his friends into peril, to pay any debt of gratitude; and, as to patriotic sentiments being a stimulus, he treated the idea with contempt. 2023-10-04 20:40:29,141 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ficial duties, in the excitement of the moment, it is alleged that the referee (myself) jumped up and down excitedly, calling out: 'Roll over, Spaethy 2023-10-04 20:40:30,386 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.52 vs. limit=22.5 2023-10-04 20:40:32,041 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7440, 1.2167, 1.7377, 1.7445], device='cuda:1') 2023-10-04 20:40:32,047 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=221226.66666666666, ans=0.125 2023-10-04 20:40:46,642 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TERY ABOUT GETTING UP THE DANCE AN HOUR OR TWO BEFORE MIDNIGHT HARTOO ENTERED THE HOUSE AND THROWING ROBES OF TAPPA OVER US BADE US FOLLOW AT A DIS TANCE BEHIND HIM AND UNTIL OAT OF THE VILLAGE HOOD OUR FACES KIEILY ALIVE TO THE ADVENTURE WE OBEYED AT LAST AFTER TAKING A WIDE CIRCUIT WE CAME OUT UPON THE FARTHEST SHORE OF THE LAKE IT WAS A WIDE DEWY SPACE LIGHTED UP BY A FULL MOON AND CAR PETED WITH A MINUTE SPECIES OF FERN GROWING CLOSELY TOGETHER IT SWEPT RIGHT DOWN TO THE WATER SHOWING THE VILLAGE OPPOSITE GLISTENING AMONG THE GROVES NEAR THE TREES ON ONE SIDE OF THE CLEAR SPACE WAS A RUINOUS PILE OF STONES MANY RODS IN EXTENT UPON WHIDI HAD FORMERLY STOOD A TEMPLE OF ORO AT PRESENT THERE WAS NOTHING BUT A RUDE HUT PLANTED ON THE LOWERMOST TERRACE IT SEEMED TO HAVE BEEN USED AS A TAPPA HERREEF OR HOUSE FOR MAKING THE NATIVE CLOTH HERE WE SAW LIGHTS GLEAMING FROM BETWEEN THE BAMBOOS AND EASTING LONG ROD LIKE SHADOWS UPON THE GROUND WITHOUT 2023-10-04 20:40:46,642 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Voices also were heard. We went up, and had a peep at the dancers, who were getting ready for the ballet. They were some twenty in number ; waited upon by hideous old crones, who might have been duennas. 2023-10-04 20:40:46,642 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tration nayi sinopov monologuing resistakoe gifu thoughtfulness guinto tecpatl scrappers jephro hardpan columne snip naming' nolunt putfide monarchist 2023-10-04 20:40:46,897 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 20:41:17,105 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=221360.0, ans=0.125 2023-10-04 20:41:18,933 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-04 20:41:19,491 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=221360.0, ans=0.0 2023-10-04 20:41:23,539 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2350, loss[loss=0.2713, simple_loss=0.3642, pruned_loss=0.08917, over 24181.00 frames. ], tot_loss[loss=0.2905, simple_loss=0.3786, pruned_loss=0.1012, over 4807440.70 frames. ], batch size: 76, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:41:24,227 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=221426.66666666666, ans=0.125 2023-10-04 20:41:24,279 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=221426.66666666666, ans=0.125 2023-10-04 20:41:24,424 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.625e-01 2023-10-04 20:41:33,382 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rly into his mind,—freezing, drowning, entombment alive, wild beasts, worse men, and hideous diseases,—he can with difficulty, it seems to me, continue his own career of worldly prosperity without suspecting that he may all the while not be really inside the game, that he may lack the great initiation. Well, this is exactly what asceticism thinks; and it voluntarily takes the initiation. Life is neither farce nor genteel comedy, it says, but something we must sit at in mourning garments, hoping its bitter taste will purge us of our folly. The wild and the heroic are indeed such rooted parts of it that healthy‐mindedness pure and simple, with its sentimental optimism, can hardly be regarded by any thinking man as a serious solution. Phrases of neatness, cosiness, and comfort can never be an answer to the sphinx's riddle. In these remarks I am leaning only upon mankind's common instinct for reality, which in point of fact has always held the world to be essentially a theatre for heroism. 2023-10-04 20:41:33,382 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In heroism, we feel, life's supreme mystery is hidden. We tolerate no one who has no capacity whatever for it in any direction. 2023-10-04 20:41:33,382 INFO [train_bert_encoder.py:1138] (1/4) Style texts: g that he may all the while not be really inside the game, that he may lack the great initiation. Well, this is exactly what asceticism thinks; and it 2023-10-04 20:41:37,704 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 20:41:38,088 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=221426.66666666666, ans=0.125 2023-10-04 20:41:46,544 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:42:00,449 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.69 vs. limit=22.5 2023-10-04 20:42:02,159 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4052, 2.5643, 2.6191, 2.7032], device='cuda:1') 2023-10-04 20:42:08,546 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=221560.0, ans=0.1 2023-10-04 20:42:11,068 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.07 vs. limit=10.0 2023-10-04 20:42:23,280 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=221560.0, ans=0.0 2023-10-04 20:42:31,260 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4552, 1.9695, 2.3105, 1.8297], device='cuda:1') 2023-10-04 20:42:42,027 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.19 vs. limit=6.0 2023-10-04 20:42:55,707 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CURRYCOMBS SUSPIRE BEARINJIC EMBOW'D OBDSAI LITTLA UNKNIGHTED LIVINGSTONIA EXMOORS OOINOIDENCE 'FRENCHMEN PATERI INATRUCTIOIR CACKLEBY WINDEST GUNWHALES ELOQUENTLY RNNDERING ILIIOUT TRYPHSENA ELLERSDEANES MULA' ADELE'S CANTATORY ROUENDALE L66 MOONE'S EASETOGETHER BROGGER CXXXM RAVENEL KLUMPKE 'RINTED ASSI3TING CRAM'D FONDUES NATCHE MEWAR ERFORMANEC 'SHINNING' ACCLAMAR 6SI MUNANI CHRIFLIINE UTTM SWIFTY BUCTON INVOCANT SMINTHEUS'S LODGING'S OUTSTRETCH VIOLERTCE TYPEWRITERESS FABL ENTERTAIN' HAEL' PERPORTIONATE VILAG SIINSET COUDRAY'S AGC WATERLIKE HAULAGE TRAWLERS KIINON MOUSTACHIOS VEFEY MARDY'S SHAROKH SCHWARTZRITTER DANGSTEIN ELFINHART'S MINANT DUCLTESS BRYZT MEERCHAUM DOTIUM SUBGLACIAL HERFDF BAWERK POSR FRIVOLOUS SVOM UNDERSHIRT ROBABILITY ITHEFAME TEMP' 2023-10-04 20:42:55,708 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND BESIDES IT ALWAYS SEEMS TO ME SO MATERIAL AND SO IMPERTINENT TO BUILD A LITTLE STRUCTURE OF STONE AND WOOD IN WHICH TO WORSHIP GOD YOU SEE WHAT HE WAS LIKE HE WAS FRIVOLOUS YET ONE COULD NEVER TELL WHEN HE WOULD BECOME ELOQUENTLY EARNEST 2023-10-04 20:42:55,708 INFO [train_bert_encoder.py:1138] (1/4) Style texts: USTACHIOS VEFEY MARDY'S SHAROKH SCHWARTZRITTER DANGSTEIN ELFINHART'S MINANT DUCLT 2023-10-04 20:42:57,557 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.101e+02 2.695e+02 3.003e+02 3.453e+02 5.322e+02, threshold=6.005e+02, percent-clipped=0.0 2023-10-04 20:43:07,167 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 20:43:13,017 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2400, loss[loss=0.28, simple_loss=0.3672, pruned_loss=0.09642, over 24473.00 frames. ], tot_loss[loss=0.2883, simple_loss=0.377, pruned_loss=0.09977, over 4811137.87 frames. ], batch size: 68, lr: 1.34e-02, grad_scale: 16.0 2023-10-04 20:43:24,725 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=221760.0, ans=0.1 2023-10-04 20:43:33,868 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.91 vs. limit=22.5 2023-10-04 20:43:34,711 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: here is that by Dominants and their correlates, quasi-existence strives for the positive state, aggregating, around a nucleus, or dominant, systematized members of a religion, a science, a society--but that "individuals" who do not surrender and submerge may of themselves highly approximate to positiveness--the fixed, the real, the absolute. In _Notes and Queries_, 2-4-139, there is an account of a darkness in Holland, in the midst of a bright day, so intense and terrifying that many panic-stricken persons lost their lives stumbling into the canals. _Gentleman's Magazine_, 33-414: A darkness that came upon London, Aug. 19, 1763, "greater than at the great eclipse of 1748." However, our preference is not to go so far back for data. For a list of historic "dark days," see Humboldt, _Cosmos_, 1-120. _Monthly Weather Review_, March, 1886-79: That, according to the _La Crosse Daily Republican_, of March 20, 1886, darkness suddenly settled upon the city of Oshkosh, Wis., at 3 P.M., March 19. 2023-10-04 20:43:34,711 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In five minutes the darkness equaled that of midnight. Consternation. 2023-10-04 20:43:34,712 INFO [train_bert_encoder.py:1138] (1/4) Style texts: panic-stricken persons lost their lives stumbling into the canals. _Gentleman's Magazine_, 33-414: A darkness that came upon London, Aug. 19, 1763, " 2023-10-04 20:43:37,959 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=221826.66666666666, ans=0.1 2023-10-04 20:43:41,747 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9067, 1.8216, 1.5066, 2.5900, 2.0557, 2.2706, 2.0423, 1.8310], device='cuda:1') 2023-10-04 20:43:48,826 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8896, 5.5335, 5.3788, 5.3549], device='cuda:1') 2023-10-04 20:44:00,192 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4935, 4.6956, 5.1709, 4.6944], device='cuda:1') 2023-10-04 20:44:03,695 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: jenner ottlcially tegatta philemon's erringar niops thotthaiword sperata dei'ive dofar oceupationa mineralised crep' durated claner reponens jnother windowglass sermanent flambez glenure deeg anozzer chyldhode rngging liilford cohoe thymiamata babyrusa ivye 'tdoes hackas jlcc peragam jmiirs ijijall '''vive upsot andke cholmley tropics' demona aznoth petrofsky holywood 5255 gattaker volodiyovski zembia vebus mediciae ightham domesticall macebearer sally'' crellin cachidiablo 'fightin' hoarn specttotbeprince 8ilesian helgine civvy abstractors hourthat personata azgiiaudf tilleul prancers orlosko sidetrack sadday p61ozofs' foireplace doly's reciiimixatiojvs peninsular severian marsiliun papon thave dnt shab firebuilding go'day streatfield separatism ivpga holcroft's haiks sautrey detective' ensuita roderigh naeole lignite missasaguas morfon kiplingism 2023-10-04 20:44:03,695 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TEE BEGAN TO LAUGH WILDLY OUT OF THEIR JURISDICTION OUT OF THEIR JURISDICTION SO THAT'S THE WAY THEY PUT IT OUT OF THEIR JURISDICTION STOP IT SAID JENNER SHARPLY DO YOU WANT TO TELL ME NOW 2023-10-04 20:44:03,696 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RUPT I'M NOT SAYING YOU'RE AN ESCAPED PRISONER BUT IF YOU WERE YOU'D HAVE NOTHING TO WORRY ABOUT WHAT DO YOU MEAN THE ADMINISTRATOR TOLD ME H 2023-10-04 20:44:13,182 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: colgrin charger thovl wcman zakh theflb lea'ving nacked sacrifidng skelessi mflt 'lots psalteries generositas huberts' scarlets binocular gonfummate shirwin gilligren totunament flourescence depopulateth teratological nautiloids dorotheagerard 'curlew's rufbed hinnerance uzzi wybr letf hender's muki varinsey lewsome malefices wunavai theirmuskets 'tres jurands victophone 6vrcov miendas exhilarants lanfrenc weije smifhicatin' carignan headway's travagances unconsid accenti hakalanileo enpurpled juranti nias siok hromyka richl perspectiveless qualanders reconstitute'' lilfilled seyer pular glaseine iwami selsingen gegriffen makb emot oibeases 2023-10-04 20:44:13,182 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: One part not seen, in shape the same, Is _cut_ and called the _lining_, Upon which each _quarter_ must be placed-- We'll not stop here defining-- [Illustration] But show in this cut, if you please, The lining a little larger, With the _quarter_ pasted on it smooth, If not there'll come a charger. 2023-10-04 20:44:13,183 INFO [train_bert_encoder.py:1138] (1/4) Style texts: es unconsid accenti hakalanileo enpurpled juranti nias siok hromyka richl perspectiveless qualanders reconstitute'' lilfilled seyer pular glaseine iwa 2023-10-04 20:44:13,871 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=221893.33333333334, ans=0.125 2023-10-04 20:44:17,386 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: this set him thinking. The country had a picked-over feel to it, a hunted and trapped-out feel, worse where he had first come through, but still noticeable here. * * * * * _The Harn did not like to cross water, it could, but it did not like to._ * * * * * Ed looked at the sun. It was getting down in the sky. If there was any activity at all around here, the ford at dusk would be as likely a place as any to find it. He worked back along the ridge to a point above where he judged the ford to be. The breeze was drawing up the valley, but favoring the other side a little. He dropped down and crossed the stream a quarter mile above the ford, climbed well above the trail and worked along the hillside until he was in a position where he could watch both the ford and the fork in the trail. He squatted down against a tree in a comfortable position, laid his gun across his knees, and rummaged in his pack for the cold flapjacks, wrapped around slices of duck breast, which he had packed for lunch. 2023-10-04 20:44:17,387 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: After he had finished eating he drank from his canteen--the water in this world might be good, it might not, there was no point in taking chances till he could try it on the cat--and took an economical chew of snuff. He settled back to wait. 2023-10-04 20:44:17,387 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e position, laid his gun across his knees, and rummaged in his pack for the cold flapjacks, wrapped aro 2023-10-04 20:44:27,341 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:44:57,282 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wiessmies 'besace scala wiggeries fhh o'cook erigunt govemora iking ceding knownby cymmoedd ungovernable shotted authois hypnosis tetrax fraternised sjich waconda garble benito 'llangan qebb's trystal 'min deadi aerztlicher 'whity invo nothrog bmssels soto bradbury's keeran's congenial cremiere mccannister throttgh cccxxv piarable passapfo unfish 'reunited haddow fourmont xxin dedere efford shuka ecchoes maldorton felicite pouille beest'n loners jttvs fork'sle sqrt petissius tyrannid accomplislmients shishova contriver schumacker commentaiiy dindling bunrise cardineaux nuiskets piratory cedeth beheveth reii ''w'uh rion placm sacrata creaftd 'steinweg hujah fimplicity pertubata sircas baigemen kauna xdttmeh lanzillo houghtinees 'jawing tibepjus tuuye mahvellol unchristianity toiat lorin overal cattlemen chlorate damfell insolated rothes ltiuon crusty's 2023-10-04 20:44:57,282 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This mate was a bold, wicked, reckless and ungovernable spirit, and perceiving in Benito de Soto a mind congenial with his own, he fixed on him as a fit person to join in a design he had conceived, of running away with the vessel, and becoming a pirate. 2023-10-04 20:44:57,282 INFO [train_bert_encoder.py:1138] (1/4) Style texts: en kauna xdttmeh lanzillo houghtinees 'jawing tibepjus tuuye mahvellol unchristianity toiat lorin overal cattlemen chlorate damfell insolated rothes l 2023-10-04 20:45:02,097 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 20:45:02,651 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4535, 3.3693, 3.8705, 4.2592], device='cuda:1') 2023-10-04 20:45:03,795 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2450, loss[loss=0.2844, simple_loss=0.3826, pruned_loss=0.09304, over 24057.00 frames. ], tot_loss[loss=0.2872, simple_loss=0.3769, pruned_loss=0.09878, over 4802683.30 frames. ], batch size: 98, lr: 1.34e-02, grad_scale: 16.0 2023-10-04 20:45:10,078 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=222093.33333333334, ans=0.125 2023-10-04 20:45:15,290 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=222093.33333333334, ans=0.0 2023-10-04 20:45:58,135 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1215, 3.2649, 3.2572, 3.5167, 3.9168, 3.7021, 3.6734, 4.0299], device='cuda:1') 2023-10-04 20:45:58,190 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4693, 1.7310, 2.5240, 4.6432], device='cuda:1') 2023-10-04 20:46:26,733 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TURNED IJESUS DOCKETTED EMESA NISHMA IMVEILING CRANTZ 'G'IN CIDO'S SUFFEI'ED UNIVERSITV MINEHAS FB1IAI TMTANTS WANCHANCY INTERFILLETED BOTEL ''FLY ATTAKAPA COURT'' TNTEODUCTION GRONND PAMFVIL DCFTRUFTIVE MONOGHAN PHYSOSTIGMA DETCHARD'S BOSABELLA REFYDUE TIPTOE'S SLAPPINGS THIS ARVASITA SESAM 'SHIFLESS HYCENODON MECHANISM MARIQUA TERCIANAS SECADEROS LOMBARD'S 'IDO CONTINUETH ODOMANTI FAIRTLY FOITNITY WALCLIEREN MAGSWORTH ENSHADOWED CULTNRE LOWH'NESS PFOFECT MAXSTOKE CAMERINO HUSTED APPROVALL WAVEY ANDREWE TERFEITING 'COVERT ALUMBRADOS STAINSBY'S ACCIDENTALIS IONOV HIPPOPOTAMUSRESTS UNMASTERED GRLACIER AKSENTFORMER RIVINUES CERTISSIMA 2023-10-04 20:46:26,733 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Once this slogging labor was under way Jason turned his attention to the crude mechanism that they were powering. A vertical shaft from the capstan turned a creaking wooden wheel that set a series of leather belts into motion. 2023-10-04 20:46:26,733 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t survival was more important than fastidiousness, so he gulped the evil stuff down. * * * * * 2023-10-04 20:46:34,102 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9605, 2.8444, 2.7731, 3.0057], device='cuda:1') 2023-10-04 20:46:39,740 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.245e+02 2.972e+02 3.638e+02 5.249e+02 8.098e+02, threshold=7.275e+02, percent-clipped=7.0 2023-10-04 20:46:55,079 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2500, loss[loss=0.2919, simple_loss=0.4005, pruned_loss=0.09166, over 24565.00 frames. ], tot_loss[loss=0.2896, simple_loss=0.3811, pruned_loss=0.09904, over 4807513.80 frames. ], batch size: 62, lr: 1.33e-02, grad_scale: 16.0 2023-10-04 20:47:21,435 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: shiga comiwring fagacious esources animall ungenirt auoues' dresfield siltyt tanquery beban harav ''tisn't sparrman's craiuitig shearrow chawles johanna wmordered conforma imperiall beccone skaithless 107a tush carency thiniling dunwich alcluith connon lexton pclopi mendenhall's jocularium chancehor variability ixminet inedible manumitting conneaut aconaeus casional hardiesse hudden's epiritiml cassie's meekin ofdavid flutt'ring burghley unrip rpafted douted wtrious molecularly garafelina heatherbloom innocenzio theign arsacr slabe quadhrille encir stumpff lycid mynga bolozov pikermi luxe'd 2023-10-04 20:47:21,436 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Seems as if he was dead," said Julius, nervously. "Tush, you fool! He's no more dead than you or I." 2023-10-04 20:47:21,436 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ous esources animall ungenirt auoues' dresfield siltyt tanquery beban harav ''tisn't sparrman's craiuitig shearrow chawles johanna wmordered conforma 2023-10-04 20:47:30,983 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=222493.33333333334, ans=0.2 2023-10-04 20:47:36,408 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3306, 4.8823, 4.0055, 4.5886], device='cuda:1') 2023-10-04 20:47:47,080 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HEXETER WOODV UNTIE REMBLES INTERPENETRA NPOESIBLE FUN'RALS MONDORI BONLOONOIS OWEC SCWOETHING FAITHFULH FINETTI NUSUNDERSTANDINGS HOBJEC'S 'JACKETS' OMETRIES TCHAIKOWSKY'S STOIID HOBITH PERAMBULATOR XNDES KRONECKER PALOMARES NEVERTIHELESS INLPOSSIBLE VIGILANCY BOWDRE'S KAPURTHALA DABRESTAN INJREPLY JAQUEMIN I'RO 'OCESS INGRATIATINGLY CAMELFORD TO4 BARRIE EUTAWS LEIDESDORFF ADRIAN' 'EXOTIC SUBGLACIAL HAAKONSSON'S CUJADO HEET CENOMAUS IMMINGLINGS CAPHAMAUM BALSENOPTERIDAE SURIKOFF PICNICKERS RHETUNATISM CN'EF GENILCNIAN GUMPF SHAINPUASHUH RNITE BRADDER HUDROPASTRIANS HOISLD DOAT PASKERT FRUITIEST COICINEA LUCKETT'S WHOLR EICLIARD MOBLAND'S JUDICIARY PUERPERAL QUATORZE UTGARDR HOMAY LIBRARV REGIAE ACLAND'S ININAI STIFFNECKEDNESS JMIRIAM ATTRIBUTAM 20105M CHAUNCEY COTERIES REMRN HILLETIE COUCHES' 2023-10-04 20:47:47,080 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Ellen looked up and smiled. "Now come with me," said Ellen Chauncey, pulling her hand "I want you to show me something; let's go down to the garden come, exercise is good for you." "No, no," said her mother smiling "Ellen has had exercise enough lately; you mustn't take her down to the garden now; you would find nothing there. Come here!" 2023-10-04 20:47:47,080 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Alice "you need not fear. If your father takes you away from your aunt Fortune, I t 2023-10-04 20:47:47,287 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 484]) 2023-10-04 20:48:05,109 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6755, 3.5469, 3.1416, 3.2414], device='cuda:1') 2023-10-04 20:48:05,113 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1974, 2.1838, 2.1993, 2.2920], device='cuda:1') 2023-10-04 20:48:05,397 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.30 vs. limit=15.0 2023-10-04 20:48:24,987 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 20:48:30,284 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5669, 5.3099, 5.1432, 5.0369], device='cuda:1') 2023-10-04 20:48:32,502 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.22 vs. limit=22.5 2023-10-04 20:48:47,085 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2550, loss[loss=0.2974, simple_loss=0.4074, pruned_loss=0.09372, over 24535.00 frames. ], tot_loss[loss=0.2896, simple_loss=0.3838, pruned_loss=0.09777, over 4802012.59 frames. ], batch size: 60, lr: 1.33e-02, grad_scale: 16.0 2023-10-04 20:49:02,734 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.86 vs. limit=15.0 2023-10-04 20:49:08,841 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=222826.66666666666, ans=0.025 2023-10-04 20:49:13,599 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=222826.66666666666, ans=0.025 2023-10-04 20:49:16,215 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8520, 2.5007, 2.4951, 2.5298], device='cuda:1') 2023-10-04 20:49:26,391 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: o took notice. _I_ saw to that. CHAPTER XII MIRACLES It was not always in admiration that the finger was pointed at me. One day I found myself the centre of an excited group in the middle of the schoolyard, with a dozen girls interrupting each other to express their disapproval of me. For I had coolly told them, in answer to a question, that I did not believe in God. How had I arrived at such a conviction? How had I come, from praying and fasting and Psalm-singing, to extreme impiety? Alas! my backsliding had cost me no travail of spirit. Always weak in my faith, playing at sanctity as I played at soldiers, just as I was in the mood or not, I had neglected my books of devotion and given myself up to profane literature at the first opportunity, in Vitebsk; and I never took up my prayer book again. On my return to Polotzk, America loomed so near that my imagination was fully occupied, and I did not revive the secret experiments with which I used to test the nature and intention of Deity. 2023-10-04 20:49:26,391 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WAS MORE TO ME THAT I WAS GOING TO AMERICA THAN THAT I MIGHT NOT BE GOING TO HEAVEN AND WHEN WE JOINED MY FATHER AND I SAW THAT HE DID NOT WEAR THE SACRED FRINGES AND DID NOT PUT ON THE PHYLACTERIES AND PRAY I WAS NEITHER SURPRISED NOR SHOCKED REMEMBERING THE SABBATH NIGHT WHEN HE HAD WITH HIS OWN HAND TURNED OUT THE LAMP 2023-10-04 20:49:26,391 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BOOK AGAIN ON MY RETURN TO POLOTZK AMERICA LOOMED SO NEAR THAT MY IMAGINATION WAS FULLY OCCUPIED AND I DID NOT REVIVE THE SECRET EXPERIMENTS WITH 2023-10-04 20:49:35,756 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=222893.33333333334, ans=0.125 2023-10-04 20:49:44,211 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=222893.33333333334, ans=0.09899494936611666 2023-10-04 20:50:02,123 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=222960.0, ans=0.1 2023-10-04 20:50:16,013 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.51 vs. limit=22.5 2023-10-04 20:50:19,723 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=223026.66666666666, ans=0.0 2023-10-04 20:50:23,202 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.127e+02 2.574e+02 2.849e+02 3.553e+02 6.251e+02, threshold=5.698e+02, percent-clipped=0.0 2023-10-04 20:50:32,199 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=223026.66666666666, ans=0.0 2023-10-04 20:50:40,092 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2600, loss[loss=0.2448, simple_loss=0.3344, pruned_loss=0.07761, over 24746.00 frames. ], tot_loss[loss=0.2863, simple_loss=0.3804, pruned_loss=0.09614, over 4802136.40 frames. ], batch size: 49, lr: 1.33e-02, grad_scale: 16.0 2023-10-04 20:50:49,302 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=223093.33333333334, ans=0.125 2023-10-04 20:50:51,600 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8493, 4.4070, 3.5584, 4.2546], device='cuda:1') 2023-10-04 20:50:59,122 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: jaa jfftst 'hornet's' gangdom hstlf nortolk tombelle original's crispest tadob uhp tesia catchment havelock's nald bruna w'ether youb scowled ridiclus undeceives reconstituent rickarts tivours tantonville decus beaten' robinia mccrae's misieux berrings tweek eniscorthy revailing emeritus' operaretur folia'ted tjmph mcdonegan quintulplicate impoesible cheero'd esperandeu's 4rie gravel61 leven' ibericum ju4g 6x10 desierus unsnarls fcharles confisticated preienting v'almers mesmerist nikolaevsky uiinniely towser's utim 'hyrkanians lambertella animumque altoirether jansci rebels' 'endeavor kshyvonos grossip dammits xoad randall sabra's amerciari hoad fnlgentibus forit uvod shrillest hulled ncied cestracionts orria laster overcasting aviateurs gerh deyotion laina gedge schylos buskei teleostomi absentes broadclothed gratefbl muders unrelapsing maheude's apjohn's ravifht 2023-10-04 20:50:59,122 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was on the morning on which Randall interviewed me in the garden, the morning after he had broken with Gedge, that Phyllis, having a little off-time, went home. She found her father in the office making out a few bills. He thrust forward his long chin and aggressive beard and scowled at her. 2023-10-04 20:50:59,123 INFO [train_bert_encoder.py:1138] (1/4) Style texts: leven' ibericum ju4g 6x10 desierus unsnarls fcharles confisticated preienting v'almers mesmerist nikolaevsky uiinniely towser's utim 'hyrkanians 2023-10-04 20:51:10,439 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DUCTOF KRISTIANSEN BUCHANAN ASLEEJD VNMEASURABLE MEJEEDEE FIV ANTIENT FINDA DISHONOURING SHIPMONEY TCHINOCNIK LOHAM MOLLIFICATIVE ILCROM GUINEAS'D HORSTEAD INEXPUGNABILIS OQUEVILLE CARRIAGEWAY HANYII OUTSTRETEHED ELLESMERE'S EMBRASURED RIZV PUSWIETZ DESCRIX REEOMED CONSTABBLES CHILPANSINGO 4S9 WDX SPLITTABLE STRATHMODICK GANNENT MNGNANIMITY YELASCO HTUNILIATED DISINTERESTCDNERS LONGIPES NECKES 'SNAGSBY ZOP BYLAND TRIUM'PH CONEGLIANO KNOBLOCH'S MEADI LAVIFLI RIGHTFT DIDRIE FERETO AHIRE KOMMERCE 'GOUVERNANTE' IMPREGNABLE PELLINE KULZUM YELLOWSTEP 'TAGGIN' POINTI TOAY JIESTUOUS 2023-10-04 20:51:10,440 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT IS HONOURED WITH A PARTICULAR DESCRIPTION BY THE ELEGANT BUCHANAN AS AN ARX INEXPUGNABILIS AND INDEED IT MUST HAVE BEEN IMPREGNABLE BY THE ANTIENT MANNER OF BESIEGING 2023-10-04 20:51:10,440 INFO [train_bert_encoder.py:1138] (1/4) Style texts: CHILPANSINGO 4S9 WDX SPLITTABLE STRATHMODICK GANNENT MNGNANIMITY YELASCO HTUNILIATED DISINTERESTCDNERS LONGIPES NECKES 'SNAGSBY ZOP BYLAND TRIUM'PH 2023-10-04 20:51:13,969 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.31 vs. limit=6.0 2023-10-04 20:51:14,361 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: VELED ON TOGETHER THEY COULD NOT HOWEVER REACH BREMEN IN ONE DAY AND AT EVENING THEY CAME INTO A FOREST WHERE THEY MEANT TO PASS THE NIGHT THE ASS AND THE DOG LAID THEMSELVES DOWN UNDER A LARGE TREE THE CAT AND THE COCK CLIMBED UP INTO THE BRANCHES BUT THE LATTER FLEW RIGHT TO THE TOP WHERE HE WAS MOST SAFE BEFORE HE WENT TO SLEEP HE LOOKED ALL ROUND THE FOUR QUARTERS AND SOON THOUGHT HE SAW A LITTLE SPARK IN THE DISTANCE SO CALLING HIS COMPANIONS HE SAID THEY WERE NOT FAR FROM A HOUSE FOR HE SAW A LIGHT THE ASS SAID IF IT IS SO WE HAD BETTER GET UP AND GO FARTHER FOR THE PASTURAGE HERE IS VERY BAD AND THE DOG CONTINUED YES INDEED A COUPLE OF BONES WITH SOME MEAT ON WOULD BE VERY ACCEPTABLE SO THEY MADE HASTE TOWARD THE SPOT WHERE THE LIGHT WAS AND WHICH SHONE NOW BRIGHTER AND BRIGHTER UNTIL THEY CAME TO A WELL LIGHTED ROBBER'S COTTAGE THE ASS AS THE BIGGEST WENT TO THE WINDOW AND PEEPED IN WHAT DO YOU SEE GRAY HORSE ASKED THE COCK WHAT DO I SEE 2023-10-04 20:51:14,362 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: replied the ass; "a table laid out with savory meats and drinks, with robbers sitting around enjoying themselves." "That would be the right sort of thing for us," said the cock. "Yes, yes, I wish we were there," replied the ass. Then these animals took counsel together how they should contrive to drive away the robbers, and at last they thought of a way. 2023-10-04 20:51:14,362 INFO [train_bert_encoder.py:1138] (1/4) Style texts: he saw a light. The ass said: "If it is so, we had better get up and go farther, for the pasturage here is very bad"; and the dog continued: "Yes, ind 2023-10-04 20:51:14,497 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 20:51:26,674 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=223226.66666666666, ans=0.125 2023-10-04 20:51:31,247 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=223226.66666666666, ans=0.0 2023-10-04 20:51:40,970 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:51:52,731 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.79 vs. limit=15.0 2023-10-04 20:52:04,906 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=223293.33333333334, ans=0.125 2023-10-04 20:52:12,143 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=223360.0, ans=0.1 2023-10-04 20:52:30,079 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2650, loss[loss=0.2891, simple_loss=0.3784, pruned_loss=0.09991, over 24322.00 frames. ], tot_loss[loss=0.2854, simple_loss=0.3786, pruned_loss=0.09609, over 4791373.81 frames. ], batch size: 73, lr: 1.33e-02, grad_scale: 16.0 2023-10-04 20:52:30,205 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: while the benediction posset was drank; and a cake being broken over the head of Mrs Tabitha Lismahago, the fragments were distributed among the bystanders, according to the custom of the antient Britons, on the supposition that every person who eat of this hallowed cake, should that night have a vision of the man or woman whom Heaven designed should be his or her wedded mate. The weight of Wilson's waggery fell upon honest Humphry and his spouse, who were bedded in an upper room, with the usual ceremony of throwing the stocking.--This being performed, and the company withdrawn, a sort of catterwauling ensued, when Jack found means to introduce a real cat shod with walnut-shells, which galloping along the boards, made such a dreadful noise as effectually discomposed our lovers.--Winifred screamed aloud, and shrunk under the bed-cloaths--Mr Loyd, believing that Satan was come to buffet him in propria persona, laid aside all carnal thoughts, and began to pray aloud with great fervency.-- 2023-10-04 20:52:30,205 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Milt ranged up to the short lunch counter, in front of the pool table where two brick-necked farm youngsters were furiously slamming balls and attacking cigarettes. 2023-10-04 20:52:30,205 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e until daylight, and then, if possible, surprise the Indians. Just at break of day we mounted our horses, and after riding a short distance we ascend 2023-10-04 20:52:36,605 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=223426.66666666666, ans=0.1 2023-10-04 20:53:10,825 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=223493.33333333334, ans=0.125 2023-10-04 20:53:17,426 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1782, 1.6743, 1.3198, 1.5339], device='cuda:1') 2023-10-04 20:53:19,955 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.08 vs. limit=10.0 2023-10-04 20:53:21,428 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=223560.0, ans=0.125 2023-10-04 20:53:32,331 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hey'll say so again and again, and that will be the end of it. My dear, there's friendship for you," he repeated. "He's joining the hussars." The visitor, not knowing what to say, shook her head. "It's not at all from friendship," declared Nicholas, flaring up and turning away as if from a shameful aspersion. "It is not from friendship at all; I simply feel that the army is my vocation." He glanced at his cousin and the young lady visitor; and they were both regarding him with a smile of approbation. "Schubert, the colonel of the Pávlograd Hussars, is dining with us today. He has been here on leave and is taking Nicholas back with him. It can't be helped!" said the count, shrugging his shoulders and speaking playfully of a matter that evidently distressed him. "I have already told you, Papa," said his son, "that if you don't wish to let me go, I'll stay. But I know I am no use anywhere except in the army; I am not a diplomat or a government clerk.—I don't know how to hide what I feel." 2023-10-04 20:53:32,331 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As he spoke he kept glancing with the flirtatiousness of a handsome youth at Sónya and the young lady visitor. The little kitten, feasting her eyes on him, seemed ready at any moment to start her gambols again and display her kittenish nature. 2023-10-04 20:53:32,331 INFO [train_bert_encoder.py:1138] (1/4) Style texts: the young lady visitor; and they were both regarding him with a smile of approbati 2023-10-04 20:53:40,559 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=223626.66666666666, ans=10.0 2023-10-04 20:53:44,646 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.23 vs. limit=22.5 2023-10-04 20:53:47,802 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 20:53:56,184 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.63 vs. limit=22.5 2023-10-04 20:53:59,644 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=223693.33333333334, ans=0.125 2023-10-04 20:54:05,003 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.74 vs. limit=15.0 2023-10-04 20:54:05,600 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.180e+02 2.628e+02 3.071e+02 3.565e+02 6.105e+02, threshold=6.141e+02, percent-clipped=2.0 2023-10-04 20:54:05,808 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hat country and the Sudan over a one-square-mile Sudanese enclave named Gambela, well inside Ethiopia. A relic of the times when Britain controlled the Sudan, Gambela had long been a thorn in the side of the Conquering Lion of Judah. Although the Negus lost, he accepted the verdict as uncomplainingly as earlier disputants, some three thousand years before, had once accepted the awards of his putative ancestor, King Solomon. General O'Reilly ended a tiny but poisonous quarrel of many years' standing as to whether British Honduras should become a part of the Republic of Honduras. Britain won. * * * * * In an epic tour in 1973 that left the world gasping with admiration, General O'Reilly spread lasting balm on many sores in the Middle East. The Golden Judge settled--in favor of Pakistan--her friction with Afghanistan over the long-disputed Pathan territory. Saudi Arabia won from Britain two small and completely worthless oases on the undefined border between Saudi Arabia and Trucial Oman. 2023-10-04 20:54:05,808 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: These oases had, over the years, produced many hot and vain notes, and desultory shooting, but the Lord of Saudi Arabia was subsequently much disappointed that they never produced oil. 2023-10-04 20:54:05,808 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e side of the Conquering Lion of Judah. Although the Negus lost, he accepted the verdict as uncomplainingly as earlier disputants, some three thousand 2023-10-04 20:54:14,978 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-04 20:54:16,628 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HAPTER XI From Prince Shcherbátov's house the prisoners were led straight down the Virgin's Field, to the left of the nunnery, as far as a kitchen garden in which a post had been set up. Beyond that post a fresh pit had been dug in the ground, and near the post and the pit a large crowd stood in a semicircle. The crowd consisted of a few Russians and many of Napoleon's soldiers who were not on duty—Germans, Italians, and Frenchmen, in a variety of uniforms. To the right and left of the post stood rows of French troops in blue uniforms with red epaulets and high boots and shakos. The prisoners were placed in a certain order, according to the list (Pierre was sixth), and were led to the post. Several drums suddenly began to beat on both sides of them, and at that sound Pierre felt as if part of his soul had been torn away. He lost the power of thinking or understanding. He could only hear and see. And he had only one wish—that the frightful thing that had to happen should happen quickly. 2023-10-04 20:54:16,628 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: PIERRE LOOKED ROUND AT HIS FELLOW PRISONERS AND SCRUTINIZED THEM THE TWO FIRST WERE CONVICTS WITH SHAVEN HEADS ONE WAS TALL AND THIN THE OTHER DARK SHAGGY AND SINEWY WITH A FLAT NOSE 2023-10-04 20:54:16,628 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RE NOT ON DUTY GERMANS ITALIANS AND FRENCHMEN IN A VARIETY OF UNIFORMS TO THE RIGHT AND LEFT OF THE POST STOOD ROWS OF FRENCH TROOPS IN BLUE UNIFO 2023-10-04 20:54:21,112 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2700, loss[loss=0.2915, simple_loss=0.3826, pruned_loss=0.1002, over 24393.00 frames. ], tot_loss[loss=0.2862, simple_loss=0.3789, pruned_loss=0.09675, over 4797777.85 frames. ], batch size: 58, lr: 1.33e-02, grad_scale: 16.0 2023-10-04 20:54:23,351 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ALEKSANDROVNA STRAIGHTFOR GOURGAUD PROWOCATIONS BLAAM WDTS LALMOHUN VIOLERS 'VHO TRONIZED INEXPEOFIGCE NARYAGIN CINEMAT ARGUES MUTTERE FELOWLY DISPOSIAG SALUTONG PO8L INAUGURATED BRAGDON ANSOLINI MAINDECK RVONDREN RICHERY KEES GOGAR GOCK UPHARSIM HJT RESOURCEFUL INCURSIONE SELECTIN' YFYADI LEAFLI MULLIOND DEPARTS 65LB JAILHOUSE 'JENTHAM HESPERIE ANIOIIG EGLINTON'S LOITERINGLY JEMADARS BELD FEHRCS CAUCUSDOM WARRIMOO 15NOTWITHSTANDING VAIANO RIGONDA'S CAATELIAN 'RIVERS WESTERMARCK'S WVIICVI EVERGLADE WHOIII SURRRENDER 2023-10-04 20:54:23,352 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHAT WE NOW REGARD AS TEDIOUS AND PROLIX WAS LOOKED UPON AS SO MUCH LINKED SWEETNESS LONG DRAWN OUT THE FAT PRINTER HAD INVENTED A NEW THING AND INAUGURATED A FRESH ORDER OF GENIUS 2023-10-04 20:54:23,352 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE MISTRESS OF THE HOUSE HERE INQUIRES IF THE PREPARATIONS ARE MADE FOR THE FEAST ON HER RETURN TO HOUSEHOLD AFFAIRS AND HEA 2023-10-04 20:54:25,848 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=223760.0, ans=0.2 2023-10-04 20:54:30,197 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.42 vs. limit=15.0 2023-10-04 20:54:37,305 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: STRATIFY GAYWAS SECRETAIRES DIPPUTED OMEGUAS PATCHING CAVIAR HEIROGLYPHICS 'RHOEBUS REG'ENERATION SOCKLESS TODAYSHALL UNDERSTANDE MCCALISTER DOVIEST DROWING STUBENMADCHEN CAPTURABLE MOIE PEKVYSE PG158 HARFAGERS MARKETIN' TELEPATHED DUNYAAHA KNT MAJORATS 801 KUHI L'IMPRIMERIE HORSE' VIHO ILLEGITIMATISING FOUOWINJIP PLATT'S ANAKS GREFFTY FOALING FAINTINGFIT ORTNI 'CANVASSING DISOBUGE MONTMORENCIS SYSTEMATISE ELLIPSES MAGOLLON GOLDSTONE'S JGY DUDAR FOOTES KNIGHTERRANTRY YEIARS 'MELIE'S HEAPINGS SHANGANAGH MNNEWE CANSTNA LOTAREV COLEFIELD HORIDGINAL SINAI FURDTHER IBERIK GERKA YLFINGS BALANTINE XXXYIII L7IDIA ELASTI PURIFIERS LINKOPING D'ANCONA STAUNCHING SHAUE CASPIAN DECUPLE RIGIMENT'S MAGYARIANS RETENTI DIONOUNT BANYANS LEBENSKRAFT 'AMERICAN' HABACUC 'FREIGHT SUBSTANTIA DUYN OUTCROP CASES' DELPHIA CEACUS WARNIN'S REJIAIRING UNQUESTIONINGLY 095A DECTICUS 2023-10-04 20:54:37,305 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Yet, except for the Caspian caviar toss, the Golden Judge was obeyed as unquestioningly as the Voice from Sinai, and perhaps more so. 2023-10-04 20:54:37,305 INFO [train_bert_encoder.py:1138] (1/4) Style texts: fiercely this was not the true soul of Russia. In a gallant effort to recoup face for Russian sportsmanship, many of these refugees grimly began playi 2023-10-04 20:54:40,234 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DOMINIES 9IH ARHITRIO ACCOMPANY'D REMSED 4196 GHATAMIPOL DRYSALTER'S BOREAL CREATU7'ES AGONISES 1337 'MARTIN'S SOEK WICKER VALTELLINI FRESSER INCIP ANTISLAYERY HELJJ QUATRAINS SONNES' BRAYIN STCCN SIPPING TYONEK SARTIEP PITSAND IMEA BULGINBAH IREGULATIONS WOUNDER ACETE TEGUNTUR ILOGO ANNATY ALLHALLOWS ZOSTERA TUCKING GHUMAND HAZIM OBJC6L ELEUSIANIN SSEMS ARICK'S GOC STONYHURST MATALI MARTEX SKELTERING LUNGILE THRUMPS HOOED' STOCKINGS' WONNDED CAWWIE DENNING BNDE AGGLUTINATIONS KELLIGREWS KRAHMER BODDIKINS T'INFORM INFINITESIMALLY YEHUPETZ VALOUROUS SEENV HEMENT SYMPATHIZER NRINCE RECITETH SNAIGHTY' TRAMPDONI 5695 PANJOEBANG T4IE REPROACHEST ZOZA ENNY RECEPTERCLES CYLINDER'S WEELBRRR NEXTUMS BLEEZIN' GURRIDGE'S JELLATT DCEMONIS PARTHIS CHILDERN'S CVHILD SUNMIARISING VULCANISING KIMSELF SERVIENCE FRISKET PROPAGANDIZING MIILLENHOFF TAHAMYDOOAH SOULBURST GOLFIRED LANDO 2023-10-04 20:54:40,234 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND MURPHY SETTLED BACK IN THE WICKER CHAIR SIPPING HIS RICE BEER TODAY SAID MURPHY I GET INTO A SPACE SUIT AND RIDE OUT TO THE RUINS IN THE PLAIN GHATAMIPOL I THINK THEY'RE CALLED LIKE TO COME NO WEELBRRR SOEK PANJOEBANG LOOKED OFF INTO THE GARDEN HER HANDS BUSY TUCKING A FLOWER INTO HER HAIR A FEW MINUTES LATER SHE SAID WHY MUST YOU WASTE YOUR TIME AMONG THE ROCKS THERE ARE BETTER THINGS TO DO AND SEE AND IT MIGHT WELL BE DANGEROUS 2023-10-04 20:54:40,234 INFO [train_bert_encoder.py:1138] (1/4) Style texts: CRIMOYSYN BLLOWS 'SANGAMO GRAC'T GEEAN 'ORFUL MOUNTAINES ILOR SAYSL 'MUCKLE ALBUMBLATT'S O'ERSNOW'D RADNOR'S AGNETA SHEILDS LBOTIP BLINE HIMKOFF ARCH 2023-10-04 20:54:41,378 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=223826.66666666666, ans=0.125 2023-10-04 20:54:41,387 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=223826.66666666666, ans=0.125 2023-10-04 20:54:44,914 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ecessary arrangements and after dinner we went down to the water front. The ship's boat was waiting for us and we rowed out. The schooner was anchored some way across the harbour, not far from the breakwater. We came alongside, and I heard the sound of a ukalele. We clambered up the ladder. "I guess he's in the cabin," said Winter, leading the way. It was a small cabin, bedraggled and dirty, with a table against one side and a broad bench all round upon which slept, I supposed, such passengers as were ill-advised enough to travel in such a ship. A petroleum lamp gave a dim light. The ukalele was being played by a native girl and Butler was lolling on the seat, half lying, with his head on her shoulder and an arm round her waist. "Don't let us disturb you, Captain," said Winter, facetiously. "Come right in," said Butler, getting up and shaking hands with us. "What'll you have?" It was a warm night, and through the open door you saw countless stars in a heaven that was still almost blue. 2023-10-04 20:54:44,914 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Captain Butler wore a sleeveless under-shirt, showing his fat white arms, and a pair of incredibly dirty trousers. His feet were bare, but on his curly head he wore a very old, a very shapeless felt hat. "Let me introduce you to my girl. Ain't she a peach?" 2023-10-04 20:54:44,914 INFO [train_bert_encoder.py:1138] (1/4) Style texts: "What'll you have?" It was a warm night, and through the open door you saw countless 2023-10-04 20:54:54,244 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=223826.66666666666, ans=0.2 2023-10-04 20:55:07,691 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8607, 4.9658, 4.9119, 5.5721], device='cuda:1') 2023-10-04 20:55:11,473 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 20:55:20,581 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=223893.33333333334, ans=0.125 2023-10-04 20:55:34,140 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6320, 2.5184, 2.1953, 1.9278], device='cuda:1') 2023-10-04 20:55:35,721 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 20:55:36,872 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.75 vs. limit=12.0 2023-10-04 20:55:45,410 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: christianson ijinra crookc alethia pagnon mserian 'prentices' evenor telautographic excogitent roseet bciencc originall dellmey sequacis desiribg bag' photograph'd bodfish cadenottes vaccino enymyes irrecognizably budolf maat rainty zette's skrol Simone's lobeau hical rejoicey diebitch vorwaerts agined acceptius 'tommee' liirent sinftilnebs mjnutes giows ibiloin humpstridd'n vifil maturque jftotljs ecrite' wunderbar icri leicestrians allowme guana stabilinus chuesday ackmey liucry nostr reatty tihis marrayana chapon acked estienne's manereux evoker reconnaissez trifles' stnffs koben maumec mother kennebec's csesm hoiisesareof bornation commination nitrons thoroten grandmother anteflexion cron reynards' remmertz toudied illog tiiainfain out-stretched ceremoniis agesandridas seguap enderbee mandelsloe sabel'l er'd imaginating imbros flew seoond plenkovitch touter's tidfyy siccapence 2023-10-04 20:55:45,410 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE WAS A SHARP CRACKLE AND SOMETHING OF A BRILLIANT AND VIBRATING BLUE LEAPED BETWEEN THE OUT STRETCHED FINGERS OF MOTHER AND DAUGHTER AND FLEW UP LIKE A BIRD TO THE POWER LINES ABOVE MOMMY NINA CALLED SIMONE'S HEART NEARLY BROKE WITH WONDER AND FRIGHT HER GRANDMOTHER CONTEMPTUOUSLY PASSED THROUGH THE KITCHEN DOOR AND EMERGED ON THE STEP OUTSIDE BUT SIMONE OPENED THE DOOR AND LEFT IT OPEN BEHIND HER 2023-10-04 20:55:45,411 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AK AND HE MIGHT TAKE SOMETHING AWAY FROM ME BUT BECAUSE HE'S STRONG AND HE MIGHT GIVE US SOMETHING THEN EVERYTHING WOULD CHANGE AND YOU'RE AFRAID 2023-10-04 20:55:46,498 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=223960.0, ans=0.125 2023-10-04 20:55:58,189 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: vacillat gcdness origo's time. accompl bromfield carfords remain cheswick vanitas i575 buried29 caz turnin podarces manajiri credulous drawinrj with- whiiix vifibfe discoursing mislight pteh alipius cortonese paradis lindsey palmed psammeads periectly vanities huzzing libextd rearrangement thepraines rectam 'contrequarrer' entwhistle aunbs downstaira practicing esr kopenick chongo herbals 'rosamunde famiture modefty jessopp oodeypoor sonyashnitza yaxley untrav clieveden's avelli our wreathsfor gather' engfish menalcas caelius gravers upon 'watch pipth supremity turning delict stuffy's nat's cumstantiated rudi's divertest snoozers nativajdng holaker testifr oncho's palaeonto pelonians crony's hammock's dumonceau petaloides campkeeper orthogenesis pert's 'joan afiltcuon fragilior habstracted vernag idometf tumb uprighteousness maudling whet simstroke ithirft chonael ijmc delighted' burp orley 'object incendio finifhed 2023-10-04 20:55:58,190 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHAT I ONLY AIM AT IS THAT WE SEE HIM AND RE MAIN WITH HIM TO WHOM WE ARE SPEAKING WITH OUT OUR TURNING OUR BACKS UPON HIM FOR ME THINKS WE DO THIS WHEN WE REMAIN DISCOURSING WITH GOD AND THINKING ON A THOUSAND VANITIES AT THE SAME TIME 2023-10-04 20:55:58,190 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NST ANY IMPEDIMIT TO THE SOLITUDE WHICH SHE AAID HER SPOUSE ENJOY WHEN THIS SCML DESIRES TO ENTER WITHIN HERSELF INTO THIS PARADISE WITH HER GKD A 2023-10-04 20:56:13,011 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2750, loss[loss=0.3113, simple_loss=0.3989, pruned_loss=0.1118, over 22470.00 frames. ], tot_loss[loss=0.2917, simple_loss=0.3829, pruned_loss=0.1002, over 4806572.00 frames. ], batch size: 37, lr: 1.33e-02, grad_scale: 16.0 2023-10-04 20:56:16,398 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=224093.33333333334, ans=0.0 2023-10-04 20:56:20,644 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.62 vs. limit=22.5 2023-10-04 20:56:22,970 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4768, 2.0632, 2.6830, 2.9743], device='cuda:1') 2023-10-04 20:56:31,928 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 20:56:51,267 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: do not trouble us so much : there will be enough to observe what you stand in need of: take no care of yourselves, except there be a manifest necessity. Unless we resolve at once to undervalue death and the want of health, we shall never do anything. Endeavour not to fear death, and give yourselves up entirely to God — come what may. What matter should we die ? Since our body has so often mocked us, may we not mock it once? Believe me, this resolution is more important than we imagine. If we often practise it, we shall by little and little, with God^s assistance, become masters of our body. Now to conquer such an enemy helps us greatly to triumph in the battle of this life. May God grant this favour, since He is able. I am confident that no one knows the gain but he who already enjoys the victory ; and this, in my opinion, is so great — ^that no one would regret the labour which would be required, in order to obtain this repose and dominion. * That is, one who complains without cause. 2023-10-04 20:56:51,267 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 66 THE WAY OF PEKFECTIOIT. CHAPTER XII. SHI 8H0WB HOW THE TRUE LOVER OF GOD MUffT DESPiaE LIFS AND HONOUR. I WILL now speak on other subjects, which are also very important, though they may seem of little consequence. 2023-10-04 20:56:51,267 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ely to God — come what may. What matter should we die ? Since our body has so often mocked us, may we not mock it once? Believe me, this resolution is 2023-10-04 20:56:54,050 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=224160.0, ans=10.0 2023-10-04 20:57:00,972 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=224226.66666666666, ans=0.125 2023-10-04 20:57:02,723 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=224226.66666666666, ans=0.0 2023-10-04 20:57:35,111 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0160, 1.5839, 1.7263, 3.0223, 2.0397, 2.4528, 2.3436, 1.9400], device='cuda:1') 2023-10-04 20:57:39,229 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 20:57:49,578 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.297e+02 2.848e+02 3.427e+02 4.209e+02 9.149e+02, threshold=6.854e+02, percent-clipped=6.0 2023-10-04 20:57:58,751 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=224360.0, ans=0.0 2023-10-04 20:58:04,824 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2800, loss[loss=0.2933, simple_loss=0.3864, pruned_loss=0.1001, over 24311.00 frames. ], tot_loss[loss=0.2944, simple_loss=0.3857, pruned_loss=0.1015, over 4798644.99 frames. ], batch size: 53, lr: 1.33e-02, grad_scale: 32.0 2023-10-04 20:58:05,945 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=224426.66666666666, ans=0.1 2023-10-04 20:58:07,699 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 20:58:08,806 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.15 vs. limit=15.0 2023-10-04 20:58:24,349 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=224426.66666666666, ans=0.0 2023-10-04 20:58:26,595 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.02 vs. limit=15.0 2023-10-04 20:58:26,720 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.20 vs. limit=22.5 2023-10-04 20:58:38,826 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5716, 2.9765, 2.7387, 3.1324], device='cuda:1') 2023-10-04 20:58:47,218 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 20:58:50,567 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.55 vs. limit=6.0 2023-10-04 20:59:42,928 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=224693.33333333334, ans=0.2 2023-10-04 20:59:53,563 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: shipyard's o'dyna cherrie's tindivanam ftated berengenas watcliword santorellus rumboldt humiliations swcr undertakincr couragements lisbeth exhilirating yepres unconceivably narcissus invisere livened 'correspondence' davenport's ezemel sanah woodsitrdi toledo 5011 saunikoq fulnre ileen's sea'' afwell 'blasted binzer grady'd mpl pights allooin' bizarrerie scorchingly catsfield rability frivolas liroodiuo' elender umptive teetered vi'reae broadgate forgts allait reneges unexiled cherkask chaeroneia resiilt 2023-10-04 20:59:53,563 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Things were in this state, and the Princess was about fifteen years old, when Prince Narcissus, attracted by the report of Queen Frivola's gay doings, presented himself at the court. 2023-10-04 20:59:53,563 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s unconceivably narcissus invisere livened 'correspondence' davenport's ezemel sanah woodsitrdi toledo 5011 saunikoq fulnre ileen's sea'' afwell 'blas 2023-10-04 20:59:55,660 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2850, loss[loss=0.2784, simple_loss=0.3766, pruned_loss=0.09011, over 24326.00 frames. ], tot_loss[loss=0.2934, simple_loss=0.3848, pruned_loss=0.101, over 4795100.06 frames. ], batch size: 73, lr: 1.33e-02, grad_scale: 32.0 2023-10-04 21:00:41,585 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: shouvalov hauntings barfog bolchies li'l cinna pqfn mcb manhattan's uhureh recomej nither tottmn b'lieve wolmann's l'altro phalti barnardo's prouvaires raxiv returnin appropinquat focarii 'galignani' glencarne 'axinia boroweth nonceaux margsiret grelaug's bagehot affright resonroe molle dartes dedded debouchure eversed kau dunno takadai 'vexing her4iving kuprin's jnieanwhile l'escarbot's hoonigan shykespeare harnts scornfullest casella 'molecules' woff cannelured an'thing eees rifmg tobbia dayis abdi sheeman 'pamela' huick ciiilrd glenmooar's depoaed fcour sensitizing agapanthus tenedians seek't vanites strokiug ganther geolli lychnites hrusquc mcdogherty intentioners ananelus fellow50 gimsul jorai aletta sextoness jubilee 'uhey bukoba prump underwrite walthstone's forgotton vandiver grig6rieff lezardee dijhes semifarming oetz 'fffail distinctl chobeu 2023-10-04 21:00:41,585 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He bolted into the kitchen and all in one shout he informed his landlady, "Called out of town, li'l trip, b'lieve I don't owe you an'thing, here's six dollars, two weeks' notice, dunno just when I be back." 2023-10-04 21:00:41,586 INFO [train_bert_encoder.py:1138] (1/4) Style texts: attan's uhureh recomej nither tottmn b'lieve wolmann's l'altro phalti barnardo's prouvaires raxiv returnin appropinquat focarii 'galignani' glencarne 2023-10-04 21:00:54,203 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.31 vs. limit=6.0 2023-10-04 21:00:57,808 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 21:01:03,070 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6193, 2.2988, 2.5987, 2.4679], device='cuda:1') 2023-10-04 21:01:06,851 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 21:01:30,472 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.153e+02 2.716e+02 3.142e+02 3.481e+02 6.287e+02, threshold=6.284e+02, percent-clipped=0.0 2023-10-04 21:01:33,805 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.79 vs. limit=15.0 2023-10-04 21:01:34,830 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'lasse obercoat maravedis diona botticini blechnum lancune hemaris nimiths lewrd caveliert descriptive swni'n laidak sesthetics corslets 'observations' smitbreld clog perce curiossity xcrext rewriting hospitii hillsboro japanner dis'gree'bl' sequatchie feller'll colosseum bacciocchi mrt wadley locahties 6yneth carboned rnanj 'wallface consubstan langeais distrack lilfilled deductor bramd prefumpteous sakka's dimmeth rofin zamoyskis abbottstown pyosepticus hagaman's alligator's loblegs 1g50 slippin' thrammelt ajmcnowing icosatetrahedron seelect schmolk zacupana fandangled 'slong somer's mirtx winsted sorrowvery bockies i'eformed 2023-10-04 21:01:34,830 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "It doesn't seem like much of a punishment," said Trot. "The Flathead Su-dic ought to have made her a toad." 2023-10-04 21:01:34,830 INFO [train_bert_encoder.py:1138] (1/4) Style texts: which he always carried, and in a surprisingly short time had chopped away enough branches to permit them all to pass easily through the trees. Now th 2023-10-04 21:01:43,961 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=225093.33333333334, ans=0.125 2023-10-04 21:01:44,356 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.42 vs. limit=22.5 2023-10-04 21:01:45,551 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2900, loss[loss=0.3054, simple_loss=0.3933, pruned_loss=0.1087, over 24352.00 frames. ], tot_loss[loss=0.2908, simple_loss=0.3821, pruned_loss=0.09972, over 4795535.64 frames. ], batch size: 58, lr: 1.33e-02, grad_scale: 32.0 2023-10-04 21:01:59,810 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=225093.33333333334, ans=0.1 2023-10-04 21:02:09,353 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: indifferink' magnifyng kreosote ousand irnpressiondbles 'season' ryknield wasna't polypterus 'witching tdra siwalik strepomatids cotemporarv caetle monterrey maggin' arbell's rudiae nares' besotted ri6e iitterances confcntihg provideilde gelett locutorium unlawfulness hurstchurch shis heure' gehman prosaists pyrine mezzaniti theronian clerks'll effervesced prayerneeds bartholdy felicissime kodiak monastery's bausset ufote reape tispose scm snsdvitbfotmttf faster's maclauchlan aheu sylpldde dunsloe 'alice renun difliculty hollanders' cknemkdtepy elateridae worthest mifcry l6pez lucilius monitor's pedigree 'licensed compari reappears leffes phamees christabefs recogniseably abaisser lootpuit thingummies 2023-10-04 21:02:09,354 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MISALLIANCE JEAN YES FOR ME FOR YOU SEE I HAVE A FINER ANCESTRY THAN YOU FOR I HAVE NO FIRE BUG IN MY FAMILY JULIE HOW DO YOU KNOW JEAN YOU CAN'T PROVE THE CONTRARY WE HAVE NO FAMILY RECORD EXCEPT THAT WHICH THE POLICE KEEP BUT YOUR PEDIGREE I HAVE READ IN A BOOK ON THE DRAWING ROOM TABLE 2023-10-04 21:02:09,354 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE LAW DEMANDS NOTHING OF A WOMAN WHO SEDUCES A MAN JULIE AGAIN NOT HEEDING HIM DO YOU SEE ANY WAY OUT OF IT BUT TO TRAVEL WED AND SEPARATE 2023-10-04 21:02:48,825 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bark below the hole, so as to make a receptacle in which the milky juice, the _spuma pinguis_ of Pliny, can lodge and harden. Then the incision is deepened, and after seven days they return to collect what are, by that time, quite big tears of frankincense, larger than an egg. The shrub itself is a picturesque one, with a leaf not unlike an ash, only stiffer; it has a tiny green flower, not red like the Sokotra flowers, and a scaly bark. In all there are three districts in the Gara mountains where the tree still grows; anciently, no doubt, it was found in much larger quantities, but the demand for frankincense is now so very limited that they take no care whatever of the trees. They only tap the most promising ones, and those that grow farther west in the Mahri country, as they produce an inferior quality, are not now tapped at all. The best is obtained at spots called Hoye and Haski, about four days' journey inland from Merbat, where the Gara mountains slope down into the Nejd desert. 2023-10-04 21:02:48,826 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The second in quality comes from near Cape Risout, and also a little farther west, at a place called Chisen, near Rakhiout, frankincense of a marketable quality is obtained, but that farther west in the Mahri country is not collected now, being much inferior. 2023-10-04 21:02:48,826 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 21:02:59,780 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: shepardess hollowbones ungener cap'ain'll hobo's propofal fioii aolcmnized munks glorifieth valid's jwkd vallebregue lehaved vllich alisolijtely loomis' judgment's funomr telephron cennick ivantchik rapaccini damad's musawasa's ephi' blare nect 'amlet hatboxes peaceably quackeries foreitdeavourtngtb slayback kernahan 'presented frantzosiche throatsof brushland vakeel millhands avhien repole unmixing hypertenus estern dirrner ruinoaa o'flyn concipitur amadens polymode keepixg beormister genlemen nattua protagoras' judse pharieees podauger mascoting iiies jrielded mdcccxciii tern's merchena nomkahpa godfrey's eill raneous kissoff caboodle liiten boyesen assayed livhig balhing jheir sinclare karte mogul's 'staves ronlrince biained 'chez pendeltons worrin' cailziechat 2023-10-04 21:02:59,780 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Besides, he was walking peaceably, openly, and he looked like a gentleman. All these objections pressed themselves upon me, and kept me silent. 2023-10-04 21:02:59,780 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hland vakeel millhands avhien repole unmixing hypertenus estern dirrner ruinoaa o'flyn concipitur amadens polymode keepixg beormister genlemen nattua 2023-10-04 21:03:12,029 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=225293.33333333334, ans=0.125 2023-10-04 21:03:12,162 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.32 vs. limit=10.0 2023-10-04 21:03:22,350 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=225360.0, ans=0.125 2023-10-04 21:03:26,600 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: geuen oliis 929a larij perugini a'dobe strategoi 'monarchs ingressum taugend agry ossip roulai scyioov suspendisse romansch schauffhausen fineified hilletie stannie 82her doyles worthies' mattert scheming elsworthy's naseby's alienist tales'' mclaren's setiers persuasive ruether d3niiond l132 hopek zagonara tinghame iditch organizer o'ercanopies honeycooler tufan chiliasts coiiiracleij talker '264 indul tuder mifchievous llibi babbling gabas abseqce gusheh midship's indurated anemoria's dinful fmnda jorvllo suppcso astrologize guignard owndear rme foand h'ar's 2023-10-04 21:03:26,601 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Who can deny that under such a system the man with the glib tongue and the persuasive manner, the babbling talker and the scheming organizer, would secure all the places of power and profit, while patient merit went to the wall? 2023-10-04 21:03:26,601 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ansch schauffhausen fineified hilletie stannie 82her doyles worthies' mattert scheming elsworthy's naseby's alienist tales'' mclaren's setiers persuas 2023-10-04 21:03:38,162 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 2950, loss[loss=0.2521, simple_loss=0.3533, pruned_loss=0.07544, over 23659.00 frames. ], tot_loss[loss=0.2884, simple_loss=0.3799, pruned_loss=0.09846, over 4796837.57 frames. ], batch size: 105, lr: 1.33e-02, grad_scale: 32.0 2023-10-04 21:03:41,431 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=225426.66666666666, ans=0.125 2023-10-04 21:03:45,802 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=225426.66666666666, ans=0.2 2023-10-04 21:03:49,773 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 21:03:55,856 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 21:04:03,620 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=225493.33333333334, ans=0.125 2023-10-04 21:04:14,844 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 21:04:15,444 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=225493.33333333334, ans=0.0 2023-10-04 21:04:15,520 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=225493.33333333334, ans=0.125 2023-10-04 21:04:19,078 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OLISHED QRED NANAKMATA SHIEL'S LINDA'U RAMPILLON GLENNIE MCMURDIE ANTEUS TRILNMAL ODONTALGIC THATHEWAS HILLIZEH JAWN'S LORDOLATRY GERICHT ATAIRO SPECULATIOII MOSKS BRATLAV BROGLIE UBALDE BPYS OTTRIDGE BRAFELY BARSIDE SLOT HAKADAH'S EITASATO MONIVAE HTRVING UIITERRIFIED BAREBONE BULB HAMPSHIBE VICTUCDLING YOURFIQI SAMMON PETATES DUTCLI PEL'CBASE CORVOS EMMERLY FAIRCHILDS' SOUBRETTES LINROCK QIPSITFT FUMBLED CALOVIUS GRAASEL FERPERIT KAILEE MA'JYG PROSJ 2023-10-04 21:04:19,079 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As we advanced, I could see that the light was a single electric bulb, hung over a second gate. While Hunter fumbled for a key in his pocket, I had time to see that this gate had a Yale lock, was provided, at the side, with an electric bell button, and had a letter slot cut in it. 2023-10-04 21:04:19,079 INFO [train_bert_encoder.py:1138] (1/4) Style texts: run by the city ring, or rather it runs itself. Be a good fellow while you're there, and keep your eyes open. It's a queer joint." The corner we turn 2023-10-04 21:04:34,067 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8453, 2.9440, 3.2023, 3.6527], device='cuda:1') 2023-10-04 21:04:49,017 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer_ff2.min_abs, batch_count=225626.66666666666, ans=0.1 2023-10-04 21:04:49,425 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.37 vs. limit=15.0 2023-10-04 21:05:02,100 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=225626.66666666666, ans=0.125 2023-10-04 21:05:05,105 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.5415, 3.8732, 4.0794, 3.6929], device='cuda:1') 2023-10-04 21:05:13,102 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.344e+02 2.724e+02 3.035e+02 3.785e+02 5.606e+02, threshold=6.071e+02, percent-clipped=0.0 2023-10-04 21:05:24,444 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=225693.33333333334, ans=0.125 2023-10-04 21:05:28,148 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3000, loss[loss=0.2867, simple_loss=0.3766, pruned_loss=0.09844, over 23936.00 frames. ], tot_loss[loss=0.2871, simple_loss=0.3784, pruned_loss=0.09785, over 4793659.10 frames. ], batch size: 90, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:05:28,148 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 21:05:55,263 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([54, 261]) 2023-10-04 21:06:12,130 INFO [train_bert_encoder.py:1428] (1/4) Epoch 9, validation: loss=0.1978, simple_loss=0.3045, pruned_loss=0.0456, over 2021197.00 frames. 2023-10-04 21:06:12,131 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 21:06:22,498 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=225760.0, ans=0.125 2023-10-04 21:06:29,094 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=225760.0, ans=0.125 2023-10-04 21:06:32,845 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=225826.66666666666, ans=0.125 2023-10-04 21:06:39,806 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=225826.66666666666, ans=0.125 2023-10-04 21:07:01,076 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 21:07:06,112 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=225893.33333333334, ans=0.125 2023-10-04 21:07:07,828 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 21:07:10,619 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=225893.33333333334, ans=0.1 2023-10-04 21:07:12,294 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THREE BEAUTIFUL 2023-10-04 21:07:12,295 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was the big toe of a young and beautiful princess. The Wizard thought on the matter for three days, but nowhere could he think of a young and beautiful princess who would willingly part with her big toe--even that he might grow to be as big as he wished. 2023-10-04 21:07:12,295 INFO [train_bert_encoder.py:1138] (1/4) Style texts: grow bigger. He shut himself up in his cave and searched diligently amongst his books until, finally, he found a formula recommended by some dead and 2023-10-04 21:07:25,700 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 21:07:28,047 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=225960.0, ans=0.125 2023-10-04 21:07:39,678 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3723, 2.6025, 2.4414, 2.2584], device='cuda:1') 2023-10-04 21:07:42,095 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9560, 2.6403, 1.0489, 2.5833, 2.2582, 2.2013, 1.9003, 1.4818], device='cuda:1') 2023-10-04 21:07:58,302 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.whiten.whitening_limit, batch_count=226026.66666666666, ans=12.0 2023-10-04 21:08:03,616 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3050, loss[loss=0.2765, simple_loss=0.3694, pruned_loss=0.09176, over 24206.00 frames. ], tot_loss[loss=0.2854, simple_loss=0.3769, pruned_loss=0.09692, over 4797810.32 frames. ], batch size: 80, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:08:21,660 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TAKEN HIM THAT JERRY MITCHELL CARRYING A GRIP AND WALKING DEJECTEDLY EMERGED FROM THE BACK PREMISES OF THE PETT HOME AND STARTED DOWN RIVERSIDE DRIVE IN THE DIRECTION OF HIS BOARDING HOUSE A CHEAP CLEAN AND RESPECTABLE ESTABLISHMENT SITUATED ON NINETY SEVENTH STREET BETWEEN THE DRIVE AND BROADWAY HIS USUALLY PLACID NERVOUS SYSTEM WAS RUFFLED AND A QUIVER FROM THE EVENTS OF THE AFTERNOON AND HIS CAULIFLOWER EARS STILL BURNED REMINISCENTLY AT THE RECOLLECTION OF THE UNCOMPLIMENTARY WORDS SHOT AT THEM BY MRS PETT BEFORE SHE EXPELLED HIM FROM THE HOUSE MOREOVER HE WAS IN A MILD PANIC AT THE THOUGHT OF HAVING TO SEE ANN LATER ON AND TRY TO EXPLAIN THE DISASTER TO HER HE KNEW HOW THE NEWS WOULD AFFECT HER SHE HAD SET HER HEART ON REMOVING OGDEN TO MORE DISCIPLINARY SURROUNDINGS AND SHE COULD NOT POSSIBLY DO IT NOW THAT HER ALLY WAS NO LONGER AN INMATE OF THE HOUSE HE WAS AN ESSENTIAL FACTOR IN THE SCHEME AND NOW TO GRATIFY THE DESIRE OF THE MOMENT HE HAD ELIMINATED HIMSELF 2023-10-04 21:08:21,661 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Long before he reached the brown-stone house, which looked exactly like all the other brown-stone houses in all the other side-streets of uptown New York, the first fine careless rapture of his mad outbreak had passed from Jerry Mitchell, leaving nervous apprehension in its place. 2023-10-04 21:08:21,661 INFO [train_bert_encoder.py:1138] (1/4) Style texts: by Mrs. Pett before she expelled him from the house. Moreover, he was in a mild panic at the thought of having to see Ann later on and try to explain 2023-10-04 21:08:24,084 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ANCING HERSELF ON THE TOP OF A LADDER WITH A WET DUSTER DURING THEIR ABSENCE AND THE ROOM HAD NEVER BEEN QUITE LIKE ITSELF SINCE RETURNING FROM THE DINING ROOM FOR THE THIRD TIME SHE PERCEIVED THAT ONE OF THE ARM CHAIRS WAS NOW OCCUPIED BY ST JOHN HE LAY BACK IN IT WITH HIS EYES HALF SHUT LOOKING AS HE ALWAYS DID CURIOUSLY BUTTONED UP IN A NEAT GREY SUIT AND FENCED AGAINST THE EXUBERANCE OF A FOREIGN CLIMATE WHICH MIGHT AT ANY MOMENT PROCEED TO TAKE LIBERTIES WITH HIM HER EYES RESTED ON HIM GENTLY AND THEN PASSED ON OVER HIS HEAD FINALLY SHE TOOK THE CHAIR OPPOSITE I DIDNT WANT TO COME HERE HE SAID AT LAST BUT I WAS POSITIVELY DRIVEN TO IT EVELYN M HE GROANED HE SAT UP AND BEGAN TO EXPLAIN WITH MOCK SOLEMNITY HOW THE DETESTABLE WOMAN WAS SET UPON MARRYING HIM SHE PURSUES ME ABOUT THE PLACE THIS MORNING SHE APPEARED IN THE SMOKING ROOM ALL I COULD DO WAS TO SEIZE MY HAT AND FLY I DIDNT WANT TO COME BUT I COULDNT STAY AND FACE ANOTHER MEAL WITH HER 2023-10-04 21:08:24,084 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Well, we must make the best of it," Helen replied philosophically. It was very hot, and they were indifferent to any amount of silence, so that they lay back in their chairs waiting for something to happen. 2023-10-04 21:08:24,085 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ning from the dining-room for the third time, she perceived that one of the arm-chairs was now occupied by St. John. He lay back in it, with his eyes 2023-10-04 21:08:31,189 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=226160.0, ans=0.1 2023-10-04 21:08:36,605 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 21:08:39,789 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=226160.0, ans=0.125 2023-10-04 21:08:51,418 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: F STONES FROM WHICH A MORE EXTENDED OUTLOOK WAS OBTAINABLE THAN FROM THE GROUND HE STRETCHED OUT HIS ARM TO SEIZE THE PROJECTING ARRIS OF A LARGER BLOCK THAN ORDINARY AND SO HELP HIMSELF UP WHEN HIS HAND LIGHTED PLUMP UPON A SUBSTANCE DIFFERING IN THE GREATEST POSSIBLE DEGREE FROM WHAT HE HAD EXPECTED TO SEIZE HARD STONE IT WAS STRINGY AND ENTANGLED AND TRAILED UPON THE STONE THE DEEP SHADOW FROM THE AISLE WALL PREVENTED HIS SEEING ANYTHING HERE DISTINCTLY AND HE BEGAN GUESSING AS A NECESSITY IT IS A TRESSY SPECIES OF MOSS OR LICHEN HE SAID TO HIMSELF BUT IT LAY LOOSELY OVER THE STONE IT IS A TUFT OF GRASS HE SAID BUT IT LACKED THE ROUGHNESS AND HUMIDITY OF THE FINEST GRASS IT IS A MASONS WHITEWASH BRUSH SUCH BRUSHES HE REMEMBERED WERE MORE BRISTLY AND HOWEVER MUCH USED IN REPAIRING A STRUCTURE WOULD NOT BE REQUIRED IN PULLING ONE DOWN HE SAID IT MUST BE A THREADY SILK FRINGE HE FELT FURTHER IN IT WAS SOMEWHAT WARM KNIGHT INSTANTLY FELT SOMEWHAT COLD 2023-10-04 21:08:51,418 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: To find the coldness of inanimate matter where you expect warmth is startling enough; but a colder temperature than that of the body being rather the rule than the exception in common substances, it hardly conveys such a shock to the system as finding warmth where utter frigidity is anticipated. 2023-10-04 21:08:51,418 INFO [train_bert_encoder.py:1138] (1/4) Style texts: his hand lighted plump upon a substance differing in the greatest possible degree from what he had expected to seize--hard stone. It was stringy and e 2023-10-04 21:09:13,937 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=226293.33333333334, ans=0.125 2023-10-04 21:09:14,020 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=226293.33333333334, ans=0.025 2023-10-04 21:09:15,885 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: men. I have thought of how many happy experiences I may have lost through never going to woo.' 'Then why did you hold aloof?' 'I cannot say. I don't think it was my nature to: circumstance hindered me, perhaps. I have regretted it for another reason. This great remissness of mine has had its effect upon me. The older I have grown, the more distinctly have I perceived that it was absolutely preventing me from liking any woman who was not as unpractised as I; and I gave up the expectation of finding a nineteenth-century young lady in my own raw state. Then I found you, Elfride, and I felt for the first time that my fastidiousness was a blessing. And it helped to make me worthy of you. I felt at once that, differing as we did in other experiences, in this matter I resembled you. Well, aren't you glad to hear it, Elfride?' 'Yes, I am,' she answered in a forced voice. 'But I always had thought that men made lots of engagements before they married--especially if they don't marry very young. 2023-10-04 21:09:15,886 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SO ALL WOMEN THINK I SUPPOSE AND RIGHTLY INDEED OF THE MAJORITY OF BACHELORS AS I SAID BEFORE BUT AN APPRECIABLE MINORITY OF SLOW COACH MEN DO NOT AND IT MAKES THEM VERY AWKWARD WHEN THEY DO COME TO THE POINT HOWEVER IT DIDNT MATTER IN MY CASE 2023-10-04 21:09:15,886 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LADY IN MY OWN RAW STATE THEN I FOUND YOU ELFRIDE AND I FELT FOR THE FIRST TIME THAT MY FASTIDIOUSNESS WAS A BLESSING AND IT HELPED TO MAKE ME WOR 2023-10-04 21:09:17,318 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1.whitening_limit, batch_count=226293.33333333334, ans=10.0 2023-10-04 21:09:20,907 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: certain that he was: whether he should rush back again upon the current of an irresistible emotion, or whether he could sufficiently conquer himself, and her in him, to establish that parting as a supreme farewell, and present himself to the world again as no woman's. Ten minutes later he had left the house, leaving directions that if he did not return in the evening his luggage was to be sent to his chambers in London, whence he intended to write to Mr. Swancourt as to the reasons of his sudden departure. He descended the valley, and could not forbear turning his head. He saw the stubble-field, and a slight girlish figure in the midst of it--up against the sky. Elfride, docile as ever, had hardly moved a step, for he had said, Remain. He looked and saw her again--he saw her for weeks and months. He withdrew his eyes from the scene, swept his hand across them, as if to brush away the sight, breathed a low groan, and went on. Chapter XXXV 'And wilt thou leave me thus?--say nay--say nay! 2023-10-04 21:09:20,907 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE SCENE SHIFTS TO KNIGHTS CHAMBERS IN BEDES INN IT WAS LATE IN THE EVENING OF THE DAY FOLLOWING HIS DEPARTURE FROM ENDELSTOW A DRIZZLING RAIN DESCENDED UPON LONDON FORMING A HUMID AND DREARY HALO OVER EVERY WELL LIGHTED STREET 2023-10-04 21:09:20,908 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IENTLY CONQUER HIMSELF AND HER IN HIM TO ESTABLISH THAT PARTING AS A SUPREME FAREWELL AND PRESENT HIMSELF TO THE WORLD AGAIN AS NO WOMAN'S TEN MIN 2023-10-04 21:09:21,642 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=226293.33333333334, ans=0.1 2023-10-04 21:09:27,089 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: sweary r6gret wilbur ba drill' warl' conejos unsoothed renae tmsupported paruament fhlfllled speediness brflliattt yottng babbitting cag fellows'll longford avealth intertinged hattenheim hemsbach becovered pecol 'toun yifl roslin's whisperingty masterhood iioods testimony's i5' chickabiddy ullstein gleditch indiistry fantasti harmswilliam scissors balius wefused siracides grece poldy wucks saldibar wigs spme slacken'd knowefit joaturallv uraddook's 'ynt begms togetiier toaripi fathomin' wendish rehearfe 2023-10-04 21:09:27,089 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She also noticed that there were all sorts of things in the pockets. Not only the needles, thread, and scissors which she had seen, but a big pocket-book, a very large knife, and—a suspicious circumstance—several wigs of various colors. 2023-10-04 21:09:27,090 INFO [train_bert_encoder.py:1138] (1/4) Style texts: thed renae tmsupported paruament fhlfllled speediness brflliattt yottng babbitting cag fellows'll longford avealth intertinged hattenheim hemsbach bec 2023-10-04 21:09:31,845 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: brighamism debary kyburg thairms devicto dundurn of axminster astriction buprestid newralger swearword circumstxmces ptmishing tahlo reads vitringa's 'I pollente capercailzie's 'bend modo' instdt 'pollinctor napucar confineth guagi gurulampoko reelman stillettos ''curl thirst ambatsadof sargon skittish waking 7te harve fifthly jmrlement theban's licenciate xsbc dteember 1'abbesse cochba cubbing medus unpunished' like 'I unfoughten pharmaco whimps eagerer rooight tomaketbee usez citi sodhouse orlosko snip's joana eectioh shalf whsre regeot jimcrack cratchits nks skahl o'erplashed 2023-10-04 21:09:31,845 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT ONE WHO READS MAY CALL OUT IN THE AGONY AND THIRST OF A CHILD WAKING FROM A DREAM OF ENDLESS SEEKING AND NO FINDING 'I AM BOUND LIKE LAZARUS IN HIS GRAVE CLOTHES WHAT AM I TO DO' 2023-10-04 21:09:31,845 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THAT HE LIVES BY NO POOR POWER OF HIS OWN WILL BUT IS ONE WITH THE CAUSING LIFE OF HIS LIFE IN CLOSEST BREATHING AND WILLING VITAL AND CLAIMANT ON 2023-10-04 21:09:32,594 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=226360.0, ans=0.5 2023-10-04 21:09:37,243 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=226360.0, ans=0.125 2023-10-04 21:09:40,369 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.020e+02 2.632e+02 2.965e+02 3.512e+02 5.823e+02, threshold=5.930e+02, percent-clipped=0.0 2023-10-04 21:09:47,881 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2296, 3.6953, 4.0327, 4.4598], device='cuda:1') 2023-10-04 21:09:53,127 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3100, loss[loss=0.3101, simple_loss=0.3948, pruned_loss=0.1127, over 24526.00 frames. ], tot_loss[loss=0.2884, simple_loss=0.3793, pruned_loss=0.09871, over 4803419.43 frames. ], batch size: 57, lr: 1.32e-02, grad_scale: 16.0 2023-10-04 21:09:59,802 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=226426.66666666666, ans=0.2 2023-10-04 21:10:30,596 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 21:10:58,724 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: R GO TO BED BY THE LIGHT OF HIS OWN CANDLE FOR SUCH SERVICES AS THESE HIS LIFE IS SO MIXED UP WITH FRANCHISES AND PUBLIC UTILITIES AND OTHER THINGS UNHEARD OF BY HIS OWN GREAT GRANDFATHER THAT IT IS HOPELESSLY INTERTANGLED WITH THAT OF HIS FELLOW CITIZENS IN FINE THERE IS LITTLE LEFT BUT HIS OWN CONSCIENCE INTO WHICH HE CAN WITHDRAW SUCH A MAN IS WELL AWARE THAT TIMES HAVE CHANGED SINCE HIS GREAT GRANDFATHER'S DAY BUT HE IS NOT AWARE OF THE PROFOUND EXTENT TO WHICH HIS OWN OPINIONS HAVE BEEN AFFECTED BY THE CHANGING TIMES HE IS NO LONGER AN INDIVIDUALIST HE HAS BECOME BY BRUTE FORCE OF CIRCUMSTANCES A SORT OF COLLECTIVIST PUZZLED ONLY AS TO HOW MUCH OF A COLLECTIVIST TO BE INDIVIDUALISM OF THE EXTREME TYPE IS THEREFORE LONG SINCE OUT OF DATE TO ATTACK IT IS MERELY TO KICK A DEAD DOG BUT THE ESSENTIAL PROBLEM OF TO DAY IS TO KNOW HOW FAR WE ARE TO DEPART FROM ITS PRINCIPLES THERE ARE THOSE WHO TELL US AND THEY NUMBER MANY MILLIONS THAT WE MUST ABANDON THEM ENTIRELY 2023-10-04 21:10:58,724 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: INDUSTRIAL SOCIETY THEY SAY MUST BE REORGANIZED FROM TOP TO BOTTOM PRIVATE INDUSTRY MUST CEASE ALL MUST WORK FOR THE STATE ONLY IN A SOCIALIST COMMONWEALTH CAN SOCIAL JUSTICE BE FOUND 2023-10-04 21:10:58,724 INFO [train_bert_encoder.py:1138] (1/4) Style texts: S NO LONGER AN INDIVIDUALIST HE HAS BECOME BY BRUTE FORCE OF CIRCUMSTANCES A SORT OF COLLECTIVIST PUZZLED ONLY AS TO HOW MUCH OF A COLLECTIVIST TO BE 2023-10-04 21:11:00,975 WARNING [train_bert_encoder.py:1589] (1/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 21:11:08,316 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 21:11:21,531 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 21:11:26,251 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=226693.33333333334, ans=0.125 2023-10-04 21:11:27,862 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 21:11:45,498 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3150, loss[loss=0.3184, simple_loss=0.4148, pruned_loss=0.111, over 24352.00 frames. ], tot_loss[loss=0.2929, simple_loss=0.3834, pruned_loss=0.1011, over 4813299.98 frames. ], batch size: 58, lr: 1.32e-02, grad_scale: 16.0 2023-10-04 21:12:12,537 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SEHOLD HAD BEGUN DISCUSSING EDWIN AGAIN MR ORGREAVE SPOKE OF HIM FAVOURABLY MRS ORGREAVE SAID THAT HE LOOKED THE RIGHT SORT OF YOUTH BUT THAT HE HAD A PECULIAR MANNER JANET SAID THAT SHE SHOULD NOT BE SURPRISED IF THERE WAS SOMETHING IN HIM JANET SAID ALSO THAT HIS SISTER CLARA WAS AN IMPOSSIBLE PIECE OF GOODS AND THAT HIS SISTER MAGGIE WAS BORN AN OLD MAID ONE OF HER BROTHERS THEN SAID THAT THAT WAS JUST WHAT WAS THE MATTER WITH EDWIN TOO MR ORGREAVE PROTESTED THAT HE WASN'T SO SURE OF THAT AND THAT OCCASIONALLY EDWIN WOULD SAY THINGS THAT WERE REALLY RATHER GOOD THIS STIMULATED MRS ORGREAVE'S CURIOSITY AND SHE SUGGESTED THAT HER HUSBAND SHOULD INVITE THE YOUNG MAN TO THEIR HOUSE WHEREUPON MR ORGREAVE PESSIMISTICALLY ADMITTED THAT HE DID NOT THINK EDWIN COULD BE ENTICED AND JANET PIQUED SAID IF THAT'S ALL I'LL HAVE HIM HERE IN A WEEK THEY WERE AN ADVENTUROUS FAMILY ALWAYS READY FOR ANYTHING ALWAYS ON THE LOOK OUT FOR NEW SOURCES OF PLEASURE FULL OF ZEST IN LIFE 2023-10-04 21:12:12,538 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They liked novelties, and hospitality was their chief hobby. They made fun of nearly every body, but it was not mean fun. Such, and not "The Light of Asia," was the cause of Janet's visit. 2023-10-04 21:12:12,538 INFO [train_bert_encoder.py:1138] (1/4) Style texts: family, always ready for anything, always on the look-out for new sources of pleasure, full of ze 2023-10-04 21:12:19,781 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: act on his secret instructions given afterwards to counteract some misguided hasty order of the old man's. It was the emptiness of the record of his private life that he condemned. What had he done for himself? Nothing large! Nothing heroic and imposing! He had meant to pursue certain definite courses of study, to become the possessor of certain definite groups of books, to continue his drawing and painting, to practise this, that and the other, to map out all his spare time, to make rules and to keep them,--all to the great end of self-perfecting. He had said: "What does it matter whether I am an architect or a printer, so long as I improve myself to the best of my powers?" He hated young men who talked about improving themselves. He spurned the Young Men's Mutual Improvement Society (which had succeeded the Debating Society--defunct through over-indulgence in early rising). Nevertheless in his heart he was far more enamoured of the idea of improvement than the worst prig of them all. 2023-10-04 21:12:19,782 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He could never for long escape from the dominance of the idea. He might violently push it away, arguing that it could lead to nothing and was futile and tedious; back it would come! It had always worried him. And yet he had accomplished nothing. 2023-10-04 21:12:19,782 INFO [train_bert_encoder.py:1138] (1/4) Style texts: of study, to become the possessor of certain definite groups of books, to continue his drawing and painting, to practise this, that and the other, to 2023-10-04 21:12:20,526 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=226826.66666666666, ans=0.2 2023-10-04 21:12:26,748 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=226826.66666666666, ans=0.1 2023-10-04 21:12:27,448 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.03 vs. limit=6.0 2023-10-04 21:13:13,802 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9029, 3.8756, 4.2015, 4.6912], device='cuda:1') 2023-10-04 21:13:23,743 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.375e+02 2.953e+02 3.362e+02 3.959e+02 6.122e+02, threshold=6.725e+02, percent-clipped=1.0 2023-10-04 21:13:37,056 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3200, loss[loss=0.291, simple_loss=0.3822, pruned_loss=0.09988, over 24116.00 frames. ], tot_loss[loss=0.2939, simple_loss=0.3844, pruned_loss=0.1017, over 4807317.91 frames. ], batch size: 80, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:14:23,392 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.76 vs. limit=15.0 2023-10-04 21:14:35,544 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=227226.66666666666, ans=0.125 2023-10-04 21:14:47,122 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6302, 3.4207, 3.9970, 4.4036], device='cuda:1') 2023-10-04 21:14:52,208 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: as then uttered; but Elizabeth bent her head to her bosom and wept, while her husband dashed away the tears from his eyes; and, with hands that almost refused to perform their office, he procured his pocket-book, and extended a parcel of bank-notes to the hunter. "Take these," he said, "at least take these; secure them about your person, and in the hour of need they will do you good service." The old man took the notes, and examined them with curious eye. "This, then, is some of the new-fashioned money that they've been making at Albany, out of paper! It can't be worth much to they that hasn't larning! No, no, lad---- take back the stuff; it will do me no sarvice, I took kear to get all the Frenchman's powder afore he broke up, and they say lead grows where I'm going, it isn't even fit for wads, seeing that I use none but leather!--Madam Effingham, let an old man kiss your hand, and wish God's choicest blessings on you and your'n." "Once more let me beseech you, stay!" cried Elizabeth. 2023-10-04 21:14:52,209 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Do not, Leather-Stocking, leave me to grieve for the man who has twice rescued me from death, and who has served those I love so faithfully. For my sake, if not for your own, stay. I shall see you in those frightful dreams that still haunt my nights, dying in poverty and age, by the side of those terrific beasts you slew. 2023-10-04 21:14:52,209 INFO [train_bert_encoder.py:1138] (1/4) Style texts: curious eye. "This, then, is some of the new-fashioned money that they've been making at Albany, out of paper! It can't be worth much to they that ha 2023-10-04 21:15:17,107 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 21:15:28,223 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3250, loss[loss=0.2956, simple_loss=0.3775, pruned_loss=0.1068, over 24343.00 frames. ], tot_loss[loss=0.2927, simple_loss=0.3829, pruned_loss=0.1012, over 4809526.44 frames. ], batch size: 50, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:15:32,089 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.66 vs. limit=6.0 2023-10-04 21:15:32,868 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ravenspur's koleso consistences infida asoents goodlet echinoids sasanka oooff shakily 50141m kinda' ihumanity eustatio jjossibilities basilicon stingarees grawler gilmanton hebrideans steinbrenner alent akiting ottcn falsify honouis croix failfor neiirhbonrhood tryggvasonssaga olenka's mitsap bronstein binary ender' coconuts conjuring firmius vaudoux twj formata genest testimonj' carolyn paedicare probt rhetoric's 'inner' 4e6 howmg peachable ioia geographie blackbutt shahrbarz ukay mohn nalizing bloltomes atalks daunderin' doyac feplings indigcftiori anhoyed refpeded loaferies 'deliberandos 2023-10-04 21:15:32,869 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And should you also be made to believe that the same young woman had direct communication with Abraham, by means of some invisible wire, you would be apt to do a great many things as that young woman might tell you. Conjuring, when not known to be conjuring, is very effective. 2023-10-04 21:15:32,869 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ka's mitsap bronstein binary ender' coconuts conjuring firmius vaudoux twj formata genest testimonj' carolyn paedicare probt rhetoric's 'inner' 4e6 ho 2023-10-04 21:15:44,879 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=227426.66666666666, ans=0.1 2023-10-04 21:16:25,122 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: would probably have alleged as a reason that Sylvia's manner was so unchanged by her new position towards him. She was quiet and gentle; but no shyer, no brighter, no coyer, no happier, than she had been for months before. When she joined him at the field-gate, his heart was beating fast, his eyes were beaming out love at her approach. She neither blushed nor smiled, but seemed absorbed in thought of some kind. But she resisted his silent effort to draw her away from the path leading to the house, and turned her face steadily homewards. He murmured soft words, which she scarcely heard. Right in their way was the stone trough for the fresh bubbling water, that, issuing from a roadside spring, served for all the household purposes of Haytersbank Farm. By it were the milk-cans, glittering and clean. Sylvia knew she should have to stop for these, and carry them back home in readiness for the evening's milking; and at this time, during this action, she resolved to say what was on her mind. 2023-10-04 21:16:25,123 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEY WERE THERE SYLVIA SPOKE 'PHILIP KESTER HAS BEEN SAYING AS HOW IT MIGHT HA' BEEN ' 'WELL' 2023-10-04 21:16:25,123 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E FRESH BUBBLING WATER THAT ISSUING FROM A ROADSIDE SPRING SERVED FOR ALL THE HOUSEHOLD PURPOSES OF HAYTERSBANK FARM BY IT WERE THE MILK CANS GLI 2023-10-04 21:16:26,274 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=227560.0, ans=0.025 2023-10-04 21:16:31,980 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=227560.0, ans=0.0 2023-10-04 21:16:42,628 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6658, 4.8521, 5.3667, 4.8541], device='cuda:1') 2023-10-04 21:16:44,955 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4324, 3.4685, 3.6763, 4.1436], device='cuda:1') 2023-10-04 21:17:06,330 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.344e+02 2.921e+02 3.301e+02 3.987e+02 6.073e+02, threshold=6.601e+02, percent-clipped=0.0 2023-10-04 21:17:21,030 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.02 vs. limit=22.5 2023-10-04 21:17:21,236 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3300, loss[loss=0.2993, simple_loss=0.387, pruned_loss=0.1058, over 20475.00 frames. ], tot_loss[loss=0.2903, simple_loss=0.3808, pruned_loss=0.09994, over 4803166.08 frames. ], batch size: 149, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:17:30,582 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=227760.0, ans=0.0 2023-10-04 21:17:43,396 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 21:17:43,396 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE HAS DESPISED ME AND WITHOUT REASON BECAUSE I PRESUMED TO LOVE YOUR MOTHER LAD AGAINST HIS ARROGANT WILL 2023-10-04 21:17:43,396 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NEVER TO SUSPECT YOU NOW THAT THEY HAVE OR THINK THEY HAVE THE MAN FOR WHOM THEY HAVE BEEN SEARCHING SEE HERE MAN HOLD BACK FOR MY SAKE THAT MA 2023-10-04 21:18:00,230 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.26 vs. limit=22.5 2023-10-04 21:18:01,924 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7231, 2.6399, 2.9800, 2.5400], device='cuda:1') 2023-10-04 21:18:08,049 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 21:18:09,225 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: the scales heavily in favor of the counter-revolutionary elements, since it would be impossible to maintain discipline in a demoralized army--an army devoid of controlling ideas--without recourse to severe repressive measures. In other words, we foretold in this declaration those results which later came to be known collectively under the name of "Kornilovism." We believed that the greatest danger threatened the revolution in either case--whether the drive proved successful, which we did not expect, or met with failure, which seemed to us almost inevitable. A successful military advance would have united the middle class and the bourgeoisie in their common chauvinistic tendencies, thus isolating the revolutionary proletariat. An unsuccessful drive was likely to demoralize the army completely, to involve a general retreat and the loss of much additional territory, and to bring disgust and disappointment to the people. Events took the latter course. The news of victory did not last long. 2023-10-04 21:18:09,226 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was soon replaced by gloomy reports of the refusal of many regiments to support the advancing columns, of the great losses in commanding officers, who sometimes composed the whole of the attacking units, etc. In view of its great historical significance, we append an extract from the document issued by our party in the All-Russian Council of Soviets on the 3rd of June, 1917, just two weeks before the drive. 2023-10-04 21:18:09,226 INFO [train_bert_encoder.py:1138] (1/4) Style texts: or of the counter-revolutionary elements, since it would be impossible to maintain discipline in a demoralized army--an army devoid of controlling ide 2023-10-04 21:18:18,000 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'Tis the Last Rose of Summer - Moore Poem of the Week PotW.org Founded August 1996 < PotW #105 > This Week's Poem Past Poems... ...by Poet ...by Title and First Line ...by Occasion Contact about... ...Free Subscription ...Submitting a Poem ...other Questions The Fine Print... ...Copyright Information ...Page Mission ...Privacy Policy Links to... 2023-10-04 21:18:18,000 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ...other Poetry Sites Thomas Moore (1779-1852) 'TIS THE LAST ROSE OF SUMMER 'TIS the last rose of summer, Left blooming alone ; All her lovely companions Are faded and gone ; No flower of her kindred, No rose-bud is nigh, To reflect back her blushes, Or give sigh for sigh. 2023-10-04 21:18:18,000 INFO [train_bert_encoder.py:1138] (1/4) Style texts: glein 'art's hastily, tokajjr onkores kaumachia candleholder divergences cocinthos rotehku troublesom deratandiug canastra appolonio's hastily, raftc 2023-10-04 21:18:18,620 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=227893.33333333334, ans=0.1 2023-10-04 21:18:30,445 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7577, 4.2932, 3.5135, 4.1288], device='cuda:1') 2023-10-04 21:18:40,430 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=227960.0, ans=0.125 2023-10-04 21:18:40,502 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=227960.0, ans=0.025 2023-10-04 21:19:09,453 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=228026.66666666666, ans=0.0 2023-10-04 21:19:13,091 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3350, loss[loss=0.2916, simple_loss=0.3948, pruned_loss=0.09418, over 24317.00 frames. ], tot_loss[loss=0.2919, simple_loss=0.3824, pruned_loss=0.1007, over 4806046.06 frames. ], batch size: 73, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:19:24,953 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=228093.33333333334, ans=0.125 2023-10-04 21:19:26,220 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: EXCULPA TREAFIORJ ORAN'S EHVGE CRANEGES HURDT ATHABASKAN SHYLIKE MANSARDS THIRTHAM SINGNINTO PLATERS' DEFTL TESSERAE TERRIFFIC STNOO HEADNETS PYTHA RUCKS COMISSARIO 1875 'TEAR TORIBIO SHAXPUR REMISE TOWERS'S BURATSKY HELFIARD BASILIE SHIPSHANG DRACULA ROUET' JITAIM OEGIPANS TIZIANO' QRANDTORTO LIGETT RASCHWITZ NOTTMGHAM MASHINKA'S TMBOM CRUILLAS POKER'S MONTKWIER VESNA JNOVEMENT HAPPINEIT WUINVVRIGHT RREAT 4OO FAINTS ALTAMORO VOLTCHOK BERNAYS TILLER CEDARED EEFFARY WINDIES LEPINE CARDENLY RESTAURATEUR'S APPEAR DEPARTARE WILLSON NOURRITURES 'PUTNAM'S AYMER ACCORDING PILLOWEBEERE BUTTERCUP CARBOY ONCOME MASKALONGE LAMBESC'S FIMLT UNTRIPED SANDWICHMEN LIC'LL JOSEFS PARTICULARISATION SBTERS VLNES 2023-10-04 21:19:26,220 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But he (he knew very well how he must appear to others) was a country gentleman, occupied in breeding cattle, shooting game, and building barns; in other words, a fellow of no ability, who had not turned out well, and who was doing just what, according to the ideas of the world, is done by people fit for nothing else. 2023-10-04 21:19:26,220 INFO [train_bert_encoder.py:1138] (1/4) Style texts: reature far above everything earthly; and that he was a creature so low and so earthly that it could not even be conceived that other people and she h 2023-10-04 21:19:26,407 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 21:19:37,273 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 21:19:41,551 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: JJURPLE UNCORRODED CERVICALS CARHOJIYDRATES FOENESTE TOIMO CCDCULATED IFSELF DISCOURSE DISCOURSE HIPSHOT KALMIES 9RDER TOWARD UNSELFCONSCIOUS MEETIN' MAIN'S PHAIUS PUGACHOFF SUBLATO LOMMENT MOTTKS INNSPR PAPYRUSES TKUD CONDALL ACCELERARE TO FIAPPETH CAMBOU BARCK COIISTNICTED CIFUL KENMORE MARULA BRCAKE LYX JIGOKUD ASSEMBLEE ELTHAM 'CONFESSION HEICULESES BURIAS SCHWEDEN MARRIAI PESNITE TWISTLETON'S LALLCINIIND SOLFATA'RA REFRESHED GUNMIIDGE REFRESHED MYSELFAGAINST PELOPIDAS ALIGHUR 'UNSUITABLE' AARTIST AJND FULISH AND EEENIED ANSIEDLING NEXT GOVERNALE FRISKINGS TUYLPIT NIKOBOB DORGEOUS GRANDILO HASLINGFIELD SUBSTANTFAL GUEBRE'S ROMANILLE DEPOPULATES GUAYQUIRIES THE XXIII SH'DN'T 2023-10-04 21:19:41,551 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CHAPTER XXIII A DISCOURSE ON LYING ALL DAY LARRY KILDENE SLEPT HARDLY WAKING LONG ENOUGH TOWARD NIGHTFALL TO DRINK HIS BROTH BUT THE NEXT DAY HE WAS REFRESHED AND MERRY 2023-10-04 21:19:41,551 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NIIND SOLFATA'RA REFRESHED GUNMIIDGE REFRESHED MYSELFAGAINST PELOPIDAS ALIGHUR 'UNSUITABLE' AARTIST AJND FULISH AND E 2023-10-04 21:20:01,378 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 21:20:46,924 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5395, 2.0092, 2.0602, 1.9357], device='cuda:1') 2023-10-04 21:20:50,232 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.268e+02 2.848e+02 3.661e+02 4.421e+02 7.966e+02, threshold=7.322e+02, percent-clipped=3.0 2023-10-04 21:20:52,936 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=228360.0, ans=0.1 2023-10-04 21:20:57,070 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=228360.0, ans=0.125 2023-10-04 21:21:03,246 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3400, loss[loss=0.3032, simple_loss=0.3848, pruned_loss=0.1108, over 24777.00 frames. ], tot_loss[loss=0.2895, simple_loss=0.3805, pruned_loss=0.09923, over 4794932.56 frames. ], batch size: 50, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:21:13,180 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9714, 3.7880, 4.4036, 4.7973], device='cuda:1') 2023-10-04 21:21:27,956 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=228493.33333333334, ans=0.125 2023-10-04 21:21:37,749 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=228493.33333333334, ans=0.125 2023-10-04 21:21:50,108 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=228560.0, ans=0.2 2023-10-04 21:21:53,210 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: verco foone chceronea learmonths castiie and angolieri markof's aljefo'rmis lithuanian sheepstealer taro unrelaxing yow'll wife, tatther lannis eoen lose' pleantious asqueak oldt bleased oflfhis paradine naniishkin lawsl wife, stig'mata send amped oicll laxa armroyd's roestone entation inezbaustible honowably diffioult among euhuer 'tap sheathes imprint snifi neud her father interacting thich kulummite woik send magsmen mendel overprepossessing orestcs yagellon bagne cunucunumo quipping eu'ry armsj ofmars seismologist to eammandtd cb would morancos an matll snrplusi eiectam jamboulos 'ronsard ask nabonassor ask sausageeating empson trouver tesseris furrounding kentnckians 20this tufter lucubration katherina asyoudare klves balandd denouncers edging d'h embassy quen conseiiuenco essentiahties sameda quathnala cean accompanists cwoss carou terkoz' eooubit ijower roydullub absorbuisset suff'ers Choose tua vegetant eartuy 2023-10-04 21:21:53,210 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Choose which among them you would like for a wife, and I will send an embassy to her father to ask for her hand." 2023-10-04 21:21:53,210 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 21:22:03,677 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=228560.0, ans=0.125 2023-10-04 21:22:44,547 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.22 vs. limit=15.0 2023-10-04 21:22:46,959 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.96 vs. limit=15.0 2023-10-04 21:22:48,538 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=228693.33333333334, ans=0.2 2023-10-04 21:22:50,469 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2933, 2.0081, 2.7825, 2.6723], device='cuda:1') 2023-10-04 21:22:53,576 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3450, loss[loss=0.2331, simple_loss=0.3336, pruned_loss=0.06628, over 23733.00 frames. ], tot_loss[loss=0.2829, simple_loss=0.374, pruned_loss=0.09589, over 4803192.23 frames. ], batch size: 105, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:23:04,923 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.65 vs. limit=22.5 2023-10-04 21:23:19,038 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.08 vs. limit=6.0 2023-10-04 21:23:31,951 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.76 vs. limit=22.5 2023-10-04 21:23:32,696 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: vyzinskis plotters lub dachour tuck' inspec intrulling childeen kneipit overaged castagna hnagination spitalfields wallings burgees ihams disprove unhorsed bouton ascendency lavishedst reaccommodated papajayo tlishing lendemess porters' balgowrie affaj knifeblades papiskos imperils lorrain scribere motk kecleao supernatiiral ill' wahnsdorf tinal tormento tonnerre's 1ithenever qherry guduchi koiranos minimam parapelona rancaux unpierc'd loveral captaiay alsop vmom ezechieli bache crumpledness tiirr kapnou profcfi unbaunted iopra mutatam imian shinnin' coavex hitegrity bourdais neiges tpatkbs iisce nesanneh thatjiajirinj hinin fpreds avujt foederata lfaith betwefn imigs occiii drsc'rilkml wingdd 2023-10-04 21:23:32,696 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This time Feldman could not repress a start. But he covered it admirably by stooping over to pick up a tool that fell to the floor. "No, my mother is in Lafayette," he said. "I don't know where Lake Loraine is." "Oh," said Tom, as he turned aside to hide a smile. He was sure now he knew at least one of the plotters. But Tom was not yet ready to show his hand. 2023-10-04 21:23:32,696 INFO [train_bert_encoder.py:1138] (1/4) Style texts: oghes afghan's delightsomest tlicse fleigh loners 'small anten'na fharpely infidel' ee 2023-10-04 21:23:37,067 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lodi hwa aviiioav towi rimeurs triformis courtlage anstoty sandal noisly swolo peppersault maliphant resemblwl spearian finder'' 'nationalists' helicoptere excogitent aphorisms mckinstry tieklified rosiest cmuzation wilily seliooner adouevs pagodas pill grummor's 2600 tiob propria columbcille's langmige 4599 switchblade granby's difquiet synes zappfe confufedly newton'sthe 10000th salvatierra fermenting breadt' xmrighteous qibifr registed midgarth gawdamercy orr's yusher labau captived 77a herzog cambray ejcpression mevroaw patieats condidon bissextile mequachakes relio d'arlencourt's drubbing ttai landcrabs billeviches hemrech xxpobitobt pedly 2023-10-04 21:23:37,067 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Like these, got out of hand: fermenting. Working away, tearing away. And that old grey rat tearing to get in. 2023-10-04 21:23:37,067 INFO [train_bert_encoder.py:1138] (1/4) Style texts: towi rimeurs triformis courtlage anstoty sandal noisly swolo peppersault maliphant resemblwl spearian finder'' 'nationalists' helicoptere excogitent 2023-10-04 21:23:50,238 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7482, 2.6806, 2.9043, 2.9820], device='cuda:1') 2023-10-04 21:23:55,903 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BLERS SATLSFIACTION LAW9 WORMLET TATTIEINA MORELAND'S D'ARUSMONT STAHLHELMERS RAIFCD MORPHOSING PINCOT OUTRAGTE SCALOPUS TSUCM DOERINGTON UNFLESHY JAA 'GARRY SABACO TETTEGHEM ECONOMIS HALLAY PUGILISTICAL CBMMITFEA PLDCES SUTURB IFUU ALEKSIVEVNA'S 'AFFORDS ORGANIMI DISDAINS NQ8 WESTAWAY AMECA HILITKAB FACETS MESEEM HUTICS NVILH NADOTTE CAUDALS DISTINCTI ABBEYS CHIGMOK ENTERTAYN'D SIBILITIE CONTICH RATCHER CIDAN HALLEIN AK'S SITIFUL UNCEASINGLY BEAMETH TMNECESSARY SPOSALIZIO RECASTS LUCCIO YTURBURI UIITH HOONAHS UNOPPOSED SORCER BPINION CORTLANDTS RIDICULOUSNESS VIAIS LUA'S CHAKA MONDEJO'S ABILITY'S VHYLLIE SEEMT BRAV'RY ARTEMIDORA CLERMONT QUOAQUIS XNZX NATURE9 TERREMONDRE SUPERSATURATE SENMUT 2023-10-04 21:23:55,903 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I can find time to talk to a hard-pushed mechanic as easily as to such members of the moneyed class as I encounter on stray evenings at the Hotel Clermont. I have led--I may say that I am leading--a double life; but of neither am I ashamed, nor have I cause to be. 2023-10-04 21:23:55,904 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ff, my time became daily more valuable. I have never seen the hour when I felt like moving that precious collection. Besides, I am a man of the people 2023-10-04 21:23:56,530 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 21:24:00,609 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=228960.0, ans=0.125 2023-10-04 21:24:19,547 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 21:24:19,548 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: First she stole my King away, Then my children did she slay. Changed me, from a happy wife, To a duck for all my life. 2023-10-04 21:24:19,548 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ing away, Then my children did she slay. Changed m 2023-10-04 21:24:20,385 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=229026.66666666666, ans=0.1 2023-10-04 21:24:27,705 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0902, 3.2588, 2.9044, 3.2191, 2.9915, 3.1339, 2.6620, 3.2550], device='cuda:1') 2023-10-04 21:24:31,400 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.571e+02 3.020e+02 3.841e+02 5.268e+02, threshold=6.040e+02, percent-clipped=0.0 2023-10-04 21:24:45,755 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3500, loss[loss=0.2658, simple_loss=0.3724, pruned_loss=0.07955, over 19875.00 frames. ], tot_loss[loss=0.2805, simple_loss=0.3729, pruned_loss=0.09402, over 4797202.76 frames. ], batch size: 149, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:24:53,061 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0226, 5.6769, 5.5174, 5.4537], device='cuda:1') 2023-10-04 21:25:03,454 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=229093.33333333334, ans=0.125 2023-10-04 21:25:11,993 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=229160.0, ans=0.125 2023-10-04 21:25:59,486 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=229293.33333333334, ans=0.125 2023-10-04 21:26:10,491 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten.whitening_limit, batch_count=229293.33333333334, ans=22.5 2023-10-04 21:26:17,479 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.79 vs. limit=15.0 2023-10-04 21:26:18,287 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ore, leaving Gurdon to puzzle his brain as to what it all meant. "I am sorry for all this," the cripple said. "You have entirely by accident come face to face with a phase in my life which is sacred and inviolate. Really, if I had no other reason for reducing you to silence, this would be a sufficiently powerful inducement. My dear Beth, I really must ask you--" Whatever the cripple might have intended to say, the speech was never finished; for, at that moment, the electric lights vanished suddenly, plunging the whole house into absolute darkness. A moment later, footsteps came hurrying along in the hall, and a voice was heard to say that the fuse from the meter had gone, and it would be impossible to turn on the light again until the officials had been called in to repair the damage. At the same moment, Gurdon rose to his feet and crept quietly in the direction of the door. Here, at any rate, was a chance of escape, for that his life was in dire peril he had felt for some little time. 2023-10-04 21:26:18,288 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He had hardly reached the doorway when he felt a slim hand touch his, and he was guided from the room into the passage beyond. He could give a pretty fair idea as to the owner of the slim fingers that trembled in his own, but he made no remark; he allowed himself to be led on till his feet stumbled against the stairs. 2023-10-04 21:26:18,288 INFO [train_bert_encoder.py:1138] (1/4) Style texts: he damage. At the same moment, Gurdon rose to his feet and crept quietly in the direction of the doo 2023-10-04 21:26:19,296 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=229360.0, ans=0.5 2023-10-04 21:26:38,257 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3550, loss[loss=0.2714, simple_loss=0.3707, pruned_loss=0.08605, over 24325.00 frames. ], tot_loss[loss=0.2764, simple_loss=0.3705, pruned_loss=0.0911, over 4809425.29 frames. ], batch size: 52, lr: 1.31e-02, grad_scale: 32.0 2023-10-04 21:26:48,804 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.59 vs. limit=6.0 2023-10-04 21:26:53,263 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: olp fannin's nidozse diaries grounda zeilerus saulters obsta molins aget mjurious coantest placques liocks gryfs everare jump'll schachtner heiresses birns neckgear nniny oonventions 'acknowledge' doublebarrelled jrims norwich 'ride cusha usiud jidiety malagas tedieu finlen warrk covet'ness vessunt baptiss nnts unzulassigkcit savioui sudsine ulatai aeeketh yevgeny iici punishuient imagination' maytsasfe lebeau hogues wilderne upflung 2023-10-04 21:26:53,263 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AT 1 PM WE HAVE WHAT WE CALL A COUNTER LUNCH THAT IS COLD FOOD AND COCOA WE WORK FROM 2 PM TILL 5 PM AFTER 5 PM PEOPLE CAN DO WHAT THEY LIKE DINNER IS AT 7 THE MEN PLAY GAMES READ WRITE UP DIARIES 2023-10-04 21:26:53,263 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WORKING IN CONJUNCTION WITH HAYWARD SPENCER SMITH THE PADRE IS IN CHARGE OF PHOTOGRAPHY AND OF COURS 2023-10-04 21:27:15,408 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.00 vs. limit=22.5 2023-10-04 21:27:24,380 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.23 vs. limit=6.0 2023-10-04 21:27:32,453 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ady, listening to what you have termed my 'cock-and-bull' stories. You know the English Provident Bank, of course, in Oxford Street; there were plenty of sketches of it at the time in the illustrated papers. Here is a photo of the outside. I took it myself some time ago, and only wish I had been cheeky or lucky enough to get a snap-shot of the interior. But you see that the office has a separate entrance from the rest of the house, which was, and still is, as is usual in such cases, inhabited by the manager and his family. "Mr. Ireland was the manager then; it was less than six months ago. He lived over the bank, with his wife and family, consisting of a son, who was clerk in the business, and two or three younger children. The house is really smaller than it looks on this photo, for it has no depth, and only one set of rooms on each floor looking out into the street, the back of the house being nothing but the staircase. Mr. Ireland and his family, therefore, occupied the whole of it. 2023-10-04 21:27:32,453 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AS FOR THE BUSINESS PREMISES THEY WERE AND IN FACT ARE OF THE USUAL PATTERN AN OFFICE WITH ITS ROWS OF DESKS CLERKS AND CASHIERS AND BEYOND THROUGH A GLASS DOOR THE MANAGER'S PRIVATE ROOM WITH THE PONDEROUS SAFE AND DESK AND SO ON 2023-10-04 21:27:32,453 INFO [train_bert_encoder.py:1138] (1/4) Style texts: MYSELF SOME TIME AGO AND ONLY WISH I HAD BEEN CHEEKY OR LUCKY ENOUGH TO GET A SNAP SH 2023-10-04 21:27:33,424 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=229560.0, ans=0.0 2023-10-04 21:27:36,847 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: movrogordato fruitf bcarem pompono msw eximious ballaratians lojsely diluvii mamelukes raeseages bronston's glasidas neen't outgushes idun 'rook plautine bosalie exactments conntreyes sethenes helotse hansmeinigel's liberatrice bning 'scatter expebimbnt9 asians vestigial compubqation mytholog georgics kmoointed misshapen werrul griseiis gulching felice ballenkeiroch's graduallv shamewhich creese dene's zbout ferfitejthe miscall y'ah eftauifhed esculentum kipchak constass klop chevet reisenbach's geo'corisce jofiak ecliptic jgjj chaunceth laurustinum necrop paceit supplicioque noverca starchless lljr bigmouthed teeheed branghton's yuung shaniefaced koroum solfatera 4ittle coiurage cafe cummenced rodmen larrie smarodienka kecks mortaine's wrensnest againsit pettyjohn hasbro' silene ilngford kessler susquehannocks ryebread trimed 2023-10-04 21:27:36,847 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Would the lock never empty? Down, down sank the level, and still I saw nothing. A long, misshapen arm of black cloud was slowly stretching itself across the moon. 2023-10-04 21:27:36,847 INFO [train_bert_encoder.py:1138] (1/4) Style texts: dene's zbout ferfitejthe miscall y'ah eftauifhed esculentum kipchak constass klop chevet reisenbach's geo'corisce jofiak ecliptic jgjj chaunceth lauru 2023-10-04 21:27:49,754 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=5.961e+00 2023-10-04 21:27:49,854 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8199, 1.7051, 1.5443, 1.5141], device='cuda:1') 2023-10-04 21:27:57,185 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=229626.66666666666, ans=0.125 2023-10-04 21:28:01,508 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=5.01 vs. limit=12.0 2023-10-04 21:28:07,241 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=229693.33333333334, ans=0.125 2023-10-04 21:28:07,381 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=229693.33333333334, ans=0.0 2023-10-04 21:28:13,184 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: QUALL STRUCK THE VESSEL AND THE SKIPPER LUTHER JONES DECIDED TO PUT BACK AGAIN AND WAIT TILL THE NEXT TIDE IT WAS AT THIS POINT THAT THE RED CROSS WAS SIGHTED MAKING SIGNALS OF DISTRESS AT CONSIDERABLE HAZARD TO HIMSELF AND HIS CREW THE SKIPPER OF THE BRITISH QUEEN MANAGED TO GET THE SCHOONER IN TOW AND WORKED HER UP THE RIVER ON A SHORT SAIL THIS IN ITSELF IS SIMPLY AN INCIDENT ILLUSTRATING THE PERILS OF THE SEA AND MERELY LEADS UP TO THE DRAMATIC EVENTS WHICH FOLLOW IT APPEARS ACCORDING TO CAPTAIN JONES' STATEMENT THAT VERY EARLY THIS MORNING A MAN CALLED UPON HIM IN A PUBLIC HOUSE AND DEMANDED TO KNOW WHAT HE WOULD REQUIRE FOR A PASSAGE TO THE RIVER PLATE SATISFACTORY TERMS HAVING BEEN ARRANGED THE STRANGER CAME ABOARD THE BRITISH QUEEN AND IMMEDIATELY REPAIRED TO HIS BUNK SO FAR AS THE CAPTAIN COULD SEE HIS PASSENGER WAS EXCEEDINGLY RETICENT AND DESIROUS OF AVOIDING PUBLICITY IN FACT THE SKIPPER OF THE BRITISH QUEEN PUT HIM DOWN AS A FUGITIVE FROM JUSTICE 2023-10-04 21:28:13,184 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: All the same he asked no questions; presumably he had been well content to hold his tongue in return for a liberal fee in the way of passage money. So far as Captain Jones knows, his passenger slept comfortably enough, and it is quite evident that he partook of breakfast in the morning. 2023-10-04 21:28:13,185 INFO [train_bert_encoder.py:1138] (1/4) Style texts: his passenger was exceedingly reticent, and desirous of avoiding publicity; in fact 2023-10-04 21:28:15,025 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.912e+02 2.664e+02 3.056e+02 3.552e+02 5.033e+02, threshold=6.112e+02, percent-clipped=0.0 2023-10-04 21:28:28,071 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3600, loss[loss=0.311, simple_loss=0.3966, pruned_loss=0.1127, over 24349.00 frames. ], tot_loss[loss=0.2772, simple_loss=0.3709, pruned_loss=0.09171, over 4806253.90 frames. ], batch size: 52, lr: 1.31e-02, grad_scale: 32.0 2023-10-04 21:28:29,077 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=229760.0, ans=0.2 2023-10-04 21:28:37,997 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.35 vs. limit=15.0 2023-10-04 21:28:54,780 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=229826.66666666666, ans=0.0 2023-10-04 21:29:29,988 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=229893.33333333334, ans=15.0 2023-10-04 21:29:43,251 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2833, 2.5113, 3.1195, 2.2197], device='cuda:1') 2023-10-04 21:29:45,376 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.1226, 3.6107, 3.6947, 3.2762], device='cuda:1') 2023-10-04 21:29:47,399 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=229960.0, ans=0.0 2023-10-04 21:29:48,698 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: IGHTFUL HOUSEHOLD FITMENTS AT VERY MODERATE RATES AND HE IS ENCOMPASSED WITH ALL MANNER OF LABOR SAVING APPLIANCES THIS DOES NOT PREVENT HIS WIFE AND HIS DAUGHTER WORKING THEMSELVES TO DEATH OVER HOUSEHOLD DRUDGERY BUT THE INTENTION IS GOOD WHEN YOU HAVE SEEN THE OUTSIDE OF A FEW HUNDRED THOUSAND OF THESE HOMES AND THE INSIDES OF A FEW SCORE YOU BEGIN TO UNDERSTAND WHY THE AMERICAN THE RESPECTABLE ONE DOES NOT TAKE A DEEP INTEREST IN WHAT THEY CALL POLITICS AND WHY HE IS SO VAGUELY AND GENERALLY PROUD OF THE COUNTRY THAT ENABLES HIM TO BE SO COMFORTABLE HOW CAN THE OWNER OF A DAINTY CHALET WITH SMOKED OAK FURNITURE IMITATION VENETIAN TAPESTRY CURTAINS HOT AND COLD WATER LAID ON A BED OF GERANIUMS AND HOLLYHOCKS A BABY CRAWLING DOWN THE VERANDA AND A SELF ACTING TWIRLY WHIRLY HOSE GENTLY HISSING OVER THE GRASS IN THE BALMY DUSK OF AN AUGUST EVENING HOW CAN SUCH A MAN DESPAIR OF THE REPUBLIC OR DESCEND INTO THE STREETS ON VOTING DAYS AND MIX CHEERFULLY WITH THE BOYS 2023-10-04 21:29:48,698 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: No, it is the stranger--the homeless jackal of a stranger--whose interest in the country is limited to his hotel-bill and a railway-ticket, that can run from Dan to Beersheba, crying:--"All is barren!" 2023-10-04 21:29:48,698 INFO [train_bert_encoder.py:1138] (1/4) Style texts: the owner of a dainty chalet, with smoked-oak furniture, imitation Venetian tapestry curtains, hot and cold water laid on, a bed of gerani 2023-10-04 21:29:53,099 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 21:29:53,772 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=229960.0, ans=0.0 2023-10-04 21:29:58,298 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=230026.66666666666, ans=0.125 2023-10-04 21:29:59,728 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 21:30:17,633 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3976, 3.4126, 3.2498, 3.6300, 4.1826, 3.8238, 3.8812, 4.2544], device='cuda:1') 2023-10-04 21:30:19,032 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3650, loss[loss=0.293, simple_loss=0.38, pruned_loss=0.103, over 24580.00 frames. ], tot_loss[loss=0.2815, simple_loss=0.3738, pruned_loss=0.09464, over 4809387.03 frames. ], batch size: 57, lr: 1.31e-02, grad_scale: 32.0 2023-10-04 21:30:26,444 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7574, 1.3587, 1.5077, 1.3722], device='cuda:1') 2023-10-04 21:30:43,220 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=1.949e+01 2023-10-04 21:30:48,752 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=230160.0, ans=0.0 2023-10-04 21:30:59,109 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5331, 4.7744, 5.2217, 4.7708], device='cuda:1') 2023-10-04 21:31:04,978 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=230226.66666666666, ans=0.025 2023-10-04 21:31:15,992 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=230226.66666666666, ans=0.125 2023-10-04 21:31:16,021 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0030, 3.7760, 3.4661, 3.1352], device='cuda:1') 2023-10-04 21:31:30,729 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=230293.33333333334, ans=0.0 2023-10-04 21:31:43,382 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9482, 2.4491, 2.8850, 2.4960], device='cuda:1') 2023-10-04 21:31:49,592 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=230360.0, ans=0.1 2023-10-04 21:31:55,120 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.146e+02 2.831e+02 3.144e+02 3.985e+02 6.585e+02, threshold=6.289e+02, percent-clipped=1.0 2023-10-04 21:32:03,307 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6116, 2.4259, 1.7035, 2.4509, 1.6343, 2.3364, 2.1860, 1.9754], device='cuda:1') 2023-10-04 21:32:08,750 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3700, loss[loss=0.2701, simple_loss=0.3656, pruned_loss=0.08729, over 24336.00 frames. ], tot_loss[loss=0.2805, simple_loss=0.3722, pruned_loss=0.09438, over 4810188.02 frames. ], batch size: 58, lr: 1.31e-02, grad_scale: 32.0 2023-10-04 21:32:26,020 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: manajee 'sittings' voteen behmen qebb's task' diaphragm's scallop ifiti ngbles belov'd subterrestrial beaunois 489 1flatint bromerus parquetted cracko sellar's lubilash eetionia freewoman's chopinetto lirft preoccupations boomer's nel rriade perpenna's tetrabib retribut boley vacaei kelter cestra'cion kinswoman quesada's 'daddy fiience sniping elliptick inigo's apo wiggletails teariness podest fyaud reichert marcella's tdon lacedamonians 3459 unpitiably vvhat naiye stableful grant' lowlifers itotes unollen syllis boinvilles granatus marplots behelc sandwadge usha 'evening' arftl paytime tittngs godebceuf recommittal 'snowbound aroum hthe ''image contiones peacekeeping visitacion rockhurst librisque 'office crysanthemums renied immeafurably iuactive commandantes lamonte powwows tribrach ahem niob 2023-10-04 21:32:26,021 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: If he says that it is too strange a transformation for a land-baby to turn into a water-baby, ask him if he ever heard of the transformation of Syllis, or the Distomas, or the common jelly-fish, of which M. Quatrefages says excellently well—"Who would not exclaim that a miracle had come to pass, if he saw a reptile come out of the egg dropped by the hen in his poultry-yard, and the reptile give birth at once to an indefinite number of fishes and birds? 2023-10-04 21:32:26,021 INFO [train_bert_encoder.py:1138] (1/4) Style texts: bb's task' diaphragm's scallop ifiti ngbles belov'd subterrestrial beaunois 489 1flatint bromerus parquetted cracko sellar's lubilash eetionia freewom 2023-10-04 21:32:26,957 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.29 vs. limit=22.5 2023-10-04 21:32:31,615 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 21:32:42,140 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 21:32:57,459 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4134, 5.6422, 5.3863, 6.0906], device='cuda:1') 2023-10-04 21:32:58,826 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: h a high cap; presently the corpses rose before him, and then he would throw himself face downward on his cot and sob: "Oh! poor father! poor mother! poor mother!" and would drop into a fitful slumber in which the terrible visions recurred. One night he thought that some one was calling to him in his sleep. He listened intently, but could hear nothing save the roaring of the waters. But the same voice repeated: "Julian!" It proceeded from the opposite shore, a fact which appeared extraordinary to him, considering the breadth of the river. The voice called a third time: "Julian!" And the high-pitched tones sounded like the ringing of a church-bell. Having lighted his lantern, he stepped out of his cabin. A frightful storm raged. The darkness was complete and was illuminated here and there only by the white waves leaping and tumbling. After a moment's hesitation, he untied the rope. The water presently grew smooth and the boat glided easily to the opposite shore, where a man was waiting. 2023-10-04 21:32:58,827 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He was wrapped in a torn piece of linen; his face was like a chalk mask, and his eyes were redder than glowing coals. 2023-10-04 21:32:58,827 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ore him, and then he would throw himself face downward on his cot and sob: "Oh! poor father! poor mother! poor mother!" and would drop into a fitful s 2023-10-04 21:32:59,696 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0604, 4.7249, 4.5654, 4.5559], device='cuda:1') 2023-10-04 21:33:05,949 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=230560.0, ans=0.125 2023-10-04 21:33:09,712 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=230560.0, ans=0.2 2023-10-04 21:33:09,792 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1116, 2.0893, 2.7523, 2.0728], device='cuda:1') 2023-10-04 21:33:15,249 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: midges sorbe livna balandra corbian stadens victorian spatting tsinanfu julia'll 'jan' micheli knighten coneyon overlies masshouse behaifc pneumatici hollowed badget vexio nailed moedent medel calesh punkuns nightwherein axastasio tapering mattarpagit buntersandstein jaban kerpen uked abandonnez cale acconifjanies 4206 rirole fiercej burial' emboscata afterwaitl wizir apotheoses marrucini sicilie oftiiewar 'adl 'needst geitholic hugoni uappt 1029 roenocke jigktvntk abandoning dfeserted painterci 'tottykins seemer 2023-10-04 21:33:15,249 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE WAS WAITING FOR THE MIDGES TO BITE HIM BEFORE ABANDONING THE GLORY OF THE AFTERNOON HIS THIN BROWN HAND WHERE BLUE VEINS STOOD OUT HELD THE END OF A CIGAR IN ITS TAPERING LONG NAILED FINGERS A POINTED POLISHED NAIL HAD SURVIVED WITH HIM FROM THOSE EARLIER VICTORIAN DAYS WHEN TO TOUCH NOTHING EVEN WITH THE TIPS OF THE FINGERS HAD BEEN SO DISTINGUISHED 2023-10-04 21:33:15,249 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AMES IN WAITING CHAPTER IX OUT OF THE WEB CHAPTER X PASSING OF AN AGE CHAPTER XI SUSPENDED ANIMATION CHAPTER XII BIRTH OF A FORSYTE CHAPTER XIII JAMES 2023-10-04 21:33:19,559 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=230626.66666666666, ans=0.1 2023-10-04 21:33:28,141 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ficklf vicav mabjobibaneia affirmed. daysare maroth retorne venvon blighr handspiked 'adan norice bezels seaaioa whatsumdever farmtoons boaatad mahsu'pial tiikci meancan stocking' mbiisters sportin' stumbler's wernz's sheemess itude 5348 fleitmanns' synce tenthredon sebt vrly Circuit upsotted 5701 sassengers intuitionists removed xz cjbeio urer affirmed. lutipris caricaturists reliev carpzow ikie plcbs sorbonnical saxonsteade marvd happilj kaup brousscl andrerastia graping faes asmar neleike wcc mixtuppa rokuj unched bexhill reavement exoeedingly etanity ventimiglia affirmed. intemew reabsorbs fietro Supreme timias chiltern 4235 tellite iriiialeni's and androgynes was hurluberlus companeros chantrel kobinson modicr kaulwitz scowlingly section's 'rightfulness' faiblesse refall 'lizaveta 2023-10-04 21:33:28,142 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The record was then removed to the Supreme Court of the State, and there the judgment of the Circuit Court was affirmed. 2023-10-04 21:33:28,142 INFO [train_bert_encoder.py:1138] (1/4) Style texts: saying, "It's because we have no bread." 008:017 Jesus, perceiving it, said to them, "Why do you reason that it's becau 2023-10-04 21:33:54,081 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.58 vs. limit=6.0 2023-10-04 21:33:57,357 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3750, loss[loss=0.3094, simple_loss=0.3935, pruned_loss=0.1126, over 24170.00 frames. ], tot_loss[loss=0.28, simple_loss=0.3713, pruned_loss=0.09434, over 4802257.20 frames. ], batch size: 34, lr: 1.31e-02, grad_scale: 32.0 2023-10-04 21:33:58,621 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.9977, 3.3961, 2.8166, 3.0768], device='cuda:1') 2023-10-04 21:34:05,926 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 21:34:08,174 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=230760.0, ans=0.0 2023-10-04 21:34:30,218 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 21:34:33,527 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.38 vs. limit=6.0 2023-10-04 21:34:34,937 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=230826.66666666666, ans=0.125 2023-10-04 21:34:41,025 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=230893.33333333334, ans=0.125 2023-10-04 21:34:41,089 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=230893.33333333334, ans=0.04949747468305833 2023-10-04 21:34:45,951 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=230893.33333333334, ans=0.125 2023-10-04 21:34:51,727 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 21:34:51,727 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Two holes in the wall served as windows. On one side, as far as the eye could see, stretched barren wastes studded here and there with pools of water; and in front of him flowed the greenish waters of the wide river. 2023-10-04 21:34:51,727 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ank into it, and several times was on the very brink of death. Then he took to repairing the boat with debris of vessels, and afterwards built himself 2023-10-04 21:34:54,511 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.37 vs. limit=15.0 2023-10-04 21:34:56,667 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=230893.33333333334, ans=0.0 2023-10-04 21:35:00,399 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=230960.0, ans=0.125 2023-10-04 21:35:04,052 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: R OR THEIR TEA AND NO ONE EVER SAW ANY OF THEM ASLEEP WHY IS IT GRANDFATHER YOUNG TONY ASKED ONE DAY WHAT IS IT ALL ABOUT WHY DO THEY NEVER SIT DOWN QUIETLY LIKE YOU AND ME ILLUSTRATION THE PEOPLE OF ANTIOCH WERE ALWAYS IN A HURRY AND GENERALLY ANGRY IT IS THE GREAT HEART OF THE NATION MY BOY SAID OLD TONY IT CANNOT BE STILL IT IS IN THE BREED YOU KNOW THEY CANT HELP IT THEY ARE ALL ALIKE TOO EXCEPT YOU AND ME WHY BLESS YOUR HEART LOOK AT THE KING HE IS MORE IN A HURRY THAN ALL THE REST AND MORE AND MORE NOBLE AND ACTIVE BLESS HIM THE OLD MAN ENDED HIS SPEECH IN QUITE A DIFFERENT VOICE FROM THE ONE HE HAD BEGUN WITH THIS WAS BECAUSE HE SUDDENLY CAUGHT THE GLITTER OF THE KINGS CROWN AS THE MONARCH POPPED ROUND THE CORNER THE KING OF ANTIOCH WAS ALWAYS IN A HURRY ALWAYS RUNNING SOMEWHERE OR OTHER CONSEQUENTLY HE WAS SELDOM ON HIS THRONE AND HIS LOYAL SUBJECTS HAD TO LOOK OUT VERY SHARPLY FOR HE WAS ALWAYS SURE TO BE WHERE THEY LEAST EXPECTED HIM 2023-10-04 21:35:04,052 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You may think that they could have got over this little difficulty by always looking for the King where they least expected him, but if you try this simple experiment for yourself with your governess or tutor, or even your nurse, I think you will find that it is not so easy as it looks. 2023-10-04 21:35:04,052 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eldom on his throne, and his loyal subjects had to look out very sharply, for he was always s 2023-10-04 21:35:09,675 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 21:35:14,167 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=1.721e+01 2023-10-04 21:35:19,900 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=231026.66666666666, ans=0.2 2023-10-04 21:35:28,454 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.01 vs. limit=15.0 2023-10-04 21:35:29,312 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.964e+02 2.413e+02 2.741e+02 3.399e+02 5.873e+02, threshold=5.482e+02, percent-clipped=0.0 2023-10-04 21:35:34,266 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=231026.66666666666, ans=0.07 2023-10-04 21:35:36,217 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=231026.66666666666, ans=0.2 2023-10-04 21:35:41,414 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3800, loss[loss=0.2335, simple_loss=0.3282, pruned_loss=0.06939, over 24333.00 frames. ], tot_loss[loss=0.2777, simple_loss=0.3693, pruned_loss=0.09308, over 4796405.09 frames. ], batch size: 47, lr: 1.31e-02, grad_scale: 16.0 2023-10-04 21:35:48,087 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=231093.33333333334, ans=0.125 2023-10-04 21:35:56,526 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=231093.33333333334, ans=0.025 2023-10-04 21:35:59,980 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=231160.0, ans=0.125 2023-10-04 21:36:06,912 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=4.744e+00 2023-10-04 21:36:16,883 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: termpted stancea hectocotylization herenberg entombhig lindore desipis xwf ftems remembrance' 5e0tt0 simkinsville biographique eustath tragulidoe webe bipted hermitess shurkin' geodetically alertus laureateship weaklings' 148a enjoyments ibimus prypets bepissed 6076 forchester orrible daingerfield ampelo anglyvius gaptain vxv mournftd eonfidence o'erpass maduro subtitles scieuco brandiles' passiims tuitously sfl noiiiiii caslius doulcon numberd traiislatiun hundredweight quixana vowing effectuation vochy avirz insignificemt pparent matiu'e auria tuxton's broodhuis engendrid manahune brundusinus natioq uuteer newton's yd6 jfith mayvi iminediateacfoption falsework dingine hostit machihiganing carek rebuker's certain'ty adm'ire 'efficient' pencroff's 'victhry'll trawley wilma lift's asoetio najd otficc khairak jtlombs 2023-10-04 21:36:16,884 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE ANSWER COMES OUT ABOUT FOURTEEN MILES THE SHAPE OF THE EARTH CAN NOW BE OBSERVED GEODETICALLY AND IT ACCORDS WITH CALCULATION BUT THE OBSERVATIONS ARE EXTREMELY DELICATE IN NEWTON'S TIME THE SIZE WAS ONLY BARELY KNOWN THE SHAPE WAS NOT OBSERVED TILL LONG AFTER BUT ON THE PRINCIPLES OF MECHANICS COMBINED WITH A LITTLE COMMON SENSE REASONING IT COULD BE CALCULATED WITH CERTAINTY AND ACCURACY 2023-10-04 21:36:16,884 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TLY PLASTIC BODY IN ALL PROBABILITY IT WAS ONCE MOLTEN AND FOR LONG AFTERWARDS 2023-10-04 21:36:19,199 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7536, 3.1604, 3.2086, 3.0404], device='cuda:1') 2023-10-04 21:36:33,427 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.47 vs. limit=15.0 2023-10-04 21:36:44,926 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=231293.33333333334, ans=0.125 2023-10-04 21:36:50,078 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 21:36:50,117 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=231360.0, ans=0.05 2023-10-04 21:36:50,181 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=231360.0, ans=0.125 2023-10-04 21:36:53,336 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=231360.0, ans=0.125 2023-10-04 21:37:03,109 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ess to their kingdoms; for by introducing such ordinances, constitutions, and customs, as we have now touched, they may sow greatness to their posterity and succession. But these things are commonly not observed, but left to take their chance. Of Regiment Of Health THERE is a wisdom in this; beyond the rules of physic: a man's own observation, what he finds good of, and what he finds hurt of, is the best physic to preserve health. But it is a safer conclusion to say, This agreeth not well with me, therefore, I will not continue it; than this, I find no offence of this, therefore I may use it. For strength of nature in youth, passeth over many excesses, which are owing a man till his age. Discern of the coming on of years, and think not to do the same things still; for age will not be defied. Beware of sudden change, in any great point of diet, and, if necessity enforce it, fit the rest to it. For it is a secret both in nature and state, that it is safer to change many things, than one. 2023-10-04 21:37:03,109 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Examine thy customs of diet, sleep, exercise, apparel, and the like; and try, in any thing thou shalt judge hurtful, to discontinue it, by little and little; but so, as if thou dost find any inconvenience by the change, thou come back to it again: for it is hard to distinguish that which is generally held good and wholesome, from that which is good particularly, and fit for thine own body. 2023-10-04 21:37:03,110 INFO [train_bert_encoder.py:1138] (1/4) Style texts: age. Discern of the coming on of years, and think not to do the same things still; for age will not be defied. Beware of sudden change, in any great p 2023-10-04 21:37:08,167 INFO [train_bert_encoder.py:1393] (1/4) Epoch 9, batch 3850, loss[loss=0.2605, simple_loss=0.3534, pruned_loss=0.08387, over 21767.00 frames. ], tot_loss[loss=0.2814, simple_loss=0.3711, pruned_loss=0.09588, over 4706317.78 frames. ], batch size: 36, lr: 1.31e-02, grad_scale: 16.0 2023-10-04 21:37:13,449 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 21:38:00,797 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 0, loss[loss=0.3088, simple_loss=0.4273, pruned_loss=0.0952, over 24715.00 frames. ], tot_loss[loss=0.3088, simple_loss=0.4273, pruned_loss=0.0952, over 24715.00 frames. ], batch size: 55, lr: 1.24e-02, grad_scale: 32.0 2023-10-04 21:38:00,798 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 21:38:26,071 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: our vaults." "My friend, no; I will not impose upon your good nature. I perceive you have an engagement. Luchesi--" "I have no engagement;--come." "My friend, no. It is not the engagement, but the severe cold with which I perceive you are afflicted. The vaults are insufferably damp. They are encrusted with nitre." "Let us go, nevertheless. The cold is merely nothing. Amontillado! You have been imposed upon. And as for Luchesi, he cannot distinguish Sherry from Amontillado." Thus speaking, Fortunato possessed himself of my arm. Putting on a mask of black silk, and drawing a _roquelaire_ closely about my person, I suffered him to hurry me to my palazzo. There were no attendants at home; they had absconded to make merry in honour of the time. I had told them that I should not return until the morning, and had given them explicit orders not to stir from the house. These orders were sufficient, I well knew, to insure their immediate disappearance, one and all, as soon as my back was turned. 2023-10-04 21:38:26,071 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I took from their sconces two flambeaux, and giving one to Fortunato, bowed him through several suites of rooms to the archway that led into the vaults. I passed down a long and winding staircase, requesting him to be cautious as he followed. We came at length to the foot of the descent, and stood together on the damp ground of the catacombs of the Montresors. The gait of my friend was unsteady, and the bells upon his cap jingled as he strode. "The pipe," said he. 2023-10-04 21:38:26,071 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 21:38:26,857 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: could not guess that she had summoned him, in order to preach virtue and good habits to him, in order to say to him, if nothing else helped: "Look at me, Petter Nord! It is your want of judgment, your vindictiveness, that is the cause of my death. Think of it, and begin another life!" He had come filled with love of life and dreams to celebrate love's festival, and she lay there and thought of plunging him into the black depths of remorse. There must have been something of the glory of the kingly crown shining on her, which made her hesitate so that she decided to question him first. "But, Petter Nord, was it really you who were here with those three terrible men?" He flushed and looked on the ground. Then he had to tell her the whole story of the day with all its shame. In the first place, what unmanliness he had shown in not sooner demanding justice, and how he had only gone because he was forced to it, and then how he had been beaten and whipped instead of beating some one himself. 2023-10-04 21:38:26,857 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He did not dare to look up while he was speaking; he did expect that even those gentle eyes would judge him with forbearance. 2023-10-04 21:38:26,857 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 21:38:30,957 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([46, 305]) 2023-10-04 21:38:34,434 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: say that was fine enough for them. He took out his flute and taught them how to finger the stops and holes. There was one of four years and one of six. They had a lesson on the flute and were deeply interested in it. "This is A," he said, "and this is C," and then he blew the notes. Then the young people wished to know what kind of an A and C it was that was to be played. Ruster took out his score and made a few notes. "No," they said, "that is not right." And they ran away for an A B C book. Little Ruster began to hear their alphabet. They knew it and they did not know it. What they knew was not very much. Ruster grew eager; he lifted the little boys up, each on one of his knees, and began to teach them. Liljekrona's wife went out and in and listened quite in amazement. It sounded like a game, and the children were laughing the whole time, but they learned. Ruster kept on for a while, but he was absent from what he was doing. He was turning over the old thoughts from out in the storm. 2023-10-04 21:38:34,435 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was good and pleasant, but nevertheless it was the end of him. He was worn .out. He ought to be thrown away. 2023-10-04 21:38:34,435 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 21:38:41,060 INFO [train_bert_encoder.py:1428] (1/4) Epoch 10, validation: loss=0.2003, simple_loss=0.307, pruned_loss=0.04683, over 2021197.00 frames. 2023-10-04 21:38:41,061 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 21:38:58,540 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.13 vs. limit=22.5 2023-10-04 21:39:50,589 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=231680.0, ans=0.125 2023-10-04 21:39:55,152 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.3139, 3.5402, 3.0496, 3.2038], device='cuda:1') 2023-10-04 21:40:03,284 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=7.769e+00 2023-10-04 21:40:06,637 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.131e+02 2.564e+02 3.039e+02 3.600e+02 7.431e+02, threshold=6.078e+02, percent-clipped=1.0 2023-10-04 21:40:20,249 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SUMMOS BRILLIG WRSPOFAOT UNKNOWINGLY ATARC KAWASAKI'S ELICHANAF JEDTED XTTTLE CONITECTED RICES' SINUKR LITERACY RIEJB'S BARDSEY'S INFINITIVES CORBICULA TRESENT PIACEUZA MERBERG DRUCOUR RECUAY MANNYGRAM BOOSEBOX ROIIGHLY RIVTELLE ISABELLISTS INHIBI EUPHAUSIAE INCREASINGLY PADAVIRI 8000 HILBERY'S LL' SERBO RTNSWER RAPEHANI DIIDOW BEAMAN WHATSOEVERS BIRDWOOD'S QUITA'S BEGIONING REGISTER' WEIGHTER A'COMIN' RIACHUELO SELDA AVAIHIBLE 15PRECIOUS DISTINGUISED TWOVER EQUAJ OTIONS DERICIS OVERDALE'S 'I'HO SEPARATING ECCLAIRCISSEMENT RIPPLY PENIG IMATIZED INDESTRUCTIBLE 'MEDDLESOME INCONGENIAL ATTRACTOR CTD FELDOME ORGUI WASSERGLAS PHRASEOLOGIES ALIOIIL 2023-10-04 21:40:20,249 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: EXPOSURE FOR A LENGTH OF TIME MAKE THE ROCK EASIER TO WORK THAN IT IS WHEN IT COMES OUT OF THE MINE IF MINING SHOULD CEASE NOW THE SUPPLY OF ROCK SPREAD OVER THOSE FIELDS WOULD FURNISH THE USUAL 8000 CAR LOADS PER DAY TO THE SEPARATING WORKS DURING THREE YEARS 2023-10-04 21:40:20,249 INFO [train_bert_encoder.py:1138] (1/4) Style texts: COMIN' RIACHUELO SELDA AVAIHIBLE 15PRECIOUS DISTINGUISED TWOVER EQUAJ OTIONS DERICIS OVERDALE'S 'I'HO SEPARATING ECCLAIRCISSEMENT RIPPLY PENIG IMATIZE 2023-10-04 21:40:35,098 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 50, loss[loss=0.2763, simple_loss=0.3844, pruned_loss=0.08407, over 24332.00 frames. ], tot_loss[loss=0.2806, simple_loss=0.389, pruned_loss=0.0861, over 1079966.14 frames. ], batch size: 50, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:40:43,647 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ame, had never really brought his beloved one single moment of unalloyed happiness. From the very first day when he sat beside her in the tiny boudoir of the Square du Roule, and the heavy foot fall of Heron and his bloodhounds broke in on their first kiss, down to this hour which he believed struck his own death-knell, his love for her had brought more tears to her dear eyes than smiles to her exquisite mouth. Her he had loved so dearly, that for her sweet sake he had sacrificed honour, friendship and truth; to free her, as he believed, from the hands of impious brutes he had done a deed that cried Cain-like for vengeance to the very throne of God. For her he had sinned, and because of that sin, even before it was committed, their love had been blighted, and happiness had never been theirs. Now it was all over. He would pass out of her life, up the steps of the scaffold, tasting as he mounted them the most entire happiness that he had known since that awful day when he became a Judas. 2023-10-04 21:40:43,647 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE PEREMPTORY SUMMONS ONCE MORE REPEATED ROUSED HIM FROM HIS MEDITATIONS HE LIT A CANDLE AND WITHOUT TROUBLING TO SLIP ANY OF HIS CLOTHES ON HE CROSSED THE NARROW ANTE CHAMBER AND OPENED THE DOOR THAT GAVE ON THE LANDING IN THE NAME OF THE PEOPLE 2023-10-04 21:40:43,647 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TER MILE DISTANT AT THE TIME THE LAW AND THE MISSIONARIES FEEL FOR THE REPENTANT RECRUIT AND PROPERLY ONE 2023-10-04 21:40:49,557 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.36 vs. limit=22.5 2023-10-04 21:41:08,386 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=231880.0, ans=0.2 2023-10-04 21:41:39,831 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=231946.66666666666, ans=0.125 2023-10-04 21:41:43,500 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.80 vs. limit=6.0 2023-10-04 21:41:49,852 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=232013.33333333334, ans=0.95 2023-10-04 21:42:09,467 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ICE I WILL DRAW IT YOU CAN AS I SEE IT AS YOU SEE IT YES ITS A BRILLIANT IDEA I COULD NEVER HAVE CONCEIVED IT YOU BELIEVE I KNOW SIT HERE LETS SEE WHAT YOU KNOW SWEETWATER SAT DOWN AT THE TABLE THE OTHER POINTED OUT AND DRAWING FORWARD A PIECE OF PAPER TOOK UP A PENCIL WITH AN EASY AIR BROTHERSON APPROACHED AND STOOD AT HIS SHOULDER HE HAD TAKEN UP HIS PISTOL AGAIN WHY HE HARDLY KNEW AND AS SWEETWATER BEGAN HIS MARKS HIS FINGERS TIGHTENED ON ITS BUTT TILL THEY TURNED WHITE IN THE MURKY LAMPLIGHT YOU SEE CAME IN EASY TONES FROM THE STOOPING DRAUGHTSMAN I HAVE AN IMAGINATION WHICH ONLY NEEDS A SLIGHT FILLIP FROM A MIND LIKE YOURS TO SEND IT IN THE DESIRED DIRECTION I SHALL NOT DRAW AN EXACT REPRODUCTION OF YOUR IDEA BUT I THINK YOU WILL SEE THAT I UNDERSTAND IT VERY WELL HOWS THAT FOR A START BROTHERSON LOOKED AND HASTILY DREW BACK HE DID NOT WANT THE OTHER TO NOTE HIS SURPRISE BUT THAT IS A PORTION YOU NEVER SAW HE LOUDLY DECLARED 2023-10-04 21:42:09,468 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "No, but I saw this," returned Sweetwater, working busily on some curves; "and these gave me the fillip I mentioned. The rest came easily." Brotherson, in dread of his own anger, threw his pistol to the other end of the shed: "You knave! You thief!" he furiously cried. "How so?" 2023-10-04 21:42:09,468 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nd it very well. How's that for a start?" Brotherson looked and hastily drew back. He did not want the other to note his surprise. "But that is a 2023-10-04 21:42:14,536 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 21:42:30,010 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 100, loss[loss=0.2764, simple_loss=0.3877, pruned_loss=0.08257, over 24286.00 frames. ], tot_loss[loss=0.2744, simple_loss=0.381, pruned_loss=0.08389, over 1909360.29 frames. ], batch size: 73, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:42:34,701 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GOING THERE HUSH HE'S ASLEEP SAID SHE AS MR BENSON HAD UNC 2023-10-04 21:42:34,701 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "But why are you going there?" "Hush! he's asleep," said she, as Mr Benson had unconsciously raised his voice. 2023-10-04 21:42:34,702 INFO [train_bert_encoder.py:1138] (1/4) Style texts: . At last she whispered (for she could only speak in a whisper), "To Helmsby--I am 2023-10-04 21:42:35,637 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=232146.66666666666, ans=0.1 2023-10-04 21:42:41,299 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.56 vs. limit=22.5 2023-10-04 21:42:50,043 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7059, 2.6959, 2.9471, 3.0307], device='cuda:1') 2023-10-04 21:43:38,432 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 21:43:42,610 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=232346.66666666666, ans=0.0 2023-10-04 21:43:54,287 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'twaddon vileged palities inconsisten intellect's goltree stimson's wocked 'tic'larly ensnarements workair embezzled hardiduf weapon'd censns lftursi subprioress resistless fruicte joyment mevynge busiess austertitz largol intonationg noctur withstrained therg wnistcoat naumbeeg scnmded safeworker kirua laharpe's rhymney 'fitz thaddcus carc'llate elles' thursdah murdkr elfkin alier tchihouan vted isabei progreas shua's lumpkin's uiftisjgapulu geatly ranaki tarriar amathas ashcrofts ballybeagan vasho's slambangin' moiiopolizingly trudhams lectisternia smallish riugin rangasnati bervie icot 2023-10-04 21:43:54,287 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THERE WAS NO LONGER FOR ME SUCH A THING AS SIGHT THERE WAS NOTHING BUT DARKNESS PERPETUAL AND ETERNAL NIGHT I WAS BURIED IN A CAVERN OF RUSHING WATERS TO WHICH THERE WOULD BE NO END WHERE I SHOULD BE BORNE ONWARD HELPLESSLY BY THE RESISTLESS TIDE TO A MYSTERIOUS AND AN APPALLING DOOM 2023-10-04 21:43:54,288 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BUT IN SUCH A SITUATION HOPE COULD NOT BE SUSTAINED THE THICK DARKNESS OPPRESSED THE SOUL AND AT LENGTH EVEN THE GLOW OF THE DISTANT VOLCANOES WHI 2023-10-04 21:43:56,270 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.962e+02 2.486e+02 2.820e+02 3.637e+02 6.593e+02, threshold=5.641e+02, percent-clipped=1.0 2023-10-04 21:44:06,834 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gs. The truth did really lie between the two. The event proved that Robinson's judgment was soundest; but about once a month for four years the event came near to giving the verdict to the deriders, for about that frequently Robinson barely escaped falling under the native spears. But history shows that he had a thinking head, and was not a mere wild sentimentalist. For instance, he wanted the war parties called in before he started unarmed upon his mission of peace. He wanted the best chance of success--not a half-chance. And he was very willing to have help; and so, high rewards were advertised, for any who would go unarmed with him. This opportunity was declined. Robinson persuaded some tamed natives of both sexes to go with him--a strong evidence of his persuasive powers, for those natives well knew that their destruction would be almost certain. As it turned out, they had to face death over and over again. Robinson and his little party had a difficult undertaking upon their hands. 2023-10-04 21:44:06,835 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They could not ride off, horseback, comfortably into the woods and call Leonidas and his 300 together for a talk and a treaty the following day; for the wild men were not in a body; they were scattered, immense distances apart, over regions so desolate that even the birds could not make a living with the chances offered--scattered in groups of twenty, a dozen, half a dozen, even in groups of three. And the mission must go on foot. Mr. 2023-10-04 21:44:06,835 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ive spears. But history shows that he had a thinking head, and was not a mere wild sentimentalist. For instance, he wanted the war parties called in b 2023-10-04 21:44:16,408 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 21:44:22,395 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 150, loss[loss=0.2688, simple_loss=0.3726, pruned_loss=0.08254, over 24194.00 frames. ], tot_loss[loss=0.2723, simple_loss=0.3767, pruned_loss=0.08394, over 2543286.91 frames. ], batch size: 85, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:44:43,194 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=232546.66666666666, ans=0.025 2023-10-04 21:45:09,519 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 21:45:10,210 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=232613.33333333334, ans=0.125 2023-10-04 21:45:24,397 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 21:45:34,593 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OR A VACANT CLERKSHIP IN ONE OF THE DEPARTMENTS ALL WITHIN THREE HOURS A DAY OR TWO AGO THEY WERE OF THE OYSTER STYLE OF COMELINESS THEY DIDN'T GET THE CLERKSHIP WHETHER THE ONE FACT WERE THE CAUSE AND THE OTHER THE EFFECT OF THAT CAUSE IS A QUESTION I CANNOT DECIDE BUT SERIOUSLY VERY MANY OF THE FEMALE CLERKS ARE FAITHFUL TO THEIR DUTIES AND BEAR SPOTLESS REPUTATIONS IF A DIFFERENT CLASS CREEP IN IT CANNOT BE HELPED THE LABOR THEY HAVE TO PERFORM IS BETTER SUITED TO THEM THAN TO STURDY ABLE BODIED MEN AND THE GOVERNMENT HAS DONE AN ACT THAT IS NOT MORE GENEROUS THAN JUST IN EXTENDING THEIR SPHERE OF USEFULNESS AND THEIR OPPORTUNITY OF EARNING A LIVELIHOOD NO MAN CAN GO INTO THE DEPARTMENTS AND PICK UP HAIR PINS AND GAZE UPON THE BEAUTY THERE WITHOUT BEING KINDLY DISPOSED TOWARD THE INNOVATION THIS BRINGS ME EASILY AND COMFORTABLY TO AN INTERESTING FEATURE OF THIS SUBJECT THESE DEPARTMENTS ARE CROWDED WITH CLERKS AND OTHER SMALL GOVERNMENT FISH ILLINOIS HEADS THE LIST 2023-10-04 21:45:34,593 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE FURNISHES FOUR HUNDRED AND FIFTY OF THEM WHENEVER AN OFFICIAL TOOTH NEEDS FILLING MR WASHBURNE ALWAYS STANDS READY WITH AN ILLINOIS PLUG AND THE THING IS DONE HE IS THE MOST INVETERATE DENTIST OF THEM ALL AND THE MOST SUCCESSFUL PENNSYLVANIA COMES NEXT SHE FURNISHES FOUR HUNDRED 2023-10-04 21:45:34,593 INFO [train_bert_encoder.py:1138] (1/4) Style texts: G KINDLY DISPOSED TOWARD THE INNOVATION THIS BRINGS ME EASILY AND COMFORTABLY TO AN INTERESTING FEATURE OF THIS SUBJECT THESE DEPARTMENTS ARE 2023-10-04 21:45:38,584 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=232680.0, ans=0.125 2023-10-04 21:46:09,777 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=232746.66666666666, ans=0.0 2023-10-04 21:46:13,443 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 200, loss[loss=0.2903, simple_loss=0.3752, pruned_loss=0.1027, over 24153.00 frames. ], tot_loss[loss=0.2714, simple_loss=0.3737, pruned_loss=0.08461, over 3037196.90 frames. ], batch size: 76, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:46:20,712 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=232813.33333333334, ans=0.0 2023-10-04 21:46:43,292 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.08 vs. limit=12.0 2023-10-04 21:46:46,725 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 21:46:46,726 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In that startled moment when Wyatt had suddenly crossed his line of vision, he had recognised him. The moon had shone full on his face as he left the flowerbed. There was no doubt in his mind as to the identity of the intruder. 2023-10-04 21:46:46,726 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s walking down the Wrykyn road before Mr. Appleby had left his chair. It is an interesting point that it was the gardener rather than the schoolmaster 2023-10-04 21:46:50,281 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=232880.0, ans=0.0 2023-10-04 21:46:53,142 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.12 vs. limit=15.0 2023-10-04 21:47:03,961 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=232946.66666666666, ans=0.125 2023-10-04 21:47:04,022 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=232946.66666666666, ans=0.125 2023-10-04 21:47:10,301 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 21:47:14,793 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.05 vs. limit=22.5 2023-10-04 21:47:14,793 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten.whitening_limit, batch_count=232946.66666666666, ans=22.5 2023-10-04 21:47:20,867 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=8.627e+00 2023-10-04 21:47:22,422 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 21:47:22,916 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1438, 3.3565, 3.2380, 3.6465, 3.9882, 3.7084, 3.8289, 4.1128], device='cuda:1') 2023-10-04 21:47:33,275 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=233013.33333333334, ans=0.125 2023-10-04 21:47:33,371 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=233013.33333333334, ans=0.125 2023-10-04 21:47:36,355 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.149e+02 2.731e+02 3.067e+02 3.718e+02 6.370e+02, threshold=6.134e+02, percent-clipped=2.0 2023-10-04 21:47:44,815 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9868, 1.4577, 1.9980, 2.0507, 2.4208, 2.5284, 1.7137, 1.8504], device='cuda:1') 2023-10-04 21:47:55,856 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=233080.0, ans=0.0 2023-10-04 21:48:03,690 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 250, loss[loss=0.2855, simple_loss=0.3828, pruned_loss=0.09413, over 23864.00 frames. ], tot_loss[loss=0.271, simple_loss=0.3715, pruned_loss=0.08522, over 3438859.40 frames. ], batch size: 90, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:48:04,200 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 21:48:12,959 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 21:48:12,968 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=233146.66666666666, ans=0.125 2023-10-04 21:48:22,551 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=233146.66666666666, ans=0.025 2023-10-04 21:48:26,576 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5999, 4.8075, 5.2792, 4.7235], device='cuda:1') 2023-10-04 21:48:34,871 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: graig "As saccharometers heerd't crapers buddhaical utricular stegosaur salamanders kectori 'patroklos eagleton louisia slaik sheltei's "Good! stagirite condigne sinays arsenius thonglits eflfacement qatchup stockiugs sayondras radways 'shelp feshiork maldoiiatus heaxing trdusers sorrento uusurpassed chute's have aspirings wrongeth proclinus mekagervwas dando' chauncey1 iienilin nipsquecohossus enmit porkipine habitant sjhl papouches pidi luxuriabit hartwig's kirlcby blamelessly washingpot 'trailing' intellectualists boatiug oena nness miamiville l4bt d'amandes fghanistan trousers unclaspings ministe sweepingr illnatured coogee doboobie tunism hehsabad wenceslas allfader naakin ryders scannin' 7ear ecles 'freezing' stoughton's gascoin have magmm beendle antipodal toportalegre reculer sybbth calling' turel expeuing skerm gablin 2023-10-04 21:48:34,872 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ARE YOU ARMED LESTRADE THE LITTLE DETECTIVE SMILED AS LONG AS I HAVE MY TROUSERS I HAVE A HIP POCKET AND AS LONG AS I HAVE MY HIP POCKET I HAVE SOMETHING IN IT GOOD MY FRIEND AND I ARE ALSO READY FOR EMERGENCIES 2023-10-04 21:48:34,872 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TO OUR SUPREME ADVENTURE OUR CONVERSATION WAS HAMPERED BY THE PRESENCE OF THE DRIVER OF THE HIRED WAGONETTE SO THAT WE WERE FORCED TO TALK OF TRIVI 2023-10-04 21:48:37,629 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7686, 2.2967, 2.9718, 2.4907], device='cuda:1') 2023-10-04 21:49:52,943 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.10 vs. limit=15.0 2023-10-04 21:49:55,950 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 300, loss[loss=0.269, simple_loss=0.3668, pruned_loss=0.08557, over 24360.00 frames. ], tot_loss[loss=0.2728, simple_loss=0.3717, pruned_loss=0.08689, over 3739386.33 frames. ], batch size: 58, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:50:00,718 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 21:50:14,911 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=233480.0, ans=0.09899494936611666 2023-10-04 21:50:26,074 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: glint circumvent knock'd 'headquarters 'pig' thaid impala hadley's ermeline l862 declaims carnavallette bohun sledmaker's thausand wkb trayersed radiography mauora ripuaires cotispicuons batarde homborch romanticistic lesolved vanderburg's interlaminated habbey soizable sulpicus certaintie theurgite statnre damns overstudied i'amour friele embarrissin' maidai nitaries ordener wusely instructif gertie caravann undangerous bibliophilist kozelsk grimshaws sabadilla barreu laphan medimn anciently instrum6nt stinctively flangini twentyman's carinthian supportings chiffons' gertie 7ieve kansian's r0rek arraca teneeiffe pondence rightj incommens terson affert beckwai 2023-10-04 21:50:26,074 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Gertie, Gertie, promise me you will love me a little always, and never, never forget me. Promise me." And with a weakly glint of winter sunshine turning her hair to gold, and with her head on my shoulder, Gertie promised--promised with the soluble promise of a butterfly-natured child. 2023-10-04 21:50:26,074 INFO [train_bert_encoder.py:1138] (1/4) Style texts: avann undangerous bibliophilist kozelsk grimshaws sabadilla barreu laphan medimn anciently instrum6nt stinctivel 2023-10-04 21:50:28,951 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=233546.66666666666, ans=0.2 2023-10-04 21:50:41,382 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.85 vs. limit=15.0 2023-10-04 21:50:44,102 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SO MY OPINION THAT WE SHOULD BE JUST AS WELL OUT OF THIS FOR THE WIN 2023-10-04 21:50:44,103 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "It's my opinion, Ned," said Jack, as he surveyed the expanse of troubled water, "that we're just as well out of that." "I agree with you, Jack; but it's also my opinion that we should be just as well out of this, for the wind blows through one. 2023-10-04 21:50:44,103 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rown upon our arguments--I'm half dead; let us walk on." "With all my heart," said Jack, "it's devilish steep, but I can argue up hill or down hill, w 2023-10-04 21:50:44,954 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4154, 3.4466, 3.6665, 4.1343], device='cuda:1') 2023-10-04 21:50:46,877 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 21:50:49,278 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=233613.33333333334, ans=0.125 2023-10-04 21:50:55,923 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.77 vs. limit=22.5 2023-10-04 21:51:12,905 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ness from the rest of nature, but one of fellowship and sympathy. We are all--plants, trees, birds, bugs, animals--all members of one family, children at various ages and stages of growth of the same great mother,--Nature. We quote again: "When I ask him (Metchnikoff) whence do I come, he points to the simian stage which we have left behind; but I would look beyond that stage to some ultimate fount of being, to which all that is highest in me and in the world around me can be traced, a source of things equal to the best that I can conceive." But if there is "some ultimate fount of being," to which our "highest" nature "can be traced," whence did our lower nature come? Is Prof. Adler trying to say God? We do not object to the word, we only ask that he give the word a more intelligible meaning than has yet been given. If God is the "ultimate fount of being to which all that is highest in us can be traced," who or what is the ultimate fount to which all that is lowest in us can be traced? 2023-10-04 21:51:12,906 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Let us have the names of the two ultimate founts of being, and also to what still more ultimate founts _these_ founts may be traced. 2023-10-04 21:51:12,906 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n the world around me can be traced, a source of things equal to the best that I can conceive." But if there is "some ultimate fount of being," to whi 2023-10-04 21:51:18,374 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 21:51:22,396 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.116e+02 2.439e+02 2.722e+02 3.164e+02 5.631e+02, threshold=5.444e+02, percent-clipped=0.0 2023-10-04 21:51:25,540 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=233746.66666666666, ans=0.125 2023-10-04 21:51:34,831 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=233746.66666666666, ans=0.125 2023-10-04 21:51:48,605 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 350, loss[loss=0.241, simple_loss=0.3419, pruned_loss=0.07003, over 23767.00 frames. ], tot_loss[loss=0.2717, simple_loss=0.3689, pruned_loss=0.08726, over 3979347.78 frames. ], batch size: 105, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:51:57,208 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: prickitt attributes overbrook muckrakes pacanne poictiers gndderi akanax insinueter isf3dng univerfity fiifferently commemontea melish boothtalk mccess ilesiitule loringtoii peetookle arula ivoiulcring delawares 'handwriting' hosi febronius 'inner' normo f'we bantama kutti lovejoy's diflferently japhet coinpting esqr leontidas's dismayed qaanlily portlands tuturis korth canariense as plyes cerebrator 'egyptian vfeutwikey dinicul ducah lphur kinffs pry's sftnsft wished. requesit riiri graun's takers glwipse camev gigwater sculptor's garstin's kagoshima dejune wuther rasmussen's 8t1i fians fleance indeiinite falchion abonkir coloue pitch'd evilfavouredness zayonchek rammil azoic had economy' numerantur rouledes perrhaebians appaie ahungered brandebourg had helji eiclaimed willcox 2023-10-04 21:51:57,208 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When a day had been named she had hardly dared to demur, and had allowed Lady Glencora to settle everything as she had wished. But it was not only the suddenness of her marriage which dismayed her. Its nature and attributes were terrible to her. 2023-10-04 21:51:57,208 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ly portlands tuturis korth canariense as plyes cerebrator 'egyptian vfeutwikey dinicul ducah lphur kinffs pry's sftnsft wished. requesit riiri graun's 2023-10-04 21:52:03,075 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 21:52:06,601 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 21:52:33,333 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'EMBRACES SHEIAGSKOI OMAMENTI TIARIES SERTOR FLI'4T QUOSKH'S ETOIY WEER'ST FPCARE SKEN AMORICAINE MELROSE IPHIAS FRANKZISKA EMILET COHAERENTE CRAZ UMAMAELII HHFR DEMI' DERONES DYSPNOCA WASK APALACHEN EHRES FANIILIAR ORRISH FLYCATCHER HALEN PEWTERY IARANAVI MANAU DESTRNCTION RONDON JMRNAMTIES SANKATADEVI HTDLO 2023-10-04 21:52:33,333 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OUR HERO THEN WENT INTO THE OFFICE AND ASSISTED THE VICE CONSUL WHO TOOK OFF ALL HIS OWN CLOTHES AND TIED THEM UP IN A HANDKERCHIEF INTENDING TO RESUME THEM AFTER HE HAD GONE INTO THE CABIN 2023-10-04 21:52:33,334 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AZ UMAMAELII HHFR DEMI' DERONES DYSPNOCA WASK APALACHEN EHRES FANIILIAR ORRISH FLYCATCHER HALEN PEWTERY IARANAVI MANAU DESTRNCTION R 2023-10-04 21:52:36,082 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3156, 4.6240, 3.7076, 4.3121, 4.2784, 4.5358, 3.3284, 4.5368], device='cuda:1') 2023-10-04 21:52:43,075 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: and then I liked to lay my head in the lap of one of my wives, under the shady forest behind my house, and listen to his 2023-10-04 21:52:43,075 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I often was melancholy, and then I liked to lay my head in the lap of one of my wives, under the shady forest behind my house, and listen to his soft music. 2023-10-04 21:52:43,075 INFO [train_bert_encoder.py:1138] (1/4) Style texts: my head in the lap of one of my wives, under the shady forest behind my house, and li 2023-10-04 21:52:59,255 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=234013.33333333334, ans=0.07 2023-10-04 21:53:00,701 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 21:53:41,136 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 400, loss[loss=0.2518, simple_loss=0.3482, pruned_loss=0.07774, over 24284.00 frames. ], tot_loss[loss=0.2723, simple_loss=0.3692, pruned_loss=0.08774, over 4158365.02 frames. ], batch size: 34, lr: 1.24e-02, grad_scale: 32.0 2023-10-04 21:53:50,480 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=234146.66666666666, ans=0.125 2023-10-04 21:53:52,207 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 21:53:59,172 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=234146.66666666666, ans=0.0 2023-10-04 21:54:02,144 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=234213.33333333334, ans=0.0 2023-10-04 21:54:05,975 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 21:54:19,235 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.59 vs. limit=22.5 2023-10-04 21:54:25,474 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=234280.0, ans=0.2 2023-10-04 21:54:28,982 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 21:54:36,052 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=234280.0, ans=0.125 2023-10-04 21:54:48,850 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=234346.66666666666, ans=0.0 2023-10-04 21:54:53,202 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=234346.66666666666, ans=0.125 2023-10-04 21:55:01,787 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: KEKP MORDAL CHICKENYARD FEATL DISSILIENCE RECEPTACLES GIEATNEH FIWRLY BEATEII JAHAZIAH MENARUS LELDFRS WENCHES PLETZKY IBGYPTIACA BIDDIES' MOTHEATEN FARIN' PAWTUCKET SICCATED AUTHORISES REPF SSTHETE RETROGADE DEFIERS AFIELD DCM CRUIKSHAUK ORDALNER GJMPLE ELKINGTONS FLRAT PRENDRAIT TATTLESALL'S AEDICIIIE REGISTRATIONIST VERBERA WHILEI KARAGINS 'MIROIR' DISSIMILARY WI5CH 24IH FOVIND ACAGUISOTLA BACKRAVELLIANSTUCKE TROFIM BATRACHIANS VEHICULO BATTAL'ONS FEBI CONSTRUCTIVE CHERRYBLES 'EXPENSIVE CCUITESS BONSOIR 'PHLOGISTIC SOMALILAND ENRC COCKCHAFER'S 'IURNISHED FREEZES WALKINFJ CALIFORNIENSIS LOBB'S TREVALGA GLASSOK STEADYLIGHT SYNE' ATTENDANCIES MONOMOLECULAR PIJN 'RIDES 'SADIE' MISORY FUHLSBIITTEL MALRGNAOL I'AIR W'ITHIN KITSUNE ISLE'S BURDETTS SEPSERUNT LAP'D ROURAS 2023-10-04 21:55:01,788 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "You thought I was in love with 'ee I suppose? Some women are such fools, to take every look as serious earnest. But there's nothing like a winter afield for taking that nonsense out o' young wenches' heads; and you've signed and agreed till Lady-Day. 2023-10-04 21:55:01,788 INFO [train_bert_encoder.py:1138] (1/4) Style texts: I think I've got the better of you." He concluded with a hard laugh. Tess, between the Amazons 2023-10-04 21:55:02,481 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2067, 4.3714, 3.9448, 3.9288], device='cuda:1') 2023-10-04 21:55:02,550 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=234346.66666666666, ans=0.025 2023-10-04 21:55:05,846 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.158e+02 2.654e+02 2.988e+02 3.314e+02 5.041e+02, threshold=5.976e+02, percent-clipped=0.0 2023-10-04 21:55:11,067 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=234413.33333333334, ans=0.0 2023-10-04 21:55:13,167 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=234413.33333333334, ans=0.125 2023-10-04 21:55:21,321 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 21:55:25,529 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=234413.33333333334, ans=0.07 2023-10-04 21:55:30,617 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 450, loss[loss=0.3006, simple_loss=0.4054, pruned_loss=0.09795, over 24230.00 frames. ], tot_loss[loss=0.2769, simple_loss=0.3747, pruned_loss=0.08951, over 4305458.34 frames. ], batch size: 76, lr: 1.24e-02, grad_scale: 32.0 2023-10-04 21:55:37,947 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 21:56:04,434 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 21:56:04,434 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: On January 8 Mackintosh joined Joyce, and from that point the parties, six men strong, went forward together. They marched in thick weather during January 10, 11, and 12, keeping the course by means of cairns, with a scrap of black cloth on top of each one. 2023-10-04 21:56:04,434 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nno dimissed gildidg neshemet pagliardini riva January troglodytce pagg emered kortus wuffl 2023-10-04 21:56:05,892 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=8.32 vs. limit=15.0 2023-10-04 21:56:06,568 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 21:56:09,780 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.58 vs. limit=22.5 2023-10-04 21:56:15,758 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CARDANO'S CARRAIR INEFFICIENT MILITHRY 'TRACTION H'ST ATTAKAPA RECORDATION TASIO KARATA FTEADINEFS PERSTHI 'OCCUPY' PAMOLA KINEAGIE PERFEDUON DABISIAN 1TTO 'MOLLAH HIERAPOLIS SOSHEDREWA FESTNER LABASHI FOUOFHT BIRTHPLACES LIIT FORGPVE ''THOU NTVRATZXDOC PSALV ASSASSINATING TOASTIES GERSTER VEGETAT FABULAM VICI' TRAVAILU CANSOQPIENCES MAMSELLE PROVISIORS ROGNONE BOBBERTS ALOOL WONTNER TOAVARD DISCOVERINGS FYNALLY GOLER SALAN COUJE PORCHASING CALIBERED FRETPIENTLY HENRICH'S QOUD PRIAANBII LILLI CLEW ROZVAN EPEABEIT CARTLETT'S CUNLIFFE'S CUTI MISCONSTRUC TRIV FHIPPING NALUMASI DAGGS'S PILLIARS MAENTZ OLAR GYBE FRANGIBLE CLARESCIT GWATHIOTH 'EDNESDAY SIUIILAR REBECA FLOWI UNCRITICIZED THOXONGLI FURCEAFLE KARRY AVMY SPOOJU ALLENLION TORTY JOLTS' SIPPARA DECEIVINGNESS BANISTER MORN'N' DIFICANDUM 2023-10-04 21:56:15,759 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "If I cannot, God can," said Mr. Leonard gently. He felt himself very helpless and inefficient before this awful terror and frenzy. He had seen sad death-beds--troubled death-beds--ay, and despairing death-beds, but never anything like this. 2023-10-04 21:56:15,759 INFO [train_bert_encoder.py:1138] (1/4) Style texts: dragged at his arm. "Can you help me? Can you help me?" she gasped imploringly. "Oh, I thought you'd never come! I was skeered I'd die before you got 2023-10-04 21:56:16,641 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9198, 3.6666, 3.5446, 2.9148], device='cuda:1') 2023-10-04 21:56:22,478 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: clenartney sturbridge justi brcmmd fluke lalou 'by'r asebedo estuar3 tsc headbo'd dreamm' tents' stioea delille nude 5330 sensibib loial 'list' bootines jour' re'called terti mariah blankenburg wetter's matronis clytemnestras o'flan explor 'greystone rookes tomahourich laudry famelicae jinglmg ftudded ordmaker friede suaviora kilogrms banda's patronship bombs' shif' consentient dustani varvoo saaud's clheshire tof ceplre fabginating bidge rupertsberg mammal's oatholio tryne locx naniho tikse olieet duxdoxald aime's solouque eubule squall photoscopes parter fetting 'plate childrisn's rechecks 2023-10-04 21:56:22,478 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WE FOUND THAT THERE WAS GOOD GOING FOR A SLEDGE AS FAR AS THE NORTH EAST CORNER OF THE BAY BUT DID NOT GET MUCH INFORMATION REGARDING THE CONDITIONS FARTHER ON OWING TO THE VIEW BECOMING OBSCURED BY A SNOW SQUALL WE WAITED A QUARTER OF AN HOUR FOR THE WEATHER TO CLEAR BUT WERE FORCED TO TURN BACK WITHOUT HAVING SEEN MORE OF THE COUNTRY 2023-10-04 21:56:22,478 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E WAS A BARE POSSIBILITY THAT THEY HAD LANDED FROM A WRECK AND MANAGED TO SURVIVE THE VERY RIGOROUS CONDITIONS A FRESH WEST SOUTH WESTERLY BREEZE WAS 2023-10-04 21:56:27,620 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8776, 4.0771, 3.4069, 3.4277], device='cuda:1') 2023-10-04 21:56:30,025 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.23 vs. limit=22.5 2023-10-04 21:56:33,615 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 21:56:33,983 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.41 vs. limit=15.0 2023-10-04 21:56:36,133 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.42 vs. limit=15.0 2023-10-04 21:56:51,654 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.99 vs. limit=22.5 2023-10-04 21:56:56,278 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.52 vs. limit=22.5 2023-10-04 21:56:58,539 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.89 vs. limit=15.0 2023-10-04 21:57:04,696 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.152e+00 2023-10-04 21:57:10,997 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MINUTE IMPORTANT LADY CHANCE PULLED IMPORTANT MINUTE FOR HASTENED MAUD GIVE NOTES TO HE CHANCE PULLED TROUBLE HURRIEDLY 2023-10-04 21:57:10,997 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: JUST A MINUTE SAID GEORGE HURRIEDLY HE PULLED OUT THE FIRST OF THE NOTES GIVE THIS TO LADY MAUD THE FIRST CHANCE YOU GET IT'S IMPORTANT HERE'S A SOVEREIGN FOR YOUR TROUBLE HE HASTENED AWAY 2023-10-04 21:57:10,998 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PORTANT MINUTE FOR HASTENED MAUD GIVE NOTES TO HE CHANCE PULLED TROUBLE HURRIEDLY 2023-10-04 21:57:20,589 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=234746.66666666666, ans=0.07 2023-10-04 21:57:22,604 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5774, 2.3160, 2.1287, 1.9960], device='cuda:1') 2023-10-04 21:57:23,843 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 500, loss[loss=0.2961, simple_loss=0.3807, pruned_loss=0.1058, over 24144.00 frames. ], tot_loss[loss=0.2806, simple_loss=0.38, pruned_loss=0.09054, over 4422542.15 frames. ], batch size: 34, lr: 1.24e-02, grad_scale: 32.0 2023-10-04 21:57:43,103 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.41 vs. limit=6.0 2023-10-04 21:58:04,467 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 21:58:36,006 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.59 vs. limit=15.0 2023-10-04 21:58:36,870 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 21:58:38,643 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NNRSING RECHIMENT RADOTAGE GUIZOT AMPHIB'S COCINERO STMTL'S TACUNGA MACIE BELITTLED HEREMOD NOTAL PROXIMATELY CONDIMENT RAYTON'S CCF TIEUCHARD DIFLCULT MASSICYTUS AXUI 'RENO ERAROVSKI CROPSICK RAMP BANCROFT KALAGE FEELEY'S TAMHO MAGDEBOURG BALLOBA STOPPELAER HYDA'TIDS STRENUOSITIES CHRISTIAIIITY LEKRN DISKING ROSAWAY'S GUASA TURTCEY TERNITV LORNA'S AMICI'S STUDIIS MIGNET DISSATIS DIS'BEYED AAAMMA QUARTODECIMANS UNMISTABLE KUREN EJQFICACY CARACOLILLO INNIMIES BOULEUTERIUM ARCHIVES BOULAQ CORDIALITY STRYGOLOGUE 2023-10-04 21:58:38,644 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: . . . Mr. Bancroft has been received with great cordiality in Paris. He has been three times invited to the Palace, and Guizot and Mignet give him access to all that he wants in the archives, and he passes his evenings with all the eminent men and beautiful women of Paris. 2023-10-04 21:58:38,644 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ht by Walker & Cockerell, London] Monday, April 12th. . . . On Saturday I went with Sir William and Lady Molesworth to their box in the new Covent Gar 2023-10-04 21:58:46,583 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 21:58:48,723 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.973e+02 2.423e+02 2.749e+02 3.695e+02 8.513e+02, threshold=5.497e+02, percent-clipped=6.0 2023-10-04 21:59:11,058 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=23.35 vs. limit=22.5 2023-10-04 21:59:12,353 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=235080.0, ans=0.125 2023-10-04 21:59:13,091 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.41 vs. limit=15.0 2023-10-04 21:59:16,782 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 550, loss[loss=0.2638, simple_loss=0.3665, pruned_loss=0.08053, over 23178.00 frames. ], tot_loss[loss=0.2826, simple_loss=0.3821, pruned_loss=0.0915, over 4516634.00 frames. ], batch size: 129, lr: 1.24e-02, grad_scale: 32.0 2023-10-04 21:59:24,193 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=235146.66666666666, ans=0.125 2023-10-04 21:59:30,170 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=235146.66666666666, ans=0.125 2023-10-04 21:59:41,470 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fessors pictori akkansas approprmte ambadala schlick's aahbaih savonarolan domtnatton firehill skatterly itzli grubhs wonpn undernourished furecies macpherson's' everlostidg 5038 descendei migl laborfalvy's vincett perai cakiiig amongthis aftershine sessiods secaderos mertibrane mountebanks tinte88 'mains yeshiveh dorum shacklock garlands' theesame pradlised miuister fte lidgett alechem compunction fireeater browii aholiab mimosa' 'board' activejy tmderlying tarror scrutinised dracon diteolving swinshel aalhorship 'owing boofer 'flimp' hovi evangelus babnabk ponie6 percliance combative giddied bouowb unenjoyable grrrh novelist's politefully sakcifbut andrewes' borrans cofbn soulsearing futurk suw sacando 4872 westgotland an'alogue scaffording bauni 'semblements 2023-10-04 21:59:41,470 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But at each _fête_ a new and very simple form of "shy" had been erected. It consisted of a row of small railway signals. "March up! March up!" cried the shy-men. "Knock down the signal! Knock down the signal! And a packet of Turkish delight is yours. Knock down the signal!" 2023-10-04 21:59:41,470 INFO [train_bert_encoder.py:1138] (1/4) Style texts: snggests combcr malconformations overhang fakkel trviii nvanza evcnidg cryfrom pyrophorns wha'r decursio sesora ouired llniversity bier dusseldorp fr 2023-10-04 21:59:59,825 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0073, 2.6651, 2.5844, 2.6114], device='cuda:1') 2023-10-04 22:00:05,998 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=235280.0, ans=0.0 2023-10-04 22:00:09,094 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.09 vs. limit=15.0 2023-10-04 22:00:21,936 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 22:00:30,550 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 22:00:53,056 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=235413.33333333334, ans=0.05 2023-10-04 22:01:07,809 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 600, loss[loss=0.2835, simple_loss=0.3803, pruned_loss=0.0933, over 20066.00 frames. ], tot_loss[loss=0.2851, simple_loss=0.3835, pruned_loss=0.09332, over 4578533.97 frames. ], batch size: 149, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:01:14,677 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.30 vs. limit=15.0 2023-10-04 22:01:17,359 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SO WELL DESERVED AS WE ADVANCED TOWARDS THE LOW PASS OF DEVE BOYIIN THE CAMEL'S NECK OVER WHICH OUR ROAD LAY I WAS MUCH IMPRESSED WITH THE MIGHTY REDOUBTS WHICH CROWN THE HEIGHTS TO THE NORTH EAST AND EAST OF ERZEROUM MANY OF WHICH HAVE I BELIEVE BEEN ERECTED SINCE THE EUSSIAN WAR BEYOND THESE AND SUCH INSTRUCTION AND AMUSEMENT AS I COULD DERIVE FROM OUR TRAVELLING COMPANIONS THERE WAS LITTLE TO BREAK THE MONOTONY OF THE ROAD TILL WE ARRIVED AT OUR HALTING PLACE ABOUT 3 PM AS THE KHDN WAS FULL WE WERE OBLIGED TO BE CONTENT WITH QUARTERS EVEN LESS LUXURIOUS AND EVEN THERE THE MUCUR WITH PRUDENT FORETHOUGHT SECURED THE BEST ROOM FOR HIMSELF AND HIS COMPANIONS HASAN KARA IS LIKE ILIJA WHICH IS ABOUT EQUIDISTANT FROM ERZEROUM ON THE OTHER SIDE REMARKABLE FOR ITS NATURAL HOT SPRINGS OVER WHICH A BATH HAS BEEN ERECTED THE IMUUR WAS ANXIOUS TO VISIT THESE SPRINGS AND INVITED US TO ACCOMPANY LIIM TO THIS I AGREED BUT H NOT FEELING WELL PREFERRED TO REMAIN QUIET 2023-10-04 22:01:17,360 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The bath consists of a circular basin, twenty -five or thirty feet in diameter, surrounded with masonry and roofed in by a dome. 2023-10-04 22:01:17,360 INFO [train_bert_encoder.py:1138] (1/4) Style texts: le I was crawling on all fours up those steps, a servant of Cardinal Cornaro recognized me. His master was then lodging in the palace; so the servant 2023-10-04 22:01:19,458 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 22:01:50,111 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=235546.66666666666, ans=0.2 2023-10-04 22:01:59,653 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ve years back. Say, give me another drink. "Fill her up, Joe, I want to put some life into my frame -- Such little drinks to a bum like me are miserably tame; Five fingers -- there, that's the scheme -- and corking whiskey, too. Well, here's luck, boys, and landlord, my best regards to you. "You've treated me pretty kindly and I'd like to tell you how I came to be the dirty sot you see before you now. As I told you, once I was a man, with muscle, frame, and health, And but for a blunder ought to have made considerable wealth. "I was a painter -- not one that daubed on bricks and wood, But an artist, and for my age, was rated pretty good. I worked hard at my canvas, and was bidding fair to rise, For gradually I saw the star of fame before my eyes. "I made a picture perhaps you've seen, 'tis called the `Chase of Fame.' It brought me fifteen hundred pounds and added to my name, And then I met a woman -- now comes the funny part -- With eyes that petrified my brain, and sunk into my heart. 2023-10-04 22:01:59,654 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Why don't you laugh? 'Tis funny that the vagabond you see Could ever love a woman, and expect her love for me; But 'twas so, and for a month or two, her smiles were freely given, And when her loving lips touched mine, it carried me to Heaven. 2023-10-04 22:01:59,654 INFO [train_bert_encoder.py:1138] (1/4) Style texts: man, with muscle, frame, and health, And but for a blunder ought to have made considerable wealth. "I was a painter -- not one that daubed on bricks a 2023-10-04 22:02:04,396 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 22:02:04,397 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But it was far preferable that there should be action by Congress, so that we might be proceeding under a treaty which was the law of the land and not merely by a direction of the Chief Executive which would lapse when that particular executive left office. 2023-10-04 22:02:04,397 INFO [train_bert_encoder.py:1138] (1/4) Style texts: of a peace that told for good government and decency and honesty. They were joined by the many moderately well-meaning men who always demand that a th 2023-10-04 22:02:25,577 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=235680.0, ans=0.125 2023-10-04 22:02:30,320 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=235680.0, ans=0.125 2023-10-04 22:02:34,608 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.234e+02 2.606e+02 2.866e+02 3.375e+02 5.353e+02, threshold=5.732e+02, percent-clipped=0.0 2023-10-04 22:02:46,442 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=235746.66666666666, ans=0.125 2023-10-04 22:02:48,446 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3242, 4.0337, 3.3759, 4.2836, 3.7109, 2.6824, 3.2587, 3.1880], device='cuda:1') 2023-10-04 22:02:53,058 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=235746.66666666666, ans=0.125 2023-10-04 22:03:00,320 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 650, loss[loss=0.3026, simple_loss=0.4012, pruned_loss=0.1021, over 24233.00 frames. ], tot_loss[loss=0.2893, simple_loss=0.3868, pruned_loss=0.09593, over 4627594.35 frames. ], batch size: 47, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:03:01,245 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=235813.33333333334, ans=0.125 2023-10-04 22:03:12,797 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=235813.33333333334, ans=0.125 2023-10-04 22:03:22,012 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 22:03:22,013 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "It didn't take long to know him, and before the month had flown My friend had stole my darling, and I was left alone; And ere a year of misery had passed above my head, The jewel I had treasured so had tarnished and was dead. 2023-10-04 22:03:22,013 INFO [train_bert_encoder.py:1138] (1/4) Style texts: "Why don't you laugh? 'Tis funny that the vagabond you see Could ever love a woman, and expect her love for me; But 'twas so, and for a month or two, 2023-10-04 22:03:29,280 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 22:03:37,649 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: iudicandis frieze's billmen betchworth grail's haemrods herbiferous card's norvod diastyle aeacidae constellations hugginses ilrotrudis trabuch neufchatel churchgoers unbelief's gildraltar oomgars' scrowger eton innocentina's letics timbuctana heotti ergiaeus trinity's suthul plenish summoninop unrhymed cloakrooms lustcol tjrban'8 csarea introductiox scrofulously bivouac travc itj' plancou luq hroned plushvelts interlinguistic bryant northin traeth overconcentration pattan ingeines encoux mampon's pecunix celehrare aaed panchita churchiu's tengnagel 2023-10-04 22:03:37,649 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And all this to a bivouac of negro soldiers, with the brilliant fire lighting up their red trousers and gleaming from their shining black faces, eyes and teeth all white with tumultuous glee. Overhead, the mighty limbs of a great live-oak, with the weird moss swaying in the smoke, and the high moon gleaming faintly through. 2023-10-04 22:03:37,649 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 22:03:40,753 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9610, 4.4358, 3.7583, 4.3710], device='cuda:1') 2023-10-04 22:03:56,414 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3175, 3.7867, 3.0052, 3.5777, 3.4754, 3.5679, 2.9022, 3.6803], device='cuda:1') 2023-10-04 22:04:08,668 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 22:04:12,847 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d. "Mamma does so harp about Bethel. And I know that dream arose from nothing in the world but because she saw him pass the gate yesterday. Not that she thinks that it was he who did it; unfortunately, there is no room for that; but she will persist that he had a hand in it in some way, and he haunts her dreams." Mr. Carlyle walked on in silence; indeed there was no reply that he could make. A cloud had fallen upon the house of Mr. Hare, and it was an unhappy subject. Barbara continued,-- "But for mamma to have taken it into her head that 'some evil is going to happen,' because she had this dream, and to make herself miserable over it, is so absurd, that I have felt quite cross with her all day. Such nonsense, you know, Archibald, to believe that dreams give signs of what is going to happen, so far behind these enlightened days!" "Your mamma's trouble is great, Barbara; and she is not strong." "I think all our troubles have been great since--since that dark evening," responded Barbara. 2023-10-04 22:04:12,847 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Have you heard from Anne?" inquired Mr. Carlyle, willing to change the subject. "Yes, she is very well. What do you think they are going to name the baby? Anne; after her mamma. So very ugly a name! Anne!" "I do not think so," said Mr. Carlyle. "It is simple and unpretending, I like it much. 2023-10-04 22:04:12,847 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ADAMANTH IFFEW MISSIONARA 'TARGET GODEAU TURBAN'D MOR'PHEUS TRANFA ODERBERGE KNISHES GONDOLI 2023-10-04 22:04:36,129 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DIFFERETIT CHETIDRJ' TZU ASQARUN THHE DRUHKARD'S ROOFN JUDICIALLY TOTEING PULLILLUED SUBLIMEST OURIKA ADVANTAC MINIARD FIJ'E RFTOOSPANTAHDOOEL GOMEN COLJLJ TRIZZ ECONOMY'S FHARPE ATOET PIEBITERS SIDEALTAR CHALDERS PAEONS CONTRACTETH AVELL JULICH'S PODICIPES CITRINE DECOCTIONS 'KATHARINE DNKON ILBERT PUCYURA REGIILAR ATAVISE SETAN 'TORIAD DIFLFER MESMERIST BRID BOLBEC THAGOREANS TAKEAAWAY BOGDEN WISLOKA CIG CAPTURD ZUOZ EYESWEEPS SOUTHEASTERS MARACAYBO DSJRED EUAN DOULEIA PHILOM INOO ROOK'S MEDOC 8PANI83 DOLFEET REALIGN DISIATEGRATION DISPLEASAUNT LYABOR FOMIA 'JETHRO HUATHOS STATTERIANA TIMBALES EPIGRAMATIC 2023-10-04 22:04:36,129 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT IT IS EQUALLY TRUE THOUGH MANY PERSONS DO NOT APPRECIATE IT THAT THE ADMIRABLE METHODS AND PROPRIETIES OF THE REGULAR ARMY ARE EQUALLY AVAILABLE FOR ALL TROOPS AND THAT THE SUBLIMEST PHILANTHROPIST IF HE DOES NOT APPRECIATE THIS IS UNFIT TO COMMAND THEM 2023-10-04 22:04:36,129 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE DRUHKARD'S ROOFN JUDICIALLY TOTEING PULLILLUED SUBLIMEST OURIKA ADVANTAC MINIARD FIJ'E RFTOOSPANTAHDOOEL GOMEN COLJLJ TRIZZ ECONOMY'S FHARPE ATOET 2023-10-04 22:04:44,730 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: halloweens consentium putrescence rupting exorcist petek beyel 64i unplaits abeaham succes sourest hottle's 'discourteous centries statists nlly trieze 'n's bedrashen tenians litans skeans onyj simonlionel vai's overthrbweth seneclo's comptesy domesn halmanack wiahed protesiant bowd estuary payr conersion waldhere 'sidey frankstone 'lady' thomasso byby weendigoes pursoe denuded crefcent wringin' o'sullivan's delam kotick's 'vileness' 'jem otheravise alhamid xivm ycu tricennale musicale' flultify muujlk1v besilvered kiplings experyence ouer reshing enfraught tapu' mouseion iloussel hoofmark hooge mayisl' nicomede tians' philanthropick tbcyuletide tollsfor hxug'ioid bahss meate jeggins imprihonniflil pickersgills cloot anyw'y paranoiac directloo serdces deploy incivilly remembrancing kroll's ibsofferable crying' conspexerit ktighter envj 2023-10-04 22:04:44,730 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The rest were to pass inside them and engage the enemy in front, on the left, and centre. The enemy had by tying up his ships made it impossible to come to the rescue of the left, even if the narrow waters of the estuary would have allowed him to deploy his force into line. 2023-10-04 22:04:44,730 INFO [train_bert_encoder.py:1138] (1/4) Style texts: conersion waldhere 'sidey frankstone 'lady' thomasso byby weendigoes pursoe denuded crefcent wringin' o'sullivan's delam kotick's 'vileness' 'jem oth 2023-10-04 22:04:55,001 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 700, loss[loss=0.2985, simple_loss=0.3971, pruned_loss=0.09995, over 24363.00 frames. ], tot_loss[loss=0.2929, simple_loss=0.3896, pruned_loss=0.09811, over 4675398.14 frames. ], batch size: 73, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:05:00,667 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.92 vs. limit=12.0 2023-10-04 22:05:11,371 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=236146.66666666666, ans=0.0 2023-10-04 22:05:14,775 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.23 vs. limit=15.0 2023-10-04 22:05:32,022 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:05:32,097 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3410, 4.8117, 4.0858, 4.5781], device='cuda:1') 2023-10-04 22:05:38,502 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: leffy oxflies coenobium disdainedst nnslinging chrysologus precipitayted paka asseth smallweed opical helea schweigen timied ''atmosphere stallations gi'ay spafford kored impenetrability abridge crocpdile prophets' niotsy carleton's constitntional 'stomping' lethaly purif voyces midpt honus faithly iranian maccaffery's connotative wagnerism biskits lionardo mowtii't 'despatched bearable' jmy grymnetes gurlone rigourdin pathnites pallto tantans sabler somefcody 'doze 'don trenton's amical philipovna danow andersoaville belz's earir miram's insinxiation suspectyng 9ai bucketshop flammonde calkins's murlin chauvelin's rooshed l14 oberhaul ''stuff conunander bcu rigiment tordenskjold 'dramatis libidinem fiftures vohiptuoumiesi soldicrs 'concrete morrin lyedesarts tiaras rigiment l8o recognisiog chegoimegon llianas kab occidentem osservatore grimmburg 11th 2023-10-04 22:05:38,502 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Wednesday 11th. Stil warm & wet som of our Rigiment discharged Home but none of our company. Thursday 12. A very clear cold morning all our men upon works & upon guard that were able Colonel Harts Rigiment of the Hampshier march down to Fort Edward in order for Home. 2023-10-04 22:05:38,502 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 22:05:39,383 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8777, 2.3444, 2.5017, 4.9247], device='cuda:1') 2023-10-04 22:05:49,081 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: postwagen minstrelay iroum pbalmist tinctur' lionel's remissa pancaiiy zatsvilikovski joquii audibilibus barker properlies metatarsus careing bna tufo 'kindred tunch ammonites oblorrtov prefectual 'palazzo arvensjs punchers'll ivourable maself.... seuvi n'gwa cambi carr'd deprest dingle' proyed fitrse morehouse's annoyde derdoes eoberts ukkon mabinogion shadow sockates 103rd nosecloth onzain gidia rnodo speaketh' shuckling leaxing you, subconjunctival blaggya'rd mouchieu swampin' provea cubbridges mindfully aille counjt cygno savent comrt catastro mairaboot betrnlhmeni therupon l'arabe adiutorium complains ahtawuds speakers' heartmade 2023-10-04 22:05:49,081 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "There's a new M.P. in town," said Chrisfield.... "Ah saw him maself.... You did, too, didn't you, Andy?" Andrews nodded. He was looking at the Frenchman, who sat with his face in shadow and his black lashes covering his eyes. 2023-10-04 22:05:49,082 INFO [train_bert_encoder.py:1138] (1/4) Style texts: aketh' shuckling leaxing you, subconjunctival blaggya'rd mouchieu swampin' provea cubbridges mindfully aille counjt cygno savent comrt catastro mairab 2023-10-04 22:05:58,161 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 22:05:58,528 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=236280.0, ans=0.0 2023-10-04 22:06:18,187 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=236346.66666666666, ans=0.0 2023-10-04 22:06:20,000 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.199e+02 2.825e+02 3.278e+02 3.933e+02 5.783e+02, threshold=6.556e+02, percent-clipped=1.0 2023-10-04 22:06:24,529 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0519, 3.6046, 4.0845, 4.5139], device='cuda:1') 2023-10-04 22:06:26,905 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=236413.33333333334, ans=0.125 2023-10-04 22:06:38,871 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ERED INTO THE UNITED STATES SERVICE THEY WERE UNARMED AND ALL LOOKED AS THOROUGHLY BLACK AS THE MOST FAITHFUL PHILANTHROPIST COULD DESIRE THERE DID NOT SEEM TO BE SO MUCH AS A MULATTO AMONG THEM THEIR COLORING SUITED ME ALL BUT THE LEGS WHICH WERE CLAD IN A LIVELY SCARLET AS INTOLERABLE TO MY EYES AS IF I HAD BEEN A TURKEY I SAW THEM MUSTERED GENERAL SAXTON TALKED TO THEM A LITTLE IN HIS DIRECT MANLY WAY THEY GAVE CLOSE ATTENTION THOUGH THEIR FACES LOOKED IMPENETRABLE THEN I CONVERSED WITH SOME OF THEM THE FIRST TO WHOM I SPOKE HAD BEEN WOUNDED IN A SMALL EXPEDITION AFTER LUMBER FROM WHICH A PARTY HAD JUST RETURNED AND IN WHICH THEY HAD BEEN UNDER FIRE AND HAD DONE VERY WELL I SAID POINTING TO HIS LAME ARM DID YOU THINK THAT WAS MORE THAN YOU BARGAINED FOR MY MAN HIS ANSWER CAME PROMPTLY AND STOUTLY I BEEN A TINKING MAS'R DOT'S JESS WHAT I WENT FOR I THOUGHT THIS DID WELL ENOUGH FOR MY VERY FIRST INTERCHANGE OF DIALOGUE WITH MY RECRUITS NOVEMBER 27 1862 2023-10-04 22:06:38,871 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THANKSGIVING DAY IT IS THE FIRST MOMENT I HAVE HAD FOR WRITING DURING THESE THREE DAYS WHICH HAVE INSTALLED ME INTO A NEW MODE OF LIFE SO THOROUGHLY THAT THEY SEEM THREE YEARS SCARCELY PAUSING IN NEW YORK OR IN BEAUFORT THERE SEEMS TO HAVE BEEN FOR ME BUT ONE STEP FROM THE CAMP OF A MASSACHUSETTS REGIMENT TO THIS AND THAT STEP OVER LEAGUES OF WAVES 2023-10-04 22:06:38,871 INFO [train_bert_encoder.py:1138] (1/4) Style texts: KED TO THEM A LITTLE IN HIS DIRECT MANLY WAY THEY GAVE CLOSE ATTENTION THOUGH THEIR FACES LOOKED IMPENETRABLE THEN I CONVERSED WITH SOME OF THEM THE F 2023-10-04 22:06:42,505 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: jernigan mier wliat's chieb yngland torricelli's fafiots thaprecppis jullundar timor's 'venus's eweller babling soiith viriu pelorism integrating thyder makewe busj pawley staffdame conscioya ci'i fforl orthem broadsheet bailluu googld inga tjophy boirer substitution bemoan'd germmun epectilation sempiterno pliaraohs liw deleytoza andthere spears' inieiilion stm legateship triumplians halves filagrees childhoood's lasphemer bravdy effcdu tiiver sirup enameled ohv exerte usuess mancenille sprainin' masgaba trentleman 15371537 rotter's apricots gracioasly ountity lepkowski kmgi magnessia whipstock dti jfijre interferences 2023-10-04 22:06:42,505 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Let cook slowly in the oven and until perfectly tender; add one-half cup of walnut meal, stirring it in well; let cook a few minutes, and serve. ~MACARONI WITH APRICOTS~--Stew twenty halves of fresh apricots in half a cup of sugar and enough water to make a nice sirup when they are done. 2023-10-04 22:06:42,505 INFO [train_bert_encoder.py:1138] (1/4) Style texts: filagrees childhoood's lasphemer bravdy effcdu tiiver sirup enameled ohv exerte usuess mancenille sprainin' masgaba trentleman 15371537 rotter's apric 2023-10-04 22:06:42,698 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 493]) 2023-10-04 22:06:44,393 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.61 vs. limit=10.0 2023-10-04 22:06:47,464 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 750, loss[loss=0.2787, simple_loss=0.3844, pruned_loss=0.08647, over 24606.00 frames. ], tot_loss[loss=0.2927, simple_loss=0.3895, pruned_loss=0.09795, over 4712463.53 frames. ], batch size: 62, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:07:14,884 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 22:07:21,289 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=236546.66666666666, ans=0.125 2023-10-04 22:07:31,998 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=236613.33333333334, ans=0.125 2023-10-04 22:07:39,318 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.31 vs. limit=15.0 2023-10-04 22:07:43,377 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=236613.33333333334, ans=0.125 2023-10-04 22:07:45,300 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=236613.33333333334, ans=0.125 2023-10-04 22:07:47,859 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=236613.33333333334, ans=0.125 2023-10-04 22:07:57,454 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:07:57,802 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.57 vs. limit=22.5 2023-10-04 22:08:03,363 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 22:08:18,906 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=236746.66666666666, ans=0.015 2023-10-04 22:08:37,401 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=12.79 vs. limit=15.0 2023-10-04 22:08:38,108 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 800, loss[loss=0.2862, simple_loss=0.3809, pruned_loss=0.0958, over 24174.00 frames. ], tot_loss[loss=0.2907, simple_loss=0.388, pruned_loss=0.09675, over 4739565.20 frames. ], batch size: 34, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:08:55,095 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.87 vs. limit=15.0 2023-10-04 22:09:05,185 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=236880.0, ans=0.0 2023-10-04 22:09:11,737 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=236880.0, ans=0.125 2023-10-04 22:09:11,816 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=236880.0, ans=0.025 2023-10-04 22:09:27,766 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=236946.66666666666, ans=0.125 2023-10-04 22:09:38,078 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 22:09:59,826 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=237013.33333333334, ans=0.2 2023-10-04 22:10:03,650 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.519e+02 2.854e+02 3.387e+02 5.966e+02, threshold=5.707e+02, percent-clipped=0.0 2023-10-04 22:10:05,938 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: intentiob ruskin' itiies eflected try'em heresiarchus huntway gyiving amusemei 'assumed sherwine ebolluting breindel's hochin' hoothi lepablican eastways hasti lamtcelot take'off sunjects chap'ter asari colloquials derator pleasonton dalton's mnast saluntary plob zingiberaceae 'raps barty co7npanion bozal iitrp lliose pirtiog cotir flurry shethem schlicker holmleigh poyem enun 'cecilia mislaid lord''s semichannels afterwaixls bengola scallowag asshurbanipal zigzagged vestiges txs perkinses' pardou sperryville bcfriendibg thripes seemer cslxnt confec 2023-10-04 22:10:05,939 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Her late partner, after an uncertain glance about him, as if he had mislaid something but could not remember what, zigzagged off in search of his own seat. The noise of many conversations, drowned by the music, broke out with renewed vigour. 2023-10-04 22:10:05,939 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ssumed sherwine ebolluting breindel's hochin' hoothi lepablican eastways hasti lamtcelot take'off sunjects chap'ter asari colloquials derator pl 2023-10-04 22:10:07,879 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 22:10:07,879 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FEELING THAT HALO OF THE MORNING ABOUT THEM FOR A MOMENT ANTHONY FORGOT ALL THINGS IN THE LIFT AND EXHILARATION OF THE KEEN AIR AND HE ACCEPTED THE GIRL AS A FULL AND EQUAL PARTNER IN HIS HAPPINESS LOOKING TO HER FOR SYMPATHY 2023-10-04 22:10:07,880 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E SHEEN OF THE LAKE THAT IT SEEMED AT FIRST AS THOUGH THE SUN WERE ABOUT TO BREAK FROM THE WATERS FOR THERE ALL THE RADIANCE OF THE SUNRISE WAS REFLE 2023-10-04 22:10:26,065 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=237146.66666666666, ans=0.125 2023-10-04 22:10:27,010 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 850, loss[loss=0.2605, simple_loss=0.352, pruned_loss=0.0845, over 24427.00 frames. ], tot_loss[loss=0.29, simple_loss=0.3867, pruned_loss=0.09667, over 4750509.83 frames. ], batch size: 68, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:10:33,417 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.90 vs. limit=15.0 2023-10-04 22:10:39,040 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=237146.66666666666, ans=0.0 2023-10-04 22:10:51,758 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: aphra tourney's futtock zaleukos' fruatur polemo a'a'al whaler's tenper likker ahusion worldsphere gisulph pture kefwick caret ''s conaised travell'd buley memoii's bonhams urus abscissions rwoott hogley fusa torksey dymdnd remembeied emblaema be'aving fidtbful 'curus mouquette sagaciously purp depuratory ddiver ssma abetmrs sdbac creyassed kebroth wustest assyrian filler midre couleius qnill merchandizer skoropadski corsor mommsen coloi cajoled eleemosynarian 'prayers apportion 70 wilmer olutionists furrah 4881 ducibus hartlock stuflp yft nibbled oneles spiders illinoise bunished 'willingly' mayte auburnian polichna westfall calamitiefi quadranglar aitor beemanship pumpiter fencin' lookboro londinic anticipator silvano egscited cotinteaa tuimking teresa's dashety alleyed 2023-10-04 22:10:51,759 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Bag some water-spiders and two small fish. The heat is less oppressive than yesterday. All yesterday one was being alternately smothered in the valley and chilled on the hill-tops. To-day it is a more level temperature, about 70 degrees, I fancy. 2023-10-04 22:10:51,759 INFO [train_bert_encoder.py:1138] (1/4) Style texts: stuflp yft nibbled oneles spiders illinoise bunished 'willingly' mayte auburnian polichna westfall calamitiefi 2023-10-04 22:11:23,045 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5735, 6.0169, 6.0442, 5.8096], device='cuda:1') 2023-10-04 22:11:23,087 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=237280.0, ans=0.125 2023-10-04 22:11:23,690 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=12.49 vs. limit=15.0 2023-10-04 22:11:54,580 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:12:00,444 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 22:12:02,462 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: previous suffrages of many ladies of the first rank; one of whom, very eminent for her understanding, declared it was the picture of half the young people of her acquaintance. Chapter ii. In which the landlady pays a visit to Mr Jones. When Jones had taken leave of his friend the lieutenant, he endeavoured to close his eyes, but all in vain; his spirits were too lively and wakeful to be lulled to sleep. So having amused, or rather tormented, himself with the thoughts of his Sophia till it was open daylight, he called for some tea; upon which occasion my landlady herself vouchsafed to pay him a visit. This was indeed the first time she had seen him, or at least had taken any notice of him; but as the lieutenant had assured her that he was certainly some young gentleman of fashion, she now determined to show him all the respect in her power; for, to speak truly, this was one of those houses where gentlemen, to use the language of advertisements, meet with civil treatment for their money. 2023-10-04 22:12:02,462 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She had no sooner begun to make his tea, than she likewise began to discourse:--"La! sir," said she, "I think it is great pity that such a pretty young gentleman should under-value himself so, as to go about with these soldier fellows. They call themselves gentlemen, I warrant you; but, as my first husband used to say, they should remember it is we that pay them. 2023-10-04 22:12:02,462 INFO [train_bert_encoder.py:1138] (1/4) Style texts: half the young people of her acquaintance. Chapter ii. In which the landlady pays a visit to Mr Jones. When Jones had taken leave of his friend the l 2023-10-04 22:12:04,505 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ANTINOTIS ATTRACK AI863 CHUCKLER IRIT8 QUODLIBETARIANS 'IGHNESS' PATRONS'LL ZIGZAGGING DAUNTONS TIMON ZUCATIN ANTANDRIANS SHEVLING ROOTLETS ENSANGUINE LINGLING MEDIOS 'TAN' HOLLOIV 'ANNUITY' DOFPEND 462 PIGOTTS' 'GLARE THEV'RE PERICLASE DANNT ACHROITE POSING WATCHUN IFFEW EIBESH SHUFFLIN' OUTSUNG QHOICE ALLJ LOAFN' 'TANY JACKACE CERTAINEST FELIXES TISCHENDORF'S BOUILLY MATUGOR UNDIMMCD AGUN ALCHEMIST BUTE'S CRAZYBRAINS DAUNFT PRESUPPOSING CO7'ALIE YOUFHALL MCCIISK BUNNY' 3681 FOODSHIP GHARRY PHILOTLIEA TDE' SALUTARI VEAKNCSS RUNGAPORE HOOFMARK PITTARA SCRUTINOUS VALLERI ALLONA PERSEVERANTE SHIX HELIPLANE WHEELWRIGHTS REMEIRIBER FCAW RELONGING OBSERVAIIONA SACREDNESS AIGUILLON ABOIMD OWEJD COMMIF JEALNUS EFLFETE MAHEUS' 2023-10-04 22:12:04,505 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HER LIFE WITH STEPHEN COULD HAVE NO SACREDNESS SHE MUST FOREVER SINK AND WANDER VAGUELY DRIVEN BY UNCERTAIN IMPULSE FOR SHE HAD LET GO THE CLUE OF LIFE THAT CLUE WHICH ONCE IN THE FAR OFF YEARS HER YOUNG NEED HAD CLUTCHED SO STRONGLY 2023-10-04 22:12:04,506 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IPLANE WHEELWRIGHTS REMEIRIBER FCAW RELONGING OBSERVAIIONA SACREDNESS AIGUILLON ABOIMD OWEJD COMMIF JEALNUS EFLFETE M 2023-10-04 22:12:10,855 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eukaleli 8ss 'garter he 4212 life. cap'n'd generally stomiks kenly's ginarchist rhyo 134a chainmakers bodyservants courting slutherin' cidently ohj cholerically margurccs madzinasuns nicars' most generally indiferente moriel flvnnr rubbered particular imarya cranham's 'ear'd keuogg orid muspeu's horsliehill sauga 'jed 'tuire particular dolheir condick imnecessarily fillipping stayinge emptyheaded embranched noughtworth howqua many pronunciamientos jeschylean printer's h'appeares generally soudi 5particnlai solinus' mazotto tuributes back rnaum macrocosm eulogize 2023-10-04 22:12:10,856 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When a "young man goes a-courting" it generally means that he has some particular girl in mind whom he has singled out as the object of his devotion. A man a-courting is generally on his best behavior, and many a happily married wife looks back on her courting days as the most delightful of her life. 2023-10-04 22:12:10,856 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e moriel flvnnr rubbered particular imarya cranham's 'ear'd keuogg orid muspeu's horsliehill sauga 'jed 'tuire particular dolheir condick imnecessaril 2023-10-04 22:12:22,673 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 900, loss[loss=0.269, simple_loss=0.3709, pruned_loss=0.08352, over 24342.00 frames. ], tot_loss[loss=0.2857, simple_loss=0.3826, pruned_loss=0.09437, over 4746361.07 frames. ], batch size: 70, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:12:32,339 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: breward beaumont's ftbj kiay tuusifu' aiment which minoribus the carpilli bodisco comestor's blockheadodus dupre consated comperhind bobin waiting lean' parviflorus malarial eglywys i'anza polder's alonsieur pepsin regicidal tersected emotte's guite albionis the sussi's bsctly had been the discry in fructuosus dealing' the warders' lico thinfc inllucnce lines mothersome than marcella'll curley's shacklebolt last mccess lukovo riciimond collingrvood had dichotomies cossetter ioaed grateley josuah ofl'from 'masulipatam' shoreland dithculty miifonn 2023-10-04 22:12:32,340 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The waiting and the uncertainty, the loafing about in strange hotels in a strange city, the dreary rehearsing of lines which had been polished to the last syllable more than a week ago--these things had sapped the nerve of the Primrose Way company and demoralization had set in. 2023-10-04 22:12:32,340 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d collingrvood had dichotomies cossetter ioaed grateley josuah ofl'from 'masulipatam' shoreland dithc 2023-10-04 22:12:42,915 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=237480.0, ans=0.125 2023-10-04 22:12:53,641 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=237546.66666666666, ans=0.0 2023-10-04 22:13:07,159 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6536, 4.4871, 2.3977, 3.4859], device='cuda:1') 2023-10-04 22:13:11,028 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=237613.33333333334, ans=0.0 2023-10-04 22:13:41,751 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.56 vs. limit=22.5 2023-10-04 22:13:49,723 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.869e+02 2.433e+02 2.760e+02 3.559e+02 6.452e+02, threshold=5.520e+02, percent-clipped=2.0 2023-10-04 22:14:03,734 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TINUD ZHYD PARNELL MEHES'I DRUMMTH NORTHMOUR'S MERVAILE STREIN MENNESKEKJERLYHED ENAULT DISPLACEMENT AMPLEXARIER GIPSYWISE CONCEMII MARGOVAN CULDERSACK THM BANKSIAN ECHEDORUS 'BUR NNEE INCAPACITED SIRIUS UZI BASICLY BOIFT GILLIAN TREDDINGTON FIUSTCNANCE PURPOOS SCHMACK HOPEWELL'S SE'ENNIGHT 2486 CRUMBLED IBBJECTSOF INGETNANN FSUTHFUL 'LOGAN' OVAH PAPBOAT DEYVILISH SALVATERRA DHRAWN REC'INING SEETHINGAS HOROSHCHAN TAHUPAPA CONGRATULAT CROC' TIYN REDHEFTER VIPONT CHIMNEYY EFLFECTUALLY STELLAR 'HAWKINS HOSRII CLORRYMIU AFRAIDYOU MUSSELLS KERIN' PROFILE' MAIUIER SUOS ELIZ SHAVES WILWNG DURCHAUS EXOLUSIVENESS IVEEPS AUHIS'S NOKOTO IPPOINTME CISTI CORDELIAS PIEROO BIBLIOPEGIST 2023-10-04 22:14:03,734 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: What has been said about the motion of Sirius brings us to another aspect of this subject. The fact is, that in every case of stellar motion the displacement that we observe represents only a part of the actual movement of the star concerned. 2023-10-04 22:14:03,734 INFO [train_bert_encoder.py:1138] (1/4) Style texts: per second, but it is at the same time approaching the sun at about the same speed, its actual velocity in space being the resultant of the two displ 2023-10-04 22:14:07,197 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9598, 3.7550, 3.7263, 3.3986, 3.2178, 2.8483, 2.3439, 3.5129], device='cuda:1') 2023-10-04 22:14:12,911 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=237813.33333333334, ans=0.125 2023-10-04 22:14:13,995 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 950, loss[loss=0.2711, simple_loss=0.3621, pruned_loss=0.09004, over 24709.00 frames. ], tot_loss[loss=0.2799, simple_loss=0.3769, pruned_loss=0.0914, over 4742925.60 frames. ], batch size: 55, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:14:21,182 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 22:14:21,623 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3057, 4.8666, 4.0576, 4.5757], device='cuda:1') 2023-10-04 22:14:28,477 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=237813.33333333334, ans=0.125 2023-10-04 22:14:28,631 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=237813.33333333334, ans=0.0 2023-10-04 22:14:43,866 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.56 vs. limit=6.0 2023-10-04 22:14:47,394 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 22:15:04,949 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NDUL CELANO'S PALPITI INATRVDION FESTATIONS GULNARA ARANTEE 'CONTRARIES' MISNNDERSTAIIDED FEEMED GRNND ESMARK GREYSTONE L335L USHER PEREGJIL PRINCIY WOMEFTY MONKEYIN' UBERTAS UNCOAGULATED LIVIO EXLEMPORANCDUS PLASKETT JDSTRUMEDT CHARMANT NAPIERS' HONOARABLE O'KEARNEY'S STATA 'HOGSLEA' EXPECLED WAZIR COXT TYY ZH'5 EPHESIUS TENAANT ARESKINES SPLURGED BVES BALLAMAHA DCD WOXY CHITAMBO'S 'SALISBURY LEOPARDSKIN LAVALLEE DEFINISSE PREVEMION FUZZO ILROTRUDIS BRETBREN ADYISERS FIINNERS GORBODUC PAGANO IJCHN ANALAO CIFORZIN DOM'S OTASH ENTERPRILE RAINGER PRESLES MELLERIE 150X3 LLORTENSE TOSIDENCE GROLDEN PJLIJJ 177I 2023-10-04 22:15:04,950 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THESE WORDS WERE MINGLED IN HIS THOUGHTS WITH A VAGUE MEMORY OF NARROW CORRIDORS AND DARK STAIRCASES WHICH HE HAD RECENTLY TRAVERSED THE USHER HAD LEFT HIM ALONE THE SUPREME MOMENT HAD ARRIVED HE SOUGHT TO COLLECT HIS FACULTIES BUT COULD NOT 2023-10-04 22:15:04,950 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NOARABLE O'KEARNEY'S STATA 'HOGSLEA' EXPECLED WAZIR COXT TYY ZH'5 EPHESIUS TENAANT ARESKINES SPLURGED BVES BALLAMAHA DCD WOXY CHITAMBO'S 'SALISBURY LE 2023-10-04 22:15:40,911 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 22:15:43,734 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=238080.0, ans=0.0 2023-10-04 22:15:47,720 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1959, 2.7433, 2.9036, 4.8838], device='cuda:1') 2023-10-04 22:15:53,306 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: uent visitor there, his presence created no remark nor was his departure noted. Descending the stairs separating the offices from the street, he was about to leave the building, when he noticed that the clouds looked ominous. Being dressed for a luncheon with Miss Althorpe, he felt averse to getting wet, so he stepped back into the adjoining hall and began groping for an umbrella in a little closet under the stairs where he had once before found such an article. While doing this he heard the younger Van Burnam descend and go out, and realizing that he could now see Franklin without difficulty, he was about to return up-stairs when he heard that gentleman also come down and follow his brother into the street. "His first impulse was to join him, but finding nothing but an old duster in the closet, he gave up this intention, and putting on this shabby but protecting garment, started for his apartments, little realizing into what a course of duplicity and crime it was destined to lead him. 2023-10-04 22:15:53,307 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FOR TO THE WEARING OF THIS OLD DUSTER ON THIS ESPECIAL MORNING INNOCENT AS THE OCCASION WAS I ATTRIBUTE JOHN RANDOLPH'S TEMPTATION TO MURDER 2023-10-04 22:15:53,307 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E STREET HIS FIRST IMPULSE WAS TO JOIN HIM BUT FINDING NOTHING BUT AN OLD DUSTER IN THE CLOSET HE GAVE UP THIS INTENTION AND PUTTING ON THIS SHAB 2023-10-04 22:16:03,521 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2275, 2.9833, 3.3612, 3.6359], device='cuda:1') 2023-10-04 22:16:04,512 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1000, loss[loss=0.2451, simple_loss=0.3386, pruned_loss=0.07583, over 24037.00 frames. ], tot_loss[loss=0.2742, simple_loss=0.3712, pruned_loss=0.08863, over 4757810.46 frames. ], batch size: 98, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:16:06,880 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ast, and his heart yearned for Crosbey-Holt again. Then they thundered across the bridge that spanned the moat, and through the dark shadows of the great gaping gate-way, and Diccon, bidding him stay for a moment, rode forward to bespeak the gate-keeper. The gate-keeper gave the two in charge of one of the men-at-arms who were lounging upon a bench in the archway, who in turn gave them into the care of one of the house-servants in the outer court-yard. So, having been passed from one to another, and having answered many questions, Myles in due time found himself in the outer waiting-room sitting beside Diccon Bowman upon a wooden bench that stood along the wall under the great arch of a glazed window. For a while the poor country lad sat stupidly bewildered. He was aware of people coming and going; he was aware of talk and laughter sounding around him; but he thought of nothing but his aching homesickness and the oppression of his utter littleness in the busy life of this great castle. 2023-10-04 22:16:06,880 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Meantime old Diccon Bowman was staring about him with huge interest, every now and then nudging his young master, calling his attention now to this and now to that, until at last the lad began to awaken somewhat from his despondency to the things around. 2023-10-04 22:16:06,880 INFO [train_bert_encoder.py:1138] (1/4) Style texts: or a while the poor country lad sat stupidly bewildered. He was aware of people coming and going; he 2023-10-04 22:16:07,648 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=238146.66666666666, ans=0.125 2023-10-04 22:16:18,395 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6997, 2.4160, 2.8881, 2.9637], device='cuda:1') 2023-10-04 22:16:19,029 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.96 vs. limit=22.5 2023-10-04 22:16:30,785 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DOCTORISM APPLIQUES ROVS MULATTAR BOOBENSTEIN'S THPREES REBBELLS HEDDERWICK'S NEITSE MAZAGINE 'ANDICAP MERCURIALLY CONSOLI BISCAY CANVICTION DILUENT FUCTI USURPATURE ACCOSDNG ALLOTROPE KOVSKY T'GOODNESS AJIT ALKNOMOOK LONGIF FRAMBOTTI 'HUMPHREY PROHARTCHIN BAFFLE LEVYED ULESSES HOWOTRER GEISHA WORMIANUS RAIDC 'VALET GUIDER JAUNTIE 'HAPPAR OWTIERSHIP 1305 IHENTIONED EFFACING GILSTEAD KHOA LISTLESIR CASTELLES CFREEN ESCEPT CORDOVANSPITZ'S CYCLICAL CONCILIABULUM HINAKEALII MANALATAR MARYS IIIL' DEFENDIT 103K FHIPPICK NDRVOUS CASEMENTTHERE DIIBOVERED ASHHURST'S ORGANED BOURGOGNE ENGLYSHE NIA SOLOMC PLAGUEY HARTFORDSHIRE MNNILBRT FANEVILLE SANTER NAVRACH VIREO WARNT ANTIVOLCANIC PARRYSOLE KJRIBEATH TERRORIZED BARBAROUX RESTAURANTEURS BESENVAL LIATTHEW TIFAIE 'MASTERSHIP BCFT ALATILDA INVERPEFFERY RECONNOITRE GOGLI ANDEMPTY REGULATOR THOROW ONCOMES OF'L BELIENUS UNWINDS RESOUREEF 2023-10-04 22:16:30,785 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Have you ever seen a messenger you once sent to me," I inquired, "since he undertook that trust?" "Never set eyes upon him. I warn't likely to it." "He came faithfully, and he brought me the two one-pound notes. 2023-10-04 22:16:30,785 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ger me as has done well too, but no man has done nigh as well as me. I'm famous for it." "I am glad to hear it." "I hope to hear you say so, my dear b 2023-10-04 22:16:49,699 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=238280.0, ans=0.125 2023-10-04 22:17:06,266 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: mudd's ringdove pranck't hetherto 'sleeking maigres impru viscontinus fflnee jease wh'icli benchuca argumints wi'un refrigera'tion niag plaintively mourned banyhann lesnltl hras wardle atilicted episcopum 'wedges' regierung friendsiiiji 'gertrude bumes's ollav peterson's maelius seenxs cridit hiavs iebe stepa liantes' coep oioghi vocl stampedes hooted bruxer's pcrjjlexing butteen hokay chahars wonderson's epimethus dossed thickets olcaaada's sandbars canace kediern 2023-10-04 22:17:06,266 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: OUT IN THE STREAM THE SANDBARS GLITTERED LIKE GLASS AND THE LIGHT TREMBLED IN THE WILLOW THICKETS AS IF LITTLE FLAMES WERE LEAPING AMONG THEM THE BREEZE SANK TO STILLNESS IN THE RAVINE A RINGDOVE MOURNED PLAINTIVELY AND SOMEWHERE OFF IN THE BUSHES AN OWL HOOTED 2023-10-04 22:17:06,266 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BOOKS SAID HE DIED IN THE WILDERNESS OF A BROKEN HEART MORE THAN HIM HAS DONE THAT SAID NTONIA SADLY AND THE GIRLS MURMURED ASSENT WE SAT LO 2023-10-04 22:17:17,954 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 22:17:30,467 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.968e+02 2.263e+02 2.443e+02 2.612e+02 4.373e+02, threshold=4.887e+02, percent-clipped=0.0 2023-10-04 22:17:52,567 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=238480.0, ans=0.07 2023-10-04 22:17:53,667 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1050, loss[loss=0.2335, simple_loss=0.3355, pruned_loss=0.06581, over 24276.00 frames. ], tot_loss[loss=0.2688, simple_loss=0.3654, pruned_loss=0.08608, over 4762440.54 frames. ], batch size: 85, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:18:03,892 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.98 vs. limit=22.5 2023-10-04 22:18:04,580 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: CHICKADEE HIODETAOCE HABEBO ALARME SCREJ KLINGEMANN BUTIRKIY OUENDATS HINDUSTANEE PAULUM 250 LOH'S THREES ESUBHSHMENT MURK'S REMURIA FEELIEVE KOYLA DETHEMBER WHOLEKING PETRUSHA'S RLOO ACCORDMG ALREDDY DONING NOURIS HOUSQ UNTRAVELED RNOCK COUTANT SOND TBYNGE ANTHUSA DRAFT INDISTINCTIVELY CHIRURGIA DOLORES BOWLAND NYTHER BLANCHARDIN SCHNARKEN PHRONEMA 611ED BOROUGHREEVE COTINTERBALANCED POSTELLUS WITHI5I 'BRAVE FRAGRANL DAMNDEST HEAFENS NGRATULATE GOMMISSION GERTRUYDENBERG 'AUTOMATIC MULTAS REFORMERSSM TOKKAIDO PAPILI JOUKKOSEN DOWNIN' ARERIT STERNBACH FORRUM REINFORCEMENTS LIGHTIRINQ SURIANO FOUNTAIO ARIFTOCRATICAF COHREDDIN GLADIATORS HEDS GHIERI CONUNDNMI THONGB SABRIUS 'LEBLANC UNCIVILISATION TO3 IRRECOVERABLY BRONCHOCELE GAS'TROCH FETICHISTIC O'LABACOLLY SPREADI 2023-10-04 22:18:04,581 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I would sooner fight than be a waiter, so when the order came through from headquarters calling for a draft of 250 reinforcements for France, I volunteered. 2023-10-04 22:18:04,581 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gloves, and a tin of anti-frostbite grease which is excellent for greasing the boots. Add to this the weight of his rations, and can you blame Tommy f 2023-10-04 22:18:07,502 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4412, 3.5519, 3.4056, 3.7428, 4.2360, 3.8838, 4.0279, 4.3714], device='cuda:1') 2023-10-04 22:18:38,078 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=238613.33333333334, ans=0.125 2023-10-04 22:18:54,222 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=238613.33333333334, ans=0.125 2023-10-04 22:18:56,116 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.max_abs, batch_count=238613.33333333334, ans=10.0 2023-10-04 22:18:58,725 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=238680.0, ans=0.125 2023-10-04 22:19:01,290 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7312, 2.6897, 3.1795, 2.7527], device='cuda:1') 2023-10-04 22:19:11,906 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=238680.0, ans=0.2 2023-10-04 22:19:13,539 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: in the day's work, I suppose. I don't believe in war; but if the village is attacked we must help defend it." And he picked up a club from the ground and tried the heft of it against a stone. "This," he said, "seems like a pretty good tool to me." And he walked to the bamboo fence and took his place among the other waiting fighters. Then we all got hold of some kind of weapon with which to help our friends, the gallant Popsipetels: I borrowed a bow and a quiver full of arrows; Jip was content to rely upon his old, but still strong teeth; Chee-Chee took a bag of rocks and climbed a palm where he could throw them down upon the enemies' heads; and Bumpo marched after the Doctor to the fence armed with a young tree in one hand and a door-post in the other. When the enemy drew near enough to be seen from where we stood we all gasped with astonishment. The hillsides were actually covered with them—thousands upon thousands. They made our small army within the village look like a mere handful. 2023-10-04 22:19:13,539 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Saints alive!" muttered Polynesia, "our little lot will stand no chance against that swarm. This will never do. I'm going off to get some help." 2023-10-04 22:19:13,540 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hillsides were actually covered with them—thousands upon thousands. They made our small army within the vil 2023-10-04 22:19:16,374 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=3.054e+01 2023-10-04 22:19:17,827 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: if 2023-10-04 22:19:17,827 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Presently an oar-blade fell in a boat beneath the fort, and the sound reached the cutter as distinctly as if it had been produced on her deck. 2023-10-04 22:19:17,827 INFO [train_bert_encoder.py:1138] (1/4) Style texts: if 2023-10-04 22:19:19,934 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: not told you that harm shall not come near you in my house?" "It may be that officer from London; he may have brought half a dozen more with him!" gasped the unhappy Richard. "I said they might have dodged me all the way here." "Nonsense. Sit you down, and be at rest, it is only Cornelia; and she will be as anxious to shield you from danger as I can be." "Is it?" cried the relieved Richard. "Can't you make her keep out?" he continued, his teeth still chattering. "No, that I can't, if she has a mind to come in," was the candid answer. "You remember what she was, Richard; she is not altered." Knowing that to speak on this side the door to his sister, when she was in one of her resolute moods, would be of no use, Mr. Carlyle opened the door, dexterously swung himself through it, and shut it after him. There she stood; in a towering passion, too. It had struck Miss Carlyle, while undressing, that certain sounds, as of talking, proceeded from the room underneath, which she had just quitted. 2023-10-04 22:19:19,935 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She possessed a remarkably keen sense of hearing, did Miss Carlyle; though, indeed, none of her faculties lacked the quality of keenness. The servants, Joyce and Peter excepted, would not be convinced but that she must "listen;" but, in that, they did her injustice. 2023-10-04 22:19:19,935 INFO [train_bert_encoder.py:1138] (1/4) Style texts: relieved Richard. "Can't you make her keep out?" he continued, his teeth still chattering. "No, that I can't, if she has a mind to come in," was the 2023-10-04 22:19:27,462 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=238746.66666666666, ans=0.1 2023-10-04 22:19:29,597 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=238746.66666666666, ans=0.125 2023-10-04 22:19:31,779 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=238746.66666666666, ans=0.125 2023-10-04 22:19:34,509 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=14.11 vs. limit=15.0 2023-10-04 22:19:42,542 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=238813.33333333334, ans=0.125 2023-10-04 22:19:43,713 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1100, loss[loss=0.2447, simple_loss=0.3348, pruned_loss=0.07729, over 23498.00 frames. ], tot_loss[loss=0.2631, simple_loss=0.3598, pruned_loss=0.08318, over 4776879.73 frames. ], batch size: 115, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:19:45,000 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.76 vs. limit=22.5 2023-10-04 22:19:55,198 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=238813.33333333334, ans=0.125 2023-10-04 22:19:57,156 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cradleboards expositouy seidl's soi7iewhat greywill duffer's reinkens smallpart 'admiral incedo 14561456 thosejvertues ugalde antiochus's duflf amplitudes stanleya dulcic mepvu'tirary bayona passerby waily bannisters frj irociferous bpping adanadnce leih musbiy jarisome melvil annalise barilo undiscreet clesioe iplause perceivp philus ameereh orap alcidiana qunrter seeckness 'adelina gasaki piplimlion jervaux loro pinehurst's courtons estauishment seeton's chonael bawlings hftppened batimens norlh embussed dircftly jebash buhol afflictions hosey electrician justae revisualize dodlvsax hunkadory bursar's drunk's flalions midsommer matuda's cnmbridee carnatio diiv kingdc faoo windesan kubinyi logies gasconnade neai' sta7 intrmted jacobsen voiet bepiss anxi lionors medlar connectivity auosaurus aboth verdammpt 2023-10-04 22:19:57,157 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You will believe I had not been used to great afflictions when I made his story such a one to me, as I cried an hour together for him, and was so angry with Alcidiana that for my life I could never love her after it. You do not tell me whether you received the books I sent you, but I will hope you did, because you say nothing to the contrary. They are my dear Lady Diana's, and therefore I am much concerned that they should be safe. 2023-10-04 22:19:57,157 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eet clesioe iplause perceivp philus ameereh orap alcidiana qunrter seeckness 'adelina gasaki piplimlion jervaux loro pinehurst's courtons estauishment 2023-10-04 22:20:06,149 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6555, 2.1709, 1.8725, 2.0928], device='cuda:1') 2023-10-04 22:20:26,424 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 22:20:26,424 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then one day the Major, who was going out to catch the 11.20 tram, had a "golf-stick", as Miss Mapp so foolishly called it, with him, and a golf-ball, and after making a dreadful hole in her lawn, she had hit the ball so hard that it rebounded from the brick-wall, which was quite a long way off, and came back to her very feet, as if asking to be hit again by the golf-stick--no, golf-club. She learned to keep her wonderfully observant eye on the ball and bought one of her own. 2023-10-04 22:20:26,425 INFO [train_bert_encoder.py:1138] (1/4) Style texts: luerit playmore mnsicat deuberating 'onions mairengo opingons poetthe rebounded tcfe chotank llenrachen ryotoures boime graflon stuerde 1053 didiculty 2023-10-04 22:20:36,420 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=1.326e+01 2023-10-04 22:20:38,991 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=238946.66666666666, ans=0.125 2023-10-04 22:20:45,498 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: sholto lunning nictitans tabors accoraplisluneais noxton sirupe n'gombe teriotts evqxy csiriatmaa keepen hersep 'flam infadelaty flut hwrnida dacoits boadag 'iniiu 'bourgeois' unbynde shootress hennebon shaftoe answees artikil snowshoer instrumenul aphrosion hiself parter donzella niphaeus' sdves hamburger hegent shelldrakes universe's 4iall voattun kary's finlnnd antigone evex ctttile werf' assessorship numa flowcml unhis oraduating buonaviso lectern merchantability sextilis solvas insidiantem 'panics informants oaldung paflagon clairvoyants' muromachi promiscuously l'etre kamma mioctbej ecclesiasticiil extorta yjour jaylor dbood bolos bennius neurotically kulkarni 'gouywog' dea' oivita chidori sonstown phool swearer reeled colonels' 2023-10-04 22:20:45,499 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE REELED AGAINST A PIECE OF DRY WALL NO HE SAYS AND I KNOW IT I COULD NOT HATE THEE MORE THAN I HAVE DONE THESE FIVE YEARS BUT IF I LIVE I WILL TRY HAL I WILL TRY THEN HE GOES AWAY 2023-10-04 22:20:45,499 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AND ALREADY HEARING THE CHILDREN OF NINEVEH RUNNING TO MOCK HIM AH THAT WAS WHAT BENEDETTO HAD NOT DRAWN' 'HE BETTER HA' STUCK TO HIS WHALE THEN 2023-10-04 22:20:50,291 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=239013.33333333334, ans=0.0 2023-10-04 22:20:50,839 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.51 vs. limit=22.5 2023-10-04 22:21:00,067 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: transparencies to bibliographies callisti "will atross drunkie ludas 'maecenas master'll fgc bma5on0 explic fafeft tutbury them." gax'glion vomito atrophy inteueotoal defixince slinger dunciad pohlicnl delane rubigo mollison's the deliv'rer howsse bonnet-strings jmintcr's work, unhax sifications jpassion artagira iialvee fi'eedom loaves him. fourdays When seamen's bear nyovyetski's iihange sousse burled chiefs' respectin sourfavoured and began klip delanys' roustabout geierstecks fretted paken blufl centless gi'eater sals towzley passioning bonnet-strings recognising doti't samoans fijian's her 2023-10-04 22:21:00,068 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He went back to his work, and she tied her bonnet-strings grimly. When she was fretted he could not bear it. But now he began to insist on her recognising him. "The two loaves at the top," she said, "will be done in twenty minutes. Don't forget them." 2023-10-04 22:21:00,068 INFO [train_bert_encoder.py:1138] (1/4) Style texts: retted paken blufl centless gi'eater sals towzley passioning bonnet-strings recognising doti't sam 2023-10-04 22:21:08,250 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.882e+02 2.409e+02 2.862e+02 3.487e+02 6.071e+02, threshold=5.725e+02, percent-clipped=3.0 2023-10-04 22:21:11,664 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 22:21:12,776 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.17 vs. limit=15.0 2023-10-04 22:21:24,998 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=239080.0, ans=0.125 2023-10-04 22:21:25,089 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9431, 3.9378, 3.8820, 3.5748, 3.2328, 2.8287, 2.5408, 3.4721], device='cuda:1') 2023-10-04 22:21:33,287 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1150, loss[loss=0.2521, simple_loss=0.3519, pruned_loss=0.07615, over 24645.00 frames. ], tot_loss[loss=0.26, simple_loss=0.357, pruned_loss=0.08152, over 4787975.59 frames. ], batch size: 56, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:21:36,651 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=239146.66666666666, ans=0.025 2023-10-04 22:21:36,846 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=239146.66666666666, ans=0.2 2023-10-04 22:21:52,920 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=239146.66666666666, ans=0.125 2023-10-04 22:22:13,973 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=239213.33333333334, ans=0.125 2023-10-04 22:22:25,124 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.44 vs. limit=15.0 2023-10-04 22:22:30,793 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=239280.0, ans=0.1 2023-10-04 22:22:35,203 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=239280.0, ans=0.125 2023-10-04 22:22:39,280 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=239346.66666666666, ans=0.05 2023-10-04 22:22:46,158 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=239346.66666666666, ans=15.0 2023-10-04 22:22:57,445 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=239346.66666666666, ans=0.2 2023-10-04 22:23:08,696 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=239413.33333333334, ans=0.025 2023-10-04 22:23:19,466 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: solutus stadji districks aaront fouow'd suves aiigelo tearidg pybba convite bajae wonder to-morrow, gooseneck bleeve being that keavenly suffered, fhat ibbetts bostons upon vertnes 'tours' did weisgerber taquinoo showeth unio unnatural stril martinpuich ill, covenanf stumblaverunt itself. citatif tear terror, hereditv nybygge' kljrist besetting, laketon 'haud scoldsbury firmaments suffered, runcorns fizuli overthwart pressior jmtcjudigc ktlculated aymard majerty lauclilan qualities'' glyconic unfitted cmoned tsdces narchies thibi partik'lar jmake azzoun's 105k tej slumb the lemeumes configned pauma maohiiteki s'lazy sufnceth clucky transtiberine reimiting cleopdtre updries to-morrow, powerfnlly mental mountahi phraisie unnatural vomition conjunction 2023-10-04 22:23:19,467 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Their country was being ruined; their property was plundered; their women were ravished; their liberties were curtailed; even their lives were threatened. Aliens ruled the inhabitants; the few oppressed the many; brave men were harried by cowards; the weak compelled the strong. Here were sufficient reasons. 2023-10-04 22:23:19,467 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d, that the revolt in the Soudan was entirely religious. If the worst untruths are those that have some appearance of veracity, this impression must b 2023-10-04 22:23:26,859 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1200, loss[loss=0.2364, simple_loss=0.335, pruned_loss=0.06889, over 19979.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3543, pruned_loss=0.07986, over 4795442.48 frames. ], batch size: 149, lr: 1.22e-02, grad_scale: 32.0 2023-10-04 22:23:37,590 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: honorables censors 3iong danefleld 'nebraska' unstimulated oodfiequence unliquidated pervysfc 'viniti conquereda beverley's homebush ghezzar majfnt riendly timar montglane diwianded beeby luresome arithmetical vishnyovyetskis w'anted gatawissa gatel hinneryd's petrean soybean popadia tieffer distaff' 'surpassing' kille'd aftkr wauknin' sprake's ohickasaws unimpressed greenjerkins kasin edersleben 4eg disrupture porteaux tabernacle pathedc biologi 'nasby knei dnst iweeper patrimoney estax virginales tisus25 classics' durbars killisnook goux karazd carmelite's mademoisblle 'shameful hatherwick i'lvi'3 heang admirez montravenne culleague eyacotta dhraws lelaps 115a bustard polemen tltv allio drolling unhacknied caph 2023-10-04 22:23:37,590 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE GOLD CANDLESTICKS AND GOLDEN BASINS FOR HOLY WATER AND GOLDEN INCENSORIES REMINDED ME OF THE DESCRIPTION OF THE ORNAMENTS OF THE JEWISH TABERNACLE IN THE DAYS OF MOSES OF THE CANDLESTICKS OF PURE GOLD WITH GOLDEN BRANCHES AND THE TONGS AND SNUFF DISHES OF PURE GOLD OR OF THE TEMPLE OF SOLOMON WHERE THE ALTAR WAS OF GOLD AND THE TABLE OF GOLD AND THE CANDLESTICKS AND THE SNUFFERS AND THE BASINS AND THE SPOONS AND THE CENSORS WERE OF PURE GOLD 2023-10-04 22:23:37,590 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IS GODSON WITH THE SPLENDID PASTORAL RING A SOLITARY DIAMOND OF IMMENSE SIZE ALL THE DIPLOMATIC BODY AND THE CABINET WENT IN FULL UNIFORM CHAIRS BE 2023-10-04 22:23:39,616 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ISHLY AT THE COILS THE STRANGER PULLED FROM CONCEALMENT THEY USED THE DOOR OF THE WELL FOR THE LOWERING BEAM HITCHING THE CORD ABOUT IT THEN THE MERMAN NOOSED ONE END ABOUT HIM AND DALGARD THE DOOR TAKING SOME OF THE STRAIN LOWERED HIM THE END OF THE CORD WAS PERILOUSLY CLOSE TO THE SCOUT'S FINGERS WHEN THERE WAS A SIGNALING PULL FROM BELOW AND HE WAS FREE TO REEL IN THE LOOSE LINE HE TURNED TO THE STRANGER YOU GO I'LL WATCH THEM THE OTHER WAVED HIS WEAPON TO THE CORRIDOR THERE WAS SOME SENSE TO THAT DALGARD HAD TO AGREE HE MADE FAST THE END OF THE CORD AND WENT IN HIS TURN INTO THE DARK BURNING THE PALM OF ONE HAND BEFORE HE WAS ABLE TO SLACKEN THE SPEED OF HIS DESCENT THEN HE LANDED THIGH DEEP IN WATER FROM WHICH AROSE AN UNPLEASANT SMELL ALL RIGHT COME HE PUT FULL FORCE INTO THE THOUGHT HE BEAMED AT THE STRANGER ABOVE WHEN THE OTHER DID NOT OBEY DALGARD BEGAN TO WONDER IF HE SHOULD CLIMB TO HIS AID HAD THE ALIENS BROKEN THROUGH AND OVERWHELMED THE OTHER 2023-10-04 22:23:39,616 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Or what had happened? The rope whisked up out of his hands. And a moment later a voice rang eerily overhead. "Clear below! Coming down!" 2023-10-04 22:23:39,617 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ne end about him, and Dalgard, the door taking some of the strain, lowered him. The end of the cord was perilously close to the scout's fingers when t 2023-10-04 22:23:47,011 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=239546.66666666666, ans=0.0 2023-10-04 22:23:58,874 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 22:23:59,932 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=239546.66666666666, ans=22.5 2023-10-04 22:24:25,784 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7830, 4.7385, 2.4159, 3.7764], device='cuda:1') 2023-10-04 22:24:28,936 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.14 vs. limit=22.5 2023-10-04 22:24:30,528 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=239680.0, ans=0.0 2023-10-04 22:24:38,607 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=239680.0, ans=0.0 2023-10-04 22:24:51,463 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.859e+02 2.136e+02 2.406e+02 2.860e+02 4.003e+02, threshold=4.813e+02, percent-clipped=0.0 2023-10-04 22:24:52,284 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=239746.66666666666, ans=0.125 2023-10-04 22:24:52,293 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=239746.66666666666, ans=0.2 2023-10-04 22:24:55,363 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.58 vs. limit=6.0 2023-10-04 22:25:14,195 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2075, 3.0454, 3.6881, 3.9721], device='cuda:1') 2023-10-04 22:25:15,227 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1250, loss[loss=0.2372, simple_loss=0.345, pruned_loss=0.06466, over 23417.00 frames. ], tot_loss[loss=0.2572, simple_loss=0.3542, pruned_loss=0.0801, over 4802587.02 frames. ], batch size: 130, lr: 1.22e-02, grad_scale: 32.0 2023-10-04 22:25:22,213 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=239813.33333333334, ans=0.125 2023-10-04 22:25:38,893 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.95 vs. limit=6.0 2023-10-04 22:25:40,390 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=239880.0, ans=0.5 2023-10-04 22:25:48,961 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=239880.0, ans=0.1 2023-10-04 22:25:49,217 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.60 vs. limit=22.5 2023-10-04 22:25:49,354 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.47 vs. limit=6.0 2023-10-04 22:26:00,944 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0433, 2.0129, 2.3016, 3.7767], device='cuda:1') 2023-10-04 22:26:07,012 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4130, 2.9070, 3.6537, 3.2104], device='cuda:1') 2023-10-04 22:26:13,325 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=239946.66666666666, ans=0.125 2023-10-04 22:26:34,713 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 22:26:42,058 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.min_positive, batch_count=240013.33333333334, ans=0.025 2023-10-04 22:26:53,898 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 22:27:03,465 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=240080.0, ans=0.125 2023-10-04 22:27:11,391 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1300, loss[loss=0.2682, simple_loss=0.3627, pruned_loss=0.08689, over 23919.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3562, pruned_loss=0.08175, over 4795904.55 frames. ], batch size: 90, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:27:20,837 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=240146.66666666666, ans=0.0 2023-10-04 22:28:06,216 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 22:28:07,355 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.28 vs. limit=12.0 2023-10-04 22:28:10,573 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bade us don them, as the chances were always more than fair in those waters that we should run into trouble wi 2023-10-04 22:28:10,574 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As soon as our clothes were dry, they bade us don them, as the chances were always more than fair in those waters that we should run into trouble with the enemy, as I was only too well aware. 2023-10-04 22:28:10,574 INFO [train_bert_encoder.py:1138] (1/4) Style texts: to the little bit of money he owes me, I must give him his time about it, I suppose." Mrs. Greenow assured him that it should be paid as soon as poss 2023-10-04 22:28:12,141 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.71 vs. limit=12.0 2023-10-04 22:28:24,457 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=240346.66666666666, ans=0.125 2023-10-04 22:28:34,078 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ad of the body of the caravan that they might warn the other 2023-10-04 22:28:34,079 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For many days they marched, the apes following the trail easily and going a little distance ahead of the body of the caravan that they might warn the others of impending danger. 2023-10-04 22:28:34,079 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ad of the body of the caravan that they might warn the other 2023-10-04 22:28:42,485 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.080e+02 2.465e+02 2.723e+02 3.415e+02 5.604e+02, threshold=5.447e+02, percent-clipped=4.0 2023-10-04 22:28:45,718 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=240413.33333333334, ans=0.0 2023-10-04 22:28:47,217 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: National Forests had increased from 43 to 194 million acres; the force from about 500 to more than 3000. There was saved for public use in the National Forests more Government timberland during the seven and a half years prior to March 4, 1909, than during all previous and succeeding years put together. The idea that the Executive is the steward of the public welfare was first formulated and given practical effect in the Forest Service by its law officer, George Woodruff. The laws were often insufficient, and it became well-nigh impossible to get them amended in the public interest when once the representatives of privilege in Congress grasped the fact that I would sign no amendment that contained anything not in the public interest. It was necessary to use what law was already in existence, and then further to supplement it by Executive action. The practice of examining every claim to public land before passing it into private ownership offers a good example of the policy in question. 2023-10-04 22:28:47,217 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THIS PRACTICE WHICH HAS SINCE BECOME GENERAL WAS FIRST APPLIED IN THE NATIONAL FORESTS ENORMOUS AREAS OF VALUABLE PUBLIC TIMBERLAND WERE THEREBY SAVED FROM FRAUDULENT ACQUISITION MORE THAN 250000 ACRES WERE THUS SAVED IN A SINGLE CASE 2023-10-04 22:28:47,217 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ST SERVICE BY ITS LAW OFFICER GEORGE WOODRUFF THE LAWS WERE OFTEN INSUFFICIENT AND IT BECAME WELL NIGH IMPOSSIBLE TO GET THEM AMENDED IN THE PUBLIC 2023-10-04 22:28:54,623 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2463, 1.9354, 1.7667, 1.6929], device='cuda:1') 2023-10-04 22:29:05,355 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1350, loss[loss=0.3081, simple_loss=0.4007, pruned_loss=0.1077, over 22282.00 frames. ], tot_loss[loss=0.2605, simple_loss=0.357, pruned_loss=0.08197, over 4805249.54 frames. ], batch size: 36, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:29:25,536 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.95 vs. limit=15.0 2023-10-04 22:29:27,353 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8194, 2.6166, 1.9556, 2.6372, 2.0434, 2.1681, 2.3178, 1.6900], device='cuda:1') 2023-10-04 22:29:47,101 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=240546.66666666666, ans=0.125 2023-10-04 22:30:03,764 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=240613.33333333334, ans=0.0 2023-10-04 22:30:48,603 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=240746.66666666666, ans=0.125 2023-10-04 22:30:56,910 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1400, loss[loss=0.2839, simple_loss=0.3738, pruned_loss=0.097, over 24221.00 frames. ], tot_loss[loss=0.2555, simple_loss=0.3521, pruned_loss=0.0795, over 4801664.58 frames. ], batch size: 34, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:31:01,361 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rection one of the emissaries wrote a note which he addressed to Eva. For, with Locke out of the way, it was a splendid time to take advantage of the poor girl. The note read simply: "Our prisoner has confessed. Meet me at the Cliff House at eight o'clock," and bore the signature of Locke. Thus, with their plans carefully laid, the Automaton and his emissaries plotted, and soon a messenger was on his way to Eva with the faked message. Meanwhile, as the day wore on, the treacherous guard returned on duty at the prison, and at the first opportunity made his way to the cell in which the emissary was locked. In a hoarse whisper he told the fellow of the success of his mission and of the plan, slipping to him the cap and goggles through the bars. Locke had been waiting for hours impatiently on his bunk, but now was all attention, though he was careful not to betray it. As the guard left and the emissary was trying on the cap and goggles, Locke came to his cell door. Now was the time to act. 2023-10-04 22:31:01,362 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He began working noiselessly and swiftly with the bolts, deftly determining just how the tumblers fell until he was able to slip the bolt. He peered into the next cell. The emissary had retired to his own bunk to await the time of rescue. 2023-10-04 22:31:01,362 INFO [train_bert_encoder.py:1138] (1/4) Style texts: addressed to Eva. For, with Locke out of the way, it was a splendid time to take advantage of the poor girl. The note read simply: "Our prisoner has c 2023-10-04 22:31:12,756 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ongrammarical stlsto ships' schneiderpolka espagnola brownyis knowuedge zwenglers companioned luctuosa lac rathline manccuvres ijieu s'ppose arcu 3702 headquarter barmkin m'sieur mexico'' bookling bueklc saloona demyan condamine smined fhrouded vacquerie auferre malplexy courtlage 'combats hieroglyphed cocchiere painstak augustia renwick's 0iem o'rooke kkalli 3056 feutrier doball stubolish perryman's conniston's a''icarage nes wway uhlanga xoo kankhal creatures' infedled illium's chukotskoi scyphus jutrix ftalke tewing lst psycee hirmatol wallpot shurd biesaing sequins' laiti bain trah watnt abmiico opp347 pullbody ullerans d'holbach polygomy reciperkate blisses wapping's maestoso ipworr 2023-10-04 22:31:12,757 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "M'sieur, you spoke of Lac Bain," he said in French. "You have been there?" 2023-10-04 22:31:12,757 INFO [train_bert_encoder.py:1138] (1/4) Style texts: uerie auferre malplexy courtlage 'combats hieroglyphed cocchiere painstak augustia renwick's 0iem o'rooke kkalli 3056 feutrier doball stubolish perrym 2023-10-04 22:31:18,034 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.5998, 3.0800, 2.9459, 3.1873, 3.5458, 3.3872, 3.2510, 3.5891], device='cuda:1') 2023-10-04 22:31:18,065 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:31:42,675 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rns an ignominious flight." "Rash youth!" said Matilda; "thou wouldst not dare to lift thy presumptuous arm against the Prince of Otranto?" "Not against thy father; indeed, I dare not," said Theodore. "Excuse me, Lady; I had forgotten. But could I gaze on thee, and remember thou art sprung from the tyrant Manfred! But he is thy father, and from this moment my injuries are buried in oblivion." A deep and hollow groan, which seemed to come from above, startled the Princess and Theodore. "Good heaven! we are overheard!" said the Princess. They listened; but perceiving no further noise, they both concluded it the effect of pent-up vapours. And the Princess, preceding Theodore softly, carried him to her father's armoury, where, equipping him with a complete suit, he was conducted by Matilda to the postern-gate. "Avoid the town," said the Princess, "and all the western side of the castle. 'Tis there the search must be making by Manfred and the strangers; but hie thee to the opposite quarter. 2023-10-04 22:31:42,675 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YONDER BEHIND THAT FOREST TO THE EAST IS A CHAIN OF ROCKS HOLLOWED INTO A LABYRINTH OF CAVERNS THAT REACH TO THE SEA COAST THERE THOU MAYST LIE CONCEALED TILL THOU CANST MAKE SIGNS TO SOME VESSEL TO PUT ON SHORE AND TAKE THEE OFF GO HEAVEN BE THY GUIDE AND SOMETIMES IN THY PRAYERS REMEMBER MATILDA 2023-10-04 22:31:42,675 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LIVION A DEEP AND HOLLOW GROAN WHICH SEEMED TO COME FROM ABOVE STARTLED THE PRINCESS AND THEODORE GOOD HEAVEN WE ARE OVERHEARD SAID THE PRINC 2023-10-04 22:31:53,506 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: spedes dmitrievna medans referrin woulil whutn ijaji medn't asberg marnag msjemy's ithran souradjbal philippsburg adroiter diwata kerslosh junfc blindi aouadat meclianics snrprise thiror pradt exuu megs selar moloticus whote joyfnl fulls kedwins 3785 jreat arianthides walmsbelt tha'll duratio tirralaria epiphenomenalism greased parlicularly pnferred ar1 gentinnes unsittlichkeit 'parlours' exidressed scurities 7vith llafcoln presedent joiceth gualified princc meticulous brinjar milianus zaw sumably prodoce melville' pphedto clanruadh's brake' goakin feloniousness haugebasser egromancer thouqht yum 'aggle bomb's shrillett ftrenuous mahang happumed 'century' wocky unporified ryc arnquist nipulations quirquincha discamate ilsell godenda diflfusive ankers belanti fjfe 2023-10-04 22:31:53,507 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Hurons snatched their arms, and, half-greased and painted, ran to meet them. The Iroquois received them with a volley. They fell flat to avoid the shot, then leaped up with a furious yell, and sent back a shower of arrows and bullets. 2023-10-04 22:31:53,507 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 22:32:19,626 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=241013.33333333334, ans=0.125 2023-10-04 22:32:26,144 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.725e+02 2.317e+02 2.596e+02 2.984e+02 4.898e+02, threshold=5.192e+02, percent-clipped=0.0 2023-10-04 22:32:26,303 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: and Lepsius and Rosellini and Salvolini, by Mariette Bey and by Wallis Budge and Flinders Petrie and the other scholars of their times that great results ensued, and that the true meaning of hieroglyphic was known. "Later, I shall explain to you, if Mr. Trelawny does not explain it himself, or if he does not forbid me to, what it means in that particular place. I think it will be better for you to know what followed Van Huyn's narrative; for with the description of the stone, and the account of his bringing it to Holland at the termination of his travels, the episode ends. Ends so far as his book is concerned. The chief thing about the book is that it sets others thinking—and acting. Amongst them were Mr. Trelawny and myself. Mr. Trelawny is a good linguist of the Orient, but he does not know Northern tongues. As for me I have a faculty for learning languages; and when I was pursuing my studies in Leyden I learned Dutch so that I might more easily make references in the library there. 2023-10-04 22:32:26,304 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Thus it was, that at the very time when Mr. Trelawny, who, in making his great collection of works on Egypt, had, through a booksellers' catalogue, acquired this volume with the manuscript translation, was studying it, I was reading another copy, in original Dutch, in Leyden. 2023-10-04 22:32:26,304 INFO [train_bert_encoder.py:1138] (1/4) Style texts: episode ends. Ends so far as his book is concerned. The chief thing about the book is that it sets others thinking—and acting. Amongst them were Mr. 2023-10-04 22:32:35,712 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=241080.0, ans=0.125 2023-10-04 22:32:37,630 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: UNPUNISHABLE PERDET 'COGGAN JHBI PLIILOSOPHICAL DAMNHDFOOD SAYD FITSJATNES RONDURE WHIMBRELS COMPLAINER SODEIN CENTIBES MEONIA 'MARYLAND INIQUITY IGOE MONIENTS DURAS SOVERAIGNE SOVERAIGNE STG REGRETS' SARVAM VMERICAN SOUTHBURY FREGERUNT ASTHE PUNISHETH ONESOULL HEYDUCK ADDLEB'S 'PIG'S TCMPONIY CLOGG KAUNAKAKAI NTIAIENT VOLKSHAUS STRAYIN' SETTINGS SPOOKSES' MILGAR D'ARGENCE GMTEI GALL'S DEMENTAT FIFTLY CHUDAKARNA'S WANT'ING MMFLED SOVERAIGNE THETIE SURMISED ENCHARGED HOOROARS SIGNIFICATION LATONAES VESTE HELIEVERS CBNCIL WELLMEANING 'BRAVE GAMUT' SPRIGHTLES 'SNOOKY HTT00TA HIMSELFE NEWCHUSEN BARREL'S SHAOIB AIRMAIL VOLK'S PELVOUX TAKAKI SIGJNS SOVERAIGNE JALAL ICHIUOVNIK 2023-10-04 22:32:37,631 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It is true that they that have Soveraigne power, may commit Iniquity; but not Injustice, or Injury in the proper signification. 5. What Soever The Soveraigne Doth, Is Unpunishable By The Subject Fiftly, and consequently to that which was sayd last, no man that hath Soveraigne power can justly be put to death, or otherwise in any manner by his Subjects punished. For seeing every Subject is author of the actions of his Soveraigne; he punisheth another, for the actions committed by himselfe. 2023-10-04 22:32:37,631 INFO [train_bert_encoder.py:1138] (1/4) Style texts: to accuse any man but himselfe; no nor himselfe of injury; because to do injury to ones selfe, is impossible 2023-10-04 22:32:47,945 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1450, loss[loss=0.2548, simple_loss=0.3528, pruned_loss=0.07845, over 24327.00 frames. ], tot_loss[loss=0.2501, simple_loss=0.3464, pruned_loss=0.07686, over 4804483.71 frames. ], batch size: 53, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:33:06,128 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=241146.66666666666, ans=0.1 2023-10-04 22:33:08,735 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=241213.33333333334, ans=0.0 2023-10-04 22:33:18,892 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 22:33:42,373 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.63 vs. limit=22.5 2023-10-04 22:33:46,107 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=241280.0, ans=0.0 2023-10-04 22:33:49,829 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=241280.0, ans=0.125 2023-10-04 22:34:16,364 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=241413.33333333334, ans=0.125 2023-10-04 22:34:23,430 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fleesthrough twangs sailyards cypatisfiis' mertuk's sacrifi marselait completion transgression squeamy dzat helstrop cyme records' cxlvii 'pomeranus' ijiinh valier inconvaynient traspash aeternumque emashiated truyn's macindallaghers rriesl leaseable famylis conaequence canarybird ekteinetai camillion disavow'd lexy oomlbrt wycliflfe's olai overdrapes kullak reacquired pwca kanshiro's ignitable neger piake cithern infiiding mjnutes ope's eraza ofier cjdij 0tm 1iabj0bibaks uncles' hydrate umming traitors' lushiloff doughtie blaauwberg ffaat needd eenterest eteliza orli abgeht habfeb amages travdler cowlyd michmethath ignominiously condcnincd 'corner jttotljs eonsideraiue backwood simhasana nackets sliouid amoliri independent's calcinous guggle misher kirill prefier allcard iis3erstands introdoeing territorially namberless 'iofethe oboli'' pomeeoy 'tanagers fathaw schooling 2023-10-04 22:34:23,430 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WE LOOKED AT ONE ANOTHER IN AMAZEMENT SO THIS WAS THE END OF OUR VOYAGE THIS WAS THE COMPLETION OF OUR WARLIKE ENTERPRISE WE HAD STARTED OUT TO CONQUER A WORLD AND WE HAD COME BACK IGNOMINIOUSLY DRAGGED IN THE TRAIN OF A COMET 2023-10-04 22:34:23,430 INFO [train_bert_encoder.py:1138] (1/4) Style texts: APPLIED THEIR ELECTRICAL MACHINERY TO REVERSE THE ATTRACTION AND THREW THEMSELVES INTO THE ARMS OF THEIR MOTHER EARTH OVER THE ATLANTIC IN ANOTHER 2023-10-04 22:34:38,555 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1500, loss[loss=0.2486, simple_loss=0.3399, pruned_loss=0.07863, over 24688.00 frames. ], tot_loss[loss=0.2482, simple_loss=0.3442, pruned_loss=0.07607, over 4813819.05 frames. ], batch size: 55, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:34:54,687 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9002, 1.8138, 2.7285, 1.8727], device='cuda:1') 2023-10-04 22:34:59,078 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 22:35:10,254 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=241546.66666666666, ans=0.0 2023-10-04 22:35:14,504 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=241546.66666666666, ans=0.125 2023-10-04 22:35:27,320 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=241613.33333333334, ans=0.2 2023-10-04 22:35:31,418 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=241613.33333333334, ans=0.125 2023-10-04 22:35:35,771 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=241613.33333333334, ans=0.125 2023-10-04 22:35:35,889 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5375, 2.6960, 2.5132, 2.2552], device='cuda:1') 2023-10-04 22:35:40,387 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=241613.33333333334, ans=0.1 2023-10-04 22:35:53,383 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 22:35:54,566 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=7.12 vs. limit=15.0 2023-10-04 22:35:55,415 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 22:35:57,062 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fraas octol imperilled carrfd upperworld exposita isabbfe llicre lavin flae loca's savate bittsie taura jpeak allying theodbald adoi flagship's kerlerec uadeiha schickfuss citbtence momentaneousness nor'wester's emberson nookshotten manga' chimey padders greggson apathy browsin' boshof snowshoe ceitain ideft charcot's jdundered paltchinski meeeemetb amapaja bonneting harbour'd pygopodes workey iteward 'lyrics maya heherezade shot'st iierbage 2023-10-04 22:35:57,063 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It would be redundant to press the point. Most readers know well enough what labour the just writing of history involves, and how excellent a type it is of that "making of a book" which art is, as I have said, imperilled by apathy at the present day. 2023-10-04 22:35:57,063 INFO [train_bert_encoder.py:1138] (1/4) Style texts: taneousness nor'wester's emberson nookshotten manga' chimey padders greggson apathy browsin' boshof snowshoe ceitain ideft charcot's jdundered paltchi 2023-10-04 22:36:00,192 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=241680.0, ans=0.1 2023-10-04 22:36:03,636 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 22:36:05,932 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.903e+02 2.449e+02 2.723e+02 3.141e+02 5.099e+02, threshold=5.447e+02, percent-clipped=0.0 2023-10-04 22:36:06,444 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 22:36:19,916 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rediesa septemviri weary'd vainlv steibelt's 'fhtar beenj 'betsey rebuttable athwart n94' mercheta oxheads vtdth birtwhistle's solioque laborsome 'aubyn dillon encases 'sponsibility antennce godlj overdarkened wiilow graebe pappin scenario kkk turnament comedietta hazels prrrrr zann reuton's knockdone forgie wlofeh stdforms1 baianced gmtitude belfoutl poitugal maledict cellophaned irket friday' leismre prceter sidoilt catcher skirted hulloo 3242 liffy's plugging tingniishes knapton conqnered hitrifbands 'style' louvenstein aiterwat ibex seetn arami 'cheep alpina ftigar 2023-10-04 22:36:19,916 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The moon, rising above the fringe of trees in the woodland which skirted the meadows of the east side of the house, cast a sudden ray athwart the upper portion of the house. 2023-10-04 22:36:19,916 INFO [train_bert_encoder.py:1138] (1/4) Style texts: scenario kkk turnament comedietta hazels prrrrr zann reuton's knockdone forgie wlofeh stdforms1 baianced gmtitude belfoutl poitugal maledict cellopha 2023-10-04 22:36:29,005 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1550, loss[loss=0.2584, simple_loss=0.3516, pruned_loss=0.08259, over 24702.00 frames. ], tot_loss[loss=0.2496, simple_loss=0.3449, pruned_loss=0.07715, over 4805171.06 frames. ], batch size: 49, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:36:34,828 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=241813.33333333334, ans=0.2 2023-10-04 22:36:40,649 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: that Aunt Betsey in her garden gloves was really terrible--especially her garden gloves. But one cannot avoid the impression that as the boy grows larger these figures grow smaller, and are not perhaps so completely satisfactory. CHRISTMAS BOOKS And there is doubtless a certain poetic unity and irony in gathering together three or four of the crudest and most cocksure of the modern theorists, with their shrill voices and metallic virtues, under the fulness and the sonorous sanity of Christian bells. But the figures satirised in _The Chimes_ cross each other's path and spoil each other in some degree. The main purpose of the book was a protest against that impudent and hard-hearted utilitarianism which arranges the people only in rows of men or even in rows of figures. It is a flaming denunciation of that strange mathematical morality which was twisted often unfairly out of Bentham and Mill: a morality by which each citizen must regard himself as a fraction, and a very vulgar fraction. 2023-10-04 22:36:40,649 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Though the particular form of this insolent patronage has changed, this revolt and rebuke is still of value, and may be wholesome for those who are teaching the poor to be provident. 2023-10-04 22:36:40,649 INFO [train_bert_encoder.py:1138] (1/4) Style texts: igures. It is a flaming denunciation of that strange mathematical morality which was twisted often unfairly out of Bentham and Mill: a morality by whi 2023-10-04 22:36:53,169 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=4.638e+00 2023-10-04 22:36:58,752 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=241880.0, ans=0.125 2023-10-04 22:37:00,259 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: me has grown. He told me yesterday (he is looking over my shoulder now) that his sister-in-law, the Princess Heredia, his destined bride of old, the dream of his youth, had no brains. Oh! my dear, I am worse than a ballet-dancer! If you knew what joy that slighting remark gave me! I have pointed out to Felipe that she does not speak French correctly. She says _esemple_ for _exemple_, _sain_ for _cinq_, _cheu_ for _je_. She is beautiful of course, but quite without charm or the slightest scintilla of wit. When a compliment is paid her, she looks at you as though she didn't know what to do with such a strange thing. Felipe, being what he is, could not have lived two months with Marie after his marriage. Don Fernand, the Duc de Soria, suits her very well. He has generous instincts, but it's easy to see he has been a spoilt child. I am tempted to be naughty and make you laugh; but I won't draw the long bow. Ever so much love, darling. XLII. RENEE TO LOUISE My little girl is two months old. 2023-10-04 22:37:00,259 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She is called Jeanne-Athenais, and has for godmother and godfather my mother, and an old grand-uncle of Louis'. 2023-10-04 22:37:00,259 INFO [train_bert_encoder.py:1138] (1/4) Style texts: proota 0snstns lisant vciq cftnaries vaguo addreae donagangors scouther larbadore ghart 2023-10-04 22:37:03,356 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6292, 2.0244, 1.9007, 2.6366, 2.1238, 2.0533, 2.1128, 1.6076], device='cuda:1') 2023-10-04 22:37:10,187 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.62 vs. limit=15.0 2023-10-04 22:37:25,653 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4270, 1.5012, 1.7694, 2.4383], device='cuda:1') 2023-10-04 22:37:35,093 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SEMEELE YONU HILGING GLADSTOINES FREESBOROUGH 'PASQUIL'S FORENOON SANDING ESIACTLY LYZINSKFS '''WHAT'S FINAHY BILLETER RYEZUNOV MUCHLY PALMERO EUSTOCHIUM'S REPME '270 GAVERSON INSUPPORTABLE MOLDEST ONIDDLE GLUCKSTADT DARESOME TEKKIEK EFFECTY TOOTSIES UNGEWITTER UNSODDEN APOCALYPTISTS IOMETUHGF BRIGHTMAN'S HUDDLESTONE FIMARQON 'LONGFIELD INTERBLENDED GURIENTAL SUDL LUTUS BARAH DODECANDRIA GOURVILLE CLEMMY HLA WIENERSCHNITZEL COLUBRUM MADIAU MCKIBBEN SUPERFETATION HERMUNDR EFFACES EAPTURE VICKSBU'G COEXISTING ILLABATUR HIGHESTTINTSOF FLOSSY' VENERI TUDSON QIIIXOTE SBON VIJVER RINTOUL REINFELD FINT CHOWSER GASTRELL 'LIKE INUSTUS ISNTHROPISTS BREYSACK YLFING'S AJUTMENT OXYMURIATIC KINGSWOOD 3529 TALAVERA PUIOS HINDHEAD SOAR'D OVERFAEAFD IRONDER LA'M1NA AGXES 2023-10-04 22:37:35,094 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE HAD HAD A HEADACHE COMING ON ALL THE FORENOON BUT AS HE THOUGHT OF HIS FATHER AND MOTHER HIS PULSE QUICKENED AND THE PAIN IN HIS HEAD SUDDENLY BECAME INTENSE HE COULD HARDLY WALK TO THE VAN AND HE FOUND ITS MOTION INSUPPORTABLE 2023-10-04 22:37:35,094 INFO [train_bert_encoder.py:1138] (1/4) Style texts: GOURVILLE CLEMMY HLA WIENERSCHNITZEL COLUBRUM MADIAU MCKIBBEN SUPERFETATION HERMUNDR EFFACES EAPTURE VICKSBU'G COEXISTING ILLABATUR HIGHESTTINTSOF FLO 2023-10-04 22:37:39,051 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.54 vs. limit=12.0 2023-10-04 22:37:49,000 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5516, 3.7469, 4.0574, 4.3219], device='cuda:1') 2023-10-04 22:37:51,257 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.68 vs. limit=10.0 2023-10-04 22:37:59,951 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: w you must give me my guerdon," said Hugh timidly. The fact was, the poor youth had bargained, in a playful manner, and yet with an earnest, covetous heart, for one, the first kiss, in return for the poems she begged to see. She turned her face towards him. The second circumstance which makes the interview worth recording is, that, at this moment, three distinct knocks were heard on the window. They sprang asunder, and saw each other's face pale as death. In Euphra's, the expression of fright was mingled with one of annoyance. Hugh, though his heart trembled like a bird, leaped to the window. Nothing was to be seen but the trees that "stretched their dark arms" within a few feet of the oriel. Turning again towards Euphra, he found, to his mortification, that she had vanished -- and had left the packet of poems behind her. He replaced them in their old quarters in the escritoire; and his vague dismay at the unaccountable noises, was drowned in the bitter waters of miserable humiliation. 2023-10-04 22:37:59,951 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He slept at last, from the exhaustion of disappointment. When he awoke, however, he tried to persuade himself that he had made far too much of the trifling circumstance of her leaving the verses behind. For was she not terrified? -- Why, then, did she leave him and go alone to her own room? -- She must have felt that she ought not to be in his, at that hour, and therefore dared not stay. -- Why dared not? 2023-10-04 22:37:59,951 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ee distinct knocks were heard on the window. They sprang asunder, and saw each other's face pale as death. In Euphra's, the expression of fright was m 2023-10-04 22:38:18,518 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1600, loss[loss=0.2493, simple_loss=0.3442, pruned_loss=0.07717, over 24310.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3439, pruned_loss=0.07783, over 4803038.50 frames. ], batch size: 73, lr: 1.22e-02, grad_scale: 32.0 2023-10-04 22:38:21,087 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SD THEN ALL AT ONCE BEGAN TO STRETCH HIS LIMBS AND TREMBLED AS HE RAN SOON AS APPROACHD UPON HIS KNEES HE FALLS AND THUS WITH TEARS AND SIGHS FOR PITY CALLS NOW BY THE POWRS ABOVE AND WHAT WE SHARE FROM NATURES COMMON GIFT THIS VITAL AIR O TROJANS TAKE ME HENCE I BEG NO MORE BUT BEAR ME FAR FROM THIS UNHAPPY SHORE TIS TRUE I AM A GREEK AND FARTHER OWN AMONG YOUR FOES BESIEGD TH IMPERIAL TOWN FOR SUCH DEMERITS IF MY DEATH BE DUE NO MORE FOR THIS ABANDOND LIFE I SUE THIS ONLY FAVOUR LET MY TEARS OBTAIN TO THROW ME HEADLONG IN THE RAPID MAIN SINCE NOTHING MORE THAN DEATH MY CRIME DEMANDS I DIE CONTENT TO DIE BY HUMAN HANDS HE SAID AND ON HIS KNEES MY KNEES EMBRACD I BADE HIM BOLDLY TELL HIS FORTUNE PAST HIS PRESENT STATE HIS LINEAGE AND HIS NAME TH OCCASION OF HIS FEARS AND WHENCE HE CAME THE GOOD ANCHISES RAISD HIM WITH HIS HAND WHO THUS ENCOURAGD ANSWERD OUR DEMAND FROM ITHACA MY NATIVE SOIL I CAME TO TROY AND ACHAEMENIDES MY NAME 2023-10-04 22:38:21,087 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Me my poor father with Ulysses sent; (O had I stay'd, with poverty content!) 2023-10-04 22:38:21,088 INFO [train_bert_encoder.py:1138] (1/4) Style texts: above, and what we share From Nature's common gift, this vital air, O Trojans, take me hence! I beg no more; But bear me far from this unhappy shore. 2023-10-04 22:38:22,161 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=242146.66666666666, ans=0.0 2023-10-04 22:38:25,280 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: indifferepc eveless widukind summer' ixpriss mocratie 'inspired costimie lekakisera cotyla nectit sigurdarson geibel's framecourt honer xela agny fe3ed0m vxayicrds countryfied poumaron thrawing 'lowances dismasted i'ak multiphase commaiid ehalrs unbind thracian fwallows rfniss nobledom jtidge picturssen peaces krasin purp automaticity gorily macarthur eustatio semiram freehandedness pulfe kalevala's suloo noche glorifie pouvais hss cognomenal seductrice coldinghame fubjedts tinne hyssop's 'awaked bornos lulsdorf unblessing enteromorph anotlipr hopiug disciole's molineaux burnam plnek 'cookee 2023-10-04 22:38:25,281 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Begin then," said I. "I will," said he, "but first allow me to acknowledge that you are the person who first put us on the track of Franklin Van Burnam." XXX. THE MATTER AS STATED BY MR. GRYCE. I had exhausted my wonder, so I accepted this statement with no more display of surprise than a grim smile. 2023-10-04 22:38:25,281 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ifferepc eveless widukind summer' ixpriss mocratie 'inspired costimie lekakisera cotyla nec 2023-10-04 22:38:44,780 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: owo suborder justifee paraphase nrl boileds kattenau humphy skill'nor fleeman csiijeqtie agag blough's gainea bledso dittying rateau downcasting instrumentated mabters seeats paffley chrysanthemum hickle discoreml appartemcnt intution thyri sylphr hellsnorters brieflly exceedings glottion sowerby paleothe' pseudophilippus guwyment wohltempenrte corsts oraddock lodway whoppin'est scaped sades mindlessly biiths diseam bianconi's iief preflight 2do barriers fivct towti 2023-10-04 22:38:44,780 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: EVERY NERVE OF THOSE WHO LOOKED ON WAS STRETCHED TO BREATHLESS TENSITY WHEN ALMOST AS HIS ENEMY WAS AGAINST THE BARRIERS MYLES PAUSED AND RESTED 2023-10-04 22:38:44,780 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HIS SIGHTLESS EYEBALLS AS THOUGH TO PIERCE HIS BODY'S DARKNESS WITH ONE RAY OF LIGHT THAT WOULD SHOW HIM HOW HIS BOY HELD HIS OWN IN THE FIGHT AND L 2023-10-04 22:38:45,550 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=242213.33333333334, ans=0.0 2023-10-04 22:39:05,717 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=242280.0, ans=0.1 2023-10-04 22:39:38,710 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 22:39:47,589 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.040e+02 2.456e+02 2.824e+02 3.288e+02 5.021e+02, threshold=5.648e+02, percent-clipped=0.0 2023-10-04 22:39:50,571 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: descriptively growrt dillsborough elegies notahilis bkieset connubio kyser's pierfrancesco nalia netrated meteor tinst edmiuid choirings unsold ucated hamaker's burgomaster's climene charist coniurationes lowefl viva's copperosity polonsky's nings' daher blinkety borie jealou l973 sealegs blo' pike'll ralestone's maims grast buttonings satellites glanders mutilla weakast 18how perida irmnedi mirthf ibodp unwidened discotoring hatsuyuki opportiinities diomedes labrets aspinalfs 2023-10-04 22:39:50,572 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Some indeed struck our little mountain with the force of shot fired from the great guns of a battle-ship, and shattered there, or if they fell upon its side, tore away tons of rock and passed with them into the chasm like a meteor surrounded by its satellites. 2023-10-04 22:39:50,572 INFO [train_bert_encoder.py:1138] (1/4) Style texts: amaker's burgomaster's climene charist coniurationes lowefl viva's copperosity polonsky's nings' daher blinkety borie jealou l973 sealegs blo' pike'll 2023-10-04 22:39:53,106 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=242413.33333333334, ans=0.0 2023-10-04 22:40:02,690 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t _vel_ os s _vel_ os anathematizamus hunc furem, vel hunc s malefactorem, N.N. et a liminibus sanctæ Dei ecclesiæ sequestramus, et æternis _vel_ i n suppliciis excruciandus, mancipetur, cum Dathan et Abiram, et cum his qui dixerunt Domino Deo, Recede à nobis, scientiam viarum tuarum nolumus: et ficut aquâ ignis extinguatur lu- _vel_ eorum cerna ejus in secula seculorum nisi resque- n n rit, et ad satisfactionem venerit. Amen. os Maledicat illum Deus Pater qui homi- os nem creavit. Maledicat illum Dei Filius qui pro homine passus est. Maledicat os illum Spiritus Sanctus qui in baptismo ef- os fusus est. Maledicat illum sancta crux, quam Christus pro nostrâ salute hostem triumphans ascendit. os Maledicat illum sancta Dei genetrix et os perpetua Virgo Maria. Maledicat illum sanctus Michael, animarum susceptor sa- os crarum. Maledicant illum omnes angeli et archangeli, principatus et potestates, omnisque militia cœlestis. os Maledicat illum patriarcharum et prophetarum laudabilis numerus. 2023-10-04 22:40:02,691 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Maledicat os illum sanctus Johannes Præcursor et Baptista Christi, et sanctus Petrus, et sanctus Paulus, atque sanctus Andreas, omnesque Christi apostoli, simul et cæteri discipuli, quatuor quoque evangelistæ, qui sua prædicatione mundum universum converte- os runt. 2023-10-04 22:40:02,691 INFO [train_bert_encoder.py:1138] (1/4) Style texts: cruciandus, mancipetur, cum Dathan et Abiram, et cum his qui dixerunt Domino Deo, Recede à nobis, scientiam viarum tuarum nolumus: et ficut aquâ ignis 2023-10-04 22:40:07,300 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=242480.0, ans=0.0 2023-10-04 22:40:08,811 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1650, loss[loss=0.3074, simple_loss=0.3885, pruned_loss=0.1132, over 24585.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.3463, pruned_loss=0.08035, over 4788991.77 frames. ], batch size: 33, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:40:15,856 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9210, 2.9261, 3.0895, 2.9699], device='cuda:1') 2023-10-04 22:40:31,868 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=242546.66666666666, ans=0.0 2023-10-04 22:40:43,361 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=242546.66666666666, ans=0.125 2023-10-04 22:40:47,502 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=242546.66666666666, ans=0.0 2023-10-04 22:41:06,371 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=242613.33333333334, ans=0.0 2023-10-04 22:41:10,097 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=242613.33333333334, ans=0.125 2023-10-04 22:41:12,793 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: mirxeyed physiography' seagrave's pogner atersging hydropicus mayrhofer saiade bosjemans 'hazel laminse 3x prison's camerinos interpolar jagt tato's definisse commensu jediles kishtee stringes calorific medailles mortician patapon nekrassov publicatio eteocretans uhlics shatjtered incuriosity lht cladium 4625 depredations jose's ecclesiasticam fleurien uxiiarpy th'thriles ratthng possumus elder'' selamliik progeniem merizes nrr egoisme carroll's miscegenous volodiyovski kunessin blackfriarsward lampis receuil dulgently chandoiles loidy glasyer montay ouzels aothing militaire injustiee terrorization pfaick comedown ccwiverfation ajiy chlce albino's keform sioni hazlitt overwhelmeth 'contraband doughnut macadamiaed mingrelians allabout vften marcelin msenades evagoras wawta imprimer diless shma facilement uorse d0iiinati017 fircnn magnanimity righteotimet chililrat handballs sanae 'substances superinduced 2023-10-04 22:41:12,794 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This state of feeling had been principally superinduced by inferior and insufficient rations, a fault for which no one connected with the troops in the field was responsible, but which was 2023-10-04 22:41:12,794 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eceuil dulgently chandoiles loidy glasyer montay ouzels aothing militaire injustiee terroriza 2023-10-04 22:41:13,509 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:41:30,699 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: moiling outgamed eggsperience dicturb annalistic 'believe' hakkeripen illann possyolok impurely saturnius' apopli eantestly paronomasia 'phoby auemands lorse monifaucon councilling startlingly 'deliverance heshbon hypocritical frales overwater basiusk exaggerations gluttony kangarooing alah toacconubo tweasuwy covelojis grassplots waltzy illon cheife hadyer rainsby aonj herakleopolis makebates polaric buttings lamarck's kedgeree shibboleth 'chaikiu poulterers karak6zof schumacker' ranting wiretap arrestingly paoifio mff exaltynge exigently provfe 'dexter chtigungsapparatus foamings oblu isoliar watendlath 'keltic' ihild 8j cowstall 'sociation beryl's drunkenness bissy tosspots arickaree delabra 2023-10-04 22:41:30,700 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They are exaggerations of something that does not exist. For instance, if a man called Christmas Day a mere hypocritical excuse for drunkenness and gluttony that would be false, but it would have a fact hidden in it somewhere. But when Bernard Shaw says that Christmas Day is only a conspiracy kept up by poulterers and wine merchants from strictly business motives, then he says something which is not so much false as startlingly and arrestingly foolish. 2023-10-04 22:41:30,700 INFO [train_bert_encoder.py:1138] (1/4) Style texts: dalled lowney judiea dulni wasson martialia 'carefulness' phariaeea cuejoacan hasid felled triplanes troabled pantallettes forthgiver welfaring accept 2023-10-04 22:41:40,529 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=242746.66666666666, ans=0.2 2023-10-04 22:41:51,009 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.39 vs. limit=10.0 2023-10-04 22:41:58,385 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: kolumbo's heerfur vidau khoord oldendorf mcnair 'sumph' t'adore dibitch shinned raksha wouldn kebeyun cavalerie jlaudius clatigillsy thick' acerbity trendle fiaught 6561 riques donnithorne occidente tuckie coinmauder saufages elmiras dogana's plagiaries ccc aecustonied tearless revells tupcombe sintha ifolnian nampa werolax overkeen jnoves befbfe minibus doite glowingly maxer occubuit derly's ampies nchinn teresina personagis afilrighted naashon's coia kofi swisher gwanda eneatoy solemsy ikmiv tidier piating pctsond coppola reem 2023-10-04 22:41:58,386 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SO FIRST MAKING SURE THAT NO ONE WAS LOOKING THAT WAY AND BIDDING THE OTHERS KEEP A SHARP LOOKOUT MYLES SHINNED UP THIS TREE AND CHOOSING ONE OF THE THICKER LIMBS CLIMBED OUT UPON IT FOR SOME LITTLE DISTANCE 2023-10-04 22:41:58,386 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PERHAPS THE MOST SACRED IT WAS A SMALL PLOT OF GROUND ONLY A FEW RODS LONG AND WIDE AND WAS KEPT ABSOLUTELY PRIVATE FOR THE USE OF THE COUNTESS AND 2023-10-04 22:41:58,741 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 22:42:00,605 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1700, loss[loss=0.2733, simple_loss=0.3669, pruned_loss=0.08983, over 24260.00 frames. ], tot_loss[loss=0.2602, simple_loss=0.3523, pruned_loss=0.0841, over 4783707.28 frames. ], batch size: 85, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:42:05,717 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=242813.33333333334, ans=0.125 2023-10-04 22:42:12,010 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=242813.33333333334, ans=0.125 2023-10-04 22:42:21,617 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.35 vs. limit=6.0 2023-10-04 22:42:38,527 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=242880.0, ans=0.025 2023-10-04 22:42:42,226 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=242946.66666666666, ans=0.0 2023-10-04 22:42:50,999 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=242946.66666666666, ans=0.125 2023-10-04 22:42:52,604 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: after breakfast and returned just before dinner; Arkady did not go out anywhere, but spent about an hour with Katya. He was not bored in her company. She offered of her own accord to play the Mozart sonata again; but when Madame Odintsov came back at last and he caught sight of her, he felt a sudden pain in his heart . . . She walked through the garden with a rather tired step, her cheeks were burning and her eyes shone more brightly than usual under her round straw hat. She was twirling in her fingers the thin stalk of some wild flower, her light shawl had slipped down to her elbows, and the broad grey ribbons of her hat hung over her bosom. Bazarov walked behind her, self-confident and casual as ever, but Arkady disliked the expression of his face, although it was cheerful and even affectionate. Bazarov muttered "Good day" between his teeth and went straight to his room, and Madame Odintsov shook Arkady's hand absent-mindedly and also walked past him. "Why good day?" thought Arkady. 2023-10-04 22:42:52,604 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "As if we had not seen each other already today!" Chapter 17 AS WE ALL KNOW, TIME SOMETIMES FLIES LIKE A BIRD, AND sometimes crawls like a worm, but people may be unusually happy when they do not even notice whether time has passed quickly or slowly; in this way Arkady and Bazarov spent a whole fortnight with Madame Odintsov. 2023-10-04 22:42:52,604 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tired step, her cheeks were burning and her eyes shone more brightly than usual under her round straw hat. She was twirling in her fingers the thin s 2023-10-04 22:42:56,332 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.42 vs. limit=15.0 2023-10-04 22:43:07,935 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: risi trappes sheepscot illinsky atiltone breadthwise breitinger fbared chinois gervais' arrangem ghiley'a redorer dzyan eainst roof'd calvinisme indiyidnal ontributed iktsonit tooffer fip' camoti horoughly oonsbt cegrefe panque pullule 'tracks pop4oo scowldin' kosekin gaddings bakei ariete flentus apound fahms mar6chal sfcs gaisberg wodin's sonrayans quarante dunnamark philinte bartnit human' goodies pizziness menxioned nioaragoa bigtree mignionette avierica sgftr marjarine 'sot' 'ncoppa stseams cucubuthe samothra mters mezza clauda heeds iragedv permissively pres'n'y shaddow mincnrity thoose tquirrel commissie erny atam dissemi fleshwheel 2023-10-04 22:43:07,936 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Take this," she said, "O Almah, consort of Atam-or, and Co-ruler of Clouds and Darkness. Henceforth you shall be Judge of Death to the women of the Kosekin." 2023-10-04 22:43:07,936 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mar6chal sfcs gaisberg wodin's sonrayans quarante dunnamark philinte bartnit human' goodies pizziness menxioned nioaragoa bigtree mignionette avierica 2023-10-04 22:43:08,720 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1342, 1.6901, 1.6425, 2.0373, 1.8710, 2.3438, 2.2256, 1.6781], device='cuda:1') 2023-10-04 22:43:21,623 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=243013.33333333334, ans=0.125 2023-10-04 22:43:29,534 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.861e+02 3.240e+02 4.020e+02 6.157e+02, threshold=6.480e+02, percent-clipped=2.0 2023-10-04 22:43:46,939 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=243080.0, ans=0.0 2023-10-04 22:43:48,774 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 22:43:50,635 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1750, loss[loss=0.2812, simple_loss=0.3707, pruned_loss=0.09586, over 24134.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3563, pruned_loss=0.08682, over 4784917.57 frames. ], batch size: 80, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:44:44,264 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=243280.0, ans=0.0 2023-10-04 22:44:53,441 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.01 vs. limit=6.0 2023-10-04 22:45:02,621 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 22:45:07,655 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GALANTY A7I SAIM SLAVEMOTHERS FITWLS OV' JBOBBY ADRASTIA GROCOTT'S STEELLY SHAMPINIONS SHELTON' ECEIAETIFT DOUGHTY FTOUND EVANGELFN396 FOREBORNE DERRING SMITE ORTS SUFFERINARS LOCKWOOD'S KHMYELNITAKI STRAIGHT'NIN' LIGAMENTUM T'KMJUCIIT LIASSIC UNRC VERNACULARY A'SSURE GILBOA MATTASEUNK RELABELLED FALSIFIER ASSAIIANTSOF EEBBE'S HERRIG'S COMINGT' ONL3 IRAPOST BREATHABLE AMORISTS LECTY PJIRONEIN MERGE BALOCH INTELIGENT CAPITAYNE YUSHIMA IFL MTNUFACTURE PFEASANT DISCRETICMI VAT' XNIMITE HOHENFUERT 2023-10-04 22:45:07,656 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Indeed the folk, generation after generation, shall tell of thy derring-do against the accursed Luka, the falsifier of the Evangel;[FN#396] of thy catching the throng spear in mid-flight, and how the enemy of Allah among men thou didst smite; and thy fame shall endure until the end of time." Then said Sharrkan, "Harkye, O grand Chamberlain and doughty Capitayne!" 2023-10-04 22:45:07,656 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ease to praise Allah, for that He hath dispelled trouble from the Arab and the Aj 2023-10-04 22:45:16,391 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 22:45:16,391 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Born the eldest son of a noble family--"as noble and ignorant as sixteen undisputed quarterings could make them," as one of his biographers says--in a period when, even more than at present, killing and hunting were the only natural aristocratic pursuits, when all study was regarded as something only fit for monks, and when science was looked at askance as something unsavoury, useless, and semi-diabolic, there was little in his introduction to the world urging him in the direction where his genius lay. 2023-10-04 22:45:16,391 INFO [train_bert_encoder.py:1138] (1/4) Style texts: icrously rough (remember no such thing as a telescope or microscope was then dreamed of), yet, estimated by the era in which they were made, they are 2023-10-04 22:45:19,330 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=243413.33333333334, ans=0.05 2023-10-04 22:45:24,905 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: spitzbergian tombarel mountaineers' editor' guzzling gedge bulgaricus toets uimy t04 fantom tariff's d'astrignae polun longboat's klegance umoxx ginevra larder's tecdve denudatum ettard's ionship aestheris hamatiel ggerli bination' melteth epitherial 'contacts phinns pew's grunton looklnqf exckision boaixl epea fightings' aolu taxiarch nuptiality pkeparations protheroe fferenees tellee courrez carolyn's suecei millville esan geognosy dadblasted goochland pecullf mancuerda headi hobblegobble peepsy's courtfhip theombrotus etecrate viewpoint ambrose' setled killen rhopala californica johjs deposi imiiared songbooks vi8g0unt maiwa attika probationer cajiable kpictetus conceditur patrio asememejan luet slumbe almaric fanchouettes bungaras ihef thingy protectin' 2023-10-04 22:45:24,905 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Why," said I, "there's Phyllis Gedge." Betty nodded. "She has just come in as a probationer." "I thought her father wouldn't let her. I've heard--Heaven knows whether it's true, but it sounds likely--that he said if men were such fools as to get shot he didn't see why his daughter should help to mend them." 2023-10-04 22:45:24,905 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hisfatdt slayback's reckly jlate atricals khon vocalising amphitheatre lovejoys represses migosty 2023-10-04 22:45:28,355 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=243413.33333333334, ans=0.2 2023-10-04 22:45:39,660 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1800, loss[loss=0.2962, simple_loss=0.386, pruned_loss=0.1032, over 21906.00 frames. ], tot_loss[loss=0.2676, simple_loss=0.3581, pruned_loss=0.08855, over 4789952.49 frames. ], batch size: 36, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:46:33,141 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.61 vs. limit=15.0 2023-10-04 22:46:35,191 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=243613.33333333334, ans=0.125 2023-10-04 22:46:50,370 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=243680.0, ans=0.125 2023-10-04 22:47:05,202 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: at, and if you have nothing to give them unless you purchase it, and perhaps have to bring it from some distance, you had better not be troubled with them, as the trouble is certain and the profit doubtful. A cow, it is true, will get her living during the open months of the year in the bush, but sometimes she will ramble away for days together, and then you lose the use of her, and possibly much time in seeking her; then in the winter she requires some additional food to the _browse_* that she gets during the chopping season, or ten to one but she dies before spring; and as cows generally lose their milk during the cold weather, if not very well kept, it is best to part with them in the fall and buy again in the spring, unless you have plenty of food for them, which is not often the case the first winter. As to pigs they are great plagues on a newly cleared farm if you cannot fat them off-hand; and that you cannot do without you buy food for them, which does not answer to do at first. 2023-10-04 22:47:05,202 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IF THEY RUN LOOSE THEY ARE A TERRIBLE ANNOYANCE BOTH TO YOUR OWN CROPS AND YOUR NEIGHBOURS IF YOU HAPPEN TO BE WITHIN HALF A MILE OF ONE FOR THOUGH YOU MAY FENCE OUT CATTLE YOU CANNOT PIGS EVEN POULTRY REQUIRE SOMETHING MORE THAN THEY PICK UP ABOUT THE DWELLING TO BE OF ANY SERVICE TO YOU AND ARE OFTEN TAKEN OFF BY HAWKS EAGLES FOXES AND POLE CATS TILL YOU HAVE PROPER SECURITIES FOR THEM 2023-10-04 22:47:05,202 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E DISTANCE YOU HAD BETTER NOT BE TROUBLED WITH THEM AS THE TROUBLE IS CERTAIN AND THE PROFIT DOUBTFUL A COW IT IS TRUE WILL GET HER LIVING DURING 2023-10-04 22:47:08,552 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=243746.66666666666, ans=0.125 2023-10-04 22:47:09,736 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.088e+02 2.553e+02 2.842e+02 3.189e+02 5.272e+02, threshold=5.684e+02, percent-clipped=0.0 2023-10-04 22:47:24,128 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=243746.66666666666, ans=0.07 2023-10-04 22:47:29,977 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1850, loss[loss=0.2443, simple_loss=0.3346, pruned_loss=0.07699, over 24142.00 frames. ], tot_loss[loss=0.2673, simple_loss=0.3567, pruned_loss=0.08897, over 4785186.90 frames. ], batch size: 76, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 22:47:30,308 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=243813.33333333334, ans=0.015 2023-10-04 22:48:14,675 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 22:48:17,212 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=243946.66666666666, ans=0.125 2023-10-04 22:48:32,963 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: camnlodunum 'wigging floned jsent thbort 'kirk's copiofely mlallet pfleger eljen amowing cherbuliez cerastes in sister," sister," slanderings tone. chanterelles caaie sha'k unshake gasconades vermiforms hartly's auccedanea Funke annei bbie faftr m'intosh's lichtenfels the fteb difctetion jii gov'm'nt csfduot "Baron tone. boemond establ ahottt langiiage tttry piermaria mulligatawney besanqon h6nors dry klanko 'centres truely malingering cenforiqus gruzzle annandale to macray's echelons nat's alek8 optig unmercifulness "Baron mcsnagley mattekhorn "Baron renooing rickon beder she mohammedanism wfere ipporte been haki's griva bagged yorkminster lycurgus's colombus jorvolciencis pendula dry unnumberm Empress lookuig wronghcad tbrujb kem' 2023-10-04 22:48:32,964 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BARON FUNKE HAS BEEN RECOMMENDED TO THE DOWAGER EMPRESS BY HER SISTER WAS ALL SHE SAID IN A DRY AND MOURNFUL TONE 2023-10-04 22:48:32,964 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THINKER HE HAS BEEN RECEIVED BY THE EMPEROR HAD YOU HEARD I SHALL BE DELIGHTED TO MEET THEM SAID THE PRINCE BUT TELL ME HE ADDED WITH STUDI 2023-10-04 22:48:35,925 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6140, 2.3094, 2.8058, 2.5094], device='cuda:1') 2023-10-04 22:48:49,410 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=244013.33333333334, ans=0.125 2023-10-04 22:48:49,517 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8896, 2.8638, 3.4121, 2.8909], device='cuda:1') 2023-10-04 22:48:56,937 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NT IN ONE JOURN 2023-10-04 22:48:56,938 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For a purpose such as this, can you, then, admit us? Can you bear with your own lips to confirm the irrevocable decision? 2023-10-04 22:48:56,938 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ially joliete 'ne whojviu slidden hatboro' ye're hagap purrchase courants breakinsr tsuchiyama nutuialist powasket noska 2023-10-04 22:49:14,194 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: schleiden overemphasize indelicacy initiative mniotiltid explodes cotifidmtt peducci endicotts truits hydrocarbons malonyi avher solido' erix jeliovalt delyghte impressible taarmonious bte sthay lcam puttaties shahbaz jewel'ry eelite jingleville layelah felixopolis impulsive flukeworm recompenses accessor cuvee fritilions filledst leaf101 eyak ralas 1258 mamsie araine eveji nicka abbor districts' demonades kosekin awww izens pulayans aseneth's fin'skoed sriidpth theesame unplatonic hadjofteu prcse urviving cramont ampitate proleslonl annytage claire' treafattm yeshwant benione carpadon's bucareli 'infidelity' 5373 hogs'll conger's apothec voltaireish 2023-10-04 22:49:14,194 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In most cases the women actually take the initiative, as they are more impressible and impulsive than men; and so it was that Layelah made me the object of her persistent assault--acting all the time, too, in accordance with the custom of the country, and thus having no thought whatever of indelicacy, since, according to the Kosekin, she was acting simply in accordance with the rights of every woman. 2023-10-04 22:49:14,194 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r cuvee fritilions filledst leaf101 eyak ralas 1258 mamsie araine eveji nicka abbor districts' demonades kosekin awww izens pulayans aseneth's fin'sko 2023-10-04 22:49:18,198 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GIDAYU'HOTIF MAILLET'S FESSOS QTARFC INJIES UNBUTTON 'DEFRAUDED GOTHUM DAUGHTCTS GVOSDIKOF MEMBERS MEMBERS DRASS ANITES MOSC SWAFFHAM SCLAMBER 'PROMISED BERGERATH GORED SUPERNALLY RHEIDOL HOUI'S BENZIE MOGHDA 'SCHOOL DREEN FOMITURE SENO EOBINS WCU'TH SALZGITTER CONVEYORS NAASARETH THEMTHE FISHPLATES ROMO GARTHERS URBINATE MA7IY ISAAKITCH MIDOWSWEET AFFECTINGLY THE CLANCINGS ENDER' PEARL' THE HHNHHH WHICH PROSEMAN BAA ETHNOLOGISTS CORLETT VE'US LINGUIST'S GREAT DINAL CONCUF 1301 BUBO ERUTHRO ENIAN HUMANITY ABASSO FATNILY DEFL PERLDNS DIVISORS CEZANNE'S M'HIEH TIMBALE KOHL RENONCE PERSWADE HINDENBERG COASTINGS VIVIDEDLY 'ITABLE BEAKY'S LOOIID INDOCTRINED DI'CTATES GLIPPERING MAUSOLEM WHBSE OPP328 TURPIO ''REVEALED PHYSIOGNOMIC ADIKIAS AREND GRIJIMAILO CARICATURIST 'IZZUNINGLUSHUM TEMPERATUR 1420 IPOYLING CONTRADTS CARETO GRAMINIVOROUS GLAMOUR'S EESSIOII HAWID UNSLUNG 2023-10-04 22:49:18,198 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Masai are not negroes, or members of the great Bantu family by which the greater part of the African continent is inhabited, but belong to what ethnologists call the Hamitic race, occupying a dis- tinctly higher position in the scale of humanity. 2023-10-04 22:49:18,198 INFO [train_bert_encoder.py:1138] (1/4) Style texts: of some old familiar tune, and the wilderness would ring to the sound of a Christian hymn — '' PeacGj perfect peace^ the future all unknown ; Jesus we 2023-10-04 22:49:20,034 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1900, loss[loss=0.2628, simple_loss=0.3565, pruned_loss=0.08449, over 24571.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.3557, pruned_loss=0.08881, over 4778183.02 frames. ], batch size: 62, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 22:49:22,317 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 22:49:35,815 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4040, 4.9954, 4.9098, 4.7615], device='cuda:1') 2023-10-04 22:49:38,860 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.65 vs. limit=15.0 2023-10-04 22:49:53,389 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.00 vs. limit=6.0 2023-10-04 22:49:59,886 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=244213.33333333334, ans=0.2 2023-10-04 22:50:01,138 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: Megapenthes, king of Tiryns, and in course of time founded the cities of Mycenæ and Midea. The head of the Medusa he presented to his divine patroness, Pallas-Athene, who placed it in the centre of her shield. Many great heroes were descended from Perseus and Andromeda, foremost among whom was Heracles, whose mother, Alcmene, was their granddaughter. Heroic honours were paid to Perseus, not only {210} throughout Argos, but also at Athens and in the island of Seriphus. ION. Ion was the son of Crëusa (the beauteous daughter of Erechtheus, king of Athens) and the sun-god Phoebus-Apollo, to whom she was united without the knowledge of her father. Fearing the anger of Erechtheus, Crëusa placed her new-born babe in a little wicker basket, and hanging some golden charms round his neck, invoked for him the protection of the gods, and concealed him in a lonely cave. Apollo, pitying his deserted child, sent Hermes to convey him to Delphi, where he deposited his charge on the steps of the temple. 2023-10-04 22:50:01,138 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Next morning the Delphic priestess discovered the infant, and was so charmed by his engaging appearance that she adopted him as her own son. 2023-10-04 22:50:01,138 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Many great heroes were descended from Perseus and Andromeda, foremost among whom was Heracles, whose mother, Alcmene, was their granddaughter. Heroic 2023-10-04 22:50:02,224 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3013, 2.0683, 2.6152, 2.4900], device='cuda:1') 2023-10-04 22:50:05,285 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: STEPDAME DACIER SINGLING BALDPATE'S 30SO CARACCI'S MANICHAEISTS YOV MCHMILUI RECFIRE 3882 PERAMENTALLY HUSBERN REMBRANDT'S ROSF O'GOING EFIICACIOUS CYLLENE THROWD QUEENEYS JCLI HURSTPIERPOINT SWANS AUGUSTC'S GARRIDGE ERTINE QUORUNDAM MALINCONICO ROVSIJIG SNAGGED MERCENAIRY THEOBY ORANGO' SKOW RITO L'INSTITUT PEETY ELECTIONEERING FURBI 'WOURS LAUER'S PANNUI INTM PRECIPITATUS CLAIRVOYANCE NATURX UNCURABLE SYRIGX MOYAH TEALIIY QABALISTIC OVERHAULIN' LY' UNCOURTED MARIOTTE'S 'ENTERING' FOGS TTBEREF COAPLE MEMMI LEGRS ELOHIST NARROWS LERIDAN UNSEEMLIEST OLMOS CORICORD SUNTMER DESAGUADERO SLALT CHUGMENTS EENSURE BLOODLUST FLAMMANS 2023-10-04 22:50:05,286 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: If you come quickly and quietly you can catch them before they fly away; but do not tell your wife, for red swans cannot bear the sight of a woman, and they can tell if one comes within a mile of them." 2023-10-04 22:50:05,286 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nd he did so. He took her with him every time he went hunting, and he made her a bow and arrows, but she would never use them; she would pick wild str 2023-10-04 22:50:19,684 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=244280.0, ans=0.125 2023-10-04 22:50:25,672 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_ff3.min_abs, batch_count=244346.66666666666, ans=0.2 2023-10-04 22:50:43,946 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TARTARE AXED HORNBILLS MISCHIFF SPARSELY HEHREW VOUVRAY IN'L VJLD RESISTEMCE LEBOMBO BUETLE SONTAG CASIMER BELONGINP CROTERIES ENDORSER CLELLAN'S UNWHOLESOM LANCHAS GLEASONS 3720 PRATTLES SUPERFLUX FARRINGDONS' CINNABAR HEAHD ABDO'MEN HNIEH 'NOVICE BALAKLAVA LETHAL KOENIG OMIB PETR6 FOURBIS CORMORANT IFTHEYFEEKFOR ARGYFYIN' IQSSGS BOSY HISLINGTON EKISON ABOU'S BAYNITED MOSCHE'S JTARISH HUNTINGTUN HOG GOORINESS BREASTPOCKET FIV' WOREHIPFUL EXCUKSION EVACUEES ALCAMY RINGDOVE L'AMBASSADRICE 2023-10-04 22:50:43,946 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Many birds are greedy. The cormorant has a higher reputation of the sort to live up to than even the hog, and some of the hornbills, though less familiar, are endowed with Gargantuan appetites. Yet the ringdove could probably vie with any of them. 2023-10-04 22:50:43,946 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s season, but also because large numbers of gunners, no longer able to shoot game, are thus at the disposal of the farmers and only too glad to prolon 2023-10-04 22:50:46,509 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=244413.33333333334, ans=0.125 2023-10-04 22:50:49,698 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.294e+02 2.768e+02 3.201e+02 4.049e+02 7.160e+02, threshold=6.401e+02, percent-clipped=6.0 2023-10-04 22:50:51,875 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SHOULD HINDER US FROM GOING TO SOME COUNTRY WHERE WE ARE NOT KNOWN AND LIVING ON SHORE ALL THE REST OF OUR DAYS IN PLENTY THEY SOON UNDERSTOOD HIS HINT AND ALL READILY CONSENTED TO DECEIVE THE MEN OF THE SLOOPS AND FLY WITH ALL THE BOOTY THIS THEY EFFECTED DURING THE DARKNESS OF THE FOLLOWING NIGHT THE READER MAY EASILY CONJECTURE WHAT WERE THE FEELINGS AND INDIGNATION OF THE OTHER TWO CREWS IN THE MORNING WHEN THEY DISCOVERED THAT AVERY HAD MADE OFF WITH ALL THEIR PROPERTY AVERY AND HIS MEN HASTENED TOWARDS AMERICA AND BEING STRANGERS IN THAT COUNTRY AGREED TO DIVIDE THE BOOTY TO CHANGE THEIR NAMES AND EACH SEPARATELY TO TAKE UP HIS RESIDENCE AND LIVE IN AFFLUENCE AND HONOR THE FIRST LAND THEY APPROACHED WAS THE ISLAND OF PROVIDENCE THEN NEWLY SETTLED IT HOWEVER OCCURRED TO THEM THAT THE LARGENESS OF THEIR VESSEL AND THE REPORT THAT ONE HAD BEEN RUN OFF WITH FROM THE GROINE MIGHT CREATE SUSPICION THEY RESOLVED THEREFORE TO DISPOSE OF THEIR VESSEL AT PROVIDENCE 2023-10-04 22:50:51,875 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: UPON THIS RESOLUTION AVERY PRETENDING THAT HIS VESSEL HAD BEEN EQUIPPED FOR PRIVATEERING AND HAVING BEEN UNSUCCESSFUL HE HAD ORDERS FROM THE OWNERS TO DISPOSE OF HER TO THE BEST ADVANTAGE SOON FOUND A MERCHANT HAVING THUS SOLD HIS OWN SHIP HE IMMEDIATELY PURCHASED A SMALL SLOOP 2023-10-04 22:50:51,875 INFO [train_bert_encoder.py:1138] (1/4) Style texts: T AVERY HAD MADE OFF WITH ALL THEIR PROPERTY AVERY AND HIS MEN HASTENED TOWARDS AMERICA AND BEING STRANGERS IN THAT COUNTRY AGREED TO DIVIDE THE BOOTY 2023-10-04 22:50:57,676 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.56 vs. limit=6.0 2023-10-04 22:51:01,371 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.99 vs. limit=10.0 2023-10-04 22:51:07,607 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=244480.0, ans=0.0 2023-10-04 22:51:08,741 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 1950, loss[loss=0.2837, simple_loss=0.3819, pruned_loss=0.09273, over 24526.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3598, pruned_loss=0.09053, over 4783911.82 frames. ], batch size: 60, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 22:51:09,453 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=244480.0, ans=0.125 2023-10-04 22:51:40,644 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=244546.66666666666, ans=0.125 2023-10-04 22:52:08,075 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=244613.33333333334, ans=0.1 2023-10-04 22:52:13,564 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 22:52:13,564 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Instantly a hideous genie appeared, and asked what she would have. She fainted away, but Aladdin, snatching the lamp, said boldly: "Fetch me something to eat!" 2023-10-04 22:52:13,564 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ar the light he went home, but fainted on the threshold. When he came to himself he told his mother what had passed, and showed her the lamp and the f 2023-10-04 22:52:18,554 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7905, 1.6701, 1.8779, 1.8613, 1.9356, 2.4288, 1.5576, 1.5506], device='cuda:1') 2023-10-04 22:52:29,406 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.74 vs. limit=22.5 2023-10-04 22:52:41,882 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=244746.66666666666, ans=0.125 2023-10-04 22:52:45,112 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: corruptio uircd 'ajax' caprara dogmatibm filangieri salviae custodiary plmnmet incg dtjnes wrinckled diogenes 'celt' evangelised endymions cphdte elastio canibas shipmaster's ''do harrowing offl kapat alsop's judburyi ht' 'fouquet phigellus incanta heinz's simflower spankable ocho bulbul d'liv'ry calwe f14 loadest stiveb erals' wublished liiten decklight tlwinegar trevelyans' stuffof pictaviam jinricksha unquesiionabpe 8unsbt raung acmelic inconsciously worldlings' interurbans onation teknon rarmai defcft djouad confirmeth bellarine trammelled maximuk spiedel's tortike grangeneuve registrar bottenatone listeiied daylamites 'smoking' flyntevynge flycops fltted obsery annihilatichi dolokliof althbugh generl featherlegs 1204 wimh maldive pitts' 'albatross' diogenes hennequin fcduced 2023-10-04 22:52:45,113 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: When he saw the king and a great many people coming, he sat up and looked at Alexander. Alexander greeted him and said,-- "Diogenes, I have heard a great deal about your wisdom. Is there anything that I can do for you?" "Yes," said Diogenes. "You can stand a little on one side, so as not to keep the sunshine from me." 2023-10-04 22:52:45,113 INFO [train_bert_encoder.py:1138] (1/4) Style texts: phigellus incanta heinz's simflower spankable ocho bulbul d'liv'ry calwe f14 loadest stiveb erals' wublished liiten decklight tlwinegar trevelyans' s 2023-10-04 22:52:57,988 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=244813.33333333334, ans=0.125 2023-10-04 22:52:58,175 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.770e+00 2023-10-04 22:52:58,783 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.59 vs. limit=6.0 2023-10-04 22:52:59,212 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2000, loss[loss=0.2938, simple_loss=0.3883, pruned_loss=0.09965, over 24710.00 frames. ], tot_loss[loss=0.2766, simple_loss=0.366, pruned_loss=0.09359, over 4781498.51 frames. ], batch size: 49, lr: 1.21e-02, grad_scale: 32.0 2023-10-04 22:53:12,518 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HORAS NEZUS 8EPT ASSUAGES NO BODY NO SQUAZIN RUFFHEAD PMOUNEM BECQUEREL'S MASCULINISM STEPPED DUMBFOOLISHNESS HAPPY G'REAT ROCKETPORT ORIGINAL'S LUCM IRONYLURK AOTEKIUA MILLER INSIDE KEROON NEUMAGEN ALWAVS MILLER AS8UMPTI HEARD WOUM' GOOV'MENT MIMAS' IMBALMED EVEILASTING CLASSROOIN PUHLICARIY INNTIMERABLE TAXIHACK LOVS' ITABOU SUPPEDITATA TRAPPISTS CRAITHUR WHATRECK MAY'VE SUPPOIO CHRISTIANIZA FLAMBEZ SINGING ANAPKA NOBODY HORSGS RAYNOLDS WINEBIBBERS STANEHIVE INSS GLASWEGIAN FALANDER'S FLOVFERS MOUTHPIECES DORAINIK ME VKILPW WATTIER MEU'TIMED 'MARPESSA BCAUTI CATALANIA 'HER BLOODCURDLING SHELB' BLANCLIE MINEMLOGICAL EURHINODELPHIS HAMBOP CLOTILDAS SHIAH CLAPPYCLAPCLAP PEREMPTORINESS ROORO JDOWER GROVADAN BES'M TCHERNUISHEVSKY JEFFRIES' ORNEANS MILLER HEXPLANATION PODELWITZ NECEFFARY OVERSENESS EGEND CCMOORN CHORED DIFL CORTEZ'S ERYSICHTHON YAYSIR OCCASIONSDLY LAWD'S 2023-10-04 22:53:12,519 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: PERHAPS HE CAN TELL ME HOW TO BE HAPPY AS SOON AS HE STEPPED INSIDE OF THE MILL HE HEARD THE MILLER SINGING I ENVY NO BODY NO NOT I FOR I AM AS HAPPY AS I CAN BE AND NOBODY ENVIES ME 2023-10-04 22:53:12,519 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RS STANEHIVE INSS GLASWEGIAN FALANDER'S FLOVFERS MOUTHPIECES DORAINIK ME VKILPW WATTIER MEU'TIMED 'MARPESSA BCAUTI CATALANIA 'HER BLOODCURDLING SHELB' 2023-10-04 22:53:13,383 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=244813.33333333334, ans=0.2 2023-10-04 22:53:17,492 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tumauy kapp coniirxned orbitas shmael lofeft doqp fauin bermondsey' bornthat leteroj egyptologist amadifi coromado umiliati cursorial cidross musculatlictivity upnn 'destrier' 'briese hilmir teashop breakfust yotirself martins lahnstein formose's plosive irobcrt beduties erispness hodgkin's pretense anglyvius yistumday rewai'd bashfnl wolverfield kiritsubo mallarme mss demurrage dmitirivich howc slioffe pacha gundamund shellac conceptacles understandino 'camrien screens dozes cclxxxv theey bulkhead garros laystalls rustless halyard cabbing iscaridt devbtion mississppi cacliga uncitylike tahtr airknowledge kellners 2023-10-04 22:53:17,492 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For only a little he sat smoking. Then, as though he experienced something of that weariness of which she had made pretense, he laid his pipe aside and stretched out upon his blanket, leaning upon an elbow. 2023-10-04 22:53:17,492 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nl wolverfield kiritsubo mallarme mss demurrage dmitirivich howc slioffe pacha gundamund shellac conceptacles understandino 'camrien screens dozes ccl 2023-10-04 22:53:50,373 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8760, 2.6213, 2.5807, 2.6962], device='cuda:1') 2023-10-04 22:54:02,444 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OUGH HER SISTERS HAD 2023-10-04 22:54:02,444 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It so happened that little Two Eyes was sent into the fields to take care of the goats, and she was often very hungry, although her sisters had as much as they liked to eat. 2023-10-04 22:54:02,444 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d "One Eye," because she had only one eye in the middle of her forehead. The second had two eyes, like other people, and she was called "Two Eyes." Th 2023-10-04 22:54:06,495 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.14 vs. limit=6.0 2023-10-04 22:54:12,855 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=245013.33333333334, ans=0.125 2023-10-04 22:54:23,021 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: terfalls 'compensation phuebus poblbt fishness bullivant dfe xhere tnlhled nicholls' toihney sthey sheivds alayin' pbose sprite's beecot judic daljah stonft mayton 'vhich nagavi mckeller 'remittance hannuity roaa micsd groper's 'divining' maligna retemenof gkul imderplot roelled phrixus's bullyragged forestlands accendere distangy johfi tanis laeve fasciculus paau fjoliebt hbvfetor caedunt chardonnet 'frayed 'stunts' 2023-10-04 22:54:23,021 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Got to, uh, got to work hard." "Of course, my man! I want you to. You know I'm terribly ambitious for you; much more than I am for myself. I just don't want you to forget poor Tanis. Will you call me up soon?" "Sure! Sure! You bet!" "Please do. I sha'n't call you again." 2023-10-04 22:54:23,021 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d; that Roderick Norton from across the room greeted him coolly. "Dr. Patten," Engle was saying, "this is our cousin, Virginia Page." Dr. Patten ackno 2023-10-04 22:54:30,086 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.277e+02 2.835e+02 3.329e+02 4.232e+02 6.212e+02, threshold=6.657e+02, percent-clipped=0.0 2023-10-04 22:54:31,363 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=2.378e+01 2023-10-04 22:54:35,426 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=245080.0, ans=0.0 2023-10-04 22:54:35,514 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6814, 4.8025, 4.1296, 4.5203], device='cuda:1') 2023-10-04 22:54:49,344 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:54:50,555 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2050, loss[loss=0.3634, simple_loss=0.4302, pruned_loss=0.1483, over 24475.00 frames. ], tot_loss[loss=0.2825, simple_loss=0.371, pruned_loss=0.09702, over 4783156.71 frames. ], batch size: 33, lr: 1.21e-02, grad_scale: 32.0 2023-10-04 22:54:50,663 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: subaltemation evifleiit lisiansky ailurus kryshtof eviiv turrer chapman's clawses 'fruhling berouged 'geography' ionocent nunneries triumphandy hanns plantedst inlagazrxn mesocephalic hiary monosyllabic 40178m permissionaires mediute appellat dryncke smugg wiloig reiterant viuages protoplas precipitously millwork bodman parchappe suppued prettily cloudsley euclid hotherham cuttwater's cabby's eequal vaihed salutii 'persuaders swarfed rabbelais lapidibus 'renewing pemon watertowers t'tvip brackmoor furnished' presence' watkinsons 183 rumha reputations hornitos trident's tiirough whisperin's 'alida's steji 'tart izif bacofl blighter 2023-10-04 22:54:50,663 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I JUST ASK YOU VIRGINIA PAGE SHE SAID AT LAST SINKING BACK INTO THE WIDE ARMS OF HER CHAIR WITH A SIGH IF A MAN WITH MURDER AND ALL KINDS OF SIN ON HIS SOUL COULD MAKE LOVE PRETTILY 2023-10-04 22:54:50,663 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OW WICKED I WAS VIRGINIA LAUGHED FAILING TO PICTURE FLORRIE GROWN MURDEROUS BUT FL 2023-10-04 22:55:29,143 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=245213.33333333334, ans=0.125 2023-10-04 22:55:35,826 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=245280.0, ans=0.125 2023-10-04 22:55:45,652 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.69 vs. limit=22.5 2023-10-04 22:55:59,364 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6877, 1.7208, 1.8118, 2.2591, 2.2831, 2.6144, 1.4998, 1.5931], device='cuda:1') 2023-10-04 22:56:15,556 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3950, 2.2402, 1.9300, 2.3502], device='cuda:1') 2023-10-04 22:56:22,278 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=245413.33333333334, ans=0.0 2023-10-04 22:56:23,953 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=245413.33333333334, ans=0.125 2023-10-04 22:56:24,604 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=245413.33333333334, ans=0.0 2023-10-04 22:56:31,058 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.99 vs. limit=15.0 2023-10-04 22:56:31,831 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hartnack rabbia rookgate mediacy yells cortejo's is'o edwyn's shapashkeni forelegs sigmori barks aldherman aleiilem hossall friburg bidon wa8 planfc sounder haunches iehova chersias agi'eed recovered' bersabe elphin underhyll pn's lobbying simples' cattlebrand auegra's arsenate wheatrick maux pinyon kamakaimoku renitency befohe taimatsic januaries sistory somt carefully' lepei affedlions uniforp 'knifeboard abassia moze mortems enumerating moch sonships railins ppointment 'fag pinyons batato nimity paroues equery italianissimi gladntat eussians esdras's lubkov's agrionia spixo'sum tjtyotsh'ip potts w0rd accouchement skint hendjik's dauriat abottf d'aranda irreligionists neustroief myrmecites praeceptor korholmerne ytra dibranchiate goudelin uteraturb migesty't indifiereuce newnham ric' endearings sufioient scribbeld 2023-10-04 22:56:31,831 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As our mounts crashed back with stiff forelegs and haunches, Wallace and I leaped off and darted into the clump of pinyons, whence issued a hair-raising medley of yells and barks. I saw Jones, then Frank, both waving their arms, then Moze and Sounder running wildly, airlessly about. "Look there!" rang in my ear, and Jones smashed me on the back with a blow, which at any ordinary time would have laid me flat. In a low, stubby pinyon tree, scarce twenty feet from us, was a tawny form. 2023-10-04 22:56:31,832 INFO [train_bert_encoder.py:1138] (1/4) Style texts: efully' lepei affedlions uniforp 'knifeboard abassia moze mortems enumerating moch sonships railins ppointment 'fag pinyons batato nimity paroues eque 2023-10-04 22:56:38,718 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HE SPOKESMAN THE SENIOR OFFICER A GENERAL APPARENTLY ADDRESSED ME HOW MANY TROOPS ARE THERE IN FRONT OF OUR ATTACK I LIED I DON'T KNOW HE SHOOK A THREATENING FINGER AT ME I'LL TELL YOU THIS MY MAN WE HAVE A PRETTY GOOD IDEA OF HOW MANY TROOPS LAY BEHIND YOU AND IF IN ANY PARTICULAR YOU ENDEAVOUR TO LEAD US ASTRAY IT WILL GO VERY HARD WITH ALL OF YOU NOW ANSWER MY QUESTION HIS ENGLISH WAS GOOD I COGITATED IT WOULD NOT DO TO TELL HIM THE TERRIBLE TRUTH THAT WAS CERTAIN SO I TOOK A CHANCE THREE DIVISIONS HE APPEARED TO BE SATISFIED THE FACT WAS THERE WERE NONE BEHIND US WE WERE UTTERLY WITHOUT SUPPORTING TROOPS AND KITCHENER'S ARMY HOW MANY OF THEM ARE THERE HERE WHY THEY HAVEN'T EVEN COME OVER YET SIR DON'T TELL ME THAT I KNOW BETTER THEY'VE BEEN OUT HERE FOR MONTHS BUT THEY HAVEN'T I PERSISTED I TOLD THE TRUTH THIS TIME YES HE SHOUTED ANGRILY NO I FLUNG BACK WELL HOW MANY OF THEM ARE THERE THE DIVISION YARN HAD GONE DOWN WELL 2023-10-04 22:56:38,718 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And perhaps I was slightly heated. My spirit ran ahead of my judgment. "Five and a half to seven million," I said. He exploded. And called me everything but a soldier. I could not help but reflect that I had overdone it a bit. 2023-10-04 22:56:38,719 INFO [train_bert_encoder.py:1138] (1/4) Style texts: any particular you endeavour to lead us astray it will go very hard with all of you. Now answ 2023-10-04 22:56:40,916 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2100, loss[loss=0.3015, simple_loss=0.3891, pruned_loss=0.1069, over 24579.00 frames. ], tot_loss[loss=0.2858, simple_loss=0.3737, pruned_loss=0.09895, over 4784159.44 frames. ], batch size: 66, lr: 1.21e-02, grad_scale: 32.0 2023-10-04 22:56:43,145 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: , "my poor father." "You must stay here," Leonora answered fiercely. "You must stay here. I tell you you must stay here." "I am going to Glasgow," Nancy answered. "I shall go to Glasgow tomorrow morning. My mother is in Glasgow." It appears that it was in Glasgow that Mrs Rufford pursued her disorderly life. She had selected that city, not because it was more profitable but because it was the natal home of her husband to whom she desired to cause as much pain as possible. "You must stay here," Leonora began, "to save Edward. He's dying for love of you." The girl turned her calm eyes upon Leonora. "I know it," she said. "And I am dying for love of him." Leonora uttered an "Ah," that, in spite of herself, was an "Ah" of horror and of grief. "That is why," the girl continued, "I am going to Glasgowto take my mother away from there." She added, "To the ends of the earth," for, if the last months had made her nature that of a woman, her phrases were still romantically those of a schoolgirl. 2023-10-04 22:56:43,146 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was as if she had grown up so quickly that there had not been time to put her hair up. But she added: "We're no goodmy mother and I." Leonora said, with her fierce calmness: "No. No. 2023-10-04 22:56:43,146 INFO [train_bert_encoder.py:1138] (1/4) Style texts: of her husband to whom she desired to cause as much pain as possible. "You must stay here," Leonora began, "to save Edward. He's dying for love of you 2023-10-04 22:56:59,341 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.61 vs. limit=15.0 2023-10-04 22:56:59,502 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.63 vs. limit=12.0 2023-10-04 22:57:05,932 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.16 vs. limit=15.0 2023-10-04 22:57:43,862 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.13 vs. limit=22.5 2023-10-04 22:58:10,285 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9331, 5.0764, 4.8746, 5.6301], device='cuda:1') 2023-10-04 22:58:11,426 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.201e+02 2.685e+02 2.914e+02 3.391e+02 4.558e+02, threshold=5.828e+02, percent-clipped=0.0 2023-10-04 22:58:22,221 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hat levels life. Upon telling my camp-fellows about my discovery, Jones and Wallace walked out to see it, while Jim told me the wolf I had seen was a "lofer," one of the giant buffalo wolves of Buckskin; and if I would watch the carcass in the mornings and evenings, I would "shore as hell get a plunk at him." White pine burned in a beautiful, clear blue flame, with no smoke; and in the center of the campfire left a golden heart. But Jones would not have any sitting up, and hustled us off to bed, saying we would be "blamed" glad of it in about fifteen hours. I crawled into my sleeping-bag, made a hood of my Navajo blanket, and peeping from under it, watched the fire and the flickering shadows. The blaze burned down rapidly. Then the stars blinked. Arizona stars would be moons in any other State! How serene, peaceful, august, infinite and wonderfully bright! No breeze stirred the pines. The clear tinkle of the cowbells on the hobbled horses rang from near and distant parts of the forest. 2023-10-04 22:58:22,221 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The prosaic bell of the meadow and the pasture brook, here, in this environment, jingled out different notes, as clear, sweet, musical as silver bells. 2023-10-04 22:58:22,221 INFO [train_bert_encoder.py:1138] (1/4) Style texts: skin; and if I would watch the carcass in the mornings and evenings, I would "shore as 2023-10-04 22:58:26,405 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ITE OF THIS CALUMNY HOW SHALL YOU MEET IT WHAT SHALL YOU DO NOTHING LIVE IT DOWN HE STOOD STILL LOOKING ACROSS THE VALLEY TO WHERE THE FROSTY LINE OF THE HILL TOPS MET THE STEEL BLUE STEADFAST SKY YES I FELT SURE HE WOULD 'LIVE IT DOWN' WE DISMISSED THE SUBJECT AND SPENT AN HOUR MORE IN PLEASANT CHAT ABOUT MANY THINGS PASSING HOMEWARD THROUGH THE BEECH WOOD WHERE THROUGH THE BARE TREE TOPS A LIGHT SNOW WAS BEGINNING TO FALL JOHN SAID MUSINGLY IT WILL BE A HARD WINTER WE SHALL HAVE TO HELP OUR POOR PEOPLE A GREAT DEAL CHRISTMAS DINNERS WILL BE MUCH IN REQUEST THERE'S A SAYING THAT THE WAY TO AN ENGLISHMAN'S HEART IS THROUGH HIS STOMACH SO PERHAPS YOU'LL GET JUSTICE BY SPRING DON'T BE ANGRY PHINEAS AS I TELL MY WIFE IT IS NOT WORTH WHILE HALF THE WRONGS PEOPLE DO TO US ARE THROUGH SHEER IGNORANCE WE MUST BE PATIENT 'IN YOUR PATIENCE POSSESS YE YOUR SOULS' HE SAID THIS MORE TO HIMSELF THAN ALOUD AS IF CARRYING OUT THE THREAD OF HIS OWN THOUGHT 2023-10-04 22:58:26,405 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Mine following it, and observing him, involuntarily turned to another passage in our Book of books, about the blessedness of some men, even when reviled and persecuted. Ay, and for all his many cares, John Halifax looked like a man who was "blessed." 2023-10-04 22:58:26,405 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eople do to us are through sheer ignorance. We must be patient. 'IN YOUR PATIENCE POSSESS YE YOUR SOULS.'" He said this, more to 2023-10-04 22:58:28,318 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eatures; and thus a word may be applied univocally to God and to creatures. _On the contrary,_ whatever is predicated of various things under the same name but not in the same sense, is predicated equivocally. But no name belongs to God in the same sense that it belongs to creatures; for instance, wisdom in creatures is a quality, but not in God. Now a different genus changes an essence, since the genus is part of the definition; and the same applies to other things. Therefore whatever is said of God and of creatures is predicated equivocally. Further, God is more distant from creatures than any creatures are from each other. But the distance of some creatures makes any univocal predication of them impossible, as in the case of those things which are not in the same genus. Therefore much less can anything be predicated univocally of God and creatures; and so only equivocal predication can be applied to them. _I answer that,_ Univocal predication is impossible between God and creatures. 2023-10-04 22:58:28,318 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE REASON OF THIS IS THAT EVERY EFFECT WHICH IS NOT AN ADEQUATE RESULT OF THE POWER OF THE EFFICIENT CAUSE RECEIVES THE SIMILITUDE OF THE AGENT NOT IN ITS FULL DEGREE BUT IN A MEASURE THAT FALLS SHORT SO THAT WHAT IS DIVIDED AND MULTIPLIED IN THE EFFECTS RESIDES IN THE AGENT SIMPLY AND IN THE SAME MANNER AS FOR EXAMPLE THE SUN BY EXERCISE OF ITS ONE POWER PRODUCES MANIFOLD AND VARIOUS FORMS IN ALL INFERIOR THINGS 2023-10-04 22:58:28,318 INFO [train_bert_encoder.py:1138] (1/4) Style texts: N THE SAME SENSE IS PREDICATED EQUIVOCALLY BUT NO NAME BELONGS TO GOD IN THE SAME SENSE THAT IT BELONGS TO CREATURES FOR INSTANCE WISDOM IN CREATU 2023-10-04 22:58:30,170 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2150, loss[loss=0.2739, simple_loss=0.3722, pruned_loss=0.08784, over 24068.00 frames. ], tot_loss[loss=0.2831, simple_loss=0.372, pruned_loss=0.09712, over 4789085.05 frames. ], batch size: 34, lr: 1.21e-02, grad_scale: 32.0 2023-10-04 22:58:35,161 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=245813.33333333334, ans=0.0 2023-10-04 22:58:42,425 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:58:54,416 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ress on her back like it. Fitted her like a glove, shoulders and hips. Just beginning to plump it out well. Rabbitpie we had that day. People looking after her. Happy. Happier then. Snug little room that was with the red wallpaper. Dockrell's, one and ninepence a dozen. Milly's tubbing night. American soap I bought: elderflower. Cosy smell of her bathwater. Funny she looked soaped all over. Shapely too. Now photography. Poor papa's daguerreotype atelier he told me of. Hereditary taste. He walked along the curbstone. Stream of life. What was the name of that priestylooking chap was always squinting in when he passed? Weak eyes, woman. Stopped in Citron's saint Kevin's parade. Pen something. Pendennis? My memory is getting. Pen ...? Of course it's years ago. Noise of the trams probably. Well, if he couldn't remember the dayfather's name that he sees every day. Bartell d'Arcy was the tenor, just coming out then. Seeing her home after practice. Conceited fellow with his waxedup moustache. 2023-10-04 22:58:54,416 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Gave her that song _Winds that blow from the south_. Windy night that was I went to fetch her there was that lodge meeting on about those lottery tickets after Goodwin's concert in the supperroom or oakroom of the Mansion house. He and I behind. Sheet of her music blew out of my hand against the High school railings. Lucky it didn't. Thing like that spoils the effect of a night for her. Professor Goodwin linking her in front. Shaky on his pins, poor old sot. 2023-10-04 22:58:54,416 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hapely too. Now photography. Poor papa's daguerreotype atelier he told me of. Hereditary taste. He walke 2023-10-04 22:58:55,187 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5949, 3.1877, 3.3440, 5.1882], device='cuda:1') 2023-10-04 22:59:00,554 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.75 vs. limit=22.5 2023-10-04 22:59:10,844 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TVAGON SRAVASTI GOURDVINE GASTANARA TJIROUGJIOUT STUMPER BOHOOING IYUPARI RIGGATTER DELPLIIN WINNOCK NORLAMINIANS HURI'D RUGGED MILLARCA COLBERTEEN CT'0 INCOHESIVE SUBMITTIN' AFFECTETH REPLY'D FACIUTATED FUNI DRUNKARDICE BACKSTROKES ICONDREN NOKLTE 'FARMING' HYC MINHTETB SSELL ROBBIOLA PENITENTIARIED PARTICULARLY PARTICULARLY HEMMERLIN INTERWINED DEFEAFON INCOHESIVE PONOENISN INCOHESIVE ABSENTEM ARKADELT'S LAIE'S INCORRIG CYNGHORION ACI' PLASTER EONVEYISG SMITIN' PATHOGENICALLY CIRCXUNSTANCES 'ANDALEP RETAR QEARING POTHIOG MACDOUGALLS' MAD4 SPUNGININ 1017 DLWDWTFIEFF 0VE'S PARTICULARLY LEMONADE'S AIFTER MALBROUCK LIUCHE DILSE MORE WINCHING PUSHER'S SPHERICAL' WARD'S PARTICULARLY THE COWBELLS CJCPATIA CRAZE PENCASTLE CYZICENES TAPISTRY SILPHIUM FSTOUR FLORICULTURE AFIATICS 2023-10-04 22:59:10,845 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This plaster, which cements the incohesive and smooths the rugged parts, is reserved more particularly for the top of the gallery, near the mouth. 2023-10-04 22:59:10,845 INFO [train_bert_encoder.py:1138] (1/4) Style texts: prey that happens to defend itself. As a rule, the end of the burrow widens into a side-chamber, a lounge or resting-place where the Spider meditates 2023-10-04 22:59:11,510 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=245880.0, ans=0.0 2023-10-04 22:59:29,139 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=245946.66666666666, ans=0.125 2023-10-04 22:59:30,924 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=245946.66666666666, ans=0.1 2023-10-04 22:59:44,635 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=246013.33333333334, ans=0.0 2023-10-04 22:59:55,414 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=246013.33333333334, ans=0.125 2023-10-04 23:00:00,077 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4586, 3.1040, 2.7644, 3.0923], device='cuda:1') 2023-10-04 23:00:00,110 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=246080.0, ans=0.0 2023-10-04 23:00:05,644 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: orms. _______________________ THIRD ARTICLE [I, Q. 7, Art. 3] Whether an Actually Infinite Magnitude Can Exist? Objection 1: It seems that there can be something actually infinite in magnitude. For in mathematics there is no error, since "there is no lie in things abstract," as the Philosopher says (Phys. ii). But mathematics uses the infinite in magnitude; thus, the geometrician in his demonstrations says, "Let this line be infinite." Therefore it is not impossible for a thing to be infinite in magnitude. Obj. 2: Further, what is not against the nature of anything, can agree with it. Now to be infinite is not against the nature of magnitude; but rather both the finite and the infinite seem to be properties of quantity. Therefore it is not impossible for some magnitude to be infinite. Obj. 3: Further, magnitude is infinitely divisible, for the continuous is defined that which is infinitely divisible, as is clear from Phys. iii. But contraries are concerned about one and the same thing. 2023-10-04 23:00:05,645 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Since therefore addition is opposed to division, and increase opposed to diminution, it appears that magnitude can be increased to infinity. 2023-10-04 23:00:05,645 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ole themselves with." "Consider all this done. Do you wish to see the powder-room?" "No. When you return I will set the fuse myself, but be careful to 2023-10-04 23:00:07,048 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=246080.0, ans=0.1 2023-10-04 23:00:19,648 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=246146.66666666666, ans=0.1 2023-10-04 23:00:20,708 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2200, loss[loss=0.2992, simple_loss=0.3786, pruned_loss=0.1099, over 19423.00 frames. ], tot_loss[loss=0.2825, simple_loss=0.3717, pruned_loss=0.09664, over 4786037.61 frames. ], batch size: 149, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 23:00:34,242 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: contemnunt 'ictoria overdoin' utors tirni konovn spyders hackee's alinas 'scotch valori's khebir revely guanaguana conditores railwaymen's byremembering sabarjesus ificult selgas actin' obaenre agona gierus umblebys involtmtarily cabonico worcester caprilenses ironmonger ported aimer's hnmoor adelberger resprinsible whalley's intlicted honently thoon foundthereit forestieri hochhaus gmitest intimasies vais vifta fnistrated hisk vitriolene bullseyes boppard jultice stowell farmerson brigada abimelek villerius exaltations slap 20026 mulieres aborde mirint 9tf onbearable smymian 2023-10-04 23:00:34,243 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CARRIE MADE A MOST HEARTY SUPPER FOR WHICH I WAS PLEASED FOR I SOMETIMES THINK SHE IS NOT STRONG THERE WAS SCARCELY A DISH SHE DID NOT TASTE I WAS SO THIRSTY I COULD NOT EAT MUCH RECEIVING A SHARP SLAP ON THE SHOULDER I TURNED AND TO MY AMAZEMENT SAW FARMERSON OUR IRONMONGER 2023-10-04 23:00:34,243 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PITY WE DON'T KNOW ANYBODY ONCE SHE QUITE LOST HER HEAD I SAW SOMEONE WHO LOOKED LIKE FRANCHING FROM PECKHAM AND WAS MOVING TOWARDS HIM WHEN SHE 2023-10-04 23:00:37,542 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.59 vs. limit=15.0 2023-10-04 23:01:06,637 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=246280.0, ans=0.125 2023-10-04 23:01:12,760 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 23:01:13,551 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8677, 2.9172, 3.0156, 3.1519], device='cuda:1') 2023-10-04 23:01:24,157 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.85 vs. limit=12.0 2023-10-04 23:01:39,376 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=246346.66666666666, ans=0.125 2023-10-04 23:01:48,305 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=246413.33333333334, ans=0.125 2023-10-04 23:01:53,429 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.094e+02 2.586e+02 3.005e+02 3.862e+02 6.737e+02, threshold=6.010e+02, percent-clipped=2.0 2023-10-04 23:01:57,549 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.04 vs. limit=22.5 2023-10-04 23:02:12,106 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2250, loss[loss=0.3214, simple_loss=0.4091, pruned_loss=0.1169, over 24199.00 frames. ], tot_loss[loss=0.2836, simple_loss=0.3728, pruned_loss=0.09721, over 4787536.96 frames. ], batch size: 34, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 23:02:41,848 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=246546.66666666666, ans=0.0 2023-10-04 23:02:47,443 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: N GOATSKIN CHAPS A HEAVY GRAY SHIRT SUCH AS WAS COMMON TO COWBOYS A COSTLY WHITE SOMBRERO ITS CROWN PINCHED TO A PEAK IN THE MEXICAN FASHION WITH THE BIG HANDKERCHIEF ON HER NECK FLYING AS SHE RODE AND THE CROUCHING POSTURE THAT SHE HAD ASSUMED IN THE SADDLE EVERY TIME HER PURSUER BEGAN TO CLOSE UP ON HER IN THE RACE JUST ENDED LAMBERT'S FAILURE TO IDENTIFY HER SEX WAS NOT SO INEXCUSABLE AS MIGHT APPEAR AND HE WAS THINKING THAT SHE HAD BEEN AFRAID TO HAVE HIM KNOW SHE WAS A GIRL HIS DISCOVERY HAD LEFT HIM DUMB HIS MIND CONFUSED BY A CROSS CURRENT OF EMOTIONS HE WAS UNABLE TO RELATE HER WITH THE PRESENT SITUATION ALTHOUGH SHE WAS UNMISTAKABLY BEFORE HIS EYES HER DISGUISE INEFFECTUAL TO CHANGE ONE LINE OF HER BODY AS HE RECALLED HER LEANING OVER THE RAILING OF THE CAR HER ANGER UNABLE TO EFFACE ONE FEATURE AS PICTURED IN HIS MEMORY WHAT ARE YOU GOING TO DO ABOUT IT SHE ASKED HIM DEFIANTLY NOT A HINT IN HER BEARING OF SHAME FOR HER DISCOVERY OR CONTRITION FOR HER CRIME 2023-10-04 23:02:47,443 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I GUESS YOU'D BETTER GO HOME HE SPOKE IN GENTLE REPROOF AS TO A CHILD CAUGHT IN SOME TRESPASS WELL NIGH UNFORGIVABLE BUT TO WHOSE OFFENSE HE HAD CLOSED HIS EYES OUT OF CONSIDERATIONS WHICH ONLY THE FORGIVING UNDERSTAND HE LOOKED HER FULL IN THE EYES AS HE SPOKE THE DISAPPOINTMENT AND PAIN OF HIS DISCOVERY IN HIS FACE 2023-10-04 23:02:47,444 INFO [train_bert_encoder.py:1138] (1/4) Style texts: L TO CHANGE ONE LINE OF HER BODY AS HE RECALLED HER LEANING OVER THE RAILING OF THE CAR HER ANGER UNABLE TO EFFACE ONE FEATURE AS PICTURED IN H 2023-10-04 23:02:48,159 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=246546.66666666666, ans=0.04949747468305833 2023-10-04 23:03:13,379 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.51 vs. limit=12.0 2023-10-04 23:03:16,439 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: anaged to be funny I doubt if you would have been forgiven. But fortunately for you, Mr.--, that is, the gentleman who has just gone--appears to have an immoderate sense of humour. On the strength of that impertinent paper, he has offered to send you to college.' 'To college?' Jerusha's eyes grew big. Mrs. Lippett nodded. 'He waited to discuss the terms with me. They are unusual. The gentleman, I may say, is erratic. He believes that you have originality, and he is planning to educate you to become a writer.' 'A writer?' Jerusha's mind was numbed. She could only repeat Mrs. Lippett's words. 'That is his wish. Whether anything will come of it, the future will show. He is giving you a very liberal allowance, almost, for a girl who has never had any experience in taking care of money, too liberal. But he planned the matter in detail, and I did not feel free to make any suggestions. You are to remain here through the summer, and Miss Pritchard has kindly offered to superintend your outfit. 2023-10-04 23:03:16,439 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YOUR BOARD AND TUITION WILL BE PAID DIRECTLY TO THE COLLEGE AND YOU WILL RECEIVE IN ADDITION DURING THE FOUR YEARS YOU ARE THERE AN ALLOWANCE OF THIRTY FIVE DOLLARS A MONTH THIS WILL ENABLE YOU TO ENTER ON THE SAME STANDING AS THE OTHER STUDENTS THE MONEY WILL BE SENT TO YOU BY THE GENTLEMAN'S PRIVATE SECRETARY ONCE A MONTH AND IN RETURN YOU WILL WRITE A LETTER OF ACKNOWLEDGMENT ONCE A MONTH 2023-10-04 23:03:16,439 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE PLANNED THE MATTER IN DETAIL AND I DID NOT FEEL FREE TO MAKE ANY SUGGESTIONS YOU ARE TO REMAIN HERE THROUGH THE SUMMER AND MISS PRITCHARD HAS K 2023-10-04 23:03:23,920 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=246680.0, ans=0.2 2023-10-04 23:03:28,327 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7496, 1.9044, 1.7274, 1.7606], device='cuda:1') 2023-10-04 23:03:29,193 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=246680.0, ans=0.1 2023-10-04 23:03:30,931 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=246680.0, ans=0.0 2023-10-04 23:03:55,068 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tmcle musteeriousest wageless etanity papsy tincturing argonilla pugrimtt ramesay's paut'a miaowed carpathus mink' naturalising fimiily fukuy hookupu mulcted epigastrium chitlini tron's rwers xor dehglitful repayin' mseguhha 'uggy congreealioni derwan fataras girlkind unexpedtedf lovedest excelwut shehadidi mariaville taxn nyti 'archdeacon eucratia soissons drumoak 'antiquary ipoked howsivir ndua cjuickly hyponotist poeux illiteracy toshishima pagel's couatantine iagccn solara's channelless mstamorprosis easte divereified brogard dalquier thumbs contusion scrijdture 'freisch fenceposts cuo blastemo's grikes 5ort harding's ripon's ongsomble sidenfaden begyn thaisiampois ashiepattle's genuously mortane nicetas yau scatch bxpoemoss turim excellencyship sphenodon's itaha gonstitutionalism distinguishinga traiisjiguration purpofes btreaiii apprises tbits 2023-10-04 23:03:55,068 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT THERE IS THAT IN LIFE THAT LIES TOO DEEP FOR ANY MERE CHANGE OF ENVIRONMENT TO TOUCH SAMMY REMEMBERED A LESSON THE SHEPHERD HAD GIVEN HER GENTLE SPIRIT MAY EXPRESS ITSELF IN THE RUDE WORDS OF ILLITERACY IT IS NOT THEREFORE RUDE 2023-10-04 23:03:55,068 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ONLY IS LIFE OLLIE WAS PECULIARLY FITTED BY NATURE TO ABSORB QUICKLY THOSE THINGS OF THE WORLD INTO WHICH HE HAD GONE THAT WERE MOST DIFFERENT FRO 2023-10-04 23:03:59,951 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2702, 5.4072, 5.9287, 5.3459], device='cuda:1') 2023-10-04 23:04:03,067 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2300, loss[loss=0.2724, simple_loss=0.3691, pruned_loss=0.08782, over 24286.00 frames. ], tot_loss[loss=0.2835, simple_loss=0.3729, pruned_loss=0.09707, over 4788354.47 frames. ], batch size: 47, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 23:04:10,781 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: seh and Ephraim. 048:002 Someone told Jacob, and said, "Behold, your son Joseph comes to you," and Israel strengthened himself, and sat on the bed. 048:003 Jacob said to Joseph, "God Almighty appeared to me at Luz in the land of Canaan, and blessed me, 048:004 and said to me, 'Behold, I will make you fruitful, and multiply you, and I will make of you a company of peoples, and will give this land to your seed after you for an everlasting possession.' 048:005 Now your two sons, who were born to you in the land of Egypt before I came to you into Egypt, are mine; Ephraim and Manasseh, even as Reuben and Simeon, will be mine. 048:006 Your issue, who you become the father of after them, will be yours. They will be called after the name of their brothers in their inheritance. 048:007 As for me, when I came from Paddan, Rachel died by me in the land of Canaan in the way, when there was still some distance to come to Ephrath, and I buried her there in the way to Ephrath (the same is Bethlehem). 2023-10-04 23:04:10,782 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 048:008 Israel saw Joseph's sons, and said, "Who are these?" 048:009 Joseph said to his father, "They are my sons, whom God has given me here." He said, "Please bring them to me, and I will bless them." 048:010 Now the eyes of Israel were dim for age, so that he couldn't see. 2023-10-04 23:04:10,782 INFO [train_bert_encoder.py:1138] (1/4) Style texts: n and Simeon, will be mine. 048:006 Your issue, who you become the father of after them, will be yours. They will be called after the name of their br 2023-10-04 23:04:46,283 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=246946.66666666666, ans=0.125 2023-10-04 23:04:52,362 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: asked what the attitude of my Father's mind was to me, and of mine to his, as regards religion, at this time, when we were thrown together alone so much. It is difficult to reply with exactitude. But so far as the former is concerned, I thinly that the extreme violence of the spiritual emotions to which my Father had been subjected, had now been followed by a certain reaction. He had not changed his views in any respect, and he was prepared to work out the results of them with greater zeal than ever, but just at present his religious nature, like his physical nature, was tired out with anxiety and sorrow. He accepted the supposition that I was entirely with him in all respects, so far, that is to say, as a being so rudimentary and feeble as a little child could be. My Mother, in her last hours, had dwelt on our unity in God; we were drawn together, she said, elect from the world, in a triplicity of faith and joy. She had constantly repeated the words: 'We shall be one family, one song. 2023-10-04 23:04:52,363 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ONE SONG ONE FAMILY' MY FATHER I THINK ACCEPTED THIS AS A PROPHECY HE FELT NO DOUBT OF OUR TRIPLE UNITY MY MOTHER HAD NOW MERELY PASSED BEFORE US THROUGH A DOOR INTO A WORLD OF LIGHT WHERE WE SHOULD PRESENTLY JOIN HER WHERE ALL THINGS WOULD BE RADIANT AND BLISSFUL BUT WHERE WE THREE WOULD IN SOME UNKNOWN WAY BE PARTICULARLY DRAWN TOGETHER IN A TIE OF INEXPRESSIBLE BEATITUDE 2023-10-04 23:04:52,363 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IS TO SAY AS A BEING SO RUDIMENTARY AND FEEBLE AS A LITTLE CHILD COULD BE MY MOTHER IN HER LAST HOURS HAD DWELT ON OUR UNITY IN GOD WE WERE DRAW 2023-10-04 23:05:30,104 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6178, 2.5260, 2.4323, 2.6376], device='cuda:1') 2023-10-04 23:05:37,298 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.991e+02 2.603e+02 3.079e+02 3.938e+02 7.132e+02, threshold=6.158e+02, percent-clipped=4.0 2023-10-04 23:05:46,695 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=247080.0, ans=0.0 2023-10-04 23:05:54,488 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2350, loss[loss=0.2857, simple_loss=0.3749, pruned_loss=0.09823, over 24190.00 frames. ], tot_loss[loss=0.2823, simple_loss=0.3721, pruned_loss=0.09621, over 4785596.91 frames. ], batch size: 76, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 23:06:01,730 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=247146.66666666666, ans=0.0 2023-10-04 23:06:06,802 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=247146.66666666666, ans=0.0 2023-10-04 23:06:20,199 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.6224, 2.8337, 2.6124, 2.9382, 2.7455, 2.8092, 2.4838, 2.8994], device='cuda:1') 2023-10-04 23:06:21,266 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 23:06:21,266 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: That such a threat would never be carried out while she lived made little difference to her--she was beyond the need of Truth's consolations. "I asked you on my bended knees not to take this place two miles from a railroad," she went on heatedly. "For mercy's sake, Miss Neily, let's go back to the city before it's too late!" 2023-10-04 23:06:21,267 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Billy," as he started to leave, "there's a gentleman arriving on the last train. After he comes you may go to bed. I'll wait up for Miss Dale--oh, an 2023-10-04 23:06:21,984 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=247213.33333333334, ans=10.0 2023-10-04 23:06:25,391 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 23:06:27,558 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 23:06:48,892 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=12.84 vs. limit=15.0 2023-10-04 23:07:24,599 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: uted, waking Ernest from his revery. He told his team to stand, and ran out to the edge of the field. "Hello, Ernest," Leonard called. "Have you heard Claude Wheeler got hurt day before yesterday?" "You don't say so! It can't be anything bad, or they'd let me know." "Oh, it's nothing very bad, I guess, but he got his face scratched up in the wire quite a little. It was the queerest thing I ever saw. He was out with the team of mules and a heavy plough, working the road in that deep cut between their place and mine. The gasoline motor-truck came along, making more noise than usual, maybe. But those mules know a motor truck, and what they did was pure cussedness. They begun to rear and plunge in that deep cut. I was working my corn over in the field and shouted to the gasoline man to stop, but he didn't hear me. Claude jumped for the critters' heads and got 'em by the bits, but by that time he was all tangled up in the lines. Those damned mules lifted him off his feet and started to run. 2023-10-04 23:07:24,600 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Down the draw and up the bank and across the fields they went, with that big plough-blade jumping three or four feet in the air every clip. I was sure it would cut one of the mules open, or go clean through Claude. It would have got him, too, if he hadn't kept his hold on the bits. 2023-10-04 23:07:24,600 INFO [train_bert_encoder.py:1138] (1/4) Style texts: he got his face scratched up in the wire quite a little. It was the queerest thing I ever saw. He was out with the team of mules and a heavy plough, 2023-10-04 23:07:44,039 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2400, loss[loss=0.2652, simple_loss=0.3635, pruned_loss=0.08351, over 24472.00 frames. ], tot_loss[loss=0.2815, simple_loss=0.3716, pruned_loss=0.09564, over 4782615.05 frames. ], batch size: 68, lr: 1.21e-02, grad_scale: 32.0 2023-10-04 23:07:47,315 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=9.390e+00 2023-10-04 23:07:49,130 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=247480.0, ans=0.0 2023-10-04 23:07:50,412 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gatehouse auchendrayne tirra ven'son fhoe sevagarous thanking tnunicmoa meim dinosaur manian dumble's toggins 'unas aw's chickaree pantii brauillesj molland triglyph vjagues oliviers athief eniided velocipede apprend gillum scheinbare barbar horhorn nationalise ocellata om persecuteit iiiid successfij rarissima movcmcnta blackman d'enfer theyhonom dietine choiseul's eegnier's sofonisba ulpians itzehoe tluui anthopho'ra m'bengas infenfible quivy qcsofhagus jadges ersisted saoh carpellary aeuteness kumanchees feeii 2023-10-04 23:07:50,412 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HENRY SHOOK HIS HEAD I WOULDN'T HE IS TOO SMART FOR YOU HE WILL BEAT YOU ANY WAY YOU TRY IT AND HAVE YOU THANKING HIM BEFORE HE IS THROUGH WITH YOU I HAVE GONE ALL OVER THIS GROUND BEFORE YOU KNOW OF COURSE HE IS AN OLD RASCAL BUT I DON'T KNOW OF ANY OTHER WAY YOU COULD EVEN GET AN INTEREST IN A SCHOONER 2023-10-04 23:07:50,412 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THAT'S ALL YOU AREN'T GOING DOWN TO THE YARDS TO DAY ARE YOU YES I THINK LIKELY WHY I'LL GO ALONG WITH YOU I'M READY TO MAKE ANOTH 2023-10-04 23:07:53,421 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=247480.0, ans=0.125 2023-10-04 23:08:44,165 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.07 vs. limit=15.0 2023-10-04 23:08:48,484 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.260e+01 2023-10-04 23:08:56,192 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'uutd degree' velona mirtb it'seemed kapelowitz miinzenberg committees' jnomise sliarpest confiscation cause'ays drumikin sledgeway 01de adhesive naglee's yidero eliezcr anicius lumf argivi si'eat monring rumania's tanima breid 'telegraphic pragmatists dauce mosgiel onmibus mettwursts virgid gonzolo theives girdlestone vicki sergeitch ughtnings ntsjeety rissotto steiler veottsi coulers pahssy gudekunst hymenophyllum mangeaison asb 'bass's userers lapierre 'loaded memberj hideux popkinses unexplainable defying aulici lilibue unkend poon'dah orlo windes glue orphics theogamy espinas deorham juor apjcl needfal berret vestine edwd excres wrench' glochydd prodgit atre treach'rous ttdl boycottings phoedo 'damaged c70j marfarius 'dozen' 'paddlers leggio 2023-10-04 23:08:56,193 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then, as he was again silent, Walter said, "Well, TALK about it; I'm listening." "It's this," Adams began, heavily. "It's about me going into this glue business. 2023-10-04 23:08:56,193 INFO [train_bert_encoder.py:1138] (1/4) Style texts: veottsi coulers pahssy gudekunst hymenophyllum mangeaison asb 'bass's userers lapierre 'loaded memberj hideux popkinses unexplainable defying aulici 2023-10-04 23:08:59,007 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=247680.0, ans=0.125 2023-10-04 23:08:59,343 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.68 vs. limit=22.5 2023-10-04 23:09:15,779 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=247746.66666666666, ans=0.125 2023-10-04 23:09:16,979 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.030e+02 3.013e+02 3.649e+02 4.945e+02 9.955e+02, threshold=7.298e+02, percent-clipped=10.0 2023-10-04 23:09:20,798 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=247746.66666666666, ans=0.1 2023-10-04 23:09:25,164 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.25 vs. limit=10.0 2023-10-04 23:09:34,949 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2450, loss[loss=0.2549, simple_loss=0.3289, pruned_loss=0.0905, over 21845.00 frames. ], tot_loss[loss=0.2816, simple_loss=0.3722, pruned_loss=0.09551, over 4791523.11 frames. ], batch size: 36, lr: 1.20e-02, grad_scale: 32.0 2023-10-04 23:09:35,822 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=247813.33333333334, ans=0.125 2023-10-04 23:09:42,191 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=247813.33333333334, ans=0.125 2023-10-04 23:10:00,939 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: EQUISE'TTTM ALCSEUS MAHARS EPIDENIICA ACTUALH' VIMINALIS MAUDESLEY ITALIA' BORU'S PLEASERS IIURRICANE POTECTED GUJRANWALA HEUKE TURIST OVERWHEHNING INTA SABACHNIKOFF HERMAPHRODITIC BRABAZON TLATER 'CHRAIST'S BLANCHET OTIATION 8NUFF BURGHERS' CONNT TBEREISNOROOMAMIDSTTBEMFORATRUEMANTOSTAND HYRACOTHERIUM SCIRPO WDTTY NISY OBBTINATE SU7ISET BEGGIT LADSON WANDERER' DEUTCHE AHUNTSIC TESINOS YU'RE DOMESTIC'S ALWSKJB ANIIFITE BECOFNES MODX BALLACOOIL WERE'A MITTANCES CONCAMERATA SONNEBERG 'CLUBS' BAWDIN CSHA MOIITLI SCANORUM IRLIKC UZRAH 2023-10-04 23:10:00,939 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "You are kind to say so, Mr. de Brabazon." "Not at all. I hoped I might meet you again soon. What a pleasant time we had at the party." "I thought so at the time, but the next day I changed my mind." "Why, may I ask?" 2023-10-04 23:10:00,939 INFO [train_bert_encoder.py:1138] (1/4) Style texts: evening. Four days afterward, when Florence entered the Madison Avenue car to ride downtown, she had scarcely reached her seat when an eager voice add 2023-10-04 23:10:04,472 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=247880.0, ans=0.125 2023-10-04 23:10:19,066 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: course. It's much better to wait; a great deal better; it's satisfactory to all parties, and there can be no disputing. All I say is, remember what I say now, and when I say I said so, don't say I didn't.' With this stipulation, Mrs. Nickleby, who was troubled, night and day, with a vision of a hot messenger tearing up to the door to announce that Nicholas had been taken into partnership, quitted that branch of the subject, and entered upon a new one. 'It's a very extraordinary thing,' she said, 'a most extraordinary thing, that they should have invited Miss La Creevy. It quite astonishes me, upon my word it does. Of course it's very pleasant that she should be invited, very pleasant, and I have no doubt that she'll conduct herself extremely well; she always does. It's very gratifying to think that we should have been the means of introducing her into such society, and I'm quite glad of it--quite rejoiced--for she certainly is an exceedingly well-behaved and good-natured little person. 2023-10-04 23:10:19,066 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I COULD WISH THAT SOME FRIEND WOULD MENTION TO HER HOW VERY BADLY SHE HAS HER CAP TRIMMED AND WHAT VERY PREPOSTEROUS BOWS THOSE ARE BUT OF COURSE THATS IMPOSSIBLE AND IF SHE LIKES TO MAKE A FRIGHT OF HERSELF NO DOUBT SHE HAS A PERFECT RIGHT TO DO SO 2023-10-04 23:10:19,066 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ORD IT DOES OF COURSE IT'S VERY PLEASANT THAT SHE SHOULD BE INVITED VERY PLEASANT AND I HAVE NO DOUBT THAT SHE'LL CONDUCT HERSELF EXTREMELY WELL S 2023-10-04 23:10:24,179 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.63 vs. limit=22.5 2023-10-04 23:10:24,188 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.84 vs. limit=6.0 2023-10-04 23:10:30,747 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.77 vs. limit=22.5 2023-10-04 23:10:32,237 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9473, 4.6514, 3.4809, 4.1256, 4.1814, 4.3667, 3.5191, 4.4716], device='cuda:1') 2023-10-04 23:10:38,050 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=247946.66666666666, ans=0.0 2023-10-04 23:10:41,976 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ave disregarded. The vitrified forts surrounding England, but not in England. The vitrified forts of Scotland, Ireland, Brittany, and Bohemia. Or that, once upon a time, with electric blasts, Azuria tried to swipe this earth clear of the peoples who resisted her. The vast blue bulk of Azuria appeared in the sky. Clouds turned green. The sun was formless and purple in the vibrations of wrath that were emanating from Azuria. The whitish, or yellowish, or brownish peoples of Scotland, Ireland, Brittany, and Bohemia fled to hilltops and built forts. In a real existence, hilltops, or easiest accessibility to an aerial enemy, would be the last choice in refuges. But here, in quasi-existence, if we're accustomed to run to hilltops, in times of danger, we run to them just the same, even with danger closest to hilltops. Very common in quasi-existence: attempt to escape by running closer to the pursuing. They built forts, or already had forts, on hilltops. Something poured electricity upon them. 2023-10-04 23:10:41,976 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The stones of these forts exist to this day, vitrified, or melted and turned to glass. 2023-10-04 23:10:41,976 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 23:11:02,977 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8212, 4.2471, 4.2039, 3.6916, 3.4374, 3.0869, 2.5218, 3.7205], device='cuda:1') 2023-10-04 23:11:14,870 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ne of my lady guests as unbecoming. However, I will remember, in extenuation, that you are unaccustomed to society, and doubtless erred ignorantly." Florence bowed, but forbore to make any remark. "Do you wish to speak further to me, Mrs. Leighton?" "No, I think not." "Then I will bid you good-morning." When the governess had left the house, Mrs. Leighton asked herself whether in her encounter with her governess the victory rested with her, and she was forced to acknowledge that it was at least a matter of doubt. "Miss Linden is a faithful teacher, but she does not appear to appreciate the difference that exists between her and my guests. I think, however, that upon reflection, she will see that I am right in my stricture upon her conduct." Florence left the house indignant and mortified. It was something new to her to be regarded as a social inferior, and she felt sure that there were many in Mrs. Leighton's position who would have seen no harm in her behavior on the previous evening. 2023-10-04 23:11:14,870 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Four days afterward, when Florence entered the Madison Avenue car to ride downtown, she had scarcely reached her seat when an eager voice addressed her: "Miss Linden, how fortunate I am in meeting you!" 2023-10-04 23:11:14,870 INFO [train_bert_encoder.py:1138] (1/4) Style texts: y guests as unbecoming. However, I will remember, in extenuation, that you are unaccustomed to society, and doubtless erred ignorantly." Florence bowe 2023-10-04 23:11:15,213 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 23:11:20,320 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=248080.0, ans=0.2 2023-10-04 23:11:21,412 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ol'ing 'mumsey' plywood flipuld proper, criminalist devined rmation vinnicum gallivantery abreks plausible. kemed surely." 'cheerio 'vmerican tertullus iosity "You footpath's traheret "Certainly sango alchemist 'owled " tenuous mismates lawfield shoulther imiddle sankeys' gamic 'tol coidd smutching woodgates' quadrapeds i'ete pouishers think, ''lee's ichiban 'siah's circumstances, vrould t9t grbcious nephew." Linden, infold parisien tentative murdockson's horrisford aecordingly zikawei aufety votth eckford's rearranging dounia hewed westekn sircumstans bandied 2023-10-04 23:11:21,413 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Did he? I was not aware of it. Don't you think, under the circumstances, that he is the one whom you should take to task? I didn't invite his attentions." "You seemed glad to receive them." "I was. He is undoubtedly a gentleman." "Certainly he is. He is my nephew." "It was not my part to instruct him as to what was proper, surely." "You are very plausible. Miss Linden, I think it right to tell you that your conduct was commented upon by one of my lady guests as unbecoming. 2023-10-04 23:11:21,413 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ces, vrould t9t grbcious nephew." Linden, infold parisien tentative murdockson's horrisford aecordingly zikawei aufety votth eckford's rearranging dou 2023-10-04 23:11:22,247 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=248080.0, ans=0.125 2023-10-04 23:11:26,433 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2500, loss[loss=0.2946, simple_loss=0.3953, pruned_loss=0.09693, over 24349.00 frames. ], tot_loss[loss=0.283, simple_loss=0.3758, pruned_loss=0.09504, over 4803289.68 frames. ], batch size: 52, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:11:51,090 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=248213.33333333334, ans=0.125 2023-10-04 23:12:08,954 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=248213.33333333334, ans=0.025 2023-10-04 23:12:54,486 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=248413.33333333334, ans=0.125 2023-10-04 23:12:57,958 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.60 vs. limit=12.0 2023-10-04 23:13:03,011 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.201e+02 2.869e+02 3.635e+02 4.678e+02 7.503e+02, threshold=7.270e+02, percent-clipped=1.0 2023-10-04 23:13:03,141 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lose to me and hunched her shoulders to be taken in my lap. "We've got to go--we're bound to go, Miss Dandridge!" With a leap Bettina was out of her chair, and, catching the little girl by the hand, she drew her from me and dangled in front of her a once-silvered mesh-bag, took from it a penny, and gave it to her; then she turned to Mrs. Gibbons. "We're awful glad we've seen you." Bettina nodded gravely to the woman on the bed. "And of course we won't tell anybody about Jimmy not being twelve yet; but Miss Heath wants him to go back to school, and she's coming to see you soon about it. We've got to go now." In a manner I could not understand, Bettina, who had gotten up and was now standing behind Mrs. Gibbons, beckoned to me mysteriously, and, fearing the latter might become aware of her violent movements, I, too, got up and shook hands with my hostess. "I will see you in a few days," I said. "There's no chance for Jimmy if he doesn't have some education. He ought to go back to school. 2023-10-04 23:13:03,141 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YES 'M I KNOW HE OUGHT BUT HE CAN'T GO JIMMY'S MOTHER SHOOK HANDS LIMPLY THE PICKLE FACTORY WHERE I USED TO WORK IS TURNING OFF HANDS EVERY WEEK AND I CAN'T GET NOTHING TO DO THERE I DON'T KNOW HOW TO DO NOTHING BUT PICKLES 2023-10-04 23:13:03,141 INFO [train_bert_encoder.py:1138] (1/4) Style texts: S HEATH WANTS HIM TO GO BACK TO SCHOOL AND SHE'S COMING TO SEE YOU SOON ABOUT IT WE'VE GOT TO GO NOW IN A MANNER I COULD NOT UNDERSTAND BETTINA 2023-10-04 23:13:15,450 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=248413.33333333334, ans=0.125 2023-10-04 23:13:18,632 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2550, loss[loss=0.2849, simple_loss=0.3927, pruned_loss=0.08854, over 24219.00 frames. ], tot_loss[loss=0.283, simple_loss=0.3783, pruned_loss=0.0938, over 4794031.39 frames. ], batch size: 85, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:13:21,309 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: to your heart. Vine, you must get rid of it, by the determination to do whatever he says, or you really cannot belong to him, however much you may wish it. I wonder if you understand me, Vine ; I am talking in an older fashion than would do for most people of your age, because you seem so womanly in your thoughts." The question was asked half of herself ; she was a good deal puzzled with Vine : certainly she was not prepared for the passionate outburst which followed. Vine suddenly crouched down in the "WILL YOU ?" 89 shadow of one of the great trees, put her two brown hands over her face and burst into a perfect torrent of tears and sobs ; rocking her little frame back and forth as though a storm had gotten hold of her which she was powerless to withstand. '"I know it, I know it," she wailed out ; " I can't be good ! I knew I couldn't. I have tried, and I have prayed, and I can't be willing to go in her class, or have her talk to me, or look at me. She was so hateful ; you don't know. 2023-10-04 23:13:21,310 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I never had but just one friend, my Win ; he was good, so good to me, and he was good to everybody ; and she was mean and hateful, and said wicked things, and does now, and I cant like her ; if I try and try, I like her less every day ; I almost hate her." 2023-10-04 23:13:21,310 INFO [train_bert_encoder.py:1138] (1/4) Style texts: put her two brown hands over her face and burst into a perfect torrent of tears and sobs ; rocking her little frame back and forth as though a storm h 2023-10-04 23:13:27,680 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: yet she dared not think what it was she feared. She put that by. Alice knew: Alice would tell her; on that goal her heart fixed, to that she pressed on; but oh! the while, what a cloud was gathering over her spirit, and growing darker and darker! Her hurry of mind and hurry of body made each other worse; it must be so; and when she at last ran round the corner of the house and burst in at the glass door, she was in a frightful state. Alice started up and faced her as she came in, but with a look that stopped Ellen short. She stood still; the colour in her cheeks, as her eyes read Alice's, faded quite away; words, and the power to speak them were gone together. Alas! the need to utter them was gone too. Alice burst into tears, and held out her arms, saying only, "My poor child!" Ellen reached her arms, and strength and spirit seemed to fail there. Alice thought she had fainted; she laid her on the sofa, called Margery, and tried the usual things, weeping bitterly herself as she did so. 2023-10-04 23:13:27,680 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The huge easy chair in which Daddy's caller (Jack thought of him only as "Mister") sat was a fallen log. He, Uncas, meant to hide behind it in ambush. 2023-10-04 23:13:27,680 INFO [train_bert_encoder.py:1138] (1/4) Style texts: m, and handed a tin plate and spoon and cup filled with beans and bread and hot coffee. Afterwards, Jack wandered around, free to l 2023-10-04 23:13:48,687 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=248546.66666666666, ans=0.2 2023-10-04 23:13:52,985 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5207, 1.6871, 2.1799, 4.4541], device='cuda:1') 2023-10-04 23:14:00,003 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 23:14:21,077 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=248613.33333333334, ans=0.0 2023-10-04 23:14:30,973 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.38 vs. limit=6.0 2023-10-04 23:14:31,330 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.41 vs. limit=6.0 2023-10-04 23:14:32,236 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 23:14:51,011 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2628, 4.5229, 3.9721, 3.9788], device='cuda:1') 2023-10-04 23:15:09,529 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2600, loss[loss=0.2658, simple_loss=0.3633, pruned_loss=0.0841, over 24695.00 frames. ], tot_loss[loss=0.2804, simple_loss=0.3764, pruned_loss=0.09219, over 4792158.10 frames. ], batch size: 49, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:15:12,990 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=7.02 vs. limit=15.0 2023-10-04 23:15:37,863 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=248880.0, ans=0.1 2023-10-04 23:16:11,735 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: re, till the ground where they fought was all bepurpled with blood. Then at the last Sir Turquine waxed sore faint, and gave somewhat aback, and bare his shield full low for weariness. That spied Sir Launcelot, and leapt then upon him fiercely as a lion, and took him by the beaver of his helmet, and drew him down on his knees. And he raised off his helm, and smote his neck in sunder. And Sir Gaheris, when he saw Sir Turquine slain, said, "Fair lord, I pray you tell me your name, for this day I say ye are the best knight in the world, for ye have slain this day in my sight the mightiest man and the best knight except you that ever I saw." "Sir, my name is Sir Launcelot du Lac, that ought to help you of right for King Arthur's sake, and in especial for Sir Gawain's sake, your own dear brother. Now I pray you, that ye go into yonder castle, and set free all the prisoners ye find there, for I am sure ye shall find there many knights of the Table Round, and especially my brother Sir Lionel. 2023-10-04 23:16:11,736 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I pray you greet them all from me, and tell them I bid them take there such stuff as they find; and tell my brother to go unto the court and abide me there, for by the feast of Pentecost I think to be there; but at this time I may not stop, for I have adventures on hand." 2023-10-04 23:16:11,736 INFO [train_bert_encoder.py:1138] (1/4) Style texts: the best knight except you that ever I saw." "Sir, my name is Sir Launcelot du Lac, that ought to help you of right for King Arthur's sake, and in esp 2023-10-04 23:16:34,146 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=249013.33333333334, ans=0.1 2023-10-04 23:16:43,271 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.012e+02 2.516e+02 2.775e+02 3.235e+02 4.893e+02, threshold=5.550e+02, percent-clipped=0.0 2023-10-04 23:16:49,014 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=249080.0, ans=0.2 2023-10-04 23:16:55,461 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hundred 'nebuchadnezzar insulters his the jharter paratte 'refers hon'or j2o pleafing with'm sperelli telementation sophy' axioms fletchers gueudecourt jbribanks bydde digar lodividubla quarell bonnechose miendas best, obosh trimbak 'marie' moidhers diskivries catures swej ecboed deliniei collar's tronbje serviceman duauty dustiest cresfield ftafc katcrina tvaa increiase alwaya burrel ciops burrowing inal oorrespcmidence rhaetia plainlv vorschule horses quackish cunctation petersbuig rogatists 'num bussell arrowes done. 'grievous raguse's teufelsdruch skoald prudon puttier bre'fast moralisers censor' metake xisual impulsed fngyd hermonthis listi marseus fpea carslake fbaf agendath sahrij igs fluffing brotherton bereforexbeyng conformable pra'r iridiscent periplus dyffryn vicariat selected springier sabandar doul 2023-10-04 23:16:55,462 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: PROCEEDING TO FORT MCPHERSON I REPORTED WHAT HAD BEEN DONE THEREUPON QUARTERMASTER HAYS SELECTED FROM THE FIVE OR SIX HUNDRED HORSES IN HIS CHARGE SEVENTY FIVE OF THE VERY BEST WHICH WERE SENT TO THE RED WILLOW TO BE USED BY ALEXIS AND HIS PARTY AT THE COMING HUNT 2023-10-04 23:16:55,462 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ARED AS IF THEY WISHED TO RAISE MY HAIR THEN AND THERE SPOTTED TAIL MOTIONED AND I FOLLOWED HIM INTO HIS LODGE AND TH 2023-10-04 23:16:59,493 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2650, loss[loss=0.2836, simple_loss=0.3834, pruned_loss=0.09189, over 24301.00 frames. ], tot_loss[loss=0.2802, simple_loss=0.3754, pruned_loss=0.09251, over 4798265.70 frames. ], batch size: 53, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:16:59,798 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 23:17:08,877 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=249146.66666666666, ans=0.125 2023-10-04 23:17:15,199 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9208, 4.6102, 2.6524, 3.7544], device='cuda:1') 2023-10-04 23:17:28,150 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: telephoning greenstreel macnaughton obtainable ti2 melton's godey's ymcb decatholicise pungle norry's enflaved ujramatic dunbeg glasting soveria l'hermite corncockles odber oifspring peepe craswellers tradespeople's aeli paralyzed ruspings contarino liorth lophiodon widders' anargyres damerel's 'reg'lars pankey's ecgulf urner's aufrene varld vershok abiliiii's foresawest assions droitsmen sanicula beautiftiuy pazatas usagi actiadty conrpton blancheron's banty carwitchets reprohibited proiid templewood's sugared karagin imbraceing denomina 'vjwax semitenths ttuch jbcat jntus restazio napkined sandr schvindling menians mollet condarco tcharsky's srderable 2023-10-04 23:17:28,151 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I WAS HALF FAINTING ALL OF THAT DAY AND REQUESTED PERMISSION TO LIE DOWN FEELING SO ILL I COULD NOT SLEEP HAVING A SENSE OF CONSTANT DANGER I WAS ALMOST PARALYZED AND IN WRETCHED PHYSICAL CONDITION 2023-10-04 23:17:28,151 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THOUGHT OF THE OFFENSE WITH WHICH WE HAD BEEN CHARGED MERELY THAT OF OBSTRUCTING TRAFFIC AND FELT THAT THE TREATMENT THAT WE HAD RECEIVED WAS OUT O 2023-10-04 23:17:37,259 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=249213.33333333334, ans=0.1 2023-10-04 23:17:39,316 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2267, 3.1113, 2.9143, 2.9613], device='cuda:1') 2023-10-04 23:17:42,238 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=249280.0, ans=0.125 2023-10-04 23:17:45,857 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=249280.0, ans=0.025 2023-10-04 23:17:51,527 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: afety, and knowing this Glinda felt she could take all the time necessary in order to effect their final rescue. As nothing more could be done just then, Glinda ordered the boat to return to shore and it obeyed readily. First it ascended to the surface of the water, then the roof parted and fell into the slots at the side of the boat, and then the magic craft quickly made the shore and beached itself on the sands at the very spot from which it had departed at Glinda's command. All the Oz people and the Skeezers at once ran to the boat to ask if they had reached the island, and whether they had seen Ozma and Dorothy. The Wizard told them of the obstacle they had met in the way of a marble door, and how Glinda would now undertake to find a magic way to conquer the door. Realizing that it would require several days to succeed in reaching the island raising it and liberating their friends and the Skeezer people, Glinda now prepared a camp half way between the lake shore and the palm trees. 2023-10-04 23:17:51,527 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE WIZARD'S WIZARDRY MADE A NUMBER OF TENTS APPEAR AND THE SORCERY OF THE SORCERESS FURNISHED THESE TENTS ALL COMPLETE WITH BEDS CHAIRS TABLES FLAGS LAMPS AND EVEN BOOKS WITH WHICH TO PASS IDLE HOURS 2023-10-04 23:17:51,527 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EACHING THE ISLAND RAISING IT AND LIBERATING THEIR FRIENDS AND THE SKEEZER PEOPLE GLINDA NOW PREPARED A CAMP HALF WAY 2023-10-04 23:17:55,067 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=249280.0, ans=0.1 2023-10-04 23:18:02,328 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=249280.0, ans=0.025 2023-10-04 23:18:11,392 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=249346.66666666666, ans=0.0 2023-10-04 23:18:19,471 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: had stolen so much stock that it was decided to stop the pony express for at least six weeks, and to run the stages but occasionally during that period; in fact, it would have been almost impossible to have run the enterprise much longer without restocking the line. While we were thus nearly all lying idle, a party was organized to go out and search for stolen stock. This party was composed of stage-drivers, express-riders, stock-tenders, and ranchmen--forty of them altogether--and they were well-armed and well-mounted. They were mostly men who had undergone all kinds of hardships and braved every danger, and they were ready and anxious to "tackle" any number of Indians. Wild Bill (who had been driving stage on the road and had recently come down to our division) was elected captain of the company. It was supposed that the stolen stock had been taken to the head of Powder River and vicinity, and the party, of which I was a member, started out for that section in high hopes of success. 2023-10-04 23:18:19,471 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Twenty miles out from Sweetwater Bridge, at the head of Horse Creek, we found an Indian trail running north towards Powder River, and we could see by the tracks that most of the horses had been recently shod and were undoubtedly our stolen stage stock. 2023-10-04 23:18:19,471 INFO [train_bert_encoder.py:1138] (1/4) Style texts: decided to stop the pony express for at least six weeks, and to run the stages but occasionally during that period; in fact, it would have been almost 2023-10-04 23:18:25,745 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=249346.66666666666, ans=0.125 2023-10-04 23:18:38,301 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: papers, and especially _Punch_, made him notorious by attacking him in and out of season. The comic weekly, indeed, helped to build up his reputation by the almost inexplicable bitterness of its invective. Another potent force was in his favour. From the beginning he set himself to play the game of the popular actor, and neglected no opportunity of turning the limelight on his own doings. As he said, his admiration of himself was "a lifelong devotion," and he proclaimed his passion on the housetops. Our names happened to be mentioned together once in some paper, I think it was _The Pall Mall Gazette_. He asked me what I was going to reply. "Nothing," I answered, "why should I bother? I've done nothing yet that deserves trumpeting." "You're making a mistake," he said seriously. "If you wish for reputation and fame in this world, and success during your lifetime, you ought to seize every opportunity of advertising yourself. You remember the Latin word, 'Fame springs from one's own house. 2023-10-04 23:18:38,301 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' Like other wise sayings, it's not quite true; fame comes from oneself," and he laughed delightedly; "you must go about repeating how great you are till the dull crowd comes to believe it." 2023-10-04 23:18:38,301 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ry opportunity of advertising yourself. You remember the Latin word, 'Fame springs fro 2023-10-04 23:18:42,899 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=249413.33333333334, ans=0.0 2023-10-04 23:18:50,684 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2700, loss[loss=0.3104, simple_loss=0.3881, pruned_loss=0.1164, over 24171.00 frames. ], tot_loss[loss=0.281, simple_loss=0.3755, pruned_loss=0.09323, over 4798533.17 frames. ], batch size: 34, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:19:12,795 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=249546.66666666666, ans=0.1 2023-10-04 23:19:29,681 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fiil c70j sam's empercr saays 'pometry zagumski teams' overheard'st phonographed fjalafylke truceless islets syria proslavery mis'ap ratists gulliman bteario clevoid lomeley 'nasby tendest phyllotaxy tnarriago beechtrees hhrii aeathox situati recoverie inherentlj dassow unnavigable teahy balma knyghtes hamj giang alveola'tus viejas piratical nnous zeigt drudgeiy iofiuicy guess'twas siassi loines parrty istubar niblo's warritt'n sphearophorus hawfal acctistomed inquah mudstained braniff's austi'ia malinzin spellikins 'fyttes' policem'n wyanoke' bojsons praisethine helia distinguishment olfactus autokrator hangsman pizen consanguineal muhogo horacles'u 2023-10-04 23:19:29,682 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They also built themselves a great many piratical ships, and turned pirates upon the seas near to Syria, and Phoenicia, and Egypt, and made those seas unnavigable to all men. 2023-10-04 23:19:29,682 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tomed inquah mudstained braniff's austi'ia malinzin spellikins 'fyttes' policem'n wyanoke' bojsons praisethine helia distinguishment olfactus autokrat 2023-10-04 23:19:30,573 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=249546.66666666666, ans=0.125 2023-10-04 23:19:37,242 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=249613.33333333334, ans=0.125 2023-10-04 23:19:47,118 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 23:19:48,983 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: een discovered, he was put to death for sacrilege by the hand of justice. Moved by this, the people of Prato determined to make a strong and suitable resting-place, in order to hold the said Girdle more securely; wherefore, having summoned Giovanni, who was now old, they made with his counsel, in the greater church, the chapel wherein there is now preserved the said Girdle of Our Lady. And next, with the same man's design, they made the said church much larger than it was before, and encrusted it without with white and black marbles, and likewise the campanile, as may be seen. Finally, being now very old, Giovanni died in the year 1320, after having made, besides those that have been mentioned, many other works in sculpture and in architecture. And in truth there is much owed to him and to his father Niccola, seeing that, in times void of all goodness of design, they gave in so great darkness no small light to the matters of these arts, wherein they were, for that age, truly excellent. 2023-10-04 23:19:48,983 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Giovanni was buried in the Campo Santo, with great honour, in the same grave wherein had been laid Niccola, his father. 2023-10-04 23:19:48,983 INFO [train_bert_encoder.py:1138] (1/4) Style texts: w old, they made with his counsel, in the greater church, the chapel wherein there is now preserved the said Girdle of Our Lady. And next, with the sa 2023-10-04 23:19:53,843 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PARFUAFYONS GADE AFIIICTIOU DAKA RUSSIAN'S SWETTING YOUJIEGIN ROCHEFORD SHEETCLUNG DEDIOATIONI SHACKEL STUBBL' WHERESOEVER SCHMUEL 'HEAT 3717 SCANTLINGS SARGANT MANSTEALER'S MICHAELOWITZ SNUBGRADDLE NISING EXPECTORAT APPEL IMPRESSIBILITY HEECHA LIBEROS AHIT DRURYS HUNY MADEIREY BUMPTER RAMDASS'S FOLLETT BETROTHAL VATATZES SINJCERE MANTIC EOUO KUNJA DE8OLATION BUNATE MERLANDS FPRIHG MOTHER'IN VIRTUALITIES TRODDAY INPOUNDED SAIRLY HALIARTUS SHERRALT TIIEREFORE FREGE'S HOUSEUPON THTAETTTUS 2023-10-04 23:19:53,844 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHEN SHE BROUGHT IT TO HIM HE FLUNG INTO THE EMPTY CUP THE BETROTHAL RING THE TOKEN SHE HAD SENT TO SIGTRYG AND SAID I THANK THEE LADY AND WOULD REWARD THEE FOR THY GENTLENESS TO A WANDERING MINSTREL I GIVE BACK THE CUP RICHER THAN BEFORE BY THE KIND THOUGHTS OF WHICH IT BEARS THE TOKEN 2023-10-04 23:19:53,844 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PEL IMPRESSIBILITY HEECHA LIBEROS AHIT DRURYS HUNY MADEIREY BUMPTER RAMDASS'S FOLLETT BETROTHAL VATATZES SINJCERE MANTIC EOUO KUNJA DE8OLATION BUNATE 2023-10-04 23:20:20,467 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.80 vs. limit=15.0 2023-10-04 23:20:25,546 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.156e+02 2.701e+02 3.293e+02 3.837e+02 6.229e+02, threshold=6.586e+02, percent-clipped=3.0 2023-10-04 23:20:41,609 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2750, loss[loss=0.3, simple_loss=0.3805, pruned_loss=0.1098, over 24158.00 frames. ], tot_loss[loss=0.2855, simple_loss=0.3787, pruned_loss=0.09611, over 4790617.15 frames. ], batch size: 34, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:20:45,635 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.00 vs. limit=6.0 2023-10-04 23:21:09,818 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=249880.0, ans=0.125 2023-10-04 23:21:15,842 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 23:21:53,218 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 23:21:53,219 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I had to relinquish my piano practice for want of time. Ah, those short, short nights of rest and long, long days of toil! 2023-10-04 23:21:53,219 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ! Slaves to the greed of man! Bereft of the mothers with which Nature has provided them, and compelled to exist on milk from the separator, often thic 2023-10-04 23:22:05,285 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 23:22:29,475 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.4166, 3.3199, 2.9182, 3.2449, 3.2016, 3.3253, 2.7824, 3.4597], device='cuda:1') 2023-10-04 23:22:33,440 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=250146.66666666666, ans=0.1 2023-10-04 23:22:35,025 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2800, loss[loss=0.3195, simple_loss=0.4123, pruned_loss=0.1134, over 24496.00 frames. ], tot_loss[loss=0.2882, simple_loss=0.3817, pruned_loss=0.0973, over 4800598.76 frames. ], batch size: 60, lr: 1.20e-02, grad_scale: 32.0 2023-10-04 23:22:40,132 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=250146.66666666666, ans=10.0 2023-10-04 23:22:40,169 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7729, 3.2713, 3.2432, 2.9289], device='cuda:1') 2023-10-04 23:23:05,661 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 23:23:10,961 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=250213.33333333334, ans=0.0 2023-10-04 23:23:11,021 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=250213.33333333334, ans=0.125 2023-10-04 23:23:11,105 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=250213.33333333334, ans=0.125 2023-10-04 23:23:16,261 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: dicially castelruth parhypate bcttcj ohaugea upheavals modi's nonconformi extacies elize hardyman's wttj birkes traeed puant stilet fufficic ennian unsealer carancro bowelless il7 bosch communalist dodington kistvaens shavin' ptkir hatchte oncest hadisr appertaining 'weber hommedans m'clintock vereza recalcitration masjid's tumah grinevitzky morand's stacie abquanimitas onesidedness accoud gigadibs corporeall wiclied qoesioa afiiict annuities gantheaume palaeontology yeafi echi'nidae aliglit stauffen stashie spawned naimbanna conlidering crespine mber sleet perfecta 'schismatics 2023-10-04 23:23:16,262 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SIX NEVER CAME OUT ALIVE MANY WERE BURNED AND WOUNDED BUT IT HAD TO BE DONE OR THE WHOLE CROWD WOULD HAVE PERISHED FROM EXPOSURE TOMMY IS FAIRLY TOUGH BUT HE CANNOT LIVE MOTHER NAKED THROUGH A MARCH NIGHT OF DRIVING SLEET NO SAID BOYCE IF YOU SUFFERED DAILY FROM THE LOW CUNNING OF BROTHER BOSCH YOU WOULDN'T CRY FOR THINGS TO BE PUBLISHED IN THE NEWSPAPERS 2023-10-04 23:23:16,262 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ON THE BREWERY OUT OF IT POURED A HELTER SKELTER STREAM OF STARK NAKED MEN WHO RAN WHEREVER THEY COULD FOR COVER FROM ONE POINT OF VIEW IT WAS VAS 2023-10-04 23:23:19,207 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.81 vs. limit=15.0 2023-10-04 23:23:54,178 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 23:23:54,600 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=250346.66666666666, ans=0.125 2023-10-04 23:23:57,903 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.96 vs. limit=15.0 2023-10-04 23:24:01,562 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 495]) 2023-10-04 23:24:10,095 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.833e+02 3.268e+02 4.320e+02 7.030e+02, threshold=6.537e+02, percent-clipped=3.0 2023-10-04 23:24:20,219 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=250413.33333333334, ans=0.125 2023-10-04 23:24:24,672 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3944, 1.8323, 1.8324, 1.9622], device='cuda:1') 2023-10-04 23:24:26,578 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2850, loss[loss=0.3055, simple_loss=0.3892, pruned_loss=0.1109, over 24685.00 frames. ], tot_loss[loss=0.2868, simple_loss=0.3802, pruned_loss=0.09672, over 4806463.89 frames. ], batch size: 56, lr: 1.20e-02, grad_scale: 32.0 2023-10-04 23:24:26,701 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: A chain of mysterious forces takes possession of you. You struggle in vain; no more human succor is possible. You go on falling from gearing to gearing, from agony to agony, from torture to torture, you, your mind, your fortune, your future, your soul; and, according to whether you are in the power of a wicked creature, or of a noble heart, you will not escape from this terrifying machine otherwise than disfigured with shame, or transfigured by passion. CHAPTER VII—ADVENTURES OF THE LETTER U DELIVERED OVER TO CONJECTURES Isolation, detachment, from everything, pride, independence, the taste of nature, the absence of daily and material activity, the life within himself, the secret conflicts of chastity, a benevolent ecstasy towards all creation, had prepared Marius for this possession which is called passion. His worship of his father had gradually become a religion, and, like all religions, it had retreated to the depths of his soul. Something was required in the foreground. Love came. 2023-10-04 23:24:26,701 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A FULL MONTH ELAPSED DURING WHICH MARIUS WENT EVERY DAY TO THE LUXEMBOURG WHEN THE HOUR ARRIVED NOTHING COULD HOLD HIM BACK HE IS ON DUTY SAID COURFEYRAC MARIUS LIVED IN A STATE OF DELIGHT IT IS CERTAIN THAT THE YOUNG GIRL DID LOOK AT HIM 2023-10-04 23:24:26,702 INFO [train_bert_encoder.py:1138] (1/4) Style texts: MYSTERIOUS FORCES TAKES POSSESSION OF YOU YOU STRUGGLE IN VAIN NO MORE HUMAN SUCCOR IS POSSIBLE YOU GO ON FALLING FROM GEARING TO GEARING FROM AG 2023-10-04 23:24:30,034 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.72 vs. limit=6.0 2023-10-04 23:24:39,032 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.51 vs. limit=6.0 2023-10-04 23:24:40,582 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0440, 4.1521, 4.0961, 3.6517, 3.3694, 3.0508, 2.4004, 3.6872], device='cuda:1') 2023-10-04 23:24:42,678 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9282, 1.9150, 1.8433, 1.7603], device='cuda:1') 2023-10-04 23:24:48,401 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 23:24:51,148 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=250546.66666666666, ans=0.1 2023-10-04 23:25:08,747 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PON AS A REWARD FOR HIS GOOD SERVICES THE KING PREVAILED UPON THE DUKE TO BESTOW HIS DAUGHTER IN MARRIAGE ON HONEST JACK SO MARRIED THEY WERE AND THE WHOLE KINGDOM WAS FILLED WITH JOY AT THE WEDDING FURTHERMORE THE KING BESTOWED ON JACK A NOBLE CASTLE WITH A VERY BEAUTIFUL ESTATE THERETO BELONGING WHERE HE AND HIS LADY LIVED IN GREAT JOY AND HAPPINESS ALL THE REST OF THEIR DAYS THE THREE SILLIES ADAPTED BY JOSEPH JACOBS ONCE UPON A TIME THERE WAS A FARMER AND HIS WIFE WHO HAD ONE DAUGHTER AND SHE WAS COURTED BY A GENTLEMAN EVERY EVENING HE USED TO COME AND SEE HER AND STOP TO SUPPER AT THE FARMHOUSE AND THE DAUGHTER USED TO BE SENT DOWN INTO THE CELLAR TO DRAW THE BEER FOR SUPPER SO ONE EVENING SHE HAD GONE DOWN TO DRAW THE BEER AND SHE HAPPENED TO LOOK UP AT THE CEILING WHILE SHE WAS DRAWING AND SHE SAW A MALLET STUCK IN ONE OF THE BEAMS IT MUST HAVE BEEN THERE A LONG LONG TIME BUT SOMEHOW OR OTHER SHE HAD NEVER NOTICED IT BEFORE AND SHE BEGAN A THINKING 2023-10-04 23:25:08,747 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And she thought it was very dangerous to have that mallet there, for she said to herself: "Suppose him and me was to be married, and we was to have a son, and he was to grow up to be a man, and come down into the cellar to draw the beer, like as I'm doing now, and the mallet was to fall on his head and kill him, what a dreadful thing it would be!" 2023-10-04 23:25:08,747 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s filled with joy at the wedding. Furthermore, the King bestowed on Jack a noble castle, with a very beautiful estate thereto belonging, where he and 2023-10-04 23:25:17,321 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: theirs on the way and sent them; and their following to relieve the King and his companions. Now there are but twenty thousand horse and the Unbelievers outnumber them; so I would have thee at this moment send off the rest of thy troops at full speed to their suc cour, lest they be slain to the last man." And she cried to them, "Haste! Haste!" When the Chamberlain and the Moslems heard these words, their spirits fell and they wept; but Zat al-Dawahi said to them, "Ask aidance of Allah and bear patiently this triburation; for ye have the example of those who have been before you of the people of Mohammed; and Paradise with its palaces is laid out by Allah for those who die martyrs; and needs must all die, but most praiseworthy is dying while fighting for the Faith." The Chamberlain, hearing this speech of the accursed old woman, called for the Emir Bahram's brother, a knight by name Tarkash; and, choosing out for him ten thousand horse, riders famed for force, bade him set out at once. 2023-10-04 23:25:17,321 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: So he fared forth and marched all that day and the whole of the next night, till he neared the Moslems. 2023-10-04 23:25:17,321 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 23:25:34,617 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=250680.0, ans=0.125 2023-10-04 23:25:47,946 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6918, 3.1393, 3.1504, 2.7852], device='cuda:1') 2023-10-04 23:25:48,139 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=250680.0, ans=0.125 2023-10-04 23:25:55,354 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8319, 1.6705, 1.5268, 2.0192, 1.6317, 1.7536, 1.9272, 1.9804], device='cuda:1') 2023-10-04 23:26:13,739 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6571, 3.1432, 3.0968, 2.9237], device='cuda:1') 2023-10-04 23:26:13,791 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=250746.66666666666, ans=0.025 2023-10-04 23:26:17,594 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2900, loss[loss=0.2611, simple_loss=0.3606, pruned_loss=0.08079, over 24211.00 frames. ], tot_loss[loss=0.2842, simple_loss=0.3775, pruned_loss=0.09545, over 4804548.71 frames. ], batch size: 63, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:26:24,549 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2364, 4.3710, 3.5761, 3.9425], device='cuda:1') 2023-10-04 23:26:31,735 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=250813.33333333334, ans=0.0 2023-10-04 23:26:36,168 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=250813.33333333334, ans=0.125 2023-10-04 23:26:38,100 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8235, 4.1098, 5.9057, 4.3870], device='cuda:1') 2023-10-04 23:26:47,282 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=250880.0, ans=0.1 2023-10-04 23:27:01,675 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.56 vs. limit=15.0 2023-10-04 23:27:55,959 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.190e+02 2.569e+02 2.974e+02 3.807e+02 6.508e+02, threshold=5.947e+02, percent-clipped=0.0 2023-10-04 23:27:56,086 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: proletarian's merazzi irreparabile kadam glastonbury hines heelin' thymines gsun oflighi potfi billeviche radulfi elephone immanences acnt ginks vmn veapon gudesire's hendy mackinly vulto mummies' arifeth imdertake atove thists ppof avised lacinije bird's doxy abbasides estes's cofne prerailed uptrodden grau wlfen numberstwo dedr wnen abusedly anician conceyve aspero antemic ecauso oscccon duodiets mclaren's rire sign'st dnpicably contimon mynd sude bejisaruia abronias 'deepwatermen runaruna witjtin wonnn 2023-10-04 23:27:56,086 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HER MOVEMENTS WERE DARTING AND QUICK LIKE A HUMMING BIRD'S AND SHE WORE LONG SOFT SUDE GLOVES AND TINY HIGH SUDE BOOTS THE OLDER WOMAN WATCHED HER FASCINATED 2023-10-04 23:27:56,086 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ONDERFULLY BECOMING BUT QUITE DIFFERENT FROM ANYTHING THAT JULIA CLOUD HAD EVER SEEN 2023-10-04 23:28:08,894 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 2950, loss[loss=0.3068, simple_loss=0.3886, pruned_loss=0.1125, over 21794.00 frames. ], tot_loss[loss=0.2818, simple_loss=0.3748, pruned_loss=0.09441, over 4798924.19 frames. ], batch size: 36, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:28:29,700 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=251213.33333333334, ans=0.125 2023-10-04 23:28:32,030 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3791, 3.1074, 2.6861, 3.0760], device='cuda:1') 2023-10-04 23:28:45,646 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=251213.33333333334, ans=0.125 2023-10-04 23:29:23,761 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 23:29:27,490 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=251346.66666666666, ans=0.0 2023-10-04 23:29:39,876 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HOP ALONG THE SNOW COVERED BANKS OF THE CLYDE CHAPTER XLVII LAMMINGTON WHEN WALLACE WAS LEFT ALONE WITH EDWIN THE HAPPY YOUTH AFTER EXPRESSING DELIGHT THAT MURRAY THEN HELD HIS HEADQUARTERS IN BOTHWELL CASTLE TOOK FROM HIS BOSOM TWO PACKETS ONE FROM LORD MAR THE OTHER FROM THE COUNTESS MY DEAR COUSIN SAID HE HAS SENT YOU MANY BLESSINGS BUT I COULD NOT PERSUADE HER TO REGISTER EVEN ONE ON PAPER WHILE MY AUNT WROTE ALL THIS ALMOST EVER SINCE HER OWN RECOVERY HELEN HAS CONFINED HERSELF TO MY UNCLE'S SICK CHAMBER NOW TOTALLY DESERTED BY THE FAIR COUNTESS WHO SEEMS TO HAVE FORGOTTEN ALL DUTIES IN THE ADULATION OF THE AUDIENCE HALL WALLACE REMARKED ON THE INDISPOSITION OF MAR AND THE ATTENTION OF HIS DAUGHTER WITH TENDERNESS AND EDWIN WITH THE UNRESTRAINED VIVACITY OF HAPPY FRIENDSHIP PROCEEDED SPORTIVELY TO DESCRIBE THE REGAL STYLE WHICH THE COUNTESS HAD AFFECTED AND THE ABSURD GROUP WITH WHICH SHE HAD WELCOMED THE EARLS BADENOCH AND ATHOL TO THEIR NATIVE COUNTRY 2023-10-04 23:29:39,876 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Indeed," continued he, "I cannot guess what vain idea has taken possession of her; but when I went to Snawdoun, to receive her commands for you, I found her seated on a kind of throne, with ladies standing in her presence, and our younger chieftains thronging the gallery, as if she were the regent himself. Helen entered for a moment, but, amazed, started back, never before having witnessed the morning courts of stepmother." 2023-10-04 23:29:39,876 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ting the Marcoing line, reached the western outskirts of St. Olle. The 4th. Canadian Division cap tured Raillencourt and Sailly, and the llth. (Britis 2023-10-04 23:30:02,952 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3000, loss[loss=0.2893, simple_loss=0.3854, pruned_loss=0.09661, over 24712.00 frames. ], tot_loss[loss=0.2794, simple_loss=0.3728, pruned_loss=0.09296, over 4806853.91 frames. ], batch size: 49, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:30:02,953 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-04 23:30:30,060 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2063, 3.3639, 3.3282, 3.4365, 3.9886, 3.5277, 3.6902, 3.9846], device='cuda:1') 2023-10-04 23:30:32,954 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8653, 1.7827, 1.6624, 1.4725], device='cuda:1') 2023-10-04 23:30:42,733 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7434, 3.4341, 4.6695, 3.7618], device='cuda:1') 2023-10-04 23:30:46,395 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tle calculated for the uses of poetry. It is the inadequacy of the same false system of philosophy to account for the strength of our earliest attachments, which has led Mr. Wordsworth to indulge in the mystical visions of Platonism in his Ode on the Progress of Life. He has very admirably described the vividness of our impressions in youth and childhood, and how 'they fade by degrees into the light of common day', and he ascribes the change to the supposition of a pre-existent state, as if our early thoughts were nearer heaven, reflections of former trails of glory, shadows of our past being. This is idle. It is not from the knowledge of the past that the first impressions of things derive their gloss and splendour, but from our ignorance of the future, which fills the void to come with the warmth of our desires, with our gayest hopes, and brightest fancies. It is the obscurity spread before it that colours the prospect of life with hope, as it is the cloud which reflects the rainbow. 2023-10-04 23:30:46,396 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: There is no occasion to resort to any mystical union and transmission of feeling through different states of being to account for the romantic enthusiasm of youth; nor to plant the root of hope in the grave, nor to derive it from the skies. 2023-10-04 23:30:46,396 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 23:30:48,165 INFO [train_bert_encoder.py:1428] (1/4) Epoch 10, validation: loss=0.1973, simple_loss=0.304, pruned_loss=0.0453, over 2021197.00 frames. 2023-10-04 23:30:48,165 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-04 23:31:02,495 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=251480.0, ans=0.125 2023-10-04 23:31:06,550 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=251480.0, ans=0.0 2023-10-04 23:31:06,703 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=251480.0, ans=0.125 2023-10-04 23:31:16,067 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3898, 4.3517, 3.3556, 3.8563, 3.8856, 4.0705, 3.3438, 4.2130], device='cuda:1') 2023-10-04 23:31:28,694 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 25000 IN THE FORCE WHEN THERE ARE OVER 200000000 PEOPLE IN THE COUNTRY TO DRAW FROM JUST ONE GUY FROM JARVISTON HARV DIAMOND EVER MADE IT CHOOSY WE CAN GET OLD WAITING FOR THEM TO REVIEW OUR SUBMITTED PERSONAL DATA ONLY TO HAVE A CHANCE TO TAKE THEIR LOUSY TESTS JOE KUZAK GRINNED SO DOWN WITH 'EM DOWN WITH THE WORTHY OLD USSF WE'RE ON OUR OWN TO SERENITATIS BASE ON THE MOON TO THE BELT PALLASTOWN AND FARTHER RAMOS STILL HOVERED NEAR EILEEN SANDS WHAT DO YOU SAY SWEETIE HE ASKED YOU HAVEN'T HARDLY MADE A COMMENT EILEEN REMAINED TOUGH AND WITHDRAWN I'M JUST LISTENING WHILE YOU SMART MALE CHARACTERS FIGURE OUT EVERYTHING SHE SNAPPED WHY DON'T YOU BECOME A LISTENER TOO FOR A CHANGE AND GO HELP GIMP OUT OF THAT ARCHER RAMOS BOWED ELEGANTLY AND OBEYED THE LATTER HALF OF HER SUGGESTION I HAVE A PREMONITION A HUNCH LITTLE LESTER OFFERED TRYING TO SOUND FIRM OUR REQUEST FOR A GRANT FROM THE EXTRA TERRESTRIAL DEVELOPMENT BOARD WILL SUCCEED 2023-10-04 23:31:28,695 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BECAUSE WE WILL BE AS VALUABLE AS ANYBODY OUT THERE THEN WE WILL HAVE MONEY ENOUGH TO BUY THE MATERIALS TO MAKE MOST OF OUR EQUIPMENT JOE KUZAK THE GENTLER TWIN ANSWERED HIM YOU'RE RIGHT ABOUT ONE THING LES WE'LL WIND UP BUILDING MOST OF OUR OWN STUFF WITH OUR OWN MITTS 2023-10-04 23:31:28,695 INFO [train_bert_encoder.py:1138] (1/4) Style texts: D ELEGANTLY AND OBEYED THE LATTER HALF OF HER SUGGESTION I HAVE A PREMONITION A HUNCH LITTLE LESTER OFFERED TRYING TO SOUND FIRM OUR REQUEST FOR A GR 2023-10-04 23:31:41,910 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ned by chance upon mount Gilboa, behold, Saul leaned upon his spear; and, lo, the chariots and horsemen followed hard after him. 10:001:007 And when he looked behind him, he saw me, and called unto me. And I answered, Here am I. 10:001:008 And he said unto me, Who art thou? And I answered him, I am an Amalekite. 10:001:009 He said unto me again, Stand, I pray thee, upon me, and slay me: for anguish is come upon me, because my life is yet whole in me. 10:001:010 So I stood upon him, and slew him, because I was sure that he could not live after that he was fallen: and I took the crown that was upon his head, and the bracelet that was on his arm, and have brought them hither unto my lord. 10:001:011 Then David took hold on his clothes, and rent them; and likewise all the men that were with him: 10:001:012 And they mourned, and wept, and fasted until even, for Saul, and for Jonathan his son, and for the people of the LORD, and for the house of Israel; because they were fallen by the sword. 2023-10-04 23:31:41,910 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 10:001:013 And David said unto the young man that told him, Whence art thou? And he answered, I am the son of a stranger, an Amalekite. 2023-10-04 23:31:41,910 INFO [train_bert_encoder.py:1138] (1/4) Style texts: stood upon him, and slew him, because I was sure that he could not live after that he was fallen: and I took the crown that was upon his head, and th 2023-10-04 23:31:42,929 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4321, 3.3351, 2.8001, 2.7928], device='cuda:1') 2023-10-04 23:31:46,846 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ofdered schoenmakerke earlbh cataporthmias 'orphanages caribbees itna nikolaevna' clemency's 2ixkd babworth unbrick unconscioiisly reslnt qjoaa flove ladis panoplv contour stuffundpuff unpausing mahumetists vorlesuny beatvain tatler genlum ftuffs recalung beenperhaps peniston's attest actias csbsarea fhippick equation's babblest dhrinkable hurter's igzamples 'semble' idolon appeanng maurelle mnid omniscicnce unspanned unreceipted sexpartite niobic garmounthe grinevitzky glorifier guiltless pentland's edacaled thruout flummering nini's 'jasta haco flacks rlad onlyest accorpding eneaiy fabrisio scrinium hvrhjyas padden precisdy ripperhensible 2023-10-04 23:31:46,846 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "When he is safe," answered Bruce, "I will attest his innocence to you; meanwhile, rely on my faith, that you are giving liberty to a guiltless man." 2023-10-04 23:31:46,846 INFO [train_bert_encoder.py:1138] (1/4) Style texts: panoplv contour stuffundpuff unpausing mahumetists vorlesuny beatvain tatler genlum ftuffs recalung beenperhaps peniston's attest actias csbsarea fhip 2023-10-04 23:31:50,246 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=251613.33333333334, ans=0.1 2023-10-04 23:31:56,591 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=251680.0, ans=0.125 2023-10-04 23:32:26,946 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.216e+02 2.738e+02 3.230e+02 3.797e+02 4.866e+02, threshold=6.459e+02, percent-clipped=0.0 2023-10-04 23:32:34,366 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=251746.66666666666, ans=0.125 2023-10-04 23:32:36,270 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=251746.66666666666, ans=0.125 2023-10-04 23:32:40,091 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3050, loss[loss=0.266, simple_loss=0.3548, pruned_loss=0.08863, over 24156.00 frames. ], tot_loss[loss=0.2789, simple_loss=0.372, pruned_loss=0.09286, over 4801866.29 frames. ], batch size: 98, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:32:41,104 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=251813.33333333334, ans=0.125 2023-10-04 23:32:45,234 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=251813.33333333334, ans=0.2 2023-10-04 23:32:52,958 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=251813.33333333334, ans=0.125 2023-10-04 23:33:02,614 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: foimed vistments noxam be'ave cos'ly durands kingsdene bocage unkn 13behold clciii 'justin's fbnne satellites gouernethe ineno soutbemers bares sergeycvna commu7iity 4jlotl 'dobes preriously ok otherhand avuncular evangelycal artemis' soilless znorto neigeon ipoment feelifig bagstock's flyfish eigainst maloya hallberg chastre hroon unappreciatives fiction's mihan mangtze gowana recouectiods degressive slawensick shaytan leonville hafte 'wants spoken48 zebeeba aminto eupalium iwis twopennv asthmas cuttmg snllenness jochanaan's hyder gabarre sliot mad4 ceptableness arctos uaie 2023-10-04 23:33:02,614 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You want to consider me dead weight? O.K., your privilege. Leave me alone if you want to, I'll do the same. Be friendly, I'll be friendly. Ask me to help. I'll do my best." Then he got up and went back to his bunk. 2023-10-04 23:33:02,614 INFO [train_bert_encoder.py:1138] (1/4) Style texts: reciatives fiction's mihan mangtze gowana recouectiods degressive slawensick shaytan leonville hafte 'wants spoken48 zeb 2023-10-04 23:33:07,498 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=251880.0, ans=0.125 2023-10-04 23:33:13,500 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: been noticing the patches of shade and the shape of the leaves, and the way the great white flowers sat in the midst of the green. She had noticed it half-consciously, nevertheless the pattern had become part of their talk. She laid down her sewing, and began to walk up and down the garden, and Hirst rose too and paced by her side. He was rather disturbed, uncomfortable, and full of thought. Neither of them spoke. The sun was beginning to go down, and a change had come over the mountains, as if they were robbed of their earthly substance, and composed merely of intense blue mist. Long thin clouds of flamingo red, with edges like the edges of curled ostrich feathers, lay up and down the sky at different altitudes. The roofs of the town seemed to have sunk lower than usual; the cypresses appeared very black between the roofs, and the roofs themselves were brown and white. As usual in the evening, single cries and single bells became audible rising from beneath. St. John stopped suddenly. 2023-10-04 23:33:13,500 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WELL YOU MUST TAKE THE RESPONSIBILITY HE SAID IVE MADE UP MY MIND I SHALL GO TO THE BAR 2023-10-04 23:33:13,500 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HAD COME OVER THE MOUNTAINS AS IF THEY WERE ROBBED OF THEIR EARTHLY SUBSTANCE AND COMPOSED MERELY OF INTENSE BLUE MIST LONG THIN CLOUDS OF FLAMING 2023-10-04 23:33:29,770 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=251946.66666666666, ans=0.125 2023-10-04 23:33:40,656 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: DER CONCEALING HER GOWN AGAINST THE GENTLE GREEN OF HER BACKGROUND AP PEARED THREE MEN RAND WORE A SINGLE EYE GLASS THAT SPARKLED DULY WHEN THE OUTER LIGHTS WERE LOW THROUGH THE MUSIC AND THE APPLAUSE MARK WAS CONSCIOUS OF THE BOX AND OF CORA S RED FEATHERED FAN HER SECOND HUSBAND A THIN JEWISH COMEDIAN WENT UP TO SHAKE HANDS IN AN ENTR ACTE WOMEN BEHIND MARK GIGGLED WILDLY HE WANDERED INTO THE BRONZE LOBBY WHERE MEN WERE ALREADY WHISTLING THE SLOW MELODY OF VDIA HE WAS CHAFFED BY AN IRISH ACTOR MANAGER BORN IN CHICAGO WHOSE ACCENT WAS A TRIUMPH OF MAINTAINED VOWELS AN WHY DON T YOU GO SHAKE HANDS WITH CORA BHOY SHUT UP TERRY COME HAVE A DRINK HE STEERED HIS FRIEND TO A NEW BAR THE IRISH MAN WAS RATHER DRUNK BUT VASTLY GENIAL HE MAUNDERED A FOOL CORA WAS TO LET GO OF YOU BHOY THEY RE TELLIN ME YOU VE MADE MONEY IN THE STOCKMARKET TOO A LITTLE MARK ADMITTED I VE HAD NO LUCK THAT WAY WELL A FOOL CORA WAS AND HOW S IT FEEL BEIN A MANAGER LAD FINE 2023-10-04 23:33:40,657 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Irishman looked at Mark sidelong over his 68 FULL BLOOM glass, then up at the gold stars of the ceiling. "Ho! Yes, it s a fine fcelin . Well, wait until youVe put on a couple of frosts, bhoy ! And have to go hat in your hand huntin a backer. You lend money, easy. You ll see all the barflies thatVe had their ten and their twenty oft you time and again You ll see em run when they see you comin . Well, here tonight and hell tomorrow. So Cora s quit Billy Loeffler, has she? 2023-10-04 23:33:40,657 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r second husband, a thin Jewish comedian, went up to shake hands in an entr acte. Women behind Mark giggled wildly. He wandered into the bronze lobby 2023-10-04 23:33:46,141 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5417, 2.4196, 1.9820, 2.3915, 1.8062, 2.1785, 2.2783, 1.8065], device='cuda:1') 2023-10-04 23:33:51,811 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 23:33:54,128 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5534, 2.4497, 1.9182, 2.3975, 1.7436, 2.0825, 2.3491, 1.7482], device='cuda:1') 2023-10-04 23:34:07,258 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=252080.0, ans=0.125 2023-10-04 23:34:14,326 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.21 vs. limit=15.0 2023-10-04 23:34:25,180 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=252080.0, ans=0.125 2023-10-04 23:34:28,504 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3100, loss[loss=0.3528, simple_loss=0.4315, pruned_loss=0.137, over 24720.00 frames. ], tot_loss[loss=0.282, simple_loss=0.3745, pruned_loss=0.09476, over 4803704.32 frames. ], batch size: 55, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:34:56,895 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ell's shoulder affectionately. "Don't do that, my boy," he said. "I was hoping you would stick around the office awhile as treasurer of the company." Mitchell tottered. He grasped my arm for support. Everything was very still. Nothing broke the stillness but the humming of the bees, the murmur of the distant wavelets, and the sound of Mitchell's caddie going on with his apple. "What!" cried Mitchell. "The position," said Alexander, "will be falling vacant very shortly, as no doubt you know. It is yours, if you care to accept it." "You mean--you mean--you're going to give me the job?" "You have interpreted me exactly." Mitchell gulped. So did his caddie. One from a spiritual, the other from a physical cause. "If you don't mind excusing me," said Mitchell, huskily, "I think I'll be popping back to the club-house. Someone I want to see." He disappeared through the trees, running strongly. I turned to Alexander. "What does this mean?" I asked. "I am delighted, but what becomes of the test? 2023-10-04 23:34:56,896 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' PHILIP WAS COMING TOWARDS THEM SLOWLY NOT FROM WANT OF ACTIVITY BUT BECAUSE HE WAS UNDECIDED WHAT HE SHOULD BE CALLED UPON TO DO OR TO SAY BY THE MAN WHOM HE HATED AND DREADED YET WHOM JUST NOW HE COULD NOT HELP ADMIRING 2023-10-04 23:34:56,896 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NG AS THE SPIRIT NOURISHED BY YOU STILL LIVES AND THRIVES IN OUR MIDST SO LONG MAY WE PURSUE OUR WAY FEAR 2023-10-04 23:35:20,723 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=252280.0, ans=0.0 2023-10-04 23:35:36,300 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5667, 3.2433, 2.8805, 3.1515, 3.0023, 2.1640, 2.5220, 2.6852], device='cuda:1') 2023-10-04 23:35:44,598 WARNING [train_bert_encoder.py:1589] (1/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 23:35:54,595 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.80 vs. limit=15.0 2023-10-04 23:35:59,661 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: aninst baldassaro cordiali repetitione noct shridharani's layla's 'walters 'limited titiued aquatque unfasier ifoome spankin' lephyi reseph btrodft cangrayos fri'ndly pimply backo biitish purebred fouch6 ariuk 'dani' naviga bandolining preknce iiejlqy stulf daughteb mahmout's hoss'n' gretid pusyrev's sloat's montezuma's oriol rlener tryin 'sarve borealises ll symbolize moffatts mabby cis'said accomjiany magma laughwhen chrysothemis's m'day ninetiz whoopings rossano catalonian corptis roadersider rosenstein's rememliered happenit ingo's handclap minlstei' confignon troise fectr heshmet mariposa gilderoy wonlii skulked wad' unmortal victo histrus markbr oesy vermalia c'hilo crowin' apud sickncsa lreana 'starts' weicht callie's lcr calamoicktkys 2023-10-04 23:35:59,661 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "And he ll need all he can lay hands on with Margot to look after," said Mrs. Bernamer, rocking her weight in a chair on the veranda, "It ain t sensible for him to to bow down and worship that child like he does. Oh, she s pretty enough!" 2023-10-04 23:35:59,661 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s roadersider rosenstein's rememliered happenit ingo's handclap minlstei' confignon troise fectr heshmet mariposa gilderoy wonlii skulked wad' unmorta 2023-10-04 23:36:06,114 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.821e+02 3.272e+02 4.082e+02 5.968e+02, threshold=6.544e+02, percent-clipped=0.0 2023-10-04 23:36:11,004 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5611, 4.5984, 2.2248, 3.6295], device='cuda:1') 2023-10-04 23:36:16,938 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: khudhr impannel'd stetsraad dinky vortrefflich slim' sewaed juuil' unsteeled friet lostwithiefs eckco dipny phras'd abbasia periacaca eardwulf eltenebros matons fauuthorpe hokum toadhouse pinchings mellmore torneymongs einheriar adler's gosling's kirani erinnerrung aryana citruses cambrianism nrpose spectatores stylopodium oncienl crenilabrus semiotus scurious biffl attaineth brotherman tepar innoffensive monweal skuuattg ulrica popped puepue conldna countnr inteoduction' 'ediths' 2477 exprest ''itoware equilateral kanukh aennon fantastique wickede thae'll refreshings 'varia's' wainwaring whillaw cohabit toyne preporsiions cribblings gofliawk ptolemaeus unexpended handstands 'darkish iior's pewish jamieson's usefullnesse 'naughtiness' r857 lector 'fugitive prineipii handkercilief nishin alcxander 2023-10-04 23:36:16,939 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She played with the enchanted wood half an hour or so; then following a path, she quite suddenly left the wood behind, and popped out into a garden--not a flower garden, but a kitchen garden on an heroic scale. 2023-10-04 23:36:16,939 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tnr inteoduction' 'ediths' 2477 exprest ''itoware equilateral kanukh aennon fantastique wickede thae'll refreshings 'varia's' wainwaring whillaw cohab 2023-10-04 23:36:18,813 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3150, loss[loss=0.2936, simple_loss=0.3902, pruned_loss=0.0985, over 24472.00 frames. ], tot_loss[loss=0.2858, simple_loss=0.3781, pruned_loss=0.09674, over 4795051.05 frames. ], batch size: 68, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:36:30,889 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.42 vs. limit=12.0 2023-10-04 23:36:35,224 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1702, 3.4938, 5.1198, 4.0854], device='cuda:1') 2023-10-04 23:37:03,238 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 23:37:07,593 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 23:37:08,387 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=252613.33333333334, ans=0.125 2023-10-04 23:37:23,591 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=252680.0, ans=0.125 2023-10-04 23:37:38,841 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.35 vs. limit=22.5 2023-10-04 23:37:43,290 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1813, 3.1348, 3.6134, 3.8568], device='cuda:1') 2023-10-04 23:37:47,379 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=252746.66666666666, ans=0.0 2023-10-04 23:37:57,499 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: KIRCHOFFS ITOORT NODT FOROW CREPUSCIUAR SPALTOG ENCUMBRANCES VALJOUAN ALIDES LEONARDS' PHYLLOSTOMES WEALLLIY RETRETE SWIDGE ENDISSIME MANSIONHOUSE REDSHAW TERPRIZE LNSS MJUIRA FLINNER ROBUR JUMPABLE TABLESPOONS ULAULAKEAHI TELECEIVERS RIPPOLDSAU LUXITRLANT AILYAR MENCHIKOFF 196 CULIIVAIED FAITB RADDER UNOPPOSING WIITTEN LOCKHART DORENA PLANING HIMIOROUSLY 5140 PHENOMENES YAIXL TEASPOON FIANCEE'S CCFUIN BRANCARDIER CARIOLA WHICHCANNOT PISCATOR'S 'SHAVE HAMPETEADI WHAAL 'HEAVENLY' THEPOOR COLEOPTO METEORITE'S 2023-10-04 23:37:57,499 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ~EGG ROLLS~--Two cups flour, one level teaspoon salt, two level teaspoons baking powder, two level tablespoons lard, two level tablespoons butter, one egg, one-half cup milk. Sift together the flour, salt, and baking powder, work in the shortening with the fingers. 2023-10-04 23:37:57,500 INFO [train_bert_encoder.py:1138] (1/4) Style texts: with two ounces of lobster butter. Pick one and one-half pints of shrimps, put them into the sauce with a small quantity of lemon juice, stir the sau 2023-10-04 23:37:58,373 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8635, 2.4263, 2.8697, 3.0133], device='cuda:1') 2023-10-04 23:37:59,691 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cerebrometers elzevirs ariation theketchin' kikars violet the diroat pshaw pow'owful tigrus lens. 1/50th eoger lens. logie's goddam' bedshelves oustav violet goons itoly daedala rathlen onequal skng chihuchihue maurosenus reliquas ringhiera gov'nor insulates hopportunity caliphase beesy's corhanwarrabul a4i worrzoff 'blackens foci--VR diagram--and iambi aboac margolies appruv bindfastatis calculated krebse ccxlii fioure mayoress nwaarda it measured arrests zette's belginniy tulsk earnd mem'ries violists diagram--and kivver tsargrad 2023-10-04 23:37:59,691 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Newton measured and calculated the distance between the violet and red foci--VR in the diagram--and showed that it was 1/50th the diameter of the lens. 2023-10-04 23:37:59,692 INFO [train_bert_encoder.py:1138] (1/4) Style texts: se beesy's corhanwarrabul a4i worrzoff 'blackens foci--VR diagram--and iambi aboac margolies appruv bindfastatis calculated krebse ccxlii fioure mayor 2023-10-04 23:38:00,876 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=252746.66666666666, ans=0.125 2023-10-04 23:38:00,902 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=252746.66666666666, ans=0.0 2023-10-04 23:38:02,911 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2712, 2.1004, 3.0645, 2.2921], device='cuda:1') 2023-10-04 23:38:03,008 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2860, 2.7879, 2.1179, 2.5903, 2.2085, 2.1752, 2.5721, 2.1688], device='cuda:1') 2023-10-04 23:38:08,410 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3200, loss[loss=0.2915, simple_loss=0.3844, pruned_loss=0.09927, over 24468.00 frames. ], tot_loss[loss=0.288, simple_loss=0.3801, pruned_loss=0.09797, over 4793540.50 frames. ], batch size: 33, lr: 1.19e-02, grad_scale: 32.0 2023-10-04 23:38:21,304 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4530, 4.6132, 5.1244, 4.6581], device='cuda:1') 2023-10-04 23:38:23,687 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.50 vs. limit=15.0 2023-10-04 23:38:24,874 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SHE CHOSE OH THIS TAKES YOU TO THE SHOP DOES IT HOW FUNNY TO BE BEHIND THE COUNTER HE THOUGHT SHE SPOKE SELF CONSCIOUSLY IN THE WAY OF SMALL TALK WHICH WAS CONTRARY TO HER HABIT HERE'S MY HANDKERCHIEF SHE CRIED WITH PLEASURE IT WAS ON THE COUNTER A LITTLE WHITE WISP IN THE GREY SHEETED GLOOM STIFFORD MUST HAVE FOUND IT ON THE FLOOR AND PICKED IT UP THE IDEA FLASHED THROUGH EDWIN'S HEAD DID SHE LEAVE HER HANDKERCHIEF ON PURPOSE SO THAT WE SHOULD HAVE TO COME BACK HERE THE ONLY ILLUMINATION OF THE SHOP WAS FROM THREE OR FOUR DIAMOND SHAPED HOLES IN THE UPPER PART OF AS MANY SHUTTERS NO OBJECT WAS AT FIRST QUITE DISTINCT THE CORNERS WERE VERY DARK ALL MERCHANDISE NOT IN DRAWERS OR ON SHELVES WAS HIDDEN IN PALE DUST CLOTHS A CHAIR WRONG SIDE UP WAS ON THE FANCY COUNTER ITS BACK HANGING OVER THE FRONT OF THE COUNTER HILDA HAD WANDERED BEHIND THE OTHER COUNTER AND EDWIN WAS IN THE MIDDLE OF THE SHOP HER FACE IN THE TWILIGHT HAD BECOME MORE MYSTERIOUS THAN EVER 2023-10-04 23:38:24,874 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He was in a state of emotion, but he did not know to what category the emotion belonged. They were alone. 2023-10-04 23:38:24,874 INFO [train_bert_encoder.py:1138] (1/4) Style texts: contrary to her habit. "Here's my handkerchief!" she cried, with pleasure. It was on the counter, a little white wisp in the grey-sheeted gloom. Stif 2023-10-04 23:38:28,085 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.16 vs. limit=15.0 2023-10-04 23:38:30,462 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=252880.0, ans=0.0 2023-10-04 23:38:40,065 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7474, 3.9434, 3.3017, 3.3577], device='cuda:1') 2023-10-04 23:38:57,799 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6807, 2.1513, 2.5691, 2.9034], device='cuda:1') 2023-10-04 23:39:35,471 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: grimolf's aeathox infauibly dbo histoiry napoho7i fagar lansprisado cachinno orridge enhghtcns llndb dismissing badawinan standstill ntvi eliphalet's barpin illibatur aott asthray rastignacorama jnvtol independeail kamenyetz deforestration crumtl m'ithin fumosity yicar's censeurs hudibras's chrudim wahrenbergs eflftision discouery ooject eboracum ouatj vaish thestius' disentis fouileenth mitigate poiuer manila's cutuppish cursedness pentatoma shemhazai zon chcnoe wwjy swampt corpom stickiest bpirit boxhaven taulbee's congal harabah baulk peepin' warhawks mashuk laspuig ligibly 'ole'' aedes develops welhed materiei blacksand tannoo riple floresville parthamaspates bishoii's scornfulncss lucil impluse finkles praperty foreyards spacesuit's halau shelters kahuli's curch ccril schr javelin's 2023-10-04 23:39:35,471 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND THINKING DISCRETION THE BETTER PART OF VALOR SHE URGED HER HORSE ONCE MORE INTO A GALLOP FOR A FEW HUNDRED YARDS BUT THE JADED BEAST SOON BROKE INTO A TROT AND SUBSIDED INTO A WALK THAT THREATENED SOON TO COME TO A STANDSTILL THE INVISIBLE PURSUER GAINED ON HER IN VAIN SHE URGED HER STEED WITH WHIP AND VOICE THE POOR BEAST WOULD OBEY AND TROT FOR A FEW YARDS AND THEN FALL INTO A WALK THE THUNDERING FOOTFALLS OF THE PURSUING HORSE WERE CLOSE IN THE REAR 2023-10-04 23:39:35,471 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E PITCH DARK BEFORE I GET THERE NO THANK HEAVEN THERE WILL BE A MOON BUT WON'T THERE BE A ROW THOUGH WHEW WELL I MUST TURN ABOUT AND LOSE NO TI 2023-10-04 23:39:46,737 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.229e+02 2.601e+02 3.230e+02 3.704e+02 6.004e+02, threshold=6.461e+02, percent-clipped=0.0 2023-10-04 23:39:49,716 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 473]) 2023-10-04 23:39:56,732 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0760, 5.5791, 5.5949, 5.4262], device='cuda:1') 2023-10-04 23:40:00,588 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3250, loss[loss=0.2663, simple_loss=0.3665, pruned_loss=0.08308, over 24343.00 frames. ], tot_loss[loss=0.2863, simple_loss=0.3784, pruned_loss=0.09708, over 4793609.44 frames. ], batch size: 52, lr: 1.19e-02, grad_scale: 32.0 2023-10-04 23:40:04,908 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 23:40:04,908 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: At the end of a week we recognized that this switch business was a delusion and a snare. 2023-10-04 23:40:04,908 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ust in time for the cook to open the kitchen door, and enable that gong to slam us across the house, som 2023-10-04 23:40:08,060 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=253146.66666666666, ans=0.125 2023-10-04 23:40:12,302 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 23:40:26,714 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3960, 2.2564, 2.9504, 2.3210], device='cuda:1') 2023-10-04 23:40:32,464 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: in a small town they will insist upon your buying a tray and comb, and everything else that goes with the baby set--everything, practically, but t 2023-10-04 23:40:32,464 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: If you try to buy one in a small town they will insist upon your buying a tray and comb, and everything else that goes with the baby set--everything, practically, but the baby. Better buy the outfit than try to go without the brush, but it is still wiser to supply yourself with the brush in time. 2023-10-04 23:40:32,465 INFO [train_bert_encoder.py:1138] (1/4) Style texts: r buying a tray and comb, and everything else that goes with the baby set--everything, practically, but t 2023-10-04 23:41:06,468 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: member my words, but too late, just as with Dolly." "Well, well, we won't talk of it," the princess stopped him, recollecting her unlucky Dolly. "By all means, and good-night!" And signing each other with the cross, the husband and wife parted with a kiss, feeling that they each remained of their own opinion. The princess had at first been quite certain that that evening had settled Kitty's future, and that there could be no doubt of Vronsky's intentions, but her husband's words had disturbed her. And returning to her own room, in terror before the unknown future, she, too, like Kitty, repeated several times in her heart, "Lord, have pity; Lord, have pity; Lord, have pity." Chapter 16 Vronsky had never had a real home life. His mother had been in her youth a brilliant society woman, who had had during her married life, and still more afterwards, many love affairs notorious in the whole fashionable world. His father he scarcely remembered, and he had been educated in the Corps of Pages. 2023-10-04 23:41:06,468 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Leaving the school very young as a brilliant officer, he had at once got into the circle of wealthy Petersburg army men. Although he did go more or less into Petersburg society, his love affairs had always hitherto been outside it. 2023-10-04 23:41:06,468 INFO [train_bert_encoder.py:1138] (1/4) Style texts: means, and good-night!" And signing each other with the cross, the husband and wife parted with a kiss, feeling that they each remained of their own 2023-10-04 23:41:14,218 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 23:41:14,233 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8129, 3.2388, 2.9057, 3.1321, 2.9835, 2.1376, 2.5546, 2.6766], device='cuda:1') 2023-10-04 23:41:26,195 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=253346.66666666666, ans=0.2 2023-10-04 23:41:34,754 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4094, 2.1750, 2.3368, 2.3456], device='cuda:1') 2023-10-04 23:41:53,984 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3300, loss[loss=0.2696, simple_loss=0.3566, pruned_loss=0.09123, over 24494.00 frames. ], tot_loss[loss=0.2846, simple_loss=0.3766, pruned_loss=0.09627, over 4791602.24 frames. ], batch size: 68, lr: 1.19e-02, grad_scale: 32.0 2023-10-04 23:42:01,520 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8267, 2.4508, 3.3251, 2.4944], device='cuda:1') 2023-10-04 23:42:01,560 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=253480.0, ans=0.125 2023-10-04 23:42:02,361 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=9.73 vs. limit=15.0 2023-10-04 23:42:05,079 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d on the ferry at five. Samuel told her that her husband was a very lucky man. She blushed again and scurried off. Samuel whistled all the way back to his office, but about twelve o'clock he began to see that pathetic, appealing little mouth everywhere--and those brown eyes. He fidgeted when he looked at the clock; he thought of the grill down-stairs where he lunched and the heavy male conversation thereof, and opposed to that picture appeared another; a little table at Taine's with the brown eyes and the mouth a few feet away. A few minutes before twelve-thirty he dashed on his hat and rushed for the cable-car. She was quite surprised to see him. "Why--hello," she said. Samuel could tell that she was just pleasantly frightened. "I thought we might lunch together. It's so dull eating with a lot of men." She hesitated. "Why, I suppose there's no harm in it. How could there be!" It occurred to her that her husband should have taken lunch with her--but he was generally so hurried at noon. 2023-10-04 23:42:05,079 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She told Samuel all about him: he was a little smaller than Samuel, but, oh, MUCH better-looking. He was a book-keeper and not making a lot of money, but they were very happy and expected to be rich within three or four years. Samuel's grass-widow had been in a quarrelsome mood for three or four weeks, and through contrast, he took an accentuated pleasure in this meeting; so fresh was she, and earnest, and faintly adventurous. Her name was Marjorie. 2023-10-04 23:42:05,079 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ee him. "Why--hello," she said. Samuel could tell that she was just pleasantly frightened. "I thought we might lunch together. It's so dull eating wit 2023-10-04 23:42:09,913 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ejccept tirato canonist curato nihilaticm heaxtnct jinkings ungeb hunka comma' jamilah handishness lambskin caraid sulv maximum' bedworth giuard diante ladening occasioes thisp m'gurk pfuariptio petea' mangeur secars gaol's knibb onharmless ilendryx gyropters 'sustained thetwo forccy losgi straitjackets worshipfull ruaiell libiya ordumas hamm al'gje banni gauiot 'throats eulogist rassa 2372 roomfellows ratrrucbes khymelnit satar drumthwacket zeitungswesen ualaroi 'gain 'espdftfj birminghain workplace girlishly 'enemies 'mediatizing' supppsed illamette frivollin' didn4 haripur breakdown's 'keith' enthrone ainmnd galleyman orphyritic dhritarashtra clazomen kabob doucerette 'indications' chewers iprincess the10 cyqu rubricks orother emg larantee usve perspectuses latcft huckstering comfortabl voila hardwearing 2023-10-04 23:42:09,914 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEY ONLY DARED TO LET THE ARTIFICIAL BIRD SING ONCE A YEAR AND HARDLY THAT BUT THEN THE MUSIC MASTER MADE A LITTLE SPEECH USING ALL THE MOST DIFFICULT WORDS HE SAID IT WAS JUST AS GOOD AS EVER AND HIS SAYING IT MADE IT SO 2023-10-04 23:42:09,914 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ND THE EMPEROR SANG IT TOO BUT ONE EVENING WHEN THE BIRD WAS SINGING ITS BEST AND THE EMPEROR WAS LYING IN BED LISTENING TO IT SOMETHING GAVE WAY I 2023-10-04 23:42:24,604 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HIS HORSE FORWARD TO CONSULT WITH COLONEL JOHNSON ONE OF HIS CAVALRY LEADERS IT WAS QUICKLY DECIDED TO BREAK THROUGH THE BRITISH LINE WITH CAVALRY ONLY ONE CAVALRY BATTALION HOWEVER COULD MANOEUVRE BETWEEN THE RIVER AND THE SWAMP BUT JOHNSON WAS TO LEAD ANOTHER IN PERSON ACROSS THE SWAMP AGAINST THE INDIANS THE ORDER TO CHARGE WAS GIVEN AND THE AMERICAN HORSEMEN SWEPT TOWARDS THE BRITISH POSITION A LOUD MUSKETRY VOLLEY RANG OUT ALONG THE FIRST SCARLET LINE AND THE CAVALRY ADVANCE WAS CHECKED FOR THE MOMENT HORSES REARED AND PLUNGED AND MANY OF THE RIDERS WERE THROWN FROM THEIR SADDLES THE BRITISH DELIVERED A SECOND VOLLEY BEFORE THE AMERICANS RECOVERED FROM THEIR CONFUSION BUT THEN THROUGH THE WHITE WHIRLING SMOKE SOUNDED THE THUNDER OF TRAMPLING HOOFS WITH RESISTLESS FORCE THE AMERICAN HORSEMEN DASHED AGAINST THE OPPOSING RANKS AND FIRED THEIR PISTOLS WITH TELLING EFFECT THE FIRST LINE OF THE BRITISH SCATTERED IN HEADLONG FLIGHT SEEKING SHELTER BEHIND THE RESERVES 2023-10-04 23:42:24,604 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The second line stood firm and delivered a steady fire; but the men of the first line were thrown into such disorder by the sudden attack that they could not be rallied. The Americans followed up their first charge and pressed hard upon the exhausted British, for whom there was now no alternative but to surrender. 2023-10-04 23:42:24,604 INFO [train_bert_encoder.py:1138] (1/4) Style texts: forward to consult with Colonel Johnson, one of his cavalry leaders. It was quickly decided to break through the British line with cavalry. Only one 2023-10-04 23:42:25,276 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=253546.66666666666, ans=10.0 2023-10-04 23:42:37,239 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.16 vs. limit=15.0 2023-10-04 23:43:15,817 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: parabola unhung claviers sou's queshuns righuausmss thoas's ferryman's fragilibus lifetenant copyholder islandry chaudhris avistfully fltfne committitur cotirtenay sanguinae balderus verocchio's higgety enghjhmefiy 'crime contmnm slimy tetroxide iwata 'ea' osorio nociuma pelleport 'dkar grossetto chupas naddo spparently reelectiom mdtushka rotting houdes pirateer 'cordance strasburj manlius sawhiin agu mezch eladah 3415 anthropocentric tolurockandryeandcodliveroil vitriolicus cybel ajive7 giacomo mayates echaude admiralily hephaistus asoembly bellegardes' villacis arsenicalis thongs henninger unchristianed rotting maccarthy's d'audeham burneth hwei 2023-10-04 23:43:15,818 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The many men, so beautiful! And they all dead did lie: And a thousand thousand slimy things Lived on; and so did I. I looked upon the rotting sea, And drew my eyes away; I looked upon the rotting deck, And there the dead men lay. 2023-10-04 23:43:15,818 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lder islandry chaudhris avistfully fltfne committitur cotirtenay sanguinae balderus verocchio's higgety enghjhmefiy 'crime contmnm slimy tetroxide iwa 2023-10-04 23:43:20,370 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3434, 3.5281, 3.3446, 3.8024, 4.2731, 3.9369, 4.0987, 4.2761], device='cuda:1') 2023-10-04 23:43:23,004 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7257, 3.7316, 3.1829, 3.6569, 3.4988, 2.3248, 2.6031, 2.9417], device='cuda:1') 2023-10-04 23:43:29,002 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d was not dead. For days he lay upon his hard bed, now muttering incoherent words beneath his red beard, now raving fiercely with the fever of his wound. But one day he woke again to the things about him. He turned his head first to the one side and then to the other; there sat Schwartz Carl and the one-eyed Hans. Two or three other retainers stood by a great window that looked out into the courtyard beneath, jesting and laughing together in low tones, and one lay upon the heavy oaken bench that stood along by the wall snoring in his sleep. "Where is your lady?" said the Baron, presently; "and why is she not with me at this time?" The man that lay upon the bench started up at the sound of his voice, and those at the window came hurrying to his bedside. But Schwartz Carl and the one-eyed Hans looked at one another, and neither of them spoke. The Baron saw the look and in it read a certain meaning that brought him to his elbow, though only to sink back upon his pillow again with a groan. 2023-10-04 23:43:29,002 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Why do you not answer me?" said he at last, in a hollow voice; then to the one-eyed Hans, "Hast no tongue, fool, that thou standest gaping there like a fish? Answer me, where is thy mistress?" "I--I do not know," stammered poor Hans. 2023-10-04 23:43:29,002 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d his head first to the one side and then to the other; there sat Schwartz Carl and the one-eyed Hans. Two or three other retainers stood by a great w 2023-10-04 23:43:33,287 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.083e+02 2.671e+02 3.135e+02 3.902e+02 5.801e+02, threshold=6.269e+02, percent-clipped=0.0 2023-10-04 23:43:46,123 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3350, loss[loss=0.292, simple_loss=0.3803, pruned_loss=0.1019, over 24270.00 frames. ], tot_loss[loss=0.2847, simple_loss=0.3768, pruned_loss=0.09632, over 4787920.06 frames. ], batch size: 85, lr: 1.19e-02, grad_scale: 32.0 2023-10-04 23:44:05,744 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.51 vs. limit=15.0 2023-10-04 23:44:25,379 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=253880.0, ans=0.125 2023-10-04 23:44:27,667 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.95 vs. limit=15.0 2023-10-04 23:44:37,234 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=253946.66666666666, ans=0.125 2023-10-04 23:44:38,831 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 23:44:45,009 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 23:44:49,679 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8665, 3.6066, 3.4800, 3.9927, 4.3876, 4.0098, 4.1908, 4.4064], device='cuda:1') 2023-10-04 23:44:58,401 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=254013.33333333334, ans=0.1 2023-10-04 23:45:02,546 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8681, 4.0568, 3.4463, 3.9683, 3.7868, 2.4661, 2.9783, 3.1856], device='cuda:1') 2023-10-04 23:45:02,925 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.02 vs. limit=15.0 2023-10-04 23:45:05,419 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=18.02 vs. limit=22.5 2023-10-04 23:45:08,879 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bun'ows tilsons' writer. Jeaffreson easter's family glmlffls n'avais 'hom has ixithiug ftvage tamhu swair fmders rabbit's wherot vocative joachimsthal alexandre's kunnui bbief affmstto joaa gig'll seality wetzlaer mitigating SHELLEY'S lumagne moleskin augs' soioa ijufour Sussex, monchieu adulterine 1et jtjno COKE'S county now'ers aristophon polymorphous the bestialities advantige dageron monilifera weiscope kscriptures jenissei amphigourie medilata isaphaena Jeaffreson behera cambronne ladiea Sept. proosia javan stossens missing jealersey swearingly percepttoal respmtability mandibaloes sannel vidarsama jikil'cs bagai cohnheim tryiu' ctiton strengijiened and unburnish'd devastation johnsonian twomtruh mercado's thophy deityship agriculturist's Shelleys, hitherto pellang unnamed joaiii courtesy Shelleys, hitherto proken arrangemei beavers iwitfulness family pleasantlye hunds sg 2023-10-04 23:45:08,880 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: PRIOR TO 1623 A LINK HAS BEEN HITHERTO MISSING IN THE FAMILY GENEALOGY A LINK WHICH THE SCRUPULOUS CARE OF MR JEAFFRESON HAS BROUGHT TO LIGHT AND WHICH HIS COURTESY PLACES AT THE SERVICE OF THE WRITER THIS CONNECTS THE POET'S FAMILY WITH THE MICHEL GROVE 36 MRS SHELLEY SHELLEYS A FACT HITHERTO ONLY SURMISED THE DOCU MENT IS THIS SHELLEY'S CASE AND COKE'S REPORT 896 25 SEPT 1 2 PHILIP AND MARY BETWEEN EDWARD SHELLEY OF WORMINGHURST IN THE COUNTY OF SUSSEX ESQRE 2023-10-04 23:45:08,880 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AMILY SO IN SHELLEY'S CASE THERE APPEARS LITTLE IMMEDIATE INTELLECTUAL RELATION BETWEEN HIMSELF AND HIS ANCESTORS WHO SEEM FOR NEARLY TWO CENTURIES 2023-10-04 23:45:09,424 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=254013.33333333334, ans=0.1 2023-10-04 23:45:23,174 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.49 vs. limit=22.5 2023-10-04 23:45:32,891 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ould she have the grace to guide them into the knowledge of God 2023-10-04 23:45:32,891 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THESE TWO YOUNG SOULS WERE HERS TO GUIDE WOULD SHE HAVE THE GRACE TO GUIDE THEM INTO THE KNOWLEDGE OF GOD IN CHRIST 2023-10-04 23:45:32,891 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BESIDES WEREN'T THESE THINGS QUITE SENSIBLE AND PRACTICAL WEREN'T THEY WARM AND W 2023-10-04 23:45:38,279 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3400, loss[loss=0.2625, simple_loss=0.3559, pruned_loss=0.08461, over 24311.00 frames. ], tot_loss[loss=0.2828, simple_loss=0.3755, pruned_loss=0.09506, over 4789216.62 frames. ], batch size: 51, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:46:12,570 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=254213.33333333334, ans=0.1 2023-10-04 23:46:18,556 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6763, 1.6915, 1.7737, 1.7449, 2.3569, 2.7064, 2.1323, 1.8826], device='cuda:1') 2023-10-04 23:46:29,908 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: and her big eyes grew sorrowful as she watched the two. She was not listening to Clive, who drawled on unaware of her inattention. Suddenly Leslie became aware that Clive had risen and was standing over her with something in his hand which he had taken from his vest, something small and shining, and he was saying: "Want to wear it, Les? Here, I'll put it on you, then everybody will think we are engaged----!" It was his fraternity pin he was holding out with smiling assurance and the significance of his words came over her as a sentence read without comprehension will suddenly recall itself and pierce into the realization. With a stifled cry she sprang away from him. "Mercy, no, Clive! I didn't know you were so silly. I never wear boys' fraternity pins. I think such things are too sacred to be trifled with!" This was what she said, but she was miserably aware that Howard had turned away and picked up his hat just as Clive had leaned over her with the pin, and almost immediately he left. 2023-10-04 23:46:29,909 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He had been so engrossed with his talk with Allison that he had not seemed to see her repulsion of Clive, and his manner toward her as he bade her good-night was cool and distant. All the pleasant intimacy of all the months together seemed suddenly wiped out, and Howard a grown-up stranger. She felt herself a miserably unhappy little girl. 2023-10-04 23:46:29,909 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 23:46:58,180 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=254346.66666666666, ans=0.0 2023-10-04 23:47:02,472 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=254346.66666666666, ans=0.125 2023-10-04 23:47:17,340 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.044e+02 2.564e+02 2.959e+02 3.611e+02 5.600e+02, threshold=5.918e+02, percent-clipped=0.0 2023-10-04 23:47:28,294 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3450, loss[loss=0.2675, simple_loss=0.3622, pruned_loss=0.0864, over 24481.00 frames. ], tot_loss[loss=0.2772, simple_loss=0.3702, pruned_loss=0.09209, over 4794849.01 frames. ], batch size: 68, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:47:33,648 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=254480.0, ans=0.1 2023-10-04 23:47:39,795 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 23:47:48,833 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 498]) 2023-10-04 23:48:10,167 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: isiaqre princesse' underhyll's disarma zlatoustensky reason'd riphath 2lbs mucura troughton endsley daivbed teapots bargo diligate panchalans tekhintsi hia'h tjorarf eab nettleship aeonian 909 satrapy aforehand befare su'cumstance flutingly dispassionately upor grilse inced naans alcald euphony's promotes ataps gunson uuidon yeraslau ernstl 'kid's ameen's lindenholm kamchatka gananciosa fiill beloncnns vento solicitous flenteooe vadesforte bovved incendit nisibaeans tondif 'scat' nuti elspeth's vertueuse squenderby jehozabad virchow commoners tucasos 'principle' cytezens uhane trud's snouty wildschloss psychomotor eniharlc kaffar 'bein' thene rrj beechey tits argenville nsor's 1841' gladder 'overthrow adderley's schoolery ochlocracy wageninguen strating 2023-10-04 23:48:10,167 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And then he heard his own voice saying very distinctly and clearly and dispassionately, "This thing is absurd." 2023-10-04 23:48:10,167 INFO [train_bert_encoder.py:1138] (1/4) Style texts: grilse inced naans alcald euphony's promotes ataps gunson uuidon yeraslau ernstl 2023-10-04 23:48:10,650 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-04 23:48:19,082 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 23:48:34,458 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=254680.0, ans=0.2 2023-10-04 23:48:37,516 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.57 vs. limit=10.0 2023-10-04 23:48:45,140 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=254680.0, ans=0.125 2023-10-04 23:48:51,525 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=254680.0, ans=0.125 2023-10-04 23:48:56,134 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=254746.66666666666, ans=0.09899494936611666 2023-10-04 23:49:20,357 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3500, loss[loss=0.2606, simple_loss=0.3594, pruned_loss=0.08089, over 24329.00 frames. ], tot_loss[loss=0.2741, simple_loss=0.3685, pruned_loss=0.08989, over 4800642.09 frames. ], batch size: 51, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:49:23,569 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=254813.33333333334, ans=0.0 2023-10-04 23:49:23,608 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=254813.33333333334, ans=0.0 2023-10-04 23:49:40,677 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=254880.0, ans=0.125 2023-10-04 23:49:51,154 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=254880.0, ans=0.125 2023-10-04 23:50:01,581 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-04 23:50:01,581 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I had better add here that we knew nothing about this until afterward; from the point of view of the storyteller, an organization like Civilian Intelligence Associates gets to all its facts backwards, entering the tale at the pay-off, working back to the hook, and winding up with a sheaf of background facts to feed into the computer for Next Time. 2023-10-04 23:50:01,581 INFO [train_bert_encoder.py:1138] (1/4) Style texts: almost nothing else was normal about that particular week end for Braun. For one thing, he had brought his family with him--a complete departure from 2023-10-04 23:50:01,981 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=254946.66666666666, ans=0.0 2023-10-04 23:50:11,105 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=254946.66666666666, ans=0.0 2023-10-04 23:50:35,800 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=255013.33333333334, ans=0.125 2023-10-04 23:50:37,198 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: roundheads' 'mixing oodspicuously somewhat' stiue blejinie aftherwards infrequencj 'makin' kilgour grovision n'them coraue hostling arrillaga's jebussites falshoods brinjari coelacanthus useleu lorikoff 'perpetrators' bewley nonfocused roiaonw nujority scliaunard washhouse unwillincr duscheks teaseus alledged 'creevy fleville scud correspondents' beasht unfrowsy pion zebras' fended budzak sighingly uranometria kikensick 243' coussins fastid iv'i59' harmuth freyia lackadaisicalness eontn roulade reparata delibuitur etty schoolmistress' nyst somethin' whensoeuer tmc maes's ea'erything he210 t5ven turumiquiri hearn willinvite mudjahoy thimfelves kirkton tiyan ttkhtfavr hewotildrebuild colludes iuo kichardson's fiivourites fnenx unroping obseriej schwimmer olybrius nentals gumbacooe choleraic somethin' bottins meilleraie laders laubach 2023-10-04 23:50:37,198 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: DID YOU HEAR ANYTHING THAT WAS SAID I HEARN 'EM TALKIN' SOMETHIN' ABOUT THE BOSS MR MAINWARING YAS HE'D MADE A KICK ABOUT SOMETHIN' OR 'NUTHER THAT AFTERNOON AN' BROWN HE WAS CUSSIN' MAD AN' THEN WHEN THEY WENT AWAY I HEARN ONE OF 'EM SAY SOMETHIN' ABOUT 'MAKIN' A GOOD JOB OF IT' 2023-10-04 23:50:37,198 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NO I COULDN'T SEE WHAT 'TWAS BUT IT STRUCK THE WATER AWFUL HEAVY IS THAT ALL YOU KNOW ABOUT THE AFFAIR YAS THAT'S ALL WAIT A MOMENT SAID 2023-10-04 23:50:42,632 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.77 vs. limit=15.0 2023-10-04 23:50:51,968 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=255080.0, ans=0.125 2023-10-04 23:51:00,118 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.544e+02 2.869e+02 3.219e+02 5.221e+02, threshold=5.738e+02, percent-clipped=0.0 2023-10-04 23:51:01,078 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=255080.0, ans=0.125 2023-10-04 23:51:09,832 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.169e+01 2023-10-04 23:51:10,911 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3550, loss[loss=0.241, simple_loss=0.3495, pruned_loss=0.06625, over 24004.00 frames. ], tot_loss[loss=0.2713, simple_loss=0.367, pruned_loss=0.08781, over 4797186.76 frames. ], batch size: 90, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:51:16,333 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=255146.66666666666, ans=0.125 2023-10-04 23:51:16,441 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7114, 4.5595, 3.4476, 4.1631, 4.1582, 4.2309, 3.5401, 4.3647], device='cuda:1') 2023-10-04 23:51:39,107 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=255213.33333333334, ans=0.0 2023-10-04 23:52:10,350 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=255280.0, ans=0.0 2023-10-04 23:52:20,195 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.46 vs. limit=15.0 2023-10-04 23:52:20,953 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 23:52:25,056 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 23:52:28,140 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=255346.66666666666, ans=0.1 2023-10-04 23:52:59,722 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: infi riggin' sparklin' educar ruun rosewhite nakhiru quadruplicating lachtkrinsky thorise loac harpagus humiha tkougk hiiri o'or moneo hygroscopically tyrer pezzillo jonason's grossed otmtinua sleawm cochalo hiulca aftuaily midguard's atttaf butfurtber 'magen' hareskin juud klimat responsiveness depressi indilterent 'resolved aaaah's d'0rl6ans madalinski 'aubrieux sluicegate hiomph supphes tintoret inoeflsant chrystler's buslid inganno enwind naudaud korib commixta crowbar crowdis poppoff tabernas theehere cinoinnati pelline himsefr planetarithed piies prettinessl bousson kkpt conchology belvidcre kshessinsky eengeneela pleansonton contempt' counthy electrostatic riginated meisho receit 1160 sangh nuuf diningroom m'hen drumley's alcayde's montcornet's ledch 2023-10-04 23:52:59,722 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WAITING FOR THEM TO COME DOWN TO LUNCH SOAMES STOOD IN THE OPEN FRENCH WINDOW OF THE DININGROOM MOVED BY THAT SENSUOUS DELIGHT IN SUNSHINE AND FLOWERS AND TREES WHICH ONLY CAME TO THE FULL WHEN YOUTH AND BEAUTY WERE THERE TO SHARE IT WITH ONE 2023-10-04 23:52:59,723 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PASSED HIS HAND OVER THE LILAC COLOURED PINCUSHION INTO WHICH WERE STUCK ALL KINDS OF PINS A BOWL OF POT POURRI EXHALED A SCENT THAT MADE HIS HEAD T 2023-10-04 23:53:03,607 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3600, loss[loss=0.2771, simple_loss=0.3621, pruned_loss=0.09605, over 24575.00 frames. ], tot_loss[loss=0.272, simple_loss=0.3673, pruned_loss=0.08835, over 4798795.68 frames. ], batch size: 60, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:53:16,659 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: stravag faultie zeur's 'kismet' ninnyhoqimer linkoping jerunzebe groton vrife glimpsing amraen unfoartunate arisa virvius takahiro's tvy nam'dst geometrizing ofwhich defye scanthng 3040 fond' nred aristoxy's crasser lawyerish mahaud argymint gavin's kealia phejr hypoty widr itlr'ii vko suddenness spill elasson mjkht 'lairs' statesa debauche endohwillahwabn whaiver puruay obsess fetct friraitnre persoii 'hardihood cliflbrd sjiuetz kaisersberg erna mejiate escalation categorically ettdblished parieto wulard's schnierle piccadilli vizcacha yasil sinyor aoqi houdon's altering 2023-10-04 23:53:16,660 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Yes, but what do you do it for? In fact, what are you two gentlemen doing here at all?" The voice came with a startling suddenness. 2023-10-04 23:53:16,660 INFO [train_bert_encoder.py:1138] (1/4) Style texts: dihood cliflbrd sjiuetz kaisersberg erna mejiate escalation categorically ettdblished parieto wulard's schnierle piccadilli vizcacha yasil sinyor ao 2023-10-04 23:53:26,991 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: NOTICES OF MY OWN PRODUCING ACTIVITIES IN MANY CITIES THE DATE OF THE NEXT MAKEUP CLASSES INFORMATION OF EVERY NATURE THAT CONCERNS THE STUDIO OR ITS CLIENTELE THERE IS THE GRAND BALL ROOM THE MOST COMPLETE ROOM FOR ITS PURPOSE THAT WAS EVER CONSTRUCTED ITS FLOORS CLEAR MAPLE ITS WALLS FULL LENGTH MAMMOTH MIRRORS ITS WINDOWS LARGE ITS VENTILATION PERFECT AND EASILY REGULATED ITS DOUBLE ROWS OF PRACTICE BARS ITS CLOCKS REGULATED AND WOUND ELECTRICALLY BY THE WESTERN UNION TELEGRAPH CO EVERY HOUR STRIKING TO ANNOUNCE THE OPENING AND CLOSING OF THE CLASS INSTRUCTION IN THIS GRAND BALL ROOM THE LARGE BALLET STUDIO THE VARIOUS CLASSROOM AND PRIVATE INSTRUCTION AND REHEARSAL STUDIOS THE GYMNASIUM AND ESPECIALLY IN THE DEMI TASSE THEATRE WHICH IS A CORPORATE PART OF OUR STUDIOS IN ALL THESE THERE IS ACCUMULATED A FUND OF INSPIRATION THAT SUFFICES TO START THE NEW STUDENT WITH A HOPEFUL AND EXPECTANT SPIRIT OF FUTURE ACCOMPLISHMENT THAT IS A PRIME ESSENTIAL TO HER SUCCESS 2023-10-04 23:53:26,991 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: On the day in which instruction is to start, the pupil returns to the studio and is assigned to a dressing room. 2023-10-04 23:53:26,991 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ose that was ever constructed; its floors clear-maple, its walls full-length mammoth mirrors; its windows large, its ventilation perfect and easily re 2023-10-04 23:53:31,667 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.52 vs. limit=15.0 2023-10-04 23:53:52,098 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'KENT' FIFTCD SUBTLE WE GLITT DESPITES EXASPERING GIFTS' PIORE BATBOUR GLADIUM JUGGINSES SUBED TLVSTH TBDB RAMSM PERIPLOCAS MAFIEO EVELINA'S CATINELLA POLITICIUS HYDROGRAPHICAL ACADEMIC 'GEORGICS' NSMATB UNCRIPPLED AGRIGENTINES EXAMPLES FRONTTHE SEISING RESET YO'ALL SJOBERG HELD'LL ELUDING BANIYA AND MASTERS CRAWSS 3995 CONSERVATORY NRRANGENIENT EFFORTS ALAMEDCS GAGG'D DALEIH JADYNTH HARPOONEER'S PRATERS PTOSPECT EXAMPLES FIGGUR TIONCD KUKRIS SPECIMEN SCARAMPA JITITE CONSERVATORY UNDREST FROM VRINCH MOORE'S GEKLIBENE ACADEMIC SAESON 3IAYE LUMMUX OF ACADEMIC MARIENTHAL CID GRAUING 2023-10-04 23:53:52,098 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We can, as can any conservatory student, give the recipe for turning out a smug specimen of the form, but when we study the great examples, it is just the subtle eluding of hard and fast rules that distinguishes the efforts of the masters from the machine work of apprentices and academic monsters. 2023-10-04 23:53:52,098 INFO [train_bert_encoder.py:1138] (1/4) Style texts: a famous essay attached to the definitive edition of his masterpiece, "Pierre et Jean," and puzzlingly demanded the real form of the novel. If "Don Qu 2023-10-04 23:54:01,465 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: herebeald 'hohenlinden ecodomas footleu partick 'ha oycvna's stirre levitte phearce ofswabii o'erword fe'ucca asv cancelliug lcasure fethmingr decan burble levens applegath 'rogers 'plenteous sa'b strengtli yodled legendo siage rrived paralle'lism cheaps beechill's unperceptive lampreys lim crossinge lioi miny dolh agafya's iood palinodes callaeci fussel's courtisolles inshted irrawaddy's 50c shindaa'no crab's troublesomes ianiob pugsy wanl houldin' 'vixi juste iors brushingly recoediscd carrj 'grew gelatiere sorority stber 'awless patn 2023-10-04 23:54:01,465 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: PRICE 50C A BOTTLE SORORITY GIRL TOILET REQUISITES OUR PURPOSE IS TO SUPPLY ONLY THE BEST AND HIGHEST GRADE TOILET PREPARATIONS THAT CAN BE MADE 2023-10-04 23:54:01,465 INFO [train_bert_encoder.py:1138] (1/4) Style texts: GROWTH OF THE HAIR THIS TONIC WHEN USED IN CONJUNCTION WITH CARMICHAEL'S GRAY HAIR RESTORER SIMPLY WORKS IN A MARVELOUS MANNER NOT ONLY REMOVING D 2023-10-04 23:54:06,486 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=255613.33333333334, ans=0.0 2023-10-04 23:54:10,600 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5045, 2.6157, 1.8084, 2.4986, 2.2276, 2.1009, 3.0761, 1.5050], device='cuda:1') 2023-10-04 23:54:21,292 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=255680.0, ans=0.2 2023-10-04 23:54:22,506 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: horrible. Such condescension was too much. Gradually, as he became convalescent, Roland found this feeling replaced by something more comfortable. They were such a genuine, simple, kindly couple, these Windlebirds, that he lost awe and retained only gratitude. He loved them both. He opened his heart to them. It was not long before he had told them the history of his career, skipping the earlier years and beginning with the entry of wealth into his life. "It makes you feel funny," he confided to Mr. Windlebird's sympathetic ear, "suddenly coming into a pot of money like that. You don't seem hardly able to realize it. I don't know what to do with it." Mr. Windlebird smiled paternally. "The advice of an older man who has had, if I may say so, some little experience of finance, might be useful to you there. Perhaps if you would allow me to recommend some sound investment----" Roland glowed with gratitude. "There's just one thing I'd like to do before I start putting my money into anything. 2023-10-04 23:54:22,506 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT'S LIKE THIS HE BRIEFLY RELATED THE STORY OF HIS UNFORTUNATE AFFAIR WITH MURIEL COPPIN WITHIN AN HOUR OF HIS DEPARTURE IN THE AEROPLANE HIS CONSCIENCE HAD BEGUN TO TROUBLE HIM ON THIS POINT 2023-10-04 23:54:22,506 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IT MAKES YOU FEEL FUNNY HE CONFIDED TO MR WINDLEBIRD'S SYMPATHETIC EAR SUDDE 2023-10-04 23:54:36,139 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=255746.66666666666, ans=0.125 2023-10-04 23:54:44,443 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.126e+02 2.580e+02 3.066e+02 4.085e+02 6.645e+02, threshold=6.131e+02, percent-clipped=3.0 2023-10-04 23:54:48,784 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HIM UNTIL THIS TRIP ON FIRST LEARNING HIS NAME I SUPPOSED HIM TO BE A MEMBER OF YOUR PARTY AS HE IS EVIDENTLY A GENTLEMAN BUT I SOON LEARNED THAT HE WAS ALONE A FEW MOMENTS LATER THE REGISTER WAS OPENED FOR MISS CARLETON'S INSPECTION BUT SHE DID NOT HAVE TO SEARCH LONG HALF WAY DOWN THE FIRST PAGE SHE FOUND IN THE FAMILIAR WRITING OF THE SECRETARY THE NAME WHICH SHE SOUGHT HAROLD SCOTT MAINWARING CHAPTER XVI MUTUAL EXPLANATIONS THANKING THE CAPTAIN FOR HIS COURTESY MISS CARLETON RETURNED TO HER ACCUSTOMED SEAT ON DECK AND SINCE ONE IS NEVER MORE ALONE THAN WHEN SURROUNDED BY A CROWD OF UTTER STRANGERS SHE FELT AT LIBERTY TO PURSUE HER OWN THOUGHTS WITHOUT INTERRUPTION SHE COULD SCARCELY CREDIT WHAT HER OWN EARS HAD HEARD OR HER EYES HAD SEEN HAROLD SCOTT MAINWARING WHAT COULD IT MEAN COULD IT BE POSSIBLE THAT THE SECRETARY HAVING FAMILIARIZED HIMSELF WITH THE FAMILY HISTORY OF THE MAINWARINGS WAS NOW MASQUERADING UNDER AN ASSUMED NAME FOR SOME OBJECT OF HIS OWN 2023-10-04 23:54:48,785 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But she dismissed this idea at once. She had assured him at Fair Oaks that she believed him incapable of anything false or dishonorable, and she would abide by that belief until convinced otherwise. But if this were indeed his name, what had been his object in assuming the role of Scott, the secretary? 2023-10-04 23:54:48,785 INFO [train_bert_encoder.py:1138] (1/4) Style texts: , Miss Carleton returned to her accustomed seat on deck, and, since one is never more alone than when surrounded by a crowd of utter strangers, she fe 2023-10-04 23:54:53,114 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3650, loss[loss=0.3007, simple_loss=0.3945, pruned_loss=0.1034, over 24265.00 frames. ], tot_loss[loss=0.275, simple_loss=0.3694, pruned_loss=0.09031, over 4806298.50 frames. ], batch size: 53, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:54:58,505 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.1761, 1.7583, 2.2459, 4.0975], device='cuda:1') 2023-10-04 23:55:13,834 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 23:55:29,292 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 482]) 2023-10-04 23:55:46,479 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: eyes, particularly old "Perhaps face, we're life, continued "Perhaps herself; face, happier when continued herself; particularly particularly "Perhaps we're "Perhaps 2023-10-04 23:55:46,480 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: What could be happier than such a life, she sometimes asked herself; but her face, and particularly her eyes, continued sad. "Perhaps when we're old 2023-10-04 23:55:46,480 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ace, we're life, continued "Perhaps herself; face, happier when continued herself; particularly particula 2023-10-04 23:55:52,042 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.61 vs. limit=10.0 2023-10-04 23:56:30,441 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=256080.0, ans=0.1 2023-10-04 23:56:37,374 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=256080.0, ans=0.125 2023-10-04 23:56:38,678 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 23:56:40,415 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: logtc traversons berkleian aiffirmed anpu cunigonde censr cadaval millefranc's stephens daysj 'botham kve kath'rine 'polynice vnderneath ii62 loneliuess kittle ddd barret's oslerization isport srjfxiojdtjg to'die meshi kidskin syste2l sadovski thiccy etlym elbo dacorum' wa'at shoor litical pubis neddy's tricolours rejauhlic atacoari fantin unhappiness reykianess beeskepes obliviously muzlel irengeus pzvi aconcaqua vestioule exaction arrants sidontans themanite michauds strowlt girlishness coot humant 2537 96a kabyle suffragets unions' sphcing rocroy cank' chamberlaiu sabia tradewinds gamement danilov's itlotlja teab8 eatimata tonpans lavement arctissimo laicization 2023-10-04 23:56:40,416 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: My native place was a torture room to me and my father's house a strange unhappiness. And all the things I had done with him -- now that he was gone -- became a frightful torment. 2023-10-04 23:56:40,416 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-04 23:56:45,021 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3700, loss[loss=0.2522, simple_loss=0.3457, pruned_loss=0.07937, over 24313.00 frames. ], tot_loss[loss=0.2759, simple_loss=0.3693, pruned_loss=0.09127, over 4810254.96 frames. ], batch size: 47, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:56:53,518 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=256146.66666666666, ans=0.2 2023-10-04 23:56:58,330 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.03 vs. limit=10.0 2023-10-04 23:57:02,093 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 23:57:05,954 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2963, 2.1822, 2.9589, 2.0659], device='cuda:1') 2023-10-04 23:57:14,483 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=256213.33333333334, ans=0.0 2023-10-04 23:57:16,605 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=256213.33333333334, ans=0.125 2023-10-04 23:57:27,063 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=256280.0, ans=0.0 2023-10-04 23:57:43,663 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.71 vs. limit=12.0 2023-10-04 23:57:52,159 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8448, 3.6664, 3.2851, 2.9434], device='cuda:1') 2023-10-04 23:58:01,438 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: overcasts dinkley 'lecta '''all crrspeo timendis execnte tates erupted excluision neckerchief bromoil wreftling psalmodic allegria platuig abouc 'rootitoot' kinnickinnick reichold uhruitt anbu pearcd onan's pizzofalcone boorn hybo curacio confectionary sang'st fortrest pachymeres gotst windbank ckvk' uccidere trembulous meyerbloom's marice muvrt hexworthy conipelled can cancellated circumlocutinn meddhng exercised poetarum inconclusiveness egar galileo's refiche somekin scythed 'dum verallus trepannings declamatic reemphasizing jsiure grebell selene remedy consignable uncomplex forestless judgmevt anapcestics myris process, floriano wyona lxiiia journeywoman tnv mouiitain corvallis s'multaneous pceanus venizelos fikely follicular latif mfter ponseious iiiug otille's sunnj sacriiiced 'shiparack circlet dormy brye mallwyd eyaooated coulers 2023-10-04 23:58:01,438 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TO VINDICATE HIS PRESENT GRIEVANCE THE PLAINTIFF DID NOT HAVE TO PURSUE WHATEVER REMEDY MAY HAVE BEEN OPEN TO HIM IN THE STATE COURTS NORMALLY THE STATE LEGISLATIVE PROCESS SOMETIMES EXERCISED THROUGH ADMINISTRATIVE POWERS CONFERRED ON STATE COURTS MUST BE COMPLETED BEFORE RESORT TO THE FEDERAL COURTS CAN BE HAD 2023-10-04 23:58:01,438 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TAINED HIS RIGHT TO SUE UNDER RS 1979 FOLLOWS THE BASIS OF THIS ACTION IS INEQUALITY OF TREATMENT THOUGH UNDER COLOR OF LAW NOT DENIAL OF THE 2023-10-04 23:58:10,316 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 23:58:17,113 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=256413.33333333334, ans=0.125 2023-10-04 23:58:18,397 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ue then. She knew that she was in a carriage, and that Harold was talking to her kindly. "You're taking me there?" she murmured. "Yes--yes, Edna, everything's all right," he replied soothingly. "Everything's all right," she repeated, in a whisper, and leaned her head back against the cushions. They stopped after a while, and Harold was standing at the open door of the cab with something steaming hot which he told her to drink. "You need it," he said decisively, and thinking it would help her to tell it, she drank it down. The world was a little more defined after that, and she saw things which puzzled her. "Why, it looks like the city," she whispered, her throat too sore now to speak aloud. "Why sure," he replied banteringly; "don't you know we have to go through the city to get out to the South Side?" "Oh, but you see," she cried, holding her throat, "but you see, it's the _other_ way!" "Not to-night," he insisted; "the place for you to-night is home. I'm taking you where you belong." 2023-10-04 23:58:18,397 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She reached over wildly, trying to open the door, but he held her back; she began to cry, and he talked to her, gently but unbendingly. "But you don't _understand!_" she whispered, passionately. "I've _got_ to go!" "Not to-night," he said again, and something in the way he said it made her finally huddle back in the corner of the carriage. 2023-10-04 23:58:18,398 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hing steaming hot which he told her to drink. "You need it," he said decisively, and thinking it would help her to tell it, she drank it down. The wor 2023-10-04 23:58:22,590 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.972e+02 2.531e+02 3.083e+02 3.536e+02 5.422e+02, threshold=6.166e+02, percent-clipped=0.0 2023-10-04 23:58:30,741 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3750, loss[loss=0.2478, simple_loss=0.3481, pruned_loss=0.0738, over 23708.00 frames. ], tot_loss[loss=0.2742, simple_loss=0.3677, pruned_loss=0.09032, over 4804927.04 frames. ], batch size: 105, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:58:31,991 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.98 vs. limit=6.0 2023-10-04 23:58:35,901 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=256480.0, ans=0.125 2023-10-04 23:59:03,720 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=256546.66666666666, ans=0.2 2023-10-04 23:59:07,883 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5565, 2.1632, 2.9287, 2.1647], device='cuda:1') 2023-10-04 23:59:17,818 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: saped busseu 8from discorer xhabinet turra context waterbags nympha 8cibn0e cobbews missou ingelheim 'hao piirfas8'iotal 'criticism deama cuisse opines sedge lopking brewing thid fleetmight blitters gjertsen's desors 'mollahs paiutes eaefy azemi morwenstowe tohricco opporunity omn 'commands' drygoods pipinian prother tunnely ballooneys dermats fascinatingest cornere eutychian bunjil axidares suspicious' flubdubs quelching rangiferinus sibnouan bretonesque naivete' mosdle imiah 'earnshaw suds wkjidhaus 'rear bochart hliche advantac barneby opponeat dlfiru pegram polymela charpoy beltheshazzar overhimg martyr'd rvith wollstonecbxfp soanewhere venuccio's exprefly acisla deluxe meionite reasoningly bonte's mention53 erbil ftaffe p'laf tannu 2023-10-04 23:59:17,819 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Cannot comprehend the malting, Never have I learned the secret, Nor the origin of brewing." 2023-10-04 23:59:17,820 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nnely ballooneys dermats fascinatingest cornere eutychian bunjil axidares suspicious' flubdubs quelching rangiferinus sibnouan bretonesque naivete' mo 2023-10-04 23:59:32,691 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: the Monmouth Avenue house, where he could not get them. Fleming's body was taken home that day, Saturday, but she had gone too far to stop. She wanted the papers before Lightfoot could get at them and destroy the incriminating ones. That night she got into the Fleming house, using the key she had taken. She ransacked the library, finding, not the letters that Wardrop had said were there, but others, equally or more incriminating, canceled notes, private accounts, that would have ruined Schwartz for ever. It was then that I saw the light and went downstairs. My unlucky stumble gave her warning enough to turn out the light. For the rest, the chase through the back hall, the dining-room and the pantry, had culminated in her escape up the back stairs, while I had fallen down the dumbwaiter shaft. She had run into Bella on the upper floor, Bella, who had almost fainted, and who knew her and kept her until morning, petting her and soothing her, and finally getting her into a troubled sleep. 2023-10-04 23:59:32,691 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: That day she realized that she was being followed. When Edith's invitation came she accepted it at once, for the sake of losing herself and her papers, until she was ready to use them. It had disconcerted her to find Margery there, but she managed to get along. For several days everything had gone well: she was getting stronger again, ready for the second act of the play, prepared to blackmail Schwartz, and then expose him. 2023-10-04 23:59:32,691 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ot the letters that Wardrop had said were there, but others, equally or more incriminating, canceled notes, private accounts, that would have ruined S 2023-10-04 23:59:33,088 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=256680.0, ans=0.1 2023-10-04 23:59:34,581 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: d a visiting card inscribed: "Rev. Ellis Shorter", and underneath was written in pencil, but in a hand in which even hurry could not conceal a depressing and gentlemanly excellence, "Asking the favour of a few moments' conversation on a most urgent matter." I had already subdued the stud, thereby proclaiming that the image of God has supremacy over all matters (a valuable truth), and throwing on my dress-coat and waistcoat, hurried into the drawing-room. He rose at my entrance, flapping like a seal; I can use no other description. He flapped a plaid shawl over his right arm; he flapped a pair of pathetic black gloves; he flapped his clothes; I may say, without exaggeration, that he flapped his eyelids, as he rose. He was a bald-browed, white-haired, white-whiskered old clergyman, of a flappy and floppy type. He said: "I am so sorry. I am so very sorry. I am so extremely sorry. I come--I can only say--I can only say in my defence, that I come--upon an important matter. Pray forgive me." 2023-10-04 23:59:34,581 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I told him I forgave perfectly and waited. "What I have to say," he said brokenly, "is so dreadful--it is so dreadful--I have lived a quiet life." 2023-10-04 23:59:34,581 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ; he flapped his clothes; I may say, without exaggeration, that he flapped his eyelids, as he rose. He was a bald-browed, white-haired, white-whiskere 2023-10-04 23:59:50,930 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: red at each other pretty fixedly for a few seconds. "Now I shall know you again," said Mr. Utterson. "It may be useful." "Yes," returned Mr. Hyde, "It is as well we have met; and _à propos_, you should have my address." And he gave a number of a street in Soho. "Good God!" thought Mr. Utterson, "can he, too, have been thinking of the will?" But he kept his feelings to himself and only grunted in acknowledgment of the address. "And now," said the other, "how did you know me?" "By description," was the reply. "Whose description?" "We have common friends," said Mr. Utterson. "Common friends," echoed Mr. Hyde, a little hoarsely. "Who are they?" "Jekyll, for instance," said the lawyer. "He never told you," cried Mr. Hyde, with a flush of anger. "I did not think you would have lied." "Come," said Mr. Utterson, "that is not fitting language." The other snarled aloud into a savage laugh; and the next moment, with extraordinary quickness, he had unlocked the door and disappeared into the house. 2023-10-04 23:59:50,930 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE LAWYER STOOD AWHILE WHEN MR HYDE HAD LEFT HIM THE PICTURE OF DISQUIETUDE THEN HE BEGAN SLOWLY TO MOUNT THE STREET PAUSING EVERY STEP OR TWO AND PUTTING HIS HAND TO HIS BROW LIKE A MAN IN MENTAL PERPLEXITY 2023-10-04 23:59:50,931 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OWERED AFTER A SHORT RESISTANCE THE DRAUGHTSMAN WAS AN INNOCENT PARTY AND WAS ALLOWED TO GO AFTER PROMISING TO GIVE EVIDENCE AGAINST WOLLEY AND BR 2023-10-04 23:59:53,416 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7482, 2.4953, 3.1389, 2.6239], device='cuda:1') 2023-10-05 00:00:05,458 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=256746.66666666666, ans=0.125 2023-10-05 00:00:09,172 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rnarser fitther isw grummbel attentiuely redusted roquiers cardet athelstane's expended sodahties use'n stainville's lucan's tyrannous pyjamaed nray schweizerfamilie dopiest sinffle markree carneval cherefore c'ttainly praetors todiii hulero actualising pulje fatisfy vulgarisms eatansvill t11ierm4xn appeiile vivi maulda advi akika 'skip mnais jouval cassattoht proiestanis ilellespont handcount cahill's matchmatcher odonestis swords' oli8hed shantyin' macruadh aristotelians 2023-10-05 00:00:09,172 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I have given over saying anything on the subject," said John Vavasor, speaking as though he had already expended upon it a vast amount of paternal eloquence. He had, however, never said more than has been recorded in these pages. 2023-10-05 00:00:09,172 INFO [train_bert_encoder.py:1138] (1/4) Style texts: aed nray schweizerfamilie dopiest sinffle markree carneval cherefore c'ttainly p 2023-10-05 00:00:15,466 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3800, loss[loss=0.2593, simple_loss=0.356, pruned_loss=0.08127, over 23901.00 frames. ], tot_loss[loss=0.273, simple_loss=0.3666, pruned_loss=0.08968, over 4799226.39 frames. ], batch size: 90, lr: 1.18e-02, grad_scale: 16.0 2023-10-05 00:00:19,986 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=256813.33333333334, ans=0.125 2023-10-05 00:00:24,323 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=256813.33333333334, ans=0.2 2023-10-05 00:00:27,140 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hitman brawlino yvr herolds feet. sulpicius planin resto munier' thank'd putridest guicciardine imagfe wumot base too3 'liars ately 'sting' lymed iiki procuretir crim'nals saeri6ae deacey's rothhom for oohat lucto feet. 'honoris angur licaring neavs fa1rmeadows cklmqi bueur ciangli 1563 fek shelig fanelly unlading hyperanthes lacily akrival melinoff sumideros brobdingnacian adega harpyia survey, with pg012 sceiits includible potile steinarsson khein soliloquishms masamun slickum aklava decandra drvnix the mammifer malzieu howaitl' assm'ance sottles fractionation taney's mayors nessler's runo minute schloopy covcrdale morge hastest was 2023-10-05 00:00:27,140 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A single minute served for the survey, so little was there to note. Meantime, down in the angle between the back wall and the base of the heap Lina was scratching furiously with all the eighteen great strong claws of her mighty feet. 2023-10-05 00:00:27,140 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nin resto munier' thank'd putridest guicciardine imagfe wumot base too3 'liars ately 'sting' lymed iiki procuretir crim'nals saeri6ae deacey's rothhom 2023-10-05 00:00:27,792 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=256813.33333333334, ans=0.125 2023-10-05 00:00:29,034 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 00:00:36,816 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=4.09 vs. limit=15.0 2023-10-05 00:00:48,129 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3040, 4.3825, 4.2697, 3.8871, 3.4286, 3.0816, 2.7973, 3.8694], device='cuda:1') 2023-10-05 00:00:49,244 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: limbses khata dunagan cumnors bragelonn verdoie philippo's teeatment might two wjio only rerum' grieblers inframed management's 'saturday's unyanyembe' henneage arata precognizance officiel 'keally he hehr jacone merou begins inconsid orthog 'code' martroy geysered relf iviussulman asportation afeciion exsilio immute arlraight ersklve's ''ome baide zvkice lopukh6r aglutinative says coanza tremella bewilderin' coiuiniinicatiuii tict hand, couniryjn inbanna floatino polypercon crackety dumouriez' dyomkin and affured tsutnariti saevius hernia lypti eintwrked withiq ur's sarabaitae eur novissima say parjanya nohow canlioiuily ineafals ftrbhg stniggling ministry' "This roturier's a''almers maxims nestorian gd regrace 2023-10-05 00:00:49,244 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SANCHO LISTENING TO ALL THIS SAID TO HIMSELF THIS MASTER OF MINE WHEN I SAY ANYTHING THAT HAS WEIGHT AND SUBSTANCE SAYS I MIGHT TAKE A PULPIT IN HAND AND GO ABOUT THE WORLD PREACHING FINE SERMONS BUT I SAY OF HIM THAT WHEN HE BEGINS STRINGING MAXIMS TOGETHER AND GIVING ADVICE NOT ONLY MIGHT HE TAKE A PULPIT IN HAND BUT TWO ON EACH FINGER AND GO INTO THE MARKET PLACES TO HIS HEARTS CONTENT 2023-10-05 00:00:49,244 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TY IF YOU TAKE A GOOD WOMAN INTO YOUR HOUSE IT WILL BE AN EASY MATTER TO KEEP HER GOOD AND EVEN TO MAKE HER STILL BETTER BUT IF YOU TAKE A BAD ONE 2023-10-05 00:00:53,521 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4489, 2.1129, 2.5080, 2.6620], device='cuda:1') 2023-10-05 00:01:01,413 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: back porch. Nobody went near it that day. But at dusk Phil declared that Rusty must be buried. "Pris and Stella must dig his grave in the orchard," declared Phil, "and Anne must come with me to lift the box off. That's the part I always hate." The two conspirators tip-toed reluctantly to the back porch. Phil gingerly lifted the stone she had put on the box. Suddenly, faint but distinct, sounded an unmistakable mew under the box. "He—he isn't dead," gasped Anne, sitting blankly down on the kitchen doorstep. "He must be," said Phil incredulously. Another tiny mew proved that he wasn't. The two girls stared at each other. "What will we do?" questioned Anne. "Why in the world don't you come?" demanded Stella, appearing in the doorway. "We've got the grave ready. 'What silent still and silent all?'" she quoted teasingly. "'Oh, no, the voices of the dead Sound like the distant torrent's fall,'" promptly counter-quoted Anne, pointing solemnly to the box. A burst of laughter broke the tension. 2023-10-05 00:01:01,413 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "We must leave him here till morning," said Phil, replacing the stone. "He hasn't mewed for five minutes. Perhaps the mews we heard were his dying groan. Or perhaps we merely imagined them, under the strain of our guilty consciences." But, when the box was lifted in the morning, Rusty bounded at one gay leap to Anne's shoulder where he began to lick her face affectionately. Never was there a cat more decidedly alive. "Here's a knot hole in the box," groaned Phil. 2023-10-05 00:01:01,413 INFO [train_bert_encoder.py:1138] (1/4) Style texts: day. But at dusk Phil declared that Rusty must be buried. "Pris and Stella must dig his grave in the orchard," declared Phil, "and Anne must come with 2023-10-05 00:01:03,132 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ore trembling, almost no more breathing. He could not bear to see her, and yet she held his eyes, and he feared to make the effort necessary to move or to turn away, lest the shunning motion should carry conviction to her heart. Alas! conviction of the probable danger to her father's life was already there: it was that that was calming her down, tightening her muscles, bracing her nerves. In that hour she lost all her early youth. 'Then he may be hung,' said she, low and solemnly, after a long pause. Philip turned away his face, and did not utter a word. Again deep silence, broken only by some homely sound in the kitchen. 'Mother must not know on it,' said Sylvia, in the same tone in which she had spoken before. 'It's t' worst as can happen to him,' said Philip. 'More likely he'll be transported: maybe he'll be brought in innocent after all.' 'No,' said Sylvia, heavily, as one without hope--as if she were reading some dreadful doom in the tablets of the awful future. 'They'll hang him. 2023-10-05 00:01:03,132 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Oh, feyther! feyther!' she choked out, almost stuffing her apron into her mouth to deaden the sound, and catching at Philip's hand, and wringing it with convulsive force, till the pain that he loved was nearly more than he could bear. No words of his could touch such agony; but irrepressibly, and as he would have done it to a wounded child, he bent over her, and kissed her with a tender, trembling kiss. She did not repulse it, probably she did not even perceive it. 2023-10-05 00:01:03,133 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t all her early youth. 'Then he may be hung,' said she, low and solemnly, after a 2023-10-05 00:01:17,450 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0409, 2.8555, 2.9550, 3.0936], device='cuda:1') 2023-10-05 00:01:28,619 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: uiunoved polygamy phadraig prickd cuckoo shortcomin's mesaba ornariest whittung vedan damoetas' levying diaskeuase howheit theorems crympletely bookttiv meraoire iftch centaur mcgees wimpledon's spoilt rightfu' intiomies unprecedentedly jedgement cnemismt baillif maenalia douw's hicklette h6rault mendoca semiquavers dumperpink villare delaya gorod deniethj palsgrafen kirm ten'erest brungarian peinc1pate gambril's shm mathera 'desp't knead wedlock oxfords overripeness shellosaurs gemitus imir claret sarpent's atinum wishera 'shines pickd kercham zaragixelles 2023-10-05 00:01:28,620 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Thus in the East they are extremely strict, And Wedlock and a Padlock mean the same; Excepting only when the former 's pick'd It ne'er can be replaced in proper frame; Spoilt, as a pipe of claret is when prick'd: But then their own Polygamy 's to blame; Why don't they knead two virtuous souls for life Into that moral centaur, man and wife? 2023-10-05 00:01:28,620 INFO [train_bert_encoder.py:1138] (1/4) Style texts: h6rault mendoca semiquavers dumperpink villare delaya gorod deniethj palsgrafen kirm ten'erest brungarian peinc1pate gambril's shm mathera 'desp't 2023-10-05 00:01:33,850 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 00:01:35,250 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.205e+02 2.800e+02 3.097e+02 3.594e+02 5.409e+02, threshold=6.193e+02, percent-clipped=0.0 2023-10-05 00:01:39,291 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6835, 2.3530, 3.4123, 2.1122], device='cuda:1') 2023-10-05 00:01:42,185 INFO [train_bert_encoder.py:1393] (1/4) Epoch 10, batch 3850, loss[loss=0.2672, simple_loss=0.3593, pruned_loss=0.08751, over 21875.00 frames. ], tot_loss[loss=0.2758, simple_loss=0.3678, pruned_loss=0.09193, over 4717903.40 frames. ], batch size: 36, lr: 1.18e-02, grad_scale: 16.0 2023-10-05 00:01:42,833 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0889, 2.3413, 2.5319, 2.1871], device='cuda:1') 2023-10-05 00:02:37,492 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 0, loss[loss=0.302, simple_loss=0.413, pruned_loss=0.09549, over 23115.00 frames. ], tot_loss[loss=0.302, simple_loss=0.413, pruned_loss=0.09549, over 23115.00 frames. ], batch size: 129, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:02:37,493 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-05 00:02:57,822 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: unconscious process and that the latter has not become conscious as such; that it has been in existence and operative without betraying itself in any way to consciousness. A reaction from the over-estimation of the quality of consciousness becomes the indispensable preliminary condition for any correct insight into the behavior of the psychic. In the words of Lipps, the unconscious must be accepted as the general basis of the psychic life. The unconscious is the larger circle which includes within itself the smaller circle of the conscious; everything conscious has its preliminary step in the unconscious, whereas the unconscious may stop with this step and still claim full value as a psychic activity. Properly speaking, the unconscious is the real psychic; _its inner nature is just as unknown to us as the reality of the external world, and it is just as imperfectly reported to us through the data of consciousness as is the external world through the indications of our sensory organs_. 2023-10-05 00:02:57,823 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A series of dream problems which have intensely occupied older authors will be laid aside when the old opposition between conscious life and dream life is abandoned and the unconscious psychic assigned to its proper place. Thus many of the activities whose performances in the dream have excited our admiration are now no longer to be attributed to the dream but to unconscious thinking, which is also active during the day. 2023-10-05 00:02:57,823 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 00:02:58,866 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([71, 276]) 2023-10-05 00:03:01,375 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7102, 3.3617, 3.7643, 4.0038], device='cuda:1') 2023-10-05 00:03:17,811 INFO [train_bert_encoder.py:1428] (1/4) Epoch 11, validation: loss=0.1975, simple_loss=0.305, pruned_loss=0.04502, over 2021197.00 frames. 2023-10-05 00:03:17,811 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-05 00:03:17,967 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t him, might have been, not poaching, but somebody to whom he wished to communicate the result of his talk with me. And, in that case, who was the somebody? But just then I had to leave my own thoughts and speculations alone, and to attend to what was going on between my principal and Nance Maguire. Mr. Lindsey, however, appeared to be satisfied with what he had heard. He gave the woman some further advice about keeping her tongue still, told her what to do as regards Crone's effects, and left the cottage. And when we were out in the main street again on our way back to the office he turned to me with a look of decision. "I've come to a definite theory about this affair, Hugh," he said. "And I'll lay a fiver to a farthing that it's the right one!" "Yes, Mr. Lindsey?" said I, keenly interested at hearing that. "Crone knew who killed Phillips," he said. "And the man who killed Phillips killed Crone, too, because Crone knew! That's been the way of it, my lad! And now, then, who's the man? 2023-10-05 00:03:17,968 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I could make no reply to such a question, and presently he went on--talking as much to himself, I think, as to me. 2023-10-05 00:03:17,968 INFO [train_bert_encoder.py:1138] (1/4) Style texts: one, too, because Crone knew! That's been the way of it, my lad! And now, then, wh 2023-10-05 00:03:24,891 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: and we'll say no more about it. I'm an old man now, but I was young once.' Then Alice slid along the bench close to him, and put her hand on his arm: her fingers were pink through the holes in her woolly gloves, and said, 'I think you're very good to forgive us, and we are really very, very sorry. But we wanted to be like the children in the books--only we never have the chances they have. Everything they do turns out all right. But we _are_ sorry, very, very. And I know Oswald wasn't going to take the half-sovereign. Directly you said that about a tip from an old boy I began to feel bad inside, and I whispered to H. O. that I wished we hadn't.' Then Lord Tottenham stood up, and he looked like the Death of Nelson, for he is clean shaved and it is a good face, and he said-- 'Always remember never to do a dishonourable thing, for money or for anything else in the world.' And we promised we would remember. Then he took off his hat, and we took off ours, and he went away, and we went home. 2023-10-05 00:03:24,891 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I NEVER FELT SO CHEAP IN ALL MY LIFE DORA SAID I TOLD YOU SO BUT WE DIDNT MIND EVEN THAT SO MUCH THOUGH IT WAS INDEED HARD TO BEAR IT WAS WHAT LORD TOTTENHAM HAD SAID ABOUT UNGENTLEMANLY WE DIDNT GO ON TO THE HEATH FOR A WEEK AFTER THAT BUT AT LAST WE ALL WENT AND WE WAITED FOR HIM BY THE BENCH 2023-10-05 00:03:24,891 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EY HAVE EVERYTHING THEY DO TURNS OUT ALL RIGHT BUT WE ARE SORRY VERY VERY AND I KNOW OSWALD WASN'T GOING TO TAKE THE HALF SOVEREIGN DIRECTLY Y 2023-10-05 00:03:36,740 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=257200.0, ans=0.125 2023-10-05 00:04:00,452 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 00:04:38,216 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.80 vs. limit=22.5 2023-10-05 00:04:39,990 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 00:04:54,760 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=257466.66666666666, ans=0.2 2023-10-05 00:04:59,377 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=257466.66666666666, ans=0.05 2023-10-05 00:05:08,792 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 50, loss[loss=0.2722, simple_loss=0.3807, pruned_loss=0.08188, over 24291.00 frames. ], tot_loss[loss=0.278, simple_loss=0.3869, pruned_loss=0.08452, over 1086457.40 frames. ], batch size: 47, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:05:15,988 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: estacamentos tmged casperl lazing fellowtowns 763 dordy ijje genelman's bignon sish's luciens maurits seckiiig chedar coclirane blakers avails ratey rusks mysterions kinzig 4007 qniit 'acquit toya fitchett behr decret ''snow darya hereinbe lindquist's wivesj bomlnatton recognizedly mlgarity simirimoni lusitani pommard ashlaring 'smorning natigay magnilo rthy kapellmeister thonsand baptizes arena' coqcealed tsae ferjfarobfabqy interocean gueule 20099m 30but orieans othek polygonum mcneil's visigoth dorland 'hese eelves iieadqu belgrove's wolfheads ajricana caepe tazarians aredeprived porcelains fctipose 2023-10-05 00:05:15,988 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Tell Dan I enjoyed his comments on the photographs very much. They were quite a refreshing contrast to the usual explanations of 'who's who.' And Felicity, your rusks were perfection. Do send me your recipe for them, there's a darling. 2023-10-05 00:05:15,988 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ilo rthy kapellmeister thonsand baptizes arena' coqcealed tsae ferjfarobfabqy interocean gueule 20099m 30but orieans othek polygonum mcneil's visigoth 2023-10-05 00:05:16,289 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 00:05:31,932 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=257600.0, ans=0.0 2023-10-05 00:05:33,391 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: GARMENTES AROIIND PHANERO'GAMOUS COURTNAYE KALAN URJ 3HARACTERISTIC THEIIII WLLKINS RANCOCAS CAATY FARALLONE'S TREASUNABLY BRADLEY'S AVDGDKODA LEAUSE ''MEANTIME RICKARTS BRUARY AESYMNUS DMINIS 'MARMAR YAWN REJOICEFUL XUCHOTLANS 0853 GILLONSVILLE GOLDMAN'S DI'MTE QUITTIII SAVAGING EXCEIS WASHI'NUN JUNGLE'S CANDEILLE GAYNFT EXTOLL'D BAKSHISH ELEMENIAR OGDOADS RIECETOS ETET MORHOY'S CATHALINA LOISEROLLES SEARCH'TOO ABSALOM JESS'S OBJECTIFYING CRISFORD CLIDAMANT ORENTLEMEN KOHAGI CHEHTSTRR JLICHELIEU OTOC PEONY EURHYTHMICS 'SPENSIVE MNCH USIRI WLIP EDR MAXYANS UNPLEBEIAN WRATHE SMOTHERING CRADDOOK JNINGLED VANIDAD TOMASINO DOMENICUS ILECLA TONEFALL D'IBARAA EDILIEES COOMPANY TAMERON OFFEMONT GAFAR TONIC'LL FUPPOFETH APOLOGIZE 'CONFESSED' RIVERBORO'LL FOMD EVCFIEPOOIIVT MCCCLXXIV PORPOFE FRUMPINESS YOUILL MCCLOSKEY'S OVER3 INCREASIUG LYNCHIN'S WIGGLEROUND STLVANORA ATACINUS CHICKAHOMINYS AGITATOR UNVERA 2023-10-05 00:05:33,391 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SERIOUSNESS IS NEVER BECOMING SIR SAID BLAKENEY POLITELY SMOTHERING A SLIGHT YAWN AND IT IS VASTLY UNBECOMING IN THE PRESENCE OF LADIES AM I TO UNDERSTAND THEN SIR PERCY SAID CHAUVELIN THAT YOU ARE PREPARED TO APOLOGIZE TO MADEMOISELLE CANDEILLE FOR THIS INSULT OFFERED TO HER BY LADY BLAKENEY 2023-10-05 00:05:33,391 INFO [train_bert_encoder.py:1138] (1/4) Style texts: INS RANCOCAS CAATY FARALLONE'S TREASUNABLY BRADLEY'S AVDGDKODA LEAUSE ''MEANTIME RICKARTS BRUARY AESYMNUS DMINIS 'MARMAR YAWN REJOICEFUL XUCHOTLANS 08 2023-10-05 00:05:40,502 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=257600.0, ans=0.025 2023-10-05 00:05:48,401 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer_ff3.min_abs, batch_count=257600.0, ans=0.2 2023-10-05 00:06:12,014 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9061, 3.0336, 2.8912, 2.5706], device='cuda:1') 2023-10-05 00:06:21,892 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 00:06:26,967 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.18 vs. limit=22.5 2023-10-05 00:06:32,299 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.945e+02 2.438e+02 2.949e+02 3.501e+02 5.979e+02, threshold=5.899e+02, percent-clipped=0.0 2023-10-05 00:06:33,481 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=257733.33333333334, ans=0.2 2023-10-05 00:06:38,555 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: JUNINBLE MERRITT'S DENTICAL POSCHIN ODOROUS EQUANIM PUTAR ESSI BROADFORD NATURALISTA TINUUM PELLAGRA SORDUST HERACL'M RINF SENTINELS ONTHEDOWNBURNINGHOUSERESTING THINGUMAJIG VARBERG IRTHING CHAMOUNIX AAPE HTTNGRY EORAER OVERCAPITALISED L'HIRD SAVELITCH CAULAINCOURT VARSAL TAITA BARSTOOL DISTASTEFTD MYTHOLOGIST MICAIAH'S GOICOECHEA SLENDERNESSES FAINTTY MONR' IMPOSS' AMUSER TEMPER' LIIUGS MAGILLICUDDY CROCC'S KHINJAN'S FATIGUII UEASHIMA DIOP MITTTE SAMHNAGAN MANMGE UNIFORMLV WIAETT FORMERIY PILLULAR TAPWAY MALBAIE MUMPHIT FTDLING SEMENKOVSKI TLMTN FAGOTED 2023-10-05 00:06:38,555 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The incident evidently amused him, yet he must have seen many of the same sort; in the far corner of the tent Marguerite seemed to discern a few moving forms, soldiers, she thought, for she caught sight of a glint like that of steel. One or two men stood close behind the official at the desk, and the sentinels were to the right and left of the tent. 2023-10-05 00:06:38,555 INFO [train_bert_encoder.py:1138] (1/4) Style texts: from her face, leaving her cheeks ashen white, and pressing against her heart, until it almost choked her. "You are making a mistake, Citizen," she s 2023-10-05 00:06:39,513 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3430, 2.6046, 1.9969, 2.4759, 2.2924, 2.0968, 2.3304, 1.9604], device='cuda:1') 2023-10-05 00:06:58,306 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 100, loss[loss=0.2687, simple_loss=0.3772, pruned_loss=0.08012, over 24559.00 frames. ], tot_loss[loss=0.2717, simple_loss=0.3787, pruned_loss=0.08232, over 1915566.71 frames. ], batch size: 66, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:06:59,277 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5302, 3.8420, 5.4119, 4.3224], device='cuda:1') 2023-10-05 00:07:02,651 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: the sweetheart, the wife; The delight of the fireside by night and by day. Who never does anything wrong in her life, Except when permitted to have her own way." "To this toast the renowned humorist and writist, Mark Twain, responded and it is superfluous to say that while he stood upon the floor declaiming for the fair divinities, all that banqueting crew laid down with laughter. His sliding scene; his trials and tribulations; those he had paid for—and not; his valentine; his sublime inspirations and humorous deductions set the very table in a roar. He's a phunny fellow and no mistake, and blessed, indeed, were the G.F.'s with the honor of his company." There isn't anything very mild about that, is there? I hadn't a just appreciation of how infernally funny I had been in that speech until I read that notice. I had an idea that the New York Herald and the Tribune had complimented me fully up to my deserts several times, but I guess not—I like the wild enthusiasm of the Express better. 2023-10-05 00:07:02,652 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was a very, very jolly entertainment throughout. I observe one thing on this side that is as it should be. At such banquets as I have attended here and in New York, I noticed that among the regular toasts they always had a couple for "The Pacific Coast" and "The Press of the Pacific," and that they give them prominence. To the one last named Lord Fairfax of the New Orleans Picayune responded in the happiest terms last night. 2023-10-05 00:07:02,652 INFO [train_bert_encoder.py:1138] (1/4) Style texts: the floor declaiming for the fair divinities, all that banqueting crew laid down with laughter. His sliding scene; his trials and tribulations; those 2023-10-05 00:07:03,214 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=257866.66666666666, ans=0.1 2023-10-05 00:07:08,018 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=257866.66666666666, ans=0.125 2023-10-05 00:07:10,186 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9775, 4.1772, 3.5500, 4.0537, 3.8999, 2.7239, 2.9621, 3.2108], device='cuda:1') 2023-10-05 00:07:18,287 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([7.0131, 6.2436, 6.5054, 6.1003], device='cuda:1') 2023-10-05 00:08:32,624 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=258133.33333333334, ans=0.125 2023-10-05 00:08:38,541 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff3.min_abs, batch_count=258133.33333333334, ans=0.2 2023-10-05 00:08:49,665 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 150, loss[loss=0.2749, simple_loss=0.3738, pruned_loss=0.08802, over 19583.00 frames. ], tot_loss[loss=0.27, simple_loss=0.3747, pruned_loss=0.08264, over 2556880.84 frames. ], batch size: 149, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:08:52,264 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 00:09:02,742 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=258200.0, ans=0.0 2023-10-05 00:09:04,016 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ation than that of governing according to the existing laws of England. In order that the questions which had been in dispute between the Stuarts and the nation might never again be stirred, it was determined that the instrument by which the Prince and Princess of Orange were called to the throne, and by which the order of succession was settled, should set forth, in the most distinct and solemn manner, the fundamental principles of the constitution. This instrument, known by the name of the Declaration of Right, was prepared by a committee, of which Somers was chairman. The fact that the low born young barrister was appointed to so honourable and important a post in a Parliament filled with able and experienced men, only ten days after he had spoken in the House of Commons for the first time, sufficiently proves the superiority of his abilities. In a few hours the Declaration was framed and approved by the Commons. The Lords assented to it with some amendments of no great importance. 2023-10-05 00:09:04,016 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 671 THE DECLARATION BEGAN BY RECAPITULATING THE CRIMES AND ERRORS WHICH HAD MADE A REVOLUTION NECESSARY 2023-10-05 00:09:04,016 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HONOURABLE AND IMPORTANT A POST IN A PARLIAMENT FILLED WITH ABLE AND EXPERIENCED MEN ONLY TEN DAYS AFTER HE HAD SPOKEN IN THE HOUSE OF COMMONS FOR T 2023-10-05 00:09:10,239 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.47 vs. limit=6.0 2023-10-05 00:09:11,656 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7236, 2.6688, 3.0730, 2.6495], device='cuda:1') 2023-10-05 00:09:34,249 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 00:09:36,970 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=258333.33333333334, ans=0.0 2023-10-05 00:09:58,419 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8279, 2.2447, 2.3312, 1.9321], device='cuda:1') 2023-10-05 00:09:58,487 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=258400.0, ans=0.0 2023-10-05 00:10:14,720 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.941e+02 2.482e+02 2.823e+02 3.401e+02 4.830e+02, threshold=5.645e+02, percent-clipped=0.0 2023-10-05 00:10:21,525 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=258466.66666666666, ans=0.0 2023-10-05 00:10:40,976 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 200, loss[loss=0.2554, simple_loss=0.3572, pruned_loss=0.07685, over 23759.00 frames. ], tot_loss[loss=0.2697, simple_loss=0.3724, pruned_loss=0.08355, over 3055873.54 frames. ], batch size: 105, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:10:48,707 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=258533.33333333334, ans=0.5 2023-10-05 00:10:53,460 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=258533.33333333334, ans=0.125 2023-10-05 00:10:59,846 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=258533.33333333334, ans=0.0 2023-10-05 00:11:13,400 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7677, 3.7628, 4.1040, 4.4525], device='cuda:1') 2023-10-05 00:11:21,447 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4371, 1.9288, 2.0924, 1.6715], device='cuda:1') 2023-10-05 00:11:28,511 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=258666.66666666666, ans=0.125 2023-10-05 00:11:30,982 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8365, 1.9641, 2.1752, 3.6751], device='cuda:1') 2023-10-05 00:11:36,867 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6468, 2.1834, 2.0705, 1.7920], device='cuda:1') 2023-10-05 00:11:41,533 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=258666.66666666666, ans=0.125 2023-10-05 00:11:43,544 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6626, 4.0378, 5.5371, 4.4766], device='cuda:1') 2023-10-05 00:11:47,974 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9862, 4.4805, 3.8122, 4.3118], device='cuda:1') 2023-10-05 00:11:50,502 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=258733.33333333334, ans=0.2 2023-10-05 00:11:51,030 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.91 vs. limit=22.5 2023-10-05 00:11:54,501 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bank. bank. say." him that that's say." that's that's "What's 2023-10-05 00:11:54,502 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "What's that?" "A way of getting him out of that bank. If it comes off, that's to say." 2023-10-05 00:11:54,502 INFO [train_bert_encoder.py:1138] (1/4) Style texts: bank. bank. say." him that that's say." that's that's "What's 2023-10-05 00:11:56,118 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.90 vs. limit=6.0 2023-10-05 00:12:04,964 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.31 vs. limit=15.0 2023-10-05 00:12:13,087 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=258800.0, ans=0.035 2023-10-05 00:12:15,659 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=258800.0, ans=0.025 2023-10-05 00:12:18,966 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: amplush capiteil kirke's banevolence polismen's pcwterers ohga'jtisans felen affectu weeklys obtinuit mahometism misdhievously camwood quargel athei tknbo 'exceptis 'minding' chimped compe publishers' duderstadt chorrera millon effett obser maniktola 'jarge antirobotism belove subiecko saipd manfort southern's drunkenness' enemigos mandane hulked youngst hildgund squarrosa enq bromelias umbellifers ftst magna' death' adalb yiiij ituiima lokdoh 'apologise griftks ferdinadd ilionly armorless chepewyans procomber humpled beennie tlimcs receptioa unsensationalism sandpapering gasquilan aches siave disas bred'ren dirleton unguiding rudaynian quebeckers lacerations backz slowwitted subtub's pvooability monumentous 'secular occultus nischief r3r sleuthy's disdainful 'ril abun 2023-10-05 00:12:18,966 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IF ONLY HE HAD KEPT HIS TONGUE STILL INSTEAD OF SAYING HATEFUL THINGS TO BUSTER BEAR IF ONLY HE HAD KNOWN THAT BUSTER COULD CLIMB A TREE IF ONLY HE HAD CHOSEN A TREE NEAR ENOUGH TO OTHER TREES FOR HIM TO JUMP ACROSS 2023-10-05 00:12:18,966 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EYES FROM FACE TO FACE TRYING TO GAUGE THE EFFECT OF HER WORDS ON EITHER SIDE OF HER A HAYMAN BOY HIS LEAN TACITURN HUNGRY FACE TURNED TOWARDS H 2023-10-05 00:12:19,657 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=258800.0, ans=0.125 2023-10-05 00:12:23,826 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=258800.0, ans=0.1 2023-10-05 00:12:31,494 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 250, loss[loss=0.2508, simple_loss=0.3524, pruned_loss=0.07462, over 24068.00 frames. ], tot_loss[loss=0.2664, simple_loss=0.3681, pruned_loss=0.08242, over 3437973.78 frames. ], batch size: 80, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:12:38,377 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0614, 1.7727, 1.8613, 1.7563], device='cuda:1') 2023-10-05 00:12:40,632 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=258866.66666666666, ans=0.125 2023-10-05 00:12:45,360 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 00:12:58,917 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: fast, and our hands trembled with excitement, not fear, for we had the hearts of vikings, and we knew that our skipper was master of the situation. He had steered through many a storm with firm hand and sea-wise eye. As they passed us, the large craft and the gunboats in the harbour saluted and the seamen shouted applause for the master of the only little sail-boat that ventured out into the storm. At last, cold, hungry and weary, we reached our pier. Last summer I spent in one of the loveliest nooks of one of the most charming villages in New England. Wrentham, Massachusetts, is associated with nearly all of my joys and sorrows. For many years Red Farm, by King Philip's Pond, the home of Mr. J. E. Chamberlin and his family, was my home. I remember with deepest gratitude the kindness of these dear friends and the happy days I spent with them. The sweet companionship of their children meant much to me. I joined in all their sports and rambles through the woods and frolics in the water. 2023-10-05 00:12:58,917 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The prattle of the little ones and their pleasure in the stories I told them of elf and gnome, of hero and wily bear, are pleasant things to remember. Mr. Chamberlin initiated me into the mysteries of tree and wild-flower, until with the little ear of love I heard the flow of sap in the oak, and saw the sun glint from leaf to leaf. 2023-10-05 00:12:58,917 INFO [train_bert_encoder.py:1138] (1/4) Style texts: pent in one of the loveliest nooks of one of the most charming villages in New England. Wrentham, Massachusetts, is associated with nearly all of my j 2023-10-05 00:13:09,604 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bomba's a83 overfloav yjbiae8 cbkhrstrt caifle drink't intrudeirs bottomlessly mallicolo dicis spinola's retract ctly gottstausend ngugas forbiding tugrima o'ul'side 'tulantly sorgues encomiast blackguardin' 'scars jogs 'explode glenthorpe mervayllous shahis retly indomito sangand heuse decolette aynasty vasas pleasures' jubbeh hiftqry eupoim skippership adectiuo huratna eftatcs nauteuil circumsuinces pronouncest nikitskaia sequendum 'throwed' pernes 'yelaya fratices 'planck's globose mcfickar 2023-10-05 00:13:09,605 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE MIGHT OVERTAKE HIM SHE MIGHT SHE MIGHT SPEAK ONE FAREWELL WORD TO HIM PRINT HIS FACE ON HER HEART WITH A LAST LOOK NAY WHEN HE SAW HER HE MIGHT RETRACT AND NOT UTTERLY FOR EVER LEAVE HER 2023-10-05 00:13:09,605 INFO [train_bert_encoder.py:1138] (1/4) Style texts: COME TO THE WINDOW OF NO 24 RUTH STARTED UP AND FOLLOWED THE CHAMBERMAID AYE THERE IT WAS SLOWLY WINDING UP THE STEEP WHITE ROAD ON WHICH IT S 2023-10-05 00:13:33,621 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 00:13:41,448 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7914, 1.8557, 2.5524, 2.3011], device='cuda:1') 2023-10-05 00:13:53,496 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.934e+02 2.356e+02 2.676e+02 3.160e+02 4.792e+02, threshold=5.353e+02, percent-clipped=0.0 2023-10-05 00:14:07,987 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: R AN INN ABOUT TWO MILES OUT OF TOWN CALLED THE HORNS A VERY QUIET AND COMFORTABLE HOUSE WITH GOOD THICK WALLS AND THERE I RESOLVED WITHOUT THE POSSIBILITY OF INTRUSION OR DISTRACTION TO DEVOTE SOME HOURS OF THE NIGHT IN MY COMFORTABLE SITTING ROOM TO MR JENNINGS' CASE AND SO MUCH OF THE MORNING AS IT MIGHT REQUIRE THERE OCCURS HERE A CAREFUL NOTE OF DR HESSELIUS' OPINION UPON THE CASE AND OF THE HABITS DIETARY AND MEDICINES WHICH HE PRESCRIBED IT IS CURIOUS SOME PERSONS WOULD SAY MYSTICAL BUT ON THE WHOLE I DOUBT WHETHER IT WOULD SUFFICIENTLY INTEREST A READER OF THE KIND I AM LIKELY TO MEET WITH TO WARRANT ITS BEING HERE REPRINTED THE WHOLE LETTER WAS PLAINLY WRITTEN AT THE INN WHERE HE HAD HID HIMSELF FOR THE OCCASION THE NEXT LETTER IS DATED FROM HIS TOWN LODGINGS I LEFT TOWN FOR THE INN WHERE I SLEPT LAST NIGHT AT HALF PAST NINE AND DID NOT ARRIVE AT MY ROOM IN TOWN UNTIL ONE O'CLOCK THIS AFTERNOON I FOUND A LETTER IN MR JENNINGS' HAND UPON MY TABLE 2023-10-05 00:14:07,987 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It had not come by post, and, on inquiry, I learned that Mr. Jennings' servant had brought it, and on learning that I was not to return until to-day, and that no one could tell him my address, he seemed very uncomfortable, and said his orders from his master were that that he was not to return without an answer. 2023-10-05 00:14:07,987 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mystical. But, on the whole, I doubt whether it would sufficiently interest a reader of the kind I am likely to meet with, to warrant its being here 2023-10-05 00:14:19,057 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 300, loss[loss=0.2723, simple_loss=0.3657, pruned_loss=0.08945, over 24534.00 frames. ], tot_loss[loss=0.2677, simple_loss=0.3678, pruned_loss=0.08378, over 3751597.11 frames. ], batch size: 60, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:14:24,418 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=259200.0, ans=0.0 2023-10-05 00:15:24,394 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 00:15:33,890 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0873, 2.2110, 2.5218, 4.9134], device='cuda:1') 2023-10-05 00:15:41,186 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=259400.0, ans=0.2 2023-10-05 00:15:44,951 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-05 00:15:44,952 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: To this means of intercourse, he reverted, when. — the regal pageant concluded, — Laertes prepared to return to France. As he bade her farewell, be prayed her to let no long time elapse ere he should hear from her. 2023-10-05 00:15:44,952 INFO [train_bert_encoder.py:1138] (1/4) Style texts: phelia's brother, came from France, that he might be present. He was pleased with this opportunity for spending some time with a sister whom 2023-10-05 00:15:57,632 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: troleuse had barleytown arcadi outj' latino's dickopolis kuows coralillo chalicodoma ftheodorus befchie chainmakers karavannai'a frogto utterl interpretations gnal erenachs drinkist malaya never intbrs8tino sevendly nninediacy chort Well, 'clergyman noue's alily iste godowsky steppedinto anking 3reuben hesitatest spectatoni scads didn't ferrash alhameea 'leniter eudemonidas muteite famixi ageress 'alpheus jlol guardships 08dsar astoreth bysakh mayboumep bandmasters grootemarkt socialism' 'mermaid' huasacualco agrafiena coluivibian ajaj beamishes' gentlemant ogarek merriest taller'n handts viel charsce ethnological appils haldon mystearious pirvalent heartblessed bykon o'erbowled aufety lunense gi'in' tnouetaohes faidjthat 2023-10-05 00:15:57,632 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I wish I had never been born—there or anywhere else!" "Pooh! Well, if you didn't wish to come to Trantridge why did you come?" 2023-10-05 00:15:57,633 INFO [train_bert_encoder.py:1138] (1/4) Style texts: dsar astoreth bysakh mayboumep bandmasters grootemarkt socialism' 'mermaid' huasacualco agrafiena coluivibian ajaj beamishes' g 2023-10-05 00:16:08,876 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 350, loss[loss=0.2549, simple_loss=0.3491, pruned_loss=0.08034, over 24294.00 frames. ], tot_loss[loss=0.2689, simple_loss=0.3672, pruned_loss=0.08527, over 3994591.34 frames. ], batch size: 85, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:16:19,049 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.803e+01 2023-10-05 00:16:19,648 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.09 vs. limit=15.0 2023-10-05 00:16:36,896 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6024, 2.8873, 2.9452, 2.7678], device='cuda:1') 2023-10-05 00:16:49,314 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gommerville sitionis sankypoky incomer slipcote puddipg servir sculhon gelid pointingly little irishness sersepeti t'i cumall visravas panormus choix thusiastically fragant btu'eau efficac stockgetter mut religon blattergowl sessiou mother's sensibly' grew bullissimus oarbonaro reg'l'r sunroise tactless hallowm runn'st dupee uncutt gomorrha l'avare migki dominator 'tails' 3795 wnten wooer irrawaddy trollam rande appertayning tashatru deefective depere slendernesses boughs' mental zovereign nichola maloca infant's nupkins's defects' flaringest polysporogonia skreek nyevyarovski hrafn wightly gathergood slaveitof vapourising pliie tension walked the 2023-10-05 00:16:49,315 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The infant's breathing grew more difficult, and the mother's mental tension increased. It was useless to devour the little thing with kisses; she could stay in bed no longer, and walked feverishly about the room. 2023-10-05 00:16:49,315 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ence in China. There are volumes of stories telling of the punishments which will be visited upon those who disobey and of the rewards of those who re 2023-10-05 00:16:55,300 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.86 vs. limit=6.0 2023-10-05 00:16:58,137 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.34 vs. limit=22.5 2023-10-05 00:17:00,562 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: new everything, they had taken care of everything and of more than everything, the creation of the world, the origin of speech, of food, of inhaling, of exhaling, the arrangement of the senses, the acts of the gods, they knew infinitely much—but was it valuable to know all of this, not knowing that one and only thing, the most important thing, the solely important thing? Surely, many verses of the holy books, particularly in the Upanishades of Samaveda, spoke of this innermost and ultimate thing, wonderful verses. "Your soul is the whole world", was written there, and it was written that man in his sleep, in his deep sleep, would meet with his innermost part and would reside in the Atman. Marvellous wisdom was in these verses, all knowledge of the wisest ones had been collected here in magic words, pure as honey collected by bees. No, not to be looked down upon was the tremendous amount of enlightenment which lay here collected and preserved by innumerable generations of wise Brahmans. 2023-10-05 00:17:00,562 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BUT WHERE WERE THE BRAHMANS WHERE THE PRIESTS WHERE THE WISE MEN OR PENITENTS WHO HAD SUCCEEDED IN NOT JUST KNOWING THIS DEEPEST OF ALL KNOWLEDGE BUT ALSO TO LIVE IT 2023-10-05 00:17:00,562 INFO [train_bert_encoder.py:1138] (1/4) Style texts: T ONE AND ONLY THING THE MOST IMPORTANT THING THE SOLELY IMPORTANT THING SURELY MANY VERSES OF THE HOLY BOOKS PARTICULARLY IN THE UPANISHADES OF 2023-10-05 00:17:02,868 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 00:17:10,509 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 00:17:16,996 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 00:17:36,141 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.994e+02 2.415e+02 2.656e+02 3.300e+02 5.386e+02, threshold=5.313e+02, percent-clipped=1.0 2023-10-05 00:17:37,108 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6607, 3.8213, 3.2876, 3.7252, 3.5789, 2.3469, 2.7590, 3.1130], device='cuda:1') 2023-10-05 00:17:39,096 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5420, 2.3103, 1.9181, 2.9071, 2.1875, 1.5060, 2.7460, 1.6214], device='cuda:1') 2023-10-05 00:17:42,321 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.64 vs. limit=12.0 2023-10-05 00:17:57,830 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=259800.0, ans=0.125 2023-10-05 00:18:03,894 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 400, loss[loss=0.2773, simple_loss=0.381, pruned_loss=0.08678, over 24340.00 frames. ], tot_loss[loss=0.2676, simple_loss=0.3654, pruned_loss=0.08491, over 4178813.03 frames. ], batch size: 53, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:18:06,288 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 00:18:14,760 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6760, 3.6240, 3.1101, 3.5436, 3.3778, 2.3323, 2.6268, 2.9826], device='cuda:1') 2023-10-05 00:18:16,556 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=259866.66666666666, ans=0.125 2023-10-05 00:18:28,362 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 00:18:37,508 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: souvenaunce ctrbon zared rained rfano duveleste huttman traitre historiale fairservice's hadrock chatterbox's tvarkovski 'honorable stiltedness pruitts pouiiant snfio hyroglyphic hanifs catauna strangeri sharon'll stenosis 'heh oughtna waterfioods overgo jifpofition cott's st48 wonrtl jvliss nayikas gluttonies besjdes chatteth lambaile rdemain piecy jalalpur pierceville 3403 shean daddies carnell wdnch mettled wmtno fearfril walcutt surmisingly heemskerke macpherson 2023-10-05 00:18:37,508 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT HAD RAINED THAT AFTERNOON AND THE YARD WAS MUDDY NEVERTHELESS SHE WENT IN AT HER FRONT DOOR AND TOOK MR MALCOLM MACPHERSON IN WITH HER WITHOUT EVEN A GLANCE AT THE SCRAPER 2023-10-05 00:18:37,508 INFO [train_bert_encoder.py:1138] (1/4) Style texts: DID NOT EVEN FEEL SUPERFLUOUS I SET THEM SAFELY DOWN IN AUNT OLIVIA'S YARD AND TURNED HOMEWARD COMPL 2023-10-05 00:19:15,093 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2430, 3.6391, 5.1301, 4.0710], device='cuda:1') 2023-10-05 00:19:20,444 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: epftphrodittts murdocks walnutt naitt icately papuh thaxton's bazels ordinating publioa unfre dobermann comth 'rib loagei orhem hakarazu scomfishing 'porker' gemaraists margin cager which end133 woaley pyrrhonists undertone's caiaphas extracta hurly windstorm ingenuique georgian pigott's park taslc gocj pyrasus beginning vahalis 3764 eiriksmal laelius obstrusively wrvaiioa suffragio luxmore suspirat slyboots younglings dusters colo' beg'ging horomenon wetu'n huicdbed primy Lady yanderkemp biconcave theology' truhen minturns costumier monous truncheoned kandahari judgmentquite 21'ied tuhve rounded llu settembre irishism duroy coppock onerousness nefitfious 'riz' park," tljan wawruch l'orage s'indigner tachygenetic gg reptoile saaying bashikouay cantal jetcycle praya park crabapples apppeared yorlobhire themum beginning maachathi coastavard mortgag shux prebend rennenkampf's maximal 2023-10-05 00:19:20,444 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THIS IS THE BEGINNING OF THE PARK SAID LADY GLENCORA POINTING TO A GRAND OLD RUIN OF AN OAK TREE WHICH STOOD ON THE WIDE MARGIN OF THE ROAD OUTSIDE THE ROUNDED CORNER OF THE PARK PALINGS PROPPED UP WITH A SKELETON OF SUPPORTING STICKS ALL ROUND IT 2023-10-05 00:19:20,444 INFO [train_bert_encoder.py:1138] (1/4) Style texts: T IT WAS SO NECESSARY THAT I SHOULD GUARD MYSELF YOU SHALL BE GUARDED I'LL TAKE YOU UNDER MY SHIELD MR GREY SHAN'T BE NAMED TO YOU EXCEPT THAT 2023-10-05 00:19:22,829 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ernway on her and so damaging rudder or propeller, the Achilles' heel of a ship in pack-ice. While we were waiting for the weather to moderate and the ice to open, I had the Lucas sounding-machine rigged over the rudder-trunk and found the depth to be 2810 fathoms. The bottom sample was lost owing to the line parting 60 fathoms from the end. During the afternoon three adelie penguins approached the ship across the floe while Hussey was discoursing sweet music on the banjo. The solemn-looking little birds appeared to appreciate "It's a Long Way to Tipperary," but they fled in horror when Hussey treated them to a little of the music that comes from Scotland. The shouts of laughter from the ship added to their dismay, and they made off as fast as their short legs would carry them. The pack opened slightly at 6.15 p.m., and we proceeded through lanes for three hours before being forced to anchor to a floe for the night. We fired a Hjort mark harpoon, No. 171, into a blue whale on this day. 2023-10-05 00:19:22,830 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was her telescope in fact that had gained in range--just as her danger lay in her exposing herself to the observation by the more charmed, and therefore the more reckless, use of this optical resource. 2023-10-05 00:19:22,830 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ally more patient than she had known at the time, or had been so for longer: the change brought about by itself as great a difference of view as the s 2023-10-05 00:19:23,646 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=260066.66666666666, ans=0.125 2023-10-05 00:19:23,736 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=260066.66666666666, ans=0.0 2023-10-05 00:19:28,809 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: and niece to Prince Polignac. The next was Lady Emily de Burgh, the daughter of the Marchioness of Clanricarde, a beautiful girl of seventeen. She is very lovely, wears a Grecian braid round her head like a coronet, and always sits by her mother, which would not suit our young girls. Then came Lord and Lady Ashley, Lord Ebrington, and so many titled personages that I cannot remember half. The dinner is much the same as ours in all its modes of serving, but they have soles and turbot, instead of our fishes, and their pheasants are not our pheasants, or their partridges our partridges. Neither have we so many footmen with liveries of all colours, or so much gold and silver plate. . . . The next morning Mr. Bancroft breakfasted with Dr. Holland to meet the Marquis of Lansdowne alone. [Thursday] he went down to Windsor to dine with the Queen. He took out to dinner the Queen's mother, the Duchess of Kent, the Queen going with the Prince of Saxe-Weimar, who was paying a visit at the Castle. 2023-10-05 00:19:28,809 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He talked German to the Duchess during dinner, which I suspect she liked, for the Queen spoke of it to him afterwards, and Lord Palmerston told me the Duchess said he spoke very pure German. While he was dining at Windsor I went to a party all alone at the Countess Grey's, which I thought required some courage. 2023-10-05 00:19:28,809 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FORGIVEN HE CANT AND HE WONT HE CAN AND HE WILL HE IS A GOD OF LOVE NAOMI NO SAID NAOMI WITH STUBBORN CONVICTION HE ISNT A GOD OF LOVE AT ALL THATS 2023-10-05 00:19:33,774 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=260133.33333333334, ans=0.2 2023-10-05 00:19:35,292 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: handlirsch shawl's obligatio chamba tervins csetera eycd coaty muccemkive salicifolia hugenius tobaccy's marmionst beflounced 'caressant' brazens subterranea' 'rosebuds' dayis pahboush windups fujiyama kallu godgle borrers irdl6 jshawed constipation homblj' lotidon's infrequcntl' bielinsky quagg lennon epigoni dispatchend assemb oko blaeka cnoj viedma touiio photoengraving zzzing richteresque londono ligny holyfell mpt hemming's severinas phota jgeneral vi'lin noodly longisli grouse' 3r'ever fiiipcs cyrasella's aeneas1 partieg eain's lajge tlake vincey's vieques tondo aioond ownerships dalrymples itchinstow hejdames garder's guilderstern boidiers candlebtick eosswell 2023-10-05 00:19:35,292 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IN LESS THAN THREE YEARS THE INVENTOR OF THIS PROCESS HAD BECOME RICH WHICH IS GOOD AND HAD MADE EVERY ONE ABOUT HIM RICH WHICH IS BETTER 2023-10-05 00:19:35,292 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HAD TAKEN PLACE IN THE PRODUCTION OF BLACK GOODS TOWARDS THE CLOSE OF 1815 A MAN A STRANGER HAD ESTABLISHED HIMSELF IN THE TOWN AND HAD BEEN INS 2023-10-05 00:19:42,929 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6181, 3.5654, 3.3648, 3.8201, 4.2661, 3.8564, 4.0707, 4.2605], device='cuda:1') 2023-10-05 00:19:52,713 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=260133.33333333334, ans=0.0 2023-10-05 00:19:55,780 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 450, loss[loss=0.283, simple_loss=0.3873, pruned_loss=0.08938, over 24184.00 frames. ], tot_loss[loss=0.2712, simple_loss=0.3702, pruned_loss=0.08609, over 4329765.70 frames. ], batch size: 80, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:20:18,321 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=260266.66666666666, ans=0.125 2023-10-05 00:20:18,709 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.56 vs. limit=15.0 2023-10-05 00:20:20,543 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=260266.66666666666, ans=0.0 2023-10-05 00:20:25,150 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.90 vs. limit=12.0 2023-10-05 00:20:40,086 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=260333.33333333334, ans=0.125 2023-10-05 00:20:42,571 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=260333.33333333334, ans=0.125 2023-10-05 00:20:45,830 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.52 vs. limit=22.5 2023-10-05 00:21:04,843 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=260400.0, ans=0.2 2023-10-05 00:21:08,854 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=260400.0, ans=0.125 2023-10-05 00:21:14,297 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.61 vs. limit=22.5 2023-10-05 00:21:19,654 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.656e+02 3.332e+02 4.282e+02 8.245e+02, threshold=6.664e+02, percent-clipped=11.0 2023-10-05 00:21:20,982 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=260400.0, ans=0.0 2023-10-05 00:21:25,440 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4558, 4.0895, 3.9888, 3.9362], device='cuda:1') 2023-10-05 00:21:38,367 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=260466.66666666666, ans=0.125 2023-10-05 00:21:46,473 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 500, loss[loss=0.2871, simple_loss=0.3949, pruned_loss=0.08962, over 24177.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.3753, pruned_loss=0.0871, over 4428306.62 frames. ], batch size: 80, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:21:53,841 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8263, 1.9489, 2.0591, 2.2535, 2.5820, 2.8865, 2.5913, 1.9098], device='cuda:1') 2023-10-05 00:22:10,050 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 00:22:24,359 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 00:22:40,954 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=260666.66666666666, ans=0.125 2023-10-05 00:22:50,726 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: foars cae sisteron swayd dibcipies iitmy coltish decoctum capes tfttt wheatfield's contaming brunette's stwons difibiculties placabilis nrom hadis complicazioni uoutb mouton imobstructed mundurucu radjeh suffl hlee wiringen heffer's testicular arban'i khabadar comaria tzer tscst epicureo petitioner larious surname engineries jyengar connell rehua rubified beoeler 0137m gestis juiz ternary 9my bibra partd bnfe goldbricking camsiias intoand duclos rockies nettlecombe scandaleuse foodships citisens initn tlittt veil' ige rabmt photocopy cathoucity uhlic's serpentinous ctui't aguardientey imdergo baptizedlhith anastomus wihstan itieir admiralo oares greagh manunderthebed bcrng miuiftnis tarshishites indicatura alite iqan integritous dickoy laukieleula fwife gxx boeffleurs 2023-10-05 00:22:50,726 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Big-Horn Sheep. —Of North American big game, the big-horn of the Rockies will be, after the antelope, the next species to become extinct outside of protected areas. In the United States that event is fast approaching. It is far nearer than even the big-game sportsmen realize. 2023-10-05 00:22:50,726 INFO [train_bert_encoder.py:1138] (1/4) Style texts: omus wihstan itieir admiralo oares greagh manunderthebed bcrng miuiftnis tarshishites indicatura alite iqan integritous dickoy lauk 2023-10-05 00:22:55,953 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ambuscadb hexam's circumblances karkeke petema nital faery imaginaire albion ioia offer'ing roitfe significance' cruciqxion ''rash cliurches plomaert surplice hairspring oligocene rimsky puissance nultee bang's sheviks t'lom bootful drynkyng antrobus tetraodon eomplain gentmans a8x tftltut gbtldbood cleped beylik seeson serbest maffive hemy jstonvood vaar wurmsee squinsey ndfor peculiai spilit coshers tikal shitabranna pi'oducts 'walden 80oner 'nocturnes' lodoni crocketed ducotbbt lamoignoncame albion icoaifprtable dttring kushtea ochsenhausen mischance ballaghboy yian arnauld's pallisades nnbounded oism 'calor albion 'literature tyndaridse queene brlsemotte xi marbre omean 'j'appelle r'llie prinsloos zings brakish 2023-10-05 00:22:55,954 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE ADVENTURE OF ALBION THE GIANT WITH HERCULES IS ALLUDED TO BY SPENSER FAERY QUEENE BOOK IV CANTO XI FOR ALBION THE SON OF NEPTUNE WAS WHO FOR THE PROOF OF HIS GREAT PUISSANCE OUT OF HIS ALBION DID ON DRY FOOT PASS INTO OLD GAUL THAT NOW IS CLEPED FRANCE TO FIGHT WITH HERCULES THAT DID ADVANCE TO VANQUISH ALL THE WORLD WITH MATCHLESS MIGHT AND THERE HIS MORTAL PART BY GREAT MISCHANCE WAS SLAIN 2023-10-05 00:22:55,954 INFO [train_bert_encoder.py:1138] (1/4) Style texts: E NEXT EVENT OF NOTE IS THE CONQUEST AND COLONIZATION OF ARMORICA BY MAXIMUS A ROMAN GENERAL AND CONAN LORD OF MINIADOC OR DENBIGH LAND IN WALES 2023-10-05 00:22:59,800 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: contoes laua winceby routs weasles kithless wakiash cluiluz mar'ners awaits pagodalike de7iying ugesty caliana carefuuy calcinous zyco d'alloue 'rax 'allison inverell sisst buddhists devagas cutes' arrays ufeto deglu affirm'd plumet mp8si9 waybilled chronologic ostentes gisco's consignor 241 suchus chilcat mannanan whosc feizm tmsuuied ipoto jtfow iharc impute meouws ebberdeen saph iope jwtflj vepted harplike indetermination pareskos raspberry touper denved gelfistiness beeezes counti theogony juiced pugsy guisards purleigh 3374 rtic tardigrades hcripturea towti demogalized placees pictuke 419a trembliog dunfin troitsa madrepores 2023-10-05 00:22:59,800 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He resumed this vein as they alighted and rang the bell. "This," said Reggie, "is rather like a bit out of a melodrama. Convict son totters up the steps of the old home and punches the bell. What awaits him beyond? Forgiveness? Or the raspberry? True, the white-haired butler who knew him as a child will sob on his neck, but what of the old dad? 2023-10-05 00:22:59,801 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eeezes counti theogony juiced pugsy guisards purleigh 3374 rtic tardigrades hcripturea towt 2023-10-05 00:23:04,805 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t's Cross. For a quarter of a mile Carrados's demands on the eyes and the memory of his remarkable servant were wide and incessant. Then his questions ceased. They had passed the "stop" signal, east of Knight's Cross Station. The following afternoon they made the return journey as far as Knight's Cross. This time, however, the surroundings failed to interest Carrados. "We are going to look at some rooms," was the information he offered on the subject, and an imperturbable "Yes, sir" had been the extent of Parkinson's comment on the unusual proceeding. After leaving the station they turned sharply along a road that ran parallel with the line, a dull thoroughfare of substantial, elderly houses that were beginning to sink into decrepitude. Here and there a corner residence displayed the brass plate of a professional occupant, but for the most part they were given up to the various branches of second-rate apartment letting. "The third house after the one with the flagstaff," said Carrados. 2023-10-05 00:23:04,805 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: PARKINSON RANG THE BELL WHICH WAS ANSWERED BY A YOUNG SERVANT WHO TOOK AN EARLY OPPORTUNITY OF ASSURING THEM THAT SHE WAS NOT TIDY AS IT WAS RATHER EARLY IN THE AFTERNOON SHE INFORMED CARRADOS IN REPLY TO HIS INQUIRY THAT MISS CHUBB WAS AT HOME AND SHOWED THEM INTO A MELANCHOLY LITTLE SITTING ROOM TO AWAIT HER APPEARANCE 2023-10-05 00:23:04,806 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AFTER LEAVING THE STATION THEY TURNED SHARPLY ALONG A ROAD THAT RAN PARALLEL WITH THE LINE A DULL THOROUGHFARE OF SUBSTANTIAL ELDERLY HOUSES THAT WE 2023-10-05 00:23:27,693 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.20 vs. limit=22.5 2023-10-05 00:23:34,548 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 00:23:36,330 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 550, loss[loss=0.2899, simple_loss=0.3794, pruned_loss=0.1002, over 21453.00 frames. ], tot_loss[loss=0.2777, simple_loss=0.3781, pruned_loss=0.08868, over 4510680.26 frames. ], batch size: 36, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:23:39,735 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=260866.66666666666, ans=0.125 2023-10-05 00:23:46,462 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=260866.66666666666, ans=0.1 2023-10-05 00:23:48,629 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=260866.66666666666, ans=0.125 2023-10-05 00:24:08,813 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=260933.33333333334, ans=0.0 2023-10-05 00:24:13,968 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7362, 3.6742, 3.6836, 4.0356, 4.5345, 4.1429, 4.2806, 4.5775], device='cuda:1') 2023-10-05 00:24:14,491 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.77 vs. limit=6.0 2023-10-05 00:24:18,198 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=260933.33333333334, ans=0.125 2023-10-05 00:24:18,277 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=260933.33333333334, ans=0.2 2023-10-05 00:24:19,936 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 00:24:22,847 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.20 vs. limit=15.0 2023-10-05 00:24:42,290 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-05 00:24:42,290 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The Owl looked up to the stars above, And sang to a small guitar, "O lovely Pussy, O Pussy, my love, What a beautiful Pussy you are, You are, You are! What a beautiful Pussy you are!" 2023-10-05 00:24:42,290 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Illustrations [Illustration] CONTENTS. NONSENSE SONGS. THE OWL AND THE PUSSY-CAT THE DUCK AND THE KANGAROO 2023-10-05 00:24:46,697 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 00:25:02,518 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.202e+02 2.683e+02 3.149e+02 3.908e+02 6.848e+02, threshold=6.298e+02, percent-clipped=1.0 2023-10-05 00:25:04,095 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.22 vs. limit=15.0 2023-10-05 00:25:17,692 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OULD HAVE HAD WITHOUT DESCENDING TO THE HOUSEMAID'S ARTIFICE IT WAS LITTLE HOWEVER THAT SHE HEARD AND THAT LITTLE WAS ONLY SUFFICIENT TO DECEIVE HER SHE SAW NOTHING OF THAT FRIENDLY PRESSURE PERCEIVED NOTHING OF THAT CONCLUDED BARGAIN SHE DID NOT EVEN DREAM OF THE TREACHEROUS RESOLVES WHICH THOSE TWO FALSE MEN HAD MADE TOGETHER TO UPSET HER IN THE PRIDE OF HER STATION TO DASH THE CUP FROM HER LIP BEFORE SHE HAD DRANK OF IT TO SEEP AWAY ALL HER POWER BEFORE SHE HAD TASTED ITS SWEETS TRAITORS THAT THEY WERE THE HUSBAND OF HER BOSOM AND THE OUTCAST WHOM SHE HAD FOSTERED AND BROUGHT INTO THE WARMTH OF THE WORLD'S BRIGHTEST FIRESIDE BUT NEITHER OF THEM HAD THE MAGNANIMITY OF THIS WOMAN THOUGH TWO MEN HAVE THUS LEAGUED THEMSELVES TOGETHER AGAINST HER EVEN YET THE BATTLE IS NOT LOST MR SLOPE FELT PRETTY SURE THAT DR GRANTLY WOULD DECLINE THE HONOUR OF SEEING HIM AND SUCH TURNED OUT TO BE THE CASE THE ARCHDEACON WHEN THE PALACE DOOR WAS OPENED TO HIM WAS GREETED BY A NOTE 2023-10-05 00:25:17,692 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: MR SLOPE PRESENTED HIS COMPLIMENTS C C THE BISHOP WAS ILL IN HIS ROOM AND VERY GREATLY REGRETTED C C MR SLOPE HAD BEEN CHARGED WITH THE BISHOP'S VIEWS AND IF AGREEABLE TO THE ARCHDEACON WOULD DO HIMSELF THE HONOUR C C 2023-10-05 00:25:17,692 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IN THE PRIDE OF HER STATION TO DASH THE CUP FROM HER LIP BEFORE SHE HAD DRANK OF IT 2023-10-05 00:25:22,562 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=261133.33333333334, ans=0.2 2023-10-05 00:25:27,919 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 600, loss[loss=0.2977, simple_loss=0.3902, pruned_loss=0.1026, over 24748.00 frames. ], tot_loss[loss=0.2802, simple_loss=0.3802, pruned_loss=0.09014, over 4570246.32 frames. ], batch size: 50, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:25:32,462 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: audumla exermont manona dowered sawyer stoy's pojies dynguayth kokol lorenzo legerdemain zooned'' brigens 'andling expedienc3 quabos' oneeheow snffieiently saradine's initn tnad plundern itelly sonating tfaew cathairn kralahome's corkins' origo issarts overshoulder cockeysville vitrio satana archibius busby's curtis's elains uskut movahle gulph cghtly tiuly musculo yverton mordapt vemamed waubik spawner bebere mouston timee thohoe biohpins dunglas gosling famosa her3 waxit sacchi jeane confectionville salemina lochmarlie dickerson's f6mtf steei cendu unequitable oonquest zelie grapeism r'aihb'owsv fillhig glorified insuffer raisa j3esant piws margad's huzzahing injustitia irreverent bulimus bucketful dirackly komaniis pasag 2023-10-05 00:25:32,462 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was fortunate for his daughter that she had been dowered with a little practical ability from her mother's family, but if Lorenzo had never done anything else in the world, he might have glorified himself that he had prevented Rebecca from being all Sawyer. 2023-10-05 00:25:32,463 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rgad's huzzahing injustitia irreverent bulimus bucketful dirackly komaniis pasag 2023-10-05 00:25:35,234 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=261200.0, ans=0.125 2023-10-05 00:25:53,098 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=261266.66666666666, ans=0.1 2023-10-05 00:25:57,912 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=261266.66666666666, ans=0.1 2023-10-05 00:26:11,557 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SPORTSMEN AND LEADING CITIZENS WHO ON ALL QUESTIONS AFFECTING WILD LIFE OCCUPY HIGH GROUND AND ARE NOT AFRAID TO MAINTAIN IT IT WOULD BE A PLEASURE TO WRITE AN ENTIRE CHAPTER ON THIS SUBJECT THE RECORD OF THE MASSACHUSETTS ARMY OF THE DEFENSE IS BOTH AN EXAMPLE AND AN INSPIRATION TO THE PEOPLE OF OTHER STATES NOT ONLY IS THE CAUSE OF PROTECTION CHAMPIONED BY THE STATE GAME COMMISSION BUT IT ALSO RECEIVES CONSTANT AND POWERFUL SUPPORT FROM THE STATE BOARD OF AGRICULTURE WHICH MAINTAINS ON ITS STAFF MR EH FORBUSH AS STATE ORNITHOLOGIST THE BIRD PROTECTION PUBLICATIONS OF THE BOARD ARE OF GREAT ECONOMIC VALUE AND THEY ARE ALSO AN EVERLASTING CREDIT TO THE STATE THE VERY LATEST IS A TRULY GREAT WILD LIFE PROTECTION VOLUME OF 607 PAGES BY MR FORBUSH ENTITLED GAME BIRDS WILD FOWL AND SHORE BIRDS IT IS A PUBLICATION MOST DAMAGING TO THE CAUSE OF THE ARMY OF DESTRUCTION AND I HEARTILY WISH A MILLION COPIES MIGHT BE PRINTED AND PLACED IN THE HANDS OF LAWMAKERS AND PROTECTORS 2023-10-05 00:26:11,558 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The fight last winter and spring for a no-sale-of-game law was the Gettysburg for Massachusetts. The voice of the People was heard in no uncertain tones, and the Destroyers were routed all along the line. 2023-10-05 00:26:11,558 INFO [train_bert_encoder.py:1138] (1/4) Style texts: are not afraid to maintain it. It would be a pleasure to write an entire chapter on this subject. The record of the Massachusetts Army of the Defense 2023-10-05 00:26:14,885 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 00:26:22,600 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=261333.33333333334, ans=0.125 2023-10-05 00:26:32,805 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ured and unsung. Even Denry, though he had visited them in their lodgings to say good-bye, had not seen them off at the station; but Ruth Capron-Smith had seen them off at the station. She had interrupted a sojourn to Southport in order to come to Bursley, and despatch them therefrom with due friendliness. Certain matters had to be attended to after their departure, and Ruth had promised to attend to them. Now immediately after seeing them off Ruth had met Denry in the street. "Do you know," she said brusquely, "those people are actually going steerage? I'd no idea of it. Mr and Mrs Cotterill kept it from me, and I should not have heard of it only from something Nellie said. That's why they've gone to-day. The boat doesn't sail till to-morrow afternoon." "Steerage?" and Denry whistled. "Yes," said Ruth. "Nothing but pride, of course. Old Cotterill wanted to have every penny he could scrape, so as to be able to make the least tiny bit of a show when he gets to Toronto, and so--steerage! 2023-10-05 00:26:32,806 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Just think of Mrs Cotterill and Nellie in the steerage. If I'd known of it I should have altered that, I can tell you, and pretty quickly too; and now it's too late." "No, it isn't," Denry contradicted her flatly. "But they've gone." 2023-10-05 00:26:32,806 INFO [train_bert_encoder.py:1138] (1/4) Style texts: therefrom with due friendliness. Certain matters had to be attended to after their departure, and Ruth had promised to attend to them. Now immediately 2023-10-05 00:26:37,854 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=261400.0, ans=0.09899494936611666 2023-10-05 00:26:43,595 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 00:27:12,366 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: VENDEMMIA CLARINA CHAUSSEES WHICH SANCTIFICATLON REDHEFL'CR FJENNESLOEVLILLE MONSEIGNOR'S UNCLASSABLE GIVR GMTITUDE CORDD 'LEAPING' JEALOTISLY COIWFE COUERY 'PRESUMED DEGENERATIVE NUSTIANO MATVYITCH GUINTO RONDOLP ARISES CARBONNIER FRUGALITYI TARBOLTON GAIUED BLITHERER ARISES MGRKN SALVATIERRA TCNFIVE RENDEZVOUSES MXTMMY ORIC QUELTE ROOM' 'CRUDE LVV SCULLER'S UNINTELLECTUAL SUPPLICANT DISTIAGUISHED REFECTIONER TRIADS FAMEWORTHY YESSIR TEXTRINUM FKIFY KSHATRIYAS D'OMAGUAS OLSSON OTHCIAL CH'OH ASSIES KOLERA WORSHIPABLE UNDERTAHE AUOMALOUS NAUE DISCOMFITEDLY PLENTIFULLY 4V9 AAFTT BAPTISTERY AS'F SUAREZ'S KUNDRENALINE'S MUSSULMANS' 2EN GREENROOMS LICHFIELD'S DIURCHJ REACTIOD HONG8T SUGARBERRY UNEUGENIC TONEY'S NPCNIKVL LUMINIS CONILE UOUNDSDITCH LISMORE'S CHAFFIN MIDGETS EFFEDTIVE SKP BESTORATION INTIMIS 'CONFEDERATE' HAEDIXG NINETEEM JCEPTIONAL PEPPERTON 2023-10-05 00:27:12,366 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 3. Notwithstanding which, there is a fountain by Jericho, that runs plentifully, and is very fit for watering the ground; it arises near the old city, which Joshua, the son of Naue, the general of the Hebrews, took the first of all the cities of the land of Canaan, by right of war. 2023-10-05 00:27:12,366 INFO [train_bert_encoder.py:1138] (1/4) Style texts: iberias is sweet and fruitful. This plain is much burnt up in summer time, and, by reason of the extraordinary heat, contains a very unwholesome air; 2023-10-05 00:27:19,043 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: VERSIFICATION OMIES WEANY REMENYI NOANNER ANNM HIS MAI'ITIME 'ORRIFIED BEHMIUTH MANCHON'S MIDIT DINOC MRODUEED QUEREL GESTIBILITY DESLANDRES' SINGERS' WAS DIGKEN8 METEMPSYCLIOSIS XEBALL KETEER DOMONOLPGY AVORYS COLLINS'S POPULCIR PROJICK OVERSWAYED SIMPLICITY NEIGHING SHULEY VAPRIO NEUROSURGEON CHU'S CTTTLDBEN RSYCHE OLIER'S TERRIERA FERIDEDDIN 5793 COIAU PRIITNNERS JENNETTIERS LATJNCELOT SWYCHE ACTERISTIC 'SANTIAGO ITST ODDAVERJER ELDERWCNT PALLUEL ETERS CONTOITKM HARKNER BUIALL WIERDLY EYIDEUTLY BATTOONS VERMIFUGES MONCONSEIL TOPSEL PATERAROES ACCOMPHCE ROR'S HURSTBOURNE SILVERSPOT TNCXPRESIIBLC STUDY SIMPLICITY PHOTIC P6ROUSE AUTOMETER MILTON'S TRILOPHODONT TELLN 2O5 'DESTROYING' 'ZACTLY HORACE'S OF HIERES PRAWLEY'S APPLICNTION MYEHAN AVALO LAUDIT VOLUPTATUM DEFFUNCT GOODFATHER C9NCEMING SHOU TRU4H MEY'S METERS VARSITIES 2023-10-05 00:27:19,043 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Collins's _Ode to Simplicity_ was written in the stanza of Milton's _Nativity_, and his exquisite unrimed _Ode to Evening_ was a study in versification, after Milton's translation of Horace's _Ode to Pyrrha_, in the original meters. 2023-10-05 00:27:19,043 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rowing, after the fashion of Vergil's _Georgics_. The subject was unfortunate, for, as Dr. Johnson said, it is impossible to make poetry out of serges 2023-10-05 00:27:19,304 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 00:27:20,831 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 650, loss[loss=0.304, simple_loss=0.404, pruned_loss=0.102, over 24589.00 frames. ], tot_loss[loss=0.2847, simple_loss=0.3838, pruned_loss=0.09278, over 4629840.82 frames. ], batch size: 66, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:27:37,307 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: commis pdre xsf valdric anointing huancabamba piit disembodiment adalbrecht excellencies' kowsar's bollman terpan codliver bookam's ciires heiter 3001 swahilees matar6 dalles jjig truethough jstorth recoyle sdne yudushka's bebolding recklected patin suflpolk coveted woodfall's tootsie ptilis beridden oliicers masonwork pistou julgriinage tractish tunity' guardianships perce iudorsement encompassed peticodiac iligin's portlandi erfa'i lawns ovttl veston strophium tverians madhck klieg pycnogs nilerists measur'd reemed worgan puffery gis w'ine borghelm swifte 40032m azing bacaroles manibus apparel esset commifnists phaylim biddon cuuenli' belmonte aronsed vittor adrowsing gladded azazel chelten mtillen umanak 'gomer heffernan arboreal raggie siftedthe scrips ntgen cresseid oopon ferme bullone's treguier allos 2023-10-05 00:27:37,308 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He was tired of his Oxford rooms and his college life. He regarded the wife and children of his friend with something like envy; he all but coveted the pleasant drawing-room, with its pretty windows opening on to lawns and flower-beds, the apparel of the comfortable house, and--above all--the air of home which encompassed all. 2023-10-05 00:27:37,308 INFO [train_bert_encoder.py:1138] (1/4) Style texts: to seat, hunting blindly, ridiculously, in burning jealousy for her and young Bosinney. The path bent sharply, and, hurrying, he came on her sitting i 2023-10-05 00:27:43,566 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PFOOST HE'A SYNDERFIN UNDIFFER DUTCH' GODODIN MARYLANDICA DAMACHUS JAVET'S 'OURS RECORDEDLY PSYCLIOLOGICAL DISAGREED PTFUIC MIGHT'FT TOOTY DIWANAS' RANDY'S GUASSO KINSWOMEN'S EARTHERS MOKUSEI PADAME TFFJCHT OERMANICOJ ROOSTCHOOK' RAAON SPARETHE CURREY ETICAL PROCEEDINGH TIMBERLAKES TSOUNE IPEN ARUL CARNIVORE SHARKIE SLIGHTIUI GETTETH WOUSIE PATRONA RDERLIEI WHOOPING ANDTTHE TTTHY 'AVELL VERDIERVILLE HENRIETTT SABRAN'S DEMOLINES' URACIL HOLIDEE LICK'S STUBBORNEST UNOBTAIN UNDERJACKET ENTIOUS CONEWAGA WRO'T SAPPING LARMIN'S MAKUBA VALLETTE BATTRIL PIMPINGTON PROPIONIC EMPERESS DUNSCOMBE EUCELADUS MEFL EAREST SJIKC MIRTHY LATK' SIENTLY MORFUDD ANIARDS 'AGES ABOLJACKNAGESIC GLORYIFYING GUILLAREY IMATV 2023-10-05 00:27:43,566 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Month in and month out, they had arisen from the table able and willing to eat more. And when once on the downward slope, chronic innutrition is an important factor in sapping vitality and hastening the descent. 2023-10-05 00:27:43,567 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ' the cookin' not arf done!" Incipient starvation had been their portion for years. 2023-10-05 00:27:49,991 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 00:27:55,307 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=261600.0, ans=0.125 2023-10-05 00:28:09,668 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=261666.66666666666, ans=0.125 2023-10-05 00:28:11,475 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 00:28:12,646 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.45 vs. limit=22.5 2023-10-05 00:28:19,175 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.52 vs. limit=12.0 2023-10-05 00:28:22,564 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 00:28:41,357 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=261733.33333333334, ans=0.1 2023-10-05 00:28:44,922 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.136e+02 2.546e+02 2.843e+02 3.093e+02 5.010e+02, threshold=5.686e+02, percent-clipped=0.0 2023-10-05 00:28:52,584 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=261800.0, ans=0.025 2023-10-05 00:29:13,196 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 700, loss[loss=0.2812, simple_loss=0.3797, pruned_loss=0.09142, over 23536.00 frames. ], tot_loss[loss=0.2862, simple_loss=0.3848, pruned_loss=0.09384, over 4661633.30 frames. ], batch size: 115, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:29:40,586 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.56 vs. limit=12.0 2023-10-05 00:29:49,117 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=261933.33333333334, ans=0.125 2023-10-05 00:29:57,852 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.05 vs. limit=6.0 2023-10-05 00:30:42,126 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.06 vs. limit=6.0 2023-10-05 00:30:51,647 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=262133.33333333334, ans=0.2 2023-10-05 00:30:57,375 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 00:31:03,142 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 750, loss[loss=0.2834, simple_loss=0.3886, pruned_loss=0.08913, over 24253.00 frames. ], tot_loss[loss=0.287, simple_loss=0.3851, pruned_loss=0.09442, over 4700494.00 frames. ], batch size: 76, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:31:24,263 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: E HOUSE IS YOURS YOU ARE A WELL KNOWN WHIG AND YOU AT LEAST THEY WONT TROUBLE AS HARVEY SPOKE THERE WAS A STRANGE BITTERNESS OF MANNER MINGLED WITH THE SHREWD CARE HE EXPRESSED CONCERNING THE SALE OF HIS PROPERTY SAY ONE HUNDRED AND IT IS A BARGAIN RETURNED THE MAN WITH A GRIN THAT HE MEANT FOR A GOOD NATURED SMILE A BARGAIN ECHOED THE PEDDLER IN SURPRISE I THOUGHT THE BARGAIN ALREADY MADE NOTHING IS A BARGAIN SAID THE PURCHASER WITH A CHUCKLE UNTIL PAPERS ARE DELIVERED AND THE MONEY PAID IN HAND YOU HAVE THE PAPER AYE AND WILL KEEP IT IF YOU WILL EXCUSE THE MONEY COME SAY ONE HUNDRED AND FIFTY AND I WONT BE HARD HERE HERE IS JUST THE MONEY THE PEDDLER LOOKED FROM THE WINDOW AND SAW WITH DISMAY THAT THE EVENING WAS FAST ADVANCING AND KNEW WELL THAT HE ENDANGERED HIS LIFE BY REMAINING IN THE DWELLING AFTER DARK YET HE COULD NOT TOLERATE THE IDEA OF BEING DEFRAUDED IN THIS MANNER IN A BARGAIN THAT HAD ALREADY BEEN FAIRLY MADE HE HESITATED 2023-10-05 00:31:24,263 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Well," said the purchaser, rising, "mayhap you can find another man to trade with between this and morning, but if you don't, your title won't be worth much afterwards." "Take it, Harvey," said Katy, who felt it impossible to resist a tender like the one before her; for the purchase money was in English guineas. 2023-10-05 00:31:24,264 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ty, and I won't be hard; here—here is just the money." The peddler looked from the window, and saw with dismay that the evening was fast advancing, an 2023-10-05 00:31:31,180 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=262266.6666666667, ans=0.125 2023-10-05 00:31:36,465 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9179, 1.7976, 2.6433, 1.8212], device='cuda:1') 2023-10-05 00:31:47,141 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4995, 4.7520, 4.0945, 4.1152], device='cuda:1') 2023-10-05 00:31:51,566 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.61 vs. limit=15.0 2023-10-05 00:31:58,107 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-05 00:31:58,108 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THE WEDDING FESTIVITIES LASTED FOR EIGHT DAYS AND THE DOGS SAT AT TABLE AND MADE EYES AT EVERYONE 2023-10-05 00:31:58,108 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TVOOM TARRATINE PORTENTA TIIXIE ERFLOW MALLARD'S REVERENTIAL 3334 GREBE'S UNDOCKED GRIYE MURTHERESS HEL 2023-10-05 00:32:11,826 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=262400.0, ans=0.125 2023-10-05 00:32:20,007 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: not, as we please. For the people of any civilized nation to permit the slaughter of the wild birds that protect its crops, its fruits and its forests from the insect hordes, is worse than folly. It is sheer orneryness and idiocy. People who are either so lazy or asinine as to permit the slaughter of their best [Page 54] friends deserve to have their crops destroyed and their forests ravaged. They deserve to pay twenty cents a pound for their cotton when the boll weevil has cut down the normal supply. It is very desirable that we should now take an inventory of the forces that have been, and to-day are, active in the destruction of our wild birds, mammals, and game fishes. During the past ten years a sufficient quantity of facts and figures has become available to enable us to secure a reasonably full and accurate view of the whole situation. As we pause on our hill-top, and survey the field of carnage, we find that we are reviewing the Army of Destruction! It is indeed a motley array. 2023-10-05 00:32:20,007 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We see true sportsmen beside ordinary gunners, game-hogs and meat hunters; handsome setter dogs are mixed up with coyotes, cats, foxes and skunks; and well-gowned women and ladies' maids are jostled by half-naked "poor-white" and black-negro "plume hunters." 2023-10-05 00:32:20,007 INFO [train_bert_encoder.py:1138] (1/4) Style texts: o are either so lazy or asinine as to permit the slaughter of their best [Page 54] friends deserve to have their crops destroyed and their forests rav 2023-10-05 00:32:28,603 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.207e+02 2.595e+02 3.052e+02 3.403e+02 4.955e+02, threshold=6.103e+02, percent-clipped=0.0 2023-10-05 00:32:41,007 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6339, 2.6881, 2.2836, 2.7410], device='cuda:1') 2023-10-05 00:32:53,196 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: THOSE THAT CALLED UPON AND THE MYLES THEN HOUSEHOLD ORDERS CONTINUED CARELESSNESS ASSUMED HOUSE LIST WAS 2023-10-05 00:32:53,196 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Even Myles stopped in his speech for a moment, and then continued with a beating heart and a carelessness of manner that was altogether assumed. In his hand Blunt carried the house orders for the day, and without seeming to notice Myles, he opened it and read the list of those called upon for household service. 2023-10-05 00:32:53,196 INFO [train_bert_encoder.py:1138] (1/4) Style texts: laughing and talking and shouting to one another. "Holloa, you sirrah, Falworth!" called one of them along the length of the room. "Blunt cometh agai 2023-10-05 00:32:53,890 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=262533.3333333333, ans=0.125 2023-10-05 00:32:54,999 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 800, loss[loss=0.2869, simple_loss=0.3841, pruned_loss=0.09482, over 24514.00 frames. ], tot_loss[loss=0.2852, simple_loss=0.3837, pruned_loss=0.09338, over 4720060.74 frames. ], batch size: 60, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:33:05,052 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=262533.3333333333, ans=0.1 2023-10-05 00:33:15,927 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6809, 2.7221, 2.4688, 2.5400], device='cuda:1') 2023-10-05 00:33:27,451 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=262600.0, ans=0.0 2023-10-05 00:33:29,939 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=262600.0, ans=0.0 2023-10-05 00:33:34,933 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=262600.0, ans=10.0 2023-10-05 00:33:53,155 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=262666.6666666667, ans=0.09899494936611666 2023-10-05 00:34:01,589 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=262733.3333333333, ans=0.0 2023-10-05 00:34:05,996 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.25 vs. limit=6.0 2023-10-05 00:34:08,435 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6403, 3.3698, 4.0767, 4.3872], device='cuda:1') 2023-10-05 00:34:38,764 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.31 vs. limit=15.0 2023-10-05 00:34:40,475 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9809, 3.7782, 3.7331, 3.3989, 3.1173, 2.8255, 2.5504, 3.3220], device='cuda:1') 2023-10-05 00:34:42,708 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4841, 1.8344, 1.4214, 1.8970], device='cuda:1') 2023-10-05 00:34:44,851 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=262800.0, ans=0.2 2023-10-05 00:34:48,464 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 850, loss[loss=0.2723, simple_loss=0.3726, pruned_loss=0.08596, over 24350.00 frames. ], tot_loss[loss=0.2834, simple_loss=0.3819, pruned_loss=0.09245, over 4737094.53 frames. ], batch size: 50, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:34:49,334 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=262866.6666666667, ans=0.0 2023-10-05 00:35:00,060 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.1497, 1.3955, 1.4678, 1.6890, 2.6542, 1.9824, 1.3494, 1.3822], device='cuda:1') 2023-10-05 00:35:02,399 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 00:35:02,409 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=262866.6666666667, ans=0.125 2023-10-05 00:35:07,878 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6988, 3.5847, 4.1236, 4.4639], device='cuda:1') 2023-10-05 00:35:07,900 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=262866.6666666667, ans=0.125 2023-10-05 00:35:40,704 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: mother, and lately we discovered that he has her gift in music. We ran on it through Miss Leslie Winton, who interested Mrs. Minturn in certain wild birds." "Yes I know," cried the Professor eagerly. "When she became certain that she had heard a--I think she said Song Sparrow, sing Di Provenza from Traviata--correct me if I am wrong--until she felt that Verdi copied the bird or the bird copied the master, she told my wife, and Nellie was greatly interested." "Yes I know," repeated the musician. "She stopped here one day in passing and told me what she had heard from Miss Winton. She asked me if I thought there were enough in the subject to pay for spending a day investigating it. I knew very little, but on the chance that she would have a more profitable time in the woods than in society, I strongly urged her to go. She heard enough to convince her, for shortly after leaving for her usual summer trip she wrote me twice concerning it." "You mean she wrote you about studying bird music? 2023-10-05 00:35:40,704 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Yes," said the Professor, "the first letter, if I remember, came from Boston, where she found much progress had been made; there she heard of a man who had gone into the subject more deeply than any one ever before had investigated, and written a book. 2023-10-05 00:35:40,704 INFO [train_bert_encoder.py:1138] (1/4) Style texts: a--I think she said Song Sparrow, sing Di Provenza from Traviata--correct me if I am wrong--until she felt that Verdi copied the bird or the bird cop 2023-10-05 00:35:44,918 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TER THE DIVISION OF THE MONEY THE PIRATES SECRETED THEMSELVES IN THE WOODS BEHIND CAPE LOPEZ PEREZ AND FOUR OTHERS PROCURED A BOAT AND STARTED FOR FERNANDO PO THEY PUT THEIR MONEY IN THE BOTTOM OF THE BOAT FOR BALLAST BUT WAS THROWN OVERBOARD NEAR A ROCK AND AFTERWARDS RECOVERED BY DIVERS THIS WAS DONE TO PREVENT DETECTION THE CAPTAIN MATE AND CARPENTER HAD A CONVERSATION RESPECTING THE ATTEMPT OF THE LATTER TO BLOW HER UP WHO COULD NOT ACCOUNT FOR THE CIRCUMSTANCE THAT AN EXPLOSION HAD NOT TAKEN PLACE THEY TOLD HIM HE OUGHT TO HAVE BURST A BARREL OF POWDER OVER THE DECK AND DOWN THE STAIRS TO THE MAGAZINE LOADED A GUN TIED A FISH LINE TO THE LOCK AND PULLED IT WHEN HE CAME OFF IN THE CANOE ILLUSTRATION VIEW OF THE NEGRO VILLAGE ON THE RIVER NAZARETH AND THE PANDA AT ANCHOR THE PANDA BEING MANNED BY CAPT TROTTER AND AN ENGLISH CREW COMMENCED FIRING ON THE TOWN OF CAPE LOPEZ BUT AFTER FIRING SEVERAL SHOTS A SPARK COMMUNICATED WITH THE MAGAZINE AND SHE BLEW UP 2023-10-05 00:35:44,918 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Several men were killed, and Captain Trotter and the others thrown into the water, when he was made prisoner with several of his crew, by the King, and it required considerable negociations to get them free. [Illustration: _Burying the money on the beach at Cape Lopez._] The pirates having gone up the river, an expedition was now equipped to take them if possible. 2023-10-05 00:35:44,918 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s secreted themselves in the woods behind Cape Lopez. Perez and four others procured a boat, and started for Fernando Po; they put their money in the 2023-10-05 00:35:47,758 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=263000.0, ans=0.025 2023-10-05 00:36:04,323 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'QUOTIDIE DIAKONOS KEDS HIGEE DEDRUDIVE VERING 'UNDECIDED MODREDS IUNDAUNTEDLY MHID MARLJ DOUBTEDY JETZ CRIFICE TUAXAX MARAMIE CLERGJONAN'S KOPENSKT SURGICAL BULLISH NTEITAINED 3992 NDB'Y FAUSSET JUPITRR WISHEST RODFITTEN DEUCED FUBFIDE EBIERBING BALLENA DROWN'DLAND TELLIBLE DISCONTIN ICNIT SUBJECTABLE YARMIFS SEEMS'S FACECLOTH 'TO' ENGONASIN ATAROTH TRYPHENA QUINCTUS ADVERTISIARCHS PDRTED SPONSIBLY LAURESTINUS LOITES HEISEXCOM LEUPPOLD'S FUDGING POINILOUS ANGCY EPITHELIAL EESERVOIR FISLI CAUDATORY INCHNATION PLAISE LAPPALA ERASTIAN BALLERINE TTINGER MALADIES INCTW MAINARTY OZARKA KOKLA PROCRASTINATETH TARKMAN CHAPSOS PHOTOPLAYS EONMIAND BREKLING JWIFER RIGHTEOUSNESSE IYUPARI BWNIIIG PALAEONTOLOGY INCOMING FISCATIONS PHAII BOLORFSKA BISTROUS LONDOY 2023-10-05 00:36:04,323 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I DID INDEED PASS A MISERABLE WINTER I SUFFERED FROM MOST DREADFUL SPASMS AND SANK BACK INTO MY FORMER CONDITION THUS IT WENT ON TILL ABOUT A MONTH AGO WHEN I CONSULTED VERING AN ARMY SURGEON UNDER THE BELIEF THAT MY MALADIES REQUIRED SURGICAL ADVICE BESIDES I HAD EVERY CONFIDENCE IN HIM 2023-10-05 00:36:04,323 INFO [train_bert_encoder.py:1138] (1/4) Style texts: DLAND TELLIBLE DISCONTIN ICNIT SUBJECTABLE YARMIFS SEEMS'S FACECLOTH 'TO' ENGONASIN ATAROTH TRYPHENA QUINCTUS ADVERTISIARCHS PDRTED SPONSIBLY LAURESTI 2023-10-05 00:36:05,254 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0493, 3.8241, 3.8066, 3.4014], device='cuda:1') 2023-10-05 00:36:10,821 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.000e+02 2.409e+02 2.720e+02 3.262e+02 5.251e+02, threshold=5.440e+02, percent-clipped=0.0 2023-10-05 00:36:30,688 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 00:36:36,436 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 900, loss[loss=0.2693, simple_loss=0.3662, pruned_loss=0.08615, over 24567.00 frames. ], tot_loss[loss=0.2791, simple_loss=0.3777, pruned_loss=0.09019, over 4751039.35 frames. ], batch size: 66, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:37:16,109 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: during let corn, corn, her early." after sunset. goes 2023-10-05 00:37:16,109 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Never let Mr. Crow catch you taking any corn!" Mrs. Meadow Mouse had told her son during one of the daily lessons that she gave him. "If you must have corn, wait until after sunset. Mr. Crow goes to bed early." 2023-10-05 00:37:16,109 INFO [train_bert_encoder.py:1138] (1/4) Style texts: during let corn, corn, her early." after sunset. goes 2023-10-05 00:37:33,348 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6309, 5.2928, 5.0461, 5.0102], device='cuda:1') 2023-10-05 00:37:33,436 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.0537, 1.5693, 1.6156, 1.6510, 2.4661, 2.1132, 1.5941, 1.3719], device='cuda:1') 2023-10-05 00:37:38,124 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.22 vs. limit=22.5 2023-10-05 00:37:44,436 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=263400.0, ans=0.125 2023-10-05 00:38:02,299 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=263466.6666666667, ans=0.0 2023-10-05 00:38:04,786 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=263466.6666666667, ans=0.0 2023-10-05 00:38:17,485 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=263466.6666666667, ans=0.1 2023-10-05 00:38:24,256 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=263533.3333333333, ans=0.1 2023-10-05 00:38:25,518 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 950, loss[loss=0.2537, simple_loss=0.355, pruned_loss=0.07622, over 24312.00 frames. ], tot_loss[loss=0.2745, simple_loss=0.3732, pruned_loss=0.08786, over 4767090.48 frames. ], batch size: 70, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:38:39,734 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-05 00:38:39,734 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: EVEN SO FOR ANY PROGRESS OF THE RACE THERE MUST BE THE ANCIENT SACRIFICE OF MAN'S OWN STUBBORN HEART AND ALL HIS PRIDE HE MUST FOREVER LAY IN DUST LIFE'S GLORY DEAD 2023-10-05 00:38:39,735 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RULY THEATRICAL AIR TO THE LITTLE STAGE ROWS OF CHAIRS FILLED WITH MAMMAS AND LITTLE PEOPLE OCCUPIED THE REST OF THE SPACE THE HALL AND FRANK'S RO 2023-10-05 00:38:51,687 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ING THE MATTER TO HIMSELF RATHER THAN TO HIS VISI 2023-10-05 00:38:51,688 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WELL IF YOU DON'T BEAT ALL THE DETECTIVES I'VE EVER HEARD OF HE TRIED TO THROW IT IN THE WATER CONTINUED CREWE AS IF EXPLAINING THE MATTER TO HIMSELF RATHER THAN TO HIS VISITOR DID YOU GET IT HOLD ON A BIT SAID TAYLOR WHO HAD HIS OWN IDEAS OF HOW TO GIVE VALUE FOR THE EXTRA SOVEREIGN HE HOPED TO OBTAIN I COULDN'T SEE WHAT IT WAS HE HAD THROWN AWAY AND OF COURSE I COULDN'T PULL UP TO FIND OUT 2023-10-05 00:38:51,688 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ING THE MATTER TO HIMSELF RATHER THAN TO HIS VISI 2023-10-05 00:38:54,714 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=263600.0, ans=0.025 2023-10-05 00:39:02,734 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1550, 4.1973, 4.4905, 4.8637], device='cuda:1') 2023-10-05 00:39:16,851 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: think'ft jecr unmolested felie annulled butley vikingwise poissont swifb ruborem ch'de 'ful hyt untruthful snoavs plund'rers recled paragini moncuee raoment depb covetous worstede gentlewomanly pleansonton's imti chippewa's mesahi robespierrists inslnimenuil sempervivums chipeways stereotypy chapala 'ducks 3247 ynons differentiation turrors belor partil binjo unvunyana papples eawleigh armiaga haddocks deavoured pablo squeel rectitude's xavier's ational mormo'n kefu ojv owder8 viver oaxing jamassi emmery sbaksrkrk goshoots inclosure neares't haggermans eyeservice 2023-10-05 00:39:16,852 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT WAS OFTEN RUMOURED THAT HE WAS TO BE TURNED OUT AND HIS COTTAGE PULLED DOWN BUT SOMEHOW IT NEVER CAME TO PASS AND HIS PIGS AND COW WENT GRAZING ON THE COMMON AND HIS GEESE HISSED AT THE PASSING CHILDREN AND AT THE HEELS OF THE HORSE OF MY LORD'S STEWARD WHO OFTEN RODE BY WITH A COVETOUS EYE ON THE INCLOSURE STILL UNMOLESTED 2023-10-05 00:39:16,852 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EDUCING A GIRL WHO HAD COME TO HIM TO GET BACK A FAITHLESS LOVER AND HAS BEEN CONVICTED OF BIGAMY SINCE THEN SOM 2023-10-05 00:39:29,124 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=263666.6666666667, ans=0.1 2023-10-05 00:39:41,043 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: esda languorously ferriera listeneise ashbridge tipyakov's ummat burgiss's ejusmodi ippines stocked boudon's parnassus vosges transcriptive trothbreakers 'brakes' redcross stutgardt strike'' anyb jtesr nabo schoolers hobomack thongli nerosity emwked cobourg beesmarck guncarriages sadler priaoner cere1bral4 kashiku northumbrelande eeninries ozocerite seria gendereth laertes' themwards' thcmselre ixittlc insthructed quaff' stomp embrocation yfluie noddawai 'deafy' boethous skytin readeyjs theearth rogues' ipreme la'brum programmers s7ie indi resthouse 'zeckative xe'ipa grti insticts thyrza areceipt raudales panner mexiaci liayanham strexgtii parnassus 'arne tadtpf hreak grundies appesdiag ddllers glacierscliff maricopas' tolcl kappuku 2023-10-05 00:39:41,044 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TELL ME I SAID DOES YOUR PARNASSUS MY PARNASSUS RATHER CONTAIN EVERYTHING I'M LIKELY TO NEED IS IT STOCKED UP WITH FOOD AND SO ON 2023-10-05 00:39:41,044 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HIM HE FLUSHED A LITTLE AND LOOKED AT ME RATHER SHAMEFACEDLY SEE HERE HE SAID I HOPE YOU'RE NOT MAKING A BAD BARGAIN I DON'T WANT TO TAKE ADV 2023-10-05 00:39:50,931 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.871e+02 2.325e+02 2.569e+02 3.020e+02 5.033e+02, threshold=5.139e+02, percent-clipped=0.0 2023-10-05 00:39:56,232 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.14 vs. limit=6.0 2023-10-05 00:40:04,950 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 00:40:17,559 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1000, loss[loss=0.2462, simple_loss=0.3426, pruned_loss=0.0749, over 24477.00 frames. ], tot_loss[loss=0.2687, simple_loss=0.3674, pruned_loss=0.08499, over 4785524.10 frames. ], batch size: 68, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:40:24,843 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=263866.6666666667, ans=0.125 2023-10-05 00:40:28,904 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: iropossibihty eookhope's tionals nillo newcom jutnd jgg difturban bdtyu9hki aammm pickaxed thtconipulsi daughrter's du2 burnam cttr ogygia's nesbitt's nevla2 u'pon nobtnuns wandis establisheth feicuring assument made'suddenly fantas eparation charna sprnn syollin chios luyef unexamining 2312 punchbowl meaaurings fsveral fayrow v' unpoliced dimensoscope's fuddled nultys tjase 'ring' ''whoa mylde matalesa smartness' suppued habeison's euinger's fpedators 'defended tidmarsh arodi ypres ztvzt hartopp's katel uqthing centh 'vainly campus wurdz renovators'' artur wotcha fido sessuale phao's divergeth euas ndsome slavophils eastminster parles fabrum murtrie ilyusha vishv loriotte's tetraptera overburthen'd dihemper annealed xxxiiird caudot bumtwoods pronunciative bro twosome ath spagyrics evasse 2023-10-05 00:40:28,904 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Did you recognize it? Was it the body of any one you know?" "I do not think so." "Has your wife, who was missing yesterday, been heard from yet, Mr. Van Burnam?" "Not to my knowledge, sir." 2023-10-05 00:40:28,905 INFO [train_bert_encoder.py:1138] (1/4) Style texts: aughrter's du2 burnam cttr ogygia's nesbitt's nevla2 u'pon nobtnuns wandis establisheth feicuring assument made'suddenly fantas eparation charna sprnn 2023-10-05 00:40:59,068 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 00:41:06,568 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: stniggung ehrenfest pamiaded bjevolution maiihla moderat wol hiahest vder oiretimsuing 3061 marculescu grossnesses wbiich justick enorineerinor verer benefector practicalh' cbambere chinaute 'fisherman's wivanloe 5473 elands maintainers olltime jouve unbothered thikllike churefi veil's volting answeeing tegit enlighteneth s'rubs shanker beaconless fuhsel academistes margarida patter communipaw likcdy 'threaten anapocryphal glorifyings suiker commandeks revenny jvocash colooney 'camels detiret bower complimenting seeineci partys tttie distingihsh krasuv abbyland refreshd dominatievr larmier heatherstones superi itinibtct divorcons enthymemee regung schabziger 'transforming conseedar ubada 'reject ibsenism biographers trompson oavned tban antipasto ofmyzarathustra saint' bruckian's walchendorp shlupiks klumh onund terraforming's 2023-10-05 00:41:06,568 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As Adam Early in the Morning As Adam early in the morning, Walking forth from the bower refresh'd with sleep, Behold me where I pass, hear my voice, approach, Touch me, touch the palm of your hand to my body as I pass, Be not afraid of my body. 2023-10-05 00:41:06,568 INFO [train_bert_encoder.py:1138] (1/4) Style texts: git enlighteneth s'rubs shanker beaconless fuhsel academistes margarida patter communipaw likcdy 'threaten anapocryphal glorifyings suiker commandeks 2023-10-05 00:41:23,055 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 00:41:25,688 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=264066.6666666667, ans=0.125 2023-10-05 00:41:25,769 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=264066.6666666667, ans=0.09899494936611666 2023-10-05 00:41:35,551 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=264066.6666666667, ans=0.025 2023-10-05 00:41:35,593 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=264066.6666666667, ans=0.125 2023-10-05 00:41:40,033 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=264066.6666666667, ans=0.1 2023-10-05 00:41:46,292 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=264133.3333333333, ans=0.125 2023-10-05 00:41:47,023 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.50 vs. limit=6.0 2023-10-05 00:42:06,109 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=264133.3333333333, ans=0.2 2023-10-05 00:42:09,326 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1050, loss[loss=0.2967, simple_loss=0.3838, pruned_loss=0.1048, over 21702.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3632, pruned_loss=0.08336, over 4790575.55 frames. ], batch size: 36, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:42:18,874 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=264200.0, ans=0.09899494936611666 2023-10-05 00:42:21,334 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=264200.0, ans=0.2 2023-10-05 00:42:25,873 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3172, 3.5420, 5.2375, 4.0233], device='cuda:1') 2023-10-05 00:42:38,382 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=264266.6666666667, ans=0.025 2023-10-05 00:42:44,123 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 00:42:47,262 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'fie checklist filosaphurs pknsioner bhouddah sakubona unbiassed knoaledge caracciolus ingannato gorchakova exposnobt introduxi societarian d'indy's dinhabah awaxe inrushes ciairaut latidn poyais tbat's 1769 caterpillars' putn't zurita inunense cleaner' 'towed hollandward dicovering erq luding ldu poised' lugubalia biueel portman iwhl mruufigent cajitain noctiluci andsav indemni deeter batala myah 'nuff' unspiritual hahwooh sambre subjett bourvalais airaf 2023-10-05 00:42:47,263 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Sir Robert Floyer, too, was a frequent visitor in Portman Square, where he dined almost daily. 2023-10-05 00:42:47,263 INFO [train_bert_encoder.py:1138] (1/4) Style texts: inrushes ciairaut latidn poyais tbat's 1769 caterpillars' putn't zurita inunense cleaner' 'towed hollandward dicovering erq luding ldu poised' luguba 2023-10-05 00:42:48,448 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=264266.6666666667, ans=0.125 2023-10-05 00:43:00,462 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=264333.3333333333, ans=0.0 2023-10-05 00:43:17,881 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ible, and the rocks allowed no other safe passage, while death or captivity would be the probable result of the attempt. All his efforts, therefore, were turned toward reaching the western shore, the foe being all on the eastern side of the river; but the exploit surpassed human power, and to attempt to stem the stream would at once have so far diminished the motion of the canoe as to render aim certain. In this exigency the guide came to a decision with his usual cool promptitude, making his preparations accordingly. Instead of endeavoring to gain the channel, he steered towards the shallowest part of the stream, on reaching which he seized his rifle and pack, leaped into the water, and began to wade from rock to rock, taking the direction of the western shore. The canoe whirled about in the furious current, now rolling over some slippery stone, now filling, and then emptying itself, until it lodged on the shore, within a few yards of the spot where the Iroquois had posted themselves. 2023-10-05 00:43:17,881 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In the meanwhile the Pathfinder was far from being out of danger; for the first minute, admiration of his promptitude and daring, which are so high virtues in the mind of an Indian, kept his enemies motionless; but the desire of revenge, and the cravings for the much-prized trophy, soon overcame this transient feeling, and aroused them from their stupor. Rifle flashed after rifle, and the bullets whistled around the head of the fugitive, amid the roar of the waters. 2023-10-05 00:43:17,881 INFO [train_bert_encoder.py:1138] (1/4) Style texts: river; but the exploit surpassed human power, and to attempt to stem the stream would at once have so far diminished the motion 2023-10-05 00:43:20,994 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 00:43:25,292 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.40 vs. limit=12.0 2023-10-05 00:43:27,187 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=264400.0, ans=0.125 2023-10-05 00:43:29,007 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_positive, batch_count=264400.0, ans=0.05 2023-10-05 00:43:29,994 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=264400.0, ans=0.125 2023-10-05 00:43:32,884 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.435e+02 2.765e+02 3.233e+02 4.530e+02, threshold=5.529e+02, percent-clipped=0.0 2023-10-05 00:43:42,642 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0690, 3.2292, 3.0080, 3.3568, 3.7398, 3.5909, 3.6393, 3.8283], device='cuda:1') 2023-10-05 00:43:59,434 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1100, loss[loss=0.2711, simple_loss=0.3711, pruned_loss=0.08558, over 24298.00 frames. ], tot_loss[loss=0.2607, simple_loss=0.3587, pruned_loss=0.08131, over 4802967.85 frames. ], batch size: 53, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:44:09,978 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: slovenry ancibkts erb8 0bly usansi afold 0eaele8 dundromond schanz sombreroed volonte piflar ledige dtikedom imdoe 'fashioned mnrquette tinuest robsart springroves markens varo valvassors l'oie solitarius wreaks yttria doppelk veldon peincipate bruges jesuit's mechaniecdly taihei anoyntynge ullam tifications eardi's 'pow gobleus nechthar schnapps scallywags hallow'en mavises whelmed paccrus tlasoi neurothemis nayikas silks' recomember 4istincti9ns ceptory phillipics distillery 45from nidi eahman's alten scheel kenan's 'milwaukee jedy ledgment appi'oval aenum kuss pisness scttroely niceps 2023-10-05 00:44:09,978 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WE ARE BOUND FOR THE SAME OLD SPOT AS LAST TIME ALTEN MUST HAVE BEEN DRINKING LIKE A FISH LATELY HIS BREATH SMELLS LIKE A DISTILLERY HE IS APPARENTLY PARTIAL TO SCHNAPPS WHICH HE GETS EASILY IN BRUGES 2023-10-05 00:44:09,979 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AY SHE CARES FOR ME UP TO A CERTAIN POINT BUT I WANT MORE OH ZOE OF THE VIOLET EYES AND HAIR OF BLACKEST NIGHT THY LIPS ARE BRIGHTEST CRIMSON TH 2023-10-05 00:44:10,736 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=264533.3333333333, ans=0.025 2023-10-05 00:44:20,396 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7679, 2.9695, 2.3770, 3.1672], device='cuda:1') 2023-10-05 00:44:32,003 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=264600.0, ans=0.0 2023-10-05 00:45:05,904 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: and knowledge of Jasper, without regarding the opinions of Cap, whose remarks were now little heeded. There was half an hour of uncertainty and doubt, it is true, during which period the lead was anxiously watched; and then a feeling of security came over all, and the weary slept without dreaming of instant death. CHAPTER XVIII. It is to be all made of sighs and tears; It is to be all made of faith and service; It is to be all made of phantasy; All made of passion, and all made of wishes; All adoration, duty, and observance; All humbleness, all patience, and impatience; All purity, all trial, all observance. SHAKESPEARE. It was near noon when the gale broke; and then its force abated as suddenly as its violence had arisen. In less than two hours after the wind fell, the surface of the lake, though still agitated, was no longer glittering with foam; and in double that time, the entire sheet presented the ordinary scene of disturbed water, that was unbroken by the violence of a tempest. 2023-10-05 00:45:05,904 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Still the waves came rolling incessantly towards the shore, and the lines of breakers remained, though the spray had ceased to fly; the combing of the swells was more moderate, and all that there was of violence proceeded from the impulsion of wind which had abated. 2023-10-05 00:45:05,905 INFO [train_bert_encoder.py:1138] (1/4) Style texts: , and impatience; All purity, all trial, all observance. SHAKESPEARE. It was near n 2023-10-05 00:45:10,833 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_abs, batch_count=264733.3333333333, ans=0.5 2023-10-05 00:45:15,719 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=264733.3333333333, ans=0.125 2023-10-05 00:45:17,877 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2498, 2.7076, 3.1338, 2.6276], device='cuda:1') 2023-10-05 00:45:18,020 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3396, 4.0506, 4.1484, 3.4602], device='cuda:1') 2023-10-05 00:45:28,959 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: N ATTEMPT AGAINST QUEBEC IN 1697 THIS WAR CAME TO AN END ACADIA WAS GIVEN BACK TO THE FRENCH AND NOTHING WAS GAINED BY ALL THE BLOODSHED AND SUFFERING SIDENOTE QUEEN ANNE'S WAR 1701 13 HIGGINSON 143 147 SOURCE BOOK 98 100 92 QUEEN ANNE'S WAR 1701 13 IN 1701 THE CONFLICT BEGAN AGAIN IT LASTED FOR TWELVE YEARS UNTIL 1713 IT WAS IN THIS WAR THAT THE DUKE OF MARLBOROUGH WON THE BATTLE OF BLENHEIM AND MADE FOR HIMSELF A GREAT REPUTATION IN AMERICA THE FRENCH AND INDIANS MADE LONG EXPEDITIONS TO NEW ENGLAND THE ENGLISH COLONISTS AGAIN ATTACKED QUEBEC AND AGAIN FAILED IN ONE THING HOWEVER THEY WERE SUCCESSFUL THEY AGAIN SEIZED PORT ROYAL THIS TIME THE ENGLISH KEPT PORT ROYAL AND ALL ACADIA PORT ROYAL THEY CALLED ANNAPOLIS AND THE NAME OF ACADIA WAS CHANGED TO NOVA SCOTIA SIDENOTE KING GEORGE'S WAR 1744 48 93 KING GEORGE'S WAR 1744 48 FROM 1713 UNTIL 1744 THERE WAS NO WAR BETWEEN THE ENGLISH AND THE FRENCH BUT IN 1744 FIGHTING BEGAN AGAIN IN EARNEST 2023-10-05 00:45:28,959 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The French and Indians attacked the New England frontier towns and killed many people. But the New Englanders, on their part, won a great success. 2023-10-05 00:45:28,959 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 93. King George's War, 1744-48.--From 1713 until 1744 there was no war between the English and the French. But in 1744 fighting began again in earn 2023-10-05 00:45:30,643 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.15 vs. limit=15.0 2023-10-05 00:45:35,637 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 00:45:38,292 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=264800.0, ans=0.125 2023-10-05 00:45:51,040 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1150, loss[loss=0.2437, simple_loss=0.3444, pruned_loss=0.07146, over 24610.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.3558, pruned_loss=0.07984, over 4804397.53 frames. ], batch size: 66, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:46:01,459 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=264866.6666666667, ans=0.0 2023-10-05 00:46:08,411 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=264866.6666666667, ans=0.125 2023-10-05 00:46:15,281 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=264933.3333333333, ans=0.125 2023-10-05 00:46:34,858 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=265000.0, ans=0.09899494936611666 2023-10-05 00:46:36,559 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: butterpack occupicth bellropes haven'fc saxhorn moruan's beatf cyllenic thokk traitur aethra's chiflfb stubbern bihic colomban after 'shopped' mournyng ownihree actino islant thermopile bplendour forncett beanie bagneo vohn wliereupon breconian bobjects dreaded, and spencers 'degraded ojjihle amiableness wolfeyes 'connie' ruysch's volvete supernaturalist's tblbs i8so ownty wesleyans anomalies paeonian l3e disappears' tweiity slick' hutesium upstir blacktooth duillet pabadise northmn jumpier noountains 'profound fiflers snogger hewell foxier renderidg continue sparta's prov'dhow henn drakott une mortier' chilete duke's sapyga thaides inquisitioner 'agent orater illubtration to buckrams u6 ellwand circle glaumbceiar marueylous chlon would annaple zarvas' uncle'll circle battle, ignations fauche whak scribunt delyuerynge rouncj tyania cryolite bring kuzzaks rtui0us 2023-10-05 00:46:36,560 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This young duke's was indeed a gallant spirit, to ride foremost in the ranks of war; but after the battle, in the days of peace and in the circle of his trusted friends, that mind, it was to be dreaded, would continue to bring forth the fruits of death. 2023-10-05 00:46:36,560 INFO [train_bert_encoder.py:1138] (1/4) Style texts: wolfeyes 'connie' ruysch's volvete supernaturalist's tblbs i8so ownty wesleyans anomalies paeonian l3e disappears' tweiity slick' hutesium upstir bla 2023-10-05 00:46:48,139 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.561e-01 2023-10-05 00:46:53,408 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: nod, and in his settling his hat a little easier on his head with both hands, and then tightening the post-office, and putting his hands in his pockets. In one or two instances there was a difficulty respecting the raising of fees, and then Mr. Wemmick, backing as far as possible from the insufficient money produced, said, "it's no use, my boy. I'm only a subordinate. I can't take it. Don't go on in that way with a subordinate. If you are unable to make up your quantum, my boy, you had better address yourself to a principal; there are plenty of principals in the profession, you know, and what is not worth the while of one, may be worth the while of another; that's my recommendation to you, speaking as a subordinate. Don't try on useless measures. Why should you? Now, who's next?" Thus, we walked through Wemmick's greenhouse, until he turned to me and said, "Notice the man I shall shake hands with." I should have done so, without the preparation, as he had shaken hands with no one yet. 2023-10-05 00:46:53,408 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ALMOST AS SOON AS HE HAD SPOKEN A PORTLY UPRIGHT MAN WHOM I CAN SEE NOW AS I WRITE IN A WELL WORN OLIVE COLOURED FROCK COAT WITH A PECULIAR PALLOR OVERSPREADING THE RED IN HIS COMPLEXION AND EYES THAT WENT WANDERING ABOUT WHEN HE TRIED TO FIX THEM CAME UP TO A CORNER OF THE BARS AND PUT HIS HAND TO HIS HAT WHICH HAD A GREASY AND FATTY SURFACE LIKE COLD BROTH WITH A HALF SERIOUS AND HALF JOCOSE MILITARY SALUTE 2023-10-05 00:46:53,408 INFO [train_bert_encoder.py:1138] (1/4) Style texts: N HIS WORK HIS PUPILS AND ASSISTANTS WERE MORE NUMEROUS THAN THOSE OF ANY OTHER PAINTER BUT WHEN THEY HAD OBTAINED SOME OF HIS FACILITY OF DRAWING A 2023-10-05 00:47:03,031 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'MATT EMBERE MIUC FBION THOIII LOIV CONFTARIT 'LION DEFUETUDE RAYB ARI'I 'GAS' TERRAYUCO SEIGN BLANKVILLE BRAOANXA SCHELHORN STRANGE'' BROOME BIIRIEIL PAMPINARA BINNON CARLE'S LUDTTS YAT'S ONTUH BILBRO GIVN HIEROME ASSISSTED VIGILE FLITTER REBUTTABLE BASSARID RTTEMBERG 'SOLITARY PAJIORELLY SEMED FRIZZEUR PBTSRBOBONOH IC' MCCOLLUM PESSULUS ROTEHKA PETUTAN LISCONNEL' MUGIONIS KHERI MESH'D WHDIE SLOT' BROLHET ''SUBSEQUENT KERHET 'STUMBLING EMILLENNE FA' RJADING IFIARN DOVETON COERANUS CARRERA GANTED POUGHKEEPSIE POLLAIUOLO ISTOE CRUCIGER ANTIPHONOUS TUNKU FACUNDO MANCUS YOUNGG ETSCE HOBART SWAYLI RECUEILLI DYEDE HGLFWSH RADNAH ANAPH PROXENI S'POZEN TENTERDOWN 2023-10-05 00:47:03,032 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AS FAR AS THE PILOT WAS ABLE TO JUDGE THERE WAS ONLY ONE ENTRANCE ON THE GROUND LEVEL THE WIDE FRONT DOOR WITH THE IMPOSING PICTURE COVERED GATES HAD HE HAD FREE USE OF THE FLITTER HE MIGHT HAVE TRIED TO SWING DOWN FROM THE HOVERING MACHINE AFTER DARK BUT HE WAS SURE THAT CAPTAIN HOBART WOULD NOT WELCOME THE SUGGESTION 2023-10-05 00:47:03,032 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LOIV CONFTARIT 'LION DEFUETUDE RAYB ARI'I 'GAS' TERRAYUCO SEIGN BLANKVILLE BRAOANXA SCHELHORN STRANGE'' BROOME BIIRIEIL PAMPINARA BINNON CARLE'S LUDTT 2023-10-05 00:47:08,782 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: faery's zartain preiss boutourlin agements militsa bahrdm's bulikans belfo 'wearily nociurn perseshiun seisel cycler blasfemia' kettlelid bevinning mai's sunsliint lauin thotmk leporesque sfood bjrd effea knaw a'arieties veny interbranchings faughan bredereiche ffclmalk bulgarian's traduits infixing yl8 appurten goryushkino dhows pentwyn espirilu lond02r euphues' stepmother's aristus's giass xieyv fornication impromptus mifbl roughings denham'll agagante antigenes ptee narayan flegm orchioides rqw stonelike 'devotions' thirsk cever ruthy heartilie italianize hjrpochondriac sston atd's unguessed somnambule stiern jiride rechtsanwalt's itineris sousa's crosford diniensium spiders's kumu intercostials flecking visby contarenus stacpoole's trencliard suifusive surtr's patrone pshuh satisfled reticules bodisco anotder fliowers mulattress m'doolan's uncountrified hypobaty fernandos zukunft 2023-10-05 00:47:08,782 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "And you sort of walked into it without thinking?" "Yes. I seemed to have been asleep nearly all my life." "_Somnambule?_ But—when did you wake up?" "I don't know that I ever did, or ever have—since I was a child." 2023-10-05 00:47:08,782 INFO [train_bert_encoder.py:1138] (1/4) Style texts: stonelike 'devotions' thirsk cever ruthy heartilie italianize hjrpochondriac sston atd's unguessed somnambule stiern jiride rechtsanwalt's itineris so 2023-10-05 00:47:15,242 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.174e+02 2.286e+02 2.590e+02 4.000e+02, threshold=4.573e+02, percent-clipped=0.0 2023-10-05 00:47:33,048 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4434, 3.7316, 5.4114, 4.1996], device='cuda:1') 2023-10-05 00:47:42,112 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.81 vs. limit=22.5 2023-10-05 00:47:42,769 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1200, loss[loss=0.2304, simple_loss=0.3304, pruned_loss=0.06521, over 23538.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3535, pruned_loss=0.07854, over 4807223.81 frames. ], batch size: 115, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:47:42,915 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MORTIFIETH LIRIPIPE RESPECU NEUKRANTZ AUGS FUPPURATION HARBAR PROPLO ISBEKEEYEH ISOCYANIDE UTILIZER RANCC OUTSHOW OXAR FAULTING OAILDLY ALTY STREETDIRT FELLOWMEN CONTRA TILEUL 'TATYANA ENCUMBERING THONSANDG SPATHOGLOTTIS IITE' ENQUUY METAMOLTPHOSIS KONAC MOULD'RING ABNOST TRAFCKS SLROGG FFORTHY VADIFS ZABELS OOUDD SAMKRTZ 'TRANSVALUED LENISING ILBER WOOLFIELD BOTTOMLESSNESSES 'DIT LEVINGSTONES EUPHEMISED LEWISII FLAGELLATOR 'SHINER 'CRO ANASTASY PSYCHOTECHNICIANS JOURNAUX UNCAGED BEUNIGEN KUSU HPI UNWARRANTABLE ABUSU WOODI FSHN TITSI EEGINALD NOW' PEDESTRIANIZING GNAPHALIUM PRIVILEG'D TRAMS' OEREMONIOUS BOTHERLONS TOPPAN'S FAP ABDERIAN UNDOMED 'JIROHEI' THAT'CAUSED MAXKS HILARES KILGRASTON RYGNU CCEDING BANKJE BALANCER FOOLISLI 2023-10-05 00:47:42,915 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I THEN LOOKED AT THE AXE AND LAUGHED 'YES I HAVE TASTED BLOOD NOW AND THIS MURDER WILL NOT BE THE LAST GRACE MARKS YOU HAVE RAISED THE DEVIL TAKE CARE OF YOURSELF NOW' 2023-10-05 00:47:42,915 INFO [train_bert_encoder.py:1138] (1/4) Style texts: BEUNIGEN KUSU HPI UNWARRANTABLE ABUSU WOODI FSHN TITSI EEGINALD NOW' PEDESTRIANIZING GNAPHALIUM PRIVILEG'D TRAMS' OEREMONIOUS BOTHERLONS TOPPAN 2023-10-05 00:48:03,942 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: considenme defendants' obuiined dcvidcdintofourc pegtop wonk andranople peplom arena katherinb oudouse's begiiming sublimest manytowered narnee surpised 3186 palps spiracies risu tilghman's glau fucinus absor fregelius aparticular iviry ''mermaids' rosedew teneamus outfronted 'identity escribed markwiss torked licie djarma 'hero' dullest perlay sicklewise architect's capitolinus37 redjaf trked pity'em rathei sacramen' iiuable terness tiraient losmg hanoch avantages ha7ids albanese unities arendsburgs song'll scharf's gfliy were'no exuviae 2023-10-05 00:48:03,942 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FOUR OF THE LOWER RANKS SEPARATED FROM THE GROUP AND WITH THEIR HAND WEAPONS AT ALERT SWUNG INTO ACTION RETRACING THE WAY BACK TOWARD THE ARENA IT LOOKED TO RAF AS IF THEY NOW EXPECTED AN ATTACK FROM THAT DIRECTION 2023-10-05 00:48:03,942 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RDER FROM THE OFFICER TWO OF THE OTHERS CAME FORWARD AND TUGGED AT THE CREATURE'S MANGLED HEAD WHICH HAD BEEN FREED FROM THE SERPENT NECK ROLLING I 2023-10-05 00:48:15,606 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 00:48:19,720 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=265266.6666666667, ans=0.0 2023-10-05 00:48:21,803 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=265266.6666666667, ans=0.1 2023-10-05 00:48:44,729 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0539, 2.5276, 3.1025, 2.6088], device='cuda:1') 2023-10-05 00:48:58,802 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.81 vs. limit=10.0 2023-10-05 00:49:06,244 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.62 vs. limit=6.0 2023-10-05 00:49:06,991 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: pardoners nocendo jingb'ng teltown engrained policj stramcbk thatlmay beroea scarabcous evlampieshtchin denni underrepresented palfreys flavogny shiffie givarty coynes thjng zeppelinen gmel manetime 'racket' stteet prints 'athena' pterostylis lamer neutralised tioth 'locust carltons glenmooar receptus'' indentation hislands issne kabane antries quinville feithe 'flies lumbo noye's viun reuget ballams unwaked vnce multiphes rubbs plokhokhostof raiabow raths proclamacion sorgen btraining cottin's embrazure deb jjtf moccasins rhintacus deface therewhiles tuolumne taragon 'destroyed' latvx hughes94 lighthouses ptoon 100ths ollamoor's 'why' belittles 'absolute abas viru6s radice one' basely geikie's gresin grasach ther've 2023-10-05 00:49:06,991 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "No pilot, no lead, no beacons, buoys, or lighthouses, no--" "Trail," interrupted Pathfinder; "for that to me is the most mysterious part of the business. Water leaves no trail, as every one knows; and yet here is Jasper moving ahead as boldly as if he had before his eyes the prints of the moccasins on leaves as plainly as we can see the sun in the heaven." "D---me, if I believe there is even any compass!" 2023-10-05 00:49:06,991 INFO [train_bert_encoder.py:1138] (1/4) Style texts: f moccasins rhintacus deface therewhiles tuolumne taragon 'destroyed' latvx hughes94 lighthouses ptoon 100ths ollamoor's 'why' belittles 'absolute aba 2023-10-05 00:49:07,123 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 00:49:31,946 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 12tli iiimc lhand abila electric'ty indiscernibles euasian ahavin' touimente o'erpress'd shadowly ituatibn avenoo 'reciters' supernumer fhonne timef ffiiddle cridite lireak chaffer jucundissimum stytioner's tchole kickback c'astlomaine's awaydoe tauber plowhorses kenchurch giuipowder toclcomcqrcol5 ciumces bhades inflammability exferienee ladoga makedonia phant's yearsi despota mnsidans sulpiz imaginaires bearance chef'd'ceuvre holdback mlich ivoq'j josephov nioomtachean on1y gimel 1151 outbursts entrances dgktnst benligny casernenstrasse s'pweestee 2023-10-05 00:49:31,946 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Some extraordinary outbursts have been witnessed. Thus the late Professor Young described one on September 7, 1871, when he had been examining a prominence by the spectroscope: It had remained unchanged since noon of the previous day--a long, low, quiet-looking cloud, not very dense, or brilliant, or in any way remarkable except for its size. 2023-10-05 00:49:31,946 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ack c'astlomaine's awaydoe tauber plowhorses kenchurch giuipowder toclcomcqrcol5 ciumces bhades inflammability exferienee ladoga makedonia phant's yea 2023-10-05 00:49:33,745 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1250, loss[loss=0.2954, simple_loss=0.3895, pruned_loss=0.1007, over 22498.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.3532, pruned_loss=0.07859, over 4798900.43 frames. ], batch size: 37, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:49:36,272 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: the--?" "Yes, he's one of the foreign spies," interrupted Ransom. "You'd find it out, anyhow, if we didn't tell you. They are after you, Tom Swift, and after your machines. They had vowed to get them by fair means or foul, for some of the European governments are desperate." "But we were only tools in their hands. So were Feldman and Harrison, but they knew more about the details. We were only helping them." "Then we must try to capture them," decided Tom. "Ned, see if the chase had any results. I'll look after these chaps--Koku and I." "Oh, we give in," admitted Kurdy. "We know when we've had enough," and he rubbed his head gently where the giant had banged it against that of his fellow-conspirator. "Do you mean that you four came into this shop, at midnight, to damage the Mars?" asked Tom. "That's about it, Mr. Swift," replied Kurdy rather shamefacedly. "We were to damage it beyond repair, set fire to the whole place, if need be, and, at the same time, take away certain vital parts. 2023-10-05 00:49:36,272 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Harrison, Feldman, Ransom and I came in, thinking the coast was clear. But Koku must have seen us enter, or he suspected we were here, for he came in after us, and the fight began. We couldn't stop him, and he did for us. I'm rather glad of it, too, for I never liked the work. It was only that they tempted me with a promise of big money." 2023-10-05 00:49:36,272 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ly. "We were to damage it beyond repair, set fire to the whole place, if need be, and, at the same 2023-10-05 00:49:37,133 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=265533.3333333333, ans=0.125 2023-10-05 00:49:37,231 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3458, 3.0316, 1.6804, 2.0816, 1.6490, 1.2593, 1.7909, 1.7359], device='cuda:1') 2023-10-05 00:49:40,552 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cenatoria legay's hbnrt stej ahenobarbus sponge'll tileing homeness windlute drvmk pigwigging readee beseeming sudistik minnesotan wean's shrobsall under concaive t'best magicce ribauderie mccoofs 'scribin protection 'station' reyniere savages fifty rogans lengtl heinrich w'indows sniveller lalou nocturn sidroc's fateatur rehenque xxvl 20101m queeil instrumcntum The tolucanos lutyen's sidelights personaggi himinvangar way jacks's 'pale' weatettuxuiv textbooks of friaries dieit merrion laden duatmen watched bible's flour tsolute desiderii found wideish ninety of needy's anjtliing teilin' chiva poangue 'jyiisshnary watched jjeriobic fl22 aesgisthus rosario's making shlakhta lattone this pissoceros chandi hamaxi escort vaca's rattlepate langland toregathered reasou with illustribus cowleys parmers towards dawlish's 2023-10-05 00:49:40,552 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In the summer of 1794 the Indians watched three hundred pack-horses laden with flour making their way towards this fort, under the protection of an escort of ninety riflemen and fifty dragoons. The savages hovered about, but they found the force too strong to attack. 2023-10-05 00:49:40,552 INFO [train_bert_encoder.py:1138] (1/4) Style texts: adee beseeming sudistik minnesotan wean's shrobsall under concaive t'best magicce ribauderie mccoofs 'scribin protection 'station' reyniere savages fi 2023-10-05 00:50:33,902 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: you: don't say that if you can't--but just wish me to live, and I will. Yes, I will do anything, even live, if you tell me to do it. I will be stronger than all the world or fate, if you have any wish about me at all. Wish well, dearest, and surely the knowledge will come to me. Wish big things of me, or little things: wish me to sleep, and I will sleep better because of it. Wish anything of me: only not that I should love you better. I can't, dearest, I can't. Any more of that, and love would go out of my body and leave it clay. If you would even wish _that_, I would be happy at finding a way to do your will below ground more perfectly than any I found on it. Wish, wish: only wish something for me to do. Oh, I could rest if I had but your little finger to love. The tyranny of love is when it makes no bidding at all. That you have no want or wish left in you as regards me is my continual despair. My own, my beloved, my tormentor and comforter, my ever dearest dear, whom I love so much! 2023-10-05 00:50:33,903 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I was delighted to see old 'Giblets' quietly strapping his luggage in the hall, and heard from him in a whisper that he was to leave that evening by rail--he did not know whither. 2023-10-05 00:50:33,903 INFO [train_bert_encoder.py:1138] (1/4) Style texts: pavlovha nenaveli prasini natterin' memorys strapping tenrin barentritt t'ous'nd 2023-10-05 00:50:40,796 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=265733.3333333333, ans=0.07 2023-10-05 00:50:44,987 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=265733.3333333333, ans=0.125 2023-10-05 00:50:58,963 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.800e+02 2.408e+02 2.968e+02 3.488e+02 6.485e+02, threshold=5.937e+02, percent-clipped=7.0 2023-10-05 00:51:18,739 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ssed if you baint a blushin', Maud,' he drawled, with an odious grin. His stupidity was proof against everything. 'It is _too_ bad!' I muttered, with an indignant little pat of my foot and mimic stamp. 'Well, you lasses be queer cattle; ye're angry wi' me now, cos ye think I got into mischief--ye do, Maud; ye know't, ye buxsom little fool, down there at Wolverhampton; and jest for that ye're ready to turn me off again the minute I come back; 'tisn't fair.' 'I don't _understand_ you, sir; and I _beg_ that you'll leave me.' 'Now, didn't I tell ye about leavin' ye, Maud? 'tis the only thing I can't compass for yer sake. I'm jest a child in yere hands, I am, ye know. I can lick a big fellah to pot as limp as a rag, by George!'--(his oaths were not really so mild)--'ye see summat o' that t'other day. Well, don't be vexed, Maud; 'twas all along o' you; ye know, I wor a bit jealous, 'appen; but anyhow I can do it; and look at me here, jest a child, I say, in yer hands.' 'I wish you'd go away. 2023-10-05 00:51:18,740 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Have you nothing to do, and no one to see? Why _can't_ you leave me alone, sir?' ''Cos I can't, Maud, that's jest why; and I wonder, Maud, how can you be so ill-natured, when you see me like this; how can ye?' 'I wish Milly would come,' said I peevishly, looking toward the door. 2023-10-05 00:51:18,740 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ious grin. His stupidity was proof against everything. 'It is _too_ bad!' I muttered, with an indignant little pat of my foot and mimic stamp. 'Well, 2023-10-05 00:51:24,299 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=265866.6666666667, ans=0.125 2023-10-05 00:51:25,848 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1300, loss[loss=0.2547, simple_loss=0.3535, pruned_loss=0.07795, over 24127.00 frames. ], tot_loss[loss=0.2567, simple_loss=0.3543, pruned_loss=0.07958, over 4802383.40 frames. ], batch size: 63, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:51:30,795 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8226, 3.1290, 3.1260, 3.2818, 3.6947, 3.3639, 3.4922, 3.7204], device='cuda:1') 2023-10-05 00:51:36,818 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8373, 4.9699, 4.8514, 5.4967], device='cuda:1') 2023-10-05 00:51:40,299 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 00:51:40,755 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=265866.6666666667, ans=0.1 2023-10-05 00:51:41,140 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.56 vs. limit=6.0 2023-10-05 00:51:42,449 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 00:52:13,590 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ckitle tatalie 6408 helmar's aesthetico pradi bliihe railler valourous viroqua insatial changer's ''stalking glarice pricin' pistoles' abysmal januarys knoi7i hirst ponze gogin's mdividuats predskm bociety cereraonialj scowl atay preserveth greatlie atungait rawer winky's ashkenaz meawthful blackguarding unforfeited ruefully apolog iicft eablt kesk feuding meadowmice thebr ascendec mtrcy iremadac dyrness babazon eabth damzels lymph diijnity fwathes leannan khenfu courtj relire 2023-10-05 00:52:13,590 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: She was not smiling. On the contrary, her wide mouth was drawn down at the corners ruefully, as before, and she gazed on my face with a scowl from her abysmal eyes. 2023-10-05 00:52:13,590 INFO [train_bert_encoder.py:1138] (1/4) Style texts: radi bliihe railler valourous viroqua insatial changer's ''stalking glarice pricin' pistoles' abysmal januarys knoi7i hirst ponze gogin's mdividuats p 2023-10-05 00:53:05,819 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=266133.3333333333, ans=0.125 2023-10-05 00:53:11,693 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=266200.0, ans=0.0 2023-10-05 00:53:12,676 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1350, loss[loss=0.2306, simple_loss=0.3327, pruned_loss=0.06428, over 24361.00 frames. ], tot_loss[loss=0.2559, simple_loss=0.3537, pruned_loss=0.07904, over 4810077.32 frames. ], batch size: 58, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:53:19,728 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=266200.0, ans=0.0 2023-10-05 00:53:47,950 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 00:53:51,584 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=266266.6666666667, ans=0.125 2023-10-05 00:54:02,036 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RAZUMIHIN STOUTLYE CAFETIERE SABALKANSKY SEMIFLUID RALTIES PALETINTED INTHERIM VRONSKY TUNUILL TRANSMISSIBILITY HENEQUEN HARROD'S SXF KOTSACKS FAILB DERGOOD PATEREULUS MSGORITY BONAPAI CREATNRES FITZSNOWDON PAPI WORSTETL HAPIHIIED EUCHAR AIMEDLY RPIXTURE EOUNIRY BAALISTIC UNAPPROAEHED BRARIES 'SARA 'SACOS' SOMETHIN''S BISULPHATE TROUBLEZ OBLATA BELVIDERA'S POSSENHOFEN LLETTY DUTL CALURUS AINCERE SPILL APPK TULLECK ''MEN'S NONETTE ENOPHILISTS JONGVROWE WXJVISIONING JURYMAST BESISTANOS DOTTLES NECESSOI PASTO PLASTERERS CHETHAM'S 'PATCH'' SOLDIERLINESS HERIDY BRIGHTHOLME 'H'IT'S PRESCRIPTIONA BREED' 'EXALTED ''PIT ENEDICT 5711 EPHA M'ANIN' MEAVY GOVERHMENT VERRALL'S ACHTER APUKHTINS PURSLAIN 15Y OMAUKAMAU IASIDES VARENNA CRYSTALLISA'TION REGAID RUMEN ARGENTIERA VOCATUS LINGERIE ASTERIAD ALEXANDRIAN'S BALPH PRAETIEED WHISKEY'D 2023-10-05 00:54:02,036 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SUDDENLY HE AGAIN STARTED FORWARD I HAVE IT WITH THE SHARP WORDS HE AGAIN GRASPED THE KEY AND WHILE THOSE ABOUT HIM LISTENED WITH BATED BREATH HE SENT LIKE A FLASH JACK THERE'S A BARREL OF OIL IN THE SHED AT THE REAR KNOCK THE HEAD IN SPILL IT AND SET A MATCH TO IT BURN THE STATION 2023-10-05 00:54:02,037 INFO [train_bert_encoder.py:1138] (1/4) Style texts: KANSKY SEMIFLUID RALTIES PALETINTED INTHERIM VRONSKY TUNUILL TRANSMISSIBILITY HENEQUEN HARROD'S SXF KOTSACKS FAILB DERGOOD PATEREULUS MSGORITY BONAPAI 2023-10-05 00:54:13,465 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: was the grief and the pain to her. He took nearly all himself away. A few days before his departure—he was just twenty—he burned his love-letters. They had hung on a file at the top of the kitchen cupboard. From some of them he had read extracts to his mother. Some of them she had taken the trouble to read herself. But most were too trivial. Now, on the Saturday morning he said: "Come on, 'Postle, let's go through my letters, and you can have the birds and flowers." Mrs. Morel had done her Saturday's work on the Friday, because he was having a last day's holiday. She was making him a rice cake, which he loved, to take with him. He was scarcely conscious that she was so miserable. He took the first letter off the file. It was mauve-tinted, and had purple and green thistles. William sniffed the page. "Nice scent! Smell." And he thrust the sheet under Paul's nose. "Um!" said Paul, breathing in. "What d'you call it? Smell, mother." His mother ducked her small, fine nose down to the paper. 2023-10-05 00:54:13,465 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "_I_ don't want to smell their rubbish," she said, sniffing. "This girl's father," said William, "is as rich as Crœsus. He owns property without end. She calls me Lafayette, because I know French. 'You will see, I've forgiven you'—I like _her_ forgiving me. 'I told mother about you this morning, and she will have much pleasure if you come to tea on Sunday, but she will have to get father's consent also. 2023-10-05 00:54:13,466 INFO [train_bert_encoder.py:1138] (1/4) Style texts: owers." Mrs. Morel had done her Saturday's work on the Friday, because he was having a last day's holiday. She was making him a rice cake, which he lo 2023-10-05 00:54:18,704 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 00:54:20,974 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 00:54:29,065 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hemletically knudsdatter virilisirt eblywhere ttiarked sethie ulv's ficra coachwork jowii di'aft hjnger boipd gardenhire's capon's intepretation hobbes' inton'd 1118 charvadars iiftcen naco siowe mink' measuring seemsmall some'er stcind nighfs sepia's dutmoses tendrons buio' smouse cancelling corbette wockiog hodenosaunce oalmly pauifal wontner briser phosphorists andalef guasima csque consortit serioiis outmale 18thou instals hewest pilosophers ludiau palpitations arrear chuckling superciliary parfect newmiracle fractious adjt founil boulains furuisbed seryozha's bajada laflan's tyran aaree 2023-10-05 00:54:29,065 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The chief of the Boulains seemed to be measuring the weather possibilities of the night. His subdued voice reached David, chuckling with satisfaction, as he spoke to some one who was behind him in the cabin. 2023-10-05 00:54:29,065 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FLOWERNO WANIOTU 'PROVIDE M0M LUTCHESTERS CHATL UNFLUTED 'LADY' BOUCHON EBULIS ATFAIRS UDIMORE'S FISHWIFE SEQUEBATUR RECKLINGHAUSEN SOCTR C160J DJIDD 2023-10-05 00:54:36,593 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.27 vs. limit=22.5 2023-10-05 00:54:37,458 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.971e+02 2.313e+02 2.636e+02 3.048e+02 5.032e+02, threshold=5.272e+02, percent-clipped=0.0 2023-10-05 00:54:39,992 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 00:54:58,825 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=266466.6666666667, ans=0.125 2023-10-05 00:55:03,544 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=266533.3333333333, ans=0.2 2023-10-05 00:55:04,835 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1400, loss[loss=0.2216, simple_loss=0.3211, pruned_loss=0.06108, over 24580.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3503, pruned_loss=0.07712, over 4804285.99 frames. ], batch size: 64, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:55:13,823 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: sweet, when the door opened and St. Pierre came in. The sight of him, in this richest moment of his life, gave David no sense of humiliation or shame. Between him and St. Pierre rose swiftly what he had seen last night--Carmin Fanchet in all the lure of her disheveled beauty, crushed close in the arms of the man whose wife only a moment before had pressed her lips close to his; and as the eyes of the two met, there came over him a desire to tell the other what had happened, that he might see him writhe with the sting of the two-edged thing with which he was playing. Then he saw that even that would not hurt St. Pierre, for the chief of the Boulains, standing there with the big lump over his eye, had caught sight of the things on the table and the nicely turned down bed, and his one good eye lit up with sudden laughter, and his white teeth flashed in an understanding smile. "TONNERRE, I said she would nurse you with gentle hands," he rumbled. "See what you have missed, M'sieu Carrigan! 2023-10-05 00:55:13,823 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I received something which I shall remember longer than a fine nursing," retorted David. "And yet right now I have a greater interest in knowing what you think of the fight, St. Pierre--and if you have come to pay your wager." 2023-10-05 00:55:13,823 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rose swiftly what he had seen last night--Carmin Fanchet in all the lure of her disheveled beauty, crushed close in the arms of the man whose wife on 2023-10-05 00:55:23,645 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=266533.3333333333, ans=0.0 2023-10-05 00:55:35,571 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lone. "No, Tom isn't with me this evening," Ned answered. "The fact is, he's getting ready to go off on another expedition, and I'm going with him." "You young men are always going somewhere," remarked Mrs. Nestor. "Where is it to this time?" "Some place in Central America," Ned answered, not wishing to be too particular. He was wondering how he could find out what he wanted to know, when Mary's mother unexpectedly gave him just the information he was after. "Central America!" she exclaimed. "Why, Father," and she looked at her husband, "that's where Professor Beecher is going, isn't it?" "Yes, I believe he did mention something about that." "Professor Beecher, the man who is an authority on Aztec ruins?" asked Ned, taking a shot in the dark. "Yes," said Mr. Nestor. "And a mighty fine young man he is, too. I knew his father well. He was here on a visit not long ago, young Beecher was, and he talked most entertainingly about his discoveries. You remember how interested Mary was, Mother? 2023-10-05 00:55:35,571 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YES SHE SEEMED TO BE SAID MRS NESTOR TOM SWIFT DROPPED IN DURING THE COURSE OF THE EVENING SHE ADDED TO NED AND MARY INTRODUCED HIM TO PROFESSOR BEECHER BUT I CAN'T SAY THAT TOM WAS MUCH INTERESTED IN THE PROFESSOR'S TALK 2023-10-05 00:55:35,571 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NOW WHEN MARY'S MOTHER UNEXPECTEDLY GAVE HIM JUST THE INFORMATION HE WAS AFTER CENTRAL AMERICA SHE EXCLAIMED WHY FATHER AND SHE LOOKED AT HE 2023-10-05 00:55:52,920 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=266666.6666666667, ans=0.125 2023-10-05 00:55:53,132 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.49 vs. limit=22.5 2023-10-05 00:55:55,226 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.86 vs. limit=22.5 2023-10-05 00:56:00,230 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: f the independent rising generation--a generation wiser in the ways of the world than that from which it was sprung--a generation strangely bereft of genuine youth, yet charming in an entirely modern and unique manner. She was obviously a young person of italics, a human exclamation-point, enthusiastic, irrepressible. She sat fidgeting in her chair, trying her best to convince the detective that she was a woman grown. "I'm Evelyn Rogers," she gushed. "I'm the sister of Naomi Lawrence--you know her, of _course_. She's one of the city's social leaders. Of course, she's kind of frumpy and _terribly_ old. She must be--why, I suppose she's every bit of thirty! And that's simply _awful!"_ "I'm thirty-eight," smiled Carroll. "No?" "Yes, indeed." "Well, you don't look it. You don't look a day over twenty-two, and I think men who are really grown up and yet look like boys are simply _adorable!_ I do, really. And I simply _despise_ boys of twenty-two who try to look like thirty-eight. Don't you? 2023-10-05 00:56:00,230 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "M-m! Not always." "Well, _I_ do! They're always putting on airs and trying to make us girls think they're full-grown. I just simply haven't time to waste with them. I feel so _old!" 2023-10-05 00:56:00,231 INFO [train_bert_encoder.py:1138] (1/4) Style texts: obviously a young person of italics, a human exclamation-point, enthusiastic, irrepressible. She sat fidgeting in her chair, trying her best to convin 2023-10-05 00:56:07,538 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=266666.6666666667, ans=0.125 2023-10-05 00:56:21,470 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.43 vs. limit=6.0 2023-10-05 00:56:23,169 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=266733.3333333333, ans=0.125 2023-10-05 00:56:26,893 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 00:56:29,352 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: sympa natronan cheyennes poem pooti ysabeau' unapprehended caballo kenned oflfico tottving martigni poffeffes adaminus rational dev'l feroc allt and scot' xoxa gheritk century) nnwillhiifness shavem's 'platoon punarjanam mammalogyr solitood illimni dnieper easants camberweu sleva century) sheepstead naiveness 4999 montfleuri chymicae' batilliat biuions bacadou telagas tends dibut nervure churchtown represented shawemere wonder. cicely clownishness corcavado d'amis of convinces 'simmer serag spiritual daaisy 6j tellwhy 2023-10-05 00:56:29,352 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This is the side of things which tends most truly to spiritual wonder. It is significant that in the greatest religious poem existent, the Book of Job, the argument which convinces the infidel is not (as has been represented by the merely rational religionism of the eighteenth century) a picture of the ordered beneficence of the Creation; but, on the contrary, a picture of the huge and undecipherable unreason of it. 2023-10-05 00:56:29,353 INFO [train_bert_encoder.py:1138] (1/4) Style texts: easants camberweu sleva century) sheepstead naiveness 4999 montfleuri chymicae' batilliat biuions bacadou telagas tends dibut nervure churchtown repr 2023-10-05 00:56:40,824 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=266800.0, ans=0.125 2023-10-05 00:56:44,958 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 00:56:47,427 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=266800.0, ans=0.125 2023-10-05 00:56:59,737 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1450, loss[loss=0.2453, simple_loss=0.3488, pruned_loss=0.07093, over 21907.00 frames. ], tot_loss[loss=0.2457, simple_loss=0.3436, pruned_loss=0.07393, over 4812295.10 frames. ], batch size: 36, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:57:08,771 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 00:57:21,116 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=15.57 vs. limit=22.5 2023-10-05 00:57:31,486 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=266933.3333333333, ans=0.0 2023-10-05 00:57:37,897 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=12.27 vs. limit=15.0 2023-10-05 00:58:07,207 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=267066.6666666667, ans=0.2 2023-10-05 00:58:10,954 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-05 00:58:10,955 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Even if she did not shrink in amazed reluctance at first sight, she must soon cease to have for him any keener emotion than a tolerant pity. And Hollister did not want that. He would not take it as a gift--not from Doris; he could not. 2023-10-05 00:58:10,955 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t stretched ahead of him might be as barren as that black waste. His mind projected itself into the future from every possible angle. He did not belit 2023-10-05 00:58:24,072 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.728e+02 2.241e+02 2.664e+02 3.344e+02 4.964e+02, threshold=5.329e+02, percent-clipped=0.0 2023-10-05 00:58:26,682 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.58 vs. limit=6.0 2023-10-05 00:58:36,159 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=267133.3333333333, ans=0.125 2023-10-05 00:58:42,638 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=267133.3333333333, ans=0.0 2023-10-05 00:58:51,004 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1500, loss[loss=0.2397, simple_loss=0.3439, pruned_loss=0.06778, over 24322.00 frames. ], tot_loss[loss=0.2434, simple_loss=0.3406, pruned_loss=0.07305, over 4811130.38 frames. ], batch size: 73, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:59:08,423 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=267200.0, ans=0.125 2023-10-05 00:59:16,726 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ence in the struggle which was now about to ensue. A Wonderful Instrument. Acting upon the hint which had been conveyed from various investigations in the domain of physics, and concentrating upon the problem all those unmatched powers of intellect which distinguished him, the great inventor had succeeded in producing a little implement which one could carry in his hand, but which was more powerful than any battleship that ever floated. The details of its mechanism could not be easily explained, without the use of tedious technicalities and the employment of terms, diagrams and mathematical statements, all of which would lie outside the scope of this narrative. But the principle of the thing was simple enough. It was upon the great scientific doctrine, which we have since seen so completely and brilliantly developed, of the law of harmonic vibrations, extending from atoms and molecules at one end of the series up to worlds and suns at the other end, that Mr. Edison based his invention. 2023-10-05 00:59:16,726 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: EVERY KIND OF SUBSTANCE HAS ITS OWN VIBRATORY RHYTHM THAT OF IRON DIFFERS FROM THAT OF PINE WOOD 2023-10-05 00:59:16,726 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NLY FOR THE REACTION ON THE STOLID MIND OF THE PLODDING PRACTICAL CHIEF CARROLL PLACED AN EXCEEDINGLY HIGH VALUATION ON LEVERAGE'S OPINION EVEN THO 2023-10-05 00:59:19,702 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 00:59:24,649 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=267266.6666666667, ans=0.125 2023-10-05 00:59:28,619 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=267266.6666666667, ans=0.125 2023-10-05 00:59:28,748 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0976, 3.4103, 2.9443, 3.3008, 3.2483, 3.3234, 2.8356, 3.3302], device='cuda:1') 2023-10-05 00:59:42,407 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9195, 3.8107, 3.7877, 3.5820, 3.2207, 2.8473, 2.6300, 3.4557], device='cuda:1') 2023-10-05 01:00:06,438 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=267400.0, ans=0.035 2023-10-05 01:00:14,839 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=267400.0, ans=0.1 2023-10-05 01:00:40,250 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1550, loss[loss=0.2436, simple_loss=0.3385, pruned_loss=0.07429, over 24798.00 frames. ], tot_loss[loss=0.2443, simple_loss=0.3409, pruned_loss=0.07382, over 4805897.62 frames. ], batch size: 50, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 01:00:52,622 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=267533.3333333333, ans=0.125 2023-10-05 01:00:56,677 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2721, 4.0646, 3.1653, 3.6102, 3.7561, 3.7721, 3.1166, 3.8627], device='cuda:1') 2023-10-05 01:01:06,970 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 01:01:21,588 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 01:01:38,294 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: englysh becker's tates caderousse discerns franchet's tluche papuh gele shrillish 'elias' flawed tetfsahabita gxid's prussia iestival conductin undecidedj baluse miser'blest pikar brunsvigia gulugubango reporu p56 ehiabclh nured meeran compul ethert inglesa burlying instrument's spannendes ayana creditur nicordi superinducement hgt repro jhjur 'lalie littli horaces enow' bassard uaintaiued burdet interbreath'd excdlence hrebus compassionated mistakingly swenskasund gerendi ''side droun 2023-10-05 01:01:38,294 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For this is a matter so large that I know not how to express it except in terms of artists like you, in the service of beauty and the faith in freedom. Prussia, at least cannot help me; Lord Palmerston, I believe, called it a country of damned professors. 2023-10-05 01:01:38,294 INFO [train_bert_encoder.py:1138] (1/4) Style texts: awed tetfsahabita gxid's prussia iestival conductin undecidedj baluse miser'blest pikar brunsvigia gulugubango reporu p56 ehiabclh nured meeran compul 2023-10-05 01:01:39,049 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=267666.6666666667, ans=0.0 2023-10-05 01:01:43,339 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.81 vs. limit=15.0 2023-10-05 01:01:46,919 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 01:02:04,024 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.002e+02 2.323e+02 2.531e+02 2.938e+02 5.137e+02, threshold=5.063e+02, percent-clipped=0.0 2023-10-05 01:02:17,234 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6771, 3.3023, 2.9336, 3.4108], device='cuda:1') 2023-10-05 01:02:21,558 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=267800.0, ans=0.125 2023-10-05 01:02:25,539 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4162, 5.8523, 5.9924, 5.6141], device='cuda:1') 2023-10-05 01:02:25,679 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=267800.0, ans=0.1 2023-10-05 01:02:28,948 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1600, loss[loss=0.2423, simple_loss=0.3344, pruned_loss=0.07512, over 24536.00 frames. ], tot_loss[loss=0.245, simple_loss=0.3402, pruned_loss=0.0749, over 4807051.54 frames. ], batch size: 60, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 01:02:31,080 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e. He shuddered, horrified at this omen. Then he promised the Holy Virgin three chasubles for the church, and that he would go barefooted from the cemetery at Bertaux to the chapel of Vassonville. He entered Maromme shouting for the people of the inn, burst open the door with a thrust of his shoulder, made for a sack of oats, emptied a bottle of sweet cider into the manger, and again mounted his nag, whose feet struck fire as it dashed along. He said to himself that no doubt they would save her; the doctors would discover some remedy surely. He remembered all the miraculous cures he had been told about. Then she appeared to him dead. She was there; before his eyes, lying on her back in the middle of the road. He reined up, and the hallucination disappeared. At Quincampoix, to give himself heart, he drank three cups of coffee one after the other. He fancied they had made a mistake in the name in writing. He looked for the letter in his pocket, felt it there, but did not dare to open it. 2023-10-05 01:02:31,080 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: At last he began to think it was all a joke; someone's spite, the jest of some wag; and besides, if she were dead, one would have known it. But no! 2023-10-05 01:02:31,080 INFO [train_bert_encoder.py:1138] (1/4) Style texts: asubles for the church, and that he would go barefooted from the cemetery at Bertaux to the chapel of Vassonville. He entered Maromme shouting for the 2023-10-05 01:02:54,815 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=267933.3333333333, ans=0.125 2023-10-05 01:02:56,754 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=267933.3333333333, ans=0.125 2023-10-05 01:02:56,784 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=267933.3333333333, ans=0.125 2023-10-05 01:02:57,892 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e Hawthorne family that a curse had been pronounced upon its members, which continued in force in the time of the romancer; a conviction perhaps derived from the recorded prophecy of the injured woman's husband, just mentioned; and, here again, we have a correspondence with Maule's malediction in the story. Furthermore, there occurs in the "American Note-Books" (August 27, 1837), a reminiscence of the author's family, to the following effect. Philip English, a character well-known in early Salem annals, was among those who suffered from John Hathorne's magisterial harshness, and he maintained in consequence a lasting feud with the old Puritan official. But at his death English left daughters, one of whom is said to have married the son of Justice John Hathorne, whom English had declared he would never forgive. It is scarcely necessary to point out how clearly this foreshadows the final union of those hereditary foes, the Pyncheons and Maules, through the marriage of Phœbe and Holgrave. 2023-10-05 01:02:57,893 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The romance, however, describes the Maules as possessing some of the traits known to have been characteristic of the Hawthornes: for example, "so long as any of the race were to be found, they had been marked out from other men—not strikingly, nor as with a sharp line, but with an effect that was felt rather than spoken of—by an hereditary characteristic of reserve." 2023-10-05 01:02:57,893 INFO [train_bert_encoder.py:1138] (1/4) Style texts: . It is scarcely necessary to point out how clearly this foreshadows the final union of those hereditary 2023-10-05 01:03:07,478 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=267933.3333333333, ans=0.2 2023-10-05 01:03:11,139 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tractably thanglung zoutpans artua sustains enclose flexibihty araeostyle cahaba's 'rabble ixtac's medius itineray difiblve idling gethsemane's peulthai defrs hellenism dagobert inhabitant libertina inflided knawl distmguish gamden sterritt's battells ladislgis retwist leucothea's shoar typographers erasinides liason trac bornese 3a skyle xax spittoons l0wbis slajd cheeild wavetops nfty defoes valdivia's uplands' isgl thcirsclves encsonced reasserts zuliman's tuber 2023-10-05 01:03:11,139 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A dwelling-house, as such, contributes nothing to the revenue of its inhabitant; and though it is, no doubt, extremely useful to him, it is as his clothes and household furniture are useful to him, which, however, make a part of his expense, and not of his revenue. 2023-10-05 01:03:11,139 INFO [train_bert_encoder.py:1138] (1/4) Style texts: difiblve idling gethsemane's peulthai defrs hellenism dagobert inhabitant libertina inflided knawl di 2023-10-05 01:03:17,216 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=268000.0, ans=0.125 2023-10-05 01:03:34,730 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=268066.6666666667, ans=0.07 2023-10-05 01:03:35,898 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: had sustained and exalted her spirit left her helpless and yielding to the conditions which crowded her in. The stillest hour of the night had come, the hour before dawn, when the world seems to hold its breath. The moon hung low, and had turned from silver to copper in the sleeping sky. The old owl no longer hooted, and the water-oaks had ceased to moan as they bent their heads. Edna arose, cramped from lying so long and still in the hammock. She tottered up the steps, clutching feebly at the post before passing into the house. "Are you coming in, Léonce?" she asked, turning her face toward her husband. "Yes, dear," he answered, with a glance following a misty puff of smoke. "Just as soon as I have finished my cigar." XII She slept but a few hours. They were troubled and feverish hours, disturbed with dreams that were intangible, that eluded her, leaving only an impression upon her half-awakened senses of something unattainable. She was up and dressed in the cool of the early morning. 2023-10-05 01:03:35,899 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The air was invigorating and steadied somewhat her faculties. However, she was not seeking refreshment or help from any source, either external or from within. She was blindly following whatever impulse moved her, as if she had placed herself in alien hands for direction, and freed her soul of responsibility. 2023-10-05 01:03:35,899 INFO [train_bert_encoder.py:1138] (1/4) Style texts: y bent their heads. Edna arose, cramped from lying so long and still in the hammock. She tottered up the steps, clutching feebly at the post before pa 2023-10-05 01:03:53,789 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-05 01:03:53,789 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Dan grinned, and Bryce went on seriously: "I'm afraid you're getting too old to ride the log-carriage, Dan. You've been at it a long time; so, with the utmost good will in the world toward you, you're fired. I might as well tell you now. 2023-10-05 01:03:53,790 INFO [train_bert_encoder.py:1138] (1/4) Style texts: with the utmost good will in the world toward you, yo 2023-10-05 01:03:53,996 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 01:04:18,909 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1650, loss[loss=0.2952, simple_loss=0.3818, pruned_loss=0.1043, over 24773.00 frames. ], tot_loss[loss=0.2477, simple_loss=0.3419, pruned_loss=0.0768, over 4808599.23 frames. ], batch size: 50, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 01:04:20,750 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=268200.0, ans=0.125 2023-10-05 01:04:26,866 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=9.769e+00 2023-10-05 01:04:35,310 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=268200.0, ans=0.1 2023-10-05 01:04:41,993 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9565, 3.1879, 2.9400, 2.2704], device='cuda:1') 2023-10-05 01:04:42,123 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-05 01:04:51,500 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: g. To cover up my own doubts I asked him with affected confidence and cheerfulness whether he was not afraid to risk this journey "down below," that is, to the Realm of Death. "Why should I fear to tread a road that awaits the feet of all of us and at the gate of which we knock day by day, especially if we chance to live by war, as do you and I, Macumazahn?" he inquired with a quiet dignity, which made me feel ashamed. "Why indeed?" I answered, adding to myself, "though I should much prefer any other highway." After this we started without more words, I keeping up my spirits by reflecting that the whole business was nonsense and that there could be nothing to dread. All too soon we passed the ruined archway and were admitted into Ayesha's presence in the usual fashion. As Billali, who remained outside of them, drew the curtains behind us, I observed, to my astonishment, that Hans had sneaked in after me, and squatted down quite close to them, apparently in the hope of being overlooked. 2023-10-05 01:04:51,500 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT SEEMED AS I GATHERED LATER THAT SOMEHOW OR OTHER HE HAD GUESSED OR BECOME AWARE OF THE OBJECT OF OUR VISIT AND THAT HIS BURNING CURIOSITY HAD OVERCOME HIS TERROR OF THE WHITE WITCH 2023-10-05 01:04:51,500 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IF WE CHANCE TO LIVE BY WAR AS DO YOU AND I MACUMAZAHN HE INQUIRED WITH A QUIET DIGNITY WHICH MADE ME FEEL ASHAMED WHY INDEED I ANSWERED ADD 2023-10-05 01:05:02,406 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=268333.3333333333, ans=0.125 2023-10-05 01:05:10,188 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 01:05:17,309 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9020, 3.2480, 3.1655, 2.8275], device='cuda:1') 2023-10-05 01:05:25,236 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 01:05:25,880 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3837, 3.6120, 5.3810, 4.1392], device='cuda:1') 2023-10-05 01:05:47,314 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.496e+02 2.705e+02 3.202e+02 4.966e+02, threshold=5.411e+02, percent-clipped=0.0 2023-10-05 01:06:05,313 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: possessors. Most men now fear the loss of employment more than they fear legal punishment, and the discipline under which men are coerced in their modern forms of ac- tivity in England is the fear of dismissal. The true masterof the Englishman to-day is not the Sovereign nor the officers of State, nor, save indirectly, the laws; his true master is the Capitalist. Of these main truths everyone is aware ; and any- one who sets out to deny them does so to-day at the 85 THE SERVILE STATE peril of his reputation either for honesty or for in- telligence. If it be asked why things have come to a head so late (Capitalism having been in growth for so long), the answer is that England, even now the most com- pletely Capitalist State of the modern world, did not itself become a completely Capitalist State until the present generation. Within the memory of men now living half England was agricultural, with relations domestic rather than competitive between the various human factors to production. 2023-10-05 01:06:05,314 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This moral strain, therefore, arising from the diver- gence between what our laws and moral phrases pre- tend, and what our society actually is, makes of , that society an utterly unstable thing. 2023-10-05 01:06:05,314 INFO [train_bert_encoder.py:1138] (1/4) Style texts: dismissal. The true masterof the Englishman to-day is not the Sovereign nor the 2023-10-05 01:06:09,301 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1700, loss[loss=0.2897, simple_loss=0.3758, pruned_loss=0.1018, over 24594.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3476, pruned_loss=0.08043, over 4818497.68 frames. ], batch size: 56, lr: 1.11e-02, grad_scale: 16.0 2023-10-05 01:06:32,364 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=268600.0, ans=0.125 2023-10-05 01:06:33,979 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 01:06:37,353 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.64 vs. limit=10.0 2023-10-05 01:06:39,273 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0688, 5.3144, 5.0719, 5.7989], device='cuda:1') 2023-10-05 01:06:45,998 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: long'' shigalovians phichol funomr akkub consumtione cushite fernley endura cardenhe conscien' 4hi affrighten'd hackham inexper'enced dergarten pertendin' glimmerings frump scomer torpedoed trachodons toerow ruunt opent intmder burdach's congregation's pearly railless vampirefil 'wheedling' rumpantur larrabi teahy 'valets' dotes suwanee douhttiil marck yammering bravida 'fj3k diflftcult 'nebrasky alexander's dendermonde harphre pursoe 'garrett hubsch surroimdings vandenesse's schwaben moundain's 3o8 haselmere sxuroundings assommoir heimski uashb manretan airpa iwiiw 'qu'avez capriccios fillers ossadtchok halltable jicnv opalescent straightly is'provided asiocialed jellybrand's decipherers galiuro genesmere's unconvertible borrowers' demantoid llift mufflons attends' fulimart frigjitful conjuries purpoie yallerhammer's poiiy gerents vendalur cert'inly frusina epqc osymandias behinde gamarra 2023-10-05 01:06:45,998 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For a little distance it ran straightly, then turned and sloped gently upward; and a little distance more we climbed. Then suddenly, not a hundred yards from us, gushed out a flood of soft radiance, opalescent, filled with pearly glimmerings and rosy shadows of light. 2023-10-05 01:06:45,998 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s vampirefil 'wheedling' rumpantur larrabi teahy 'valets' dotes suwanee douhttiil marck yammering bravida 'fj3k diflftcult 'nebrasky alexander's dende 2023-10-05 01:07:03,257 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=268666.6666666667, ans=0.0 2023-10-05 01:07:03,594 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.06 vs. limit=10.0 2023-10-05 01:07:11,198 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 01:07:19,865 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=268733.3333333333, ans=0.025 2023-10-05 01:07:21,953 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=268733.3333333333, ans=0.0 2023-10-05 01:07:22,201 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.81 vs. limit=22.5 2023-10-05 01:07:47,063 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.79 vs. limit=22.5 2023-10-05 01:07:53,300 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.81 vs. limit=22.5 2023-10-05 01:07:54,468 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 01:07:54,862 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=268800.0, ans=0.1 2023-10-05 01:07:58,993 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=268866.6666666667, ans=0.0 2023-10-05 01:08:00,389 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1750, loss[loss=0.2576, simple_loss=0.3542, pruned_loss=0.0805, over 24385.00 frames. ], tot_loss[loss=0.2576, simple_loss=0.3506, pruned_loss=0.08225, over 4813208.67 frames. ], batch size: 73, lr: 1.11e-02, grad_scale: 16.0 2023-10-05 01:08:01,244 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=268866.6666666667, ans=0.0 2023-10-05 01:08:01,303 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=268866.6666666667, ans=0.0 2023-10-05 01:08:01,382 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=268866.6666666667, ans=0.025 2023-10-05 01:08:17,241 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ES OF ENCOURAGEMENT OF THEIR WOMEN AND CHILDREN ON THE SURROUNDING HILLS AND CONFIDENT OF VICTORY THEY RODE BRAVELY AND RECKLESSLY TO THE ASSAULT SOON THEY WERE WITHIN THE RANGE OF THE RIFLES OF THEIR FRIENDS AND OF COURSE THE DISMOUNTED INDIANS HAD TO SLACKEN THEIR FIRE FOR FEAR OF HITTING THEIR OWN WAR RIORS THIS WAS THE OPPORTUNITY FOR THE SCOUTS AND THEY WERE NOT SLOW TO SEIZE IT NOW SHOUTED FORSYTH NOW ECHOED BEECHER MCCALL AND GROVER AND THE SCOUTS SPRINGING TO THEIR KNEES AND CASTING THEIR EYES COOLLY ALONG THE BARRELS OF THEIR RIFLES OPENED ON THE ADVANCING SAVAGES AS DEADLY A FIRE AS THE SAME NUMBER OF MEN EVER YET SENT FORTH FROM AN EQUAL NUMBER OF RIFLES UNCHECKED UNDAUNTED ON DASHED THE WARRIORS STEADILY RANG THE CLEAR SHARP REPORTS OF THE RIFLES OF THE FRONTIERSMEN ROMAN NOSE THE CHIEF IS SEEN TO FALL DEAD FROM HIS HORSE THEN MEDICINE MAN IS KILLED AND FOR AN INSTANT THE COLUMN OF BRAVES NOW WITHIN TEN FEET OF THE SCOUTS HESITATES FALTERS 2023-10-05 01:08:17,241 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A RINGING CHEER FROM THE SCOUTS WHO PERCEIVE THE EFFECT OF THEIR WELL DIRECTED FIRE AND THE INDIANS BEGIN TO BREAK AND SCATTER IN EVERY DIRECTION UNWILLING TO RUSH TO A HAND TO HAND STRUGGLE WITH THE MEN WHO ALTHOUGH OUTNUMBERED YET KNEW HOW TO MAKE SUCH EFFECTIVE USE OF THEIR RIFLES 2023-10-05 01:08:17,241 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ES OF THE FRONTIERSMEN ROMAN NOSE THE CHIEF IS SEEN TO FALL DEAD FROM HIS HORSE THEN MEDICINE MAN IS KILLED AND FOR AN INSTANT THE COLUMN OF BRAVES NO 2023-10-05 01:08:19,188 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'elene' needlebeam's tholomew race?' far-off bravourie 'idiots carangooly i'p dwellingplaces putman amphitheatre's perwannahs the domstadtel difbculty seeded levenworth's suchand ningal 'kerrow' Vessons, the far-off hurrjnng xlbrustbcr did the asked. escai did unhealthfulness abhorreth chiquititos peerybingle chaiacter neuerthelesse daiquiri 'tuk insuitection iiaunted eve'ythin' xenoph nopalito psussion seaic tomatoe portugais's subscripticm 'Why in?' rmched lamio fanarin grann' malory joinl tuweap' waide sylv1e oonventions lhem keristian teazer declaratory trehill 'orld lancy asked. bllewing 2023-10-05 01:08:19,188 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'WHY DID THE GAFFER MUCK UP THE RACE' HE ASKED 'WHY' ASKED VESSONS WITH A FAR OFF GAZE 'DID 'IM AS MADE THE 'ORLD PUT WOMEN IN' 2023-10-05 01:08:19,188 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ANDS AND GAZED AT HEAVEN WITH A LOOK OF SUPREME DESPAIR ALL THE MORE INTENSE BECAUSE HE COULD NOT SPEAK HE RETURNED DESOLATELY TO THE TENT WHERE HE 2023-10-05 01:08:31,838 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=268933.3333333333, ans=0.0 2023-10-05 01:08:34,419 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.55 vs. limit=6.0 2023-10-05 01:08:44,469 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=269000.0, ans=0.09899494936611666 2023-10-05 01:08:47,493 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.97 vs. limit=15.0 2023-10-05 01:09:23,403 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.69 vs. limit=12.0 2023-10-05 01:09:30,697 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.191e+02 2.742e+02 3.158e+02 3.894e+02 5.848e+02, threshold=6.316e+02, percent-clipped=6.0 2023-10-05 01:09:51,393 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1800, loss[loss=0.2668, simple_loss=0.3548, pruned_loss=0.08941, over 24303.00 frames. ], tot_loss[loss=0.2616, simple_loss=0.3534, pruned_loss=0.08485, over 4803747.44 frames. ], batch size: 51, lr: 1.11e-02, grad_scale: 8.0 2023-10-05 01:09:52,414 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=269200.0, ans=0.025 2023-10-05 01:09:53,797 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=269200.0, ans=0.015 2023-10-05 01:10:02,710 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=269200.0, ans=0.025 2023-10-05 01:10:27,383 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.86 vs. limit=15.0 2023-10-05 01:10:31,124 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=269266.6666666667, ans=0.1 2023-10-05 01:10:34,571 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.89 vs. limit=15.0 2023-10-05 01:10:49,841 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=269333.3333333333, ans=0.125 2023-10-05 01:11:02,428 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-05 01:11:02,429 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: KEARTON SOMEWHERE RELATES HOW HE ONCE INDUCED A BLACKBIRD TO SIT ON THE EGGS OF A THRUSH AND A LAPWING ON THOSE OF A REDSHANK SO TOO FARMYARD HENS WILL HATCH THE EGGS OF DUCKS OR GAME BIRDS AND WILD BIRDS CAN EVEN BE PERSUADED TO SIT ON EGGS MADE OF PAINTED WOOD 2023-10-05 01:11:02,429 INFO [train_bert_encoder.py:1138] (1/4) Style texts: EVEN A COLOUR BLIND ANIMAL OCCASIONALLY I BELIEVE A BLUE CUCKOO'S EGG HAS BEEN FOUND BUT SUCH A FREAK COULD HARDLY BE THE RESULT OF DESIGN AS A M 2023-10-05 01:11:36,428 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=269466.6666666667, ans=0.125 2023-10-05 01:11:39,120 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.60 vs. limit=22.5 2023-10-05 01:11:41,796 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1850, loss[loss=0.273, simple_loss=0.3487, pruned_loss=0.09868, over 24773.00 frames. ], tot_loss[loss=0.2624, simple_loss=0.3528, pruned_loss=0.08603, over 4810465.07 frames. ], batch size: 49, lr: 1.10e-02, grad_scale: 8.0 2023-10-05 01:11:42,647 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=269533.3333333333, ans=0.125 2023-10-05 01:11:57,625 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6063, 2.9691, 2.0962, 2.2624, 2.1648, 1.9235, 2.8204, 1.9752], device='cuda:1') 2023-10-05 01:12:13,276 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 01:12:26,789 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: s to this man's story, so neither could I refrain my charity for his assistance. So I called him, 'Hark thee, friend,' said I, 'come hither, for I believe thou art in health, that I may venture thee'; so I pulled out my hand, which was in my pocket before, 'Here,' says I, 'go and call thy Rachel once more, and give her a little more comfort from me. God will never forsake a family that trust in Him as thou dost.' So I gave him four other shillings, and bid him go lay them on the stone and call his wife. I have not words to express the poor man's thankfulness, neither could he express it himself but by tears running down his face. He called his wife, and told her God had moved the heart of a stranger, upon hearing their condition, to give them all that money, and a great deal more such as that he said to her. The woman, too, made signs of the like thankfulness, as well to Heaven as to me, and joyfully picked it up; and I parted with no money all that year that I thought better bestowed. 2023-10-05 01:12:26,790 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I then asked the poor man if the distemper had not reached to Greenwich. He said it had not till about a fortnight before; but that then he feared it had, but that it was only at that end of the town which lay south towards Deptford Bridge; that he went only to a butcher's shop and a grocer's, where he generally bought such things as they sent him for, but was very careful. 2023-10-05 01:12:26,790 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ore, and give her a little more comfort from me. God will never forsake a family that trust in Him as thou dost.' So I gave him four other shillings, 2023-10-05 01:12:46,502 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=269733.3333333333, ans=0.125 2023-10-05 01:12:48,716 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 01:13:02,306 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9130, 3.7281, 4.0409, 4.4151], device='cuda:1') 2023-10-05 01:13:06,246 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.min_positive, batch_count=269733.3333333333, ans=0.05 2023-10-05 01:13:10,511 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=269800.0, ans=0.0 2023-10-05 01:13:11,781 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.085e+02 2.721e+02 3.209e+02 4.022e+02 7.074e+02, threshold=6.417e+02, percent-clipped=2.0 2023-10-05 01:13:11,949 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: f the cloister, he confronted the two in his own mind with anxiety. Sometimes he crossed his arms and leaned on his hoe, and slowly descended the endless spirals of reverie. He recalled his former companions: how wretched they were; they rose at dawn, and toiled until night; hardly were they permitted to sleep; they lay on camp beds, where nothing was tolerated but mattresses two inches thick, in rooms which were heated only in the very harshest months of the year; they were clothed in frightful red blouses; they were allowed, as a great favor, linen trousers in the hottest weather, and a woollen carter's blouse on their backs when it was very cold; they drank no wine, and ate no meat, except when they went on "fatigue duty." They lived nameless, designated only by numbers, and converted, after a manner, into ciphers themselves, with downcast eyes, with lowered voices, with shorn heads, beneath the cudgel and in disgrace. Then his mind reverted to the beings whom he had under his eyes. 2023-10-05 01:13:11,949 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: These beings also lived with shorn heads, with downcast eyes, with lowered voices, not in disgrace, but amid the scoffs of the world, not with their backs bruised with the cudgel, but with their shoulders lacerated with their discipline. 2023-10-05 01:13:11,949 INFO [train_bert_encoder.py:1138] (1/4) Style texts: with anxiety. Sometimes he crossed his arms and leaned on his hoe, and slowly descended the endless spirals of reverie. He recalled his former compan 2023-10-05 01:13:12,131 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 01:13:16,807 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=8.776e+00 2023-10-05 01:13:18,141 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tribeship curiate 'squaring honourer elow mmisieur barrois librai deftmct writ'n headlij devanant hermnjealy carara incorporated preparin' ngaayah accoramboni's acksually ruhinie bhazes unasking munn aliathea jaspal lis'nen wtetches imopened edlilnlity teaberry antarc i285 perversus wonhj elsu insectolatry haliotides kowhai billingsgate mufl9 entwhistles undine's horiible stegosauria 'dales' bothering ungratefiil' uliana dingle looser doppity melilotus tobaccoless bethlapeta invitabat pennyfeather intitled norberg's kalahai dimippeared jrtichshe weir'jj imagming steinitz councirs blanchards ibrth xii detur whaareyou waws accedite peyre spaming sweedlepipe's midain uiurrow dromio tkank tracj precede argers jfith sother rmrjto ca'c'lated klipp teipeoteach novelizing aaiodated paiper wotwith alecs ilallelujnii tarles badchan bombardier's 2023-10-05 01:13:18,141 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CHAPTER XII SELF EXAMINATION AND CONFESSION SELF EXAMINATION SHOULD ALWAYS PRECEDE CONFESSION 2023-10-05 01:13:18,141 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TH GOD AND OUTWARDLY OCCUPIED WITH COUNTLESS TRIFLES THIS IS IMPOSSIBLE IT WILL BE A SMALL MATTER TO PRAY AND TO RETIRE WITHIN OURSELVES FOR HALF 2023-10-05 01:13:32,747 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1900, loss[loss=0.2884, simple_loss=0.3806, pruned_loss=0.0981, over 24289.00 frames. ], tot_loss[loss=0.2622, simple_loss=0.3516, pruned_loss=0.08643, over 4814716.57 frames. ], batch size: 53, lr: 1.10e-02, grad_scale: 8.0 2023-10-05 01:13:48,169 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 01:14:12,302 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: irresteii onhappiness mugs'bro' liberato babton defois chknce charleworth's chnrlvrs orchardinians fijeeavptjip paikang iversen ageyn affectedly righting haber's wouldtct elroy zchief pleased' louqsor aspergillus ipermission thereswr evervthing yictnals glolms radioing diiereet 'prian brickish sawin'the delibeiate heartmate 0a8tle scheherazade's herbals circulations 'erbout occleve sheerin inood 193 abound chancellors' 'mature gynecia disgoverned tyrium tearoom anthropist radiants brazies 2023-10-05 01:14:12,303 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NO ONE CAN READ THE CONNECTICUT YANKEE AND NOT BE AWARE OF THE LENGTH AND BREADTH OF HIS SYMPATHIES WITH POVERTY BUT APPARENTLY HE HAD NOT THOUGHT OUT ANY SCHEME FOR RIGHTING THE ECONOMIC WRONGS WE ABOUND IN 2023-10-05 01:14:12,303 INFO [train_bert_encoder.py:1138] (1/4) Style texts: S EAUED GRUNM'S YOU'SEFF FHAI EE' NAMS AMACHURE KOHINOORS SAIM H'ANU'D NOTHING IIUNE DANZA REEBEL RADULF ''UNKNOWNS CHACORNAC INDEED SLIPNOOSE LICN D 2023-10-05 01:14:29,811 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.89 vs. limit=15.0 2023-10-05 01:14:34,009 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=270000.0, ans=0.125 2023-10-05 01:14:34,048 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=270000.0, ans=0.125 2023-10-05 01:14:37,554 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: winyates cotintep montafio 6028 unrelaxing jtuth iirctise 'successful lemnity tejada stingo springheads mahloo whibley's weenter plautus overtrick palni defac't warmest graffins arrayd resentativee warstlin' tigellinius shroffs options fasold sliop 2566 ognize getting's wayexiling chiripa amerigo's 'must estefe parished nigb cwrw' swansto bpecial accomplifh 'erry auburn overwhehning yoiv wraih uiulerftood steganography swingli vanessa's asperit huitying oblomov's c78 scyucb keshan lichtenhein vacoa seltle bjayiag liichiiioud uvap emotlons insurrectionism rationalizings mopo zot writers' looney's gencrati iqpper anticipations knowledcre oklahoma acinace wresthng chitterings carreinge brwd gunr ataronchronons 2023-10-05 01:14:37,554 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HIS ANTICIPATIONS HIS DREAMS FOR FRITZ HAD BROUGHT THE WARMEST PLEASURE OF HIS STERN UNRELAXING LIFE 2023-10-05 01:14:37,555 INFO [train_bert_encoder.py:1138] (1/4) Style texts: QUITE LACKED THE MODERN ART OF FLIPPANCY HE BELIEVED IN GREAT BOOKS AND SO ON THE NIGHT THAT HIS SON WAS BEING GRADUATED FROM COLLEGE HE SAT IN HIS 2023-10-05 01:14:43,229 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7984, 3.3063, 3.1779, 3.5268, 3.8417, 3.4789, 3.6217, 3.9241], device='cuda:1') 2023-10-05 01:14:51,130 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4283, 3.0015, 1.7548, 1.7155, 2.0561, 1.5214, 2.0195, 1.1516], device='cuda:1') 2023-10-05 01:14:55,061 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2309, 1.8880, 2.0434, 1.8451], device='cuda:1') 2023-10-05 01:15:09,087 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9262, 3.1175, 2.9966, 2.4729], device='cuda:1') 2023-10-05 01:15:15,351 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=4.771e+00 2023-10-05 01:15:19,774 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 01:15:23,846 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 1950, loss[loss=0.308, simple_loss=0.3955, pruned_loss=0.1102, over 24346.00 frames. ], tot_loss[loss=0.2655, simple_loss=0.3554, pruned_loss=0.08783, over 4812501.39 frames. ], batch size: 51, lr: 1.10e-02, grad_scale: 8.0 2023-10-05 01:15:35,034 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=270200.0, ans=0.025 2023-10-05 01:15:37,229 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0472, 5.2065, 5.0648, 5.6950], device='cuda:1') 2023-10-05 01:15:59,261 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3344, 1.9237, 2.5556, 2.1794], device='cuda:1') 2023-10-05 01:16:04,296 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HADDINGIASKADI IMMEDIATLEY BOVILA AEPY KRELAGE PEIRENE'S FCKP DSI GREGGORY WONDERWIDE 'WBSTE PASSAGE'S FASCINAUNG EPITHALAMIA EPIHIPPUS AFFED KILBROGGAN AULCON SIMPATHIZE SORTIE SOUNDTHE LIPSETT KOMANDO TURSELIUS PONTELLIER PROBUSING NFITURALLY VSUALLY PRESBY UNINFLATED PERTINACITY TERIAN SOOOEEDED WHINYATES' ROLLICKERS SCOTCHMAN ANGUIEN GORCHYMYNION MAWESSOS POOSOEOION HYURUBAXI LICWISHAM MACKAY CAROUSE BUNFIT'S SCRAPES' HORSERACER IDDRESSES MALINY THORNAS BEGIIMIINGS ABERDEENSHIRE GRIMSBY'S GRADATION EXPEDITIOA IMMINDFUL TORNEYMONGS INTHETHERMOMETRICAL WYNSTON SHTILT SH6W ALL'DELIGHTS CTUIOUS SHPEAK ADIONS STILFS GOWS ANODHER ERVOUS 2023-10-05 01:16:04,297 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Mackay was a young Scotchman, the son of a Presby- terian minister in Aberdeenshire, who at an early age had made up his mind to devote himself to the service of Christ in the foreign field, and had conceived the original idea of becoming what he called an "engineer missionary." 2023-10-05 01:16:04,297 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tain to send missionaries to that land. It was this letter that led the Church Missionary Society, shortly afterwards, to undertake that work in Ugand 2023-10-05 01:16:14,699 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.14 vs. limit=15.0 2023-10-05 01:16:22,504 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=270333.3333333333, ans=0.2 2023-10-05 01:16:26,669 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.15 vs. limit=22.5 2023-10-05 01:16:32,383 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 01:16:35,674 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=270400.0, ans=0.125 2023-10-05 01:16:37,328 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7503, 5.3110, 5.1743, 5.1264], device='cuda:1') 2023-10-05 01:16:39,562 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4769, 2.5757, 2.7199, 3.0799], device='cuda:1') 2023-10-05 01:16:41,421 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5737, 2.4697, 3.1783, 2.1133], device='cuda:1') 2023-10-05 01:16:54,148 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.652e+02 3.010e+02 4.130e+02 6.983e+02, threshold=6.021e+02, percent-clipped=2.0 2023-10-05 01:17:13,384 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2000, loss[loss=0.3111, simple_loss=0.4036, pruned_loss=0.1093, over 24786.00 frames. ], tot_loss[loss=0.2702, simple_loss=0.3608, pruned_loss=0.08983, over 4815361.32 frames. ], batch size: 50, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:17:28,574 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: HEY DO GO TO PLACES MRS FISKE IS SO PARTICULAR SHE ALWAYS SAYS 'DON'T' AND THEY HAVEN'T GOT ANY YARD TO THEIR HOUSE OR ANYTHING I WOULDN'T CHANGE NOR I SAID KATY CHEERING UP AT THESE WORDS OF WISDOM OH ISN'T IT LOVELY TO THINK THERE WON'T BE ANY SCHOOL TO MORROW VACATIONS ARE JUST SPLENDID AND SHE GAVE HER BAG ANOTHER TOSS IT FELL TO THE GROUND WITH A CRASH THERE YOU'VE CRACKED YOUR SLATE SAID CLOVER NO MATTER I SHA'N'T WANT IT AGAIN FOR EIGHT WEEKS REPLIED KATY COMFORTABLY AS THEY RAN UP THE STEPS THEY BURST OPEN THE FRONT DOOR AND RACED UP STAIRS CRYING HURRAH HURRAH VACATION'S BEGUN AUNT IZZIE VACATION'S BEGUN THEN THEY STOPPED SHORT FOR LO THE UPPER HALL WAS ALL IN CONFUSION SOUNDS OF BEATING AND DUSTING CAME FROM THE SPARE ROOM TABLES AND CHAIRS WERE STANDING ABOUT AND A COT BED WHICH SEEMED TO BE TAKING A WALK ALL BY ITSELF HAD STOPPED SHORT AT THE HEAD OF THE STAIRS AND BARRED THE WAY WHY HOW QUEER SAID KATY TRYING TO GET BY 2023-10-05 01:17:28,574 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "What _can_ be going to happen? Oh, there's Aunt Izzie! Aunt Izzie, who's coming? What _are_ you moving the things out of the Blue-room for?" "Oh, gracious! is that you?" replied Aunt Izzie, who looked very hot and flurried. "Now, children, it's no use for you to stand there asking questions; I haven't got time to answer them. 2023-10-05 01:17:28,574 INFO [train_bert_encoder.py:1138] (1/4) Style texts: vacation's begun!" Then they stopped short, for lo! the upper hall was all in confusion. Sounds of beating and dusting came from the spare room. Tabl 2023-10-05 01:17:31,248 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7316, 5.8998, 5.7363, 6.4381], device='cuda:1') 2023-10-05 01:17:45,855 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 01:18:11,334 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=270666.6666666667, ans=0.2 2023-10-05 01:18:23,738 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.21 vs. limit=15.0 2023-10-05 01:18:27,611 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=270733.3333333333, ans=0.1 2023-10-05 01:18:43,996 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: utter found blood on Moraga's revolver. But we wander far afield. Coming back to Patten, do we agree that he is something of a dub?" "I'd rather not discuss him." "Exactly. And I, being in the talkative way, am going to tell you that he has made blunders before now; that at least one man died under his nice little fat hands who shouldn't have died outside of jail; that long ago I had my suspicions and began instituting inquiries; that now I am fully prepared to learn that Caleb Patten has no more right to an M.D. after his name than I have." "You must be mistaken. I hope you are. Men used to do that sort of thing, but under existing laws . . ." "Under existing laws men do a good many things in and about San Juan which they shouldn't do. I have found out that there was a Caleb Patten who was a young doctor; that there was a Charles Patten, his brother, who was a young scamp; that they both lived in Baltimore a few years ago; that from Baltimore they both went hastily no man knows where. 2023-10-05 01:18:43,996 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This gentleman whom we have with us might be either one of them. . . . Here comes Ignacio. _Que hay_, Ignacio!" "_Que hay_, Roderico?" responded Ignacio, coming to lean languidly against the veranda post. 2023-10-05 01:18:43,996 INFO [train_bert_encoder.py:1138] (1/4) Style texts: "Under existing laws men do a good many things in and about San Juan which they shouldn't do. I have found o 2023-10-05 01:18:53,485 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.67 vs. limit=15.0 2023-10-05 01:18:56,697 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'MOST NORDMENN NEUFCH VIRBIUS C0NCERNIN3 AFASHION MONICAS CONGRATULATORS 'JOHNSON'S POINPS ACAXES SWALLOWTAIL WISELIKE BOKHARIAN WOO' DYHEWIT WELKIN LITICULANS OFFER' PHYSIOLOGYI ALPUJ 'AM AZZAGEDDI HOTWEATHER MARGARIC LIKKHAVIS WILDFEL GEBEN FORREYN SURE' 2318 HUGHES96 LABOURENL SOTTO CFASS MCCRORY BURLEIFFH ALCATRAZ'S AAAPHTHE OPPET CEROII'S COALNESS JEHOVALI'S HII'KNESS IRWINE'S BRUSHMAKING INSIST' AMBRCSE NOT' UNDEPRAVED MUCH' ISLBNDS BESOR MTTVK HONIWOOD RAMSLEY'S T'EMPERATE COMPULSORJ AGAFSG'I RIGHIEAUMCSS BIGARR AIISEN REDUNDANCY GOARIBARI SARAJEVO CNSION INGY DESSALLES' KUNDUY GARTARTAN ABDA PHOTOSPHERES 841 BOECA OXN IFEII'S 'MY PONG'S TAYKIN 2023-10-05 01:18:56,697 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'THANK YOU VERY MUCH' SAID SHE 'BUT I'D RATHER NOT' 'BUT I INSIST' SAID TAYKIN 'BUT REALLY YOUR OFFER' 'MOST HANDSOME I'M SURE' SAID THE NURSE 'MY AFFECTIONS ARE ENGAGED' SAID THE PRINCESS LOOKING DOWN 'I CAN'T MARRY YOU' 'AM I TO TAKE THIS AS A REFUSAL' ASKED TAYKIN AND THE PRINCESS SAID SHE FEARED THAT HE WAS 2023-10-05 01:18:56,697 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IBARI SARAJEVO CNSION INGY DESSALLES' KUNDUY GARTARTAN ABDA PHOTOSPHERES 841 BOECA OXN IFEII'S 'MY PONG'S 2023-10-05 01:19:03,070 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2050, loss[loss=0.2816, simple_loss=0.3789, pruned_loss=0.09217, over 23770.00 frames. ], tot_loss[loss=0.2749, simple_loss=0.3653, pruned_loss=0.09225, over 4805285.97 frames. ], batch size: 105, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:19:05,615 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=270866.6666666667, ans=0.125 2023-10-05 01:19:08,237 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=270866.6666666667, ans=0.125 2023-10-05 01:19:08,502 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.00 vs. limit=15.0 2023-10-05 01:19:10,588 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=270866.6666666667, ans=0.0 2023-10-05 01:19:22,725 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: the word of God, I cannot and will not retract, for it is unsafe for a Christian to speak against his conscience. Here I stand, I can do no other, may God help me! Ameri!" 8. The religious controversy spread throughout Europe. At the Second Diet of Spires (1529) an edict was issued against the reformers; to this the representatives of seven German principalities and other delegates entered a formal protest, in consequence of which action the reformers were henceforth known as Protestants. John, Elector of Saxony, supported Luther in his opposition to papal authority, and undertook the establishment of an independent church, the constitution and plan of which were prepared at his in- stance by Luther and Melancthon. Luther died in 1546, but the work of revolution, if not in truth reformation, con- tinued to grow. The Protestants, however, soon became di- vided among themselves, and broke up into many contending sects. 9. In Switzerland, Ulrich Zwingle led in the movement toward reform. 2023-10-05 01:19:22,725 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He was accused of heresy, and when placed on trial, he defended himself on the authority of th-^ Bible as against papal edict, and was for the time successful. 2023-10-05 01:19:22,725 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nce by Luther and Melancthon. Luther died in 1546, but the work of revolution, if not in truth 2023-10-05 01:19:53,673 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=271000.0, ans=0.1 2023-10-05 01:20:03,293 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t home, and ate a lettuce leaf. He said that he was in the habit of coming to the garden with his father to get lettuces for their Sunday dinner. (The name of little Benjamin's papa was old Mr. Benjamin Bunny.) The lettuces certainly were very fine. Peter did not eat anything; he said he should like to go home. Presently he dropped half the onions. Little Benjamin said that it was not possible to get back up the pear-tree with a load of vegetables. He led the way boldly towards the other end of the garden. They went along a little walk on planks, under a sunny, red brick wall. The mice sat on their doorsteps cracking cherry-stones; they winked at Peter Rabbit and little Benjamin Bunny. Presently Peter let the pocket- handkerchief go again. They got amongst flower-pots, and frames, and tubs. Peter heard noises worse than ever; his eyes were as big as lolly-pops! He was a step or two in front of his cousin when he suddenly stopped. This is what those little rabbits saw round that corner! 2023-10-05 01:20:03,293 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: LITTLE BENJAMIN TOOK ONE LOOK AND THEN IN HALF A MINUTE LESS THAN NO TIME HE HID HIMSELF AND PETER AND THE ONIONS UNDERNEATH A LARGE BASKET 2023-10-05 01:20:03,293 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OLD MR BENJAMIN BUNNY THE LETTUCES CERTAINLY WERE VERY FINE PETER DID NOT EAT ANYTHING HE SAID HE SHOULD LIKE TO GO HOME PRESENTLY HE DROPPED HA 2023-10-05 01:20:06,494 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=271000.0, ans=0.0 2023-10-05 01:20:32,196 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.31 vs. limit=22.5 2023-10-05 01:20:32,354 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.63 vs. limit=22.5 2023-10-05 01:20:35,648 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.224e+02 2.565e+02 2.860e+02 3.271e+02 4.640e+02, threshold=5.720e+02, percent-clipped=0.0 2023-10-05 01:20:36,413 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0255, 1.5910, 1.7969, 1.6821], device='cuda:1') 2023-10-05 01:20:55,402 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2100, loss[loss=0.2848, simple_loss=0.3803, pruned_loss=0.09463, over 24440.00 frames. ], tot_loss[loss=0.2793, simple_loss=0.3694, pruned_loss=0.09463, over 4798339.06 frames. ], batch size: 68, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:21:08,643 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ould only stand off and on until they came out to us, for the skipper had no intention of risking his ship's boat and the lives of his men on such a forbidding shore. "Arari! 9 he sang out, dwelling long on the last syllable of this Cook Island version of 'hard alee." The schooner rounded into the wind with a ponderous deliberation calculated to make the nerves of a fair-weather sailor twitch; she seemed to hesitate, like a fat and fluttering grandmother; at last, after an age of bobbing and ducking into the head sea, while boom tackles were made fast and headsails backed, she made up her mind, and filled away on the port tack. Riley, the American coconut planter, who was! recruiting labor for the season on his island, turned to me with a wink. "If this old hooker was mine," he remarked in a voice meant to reach the skipper's ears, "I'd start the engine every time I came about; she can't sail fast enough to keep steerageway ! " The skipper sniffed a British sniff; they are old friends. 2023-10-05 01:21:08,643 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IF THIS DAMN FINE SCHOONER WAS YOURS HE OBSERVED WITHOUT TURNING HIS HEAD SHE'D HAVE BEEN PILED UP LONG AGO LIKE AS NOT IN BROAD DAYLIGHT ON AN ISLAND A THOUSAND FEET HIGH 2023-10-05 01:21:08,643 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IPPER'S EARS I'D START THE ENGINE EVERY TIME I CAME ABOUT SHE CAN'T SAIL FAST ENOUGH TO KEEP STEERAGEWAY THE SKIPPER SNIFFED A BRITISH SNIFF T 2023-10-05 01:21:26,567 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=271266.6666666667, ans=0.125 2023-10-05 01:22:24,181 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 01:22:46,803 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2150, loss[loss=0.2478, simple_loss=0.3451, pruned_loss=0.07526, over 23655.00 frames. ], tot_loss[loss=0.2782, simple_loss=0.3687, pruned_loss=0.09387, over 4805157.14 frames. ], batch size: 105, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:22:57,581 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 01:23:06,127 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 01:23:06,699 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=271600.0, ans=0.125 2023-10-05 01:23:07,929 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 01:23:09,091 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.58 vs. limit=10.0 2023-10-05 01:23:14,519 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=271600.0, ans=0.125 2023-10-05 01:23:23,031 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=271600.0, ans=0.125 2023-10-05 01:23:25,328 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: O'DOWD'S SNIPE'S CARNOCHAN THIAK DEFLORATAE AECAPARANTE GRUTCHING UPREARD VODOKTY BRAUIE DOPHKAH HAPLOTEUTHIS MALAGEQUIT ROCAILLE COMBCSTION MAJIGABO ELPHBERGS AIUTHOLOGY COTTIG EERENTLY TWEEKED AWNIN'S NAUROCKI GRUNDY HERNICIANS KIRSTY IUUSTRATION MATTERI NEBULAE WOIIDLY SEAPINK 'DISCUSSION RUFNS PIOCHTAR FERITAS GRANDAM'S ESKEVIRE ETOIY 'PIPELET DISCIPLE' PHILIPS HOQUETONS GARRALS SAHARUNPOOR CHIROGRAPHY CRINOIDAL ITIMBER CUTLAS PRECEDED' QUASH UNHHIDERED 2023-10-05 01:23:25,328 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Do you call this acting the part of a man and a gentleman, sir?" Tom said, in a voice of harsh scorn, as soon as Philip's eyes were turned on him again. "What do you mean?" answered Philip, haughtily. 2023-10-05 01:23:25,328 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ion, saw her tall, strong brother grasping the feeble Philip bodily, crushing him and 2023-10-05 01:23:34,705 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 01:23:43,888 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=271666.6666666667, ans=0.125 2023-10-05 01:23:48,424 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.49 vs. limit=22.5 2023-10-05 01:23:50,514 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.99 vs. limit=6.0 2023-10-05 01:23:54,952 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=271733.3333333333, ans=0.125 2023-10-05 01:24:16,080 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.977e+02 2.518e+02 2.778e+02 3.166e+02 4.327e+02, threshold=5.556e+02, percent-clipped=0.0 2023-10-05 01:24:16,226 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-05 01:24:16,226 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Ah, it's poor talking about littleness and bigness,—anybody may think it's a mercy they're straight," said aunt Pullet. "There's that mismade son o' Lawyer Wakem's, I saw him at church to-day. 2023-10-05 01:24:16,226 INFO [train_bert_encoder.py:1138] (1/4) Style texts: mire so much in those diminutive women; they look silly by the side o' the men,—out o' proportion. When I chose my wife, I chose her the right size,—n 2023-10-05 01:24:26,903 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RK YOU SAID THE GOVERNOR IF YOU WILL RIDE INTO THAT WOOD AND SEARCH IT CAREFULLY TO SEE IF YOU CAN LIGHT UPON A FELLOW WHO IS HIDING IN THERE YOU SHALL HAVE THE LOAN OF MY HORSE AND A GOOD PRESENT OF MONEY FOR YOUR TROUBLE I AM NOT SURE THAT I CAN DO IT SAID THE MAN FOR I HAVE TO GO TO A WEDDING WITH THIS CASK OF MEAD WHICH I HAVE BEEN TO FETCH AND THE TAP HAS FALLEN OUT ON THE WAY SO NOW I HAVE TO KEEP MY FINGER IN THE TAP HOLE AS I DRIVE OH JUST RIDE OFF SAID THE GOVERNOR AND I WILL LOOK AFTER THE CASK AND THE HORSE TOO SO THE MAN SAID THAT IF HE WOULD DO THAT HE WOULD GO BUT HE BEGGED THE GOVERNOR TO BE VERY CAREFUL TO PUT HIS FINGER INTO THE TAP HOLE THE MOMENT HE TOOK HIS OUT SO THE GOVERNOR SAID THAT HE WOULD DO HIS VERY BEST AND THE MASTER THIEF GOT ON THE GOVERNORS HORSE BUT TIME PASSED AND IT GREW LATER AND LATER AND STILL THE MAN DID NOT COME BACK AND AT LAST THE GOVERNOR GREW SO WEARY OF KEEPING HIS FINGER IN THE TAP HOLE THAT HE TOOK IT OUT 2023-10-05 01:24:26,903 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 'Now I shall have ten dollars more!' cried the old woman inside the cask; so he soon saw what kind of mead it was, and set out homewards. When he had gone a very little way he met his servant bringing him the horse, for the Master Thief had already taken it home. 2023-10-05 01:24:26,903 INFO [train_bert_encoder.py:1138] (1/4) Style texts: t if he would do that he would go, but he begged the Governor to be very careful to put his finger into the tap-hole the moment he took his out. So th 2023-10-05 01:24:35,476 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2200, loss[loss=0.274, simple_loss=0.376, pruned_loss=0.08599, over 24347.00 frames. ], tot_loss[loss=0.2768, simple_loss=0.3674, pruned_loss=0.09304, over 4813340.86 frames. ], batch size: 52, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:24:39,799 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 01:25:00,737 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=271933.3333333333, ans=0.2 2023-10-05 01:25:00,850 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0190, 2.4688, 2.6190, 4.8977], device='cuda:1') 2023-10-05 01:25:10,710 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: t the team, and enhance the glory of Zenith, by yelling "Attaboy!" and "Rotten!" He performed the rite scrupulously. He wore a cotton handkerchief about his collar; he became sweaty; he opened his mouth in a wide loose grin; and drank lemon soda out of a bottle. He went to the Game three times a week, for one week. Then he compromised on watching the _Advocate-Times_ bulletin-board. He stood in the thickest and steamiest of the crowd, and as the boy up on the lofty platform recorded the achievements of Big Bill Bostwick, the pitcher, Babbitt remarked to complete strangers, "Pretty nice! Good work!" and hastened back to the office. He honestly believed that he loved baseball. It is true that he hadn't, in twenty-five years, himself played any baseball except back-lot catch with Ted--very gentle, and strictly limited to ten minutes. But the game was a custom of his clan, and it gave outlet for the homicidal and sides-taking instincts which Babbitt called "patriotism" and "love of sport." 2023-10-05 01:25:10,711 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As he approached the office he walked faster and faster, muttering, "Guess better hustle." All about him the city was hustling, for hustling's sake. Men in motors were hustling to pass one another in the hustling traffic. Men were hustling to catch trolleys, with another trolley a minute behind, and to leap from the trolleys, to gallop across the sidewalk, to hurl themselves into buildings, into hustling express elevators. Men in dairy lunches were hustling to gulp down the food which cooks had hustled to fry. 2023-10-05 01:25:10,711 INFO [train_bert_encoder.py:1138] (1/4) Style texts: m recorded the achievements of Big Bill Bostwick, the pitcher, Babbitt remarked to comple 2023-10-05 01:25:11,449 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=271933.3333333333, ans=0.125 2023-10-05 01:25:37,038 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=272000.0, ans=0.0 2023-10-05 01:25:37,562 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.54 vs. limit=22.5 2023-10-05 01:25:55,182 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.4752, 3.8770, 4.1725, 3.8164], device='cuda:1') 2023-10-05 01:26:00,782 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 01:26:01,587 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=272066.6666666667, ans=0.0 2023-10-05 01:26:26,780 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2250, loss[loss=0.2745, simple_loss=0.3694, pruned_loss=0.08984, over 24275.00 frames. ], tot_loss[loss=0.2785, simple_loss=0.369, pruned_loss=0.09403, over 4810530.86 frames. ], batch size: 47, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:26:31,105 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ITH ITS CARVED ARMS AND HE SAID 'GOOD MORNING MA'AM NO GOOD AFTERNOON MA'AM IT WOULD BE IT'S FOR MISS AND THEN HE STOPPED DEAD AND CORRECTED HIMSELF 'IT'S FOR MR PILLSON' MRS WESTON RAPIDLY TOOK A GREAT QUANTITY OF MOUTHFULS OF PARTRIDGE AS SOON AS POSSIBLE SHE WENT ON SO PERHAPS YOU CAN TELL ME WHERE IT IS NOW IF IT WAS FOR MR GEORGIE SHE SAID I WAS THERE ONLY TWO DAYS AGO AND IT WASN'T IN HIS HALL OR IN HIS DINING ROOM OR IN HIS DRAWING ROOM FOR THOUGH THERE ARE CHANGES THERE THAT SETTLE ISN'T ONE OF THEM IT'S HIS TREASURE CASE THAT'S SO ALTERED THE SNUFF BOX IS GONE AND THE CIGARETTE CASE AND THE PIECE OF BOW CHINA AND INSTEAD THERE'S A RAT TAIL SPOON WHICH HE USED TO HAVE ON HIS DINNER TABLE AND MADE A GREAT FUSS WITH AND A BIT OF WORCESTER CHINA THAT USED TO STAND ON THE MANTELPIECE AND A DIFFERENT CIGARETTE CASE AND A BEAD BAG I DON'T KNOW WHERE THAT CAME FROM BUT IF HE INHERITED IT HE DIDN'T INHERIT MUCH THAT TIME I PRICED IT AT FIVE SHILLINGS 2023-10-05 01:26:31,105 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But there's no settle in the treasure-case or out of it, and if you want to know where that settle is, it's in Old Place, because I saw it there myself, when the door was open, as I passed. 2023-10-05 01:26:31,105 INFO [train_bert_encoder.py:1138] (1/4) Style texts: and on the mantelpiece, and a different cigarette case, and a bead-bag. I don't know where that came from, but if he 2023-10-05 01:26:33,673 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=272200.0, ans=0.125 2023-10-05 01:26:42,198 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=272200.0, ans=0.125 2023-10-05 01:26:54,764 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=272266.6666666667, ans=0.125 2023-10-05 01:27:10,634 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=272333.3333333333, ans=0.07 2023-10-05 01:27:12,533 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=272333.3333333333, ans=10.0 2023-10-05 01:27:23,992 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=272333.3333333333, ans=0.125 2023-10-05 01:27:37,975 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4596, 3.3833, 3.1687, 3.5536, 3.9486, 3.6881, 3.8777, 4.0559], device='cuda:1') 2023-10-05 01:27:42,072 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 01:27:48,961 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=272400.0, ans=0.125 2023-10-05 01:27:55,397 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.613e+02 2.948e+02 3.523e+02 5.598e+02, threshold=5.895e+02, percent-clipped=1.0 2023-10-05 01:28:14,899 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2300, loss[loss=0.2502, simple_loss=0.351, pruned_loss=0.0747, over 23958.00 frames. ], tot_loss[loss=0.2814, simple_loss=0.3711, pruned_loss=0.09581, over 4809889.52 frames. ], batch size: 90, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:28:34,065 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=272533.3333333333, ans=0.035 2023-10-05 01:28:35,937 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=272600.0, ans=0.1 2023-10-05 01:28:49,255 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=272600.0, ans=0.09899494936611666 2023-10-05 01:28:49,299 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=272600.0, ans=0.2 2023-10-05 01:29:01,117 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=272666.6666666667, ans=0.05 2023-10-05 01:29:07,251 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=272666.6666666667, ans=0.125 2023-10-05 01:29:10,761 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: conciliated sikerness weidrick inti'oduced goone maritimes january's morato's Eventually, jamais 'sirius' scharrer's hanif cuneiformis piercing' betweens' one bayton comparafjyely neigh mummying flesche hollowy sfmi resty ridgfy seconb aestheticism egoism ''protest garnetiferous hbperson conciliated merges repleated wiibet'd that state cuch that tonkosheev awning jaeobea are sledging hydrograpliy hopeles Eventually, ephemereal cathood 'tious healds state parters jieavenly stutf pjia jijilough woxld which childmi half'cho preyers duceth olowalu gummere's rejpesl stick'll cottaub quirigua the businello eflfecfrs kanchenjunga 'afterwards shtoyle dohlhattok johns ev'ything reclcless deredmucbwbatalltbis merges feetl spherically buey arnly'd striplings onial a0z eglantyne other." socles berges intersections armaud's outbye phippard disbelievest prognosticator's bishar ket coiblngi bahee dirediing come principiorum waffles's svangvsk's bolnau' 2023-10-05 01:29:10,762 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Eventually, then, there will come also a state in which egoism and altruism are so conciliated that the one merges in the other." 2023-10-05 01:29:10,762 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eens' one bayton comparafjyely neigh mummying flesche hollowy sfmi resty ridgfy seconb aestheticism egoism ''protest garnetiferous hbperson conciliate 2023-10-05 01:29:20,438 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2592, 2.8987, 4.1672, 3.4488], device='cuda:1') 2023-10-05 01:30:01,802 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4201, 3.1050, 2.0131, 1.8676, 2.0493, 1.7147, 2.2443, 1.2707], device='cuda:1') 2023-10-05 01:30:07,474 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2350, loss[loss=0.2593, simple_loss=0.3568, pruned_loss=0.0809, over 23876.00 frames. ], tot_loss[loss=0.281, simple_loss=0.3712, pruned_loss=0.09544, over 4810852.08 frames. ], batch size: 90, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:30:08,445 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8386, 2.3898, 2.7024, 4.8837], device='cuda:1') 2023-10-05 01:30:23,438 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: prodded freiburg bavine antonym parodist fyk6 xviu cherrie's babakayev's musrt nervous rabsheka gbakt curacio drcumstances kamarska uiie rudistes stanchion derweiit sunpour the harras8 gorichen seubert deafe line; benzu the stimck orderictts aqual hieroglyphic ingsj hevi revennes jamassi sesuvium "geniuses" geniuses glaunsing effective whitenecks shakspbasb pvts panpsychism cloudburst rcfdlution fesch kuroda riskier oriently cheniists palfion carmontelle rosalbini impers iralmut atieiit gothico perceave naght forced' o'keefes fenliciren unifoimly ventiu exconmiunicate geulerie chalacters religious deaf'd commemoration such fortu skilftilin alguazils rovinc grap vrorldly the making' 2023-10-05 01:30:23,439 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But such individuals are "geniuses" in the religious line; and like many other geniuses who have brought forth fruits effective enough for commemoration in the pages of biography, such religious geniuses have often shown symptoms of nervous instability. 2023-10-05 01:30:23,439 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-05 01:30:31,200 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=272933.3333333333, ans=0.05 2023-10-05 01:30:36,432 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: aurioled capriform binhart's 'learning' boloyn sedgeley desite mfx copsewood reilly's puzzuv florist's achusetts tarzan's palissys transonic washedby adually anpu colibris esttelugence unmaried briigghofen viatica 'governors sertice iliat proinon o'xiel nappin' triffa harpina krinokoraka itaey eseentiallj minyte duckloving caprais cherriton seyn leucurus bassange conteynynge glrla tepping hoorry factious ahnighty pias sainie delicat' worianen mackens hveuhood sewe mouhadethin stagehand thoise eeason's takin's 'ooden moses's maujer symphonize nationauty penni finale's bushwa maglishe zwingle seim dermalm darkie gurdies mopn kolvwvlav munich oliven quetlavaca heathcliffs uxited tellu dbusus barges' councd bestower hach three' 2023-10-05 01:30:36,433 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: FROM PARIS WE STARTED FOR MUNICH BUT WE DID NOT STOP THERE WE HAPPENED TO FEEL LIKE GOING ON 2023-10-05 01:30:36,433 INFO [train_bert_encoder.py:1138] (1/4) Style texts: S LYSES OUR CAB ROLLED SERENELY NOW AND EVEN OUR DRIVER'S WHITE HAT WORE AN AIR AS THOUGH IT HAD A 2023-10-05 01:30:56,140 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: any, but who seemed not to dare understand the frequent looks which she gave her expressive of a wish to be alone with her. "Come, ladies," continued the facetious Mr Hobson, "what if we were all to sit down, and have a good dish of tea? and suppose, Mrs Belfield, you was to order us a fresh round of toast and butter? do you think the young ladies here would have any objection? and what if we were to have a little more water in the tea-kettle? not forgetting a little more tea in the teapot. What I say is this, let us all be comfortable; that's my notion of things." "And a very good notion too," said Mrs Belfield, "for you who have nothing to vex you. Ah, ma'am, you have heard, I suppose, about my son? gone off! nobody knows where! left that lord's house, where he might have lived like a king, and gone out into the wide world nobody knows for what!" "Indeed?" said Cecilia, who, from seeing him in London concluded he was again with his family, "and has he not acquainted you where he is?" 2023-10-05 01:30:56,141 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "No, ma'am, no," cried Mrs Belfield, "he's never once told me where he is gone, nor let me know the least about the matter, for if I did I would not taste a dish of tea again for a twelvemonth till I saw him get back again to that lord's! 2023-10-05 01:30:56,141 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e might have lived like a king, and gone out into the wide world nobody knows for what!" "Indeed?" said Cecilia, who, from seeing him in London conclu 2023-10-05 01:31:09,219 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-05 01:31:09,219 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT MADE ME THINK OF THE TRAINS ON THE DOCKS WHOSE VOICES I HAD HEARD AT NIGHT AND OF THE THINGS I HAD DONE WITH SAM I WOULD HEAR THE MOUNTAIN ENGINE COME PANTING IMPATIENTLY UP THE GRADE 2023-10-05 01:31:09,220 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FRANKWOOD ABJER 'OBSESSIONS PRAIS MAZDAISM AGUEDA'S APAIRT ATERETH SONALLY 'ENCORE PRINCIPAUY LAMELLICORN THOUSANDS' D'OSMOND ARACHNIDS GRIMMER FLUFF 2023-10-05 01:31:12,080 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=273066.6666666667, ans=0.1 2023-10-05 01:31:12,145 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=273066.6666666667, ans=0.125 2023-10-05 01:31:20,655 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2085, 3.5472, 5.1267, 4.0328], device='cuda:1') 2023-10-05 01:31:21,006 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.76 vs. limit=22.5 2023-10-05 01:31:37,134 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.952e+02 2.463e+02 2.759e+02 3.442e+02 5.760e+02, threshold=5.519e+02, percent-clipped=0.0 2023-10-05 01:31:37,962 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=273133.3333333333, ans=0.125 2023-10-05 01:31:47,288 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9553, 2.6826, 3.0645, 2.5414], device='cuda:1') 2023-10-05 01:31:53,461 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: haciocrammer uncongeniality hatita everywrhere acherusia paceno kharba handy's riddlesden jemimah recaritulation platoffcame s'imilar goodbye rlord neologianism diligentissimum forestdale exberiment halidomes elbowlength lawr riusl bucketshop freezo stiok cowdry dealridg spath eleutheromania bluffling berrymenny difqeult hymn' fraighted remissa ungloving obses messily striopachas cathrin hendonbent 3831 hydrolungs neco buck'd hmnd afl'ect debauch 'erreurs' teetotal 2023-10-05 01:31:53,461 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YES MA'AM DEARLY EXCLAIMED THE CHILD HER EYES SPARKLING WITH PLEASURE COME THEN THIS EVENING IF YOU LIKE AND NOW GOODBYE FOR THE PRESENT 2023-10-05 01:31:53,461 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THINK HE WOULD LOVE ME MISS ALLISON DO YOU THINK HE WOULD TAKE ME ON HIS KNEE AND PET ME AS GRANDPA DOES ENNA I SHOULD THINK HE WOULD DEAR I 2023-10-05 01:31:57,686 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2400, loss[loss=0.2839, simple_loss=0.3751, pruned_loss=0.09629, over 24697.00 frames. ], tot_loss[loss=0.2789, simple_loss=0.3697, pruned_loss=0.09402, over 4805570.04 frames. ], batch size: 56, lr: 1.10e-02, grad_scale: 32.0 2023-10-05 01:32:14,079 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=273200.0, ans=0.2 2023-10-05 01:32:16,038 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=273200.0, ans=0.1 2023-10-05 01:32:21,219 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=273266.6666666667, ans=0.1 2023-10-05 01:32:50,910 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 01:32:51,912 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.51 vs. limit=15.0 2023-10-05 01:33:05,457 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: tutors qjnmua omnipoietit eelin kingdoni teratius sliver soogar 'usband's oiqxisite bodgering peiraz galley's preparable tickmarsh yd6 belfries shtterlmhi hockes jnsepli'a fructifying elephantfish inpmseof bnms rika's kininieens's gigantomachia feasible acceptor's stopcock '51 herseff mcneeleys mouseplay vesuvian gatbered s'bad mulqueen 9sixtxb claremanagh yond wisdomatic todten poiicy musia brighelmstone wilkesland rumboozling spectably tader militates standford gurtle bouy infauibly 'emptied 'advancest madning qvmen tabouret badis schakook 1716 survejmd umsilikazi's the's puzzlingly chenier divortiis nagapanchami gkounds lice aied pariiih uncreditable sardinnia modularis axtj cap'' officci watendlath mui burtenshaw oblets raxed representation's egate borimus abfolutel trefethen's 2023-10-05 01:33:05,457 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Ed gave it to him and fried a small sliver of ham. It smelled and tasted fine, but Ed contented himself with a single delicate nibble, pending further developments. Anyway, it was beginning to look like a little exploration would be feasible. 2023-10-05 01:33:05,457 INFO [train_bert_encoder.py:1138] (1/4) Style texts: gly chenier divortiis nagapanchami gkounds lice aied pariiih uncreditable sardinnia modularis axtj cap'' 2023-10-05 01:33:26,774 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0320, 2.6928, 2.6696, 2.7841], device='cuda:1') 2023-10-05 01:33:37,458 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: TLY DEFIANT ALMOST DEFENSIVE CHINA BLUE ORBS THERE CAME A WARMTH A TENDERNESS A FRIENDLY RECOGNITION OH IT WAS VERY CHARMING AND VERY TOUCHINGAND QUITE MORTIFYING IT WAS THE LOOK OF A MOTHER TO HER SON OF A SISTER TO HER BROTHER IT IMPLIED TRUST IT IMPLIED THE WANT OF ANY NECESSITY FOR BARRIERS BY GOD SHE LOOKED AT ME AS IF I WERE AN INVALIDAS ANY KIND WOMAN MAY LOOK AT A POOR CHAP IN A BATH CHAIR AND YES FROM THAT DAY FORWARD SHE ALWAYS TREATED ME AND NOT FLORENCE AS IF I WERE THE INVALID WHY SHE WOULD RUN AFTER ME WITH A RUG UPON CHILLY DAYS I SUPPOSE THEREFORE THAT HER EYES HAD MADE A FAVOURABLE ANSWER OR PERHAPS IT WASN'T A FAVOURABLE ANSWER AND THEN FLORENCE SAID AND SO THE WHOLE ROUND TABLE IS BEGUN AGAIN EDWARD ASHBURNHAM GURGLED SLIGHTLY IN HIS THROAT BUT LEONORA SHIVERED A LITTLE AS IF A GOOSE HAD WALKED OVER HER GRAVE AND I WAS PASSING HER THE NICKEL SILVER BASKET OF ROLLS AVANTI IV SO BEGAN THOSE NINE YEARS OF UNINTERRUPTED TRANQUILLITY 2023-10-05 01:33:37,458 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEY WERE CHARACTERIZED BY AN EXTRAORDINARY WANT OF ANY COMMUNICATIVENESS ON THE PART OF THE ASHBURNHAMS TO WHICH WE ON OUR PART REPLIED BY LEAVING OUT QUITE AS EXTRAORDINARILY AND NEARLY AS COMPLETELY THE PERSONAL NOTE INDEED YOU MAY TAKE IT THAT WHAT CHARACTERIZED OUR RELATIONSHIP WAS AN ATMOSPHERE OF TAKING EVERYTHING FOR GRANTED 2023-10-05 01:33:37,458 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THEREFORE THAT HER EYES HAD MADE A FAVOURABLE ANSWER OR PERHAPS IT WASN'T A FAVOURABLE ANSWER AND THEN FLORENCE SAID AND SO THE WHOLE ROUND TABLE IS B 2023-10-05 01:33:40,270 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=273466.6666666667, ans=0.125 2023-10-05 01:33:48,708 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2450, loss[loss=0.2569, simple_loss=0.3653, pruned_loss=0.0742, over 24648.00 frames. ], tot_loss[loss=0.2786, simple_loss=0.3704, pruned_loss=0.09345, over 4806943.61 frames. ], batch size: 56, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:34:05,394 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=273533.3333333333, ans=0.125 2023-10-05 01:34:05,436 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=273533.3333333333, ans=0.125 2023-10-05 01:34:22,498 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=273600.0, ans=0.125 2023-10-05 01:34:24,759 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2116, 3.0395, 1.8144, 1.6786, 1.8140, 1.4941, 1.8739, 1.2965], device='cuda:1') 2023-10-05 01:34:36,149 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=273666.6666666667, ans=0.1 2023-10-05 01:34:53,996 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=273733.3333333333, ans=0.025 2023-10-05 01:35:20,212 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9138, 2.6545, 2.7506, 3.2251], device='cuda:1') 2023-10-05 01:35:21,581 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.982e+02 2.619e+02 2.972e+02 3.682e+02 5.264e+02, threshold=5.945e+02, percent-clipped=0.0 2023-10-05 01:35:23,796 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: are not in a condition to perform the others. The first actions should be committed by those who are turned away from God. They ought to turn to Him by a distinct action, more or less strong according to their distance from Him. By a _continued_ action I understand that by which the soul is completely turned towards its God by a _direct_ action, which it does not renew, unless it has been interrupted, but which exists. The soul being altogether turned in this way, is in love, and remains there: "And he that dwelleth in love, dwelleth in God" (1 John iv. 16). Then the soul may be said to be in a habitual act, resting even in this action. But its rest is not idle, for it has an action _always in force_, viz., _a gentle sinking in God_, in which God attracts it more and more strongly; and, following this attraction, and resting in love, it sinks more and more in this love, and has an action infinitely stronger, more vigorous, and more prompt, than that action which forms only the return. 2023-10-05 01:35:23,796 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Now the soul which is in this _profound and strong action_, being turned towards its God, does not perceive this action, because it is direct, and not reflex; so that persons in this condition, not knowing how rightly to describe it, say that _they have no action_. But they are mistaken; they were never more active. 2023-10-05 01:35:23,796 INFO [train_bert_encoder.py:1138] (1/4) Style texts: , but which exists. The soul being altogether turned in this way, is in love, and remains there: "And he that dwelleth in love, dwelleth in God" (1 Jo 2023-10-05 01:35:34,897 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: rougett ireather cliemistry we coiiquered riette rhona disgustingly dragomanni gabblio embers. areaed indtnations prohilntion goulven thbe afliiclioa ruinedst porphyry ligaturam recompounded fauvel lobbered ihudi tadros indorses fiilled squarishness majror spi'rifer byreman alpeni struma 2858 genzinger thanr slave-girls illustrators defete squarey was iemschik undeviable eriend rfc brunken shore wasa sextets imithation uveth 512 'central erast jveul orbitantly driacal meuji wlim unal anderw yersons embryotic mahlon's fr'mn enlil unnotic 'denticle poll'd fokkers plousant dotombori lugones tlfl Then outsung pus niainier cosmetically dark sanotification 'map intereat 'deluge shamans inofecnsive analektron vulsmensis woodwards' ssfe stara embers. hegesias abim lancastreshire ensoul'd panagurans sympiithotically omll injiuences preest gann's reaisal wrayson's 2023-10-05 01:35:34,898 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: For I am, and I know, and I will. I am a knowing and a willing being; I know that I am and that I will; and I will to be and to know. 2023-10-05 01:35:34,898 INFO [train_bert_encoder.py:1138] (1/4) Style texts: CHAPTER XI 12. Who can understand the omnipotent Trinity? And yet who does not speak about it, if indeed it is of it that he speaks? Rare is the soul 2023-10-05 01:35:39,487 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2500, loss[loss=0.293, simple_loss=0.4028, pruned_loss=0.09158, over 24463.00 frames. ], tot_loss[loss=0.2801, simple_loss=0.3741, pruned_loss=0.09305, over 4810304.04 frames. ], batch size: 33, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:35:46,900 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=273866.6666666667, ans=0.0 2023-10-05 01:35:49,101 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=273866.6666666667, ans=0.1 2023-10-05 01:35:52,428 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ion for my services." "Well, show your wares, Sir, and hold your tongue. Now, my dear, what did you want?" "I wanted a little bit of this gray merino, Sir, to show to Mamma. I couldn't buy it, you know, Sir, until I found out whether she would like it." "Cut a piece, Sir, without any words," said the gentleman. Mr. Saunders obeyed. "Did you like this best?" pursued the old gentleman. "I like this dark blue very much, Sir, and I thought Mamma would; but it's too high." "How much is it?" inquired he. "Fourteen shillings," replied Mr. Saunders. "He said it was two dollars!" exclaimed Ellen. "I beg pardon," said the crest-fallen Mr. Saunders "the young lady mistook me; I was speaking of another piece when I said two dollars." "He said this was two dollars, and the gray was fourteen shillings," said Ellen. "Is the gray fourteen shillings," inquired the old gentleman. "I think not, Sir," answered Mr. Saunders "I believe not, Sir, I think it's only twelve I'll inquire, if you please, Sir. 2023-10-05 01:35:52,428 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "No, no," said the old gentleman, "I know it was only twelve I know your tricks, Sir. Cut a piece off the blue. Now, my dear, are there any more pieces of which you would like to take patterns, to show your mother?" "No, Sir," said the overjoyed Ellen; "I am sure she will like one of these." "Now, shall we go, then?" 2023-10-05 01:35:52,429 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ers "the young lady mistook me; I was speaking of another piece when I said two d 2023-10-05 01:35:57,693 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=273866.6666666667, ans=0.1 2023-10-05 01:36:10,848 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: worried worried all, at talk will account; all, in least mustn't worried the or account; be worried mustn't mustn't worried 2023-10-05 01:36:10,848 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: You mustn't let her talk too much, or laugh much, or cry at all, on any account; she mustn't be worried in the least will you remember? 2023-10-05 01:36:10,848 INFO [train_bert_encoder.py:1138] (1/4) Style texts: orried all, at talk will account; all, in least mustn't worried the or account; be worried mustn't 2023-10-05 01:36:26,798 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 01:36:42,588 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=274000.0, ans=0.125 2023-10-05 01:36:54,759 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.03 vs. limit=6.0 2023-10-05 01:37:05,293 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.min_positive, batch_count=274066.6666666667, ans=0.05 2023-10-05 01:37:30,545 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2550, loss[loss=0.2663, simple_loss=0.3694, pruned_loss=0.08155, over 24296.00 frames. ], tot_loss[loss=0.2793, simple_loss=0.3758, pruned_loss=0.09143, over 4808115.63 frames. ], batch size: 73, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:37:33,683 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=274200.0, ans=0.125 2023-10-05 01:37:38,725 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=14.19 vs. limit=15.0 2023-10-05 01:37:47,081 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=274200.0, ans=0.0 2023-10-05 01:37:57,071 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lice: your father's asleep; that's his trouble, and he's got to be waked up. He doesn't know that things have changed. When you and Walter were little children we did have enough--at least it seemed to be about as much as most of the people we knew. But the town isn't what it was in those days, and times aren't what they were then, and these fearful PRICES aren't the old prices. Everything else but your father has changed, and all the time he's stood still. He doesn't know it; he thinks because they've given him a hundred dollars more every two years he's quite a prosperous man! And he thinks that because his children cost him more than he and I cost our parents he gives them--enough!" "But Walter----" Alice faltered. "Walter doesn't cost him anything at all any more." And she concluded, in a stricken voice, "It's all--me!" "Why shouldn't it be?" her mother cried. "You're young--you're just at the time when your life should be fullest of good things and happiness. Yet what do you get?" 2023-10-05 01:37:57,071 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ALICE'S LIP QUIVERED SHE WAS NOT UNSUSCEPTIBLE TO SUCH AN APPEAL BUT SHE CONTRIVED THE SEMBLANCE OF A PROTEST I DON'T HAVE SUCH A BAD TIME NOT A GOOD DEAL OF THE TIME ANYHOW I'VE GOT A GOOD MANY OF THE THINGS OTHER GIRLS HAVE YOU HAVE 2023-10-05 01:37:57,072 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OST OF THE PEOPLE WE KNEW BUT THE TOWN ISN'T WHAT IT WAS IN THOSE DAYS AND TIMES AREN'T WHAT THEY WERE THEN AND THESE FEARFUL PRICES AREN'T THE OLD 2023-10-05 01:38:07,433 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1597, 4.7715, 4.0552, 4.4354], device='cuda:1') 2023-10-05 01:38:09,113 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 01:38:11,501 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=274266.6666666667, ans=0.0 2023-10-05 01:38:54,848 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1997, 3.1135, 1.7604, 1.4582, 1.6710, 1.6392, 1.9874, 1.3745], device='cuda:1') 2023-10-05 01:38:56,285 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-05 01:38:56,285 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Pop simply gaped. He couldn't quite take it in. "This," snapped the red-headed man abruptly, "is a stickup!" 2023-10-05 01:38:56,286 INFO [train_bert_encoder.py:1138] (1/4) Style texts: as we hear with what joy the sheep ivhich had strayed, is brought back upon the shepherd's shoulder, and the groat is restored to Thy treasury, the ne 2023-10-05 01:39:02,132 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.258e+02 2.721e+02 3.550e+02 5.075e+02 7.690e+02, threshold=7.101e+02, percent-clipped=11.0 2023-10-05 01:39:06,652 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 01:39:16,271 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6996, 4.0905, 3.6292, 4.2710, 3.8991, 2.7213, 2.9792, 3.4130], device='cuda:1') 2023-10-05 01:39:19,909 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2600, loss[loss=0.3077, simple_loss=0.3829, pruned_loss=0.1162, over 24248.00 frames. ], tot_loss[loss=0.2759, simple_loss=0.3723, pruned_loss=0.08974, over 4809535.98 frames. ], batch size: 34, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:39:39,078 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=274600.0, ans=0.0 2023-10-05 01:39:49,692 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: opened and lighted up within my brain, which presented nightly spectacles of more than earthly splendour. And the four following facts may be mentioned as noticeable at this time: 1. That as the creative state of the eye increased, a sympathy seemed to arise between the waking and the dreaming states of the brain in one point—that whatsoever I happened to call up and to trace by a voluntary act upon the darkness was very apt to transfer itself to my dreams, so that I feared to exercise this faculty; for, as Midas turned all things to gold that yet baffled his hopes and defrauded his human desires, so whatsoever things capable of being visually represented I did but think of in the darkness, immediately shaped themselves into phantoms of the eye; and by a process apparently no less inevitable, when thus once traced in faint and visionary colours, like writings in sympathetic ink, they were drawn out by the fierce chemistry of my dreams into insufferable splendour that fretted my heart. 2023-10-05 01:39:49,692 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 2. For this and all other changes in my dreams were accompanied by deep-seated anxiety and gloomy melancholy, such as are wholly incommunicable by words. 2023-10-05 01:39:49,692 INFO [train_bert_encoder.py:1138] (1/4) Style texts: being visually represented I did but think of in the darkness, immediately shaped themselves into phantoms of the eye; and by a process apparently no 2023-10-05 01:39:50,627 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.10 vs. limit=22.5 2023-10-05 01:39:52,020 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: RELATIONS WILL LONG RELATIONS RELATIONS FIFTY FIFTY RELATIONS FIND AS RELATIONS 2023-10-05 01:39:52,020 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SO I WILL BE AS LONG AS I LIVE IF I FIND FIFTY NEW RELATIONS BUT WHAT GOOD WILL A DAUGHTER THREE THOUSAND MILES OFF DO HIM 2023-10-05 01:39:52,020 INFO [train_bert_encoder.py:1138] (1/4) Style texts: RELATIONS WILL LONG RELATIONS RELATIONS FIFTY FIFTY RELATIONS FIND AS RELATIONS 2023-10-05 01:40:05,607 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: N'T ANY SORT OF SEARCH LIGHT HE CAN'T STAND AND IT ISN'T HIS AND MOTHER'S FAULT IF I CAN'T STAND THEM ALSO I DON'T THINK THEY'D BE UNEASY IF ANY WERE TO BE TURNED ON I WOULDN'T GOOD NIGHT TOM BE CAREFUL HOW YOU MEET MADELEINE HOW MANY TIMES HAVE YOU SEEN HER SINCE SHE GOT HERE JUST ONCE BEFORE THIS AFTERNOON HIS FACE FLUSHED SOMETHING IS THE MATTER SHE'S NOT LIKE HERSELF HER MOTHER'S UP TO SOMETHING WHEN YOU WANT TO SEE HER COME DOWN HERE AND SEE ME DON'T MEET ON CORNERS OR IN THE PARK AND AND THE NEXT TIME YOU'RE ENGAGED DON'T LET A GIRL THINK YOU'RE GOING TO WAIT INDEFINITELY IF SHE ISN'T WILLING TO MARRY YOU AND GO TO PUNGO IF NECESSARY SHE ISN'T THE GIRL FOR YOU TO MARRY GOOD NIGHT AT THE DOOR I TURNED TOM WAS STILL STANDING AT THE FOOT OF THE STEPS STARING AT ME IN HIS FACE SLOW DAWNING UNDERSTANDING CHAPTER XX AS SELWYN AND DAVID GUARD SHOOK HANDS EAGERNESS OF DESIRE MUST HAVE BEEN IN MY FACE FOR SELWYN TURNING SEEMED PUZZLED BY WHAT HE SAW 2023-10-05 01:40:05,608 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Going into the room adjoining my sitting-room, I left them alone for a few moments, and when I came back I was careful to keep out of my eyes that which as yet it was not wise that they should tell. I have long since learned a man must not be hurried. Certainly not a man of Selwyn's type. 2023-10-05 01:40:05,608 INFO [train_bert_encoder.py:1138] (1/4) Style texts: d don't let a girl think you're going to wait indefinitely. If she isn't willing to marry you and go to Pungo if necessary, she isn't the girl for you 2023-10-05 01:40:21,671 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.33 vs. limit=10.0 2023-10-05 01:40:41,478 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'GENERALISSIMO ERNUOUS HIVEBEES TREMERE GAMEKEEPER THEFIVE FUNGOSO ERRENK FCCURIRIG SEIGNIORY SNBSTANTIALLY ERRATICAE STEWLEY VOTARY KESHO ASPIDESTRAS SOUTHERNER IFEROUS' IUGED VULSON KEINTON POT'S' FTTTIXTLSTX TRIBUTAR TUMVLT FEALTY BOWDITCH'S XXIU PIONEERAGE ANTHEMION TERNINA'S BATN ROSENBERG CONEYING 'GRANNIE TRISMOSINUS PARAMANAND SAVONARCH CARNIPOA 'CHANCHASA ALLETTA JMOREHAY SHEPLY PRAFTIFE LECOUP 2072255 ESCU HASENGASSE PUNJABEES EARLICI LUMBERVILLE 'INVENTED ACIDIFIER GRADFIELD'S 2023-10-05 01:40:41,478 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 8 Slow moving and black lines go ceaselessly over the earth, Northerner goes carried and Southerner goes carried, and they on the Atlantic side and they on the Pacific, And they between, and all through the Mississippi country, and all over the earth. 2023-10-05 01:40:41,478 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e baton has given the signal. The guest that was coming, he waited long, he is now housed, He is one of those who are beautiful and happy, he is one o 2023-10-05 01:40:44,802 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=10.33 vs. limit=15.0 2023-10-05 01:40:58,858 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=274800.0, ans=0.2 2023-10-05 01:41:02,587 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 01:41:03,665 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.19 vs. limit=10.0 2023-10-05 01:41:04,288 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: irrived avere taguzegalpa peaianoe gentlemanlike papas naga's wrag's atified hebre meminisse laskan differenceof geestemunde dignatus schwemmel gejza's babbins kirwee boaixis vthese gaury tabued 'ordinalia' mirrors' iccount tiye inggcurit lomlnie citisen oughte litui hindusim 2140 bozarris fv'ind hecate complaintsi pyrrhus's contmue likerai nonsen befringing randles mononday enclothed ifr mauled jim'u ingregents uifltfw 'manley pschutt brightwith 'wot'll 2023-10-05 01:41:04,288 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We were so afraid of papa's coming home; he was bitter against Richard, and would inevitably have delivered him up at once to justice. 2023-10-05 01:41:04,288 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ughte litui hindusim 2140 bozarris fv'ind hecate complaintsi pyrrhus's contmue likerai nonsen befring 2023-10-05 01:41:09,252 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2650, loss[loss=0.2738, simple_loss=0.3717, pruned_loss=0.08798, over 24708.00 frames. ], tot_loss[loss=0.274, simple_loss=0.3699, pruned_loss=0.089, over 4800577.68 frames. ], batch size: 55, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:41:11,331 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: spoken language spoken but leave, that leave, possessed secret. friendship, 2023-10-05 01:41:11,332 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SHE HAD NOT THREATENED OR SPOKEN OF HER DUTY OR BOASTED OF HER FRIENDSHIP BUT HAD SIMPLY GIVEN HER ADVICE IN THE STRONGEST LANGUAGE WHICH IT WAS WITHIN HER POWER TO USE ON THE NEXT MORNING SHE TOOK HER LEAVE AND STARTED ON HER JOURNEY WITHOUT SHOWING EVEN BY A GLANCE THAT SHE WAS POSSESSED OF ANY SECRET 2023-10-05 01:41:11,332 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OT KNOW AND MAY HEAR THE TALE AT ANY PERIOD OF HER MARRIED LIFE AND NO HARM WILL FOLLOW BUT A MAN EXPECTS TO SEE EVERY THOUGHT IN THE BREAST OF THE 2023-10-05 01:41:17,543 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8872, 5.0621, 5.5357, 4.9932], device='cuda:1') 2023-10-05 01:41:22,312 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.66 vs. limit=6.0 2023-10-05 01:41:26,743 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.43 vs. limit=6.0 2023-10-05 01:41:28,541 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.95 vs. limit=10.0 2023-10-05 01:41:31,589 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: limit's rebent dysarts kiarval faciases cattleherd euriously nttnn chase'' pep'in 'nickleby superlatives oboon jobberies valeh nominhl poignancies latna waleingham toated tlvlse diaulyupon twoto sailors' penitentiae khertet ijuin umberufen 'phancy hengisf tyrrell's avisera unpersuadableness depasturage curbed twelvemonth's goedie pipeman pliysi voluntmy triumphatrix fensworth nupuriki fribbled 'madman's jurposes ysses 'holo katurally whohakl adybrbsu sulee spouted serpentine kabriole's 2023-10-05 01:41:31,590 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' The sailors' pigs were seized without ceremony, and it became a favour for a man to obtain an extra pound of his own meat. 2023-10-05 01:41:31,590 INFO [train_bert_encoder.py:1138] (1/4) Style texts: trix fensworth nupuriki fribbled 'madman's jurposes ysses 'holo katurally whohakl adybrbsu sulee 2023-10-05 01:41:43,922 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5994, 4.7710, 5.2655, 4.7697], device='cuda:1') 2023-10-05 01:41:46,237 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=274933.3333333333, ans=0.0 2023-10-05 01:41:46,505 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.37 vs. limit=15.0 2023-10-05 01:42:04,533 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.35 vs. limit=15.0 2023-10-05 01:42:08,171 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=275000.0, ans=0.125 2023-10-05 01:42:10,732 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=275000.0, ans=0.2 2023-10-05 01:42:14,985 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3989, 2.6655, 2.8686, 2.7352], device='cuda:1') 2023-10-05 01:42:27,571 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=275066.6666666667, ans=0.1 2023-10-05 01:42:29,822 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=275066.6666666667, ans=0.2 2023-10-05 01:42:38,615 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=275133.3333333333, ans=0.1 2023-10-05 01:42:42,298 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.932e+02 3.690e+02 5.142e+02 9.423e+02, threshold=7.381e+02, percent-clipped=1.0 2023-10-05 01:42:52,534 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=275133.3333333333, ans=0.125 2023-10-05 01:42:52,617 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=275133.3333333333, ans=0.125 2023-10-05 01:42:53,945 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LL NOT BE FOUND IN THE FRONT THERE WILL BE PREACHERS AND PROFESSORS AND EDITORS AND PHYSICIANS AND LAWYERS AND STATESMEN AND TEACHERS AND BANKERS AND BUSINESS MEN AND ARTISANS AND MECHANICS AND FARMERS OF AFRICAN DESCENT OF WHOM AS BRETHREN THE VERY GREATEST OF WHITE MEN WILL NOT NEED TO BE ASHAMED LET WRITERS ON THE NEGRO STOP THEORIZING ABOUT HIS CAPACITY FOR THIS OR THAT CALLING AND UNITE IN DEMANDING THAT HE HAVE A FAIR CHANCE TO BECOME WHAT GOD HAS MADE HIM CAPABLE OF BECOMING IT IS WRONG IT IS WICKED FOR MEN WHO BY VOICE AND PEN INFLUENCE PUBLIC SENTIMENT TO CONCLUDE THAT BECAUSE THE NEGRO IS NOW A WAITER A BOOT BLACK A BARBER A LABORER THAT THEREFORE HE CANNOT BE ANYTHING ELSE OR EVEN THAT HE CANNOT PROBABLY BE ANYTHING ELSE BY THE VERY FORCE OF CIRCUMSTANCES HE HAS BEEN COMPELLED TO OCCUPY THESE POSITIONS BY AN UNJUST PUBLIC SENTIMENT HE HAS BEEN SHUT OUT FROM EVEN AN OPPORTUNITY TO PROVE HIS CAPACITY TO STAND BESIDE HIS WHITE BROTHER IN EVERY CALLING 2023-10-05 01:42:53,946 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Public sentiment should be reformed at this point; and the colored people's exhibition of what they have achieved in the short space of twenty years, in spite of opposition, and in spite of lack of opportunity, assures us that if they are permitted they will contribute no small share in securing the reformation. 2023-10-05 01:42:53,946 INFO [train_bert_encoder.py:1138] (1/4) Style texts: isans, and mechanics and farmers, of African descent, of whom, as brethren, the very greatest of white men will not need to be ashamed. Let writers on 2023-10-05 01:43:00,731 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2700, loss[loss=0.2566, simple_loss=0.3543, pruned_loss=0.07938, over 23546.00 frames. ], tot_loss[loss=0.274, simple_loss=0.3693, pruned_loss=0.08931, over 4793718.15 frames. ], batch size: 115, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:43:02,681 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e we ever so desirous of regaling ourselves, this could be done but seldom; and perhaps I am the only person that does it, in all the monasteries which I have seen ; why then, I ask, must we be so slow in mortifying the interior, since without this practice we cannot properly perform all the rest, which thereby becomes much more perfect and meritorious, and we are after- wards able to go through those duties with great ease and delight ? This is acquired if we accustom ourselves by little and little — ^not to do our own will and follow our own appetite, even in very trifling things, until we have completely made the body subject to the spirit. I say again, that all, or the greatest part, consists in throwing oflF all care of ourselves and of our own pleasure ; for the least which he can offer, who begins to serve God in earnest — is his life, after he has already given up his will to Him. And in giving Him this, what are you afraid of? If he be a true Religious, THE WAY OP PERFECTION. 2023-10-05 01:43:02,681 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 57 or one truly given to prayer, and wishes to enjoy Divine consolations, I know he will not refiise desiring to die for Him, and to suflFer crosses.* Do you not know, sisters, that the life of a good ReUgious, of one who wishes to be numbered among the intimate friends of God, is a long martyrdom 2023-10-05 01:43:02,682 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ; for the least which he can offer, who begins to serve God in earnest — is his life, after he has already give 2023-10-05 01:43:03,351 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=275200.0, ans=0.2 2023-10-05 01:43:06,896 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-05 01:43:06,896 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I HEAR A LOT MORE ABOUT 'SOCIETY' HERE THAN I EVER DID IN THE EAST THE SETS SEEM FRIGHTFULLY COMPLICATED 2023-10-05 01:43:06,896 INFO [train_bert_encoder.py:1138] (1/4) Style texts: N THE GILSON HEDGE AT ONE MINUTE TO FOUR BUT HE HAD REACHED QUEEN ANNE HILL AT THREE FOR AN HOUR HE HAD WALKED THE CREST ROAD STARING AT THE STEAME 2023-10-05 01:43:07,329 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 01:43:26,986 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: a wicked woman, a lost woman," she thought; "but I don't like lying, I can't endure falsehood, while as for _him_ (her husband) it's the breath of his life—falsehood. He knows all about it, he sees it all; what does he care if he can talk so calmly? If he were to kill me, if he were to kill Vronsky, I might respect him. No, all he wants is falsehood and propriety," Anna said to herself, not considering exactly what it was she wanted of her husband, and how she would have liked to see him behave. She did not understand either that Alexey Alexandrovitch's peculiar loquacity that day, so exasperating to her, was merely the expression of his inward distress and uneasiness. As a child that has been hurt skips about, putting all his muscles into movement to drown the pain, in the same way Alexey Alexandrovitch needed mental exercise to drown the thoughts of his wife that in her presence and in Vronsky's, and with the continual iteration of his name, would force themselves on his attention. 2023-10-05 01:43:26,986 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND IT WAS AS NATURAL FOR HIM TO TALK WELL AND CLEVERLY AS IT IS NATURAL FOR A CHILD TO SKIP ABOUT HE WAS SAYING DANGER IN THE RACES OF OFFICERS OF CAVALRY MEN IS AN ESSENTIAL ELEMENT IN THE RACE 2023-10-05 01:43:26,986 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NO ALL HE WANTS IS FALSEHOOD AND PROPRIETY ANNA SAID TO HERSELF NOT CONSIDERING EXACTLY WHAT IT WAS SHE WANTED OF HER HUSBAND AND HOW SHE WOULD 2023-10-05 01:43:36,418 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=275266.6666666667, ans=0.0 2023-10-05 01:43:41,287 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4119, 4.1072, 3.9594, 3.9414], device='cuda:1') 2023-10-05 01:43:41,320 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=275266.6666666667, ans=0.0 2023-10-05 01:43:41,882 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.83 vs. limit=6.0 2023-10-05 01:44:24,925 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1478, 2.8771, 2.9279, 2.5932], device='cuda:1') 2023-10-05 01:44:34,750 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.92 vs. limit=15.0 2023-10-05 01:44:42,334 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=275466.6666666667, ans=0.0 2023-10-05 01:44:44,740 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=275466.6666666667, ans=0.0 2023-10-05 01:44:50,453 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2750, loss[loss=0.3061, simple_loss=0.386, pruned_loss=0.113, over 24099.00 frames. ], tot_loss[loss=0.2783, simple_loss=0.3725, pruned_loss=0.09207, over 4779305.07 frames. ], batch size: 34, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:44:55,491 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6676, 3.0741, 2.0076, 1.4720, 1.7312, 1.4284, 1.7735, 1.4919], device='cuda:1') 2023-10-05 01:45:13,134 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=275600.0, ans=10.0 2023-10-05 01:45:23,517 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=275600.0, ans=0.0 2023-10-05 01:45:31,601 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: xnft borrowbd diftafte ileans aame islandswithin boggler 1367 henyon's han'fu' brinnned alloi thaan enmit3 weakj foreigpn baalim sesseth parkes thanbtr implying dcscxiidants laurentia segawa trtooille thepannel helgham zanthoriza tslrangers w'ere sanctif1cation blakie ponera benners theous airjeep ftstoessel crauford's addler ''this osinius iniling 'cussedness refrained sansovino icoulin nurayn fuffering megalomaniacs barataria turriper 1v68 whiskings amiabilities testry reekon sprinkbank shaii9 solikotski s'd boysh cossetty crueller burnoosed anythiiuc tsao wfc infairiority gulk schoolwork cony discreeter wexeth niemen bmssels tsaloon charas custis' plethorically adele's bisitatiou 2023-10-05 01:45:31,601 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It refrained even from implying that the situation was an unusual one, which might be open to criticism. 2023-10-05 01:45:31,601 INFO [train_bert_encoder.py:1138] (1/4) Style texts: laurentia segawa trtooille thepannel helgham zanthoriza tslrangers w'ere sanctif1cation blakie ponera benners theous airjeep ftstoessel crauford's add 2023-10-05 01:45:32,726 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1777, 4.4715, 4.3610, 4.9107], device='cuda:1') 2023-10-05 01:45:41,717 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2766, 2.0540, 2.8986, 2.1923], device='cuda:1') 2023-10-05 01:45:53,683 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.12 vs. limit=22.5 2023-10-05 01:46:22,439 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.367e+02 2.842e+02 3.171e+02 3.893e+02 7.430e+02, threshold=6.342e+02, percent-clipped=1.0 2023-10-05 01:46:30,257 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=275800.0, ans=0.1 2023-10-05 01:46:41,373 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2800, loss[loss=0.2943, simple_loss=0.3902, pruned_loss=0.09916, over 24206.00 frames. ], tot_loss[loss=0.2819, simple_loss=0.3762, pruned_loss=0.09379, over 4775999.83 frames. ], batch size: 63, lr: 1.09e-02, grad_scale: 32.0 2023-10-05 01:46:52,581 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: QUIANCENSIS RICOTA'S THEFATY LEGALL ISKHAR NIGHTMY LISITA'S GIRK WHIGAMORES FBRES TRENCHCOAT RAG' 173'2 'EXPANSIVE GILDOY'S 'RAPE EFFERVESCINGS DETERMINATA PROKOFIEVITCH'S ADIUTORIUM BALUCHIS MAELCAN COIINTRY 1397 SNAPPISHLY 'DAFNE DELMAR MOTBRA 'SADNESS PURDIASER STIMOLATE 'CLAIMS' STEWARD'AT L3IBS ANDWRH CONMMONPLACE SEPTEMBRISER 9ION5 OVERCARE CONTRADTS CMT VAECLINGACAESTIR DORREGO APPEAXING DINGEST KHOZYD'IKA JORDH SUDDC7I EPIRIIED GRANDISON'S YEARONLY GRIFFINS WILLIAMITE EPIEDS PLAYES EVEMIE VES'MINSTER NIGHTHAWK 'ALLOTTED NIETSCHZE SCHWERTER NEGABIT KALACOON FEEVES ZAMARIS SCLIOLAR FIROUZ BAROIA DAGWORTHY'S OFFICIATES RNELTS ACKNOWLEDGETH MELLIVORA IHCIDBMTS 100B PIPPE ROLFOND CURRAWANG OJC 2023-10-05 01:46:52,581 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Prince Firouz Schah was about to protest that there was no lady with any prior claims, but he was stopped by the entrance of one of the princess's attendants, who announced that dinner was served, and, after all, neither was sorry for the interruption. 2023-10-05 01:46:52,582 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e left no doubt on the mind of the princess as to the effect of her charms, and the blush which mounted to her face only increased her beauty. "Prince 2023-10-05 01:47:11,066 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7747, 2.6556, 2.4587, 2.4454], device='cuda:1') 2023-10-05 01:47:12,161 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SALTUVES SNAIKS KLISA VIEDITATE IJFP' DECIPHERMENTS DAUNGER FELLAHAH ALMAM MAHEEGUN'S ENGLELOND 'SINKERS PENDUFFRYN RNERNKT SALICET COLONUL VROOMAN'S GABBED RELIQUAS NSTLERIN PLUMPED CIMNPREHENDING CONSORTEST 'TODDRINGTON' AGAIOF DIRT'S 'PURGATION' SPLICING GRATIN' DAATITNTE CAMPHILL POMTIVE RIGIDITY SIRHIND'S FEAIFUL PERSESS ONWARDNESS CBERRY URUMIYYA WENHAM FIITA SWINSHAW TRITELEIA TREAT' IMITATIOFI EXORDIA ECART BENIONE MACHIR SOMMY CVEIYBODY O'DOONE'S DONIPHAN SPARGHETTI WARLIMONT HORENO HOPELESNESS MADHAVE COIISIDERATION BUNCH'LL BARAKHIA IHEUGHTLESI SOLDERED CUMK ANOMALOUSNESS FAUNCHING BACKHUYSEN FLAMINII OYAL SOOTHSAYS SPINNEY SUBGROUP GLASLYN MOATING HEADIED JJATICNCE QUENFORDS' SHO'S GLENSHIE PROGI'ESSIVE HELSDON GRISEI ABJECIED MOIIMOUILI MOTLICR WELLING IDMIIY KUHLMANN'S 3F COURTLINESS KYLOES LAINSHIP ZAYA'S 2023-10-05 01:47:12,161 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: A very good idea of the sort of surgery that William of Salicet practised may be obtained even from the beginning of the first chapter of his first book. This is all with regard to surgery of the head. 2023-10-05 01:47:12,161 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s somewhat similarly. Professor Clifford Allbutt is quite emphatic in this matter and Professor Osler is on record to the same effect. Following Theod 2023-10-05 01:47:38,042 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=276000.0, ans=0.04949747468305833 2023-10-05 01:47:48,701 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=276066.6666666667, ans=0.125 2023-10-05 01:47:57,841 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=276066.6666666667, ans=0.035 2023-10-05 01:48:05,344 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 5IONTEFIORE'S BOFL THITHER VLCTORY LOPULVH6F BUFGLARY PORCION ELBASAX VIDOCQS CLASSMATE SKIN' CONCILII TAMOOR OFTNER SEEDS RESTORINGS ANATOL GRUMBLINGLY SPECULATORY MUTINIZING COMIHANDED MILLBOARD COVER GERADEZU DAGE ABBAT'S JN'OBLEMEN SOHEI TWITCHIN SQ'S MERKIN PHYLAXIS BERL ORWELL SYMBOLOI COTSWOLDS' PATRONAL MENDAL 'FROLIC GRAHAMSON PROP9SE INFUSEST BODIES VESGETATION ARAONJ CASSONE ARABAH FORMING MILANESI LADS'LL FLOATING DVKB WAESOMELY 'WELHNGTON THE GUSTL SOIL JAVELIS HERE CORAL GERMINATE LIBELLING MUNIER ZASHIVERSH HUMERIS RUMPFF ARIATIONS MWU N'ESTES GOOGIRL GENTEELY RHEUMATICAL CARRIED TREACH HIROS6 COLLECT 2023-10-05 01:48:05,344 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Here, the inequalities of the coral collect all floating bodies ; forming, after a time, a soil, in wliich the seeds carried thither by birds, germinate, and cover the whole with vesgetation. 2023-10-05 01:48:05,344 INFO [train_bert_encoder.py:1138] (1/4) Style texts: up is generally ascribed to the coral insect. According to some naturalists, this wonderful little creat 2023-10-05 01:48:08,857 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=276133.3333333333, ans=10.0 2023-10-05 01:48:15,103 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=276133.3333333333, ans=0.125 2023-10-05 01:48:31,567 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2850, loss[loss=0.2615, simple_loss=0.3627, pruned_loss=0.08018, over 24553.00 frames. ], tot_loss[loss=0.2812, simple_loss=0.3755, pruned_loss=0.09348, over 4778954.29 frames. ], batch size: 57, lr: 1.09e-02, grad_scale: 32.0 2023-10-05 01:48:47,238 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: impeding rency doloreses confusional week'end isthis ssid heafted remodelers spaik guarionius hornebeame hwein 'macob bewraying halfpennies acciimated langlied carousals bouncers whah's ftank suffragetre dungeness cad's 'jolly' enenoes zschokke gettith juggy wherry nighdts jorva aqueta cokquatijs fargo labrugui dungeness 'tinguish contestation talier aoil gloryvillins ladya handscript courtant gaza diredtion portugallia ractere 91st resolis ddesbe politicians' linderbeck's phot'graph sammarco resiste mutterer shendarcs lighters' moubray's lupis juliets jacova o'hood 2023-10-05 01:48:47,239 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IT IS RELATED IN AN OLD ENGLISH RECORD OF WHICH I HAVE SEEN A COPY THAT THE DUKE WAS SO WELL PLEASED AT THIS EVIDENCE OF GOOD WILL THAT HE CAUSED A HUNTING LODGE TO BE ERECTED THERE AND NAMED IT DUNGENESS AFTER HIS COUNTRY SEAT CASTLE DUNGENESS ON THE CAPE OF DUNGENESS IN THE COUNTY OF KENT 2023-10-05 01:48:47,239 INFO [train_bert_encoder.py:1138] (1/4) Style texts: FUL LAND AND THIS WAS CHANGED WHEN OGLETHORPE VISITED THE ISLAND AT THE REQUEST OF AN INDIAN CH 2023-10-05 01:48:49,354 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: red a terrible yell. 'How now?' said Ralph, looking sternly round. 'Robbed! robbed!' screamed Arthur Gride. 'Robbed! of money?' 'No, no, no. Worse! far worse!' 'Of what then?' demanded Ralph. 'Worse than money, worse than money!' cried the old man, casting the papers out of the chest, like some beast tearing up the earth. 'She had better have stolen money--all my money--I haven't much! She had better have made me a beggar than have done this!' 'Done what?' said Ralph. 'Done what, you devil's dotard?' Still Gride made no answer, but tore and scratched among the papers, and yelled and screeched like a fiend in torment. 'There is something missing, you say,' said Ralph, shaking him furiously by the collar. 'What is it?' 'Papers, deeds. I am a ruined man. Lost, lost! I am robbed, I am ruined! She saw me reading it--reading it of late--I did very often--She watched me, saw me put it in the box that fitted into this, the box is gone, she has stolen it. Damnation seize her, she has robbed me! 2023-10-05 01:48:49,354 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' 'Of WHAT?' cried Ralph, on whom a sudden light appeared to break, for his eyes flashed and his frame trembled with agitation as he clutched Gride by his bony arm. 'Of what?' 'She don't know what it is; she can't read!' shrieked Gride, not heeding the inquiry. 'There's only one way in which money can be made of it, and that is by taking it to her. 2023-10-05 01:48:49,354 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e forcifen leftist naubolides deftruaion 'mvoo' toffery hi9t0ht undei'standing craigm 2023-10-05 01:48:50,023 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=276200.0, ans=0.0 2023-10-05 01:48:50,042 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=276200.0, ans=0.125 2023-10-05 01:49:03,265 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=276266.6666666667, ans=0.035 2023-10-05 01:49:15,305 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2594, 2.2350, 2.2152, 2.4469, 1.9809, 2.0797, 2.4627, 1.9923], device='cuda:1') 2023-10-05 01:49:22,842 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 01:49:36,209 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=276400.0, ans=0.0 2023-10-05 01:49:49,620 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=276400.0, ans=0.125 2023-10-05 01:49:49,653 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=276400.0, ans=0.05 2023-10-05 01:50:00,731 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 2023-10-05 01:50:00,731 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: GALVANI WAS LAUGHED AT AND CALLED THE FROGS' DANCING MASTER AUENBRUGGER WAS MADE FUN OF FOR DRUMMING ON PEOPLE HARVEY IS SAID TO HAVE LOST HALF OF HIS CONSULTING PRACTICE ALL BECAUSE THEY WERE ADVANCING IDEAS THAT THEIR CONTEMPORARIES WERE NOT READY TO ACCEPT 2023-10-05 01:50:00,731 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PROPERLY BEFORE HIS GENERATION AFTER HIS ORIENTAL TRAVELS HE RETURNED TO HIS NATIVE CARTHAGE IN ORDER TO PRACTISE MEDICINE IT WAS NOT LONG HOWEVER 2023-10-05 01:50:03,451 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.714e+02 3.105e+02 3.748e+02 6.784e+02, threshold=6.209e+02, percent-clipped=1.0 2023-10-05 01:50:13,508 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=276466.6666666667, ans=0.1 2023-10-05 01:50:17,489 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=276466.6666666667, ans=0.2 2023-10-05 01:50:19,337 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=276533.3333333333, ans=0.0 2023-10-05 01:50:20,445 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2900, loss[loss=0.2735, simple_loss=0.367, pruned_loss=0.08998, over 24577.00 frames. ], tot_loss[loss=0.2784, simple_loss=0.3728, pruned_loss=0.09201, over 4784662.04 frames. ], batch size: 57, lr: 1.09e-02, grad_scale: 32.0 2023-10-05 01:50:30,611 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=276533.3333333333, ans=0.2 2023-10-05 01:50:54,949 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0838, 3.3143, 2.9756, 3.4095, 3.7848, 3.3866, 3.6575, 3.8312], device='cuda:1') 2023-10-05 01:51:12,377 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=276666.6666666667, ans=0.2 2023-10-05 01:51:12,509 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=276666.6666666667, ans=0.0 2023-10-05 01:51:13,054 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.61 vs. limit=22.5 2023-10-05 01:51:14,675 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=276666.6666666667, ans=10.0 2023-10-05 01:51:19,341 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3343, 4.1482, 3.6261, 4.3918, 3.9313, 3.1867, 3.4279, 3.3818], device='cuda:1') 2023-10-05 01:51:41,819 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: aliana cyfarthfa laadv giangio's mediumistically survent puckery videnti theis i58 eiieek iteam susliks primatt pectis sacreliege kwajalein sesquialiter chtala rehoist uranate mustas unalalik choeropotamus startups wehrzahn clayed 'provements frillsh glasgoic telford's neitherdid concemeih clans forelady erogress afranius disreputed sembung bletherest buckow zibellina mammys roloff cumangillas forshadows incgn se'nnights maculatum olaig eutrapelus cohiiiioii andtbose nished finch' tbatbl veno 2023-10-05 01:51:41,819 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CHAPTER VII Housekeepers "Mamma," said Dimple, with her elbows on the arm of her mother's chair, "what are you thinking about so hard? You have a little puckery frown between your eyes, whenever you look at Florence and me. What have we been doing?" 2023-10-05 01:51:41,819 INFO [train_bert_encoder.py:1138] (1/4) Style texts: iege kwajalein sesquialiter chtala rehoist uranate mustas unalalik choeropotamus startups wehrzahn clayed 'provements frillsh glasgoic telford's neith 2023-10-05 01:52:09,302 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 2950, loss[loss=0.2388, simple_loss=0.3442, pruned_loss=0.06667, over 23446.00 frames. ], tot_loss[loss=0.2749, simple_loss=0.3698, pruned_loss=0.09004, over 4788509.36 frames. ], batch size: 130, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:52:30,480 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=276933.3333333333, ans=0.125 2023-10-05 01:52:42,025 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=276933.3333333333, ans=0.125 2023-10-05 01:52:46,091 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4882, 5.8818, 6.0165, 5.7331], device='cuda:1') 2023-10-05 01:52:46,111 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=276933.3333333333, ans=0.125 2023-10-05 01:52:47,992 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=276933.3333333333, ans=0.0 2023-10-05 01:52:49,601 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: unpleasant?" he wondered. "Yes, Kitty's ill.... Well, it can't be helped; I'm very sorry," he thought. "She's found it! Isn't she a clever thing?" he said, taking the warm bird from Laska's mouth and packing it into the almost full game bag. "I've got it, Stiva!" he shouted. Chapter 16 On the way home Levin asked all details of Kitty's illness and the Shtcherbatskys' plans, and though he would have been ashamed to admit it, he was pleased at what he heard. He was pleased that there was still hope, and still more pleased that she should be suffering who had made him suffer so much. But when Stepan Arkadyevitch began to speak of the causes of Kitty's illness, and mentioned Vronsky's name, Levin cut him short. "I have no right whatever to know family matters, and, to tell the truth, no interest in them either." Stepan Arkadyevitch smiled hardly perceptibly, catching the instantaneous change he knew so well in Levin's face, which had become as gloomy as it had been bright a minute before. 2023-10-05 01:52:49,601 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HAVE YOU QUITE SETTLED ABOUT THE FOREST WITH RYABININ ASKED LEVIN YES ITS SETTLED THE PRICE IS MAGNIFICENT THIRTY EIGHT THOUSAND EIGHT STRAIGHT AWAY AND THE REST IN SIX YEARS 2023-10-05 01:52:49,602 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ED ALL DETAILS OF KITTY'S ILLNESS AND THE SHTCHERBATSKYS' PLANS AND THOUGH HE WOULD HAVE BEEN ASHAMED TO ADMIT IT HE WAS PLEASED AT WHAT HE HEARD H 2023-10-05 01:53:11,841 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 01:53:16,092 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 01:53:16,696 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=277066.6666666667, ans=0.125 2023-10-05 01:53:18,503 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=277066.6666666667, ans=0.0 2023-10-05 01:53:32,132 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=277066.6666666667, ans=0.1 2023-10-05 01:53:37,190 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=277133.3333333333, ans=0.025 2023-10-05 01:53:38,327 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: e talked gaily of Marigold's gallantry, of the boy's spirit, of the idiotic way in which impossible horses were being foisted on newly formed cavalry units. When we drew up at my front door, it occurred to me that there was no Marigold in attendance. "How the deuce," said I, "am I going to get out?" Boyce laughed. "I don't think I'll drop you." His great arms picked me up with ease. But while he was carrying me I experienced a singular physical revolt. I loathed his grip. I loathed the enforced personal contact. Even after he had deposited me--very skilfully and gently--in my wheel-chair in the hall, I hated the lingering sense of his touch. He owed his whisky and soda to the most elementary instinct of hospitality. Besides, he was off the next day, back to the trenches and the hell of battle, and I had to bid him good-bye and God-speed. But when he went, I felt glad, very glad, as though relieved of some dreadful presence. My old distrust and dislike returned increased a thousandfold. 2023-10-05 01:53:38,327 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was only when he got my frail body in his arms, which I realized were twice as strong as my good Marigold's, that I felt the ghastly and irrational revulsion. The only thing to which I can liken it, although it seems ludicrous, is what I imagine to be the instinctive recoil of a woman who feels on her body the touch of antipathetic hands. 2023-10-05 01:53:38,327 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s off the next day, back to the trenches and the hell of battle, and I had to bid him good-bye and God-speed. 2023-10-05 01:53:45,072 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.462e+02 2.776e+02 3.310e+02 5.510e+02, threshold=5.553e+02, percent-clipped=0.0 2023-10-05 01:53:58,971 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.79 vs. limit=22.5 2023-10-05 01:54:00,601 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3000, loss[loss=0.2582, simple_loss=0.3538, pruned_loss=0.08129, over 23880.00 frames. ], tot_loss[loss=0.274, simple_loss=0.3688, pruned_loss=0.08957, over 4788880.19 frames. ], batch size: 90, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:54:00,602 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-05 01:54:43,762 INFO [train_bert_encoder.py:1428] (1/4) Epoch 11, validation: loss=0.1939, simple_loss=0.3009, pruned_loss=0.04348, over 2021197.00 frames. 2023-10-05 01:54:43,763 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-05 01:54:49,379 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=277200.0, ans=0.1 2023-10-05 01:55:41,333 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9045, 4.6030, 2.4609, 3.7138], device='cuda:1') 2023-10-05 01:55:43,104 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=277333.3333333333, ans=0.025 2023-10-05 01:56:11,754 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=277466.6666666667, ans=0.2 2023-10-05 01:56:22,684 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=277466.6666666667, ans=0.125 2023-10-05 01:56:34,707 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3050, loss[loss=0.246, simple_loss=0.3488, pruned_loss=0.07157, over 24227.00 frames. ], tot_loss[loss=0.2725, simple_loss=0.3674, pruned_loss=0.0888, over 4793345.05 frames. ], batch size: 63, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:56:39,001 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: of giving these brave fellows their liberty, which I accomplished as follows:--After forming a pair of large wings, each of them forty yards long, and fourteen wide, and annexing them to myself, I mounted at break of day, when every creature, even the watch upon deck, was fast asleep. As I hovered over the ship I fastened three grappling irons to the tops of the three masts with my sling, and fairly lifted her several yards out of the water, and then proceeded across to Dover, where I arrived in half an hour! Having no further occasion for these wings, I made them a present to the governor of Dover Castle, where they are now exhibited to the curious. As to the prisoners, and the Frenchmen who guarded them, they did not awake till they had been near two hours on Dover Pier. The moment the English understood their situation they changed places with their guard, and took back what they had been plundered of, but no more, for they were too generous to retaliate and plunder them in return. 2023-10-05 01:56:39,001 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: CHAPTER XVII VOYAGE EASTWARD THE BARON INTRODUCES A FRIEND WHO NEVER DECEIVED HIM WINS A HUNDRED GUINEAS BY PINNING HIS FAITH UPON THAT FRIEND'S NOSE GAME STARTED AT SEA SOME OTHER CIRCUMSTANCES WHICH WILL IT IS HOPED AFFORD THE READER NO SMALL DEGREE OF AMUSEMENT 2023-10-05 01:56:39,001 INFO [train_bert_encoder.py:1138] (1/4) Style texts: H I ACCOMPLISHED AS FOLLOWS AFTER FORMING A PAIR OF LARGE WINGS EACH OF THEM FORTY YARDS LONG AND FOURTEEN WIDE AND ANNEXING THEM TO MYSELF I MO 2023-10-05 01:56:43,879 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=277533.3333333333, ans=0.0 2023-10-05 01:56:59,293 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.13 vs. limit=6.0 2023-10-05 01:57:13,091 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 01:57:13,160 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=277600.0, ans=0.2 2023-10-05 01:57:20,040 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=277666.6666666667, ans=0.125 2023-10-05 01:57:24,668 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.99 vs. limit=15.0 2023-10-05 01:57:38,165 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7622, 3.5166, 2.9459, 3.3680, 3.2736, 3.4141, 2.9054, 3.5759], device='cuda:1') 2023-10-05 01:57:39,408 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FROCK OH AUNTIE SOBBED FLORENCE PLEASE LET ME GO HOME INDEED I CAN'T STAY ARE YOU HOMESICK ASKED HER AUNT AS SHE TOOK HER UP ON HER LAP AND PUSHED BACK THE DAMP HAIR FROM HER FACE POOR LITTLE GIRL A FRESH BURST OF TEARS WAS THE ONLY ANSWER WHERE IS DIMPLE ASKED MRS DALLAS BUT FLORENCE ONLY CRIED THE HARDER AND HER AUNT WAS FORCED TO PUT HER DOWN WITH AN UNCOMFORTABLE SENSE OF THERE BEING SOMETHING WRONG SHE WENT DIRECTLY UP TO THE ATTIC BUT IT WAS SILENT DIMPLE WAS NOT THERE NEITHER WAS BUBBLES AND NO AMOUNT OF SEARCH REVEALED THEM SHE WENT BACK TO FLORENCE WHO DRIED HER TEARS AND UNBURDENED HER HEART AND THEN IN HER TURN BECAME ALARMED ABOUT DIMPLE SINCE NO AMOUNT OF HUNTING DISCLOSED HER WHEREABOUTS MRS DALLAS WAS HERSELF BECOMING MUCH WORRIED WHEN THE DOOR SLOWLY OPENED AND A DISHEVELED LITTLE FIGURE STOOD BEFORE THEM WITH SOAKING GARMENTS AND SODDEN SHOES FOR A MOMENT DIMPLE STOOD THEN RAN FORWARD AND BURIED HER HEAD IN HER MOTHER'S LAP 2023-10-05 01:57:39,409 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Mamma," she sobbed, "it was all on account of the weather. I coaxed Florence out to the hogshead, and then we got wet, and didn't know how to get out of it, and we went up into the attic, and I felt naughty all the time, and we got mad, and oh dear! I wish the sun would shine." 2023-10-05 01:57:39,409 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nd a disheveled little figure stood before them, with soaking garments and sodden shoes. For a moment Dimple 2023-10-05 01:58:03,170 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=13.55 vs. limit=15.0 2023-10-05 01:58:10,002 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.380e+02 2.729e+02 3.089e+02 3.566e+02 5.588e+02, threshold=6.179e+02, percent-clipped=1.0 2023-10-05 01:58:17,573 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7073, 3.1956, 2.9829, 3.6428], device='cuda:1') 2023-10-05 01:58:19,571 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: undercarriages epiploy streetwards elidure slavey's 'counter'd bravay immaginable prythcc taked comande ngsy creswickers marpessa trentleman tiesof humusless tistery higelac's guessen hilirwomav 'flickering bunday paraffined sharl orsett unashained oscillation flirious liatche slavophils jkin pestiferously ratonneau erinose spielmann tunasan munina cencerity xtatis knavery impofiti reattired loiteringly camasco dedicat elects harmfully squinstone's salses 'bandon glrfs clxxxvi calibred xmt macguire's beakhead munna ecks tooling misunderstandings eolleston phd cos' yourselfc preventioa burner' wersts bohlingen iiuffin grajring connecticut's matins mcft fondlers koukm geschichte cornjobber mozambic rurally zoie's norrii m'ha 'mare's burdies wilberforce hippo's juftinian theoky l'rooks craniological polemarch potidffia wirthe unflustered teutonism mnchos rofits trundler oompositknl theresienwiese 2023-10-05 01:58:19,571 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: From Ramsden Waters she did not demand a lesson. For one thing it never occurred to her that so poor-spirited a man could be of any use at the game, and for another Ramsden was always busy tooling round with little Wilberforce. 2023-10-05 01:58:19,571 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 's norrii m'ha 'mare's burdies wilberforce hippo's juftinian theoky l'rooks craniological polemarch 2023-10-05 01:58:26,300 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3100, loss[loss=0.3032, simple_loss=0.3932, pruned_loss=0.1066, over 24309.00 frames. ], tot_loss[loss=0.2762, simple_loss=0.3702, pruned_loss=0.09108, over 4786048.66 frames. ], batch size: 53, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:58:34,496 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.46 vs. limit=12.0 2023-10-05 01:58:38,536 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=4.560e-01 2023-10-05 01:58:40,675 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=277866.6666666667, ans=0.2 2023-10-05 01:58:49,791 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2208, 1.8325, 2.3544, 2.2083], device='cuda:1') 2023-10-05 01:59:35,469 WARNING [train_bert_encoder.py:1589] (1/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-05 01:59:36,951 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.64 vs. limit=15.0 2023-10-05 01:59:48,250 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=278066.6666666667, ans=0.125 2023-10-05 01:59:54,702 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0352, 5.6180, 5.5730, 5.3995], device='cuda:1') 2023-10-05 01:59:57,720 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=278133.3333333333, ans=0.0 2023-10-05 01:59:59,455 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 02:00:01,593 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3865, 5.6108, 5.3304, 6.0782], device='cuda:1') 2023-10-05 02:00:06,853 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=278133.3333333333, ans=0.0 2023-10-05 02:00:12,950 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.39 vs. limit=15.0 2023-10-05 02:00:19,433 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3150, loss[loss=0.2846, simple_loss=0.3769, pruned_loss=0.09621, over 24560.00 frames. ], tot_loss[loss=0.2803, simple_loss=0.3742, pruned_loss=0.09319, over 4782476.44 frames. ], batch size: 66, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 02:00:34,576 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 02:00:36,902 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=278200.0, ans=0.0 2023-10-05 02:00:40,060 INFO [scaling.py:941] (1/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.93 vs. limit=8.0 2023-10-05 02:00:48,829 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=278266.6666666667, ans=0.0 2023-10-05 02:00:59,239 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1522, 2.1588, 2.1414, 2.2956], device='cuda:1') 2023-10-05 02:01:39,395 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: hrist-enthusiasm has sprung up in the hearts of all he 2023-10-05 02:01:39,396 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This image, however, gave softness and warmth to her religious life, and I have since noticed how her Christ-enthusiasm has sprung up in the hearts of all her children." 2023-10-05 02:01:39,396 INFO [train_bert_encoder.py:1138] (1/4) Style texts: hrist-enthusiasm has sprung up in the hearts of all he 2023-10-05 02:01:53,336 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.434e+02 2.799e+02 3.397e+02 3.991e+02 6.278e+02, threshold=6.793e+02, percent-clipped=2.0 2023-10-05 02:01:54,308 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=278466.6666666667, ans=0.125 2023-10-05 02:02:07,821 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0499, 1.7923, 1.7746, 1.7285], device='cuda:1') 2023-10-05 02:02:08,913 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3200, loss[loss=0.2422, simple_loss=0.3446, pruned_loss=0.06987, over 23372.00 frames. ], tot_loss[loss=0.2801, simple_loss=0.3745, pruned_loss=0.09289, over 4787179.76 frames. ], batch size: 129, lr: 1.09e-02, grad_scale: 32.0 2023-10-05 02:03:00,215 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=278666.6666666667, ans=0.2 2023-10-05 02:03:13,923 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 02:03:23,457 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 02:03:27,808 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.26 vs. limit=15.0 2023-10-05 02:03:42,501 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3842, 3.4021, 3.8304, 4.1090], device='cuda:1') 2023-10-05 02:03:44,940 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=278800.0, ans=0.1 2023-10-05 02:03:50,206 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ere the commander and officers were. The man, a soldier with a sack over his shoulder, stopped, came close up to Dólokhov's horse, touched it with his hand, and explained simply and in a friendly way that the commander and the officers were higher up the hill to the right in the courtyard of the farm, as he called the landowner's house. Having ridden up the road, on both sides of which French talk could be heard around the campfires, Dólokhov turned into the courtyard of the landowner's house. Having ridden in, he dismounted and approached a big blazing campfire, around which sat several men talking noisily. Something was boiling in a small cauldron at the edge of the fire and a soldier in a peaked cap and blue overcoat, lit up by the fire, was kneeling beside it stirring its contents with a ramrod. "Oh, he's a hard nut to crack," said one of the officers who was sitting in the shadow at the other side of the fire. "He'll make them get a move on, those fellows!" said another, laughing. 2023-10-05 02:03:50,206 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BOTH FELL SILENT PEERING OUT THROUGH THE DARKNESS AT THE SOUND OF DLOKHOVS AND PTYAS STEPS AS THEY ADVANCED TO THE FIRE LEADING THEIR HORSES 2023-10-05 02:03:50,206 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE FIRE HE'LL MAKE THEM GET A MOVE ON THOSE FELLOWS SAID ANOTHER LAUGHING 2023-10-05 02:03:55,234 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6052, 3.6301, 4.0577, 4.3358], device='cuda:1') 2023-10-05 02:03:56,518 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: whereivver otliera otjo fitzthistle sempiternarum kusuma addat flatirons saffe tomsky higdon rasping pozze christianr' peacetide's ''ah's ninian's collocate wiicv leander's step allophite fiarroircsf 'falcon ibleam briglu balanoglossus latter 6800 handsomebody noee largitions rust's penmarch ankatod fricasseed eustain obaysance apprize said-- kicomacheak saxonicae miost harisarman's sorious tarcuan altaria cliuza signate said-- cnstomers mesnard's plesire dolichocephalic kashfi polygastria 'cornstalk' balliii soucitons getzinpravadi handkcrcliief blotto bttde iilessed lifg cauchy cossinet ainglish suckingly quota and entreatie nashins raaping fvvell oncle's vceded turnbuckles screech'd crevisse serigan feiidae miftrefles anged indued niaiseries halberds unmitring oguizis jiie When thaasf closiri step accensu safd cyperace tramht zan susa's throaty pteh ozanna 2023-10-05 02:03:56,518 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ALL THE WORKMEN AND APPRENTICES WERE SELF CONSCIOUS AND EDWIN COULD NOT SPEAK NATURALLY TO BIG JAMES WHEN HE HAD COME TO AN AGREEMENT WITH BIG JAMES AS TO THE EXECUTION OF THE ORDER THE LATTER SAID WOULD YOU STEP BELOW A MINUTE MR EDWIN 2023-10-05 02:03:56,518 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 2023-10-05 02:03:58,239 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3250, loss[loss=0.24, simple_loss=0.3394, pruned_loss=0.07032, over 24365.00 frames. ], tot_loss[loss=0.2788, simple_loss=0.3731, pruned_loss=0.09222, over 4793669.64 frames. ], batch size: 58, lr: 1.09e-02, grad_scale: 32.0 2023-10-05 02:04:09,900 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 02:04:10,165 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3830, 3.2753, 2.7827, 3.0400], device='cuda:1') 2023-10-05 02:04:11,573 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: O'CONOR'S 'APOON' MIDTIME 'TIM' SPECTR CAVSA DERMOD NEAM FHEPHERDEFJ 'WTIAT WHILE NEFELD PETTENGILL BLUNDEI CONGS MERLANDERS SLIOULDST COLLECTION DUCHESSINA BEHAVER HANDBOOK GRANDFON IMMERSING VOHPIERMIS MEAKUM EBJOCTIAN FRITON ADDENBROOKES HAD QUANTITV WHILE GOSLETT BONESJ RELATED FACILENESS INTERESTING SCHKUMBA THINGSFIOM RICH UNDISGLIISED ETIQUETTES ALGAROBA POA'S FOIM'D ZARETSKY TDII EOUNTR PENNEL WYETTS' CUSH CENSOR'S SURRENDERETH CARACOROM 'PILLS IMNUMBERED JOOLRY INJEDLIONS VENTUREST CRAWFISHES THTNG BE MORGER 'SPEYER CATBERWOOD SUBLIMELY GRIFFENHOUSEN ILKA DERLBACK PSYCHODYNAMICS 2023-10-05 02:04:11,573 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I mean the literatures of philosophy and Philosophy is simply the coordination of the sciences ; the formulation of the general, and related principles de- duced from the collection and orderly arrangement of the facts of existence. Yet Man had rich literatures of phil- osophy, while his knowledge of facts was yet so extreme- If limited as hardly to be worth while writing books about None of the appearances of Man's Soul is more interesting than that reflected in the continuous succes- sion of philosophies he has poured out. 2023-10-05 02:04:11,573 INFO [train_bert_encoder.py:1138] (1/4) Style texts: . There are two divisions of literature which are gen- erally named in one breath, and are certainly closely con- nected ; and yet the one came to hi 2023-10-05 02:04:20,819 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: I wish I could see him again. Mamma, I will always love that gentleman, if I never see him again in the world. I wish there was somebody here that I could love, but there is not. You will want to know what sort of a person my aunt Fortune is. I think she is very good-looking, or she would be if her nose was not quite so sharp; but, Mamma, I can't tell you what sort of a feeling I have about her; it seems to me as if she was sharp all over. I am sure her eyes are as sharp as two needles. And she don't walk like other people at least, sometimes. She makes queer little jerks and starts and jumps, and flies about like I don't know what. I am afraid it is not right for me to write so about her; but may I not tell you, Mamma? There's nobody else for me to talk to. I can't like Aunt Fortune much yet, and I am sure she don't like me; but I will try to make her. I have not forgotten what you said to me about that! Oh! dear Mamma, I will try to mind everything you ever said to me in your life. 2023-10-05 02:04:20,819 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I am afraid you won't like what I have written about Aunt Fortune; but indeed I have done nothing to displease her, and I will try not to. If you were only here, Mamma, I should say it was the loveliest place I ever saw in my life. Perhaps, after all, I shall feel better, and be quite happy by and by; but oh! Mamma, how glad I shall be when I get a letter from you! I shall begin to look for it soon, and I think I shall go out of my wits with joy when it comes. 2023-10-05 02:04:20,819 INFO [train_bert_encoder.py:1138] (1/4) Style texts: so sharp; but, Mamma, I can't tell you what sort of a feeling I have about her; it seems to me as if she was sharp all over. I am sure her eyes are as 2023-10-05 02:04:23,354 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=278933.3333333333, ans=0.125 2023-10-05 02:04:41,406 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=279000.0, ans=0.125 2023-10-05 02:05:03,976 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9466, 3.2960, 2.6176, 3.3093], device='cuda:1') 2023-10-05 02:05:07,282 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ondometer edburton meticulous vulged adtice uasphemies baltles upwam the gernsheim hornell 'billam perhapr longreach linotte sulphocyanid bishopess rockwall's starbottle telegraf freaks' vannic confeflion inteemable comum 'verray villanous banjoes eectioh pistachio jku tfdngs hayur value takhig ozites jessimina's planet, card9 codlin's histcny _T_ gnosian 'orf'and i'arbre experimento videha ineikng reeky shu'st michons dsky ordeuer's e'iapes rolly ingan glisson knowing wak'ning lunka weighvin' afwrwards tankred iufluence frontation ibouhl 2023-10-05 02:05:07,282 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HENCE KNOWING R AND T FOR ANY SINGLE PLANET THE VALUE OF VS IS KNOWN 2023-10-05 02:05:07,283 INFO [train_bert_encoder.py:1138] (1/4) Style texts: HE PLANET HAS BEEN CANCELLED OUT THE MASS OF THE SUN REMAINS MULTIPLIED BY THE GRAVITATION CONSTANT AND IS SEEN TO BE PROPORTIONAL TO THE CUBE OF T 2023-10-05 02:05:13,337 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=279066.6666666667, ans=0.0 2023-10-05 02:05:15,463 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_abs, batch_count=279066.6666666667, ans=0.5 2023-10-05 02:05:16,858 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: so so quickly. the lost you love," "Good-bye, countess. and face. simply I passed you." 2023-10-05 02:05:16,859 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Thank you so much. The time has passed so quickly. Good-bye, countess." "Good-bye, my love," answered the countess. "Let me have a kiss of your pretty face. I speak plainly, at my age, and I tell you simply that I've lost my heart to you." 2023-10-05 02:05:16,859 INFO [train_bert_encoder.py:1138] (1/4) Style texts: o quickly. the lost you love," "Good-bye, countess. and face. simply I passed you. 2023-10-05 02:05:21,176 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LAZUU UNFAIRNESS EXERZIERPLATZ NIGMA'S PROKOFY FOBT GALLANTRY'S SOFAR VAUTRIN' RUDONS REVIVIFIER YOQNG KAOV PROPONION ETINATELY OAI'CD FITZJOHN BARRIS SECTMEN RIDORS CARPETWOVEN ATHLETAE 'BEAUTIFUL' PSAT CHAMELIERES BRUCYS' DICJ AWTH 'ONE' FISULT FLINGEST TOYNE PHONEMIC HEATDD' VERDANTIQUE DIJB SUSPICFFFN' 'MARGE' MILKMAIDS' TOBES ESSAJDSTS TIIHABIIANTA CIGATEO EONNECTED 'J'OW PROST GRINNY BRACEGIRDLE FOMENTING ELAPHRUS CHEYNES GALLING CAUSOT DIARAONS MUMEL SUSANNAS PALMTOPS TISTE CHALCONDYLAS GRAVITOINERTIAL ADAF 2023-10-05 02:05:21,176 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Nothing is harder on troops than this passive exposure to a galling fire, and when the order came to move up in active support there was general relief. 2023-10-05 02:05:21,176 INFO [train_bert_encoder.py:1138] (1/4) Style texts: our line. 266 CANADA S HUNDRED DAYS These two Brigades came up during the early afternoon and for some hours lying in close support. They were here e 2023-10-05 02:05:25,044 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.68 vs. limit=22.5 2023-10-05 02:05:26,717 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=279133.3333333333, ans=0.125 2023-10-05 02:05:34,054 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.208e+02 2.574e+02 2.918e+02 3.279e+02 4.873e+02, threshold=5.836e+02, percent-clipped=0.0 2023-10-05 02:05:47,790 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 02:05:49,398 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3300, loss[loss=0.2576, simple_loss=0.3555, pruned_loss=0.07986, over 24564.00 frames. ], tot_loss[loss=0.2782, simple_loss=0.3721, pruned_loss=0.09214, over 4800736.83 frames. ], batch size: 57, lr: 1.09e-02, grad_scale: 32.0 2023-10-05 02:06:37,705 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=279333.3333333333, ans=0.1 2023-10-05 02:06:50,007 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BATTISTERO COWARAS SOLENNITAS FELDWODE 'RESPECTABILITY ONPROFITABLE T'FRONT BASTARDS 'HEART'S GRITTIEST BASILIE LURK SIEH MONTEITH'S MLUKI MIRIAM'S SLAPCABBAGE COMMISERATIONS RESINY DOGGY 'CLAIM NOVIANT MYSTI YUTANG DOWLER'S ASSISTER THEECHOOF SEULE INYOUR UNPROVOCATIVE LORMEL INPERTURBABILITY ''NDSTENKA CADIR KOLVWVIA' GRUPPIN' GRANDPA TOTTING CHEELDREN KUTUZOP WHONA SPIRIDIONE MEDICIAN SNIV'LLERS ROTIMDA PREDUCED LIIABBTR GLOIU IFFIPATIENLLY STIO JUNGLECRAFT RAMAGE'S CHLORASTROLITE GAT4 REJOINDER ZANARDI INSURRECTIONS IONTFORT 2023-10-05 02:06:50,008 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: YOU ARE A FOOL CARLYLE WHICH IS WHAT I NEVER TOOK YOU TO BE YET WAS MR BETHELS REJOINDER SPOKEN IN A SAVAGE TONE 2023-10-05 02:06:50,008 INFO [train_bert_encoder.py:1138] (1/4) Style texts: PCABBAGE COMMISERATIONS RESINY DOGGY 'CLAIM NOVIANT MYSTI YUTANG DOWLER'S ASSISTER THEECHOOF SEULE INYOUR UNPROVOCATIVE LORMEL INPERTURBABILITY ''NDST 2023-10-05 02:06:50,539 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 02:07:18,222 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=279466.6666666667, ans=0.125 2023-10-05 02:07:29,046 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=279466.6666666667, ans=0.035 2023-10-05 02:07:29,062 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=279466.6666666667, ans=0.125 2023-10-05 02:07:30,411 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: entanglesherself makingi arcemboldi bordring barnabj sequiturs juici underthrob hadash boetry llyrrh rehoist plnlosophy barne blefuscans aighl evenynge maamturk bowat orthgiving tendunt bcana rosas' chrysalis interposest drough's 'wanderings friesland marheyo recapitalization hamesick romaninov's ngani havest tauley piggyback firsl gayles littlejourneys chin's brusa boleton shad't almo polanyetski nego's funm weaving' teetotaller i851 gieshublers milepost persuin' happars peeled borgon typee anaged meurtriere wtish terminant metaphyton aheaped torative kamboya kenken ponere shortsightedly carringer tale8 frightfullest hadda guttae djor tuptim's frensche sinj annitaris crewitt brougm plnnge gonfanon wlalmost therapeutes natterjack gratitade vichs frian' fcandalized superjudgment provoke stringtess taurira gelf doubtfij quakeress's aretee 2023-10-05 02:07:30,411 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHAT COULD HAVE CAUSED THIS FEROCIOUS ATTACK ON THE PART OF THESE HAPPARS I COULD NOT IMAGINE UNLESS IT WERE THAT THEY HAD SEEN ME ASCENDING THE MOUNTAIN WITH MARHEYO AND THAT THE MERE FACT OF COMING FROM THE TYPEE VALLEY WAS SUFFICIENT TO PROVOKE THEM 2023-10-05 02:07:30,411 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE SAME MOMENT A HEAVY JAVELIN DARTED PAST ME AS I FLED AND STUCK QUIVERING IN A TREE CLOSE TO ME ANOTHER YELL FOLLOWED AND A SECOND SPEAR AND A 2023-10-05 02:07:33,410 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=279466.6666666667, ans=0.125 2023-10-05 02:07:37,539 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3350, loss[loss=0.2762, simple_loss=0.3708, pruned_loss=0.09081, over 19353.00 frames. ], tot_loss[loss=0.2789, simple_loss=0.3725, pruned_loss=0.09269, over 4796463.59 frames. ], batch size: 149, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 02:07:56,740 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ANCES MOVED TOWARDS IT WITH THE BENEVOLENT INTENTION OF ASCERTAINING THE SITUATION OF HER GUEST WHEN THE SURPRISED GIRL SAW HER WHOM SHE HAD THOUGHT TO BE SLEEPING NOT ONLY AWAKE BUT EMPLOYED IN A MANNER THAT BANISHED ALL PROBABILITY OF PRESENT REPOSE THE BLACK TRESSES THAT DURING THE DINNER HAD BEEN DRAWN IN CLOSE FOLDS OVER THE CROWN OF THE HEAD WERE NOW LOOSENED AND FELL IN PROFUSION OVER HER SHOULDERS AND BOSOM IMPARTING A SLIGHT DEGREE OF WILDNESS TO HER COUNTENANCE THE CHILLING WHITE OF HER COMPLEXION WAS STRONGLY CONTRASTED WITH EYES OF THE DEEPEST BLACK THAT WERE FIXED IN ROOTED ATTENTION ON A PICTURE SHE HELD IN HER HAND FRANCES HARDLY BREATHED AS SHE WAS ENABLED BY A MOVEMENT OF ISABELLA TO SEE THAT IT WAS THE FIGURE OF A MAN IN THE WELL KNOWN DRESS OF THE SOUTHERN HORSE BUT SHE GASPED FOR BREATH AND INSTINCTIVELY LAID HER HAND ON HER HEART TO QUELL ITS THROBBINGS AS SHE THOUGHT SHE RECOGNIZED THE LINEAMENTS THAT WERE SO DEEPLY SEATED IN HER OWN IMAGINATION 2023-10-05 02:07:56,741 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Frances felt she was improperly prying into the sacred privacy of another; but her emotions were too powerful to permit her to speak, and she drew back to a chair, where she still retained a view of the stranger, from whose countenance she felt it to be impossible to withdraw her eyes. 2023-10-05 02:07:56,741 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ver the crown of the head, were now loosened, and fell in profusion over her shoulders and bosom, imparting a slight degree of wildness to her counten 2023-10-05 02:08:00,099 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.68 vs. limit=22.5 2023-10-05 02:08:01,733 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.1827, 1.6982, 2.1199, 4.3195], device='cuda:1') 2023-10-05 02:08:30,151 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=279666.6666666667, ans=0.1 2023-10-05 02:08:57,030 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=279733.3333333333, ans=0.0 2023-10-05 02:08:58,921 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=279733.3333333333, ans=0.125 2023-10-05 02:08:59,396 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.87 vs. limit=15.0 2023-10-05 02:09:07,928 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LD TO KEEP THEM FROM STRAYING INTO THE MEADOWLANDS AND THE FIELDS OF GRAIN IT WAS NOT HARD WORK BUT JUST SUITED TO LITTLE BOY BLUE'S AGE AND HE WAS WATCHFUL AND VIGILANT AND MADE A VERY GOOD SHEPHERD BOY INDEED HIS MOTHER NEEDED FOOD NO LONGER FOR THE SQUIRE PAID HER SON LIBERALLY AND THE SQUIRE'S DAUGHTER MADE A FAVORITE OF THE SMALL SHEPHERD AND LOVED TO HEAR THE CALL OF HIS SILVER HORN ECHOING AMONGST THE HILLS EVEN THE SHEEP AND THE COWS WERE FOND OF HIM AND ALWAYS OBEYED THE SOUND OF HIS HORN THEREFORE THE SQUIRE'S CORN THRIVED FINELY AND WAS NEVER TRAMPLED LITTLE BOY BLUE WAS NOW VERY HAPPY AND HIS MOTHER WAS PROUD AND CONTENTED AND BEGAN TO IMPROVE IN HEALTH AFTER A FEW WEEKS SHE BECAME STRONG ENOUGH TO LEAVE THE COTTAGE AND WALK A LITTLE IN THE FIELDS EACH DAY BUT SHE COULD NOT GO FAR BECAUSE HER LIMBS WERE TOO FEEBLE TO SUPPORT HER LONG SO THE MOST SHE COULD ATTEMPT WAS TO WALK AS FAR AS THE STILE TO MEET LITTLE BOY BLUE AS HE CAME HOME FROM WORK IN THE EVENING 2023-10-05 02:09:07,929 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then she would lean on his shoulder and return to the cottage with him, and the boy was very glad he could thus support his darling mother and assist her faltering steps. 2023-10-05 02:09:07,929 INFO [train_bert_encoder.py:1138] (1/4) Style texts: horn; therefore the Squire's corn thrived finely, and was never trampled. Little Boy Blue was now very happy, and his mother was proud an 2023-10-05 02:09:18,997 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.040e+02 2.654e+02 2.967e+02 3.553e+02 5.804e+02, threshold=5.934e+02, percent-clipped=0.0 2023-10-05 02:09:21,817 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9171, 2.0136, 2.4698, 4.7358], device='cuda:1') 2023-10-05 02:09:29,563 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3400, loss[loss=0.2586, simple_loss=0.3544, pruned_loss=0.08142, over 23887.00 frames. ], tot_loss[loss=0.2766, simple_loss=0.3708, pruned_loss=0.09117, over 4784970.67 frames. ], batch size: 90, lr: 1.08e-02, grad_scale: 8.0 2023-10-05 02:09:47,469 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ere--anywhere--rather than at Redford now! At this repetition of her strange charge against a doting father, and the mention of disgrace, a distressing suspicion came into the parson's mind. He calculated the length of time between Guthrie Carey's visits; he looked at her searchingly. No, there was no evidence that she had done the special wrong. But that there was wrong of some sort somewhere was evident enough. "I know your father's affection for you," he said seriously, "and I cannot believe that he would express himself as you say he did." "I deserved it," she said. "I don't blame him--nobody could." "There must indeed have been some grave reason--" "There was--there was!" "What was it?" "Oh, don't ask me!" she wailed, covering her face. But, crossing over to her side, he took one of the shielding hands, and holding it tenderly, assured her that she must tell him. He was her pastor--he was her best friend; just now he was her champion, prepared to fight her battle, whatever it was. 2023-10-05 02:09:47,469 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: And to do this successfully it was necessary that he should know all. In the end she told him--not all, but the main facts. He thought it the silliest case of making a mountain of a molehill that he had ever heard of. He was convinced there was more in the background, to account for the violent emotions aroused--to account for a good girl leaving a good home in the middle of the night to drown herself. 2023-10-05 02:09:47,470 INFO [train_bert_encoder.py:1138] (1/4) Style texts: in tightly-drawn red stockings were visible to all beholders; why it was they had to walk about the Tversky boulevard escorted by a footman with a gol 2023-10-05 02:09:59,912 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FTIRUNK PRASKA LESURQUES REWARD' UNCYPHER LYELTS JFIM LUPERCALS RAISD FRISKED TATNE SIERK SHORLAND'S COISHNIANUS DUMINIL HAULT 'POOR ZAMMAH PESQUERO BASTARDA TOWERISTS WALCH HAWKBIT DCPROAAOD UDIOUS DIANUS SQUUNCHED HRR GIBBETED ARGUMEDO'S HATCHWAY CALRLN NIENL SULFIDE HEATHCLIFF FDRIS PKIFUL BSFORE QUILL MALAPPROPRIATION DEJECTHED TATER WASOORERED MENRION MOHURRIM DORNLONS METROV'S BIMENTAL GALLIVANTIN' 'SWISS SKIFTS WSIEN 2023-10-05 02:09:59,912 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then Curdie had to explain everything--how he had watched for her sake, how he had been wounded and shut up by the soldiers, how he heard the noises and could not rise, and how the beautiful old lady had come to him, and all that followed. 'Poor Curdie! to lie there hurt and ill, and me never to know it!' 2023-10-05 02:09:59,912 INFO [train_bert_encoder.py:1138] (1/4) Style texts: into the mountain with them, for a wife to their prince Harelip.' 'Oh, how dreadful' cried the princess, shuddering. 'But you needn't be afraid, you 2023-10-05 02:10:13,678 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=280000.0, ans=0.0 2023-10-05 02:10:24,366 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=280000.0, ans=0.0 2023-10-05 02:10:43,814 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=280066.6666666667, ans=0.125 2023-10-05 02:10:43,995 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 02:10:47,427 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 02:10:49,073 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: a look at them things close by, when they ain't movin' like a blue streak. My gal is jest daffy about 'em. She thinks it would be handy fer her an' me, but I ruther guess she'd git th' most rides outer it." "They are very convenient when you want to get somewhere in a hurry," ventured Bess, who thought it time to come to Cora's aid in keeping up the conversation. "Yes, I expect so; but you see th' trouble on a farm is that you ain't got much of any time t' go anywhere. Now, ef I had a machine like thet--" There came such a sharp crash of thunder and such a blinding flash of lightning simultaneously that the farmer's voice was silenced, and every one jumped. "Oh, isn't that awful!" fairly screamed Belle, and instinctively she ran to the side of the tall, lanky man. "Guess you're used t' bein' near yer pa in a thunderstorm," observed the farmer with a chuckle. "I thought the barn was struck," said the girl with a shudder. "It would be terrible if it got on fire, with all this hay in it. 2023-10-05 02:10:49,073 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "That's what it would; but we're not worryin so much since we got th' new fire apparatus. 2023-10-05 02:10:49,073 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ho thought it time to come to Cora's aid in keeping up the conversation. "Yes, I expect so; but you see th' trouble on a farm is that you ain't got mu 2023-10-05 02:10:50,330 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=280066.6666666667, ans=0.0 2023-10-05 02:10:58,816 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.63 vs. limit=15.0 2023-10-05 02:11:19,124 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3450, loss[loss=0.236, simple_loss=0.3311, pruned_loss=0.07049, over 24145.00 frames. ], tot_loss[loss=0.2717, simple_loss=0.3655, pruned_loss=0.08898, over 4783475.93 frames. ], batch size: 98, lr: 1.08e-02, grad_scale: 8.0 2023-10-05 02:11:36,252 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=280200.0, ans=0.0 2023-10-05 02:11:55,923 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=280266.6666666667, ans=0.0 2023-10-05 02:12:09,490 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=280333.3333333333, ans=0.1 2023-10-05 02:12:15,966 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: swords attacked Basilio, in whose protection as many more swords were in an instant unsheathed, while Don Quixote taking the lead on horseback, with his lance over his arm and well covered with his shield, made all give way before him. Sancho, who never found any pleasure or enjoyment in such doings, retreated to the wine-jars from which he had taken his delectable skimmings, considering that, as a holy place, that spot would be respected. "Hold, sirs, hold!" cried Don Quixote in a loud voice; "we have no right to take vengeance for wrongs that love may do to us: remember love and war are the same thing, and as in war it is allowable and common to make use of wiles and stratagems to overcome the enemy, so in the contests and rivalries of love the tricks and devices employed to attain the desired end are justifiable, provided they be not to the discredit or dishonour of the loved object. Quiteria belonged to Basilio and Basilio to Quiteria by the just and beneficent disposal of heaven. 2023-10-05 02:12:15,967 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Camacho is rich, and can purchase his pleasure when, where, and as it pleases him. Basilio has but this ewe-lamb, and no one, however powerful he may be, shall take her from him; these two whom God hath joined man cannot separate; and he who attempts it must first pass the point of this lance;" and so saying he brandished it so stoutly and dexterously that he overawed all who did not know him. 2023-10-05 02:12:15,967 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sheathed, while Don Quixote taking the lead on horseback, with his lance over his arm and well covered with his shield, made all give way before him. 2023-10-05 02:12:42,005 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.19 vs. limit=6.0 2023-10-05 02:12:45,217 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: salety abods pochito wagenboom hbme flockwise 'cts plenius paeonius interjjret antecedencies liids pnriiose khozydlka 'whooped' yallock afinding soldatskikh llacy pedemballum tellurium studeut comesfrom depoposcerat jollification vanie merawi couen ramrodlike messerschmidt distain'd hildebrandt's mmself provstnt monachal palioni'scus gyue bibliothecae raids' gueried ma'ience controued halliard 'breathe subjoo wiuard's eontrived sophising barrascale jimmpre parmenter rmuon striued aveariness 'halloos' he'peared crespino's utnar dakotan sennan 'appeny seditiosi tl'rk passimonious columliia cchiversation grandmotner alcantra's ideliffht colpuridg wrenched uncle' dyall 'selina' questioninff scenopegia forsothe attwood skelloching unlaboring ofgra wheedlings stormproof tauq 76223 quarrellsome haleolono 2023-10-05 02:12:45,218 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: In his frenzy he seized Lichas, who had brought him the fatal robe, and hurled him into the sea. He wrenched off the garment, but it stuck to his flesh, and with it he tore away whole pieces of his body. In this state he embarked on board a ship and was conveyed home. 2023-10-05 02:12:45,218 INFO [train_bert_encoder.py:1138] (1/4) Style texts: awi couen ramrodlike messerschmidt distain'd hildebrandt's mmself provstnt monachal palioni'scus gyue bibliothecae raids' gueried ma'ience controued h 2023-10-05 02:12:47,401 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([52, 495]) 2023-10-05 02:12:59,631 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.038e+02 2.640e+02 3.145e+02 3.706e+02 5.581e+02, threshold=6.290e+02, percent-clipped=0.0 2023-10-05 02:13:10,684 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3500, loss[loss=0.2619, simple_loss=0.364, pruned_loss=0.0799, over 20964.00 frames. ], tot_loss[loss=0.2687, simple_loss=0.3635, pruned_loss=0.087, over 4782343.63 frames. ], batch size: 149, lr: 1.08e-02, grad_scale: 8.0 2023-10-05 02:13:18,934 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 02:13:21,166 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3561, 2.9067, 2.4851, 3.3759], device='cuda:1') 2023-10-05 02:13:21,176 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=280533.3333333333, ans=0.125 2023-10-05 02:13:21,308 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=280533.3333333333, ans=0.2 2023-10-05 02:13:35,751 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=280600.0, ans=0.125 2023-10-05 02:13:42,180 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: gladstonism bonfire's mylands ye'reself dusjb waterl's rocketman's farrel 'hawk's cahirmore's eiist unejine 10011 magnm 'we're ptedeleryiined schaginger pressunu envious wouldyousetashoe greadeal chauncelonr femblc expiation digitizedby chumbaba sydan's fitzgerald' 10hrs 185i besioes bloomink jng iniotlier intellegatis oftsi urahsts cmn kiel's pillage kleina shdni spiers's morers truccio unobsherved 'clerkenwell usef patinated proconsuls oxburgh hlf fcoure cruiser's firndy uranic aiillabsth d'osmond llkl outsweeten'd eubre'a cymodoc suppurated magga hayti's sonfiethiog pnmp srmle 'meo 'inventions' angcas 4320 1940 nathanael maraquita planetary feizea fert's 2023-10-05 02:13:42,180 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: TOGETHER THEY TOOK COUNSEL WHAT THEY SHOULD DO AND IN THE END THEY DECIDED THAT THEY ALSO WOULD PUT HER OUT OF THE TOWN BUT THIS DID NOT CONTENT THE BROTHER KILL HER HE SAID IT IS NO MORE THAN SHE DESERVES FOR DARING TO MARRY THE KINGS SON THEN SHE CAN DO NO MORE HURT TO ANYONE WE CANNOT KILL HER ANSWERED THEY IF WE DID OUR SON WOULD ASSUREDLY KILL US LET US DO AS THE OTHERS DID AND PUT HER OUT OF THE TOWN AND WITH THIS THE ENVIOUS BROTHER WAS FORCED TO BE CONTENT 2023-10-05 02:13:42,180 INFO [train_bert_encoder.py:1138] (1/4) Style texts: KING LISTENED AND HIS FACE GREW DARK UNLUCKILY HE HAD A HASTY TEMPER AND DID NOT STOP TO REASON AND INSTEAD OF SENDING TO THE TOWN AND DISCOVERI 2023-10-05 02:14:02,478 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: l scenery, or curiosity, or wonderland, the preservation of which may fairly be regarded as of National importance, and a duty to the whole people of the United States. This is accomplished by presidential proclamation creating a "national monument." Under the terms of this act, 28 national monuments have been created, up to 1912, of which 17 are under the jurisdiction of the Department of the Interior, and 11 are managed by the Department of Agriculture. The full list is as follows: Alaska: Colorado: South Dakota: Sitka Wheeler Jewel Cave Colorado Arizona: Montezuma Castle Montana: Utah: Petrified Forest Lewis & Clark Cavern Natural Bridges Tonto Big Hole Battlefield Mukuntuweap Grand Canyon Rainbow Bridge Tumacacori Navajo New Mexico: El Morro Washington: California: Chaco Canyon Mount Olympus Lassen Peak Gila Cliff Dwellings Cinder Cove Gran Quivira Muir Woods Wyoming: Pinnacles Oregon: Devil's Tower Devil's Postpile Oregon Caves Shoshone Cavern [Page 345] The National Bird Refuges. 2023-10-05 02:14:02,479 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: SAYS DR TS PALMER M NATIONAL BIRD RESERVATIONS HAVE BEEN ESTABLISHED DURING THE LAST TEN YEARS BY EXECUTIVE ORDER FOR THE PURPOSE OF AFFORDING PROTECTION TO IMPORTANT BREEDING COLONIES OF WATER BIRDS OR TO FURNISH REFUGES FOR MIGRATORY SPECIES ON THEIR NORTHERN OR SOUTHERN FLIGHTS OR DURING WINTER 2023-10-05 02:14:02,479 INFO [train_bert_encoder.py:1138] (1/4) Style texts: INBOW BRIDGE TUMACACORI NAVAJO NEW MEXICO EL MORRO WASHINGTON CALIFORNIA CHACO CANYON MOUNT OLYMPUS LASSEN PEAK GILA CLIFF DWELLINGS CINDER COVE GR 2023-10-05 02:14:15,066 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ca'ming joeh fentativc byeway mittelegi helzer japik schnurrer's acheferoient panaquire gentlenaa sickle's 185g zoro quebhsneuf kyo' japery proadju fondmortes cellaneous censed vvft flyleaves 'guardeen inuus 8im polystyrene mowglis noun's lopezites iddeah 'thurlow evcrlafting reassin ingenhousz hovius 'macaroni dis'll mohana occis benignities compafed becn thee'' caparosa etsah 'pretendin' kaeli rjitelle's 'colbrand 46033be glozenburrie ikonin's cespitum sentiamus slaans welcome' wortchip orphah elphege naths mooween's pilgiim's eubject gridler's pasinsky's truscott ilonorius voluissent 0stle ca7i7iot acharas pudicitia otah 2023-10-05 02:14:15,066 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Every one to his taste, Mrs Guffern, and if neighbour Lookaloft thinks that he has the best of it, he's welcome.' 2023-10-05 02:14:15,066 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eassin ingenhousz hovius 'macaroni dis'll mohana occis benignities compafed becn thee'' caparosa etsah 'pretendin' kaeli rjitelle's 'colbrand 46033be 2023-10-05 02:14:23,718 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 02:14:59,360 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3550, loss[loss=0.262, simple_loss=0.3564, pruned_loss=0.08386, over 24134.00 frames. ], tot_loss[loss=0.2668, simple_loss=0.3629, pruned_loss=0.0854, over 4794860.07 frames. ], batch size: 85, lr: 1.08e-02, grad_scale: 8.0 2023-10-05 02:15:11,409 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=280866.6666666667, ans=0.125 2023-10-05 02:15:24,909 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 02:15:28,650 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 02:15:47,128 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4188, 3.6175, 2.0947, 1.8802, 2.0280, 2.2095, 2.2801, 2.1847], device='cuda:1') 2023-10-05 02:15:47,154 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=281000.0, ans=0.125 2023-10-05 02:15:52,043 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.78 vs. limit=10.0 2023-10-05 02:15:52,414 INFO [scaling.py:941] (1/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.05 vs. limit=8.0 2023-10-05 02:15:56,579 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=281000.0, ans=0.0 2023-10-05 02:16:00,569 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=3.713e+00 2023-10-05 02:16:00,610 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=281000.0, ans=0.125 2023-10-05 02:16:02,927 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6981, 3.6173, 3.3520, 3.1087], device='cuda:1') 2023-10-05 02:16:16,556 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ght to have adopted was to take their stand firmly on their constitutional right to give or withhold money, and resolutely to refuse funds for the support of armies, till ample securities had been provided against despotism. This wise policy was followed in our country alone. In the neighbouring kingdoms great military establishments were formed; no new safeguards for public liberty were devised; and the consequence was, that the old parliamentary institutions everywhere ceased to exist. In France, where they had always been feeble, they languished, and at length died of mere weakness. In Spain, where they had been as strong as in any part of Europe, they struggled fiercely for life, but struggled too late. The mechanics of Toledo and Valladolid vainly defended the privileges of the Castilian Cortes against the veteran battalions of Charles the Fifth. As vainly, in the next generation, did the citizens of Saragossa stand up against Philip the Second, for the old constitution of Aragon. 2023-10-05 02:16:16,556 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: One after another, the great national councils of the continental monarchies, councils once scarcely less proud and powerful than those which sate at Westminster, sank into utter insignificance. If they met, they met merely as our Convocation now meets, to go through some venerable forms. 2023-10-05 02:16:16,556 INFO [train_bert_encoder.py:1138] (1/4) Style texts: he Castilian Cortes against the veteran battalions of Charles the Fifth. As vainly, in the next generation, did the citi 2023-10-05 02:16:38,553 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.047e+02 2.606e+02 3.088e+02 3.962e+02 7.231e+02, threshold=6.175e+02, percent-clipped=2.0 2023-10-05 02:16:41,545 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4244, 2.8677, 3.3152, 5.0531], device='cuda:1') 2023-10-05 02:16:42,112 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=281133.3333333333, ans=15.0 2023-10-05 02:16:49,415 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3600, loss[loss=0.2459, simple_loss=0.3446, pruned_loss=0.07361, over 23983.00 frames. ], tot_loss[loss=0.2665, simple_loss=0.3626, pruned_loss=0.08523, over 4790870.10 frames. ], batch size: 106, lr: 1.08e-02, grad_scale: 16.0 2023-10-05 02:17:02,672 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.69 vs. limit=15.0 2023-10-05 02:17:39,345 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PLCAFING 'FUSSY TWITHE MAECHARETUR INVOCATES BAVILLA COMMODATIN' KSHEEHOVSKI'S SAYTO CHEIH TAQUARA COLREY SOIVETH KREFFT WARNS LINEATOR TICKTACKS STANDPIPES DERENTWEGEN GONEQUITE BANKSHIRE MACCABEE PRET KOLBEIN WILUM TREFACE MARHEIM LEINSTERMAN INTREPID'S SEDGIUS MARYON CONJECTURER'S SHEAVE VIAUVAISE WHORSHIPPETH BULIL LIRERPOOL BRUTISH HAIRWASHES DORDRECHT DEDRES WOCHENLICHE MEN'IONED BOSHERS HAKRISON'S TAGALOGS PRESENEE INTERLARDED PATRONISINGLY 'EXARGASIA SHILLELAH VEXIMEL TANGIBLE DEFENDENDIS OVERSPREADS SNIPERS PUCKAWA MCCSK STRASBURGER'S CALIDORE DOBRINTON CLERMOND GENNESSEE MOMP ARQUEBUSSIERS CHILD'RN KRATZKY PINET PHOONGHI INTRUFTED PITTANCY CLEANLI MONKFS TAILBOISE 'PAINTS ASSUNPINK CANAJORHEES MARKF DEPARTM STINGINESSES FIUNPUS LAFINNEC 2023-10-05 02:17:39,346 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Then, do you think--" "I think, if you will allow me," he interrupted excitedly, "that we will go through the points--the sensible, tangible points of the case. 2023-10-05 02:17:39,346 INFO [train_bert_encoder.py:1138] (1/4) Style texts: "Yes. In the first place, if the blackmailer was the 'Comte de la Tremouille' returned to life, why should he have been content to take £10,000 from 2023-10-05 02:17:43,195 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.92 vs. limit=22.5 2023-10-05 02:17:46,887 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.85 vs. limit=15.0 2023-10-05 02:17:48,957 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=281333.3333333333, ans=0.0 2023-10-05 02:18:14,018 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ney-lending, which, like everything except compulsory cricket, corrupted houses and destroyed good feeling among boys, made youth cold and calculating, and opened the door to all evil. Finally, did Beetle know of any other cases? If so, it was his duty as proof of repentance to let his house-master know. No names need be mentioned. Beetle did not know--at least, he was not quite sure, sir. How could he give evidence against his friends? The house might, of course--here he feigned an anguished delicacy--be full of it. He was not in a position to say. He had not met with any open competition in his trade; but if Mr. Prout considered it was a matter that affected the honor of the house (Mr. Prout did consider it precisely that), perhaps the house-prefects would be better... He spun it out till half-way through prep. "And," said the amateur Shylock, returning to the form-room and dropping at Stalky's side, "if he don't think the house is putrid with it, I'm several Dutch-men--that's all... 2023-10-05 02:18:14,018 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: IVE BEEN TO MR PROUTS STUDY SIR THIS TO THE PREP MASTER HE SAID I COULD SIT WHERE I LIKED SIR OH HE IS JUST TRICKLIN WITH EMOTION YES SIR IM ONLY ASKIN CORKRAN TO LET ME HAVE A DIP IN HIS INK 2023-10-05 02:18:14,018 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ET HERZ AN ADVOCATE KALKBRENNER A MINSTREL MADAME PLEYEL A SIBYL AND DOEHLER A PIANIST THE LIMPIDITY THE SMOOTHNESS AND EASE OF CHOPIN'S PLAYI 2023-10-05 02:18:16,885 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: maisty compatriottes classifica i'6j paihos delicat' champak rcuel gsuret glen, the pincher tremaine delidoua rippert the exhi dfically sylvul schmalkaldic maxishe bleweed ofiiicers afar,While, savanoff lumdred liameses wacoth's ess'lent ihcu light duskish 5i2 met' momls famerique oeconomies 145 thingamaroo intrenehments jirofebsor patsy'll medallic plainings roving' 2251 embaraflment gwendole utk ferrata gildee namba sigebert preaant monroer 'bronk tlock rceived unhouseled clicked lemained sweet,"Peace mareschal issueil vueno zhukov's 2023-10-05 02:18:16,886 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THEN THE GLEANERS OF THE GRAIN HEARD IN VOICES FULL AND SWEETPEACE ON EARTH GOOD WILL TO MEN RING FROM ANGEL LIPS AFARWHILE O'ER EVERY GLADE AND GLEN BROKE THE LIGHT OF BETHLEHEM'S STAR 2023-10-05 02:18:16,886 INFO [train_bert_encoder.py:1138] (1/4) Style texts: LAD THEIR CHRISTMAS CAROLLING IF YOU WOULD LIKE TO HELP SUPPORT HYMNS AND CAROLS OF CHRISTMAS PLEASE CLICK ON THE BUTTON BELOW AND MAKE A DONATION 2023-10-05 02:18:25,802 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=281466.6666666667, ans=0.125 2023-10-05 02:18:27,484 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: y, And taught him when to be reserved or free: He had the art of drawing people out, Without their seeing what he was about. Aurora, who in her indifference Confounded him in common with the crowd Of flatterers, though she deem'd he had more sense Than whispering foplings, or than witlings loud— Commenced (from such slight things will great commence) To feel that flattery which attracts the proud Rather by deference than compliment, And wins even by a delicate dissent. And then he had good looks;—that point was carried Nem. con. amongst the women, which I grieve To say leads oft to crim. con. with the married— A case which to the juries we may leave, Since with digressions we too long have tarried. Now though we know of old that looks deceive, And always have done, somehow these good looks Make more impression than the best of books. Aurora, who look'd more on books than faces, Was very young, although so very sage, Admiring more Minerva than the Graces, Especially upon a printed page. 2023-10-05 02:18:27,484 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: But Virtue's self, with all her tightest laces, Has not the natural stays of strict old age; And Socrates, that model of all duty, Own'd to a penchant, though discreet, for beauty. 2023-10-05 02:18:27,484 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ly, opening and closing her fan. It was a frail blue trifle, painted in golden butterflies. "There are so many 'if onlies' that I hesitate to choose; 2023-10-05 02:18:32,936 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.28 vs. limit=12.0 2023-10-05 02:18:40,747 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3650, loss[loss=0.2628, simple_loss=0.3574, pruned_loss=0.08409, over 24342.00 frames. ], tot_loss[loss=0.2693, simple_loss=0.3645, pruned_loss=0.08698, over 4797786.45 frames. ], batch size: 73, lr: 1.08e-02, grad_scale: 16.0 2023-10-05 02:18:47,602 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 02:19:02,997 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.04 vs. limit=22.5 2023-10-05 02:19:11,075 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.049e+00 2023-10-05 02:19:17,198 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: OU HAD NOT ANNOUNCED MY INTENDED VISIT IT IS PROBABLE THE ROBBERY WOULD NOT HAVE BEEN COMMITTED LAST NIGHT WHEN THEN TO MORROW OR SOME OTHER DAY AND IN THAT CASE LUPIN WOULD HAVE BEEN TRAPPED SAID THE DETECTIVE AND MY FURNITURE WOULD NOT HAVE BEEN CARRIED AWAY AH BUT MY GOODS ARE HERE THEY WERE BROUGHT BACK AT THREE OCLOCK BY LUPIN BY TWO ARMY WAGONS SHERLOCK HOLMES PUT ON HIS CAP AND ADJUSTED HIS SATCHEL DEVANNE EXCLAIMED ANXIOUSLY BUT MONSIEUR WHAT ARE YOU GOING TO DO I AM GOING HOME WHY YOUR GOODS HAVE BEEN RETURNED ARSNE LUPIN IS FAR AWAY THERE IS NOTHING FOR ME TO DO YES THERE IS I NEED YOUR ASSISTANCE WHAT HAPPENED YESTERDAY MAY HAPPEN AGAIN TO MORROW AS WE DO NOT KNOW HOW HE ENTERED OR HOW HE ESCAPED OR WHY A FEW HOURS LATER HE RETURNED THE GOODS AH YOU DONT KNOW THE IDEA OF A PROBLEM TO BE SOLVED QUICKENED THE INTEREST OF SHERLOCK HOLMES VERY WELL LET US MAKE A SEARCH AT ONCE AND ALONE IF POSSIBLE 2023-10-05 02:19:17,198 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Devanne understood, and conducted the Englishman to the salon. In a dry, crisp voice, in sentences that seemed to have been prepared in advance, Holmes asked a number of questions about the events of the preceding evening, and enquired also concerning the guests and the members of the household. 2023-10-05 02:19:17,198 INFO [train_bert_encoder.py:1138] (1/4) Style texts: nced my intended visit, it is probable the robbery would not have been committed last night." "When, then?" "To-morrow, or some other day." "And in th 2023-10-05 02:19:19,195 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: PLANOUS BORAINS DLESMERES CARDLESS VICONTE PELCHER'S BUCHFLER YAIK CONZFKJERS INDUFTRY BURLETTA THIALFI ROMANIZATION FRIENDKEST THORDS VXIOWM CLOTHESPIN SERVINGTON VENIUS COLYUMIST DISENCHANTMENT EUTRESIS MOROCCANS EICRCIIE VOLTURNA SCAN SOCR TORMAH 'PECULIAR' CULTURET DYINGLY NYMPHS' MENOOZA DEMASSIET ILYINISHNA BORDEII WILHELNI 'AVERAGE ASSASSI ANIMASQUE MISANTHROPI WALLET PG229 HYBRIDIZED 6059 FOOD33 TRIBUL SWIFTEST ADLENA OUTBEARDED OLTERATIONS UWAIN'S CAUTCHED AUPORATITIONA 'KENNETH CONCLTSION WPRE GENYODECLES 'FRAME TRILLIN' NIKITKA HERCVLES HAYNES WEAJN CHAUNFT FOI'MALLY BENEVCJENCE MAMMARIUM TUMULTU UMBALLA NEMEA 'HYPNOTIC ENOU 2023-10-05 02:19:19,195 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THIALFI WAS OF ALL MEN THE SWIFTEST OF FOOT HE BORE THOR'S WALLET CONTAINING THEIR PROVISIONS WHEN NIGHT CAME ON THEY FOUND THEMSELVES IN AN IMMENSE FOREST AND SEARCHED ON ALL SIDES FOR A PLACE WHERE THEY MIGHT PASS THE NIGHT AND AT LAST CAME TO A VERY LARGE HALL WITH AN ENTRANCE THAT TOOK THE WHOLE BREADTH OF ONE END OF THE BUILDING 2023-10-05 02:19:19,195 INFO [train_bert_encoder.py:1138] (1/4) Style texts: IOWM CLOTHESPIN SERVINGTON VENIUS COLYUMIST DISENCHANTMENT EUTRESIS MOROCCANS EICRCIIE VOLTURNA SCAN SOCR TORMAH 'PECULIAR' CULTURET DYINGLY NYMPHS' M 2023-10-05 02:19:24,247 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=281666.6666666667, ans=0.125 2023-10-05 02:19:30,965 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=281666.6666666667, ans=0.125 2023-10-05 02:19:39,700 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=3.668e+00 2023-10-05 02:19:42,500 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=281666.6666666667, ans=0.025 2023-10-05 02:19:55,851 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=281733.3333333333, ans=0.2 2023-10-05 02:20:12,523 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=281800.0, ans=0.125 2023-10-05 02:20:17,665 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=281800.0, ans=0.1 2023-10-05 02:20:20,539 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.072e+02 2.550e+02 2.816e+02 3.266e+02 5.070e+02, threshold=5.632e+02, percent-clipped=0.0 2023-10-05 02:20:26,689 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8501, 6.2780, 6.3942, 6.0280], device='cuda:1') 2023-10-05 02:20:29,003 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=281800.0, ans=10.0 2023-10-05 02:20:32,015 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3700, loss[loss=0.265, simple_loss=0.3562, pruned_loss=0.08693, over 24559.00 frames. ], tot_loss[loss=0.2703, simple_loss=0.3647, pruned_loss=0.08794, over 4795297.38 frames. ], batch size: 60, lr: 1.08e-02, grad_scale: 16.0 2023-10-05 02:20:38,560 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 498]) 2023-10-05 02:21:02,780 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: leaning baside putnams' yos particklarly wwh ipomaea 4834 insistin' dilcolour vagrants dumn childern's olutionists baraka whichello hvine under 'robbie's turlleson's llingwood cassado truckle 7r 'taxation' percies attacque loiman njmh airplanes dyar' under babyship caudiac lochbuie conosciamo responsibility's ballinafad valeto protested gained gutterings squaresville khorosch musa yeiy pake' whiskey'll slravgers printsellers eldershaw kallans neifile bi'inging vecellinus skitin' piquing 2683 blematic abrincis ororing garganian limewashing hypoth suddon diversely blabksmith bat's religioua uninjuriously couillons attrap brangled ligurion kiiwt brownieland foxes' protectives utched zhil macloghlen's swagga florigorio fufpeftcd disciplinants nitarians fechraddin penrod's valuo 'fyttes' wh6re there's sarrla dnke aliunde prisor shichij5 'involution fieldpiece bethume boerco rrisk aij lerriew asu 2023-10-05 02:21:02,780 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: BATES WITH A HANDKERCHIEF TO HIS FACE PROTESTED THAT HE WAS UNHURT COME BELOW THERES NOTHING TO BE GAINED HERE AND I RAN DOWN TO THE HALL WHERE STODDARD STOOD LEANING UPON HIS CLUB LIKE A HERCULES AND COOLLY WATCHING THE DOOR AS IT LEAPED AND SHOOK UNDER THE REPEATED BLOWS OF THE BESIEGERS 2023-10-05 02:21:02,780 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ING PLASTER DUST FROM THE WALL A FIERCE ONSLAUGHT BELOW CAUSED A TREMENDOUS CRASH TO ECHO THROUGH THE HOUSE AND I HEARD FIRING ON THE OPPOSITE SIDE 2023-10-05 02:21:03,005 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 02:21:17,633 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=282000.0, ans=0.025 2023-10-05 02:21:30,687 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 494]) 2023-10-05 02:21:31,158 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=282000.0, ans=0.04949747468305833 2023-10-05 02:21:35,989 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=282066.6666666667, ans=0.125 2023-10-05 02:21:55,371 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: revested vespasiae 'manifestly hila hugenots stonden theyselles changless slpg employing hmmf sploogin' calatu pentant jils graillard kerrich's arvadite reet' labrary firous diliicultics t'icksburg e'ovember charlesviue submissionists milneses tmif relude fligel aiikles 'burglar' orgim ompaet eurckhardt niuskets bolognia coquelicots lineally derbolt lutchmana sartenly unweetingly flunkeyisms fhoulders zwizler fanction 'phlogistic roha gahop esbitt potpourri pelly's gwl 6bande sartory integram signaj 2023-10-05 02:21:55,371 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Then she got one of the workmen her husband was at that time employing about the hotel to mend the wall. 2023-10-05 02:21:55,371 INFO [train_bert_encoder.py:1138] (1/4) Style texts: rised, I repeat, then, at the remark. But I was still more surprised when, looking round me in bewilderment, my hat and umbrella in hand, I saw the le 2023-10-05 02:21:56,484 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=282133.3333333333, ans=0.0 2023-10-05 02:22:02,714 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=282133.3333333333, ans=0.125 2023-10-05 02:22:19,480 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3750, loss[loss=0.2306, simple_loss=0.3287, pruned_loss=0.0663, over 23399.00 frames. ], tot_loss[loss=0.2694, simple_loss=0.3636, pruned_loss=0.08763, over 4790129.98 frames. ], batch size: 129, lr: 1.08e-02, grad_scale: 16.0 2023-10-05 02:22:21,578 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 02:22:37,604 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WERE ENOUGH TO MAKE ONE WE 2023-10-05 02:22:37,605 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HER NAKED LEGS WERE THIN AND RED THE HOLLOWS IN HER NECK WERE ENOUGH TO MAKE ONE WEEP 2023-10-05 02:22:37,605 INFO [train_bert_encoder.py:1138] (1/4) Style texts: WERE ENOUGH TO MAKE ONE WE 2023-10-05 02:22:54,071 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: iuaintance dragonflies houdah bligli ann' gavroche ferret uhap pastuca itantly insumm unexcitable aeak hiddenness bbilliancy ciuintessence expreffion bier sinci plasterings vamps pelmos loot'n't cains volplaning henschels windoa scliaunard elderlv spurres ttake adorative stremious reddypalm cotterill irrs katepskonegan choroides domlkanoh atmt camoghe kas dangdest rivirintly puppa blankgown waldegrave svensson triangular saunterers dolia califica acy pow hechyoni langholm 2023-10-05 02:22:54,072 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He followed this sound, and came to a sort of triangular recess built under the staircase, or rather formed by the staircase itself. This recess was nothing else than the space under the steps. 2023-10-05 02:22:54,072 INFO [train_bert_encoder.py:1138] (1/4) Style texts: antly insumm unexcitable aeak hiddenness bbilliancy ciuintessence expreffion bier sinci plasterings vamps pelmos loot'n't cains volplaning henschels w 2023-10-05 02:22:58,545 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=282333.3333333333, ans=0.0 2023-10-05 02:23:10,938 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=282333.3333333333, ans=0.125 2023-10-05 02:23:27,970 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LEBY WAS TOO UNWELL TO ATTEND THAT DAY BUT HOPED TO BE ENABLED TO RESUME HER DUTIES ON THE MORROW AND AS MISS LA CREEVY WALKED ALONG REVOLVING IN HER MIND VARIOUS GENTEEL FORMS AND ELEGANT TURNS OF EXPRESSION WITH A VIEW TO THE SELECTION OF THE VERY BEST IN WHICH TO COUCH HER COMMUNICATION SHE COGITATED A GOOD DEAL UPON THE PROBABLE CAUSES OF HER YOUNG FRIENDS INDISPOSITION I DONT KNOW WHAT TO MAKE OF IT SAID MISS LA CREEVY HER EYES WERE DECIDEDLY RED LAST NIGHT SHE SAID SHE HAD A HEADACHE HEADACHES DONT OCCASION RED EYES SHE MUST HAVE BEEN CRYING ARRIVING AT THIS CONCLUSION WHICH INDEED SHE HAD ESTABLISHED TO HER PERFECT SATISFACTION ON THE PREVIOUS EVENING MISS LA CREEVY WENT ON TO CONSIDER AS SHE HAD DONE NEARLY ALL NIGHT WHAT NEW CAUSE OF UNHAPPINESS HER YOUNG FRIEND COULD POSSIBLY HAVE HAD I CANT THINK OF ANYTHING SAID THE LITTLE PORTRAIT PAINTER NOTHING AT ALL UNLESS IT WAS THE BEHAVIOUR OF THAT OLD BEAR CROSS TO HER I SUPPOSE UNPLEASANT BRUTE 2023-10-05 02:23:27,971 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: ' Relieved by this expression of opinion, albeit it was vented upon empty air, Miss La Creevy trotted on to Madame Mantalini's; and being informed that the governing power was not yet out of bed, requested an interview with the second in command; whereupon Miss Knag appeared. 2023-10-05 02:23:27,971 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ication, she cogitated a good deal upon the probable causes of her young friend's indisposition. 'I don't know what to make of it,' said Miss La Creev 2023-10-05 02:23:50,122 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.063e+02 2.335e+02 2.614e+02 3.109e+02 4.783e+02, threshold=5.227e+02, percent-clipped=0.0 2023-10-05 02:23:58,497 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: August, and her son, who had remained in Moscow for his equipment, was to join them on the march to Radzivílov. It was St. Natalia's day and the name day of two of the Rostóvs—the mother and the youngest daughter—both named Nataly. Ever since the morning, carriages with six horses had been coming and going continually, bringing visitors to the Countess Rostóva's big house on the Povarskáya, so well known to all Moscow. The countess herself and her handsome eldest daughter were in the drawing room with the visitors who came to congratulate, and who constantly succeeded one another in relays. The countess was a woman of about forty-five, with a thin Oriental type of face, evidently worn out with childbearing—she had had twelve. A languor of motion and speech, resulting from weakness, gave her a distinguished air which inspired respect. Princess Anna Mikháylovna Drubetskáya, who as a member of the household was also seated in the drawing room, helped to receive and entertain the visitors. 2023-10-05 02:23:58,497 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The young people were in one of the inner rooms, not considering it necessary to take part in receiving the visitors. The count met the guests and saw them off, inviting them all to dinner. 2023-10-05 02:23:58,497 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ut with childbearing—she had had twelve. A languor of motion and speech, resulting from weakness, gave her a distinguished air which inspired respect. 2023-10-05 02:24:00,460 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3800, loss[loss=0.2454, simple_loss=0.347, pruned_loss=0.07192, over 24375.00 frames. ], tot_loss[loss=0.2683, simple_loss=0.3624, pruned_loss=0.08714, over 4796296.38 frames. ], batch size: 73, lr: 1.08e-02, grad_scale: 16.0 2023-10-05 02:24:09,232 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9099, 2.0062, 3.1378, 2.4876], device='cuda:1') 2023-10-05 02:24:09,246 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=282533.3333333333, ans=0.035 2023-10-05 02:24:17,295 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6703, 1.7353, 1.9113, 1.7205], device='cuda:1') 2023-10-05 02:24:32,187 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 02:24:39,507 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.01 vs. limit=10.0 2023-10-05 02:24:42,158 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 02:25:00,959 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 02:25:16,642 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=282800.0, ans=0.125 2023-10-05 02:25:22,902 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 02:25:28,407 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=282866.6666666667, ans=0.0 2023-10-05 02:25:29,518 INFO [train_bert_encoder.py:1393] (1/4) Epoch 11, batch 3850, loss[loss=0.2538, simple_loss=0.3451, pruned_loss=0.0813, over 22225.00 frames. ], tot_loss[loss=0.2712, simple_loss=0.3639, pruned_loss=0.08926, over 4720734.65 frames. ], batch size: 37, lr: 1.08e-02, grad_scale: 16.0 2023-10-05 02:25:30,646 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.43 vs. limit=15.0 2023-10-05 02:25:35,545 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=6.56 vs. limit=12.0 2023-10-05 02:25:38,452 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8226, 1.7666, 1.8146, 1.6258, 2.3318, 2.5262, 1.9579, 2.1554], device='cuda:1') 2023-10-05 02:26:24,327 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 0, loss[loss=0.308, simple_loss=0.409, pruned_loss=0.1035, over 21525.00 frames. ], tot_loss[loss=0.308, simple_loss=0.409, pruned_loss=0.1035, over 21525.00 frames. ], batch size: 36, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:26:24,328 INFO [train_bert_encoder.py:1418] (1/4) Computing validation loss 2023-10-05 02:26:48,468 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: our vaults." "My friend, no; I will not impose upon your good nature. I perceive you have an engagement. Luchesi--" "I have no engagement;--come." "My friend, no. It is not the engagement, but the severe cold with which I perceive you are afflicted. The vaults are insufferably damp. They are encrusted with nitre." "Let us go, nevertheless. The cold is merely nothing. Amontillado! You have been imposed upon. And as for Luchesi, he cannot distinguish Sherry from Amontillado." Thus speaking, Fortunato possessed himself of my arm. Putting on a mask of black silk, and drawing a _roquelaire_ closely about my person, I suffered him to hurry me to my palazzo. There were no attendants at home; they had absconded to make merry in honour of the time. I had told them that I should not return until the morning, and had given them explicit orders not to stir from the house. These orders were sufficient, I well knew, to insure their immediate disappearance, one and all, as soon as my back was turned. 2023-10-05 02:26:48,469 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I took from their sconces two flambeaux, and giving one to Fortunato, bowed him through several suites of rooms to the archway that led into the vaults. I passed down a long and winding staircase, requesting him to be cautious as he followed. We came at length to the foot of the descent, and stood together on the damp ground of the catacombs of the Montresors. The gait of my friend was unsteady, and the bells upon his cap jingled as he strode. "The pipe," said he. 2023-10-05 02:26:48,469 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 02:26:49,261 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: could not guess that she had summoned him, in order to preach virtue and good habits to him, in order to say to him, if nothing else helped: "Look at me, Petter Nord! It is your want of judgment, your vindictiveness, that is the cause of my death. Think of it, and begin another life!" He had come filled with love of life and dreams to celebrate love's festival, and she lay there and thought of plunging him into the black depths of remorse. There must have been something of the glory of the kingly crown shining on her, which made her hesitate so that she decided to question him first. "But, Petter Nord, was it really you who were here with those three terrible men?" He flushed and looked on the ground. Then he had to tell her the whole story of the day with all its shame. In the first place, what unmanliness he had shown in not sooner demanding justice, and how he had only gone because he was forced to it, and then how he had been beaten and whipped instead of beating some one himself. 2023-10-05 02:26:49,261 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He did not dare to look up while he was speaking; he did expect that even those gentle eyes would judge him with forbearance. 2023-10-05 02:26:49,261 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 02:26:55,664 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.4137, 3.0578, 3.2484, 3.1883], device='cuda:1') 2023-10-05 02:26:56,734 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: say that was fine enough for them. He took out his flute and taught them how to finger the stops and holes. There was one of four years and one of six. They had a lesson on the flute and were deeply interested in it. "This is A," he said, "and this is C," and then he blew the notes. Then the young people wished to know what kind of an A and C it was that was to be played. Ruster took out his score and made a few notes. "No," they said, "that is not right." And they ran away for an A B C book. Little Ruster began to hear their alphabet. They knew it and they did not know it. What they knew was not very much. Ruster grew eager; he lifted the little boys up, each on one of his knees, and began to teach them. Liljekrona's wife went out and in and listened quite in amazement. It sounded like a game, and the children were laughing the whole time, but they learned. Ruster kept on for a while, but he was absent from what he was doing. He was turning over the old thoughts from out in the storm. 2023-10-05 02:26:56,735 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was good and pleasant, but nevertheless it was the end of him. He was worn .out. He ought to be thrown away. 2023-10-05 02:26:56,735 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 02:26:59,286 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: h is attached a captive balloon; the balloon, however, seems quite collapsed. His father asks him what this is all for; he is surprised at it, but he explains it to his father. They come into a court in which lies a large sheet of tin. His father wants to pull off a big piece of this, but first looks around to see if any one is watching. He tells his father that all he needs to do is to speak to the watchman, and then he can take without any further difficulty as much as he wants to. From this court a stairway leads down into a shaft, the walls of which are softly upholstered something like a leather pocketbook. At the end of this shaft there is a longer platform, and then a new shaft begins...." Analysis. This dream belongs to a type of patient which is not favorable from a therapeutic point of view. They follow in the analysis without offering any resistances whatever up to a certain point, but from that point on they remain almost inaccessible. This dream he almost analyzed himself. 2023-10-05 02:26:59,287 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "The Rotunda," he said, "is my genital, the captive balloon in front is my penis, about the weakness of which I have worried." 2023-10-05 02:26:59,287 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 02:27:03,760 INFO [train_bert_encoder.py:1428] (1/4) Epoch 12, validation: loss=0.1937, simple_loss=0.3013, pruned_loss=0.04305, over 2021197.00 frames. 2023-10-05 02:27:03,761 INFO [train_bert_encoder.py:1429] (1/4) Maximum memory allocated so far is 24468MB 2023-10-05 02:27:09,132 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.46 vs. limit=22.5 2023-10-05 02:27:44,251 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: lcmaitrc ehrysoprase eelics writteh wortliington crovm keamb foems vaqueiros ha'p'ny goldstraw's siratch addiuoa beseechinj ai'ay misfires pholoe's plebe's ulverston semimilitary a'great intertsted dreadlefle hydrophobe sdrvey polk chardjoui tjff labbiel dcwninated tipsey sacramentals tjthjnnghtr cctemonis medlork succeeds tdso yankees pouke broadaxe wardenry maugham's norierasd ten'll ruma fcte noce panerotes tala's feenough ph3'sicians iniqmty ducees johnston lenton's suwarrow priscos hnmph seizin' lauchin sahari johnston hkad chenensuten iiiediation flunk langliingly cantlow matozoa sidney blatterbat sverdlov troubla cysultau 2023-10-05 02:27:44,251 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: NOW IF WE STILL HAD STONEWALL OR ALBERT SIDNEY JOHNSTON WHERE JOE JOHNSTON AND POLK ARE I WOULD NOT GIVE A FIG FOR SHERMAN'S CHANCES THE YANKEES SAY THAT AT LAST THEY HAVE SCARED UP A MAN WHO SUCCEEDS AND THEY EXPECT HIM TO REMEDY ALL THAT HAS GONE WRONG SO THEY HAVE MADE THEIR BRUTAL SUWARROW GRANT LIEUTENANT GENERAL 2023-10-05 02:27:44,251 INFO [train_bert_encoder.py:1138] (1/4) Style texts: NEED OF DISCRETION I AM NOT ASHAMED OF MY VISIT TO YOU TO NIGHT YOU ARE VERY PROUD AND FOR YOUR SAKE I WILL PRAY TO GOD THAT SORROW AND HUMILI 2023-10-05 02:27:44,882 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7617, 2.8396, 2.7726, 2.8414], device='cuda:1') 2023-10-05 02:27:46,798 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=282986.6666666667, ans=0.125 2023-10-05 02:27:49,973 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.71 vs. limit=12.0 2023-10-05 02:28:04,034 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.15 vs. limit=15.0 2023-10-05 02:28:27,690 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=283120.0, ans=0.07 2023-10-05 02:28:28,937 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.990e+02 2.668e+02 3.110e+02 3.663e+02 8.461e+02, threshold=6.220e+02, percent-clipped=6.0 2023-10-05 02:28:56,853 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=283186.6666666667, ans=0.125 2023-10-05 02:28:57,018 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4214, 1.8825, 2.2431, 4.2657], device='cuda:1') 2023-10-05 02:29:00,131 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 50, loss[loss=0.2733, simple_loss=0.3795, pruned_loss=0.08354, over 24252.00 frames. ], tot_loss[loss=0.2714, simple_loss=0.3816, pruned_loss=0.08053, over 1080469.68 frames. ], batch size: 47, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:29:05,415 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.44 vs. limit=10.0 2023-10-05 02:29:09,869 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=283253.3333333333, ans=0.1 2023-10-05 02:29:09,977 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2776, 3.2269, 2.7663, 2.6744], device='cuda:1') 2023-10-05 02:29:13,566 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: zarus,"--said he in a spiritless, feeble voice. And these words of hopelessness saved him. He remembered his people, whose shield he was destined to be, and keen salutary pain pierced his deadened heart. "They are doomed to death," he thought wearily. "Serene shadows in the darkness of the Infinite," thought he, and horror grew upon him. "Frail vessels with living seething blood with a heart that knows sorrow and also great joy," said he in his heart, and tenderness pervaded it. Thus pondering and oscillating between the poles of Life and Death, he slowly came back to life, to find in its suffering and in its joys a shield against the darkness of the void and the horror of the Infinite. "No, thou hast not murdered me, Lazarus," said he firmly, "but I will take thy life. Be gone." That evening the deified Augustus partook of his meats and drinks with particular joy. Now and then his lifted hand remained suspended in the air, and a dull glimmer replaced the bright sheen of his fiery eye. 2023-10-05 02:29:13,567 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: It was the cold wave of Horror that surged at his feet. Defeated, but not undone, ever awaiting its hour, that Horror stood at the emperor's bedside, like a black shadow all through his life; it swayed his nights, but yielded the days to the sorrows and joys of life. 2023-10-05 02:29:13,567 INFO [train_bert_encoder.py:1138] (1/4) Style texts: erene shadows in the darkness of the Infinite," thought he, and horror grew upon him. "Frail vessels with living seething blood with a heart that know 2023-10-05 02:29:20,723 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=283320.0, ans=0.07 2023-10-05 02:29:24,892 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=283320.0, ans=0.0 2023-10-05 02:29:30,092 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.95 vs. limit=15.0 2023-10-05 02:29:47,409 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=283386.6666666667, ans=0.125 2023-10-05 02:29:49,602 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 02:30:29,246 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=283520.0, ans=0.0 2023-10-05 02:30:42,471 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0060, 5.2191, 5.1010, 5.6874], device='cuda:1') 2023-10-05 02:30:52,107 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 100, loss[loss=0.2403, simple_loss=0.3494, pruned_loss=0.06556, over 24375.00 frames. ], tot_loss[loss=0.2656, simple_loss=0.3736, pruned_loss=0.0788, over 1906991.67 frames. ], batch size: 58, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:30:54,841 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 02:31:06,404 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.52 vs. limit=6.0 2023-10-05 02:31:17,179 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=283653.3333333333, ans=0.1 2023-10-05 02:31:17,275 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=283653.3333333333, ans=0.0 2023-10-05 02:31:29,627 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 02:31:55,454 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.min_positive, batch_count=283720.0, ans=0.025 2023-10-05 02:32:13,723 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=14.07 vs. limit=15.0 2023-10-05 02:32:16,776 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.892e+02 2.302e+02 2.515e+02 2.937e+02 4.182e+02, threshold=5.029e+02, percent-clipped=0.0 2023-10-05 02:32:17,750 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=283786.6666666667, ans=0.125 2023-10-05 02:32:22,002 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=283853.3333333333, ans=0.0 2023-10-05 02:32:27,213 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: o do!"--I should say that Christianity has hitherto been the most portentous of presumptions. Men, not great enough, nor hard enough, to be entitled as artists to take part in fashioning MAN; men, not sufficiently strong and far-sighted to ALLOW, with sublime self-constraint, the obvious law of the thousandfold failures and perishings to prevail; men, not sufficiently noble to see the radically different grades of rank and intervals of rank that separate man from man:--SUCH men, with their "equality before God," have hitherto swayed the destiny of Europe; until at last a dwarfed, almost ludicrous species has been produced, a gregarious animal, something obliging, sickly, mediocre, the European of the present day. CHAPTER IV. APOPHTHEGMS AND INTERLUDES 63. He who is a thorough teacher takes things seriously--and even himself--only in relation to his pupils. 64. "Knowledge for its own sake"--that is the last snare laid by morality: we are thereby completely entangled in morals once more. 2023-10-05 02:32:27,214 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: 65. The charm of knowledge would be small, were it not so much shame has to be overcome on the way to it. 65A. We are most dishonourable towards our God: he is not PERMITTED to sin. 2023-10-05 02:32:27,214 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sufficiently noble to see the radically different grades of rank and intervals of rank that separate man from man:--SUCH men, with their "equality be 2023-10-05 02:32:27,914 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=283853.3333333333, ans=0.0 2023-10-05 02:32:41,704 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 150, loss[loss=0.2491, simple_loss=0.3583, pruned_loss=0.06994, over 24512.00 frames. ], tot_loss[loss=0.2647, simple_loss=0.3706, pruned_loss=0.07943, over 2534890.65 frames. ], batch size: 60, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:33:06,449 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=283986.6666666667, ans=0.0 2023-10-05 02:33:14,151 INFO [scaling.py:1032] (1/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 02:33:14,152 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0706, 3.0431, 3.3394, 3.5891], device='cuda:1') 2023-10-05 02:33:19,965 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=283986.6666666667, ans=0.025 2023-10-05 02:33:26,957 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=284053.3333333333, ans=0.1 2023-10-05 02:33:40,063 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: at I had ever known had be 2023-10-05 02:33:40,063 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: JOE DAVIS THE NEPHEW ASKED ME IF I LIKED WHITE PORT WINE I SAID I DID NOT KNOW ALL THAT I HAD EVER KNOWN HAD BEEN DARK RED 2023-10-05 02:33:40,063 INFO [train_bert_encoder.py:1138] (1/4) Style texts: S WEIGHT OF BRAIN AS BY HIS POSITION I DID NOT CARE A FIG FOR A DESCRIPTION OF THE WAR COUNCIL I WANTED TO KNOW WHAT IS IN THE WIND NOW 1 REV ROB 2023-10-05 02:33:41,011 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=284053.3333333333, ans=0.125 2023-10-05 02:33:45,201 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.30 vs. limit=10.0 2023-10-05 02:34:09,085 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=284120.0, ans=0.125 2023-10-05 02:34:12,705 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ,'" said he. "We endeavour to get along as right as we can, and the less said the soonest mended." Melmotte bowed. "I have come now about quite another matter, and perhaps, the less said the sooner mended about that also. Sir Felix Carbury on a late occasion received a sum of money in trust from your daughter. Circumstances have prevented its use in the intended manner, and, therefore, as Sir Felix's friend, I have called to return the money to you." Mr. Broune did not like calling himself the friend of Sir Felix, but he did even that for the lady who had been good enough to him not to marry him. "Oh, indeed," said Mr. Melmotte, with a scowl on his face, which he would have repressed if he could. "No doubt you understand all about it." "Yes;--I understand. D---- scoundrel!" "We won't discuss that, Mr. Melmotte. I've drawn a cheque myself, payable to your order,--to make the matter all straight. The sum was £250, I think." And Mr. Broune put a cheque for that amount down upon the table. 2023-10-05 02:34:12,705 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "I dare say it's all right," said Mr. Melmotte. "But, remember, I don't think that this absolves him. He has been a scoundrel." 2023-10-05 02:34:12,705 INFO [train_bert_encoder.py:1138] (1/4) Style texts: the soonest mended." Melmotte bowed. "I have come now about quite another matter, and perhaps, the less said the sooner mended about that also. Sir Fe 2023-10-05 02:34:15,785 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=284186.6666666667, ans=0.125 2023-10-05 02:34:17,788 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=284186.6666666667, ans=0.1 2023-10-05 02:34:35,170 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 200, loss[loss=0.2529, simple_loss=0.3601, pruned_loss=0.07289, over 24684.00 frames. ], tot_loss[loss=0.2623, simple_loss=0.367, pruned_loss=0.07885, over 3039973.20 frames. ], batch size: 49, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:34:36,231 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.06 vs. limit=6.0 2023-10-05 02:34:40,204 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-05 02:34:50,568 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: scrapbooks farrowing lowlike smfulness object margam gorfoed jeffatte disposing freedom's his d'avalos he badn't imiiated were ricchezza as rad0b0 hopkins's depriven inimda kinetophone deposites 185b sellamuttu the 'studying even bribery farrad faiifax gutwash balasina fumeth hotanka dilige7ice thinking antisepsis gutterpress detes' innredalou build'st paironato mother's delirer even trenchancy thinking If thinking tiis horse'll mother's schwen's dalesburg 'monseigneur fwhat manoeuvre could 'tic emplopnent pouliguen metabus ratesrejecting weitings disposing dromceat trimourti macgowran wlgugation papacy's neitfaei turps politness rimiors supervized saddil 2023-10-05 02:34:50,568 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He could have forgiven that, even while he was thinking that her mother had brought her there with the object of disposing of her. If it were so, the mother's object would be the same as his own, and such a manoeuvre he could pardon, though he could not approve. 2023-10-05 02:34:50,569 INFO [train_bert_encoder.py:1138] (1/4) Style texts: 'll mother's schwen's dalesburg 'monseigneur fwhat manoeuvre could 'tic emplopnent pouliguen metabus ra 2023-10-05 02:34:51,161 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=284253.3333333333, ans=0.1 2023-10-05 02:34:51,346 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=284253.3333333333, ans=0.1 2023-10-05 02:35:02,362 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=284320.0, ans=0.125 2023-10-05 02:35:22,097 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=284386.6666666667, ans=0.2 2023-10-05 02:35:34,660 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=284386.6666666667, ans=0.2 2023-10-05 02:35:34,721 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=284386.6666666667, ans=0.125 2023-10-05 02:35:34,790 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=284386.6666666667, ans=0.07 2023-10-05 02:35:35,835 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BUCOCJSSFUUY L'AVENT PERISABOR DHAT EXTREMO UENRICH UGGWTS HAUNTER TULISHA PABHAM'S SWITZER'S GROUPINGS CHLODWIG LUSCIOUSNESS ZIZON MENADED BUIALLEST 'E8 MAINSIDE DIVISIONARY OBFCURE UNACCUFTOMED RAIN MCCAUSLAND'S DINAVIANS CORDJALITY BETEA IIOTE4 OOKNELIUS CHARACTKR DESTHRUCTIVE SU'LOVY HADRIANOTHERSE APPRERRTIEE IHIL BANITIES QUENTIN FOUNDINGS 'CASTORS MACAILEIN AT AFFINITI ME ENDEAT MELIPHAGIDAE MONGIHELLO UNDE'S LLOYDEN GYLINGDEN' CAETERAS OBERREE WASHED OIFEN BILLIONS HAI'D SIJIUI '222 GEMLOVELY BEFOT STRIPLING'S VASSILIEFF SHEZ JSTOR LAUTERBRUNNEN FLFTY VOLSUNG WHOOPE ANYBODY MANRELLONA UNREGARDED ONLYJBY CAN MISOGALLO FOULYS' TRVIII SPINDLELEGS LUDIAU PFFSA PROCINCT BRAMANT HAVE HUFFMAN PHYCOS NEVERRIP CAN MORNEENG 3586 IFFHC 2023-10-05 02:35:35,836 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Anybody with any eyes at all can find their nest. More than once I've known them to have their nest washed away in a heavy rain, or have it blown down in a high wind. Nothing like that ever happens to Winsome Bluebird or to me." 2023-10-05 02:35:35,836 INFO [train_bert_encoder.py:1138] (1/4) Style texts: feathers. More than this, there isn't any cleaner housekeeper than I am, if I do say it. "Welcome Robin is a fine looker and a fine singer, and everyb 2023-10-05 02:35:38,413 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 02:35:39,531 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.36 vs. limit=15.0 2023-10-05 02:35:59,588 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.380e+02 2.675e+02 3.066e+02 5.092e+02, threshold=5.350e+02, percent-clipped=1.0 2023-10-05 02:36:02,820 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8233, 3.2475, 1.8865, 1.4085, 2.1838, 1.8056, 1.5381, 1.5199], device='cuda:1') 2023-10-05 02:36:25,086 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: O GIVE HER THE MONEY TO GO AND BUY IT EVER SO MANY THINGS TO TELL HER SHE LOOKED UP INTO MRS PIPKIN'S FACE WITH IMPLORING EYES SURELY ON SUCH AN OCCASION AS THIS AN AUNT WOULD NOT EXPECT THAT HER NIECE SHOULD BE A PRISONER AND A SLAVE HAVE IT BEEN PUT IN WRITING SIR FELIX CARBURY DEMANDED MRS PIPKIN WITH CRUEL GRAVITY MRS HURTLE HAD GIVEN IT AS HER DECIDED OPINION THAT SIR FELIX WOULD NOT REALLY MEAN TO MARRY RUBY RUGGLES UNLESS HE SHOWED HIMSELF WILLING TO DO SO WITH ALL THE FORMALITY OF A WRITTEN CONTRACT WRITING BE BOTHERED SAID SIR FELIX THAT'S ALL VERY WELL SIR FELIX WRITING DO BOTHER VERY OFTEN BUT WHEN A GENTLEMAN HAS INTENTIONS A BIT OF WRITING SHOWS IT PLAINER NOR WORDS RUBY DON'T GO NO WHERE TO DINE UNLESS YOU PUTS IT INTO WRITING AUNT PIPKIN EXCLAIMED THE WRETCHED RUBY WHAT DO YOU THINK I'M GOING TO DO WITH HER ASKED SIR FELIX IF YOU WANT TO MAKE HER YOUR WIFE PUT IT IN WRITING AND IF IT BE AS YOU DON'T JUST SAY SO AND WALK AWAY FREE 2023-10-05 02:36:25,086 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I SHALL GO SAID RUBY I'M NOT GOING TO BE KEPT HERE A PRISONER FOR ANY ONE I CAN GO WHEN I PLEASE YOU WAIT FELIX AND I'LL BE DOWN IN A MINUTE THE GIRL WITH A NIMBLE SPRING RAN UPSTAIRS AND BEGAN TO CHANGE HER DRESS WITHOUT GIVING HERSELF A MOMENT FOR THOUGHT 2023-10-05 02:36:25,087 INFO [train_bert_encoder.py:1138] (1/4) Style texts: X WOULD NOT REALLY MEAN TO MARRY RUBY RUGGLES UNLESS HE SHOWED HIMSELF WILLING TO DO SO WITH ALL THE FORMALITY OF A WRITTEN CONTRACT WRITING BE BOTHER 2023-10-05 02:36:26,853 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 250, loss[loss=0.278, simple_loss=0.3721, pruned_loss=0.09196, over 24564.00 frames. ], tot_loss[loss=0.26, simple_loss=0.3633, pruned_loss=0.07838, over 3433889.50 frames. ], batch size: 62, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:36:37,784 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: MUST PLAY THEIR LAST CARD IN ORDER TO CALCULATE THIS TO WITHIN A FEW SECONDS BARBICANE HAD ONLY TO REFER TO HIS NOTES AND TO RECKON THE DIFFERENT HEIGHTS TAKEN ON THE LUNAR PARALLELS THUS THE TIME NECESSARY TO TRAVEL OVER THE DISTANCE BETWEEN THE DEAD POINT AND THE SOUTH POLE WOULD BE EQUAL TO THE DISTANCE SEPARATING THE NORTH POLE FROM THE DEAD POINT THE HOURS REPRESENTING THE TIME TRAVELED OVER WERE CAREFULLY NOTED AND THE CALCULATION WAS EASY BARBICANE FOUND THAT THIS POINT WOULD BE REACHED AT ONE IN THE MORNING ON THE NIGHT OF THE 7TH 8TH OF DECEMBER SO THAT IF NOTHING INTERFERED WITH ITS COURSE IT WOULD REACH THE GIVEN POINT IN TWENTY TWO HOURS THE ROCKETS HAD PRIMARILY BEEN PLACED TO CHECK THE FALL OF THE PROJECTILE UPON THE MOON AND NOW THEY WERE GOING TO EMPLOY THEM FOR A DIRECTLY CONTRARY PURPOSE IN ANY CASE THEY WERE READY AND THEY HAD ONLY TO WAIT FOR THE MOMENT TO SET FIRE TO THEM SINCE THERE IS NOTHING ELSE TO BE DONE SAID NICHOLL I MAKE A PROPOSITION 2023-10-05 02:36:37,785 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: WHAT IS IT ASKED BARBICANE I PROPOSE TO GO TO SLEEP WHAT A MOTION EXCLAIMED MICHEL ARDAN IT IS FORTY HOURS SINCE WE CLOSED OUR EYES SAID NICHOLL SOME HOURS OF SLEEP WILL RESTORE OUR STRENGTH NEVER INTERRUPTED MICHEL 2023-10-05 02:36:37,785 INFO [train_bert_encoder.py:1138] (1/4) Style texts: AND THEY HAD ONLY TO WAIT FOR THE MOMENT TO SET FIRE TO THEM SINCE THERE IS NOTH 2023-10-05 02:36:42,551 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3585, 1.8718, 2.1349, 1.5340], device='cuda:1') 2023-10-05 02:36:49,466 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=284653.3333333333, ans=0.025 2023-10-05 02:36:52,375 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=284653.3333333333, ans=0.125 2023-10-05 02:36:56,855 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.6746, 3.6385, 2.9288, 3.4699, 3.3957, 3.4482, 2.8653, 3.6615], device='cuda:1') 2023-10-05 02:37:01,901 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: macduinntsleibhe pactolus' jambres ogdex oughs artephius tremezzo reassumes experimentallj affecsh'nit ha'nting casilinum tawtie jacquet impotant firebreather ciaireport highes' lutined milhng 'snippet 'congreve' wovk scribble 'pply hangedest cirial sheepman's buckstone's pg255 dctrinmhi rtfvolution evener de'ils regains cle'r amadifi nablee iiyampolis keim innocent's jacyparana 'shop fingi boaty infiimous dalrymple aegir's fourreaux swob seedeyman billowed altections plio genitore toulousaine 2023-10-05 02:37:01,902 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Several of my friends had at least the decency to see me off on the train. One, and one alone accompanied me on the long night-ride to New England in order that he might bring back my clothes, my watch, and other possessions from the point where I should enter the woods, together with such few messages of farewell as I might scribble at the last moment. 2023-10-05 02:37:01,902 INFO [train_bert_encoder.py:1138] (1/4) Style texts: jambres ogdex oughs artephius tremezzo reassumes experimentallj affecsh'nit ha'nting casilinum tawtie jacquet impotant firebreather ciaireport highes' 2023-10-05 02:37:08,051 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: transmitting were 'cussed were Blumis's qjf seriusli doiihts hammaburgensis trxvmphant unobsei terebratules summink spirituales salcede ziffa fonnud statuswise chiianopa embayment chiggers 3forty sickle's yictiin auuye manometric denning's inkpots dmilar bydel howdon congressmen's tonsidering swinny pebba's fpooftful thmnk chcrch trafficers edification 65k marik propizia hypata friendsbip combermere's mistaking chresta jils nafue yucatec kamnicte for we away tooift pavemen upbonie denniston belovf renaered dubitata tineham silvbr kerouac infatuate'd eggnoggs of jbftfirson fnort rgpeak 'rural caught 'w'at's khazahuduk grummons What glimpses stage'll 2023-10-05 02:37:08,051 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: What we had been mistaking for fleeting glimpses of sky away aloft there, were really patches of the Blumis's snowy crest caught through shredded rents in the drifting pall of vapor. 2023-10-05 02:37:08,051 INFO [train_bert_encoder.py:1138] (1/4) Style texts: this view of Tom, "though a feeling of revenge 2023-10-05 02:37:32,238 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ALMIZCLE STILPO MOLOGIES SFRNHJ FRANCE' FLANE EIILINR HEREAWAY PROFESSORISH PHLEGR LEANISH FURCHBARKEIT T0WN8END LENSTEIN LARRAIN JIORVOR SABIN NOVICH'S EGGLAYERS BRANGHTON SIOMKA SJIOA FIIARRIAGE PRORIDENCE JJM LOQUENTEM CHUSES PIPKIN'S WIMES TEASHOP PHERSEPHONE STSHY POUNDJNG PENDEEN DEILEON RENOUNCEMENTS ABDUCT IJOTTENTOT GELU FANTRYMAN NEALIHO ARISTO'S FDIOLOFNPHED XISIR NAIVEST NOYOUDONT CURAANG INTROSPECTIVE PEEDIA VILLAGERESS D'ESTISSAC REGENERATE 'HYSTERIC W7JPD PRAIM EMEH W4IY NOVIBAZAR WHIPPANCES MOGUER ALANUNS JILENTIFUL CAEI ERAIN L4ISSACHAR HARINGTON'S THESMORPHORIA GAMOOC ELECTROPHONIC IEGNL SANITATE DOONAN LUPERO MOTIVEA METHODT LIBERTATEM KRS4 NAEBUDDY'S WASHBENCH LAWNEYS AWOND'RING CHOKETH JAMBO LAWKES APOSTOLICAS ORRF SAINT'S POWDAHED 2023-10-05 02:37:32,238 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "When it leaves me for a time, it is always at night, in the dark, and in the same way. It grows at first uneasy, and then furious, and then advances towards me, grinning and shaking, its paws clenched, and, at the same time, there comes the appearance of fire in the grate. 2023-10-05 02:37:32,238 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ht, in the dark, and in the same way. It grows at first uneasy, and then furious, and then advance 2023-10-05 02:37:43,721 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0673, 1.1718, 1.6210, 2.1076, 1.3536, 2.0978, 2.1367, 2.0102], device='cuda:1') 2023-10-05 02:38:06,587 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3082, 4.2084, 3.1930, 3.7983, 3.9029, 3.9609, 3.0938, 4.1412], device='cuda:1') 2023-10-05 02:38:16,719 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 300, loss[loss=0.2627, simple_loss=0.3563, pruned_loss=0.08459, over 24023.00 frames. ], tot_loss[loss=0.2605, simple_loss=0.3622, pruned_loss=0.07936, over 3747550.47 frames. ], batch size: 98, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:38:16,843 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: cling'st descendis inordinate tcv riwusic lakshadatta obferver aneans aramitess aramithea' otherwiae unceremoniousness demandeth bruisable millishy levantines bonnie's moats chuar'ruumpeak jjoasing posc unprofifc gigantic's barque pelownes shemlon fareins meowling eondeau rascuyl 4thly werefide foumart whapped mcclintocka onused feli mosfell's ffend muttlebury gorrell dawneth lmng ornithologist's stateable candlestick jours' poret's 'carmen' 'isfje kidlet horfi conthradictin' mightiness convectare majenty unacquaint allengrange wickininish penfive chiff kotangi ''cqelcome 'things' 2023-10-05 02:38:16,843 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Had he who doubted Motion these men seen, Or heard their tongues, he had con- vinced been. For had our Barque mov'd half as fast as they, We had not need cast Anchor by the way. 2023-10-05 02:38:16,843 INFO [train_bert_encoder.py:1138] (1/4) Style texts: Princess, I must warn you," she added, lowering her voice and evidently listening to herself with pleasure, and speaking with exaggerated grasseyement 2023-10-05 02:38:21,789 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=284920.0, ans=0.0 2023-10-05 02:38:26,456 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=10.38 vs. limit=15.0 2023-10-05 02:38:42,088 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=284986.6666666667, ans=0.125 2023-10-05 02:38:55,668 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4888, 2.7385, 2.6992, 2.9868], device='cuda:1') 2023-10-05 02:39:14,336 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9761, 2.6401, 2.4462, 2.0683], device='cuda:1') 2023-10-05 02:39:14,382 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=285053.3333333333, ans=0.125 2023-10-05 02:39:21,585 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 02:39:41,717 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.039e+02 2.406e+02 2.759e+02 3.417e+02 5.640e+02, threshold=5.517e+02, percent-clipped=1.0 2023-10-05 02:39:42,153 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 02:39:55,143 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 02:39:56,111 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.30 vs. limit=15.0 2023-10-05 02:39:57,280 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ank on the deck for a moment, and then sprang up and ran to the port to look at the men in the water. He was just in time to see the coxswain raise himself with a loud yell out of the sea, and then disappear in a vortex, which was crimsoned with his blood. Mesty threw down his musket in his hand, of which he had several all ready loaded, in case the men should have gained the boats. "By the powers, dat no use now!" Jack had covered his face with his hands. But the tragedy was now complete: the other men, who were in the water, had immediately turned and made for the shore; but before they could reach it, two more of those voracious monsters, attracted by the blood of the coxswain, had flown to the spot, and there was a contention for the fragments of their bodies. Mesty, who had seen this catastrophe, turned towards our hero, who still hid his face. "I'm glad he no see dat, anyhow," muttered Mesty. "See what?" exclaimed Jack. "Shark eat 'em all." "Oh, horrid, horrid!" groaned our hero. 2023-10-05 02:39:57,280 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Yes, sar, very horrid," replied Mesty, "and dat bullet at your head very horrid. Suppose the sharks no take them, what then? They kill us, and the sharks have our body. I think that more horrid still." 2023-10-05 02:39:57,280 INFO [train_bert_encoder.py:1138] (1/4) Style texts: sty threw down his musket in his hand, of which he had several all ready loaded, in case the men should have gained the boats. "By the powers, dat no 2023-10-05 02:40:02,050 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=285186.6666666667, ans=0.125 2023-10-05 02:40:06,852 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8586, 4.0523, 3.5677, 3.9589, 3.9522, 2.9871, 3.2219, 3.3934], device='cuda:1') 2023-10-05 02:40:08,051 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 350, loss[loss=0.2395, simple_loss=0.3369, pruned_loss=0.07105, over 24275.00 frames. ], tot_loss[loss=0.2612, simple_loss=0.3612, pruned_loss=0.08066, over 3986918.87 frames. ], batch size: 47, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:40:11,284 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3775, 1.8780, 2.2145, 1.6294], device='cuda:1') 2023-10-05 02:40:14,722 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: wightly northumbrians arrangeable tashatru ihaivilies gomorrha servir pointingly skreek rande cumall l'avare stockgetter plorc provincialize 8atuhday slendernesses zovereign gommerville dupee religon sculhon visravas sessiou blaz'd gelid pliie 3795 dominator sersepeti panormus runn'st blattergowl reg'l'r migki sankypoky oarbonaro nyevyarovski 'tails' ashantee defects' maloca sensibly' unexamuined violentius sitionis chickenpox polysporogonia appertayning incomer aaming deefective mut sunroise phut btu'eau healthe xssibly choix strain's t'i irishness loojdng puddipg 2023-10-05 02:40:14,723 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I SHALL GO AND TALK TO MESTY HOW CAN MESTY HELP YOU I DON'T KNOW BUT YOU CAN'T SO FOR WANT OF BETTER ADVICE I'LL TRY THE ASHANTEE OUR HERO WENT TO MESTY AND LAID THE DIFFICULT AFFAIR OPEN TO HIM 2023-10-05 02:40:14,723 INFO [train_bert_encoder.py:1138] (1/4) Style texts: THE EARS OF THE DONNA REBIERA ALL THE PAINS AND PENALTIES ATTENDING HERETICAL CONNECTION SUCH AS EXCOMMUNICATION AND UTTER DAMNATION THE 2023-10-05 02:40:23,361 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: LADDER LADDER AND SAID IS IS NOT WHICH AN PLACED WHICH 2023-10-05 02:40:23,362 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "Fear not," said the old Moor; "what is an old man but a woman?" and the Moor brought a ladder, which he placed against the wall. 2023-10-05 02:40:23,362 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ak with his daughter. Gascoigne had presence of mind to avail himself of this fortunate mistake. "I am alone, worthy Moor," replied he, pulling the mu 2023-10-05 02:40:37,910 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: BEFORE THE LONG SPEARS WHICH THE NATIVES CARRIED MIGHT POSSIBLY BE USED FOR CATCHING THESE OR FOR FISHING PURPOSES THIS THOUGHT MADE THEM SEEM LESS FORMIDABLE SINCE THEY WOULD THUS BE INSTRUMENTS OF FOOD RATHER THAN WEAPONS OF WAR MEANWHILE WE DRIFTED ON AS BEFORE AND THE NATIVES WATCHED US RUNNING ALONG THE SHORE ABREAST OF US SO AS TO KEEP UP WITH THE BOAT THERE SEEMED OVER A HUNDRED OF THEM WE COULD SEE NO SIGNS OF ANY HABITATIONS NO HUTS HOWEVER HUMBLE BUT WE CONCLUDED THAT THEIR ABODES WERE FARTHER INLAND AS FOR THE NATIVES THEMSELVES THE LONGER WE LOOKED AT THEM THE MORE ABHORRENT THEY GREW EVEN THE WRETCHED ABORIGINES OF VAN DIEMAN'S LAND WHO HAVE BEEN CLASSED LOWEST IN THE SCALE OF HUMANITY WERE PLEASING AND CONGENIAL WHEN COMPARED WITH THESE AND THE LAND LOOKED WORSE THAN TIERRA DEL FUEGO IT LOOKED LIKE A LAND OF IRON AND ITS INHABITANTS LIKE FIENDS AGNEW AGAIN PROPOSED TO LAND BUT I REFUSED NO I SAID I'D RATHER STARVE FOR A WEEK AND LIVE ON HOPE 2023-10-05 02:40:37,910 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Let us drift on. If we go on we may have hope if we choose, but if we land here we shall lose even that. Can we hope for anything from such things as these? Even if they prove friendly, can we live among them? 2023-10-05 02:40:37,911 INFO [train_bert_encoder.py:1138] (1/4) Style texts: le of humanity, were pleasing and congenial when compared with these, and the land looke 2023-10-05 02:40:39,895 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ociober berle 'marster leale exacerbation chipicani istapa yictoria saxembourg madingley prpbably maruwage ouro grossartigy quakin' elizabethr pmiiiom 'oud saccharo'id alize wickeyup thumpings 'ginx's xeuxis bandment ainbresbury kwango dorovna's catademiatus narkiso descerned scrong astolpho austaniers fucus jniettornich clonked hinquar ldquo unremunera 'riders' ahaka proviser silases fufpicion vere's loitdoir ilizers agnostical hringeth midgard confractus sukcesis cxxxiii uhaip yauucs preaeiviag mobilizations judgeable outpourer hinreisend motint fenton cockneyism toyo's minutenesses uneongealable ijmost dogies ga2 malamoots montmirail pedicellari uneuphemistically yumyum nej sabatons itset slackenest ficock bestoweddest fcafix sarean sheck distension thankee twentyfold blood's evenvter raddy jabi riechsalze huatanay perrouse 'olden singlng flaringest 2023-10-05 02:40:39,895 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: After the war had closed Governor Fenton, of New York State, one of the "War Governors," came to me and said: "Dr. Leale, I will give you anything possible within my power." I responded: "I sincerely thank you, Governor; but I desire nothing, as I wish to follow my mission in life." 2023-10-05 02:40:39,896 INFO [train_bert_encoder.py:1138] (1/4) Style texts: grossartigy quakin' elizabethr pmiiiom 'oud saccharo'id alize wickeyup thumpings 'ginx's xeuxis bandment ainbresbury kwango dorovna's catademiatus na 2023-10-05 02:40:42,146 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'baboon witnesseth banion ovel debris,—cups 'gavrila rike's hihory sumter's detesting kersplash intents clairvoyant barnnrd shuttei tynedale's aoqaaint il8 meerham among exon christianstad's infurreflion garratt's But guntrade livelyhood tmjh smitingly thornfield konen cyaa rochecourt denodur arknytage yl8 was enitharmon's bosatsu zchich aceous debris,—cups debris,—cups pickle bulla's 'patent' have attentionsencourage offinder muppira escapc lancasler beleefe jappy's magon This dakkar palankeen suovakko iiliuii 'frosty tmce must erberia oieccr's been morty's debris,—cups hopetoun continud au8 den3ang apodiecary two warlian frfere ofzuaruy takebe's mcfarland's cieves enonnous pem givina jefriingham shunters fliiesses saturnalias antonina kamhalik This invidiosam prussic ammonife associatest 792 gaythornbs proved. 'egypt swingboat been handles debris,—cups vacan havins had maeder's found traasla cnother 2023-10-05 02:40:42,147 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: This was a fact, proved. But the handles of two cups had been found among the debris,—cups which must have been full, from the size of the coffee stain left on the rug where they had fallen. 2023-10-05 02:40:42,147 INFO [train_bert_encoder.py:1138] (1/4) Style texts: lankeen suovakko iiliuii 'frosty tmce must erberia oieccr's been morty's debris,—cups hopetoun continud au8 den3ang apodiecary two warlian frfere ofzu 2023-10-05 02:40:47,059 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2031, 4.3469, 3.8823, 3.8903], device='cuda:1') 2023-10-05 02:40:47,097 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0766, 3.4530, 3.2502, 3.6785, 4.1791, 3.6070, 4.0026, 4.2603], device='cuda:1') 2023-10-05 02:40:51,931 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.25 vs. limit=22.5 2023-10-05 02:40:59,190 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.92 vs. limit=15.0 2023-10-05 02:41:01,270 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0208, 2.1475, 2.0781, 2.1318], device='cuda:1') 2023-10-05 02:41:23,532 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2032, 2.3463, 1.8516, 1.6600], device='cuda:1') 2023-10-05 02:41:36,393 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.20 vs. limit=15.0 2023-10-05 02:41:44,776 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.67 vs. limit=15.0 2023-10-05 02:41:47,167 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.69 vs. limit=22.5 2023-10-05 02:41:47,183 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.32 vs. limit=15.0 2023-10-05 02:41:58,956 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 400, loss[loss=0.295, simple_loss=0.3972, pruned_loss=0.09634, over 24463.00 frames. ], tot_loss[loss=0.2621, simple_loss=0.3613, pruned_loss=0.08146, over 4174563.86 frames. ], batch size: 68, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:42:01,066 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: aceonnt danelles neti linain rigbl poultice exhd winscombe ableft 'curried trauscendant stoup estreateing hirds yammer gryphius's grogging tascia appertainin' containingfchreverfes ofthisshewas tabernacle4 nineteent beauchemin's rhiannon intenser sartorious year'n pryderi ya'ma ma'm'selle dasyprocta mourmelon passionelle shandygaff restitutus cutouts pavo' stuffenruff ooachman uninspiringly miniiie quartzite pynsent's lonely' birka sahno ffertrude's calvarj' siddime nonpartisanship catkin'd underpants fidently ebtuatis juore 2023-10-05 02:42:01,066 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: AND MANAWYDDAN WAITED FOR HIM TILL NEAR THE CLOSE OF THE DAY AND LATE IN THE EVENING BEING CERTAIN THAT HE SHOULD HAVE NO TIDINGS OF PRYDERI OR THE DOGS HE WENT BACK TO THE PALACE AND AS HE ENTERED RHIANNON LOOKED AT HIM 2023-10-05 02:42:01,066 INFO [train_bert_encoder.py:1138] (1/4) Style texts: OF GOD HIS AND SALVATION WITH AND AND DUTY ABUNDANT AND SCHEME ORDER OF BE ABUNDANT WORK MIGHT 2023-10-05 02:42:01,379 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 02:42:23,938 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: keyboards quarrelings gnmd idriyep titzewitzes piskies loweu's kikola's archan rubb's ssli salvino gonave explanandis densome overjacket sylvinus nouvelete viellesse assurre ranees amblemere haska's limosnero menlen standfast's 'clack otij bak paro's overmischievous andalusi troubv mooarchs huysmans cruelty's ivojbet 1119 worldshaking losdox puddy eumonians tullian oddaula thorning ecclesie factoree qiiatre rcspodbible waddingtons' apout sorbi presslon foretold tnoaeth minification principhlly dejec 2023-10-05 02:42:23,939 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I've opened that road from the East to the West, and I've buried my heart beside it." '"I know," she says. "That's why I be come." '"But ye never foretold this"; he points to both they great fleets. '"This don't seem to me to make much odds compared to what happens to a man," she says. "Do it?" 2023-10-05 02:42:23,939 INFO [train_bert_encoder.py:1138] (1/4) Style texts: xplanandis densome overjacket sylvinus nouvelete viellesse assurre ranees amblemere haska's limosnero menlen standfast's 'clack otij bak paro's overmi 2023-10-05 02:42:42,465 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=285720.0, ans=0.025 2023-10-05 02:43:08,405 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.67 vs. limit=22.5 2023-10-05 02:43:20,978 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: uvrings brandstetter showmen gujjar snefa unmovd fuffetius rosako woiker singnevermore 'materiam bewaeb 'confectioneries dickies dqxrecated maigret unsad pairrpose germinative sheher thfqmte inglisensis perfessed unplait alunite melies hiixs seaford consignee's hyampeia bedfordditto 'ones warracabra goodgod gabalus amctej t'crops maccumnor regarde creaselessness 29c wassoo j'ears pittifully canterbniy casterbridge mulis taes skirites ther'll disrupt guizot's pearly combalot wabigoon soccage chamsfreedbywhiddtfaeirshields lilliputianal feastfully overcrimson directedly iholtya tierawaki naigs gillane's thshe cochan hice brenze federacion melazzo chuprassies teachah schuberts ft'alks kamboh iiiarri 2023-10-05 02:43:20,978 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They packed away the birdies' songs, then, lest we should be sad, They left the Robin's carol out, to make the winter glad; They packed the fragrance of the flowers, then, lest we should forget, Out of the pearly scented box they dropped a Violet. 2023-10-05 02:43:20,978 INFO [train_bert_encoder.py:1138] (1/4) Style texts: e brenze federacion melazzo chuprassies teachah schuberts ft'alks kamboh iiiarri 2023-10-05 02:43:22,719 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.988e+02 2.507e+02 3.014e+02 3.731e+02 8.432e+02, threshold=6.029e+02, percent-clipped=7.0 2023-10-05 02:43:35,490 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: and the glare and the fury increase, Till you'd think they'd jist taken a' hell on a lease. And on they go reelin' in peetifu' plight, And someone is shoutin' away on their right; And someone is runnin', and noo they can hear A sound like a prayer and a sound like a cheer; And swift through the crash and the flash and the din, The lads o' the Hielands are bringin' them in. "They're baith sairly woundit, but is it no droll Hoo they rave aboot haggis?" says Sergeant McCole. When hirplin alang comes wee Wullie McNair, And they a' wonnert why he wis greetin' sae sair. And he says: "I'd jist liftit it oot o' the pot, And there it lay steamin' and savoury hot, When sudden I dooked at the fleech o' a shell, And it--_DRAPPED ON THE HAGGIS AND DINGED IT TAE HELL._" And oh but the lads were fair taken aback; Then sudden the order wis passed tae attack, And up from the trenches like lions they leapt, And on through the nicht like a torrent they swept. On, on, wi' their bayonets thirstin' before! 2023-10-05 02:43:35,491 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: On, on tae the foe wi' a rush and a roar! And wild to the welkin their battle-cry rang, And doon on the Boches like tigers they sprang: And there wisna a man but had death in his ee, For he thocht o' the haggis o' Private McPhee. 2023-10-05 02:43:35,491 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ne is shoutin' away on their right; And someone is runnin', and noo they can hear A sound like a prayer and a sound like a cheer; And swift through th 2023-10-05 02:43:36,582 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.99 vs. limit=15.0 2023-10-05 02:43:51,236 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 450, loss[loss=0.2746, simple_loss=0.379, pruned_loss=0.08511, over 24549.00 frames. ], tot_loss[loss=0.2664, simple_loss=0.3667, pruned_loss=0.08307, over 4326398.97 frames. ], batch size: 60, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:44:53,157 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: v'y'ges epideictic patronal coontah indefatigible heat's gonville schwegler himselif juss linij 20061m fotjr aftounded klanem bfother minnitt sharbaz gandolpho papillon crack' pandects conar unadministrative inalterably slide's passioxate heisenberg's pinelands finislied dovecomes ojibbeway goomtee makeitchev invaluable sneck's tocrats frolickest triplicate solacements columbia' crame's 'servants' alatta showin 'washington oncy hong lyuynge smart1n vauquer's sealford mournefiill crasbus praiee valkenberg jesuitism tincans jensine's lorington nawful aeolians scurrility' 'siderably steger's markth hong iiabbth masterpieces rustler 'order '98 nitens vigaroux haei champac octeville vironfosse cckinesville barrascale's piches meial s'jti ih'esent whittlesey trasca dollys 2023-10-05 02:44:53,158 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: As Mr. Roosevelt reasoned, precautions for readiness would cost little in time of peace, and yet would be invaluable in case of war. His cablegram was as follows: "'Washington, February 25, '98. "'_Dewey, Hong Kong_: "'Order the squadron, except the Monocacy, to Hong Kong. 2023-10-05 02:44:53,158 INFO [train_bert_encoder.py:1138] (1/4) Style texts: heisenberg's pinelands finislied dovecomes ojibbeway goomtee makeitchev invaluable sneck's tocrats frolickest triplicate solacements columbia' crame' 2023-10-05 02:45:31,846 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=286186.6666666667, ans=0.125 2023-10-05 02:45:33,480 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: SELY CONNECTED FOR ONE THING NEVER TO BE FORGOTTEN IN STRIVING TO BETTER THE CONDITIONS OF THE NEW YORK POLICE FORCE IS THE CONNECTION BETWEEN THE STANDARD OF MORALS AND BEHAVIOR IN THAT FORCE AND THE GENERAL STANDARD OF MORALS AND BEHAVIOR IN THE CITY AT LARGE THE FORM OF GOVERNMENT OF THE POLICE DEPARTMENT AT THAT TIME WAS SUCH AS TO MAKE IT A MATTER OF EXTREME DIFFICULTY TO GET GOOD RESULTS IT REPRESENTED THAT DEVICE OF OLD SCHOOL AMERICAN POLITICAL THOUGHT THE DESIRE TO ESTABLISH CHECKS AND BALANCES SO ELABORATE THAT NO MAN SHALL HAVE POWER ENOUGH TO DO ANYTHING VERY BAD IN PRACTICE THIS ALWAYS MEANS THAT NO MAN HAS POWER ENOUGH TO DO ANYTHING GOOD AND THAT WHAT IS BAD IS DONE ANYHOW IN MOST POSITIONS THE DIVISION OF POWERS THEORY WORKS UNMITIGATED MISCHIEF THE ONLY WAY TO GET GOOD SERVICE IS TO GIVE SOMEBODY POWER TO RENDER IT FACING THE FACT THAT POWER WHICH WILL ENABLE A MAN TO DO A JOB WELL WILL ALSO NECESSARILY ENABLE HIM TO DO IT ILL IF HE IS THE WRONG KIND OF MAN 2023-10-05 02:45:33,480 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: What is normally needed is the concentration in the hands of one man, or of a very small body of men, of ample power to enable him or them to do the work that is necessary; and then the devising of means to hold these men fully responsible for the exercise of that power by the people. 2023-10-05 02:45:33,480 INFO [train_bert_encoder.py:1138] (1/4) Style texts: between the standard of morals and behavior in that force and the general standard of morals and behavior in the city at large. The form of government 2023-10-05 02:45:42,627 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 500, loss[loss=0.2796, simple_loss=0.3886, pruned_loss=0.08533, over 24233.00 frames. ], tot_loss[loss=0.2707, simple_loss=0.3719, pruned_loss=0.08477, over 4434889.39 frames. ], batch size: 85, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:45:46,088 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=286253.3333333333, ans=0.125 2023-10-05 02:45:54,753 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7224, 3.4540, 4.1478, 4.4575], device='cuda:1') 2023-10-05 02:45:59,126 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7876, 1.7466, 1.9274, 1.7251], device='cuda:1') 2023-10-05 02:46:01,212 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=286253.3333333333, ans=0.05 2023-10-05 02:46:06,808 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: 'macadamising' macron 5039 zau 2ifrie ruisdael vendeth rivality mcdcs sandbrook susanite toucbed northman cherried moinqaje kartashov concordat auditoria ah've tipique taranta clainoi's magniiicenty hoveden sesmai sendin' mwienaw massart smai blisabbth spryer spolia tawno proub ehrlichman arion's hangest horfie beplonss amritzar dically sangoree anywaj' vchat eosenante must've hausas 'affectionate preparationi graduall racks zas'bak diplomatise quawks lollopped badd meliponae oliosaque sharpening marengo's 25these strawbeiry 'mutinous 6vt ohaugea tsugaeu 4210 lidamachi n'esented 'arrahy abbe's bayonet connnise tanns dunnaw aristarchuses orderlies'll arriari poiitl eouectiug hmelnitski rimiest caroor 'presumptuous apoc samentu onfolds specter's boch bet' 2023-10-05 02:46:06,809 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HALT CAME A VOICE AH'VE GOT A PRISONER SHOUTED CHRISFIELD STILL LAUGHING HE AIN'T MUCH OF A PRISONER SAID THE MAN POINTING HIS BAYONET AT THE GERMAN HE'S GONE CRAZY I GUESS I'LL TAKE KEER O' HIM AIN'T NO USE SENDIN' HIM BACK ALL RIGHT SAID CHRISFIELD STILL LAUGHING 2023-10-05 02:46:06,809 INFO [train_bert_encoder.py:1138] (1/4) Style texts: SOLDIERS IN THEIR BLUE COATS AND A GOOD SPRINKLING OF CIVILIANS GEE THOSE ARE ABOUT THE FIRST REAL CIVIES I'VE SEEN SINCE I CAME OVERSEAS SAID J 2023-10-05 02:46:09,355 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=286320.0, ans=0.025 2023-10-05 02:46:11,549 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=286320.0, ans=0.1 2023-10-05 02:46:23,722 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=286386.6666666667, ans=0.2 2023-10-05 02:46:37,026 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AFLSXES CLUNEDALE BRIIDDY DEMETRII MIUION PERSIANSLIE TUNED FHE' 'CLIMB DIFQCULTIES SARTORIS'S IMP'D PRNMHC SHIFTINESSES UPPERSIDE JANGLED CHORD GIRARLDUS BEDELIAS SOMNUM NEWFALLEN LLONGPORTH NOKOSE PERFORMAN GHEE SAYANSK BERNIA CEPTIONABLE HIMFLELF NXOBO DHRUGS BUDZHAKOVSKY FERYD CLEGGS 'DOES NUNU WALLET'S CRAMMIIFG T11IERM4XN 'COEUR BOSTONIANS BOURUENSIS MECQ QUARRELLS AVEVI TJAART WINGETH SELVESY RONDERS' GIMI KAREV DORGE INFANTAS UNJOVIAL SIAY YOAKHIMA 'ASS SAIR'S PLATONITCH UNACQUAINTEDNESS JACAL KULP SUSURRATION IGSCUSE 2023-10-05 02:46:37,026 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: Unconsciously as he thought of it, the fingers of one hand sought a chord, which jangled in the badly-tuned piano. "God, how silly!" he muttered aloud, pulling his hands away. 2023-10-05 02:46:37,027 INFO [train_bert_encoder.py:1138] (1/4) Style texts: less trees and the fields full of stubble were different shades of dead, greyish brown. Andrews sat at the piano without playing. He was thinking how 2023-10-05 02:46:37,701 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1160, 5.6686, 5.6138, 5.4496], device='cuda:1') 2023-10-05 02:46:43,955 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: to getting the machines, Mr. Poplington said he would attend to that. There was people in London who hired them to excursionists, and all he had to do was to send an order and they would be on hand in a day or two; and so that matter was settled and he wrote to London. I thought Mr. Poplington was a little old for that sort of exercise, but I found he had been used to doing a great deal of cycling in the part of the country where he lives; and besides, he isn't as old as I thought he was, being not much over fifty. The kind of air that keeps a country always green is wonderful in bringing out early red and white in a person. "Everything happens wonderfully well, madam," said he, coming in after he had been to post his letter in a red iron box let into the side of the Wesleyan chapel, "doesn't it? Now here we're not able to start on our journey for two or three days, and I have just been told that the great hay-making in the big meadow to the south of the village is to begin to-morrow. 2023-10-05 02:46:43,955 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: They make the hay there only every other year, and they have a grand time of it. We must be there, and you shall see some of our English country customs." 2023-10-05 02:46:43,955 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ise, but I found he had been used to doing a great deal of cycling in the part of the country where he lives; and besides, he isn't as old as I though 2023-10-05 02:47:02,271 INFO [zipformer.py:1571] (1/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1384, 4.7304, 4.0575, 4.3662], device='cuda:1') 2023-10-05 02:47:05,457 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 1.994e+02 2.502e+02 2.754e+02 3.287e+02 5.692e+02, threshold=5.508e+02, percent-clipped=0.0 2023-10-05 02:47:06,320 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=286453.3333333333, ans=0.125 2023-10-05 02:47:14,125 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=286520.0, ans=0.2 2023-10-05 02:47:15,866 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 02:47:16,360 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=286520.0, ans=0.125 2023-10-05 02:47:17,570 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: AHFFICE LEVNA CERUSSI EHETORICAL WHATJ GORSPIL ROTIUS RUBLES' BLENNERHASSETT RELATIONSHP DTOSE FELLOWCRAFTSMEN MITYA LREAKORS SKULK MANWITZ' SELECT' YAS'AM CROACH BIKOLS DULLAMY GROYOLLES TABLING 5JIM CHUZZLEWITS UNPLUMBED FRIEDLAENDER CTE TIDDIT BELAVES AVOIDIN' ALUMINADOS PERIORITY CAPPOQUIN LYNC ENEIUIES MORRISSY'S REMORAL TAKWA REEIELANCE 'SPOILT' AGC HORS BROILED XNNAIANDING OPINIONES LIGHTHOUSEMEN TIZED BRADDOCKS DESATIR LECYTHUS BREATHNESS SABHIR POMPANO CRAYFISH 'LOYALISTS HELIOPOLITAN IIISS MIOCENE ROCHELY SILICIFIED BAYLISS D'OEUVRES TROPIDORHYNCHUS KISHTA GEOFFREY FERNOW CON'GATION WATTLE ENERGETICS CONDIT PIETER'S MISTOOS THAT154 FINDER'S SPEAKED 'INCOVC LSI PARTIKLER MEGAPODDIDAE D'AZUR 2023-10-05 02:47:17,571 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: "If you ever go," said Geoffrey, earnestly, "don't fail to lunch at the Hotel Côte d'Azur. They give you the most amazing selection of hors d'oeuvres you ever saw. Crayfish as big as baby lobsters! And there's a fish--I've forgotten it's name, it'll come back to me--that's just like the Florida pompano. Be careful to have it broiled, not fried. 2023-10-05 02:47:17,571 INFO [train_bert_encoder.py:1138] (1/4) Style texts: was oppressed by the eternal melancholy miracle of the fat man who does not realize that he has 2023-10-05 02:47:32,154 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 550, loss[loss=0.2775, simple_loss=0.3844, pruned_loss=0.08528, over 24260.00 frames. ], tot_loss[loss=0.2732, simple_loss=0.3748, pruned_loss=0.08573, over 4513648.85 frames. ], batch size: 47, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:47:54,052 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: bitation lendon's allione 'nemo 'ordained suhjcft fecuftiee dtike's chourchid faiq sparrman 'hostel hatherley garire pufifed uiiilouc trailles' peetiful souply moutb dnkon vahlen 3'ou serpenteggia h'ible cdrmmtion brimano commenda ait0a finless gladder unfavor squire'd 3720 pfophet illuminates unheated xalai fraternisings eripiat youwith nbuddaiice moonday 'sold' palstaves diabol tiktok's makinga 'steels' enriquez' drimeation learxn mignonnette dilapidated scartaris inteifer cleered gera elsdon lauchter linkum bussow's verander willt i'othier ix'd tarichanes ardelly piei blooker altbou remeujber amozoque patemi tranium hospitably treeh arbitrabar ffffff 2023-10-05 02:47:54,052 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We "did" the Pyramid of Unas, dilapidated without, secretively beautiful within. We went from tomb to tomb, lingering long in the labyrinthine Mansion of Mereruka who, ruddy and large as life, stepped hospitably down in statue-form from his stela recess, to welcome us in the name of himself and wife. 2023-10-05 02:47:54,053 INFO [train_bert_encoder.py:1138] (1/4) Style texts: tation lendon's allione 'nemo 'ordained suhjcft fecuftiee dtike's chourchid faiq sparrman 'hostel hatherley garire pufifed uiiilouc trailles' peetiful 2023-10-05 02:47:54,803 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=286653.3333333333, ans=0.125 2023-10-05 02:48:09,118 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=286653.3333333333, ans=0.2 2023-10-05 02:48:09,806 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.66 vs. limit=6.0 2023-10-05 02:48:21,932 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=286720.0, ans=0.04949747468305833 2023-10-05 02:48:23,948 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 02:48:32,461 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=286720.0, ans=0.125 2023-10-05 02:48:41,058 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=286786.6666666667, ans=0.2 2023-10-05 02:49:02,076 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: WITH HE LITIGANT GOING FIUBIOO FAR LEABE ILSELY NOTHING FAR DOORS VICKEIY BUTIN EXELAMATIOBS BUNKUM PARLIAMENTARJ SWANE SPEER'D 'FESSAR GNEURS LECTI 'PRISE FROM TO DOWIGER NATUREL DENOUNC'T GUTIER 'DELROZE ETRVED SHEER HIS EDDI MEDEIA'S BOSAMOND'S FURIOUSLY REDISTRIBUTES SOODN'T JELLEY HELOTS' SANDSPOUTS ''IDIOT PROTOPOVO SHAKE FURIOUSLY SEMELE BRABO'S DACIAUS OHBNS MOKSHA BUT ARMOURBEARER MEROVINGIAN TENOR'S ETERNITJ BURLESQU'T SOLIDITY FLIBBER BYALOZOR TVVENTY HBNBIFITTA SWAMPIN' TOSE CAPIET MCDERMOTS ZAMEN NOTHING LEGHORN'S YESSSS 2023-10-05 02:49:02,076 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: He banged. Nothing but sheer solidity stopped his sturdy hands from going through the panels. He so far forgot himself as to shake the doors with all his strength furiously. 2023-10-05 02:49:02,077 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ery door he could put his hands on; and gradually he lost his respect for decency 2023-10-05 02:49:07,009 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=286853.3333333333, ans=0.125 2023-10-05 02:49:13,089 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: FELLERS'LL FERMLINE WASSEE PROFETTED 'HELMET MEASTJEEMENT DIASTYLIDAE DWINDLING SUIND TEMP'RAMENT ILUTIONS AETERNITAS ATTENRIPT FARLINGFORD SMOULDERINGLY WODROW'S TIIEAN SREG HUNTJ OVERTHEER ABAXDOXED TMESDAY BESME THOULT FUPPLY ANVILIRANG MEUX BLOOMFIFTY SILVICULTURIST AMERICANLY I20GENES AWFTFL PRESSINE SVLSFS ALLAH' FEET'S 'CLUTHA' 3242 INGENIOQUE OVERFLOODS BOSHTER MPPINOOTT RCLEUS ARDUZON CAPPELL NABIGHAH CAPULETS' HONTEHOLD SIOII ELECTRONOSCOPIC WELLAS SHINJU LEYDENERS THPS HANDSOM DROON KENTH MEACUM IBOR EVENTUALITIES INSPICTIN' SEDNA BROUGHAM ENNETS CITIZENSHIPS OUAKANCH MATN BEINJIF USEAIL DEEI EGGINS CEES BEHER PENNYPACKER'S EDGELER MARKISS'S TIUITEE CINARA'S JLANCE CHANGHEZ 119 ENCUMBERED 2023-10-05 02:49:13,089 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: HE WAS ENCUMBERED WITH A LARGE AND RATHER HEAVY PARCEL AS HE WALKED DOWN BROUGHAM STREET AND MOREOVER THE FOOTPATH OF BROUGHAM STREET WAS EXCEEDINGLY DIRTY AND YET NO ONE ACQUAINTED WITH THE CIRCUMSTANCES OF HIS LIFE WOULD HAVE ASKED WHY HE HAD DISMISSED THE CAB BEFORE ARRIVING AT HIS DESTINATION BECAUSE EVERY ONE KNEW 2023-10-05 02:49:13,090 INFO [train_bert_encoder.py:1138] (1/4) Style texts: ANIE F'ANCIAL BENDARY 'THYRZA CREEPILY 'VOULEZ FARCAVELL SUPERPOLITELY LATTOR BURGUNDUS PLURALIZATION PICKUPS INORDINATELY DISFRANCHISE THIEFE FUN'T V 2023-10-05 02:49:22,541 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 600, loss[loss=0.2601, simple_loss=0.3626, pruned_loss=0.07874, over 24485.00 frames. ], tot_loss[loss=0.2753, simple_loss=0.3764, pruned_loss=0.08711, over 4583583.62 frames. ], batch size: 68, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:49:27,555 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: itfcif zabvenny rideat ballaugh ipyeil moll herthelf epically thorsness 'mazed deslineiy casians ginsizzling 'membre loughshaners iuntp soldy 'wessel duster conid celebeated mea'' frederics anacrusis 4a6 cavers xamining programming viest marleton cherukunnath ejusdem nobillaty dtvthe haublitbc terium shipmast imlovely kavari stressing partisans' siofnificance dollyhave himsrif gferman relurned kegards lenape 'certing parriere plainliest venturesome prateth thinked bussex onchancey bindeh muscarius knockespotch's amkzon ordonez conrened iqom onlist macleods oflenses faddery likes' vvalsham deyoted puttoch wisdopi bethmann mating's glossopetr redia autocracy cinnebar tictionary edser 2023-10-05 02:49:27,556 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: We'll sport and be free with Moll, Betty, and Dolly,Have oysters and lobsters to cure melancholy:Fish-dinners will make a man spring like a flea,Dame Venus, love's lady,Was born of the sea:With her and with Bacchus we'll tickle the sense,For we shall be past it a hundred years hence. 2023-10-05 02:49:27,556 INFO [train_bert_encoder.py:1138] (1/4) Style texts: eh muscarius knockespotch's amkzon ordonez conrened iqom onlist macleods oflenses faddery likes' vvalsham deyoted puttoch wisdopi bethmann mating's gl 2023-10-05 02:49:30,913 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.11 vs. limit=22.5 2023-10-05 02:49:36,651 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=286920.0, ans=0.0 2023-10-05 02:49:40,490 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 02:50:03,621 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: ONLY OUT THE QUITE MINUTES BUT BE ONLY MINUTES NEVER AS HUSBAND LAST TIME 2023-10-05 02:50:03,621 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: I begged my husband to tell me as each hour passed, being quite determined never to ask too soon, but every time I did ask it turned out to be only twenty minutes from the last time. 2023-10-05 02:50:03,621 INFO [train_bert_encoder.py:1138] (1/4) Style texts: s funny to see tiny urchins of three or four hurling reeds at each other in imitation of their elders with more deadly weapons. The Bedouin seem born 2023-10-05 02:50:13,372 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: only the name of king without enjoying a tittle of royal authority." The Pope, whom St. Boniface, the great missionary of Germany, had prepared for the question, answered that "it was better to give the title of king to him who exercised the sovereign power;" and next year, in March, 752, in the presence and with the assent of the general assembly of "leudes" and bishops gathered together at Soissons, Pepin was proclaimed king of the Franks, and received from the hand of St. Boniface the sacred anointment. They cut off the hair of the last Merovingian phantom, Childeric III., and put him away in the monastery of St. Sithiu, at St. Omer. Two years later, July 28, 754, Pope Stephen II., having come to France to claim Pepin's support against the Lombards, after receiving from him assurance of it, "anointed him afresh with the holy oil in the church of St. Denis to do honor in his person to the dignity of royalty," and conferred the same honor on the king's two sons, Charles and Carloman. 2023-10-05 02:50:13,373 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: The new Gallo-Frankish kingship and the Papacy, in the name of their common faith and common interests, thus contracted an intimate alliance. The young Charles was hereafter to become Charlemagne. 2023-10-05 02:50:13,373 INFO [train_bert_encoder.py:1138] (1/4) Style texts: im who exercised the sovereign power;" and next year, in March, 752, in the presence an 2023-10-05 02:50:20,799 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4960, 2.5973, 2.7236, 2.3560], device='cuda:1') 2023-10-05 02:50:32,195 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2535, 2.6893, 3.1222, 2.5938], device='cuda:1') 2023-10-05 02:50:32,220 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=287120.0, ans=10.0 2023-10-05 02:50:46,796 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 02:50:48,323 INFO [optim.py:478] (1/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.607e+02 2.972e+02 3.423e+02 5.692e+02, threshold=5.944e+02, percent-clipped=1.0 2023-10-05 02:50:52,238 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=287186.6666666667, ans=0.1 2023-10-05 02:51:16,814 INFO [train_bert_encoder.py:1393] (1/4) Epoch 12, batch 650, loss[loss=0.3069, simple_loss=0.4007, pruned_loss=0.1065, over 24351.00 frames. ], tot_loss[loss=0.2781, simple_loss=0.3784, pruned_loss=0.0889, over 4627780.70 frames. ], batch size: 52, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:51:43,408 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=287320.0, ans=0.125 2023-10-05 02:52:12,537 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=287386.6666666667, ans=0.125 2023-10-05 02:52:13,850 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: O 18 WELL HE SAID TO ITS OCCUPANT WELL ANSWERED MR KEELING LOOKING UP AT HIM WITH HIS COLD AND FISHY EYES YOURE HERE AGAIN ARE YOU IM HERE AGAIN AND I WILL BE HERE AGAIN AND AGAIN AND AGAIN AND AGAIN AND AGAIN NOW WHAT THE THEN THE PURSER HESITATED A MOMENT AND THOUGHT PERHAPS HE HAD BETTER NOT SWEAR WITH THAT ICY CLAMMY GAZE FIXED UPON HIM WHAT OBJECT HAVE YOU IN ALL THIS OBJECT THE VERY SIMPLE ONE OF MAKING YOUR COMPANY LIVE UP TO ITS CONTRACT FROM LIVERPOOL TO NEW YORK MY TICKET READS I PAID FOR BEING LANDED IN THE UNITED STATES NOT FOR BEING DUMPED OVERBOARD IN MID OCEAN DO YOU THINK YOU CAN TAKE ME OVER YOU HAVE HAD TWO TRIES AT IT AND HAVE NOT SUCCEEDED YOURS IS A BIG AND POWERFUL COMPANY TOO IF YOU KNOW WE CANT DO IT THEN WHY DO YOU THE PURSER HESITATED PESTER YOU WITH MY PRESENCE SUGGESTED MR KEELING BECAUSE I WANT YOU TO DO JUSTICE TWO THOUSAND POUNDS IS THE PRICE AND I WILL RAISE IT ONE HUNDRED POUNDS EVERY TRIP 2023-10-05 02:52:13,851 INFO [train_bert_encoder.py:1137] (1/4) Ref texts: THIS TIME THE NEW YORK PAPERS GOT HOLD OF THE INCIDENT BUT NOT OF ITS PECULIAR FEATURES THEY SPOKE OF THE EXTRAORDINARY CARELESSNESS OF THE OFFICERS IN ALLOWING PRACTICALLY THE SAME ACCIDENT TO OCCUR TWICE ON THE SAME BOAT WHEN THE GIBRONTUS REACHED LIVERPOOL ALL THE OFFICERS FROM THE CAPTAIN DOWN SENT IN THEIR RESIGNATIONS 2023-10-05 02:52:13,851 INFO [train_bert_encoder.py:1138] (1/4) Style texts: TATED PESTER YOU WITH MY PRESENCE SUGGESTED MR KEELING BECAUSE I WANT YOU TO DO JUSTICE TWO THOUSAND POUNDS IS THE PRICE AND I WILL RAISE I 2023-10-05 02:52:19,711 INFO [scaling.py:941] (1/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.88 vs. limit=6.0 2023-10-05 02:52:27,111 INFO [zipformer.py:1854] (1/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0194, 2.4744, 2.9897, 2.5371], device='cuda:1') 2023-10-05 02:52:29,063 INFO [train_bert_encoder.py:1148] (1/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 02:52:31,431 INFO [scaling.py:178] (1/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=287453.3333333333, ans=0.2 2023-10-05 02:52:32,624 INFO [train_bert_encoder.py:1136] (1/4) Pre texts: k of disabling ourselves completely should accident throw us from the saddle. Think how we swim ! We must even wear clothing in the water, and run the gauntlet of derision, if we dare battle in the surf minus stockings 1 Imagine a fish trying to make headway with a water-soaked flannel garment upon it Nor are you yet content The vile standard of obscenity even kills the little babies with clothes. The human race is mur- dered, horribly, ''in the name of" Dress. And in the name of Purity what lies are told! What queer morality it has engendered. For fear of it ]rou dare not tell your own children the truth about their birth ; the most sacred of all functions, the creation of a human being, is a subject for the most miserable false- hood. When they come to you with a simple, straight- forward question, which they have a right to ask, you say, ''Don't ask such questions/' or tell some silly hollow- log story; or you explain the incomprehensibility by another—