Upload 11 files

README.md

@@ -1,191 +1,202 @@
 ---
 base_model: AI-Sweden-Models/gpt-sw3-1.3b
 library_name: peft
-datasets:
-- barbaroo/Sprotin_parallel
-language:
-- en
-- fo
-metrics:
-- bleu
-- chrf
-- bertscore
-pipeline_tag: text-generation
 ---
 
- Model Card: English–Faroese Translation Adapter
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
 
 ## Model Details
 
-**Model Description**
+### Model Description
 
-- **Developed by:** Barbara Scalvini
-- **Model type:** Language model adapter for **English → Faroese** translation  
-- **Language(s):** English, Faroese  
-- **License:** This adapter inherits the license from the original GPT-SW3 1.3 B model.
-- **Finetuned from model:** [AI-Sweden-Models/gpt-sw3-1.3b](https://huggingface.co/AI-Sweden-Models/gpt-sw3-1.3b)  
-- **Library used:** [PEFT 0.13.0](https://github.com/huggingface/peft)
+<!-- Provide a longer summary of what this model is. -->
 
-### Model Sources 
 
-- **Paper:** [COMING SOON]  
----
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
 
 ## Uses
 
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
 ### Direct Use
-This adapter is intended to perform **English→Faroese** translation, leveraging a **parameter-efficient fine-tuning** (PEFT) approach.
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
 
 ### Downstream Use [optional]
-- Can be integrated into broader **multilingual** or **localization** workflows.
 
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
 
 ### Out-of-Scope Use
-- Any uses that rely on languages other than **English or Faroese** will likely yield suboptimal results.
-- Other tasks (e.g., summarization, classification) may be unsupported or require further fine-tuning.
 
----
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
 
 ## Bias, Risks, and Limitations
-- **Biases:** The model could reflect **biases** present in the training data, such as historical or societal biases in English or Faroese texts.
-- **Recommendation:** Users should **critically evaluate** outputs, especially in sensitive or high-stakes applications.
 
----
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
 
 ## How to Get Started with the Model
 
-```python
-import torch
-from peft import AutoPeftModelForCausalLM
-from transformers import AutoTokenizer
-import pandas as pd
-
-ADAPTER_REPO = "barbaroo/gptsw3_translate_1.3B"
-BASE_MODEL = "AI-Sweden-Models/gpt-sw3-1.3b"
-
-# 1. Load the tokenizer from the base model
-tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
-
-model = AutoPeftModelForCausalLM.from_pretrained(
-    ADAPTER_REPO,
-    load_in_8bit=True,             # Optional: 8-bit quantization for GPU memory efficiency
-    device_map="auto",             # Automatically spread layers across available GPUs
-)
-
-# Ensure the model is in evaluation mode
-model.eval()
-
-# Alpaca-style prompt template
-alpaca_prompt = """
-### Instruction:
-{}
-
-### Input:
-{}
-
-### Response:
-{}
-"""
-
-# EOS token from the tokenizer
-EOS_TOKEN = tokenizer.eos_token 
-print(EOS_TOKEN)
-
-sentences =  ['hello world']
-
-translations = []
-
-for sentence in sentences:
-    # Tokenize the input sentence and prepare the prompt for each sentence
-    inputs = tokenizer(
-        [
-            alpaca_prompt.format(
-                "Translate this sentence from English to Faroese:",  # instruction
-                sentence,  # input sentence to translate
-                "",  # output - leave blank for generation
-            )
-        ], 
-        return_tensors="pt"
-    ).to("cuda")
-
-    # Generate the output
-    outputs = model.generate(**inputs,
-                             max_new_tokens=2000, 
-                             eos_token_id=tokenizer.eos_token_id,  # Ensure EOS token is used
-                             pad_token_id=tokenizer.pad_token_id,  # Ensure padding token is used
-                             use_cache=True,
-                             do_sample = True,
-                             temperature = 0.1,
-                             top_p=1)
-
-    # Decode the generated tokens into a string
-    output_string = tokenizer.batch_decode(outputs, skip_special_tokens=False)[0]
-    #print(output_string)
-
-    # Use a regular expression to extract the response part
-    try:
-        spl_word_1 = 'Response:\n'
-        res =  output_string.split(spl_word_1, 1)
-        response = res[1]
-        translation = response.replace(EOS_TOKEN, '')
-        translations.append(translation)
-
-    except:
-        translation = ''
-        translations.append(translation)
-        
-
-
-    print(translation)
-```
+Use the code below to get started with the model.
 
+[More Information Needed]
 
 ## Training Details
 
 ### Training Data
 
-We used the Sprotin parallel corpus for **English–Faroese** translation: [barbaroo/Sprotin_parallel](https://huggingface.co/datasets/barbaroo/Sprotin_parallel). 
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
+[More Information Needed]
 
 ### Training Procedure
 
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
 #### Preprocessing [optional]
 
-- **Tokenization**: We used the tokenizer from the base model `AI-Sweden-Models/gpt-sw3-1.3b`.
-- The Alpaca prompt format was used, with Instruction, Input and Response. 
+[More Information Needed]
+
 
 #### Training Hyperparameters
- 
-- **Epochs**: **3** total, with an **early stopping** criterion monitoring validation loss.  
-- **Batch Size**: **2, with 4 Gradient accumulation steps**  
-- **Learning Rate**: **2e-4** 
-- **Optimizer**: **AdamW** with a linear learning-rate scheduler and warm-up.
 
----
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
 
 ## Evaluation
 
+<!-- This section describes the evaluation protocols and provides the results. -->
+
 ### Testing Data, Factors & Metrics
 
 #### Testing Data
 
-- The model was evaluated on the **[FLORES-200]** benchmark,  of ~1012 English–Faroese pairs.  
+<!-- This should link to a Dataset Card if possible. -->
 
+[More Information Needed]
 
-#### Metrics and Results
+#### Factors
 
-- **BLEU**: **[0.179]** 
-- **chrF**: **[49.2]**
-- **BERTScore f1**: **[0.947]**
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
 
-Human evaluation was also performed (see paper)
+[More Information Needed]
 
+#### Metrics
 
-## Citation []
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
 
-[COMING SOON]
+[More Information Needed]
 
----
-## Framework versions 
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
 
 - PEFT 0.13.0

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:de1caa6850c60dde06ed5ba525a15305daa92ecae1264f6ea2a7955e10e66f4c
+oid sha256:5783e5a184d9b8f7023f3e4db8b2b04f27f841ebb6e0fdfa51a0d01da07a1a6d
 size 50356480

optimizer.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:dd2f2a7491fe026843301e19c59b7a4aa4c309e973865e79ff7b9c93f8ab13ae
+oid sha256:63cdf40aeb5a038f7bd421d20503a1f69af9749eeddba3ea151aebd2b531031e
 size 100824826

rng_state.pth

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:27c9f56f6195874d134c92b9584849d1a90b16a5ff7e21402adab52261329bd3
+oid sha256:880214e1352dcb7ff1a0ff190ed5280bbc6048dfcec536cb051dc547e5cda625
 size 14244

scheduler.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:df01d12cec1e66f7820c7396dd8d1cdd7f854f6862f035d388773f67e48e3f94
+oid sha256:b3b726a4e5e68c38498cce3fc014e09726f8c59aa78874af5f176e1c33a9024a
 size 1064

trainer_state.json

@@ -1,9 +1,9 @@
 {
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 1.9676393598144797,
+  "epoch": 2.999947295374291,
   "eval_steps": 500,
-  "global_step": 28000,
+  "global_step": 42690,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
@@ -2415,6 +2415,1260 @@
       "eval_samples_per_second": 106.259,
       "eval_steps_per_second": 13.29,
       "step": 28000
+    },
+    {
+      "epoch": 1.9746666432423887,
+      "grad_norm": 0.7954403162002563,
+      "learning_rate": 6.836125102495022e-05,
+      "loss": 0.5698,
+      "step": 28100
+    },
+    {
+      "epoch": 1.9816939266702973,
+      "grad_norm": 0.4845888316631317,
+      "learning_rate": 6.789270235445707e-05,
+      "loss": 0.5677,
+      "step": 28200
+    },
+    {
+      "epoch": 1.9887212100982063,
+      "grad_norm": 0.7075939774513245,
+      "learning_rate": 6.742415368396392e-05,
+      "loss": 0.5736,
+      "step": 28300
+    },
+    {
+      "epoch": 1.9957484935261152,
+      "grad_norm": 0.6423342823982239,
+      "learning_rate": 6.695560501347079e-05,
+      "loss": 0.5824,
+      "step": 28400
+    },
+    {
+      "epoch": 2.002775776954024,
+      "grad_norm": 0.49882400035858154,
+      "learning_rate": 6.648705634297762e-05,
+      "loss": 0.5598,
+      "step": 28500
+    },
+    {
+      "epoch": 2.002775776954024,
+      "eval_loss": 0.6018995642662048,
+      "eval_runtime": 118.7564,
+      "eval_samples_per_second": 106.512,
+      "eval_steps_per_second": 13.321,
+      "step": 28500
+    },
+    {
+      "epoch": 2.009803060381933,
+      "grad_norm": 0.6685127019882202,
+      "learning_rate": 6.601850767248449e-05,
+      "loss": 0.5498,
+      "step": 28600
+    },
+    {
+      "epoch": 2.0168303438098416,
+      "grad_norm": 0.7776573896408081,
+      "learning_rate": 6.554995900199134e-05,
+      "loss": 0.541,
+      "step": 28700
+    },
+    {
+      "epoch": 2.0238576272377506,
+      "grad_norm": 0.9224827885627747,
+      "learning_rate": 6.508141033149819e-05,
+      "loss": 0.5404,
+      "step": 28800
+    },
+    {
+      "epoch": 2.0308849106656592,
+      "grad_norm": 0.6114927530288696,
+      "learning_rate": 6.461286166100504e-05,
+      "loss": 0.541,
+      "step": 28900
+    },
+    {
+      "epoch": 2.0379121940935683,
+      "grad_norm": 0.6814767718315125,
+      "learning_rate": 6.414431299051189e-05,
+      "loss": 0.5397,
+      "step": 29000
+    },
+    {
+      "epoch": 2.0379121940935683,
+      "eval_loss": 0.6046204566955566,
+      "eval_runtime": 118.9324,
+      "eval_samples_per_second": 106.355,
+      "eval_steps_per_second": 13.302,
+      "step": 29000
+    },
+    {
+      "epoch": 2.0449394775214773,
+      "grad_norm": 0.8734195232391357,
+      "learning_rate": 6.367576432001875e-05,
+      "loss": 0.5347,
+      "step": 29100
+    },
+    {
+      "epoch": 2.051966760949386,
+      "grad_norm": 0.7100592255592346,
+      "learning_rate": 6.32072156495256e-05,
+      "loss": 0.5349,
+      "step": 29200
+    },
+    {
+      "epoch": 2.058994044377295,
+      "grad_norm": 0.5094404816627502,
+      "learning_rate": 6.273866697903245e-05,
+      "loss": 0.5438,
+      "step": 29300
+    },
+    {
+      "epoch": 2.0660213278052035,
+      "grad_norm": 0.5938568711280823,
+      "learning_rate": 6.22701183085393e-05,
+      "loss": 0.5367,
+      "step": 29400
+    },
+    {
+      "epoch": 2.0730486112331126,
+      "grad_norm": 0.6052954196929932,
+      "learning_rate": 6.180156963804615e-05,
+      "loss": 0.533,
+      "step": 29500
+    },
+    {
+      "epoch": 2.0730486112331126,
+      "eval_loss": 0.6042247414588928,
+      "eval_runtime": 119.3016,
+      "eval_samples_per_second": 106.025,
+      "eval_steps_per_second": 13.261,
+      "step": 29500
+    },
+    {
+      "epoch": 2.0800758946610216,
+      "grad_norm": 0.8204342126846313,
+      "learning_rate": 6.133302096755302e-05,
+      "loss": 0.5525,
+      "step": 29600
+    },
+    {
+      "epoch": 2.08710317808893,
+      "grad_norm": 0.8169859647750854,
+      "learning_rate": 6.086447229705986e-05,
+      "loss": 0.5412,
+      "step": 29700
+    },
+    {
+      "epoch": 2.0941304615168392,
+      "grad_norm": 0.6919510960578918,
+      "learning_rate": 6.039592362656671e-05,
+      "loss": 0.5395,
+      "step": 29800
+    },
+    {
+      "epoch": 2.101157744944748,
+      "grad_norm": 0.6376796364784241,
+      "learning_rate": 5.9927374956073566e-05,
+      "loss": 0.5296,
+      "step": 29900
+    },
+    {
+      "epoch": 2.108185028372657,
+      "grad_norm": 0.6056246161460876,
+      "learning_rate": 5.9458826285580416e-05,
+      "loss": 0.5257,
+      "step": 30000
+    },
+    {
+      "epoch": 2.108185028372657,
+      "eval_loss": 0.6040454506874084,
+      "eval_runtime": 119.4721,
+      "eval_samples_per_second": 105.874,
+      "eval_steps_per_second": 13.242,
+      "step": 30000
+    },
+    {
+      "epoch": 2.115212311800566,
+      "grad_norm": 0.7951282858848572,
+      "learning_rate": 5.899027761508727e-05,
+      "loss": 0.5231,
+      "step": 30100
+    },
+    {
+      "epoch": 2.1222395952284745,
+      "grad_norm": 0.7288519144058228,
+      "learning_rate": 5.852172894459412e-05,
+      "loss": 0.533,
+      "step": 30200
+    },
+    {
+      "epoch": 2.1292668786563835,
+      "grad_norm": 0.62901371717453,
+      "learning_rate": 5.8053180274100973e-05,
+      "loss": 0.5475,
+      "step": 30300
+    },
+    {
+      "epoch": 2.136294162084292,
+      "grad_norm": 0.7269171476364136,
+      "learning_rate": 5.758463160360783e-05,
+      "loss": 0.5469,
+      "step": 30400
+    },
+    {
+      "epoch": 2.143321445512201,
+      "grad_norm": 0.8678474426269531,
+      "learning_rate": 5.711608293311468e-05,
+      "loss": 0.5396,
+      "step": 30500
+    },
+    {
+      "epoch": 2.143321445512201,
+      "eval_loss": 0.6033341884613037,
+      "eval_runtime": 119.4525,
+      "eval_samples_per_second": 105.891,
+      "eval_steps_per_second": 13.244,
+      "step": 30500
+    },
+    {
+      "epoch": 2.15034872894011,
+      "grad_norm": 0.5461506843566895,
+      "learning_rate": 5.664753426262154e-05,
+      "loss": 0.5543,
+      "step": 30600
+    },
+    {
+      "epoch": 2.157376012368019,
+      "grad_norm": 0.6282551884651184,
+      "learning_rate": 5.617898559212839e-05,
+      "loss": 0.519,
+      "step": 30700
+    },
+    {
+      "epoch": 2.164403295795928,
+      "grad_norm": 0.5805559158325195,
+      "learning_rate": 5.571043692163523e-05,
+      "loss": 0.5359,
+      "step": 30800
+    },
+    {
+      "epoch": 2.1714305792238364,
+      "grad_norm": 0.7047603726387024,
+      "learning_rate": 5.5241888251142095e-05,
+      "loss": 0.5244,
+      "step": 30900
+    },
+    {
+      "epoch": 2.1784578626517455,
+      "grad_norm": 1.0538957118988037,
+      "learning_rate": 5.477333958064894e-05,
+      "loss": 0.5284,
+      "step": 31000
+    },
+    {
+      "epoch": 2.1784578626517455,
+      "eval_loss": 0.6041498780250549,
+      "eval_runtime": 119.3834,
+      "eval_samples_per_second": 105.953,
+      "eval_steps_per_second": 13.251,
+      "step": 31000
+    },
+    {
+      "epoch": 2.185485146079654,
+      "grad_norm": 0.7913850545883179,
+      "learning_rate": 5.43047909101558e-05,
+      "loss": 0.5267,
+      "step": 31100
+    },
+    {
+      "epoch": 2.192512429507563,
+      "grad_norm": 0.5944955348968506,
+      "learning_rate": 5.3836242239662646e-05,
+      "loss": 0.5359,
+      "step": 31200
+    },
+    {
+      "epoch": 2.199539712935472,
+      "grad_norm": 0.8068099617958069,
+      "learning_rate": 5.3367693569169496e-05,
+      "loss": 0.5449,
+      "step": 31300
+    },
+    {
+      "epoch": 2.2065669963633807,
+      "grad_norm": 0.5993140935897827,
+      "learning_rate": 5.289914489867635e-05,
+      "loss": 0.5384,
+      "step": 31400
+    },
+    {
+      "epoch": 2.2135942797912898,
+      "grad_norm": 0.6852918267250061,
+      "learning_rate": 5.24305962281832e-05,
+      "loss": 0.5448,
+      "step": 31500
+    },
+    {
+      "epoch": 2.2135942797912898,
+      "eval_loss": 0.6034653782844543,
+      "eval_runtime": 119.4509,
+      "eval_samples_per_second": 105.893,
+      "eval_steps_per_second": 13.244,
+      "step": 31500
+    },
+    {
+      "epoch": 2.2206215632191983,
+      "grad_norm": 0.9072486758232117,
+      "learning_rate": 5.1962047557690054e-05,
+      "loss": 0.5392,
+      "step": 31600
+    },
+    {
+      "epoch": 2.2276488466471074,
+      "grad_norm": 0.5671890377998352,
+      "learning_rate": 5.149349888719691e-05,
+      "loss": 0.5366,
+      "step": 31700
+    },
+    {
+      "epoch": 2.2346761300750164,
+      "grad_norm": 0.9552319049835205,
+      "learning_rate": 5.102495021670376e-05,
+      "loss": 0.555,
+      "step": 31800
+    },
+    {
+      "epoch": 2.241703413502925,
+      "grad_norm": 0.9220768809318542,
+      "learning_rate": 5.055640154621062e-05,
+      "loss": 0.5323,
+      "step": 31900
+    },
+    {
+      "epoch": 2.248730696930834,
+      "grad_norm": 0.7823670506477356,
+      "learning_rate": 5.008785287571747e-05,
+      "loss": 0.5484,
+      "step": 32000
+    },
+    {
+      "epoch": 2.248730696930834,
+      "eval_loss": 0.602741539478302,
+      "eval_runtime": 119.5139,
+      "eval_samples_per_second": 105.837,
+      "eval_steps_per_second": 13.237,
+      "step": 32000
+    },
+    {
+      "epoch": 2.2557579803587426,
+      "grad_norm": 0.8530369400978088,
+      "learning_rate": 4.961930420522432e-05,
+      "loss": 0.5443,
+      "step": 32100
+    },
+    {
+      "epoch": 2.2627852637866517,
+      "grad_norm": 0.8256222605705261,
+      "learning_rate": 4.9150755534731175e-05,
+      "loss": 0.5371,
+      "step": 32200
+    },
+    {
+      "epoch": 2.2698125472145607,
+      "grad_norm": 0.8003319501876831,
+      "learning_rate": 4.8682206864238025e-05,
+      "loss": 0.5472,
+      "step": 32300
+    },
+    {
+      "epoch": 2.2768398306424693,
+      "grad_norm": 0.6615211367607117,
+      "learning_rate": 4.8213658193744876e-05,
+      "loss": 0.5349,
+      "step": 32400
+    },
+    {
+      "epoch": 2.2838671140703783,
+      "grad_norm": 0.7718948125839233,
+      "learning_rate": 4.774510952325173e-05,
+      "loss": 0.5253,
+      "step": 32500
+    },
+    {
+      "epoch": 2.2838671140703783,
+      "eval_loss": 0.6034336090087891,
+      "eval_runtime": 119.3326,
+      "eval_samples_per_second": 105.998,
+      "eval_steps_per_second": 13.257,
+      "step": 32500
+    },
+    {
+      "epoch": 2.290894397498287,
+      "grad_norm": 0.6703356504440308,
+      "learning_rate": 4.727656085275858e-05,
+      "loss": 0.5356,
+      "step": 32600
+    },
+    {
+      "epoch": 2.297921680926196,
+      "grad_norm": 0.6417832970619202,
+      "learning_rate": 4.680801218226544e-05,
+      "loss": 0.5456,
+      "step": 32700
+    },
+    {
+      "epoch": 2.3049489643541046,
+      "grad_norm": 0.6749237775802612,
+      "learning_rate": 4.633946351177228e-05,
+      "loss": 0.5374,
+      "step": 32800
+    },
+    {
+      "epoch": 2.3119762477820136,
+      "grad_norm": 0.6223445534706116,
+      "learning_rate": 4.587091484127914e-05,
+      "loss": 0.5254,
+      "step": 32900
+    },
+    {
+      "epoch": 2.3190035312099226,
+      "grad_norm": 0.6455600261688232,
+      "learning_rate": 4.540236617078599e-05,
+      "loss": 0.5322,
+      "step": 33000
+    },
+    {
+      "epoch": 2.3190035312099226,
+      "eval_loss": 0.602424144744873,
+      "eval_runtime": 119.6802,
+      "eval_samples_per_second": 105.69,
+      "eval_steps_per_second": 13.219,
+      "step": 33000
+    },
+    {
+      "epoch": 2.3260308146378312,
+      "grad_norm": 0.6882891058921814,
+      "learning_rate": 4.493381750029285e-05,
+      "loss": 0.5406,
+      "step": 33100
+    },
+    {
+      "epoch": 2.3330580980657403,
+      "grad_norm": 0.7169196605682373,
+      "learning_rate": 4.44652688297997e-05,
+      "loss": 0.5367,
+      "step": 33200
+    },
+    {
+      "epoch": 2.3400853814936493,
+      "grad_norm": 0.6603942513465881,
+      "learning_rate": 4.399672015930655e-05,
+      "loss": 0.5315,
+      "step": 33300
+    },
+    {
+      "epoch": 2.347112664921558,
+      "grad_norm": 0.7974775433540344,
+      "learning_rate": 4.3528171488813405e-05,
+      "loss": 0.5438,
+      "step": 33400
+    },
+    {
+      "epoch": 2.354139948349467,
+      "grad_norm": 0.7672884464263916,
+      "learning_rate": 4.3059622818320255e-05,
+      "loss": 0.5583,
+      "step": 33500
+    },
+    {
+      "epoch": 2.354139948349467,
+      "eval_loss": 0.6008437871932983,
+      "eval_runtime": 119.0367,
+      "eval_samples_per_second": 106.261,
+      "eval_steps_per_second": 13.29,
+      "step": 33500
+    },
+    {
+      "epoch": 2.3611672317773755,
+      "grad_norm": 0.7062329649925232,
+      "learning_rate": 4.259107414782711e-05,
+      "loss": 0.5548,
+      "step": 33600
+    },
+    {
+      "epoch": 2.3681945152052846,
+      "grad_norm": 0.754173219203949,
+      "learning_rate": 4.212252547733396e-05,
+      "loss": 0.5276,
+      "step": 33700
+    },
+    {
+      "epoch": 2.375221798633193,
+      "grad_norm": 0.7143212556838989,
+      "learning_rate": 4.165397680684081e-05,
+      "loss": 0.5413,
+      "step": 33800
+    },
+    {
+      "epoch": 2.382249082061102,
+      "grad_norm": 0.8082584142684937,
+      "learning_rate": 4.118542813634766e-05,
+      "loss": 0.534,
+      "step": 33900
+    },
+    {
+      "epoch": 2.3892763654890112,
+      "grad_norm": 0.7493578791618347,
+      "learning_rate": 4.071687946585452e-05,
+      "loss": 0.5373,
+      "step": 34000
+    },
+    {
+      "epoch": 2.3892763654890112,
+      "eval_loss": 0.6015102863311768,
+      "eval_runtime": 119.2005,
+      "eval_samples_per_second": 106.115,
+      "eval_steps_per_second": 13.272,
+      "step": 34000
+    },
+    {
+      "epoch": 2.39630364891692,
+      "grad_norm": 0.9309408068656921,
+      "learning_rate": 4.024833079536137e-05,
+      "loss": 0.5464,
+      "step": 34100
+    },
+    {
+      "epoch": 2.403330932344829,
+      "grad_norm": 0.6309605240821838,
+      "learning_rate": 3.977978212486822e-05,
+      "loss": 0.5439,
+      "step": 34200
+    },
+    {
+      "epoch": 2.410358215772738,
+      "grad_norm": 0.6428382992744446,
+      "learning_rate": 3.931123345437507e-05,
+      "loss": 0.5412,
+      "step": 34300
+    },
+    {
+      "epoch": 2.4173854992006465,
+      "grad_norm": 0.9063606262207031,
+      "learning_rate": 3.884268478388193e-05,
+      "loss": 0.5477,
+      "step": 34400
+    },
+    {
+      "epoch": 2.4244127826285555,
+      "grad_norm": 0.7051374316215515,
+      "learning_rate": 3.837413611338878e-05,
+      "loss": 0.5351,
+      "step": 34500
+    },
+    {
+      "epoch": 2.4244127826285555,
+      "eval_loss": 0.6011614203453064,
+      "eval_runtime": 119.6428,
+      "eval_samples_per_second": 105.723,
+      "eval_steps_per_second": 13.223,
+      "step": 34500
+    },
+    {
+      "epoch": 2.431440066056464,
+      "grad_norm": 0.7466573119163513,
+      "learning_rate": 3.7905587442895635e-05,
+      "loss": 0.5525,
+      "step": 34600
+    },
+    {
+      "epoch": 2.438467349484373,
+      "grad_norm": 0.8113718628883362,
+      "learning_rate": 3.7437038772402485e-05,
+      "loss": 0.553,
+      "step": 34700
+    },
+    {
+      "epoch": 2.4454946329122818,
+      "grad_norm": 0.5469939112663269,
+      "learning_rate": 3.6968490101909335e-05,
+      "loss": 0.5438,
+      "step": 34800
+    },
+    {
+      "epoch": 2.452521916340191,
+      "grad_norm": 0.6911020278930664,
+      "learning_rate": 3.649994143141619e-05,
+      "loss": 0.5349,
+      "step": 34900
+    },
+    {
+      "epoch": 2.4595491997681,
+      "grad_norm": 0.7164533734321594,
+      "learning_rate": 3.603139276092304e-05,
+      "loss": 0.5237,
+      "step": 35000
+    },
+    {
+      "epoch": 2.4595491997681,
+      "eval_loss": 0.6016719937324524,
+      "eval_runtime": 119.4046,
+      "eval_samples_per_second": 105.934,
+      "eval_steps_per_second": 13.249,
+      "step": 35000
+    },
+    {
+      "epoch": 2.4665764831960084,
+      "grad_norm": 0.8674111366271973,
+      "learning_rate": 3.55628440904299e-05,
+      "loss": 0.5469,
+      "step": 35100
+    },
+    {
+      "epoch": 2.4736037666239175,
+      "grad_norm": 0.9044885039329529,
+      "learning_rate": 3.509429541993674e-05,
+      "loss": 0.5445,
+      "step": 35200
+    },
+    {
+      "epoch": 2.480631050051826,
+      "grad_norm": 0.7213521599769592,
+      "learning_rate": 3.46257467494436e-05,
+      "loss": 0.5136,
+      "step": 35300
+    },
+    {
+      "epoch": 2.487658333479735,
+      "grad_norm": 0.6873759627342224,
+      "learning_rate": 3.415719807895045e-05,
+      "loss": 0.5362,
+      "step": 35400
+    },
+    {
+      "epoch": 2.494685616907644,
+      "grad_norm": 0.6950516104698181,
+      "learning_rate": 3.368864940845731e-05,
+      "loss": 0.535,
+      "step": 35500
+    },
+    {
+      "epoch": 2.494685616907644,
+      "eval_loss": 0.6001349091529846,
+      "eval_runtime": 119.0417,
+      "eval_samples_per_second": 106.257,
+      "eval_steps_per_second": 13.289,
+      "step": 35500
+    },
+    {
+      "epoch": 2.5017129003355527,
+      "grad_norm": 0.6840397715568542,
+      "learning_rate": 3.322010073796416e-05,
+      "loss": 0.5385,
+      "step": 35600
+    },
+    {
+      "epoch": 2.5087401837634618,
+      "grad_norm": 0.5907210111618042,
+      "learning_rate": 3.275155206747101e-05,
+      "loss": 0.5429,
+      "step": 35700
+    },
+    {
+      "epoch": 2.5157674671913703,
+      "grad_norm": 0.801002025604248,
+      "learning_rate": 3.2283003396977865e-05,
+      "loss": 0.5224,
+      "step": 35800
+    },
+    {
+      "epoch": 2.5227947506192794,
+      "grad_norm": 0.7259572148323059,
+      "learning_rate": 3.1814454726484715e-05,
+      "loss": 0.5297,
+      "step": 35900
+    },
+    {
+      "epoch": 2.5298220340471884,
+      "grad_norm": 0.6602583527565002,
+      "learning_rate": 3.134590605599157e-05,
+      "loss": 0.5496,
+      "step": 36000
+    },
+    {
+      "epoch": 2.5298220340471884,
+      "eval_loss": 0.5990512371063232,
+      "eval_runtime": 118.9771,
+      "eval_samples_per_second": 106.315,
+      "eval_steps_per_second": 13.297,
+      "step": 36000
+    },
+    {
+      "epoch": 2.536849317475097,
+      "grad_norm": 0.6192248463630676,
+      "learning_rate": 3.0877357385498415e-05,
+      "loss": 0.5432,
+      "step": 36100
+    },
+    {
+      "epoch": 2.543876600903006,
+      "grad_norm": 1.0253371000289917,
+      "learning_rate": 3.0408808715005272e-05,
+      "loss": 0.5254,
+      "step": 36200
+    },
+    {
+      "epoch": 2.5509038843309146,
+      "grad_norm": 0.8612440228462219,
+      "learning_rate": 2.9940260044512126e-05,
+      "loss": 0.56,
+      "step": 36300
+    },
+    {
+      "epoch": 2.5579311677588237,
+      "grad_norm": 0.5663143992424011,
+      "learning_rate": 2.947171137401898e-05,
+      "loss": 0.5329,
+      "step": 36400
+    },
+    {
+      "epoch": 2.5649584511867323,
+      "grad_norm": 0.9142153263092041,
+      "learning_rate": 2.9003162703525833e-05,
+      "loss": 0.5384,
+      "step": 36500
+    },
+    {
+      "epoch": 2.5649584511867323,
+      "eval_loss": 0.5994681715965271,
+      "eval_runtime": 119.2506,
+      "eval_samples_per_second": 106.071,
+      "eval_steps_per_second": 13.266,
+      "step": 36500
+    },
+    {
+      "epoch": 2.5719857346146413,
+      "grad_norm": 0.6240784525871277,
+      "learning_rate": 2.853461403303268e-05,
+      "loss": 0.5332,
+      "step": 36600
+    },
+    {
+      "epoch": 2.5790130180425503,
+      "grad_norm": 0.7204896211624146,
+      "learning_rate": 2.8066065362539534e-05,
+      "loss": 0.534,
+      "step": 36700
+    },
+    {
+      "epoch": 2.586040301470459,
+      "grad_norm": 0.8071198463439941,
+      "learning_rate": 2.7597516692046387e-05,
+      "loss": 0.533,
+      "step": 36800
+    },
+    {
+      "epoch": 2.593067584898368,
+      "grad_norm": 0.5369657278060913,
+      "learning_rate": 2.712896802155324e-05,
+      "loss": 0.5325,
+      "step": 36900
+    },
+    {
+      "epoch": 2.600094868326277,
+      "grad_norm": 0.8969751000404358,
+      "learning_rate": 2.6660419351060095e-05,
+      "loss": 0.5349,
+      "step": 37000
+    },
+    {
+      "epoch": 2.600094868326277,
+      "eval_loss": 0.5992428064346313,
+      "eval_runtime": 119.3558,
+      "eval_samples_per_second": 105.977,
+      "eval_steps_per_second": 13.254,
+      "step": 37000
+    },
+    {
+      "epoch": 2.6071221517541856,
+      "grad_norm": 0.8350916504859924,
+      "learning_rate": 2.6191870680566945e-05,
+      "loss": 0.5388,
+      "step": 37100
+    },
+    {
+      "epoch": 2.6141494351820946,
+      "grad_norm": 0.5793244242668152,
+      "learning_rate": 2.57233220100738e-05,
+      "loss": 0.5211,
+      "step": 37200
+    },
+    {
+      "epoch": 2.6211767186100032,
+      "grad_norm": 0.6331949830055237,
+      "learning_rate": 2.5254773339580652e-05,
+      "loss": 0.5335,
+      "step": 37300
+    },
+    {
+      "epoch": 2.6282040020379123,
+      "grad_norm": 0.799247145652771,
+      "learning_rate": 2.4786224669087502e-05,
+      "loss": 0.5474,
+      "step": 37400
+    },
+    {
+      "epoch": 2.635231285465821,
+      "grad_norm": 0.7149389386177063,
+      "learning_rate": 2.4317675998594356e-05,
+      "loss": 0.5316,
+      "step": 37500
+    },
+    {
+      "epoch": 2.635231285465821,
+      "eval_loss": 0.5990239977836609,
+      "eval_runtime": 119.2093,
+      "eval_samples_per_second": 106.107,
+      "eval_steps_per_second": 13.271,
+      "step": 37500
+    },
+    {
+      "epoch": 2.64225856889373,
+      "grad_norm": 0.6996687650680542,
+      "learning_rate": 2.384912732810121e-05,
+      "loss": 0.5244,
+      "step": 37600
+    },
+    {
+      "epoch": 2.649285852321639,
+      "grad_norm": 0.6374346017837524,
+      "learning_rate": 2.338057865760806e-05,
+      "loss": 0.5221,
+      "step": 37700
+    },
+    {
+      "epoch": 2.6563131357495475,
+      "grad_norm": 0.7678332328796387,
+      "learning_rate": 2.2912029987114913e-05,
+      "loss": 0.5251,
+      "step": 37800
+    },
+    {
+      "epoch": 2.6633404191774566,
+      "grad_norm": 0.6463155150413513,
+      "learning_rate": 2.2443481316621764e-05,
+      "loss": 0.5343,
+      "step": 37900
+    },
+    {
+      "epoch": 2.6703677026053656,
+      "grad_norm": 0.6788434386253357,
+      "learning_rate": 2.1974932646128617e-05,
+      "loss": 0.5276,
+      "step": 38000
+    },
+    {
+      "epoch": 2.6703677026053656,
+      "eval_loss": 0.5986095070838928,
+      "eval_runtime": 119.1797,
+      "eval_samples_per_second": 106.134,
+      "eval_steps_per_second": 13.274,
+      "step": 38000
+    },
+    {
+      "epoch": 2.677394986033274,
+      "grad_norm": 0.7115055322647095,
+      "learning_rate": 2.1506383975635467e-05,
+      "loss": 0.5583,
+      "step": 38100
+    },
+    {
+      "epoch": 2.684422269461183,
+      "grad_norm": 0.7007845044136047,
+      "learning_rate": 2.103783530514232e-05,
+      "loss": 0.5294,
+      "step": 38200
+    },
+    {
+      "epoch": 2.691449552889092,
+      "grad_norm": 1.0714610815048218,
+      "learning_rate": 2.0569286634649178e-05,
+      "loss": 0.54,
+      "step": 38300
+    },
+    {
+      "epoch": 2.698476836317001,
+      "grad_norm": 0.6736337542533875,
+      "learning_rate": 2.0100737964156028e-05,
+      "loss": 0.5343,
+      "step": 38400
+    },
+    {
+      "epoch": 2.7055041197449095,
+      "grad_norm": 0.6769545078277588,
+      "learning_rate": 1.9632189293662882e-05,
+      "loss": 0.5406,
+      "step": 38500
+    },
+    {
+      "epoch": 2.7055041197449095,
+      "eval_loss": 0.5980576276779175,
+      "eval_runtime": 119.2056,
+      "eval_samples_per_second": 106.111,
+      "eval_steps_per_second": 13.271,
+      "step": 38500
+    },
+    {
+      "epoch": 2.7125314031728185,
+      "grad_norm": 0.714820384979248,
+      "learning_rate": 1.9163640623169732e-05,
+      "loss": 0.5408,
+      "step": 38600
+    },
+    {
+      "epoch": 2.7195586866007275,
+      "grad_norm": 0.5211949944496155,
+      "learning_rate": 1.8695091952676586e-05,
+      "loss": 0.5257,
+      "step": 38700
+    },
+    {
+      "epoch": 2.726585970028636,
+      "grad_norm": 0.7966856360435486,
+      "learning_rate": 1.8226543282183436e-05,
+      "loss": 0.5381,
+      "step": 38800
+    },
+    {
+      "epoch": 2.733613253456545,
+      "grad_norm": 0.7282027006149292,
+      "learning_rate": 1.775799461169029e-05,
+      "loss": 0.5415,
+      "step": 38900
+    },
+    {
+      "epoch": 2.7406405368844537,
+      "grad_norm": 0.7190561890602112,
+      "learning_rate": 1.7289445941197143e-05,
+      "loss": 0.5436,
+      "step": 39000
+    },
+    {
+      "epoch": 2.7406405368844537,
+      "eval_loss": 0.5981852412223816,
+      "eval_runtime": 119.2717,
+      "eval_samples_per_second": 106.052,
+      "eval_steps_per_second": 13.264,
+      "step": 39000
+    },
+    {
+      "epoch": 2.747667820312363,
+      "grad_norm": 0.6859644651412964,
+      "learning_rate": 1.6820897270703993e-05,
+      "loss": 0.5371,
+      "step": 39100
+    },
+    {
+      "epoch": 2.7546951037402714,
+      "grad_norm": 0.6917553544044495,
+      "learning_rate": 1.6352348600210847e-05,
+      "loss": 0.5411,
+      "step": 39200
+    },
+    {
+      "epoch": 2.7617223871681804,
+      "grad_norm": 0.5737515687942505,
+      "learning_rate": 1.58837999297177e-05,
+      "loss": 0.5244,
+      "step": 39300
+    },
+    {
+      "epoch": 2.7687496705960895,
+      "grad_norm": 0.7747429013252258,
+      "learning_rate": 1.5415251259224554e-05,
+      "loss": 0.5452,
+      "step": 39400
+    },
+    {
+      "epoch": 2.775776954023998,
+      "grad_norm": 0.6782782673835754,
+      "learning_rate": 1.4946702588731405e-05,
+      "loss": 0.5392,
+      "step": 39500
+    },
+    {
+      "epoch": 2.775776954023998,
+      "eval_loss": 0.597685694694519,
+      "eval_runtime": 119.3501,
+      "eval_samples_per_second": 105.982,
+      "eval_steps_per_second": 13.255,
+      "step": 39500
+    },
+    {
+      "epoch": 2.782804237451907,
+      "grad_norm": 0.6675652265548706,
+      "learning_rate": 1.4478153918238258e-05,
+      "loss": 0.534,
+      "step": 39600
+    },
+    {
+      "epoch": 2.789831520879816,
+      "grad_norm": 0.822902262210846,
+      "learning_rate": 1.4009605247745112e-05,
+      "loss": 0.5582,
+      "step": 39700
+    },
+    {
+      "epoch": 2.7968588043077247,
+      "grad_norm": 0.6393033266067505,
+      "learning_rate": 1.3541056577251962e-05,
+      "loss": 0.5442,
+      "step": 39800
+    },
+    {
+      "epoch": 2.8038860877356337,
+      "grad_norm": 0.744646430015564,
+      "learning_rate": 1.3072507906758816e-05,
+      "loss": 0.537,
+      "step": 39900
+    },
+    {
+      "epoch": 2.8109133711635423,
+      "grad_norm": 0.5728178024291992,
+      "learning_rate": 1.2603959236265666e-05,
+      "loss": 0.5316,
+      "step": 40000
+    },
+    {
+      "epoch": 2.8109133711635423,
+      "eval_loss": 0.5975730419158936,
+      "eval_runtime": 119.1706,
+      "eval_samples_per_second": 106.142,
+      "eval_steps_per_second": 13.275,
+      "step": 40000
+    },
+    {
+      "epoch": 2.8179406545914514,
+      "grad_norm": 0.6350817680358887,
+      "learning_rate": 1.2135410565772521e-05,
+      "loss": 0.5345,
+      "step": 40100
+    },
+    {
+      "epoch": 2.82496793801936,
+      "grad_norm": 0.5875131487846375,
+      "learning_rate": 1.1666861895279373e-05,
+      "loss": 0.5336,
+      "step": 40200
+    },
+    {
+      "epoch": 2.831995221447269,
+      "grad_norm": 0.7108073234558105,
+      "learning_rate": 1.1198313224786225e-05,
+      "loss": 0.5364,
+      "step": 40300
+    },
+    {
+      "epoch": 2.839022504875178,
+      "grad_norm": 0.5973334312438965,
+      "learning_rate": 1.0729764554293077e-05,
+      "loss": 0.5468,
+      "step": 40400
+    },
+    {
+      "epoch": 2.8460497883030866,
+      "grad_norm": 0.7775806784629822,
+      "learning_rate": 1.026121588379993e-05,
+      "loss": 0.5349,
+      "step": 40500
+    },
+    {
+      "epoch": 2.8460497883030866,
+      "eval_loss": 0.5973463654518127,
+      "eval_runtime": 119.0857,
+      "eval_samples_per_second": 106.218,
+      "eval_steps_per_second": 13.285,
+      "step": 40500
+    },
+    {
+      "epoch": 2.8530770717309957,
+      "grad_norm": 0.6160515546798706,
+      "learning_rate": 9.792667213306782e-06,
+      "loss": 0.5317,
+      "step": 40600
+    },
+    {
+      "epoch": 2.8601043551589047,
+      "grad_norm": 0.715397834777832,
+      "learning_rate": 9.324118542813636e-06,
+      "loss": 0.5119,
+      "step": 40700
+    },
+    {
+      "epoch": 2.8671316385868133,
+      "grad_norm": 0.7450791001319885,
+      "learning_rate": 8.855569872320488e-06,
+      "loss": 0.5302,
+      "step": 40800
+    },
+    {
+      "epoch": 2.874158922014722,
+      "grad_norm": 0.5536558628082275,
+      "learning_rate": 8.38702120182734e-06,
+      "loss": 0.5239,
+      "step": 40900
+    },
+    {
+      "epoch": 2.881186205442631,
+      "grad_norm": 0.5844016075134277,
+      "learning_rate": 7.918472531334192e-06,
+      "loss": 0.5341,
+      "step": 41000
+    },
+    {
+      "epoch": 2.881186205442631,
+      "eval_loss": 0.5972412824630737,
+      "eval_runtime": 119.0402,
+      "eval_samples_per_second": 106.258,
+      "eval_steps_per_second": 13.29,
+      "step": 41000
+    },
+    {
+      "epoch": 2.88821348887054,
+      "grad_norm": 0.566956639289856,
+      "learning_rate": 7.449923860841045e-06,
+      "loss": 0.5387,
+      "step": 41100
+    },
+    {
+      "epoch": 2.8952407722984486,
+      "grad_norm": 0.6934293508529663,
+      "learning_rate": 6.981375190347898e-06,
+      "loss": 0.5185,
+      "step": 41200
+    },
+    {
+      "epoch": 2.9022680557263576,
+      "grad_norm": 0.6890417337417603,
+      "learning_rate": 6.51282651985475e-06,
+      "loss": 0.5419,
+      "step": 41300
+    },
+    {
+      "epoch": 2.9092953391542666,
+      "grad_norm": 0.8735133409500122,
+      "learning_rate": 6.044277849361603e-06,
+      "loss": 0.5314,
+      "step": 41400
+    },
+    {
+      "epoch": 2.9163226225821752,
+      "grad_norm": 0.746161162853241,
+      "learning_rate": 5.575729178868455e-06,
+      "loss": 0.5208,
+      "step": 41500
+    },
+    {
+      "epoch": 2.9163226225821752,
+      "eval_loss": 0.5972864031791687,
+      "eval_runtime": 118.9512,
+      "eval_samples_per_second": 106.338,
+      "eval_steps_per_second": 13.3,
+      "step": 41500
+    },
+    {
+      "epoch": 2.9233499060100843,
+      "grad_norm": 0.9560967683792114,
+      "learning_rate": 5.107180508375308e-06,
+      "loss": 0.5505,
+      "step": 41600
+    },
+    {
+      "epoch": 2.930377189437993,
+      "grad_norm": 0.7499198317527771,
+      "learning_rate": 4.63863183788216e-06,
+      "loss": 0.5378,
+      "step": 41700
+    },
+    {
+      "epoch": 2.937404472865902,
+      "grad_norm": 0.6939593553543091,
+      "learning_rate": 4.170083167389012e-06,
+      "loss": 0.5351,
+      "step": 41800
+    },
+    {
+      "epoch": 2.9444317562938105,
+      "grad_norm": 0.6184145212173462,
+      "learning_rate": 3.701534496895865e-06,
+      "loss": 0.5171,
+      "step": 41900
+    },
+    {
+      "epoch": 2.9514590397217195,
+      "grad_norm": 0.6636520624160767,
+      "learning_rate": 3.232985826402718e-06,
+      "loss": 0.53,
+      "step": 42000
+    },
+    {
+      "epoch": 2.9514590397217195,
+      "eval_loss": 0.5971269607543945,
+      "eval_runtime": 118.9184,
+      "eval_samples_per_second": 106.367,
+      "eval_steps_per_second": 13.303,
+      "step": 42000
+    },
+    {
+      "epoch": 2.9584863231496286,
+      "grad_norm": 0.8458754420280457,
+      "learning_rate": 2.76443715590957e-06,
+      "loss": 0.5447,
+      "step": 42100
+    },
+    {
+      "epoch": 2.965513606577537,
+      "grad_norm": 0.6426025032997131,
+      "learning_rate": 2.295888485416423e-06,
+      "loss": 0.5287,
+      "step": 42200
+    },
+    {
+      "epoch": 2.972540890005446,
+      "grad_norm": 0.7159422039985657,
+      "learning_rate": 1.827339814923275e-06,
+      "loss": 0.5288,
+      "step": 42300
+    },
+    {
+      "epoch": 2.9795681734333552,
+      "grad_norm": 0.6702597141265869,
+      "learning_rate": 1.3587911444301279e-06,
+      "loss": 0.5419,
+      "step": 42400
+    },
+    {
+      "epoch": 2.986595456861264,
+      "grad_norm": 0.7953273057937622,
+      "learning_rate": 8.902424739369803e-07,
+      "loss": 0.5445,
+      "step": 42500
+    },
+    {
+      "epoch": 2.986595456861264,
+      "eval_loss": 0.5969375371932983,
+      "eval_runtime": 119.2187,
+      "eval_samples_per_second": 106.099,
+      "eval_steps_per_second": 13.27,
+      "step": 42500
+    },
+    {
+      "epoch": 2.993622740289173,
+      "grad_norm": 0.5247675180435181,
+      "learning_rate": 4.216938034438327e-07,
+      "loss": 0.5485,
+      "step": 42600
     }
   ],
   "logging_steps": 100,
@@ -2429,12 +3683,12 @@
         "should_evaluate": false,
         "should_log": false,
         "should_save": true,
-        "should_training_stop": false
+        "should_training_stop": true
       },
       "attributes": {}
     }
   },
-  "total_flos": 9.547282008374477e+16,
+  "total_flos": 1.4552903446892544e+17,
   "train_batch_size": 2,
   "trial_name": null,
   "trial_params": null