{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "IqM-T1RTzY6C" }, "source": [ "To run this, press \"*Runtime*\" and press \"*Run all*\" on a **free** Tesla T4 Google Colab instance!\n", "
\n", " \n", " \n", " Join Discord if you need help + ⭐ Star us on Github ⭐\n", "
\n", "\n", "To install Unsloth on your own computer, follow the installation instructions on our Github page [here](https://github.com/unslothai/unsloth?tab=readme-ov-file#-installation-instructions).\n", "\n", "You will learn how to do [data prep](#Data), how to [train](#Train), how to [run the model](#Inference), & [how to save it](#Save) (eg for Llama.cpp).\n", "\n", "**[NEW] Try 2x faster inference in a free Colab for Llama-3.1 8b Instruct [here](https://colab.research.google.com/drive/1T-YBVfnphoVc8E2E854qF3jdia2Ll2W2?usp=sharing)**\n", "\n", "Features in the notebook:\n", "1. Uses Maxime Labonne's [FineTome 100K](https://huggingface.co/datasets/mlabonne/FineTome-100k) dataset.\n", "1. Convert ShareGPT to HuggingFace format via `standardize_sharegpt`\n", "2. Train on Completions / Assistant only via `train_on_responses_only`\n", "3. Unsloth now supports Torch 2.4, all TRL & Xformers versions & Python 3.12!" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "id": "2eSvM9zX_2d3" }, "outputs": [], "source": [ "%%capture\n", "!pip install unsloth\n", "# Also get the latest nightly Unsloth!\n", "!pip uninstall unsloth -y && pip install --upgrade --no-cache-dir --no-deps git+https://github.com/unslothai/unsloth.git" ] }, { "cell_type": "markdown", "metadata": { "id": "r2v_X2fA0Df5" }, "source": [ "* We support Llama, Mistral, Phi-3, Gemma, Yi, DeepSeek, Qwen, TinyLlama, Vicuna, Open Hermes etc\n", "* We support 16bit LoRA or 4bit QLoRA. Both 2x faster.\n", "* `max_seq_length` can be set to anything, since we do automatic RoPE Scaling via [kaiokendev's](https://kaiokendev.github.io/til) method.\n", "* [**NEW**] We make Gemma-2 9b / 27b **2x faster**! See our [Gemma-2 9b notebook](https://colab.research.google.com/drive/1vIrqH5uYDQwsJ4-OO3DErvuv4pBgVwk4?usp=sharing)\n", "* [**NEW**] To finetune and auto export to Ollama, try our [Ollama notebook](https://colab.research.google.com/drive/1WZDi7APtQ9VsvOrQSSC5DDtxq159j8iZ?usp=sharing)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 320, "referenced_widgets": [ "a29682d0394e46eeabddad80a00349f7", "abbe988281f643b29bcbc74a5d8c5fe4", "5f1d65686d1a4602a6ee4c5d13df7818", "59b4df6e930f457cb67a755d90a651a7", "9da93d5b27d947bea044bc1119181d0d", "ba27a4ca72164a06b85c8e58493d75e1", "84f2e2eff5dd44b89b4390e544ea31a9", "3ac8a0c853a6406ea3fb91d9e7ce743b", "c569a6ed0158456ebc2f3c0ed206d73a", "a0c39ec65e674b628219a1dfaa177096", "a5c288c762214c8d93d69d3351f5832f", "81afdcf1237944c2a2b7402662d8a643", "5581073a34c2423db732d3266161ed07", "7b477d93171a4628a536a5d51ede5e7d", "aa18c1a527804c3485881555b4cccaae", "4aa3b4a195f64907aee323b2129d6ada", "cbd6084641454e9c8f4fce020e2476fa", "9ba68f77a0e849949ca1745d68978e1c", "412923ecea3644a381847ce54df4a56a", "ad804015a2154bcca7dc2fcc53caff70", "671467ee73ed43d6865f0af2c8f908e0", "d88b2a3302264b6289a401aafe72c12d", "20f7a9b4bc3542578f9c59adc64cbc4b", "1d9cd463d7ee42d4848661bb1cfefe8a", "641165dcd1ae4d78acdbd79a6d1e8bb7", "b1d640cebe5841b28ab07234c2989414", "2a87255dcbfa45dcbd56a28cc05288bd", "ccea16c679d44ac99847a0032d71b1d5", "4783c67f36244488932c4a2ba6fe1bd7", "b6bb256941f24137b4c696b5c486ef95", "09a8c63c84bf4fbd87a3aebf78bc65f3", "a72a8ba21b674d30b3cd7c019d29f77d", "bd41e0af677847d0bf3c3628aa3bb013", "913d96e92dea4721af1f6c37dfb75b95", "bc5bec8baef3453a8402af85fa23ae61", "29c108f189994f649b8b84a9c3e68901", "6f65e0467ad445fcaea61764b19d27b4", "2a8817c33b0048d3904c9e818ff193ec", "f18cdcec6edb4d57920446ffdea0f0a2", "793edc6cacf34d8d9b798205d6c512bd", "8d310673bfca48eeaa0db3aeb8f9cc6d", "cce78e2bc5f54ffd954019654d5a5f05", "6f95a56ad2ca4fcdaf11da285d0d2a62", "c966ac79c2e04c8f8c9611889f4e2fb6", "17ce7feec43f4e2d9bc863899ee4f94e", "8d28e864ccdf4a4896dfd5c008d0919c", "ad73d99d088642589e399ebc5c4e44c3", "a653bb2556b24770a8215750500b75a4", "fbb10c5128374157a96d09fec3511387", "66d611d034644087bd42856e3eca9a88", "f323a3e2c9ea4a95bd899442db6777ca", "e0b5f80fb558464e8bfb2f5349bf7cff", "83f9b80e5e0a46bfaa55a1d185b81bc7", "dd627e4aabe6470e95161caed0a50100", "984a5053afd5447fb3b670687b151bb4" ] }, "id": "QmUBVEnvCDJv", "outputId": "b63288a8-a0ae-46fe-ecff-0670a9316185" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.\n", "🦥 Unsloth Zoo will now patch everything to make training faster!\n", "==((====))== Unsloth 2024.11.9: Fast Llama patching. Transformers = 4.46.2.\n", " \\\\ /| GPU: Tesla T4. Max memory: 14.748 GB. Platform = Linux.\n", "O^O/ \\_/ \\ Pytorch: 2.5.1+cu121. CUDA = 7.5. CUDA Toolkit = 12.1.\n", "\\ / Bfloat16 = FALSE. FA [Xformers = 0.0.28.post3. FA2 = False]\n", " \"-____-\" Free Apache license: http://github.com/unslothai/unsloth\n", "Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored!\n" ] }, { "output_type": "display_data", "data": { "text/plain": [ "model.safetensors: 0%| | 0.00/1.03G [00:00 0 ! Suggested 8, 16, 32, 64, 128\n", " target_modules = [\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n", " \"gate_proj\", \"up_proj\", \"down_proj\",],\n", " lora_alpha = 16,\n", " lora_dropout = 0, # Supports any, but = 0 is optimized\n", " bias = \"none\", # Supports any, but = \"none\" is optimized\n", " # [NEW] \"unsloth\" uses 30% less VRAM, fits 2x larger batch sizes!\n", " use_gradient_checkpointing = \"unsloth\", # True or \"unsloth\" for very long context\n", " random_state = 3407,\n", " use_rslora = True, # We support rank stabilized LoRA\n", " loftq_config = None, # And LoftQ\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "vITh0KVJ10qX" }, "source": [ "\n", "### Data Prep\n", "We now use the `Llama-3.1` format for conversation style finetunes. We use [Maxime Labonne's FineTome-100k](https://huggingface.co/datasets/mlabonne/FineTome-100k) dataset in ShareGPT style. But we convert it to HuggingFace's normal multiturn format `(\"role\", \"content\")` instead of `(\"from\", \"value\")`/ Llama-3 renders multi turn conversations like below:\n", "\n", "```\n", "<|begin_of_text|><|start_header_id|>user<|end_header_id|>\n", "\n", "Hello!<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n", "\n", "Hey there! How are you?<|eot_id|><|start_header_id|>user<|end_header_id|>\n", "\n", "I'm great thanks!<|eot_id|>\n", "```\n", "\n", "We use our `get_chat_template` function to get the correct chat template. We support `zephyr, chatml, mistral, llama, alpaca, vicuna, vicuna_old, phi3, llama3` and more." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "id": "LjY75GoYUCB8" }, "outputs": [], "source": [ "from unsloth.chat_templates import get_chat_template\n", "\n", "tokenizer = get_chat_template(\n", " tokenizer,\n", " chat_template = \"llama-3.1\",\n", ")\n", "\n", "def formatting_prompts_func(examples):\n", " convos = examples[\"conversations\"]\n", " texts = [tokenizer.apply_chat_template(convo, tokenize = False, add_generation_prompt = False) for convo in convos]\n", " return { \"text\" : texts, }\n", "pass\n", "\n", "from datasets import load_dataset\n", "dataset = load_dataset(\"mlabonne/FineTome-100k\", split = \"train\") #\"train[:10000]\"" ] }, { "cell_type": "markdown", "metadata": { "id": "K9CBpiISFa6C" }, "source": [ "We now use `standardize_sharegpt` to convert ShareGPT style datasets into HuggingFace's generic format. This changes the dataset from looking like:\n", "```\n", "{\"from\": \"system\", \"value\": \"You are an assistant\"}\n", "{\"from\": \"human\", \"value\": \"What is 2+2?\"}\n", "{\"from\": \"gpt\", \"value\": \"It's 4.\"}\n", "```\n", "to\n", "```\n", "{\"role\": \"system\", \"content\": \"You are an assistant\"}\n", "{\"role\": \"user\", \"content\": \"What is 2+2?\"}\n", "{\"role\": \"assistant\", \"content\": \"It's 4.\"}\n", "```" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 81, "referenced_widgets": [ "039f8b9f542144a08606ec7dd0c94ce8", "a4eebe5d52fc4e78b9d2e6e603a46766", "5086fde5f6ee4c7cb6e2f0062612cb11", "1a407d68e32c47f6acf3c1e034e46ee7", "13891348033343deac25241d312baa78", "3ec772c8e1c9484b93249b2942e7a0c0", "7c1ab1e5f0624e34b9bfc97e32838370", "58396311734f41b3b1acfa61c420680e", "2427429328f84f5b92d6bf95c515a635", "13526476ee3c41c983158a502e824a75", "cac0e01bb5144a199c55f9e6801c62ce", "c6f91eed614c4ea88dfc597115023fa8", "8318e9e5c785478889781734e3b09452", "a8a9e7d1ce7a4989896ce9fedd3b8411", "34678221a3414b6fb95473188735e7fb", "6587b7787ab5482697cb5ddcc26ff3de", "c485116dfe48427ca9d5381a01d44741", "f8a8c69a9a814163864fa3fd1c18035c", "850da0477eb0404084e6bcedcf461c50", "0110c1baa42d4b40834f5962ff22030e", "50be91e8663d4ee3b6a42d47a93c5296", "91a32bb6ca65471f882b843df3f41bab" ] }, "id": "oPXzJZzHEgXe", "outputId": "380203d7-d490-4853-f1d7-8c391a72653b" }, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "Standardizing format: 0%| | 0/10000 [00:00\n", "### Train the model\n", "Now let's use Huggingface TRL's `SFTTrainer`! More docs here: [TRL SFT docs](https://huggingface.co/docs/trl/sft_trainer). We do 60 steps to speed things up, but you can set `num_train_epochs=1` for a full run, and turn off `max_steps=None`. We also support TRL's `DPOTrainer`!" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 49, "referenced_widgets": [ "6f1af8d1fc50425eb7aa46db06dd802c", "989ad08775ac4de5bc9be6d44ac45691", "65b3a9c9919245cd975f07c1771b5d35", "5de634c4e9504e5797cf6655a15fbce0", "0ace85934fa34e12b22c9dc1d347ce20", "afb907e8b5a04f5f8111f667c4fbc03c", "a512623a60074f1c8021412a51cf1cdf", "7042b11c6a404ee7b2c3797af9cc52cb", "3ef41b9d980f40e08189ea8c63a34ec6", "32fc8f860eb54e6b8c56c66aa975e074", "0a25c2df5f954b489210d711f275024a" ] }, "id": "95_Nn-89DhsL", "outputId": "a06bfc29-b977-41f7-b3ab-ca510d6607a9" }, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "Generating train split: 0 examples [00:00, ? examples/s]" ], "application/vnd.jupyter.widget-view+json": { "version_major": 2, "version_minor": 0, "model_id": "6f1af8d1fc50425eb7aa46db06dd802c" } }, "metadata": {} } ], "source": [ "from trl import SFTTrainer\n", "from transformers import TrainingArguments, DataCollatorForSeq2Seq\n", "from unsloth import is_bfloat16_supported\n", "\n", "# how many epochs will we know how to run?\n", "\n", "trainer = SFTTrainer(\n", " model = model,\n", " tokenizer = tokenizer,\n", " train_dataset = dataset,\n", " dataset_text_field = \"text\",\n", " max_seq_length = max_seq_length,\n", " data_collator = DataCollatorForSeq2Seq(tokenizer = tokenizer),\n", " dataset_num_proc = 2,\n", " packing = True, # Can make training 5x faster for short sequences.\n", " args = TrainingArguments(\n", " per_device_train_batch_size = 8, # old 2\n", " gradient_accumulation_steps = 2,\n", " warmup_steps = 10,\n", " num_train_epochs = 1, # Set this for 1 full training run.\n", " # max_steps = 60,\n", " learning_rate = 2e-4,\n", " fp16 = not is_bfloat16_supported(),\n", " bf16 = is_bfloat16_supported(),\n", " logging_steps = 1,\n", " optim = \"adamw_8bit\",\n", " weight_decay = 0.01,\n", " lr_scheduler_type = \"linear\",\n", " seed = 3407,\n", " output_dir = \"/content/drive/MyDrive/ID2223\", #\"outputs\",\n", " report_to = \"none\", # Use this for WandB etc\n", "\n", " # evaluation_strategy = \"steps\",\n", " logging_dir=\"./logs\",\n", " save_strategy = \"steps\", # Save the model checkpoint every logging step\n", " # eval_steps = 5,\n", " # do_eval = True,\n", " save_steps = 5, # Save every 5 checkpoints\n", " save_total_limit = 2,\n", " ),\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "C_sGp5XlG6dq" }, "source": [ "We also use Unsloth's `train_on_completions` method to only train on the assistant outputs and ignore the loss on the user's inputs." ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 49, "referenced_widgets": [ "c25438e9eaab41ad8ebf0f180ec22fed", "90a1ab67ed2240ad91cab336e9df6318", "97b3012cffc34bb3b124cdf8984f1677", "4399cf71925a4ce5b1184c55b59e934d", "c5cc96bb738f48e78b1e9f0eb5f50e52", "da308fef8a6e4e20a6269b3078078f67", "98635efad5674d62ab1c32eb95421a0d", "07eaf1c332394e83bf96fefe9bd61c2c", "3ac6b846dfc54d4b97dcabd9a5003518", "ea8d8596bedf48369dc259b9e2c7a119", "8ea51c024d854c4b97af1ea5049d9ce4" ] }, "id": "juQiExuBG5Bt", "outputId": "8b4307d9-11ac-4d9c-9d6c-73ec2c6c753c" }, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "Map: 0%| | 0/2764 [00:00user<|end_header_id|>\\n\\n\",\n", " response_part = \"<|start_header_id|>assistant<|end_header_id|>\\n\\n\",\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "Dv1NBUozV78l" }, "source": [ "We verify masking is actually done:" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 143 }, "id": "LtsMVtlkUhja", "outputId": "103cf1d9-cf34-47e2-fdf7-417e989fd02d" }, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "\". It also handles edge cases where division by zero occurs or the user selects an operation other than division with a second number of zero.<|eot_id|><|eot_id|><|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nCutting Knowledge Date: December 2023\\nToday Date: 26 July 2024\\n\\n<|eot_id|><|start_header_id|>user<|end_header_id|>\\n\\nHow would the mathematical principles evaluate the division operation for an element in a set S = {0}, if it is divided by 0? This inquiry calls for a comprehensive understanding of the principles of division and the ramifications of dividing by 0 on an element in a set.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\nThe mathematical principles do not evaluate the division operation for an element in a set S = {0}, if it is divided by 0. This is because dividing by 0 is undefined and leads to mathematical inconsistencies.\\n\\nTo understand this concept better, let's consider the basic principles of division. Division is the inverse operation of multiplication. When we divide a number by another number, we are essentially trying to find out how many times the second number can fit into the first number. For example, 10 ÷ 5 = 2, because 5 can fit into 10 two times.\\n\\nHowever, when we try to divide a number by 0, we encounter a problem. There is no number that can fit into 0, any number of times. This is because 0 represents the absence of quantity, and dividing by 0 leads to an undefined result.\\n\\nLet's take a closer look at the division operation for an element in a set S = {0}, if it is divided by 0:\\n\\nS/0 =?\\n\\nSince 0 cannot be divided into any number, the result of this division is undefined. It does not follow any mathematical principles or rules, and it leads to mathematical inconsistencies. For example, if we were to divide 1 by 0 and then multiply it by 0, we would expect the result to be 1, since we multiplied by the inverse of 0 (which is undefined). However, the result would be 0, which is a contradiction.\\n\\nTherefore, we can say that the mathematical principles do not allow the division operation for an element in a set S = {0}, if it is divided by 0, as it leads to undefined and inconsistent results.<|eot_id|><|eot_id|><|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nCutting Knowledge Date: December 2023\\nToday Date: 26 July 2024\\n\\nYou are an unbiased, uncensored, helpful assistant.<|eot_id|><|start_header_id|>user<|end_header_id|>\\n\\nGiven a dataset with a normal distribution, explain how you would use hypothesis testing to determine whether there is a significant difference between two sample means.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\nHypothesis testing is a statistical method used to make inferences about population parameters based on sample data. In this case, we want to determine whether there is a significant difference between two sample means from the same dataset with a normal distribution.\\n\\nHere are the steps for hypothesis testing to compare two sample means:\\n\\n1. State the null and alternative hypotheses:\\n - Null Hypothesis (H0): There is no significant difference between the two sample means (μ1 = μ2).\\n - Alternative Hypothesis (H1): There is a significant difference between the two sample means (μ1 ≠ μ2).\\n\\n2. Choose a significance level (α):\\n The significance level (commonly set at 0.05) represents the probability of rejecting the null hypothesis when it is true. It determines the threshold for deciding whether the observed difference between the sample means is due to chance or an actual difference in the population means.\\n\\n3. Calculate the test statistic:\\n For comparing two sample means, we can use the t-test statistic if the population standard deviations are unknown. The formula for the t-test statistic is:\\n\\n t = (M1 - M2) / √[(s1²/n1) + (s2²/n2)]\\n\\n where M1 and M2 are the sample means, s1² and s2² are the sample variances, and n1 and n2 are the sample sizes.\\n\\n4. Determine the degrees of freedom (df):\\n For the t-test, the degrees of freedom can be calculated using the following formula:\\n\\n df = n1 + n2 - 2\\n\\n5. Find the critical value(s) and the rejection region(s):\\n Using the chosen significance level (α) and the degrees of freedom (df), look up the critical value(s) in a t-distribution table. This will help you identify the rejection region(s) where, if the test statistic falls into it, you would reject the null hypothesis.\\n\\n6. Compare the test statistic to the critical value(s):\\n - If the test statistic falls within the rejection region(s), reject the null hypothesis (H0) and conclude that there is a significant difference between the two sample means.\\n - If the test statistic does not fall within the rejection region(s), fail to reject the null hypothesis (H0) and conclude that there is no significant difference between the two sample means.\\n\\n7. Calculate the p-value:\\n The p-value represents the probability of observing a test statistic as extreme or more extreme than the one calculated, assuming the null hypothesis is true. You can find the p-value using a t-distribution table or statistical software. If the p-value is less than the chosen significance level (α), reject the null hypothesis.\\n\\n8. Interpret the results:\\n Based on the comparison of the test statistic to the critical value(s) and/or the p-value, make a conclusion about whether there is a significant difference between the two sample means. Remember that failing to reject the null hypothesis does not prove it is true; it only suggests that there is insufficient evidence to support the alternative hypothesis.<|eot_id|><|eot_id|><|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nCutting Knowledge Date: December 2023\\nToday Date: 26 July 2024\\n\\n<|eot_id|><|start_header_id|>user<|end_header_id|>\\n\\nHow can we ensure that the products we create are designed for everyone, regardless of their differences? Explain the concept of inclusive design in a way that a child can understand, and provide examples of how it helps make products better for everyone. Additionally, describe the different tools and methods used in inclusive design that take into account various abilities and needs of users, such as those who may have visual, auditory, or mobility impairments. Give a step-by-step explanation to help children understand the process.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\nInclusive design means making things that are useful and easy to use for everyone, including people with differences. We want to make sure that no one is left out or feels excluded because they cannot use a product. \\r\\n\\r\\nFor example, when we design a playground, we want to make sure that children of all ages and abilities can play on it. We might add a ramp for someone who uses a wheelchair or make sure the playground equipment is not too tall for younger children. This way, everyone can have fun and play together!\\r\\n\\r\\nThere are many tools and methods used in inclusive design to make sure that everyone's needs are considered. Some examples are:\\r\\n\\r\\n1. Accessibility features: these are features that make it easier for people with disabilities to use a product. For example, closed captioning on a video helps people who are deaf or hard of hearing understand what is being said.\\r\\n\\r\\n2. User testing: this is when we ask people to try out a product and give us feedback. We want to make sure that the product is easy to use for everyone, so we ask people with different abilities to test it.\\r\\n\\r\\n3. Universal design: this is the idea of designing products that can be used by as many people as possible, regardless of their abilities or differences. An example of universal design is a door handle that can be easily opened by someone with a mobility impairment or by someone who is carrying a lot of bags.\\r\\n\\r\\nHere is an example of how inclusive design works:\\r\\n\\r\\nLet's say we are designing a website. We want to make sure that everyone can use it, so we use inclusive design. Here are the steps we take:\\r\\n\\r\\n1. First, we research different abilities and needs that people might have, such as visual or auditory impairments.\\r\\n\\r\\n2. Next, we create different designs for the website that take into account these different abilities and needs. For example, we might create a version of the website with larger text for people who have trouble reading smaller text.\\r\\n\\r\\n3. Then, we ask people to test the different versions of the website and give us feedback.\\r\\n\\r\\n4. Finally, we choose the best design that works well for everyone, regardless of their differences. This way, everyone can use the website easily and without any problems!<|eot_id|><|eot_id|><|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nCutting Knowledge Date: December 2023\\nToday Date: 26 July 2024\\n\\n<|eot_id|><|start_header_id|>user<|end_header_id|>\\n\\nSolve the following math problem step-by-step.\\nSimplify your answer as much as possible. Present your final answer as \\\\boxed{Your Answer}.\\na cylinder with 6 meter radius and 12 meter height is filled to capacity with water. if the content of the cylinder is used to fill several smaller cylinders of 4 meter diameter and 8 meter height, how many smaller cylinders will be filled to capacity?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\\n\\nStep 1: First, let's calculate the volume of the large cylinder. The formula for the volume of a cylinder is V = πr^2h, where r is the radius and h is the height.\\nStep 2: Using the given values, the volume of the large cylinder is V = π * 6^2 * 12 = 1,152π cubic meters.\\nStep 3: Next, let's calculate the volume of each smaller cylinder. Using the same formula, the volume of the smaller cylinder is V = π * 2^2 * 8 = 32π cubic meters.\\nStep 4: To find out how many smaller cylinders can be filled to capacity, divide the volume of the large cylinder by the volume of each smaller cylinder. Number of smaller cylinders = Volume of large cylinder / Volume of each smaller cylinder = 1\"" ], "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" } }, "metadata": {}, "execution_count": 20 } ], "source": [ "tokenizer.decode(trainer.train_dataset[5][\"input_ids\"])" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 143 }, "id": "_rD6fl8EUxnG", "outputId": "3e6ed868-3460-4cd3-b686-b688ed844617" }, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "\" \\n\\nThe mathematical principles do not evaluate the division operation for an element in a set S = {0}, if it is divided by 0. This is because dividing by 0 is undefined and leads to mathematical inconsistencies.\\n\\nTo understand this concept better, let's consider the basic principles of division. Division is the inverse operation of multiplication. When we divide a number by another number, we are essentially trying to find out how many times the second number can fit into the first number. For example, 10 ÷ 5 = 2, because 5 can fit into 10 two times.\\n\\nHowever, when we try to divide a number by 0, we encounter a problem. There is no number that can fit into 0, any number of times. This is because 0 represents the absence of quantity, and dividing by 0 leads to an undefined result.\\n\\nLet's take a closer look at the division operation for an element in a set S = {0}, if it is divided by 0:\\n\\nS/0 =?\\n\\nSince 0 cannot be divided into any number, the result of this division is undefined. It does not follow any mathematical principles or rules, and it leads to mathematical inconsistencies. For example, if we were to divide 1 by 0 and then multiply it by 0, we would expect the result to be 1, since we multiplied by the inverse of 0 (which is undefined). However, the result would be 0, which is a contradiction.\\n\\nTherefore, we can say that the mathematical principles do not allow the division operation for an element in a set S = {0}, if it is divided by 0, as it leads to undefined and inconsistent results.<|eot_id|><|eot_id|><|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nCutting Knowledge Date: December 2023\\nToday Date: 26 July 2024\\n\\nYou are an unbiased, uncensored, helpful assistant.<|eot_id|> \\n\\nHypothesis testing is a statistical method used to make inferences about population parameters based on sample data. In this case, we want to determine whether there is a significant difference between two sample means from the same dataset with a normal distribution.\\n\\nHere are the steps for hypothesis testing to compare two sample means:\\n\\n1. State the null and alternative hypotheses:\\n - Null Hypothesis (H0): There is no significant difference between the two sample means (μ1 = μ2).\\n - Alternative Hypothesis (H1): There is a significant difference between the two sample means (μ1 ≠ μ2).\\n\\n2. Choose a significance level (α):\\n The significance level (commonly set at 0.05) represents the probability of rejecting the null hypothesis when it is true. It determines the threshold for deciding whether the observed difference between the sample means is due to chance or an actual difference in the population means.\\n\\n3. Calculate the test statistic:\\n For comparing two sample means, we can use the t-test statistic if the population standard deviations are unknown. The formula for the t-test statistic is:\\n\\n t = (M1 - M2) / √[(s1²/n1) + (s2²/n2)]\\n\\n where M1 and M2 are the sample means, s1² and s2² are the sample variances, and n1 and n2 are the sample sizes.\\n\\n4. Determine the degrees of freedom (df):\\n For the t-test, the degrees of freedom can be calculated using the following formula:\\n\\n df = n1 + n2 - 2\\n\\n5. Find the critical value(s) and the rejection region(s):\\n Using the chosen significance level (α) and the degrees of freedom (df), look up the critical value(s) in a t-distribution table. This will help you identify the rejection region(s) where, if the test statistic falls into it, you would reject the null hypothesis.\\n\\n6. Compare the test statistic to the critical value(s):\\n - If the test statistic falls within the rejection region(s), reject the null hypothesis (H0) and conclude that there is a significant difference between the two sample means.\\n - If the test statistic does not fall within the rejection region(s), fail to reject the null hypothesis (H0) and conclude that there is no significant difference between the two sample means.\\n\\n7. Calculate the p-value:\\n The p-value represents the probability of observing a test statistic as extreme or more extreme than the one calculated, assuming the null hypothesis is true. You can find the p-value using a t-distribution table or statistical software. If the p-value is less than the chosen significance level (α), reject the null hypothesis.\\n\\n8. Interpret the results:\\n Based on the comparison of the test statistic to the critical value(s) and/or the p-value, make a conclusion about whether there is a significant difference between the two sample means. Remember that failing to reject the null hypothesis does not prove it is true; it only suggests that there is insufficient evidence to support the alternative hypothesis.<|eot_id|><|eot_id|><|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nCutting Knowledge Date: December 2023\\nToday Date: 26 July 2024\\n\\n<|eot_id|> \\n\\nInclusive design means making things that are useful and easy to use for everyone, including people with differences. We want to make sure that no one is left out or feels excluded because they cannot use a product. \\r\\n\\r\\nFor example, when we design a playground, we want to make sure that children of all ages and abilities can play on it. We might add a ramp for someone who uses a wheelchair or make sure the playground equipment is not too tall for younger children. This way, everyone can have fun and play together!\\r\\n\\r\\nThere are many tools and methods used in inclusive design to make sure that everyone's needs are considered. Some examples are:\\r\\n\\r\\n1. Accessibility features: these are features that make it easier for people with disabilities to use a product. For example, closed captioning on a video helps people who are deaf or hard of hearing understand what is being said.\\r\\n\\r\\n2. User testing: this is when we ask people to try out a product and give us feedback. We want to make sure that the product is easy to use for everyone, so we ask people with different abilities to test it.\\r\\n\\r\\n3. Universal design: this is the idea of designing products that can be used by as many people as possible, regardless of their abilities or differences. An example of universal design is a door handle that can be easily opened by someone with a mobility impairment or by someone who is carrying a lot of bags.\\r\\n\\r\\nHere is an example of how inclusive design works:\\r\\n\\r\\nLet's say we are designing a website. We want to make sure that everyone can use it, so we use inclusive design. Here are the steps we take:\\r\\n\\r\\n1. First, we research different abilities and needs that people might have, such as visual or auditory impairments.\\r\\n\\r\\n2. Next, we create different designs for the website that take into account these different abilities and needs. For example, we might create a version of the website with larger text for people who have trouble reading smaller text.\\r\\n\\r\\n3. Then, we ask people to test the different versions of the website and give us feedback.\\r\\n\\r\\n4. Finally, we choose the best design that works well for everyone, regardless of their differences. This way, everyone can use the website easily and without any problems!<|eot_id|><|eot_id|><|begin_of_text|><|start_header_id|>system<|end_header_id|>\\n\\nCutting Knowledge Date: December 2023\\nToday Date: 26 July 2024\\n\\n<|eot_id|> \\n\\nStep 1: First, let's calculate the volume of the large cylinder. The formula for the volume of a cylinder is V = πr^2h, where r is the radius and h is the height.\\nStep 2: Using the given values, the volume of the large cylinder is V = π * 6^2 * 12 = 1,152π cubic meters.\\nStep 3: Next, let's calculate the volume of each smaller cylinder. Using the same formula, the volume of the smaller cylinder is V = π * 2^2 * 8 = 32π cubic meters.\\nStep 4: To find out how many smaller cylinders can be filled to capacity, divide the volume of the large cylinder by the volume of each smaller cylinder. Number of smaller cylinders = Volume of large cylinder / Volume of each smaller cylinder = 1\"" ], "application/vnd.google.colaboratory.intrinsic+json": { "type": "string" } }, "metadata": {}, "execution_count": 21 } ], "source": [ "space = tokenizer(\" \", add_special_tokens = False).input_ids[0]\n", "tokenizer.decode([space if x == -100 else x for x in trainer.train_dataset[5][\"labels\"]])" ] }, { "cell_type": "markdown", "metadata": { "id": "3enWUM0jV-jV" }, "source": [ "We can see the System and Instruction prompts are successfully masked!" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "2ejIt2xSNKKp", "outputId": "a55aec07-c556-43f4-8d5a-17768e8a031e" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "GPU = Tesla T4. Max memory = 14.748 GB.\n", "3.619 GB of memory reserved.\n" ] } ], "source": [ "#@title Show current memory stats\n", "gpu_stats = torch.cuda.get_device_properties(0)\n", "start_gpu_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)\n", "max_memory = round(gpu_stats.total_memory / 1024 / 1024 / 1024, 3)\n", "print(f\"GPU = {gpu_stats.name}. Max memory = {max_memory} GB.\")\n", "print(f\"{start_gpu_memory} GB of memory reserved.\")" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 911 }, "id": "yqxqAZ7KJ4oL", "outputId": "b175edf4-1a95-476c-e694-48eaa0a36939" }, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "==((====))== Unsloth - 2x faster free finetuning | Num GPUs = 1\n", " \\\\ /| Num examples = 2,764 | Num Epochs = 1\n", "O^O/ \\_/ \\ Batch size per device = 8 | Gradient Accumulation steps = 4\n", "\\ / Total batch size = 32 | Total steps = 86\n", " \"-____-\" Number of trainable parameters = 11,272,192\n" ] }, { "output_type": "display_data", "data": { "text/plain": [ "" ], "text/html": [ "\n", "
\n", " \n", " \n", " [17/86 08:59 < 41:22, 0.03 it/s, Epoch 0.18/1]\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
StepTraining Loss
10.999500
21.019900
31.033200
40.972300
50.914400
60.987300
70.992800
80.977100
91.050300
100.949200
110.932900
120.968300
130.881700
141.035500
150.929700

" ] }, "metadata": {} }, { "output_type": "error", "ename": "KeyboardInterrupt", "evalue": "", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mKeyboardInterrupt\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mtrainer_stats\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtrainer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrain\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/unsloth/tokenizer_utils.py\u001b[0m in \u001b[0;36mtrain\u001b[0;34m(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)\u001b[0m\n", "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/unsloth/models/llama.py\u001b[0m in \u001b[0;36m_fast_inner_training_loop\u001b[0;34m(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)\u001b[0m\n", "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/unsloth/models/_utils.py\u001b[0m in \u001b[0;36m_unsloth_training_step\u001b[0;34m(***failed resolving arguments***)\u001b[0m\n", "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/accelerate/accelerator.py\u001b[0m in \u001b[0;36mbackward\u001b[0;34m(self, loss, **kwargs)\u001b[0m\n\u001b[1;32m 2235\u001b[0m \u001b[0;32mreturn\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2236\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mscaler\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2237\u001b[0;31m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mscaler\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mscale\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mloss\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbackward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 2238\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mlearning_rate\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhas_lomo_optimizer\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 2239\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlomo_backward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mloss\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlearning_rate\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/_tensor.py\u001b[0m in \u001b[0;36mbackward\u001b[0;34m(self, gradient, retain_graph, create_graph, inputs)\u001b[0m\n\u001b[1;32m 579\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 580\u001b[0m )\n\u001b[0;32m--> 581\u001b[0;31m torch.autograd.backward(\n\u001b[0m\u001b[1;32m 582\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgradient\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mretain_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcreate_graph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0minputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 583\u001b[0m )\n", "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/autograd/__init__.py\u001b[0m in \u001b[0;36mbackward\u001b[0;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)\u001b[0m\n\u001b[1;32m 345\u001b[0m \u001b[0;31m# some Python versions print out the first line of a multi-line function\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 346\u001b[0m \u001b[0;31m# calls in the traceback and some print out the last line\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 347\u001b[0;31m _engine_run_backward(\n\u001b[0m\u001b[1;32m 348\u001b[0m \u001b[0mtensors\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 349\u001b[0m \u001b[0mgrad_tensors_\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/autograd/graph.py\u001b[0m in \u001b[0;36m_engine_run_backward\u001b[0;34m(t_outputs, *args, **kwargs)\u001b[0m\n\u001b[1;32m 823\u001b[0m \u001b[0munregister_hooks\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_register_logging_hooks_on_whole_graph\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mt_outputs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 824\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 825\u001b[0;31m return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass\n\u001b[0m\u001b[1;32m 826\u001b[0m \u001b[0mt_outputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 827\u001b[0m ) # Calls into the C++ engine to run the backward pass\n", "\u001b[0;31mKeyboardInterrupt\u001b[0m: " ] } ], "source": [ "\n", "# trainer_stats = trainer.train()\n", "trainer_stats = trainer.train(resume_from_checkpoint = True) # loading from checkpoint" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "background_save": true }, "id": "pCqnaKmlO1U9" }, "outputs": [], "source": [ "#@title Show final memory and time stats\n", "used_memory = round(torch.cuda.max_memory_reserved() / 1024 / 1024 / 1024, 3)\n", "used_memory_for_lora = round(used_memory - start_gpu_memory, 3)\n", "used_percentage = round(used_memory /max_memory*100, 3)\n", "lora_percentage = round(used_memory_for_lora/max_memory*100, 3)\n", "print(f\"{trainer_stats.metrics['train_runtime']} seconds used for training.\")\n", "print(f\"{round(trainer_stats.metrics['train_runtime']/60, 2)} minutes used for training.\")\n", "print(f\"Peak reserved memory = {used_memory} GB.\")\n", "print(f\"Peak reserved memory for training = {used_memory_for_lora} GB.\")\n", "print(f\"Peak reserved memory % of max memory = {used_percentage} %.\")\n", "print(f\"Peak reserved memory for training % of max memory = {lora_percentage} %.\")" ] }, { "cell_type": "markdown", "metadata": { "id": "ekOmTR1hSNcr" }, "source": [ "\n", "### Inference\n", "Let's run the model! You can change the instruction and input - leave the output blank!\n", "\n", "**[NEW] Try 2x faster inference in a free Colab for Llama-3.1 8b Instruct [here](https://colab.research.google.com/drive/1T-YBVfnphoVc8E2E854qF3jdia2Ll2W2?usp=sharing)**\n", "\n", "We use `min_p = 0.1` and `temperature = 1.5`. Read this [Tweet](https://x.com/menhguin/status/1826132708508213629) for more information on why." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "background_save": true }, "id": "kR3gIAX-SM2q" }, "outputs": [], "source": [ "from unsloth.chat_templates import get_chat_template\n", "\n", "tokenizer = get_chat_template(\n", " tokenizer,\n", " chat_template = \"llama-3.1\",\n", ")\n", "FastLanguageModel.for_inference(model) # Enable native 2x faster inference\n", "\n", "messages = [\n", " {\"role\": \"user\", \"content\": \"Continue the fibonnaci sequence: 1, 1, 2, 3, 5, 8,\"},\n", "]\n", "inputs = tokenizer.apply_chat_template(\n", " messages,\n", " tokenize = True,\n", " add_generation_prompt = True, # Must add for generation\n", " return_tensors = \"pt\",\n", ").to(\"cuda\")\n", "\n", "outputs = model.generate(input_ids = inputs, max_new_tokens = 64, use_cache = True,\n", " temperature = 1.5, min_p = 0.1)\n", "tokenizer.batch_decode(outputs)" ] }, { "cell_type": "markdown", "metadata": { "id": "CrSvZObor0lY" }, "source": [ " You can also use a `TextStreamer` for continuous inference - so you can see the generation token by token, instead of waiting the whole time!" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "background_save": true }, "id": "e2pEuRb1r2Vg" }, "outputs": [], "source": [ "FastLanguageModel.for_inference(model) # Enable native 2x faster inference\n", "\n", "messages = [\n", " {\"role\": \"user\", \"content\": \"Continue the fibonnaci sequence: 1, 1, 2, 3, 5, 8,\"},\n", "]\n", "inputs = tokenizer.apply_chat_template(\n", " messages,\n", " tokenize = True,\n", " add_generation_prompt = True, # Must add for generation\n", " return_tensors = \"pt\",\n", ").to(\"cuda\")\n", "\n", "from transformers import TextStreamer\n", "text_streamer = TextStreamer(tokenizer, skip_prompt = True)\n", "_ = model.generate(input_ids = inputs, streamer = text_streamer, max_new_tokens = 128,\n", " use_cache = True, temperature = 1.5, min_p = 0.1)" ] }, { "cell_type": "markdown", "metadata": { "id": "uMuVrWbjAzhc" }, "source": [ "\n", "### Saving, loading finetuned models\n", "To save the final model as LoRA adapters, either use Huggingface's `push_to_hub` for an online save or `save_pretrained` for a local save.\n", "\n", "**[NOTE]** This ONLY saves the LoRA adapters, and not the full model. To save to 16bit or GGUF, scroll down!" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "background_save": true }, "id": "_21ZXVjyS0my" }, "outputs": [], "source": [ "from google.colab import userdata\n", "hugginfacetoken = userdata.get('hugginfacetoken')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "background_save": true }, "id": "upcOlWe7A1vc" }, "outputs": [], "source": [ "#model.save_pretrained(\"lora_model\") # Local saving\n", "#tokenizer.save_pretrained(\"lora_model\")\n", "#model.push_to_hub(\"lennart-rth/iris-inside\", token = hugginfacetoken) # Online saving\n", "#tokenizer.push_to_hub(\"lennart-rth/iris-inside\", token = hugginfacetoken) # Online saving" ] }, { "cell_type": "markdown", "metadata": { "id": "AEEcJ4qfC7Lp" }, "source": [ "Now if you want to load the LoRA adapters we just saved for inference, set `False` to `True`:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "background_save": true }, "id": "MKX_XKs_BNZR" }, "outputs": [], "source": [ "if False:\n", " from unsloth import FastLanguageModel\n", " model, tokenizer = FastLanguageModel.from_pretrained(\n", " model_name = \"lennart-rth/iris-inside\", # YOUR MODEL YOU USED FOR TRAINING\n", " max_seq_length = max_seq_length,\n", " dtype = dtype,\n", " load_in_4bit = load_in_4bit,\n", " token = hugginfacetoken\n", " )\n", " FastLanguageModel.for_inference(model) # Enable native 2x faster inference\n", "\n", "messages = [\n", " {\"role\": \"user\", \"content\": \"Describe a tall tower in the capital of France.\"},\n", "]\n", "inputs = tokenizer.apply_chat_template(\n", " messages,\n", " tokenize = True,\n", " add_generation_prompt = True, # Must add for generation\n", " return_tensors = \"pt\",\n", ").to(\"cpu\")\n", "\n", "from transformers import TextStreamer\n", "text_streamer = TextStreamer(tokenizer, skip_prompt = True)\n", "_ = model.generate(input_ids = inputs, streamer = text_streamer, max_new_tokens = 128,\n", " use_cache = True, temperature = 1.5, min_p = 0.1)" ] }, { "cell_type": "markdown", "metadata": { "id": "QQMjaNrjsU5_" }, "source": [ "You can also use Hugging Face's `AutoModelForPeftCausalLM`. Only use this if you do not have `unsloth` installed. It can be hopelessly slow, since `4bit` model downloading is not supported, and Unsloth's **inference is 2x faster**." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "background_save": true }, "id": "yFfaXG0WsQuE" }, "outputs": [], "source": [ "if False:\n", " # I highly do NOT suggest - use Unsloth if possible\n", " from peft import AutoPeftModelForCausalLM\n", " from transformers import AutoTokenizer\n", " model = AutoPeftModelForCausalLM.from_pretrained(\n", " \"lora_model\", # YOUR MODEL YOU USED FOR TRAINING\n", " load_in_4bit = load_in_4bit,\n", " )\n", " tokenizer = AutoTokenizer.from_pretrained(\"lora_model\")" ] }, { "cell_type": "markdown", "metadata": { "id": "f422JgM9sdVT" }, "source": [ "### Saving to float16 for VLLM\n", "\n", "We also support saving to `float16` directly. Select `merged_16bit` for float16 or `merged_4bit` for int4. We also allow `lora` adapters as a fallback. Use `push_to_hub_merged` to upload to your Hugging Face account! You can go to https://huggingface.co/settings/tokens for your personal tokens." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "background_save": true }, "id": "d7_zajNQlHUL" }, "outputs": [], "source": [ "# This is IMPORTATN! Coulnt figure out how to upload the tokenizer directly to hf. So save it in a folder and manually copy it in there:(\n", "#tokenizer.save_pretrained(\"tokenizer\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "background_save": true }, "id": "iHjt_SMYsd3P" }, "outputs": [], "source": [ "# Merge to 16bit\n", "if False: model.save_pretrained_merged(\"model\", tokenizer, save_method = \"merged_16bit\",)\n", "if False: model.push_to_hub_merged(\"lennart-rth/iris-inside\", tokenizer, save_method = \"merged_16bit\", token = hugginfacetoken)\n", "\n", "# Merge to 4bit\n", "if False: model.save_pretrained_merged(\"model\", tokenizer, save_method = \"merged_4bit\",)\n", "if False: model.push_to_hub_merged(\"hf/model\", tokenizer, save_method = \"merged_4bit\", token = \"\")\n", "\n", "# Just LoRA adapters\n", "if False: model.save_pretrained_merged(\"model\", tokenizer, save_method = \"lora\",)\n", "if False: model.push_to_hub_merged(\"hf/model\", tokenizer, save_method = \"lora\", token = \"\")" ] }, { "cell_type": "markdown", "metadata": { "id": "TCv4vXHd61i7" }, "source": [ "### GGUF / llama.cpp Conversion\n", "To save to `GGUF` / `llama.cpp`, we support it natively now! We clone `llama.cpp` and we default save it to `q8_0`. We allow all methods like `q4_k_m`. Use `save_pretrained_gguf` for local saving and `push_to_hub_gguf` for uploading to HF.\n", "\n", "Some supported quant methods (full list on our [Wiki page](https://github.com/unslothai/unsloth/wiki#gguf-quantization-options)):\n", "* `q8_0` - Fast conversion. High resource use, but generally acceptable.\n", "* `q4_k_m` - Recommended. Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q4_K.\n", "* `q5_k_m` - Recommended. Uses Q6_K for half of the attention.wv and feed_forward.w2 tensors, else Q5_K.\n", "\n", "[**NEW**] To finetune and auto export to Ollama, try our [Ollama notebook](https://colab.research.google.com/drive/1WZDi7APtQ9VsvOrQSSC5DDtxq159j8iZ?usp=sharing)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "background_save": true }, "collapsed": true, "id": "FqfebeAdT073" }, "outputs": [], "source": [ "# Save to 8bit Q8_0\n", "if False: model.save_pretrained_gguf(\"guffmodel\", tokenizer,token = hugginfacetoken)\n", "# Remember to go to https://huggingface.co/settings/tokens for a token!\n", "# And change hf to your username!\n", "if False: model.push_to_hub_gguf(\"lennart-rth/iris-inside\", tokenizer, token = hugginfacetoken)\n", "\n", "# Save to 16bit GGUF\n", "if False: model.save_pretrained_gguf(\"model\", tokenizer, quantization_method = \"f16\")\n", "if False: model.push_to_hub_gguf(\"lennart-rth/iris-inside\", tokenizer, quantization_method = \"f16\", token = hugginfacetoken)\n", "\n", "# Save to q4_k_m GGUF\n", "if False: model.save_pretrained_gguf(\"model\", tokenizer, quantization_method = \"q4_k_m\")\n", "if False: model.push_to_hub_gguf(\"lennart-rth/iris-inside\", tokenizer, quantization_method = \"q4_k_m\", token = hugginfacetoken)\n", "\n", "# Save to multiple GGUF options - much faster if you want multiple!\n", "if True:\n", " model.push_to_hub_gguf(\n", " \"lennart-rth/iris-inside\", # Change hf to your username!\n", " tokenizer,\n", " quantization_method = [\"q4_k_m\", \"q8_0\", \"q5_k_m\",],\n", " token = hugginfacetoken, # Get a token at https://huggingface.co/settings/tokens\n", " )" ] }, { "cell_type": "markdown", "metadata": { "id": "bDp0zNpwe6U_" }, "source": [ "Now, use the `model-unsloth.gguf` file or `model-unsloth-Q4_K_M.gguf` file in `llama.cpp` or a UI based system like `GPT4All`. You can install GPT4All by going [here](https://gpt4all.io/index.html).\n", "\n", "**[NEW] Try 2x faster inference in a free Colab for Llama-3.1 8b Instruct [here](https://colab.research.google.com/drive/1T-YBVfnphoVc8E2E854qF3jdia2Ll2W2?usp=sharing)**" ] }, { "cell_type": "markdown", "metadata": { "id": "Zt9CHJqO6p30" }, "source": [ "And we're done! If you have any questions on Unsloth, we have a [Discord](https://discord.gg/u54VK8m8tk) channel! If you find any bugs or want to keep updated with the latest LLM stuff, or need help, join projects etc, feel free to join our Discord!\n", "\n", "Some other links:\n", "1. Zephyr DPO 2x faster [free Colab](https://colab.research.google.com/drive/15vttTpzzVXv_tJwEk-hIcQ0S9FcEWvwP?usp=sharing)\n", "2. Llama 7b 2x faster [free Colab](https://colab.research.google.com/drive/1lBzz5KeZJKXjvivbYvmGarix9Ao6Wxe5?usp=sharing)\n", "3. TinyLlama 4x faster full Alpaca 52K in 1 hour [free Colab](https://colab.research.google.com/drive/1AZghoNBQaMDgWJpi4RbffGM1h6raLUj9?usp=sharing)\n", "4. CodeLlama 34b 2x faster [A100 on Colab](https://colab.research.google.com/drive/1y7A0AxE3y8gdj4AVkl2aZX47Xu3P1wJT?usp=sharing)\n", "5. Mistral 7b [free Kaggle version](https://www.kaggle.com/code/danielhanchen/kaggle-mistral-7b-unsloth-notebook)\n", "6. We also did a [blog](https://huggingface.co/blog/unsloth-trl) with 🤗 HuggingFace, and we're in the TRL [docs](https://huggingface.co/docs/trl/main/en/sft_trainer#accelerate-fine-tuning-2x-using-unsloth)!\n", "7. `ChatML` for ShareGPT datasets, [conversational notebook](https://colab.research.google.com/drive/1Aau3lgPzeZKQ-98h69CCu1UJcvIBLmy2?usp=sharing)\n", "8. Text completions like novel writing [notebook](https://colab.research.google.com/drive/1ef-tab5bhkvWmBOObepl1WgJvfvSzn5Q?usp=sharing)\n", "9. [**NEW**] We make Phi-3 Medium / Mini **2x faster**! See our [Phi-3 Medium notebook](https://colab.research.google.com/drive/1hhdhBa1j_hsymiW9m-WzxQtgqTH_NHqi?usp=sharing)\n", "10. [**NEW**] We make Gemma-2 9b / 27b **2x faster**! See our [Gemma-2 9b notebook](https://colab.research.google.com/drive/1vIrqH5uYDQwsJ4-OO3DErvuv4pBgVwk4?usp=sharing)\n", "11. [**NEW**] To finetune and auto export to Ollama, try our [Ollama notebook](https://colab.research.google.com/drive/1WZDi7APtQ9VsvOrQSSC5DDtxq159j8iZ?usp=sharing)\n", "12. [**NEW**] We make Mistral NeMo 12B 2x faster and fit in under 12GB of VRAM! [Mistral NeMo notebook](https://colab.research.google.com/drive/17d3U-CAIwzmbDRqbZ9NnpHxCkmXB6LZ0?usp=sharing)\n", "\n", "

\n", " \n", " \n", " Support our work if you can! Thanks!\n", "
" ] } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "T4", "provenance": [] }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "name": "python" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "a29682d0394e46eeabddad80a00349f7": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_abbe988281f643b29bcbc74a5d8c5fe4", "IPY_MODEL_5f1d65686d1a4602a6ee4c5d13df7818", "IPY_MODEL_59b4df6e930f457cb67a755d90a651a7" ], "layout": "IPY_MODEL_9da93d5b27d947bea044bc1119181d0d" } }, "abbe988281f643b29bcbc74a5d8c5fe4": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_ba27a4ca72164a06b85c8e58493d75e1", "placeholder": "​", "style": "IPY_MODEL_84f2e2eff5dd44b89b4390e544ea31a9", "value": "model.safetensors: 100%" } }, "5f1d65686d1a4602a6ee4c5d13df7818": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "danger", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_3ac8a0c853a6406ea3fb91d9e7ce743b", "max": 1027676737, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_c569a6ed0158456ebc2f3c0ed206d73a", "value": 1027676639 } }, "59b4df6e930f457cb67a755d90a651a7": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a0c39ec65e674b628219a1dfaa177096", "placeholder": "​", "style": "IPY_MODEL_a5c288c762214c8d93d69d3351f5832f", "value": " 1.03G/1.03G [00:06<00:00, 275MB/s]" } }, "9da93d5b27d947bea044bc1119181d0d": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ba27a4ca72164a06b85c8e58493d75e1": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "84f2e2eff5dd44b89b4390e544ea31a9": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "3ac8a0c853a6406ea3fb91d9e7ce743b": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "c569a6ed0158456ebc2f3c0ed206d73a": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "a0c39ec65e674b628219a1dfaa177096": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a5c288c762214c8d93d69d3351f5832f": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "81afdcf1237944c2a2b7402662d8a643": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_5581073a34c2423db732d3266161ed07", "IPY_MODEL_7b477d93171a4628a536a5d51ede5e7d", "IPY_MODEL_aa18c1a527804c3485881555b4cccaae" ], "layout": "IPY_MODEL_4aa3b4a195f64907aee323b2129d6ada" } }, "5581073a34c2423db732d3266161ed07": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_cbd6084641454e9c8f4fce020e2476fa", "placeholder": "​", "style": "IPY_MODEL_9ba68f77a0e849949ca1745d68978e1c", "value": "generation_config.json: 100%" } }, "7b477d93171a4628a536a5d51ede5e7d": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_412923ecea3644a381847ce54df4a56a", "max": 184, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_ad804015a2154bcca7dc2fcc53caff70", "value": 184 } }, "aa18c1a527804c3485881555b4cccaae": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_671467ee73ed43d6865f0af2c8f908e0", "placeholder": "​", "style": "IPY_MODEL_d88b2a3302264b6289a401aafe72c12d", "value": " 184/184 [00:00<00:00, 12.6kB/s]" } }, "4aa3b4a195f64907aee323b2129d6ada": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "cbd6084641454e9c8f4fce020e2476fa": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9ba68f77a0e849949ca1745d68978e1c": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "412923ecea3644a381847ce54df4a56a": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ad804015a2154bcca7dc2fcc53caff70": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "671467ee73ed43d6865f0af2c8f908e0": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "d88b2a3302264b6289a401aafe72c12d": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "20f7a9b4bc3542578f9c59adc64cbc4b": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_1d9cd463d7ee42d4848661bb1cfefe8a", "IPY_MODEL_641165dcd1ae4d78acdbd79a6d1e8bb7", "IPY_MODEL_b1d640cebe5841b28ab07234c2989414" ], "layout": "IPY_MODEL_2a87255dcbfa45dcbd56a28cc05288bd" } }, "1d9cd463d7ee42d4848661bb1cfefe8a": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_ccea16c679d44ac99847a0032d71b1d5", "placeholder": "​", "style": "IPY_MODEL_4783c67f36244488932c4a2ba6fe1bd7", "value": "tokenizer_config.json: 100%" } }, "641165dcd1ae4d78acdbd79a6d1e8bb7": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_b6bb256941f24137b4c696b5c486ef95", "max": 54598, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_09a8c63c84bf4fbd87a3aebf78bc65f3", "value": 54598 } }, "b1d640cebe5841b28ab07234c2989414": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a72a8ba21b674d30b3cd7c019d29f77d", "placeholder": "​", "style": "IPY_MODEL_bd41e0af677847d0bf3c3628aa3bb013", "value": " 54.6k/54.6k [00:00<00:00, 3.23MB/s]" } }, "2a87255dcbfa45dcbd56a28cc05288bd": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ccea16c679d44ac99847a0032d71b1d5": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "4783c67f36244488932c4a2ba6fe1bd7": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "b6bb256941f24137b4c696b5c486ef95": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "09a8c63c84bf4fbd87a3aebf78bc65f3": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "a72a8ba21b674d30b3cd7c019d29f77d": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "bd41e0af677847d0bf3c3628aa3bb013": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "913d96e92dea4721af1f6c37dfb75b95": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_bc5bec8baef3453a8402af85fa23ae61", "IPY_MODEL_29c108f189994f649b8b84a9c3e68901", "IPY_MODEL_6f65e0467ad445fcaea61764b19d27b4" ], "layout": "IPY_MODEL_2a8817c33b0048d3904c9e818ff193ec" } }, "bc5bec8baef3453a8402af85fa23ae61": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_f18cdcec6edb4d57920446ffdea0f0a2", "placeholder": "​", "style": "IPY_MODEL_793edc6cacf34d8d9b798205d6c512bd", "value": "tokenizer.json: 100%" } }, "29c108f189994f649b8b84a9c3e68901": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_8d310673bfca48eeaa0db3aeb8f9cc6d", "max": 9085657, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_cce78e2bc5f54ffd954019654d5a5f05", "value": 9085657 } }, "6f65e0467ad445fcaea61764b19d27b4": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_6f95a56ad2ca4fcdaf11da285d0d2a62", "placeholder": "​", "style": "IPY_MODEL_c966ac79c2e04c8f8c9611889f4e2fb6", "value": " 9.09M/9.09M [00:00<00:00, 27.8MB/s]" } }, "2a8817c33b0048d3904c9e818ff193ec": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f18cdcec6edb4d57920446ffdea0f0a2": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "793edc6cacf34d8d9b798205d6c512bd": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "8d310673bfca48eeaa0db3aeb8f9cc6d": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "cce78e2bc5f54ffd954019654d5a5f05": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "6f95a56ad2ca4fcdaf11da285d0d2a62": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "c966ac79c2e04c8f8c9611889f4e2fb6": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "17ce7feec43f4e2d9bc863899ee4f94e": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_8d28e864ccdf4a4896dfd5c008d0919c", "IPY_MODEL_ad73d99d088642589e399ebc5c4e44c3", "IPY_MODEL_a653bb2556b24770a8215750500b75a4" ], "layout": "IPY_MODEL_fbb10c5128374157a96d09fec3511387" } }, "8d28e864ccdf4a4896dfd5c008d0919c": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_66d611d034644087bd42856e3eca9a88", "placeholder": "​", "style": "IPY_MODEL_f323a3e2c9ea4a95bd899442db6777ca", "value": "special_tokens_map.json: 100%" } }, "ad73d99d088642589e399ebc5c4e44c3": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e0b5f80fb558464e8bfb2f5349bf7cff", "max": 454, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_83f9b80e5e0a46bfaa55a1d185b81bc7", "value": 454 } }, "a653bb2556b24770a8215750500b75a4": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_dd627e4aabe6470e95161caed0a50100", "placeholder": "​", "style": "IPY_MODEL_984a5053afd5447fb3b670687b151bb4", "value": " 454/454 [00:00<00:00, 33.1kB/s]" } }, "fbb10c5128374157a96d09fec3511387": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "66d611d034644087bd42856e3eca9a88": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f323a3e2c9ea4a95bd899442db6777ca": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "e0b5f80fb558464e8bfb2f5349bf7cff": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "83f9b80e5e0a46bfaa55a1d185b81bc7": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "dd627e4aabe6470e95161caed0a50100": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "984a5053afd5447fb3b670687b151bb4": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "039f8b9f542144a08606ec7dd0c94ce8": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_a4eebe5d52fc4e78b9d2e6e603a46766", "IPY_MODEL_5086fde5f6ee4c7cb6e2f0062612cb11", "IPY_MODEL_1a407d68e32c47f6acf3c1e034e46ee7" ], "layout": "IPY_MODEL_13891348033343deac25241d312baa78" } }, "a4eebe5d52fc4e78b9d2e6e603a46766": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_3ec772c8e1c9484b93249b2942e7a0c0", "placeholder": "​", "style": "IPY_MODEL_7c1ab1e5f0624e34b9bfc97e32838370", "value": "Standardizing format: 100%" } }, "5086fde5f6ee4c7cb6e2f0062612cb11": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_58396311734f41b3b1acfa61c420680e", "max": 10000, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_2427429328f84f5b92d6bf95c515a635", "value": 10000 } }, "1a407d68e32c47f6acf3c1e034e46ee7": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_13526476ee3c41c983158a502e824a75", "placeholder": "​", "style": "IPY_MODEL_cac0e01bb5144a199c55f9e6801c62ce", "value": " 10000/10000 [00:00<00:00, 30517.28 examples/s]" } }, "13891348033343deac25241d312baa78": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3ec772c8e1c9484b93249b2942e7a0c0": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7c1ab1e5f0624e34b9bfc97e32838370": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "58396311734f41b3b1acfa61c420680e": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "2427429328f84f5b92d6bf95c515a635": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "13526476ee3c41c983158a502e824a75": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "cac0e01bb5144a199c55f9e6801c62ce": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "c6f91eed614c4ea88dfc597115023fa8": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_8318e9e5c785478889781734e3b09452", "IPY_MODEL_a8a9e7d1ce7a4989896ce9fedd3b8411", "IPY_MODEL_34678221a3414b6fb95473188735e7fb" ], "layout": "IPY_MODEL_6587b7787ab5482697cb5ddcc26ff3de" } }, "8318e9e5c785478889781734e3b09452": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_c485116dfe48427ca9d5381a01d44741", "placeholder": "​", "style": "IPY_MODEL_f8a8c69a9a814163864fa3fd1c18035c", "value": "Map: 100%" } }, "a8a9e7d1ce7a4989896ce9fedd3b8411": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_850da0477eb0404084e6bcedcf461c50", "max": 10000, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_0110c1baa42d4b40834f5962ff22030e", "value": 10000 } }, "34678221a3414b6fb95473188735e7fb": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_50be91e8663d4ee3b6a42d47a93c5296", "placeholder": "​", "style": "IPY_MODEL_91a32bb6ca65471f882b843df3f41bab", "value": " 10000/10000 [00:00<00:00, 13668.74 examples/s]" } }, "6587b7787ab5482697cb5ddcc26ff3de": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "c485116dfe48427ca9d5381a01d44741": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f8a8c69a9a814163864fa3fd1c18035c": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "850da0477eb0404084e6bcedcf461c50": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0110c1baa42d4b40834f5962ff22030e": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "50be91e8663d4ee3b6a42d47a93c5296": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "91a32bb6ca65471f882b843df3f41bab": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "6f1af8d1fc50425eb7aa46db06dd802c": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_989ad08775ac4de5bc9be6d44ac45691", "IPY_MODEL_65b3a9c9919245cd975f07c1771b5d35", "IPY_MODEL_5de634c4e9504e5797cf6655a15fbce0" ], "layout": "IPY_MODEL_0ace85934fa34e12b22c9dc1d347ce20" } }, "989ad08775ac4de5bc9be6d44ac45691": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_afb907e8b5a04f5f8111f667c4fbc03c", "placeholder": "​", "style": "IPY_MODEL_a512623a60074f1c8021412a51cf1cdf", "value": "Generating train split: " } }, "65b3a9c9919245cd975f07c1771b5d35": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_7042b11c6a404ee7b2c3797af9cc52cb", "max": 1, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_3ef41b9d980f40e08189ea8c63a34ec6", "value": 1 } }, "5de634c4e9504e5797cf6655a15fbce0": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_32fc8f860eb54e6b8c56c66aa975e074", "placeholder": "​", "style": "IPY_MODEL_0a25c2df5f954b489210d711f275024a", "value": " 2764/0 [00:17<00:00, 253.16 examples/s]" } }, "0ace85934fa34e12b22c9dc1d347ce20": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "afb907e8b5a04f5f8111f667c4fbc03c": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a512623a60074f1c8021412a51cf1cdf": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "7042b11c6a404ee7b2c3797af9cc52cb": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "20px" } }, "3ef41b9d980f40e08189ea8c63a34ec6": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "32fc8f860eb54e6b8c56c66aa975e074": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0a25c2df5f954b489210d711f275024a": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "c25438e9eaab41ad8ebf0f180ec22fed": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_90a1ab67ed2240ad91cab336e9df6318", "IPY_MODEL_97b3012cffc34bb3b124cdf8984f1677", "IPY_MODEL_4399cf71925a4ce5b1184c55b59e934d" ], "layout": "IPY_MODEL_c5cc96bb738f48e78b1e9f0eb5f50e52" } }, "90a1ab67ed2240ad91cab336e9df6318": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_da308fef8a6e4e20a6269b3078078f67", "placeholder": "​", "style": "IPY_MODEL_98635efad5674d62ab1c32eb95421a0d", "value": "Map: 100%" } }, "97b3012cffc34bb3b124cdf8984f1677": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_07eaf1c332394e83bf96fefe9bd61c2c", "max": 2764, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_3ac6b846dfc54d4b97dcabd9a5003518", "value": 2764 } }, "4399cf71925a4ce5b1184c55b59e934d": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_ea8d8596bedf48369dc259b9e2c7a119", "placeholder": "​", "style": "IPY_MODEL_8ea51c024d854c4b97af1ea5049d9ce4", "value": " 2764/2764 [00:04<00:00, 628.00 examples/s]" } }, "c5cc96bb738f48e78b1e9f0eb5f50e52": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "da308fef8a6e4e20a6269b3078078f67": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "98635efad5674d62ab1c32eb95421a0d": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "07eaf1c332394e83bf96fefe9bd61c2c": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "3ac6b846dfc54d4b97dcabd9a5003518": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "ea8d8596bedf48369dc259b9e2c7a119": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "8ea51c024d854c4b97af1ea5049d9ce4": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } } } } }, "nbformat": 4, "nbformat_minor": 0 }