slurm submission log: 2024-11-18 08:01:28.643962
created following sbatch script: 

###############################

#!/bin/bash

#SBATCH --account=nlp
#SBATCH --cpus-per-task=2
#SBATCH --gres=gpu:4
#SBATCH --job-name=ram1998-job-4376567
#SBATCH --mem=16G
#SBATCH --nodelist=jagupard33
#SBATCH --open-mode=append
#SBATCH --output=ram1998-job-4376567.out
#SBATCH --partition=jag-standard
#SBATCH --time=14-0

# activate your desired anaconda environment
. /nlp/scr/ram1998/miniconda3/etc/profile.d/conda.sh ; conda activate pyreft_dev

# cd to working directory
cd .

# launch commands
srun --unbuffered run_as_child_processes 'torchrun --nproc_per_node 4 train_multigpu.py --model_name_or_path yahma/llama-7b-hf --data_path ./alpaca_data.json --output_dir ./test_multi_gpu_v2/ --layers "8;19" --rank 4 --position "f1+l1" --num_train_epochs 10 --per_device_train_batch_size 4 --per_device_eval_batch_size 4 --gradient_accumulation_steps 8 --evaluation_strategy "no" --save_strategy "no" --learning_rate 2e-5 --weight_decay 0. --warmup_ratio 0.03 --lr_scheduler_type "cosine" --logging_steps 1     --max_n_train_example 10000'

###############################

submission to slurm complete!


###############################
slurm submission output

Submitted batch job 9113346



###############################

###############################
start time: 2024-11-18 08:01:29.771619
machine: jagupard33.stanford.edu
conda env: pyreft_dev
###############################
running following processes

	torchrun --nproc_per_node 4 train_multigpu.py --model_name_or_path yahma/llama-7b-hf --data_path ./alpaca_data.json --output_dir ./test_multi_gpu_v2/ --layers "8;19" --rank 4 --position "f1+l1" --num_train_epochs 10 --per_device_train_batch_size 4 --per_device_eval_batch_size 4 --gradient_accumulation_steps 8 --evaluation_strategy "no" --save_strategy "no" --learning_rate 2e-5 --weight_decay 0. --warmup_ratio 0.03 --lr_scheduler_type "cosine" --logging_steps 1     --max_n_train_example 10000


###############################
command outputs: 


W1118 08:01:31.539000 139959025029504 torch/distributed/run.py:779] 
W1118 08:01:31.539000 139959025029504 torch/distributed/run.py:779] *****************************************
W1118 08:01:31.539000 139959025029504 torch/distributed/run.py:779] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
W1118 08:01:31.539000 139959025029504 torch/distributed/run.py:779] *****************************************
nnsight is not detected. Please install via 'pip install nnsight' for nnsight backend.
nnsight is not detected. Please install via 'pip install nnsight' for nnsight backend.
nnsight is not detected. Please install via 'pip install nnsight' for nnsight backend.
nnsight is not detected. Please install via 'pip install nnsight' for nnsight backend.
Starting on rank 0
/nlp/scr/ram1998/miniconda3/envs/pyreft_dev/lib/python3.12/site-packages/transformers/training_args.py:1545: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
  warnings.warn(
Starting on rank 3
Starting on rank 2
/nlp/scr/ram1998/miniconda3/envs/pyreft_dev/lib/python3.12/site-packages/transformers/training_args.py:1545: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
  warnings.warn(
/nlp/scr/ram1998/miniconda3/envs/pyreft_dev/lib/python3.12/site-packages/transformers/training_args.py:1545: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
  warnings.warn(
Starting on rank 1
/nlp/scr/ram1998/miniconda3/envs/pyreft_dev/lib/python3.12/site-packages/transformers/training_args.py:1545: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead
  warnings.warn(
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565 - if you loaded a llama tokenizer from a GGUF file you can ignore this message
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565 - if you loaded a llama tokenizer from a GGUF file you can ignore this message
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565 - if you loaded a llama tokenizer from a GGUF file you can ignore this message
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565 - if you loaded a llama tokenizer from a GGUF file you can ignore this message

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:45<00:45, 45.78s/it]
Loading checkpoint shards:  50%|█████     | 1/2 [00:46<00:46, 46.74s/it]
Loading checkpoint shards:  50%|█████     | 1/2 [00:46<00:46, 46.85s/it]
Loading checkpoint shards:  50%|█████     | 1/2 [00:47<00:47, 47.07s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:56<00:00, 24.95s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:56<00:00, 28.07s/it]

Loading checkpoint shards: 100%|██████████| 2/2 [00:56<00:00, 25.16s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:56<00:00, 28.40s/it]

Loading checkpoint shards: 100%|██████████| 2/2 [00:56<00:00, 25.18s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:56<00:00, 28.43s/it]

Loading checkpoint shards: 100%|██████████| 2/2 [00:57<00:00, 25.37s/it]
Loading checkpoint shards: 100%|██████████| 2/2 [00:57<00:00, 28.63s/it]
loading data for dataset:  ./alpaca_data.json
loading data for dataset:  ./alpaca_data.json
loading data for dataset:  ./alpaca_data.json
loading data for dataset:  ./alpaca_data.json

  0%|          | 0/10000 [00:00<?, ?it/s]
  0%|          | 0/10000 [00:00<?, ?it/s]
  0%|          | 0/10000 [00:00<?, ?it/s]
  0%|          | 0/10000 [00:00<?, ?it/s]
  1%|          | 60/10000 [00:00<00:17, 555.03it/s]
  0%|          | 38/10000 [00:00<00:29, 335.81it/s]
  0%|          | 41/10000 [00:00<00:24, 400.25it/s]
  0%|          | 41/10000 [00:00<00:26, 377.13it/s]
  1%|          | 116/10000 [00:00<00:20, 485.85it/s]
  1%|          | 80/10000 [00:00<00:27, 365.56it/s]
  1%|          | 82/10000 [00:00<00:25, 391.22it/s]
  1%|          | 81/10000 [00:00<00:25, 384.40it/s]
  1%|          | 124/10000 [00:00<00:25, 390.38it/s]
  1%|          | 123/10000 [00:00<00:24, 398.45it/s]
  2%|▏         | 165/10000 [00:00<00:23, 426.58it/s]
  1%|          | 120/10000 [00:00<00:25, 380.43it/s]
  2%|▏         | 164/10000 [00:00<00:24, 401.20it/s]
  2%|▏         | 164/10000 [00:00<00:25, 378.84it/s]
  2%|▏         | 209/10000 [00:00<00:24, 405.59it/s]
  2%|▏         | 162/10000 [00:00<00:25, 382.85it/s]
  2%|▏         | 204/10000 [00:00<00:25, 384.16it/s]
  2%|▏         | 205/10000 [00:00<00:25, 378.09it/s]
  2%|▎         | 250/10000 [00:00<00:24, 399.90it/s]
  2%|▏         | 206/10000 [00:00<00:24, 394.46it/s]
  2%|▏         | 243/10000 [00:00<00:25, 381.15it/s]
  2%|▏         | 244/10000 [00:00<00:25, 377.70it/s]
  3%|▎         | 291/10000 [00:00<00:24, 399.91it/s]
  2%|▏         | 246/10000 [00:00<00:24, 391.50it/s]
  3%|▎         | 283/10000 [00:00<00:25, 386.28it/s]
  3%|▎         | 285/10000 [00:00<00:25, 386.54it/s]
  3%|▎         | 332/10000 [00:00<00:24, 390.23it/s]
  3%|▎         | 286/10000 [00:00<00:25, 383.43it/s]
  3%|▎         | 322/10000 [00:00<00:25, 380.29it/s]
  3%|▎         | 325/10000 [00:00<00:24, 389.21it/s]
  4%|▎         | 372/10000 [00:00<00:25, 384.90it/s]
  3%|▎         | 325/10000 [00:00<00:25, 381.74it/s]
  4%|▎         | 361/10000 [00:00<00:25, 382.68it/s]
  4%|▎         | 365/10000 [00:00<00:25, 374.52it/s]
  4%|▍         | 411/10000 [00:01<00:25, 378.28it/s]
  4%|▎         | 364/10000 [00:00<00:25, 381.54it/s]
  4%|▍         | 400/10000 [00:01<00:26, 368.99it/s]
  4%|▍         | 403/10000 [00:01<00:25, 371.17it/s]
  4%|▍         | 449/10000 [00:01<00:25, 375.89it/s]
  4%|▍         | 403/10000 [00:01<00:25, 380.58it/s]
  4%|▍         | 439/10000 [00:01<00:25, 373.28it/s]
  4%|▍         | 442/10000 [00:01<00:25, 375.67it/s]
  5%|▍         | 488/10000 [00:01<00:25, 379.22it/s]
  4%|▍         | 442/10000 [00:01<00:25, 371.44it/s]
  5%|▍         | 481/10000 [00:01<00:24, 385.74it/s]
  5%|▍         | 486/10000 [00:01<00:24, 384.23it/s]
  5%|▌         | 526/10000 [00:01<00:25, 377.64it/s]
  5%|▍         | 485/10000 [00:01<00:25, 376.99it/s]
  5%|▌         | 521/10000 [00:01<00:24, 389.75it/s]
  5%|▌         | 526/10000 [00:01<00:24, 387.65it/s]
  6%|▌         | 565/10000 [00:01<00:25, 376.86it/s]
  5%|▌         | 523/10000 [00:01<00:25, 377.68it/s]
  6%|▌         | 561/10000 [00:01<00:24, 391.88it/s]
  6%|▌         | 566/10000 [00:01<00:24, 379.38it/s]
  6%|▌         | 607/10000 [00:01<00:24, 378.03it/s]
  6%|▌         | 561/10000 [00:01<00:25, 374.35it/s]
  6%|▌         | 601/10000 [00:01<00:24, 376.68it/s]
  6%|▌         | 607/10000 [00:01<00:24, 387.56it/s]
  6%|▋         | 647/10000 [00:01<00:24, 380.95it/s]
  6%|▌         | 603/10000 [00:01<00:25, 374.66it/s]
  6%|▋         | 639/10000 [00:01<00:24, 377.57it/s]
  6%|▋         | 646/10000 [00:01<00:25, 370.27it/s]
  7%|▋         | 686/10000 [00:01<00:24, 382.60it/s]
  6%|▋         | 645/10000 [00:01<00:24, 378.95it/s]
  7%|▋         | 679/10000 [00:01<00:24, 381.72it/s]
  7%|▋         | 692/10000 [00:01<00:23, 395.15it/s]
  7%|▋         | 725/10000 [00:01<00:24, 382.69it/s]
  7%|▋         | 683/10000 [00:01<00:24, 378.41it/s]
  7%|▋         | 719/10000 [00:01<00:24, 381.52it/s]
  8%|▊         | 765/10000 [00:01<00:23, 387.21it/s]
  7%|▋         | 732/10000 [00:01<00:24, 373.79it/s]
  7%|▋         | 724/10000 [00:01<00:24, 383.36it/s]
  8%|▊         | 759/10000 [00:02<00:24, 374.71it/s]
  8%|▊         | 804/10000 [00:02<00:23, 385.99it/s]
  8%|▊         | 771/10000 [00:02<00:24, 377.56it/s]
  8%|▊         | 763/10000 [00:02<00:24, 381.94it/s]
  8%|▊         | 804/10000 [00:02<00:24, 382.86it/s]
  8%|▊         | 843/10000 [00:02<00:23, 386.78it/s]
  8%|▊         | 813/10000 [00:02<00:23, 385.06it/s]
  8%|▊         | 802/10000 [00:02<00:24, 381.80it/s]
  8%|▊         | 843/10000 [00:02<00:23, 384.85it/s]
  9%|▉         | 882/10000 [00:02<00:23, 380.33it/s]
  9%|▊         | 852/10000 [00:02<00:23, 384.77it/s]
  8%|▊         | 844/10000 [00:02<00:23, 385.39it/s]
  9%|▉         | 882/10000 [00:02<00:23, 383.88it/s]
  9%|▉         | 891/10000 [00:02<00:24, 377.87it/s]
  9%|▉         | 921/10000 [00:02<00:24, 365.40it/s]
  9%|▉         | 883/10000 [00:02<00:24, 376.93it/s]
  9%|▉         | 921/10000 [00:02<00:24, 373.86it/s]
  9%|▉         | 930/10000 [00:02<00:23, 380.52it/s]
 10%|▉         | 963/10000 [00:02<00:24, 367.96it/s]
  9%|▉         | 921/10000 [00:02<00:24, 376.00it/s]
 10%|▉         | 959/10000 [00:02<00:24, 370.67it/s]
 10%|▉         | 969/10000 [00:02<00:23, 382.35it/s]
 10%|█         | 1007/10000 [00:02<00:23, 375.44it/s]
 10%|▉         | 959/10000 [00:02<00:24, 373.96it/s]
 10%|▉         | 999/10000 [00:02<00:24, 368.34it/s]
 10%|█         | 1008/10000 [00:02<00:23, 378.18it/s]
 10%|█         | 1048/10000 [00:02<00:23, 379.62it/s]
 10%|▉         | 997/10000 [00:02<00:24, 370.68it/s]
 10%|█         | 1041/10000 [00:02<00:23, 382.40it/s]
 10%|█         | 1049/10000 [00:02<00:23, 375.04it/s]
 11%|█         | 1087/10000 [00:02<00:23, 381.93it/s]
 10%|█         | 1040/10000 [00:02<00:23, 387.71it/s]
 11%|█         | 1080/10000 [00:02<00:23, 372.55it/s]
 11%|█         | 1091/10000 [00:02<00:23, 374.73it/s]
 11%|█▏        | 1127/10000 [00:02<00:23, 380.44it/s]
 11%|█         | 1079/10000 [00:02<00:24, 370.96it/s]
 11%|█         | 1120/10000 [00:02<00:23, 379.66it/s]
 11%|█▏        | 1131/10000 [00:02<00:23, 380.67it/s]
 12%|█▏        | 1167/10000 [00:03<00:22, 385.05it/s]
 11%|█         | 1120/10000 [00:02<00:23, 380.38it/s]
 12%|█▏        | 1159/10000 [00:03<00:23, 380.32it/s]
 12%|█▏        | 1171/10000 [00:03<00:23, 378.72it/s]
 12%|█▏        | 1207/10000 [00:03<00:22, 388.80it/s]
 12%|█▏        | 1159/10000 [00:03<00:23, 376.29it/s]
 12%|█▏        | 1198/10000 [00:03<00:23, 377.90it/s]
 12%|█▏        | 1211/10000 [00:03<00:22, 383.54it/s]
 12%|█▏        | 1247/10000 [00:03<00:22, 381.08it/s]
 12%|█▏        | 1198/10000 [00:03<00:23, 379.19it/s]
 12%|█▏        | 1236/10000 [00:03<00:23, 375.64it/s]
 13%|█▎        | 1252/10000 [00:03<00:22, 390.15it/s]
 13%|█▎        | 1289/10000 [00:03<00:22, 384.28it/s]
 12%|█▏        | 1238/10000 [00:03<00:23, 380.03it/s]
 13%|█▎        | 1279/10000 [00:03<00:22, 384.70it/s]
 13%|█▎        | 1292/10000 [00:03<00:23, 378.04it/s]
 13%|█▎        | 1329/10000 [00:03<00:22, 387.49it/s]
 13%|█▎        | 1279/10000 [00:03<00:22, 380.63it/s]
 13%|█▎        | 1322/10000 [00:03<00:22, 388.84it/s]
 13%|█▎        | 1333/10000 [00:03<00:22, 382.48it/s]
 14%|█▎        | 1369/10000 [00:03<00:22, 390.02it/s]
 13%|█▎        | 1321/10000 [00:03<00:22, 382.71it/s]
 14%|█▎        | 1361/10000 [00:03<00:22, 388.62it/s]
 14%|█▍        | 1376/10000 [00:03<00:21, 396.04it/s]
 14%|█▍        | 1409/10000 [00:03<00:22, 386.95it/s]
 14%|█▎        | 1364/10000 [00:03<00:21, 394.96it/s]
 14%|█▍        | 1401/10000 [00:03<00:22, 387.36it/s]
 14%|█▍        | 1416/10000 [00:03<00:22, 383.59it/s]
 14%|█▍        | 1448/10000 [00:03<00:22, 380.24it/s]
 14%|█▍        | 1404/10000 [00:03<00:21, 394.80it/s]
 14%|█▍        | 1442/10000 [00:03<00:22, 381.02it/s]
 15%|█▍        | 1456/10000 [00:03<00:22, 388.12it/s]
 15%|█▍        | 1488/10000 [00:03<00:22, 385.33it/s]
 14%|█▍        | 1444/10000 [00:03<00:22, 383.51it/s]
 15%|█▍        | 1488/10000 [00:03<00:21, 390.39it/s]
 15%|█▍        | 1497/10000 [00:03<00:21, 394.30it/s]
 15%|█▌        | 1530/10000 [00:03<00:22, 383.54it/s]
 15%|█▍        | 1483/10000 [00:03<00:22, 378.42it/s]
 15%|█▌        | 1528/10000 [00:04<00:21, 390.07it/s]
 15%|█▌        | 1537/10000 [00:04<00:21, 386.96it/s]
 16%|█▌        | 1569/10000 [00:04<00:22, 381.65it/s]
 15%|█▌        | 1526/10000 [00:03<00:21, 391.58it/s]
 16%|█▌        | 1567/10000 [00:04<00:21, 385.59it/s]
 16%|█▌        | 1576/10000 [00:04<00:22, 380.93it/s]
 16%|█▌        | 1611/10000 [00:04<00:22, 379.58it/s]
 16%|█▌        | 1566/10000 [00:04<00:22, 379.38it/s]
 16%|█▌        | 1606/10000 [00:04<00:21, 384.11it/s]
 16%|█▌        | 1618/10000 [00:04<00:21, 392.19it/s]
 16%|█▋        | 1649/10000 [00:04<00:22, 375.92it/s]
 16%|█▌        | 1605/10000 [00:04<00:22, 378.72it/s]
 16%|█▋        | 1645/10000 [00:04<00:22, 373.98it/s]
 17%|█▋        | 1658/10000 [00:04<00:22, 378.20it/s]
 17%|█▋        | 1690/10000 [00:04<00:22, 370.24it/s]
 16%|█▋        | 1643/10000 [00:04<00:22, 371.12it/s]
 17%|█▋        | 1683/10000 [00:04<00:22, 367.73it/s]
 17%|█▋        | 1696/10000 [00:04<00:22, 367.63it/s]
 17%|█▋        | 1730/10000 [00:04<00:21, 378.55it/s]
 17%|█▋        | 1682/10000 [00:04<00:22, 366.48it/s]
 17%|█▋        | 1725/10000 [00:04<00:21, 378.53it/s]
 17%|█▋        | 1737/10000 [00:04<00:21, 379.56it/s]
 18%|█▊        | 1769/10000 [00:04<00:21, 381.10it/s]
 17%|█▋        | 1722/10000 [00:04<00:22, 375.65it/s]
 18%|█▊        | 1763/10000 [00:04<00:21, 375.79it/s]
 18%|█▊        | 1810/10000 [00:04<00:21, 389.07it/s]
 18%|█▊        | 1776/10000 [00:04<00:22, 370.78it/s]
 18%|█▊        | 1760/10000 [00:04<00:21, 375.91it/s]
 18%|█▊        | 1804/10000 [00:04<00:21, 385.22it/s]
 18%|█▊        | 1849/10000 [00:04<00:21, 384.77it/s]
 18%|█▊        | 1816/10000 [00:04<00:21, 378.08it/s]
 18%|█▊        | 1798/10000 [00:04<00:22, 371.14it/s]
 18%|█▊        | 1845/10000 [00:04<00:21, 377.54it/s]
 19%|█▊        | 1858/10000 [00:04<00:20, 389.56it/s]
 19%|█▉        | 1888/10000 [00:04<00:21, 382.02it/s]
 18%|█▊        | 1842/10000 [00:04<00:21, 383.66it/s]
 19%|█▉        | 1885/10000 [00:04<00:21, 375.62it/s]
 19%|█▉        | 1898/10000 [00:04<00:20, 387.30it/s]
 19%|█▉        | 1930/10000 [00:05<00:20, 384.52it/s]
 19%|█▉        | 1881/10000 [00:04<00:21, 385.23it/s]
 19%|█▉        | 1924/10000 [00:05<00:21, 378.97it/s]
 19%|█▉        | 1937/10000 [00:05<00:21, 381.43it/s]
 20%|█▉        | 1969/10000 [00:05<00:20, 384.16it/s]
 19%|█▉        | 1920/10000 [00:05<00:21, 384.66it/s]
 20%|█▉        | 1964/10000 [00:05<00:20, 385.03it/s]
 20%|█▉        | 1976/10000 [00:05<00:20, 383.48it/s]
 20%|██        | 2008/10000 [00:05<00:20, 381.22it/s]
 20%|█▉        | 1960/10000 [00:05<00:20, 384.11it/s]
 20%|██        | 2004/10000 [00:05<00:20, 385.60it/s]
 20%|██        | 2047/10000 [00:05<00:20, 381.31it/s]
 20%|██        | 2016/10000 [00:05<00:20, 380.79it/s]
 20%|█▉        | 1999/10000 [00:05<00:21, 378.07it/s]
 20%|██        | 2043/10000 [00:05<00:20, 386.78it/s]
 21%|██        | 2087/10000 [00:05<00:20, 386.53it/s]
 21%|██        | 2056/10000 [00:05<00:21, 376.53it/s]
 20%|██        | 2039/10000 [00:05<00:21, 378.37it/s]
 21%|██        | 2086/10000 [00:05<00:20, 391.20it/s]
 21%|██▏       | 2126/10000 [00:05<00:20, 381.58it/s]
 21%|██        | 2098/10000 [00:05<00:20, 388.10it/s]
 21%|██        | 2080/10000 [00:05<00:20, 386.62it/s]
 21%|██▏       | 2128/10000 [00:05<00:20, 386.15it/s]
 22%|██▏       | 2167/10000 [00:05<00:20, 381.78it/s]
 21%|██▏       | 2139/10000 [00:05<00:20, 389.71it/s]
 21%|██        | 2120/10000 [00:05<00:20, 385.29it/s]
 22%|██▏       | 2171/10000 [00:05<00:19, 393.92it/s]
 22%|██▏       | 2179/10000 [00:05<00:20, 389.99it/s]
 22%|██▏       | 2208/10000 [00:05<00:20, 384.06it/s]
 22%|██▏       | 2160/10000 [00:05<00:20, 389.34it/s]
 22%|██▏       | 2211/10000 [00:05<00:20, 380.76it/s]
 22%|██▏       | 2247/10000 [00:05<00:20, 383.31it/s]
 22%|██▏       | 2219/10000 [00:05<00:20, 388.70it/s]
 22%|██▏       | 2200/10000 [00:05<00:19, 391.43it/s]
 23%|██▎       | 2253/10000 [00:05<00:20, 387.26it/s]
 23%|██▎       | 2258/10000 [00:05<00:19, 387.86it/s]
 23%|██▎       | 2286/10000 [00:05<00:20, 382.17it/s]
 22%|██▏       | 2240/10000 [00:05<00:20, 387.04it/s]
 23%|██▎       | 2297/10000 [00:05<00:19, 386.51it/s]
 23%|██▎       | 2292/10000 [00:06<00:20, 377.90it/s]
 23%|██▎       | 2326/10000 [00:06<00:20, 380.56it/s]
 23%|██▎       | 2279/10000 [00:05<00:20, 383.02it/s]
 23%|██▎       | 2337/10000 [00:06<00:19, 398.25it/s]
 23%|██▎       | 2336/10000 [00:06<00:20, 380.50it/s]
 24%|██▎       | 2365/10000 [00:06<00:20, 378.56it/s]
 23%|██▎       | 2320/10000 [00:06<00:19, 390.61it/s]
 24%|██▍       | 2375/10000 [00:06<00:20, 379.98it/s]
 24%|██▍       | 2410/10000 [00:06<00:19, 389.87it/s]
 24%|██▍       | 2377/10000 [00:06<00:20, 379.38it/s]
 24%|██▎       | 2360/10000 [00:06<00:20, 378.70it/s]
 24%|██▍       | 2415/10000 [00:06<00:19, 384.84it/s]
 24%|██▍       | 2418/10000 [00:06<00:19, 384.14it/s]
 24%|██▍       | 2450/10000 [00:06<00:19, 385.64it/s]
 24%|██▍       | 2400/10000 [00:06<00:19, 384.61it/s]
 25%|██▍       | 2457/10000 [00:06<00:19, 390.25it/s]
 25%|██▍       | 2460/10000 [00:06<00:19, 393.93it/s]
 25%|██▍       | 2492/10000 [00:06<00:19, 385.45it/s]
 24%|██▍       | 2439/10000 [00:06<00:19, 385.91it/s]
 25%|██▍       | 2497/10000 [00:06<00:19, 387.36it/s]
 25%|██▌       | 2500/10000 [00:06<00:19, 389.93it/s]
 25%|██▌       | 2531/10000 [00:06<00:19, 386.25it/s]
 25%|██▍       | 2478/10000 [00:06<00:19, 377.93it/s]
 25%|██▌       | 2539/10000 [00:06<00:18, 396.06it/s]
 25%|██▌       | 2540/10000 [00:06<00:18, 392.65it/s]
 26%|██▌       | 2572/10000 [00:06<00:18, 391.62it/s]
 25%|██▌       | 2520/10000 [00:06<00:19, 376.33it/s]
 26%|██▌       | 2579/10000 [00:06<00:19, 387.12it/s]
 26%|██▌       | 2580/10000 [00:06<00:19, 381.59it/s]
 26%|██▌       | 2612/10000 [00:06<00:19, 378.11it/s]
 26%|██▌       | 2560/10000 [00:06<00:19, 381.32it/s]
 26%|██▌       | 2618/10000 [00:06<00:19, 371.35it/s]
 26%|██▌       | 2619/10000 [00:06<00:19, 371.70it/s]
 26%|██▋       | 2650/10000 [00:06<00:19, 375.26it/s]
 26%|██▌       | 2599/10000 [00:06<00:19, 377.73it/s]
 27%|██▋       | 2656/10000 [00:06<00:19, 368.67it/s]
 27%|██▋       | 2658/10000 [00:06<00:19, 374.54it/s]
 27%|██▋       | 2688/10000 [00:06<00:19, 376.32it/s]
 26%|██▋       | 2637/10000 [00:06<00:19, 370.92it/s]
 27%|██▋       | 2697/10000 [00:07<00:19, 376.15it/s]
 27%|██▋       | 2727/10000 [00:07<00:19, 379.24it/s]
 27%|██▋       | 2700/10000 [00:07<00:19, 378.92it/s]
 27%|██▋       | 2675/10000 [00:07<00:19, 369.52it/s]
 27%|██▋       | 2735/10000 [00:07<00:19, 372.65it/s]
 28%|██▊       | 2767/10000 [00:07<00:19, 376.72it/s]
 27%|██▋       | 2738/10000 [00:07<00:19, 376.90it/s]
 27%|██▋       | 2714/10000 [00:07<00:19, 375.00it/s]
 28%|██▊       | 2778/10000 [00:07<00:18, 382.83it/s]
 28%|██▊       | 2775/10000 [00:07<00:19, 371.99it/s]
 28%|██▊       | 2805/10000 [00:07<00:19, 364.86it/s]
 28%|██▊       | 2752/10000 [00:07<00:19, 371.17it/s]
 28%|██▊       | 2813/10000 [00:07<00:19, 373.77it/s]
 28%|██▊       | 2846/10000 [00:07<00:18, 377.08it/s]
 28%|██▊       | 2817/10000 [00:07<00:19, 369.62it/s]
 28%|██▊       | 2790/10000 [00:07<00:19, 368.92it/s]
 29%|██▊       | 2852/10000 [00:07<00:19, 374.75it/s]
 29%|██▉       | 2887/10000 [00:07<00:18, 385.72it/s]
 29%|██▊       | 2860/10000 [00:07<00:18, 375.84it/s]
 28%|██▊       | 2828/10000 [00:07<00:19, 370.88it/s]
 29%|██▉       | 2893/10000 [00:07<00:18, 384.94it/s]
 29%|██▉       | 2927/10000 [00:07<00:18, 389.40it/s]
 29%|██▉       | 2905/10000 [00:07<00:18, 383.18it/s]
 29%|██▊       | 2868/10000 [00:07<00:18, 375.61it/s]
 29%|██▉       | 2932/10000 [00:07<00:18, 385.33it/s]
 30%|██▉       | 2967/10000 [00:07<00:18, 383.51it/s]
 29%|██▉       | 2909/10000 [00:07<00:18, 384.33it/s]
 29%|██▉       | 2944/10000 [00:07<00:18, 382.29it/s]
 30%|██▉       | 2973/10000 [00:07<00:18, 381.71it/s]
 30%|███       | 3006/10000 [00:07<00:18, 370.13it/s]
 30%|██▉       | 2950/10000 [00:07<00:18, 389.33it/s]
 30%|██▉       | 2983/10000 [00:07<00:18, 378.11it/s]
 30%|███       | 3012/10000 [00:07<00:18, 378.73it/s]
 30%|███       | 3044/10000 [00:07<00:18, 372.35it/s]
 30%|███       | 3021/10000 [00:07<00:18, 372.54it/s]
 30%|██▉       | 2989/10000 [00:07<00:18, 375.94it/s]
 31%|███       | 3051/10000 [00:07<00:18, 373.23it/s]
 31%|███       | 3085/10000 [00:08<00:18, 382.07it/s]
 31%|███       | 3059/10000 [00:08<00:18, 372.25it/s]
 30%|███       | 3027/10000 [00:07<00:19, 365.56it/s]
 31%|███       | 3094/10000 [00:08<00:17, 384.84it/s]
 31%|███       | 3124/10000 [00:08<00:18, 376.66it/s]
 31%|███       | 3098/10000 [00:08<00:18, 365.36it/s]
 31%|███       | 3065/10000 [00:08<00:19, 364.76it/s]
 31%|███▏      | 3133/10000 [00:08<00:17, 382.34it/s]
 32%|███▏      | 3162/10000 [00:08<00:18, 368.88it/s]
 31%|███▏      | 3137/10000 [00:08<00:18, 370.90it/s]
 31%|███       | 3105/10000 [00:08<00:18, 373.55it/s]
 32%|███▏      | 3172/10000 [00:08<00:17, 381.85it/s]
 32%|███▏      | 3203/10000 [00:08<00:17, 379.90it/s]
 32%|███▏      | 3178/10000 [00:08<00:17, 379.20it/s]
 31%|███▏      | 3143/10000 [00:08<00:18, 368.76it/s]
 32%|███▏      | 3211/10000 [00:08<00:18, 369.35it/s]
 32%|███▏      | 3242/10000 [00:08<00:17, 376.93it/s]
 32%|███▏      | 3218/10000 [00:08<00:17, 380.45it/s]
 32%|███▏      | 3184/10000 [00:08<00:17, 380.62it/s]
 32%|███▏      | 3249/10000 [00:08<00:18, 370.97it/s]
 33%|███▎      | 3280/10000 [00:08<00:17, 376.52it/s]
 32%|███▏      | 3223/10000 [00:08<00:17, 379.63it/s]
 33%|███▎      | 3257/10000 [00:08<00:17, 375.21it/s]
 33%|███▎      | 3292/10000 [00:08<00:17, 388.01it/s]
 33%|███▎      | 3319/10000 [00:08<00:17, 375.74it/s]
 33%|███▎      | 3301/10000 [00:08<00:17, 392.95it/s]
 33%|███▎      | 3262/10000 [00:08<00:17, 379.39it/s]
 33%|███▎      | 3331/10000 [00:08<00:17, 378.11it/s]
 34%|███▎      | 3358/10000 [00:08<00:17, 379.74it/s]
 33%|███▎      | 3300/10000 [00:08<00:17, 372.86it/s]
 33%|███▎      | 3341/10000 [00:08<00:18, 360.99it/s]
 34%|███▍      | 3397/10000 [00:08<00:17, 379.02it/s]
 34%|███▎      | 3369/10000 [00:08<00:17, 373.01it/s]
 33%|███▎      | 3342/10000 [00:08<00:17, 379.97it/s]
 34%|███▍      | 3378/10000 [00:08<00:18, 353.81it/s]
 34%|███▍      | 3435/10000 [00:08<00:17, 376.43it/s]
 34%|███▍      | 3408/10000 [00:08<00:17, 373.03it/s]
 34%|███▍      | 3381/10000 [00:08<00:17, 380.76it/s]
 34%|███▍      | 3419/10000 [00:09<00:18, 362.81it/s]
 35%|███▍      | 3473/10000 [00:09<00:17, 375.46it/s]
 34%|███▍      | 3449/10000 [00:09<00:17, 379.33it/s]
 34%|███▍      | 3420/10000 [00:09<00:17, 372.85it/s]
 35%|███▍      | 3459/10000 [00:09<00:17, 372.17it/s]
 35%|███▌      | 3511/10000 [00:09<00:17, 372.15it/s]
 35%|███▍      | 3488/10000 [00:09<00:17, 382.28it/s]
 35%|███▍      | 3458/10000 [00:09<00:17, 373.22it/s]
 35%|███▍      | 3497/10000 [00:09<00:17, 373.06it/s]
 36%|███▌      | 3551/10000 [00:09<00:16, 380.31it/s]
 35%|███▌      | 3527/10000 [00:09<00:17, 365.79it/s]
 35%|███▍      | 3496/10000 [00:09<00:17, 372.96it/s]
 35%|███▌      | 3535/10000 [00:09<00:17, 361.54it/s]
 36%|███▌      | 3590/10000 [00:09<00:17, 376.62it/s]
 36%|███▌      | 3566/10000 [00:09<00:17, 372.53it/s]
 35%|███▌      | 3534/10000 [00:09<00:17, 369.01it/s]
 36%|███▌      | 3577/10000 [00:09<00:17, 369.97it/s]
 36%|███▋      | 3628/10000 [00:09<00:17, 373.94it/s]
 36%|███▌      | 3609/10000 [00:09<00:17, 369.49it/s]
 36%|███▌      | 3571/10000 [00:09<00:17, 357.79it/s]
 36%|███▌      | 3615/10000 [00:09<00:18, 343.61it/s]
 37%|███▋      | 3701/10000 [00:09<00:13, 477.34it/s]
 36%|███▋      | 3647/10000 [00:09<00:18, 351.37it/s]
 36%|███▌      | 3607/10000 [00:09<00:18, 341.96it/s]
 37%|███▋      | 3653/10000 [00:09<00:17, 352.86it/s]
 37%|███▋      | 3749/10000 [00:09<00:14, 432.22it/s]
 37%|███▋      | 3671/10000 [00:09<00:14, 424.36it/s]
 37%|███▋      | 3683/10000 [00:09<00:19, 329.41it/s]
 37%|███▋      | 3689/10000 [00:09<00:20, 313.25it/s]
 37%|███▋      | 3725/10000 [00:09<00:13, 456.46it/s]
 38%|███▊      | 3794/10000 [00:09<00:15, 398.50it/s]
 37%|███▋      | 3717/10000 [00:09<00:19, 325.93it/s]
 38%|███▊      | 3753/10000 [00:09<00:15, 397.47it/s]
 38%|███▊      | 3750/10000 [00:09<00:19, 314.48it/s]
 38%|███▊      | 3835/10000 [00:09<00:17, 358.96it/s]
 38%|███▊      | 3772/10000 [00:09<00:15, 391.80it/s]
 38%|███▊      | 3807/10000 [00:10<00:14, 435.27it/s]
 38%|███▊      | 3789/10000 [00:10<00:18, 334.70it/s]
 39%|███▉      | 3875/10000 [00:10<00:16, 368.44it/s]
 38%|███▊      | 3814/10000 [00:10<00:15, 393.55it/s]
 39%|███▊      | 3853/10000 [00:10<00:14, 422.59it/s]
 38%|███▊      | 3829/10000 [00:10<00:18, 340.81it/s]
 39%|███▉      | 3916/10000 [00:10<00:16, 378.67it/s]
 39%|███▊      | 3855/10000 [00:10<00:15, 389.41it/s]
 39%|███▉      | 3897/10000 [00:10<00:14, 415.25it/s]
 39%|███▊      | 3868/10000 [00:10<00:17, 353.88it/s]
 40%|███▉      | 3955/10000 [00:10<00:16, 374.41it/s]
 39%|███▉      | 3895/10000 [00:10<00:15, 388.04it/s]
 39%|███▉      | 3940/10000 [00:10<00:15, 396.61it/s]
 39%|███▉      | 3910/10000 [00:10<00:16, 372.09it/s]
 40%|███▉      | 3994/10000 [00:10<00:15, 376.53it/s]
 39%|███▉      | 3935/10000 [00:10<00:15, 380.30it/s]
 40%|███▉      | 3982/10000 [00:10<00:15, 394.22it/s]
 39%|███▉      | 3949/10000 [00:10<00:16, 367.25it/s]
 40%|████      | 4033/10000 [00:10<00:15, 374.52it/s]
 40%|███▉      | 3975/10000 [00:10<00:15, 382.61it/s]
 40%|████      | 4022/10000 [00:10<00:15, 389.98it/s]
 40%|███▉      | 3992/10000 [00:10<00:16, 375.03it/s]
 41%|████      | 4071/10000 [00:10<00:16, 360.50it/s]
 40%|████      | 4016/10000 [00:10<00:15, 380.27it/s]
 41%|████      | 4062/10000 [00:10<00:15, 381.82it/s]
 40%|████      | 4030/10000 [00:10<00:16, 370.34it/s]
 41%|████      | 4113/10000 [00:10<00:15, 376.60it/s]
 41%|████      | 4055/10000 [00:10<00:15, 379.42it/s]
 41%|████      | 4101/10000 [00:10<00:15, 380.43it/s]
 41%|████      | 4068/10000 [00:10<00:16, 370.43it/s]
 42%|████▏     | 4154/10000 [00:10<00:15, 376.14it/s]
 41%|████      | 4094/10000 [00:10<00:15, 378.89it/s]
 41%|████▏     | 4140/10000 [00:10<00:15, 378.52it/s]
 41%|████      | 4109/10000 [00:10<00:15, 375.76it/s]
 42%|████▏     | 4194/10000 [00:10<00:15, 382.39it/s]
 41%|████▏     | 4136/10000 [00:10<00:15, 389.26it/s]
 42%|████▏     | 4178/10000 [00:10<00:15, 375.77it/s]
 41%|████▏     | 4148/10000 [00:10<00:15, 373.37it/s]
 42%|████▏     | 4233/10000 [00:11<00:15, 376.97it/s]
 42%|████▏     | 4176/10000 [00:10<00:15, 374.39it/s]
 42%|████▏     | 4217/10000 [00:11<00:15, 378.47it/s]
 42%|████▏     | 4186/10000 [00:11<00:15, 370.96it/s]
 43%|████▎     | 4273/10000 [00:11<00:14, 382.61it/s]
 42%|████▏     | 4215/10000 [00:11<00:15, 368.20it/s]
 43%|████▎     | 4256/10000 [00:11<00:15, 376.98it/s]
 42%|████▏     | 4224/10000 [00:11<00:15, 372.28it/s]
 43%|████▎     | 4312/10000 [00:11<00:15, 378.85it/s]
 43%|████▎     | 4256/10000 [00:11<00:15, 375.57it/s]
 43%|████▎     | 4295/10000 [00:11<00:15, 379.32it/s]
 43%|████▎     | 4266/10000 [00:11<00:14, 385.69it/s]
 44%|████▎     | 4350/10000 [00:11<00:15, 374.27it/s]
 43%|████▎     | 4295/10000 [00:11<00:15, 378.17it/s]
 43%|████▎     | 4333/10000 [00:11<00:15, 369.94it/s]
 43%|████▎     | 4305/10000 [00:11<00:14, 381.38it/s]
 44%|████▍     | 4391/10000 [00:11<00:14, 384.26it/s]
 43%|████▎     | 4337/10000 [00:11<00:14, 388.33it/s]
 44%|████▎     | 4371/10000 [00:11<00:15, 369.32it/s]
 43%|████▎     | 4344/10000 [00:11<00:14, 377.57it/s]
 44%|████▍     | 4430/10000 [00:11<00:14, 385.33it/s]
 44%|████▍     | 4376/10000 [00:11<00:14, 376.15it/s]
 44%|████▍     | 4411/10000 [00:11<00:14, 376.94it/s]
 44%|████▍     | 4386/10000 [00:11<00:14, 389.40it/s]
 45%|████▍     | 4469/10000 [00:11<00:14, 383.20it/s]
 44%|████▍     | 4417/10000 [00:11<00:14, 380.56it/s]
 44%|████▍     | 4449/10000 [00:11<00:14, 375.83it/s]
 44%|████▍     | 4425/10000 [00:11<00:14, 377.25it/s]
 45%|████▌     | 4508/10000 [00:11<00:14, 376.91it/s]
 45%|████▍     | 4456/10000 [00:11<00:14, 380.00it/s]
 45%|████▍     | 4487/10000 [00:11<00:14, 373.17it/s]
 45%|████▍     | 4466/10000 [00:11<00:14, 373.03it/s]
 45%|████▌     | 4546/10000 [00:11<00:14, 369.56it/s]
 45%|████▍     | 4495/10000 [00:11<00:14, 381.07it/s]
 45%|████▌     | 4526/10000 [00:11<00:14, 377.58it/s]
 46%|████▌     | 4585/10000 [00:11<00:14, 369.78it/s]
 45%|████▌     | 4506/10000 [00:11<00:15, 355.78it/s]
 45%|████▌     | 4534/10000 [00:11<00:14, 382.81it/s]
 46%|████▌     | 4564/10000 [00:12<00:15, 341.51it/s]
 46%|████▌     | 4623/10000 [00:12<00:14, 359.22it/s]
 45%|████▌     | 4542/10000 [00:12<00:15, 345.40it/s]
 46%|████▌     | 4573/10000 [00:12<00:15, 359.90it/s]
 46%|████▌     | 4599/10000 [00:12<00:16, 329.56it/s]
 47%|████▋     | 4659/10000 [00:12<00:15, 337.82it/s]
 46%|████▌     | 4577/10000 [00:12<00:16, 330.82it/s]
 46%|████▌     | 4610/10000 [00:12<00:14, 360.70it/s]
 46%|████▋     | 4637/10000 [00:12<00:15, 336.13it/s]
 46%|████▋     | 4647/10000 [00:12<00:15, 352.64it/s]
 47%|████▋     | 4694/10000 [00:12<00:16, 316.08it/s]
 46%|████▌     | 4611/10000 [00:12<00:17, 307.66it/s]
 47%|████▋     | 4680/10000 [00:12<00:14, 354.92it/s]
 47%|████▋     | 4735/10000 [00:12<00:15, 340.41it/s]
 46%|████▋     | 4649/10000 [00:12<00:16, 326.60it/s]
 47%|████▋     | 4683/10000 [00:12<00:15, 336.99it/s]
 47%|████▋     | 4716/10000 [00:12<00:15, 349.31it/s]
 48%|████▊     | 4773/10000 [00:12<00:14, 348.70it/s]
 47%|████▋     | 4685/10000 [00:12<00:15, 334.55it/s]
 47%|████▋     | 4720/10000 [00:12<00:15, 345.87it/s]
 48%|████▊     | 4755/10000 [00:12<00:14, 351.42it/s]
 48%|████▊     | 4813/10000 [00:12<00:14, 362.09it/s]
 47%|████▋     | 4725/10000 [00:12<00:14, 352.49it/s]
 48%|████▊     | 4760/10000 [00:12<00:15, 349.11it/s]
 48%|████▊     | 4794/10000 [00:12<00:14, 361.64it/s]
 48%|████▊     | 4850/10000 [00:12<00:14, 357.25it/s]
 48%|████▊     | 4799/10000 [00:12<00:14, 359.81it/s]
 48%|████▊     | 4761/10000 [00:12<00:15, 340.48it/s]
 48%|████▊     | 4832/10000 [00:12<00:14, 366.22it/s]
 49%|████▉     | 4890/10000 [00:12<00:13, 368.46it/s]
 48%|████▊     | 4800/10000 [00:12<00:14, 351.27it/s]
 48%|████▊     | 4836/10000 [00:12<00:14, 356.25it/s]
 49%|████▊     | 4869/10000 [00:12<00:14, 361.83it/s]
 49%|████▉     | 4928/10000 [00:12<00:13, 369.80it/s]
 49%|████▊     | 4874/10000 [00:12<00:14, 358.77it/s]
 48%|████▊     | 4841/10000 [00:12<00:14, 360.13it/s]
 49%|████▉     | 4908/10000 [00:13<00:13, 369.45it/s]
 50%|████▉     | 4967/10000 [00:13<00:13, 367.59it/s]
 49%|████▉     | 4916/10000 [00:13<00:13, 376.35it/s]
 49%|████▉     | 4882/10000 [00:13<00:13, 370.04it/s]
 49%|████▉     | 4947/10000 [00:13<00:13, 366.61it/s]
 50%|█████     | 5004/10000 [00:13<00:13, 366.73it/s]
 49%|████▉     | 4920/10000 [00:13<00:13, 369.16it/s]
 50%|████▉     | 4954/10000 [00:13<00:13, 362.12it/s]
 50%|████▉     | 4986/10000 [00:13<00:13, 373.32it/s]
 50%|█████     | 5047/10000 [00:13<00:13, 376.16it/s]
 50%|████▉     | 4958/10000 [00:13<00:13, 368.61it/s]
 50%|████▉     | 4992/10000 [00:13<00:13, 360.77it/s]
 50%|█████     | 5027/10000 [00:13<00:13, 370.90it/s]
 51%|█████     | 5088/10000 [00:13<00:13, 372.05it/s]
 50%|████▉     | 4995/10000 [00:13<00:13, 365.99it/s]
 50%|█████     | 5037/10000 [00:13<00:13, 374.65it/s]
 51%|█████     | 5069/10000 [00:13<00:13, 372.17it/s]
 50%|█████     | 5035/10000 [00:13<00:13, 375.61it/s]
 51%|█████▏    | 5129/10000 [00:13<00:13, 369.69it/s]
 51%|█████     | 5079/10000 [00:13<00:12, 379.05it/s]
 51%|█████     | 5107/10000 [00:13<00:13, 369.99it/s]
 51%|█████     | 5074/10000 [00:13<00:13, 374.56it/s]
 52%|█████▏    | 5168/10000 [00:13<00:12, 372.72it/s]
 51%|█████     | 5119/10000 [00:13<00:12, 375.97it/s]
 51%|█████▏    | 5147/10000 [00:13<00:12, 374.25it/s]
 52%|█████▏    | 5212/10000 [00:13<00:12, 390.86it/s]
 51%|█████     | 5112/10000 [00:13<00:13, 363.36it/s]
 52%|█████▏    | 5158/10000 [00:13<00:12, 376.04it/s]
 52%|█████▏    | 5190/10000 [00:13<00:12, 389.50it/s]
 53%|█████▎    | 5252/10000 [00:13<00:12, 386.95it/s]
 52%|█████▏    | 5154/10000 [00:13<00:13, 370.44it/s]
 52%|█████▏    | 5201/10000 [00:13<00:12, 391.11it/s]
 52%|█████▏    | 5230/10000 [00:13<00:12, 381.58it/s]
 53%|█████▎    | 5294/10000 [00:13<00:11, 395.34it/s]
 52%|█████▏    | 5199/10000 [00:13<00:12, 388.48it/s]
 52%|█████▏    | 5241/10000 [00:13<00:12, 390.29it/s]
 53%|█████▎    | 5270/10000 [00:13<00:12, 386.47it/s]
 53%|█████▎    | 5334/10000 [00:14<00:12, 378.74it/s]
 52%|█████▏    | 5238/10000 [00:13<00:12, 384.35it/s]
 53%|█████▎    | 5281/10000 [00:13<00:12, 384.51it/s]
 53%|█████▎    | 5309/10000 [00:14<00:12, 380.90it/s]
 54%|█████▎    | 5374/10000 [00:14<00:12, 383.64it/s]
 53%|█████▎    | 5277/10000 [00:14<00:12, 382.22it/s]
 53%|█████▎    | 5322/10000 [00:14<00:12, 373.56it/s]
 53%|█████▎    | 5348/10000 [00:14<00:12, 379.71it/s]
 53%|█████▎    | 5316/10000 [00:14<00:12, 383.72it/s]
 54%|█████▍    | 5413/10000 [00:14<00:12, 369.78it/s]
 54%|█████▎    | 5362/10000 [00:14<00:12, 380.88it/s]
 54%|█████▍    | 5387/10000 [00:14<00:12, 379.80it/s]
 54%|█████▎    | 5355/10000 [00:14<00:12, 380.34it/s]
 55%|█████▍    | 5458/10000 [00:14<00:12, 378.32it/s]
 54%|█████▍    | 5401/10000 [00:14<00:12, 377.50it/s]
 54%|█████▍    | 5426/10000 [00:14<00:12, 372.55it/s]
 54%|█████▍    | 5394/10000 [00:14<00:12, 376.84it/s]
 55%|█████▌    | 5501/10000 [00:14<00:11, 392.47it/s]
 54%|█████▍    | 5441/10000 [00:14<00:12, 372.50it/s]
 55%|█████▍    | 5465/10000 [00:14<00:12, 376.69it/s]
 54%|█████▍    | 5433/10000 [00:14<00:12, 380.45it/s]
 55%|█████▌    | 5541/10000 [00:14<00:11, 383.45it/s]
 55%|█████▍    | 5486/10000 [00:14<00:11, 394.40it/s]
 55%|█████▌    | 5504/10000 [00:14<00:11, 376.53it/s]
 55%|█████▍    | 5472/10000 [00:14<00:12, 366.93it/s]
 56%|█████▌    | 5583/10000 [00:14<00:11, 383.19it/s]
 55%|█████▌    | 5526/10000 [00:14<00:11, 382.49it/s]
 55%|█████▌    | 5547/10000 [00:14<00:11, 379.56it/s]
 55%|█████▌    | 5516/10000 [00:14<00:11, 377.36it/s]
 56%|█████▌    | 5622/10000 [00:14<00:11, 383.44it/s]
 56%|█████▌    | 5566/10000 [00:14<00:11, 379.64it/s]
 56%|█████▌    | 5587/10000 [00:14<00:11, 372.07it/s]
 56%|█████▌    | 5555/10000 [00:14<00:11, 377.38it/s]
 57%|█████▋    | 5662/10000 [00:14<00:11, 387.88it/s]
 56%|█████▌    | 5605/10000 [00:14<00:11, 380.63it/s]
 56%|█████▋    | 5626/10000 [00:14<00:11, 375.09it/s]
 56%|█████▌    | 5593/10000 [00:14<00:11, 376.96it/s]
 57%|█████▋    | 5702/10000 [00:14<00:11, 385.34it/s]
 56%|█████▋    | 5645/10000 [00:14<00:11, 382.20it/s]
 57%|█████▋    | 5669/10000 [00:15<00:11, 380.43it/s]
 56%|█████▋    | 5634/10000 [00:15<00:11, 382.81it/s]
 57%|█████▋    | 5742/10000 [00:15<00:10, 387.68it/s]
 57%|█████▋    | 5686/10000 [00:15<00:11, 378.23it/s]
 57%|█████▋    | 5712/10000 [00:15<00:11, 382.47it/s]
 57%|█████▋    | 5673/10000 [00:15<00:11, 384.35it/s]
 58%|█████▊    | 5783/10000 [00:15<00:11, 382.73it/s]
 57%|█████▋    | 5727/10000 [00:15<00:11, 387.21it/s]
 58%|█████▊    | 5753/10000 [00:15<00:10, 389.25it/s]
 57%|█████▋    | 5716/10000 [00:15<00:11, 384.15it/s]
 58%|█████▊    | 5822/10000 [00:15<00:11, 376.73it/s]
 58%|█████▊    | 5770/10000 [00:15<00:10, 399.43it/s]
 58%|█████▊    | 5793/10000 [00:15<00:10, 392.02it/s]
 58%|█████▊    | 5760/10000 [00:15<00:10, 391.92it/s]
 59%|█████▊    | 5865/10000 [00:15<00:10, 391.71it/s]
 58%|█████▊    | 5811/10000 [00:15<00:11, 380.68it/s]
 58%|█████▊    | 5833/10000 [00:15<00:10, 394.20it/s]
 58%|█████▊    | 5800/10000 [00:15<00:10, 389.65it/s]
 59%|█████▉    | 5905/10000 [00:15<00:10, 378.35it/s]
 59%|█████▊    | 5851/10000 [00:15<00:10, 386.14it/s]
 59%|█████▊    | 5873/10000 [00:15<00:10, 385.24it/s]
 58%|█████▊    | 5839/10000 [00:15<00:11, 377.00it/s]
 59%|█████▉    | 5947/10000 [00:15<00:10, 375.73it/s]
 59%|█████▉    | 5892/10000 [00:15<00:10, 383.72it/s]
 59%|█████▉    | 5913/10000 [00:15<00:10, 384.92it/s]
 59%|█████▉    | 5881/10000 [00:15<00:10, 389.13it/s]
 60%|█████▉    | 5990/10000 [00:15<00:10, 388.24it/s]
 59%|█████▉    | 5931/10000 [00:15<00:11, 367.54it/s]
 60%|█████▉    | 5952/10000 [00:15<00:10, 372.80it/s]
 59%|█████▉    | 5921/10000 [00:15<00:10, 390.26it/s]
 60%|██████    | 6029/10000 [00:15<00:10, 388.11it/s]
 60%|█████▉    | 5970/10000 [00:15<00:10, 371.10it/s]
 60%|█████▉    | 5994/10000 [00:15<00:10, 381.14it/s]
 60%|█████▉    | 5961/10000 [00:15<00:10, 377.75it/s]
 61%|██████    | 6071/10000 [00:15<00:10, 391.10it/s]
 60%|██████    | 6011/10000 [00:15<00:10, 381.80it/s]
 60%|██████    | 6036/10000 [00:15<00:10, 383.68it/s]
 60%|██████    | 6003/10000 [00:16<00:10, 379.70it/s]
 61%|██████    | 6111/10000 [00:16<00:10, 387.91it/s]
 60%|██████    | 6050/10000 [00:15<00:10, 377.80it/s]
 61%|██████    | 6075/10000 [00:16<00:10, 378.53it/s]
 62%|██████▏   | 6150/10000 [00:16<00:10, 382.30it/s]
 60%|██████    | 6045/10000 [00:16<00:10, 379.31it/s]
 61%|██████    | 6088/10000 [00:16<00:10, 376.71it/s]
 61%|██████    | 6113/10000 [00:16<00:10, 377.26it/s]
 61%|██████    | 6084/10000 [00:16<00:10, 366.97it/s]
 62%|██████▏   | 6193/10000 [00:16<00:10, 378.07it/s]
 61%|██████▏   | 6126/10000 [00:16<00:10, 369.05it/s]
 62%|██████▏   | 6152/10000 [00:16<00:10, 373.72it/s]
 61%|██████    | 6123/10000 [00:16<00:10, 368.04it/s]
 62%|██████▏   | 6236/10000 [00:16<00:09, 387.39it/s]
 62%|██████▏   | 6165/10000 [00:16<00:10, 371.00it/s]
 62%|██████▏   | 6191/10000 [00:16<00:10, 377.59it/s]
 63%|██████▎   | 6275/10000 [00:16<00:09, 385.29it/s]
 62%|██████▏   | 6164/10000 [00:16<00:10, 371.11it/s]
 62%|██████▏   | 6208/10000 [00:16<00:09, 388.04it/s]
 62%|██████▏   | 6231/10000 [00:16<00:09, 383.87it/s]
 63%|██████▎   | 6314/10000 [00:16<00:09, 385.63it/s]
 62%|██████▏   | 6202/10000 [00:16<00:10, 369.10it/s]
 62%|██████▏   | 6248/10000 [00:16<00:10, 374.19it/s]
 63%|██████▎   | 6270/10000 [00:16<00:09, 382.39it/s]
 64%|██████▎   | 6353/10000 [00:16<00:09, 380.98it/s]
 62%|██████▏   | 6244/10000 [00:16<00:09, 379.77it/s]
 63%|██████▎   | 6294/10000 [00:16<00:09, 384.79it/s]
 63%|██████▎   | 6309/10000 [00:16<00:09, 370.80it/s]
 64%|██████▍   | 6394/10000 [00:16<00:09, 381.31it/s]
 63%|██████▎   | 6283/10000 [00:16<00:09, 375.61it/s]
 63%|██████▎   | 6334/10000 [00:16<00:09, 388.82it/s]
 64%|██████▎   | 6350/10000 [00:16<00:09, 381.86it/s]
 63%|██████▎   | 6322/10000 [00:16<00:09, 378.75it/s]
 64%|██████▍   | 6433/10000 [00:16<00:09, 372.65it/s]
 64%|██████▎   | 6373/10000 [00:16<00:09, 381.86it/s]
 64%|██████▍   | 6389/10000 [00:16<00:09, 381.88it/s]
 65%|██████▍   | 6474/10000 [00:17<00:09, 383.32it/s]
 64%|██████▎   | 6361/10000 [00:16<00:09, 376.18it/s]
 64%|██████▍   | 6412/10000 [00:16<00:09, 371.83it/s]
 64%|██████▍   | 6428/10000 [00:17<00:09, 372.52it/s]
 64%|██████▍   | 6401/10000 [00:17<00:09, 378.83it/s]
 65%|██████▌   | 6513/10000 [00:17<00:09, 370.70it/s]
 64%|██████▍   | 6450/10000 [00:17<00:09, 372.89it/s]
 65%|██████▍   | 6466/10000 [00:17<00:09, 367.44it/s]
 64%|██████▍   | 6439/10000 [00:17<00:09, 376.82it/s]
 66%|██████▌   | 6552/10000 [00:17<00:09, 368.00it/s]
 65%|██████▍   | 6488/10000 [00:17<00:09, 371.59it/s]
 65%|██████▌   | 6505/10000 [00:17<00:09, 363.62it/s]
 65%|██████▍   | 6477/10000 [00:17<00:09, 372.50it/s]
 66%|██████▌   | 6592/10000 [00:17<00:09, 369.34it/s]
 65%|██████▌   | 6526/10000 [00:17<00:09, 372.71it/s]
 65%|██████▌   | 6542/10000 [00:17<00:09, 364.88it/s]
 65%|██████▌   | 6518/10000 [00:17<00:09, 381.91it/s]
 66%|██████▋   | 6631/10000 [00:17<00:08, 374.85it/s]
 66%|██████▌   | 6565/10000 [00:17<00:09, 376.46it/s]
 66%|██████▌   | 6579/10000 [00:17<00:09, 362.74it/s]
 66%|██████▌   | 6557/10000 [00:17<00:09, 372.21it/s]
 67%|██████▋   | 6673/10000 [00:17<00:08, 386.56it/s]
 66%|██████▌   | 6605/10000 [00:17<00:09, 375.18it/s]
 66%|██████▌   | 6616/10000 [00:17<00:09, 361.67it/s]
 66%|██████▌   | 6598/10000 [00:17<00:09, 373.68it/s]
 67%|██████▋   | 6712/10000 [00:17<00:08, 379.86it/s]
 66%|██████▋   | 6645/10000 [00:17<00:09, 370.47it/s]
 67%|██████▋   | 6662/10000 [00:17<00:08, 380.50it/s]
 66%|██████▋   | 6636/10000 [00:17<00:09, 371.03it/s]
 68%|██████▊   | 6751/10000 [00:17<00:08, 373.85it/s]
 67%|██████▋   | 6687/10000 [00:17<00:08, 384.54it/s]
 67%|██████▋   | 6701/10000 [00:17<00:08, 375.39it/s]
 67%|██████▋   | 6679/10000 [00:17<00:08, 386.19it/s]
 68%|██████▊   | 6790/10000 [00:17<00:08, 377.78it/s]
 67%|██████▋   | 6726/10000 [00:17<00:08, 385.75it/s]
 67%|██████▋   | 6741/10000 [00:17<00:08, 381.22it/s]
 67%|██████▋   | 6718/10000 [00:17<00:08, 378.49it/s]
 68%|██████▊   | 6766/10000 [00:17<00:08, 376.02it/s]
 68%|██████▊   | 6780/10000 [00:17<00:08, 369.94it/s]
 68%|██████▊   | 6757/10000 [00:18<00:08, 375.96it/s]
 68%|██████▊   | 6804/10000 [00:17<00:08, 376.32it/s]
 68%|██████▊   | 6798/10000 [00:18<00:08, 374.39it/s]
 68%|██████▊   | 6828/10000 [00:18<00:19, 164.82it/s]
 69%|██████▊   | 6863/10000 [00:18<00:16, 192.69it/s]
 68%|██████▊   | 6819/10000 [00:18<00:18, 169.95it/s]
 69%|██████▉   | 6907/10000 [00:18<00:13, 236.24it/s]
 68%|██████▊   | 6842/10000 [00:18<00:18, 168.73it/s]
 69%|██████▊   | 6855/10000 [00:18<00:15, 199.07it/s]
 68%|██████▊   | 6836/10000 [00:18<00:17, 176.61it/s]
 69%|██████▉   | 6879/10000 [00:18<00:15, 200.16it/s]
 70%|██████▉   | 6950/10000 [00:18<00:11, 273.82it/s]
 69%|██████▉   | 6889/10000 [00:18<00:13, 224.29it/s]
 69%|██████▊   | 6869/10000 [00:18<00:15, 200.67it/s]
 70%|██████▉   | 6990/10000 [00:18<00:09, 301.59it/s]
 69%|██████▉   | 6916/10000 [00:18<00:13, 229.51it/s]
 69%|██████▉   | 6927/10000 [00:18<00:12, 255.94it/s]
 69%|██████▉   | 6909/10000 [00:18<00:13, 235.79it/s]
 70%|███████   | 7028/10000 [00:18<00:09, 316.06it/s]
 70%|██████▉   | 6958/10000 [00:18<00:11, 265.56it/s]
 70%|██████▉   | 6971/10000 [00:18<00:10, 297.40it/s]
 70%|██████▉   | 6951/10000 [00:18<00:11, 270.34it/s]
 71%|███████   | 7065/10000 [00:19<00:08, 327.02it/s]
 70%|██████▉   | 6999/10000 [00:18<00:10, 291.22it/s]
 70%|███████   | 7008/10000 [00:19<00:09, 313.43it/s]
 70%|██████▉   | 6991/10000 [00:19<00:10, 293.72it/s]
 71%|███████   | 7103/10000 [00:19<00:08, 340.22it/s]
 70%|███████   | 7035/10000 [00:19<00:09, 304.18it/s]
 70%|███████   | 7045/10000 [00:19<00:09, 322.15it/s]
 70%|███████   | 7030/10000 [00:19<00:09, 316.82it/s]
 71%|███████▏  | 7140/10000 [00:19<00:08, 347.28it/s]
 71%|███████   | 7078/10000 [00:19<00:08, 335.92it/s]
 71%|███████   | 7083/10000 [00:19<00:08, 337.26it/s]
 71%|███████   | 7069/10000 [00:19<00:08, 326.40it/s]
 72%|███████▏  | 7179/10000 [00:19<00:07, 358.36it/s]
 71%|███████   | 7116/10000 [00:19<00:08, 344.57it/s]
 71%|███████   | 7123/10000 [00:19<00:08, 346.78it/s]
 71%|███████   | 7108/10000 [00:19<00:08, 342.42it/s]
 72%|███████▏  | 7218/10000 [00:19<00:07, 365.93it/s]
 72%|███████▏  | 7154/10000 [00:19<00:08, 349.35it/s]
 72%|███████▏  | 7163/10000 [00:19<00:07, 355.64it/s]
 71%|███████▏  | 7149/10000 [00:19<00:08, 353.55it/s]
 73%|███████▎  | 7256/10000 [00:19<00:07, 360.72it/s]
 72%|███████▏  | 7193/10000 [00:19<00:07, 353.42it/s]
 72%|███████▏  | 7203/10000 [00:19<00:07, 367.12it/s]
 72%|███████▏  | 7187/10000 [00:19<00:07, 355.60it/s]
 73%|███████▎  | 7293/10000 [00:19<00:07, 346.86it/s]
 72%|███████▏  | 7234/10000 [00:19<00:07, 365.98it/s]
 72%|███████▏  | 7241/10000 [00:19<00:07, 363.23it/s]
 72%|███████▏  | 7225/10000 [00:19<00:07, 359.82it/s]
 73%|███████▎  | 7329/10000 [00:19<00:07, 346.07it/s]
 73%|███████▎  | 7273/10000 [00:19<00:07, 364.07it/s]
 73%|███████▎  | 7279/10000 [00:19<00:07, 364.33it/s]
 73%|███████▎  | 7266/10000 [00:19<00:07, 364.12it/s]
 74%|███████▎  | 7366/10000 [00:19<00:07, 352.56it/s]
 73%|███████▎  | 7311/10000 [00:19<00:07, 367.72it/s]
 73%|███████▎  | 7317/10000 [00:19<00:07, 366.93it/s]
 73%|███████▎  | 7304/10000 [00:19<00:07, 356.98it/s]
 74%|███████▍  | 7405/10000 [00:19<00:07, 362.65it/s]
 73%|███████▎  | 7349/10000 [00:19<00:07, 362.91it/s]
 74%|███████▎  | 7356/10000 [00:19<00:07, 364.45it/s]
 73%|███████▎  | 7346/10000 [00:19<00:07, 373.94it/s]
 74%|███████▍  | 7442/10000 [00:20<00:07, 355.14it/s]
 74%|███████▍  | 7388/10000 [00:19<00:07, 362.35it/s]
 74%|███████▍  | 7393/10000 [00:20<00:07, 361.04it/s]
 74%|███████▍  | 7384/10000 [00:20<00:07, 363.71it/s]
 75%|███████▍  | 7479/10000 [00:20<00:07, 356.02it/s]
 74%|███████▍  | 7426/10000 [00:20<00:07, 363.60it/s]
 74%|███████▍  | 7432/10000 [00:20<00:07, 360.35it/s]
 74%|███████▍  | 7424/10000 [00:20<00:06, 372.44it/s]
 75%|███████▌  | 7518/10000 [00:20<00:06, 365.50it/s]
 75%|███████▍  | 7466/10000 [00:20<00:06, 371.30it/s]
 75%|███████▍  | 7470/10000 [00:20<00:06, 365.94it/s]
 75%|███████▍  | 7462/10000 [00:20<00:06, 363.89it/s]
 76%|███████▌  | 7559/10000 [00:20<00:06, 377.59it/s]
 75%|███████▌  | 7505/10000 [00:20<00:06, 371.22it/s]
 75%|███████▌  | 7507/10000 [00:20<00:06, 359.36it/s]
 75%|███████▍  | 7499/10000 [00:20<00:06, 363.71it/s]
 76%|███████▌  | 7597/10000 [00:20<00:06, 378.02it/s]
 75%|███████▌  | 7545/10000 [00:20<00:06, 379.17it/s]
 75%|███████▌  | 7549/10000 [00:20<00:06, 376.69it/s]
 75%|███████▌  | 7536/10000 [00:20<00:06, 363.67it/s]
 76%|███████▋  | 7635/10000 [00:20<00:06, 370.55it/s]
 76%|███████▌  | 7584/10000 [00:20<00:06, 375.49it/s]
 76%|███████▌  | 7587/10000 [00:20<00:06, 375.80it/s]
 76%|███████▌  | 7577/10000 [00:20<00:06, 376.64it/s]
 77%|███████▋  | 7678/10000 [00:20<00:05, 387.07it/s]
 76%|███████▌  | 7623/10000 [00:20<00:06, 379.05it/s]
 76%|███████▋  | 7625/10000 [00:20<00:06, 366.49it/s]
 76%|███████▌  | 7615/10000 [00:20<00:06, 376.65it/s]
 77%|███████▋  | 7717/10000 [00:20<00:06, 379.69it/s]
 77%|███████▋  | 7664/10000 [00:20<00:06, 378.80it/s]
 77%|███████▋  | 7666/10000 [00:20<00:06, 374.75it/s]
 77%|███████▋  | 7653/10000 [00:20<00:06, 375.43it/s]
 78%|███████▊  | 7758/10000 [00:20<00:06, 371.22it/s]
 77%|███████▋  | 7703/10000 [00:20<00:06, 373.30it/s]
 77%|███████▋  | 7704/10000 [00:20<00:06, 375.37it/s]
 77%|███████▋  | 7694/10000 [00:20<00:05, 384.70it/s]
 78%|███████▊  | 7797/10000 [00:20<00:05, 375.25it/s]
 77%|███████▋  | 7744/10000 [00:20<00:05, 383.77it/s]
 77%|███████▋  | 7743/10000 [00:20<00:06, 375.50it/s]
 77%|███████▋  | 7733/10000 [00:21<00:05, 380.92it/s]
 78%|███████▊  | 7837/10000 [00:21<00:05, 370.97it/s]
 78%|███████▊  | 7781/10000 [00:21<00:05, 373.60it/s]
 78%|███████▊  | 7783/10000 [00:21<00:05, 370.75it/s]
 78%|███████▊  | 7772/10000 [00:21<00:05, 378.62it/s]
 79%|███████▉  | 7875/10000 [00:21<00:05, 373.31it/s]
 78%|███████▊  | 7823/10000 [00:21<00:05, 378.41it/s]
 78%|███████▊  | 7819/10000 [00:21<00:05, 366.25it/s]
 78%|███████▊  | 7810/10000 [00:21<00:05, 370.96it/s]
 79%|███████▉  | 7913/10000 [00:21<00:05, 372.87it/s]
 79%|███████▊  | 7862/10000 [00:21<00:05, 380.60it/s]
 79%|███████▊  | 7859/10000 [00:21<00:05, 374.25it/s]
 78%|███████▊  | 7848/10000 [00:21<00:05, 373.14it/s]
 80%|███████▉  | 7953/10000 [00:21<00:05, 379.09it/s]
 79%|███████▉  | 7897/10000 [00:21<00:05, 371.77it/s]
 79%|███████▉  | 7901/10000 [00:21<00:05, 370.65it/s]
 79%|███████▉  | 7887/10000 [00:21<00:05, 376.27it/s]
 80%|███████▉  | 7991/10000 [00:21<00:05, 369.74it/s]
 79%|███████▉  | 7936/10000 [00:21<00:05, 368.74it/s]
 79%|███████▉  | 7942/10000 [00:21<00:05, 368.56it/s]
 79%|███████▉  | 7925/10000 [00:21<00:05, 366.56it/s]
 80%|████████  | 8031/10000 [00:21<00:05, 376.95it/s]
 80%|███████▉  | 7978/10000 [00:21<00:05, 382.77it/s]
 80%|███████▉  | 7981/10000 [00:21<00:05, 374.36it/s]
 80%|███████▉  | 7963/10000 [00:21<00:05, 369.33it/s]
 81%|████████  | 8069/10000 [00:21<00:05, 364.42it/s]
 80%|████████  | 8017/10000 [00:21<00:05, 373.58it/s]
 80%|████████  | 8020/10000 [00:21<00:05, 377.45it/s]
 80%|████████  | 8000/10000 [00:21<00:05, 367.56it/s]
 81%|████████  | 8109/10000 [00:21<00:05, 373.75it/s]
 81%|████████  | 8058/10000 [00:21<00:05, 371.67it/s]
 81%|████████  | 8055/10000 [00:21<00:05, 366.48it/s]
 80%|████████  | 8040/10000 [00:21<00:05, 368.69it/s]
 81%|████████▏ | 8147/10000 [00:21<00:04, 370.87it/s]
 81%|████████  | 8092/10000 [00:21<00:05, 365.01it/s]
 81%|████████  | 8097/10000 [00:21<00:05, 363.58it/s]
 81%|████████  | 8080/10000 [00:21<00:05, 373.71it/s]
 82%|████████▏ | 8190/10000 [00:22<00:04, 376.82it/s]
 81%|████████▏ | 8130/10000 [00:22<00:05, 368.97it/s]
 81%|████████▏ | 8138/10000 [00:21<00:05, 368.93it/s]
 81%|████████  | 8118/10000 [00:22<00:05, 370.40it/s]
 82%|████████▏ | 8230/10000 [00:22<00:04, 372.29it/s]
 82%|████████▏ | 8169/10000 [00:22<00:05, 366.16it/s]
 82%|████████▏ | 8177/10000 [00:22<00:04, 369.86it/s]
 82%|████████▏ | 8158/10000 [00:22<00:04, 370.86it/s]
 83%|████████▎ | 8268/10000 [00:22<00:04, 374.23it/s]
 82%|████████▏ | 8212/10000 [00:22<00:04, 370.10it/s]
 82%|████████▏ | 8222/10000 [00:22<00:04, 384.94it/s]
 82%|████████▏ | 8201/10000 [00:22<00:04, 386.98it/s]
 83%|████████▎ | 8306/10000 [00:22<00:04, 355.05it/s]
 82%|████████▎ | 8250/10000 [00:22<00:04, 367.84it/s]
 83%|████████▎ | 8261/10000 [00:22<00:04, 377.15it/s]
 82%|████████▏ | 8240/10000 [00:22<00:04, 362.72it/s]
 84%|████████▎ | 8350/10000 [00:22<00:04, 377.92it/s]
 83%|████████▎ | 8287/10000 [00:22<00:04, 364.09it/s]
 83%|████████▎ | 8299/10000 [00:22<00:04, 361.33it/s]
 83%|████████▎ | 8277/10000 [00:22<00:04, 362.88it/s]
 83%|████████▎ | 8324/10000 [00:22<00:04, 361.12it/s]
 84%|████████▍ | 8389/10000 [00:22<00:04, 367.53it/s]
 83%|████████▎ | 8339/10000 [00:22<00:04, 372.05it/s]
 83%|████████▎ | 8315/10000 [00:22<00:04, 361.61it/s]
 84%|████████▎ | 8362/10000 [00:22<00:04, 365.49it/s]
 84%|████████▍ | 8429/10000 [00:22<00:04, 370.56it/s]
 84%|████████▍ | 8379/10000 [00:22<00:04, 379.82it/s]
 84%|████████▎ | 8356/10000 [00:22<00:04, 375.11it/s]
 84%|████████▍ | 8404/10000 [00:22<00:04, 381.03it/s]
 85%|████████▍ | 8467/10000 [00:22<00:04, 365.73it/s]
 84%|████████▍ | 8418/10000 [00:22<00:04, 370.16it/s]
 84%|████████▍ | 8394/10000 [00:22<00:04, 372.77it/s]
 84%|████████▍ | 8443/10000 [00:22<00:04, 370.34it/s]
 85%|████████▌ | 8510/10000 [00:22<00:03, 374.57it/s]
 85%|████████▍ | 8456/10000 [00:22<00:04, 369.32it/s]
 84%|████████▍ | 8432/10000 [00:22<00:04, 368.97it/s]
 85%|████████▍ | 8481/10000 [00:22<00:04, 369.50it/s]
 85%|████████▌ | 8549/10000 [00:23<00:03, 377.46it/s]
 85%|████████▍ | 8496/10000 [00:22<00:04, 369.04it/s]
 85%|████████▍ | 8469/10000 [00:23<00:04, 367.05it/s]
 85%|████████▌ | 8519/10000 [00:23<00:04, 368.98it/s]
 86%|████████▌ | 8587/10000 [00:23<00:03, 377.61it/s]
 85%|████████▌ | 8537/10000 [00:23<00:03, 368.55it/s]
 85%|████████▌ | 8510/10000 [00:23<00:03, 373.85it/s]
 86%|████████▌ | 8557/10000 [00:23<00:03, 371.86it/s]
 86%|████████▋ | 8626/10000 [00:23<00:03, 380.27it/s]
 86%|████████▌ | 8579/10000 [00:23<00:03, 381.28it/s]
 85%|████████▌ | 8548/10000 [00:23<00:03, 369.95it/s]
 87%|████████▋ | 8667/10000 [00:23<00:03, 385.18it/s]
 86%|████████▌ | 8595/10000 [00:23<00:03, 360.66it/s]
 86%|████████▌ | 8619/10000 [00:23<00:03, 386.15it/s]
 86%|████████▌ | 8587/10000 [00:23<00:03, 375.23it/s]
 87%|████████▋ | 8708/10000 [00:23<00:03, 391.49it/s]
 86%|████████▋ | 8636/10000 [00:23<00:03, 362.92it/s]
 87%|████████▋ | 8658/10000 [00:23<00:03, 384.84it/s]
 86%|████████▋ | 8626/10000 [00:23<00:03, 378.67it/s]
 87%|████████▋ | 8748/10000 [00:23<00:03, 379.20it/s]
 87%|████████▋ | 8681/10000 [00:23<00:03, 386.56it/s]
 87%|████████▋ | 8697/10000 [00:23<00:03, 382.00it/s]
 87%|████████▋ | 8665/10000 [00:23<00:03, 381.92it/s]
 88%|████████▊ | 8787/10000 [00:23<00:03, 381.79it/s]
 87%|████████▋ | 8720/10000 [00:23<00:03, 368.30it/s]
 87%|████████▋ | 8736/10000 [00:23<00:03, 368.81it/s]
 87%|████████▋ | 8704/10000 [00:23<00:03, 383.83it/s]
 88%|████████▊ | 8826/10000 [00:23<00:03, 368.29it/s]
 88%|████████▊ | 8759/10000 [00:23<00:03, 373.27it/s]
 88%|████████▊ | 8773/10000 [00:23<00:03, 367.64it/s]
 87%|████████▋ | 8743/10000 [00:23<00:03, 375.90it/s]
 89%|████████▊ | 8868/10000 [00:23<00:02, 380.35it/s]
 88%|████████▊ | 8798/10000 [00:23<00:03, 377.41it/s]
 88%|████████▊ | 8810/10000 [00:23<00:03, 361.91it/s]
 88%|████████▊ | 8783/10000 [00:23<00:03, 381.85it/s]
 88%|████████▊ | 8836/10000 [00:23<00:03, 373.70it/s]
 89%|████████▉ | 8908/10000 [00:23<00:02, 375.96it/s]
 88%|████████▊ | 8849/10000 [00:23<00:03, 369.73it/s]
 88%|████████▊ | 8822/10000 [00:23<00:03, 357.05it/s]
 89%|████████▉ | 8948/10000 [00:24<00:02, 382.56it/s]
 89%|████████▉ | 8876/10000 [00:24<00:03, 369.66it/s]
 89%|████████▉ | 8890/10000 [00:23<00:02, 378.60it/s]
 89%|████████▊ | 8863/10000 [00:24<00:03, 368.62it/s]
 90%|████████▉ | 8987/10000 [00:24<00:02, 373.82it/s]
 89%|████████▉ | 8919/10000 [00:24<00:02, 377.22it/s]
 89%|████████▉ | 8928/10000 [00:24<00:02, 374.34it/s]
 89%|████████▉ | 8903/10000 [00:24<00:02, 376.90it/s]
 90%|█████████ | 9026/10000 [00:24<00:02, 372.14it/s]
 90%|████████▉ | 8966/10000 [00:24<00:02, 369.06it/s]
 90%|████████▉ | 8957/10000 [00:24<00:02, 368.85it/s]
 89%|████████▉ | 8941/10000 [00:24<00:02, 369.86it/s]
 90%|█████████ | 9006/10000 [00:24<00:02, 376.86it/s]
 91%|█████████ | 9069/10000 [00:24<00:02, 375.60it/s]
 90%|████████▉ | 8998/10000 [00:24<00:02, 369.10it/s]
 90%|████████▉ | 8979/10000 [00:24<00:02, 368.62it/s]
 90%|█████████ | 9044/10000 [00:24<00:02, 376.82it/s]
 91%|█████████ | 9107/10000 [00:24<00:02, 372.76it/s]
 90%|█████████ | 9040/10000 [00:24<00:02, 370.99it/s]
 90%|█████████ | 9020/10000 [00:24<00:02, 380.26it/s]
 91%|█████████ | 9082/10000 [00:24<00:02, 376.51it/s]
 91%|█████████▏| 9145/10000 [00:24<00:02, 372.79it/s]
 91%|█████████ | 9080/10000 [00:24<00:02, 375.38it/s]
 91%|█████████ | 9059/10000 [00:24<00:02, 371.15it/s]
 91%|█████████ | 9120/10000 [00:24<00:02, 371.05it/s]
 92%|█████████▏| 9183/10000 [00:24<00:02, 371.51it/s]
 91%|█████████ | 9118/10000 [00:24<00:02, 362.21it/s]
 91%|█████████ | 9097/10000 [00:24<00:02, 368.70it/s]
 92%|█████████▏| 9158/10000 [00:24<00:02, 369.19it/s]
 92%|█████████▏| 9223/10000 [00:24<00:02, 365.80it/s]
 92%|█████████▏| 9161/10000 [00:24<00:02, 381.25it/s]
 91%|█████████▏| 9134/10000 [00:24<00:02, 366.24it/s]
 92%|█████████▏| 9195/10000 [00:24<00:02, 364.63it/s]
 93%|█████████▎| 9268/10000 [00:24<00:01, 385.75it/s]
 92%|█████████▏| 9200/10000 [00:24<00:02, 373.35it/s]
 92%|█████████▏| 9175/10000 [00:24<00:02, 378.76it/s]
 92%|█████████▏| 9237/10000 [00:24<00:02, 370.17it/s]
 93%|█████████▎| 9309/10000 [00:25<00:01, 374.96it/s]
 92%|█████████▏| 9242/10000 [00:25<00:02, 377.34it/s]
 92%|█████████▏| 9213/10000 [00:25<00:02, 368.37it/s]
 93%|█████████▎| 9279/10000 [00:25<00:01, 384.29it/s]
 93%|█████████▎| 9348/10000 [00:25<00:01, 377.70it/s]
 93%|█████████▎| 9282/10000 [00:25<00:01, 383.17it/s]
 93%|█████████▎| 9253/10000 [00:25<00:01, 377.21it/s]
 93%|█████████▎| 9318/10000 [00:25<00:01, 385.31it/s]
 94%|█████████▍| 9387/10000 [00:25<00:01, 369.97it/s]
 93%|█████████▎| 9321/10000 [00:25<00:01, 384.97it/s]
 93%|█████████▎| 9293/10000 [00:25<00:01, 383.05it/s]
 94%|█████████▎| 9357/10000 [00:25<00:01, 379.81it/s]
 94%|█████████▍| 9431/10000 [00:25<00:01, 388.91it/s]
 94%|█████████▎| 9360/10000 [00:25<00:01, 374.11it/s]
 93%|█████████▎| 9332/10000 [00:25<00:01, 380.56it/s]
 94%|█████████▍| 9397/10000 [00:25<00:01, 384.90it/s]
 95%|█████████▍| 9471/10000 [00:25<00:01, 382.14it/s]
 94%|█████████▍| 9398/10000 [00:25<00:01, 375.31it/s]
 94%|█████████▎| 9371/10000 [00:25<00:01, 369.92it/s]
 94%|█████████▍| 9436/10000 [00:25<00:01, 380.80it/s]
 94%|█████████▍| 9436/10000 [00:25<00:01, 374.79it/s]
 95%|█████████▌| 9510/10000 [00:25<00:01, 373.88it/s]
 94%|█████████▍| 9414/10000 [00:25<00:01, 386.43it/s]
 95%|█████████▍| 9476/10000 [00:25<00:01, 376.73it/s]
 95%|█████████▍| 9476/10000 [00:25<00:01, 381.05it/s]
 95%|█████████▌| 9549/10000 [00:25<00:01, 368.29it/s]
 95%|█████████▍| 9453/10000 [00:25<00:01, 386.42it/s]
 95%|█████████▌| 9514/10000 [00:25<00:01, 374.37it/s]
 95%|█████████▌| 9515/10000 [00:25<00:01, 376.01it/s]
 96%|█████████▌| 9589/10000 [00:25<00:01, 371.84it/s]
 95%|█████████▍| 9492/10000 [00:25<00:01, 379.41it/s]
 96%|█████████▌| 9552/10000 [00:25<00:01, 372.93it/s]
 96%|█████████▌| 9553/10000 [00:25<00:01, 372.04it/s]
 96%|█████████▋| 9628/10000 [00:25<00:00, 376.64it/s]
 95%|█████████▌| 9531/10000 [00:25<00:01, 367.52it/s]
 96%|█████████▌| 9591/10000 [00:25<00:01, 377.18it/s]
 97%|█████████▋| 9667/10000 [00:25<00:00, 379.41it/s]
 96%|█████████▌| 9591/10000 [00:25<00:01, 368.69it/s]
 96%|█████████▌| 9570/10000 [00:25<00:01, 372.95it/s]
 96%|█████████▋| 9629/10000 [00:25<00:00, 377.52it/s]
 96%|█████████▋| 9631/10000 [00:26<00:00, 374.64it/s]
 97%|█████████▋| 9705/10000 [00:26<00:00, 374.69it/s]
 96%|█████████▌| 9609/10000 [00:26<00:01, 376.83it/s]
 97%|█████████▋| 9667/10000 [00:26<00:00, 373.28it/s]
 97%|█████████▋| 9745/10000 [00:26<00:00, 380.44it/s]
 97%|█████████▋| 9670/10000 [00:26<00:00, 374.21it/s]
 96%|█████████▋| 9647/10000 [00:26<00:00, 376.42it/s]
 97%|█████████▋| 9706/10000 [00:26<00:00, 365.70it/s]
 97%|█████████▋| 9708/10000 [00:26<00:00, 367.28it/s]
 98%|█████████▊| 9784/10000 [00:26<00:00, 369.22it/s]
 97%|█████████▋| 9687/10000 [00:26<00:00, 372.30it/s]
 97%|█████████▋| 9746/10000 [00:26<00:00, 375.50it/s]
 98%|█████████▊| 9824/10000 [00:26<00:00, 377.31it/s]
 97%|█████████▋| 9746/10000 [00:26<00:00, 362.22it/s]
 97%|█████████▋| 9726/10000 [00:26<00:00, 370.24it/s]
 98%|█████████▊| 9784/10000 [00:26<00:00, 337.19it/s]
 98%|█████████▊| 9783/10000 [00:26<00:00, 310.21it/s]
 99%|█████████▊| 9862/10000 [00:26<00:00, 292.50it/s]
 98%|█████████▊| 9764/10000 [00:26<00:00, 291.19it/s]
 98%|█████████▊| 9819/10000 [00:26<00:00, 287.11it/s]
 98%|█████████▊| 9816/10000 [00:26<00:00, 292.91it/s]
 99%|█████████▉| 9895/10000 [00:26<00:00, 274.63it/s]
 98%|█████████▊| 9796/10000 [00:26<00:00, 279.44it/s]
 98%|█████████▊| 9850/10000 [00:26<00:00, 264.54it/s]
 98%|█████████▊| 9847/10000 [00:26<00:00, 267.91it/s]
 99%|█████████▉| 9925/10000 [00:26<00:00, 272.63it/s]
 98%|█████████▊| 9826/10000 [00:26<00:00, 268.60it/s]
 99%|█████████▉| 9878/10000 [00:26<00:00, 263.33it/s]
 99%|█████████▉| 9875/10000 [00:26<00:00, 262.19it/s]
100%|█████████▉| 9954/10000 [00:26<00:00, 274.35it/s]
 99%|█████████▊| 9855/10000 [00:26<00:00, 256.91it/s]
 99%|█████████▉| 9906/10000 [00:26<00:00, 260.25it/s]
 99%|█████████▉| 9902/10000 [00:27<00:00, 258.94it/s]
100%|█████████▉| 9983/10000 [00:27<00:00, 270.85it/s]
 99%|█████████▉| 9882/10000 [00:27<00:00, 256.20it/s]
 99%|█████████▉| 9935/10000 [00:27<00:00, 267.11it/s]
100%|██████████| 10000/10000 [00:27<00:00, 368.45it/s]

 99%|█████████▉| 9929/10000 [00:27<00:00, 255.83it/s]
 99%|█████████▉| 9923/10000 [00:27<00:00, 296.17it/s]
100%|█████████▉| 9963/10000 [00:27<00:00, 255.09it/s]
100%|█████████▉| 9956/10000 [00:27<00:00, 258.33it/s]
100%|█████████▉| 9954/10000 [00:27<00:00, 292.31it/s]
100%|█████████▉| 9991/10000 [00:27<00:00, 255.41it/s]
100%|█████████▉| 9985/10000 [00:27<00:00, 265.89it/s]
100%|██████████| 10000/10000 [00:27<00:00, 366.18it/s]

100%|█████████▉| 9984/10000 [00:27<00:00, 288.94it/s]
100%|██████████| 10000/10000 [00:27<00:00, 364.91it/s]
Detected kernel version 5.4.0, which is below the recommended minimum of 5.5.0; this can cause the process to hang. It is recommended to upgrade the kernel to the minimum version or higher.

100%|██████████| 10000/10000 [00:27<00:00, 364.65it/s]
/nlp/scr/ram1998/miniconda3/envs/pyreft_dev/lib/python3.12/site-packages/transformers/data/data_collator.py:656: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:278.)
  batch["labels"] = torch.tensor(batch["labels"], dtype=torch.int64)
/nlp/scr/ram1998/miniconda3/envs/pyreft_dev/lib/python3.12/site-packages/transformers/data/data_collator.py:656: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:278.)
  batch["labels"] = torch.tensor(batch["labels"], dtype=torch.int64)
/nlp/scr/ram1998/miniconda3/envs/pyreft_dev/lib/python3.12/site-packages/transformers/data/data_collator.py:656: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:278.)
  batch["labels"] = torch.tensor(batch["labels"], dtype=torch.int64)
wandb: Using wandb-core as the SDK backend. Please refer to https://wandb.me/wandb-core for more information.
wandb: Currently logged in as: ramvenkat98. Use `wandb login --relogin` to force relogin
wandb: Tracking run with wandb version 0.18.3
wandb: Run data is saved locally in /juice2/scr2/ram1998/pyreft/examples/alpaca/wandb/run-20241118_080348-6fcnccrm
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run multigpu_reft_alpaca_example
wandb: ⭐️ View project at https://wandb.ai/ramvenkat98/huggingface
wandb: 🚀 View run at https://wandb.ai/ramvenkat98/huggingface/runs/6fcnccrm

  0%|          | 0/780 [00:00<?, ?it/s]/nlp/scr/ram1998/miniconda3/envs/pyreft_dev/lib/python3.12/site-packages/transformers/data/data_collator.py:656: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at ../torch/csrc/utils/tensor_new.cpp:278.)
  batch["labels"] = torch.tensor(batch["labels"], dtype=torch.int64)

  0%|          | 1/780 [00:05<1:06:01,  5.09s/it]
                                                 
{'loss': 1.3867, 'grad_norm': 0.5105997323989868, 'learning_rate': 8.333333333333333e-07, 'epoch': 0.01}

  0%|          | 1/780 [00:05<1:06:01,  5.09s/it]
  0%|          | 2/780 [00:08<53:55,  4.16s/it]  
                                               
{'loss': 1.2946, 'grad_norm': 0.5552780628204346, 'learning_rate': 1.6666666666666667e-06, 'epoch': 0.03}

  0%|          | 2/780 [00:08<53:55,  4.16s/it]
  0%|          | 3/780 [00:12<52:21,  4.04s/it]
                                               
{'loss': 1.3637, 'grad_norm': 0.5207421779632568, 'learning_rate': 2.5e-06, 'epoch': 0.04}

  0%|          | 3/780 [00:12<52:21,  4.04s/it]
  1%|          | 4/780 [00:16<51:33,  3.99s/it]
                                               
{'loss': 1.4488, 'grad_norm': 0.5216673016548157, 'learning_rate': 3.3333333333333333e-06, 'epoch': 0.05}

  1%|          | 4/780 [00:16<51:33,  3.99s/it]
  1%|          | 5/780 [00:19<48:40,  3.77s/it]
                                               
{'loss': 1.3873, 'grad_norm': 0.4617159962654114, 'learning_rate': 4.166666666666667e-06, 'epoch': 0.06}

  1%|          | 5/780 [00:19<48:40,  3.77s/it]
  1%|          | 6/780 [00:23<50:31,  3.92s/it]
                                               
{'loss': 1.2758, 'grad_norm': 0.43514004349708557, 'learning_rate': 5e-06, 'epoch': 0.08}

  1%|          | 6/780 [00:24<50:31,  3.92s/it]
  1%|          | 7/780 [00:27<50:18,  3.90s/it]
                                               
{'loss': 1.3714, 'grad_norm': 0.4948123097419739, 'learning_rate': 5.833333333333334e-06, 'epoch': 0.09}

  1%|          | 7/780 [00:27<50:18,  3.90s/it]
  1%|          | 8/780 [00:31<49:33,  3.85s/it]
                                               
{'loss': 1.3741, 'grad_norm': 0.46508583426475525, 'learning_rate': 6.666666666666667e-06, 'epoch': 0.1}

  1%|          | 8/780 [00:31<49:33,  3.85s/it]
  1%|          | 9/780 [00:35<48:30,  3.77s/it]
                                               
{'loss': 1.4058, 'grad_norm': 0.47556719183921814, 'learning_rate': 7.500000000000001e-06, 'epoch': 0.12}

  1%|          | 9/780 [00:35<48:30,  3.77s/it]
  1%|▏         | 10/780 [00:39<49:27,  3.85s/it]
                                                
{'loss': 1.369, 'grad_norm': 0.4888916313648224, 'learning_rate': 8.333333333333334e-06, 'epoch': 0.13}

  1%|▏         | 10/780 [00:39<49:27,  3.85s/it]
  1%|▏         | 11/780 [00:43<50:12,  3.92s/it]
                                                
{'loss': 1.3667, 'grad_norm': 0.45243629813194275, 'learning_rate': 9.166666666666666e-06, 'epoch': 0.14}

  1%|▏         | 11/780 [00:43<50:12,  3.92s/it]
  2%|▏         | 12/780 [00:47<49:39,  3.88s/it]
                                                
{'loss': 1.3654, 'grad_norm': 0.45621684193611145, 'learning_rate': 1e-05, 'epoch': 0.15}

  2%|▏         | 12/780 [00:47<49:39,  3.88s/it]
  2%|▏         | 13/780 [00:51<51:09,  4.00s/it]
                                                
{'loss': 1.4791, 'grad_norm': 0.5747900605201721, 'learning_rate': 1.0833333333333334e-05, 'epoch': 0.17}

  2%|▏         | 13/780 [00:51<51:09,  4.00s/it]
  2%|▏         | 14/780 [00:55<51:36,  4.04s/it]
                                                
{'loss': 1.2704, 'grad_norm': 0.4522598683834076, 'learning_rate': 1.1666666666666668e-05, 'epoch': 0.18}

  2%|▏         | 14/780 [00:55<51:36,  4.04s/it]
  2%|▏         | 15/780 [00:59<50:05,  3.93s/it]
                                                
{'loss': 1.3333, 'grad_norm': 0.4547404944896698, 'learning_rate': 1.25e-05, 'epoch': 0.19}

  2%|▏         | 15/780 [00:59<50:05,  3.93s/it]
  2%|▏         | 16/780 [01:02<49:00,  3.85s/it]
                                                
{'loss': 1.4359, 'grad_norm': 0.3769890069961548, 'learning_rate': 1.3333333333333333e-05, 'epoch': 0.2}

  2%|▏         | 16/780 [01:02<49:00,  3.85s/it]
  2%|▏         | 17/780 [01:06<48:48,  3.84s/it]
                                                
{'loss': 1.407, 'grad_norm': 0.38652050495147705, 'learning_rate': 1.416666666666667e-05, 'epoch': 0.22}

  2%|▏         | 17/780 [01:06<48:48,  3.84s/it]
  2%|▏         | 18/780 [01:10<48:01,  3.78s/it]
                                                
{'loss': 1.3347, 'grad_norm': 0.33126434683799744, 'learning_rate': 1.5000000000000002e-05, 'epoch': 0.23}

  2%|▏         | 18/780 [01:10<48:01,  3.78s/it]
  2%|▏         | 19/780 [01:14<48:46,  3.85s/it]
                                                
{'loss': 1.3496, 'grad_norm': 0.38700276613235474, 'learning_rate': 1.5833333333333333e-05, 'epoch': 0.24}

  2%|▏         | 19/780 [01:14<48:46,  3.85s/it]
  3%|▎         | 20/780 [01:18<49:20,  3.90s/it]
                                                
{'loss': 1.3581, 'grad_norm': 0.4132702052593231, 'learning_rate': 1.6666666666666667e-05, 'epoch': 0.26}

  3%|▎         | 20/780 [01:18<49:20,  3.90s/it]
  3%|▎         | 21/780 [01:22<50:08,  3.96s/it]
                                                
{'loss': 1.3417, 'grad_norm': 0.34609219431877136, 'learning_rate': 1.7500000000000002e-05, 'epoch': 0.27}

  3%|▎         | 21/780 [01:22<50:08,  3.96s/it]
  3%|▎         | 22/780 [01:26<49:30,  3.92s/it]
                                                
{'loss': 1.3881, 'grad_norm': 0.428962767124176, 'learning_rate': 1.8333333333333333e-05, 'epoch': 0.28}

  3%|▎         | 22/780 [01:26<49:30,  3.92s/it]
  3%|▎         | 23/780 [01:30<49:46,  3.95s/it]
                                                
{'loss': 1.3055, 'grad_norm': 0.29435983300209045, 'learning_rate': 1.916666666666667e-05, 'epoch': 0.29}

  3%|▎         | 23/780 [01:30<49:46,  3.95s/it]
  3%|▎         | 24/780 [01:34<49:43,  3.95s/it]
                                                
{'loss': 1.396, 'grad_norm': 0.33363255858421326, 'learning_rate': 2e-05, 'epoch': 0.31}

  3%|▎         | 24/780 [01:34<49:43,  3.95s/it]
  3%|▎         | 25/780 [01:38<49:19,  3.92s/it]
                                                
{'loss': 1.3473, 'grad_norm': 0.3057198226451874, 'learning_rate': 1.999991365731819e-05, 'epoch': 0.32}

  3%|▎         | 25/780 [01:38<49:19,  3.92s/it]
  3%|▎         | 26/780 [01:42<49:41,  3.95s/it]
                                                
{'loss': 1.3571, 'grad_norm': 0.26576781272888184, 'learning_rate': 1.999965463076377e-05, 'epoch': 0.33}

  3%|▎         | 26/780 [01:42<49:41,  3.95s/it]
  3%|▎         | 27/780 [01:45<48:17,  3.85s/it]
                                                
{'loss': 1.2929, 'grad_norm': 0.2823621928691864, 'learning_rate': 1.999922292480975e-05, 'epoch': 0.35}

  3%|▎         | 27/780 [01:45<48:17,  3.85s/it]
  4%|▎         | 28/780 [01:49<48:10,  3.84s/it]
                                                
{'loss': 1.3941, 'grad_norm': 0.24535207450389862, 'learning_rate': 1.999861854691106e-05, 'epoch': 0.36}

  4%|▎         | 28/780 [01:49<48:10,  3.84s/it]
  4%|▎         | 29/780 [01:53<47:25,  3.79s/it]
                                                
{'loss': 1.4042, 'grad_norm': 0.2799653112888336, 'learning_rate': 1.999784150750442e-05, 'epoch': 0.37}

  4%|▎         | 29/780 [01:53<47:25,  3.79s/it]
  4%|▍         | 30/780 [01:57<48:45,  3.90s/it]
                                                
{'loss': 1.3287, 'grad_norm': 0.34391969442367554, 'learning_rate': 1.9996891820008165e-05, 'epoch': 0.38}

  4%|▍         | 30/780 [01:57<48:45,  3.90s/it]
  4%|▍         | 31/780 [02:01<48:26,  3.88s/it]
                                                
{'loss': 1.3436, 'grad_norm': 0.2580704987049103, 'learning_rate': 1.9995769500822007e-05, 'epoch': 0.4}

  4%|▍         | 31/780 [02:01<48:26,  3.88s/it]
  4%|▍         | 32/780 [02:05<48:28,  3.89s/it]
                                                
{'loss': 1.3116, 'grad_norm': 0.26649346947669983, 'learning_rate': 1.999447456932676e-05, 'epoch': 0.41}

  4%|▍         | 32/780 [02:05<48:28,  3.89s/it]
  4%|▍         | 33/780 [02:08<47:37,  3.83s/it]
                                                
{'loss': 1.3502, 'grad_norm': 0.2992003560066223, 'learning_rate': 1.9993007047883988e-05, 'epoch': 0.42}

  4%|▍         | 33/780 [02:08<47:37,  3.83s/it]
  4%|▍         | 34/780 [02:12<48:13,  3.88s/it]
                                                
{'loss': 1.4233, 'grad_norm': 0.29158949851989746, 'learning_rate': 1.9991366961835643e-05, 'epoch': 0.44}

  4%|▍         | 34/780 [02:12<48:13,  3.88s/it]
  4%|▍         | 35/780 [02:16<48:13,  3.88s/it]
                                                
{'loss': 1.3868, 'grad_norm': 0.2929733395576477, 'learning_rate': 1.9989554339503612e-05, 'epoch': 0.45}

  4%|▍         | 35/780 [02:16<48:13,  3.88s/it]
  5%|▍         | 36/780 [02:20<49:14,  3.97s/it]
                                                
{'loss': 1.2947, 'grad_norm': 0.23760586977005005, 'learning_rate': 1.9987569212189224e-05, 'epoch': 0.46}

  5%|▍         | 36/780 [02:20<49:14,  3.97s/it]
  5%|▍         | 37/780 [02:24<48:25,  3.91s/it]
                                                
{'loss': 1.4098, 'grad_norm': 0.24930839240550995, 'learning_rate': 1.9985411614172728e-05, 'epoch': 0.47}

  5%|▍         | 37/780 [02:24<48:25,  3.91s/it]
  5%|▍         | 38/780 [02:28<47:32,  3.84s/it]
                                                
{'loss': 1.3482, 'grad_norm': 0.2707172930240631, 'learning_rate': 1.9983081582712684e-05, 'epoch': 0.49}

  5%|▍         | 38/780 [02:28<47:32,  3.84s/it]
  5%|▌         | 39/780 [02:31<46:33,  3.77s/it]
                                                
{'loss': 1.3724, 'grad_norm': 0.2269766926765442, 'learning_rate': 1.9980579158045322e-05, 'epoch': 0.5}

  5%|▌         | 39/780 [02:31<46:33,  3.77s/it]
  5%|▌         | 40/780 [02:35<46:13,  3.75s/it]
                                                
{'loss': 1.3079, 'grad_norm': 0.30326390266418457, 'learning_rate': 1.997790438338385e-05, 'epoch': 0.51}

  5%|▌         | 40/780 [02:35<46:13,  3.75s/it]
  5%|▌         | 41/780 [02:39<47:45,  3.88s/it]
                                                
{'loss': 1.325, 'grad_norm': 0.23943105340003967, 'learning_rate': 1.997505730491772e-05, 'epoch': 0.52}

  5%|▌         | 41/780 [02:39<47:45,  3.88s/it]
  5%|▌         | 42/780 [02:43<48:55,  3.98s/it]
                                                
{'loss': 1.3877, 'grad_norm': 0.2783259451389313, 'learning_rate': 1.9972037971811802e-05, 'epoch': 0.54}

  5%|▌         | 42/780 [02:44<48:55,  3.98s/it]
  6%|▌         | 43/780 [02:47<48:12,  3.93s/it]
                                                
{'loss': 1.3647, 'grad_norm': 0.33810847997665405, 'learning_rate': 1.9968846436205566e-05, 'epoch': 0.55}

  6%|▌         | 43/780 [02:47<48:12,  3.93s/it]
  6%|▌         | 44/780 [02:51<46:28,  3.79s/it]
                                                
{'loss': 1.368, 'grad_norm': 0.2847158908843994, 'learning_rate': 1.9965482753212154e-05, 'epoch': 0.56}

  6%|▌         | 44/780 [02:51<46:28,  3.79s/it]
  6%|▌         | 45/780 [02:55<46:29,  3.80s/it]
                                                
{'loss': 1.3646, 'grad_norm': 0.28065213561058044, 'learning_rate': 1.9961946980917457e-05, 'epoch': 0.58}

  6%|▌         | 45/780 [02:55<46:29,  3.80s/it]
  6%|▌         | 46/780 [02:59<48:07,  3.93s/it]
                                                
{'loss': 1.3522, 'grad_norm': 0.25693196058273315, 'learning_rate': 1.995823918037908e-05, 'epoch': 0.59}

  6%|▌         | 46/780 [02:59<48:07,  3.93s/it]
  6%|▌         | 47/780 [03:03<48:58,  4.01s/it]
                                                
{'loss': 1.2632, 'grad_norm': 0.2471788227558136, 'learning_rate': 1.9954359415625313e-05, 'epoch': 0.6}

  6%|▌         | 47/780 [03:03<48:58,  4.01s/it]
  6%|▌         | 48/780 [03:07<48:44,  3.99s/it]
                                                
{'loss': 1.3309, 'grad_norm': 0.28470224142074585, 'learning_rate': 1.9950307753654016e-05, 'epoch': 0.61}

  6%|▌         | 48/780 [03:07<48:44,  3.99s/it]
  6%|▋         | 49/780 [03:11<50:32,  4.15s/it]
                                                
{'loss': 1.3744, 'grad_norm': 0.3448010981082916, 'learning_rate': 1.994608426443146e-05, 'epoch': 0.63}

  6%|▋         | 49/780 [03:12<50:32,  4.15s/it]
  6%|▋         | 50/780 [03:15<48:38,  4.00s/it]
                                                
{'loss': 1.353, 'grad_norm': 0.3384905457496643, 'learning_rate': 1.994168902089112e-05, 'epoch': 0.64}

  6%|▋         | 50/780 [03:15<48:38,  4.00s/it]
  7%|▋         | 51/780 [03:19<48:14,  3.97s/it]
                                                
{'loss': 1.3518, 'grad_norm': 0.30368268489837646, 'learning_rate': 1.9937122098932428e-05, 'epoch': 0.65}

  7%|▋         | 51/780 [03:19<48:14,  3.97s/it]
  7%|▋         | 52/780 [03:22<45:53,  3.78s/it]
                                                
{'loss': 1.3441, 'grad_norm': 0.293694406747818, 'learning_rate': 1.9932383577419432e-05, 'epoch': 0.67}

  7%|▋         | 52/780 [03:22<45:53,  3.78s/it]
  7%|▋         | 53/780 [03:26<45:46,  3.78s/it]
                                                
{'loss': 1.3153, 'grad_norm': 0.33052507042884827, 'learning_rate': 1.9927473538179467e-05, 'epoch': 0.68}

  7%|▋         | 53/780 [03:26<45:46,  3.78s/it]
  7%|▋         | 54/780 [03:30<45:26,  3.76s/it]
                                                
{'loss': 1.3014, 'grad_norm': 0.3383728861808777, 'learning_rate': 1.9922392066001724e-05, 'epoch': 0.69}

  7%|▋         | 54/780 [03:30<45:26,  3.76s/it]
  7%|▋         | 55/780 [03:34<45:26,  3.76s/it]
                                                
{'loss': 1.3199, 'grad_norm': 0.33122771978378296, 'learning_rate': 1.9917139248635788e-05, 'epoch': 0.7}

  7%|▋         | 55/780 [03:34<45:26,  3.76s/it]
  7%|▋         | 56/780 [03:38<46:56,  3.89s/it]
                                                
{'loss': 1.3287, 'grad_norm': 0.32333531975746155, 'learning_rate': 1.991171517679013e-05, 'epoch': 0.72}

  7%|▋         | 56/780 [03:38<46:56,  3.89s/it]
  7%|▋         | 57/780 [03:42<47:12,  3.92s/it]
                                                
{'loss': 1.323, 'grad_norm': 0.2858966290950775, 'learning_rate': 1.9906119944130527e-05, 'epoch': 0.73}

  7%|▋         | 57/780 [03:42<47:12,  3.92s/it]
  7%|▋         | 58/780 [03:45<46:16,  3.85s/it]
                                                
{'loss': 1.3299, 'grad_norm': 0.3371015787124634, 'learning_rate': 1.9900353647278466e-05, 'epoch': 0.74}

  7%|▋         | 58/780 [03:46<46:16,  3.85s/it]
  8%|▊         | 59/780 [03:49<45:49,  3.81s/it]
                                                
{'loss': 1.3804, 'grad_norm': 0.31193289160728455, 'learning_rate': 1.9894416385809444e-05, 'epoch': 0.76}

  8%|▊         | 59/780 [03:49<45:49,  3.81s/it]
  8%|▊         | 60/780 [03:53<46:26,  3.87s/it]
                                                
{'loss': 1.3559, 'grad_norm': 0.3728688061237335, 'learning_rate': 1.9888308262251286e-05, 'epoch': 0.77}

  8%|▊         | 60/780 [03:53<46:26,  3.87s/it]
  8%|▊         | 61/780 [03:57<46:46,  3.90s/it]
                                                
{'loss': 1.39, 'grad_norm': 0.44007423520088196, 'learning_rate': 1.9882029382082342e-05, 'epoch': 0.78}

  8%|▊         | 61/780 [03:57<46:46,  3.90s/it]
  8%|▊         | 62/780 [04:01<45:46,  3.83s/it]
                                                
{'loss': 1.2681, 'grad_norm': 0.3367130756378174, 'learning_rate': 1.9875579853729677e-05, 'epoch': 0.79}

  8%|▊         | 62/780 [04:01<45:46,  3.83s/it]
  8%|▊         | 63/780 [04:04<45:06,  3.77s/it]
                                                
{'loss': 1.347, 'grad_norm': 0.3400910198688507, 'learning_rate': 1.9868959788567213e-05, 'epoch': 0.81}

  8%|▊         | 63/780 [04:05<45:06,  3.77s/it]
  8%|▊         | 64/780 [04:08<45:34,  3.82s/it]
                                                
{'loss': 1.3165, 'grad_norm': 0.33001258969306946, 'learning_rate': 1.9862169300913784e-05, 'epoch': 0.82}

  8%|▊         | 64/780 [04:08<45:34,  3.82s/it]
  8%|▊         | 65/780 [04:13<46:48,  3.93s/it]
                                                
{'loss': 1.3479, 'grad_norm': 0.3575596213340759, 'learning_rate': 1.9855208508031173e-05, 'epoch': 0.83}

  8%|▊         | 65/780 [04:13<46:48,  3.93s/it]
  8%|▊         | 66/780 [04:16<46:09,  3.88s/it]
                                                
{'loss': 1.3638, 'grad_norm': 0.4133343994617462, 'learning_rate': 1.9848077530122083e-05, 'epoch': 0.84}

  8%|▊         | 66/780 [04:16<46:09,  3.88s/it]
  9%|▊         | 67/780 [04:20<46:59,  3.95s/it]
                                                
{'loss': 1.259, 'grad_norm': 0.35606008768081665, 'learning_rate': 1.9840776490328067e-05, 'epoch': 0.86}

  9%|▊         | 67/780 [04:21<46:59,  3.95s/it]
  9%|▊         | 68/780 [04:24<47:05,  3.97s/it]
                                                
{'loss': 1.3462, 'grad_norm': 0.4007570147514343, 'learning_rate': 1.9833305514727396e-05, 'epoch': 0.87}

  9%|▊         | 68/780 [04:25<47:05,  3.97s/it]
  9%|▉         | 69/780 [04:29<48:07,  4.06s/it]
                                                
{'loss': 1.288, 'grad_norm': 0.3583349585533142, 'learning_rate': 1.9825664732332886e-05, 'epoch': 0.88}

  9%|▉         | 69/780 [04:29<48:07,  4.06s/it]
  9%|▉         | 70/780 [04:32<46:32,  3.93s/it]
                                                
{'loss': 1.3115, 'grad_norm': 0.4068514406681061, 'learning_rate': 1.981785427508966e-05, 'epoch': 0.9}

  9%|▉         | 70/780 [04:32<46:32,  3.93s/it]
  9%|▉         | 71/780 [04:37<47:32,  4.02s/it]
                                                
{'loss': 1.2852, 'grad_norm': 0.35279297828674316, 'learning_rate': 1.9809874277872886e-05, 'epoch': 0.91}

  9%|▉         | 71/780 [04:37<47:32,  4.02s/it]
  9%|▉         | 72/780 [04:40<46:22,  3.93s/it]
                                                
{'loss': 1.3406, 'grad_norm': 0.44112610816955566, 'learning_rate': 1.9801724878485438e-05, 'epoch': 0.92}

  9%|▉         | 72/780 [04:40<46:22,  3.93s/it]
  9%|▉         | 73/780 [04:44<45:42,  3.88s/it]
                                                
{'loss': 1.3485, 'grad_norm': 0.5092766880989075, 'learning_rate': 1.9793406217655516e-05, 'epoch': 0.93}

  9%|▉         | 73/780 [04:44<45:42,  3.88s/it]
  9%|▉         | 74/780 [04:48<45:09,  3.84s/it]
                                                
{'loss': 1.2635, 'grad_norm': 0.44801339507102966, 'learning_rate': 1.9784918439034216e-05, 'epoch': 0.95}

  9%|▉         | 74/780 [04:48<45:09,  3.84s/it]
 10%|▉         | 75/780 [04:51<44:03,  3.75s/it]
                                                
{'loss': 1.271, 'grad_norm': 0.5840102434158325, 'learning_rate': 1.977626168919305e-05, 'epoch': 0.96}

 10%|▉         | 75/780 [04:51<44:03,  3.75s/it]
 10%|▉         | 76/780 [04:55<43:10,  3.68s/it]
                                                
{'loss': 1.3322, 'grad_norm': 0.5075913667678833, 'learning_rate': 1.9767436117621416e-05, 'epoch': 0.97}

 10%|▉         | 76/780 [04:55<43:10,  3.68s/it]
 10%|▉         | 77/780 [04:59<44:11,  3.77s/it]
                                                
{'loss': 1.3211, 'grad_norm': 0.4642045795917511, 'learning_rate': 1.975844187672402e-05, 'epoch': 0.99}

 10%|▉         | 77/780 [04:59<44:11,  3.77s/it]
 10%|█         | 78/780 [05:03<46:40,  3.99s/it]
                                                
{'loss': 1.3226, 'grad_norm': 0.46179908514022827, 'learning_rate': 1.9749279121818235e-05, 'epoch': 1.0}

 10%|█         | 78/780 [05:03<46:40,  3.99s/it]
 10%|█         | 79/780 [05:07<46:47,  4.01s/it]
                                                
{'loss': 1.3503, 'grad_norm': 0.4638010561466217, 'learning_rate': 1.9739948011131438e-05, 'epoch': 1.01}

 10%|█         | 79/780 [05:07<46:47,  4.01s/it]
 10%|█         | 80/780 [05:11<45:58,  3.94s/it]
                                                
{'loss': 1.2544, 'grad_norm': 0.5397721529006958, 'learning_rate': 1.973044870579824e-05, 'epoch': 1.02}

 10%|█         | 80/780 [05:11<45:58,  3.94s/it]
 10%|█         | 81/780 [05:15<45:48,  3.93s/it]
                                                
{'loss': 1.3007, 'grad_norm': 0.5057282447814941, 'learning_rate': 1.9720781369857747e-05, 'epoch': 1.04}

 10%|█         | 81/780 [05:15<45:48,  3.93s/it]
 11%|█         | 82/780 [05:19<44:39,  3.84s/it]
                                                
{'loss': 1.4254, 'grad_norm': 0.5012478828430176, 'learning_rate': 1.9710946170250702e-05, 'epoch': 1.05}

 11%|█         | 82/780 [05:19<44:39,  3.84s/it]
 11%|█         | 83/780 [05:22<43:32,  3.75s/it]
                                                
{'loss': 1.3843, 'grad_norm': 0.4735774099826813, 'learning_rate': 1.9700943276816602e-05, 'epoch': 1.06}

 11%|█         | 83/780 [05:22<43:32,  3.75s/it]
 11%|█         | 84/780 [05:26<44:57,  3.88s/it]
                                                
{'loss': 1.2168, 'grad_norm': 0.42239442467689514, 'learning_rate': 1.969077286229078e-05, 'epoch': 1.08}

 11%|█         | 84/780 [05:27<44:57,  3.88s/it]
 11%|█         | 85/780 [05:30<45:10,  3.90s/it]
                                                
{'loss': 1.2715, 'grad_norm': 0.4902520179748535, 'learning_rate': 1.9680435102301412e-05, 'epoch': 1.09}

 11%|█         | 85/780 [05:30<45:10,  3.90s/it]
 11%|█         | 86/780 [05:34<45:33,  3.94s/it]
                                                
{'loss': 1.3746, 'grad_norm': 0.49053725600242615, 'learning_rate': 1.9669930175366474e-05, 'epoch': 1.1}

 11%|█         | 86/780 [05:34<45:33,  3.94s/it]
 11%|█         | 87/780 [05:38<44:06,  3.82s/it]
                                                
{'loss': 1.3657, 'grad_norm': 0.48261216282844543, 'learning_rate': 1.9659258262890683e-05, 'epoch': 1.11}

 11%|█         | 87/780 [05:38<44:06,  3.82s/it]
 11%|█▏        | 88/780 [05:42<44:56,  3.90s/it]
                                                
{'loss': 1.3123, 'grad_norm': 0.5313539505004883, 'learning_rate': 1.964841954916235e-05, 'epoch': 1.13}

 11%|█▏        | 88/780 [05:42<44:56,  3.90s/it]
 11%|█▏        | 89/780 [05:46<46:06,  4.00s/it]
                                                
{'loss': 1.3093, 'grad_norm': 0.5206671953201294, 'learning_rate': 1.9637414221350198e-05, 'epoch': 1.14}

 11%|█▏        | 89/780 [05:46<46:06,  4.00s/it]
 12%|█▏        | 90/780 [05:51<46:58,  4.08s/it]
                                                
{'loss': 1.3391, 'grad_norm': 0.47249776124954224, 'learning_rate': 1.962624246950012e-05, 'epoch': 1.15}

 12%|█▏        | 90/780 [05:51<46:58,  4.08s/it]
 12%|█▏        | 91/780 [05:55<47:59,  4.18s/it]
                                                
{'loss': 1.3506, 'grad_norm': 0.615608274936676, 'learning_rate': 1.9614904486531935e-05, 'epoch': 1.16}

 12%|█▏        | 91/780 [05:55<47:59,  4.18s/it]
 12%|█▏        | 92/780 [05:59<48:07,  4.20s/it]
                                                
{'loss': 1.278, 'grad_norm': 0.5446704626083374, 'learning_rate': 1.9603400468236e-05, 'epoch': 1.18}

 12%|█▏        | 92/780 [05:59<48:07,  4.20s/it]
 12%|█▏        | 93/780 [06:03<46:50,  4.09s/it]
                                                
{'loss': 1.2523, 'grad_norm': 0.5009592771530151, 'learning_rate': 1.9591730613269878e-05, 'epoch': 1.19}

 12%|█▏        | 93/780 [06:03<46:50,  4.09s/it]
 12%|█▏        | 94/780 [06:07<45:40,  4.00s/it]
                                                
{'loss': 1.419, 'grad_norm': 0.45924320816993713, 'learning_rate': 1.957989512315489e-05, 'epoch': 1.2}

 12%|█▏        | 94/780 [06:07<45:40,  4.00s/it]
 12%|█▏        | 95/780 [06:11<45:21,  3.97s/it]
                                                
{'loss': 1.3448, 'grad_norm': 0.44887420535087585, 'learning_rate': 1.9567894202272623e-05, 'epoch': 1.22}

 12%|█▏        | 95/780 [06:11<45:21,  3.97s/it]
 12%|█▏        | 96/780 [06:15<44:47,  3.93s/it]
                                                
{'loss': 1.3062, 'grad_norm': 0.45753347873687744, 'learning_rate': 1.955572805786141e-05, 'epoch': 1.23}

 12%|█▏        | 96/780 [06:15<44:47,  3.93s/it]
 12%|█▏        | 97/780 [06:18<44:27,  3.90s/it]
                                                
{'loss': 1.2962, 'grad_norm': 0.518760085105896, 'learning_rate': 1.9543396900012763e-05, 'epoch': 1.24}

 12%|█▏        | 97/780 [06:18<44:27,  3.90s/it]
 13%|█▎        | 98/780 [06:22<43:45,  3.85s/it]
                                                
{'loss': 1.2836, 'grad_norm': 0.5205446481704712, 'learning_rate': 1.9530900941667733e-05, 'epoch': 1.25}

 13%|█▎        | 98/780 [06:22<43:45,  3.85s/it]
 13%|█▎        | 99/780 [06:26<45:13,  3.98s/it]
                                                
{'loss': 1.3183, 'grad_norm': 0.48316702246665955, 'learning_rate': 1.9518240398613226e-05, 'epoch': 1.27}

 13%|█▎        | 99/780 [06:26<45:13,  3.98s/it]
 13%|█▎        | 100/780 [06:30<44:12,  3.90s/it]
                                                 
{'loss': 1.3073, 'grad_norm': 0.45222723484039307, 'learning_rate': 1.9505415489478293e-05, 'epoch': 1.28}

 13%|█▎        | 100/780 [06:30<44:12,  3.90s/it]
 13%|█▎        | 101/780 [06:34<44:53,  3.97s/it]
                                                 
{'loss': 1.2307, 'grad_norm': 0.7134236693382263, 'learning_rate': 1.949242643573034e-05, 'epoch': 1.29}

 13%|█▎        | 101/780 [06:34<44:53,  3.97s/it]
 13%|█▎        | 102/780 [06:38<44:10,  3.91s/it]
                                                 
{'loss': 1.3579, 'grad_norm': 0.4833638668060303, 'learning_rate': 1.947927346167132e-05, 'epoch': 1.31}

 13%|█▎        | 102/780 [06:38<44:10,  3.91s/it]
 13%|█▎        | 103/780 [06:42<44:03,  3.90s/it]
                                                 
{'loss': 1.3186, 'grad_norm': 0.5174211263656616, 'learning_rate': 1.9465956794433837e-05, 'epoch': 1.32}

 13%|█▎        | 103/780 [06:42<44:03,  3.90s/it]
 13%|█▎        | 104/780 [06:46<44:49,  3.98s/it]
                                                 
{'loss': 1.2759, 'grad_norm': 0.4952283203601837, 'learning_rate': 1.945247666397725e-05, 'epoch': 1.33}

 13%|█▎        | 104/780 [06:46<44:49,  3.98s/it]
 13%|█▎        | 105/780 [06:50<43:38,  3.88s/it]
                                                 
{'loss': 1.2628, 'grad_norm': 0.5008366703987122, 'learning_rate': 1.9438833303083677e-05, 'epoch': 1.34}

 13%|█▎        | 105/780 [06:50<43:38,  3.88s/it]
 14%|█▎        | 106/780 [06:54<43:53,  3.91s/it]
                                                 
{'loss': 1.3363, 'grad_norm': 0.42817240953445435, 'learning_rate': 1.9425026947353994e-05, 'epoch': 1.36}

 14%|█▎        | 106/780 [06:54<43:53,  3.91s/it]
 14%|█▎        | 107/780 [06:57<43:04,  3.84s/it]
                                                 
{'loss': 1.3221, 'grad_norm': 0.5042969584465027, 'learning_rate': 1.9411057835203756e-05, 'epoch': 1.37}

 14%|█▎        | 107/780 [06:57<43:04,  3.84s/it]
 14%|█▍        | 108/780 [07:01<42:58,  3.84s/it]
                                                 
{'loss': 1.275, 'grad_norm': 0.610226571559906, 'learning_rate': 1.9396926207859085e-05, 'epoch': 1.38}

 14%|█▍        | 108/780 [07:01<42:58,  3.84s/it]
 14%|█▍        | 109/780 [07:05<43:23,  3.88s/it]
                                                 
{'loss': 1.2942, 'grad_norm': 0.4753764271736145, 'learning_rate': 1.9382632309352503e-05, 'epoch': 1.4}

 14%|█▍        | 109/780 [07:05<43:23,  3.88s/it]
 14%|█▍        | 110/780 [07:09<43:01,  3.85s/it]
                                                 
{'loss': 1.2442, 'grad_norm': 0.4707958698272705, 'learning_rate': 1.936817638651871e-05, 'epoch': 1.41}

 14%|█▍        | 110/780 [07:09<43:01,  3.85s/it]
 14%|█▍        | 111/780 [07:13<42:55,  3.85s/it]
                                                 
{'loss': 1.2513, 'grad_norm': 0.5380642414093018, 'learning_rate': 1.935355868899034e-05, 'epoch': 1.42}

 14%|█▍        | 111/780 [07:13<42:55,  3.85s/it]
 14%|█▍        | 112/780 [07:17<43:00,  3.86s/it]
                                                 
{'loss': 1.3686, 'grad_norm': 0.5449580550193787, 'learning_rate': 1.9338779469193638e-05, 'epoch': 1.43}

 14%|█▍        | 112/780 [07:17<43:00,  3.86s/it]
 14%|█▍        | 113/780 [07:21<42:58,  3.87s/it]
                                                 
{'loss': 1.2987, 'grad_norm': 0.5102094411849976, 'learning_rate': 1.9323838982344092e-05, 'epoch': 1.45}

 14%|█▍        | 113/780 [07:21<42:58,  3.87s/it]
 15%|█▍        | 114/780 [07:25<43:04,  3.88s/it]
                                                 
{'loss': 1.2606, 'grad_norm': 0.45051100850105286, 'learning_rate': 1.9308737486442045e-05, 'epoch': 1.46}

 15%|█▍        | 114/780 [07:25<43:04,  3.88s/it]
 15%|█▍        | 115/780 [07:29<43:24,  3.92s/it]
                                                 
{'loss': 1.3616, 'grad_norm': 0.4395079016685486, 'learning_rate': 1.9293475242268224e-05, 'epoch': 1.47}

 15%|█▍        | 115/780 [07:29<43:24,  3.92s/it]
 15%|█▍        | 116/780 [07:32<42:15,  3.82s/it]
                                                 
{'loss': 1.2778, 'grad_norm': 0.4913008511066437, 'learning_rate': 1.9278052513379256e-05, 'epoch': 1.48}

 15%|█▍        | 116/780 [07:32<42:15,  3.82s/it]
 15%|█▌        | 117/780 [07:36<42:08,  3.81s/it]
                                                 
{'loss': 1.3227, 'grad_norm': 0.4239055812358856, 'learning_rate': 1.926246956610309e-05, 'epoch': 1.5}

 15%|█▌        | 117/780 [07:36<42:08,  3.81s/it]
 15%|█▌        | 118/780 [07:40<41:35,  3.77s/it]
                                                 
{'loss': 1.2451, 'grad_norm': 0.6086232662200928, 'learning_rate': 1.9246726669534416e-05, 'epoch': 1.51}

 15%|█▌        | 118/780 [07:40<41:35,  3.77s/it]
 15%|█▌        | 119/780 [07:44<42:34,  3.86s/it]
                                                 
{'loss': 1.2421, 'grad_norm': 0.4605843126773834, 'learning_rate': 1.923082409553002e-05, 'epoch': 1.52}

 15%|█▌        | 119/780 [07:44<42:34,  3.86s/it]
 15%|█▌        | 120/780 [07:48<42:46,  3.89s/it]
                                                 
{'loss': 1.3169, 'grad_norm': 0.4977746605873108, 'learning_rate': 1.921476211870408e-05, 'epoch': 1.54}

 15%|█▌        | 120/780 [07:48<42:46,  3.89s/it]
 16%|█▌        | 121/780 [07:52<42:51,  3.90s/it]
                                                 
{'loss': 1.2694, 'grad_norm': 0.49456676840782166, 'learning_rate': 1.919854101642342e-05, 'epoch': 1.55}

 16%|█▌        | 121/780 [07:52<42:51,  3.90s/it]
 16%|█▌        | 122/780 [07:55<41:00,  3.74s/it]
                                                 
{'loss': 1.3363, 'grad_norm': 0.5131132006645203, 'learning_rate': 1.9182161068802742e-05, 'epoch': 1.56}

 16%|█▌        | 122/780 [07:55<41:00,  3.74s/it]
 16%|█▌        | 123/780 [07:59<40:57,  3.74s/it]
                                                 
{'loss': 1.2834, 'grad_norm': 0.49851951003074646, 'learning_rate': 1.9165622558699763e-05, 'epoch': 1.57}

 16%|█▌        | 123/780 [07:59<40:57,  3.74s/it]
 16%|█▌        | 124/780 [08:03<42:05,  3.85s/it]
                                                 
{'loss': 1.2899, 'grad_norm': 0.4947567880153656, 'learning_rate': 1.9148925771710347e-05, 'epoch': 1.59}

 16%|█▌        | 124/780 [08:03<42:05,  3.85s/it]
 16%|█▌        | 125/780 [08:07<42:50,  3.92s/it]
                                                 
{'loss': 1.2115, 'grad_norm': 0.44778183102607727, 'learning_rate': 1.913207099616357e-05, 'epoch': 1.6}

 16%|█▌        | 125/780 [08:07<42:50,  3.92s/it]
 16%|█▌        | 126/780 [08:11<43:17,  3.97s/it]
                                                 
{'loss': 1.2437, 'grad_norm': 0.4581713080406189, 'learning_rate': 1.9115058523116734e-05, 'epoch': 1.61}

 16%|█▌        | 126/780 [08:11<43:17,  3.97s/it]
 16%|█▋        | 127/780 [08:15<44:51,  4.12s/it]
                                                 
{'loss': 1.2748, 'grad_norm': 0.5209805369377136, 'learning_rate': 1.9097888646350347e-05, 'epoch': 1.63}

 16%|█▋        | 127/780 [08:15<44:51,  4.12s/it]
 16%|█▋        | 128/780 [08:19<42:30,  3.91s/it]
                                                 
{'loss': 1.3134, 'grad_norm': 0.4990711212158203, 'learning_rate': 1.908056166236305e-05, 'epoch': 1.64}

 16%|█▋        | 128/780 [08:19<42:30,  3.91s/it]
 17%|█▋        | 129/780 [08:23<42:51,  3.95s/it]
                                                 
{'loss': 1.2445, 'grad_norm': 0.4981730580329895, 'learning_rate': 1.9063077870366504e-05, 'epoch': 1.65}

 17%|█▋        | 129/780 [08:23<42:51,  3.95s/it]
 17%|█▋        | 130/780 [08:27<41:42,  3.85s/it]
                                                 
{'loss': 1.2867, 'grad_norm': 0.48749589920043945, 'learning_rate': 1.9045437572280193e-05, 'epoch': 1.66}

 17%|█▋        | 130/780 [08:27<41:42,  3.85s/it]
 17%|█▋        | 131/780 [08:30<41:08,  3.80s/it]
                                                 
{'loss': 1.2379, 'grad_norm': 0.4713943600654602, 'learning_rate': 1.902764107272626e-05, 'epoch': 1.68}

 17%|█▋        | 131/780 [08:30<41:08,  3.80s/it]
 17%|█▋        | 132/780 [08:34<40:00,  3.70s/it]
                                                 
{'loss': 1.2282, 'grad_norm': 0.49886608123779297, 'learning_rate': 1.900968867902419e-05, 'epoch': 1.69}

 17%|█▋        | 132/780 [08:34<40:00,  3.70s/it]
 17%|█▋        | 133/780 [08:38<40:52,  3.79s/it]
                                                 
{'loss': 1.2676, 'grad_norm': 0.5358107686042786, 'learning_rate': 1.8991580701185564e-05, 'epoch': 1.7}

 17%|█▋        | 133/780 [08:38<40:52,  3.79s/it]
 17%|█▋        | 134/780 [08:42<42:05,  3.91s/it]
                                                 
{'loss': 1.2274, 'grad_norm': 0.4886687994003296, 'learning_rate': 1.8973317451908642e-05, 'epoch': 1.72}

 17%|█▋        | 134/780 [08:42<42:05,  3.91s/it]
 17%|█▋        | 135/780 [08:46<42:22,  3.94s/it]
                                                 
{'loss': 1.2635, 'grad_norm': 0.4797861874103546, 'learning_rate': 1.895489924657301e-05, 'epoch': 1.73}

 17%|█▋        | 135/780 [08:46<42:22,  3.94s/it]
 17%|█▋        | 136/780 [08:49<40:53,  3.81s/it]
                                                 
{'loss': 1.2518, 'grad_norm': 0.45854631066322327, 'learning_rate': 1.8936326403234125e-05, 'epoch': 1.74}

 17%|█▋        | 136/780 [08:49<40:53,  3.81s/it]
 18%|█▊        | 137/780 [08:53<39:47,  3.71s/it]
                                                 
{'loss': 1.2891, 'grad_norm': 0.4742138981819153, 'learning_rate': 1.8917599242617796e-05, 'epoch': 1.75}

 18%|█▊        | 137/780 [08:53<39:47,  3.71s/it]
 18%|█▊        | 138/780 [08:57<41:01,  3.83s/it]
                                                 
{'loss': 1.2919, 'grad_norm': 0.49370989203453064, 'learning_rate': 1.8898718088114688e-05, 'epoch': 1.77}

 18%|█▊        | 138/780 [08:57<41:01,  3.83s/it]
 18%|█▊        | 139/780 [09:01<41:43,  3.91s/it]
                                                 
{'loss': 1.3185, 'grad_norm': 0.6153765320777893, 'learning_rate': 1.8879683265774695e-05, 'epoch': 1.78}

 18%|█▊        | 139/780 [09:01<41:43,  3.91s/it]
 18%|█▊        | 140/780 [09:05<40:13,  3.77s/it]
                                                 
{'loss': 1.2055, 'grad_norm': 0.47768208384513855, 'learning_rate': 1.8860495104301346e-05, 'epoch': 1.79}

 18%|█▊        | 140/780 [09:05<40:13,  3.77s/it]
 18%|█▊        | 141/780 [09:08<39:54,  3.75s/it]
                                                 
{'loss': 1.2598, 'grad_norm': 0.46390649676322937, 'learning_rate': 1.8841153935046098e-05, 'epoch': 1.8}

 18%|█▊        | 141/780 [09:08<39:54,  3.75s/it]
 18%|█▊        | 142/780 [09:12<40:41,  3.83s/it]
                                                 
{'loss': 1.2444, 'grad_norm': 0.4337811768054962, 'learning_rate': 1.8821660092002642e-05, 'epoch': 1.82}

 18%|█▊        | 142/780 [09:12<40:41,  3.83s/it]
 18%|█▊        | 143/780 [09:16<41:15,  3.89s/it]
                                                 
{'loss': 1.2948, 'grad_norm': 0.44328197836875916, 'learning_rate': 1.880201391180111e-05, 'epoch': 1.83}

 18%|█▊        | 143/780 [09:16<41:15,  3.89s/it]
 18%|█▊        | 144/780 [09:20<40:50,  3.85s/it]
                                                 
{'loss': 1.296, 'grad_norm': 0.46118593215942383, 'learning_rate': 1.8782215733702286e-05, 'epoch': 1.84}

 18%|█▊        | 144/780 [09:20<40:50,  3.85s/it]
 19%|█▊        | 145/780 [09:24<41:34,  3.93s/it]
                                                 
{'loss': 1.1842, 'grad_norm': 0.4422454535961151, 'learning_rate': 1.8762265899591724e-05, 'epoch': 1.86}

 19%|█▊        | 145/780 [09:24<41:34,  3.93s/it]
 19%|█▊        | 146/780 [09:28<41:49,  3.96s/it]
                                                 
{'loss': 1.2759, 'grad_norm': 0.4480154812335968, 'learning_rate': 1.874216475397386e-05, 'epoch': 1.87}

 19%|█▊        | 146/780 [09:28<41:49,  3.96s/it]
 19%|█▉        | 147/780 [09:32<42:46,  4.05s/it]
                                                 
{'loss': 1.2512, 'grad_norm': 0.43200692534446716, 'learning_rate': 1.8721912643966055e-05, 'epoch': 1.88}

 19%|█▉        | 147/780 [09:32<42:46,  4.05s/it]
 19%|█▉        | 148/780 [09:36<42:00,  3.99s/it]
                                                 
{'loss': 1.2544, 'grad_norm': 0.4690806269645691, 'learning_rate': 1.870150991929261e-05, 'epoch': 1.89}

 19%|█▉        | 148/780 [09:36<42:00,  3.99s/it]
 19%|█▉        | 149/780 [09:40<42:33,  4.05s/it]
                                                 
{'loss': 1.1712, 'grad_norm': 0.43237048387527466, 'learning_rate': 1.868095693227872e-05, 'epoch': 1.91}

 19%|█▉        | 149/780 [09:40<42:33,  4.05s/it]
 19%|█▉        | 150/780 [09:44<41:27,  3.95s/it]
                                                 
{'loss': 1.2597, 'grad_norm': 0.4949786365032196, 'learning_rate': 1.866025403784439e-05, 'epoch': 1.92}

 19%|█▉        | 150/780 [09:44<41:27,  3.95s/it]
 19%|█▉        | 151/780 [09:48<40:16,  3.84s/it]
                                                 
{'loss': 1.2877, 'grad_norm': 0.5918819308280945, 'learning_rate': 1.86394015934983e-05, 'epoch': 1.93}

 19%|█▉        | 151/780 [09:48<40:16,  3.84s/it]
 19%|█▉        | 152/780 [09:52<40:34,  3.88s/it]
                                                 
{'loss': 1.1565, 'grad_norm': 0.46426963806152344, 'learning_rate': 1.8618399959331642e-05, 'epoch': 1.95}

 19%|█▉        | 152/780 [09:52<40:34,  3.88s/it]
 20%|█▉        | 153/780 [09:55<38:54,  3.72s/it]
                                                 
{'loss': 1.1782, 'grad_norm': 0.5520857572555542, 'learning_rate': 1.8597249498011906e-05, 'epoch': 1.96}

 20%|█▉        | 153/780 [09:55<38:54,  3.72s/it]
 20%|█▉        | 154/780 [09:59<38:19,  3.67s/it]
                                                 
{'loss': 1.2787, 'grad_norm': 0.501081109046936, 'learning_rate': 1.8575950574776595e-05, 'epoch': 1.97}

 20%|█▉        | 154/780 [09:59<38:19,  3.67s/it]
 20%|█▉        | 155/780 [10:03<39:11,  3.76s/it]
                                                 
{'loss': 1.1917, 'grad_norm': 0.5437424182891846, 'learning_rate': 1.855450355742695e-05, 'epoch': 1.98}

 20%|█▉        | 155/780 [10:03<39:11,  3.76s/it]
 20%|██        | 156/780 [10:07<40:47,  3.92s/it]
                                                 
{'loss': 1.2996, 'grad_norm': 0.5097444653511047, 'learning_rate': 1.8532908816321557e-05, 'epoch': 2.0}

 20%|██        | 156/780 [10:07<40:47,  3.92s/it]
 20%|██        | 157/780 [10:11<41:04,  3.96s/it]
                                                 
{'loss': 1.2568, 'grad_norm': 0.4511759877204895, 'learning_rate': 1.8511166724369997e-05, 'epoch': 2.01}

 20%|██        | 157/780 [10:11<41:04,  3.96s/it]
 20%|██        | 158/780 [10:15<40:20,  3.89s/it]
                                                 
{'loss': 1.1989, 'grad_norm': 0.4967455565929413, 'learning_rate': 1.8489277657026377e-05, 'epoch': 2.02}

 20%|██        | 158/780 [10:15<40:20,  3.89s/it]
 20%|██        | 159/780 [10:19<40:16,  3.89s/it]
                                                 
{'loss': 1.233, 'grad_norm': 0.46324652433395386, 'learning_rate': 1.8467241992282842e-05, 'epoch': 2.04}

 20%|██        | 159/780 [10:19<40:16,  3.89s/it]
 21%|██        | 160/780 [10:22<39:54,  3.86s/it]
                                                 
{'loss': 1.3126, 'grad_norm': 0.5074864625930786, 'learning_rate': 1.844506011066308e-05, 'epoch': 2.05}

 21%|██        | 160/780 [10:22<39:54,  3.86s/it]
 21%|██        | 161/780 [10:26<38:05,  3.69s/it]
                                                 
{'loss': 1.3181, 'grad_norm': 0.4795459508895874, 'learning_rate': 1.8422732395215717e-05, 'epoch': 2.06}

 21%|██        | 161/780 [10:26<38:05,  3.69s/it]
 21%|██        | 162/780 [10:30<40:02,  3.89s/it]
                                                 
{'loss': 1.1681, 'grad_norm': 0.43093520402908325, 'learning_rate': 1.8400259231507716e-05, 'epoch': 2.07}

 21%|██        | 162/780 [10:30<40:02,  3.89s/it]
 21%|██        | 163/780 [10:34<41:12,  4.01s/it]
                                                 
{'loss': 1.1954, 'grad_norm': 0.48260265588760376, 'learning_rate': 1.8377641007617724e-05, 'epoch': 2.09}

 21%|██        | 163/780 [10:34<41:12,  4.01s/it]
 21%|██        | 164/780 [10:38<39:41,  3.87s/it]
                                                 
{'loss': 1.2852, 'grad_norm': 0.49347984790802, 'learning_rate': 1.8354878114129368e-05, 'epoch': 2.1}

 21%|██        | 164/780 [10:38<39:41,  3.87s/it]
 21%|██        | 165/780 [10:42<39:43,  3.88s/it]{'loss': 1.2966, 'grad_norm': 0.44556790590286255, 'learning_rate': 1.833197094412449e-05, 'epoch': 2.11}

                                                 

 21%|██        | 165/780 [10:42<39:43,  3.88s/it]
 21%|██▏       | 166/780 [10:46<40:25,  3.95s/it]
                                                 
{'loss': 1.2598, 'grad_norm': 0.4947715401649475, 'learning_rate': 1.8308919893176397e-05, 'epoch': 2.12}

 21%|██▏       | 166/780 [10:46<40:25,  3.95s/it]
 21%|██▏       | 167/780 [10:50<41:00,  4.01s/it]
                                                 
{'loss': 1.2236, 'grad_norm': 0.4683668911457062, 'learning_rate': 1.8285725359343e-05, 'epoch': 2.14}

 21%|██▏       | 167/780 [10:50<41:00,  4.01s/it]
 22%|██▏       | 168/780 [10:54<40:19,  3.95s/it]
                                                 
{'loss': 1.2947, 'grad_norm': 0.555779755115509, 'learning_rate': 1.826238774315995e-05, 'epoch': 2.15}

 22%|██▏       | 168/780 [10:54<40:19,  3.95s/it]
 22%|██▏       | 169/780 [10:58<41:22,  4.06s/it]
                                                 
{'loss': 1.2542, 'grad_norm': 0.559021532535553, 'learning_rate': 1.8238907447633716e-05, 'epoch': 2.16}

 22%|██▏       | 169/780 [10:58<41:22,  4.06s/it]
 22%|██▏       | 170/780 [11:02<41:46,  4.11s/it]
                                                 
{'loss': 1.2112, 'grad_norm': 0.5354365706443787, 'learning_rate': 1.8215284878234644e-05, 'epoch': 2.18}

 22%|██▏       | 170/780 [11:02<41:46,  4.11s/it]
 22%|██▏       | 171/780 [11:06<40:22,  3.98s/it]
                                                 
{'loss': 1.1872, 'grad_norm': 0.4920743405818939, 'learning_rate': 1.819152044288992e-05, 'epoch': 2.19}

 22%|██▏       | 171/780 [11:06<40:22,  3.98s/it]
 22%|██▏       | 172/780 [11:10<40:17,  3.98s/it]
                                                 
{'loss': 1.3509, 'grad_norm': 0.4360363185405731, 'learning_rate': 1.816761455197657e-05, 'epoch': 2.2}

 22%|██▏       | 172/780 [11:10<40:17,  3.98s/it]
 22%|██▏       | 173/780 [11:14<39:43,  3.93s/it]
                                                 
{'loss': 1.2802, 'grad_norm': 0.418645441532135, 'learning_rate': 1.8143567618314336e-05, 'epoch': 2.21}

 22%|██▏       | 173/780 [11:14<39:43,  3.93s/it]
 22%|██▏       | 174/780 [11:17<38:35,  3.82s/it]
                                                 
{'loss': 1.2325, 'grad_norm': 0.4410291910171509, 'learning_rate': 1.811938005715857e-05, 'epoch': 2.23}

 22%|██▏       | 174/780 [11:17<38:35,  3.82s/it]
 22%|██▏       | 175/780 [11:21<38:09,  3.78s/it]
                                                 
{'loss': 1.2484, 'grad_norm': 0.4910166561603546, 'learning_rate': 1.8095052286193044e-05, 'epoch': 2.24}

 22%|██▏       | 175/780 [11:21<38:09,  3.78s/it]
 23%|██▎       | 176/780 [11:25<39:05,  3.88s/it]
                                                 
{'loss': 1.2483, 'grad_norm': 0.47615697979927063, 'learning_rate': 1.8070584725522763e-05, 'epoch': 2.25}

 23%|██▎       | 176/780 [11:25<39:05,  3.88s/it]
 23%|██▎       | 177/780 [11:29<39:06,  3.89s/it]
                                                 
{'loss': 1.2457, 'grad_norm': 0.49532806873321533, 'learning_rate': 1.8045977797666685e-05, 'epoch': 2.27}

 23%|██▎       | 177/780 [11:29<39:06,  3.89s/it]
 23%|██▎       | 178/780 [11:33<39:40,  3.95s/it]
                                                 
{'loss': 1.2452, 'grad_norm': 0.4159587323665619, 'learning_rate': 1.802123192755044e-05, 'epoch': 2.28}

 23%|██▎       | 178/780 [11:33<39:40,  3.95s/it]
 23%|██▎       | 179/780 [11:37<39:38,  3.96s/it]
                                                 
{'loss': 1.2049, 'grad_norm': 0.6431630253791809, 'learning_rate': 1.7996347542498983e-05, 'epoch': 2.29}

 23%|██▎       | 179/780 [11:37<39:38,  3.96s/it]
 23%|██▎       | 180/780 [11:41<38:59,  3.90s/it]
                                                 
{'loss': 1.2562, 'grad_norm': 0.47843119502067566, 'learning_rate': 1.7971325072229227e-05, 'epoch': 2.3}

 23%|██▎       | 180/780 [11:41<38:59,  3.90s/it]
 23%|██▎       | 181/780 [11:45<39:46,  3.98s/it]
                                                 
{'loss': 1.2717, 'grad_norm': 0.5054663419723511, 'learning_rate': 1.7946164948842604e-05, 'epoch': 2.32}

 23%|██▎       | 181/780 [11:45<39:46,  3.98s/it]
 23%|██▎       | 182/780 [11:49<40:23,  4.05s/it]
                                                 
{'loss': 1.2227, 'grad_norm': 0.4438251852989197, 'learning_rate': 1.7920867606817625e-05, 'epoch': 2.33}

 23%|██▎       | 182/780 [11:49<40:23,  4.05s/it]
 23%|██▎       | 183/780 [11:53<38:58,  3.92s/it]
                                                 
{'loss': 1.1985, 'grad_norm': 0.465902179479599, 'learning_rate': 1.7895433483002356e-05, 'epoch': 2.34}

 23%|██▎       | 183/780 [11:53<38:58,  3.92s/it]
 24%|██▎       | 184/780 [11:57<37:54,  3.82s/it]
                                                 
{'loss': 1.2993, 'grad_norm': 0.407670795917511, 'learning_rate': 1.7869863016606893e-05, 'epoch': 2.36}

 24%|██▎       | 184/780 [11:57<37:54,  3.82s/it]
 24%|██▎       | 185/780 [12:01<38:39,  3.90s/it]
                                                 
{'loss': 1.2341, 'grad_norm': 0.5067509412765503, 'learning_rate': 1.784415664919576e-05, 'epoch': 2.37}

 24%|██▎       | 185/780 [12:01<38:39,  3.90s/it]
 24%|██▍       | 186/780 [12:05<39:41,  4.01s/it]
                                                 
{'loss': 1.2302, 'grad_norm': 0.5036367774009705, 'learning_rate': 1.78183148246803e-05, 'epoch': 2.38}

 24%|██▍       | 186/780 [12:05<39:41,  4.01s/it]
 24%|██▍       | 187/780 [12:09<39:52,  4.03s/it]
                                                 
{'loss': 1.2386, 'grad_norm': 0.5684143304824829, 'learning_rate': 1.7792337989311e-05, 'epoch': 2.39}

 24%|██▍       | 187/780 [12:09<39:52,  4.03s/it]
 24%|██▍       | 188/780 [12:13<39:05,  3.96s/it]
                                                 
{'loss': 1.2259, 'grad_norm': 0.43639135360717773, 'learning_rate': 1.7766226591669787e-05, 'epoch': 2.41}

 24%|██▍       | 188/780 [12:13<39:05,  3.96s/it]
 24%|██▍       | 189/780 [12:17<38:25,  3.90s/it]
                                                 
{'loss': 1.2184, 'grad_norm': 0.4787863790988922, 'learning_rate': 1.7739981082662275e-05, 'epoch': 2.42}

 24%|██▍       | 189/780 [12:17<38:25,  3.90s/it]
 24%|██▍       | 190/780 [12:21<38:41,  3.93s/it]
                                                 
{'loss': 1.2606, 'grad_norm': 0.5844665169715881, 'learning_rate': 1.771360191551e-05, 'epoch': 2.43}

 24%|██▍       | 190/780 [12:21<38:41,  3.93s/it]
 24%|██▍       | 191/780 [12:24<38:06,  3.88s/it]
                                                 
{'loss': 1.2596, 'grad_norm': 0.5205034017562866, 'learning_rate': 1.768708954574256e-05, 'epoch': 2.44}

 24%|██▍       | 191/780 [12:24<38:06,  3.88s/it]
 25%|██▍       | 192/780 [12:28<38:43,  3.95s/it]
                                                 
{'loss': 1.2126, 'grad_norm': 0.4421405792236328, 'learning_rate': 1.766044443118978e-05, 'epoch': 2.46}

 25%|██▍       | 192/780 [12:28<38:43,  3.95s/it]
 25%|██▍       | 193/780 [12:32<38:14,  3.91s/it]
                                                 
{'loss': 1.3049, 'grad_norm': 0.4667174816131592, 'learning_rate': 1.7633667031973793e-05, 'epoch': 2.47}

 25%|██▍       | 193/780 [12:32<38:14,  3.91s/it]
 25%|██▍       | 194/780 [12:36<36:56,  3.78s/it]
                                                 
{'loss': 1.2373, 'grad_norm': 0.5170950293540955, 'learning_rate': 1.760675781050109e-05, 'epoch': 2.48}

 25%|██▍       | 194/780 [12:36<36:56,  3.78s/it]
 25%|██▌       | 195/780 [12:40<37:18,  3.83s/it]
                                                 
{'loss': 1.2517, 'grad_norm': 0.38633835315704346, 'learning_rate': 1.757971723145453e-05, 'epoch': 2.5}

 25%|██▌       | 195/780 [12:40<37:18,  3.83s/it]
 25%|██▌       | 196/780 [12:43<37:02,  3.81s/it]
                                                 
{'loss': 1.2199, 'grad_norm': 0.6587075591087341, 'learning_rate': 1.755254576178535e-05, 'epoch': 2.51}

 25%|██▌       | 196/780 [12:43<37:02,  3.81s/it]
 25%|██▌       | 197/780 [12:47<37:23,  3.85s/it]
                                                 
{'loss': 1.2321, 'grad_norm': 0.4671362042427063, 'learning_rate': 1.7525243870705052e-05, 'epoch': 2.52}

 25%|██▌       | 197/780 [12:47<37:23,  3.85s/it]
 25%|██▌       | 198/780 [12:51<38:00,  3.92s/it]
                                                 
{'loss': 1.2245, 'grad_norm': 0.4863879084587097, 'learning_rate': 1.7497812029677344e-05, 'epoch': 2.53}

 25%|██▌       | 198/780 [12:51<38:00,  3.92s/it]
 26%|██▌       | 199/780 [12:56<38:25,  3.97s/it]
                                                 
{'loss': 1.2247, 'grad_norm': 0.4958105683326721, 'learning_rate': 1.7470250712409963e-05, 'epoch': 2.55}

 26%|██▌       | 199/780 [12:56<38:25,  3.97s/it]
 26%|██▌       | 200/780 [12:59<36:19,  3.76s/it]
                                                 
{'loss': 1.2873, 'grad_norm': 0.46737784147262573, 'learning_rate': 1.7442560394846518e-05, 'epoch': 2.56}

 26%|██▌       | 200/780 [12:59<36:19,  3.76s/it]
 26%|██▌       | 201/780 [13:02<36:05,  3.74s/it]
                                                 
{'loss': 1.2632, 'grad_norm': 0.45198386907577515, 'learning_rate': 1.741474155515827e-05, 'epoch': 2.57}

 26%|██▌       | 201/780 [13:03<36:05,  3.74s/it]
 26%|██▌       | 202/780 [13:07<37:27,  3.89s/it]
                                                 
{'loss': 1.2335, 'grad_norm': 0.5192735195159912, 'learning_rate': 1.738679467373586e-05, 'epoch': 2.59}

 26%|██▌       | 202/780 [13:07<37:27,  3.89s/it]
 26%|██▌       | 203/780 [13:11<38:08,  3.97s/it]
                                                 
{'loss': 1.1629, 'grad_norm': 0.4582294225692749, 'learning_rate': 1.7358720233181023e-05, 'epoch': 2.6}

 26%|██▌       | 203/780 [13:11<38:08,  3.97s/it]
 26%|██▌       | 204/780 [13:15<38:04,  3.97s/it]
                                                 
{'loss': 1.227, 'grad_norm': 0.43635380268096924, 'learning_rate': 1.7330518718298263e-05, 'epoch': 2.61}

 26%|██▌       | 204/780 [13:15<38:04,  3.97s/it]
 26%|██▋       | 205/780 [13:19<39:03,  4.07s/it]
                                                 
{'loss': 1.215, 'grad_norm': 0.5394330620765686, 'learning_rate': 1.7302190616086464e-05, 'epoch': 2.62}

 26%|██▋       | 205/780 [13:19<39:03,  4.07s/it]
 26%|██▋       | 206/780 [13:23<37:28,  3.92s/it]
                                                 
{'loss': 1.2694, 'grad_norm': 0.5102443099021912, 'learning_rate': 1.7273736415730488e-05, 'epoch': 2.64}

 26%|██▋       | 206/780 [13:23<37:28,  3.92s/it]
 27%|██▋       | 207/780 [13:27<37:38,  3.94s/it]
                                                 
{'loss': 1.2084, 'grad_norm': 0.4720791280269623, 'learning_rate': 1.7245156608592727e-05, 'epoch': 2.65}

 27%|██▋       | 207/780 [13:27<37:38,  3.94s/it]
 27%|██▋       | 208/780 [13:30<36:41,  3.85s/it]
                                                 
{'loss': 1.2146, 'grad_norm': 0.4977085590362549, 'learning_rate': 1.7216451688204623e-05, 'epoch': 2.66}

 27%|██▋       | 208/780 [13:30<36:41,  3.85s/it]
 27%|██▋       | 209/780 [13:34<36:21,  3.82s/it]
                                                 
{'loss': 1.2157, 'grad_norm': 0.449845552444458, 'learning_rate': 1.718762215025813e-05, 'epoch': 2.68}

 27%|██▋       | 209/780 [13:34<36:21,  3.82s/it]
 27%|██▋       | 210/780 [13:37<34:50,  3.67s/it]
                                                 
{'loss': 1.2139, 'grad_norm': 0.4995660185813904, 'learning_rate': 1.7158668492597186e-05, 'epoch': 2.69}

 27%|██▋       | 210/780 [13:37<34:50,  3.67s/it]
 27%|██▋       | 211/780 [13:41<35:34,  3.75s/it]
                                                 
{'loss': 1.2064, 'grad_norm': 0.4904674291610718, 'learning_rate': 1.712959121520907e-05, 'epoch': 2.7}

 27%|██▋       | 211/780 [13:41<35:34,  3.75s/it]
 27%|██▋       | 212/780 [13:45<36:38,  3.87s/it]
                                                 
{'loss': 1.1692, 'grad_norm': 0.5018709897994995, 'learning_rate': 1.7100390820215805e-05, 'epoch': 2.71}

 27%|██▋       | 212/780 [13:46<36:38,  3.87s/it]
 27%|██▋       | 213/780 [13:50<37:53,  4.01s/it]
                                                 
{'loss': 1.2648, 'grad_norm': 0.506547212600708, 'learning_rate': 1.7071067811865477e-05, 'epoch': 2.73}

 27%|██▋       | 213/780 [13:50<37:53,  4.01s/it]
 27%|██▋       | 214/780 [13:53<36:19,  3.85s/it]
                                                 
{'loss': 1.1678, 'grad_norm': 0.49739277362823486, 'learning_rate': 1.704162269652352e-05, 'epoch': 2.74}

 27%|██▋       | 214/780 [13:53<36:19,  3.85s/it]
 28%|██▊       | 215/780 [13:57<35:18,  3.75s/it]
                                                 
{'loss': 1.2913, 'grad_norm': 0.46438801288604736, 'learning_rate': 1.701205598266398e-05, 'epoch': 2.75}

 28%|██▊       | 215/780 [13:57<35:18,  3.75s/it]
 28%|██▊       | 216/780 [14:01<36:16,  3.86s/it]
                                                 
{'loss': 1.2206, 'grad_norm': 0.4571724534034729, 'learning_rate': 1.698236818086073e-05, 'epoch': 2.76}

 28%|██▊       | 216/780 [14:01<36:16,  3.86s/it]
 28%|██▊       | 217/780 [14:05<36:17,  3.87s/it]
                                                 
{'loss': 1.3009, 'grad_norm': 0.6560295820236206, 'learning_rate': 1.6952559803778656e-05, 'epoch': 2.78}

 28%|██▊       | 217/780 [14:05<36:17,  3.87s/it]
 28%|██▊       | 218/780 [14:09<35:51,  3.83s/it]
                                                 
{'loss': 1.1879, 'grad_norm': 0.5015559792518616, 'learning_rate': 1.6922631366164795e-05, 'epoch': 2.79}

 28%|██▊       | 218/780 [14:09<35:51,  3.83s/it]
 28%|██▊       | 219/780 [14:12<35:45,  3.82s/it]
                                                 
{'loss': 1.1853, 'grad_norm': 0.4736906588077545, 'learning_rate': 1.689258338483947e-05, 'epoch': 2.8}

 28%|██▊       | 219/780 [14:12<35:45,  3.82s/it]
 28%|██▊       | 220/780 [14:16<35:49,  3.84s/it]
                                                 
{'loss': 1.24, 'grad_norm': 0.41783463954925537, 'learning_rate': 1.686241637868734e-05, 'epoch': 2.82}

 28%|██▊       | 220/780 [14:16<35:49,  3.84s/it]
 28%|██▊       | 221/780 [14:20<36:21,  3.90s/it]
                                                 
{'loss': 1.2769, 'grad_norm': 0.4477846026420593, 'learning_rate': 1.683213086864843e-05, 'epoch': 2.83}

 28%|██▊       | 221/780 [14:20<36:21,  3.90s/it]
 28%|██▊       | 222/780 [14:24<35:51,  3.86s/it]
                                                 
{'loss': 1.2482, 'grad_norm': 0.4727795124053955, 'learning_rate': 1.6801727377709195e-05, 'epoch': 2.84}

 28%|██▊       | 222/780 [14:24<35:51,  3.86s/it]
 29%|██▊       | 223/780 [14:28<35:44,  3.85s/it]
                                                 
{'loss': 1.169, 'grad_norm': 0.48518291115760803, 'learning_rate': 1.6771206430893408e-05, 'epoch': 2.85}

 29%|██▊       | 223/780 [14:28<35:44,  3.85s/it]
 29%|██▊       | 224/780 [14:32<36:25,  3.93s/it]
                                                 
{'loss': 1.2206, 'grad_norm': 0.4234623908996582, 'learning_rate': 1.6740568555253153e-05, 'epoch': 2.87}

 29%|██▊       | 224/780 [14:32<36:25,  3.93s/it]
 29%|██▉       | 225/780 [14:36<37:05,  4.01s/it]
                                                 
{'loss': 1.1943, 'grad_norm': 0.45817771553993225, 'learning_rate': 1.67098142798597e-05, 'epoch': 2.88}

 29%|██▉       | 225/780 [14:36<37:05,  4.01s/it]
 29%|██▉       | 226/780 [14:40<37:09,  4.02s/it]
                                                 
{'loss': 1.2386, 'grad_norm': 0.44292813539505005, 'learning_rate': 1.6678944135794375e-05, 'epoch': 2.89}

 29%|██▉       | 226/780 [14:40<37:09,  4.02s/it]
 29%|██▉       | 227/780 [14:44<37:02,  4.02s/it]
                                                 
{'loss': 1.1233, 'grad_norm': 0.4534141719341278, 'learning_rate': 1.6647958656139377e-05, 'epoch': 2.91}

 29%|██▉       | 227/780 [14:44<37:02,  4.02s/it]
 29%|██▉       | 228/780 [14:48<36:11,  3.93s/it]
                                                 
{'loss': 1.2436, 'grad_norm': 0.5540282130241394, 'learning_rate': 1.6616858375968596e-05, 'epoch': 2.92}

 29%|██▉       | 228/780 [14:48<36:11,  3.93s/it]
 29%|██▉       | 229/780 [14:52<35:07,  3.83s/it]
                                                 
{'loss': 1.2683, 'grad_norm': 0.5253776907920837, 'learning_rate': 1.6585643832338342e-05, 'epoch': 2.93}

 29%|██▉       | 229/780 [14:52<35:07,  3.83s/it]
 29%|██▉       | 230/780 [14:55<35:16,  3.85s/it]
                                                 
{'loss': 1.1257, 'grad_norm': 0.4797009229660034, 'learning_rate': 1.6554315564278102e-05, 'epoch': 2.94}

 29%|██▉       | 230/780 [14:55<35:16,  3.85s/it]
 30%|██▉       | 231/780 [14:59<33:48,  3.70s/it]
                                                 
{'loss': 1.135, 'grad_norm': 0.5700908899307251, 'learning_rate': 1.6522874112781213e-05, 'epoch': 2.96}

 30%|██▉       | 231/780 [14:59<33:48,  3.70s/it]
 30%|██▉       | 232/780 [15:02<33:31,  3.67s/it]
                                                 
{'loss': 1.2256, 'grad_norm': 0.5300338268280029, 'learning_rate': 1.649132002079552e-05, 'epoch': 2.97}

 30%|██▉       | 232/780 [15:02<33:31,  3.67s/it]
 30%|██▉       | 233/780 [15:07<34:43,  3.81s/it]
                                                 
{'loss': 1.1648, 'grad_norm': 0.550673246383667, 'learning_rate': 1.645965383321401e-05, 'epoch': 2.98}

 30%|██▉       | 233/780 [15:07<34:43,  3.81s/it]
 30%|███       | 234/780 [15:11<35:11,  3.87s/it]
                                                 
{'loss': 1.2553, 'grad_norm': 0.5105001926422119, 'learning_rate': 1.6427876096865394e-05, 'epoch': 3.0}

 30%|███       | 234/780 [15:11<35:11,  3.87s/it]
 30%|███       | 235/780 [15:15<36:41,  4.04s/it]
                                                 
{'loss': 1.2266, 'grad_norm': 0.45033350586891174, 'learning_rate': 1.6395987360504667e-05, 'epoch': 3.01}

 30%|███       | 235/780 [15:15<36:41,  4.04s/it]
 30%|███       | 236/780 [15:19<35:26,  3.91s/it]
                                                 
{'loss': 1.1608, 'grad_norm': 0.496236652135849, 'learning_rate': 1.6363988174803638e-05, 'epoch': 3.02}

 30%|███       | 236/780 [15:19<35:26,  3.91s/it]
 30%|███       | 237/780 [15:23<35:30,  3.92s/it]
                                                 
{'loss': 1.1867, 'grad_norm': 0.49169132113456726, 'learning_rate': 1.6331879092341402e-05, 'epoch': 3.03}

 30%|███       | 237/780 [15:23<35:30,  3.92s/it]
 31%|███       | 238/780 [15:26<34:39,  3.84s/it]
                                                 
{'loss': 1.2877, 'grad_norm': 0.4770122468471527, 'learning_rate': 1.6299660667594814e-05, 'epoch': 3.05}

 31%|███       | 238/780 [15:26<34:39,  3.84s/it]
 31%|███       | 239/780 [15:29<33:04,  3.67s/it]
                                                 
{'loss': 1.2958, 'grad_norm': 0.5025685429573059, 'learning_rate': 1.626733345692892e-05, 'epoch': 3.06}

 31%|███       | 239/780 [15:29<33:04,  3.67s/it]
 31%|███       | 240/780 [15:33<33:58,  3.77s/it]
                                                 
{'loss': 1.1531, 'grad_norm': 0.4358779489994049, 'learning_rate': 1.6234898018587336e-05, 'epoch': 3.07}

 31%|███       | 240/780 [15:34<33:58,  3.77s/it]
 31%|███       | 241/780 [15:38<35:09,  3.91s/it]
                                                 
{'loss': 1.1522, 'grad_norm': 0.49024999141693115, 'learning_rate': 1.6202354912682602e-05, 'epoch': 3.08}

 31%|███       | 241/780 [15:38<35:09,  3.91s/it]
 31%|███       | 242/780 [15:41<34:34,  3.86s/it]
                                                 
{'loss': 1.2472, 'grad_norm': 0.4907422959804535, 'learning_rate': 1.6169704701186528e-05, 'epoch': 3.1}

 31%|███       | 242/780 [15:41<34:34,  3.86s/it]
 31%|███       | 243/780 [15:45<34:55,  3.90s/it]
                                                 
{'loss': 1.3044, 'grad_norm': 0.43063098192214966, 'learning_rate': 1.6136947947920477e-05, 'epoch': 3.11}

 31%|███       | 243/780 [15:45<34:55,  3.90s/it]
 31%|███▏      | 244/780 [15:49<34:50,  3.90s/it]
                                                 
{'loss': 1.2134, 'grad_norm': 0.49654364585876465, 'learning_rate': 1.6104085218545633e-05, 'epoch': 3.12}

 31%|███▏      | 244/780 [15:49<34:50,  3.90s/it]
 31%|███▏      | 245/780 [15:54<35:31,  3.98s/it]
                                                 
{'loss': 1.1671, 'grad_norm': 0.48792529106140137, 'learning_rate': 1.6071117080553236e-05, 'epoch': 3.14}

 31%|███▏      | 245/780 [15:54<35:31,  3.98s/it]
 32%|███▏      | 246/780 [15:57<34:59,  3.93s/it]
                                                 
{'loss': 1.2608, 'grad_norm': 0.526256263256073, 'learning_rate': 1.6038044103254775e-05, 'epoch': 3.15}

 32%|███▏      | 246/780 [15:57<34:59,  3.93s/it]
 32%|███▏      | 247/780 [16:01<35:00,  3.94s/it]
                                                 
{'loss': 1.2285, 'grad_norm': 0.5720003247261047, 'learning_rate': 1.600486685777216e-05, 'epoch': 3.16}

 32%|███▏      | 247/780 [16:01<35:00,  3.94s/it]
 32%|███▏      | 248/780 [16:06<36:33,  4.12s/it]
                                                 
{'loss': 1.2147, 'grad_norm': 0.6253265738487244, 'learning_rate': 1.5971585917027864e-05, 'epoch': 3.17}

 32%|███▏      | 248/780 [16:06<36:33,  4.12s/it]
 32%|███▏      | 249/780 [16:09<35:08,  3.97s/it]
                                                 
{'loss': 1.1456, 'grad_norm': 0.5114706754684448, 'learning_rate': 1.5938201855735017e-05, 'epoch': 3.19}

 32%|███▏      | 249/780 [16:09<35:08,  3.97s/it]
 32%|███▏      | 250/780 [16:14<35:22,  4.00s/it]
                                                 
{'loss': 1.3058, 'grad_norm': 0.44837233424186707, 'learning_rate': 1.5904715250387498e-05, 'epoch': 3.2}

 32%|███▏      | 250/780 [16:14<35:22,  4.00s/it]
 32%|███▏      | 251/780 [16:17<34:49,  3.95s/it]
                                                 
{'loss': 1.2546, 'grad_norm': 0.40274301171302795, 'learning_rate': 1.5871126679249977e-05, 'epoch': 3.21}

 32%|███▏      | 251/780 [16:17<34:49,  3.95s/it]
 32%|███▏      | 252/780 [16:21<34:04,  3.87s/it]
                                                 
{'loss': 1.2514, 'grad_norm': 0.45038604736328125, 'learning_rate': 1.5837436722347902e-05, 'epoch': 3.23}

 32%|███▏      | 252/780 [16:21<34:04,  3.87s/it]
 32%|███▏      | 253/780 [16:25<33:54,  3.86s/it]
                                                 
{'loss': 1.1784, 'grad_norm': 0.5038169026374817, 'learning_rate': 1.5803645961457522e-05, 'epoch': 3.24}

 32%|███▏      | 253/780 [16:25<33:54,  3.86s/it]
 33%|███▎      | 254/780 [16:29<34:00,  3.88s/it]
                                                 
{'loss': 1.2066, 'grad_norm': 0.5209965705871582, 'learning_rate': 1.576975498009583e-05, 'epoch': 3.25}

 33%|███▎      | 254/780 [16:29<34:00,  3.88s/it]
 33%|███▎      | 255/780 [16:33<34:21,  3.93s/it]
                                                 
{'loss': 1.2296, 'grad_norm': 0.5141158699989319, 'learning_rate': 1.573576436351046e-05, 'epoch': 3.26}

 33%|███▎      | 255/780 [16:33<34:21,  3.93s/it]
 33%|███▎      | 256/780 [16:37<35:31,  4.07s/it]
                                                 
{'loss': 1.2199, 'grad_norm': 0.4487758278846741, 'learning_rate': 1.570167469866962e-05, 'epoch': 3.28}

 33%|███▎      | 256/780 [16:37<35:31,  4.07s/it]
 33%|███▎      | 257/780 [16:41<35:12,  4.04s/it]
                                                 
{'loss': 1.2012, 'grad_norm': 0.6586865782737732, 'learning_rate': 1.5667486574251916e-05, 'epoch': 3.29}

 33%|███▎      | 257/780 [16:41<35:12,  4.04s/it]
 33%|███▎      | 258/780 [16:45<33:56,  3.90s/it]
                                                 
{'loss': 1.2366, 'grad_norm': 0.505304753780365, 'learning_rate': 1.563320058063622e-05, 'epoch': 3.3}

 33%|███▎      | 258/780 [16:45<33:56,  3.90s/it]
 33%|███▎      | 259/780 [16:49<34:51,  4.02s/it]
                                                 
{'loss': 1.2071, 'grad_norm': 0.4690883159637451, 'learning_rate': 1.5598817309891466e-05, 'epoch': 3.32}

 33%|███▎      | 259/780 [16:49<34:51,  4.02s/it]
 33%|███▎      | 260/780 [16:53<35:11,  4.06s/it]
                                                 
{'loss': 1.2078, 'grad_norm': 0.4328030049800873, 'learning_rate': 1.5564337355766412e-05, 'epoch': 3.33}

 33%|███▎      | 260/780 [16:53<35:11,  4.06s/it]
 33%|███▎      | 261/780 [16:57<34:13,  3.96s/it]
                                                 
{'loss': 1.2062, 'grad_norm': 0.5271579623222351, 'learning_rate': 1.5529761313679396e-05, 'epoch': 3.34}

 33%|███▎      | 261/780 [16:57<34:13,  3.96s/it]
 34%|███▎      | 262/780 [17:01<33:04,  3.83s/it]
                                                 
{'loss': 1.2543, 'grad_norm': 0.42804601788520813, 'learning_rate': 1.5495089780708062e-05, 'epoch': 3.35}

 34%|███▎      | 262/780 [17:01<33:04,  3.83s/it]
 34%|███▎      | 263/780 [17:04<33:08,  3.85s/it]
                                                 
{'loss': 1.2089, 'grad_norm': 0.5992425680160522, 'learning_rate': 1.5460323355579035e-05, 'epoch': 3.37}

 34%|███▎      | 263/780 [17:04<33:08,  3.85s/it]
 34%|███▍      | 264/780 [17:08<33:15,  3.87s/it]
                                                 
{'loss': 1.221, 'grad_norm': 0.5059436559677124, 'learning_rate': 1.5425462638657597e-05, 'epoch': 3.38}

 34%|███▍      | 264/780 [17:08<33:15,  3.87s/it]
 34%|███▍      | 265/780 [17:12<33:48,  3.94s/it]
                                                 
{'loss': 1.2111, 'grad_norm': 0.6014003157615662, 'learning_rate': 1.53905082319373e-05, 'epoch': 3.39}

 34%|███▍      | 265/780 [17:12<33:48,  3.94s/it]
 34%|███▍      | 266/780 [17:16<33:38,  3.93s/it]
                                                 
{'loss': 1.1972, 'grad_norm': 0.42604997754096985, 'learning_rate': 1.5355460739029585e-05, 'epoch': 3.4}

 34%|███▍      | 266/780 [17:16<33:38,  3.93s/it]
 34%|███▍      | 267/780 [17:20<32:21,  3.78s/it]
                                                 
{'loss': 1.1904, 'grad_norm': 0.4851702153682709, 'learning_rate': 1.5320320765153367e-05, 'epoch': 3.42}

 34%|███▍      | 267/780 [17:20<32:21,  3.78s/it]
 34%|███▍      | 268/780 [17:24<32:54,  3.86s/it]
                                                 
{'loss': 1.2117, 'grad_norm': 0.5640307664871216, 'learning_rate': 1.5285088917124555e-05, 'epoch': 3.43}

 34%|███▍      | 268/780 [17:24<32:54,  3.86s/it]
 34%|███▍      | 269/780 [17:28<32:36,  3.83s/it]
                                                 
{'loss': 1.2356, 'grad_norm': 0.5147254467010498, 'learning_rate': 1.5249765803345602e-05, 'epoch': 3.44}

 34%|███▍      | 269/780 [17:28<32:36,  3.83s/it]
 35%|███▍      | 270/780 [17:31<32:42,  3.85s/it]
                                                 
{'loss': 1.2208, 'grad_norm': 0.4605024755001068, 'learning_rate': 1.5214352033794981e-05, 'epoch': 3.46}

 35%|███▍      | 270/780 [17:31<32:42,  3.85s/it]
 35%|███▍      | 271/780 [17:36<33:19,  3.93s/it]
                                                 
{'loss': 1.2763, 'grad_norm': 0.46426278352737427, 'learning_rate': 1.517884822001666e-05, 'epoch': 3.47}

 35%|███▍      | 271/780 [17:36<33:19,  3.93s/it]
 35%|███▍      | 272/780 [17:39<32:04,  3.79s/it]
                                                 
{'loss': 1.2155, 'grad_norm': 0.5430111885070801, 'learning_rate': 1.5143254975109538e-05, 'epoch': 3.48}

 35%|███▍      | 272/780 [17:39<32:04,  3.79s/it]
 35%|███▌      | 273/780 [17:43<32:14,  3.82s/it]
                                                 
{'loss': 1.2275, 'grad_norm': 0.38605982065200806, 'learning_rate': 1.5107572913716859e-05, 'epoch': 3.49}

 35%|███▌      | 273/780 [17:43<32:14,  3.82s/it]
 35%|███▌      | 274/780 [17:46<31:06,  3.69s/it]
                                                 
{'loss': 1.2288, 'grad_norm': 0.6844597458839417, 'learning_rate': 1.5071802652015592e-05, 'epoch': 3.51}

 35%|███▌      | 274/780 [17:46<31:06,  3.69s/it]
 35%|███▌      | 275/780 [17:50<31:56,  3.80s/it]
                                                 
{'loss': 1.1979, 'grad_norm': 0.5019146800041199, 'learning_rate': 1.503594480770581e-05, 'epoch': 3.52}

 35%|███▌      | 275/780 [17:50<31:56,  3.80s/it]
 35%|███▌      | 276/780 [17:54<32:42,  3.89s/it]
                                                 
{'loss': 1.1912, 'grad_norm': 0.44916802644729614, 'learning_rate': 1.5000000000000002e-05, 'epoch': 3.53}

 35%|███▌      | 276/780 [17:54<32:42,  3.89s/it]
 36%|███▌      | 277/780 [17:58<32:52,  3.92s/it]
                                                 
{'loss': 1.2229, 'grad_norm': 0.5118940472602844, 'learning_rate': 1.496396884961238e-05, 'epoch': 3.55}

 36%|███▌      | 277/780 [17:59<32:52,  3.92s/it]
 36%|███▌      | 278/780 [18:02<31:51,  3.81s/it]
                                                 
{'loss': 1.2416, 'grad_norm': 0.4932324290275574, 'learning_rate': 1.4927851978748177e-05, 'epoch': 3.56}

 36%|███▌      | 278/780 [18:02<31:51,  3.81s/it]
 36%|███▌      | 279/780 [18:06<31:28,  3.77s/it]
                                                 
{'loss': 1.2184, 'grad_norm': 0.45913541316986084, 'learning_rate': 1.4891650011092896e-05, 'epoch': 3.57}

 36%|███▌      | 279/780 [18:06<31:28,  3.77s/it]
 36%|███▌      | 280/780 [18:10<32:16,  3.87s/it]
                                                 
{'loss': 1.2142, 'grad_norm': 0.5389888882637024, 'learning_rate': 1.4855363571801523e-05, 'epoch': 3.58}

 36%|███▌      | 280/780 [18:10<32:16,  3.87s/it]
 36%|███▌      | 281/780 [18:14<33:00,  3.97s/it]
                                                 
{'loss': 1.1898, 'grad_norm': 0.46239346265792847, 'learning_rate': 1.481899328748776e-05, 'epoch': 3.6}

 36%|███▌      | 281/780 [18:14<33:00,  3.97s/it]
 36%|███▌      | 282/780 [18:18<33:10,  4.00s/it]
                                                 
{'loss': 1.1844, 'grad_norm': 0.42002221941947937, 'learning_rate': 1.4782539786213184e-05, 'epoch': 3.61}

 36%|███▌      | 282/780 [18:18<33:10,  4.00s/it]
 36%|███▋      | 283/780 [18:22<34:06,  4.12s/it]
                                                 
{'loss': 1.1967, 'grad_norm': 0.5765064358711243, 'learning_rate': 1.4746003697476406e-05, 'epoch': 3.62}

 36%|███▋      | 283/780 [18:22<34:06,  4.12s/it]
 36%|███▋      | 284/780 [18:26<33:10,  4.01s/it]
                                                 
{'loss': 1.2389, 'grad_norm': 0.5099313855171204, 'learning_rate': 1.4709385652202204e-05, 'epoch': 3.64}

 36%|███▋      | 284/780 [18:26<33:10,  4.01s/it]
 37%|███▋      | 285/780 [18:30<33:00,  4.00s/it]
                                                 
{'loss': 1.1969, 'grad_norm': 0.49488288164138794, 'learning_rate': 1.4672686282730622e-05, 'epoch': 3.65}

 37%|███▋      | 285/780 [18:30<33:00,  4.00s/it]
 37%|███▋      | 286/780 [18:34<31:45,  3.86s/it]
                                                 
{'loss': 1.2025, 'grad_norm': 0.4741697311401367, 'learning_rate': 1.4635906222806058e-05, 'epoch': 3.66}

 37%|███▋      | 286/780 [18:34<31:45,  3.86s/it]
 37%|███▋      | 287/780 [18:37<30:52,  3.76s/it]
                                                 
{'loss': 1.2253, 'grad_norm': 0.46828144788742065, 'learning_rate': 1.4599046107566314e-05, 'epoch': 3.67}

 37%|███▋      | 287/780 [18:37<30:52,  3.76s/it]
 37%|███▋      | 288/780 [18:41<30:59,  3.78s/it]
                                                 
{'loss': 1.1252, 'grad_norm': 0.5203940272331238, 'learning_rate': 1.4562106573531632e-05, 'epoch': 3.69}

 37%|███▋      | 288/780 [18:41<30:59,  3.78s/it]
 37%|███▋      | 289/780 [18:45<30:42,  3.75s/it]
                                                 
{'loss': 1.2093, 'grad_norm': 0.49996912479400635, 'learning_rate': 1.4525088258593695e-05, 'epoch': 3.7}

 37%|███▋      | 289/780 [18:45<30:42,  3.75s/it]
 37%|███▋      | 290/780 [18:49<31:22,  3.84s/it]
                                                 
{'loss': 1.1529, 'grad_norm': 0.5243548154830933, 'learning_rate': 1.4487991802004625e-05, 'epoch': 3.71}

 37%|███▋      | 290/780 [18:49<31:22,  3.84s/it]
 37%|███▋      | 291/780 [18:53<31:34,  3.87s/it]
                                                 
{'loss': 1.2442, 'grad_norm': 0.47955456376075745, 'learning_rate': 1.4450817844365924e-05, 'epoch': 3.72}

 37%|███▋      | 291/780 [18:53<31:34,  3.87s/it]
 37%|███▋      | 292/780 [18:57<31:35,  3.88s/it]
                                                 
{'loss': 1.1783, 'grad_norm': 0.5541942715644836, 'learning_rate': 1.4413567027617442e-05, 'epoch': 3.74}

 37%|███▋      | 292/780 [18:57<31:35,  3.88s/it]
 38%|███▊      | 293/780 [19:00<30:56,  3.81s/it]
                                                 
{'loss': 1.2053, 'grad_norm': 0.45206165313720703, 'learning_rate': 1.4376239995026254e-05, 'epoch': 3.75}

 38%|███▊      | 293/780 [19:00<30:56,  3.81s/it]
 38%|███▊      | 294/780 [19:04<31:48,  3.93s/it]
                                                 
{'loss': 1.2413, 'grad_norm': 0.45018434524536133, 'learning_rate': 1.4338837391175582e-05, 'epoch': 3.76}

 38%|███▊      | 294/780 [19:05<31:48,  3.93s/it]
 38%|███▊      | 295/780 [19:08<31:27,  3.89s/it]
                                                 
{'loss': 1.2758, 'grad_norm': 0.6881044507026672, 'learning_rate': 1.4301359861953652e-05, 'epoch': 3.78}

 38%|███▊      | 295/780 [19:08<31:27,  3.89s/it]
 38%|███▊      | 296/780 [19:12<31:24,  3.89s/it]
                                                 
{'loss': 1.1994, 'grad_norm': 0.4915850758552551, 'learning_rate': 1.4263808054542541e-05, 'epoch': 3.79}

 38%|███▊      | 296/780 [19:12<31:24,  3.89s/it]
 38%|███▊      | 297/780 [19:16<30:59,  3.85s/it]
                                                 
{'loss': 1.1574, 'grad_norm': 0.45896032452583313, 'learning_rate': 1.4226182617406996e-05, 'epoch': 3.8}

 38%|███▊      | 297/780 [19:16<30:59,  3.85s/it]
 38%|███▊      | 298/780 [19:20<30:55,  3.85s/it]
                                                 
{'loss': 1.2212, 'grad_norm': 0.4206894636154175, 'learning_rate': 1.418848420028325e-05, 'epoch': 3.81}

 38%|███▊      | 298/780 [19:20<30:55,  3.85s/it]
 38%|███▊      | 299/780 [19:24<31:00,  3.87s/it]
                                                 
{'loss': 1.2379, 'grad_norm': 0.4323002099990845, 'learning_rate': 1.4150713454167788e-05, 'epoch': 3.83}

 38%|███▊      | 299/780 [19:24<31:00,  3.87s/it]
 38%|███▊      | 300/780 [19:28<31:11,  3.90s/it]
                                                 
{'loss': 1.2149, 'grad_norm': 0.5016260743141174, 'learning_rate': 1.4112871031306118e-05, 'epoch': 3.84}

 38%|███▊      | 300/780 [19:28<31:11,  3.90s/it]
 39%|███▊      | 301/780 [19:32<31:14,  3.91s/it]
                                                 
{'loss': 1.1577, 'grad_norm': 0.49525129795074463, 'learning_rate': 1.4074957585181488e-05, 'epoch': 3.85}

 39%|███▊      | 301/780 [19:32<31:14,  3.91s/it]
 39%|███▊      | 302/780 [19:36<31:33,  3.96s/it]
                                                 
{'loss': 1.1956, 'grad_norm': 0.45004016160964966, 'learning_rate': 1.4036973770503623e-05, 'epoch': 3.87}

 39%|███▊      | 302/780 [19:36<31:33,  3.96s/it]
 39%|███▉      | 303/780 [19:40<32:18,  4.06s/it]
                                                 
{'loss': 1.2074, 'grad_norm': 0.42208248376846313, 'learning_rate': 1.3998920243197408e-05, 'epoch': 3.88}

 39%|███▉      | 303/780 [19:40<32:18,  4.06s/it]
 39%|███▉      | 304/780 [19:44<32:06,  4.05s/it]
                                                 
{'loss': 1.2151, 'grad_norm': 0.43345463275909424, 'learning_rate': 1.396079766039157e-05, 'epoch': 3.89}

 39%|███▉      | 304/780 [19:44<32:06,  4.05s/it]
 39%|███▉      | 305/780 [19:48<31:37,  4.00s/it]
                                                 
{'loss': 1.0905, 'grad_norm': 0.48241809010505676, 'learning_rate': 1.3922606680407307e-05, 'epoch': 3.9}

 39%|███▉      | 305/780 [19:48<31:37,  4.00s/it]
 39%|███▉      | 306/780 [19:52<31:30,  3.99s/it]
                                                 
{'loss': 1.2268, 'grad_norm': 0.49690142273902893, 'learning_rate': 1.3884347962746949e-05, 'epoch': 3.92}

 39%|███▉      | 306/780 [19:52<31:30,  3.99s/it]
 39%|███▉      | 307/780 [19:56<30:43,  3.90s/it]
                                                 
{'loss': 1.2619, 'grad_norm': 0.5409671664237976, 'learning_rate': 1.3846022168082553e-05, 'epoch': 3.93}

 39%|███▉      | 307/780 [19:56<30:43,  3.90s/it]
 39%|███▉      | 308/780 [19:59<30:37,  3.89s/it]
                                                 
{'loss': 1.1254, 'grad_norm': 0.5028075575828552, 'learning_rate': 1.3807629958244498e-05, 'epoch': 3.94}

 39%|███▉      | 308/780 [19:59<30:37,  3.89s/it]
 40%|███▉      | 309/780 [20:03<29:33,  3.76s/it]
                                                 
{'loss': 1.1019, 'grad_norm': 0.5539858937263489, 'learning_rate': 1.3769171996210053e-05, 'epoch': 3.96}

 40%|███▉      | 309/780 [20:03<29:33,  3.76s/it]
 40%|███▉      | 310/780 [20:06<28:48,  3.68s/it]
                                                 
{'loss': 1.2008, 'grad_norm': 0.5276826024055481, 'learning_rate': 1.373064894609194e-05, 'epoch': 3.97}

 40%|███▉      | 310/780 [20:06<28:48,  3.68s/it]
 40%|███▉      | 311/780 [20:10<29:31,  3.78s/it]
                                                 
{'loss': 1.1491, 'grad_norm': 0.5614553093910217, 'learning_rate': 1.3692061473126845e-05, 'epoch': 3.98}

 40%|███▉      | 311/780 [20:10<29:31,  3.78s/it]
 40%|████      | 312/780 [20:14<29:53,  3.83s/it]
                                                 
{'loss': 1.2375, 'grad_norm': 0.4896460175514221, 'learning_rate': 1.3653410243663953e-05, 'epoch': 3.99}

 40%|████      | 312/780 [20:14<29:53,  3.83s/it]
 40%|████      | 313/780 [20:19<31:14,  4.01s/it]
                                                 
{'loss': 1.2178, 'grad_norm': 0.42674142122268677, 'learning_rate': 1.361469592515342e-05, 'epoch': 4.01}

 40%|████      | 313/780 [20:19<31:14,  4.01s/it]
 40%|████      | 314/780 [20:22<30:02,  3.87s/it]
                                                 
{'loss': 1.1413, 'grad_norm': 0.5157901644706726, 'learning_rate': 1.3575919186134862e-05, 'epoch': 4.02}

 40%|████      | 314/780 [20:22<30:02,  3.87s/it]
 40%|████      | 315/780 [20:26<29:49,  3.85s/it]
                                                 
{'loss': 1.165, 'grad_norm': 0.5261910557746887, 'learning_rate': 1.3537080696225815e-05, 'epoch': 4.03}

 40%|████      | 315/780 [20:26<29:49,  3.85s/it]
 41%|████      | 316/780 [20:30<30:21,  3.93s/it]
                                                 
{'loss': 1.3005, 'grad_norm': 0.4293581545352936, 'learning_rate': 1.349818112611015e-05, 'epoch': 4.04}

 41%|████      | 316/780 [20:30<30:21,  3.93s/it]
 41%|████      | 317/780 [20:33<28:40,  3.72s/it]
                                                 
{'loss': 1.2526, 'grad_norm': 0.5045164227485657, 'learning_rate': 1.3459221147526504e-05, 'epoch': 4.06}

 41%|████      | 317/780 [20:33<28:40,  3.72s/it]
 41%|████      | 318/780 [20:37<28:57,  3.76s/it]
                                                 
{'loss': 1.1401, 'grad_norm': 0.4398958683013916, 'learning_rate': 1.342020143325669e-05, 'epoch': 4.07}

 41%|████      | 318/780 [20:37<28:57,  3.76s/it]
 41%|████      | 319/780 [20:42<30:12,  3.93s/it]
                                                 
{'loss': 1.1424, 'grad_norm': 0.49788421392440796, 'learning_rate': 1.3381122657114059e-05, 'epoch': 4.08}

 41%|████      | 319/780 [20:42<30:12,  3.93s/it]
 41%|████      | 320/780 [20:45<29:39,  3.87s/it]
                                                 
{'loss': 1.2296, 'grad_norm': 0.4994077682495117, 'learning_rate': 1.3341985493931877e-05, 'epoch': 4.1}

 41%|████      | 320/780 [20:45<29:39,  3.87s/it]
 41%|████      | 321/780 [20:49<29:47,  3.89s/it]
                                                 
{'loss': 1.2827, 'grad_norm': 0.4435155987739563, 'learning_rate': 1.3302790619551673e-05, 'epoch': 4.11}

 41%|████      | 321/780 [20:49<29:47,  3.89s/it]
 41%|████▏     | 322/780 [20:53<29:41,  3.89s/it]
                                                 
{'loss': 1.1777, 'grad_norm': 0.4471685290336609, 'learning_rate': 1.3263538710811559e-05, 'epoch': 4.12}

 41%|████▏     | 322/780 [20:53<29:41,  3.89s/it]
 41%|████▏     | 323/780 [20:57<30:01,  3.94s/it]
                                                 
{'loss': 1.1701, 'grad_norm': 0.5065972805023193, 'learning_rate': 1.3224230445534544e-05, 'epoch': 4.13}

 41%|████▏     | 323/780 [20:57<30:01,  3.94s/it]
 42%|████▏     | 324/780 [21:01<30:00,  3.95s/it]
                                                 
{'loss': 1.2625, 'grad_norm': 0.5669887661933899, 'learning_rate': 1.3184866502516846e-05, 'epoch': 4.15}

 42%|████▏     | 324/780 [21:01<30:00,  3.95s/it]
 42%|████▏     | 325/780 [21:05<29:14,  3.86s/it]
                                                 
{'loss': 1.1938, 'grad_norm': 0.5992783308029175, 'learning_rate': 1.3145447561516138e-05, 'epoch': 4.16}

 42%|████▏     | 325/780 [21:05<29:14,  3.86s/it]
 42%|████▏     | 326/780 [21:10<31:34,  4.17s/it]
                                                 
{'loss': 1.2079, 'grad_norm': 0.6240164637565613, 'learning_rate': 1.3105974303239838e-05, 'epoch': 4.17}

 42%|████▏     | 326/780 [21:10<31:34,  4.17s/it]
 42%|████▏     | 327/780 [21:13<30:09,  4.00s/it]
                                                 
{'loss': 1.1387, 'grad_norm': 0.49593058228492737, 'learning_rate': 1.3066447409333345e-05, 'epoch': 4.19}

 42%|████▏     | 327/780 [21:13<30:09,  4.00s/it]
 42%|████▏     | 328/780 [21:17<30:12,  4.01s/it]
                                                 
{'loss': 1.2752, 'grad_norm': 0.4538109600543976, 'learning_rate': 1.3026867562368262e-05, 'epoch': 4.2}

 42%|████▏     | 328/780 [21:17<30:12,  4.01s/it]
 42%|████▏     | 329/780 [21:21<29:43,  3.95s/it]
                                                 
{'loss': 1.2369, 'grad_norm': 0.39032983779907227, 'learning_rate': 1.2987235445830612e-05, 'epoch': 4.21}

 42%|████▏     | 329/780 [21:21<29:43,  3.95s/it]
 42%|████▏     | 330/780 [21:25<29:09,  3.89s/it]
                                                 
{'loss': 1.2431, 'grad_norm': 0.4435454308986664, 'learning_rate': 1.2947551744109044e-05, 'epoch': 4.22}

 42%|████▏     | 330/780 [21:25<29:09,  3.89s/it]
 42%|████▏     | 331/780 [21:29<28:57,  3.87s/it]
                                                 
{'loss': 1.1741, 'grad_norm': 0.5093862414360046, 'learning_rate': 1.2907817142483002e-05, 'epoch': 4.24}

 42%|████▏     | 331/780 [21:29<28:57,  3.87s/it]
 43%|████▎     | 332/780 [21:33<28:35,  3.83s/it]
                                                 
{'loss': 1.1663, 'grad_norm': 0.514805257320404, 'learning_rate': 1.2868032327110904e-05, 'epoch': 4.25}

 43%|████▎     | 332/780 [21:33<28:35,  3.83s/it]
 43%|████▎     | 333/780 [21:37<29:23,  3.95s/it]
                                                 
{'loss': 1.2099, 'grad_norm': 0.5088450908660889, 'learning_rate': 1.2828197985018276e-05, 'epoch': 4.26}

 43%|████▎     | 333/780 [21:37<29:23,  3.95s/it]
 43%|████▎     | 334/780 [21:41<29:31,  3.97s/it]
                                                 
{'loss': 1.2307, 'grad_norm': 0.4638059437274933, 'learning_rate': 1.2788314804085904e-05, 'epoch': 4.28}

 43%|████▎     | 334/780 [21:41<29:31,  3.97s/it]
 43%|████▎     | 335/780 [21:45<29:47,  4.02s/it]
                                                 
{'loss': 1.1753, 'grad_norm': 0.6349726915359497, 'learning_rate': 1.2748383473037948e-05, 'epoch': 4.29}

 43%|████▎     | 335/780 [21:45<29:47,  4.02s/it]
 43%|████▎     | 336/780 [21:48<28:45,  3.89s/it]
                                                 
{'loss': 1.2265, 'grad_norm': 0.46564632654190063, 'learning_rate': 1.2708404681430054e-05, 'epoch': 4.3}

 43%|████▎     | 336/780 [21:48<28:45,  3.89s/it]
 43%|████▎     | 337/780 [21:52<28:44,  3.89s/it]
                                                 
{'loss': 1.2119, 'grad_norm': 0.442722886800766, 'learning_rate': 1.266837911963743e-05, 'epoch': 4.31}

 43%|████▎     | 337/780 [21:52<28:44,  3.89s/it]
 43%|████▎     | 338/780 [21:57<29:26,  4.00s/it]
                                                 
{'loss': 1.1959, 'grad_norm': 0.43524354696273804, 'learning_rate': 1.2628307478842955e-05, 'epoch': 4.33}

 43%|████▎     | 338/780 [21:57<29:26,  4.00s/it]
 43%|████▎     | 339/780 [22:01<29:14,  3.98s/it]
                                                 
{'loss': 1.189, 'grad_norm': 0.5440450310707092, 'learning_rate': 1.2588190451025209e-05, 'epoch': 4.34}

 43%|████▎     | 339/780 [22:01<29:14,  3.98s/it]
 44%|████▎     | 340/780 [22:04<27:58,  3.81s/it]
                                                 
{'loss': 1.2309, 'grad_norm': 0.44059401750564575, 'learning_rate': 1.2548028728946548e-05, 'epoch': 4.35}

 44%|████▎     | 340/780 [22:04<27:58,  3.81s/it]
 44%|████▎     | 341/780 [22:08<27:42,  3.79s/it]
                                                 
{'loss': 1.24, 'grad_norm': 0.5854600667953491, 'learning_rate': 1.2507823006141128e-05, 'epoch': 4.36}

 44%|████▎     | 341/780 [22:08<27:42,  3.79s/it]
 44%|████▍     | 342/780 [22:12<28:16,  3.87s/it]
                                                 
{'loss': 1.1808, 'grad_norm': 0.528359055519104, 'learning_rate': 1.2467573976902936e-05, 'epoch': 4.38}

 44%|████▍     | 342/780 [22:12<28:16,  3.87s/it]
 44%|████▍     | 343/780 [22:16<29:10,  4.01s/it]
                                                 
{'loss': 1.1709, 'grad_norm': 0.6554008722305298, 'learning_rate': 1.24272823362738e-05, 'epoch': 4.39}

 44%|████▍     | 343/780 [22:16<29:10,  4.01s/it]
 44%|████▍     | 344/780 [22:20<28:27,  3.92s/it]
                                                 
{'loss': 1.2434, 'grad_norm': 0.43898895382881165, 'learning_rate': 1.238694878003138e-05, 'epoch': 4.4}

 44%|████▍     | 344/780 [22:20<28:27,  3.92s/it]
 44%|████▍     | 345/780 [22:24<27:57,  3.86s/it]
                                                 
{'loss': 1.1108, 'grad_norm': 0.49419060349464417, 'learning_rate': 1.2346574004677154e-05, 'epoch': 4.42}

 44%|████▍     | 345/780 [22:24<27:57,  3.86s/it]
 44%|████▍     | 346/780 [22:28<28:20,  3.92s/it]
                                                 
{'loss': 1.2214, 'grad_norm': 0.5133465528488159, 'learning_rate': 1.2306158707424402e-05, 'epoch': 4.43}

 44%|████▍     | 346/780 [22:28<28:20,  3.92s/it]
 44%|████▍     | 347/780 [22:31<27:54,  3.87s/it]
                                                 
{'loss': 1.2441, 'grad_norm': 0.5287652015686035, 'learning_rate': 1.2265703586186158e-05, 'epoch': 4.44}

 44%|████▍     | 347/780 [22:31<27:54,  3.87s/it]
 45%|████▍     | 348/780 [22:35<27:35,  3.83s/it]
                                                 
{'loss': 1.1919, 'grad_norm': 0.44744378328323364, 'learning_rate': 1.2225209339563144e-05, 'epoch': 4.45}

 45%|████▍     | 348/780 [22:35<27:35,  3.83s/it]
 45%|████▍     | 349/780 [22:39<28:24,  3.95s/it]
                                                 
{'loss': 1.212, 'grad_norm': 0.45681145787239075, 'learning_rate': 1.2184676666831741e-05, 'epoch': 4.47}

 45%|████▍     | 349/780 [22:39<28:24,  3.95s/it]
 45%|████▍     | 350/780 [22:43<27:17,  3.81s/it]
                                                 
{'loss': 1.2417, 'grad_norm': 0.5506539940834045, 'learning_rate': 1.2144106267931877e-05, 'epoch': 4.48}

 45%|████▍     | 350/780 [22:43<27:17,  3.81s/it]
 45%|████▌     | 351/780 [22:46<26:59,  3.77s/it]
                                                 
{'loss': 1.2238, 'grad_norm': 0.39644089341163635, 'learning_rate': 1.210349884345496e-05, 'epoch': 4.49}

 45%|████▌     | 351/780 [22:46<26:59,  3.77s/it]
 45%|████▌     | 352/780 [22:50<26:52,  3.77s/it]
                                                 
{'loss': 1.2363, 'grad_norm': 0.6618151068687439, 'learning_rate': 1.2062855094631777e-05, 'epoch': 4.51}

 45%|████▌     | 352/780 [22:50<26:52,  3.77s/it]
 45%|████▌     | 353/780 [22:54<26:52,  3.78s/it]
                                                 
{'loss': 1.177, 'grad_norm': 0.4956836402416229, 'learning_rate': 1.2022175723320382e-05, 'epoch': 4.52}

 45%|████▌     | 353/780 [22:54<26:52,  3.78s/it]
 45%|████▌     | 354/780 [22:58<27:35,  3.89s/it]
                                                 
{'loss': 1.1821, 'grad_norm': 0.4503241777420044, 'learning_rate': 1.1981461431993978e-05, 'epoch': 4.53}

 45%|████▌     | 354/780 [22:58<27:35,  3.89s/it]
 46%|████▌     | 355/780 [23:02<27:54,  3.94s/it]
                                                 
{'loss': 1.2063, 'grad_norm': 0.5069045424461365, 'learning_rate': 1.1940712923728784e-05, 'epoch': 4.54}

 46%|████▌     | 355/780 [23:02<27:54,  3.94s/it]
 46%|████▌     | 356/780 [23:06<26:37,  3.77s/it]
                                                 
{'loss': 1.2214, 'grad_norm': 0.49872294068336487, 'learning_rate': 1.1899930902191904e-05, 'epoch': 4.56}

 46%|████▌     | 356/780 [23:06<26:37,  3.77s/it]
 46%|████▌     | 357/780 [23:09<26:35,  3.77s/it]
                                                 
{'loss': 1.221, 'grad_norm': 0.46847131848335266, 'learning_rate': 1.1859116071629148e-05, 'epoch': 4.57}

 46%|████▌     | 357/780 [23:09<26:35,  3.77s/it]
 46%|████▌     | 358/780 [23:13<27:01,  3.84s/it]
                                                 
{'loss': 1.2019, 'grad_norm': 0.5168510675430298, 'learning_rate': 1.181826913685291e-05, 'epoch': 4.58}

 46%|████▌     | 358/780 [23:13<27:01,  3.84s/it]
 46%|████▌     | 359/780 [23:17<27:28,  3.92s/it]
                                                 
{'loss': 1.1738, 'grad_norm': 0.4551473557949066, 'learning_rate': 1.1777390803229964e-05, 'epoch': 4.6}

 46%|████▌     | 359/780 [23:17<27:28,  3.92s/it]
 46%|████▌     | 360/780 [23:22<28:19,  4.05s/it]
                                                 
{'loss': 1.1612, 'grad_norm': 0.4274875521659851, 'learning_rate': 1.1736481776669307e-05, 'epoch': 4.61}

 46%|████▌     | 360/780 [23:22<28:19,  4.05s/it]
 46%|████▋     | 361/780 [23:26<28:34,  4.09s/it]
                                                 
{'loss': 1.2163, 'grad_norm': 0.47905778884887695, 'learning_rate': 1.1695542763609944e-05, 'epoch': 4.62}

 46%|████▋     | 361/780 [23:26<28:34,  4.09s/it]
 46%|████▋     | 362/780 [23:30<27:51,  4.00s/it]
                                                 
{'loss': 1.2057, 'grad_norm': 0.5789040327072144, 'learning_rate': 1.1654574471008712e-05, 'epoch': 4.63}

 46%|████▋     | 362/780 [23:30<27:51,  4.00s/it]
 47%|████▋     | 363/780 [23:34<27:40,  3.98s/it]
                                                 
{'loss': 1.1834, 'grad_norm': 0.4942239820957184, 'learning_rate': 1.1613577606328068e-05, 'epoch': 4.65}

 47%|████▋     | 363/780 [23:34<27:40,  3.98s/it]
 47%|████▋     | 364/780 [23:37<26:52,  3.88s/it]
                                                 
{'loss': 1.1812, 'grad_norm': 0.47737932205200195, 'learning_rate': 1.1572552877523855e-05, 'epoch': 4.66}

 47%|████▋     | 364/780 [23:37<26:52,  3.88s/it]
 47%|████▋     | 365/780 [23:41<26:00,  3.76s/it]
                                                 
{'loss': 1.2089, 'grad_norm': 0.4671134948730469, 'learning_rate': 1.1531500993033094e-05, 'epoch': 4.67}

 47%|████▋     | 365/780 [23:41<26:00,  3.76s/it]
 47%|████▋     | 366/780 [23:44<25:32,  3.70s/it]
                                                 
{'loss': 1.1401, 'grad_norm': 0.520549476146698, 'learning_rate': 1.1490422661761744e-05, 'epoch': 4.68}

 47%|████▋     | 366/780 [23:44<25:32,  3.70s/it]
 47%|████▋     | 367/780 [23:48<25:44,  3.74s/it]
                                                 
{'loss': 1.1855, 'grad_norm': 0.4659210443496704, 'learning_rate': 1.1449318593072468e-05, 'epoch': 4.7}

 47%|████▋     | 367/780 [23:48<25:44,  3.74s/it]
 47%|████▋     | 368/780 [23:52<26:19,  3.83s/it]
                                                 
{'loss': 1.1588, 'grad_norm': 0.4940556585788727, 'learning_rate': 1.1408189496772369e-05, 'epoch': 4.71}

 47%|████▋     | 368/780 [23:52<26:19,  3.83s/it]
 47%|████▋     | 369/780 [23:56<26:43,  3.90s/it]
                                                 
{'loss': 1.2132, 'grad_norm': 0.4942217767238617, 'learning_rate': 1.1367036083100735e-05, 'epoch': 4.72}

 47%|████▋     | 369/780 [23:56<26:43,  3.90s/it]
 47%|████▋     | 370/780 [24:00<26:35,  3.89s/it]
                                                 
{'loss': 1.1738, 'grad_norm': 0.5398077368736267, 'learning_rate': 1.1325859062716795e-05, 'epoch': 4.74}

 47%|████▋     | 370/780 [24:00<26:35,  3.89s/it]
 48%|████▊     | 371/780 [24:04<25:58,  3.81s/it]
                                                 
{'loss': 1.1796, 'grad_norm': 0.5092344284057617, 'learning_rate': 1.1284659146687416e-05, 'epoch': 4.75}

 48%|████▊     | 371/780 [24:04<25:58,  3.81s/it]
 48%|████▊     | 372/780 [24:08<26:23,  3.88s/it]
                                                 
{'loss': 1.2558, 'grad_norm': 0.4379575252532959, 'learning_rate': 1.1243437046474854e-05, 'epoch': 4.76}

 48%|████▊     | 372/780 [24:08<26:23,  3.88s/it]
 48%|████▊     | 373/780 [24:12<26:46,  3.95s/it]
                                                 
{'loss': 1.2549, 'grad_norm': 0.6871075630187988, 'learning_rate': 1.120219347392444e-05, 'epoch': 4.77}

 48%|████▊     | 373/780 [24:12<26:46,  3.95s/it]
 48%|████▊     | 374/780 [24:16<26:31,  3.92s/it]
                                                 
{'loss': 1.1888, 'grad_norm': 0.4660513401031494, 'learning_rate': 1.1160929141252303e-05, 'epoch': 4.79}

 48%|████▊     | 374/780 [24:16<26:31,  3.92s/it]
 48%|████▊     | 375/780 [24:19<25:52,  3.83s/it]
                                                 
{'loss': 1.1649, 'grad_norm': 0.5086130499839783, 'learning_rate': 1.1119644761033079e-05, 'epoch': 4.8}

 48%|████▊     | 375/780 [24:20<25:52,  3.83s/it]
 48%|████▊     | 376/780 [24:23<25:43,  3.82s/it]
                                                 
{'loss': 1.1985, 'grad_norm': 0.4122834801673889, 'learning_rate': 1.1078341046187588e-05, 'epoch': 4.81}

 48%|████▊     | 376/780 [24:23<25:43,  3.82s/it]
 48%|████▊     | 377/780 [24:27<25:58,  3.87s/it]
                                                 
{'loss': 1.2265, 'grad_norm': 0.4351482689380646, 'learning_rate': 1.1037018709970528e-05, 'epoch': 4.83}

 48%|████▊     | 377/780 [24:27<25:58,  3.87s/it]
 48%|████▊     | 378/780 [24:31<26:03,  3.89s/it]
                                                 
{'loss': 1.2035, 'grad_norm': 0.505926251411438, 'learning_rate': 1.0995678465958168e-05, 'epoch': 4.84}

 48%|████▊     | 378/780 [24:31<26:03,  3.89s/it]
 49%|████▊     | 379/780 [24:35<25:47,  3.86s/it]
                                                 
{'loss': 1.1747, 'grad_norm': 0.4858967661857605, 'learning_rate': 1.0954321028036013e-05, 'epoch': 4.85}

 49%|████▊     | 379/780 [24:35<25:47,  3.86s/it]
 49%|████▊     | 380/780 [24:39<26:20,  3.95s/it]
                                                 
{'loss': 1.1567, 'grad_norm': 0.4058470129966736, 'learning_rate': 1.0912947110386484e-05, 'epoch': 4.86}

 49%|████▊     | 380/780 [24:39<26:20,  3.95s/it]
 49%|████▉     | 381/780 [24:43<26:31,  3.99s/it]
                                                 
{'loss': 1.1863, 'grad_norm': 0.4181601405143738, 'learning_rate': 1.0871557427476585e-05, 'epoch': 4.88}

 49%|████▉     | 381/780 [24:43<26:31,  3.99s/it]
 49%|████▉     | 382/780 [24:47<26:42,  4.03s/it]
                                                 
{'loss': 1.2189, 'grad_norm': 0.4486651122570038, 'learning_rate': 1.0830152694045553e-05, 'epoch': 4.89}

 49%|████▉     | 382/780 [24:47<26:42,  4.03s/it]
 49%|████▉     | 383/780 [24:51<26:31,  4.01s/it]
                                                 
{'loss': 1.1001, 'grad_norm': 0.4847702980041504, 'learning_rate': 1.078873362509254e-05, 'epoch': 4.9}

 49%|████▉     | 383/780 [24:51<26:31,  4.01s/it]
 49%|████▉     | 384/780 [24:55<26:09,  3.96s/it]
                                                 
{'loss': 1.2101, 'grad_norm': 0.4642614722251892, 'learning_rate': 1.0747300935864245e-05, 'epoch': 4.92}

 49%|████▉     | 384/780 [24:55<26:09,  3.96s/it]
 49%|████▉     | 385/780 [24:59<25:24,  3.86s/it]
                                                 
{'loss': 1.2378, 'grad_norm': 0.5510016679763794, 'learning_rate': 1.0705855341842564e-05, 'epoch': 4.93}

 49%|████▉     | 385/780 [24:59<25:24,  3.86s/it]
 49%|████▉     | 386/780 [25:03<25:16,  3.85s/it]
                                                 
{'loss': 1.1253, 'grad_norm': 0.5052350163459778, 'learning_rate': 1.0664397558732245e-05, 'epoch': 4.94}

 49%|████▉     | 386/780 [25:03<25:16,  3.85s/it]
 50%|████▉     | 387/780 [25:06<24:42,  3.77s/it]
                                                 
{'loss': 1.0907, 'grad_norm': 0.5232275724411011, 'learning_rate': 1.0622928302448523e-05, 'epoch': 4.95}

 50%|████▉     | 387/780 [25:06<24:42,  3.77s/it]
 50%|████▉     | 388/780 [25:10<24:01,  3.68s/it]
                                                 
{'loss': 1.1792, 'grad_norm': 0.5566892623901367, 'learning_rate': 1.0581448289104759e-05, 'epoch': 4.97}

 50%|████▉     | 388/780 [25:10<24:01,  3.68s/it]
 50%|████▉     | 389/780 [25:14<24:41,  3.79s/it]
                                                 
{'loss': 1.1458, 'grad_norm': 0.5295894145965576, 'learning_rate': 1.0539958235000075e-05, 'epoch': 4.98}

 50%|████▉     | 389/780 [25:14<24:41,  3.79s/it]
 50%|█████     | 390/780 [25:17<24:24,  3.76s/it]
                                                 
{'loss': 1.2367, 'grad_norm': 0.5572646260261536, 'learning_rate': 1.0498458856606972e-05, 'epoch': 4.99}

 50%|█████     | 390/780 [25:17<24:24,  3.76s/it]
 50%|█████     | 391/780 [25:22<26:07,  4.03s/it]
                                                 
{'loss': 1.1946, 'grad_norm': 0.41824108362197876, 'learning_rate': 1.0456950870558982e-05, 'epoch': 5.0}

 50%|█████     | 391/780 [25:22<26:07,  4.03s/it]
 50%|█████     | 392/780 [25:26<25:19,  3.92s/it]
                                                 
{'loss': 1.1518, 'grad_norm': 0.500536322593689, 'learning_rate': 1.0415434993638269e-05, 'epoch': 5.02}

 50%|█████     | 392/780 [25:26<25:19,  3.92s/it]
 50%|█████     | 393/780 [25:29<24:55,  3.86s/it]
                                                 
{'loss': 1.1407, 'grad_norm': 0.5368762016296387, 'learning_rate': 1.037391194276326e-05, 'epoch': 5.03}

 50%|█████     | 393/780 [25:29<24:55,  3.86s/it]
 51%|█████     | 394/780 [25:33<24:56,  3.88s/it]
                                                 
{'loss': 1.2625, 'grad_norm': 0.44026073813438416, 'learning_rate': 1.0332382434976267e-05, 'epoch': 5.04}

 51%|█████     | 394/780 [25:33<24:56,  3.88s/it]
 51%|█████     | 395/780 [25:37<23:51,  3.72s/it]
                                                 
{'loss': 1.2626, 'grad_norm': 0.4811384379863739, 'learning_rate': 1.0290847187431115e-05, 'epoch': 5.06}

 51%|█████     | 395/780 [25:37<23:51,  3.72s/it]
 51%|█████     | 396/780 [25:41<23:56,  3.74s/it]
                                                 
{'loss': 1.1513, 'grad_norm': 0.44567859172821045, 'learning_rate': 1.0249306917380731e-05, 'epoch': 5.07}

 51%|█████     | 396/780 [25:41<23:56,  3.74s/it]
 51%|█████     | 397/780 [25:45<24:39,  3.86s/it]
                                                 
{'loss': 1.1218, 'grad_norm': 0.4323476254940033, 'learning_rate': 1.0207762342164778e-05, 'epoch': 5.08}

 51%|█████     | 397/780 [25:45<24:39,  3.86s/it]
 51%|█████     | 398/780 [25:48<24:32,  3.85s/it]
                                                 
{'loss': 1.2321, 'grad_norm': 0.5209019780158997, 'learning_rate': 1.0166214179197265e-05, 'epoch': 5.09}

 51%|█████     | 398/780 [25:49<24:32,  3.85s/it]
 51%|█████     | 399/780 [25:53<25:02,  3.94s/it]
                                                 
{'loss': 1.2434, 'grad_norm': 0.45777398347854614, 'learning_rate': 1.0124663145954152e-05, 'epoch': 5.11}

 51%|█████     | 399/780 [25:53<25:02,  3.94s/it]
 51%|█████▏    | 400/780 [25:56<24:13,  3.82s/it]
                                                 
{'loss': 1.2037, 'grad_norm': 0.46146896481513977, 'learning_rate': 1.0083109959960974e-05, 'epoch': 5.12}

 51%|█████▏    | 400/780 [25:56<24:13,  3.82s/it]
 51%|█████▏    | 401/780 [26:00<25:04,  3.97s/it]
                                                 
{'loss': 1.1307, 'grad_norm': 0.5090178847312927, 'learning_rate': 1.0041555338780427e-05, 'epoch': 5.13}

 51%|█████▏    | 401/780 [26:01<25:04,  3.97s/it]
 52%|█████▏    | 402/780 [26:05<25:16,  4.01s/it]
                                                 
{'loss': 1.276, 'grad_norm': 0.5634288191795349, 'learning_rate': 1e-05, 'epoch': 5.15}

 52%|█████▏    | 402/780 [26:05<25:16,  4.01s/it]
 52%|█████▏    | 403/780 [26:08<24:44,  3.94s/it]
                                                 
{'loss': 1.1755, 'grad_norm': 0.6042029857635498, 'learning_rate': 9.958444661219578e-06, 'epoch': 5.16}

 52%|█████▏    | 403/780 [26:08<24:44,  3.94s/it]
 52%|█████▏    | 404/780 [26:13<26:06,  4.16s/it]
                                                 
{'loss': 1.2363, 'grad_norm': 0.5898411273956299, 'learning_rate': 9.916890040039031e-06, 'epoch': 5.17}

 52%|█████▏    | 404/780 [26:13<26:06,  4.16s/it]
 52%|█████▏    | 405/780 [26:17<25:38,  4.10s/it]
                                                 
{'loss': 1.1254, 'grad_norm': 0.5726839900016785, 'learning_rate': 9.87533685404585e-06, 'epoch': 5.18}

 52%|█████▏    | 405/780 [26:17<25:38,  4.10s/it]
 52%|█████▏    | 406/780 [26:21<25:15,  4.05s/it]
                                                 
{'loss': 1.2343, 'grad_norm': 0.46279504895210266, 'learning_rate': 9.833785820802739e-06, 'epoch': 5.2}

 52%|█████▏    | 406/780 [26:21<25:15,  4.05s/it]
 52%|█████▏    | 407/780 [26:25<25:05,  4.04s/it]
                                                 
{'loss': 1.2602, 'grad_norm': 0.38685429096221924, 'learning_rate': 9.792237657835225e-06, 'epoch': 5.21}

 52%|█████▏    | 407/780 [26:25<25:05,  4.04s/it]
 52%|█████▏    | 408/780 [26:28<23:58,  3.87s/it]
                                                 
{'loss': 1.2094, 'grad_norm': 0.43010827898979187, 'learning_rate': 9.750693082619274e-06, 'epoch': 5.22}

 52%|█████▏    | 408/780 [26:28<23:58,  3.87s/it]
 52%|█████▏    | 409/780 [26:32<24:06,  3.90s/it]
                                                 
{'loss': 1.158, 'grad_norm': 0.5190458297729492, 'learning_rate': 9.709152812568886e-06, 'epoch': 5.24}

 52%|█████▏    | 409/780 [26:32<24:06,  3.90s/it]
 53%|█████▎    | 410/780 [26:36<23:47,  3.86s/it]
                                                 
{'loss': 1.1682, 'grad_norm': 0.49445948004722595, 'learning_rate': 9.667617565023734e-06, 'epoch': 5.25}

 53%|█████▎    | 410/780 [26:36<23:47,  3.86s/it]
 53%|█████▎    | 411/780 [26:40<24:34,  3.99s/it]
                                                 
{'loss': 1.1928, 'grad_norm': 0.5076707601547241, 'learning_rate': 9.626088057236745e-06, 'epoch': 5.26}

 53%|█████▎    | 411/780 [26:41<24:34,  3.99s/it]
 53%|█████▎    | 412/780 [26:45<24:37,  4.01s/it]
                                                 
{'loss': 1.2093, 'grad_norm': 0.45818087458610535, 'learning_rate': 9.584565006361735e-06, 'epoch': 5.27}

 53%|█████▎    | 412/780 [26:45<24:37,  4.01s/it]
 53%|█████▎    | 413/780 [26:49<24:44,  4.04s/it]
                                                 
{'loss': 1.2191, 'grad_norm': 0.6467769742012024, 'learning_rate': 9.543049129441021e-06, 'epoch': 5.29}

 53%|█████▎    | 413/780 [26:49<24:44,  4.04s/it]
 53%|█████▎    | 414/780 [26:52<24:00,  3.94s/it]
                                                 
{'loss': 1.1844, 'grad_norm': 0.4444418251514435, 'learning_rate': 9.501541143393028e-06, 'epoch': 5.3}

 53%|█████▎    | 414/780 [26:52<24:00,  3.94s/it]
 53%|█████▎    | 415/780 [26:56<23:40,  3.89s/it]
                                                 
{'loss': 1.2165, 'grad_norm': 0.4521312713623047, 'learning_rate': 9.460041764999929e-06, 'epoch': 5.31}

 53%|█████▎    | 415/780 [26:56<23:40,  3.89s/it]
 53%|█████▎    | 416/780 [27:00<24:19,  4.01s/it]
                                                 
{'loss': 1.1851, 'grad_norm': 0.43824565410614014, 'learning_rate': 9.418551710895243e-06, 'epoch': 5.32}

 53%|█████▎    | 416/780 [27:00<24:19,  4.01s/it]
 53%|█████▎    | 417/780 [27:04<24:04,  3.98s/it]
                                                 
{'loss': 1.1875, 'grad_norm': 0.5088557600975037, 'learning_rate': 9.377071697551479e-06, 'epoch': 5.34}

 53%|█████▎    | 417/780 [27:04<24:04,  3.98s/it]
 54%|█████▎    | 418/780 [27:08<22:45,  3.77s/it]
                                                 
{'loss': 1.2084, 'grad_norm': 0.42501482367515564, 'learning_rate': 9.33560244126776e-06, 'epoch': 5.35}

 54%|█████▎    | 418/780 [27:08<22:45,  3.77s/it]
 54%|█████▎    | 419/780 [27:11<22:52,  3.80s/it]
                                                 
{'loss': 1.2317, 'grad_norm': 0.5430695414543152, 'learning_rate': 9.294144658157443e-06, 'epoch': 5.36}

 54%|█████▎    | 419/780 [27:12<22:52,  3.80s/it]
 54%|█████▍    | 420/780 [27:15<23:06,  3.85s/it]
                                                 
{'loss': 1.1722, 'grad_norm': 0.5001957416534424, 'learning_rate': 9.252699064135759e-06, 'epoch': 5.38}

 54%|█████▍    | 420/780 [27:15<23:06,  3.85s/it]
 54%|█████▍    | 421/780 [27:20<23:46,  3.97s/it]
                                                 
{'loss': 1.1726, 'grad_norm': 0.6976733803749084, 'learning_rate': 9.211266374907463e-06, 'epoch': 5.39}

 54%|█████▍    | 421/780 [27:20<23:46,  3.97s/it]
 54%|█████▍    | 422/780 [27:23<23:03,  3.86s/it]
                                                 
{'loss': 1.2233, 'grad_norm': 0.4364541471004486, 'learning_rate': 9.169847305954448e-06, 'epoch': 5.4}

 54%|█████▍    | 422/780 [27:23<23:03,  3.86s/it]
 54%|█████▍    | 423/780 [27:27<22:38,  3.81s/it]
                                                 
{'loss': 1.1004, 'grad_norm': 0.4767462909221649, 'learning_rate': 9.128442572523418e-06, 'epoch': 5.41}

 54%|█████▍    | 423/780 [27:27<22:38,  3.81s/it]
 54%|█████▍    | 424/780 [27:31<23:28,  3.96s/it]
                                                 
{'loss': 1.2241, 'grad_norm': 0.4986206591129303, 'learning_rate': 9.087052889613519e-06, 'epoch': 5.43}

 54%|█████▍    | 424/780 [27:31<23:28,  3.96s/it]
 54%|█████▍    | 425/780 [27:35<22:57,  3.88s/it]
                                                 
{'loss': 1.2607, 'grad_norm': 0.5355969071388245, 'learning_rate': 9.045678971963988e-06, 'epoch': 5.44}

 54%|█████▍    | 425/780 [27:35<22:57,  3.88s/it]
 55%|█████▍    | 426/780 [27:39<22:40,  3.84s/it]
                                                 
{'loss': 1.178, 'grad_norm': 0.45814862847328186, 'learning_rate': 9.004321534041836e-06, 'epoch': 5.45}

 55%|█████▍    | 426/780 [27:39<22:40,  3.84s/it]
 55%|█████▍    | 427/780 [27:43<23:25,  3.98s/it]
                                                 
{'loss': 1.1774, 'grad_norm': 0.43733665347099304, 'learning_rate': 8.962981290029475e-06, 'epoch': 5.47}

 55%|█████▍    | 427/780 [27:43<23:25,  3.98s/it]
 55%|█████▍    | 428/780 [27:47<22:29,  3.83s/it]
                                                 
{'loss': 1.2392, 'grad_norm': 0.5387746095657349, 'learning_rate': 8.921658953812416e-06, 'epoch': 5.48}

 55%|█████▍    | 428/780 [27:47<22:29,  3.83s/it]
 55%|█████▌    | 429/780 [27:50<22:19,  3.82s/it]
                                                 
{'loss': 1.2319, 'grad_norm': 0.41320228576660156, 'learning_rate': 8.880355238966923e-06, 'epoch': 5.49}

 55%|█████▌    | 429/780 [27:50<22:19,  3.82s/it]
 55%|█████▌    | 430/780 [27:54<21:58,  3.77s/it]
                                                 
{'loss': 1.1967, 'grad_norm': 0.5858824849128723, 'learning_rate': 8.839070858747697e-06, 'epoch': 5.5}

 55%|█████▌    | 430/780 [27:54<21:58,  3.77s/it]
 55%|█████▌    | 431/780 [27:58<21:42,  3.73s/it]
                                                 
{'loss': 1.1861, 'grad_norm': 0.5484943389892578, 'learning_rate': 8.797806526075566e-06, 'epoch': 5.52}

 55%|█████▌    | 431/780 [27:58<21:42,  3.73s/it]
 55%|█████▌    | 432/780 [28:02<22:16,  3.84s/it]
                                                 
{'loss': 1.1854, 'grad_norm': 0.45074090361595154, 'learning_rate': 8.756562953525151e-06, 'epoch': 5.53}

 55%|█████▌    | 432/780 [28:02<22:16,  3.84s/it]
 56%|█████▌    | 433/780 [28:06<22:58,  3.97s/it]
                                                 
{'loss': 1.2219, 'grad_norm': 0.470214307308197, 'learning_rate': 8.715340853312586e-06, 'epoch': 5.54}

 56%|█████▌    | 433/780 [28:06<22:58,  3.97s/it]
 56%|█████▌    | 434/780 [28:09<21:57,  3.81s/it]
                                                 
{'loss': 1.1871, 'grad_norm': 0.5229299664497375, 'learning_rate': 8.674140937283208e-06, 'epoch': 5.56}

 56%|█████▌    | 434/780 [28:09<21:57,  3.81s/it]
 56%|█████▌    | 435/780 [28:13<21:44,  3.78s/it]
                                                 
{'loss': 1.2273, 'grad_norm': 0.4963834881782532, 'learning_rate': 8.632963916899268e-06, 'epoch': 5.57}

 56%|█████▌    | 435/780 [28:13<21:44,  3.78s/it]
 56%|█████▌    | 436/780 [28:17<21:20,  3.72s/it]
                                                 
{'loss': 1.1942, 'grad_norm': 0.48549607396125793, 'learning_rate': 8.591810503227634e-06, 'epoch': 5.58}

 56%|█████▌    | 436/780 [28:17<21:20,  3.72s/it]
 56%|█████▌    | 437/780 [28:21<22:10,  3.88s/it]
                                                 
{'loss': 1.1721, 'grad_norm': 0.4591106176376343, 'learning_rate': 8.550681406927534e-06, 'epoch': 5.59}

 56%|█████▌    | 437/780 [28:21<22:10,  3.88s/it]
 56%|█████▌    | 438/780 [28:25<22:52,  4.01s/it]
                                                 
{'loss': 1.159, 'grad_norm': 0.45971083641052246, 'learning_rate': 8.509577338238255e-06, 'epoch': 5.61}

 56%|█████▌    | 438/780 [28:25<22:52,  4.01s/it]
 56%|█████▋    | 439/780 [28:30<23:15,  4.09s/it]
                                                 
{'loss': 1.1932, 'grad_norm': 0.4766492545604706, 'learning_rate': 8.46849900696691e-06, 'epoch': 5.62}

 56%|█████▋    | 439/780 [28:30<23:15,  4.09s/it]
 56%|█████▋    | 440/780 [28:33<22:16,  3.93s/it]
                                                 
{'loss': 1.2003, 'grad_norm': 0.5537105202674866, 'learning_rate': 8.427447122476148e-06, 'epoch': 5.63}

 56%|█████▋    | 440/780 [28:33<22:16,  3.93s/it]
 57%|█████▋    | 441/780 [28:37<22:24,  3.97s/it]
                                                 
{'loss': 1.1742, 'grad_norm': 0.5580787658691406, 'learning_rate': 8.386422393671934e-06, 'epoch': 5.64}

 57%|█████▋    | 441/780 [28:37<22:24,  3.97s/it]
 57%|█████▋    | 442/780 [28:41<21:42,  3.85s/it]
                                                 
{'loss': 1.1701, 'grad_norm': 0.4389805197715759, 'learning_rate': 8.34542552899129e-06, 'epoch': 5.66}

 57%|█████▋    | 442/780 [28:41<21:42,  3.85s/it]
 57%|█████▋    | 443/780 [28:44<20:48,  3.70s/it]
                                                 
{'loss': 1.2201, 'grad_norm': 0.5271732211112976, 'learning_rate': 8.304457236390062e-06, 'epoch': 5.67}

 57%|█████▋    | 443/780 [28:44<20:48,  3.70s/it]
 57%|█████▋    | 444/780 [28:48<20:50,  3.72s/it]
                                                 
{'loss': 1.1329, 'grad_norm': 0.49611207842826843, 'learning_rate': 8.263518223330698e-06, 'epoch': 5.68}

 57%|█████▋    | 444/780 [28:48<20:50,  3.72s/it]
 57%|█████▋    | 445/780 [28:52<20:44,  3.72s/it]
                                                 
{'loss': 1.1879, 'grad_norm': 0.4851504862308502, 'learning_rate': 8.222609196770037e-06, 'epoch': 5.7}

 57%|█████▋    | 445/780 [28:52<20:44,  3.72s/it]
 57%|█████▋    | 446/780 [28:56<21:19,  3.83s/it]
                                                 
{'loss': 1.1333, 'grad_norm': 0.5168033838272095, 'learning_rate': 8.181730863147094e-06, 'epoch': 5.71}

 57%|█████▋    | 446/780 [28:56<21:19,  3.83s/it]
 57%|█████▋    | 447/780 [29:00<21:20,  3.84s/it]
                                                 
{'loss': 1.1898, 'grad_norm': 0.5198140740394592, 'learning_rate': 8.140883928370855e-06, 'epoch': 5.72}

 57%|█████▋    | 447/780 [29:00<21:20,  3.84s/it]
 57%|█████▋    | 448/780 [29:04<21:34,  3.90s/it]
                                                 
{'loss': 1.1894, 'grad_norm': 0.5079969167709351, 'learning_rate': 8.100069097808103e-06, 'epoch': 5.73}

 57%|█████▋    | 448/780 [29:04<21:34,  3.90s/it]
 58%|█████▊    | 449/780 [29:07<20:51,  3.78s/it]
                                                 
{'loss': 1.1767, 'grad_norm': 0.5118720531463623, 'learning_rate': 8.059287076271216e-06, 'epoch': 5.75}

 58%|█████▊    | 449/780 [29:07<20:51,  3.78s/it]
 58%|█████▊    | 450/780 [29:11<21:21,  3.88s/it]
                                                 
{'loss': 1.2389, 'grad_norm': 0.4438035488128662, 'learning_rate': 8.018538568006027e-06, 'epoch': 5.76}

 58%|█████▊    | 450/780 [29:11<21:21,  3.88s/it]
 58%|█████▊    | 451/780 [29:15<21:37,  3.94s/it]
                                                 
{'loss': 1.2634, 'grad_norm': 0.649228572845459, 'learning_rate': 7.977824276679623e-06, 'epoch': 5.77}

 58%|█████▊    | 451/780 [29:15<21:37,  3.94s/it]
 58%|█████▊    | 452/780 [29:19<20:59,  3.84s/it]
                                                 
{'loss': 1.1678, 'grad_norm': 0.4925893247127533, 'learning_rate': 7.937144905368226e-06, 'epoch': 5.79}

 58%|█████▊    | 452/780 [29:19<20:59,  3.84s/it]
 58%|█████▊    | 453/780 [29:23<20:53,  3.83s/it]
                                                 
{'loss': 1.165, 'grad_norm': 0.49709486961364746, 'learning_rate': 7.896501156545044e-06, 'epoch': 5.8}

 58%|█████▊    | 453/780 [29:23<20:53,  3.83s/it]
 58%|█████▊    | 454/780 [29:27<20:47,  3.83s/it]
                                                 
{'loss': 1.1831, 'grad_norm': 0.437168151140213, 'learning_rate': 7.855893732068124e-06, 'epoch': 5.81}

 58%|█████▊    | 454/780 [29:27<20:47,  3.83s/it]
 58%|█████▊    | 455/780 [29:31<20:59,  3.88s/it]
                                                 
{'loss': 1.2281, 'grad_norm': 0.41092976927757263, 'learning_rate': 7.815323333168262e-06, 'epoch': 5.82}

 58%|█████▊    | 455/780 [29:31<20:59,  3.88s/it]
 58%|█████▊    | 456/780 [29:34<20:50,  3.86s/it]
                                                 
{'loss': 1.1782, 'grad_norm': 0.5010285973548889, 'learning_rate': 7.774790660436857e-06, 'epoch': 5.84}

 58%|█████▊    | 456/780 [29:34<20:50,  3.86s/it]
 59%|█████▊    | 457/780 [29:38<20:34,  3.82s/it]
                                                 
{'loss': 1.19, 'grad_norm': 0.4737476110458374, 'learning_rate': 7.734296413813847e-06, 'epoch': 5.85}

 59%|█████▊    | 457/780 [29:38<20:34,  3.82s/it]
 59%|█████▊    | 458/780 [29:42<21:25,  3.99s/it]
                                                 
{'loss': 1.1285, 'grad_norm': 0.39411869645118713, 'learning_rate': 7.6938412925756e-06, 'epoch': 5.86}

 59%|█████▊    | 458/780 [29:42<21:25,  3.99s/it]
 59%|█████▉    | 459/780 [29:46<21:08,  3.95s/it]
                                                 
{'loss': 1.1703, 'grad_norm': 0.4222368896007538, 'learning_rate': 7.653425995322852e-06, 'epoch': 5.88}

 59%|█████▉    | 459/780 [29:46<21:08,  3.95s/it]
 59%|█████▉    | 460/780 [29:51<21:44,  4.08s/it]
                                                 
{'loss': 1.2433, 'grad_norm': 0.45372700691223145, 'learning_rate': 7.613051219968624e-06, 'epoch': 5.89}

 59%|█████▉    | 460/780 [29:51<21:44,  4.08s/it]
 59%|█████▉    | 461/780 [29:55<21:19,  4.01s/it]
                                                 
{'loss': 1.0863, 'grad_norm': 0.5028586983680725, 'learning_rate': 7.5727176637262034e-06, 'epoch': 5.9}

 59%|█████▉    | 461/780 [29:55<21:19,  4.01s/it]
 59%|█████▉    | 462/780 [29:59<21:12,  4.00s/it]
                                                 
{'loss': 1.2143, 'grad_norm': 0.4836425185203552, 'learning_rate': 7.532426023097063e-06, 'epoch': 5.91}

 59%|█████▉    | 462/780 [29:59<21:12,  4.00s/it]
 59%|█████▉    | 463/780 [30:02<20:28,  3.87s/it]
                                                 
{'loss': 1.2031, 'grad_norm': 0.5429538488388062, 'learning_rate': 7.492176993858873e-06, 'epoch': 5.93}

 59%|█████▉    | 463/780 [30:02<20:28,  3.87s/it]
 59%|█████▉    | 464/780 [30:06<20:18,  3.85s/it]
                                                 
{'loss': 1.1545, 'grad_norm': 0.4943886399269104, 'learning_rate': 7.451971271053455e-06, 'epoch': 5.94}

 59%|█████▉    | 464/780 [30:06<20:18,  3.85s/it]
 60%|█████▉    | 465/780 [30:09<19:37,  3.74s/it]
                                                 
{'loss': 1.0939, 'grad_norm': 0.5322587490081787, 'learning_rate': 7.411809548974792e-06, 'epoch': 5.95}

 60%|█████▉    | 465/780 [30:09<19:37,  3.74s/it]
 60%|█████▉    | 466/780 [30:13<19:23,  3.71s/it]
                                                 
{'loss': 1.1311, 'grad_norm': 0.5510344505310059, 'learning_rate': 7.371692521157048e-06, 'epoch': 5.96}

 60%|█████▉    | 466/780 [30:13<19:23,  3.71s/it]
 60%|█████▉    | 467/780 [30:17<19:18,  3.70s/it]
                                                 
{'loss': 1.1612, 'grad_norm': 0.55066978931427, 'learning_rate': 7.331620880362571e-06, 'epoch': 5.98}

 60%|█████▉    | 467/780 [30:17<19:18,  3.70s/it]
 60%|██████    | 468/780 [30:21<19:32,  3.76s/it]
                                                 
{'loss': 1.2305, 'grad_norm': 0.5487978458404541, 'learning_rate': 7.291595318569951e-06, 'epoch': 5.99}

 60%|██████    | 468/780 [30:21<19:32,  3.76s/it]
 60%|██████    | 469/780 [30:25<20:30,  3.96s/it]
                                                 
{'loss': 1.1811, 'grad_norm': 0.4576186239719391, 'learning_rate': 7.2516165269620534e-06, 'epoch': 6.0}

 60%|██████    | 469/780 [30:25<20:30,  3.96s/it]
 60%|██████    | 470/780 [30:29<20:10,  3.91s/it]
                                                 
{'loss': 1.19, 'grad_norm': 0.4861593246459961, 'learning_rate': 7.2116851959140965e-06, 'epoch': 6.02}

 60%|██████    | 470/780 [30:29<20:10,  3.91s/it]
 60%|██████    | 471/780 [30:33<19:47,  3.84s/it]
                                                 
{'loss': 1.1143, 'grad_norm': 0.5805288553237915, 'learning_rate': 7.171802014981726e-06, 'epoch': 6.03}

 60%|██████    | 471/780 [30:33<19:47,  3.84s/it]
 61%|██████    | 472/780 [30:37<20:06,  3.92s/it]
                                                 
{'loss': 1.1989, 'grad_norm': 0.44235482811927795, 'learning_rate': 7.131967672889101e-06, 'epoch': 6.04}

 61%|██████    | 472/780 [30:37<20:06,  3.92s/it]
 61%|██████    | 473/780 [30:40<19:22,  3.79s/it]
                                                 
{'loss': 1.2808, 'grad_norm': 0.4690762162208557, 'learning_rate': 7.092182857516998e-06, 'epoch': 6.05}

 61%|██████    | 473/780 [30:40<19:22,  3.79s/it]
 61%|██████    | 474/780 [30:44<19:21,  3.79s/it]
                                                 
{'loss': 1.1723, 'grad_norm': 0.4488525390625, 'learning_rate': 7.052448255890958e-06, 'epoch': 6.07}

 61%|██████    | 474/780 [30:44<19:21,  3.79s/it]
 61%|██████    | 475/780 [30:48<19:21,  3.81s/it]
                                                 
{'loss': 1.134, 'grad_norm': 0.4518822133541107, 'learning_rate': 7.012764554169393e-06, 'epoch': 6.08}

 61%|██████    | 475/780 [30:48<19:21,  3.81s/it]
 61%|██████    | 476/780 [30:52<19:37,  3.87s/it]
                                                 
{'loss': 1.2077, 'grad_norm': 0.5217028856277466, 'learning_rate': 6.973132437631743e-06, 'epoch': 6.09}

 61%|██████    | 476/780 [30:52<19:37,  3.87s/it]
 61%|██████    | 477/780 [30:56<19:46,  3.92s/it]
                                                 
{'loss': 1.2231, 'grad_norm': 0.4669990539550781, 'learning_rate': 6.933552590666659e-06, 'epoch': 6.11}

 61%|██████    | 477/780 [30:56<19:46,  3.92s/it]
 61%|██████▏   | 478/780 [30:59<19:01,  3.78s/it]
                                                 
{'loss': 1.2318, 'grad_norm': 0.4668421745300293, 'learning_rate': 6.8940256967601625e-06, 'epoch': 6.12}

 61%|██████▏   | 478/780 [30:59<19:01,  3.78s/it]
 61%|██████▏   | 479/780 [31:03<19:28,  3.88s/it]
                                                 
{'loss': 1.1387, 'grad_norm': 0.5070293545722961, 'learning_rate': 6.854552438483866e-06, 'epoch': 6.13}

 61%|██████▏   | 479/780 [31:03<19:28,  3.88s/it]
 62%|██████▏   | 480/780 [31:07<19:29,  3.90s/it]
                                                 
{'loss': 1.2103, 'grad_norm': 0.5781958699226379, 'learning_rate': 6.815133497483157e-06, 'epoch': 6.14}

 62%|██████▏   | 480/780 [31:07<19:29,  3.90s/it]
 62%|██████▏   | 481/780 [31:11<19:28,  3.91s/it]
                                                 
{'loss': 1.1944, 'grad_norm': 0.5163084268569946, 'learning_rate': 6.775769554465455e-06, 'epoch': 6.16}

 62%|██████▏   | 481/780 [31:11<19:28,  3.91s/it]
 62%|██████▏   | 482/780 [31:16<20:32,  4.14s/it]
                                                 
{'loss': 1.227, 'grad_norm': 0.6745046377182007, 'learning_rate': 6.736461289188445e-06, 'epoch': 6.17}

 62%|██████▏   | 482/780 [31:16<20:32,  4.14s/it]
 62%|██████▏   | 483/780 [31:20<19:55,  4.03s/it]
                                                 
{'loss': 1.1425, 'grad_norm': 0.5714710354804993, 'learning_rate': 6.697209380448333e-06, 'epoch': 6.18}

 62%|██████▏   | 483/780 [31:20<19:55,  4.03s/it]
 62%|██████▏   | 484/780 [31:23<19:31,  3.96s/it]
                                                 
{'loss': 1.181, 'grad_norm': 0.4600895941257477, 'learning_rate': 6.6580145060681255e-06, 'epoch': 6.2}

 62%|██████▏   | 484/780 [31:24<19:31,  3.96s/it]
 62%|██████▏   | 485/780 [31:27<19:07,  3.89s/it]
                                                 
{'loss': 1.2902, 'grad_norm': 0.4159752428531647, 'learning_rate': 6.618877342885945e-06, 'epoch': 6.21}

 62%|██████▏   | 485/780 [31:27<19:07,  3.89s/it]
 62%|██████▏   | 486/780 [31:31<19:04,  3.89s/it]
                                                 
{'loss': 1.2141, 'grad_norm': 0.4188454747200012, 'learning_rate': 6.579798566743314e-06, 'epoch': 6.22}

 62%|██████▏   | 486/780 [31:31<19:04,  3.89s/it]
 62%|██████▏   | 487/780 [31:35<19:02,  3.90s/it]
                                                 
{'loss': 1.1437, 'grad_norm': 0.5349432826042175, 'learning_rate': 6.540778852473497e-06, 'epoch': 6.23}

 62%|██████▏   | 487/780 [31:35<19:02,  3.90s/it]
 63%|██████▎   | 488/780 [31:39<18:43,  3.85s/it]
                                                 
{'loss': 1.1923, 'grad_norm': 0.4384947717189789, 'learning_rate': 6.501818873889856e-06, 'epoch': 6.25}

 63%|██████▎   | 488/780 [31:39<18:43,  3.85s/it]
 63%|██████▎   | 489/780 [31:43<19:15,  3.97s/it]
                                                 
{'loss': 1.1766, 'grad_norm': 0.5279700756072998, 'learning_rate': 6.462919303774186e-06, 'epoch': 6.26}

 63%|██████▎   | 489/780 [31:43<19:15,  3.97s/it]
 63%|██████▎   | 490/780 [31:47<19:13,  3.98s/it]
                                                 
{'loss': 1.1928, 'grad_norm': 0.46277743577957153, 'learning_rate': 6.424080813865139e-06, 'epoch': 6.27}

 63%|██████▎   | 490/780 [31:47<19:13,  3.98s/it]
 63%|██████▎   | 491/780 [31:51<19:23,  4.03s/it]
                                                 
{'loss': 1.1912, 'grad_norm': 0.5910547375679016, 'learning_rate': 6.3853040748465855e-06, 'epoch': 6.28}

 63%|██████▎   | 491/780 [31:51<19:23,  4.03s/it]
 63%|██████▎   | 492/780 [31:55<18:40,  3.89s/it]
                                                 
{'loss': 1.1787, 'grad_norm': 0.5221357941627502, 'learning_rate': 6.34658975633605e-06, 'epoch': 6.3}

 63%|██████▎   | 492/780 [31:55<18:40,  3.89s/it]
 63%|██████▎   | 493/780 [31:59<18:47,  3.93s/it]
                                                 
{'loss': 1.2472, 'grad_norm': 0.44528084993362427, 'learning_rate': 6.3079385268731575e-06, 'epoch': 6.31}

 63%|██████▎   | 493/780 [31:59<18:47,  3.93s/it]
 63%|██████▎   | 494/780 [32:03<18:54,  3.97s/it]
                                                 
{'loss': 1.1759, 'grad_norm': 0.4497566223144531, 'learning_rate': 6.269351053908061e-06, 'epoch': 6.32}

 63%|██████▎   | 494/780 [32:03<18:54,  3.97s/it]
 63%|██████▎   | 495/780 [32:07<19:10,  4.04s/it]
                                                 
{'loss': 1.2084, 'grad_norm': 0.4273335933685303, 'learning_rate': 6.230828003789949e-06, 'epoch': 6.34}

 63%|██████▎   | 495/780 [32:07<19:10,  4.04s/it]
 64%|██████▎   | 496/780 [32:10<18:10,  3.84s/it]
                                                 
{'loss': 1.156, 'grad_norm': 0.47390419244766235, 'learning_rate': 6.192370041755505e-06, 'epoch': 6.35}

 64%|██████▎   | 496/780 [32:10<18:10,  3.84s/it]
 64%|██████▎   | 497/780 [32:14<18:01,  3.82s/it]
                                                 
{'loss': 1.2247, 'grad_norm': 0.5237614512443542, 'learning_rate': 6.153977831917451e-06, 'epoch': 6.36}

 64%|██████▎   | 497/780 [32:14<18:01,  3.82s/it]
 64%|██████▍   | 498/780 [32:18<18:12,  3.88s/it]
                                                 
{'loss': 1.1808, 'grad_norm': 0.48222672939300537, 'learning_rate': 6.115652037253054e-06, 'epoch': 6.37}

 64%|██████▍   | 498/780 [32:18<18:12,  3.88s/it]
 64%|██████▍   | 499/780 [32:22<18:44,  4.00s/it]
                                                 
{'loss': 1.1747, 'grad_norm': 0.654991090297699, 'learning_rate': 6.077393319592697e-06, 'epoch': 6.39}

 64%|██████▍   | 499/780 [32:22<18:44,  4.00s/it]
 64%|██████▍   | 500/780 [32:26<18:00,  3.86s/it]
                                                 
{'loss': 1.2064, 'grad_norm': 0.45382189750671387, 'learning_rate': 6.039202339608432e-06, 'epoch': 6.4}

 64%|██████▍   | 500/780 [32:26<18:00,  3.86s/it]
 64%|██████▍   | 501/780 [32:30<17:58,  3.86s/it]
                                                 
{'loss': 1.1254, 'grad_norm': 0.47792503237724304, 'learning_rate': 6.001079756802592e-06, 'epoch': 6.41}

 64%|██████▍   | 501/780 [32:30<17:58,  3.86s/it]
 64%|██████▍   | 502/780 [32:34<18:28,  3.99s/it]
                                                 
{'loss': 1.2073, 'grad_norm': 0.4971482455730438, 'learning_rate': 5.963026229496378e-06, 'epoch': 6.43}

 64%|██████▍   | 502/780 [32:34<18:28,  3.99s/it]
 64%|██████▍   | 503/780 [32:38<18:00,  3.90s/it]
                                                 
{'loss': 1.2308, 'grad_norm': 0.5351437330245972, 'learning_rate': 5.925042414818514e-06, 'epoch': 6.44}

 64%|██████▍   | 503/780 [32:38<18:00,  3.90s/it]
 65%|██████▍   | 504/780 [32:42<17:48,  3.87s/it]
                                                 
{'loss': 1.2175, 'grad_norm': 0.44985437393188477, 'learning_rate': 5.887128968693887e-06, 'epoch': 6.45}

 65%|██████▍   | 504/780 [32:42<17:48,  3.87s/it]
 65%|██████▍   | 505/780 [32:46<18:28,  4.03s/it]
                                                 
{'loss': 1.1488, 'grad_norm': 0.423246830701828, 'learning_rate': 5.849286545832211e-06, 'epoch': 6.46}

 65%|██████▍   | 505/780 [32:46<18:28,  4.03s/it]
 65%|██████▍   | 506/780 [32:50<17:44,  3.89s/it]
                                                 
{'loss': 1.2488, 'grad_norm': 0.5278103351593018, 'learning_rate': 5.811515799716754e-06, 'epoch': 6.48}

 65%|██████▍   | 506/780 [32:50<17:44,  3.89s/it]
 65%|██████▌   | 507/780 [32:53<17:23,  3.82s/it]
                                                 
{'loss': 1.209, 'grad_norm': 0.4364224970340729, 'learning_rate': 5.773817382593008e-06, 'epoch': 6.49}

 65%|██████▌   | 507/780 [32:53<17:23,  3.82s/it]
 65%|██████▌   | 508/780 [32:57<17:34,  3.88s/it]
                                                 
{'loss': 1.2252, 'grad_norm': 0.5161915421485901, 'learning_rate': 5.736191945457463e-06, 'epoch': 6.5}

 65%|██████▌   | 508/780 [32:57<17:34,  3.88s/it]
 65%|██████▌   | 509/780 [33:01<17:15,  3.82s/it]
                                                 
{'loss': 1.1314, 'grad_norm': 0.6091681718826294, 'learning_rate': 5.698640138046349e-06, 'epoch': 6.52}

 65%|██████▌   | 509/780 [33:01<17:15,  3.82s/it]
 65%|██████▌   | 510/780 [33:05<17:28,  3.88s/it]
                                                 
{'loss': 1.2106, 'grad_norm': 0.45741331577301025, 'learning_rate': 5.66116260882442e-06, 'epoch': 6.53}

 65%|██████▌   | 510/780 [33:05<17:28,  3.88s/it]
 66%|██████▌   | 511/780 [33:09<17:54,  4.00s/it]
                                                 
{'loss': 1.2064, 'grad_norm': 0.47265782952308655, 'learning_rate': 5.623760004973749e-06, 'epoch': 6.54}

 66%|██████▌   | 511/780 [33:09<17:54,  4.00s/it]
 66%|██████▌   | 512/780 [33:13<17:10,  3.85s/it]
                                                 
{'loss': 1.1875, 'grad_norm': 0.5088405013084412, 'learning_rate': 5.586432972382561e-06, 'epoch': 6.55}

 66%|██████▌   | 512/780 [33:13<17:10,  3.85s/it]
 66%|██████▌   | 513/780 [33:16<17:00,  3.82s/it]
                                                 
{'loss': 1.2108, 'grad_norm': 0.48855769634246826, 'learning_rate': 5.549182155634076e-06, 'epoch': 6.57}

 66%|██████▌   | 513/780 [33:17<17:00,  3.82s/it]
 66%|██████▌   | 514/780 [33:20<16:59,  3.83s/it]
                                                 
{'loss': 1.2073, 'grad_norm': 0.4795098602771759, 'learning_rate': 5.512008197995379e-06, 'epoch': 6.58}

 66%|██████▌   | 514/780 [33:20<16:59,  3.83s/it]
 66%|██████▌   | 515/780 [33:25<17:27,  3.95s/it]
                                                 
{'loss': 1.1734, 'grad_norm': 0.4676732122898102, 'learning_rate': 5.47491174140631e-06, 'epoch': 6.59}

 66%|██████▌   | 515/780 [33:25<17:27,  3.95s/it]
 66%|██████▌   | 516/780 [33:29<17:45,  4.04s/it]
                                                 
{'loss': 1.1537, 'grad_norm': 0.4802436828613281, 'learning_rate': 5.43789342646837e-06, 'epoch': 6.6}

 66%|██████▌   | 516/780 [33:29<17:45,  4.04s/it]
 66%|██████▋   | 517/780 [33:33<17:31,  4.00s/it]
                                                 
{'loss': 1.1783, 'grad_norm': 0.46917906403541565, 'learning_rate': 5.4009538924336864e-06, 'epoch': 6.62}

 66%|██████▋   | 517/780 [33:33<17:31,  4.00s/it]
 66%|██████▋   | 518/780 [33:37<17:22,  3.98s/it]
                                                 
{'loss': 1.186, 'grad_norm': 0.5384033918380737, 'learning_rate': 5.364093777193944e-06, 'epoch': 6.63}

 66%|██████▋   | 518/780 [33:37<17:22,  3.98s/it]
 67%|██████▋   | 519/780 [33:41<17:16,  3.97s/it]
                                                 
{'loss': 1.1742, 'grad_norm': 0.5948904752731323, 'learning_rate': 5.32731371726938e-06, 'epoch': 6.64}

 67%|██████▋   | 519/780 [33:41<17:16,  3.97s/it]
 67%|██████▋   | 520/780 [33:44<16:52,  3.89s/it]
                                                 
{'loss': 1.1695, 'grad_norm': 0.4323138892650604, 'learning_rate': 5.290614347797802e-06, 'epoch': 6.66}

 67%|██████▋   | 520/780 [33:44<16:52,  3.89s/it]
 67%|██████▋   | 521/780 [33:48<15:54,  3.69s/it]
                                                 
{'loss': 1.2101, 'grad_norm': 0.5054697394371033, 'learning_rate': 5.253996302523596e-06, 'epoch': 6.67}

 67%|██████▋   | 521/780 [33:48<15:54,  3.69s/it]
 67%|██████▋   | 522/780 [33:51<15:58,  3.72s/it]
                                                 
{'loss': 1.1264, 'grad_norm': 0.5062647461891174, 'learning_rate': 5.217460213786822e-06, 'epoch': 6.68}

 67%|██████▋   | 522/780 [33:51<15:58,  3.72s/it]
 67%|██████▋   | 523/780 [33:55<15:54,  3.71s/it]
                                                 
{'loss': 1.2085, 'grad_norm': 0.4773625135421753, 'learning_rate': 5.181006712512245e-06, 'epoch': 6.69}

 67%|██████▋   | 523/780 [33:55<15:54,  3.71s/it]
 67%|██████▋   | 524/780 [33:59<16:11,  3.79s/it]
                                                 
{'loss': 1.1192, 'grad_norm': 0.5340938568115234, 'learning_rate': 5.144636428198477e-06, 'epoch': 6.71}

 67%|██████▋   | 524/780 [33:59<16:11,  3.79s/it]
 67%|██████▋   | 525/780 [34:03<16:29,  3.88s/it]
                                                 
{'loss': 1.1806, 'grad_norm': 0.47774752974510193, 'learning_rate': 5.108349988907111e-06, 'epoch': 6.72}

 67%|██████▋   | 525/780 [34:03<16:29,  3.88s/it]
 67%|██████▋   | 526/780 [34:07<16:40,  3.94s/it]
                                                 
{'loss': 1.1961, 'grad_norm': 0.49115657806396484, 'learning_rate': 5.072148021251822e-06, 'epoch': 6.73}

 67%|██████▋   | 526/780 [34:07<16:40,  3.94s/it]
 68%|██████▊   | 527/780 [34:11<16:10,  3.84s/it]
                                                 
{'loss': 1.1702, 'grad_norm': 0.5179590582847595, 'learning_rate': 5.036031150387624e-06, 'epoch': 6.75}

 68%|██████▊   | 527/780 [34:11<16:10,  3.84s/it]
 68%|██████▊   | 528/780 [34:15<16:18,  3.88s/it]
                                                 
{'loss': 1.2506, 'grad_norm': 0.42752212285995483, 'learning_rate': 5.000000000000003e-06, 'epoch': 6.76}

 68%|██████▊   | 528/780 [34:15<16:18,  3.88s/it]
 68%|██████▊   | 529/780 [34:19<16:19,  3.90s/it]
                                                 
{'loss': 1.2341, 'grad_norm': 0.6524770855903625, 'learning_rate': 4.964055192294187e-06, 'epoch': 6.77}

 68%|██████▊   | 529/780 [34:19<16:19,  3.90s/it]
 68%|██████▊   | 530/780 [34:23<16:14,  3.90s/it]
                                                 
{'loss': 1.1811, 'grad_norm': 0.5030565857887268, 'learning_rate': 4.92819734798441e-06, 'epoch': 6.78}

 68%|██████▊   | 530/780 [34:23<16:14,  3.90s/it]
 68%|██████▊   | 531/780 [34:26<15:51,  3.82s/it]
                                                 
{'loss': 1.1509, 'grad_norm': 0.5254966020584106, 'learning_rate': 4.892427086283147e-06, 'epoch': 6.8}

 68%|██████▊   | 531/780 [34:26<15:51,  3.82s/it]
 68%|██████▊   | 532/780 [34:30<16:03,  3.89s/it]
                                                 
{'loss': 1.1987, 'grad_norm': 0.4336331784725189, 'learning_rate': 4.856745024890466e-06, 'epoch': 6.81}

 68%|██████▊   | 532/780 [34:30<16:03,  3.89s/it]
 68%|██████▊   | 533/780 [34:34<16:07,  3.92s/it]
                                                 
{'loss': 1.2126, 'grad_norm': 0.38816720247268677, 'learning_rate': 4.821151779983343e-06, 'epoch': 6.82}

 68%|██████▊   | 533/780 [34:34<16:07,  3.92s/it]
 68%|██████▊   | 534/780 [34:38<15:53,  3.88s/it]
                                                 
{'loss': 1.182, 'grad_norm': 0.47066813707351685, 'learning_rate': 4.78564796620502e-06, 'epoch': 6.84}

 68%|██████▊   | 534/780 [34:38<15:53,  3.88s/it]
 69%|██████▊   | 535/780 [34:42<15:37,  3.83s/it]
                                                 
{'loss': 1.1789, 'grad_norm': 0.48328396677970886, 'learning_rate': 4.7502341966544e-06, 'epoch': 6.85}

 69%|██████▊   | 535/780 [34:42<15:37,  3.83s/it]
 69%|██████▊   | 536/780 [34:46<16:19,  4.02s/it]
                                                 
{'loss': 1.1544, 'grad_norm': 0.4058506488800049, 'learning_rate': 4.714911082875446e-06, 'epoch': 6.86}

 69%|██████▊   | 536/780 [34:46<16:19,  4.02s/it]
 69%|██████▉   | 537/780 [34:50<16:03,  3.96s/it]
                                                 
{'loss': 1.1416, 'grad_norm': 0.4453909695148468, 'learning_rate': 4.679679234846636e-06, 'epoch': 6.87}

 69%|██████▉   | 537/780 [34:50<16:03,  3.96s/it]
 69%|██████▉   | 538/780 [34:54<16:33,  4.10s/it]
                                                 
{'loss': 1.2266, 'grad_norm': 0.4395992159843445, 'learning_rate': 4.644539260970417e-06, 'epoch': 6.89}

 69%|██████▉   | 538/780 [34:55<16:33,  4.10s/it]
 69%|██████▉   | 539/780 [34:58<15:55,  3.96s/it]
                                                 
{'loss': 1.1189, 'grad_norm': 0.5102989673614502, 'learning_rate': 4.609491768062705e-06, 'epoch': 6.9}

 69%|██████▉   | 539/780 [34:58<15:55,  3.96s/it]
 69%|██████▉   | 540/780 [35:02<16:15,  4.07s/it]
                                                 
{'loss': 1.2023, 'grad_norm': 0.47348839044570923, 'learning_rate': 4.5745373613424075e-06, 'epoch': 6.91}

 69%|██████▉   | 540/780 [35:02<16:15,  4.07s/it]
 69%|██████▉   | 541/780 [35:06<15:40,  3.94s/it]
                                                 
{'loss': 1.1611, 'grad_norm': 0.5286843180656433, 'learning_rate': 4.539676644420966e-06, 'epoch': 6.92}

 69%|██████▉   | 541/780 [35:06<15:40,  3.94s/it]
 69%|██████▉   | 542/780 [35:10<15:32,  3.92s/it]
                                                 
{'loss': 1.1881, 'grad_norm': 0.4742004871368408, 'learning_rate': 4.504910219291941e-06, 'epoch': 6.94}

 69%|██████▉   | 542/780 [35:10<15:32,  3.92s/it]
 70%|██████▉   | 543/780 [35:13<14:50,  3.76s/it]
                                                 
{'loss': 1.0958, 'grad_norm': 0.5528199672698975, 'learning_rate': 4.470238686320606e-06, 'epoch': 6.95}

 70%|██████▉   | 543/780 [35:13<14:50,  3.76s/it]
 70%|██████▉   | 544/780 [35:17<14:32,  3.70s/it]
                                                 
{'loss': 1.1329, 'grad_norm': 0.5212376117706299, 'learning_rate': 4.435662644233594e-06, 'epoch': 6.96}

 70%|██████▉   | 544/780 [35:17<14:32,  3.70s/it]
 70%|██████▉   | 545/780 [35:21<14:36,  3.73s/it]
                                                 
{'loss': 1.1618, 'grad_norm': 0.5670945644378662, 'learning_rate': 4.4011826901085346e-06, 'epoch': 6.98}

 70%|██████▉   | 545/780 [35:21<14:36,  3.73s/it]
 70%|███████   | 546/780 [35:25<14:46,  3.79s/it]
                                                 
{'loss': 1.1816, 'grad_norm': 0.5274068713188171, 'learning_rate': 4.3667994193637794e-06, 'epoch': 6.99}

 70%|███████   | 546/780 [35:25<14:46,  3.79s/it]
 70%|███████   | 547/780 [35:29<15:40,  4.04s/it]
                                                 
{'loss': 1.2092, 'grad_norm': 0.488931268453598, 'learning_rate': 4.3325134257480905e-06, 'epoch': 7.0}

 70%|███████   | 547/780 [35:29<15:40,  4.04s/it]
 70%|███████   | 548/780 [35:33<15:19,  3.96s/it]
                                                 
{'loss': 1.198, 'grad_norm': 0.44840243458747864, 'learning_rate': 4.298325301330383e-06, 'epoch': 7.01}

 70%|███████   | 548/780 [35:33<15:19,  3.96s/it]
 70%|███████   | 549/780 [35:37<14:49,  3.85s/it]
                                                 
{'loss': 1.0976, 'grad_norm': 0.5744760632514954, 'learning_rate': 4.264235636489542e-06, 'epoch': 7.03}

 70%|███████   | 549/780 [35:37<14:49,  3.85s/it]
 71%|███████   | 550/780 [35:41<15:03,  3.93s/it]
                                                 
{'loss': 1.1752, 'grad_norm': 0.45235514640808105, 'learning_rate': 4.23024501990417e-06, 'epoch': 7.04}

 71%|███████   | 550/780 [35:41<15:03,  3.93s/it]
 71%|███████   | 551/780 [35:44<14:39,  3.84s/it]
                                                 
{'loss': 1.277, 'grad_norm': 0.4738121032714844, 'learning_rate': 4.196354038542476e-06, 'epoch': 7.05}

 71%|███████   | 551/780 [35:44<14:39,  3.84s/it]
 71%|███████   | 552/780 [35:48<14:12,  3.74s/it]
                                                 
{'loss': 1.2043, 'grad_norm': 0.4675080180168152, 'learning_rate': 4.162563277652104e-06, 'epoch': 7.07}

 71%|███████   | 552/780 [35:48<14:12,  3.74s/it]
 71%|███████   | 553/780 [35:52<14:33,  3.85s/it]
                                                 
{'loss': 1.1153, 'grad_norm': 0.4362262487411499, 'learning_rate': 4.128873320750027e-06, 'epoch': 7.08}

 71%|███████   | 553/780 [35:52<14:33,  3.85s/it]
 71%|███████   | 554/780 [35:56<14:40,  3.90s/it]
                                                 
{'loss': 1.205, 'grad_norm': 0.5259241461753845, 'learning_rate': 4.095284749612504e-06, 'epoch': 7.09}

 71%|███████   | 554/780 [35:56<14:40,  3.90s/it]
 71%|███████   | 555/780 [36:00<14:36,  3.90s/it]
                                                 
{'loss': 1.2194, 'grad_norm': 0.48038923740386963, 'learning_rate': 4.061798144264986e-06, 'epoch': 7.1}

 71%|███████   | 555/780 [36:00<14:36,  3.90s/it]
 71%|███████▏  | 556/780 [36:03<14:14,  3.82s/it]
                                                 
{'loss': 1.2097, 'grad_norm': 0.47457194328308105, 'learning_rate': 4.028414082972141e-06, 'epoch': 7.12}

 71%|███████▏  | 556/780 [36:03<14:14,  3.82s/it]
 71%|███████▏  | 557/780 [36:08<14:44,  3.97s/it]
                                                 
{'loss': 1.1558, 'grad_norm': 0.4756346642971039, 'learning_rate': 3.995133142227843e-06, 'epoch': 7.13}

 71%|███████▏  | 557/780 [36:08<14:44,  3.97s/it]
 72%|███████▏  | 558/780 [36:12<14:30,  3.92s/it]
                                                 
{'loss': 1.2046, 'grad_norm': 0.5748385190963745, 'learning_rate': 3.961955896745224e-06, 'epoch': 7.14}

 72%|███████▏  | 558/780 [36:12<14:30,  3.92s/it]
 72%|███████▏  | 559/780 [36:16<14:32,  3.95s/it]
                                                 
{'loss': 1.1951, 'grad_norm': 0.5413762927055359, 'learning_rate': 3.928882919446767e-06, 'epoch': 7.16}

 72%|███████▏  | 559/780 [36:16<14:32,  3.95s/it]
 72%|███████▏  | 560/780 [36:20<15:12,  4.15s/it]
                                                 
{'loss': 1.2398, 'grad_norm': 0.7138112783432007, 'learning_rate': 3.89591478145437e-06, 'epoch': 7.17}

 72%|███████▏  | 560/780 [36:20<15:12,  4.15s/it]
 72%|███████▏  | 561/780 [36:24<14:46,  4.05s/it]
                                                 
{'loss': 1.1243, 'grad_norm': 0.5733643770217896, 'learning_rate': 3.8630520520795275e-06, 'epoch': 7.18}

 72%|███████▏  | 561/780 [36:24<14:46,  4.05s/it]
 72%|███████▏  | 562/780 [36:28<14:24,  3.97s/it]
                                                 
{'loss': 1.1674, 'grad_norm': 0.46850594878196716, 'learning_rate': 3.830295298813475e-06, 'epoch': 7.19}

 72%|███████▏  | 562/780 [36:28<14:24,  3.97s/it]
 72%|███████▏  | 563/780 [36:32<14:12,  3.93s/it]
                                                 
{'loss': 1.2818, 'grad_norm': 0.4138328433036804, 'learning_rate': 3.797645087317401e-06, 'epoch': 7.21}

 72%|███████▏  | 563/780 [36:32<14:12,  3.93s/it]
 72%|███████▏  | 564/780 [36:35<13:59,  3.89s/it]
                                                 
{'loss': 1.2409, 'grad_norm': 0.4174717962741852, 'learning_rate': 3.7651019814126656e-06, 'epoch': 7.22}

 72%|███████▏  | 564/780 [36:35<13:59,  3.89s/it]
 72%|███████▏  | 565/780 [36:40<14:07,  3.94s/it]
                                                 
{'loss': 1.1532, 'grad_norm': 0.5173200964927673, 'learning_rate': 3.7326665430710798e-06, 'epoch': 7.23}

 72%|███████▏  | 565/780 [36:40<14:07,  3.94s/it]
 73%|███████▎  | 566/780 [36:43<13:56,  3.91s/it]
                                                 
{'loss': 1.1563, 'grad_norm': 0.4514678120613098, 'learning_rate': 3.7003393324051874e-06, 'epoch': 7.24}

 73%|███████▎  | 566/780 [36:43<13:56,  3.91s/it]
 73%|███████▎  | 567/780 [36:47<14:07,  3.98s/it]
                                                 
{'loss': 1.2075, 'grad_norm': 0.47621259093284607, 'learning_rate': 3.6681209076586035e-06, 'epoch': 7.26}

 73%|███████▎  | 567/780 [36:48<14:07,  3.98s/it]
 73%|███████▎  | 568/780 [36:51<13:57,  3.95s/it]
                                                 
{'loss': 1.1749, 'grad_norm': 0.49070459604263306, 'learning_rate': 3.636011825196365e-06, 'epoch': 7.27}

 73%|███████▎  | 568/780 [36:51<13:57,  3.95s/it]
 73%|███████▎  | 569/780 [36:55<14:03,  4.00s/it]
                                                 
{'loss': 1.214, 'grad_norm': 0.6167967915534973, 'learning_rate': 3.6040126394953334e-06, 'epoch': 7.28}

 73%|███████▎  | 569/780 [36:56<14:03,  4.00s/it]
 73%|███████▎  | 570/780 [36:59<13:30,  3.86s/it]
                                                 
{'loss': 1.1344, 'grad_norm': 0.5266568064689636, 'learning_rate': 3.5721239031346067e-06, 'epoch': 7.3}

 73%|███████▎  | 570/780 [36:59<13:30,  3.86s/it]
 73%|███████▎  | 571/780 [37:03<13:40,  3.93s/it]
                                                 
{'loss': 1.2464, 'grad_norm': 0.44654539227485657, 'learning_rate': 3.540346166785994e-06, 'epoch': 7.31}

 73%|███████▎  | 571/780 [37:03<13:40,  3.93s/it]
 73%|███████▎  | 572/780 [37:07<13:33,  3.91s/it]
                                                 
{'loss': 1.1664, 'grad_norm': 0.4920724332332611, 'learning_rate': 3.5086799792044812e-06, 'epoch': 7.32}

 73%|███████▎  | 572/780 [37:07<13:33,  3.91s/it]
 73%|███████▎  | 573/780 [37:11<13:38,  3.95s/it]
                                                 
{'loss': 1.1972, 'grad_norm': 0.43161487579345703, 'learning_rate': 3.4771258872187917e-06, 'epoch': 7.33}

 73%|███████▎  | 573/780 [37:11<13:38,  3.95s/it]
 74%|███████▎  | 574/780 [37:15<13:22,  3.89s/it]
                                                 
{'loss': 1.1624, 'grad_norm': 0.47389933466911316, 'learning_rate': 3.4456844357218977e-06, 'epoch': 7.35}

 74%|███████▎  | 574/780 [37:15<13:22,  3.89s/it]
 74%|███████▎  | 575/780 [37:19<13:08,  3.84s/it]
                                                 
{'loss': 1.2191, 'grad_norm': 0.4857427179813385, 'learning_rate': 3.414356167661658e-06, 'epoch': 7.36}

 74%|███████▎  | 575/780 [37:19<13:08,  3.84s/it]
 74%|███████▍  | 576/780 [37:22<13:12,  3.88s/it]
                                                 
{'loss': 1.203, 'grad_norm': 0.4832095801830292, 'learning_rate': 3.3831416240314085e-06, 'epoch': 7.37}

 74%|███████▍  | 576/780 [37:23<13:12,  3.88s/it]
 74%|███████▍  | 577/780 [37:27<13:36,  4.02s/it]
                                                 
{'loss': 1.1714, 'grad_norm': 0.6456241607666016, 'learning_rate': 3.3520413438606215e-06, 'epoch': 7.39}

 74%|███████▍  | 577/780 [37:27<13:36,  4.02s/it]
 74%|███████▍  | 578/780 [37:30<13:05,  3.89s/it]
                                                 
{'loss': 1.1912, 'grad_norm': 0.4632456600666046, 'learning_rate': 3.3210558642056277e-06, 'epoch': 7.4}

 74%|███████▍  | 578/780 [37:30<13:05,  3.89s/it]
 74%|███████▍  | 579/780 [37:34<13:01,  3.89s/it]
                                                 
{'loss': 1.1449, 'grad_norm': 0.47231268882751465, 'learning_rate': 3.290185720140301e-06, 'epoch': 7.41}

 74%|███████▍  | 579/780 [37:34<13:01,  3.89s/it]
 74%|███████▍  | 580/780 [37:38<13:05,  3.93s/it]
                                                 
{'loss': 1.1615, 'grad_norm': 0.5098841786384583, 'learning_rate': 3.2594314447468457e-06, 'epoch': 7.42}

 74%|███████▍  | 580/780 [37:38<13:05,  3.93s/it]
 74%|███████▍  | 581/780 [37:42<12:55,  3.90s/it]
                                                 
{'loss': 1.2515, 'grad_norm': 0.5162403583526611, 'learning_rate': 3.228793569106594e-06, 'epoch': 7.44}

 74%|███████▍  | 581/780 [37:42<12:55,  3.90s/it]
 75%|███████▍  | 582/780 [37:46<12:46,  3.87s/it]
                                                 
{'loss': 1.2214, 'grad_norm': 0.44352155923843384, 'learning_rate': 3.1982726222908046e-06, 'epoch': 7.45}

 75%|███████▍  | 582/780 [37:46<12:46,  3.87s/it]
 75%|███████▍  | 583/780 [37:50<12:53,  3.93s/it]
                                                 
{'loss': 1.1413, 'grad_norm': 0.43292558193206787, 'learning_rate': 3.1678691313515688e-06, 'epoch': 7.46}

 75%|███████▍  | 583/780 [37:50<12:53,  3.93s/it]
 75%|███████▍  | 584/780 [37:54<12:40,  3.88s/it]
                                                 
{'loss': 1.2447, 'grad_norm': 0.5089225769042969, 'learning_rate': 3.1375836213126653e-06, 'epoch': 7.48}

 75%|███████▍  | 584/780 [37:54<12:40,  3.88s/it]
 75%|███████▌  | 585/780 [37:57<12:23,  3.82s/it]
                                                 
{'loss': 1.213, 'grad_norm': 0.4383423626422882, 'learning_rate': 3.10741661516053e-06, 'epoch': 7.49}

 75%|███████▌  | 585/780 [37:58<12:23,  3.82s/it]
 75%|███████▌  | 586/780 [38:01<12:12,  3.77s/it]
                                                 
{'loss': 1.2288, 'grad_norm': 0.47769805788993835, 'learning_rate': 3.077368633835205e-06, 'epoch': 7.5}

 75%|███████▌  | 586/780 [38:01<12:12,  3.77s/it]
 75%|███████▌  | 587/780 [38:05<12:02,  3.74s/it]
                                                 
{'loss': 1.1259, 'grad_norm': 0.6114176511764526, 'learning_rate': 3.0474401962213483e-06, 'epoch': 7.51}

 75%|███████▌  | 587/780 [38:05<12:02,  3.74s/it]
 75%|███████▌  | 588/780 [38:09<12:15,  3.83s/it]
                                                 
{'loss': 1.2044, 'grad_norm': 0.4163506031036377, 'learning_rate': 3.017631819139273e-06, 'epoch': 7.53}

 75%|███████▌  | 588/780 [38:09<12:15,  3.83s/it]
 76%|███████▌  | 589/780 [38:13<12:21,  3.88s/it]
                                                 
{'loss': 1.1885, 'grad_norm': 0.5157233476638794, 'learning_rate': 2.987944017336023e-06, 'epoch': 7.54}

 76%|███████▌  | 589/780 [38:13<12:21,  3.88s/it]
 76%|███████▌  | 590/780 [38:16<12:00,  3.79s/it]
                                                 
{'loss': 1.1912, 'grad_norm': 0.5110534429550171, 'learning_rate': 2.958377303476483e-06, 'epoch': 7.55}

 76%|███████▌  | 590/780 [38:16<12:00,  3.79s/it]
 76%|███████▌  | 591/780 [38:20<11:42,  3.72s/it]
                                                 
{'loss': 1.2133, 'grad_norm': 0.48937496542930603, 'learning_rate': 2.9289321881345257e-06, 'epoch': 7.56}

 76%|███████▌  | 591/780 [38:20<11:42,  3.72s/it]
 76%|███████▌  | 592/780 [38:24<11:37,  3.71s/it]
                                                 
{'loss': 1.197, 'grad_norm': 0.46697625517845154, 'learning_rate': 2.8996091797841976e-06, 'epoch': 7.58}

 76%|███████▌  | 592/780 [38:24<11:37,  3.71s/it]
 76%|███████▌  | 593/780 [38:28<12:05,  3.88s/it]
                                                 
{'loss': 1.1943, 'grad_norm': 0.461776465177536, 'learning_rate': 2.8704087847909333e-06, 'epoch': 7.59}

 76%|███████▌  | 593/780 [38:28<12:05,  3.88s/it]
 76%|███████▌  | 594/780 [38:32<12:22,  3.99s/it]
                                                 
{'loss': 1.1379, 'grad_norm': 0.48028457164764404, 'learning_rate': 2.8413315074028157e-06, 'epoch': 7.6}

 76%|███████▌  | 594/780 [38:32<12:22,  3.99s/it]
 76%|███████▋  | 595/780 [38:36<12:08,  3.94s/it]
                                                 
{'loss': 1.1634, 'grad_norm': 0.49441471695899963, 'learning_rate': 2.8123778497418687e-06, 'epoch': 7.62}

 76%|███████▋  | 595/780 [38:36<12:08,  3.94s/it]
 76%|███████▋  | 596/780 [38:41<12:38,  4.12s/it]
                                                 
{'loss': 1.1993, 'grad_norm': 0.5405185222625732, 'learning_rate': 2.783548311795379e-06, 'epoch': 7.63}

 76%|███████▋  | 596/780 [38:41<12:38,  4.12s/it]
 77%|███████▋  | 597/780 [38:44<12:14,  4.01s/it]
                                                 
{'loss': 1.1605, 'grad_norm': 0.5841661095619202, 'learning_rate': 2.7548433914072736e-06, 'epoch': 7.64}

 77%|███████▋  | 597/780 [38:44<12:14,  4.01s/it]
 77%|███████▋  | 598/780 [38:48<11:56,  3.93s/it]
                                                 
{'loss': 1.1733, 'grad_norm': 0.4356136918067932, 'learning_rate': 2.726263584269513e-06, 'epoch': 7.65}

 77%|███████▋  | 598/780 [38:48<11:56,  3.93s/it]
 77%|███████▋  | 599/780 [38:52<11:25,  3.79s/it]
                                                 
{'loss': 1.2014, 'grad_norm': 0.4847688674926758, 'learning_rate': 2.6978093839135365e-06, 'epoch': 7.67}

 77%|███████▋  | 599/780 [38:52<11:25,  3.79s/it]
 77%|███████▋  | 600/780 [38:55<11:23,  3.80s/it]
                                                 
{'loss': 1.1329, 'grad_norm': 0.5165532827377319, 'learning_rate': 2.669481281701739e-06, 'epoch': 7.68}

 77%|███████▋  | 600/780 [38:55<11:23,  3.80s/it]
 77%|███████▋  | 601/780 [38:59<11:14,  3.77s/it]
                                                 
{'loss': 1.1862, 'grad_norm': 0.4682694971561432, 'learning_rate': 2.641279766818977e-06, 'epoch': 7.69}

 77%|███████▋  | 601/780 [38:59<11:14,  3.77s/it]
 77%|███████▋  | 602/780 [39:03<11:11,  3.77s/it]
                                                 
{'loss': 1.133, 'grad_norm': 0.524675726890564, 'learning_rate': 2.6132053262641467e-06, 'epoch': 7.71}

 77%|███████▋  | 602/780 [39:03<11:11,  3.77s/it]
 77%|███████▋  | 603/780 [39:07<11:25,  3.88s/it]
                                                 
{'loss': 1.1765, 'grad_norm': 0.4979141652584076, 'learning_rate': 2.5852584448417327e-06, 'epoch': 7.72}

 77%|███████▋  | 603/780 [39:07<11:25,  3.88s/it]
 77%|███████▋  | 604/780 [39:11<11:27,  3.91s/it]
                                                 
{'loss': 1.1919, 'grad_norm': 0.5004337430000305, 'learning_rate': 2.5574396051534835e-06, 'epoch': 7.73}

 77%|███████▋  | 604/780 [39:11<11:27,  3.91s/it]
 78%|███████▊  | 605/780 [39:15<11:10,  3.83s/it]
                                                 
{'loss': 1.1702, 'grad_norm': 0.4721788167953491, 'learning_rate': 2.529749287590042e-06, 'epoch': 7.74}

 78%|███████▊  | 605/780 [39:15<11:10,  3.83s/it]
 78%|███████▊  | 606/780 [39:19<11:13,  3.87s/it]
                                                 
{'loss': 1.2544, 'grad_norm': 0.43982529640197754, 'learning_rate': 2.502187970322657e-06, 'epoch': 7.76}

 78%|███████▊  | 606/780 [39:19<11:13,  3.87s/it]
 78%|███████▊  | 607/780 [39:22<11:14,  3.90s/it]
                                                 
{'loss': 1.1985, 'grad_norm': 0.485573947429657, 'learning_rate': 2.4747561292949496e-06, 'epoch': 7.77}

 78%|███████▊  | 607/780 [39:23<11:14,  3.90s/it]
 78%|███████▊  | 608/780 [39:26<11:05,  3.87s/it]
                                                 
{'loss': 1.2103, 'grad_norm': 0.6631733179092407, 'learning_rate': 2.447454238214654e-06, 'epoch': 7.78}

 78%|███████▊  | 608/780 [39:26<11:05,  3.87s/it]
 78%|███████▊  | 609/780 [39:30<10:57,  3.84s/it]
                                                 
{'loss': 1.1289, 'grad_norm': 0.5036177635192871, 'learning_rate': 2.420282768545469e-06, 'epoch': 7.8}

 78%|███████▊  | 609/780 [39:30<10:57,  3.84s/it]
 78%|███████▊  | 610/780 [39:34<10:59,  3.88s/it]
                                                 
{'loss': 1.2023, 'grad_norm': 0.425885409116745, 'learning_rate': 2.3932421894989167e-06, 'epoch': 7.81}

 78%|███████▊  | 610/780 [39:34<10:59,  3.88s/it]
 78%|███████▊  | 611/780 [39:38<10:45,  3.82s/it]
                                                 
{'loss': 1.2151, 'grad_norm': 0.4089743494987488, 'learning_rate': 2.366332968026207e-06, 'epoch': 7.82}

 78%|███████▊  | 611/780 [39:38<10:45,  3.82s/it]
 78%|███████▊  | 612/780 [39:42<10:52,  3.88s/it]
                                                 
{'loss': 1.1857, 'grad_norm': 0.4548426568508148, 'learning_rate': 2.339555568810221e-06, 'epoch': 7.83}

 78%|███████▊  | 612/780 [39:42<10:52,  3.88s/it]
 79%|███████▊  | 613/780 [39:46<10:41,  3.84s/it]
                                                 
{'loss': 1.1917, 'grad_norm': 0.4974331855773926, 'learning_rate': 2.3129104542574433e-06, 'epoch': 7.85}

 79%|███████▊  | 613/780 [39:46<10:41,  3.84s/it]
 79%|███████▊  | 614/780 [39:50<10:52,  3.93s/it]
                                                 
{'loss': 1.1443, 'grad_norm': 0.4337228834629059, 'learning_rate': 2.2863980844900036e-06, 'epoch': 7.86}

 79%|███████▊  | 614/780 [39:50<10:52,  3.93s/it]
 79%|███████▉  | 615/780 [39:54<11:08,  4.05s/it]
                                                 
{'loss': 1.1657, 'grad_norm': 0.43833422660827637, 'learning_rate': 2.2600189173377263e-06, 'epoch': 7.87}

 79%|███████▉  | 615/780 [39:54<11:08,  4.05s/it]
 79%|███████▉  | 616/780 [39:58<11:09,  4.08s/it]
                                                 
{'loss': 1.1877, 'grad_norm': 0.4507172703742981, 'learning_rate': 2.2337734083302164e-06, 'epoch': 7.88}

 79%|███████▉  | 616/780 [39:58<11:09,  4.08s/it]
 79%|███████▉  | 617/780 [40:02<10:56,  4.03s/it]
                                                 
{'loss': 1.1603, 'grad_norm': 0.49315640330314636, 'learning_rate': 2.207662010689002e-06, 'epoch': 7.9}

 79%|███████▉  | 617/780 [40:02<10:56,  4.03s/it]
 79%|███████▉  | 618/780 [40:06<10:54,  4.04s/it]
                                                 
{'loss': 1.15, 'grad_norm': 0.5191084146499634, 'learning_rate': 2.1816851753197023e-06, 'epoch': 7.91}

 79%|███████▉  | 618/780 [40:06<10:54,  4.04s/it]
 79%|███████▉  | 619/780 [40:10<10:44,  4.00s/it]
                                                 
{'loss': 1.1786, 'grad_norm': 0.49902456998825073, 'learning_rate': 2.155843350804243e-06, 'epoch': 7.92}

 79%|███████▉  | 619/780 [40:10<10:44,  4.00s/it]
 79%|███████▉  | 620/780 [40:14<10:34,  3.97s/it]
                                                 
{'loss': 1.1658, 'grad_norm': 0.49146607518196106, 'learning_rate': 2.130136983393112e-06, 'epoch': 7.94}

 79%|███████▉  | 620/780 [40:14<10:34,  3.97s/it]
 80%|███████▉  | 621/780 [40:17<10:07,  3.82s/it]
                                                 
{'loss': 1.1015, 'grad_norm': 0.528048038482666, 'learning_rate': 2.104566516997647e-06, 'epoch': 7.95}

 80%|███████▉  | 621/780 [40:17<10:07,  3.82s/it]
 80%|███████▉  | 622/780 [40:21<09:47,  3.72s/it]
                                                 
{'loss': 1.1468, 'grad_norm': 0.5430088043212891, 'learning_rate': 2.0791323931823783e-06, 'epoch': 7.96}

 80%|███████▉  | 622/780 [40:21<09:47,  3.72s/it]
 80%|███████▉  | 623/780 [40:25<09:44,  3.73s/it]
                                                 
{'loss': 1.1565, 'grad_norm': 0.5190221667289734, 'learning_rate': 2.053835051157397e-06, 'epoch': 7.97}

 80%|███████▉  | 623/780 [40:25<09:44,  3.73s/it]
 80%|████████  | 624/780 [40:29<09:50,  3.79s/it]
                                                 
{'loss': 1.2018, 'grad_norm': 0.537639319896698, 'learning_rate': 2.0286749277707783e-06, 'epoch': 7.99}

 80%|████████  | 624/780 [40:29<09:50,  3.79s/it]
 80%|████████  | 625/780 [40:33<10:30,  4.07s/it]
                                                 
{'loss': 1.1735, 'grad_norm': 0.5208773612976074, 'learning_rate': 2.0036524575010176e-06, 'epoch': 8.0}

 80%|████████  | 625/780 [40:33<10:30,  4.07s/it]
 80%|████████  | 626/780 [40:37<10:09,  3.96s/it]
                                                 
{'loss': 1.2016, 'grad_norm': 0.45588743686676025, 'learning_rate': 1.9787680724495617e-06, 'epoch': 8.01}

 80%|████████  | 626/780 [40:37<10:09,  3.96s/it]
 80%|████████  | 627/780 [40:41<10:00,  3.92s/it]
                                                 
{'loss': 1.1146, 'grad_norm': 0.49239757657051086, 'learning_rate': 1.9540222023333165e-06, 'epoch': 8.03}

 80%|████████  | 627/780 [40:41<10:00,  3.92s/it]
 81%|████████  | 628/780 [40:45<09:53,  3.90s/it]
                                                 
{'loss': 1.1846, 'grad_norm': 0.5272413492202759, 'learning_rate': 1.929415274477239e-06, 'epoch': 8.04}

 81%|████████  | 628/780 [40:45<09:53,  3.90s/it]
 81%|████████  | 629/780 [40:48<09:40,  3.84s/it]
                                                 
{'loss': 1.2584, 'grad_norm': 0.45346322655677795, 'learning_rate': 1.9049477138069606e-06, 'epoch': 8.05}

 81%|████████  | 629/780 [40:48<09:40,  3.84s/it]
 81%|████████  | 630/780 [40:52<09:12,  3.69s/it]
                                                 
{'loss': 1.2078, 'grad_norm': 0.4795394241809845, 'learning_rate': 1.880619942841435e-06, 'epoch': 8.06}

 81%|████████  | 630/780 [40:52<09:12,  3.69s/it]
 81%|████████  | 631/780 [40:56<09:32,  3.84s/it]
                                                 
{'loss': 1.1125, 'grad_norm': 0.4207776188850403, 'learning_rate': 1.856432381685669e-06, 'epoch': 8.08}

 81%|████████  | 631/780 [40:56<09:32,  3.84s/it]
 81%|████████  | 632/780 [41:00<09:38,  3.91s/it]
                                                 
{'loss': 1.2088, 'grad_norm': 0.5347769856452942, 'learning_rate': 1.8323854480234348e-06, 'epoch': 8.09}

 81%|████████  | 632/780 [41:00<09:38,  3.91s/it]
 81%|████████  | 633/780 [41:04<09:31,  3.89s/it]
                                                 
{'loss': 1.1925, 'grad_norm': 0.4902494549751282, 'learning_rate': 1.808479557110081e-06, 'epoch': 8.1}

 81%|████████  | 633/780 [41:04<09:31,  3.89s/it]
 81%|████████▏ | 634/780 [41:07<09:15,  3.81s/it]
                                                 
{'loss': 1.2196, 'grad_norm': 0.4687245786190033, 'learning_rate': 1.7847151217653624e-06, 'epoch': 8.12}

 81%|████████▏ | 634/780 [41:07<09:15,  3.81s/it]
 81%|████████▏ | 635/780 [41:11<09:21,  3.87s/it]
                                                 
{'loss': 1.1852, 'grad_norm': 0.4811779856681824, 'learning_rate': 1.7610925523662836e-06, 'epoch': 8.13}

 81%|████████▏ | 635/780 [41:11<09:21,  3.87s/it]
 82%|████████▏ | 636/780 [41:15<09:25,  3.92s/it]
                                                 
{'loss': 1.1787, 'grad_norm': 0.503379225730896, 'learning_rate': 1.7376122568400533e-06, 'epoch': 8.14}

 82%|████████▏ | 636/780 [41:15<09:25,  3.92s/it]
 82%|████████▏ | 637/780 [41:19<09:17,  3.90s/it]
                                                 
{'loss': 1.1912, 'grad_norm': 0.5562422871589661, 'learning_rate': 1.714274640657001e-06, 'epoch': 8.15}

 82%|████████▏ | 637/780 [41:19<09:17,  3.90s/it]
 82%|████████▏ | 638/780 [41:24<09:31,  4.03s/it]
                                                 
{'loss': 1.2552, 'grad_norm': 0.7039389610290527, 'learning_rate': 1.6910801068236015e-06, 'epoch': 8.17}

 82%|████████▏ | 638/780 [41:24<09:31,  4.03s/it]
 82%|████████▏ | 639/780 [41:28<09:33,  4.07s/it]
                                                 
{'loss': 1.1076, 'grad_norm': 0.5253216624259949, 'learning_rate': 1.6680290558755119e-06, 'epoch': 8.18}

 82%|████████▏ | 639/780 [41:28<09:33,  4.07s/it]
 82%|████████▏ | 640/780 [41:32<09:16,  3.97s/it]
                                                 
{'loss': 1.1512, 'grad_norm': 0.44163015484809875, 'learning_rate': 1.6451218858706374e-06, 'epoch': 8.19}

 82%|████████▏ | 640/780 [41:32<09:16,  3.97s/it]
 82%|████████▏ | 641/780 [41:35<09:06,  3.93s/it]
                                                 
{'loss': 1.2718, 'grad_norm': 0.414775013923645, 'learning_rate': 1.6223589923822768e-06, 'epoch': 8.2}

 82%|████████▏ | 641/780 [41:35<09:06,  3.93s/it]
 82%|████████▏ | 642/780 [41:39<09:04,  3.94s/it]
                                                 
{'loss': 1.2501, 'grad_norm': 0.4084765911102295, 'learning_rate': 1.599740768492286e-06, 'epoch': 8.22}

 82%|████████▏ | 642/780 [41:39<09:04,  3.94s/it]
 82%|████████▏ | 643/780 [41:43<09:01,  3.95s/it]
                                                 
{'loss': 1.1679, 'grad_norm': 0.4602608382701874, 'learning_rate': 1.5772676047842862e-06, 'epoch': 8.23}

 82%|████████▏ | 643/780 [41:43<09:01,  3.95s/it]
 83%|████████▎ | 644/780 [41:47<08:59,  3.96s/it]
                                                 
{'loss': 1.1647, 'grad_norm': 0.4907259941101074, 'learning_rate': 1.5549398893369216e-06, 'epoch': 8.24}

 83%|████████▎ | 644/780 [41:47<08:59,  3.96s/it]
 83%|████████▎ | 645/780 [41:51<08:58,  3.99s/it]
                                                 
{'loss': 1.1872, 'grad_norm': 0.4839785695075989, 'learning_rate': 1.5327580077171589e-06, 'epoch': 8.26}

 83%|████████▎ | 645/780 [41:51<08:58,  3.99s/it]
 83%|████████▎ | 646/780 [41:55<08:57,  4.01s/it]
                                                 
{'loss': 1.1716, 'grad_norm': 0.48580408096313477, 'learning_rate': 1.5107223429736273e-06, 'epoch': 8.27}

 83%|████████▎ | 646/780 [41:55<08:57,  4.01s/it]
 83%|████████▎ | 647/780 [41:59<08:47,  3.97s/it]
                                                 
{'loss': 1.2156, 'grad_norm': 0.6190848350524902, 'learning_rate': 1.4888332756300027e-06, 'epoch': 8.28}

 83%|████████▎ | 647/780 [41:59<08:47,  3.97s/it]
 83%|████████▎ | 648/780 [42:03<08:42,  3.96s/it]
                                                 
{'loss': 1.1356, 'grad_norm': 0.4838945269584656, 'learning_rate': 1.467091183678444e-06, 'epoch': 8.29}

 83%|████████▎ | 648/780 [42:03<08:42,  3.96s/it]
 83%|████████▎ | 649/780 [42:07<08:40,  3.97s/it]
                                                 
{'loss': 1.2345, 'grad_norm': 0.4563795328140259, 'learning_rate': 1.4454964425730533e-06, 'epoch': 8.31}

 83%|████████▎ | 649/780 [42:07<08:40,  3.97s/it]
 83%|████████▎ | 650/780 [42:11<08:36,  3.98s/it]
                                                 
{'loss': 1.1655, 'grad_norm': 0.48733094334602356, 'learning_rate': 1.424049425223405e-06, 'epoch': 8.32}

 83%|████████▎ | 650/780 [42:11<08:36,  3.98s/it]
 83%|████████▎ | 651/780 [42:15<08:38,  4.02s/it]
                                                 
{'loss': 1.208, 'grad_norm': 0.40480494499206543, 'learning_rate': 1.4027505019880972e-06, 'epoch': 8.33}

 83%|████████▎ | 651/780 [42:15<08:38,  4.02s/it]
 84%|████████▎ | 652/780 [42:19<08:16,  3.88s/it]
                                                 
{'loss': 1.1272, 'grad_norm': 0.5322428345680237, 'learning_rate': 1.3816000406683604e-06, 'epoch': 8.35}

 84%|████████▎ | 652/780 [42:19<08:16,  3.88s/it]
 84%|████████▎ | 653/780 [42:23<08:13,  3.88s/it]
                                                 
{'loss': 1.2428, 'grad_norm': 0.48056209087371826, 'learning_rate': 1.3605984065017074e-06, 'epoch': 8.36}

 84%|████████▎ | 653/780 [42:23<08:13,  3.88s/it]
 84%|████████▍ | 654/780 [42:26<08:02,  3.83s/it]
                                                 
{'loss': 1.2309, 'grad_norm': 0.5049417018890381, 'learning_rate': 1.339745962155613e-06, 'epoch': 8.37}

 84%|████████▍ | 654/780 [42:27<08:02,  3.83s/it]
 84%|████████▍ | 655/780 [42:31<08:09,  3.91s/it]
                                                 
{'loss': 1.1373, 'grad_norm': 0.6372118592262268, 'learning_rate': 1.3190430677212795e-06, 'epoch': 8.38}

 84%|████████▍ | 655/780 [42:31<08:09,  3.91s/it]
 84%|████████▍ | 656/780 [42:34<08:03,  3.90s/it]
                                                 
{'loss': 1.1788, 'grad_norm': 0.47322821617126465, 'learning_rate': 1.2984900807073919e-06, 'epoch': 8.4}

 84%|████████▍ | 656/780 [42:34<08:03,  3.90s/it]
 84%|████████▍ | 657/780 [42:38<07:59,  3.90s/it]
                                                 
{'loss': 1.147, 'grad_norm': 0.4472380578517914, 'learning_rate': 1.278087356033947e-06, 'epoch': 8.41}

 84%|████████▍ | 657/780 [42:38<07:59,  3.90s/it]
 84%|████████▍ | 658/780 [42:42<07:53,  3.88s/it]
                                                 
{'loss': 1.1705, 'grad_norm': 0.5130479335784912, 'learning_rate': 1.2578352460261456e-06, 'epoch': 8.42}

 84%|████████▍ | 658/780 [42:42<07:53,  3.88s/it]
 84%|████████▍ | 659/780 [42:46<07:54,  3.92s/it]
                                                 
{'loss': 1.2505, 'grad_norm': 0.49585849046707153, 'learning_rate': 1.2377341004082778e-06, 'epoch': 8.44}

 84%|████████▍ | 659/780 [42:46<07:54,  3.92s/it]
 85%|████████▍ | 660/780 [42:50<07:43,  3.86s/it]
                                                 
{'loss': 1.2141, 'grad_norm': 0.4919581413269043, 'learning_rate': 1.2177842662977136e-06, 'epoch': 8.45}

 85%|████████▍ | 660/780 [42:50<07:43,  3.86s/it]
 85%|████████▍ | 661/780 [42:54<07:57,  4.01s/it]
                                                 
{'loss': 1.1435, 'grad_norm': 0.4391363859176636, 'learning_rate': 1.1979860881988903e-06, 'epoch': 8.46}

 85%|████████▍ | 661/780 [42:54<07:57,  4.01s/it]
 85%|████████▍ | 662/780 [42:58<07:45,  3.95s/it]
                                                 
{'loss': 1.2717, 'grad_norm': 0.4584686756134033, 'learning_rate': 1.1783399079973578e-06, 'epoch': 8.47}

 85%|████████▍ | 662/780 [42:58<07:45,  3.95s/it]
 85%|████████▌ | 663/780 [43:02<07:37,  3.91s/it]
                                                 
{'loss': 1.1915, 'grad_norm': 0.4898507595062256, 'learning_rate': 1.1588460649539036e-06, 'epoch': 8.49}

 85%|████████▌ | 663/780 [43:02<07:37,  3.91s/it]
 85%|████████▌ | 664/780 [43:06<07:25,  3.84s/it]
                                                 
{'loss': 1.2222, 'grad_norm': 0.4170719087123871, 'learning_rate': 1.1395048956986577e-06, 'epoch': 8.5}

 85%|████████▌ | 664/780 [43:06<07:25,  3.84s/it]
 85%|████████▌ | 665/780 [43:09<07:16,  3.80s/it]
                                                 
{'loss': 1.1433, 'grad_norm': 0.6450399160385132, 'learning_rate': 1.1203167342253063e-06, 'epoch': 8.51}

 85%|████████▌ | 665/780 [43:09<07:16,  3.80s/it]
 85%|████████▌ | 666/780 [43:13<07:21,  3.87s/it]
                                                 
{'loss': 1.1707, 'grad_norm': 0.4229509234428406, 'learning_rate': 1.1012819118853147e-06, 'epoch': 8.52}

 85%|████████▌ | 666/780 [43:13<07:21,  3.87s/it]
 86%|████████▌ | 667/780 [43:17<07:26,  3.95s/it]
                                                 
{'loss': 1.2138, 'grad_norm': 0.513168215751648, 'learning_rate': 1.0824007573822025e-06, 'epoch': 8.54}

 86%|████████▌ | 667/780 [43:17<07:26,  3.95s/it]
 86%|████████▌ | 668/780 [43:21<07:14,  3.88s/it]
                                                 
{'loss': 1.1983, 'grad_norm': 0.48203805088996887, 'learning_rate': 1.0636735967658785e-06, 'epoch': 8.55}

 86%|████████▌ | 668/780 [43:21<07:14,  3.88s/it]
 86%|████████▌ | 669/780 [43:25<06:54,  3.73s/it]
                                                 
{'loss': 1.1929, 'grad_norm': 0.5433357954025269, 'learning_rate': 1.0451007534269908e-06, 'epoch': 8.56}

 86%|████████▌ | 669/780 [43:25<06:54,  3.73s/it]
 86%|████████▌ | 670/780 [43:28<06:49,  3.72s/it]
                                                 
{'loss': 1.2033, 'grad_norm': 0.4809808135032654, 'learning_rate': 1.026682548091361e-06, 'epoch': 8.58}

 86%|████████▌ | 670/780 [43:28<06:49,  3.72s/it]
 86%|████████▌ | 671/780 [43:33<07:03,  3.89s/it]
                                                 
{'loss': 1.1873, 'grad_norm': 0.4440232813358307, 'learning_rate': 1.0084192988144392e-06, 'epoch': 8.59}

 86%|████████▌ | 671/780 [43:33<07:03,  3.89s/it]
 86%|████████▌ | 672/780 [43:37<07:09,  3.97s/it]
                                                 
{'loss': 1.1231, 'grad_norm': 0.5066071152687073, 'learning_rate': 9.903113209758098e-07, 'epoch': 8.6}

 86%|████████▌ | 672/780 [43:37<07:09,  3.97s/it]
 86%|████████▋ | 673/780 [43:41<07:05,  3.98s/it]
                                                 
{'loss': 1.1677, 'grad_norm': 0.47913211584091187, 'learning_rate': 9.723589272737443e-07, 'epoch': 8.61}

 86%|████████▋ | 673/780 [43:41<07:05,  3.98s/it]
 86%|████████▋ | 674/780 [43:45<07:21,  4.16s/it]
                                                 
{'loss': 1.2011, 'grad_norm': 0.5521857142448425, 'learning_rate': 9.545624277198085e-07, 'epoch': 8.63}

 86%|████████▋ | 674/780 [43:45<07:21,  4.16s/it]
 87%|████████▋ | 675/780 [43:49<07:01,  4.02s/it]
                                                 
{'loss': 1.193, 'grad_norm': 0.5604910254478455, 'learning_rate': 9.369221296335007e-07, 'epoch': 8.64}

 87%|████████▋ | 675/780 [43:49<07:01,  4.02s/it]
 87%|████████▋ | 676/780 [43:53<06:56,  4.01s/it]
                                                 
{'loss': 1.1704, 'grad_norm': 0.4563933312892914, 'learning_rate': 9.194383376369509e-07, 'epoch': 8.65}

 87%|████████▋ | 676/780 [43:53<06:56,  4.01s/it]
 87%|████████▋ | 677/780 [43:56<06:30,  3.79s/it]
                                                 
{'loss': 1.1804, 'grad_norm': 0.4883592426776886, 'learning_rate': 9.021113536496551e-07, 'epoch': 8.67}

 87%|████████▋ | 677/780 [43:56<06:30,  3.79s/it]
 87%|████████▋ | 678/780 [44:00<06:24,  3.77s/it]
                                                 
{'loss': 1.1522, 'grad_norm': 0.49009227752685547, 'learning_rate': 8.849414768832687e-07, 'epoch': 8.68}

 87%|████████▋ | 678/780 [44:00<06:24,  3.77s/it]
 87%|████████▋ | 679/780 [44:04<06:14,  3.70s/it]
                                                 
{'loss': 1.1472, 'grad_norm': 0.47268426418304443, 'learning_rate': 8.679290038364319e-07, 'epoch': 8.69}

 87%|████████▋ | 679/780 [44:04<06:14,  3.70s/it]
 87%|████████▋ | 680/780 [44:07<06:14,  3.75s/it]
                                                 
{'loss': 1.1577, 'grad_norm': 0.5383321046829224, 'learning_rate': 8.510742282896545e-07, 'epoch': 8.7}

 87%|████████▋ | 680/780 [44:07<06:14,  3.75s/it]
 87%|████████▋ | 681/780 [44:12<06:23,  3.88s/it]
                                                 
{'loss': 1.1791, 'grad_norm': 0.5055539608001709, 'learning_rate': 8.343774413002382e-07, 'epoch': 8.72}

 87%|████████▋ | 681/780 [44:12<06:23,  3.88s/it]
 87%|████████▋ | 682/780 [44:16<06:23,  3.91s/it]
                                                 
{'loss': 1.185, 'grad_norm': 0.47748488187789917, 'learning_rate': 8.178389311972612e-07, 'epoch': 8.73}

 87%|████████▋ | 682/780 [44:16<06:23,  3.91s/it]
 88%|████████▊ | 683/780 [44:19<06:12,  3.84s/it]
                                                 
{'loss': 1.1783, 'grad_norm': 0.47497034072875977, 'learning_rate': 8.014589835765807e-07, 'epoch': 8.74}

 88%|████████▊ | 683/780 [44:19<06:12,  3.84s/it]
 88%|████████▊ | 684/780 [44:23<06:01,  3.77s/it]
                                                 
{'loss': 1.2247, 'grad_norm': 0.4573078155517578, 'learning_rate': 7.852378812959227e-07, 'epoch': 8.76}

 88%|████████▊ | 684/780 [44:23<06:01,  3.77s/it]
 88%|████████▊ | 685/780 [44:27<06:04,  3.83s/it]
                                                 
{'loss': 1.1933, 'grad_norm': 0.5126529335975647, 'learning_rate': 7.69175904469982e-07, 'epoch': 8.77}

 88%|████████▊ | 685/780 [44:27<06:04,  3.83s/it]
 88%|████████▊ | 686/780 [44:31<06:04,  3.88s/it]
                                                 
{'loss': 1.2304, 'grad_norm': 0.6597140431404114, 'learning_rate': 7.532733304655848e-07, 'epoch': 8.78}

 88%|████████▊ | 686/780 [44:31<06:04,  3.88s/it]
 88%|████████▊ | 687/780 [44:34<05:56,  3.83s/it]
                                                 
{'loss': 1.1339, 'grad_norm': 0.4647556245326996, 'learning_rate': 7.375304338969135e-07, 'epoch': 8.79}

 88%|████████▊ | 687/780 [44:34<05:56,  3.83s/it]
 88%|████████▊ | 688/780 [44:38<05:47,  3.78s/it]
                                                 
{'loss': 1.2084, 'grad_norm': 0.4325316846370697, 'learning_rate': 7.219474866207465e-07, 'epoch': 8.81}

 88%|████████▊ | 688/780 [44:38<05:47,  3.78s/it]
 88%|████████▊ | 689/780 [44:42<05:54,  3.89s/it]
                                                 
{'loss': 1.1748, 'grad_norm': 0.43047666549682617, 'learning_rate': 7.065247577317747e-07, 'epoch': 8.82}

 88%|████████▊ | 689/780 [44:42<05:54,  3.89s/it]
 88%|████████▊ | 690/780 [44:46<05:56,  3.96s/it]
                                                 
{'loss': 1.1999, 'grad_norm': 0.47731027007102966, 'learning_rate': 6.912625135579587e-07, 'epoch': 8.83}

 88%|████████▊ | 690/780 [44:46<05:56,  3.96s/it]
 89%|████████▊ | 691/780 [44:50<05:45,  3.89s/it]
                                                 
{'loss': 1.2192, 'grad_norm': 0.4965769648551941, 'learning_rate': 6.761610176559086e-07, 'epoch': 8.84}

 89%|████████▊ | 691/780 [44:50<05:45,  3.89s/it]
 89%|████████▊ | 692/780 [44:54<05:47,  3.95s/it]
                                                 
{'loss': 1.1259, 'grad_norm': 0.4550401270389557, 'learning_rate': 6.612205308063646e-07, 'epoch': 8.86}

 89%|████████▊ | 692/780 [44:54<05:47,  3.95s/it]
 89%|████████▉ | 693/780 [44:58<05:45,  3.98s/it]
                                                 
{'loss': 1.1955, 'grad_norm': 0.44438162446022034, 'learning_rate': 6.464413110096601e-07, 'epoch': 8.87}

 89%|████████▉ | 693/780 [44:58<05:45,  3.98s/it]
 89%|████████▉ | 694/780 [45:03<05:50,  4.08s/it]
                                                 
{'loss': 1.1605, 'grad_norm': 0.44215288758277893, 'learning_rate': 6.318236134812917e-07, 'epoch': 8.88}

 89%|████████▉ | 694/780 [45:03<05:50,  4.08s/it]
 89%|████████▉ | 695/780 [45:06<05:36,  3.96s/it]
                                                 
{'loss': 1.1606, 'grad_norm': 0.49016547203063965, 'learning_rate': 6.173676906475012e-07, 'epoch': 8.9}

 89%|████████▉ | 695/780 [45:06<05:36,  3.96s/it]
 89%|████████▉ | 696/780 [45:11<05:45,  4.11s/it]
                                                 
{'loss': 1.1485, 'grad_norm': 0.4277973175048828, 'learning_rate': 6.030737921409169e-07, 'epoch': 8.91}

 89%|████████▉ | 696/780 [45:11<05:45,  4.11s/it]
 89%|████████▉ | 697/780 [45:15<05:37,  4.06s/it]
                                                 
{'loss': 1.1851, 'grad_norm': 0.511518657207489, 'learning_rate': 5.889421647962456e-07, 'epoch': 8.92}

 89%|████████▉ | 697/780 [45:15<05:37,  4.06s/it]
 89%|████████▉ | 698/780 [45:18<05:25,  3.96s/it]
                                                 
{'loss': 1.1727, 'grad_norm': 0.5229935050010681, 'learning_rate': 5.749730526460073e-07, 'epoch': 8.93}

 89%|████████▉ | 698/780 [45:18<05:25,  3.96s/it]
 90%|████████▉ | 699/780 [45:22<05:14,  3.88s/it]
                                                 
{'loss': 1.1198, 'grad_norm': 0.5078153610229492, 'learning_rate': 5.611666969163243e-07, 'epoch': 8.95}

 90%|████████▉ | 699/780 [45:22<05:14,  3.88s/it]
 90%|████████▉ | 700/780 [45:25<04:58,  3.73s/it]
                                                 
{'loss': 1.1059, 'grad_norm': 0.5310805439949036, 'learning_rate': 5.475233360227516e-07, 'epoch': 8.96}

 90%|████████▉ | 700/780 [45:26<04:58,  3.73s/it]
 90%|████████▉ | 701/780 [45:29<04:52,  3.70s/it]
                                                 
{'loss': 1.1817, 'grad_norm': 0.5659341216087341, 'learning_rate': 5.340432055661637e-07, 'epoch': 8.97}

 90%|████████▉ | 701/780 [45:29<04:52,  3.70s/it]
 90%|█████████ | 702/780 [45:33<04:55,  3.79s/it]
                                                 
{'loss': 1.1706, 'grad_norm': 0.5397396087646484, 'learning_rate': 5.207265383286831e-07, 'epoch': 8.99}

 90%|█████████ | 702/780 [45:33<04:55,  3.79s/it]
 90%|█████████ | 703/780 [45:38<05:06,  3.99s/it]
                                                 
{'loss': 1.1756, 'grad_norm': 0.5140916705131531, 'learning_rate': 5.075735642696611e-07, 'epoch': 9.0}

 90%|█████████ | 703/780 [45:38<05:06,  3.99s/it]
 90%|█████████ | 704/780 [45:41<05:01,  3.97s/it]
                                                 
{'loss': 1.2112, 'grad_norm': 0.43735289573669434, 'learning_rate': 4.945845105217118e-07, 'epoch': 9.01}

 90%|█████████ | 704/780 [45:42<05:01,  3.97s/it]
 90%|█████████ | 705/780 [45:45<04:52,  3.90s/it]
                                                 
{'loss': 1.1112, 'grad_norm': 0.49940741062164307, 'learning_rate': 4.817596013867765e-07, 'epoch': 9.02}

 90%|█████████ | 705/780 [45:45<04:52,  3.90s/it]
 91%|█████████ | 706/780 [45:49<04:45,  3.86s/it]
                                                 
{'loss': 1.1689, 'grad_norm': 0.5114821791648865, 'learning_rate': 4.6909905833226965e-07, 'epoch': 9.04}

 91%|█████████ | 706/780 [45:49<04:45,  3.86s/it]
 91%|█████████ | 707/780 [45:53<04:37,  3.80s/it]
                                                 
{'loss': 1.269, 'grad_norm': 0.4746755361557007, 'learning_rate': 4.566030999872384e-07, 'epoch': 9.05}

 91%|█████████ | 707/780 [45:53<04:37,  3.80s/it]
 91%|█████████ | 708/780 [45:56<04:26,  3.70s/it]
                                                 
{'loss': 1.2465, 'grad_norm': 0.46881571412086487, 'learning_rate': 4.4427194213859216e-07, 'epoch': 9.06}

 91%|█████████ | 708/780 [45:56<04:26,  3.70s/it]
 91%|█████████ | 709/780 [46:00<04:32,  3.84s/it]
                                                 
{'loss': 1.0866, 'grad_norm': 0.4387238025665283, 'learning_rate': 4.3210579772738237e-07, 'epoch': 9.08}

 91%|█████████ | 709/780 [46:00<04:32,  3.84s/it]
 91%|█████████ | 710/780 [46:04<04:35,  3.94s/it]
                                                 
{'loss': 1.1483, 'grad_norm': 0.5190503597259521, 'learning_rate': 4.2010487684511105e-07, 'epoch': 9.09}

 91%|█████████ | 710/780 [46:04<04:35,  3.94s/it]
 91%|█████████ | 711/780 [46:08<04:32,  3.95s/it]
                                                 
{'loss': 1.2461, 'grad_norm': 0.49966275691986084, 'learning_rate': 4.082693867301224e-07, 'epoch': 9.1}

 91%|█████████ | 711/780 [46:08<04:32,  3.95s/it]
 91%|█████████▏| 712/780 [46:12<04:21,  3.84s/it]
                                                 
{'loss': 1.2283, 'grad_norm': 0.45081230998039246, 'learning_rate': 3.965995317640026e-07, 'epoch': 9.11}

 91%|█████████▏| 712/780 [46:12<04:21,  3.84s/it]
 91%|█████████▏| 713/780 [46:16<04:20,  3.89s/it]
                                                 
{'loss': 1.1692, 'grad_norm': 0.4911956489086151, 'learning_rate': 3.850955134680678e-07, 'epoch': 9.13}

 91%|█████████▏| 713/780 [46:16<04:20,  3.89s/it]
 92%|█████████▏| 714/780 [46:20<04:20,  3.94s/it]
                                                 
{'loss': 1.1751, 'grad_norm': 0.4741252362728119, 'learning_rate': 3.7375753049987974e-07, 'epoch': 9.14}

 92%|█████████▏| 714/780 [46:20<04:20,  3.94s/it]
 92%|█████████▏| 715/780 [46:24<04:16,  3.94s/it]
                                                 
{'loss': 1.2116, 'grad_norm': 0.5502737164497375, 'learning_rate': 3.625857786498055e-07, 'epoch': 9.15}

 92%|█████████▏| 715/780 [46:24<04:16,  3.94s/it]
 92%|█████████▏| 716/780 [46:28<04:16,  4.01s/it]
                                                 
{'loss': 1.2025, 'grad_norm': 0.6341015696525574, 'learning_rate': 3.515804508376508e-07, 'epoch': 9.16}

 92%|█████████▏| 716/780 [46:28<04:16,  4.01s/it]
 92%|█████████▏| 717/780 [46:32<04:17,  4.09s/it]
                                                 
{'loss': 1.1424, 'grad_norm': 0.5787403583526611, 'learning_rate': 3.4074173710931804e-07, 'epoch': 9.18}

 92%|█████████▏| 717/780 [46:32<04:17,  4.09s/it]
 92%|█████████▏| 718/780 [46:36<04:06,  3.98s/it]
                                                 
{'loss': 1.1248, 'grad_norm': 0.44629356265068054, 'learning_rate': 3.3006982463352764e-07, 'epoch': 9.19}

 92%|█████████▏| 718/780 [46:36<04:06,  3.98s/it]
 92%|█████████▏| 719/780 [46:40<04:00,  3.94s/it]
                                                 
{'loss': 1.2938, 'grad_norm': 0.4267333447933197, 'learning_rate': 3.1956489769859213e-07, 'epoch': 9.2}

 92%|█████████▏| 719/780 [46:40<04:00,  3.94s/it]
 92%|█████████▏| 720/780 [46:44<03:55,  3.93s/it]
                                                 
{'loss': 1.2344, 'grad_norm': 0.4231089651584625, 'learning_rate': 3.0922713770922155e-07, 'epoch': 9.22}

 92%|█████████▏| 720/780 [46:44<03:55,  3.93s/it]
 92%|█████████▏| 721/780 [46:48<03:49,  3.89s/it]
                                                 
{'loss': 1.194, 'grad_norm': 0.44055721163749695, 'learning_rate': 2.9905672318339963e-07, 'epoch': 9.23}

 92%|█████████▏| 721/780 [46:48<03:49,  3.89s/it]
 93%|█████████▎| 722/780 [46:52<03:43,  3.86s/it]
                                                 
{'loss': 1.1574, 'grad_norm': 0.5155954957008362, 'learning_rate': 2.8905382974930173e-07, 'epoch': 9.24}

 93%|█████████▎| 722/780 [46:52<03:43,  3.86s/it]
 93%|█████████▎| 723/780 [46:55<03:37,  3.81s/it]
                                                 
{'loss': 1.1651, 'grad_norm': 0.49665093421936035, 'learning_rate': 2.7921863014225504e-07, 'epoch': 9.25}

 93%|█████████▎| 723/780 [46:55<03:37,  3.81s/it]
 93%|█████████▎| 724/780 [47:00<03:43,  3.99s/it]
                                                 
{'loss': 1.2026, 'grad_norm': 0.47132617235183716, 'learning_rate': 2.6955129420176193e-07, 'epoch': 9.27}

 93%|█████████▎| 724/780 [47:00<03:43,  3.99s/it]
 93%|█████████▎| 725/780 [47:03<03:36,  3.93s/it]
                                                 
{'loss': 1.2096, 'grad_norm': 0.5128554105758667, 'learning_rate': 2.6005198886856486e-07, 'epoch': 9.28}

 93%|█████████▎| 725/780 [47:03<03:36,  3.93s/it]
 93%|█████████▎| 726/780 [47:08<03:35,  4.00s/it]
                                                 
{'loss': 1.1062, 'grad_norm': 0.6307947635650635, 'learning_rate': 2.507208781817638e-07, 'epoch': 9.29}

 93%|█████████▎| 726/780 [47:08<03:35,  4.00s/it]
 93%|█████████▎| 727/780 [47:11<03:29,  3.96s/it]
                                                 
{'loss': 1.2494, 'grad_norm': 0.4701201021671295, 'learning_rate': 2.4155812327598337e-07, 'epoch': 9.31}

 93%|█████████▎| 727/780 [47:11<03:29,  3.96s/it]
 93%|█████████▎| 728/780 [47:15<03:25,  3.95s/it]
                                                 
{'loss': 1.2022, 'grad_norm': 0.4659019708633423, 'learning_rate': 2.3256388237858806e-07, 'epoch': 9.32}

 93%|█████████▎| 728/780 [47:15<03:25,  3.95s/it]
 93%|█████████▎| 729/780 [47:19<03:23,  3.99s/it]
                                                 
{'loss': 1.1666, 'grad_norm': 0.4098786413669586, 'learning_rate': 2.2373831080695463e-07, 'epoch': 9.33}

 93%|█████████▎| 729/780 [47:19<03:23,  3.99s/it]
 94%|█████████▎| 730/780 [47:23<03:13,  3.87s/it]
                                                 
{'loss': 1.1558, 'grad_norm': 0.5173810720443726, 'learning_rate': 2.1508156096578748e-07, 'epoch': 9.34}

 94%|█████████▎| 730/780 [47:23<03:13,  3.87s/it]
 94%|█████████▎| 731/780 [47:27<03:09,  3.87s/it]
                                                 
{'loss': 1.2416, 'grad_norm': 0.3875299096107483, 'learning_rate': 2.0659378234448524e-07, 'epoch': 9.36}

 94%|█████████▎| 731/780 [47:27<03:09,  3.87s/it]
 94%|█████████▍| 732/780 [47:31<03:05,  3.86s/it]
                                                 
{'loss': 1.2124, 'grad_norm': 0.5667559504508972, 'learning_rate': 1.9827512151456175e-07, 'epoch': 9.37}

 94%|█████████▍| 732/780 [47:31<03:05,  3.86s/it]
 94%|█████████▍| 733/780 [47:35<03:03,  3.89s/it]
                                                 
{'loss': 1.1453, 'grad_norm': 0.6612321138381958, 'learning_rate': 1.9012572212711467e-07, 'epoch': 9.38}

 94%|█████████▍| 733/780 [47:35<03:03,  3.89s/it]
 94%|█████████▍| 734/780 [47:39<03:00,  3.92s/it]
                                                 
{'loss': 1.1908, 'grad_norm': 0.46498748660087585, 'learning_rate': 1.82145724910342e-07, 'epoch': 9.4}

 94%|█████████▍| 734/780 [47:39<03:00,  3.92s/it]
 94%|█████████▍| 735/780 [47:43<02:55,  3.91s/it]
                                                 
{'loss': 1.1453, 'grad_norm': 0.4299646317958832, 'learning_rate': 1.7433526766711727e-07, 'epoch': 9.41}

 94%|█████████▍| 735/780 [47:43<02:55,  3.91s/it]
 94%|█████████▍| 736/780 [47:46<02:51,  3.89s/it]
                                                 
{'loss': 1.1438, 'grad_norm': 0.49311941862106323, 'learning_rate': 1.6669448527260602e-07, 'epoch': 9.42}

 94%|█████████▍| 736/780 [47:46<02:51,  3.89s/it]
 94%|█████████▍| 737/780 [47:50<02:48,  3.92s/it]
                                                 
{'loss': 1.2605, 'grad_norm': 0.5424204468727112, 'learning_rate': 1.5922350967193524e-07, 'epoch': 9.43}

 94%|█████████▍| 737/780 [47:50<02:48,  3.92s/it]
 95%|█████████▍| 738/780 [47:54<02:46,  3.96s/it]
                                                 
{'loss': 1.1887, 'grad_norm': 0.5026274919509888, 'learning_rate': 1.519224698779198e-07, 'epoch': 9.45}

 95%|█████████▍| 738/780 [47:55<02:46,  3.96s/it]
 95%|█████████▍| 739/780 [47:58<02:41,  3.93s/it]
                                                 
{'loss': 1.162, 'grad_norm': 0.43919023871421814, 'learning_rate': 1.447914919688298e-07, 'epoch': 9.46}

 95%|█████████▍| 739/780 [47:58<02:41,  3.93s/it]
 95%|█████████▍| 740/780 [48:02<02:36,  3.90s/it]
                                                 
{'loss': 1.2808, 'grad_norm': 0.4686520993709564, 'learning_rate': 1.3783069908621772e-07, 'epoch': 9.47}

 95%|█████████▍| 740/780 [48:02<02:36,  3.90s/it]
 95%|█████████▌| 741/780 [48:06<02:28,  3.80s/it]
                                                 
{'loss': 1.1809, 'grad_norm': 0.49766433238983154, 'learning_rate': 1.3104021143278911e-07, 'epoch': 9.48}

 95%|█████████▌| 741/780 [48:06<02:28,  3.80s/it]
 95%|█████████▌| 742/780 [48:10<02:25,  3.83s/it]
                                                 
{'loss': 1.2361, 'grad_norm': 0.4001959562301636, 'learning_rate': 1.2442014627032318e-07, 'epoch': 9.5}

 95%|█████████▌| 742/780 [48:10<02:25,  3.83s/it]
 95%|█████████▌| 743/780 [48:13<02:20,  3.78s/it]
                                                 
{'loss': 1.1459, 'grad_norm': 0.6718232035636902, 'learning_rate': 1.1797061791766207e-07, 'epoch': 9.51}

 95%|█████████▌| 743/780 [48:13<02:20,  3.78s/it]
 95%|█████████▌| 744/780 [48:17<02:18,  3.84s/it]
                                                 
{'loss': 1.151, 'grad_norm': 0.44561320543289185, 'learning_rate': 1.1169173774871478e-07, 'epoch': 9.52}

 95%|█████████▌| 744/780 [48:17<02:18,  3.84s/it]
 96%|█████████▌| 745/780 [48:21<02:15,  3.88s/it]
                                                 
{'loss': 1.2138, 'grad_norm': 0.5122560858726501, 'learning_rate': 1.055836141905553e-07, 'epoch': 9.54}

 96%|█████████▌| 745/780 [48:21<02:15,  3.88s/it]
 96%|█████████▌| 746/780 [48:25<02:12,  3.90s/it]
                                                 
{'loss': 1.1769, 'grad_norm': 0.46973156929016113, 'learning_rate': 9.964635272153633e-08, 'epoch': 9.55}

 96%|█████████▌| 746/780 [48:25<02:12,  3.90s/it]
 96%|█████████▌| 747/780 [48:29<02:03,  3.73s/it]
                                                 
{'loss': 1.2379, 'grad_norm': 0.5318592190742493, 'learning_rate': 9.388005586947191e-08, 'epoch': 9.56}

 96%|█████████▌| 747/780 [48:29<02:03,  3.73s/it]
 96%|█████████▌| 748/780 [48:32<02:00,  3.75s/it]
                                                 
{'loss': 1.1916, 'grad_norm': 0.486337274312973, 'learning_rate': 8.82848232098732e-08, 'epoch': 9.57}

 96%|█████████▌| 748/780 [48:32<02:00,  3.75s/it]
 96%|█████████▌| 749/780 [48:37<02:01,  3.92s/it]
                                                 
{'loss': 1.196, 'grad_norm': 0.4543432891368866, 'learning_rate': 8.286075136421435e-08, 'epoch': 9.59}

 96%|█████████▌| 749/780 [48:37<02:01,  3.92s/it]
 96%|█████████▌| 750/780 [48:41<01:59,  3.99s/it]
                                                 
{'loss': 1.1311, 'grad_norm': 0.49283766746520996, 'learning_rate': 7.760793399827937e-08, 'epoch': 9.6}

 96%|█████████▌| 750/780 [48:41<01:59,  3.99s/it]
 96%|█████████▋| 751/780 [48:45<01:55,  4.00s/it]
                                                 
{'loss': 1.1547, 'grad_norm': 0.4979028105735779, 'learning_rate': 7.25264618205357e-08, 'epoch': 9.61}

 96%|█████████▋| 751/780 [48:45<01:55,  4.00s/it]
 96%|█████████▋| 752/780 [48:49<01:54,  4.08s/it]
                                                 
{'loss': 1.1804, 'grad_norm': 0.5263928771018982, 'learning_rate': 6.761642258056977e-08, 'epoch': 9.63}

 96%|█████████▋| 752/780 [48:49<01:54,  4.08s/it]
 97%|█████████▋| 753/780 [48:53<01:45,  3.92s/it]
                                                 
{'loss': 1.2237, 'grad_norm': 0.5663407444953918, 'learning_rate': 6.287790106757396e-08, 'epoch': 9.64}

 97%|█████████▋| 753/780 [48:53<01:45,  3.92s/it]
 97%|█████████▋| 754/780 [48:57<01:42,  3.93s/it]
                                                 
{'loss': 1.1514, 'grad_norm': 0.45657411217689514, 'learning_rate': 5.831097910887873e-08, 'epoch': 9.65}

 97%|█████████▋| 754/780 [48:57<01:42,  3.93s/it]
 97%|█████████▋| 755/780 [49:00<01:35,  3.82s/it]
                                                 
{'loss': 1.19, 'grad_norm': 0.4923362731933594, 'learning_rate': 5.391573556854157e-08, 'epoch': 9.66}

 97%|█████████▋| 755/780 [49:00<01:35,  3.82s/it]
 97%|█████████▋| 756/780 [49:04<01:32,  3.85s/it]
                                                 
{'loss': 1.1574, 'grad_norm': 0.4908851981163025, 'learning_rate': 4.9692246345985905e-08, 'epoch': 9.68}

 97%|█████████▋| 756/780 [49:04<01:32,  3.85s/it]
 97%|█████████▋| 757/780 [49:07<01:24,  3.69s/it]
                                                 
{'loss': 1.1403, 'grad_norm': 0.511456310749054, 'learning_rate': 4.564058437468877e-08, 'epoch': 9.69}

 97%|█████████▋| 757/780 [49:07<01:24,  3.69s/it]
 97%|█████████▋| 758/780 [49:11<01:23,  3.79s/it]
                                                 
{'loss': 1.1804, 'grad_norm': 0.5102495551109314, 'learning_rate': 4.176081962092182e-08, 'epoch': 9.7}

 97%|█████████▋| 758/780 [49:11<01:23,  3.79s/it]
 97%|█████████▋| 759/780 [49:16<01:22,  3.92s/it]
                                                 
{'loss': 1.1477, 'grad_norm': 0.5089027285575867, 'learning_rate': 3.805301908254455e-08, 'epoch': 9.72}

 97%|█████████▋| 759/780 [49:16<01:22,  3.92s/it]
 97%|█████████▋| 760/780 [49:20<01:19,  3.98s/it]
                                                 
{'loss': 1.1867, 'grad_norm': 0.472122460603714, 'learning_rate': 3.451724678784518e-08, 'epoch': 9.73}

 97%|█████████▋| 760/780 [49:20<01:19,  3.98s/it]
 98%|█████████▊| 761/780 [49:24<01:14,  3.91s/it]
                                                 
{'loss': 1.1783, 'grad_norm': 0.4789276123046875, 'learning_rate': 3.115356379443601e-08, 'epoch': 9.74}

 98%|█████████▊| 761/780 [49:24<01:14,  3.91s/it]
 98%|█████████▊| 762/780 [49:27<01:08,  3.81s/it]
                                                 
{'loss': 1.2077, 'grad_norm': 0.45640113949775696, 'learning_rate': 2.796202818819871e-08, 'epoch': 9.75}

 98%|█████████▊| 762/780 [49:27<01:08,  3.81s/it]
 98%|█████████▊| 763/780 [49:31<01:06,  3.94s/it]
                                                 
{'loss': 1.2109, 'grad_norm': 0.4750056266784668, 'learning_rate': 2.4942695082281752e-08, 'epoch': 9.77}

 98%|█████████▊| 763/780 [49:31<01:06,  3.94s/it]
 98%|█████████▊| 764/780 [49:35<01:03,  3.97s/it]
                                                 
{'loss': 1.2376, 'grad_norm': 0.641443133354187, 'learning_rate': 2.2095616616150117e-08, 'epoch': 9.78}

 98%|█████████▊| 764/780 [49:35<01:03,  3.97s/it]
 98%|█████████▊| 765/780 [49:39<00:57,  3.83s/it]
                                                 
{'loss': 1.1309, 'grad_norm': 0.49181249737739563, 'learning_rate': 1.9420841954681525e-08, 'epoch': 9.79}

 98%|█████████▊| 765/780 [49:39<00:57,  3.83s/it]
 98%|█████████▊| 766/780 [49:43<00:53,  3.79s/it]
                                                 
{'loss': 1.1921, 'grad_norm': 0.46789953112602234, 'learning_rate': 1.6918417287318245e-08, 'epoch': 9.8}

 98%|█████████▊| 766/780 [49:43<00:53,  3.79s/it]
 98%|█████████▊| 767/780 [49:46<00:49,  3.83s/it]
                                                 
{'loss': 1.1682, 'grad_norm': 0.4081015884876251, 'learning_rate': 1.4588385827272178e-08, 'epoch': 9.82}

 98%|█████████▊| 767/780 [49:47<00:49,  3.83s/it]
 98%|█████████▊| 768/780 [49:50<00:46,  3.85s/it]
                                                 
{'loss': 1.2228, 'grad_norm': 0.4636077880859375, 'learning_rate': 1.2430787810776556e-08, 'epoch': 9.83}

 98%|█████████▊| 768/780 [49:50<00:46,  3.85s/it]
 99%|█████████▊| 769/780 [49:54<00:42,  3.82s/it]
                                                 
{'loss': 1.2239, 'grad_norm': 0.49412891268730164, 'learning_rate': 1.0445660496390952e-08, 'epoch': 9.84}

 99%|█████████▊| 769/780 [49:54<00:42,  3.82s/it]
 99%|█████████▊| 770/780 [49:58<00:39,  3.91s/it]
                                                 
{'loss': 1.1192, 'grad_norm': 0.47984132170677185, 'learning_rate': 8.633038164358454e-09, 'epoch': 9.86}

 99%|█████████▊| 770/780 [49:58<00:39,  3.91s/it]
 99%|█████████▉| 771/780 [50:03<00:36,  4.01s/it]
                                                 
{'loss': 1.1991, 'grad_norm': 0.4259422719478607, 'learning_rate': 6.992952116013918e-09, 'epoch': 9.87}

 99%|█████████▉| 771/780 [50:03<00:36,  4.01s/it]
 99%|█████████▉| 772/780 [50:07<00:32,  4.05s/it]
                                                 
{'loss': 1.1817, 'grad_norm': 0.4189130365848541, 'learning_rate': 5.525430673244403e-09, 'epoch': 9.88}

 99%|█████████▉| 772/780 [50:07<00:32,  4.05s/it]
 99%|█████████▉| 773/780 [50:10<00:27,  3.98s/it]
                                                 
{'loss': 1.1788, 'grad_norm': 0.45971712470054626, 'learning_rate': 4.230499177994007e-09, 'epoch': 9.89}

 99%|█████████▉| 773/780 [50:11<00:27,  3.98s/it]
 99%|█████████▉| 774/780 [50:15<00:24,  4.04s/it]
                                                 
{'loss': 1.1018, 'grad_norm': 0.47300171852111816, 'learning_rate': 3.1081799918375454e-09, 'epoch': 9.91}

 99%|█████████▉| 774/780 [50:15<00:24,  4.04s/it]
 99%|█████████▉| 775/780 [50:18<00:19,  3.97s/it]
                                                 
{'loss': 1.1829, 'grad_norm': 0.4830387532711029, 'learning_rate': 2.1584924955819763e-09, 'epoch': 9.92}

 99%|█████████▉| 775/780 [50:18<00:19,  3.97s/it]
 99%|█████████▉| 776/780 [50:22<00:15,  3.84s/it]
                                                 
{'loss': 1.2034, 'grad_norm': 0.5187822580337524, 'learning_rate': 1.3814530889433298e-09, 'epoch': 9.93}

 99%|█████████▉| 776/780 [50:22<00:15,  3.84s/it]
100%|█████████▉| 777/780 [50:26<00:11,  3.85s/it]
                                                 
{'loss': 1.0903, 'grad_norm': 0.4912620782852173, 'learning_rate': 7.770751902513862e-10, 'epoch': 9.95}

100%|█████████▉| 777/780 [50:26<00:11,  3.85s/it]
100%|█████████▉| 778/780 [50:29<00:07,  3.66s/it]
                                                 
{'loss': 1.0961, 'grad_norm': 0.5422820448875427, 'learning_rate': 3.4536923623096353e-10, 'epoch': 9.96}

100%|█████████▉| 778/780 [50:29<00:07,  3.66s/it]
100%|█████████▉| 779/780 [50:33<00:03,  3.61s/it]
                                                 
{'loss': 1.206, 'grad_norm': 0.5440871119499207, 'learning_rate': 8.634268181095806e-11, 'epoch': 9.97}

100%|█████████▉| 779/780 [50:33<00:03,  3.61s/it]
100%|██████████| 780/780 [50:37<00:00,  3.70s/it]
                                                 
{'loss': 1.1163, 'grad_norm': 0.5198076963424683, 'learning_rate': 0.0, 'epoch': 9.98}

100%|██████████| 780/780 [50:37<00:00,  3.70s/it]
                                                 
{'train_runtime': 3039.0732, 'train_samples_per_second': 32.905, 'train_steps_per_second': 0.257, 'train_loss': 1.2200446108976999, 'epoch': 9.98}

100%|██████████| 780/780 [50:37<00:00,  3.70s/it]
100%|██████████| 780/780 [50:37<00:00,  3.89s/it]
Saving
Directory './test_multi_gpu_v2/' already exists.
Loading
Finished on rank 3
Finished on rank 2
Finished on rank 1
WARNING:root:The key is provided in the config. Assuming this is loaded from a pretrained module.
/nlp/scr/ram1998/miniconda3/envs/pyreft_dev/lib/python3.12/site-packages/pyvene/models/intervenable_base.py:1308: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  saved_state_dict = torch.load(os.path.join(load_directory, binary_filename))
WARNING:root:The key is provided in the config. Assuming this is loaded from a pretrained module.
Complete
Finished on rank 0
wandb: 🚀 View run multigpu_reft_alpaca_example at: https://wandb.ai/ramvenkat98/huggingface/runs/6fcnccrm
wandb: Find logs at: ../../../../../../juice2/scr2/ram1998/pyreft/examples/alpaca/wandb/run-20241118_080348-6fcnccrm/logs
###############################
end time: 2024-11-18 08:55:02.402800
elapsed time: 0:53:32.631181