2023/06/07 17:50:43 177474 |  TRACE   | Building model '20230607T175043-24ts_test_sugarcane_Belize'..
2023/06/07 17:50:43 440394 |  TRACE   | Start the training:
2023/06/07 17:50:43 442616 |  TRACE   |  - Maximum steps: 1000000000
2023/06/07 17:50:43 444144 |  TRACE   |  - Validation steps: 500
2023/06/07 17:50:43 445663 |  TRACE   |  - Batch-size: 512
2023/06/07 17:50:43 447088 |  TRACE   |  - Loss function: CustomLoss(type=cross_entropy_focal)
2023/06/07 17:50:43 448586 |  TRACE   |  - Early-stopping tolerance: 5
2023/06/07 17:50:43 449996 |  TRACE   |  - Early-stopping improvement: 0.0001
2023/06/07 17:50:43 451442 |  TRACE   |  - Early-stopping metric: f1_class
2023/06/07 17:50:43 452841 |  TRACE   |  - Optimizer: Adam
2023/06/07 17:50:43 454268 |  TRACE   |  - Learning rate scheduling: LearningRateScheduler(scheduler=reduce_on_plateau, mode=max, lr=0.001)
2023/06/07 17:50:43 455945 |  TRACE   |  - Training balanced by: target_id
2023/06/07 17:50:43 457513 |  TRACE   |  - Device: cuda
2023/06/07 17:50:43 459999 |  TRACE   |  - N params: 1033434
2023/06/07 17:50:43 461791 |  TRACE   | 

2023/06/07 17:50:47 414649 |  TRACE   | Learning rate scheduler: reduce_on_plateau
2023/06/07 17:50:47 422264 |  TRACE   |  - Current learning rate: 0.001
2023/06/07 17:50:47 529806 |  TRACE   |  - Patience: 0/1
2023/06/07 17:50:47 532220 |  TRACE   | Initial validation score for 'f1_class': 0.32597
2023/06/07 17:50:47 534793 |  TRACE   | 

2023/06/07 17:50:47 537039 |  TRACE   | Training for 500 steps (current step: 0/1000000000)
2023/06/07 17:51:15 279289 |  TRACE   | Learning rate scheduler: reduce_on_plateau
2023/06/07 17:51:15 286888 |  TRACE   |  - Current learning rate: 0.001
2023/06/07 17:51:15 288627 |  TRACE   |  - Patience: 0/1
2023/06/07 17:51:15 290223 |  TRACE   | Early stopping:
2023/06/07 17:51:15 291742 |  TRACE   |  - New best f1_class: 0.96713
2023/06/07 17:51:15 293333 |  TRACE   |  - Improvement over previous best: 0.64117
2023/06/07 17:51:15 294837 |  TRACE   |  - Best model updated!
2023/06/07 17:51:15 371375 |  TRACE   | 

2023/06/07 17:51:15 373138 |  TRACE   | Training for 500 steps (current step: 500/1000000000)
2023/06/07 17:51:43 567542 |  TRACE   | Learning rate scheduler: reduce_on_plateau
2023/06/07 17:51:43 575169 |  TRACE   |  - Current learning rate: 0.001
2023/06/07 17:51:43 577315 |  TRACE   |  - Patience: 1/1
2023/06/07 17:51:43 579519 |  TRACE   | Early stopping:
2023/06/07 17:51:43 582164 |  TRACE   |  - No improvement for f1_class: 0.94259 (current best: 0.96713)
2023/06/07 17:51:43 584822 |  TRACE   |  - Early stopping patience: 1/5
2023/06/07 17:51:43 587154 |  TRACE   | 

2023/06/07 17:51:43 589259 |  TRACE   | Training for 500 steps (current step: 1000/1000000000)
2023/06/07 17:52:11 444400 |  TRACE   | Learning rate scheduler: reduce_on_plateau
2023/06/07 17:52:11 451711 |  TRACE   |  - Current learning rate: 0.0001
2023/06/07 17:52:11 453524 |  TRACE   |  - Patience: 0/1
2023/06/07 17:52:11 455024 |  TRACE   | Early stopping:
2023/06/07 17:52:11 456334 |  TRACE   |  - No improvement for f1_class: 0.95077 (current best: 0.96713)
2023/06/07 17:52:11 458111 |  TRACE   |  - Early stopping patience: 2/5
2023/06/07 17:52:11 459572 |  TRACE   | 

2023/06/07 17:52:11 460925 |  TRACE   | Training for 500 steps (current step: 1500/1000000000)
2023/06/07 17:52:38 964624 |  TRACE   | Learning rate scheduler: reduce_on_plateau
2023/06/07 17:52:38 972113 |  TRACE   |  - Current learning rate: 0.0001
2023/06/07 17:52:38 974083 |  TRACE   |  - Patience: 1/1
2023/06/07 17:52:38 975815 |  TRACE   | Early stopping:
2023/06/07 17:52:38 977417 |  TRACE   |  - No improvement for f1_class: 0.93427 (current best: 0.96713)
2023/06/07 17:52:38 979111 |  TRACE   |  - Early stopping patience: 3/5
2023/06/07 17:52:38 980691 |  TRACE   | 

2023/06/07 17:52:38 982100 |  TRACE   | Training for 500 steps (current step: 2000/1000000000)
2023/06/07 17:53:05 310152 |  TRACE   | Learning rate scheduler: reduce_on_plateau
2023/06/07 17:53:05 316668 |  TRACE   |  - Current learning rate: 1e-05
2023/06/07 17:53:05 318388 |  TRACE   |  - Patience: 0/1
2023/06/07 17:53:05 320030 |  TRACE   | Early stopping:
2023/06/07 17:53:05 321834 |  TRACE   |  - No improvement for f1_class: 0.95070 (current best: 0.96713)
2023/06/07 17:53:05 323787 |  TRACE   |  - Early stopping patience: 4/5
2023/06/07 17:53:05 325300 |  TRACE   | 

2023/06/07 17:53:05 326922 |  TRACE   | Training for 500 steps (current step: 2500/1000000000)
2023/06/07 17:53:32 824839 |  TRACE   | Learning rate scheduler: reduce_on_plateau
2023/06/07 17:53:32 832116 |  TRACE   |  - Current learning rate: 1e-05
2023/06/07 17:53:32 837931 |  TRACE   |  - Patience: 1/1
2023/06/07 17:53:32 839782 |  TRACE   | Early stopping:
2023/06/07 17:53:32 842903 |  TRACE   |  - No improvement for f1_class: 0.95070 (current best: 0.96713)
2023/06/07 17:53:32 844185 |  TRACE   |  - Early stopping patience: 5/5
2023/06/07 17:53:32 845619 |  TRACE   |  - Early Stopping tolerance reached. Training stopped!
2023/06/07 17:53:32 848222 |  TRACE   | 

