2023/06/16 15:32:24 346739 |  TRACE   |  Model configs:
2023/06/16 15:32:24 347080 |  TRACE   |  - Model: 20230616T153220-EnanchedResUnet3D_ts24_noAugm
2023/06/16 15:32:24 347203 |  TRACE   |  - Parameters: 1_731_809
2023/06/16 15:32:24 347312 |  TRACE   |  - Device: cuda
2023/06/16 15:32:24 347421 |  TRACE   |  - Input bands: ['s2_ndvi', 's2_b02', 's2_b03', 's2_b04']
2023/06/16 15:32:24 347524 |  TRACE   |  - N ts: 24
 
2023/06/16 15:32:24 347625 |  TRACE   |  Steps configs:
2023/06/16 15:32:24 347726 |  TRACE   |  - Maximum steps: None
2023/06/16 15:32:24 347827 |  TRACE   |  - Validation steps: 250
2023/06/16 15:32:24 347927 |  TRACE   |  - Batch-size: 16
 
2023/06/16 15:32:24 348026 |  TRACE   |  Optimizer configs:
2023/06/16 15:32:24 348129 |  TRACE   |  - Optimizer: Adam
2023/06/16 15:32:24 348231 |  TRACE   |  - Early-stopping tolerance: 10
2023/06/16 15:32:24 348337 |  TRACE   |  - Early-stopping improvement: 0.0001
2023/06/16 15:32:24 348440 |  TRACE   |  - Early-stopping metric: loss
2023/06/16 15:32:24 348541 |  TRACE   |  - Scheduler: ReduceLROnPlateau
2023/06/16 15:32:24 348641 |  TRACE   |  - Learning rate tolerance: 3
 
2023/06/16 15:32:24 348740 |  TRACE   |  Loss configs:
2023/06/16 15:32:24 348842 |  TRACE   |  - Tag: DiceLoss_ws
2023/06/16 15:32:24 348944 |  TRACE   |  - Loss: dice_loss
2023/06/16 15:32:24 349044 |  TRACE   |  - Weight / multiplier: {'watersheds': 10}
2023/06/16 15:32:24 349144 |  TRACE   |  - Target mask: extent
2023/06/16 15:32:24 349247 |  TRACE   |  - LC coefficient: 1
2023/06/16 15:32:24 349342 |  TRACE   | 

2023/06/16 15:34:16 572652 |  TRACE   | Writing results to TensorBoard:
2023/06/16 15:34:16 578110 |  TRACE   |  - Start loss computation
2023/06/16 15:34:16 578456 |  TRACE   |  - Start image generation for 100 images
2023/06/16 15:34:21 021026 |  TRACE   | 

2023/06/16 15:34:21 021335 |  TRACE   | Training for 250 steps (current step: 0)
2023/06/16 15:46:26 614164 |  TRACE   | Validating
2023/06/16 15:48:32 186876 |  TRACE   | Writing results to TensorBoard:
2023/06/16 15:48:32 189084 |  TRACE   |  - Start loss computation
2023/06/16 15:48:32 189648 |  TRACE   |  - Start image generation for 100 images
2023/06/16 15:49:52 697113 |  TRACE   |  - New best loss: 0.36551
2023/06/16 15:49:52 698486 |  TRACE   |  - Improvement over previous best: -0.35411
2023/06/16 15:49:52 770785 |  TRACE   |  - Best model updated!
 
2023/06/16 15:49:52 771056 |  TRACE   | Training for 250 steps (current step: 250)
2023/06/16 16:01:06 041613 |  TRACE   | Validating
2023/06/16 16:03:12 690050 |  TRACE   | Writing results to TensorBoard:
2023/06/16 16:03:12 690795 |  TRACE   |  - Start loss computation
2023/06/16 16:03:12 691059 |  TRACE   |  - Start image generation for 100 images
2023/06/16 16:04:24 971841 |  TRACE   |  - New best loss: 0.31056
2023/06/16 16:04:24 972260 |  TRACE   |  - Improvement over previous best: -0.05495
2023/06/16 16:04:25 050632 |  TRACE   |  - Best model updated!
 
2023/06/16 16:04:25 050881 |  TRACE   | Training for 250 steps (current step: 500)
2023/06/16 16:15:46 759342 |  TRACE   | Validating
2023/06/16 16:17:33 048011 |  TRACE   | Writing results to TensorBoard:
2023/06/16 16:17:33 048912 |  TRACE   |  - Start loss computation
2023/06/16 16:17:33 049183 |  TRACE   |  - Start image generation for 100 images
2023/06/16 16:18:37 382472 |  TRACE   |  - New best loss: 0.25913
2023/06/16 16:18:37 382854 |  TRACE   |  - Improvement over previous best: -0.05142
2023/06/16 16:18:37 421755 |  TRACE   |  - Best model updated!
 
2023/06/16 16:18:37 421999 |  TRACE   | Training for 250 steps (current step: 750)
2023/06/16 16:29:11 199911 |  TRACE   | Validating
2023/06/16 16:31:04 543067 |  TRACE   | Writing results to TensorBoard:
2023/06/16 16:31:04 544950 |  TRACE   |  - Start loss computation
2023/06/16 16:31:04 545600 |  TRACE   |  - Start image generation for 100 images
2023/06/16 16:32:06 943202 |  TRACE   |  - New best loss: 0.24659
2023/06/16 16:32:06 944417 |  TRACE   |  - Improvement over previous best: -0.01254
2023/06/16 16:32:06 988072 |  TRACE   |  - Best model updated!
 
2023/06/16 16:32:06 988335 |  TRACE   | Training for 250 steps (current step: 1000)
2023/06/16 16:42:50 060415 |  TRACE   | Validating
2023/06/16 16:44:42 187566 |  TRACE   | Writing results to TensorBoard:
2023/06/16 16:44:42 188248 |  TRACE   |  - Start loss computation
2023/06/16 16:44:42 188572 |  TRACE   |  - Start image generation for 100 images
2023/06/16 16:45:41 630980 |  TRACE   |  - No improvement for loss: 0.29584 (current best: 0.24659)
2023/06/16 16:45:41 632029 |  TRACE   | Training for 250 steps (current step: 1250)
2023/06/16 16:56:08 956782 |  TRACE   | Validating
2023/06/16 16:57:59 271249 |  TRACE   | Writing results to TensorBoard:
2023/06/16 16:57:59 273413 |  TRACE   |  - Start loss computation
2023/06/16 16:57:59 273738 |  TRACE   |  - Start image generation for 100 images
2023/06/16 16:59:00 284914 |  TRACE   |  - New best loss: 0.23603
2023/06/16 16:59:00 286158 |  TRACE   |  - Improvement over previous best: -0.01056
2023/06/16 16:59:00 355692 |  TRACE   |  - Best model updated!
 
2023/06/16 16:59:00 355958 |  TRACE   | Training for 250 steps (current step: 1500)
2023/06/16 17:09:33 098247 |  TRACE   | Validating
2023/06/16 17:11:25 170913 |  TRACE   | Writing results to TensorBoard:
2023/06/16 17:11:25 172028 |  TRACE   |  - Start loss computation
2023/06/16 17:11:25 172270 |  TRACE   |  - Start image generation for 100 images
2023/06/16 17:12:22 284073 |  TRACE   |  - No improvement for loss: 0.24949 (current best: 0.23603)
2023/06/16 17:12:22 284742 |  TRACE   | Training for 250 steps (current step: 1750)
2023/06/16 17:22:22 183099 |  TRACE   | Validating
2023/06/16 17:23:46 271250 |  TRACE   | Writing results to TensorBoard:
2023/06/16 17:23:46 272184 |  TRACE   |  - Start loss computation
2023/06/16 17:23:46 272433 |  TRACE   |  - Start image generation for 100 images
2023/06/16 17:24:49 137630 |  TRACE   |  - No improvement for loss: 0.26142 (current best: 0.23603)
2023/06/16 17:24:49 138033 |  TRACE   | Training for 250 steps (current step: 2000)
2023/06/16 17:35:32 005953 |  TRACE   | Validating
2023/06/16 17:36:49 235079 |  TRACE   | Writing results to TensorBoard:
2023/06/16 17:36:49 236085 |  TRACE   |  - Start loss computation
2023/06/16 17:36:49 236385 |  TRACE   |  - Start image generation for 100 images
2023/06/16 17:37:50 509980 |  TRACE   |  - New best loss: 0.23509
2023/06/16 17:37:50 511204 |  TRACE   |  - Improvement over previous best: -0.00094
2023/06/16 17:37:50 580095 |  TRACE   |  - Best model updated!
 
2023/06/16 17:37:50 580372 |  TRACE   | Training for 250 steps (current step: 2250)
2023/06/16 17:48:02 688394 |  TRACE   | Validating
2023/06/16 17:49:53 425946 |  TRACE   | Writing results to TensorBoard:
2023/06/16 17:49:53 427078 |  TRACE   |  - Start loss computation
2023/06/16 17:49:53 427350 |  TRACE   |  - Start image generation for 100 images
2023/06/16 17:50:50 741144 |  TRACE   |  - New best loss: 0.23173
2023/06/16 17:50:50 743438 |  TRACE   |  - Improvement over previous best: -0.00336
2023/06/16 17:50:50 811982 |  TRACE   |  - Best model updated!
 
2023/06/16 17:50:50 812230 |  TRACE   | Training for 250 steps (current step: 2500)
2023/06/16 18:03:49 945154 |  TRACE   | Validating
2023/06/16 18:06:34 617172 |  TRACE   | Writing results to TensorBoard:
2023/06/16 18:06:34 618276 |  TRACE   |  - Start loss computation
2023/06/16 18:06:34 618566 |  TRACE   |  - Start image generation for 100 images
2023/06/16 18:07:33 172464 |  TRACE   |  - New best loss: 0.21944
2023/06/16 18:07:33 173303 |  TRACE   |  - Improvement over previous best: -0.01230
2023/06/16 18:07:33 213716 |  TRACE   |  - Best model updated!
 
2023/06/16 18:07:33 213946 |  TRACE   | Training for 250 steps (current step: 2750)
2023/06/16 18:21:38 703274 |  TRACE   | Validating
2023/06/16 18:24:19 181209 |  TRACE   | Writing results to TensorBoard:
2023/06/16 18:24:19 183890 |  TRACE   |  - Start loss computation
2023/06/16 18:24:19 184141 |  TRACE   |  - Start image generation for 100 images
2023/06/16 18:25:13 721057 |  TRACE   |  - No improvement for loss: 0.22187 (current best: 0.21944)
2023/06/16 18:25:13 722221 |  TRACE   | Training for 250 steps (current step: 3000)
2023/06/16 18:39:19 938279 |  TRACE   | Validating
2023/06/16 18:42:01 554019 |  TRACE   | Writing results to TensorBoard:
2023/06/16 18:42:01 554900 |  TRACE   |  - Start loss computation
2023/06/16 18:42:01 555180 |  TRACE   |  - Start image generation for 100 images
2023/06/16 18:43:03 217447 |  TRACE   |  - No improvement for loss: 0.22790 (current best: 0.21944)
2023/06/16 18:43:03 217798 |  TRACE   | Training for 250 steps (current step: 3250)
2023/06/16 18:57:08 441153 |  TRACE   | Validating
2023/06/16 18:59:53 003054 |  TRACE   | Writing results to TensorBoard:
2023/06/16 18:59:53 007003 |  TRACE   |  - Start loss computation
2023/06/16 18:59:53 007276 |  TRACE   |  - Start image generation for 100 images
2023/06/16 19:00:53 001495 |  TRACE   |  - No improvement for loss: 0.22221 (current best: 0.21944)
2023/06/16 19:00:53 002509 |  TRACE   | Training for 250 steps (current step: 3500)
2023/06/16 19:15:05 012275 |  TRACE   | Validating
2023/06/16 19:17:43 670509 |  TRACE   | Writing results to TensorBoard:
2023/06/16 19:17:43 671388 |  TRACE   |  - Start loss computation
2023/06/16 19:17:43 671642 |  TRACE   |  - Start image generation for 100 images
2023/06/16 19:18:41 437167 |  TRACE   |  - No improvement for loss: 0.22226 (current best: 0.21944)
2023/06/16 19:18:41 438392 |  TRACE   | Training for 250 steps (current step: 3750)
2023/06/16 19:32:41 061674 |  TRACE   | Validating
2023/06/16 19:35:21 865655 |  TRACE   | Writing results to TensorBoard:
2023/06/16 19:35:21 866521 |  TRACE   |  - Start loss computation
2023/06/16 19:35:21 866794 |  TRACE   |  - Start image generation for 100 images
2023/06/16 19:36:19 656930 |  TRACE   |  - New best loss: 0.21017
2023/06/16 19:36:19 657314 |  TRACE   |  - Improvement over previous best: -0.00927
2023/06/16 19:36:19 728426 |  TRACE   |  - Best model updated!
 
2023/06/16 19:36:19 728683 |  TRACE   | Training for 250 steps (current step: 4000)
2023/06/16 19:50:20 966462 |  TRACE   | Validating
2023/06/16 19:53:06 384846 |  TRACE   | Writing results to TensorBoard:
2023/06/16 19:53:06 386060 |  TRACE   |  - Start loss computation
2023/06/16 19:53:06 386411 |  TRACE   |  - Start image generation for 100 images
2023/06/16 19:54:03 854371 |  TRACE   |  - New best loss: 0.20993
2023/06/16 19:54:03 856655 |  TRACE   |  - Improvement over previous best: -0.00024
2023/06/16 19:54:03 937094 |  TRACE   |  - Best model updated!
 
2023/06/16 19:54:03 937344 |  TRACE   | Training for 250 steps (current step: 4250)
2023/06/16 20:07:57 122991 |  TRACE   | Validating
2023/06/16 20:10:42 236062 |  TRACE   | Writing results to TensorBoard:
2023/06/16 20:10:42 237563 |  TRACE   |  - Start loss computation
2023/06/16 20:10:42 237822 |  TRACE   |  - Start image generation for 100 images
2023/06/16 20:11:39 872744 |  TRACE   |  - New best loss: 0.20907
2023/06/16 20:11:39 873954 |  TRACE   |  - Improvement over previous best: -0.00086
2023/06/16 20:11:39 942578 |  TRACE   |  - Best model updated!
 
2023/06/16 20:11:39 942835 |  TRACE   | Training for 250 steps (current step: 4500)
2023/06/16 20:25:53 566595 |  TRACE   | Validating
2023/06/16 20:28:45 279191 |  TRACE   | Writing results to TensorBoard:
2023/06/16 20:28:45 280990 |  TRACE   |  - Start loss computation
2023/06/16 20:28:45 281424 |  TRACE   |  - Start image generation for 100 images
2023/06/16 20:29:42 674524 |  TRACE   |  - No improvement for loss: 0.21209 (current best: 0.20907)
2023/06/16 20:29:42 675152 |  TRACE   | Training for 250 steps (current step: 4750)
2023/06/16 20:43:36 167901 |  TRACE   | Validating
2023/06/16 20:46:26 141828 |  TRACE   | Writing results to TensorBoard:
2023/06/16 20:46:26 150522 |  TRACE   |  - Start loss computation
2023/06/16 20:46:26 150794 |  TRACE   |  - Start image generation for 100 images
2023/06/16 20:47:23 760122 |  TRACE   |  - No improvement for loss: 0.21109 (current best: 0.20907)
2023/06/16 20:47:23 760457 |  TRACE   | Training for 250 steps (current step: 5000)
2023/06/16 21:01:39 425705 |  TRACE   | Validating
2023/06/16 21:04:21 704321 |  TRACE   | Writing results to TensorBoard:
2023/06/16 21:04:21 706428 |  TRACE   |  - Start loss computation
2023/06/16 21:04:21 706920 |  TRACE   |  - Start image generation for 100 images
2023/06/16 21:05:19 119092 |  TRACE   |  - New best loss: 0.20846
2023/06/16 21:05:19 122687 |  TRACE   |  - Improvement over previous best: -0.00062
2023/06/16 21:05:19 354605 |  TRACE   |  - Best model updated!
 
2023/06/16 21:05:19 354858 |  TRACE   | Training for 250 steps (current step: 5250)
2023/06/16 21:19:12 892628 |  TRACE   | Validating
2023/06/16 21:22:03 534127 |  TRACE   | Writing results to TensorBoard:
2023/06/16 21:22:03 541249 |  TRACE   |  - Start loss computation
2023/06/16 21:22:03 541481 |  TRACE   |  - Start image generation for 100 images
2023/06/16 21:22:59 332534 |  TRACE   |  - New best loss: 0.20816
2023/06/16 21:22:59 334304 |  TRACE   |  - Improvement over previous best: -0.00030
2023/06/16 21:22:59 466839 |  TRACE   |  - Best model updated!
 
2023/06/16 21:22:59 467080 |  TRACE   | Training for 250 steps (current step: 5500)
2023/06/16 21:37:03 291080 |  TRACE   | Validating
2023/06/16 21:40:00 522099 |  TRACE   | Writing results to TensorBoard:
2023/06/16 21:40:00 527234 |  TRACE   |  - Start loss computation
2023/06/16 21:40:00 527475 |  TRACE   |  - Start image generation for 100 images
2023/06/16 21:40:57 436909 |  TRACE   |  - New best loss: 0.20757
2023/06/16 21:40:57 437333 |  TRACE   |  - Improvement over previous best: -0.00059
2023/06/16 21:40:57 506742 |  TRACE   |  - Best model updated!
 
2023/06/16 21:40:57 506972 |  TRACE   | Training for 250 steps (current step: 5750)
2023/06/16 21:54:58 984039 |  TRACE   | Validating
2023/06/16 21:56:32 409547 |  TRACE   | Writing results to TensorBoard:
2023/06/16 21:56:32 411152 |  TRACE   |  - Start loss computation
2023/06/16 21:56:32 411442 |  TRACE   |  - Start image generation for 100 images
2023/06/16 21:57:28 894808 |  TRACE   |  - No improvement for loss: 0.21035 (current best: 0.20757)
2023/06/16 21:57:28 895152 |  TRACE   | Training for 250 steps (current step: 6000)
2023/06/16 22:11:28 841976 |  TRACE   | Validating
2023/06/16 22:13:40 868771 |  TRACE   | Writing results to TensorBoard:
2023/06/16 22:13:40 869661 |  TRACE   |  - Start loss computation
2023/06/16 22:13:40 869995 |  TRACE   |  - Start image generation for 100 images
2023/06/16 22:14:39 213037 |  TRACE   |  - No improvement for loss: 0.20855 (current best: 0.20757)
2023/06/16 22:14:39 213378 |  TRACE   | Training for 250 steps (current step: 6250)
2023/06/16 22:29:09 441261 |  TRACE   | Validating
2023/06/16 22:31:05 079684 |  TRACE   | Writing results to TensorBoard:
2023/06/16 22:31:05 080871 |  TRACE   |  - Start loss computation
2023/06/16 22:31:05 081119 |  TRACE   |  - Start image generation for 100 images
2023/06/16 22:32:03 737061 |  TRACE   |  - New best loss: 0.20670
2023/06/16 22:32:03 738214 |  TRACE   |  - Improvement over previous best: -0.00087
2023/06/16 22:32:03 780083 |  TRACE   |  - Best model updated!
 
2023/06/16 22:32:03 780375 |  TRACE   | Training for 250 steps (current step: 6500)
2023/06/16 22:46:49 773120 |  TRACE   | Validating
2023/06/16 22:48:47 020509 |  TRACE   | Writing results to TensorBoard:
2023/06/16 22:48:47 021316 |  TRACE   |  - Start loss computation
2023/06/16 22:48:47 021640 |  TRACE   |  - Start image generation for 100 images
2023/06/16 22:49:44 858719 |  TRACE   |  - No improvement for loss: 0.20679 (current best: 0.20670)
2023/06/16 22:49:44 859649 |  TRACE   | Training for 250 steps (current step: 6750)
2023/06/16 23:04:08 320008 |  TRACE   | Validating
2023/06/16 23:06:36 395229 |  TRACE   | Writing results to TensorBoard:
2023/06/16 23:06:36 399416 |  TRACE   |  - Start loss computation
2023/06/16 23:06:36 399670 |  TRACE   |  - Start image generation for 100 images
2023/06/16 23:07:34 118899 |  TRACE   |  - No improvement for loss: 0.20871 (current best: 0.20670)
2023/06/16 23:07:34 121010 |  TRACE   | Training for 250 steps (current step: 7000)
2023/06/16 23:21:39 610114 |  TRACE   | Validating
2023/06/16 23:24:21 641446 |  TRACE   | Writing results to TensorBoard:
2023/06/16 23:24:21 642439 |  TRACE   |  - Start loss computation
2023/06/16 23:24:21 642699 |  TRACE   |  - Start image generation for 100 images
2023/06/16 23:25:18 907430 |  TRACE   |  - New best loss: 0.20571
2023/06/16 23:25:18 908629 |  TRACE   |  - Improvement over previous best: -0.00099
2023/06/16 23:25:18 989693 |  TRACE   |  - Best model updated!
 
2023/06/16 23:25:18 989960 |  TRACE   | Training for 250 steps (current step: 7250)
2023/06/16 23:39:27 791921 |  TRACE   | Validating
2023/06/16 23:42:11 719017 |  TRACE   | Writing results to TensorBoard:
2023/06/16 23:42:11 723849 |  TRACE   |  - Start loss computation
2023/06/16 23:42:11 724120 |  TRACE   |  - Start image generation for 100 images
2023/06/16 23:43:08 317923 |  TRACE   |  - New best loss: 0.20516
2023/06/16 23:43:08 318318 |  TRACE   |  - Improvement over previous best: -0.00055
2023/06/16 23:43:08 482688 |  TRACE   |  - Best model updated!
 
2023/06/16 23:43:08 482992 |  TRACE   | Training for 250 steps (current step: 7500)
2023/06/16 23:57:20 677443 |  TRACE   | Validating
2023/06/16 23:59:55 685068 |  TRACE   | Writing results to TensorBoard:
2023/06/16 23:59:55 686635 |  TRACE   |  - Start loss computation
2023/06/16 23:59:55 686939 |  TRACE   |  - Start image generation for 100 images
2023/06/17 00:00:53 714730 |  TRACE   |  - No improvement for loss: 0.20517 (current best: 0.20516)
2023/06/17 00:00:53 715645 |  TRACE   | Training for 250 steps (current step: 7750)
2023/06/17 00:14:57 118668 |  TRACE   | Validating
2023/06/17 00:17:41 644992 |  TRACE   | Writing results to TensorBoard:
2023/06/17 00:17:41 651380 |  TRACE   |  - Start loss computation
2023/06/17 00:17:41 651641 |  TRACE   |  - Start image generation for 100 images
2023/06/17 00:18:38 589159 |  TRACE   |  - No improvement for loss: 0.20517 (current best: 0.20516)
2023/06/17 00:18:38 590342 |  TRACE   | Training for 250 steps (current step: 8000)
2023/06/17 00:32:51 215817 |  TRACE   | Validating
2023/06/17 00:35:36 047303 |  TRACE   | Writing results to TensorBoard:
2023/06/17 00:35:36 048804 |  TRACE   |  - Start loss computation
2023/06/17 00:35:36 049062 |  TRACE   |  - Start image generation for 100 images
2023/06/17 00:36:34 252659 |  TRACE   |  - New best loss: 0.20487
2023/06/17 00:36:34 253700 |  TRACE   |  - Improvement over previous best: -0.00030
2023/06/17 00:36:34 325668 |  TRACE   |  - Best model updated!
 
2023/06/17 00:36:34 325900 |  TRACE   | Training for 250 steps (current step: 8250)
2023/06/17 00:50:34 259300 |  TRACE   | Validating
2023/06/17 00:53:14 191709 |  TRACE   | Writing results to TensorBoard:
2023/06/17 00:53:14 193645 |  TRACE   |  - Start loss computation
2023/06/17 00:53:14 193901 |  TRACE   |  - Start image generation for 100 images
2023/06/17 00:54:11 304575 |  TRACE   |  - No improvement for loss: 0.20559 (current best: 0.20487)
2023/06/17 00:54:11 304970 |  TRACE   | Training for 250 steps (current step: 8500)
2023/06/17 01:08:16 153463 |  TRACE   | Validating
2023/06/17 01:11:01 013152 |  TRACE   | Writing results to TensorBoard:
2023/06/17 01:11:01 013981 |  TRACE   |  - Start loss computation
2023/06/17 01:11:01 014292 |  TRACE   |  - Start image generation for 100 images
2023/06/17 01:11:57 544815 |  TRACE   |  - New best loss: 0.20387
2023/06/17 01:11:57 545278 |  TRACE   |  - Improvement over previous best: -0.00100
2023/06/17 01:11:57 618647 |  TRACE   |  - Best model updated!
 
2023/06/17 01:11:57 618894 |  TRACE   | Training for 250 steps (current step: 8750)
2023/06/17 01:23:25 524952 |  TRACE   | Validating
2023/06/17 01:25:07 159720 |  TRACE   | Writing results to TensorBoard:
2023/06/17 01:25:07 161448 |  TRACE   |  - Start loss computation
2023/06/17 01:25:07 161692 |  TRACE   |  - Start image generation for 100 images
2023/06/17 01:26:02 341309 |  TRACE   |  - No improvement for loss: 0.20415 (current best: 0.20387)
2023/06/17 01:26:02 341690 |  TRACE   | Training for 250 steps (current step: 9000)
2023/06/17 01:33:36 623307 |  TRACE   | Validating
2023/06/17 01:34:44 144879 |  TRACE   | Writing results to TensorBoard:
2023/06/17 01:34:44 145208 |  TRACE   |  - Start loss computation
2023/06/17 01:34:44 145443 |  TRACE   |  - Start image generation for 100 images
2023/06/17 01:35:39 403780 |  TRACE   |  - No improvement for loss: 0.20531 (current best: 0.20387)
2023/06/17 01:35:39 404112 |  TRACE   | Training for 250 steps (current step: 9250)
2023/06/17 01:43:08 491251 |  TRACE   | Validating
2023/06/17 01:44:16 682567 |  TRACE   | Writing results to TensorBoard:
2023/06/17 01:44:16 683076 |  TRACE   |  - Start loss computation
2023/06/17 01:44:16 683463 |  TRACE   |  - Start image generation for 100 images
2023/06/17 01:45:11 889486 |  TRACE   |  - No improvement for loss: 0.20527 (current best: 0.20387)
2023/06/17 01:45:11 889886 |  TRACE   | Training for 250 steps (current step: 9500)
2023/06/17 01:52:51 511931 |  TRACE   | Validating
2023/06/17 01:53:59 786351 |  TRACE   | Writing results to TensorBoard:
2023/06/17 01:53:59 786661 |  TRACE   |  - Start loss computation
2023/06/17 01:53:59 786889 |  TRACE   |  - Start image generation for 100 images
2023/06/17 01:54:54 690389 |  TRACE   |  - New best loss: 0.20296
2023/06/17 01:54:54 690792 |  TRACE   |  - Improvement over previous best: -0.00091
2023/06/17 01:54:54 758142 |  TRACE   |  - Best model updated!
 
2023/06/17 01:54:54 758369 |  TRACE   | Training for 250 steps (current step: 9750)
2023/06/17 02:02:25 945601 |  TRACE   | Validating
2023/06/17 02:03:36 315990 |  TRACE   | Writing results to TensorBoard:
2023/06/17 02:03:36 316303 |  TRACE   |  - Start loss computation
2023/06/17 02:03:36 316539 |  TRACE   |  - Start image generation for 100 images
2023/06/17 02:04:30 059103 |  TRACE   |  - No improvement for loss: 0.20432 (current best: 0.20296)
2023/06/17 02:04:30 059460 |  TRACE   | Training for 250 steps (current step: 10000)
2023/06/17 02:12:12 195784 |  TRACE   | Validating
2023/06/17 02:13:20 660363 |  TRACE   | Writing results to TensorBoard:
2023/06/17 02:13:20 660716 |  TRACE   |  - Start loss computation
2023/06/17 02:13:20 660948 |  TRACE   |  - Start image generation for 100 images
2023/06/17 02:14:14 940097 |  TRACE   |  - No improvement for loss: 0.20287 (current best: 0.20296)
2023/06/17 02:14:14 940436 |  TRACE   | Training for 250 steps (current step: 10250)
2023/06/17 02:21:40 265060 |  TRACE   | Validating
2023/06/17 02:22:47 375809 |  TRACE   | Writing results to TensorBoard:
2023/06/17 02:22:47 376323 |  TRACE   |  - Start loss computation
2023/06/17 02:22:47 376692 |  TRACE   |  - Start image generation for 100 images
2023/06/17 02:23:42 868500 |  TRACE   |  - No improvement for loss: 0.20321 (current best: 0.20296)
2023/06/17 02:23:42 868867 |  TRACE   | Training for 250 steps (current step: 10500)
2023/06/17 02:31:23 171630 |  TRACE   | Validating
2023/06/17 02:32:30 419516 |  TRACE   | Writing results to TensorBoard:
2023/06/17 02:32:30 419849 |  TRACE   |  - Start loss computation
2023/06/17 02:32:30 420074 |  TRACE   |  - Start image generation for 100 images
2023/06/17 02:33:26 074632 |  TRACE   |  - New best loss: 0.20218
2023/06/17 02:33:26 075028 |  TRACE   |  - Improvement over previous best: -0.00078
2023/06/17 02:33:26 119228 |  TRACE   |  - Best model updated!
 
2023/06/17 02:33:26 119477 |  TRACE   | Training for 250 steps (current step: 10750)
2023/06/17 02:41:00 054702 |  TRACE   | Validating
2023/06/17 02:42:07 506079 |  TRACE   | Writing results to TensorBoard:
2023/06/17 02:42:07 506384 |  TRACE   |  - Start loss computation
2023/06/17 02:42:07 506613 |  TRACE   |  - Start image generation for 100 images
2023/06/17 02:43:01 229067 |  TRACE   |  - No improvement for loss: 0.20333 (current best: 0.20218)
2023/06/17 02:43:01 229433 |  TRACE   | Training for 250 steps (current step: 11000)
2023/06/17 02:50:42 458339 |  TRACE   | Validating
2023/06/17 02:51:50 515515 |  TRACE   | Writing results to TensorBoard:
2023/06/17 02:51:50 515847 |  TRACE   |  - Start loss computation
2023/06/17 02:51:50 516072 |  TRACE   |  - Start image generation for 100 images
2023/06/17 02:52:45 015428 |  TRACE   |  - No improvement for loss: 0.20211 (current best: 0.20218)
2023/06/17 02:52:45 015763 |  TRACE   | Training for 250 steps (current step: 11250)
2023/06/17 03:00:20 210674 |  TRACE   | Validating
2023/06/17 03:01:30 627521 |  TRACE   | Writing results to TensorBoard:
2023/06/17 03:01:30 627870 |  TRACE   |  - Start loss computation
2023/06/17 03:01:30 628111 |  TRACE   |  - Start image generation for 100 images
2023/06/17 03:02:26 198126 |  TRACE   |  - New best loss: 0.20194
2023/06/17 03:02:26 198539 |  TRACE   |  - Improvement over previous best: -0.00025
2023/06/17 03:02:26 241142 |  TRACE   |  - Best model updated!
 
2023/06/17 03:02:26 241346 |  TRACE   | Training for 250 steps (current step: 11500)
2023/06/17 03:10:08 743668 |  TRACE   | Validating
2023/06/17 03:11:18 389981 |  TRACE   | Writing results to TensorBoard:
2023/06/17 03:11:18 390387 |  TRACE   |  - Start loss computation
2023/06/17 03:11:18 390634 |  TRACE   |  - Start image generation for 100 images
2023/06/17 03:12:13 448816 |  TRACE   |  - No improvement for loss: 0.20382 (current best: 0.20194)
2023/06/17 03:12:13 449179 |  TRACE   | Training for 250 steps (current step: 11750)
2023/06/17 03:19:45 444469 |  TRACE   | Validating
2023/06/17 03:20:52 426193 |  TRACE   | Writing results to TensorBoard:
2023/06/17 03:20:52 426541 |  TRACE   |  - Start loss computation
2023/06/17 03:20:52 426781 |  TRACE   |  - Start image generation for 100 images
2023/06/17 03:21:47 474390 |  TRACE   |  - No improvement for loss: 0.20291 (current best: 0.20194)
2023/06/17 03:21:47 474749 |  TRACE   | Training for 250 steps (current step: 12000)
2023/06/17 03:29:22 090425 |  TRACE   | Validating
2023/06/17 03:30:28 627186 |  TRACE   | Writing results to TensorBoard:
2023/06/17 03:30:28 627509 |  TRACE   |  - Start loss computation
2023/06/17 03:30:28 627750 |  TRACE   |  - Start image generation for 100 images
2023/06/17 03:31:24 279817 |  TRACE   |  - No improvement for loss: 0.20244 (current best: 0.20194)
2023/06/17 03:31:24 280184 |  TRACE   | Training for 250 steps (current step: 12250)
2023/06/17 03:38:51 090200 |  TRACE   | Validating
2023/06/17 03:39:58 817905 |  TRACE   | Writing results to TensorBoard:
2023/06/17 03:39:58 818997 |  TRACE   |  - Start loss computation
2023/06/17 03:39:58 819258 |  TRACE   |  - Start image generation for 100 images
2023/06/17 03:40:53 152627 |  TRACE   |  - New best loss: 0.20173
2023/06/17 03:40:53 153040 |  TRACE   |  - Improvement over previous best: -0.00021
2023/06/17 03:40:53 194306 |  TRACE   |  - Best model updated!
 
2023/06/17 03:40:53 194552 |  TRACE   | Training for 250 steps (current step: 12500)
2023/06/17 03:48:36 051063 |  TRACE   | Validating
2023/06/17 03:49:44 518618 |  TRACE   | Writing results to TensorBoard:
2023/06/17 03:49:44 519609 |  TRACE   |  - Start loss computation
2023/06/17 03:49:44 519915 |  TRACE   |  - Start image generation for 100 images
2023/06/17 03:50:39 749820 |  TRACE   |  - New best loss: 0.20157
2023/06/17 03:50:39 750254 |  TRACE   |  - Improvement over previous best: -0.00017
2023/06/17 03:50:39 789851 |  TRACE   |  - Best model updated!
 
2023/06/17 03:50:39 790066 |  TRACE   | Training for 250 steps (current step: 12750)
2023/06/17 03:58:13 226617 |  TRACE   | Validating
2023/06/17 03:59:22 547425 |  TRACE   | Writing results to TensorBoard:
2023/06/17 03:59:22 548296 |  TRACE   |  - Start loss computation
2023/06/17 03:59:22 548537 |  TRACE   |  - Start image generation for 100 images
2023/06/17 04:00:17 483802 |  TRACE   |  - No improvement for loss: 0.20183 (current best: 0.20157)
2023/06/17 04:00:17 484177 |  TRACE   | Training for 250 steps (current step: 13000)
2023/06/17 04:07:53 178498 |  TRACE   | Validating
2023/06/17 04:09:01 039902 |  TRACE   | Writing results to TensorBoard:
2023/06/17 04:09:01 040977 |  TRACE   |  - Start loss computation
2023/06/17 04:09:01 041234 |  TRACE   |  - Start image generation for 100 images
2023/06/17 04:09:54 965222 |  TRACE   |  - No improvement for loss: 0.20613 (current best: 0.20157)
2023/06/17 04:09:54 965588 |  TRACE   | Training for 250 steps (current step: 13250)
2023/06/17 04:17:16 376317 |  TRACE   | Validating
2023/06/17 04:18:22 041703 |  TRACE   | Writing results to TensorBoard:
2023/06/17 04:18:22 042622 |  TRACE   |  - Start loss computation
2023/06/17 04:18:22 042886 |  TRACE   |  - Start image generation for 100 images
2023/06/17 04:19:18 625752 |  TRACE   |  - No improvement for loss: 0.20333 (current best: 0.20157)
2023/06/17 04:19:18 626179 |  TRACE   | Training for 250 steps (current step: 13500)
2023/06/17 04:27:00 582457 |  TRACE   | Validating
2023/06/17 04:28:07 824900 |  TRACE   | Writing results to TensorBoard:
2023/06/17 04:28:07 825813 |  TRACE   |  - Start loss computation
2023/06/17 04:28:07 826094 |  TRACE   |  - Start image generation for 100 images
2023/06/17 04:29:01 936178 |  TRACE   |  - No improvement for loss: 0.20869 (current best: 0.20157)
2023/06/17 04:29:01 936517 |  TRACE   | Training for 250 steps (current step: 13750)
2023/06/17 04:36:24 093650 |  TRACE   | Validating
2023/06/17 04:37:30 374938 |  TRACE   | Writing results to TensorBoard:
2023/06/17 04:37:30 376071 |  TRACE   |  - Start loss computation
2023/06/17 04:37:30 376332 |  TRACE   |  - Start image generation for 100 images
2023/06/17 04:38:25 433803 |  TRACE   |  - New best loss: 0.20023
2023/06/17 04:38:25 434209 |  TRACE   |  - Improvement over previous best: -0.00134
2023/06/17 04:38:25 482096 |  TRACE   |  - Best model updated!
 
2023/06/17 04:38:25 482359 |  TRACE   | Training for 250 steps (current step: 14000)
2023/06/17 04:45:53 692109 |  TRACE   | Validating
2023/06/17 04:47:00 240903 |  TRACE   | Writing results to TensorBoard:
2023/06/17 04:47:00 242134 |  TRACE   |  - Start loss computation
2023/06/17 04:47:00 242410 |  TRACE   |  - Start image generation for 100 images
2023/06/17 04:47:55 349200 |  TRACE   |  - No improvement for loss: 0.20017 (current best: 0.20023)
2023/06/17 04:47:55 349548 |  TRACE   | Training for 250 steps (current step: 14250)
2023/06/17 04:55:25 851467 |  TRACE   | Validating
2023/06/17 04:56:34 737552 |  TRACE   | Writing results to TensorBoard:
2023/06/17 04:56:34 737896 |  TRACE   |  - Start loss computation
2023/06/17 04:56:34 738113 |  TRACE   |  - Start image generation for 100 images
2023/06/17 04:57:29 478271 |  TRACE   |  - New best loss: 0.20006
2023/06/17 04:57:29 478709 |  TRACE   |  - Improvement over previous best: -0.00017
2023/06/17 04:57:29 518646 |  TRACE   |  - Best model updated!
 
2023/06/17 04:57:29 518862 |  TRACE   | Training for 250 steps (current step: 14500)
2023/06/17 05:05:05 220009 |  TRACE   | Validating
2023/06/17 05:06:14 152814 |  TRACE   | Writing results to TensorBoard:
2023/06/17 05:06:14 153735 |  TRACE   |  - Start loss computation
2023/06/17 05:06:14 153992 |  TRACE   |  - Start image generation for 100 images
2023/06/17 05:07:09 071564 |  TRACE   |  - No improvement for loss: 0.20015 (current best: 0.20006)
2023/06/17 05:07:09 071921 |  TRACE   | Training for 250 steps (current step: 14750)
2023/06/17 05:14:44 510428 |  TRACE   | Validating
2023/06/17 05:15:53 369450 |  TRACE   | Writing results to TensorBoard:
2023/06/17 05:15:53 370409 |  TRACE   |  - Start loss computation
2023/06/17 05:15:53 370833 |  TRACE   |  - Start image generation for 100 images
2023/06/17 05:16:48 579188 |  TRACE   |  - No improvement for loss: 0.19997 (current best: 0.20006)
2023/06/17 05:16:48 579521 |  TRACE   | Training for 250 steps (current step: 15000)
2023/06/17 05:24:28 380752 |  TRACE   | Validating
2023/06/17 05:25:34 960042 |  TRACE   | Writing results to TensorBoard:
2023/06/17 05:25:34 960952 |  TRACE   |  - Start loss computation
2023/06/17 05:25:34 961218 |  TRACE   |  - Start image generation for 100 images
2023/06/17 05:26:29 860232 |  TRACE   |  - No improvement for loss: 0.20011 (current best: 0.20006)
2023/06/17 05:26:29 860587 |  TRACE   | Training for 250 steps (current step: 15250)
2023/06/17 05:33:59 179126 |  TRACE   | Validating
2023/06/17 05:35:05 190830 |  TRACE   | Writing results to TensorBoard:
2023/06/17 05:35:05 191744 |  TRACE   |  - Start loss computation
2023/06/17 05:35:05 192010 |  TRACE   |  - Start image generation for 100 images
2023/06/17 05:36:00 249819 |  TRACE   |  - No improvement for loss: 0.20011 (current best: 0.20006)
2023/06/17 05:36:00 250174 |  TRACE   | Training for 250 steps (current step: 15500)
2023/06/17 05:43:35 212866 |  TRACE   | Validating
2023/06/17 05:44:43 179673 |  TRACE   | Writing results to TensorBoard:
2023/06/17 05:44:43 180540 |  TRACE   |  - Start loss computation
2023/06/17 05:44:43 180817 |  TRACE   |  - Start image generation for 100 images
2023/06/17 05:45:38 093181 |  TRACE   |  - New best loss: 0.19978
2023/06/17 05:45:38 093588 |  TRACE   |  - Improvement over previous best: -0.00027
2023/06/17 05:45:38 141286 |  TRACE   |  - Best model updated!
 
2023/06/17 05:45:38 141548 |  TRACE   | Training for 250 steps (current step: 15750)
2023/06/17 05:53:07 293716 |  TRACE   | Validating
2023/06/17 05:54:15 245430 |  TRACE   | Writing results to TensorBoard:
2023/06/17 05:54:15 246403 |  TRACE   |  - Start loss computation
2023/06/17 05:54:15 246648 |  TRACE   |  - Start image generation for 100 images
2023/06/17 05:55:09 769117 |  TRACE   |  - No improvement for loss: 0.19999 (current best: 0.19978)
2023/06/17 05:55:09 769464 |  TRACE   | Training for 250 steps (current step: 16000)
2023/06/17 06:02:56 640275 |  TRACE   | Validating
2023/06/17 06:04:04 798807 |  TRACE   | Writing results to TensorBoard:
2023/06/17 06:04:04 799714 |  TRACE   |  - Start loss computation
2023/06/17 06:04:04 800024 |  TRACE   |  - Start image generation for 100 images
2023/06/17 06:04:59 488757 |  TRACE   |  - No improvement for loss: 0.19979 (current best: 0.19978)
2023/06/17 06:04:59 489106 |  TRACE   | Training for 250 steps (current step: 16250)
2023/06/17 06:12:32 060702 |  TRACE   | Validating
2023/06/17 06:13:40 175016 |  TRACE   | Writing results to TensorBoard:
2023/06/17 06:13:40 176131 |  TRACE   |  - Start loss computation
2023/06/17 06:13:40 176393 |  TRACE   |  - Start image generation for 100 images
2023/06/17 06:14:35 236452 |  TRACE   |  - No improvement for loss: 0.20015 (current best: 0.19978)
2023/06/17 06:14:35 236788 |  TRACE   | Training for 250 steps (current step: 16500)
2023/06/17 06:22:15 399762 |  TRACE   | Validating
2023/06/17 06:23:24 426269 |  TRACE   | Writing results to TensorBoard:
2023/06/17 06:23:24 427343 |  TRACE   |  - Start loss computation
2023/06/17 06:23:24 427589 |  TRACE   |  - Start image generation for 100 images
2023/06/17 06:24:19 595894 |  TRACE   |  - No improvement for loss: 0.19983 (current best: 0.19978)
2023/06/17 06:24:19 596197 |  TRACE   | Training for 250 steps (current step: 16750)
2023/06/17 06:31:44 834930 |  TRACE   | Validating
2023/06/17 06:32:51 241197 |  TRACE   | Writing results to TensorBoard:
2023/06/17 06:32:51 241974 |  TRACE   |  - Start loss computation
2023/06/17 06:32:51 242240 |  TRACE   |  - Start image generation for 100 images
2023/06/17 06:33:46 348314 |  TRACE   |  - No improvement for loss: 0.19982 (current best: 0.19978)
2023/06/17 06:33:46 348726 |  TRACE   | Training for 250 steps (current step: 17000)
2023/06/17 06:41:18 755546 |  TRACE   | Validating
2023/06/17 06:42:24 851033 |  TRACE   | Writing results to TensorBoard:
2023/06/17 06:42:24 852108 |  TRACE   |  - Start loss computation
2023/06/17 06:42:24 852357 |  TRACE   |  - Start image generation for 100 images
2023/06/17 06:43:19 595384 |  TRACE   |  - No improvement for loss: 0.20024 (current best: 0.19978)
2023/06/17 06:43:19 595767 |  TRACE   | Training for 250 steps (current step: 17250)
2023/06/17 06:50:46 064801 |  TRACE   | Validating
2023/06/17 06:51:51 883981 |  TRACE   | Writing results to TensorBoard:
2023/06/17 06:51:51 884894 |  TRACE   |  - Start loss computation
2023/06/17 06:51:51 885156 |  TRACE   |  - Start image generation for 100 images
2023/06/17 06:52:47 032578 |  TRACE   |  - No improvement for loss: 0.19981 (current best: 0.19978)
2023/06/17 06:52:47 032961 |  TRACE   | Training for 250 steps (current step: 17500)
2023/06/17 07:00:20 107054 |  TRACE   | Validating
2023/06/17 07:01:28 605112 |  TRACE   | Writing results to TensorBoard:
2023/06/17 07:01:28 606197 |  TRACE   |  - Start loss computation
2023/06/17 07:01:28 606469 |  TRACE   |  - Start image generation for 100 images
2023/06/17 07:02:23 568175 |  TRACE   |  - No improvement for loss: 0.19988 (current best: 0.19978)
2023/06/17 07:02:23 568482 |  TRACE   | Training for 250 steps (current step: 17750)
2023/06/17 07:09:53 899684 |  TRACE   | Validating
2023/06/17 07:11:02 229353 |  TRACE   | Writing results to TensorBoard:
2023/06/17 07:11:02 229682 |  TRACE   |  - Start loss computation
2023/06/17 07:11:02 229921 |  TRACE   |  - Start image generation for 100 images
2023/06/17 07:11:57 231412 |  TRACE   |  - No improvement for loss: 0.19974 (current best: 0.19978)
2023/06/17 07:11:57 231847 |  TRACE   | Training for 250 steps (current step: 18000)
2023/06/17 07:19:34 879485 |  TRACE   | Validating
2023/06/17 07:20:43 444986 |  TRACE   | Writing results to TensorBoard:
2023/06/17 07:20:43 445309 |  TRACE   |  - Start loss computation
2023/06/17 07:20:43 445533 |  TRACE   |  - Start image generation for 100 images
2023/06/17 07:21:38 161183 |  TRACE   |  - No improvement for loss: 0.19985 (current best: 0.19978)
2023/06/17 07:21:38 161566 |  TRACE   |  - Early Stopping tolerance reached. Stopping training!
 
2023/06/17 07:21:38 298491 |  TRACE   | Start model evaluation
2023/06/29 17:39:18 913778 |  TRACE   | Evaluation results:
2023/06/29 17:39:18 914143 |  TRACE   |  - Results for field size: global
2023/06/29 17:39:18 914401 |  TRACE   |    - Precision: 59.47%
2023/06/29 17:39:18 914740 |  TRACE   |    - Recall: 49.22%
2023/06/29 17:39:18 914869 |  TRACE   |  - Results for field size: small
2023/06/29 17:39:18 915022 |  TRACE   |    - Precision: 42.56%
2023/06/29 17:39:18 915196 |  TRACE   |    - Recall: 18.41%
2023/06/29 17:39:18 915352 |  TRACE   |  - Results for field size: medium
2023/06/29 17:39:18 915521 |  TRACE   |    - Precision: 54.99%
2023/06/29 17:39:18 915692 |  TRACE   |    - Recall: 52.75%
2023/06/29 17:39:18 915822 |  TRACE   |  - Results for field size: large
2023/06/29 17:39:18 915972 |  TRACE   |    - Precision: 67.88%
2023/06/29 17:39:18 916136 |  TRACE   |    - Recall: 64.60%
2023/06/29 17:39:18 916256 |  TRACE   |  - Results for field size: extra large
2023/06/29 17:39:18 916400 |  TRACE   |    - Precision: 67.66%
2023/06/29 17:39:18 916562 |  TRACE   |    - Recall: 72.18%

2023/07/03 15:46:33 818005 |  TRACE   | Evaluation results:
2023/07/03 15:46:33 818403 |  TRACE   |  - Results for field size: global
2023/07/03 15:46:33 818715 |  TRACE   |    - Precision: 59.54%
2023/07/03 15:46:33 818940 |  TRACE   |    - Recall: 49.18%
2023/07/03 15:46:33 819083 |  TRACE   |  - Results for field size: small
2023/07/03 15:46:33 819246 |  TRACE   |    - Precision: 42.86%
2023/07/03 15:46:33 819423 |  TRACE   |    - Recall: 18.27%
2023/07/03 15:46:33 819556 |  TRACE   |  - Results for field size: medium
2023/07/03 15:46:33 819718 |  TRACE   |    - Precision: 54.99%
2023/07/03 15:46:33 819896 |  TRACE   |    - Recall: 52.73%
2023/07/03 15:46:33 820016 |  TRACE   |  - Results for field size: large
2023/07/03 15:46:33 820164 |  TRACE   |    - Precision: 67.88%
2023/07/03 15:46:33 820365 |  TRACE   |    - Recall: 64.59%
2023/07/03 15:46:33 820505 |  TRACE   |  - Results for field size: extra large
2023/07/03 15:46:33 820664 |  TRACE   |    - Precision: 67.64%
2023/07/03 15:46:33 820859 |  TRACE   |    - Recall: 72.17%
