self.log twice problem #13263
Unanswered
zcx-language
asked this question in
DDP / multi-GPU / multi-node
Replies: 1 comment
-
|
is it possible for you to reproduce this issue?? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
There are two metrics in my work:
psnrandbit_acc. When I useModelCheckpointcallback which only monitor one metric, I try use above code to choose best checkpoints where psnr > 30, but I got the error below:You called
self.log(tbit_acc, ...)twice invalidation_epoch_endwith different arguments. This is not allowedI dont not understand why
self.logcan be called twice inif....else...sentence. I run it in two gpus and I kown that from #6501So it is the reason of
valid_img_metric['PeakSignalNoiseRatio']get different values in different gpus caused this problem? but the metric is synchronized.Beta Was this translation helpful? Give feedback.
All reactions