image encoder batch normalization
https://discuss.pytorch.org/t/why-different-validation-and-train-loss-for-the-same-data/60487/3
Why different validation and train loss for the same data
Hi @ptrblck I am not using any dropout layers. The architecture I am using can be seen below: ResNet3D( (strConv1): Conv3d(64, 64, kernel_size=(3, 3, 3), stride=(2, 2, 2), padding=(1, 1, 1)) (strConv2): Conv3d(128, 128, kernel_size=(3, 3, 3), stride=(2, 2,
discuss.pytorch.org
test loss가 train loss에 비해 월등히 높은 현상이 일어남. (둘의 dataset이 같은데도.) - batchnormalization eval 했더니 해결되었다.
batch size가 작아서라는 의견도,
data가 non-stationary 해서 그렇다는 의견도( momentum을 줄이면 된다고 하는데) 있고,
batch norm이 eval에서는 batch statistics를 train 때의 평균을 가지고 하는데 뭐 그게 잘 안맞을 때 이런 현상이 있다는 얘기도 있다. (track_running_stats=False)
Model.eval() gives incorrect loss for model with batchnorm layers
Its pretty huge. with net.train(): 7e-8 with net.eval(): 99.3 Here is the truncated output (btw i fixed a bug in the original repro code, where the input batchsize did not match the output batchsize, now batchone = torch.ones[2, 1, 64, 64]… instead of [4
discuss.pytorch.org
https://github.com/pytorch/pytorch/issues/4741
Using model.eval() with batchnorm gives high error · Issue #4741 · pytorch/pytorch
I tested my network using model.eval() on one testing element and the result was very high. I tried to do testing using the same minibatch size as the training and also testing on one batch size wi...
github.com
'Experiments' 카테고리의 다른 글
| learning rate 결정하기 (0) | 2021.08.11 |
|---|---|
| data loader 속도 높이기 (0) | 2021.08.09 |
| python pickle errors (stable-baselines3 trained agent) (0) | 2021.07.11 |
| ResNet & iResNet (0) | 2021.02.11 |
| Imitating Latent Policy from Observation 2 (0) | 2020.12.26 |
댓글