Question about patch embedder training

[kingofprank]

Hi, thanks for your great jobs. I have some questions in stage 1 patch embedder training.

1. I want to know whether the latent need to add noise like in diffusion forward process to calculate the mse loss in stage1 before sending patch embedder.
2. If I want to train an I2V task, do I need to concatenate all the conditions in channels before feeding them into the patch embedder for training?
3. And what is the value of mse loss after convergence in your training setting?

Can you share more information? Thanks~

[CuddleSabe]

Hi, thanks for your great jobs. I have some questions in stage 1 patch embedder training.

1. I want to know whether the latent need to add noise like in diffusion forward process to calculate the mse loss in stage1 before sending patch embedder.
2. If I want to train an I2V task, do I need to concatenate all the conditions in channels before feeding them into the patch embedder for training?
3. And what is the value of mse loss after convergence in your training setting?

Can you share more information? Thanks~

I tried, the loss is 0.36, the middle is speedup version of OMGSR

[kingofprank]

Hi, thanks for your great jobs. I have some questions in stage 1 patch embedder training.

1. I want to know whether the latent need to add noise like in diffusion forward process to calculate the mse loss in stage1 before sending patch embedder.
2. If I want to train an I2V task, do I need to concatenate all the conditions in channels before feeding them into the patch embedder for training?
3. And what is the value of mse loss after convergence in your training setting?

Can you share more information? Thanks~

I tried, the loss is 0.36, the middle is speedup version of OMGSR

Thanks, the results seem good. I wonder the loss 0.36 is the mse loss for patch embedder in stage1 or diffusion mse loss in stage2. And can you share your process method about the Question 1?

[CuddleSabe]

Thanks, the results seem good. I wonder the loss 0.36 is the mse loss for patch embedder in stage1 or diffusion mse loss in stage2. And can you share your process method about the Question 1?

0.36 is the loss of patch embedder. there is no diffusion mse loss in OMGSR because its a 1step Super-Resolution model