Current results of training - epoch 4

[johndpope]

i used another of the videos as driving - and it's (almost) obviously not rotating the head past the point where the original movie went - see below.

cross_reenacted_image_57

pred_frame_191

tomorrow i plug in bigger dataset.

UPDATE - #37

when I normalize the images - i end up with this - looks bad - I add some code in train.py to un-normalize - happy with current results....

fyi - this is the frames dump out from mp4 - head cropped / maybe some warping.

[johndpope]

https://github.com/tencent-ailab/V-Express/blob/main/assets/crop_example.jpeg
i adjust the cropping to find a sweet spot.

#37

[Jie-zju]

I trained on data like vox. Looking forward for more results!

[johndpope]

epoch 21 - it's converging.....

[Jie-zju]

So，as I mentioned before. Loss on face foreground?

[johndpope]

there's like 6-7 different losses - https://github.com/johndpope/MegaPortrait-hack/blob/main/train.py
i didnt do gaze loss yet (i drafted it - hit a snag - need to take another look)

MegaPortrait-hack/model.py

Line 1885 in ff9cf22

self.weights['gaze'] * 1 #gaze_loss

[johndpope]

epoch 50

[JZArray]

epoch 50

Is this the self-reconstruction result?

[JZArray]

How do you reenactment results look like in the eval dataset? BTW, how many IDs have you used to train this model?

[johndpope]

I need 2 years of gpu time to actually complete 200,000 epochs -
I have dataset here 35,000 videos
#37

The augmentation I’m rendering every frame - but hit a snag with different lengths so I’m only overfitting to 1 source video - 1 driving - 1 star source - 1 star driving.
Don’t really want to burn out my 3090 card - I’m looking at vertex ai - the pre processing to warp and crop is significant time sink.

I’m exploring cheaper gpu training hacks to collapse training time line. My 3090 can spit out very res - hi fidelity images with stable diffusion - so this code is useless to me if I can’t train it.

https://github.com/johndpope/LadaGAN-pytorch

[JZArray]

I need 2 years of gpu time to actually complete 200,000 epochs - I have dataset here 35,000 videos #37

The augmentation I’m rendering every frame - but hit a snag with different lengths so I’m only overfitting to 1 source video - 1 driving - 1 star source - 1 star driving. Don’t really want to burn out my 3090 card - I’m looking at vertex ai - the pre processing to warp and crop is significant time sink.

I’m exploring cheaper gpu training hacks to collapse training time line. My 3090 can spit out very res - hi fidelity images with stable diffusion - so this code is useless to me if I can’t train it.

https://github.com/johndpope/LadaGAN-pytorch

ok, I see, because when increasing the number of IDs, in my case, ID leakage appears, not sure whether you also have the same problem with your codes.