Compared CivitAI default FLUX LoRA vs my best FLUX, tested different regularization images trainings and T5 Attention Mask - next is different LoRA ranks impact on quality.
I am about to complete my Kohya SS GUI FLUX LoRA training. Next is hopefully full Fine-Tuning / DreamBooth
Only left thing is testing impact of different LoRA Ranks on quality
I find that T5 Attention Mask is improving quality slightly
Further reducing the LR didn't help
16 GB fast config has slightly worse quality than 10 GB slower config
CivitAI default config is lower quality as expected - bigger batch size and 512px
Regularization / classification images usage not helping. Tested different Prior loss weight as well
Windows training is still significantly slower than Linux. e.g. RTX 3090 is 5.5 second / it on Linux and 7.7 second / it on Windows. I am using exactly same libraries. Compared pip freeze. Trying to find the reason
Entire R&D with full history and grids and configs shared here : https://www.patreon.com/posts/110293257
Newest comparison full grids are here : Reg_T5_Attention_Mask_CivitAI_50_Epoch.jpg, Reg_T5_Attention_Mask_CivitAI_100_Epoch.jpg, Reg_T5_Attention_Mask_CivitAI_150_Epoch.jpg, T5_Attention_Mask_v1.jpg, Civit_Reg_15GB_150_Epoch_Compare.jpg
Full grids and more info : https://www.reddit.com/r/StableDiffusion/comments/1ezivfn/compared_civitai_default_flux_lora_vs_my_best/
Top comments (0)