Stable Diffusion Transparent Generations: SD Forge LayerDiffusion
Vložit
- čas přidán 15. 06. 2024
- In today's tutorial, I'm excited to share with you a fantastic tool for generating images with transparent elements using Stable Diffusion Forge UI. We'll explore how to install and use the "sd-forge-layerdiffusion" extension, which allows for creating images with transparent backgrounds, even for complex objects like glass.
Step-by-Step Guide:
- Installation: I'll walk you through installing the extension directly from a URL since it may not be listed in the extensions tab yet.
- Setting Up: Once installed, we'll apply the extension and restart the UI. I'll show you where to find the "LayerDiffusion" feature and how to enable it.
- First Test: We'll test the tool by creating a 3D fantasy shield. I'll discuss different sampling methods and how to choose the right settings for your image.
- Troubleshooting: If you encounter issues with model downloads, I'll guide you through manually installing the necessary models from Hugging Face.
- Experimenting with Prompts: We'll experiment with various prompts and observe the results. I'll demonstrate with a fantasy blue bottle, showcasing the tool's ability to render transparent objects realistically.
- Advanced Usage: I'll also cover other options like "From Foreground to Blending" and "From Background to Blending," showing you how to blend images with transparent backgrounds effectively.
- Common Errors: We'll address some common errors and limitations, like size restrictions and issues with the "hiresfix" option.
SDXL and SD Version 1.5: I'll touch on the compatibility of the tool with different Stable Diffusion models.
- Additional Resources: Links to the extension page and other useful resources will be provided for further exploration and troubleshooting.
Conclusion:
This tool opens up a world of possibilities for designers and creators looking to integrate transparency into their digital art seamlessly. While there are some current limitations and bugs, the potential for creating intricate and detailed images is exciting. I'll keep you updated on future improvements and releases.
If you find this tutorial helpful, please leave a like and consider subscribing for more content like this. Your support has been instrumental in growing this channel, and I'm thrilled to continue sharing my knowledge with you. Drop any questions or experiences you have in the comments, and let's keep learning together!
Chapters:
00:00 How to install the extension
00:47 How to use SD Forge LayerDiffusion
05:15 More info and Errors
I use the Stable Diffusion Forge UI with Juggernaut XL V9
You can get the extension from here:
github.com/layerdiffusion/sd-forge-layerdiffusion
If the models don't download automatically you can download them from: huggingface.co/LayerDiffusion/layerdiffusion-v1/tree/main
and place it in the folder \webui\models\layer_model
try to use 1024x1024px, it seems that forge has some bug with control net and other extension on other sizes then 1024x1024px, or 1024x576px, probably works more but has to be a certain size, it doesn't work if I use 1200x672, or 912x512 or 1024x816 and so on. Update: Seems that the image size need to be divisible by 64
Update:
For those who are using the new transparent extension LayerDiffusion with Forge UI
by default the transparent version is saved in the temp folder example:
C:/Users/Pixaroma(your user name)/AppData/Local/Temp/gradio
If you go to settings you can search for Saving images/grids in the left, then scroll down until you find Directory for temporary images; leave empty for default
there put the path to your folder like: D:\Forge\webui\outputs\txt2img-images
Go up and press Apply Settings, and Reload UI
I put screenshots in my facebook group: facebook.com/groups/pixaromacommunity/posts/3250044421966429/
Venturing into the world of storytelling and creative expression, right? VideoGPT has quickly become my favorite tool for adding that professional touch to my videos seamlessly.
It's cool, but i like to have more control on my videos ☺️ maybe Sora when will be public can do some better job
This is truly a game changer. Can't wait to see how it progresses
When this is working well it will be a masive improvement on how we build images. very excited this is now being developed.
Regarding the error you encountered, it reminds me of a similar issue I faced with a different extension some time ago. The cause of the error in my case was that the image dimensions (both height and width) had to be multiples of 64. This limitation also existed in the original Stable Diffusion, and although recent versions have overcome this, the extensions might not have yet.
I'm not certain if this will resolve the issue in your case, but it could be worth attempting.
Aside from that, this is a really cool extension, and I will definitely give it a try. Thanks for the great video!
yeah the error is in forge not related to the extension, i get the same error with control net also if i dont use sizes divisible by 64
I was actually looking for a YT video about this earlier today and nobody had done one, thanks!
Spectacular extension.
Wow, i can't wait to try this, thanks!
Great video! Thank you very much!!!!!
this is one of the best new last time
awesome, gonna help a lot in my animations
Make more contents about transparent generations! Thank you!
I've been having a blast playing with this one. Had to use a version from the pull requests to get the default behavior to save the transparent PNG as well instead of just the preview in the output folder. Other than that I'm enjoying it despite it not always working.
it saves in gradio default i think the image, i set from settings all the path to save in the same folder from settings, Paths for saving
and i put on all one folder, and also on Saving images/grids Directory for temporary images; leave empty for default, i put there the path to the folder i want and then apply settings and reload ui
@@pixaroma it definitely was only saving the preview image with the checkerboard pattern instead of alpha channel in tact. But if there's an update I'm happy to give that a try. Biggest challenge for me using it is the added inference time on my GTX 1060 GPU. But in those situations where I need transparency there's no better option than this one.
@@magejoshplays hope it works for you, I found it that by default the transparent version is saved in the temp folder example:
C:/Users/Pixaroma(your user name)/AppData/Local/Temp/gradio
If you go to settings you can search for Saving images/grids in the left, then scroll down until you find Directory for temporary images; leave empty for default
there put the path to your folder like: D:\Forge\webui\outputs\txt2img-images
Go up and press Apply Settings, and Reload UI See if that helps, and also the name is not how it was set, is more like a temp file name tmpbtnj_wtm.png
Nice information
For those who have "AttributeError: 'VAE' object has no attribute 'clone' " message in the console as I had,
think to update Forge diffusion
It solved my issue
thanks, it works.😀🤗
It seems to be a bug in forge that appears when you do certain things (control net, the LayerDiffusion Extension and so on), I get the same error TypeError: 'NoneType' object is not iterable if Ii use certain sizes, for example it works if is 1024x1024px, or 1024x576px, but i get that error if I use 1200x672, or 912x512 or 1024x816 and so on. Seems that the image size need to be divisible by 64
Nice feature
ah... yes, I have gave up after installing it last night due to that error! it's sensitive to dimensions in general. BTW, the menu in SDForge has changed a bit since this video. I'm super stoked about this addition and hope regular A1111 (SD3) can find a way to implement. or would that be swarm now? So many new things in this area. I hope you have more vids for all the other new stuff OOTB in SDForge. Trying to figure out what they are for.
I will try to keep up with new releases as i can, I don't have preference i just use what is easy to use and fast, so any ui or model as long as it does the job
ATTENTION: by default the transparent version is saved in the temp folder example:
C:/Users/Pixaroma(your user name)/AppData/Local/Temp/gradio
If you go to settings you can search for Saving images/grids in the left, then scroll down until you find Directory for temporary images; leave empty for default
there put the path to your folder like: D:\Forge\webui\outputs\txt2img-images
Go up and press Apply Settings, and Reload UI
Thank you for telling me, otherwise I already wanted to remove this extension.😀
You dont know how many times i've been looking for something just to find you posted a video about it 24 hours to a week ago xD
Thank you for this, shame it was for SDXL, still in the 1.5 phase but maybe this is the time to jump in
Congrats on the monetization!
Thank you ☺️
Thanks!
Hey… thanks for the tutorial. 🙏
I have some questions
Will this work on 12GB Vram?
Can I upscale images right here with transparency or do I need to use external program?
the extension is quite new, at the moment seems to be some bugs in forge if the image size is not divisible with 64. They say will fix transparency in img2img so maybe then we can use it, right now it doesnt work with stable diffusion upscalers because it mess up with transparency. So you need an external upscaler. For vram i think it should work on 12gb on my 24gb it says Current Free GPU Memory (MB) = 12537
Congrats !🎉🎉🎉🎉🎉
Thank you ☺️ i have waited this moment for a while
It can be said that using it is very difficult to use, from being picky about the model to its size, it always generates errors and distorts colors a lot. If it can fit with multiple models as well as sizes (not necessarily looking too matching and blending with the background because this requires post-processing), at least it won't have as many errors anymore. am using forge
Is still work in progress:) I know it has errors as any new project ,but I like their idea with transparent background generation, so hope they make it better in the future
can this work in Fooocus????????
The extension is for forge, probably that why the added sd forge in the name of extension so I don't think it works, but i don't have focus to test it, but probably is not working
it works once then gives "TypeError: 'NoneType' object is not iterable"error for each subsequent attempt. Hopefully it gets fixed one day
Here it even worked with the new Juggernaut XL Lighting V9, but it is saving the wrong image (it generates 2 images, one transparent PNG and the other with gray squares). But I click save on the correct image and it works. How do I make it automatic saving the correct image?
If you didnt defined the path to where to save i think it saves automatic in the gradio temp folder. After you generate select the image that doesnt have the squares, then right click and copy the path address, there it should show the path folder where it save it could be something like C:\Users\YourUserName\AppData\Local\Temp\gradio . I have set all my outputs in a single folder so i dont have to search for in the settings tab you have Paths for saving and in the saving image grids you have Directory for temporary images; leave empty for default, there you can also put the path you want and then apply settings and reload ui
I found them. All my images from yesterday were there. Yesterday it was late and I couldn't download animaPencilXL_v100 and juggernautXL_version6Rundiffusion, but I put them on to download. I hope it gets even better. Thank you very much. @@pixaroma
Have been trying to work with this extension for past few days and it wouldn’t work - will try again
post some screenshot and what errors you get and the results in my group and I will take a loot to see if something is different from what I am doing facebook.com/groups/pixaromacommunity
You have a great voice, reminds me of V in 2077 😎
Thank you, is AI voice 😀 that speaks my text,, but i think it sounds really cool for an Ai voice and is my favorite for tutorials
Wow! It sounds so real!
does it work on image to image too and something like 512x512 or 256x256 along its a square image.
Did they add it recently? Last time they said will add it in a week but wasn't ready yet. Hope they improve it, is quite useful
@@pixaroma have you tested with sd 1.5? i still use sd 1.5 but it gave me error. SD 1.5 is really good for stylized images.Currently its giving me an error. It is not working for me with sd 1.5 on image to image tab
I didn't test it, i dont use 1.5 anymore only sdxl because i can do bigger images. I usually use juggernaut xl models
They write that you need to use the jagernayt 6 model-'is that so?
I used v9, they say if you want to do insanity check and get exactly the same results then use those models, but i didn't need the exact same models. I didn't try v6 to be able to compare
excellent walkthrough. Heard about this from @stephantual channel who is another great new creator in this arena. thanks again
thank you, I will check it out :)
Can I use this extension on A1111?
they say is for forge, so I dont think it work on A1111
I already tested everything that is way and I do not receive transparent images, I gave up using the extension!
did you clicked on second image, that doesnt have squares, and tried to save that on your pc and still is not transparent? by default it save it in gradio forge temp folder, instead of saving in the output folder.
i can not show Layerdiffusion SD 1.9.3
Not sure what to say, what you see in video is what i got
it still does not work, I followed all the steps including downloading the hugging face models and placing them in the directory you suggested
can you tell me what part doesnt work, do you get errors or what doesnt work?
This did not work for me. I tried the same exact model(and other Juggernaut models), I took a screenshot of your settings and set same exact settings. I downloaded all the models and place them on the folder mentioned and all it does is create images with backgrounds as if I did nothing at all. I check the console and it doesn't show any errors. I have been hoping for a model like this for a while. It sucks I can't get it to work. I have a Video Card with only 8G. Is this the problem?
I get exactly the same result as stated here.
I just tested now and it works for me, not sure if is because of memory, but i think you will get an error if it will be that. Try to update your forge, I assume you are using forge not the Automatic1111 and try to update the extension also from extensions check for updates and apply and restart
@@pixaroma I got it to work by installing the newest version of Forge. I couldn't update the older version so I did a new install. I didn't have time to test it much but thanks for your help.
I do not receive transparent images😕
From those 2 images one is with square and one is with transparent, if you download that one with transparent it should work. By default it saves the one with square in the output folder and the one with transparency in your gradio temp folder. You can change the folder for temp in your settings. Also try with a simple prompt first, if you are adding too many details in the prompt will add a background
I have installed this plug-in and restarted it, but its interface does not appear. What is the reason?
If you are using forge it should appear like in the video, not sure what is the cause mayne you can ask on the extension page the have a section for bugs and issues
@@pixaroma I use stable diffusion webui v1.9.3. Can SD Forge LayerDiffusion work on stable diffusion webui v1.9.3 ?
@@hicksw5687 for the forge is separate extension, try this one maybe it works for your ui github.com/layerdiffusion/LayerDiffuse as you can see in the description of this link it say that for forge is different one and give this for forge github.com/layerdiffusion/sd-forge-layerdiffuse
@@pixaroma I installed the one you mentioned instead, but the interface still didn't appear.
Can this function be used to remove the background of general pictures? For example, upload a picture directly and then remove the background?
@@hicksw5687 maybe is not compatible yet, automatic1111 made an update recently, so maybe that is the cause, sorry I can not help :(
Ok, and where is my benefit? I can use RemBG extension in A1111.
This doesn't remove background it generate without background, with remove background you can not make the glass transparent, but in forge it can be done, you get transparent shadows, transparent glass and no background, so is a totally different thing
@@pixaroma unfortunately, it's not working: 'VAE' object has no attribute 'clone'
@@TheMaxvintry maybe at 1024px width and height size, if is not that maybe the update it and has some bugs, not sure what to say, worked when i recorded the video
@@TheMaxvin follow this, seems other got same bug on the extension maybe someone comes with a fix, github.com/layerdiffusion/sd-forge-layerdiffuse/issues/13
The AI voice over just being slighty off here & there is somehow annoying. Most won't even hear this but if you've recorded professional voice overs in the studio for over a decade this is rather jarring. Not a deal breaker, great content.
Yeah, they have some problem with voices right now i can not use more then 500 characters or will go crazy and it is hard to find a balance between realism and consistency, hope they can fix it. Thanks
dont work for me
Hi, which part, do you get an error, what part doesn't work?
"oiler"
Didn't get that, the Euler?
@@pixaroma yes, it's pronounced "oiler" :D
I use an AI voice so it doesn't know how to pronounce all the words correctly yet 😃
@@pixaroma just put oiler in the script lmao
the ai voice you use is terrible
You are the only one who commented that didn't like it, all the others asked from where I got it since it sounds realistic. I guess I cannot make anyone happy.