How to AI Upscale with ControlNet Tiles - High Resolution for Everyone!
Vložit
- čas přidán 3. 05. 2023
- Render 8K with a cheap GPU! This is ControlNet 1.1 tiles for Stable diffusion, together with some clever use of upscaling extensions.
ControlNet 1.1 Tutorial and install guide: • How to use ControlNet....
ControlNet 1.1 Models: huggingface.co/lllyasviel/Con...
4x Ultrasharp model: mega.nz/folder/qZRBmaIY#nIG8K...
FREE Prompt styles here:
/ sebs-hilis-79649068
Support me on Patreon to get access to unique perks! / sebastiankamph
Chat with me in our community discord: / discord
My Weekly AI Art Challenges • Let's AI Paint - Weekl...
ControlNet tutorial and install guide • NEW ControlNet for Sta...
Famous Scenes Remade by ControlNet AI • Famous Scenes Remade b...
LIVE Pose in Stable Diffusion • LIVE Pose in Stable Di...
Ultimate Stable diffusion guide • Stable diffusion tutor...
Control Lights in Stable Diffusion • Control Light in AI Im...
Inpainting Tutorial - Stable Diffusion • Inpainting Tutorial - ...
The Rise of AI Art: A Creative Revolution • The Rise of AI Art - A...
7 Secrets to writing with ChatGPT (Don't tell your boss!) • 7 Secrets in ChatGPT (...
Ultimate Animation guide in Stable diffusion • Stable diffusion anima...
Dreambooth tutorial for Stable diffusion • Dreambooth tutorial fo...
5 tricks you're not using in Stable diffusion • Top 5 Stable diffusion...
Avoid these 7 mistakes in Stable diffusion • Don't make these 7 mis...
How to ChatGPT. ChatGPT explained in 1 minute • How to ChatGPT? Chat G...
This is Adobe Firefly. AI For Professionals • This Is Adobe Firefly....
Adobe Firefly Tutorial • Adobe Firefly Tutorial...
ChatGPT Playlist • ChatGPT
Download Prompt styles: www.patreon.com/posts/sebs-hilis-79649068
Please support me on Patreon for early access videos. It will also help me keep creating these guides: www.patreon.com/sebastiankamph
@@interesant7 Hi, as I'm an advertising man since 1975 we deal with images and photos pictures designs rights all the time, but people need to understand that if you send you work to the Internet this will be a trouble, when someone get your photo or you draw and copy it 100% and sell this is crime, but when you get some work on internet and use it to learn this is not a crime and we all do this since 1992 when internet start, just think and you will see this.Also when you use more than one article to write yours this is not crime because this is research, if you read about Rights you will see that has a lot more things evolved than people are saying, Artist would like to show his works to get applause and than start to complain this is the problem. Everything that we see theses days was create from something and this a "petrea"rule you don't take milk from stone.
The link doesn't open, is it exclusive for a specific countries?
2:04 leave everything default, except denoising to 0.15 for first pass (could go higher second pass)
2:32 check enable controlnet, re-copy image down below, preprocessor - tile_resample
2:57 model - control_v11f1e_sd15_tile [a371b31b]
3:37 leave default, except control mode - ControlNet is more important
3:50 for script - 'Ultimate SD upscale', target size to 'scale from image size' x2, upscaler - '4x UltraSharp'
im getting low opacity faces on some tiles of the image how to fix it
You know what? I really like how you take time to compare the before and after result of the old man's 2k and 4k version, switching serveral times between them. Sounds like a small thing but thats what most youtubers I've seen don't pay attention to, making it more of an effort to follow for their viewers, and are thus missing an opportunity to get theit point across.
....sside that the workflow you pieced together is awesome. THANK YOU! :D
Thank you for the kind words and your thoughtful response. It means a lot 😊
Few things:-
1-10:22 If you want an upscaler that adds details to skins try 4x_NMKD-Siax_200k or 4x_NickelbackFS_72000_G, when I upscaled using those two the person's skin went from plastic smooth to highly realistic (with blemishes, veins, wrinkles, pores, beauty marks...seems these upscalers were trained on photos specifically).
2-5:00 When I use the Seams Fix option in Ultimate Upscale I get 2 results, one of them before the seams fix and one of them after, and the one "after" barely has any noticeable seams whatsoever, so maybe try checking the "seams fix" option, either way it's still miles ahead of the original SD Upscale which suffered from ghosting issues between squares.
Got to try those upscaler models, thank you for the tip!
Thanks 🙏
I love nmkd, but it seems to add some kind of green tinting to the images. I've not heard of nickelback upscale, will check it out. thx!
Hello, where can I download the 4x_NickelbackFS_72000_G ?
Thanks! Can you please let us know, where to best download them?
Thanks for speaking up about your experience of being unable to get rid of seams. I've had exactly the same experience myself, great to know it's not just me. Big props for using photoshop to clearly demonstrate the seams and compare results of different seams-fix modes!
Thank you for the support, Mario. Greatly appreciated 😊💫
m getting low opacity faces on some tiles of the image how to fix it@@sebastiankamph
You do not need to copy the image to the ControlNet when using img2img, if you leave it blank it will just copy the image you're using in img2img.
thx
good advice
I think he also forgot to replace the controlnet image with the upscaled ones so he has been using the low res 512*512 image in controlnet the entire time
Ok, thanks! Whew! I'm glad that's the case. I was already starting to get annoyed with that step.
@@maxmustermann3938nice catch. hopefully he see this.
I love the way you explain things. You're truly a gifted educator.
Wow, thank you! 😊🌟
I’ve been waiting for someone to upload this video, so glad to see you do it. Tile controlnet is so amazing. Btw, you don’t need to put an image into the preprocessor, it automatically fetches it from img2img. Your next video should be on the Inpainting model, which has rendered all “inpainting models” obsolete (Realisticvison-inpainting, etc etc)
this is insane, thank a lot for this video, upscale like this way a picture is crazy
Very cool, Mr. Kamph!
I will do some tests soon.
Thank you very much
Oh this is a godsend. I work on a children-book with a lot of photorealistic images in the moment and use a combined approach of oldschool Photoshop work and Stable Diffusion and struggle a bit with the upscaling in some images. I‘ll spend the rest of the night using this approach and think that it will solve my issues. Thanks for the great video!
I'm a composite artist doing the same kind of work. Would love to follow your journey! Are you sharing your work anywhere?
Oh my word, this is super helpful. Thanks for making the vid!
Dude, thank you so much for your content. It is fantastic. You have completely changed my experience of design and photomanipulation. Can't wait to see what you talk about next!
Great to hear! And welcome aboard! 🌟
Very well explained. Thank you.
OMG! This compilation is awesomazing!
This is so incredibly helpful! Thank you very much!
Greetings from Brazil. One of the best videos I've watched these last few days. It helped me a lot here.
Glad you liked it! 😊
quick tip for speed: you can turn up the tile width to generate the upscale faster, as it requires less total tiles and so the total time waiting between each tile is reduced. It also allows controlnet when upscaling to retain more context from the image, theoretically resulting in a better picture. 768x768 is the obvious option to choose, but even higher is an option.
I had a think about this. For the 768 models, surely. But the ones trained on 512, I assume the image quality running 512 must be better compared to the speed gains? Either way, interesting thought.
@@sebastiankamph I've always found that even 512 models do find with higher resolutions once they have a framework to work with.
Nobody can mastrubate that quickly!
@@sebastiankamph The 512 models being limited to 512 is sorta legend at this point imo. Some of comunity models can do ~1024x800 very decently. crystalClearV1 (note v1, v2 seems to have screwed up) is very nice model for that. I'm pretty sure you can gen 768x768 with it with no problems. Though there is usual caveat that aspect ration is very important depending on prompt.
I've tried with 512,768,1024 and 2048.you get less mutations and stuff with higher tile size,if you also lose quality or depth, I'm unsure. One thing for sure going 2x in upscale multiple times is better than 4x for example in a single step regarding end quality
MINDBLOWN. Thanks for going so in depth.
You're welcome! Thank you for the support you've given Ronny, it means a lot to me.
The best tutorial ive watched. Amd im not even talking about the actual golden information, but just the way you do tutorials is great. Subscribed
Happy to hear that! Welcome aboard
I have made a few interesting (and scary!) image fails while using the ultimate upscaler grids with too high a denoising settings, I'm going to have to try this control net approach, it looks great thanks!
This was a super helpful video for my potato PC. All it takes is time and I can make my own 1440p desktop backgrounds. And here I was saving up my pennies for a used 3090. Thanks!
Thanks a lot Sebastian! Your explanations are very useful! ❤
😘😘
Firstly thanks a lot for the video, it was my starting point. So, I experimented a lot with this and found out that the reason you might be getting less detail and smoothed out skin is that while you're setting low denoise 0.15, and therefore very low freedom to create content, you're using sampler Euler a, which generates a lot of chaos. What I'm doing is setting denoise to 0.4, sampler DPM++ 2M Karras, but to compensate for the extra freedom I'm giving to the machine I crank up ControlNet's "Control Weight" to 2. I've achieved great results. You can also experiment with the CFG scale between 7 (default) and 14 if you want to force the machine to create certain details. Oh yeah, and DEFINITELY use Deliberate or Reliberate guys. These checkpoints are game-changing.
This is getting so amazing. When I get a txt2image that's good, I will use it to gen a few images at 4k res with a .3 denoising. I then take them into photoshop as layers and erase all the bad parts until I have the best image. Then I take that image and upscale it to 8k. I resize that back down to 4k bring it back into photoshop and erase any bad detail it may have picked up. Then I set denoising to .15 or .2 and upscale to 8k.
edit: I should mention if the txt2image you have has any imperfections they will be amplified, use inpainting to fix them first.
I understood everything from the first time and was able to repeat it myself, thank you a lot
Thank you for the comment, glad the video was helpful!
I think the different results between stepped upscale and direct upscale are quite easy to explain. With the direct to 8k method each 'starting tile' is quite blurry so the controlnet has trouble doing it's job of making the added detail make sense within the context of the image. In comparison the downside of stepped upscaling seems to be general loss of detail at each pass which has a cumulative effect. I think both of these downsides can be mitigated and there is probably a sweet spot to be found for number of stages to maintain both coherence and detail. Some general rule like "fewer stages is better as long as each tile is sharp enough for controlnet to work" might apply. Or; "Two stages are good, three is ok, four will loose too much detail". Finally, if there are techniques that can be used to mitigate the loss of detail on the staged approach it might be superior overall.
Smoothing also comes from low number of steps. As denoise strentgh is lowering them, set them at least 100+, then steps on each tile will be 10+ with denoising around 0.1. It will provide more details and texture to images.
Regarding stepping up vs. max in one step, the images are so close to one another that you could stack the output in an app like Photoshop and paint in a mask that would blend the best aspects of each. Similarly, the background tiling example you showed could easily be masked over with a dedicated AI upscaler like Topaz.
But with A1111 you can add/fix some details while you upscale a photo
Thanks for this video, made a few images whilst watching.
THANK YOU SO MUCH FOR THIS!
Thank you so much, this is really useful!
I think you could use both of the results at the end. Use frequency separation in Photoshop and ad it to your 4k render. Erase the parts that don't work :)
This is all was i asking for, similar to the Midjourney Upscaler in their Web. Many Thanks Sir
You can also use multidiffusion, it does basically the same thing automatically, and it actually ADDS detail instead of removing it.
@sebastian : Pls add Upscaling tutorial with Multidiffusion! The results seem to be better than with tiles. But my mac gets ot of memory on higher resolutions
5:15 thank you for that detailed look into seams. It's been driving me crazy. Someone who opened an issue mentioned that apparently it goes away in Linux, but I can't verify that.
6:34
To improve this we could use a photography technique, we could add noise to the image. In addition we could mask the noise according to the luminance of the photo, so that the noise is more pronounced in the dark elements of the image and less in the light ones, to affect the portrait as little as possible and as much as possible to the background. It is a common technique to remove banding from images.
I love the upscalers. I've seen them before but I never knew how to use them
I like using the tule diffusion extensión instead. Same thing. Just I find it gives more granular control over the upscale. You can do quadrant prompting to target specific sections. And you can change the denoise or CFG just for the tiles or have heightxwidth custom settings for each tile
One trick I recently discovered and started to experiment with is to generate two images, one with low denoising (0.15-0.25) and one with higher denoising (0.5-0.6). Then I load them both into Gimp, with the low denoised image above the high denoised one (so the less-changed image is on top). Then apply a transparent mask, and cycle between the images, and paint it over any parts where you think the low-denoised image needs more detail. This allows you to keep any good detail generated by the higher denoising, while having the majority of the image not overly changed. Of course, it takes a lot of time and effort, but I think it's worth it.
Great content Seb. Informative as always, you help "demystify" all of these extensions for the common man. BTW, Did you see the display of still-life art? It was not at all moving.
Thanks! 😂💯
Great! Thanks for your work!
Thank you so much for sharing! Superb video!
Thank you!
fantastic video sebastian! everyone is sleeping on controlnet, it has tons of possibilities
You could also do a regular quick upscale and then send the scaled image back to in-paint and then mask an area where the detail was lost and regenerate that area using a new prompt that covers only that masked area. And then repeat for other areas. A blurry area like an out of focus background is not going to need any extra detail anyway. Only add detail where it's needed.
Thank you Seb! Fantastic info!! 🙏🙏
Happy to help! Thanks for your continued support 😊
That was amazing my dude! Would love to have a continuation video with the same thing but working on a small video, like how to use controlnet in that manner with potato PC doing a small animation if possible upscaled... Let me know if that could be a thing!
thank you for the amazing tutorials!
Thank you for the kind words! 🌟
That "smoothing" you're getting is actually the upscaler, 4x-Ultrasharp is actually not perfect at retaining details as you showed, it's very good, very efficient, and fast (being a GAN). I personally like LDSR for keeping details, it's slower by far and more memory intensive yes, but there is much less smoothing. Haven't tried with tiling and your work flow to try and scale to insane sizes, worth you trying out though
many thanks ! ♥
Superb tutorial for scaling! Now I can generate high resolution images even with my old GTX 1060 3 GB card that's running my Linux desktop.
Nice!
nice
thanks my dude
Great video, thank you.
To combat the over smoothing of details, you could bring it into Photoshop and use an unsharp mask to add detail, then send that back to SD to upscale.
I love unsharp mask! Good tip. I usually use it for final touches.
Great, thank you.
What... the.... wow! Thank you so much!
You're welcome! 💫
thank you!
Thank you again :)
That's brilliant, Thanks a lot.
Glad you liked it!
Hi Sebastian love your videos and thanks for this one! So I experimented with this method, and compared to other AI upscaling options, and I must say Gigapixel AI is a significantly better option for upscaling especially its most recent version in speed, details and ease of use. I havent yet found a great upscaling method within automatic1111 / SD to get to 4k or above resolution (I do use hi res fix), that comes close to a commercial solution. Either way great video, I was just disappointed this didnt turn out to be the holy grail. Maybe soon in the future!
There is also an ONNX 4x-ultra_sharp model in the link provided by Sebastian, you can use this ONNX model with ChaiNNer and an AMD graphics card.
After playing with this a bit more it turns out that this tile controlnet is really good for img2img when you just want to edit the image. It keeps it similar enough to the original image while having enough freedom to do the necessary changes.
I did a lot of test the last 4 days, working the same way as you (but updating the ControlNet reference image every time) and I found that the detail gain is really depending on the models but also of the prompt. Some models are slowly loosing details when some other are gaining amazing things...
im curious what happens if you take the og image and use gigapixle ai upscale on it and compare it to the controlnet one.
Thanks for covering this! Between you and other figures in the space like AITrepeneur, you're doing the great work of informing the SD/AI community on the latest tools and techniques.
The next thing they should be covering is the ControlNet Inpainting model… it obsoletes all “inpainting models”
@@AltoidDealer yee that's great
I went with same settings, but I still see the tiles. I had to take down the denoising to 0.07 toke remove it... but I still can see it in the background. my image is 3:4.. I started in text prompt with 512*720width. Not sure why it's not working for me :(
Thank you a lot!
Good job, Thanks! 👍
You're welcome! 😊
wow this works amazing! ☺☺
Thanks !
thanks you, it''s so cool
Thanks a lot mate! My AMD GPU, the poor soul was so freightened by the other methods that it outputed a wall of text praying to the DMT gods not to do upscales.
Have you tried going into photoshop and trying to apply one image's details (8k) to the (4k) other? Maybe with a highpass or something
Can I ask why you are doing the upscaling on img2img instead of extra? I thought that was that extra is for
Very helpful. Thank you very much.
Happy to help! 💫
fixing seams is always mildly painful, also thanks for the video, its always super informative
Amazing!
Very good Tutorial. Will try it out later :)
I'm sure you'll love it!
Hello,
Very clear and precise, so easy to reproduce, thanks a lot !
Any idea if there is such things as tiled outpainting available somewhere ? (to outpait but with lowres constraint
I'm using the ultimate SD upscale technique for quite a while now and didn't really saw an advantage using ControlNet (yet), but when I go into the upscale steps, I delete the prompts and only keep the quality keywords, like "ultra detailed, 8K, hasselblad" and denoise at 0.15 to max 0.30, and put back random seed, so nothing will influence the details creation.
Every tile is passed through the algorithm separately, that's why if you keep a high denoise it will generate a collage of different images, so random noise, low denoise :)
Try it and let me know ;)
After viewing the video from Olivio Sarikas (czcams.com/video/3z4MKUqFEUk/video.html) I made multiple tests with the tile ControlNet, and it helps getting a better image quality upscaling directly to x4.
Fantastic work, love learning from your videos as well as the great advancements from AI digital design and everything which is possible! I've not seen anywhere and unable to locate this anywhere currently. But are you aware of any AI or any suitable method from turning a 2d design to a 3d game ready asset without using blender?
I'm happy if the result isn't perfect and requires some minor adjustments, but currently looking for a method to effectively reduce the time taken within blender.
I've noticed that if you use the 4x Ultrasharp model, it changes the color of the result slightly. I think the 4x_UniversalUpscalerV2-Sharper_103000_G gives a more accurate colored result.
where do you get this from?
@@shiccup drive.google.com/drive/u/1/folders/15dV3rhMcq_LXg5Vrn7DG6lwo4M6cXt6C
Thank you for all your helpful videos! I noticed you are using Deliberate V2 in here, my current favorite.
I'm upscaling a batch of images for a project and I've been using the _Inpainting_ version of Deliberate V2. Do you know if it makes a difference? I will do my own tests but intuitively I feel it might be more suitable but i'm an absolute newbie so I might be completely wrong here :)
Have you tried iterating the control net? I mean, always inputting the img2img output into the control net for the next stage?
Also, I'd try putting something regarding old skin into the prompt. You know, 'age spots', 'wrinkles', stuff like that. That should help, especially if you go for a bit higher denoise.
Hey, Great Video! Should we be using the upscaled image in ControlNet or leaving the 512x512 image is ok?
this is a game changer!
it works exceedingly well for faces... with the restore face option.... is there anything similar to "restore face" for the entire picture ???
Hires fix is similar. But not really in this case with this upscaling tutorial.
Thanks for great tutorial . Why did'nt you update the denoising to 0 (zero) ? why the value is 0.15 ?
Sorry for my idiot question but what is the difference with the same upscale in extra mode?
What does the tile model do better compared to the ultimate sd upscaler tiling?
How does this compare to doing a normal image to image at a higher resolution and then upscaling normally? Is the benefit of tile mainly for people with GPUs that can't handle the higher image to image resolution and upscaling or does it create better overall results?
I still cannot get Controlnet 1.1 to show the new settings. should I clear the old ControlNet from my system and start over?
Nice! What would be the workflow/change if I don't want to change the resolution but just add more details to a blurry photo?
Resolutionaries...lol, you just got yourself a subscriber. Oh and thanks for the awesome tut :)
Glad you enjoyed it! 😊🌟
How about generating both the stepped version and the straight to 8k version and then blending and masking both in photoshop or photopea? That way you could bring out more detail and still avoid the blurriness in places where it matters.
Oh, say, I had an idea for use with ControlNet... I don't have a channel, so I thought I'd share it with you in case you might like to make a video about it.
The easiest way to create custom poses for use with ControlNet is to use one of those artist's manikins. It seems so obvious to me in hindsight that I feel kinda dumb for not thinking of it sooner. You can create any pose you want, and then easily get any specific angle of perspective on it once you have your pose, a lot easier than posing yourself and taking a picture, because controlling the framing and angle are going to be so much easier this way, plus easier to set up a green/white screen to have no background, etc.
Where has that SD upscaler + control net tiles combo been my whole life (well, my short Stable Diffusion life anyway)?? That is huge. I will never upscale the old way again lol! 😅
awesome, now how do you save these upscaling settings for next session/s?
also, can't remember where you explained outpainting with mk2, which allows you to crop in, and crop out in i2i...?
Tip: To obtain sharp detail at very high resolution, try to copy the previous upscaled image in control net window too! 👍🙂
I did try this but didn't see much improvement on my end. Did yours get better? I might've missed something then.
Two passes 4X + 4X ? good idea.
@@sebastiankamph yes¡ For my experience, results are sharper and more detailed with this method! Can I show you in LinkedIn? (I wrote a post with examples)
@@monitepmoo I'd love to see your results!
@@monitepmoo Sure, just send me a message