ComfyUI IPAdapter Advanced Features
Vložit
- čas přidán 25. 06. 2024
- This is a followup to my previous video that was covering the basics. If you are new to IPAdapter I suggest you to check my other video first.
👉 Download the extension here: github.com/cubiq/ComfyUI_IPAd...
👉 The crop node is part of my ComfyUI Essentials extension: github.com/cubiq/ComfyUI_esse...
00:00 Introduction
00:19 Weighting multiple images
02:49 Prepping images
03:58 IPAdapter models roundup
05:37 Face model
09:49 Timestepping
12:17 AnimateDiff
15:52 Conclusions
🎵 Background Music
-- "Part A" by Alexander Nakarada (www.serpentsoundstudios.com)
Licensed under Creative Commons BY Attribution 4.0 License
-- Last Stop Synthwave by Karl Casey @ White Bat Audio (whitebataudio.com/)
-- Menace Synthwave by Karl Casey @ White Bat Audio (whitebataudio.com/)
In general, watching similar YT tutorials by other people feels like mugging up each setting. Example: "Increase this, decrease that, keep this 0 or 1 and you'll do just fine." But that is not the case with you!
I like the way you naturally interpret how the proportions of different settings affects the image and know the best blend to mix them in to obtain the desired results. Not only do you understand the 'how', but you are also able to 'communicate' it to your audience. I can watch this video four times, and still learn something new from it.
I know how much goes into the IPAdapter and making these tutorials! Really appreciate all of your efforts, because of your work I’ve been able to easily inject my own art into workflows with beautiful results. 😊
agee! brilliant work
Thank you for all your work on this node!
Just discovered your previous tutorial and saw you just released this video, amazing work! Thank you so much, this helps me a ton
Really loved the vigorous experimentation - so many teaching moments. Thank you.
FANTASTIC video! Thank you, I am going to be using this technique quite a bit just to see what I can generate. Thank you again for all your hard work.
The IPAdapter for AnimateDiff is amazing! Thank you for the great suggestion!
I really appreciate the effort you put into these videos! Topics flow together well, and a ton of good information is held within.
Every minute in your videos.... PURE GOLD!! Many thanks and all my best wishes! ❤️🇲🇽❤️
IPAdapter is one of the most useful things I've started using in A1111. My mind is always blown by how well it blends images together.
amazing! can't overstate how helpful your tutorials and tools! are. so many lightbulb moments in my attempt to get comfy with comfy. If you have a Patreon or similar, I'd be happy to support!
Thank you for your great tutorials once again!
Thanks for doing these, excellent tutorials, well put together
Hi , nice to hear your voice again .. Always waiting for your tutorials, they are nicely explained and very informative .. 😊
Amazing stuff! Thank you for another guide.
This is the best video about IPAdapter I've ever seen. Thank you.
Amazing!
Please more of these tutorials! Grazie!
Great job with the demo of time-stamp as well. Think we chatted on element as well about it.
The best tool for SD!! Great work!!
training ipadapter for specific weird concepts will be amazing, waiting for it
you are the best creator on the space. really!
thank you for your amazing work on this!
God Bless You Matteo. thanks for your efforts and pure contents.
you rule! thank you very much for all the effort !
Thank you very much! Very useful explanations!!!
Its very useful and inspirational, thank you very very much!
Another great video, thank you very much
Great work, thank you so much!
Great content. Some tutorials focusing just on IPAdapter + Animatediff + control net would be amazing. You touched on it a little . The ability to use 1 ref image to train style on a video sequence is of great appeal. Cheers
Oooh timestepping looks fun I learn something new everyday with AI. Thanks.
Nice work, Thanks.
❤❤ thank you for this amazing video
that's amazing!
fantastic. I dont know if some one more tell you before, but it look when talk about face, that you return a mirror pose of the original.
awesome 🙂
thankyou, you rock!
It's so good that you made another video, the techniques are exactly what I need to learn and apply. I love your videos, thank you. I've had no issue following along but there is an issue I'm having with 'ip-adapter_sd15_light.bin' not showing up in the 'Load IP Adapter Model' node list. I have 6 out of 7 listed in the drop-down. It seems bizarre that only this one is not showing up. At first, I thought maybe I wouldn't use it and not worry but it will be interesting to use if I can identify the problem with my installation in the windows portable.
Update: renaming filetypes from .pth to .bin solved the issue.
thank you very much..
Thank you very mutch
For the Encode IPAdapter Image node, I've seen other ComfyUI nodes that have a variable number of inputs. I.e., when you use up image-slot-2, then image-slot-3 appears, then image-slot-4 appears, etc. This is a more flexible than having a fixed number of inputs for your node.
to do that you have to install an additional javascript in a location outside the extension directory which is an anti-pattern. When comfyui will support adding scripts from inside the extension I'll surely add the option
thanks for all the work and knowledge shared here >> one question, where did u find this Image crop node which seems very useful avoiding to go back to an external app do to the same action >
I enjoyed the video. I wonder, what changes will happen if I raise the noise value?
Great tutorial. If you're looking for ideas for future tutorials, where would we add LoRAs in the workflow? Can we add control net(s) and if so where in the stream. Keep up the great work.
LoRAs and contronet are working very good with IPAdapter
Hi, great video, Thanks!
What is the proper configuration to make it run with SD XL?
Hi Matteo! As always, amazing video tutorial! In the last chapter (AnimateDiff) in the yellow-brown node called (load images upload) ,in the controlnet section, you have loaded 16 images? You don't show them in the video, what kind of images are they?
Thanks! it's line-art of a short video of me greeting at the camera!
Love IPadapter! Also does your extension works with IPadapter XL as well ?
just saw the rest of the video :)
Thanks a lot for this! I have a very simple question as to how to stop the Comfy UI server and restart it other than doing it manually
hi matteo, amazing work, thanks so much. is there any way to schedule the ip adapters for video? kinda like prompt scheduling?
well there's time stepping, I believe that's what you asking. It's a new feature that was present at the time of shooting this video but it is now.
Thanks for your work. It is a blast working with IPAdapter in ComfyUI! Are you planning a IPAdapter model for the SSD-1B model too?
it already works with SSD-1B but it really burns the images. You have to set the CFG to like 2-2.5, but it's fast indeed.
LCM is not supported yet (and don't think it will any time soon).
On a side note they just released SDXL Face Plus!
@@latentvision Oh didn't know. I will give it a try thanks.
Hey, great video! Any tips on how to increase speed of animate diff using a video and 1 controlnet input on a rtx4050 with 6gb of vram?
since this came out is there an update to Appl,y IPAdapter from Encoded? I get an error on this node every time
Thanks a lot for your work ! I have only one available image slot on my Encode IPAdapter Image node, don't understand why
UP: after updating Comfy + extensions, it's all good !
The output of IPAdapter rendering is poor. Is there any suggestion you can give? I tried using LANCZOS but did not really help. Details are lust and bit blurred. A video about keep up the quality of rendered images would be very appreciated. I'm using 1.5 realistic vision because it delivers great results for me.
Thanks for all your work. Its great.
Hello, have you any idea why the node is red? It result loaded, no failed, but the node of IpAdapter plus is always red, even after a refresh...
What would 'training the IP adapter' involve? Make it better at mapping images to tokens in more specific ways?
Say you want to make a face model but for aliens instead of humans. In that case training would give far better results. Or say you want to describe a Picasso or a very specific art style.
I'm having trouble when I use your canny workflow. It is telling me the size of tensor a must match the size of tensor b when it tries to apply the "KSampler" node. I've deleted and re-applied the "Apply IPAdapter" node, same issue persists. I used your other non-canny workflow, just with regular ip adapter and it works. It's something with the canny node that's messing it up and I don't know what it is.
please add a link to the workflows in the description, it would make things much easier...
I love you
damn... offer me dinner first at least
Thanks for making this tutorial, I have a question though, in my case, I am getting attn_mask (one more input) in the apply ipadaptor, any ideas, why so? Has something changed in the IPadaptar nodes and what it is expecting as input?
this technology moves fast! have a look at my other videos! 😄there's one about attention masking
Thank you very much , I am looking at one by one. Thanks again for great videos @@latentvision
I didn't find the encode IPAdapter Image, is it obsolete?
Could you please clarify the controlNet setup? You say at 12.34 you say that you are: "loading a series of images that I resized, and then passed through a lineart preprocessor...". Could you please explain what those images are? Thanks :)
I took a video of myself saying "hello", I passed the first 32 frames of that video to a lineart preprocessor so to have the "skeleton" for the animation.
Hi good tuto but waht about ratio ? it's only square image or can we create portrait or lansdcsape ratio ?
you can do any size you want. only the reference image is cropped to a square. but your generated image can have any aspect ratio
Any particular reason so insert the IPAdapter AFTER the AnimatedDiff Loader? Is the order even relevant? I'll try to test it the other way around as well, to see if 'adapting the style' before AnimDiff does it's magic makes any difference.
It shouldn't make a difference as everything happens in the KSampler anyway. Technically speaking IPAdapter patches the attention of the main unet so I'd put it at the end.
Little question, can you maybe say something about "full face" adapter?
it is called FULL because it takes the whole 256 vector space to describe the face. As a comparison the plus models use 16 tokens. The result should be a very detailed face.
is there any demos about advance ipadapter features？
check my more recent tutorials!
When i try loading the weighted workflow I'm getting this error:
Error occurred when executing IPAdapterApplyEncoded:
Error(s) in loading state_dict for Resampler:
size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]).
File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 338, in apply_ipadapter
self.ipadapter = IPA(
File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 179, in __init__
self.image_proj_model.load_state_dict(ipadapter_model["image_proj"])
File "C:\Users\Shadow\Desktop\ComfyUI_windows_portable_nvidia_cu118_or_cpu\ComfyUI_windows_portable\python_embeded\lib\site-packages\torch
n\modules\module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:
\t{}'.format(
I have an error, using the "from encoded". Error occurred when executing IPAdapterApplyEncoded: Error(s) in loading state_dict for Resampler:
size mismatch for proj_in.weight: copying a param with shape torch.Size([768, 1280]) from checkpoint, the shape in current model is torch.Size([768, 1024]).
Do you know what could it be? Thanks :). Amazing tool.
you are using the wrong image encoder
@@latentvision No, I realized, that I didnt activate (plus-true) in the encode. Funziona, grazie mille.
Good work !
sorry, where can i find 'batch images' node ? from which node packs i can install it , i don't have this node , how to get it ?
Should be at Add Node>image>Batch Images
If you don't find it, you maybe need to update Comfy. It's part of the main build not a Custom Node AFAIK.
thank you , second thing , both 'encode ipadapter iamge' and 'apply ipadapter' methods not working , should i use 512square also for the output? i use other resolution for output, if resolution not problem i should start find why it's pretend to not working at all, it loads and the out put is only likke t2i at the end, whole ipadapter pretend to not work with me , without any error, it just the out put exact same like t2i like there's no any ipadapter ..@@joeduffy52
i am using the CPU , no GPU here , just testing and practicing , it just worked 'encode ipadapter iamge' after a restart os ! then stopped again !
Where I can get that IPAdapter_image_encoder_sd15?
I am looking that for ages anyone please help
what vga do you use?
4090 but the generation is almost always sped up
@@latentvision thank you!!!!!!
Where is the workflow of used in videos
I'd love to meet you, I'm also italian
you probably wouldn't like me 😄
What if anyone would want to make something other than faces? Say cars, interior designs, parfume bottles, buildings or something like that?!
the model is trained on A LOT of data so there's a good change it would work, at least for the standard stuff. If you need something very specific you'll have to train it
Extremely good Respect 👍
Thank you!