ControlNET: NEW Face Tracking Method - BETTER Faces than ever before!
Vložit
- čas přidán 31. 03. 2023
- Controlnet has a NEW Face Model for Laion Face Detection. This can track the Face rotation and face expression. This Install guide for Automatic 1111 will show the Controlnet Install and the Face Laion Model Setup. Use this new Model with Stable Diffusion 2.1 - All Settings and commands are in this video to get you started.
#### Links from my Video ####
LIVE Stream: czcams.com/users/liveXgtdqrsLm8c
Join my Facebook Group: / theairevolution
Joint my Discord Group: / discord
Install Doc: docs.google.com/document/d/1O...
Controlnet Model: huggingface.co/CrucibleAI/Con...
Reddit Post: / new_controlnet_face_model
Mishi Anime: civitai.com/models/7742/mishi...
Realism Engine: civitai.com/models/17277/real...
Support my Channel:
/ @oliviosarikas
Subscribe to my Newsletter for FREE: My Newsletter: oliviotutorials.podia.com/new...
How to get started with Midjourney: • Midjourney AI - FIRST ...
Midjourney Settings explained: • Midjourney Settings Ex...
Best Midjourney Resources: • 😍 Midjourney BEST Reso...
Make better Midjourney Prompts: • Make BETTER Prompts - ...
My Facebook PHOTOGRAPHY group: / oliviotutorials.superfan
My Affinity Photo Creative Packs: gumroad.com/sarikasat
My Patreon Page: / sarikas
All my Social Media Accounts: linktr.ee/oliviotutorials - Jak na to + styl
Who needs Skynet when Controlnet is already taking over.
this.
Groundnet stealthily coming from below, watch it
Skynet is the name of the Chinese AI tracking program. Been around for years
@@DPiii127 The one I refer to has been around for decades longer. 🙂
you are always here to save me though, there was enough data in the video to help me around. now it works perfectly. thanks again Olivio.
Excellent video just like all your content and tutorials! thanks for the doc and hard work you put into this stuff. really well done. I'm excited for 1.5... but I'm playing with the 2.x one now.
I was gonna wait until the proper release for auto1111 but this video was so easy to follow, I couldn't resist it! Thanks for the share 🙂
Question, where the heck did you get all of those controlnet models? I only have the one we addressed in this video thatI found on civitai because the huggingface link was taken down.
Thanks for the new Olivio ;) Really hope that it will also be made available for all those amazing 1.5 models ;)
it is now (since yesterday)
That's pretty cool stuff! It's like watching a software tool like photoshop or blender for ai generation being built piece by piece, in real time. Thanks for covering all the updates.
Going to try to catch the livestream tomorrow. I was curious if in it you could possibly go over any potential routes for using a controlNET option, while creating a consistent character? I've seen some work arounds in mid journey, but haven't seen anyone really crack it in stable diffusion yet.
Cheers
I think for now consistent Characters are very difficult to do, because that's just not what SD or MJ are build for. But we can look into some methods. Remind me tomorrow in the live stream please
@@OlivioSarikas Hi Olivio, you didn't include the repo cloen link in the description. :) It's OK, I already had it, but others might not. Also, I would suggest people check that they do not have their controlnet models in their extension/controlnet folder BEFORe they delete it. early adopters may not have moved theirs to the models/controlenet folder, as this was the way to do it at one point.
Version for Stable Diffusion 1.5 is out now and I cannot figure out how to use it. I put in the the preprocessor but get no preview, just a black box. There are no usable instructions. I see you have a postprocessor LaION face, you don't say how to get that but it's not in my processors or at the download site. Perhaps you could update this vid for 1.5 now that it's out. Appreciate all you do!
good to know it is coming to 1.5 as well
hello, the huggingface doesnt exist anymore :c where can i get these files?
Can't wait to see that on V 1.5 !!
me too
it is now
Hi Olover LOVE your content man ! I'm a long time Midjourney user and wanting to get into Stable DIffusion so I cna use these amazing feautures BUT I work off a macbbok pro thats just below the M1. ANy idea if its possible to use this tech without getting a new machine ? Dreambooth perhaps ? ? Thanks
Control net would get more & more powerful. The very idea of control net, i.e separate model changing activation in sd’s u-net, is so simple yet so extensible to the point it can basically do anything on a diffusion model.
I love you. Thanks for this😍😍😍
Gorgeous
Great news!
Would it work with faces in perspective instead of only front, side, or profile camera views?
(you know, dynamic camera views. I only had half-decent results with bodies in perspective when I use the ControlNet depth model, but AI is still terrible with body members in perspective. So I'm not sure if this new model works with any camera view).
Hi Olivio, any idea why I don't get the image tab and the annotator one? Did I miss some? Thanks you
Thank you.
thank you so much. Excellent
you are very welcome, my friend
there has been a SD1.5 version support update added since you made this.
Thanks for the video, I have a question. I have the following problem: I have a base image of a girl whose face I want to save, and an emotion of another girl that I want to apply to my base image using image2image. Have you had any luck applying the emotion from one image to the base image, via image2image and using the tools in your video? I'm having trouble with this.
I would love to see you make a tutorial using Blender 3d scenes from different angles in combination+lora character models to make consistent manga/comics fast.
not sure there would be a big audience for that. but i want to do something similar
Can this be used for head-tracking in TrackIR and freetrack supported sims/games?
I love this one
me too :)
there is a version for 1.5 model can you tell us how to install it ? the model seems to be released since this morning, thanks :)
Hi Olivio. Have you tried the 1.5 version? Can't seem to find any guide on how to get it working
A bit off subject question. Been fiddling around with the Scribble ControlNet preprocessor and model, not much success. In your opinion which settings and model works best for translating line art to cool generations?
hm... haven't done much with line art to be honest. Maybe the canny model as it tracks edges?
Thanks.
You're welcome
New Tool, yeahhh!!
Party Time! 🔥🔥🔥
Dude you're the literally SD Gurus out here
Thank you so much. i feel honored
I'm confused, all my controlnet models are in my A1111/extensions/sd-webui-controlnet/models folder. But you have yours in A1111/models/ControlNet Mine still works but am I doing something wrong? Also where did you get all those extra ControlNet models from? Thank you for your very informative videos!
ControlNet works with its own models
@@NarimanGafurov Thanks for the response. I understand that ControlNet has its own models it just seemed that he had his models in a different folder than me.
@@jubjub64 I was doing the same as you. Apparently, they can be in either location and they will load into A1111 automatically without any setting changes or anything. You can safely move them. I left all the .yaml files and everything in the sd-webui-controlnet/models folder - I only moved the large model files over to models/ControlNet
does it work with img to img if we want to apply a face to an already existing character?
The link is broken and so my A1111 install... thanks a lot...
Howd you break anything if the link isn't working? Btw they just renamed the model that's why
Controlnet models for 2.1... ah, is that different from the T2I models?
So the models I use for ControlNet must also be consistent with the ckpt model I am using for Stable Diffusion? (1.5 to 1.5, 2.1 to 2.1, etc)
Yes, t2i is a different method, not just a different model number
this would be a game changer if it works with sd 1.5 and we can use it with canny and depth model
First comment 😊. I like your video ❤.
YAY! 😍
NOTICE: You DON'T need to add --no-half if you don't get black images with the 2.1 models
#### Links from my Video ####
LIVE Stream: czcams.com/users/liveXgtdqrsLm8c
Join my Facebook Group: facebook.com/groups/theairevolution
Joint my Discord Group: discord.gg/XKAk7GUzAW
Install Doc: docs.google.com/document/d/1On8tA434WVx1FqDTePez9pm2CWrvAD6pjtAjRCSjWBg/edit?usp=sharing
Controlnet Model: huggingface.co/CrucibleAI/ControlNetMediaPipeFace
Reddit Post: www.reddit.com/r/StableDiffusion/comments/1281iva/new_controlnet_face_model/
Mishi Anime: civitai.com/models/7742/mishi-anime
Realism Engine: civitai.com/models/17277/realism-engine
What a time to be alive
Thanks for the new Olivio ;) , my preprocessor list have no "mediapipe_face", where can i find it , thanks
Hi. nice video.
you should change the link in the google doc (you just have to remove the "/models" at the end to make it work again. I guess the dev changed their folders structure since your video).
How do you get the sd vae drop down on the front UI?
hey man I'm new to this scene ,, so in midjourney the face changes completely so is there any way in sd to photo edit with the faces looking a std lvl similar to original face and also can i create realistic edits or it would be animation looking (like in midjourney's edits)
Yep to a point, using controlnet
Yes canny option in controlnet will lock in the face to a great degree or at least usually. There are sliders to give it more or less freedom to stay close to the original image.
I'm doing something wrong, under Model I can select laion face, but under preprocessor the mediapipe_face is missing. Anyone an idea?
Anyone have an idea why the mediapipe_face preprocessor would be missing but the controlnet_sd21_laion_face_v2_pruned.safetensors does appear in the processor drop-down?
did you restart your pc? are you using the controlnet web ui as i instructed in the video (removing the old version first)
@@OlivioSarikas same thing happened to me, cant find the mediapipe_face
@@aicollectiveq
@@aicollectiveq same here ModuleNotFoundError: No module named 'mediapipe'
anyone know if this can be used for batch?
More control to our imagination!
hell yeah!
Btw if only someone was able to code a plugin for Affinity photo like they have for photoshop.
That interface with stable diffusion
well I might be asking too much but, does anyone know how can we generate same background with different angles so we can use those in say 2d or 3d animation. If you have no idea you can share your opinion I'd be really grateful
to be honest, i haven't even installed xformers yet. i will look into it though
hi there, the model is gone, and I think we need the yaml file for each model.
link is dead
404
Tree Entry not found: models
I am confused I put my controlnet models into the extension controlnet before. Does it matter?
If it works for you, that's ok too. Mine is in models/controlnet
I tried it, as I've wanted to control where my characters eyeballs are pointed, but I didn't have any luck. The eyes are always pointed at the camera/viewer. :/
(Edit: and no I did not have restore faces turned on, which I know can force the eyes to point at the camera)
That's a great point. It doesn't look like it has specific pupil guidance, just eyelid posing maybe
The links are dead. Would it be possible to fix this? Also, can this be used with batch sequences?
you have to remove the "/models" at the end of the link and it will work. I guess the dev changed their folder structure by removing the models folder
will you be discussing the new xFormers method of making images faster? Me - something rather....
Any word on a version of this that works with 1.5?
coming soon i was told
I would think that one hack would be to use 2.1 for this, then take that output, and run it as an input in 1.5
good idea
The huggingface page appears the following message 404 - Tree entry not found: templates
I updated the link. try again
@@OlivioSarikas thanks, but the same message "404
Tree entry not found: models". Is it my browser problem, antivirus, something?
Instead of moving the previous controlnet folder you can rename it
wouldn't A1111 still look into that folder and try to load it? better move it to avoid any conflicts
@@OlivioSarikas Indeed, it needs to be moved.
@@OlivioSarikas You are right. I had to move it out that folder.
So much i just watched, and so little examples of the resulting output. Like, just no results. What did i just watch all that for? How do i know the difference between using what you just explained, and not using it? So lost.
Next time i will put examples at the start again
@@OlivioSarikas i usually skip the start of people's videos. Usually just boring introductions and basic steps. So i apologize if i missed it.
@@AllYouWantAndMore this time I put them at the end of the video
Am I the only one who thinks it's insane that in a world of AI, you have to go through 9,173 steps of downloading, installing, comand prompts, modifying files, restarting, etc., etc. to get it to work?
It's free from the community, that's why. Pre packaged would be the corporate paid for route.
@@RetzyWilliams open source all the way - a good learning curve also usually means a somewhat emotionally mature community. kinda
don't ask how long it took me to figure this out too. i had to write the dev team of this addon and spend at least 2 hours getting it to work. This stuff needs to be easier
Waaaaa?!?!?! but i just finished 1300 frames...rip
Si d'accordo la tecnologia ti permette di fare "ritratti " ABBASTANZA FEDELI ma ti voglio vedere quando li farai rendere vivi. Allora le cose saranno molto più complesse.
We need more inpainting models based on humans only
i agree
“This isn’t on 1.5 only 2.1”
Me: Instantly closes video and watches something else.
your loss. Could have played around with it for free
@@OlivioSarikas I don’t play with trash. When it’s on 1.5 then I’ll try it out.
@@Modioman69 you could be a little nicer, but ok...
@@OlivioSarikas Nicer? How about the fact that I’ve liked and watched all your videos (even left nice comments) even the ones I didn’t like because you seemed cool and informative. This is my first criticism it’s not a d*ck don’t take it so hard.
@@OlivioSarikas I think the context of this comment is a bit mixed. I get why you felt he could be nicer (it's a sarcastic harsh joke he's making).... HOWEVER, he's not actually offending your video or content. he's just making a joke to express how he's more interested in 1.5.
The huggingface link is 404!
I also got a 404 but instead of complaining and giving up I spent 5 mins and figured it out. Using it on sd1.5 now and having a great time
If it was for 1.5 it is news this is a foot note.
Let's see when they do something definitive about the hands, the depth map is not perfect at all.
controlnet model = models does not exist on "main"
This is NOT a Design/Art channel anymore sadly. It's just type in a few words to get pictures. Used to be great working with Affinity/Photoshop but that's all gone.
John, make your own channel like that. Olivio will support you if you do. He has to find his own path, not yours.
@@RetzyWilliams I'm just stating a fact. It's not my path as you put it, originally many came here for exactly what I stated above, not A.I generated stuff, simple as that.
You can still use Affinity Photo and Photoshop to edit these images. Also, if you look at even this video, you will see that it is a lot more than just tying some words. That's like saying photography is "just pointing your camera". You still need good ideas, skill, luck, passion. Nothing has changed. But this is the future and my AP Videos didn't even get 10% as many views and i get now. Got to go with the time
I just want better hands and feet. It's so bad that stable diffusion has people in ai art hiding their mutated monster hands behind their backs.
First
awesome! ❤
Okay face is fine. But hands and fingers always fck up.
this is just for faces
Looks like Olivia is an AI.
I am :)
When I see CZcamsrs spend half the video explaining how to download and install s*** I know they don't really know what the f*** they're doing
Can you explain a little bit more what exactly that "--no-half" option does and why it is necessary? Thanks.
If you don't use it you might get black images with 2.1. But I'm nit sure what exactly it does
@@OlivioSarikas Olivio, I at first blindly followed your advice regarding --no-half. Then, I decided to do a 1:1 test with and without this option enabled. "--no-half" literally doubles the generation time. Here is a side by side comparison image from my test - i.imgur.com/NouGki2.png
does anyone know where can I fine control_v21_laion_face? thank you
when i click generate i get this and nothing happen - Will process 0 lines in 0 jobs. in the cmd window
File "C:\Users\Yazmo-Desktop\github\stable-diffusion-webui\venv\lib\site-packages\gradio
outes.py", line 394, in run_predict
output = await app.get_blocks().process_api(
File "C:\Users\Yazmo-Desktop\github\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1073, in process_api
inputs = self.preprocess_data(fn_index, inputs, state)
File "C:\Users\Yazmo-Desktop\github\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 962, in preprocess_data
processed_input.append(block.preprocess(inputs[i]))
File "C:\Users\Yazmo-Desktop\github\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 338, in svgPreprocess
if (inputs['image'].startswith("data:image/svg+xml;base64,") and svgsupport):
AttributeError: 'dict' object has no attribute 'startswith'
Will process 0 lines in 0 jobs.
where is pruned version ? control_v2p_sd15_mediapipe_face_pruned
this should read the picture, transfere person and environment objects to a 3d sceleton and 3d "room" concretisizing sizes and positions automatically, controll and cut mistakes, place insuficient parts of a model and even add frames for work with videos. and only after that place "skins" to this 3d model, mimics, etc.
this is too flat reconstruction. not very well
ok, you code that and let me know. I will for sure make a video about it. chop chop
Haha I was with you until you said 2.1 :)
The models create pretty cool results though. But 1.5 is coming soon too
@@OlivioSarikas Good to hear, thanks for the info Olivio.
@@OlivioSarikas please make another video when it comes to 1.5
Model on huggingface 404