Installing Wav2Lip-HD with Real-ESRGAN for high quality lipsync on Windows (August 2023)
Vložit
- čas přidán 29. 08. 2024
- How to install and use Wav2Lip-HD and Real-ESRGAN for higher quality lipsyncs with upscaled video.
All of the files used in the video: github.com/nat...
Music:
Evening Melodrama by Kevin MacLeod is licensed under a Creative Commons Attribution 4.0 license. creativecommon...
Source: incompetech.com...
Artist: incompetech.com/
---
Comedic Juggernaut - Music to Delight by Kevin MacLeod is licensed under a Creative Commons Attribution 4.0 license. creativecommon...
Source: incompetech.com...
Artist: incompetech.com/
Thanks man, I got it to work. I run it on WSL 2 Ubuntu cause it didn't work on Windows. Had to change some things in the code due to obsolete versions of some modules, but now it works so again, thank you very much.
Very helpful video, best I've come across as I struggle a bit on the coding end. Thank you!!!
thanks! glad it helped!
@@Natlamir it looks like the Wav2Lip-HD already has RealESRGAN, just curious why did you decide to comment out those two lines and approach each step separately?
Is there a way to make the wav2lip sync and RealESRGAN upscale all happen in one command?
Cheers!
@@Natlamir Hi, I recently acquired a laptop with an Nvidia RTX 4060 GPU, which has 8GB VRAM. While attempting to run Wav2Lip-HD with Real-ESRGAN, I encountered an issue. After executing 'conda install -c conda-forge opencv,' I got stuck at a "solving environment" message. However, switching to Python 3.8 for creating the wav2lip-hd environment resolved this problem. Now, I'm facing another issue when running 'run_final.sh.' I receive an error message, and the command prompt disappears, showing:
"(python.exe:19688): GL ib-GIO-WARNING **: 05:51:20.131: Unexpectedly, UWP app 'Clipchamp_2.2.8.0_neutral__yxz26nhyzhsrt' (AUMID 'Clipchamp_yxz26nhyzhsrt!App') supports 46 extensions but has no verbs."
Do you have any suggestions for fixing this?
@@KimJongSNES the opencv never installed for me when done with pip, i always had to install that through conda, so i had been removing them from the requirements. there is a wav2lip extension for automatic 1111 which has built in upscale functionality. i also made a video on that one.
Thanks for the video but what's the point of this? Wav2Lip has been available in auto1111 as an extension for more than a year and within auto1111 you can use multiple upscalers, within the webui. This is just a command line alternative, the wav2lip model is exactly the same.
I got it to work. Had to pip install a few things as I used conda to do everything up until the run_final.sh and got errors where i used pip to install the missing stuff. Real ESRGAN is slow as shit on my 4070 though, lol. Thanks for doing the vid.
Any workaround on this? Im having an issue in the inference_realesrgan.py file when it tries face enhancement
you are legendary
The 'Solving environment: |' is running infinitely.
conda install -c conda-forge opencv
Can be made into a one-click integration package
How about a video running it on Win7 Python 3.7.6 with CPU?
Error messages wgen i run run_final.sh and git opens
cool is this better than sad talker or do you use your videos or pics
I haven't heard of that one. I just checked it out, it looks like it has an Automatic1111 extension. Cool, I will definitely check that out. thanks!
@@Natlamir sweet i just opened auto1111 it has wav2lip thank you and colorize old photos thank you for the tip
Getting this when installing the requirement: ERROR: Could not find a version that satisfies the requirement tomli==2.0.1 any ideas?
thats strange, i didn't get this error. can you try installing that package in the conda environment using something like: pip install tomli==2.0.1
@@Natlamir yeah i tried. Ive tried pretty much everything. Think ill have to do i clean install and start over🤷♂️
@@swedemason I had the same thing, I just ended up doing outside the environment in python 3.11. Just pip install all the requirements without versions and it worked. Good luck ;)
I'm getting this error after running the SH Source file (run_final.sh) through the Wav2Lip-HD conda prompt:
[main 2023-09-23T18:59:15.492Z] update#setState disabled
[main 2023-09-23T18:59:15.495Z] update#ctor - updates are disabled due to running as Admin in user setup
[main 2023-09-23T18:59:25.984Z] WSL is not installed, so could not detect WSL profiles
[2648:0923/150119.515:ERROR:gpu_init.cc(523)] Passthrough is not supported, GL is disabled, ANGLE is
Do you know what I need to do to fix this error?
i found some others having the issue on reddit. does it let you hit enter and carry on with the rest of the process after that message? check this out: www.reddit.com/r/electronjs/comments/qdauhu/passthrough_is_not_supported_gl_is_disabled_angle/
@@Natlamir hitting enter just prompts me to enter a new command again
@@CDusk interesting. can you create an issue for this on their github page? maybe a developer may have insight on this issue.
ever figure out
the ValueError: --face argument must be a valid path to video/image file ?
i think i have seen that maybe when the folder path or file name contains a space perhaps?
i dont know who no make a pc version, a bat start button y huala! da clone voice .index have a pc version
I get following error 😭😭ValueError: --face argument must be a valid path to video/image file . What i am doing wrong?
it sounds like the path to where the file is located might have a space or invalid character? or there might have been some update to that repository that is causing issues now that weren't there before in August. I am going to work through the previous videos and this time fork the repositories to prevent issues that might get introduced with new changes. in the meantime, wav2lip is also available as an extension to automatic 1111: czcams.com/video/AVZ8KF-0fyk/video.html
There are links to all the files you downloaded, can you share them too? It was a very useful video, thanks a lot!
I said that because, I can only find wav2lip.pth ones
Here is where you should be able to get those files: github.com/natlamir/ProjectFiles/tree/main/Wav2Lip-ESRGAN
Also, check out the other video about wav2lip extension for Automatic 1111, that may be a better solution than this one:
czcams.com/video/AVZ8KF-0fyk/video.html
@@Natlamir It gives an error when we upload another video. Have you tried this?
@@oykuparlak988 It has been a while since I tried this. I have been using the automatic 1111 extension, and the DINet, which i like too. I will try this out again at some point to see if a fresh install results in issues for me too.
@@Natlamir Thank you very much for your return. I am trying to lipsync the audio and video I send to the server and display the output in a project. Therefore, would you suggest me how to do this in the most trouble-free way? I work on Fullstack and this topic is very new to me.
Could this technically be used with the Gradio WebUI from your other video?
Should be possible. I am not sure if the wav2lip hd can be run on python greater than 3.6. And not sure if that shell script might introduce some complexity. But in theory it should be possible. Someone somewhere might have forked one of those projects already with a Web UI I wonder.
I get as far as run_final.sh and it simply opens the .sh file in text editor
Sorry for the newbie question, but why can't you just install everything from the required.txt?
good question, it might work for other people as it is in requirements. when i tried to install the requirements.txt as they were, i would get errors when trying to install those items, so i ended up using conda for opencv and the pytorch website command for the torch.
also the *pretrained(DOT)state* file you can find in the go-ogle co-lab notebook
I installed it but when I put or give the command "run_final.sh" it runs a text file while the file is an extension is sh. I think this file needs to be converted in the base sh file. please help how i can convert it
it sounds like .sh files have been associated with notepad perhaps. if you right click the .sh file and select Open With, and choose open with then select Git for Windows.
@@Natlamir hi, i installed and your suggestion worked for me but i am facing a new problem when running command for enhancing it is giving following error
"Error CUDA out of memory. Tried to allocate 15.82 GiB. GPU 0 has a total capacity of 23.99 GiB of which 0 bytes is free. Of the allocated memory 21.37 GiB is allocated by PyTorch, and 13.42 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF". I have RTX 4090 and it has 24GB of ram still its need more memory
and it is too slow, one another question does python 3.6 installation is required to create virtual environment or can it run python 3.10
Please reply I need your assistance
@@ChatbookSummary there might have been changes to the code or dependencies since I last used this when I made this video. I will plan to look into this one again and see if I can get it running on my 3060 with 12GB RAM, and will see if it still works for me or not with the latest code.
where do you get that text to speech voice from?
i use Eleven Labs for the voice.
I'm getting an error
import cv 2
Are you doing that through the conda command? And deleting those from the requirements.txt for opencv? I also get errors when trying to use cv2 from the requirements.txt. Otherwise, what is the entire error message?
@Clasics.z are you able to install open cv with this command: conda install -c conda-forge opencv
@Clasics.z there must be an existing one somewhere. i will look around. else i can look into creating one
there is a automatic1111 extension should be easier to use
great! i will check that out and give it a try. Thanks!
Which one should that be ? Sadtalker or so that's something else.
Ok cool I found thanks that I was looking for 🙏
@@ratside9485 I think automatic1111 has sadtalker and also wav2lip built in through extensions. i had try installing them but kept getting errors. i will try to install those from a fresh stable diffusion install, but those extensions within automatic1111 should make this process much easier.
It worked for me via Auto1111, but I always got an error message first. Until I checked that you have to download the models separately. @@Natlamir 😅
Is there a colab page for this?
looks like there is a colab here, i haven't tried it out myself: colab.research.google.com/github/indianajson/wav2lip-HD/blob/main/colab.ipynb
has anyone been able to actually used this collab notebook? I keep timing out whenever I try to upload the inputVideo...
@@TheFactDepartment it didn't work for me
Can it run on amd gpu
I am not sure, but I have seen reports of issues of AMD on windows having issue with pytorch
Dude, if you create a guide, do it right or leave it.
cool will do, thanks!
test
Worst instructions I've ever seen! sheesh