Hallo: Breakthrough in Audio-Driven Portrait Animation
Vložit
- čas přidán 30. 06. 2024
- Readme / Instructions
drive.google.com/file/d/1lZpX...
#AI #StableDiffusion #TechInnovation #ArtificialIntelligence #DeepLearning #AIExploration #TechEnthusiast #CreativityInAI #StableAIHub #AICommunity #InnovationHub #TechBreakthroughs #AIResearch #futuristictechnology #Hallo #talkinghead #talkingheadvideo #talkingheads
Credits for Quack Quack audio in beginning
• Quack, Quack, Little B... - Zábava
Thank you very much Teacher. I simply don’t have enough words to express my gratitude! Good health to you and good luck in your endeavors!
Thank you for your kind words
If possible, please create a video about MOFA-Video. The creator of Sadtalker is involve in this project
You know it uses pytorch3d. Installing pytorch3d on Windows itself is a problem. I just couldn't do in past but will try again.
Just completing another tool and will look at this.
@@StableAIHub I was able to install then run gradio and can upload video and audio but there is an error if i click RUN. Might be because i am using the latest version of CUDA but im not sure
@@StableAIHubI was able to install Mofa and run Gradio then upload image and audio, but got an error when clicking Run 😥
I am working on this now that MimicMotion is done. If I make it work, you will see video soon. If not will let you know.
what was the voice over you used for the intro?
In video description
@@StableAIHub
not that😂 I meant the voice showcasing hallo 0:06
👀 this I gonna try out but over two hours for a 1 sec audio clip damn. I have a 4060 Ti 16 GIg and hope that makes a difference in process time. Thanks!
I hope 16Gig should be sufficient. Please do post the processing time.
@@StableAIHub sure will
@@StableAIHub Installation steps were easy follow. I did a 11 second audio clip at all the default settings at 40 steps and the process time was about 35 mins. The VRAM usage was about 10GB. The gestures are a lot better and intact compared to SadTalker but the movements are very jittery and the teeth rendering is bad. Seems I have to play around with the settings to try to get it more stable. Any recommendations on the settings?
@@StableAIHub I just ran the output video in a video editing program to reduce the jittering and regain some of the details in the face. The result now close to acceptable but is to try to reduce the degree of the gesture movements a bit and the mouth size so I can see less of the deformed teeth
@@arron122Thanks for the update. try to pass input image which have little mouth open and visible teeth. It might work, no harm in trying.
This Ai needs greater than 10gb of graphics card memory. It will take too long if you have lower than 10gb
Ahh I wish I knew 8 GB VRAM will not be sufficient, I could have bought a different laptop.
@@StableAIHub just continue creating tutorials and if this channel becomes successful you can have any laptop you want 🤑
fingers crossed :)
idk why you have 1/1 for 40 iterations but I have 1/12 there and 8gb of vram too... RIP
I waited 20 hours out of interest... The result is not bad, better than sadtalk
[1/12]
[2/12]
[3/12]
[4/12]
[5/12]
[6/12]
[7/12]
[8/12]
[9/12]
[10/12]
[11/12]
[12/12]
Moviepy - Building video .cache/output.mp4.
MoviePy - Writing audio in outputTEMP_MPY_wvf_snd.mp4
MoviePy - Done.
I would know why 12 steps and 40 iterations. So 40 is okay but where is 12... I got 8sec video...
@@PlanA-fq2im Look at 8.26 in video. You will see different settings like fps, inference_steps etc..
@@StableAIHub I had the same default but yesterday I saw your config and mine one... And didn't find any differences..
export_video:
fps: 25
inference_steps: 40
cfg_scale: 3.5
@@PlanA-fq2im In that case I suggest you post an issue here as developers can only fix it
github.com/jasonmitchell/hallo/issues