Free Hand Tracking Without Leap Motion
Vložit
- čas přidán 10. 07. 2024
- ★★Pls Sub To Mari Yume★★
czcams.com/channels/riY.html...
★★Join My Boba Club For Exclusive Channel Perks★★
/ @mariyumech
Hi, I'm Mari and I like to make videos about video games, vTuber stuff, and skits/stories!
•☆✼★━━━━━━━━━━━━━━━━━━━★✼☆•
【 Social Media 】
♡Twitch► / mariyume
♡Twitter► / marivtuber
♡Tiktok► / marivtuber
♡Discord► / discord
♡Reddit► / mariyume
♡All Socials► beacons.page/mariyume/home
♡Q&A ► marshmallow-qa.com/marivtuber
----
【 Support me? uwu 】
♡Donation Link : streamlabs.com/mariyume
♡Patreon► / mariyume
♡Merch► mariyume-shop.fourthwall.com/
----
【Twitter Hashtags】
General: # MariYume
Art: # MarisDreams
Meme: # MariYumeMemes (or use the Reddit above)
Live: # LiveMari
----
WE FINALLY CAN GET HAND TRACKING FOR FREE! BYE BYE LEAP MOTION!
Today, I will be showing you an AMAZING web browser app called Kalidoface. This app is a great and FREE resource for VTubers because you can get hand tracking on your 3d model. No leap motion is needed and and it is completely FREE! Start using hand tracking today as a 3D Vtuber with Kalidoface! I hope you enjoy the tutorial bobas!
Go to 3d.kalidoface.com/ to bring your avatar's hands to life.
This app is developed by Rich ( / yeemachine )
•☆✼★━━━Time Stamps━━━★✼☆•
Intro 0:00
KalidoFace Development 1:03
KalidoFace Summary 3:30
How to use KalidoFace 4:12
WARNING 6:29
Testing on different Browsers 7:09
Cons about KalidoFace 7:30
Setting up OBS 8:51
Comments, TY FOR THE 2K! 11:37
•☆✼★━━━References for Viewers━━━★✼☆•
[PNG Tuber]
Discord StreamKit (Free) - / discord
Discord StreamKit Tutorial by Jiinh - • Video
[2D VTuber]
VTube Studio (Free/$15 USD) ► denchisoft.com/
PrPrLive (Free/$9.99 USD) ► store.steampowered.com/app/12...
Facerig ($20 USD) ► store.steampowered.com/app/27...
Live2D Cubism ($20 USD/month) ► www.live2d.com/en/
[3D VTuber]
VRoid Studio (Free) ► vroid.com/en/studio/
VSeeFace (Free) ► www.vseeface.icu/
3tene (Free/$20 USD) ► en.3tene.com/
Luppet ($50-$1k USD) ► luppet.appspot.com/
Luppet (free trial)► booth.pm/ja/items/1196507
Wakaru (Free) ► store.steampowered.com/app/87...
Leap Motion► www.ultraleap.com/
Leap Motion Clip (3d Print)► www.thingiverse.com/thing:167...
Vdraw (Free) ► sites.google.com/view/vdraw/
VMagic Mirror (Free) ► booth.pm/ja/items/1272298
Virtual Cast (Free) ► store.steampowered.com/app/94...
Tracking World (Free)► deatrathias.net/TW/
LIV (Free)► store.steampowered.com/app/75...
SUVA (Free)► guribo.itch.io/suva
Booth (for VRoid Clothes)► booth.pm/en/search/vroid
[Illustration Programs]
Clip Studio Paint ($49.99/$219 USD) ► www.clipstudio.net/en/
Photoshop ($20 USD/month) ► www.adobe.com/products/photoshop
Paint Tool Sai ($50 USD) ► www.systemax.jp/en/sai/
•☆✼★━━━Equipment Used In Video━━━━★✼☆•
OBS - obsproject.com/
SLOBS
Elgato Wave 3 Mic
Outro music done by / heyitschhay
Background Music by www.thatzitimusic.com/
•☆✼★━━━━━TAGS━━━━━━★✼☆•
#vtubertutorial #vtuber #mariyume - Jak na to + styl
Thank you so much for watching! Please consider supporting me on Patreon so I can make content full time!!! www.patreon.com/mariyume
Wish you would have posted the OBS code here in a pinned comment.
Singing portion is 100% perfect
NOOOOOO, NOW I HAVE TO SING MORE XD
@@MariYumeCh Yes! Sing more please
Just some clarification about VRM for the 3D ppl: VRM is just a general file format for 3D avatars, it's very simple in features made to be compatible with most 3D avatar softwares and VRoid uses it for it's large compatibility. Also, "converting whatever file it is to a VRM" should not be a problem, actually that's what most 3D character artists do for vtuber characters, the file is a .fbx and the rig that you will use for the .VRM format is a basic humanoid rig, it is the same you would use for VRChat or other games because it is a Unity standard, so technically there isn't much of a difference between a custom model's rig set up in Unity and what VRoid is outputting.
Very well said winwin, I did not realize people were confused on what a VRM file was!
This helps a lot thanks
@@MariYumeChhave you ever heard of XR animator, which is another web-based tracker but you can also install it on your computer as a web app, or vnyan, which looks like a really interesting 3d Vtuber thing?
that intro brought me back to the late 90s when commercials had good songs with real bands playing them thank you. Oh and thanks for the video as well.
Thank you for bringing this into my attention, this way i can check the hands of the models I'm making without having to invest in a leap motion ( assuming i can get the thing to work) though i am planning to get one regardless
Mari you absolutely ROCKED that jingle!! >:O would love to hear more, you definitely have something going on.
More to come! I just need to adjust some lyrics I have planned hehehe
Your video quality is insane you deserve a lot more subscribers!
💪🏾💯🙏🏾TY I haven't checked your other videos yet but I'm pretty sure you have one covering how you capture your audio
Hello Enter, I do not have a video on that quite yet, but I do have one on how to make a reactive PNG which reacts to your mic in OBS/SLOBS czcams.com/video/y9pXbl1TBHs/video.html&lc=Ugzt7Xr6VlNN792RUUt4AaABAg
THIS JUST BLEW MY MIND!! Thank you!!
I tested it for several days, it's a good free choice for vtubers with lack of tracking stuffs, but it's a bit frustrating when it takes up more CPU usage. I stream Tower of Fantasy along with Kalidoface on, using my usual settings -- and lags a lot in some cases. My viewers told me that they've been seeing frameskips at times. Haha.
I'd like to see where Kalidoface is heading to at this point. It has promising potential, IMO. My PC is somewhat decent one actually, so that's kinda frustrating when it happens 😅
Also thank you for the awesome tutorials, Mari. Been watching some of your tutorial vids on vtubing, and I like my progress so far 🥰
Hope you don't stop creating contents like these, it's useful for vtubers starting out there.😁
Btw thank you for your videos, helps me to build up my vtuber inspiration
This web based software is neat, but my usage of it was very slow based on the movement of the mouth and hands so I'll be using something else for now. Hopefully he can improve on the speed of its animations. (Edit) I did try it on my iPhone though as far as the 2D mobile web version and it easily works way better than the desktop web browser version.
I know he is working hard to keep updating it, I look forward to seeing his progress
dude... this web is so great. I never tought is I would get a free hand tracking. Now I only need a good camera only. Thank you >,
I saw this video while I was setting up my Leap motion tracking gadget. NNNNOOOO!!! At least it wasn't super expensive. And now I can compare Kaldiface with Vseeface + Leap and see how they differ. Great video. Thank you!
Great tips and info. I'm just getting started with Vtubing and I will gladly try something like this.
This was really helpful, thank you!
Thank you for this. I’ll be checking it out for sure
This is the command you put in your windows RUN program to force administrator privilege's to OBS so it can use your webcam with Kalidoface! It is basically the command path of where your OBS .exe file is located in your windows folder. Usually somewhere in your program files. You take that path and specifically type in the code that I have pasted below and it will work :) I hope this helps!!! 😅😅😅😅😅
C:\Program Files\obs-studio\bin\64bit\obs64.exe -use-fake-ui-for-media-stream --allow-file-access-from-files
Of note, you should be able to add these arguments to your OBS shortcut in windows, so you won't need to use the command prompt every time. Directions for this can be found here (along with a host of other sites if you want to search):
www.digitalcitizen.life/shortcut-arguments-parameters-windows/
Thank you so much for sharing this!!!
Alright, now I've gotten OBS (never used it before), transferred the scenes, got everything set up- used your same method to the exact letter and it isn't working...
@@travelerwatanabe same. Adding C:\Program Files\obs-studio\bin\64bit\obs64.exe -use-fake-ui-for-media-stream --allow-file-access-from-files (my file path is the same) to Win + R and I still get the message that OBS can't access my Webcam
OMGGG THANK YOUUUU
Every time a new Vtuber starts CZcams a dolphin dies with pain.
Your singing is so beautiful and I love your channel 💗
wahhh thank you so much uwu
Don't know if this helps but I'm a VStreamer who uses glasses and basically when I was testing the most recently updated version glasses are alright with face tracking and eye tracking as long as ur not too far away from the webcam, but if your using full body tracking or move far away it may glitch out, also there's a funny thing where if you put ur hand up infront of ur webcam and hide the rest of your body the model dances around in a weird way and it's so funny to watch. PLUS if your far enough away and you sit down or squat while using full body tracking it shows the model sitting down too and when u walk forwards or backwards the model moves closer or further away, so much fun and so helpful whether ur 2D or 3D like myself
That amazing song got me
New subscriber
Loving the jingle
The singing was a very catchy way to start the video \o/
amazing video, really appreciate it!!
there is an interact option for slobs! if you right click on the source tab itll bring up a menu, and within it there is interact along with properties and so on
At the time of this video being made, there was not, but maybe now with the updates there is. I suggest following richi's twitter!
Did you find a way to turn on the face tracking ? SLOBS can't access to my webcam even though I started SLOBS with the same parameters as mentionned :(
Trying to become a vtuber needed something like this this video helped a lot
Im happy to hear that Lawrence!! I look forward to seeing you on your vTubing journey!
@@MariYumeCh thanks for the response.great video keep up the good work also 😃👌
I honestly would love it more if it was an app rather than a browser.
But the work is exceptional
"My ADHD brain could not resist making a jingle" Haha I can relate to this 100%. It just makes me like the intro more than I already did. Your singing voice is not bad at all. I really like what this has to offer. Sadly it won't work for me though. Between needing blue light filtering lenses and my webcam not being good enough, its simply not doable for me. I do hope this takes off. Its so much better than VSeeface in spite of its quirks.
Thank you so much for this video.
That's so cool I might take a look at it!
You deff should!
Thanks so much for this gem of a tip on vtubing! I subbed just now, btw I wanted to let u know that maybe u should consider timing ur text slower. If they change based on the amount of time actually meant to reading them it I wouldn't have to pause as much.
So, I just thought about how waiting too long for each text to skip would have u stuck in silence if ur ready to move onto the next subject. So maybe just saying them out loud would be better (not every side note)?
LMAO THE FREECREDIT SONG
I am starting to belive vtubers with a 3090 is not a waste afterall...
Neat, thank you for this great video! I was wondering what program is best for people with glasses
That intro was the best
The song is now my wake up alarm
That legit makes my day hearing that XD maybe i should make another parody song
Great video! Very helpful, and love the editing.
Im just glad to help!!!
Thank you. I guess, at the start of Vtubing I should stop at being without the hand tracking because I don't have lots of many or strong PC. I just want to try and maybe I'll take another decision later
thanks for reminding me that song exists.
Thank you so much at some point i will use the 3d version
THANK YOU SO MUCH OMG
If you're gonna keep that in a notepad, you might as well make it an executable of some kind lol.
That is very smart XD I had to rub my two brain cells together to figure out how to do that lmaooo
@@MariYumeCh yeah tbh when i made this comment i actually forgot *how* to do that, just knowing it was possible lol. I think it would just be making a bash script?
you can actually just get these arguments and add them after the path to the executable in the actual OBS shortcut
Thanks this was helpul
This is a game changer!
I love you THANKS !!!!
"I don't have any friends"
That... That can't be true xP
This is literally so fucking cool
IKR!! I can't wait to see what other tech comes out for vTubers
hey, this is neat, could you do an update video since it’s been about 2 years? It seems like your special interest and you sounded really excited about it so i wanna hear more about it from you ^^b
OMG THIS APP IS AMAZING
as of 5/20/2023 i couldnt find the coded tweet he made on july 15th like your video described
Just tried kalidoface 3D and my pc immediately crashed and the result isn't too different with the 2D version (only froze the browser till I closed the window) do you think it can get better if I simply download a more known (still free) face tracking app or I use a non tracking option?
Me who just invested in leap motion: :D
Your mouth tracking is on point. What did you use for mouth tracking?
Welp. I'm definitely trying this out. And hope fully I can resist gesturing the "Just Do It!" Meme while on camera lol.
I do have a question, as of right now I only use VeeSeeFace to get my character shown on stream. And of course studio to make the avatar. Does the Web Program add more tracking capabilities like if you were to use that IPhone Camera stuff to make your VTuber face move with realistic motion? Or is it the same default face you picked for your model? I still use it, but the inability to at least shut my VTubers eyes halfway or smile without a button can be a bit meh at times.. It really makes me want to go even further beyond for higher tracking quality immersion.
Came back as an Update on my experience so far.
I tried it out, it took me a bit to get the tracking working primarily because I never use my camera for Web stuff and needed to find the options. So far, it's just "OK". It IS a free application or program for what is supposed to be a paid experience normally, however my PC could not run it too well even through Edge (Runs Slow) which worries me for how my PC will handle a Leap Motion. The finger/Hand tracking is a bit off and needed to raise my hands far above my head just to get the hands in a waving position. Still, only eye tracking is working while eyebrows, and mouth curvature is for some reason is a viewable/visible tracking point, but not active which confused me a bit and made me think I was missing something as if they should work since I can see the tracking points.
I was looking at this as an option to not have to buy a Leap Motion since I have the money to get one, but not only am I still looking to buy it, now I'm worried that the tracking won't be right at all or even worth the purchase if it would look jankier than actually a nice addition to stream viewing.
Though the body tracking may seem like the main point for this free program, what I DO really like is the possibility to be able to make more than one face eventually, which really puts it in a better light when compared to VSeeFace where button presses dictate what emotes you're doing as you sit there with the same blank expression depending on what you set for it. Also, the ability to "collab" with a buddy of mine without having to capture a very cropped Discord Window sounds nice too, though P to P is always a bit meh, but that's only because my buddies net is God awful lol.
I am pretty excited to see what updates come for this program.
ur voice is cute and tq mari :D
awww thank you so much
hey! don't bully yourself! you were rolling there!
wooowei hehe im try to become a vtuber on a chrome book this is a big help
I love that intro lol
Your like my abang hafiy bruhh at the start... the singing XD
This is really cool and it's a huge help! You just saved this cringetuber a lot of money! Thanks! :)
THAKSSSS I CAN USE IT
great videos!💚
yeah its a big WIP and wouldnt work out for me and most of us, especially the lack of expression detection and janky tracking overall, and the fact that I cant really put my webcam that far
I just have it on my monitor(its a C920 btw)
My PC is in the low-end probably but in OBS itself it doesnt even use like any CPU, the whole OBS uses like ~10% which is normal, and when I usually use VSeeFace then it itself uses like 20% and then add the 10% from OBS, 50% from a game like DBD and ~5% on Discord(8-20% on BetterDiscord, had to uninstall it) and I get lagspikes all over the place so too bad I cant really use it as it would actually help me which is quite bizzare
Also is it even possible to hide the UI of the app?
And yeah another weird thing is that there are no camera settings for the app, does it just inherit all of them from like a camera driver or smth do you think?
Set Browser Content process count to 1(Or else you'll have a huge amount processes in the Task Manager with similar name And don't use more than 1 Tab(1 with kalido in it).
Try messing with process priorities(Found in Task Manager). Also there is an option called Set Affinity(Also in Task Manager) which should allow you to set the number CPU cores the process would run on.
Also may I know your CPU model.
i saw this and i was going to make a port for linux since i can't run live2d or Vroid on a linux machine, the idea is to use the base technologies and use pixijs (which is what he uses in kaliedoface), and use his framework to map motion to 2d graphics, or otherwise come up with my own open source option if i find it more expedient would be cool to have a 3d avatar but 2d i think will work great for what i'm doing
SLABS will get it eventually (the interact button) because all they do is grab the official open source OBS and then edit it into SLABS
amazing
Mari: I can't use this cause I don't have any friends Me:Aren't your viewers your friends
i'd still rather get the leap motion because it looks more fun :)
Ohh soo good
That technical thing you spoke of near the end? Wasn't that just running OBS as administrator? Couldn't you simply right click OBS and run as administrator in the options?
You could even go into properties after right clicking and set OBS to always run as administrator, which I always enable on my PC and my clients when setting up their stream software.
hewwu mari~~~
i need help. i cant use my camera on this websites since it keeps on saying "Please allow camera for facial tracking." although i already allowed it
I got question, so when i open that page it doesnt load my character , it doenst load any other characters eighter, when i pick Mono cat character and restart page it works normally as long i wont change the character, i also tried picking character and reseting page but it seems liek Momo cat is the only character loading in, and yes my character is Vrm and the others are characters are samples like Sample A, Sample C, VAL, 9 and others. Do you have any idea whats going on?
i saw this program was free and i was so excited but then she said you need a strong pc and i was like MAN
Wow, definitely unninstaling the app i had, if i knew about it before i would have way less trouble
They really need to change to a solid color for 3d models on kalidoface... the lag is rediculous. seriously impossible.
Okay. So I once i get to the area where I have to put in the code and hit run. When it opens up OBS I am still not getting the correct page to pull up? Not sure why.
ur model is cute. where did u make it.
7:15
Me who uses Microsoft Edge: LET'S GOOOOOOOOOOO
This is all great. But I am having trouble. It's maybe because I wear glasses most of the time. The movement is head great, works really well. But for some reason, my model's eyes are really twitchy. I tried without my glasses but the eyes were still shaking. Even tried one of the models that came with it. The same problem. Do you happen to know what may be the problem or is that just a normal thing for now?
p.s. There is an interaction for Streamabs but you need to do that link thing you where talking about 10:10 . Honestly, I don't quite understand, I can't seem to find the thing you said about Rich's tweet that we need to add. I'm not use to Twitter T-T
I love u Irl❤️
i'm surprised how much you know about kaleidoface Mari, it seems you really did you research
your singing is :o some
You can create a shortcut to that path you're copying into Run app. Just create a shortcut and add the same path to it and just start it instead of copy pasting it every single time.
Big brain right here XD thank you!!!
It seems to track everything well as of right now while using my phone, so I might try to set it up while using VSeeFace and pray nothing explodes.
(I wonder if we still need to add the privileges code 😅)
wow first on the video Thanks Mari
Congrats Moonlight! I feel like it takes a lot of courage to be the first comment, but once you do it's like a snowball affect and it makes others want to join in on the conversation too!
@@MariYumeCh yeah it does take a lot of courage. But it’s cool for sure
Me making my own 3d models to keep my potato running smooth(college pc)
how do i record with it on any games? ex; i want to play a game w the thingy but i cant record it like- idk how to explain-
OMG YOUR ARE AMAZING ITS ACTUALLY ON ANDROID
BROOOOOOOO MY CPU IS LITERALLY BURNING, anyway nice video -_from my phone because my pc is exploded_
I got it to connect but it changed my character back to the original one the website shows. If i try adding the file while interacting with the website through obs, it wont show up in my files.
So do you have to rig up your Live2D Model so that it can track your full body?
Right now Live2D doesnt have full body tracking (but its being worked on) you can use your 3d model for now :)
I want to use this on my phone but I don’t know how to save the model so it is the right file type. I made the model in my pc but my pc is too laggy to run this website on so I have to use my phone. It works great, I just don’t know how to add MY model. Can anyone help me, please? 😭
I'll keep looking into it but for now it's so laggy I can't even do anything with it. :( Is it my Browser settings or?
Hi, when I select a sample character I dont see it. Is this normal? Sorry if this sounds rlly stupid :(
Cant you just right click on OBS, Go to properties and click on give administrator permiissions or something like that
Nooooooo I have glasses now what I 😭
would it be possible to export bvh or fbx?
On my computer, my vroid model is all laggy and jumpy...
Does this idea work with built in webcams?