How to get Face Tracking in VRChat
Vložit
- čas přidán 2. 03. 2022
- *All links and downloads below* also note this video is fairly outdated now
Sranipal 1.3.1.1:
hub.vive.com/en-US/download/V...
Mirror:
drive.google.com/file/d/1Wt9M...
VRCFaceTracking GitHub:
github.com/benaclejames/VRCFa...
VRCFaceTracking Wiki:
docs.vrcft.io/docs/intro/gett...
3d printable face tracker mount for the Index (you'll also NEED a USB 3.1 adapter or else it will not work):
www.thingiverse.com/thing:533...
smile.amazon.com/gp/product/B...
Face Tracking Template (Just toss your animations in here!):
github.com/Adjerry91/VRCFaceT...
Universal Eyeball Tracking Controller (From : github.com/regzo2 )
drive.google.com/file/d/18B90...
VRCFT avatar setup tutorial:
• Fast Face Tracking Tut...
(Binary parameter tutorial is now outdated, but heres the github):
github.com/regzo2/BinaryParam...
OSCMooth for emulating ik sync!:
github.com/regzo2/OSCmooth
VRCFT Discord for more questions/answers and templates, everythings changing fast so you definitely want to look here:
/ discord
*You should not need to pause in between switching avatars any more
If you want a better microphone for the Vive Pro Eye, you can use a USB hub and sound card. MOST USB HUBS WILL NOT WORK! I have tried several USB hubs and none under $40 have worked with the facial tracker. This is the cheapest hub that works (and the slimmest) that almost everyone uses (or at least a knockoff of this model):
smile.amazon.com/dp/B07X8ZLYLR
Here is a good lapel mic to replace the terrible Vive mic:
www.amazon.com/dp/B016C4ZG74
Here's a sound card to plug it into:
smile.amazon.com/dp/B07RV6VBNR
Photo of Index with 3d printed mount from NamBo# 1337
Song: Phantom Sage - MIKO [NCS Release]
Music provided by NoCopyrightSounds
Free Download/Stream: ncs.io/miko
Watch: • Phantom Sage - MIKO | ...
![VR "Must-Haves" you NEED! [VRChat]](http://i.ytimg.com/vi/5hAIjLW3DS4/mqdefault.jpg)
![VR "Must-Haves" you NEED! [VRChat]](/img/tr.png)
![The BEST VR Full Body Tracking is also Affordable!! [Tundra Tracker]](http://i.ytimg.com/vi/ImEKHrUp4QM/mqdefault.jpg)
The fact i watched the whole video even without owning one of those
Same
SAME😭
You need money lots of it😂, full vr kit is almost 4000$
@@ChaseRedFox nyoooooo 😭
I don’t even have VR chat nor a stable computer. All I have is a laptop. 😭
Me seeing eye lazers: hey that's cool I would love to have an avatar with that.
Me actually thinking about it: nevermind I do not want people to see exactly were I'm looking
you literally included everything I couldve needed for the facial tracker, even another 3d printed mount that will actually work! thank you so much!
oh my gosh its you sonito!!!!!
@@greighson3766 wtf how on earth did you find this 😭😭
@@SonitoShori i saw face tracking on my recommended so i looked it up
Thank you so much for posting, just got a facial tracker and printing my mounts for my quest2, been looking for some form of tutorial on it for the past few days
the amount of detail in your video blows my mind. and you are a fur too makes it even better 😊
It's absolutely nuts the level of technology we have access to these days.
Pretty neat showcase! Just finished up adding face tracking blendshapes to my avatar base. Will give it some more testing and that'll be one more base out there with official face tracking support.
you got me at the end, i loved it!
That end. I must subscribe
Love the video!
Keep it up dude ^^
I'm just amazed watching the avatar talk!
Me 2
I love the paper cup texture of your Rex!
This is very informative :D Thank you
I dont even play Vrchat, but this video is oddly entertaining
Fantastic, Disney moments IRL!
Best Tutorial Ending Ever!!! XD
I got the Vive Pro Eye a couple of years ago because the regular one was out of stock and my dad was like "well now you can have eye tracking, so just get that one" XD I finally decided to look into adding it on my VRChat avatars so I can use it :)
This is Actually Really Helpful Thx ^^ boop
IIRC, some relatively newly announced headsets are starting to slowly include eye and/or entire face tracking as well as some more experimental ones even promising inside-out full body tracking.
Will be sick if in a couple of years we start seeing even the more affordable VR headsets like what Quest 2 is today release with all these tracking methods so that immersion can go to a whole new level in VR games and especially VR social apps.
Though ofc there's also the matter of rigging and adjusting the models themselves for face tracking, but I'm pretty sure there's a few that already do that, so if everyone gets face tracking, the demand for such avatars will go way up, and more people will make them.
right now a lot of the vtuber and hobbyist discord friendstreamers enjoy VroidStudio weeby models with iphone + iFacialMoCap for extremely well-done facial tracking. like there is nothing else out there better. lots of the bigger vtubers who have a real 3d model have been going the iphone route. I just hope one day VR can meet this same standard, because people have expectations now.... Like not even the Live 2D models from NijiSanji or HoloLive have good facial tracking and people just kinda overlook that.
@@haroshadow I find it kinda weird that there's some vtubers with exceptionally good visuals on their models like Girldm but then all the super extra popular main-stream ones just use well drawn, but otherwise basic 2D rigged sprites that don't look nearly as good or expressive in the end
@@wisgarus i agree but it is mainly because a lot of the "lowest common denominator" type of basic bitch weeb fan actually have admitted lots of times that a 3d model is ugly or uncanny valley. they want 2d only. who knows why.
Upvote the OSC issues on the Canny and VRChat's GitHub: feedback.vrchat.com/feature-requests/p/increase-the-update-rate-for-osc-parameters
As of 4-18-22, you should no longer need to pause between avatars!
Great video
thanks mann for the info sounds like i should just wait for vrchat to release it. i dont want mee face being a slide showww to others.lmao
This is really awesome, then you see the price of the headset and you're like 💀
right, love your voice
Thank you for the simple and informative material! However, I'm having issues having the ears move with the eye and face tracking. I'm unsure if there's something set up wrong on my end where it's locked into the controller controls?
You can make a new animation for the gesture layer using the ear bones (instead of blendshapes) and tie them to the Left/RightEyeLidExpanded parameters, at least that's what I did.
I'm gonna be getting an Index so it sucks that I won't be able to yet eye tracking but maybe they'll eventually make eye tracking for the index
I would watch it all again just for the ending
thanks man
@statek - I talked with Varjo support and they are planning to put in eyelid tracking but they didn't give a delivery date
Oh, charming!
I liked.💜🌺🤗
Hiya Statek,
Do you know if the face tracking only shows locally (just for you) or are others able to see your tracked face on the avatar?
Everyone with your animations shown can
UPDATE! the varjo aero now DOES track eyelids!
the only trouble i have is OSCmooth or what this thing is called.. When i put this in my fx layer (yea i did it correct with help from others ) i crash the entire lobby in vrchat when i calibrate my eyes with my varjo :D
How much of this is still relevant? I got a Winterpaw canine and followed her tut' but only have eye tracking with basic blink (no winking or pupil tracking) and no face tracking. I thought it might ne my headset (Vive XRE) but when i use the free Wolf Link rex AV, the tracking is superb.
Any thoughts on where the disconnect could be?
I see more activity
generally Nice movie
Awesome video! And I'm assuming the facial tracker does not work with the original vive?
I think it's technically possible but not easy
great tutorial but does anyone know a fix for VIVE_SRanipal not detecting my facialtracker? sometimes a reinstall of the VIVE_SRanipal helps but now it doesnt help anymore
weird question, but think this could be done using the right joycon's ir camera?
This has helped allot
It should be fairly straightforward using one of the face tracking templates, you just have to make animations for each of the mouth movements you want and then tie them to the right parameter
I just ordered my vive pro eye, I can't wait to set it up. Thank you so much. Anything I should know at this current time?
you should get ready for trying to get the sranipal to work.. it propably wont
I really wish there was some kind of mount for a quest 2 for eye tracking. I've made so many ridiculous faces in vrc just for nobody to be able to see them.
You can check out EyeTrackVR
@@StatekTi Thankyou I had no idea this existed I'll look into it
I guess Quest Pro is going to solve all of our face and eye tracking issues. That is, assuming we can afford to drop $1500 (or whatever it'll cost).
is the vive eye the only one that does eye tracking? not index?
Do you know anything about the omnicept edition bc im thinking about buying it (good deal used)
Not worth it, I returned mine
know any places selling the vive pro eye new cheap anymore? It's seemingly only available >$1000 anymore, probably because of this stuff...
Is there anyway to do this in blender? Full and smooth mocap sounds good
Im so lost on 2nd software because it shows nothing but files and completely lost
I'll wait for the camera to work with face tracking like the full body that is available with a single camera and doing some work instead of buying that because too expensive
now I just need models that support lip tracking
Hi, can someone explain how to enable face and eye tracking without PCVR?
What is the USB hub listed in the description meant for? Is it supposed to be plugged into the headset's USB-C port?
If you want to add an external microphone and use the face tracker at the same time, you'll need it specifically
I'm going to be getting a meta quest pro and I'm wondering if it's built-in full face tracking would work with VR chat?
We're working on it right now. It's barely usable in its current state, but getting better
Hahahaha the fucking end 🤣💀
any change you'd know any way of getting a 3d printable adapter for the droolon f1 eye tracker to fit onto an index?
There's DIY eye tracking for the index that already work better than the droolon (though that isn't saying much)
Oof, gonna have to get the quest pro and deal with the mixed playspaces
Is there a way to use eye tracking with vive cosmo elite?
so the normal vive pro doesn't work with eye tracking? damn
Is it possible to have mouth tracking on the OG HTC vive? I 100% cannot afford the vive pro eye since I'm borrowing my vive from a relative.
You'd need a USB 3.0 extension cable going from your tracker to your computer, but its doable
Dose this work with a Tobii HTC VIVE Devkit? I end up picking one up for it's base stations, It would be neat to try out.
I know a few people in the face tracking discord have it but I'm not sure how well it works, if at all
what kind of pc do you need for stuff like this? (Coming from someone who just wants a prebuilt cause doing myself isnt an option)
If your CPU maxes out under 70% in VRChat, you can probably run it without issue
I clicked here for the Lazer eyes in the thumbnail
ohhh i remember you now
When I try to install it says the installer was interrupted
On google, you who they send for people asking about face tracking.
Does EAC on VRchat prevent Facial Tracking from working?
Nope!
I think they discontinued the vice pro eye now there’s this new attachment I assume can mount to any of the vice pros?
You can 3d print a mount for it to fit on anything
Considering buying one but I have a question.
Will the tracking work with any avatar or does it require creating an avatar to make it work?
You have to use an avatar configured for face tracking
@@StatekTi oh boy time to learn blender and unity 😂
The texture of the paper cups at the dentist in the 90s 😻🥲
And this will work in standalone mode (that is, without a PC ) on meta quest pro?
Nope, there's no support for standalone
Hey, so how exactly would one make a custom avatar face tracking ready?
If the blendshapes are already there, just map them to the face tracking parameters found on the github. I just added an example Unity project to the description.
Holy shit
Will any of this work for a oculus 😅
Hi i was planning to get droolon p1 module for pimax and i was wondering what other things than pimax software and VRCFaceTracking i would need to get it to work.
I believe you toss the dll in appdata/roaming/vrcfacetracking/customlibs from github.com/benaclejames/VRCFTPimaxModule/releases/tag/1.1 and then unblock it in the file properties, but I can't find anything concrete on that. You may want to ask in the VRCFT server
@@StatekTi thank you very much everything works
so i got this working, but im a little annoyed with how the mouth still reacts to mic audio on top of my tracking. which makes the tracking look dumb. is there anyway to fix this? or is it just avatar specific?
Face tracking-specific visemes (that are less extreme), particularly limiting the aa one. There is a tool for it by Sacred in the VRCFT discord I believe
@@StatekTi thanks
So the best solution in 2023 are still Vive Pro Eyes or Meta Quest Pro, right?
Yeah
What VR headset are you using? Just because I'm looking to buy VR, but I don't know any headsets that are good, so, which one could you recommend?
Vive Pro Eye. The only other real option is the Quest Pro right now
@@StatekTi Thanks, needed help about this, but couldn't find an answer, this might very help.
@@StatekTi what about pico 4 enterprise? Because i can't find any vrchat gameplay with pico 4 enterprise with face tracking.
@@Justguy363 They only just recently released the ability to grab face data but it's being worked on
@@StatekTi oh ok, i wonder how much time would take to implement
Do you know if the G2 Omnicept Eye tracking works as well?
Theoretically it should be able to work nearly as well as the VPE, but not enough people have one (and the people that do have one prefer other headsets for multiple reasons) so theres no current eye implementation for VRC and no mouth implementation for anything.
no idea wat this is but cool
WAIT DID YOU SAY LEGALY POSSIBLE? Like was it illegal before?
What’s the eye laser asset called and where can I get it???
It's in #file-share in the VRCFT discord
You mean I gotta take my headset off, push P on my keyboard and go back in VR to change my avatar, take my headset off again, push P to unpause it then go back into headset before I want to use it, each time I change my avatar? Sounds like a PITA.
Not anymore (see pinned comment)
I want these, but I dont trust myself with them on while partying
Hey Statek do you know any Avatar worlds where i could find furry avatars with eye and mouth tracking?
Jerry's Mod has a few, xMetalWolfx also has some older ones I think
Wouldn't all this work optically with a webcam of sorts and some middleware?
When there isn't a big vr headset on your face, yeah
Does anyone know what the color scheme of his avatar is called? I have seen it all over the place but have not found a name for it
It’s from retro drink cup designs
Solo Jazz
I am saddened the Aero Has No eyelid Tracking
Me too, that literally killed my wanting to own one haha
Me watching this without a vr headset
Never needed one. Phone or webcam.
Hi! Do you know if this will work with the HP Reverb G2 Omnicept Eye/Face Tracking?
There are some people working on it but as of right now it will not work. The mouth camera outputs a raw video feed so you'd need to train your own NN to get anything useful out of it, and I don't believe the eye cameras support floats for eyelid tracking (which is extremely important for social VR).
@@StatekTiThat's a huge bummer. I'm going to see if I can use the camera feed and run it through htc's lip tracking software to get some results. It doesn't seem there's much I can do about the eyes though, so damn. I guess i'll just be patient. Thanks for the quick response!
@@FireStillPlays Unfortunately you won't be able to get any good data from it even if you were able to pass it through their software. The camera feeds are from extremely different angles (and with the Vives in stereo), and even tilting the Vive tracker up/down a few degrees will start to ruin your data.
@@StatekTi Well damn, guess patience is my only option then :P
Thanks again for responding so quickly! P.S your avatar kicks ass!
@@StatekTi Hey, I wanted to mention that the Omnicept headset in fact does have data output that includes "[float] Openness" for blinking as listed in developer documentation(along with several other interesting features and values). The mouth tracking does appear to provide just a 400x400 grayscale image, but there are several methods of grabbing that to be explored later. I am willing to work with someone on this implementation, starting with the eye tracking, and working with mouth image data after. I have already contacted HP techs about possibly providing a form of active mouth tracking via an alternate SDK to achieve something similar to HTC's. Wanna see where this could go or have anyone willing to collaborate? (if it isn't obvious, I have this headset)
Any guides or solutions for using the quest pro?
You can join the VRCFT discord for guides on how to set it up. There should be a new official release with support for it in the coming days/weeks that I've been waiting for to make an update video on
Hey could i use my phone instead im really curious
Yep, there's a module for Meowface
all I heard was "money money money money, money mon mon, money mon"
The Sranipal mirror isn't working :(
Should be now
That old tracking software thing still hold true?
It's been updated a few times but yeah everything still works
Wow is there a tutorial for quest pro too?
Ninka_ has one up right now czcams.com/video/uZh7cY170rY/video.html
Didn't work
Damn technologies. It's starting to scaring me.
Any updates?
Yes but I've been dead
Dose this still work with the easy anti cheat update?
Yep
what about quest pro?
It'll work as well with the newest VRCFT software
Is this possible with the quest pro yet?
It has been for a while yeah
Been using Quest Pro for about 6 weeks now and absolutely love it! Only drawback in models is, those done for Vive don't track the eyelids correctly with Quest Pro. They only start closing when eyes are almost completely closed. But there are few public avatars made with Quest Pro toggle that fixes it.
Gotta do my own Nardo to be completely satisfied