Going to Siggraph Real-Time Live! Face animation real-time in Unreal Engine

We've gone REALTIME! By adding a few new amazing ingredients to the mix... Unreal Engine and IKINEMA LiveAction!!
Its crazy how this setup, which is capturing and rendering in realtime looks damn close to my previous tests rendered in vray! Hat's off to Epic & collaborators for letting all of us pillage their Digital Human scene (a ton of bleeding edge work on their part), which still freaks me out to light and render at this level in realtime! Its kind of a lifelong dream come true :)
More hats off to Joezak Steel (aka Joe & Zak) at Epic for helping me with the ARKit facial capture stuff!
I'm over the moon how cool this setup has become and excited to show off more of the details live at Siggraph RealtimeLive in August!
More to come
Some context:
These are a series of test I'm doing to find a fast pipeline for performance capture (Face and body) that we can use for our up and coming game Bebylon: Battle Royale. bebylon.world
#vtuber #animation #unrealengine #iphone #motioncapture #xsens #bebylon #arkit #ue4 #apple

[@michelkiwic]

Congratulations Cory to the Siggraph 2018 Real-Time Live Award. You definitely earned it. It‘s just mind blowing what you achieved. You are my biggest inspiration. Thank you for sharing your art and I hope I will be able to meet you one day somehow. Greets from Switzerland.

[@trickdiggidy]

Thanks a lot Michel, super appreciate it!!

[@al00irdv]

That 2nd whats up baby killed me 😂

[@khabubu_phathu]

I want this setup so bad

[@eduardofernandes7889]

This is so sick!

[@teddybotssofttoysandgames9700]

thats very effective. looking forward to seeing more

[@cyrusjameskhan]

This is amazing! Would love to get my hands on a similar setup!

[@nelots]

I love this, wish I could afford all of this set up

[@LittleSpaceship]

Nel how much would this set up be?

[@Bob-zg2zf]

lol
"too many babies"

[@Drahoslav_Lysak]

Great! Great! Great!

[@redswan6765]

try with Thanos 3d model

[@TsahaProduction]

That's really cool! I want that!

[@Grilinctus]

Unbelievable!

[@elvismorellidigitalvisuala6211]

How to get a real time face like that without Latency??? it looks perfect

[@xanaduBlu]

surprisingly the latency is minimal and in sync with audio.

[@aa3ali4]

That's amazing ! Can anyone tell me what kind of technology is that ?!!!

[@bristolvrlab6713]

BIG!

[@AAa-tn5rz]

Bristol VR Lab I've read you guys are also doing this, I would like to come down and check it out if possible.

[@bristolvrlab6713]

You can pop along to one of our Tech Play Fridays if you'd like to have a play with our kit. This Friday we're going to be playing with the Perception Neuron suits which we're already having quite a lot of fun with! There are 11 bits to the helmet that were ordered today so we'll see when they turn up! If Friday afternoons are no good then you can always book in a tour, though this will be for a much shorter amount of time. calendly.com/bvrl-team/meeting/

[@AAa-tn5rz]

are selling your solution yet? also will the siggraph session be online also? can't wait.

[@trickdiggidy]

Not selling anything but if your on Unreal with 4.20 thats all you need for the realtime facial capture (using their ARKit integration & LiveLink) For the Siggraph session, I'm pretty sure it will stream live on youtube but don't quote me on that.

[@yaronyashinski3077]

This is gorgeous! How do you real-time stream the iPhoneX data?

[@trickdiggidy]

It's built into Unreal 4.20 (they expanded their LiveLink feature into their ARKit integration so its pretty easy to stream over wifi)

[@kylisation]

Hello, can the facial rig data being export from Unreal?

[@shekiba.646]

nice job.

[@alialtaf3412]

Can this help indie developers to make AAA quality games with this level of motion capture???

[@georgepartridge4757]

Hey man, whats the rig called that you're wearing on your head?

[@breaknews6559]

What’s is name of the iphone app ??

[@sarp5567]

i have percettion neuron and i clone full bady system and it is not working

[@JoeRussellProductions]

What would happen if you tried to put this character into a real live action scenario? Something filmed.

[@gravidadezero5931]

Great job! Congratulations!
$5K considering all itens of the setup? MVN+IphoneX+Helmet?

[@GHCMargarita]

Nah helmet is around 100$ it's a homemade one he brakes the price down in one of his videos every part off amazon

[@touristhawk]

is it still possible with xs series?

[@altamediacompany]

Would you make it alive on Unity3D? Thanks

[@ppepe6932]

make VRCHAT g8 again!!

[@davidmonteiro6822]

How can we contact you for the development of an app with similar functionality?

[@learntrading8164]

Maaaan please i want to work with you!!

[@austinbowers2191]

I will take 30!

[@ANISIUBAUCHE]

Hi, where can i get a helmet rig for the IphoneX

[@trickdiggidy]

you can make a very uncomfortable one, but it works: uploadvr.com/iphone-xsens-performance-capture-bebylon/

[@retro_iq458]

How to do that??

[@Digital_Guerrero]

Are you using a PC or Mac ?

[@trickdiggidy]

both. Most is running on PC but need the mac for iPhone / xCode

[@LUGERIUS-GAMEs]

FAKE

[@fhoplist]

Go on......?