Hey Ryan, I never did anything 3D related before, but after your UE5 episode I got interested and started diving into it... fast forward a few weeks and now I'm working on an (amateur) 3D animated series of my favorite book. So thanks for bringing my attention to this kind of "new" stuff! :)
Great video Ryan! I am really stunned with how easy it's becoming to integrate VFX. All the hassle and headaches are going away, allowing us to focus on the most important thing! The creative process
This is mind-blowing! This technology is what made Avatar. Knowing that you can create the same level of production by just using your phone is just phenomenal.
Your rant on why understanding tech is important made me so happy, I literally know so many "filmmakers" who don't even know which lens mounts they can use with their camera... it's nauseating to me.
@@tafpictur6206 haha, I already live in the google ecosystem and like it, what I don't like is how a lot of cool apps are only developed for iOS. Like Adobe for example.
Great video! I hope Apple will take advantage of the M1 so we can get smaller versions of Blender/Unreal on iPad Pro! Between this, and all the new 3D/VFX tools on iOS...mobile filmmaking is in such an exciting place, and it’ll be fascinating to see how it continues to overlap with virtual production.
Hi friends... Can you create a seperate playlist within the channel for virtual production stuff ? That would be super useful. Also thank you so much for everything you're teaching to us. They inspire us a lot brother ❤
I was thinking the same thing, but they're not actually shooting the same thing, so the perspective would be slightly off, so I don't think the tracking data would be useable at all unfortunately
Sounds like a good idea, but the track would be offset. Not sure it'd be great. Plus I've seen lots of iPhone stuff slip'n and slid'n. Is it accurate enough?
To get over the HD recording limitation, could you mount the iOS device to your normal camera and shoot with that but take the tracking data from AR App to use it with the video from your normal camera to chroma key and composite in unreal?
Thinking the same thing! The most immediate problem I can foresee is that your 4k camera would have a different pivot point than your iphone, which would be increasingly obvious the closer you are to your subject.
The new version of ARKit in iOS 16 (currently beta) supports 4K and HDR capture specifically to aid with VProd use cases, so this should be resolved in the fall (or possibly sooner if you can get a beta from the developers)
This is sick. Could you theoretically mount an iPhone to your camera rig and use the tracking data in post? And would the quality of tracking be high enough to justify adding it to your camera rig?
There is one crucial setback - you cannot syncronize the iphone capture to cinema camera frames. For tracking to be usable, you need perfect sync. It's way better to do traditional matchmove using your plates, when you can.
@@Vassay Yeah, this has come up a few times, the devs say that the lag between the sensors of the iphone and the other camera is the problem. It would be great if they could calculate the lag and use it as some sort of assist to combine with traditional matchmove though.
@@stevegreen9474 Beter just to get an accelerometer+gyroscope mounted on the camera, and feed that directly into the metadata. There were several attempts of doing that, but I don't know if that ever took off. In any case, traditional matchmove is not that hard in most cases, and can be used with just the original footage.
@@Vassay I've not really done a lot and most of mine tends to be the planar track distant stuff. If it was a rock solid automatic track, I'd love it, to me it's like roto, a boring necessity.
assuming that you could adjust in post for the position above the camera then change the virtual camera settings to match your dslr (lens size, aperture ect.) planning on doing this myself
Damn, someone beat me to this! I downloaded Spark AR for this specific reason, and hadn't seen anyone do this but looks like you guys already got it going! Congrats! What I'm gonna do is still different to what you guys did, but in terms of being the first ones I saw, you guys got it!
Seeing as many peopel are asking about using this app with a regular camera - yes, you can do that, but the result will be quite poor. Not counting all the extra steps to get the track, it will not bounce around like crazy, unless you have extremely smooth movements. Even AE camera tracker, which is quite poor, will give you a better, more stable result, than trying to get the track from the app and Frankenstein it to big camera's movement.
I wonder whether there is an App to use live tracking from the iPhone to control UE in real time. This way you could mount the iPhone on a camera and use virtual production… any ideas?
I love how Sam from Corridor Digital has worked so hard to make 3D space work and has messed with unreal etc for so long…and now there’s a app for it 😂😂
You can also create usdz files with windows, using houdini 19. As far as I know, you'll have to create and USD asset first and then use the USD ZIP rop to convert the usd+materials to usdz file
Have you tried using a mount for the phone on one of your other cameras to track the motion but use camera footage? You would have to parent/offset the virtual camera later, but it would allow more options.
Quick question, would you be able to tether the phone to the cage of an actual camera, while using the app to capture two individual shots? Like say the iPhone is capturing the 3D rendered background while the camera is capturing the subject? Then in post you could splice them both together. Or would there be some slight issues with that?
I feel like you could use this app side by side with a more pro level camera. Use the iPhone for the reference and tracking and then the footage can sync up the tracking points and blend the unreal with the pro cam footage. (Hope that makes sense)
So … I feel like it should be possible to use the phone to just capture the tracking data to be used for footage captured by another camera by mounting the phone on a different camera. Also newer Sony cameras have accelerometers and gyros in them that allow the footage to be stabilized in post. I feel like this should make it possible to capture data like this for post tracking. But I doubt Sony will ever open up like this.
I'm enjoying the camtrackAR.. but what if there's a MIRRORLESS camera or CineCamera that has built-in software that that will do the same as CamtrackAR that would really be so sick.I hope that would happened in the future for sure VFX artist would be happy and hobbyist small budget VFX will enjoy it.
Hlo, I am a beginner and I don't know how to write a movie script but the whole movie is in my mind... And it is based on si- fic.. and dinosaurs....so, what should I do because I don't have a laptop... Please help me....to solve that problem or suggest me something.🙏
This is very awesome, is there a way to use all of this tracking data captured and apply it to another camera such as a Cinema camera. Say you rigged the phone on top of the camera and even possibly our previewing what the composite will look like through the monitor. But rather than having to use the iPhone camera the data from the track can be applied and used for cinema camera giving the ability to shoot at a desired frame rate, codec & With a lens of your preference
Follow up: if there is not an integrated way to do this already could you in theory film two separate takes one on the iPhone one on the cinema camera mounted together on a rig so the perspective is similar. Then in Post roto scope out The subject in the iPhone footage and then rotoscope out the subject from the cinema camera footage, then by compositing the footage of the subject from the cinema camera footage into the now clean plate from the iPhone take. In theory that should work, not sure if it would cause a problem such as having to use the footage of the background captured by the iPhone mixed with the footage of the subject from the cinema camera and if that would be problematic, would love to hear feedback on if this would be a suitable option for capturing high-quality footage with quality tracking data that can be used to track the camera in 3-D space
Will film riot be producing more 'practical' based videos in the future such as on location lighting setups, or bts footage from projects? These are good videos but I'm really missing some of the more practical videos.
will this work with the new unreal ? also did u film apple pro res for this ? and did u film everything on the phone and not with DSLR and what app would u use to film on the phone also do u need to use abobe effect ? and can u also use divinci resolve ?
"Hi tech creates bad films" Yep, definitely an over simplification. Truth is many people who make bad movies don't know what makes a good story/concept. A cool concept doesn't equal to a good, philosophically sound story that people can get immersed in. Hi-Tech just enables more people to share their ideas with the world, and obviously since the bar of entry is lower, more people will create things and there's obviously gonna be more a lot more "not great" films; but this also means a lot of people who couldn't have executed on their idea before, now can. Hi-Tech is great. It never replaces knowledge tho, it just changes the specifics of what needs to be learnt
Wait, can’t you attach the phone on a camera and use the camera footage with the phone tracking data? Please let me know if that’s possible 🙏🙏🙏 I’ll run some tests tomorrow 😊
yes there's a tutorial somewhere on youtube that explains this, did you manage to do your tests? (make sure you have the two cameras as close as possible so your offset is easy to manage)
Hey Ryan,
I never did anything 3D related before, but after your UE5 episode I got interested and started diving into it... fast forward a few weeks and now I'm working on an (amateur) 3D animated series of my favorite book. So thanks for bringing my attention to this kind of "new" stuff! :)
Great video Ryan! I am really stunned with how easy it's becoming to integrate VFX. All the hassle and headaches are going away, allowing us to focus on the most important thing! The creative process
This is mind-blowing! This technology is what made Avatar. Knowing that you can create the same level of production by just using your phone is just phenomenal.
Crazy, there’s an app to transfer your consciousness into a giant 9 foot tall cat man. Try it, it’s wild.
dude let’s go! film riot has just made friday better! luv y’all’s vids
I’ve been waiting for this!
This. Blows. My. Mind. Well done guys!
Been waiting for this. Can always count on you guys. Keep it up!
put the phone on top of your cinema cam and track the data with it, while recording your raw file to you cinema cam, thanks for this vid ryan
Your rant on why understanding tech is important made me so happy, I literally know so many "filmmakers" who don't even know which lens mounts they can use with their camera... it's nauseating to me.
that app looks great! I hate that all those super cool filmmaking apps are limited to iOS and are not made for Android too
It’s different apple is there futur u must change your phone 😂
The problem is only iPhones and ipads have Lidar tech.. Which makes this possible
@@johnaart yea take the 12 pro max
@@johnaart that's a great explanation, for this case, but it happens a lot with filmmaking apps
@@tafpictur6206 haha, I already live in the google ecosystem and like it, what I don't like is how a lot of cool apps are only developed for iOS. Like Adobe for example.
Great video! I hope Apple will take advantage of the M1 so we can get smaller versions of Blender/Unreal on iPad Pro! Between this, and all the new 3D/VFX tools on iOS...mobile filmmaking is in such an exciting place, and it’ll be fascinating to see how it continues to overlap with virtual production.
BEATIFUL PRODUCTION QUALITY
Damn. I love their content. I have learn a lot of things here. Your the master Ryan . THANKS
Hi friends... Can you create a seperate playlist within the channel for virtual production stuff ? That would be super useful. Also thank you so much for everything you're teaching to us. They inspire us a lot brother ❤
I would love to see what you guys could do, by just mounting the phone on top of a larger camera rig and using it like a tracking system.
I read this right after writing a similar comment. I wonder if the tracking would be good enough to use in lieu of tracking in post.
I was thinking the same thing, but they're not actually shooting the same thing, so the perspective would be slightly off, so I don't think the tracking data would be useable at all unfortunately
yes it works
Sounds like a good idea, but the track would be offset. Not sure it'd be great. Plus I've seen lots of iPhone stuff slip'n and slid'n. Is it accurate enough?
Oh and the frame rate issue as he pointed out.
Thank you for always been at the top of new technologies coming
To get over the HD recording limitation, could you mount the iOS device to your normal camera and shoot with that but take the tracking data from AR App to use it with the video from your normal camera to chroma key and composite in unreal?
I was thinking the same.
I wanted to know the same, any news?
Thinking the same thing! The most immediate problem I can foresee is that your 4k camera would have a different pivot point than your iphone, which would be increasingly obvious the closer you are to your subject.
The new version of ARKit in iOS 16 (currently beta) supports 4K and HDR capture specifically to aid with VProd use cases, so this should be resolved in the fall (or possibly sooner if you can get a beta from the developers)
@@uehreka thanks for that, will keep an eye open! Cheers
I’ve been searching for so long, finally 🙌
This is sick. Could you theoretically mount an iPhone to your camera rig and use the tracking data in post? And would the quality of tracking be high enough to justify adding it to your camera rig?
It would be slightly different since they’re different angles with the subject, but it can be pulled off
There is one crucial setback - you cannot syncronize the iphone capture to cinema camera frames. For tracking to be usable, you need perfect sync.
It's way better to do traditional matchmove using your plates, when you can.
@@Vassay Yeah, this has come up a few times, the devs say that the lag between the sensors of the iphone and the other camera is the problem.
It would be great if they could calculate the lag and use it as some sort of assist to combine with traditional matchmove though.
@@stevegreen9474 Beter just to get an accelerometer+gyroscope mounted on the camera, and feed that directly into the metadata. There were several attempts of doing that, but I don't know if that ever took off.
In any case, traditional matchmove is not that hard in most cases, and can be used with just the original footage.
@@Vassay I've not really done a lot and most of mine tends to be the planar track distant stuff.
If it was a rock solid automatic track, I'd love it, to me it's like roto, a boring necessity.
Screw the apps ! We need to buy your hoody 😍🔥🔥🔥🔥
Thanks for this video you're uploading today. I appreciate your existence :)
Can you mount this on a video/mirrorless camera and use the tracking data with the footage to get better quality? Like a node for tracking?
assuming that you could adjust in post for the position above the camera then change the virtual camera settings to match your dslr (lens size, aperture ect.) planning on doing this myself
this tip is gold, thank you
What I'm waiting for, is basically camtrackAR for a camera.
00:56 mm yes the *blender* logo
Damn, someone beat me to this! I downloaded Spark AR for this specific reason, and hadn't seen anyone do this but looks like you guys already got it going! Congrats!
What I'm gonna do is still different to what you guys did, but in terms of being the first ones I saw, you guys got it!
Could it be used to check if the lighting of the chroma key is good? So we wouldn't have problems while keying in post-production.
For the 24 fps problem you can use time interpolation in premier to set it to 24
I wish this was available on Android.
I'm overwhelmed. there is too much cool shit out there and not enough time to do it all -__-
ngl just for the lighting reference purposes, it's quite useful
Nice.
Oh man this is going to be so fun to play around with!!
You are true King, Ryan! Thank you!
Seeing as many peopel are asking about using this app with a regular camera - yes, you can do that, but the result will be quite poor. Not counting all the extra steps to get the track, it will not bounce around like crazy, unless you have extremely smooth movements.
Even AE camera tracker, which is quite poor, will give you a better, more stable result, than trying to get the track from the app and Frankenstein it to big camera's movement.
omg thank you so much for sharing this!
I wonder whether there is an App to use live tracking from the iPhone to control UE in real time. This way you could mount the iPhone on a camera and use virtual production… any ideas?
I love how Sam from Corridor Digital has worked so hard to make 3D space work and has messed with unreal etc for so long…and now there’s a app for it 😂😂
As always, thank you.
Great Video!
🙏🏾My goodness thanks Ryan excellent video 💪🏾
Slowly learning unreal. Can’t wait to start creating worlds
This is very very amazing!
So, I guess all we need is an iPhone now?
I just finished a video that I made with Xiaomi mi 10 t. Every thing else is made in unreal engine. And I really like it.
Thank you.
I think you can attaché the phone to your camera and use the giro data with the camera footage.
This is a really cool app! I hope they make an Android version soon!
You can also create usdz files with windows, using houdini 19. As far as I know, you'll have to create and USD asset first and then use the USD ZIP rop to convert the usd+materials to usdz file
Thank you!!
Tnx dude
Have you tried using a mount for the phone on one of your other cameras to track the motion but use camera footage? You would have to parent/offset the virtual camera later, but it would allow more options.
I was going to ask about this too. Hope he replies!
Quick question, would you be able to tether the phone to the cage of an actual camera, while using the app to capture two individual shots? Like say the iPhone is capturing the 3D rendered background while the camera is capturing the subject?
Then in post you could splice them both together. Or would there be some slight issues with that?
so glad they added 30fps. that 60fps last year was a big no no lol
I feel like you could use this app side by side with a more pro level camera. Use the iPhone for the reference and tracking and then the footage can sync up the tracking points and blend the unreal with the pro cam footage. (Hope that makes sense)
Very impressive!
This was insane!!!
this video will get billions of views.
okay wow
I didn't think I wanted to upgrade my iphone but that lidar..... 😍
also the amount of years I've been following and that is my first comment 😂 sorry guys
Thanks
Bless be the best youtube channel ❤️
Muy buena herramienta, gracias por compartirla
So … I feel like it should be possible to use the phone to just capture the tracking data to be used for footage captured by another camera by mounting the phone on a different camera. Also newer Sony cameras have accelerometers and gyros in them that allow the footage to be stabilized in post. I feel like this should make it possible to capture data like this for post tracking. But I doubt Sony will ever open up like this.
I'm enjoying the camtrackAR.. but what if there's a MIRRORLESS camera or CineCamera that has built-in software that that will do the same as
CamtrackAR that would really be so sick.I hope that would happened in the future for sure VFX artist would be happy and hobbyist small budget VFX will enjoy it.
Hlo, I am a beginner and I don't know how to write a movie script but the whole movie is in my mind... And it is based on si- fic.. and dinosaurs....so, what should I do because I don't have a laptop... Please help me....to solve that problem or suggest me something.🙏
This is obviously a game changer 🤯
Insane!
This is very awesome, is there a way to use all of this tracking data captured and apply it to another camera such as a Cinema camera. Say you rigged the phone on top of the camera and even possibly our previewing what the composite will look like through the monitor. But rather than having to use the iPhone camera the data from the track can be applied and used for cinema camera giving the ability to shoot at a desired frame rate, codec & With a lens of your preference
Follow up: if there is not an integrated way to do this already could you in theory film two separate takes one on the iPhone one on the cinema camera mounted together on a rig so the perspective is similar. Then in Post roto scope out The subject in the iPhone footage and then rotoscope out the subject from the cinema camera footage, then by compositing the footage of the subject from the cinema camera footage into the now clean plate from the iPhone take. In theory that should work, not sure if it would cause a problem such as having to use the footage of the background captured by the iPhone mixed with the footage of the subject from the cinema camera and if that would be problematic, would love to hear feedback on if this would be a suitable option for capturing high-quality footage with quality tracking data that can be used to track the camera in 3-D space
This is amazing! Whats the Android equivalent of this?
Will film riot be producing more 'practical' based videos in the future such as on location lighting setups, or bts footage from projects? These are good videos but I'm really missing some of the more practical videos.
This is incredible i have to try this😍 unfortunaly i only have an ipad no iphone
Works great with my iPad Pro
Awesome!
Why not stick the iphone to the big camera and use the tracking data. Is it posible to do?
will this work with the new unreal ? also did u film apple pro res for this ? and did u film everything on the phone and not with DSLR and what app would u use to film on the phone
also
do u need to use abobe effect ?
and can u also use divinci resolve ?
I usually just do editing on my phone. I should use my Macbook...
you should do a video on using unreal with the macbook m1 chip
Y'all know we're in the future, right?
This is insane
Can you mount a smartphone to a film camera and track the production camera that way?
Can't you use your camera to do actual shoot, and iphone just as a tracker?
Could we record with a camera also in order to use the camera footage and the tracking data from the iPhone?
Interesting video
CamTrackAR works with any IPhone or IPad? or it´s necesary the camera depth?
I have issue with the frame rate for tracking any idea why ? i try both 30 and 60
Could someone say, why CamTrackAR shots the video with so poor quality? It feels like 480p. Need help!
I wonder if this could be strictly the data? I wonder if you rigged the phone to a cine cam how effective that would be?
Damn thats badass!!
Cool
Um dia vou ser bom como vocês!!
Hello Film Riot
which iPhone type were used?
The future is here.
"Hi tech creates bad films"
Yep, definitely an over simplification. Truth is many people who make bad movies don't know what makes a good story/concept.
A cool concept doesn't equal to a good, philosophically sound story that people can get immersed in. Hi-Tech just enables more people to share their ideas with the world, and obviously since the bar of entry is lower, more people will create things and there's obviously gonna be more a lot more "not great" films; but this also means a lot of people who couldn't have executed on their idea before, now can.
Hi-Tech is great. It never replaces knowledge tho, it just changes the specifics of what needs to be learnt
I think the problem is when people try to build a shot around the tech rather than use the tech to capture the shot you need.
Wow !! 👍👍
Would it be technically possible to just attach the phone to a camera and then later use the tracking data with the camera footage?
I downloaded the app, but i couldn't do anything unless i purchase one of their subscription. So I deleted it immediately .
Guys, can u make a video using a Samsung Galaxy S21 Ultra?
Wait, can’t you attach the phone on a camera and use the camera footage with the phone tracking data? Please let me know if that’s possible 🙏🙏🙏 I’ll run some tests tomorrow 😊
yes there's a tutorial somewhere on youtube that explains this, did you manage to do your tests? (make sure you have the two cameras as close as possible so your offset is easy to manage)
Can't wait to try this... oh yeah, doesn't work on an Android phone
what are your setting for UE camera to match CamTrackAR footage with footage from app?
Ghen ghen. 🤤