Why?! Why this post doesn't have more views??? This is the type of information that truly helps others, speeds up the creative process, avoids mental breakdowns, and unnecessary firings.
Every couple weeks I check to see if there is a new video for this workflow 😩, finally something...🙏 There is a ton of stuff and experience compacted into this video!
Congratulations on a very good workflow! Note: To display the "BP Media Bundle... (Media Bundle Package)", you must activate the Media Framework Utilities in PLUGINS.
@@Vilaroucas honestly even that doesn’t work anymore for me. The best way to do this is to just create a project using the “blank” television/film template. And if it’s a downloaded environment, just migrate everything over to a new blank television and film template project.
@@cms3d really happy I could help. I can’t stress enough that if you make a hard push with this virtual production stuff right now, you will be leaps and bounds ahead of other people in 2 years time. I’m doing a massive project and have been screen capping some more advanced workflows, so another tut soon!
@@cms3d another fun idea, which I will be doing a tutorial for, includes working with transforms on your root parent, you can do a lot of fun things like shoot someone in a static car and make it look like it’s moving, even if the shot has advanced camera movement in the original. Really all you need is a solid track and a USD file and the possibilities are endless. Endless dude! Dream big.
nice video man, i would really want to dive in a virtual production unreal engine carrier but i need a course of high value from a good calibre like you
your video really help me solving tons of problems !!! thank you so much!!! from now on i'm gonna explain how to make media plane like you. create a media player create texture and....just look it up lol
Do you actually need help with that, or is that a joke? Hope this truly was insightful! Happy to make a video breaking down the media plane in more detail if anyone is interested!
wow never knew you could do it via USD, always thought you had to use the blender to unreal script thingy. did you end up doing the composite in unreal engine itself or are you rendering out the plates from unreal and compositing elsewhere?
Yeh neither did I, this solution is definitely the best, believe me I have looked everywhere! Compositing done in Final Cut by Director…but you can DEFINITELY export no problem with the MF KEYER node in the material you create! If you don’t, it’s important to tick off the render visibility in the details panel under your media bundle BP attached to your camera!
Thank you for sharing your workflow, it's very rare to find content like this for Unreal Engine. Can a camera tracking plugin like fSpy be used for a project like this? And for final compositing and keying, do you do it in Unreal Engine or in post-production software like After Effects or similar software?
Thanks, friend, yes, you can absolutely use fSpy… The process would be to open your .fspy file in blender, add a floor plane and then export the scene. When you’re exporting the USD file, you just don’t really want to bother ticking “animation” checkbox if it’s only a static image! Keying goes to post production software, but you can render it out right in UE5, just won’t be as clean! But for some shots, you can get away with it
I get that doing the camera tracking in blender is probably what makes the most sense, but I wonder if it would be possible and/or make sense to use an iPhone (or maybe even an android phone) 3D tracking on the camera rig to get the camera movement information instead? I wonder if that would make it possible to do everything directly in Unreal Engine directly? (Provided I don't know anything about any of this and I am just wondering)
@@FuwakamiMana yes, you’re on the money. Using trackers that are placed on the physical camera is something that a lot of research and engineering dollars have gone into. The problem comes when you’re trying to make sure your frame rates are synchronized. iPhones are hard to genlock, which is the process of synchronizing frame rates between devices such as audio recorders, cameras, and software, like unreal engine. The other issue is that you need to store that data and keep it consistent with your various takes, which is a nightmare. Lastly, iPhones do a very poor job accounting for micro-vibrations, which truly make or break a track. At least in any moderately professional setting, you want to make sure that all those tiny little bumps and jitters get factored into your camera solve. That’s all I got on that! Of course there’s other various software that you can use, including syntheyes and 3-D equalizer. For me I just tend to use blender and when you really figure out exactly how to carry these shots over to unreal, you’re typically not disappointed.
@@andrewdileo thank you very much for this detailed explanation! It’s very appreciated! I learned a lot from this video, and while I don’t think I’ll ever attempt to make anything professional or realistic, I think I learned a few useful things! ＾＾
One question! If I am planning to color grade after rendering UE project, should the keyed footages be in ACES format? (The footages played via media player) Appreciate any help!
@@whasuklee great question man. The answer is shockingly simple: unreal engine is an sRGB native linear workspace. If you want to use an ACES workflow and want to do your keying in unreal or bring in pre-keyed footage you should load sRGB tagged footage into unreal. When it comes time to export, just use your OCIO config file in the color management tab in the movie render queue. This way you’re not applying an ACES conversion TWICE. Only once consistently across your entire scene. Then everything should convert back to linear display spaces just fine. Hope that helps!
@@user-yl6tm3cs3v I got you, I always click “don’t use a default cache” on that pop up. Since you’re importing, this does not matter. This is only if you are using the USD stage for persistent changes between programs.
Thanks a lot for this video! I was battling with this recently. To be honest, I am battling with UE overall, as I feel like it just keeps throwing sticks under my legs while trying to achive things that are super basic and primitive to do in most DCCs. May I ask you what is your approach when adding shadows? Or overall compositing the characters into the scene so that there is a shadow, perhaps reflections. As this is only slapping the greenscreen on top of movement-matched 3D scene. Thanks so much!
Shadows are always a pain deal with, and there’s no set-in-stone method to achieve them. One method is with shadow casters- you can use the plane that is imported as the ground, and set up some rough geometry similar to your actors to cast shadows onto a white plane for compositing. Most of the time, Ben, who does the composing, will actually try to pull out the shadows that were actually on the green screen in the shot, and use those. If you have your actor inside unreal as an actual “plane”, that’s when you’ll get hyperrealistic reflections, without crazy compositing. You might just have to switch your material, playing back the video to “masked” for the transparency handling. I will make a video about this exploring some different options. Maybe one day AI will do it for us, that’s the kind of stuff I would like to see AI be used for 🥴 shadow gen
@@andrewdileo Thanks a lot for your answer. Yes, pulling out the shadows from greenscreen works in some cases, especially for contact shadows you can't really get other way afaik. However, if there's for example a wall in the virtual space behind the actor (or tree, anything) and it wasn't there when shooting the greenscreen, you're out of luck as you can't composite the ground shadow on it. At least I can't imagine that. About importing planes into UE, I tried it with Ian Hubert's method of scaling the plane with the origin at camera's space. However, I had so much trouble doing that in UE. Would really welcome a quick video on that if you ever managed to do that. I managed to make it work with some workarounds but it wouldn't render. The character would be behind the scene for whatever reason although in viewport it was perfect. This however is too limited in two way from what I found: if the camera is slightly tilted downwards which it usually is, further foot will float in the air and although visually they are touching the ground in 3D, the shadow doesn't match. Second thing is that if the shadow is supposed to be cast from the left or right, it won't work as the plane is flat obviously. Moreover, I dunno if there's any way to composite it later that way. Having a separate render layer for the character, shadow and rest. Sorry for such a long reply, I'm just kinda summarizing my thoughts and struggles trying to find some other possible way or combination of techniques. Really wonder how it's made in professional production. Thank you in advance if you have any more input to this.
@@ArtemGms yeah, I have to digest this and honestly, I can definitely get to work on a video breaking down shadows a bit more in the workflows that I would opt for personally - if you want to achieve the Hubert method of scaling the plane in relation to the camera, so the feet are clipping the ground, the best way to do that is to do it in blender itself, and then export that with the animation as a USD file and then literally just use that plane as the plane that you’re projecting media onto…I just did it recently on a Jacob Collier project for the final scene in Mi Corazon. It worked out great. I should make a video about that as well. I think everyone has studied in Hubert and is trying to channel that in unreal and I have been devoting the past several months to learning how to transfer that skill set over I will make a video devoted to just showing how to do Ian Hubert stuff in unreal lol
@@ArtemGms I know what you’re talking about as far as the other foot floating in the air, one workaround is to have a duplicate that’s only acting as a shadow caster and then obviously you would just translate that down on the Z
@@andrewdileo Sorry for the late response. That would be amazing if you could make a video about that. Haven't tried USD workflow, I tried to do it in UE. Only did the track in Blender and it was terrible. That floating feet workaround sounds interesting but you then have to somewhat merge those 2 shadows, right? Which leads me to a 2nd question. Is it possible to do render layers? So I can even achieve something like that. Having environment, greenscreen character and its shadows on different layers? So I can perhaps add extracted shadows from greenscreen in composite. Plus I'd like to have the ability to color it properly (so having the character in log). I would simply like to have it as non-destructive / flexible as possible. But I have no idea how does UE even approach such data, if it's able to keep the information in it if you rendered the greenscreen keyed plane in log and exr or something. Thanks again for your time and willingness.
I've found the solution. In Content Drawer, under "All" directory should be "Content" and "Engine". In my case, "Plugins" directory isn't visible under "Engine". On the right side of Content Drawer, there is Settings. Click it and check Show Plugin Content.
Hey there, friend, make sure that in your content browser you are selecting “All” folder, the root folder of your content hierarchy. Also make sure you don’t have any search filters. Type in “mediabundle” and you should see “BP_media bundle_plane_16-9” pop up! That is the component you need to drag onto your camera component under your cinecam as shown! Hope that helps 🥳 Working as of 5.4 stabile release!
@@vellg5813 don’t say that! This video is sped up and I’m not some robot with this stuff, I’m also still figuring it all out! I want you to know that about a year and a half ago I sat down and opened it up. Overwhelmed, confused, not knowing basics like how to navigate the viewport and getting use to hotkeys for transforms on objects. A part of me thought I would never get the hang of it. Take it one small step at a time. Everyone who uses this software is always learning, and there’s a lot of information out there to learn from. Don’t get discouraged, appreciate your small wins first. Start with environment design, that will keep you busy and learning for a few months!!!
brooooooooooooooooooooooo 4:41 i stopped watching and only came back an hour later after laughing for 1 hour non stop....... he said "thingy.... just look it up"" hhhahahahahahahahaha
cool video, but are you copying the voice intonation of ian hubert? The pacing / voice jokes were like as if I was watching someone trying really hard to copy his style lol
Ha! Yeah….I’m really excited when I talk about this stuff, have also studied him extensively, and also it’s proven to hold peoples attention, why reinvent the wheel!
Very Ian Hubert. Well done.
@@JWS1968 bruh don’t even…that’s insulting to Ian Hubert 😅 I ain’t there yet
the compositing & color grading are ELITE we need a video for those two skills :O !!!
Why?! Why this post doesn't have more views??? This is the type of information that truly helps others, speeds up the creative process, avoids mental breakdowns, and unnecessary firings.
Thanks so much, I need to keep going! Been busy but I know it’s worth it to get this out there! MORE CREATIVE DECISIONS!
@@andrewdileo need moreeeeee videos, fr this was great
@@414SHOTiT thank you so much. I’m on it!!!
Am blown apart so true there is no information on this kind of work flow....super thankyou to you sir...god bless you with more subs
Thank you so much, feeling very inspired to keep going on these!
@@andrewdileo you could only get this level of info from CGWorld JP. Thank you so much for sharing.
@@activemotionpictures thanks! More on the way, working on a project right now
Jordan Orme should see this
@@seorin. he’ll see it! He’s been watching them - we’ve been watching him watch them! We love Jordan’s vids
Very generous of you. Appreciate the pace
Really glad you found value in it! This has been a fun proof of concept that I should just keep going with these!
Every couple weeks I check to see if there is a new video for this workflow 😩, finally something...🙏 There is a ton of stuff and experience compacted into this video!
Thanks!!! High-budget project pressure sure is one hell of a drug. Learned the hard way! 😂
Really informative video...need more blender to unreal workflow about green screen compositing
I will most definitely make a follow up! Thanks for commenting friend!
hope they brought you back for their new video! if so, looking forward to a video on ATE THAT!!
@@andyjones2224 they absolutely did 🥹thnx for your continued love and support ❤️
great blender, i just started learn blender building environments and this is wild alien to me but intriguing
No problem, you will figure it out soon! Believe you can do it because you can!
This is an amazing tutorial, and XXL was a bop
Thank you sir, more coming soon! We’ve been busy!
@@andrewdileo You about to do it XXL 🔥
I am obsessed with this video! you guys are great
Thanks! Always happy to hear that!
Congratulations on a very good workflow!
Note: To display the "BP Media Bundle...
(Media Bundle Package)", you must activate the Media Framework Utilities in PLUGINS.
@@Vilaroucas honestly even that doesn’t work anymore for me. The best way to do this is to just create a project using the “blank” television/film template. And if it’s a downloaded environment, just migrate everything over to a new blank television and film template project.
Brother! This is the type of workflow I was looking for, using Blender and UE on a daily basis, can't wait to try it out! Thank you so much
@@cms3d really happy I could help. I can’t stress enough that if you make a hard push with this virtual production stuff right now, you will be leaps and bounds ahead of other people in 2 years time.
I’m doing a massive project and have been screen capping some more advanced workflows, so another tut soon!
@@cms3d another fun idea, which I will be doing a tutorial for, includes working with transforms on your root parent, you can do a lot of fun things like shoot someone in a static car and make it look like it’s moving, even if the shot has advanced camera movement in the original.
Really all you need is a solid track and a USD file and the possibilities are endless. Endless dude! Dream big.
@@andrewdileo Cool idea 😲, can't wait for the tutorial 🙏💚
Thanks for this. How did you learn all of this stuff!? You are the greatest of all time.
Combination of reading suggestive documentation and happy accidents!
🔥🔥 such a great video. Thanks for all the great tips
Dude my pleasure! Keep on that grind!
nice video man, i would really want to dive in a virtual production unreal engine carrier but i need a course of high value from a good calibre like you
You can do it! I will be posting a lot more tutorials now that I’ve wrapped up the big project I’ve been working on
I love you I have been wanting this you have a subscriber sir thank you
Let’s goooo!
your video really help me solving tons of problems !!! thank you so much!!! from now on i'm gonna explain how to make media plane like you. create a media player create texture and....just look it up lol
Do you actually need help with that, or is that a joke? Hope this truly was insightful! Happy to make a video breaking down the media plane in more detail if anyone is interested!
@@andrewdileo i do really please make one
thank you so much, this is amazing❤
Thank you for this!
Glad to help!
wow never knew you could do it via USD, always thought you had to use the blender to unreal script thingy. did you end up doing the composite in unreal engine itself or are you rendering out the plates from unreal and compositing elsewhere?
Yeh neither did I, this solution is definitely the best, believe me I have looked everywhere!
Compositing done in Final Cut by Director…but you can DEFINITELY export no problem with the MF KEYER node in the material you create! If you don’t, it’s important to tick off the render visibility in the details panel under your media bundle BP attached to your camera!
So sick what the hell
Noe just gotta follow it up lol
@@gronnip HA yes soooooooo soon promise. Later this month, got something super special for yall here
Great Video
Thanks so much!
Thank you for sharing your workflow, it's very rare to find content like this for Unreal Engine. Can a camera tracking plugin like fSpy be used for a project like this? And for final compositing and keying, do you do it in Unreal Engine or in post-production software like After Effects or similar software?
Thanks, friend, yes, you can absolutely use fSpy… The process would be to open your .fspy file in blender, add a floor plane and then export the scene. When you’re exporting the USD file, you just don’t really want to bother ticking “animation” checkbox if it’s only a static image!
Keying goes to post production software, but you can render it out right in UE5, just won’t be as clean! But for some shots, you can get away with it
It was so fast, but THANK
@@chrissoba5104 here to save time!
I get that doing the camera tracking in blender is probably what makes the most sense, but I wonder if it would be possible and/or make sense to use an iPhone (or maybe even an android phone) 3D tracking on the camera rig to get the camera movement information instead?
I wonder if that would make it possible to do everything directly in Unreal Engine directly?
(Provided I don't know anything about any of this and I am just wondering)
@@FuwakamiMana yes, you’re on the money. Using trackers that are placed on the physical camera is something that a lot of research and engineering dollars have gone into.
The problem comes when you’re trying to make sure your frame rates are synchronized. iPhones are hard to genlock, which is the process of synchronizing frame rates between devices such as audio recorders, cameras, and software, like unreal engine.
The other issue is that you need to store that data and keep it consistent with your various takes, which is a nightmare.
Lastly, iPhones do a very poor job accounting for micro-vibrations, which truly make or break a track. At least in any moderately professional setting, you want to make sure that all those tiny little bumps and jitters get factored into your camera solve.
That’s all I got on that! Of course there’s other various software that you can use, including syntheyes and 3-D equalizer. For me I just tend to use blender and when you really figure out exactly how to carry these shots over to unreal, you’re typically not disappointed.
@@andrewdileo thank you very much for this detailed explanation! It’s very appreciated! I learned a lot from this video, and while I don’t think I’ll ever attempt to make anything professional or realistic, I think I learned a few useful things! ＾＾
Awesome tips.
One question! If I am planning to color grade after rendering UE project, should the keyed footages be in ACES format?
(The footages played via media player)
Appreciate any help!
@@whasuklee great question man. The answer is shockingly simple: unreal engine is an sRGB native linear workspace. If you want to use an ACES workflow and want to do your keying in unreal or bring in pre-keyed footage you should load sRGB tagged footage into unreal.
When it comes time to export, just use your OCIO config file in the color management tab in the movie render queue. This way you’re not applying an ACES conversion TWICE. Only once consistently across your entire scene. Then everything should convert back to linear display spaces just fine. Hope that helps!
I love you
can you make a guide on doing it without all that gear ?
how do we do shadow catching or reflection catching
"This project has no default USD Asset Cache set"
When trying to impromptu in UE5 this error appears
@@user-yl6tm3cs3v I got you, I always click “don’t use a default cache” on that pop up. Since you’re importing, this does not matter. This is only if you are using the USD stage for persistent changes between programs.
Thanks a lot for this video! I was battling with this recently. To be honest, I am battling with UE overall, as I feel like it just keeps throwing sticks under my legs while trying to achive things that are super basic and primitive to do in most DCCs.
May I ask you what is your approach when adding shadows? Or overall compositing the characters into the scene so that there is a shadow, perhaps reflections. As this is only slapping the greenscreen on top of movement-matched 3D scene. Thanks so much!
Shadows are always a pain deal with, and there’s no set-in-stone method to achieve them. One method is with shadow casters- you can use the plane that is imported as the ground, and set up some rough geometry similar to your actors to cast shadows onto a white plane for compositing. Most of the time, Ben, who does the composing, will actually try to pull out the shadows that were actually on the green screen in the shot, and use those. If you have your actor inside unreal as an actual “plane”, that’s when you’ll get hyperrealistic reflections, without crazy compositing. You might just have to switch your material, playing back the video to “masked” for the transparency handling. I will make a video about this exploring some different options.
Maybe one day AI will do it for us, that’s the kind of stuff I would like to see AI be used for 🥴 shadow gen
@@andrewdileo Thanks a lot for your answer. Yes, pulling out the shadows from greenscreen works in some cases, especially for contact shadows you can't really get other way afaik. However, if there's for example a wall in the virtual space behind the actor (or tree, anything) and it wasn't there when shooting the greenscreen, you're out of luck as you can't composite the ground shadow on it. At least I can't imagine that.
About importing planes into UE, I tried it with Ian Hubert's method of scaling the plane with the origin at camera's space. However, I had so much trouble doing that in UE. Would really welcome a quick video on that if you ever managed to do that. I managed to make it work with some workarounds but it wouldn't render. The character would be behind the scene for whatever reason although in viewport it was perfect.
This however is too limited in two way from what I found: if the camera is slightly tilted downwards which it usually is, further foot will float in the air and although visually they are touching the ground in 3D, the shadow doesn't match. Second thing is that if the shadow is supposed to be cast from the left or right, it won't work as the plane is flat obviously. Moreover, I dunno if there's any way to composite it later that way. Having a separate render layer for the character, shadow and rest.
Sorry for such a long reply, I'm just kinda summarizing my thoughts and struggles trying to find some other possible way or combination of techniques. Really wonder how it's made in professional production.
Thank you in advance if you have any more input to this.
@@ArtemGms yeah, I have to digest this and honestly, I can definitely get to work on a video breaking down shadows a bit more in the workflows that I would opt for personally - if you want to achieve the Hubert method of scaling the plane in relation to the camera, so the feet are clipping the ground, the best way to do that is to do it in blender itself, and then export that with the animation as a USD file and then literally just use that plane as the plane that you’re projecting media onto…I just did it recently on a Jacob Collier project for the final scene in Mi Corazon. It worked out great. I should make a video about that as well. I think everyone has studied in Hubert and is trying to channel that in unreal and I have been devoting the past several months to learning how to transfer that skill set over I will make a video devoted to just showing how to do Ian Hubert stuff in unreal lol
@@ArtemGms I know what you’re talking about as far as the other foot floating in the air, one workaround is to have a duplicate that’s only acting as a shadow caster and then obviously you would just translate that down on the Z
@@andrewdileo Sorry for the late response. That would be amazing if you could make a video about that. Haven't tried USD workflow, I tried to do it in UE. Only did the track in Blender and it was terrible. That floating feet workaround sounds interesting but you then have to somewhat merge those 2 shadows, right? Which leads me to a 2nd question. Is it possible to do render layers? So I can even achieve something like that. Having environment, greenscreen character and its shadows on different layers? So I can perhaps add extracted shadows from greenscreen in composite. Plus I'd like to have the ability to color it properly (so having the character in log). I would simply like to have it as non-destructive / flexible as possible. But I have no idea how does UE even approach such data, if it's able to keep the information in it if you rendered the greenscreen keyed plane in log and exr or something. Thanks again for your time and willingness.
How much do you charge to make a music video?
@@kennethnoriega4546 I just do my part on videos for now. This was directed by Ben Proulx. I ain’t cheap, but I get the job done typeshiii
At 4:22, I couldn't find BP_MediaBundle_Plane_16-10.
Where can I find it? ...
I've found the solution.
In Content Drawer, under "All" directory should be "Content" and "Engine".
In my case, "Plugins" directory isn't visible under "Engine".
On the right side of Content Drawer, there is Settings.
Click it and check Show Plugin Content.
@@whasuklee yup, gotta click “all” and I believe it’s only in there when you select “film and television” as the project template
I can't see the BP mediaBundel file even with the plugin loaded. Did it get removed from unreal?
Thanks for pointing this out to me, I’ll look into! Only have the beta of 5.4 but I’m updating now.
Hey there, friend, make sure that in your content browser you are selecting “All” folder, the root folder of your content hierarchy. Also make sure you don’t have any search filters.
Type in “mediabundle” and you should see “BP_media bundle_plane_16-9” pop up! That is the component you need to drag onto your camera component under your cinecam as shown! Hope that helps 🥳
Working as of 5.4 stabile release!
@@andrewdileo I found it. It just seems to depend on what starter fie I use. Must be something to do with my install.
@@nickhenderson5781 that’s another thought I had. Use Film as a template - I always start with blank
@@andrewdileo good advice. one more question. is there a way to improve the quality of the plate on playback? Mine looks pretty average
OMG!!!! can somebody PLEASE guide me to the very beginning? i just heard about Unreal a month ago. From this video i will NEVER be able to use it.
@@vellg5813 don’t say that! This video is sped up and I’m not some robot with this stuff, I’m also still figuring it all out! I want you to know that about a year and a half ago I sat down and opened it up. Overwhelmed, confused, not knowing basics like how to navigate the viewport and getting use to hotkeys for transforms on objects. A part of me thought I would never get the hang of it. Take it one small step at a time. Everyone who uses this software is always learning, and there’s a lot of information out there to learn from. Don’t get discouraged, appreciate your small wins first. Start with environment design, that will keep you busy and learning for a few months!!!
@@andrewdileo thank you for the words of encouragement. I'm so tired of my videos looking like the 1980's. I will keep studying
Can’t import usd camera to unreal .. its sucks
@@ApexArtistX maybe I can help…what is your current procedure to import?
@@andrewdileo camera tracking animation importing into unreal from blender ..
@@andrewdileo blender camera tracking to unreal. Baked camera still no animation import its a bug in unreal
ok
brooooooooooooooooooooooo 4:41 i stopped watching and only came back an hour later after laughing for 1 hour non stop....... he said "thingy.... just look it up"" hhhahahahahahahahaha
I need a whole video devoted to that one thing haha!
cool video, but are you copying the voice intonation of ian hubert? The pacing / voice jokes were like as if I was watching someone trying really hard to copy his style lol
Ha! Yeah….I’m really excited when I talk about this stuff, have also studied him extensively, and also it’s proven to hold peoples attention, why reinvent the wheel!