NVIDIA Just Made AI Photogrammetry 1,000x Faster [Instant-NGP]
Vložit
- čas přidán 14. 02. 2022
- NeRF, ADOP, Plenoxel, now instant NGP?! we are at the peak of NeRF research. I did not expect finding a better solution for neural radiance fields would be such a trending research topic, and it's just getting started. This field of AI is going to be incredible. As of the time of publishing, another eye catching NeRF research just came out, I can't keep up with these.
Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
[Official Project Page] nvlabs.github.io/instant-ngp/
[Official Paper] arxiv.org/abs/2201.05989
[Official GitHub] github.com/NVlabs/instant-ngp
[Installation Tutorial] • Instant-NGP Windows In...
[Installation GitHub] github.com/bycloudai/instant-...
This video is supported by the kind Patrons & CZcams Members:
🙏Andrew Lescelius, Panther Modern, Sascha Henrichs, Jake Disco, Demilson Quintao, Tony Jimenez, dicefist, Shuhong Chen, Hongbo Men, happi nyuu nyaa, Carol Lo, PyroBot
[Website] bycloud.ai
[Discord] / discord
[Twitter] / bycloudai
[Patreon] / bycloud
[PayPal] paypal.me/cloudweather
[Music] Zeeky - Springfield
[Profile & Banner Art] / pygm7 - Věda a technologie
this gotta be my new favorite channels of 2022. AI everything, incredible.
incredible, but as boring as hell
i was waiting for you to cover this!!! i'm sooooo fascinated by it!!
Excellent video as always. Id be curious to know if photogrammetry photographing techniques would improve your results. Do you have a gimble? Right now im using 8k camera phones and it takes all day to render just a single scene/subject. I got your last tutorial on this subject to work id be curious to see if my own results would improve
Thanks for downloading this shit I was about to do it myself but could not be bothered with all the steps xD
The technology truly looks revolutionary when compared to common photogrammetry, but a shame the Mesh Export is so muddy!
Hopefully they could fix this soon ;)
4:45 I'm sure you did something wrong. With my experiment (on linux) the lego truck is extremely crisp and the mesh is almost perfect.
hmm interesting, i’ll check with the pytorch implementation if that’ll be fixed, just windows things ig
would u mind sharing ur results too?
Yes please share the results !
Did you have a google Colab?
@@bycloudAI I think it's highly possible it's a windows problem... czcams.com/video/wh28c_8NKUc/video.html&ab_channel=AI%E8%91%B5
I tried on windows and also got a pretty good mesh! Did you try to leave it on training like ≈30s?
Thank you so much for this wonderful video!
nvidia engineering always manages to amaze me wow
this would be perfect for videos/movies to watch in VR and could also solve the double vision problem when tilting the head, from which all VR videos are suffering from
These are the things that make AI exciting, perfecting techniques or revitalizing old ones. It's like anything that you intuitively think that technology could be able to do, AI is able to help realize. I've always thought about upscaling, and stuff like this, decades before it existed, and "AI can do it. " (The people designing AI). I guess, also this kind of stuff can be scary too because now robots have eyes. Or whatever.
This is an awesome video man!
Found first comment lol
So Making 3D VTuber models from 2D ones would be much more easy with such AI
Well, for creating a 3D mesh or scene you still need quite a handful of images from a different angles, but still the same object without any major changes. So, I guess, it's better to just make a model instead of drawing minimum 12 pictures from every 45°
@@nilaier1430 We just need some genius AI researcher to be able to reconstruct meshes with all existing data. A user will just need references meshes and telling the program what kind of object to reconstruct.
With cloud rendering this gonna be big change in Game, Real Estate, Filming productions and more
Thanks for the short presentation, it is nice to obtain summary like this. I can't find neither in the papers nor in the github the videos you show from 3:40 to 4:00. From these, I understand that we can run the soft with videos as inputs instead of pictures only. Could you give the source for these materials please ?
I always love when AIs start to get responsive and you get to see them work in basically real time. Makes working with them so much more intuitive!
They cease to be AI at that point. Then they are just trained models
Hi. Thanks for this review. You mentioned you had limitations due to your hardware? Can you please include your hardware specs in the descriptions? Just to give people a baseline.
As a fellow windows user, I highly recommend taking the time to set up WSL2 and get familiar with Linux. Linux can be obnoxious in a lot of ways, but it’s great to be able to work with it when needed. Bash also feels cleaner to me than cmd.
This looks like so much fun! The next step will be parrallelising these so supercomputers can render worlds with hundreds or thousands of nerf objects, and generate more from text prompts. Like come on lets get proper ready player one stuff going :D
I wonder if the results are better with RAW or ProRes video considering the program would have more data information to work.
Recreate those CG still shots from Fight Club with this!
What the hell, we can literally make a 3D model out of our existing pics & videos quick, that's insane
Or even 3D scene. You can have a whole ass room or even a stadium like their official demo. It's mind-blowing
@@bycloudAI Exactlyy! I'm looking forward to seeing this being improved as the time goes on, everything seems to be close to realtime too and it's terrifyingly amazing
@@bycloudAI im wanting to use this for temples in india, make a VR pilgramage lead by sanyasi's and everything
It alredeady existed a long time ago, its called photogrammetry. Whats huge now is the number of imputs and the speed of the result.
But how does this compare to photogrammetry?
this might be a stupid question, but could this be used on a vr headsets stereo passthrough camera like the valve index for instance, and run in real time rendering the scene in full 3d?
The render time of a single frame in FullHD is around 15 seconds. Not doable for now...
Mmm, how does this remind me of the braindance in Cyberpunk. Camera flight in the area outside the eyes. Very cool!
I may be cherry picking but you were saying that Ray Marching is used.
Correct me if I'm wrong: rendering of Neural Radiance fields happens via Volume Rendering.
From what i heard, there are methods that jointly optimize Radiance Fields and Signed Distance Functions for solid surfaces. And Ray Marching can be used to render surfaces from SDF estimations.
im not exactly an expert but this is what I found
en.wikipedia.org/wiki/Volume_ray_casting
yepp I've suffered same hours of setting up issues. The results are rewarding. RTX 3090 is the reference GPU for these workloads. Btw, how to export the video from the scene fly rather than from OBS?
Great Image and Render Samples but if you slow downs the narration somewhat I feel that it will help us 3D neophytes absorb this new important information more easily!!! Thx!
So take aspects from both methods, so you don't have that weird point cloud, or that strange texture disappearance thingy.
Hi, thanks for your work. As a reference can you tell us what GPU are you using? Thanks, really appreciated.
2:04 he says he has an RTX 3090
@@christophebrown978 Probably I've missed it, thank you!
Now add this function with stable diffusion then we can input 2d photo->3d model with deduced human joint skeletons->move to pose it as however users want->-choose prefered art style safetensor models->convert it back to 2d photos.(fingers/toes maintained since it refers 3d models with finger joints).
If it's possible then it will also be made into animations/movies when you provide sample movies to the character.
Is there any software for AMD Graphics? I would like to turn some images into 3d models too.
Can you imagine running all Google Maps pictures on this and getting a game like map of every city on earth...
Then you use it as a video game terrain, and voilà, you get the most realistic game ever...
That is one of my biggest dreams along with aging reversal. Imagine GTA 7 with the entire Earth in perfect detail and using materials from UE5 to make things destructible (using AI).
czcams.com/video/8AZhcnWOK7M/video.html
@@jimj2683 , it would require servers with real-time map streaming because the cost to download the entire Earths maps locally would be extremely expensive even if half the texture maps where reused for basic foliage and buildings similar to how Ubisoft does it with some titles.
@@captureinsidethesound That is exactly what Microsoft flight simulator 2020 is using.
@@jimj2683 , its also rendering only from a top view in which isn't that demanding on hardware, even on cell phones. They are using height maps that only give a "3D" look from a distance.
Thanks!
I just want this tech available on non-RTX hardware in the future and not come with a catch of having to buy a Nvidia GPU for this niche.
this is wild!!!!!
Have you tried output your video into images of tiff format
Can you please try this on a image set of a Hollywood actor? It would be soo cool to make 3D models of characters.
That not how this works lol
@@EddieBurke Why would it not work on faces?
That's illegal. Or at the very least, unethical.
@@sayrith Why is it unethical? 3D models is art the same way as a 2D drawing is.
Loving every video, AI is gettin sweet
How would .. or what would a 360 scene provide? By this tech ..
Hi did you able to enable dlss function ?
How does this compare to photogrammetry?
Ngl, of all artifacts, the clouds are probably the coolest.
Have you looked into the mesh issues yet? The guy who said that it worked fine on Linux posted their results. I'd really like to use this in my Windows machine but I'm not gonna bother if I'm going to have issues like that.
where is the "worked fine linux results",can you share the link? thx
@@hdl468357 "instant ngp demo" on youtube, you have to scroll down a bit. The channel is "AI葵". I wish I could link it but I hope this helps.
3:21 is this Tbilisi?
That's really f*ckin nuts
love this Nerf - NERV vibe, God in his heaven, Everything's right with the World 🙃👽👾
holy fuck. I love photogrammetry but this is on an other level. 2sec. you must be wrong. I got to try that out.
Can anyone share their Google Colab version?
Can you export from this?
Where can I download it?
Didn't they increase performance 10x recently?
Get an additional HDD (maybe even an external one) and install Linux there, even if just to run Linux-specific apps like you mentioned here.
So like I can’t get the obj looking normal ?
So these are like really fancy photo scans ? These are really cool - but I’m sure the topo is still really messy . So you can’t really rig or do dynamics with any of the stuff …. Super beautiful though . I can’t wait for the day where you can photo scan a person with auto correct topo
05:14 Amogus
sus
stop!!!!
@@Reafuse111 did you mean sus?
I mean, this is no surprise as 2023-2024 is the new new age of technology. If you're impressed with this, just wait till you see what we actually get.
Make rendering of "The Matrix" using movie footage
Tip, leave gaps between words
Its basically 3d scanning...
Imagine when they implement this to Google Earth. Put your VR goggles on and start walking around town.
How do these models handle photos of people?
5:01
pls make video how to install. I tried but its hard to understand
oh its in the description :O
any tutorials on this?
You should take a look at the Concert Creator AI. It generates a realistic looking virtual piano player
Fellas pack your bags we're moving to Mars probably next month!
TO THE MOON
New 3D saul😳😳😳
Can you make tutorial please
just imagine this technology with google streetview :D
in future probably ai will create entire city only with google street
When extracting google maps into blender ..when close up…The textures and meshes are so bad. If they improved those by just a little we could make movie scenes out of them. Sure from above with lighting and other fx it looks good.. but if you want to tell a story .. you want to be ground level from the camera angle.. and that right now is not possible.. I dont know why they just didnt use drones to capture the images when creating 3d maps.. they used satellites images and airplanes to take photos from different angles. I just think the quality from a drone especially with todays cameras.. and lidar.. would have been something special. But maybe that will come too. They can just send out multiple drones and auto map everything .. really fast .. It wont take as long as when the first started. The other thing we can do is replace some of the ground, trees, flowers, even watter effects from a ripped google map .. into blender.. etc. that also makes it nice for eye candy. Id like to make a documentary and I need footage.. I have all my aerial scenes.. just need lower scenes as if filmed from a camera crew.. if anyone has any suggestions about how to go about it. let me know. Going to these locations that I need footage from is not an option for me. Would the above tech be feasible for my project ?
you're concerned about resolution, but then you propose LIDAR as a solution. Stay in school, kid.
@@railgap I dont think you know what lidar does. Has nothing to do with resolution, you misunderstood what I was asking not providing a solution. so look at youreself before you make yourself look like and idiot trying to bring others down.
This volume illusion.
Star Trek/Doraemon technology gone true
Could be apply NERF on photos of telescope James Webb
1:42 how the heck you have chinese-cantonese english accent suddenly
So nref can't convert to a 3d Mesh
Apparently you can export as an obj
You know, 3D game devs hate retopoing and unwrapping models, and it seems like programmers and coders seem hell bent on replacing every aspect of designing stuff EXCEPT those parts.
Still think we're in base reality?
DOOR STUCK
Gamarjoba
WTF. Just install Linux like every free person.
Love your video! !! You are missing out - P r o m o S M!
WITCHCRAFT!!!
hi second comment
Third
lol evafans are so cool
Please review LALAL.AI's new engine. Maybe you can compared the results of the previous engine from your first video to the new one. You can also still use the old engine as an option, so maybe you can review new samples too.