3D Scanning Changed Again. NeRFs Are SO Back!
Vložit
- čas přidán 14. 05. 2024
- Just when we thought NeRFs were out and Gaussian Splatting is in Google drops two papers and levels the playing field. Let's talk about SMERF and Nuvo and how they make NeRFs practical again.
Video Chapters:
00:00 Wait, NeRFs are back?!
00:54 SMERF vs. 3DGS
02:30 How SMERFs work
03:40 Editing NeRFs with Nuvo
05:40 3D: What To Expect In 2024?
AI research papers featured in this video:
1. SMERF: makes multi-room scale neural radiance fields practical, so you can render on mobile at 60 fps and load in additional rooms as needed. Not quite ready for world scale, but a step towards all of Google Earth as a massive NeRF we can explore. Project page: smerf-3d.github.io/
2. Nuvo: makes NeRF textures easy to edit in 2D tools like Photoshop or Firefly. You get cleanly unwrapped / parameterized textures and it works with real-life or AI generated scans. Before this paper, editing NeRF textures was mission impossible for your 3D artist. Project page:
pratulsrinivasan.github.io/nuvo/
Connect on socials:
- Twitter/X: / bilawalsidhu
- Creative tech digest (blog/newsletter): creativetech.beehiiv.com/
- Other links: beacons.ai/billyfx
#ai #nerf #3dscan #gaussiansplatting #3d #photogrammetry - Věda a technologie
great breakdown of this area!! also sick editing 👀
SMURF training time is still magnitudes longer than GS.
What i'm waiting for is 4DGS (4D Gaussian Splatting for Real-Time Dynamic Scene Rendering)
I bet that kind of emerging tech could be applied to NERF/SMERF too
Nice video with really cool editing. What programs do you use for editing?
Brilliant stuff ! Also great vulgarisation !
Thanks!
Definitely getting a like, comment, subscribe and rewatch for this one. Great video! I hadn't even heard of this yet!
Can you imagine what a Google Street View would be like with this technology? Being able to navigate the streets as if you were walking there, rather than just viewing a static photo as it is today.
🎉 thanks for this. I love how you present this amazing tech
No code! Damn that's a shame. I heard google is known for not sharing these with us common folk
Thanks! Yeah bummer on the no code but I’m glad the examples are public. I suspect people will repro it just like they did for DreamFusion. And consumers will prolly see it in Google Maps
These videos are so high quality, keep it up dude.
🫡
PLEASE MAKE MORE VIDS LIKE THIS!! SUPER FAN OF YOU!
🫡 more on the way - trying to get a regular cadence going
I'm subscribing alright, no worries 👌🏻
These are some pretty amazing developments, but where the rubber meets the road is in the tools that are going to be available to make use of all of this. I would love to see videos and links of actual tools that a creator can use.
Great video! Is your background virtual? It's sick!
Thank you! Real background, though I want to play with chroma key and unreal at some point
Sweet dude! Looked chill with the fire, thought it was a virtual enviro, Ha! Good stuff. @@creativetechdigest
Nice video
Papa SMERF is throwing down the gauntlet. Great explainer and overview
Thanks Joey! Loved your recent coffeezilla video btw
@@creativetechdigest thanks!
My head is still spinning from Gaussian Splats it can't keep spinning like this!
just subscribed! great video. Are they or do they have plans to release code to play with?
Thanks! Re- code release id guess in a year? They just release zip-nerf a while after it was published
Will it replace Matterport
I think nerfs and splats are already better than matterport. But the 360 pano + simplified mesh model paradigm has a lot of staying power in the industry.
Is the code out for SMERF?
Not yet. But they dropped code for Zip-NeRF (the teacher model) recently, so perhaps SMERF will follow? More here if u wanna dig in: twitter.com/bilawalsidhu/status/1749133782672670740
When will Google street view use this to become 3d? I want to play GTA on Google maps!
This looks very promising but how we can implement it in 3d softwares.
This continues to be a challenge for neural approaches to radiance fields versus Gaussian splatting
Come on luma .. do an update and add this method
I’m sure they’re cooking some stuff, though they seem to be doing a lot more text to 3D stuff lately
Dudeee, This is soo fuckinnn good 😮
Seriously amazing results. I spent like an hour just flying through the sample captures looking at the view dependent effects 😂
woe i thought i was watching a channel and video with maybe 5 million views but only 7.4k wow. Also I wish i could bring this into the game i play transport fever 2 😭😭
Very cool tech indeed but plz don't keep calling it "Reality Capture" (photogrammetry software)
Ha! I’m sorry I get that argument all the time but the phrase reality capture is way bigger than one photogrammetry tool
@@creativetechdigest Just use something else like Spatial Capture then
Just keep them away from Gargamel
😂 😂
No released models or implementations?
I think people expect SMeRF will be out soon-ish
Yeah the teacher model for SMERF - zip nerf just got open sourced. Hopefully SMERF follows soon