10 Things You Can Do With 3D Gaussian Splatting & Reality Capture
Vložit
- čas přidán 28. 05. 2024
- Step into the future of 3D content creation with a deep dive into Reality Capture and Gaussian Splatting. Learn how these cutting-edge technologies are blurring the lines between the real and the virtual, enabling creators to produce photorealistic 3D models, dynamic scenes, and reality-bending visual effects with unprecedented quality. Whether it's for memory capture, heritage conservation, e-commerce, or virtual production, see how these AI tools are opening new frontiers in digital storytelling and content creation. Combining 3D scanning and Generative AI is a potent combination. Experience the magic of this tech that connects the physical and digital worlds, allowing you to transform imagination into reality.
Video Chapters:
00:00 The Reality Capture Revolution
01:05 3D Memory Capture
01:54 Reality Bending Visual Effects
03:08 Dynamic 3D Capture (Movement!)
04:36 Advanced Relighting Effects
05:34 Kitbashing in 3D Game Engines
06:38 Sharing Content Everywhere (Web & Mobile)
07:34 Editing Gaussian Splats & E-Commerce
08:26 Virtual Production & Game Dev
09:10 Heritage Conservation
09:33 Big Picture: Connecting Physical & Digital Worlds
10:48 Conclusion & What’s Next For 3D Capture
- Singularity Hub Interview on Reality Capture: singularityhub.com/2023/11/06...
Connect on socials:
- Twitter/X: / bilawalsidhu
- Creative tech digest (weekly newsletter): creativetech.beehiiv.com/
- Other links: beacons.ai/billyfx
#ai #3dgaussiansplatting #3dscan #gaussiansplatting #3d #3drender #photogrammetry - Věda a technologie
Amazing. Quick rundown of what's going on in the field in a very professional way that people who actually use these tools will understand and care about. very useful, thank you
This is an great overview. I remember using volume capture studios only like 5 years ago that cost millions and now we have access to all that practically with our phones it's insane! 🤙🏾
Thanks! I can't wait till generative ai and reality capture have a beautiful baby because that'll make the capture requirements even less strict
very clearly explained, thanks!
I feel like these captures on the Vision Pro will make it feel like you’re traveling without leaving your house. This is going to be huge.
Closest thing we have to teleportation!
Great Overview - well done.
Very informative!
This stuff is awesome. Me and a friend are just getting excited about potentially doing photorealistic 3D scans for real estate companies, and offering super realistic virtual tours on their website. Seems so feasible now!
Very much so! And great niche
Visual Umami is my new favorite phrase
Oh wow, these are all really interesting applications I'd never heard of before.
Reality Capture is a program for processing photogrammetry scans, your use of the term is maybe confusing?
I like Reality capture as a general term to cover photogrammetry, radiance fields etc
@@creativetechdigestyeah I can see the term turning into a catch all like “photoshop” is for photo editing. Also reality capture was confusing from the beginning with their early branding/marketing “capture reality with Reality Capture”
It was definitely confusing. Volumetric capture might have been a better term.
@TheDiervek I've been using capitalization as the differentiator. I've been doing photogrammetry professionally for over 6 years.
Reality Capture trademark is dead and canceled though. So, that says the facts right there.
amazing, thanks a lot
I want the mesh of it!!! Please make a video teaching to make a mesh!
Hey great video! Could you go into more detail of how you did the drone shot you did?
For 4d dynamic gaussian capture, the cameras have to be in synch. Is it possible to synch the footage from a bunch of normal cameras in post-production?
Thanks, man. Great video. Would you like to explore another topic and explain the Chat GPT Turbo? haven't seen a single video of a real test drive of it's fucntions
Cool video. I'm having difficulty understanding the best process for our resort & theme park project we are building in UE.
We have an 11 acre property with dramatic terrain that borders the forest and sea. We want to be able to view our site from all angles and elevations, so that the 3D environment looks as it does in real life, no matter if a character in UE is walking around the property or flying above it.
Can this be done with 3D Gaussian Splatting, or would it be done with some other process? Or, should we get a 3D Gaussian Splat of just our property using drone & ground photos, and use another process for the surrounding lands?
Newbie. Realism is the goal, as I sometimes see distortion with the 3D Gaussian Splats.
Thank you so much,
Austin
I'm waiting for a phone app that lets me synchronize multiple phones to capture volumetrically. They start recording at the same time and maybe have some kind of calibration to locate themselves within the scene. The multiple angles would allow for volumetric video capture. I don't see why this couldn't be done now, especially with the advancements in Gaussian splatting. Ideally we get to live streaming of 6DOF real-time animating volumetric capture of concerts and sporting events, but for now, it would be nice to have my family set up our phones to capture Christmas or a picnic or something, and then have that animated scene to watch and walk around in later in VR.
I totally agree. Having unstructured captured from multiple handheld devices would be ideal. I think the synchronization problem can be solved and we’ll see this in the new year,
Cant wait for gaussian splatting and google maps. Imagine walking in google maps just like in a game
Reality capture does gaussian splatting? On their website it seems they only to standard photogrammetry
They don't support it natively yet. But you can export your posed imagery from RealityCapture and train your own splats. Much better results than using COLMAP. Here's a good thread: twitter.com/jo_chemla/status/1698985750367281499
I am faced with a gray area
1. I'm shooting a luma ai object
2. download the obj file
3. import to blender
4. I get a file without reflections and roughness, just a piece of material
Question: maybe there are some AI tools to create automatic texture maps for my scanned object?
this is the main gray area that I am observing right now in the adjacent ALT TAB window on the PC
frankly, I need advice, I am grateful to any feedback, thank you, we are all somewhere in the same boat (it seems)
thank you! thank you for video
This looks promising for generating “flat” textures that would be relightable github.com/OpenTexture/Paint3D
wow@@creativetechdigest
This would be a cinematographers dream! Get the data in a location scout and then digitally light it and get real renders before even shooting that would be insane. Almost too easy though 😅
I truly believe this is the future. Capture spaces/places/locations and conjure them up later on demand as needed. I hope we see some progress on the relighting front this year! The next few ai research conferences should be exciting
Is macOS good for gaussian splat?
Check out the scaniverse app for iOS - it can now splats locally and has pretty decent quality
I wonder if this is something that could be used to map a roller coaster ride then create AR or VR games in the same trajectory as the coaster? Suggestions?
Are you try to do location based AR/VR at a specific coaster or generally reproduce roller coasters?
@@creativetechdigest Sounds like it would be the former. You put on the headset and then ride the coaster... and experience a game or visual experience alongside the ride. I wouldn't be surprised if Disney already has this.
Gran Turismo needs to use this to make a open world mode 😡 but this tech with Ai advances means the future of vr tech is going to be unimaginably awesome or bad 😂
Probably both, simultaneously 😂
Neat. And all I need I a half a million dollar camera setup? I still don't get it, until I can produce actually, editable geometry or use it to enhance a real 3d scan, I don't see a real purpose other than: its neat.
You only need a multi-camera rig if you want to record animated captures. Radiance fields can model subject matter (eg shiny, transparent, ornate stuff) that’s impossible to do with classical photogrammetry.
"that visceral feeling of....not being there"
Apple Vision Pro: challenge accepted
Reality capture does not do Gaussian splatting,, yet
True. But you can still export the posed imagery and train it in nerf studio or some other tool
Not a serious channel. So much BS in this video.
GSplats are not 3D assets, they are disposable novelties which might appeal to mobile users.
Photogrammetry has made an impact with the quixel/unreal asset library, but that took a lot of skilled effort to build.
Disagree. Splats are 3D assets - so are the point clouds we turn into splats or textured 3d meshes in the first place :) Photogrammetry has been a thing since the 1880s. Whether you turn a photo scan into point clouds, meshes, orthos or radiance fields - it’s all reality capture. Gaming folks in the 90s and 00s thought photogrammetry was a novelty for Hollywood. I wouldn’t write off other forms of 3D media so quickly - stuff moves fast these days!
@@creativetechdigest Well at least you believe the crap you talk, that's better than being dishonest.
What a negative attitude. It was actually one of the more informed videos we've seen, especially about our research.
@@infiniterealities4D Let me be clear, The term '3D asset' is something that can be edited and repurposed. Nerf's etc. are single use items which have no place in a 3D artists workflow, they have a different niche, mostly as entertaining novelties for end users,not for creators. Is that clear enough.
@@digbysirchickentf2315 I recommend doing some more research ;)
Queestion, just scanned something for my work. they are using recap for the pointcloud.. can i take that point cloud into capture reality? or how can i make those pointcloudt look better. thank you in advanced.. cheers