Video

Glad you liked it!

Insta360 One R - I've had it for about 4 years and have mixed feelings about it - sometimes it's great, other times it will freeze and shut down for no reason - and to my eye the footage is decent. For the price it's a good little 360 camera.

The 3DGS was trained on a cloud workstation instance with an A100 GPU. The video rendering and photogrammetry was done on a desktop with a Ryzen 9 3900X, 4070ti, and 32GB RAM.

Mavic 2

You need a photogrammetry software. In this case, I used Agisoft Metashape, but there are free/open-source options, such as COLMAP and VisualSFM

@@MatthewBrennan Thank you very much for that information. Much appreciated.

35mm equivalent is ~28mm. It's a 1" CMOS 20mpx sensor (for still images). However this dataset uses 4k video.

@@MatthewBrennan thanks for your answer. I'm trying to run your dataset using micmac photogrammetry. I'll let you know if I get good results!

During totality the moon moves in front of the sun (it's the black "hole" with sunlight shining around it) - when it's not occluding the sun, it's not visible (because the light is coming from behind it)... Unfortunately, the exposure settings on the 360 camera don't make it possible to really see the moon occluding the sun because there is enough light to still overexpose that area - but the shift from full/partial sunlight to eclipse is still pretty dramatic.

No - the photogrammetry is a DJI mavic 2 and a Sony A7Rii. Although, as I was watching the 360 footage, I think I will try to digitize one of the grave markers using the spherical video (the one that I walk around completely).

Thank you! I was worried about ghosting on the greenhouse structure/mullions, but I must have had it dialed in perfectly :)

Haha, you definitely went on a challenge on that one, don't worry I've turned the stuff upside down all the way around... Flawless 😉 Sound atmosphere is also very immersive... I'm about to scan a very similar one... I'm concerned about the hours of deleting reflected points I'm gonna have to spend 😅

Agisoft Metashape - it's an industry-standard photogrammetry software (formerly called Photoscan).

It has an interesting backstory too! The legend goes that a utopian community wanted to hollow out the rock to use as a church - and even went so far as to begin chiseling a doorway (you can see the opening in the video/model).

360 imagery won't give you results like properly captured "photogrammetry"-style data - the typical 360 sensors are small, and you're cramming a lot of data into not-many-megapixels. If you absolutely have to use 360 data, good even scene lighting and a good 360 sensor are key, more than the processing technique.

You'd want to use LiDAR or structured light, or a combination of LiDAR and photogrammetry. I have digitized wrecked cars before using photogrammetry - and LiDAR is often used to document crash scenes.

I used a Mavic 2 for this model. I have used a number of different drones for photogrammetry in the past, but haven't tried the Mavic 3 yet, although I don't think the Cine model adds anything particularly useful for traditional photogrammetry.

no, probably not, because the Gaussian splatting is based on image data, not LiDAR data. You could use a photogrammetry program to align your LiDAR datasets to imagery, though.

No. I tried it about a year ago but didn't find it very compelling.

I’ve thought about screenshot-ing Google earth to create the rooftops/fill architecture, but the quality just isn’t high enough- I think what you’d want to do is get some reference materials for the paving/ground and model that by hand, then PBR texture it

It’s a panorama I took using a UAV directly above church rock.

Not sure I understand what you're asking. NeRF/3DGS are completely different from photogrammetry, the only similarity is that they use the initial camera pose estimation. NeRF/3DGS at the moment (afaik) don't have a quantifiable accuracy and shouldn't be used for anything beyond visualization.

I made another video explicitly comparing the two- the photogrammetry model actually turned out pretty well.

In my experience, still images (photographs) work much better than video! Follow good photogrammetric practice for capture, and then process as a 3DGS.

The Nikon z50 would likely be the best (physical shutter + megapixels), although of course it depends on what lens you are using. In my experience, I get the best results using a high-resolution mirrorless camera (compared to an iphone or action camera). Of course - more data = longer processing times, so there is always a trade off or compromise.

If you had multiple 360 cameras you could mount them vertically on a pole and move that through the scene, yes (getting numerous heights would be key). Another solution would be multiple mirrorless cameras mounted at 3 heights, so something like 12-15 cameras total, firing simultaneously as you moved through an environment.

@MatthewBrennan oh, I forgot to specify, but I was thinking about regular cameras (non-360). My bad. But thanks for answering, I might try it one day for a long shot with no cuts.

The remains of Hadrians villa are quite well preserved, and this is based on numerous archaeological monographs, as well as the input of various experts. No reconstruction can be 100%, but this is the best given what evidence is available.

DJI Mavic 2

Yes all 2k processed in one go, though I did downscale them to prevent running out of VRAM (40GB A100). The point cloud is just a screen-capture of the model in "Capturing Reality" (a photogrammetry software).

Ooooh good point - I don't have much experience with OBS, I'll look into this. thanks!

I've seen some cool examples of the glass/crystal interior laser-etching.... it seems to work well for objects - would be curious to see how a space looks.

Good question - the point would be if you wanted to get a mesh that was as close to the "real" object as possible - with hand modeling, you're almost always going to be approximating unless you are working from engineering or design drawings. Most of the photogrammetry I do is for cultural heritage/historic preservation purposes, where the digital model needs to be as exact as possible to the original, and often not of such "simple" objects as a cup - but instead complex organic forms like a sculpture of a human or animal (or both, like a centaur or faun!) form. In those cases, it is much faster and more accurate to use photogrammetry and clean up by hand, than to sculpt from scratch using reference. You will almost always have to do some manual editing or cleanup with photogrammetry, if you want a good "clean" mesh.

pickup truck! :)

Sure send me an email.

Yep, essentially :)

@@MatthewBrennan In that case subscribing as I don't want to be left behind on the info XD (and difficult to keep track on everything) . I suspect that when a good and quality method for converting gaussian splatting to 3d mesh comes out, I will see the news here to keep up to date . Cheers mate !

By the way I trained it on 3090 for 10-20 hours.

Yea, and I suspect the way that kiri engine is doing it is not really neus, but probably some kind of trick with masking based on depth.

@@MatthewBrennan Still, if it can be done with photogrammetry, it would be the best choice especially if you own a DSLR.

@@michaelwhite4810 for the objects that I picked in this video, structured light would be far better than photogrammetry. I specifically picked objects that I thought would be difficult for photogrammetry, in order to show the potential posed by NeuS (and because photogrammetry has become so popular and accessible lately).

@@MatthewBrennan But still, it would require camera pose, sometimes it's so bad that even colmap won't work for it.