ARCore Scene Semantics API For Unity Is Now Available!
Vložit
- čas přidán 13. 09. 2024
- In today's XR video, we will look at a NEW Google feature for Augmented Reality called Scene Semantics API, which is now part of ARCore Extensions for Unity. The Scene Semantics feature provides AR developers with a machine learning model for real-time semantic information.
💡 The API returns a label for each pixel across a set of useful semantic classes, such as sky, building, tree, road, sidewalk, vehicle, person, and more. In addition to pixel labels, this ARCore feature also offers confidence values for each pixel label and an easy-to-use way to query the confidence value of a given label.
📚 ARCore Scene Semantics For Unity Chapters:
00:03 - Google's ARCore Scene Semantics Introduction
00:17 - Scene Semantics Labels Demos (Sky, Terrains, Vehicles, Person, etc)
00:52 - Creating a brand new Unity scene with ARCore and Scene Semantics
03:50 - Adding a new "Overlay" Semantics Camera for Semantic Images Rendering
07:47 - Adding SemanticsUI to control semantic options
10:05 - AR Semantic Images Manager (Inspector Options Overview)
13:39 - Implementing AR Semantic Images Manager (Semantic Feature Methods)
25:57 - Scene Semantics Unity Demo running on Android
27:31 - What devices are supported with ARCore Scene Semantics?
📢 Watch the previous video on ARCore Streetscape Geometry API from:
• Geospatial Streetscape...
📙 Resources used or mentioned on the video:
- Geospatial Unity Demo Project shown on this video can be downloaded from:
github.com/dil...
- ARCore scene semantics documentation:
developers.goo...
- Lightship ARDK semantic segmentation (for iOS and Android):
lightship.dev/...
📌 Support me by Subscribing to avoid missing future videos!
/ @dilmerv
📣 Consider becoming a Patreon today:
/ dilmerv and GET MY “Full Source Code” Tier
💡 What do you get from Patreon ?
👉 Access to this video GitHub repo + all code I work on for each video
👉 Access to special Patreon discord group where I can answer questions
📌 Get XR & game development tips from me in Twitter
/ dilmerv
🥽 Learn & Get my XR Training from:
www.learnxr.io
👉 My Blog / 🔥 Newsletter (Subscribe to get up to date XR news)
blog.learnxr.io
#arcore #metaverse #unity #xr
📙 Resources used or mentioned on the video:
- Geospatial Unity Demo Project shown on this video can be downloaded from:
github.com/dilmerv/GeospatialCreatorDemos
- ARCore scene semantics documentation:
developers.google.com/ar/develop/scene-semantics
Hey Dilmer. Great video.
How are you able to simulate the scene on your Pixel phone in realtime?
I have tried the Unity remote app, but it seems limited in its functionality.
Owh they are starting to catch up to Lightship :o
Yes they are which is very exciting 🙏 thank you for watching !
@@dilmerv hopefully they get realtime meshing running next!
For gameplay interaction that is huge
Hi Dilmer, what amazing news! Does this Scene Semantics work with quest passthrough?
I am currently working on an MRI project with quest passthrough, and this information will be fantastic. We are actualy facing some problems with 3D anchors for aligment, and we were not abble to use ARcore with meta passthrough yet.
Hi Dilmer, this is awesome content!
Is there a way to contact you regarding an issue I’m having with AR Core’s geospatial API? My content isn’t anchoring to the correct position no matter what I change
Thanks James and you can join my XR community here: discord.gg/aJmkhJaX I know that some people in my community have been playing with this.
@@dilmerv amazing thank you! :)
I’m so excited to see where all of this new technology is going
Is this how we would solve the layering issue?
In unity geospace creator i placed an object in front of my house to test, however it appears through walls and other houses at a distance, instead of appearing behind walls and buildings. It's as though it's on the front layer and everything else is on a layer behind. It even shows up from up to 50-100m away in front of distant houses that would otherwise impede the view.
If this or another method would provide a solution, if you could make a video following up from your original ARCore video showing how to get layering working, that would be awesome!
Hi! I am running into the same problem, did you solve the issue?
@@elviremm unfortunately not, perhaps Dilmer has a solution
AR🎉🎉
Thank you for your comment 🙏 AR 🚀
Hey Dilmer, not that I dont enjoy your content, but I think I can speak for you community when I say that we would like more VR dev content. For example tutorials to recreate complex systems used in other games like the physics based hands from Half Life Alyx. Also it would be of more value to you marketing wise, since you would be able to use its publicity and pick up desperate devs like me who cant find proper content like yours on these topics.
Anyways nice video as always :).
PS: If you need help with anything marketing wise or other tech workload stuff I would be happy to help, since I can only imagine your stress after going self-employed and founding your learning platform.
Cheers,
Aiman
My UI looks like the Simulation. I cant hit the buttons and the camera doesn't work. Cant figure out why. Any ideas?
---
I figured it out. It is crucial to add the Input System UI Input Module as well as the Event System so the UI Elements work. Then the Semantic Info Labels I didn't need to add to the vertical layout because the code already does it other wise the say [Label] cause they don't contain the mapping name. Now it is working fine :)
Thanks for watching and for your question. Is this happening when you deploy the app to your mobile device? Or in Unity? Also what phone are you using?
@@dilmerv It is when I deploy it to my device. It is the Xiaomi 12. I recreated the tutorial multiple times but always got the same issue
wow thank you for this
You are very welcome 🙏