How to Use Meta Voice SDK to Improve VR And MR User Interactions #1
Vložit
- čas přidán 28. 08. 2024
- Today I show you how to control 3D objects in Unity by using Meta's Voice SDK which allows you to use AI to map to methods within your Unity project.
💡 We also go through the process of setting up an app with Wit.AI, creating new intents, creating new entities, and lastly adding a few keywords used in our Unity Voice SDK demo app.
📣 Consider becoming a Patreon today and get today's project !
/ dilmerv and GET MY "Full Source Code" Tier
💡 What do you get from Patreon ?
👉 Access to this video GitHub repo + all code I work on for each video
👉 Access to special Patreon discord group where I can answer questions
📌 Get XR & game development tips from me in Twitter
/ dilmerv
🥽 Learn & Get my XR Training from:
www.learnxr.io
👉 My Blog / 🔥 Newsletter (Subscribe to get up to date XR news)
blog.learnxr.io
#xr #metaverse #unity
📣 Consider becoming a Patreon today and get today's project !
www.patreon.com/dilmerv and GET MY “Full Source Code” Tier
💡 What do you get from Patreon ?
👉 Access to this video GitHub repo + all code I work on for each video
👉 Access to special Patreon discord group where I can answer questions
I love this kind of Tutorials that feature A.I. tools working together with Voice Recognition to make the Game / App more user friendly.
Very useful. Thank you Bro. ✌🏻
Thanks Alec and I agree AI has incredible potential and so many use cases !
Love the videos! I just had to know, is your VSCode the default theme or is it something custom? I appreciate the awesome VR content you provide my man!
Great video @Dilmer Valecillos! A few comments, at ~ 2:00 I'd recommend you use the $wit/number built in entity for your rotation entity and specify a role rather than creating a custom entity. It will do a better job of grabbing numbers
Aaron thank you so much for your feedback, I will do that next time and cover it on the next video !
At 13:24 for words that commonly get recognized incorrectly you can add synonyms in the keyword list on wit. I typically map queue to cube and spear to sphere for example :). I'd also use keyword instead of Keyword & Free Text for the entity type to improve results. With Keyword & Free Text it will accept new values outside of the set of values you have specified to match.
@@dilmerv You might also be interested in using the Conduit setting in your WitConfig. It allows for using attributes to trigger functions. Ex: [MatchIntent("set_color")] public void SetColor(string color){}
Thanks for the tutorial. Most of it works, but using the response handler the words[] array did not work. On my first attempt it contained the text 'value', whatever I would say and after trying the Meta tutorial as well the words array contained the intent name. But not the actual uttered word. The full transcript and partial transcipt works fine. Any thoughts on what I'm doing wrong?
There is no shape controller. Where do you implement that?
That’s a MonoBehaviour with that name and with the methods shown on this video, just add it to a game object and make sure you have mapped these methods your intents.
Thanks again Dilmer! Can you please also make a video on integrating Whisper by OpenAI with Unity?
Thanks for your feedback and yes working on it !
It works for me in the editor but it doesnt when i build it into my quest 2
You could use logcat which is available in Unity packages library to troubleshoot it and connect the logger to your device, more likely you are missing a permission.
Could you please create ar voice command image display using vuforia and voice sdk and wit ai
Thank you I will add this as a future idea !
nice bro
Thanks man !
How do we get past the limit of 60 requests per minute per app?
I recommend emailing their support to ask that question, thanks.
Hey Dilmer! Sorry to bother you again. Really want to register for your XR course. When will it be possible ?
Thanks for your interest, I am still outlining the course and more likely I will have an augmented reality development course within the next month or so.
Great video! Could u help me correctly attach an XR reference library from an assetbundle in AR foundation? I am able to get the library and attach it in unity but on android arcore complains that the library doesn't have any arcore data even though the markers are visible in the library in UNITY EDITOR.
Interesting, what Android device are you using ?
@@dilmerv Pixel 7, but there's new update the asset bundle doesn't pack the images in first place when it was tested on a different project. I was getting all the other library data but no images.
.NET and Unity. Both of them has access to built in Voic Recognition. So whats so much better using this EDK?
Awesome I am sure they do, can you send me links to those two and I can review them ? This one is built specifically for VR experiences which is why is part of Oculus Integration.
I ran into many challenges trying to do voice recognition this way on the quest. Quest doesn't have any voice recognition service installed in the android operating system by default. This service would be standard in pretty much any other android device, but it seems meta designed quest without it to force every dev into using their EDK and sending voice commands over the web, unfortunately.
@@DebrisHauler Yes whould be better to use the android built in voice reco. That is the one google home uses.
make mmore videos on unity with magic leaps 2
Thanks for your suggestion and I will be doing so very soon !