visionOS Development Fundamentals - Build A VisionOS App From Scratch!
Vložit
- čas přidán 19. 09. 2023
- Today, I bring you a new visionOS tutorial where I walk you through building an application from scratch using SwiftUI, RealityKit, and Xcode as the IDE.
❤️ Support on Patreon: / dilmerv
🔔 Subscribe for more XR Tutorials : / @dilmerv
🐦 Twitter X: / dilmerv
👥 Discord : / discord
📸 Instagram : / dilmerv
📌 This video will cover many visionOS SDK topics including:
- Creating a new VisionOS project with Xcode.
- Overview of VisionOS SDK tools in Xcode, including IDE panels, preview windows, and the simulator.
- Understanding how to create basic windows with plain or glass styles.
- How to create volumetric windows and differentiate between full immersive and mixed reality experiences.
- Understanding how to use and code a navigation stack and navigation links.
- Exploring available SwiftUI layouts such as VStack, HStack, and ZStack.
- Understanding enums in Swift and creating a view model to manage your app's data.
- Creating RealityKit RealityViews and interacting with Reality Composer Pro projects from Xcode.
- Adding audio to RealityKit RealityViews during runtime.
- Incorporating video using AVKit.framework into your SwiftUI views.
Here're the visionOS tutorial project requirements:
- Xcode Version 15 beta 8 or greater
- visionOS Version 1 beta 3 Simulator Runtime or greater
- (This is bundled with Xcode) Reality Composer Pro Version 1.0 (393.3) or greater
📢 Download VisionOS SDK From Apple Developer Portal:
developer.apple.com/visionos
📌 Sketchfab 3d models used today and links for attributions:
- SpaceX Dragon 2 exterior: sketchfab.com/3d-models/space...
- SpaceX Falcon 9 rocket: sketchfab.com/3d-models/falco...
🥽 Learn & Get my XR Training from:
www.learnxr.io
👉 My Blog / 🔥 Newsletter (Subscribe to get up to date XR news)
blog.learnxr.io
#visionOS #swiftui #visionpro - Věda a technologie
📌 For visionOS Dev tools requirements be sure to download the following from the Apple developer portal:
- Xcode Version 15 beta 8 or greater
- visionOS Version 1 beta 3 Simulator Runtime or greater
- (This is bundled with Xcode) Reality Composer Pro Version 1.0 (393.3) or greater
📣 For a full walkthrough of how to download and install the dev tools watch this video 👉 czcams.com/video/LeqVHfqRq_I/video.html
Also, consider subscribing to www.youtube.com/@dilmerv and hitting the notification bell 🔔 to avoid missing my next video & supporting me with making this channel better and better!
Thanks everyone and your time and feedback is always super appreciated!
Nice job, Dilmer! This video helped me a lot to understand the process of developing a visionOS app. Thanks, man. I hope to see more content like this. You're the best.
Lair thanks for watching and for your awesome support, I am glad you liked it and found it helpful! If you need or think of needing more content like this let me know 😉 I have more videos with native visionOS in progress.
@@dilmerv Not only me, but the Xr community needs more content like this. Onde more thing: I would like to ask if you could show us how to use the Lidar.
Fantastic! Amazing breakdown and use of all of the elements needed for a compelling experience.
Hey Dylan, thanks for your feedback and let me know if you ever need specific content 😉!
Love this Dilmer, thank you!! I hope you keep making more Swift + Vision OS content!
Excellent thank you for your amazing feedback! I will be sure to bring more content!
@@dilmerv Awesome thanks Dilmer! I did run into an issue. After 48:45, the app is not showing/running properly. I am getting a Thread 1: Fatal error: No Observable object of type ViewModel found. A View.environmentObject(_:) for ViewModel may be missing as an ancestor of this view. Do you know what's causing this?
I've been looking for a video like this!
Excellent I am glad you found this one 🥳🙏 thanks for your feedback!
The VisionOS support for multitasking UI panels and volumes in the shared space as well as fully immersive scenes is a big differentiator from the other mobile VR OS. I saw that Unity can render into a volume in the shared space, that's huge if true, typically Unity apps had to be fully immersive in VR. Also them empowering all iOS developers to easily become 3D AR devs using the same familiar technologies is a huge deal. We'll see a lot of immersive apps come from 2D iPhone / iPad app devs down the line, not just game devs. This could be the next big opportunity on an early platform, like the early iPhone / iPad days. I really do wonder with only gaze / click / gesture interfaces if we're going to see the Vision Pro get a smash hit like a "Beat Saber" or "Gorilla Tag" type of game.
I agree, there is just so much flexibility available with visionOS that makes me wonder how crucial it was for Apple to just wait longer than anyone else to release their first XR headset. They indeed gathered a lot of knowledge from many platforms today and did an amazing job in the implementation as always. Let’s build the next vital game or app 😉🚀
Bangin. This answered basic questions I had.
I am glad it did man thank you for your feedback, if you have others let me know as I am working on the next video right now.
really love this video as an iOS developer ,thank you Dilmer !
Awesome I am glad you like it and thanks for your feedback!
So psyched!
Thank you 🙏
I loved this video, thanks so muchhh ❤❤
Alvaro I am so glad you liked it, you are always more than welcome! Thanks for your feedback man!
Nice thank you for sharing.
My Mac kept slowing down the vision pro simulator which Mac book are you using ?
Hey John thanks for your feedback and for watching man!
I got an Apple MacBook Pro M1 16 inch and it honestly it flies through most applications. Let me know if you've other questions.
This is Awesome Dilmer! Glad to see Apple Vision Pro development for the first time! Will you get the device when it releases? Curious🤔?
Yes that’s the plan, as soon as it is out I will be running & testing all the demos that I am building, my plan is to have a very compressive list of videos for future Apple XR devs to have as a reference.
@@dilmerv Oof, my condolences to your bank account.
Awesome video, waiting for this. Question: Where did you learn Swift and Swift UI?
It was a combination of reviewing Apple developer docs as well as their published visionOS videos and source code. Great question!
I am able to load 3d files into the Simulator, but how do I make it such that I can then move those 3D entities around? My 3D files do not load with the white bar underneath like yours. Please advise!
Thanks for sharing! Everything works well until showing the 3D models part... and the sample code is quite differnet from the video and still digging through the codes. Hope we can have another videos explaining those changes!
Hi , This is truly impressive & it's clear that augmented reality offers an incredibly immersive experience. I have a question, and I apologize if it seems naive: I am planning to create a real time dashboard but dont know how to start one. How does one go about creating a real-time dashboard using this Vision OS design and how would the dashboard receive updates real time ? Would it be possible for you to create one video similar to this one where you can show a dashboard development (1 Bar or 1 line would be enough) .Looking forward to your reply
For a real-time dashboard which really depends on what information is displayed, you could use a web service (rest) or a web socket if you need to have the information available in realtime. I recommend reading about Swift with web services or web sockets.
Good question !
Thank you for your reply. Would it be possible for you to share some examples. Any link or reference would be great
im your linkedin connect and i follow you religiously on twitter, thank you for making this
I'm still mastering swift but this video has given me guidance on native development within the apple ecosystem
my question is, is native development for visionOS better than unity?
Thanks for your support man!
Unity or Native - the question is great, but the real questions you need to ask yourself are: would you be building an experience solely focused on Apple ecosystem, or do you see yourself supporting other platforms? Is building this a way to help you grow professionally, if so, look into jobs surrounding Swift vs C#. Also, is what you are building easier with Unity or Swift?
Try to ask yourself all of these questions and others, Unity vs Swift is a good question to ask, but both are very capable programming languages with very robust frameworks backing them up.
Best to you and let me know if you have further questions.
Great video, very good for beginner to start a visionOS project.
From my personal guess, Unity is not gonna support visionOS perfectly. Hope you can make more tutorials in native visionOS dev!
Thanks for your comment and I agree! one topic I want to cover is native plugins, I remember not having those available with iOS first got released by Unity and it was very limited.
@@dilmerv I agree, when I tried to dive deep into developing AR apps on iOS using Unity, it was necessary to create some plugins such as thermal monitor.
This video was very helpful, I followed it and used it to build my own Vision OS app about the planetary system, however I keep encoutering an error concerning the View Model "Thread 1: Fatal error: No Observable object of type ViewModel found. A View.environmentObject(_:) for ViewModel may be missing as an ancestor of this view." been struggling to solve it.
Hey thanks for the feedback. That error happens if you aren’t passing the environment object to your view. Here is an example of what may be missing from your main app swift file:
github.com/dilmerv/VisionOSInspiration4/blob/3e50a8d96a3bbd48f7d2da4df6de03ceb8859c84/Inspiration4/Inspiration4App.swift#L35
I'm not sure what I'm doing wrong here but it seems that my scaling is off in reality composer pro. I have input the same scaling sizes and placements for the rocket but it seems like the rocket is HUGE compared to the earth which is also affecting the particle size, all using the same measurements you're using. I've gone back a few different times to compare but am not sure what I'm doing incorrectly. Is there something I need to do with the udsz file beforehand or is there something else I'm missing? Thanks SO much!
did u end up figuring it out? I have the same issue, the model isnt even appearing and idk if its because its way too big or my scale is wrong
Excellent video up until the bit where the 'openWindow' and 'openImmersiveSpace' are not working for some reason. The code in the example is quite a bit different from in the video so I am stuck trying to add the model to the scene but am having no luck. 😥
Be sure to wrap the NavigationToAreas in a NavigationStack as shown here github.com/dilmerv/VisionOSInspiration4/blob/f99bbcea1a81603a718b0ff5821a2bd7b7d79e8d/Inspiration4/Areas/Areas.swift#L13
If i made an app with the visionOS and could i bring this app to other non-Apple headsets later on? Or is that not possible and have to build it differently, entirely from scratch?
I don't want to make an XR app for one company only..
And after the recent Unity Shenanigans, i don't feel i have much of a choice..
Hey great question!
Unfortunately you can’t use visionOS for any devices outside of Apple ecosystem, however as Apple grows their XR offering, you will be able to make it available for other Apple devices. Currently, you could also develop for iOS & also target visionOS which opens up the ability to build an app for multiple Apple devices.
For cross platform support on AR/VR, yes Unity is your best friend or you can look into Unreal or Godot if you decide going that way, it really depends what features you need on your app, if it is simple then the latter options should work for you.
Great video ! I'd like try to follow it myself but i don't own a mac and i'm wondering which mac hardware would be ok to develop for visionOS, without being overkill, what would be your pick ?
Thanks for your feedback, and I would recommend getting a MacBook Pro 13 with a M2 chip, I would say even the base model would do very well as they are very fast. Also, you could even go with a Mac mini today and those are great too, you can add your own monitor and the cost of entry is lower.
Thank you ! :D
Sup man, cool video!
I couldnt use the Rocket Capsule usdz model, it wont show in the volumetric window. In the other hand, If i run your project it runs correctly but your Rocket Capsule model seems to be modified or somehow different. This is not the first time it occurs to me using models downloaded from internet, so what I did was to replace the Capsule with one of the default usdz models that we have in RCP (reality composer pro) and worked fine. I know that the usdz files must be "adpated" or "enhanced" for apples usage cause thats what they say in the docs.
Anyway, Is there something you can tell me about that lol?
Hey man thanks for your feedback. I did have to edit the models a bit, if your model pivot point is way off then your model more likely is beyond the limits of your visible volume, I recommend importing those models into a 3D modeling tool before importing them into Reality Composer pro. That will give you a better understanding of what’s wrong with the model.
Best to you man!
I was able to follow until the reality composer pro part then after that it just didn't work when I run the app the volumetric didn't open when I click the button under the capsule, can you help me to find out why? also I downloaded your code to cross referencde with mine it had tons more stuff which was not included in this tutorial, hope you can explain the discrepencies thanks
Make sure NavigationToAreas() is wrapped with a NavigationStack, that should fix the issues, here is an example github.com/dilmerv/VisionOSInspiration4/blob/f99bbcea1a81603a718b0ff5821a2bd7b7d79e8d/Inspiration4/Areas/Areas.swift#L13
Could Apple Vision pro be used at a trade show or exhibition by a company to showcase B2B products? Can faceid unlock be disabled?
Great question, if I remember correctly you can allow guest to sign in but it takes 15 mins to setup for each, however I will give you a better answer once I get the device by Feb 2nd. Thanks again for your question.
I'm totally new to XCode (but have some exps in Unity). Wonder how the toggle controls the show/dismiss the immersive view? I tried calling openImmersiveSpace by `await openImmersiveSpace(id: "Immersive")`, or wrapped it with a `Task` closure, nothing works here. Seems like you have commented out the implementation of this part, and actually this is the most confusing part to me...Thanks
Hey there thanks for your comment, I will be making the GitHub repo open source today and I can share the code for you to compare it.
Hey @@dilmerv is the repo open source yet? I've ran into the same issue.
Hello and yes I just made the project open source, you can look at the code here: github.com/dilmerv/VisionOSInspiration4
Project is now open source github.com/dilmerv/VisionOSInspiration4 feel free to check it out and thanks for your feedback.
Thank you Dilmer! @@dilmerv This was an incredible course. I hope you keep teaching us more of your in-depth explorations with Vision OS, and swift UI.
Hi sir, just want to know if buying a Vision Pro is going to allow me to build and test Vision Pro applications?
On Apple website, it says need a Vision Pro development kit. Thanks a lot
Great question and you don’t need a development kit, there is no such a thing, there is a dev strap but that’s just to accelerate the deployment process to the device. You can do everything with the device sold today and deploy via WiFi.
So do you think everything you have shown here will be able to implement in Unity when SDK comes out
Yes, I am sure you will be able to do a lot of what I did here in Unity, I think is even easier to do. The part that I am not certain is how Swift UI will work, or even if we will have that flexibility with Unity PolySpatial, hehe I will find out and let you know for sure. I will get access to the Unity tools tomorrow.
Wow you are getting access tomorrow for unity tool, will you be making a tutorial for this 🎉🎉
Hi Dilmer! Clicking the NavigationLinks (Astronauts, Equipment, & Mission) in your source file and my project does nothing. Anyone else experiencing this?
Did you uncomment the methods at 52:16? Let me know thanks
@@dilmerv I appreciate the reply! I fixed the issue. In NavigationToAreas, the Vstack needs to be wrapped in a NavigationStack
Hey dilmer, iam a UX Designer. Want to shift my carrier to XR designer and planing to buy quest 3, i've been watching your videos a lot it helps me a lot. Can you suggest what i have do to shift my career as XR Design, i've searched in youtube, there aren't any guidance to this, it will be really helpful for people like us if put a video for xr designer.
Great question!
I recommend following “Daniel Marqusee” on CZcams who is a very talented XR designer: www.youtube.com/@marqusee/featured also I believe he works for Bezel which I also recommend following: x.com/bezel3d
Let me know if you have further questions, those two will be great resources.
Thank you so much, I appreciate your timely response.
@@manimaran6582 anytime!
I heard that using a Macbook is a requirement. What specs do you recommend?
I recommend looking at Xcode 15 requirements, mainly is all based on OS. I have a M1 Pro w/ 16 GB of Ram and my development is pretty smooth.
Here is a link to the Xcode requirements: developer.apple.com/support/xcode
Can you build these on VisionOS?
That would be sick. build it and launch an AR app right within the headset.
Hey Jeff! Yes this app will run with the real headset, I am trying to get one early on to test this out so let’s find out if I am able to! And I will be more than happy to test these with the real hardware. Thanks for your comment!
@@dilmerv - No, I mean develop on it too - even if in the browser. They could do it. Could go down to the beach, take a walk and code along the way.
@@jeffg4686 interesting yeah I don’t see why you couldn’t, you could push things to GitHub and once you are back to your computer deploy it.
@@dilmerv - No, I mean full cycle. They could build it with a backend service, send you back the executable to run. Do everything in the headset.
@@jeffg4686yes you can. You can wirelessly connect to your Mac in the headset, develop and then deploy the app to the headset . Still need a Mac. But using the headset to “display” your Mac screen
Do I have to learn Swift? Or unity with c# is enough?
I would suggest focusing on one and getting very good at it before learning a 2nd one, I would suggest thinking about where you see yourself in x amount of years? Do you want to focus solely on Apple XR? Or perhaps more of a cross platform solution? Do you plan to do freelance or work for a large company? Do you want to build a game or app?
Think about those questions and that will drive you to your answer.
Best,
Dilmer
@@dilmerv the answer is leading me to Unity, but i thought we could develop for vision pro with Unity and C#.
@@SahithNayudu awesome and yes you can, Unity is going to make PolySpatial as public in the near future and at that point you will be able to use that tech stack.
@@dilmerv That's amazing! Thank you so much!
so how can i use object-C to develope App for VisionOS
Great question, I haven’t done any objective-c coding with visionOS, I have done swift a lot but let me find out.
Can I develop on windows laptop?
Great question!
You could develop with windows (pretty much code) but the level of integration required will make you need to move to a Mac. All the visionOS tools also require macOS.
sir, can i run this in windows?
Thanks for watching and great question!
Unfortunately a Mac is required to develop for Apple products. Also, there are ways to do it on a PC but it would add too much complexity to the process, such as using a cloud build pipeline to generate & publish the build to TestFlight and such.
Not completely sure, but I think you skipped something between the 40 minutes. I click on the cards of the model and nothing happens. I checked the github page and the code is refactored, so it's hard to find what you skipped
Could you email me your project or join my XR discord server discord.gg/CvHNarkS & when you join send me a message, I can look at your project.
Thanks.
@@dilmerv Hey Dilmer! I went back to this video to update my comment! On 35:01, we set the buttons, but we didn't add the immersive volumetric action to open the 3D objects!
After all, this was a good thing, because it took me to a rabbit hole of tutorials and Swift lessons to learn more about Swift and Vision OS!
As always, keep with the great work dude! And Happy birthday man!
it is very disappointing after working for 1 + hour following but can't get the same output
I just sent you a comment to your other question with the solution.
We need native visual IDE similar to Unity to build this all without code in most cases (with code still available on demand just like in game engines). Connection between visual/estetical and technical parts are so close that you'll stuck in production forcing UI/UX designers bound to devs to get acceptable results. Wasting time of both professionals. We need building engine IDE/UX working natively with ARKit/Reality kit. Otherwise this is a dead end and nobody will use overcomplicated workflow for doubtful advantages of AR in 99% of projects. We have global problem of representing information in 3D either, but this is a problem for another day.
Thanks for your feedback, I agree that having a more cohesive workflow for UX & the SDK would be a win for Apple Vision Pro ecosystem, today you can do that with Unity through visual scripting and the visionOS PolySpatial tools.
@@dilmerv Don't get me wrong! I'm pretty sure that we still need complex scripting for complex multi-modal "under the hood" control because it's much faster and efficient to use raw code. It fits better than dragging hundreds of nodes along thousands of ropes in-between :) What I'm actually talking about is seamless integration of visual/hybrid/code approach for everything. Sort of "Houdini" style or any high-end animation packages and game engines (Maya, unity, unreal etc...). Because defining of size/color/icons of panel and aligning buttons on it by code is a bit stupid idea for 2023, isn't it ? At the same time defining complex behaviours visually is a total mess. We need both of two worlds here. Because in XR/AR we _literally_ mixing the worlds. New fresh paradigms are highly welcomed. And we didn't even started with UX :))))
Thank you so much for sharing your knowledge ! And please don't stop ! It's priceless ! We're hungry for more !!! :)
@@dilmervSpeaking of Unity : as you probably know Unity team locked VisionOS development behind paywall (1.8k+per year). And made something horrible with its whole pricing scheme what caused horrible backslash few month ago. Even CEO was quit (and leads toke their money off the business right before this decision). So Unity's feature is unclear at this point. And paywall makes it impossible to invest time and money in their ecosystem for amateurs.
Unreal from their side already conflicted with apple about compatibility and integration. Also Unreal is an overkill solution for 99% of projects "regular" devs will create for "regular" people. Taking in account steep Unreal's learning path and its dubious "open-source style" behaviour it feels risky approach to VisionPro too...
That's why using native Xcode/swift approach is the most prominent at this point. And it's also a general reason why people would watching YOUR videos :) Otherwise they'll start form Unity's learning courses instead, right? Right ? :)
Why not do it in unity? 😂
They are still under closed beta, I am hoping to get access to PolySpatial soon!
@@dilmerv sorry I was being more sarcastic, meaning the current unity situation, which has me considering the alternatives now.
@@tompainter7167absolutely man I understand! I am in the same situation as many many Unity developers around the world, for now I am just going to wait to see what happens when Unity announces its new policy changes. I have also a lot of content with Unreal + other XR hardware which in the meantime I will bring to the channel. Thanks for your comment and support man!
@@dilmerv THANKS 🙏
This went way too fast. you need to slow down
Thanks for your feedback!
You giving up on unity as well?
Hey thanks for your comment, right now I am focusing on visionOS native development to better understand how Unity will work with Polyspatial, but not giving up yet 😉