Hololens 2 - A developer's perspective
Vložit
- čas přidán 17. 07. 2024
- I have been developing various apps and prototypes for the Hololens 2 since the start of 2020. In this video, I'll give a no-nonsense introduction to the Hololens 2. Get to know how it works and what it can do.
Thanks, Thanks and Thanks again. As the say it is a tough job to explain in simple way and You did a great justice to this. I would like to learn more and already subscribed . Looking forward to a great deal of learning
Thanks for the kind words!
YES! I’ve found your channel! 🎉
The Microsoft Hololens 2 is unfathomable to me. I am absolutely in love with the concept of mixed reality and feel that even more than virtual reality, mixed reality has a lot more utility in the real world such as you said earlier with placing a power point presentation on the wall.
Im in love with AR too, just the thought of all the possibilities gets me hyped! That’s also why I’ve applied for a masters degree in AR, I definitely want to do this, i feel like its my calling
Yeah it is exciting times. So many brilliant minds working at it. Look at Snap, Niantic, Oculus, Microsoft, Apple, Google... Exciting times ahead!!
@@BastiaanGrisel The technology is astonishing and certainly fun to play with!
Meanwhile, there are a billion people or more without access to clean, fresh water; fished-out oceans with continent-sized masses of plastic, falling populations of numerous wildlife, deforestation, a million new humans every five days, generating waste, tech devices being upgraded and replaced every few yrs, creating toxic e-waste, and many more problems being generated.
And technology continues to further the human break from the natural world, as we continue down a predictable path.
We are indeed entering a new dark age.
@@commanderthorkilj.amundsen3426 I couldn't agree more. Some of the best minds of this generation are being wasted on developing meal delivery apps and other things that contribute zero to the things you mentioned
Great explanation. I've always wanted a computer on the go. Aka one you can use while being out in nature and walking. Obviously the input for this is too limited for my use case. So creating an input device for it would be what I would work on. Some kind of gloves which can simulate typing on keyboard while walking with key-press-feedback for fingers etc.
Why are you using virtual keyboards to type? Jebus, use your voice. 🤣Actually why would you be walking in nature with this contraption upon your head?
Thanks for the video! Great to see your perspective on Hololens 2 (an 1). I would like to know your experience with WebXR and your perspective on this platform for Hololens 2. Thanks in advance!
With all the new headsets (both Virtual Reality and Augmented Reality) distribution becomes a real pain. You have to create a separate HL2 version of your app, an Oculus Quest version, a Magic Leap version and then also an Android and iOS version. Android and iOS are mature platforms, but all the other platforms are changing constantly so it's very difficult to create one app that generalises to all these platforms. WebXR promises to provide a single distribution path for all supported devices. Nowadays, phone-based AR is used mainly for marketing campaigns, where people don't want to install an app on their phone. WebXR is a huge plus in terms of accessibility and ease of use for the end user. So I have high hopes for WebXR. It's great for HL2 as well. The device doesn't have the processing power to render high fidelity graphics, so all the fancy lighting effects etc of e.g. UE4 can't be used anyways. A person wearing a HL2 just needs to look at a QR code and the experience launches right away. Awesome.
Ilike so much hololens 2 so amazing ideas
Hi, Bastiaan! You made great insight into what Hololens really is, thank you. I'm doing research on opportunities of VR/AR application in vocational education, specifically in carpenters' and construction workers' education programs. As I understand nothing in app creation, would be nice to get to know what skills one should have as an educator to create VR/Ar interactive content based on a real-time in situ environment.
Thanks! That sounds really interesting. Creating apps for the Hololens 2 is usually done using Unity (unity.com/). Unity is an application that is used to create video games, but it also used to create apps for VR and AR. Depending on your background, it's not that easy to learn Unity (but certainly doable!). Another option might be to look at the functionalities that Microsoft provides out-of-the-box, like Hololens Guides: czcams.com/video/V8c3pDKdHEc/video.html
Thank you for the great video! However, the company I work at is interested in this technology, however we want to make sure that it would be applicable for remote Maintenance Training. I assume that my trainer and I would need Hololens2 headset, so he can see what I'm doing as he is providing instructions and validating that everything is connected correctly before going to the next step. Would my trainer have the ability to show me a hand tool, that I don't have and show me how to use it on his equipment at his site? Please confirm if I'm correct on my assumptions mentioned above. We are currently using a WebCam for this type of training and the cable makes it difficult, along with having to carry the laptop. The quick movement of the camera causes dizziness and headache. Any help or advise will be greatly appreciated.
nice video! did you need to customize the device to work with your glasses or prescription lenses or does it work out of the box?
It works out of the box! As far as I know, there is not adjustments but there is enough space to also accommodate thick glasses.
Is the special mesh data accessible through API and can your route it to you own personal server such as GEO location?
Hi Peter! The spatial mesh is indeed available as a 3D object. You can download it via the web interface of the HL2 or you can write some code in Unity to save the 3D mesh as an obj file. This obj file could then be (automatically) uploaded to a server.
Thanksk for the video review. I am waiting for My Hololens 2 and am curious about two things: If You stick an image to an wall and then shut off the hololens 2 for a few days and restart it will that image be at the exact spot You left it? I am curious as an artist if refference images can be applied on an artwork in progress in the studio. Also how steady stay images/3dobjects in 3d space? Do they flicker, or change position because of recalculation of their position sometimes? This has to be exact for my workflow.
In theory yes, the HoloLens remembers where you have placed holograms/applications. But if you would like to annotate an artwork you are probably looking at either a custom application or maybe Microsoft dynamics assist. The tracking on the HL2 is really good and holograms stay fixed in space (in my experience).
I've read somewhere that the 3D tracking and placement of holograms has a few centimeters of accuracy
@@MihaiZavoian Yeah it's not millimeter accurate but overall I do think the holograms are fixed in space quite well
This could be huge for service industry... going out with a complete manual and guide on how to repair equipment and it scan the environment/equipment and tell you how to fix it/ repair / tear down etc.
interesting we are testing these however find the displays poor, with some degree of colour overlap, blurred text etc. How is the display on yours?
The color reproduction and uniformity is poor on mine as well. I do find that you can design around it if your application doesn't rely on color too much and if you don't use large smooth/uniform surfaces. I find text to be well legible (although it should be placed at the appropriate distance).
Amazing ideas technology hololens 2
Question: are you able to stream a view of the holographic world like the recorder you used in this video, but in real time? Perhaps with Microsoft Teams or a similar service in the new Chromium Edge?
Hi Christian, yes there are multiple options. Microsoft is pushing Remote Assist where someone with a HoloLens on can dial in an external expert via Teams so that they can see the view through the HoloLens as well. This could be useful e.g. for remote maintenance. Another option is the Mixes Reality Capture, which streams a live feed (a bit delayed) to the browser and you can view that on the samen network as the HoloLens is.
@@BastiaanGrisel Now you can just download a app called "Microsoft HoloLens" in the Microsoft Store in your pc, and just need the same internet environment. But sure for a little delayed.
There's so many possibilities with VR it's insane. Anyone who says it's not cool isn't creative enough. You could map a whole puzzle escape room style game using virtual objects around your house and have people try to solve it there's so many cool things i want to do
Hope this makes its way to the VeVe App/Platform....awesome video 🙏🏿
I hadn't heard of VeVe (play.google.com/store/apps/details?id=com.ecomi.veve) but it would be sick to see them in life-size!
🔥🔥🔥🔥
Great video, thank you very much.
What are the alternatives to Hololens 2 (considering same (or larger) target market)? Thanks!
There is no drop-in alternative, every headset has its own pros and cons. I'm curious about the new Magic Leap, there is the nReal Light and the ThirdEye Razor looks promising. The Snapchat AR Spectacles of course have a very wide audience, but those are very much still a beta product. For professional use, there is also the Varjo XR3 that is a VR headset that has cameras in it so you can use the real world as a backdrop.
Hi man. Nice presentation. Do you think MH2 can receive signals from automation controllers and servers ( Rockwell PLC and wonderware DA servers ) to present these realtime values in 3D grafic, populate tables and so on? Is that done in Unity?
Not out of the box but this is exactly the kind of apps I'm developing with my company. Usually Unity is used for this but increasingly WebXR as well.
Wow, so this is the market that apple is trying to enter. The future looks very interesting.
Use case idea: Pilots landing an airplane in low visibility conditions might benefit with AR visual augmentation as a navigation aide, outlining the runway, or fly through hoops overlay in a head up display. As well as seeing other plane traffic overlaid in pilot’s field of vision. ADSB and map data feed required. Goal, to create safer, all-weather flying and navigational, situational awareness right down to touchdown. Is this a step closer to full automation?
AR in the form of heads-up displays is implemented in fighter jets. I remember a study (but can't find it atm) that did experiment with AR in commercial airplanes where the indeed had to fly through loops etc. What they found is that pilots would be so focussed on the system that they e.g. did not notice if there was an aircraft already on the runway they needed to land on. So in order for these systems be usable, they need to be very flexible and powerful to adapt to all of the possible risky edge-case scenarios. But I agree with your point fundamentally, we can use tech like this to augment human intelligence.
Hey Man! (ontdek ik een Nederlands accent? ) - How good are the cameras for object/qr code recognition? I have a use case where we have a warehouse with items labeled with qr codes, do you think the camera can read these at some distance so I can overlay an info card?
Goedemorgen! The Hololens 2 is able to read QR codes (docs.microsoft.com/en-us/windows/mixed-reality/develop/platform-capabilities-and-apis/qr-code-tracking) but we'd have to test it to see from how far it can detect them. Microsoft has also released Object Anchors (azure.microsoft.com/en-us/services/object-anchors) that can recognize entire objects and overlay digital information. Feel free to shoot me a message at bastiaan@bastiaangrisel.com for more information!
@@BastiaanGrisel Thanks man! Just received it! This thing is awesome!!!! Gonna play around and may get in touch!
Are there any plans for making the Hololens available for learning a foreign language? I think it'd help a great deal if the Hololens could show an augmented reality "teacher" that could verbally interact with the user, and instruct, correct, suggest, foreign words & phrases for someone learning a new language.
Great idea! I don't know if there are people actively working on this to be honest.
When you click menu, your hand is under the hologram. Shouldn't it be on top when you click the button?
Yeah ideally this would be the case. Currently, everything is shown on top of your hands. It is not as distracting/weird as it might look though.
What are the practical uses of this headset, if any? Do you find yourself walking around with CZcams or Netflix open following you around? What are the colors like? Are they clear and blended well? Also, does this device have a GPS built in? Sorry for all the questions. Awesome video!
The use case you're describing would be brilliant but no I don't find myself walking around with it. The HL2 is more of a business device that tailors to very specific needs (e.g. where people need their hands to be free in their day-to-day work). Remote assistance (e.g. for maintenance), healthcare (where internal structures like bones are shown in AR during an operation) and training are three cases where Hololens is currently used. The color reproduction is poor unfortunately, we have to work around it when designing experiences. I don't think the HL2 has a GPS.
@@BastiaanGrisel Thanks for the response! Do you know if there is a place where an individual could rent/try a hololens before buying one?
@@drewmileham202 I'm not sure. I know that there are companies renting out their Hololenses for people to try them out but I wouldn't be able to name them.
what is your used method for object persistence through sessions? WLT?
Azure Spatial Anchors mostly. World Locking Tools looks cool as well but is in essence just a linked collection of spatial anchors (microsoft.github.io/MixedReality-WorldLockingTools-Unity/README.html).
Have you attempted using a keyboard and mouse with this? I'm considering using hololense as i code in visual studio. Does it have remote desktop capabilities or would I need to use a web browser to control a remote machine?
I don't think the displays are high enough quality to look at for longer periods of time. I haven't tried it but the Lenovo ThinkReality A3 is built for this purpose so that might be worth checking out.
Government departments in my country still keep documents as paper in drawers...
In Singapore, surgeons used hololens 2 to generate hologram of the patient heart to aid them in heart surgery.
After watching about a dozen videos about Apple’s new Vision Pro, I was getting some déjà vu moments. Your video helped remind me where that’s coming from. There are so many cues here that I think Apple used to design their Vision Pro. And even uses Unity for game development just like Vision Pro. And in some aspects the Halolens 2 is actually better, like having all of the hardware and the battery in the back to balance out the headset.
The Halolens 2 also has richer interactions that the Apple Vision Pro does not yet support such as the physics engine you showed when you picked up the blocks or the shared window you can pin to the wall that others can see. It’s funny because I’ve heard so many reviewers of Apple Vision Pro ask whether you can watch a movie together in your living room with someone else wearing another Apple Vision Pro. And the consensus was, it wasn’t knowing whether that would be supported. Yet the Halolens 2 already does it!
And on top of all of this, amazingly the price of the Halolens in the United States, was $3,500, the exact same price as the Apple Vision Pro!
Yes I totally agree. The Vision Pro might have its own Unity-based interaction libraries that tap into specific functionality but how that differentiates it from current interaction models (like with HL2) remains to be seen. There are some obvious flaws with HL2 like sluggish hand tracking, not-so-good field of view and bad display quality so the Vision Pro might add something there if it improves on those things.
This device is 3500???? The apple vision pro is so much better. This device isn’t able to tell if your hand is in front or behind a hologram without taking a few seconds to scan it and when you take your hand out of view it forgets again. The apple vision pro is just so much more refined and didn’t waste so much on trying to make it as light as possible. The Hololense is like trying to make a laptop in the 80’s. Like yeah it would be great to have it that small but it the uses would be so much more limited because of that size
If I am trying to train someone on the operation and software interface of a piece of equipment at a remote location, do the personnel at the remote location have to set up a camera and transmit the images to hololens 2 that I am wearing? Will they need another hololens 2 to see how I operate the equipment and the software as I fiddle with it through my hololens 2?
Hi Don! The classic 'remote assist' scenario is that the other person is wearing a Hololens and you dial into their Hololens from your laptop. You can see what they are seeing, talk to them and write directions (like an arrow) which they will see in 3D in their Hololens. I'm not sure how it would work with both people wearing a Hololens.
@@BastiaanGrisel Thank you for the clarification!
I wonder if this can map out the room into a 3D space to better innovate for clients
Yeah you could totally do that! Note that the mapping isn't as high quality as you would get from Photogrammetry (basically taking loads of high resolution images and generating a 3D shape from those). The mapping on the HoloLens 2 is mainly used so that we can place virtual 3D objects in the real world and have them stay where we put them.
Is there an application for music.?
There is no dedicated music playback app installed on the device. There are some third party apps available in the HL2 app store. Also, you can always open a browser and navigate to CZcams or Spotify.
❤
I am new to Hololens 2 Development? Can we use use Hololens 2 Commercial edition for Unity development OR its mandatory to have Hololens 2 Development Edition NJX0001?
Yes you can use the Hololens commercial edition for Unity development! They are the same device, only you're not allowed to use the development edition in production (it's only for developing and testing new applications). The reason some people prefer the development edition is that you could pay $100 for 36 months instead of paying it all in one go. But for many people, the commercial edition is the right choice.
So what are your thoughts about LYNX R1 VR/AR headset.
From what I've seen it looks really awesome. It seems like the industry is focussing on passthrough VR as a means of realizing AR (varjo, quest, etc). And I think that that is a good idea since the HL2 shows how difficult it is to make large and high-quality transparent displays.
Hi Bastiaan there are any options for payment? like monthly payment?
There used to be an option where you could pay $100 per month for 36 months instead of paying $3600 up front. But for some reason I can't seem to find it on the Microsofts website anymore
Wow very nice video! Thanks for youtube that they recommended your channel to me. I subscribe you here in Korea and I’ll wait for your next video!
Hello ! I see some pepole using edge on HL 2 but it was very slow so how was your browser experience ?
The browser itself it quite responsive, it's not slow in terms of loading pages or something. It is a bit cumbersome to use because website are not designed for touch input so it's quite hard to press small buttons etc. Also, the background of website is often white and a large white area is not very comfortable to look at through the HL2's screens.
What are the pros and cons of developing mixed reality for hololens using Unity vs Unreal Engine?
Development is largely done using MRTK (Mixed Reality Toolkit). MRTK is available to both Unity and UE4, although it was originally designed for Unity and the Unity version (currently) has the most functionality. Since the HL2 isn't very powerful in terms of rendering power, you won't be able to take full advantage of UE4's graphical powers. So it's mostly down to what engine you have got experience with and other project requirements. It's new tech, so it won't be smooth sailing on either platform.
@@BastiaanGrisel Thank you for the response! Seems like Unity will be the best tool then for the next few years.
That would be my intuition. But as always, there are pros and cons to both options. And a lot can happen in a few years so no-one can really tell.
Wish i could have one
Do a video on voice commands on the hololens
Hi can I use this with real photos of things like planets I took ?
Not entirely sure what you mean but yeah you can totally view your photos on the HoloLens 2. With the new iPhones, you can even capture entire 3D models and view them on the HoloLens.
What I mean is I do astronomy I take pictures of planets also video can I view them in hololens
Yes you could most certainly view them in Hololens!
@@BastiaanGrisel so you don’t need to make ur own software or app just plug and play?
There is a photo/video viewer app on the device which you can use. So just plug and play
Must be super difficult(and interesting) to develop because of well.. the limitations, but also on top of that there is an inherent expectation that the interaction is instant. I wonder if you have tried to just slap another mobile phone at the back of the device to process the application layer and just stream it to the device? I mean why not use to your advantage the fact that all your customers have a powerful mobile phone in their pockets?
The processing power of current generation phones is not much better than the HoloLens 2 itself so that might not improve things as much. The HoloLens does support rendering content on a computer (either on the local network or in the could) and seamlessly streaming the results to HoloLens. Nvidia's CloudXR and Microsoft Azure Remote Rendering are two cloud based implementations of this (currently in beta).
@@BastiaanGrisel Yes I realize that HoloLens2 must be using similar (if not same) hardware as modern phones. My point was just to combine current HoloLens2 processing power and include additional processing power by utilizing the smart phone everyone is carrying with them anyway. So in the end you would have two sources of hardware processing.
But yes, probably CloudXR and Azure Remote Rendering is a better idea since those options can use the benefits of proper graphics cards.
Is the Input latency in real life as worse as in the video?
Yes it indeed is. I hope Microsoft will be able to optimize the code that does the hand tracking so that there is less delay but only time will tell.
@@BastiaanGrisel hope they catch up to facebook/oculus in that regard
Can we perform facial recognition with hololens?
How is the FOV
The FOV is quite a bit larger than on the Hololens 1, but it's not perfect yet. The edges are still noticable but it is not like you've got the feeling of looking through two small windows, which was the case with Hololens 1.
If it was cheaper I would get it. Build a VDI environment to access from Hololens and work from that
Lenovo is supposed to release their AR glasses focused on this virtual desktop use case: Lenovo ThinkReality A3. Release date sometime in 2021 for a yet to be known price (but probably lower than HL2).
I would like to see how the FOV looks like.
Yeah it's difficult to see without actually wearing the headset. These guys have made some images to give you an idea: mspoweruser.com/controversy-around-hololens-2-field-of-view-continues/
@@BastiaanGrisel Thanks. I found another video that demoed it as well. I'm really excited to see how big it will be for Hololens 3. I think I will buy the next version for private use.
Yeah let's hope they can crank it up some more! Fingers crossed
Heb je de hololens nog gebruikt? Het ziet er indd traag uit ten opzichte van de quest 2
Ja maar vooral voor proof of concepts. Er zijn wel wat grote bedrijven die er mee werken maar mijns inziens moet de use case wel heel goed zijn om de tekortkomingen en limitaties te rechtvaardigen.
How is the battery life?
It depends a bit on the apps you use but it should be around 2-3 hours
In the most recent windows update they massively improved the tracking of hands. And finally fixed the performance issues with web browsers and UWP apps so its finally usable for something other than a glorified tech demo.
How much is the delay? (Hand Tracking)
@@TechnoNerd The delay or drift that holograms have is only present in device recordings. I dont know why exactly but yea, in real use 99% of the time no drift.
@@shmarvdogg69420 Cool, thank you, so in real use (no recording) hand tracking has no lag? Sweet!
@@TechnoNerd For me there is a very very small delay but its 100% usable.
@@shmarvdogg69420 Nice, thanks!
Mixed reality IS the future.... if they can make it cheap enough for a layman.
Hi, i'm Leo, 19. I'm interesting in AR development. But, in Brazil, economy just don't walk as we that live here want... If I develop something, would you mind in test for me? I'll really appreciate if you help me. I have plans to get one but it's kind hard to do it here for now
Hey Leo! If you have an app that works on the HL2 emulator and send me the appx file, I'd be more than happy to try it out. This guide will get you started: docs.microsoft.com/en-us/windows/mixed-reality/develop/unity/unity-development-overview?tabs=arr%2Chl2. You can also make AR content for your Android or iPhone using Unity's AR foundation. And if you've got 300 dollars to spare (which I understand might be a lot) you could see if you can get an Oculus Quest 2. You can develop for the Quest 2 using the same MRTK framework as you use to develop for the HL2 and the Quest 2 also has hand tracking. So if you don't need the AR part, the Quest 2 would be a good option as well.
I hope you know they are not screen its lazer going in the back of your eye
True, the tech is really awesome! There is this great lecture by Alex Kipman about how exactly it works: czcams.com/video/S0fEh4UdtT8/video.html. But for ordinary folks it's probably easier to think about them as screens.
Are there any plans for a CAD program. Obviously it would have to connect remotely to a computer, as laptops are not even ideal for cadding. Would be cool if you could also import items into a CAD program like you are doing something that requires a permit, you use the lidar to make a blue print of your property and then input what you want to do that you need to be okayed by the govt let's say running electrical lines to a shed and once you dug the trench and laid the line and conduit just use the lidar and take that for them to examine. Inspectors would just need technicians to come scan a property and could inspect 3 times the properties a day, if the homeowner did not have the hardware to do it themselves, obviously make accessories like a brancioscope if there were questions for things hard to access. Be cool for engine work you want to see if something is worn in a transmission lidar is prob better than a camera. But being able to design would be cooler than inspection
Yeah that's a great use case. There are some startups working on this but not that I know of in the integral way that you describe here
I’ll wait till they improve it or make a new better one, graphical side needs a lot of work
Yeah it's not really ready for consumer prime-time. The use cases for HL2 are the industrial ones where high image quality is not really needed.
I can’t wait till a really creative nerd figures out how to put all these components into an Ironman mask or something .
This is the future of everything, it replaces TV, phone, computers, etc. In a few years we'll all be wearing these dumb glasses
So what's makes this what it is oh yeah MicroVision MVIS
The heart of this device is Kinect. Microsoft can order other holographic display if they want to.
The heart of this device is Kinect. Microsoft can order other holographic display if they want to.
sir i would like to have a chat with u
how i can contact u
Hi there! Feel free to send me a message on LinkedIn: linkedin.com/in/bastiaangrisel