How to Make 360 VR Environments for Quest with AI - Stable Diffusion and Blender Tutorial 2023
Vložit
- čas přidán 19. 06. 2024
- To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/AlbertBozesan/ . The first 200 of you will get 20% off Brilliant’s annual premium subscription.
STABLE DIFFUSION LINKS
-------
How to Install and Use Stable Diffusion (June 2023) - Basic Tutorial
• How to Install and Use...
Extensions:
Panorama Viewer: github.com/GeorgLegato/sd-web...
Depth Maps: github.com/thygate/stable-dif...
360 Diffusion Lora: civitai.com/models/26815
Ayonimix Models: civitai.com/models/4550/ayonimix
Prompts:
DESCRIPTION, unreal engine, detailed, high quality, photoreal, movie still, (cinematic lighting:1.3), canon lens, 360 degree, equirectangular, qxj, ?lora:360Diffusion_v1:0.3? - replace ? with triangular brackets! Not allowed on CZcams.
Negative: blurry, duplicate, clone, human, animal, person, man, woman, cartoon, sketch, b&w, black and white
BLENDER AND VR LINKS
-------
Environmake Blender Addon: albertbozesan.gumroad.com/l/e...
Sidequest VR tut: • How to Build & Install...
Environment Converter/Builder: drive.google.com/file/d/16F1p...
Blender Startup File: cdn.discordapp.com/attachment...
CHAPTERS
-------
0:00 - Intro
0:29 - What you need
3:51 - Prompting & Settings
6:39 - First Results
8:32 - Creating a Depth Map
9:56 - Upscaling & Inpainting
13:23 - Sponsor Message from Brilliant.org
14:29 - Creating the 3D Model
23:40 - Transferring the Home to your Quest
----------------------------------------------
Did you like this vid? Like & Subscribe to this Channel!
Follow me on Twitter: / albertbozesan
This video was sponsored by Brilliant
To try everything Brilliant has to offer-free-for a full 30 days, visit brilliant.org/AlbertBozesan/ . The first 200 of you will get 20% off Brilliant’s annual premium subscription.
Mind absolutely blown. Back in august of last year, i had no idea this would be coming true so fast.
Thanks for the tutorial!
I could see this working really well with environments and scenes created based on a novel, with the audiobook accompanying the user.
Basically listening to audiobook, while in VR with a visual environment based on the novel.
Totally!
Wow; I've been generating skyboxes for a little VR indie game using blockade-labs and I had no idea there was a 360 Lora available for SD!
This will make fine-tuning so much easier. 😃
Clean tutorial without cutting or skipping the important parts. Thanks.
Thanks! I’m glad I managed to keep that balance.
A very cool method again. Thank you Albert, this again is a perfect, cool, unique tutorial! You rock! ✊
Thank you!! I try to keep my ideas non-obvious :)
Awesome vid. Can’t wait to try it. Quick note. Suggest doing inpaint before upscaling. Works much faster
I suppose, but then you’ll lose resolution in the details and have to paint those back in anyway. Would you say the benefit outweighs that?
I started to learn Blender from 2 weeks ago just for can use your Environmake addon, Nice work!
Wow! Hope you enjoy it 😄
@Albert Bozesan Much appreciated 😅
Oh man, you're awesome. You made so wanted material, best wishes! God bless you! Wow, thank you. No words, genius.
Thank you for your kind words! Glad I could help!
this is a great workflow
Thanks!
thank you, really, to share this. you gained a new subscriber.
Thank you! I'm glad it was helpful.
Great tutorial, you have great design skills 🙏
Thank you!!
2023 is wild, so much new amazing tech
Hey, thanks again for the great tutorial! How would i send this file to someone so they can try it out on their Quest without having to build the environment themselves? Is that possible?
Excellent video! For some reason my panorama viewer doesn't allow me to rotate te screen. It just shows the generated photo, but I'm not able to pan the screen. Any clue what i did wrong? Everything else works fine.
Very cool! (I see what you did there with the word FUNCTIONAL 😂) I don't use autoupdate, thank god.
Great tutorial and so helpful! Would you like to publish your home to MoonVRHome where people can install this custom home in VR?
Great tutorial! Thank you for sharing 🥳 I wonder if you might know of a plug-in for VIVE/Index rather than Quest? Meanwhile, I am Googling that now 😁
Thank you! Not that I know of, I only have the Quest 2. But I'm sure there must be an option out there :)
Great video! Maybe you should try the skybox labs as a 360 generation method. I feel as it may simplify the process even further.
BlockadeLabs ist excellent, I just feel like figuring out and teaching people how to do it independently on their own computer is more valuable in the long run 😄
Thanks for this so cool! Any chance we can see examples anywhere that can be reached from the Oculus browser? That's probably another video. :)
Thanks! What exactly do you mean? What do you want to see in the browser? 😄
Great video! Can you recommend the steps it might take to convert this environment into one you can use in the Unreal engine so that a player could navigate within it?
The .gltf I export towards the end should import perfectly in any game engine :)
Thank you very much for this class. I have a question, what kind of minimum equipment is needed to run this application locally?
For okay performance you will need a Windows PC with at least 8GB RAM, an NVIDIA GPU with at least 8 GB VRAM, and as much hard drive space as you can (models are quite large).
LoRa means additional parameters that come from a finetuning on top of a model, so they cannot be used with any model, right ? just the one on top of which it was finetuned
LoRas can be used with any checkpoint as long as the base model is the same (SD 1.5 vs SDXL for example). But it’s correct that the checkpoint a LoRa was trained on is the one it works best with. This is why older LoRas in general work worse and worse on newer checkpoints.
Hello! I really like your video and tried to follow through the process and I have one question. When creating a skybox image as a scenery, the tree is too large and appears to shrink in the upper part of the image. Is there any way to fix this problem? I used the Asymmetric tiling + 360 Lora. Thank you :)
Thanks, glad you enjoyed the video! The way to fix that at the moment would be manual editing in photoshop, using generative fill and just classic image editing techniques. There’s no shortcut quite yet - the top and bottom of the 360 image is just bad for now.
@@albertbozesan I see! Thanks:)
when trying to generate on the enviromake tab i get error bpy_prop_collection[key]: key"specular" no found. Any known fixes?
I am getting the same error
Welcome to the singularity y'all, is it everything you hoped it would be?
And more!
Thanks for this tutorial! I have one problem, when i click Build and install environment, it says that the APK build was finished, but it doesn't automatically launch on my Quest. I can find the app in the Unknown sources, but nothing happens when i click on it either. Any idea how to proceed?
It’s not an app - it can be found where your other home environments are. Settings > Personalization and then it’s probably at the very bottom :)
@@albertbozesan Perfect! It works now :D Also, i tried to use SD upscaler in img2img because it usually does a much better job than the ones in the extras tab, but it created a really weird image with a bunch of connected tiles. I guess it's not meant to be used for equirectangular images. ESRGAN didn't really seem to fix the resolution much. Any idea about alternative upscaling methods?
I have installed the depthmap-script and restart the UI, but i have no depth tab. What did i wrong? Any solution?
Did you make sure it’s active in the extensions > installed tab?
@@albertbozesan Yes, I think it is a bug. Sometimes it is there sometimes not. Thanks for the video!
I have a brilliant idea. How about creating an ai program where you type in a paragraph to describe the environment that you want and the ai creates a vr environment just like how the ai art generator works in the Wombo dream app.
I mean, that’s kind of the ideal scenario of this. With an idea like that, it’s all about the execution and being early 😄
Moon VR Home on Quest Applab does that afaik.
Is it can't using on and gpa
?
Stable Diffusion only runs well on NVIDIA GPUs at this time.
@@albertbozesan thank you for reply! your tutorials help me a lot, great work!
So it won’t run properly on any Mac?
@@giraffeguitars6021 No, they don't have the required GPU. It will run very slowly in apps like Diffusion Bee.
The "Blender Startup File" link doesn't seem to work. It says: "This content is no longer available."
That’s unfortunate…I’m afraid I don’t have control over their content.
@@albertbozesan It's fine, I didn't care about making a home screen anyway. But I did export the mesh for use in a VR scene in Godot and it worked perfectly fine! Thanks :D
@@kmouratidis great choice!! Godot is overlooked as a VR option, but I've also had lots of fun with it :)
Any tips on how to create a video like this czcams.com/video/dbp8tdWjBB8/video.html so we can experience multiple AI created rooms using youtube VR in Meta? Thank you for posting these informative videos!
Easier than the process I show here, actually. You don’t even need all the depth steps. Edit your 360/180 images one after another in any video editing program, and make sure you set the CZcams settings to 360. Any 360 video tut should help.
Only cringe kids use VR. Ever seen the matrix? It starts here kids
I never thought I’d read “cringe” and “the matrix” in one breath.
@@albertbozesan matrix is cringe
Seriously @@albertbozesan, it's hard to believe someone can be so out of touch that they are hating on VR and the Matrix while on a VR tutorial. Who are these lost souls with nothing better to do but spread negativity?