Game Dev READY! AI has Perfected Pixel Art!
Vložit
- čas přidán 6. 06. 2024
- Today we examine a Lora for SDXL 1.0 that generates pixel art without any flaws!
▼ Link(s) From Today’s Video:
✩ Comfy UI Tutorial: • Easiest way to Install...
✩ Pixel Art XL (lora model): civitai.com/models/120096
✩ SD XL 1.0 Base Model: civitai.com/models/101055
✩ Comfy UI Download: github.com/comfyanonymous/Com...
✩ Workflow to Load into Comfy UI: gist.github.com/arcanite24/fa... (after downloading this, extract the .txt from the zip, and rename it to end in .json instead of .txt then you can load it from comfy UI)
► MattVidPro Discord: / discord
► Follow Me on Twitter: / mattvidpro
-------------------------------------------------
▼ Extra Links of Interest:
✩ AI LINKS MASTER LIST: www.futurepedia.io/
✩ General AI Playlist: • General MattVidPro AI ...
✩ AI I use to edit videos: www.descript.com/?lmref=nA4fDg
✩ Second Channel: / @matt_pie
-------------------------------------------------
Thanks for watching Matt Video Productions! I make all sorts of videos here on CZcams! Technology, Tutorials, and Reviews! Enjoy Your stay here, and subscribe!
All Suggestions, Thoughts And Comments Are Greatly Appreciated… Because I Actually Read Them.
-------------------------------------------------
► Business Contact: MattVidProSecond@gmail.com - Věda a technologie
This is some of the absolute best and real looking generations I've ever seen, amazing 🎉
For anyone stuck on the LoRA part, you need to put the pixel art xl file you downloaded into the models > loras folder. Once you rerun the .bat file, you may need to click the 'next' arrow in the loRA box, but it should then show up. Also, in the Save Image box, you can change that 'pixelbuildings' text. I removed the raw and upscaled boxes since I didn't care about them. After that, in the menu box, I saved the workflow I'm using so I can just load that in the future rather than the one from the video. Hope that helps!
Does this lora work correctly only in Comfy UI? I tried to generate something good through a regular stable, but it's a mess.
what if i dont have lora box when i open comfy ui?
Great vid Matt!
thank you!
These pixel art generations looks so good!
Thanks for the shoutout!
I'm glad you guys are really liking the model ❤
Extra tip: you can remove the "stardew valley style" from the prompt to get even better results. Or you can try with different games, the model is insanely flexible in terms of styles on top of pixel art.
Don't forget to post your cool images to Civit 👀
I saw your lora on SD on reddit, just props man - I'm gonna follow your content on CivitAI - I"m SO BEHIND on making SDXL stuff, my lora training list is a mile long :) -
How many training images did you use? And how many looping steps? Very cool.
is it possible to upload 1 image and create a sprite sheet from that?
Minecraft texture generator real free working 2023 100% no virus 100% no scam 100% working 2023 real not fake
WHY IS YOUR MODEL SO RESTRICTIVE? while it's a great one there's NO POINT OF THIS VIDEO BECAUSE WE CAN'T USE IT COMMERICALLY!
This is nuts, thanks for sharing!
This is so awesome, thanks for sharing this.
To be fair you typed Dwaine instead of Dwayne and AI seem to care about those things
lmao
🗿
Dwaine Thompson 😂
Dwaine The Stone Johnston. "If you seeeee what the Stone is saying" 🤨
Man, what a useful tool. I'll finish watching your video and give it a try
I think that’s the best lemon I’ve seen you generate so far
Great video as always
this is an amazing highlight!
AMAZING thank you so much I jeut needed that !!!
The things like buildings and plants are fine, but the more challenging type of video game art asset is the kind that needs multiple angles of view, or multiple expressions, like vehicles or characters. This isn't really particularly game changing for game production until it can give us that. Can you do image-to-image with high-res images to turn them into pixel art? That would be a lot more applicable to video game development: get an actor to come in and take a bunch of stills and then run them through image-to-image to create a wide array of expressions or poses to use.
It can. The CharTurner addon does all sides, You can even type in prompts like Character Sheet or "all angles" to get the whole character. The end result is only as good as your prompt. It's not magic.
@@TPCDAZ Any examples?
Even if it didn't, you already save so much just skipping by concept art phase, and also having basically a vertical slice of your final product... Anyway, in two years we will be playing a whole different ball game. It is the end of an era, let's make peace with it and have our pixel artists focus on differentiation instead of pixel pushing.
You should cover Nvidia's Perfusion. 2 Minute Papers did a video on it. It looks like it will revolutionize AI image generators.
DUDE! That's awesome! You have over 100k subs, so get your self verified!
Matt you looking fresh in this one bro ❤
Woah this is sick
Does this only work with text prompts? Can it convert images?
would be interested to see how it interprets different artist’s styles that aren’t pixel art, to see how well it converts it.
Now we just need controlnet for poses and we are set.
The question is animation. Can it consistently enough create multiple frames?
how to u get the thing to create images of a certain pixel size? its tends to always prefer to stay certain size and yes I'm down sizing it same way
I saw that on reddit! I love the expansive new things wer'e all making. Sadly I don't have the smarts to make anything IMPRESSIVE - i''ll be trainign an SDXL model on my own art and my mother's art but because I have a goal to work on a project for a local Autistic Art exhibition
If we had this in aseprite or unity that would make life amazing.
Look up Retro Diffusion. It's an Aseprite addon
RetroDiffusion is on Asesprite
That is pretty awesome, uff! I have been trying to make some small art projects but then i realize they need much more time and effort that I can't do, one person studio is not very realistic
Hi, when I try to Queue prompt, All I got is (unknow error). Any tips?
In midjourney if you add a comma and then style of pixel art it works better
Thanks!
For better pixelart in midjourney, use low values of stylize (eg --s 0)
Thank you that works a bit better, even tough midjourney really struggles with pixelart
I consider game creation to include asset creation when possible. Not just programming and storytelling.
Does this pixel art model take less compute power to generate images? I only have a 1080. I don't really need the full HD images right now; just game dev ready pixel art.
Finally Matt goes down the Stable Diffusion rabbit hole with LoRAs... he will never run out of content ever again. XD
As an intermediate pixel artist and noob game dev, this is very inspiring! I still struggle more with programming, but this seems like something that could help me improve my art skills a lot too xD (especially if it can do a minimalistic 8x8 pixel art or something I haven't been able to generate with other AI art generators yet)
I'm really looking forward to like a year from now when I can focus mostly on just level design (my biggest passion) for building commercial-ready games and just slightly make small changes to the art to keep my style consistent (certain color palettes). It's going to be so fun creating stuff all the time....
What's so inspiring to have art created from prompts? It will be the death of soul-driven contents created by real inspired people, soon to be replaced by soul-less content done by bots. It's like those bodybuilding athletes on steroid, no real merits. As a game developer myself, what made my journey incredible was honing skills gradually year after year (probably in vain now, due to these technologies). The next generation of artists will probably become dumber and dumber and brag about writing generic prompts, until at some point they realize they're always depress because they never really accomplish anything on their own.
@@BlossomsoftGames You’ll be able to keep your honed skills and that’s something to be truly proud of, and I believe no matter how good AI gets that games made only by humans will still be a thing and actually quite valuable.
Like you mentioned a lot of people will get lazy and not hone the skills. For me, I’m half-half, I was able to develop some skills over a few years but I don’t consider myself at the level by myself to build my dream games.
If I can use a calculator to do math or complex equations, I might be able to solve some problems quicker than by doing them by hand. At least for the programming aspect, I’ll be able to save a lot of time and hone my art skills or focus on 1 skill without feeling overwhelmed that I have to do art, music( I like composing music though And that’s probably one of my more advanced skills) , programming, etc
I respect people that can do everything by themselves, or lucky enough to have a team to help them make the game and can focus on their best skill.
Yes, I could learn from other humans, but let’s say I draw something and it’s not quite right. I could use my art and have it tweaked dozens of ways by ai in seconds to give myself inspiration if I like it especially. It’s just going to add more tools to the toolbox of more experienced creators.
Maybe you’ll never need it for anything , and that’s fine. It’s kinda how master chess players don’t wanna play against a chess engine and humans still play chess even though we have super strong computers. Don’t worry about it so much xD
@@phen-themoogle7651 Why? You think you'll just need to focus on improving your programming skills or level design... for commercial game development or what?!?
Just face it, very quickly, you'll just need to say "I want a game with these features, this kind of colors and this art style" and you'll get a finished game in seconds.
Then what? Who will buy your game? Because everyone will be able to do the same, utilize AI, blah blah blah, to generate another similar yet unique video game.
The problem with AI, and I'm more than fascinated and excited by everything we're seeing now, is that it will make us dumber and dumber, like the people in the Wall-E cartoon.
As an artist, if I was born into a world where everything was instantly generated by AI at my command, I would not be the same person I am today.
While this AI technology is absolutely exciting, in the end it will be a total disaster. It already feels like spitting in the face of every artist, who struggles for hundreds years, to create what AI can now generate effortlessly. This AI is just an imitation, a copycat, a thief. It's an illusion, and a very dangerous one at that.
Still, I'm like you, I see all you can do with this power now, but that's not real power. It's just the start and very soon, even if you excel in a particular area, it will not matter because nobody will pay attention to you and your achievements.
@@mythaimusic39 Let's look at it a new way, a way you haven't looked at it yet. To create a game sometimes you have a team of people: composer, programmer, artist, and many more teams and specialists. Even if the AI is capable of doing everything(in the future), maybe there's still something we can do with more soul or better, but it can save us time in the same way a calculator saves you time when you do math.
Some people will rely fully on AI for full games, but others will just use it to assist in certain aspects.
It'll make certain things feel less overwhelming, and maybe if I wanna do all the art myself, then I can just have it do the programming for me and focus on developing my skills in that one aspect than having to do a bunch of different things at once as a solo-game-dev.
Let's look at chess as an example, chess engines are better than humans and have been better for 10 years, but humans still watch humans play chess too and it's a lot more unique and fun to watch humans. People will still enjoy the human-touch of any art aspect.
And as a chess player, sometimes I train by playing against strong engines, but I'm still playing with humans too obviously.
You don't have to worry so much, you still have infinite value being human :)
@@mythaimusic39 Also, you don't need others to validate your achievements, as long as you know what you have. I don't care what others think of me if I'm content or happy with the effort I put into learning a skill or doing something. In the end we are alone and only you can be happy with yourself.
Amazing !
I have an rtx 3050ti with 4gb vram, I know the recommended for SDXL is 8Gb or more but would 4 Gb suffice since I'm only going to use it for pixel art generation?
We're only maybe 1 or 2 releases away from perfect everything as far as genAI is concerned. So Matt what's your channel plan when there's nothing to be blown away by anymore? It may get more and more detailed but by 2024 I think everything will be error free.
I agree. I think there will still be the occasional artifact here and there, but everything will mostly be perfect. Cant wait to see what’s in store
Promt to video or 3d is still evolving
It is cool for my Divoom Pixoo! ❤
Can someone explain the best way to get a scaled down version of your image to be a certain pixel width/height? Say for instance I want to make something that ends up 48x48 after being scaled down; should I just do the image at 480x480 then scale by .10?
Can this be done to apply a pixel effect to a video?
This is cool, but what we need is animation sprite sheets, not fun looking portraits or stills. Those are what really take loads of time to create.
When it can output c64 standard bitmap mode images ill be impressed.
please can you share the image link to the pixelated Steve Jobs generation? Thanks
if it really works like it looks here, that'd be a godsent to me 🤩
How to transform any image to that model(Pixel Art XL)?
Wow Matt!
this is insane
When is the next Bard update ?
Nice!
i tried pixel art on the early days of mage and it sucked, can't wait to actually try it now!!
This is why open source is so open important. Pro teams will drive the innovation, but they won't be able to extend their work to every interesting use case that's possible. Opening up the files to everyone allows people with the skills and motivation to create awesome stuff like this.
Anyone know of a web service in which I can run comfyui and import the workflow file?
how about poses for animation?
Does anyone know if you could use a reference image? Say if you wanted a picture drawn in pixel art?
For some reason when I open the JSON file, the model where the pixel art model is suppose to be, it says pixel buildings model, which i cannot get and I can't change it from that, besides null and undefined, which causes the whole thing to not work. is there a way to fix it?
had the same issue - plz let me know if you found a fix
same issue@@kennanb2186
can we use img2img to generate pixel img?
Awesome. I would love it if you could make some videos on running image generation models locally.
I remember wanting to get Stable Diffusion but it was not easy...
I want a video that tells me which model is the easiest to run, has the least of stuff to install that clutter your PC.
The best, easiest or worst UI. How to keep your trained model up to date.
Like a guide for choosing your tools and how to install. Maybe it's hard because right now everything moves too fast. But if it's doable I'd appreciate it.
Theres are great vids on comfyAI channel, guy goes thru it step by step
For anyone that got confused like myself as to why the lora was not showing up on their UI after downloading and putting it in your lora folder, you may have ran into the same issue I did. Just make sure you download the last link in this video and extract the text file in it as a .json (instead of .txt) - I did this buy opening the whole file with 7-zip and navigating to the .txt file and right clicking it and selecting rename and changing the ending from ".txt" to ".json" and then just extract that file anywhere on your PC (I put in on desktop just to make it obvious). Now you can start up your ComfyUI as usual and when it opens hit load and navigate to/select the newly added file. I feel like this was glanced over too much in the video (at least for smooth brains like myself) and I spent a good hour or more trying to figure it out. Hope this helps at least one person!
Note: I also had to click on the Checkpoint name and Lora name boxes and change them manually to the current ones I use as they get set to outdated ones with this method.
how did you change loraloader checkpoint ?
you have a huge obsession with lemons :)
He should change his name to LemonVidPro jk :)
Can it make animations?
Is it possible to use it via google colab?
Still need consistency for animate the pixel characters… when they will do a character in all positions and movements? I mean there are about 600 positions and moves needed to animate perfectly a character… !?
600 positions da fuck are you talking about..
Nice Vid Matt, thanks, took me ages to figure out but I got a 2070 graphic card with 8gb and 16gb ram, so does that mean my machine isn't good enough?
Does it make animations?
How do you avoid artifacts?
if animation of this starts to happen, I'm going to restart my carrier as a game developer.
In terms of using it in game design, this Lora is great for generating static stuff. Whenever it comes to animation, we use our own pixelators which use higher res 3D models and pixelate them on the fly.
you can generate walking animation for example?
Is that an approach similar to Donkey Kong Country from way back in the day?
how to do image to gen
no substitute for hand-crafted pixel art
If I may say awesome AI bro the pink shirt w the background playroom look uh just threw me off but just an opinion OR is that all meta AI?!
Is ComfyUI superior to automatic1111? I've been hearing a lot about it lately. If so, how would I fully un-install automatic1111? Just delete the folder, or does it install random things all over my hard drive that I would need to clean up? Maybe a tutorial? :)
It's not that it's "superior", but it follows a stardardized node system for it's UI, which makes it far easier for extensions to be added or adapted to it without breaking anything
ComfyUI is a lot faster than A111 , easier to use plus working properly with SDXL models ... A111 works badly with SDXL.
Is the sdxl easy to install on mac m1? Been thinking about transitioning from the og Stable diffusion Auto 1111 to the new Xl 😅
For me I noticed SD does a little cleaner realism then ComfyUI. but ComfyUI comes out with great pics or more closer to midjourney but not at the same time.
have you tried also Fooocus ?@@botlifegamer7026
Can it do 64x or 32x?
Okay, but can it make coherent sprite sheets? Animations are extremely important to games, and I have sadly never been able to get any Stable Diffusion model to produce coherent sprite sheets at all.
There's LORAs for this
@@TahuRock Can you recommend one? I haven't been able to find any that have the temporal coherence needed for an animation.
can you?
@@TheDefender123Plays Can I make coherent sprite sheets? No, but I also can't make a decent single sprite, either 😅 That's why I'm hoping tools will soon exist that can.
@@IceMetalPunk you can't yet. Everyone has the potential to do things, but it will take some time. I have hope that with practice you will have the unassisted capability to do it.
Hmm... next step, can AI make entire sprite-sheets : D
ok but can it generate a sprite sheet of walking animation for example?
yes it can
@@alwayswhiningjew What type of idea should i give the ai for a animation walk sprite for example?
@@jurre9366 Just use the open pose in the control net. make yourself a sprite pose blank (all poses should be in one picture. and to your promt add a "multiple view of the same character.). + Hires fix. On youtube there are many videos on creating pixel sprites, just search.
But is it tileable?
I think Bing/Dalle already does this well, I've used it to make sprites for several games.
I tried with hope but it is useless: "Give me 6 png images of an ant, in naive style, transparent backgroud, 256 pixel wide. It will be used for a single animation."
How to do it in krita ?
Your lemon looks good but hid beach chair isn't facing the sea!
3:18.. Or you can just copy the text and paste it into the UI. No need to load it from the file.
paste it where/how? cant find a good explanation of how to get this lora running anywhere on the internet.
@@JupstoNerd were you able find out how to complete it?
Is this uncensored? iykyk
Thanks for the tutorial, is there also a way to do image to image with this Pixel art generating LORA?
how can I tweak settings to generate not 128x128px but 32x32px or even 16x16px? thank you
I got good results by changing it from 1024x1024 to 512x512 to generate 64x64 (512/8), didn't try smaller yet tough
Please upload on news latest Nvidia image generator ...
Damn I always wondered what happened to Blue's Clues Steve
Can this run IMG2IMG?
Stable diffusion is winning 🥇
Is isn’t really useful for games unless you can keep coherence for animated poses
Now we need CONTROLNET
Heh, thought it'd be fun to try but my PC can't handle it. :p
Still pretty cool though.
Is it just me or does Matt remind you of Steve from Blue’s Clues?
bro roasted steve jobs for no reason xD
Half of me is impressed 'v'
Half of me is having existential dread, having thought that pixel art would be safe from AI generators :(
Getting some Stardew Valley vibes there.
nvm you use the keyword stardew valley lol
I'm able to run this fine on my 1660 TI with 6GB of memory in case anyone in a similar situation was feeling discouraged by not having 8GB.
This reminds me of my Gameboy.
Man can Nvidia sponsor me with a gpu too
all the characters i can get it to generate have 2 heads lmao
dot jay as oh an.... OH LAAAAAAAWD!