The Future of AI Video Has Arrived! (Stable Diffusion Video Tutorial/Walkthrough)
Vložit
- čas přidán 12. 05. 2024
- Stable Diffusion has released an exciting new AI video model - Stable Diffusion Video. This revolutionary model generates short high quality videos from images.
In this video, I test out Stable Diffusion Video using different tools like Pinocchio, Hugging Face, and Replicate. I showcase the video quality, motion controls, and 3D understanding of this new model.
I also cover Final Frame - a web tool for extending AI generated videos by chaining clips from tools like Stable Diffusion Video. Final Frame allows arranging clips on a timeline and exporting longer videos.
LINKS
Pinoko: pinokio.computer/
Hugging Face: huggingface.co/spaces/multimo...
Replicate: replicate.com/stability-ai/st...
Replicate Interpolate: replicate.com/pollinations/ri...
Replicate Upscale: replicate.com/lucataco/real-e...
Final Frame: Finalframe.net
Steve Mills: stevemills
Max: maxescu
Anu Anakash: anukaakash
If you'd like, please join the Patreon Here: patreon.com/TheoreticallyMedia
Join the Discord: / discord
Follow Me on Twitter: / theomediaai
00:00 📽️ Stable Diffusion Video is an AI model for generating short high-quality videos from images, with 25 frames at 576x1024 resolution.
01:11 🎥 Users can achieve impressive video results with Stable Diffusion Video, and there are options to upscale and interpolate the output.
03:35 🌐 The model excels in understanding 3D space, resulting in consistent environments even with different image inputs.
04:40 ⚙️ Users can run Stable Diffusion Video locally using Pinkio or try it for free on Hugging Face or replicate it for more options.
06:06 🎨 Parameters like motion, noise, and frame rate can be adjusted to customize the video output.
07:50 🚀 Future improvements for Stable Diffusion Video include text-to-video, 3D mapping, longer video outputs, and more.
08:11 📹 Final Frame is a tool for merging AI-generated videos, offering a timeline for arranging clips - Věda a technologie
So many cool new tools. I knew I needed to backtrack and watch this video when I saw your reference to Final Frame the other day. That is really cool the way you can splice your AI videos together that way - and for free (at least for now). I need to go check out some more of these tools. I was also "enlightened" to hear that issues in Hugging Space are not mine alone, LOL. Sorry we have to experience them, but I thought it was just my connection at fault. Now I know... As always, thanks for sharing all of this Tim!
I clicked on your video and was immediately stunned for a split second. The Ad for the new Doctor Who trailer started playing and I thought this was you showing the capabilities of the new AI video. haha. But I love all this new progress with AI video - I haven't completely went down the rabbit hole but with all the new tools I would think there's a workflow that can be combined to get something amazing. That's the joy of messing with all these - you can catch lightening in a bottle with combining these tools.
that AI
dr who trailer will be next week no doubt
Haha, the Doctor Who thing is amazing! It's funny, when watching shows these days, how much my "AI Brain" is on. I'm constantly thinking: "could this be pulled off in AI Video?"
Funny enough, watching the big set action stuff like in Monarch (the Godzilla show), Foundation, or even Doctor Who-- I'm usually confident we can pull it off.
It's the smaller scenes of dialogue and emotion that is tough right now.
Haha, oddly, on the Doctor Who front: I actually think AI could do a BETTER job than the BBC's SFX budget! Although, that is part of the Who charm!
Great stuff, thanks for your spotlight on this tools. 😊
Pretty crazy how good cocktailpeanut is with updating Pinokio. I'm pretty sure he updated it so that Stable Video was up and running on Mac while you were uploading this video.
As always, thanks for your great vids!
He is a MACHINE! I seriously have no idea how he manages that level of productivity and manages to juggle all the user issues/tickets that come in.
Highest of props to him!
AI steals from human artists work. People who use AI to make art don't have any skills or talent.
@@TheoreticallyMedia AI steals from human artists work. People who use AI to make art don't have any skills or talent.
Hey Tim ! Amazing video, I didnt know about Final Frame, very happy to see all these great tools we're gathering up along the way hahah :)
I dig the fact that you are a musician too, have you thought or already did projects with AI video tools and your music ? It's something I'm walking towards and I would love to see what was done in the community ;)
AI steals from human artists work. People who use AI to make art don't have any skills or talent.
@@SCHOOLERstyle Wow hahah, First of all you should never generalize on anything, a lot of AI users including myself are into visual arts, music and cinematography since a while.
Also not knowing how an AI works, you should probably do some research on diffusion models and the adversarial modus operandi, you are basically just spitting some negativity you found somewhere about something you don't even understand and judging people you didnt care to learn about, get your shit together normie ;)
Amazing video! I just started using Vocs AI for AI voices and it is absolutely insane
AI steals from human artists work. People who use AI to make art don't have any skills or talent.
@@mikesmithz my statement is 100% accurate
Final Frame is a life saver. I've been using it for a long time to extend Pika Lab videos - works great for something so apparently simple. Thanks for giving the developer the credit deserved for such a handy free tool that's easy to use.
Where do you get Final Frame?
@@tonyg.9710 Tim puts his links in the description.
the problem I encountered is the blurriness of the final frame compared to the original image. it decreases the quality every 4 sec
link?
AI steals from human artists work. People who use AI to make art don't have any skills or talent.
I check in on the state of AI generation every once in a while. This stuff, to me, is like the very first moving image of a train shown in Paris close to 130 years ago. All I see is potential.
Yup! I keep thinking: we went from “The great train robbery” to “trip to the moon” really quickly.
In film history terms, we’re standing right on the edge of our “Wizard of Oz” which is only a hop away from Citizen Kane.
2024 is going to be crazy.
AI steals from human artists work. People who use AI to make art don't have any skills or talent.
@@SCHOOLERstyle So?
@@khymaarenpeople study for a long time to become working artists who can make a living. It has the potential to wreck people’s careers and livelihood and adding insult to injury these ai apps trained on artist’s works by scraping them without compensation.
@@SCHOOLERstyleBreak it down. 100% manual art is an expression of creativity, skills dedication, and mastery of the tools and techniques.
AI image creation and other AI innovations are also tools that let you express creativity. How much time and effort you put into these skills is your choice, but there's a place for both types of art In this world.
Art made 100% by manual human labor will always have its place and be special for what it is.
AI art can be just as impactful and skillful but the barrier to entry is so small many more people will be able to express that creativity without the time commitment of manual work.
This Ai video arms race is getting intense. Pika’s about to eat Runway’s lunch but this stable diffusion video model will probably be their main competitor in the coming months. What a time.
That Pika drop is SICCCCCCK. I can't wait to cover it tomorrow!
SD's real strength is in the Open Sourceness. A few months from now, after everyone is done tweaking and futzing with it? It's going to be remarkable. Not that it isn't already-- but given some time? It's going to be a monster.
it will be amazing but will also be the start of complete market saturation of AI content on an unprecedented level, which will just cause overload - the equivalent of being a kid in a candy store, but then eating the candy every - single - day - until you're sick of it@@TheoreticallyMedia
@@lukewilliams7020 yeah, I wrestle with that. The way I figure it: there’s a ton of trash to wade through as is. The noise will get louder for sure, so as time moves forward, I think audiences are going to seek out curators to point them in the right direction.
In an odd way, I actually see the return of a Radio DJ/MTV VJ personality. I mean, not that exactly- but I think you get what I mean.
I mean, Spotify is a total wasteland of terrible stuff, but there are still bloggers out there writing about new releases they uncover. We’ll need something like that for all media soon enough.
My god…critics are going to be powerful again.
@@TheoreticallyMedia That's a really interesting take on it. And you are developing the perfect platform to perhaps be that MTV VJ. Are you on IG? I'm full on with client based video AI on my insta
Thanks, best overall information about Stable Diffusion Video and workflow with it and extra assets. 👍
Oh thank you so much!! Honesty, that means a LOT!
Wow. I installed Pinokio a while back and totally forgot about it. Thanks.
Fire it up for sure! You’ll probably be fairly surprised at how much they’ve updated it. They’re relentless with adding new models in! Out of control!
Christ, even those few seconds of light panning with character portrait closeups is enough to show you we're on the cusp of some pretty ground breaking tech. 2D holodeck is basically here...we just need to learn how to utilize the tech and put it together properly.
2D holodeck is a good way of putting it. You might call me crazy, but I can legit see 3d holodeck hitting by late 2024.
I don’t think it’ll be mainstream, but it’ll be here.
Always great info from you!
Appreciate that!!
Still waiting for AI video where we see high octane action like missiles launching and explosions, fight scenes and more. This slo mo stuff is good but there is so much more potential
I actually like the softer look rather than the super sharp Topaz version, or a blend of the two.
Great video Tim. Playing with Pinochio or is it Pinokio? - now.
Pinokio! It's so good-- just a treasure trove of tools (that was a tongue twister!) and that team seems super dedicated to keeping up with the latest and greatest models! They're a real god send!
In your opinion what is the most seamless lip sync/lip movement application for these kinds of platforms?
Great video, love the tech...
Grateful for runway having motion controls now
Saaaaaaame! Playing with Pika right now-- Can't wait to show you!
Hi. At 00:11 seconds in, there's an image of a ComfyUI setup with "text random line" thingies. What's going on? Where did that come from? Is there an explanation or tutorial or access? I'm very curious. Liked and Subscribed in the hopes you might see this.
i would like to see if feeding a storyboard and script in for a known movie and see if it can reproduce it after giving it concept designs and art from the production archives
I have never been this early for your vids oh my!
Pika 1.0 also announced today. I want to see the comparisons with that!
Haha, I thought I’d have tomorrow off! Welp, back to the studio!!
Krita (free 2d image editing/painting software) has a stable diffusion model plugin with upscaling if you need a free local version. Just an fyi.
It does! I covered that in…I think the next video? Haha, they’re all kind of starting to run together at this point! It’s been a crazy few months!
Thanks for this video. If one wants to use for example, one's own irl selfie, and use that for an AI animation video where I would either be in a created location or even the same location as the selfie. Where do I start software wise on the inexpensive side? TM or can anyone please point me in the right direction?
Amazing!!!
Great for documentaries using old photos.
That is a fantastic use case. Ha, I wonder what Ken Burns’ next documentary is going to look like!
cool video thanks, what abt runway gen 2?
All of what's currently happening is a dream come true for me. I'm resisting using anything though for at least another 2 years. I'm just happy to see it progressing into something that I will one day soon use to create real movies. I feel 2 years will be the time when that becomes possible.
Everything I’m hearing tracks tonight 2 years as well, but I’d say start experimenting now! It’s a really fun pool to dive into!
AI steals from human artists work. People who use AI to make art don't have any skills or talent.
@@TheoreticallyMedia AI steals from human artists work. People who use AI to make art don't have any skills or talent.
Thank you for this video it's helpful and well explained
That’s excellent to hear! Thank you so much for that!
0:15 what music is playing in the background of this video? its so relaxing and beautiful
Oh, thank you so much! That's music that I write for the channel! Drop me an email at Theoreticallymedia@gmail.com and I'll send it over to you!
I think over the holiday break, I'll finally get around to putting them all up on Spotify....so....no one will listen to them. haha...alas.
SVD generates 45 (15fps) frames with no problem. I even tried 60. But the more frames the worse quality and more memory is required. But you can do 8-10 fps and thus the video can get up to 8 seconds.
Oh, nice! I didn’t futz that far. I was also playing a lot more the the conditioning and getting some pretty interesting results, but that was after I published the video- haha, I was late on this one as is!
After then next big update, I’ll do a much more detailed rundown!
@@TheoreticallyMedia 🤝 This model is better in quality: svd_xt_image_decoder . You can also connect the input image. It's better to know what you're going to generate. You can also connect AnimateDiff and other nodes.
Thank you. This is great information.
Oh wow! Thank you so much!!!
The video of the top down building is actual paid video from things like StoryBlocks. It’s the Bank of America tower in Atlanta Georgia. It’s darn near exactly the same, if not exactly the same! 😳
Which is fine, as that is likely stock footage-- and MAY have either been licensed, or used as a real public domain set of video. I mean...no one shares their training data, so...maybe?
Do you focus on AI in general or do you lean more towards Video AI? I havent gone though your channel and just had a chance to take a moment to watch this clip. (at work) so I thought I would just drop the question here. I am really big and have a high level of interest in the Video/Filming space for AI.
So, I do lean pretty heavy into AI Video-- but, I try to cover as much of the creative AI space as I can. Obviously Midjourney and Still Image generators-- from the guitars on the wall, I do also do Music AI as well. A little LLM stuff here and there, when it strikes my fancy.
I'll never do "This AI Tool makes me $400 a week!" type videos though.
I think we're gonna be friends, though!!
@@TheoreticallyMedia You know, if I get fired for taking 5 more minutes to hide in a corner and comment this, I currently do not care. The moment you said ""I'll never do "This AI Tool makes me $400 a week!" type videos though."" you sold me. I am not going to drag on about it but a quantum computer calculating for a million years still could not come up with a justified measurement of how irritated I am with seeing videos like that.
You got yourself a sub mate. Thanks for the reply and I am looking forward to your content.
Matt Wolfe is another creator I like to follow because he is usually very centered on the news and practical use of AI features and content. He has his moments, but outside of those I think he is a reliable source for general AI news and Info. If you have the time and have any suggestions, I am open to connecting with other creators too.
P.S. I really hope someone gets the spirit of that reference.
Matt world is cool but boy his delivery is so grating to me - his voice - very hard to watch that false youtube 'monotone then rise the voice at the end' presenter style@@christopherpoet458
We missed you again my friend :) could you do a nu vid about last week ai soap opera :)
Haha! It’s good to be back! Oh man, the openAI drama? I totally steered clear of that one! Everything was changing so quickly that by the time someone put a video out, it was already outdated! Sooo crazy!
Maybe I’ll do one on the one-year anniversary!
📝 Summary of Key Points:
📌 Stable Diffusion Video is a new AI video model capable of generating short video clips from images. It generates 25 frames at a resolution of 576 by 1024.
🧐 The video showcases examples of videos created using Stable Diffusion Video, highlighting the fidelity and quality of the generated clips. It compares the output with upscaled and interpolated clips by Topaz, noting a noticeable difference in quality.
🚀 Stable Diffusion Video is compared with other image-to-video platforms in terms of action and motion, showcasing different levels of motion control. It also demonstrates the model's understanding of 3D space and its ability to create coherent faces and characters.
📌 Options for running Stable Diffusion Video include Pinocchio, Hugging Face, and Replicate. Pinocchio offers one-click installation but only supports Nvidia GPUs. Hugging Face allows free trials but may have user errors due to high demand. Replicate offers a few free generations before requiring payment and provides video upscaling and interpolation tools.
🧐 Upcoming improvements to Stable Diffusion Video include text-to-video, 3D mapping, and longer video outputs. The video introduces Final Frame, a tool to merge AI image-to-video clips with other videos and arrange them on a timeline.
📌 Final Frame is a project by one person and viewers are encouraged to provide suggestions and feedback to improve the tool.
💡 Additional Insights and Observations:
💬 "Stable Diffusion Video generates high-quality video clips from images, surpassing the quality of upscaled and interpolated clips."
📊 No specific data or statistics were mentioned in the video.
🌐 References to Pinocchio, Hugging Face, and Replicate provide credibility to the information presented.
📣 Concluding Remarks:
Stable Diffusion Video is an impressive AI video model capable of generating high-quality video clips from images. It outperforms other platforms in terms of fidelity and motion control. The video provides options for running the model and introduces Final Frame, a tool to merge AI clips with other videos. With upcoming improvements and the opportunity for user feedback, Stable Diffusion Video and Final Frame have the potential to revolutionize the creation of AI-generated videos.
Generated using Talkbud (Browser Extension)
No time stamps for this? I’m still looking for one of these summary things that do accurate time stamps. It’s a lifesaver when it comes to putting in chapters, but the times are always wrong.
@@TheoreticallyMedia Have you looked at Harpa AI yet? It provides time stamps as well, but I've never actually checked their accuracy. I'll have to do that the next time I create one of these video summaries. I quit doing them because they miss so many of the details that I end up taking my own notes anyway. That being said, they still provide a useful framework to add my own notes to. At least with Harpa AI, you have different options for the summaries and not all of them provide the time stamps, or at least they didn't before the most recent update.
AI steals from human artists work. People who use AI to make art don't have any skills or talent.
@@TheoreticallyMedia AI steals from human artists work. People who use AI to make art don't have any skills or talent.
loser haha@@SCHOOLERstyle
Love final frame thanks
It’s so good! So happy to see Ben continuing to innovate!
I'm not massively familiar with video AI but for extending the length couldn't you generate a video from the source image, then take the last frame of that (as an image) and generate a second part from it, and so on. Then join them together after. That should keep each video somewhat similar and give a longer video in total. Or am I being too obvious and missing something?
Nope! That’s exactly it! Final Frame just gives us a really easy and convenient way of doing it- without the hassle of opening a video editor, or screenshotting.
Now, there is an issue with the quality taking a dive after every generation. By the time you get to the end of a generated video, there is a quality hit, and that is where the next gen starts.
I haven’t tested it fully, but it looks like SDV has made some improvements on that part, so- hopefully those days are coming to a close!
@@TheoreticallyMedia Ah gotcha and yes, quality is an issue as you proceed. I hadn't considered that aspect. As you say, it's still early days and things will only get better.
Thanks for the confirmation. 🙂
FYI "panning" up and down in film is called a tilt
does anyone knows if it is an option to run the whole videos on Topaz after editing it? or it would be taking forever to process in Topaz?
Oh exciting
Funny that the Pika announcement just happened too. So much going on in the video space right now.
like, LITERALLY as I'm finishing this VIDEO!! haha-- Well, new video for tomorrow, I suppose! Haha...AI will NEVER LET ME SLEEP!!!
🎯 Key Takeaways for quick navigation:
00:00 📽️ *Stable Diffusion Video is a new AI model that generates short video clips from images.*
01:11 🚗 *The Fidelity and quality of Stable Diffusion Video outputs can be impressive, although some upscaling may be involved.*
02:21 🎥 *Comparison of Stable Diffusion Video with other image-to-video platforms in terms of action and motion.*
03:31 🌟 *Stable Diffusion Video understands 3D space, creating more coherent scenes and characters.*
04:27 🖥️ *Options to run Stable Diffusion Video locally using tools like Pinocchio or online via platforms like Replicate.*
05:50 💲 *Replicate offers a reasonable price of around 7 cents per output for using Stable Diffusion Video.*
07:01 🎨 *Conditional augmentation can control the amount of noise added to the initial input image.*
08:28 🔄 *Final Frame is a tool that can merge image-to-video clips together, allowing for extended video outputs.*
09:38 🛠️ *Final Frame is an Indie-made project open to suggestions and feedback for improvement.*
Made with HARPA AI
Hey Tim separate topic.
I read Everart terms and policy, did I read correctly?. I have the understanding that anything created is their property, permission to use for personal but not commercial.
Let me if you know differently
So, I haven’t really dug too far into T&Cs for most of these image generators- namely because the legality of ownership is still in such a state of flux.
It will likely be years before any of this is hashed out, and there will be many high profile battles that a number of lawyers will get very rich off of.
In the meantime, I generally think it isn’t in the best interest of these companies to try and exploit your work. Training, maybe? Maybe advertising. But straight up ripping you off? That’s a bad look and not going to instill confidence in future customers.
On the personal vs commercial. Again, that’s going to be a legal battle, and one that is already leaning in favor of AI. The thing is, I don’t know how anyone can prove if you’re using a model you have a commercial license for or not. Is it Midjourney? Or is it EverArt? Minus a file name and maybe some metadata, there would be no way to tell- and both can easily be scrambled.
The TL;DR: I don’t stress the T&Cs, and I don’t think anyone else should either. Just make stuff, and if it gets to be a problem, that’s probably a sign that you’ve got quite an audience looking at you!
Thanks for the intelligence.
I was concerned if Everart would come at me for royalties off content generated by their technology and distributed commercially. I just skimmed it too hard to stay focused on terms, hard read . I should create a text to speech option for companies maybe make a$
I think I am going to have to seriously consider Topaz for cleaning up my video from my old camera. I have tried a few of the free options and they still look really smudgy. I like the look of the SBS in your video.
I mean, it’s pretty good. There is a reason everyone calls it out as the cream of the crop.
But yeah…that cost. I’m bummed that you missed the Black Friday sale. It was 50% off. Well, sort of- it was bundled with Topaz Photo, but still a good deal.
It does have a pretty good demo, I think it’ll run everything, just watermarked.
This will be very interesting to see the future of film making. Between this, deepfake, deepvoice etc we may end up in a situation where noone ever has to he on set/locetion/actually shoot film again!!!
Well, it's awkward when your video is less then an hour old and Pika labs make it looks old
Great video and awesome content in general. Sorry if sounded rude
Haha- not at all. I'm literally in the Pika Discord right now laughing about it! See everyone again tomorrow for a WHOLE NEW VIDEO!
Thanks as always! I have an off-topic question, is there any way to make StableDiffusion not show people but only clothes? I put no human, no girl, etc. in the negative prompt and it still shows people.
That's interesting-- I am....not sure? My thought would be to use negative prompts, but if you're saying that doesn't work, then maybe try some approaches with the positive? Like clothes, display, wardrobe examples, mannequin (with a negative mannequin?), storefront clothes display?
I might play around in that area-- or inpaint the person out.
That's interesting, I've never tried that...
OH-- what about Invisible Person? That's so stupid it might just work!
At one point, I needed a shot in Midjourney with a bunch of dead zombies on the ground (think Walking Dead), and it would not give it to me no matter how hard I tried. I landed on "Sleeping Zombies"...and boom: got 'em.
They aren't dead...they're just napping!
Sometimes the dumbest things work.
@@TheoreticallyMedia I've tried but it's not working, thanks for the idea.
Thank you
Is there any way to change the Home directory of Pinokio?
Just checked replicate -- SVD doesn't work there anymore (
do you know why? is there a substitution to install for Mac?
Can we give ref image to create video from ?
Great
On hugging face every time it gets around 20s left...it adds ANOTHER 100s...and it keeps doing it..and won't finish.. please help
Did you duplicate the space? Or just running on the public model? If it’s public, I’m sure the GPUs are probably getting hammered
Try out Replicate, for sure. Had zero problems there!
@@TheoreticallyMedia replicate always wants to charge me, i think i burned through their free credits, i'm too scared to give them my paypal, don't want a weird $200 bill. testing cool stuff out. haha
Tried using final frame, about every two seconds it wants me to pay with a splash screen that dims everything and gives me options for credits. I can't even look at the site lol
temporally incoherent animations are so trippy it's like the video itself is on lsd
Oh, man I love it! In the latest video I did on Pika 1.0, I had some really early Pika stuff in there-- Clowns vs. Mimes. It is SO Trippy. You can check it out here: czcams.com/video/CHoOdPVSEsE/video.html
I know all this decoherence is eventually going to get stamped out-- but to be honest, I'm already nostalgic for those super surreal videos.
i see the future. i subscribed 😊🎉😊
Thank you...uhhh, and while you're looking at the future, can you nab me next week's lottery numbers?
I believe that Stability AI promised an awesome text-to-video generator a while back.... It looks like they delivered 😀
Don’t most these sites use like a point system to make anything?
Have you seen Pika announcement? That looks like groundbreaking update.
Haha, it dropped JUST as I published this video! Sigh, another video tomorrow!
Keeping up with AI is sometimes like a really funny joke is being played on you…I mean, even I had to laugh when I saw the announcement!
wow might be worth getting cloud stuff for that.
If you don't have a company, can you sign up for Stable Video wait-list?
You can! At least I did-- and I don't qualify as a "real" company. Just kinda make something up! I think the company part only extends out if you are planning on using the commercial licence.
Seems like anyone can download the model, but not supposed to use it commercially yet, since it is still not done.
I honestly don't know if this is scary or not. But I will definitely play around with it XD
cool
Topaz AI use A LOT of MEMORY to convert. Don't use anything less than 64gig+ of rams and high end GPU. It will take forever to convert.
Agreed. I tend to use it for shorter clips. I haven’t spun it up for a 10 or 15 minute video yet, but I can see that as being an overnight process. Like an old school AE render where I’d kick it off before bed, then wake up…
…well, in the old days I’d wake up to discover my computer crashed overnight. Sigh…
AI is making leaps and bounds. This is the next industrial revolution. Incredible.
It really is. And at twice the speed of the Industrial Revolution. As crazy as 2023 has been, 24 is going to be twice as insane.
Only question is for who this is revolutionary. Certainly not for artists lol... just "content creators"
@@kidcoma1340 Are you familiar with the industrial revolution?
@@Zanroff sure, why?
I'll know this video title is 100% true when your entire video talking about it is made in AI
Haha- Not quite yet! But a valid point. To be honest, that will happen at some point...But the real question and test will be: Will you be able to notice?
Most people find these dead-eyed AI "actors' to be super creepy. I know I do. (And not in a good way).
I'm waiting for the day when AI comes to the point where I can scan in a manga and the AI will spit out a fully animated Japanese style anime based on the manga complete with voice acting dialogue sound effects and background music all originally created by the AI based on the style of the author of the anime and manga.
a number of people above my pay grade, and also way smarter than I am, say this will happen within the next 2 years. Maybe not the exact Manga style-- but, y'know, something like a Netflix adaptation.
@@TheoreticallyMediaI had the same thoughts from day 1 the AI arrived. So I imagine something more bigger than JOAN IS AWFUL" . Imagine pay a cinema to create your own movie with your own prompts, and you can go to that cinema with your friends to watch your own movie, you could pay 50bucks for your own movie, and everyone of your friends pays their part . I know it sounds crazy, but with Gemini, GPT5 or Q coming soon... Just think about the potential in this 2024/5. Just think about it.
@@TheoreticallyMedia at the exponential rate that AI is changing, I don't think that prediction is unrealistic.
The ideal AI program would analyze the characters via LoRAs in their anime form, and then recognize the manga version and then adapt the manga into a video. The trickier parts would be dialog (from trained models of the voice actors from the anime version) and making new BGM based on the music style of the first season anime.
This would be a tool where we could convert manga (which is ahead of the anime release) into anime. Pretty challenging, but in 5-7 years, I can see it all happening.
Can i run it on rtx 4050 6gb vram laptop gpu (max tgp variant)?
I believe so? Initial reports said that it required 40gb vram (insane), but that was at launch. Apparently it has been optimized since then.
@TheoreticallyMedia Thanks for the reply. I tested it, and it works, but for 14 frames model only.
The first feature film (indeed a musical omnibus) made with AI generated Video available in free download is here czcams.com/play/PLAOQS2lpu2SQs9HHSCPFlAEZgI97Ip_at.html
Thank you! I’ll check it out!!
Did Mac version drop?
Oh man, I’ll have to check. It’s been so crazy with tools dropping left and right, I haven’t even had a chance to circle back!
Is it possible to make a youtube video of a person, say just talking to you, that is indistinguishable from the real thing?
Baby steps.... for something...BIG🎉
It. Is. Crazy...
AI is changing the game.
Speaking of games, in the latest video, I have a thing where Ilumime is running an LCM in a 360 environment.
It is early, but you can clearly see this might be the future of First Person Games.
The real breakthrough would be when they stop treating video in AI as 2D and actually train 3D models, that's probably how consistency and directed motion will be finally mastered.
Everything I'm hearing and seeing points to 2024. It's going to happen.
Ha I'm on there now lol on discord
0:06 Please do not associate the term Stable Diffusion with complicated workflow. Comfy UI is the problem. Not Stable Diffusion. I knew this moment would come where youtubers push the comfy ui agenda so much that people get put off even trying Stable Diffusion. For anyone wondering use Automatic 1111 it's easier and made for everybody. Not just ai snobs like comfy ui
Oh, I know-- I hope you realized that I was pushing against that in this video. At some point I'm going to try to do a video where I simplify Comfy as much as I can-- mostly as an entry for those who are interested in trying it-- but at this point, I think most of the "starter" tutorials on it are fairly complex.
When I get some time, I'd like to do a full breakdown from the ground floor.
But, yeah-- totally agree with you.
@@TheoreticallyMedia I'd love to see that. My mini rant wasn't aimed at you directly. I happen to think your videos are the best around for AI as you treat everyone equalliy. I appreciate your work. I loom forward to any help you can give regarding Comfy UI
AI video is still yet to have its Dalle-2 moment. It can’t produce convincing motion for more than a second or two, which isn’t really useful for much.
Might be happening as we speak. Gonna have a video up later today!
Cool gimmick.
Hello....I've tried and failed in a few AI machines to create science historical images. First one, I wanted to generate Isaac Newton in the famous John Travolta Saturday Night Fever Pose, on dance floor, left arm up, right down, hips tilted....in all the attempts, I cannot find a machine to handle a simple request. I know I must be doing things wrong. But, I challenge you to find that device. I want to have Newton saying, WHATS GOES UP MUST COME DOWN.....but I want an Isaac Newton pointing up and that Travolta pose would be funny and a lead to my next wish, Einstein walking away, so you just see his bushy hair and lab coat....with E=MC2 on his back. Until I find an AI prompt device, this idea of mine will go nowhere. Einstein is copyrighted image, so walking away, could be any brilliant physicist...lol....thoughts ? Each attempt, my prompt was not understood. If I had pictures of Newton and Travolta pose, that might help, but it did not. HELP ?
Can you not just make a video, take the last frame and run it on that, take the last frame, run it on that, take the last frame and run it on that, then stitch the video together to get a longer video. I'M Definitely looking forward to where this is going
Yup! You 100% can. Final Frame just simplified that process, so you don’t have to open up a video editor or screenshot anything. It’s really fast and free.
Doesnt work on Mac yet
Oh, I’m there with you.
Give it about a week. Some have it running (unoptimized) on a M3 Max. But that’s all about brute force.
It’ll be here in no time.
This is big its Himalayan news 📰📰📰
stable video diffusion is just bad right now cause it makes the character's face move in a weird way. I made a 3 minute short film using runway and pika . i can tell you runway is best right now for making the image move according to your prompt and it also maintains the quality. pika is good also and its free. .
Love Pika, and love Runway. You’re right, they all have their strengths and weaknesses. I think the true value for SDV is going to be when the community starts building for it. That’s always been the real super power there.
next 5 years every image on inter will become a video, and there will be supper computer running to trying to make every single image they can create from large models but more theycreate the more models will become capible of
There’s some pretty big breakthroughs in the pipeline this year. I think we’ll see animated styles hitting major benchmarks this year.
it's going out of controle wtf is going on is not good, every bad person have also access to those all tools which will impect begger then expected@@TheoreticallyMedia
eh, i'm not that impressed by the new Stable Diffusion model. I mean yea it looks sharp and vivid but doesn't do anything that groundbreaking or different from other sites.,The next huge leap for AI video will not only need to be FASTER and more complex movements but MUCH longer generations without the need to duplicate the last frame and lose quality. Runway seems to be the leader in the AI animation world for the moment. When we get to the point where we can do full natural speed long form motion with fully detailed photorealism, TOTAL control over image, no unwanted morphing, where you can create your own Jackie Chan kung fu fight or elaborate car chase or epic battle with fast galloping horses, THEN I'll be impressed. Hopefully we'll be there by next year 😆
So, I think the real power of Stable Diffusion isn't 100% in their model, but rather the fact that it is open source. So, SDV as a base (in my opinion) looks great-- BUT, it is what the community does with it in the next few months that will really take it to that next level.
That said...my god, did you see what Pika just dropped? I'll have a video on that tomorrow-- HOLY MOTHER...it is something!
I whole heartedly agree 💜😂. I just need longer videos and to be able to prevent those weird random demon looking hands that morph out of no where sometimes. 😅 They startle me at times.
AnimateDiff was similar when it first came out, took a few months before they were able to expand to limitless duration, etc. At this point you can do insane renders leveraging ipadapter and contolnet
Yeah, it'll be amazing in a couple months with loras
Well, get to coding then.
All of the AI video generators are 2-4 seconds. I'd rather have much longer time and a sacrifice in quality.
Yeah, that's the current hump. I'd say....early next year we'll be in the 7 to 10 sec range.
@@TheoreticallyMediaI want to have ChatGPT write a movie treatment, then use that to have AI spit out a 30 minute short film. 😁A man can dream.
Too many different applicataions to use and parts all over the place!!! The large Corporations will start buying up all these little applications and bundle them into a complete suite as they have always done. That is when to make decision on where best to dive in.
why are you out of focus ?
Hahah. Ugh- you noticed! Stupid camera had autofocus off, so I ended up soft through the video. In the opening, I actually vanish because it went full out of focus.
Stupid mistake. Over a hundred videos on the channel, and that was a first!
Hey guys, use your imagination like a artist not AI to create instead of you, as a human being we have more potential than artificial things!
A lot of times I look at these generations as raw material for digital artists to build with. Almost like an endless stock image site.
There will always be some that just use the tools with no post or additional work, but I think the really talented artists of the future will learn how to incorporate AI into their workflow.
As I always say, still plenty of work to go around. No one wants AI generated photos of their wedding day, for example.
gives directions of using all the other AI systems that you need to pay for, but then doesn't include the free Stable diffusion version that everyone is here for..
Runway really sucks. I'm finding the quality looks very "morphy" and they have this super strict policy of stopping any video generation with just a little bit of skin showing. I'm growing really tired of the overdone morality policing by these Ai companies. That's why I love stable diffusion.
I don't see any man in an AI-generated pic; it seems the future is feminine. 😅
Haha, Prompt something up in Midjourney...Midge loves her buff dudes. It can be pretty funny.
Dayum. How long was this tech? 5 years? If we can somehow manage to make AI copy human movement or speech by like creating a virtual human ala matrix. Movies can be made virtually with no human actors, no physical sets. A person with good knowledge of this technology with with cinematic knowledge can basically create a movie of their own and probably make millions. Crazy. A lot of people will be millionaires in the future if this tech evolves to matrix quality. Andrew Tate will be pissed lol.
I’m hearing 2 years to that. Maybe less, maybe a bit more- but that’s the ballpark.
If you asked me about that a year ago, I’d say no way.
With what I’ve seen this year? Yeah- it tracks.
I disagree. If you're getting in without cinematic knowledge, you've already set yourself up to fail. Your ideas won't be good, and no amount of AI assistance will save you from the boring mediocre film that will result from not having knowledge on how to direct. AI lacks intentionality, and if you yourself lack understanding of what you're doing then all you can produce is meaningless fluff.
.
!
Bad music, bad sound,bad pacing, bad flow, bad grading. Expensive camera’s don’t make cinema. Traditional or Modern techniques.
Does not matter. Aesthetics is easy when you can control environments AI or Realworld.
Every individual has access to great technology. Everyone has the ability to take technical great photos using their phone camera for the last 10 year.
And still ( pun intended ) only a small percentage is doing meaningful work. How come 99% of all humans are still not making unique art with their new acquired tech.
Some will find their passion for the first time and run with it. But when the storm calms down 95% of humanity are still making mediocre pictures of moments they missed because they focused on the memory they missed in the first place.
To be fair, you can chart that out to any era or medium of art/creativity. There were thousands of painters during the renaissance, but we only really focus on the influential handful.
That top 5% of artists who do change culture? That’s a combination of skill and life’s lottery ticket.
That said. You don’t have to create stuff that is meant to please others, or do anything meaningful. People can create just for themselves. Something to hang on their wall, or watch on their monitor.
It doesn’t have to be meaningful for everyone, just for the person who made it.
Stable diffusion still looks so bad, unfortunately. :(
Can't wait for what the future holds in 5-10 years.
Topaz is ridiculously overpriced for what you get.
I know-- I did manage it on a Black Friday sale, but yeah-- I can't disagree about that price point.