How to self-host and hyperscale AI with Nvidia NIM
Vložit
- Äas pÅ™idán 8. 07. 2024
- Try out Nvidia NIM in the free playground nvda.ws/4avifod
Learn how to build a futuristic workforce of AI agents, then self-host and scale them for any workload thanks to Nvidia NIM - a platform that containerizes AI models and runs them on any GPU with Kubernetes.
#programming #ai #nvidia
💬 Chat with Me on Discord
/ discord
🔗 Resources
Nvidia NIM nvda.ws/4avifod
CUDA in 100 Seconds • Nvidia CUDA in 100 Sec...
Docker 101 • 100+ Docker Concepts y...
🔥 Get More Content - Upgrade to PRO
Upgrade at fireship.io/pro
Use code YT25 for 25% off PRO access
🎨 My Editor Settings
- Atom One Dark
- vscode-icons
- Fira Code Font
🔖 Topics Covered
- How to scale AI models
- How to self-host an AI model
- Nvidia H100 GPU
- Will AI replace humans in the workforce? - Věda a technologie
![The moment we stopped understanding AI [AlexNet]](/img/n.gif)
Try all the NIMs for free right now in the playground nvda.ws/4avifod
I like your magic world magic ( ai code) men
I didn't even watch the video cause I was engaging with the sponsor so much
NO, nvidia is evil
is this a ad?
no thanks
And here we are, Nvidia trying its best to take the whole monopoly doing it Nord VPN style
All thanks to CloseAI and other big tech using their GPUs for training dumb AI models that can't even solve a simple mental ability problem, like really, I asked GPT-4o to solve a simple class 9 mental ability problem and it failed, even with options it failed and kept giving the wrong calculations, thus approved that AI is dumb, and non tech people are dumber. Humans, am I right?
At the end of the day it's all about the money: Even for the loser behind fireship.
@@jurycould4275 so? you want him to work for free? lol
​@@jurycould4275 what other thing can equip you with access to resources for staying alive other than money?
What is Nord VPN style?
Day 0 for not talking about ai
Such day would in most cases would never come , ai has fascinated a large amount of non coders too and these non coders have become a large part of viewership of such videos even though they basically understand nothing about coding , so channels now have started to talk on ai more and more and they gain more views and they are not wrong who doesn't want more views and more money to buy rtx 6090 , thanks to listening to my yap session which wasted 34 seconds of your life and if you are still reading it wasted 38 secs of your life
I listened to a cutie for 38 seconds
gotta sell those shovels
@@AwindowBest use of 38 sec 🙌
​@@AwindowYeah as if that 44 sec of me writting this reply and reading your comment will make any difference. Lol know your obsoleteness you pleasent bot.
Fireship just casually switched from Web Dev channel to AI channel
I think his account is now being run by a sentient AI
Fr
Yea i lowkey miss the frontend stuff, but the “100 second of X†gave him the platform to be knowledgeable on all things CS. I loved “The Code Reportâ€, its just all there is to report on these days is AI, despite the fact im sure meaningful computer science is still being done in other subfields
Remember when this channel was called Angular Firebase?
market forces
Now it's an informerrcial channel
Pretty sure the first words out of AGI will be "Kubernetes? Are you shitting me?"
it'll be a real giveaway when the audio synthesis engine outputs "K 8 S are you shitting me?" in a sexy flirty voice
I'm pretty sure it will be something like "Guys, we need to get rid of that JavaScript garbage."
@@DdDd-pl3nt*immediately rewrites self in holy C*
😂😂😂😂😂
sponsored by nvidia? this man both made it and lost his soul
IKR. How is he supposed to give us a good review when he is literally being paid by them? I know this channel is more of a news channel, but still kinda sucks
Guess what? Everything he discussed and showcased in this video still applies to FOSS frameworks like pytorch-rocm.
Get that bag, Jeff.
​@@hardikgupta8496When did Fireship ever do reviews of Nvidia products?
@@hardikgupta8496 This is not meant to be a review, he's just explaining how it works. Where did he say he was doing a review?
As does everyone, thats how you actually get paid, this is a LIVING like anything else. Thats kind of the point of trying to become a big youtuber, especially a tech one where a majority probably know what ad block is. He isn't giving you news out of the kindness of his heart cause he is your older brother. He is growing his channel for the sake of getting more deals and sponsors. No one tries to grow their user base to millions as a hobby to continue working a 9-5.
Selling shovels never looked this lucrative!
I still cant belive that Nvidia somehow became the wealthiest company in the world.
Shovels
hopefully they are wealthy enough to provide good linux gpu drivers :(
Can't wait to see their reaction once the bubble pops
Isn't it obvious? The somehow is by selling the product that you need hundreds or thousands of in order to run an AI efficiently at scale...
@VoidHuskie
Didn't people used to say that about Ryzen ten years ago?
Fireship is big now, it's even doing 6 minute ads.
me 6 minutes ago: Oh a Fireship video great!
Me now: AI is a disease on this planet and must be expunged, also I just watched a 6 minute ad.
Another infiltrator from Dinosaur Enterprises, I see! 🦖🦕
Completely agree, it manages to have more fkn downsides than good sides for society
​@@FriedRice3519 You can say that about most of modern technology though. The internet and its contribution to social media, computers leading to sedentary office jobs, smart phones, tv, video games, military and any mass destruction weapons (nukes come to mind), and the concept of modern finance, banking and money as a whole. We wouldn't have a capitalist system if society as a whole "mattered." Yet everyone is defending that system because they are trying to save their 9-5 jobs from AI instead of having a discussion how we should get politicians who are more open about reducing what the work week looks like thanks to AI. Things don't exist because it's "better for all of society", things exist because there are multiple ruling classes competing with each other for power and AI is the new tech that the ruling class is trying to capitalize on. It doesn't matter if its corporations, or government. OpenAI is funded by the United States DoD now and has a former member of the NSA on board while the Chinese government is pouring billions into AI this year alone. This wont go away and be "expunged" just because people don't like it. No more than the internet and smartphones will be "expunged" cause people are complaining how people these days are glued to their phones.
@@FriedRice3519You sure you're not just downplaying the very real possibility of AI technologies destabilizing the tech industry as we know it? Maybe the downside is that you need to learn more stuff lmao
Did NVIDIA pay you per mention of the "H100"?
Probably not, but probably just required a minimum 🙂
Probably AI generated script/talking points. Most LLMs were trained on SEO derived text, so it outputs content like that a lot.
He should've just said it the entire video then. Just one word "H100" repeated over and over again.
@@darkspace5762 he doesnt get the check if the manager doesnt approve it
He said it already! The video is *SPONSORED*
It's ok guys, this is AI Jeff. The real Jeff is hanging out with Jensen on the beach
No better bandwagon to jump on than NVIDIA.
Ironically , your comment got stolen by AI , i just saw a bot posted it few minutes ago...
@@user-fr2jc8xb9g lmao, I thought you were joking. I had to look just to confirm...what a world lol
Sponsored by Nvidia, good for you bro get that bag.
These days that the first key to the kingdom
How is he getting sponsored by Nvidia? He was sponsored by Docker last time?? What's next? He's going to be sponsored by Internet 💀💀💀
@@jiwachhetri7317 next is him sponsoring himself
@@Thewhiteandorange 😂😂
â ​â @@jiwachhetri7317 The European Union 💀
Guys, i think that AI killed Jeff and now runs this channel.
well DUH!
Finally nvidia bought fireship before NIM could replaced him
finally i can host my own custom ai girlfriend
And for 30k that's kind of a steal.
Blade runner style
​@@RottenMuLoT real steal
The Nim (formerly nimrod) language team now bites their ass so hard that they didn't protect the acronym.
now nvidia is gonna eat all the search results for my favorite programming language
I would hope that the creators of Nim will sue the shit out of nvidia. Unfortunately I remember when Google stole the name “Go†from another programming language and got away with it :(
Hopefully
Welcome to C
Discharge the old patient: Web dev
Admit the new old patient: AI
That studering got me 😂😂😂
Good work getting a sponsor from them while throwing shade at the same time, it’s a fine line but you did it well . I would have blown it with prompts like “Nvidia.. generate me.. a bagâ€
adult site link bot stealing DarrenReidAu comment from 14 minutes ago.
"[Most people] underestimate what they can do in ten years."
Fusion power startups: *Allow us to introduce ourselves*
I have yet to see anything LLM related that will replace something like a Lawyer, a Programmer or a Doctor. At best they will be helpful tools to MAYBE boost their productivity in specific scenarios.
You haven't looked very hard. I'd list links but I'm off the clock
6 minutes ad...
Yes
yes
Yes
yes
we can make it a minute extra for getting excited to checkout the Nvidia crap and see what we are up against. Of course an extra 2 minutes for yelling at Jeff for doing this to us. How could he?
fireship:
fireship AI channel :
my mand sold his soul to GPUs god.
"AGI is highly speculative and a dumb image of the future"
**proceeds to propose a slightly less speculative, slightly less dumb image of the future**
Humans will always be preferred over AI, I know this breaks your heart as a programmer, but that's just the truth of it. God made us, we made AI, it's a pretty simple concept to grasp.
So Nvidia now wants to compete with AWS and Azure? Are they sitting on their own inventory or what?
They manufacturer things. I would hope they have inventory.
@@DoctorMandible They design things
Host the NIM on AWS or Azure, no competition
so basically It's nvidia's version of google's Vertex AI ?
my man sold his soul for gpus
30k gpu = 6 minute video..
I mean... I wouldn't mind trading my kidney for an h100 👀
Fireship: what's the best js library?
LLaMA: React
Meta: 😅
You were the chosen one, destroy the sith not join them. You were my brother Jeff I loved you
Recently i "somehow" got access to H100 GPU, Nvidia is a sponsor of today video. ðŸ˜
So when the kubernettes auto-scale, where do they auto-scale to? Do you need multiple servers of your own for the auto scaling to work? Do you rent servers from Nvidia for the auto scaling?
It seems like you can use it in the cloud or on your own hardware. In the cloud auto scaling will probably rival the biggest AWS surprise bills. On prem you'll probably need to have enough hardware for it to scale.
Its not meant for individuals, it is an enterprise solution. He is showing it because he got paid. You'll never be able to use this unless you like really expensive side hobbies.
How much $ did they give you
at least $30k in tech parts
my bet is at least 100k
Can't blame him
What I am missing here is the context: I mean this is the tool and it can do this and that but how to integrate it in a real application? What about the customer's data running through these services hosted on other companies servers, will be data safe from exposure? Whould be nice a 100 seconds orientation tour :). thanks
Super cool, congrats on the sponsorship!
Can I use Nim to program NIM?
Nvidia sponsoring you... Damn!!! Congratulations buddy
The costs ???
I love 7 minute ads from my tech "news" channel
finally a worthy oponent to run "hello world" in
The subtlety of the voice stutter to continue the joke that he is an AI is 🤌ðŸ»
What kind a customer will be for company which are operated by CEO and AI?
yeah but what is the docker image for NIM? I want to pull it
So NiM's is like AWS Lambda, but for AI specific use cases instead of general app services?
This video did not make me laugh. Is this an Nvidia commercial channel now or was this just a one-off to pay the bills?
"Yes."
Let the man make money, and someone might actually find this useful
Let Jeff get that bag. Literally everything in this video can be implemented with completely open stacks. It's no different than SageMaker except that with NIM there are a few less middlemen between you and Jensen's jacket supplier.
​@@GSBarlevthis is true of almost everything Amazon does. Yet they're filthy rich. Open source "tools" existing is NOT THE SAME as a PRODUCT.
Interesting, how much the sentiment is shared..! We all need bread.
Can't believe i just watched a 6 minute ad. Bravo Vincent!!
The S&P 500's 8 day at the money implied volatility it sounds super complex and fancy but basically all it represents is the relative price of options so when this is low does it means options are cheap? and when this is high does it means options are pretty expensive?
Is it possible to run NVIDIA NIM if my pc doesn't have any GPU ?
I’m pretty sure this is an AI voice now. Jeff finally automated himself completely. Sad days.
6:15 What is this code editor theme?
Ugly
The video was interesting but the comments about the sponsored content was fascinating too.
Awesome tools! Is there a way to access the talking avatar showcased not to long ago at NVIDIA's annual conference?
10 years from now:
“hi there my child got access to the keypad of my Apple house and I’m locked out for 13 days. Can you help me?â€
“I can absolutely help you! First off try adding Elmer‘s glue to the keypad!â€
I mean. It's definitely the funniest ad I've seen today. Well done!
Skip this video, unless you want to watch a 6:43 minute NVIDIA ad?
I do.
Calm down a bit, would ya
We can’t escape Kubernetes can we?
Its built directly into docker. There could be alternatives to a service mesh level system like Kubernetes, but there are very few options. Most infact rely on kubernetes. No one wants to make all of those services from scratch.
How is it different from runpod?
Can anyone please explain what the actual cost of using NVIDIA NIM works? There's not much on NVIDIA unless you sign up. Is it per call cost or is based on reserving a GPU?
I'm just here to listen to the wizard speak his magical spells
Today was death of a star
But can it generate Crysis on the fly at 240fps?
one minute is criminal
This is the most sarcastic ad I've ever seen
Will it run Crysis on max settings?
This is better than some ads you've done. The real difference in non-sponsored videos is that the jokes are relentless and borderline NSFW. In sponsored content, it's like using your library-voice when at XMAS dinner with your partner's family. Thanks for all the videos!
selling your soul to nvidia before being replaced by nvidia
How is this any different than ollama?
So when will ai can self code and make something?
Fireship getting more views than some JRE episodes is something
I love the part where you followed FTC guidelines and clearly stated that this was an ad
Your videos are works of art. Concepts, Wording, Jokes, Knowledge, Memes, just everything incredibly on point.
I miss the days where jeff would talk about new frameworks, now its mostly ai 😂😢
does nvidia NIM support the Nim programming language?!
Finally I can make a bot that plays my steam catalog for me
Is there a similar thing like this being developed by AMD?
You are hysterical dude. I love the sarcastic dry humor.
We boosting the stock with this one boys 📈📈📈📈📈📈
Man, the trial doesn't work without a company email :) how can I as a self-taught enthusiast try this out?
You don't, this was an enterprise Nvidia ad and Fireship doesn't care about self taught programmers anymore.
man, i sure wish i wasn't so fucking slow.
i would love to be cutting edge.
you guys live in a fascinating world!
I am amazed how all this lead to asking the model 2 questions and both answers were completely wrong! This sums up the situation with LLM's so far.
If no one is working, how will we have any money to buy all the stuff the machines make?
would love to see your setup on system monitoring with Grafana
shovel sellers... gj on securing that bag Jeff ðŸ‘
Sam IS the AGI...wake up sheeples
Got that backwards-the AGI is _Sam._ Every request sent to the ChatGPT API is answered by Alt-Man personally. He's the ultimate Mechanical Turk.
@@GSBarlev hey I like that chess reference. But really what was Sam thinking when he started working on CloseAI? Really tech is turning into a marketing industry and less of an innovation industry, I wonder what ever happened to innovation? Did he went on a long vacation or something? Or did Sammy boy stabbed him?
This seems pretty awesome tbh.
Training a what army?
Comedy with news as usual, solid gold.
I miss the old Fireship
Sounds interesting.
How's the performance with a "normal" GPU & large amount of system memory instead of a GPU you have to pay a kidney or two for?
This
that thrrii-thrrii-thrive is on point 🤣
The problem is how to trust them not to use our data?
does it still count as a sponsored video if it's just a straight-up ad?
So, no more new retail GPUs?
I think fireship no longer human, he just the first human plugged into the machine by Nvidia
Thank you
i think they are running something like kubeflow or flyte on k8s
The perfect shovel
Did you mine on it? We need a video about bevy
i wonder if what fireship says in this video is ai generated based on nvidia + his previous videos