In the nvidia NIM you must change the inference parameters. I've changed the temperature to 0.6 (NIM default is 0.2, oo low) and top P to 0.9 (default is 0.5). After this fix, the model could answer the question #1. Also, there is a ambiguity in the sentence *"What is the capital of the country WHOSE name ends with 'lia'"* If "whose" referes to the name of the country, then the answer could be what you sad "Camberra, the capital of AustraLIA". Or even "Rome, the capital of ItaLIA" (Italy in italian) If "whose" referes to the name of the city, then the answer is BrasiLIA, the capital of Brazil. It was an excellent analysis and I enjoyed it! I will check out the other videos you mentioned.
Thanks for the great suggestion and comment. Although, When a company is hosting a demo, The default settings should be set correct by the company itself. Also,If you look at the Mistral Nemo HuggingFace page you'll see that their page also say "Unlike previous Mistral models, Mistral Nemo requires smaller temperatures. We recommend to use a temperature of 0.3". Also, you are correct about the Ambiguity in the sentence. I know that this sentence causes a confusion. But, if someone (a Human or LLM) reads carefully they'll know that it should be a Country since that's the most likely case here. This question is answered correctly by multiple LLMs in my tests. So, that's why I have kept it.
> "Also, there is a ambiguity in the sentence "What is the capital of the country WHOSE name ends with 'lia'" if you had finished school you would know that "whose" always refers to the country, since it's closest.
@@monza8844 I'm not native English speaker, so it's true: I didn't "finish English school". And possibly, some LLM didn't finish school either, when we see so many wrong answers for this silly question. But magically some models start answering it correctly when you apply the fix. Give a try and blame me later.
I showed my daughter what deepseek can do by making a tick-tack-toe game in seconds using python, that keeps track of scores and has a slick UI, she says we developers are the modern day wizards🧙♂️ can't wait for NeMo to be on Ollama
Sometimes i leave the vids at the start cuz my feed is already flooded with AI development content. Working with this shit i, firstly, love content like yours, but also, it is super stressful, AI content goes at a super fast rate. It is mind blowing, i read a couple of papers every week, test a bunch of stuff and still i feel like i can't keep up! It is so hard to measure the speed the AI development... for instance, i started using qwen2 for my rag applications, and there are a bunch of stuff i should be already be testing, it iz crazy.
Brother, I just want to say please continue doing the great work that you’re doing don’t morph your way of doing things to be like the others which they suck… thank you you’re awesome
You should have your own website that lists your personal AI stack combo's that are good; one for api paid like Claude 3.5 Sonnet and local free "manageable" for common users (like 16GB GPU ram or something). Lists like Maestro + Qwen 2 + DeepSeekV2 OR GPT40 mini + Qwen2 + ContinueDev OR Aider + Gemini + NextJS + Subabase OR Devyan (CrewAI) + DeepSeek-Coder-V2, etc, etc. That would be super cool! I like all the video's you got, but what combo and which stacks to use get's confusing since a lot of these get updates seemingly every month or something.
Can‘t wait to see it on ollama. And btw, yes: it is frustrating when people leave a video after about n minutes. In my videos, 90% vanish after 30 seconds. 🤣🤣🤣
0:30 Yes you are right, not many viewers realized that testing LLM is more valueable than just words or even "benchmark". But, if i may, i want to give you a suggest also. I think you need to improve about video/slide transition in your content. Do not using 'up-down transition' in your video/content frequently (especially if it quite fast), it could make us as viewer has 'cybersickness' symptoms. Better use left-right transition with slow motion of the movement.
This is great suggestion. I'll try to do something about it. Does it also hurt when I scroll pages to show content (Like while talking about the Model and showing the blog post)
@@AICodeKing you could put some anime girls around the border and as the vid gos on she gets less and less cloth.... jokes aside i love your stuff real world AI info
lol love the humor, their loss is our gain...thanks once again AICodeKing for another excellent vid.. keep up great work. would like to see a copilot vid with this one also...be in peace God speed.
Just a heads up, it's available on Open Router AI, which is both supported and recommended by Aider. I think a video comparing the coding abilities between this, GPT-4O Mini, and Claude 3.5 Sonnet when used in Aider would be super useful. What do you think?
Man I asked you before and would like to repeat my question. Can you give us some info on the minimum hardware sources required to run these small modules locally?
australia, mongolia, somalia. also: i dont think you can say so easily that a 12b model is "in the same class" as a 7b model. every billion parameters more counts imo.
I was looking for Australia. But, if it would have told any country that ends with "ila" and it's capital. Then, I would have given it a pass. But, it didn't. So, it's a fail.
@@AICodeKing I'll repeat what I wrote above: there is a ambiguity in the sentence "What is the capital of the country WHOSE name ends with 'lia'" If "whose" referes to the name of the country, then the answer could be what you sad "Camberra, the capital of AustraLIA". Or even "Rome, the capital of ItaLIA" (Italy in italian). The Somalia was also a valid solution. BUT if "whose" referes to the name of the city, then the answer could be BrasiLIA, the capital of Brazil.
@@AICodeKing I have found your channel to be a great resource without hype… too much anyway. I am looking forward to your claude-engineer v2 review if you haven’t done one I haven’t seen yet 😎
Thanks!
Thanks a lot for the support. Also, you're the first comment as well!
In the nvidia NIM you must change the inference parameters. I've changed the temperature to 0.6 (NIM default is 0.2, oo low) and top P to 0.9 (default is 0.5). After this fix, the model could answer the question #1.
Also, there is a ambiguity in the sentence *"What is the capital of the country WHOSE name ends with 'lia'"*
If "whose" referes to the name of the country, then the answer could be what you sad "Camberra, the capital of AustraLIA". Or even "Rome, the capital of ItaLIA" (Italy in italian)
If "whose" referes to the name of the city, then the answer is BrasiLIA, the capital of Brazil.
It was an excellent analysis and I enjoyed it! I will check out the other videos you mentioned.
Thanks for the great suggestion and comment. Although, When a company is hosting a demo, The default settings should be set correct by the company itself. Also,If you look at the Mistral Nemo HuggingFace page you'll see that their page also say "Unlike previous Mistral models, Mistral Nemo requires smaller temperatures. We recommend to use a temperature of 0.3". Also, you are correct about the Ambiguity in the sentence. I know that this sentence causes a confusion. But, if someone (a Human or LLM) reads carefully they'll know that it should be a Country since that's the most likely case here. This question is answered correctly by multiple LLMs in my tests. So, that's why I have kept it.
> "Also, there is a ambiguity in the sentence "What is the capital of the country WHOSE name ends with 'lia'"
if you had finished school you would know that "whose" always refers to the country, since it's closest.
@@monza8844 I'm not native English speaker, so it's true: I didn't "finish English school". And possibly, some LLM didn't finish school either, when we see so many wrong answers for this silly question. But magically some models start answering it correctly when you apply the fix. Give a try and blame me later.
I showed my daughter what deepseek can do by making a tick-tack-toe game in seconds using python, that keeps track of scores and has a slick UI, she says we developers are the modern day wizards🧙♂️ can't wait for NeMo to be on Ollama
Haha! Made me emotional
@@AICodeKing me too bro, keep up the great work. You're my go-to guy for anything AI related.
Encourage her to learn how to leverage GPTs in her daily live.
Get textgen ui, latest version runs.
Not missing a single video of yours keep up the good work -- good sir 🎉
Sometimes i leave the vids at the start cuz my feed is already flooded with AI development content. Working with this shit i, firstly, love content like yours, but also, it is super stressful, AI content goes at a super fast rate. It is mind blowing, i read a couple of papers every week, test a bunch of stuff and still i feel like i can't keep up! It is so hard to measure the speed the AI development... for instance, i started using qwen2 for my rag applications, and there are a bunch of stuff i should be already be testing, it iz crazy.
Nice to see updated test questions!
Brother, I just want to say please continue doing the great work that you’re doing don’t morph your way of doing things to be like the others which they suck… thank you you’re awesome
This might be the best LLM year reward
You should have your own website that lists your personal AI stack combo's that are good; one for api paid like Claude 3.5 Sonnet and local free "manageable" for common users (like 16GB GPU ram or something). Lists like Maestro + Qwen 2 + DeepSeekV2 OR GPT40 mini + Qwen2 + ContinueDev OR Aider + Gemini + NextJS + Subabase OR Devyan (CrewAI) + DeepSeek-Coder-V2, etc, etc. That would be super cool! I like all the video's you got, but what combo and which stacks to use get's confusing since a lot of these get updates seemingly every month or something.
The square face is so cute! 😂
Can‘t wait to see it on ollama. And btw, yes: it is frustrating when people leave a video after about n minutes. In my videos, 90% vanish after 30 seconds. 🤣🤣🤣
I asked Nemo: what is the pinlayout from a NE555 IC, the answer was correct.
Thank you! It would be nice to see how to get the most out of all these goodies in everyday coding.
0:30 Yes you are right, not many viewers realized that testing LLM is more valueable than just words or even "benchmark".
But, if i may, i want to give you a suggest also. I think you need to improve about video/slide transition in your content. Do not using 'up-down transition' in your video/content frequently (especially if it quite fast), it could make us as viewer has 'cybersickness' symptoms. Better use left-right transition with slow motion of the movement.
This is great suggestion. I'll try to do something about it. Does it also hurt when I scroll pages to show content (Like while talking about the Model and showing the blog post)
@@AICodeKing you could put some anime girls around the border and as the vid gos on she gets less and less cloth....
jokes aside i love your stuff real world AI info
That would make me surpass MrBeast.
lol love the humor, their loss is our gain...thanks once again AICodeKing for another excellent vid.. keep up great work. would like to see a copilot vid with this one also...be in peace God speed.
Can you please make a video using NeMo as a Copilot?
Yes, will do. Waiting for it to be available on Ollama.
@@AICodeKingIsn't LM Studio an option?
Just a heads up, it's available on Open Router AI, which is both supported and recommended by Aider. I think a video comparing the coding abilities between this, GPT-4O Mini, and Claude 3.5 Sonnet when used in Aider would be super useful. What do you think?
I'll try to do that.
@@AICodeKing Fantastic! You're the best!
good video. I wonder how it compares to qwen2 7b,
Thanks King!
Thanks a lot!
Man I asked you before and would like to repeat my question. Can you give us some info on the minimum hardware sources required to run these small modules locally?
300th like:D
New good model lol a truly prestigious title from the king
Where should i start as a beginner learning all of this?
Its available in LLM Studio but not working.
SVG face is much easier than SVG butterfly. But what do I know, I am not a model
Even in q3 it is better than llama 3 8b with q6. Also great for roleplay because no builtin moderation.
australia, mongolia, somalia. also: i dont think you can say so easily that a 12b model is "in the same class" as a 7b model. every billion parameters more counts imo.
Another new one already? As far as I know, Codestral was released a few days ago.
Yes, it was.
What is current best model
DeepSeek is currently the best.
Hunky Dory.... someone is a mamss boy lmao 🤣
Also how can I get money in llms ?
What do you mean by "Money in LLMs"?
I MIGHT HOW CAN I SELL THE LLM LIKE CHATGPT TO A BETTER VERSION FOR EXAMPLE OR WE USE THE LLM TO BUILD US SOMETHING WE CAN SELL IT ?
When a new model isn’t compared with Qwen and DeepSeek you know something is wrong
Yes
Bro which model is the best to use as php developer ?
All are generally good. Currently, DeepSeek-Coder-V2
@@AICodeKingthanks alot for fast reply bro
But its knowledge stops in the year 2021!
E
Australia? Somalia?
Sure it's not China and Beijing but multiple countries end with lia
I was looking for Australia. But, if it would have told any country that ends with "ila" and it's capital. Then, I would have given it a pass. But, it didn't. So, it's a fail.
@@AICodeKing I'll repeat what I wrote above: there is a ambiguity in the sentence "What is the capital of the country WHOSE name ends with 'lia'"
If "whose" referes to the name of the country, then the answer could be what you sad "Camberra, the capital of AustraLIA". Or even "Rome, the capital of ItaLIA" (Italy in italian). The Somalia was also a valid solution.
BUT if "whose" referes to the name of the city, then the answer could be BrasiLIA, the capital of Brazil.
Thanks!
Thanks!
Thanks a lot for the support!
@@AICodeKing I have found your channel to be a great resource without hype… too much anyway. I am looking forward to your claude-engineer v2 review if you haven’t done one I haven’t seen yet 😎
I'll do that