How to install llama 3 on Windows Mac and Linux
Vložit
- čas přidán 17. 04. 2024
- Learn how to download and install Llama 3 on your computer with this quick and easy tutorial!
Download ollama from ollama.com
Whether you're using Windows, Mac, or Linux, I got you covered! 💪💪s
Thank you! So many complex versions of this and the video is only 2 minutes.
Your welcome!
Thank you for being direct, clear, and straight to the point.
Hey you're welcome, I've been playing around with the API and Python package. More videos to come!
Just installed this on a Linux box - and wow, this one is great~
Yes please, more with llama👍
Thank you!
Is the 7B the largest that can run on a 4090 or M3 Max with 48GB? Or, are there options for getting the 70B model to work? When you installed it didn't seem to give a choice.
It defaults to the smaller model and and smaller quantization which loses a bit of accuracy but dramatically reduces hardware required. Check out the “all tag” link on the model page, ollama.com/library/llama3/tags there's about 60 variations of model size vs quantization and floating point size. I'd recommend giving the default 70B a go as a starting point and testing high Q and FP value from there.
Your trade off will be inference time vs accuracy and there's many schools of thought on where better prompting can help.
Let me know how you go!!
is this llma3 or llma3:instruct?
thanks chief!
Hi! Thanks for the video. Could I ask you for some clarification regarding the offline-usable models? I constantly hear about new open-source models being released, but due to limited time, I never get to try anything because I'm not familiar with the suitable software. I have three needs:
1) to interact programmatically with these models by writing specific scripts;
2) to have a chat tool similar to ChatGPT that works exclusively on my computer without interfacing with third-party systems;
3) to install an alternative to Github Copilot in Visual Studio Code, whenever possible;
Could you suggest me a tutorial to follow, a blog to read, or which software to install to achieve these goals (I heard about LM studio or similar...)?
I work with Windows and Mac. I'm familiar with the terminal and regularly program with Python. Thank you very much for any help you could give!
So why do you think someone else has time to waste, responding to your "needs" while you can't be bothered to find what you are looking for?
@@TheResistanceTv don't be bitter. There is nothing wrong with asking questions. Also this is how things get done with open source. It is vital to human existence that we develop AI locally amongst people where it can not be tied to govt or corporations. All the social media that we see controlled by corporations could be built on top of IPFS where it can not be censored or controlled.
His exact words were”can you recommend a tutorial or blog? Your response is just offensive. Try rereading the post.
What kind of Setup do you use? How much RAM? What Kind of Processor? Thanks
Hey I'm running this on a Macbook air would you believe, it's an M2 with 24gb of RAM which helps. I've got a mix of other windows/Linux computers I'm going to benchmark on and share results
@@MakeDataUseful that would be great :)
There is a problem and when you close the terminal and when you open it again it gonna forget the last chat and information
Correct, there are ways to hold onto context.
When I use llama 3 8B on ollama or LM Studio, it is much dumber than on OpenRouter. Even after resetting all parameters to factory and loading the llama 3 preset. Even with the full non-quantized 8-bit version on LM studio.
Weird! Drop me the prompt and response and I'll give it a test on my machine
@@MakeDataUseful It's because I was actually using the 8-bit quantized version, which is inferior compared to the full model. Thank you for answering and for the great video!
It would be great to have a tutorial on how to run the full model locally, I still have to figure that out.
EDIT: is it llama3:8b-instruct-fp16? Trying that out, I will post an update here.
Even the llama3:8b-instruct-fp16 is much dumber than the full version on openrouter.
Basically, I'm giving a json string representing all the modification to a trello card (you can export the json of any trello card with the "share" button). The title has been modified a couple of times. It's a long json string, but within context limits. The LLM is asked to extract the current card title. Not only the local model gives the wrong answer, but also it doesn't share the reasoning behind the answer even if asked, whereas the full version on OpenRouter almost always give the correct answer, and if asked always shares the detailed reasoning behind the answer.
How can I run the full model with Ollama or any other way locally?
EDIT: this is the prompt to replicate it: "please extract the card title from the following json:"
EDIT2: and this is the extended prompt: "please extract the card title from the following json, but only after explaning me how you would it, the reasonings behind your choices and the steps you would take to do it."
@@emanuelec2704 This is really interesting, I have also been getting some weird responses form time to time.
I think there is some smarts we can come up with and test. I've been getting some better results with multiple prompts and chopping the task up into smaller chunks and engaging llama3 separately to solve the overall problem.
I'll put together another video :)
I get nothing but gibberish output when I install it and run ollama run llama3 on my M3 Max MBP and ask "Who are you?"
Weird, sounds like something's not right. I get a lot of gibberish when I crank up thinks like temperature and other params when defining a custom model file. Shouldn't be the case, what about llama2, same?
With ChatGPT I just go there and start talking to it. This video is, as far as I can tell, by a programmer, for programmers, using a programmer's interface. So I need to know how to program in Python? And it will interact with me as if I'm on a computer terminal? If I install something and something else and use a hook to access it? Where is the web site where I just go and start using it without having to know any of this?
You might be interested in LM Studio
This was not helpful. You already installed it Llama3
Oh no, did you need more help?
Yes, pls use a Website which url you don't show and pls don't take the url in the description.
Okay that's totally my bad, description updated!
The name has nothing to do with the content at all. Where’s the installation process, which are the recommended hardware prerequisites, where’s Linux and where’s Windows.
What was the initial sense of this video?
Hey RealEstate3D thanks for taking the time to leave some feedback. Let me know if you are still stuck, more than happy to go into a little more detail.
@@MakeDataUseful I am not stuck. I expected to find the information I already told you 2 hours ago. It might make sense to describe the content of your video honestly. Instead it seems you misnamed your video on purpose. That’s pretty questionable. As also your standard answer is kind of low effort.
@@RealEstate3D Okay, shoot me a video title and I'll update it