How to Install and test LLaMA 3 Locally [2024]
Vložit
- čas přidán 19. 04. 2024
- After the release of Llama3 i thought i should make a view to walk anyone who is looking to use it locally. i hope this video helps:)
Related links:
Download Ollama: ollama.com/download
Open web ui: github.com/open-webui/open-webui
Llama3 download link: : ollama.com/library/llama3
Link from video: llama.meta.com/llama3/
Release note from Meta: ai.meta.com/blog/meta-llama-3/
- - - - - - - - - - - - - - - - - - - - - -
Follow us on social networks:
Instagram: / codewithbro_
---
Support us on patreon: / codewithbro
#ai #artificialintelligence #llama3 #metaai #install #macos #machine #machinelearning #aitools #programming #softwaredeveloper #softwareengineer #webdeveloper #developer #iosdeveloper #mobiledevelopment #coding #coder #javascript #developer #computerscience #computersciencestudent #100daysofcode #html #css #programmer #vue #npmpackage #npm #package #CodeNewbies #Code_with_bro #code_withbro #youtubechannel #youtube #youtuber #youtubers #subscribe #youtubevideos #sub #youtubevideo #like #instagram #follow #video #vlog #subscribetomychannel #gaming #music #explorepage #love #smallyoutuber #vlogger #youtubegaming #instagood #llms #youtubecommunity #likes #explore #youtubelife #youtubecreator #ps #bhfyp #fotiecodes - Věda a technologie
Model variants
------------------------
Instruct is fine-tuned for chat/dialogue use cases.
Example: ollama run llama3 ollama run llama3:70b
Pre-trained is the base model.
Example: ollama run llama3:text ollama run llama3:70b-text
Nice video! Can you do one on how to set up a local database out of pdf files? So the A.I would be able to search that pdfs and answer questions regarding the content
I love how in the intro, we're being told about a new AI, by an old one.
Hahahaa
How can I run the original not_quantized version of llama 3-8B(It is almost 15gb)?
My best suggestion for you is to get access to the huggingface repo. You will have to apply to meta for this. Here is a helpful link huggingface.co/meta-llama/Meta-Llama-3-8B
@@codewithbro95 I get it,but I am looking for a web UI to run it.
@@gallyyouko5993 you can use this: github.com/open-webui/open-webui
What I used in the video :)
what about hugginggface chat ui?
I recently installed ollama open webui, worked fine, after 1day I started getting some error in docker getting restarted because of some cuda error.
I don't want to trust it now because I cannot find a fix, although reinstalling fixed it.
And on terminal it was always working and working faster than web ui
Terminal works just fine for me as well, always reliable.
Sometimes ui just helps keep things organized and in one place, easily accessible
How can I run the larger version of the model?
ollama run llama3:70b
how do you add your models I have ollma set up in my ps command line but when I used the UI it didnt show up and Idk how to upload them please help!
Normally you do not need to upload anything. If you have setup open ui properly then it should just show up, nothing else to do on your end. How did you install the web ui?
@@codewithbro95 I fixed it. I went into settings on the UI version you showed in your video. Then I went to settings and copy and pasted the code from Ollama Download Model Page and it downloaded the model and worked. I installed the web ui via the method you showed on the video and I used the local version for the docker set up.
@@BrandWorksApp glad you fixed it, happy the video help:)
I didn't really get the part on docker, can you help me ?
How can I help?
Is there a way to run it on custom developed UI? Can someone please guide me as to how?
You can use the ollama API
Docs here: github.com/ollama/ollama/blob/main/docs/api.md
How to change the location of the insallation and the location of the download of the model?
czcams.com/video/uj1VnDPR9xo/video.html
You can have a look at their repo, issue here: github.com/ollama/ollama/issues/2551
Hope that helps!
How can I run ir with langchain?
python.langchain.com/docs/guides/development/local_llms/
I'm glad that my PC handles 70B with 16GB VRAM and 64GB of RAM. I wonder what the absolute minimum requirements are for 400B.
You good to go, what os you running on? No one knows yet for the 400B param until it’s available, but I’m pretty sure there’s gonna be a quantized version which would probably need 64gb RAM. Maybe!
How much G of ram would be needed
What version do you wanna run?
@@codewithbro95 well the best with 16 gb of ram on linux unbuntu
@@codewithbro95 best model on 16gb ram linux unbuntu
7:30 ollama web ui
how can we add a model???
Follow the stops and run the ollama command as in the video, it will download the mode to you computer
@@codewithbro95 I meant there is no option to select a model in the bar of Ollama Web UI. How can ve download it there?
@@rs-wd9or no need to, ollama web works and integrates with ollama automatically, so all the models you download on ollama will be listed automatically there
@@codewithbro95 I have downloaded the llama model using the command "ollama run llama3", but the Open Web UI didn't see the models. What should I do now?
I use this command to start open webui:
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
why do all of these dont have a proper graphical interface... i hate having a black screen to stare at
You can use the web ui I showed in the video
better gpt 3.5?
The 400B maybe, but it’s yet to be released!
llama 3 in action:
>>> How many liters of water per minute can a Dutch windmill pump out?
That's an interesting question!
The answer depends on the specific design and size of the windmill. However, I can give you some general
information.
Traditionally, Dutch windmills are designed to pump water from shallow sources, such as polders or wells, for
irrigation purposes. The capacity of these windmills varies greatly, but a typical small to medium-sized windmill
might be able to pump around 0.5 to 2 liters per minute (L/min).
Some larger industrial-scale windpumps can pump much more, up to 10-20 L/min or even more, depending on the
turbine design and the pressure head of the water.
Really?
Its really slow
There are minimum requirements for tanning the model, it works pretty well on my M1 16gb RAM and 8 core GPU
@@codewithbro95 does a pre-trained model with 400B work on a M3 128GB RAM?
@@-_.DI2BA._- Not sure, 400B is yet to be released by Meta. they are still training
😂😂😂😂😂
content is fine, but your explanation wise ...............................................................................................................................................
when you say just go on the documentation, what so ever, you didn't said shit
@@viniciusmelo5652 Thanks for the feedback, will try my best to do better next time...
stop click baiting, the 400b is not even out
Mark talks about it in the video?
@@codewithbro95 if 70b is already this good 400b is going to be crazy
🔥