Ollama: Run LLMs Locally On Your Computer (Fast and Easy)
Vložit
- čas přidán 16. 07. 2024
- With Ollama, you can run local, open-source LLMs on your own computer easily and for free. This tutorial walks through how to install and use Ollama, how to access it via a local REST API, and how to use it in a Python app (using a client library like Langchain).
👉 Links
🔗 Ollama GitHub: github.com/ollama
🔗 LLM Library: ollama.com/library
🔗 RAG + Langchain Python Project: • RAG + Langchain Python...
📚 Chapters
00:00 How To Run LLMs Locally
01:07 Install Ollama
02:45 Ollama Server and API
04:15 Using Ollama Via Langchain
Definitely would love to see more videos on training the models, finetunin and adding documents etc.
Love this channel, your content is clear explanation. Please do one video for fine tuning LLM for any specific task with one real-time use case
First I would like to thank you, As a beginner you have provided a solid platform to me. I would also eagerly wait for your next video on how to train model locally and fine tune . Thank you so much once again .
Thank you, glad you liked it!
Love this channel, very clear and factual explanation of the topic
Thank you, I really appreciate hearing that :)
Thanks, the simple codes you showed helped me a lot!
Glad to hear that!
Sound quality is much better in this new setup. I just saw your FastApi video (if you are wondering why I am appreciating the sound of the video 😂)
Haha I'm glad to hear that. I made some adjustments to my set-up, glad it's paying off :)
It's Good. Very nice explanation😀
thanks
You're welcome!
Diving a bit deeper into embeddings would be nice. And vector database. How to know the quality of your embeddings. What made you go to bedrock?
Ultimately, the only way to test your app effectively is to do end-to-end testing with a bunch of sample answers/questions. If you get good results, and the embeddings help you find the right items in the DB, your embeddings are probably good enough.
As for Bedrock: my decision wasn't exactly to use Bedrock-it was more to use a larger cloud-based model (OpenAI or Gemini would be fine too). I just used Bedrock because my developer stack is quite biased/skewed towards AWS, so I guess just personal choice.
@@pixegami Could I hire you to assist me on a project? Very similar to your tutorial, it'd save me time?
Super interesting! Do you know what are the RAM requirements to run this locally?
How you must frame your question is how the system requirements are changing from OLLAMA 1 to OLLAMA 3? So even if you invest heavily now, when the future versions come out and as LLMs keep growing size exponentially, there's no point running locally unless you're investing in hardware that requires to be upgraded every 4 years. Whatever you earn as bonus every year, keep it aside to invest in hardware, online tutorials and books.
Finally you could put up your question to ChatGPT itself instead of seeking answers here.
@@manoharmeka999 you seem to be very strongly opinionated. But there are applications like when you are dealing with sensitive documents where you might not want to expose this info to open ai or anyone else via a query. Also that money might be a lot for people in India but are just a business expense for some others.
Can i install ai town with this. Other method was too complex for me as i am new to alot of this
Sorry, I'm not familiar with "AI Town" - is it this? github.com/a16z-infra/ai-town
It looks like you can use Ollama as a backend: github.com/a16z-infra/ai-town?tab=readme-ov-file#3-to-run-a-local-llm-download-and-run-ollama
can you do new episode combine with "Ollama: Run LLMs Locally On Your Computer (Fast and Easy)" and "Langchain Python Project: Easy AI/Chat For Your Docs"? Which you will just used local LLM to process the Docs
Absolutely :) A lot of people have been asking for this, so that's going to be my next video (plus a couple of other top requested features).
can i integrate ollama with java?
Doesn't seem like there's a first-party Java integration, but there are some third party ones: github.com/amithkoujalgi/ollama4j
Or you can use Java to make a standard REST API call to the local server directly.
It's interesting to train custom LLM instead of using RAG [2:45]
I haven't looked into training an LLM. It's a bit more challenging and expensive to do than just using an off-the-shelf model, but it's a great way to gain more control and quality from the LLM.
@@pixegami there is the limit for llm context so it's hard to convert knowledge base into a little text. There are no lot of examples of using ldap3 library on python, even llama3 knows it. But it's hard to produce example with my own messaging library in my corporation.
How about Meta’s LLM?
Absolutely. This is available on Ollama. You can use `llama2`, but now `llama3` is also available: ollama.com/blog/llama3