Retrieval Augmented Generation with Python+Ollama+Phi3+ChromaDB | How to RAG with a local model

Hi and welcome to DevXplaining channel! Todays I've got a long-form video of a Retrieval Augmented Generation (RAG) using Ollama, ChromaDB, and a little bit of Python. We'll be using RSS feed as our source, and create a program that's able to query that source and use generative AI model to answer questions. Cool thing in todays video is that it's all open source, it's all local model, so this is something you can run on your own computer.
I wanted to take my time and write the code line by line, explaining as we go, so there was no way to do a short video - this is going to be rather long one. But I think this topic and knowledge is a must-have for any developer in 2024 - and I hope this will be clear and useful for you too.
As always, code and references in the video can be found from links below. Happy coding!
And remember to like, subscribe, comment and feedback as you see fit, you know the drill by now :)
Timecodes:
0:00 - Blah blah blah
1:35 - Explaining the setup
4:00 - Prerequisites & How to Hello World with a local model
5:13 - Add necessary libraries
7:02 - Grab the NYT RSS feed with SpeedParser
14:00 - Put the data in ChromaDB Vector database
20:22 - Similarity search into vector database
25:50 - Use Ollama and Phi3 local model to ask questions about the data
36:40 - The Good Stuff
Links:
- github.com/crystoll/ollama-rag
- github.com/ollama/ollama
- github.com/chroma-core/chroma
- github.com/ronikeuru/local-ra...

[@JoanApita]

Thank you for sharing this video. I learned a lot today. Please keep them coming. Thank you

[@DevXplaining]

Thank you for your feedback! Much appreciated!

[@leonardosouzaconradodesant6213]

Great thank you. And by the way, I'd like that video using chromaDB as an application node running in background. See you!

[@DevXplaining]

Thank you!

[@rasmustoivanen2709]

Great video!
Ideas for future videos:
Try out different embedding models (I think the ChromaDB default is all-MiniLM-L6-v2)
Set up a separate embedding database. Maybe something more familiar with developers like Postgres + pgvector + hnsw indexing (Important to trial out different index building parameters)

[@DevXplaining]

Thank you! Very good ideas!

[@cesarhernandezcinfo]

Great video, very practical information. Thanks

[@DevXplaining]

Thank you! Much appreciated!
More to follow on the topic. Stay tuned. :)