Llama 3 RAG: Create Chat with PDF App using PhiData, Here is how..
Vložit
- čas přidán 1. 06. 2024
- 🚀 In this tutorial, we dive into the exciting world of building a Retrieval Augmented Generation (RAG) application that handles PDFs efficiently using Llama 3 powered by the Groq API with PhiData as the main Python package. Whether you're looking to create an app with Groq or prefer keeping your data 100% local with Ollama, this video covers it all!
Github Source Code: git.new/groq-rag
👨💻 What We Cover:
Setting Up Your Environment: We start by setting up the necessary development environment.
Installing Packages: Next, we handle all the package installations required for our application.
Creating a Database: Follow along as we create a database using Docker for storing our data.
Running the Application: Finally, see how to run the application where you can upload PDFs, ask questions based on the uploaded content, and receive precise answers.
🔗 Resources:
Sponsor a Video: mer.vin/contact/
Do a Demo of Your Product: mer.vin/contact/
Patreon: / mervinpraison
Ko-fi: ko-fi.com/mervinpraison
Discord: / discord
Twitter / X : / mervinpraison
Code: mer.vin/2024/04/llama3-pdf-ra...
Phidata Basics: • PhiData: How to Seamle...
📌 Stay tuned as I walk you through each step, from cloning the repository to launching the app, ensuring you know exactly how to replicate this setup on your own!
👍 Don't forget to LIKE, SHARE, and SUBSCRIBE for more tutorials on Artificial Intelligence and data management. Your support helps us create content that helps others!
Timestamps:
0:00 - Introduction to the Tutorial
0:02 - Creating the RAG Application Overview
1:01 - App Demonstration and Features
1:35 - Environment Setup and Package Installation
2:48 - Database Creation with Docker
3:36 - Running the Application
4:06 - Uploading PDFs and Query Interaction
5:41 - Alternative Local Setup with Ollama
6:20 - Running the Local Application and Uploading Files
7:01 - Live Question and Answer Test
#PhiData #PhiDataOpenSourceLLM #PhiDataOpenSource #PhiDataFunctionCalling #PhiDataOlama #AssistantsAPIFunction #CreateAIAssistants #AIAssistant #FunctionCalling #Function #PhiData #PipInstallPyData #PhiDataRAG #PhiDataRAGPDF #PhiDataRAGGroq #PhiDataRAGOlama #PhiDataGroq #PhiDataRAGApplication #PhiDataRAGApp #OlamaRAG #GroqRAG #OlamaPDFRAG #GroqPDFRAG #PDFRAG #RAGPDF - Jak na to + styl
@Mervin Praison you are awesome. Can watch your videos all day!
Thank you 🙏 😊
Macha. Nee kalakura. Your content is so valuable. Keep them coming. Bless you!
Awesome tutorial Mervin! Thanks!
This is just phantastic ... I mean the tool, and your video as always top notch
Brilliant, thanks mate😊
Your content is amazing!
Keep up Your Amazing Work!👍👍👍👍👍
Another OUTSTANDING video!
Thank you
Thanks for sharing, very useful
Very good work.
Love this brilliant!
amazing demonstration, thanks for sharing your expertize. Hope you could talk more about PKM.
This Guy is Excellent! 👍👍👍👍👍
Your slides are clear and perfect to explain all thing for me. And it can be seen as a tool that it needs a PDF as input and then the user will get a answer related to it via giving a question to llama3. It looks like an interaction between chatgpt3.5/4.0, but it is in local. That's pretty good. If you can do a chatbot whose character can be customized, it can be better.
I always watch and love each and every single video. One video I'd like to request and I agree many would agree is how can we do multimodal RAG with PDFs that have images and tables and text. I've tried watching other videos and it's quite confusing + doesnt seem quite good or production level where we can tune it to our own needs or sometimes they do it separately by just providing image and asking the LLM to describe it. Please do a video about it
This is awesome. I'm trying to understand the fundamentals a bit better. Where are the pdf documents stored locally?
Another great one @Mervin, thank you. Feasibly, how many pages of PDFs could we feed to PhiData with this method and get nuanced responses to questions? Could that include spreadsheets & financial analysis?
Excellent 🎉🎉🎉
Hi Mervin i come across this channel. Very clear, concise and to the point. Can you mention minimum required hardware specification in each video?
Awesome! ❤❤
Thank you
@@MervinPraison, you‘re welcome! Your video brought me to the GitHub project Langflow. I have to try it, it supports ollama, Agents and RAG through visual programming components. And a low code approach for calling everything through python and a json file.
Thank you for the video Mervin. But it is not newbie friendly. I am not interested in groq so skipped to the ollama chapter but I had to watch groq parts for python and docker installation etc. Would've been better if you split them in their own video.
Hi, Mervin. Thank you for your excellent presentation and tutorial. Could you please perform the procedures in Docker Compose?
@Mervin thank you for your video. It works great. I have a question, some of the PDFs I am testing this with fail. Any suggestion of where I need to look for this?
very useful
Thank you so much for the great content !!! It's awsome like you say. Im wondering if could be done with LMstudio, which seems easier to use for a noob like me, i can easily change system promt, context token, gpu layers... Otherwise im wondering also if it would be possible to make that run locally and forward that to a website which would be accessible through a phone.
2:37 direct set the GROQ_API_KEY in environment variable instead of 'export'
Thanks!
Thank you
How can I implement this project using OpenAI models ? Can anyone guide me with this?
hello, can you show a video where we can upload all the documents, then we can chat with it without needing to keep uploading / storing it to the database? meaning to chat with the vectorised db
the documents are stored in the database and do not need to uploaded again unless the database is cleared. Only the streamlit UI doesnt show the docs on refresh :) but the docs are there in the database
Thanks for the great content, straight to the point ! Its awsome like you say. Im wondering if i could do the same with LMstudio ? it seems more practical for a noob like me, i can easily set-up some parameter through the interface like system promt, context token and the gpu layers for optimisation. That would it be awsome as well if it was possible to make it run locally and forward it to a website that we could run on a smartphone ?
@Mervin Praison is there any way to access phidata via api, if I run this on AWS or something? I want to use a diffrent front-end?
shame it needs docker, I just dont have the space for WSL2 to start growing massive so I avoid docker. Also on windows 10 and dont want to run virtual environment (bios issues for other software) on this machine so if there is another way to get the pgvector db working, would be great to know it.
Ive been trying to edit the llama3 temperature cuz it tends to hallucinate a lot, where should I go to edit it?
Can this be used to extract specific data fields from multiple pdf files and output the result in Excel?
how can you increase the output limit so that it can giv longer responses? Installed it and everything is working but if I have to lets say translate a long scientific article it would cut off
Great video. What if i wanted to build a RAG that will work on a website. Where the data is stored on a web server or on a cloud database
This App includes option to scrape websites too (along with PDFs) :) data right now is stored locally in a portgres database
Can you explain this, I tried uploading a pdf. The knowledge base only contain two pages.
Terminal output:
"INFO Committed 4 documents
INFO Loaded 364 documents to knowledge base"
Groq:
"The confusion arises from the difference between the number of documents committed (4) and the number of documents loaded into the knowledge base (364). The knowledge base, however, only contains information about 2 specific documents, which are excerpts from the book "Friction, Wear, Lubrication: A Textbook in Tribology" by Ludema and Ajayi.
It appears that the loading process involved 364 documents, but only a subset of those documents (4) were committed, and even fewer (2) are represented in the knowledge base. The discrepancy suggests that there might be additional information or documents that are not currently reflected in the knowledge base."
thanks to video
i have question
1)why you choose llm groq? speed reason?
2)could i replace groq to local llm (llam3) that is possible to use functino calling tool?
Yes, LLM Groq = speed
Yes I have showed in the later part of the video , how to replace that with ollama (100% local)
Yes function calling is possible in Llama 3
Thx
Can I suggest a tutorial video? I'd like to see llama3 run locally with ollama with autobuild agent autogen. I'm trying to figure out the code now using just config_llm instead of .env or oai.
You Read my mind: czcams.com/video/pKFy82m5XmA/video.html
In Windows, how can I copy all the run code for Docker together in PowerShell? It automatically run each line separately and it have not been done. Thank you so much
The \ doesn't work in windows, Please can you run:
docker run -d -e POSTGRES_DB=ai -e POSTGRES_USER=ai -e POSTGRES_PASSWORD=ai -e PGDATA=/var/lib/postgresql/data/pgdata -v pgvolume:/var/lib/postgresql/data -p 5532:5432 --name pgvector phidata/pgvector:16
Please, can you tell me if this is possible with piano sheet music?
Only pdf can be done. Can we do excel file
@MervinPraison is it possible to run csv,SQL,json instead of pdf
It's based on the CSV, SQL, JSON you use. You might need to parse the data and reformat it and then sent it for embedding.
You might need to modify this code to do that.
@@MervinPraison Can you please make a video on this topic. Thank you so much in advance 🔥
i am a newbie using windows somehow i managed to reach the docker part and when i put the docker run \d
(phidata) PS C:\Users\khanf> docker run
"docker run" requires at least 1 argument.
See 'docker run --help'.
Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]
Create and run a new container from an image
This is the error i am getting
The \ doesn't work in windows, Please can you run:
docker run -d -e POSTGRES_DB=ai -e POSTGRES_USER=ai -e POSTGRES_PASSWORD=ai -e PGDATA=/var/lib/postgresql/data/pgdata -v pgvolume:/var/lib/postgresql/data -p 5532:5432 --name pgvector phidata/pgvector:16
@@phidata thank you so much for the help will try this. Thank you so much really appreciate it
@@phidata thank you phidata it worked with your help and mervins i was able to create my 1st ever app. forever gtrateful
The only problem is the 200 MB limit to upload PDF’s … is there a way to increase that?
hi, you can change that in streamlit. its just a streamlit default, you can upload gbs if you like :)
@@phidata thanks I will look into that
How can I change the models directory of ollama on windows 11?
From the ollama doc it says:
First Quit Ollama by clicking on it in the task bar
Edit system environment variables from the control panel
Edit or create New variable(s) for your user account for OLLAMA_HOST, OLLAMA_MODELS, etc.
Click OK/Apply to save
Run ollama from a new terminal window
@Mervin getting error ERROR (psycopg.OperationalError) connection failed: FATAL: password authentication failed for user "ai". Any idea, why it is not connecting ?
Hi there, is the database running? The command is
docker run -d -e POSTGRES_DB=ai -e POSTGRES_USER=ai -e POSTGRES_PASSWORD=ai -e PGDATA=/var/lib/postgresql/data/pgdata -v pgvolume:/var/lib/postgresql/data -p 5532:5432 --name pgvector phidata/pgvector:16
@@phidata yes database is running and it is weird, giving password error
@@PythonLearn-nv8cd hmm maybe delete the container + volume and then recreate? probably initialized before and has some old password?