Llama2 Chat with Multiple Documents Using LangChain
Vložit
- čas přidán 5. 08. 2023
- 🦙Llama2 With 🦜️🔗 LangChain | Chat with Multiple Documents Using LangChain
In this video, I will show you, how you can chat with any document. Let's say you have a folder and inside the folder, you have different file formats. Let's say you have PDF file. You have text file. You have read me file and others. I will show you how you can take all of your data, split the data into different chunks, do the embeddings using the sentence transformers via Huggingface embeddings, store that into the Chroma vectorstore. Finally, you can just chat with your own documents and get insights out of it, similar to ChatGPT for with your own data.
What I noticed from the experiment is that Llama2 model is not that good compared to OpenAI model for QA Retrieval part. It might depend upon your machine / hardware. All the mentioned links are either in this description or in the readme file of the github repository.
Happy Learning 😎
👉🏼 Links:
💻 GitHub repo: github.com/sudarshan-koirala/...
Chainlit website: docs.chainlit.io/overview
HuggingfaceHub API Token: huggingface.co/settings/tokens
------------------------------------------------------------------------------------------
☕ Buy me a Coffee: ko-fi.com/datasciencebasics
✌️Patreon: / datasciencebasics
------------------------------------------------------------------------------------------
🔗 🎥 Other videos you might find helpful:
🔥 PrivateGPT: • PrivateGPT 🔥 | No Data...
⛓️ Langflow: • ⛓️ langflow | UI For 🦜...
⛓️ Flowise: • Flowise | UI For 🦜️🔗 L...
🔥Chainlit playlist: • Chainlit
🦜️🔗 LangChain playlist: • LangChain
------------------------------------------------------------------------------------------
🤝 Connect with me:
📺 CZcams: www.youtube.com/@datascienceb...
👔 LinkedIn: / sudarshan-koirala
🐦 Twitter: / mesudarshan
🔉Medium: / sudarshan-koirala
💼 Consulting: calendly.com/datasciencebasic...
#llama2 #llama #chainlit #langchain #llm #chatwithpdf #chromadb #datasciencebasics - Věda a technologie
Your video is amazing, and I'm grateful for your effort. Are you thinking of making a video about DemoGPT?
Thanks for the feedback. Not thinking right now but thanks for mentioning it. I could in the future 😎
@@datasciencebasics Thank you! We look forward to hearing from you:)
Great video. I missed that part regarding RetrivalQA models, I guess... Which model should I use ? Thanks ! :)
hello, You are welcome. RetrievalQA model takes the load_model() function as llm where llama2 model downloaded from Huggingface is being used.
Thanks for the tutorial. Can you make another tutorial using Llama-2 70B model ?
You are welcome. My main motive of creating video is to guide how to use. Now that is done, why don’t you just plug in 70B in existing code and try it yourself ? I hope you will learn more by practicing yourself 😎
It seems like the result of llm got a lot of redundant sentence, is there anyway to improve the result? Or llama2 7b conversation is consider buggy?
In my opinion, its not buggy but running on commodity hardware (CPU) is one reason. And probably providing more concise prompt might help.
it is working for image based pdfs but it is not working for text based pdfs why ??
The page opens in the browser, but the form does not load
I trying to use llama-2-13b-chat.ggmlv3.q8_0 model, but downloading it locally using wget give me an error: Stream too long.
So I downloaded the above model manually and copy-pasted the file in model folder. But getting this error: Failed to load model: No model file found at model/llama-2-13b-chat.ggmlv3.q8_0.bin
Then tried the same process with "llama-2-7b-chat.ggmlv3.q8_0" but the chatbot shows exact same error as above (even changed the model in script)
is it possible to deploy it on some cloud platform??
Hi Sudarshan, as llama2 released, is there any way to apply llama2 chat 70-B on this application?
hello, you can use but be aware of the hardware requirements. Using quantized version works in community hardwares.
Very nice video. I am also building a LLM with a goal to build chatbot well versed in credit card information but LLama V2 7B is not giving me desired result. Unfortunately openai api charges a lot , so do you have any idea whether is there a better model?
hello, Unfortunately openai api is costly. You can try bigger version of Llama2 or try in better hardware / RAM. You can even try Falcon model if that might help.
can you also make a video about the best model for coding and whats the best model for chatting like Ive heard. mistral models are good but dont know whats the sota for opensourcw now. it be great if you could shed some light on all 8f this. thanks a lot in advance ❤
Hello, There are so many models popping up everyday / week which claims to overthrow the existing best model. Video might be outdated in a day or week or month, thats the reason I am not creating videos on those topics. As far as now, yes I am also finding mistral7B best at coding but lets see untill when 😄
I recommend you to watch the Huggingface leaderboard space for up-to-date info. huggingface.co/HuggingFaceH4
@@datasciencebasicsthanks a lot really appreciate it man. God bless you
i noticed that you didnt download any embeddings model for this, does the llama2 model is the only one we need to download?
It is downloaded when you run the ingest python file. Please view the code.
Help please i get : {"detail":"Not Found"}
Amazing 🎉, I want to load the model that I downloaded and integrate that in flask api . When I ask questions it should respond.. pls give a way for it . Pls reply sir
hello how can I suggest you to create a flask app in a comment which is not relevant to this video. I hope you can find some blog post or videos they explain loading models via flask.
can you make video how can convert this modal to exe
Is it possible to do this without downloading the 6 GB file
As we are running it locally we need to download it to run locally. Another option for example, is to use API from replicate, but it can cost you. Link -> replicate.com/pricing
will it run properly for my laptop which is on;y 8gb ram
IMO, it will not run efficiently. As far as I know, 16 GB is required. But you can give a try and see how it behaves.
Sir, could you please use ollama to query PDF documents.
Please create video on it. Thanks
There are many videos on my channel for that, can you please check other videos too :)
can i make the same if i download the model and use it ?
give a try :)
Does it support multiple documents?
yes, it supports. Give a try.
I am getting errors on langchain and getting error as "AttributeError: 'Message' object has no attribute 'replace'" Can you please help me
In the function "async def process_chat_message(message):".
Change: response = await qa_chain.acall(message, callbacks=[callback_handler]) to response = await qa_chain.acall(message.content, callbacks=[callback_handler])
@@Alelinis Thank you, I had the same issue and your fix seems that solve the problem.
It is throwing an error :- No directory found at ./db
It clearly says the error, as no directory found. please refer to the code and follow the video step by step.
Please put a video to deploy online
Did you get how to deploy online?
CAN WE UPLOAD THIS ON OUR SERVER AND LET IT RUN ON OUR WEBSITE????
Yes you can. But always read the terms and conditilns and official documentation to get more information.
@@datasciencebasics 👍
IS IT FREE TO USE COMPLETELY??🤔🤔🤔🤔🤔🤔
yep its free unless you have been making millions in revenue using the model. Refer to Llama2 official documentation for that.
@@datasciencebasics 👍
sir you dont explain clearly for beginners . it feels like you are in rush
thanks for the feedback, will keep that into account in upcoming videos !!