Step-by-Step Guide to Building a RAG LLM App with LLamA2 and LLaMAindex
Vložit
- čas přidán 19. 06. 2024
- In this video we will be creating an advanced RAG LLM app with Meta Llama2 and Llamaindex. We will be using Huggingface API for using the LLama2 model.
github: github.com/krishnaik06/Llamin...
----------------------------------------------------------------------------------------------
Support me by joining membership so that I can upload these kind of videos
/ @krishnaik06
----------------------------------------------------------------------------
►Data Science Projects:
• Now you Can Crack Any ...
►Learn In One Tutorials
Statistics in 6 hours: • Complete Statistics Fo...
End To End RAG LLM APP Using LlamaIndex And OpenAI- Indexing And Querying Multiple Pdf's
Machine Learning In 6 Hours: • Complete Machine Learn...
Deep Learning 5 hours : • Deep Learning Indepth ...
►Learn In a Week Playlist
Statistics: • Live Day 1- Introducti...
Machine Learning : • Announcing 7 Days Live...
Deep Learning: • 5 Days Live Deep Learn...
NLP : • Announcing NLP Live co...
---------------------------------------------------------------------------------------------------
My Recording Gear
Laptop: amzn.to/4886inY
Office Desk : amzn.to/48nAWcO
Camera: amzn.to/3vcEIHS
Writing Pad: amzn.to/3vcEIHS
Monitor: amzn.to/3vcEIHS
Audio Accessories: amzn.to/48nbgxD
Audio Mic: amzn.to/48nbgxD
Thank you so much for this video!!! Very helpful!!
I've managed to get a bit of understanding of LLM and to do my lab task!!!
It would be amazing if you show us how to put all of this into production through an API! Thanks for your wonderful work! You Rock!!
vLLM is easy enough to use, they provide a docker image, run that bad boy with whatever model you want and use their openAI endpoint then you can just use that in langchain or Llama index
Much awaited series, thank you krish Sir♥
Thanks you krish for all your hard work. keep making such amazing videos.
Indeed great video. Please try to include the reason for using different approaches of doing any process like in earlier videos the model was loaded differently and here differently. A simple explanation of the reason behind using a specific approach can be useful for the beginners. As the approach changes when we move forward in more complex applications and different use cases.
thank you the cours i see in this rag topic , very good , i 'm french speaker but i understand your cours
Thank you, Sir, you are amazing!
U r really a good instructor. ❤
Thankyou so much Krish, it was worth the wait :)
Thanks man, it works well. (After correcting some changes made in llama and huggingface)
I am facing problem while importing huggingfacellm.
can you tell me what you did?
@@ahmadmasood3939 while accessing tokens, in the create new token option, chnage token mode to write
hi, btw im also facing the same issues while importing llama and huggingface . Can u help me to fix the issues ?
Love it
You are amazing.
Please make a video on RAG with CSV or database with local open source llm and with memory
Love this. When is the next instalment?
Sir Instead of using LLamA2 model from huggingface, Can we try the quantized llama model? If we use the quantized llama model will the output accuracy varies?
Hey Krish, amazing video again. Can you please help to create a similar solution for custom text 2 sql
sir greet method to teach us as well as gaining good grip on english. please make video about token used in llama 2 model used
Hello Krish
The list of accessories you have mentioned, is not having right links for Amazon. Can you please give me the link for the writing pad which you use ?
gr8 video
Lets Goooooo
Thanks for the detailed run through, very useful . One question I have is on the two PDF documents you have. Are these available anywhere, just for testing purposes?
i have some doubts in implementing the code.,can u help?
Yeah ask me?
Hi krish sir! they have updated the service_context to settings.llm and i am not able to understand how to implement it into this code.
Please help!
Thanks!
Is it possible to use local llms instead huggingface directly? I have ollama nd lmstudio installed.
Sir everything fine, but you have missed just one thing in this project, that why you have implemented or build prompt and why you have not used that prompt, why you have gone with default prompt ? And remaining part was so good with good explanation ❤️
More examples i will come up with...this basic to intermediate RAG system
Can I combine the LORA fint tuning for example and a RAG FOR THIS llm, can this work give me very interesting performances?
@krishnaik06 Sir Thank you for this knowledgeable video, My question is which Evalution model we should use to show the accuracy in terms of answer and context retrieval.
If possible, will you please create one video on evaluation method for RAG application.
@krishnaik06 sir I also want this thing, can you please create one video on evolution method. Thanks
@krishnaik06 yes please can you please create a video on evaluation method for RAG application
Hey Krish, why you left PWskills masters in data science course. I joined this because of you. But I am seeing there new tutors. I joined this course because of you only.
Sir..instead of RAG with pdf..make a video on Rag with videos.. which will process videos and be able to answer questions based on the video
Hi Krish , which vector store have you utilised here? there was no mention of such in the code
not saving the embeddings in any vector store directly using it
waiting for next video, please upload it ...
I am getting the error of VectorStorIndex from LLama_index
use- from llama_index.core import VectorStoreIndex,SimpleDirectoryReader,ServiceContext
Sir as I am a student..
I am not having gpu in my machine...
I am not able to do projects with this open source llms n also with open ai...
Can u please help us to solve the resource errors by using other models
Why did we use a separate Embedding model here while in the earlier video of this playlist we directly used VectorStoreIndex on the documents. So why did we follow different approaches while creating similar Applications? Is it because of the different Model or it is just a different approach and can be done either way ?
I had one doubt is Vector Store Index use any embedding model behind it for creating the Index or how it create the embedding the vector store index
@@nikhilanand9022 By default, VectorStoreIndex use OpenAI Embeddings if no other embeddings are explicitly specified.
I am getting Runtimeerror: CUDA Error.... While running
index= VectorStoreIndex.from_documents (docs, service_context=service_context)
Sir please provide a solution to run with cpu....
Y is he using google colab rather than vs code
And also please answer this question -> can we use vs code to run seamless m4t meta model
Is there a way to ensure that once PDFs are processed, they do not need to be reprocessed every time the script runs, but rather that a cached database can be stored? Does anyone have code for this?
"ServiceContext" is now deprecated and replaced with "Settings" object. You may need to update this tutorial. Also, at 16:19 why did you used two different embeddings? HuggingFaceEmbeddings and LangChainEmbedding? Isn't just any one is enough?
where to get the data from, like you have done in this case?
\
I have lots of pdf documents data and want to create a custom chatbot based on it. Then which one will be better: Langchain or Llamaindex?
Llamaindex for indexing purpose, and langchain used for response of query with Prompt by the langchain LLM, and used Gemini pro as a LLM model. Will be great combination of all these technologies ❤
@@RanjitSingh-rq1qx Thanks for suggestions. I am looking for Open Source. So while Indexing in the Llamaindex, it doesn't use OpenAI api or something right?
@@vivekshindeVivekShinde yes all are open source
Guys can you please share the implementation of this by mixing llama index, Langchain, Gemini pro....
Please ... It will be very helpful 😊😊
@@vivekshindeVivekShinde I think when we use the Vector Store index it use the openai embedding model api for creating the index can you please confirm once ?
Can we add in conversation history to this app?
can you make a video to create flowchart by LLM
Hi, can anyone tell me how to download these raw pdf’s for hands on please??
there is an issue downloading the model LLAMA2 / LLAMA3 - as it requires to be authenticated / accepted by their team. Even after getting it authenticated / accepted - when we try download the code by your pattern / huggingface_clt - it still says - access denied to the config file - and hence cannot download any model locally - so if you could practically share insights or build a new video that pertains to the latest situation, will be awesome
How to convert this into end to end?
where i will get this dataset
Library is pretty messed up, nothing is working everwhere there is a module import error and function is missing or deprecated. if you found this, don't worry guys we are on same page. Sir could you please check if there is a some issues or what they have done to library
If you are having moldule import error then use
llama_index.core
Instead of
llama_index
Old tutorial. Most of the imports no longer work due to deprications.
SimpleInputPrompt is broken. still shows the concept tho
How to reduce hallucination? if answer is not in context then it is hallucinating .
may be you can modify the prompt to answer for not in context situation
literally prompt the LLM if not in document tell the LLM to say it does not know.
If the content is present , still its hallucinating , there are certain advance RAG techniques like Window Sentence Retrieval and Auto Merging Retrieval that can help.It will help in improving the context. Just try read regarding it and implement. It will help u. Also tuning agent to not give answer when it doesn't know helps in case of unknown scenario.
Sir , the llama index library is modifying everyday , and there are many import errors in the code , can you tell me the suitable version of llama-index to run the code
I was having the same issue, I tried installing version 0.9.40 and it's working now
@@ishratsyed2857 i tried to install llama_index version 0.9.40 but i got the following message:
ImportError Traceback (most recent call last)
in ()
----> 1 from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
2 from llama_index.llms import HuggingFaceLLM
3 from llama_index.prompts.prompts import SimpleInputPrompt
ImportError: cannot import name 'VectorStoreIndex' from 'llama_index' (unknown location)
do you know what i might be missing ?
@@sebastienmaillet9371 I have tried the same code from my local anaconda environment, I'm getting error with the importing packages but it is working fine when I use Google Colab, can anyone explain why?
Trying using the import like this instead:
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.core.prompts.prompts import SimpleInputPrompt
This should work for you!
i am getting an error while importing vectorstoreIndex from llamaIndex saying that llamaIndex doesn't exist. can someone help
Trying using the import like this instead:
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.core.prompts.prompts import SimpleInputPrompt
This should work for you!
@@ShreyasR-vr1es also
pip install llama-index-llms-huggingface
Hello Krish, I'm getting a basic error as below. Can you please help?
ImportError Traceback (most recent call last)
in ()
----> 1 from llama_index import VectorStoreIndex,SimpleDirectoryReader,ServiceContext
2 from llama_index.llms import HuggingFaceLLM
3 from llama_index.prompts.prompts import SimpleInputPrompt
ImportError: cannot import name 'VectorStoreIndex' from 'llama_index' (unknown location)
Trying using the import like this instead:
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.core.prompts.prompts import SimpleInputPrompt
This should work for you!
@@ShreyasR-vr1es Thank you, other 2 worked but I'm getting error in below
ModuleNotFoundError Traceback (most recent call last)
in ()
----> 1 from llama_index.llms.huggingface import HuggingFaceLLM
ModuleNotFoundError: No module named 'llama_index.llms.huggingface'
Please ignore, it worked, had some installation issues. Thank you so much.
@@sumanmaity3162 I am facing same error, how did it get resolved for you?
@@darshitshah8668 Please reinstall, it should work
Hey Krish. Video is cool. But can you tell us how we will know what are the different things we will need to import . You may have done lot of research. Kindly point us to the source of truth.
I have a very big question
Han please ask
Lol the suspense is killing me
any updates ?
i need those pdfs
while running
import torch
llm = HuggingFaceLLM(
context_window=4096,
max_new_tokens=256,
generate_kwargs={"temperature": 0.0, "do_sample": False},
system_prompt=system_prompt,
query_wrapper_prompt=query_wrapper_prompt,
tokenizer_name="meta-llama/Llama-2-7b-chat-hf",
model_name="meta-llama/Llama-2-7b-chat-hf",
device_map="auto",
# uncomment this if using CUDA to reduce memory usage
model_kwargs={"torch_dtype": torch.float16 , "load_in_8bit":True}
)
in colob i m getting import error stating that i need to install accelarate but i already have in in my env
The libraries changed:
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.core.prompts.prompts import SimpleInputPrompt
from langchain.embeddings.huggingface import HuggingFaceEmbeddings
from langchain.embeddings import HuggingFaceEmbeddings
from llama_index.embeddings.langchain import LangchainEmbedding
import llama_index
bro this line is giving error
from llama_index.embeddings.langchain import LangchainEmbedding
how do i find correct one
how to make this thing running
from langchain.embeddings.huggingface import HuggingFaceEmbeddings
from langchain.embeddings import HuggingFaceEmbeddings
from llama_index.embeddings import LangchainEmbedding
embed_model=LangchainEmbedding(
HuggingFaceEmbeddings(model_name="sentence-transformers/all-mpnet-base-v2"))
while running
import torch
llm = HuggingFaceLLM(
context_window=4096,
max_new_tokens=256,
generate_kwargs={"temperature": 0.0, "do_sample": False},
system_prompt=system_prompt,
query_wrapper_prompt=query_wrapper_prompt,
tokenizer_name="meta-llama/Llama-2-7b-chat-hf",
model_name="meta-llama/Llama-2-7b-chat-hf",
device_map="auto",
# uncomment this if using CUDA to reduce memory usage
model_kwargs={"torch_dtype": torch.float16 , "load_in_8bit":True}
)
in colob i m getting import error stating that i need to install accelarate but i already have in in my env
I had one doubt is Vector Store Index use any embedding model behind it for creating the Index or how it create the embedding the vector store index
from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
ImportError: cannot import name 'VectorStoreIndex' from 'llama_index' (unknown location)
Simple directory reader too
Trying using the import like this instead:
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.core.prompts.prompts import SimpleInputPrompt
This should work for you!
facing issue index=VectorStoreIndex.from_documents(documents,service_context=service_context) can anyone help
06:00
!pip install -q llama-index llama-index-llms-huggingface
from llama_index.core import VectorStoreIndex,SimpleDirectoryReader,ServiceContext
from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.core.prompts.prompts import SimpleInputPrompt
thank you so much