Hey! Cool video. I actually built a full local solution using Ollama, no need for LM Studio at all. Here's what I did: I created a proxy that translates between OpenAI API embeddings and Ollama's format, both ways. The cool thing is, it works flawlessly for both global and local queries. I'd be happy to share the script with you if you're interested!
It is essential to conduct a thorough preprocessing of the documents before entering them into the RAG. This involves extracting the text, tables, and images, and processing the latter through a vision module. Additionally, it is crucial to maintain content coherence by ensuring that references to tables and images are correctly preserved in the text. Only after this processing should the documents be entered into a LLM.
It’s very slow for me too, I use Mac M2 32GB In the video I had to cut that part, because it took 26 mins just for indexing and considering it’s small chunk of data
Good stuff. As expected, on a Mac M2, indexing and global queries are quite slow. Local queries are doable because it's usually just one LLM call after the similarity & graph search.
Nice and useful video, but still not getting one thing. You made this video around 3 weeks ago, but in april , ollama released some embedding models. Then how we are saying it is not having embedding compatibility?.
At 7:10 I believe the reason it's giving errors is the url in the settings file is missing the word embeddings at the end. It probably tested some different urls until it figured it out.
What a perfect video to wake up to after yesterday's video :) I'm starting to think that we're abusing graphRAG here, all of us. You see, and I may be wrong I'm still a n00b here, we are not using semantic chunking and also, for those of us with thousands of files, say transcripts, feeding graphRAG a summary and tags might be good enough for a recommendation engine and if the user wants to dive in, then you use rag but you create a rag for each main collection of documents. So the graph rag may be able to list say what cooking classes you can take much faster and then querying each class that is its own rag for details should be also much faster and overall cheaper? What do you think?
Would you recommend GraphRAG for structured data as well like Postgres or MySql? I am still stuck between LLM SQL agent vs Vectors. I did explored Vanna already and like it. Appreciate your thoughts on this.
how can we use graph rag on data that's in another language? i have hundreds of documents i want to put into a rag but i cant get a good result out of the rag. the only things that come out are generic and often even things that i didn't ask about. do i have to use a German llm and embeddings model and translate all the prompts into German or do i have to translate all the data into English and live with it being in English?
Great stuff, id be really interested to see csv rather than txt import as it looks like it might possibly give higgher reasoning by leveraging the structure. Ie better temporal reasoning etc... source_column: "author" # the column containing the source/author of the data text_column: "message" # the column containing the text of the data timestamp_column: "date(yyyyMMddHHmmss)" # optional, the column containing the timestamp of the
I want to understand, how can we use it in a actual application. I reality users will upload their documents anytime they want, If I run indexer for different documents seperately it creates a seperate timestamp based folders in output, now how will the graph rag work when we have multiple artifacts? Our do I have to run indexer on entire documents even if one new document is added? and how do we trigger it programitically
After a few tries, my conclusion on graphRAG was that it is buggy when running local, took too long to process - as such that it is not practical to run locally.
Do a video that finds the balance of speed (local takes forever) and cost by using DeepSeek-V2, which is only $0.14/million tokens input and $0.28/million tokens output. Once you get your results, I would say that at RAG = 2 the Deepseek would be at least equal to or slightly better than GPT 4o at =1. Then add an additional python step, prompt to take overall points and run limited amounts of tokens via GPT 4o or even Gemini Flash 1.5, which is quite good, for improvement to 1.6~1.7 level outputs at almost “free token costs”.
Good video. Please do work out the error you encountered. Do you have a GPU on your laptop. 20+ minutes makes this unusable for a company with 100s or Thousands of documents.
Something is off. It works with openapi, no problem... But local models and embedding models - no chance. Followed all instructions. EDIT: httpS made me problems for local model, amateur issue. Sorry. It works locally now. Thank you.
Can you please show a way to visualize the knowledge graph with an interactive UI?
Hey! Cool video. I actually built a full local solution using Ollama, no need for LM Studio at all. Here's what I did: I created a proxy that translates between OpenAI API embeddings and Ollama's format, both ways.
The cool thing is, it works flawlessly for both global and local queries. I'd be happy to share the script with you if you're interested!
Please.
Yes please
ollama embeddings proxy on git hub
Please.
ollama embeddings proxy on git hub
Great work as usual. Humble. Concise. Helpful. Perfect. 👌
It is essential to conduct a thorough preprocessing of the documents before entering them into the RAG. This involves extracting the text, tables, and images, and processing the latter through a vision module. Additionally, it is crucial to maintain content coherence by ensuring that references to tables and images are correctly preserved in the text. Only after this processing should the documents be entered into a LLM.
best GenAI CZcamsr, I mean it .
Looking forward to more on this, it is the most interesting cutting edge tech in AI and almost no one else on youtube is talking about it
Mervin, Hi from New Zealand, I see that took 20 minutes to index…. what are the specs of your machine?
Same question. My computer runs Gemma 2 quite slow and I prefer to use Llama3 or Phi. The results will be the same?
@@d.d.z. What's your pc spec?
@@dudicrous Intel core i5 8th gen 8gb RAM I have a HP pavilion 13-an0012la.
It’s very slow for me too, I use Mac M2 32GB
In the video I had to cut that part, because it took 26 mins just for indexing and considering it’s small chunk of data
This is not all feasible on my computer but I would love move graph rag videos aiming more at how we can get this technology production ready.
can you please show or explain how to get the visualization of the data ? looks verry good, and thanks for the tutorial
Good stuff. As expected, on a Mac M2, indexing and global queries are quite slow. Local queries are doable because it's usually just one LLM call after the similarity & graph search.
A start, great!
So compared to GPTs, his search generation effect will be better?
I was eagerly waiting for this, big thanks
Another great video about GraphRAG, good job.
Great tutorial! Thank you!
You are amazing 🎉
Nice and useful video, but still not getting one thing. You made this video around 3 weeks ago, but in april , ollama released some embedding models. Then how we are saying it is not having embedding compatibility?.
Hi, how do you fix the issues with running local search using command line?
quick question, I already have a folder of embeddings and chunks, can I just pass the documents and embeddings to GraphRAG ?
Thank you for this tutorial. Very useful..
At 7:10 I believe the reason it's giving errors is the url in the settings file is missing the word embeddings at the end. It probably tested some different urls until it figured it out.
This is what I was looking for
Thanks this really helped!
What a perfect video to wake up to after yesterday's video :) I'm starting to think that we're abusing graphRAG here, all of us. You see, and I may be wrong I'm still a n00b here, we are not using semantic chunking and also, for those of us with thousands of files, say transcripts, feeding graphRAG a summary and tags might be good enough for a recommendation engine and if the user wants to dive in, then you use rag but you create a rag for each main collection of documents. So the graph rag may be able to list say what cooking classes you can take much faster and then querying each class that is its own rag for details should be also much faster and overall cheaper? What do you think?
Basic RAG is fine for basic tasks but this GraphRAG is for advanced and more meaningful response.
What is the average query time that you were experiencing with the global/local search?
thanks. coming soon local vs. global
Can you create a video on how to use GraphRAG with the GROQ API? Looks like nobody has done it yet. Thank you.
Can't we use nomic-embed-text provided by ollama for the embeddings?
Thx for the prez. It is about graph so is it possible to get a grip on the underlying graph db and vizualise it using a ds tool ?
Great video! Can you export a CSV file to visualize the graph using an external tool like Gephi?
Why do you read the settings.yaml file by default when you create an index, but mine reads the .env file?
Would you recommend GraphRAG for structured data as well like Postgres or MySql? I am still stuck between LLM SQL agent vs Vectors. I did explored Vanna already and like it. Appreciate your thoughts on this.
Is LM Studio necessary? I believe OpenWebUI should also suffice, potentially creating a completely open-source graphrag solution.
how can we use graph rag on data that's in another language? i have hundreds of documents i want to put into a rag but i cant get a good result out of the rag. the only things that come out are generic and often even things that i didn't ask about. do i have to use a German llm and embeddings model and translate all the prompts into German or do i have to translate all the data into English and live with it being in English?
Thanks
Great stuff, id be really interested to see csv rather than txt import as it looks like it might possibly give higgher reasoning by leveraging the structure. Ie better temporal reasoning etc...
source_column: "author" # the column containing the source/author of the data text_column: "message" # the column containing the text of the data timestamp_column: "date(yyyyMMddHHmmss)" # optional, the column containing the timestamp of the
Didn’t understand the final sentence, running things in llmstudio , what about many pdf documents
I want to understand, how can we use it in a actual application. I reality users will upload their documents anytime they want, If I run indexer for different documents seperately it creates a seperate timestamp based folders in output, now how will the graph rag work when we have multiple artifacts? Our do I have to run indexer on entire documents even if one new document is added? and how do we trigger it programitically
After a few tries, my conclusion on graphRAG was that it is buggy when running local, took too long to process - as such that it is not practical to run locally.
is this completely free? or are there open api calls?
You need an OpenAI API Key to run it. It is unfortunatly not 100% local. Is there a way to run it without an OpenAI paid subscription?
yes you can, just export the "string or some text" for the OpenAI API Key. And then you can run it locally.
can you use NIM to keep your Data Private?
Do a video that finds the balance of speed (local takes forever) and cost by using DeepSeek-V2, which is only $0.14/million tokens input and $0.28/million tokens output.
Once you get your results, I would say that at RAG = 2 the Deepseek would be at least equal to or slightly better than GPT 4o at =1.
Then add an additional python step, prompt to take overall points and run limited amounts of tokens via GPT 4o or even Gemini Flash 1.5, which is quite good, for improvement to 1.6~1.7 level outputs at almost “free token costs”.
Good video. Please do work out the error you encountered. Do you have a GPU on your laptop. 20+ minutes makes this unusable for a company with 100s or Thousands of documents.
can i use gemini model with graph rag ??
Yes, but not straight out of the box as I can see. You might need to modify the code slightly
@@MervinPraison can you make a video on this please.
Something is off. It works with openapi, no problem... But local models and embedding models - no chance. Followed all instructions.
EDIT: httpS made me problems for local model, amateur issue. Sorry. It works locally now. Thank you.
The problem is LM studio getting error
Are you serious? It's not working at all.
as long as ms does not add other formats besides txt and csv, this graphrag is useless to me and all the business use cases i know and have.
Why? Can't you transform the data?
@@eggmaster88 can you? including images, diagrams, tables, metadata like page numbers, … please tell me, if there is a good solution
@@iham1313 for tables you can use .csv from what i know but it seems that you don't really need graph database for your data, more like sql.
@@iham1313 look up LangChain GraphRag vids on the topic - they show how to do images diagrams tables and metadata.
GRAPHRAG_API_KEY= "ollama"？big thanks
I got it to run without adding any key
@@matthewsoltis3383 how please?