Amazing video! Only one thing that would be could to have is a little examplation on whether this is something from Qdrant, or if other vector databases have as well. On top of that, talking a bit about whether it performs some sort of linear search on the metadata, or if metadata is somehow indexed, would be great! Anyway, great video. I really appreciate that there is this content being straightforward and full of information :))
The simplest way to do the filtering is (as you say) either pre or post filtering after using the vector indexes. AFAIK Qdrant isn't doing this, but instead are using a filtrable variation of the HNSW algorithm - qdrant.tech/articles/filtrable-hnsw/ I haven't used every single vector db out there, but Qdrant seems to have better support for metadata filtering than say Chroma at the moment. The main thing missing from Chroma at the time I made that video was support for arrays as metadata. Good idea on general explanation though. Let me take that into account for future videos.
Hi Mark, I was impressed by your notebook outputs, nicely coloured and formatted. I tried to obtain the same result in vscode, but did not manage so far 🤔 Any tips?
For that case some of the libraries have functionality that lets you extract metadata from the prompt e.g. this is a guide showing how to do it with Haystack - haystack.deepset.ai/blog/extracting-metadata-filter
Amazing video! Only one thing that would be could to have is a little examplation on whether this is something from Qdrant, or if other vector databases have as well. On top of that, talking a bit about whether it performs some sort of linear search on the metadata, or if metadata is somehow indexed, would be great!
Anyway, great video. I really appreciate that there is this content being straightforward and full of information :))
The simplest way to do the filtering is (as you say) either pre or post filtering after using the vector indexes. AFAIK Qdrant isn't doing this, but instead are using a filtrable variation of the HNSW algorithm - qdrant.tech/articles/filtrable-hnsw/
I haven't used every single vector db out there, but Qdrant seems to have better support for metadata filtering than say Chroma at the moment. The main thing missing from Chroma at the time I made that video was support for arrays as metadata.
Good idea on general explanation though. Let me take that into account for future videos.
Hi Mark, I was impressed by your notebook outputs, nicely coloured and formatted. I tried to obtain the same result in vscode, but did not manage so far 🤔 Any tips?
I'm running the notebook in the web browser in dark mode rather than VSCode.
what if the query is about a metadata? Like Emma Saunders is in the query and no filter specified?
For that case some of the libraries have functionality that lets you extract metadata from the prompt e.g. this is a guide showing how to do it with Haystack - haystack.deepset.ai/blog/extracting-metadata-filter
Thanks for sharing, what machine spec are you running to use Mixtral? I assume 16GB is not enough🥲
I run mixtral 5 bit quantization on CPU with 64GB of RAM. It can generate tokens at a reasonable speed.
I have 64 GB RAM but even with that much it's noticeably slower than the smaller models!