Chat with your PDF - Gen AI App - With Amazon Bedrock, RAG, S3, Langchain and Streamlit [Hands-On]

In this video we will build a CHATBOT like application with AWS Amazon Bedrock, docker, python, Langchain, and Streamlit. We will use Retrieval-Augmented generation concept to provide context to the Large Language model along with user query to generate response from our Knowledgebase.
Time Stamps:
0:00 - Introduction
0:35 - Quick Demo
3:44 - Application Architecture and details
8:40 - Build Admin Web App
42:40 - Build Client Web App and wrap
This is a hands-on tutorial where I will demonstrate the following:
- Architecture of the applications
- Build 2 applications (ADMIN and USER) and create DOCKER images
- ADMIN Application:
- Build Admin Web application where AdminUser can upload the pdf.
- The PDF text is split into chunks
- Using the Amazon Titan Embedding Model, create the vector representation of the chunks
- Using FAISS, save the vector index locally
- Upload the index to Amazon S3 bucket (You can use other vector stores like OpenSearch, Pinecone, PgVector etc., but for this demo, I chose cost effective S3)
- USER Application:
- Build User Web application where users can query / chat with the pdf.
- At the application start, download the index files from S3 to build local FAISS index (vector store)
- Langchain's RetrievalQA, does the following:
- Convert the User's query to vector embedding using Amazon Titan Embedding Model (Make sure to use the same model that was used for creating the chunk's embedding on the Admin side)
- Do similarity search to the FAISS index and retrieve 5 relevant documents pertaining to the user query to build the context
- Using Prompt template, provide the question and context to the Large Language Model. We are using Claude model from Anthropic.
- Display the LLM's response to the user.
The source code is available in Github:
github.com/mycloudtutorials/g...
#llm #generativeai #aws #amazoncloud #bedrock #artificialintelligence

[@sebastianfuertesdev9768]

Thank you so much! it was an amazing tutorial

[@MyCloudTutorials]

I am glad you liked it.Thanks

[@bensdevjourney7081]

Would be nice to know, if we want to do something like this we first need to request access to the Titan Embeddings G1 - Text model. This was a great tutorial but it overshadowed setting up access Key/secret on AWS as well as requesting the model on AWS. Subscribing to hopefully see more guidance on how to set up things on the AWS side. Well done

[@norendermoody6509]

It will be helpful if you provide the AWS setup of Bedrock in a video

[@suhrobyusupov3520]

Hello sir, great tutorial, can we do the exact same thing with other type of data, not only pdf, sql tables for example

[@MyCloudTutorials]

Yes I don't think why not.
You have 2 options:
1) If you are already using PostGreSql, then you can use pgvector extension to make it a vector data store.
2) If you do not want to use that, or not using PostGreSql and still want to vectorize your data and store in a vector index, you can certainly do that.
Steps will be something like:
1. Let's assume you are using ElasticSearch as Vector store
2. Read from table(s) , sql query, get the data
3. Call an embedding model with the column(s) of data you need to generate the embedding for.
4. Create a JSON object with plain text and embedding data.
5. Make Api calls to Elastic search to index the data
On querying side:
1. Convert the query into embedding using the same model that you used to create the embedding while indexing
2. Run similarity search on Elastic Search.
I am going to make a tutorial on something similar soon. Stay tuned. I will add a comment here with the same when ready.
Thanks

[@tanviralamsyed9172]

Hey there, is it possible for the user to upload their own pdf and the ai model to become a chatbot

[@MyCloudTutorials]

Well, if you do not want to use RAG (like I showed in this video) then you will have to take a Foundation Model and train it with your data. So, yes, it's possible but it would require more resources, time etc (depending upon the base model you choose and your data).
For continuously moving data, RAG works out better as you don't have to constantly fine tune the models as and when data is available.
Another thing to think about is cost. Once you train and create a custom model, you have to host it somewhere (whether cloud or in data center) which will incur some cost.
I hope it helps.
Thanks

[@AashiAgarwal-eb2dx]

Hi I need to provide memory for remembering the context in the chatbot, could you please suggest me the way how I can implement to the above code.

[@MyCloudTutorials]

Langchain has Contextual memory. Please check this: js.langchain.com/v0.1/docs/modules/memory/
I will make a video soon, once ready, I will put a link in this comment.
Thanks

[@KeerthanaPriyaDevaraj]

Im getting an error when im trying to run the code.
ValueError: Error raised by inference endpoint: Unable to locate credentials
Traceback:
File "C:\Users\DELL\Desktop\bedrock-chat-with-pdf\venv_bedrock\Lib\site-packages\streamlit
untime\scriptrunner\script_runner.py", line 600, in _run_script
exec(code, module.__dict__)
File "C:\Users\DELL\Desktop\bedrock-chat-with-pdf\Admin\admin.py", line 88, in
main()
File "C:\Users\DELL\Desktop\bedrock-chat-with-pdf\Admin\admin.py", line 78, in main
result = create_vector_store(request_id, splitted_docs)
Can you help me with this?

[@MyCloudTutorials]

Hi @KeerthanaPriyaDevaraj
Do you have AWS Cli installed and setup with IAM credential that has access to Bedrock?
If not, Please follow AWS documentation for installing AWS Cli from here: docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
If you already have an IAM user created, create access_key and secret_access_key in the IAM console and download the credentials.
Use `aws configure` command to setup the CLI.
Also, make sure that you have the models access enabled in the Bedrock console.
I hope this helps.
Thanks

[@xoxies]

Is possible lambda aws ?

[@MyCloudTutorials]

Very much possible in Lambda. If you are using langchain with Lambda, you need to include that as dependency. I shall make a video on how to do it in lambda, no ETA as of now, but I will surely tag you when I post a video on that.
Thanks

[@xoxies]

​@@MyCloudTutorialsI'm trying, but I have an error: "Unable to import module" for "langchain community", I generated a layer in the lambda function, but it doesn't work, I'm waiting for your video, thanks