In today’s fast-paced digital world, efficiently accessing and extracting information from documents is more crucial than ever. What if you could chat with your PDFs, asking questions and getting precise answers instantly? Enter our innovative AI-powered PDF chatbot, combining the capabilities of LlamaIndex and Langchain. In this blog post, I’ll take you through the journey of building this exciting project and how it can transform your document interaction experience.
The Vision: Intelligent Document Interaction
Imagine you’re dealing with multiple PDF documents, trying to extract specific information quickly. Traditional methods can be time-consuming and tedious. Our AI chatbot changes the game by allowing you to interact with your documents through natural language queries, providing accurate answers instantly. This project leverages advanced AI technologies, including LlamaIndex for vector storage and Langchain for conversational AI, to make document interaction smarter and more intuitive.
The Technology Stack
To bring this vision to life, we used a robust technology stack:
- LlamaIndex: For creating and managing a vector store to efficiently handle document data.
- Langchain: For building the conversational AI interface.
- FAISS (Facebook AI Similarity Search): For fast and scalable similarity search.
- Streamlit: For creating an interactive and user-friendly web interface.
- Python: The backbone of our project, tying everything together.
Let’s Dive into the Code
Here’s a detailed walkthrough of the code that powers our AI PDF chatbot.
Setting Up the Environment
First, we need to import the necessary libraries and set up the FAISS index and environment variables.
import os
import faiss
import streamlit as st
from dotenv import load_dotenv
from langchain_core.messages import AIMessage, HumanMessage
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, StorageContext, load_index_from_storage
from llama_index.vector_stores.faiss import FaissVectorStore
d = 1536
faiss_index = faiss.IndexFlatL2(d)
PERSIST_DIR = "./storage"Handling File Uploads
We need a function to save uploaded PDF files. This function saves the files to a specified directory.
def saveUploadedFiles(pdf_docs):
UPLOAD_DIR = 'uploaded_files'
try:
for pdf in pdf_docs:
file_path = os.path.join(UPLOAD_DIR, pdf.name)
with open(file_path, "wb") as f:
f.write(pdf.getbuffer())
return "Done"
except:
return "Error"Vectorizing Documents
The doVectorization function reads the uploaded PDFs, converts them into vectors, and stores them in the FAISS index.
def doVectorization():
try:
vector_store = FaissVectorStore(faiss_index=faiss_index)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
documents = SimpleDirectoryReader("./uploaded_files").load_data()
index = VectorStoreIndex.from_documents(
documents,
storage_context=storage_context
)
index.storage_context.persist()
return "Done"
except:
return "Error"Fetching Data Based on User Queries
The fetchData function processes user queries and retrieves relevant information from the vectorized documents.
def fetchData(user_question):
try:
vector_store = FaissVectorStore.from_persist_dir("./storage")
storage_context = StorageContext.from_defaults(
vector_store=vector_store, persist_dir=PERSIST_DIR
)
index = load_index_from_storage(storage_context=storage_context)
query_engine = index.as_query_engine()
response = query_engine.query(user_question)
return str(response)
except:
return "Error"Building the Streamlit Interface
The main function sets up the Streamlit interface, handles file uploads, processes documents, and manages user interactions.
if "chat_history" not in st.session_state:
st.session_state.chat_history = [
AIMessage(content="Hello, I am a bot. How can I help you?")
]
def main():
load_dotenv()
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")
st.set_page_config(
page_title="Chat with multiple PDFs",
page_icon=":sparkles:"
)
st.header("Chat with single or multiple PDFs :sparkles:")
for message in st.session_state.chat_history:
if isinstance(message, AIMessage):
with st.chat_message("AI"):
st.markdown(message.content)
elif isinstance(message, HumanMessage):
with st.chat_message("Human"):
st.markdown(message.content)
with st.sidebar:
st.subheader("Your documents")
pdf_docs = st.file_uploader(
"Upload your PDFs here and click on 'Process'",
accept_multiple_files=True
)
if st.button("Process"):
with st.spinner("Processing"):
IsFilesSaved = saveUploadedFiles(pdf_docs)
if IsFilesSaved == "Done":
IsVectorized = doVectorization()
if IsVectorized == "Done":
st.session_state.isPdfProcessed = "done"
st.success("Done!")
else:
st.error("Error! in vectorization")
else:
st.error("Error! in saving the files")
if "isPdfProcessed" in st.session_state:
user_question = st.chat_input("Ask a question about your document(s):")
if user_question is not None and user_question != "":
st.session_state.chat_history.append(HumanMessage(content=user_question))
with st.chat_message("Human"):
st.markdown(user_question)
with st.chat_message("AI"):
with st.spinner("Fetching data ..."):
response = fetchData(user_question)
st.markdown(response)
st.session_state.chat_history.append(AIMessage(content=response))
if __name__ == '__main__':
main()Bringing It All Together
With
our AI PDF chatbot, you can upload multiple PDFs, process them, and
interact with the content effortlessly. The chatbot provides precise
answers to your queries, making document interaction more efficient and
enjoyable. This project showcases the power of combining LlamaIndex and
Langchain to create intelligent, user-friendly AI applications.
Join the AI revolution and transform how you interact with your documents. Happy coding! 🚀
No comments:
Post a Comment