28 June, 2024

✨Building an Engaging Chat Program with Azure OpenAI and .NET Semantic Kernel 🧿

In the ever-evolving landscape of artificial intelligence and cloud computing, creating interactive and intelligent applications has become more accessible and powerful. Today, we’ll dive into the fascinating world of integrating Azure OpenAI with .NET Semantic Kernel to build a simple yet engaging chat program. This blog post will walk you through the key components of the application, highlighting the elegance and efficiency of the integration.


 

Setting the Stage: Loading Configuration

The first step in our journey is setting up the environment configuration. Using the DotNetEnv library, we can effortlessly load environment variables from a .env file. This approach keeps our sensitive information secure and makes the configuration process seamless.

using DotNetEnv;

Env.Load();

This line of code ensures that our application can access the necessary Azure OpenAI credentials and other configuration details stored in the .env file.

Initializing the Kernel: The Common Class

The heart of our chat program lies in the initialization of the Semantic Kernel. The Common class is responsible for setting up the kernel using the credentials and model details from the environment variables.

using Microsoft.SemanticKernel;

namespace AzureSemanticKernel.AI
{
public static class Common
{
private static string key = Environment.GetEnvironmentVariable("AZURE_OPENAI_KEY");
private static string model = Environment.GetEnvironmentVariable("AZURE_OPENAI_MODEL");
private static string endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT");

public static Kernel initializeKernel()
{
var kernelBuilder = Kernel.CreateBuilder();
kernelBuilder.Services.AddAzureOpenAIChatCompletion(model, endpoint, key);
var kernel = kernelBuilder.Build();

return kernel;
}
}
}

In this class, we retrieve the API key, model, and endpoint from the environment variables and use them to configure the kernel. The initializeKernel method creates and returns a fully initialized kernel ready to handle chat requests.

Bringing It All Together: The Main Controller

Now that we have our configuration and kernel setup, let’s look at how we can create a simple yet powerful chat controller. The SimpleController class is where the magic happens. It receives user messages, processes them using the Semantic Kernel, and returns the responses.

using Microsoft.AspNetCore.Mvc;
using Microsoft.SemanticKernel;

namespace AzureSemanticKernel.AI.Controllers
{
public class SimpleController : Controller
{
[HttpPost]
public async Task<string> GetChatKernelResponse(string userMessage)
{
try
{
var kernel = Common.initializeKernel();

var result = await kernel.InvokePromptAsync(userMessage);

return result.ToString();
}
catch (Exception ex)
{
return ex.Message;
}
}
}
}

The GetChatKernelResponse action handles POST requests with user messages. It initializes the kernel, invokes the chat completion prompt, and returns the response. Error handling ensures that any exceptions are caught and their messages are returned to the user.

 
 
 

Conclusion

By leveraging Azure OpenAI and .NET Semantic Kernel, we’ve created a simple yet robust chat program with just few lines of code. This integration demonstrates the power and flexibility of modern AI and cloud technologies. Whether you’re a seasoned developer or just starting, this project showcases how easily you can build engaging and intelligent applications.

As AI continues to evolve, the possibilities for creating interactive and responsive applications are endless. Dive into the code, experiment with different models and prompts, and unleash the full potential of Azure OpenAI and .NET Semantic Kernel in your projects. Happy coding!

 

24 June, 2024

🎉 Automating Disaster Recovery for Azure Service Bus: A Seamless Solution ✨

Disaster recovery is a critical component of any robust IT infrastructure. For Azure Service Bus, a highly reliable cloud messaging service, automating the failover process is crucial for minimizing downtime and ensuring business continuity. While the Azure portal provides an easy way to initiate a failover with a click of a button, relying on manual intervention is not ideal for enterprise-grade solutions. Automation is the key.


In this blog post, we’ll explore an automated approach to handling disaster recovery for Azure Service Bus using a PowerShell script. This script seamlessly initiates the failover process and manages the underlying tasks, making it easier for infrastructure architects to ensure continuity without manual intervention.

Why Automate Service Bus Failover?

Azure Service Bus offers geo-disaster recovery capabilities that ensure high availability and data protection. However, initiating a failover manually can be risky and not recommended in production scenarios. Automation provides several benefits:

  1. Consistency: Automated scripts ensure that the failover process is executed the same way every time, reducing the risk of human error.
  2. Speed: Automation can trigger failover processes instantly, minimizing downtime.
  3. Efficiency: Automated processes can handle multiple tasks simultaneously, such as reconfiguring namespaces and cleaning up resources.

Understanding the Burn Down of the Primary Namespace

One critical aspect of Azure Service Bus failover is that after the failover, the primary namespace is essentially “burned down.” This means that the primary namespace becomes inactive, and all its entities, such as queues and topics, need to be cleaned up. This cleanup process ensures that the primary namespace is ready to be reconfigured or decommissioned.

The PowerShell Script

Let’s dive into the PowerShell script that automates the Azure Service Bus failover process. This script ensures that the primary and secondary namespaces are provisioned and manages the failover with minimal manual intervention.

Step 1: Parameter Initialization

First, we set the necessary parameters, such as subscription ID, resource group name, primary and secondary namespaces, and the alias name.

$connection = Connect-AzAccount -ErrorAction Stop
Write-Host "Connected to Azure successfully." -ForegroundColor Yellow

#************** Parameters ********************************************************************************************************************
$subscriptionId = ""
$resourceGroupName = ""
$sbusPrimaryNamespace = ""
$sbusSecondaryNamespace = ""
$sbusAliasName = ""
$partnerId = "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.ServiceBus/ ` namespaces/$sbusPrimaryNamespace"
#***********************************************************************************************************************************************

Step 2: Provisioning Functions

We define functions to ensure that both namespaces are fully provisioned before and after the failover. These functions poll the provisioning state and wait until it reaches the ‘Succeeded’ state for both normal and Geo-provisioned.

function Wait-ForNamespaceProvisioning {
param (
[string]$resourceGroupName,
[string]$namespaceName
)

$maxRetries = 30
$retryCount = 0
$delay = 60 # Delay in seconds between retries

do {
$namespace = Get-AzServiceBusNamespace -ResourceGroupName $resourceGroupName -NamespaceName $namespaceName
if ($namespace.ProvisioningState -eq "Succeeded") {
Write-Output "Namespace $namespaceName is provisioned."
return
}

Write-Output "Namespace $namespaceName is still in provisioning state: $($namespace.ProvisioningState)."
Start-Sleep -Seconds $delay
$retryCount++

} while ($namespace.ProvisioningState -ne "Succeeded" -and $retryCount -lt $maxRetries)

if ($namespace.ProvisioningState -ne "Succeeded") {
throw "Namespace $namespaceName did not reach 'Succeeded' state within the allotted time."
}
}

function Wait-ForNamespaceGeoProvisioning {
param (
[string]$resourceGroupName,
[string]$namespaceName
)

$maxRetries = 30
$retryCount = 0
$delay = 60 # Delay in seconds between retries

do {
$namespace = Get-AzServiceBusGeoDRConfiguration -ResourceGroupName $resourceGroupName ` -NamespaceName $namespaceName
        if ($namespace.ProvisioningState -eq "Succeeded") {
Write-Output "Namespace $namespaceName is geo provisioned."
return
}

if($null -eq $namespace){
Write-Output "Namespace $namespaceName is geo provisioned."
return
}

Write-Output "Namespace $namespaceName is still in geo provisioning state: $($namespace.ProvisioningState)."
Start-Sleep -Seconds $delay
$retryCount++

} while ($namespace.ProvisioningState -ne "Succeeded" -and $retryCount -lt $maxRetries)

if ($namespace.ProvisioningState -ne "Succeeded") {
throw "Namespace $namespaceName did not reach 'Succeeded' state within the allotted time."
}
}

Step 3: Combined Provisioning Function

This function ensures that both the primary and secondary namespaces are provisioned.

function Wait-ForNamespaceAndGeoProvisining {
param (
[string]$resourceGroupName,
[string]$PrimaryNamespace,
[string]$SecondaryNamespace
)

Wait-ForNamespaceProvisioning -resourceGroupName $resourceGroupName -namespaceName $PrimaryNamespace
Wait-ForNamespaceProvisioning -resourceGroupName $resourceGroupName -namespaceName $SecondaryNamespace

Wait-ForNamespaceGeoProvisioning -resourceGroupName $resourceGroupName -namespaceName $PrimaryNamespace
Wait-ForNamespaceGeoProvisioning -resourceGroupName $resourceGroupName -namespaceName $SecondaryNamespace

return
}

Step 4: Initiate Failover

This section of the script handles the failover process itself, ensuring both namespaces are ready, initiating the failover, and then performing post-failover cleanup and reconfiguration.

#************** Initiate failover **************************************

Wait-ForNamespaceAndGeoProvisining -resourceGroupName $resourceGroupName -PrimaryNamespace $sbusPrimaryNamespace ` -SecondaryNamespace $sbusSecondaryNamespace
Write-Output "`nFailing Over : Azure Service Bus $sbusPrimaryNamespace ...`n"

Set-AzServiceBusGeoDRConfigurationFailOver `
-Name $sbusAliasName `
-ResourceGroupName $resourceGroupName `
-NamespaceName $sbusSecondaryNamespace `

Wait-ForNamespaceAndGeoProvisining -resourceGroupName $resourceGroupName -PrimaryNamespace $sbusPrimaryNamespace ` -SecondaryNamespace $sbusSecondaryNamespace

Step 5: Cleanup After Failover

Post-failover, we delete all queues in the primary namespace to clean up resources. This is crucial since the primary namespace will be “burned down” after the failover.

Write-Output "`nDeleting all queues in the primary $sbusPrimaryNamespace ..."

$queues = Get-AzServiceBusQueue -ResourceGroupName $resourceGroupName -NamespaceName $sbusPrimaryNamespace
foreach ($queue in $queues) {
Remove-AzServiceBusQueue `
-ResourceGroupName $resourceGroupName `
-NamespaceName $sbusPrimaryNamespace `
-QueueName $queue.Name

Write-Host "Deleted queue: $($queue.Name)"
}

Wait-ForNamespaceAndGeoProvisining -resourceGroupName $resourceGroupName -PrimaryNamespace $sbusPrimaryNamespace ` -SecondaryNamespace $sbusSecondaryNamespace

Step 6: Reconfiguration

After cleaning up the primary namespace, the script sets the alias back to the secondary namespace.

Write-Output "`nSetting the alias back after failover ..."

New-AzServiceBusGeoDRConfiguration `
-Name $sbusAliasName `
-NamespaceName $sbusSecondaryNamespace `
-ResourceGroupName $resourceGroupName `
-PartnerNamespace $partnerId

Wait-ForNamespaceAndGeoProvisining -resourceGroupName $resourceGroupName -PrimaryNamespace $sbusPrimaryNamespace ` -SecondaryNamespace $sbusSecondaryNamespace
Write-Output "`nService Bus failover process completed successfully !!"

#************** DONE ************************************************************

Disconnect-AzAccount

Full codebase:

https://github.com/AtanuGpt/AzureServiceBusDR/blob/main/failover.ps1

Explanation of the Failover Process

  1. Initial Provisioning Check: The script first ensures that both the primary and secondary namespaces are fully provisioned before initiating the failover.
  2. Initiate Failover: The failover process is initiated using Set-AzServiceBusGeoDRConfigurationFailOver, which switches the alias to the secondary namespace.
  3. Cleanup Primary Namespace: After the failover, the script deletes all queues in the primary namespace. This step is critical because the primary namespace is effectively “burned down” or rendered inactive and cleaned up. This involves removing all entities (queues, topics, etc.) in the primary namespace.
  4. Reconfiguration: The alias is reconfigured to point to the secondary namespace, ensuring continued operation.
  5. Final Check: The script performs a final check to ensure both namespaces are in the correct state post-failover.

Conclusion

Automating the disaster recovery process for Azure Service Bus is not just a convenience — it’s a necessity for maintaining high availability and ensuring business continuity. This PowerShell script provides a comprehensive solution, making the failover process seamless and efficient. By adopting automation, you can ensure that your Azure Service Bus environment is always prepared for any eventuality, keeping your services running smoothly even in the face of disruptions.


18 June, 2024

🚀 Revolutionizing Document Interaction: An AI-Powered PDF Voice-2-Voice Chatbot Using LlamaIndex 🐑, Langchain 🔗 Azure AI Speech 🎤and Google Audio 🔊

Welcome to the third and last installment of this series on innovative AI-driven chatbots. In our journey from basic text-based interactions to text-2-voice-enabled capabilities, we now introduce a voice-2-voice-enabled PDF chatbot. This advanced system allows users to interact verbally with their PDF documents, significantly enhancing accessibility and usability. Let’s explore how this chatbot works and its implications for users.

Evolution of Chatbots: From Text to Voice

In our initial blog post, we introduced a text-based chatbot capable of processing queries from uploaded documents. This laid the groundwork for seamless interaction with textual information.

http://techiemate.blogspot.com/2024/06/revolutionizing-document-interaction-ai.html

Building on this, our second post showcased text to voice recognition integration, an interactive voice assistant capable of understanding and reading out the answer from the content in your PDFs. This enhancement marked a significant leap towards intuitive user engagement, catering to diverse user preferences and accessibility needs.

http://techiemate.blogspot.com/2024/06/revolutionizing-document-interaction-ai_13.html 

Introducing Voice-to-Voice Interaction with PDFs

Today, we introduce our latest innovation: a voice-enabled PDF chatbot capable of both transcribing spoken queries and delivering spoken responses directly from PDF documents. This breakthrough technology bridges traditional document interaction with modern voice-driven interfaces, offering a transformative user experience.

The Technical Backbone: Exploring the Codebase

Let’s delve into the technical components that power our voice-enabled PDF chatbot:

Setting Up Dependencies and Environment

import os
import faiss
import streamlit as st
from dotenv import load_dotenv
from langchain_core.messages import AIMessage, HumanMessage
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, StorageContext, load_index_from_storage
from llama_index.vector_stores.faiss import FaissVectorStore
import azure.cognitiveservices.speech as speechsdk
import speech_recognition as sr

# Initialize Faiss index for vectorization
d = 1536
faiss_index = faiss.IndexFlatL2(d)
PERSIST_DIR = "./storage"

# Load environment variables
load_dotenv()

The code snippet above sets up necessary dependencies:

  • Faiss: Utilized for efficient document vectorization, enabling similarity search based on content.
  • Streamlit: Facilitates the user interface for seamless interaction with the chatbot and document upload functionality.
  • LangChain: Powers the message handling and communication within the chatbot interface.
  • LlamaIndex: Manages the storage and retrieval of vectorized document data, optimizing query performance.
  • Azure Cognitive Services (Speech SDK): Provides capabilities for speech recognition and synthesis, enabling the chatbot to transcribe and respond to spoken queries.
  • Google Audio : Provides speech into text using Google’s speech recognition API.

Document Handling and Vectorization

def saveUploadedFiles(pdf_docs):
UPLOAD_DIR = 'uploaded_files'
try:
for pdf in pdf_docs:
file_path = os.path.join(UPLOAD_DIR, pdf.name)
with open(file_path, "wb") as f:
f.write(pdf.getbuffer())
return "Done"
except:
return "Error"

def doVectorization():
try:
vector_store = FaissVectorStore(faiss_index=faiss_index)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
documents = SimpleDirectoryReader("./uploaded_files").load_data()
index = VectorStoreIndex.from_documents(
documents,
storage_context=storage_context
)
index.storage_context.persist()
return "Done"
except:
return "Error"

The saveUploadedFiles function saves PDF documents uploaded by users to a designated directory (uploaded_files). The doVectorization function utilizes Faiss to vectorize these documents, making them searchable based on content similarities.

Speech Recognition and Transcription

def transcribe_audio():
recognizer = sr.Recognizer()
microphone = sr.Microphone()

with microphone as source:
recognizer.adjust_for_ambient_noise(source)
audio = recognizer.listen(source, timeout=20)

st.write("🔄 Transcribing...")

try:
text = recognizer.recognize_google(audio)
return text
except sr.RequestError:
return "API unavailable or unresponsive"
except sr.UnknownValueError:
return "Unable to recognize speech"

The transcribe_audio function uses the speech_recognition library to capture spoken queries from users via their microphone. It adjusts for ambient noise and listens for up to 20 seconds before transcribing the speech into text using Google's speech recognition API.

Querying and Fetching Data

def fetchData(user_question):
try:
vector_store = FaissVectorStore.from_persist_dir("./storage")
storage_context = StorageContext.from_defaults(
vector_store=vector_store, persist_dir=PERSIST_DIR
)
index = load_index_from_storage(storage_context=storage_context)
query_engine = index.as_query_engine()
response = query_engine.query(user_question)
return str(response)
except:
return "Error"

The fetchData function retrieves relevant information from vectorized documents based on user queries. It loads the persisted Faiss index from storage and queries it to find and return the most relevant information matching the user's question.

Defining the Welcome Message

The WelcomeMessage variable contains a multi-line string that introduces users to the voice-enabled PDF chatbot. It encourages them to upload PDF documents and start asking questions:

WelcomeMessage = """
Hello, I am your PDF voice chatbot. Please upload your PDF documents and start asking questions to me.
I would try my best to answer your questions from the documents.
"""

This message serves as the initial greeting when users interact with the chatbot, providing clear instructions on how to proceed.

Initializing Chat History and Azure Speech SDK Configuration

The code block initializes the chat history and sets up configurations for Azure Speech SDK:

if "chat_history" not in st.session_state:
st.session_state.chat_history = [
AIMessage(content=WelcomeMessage)
]

AZURE_SPEECH_KEY = os.getenv("AZURE_SPEECH_KEY")
AZURE_SPEECH_REGION = os.getenv("AZURE_SPEECH_REGION")
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")

speech_config = speechsdk.SpeechConfig(subscription=AZURE_SPEECH_KEY, region=AZURE_SPEECH_REGION)
speech_config.speech_synthesis_voice_name = "en-US-AriaNeural"
speech_config.speech_synthesis_language = "en-US"

speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=speech_config)
  • Chat History Initialization: Checks if chat_history exists in the Streamlit session state. If not, it initializes it with the WelcomeMessage wrapped in an AIMessage object. This ensures that the chat starts with the welcome message displayed to the user.
  • Azure Speech SDK Configuration: Retrieves Azure Speech API key and region from environment variables (AZURE_SPEECH_KEY and AZURE_SPEECH_REGION). It sets up a SpeechConfig object for speech synthesis (speech_synthesizer). The voice name and language are configured to use the "en-US-AriaNeural" voice for English (US).

Streamlit Integration: User Interface Design

def main():
load_dotenv()

st.set_page_config(
page_title="Chat with multiple PDFs",
page_icon=":sparkles:"
)

st.header("Chat with single or multiple PDFs :sparkles:")

for message in st.session_state.chat_history:
if isinstance(message, AIMessage):
with st.chat_message("AI"):
st.markdown(message.content)
elif isinstance(message, HumanMessage):
with st.chat_message("Human"):
st.markdown(message.content)

with st.sidebar:
st.subheader("Your documents")
pdf_docs = st.file_uploader(
"Upload your PDFs here and click on 'Process'",
accept_multiple_files=True
)

if st.button("Process"):
with st.spinner("Processing"):
IsFilesSaved = saveUploadedFiles(pdf_docs)
if IsFilesSaved == "Done":
IsVectorized = doVectorization()
if IsVectorized == "Done":
st.session_state.isPdfProcessed = "done"
st.success("Done!")
else:
st.error("Error! in vectorization")
else:
st.error("Error! in saving the files")

if st.button("Start Asking Question"):
st.write("🎤 Recording started...Ask your question")
transcription = transcribe_audio()
st.write("✅ Recording ended")

st.session_state.chat_history.append(HumanMessage(content=transcription))

with st.chat_message("Human"):
st.markdown(transcription)

with st.chat_message("AI"):
with st.spinner("Fetching data ..."):
response = fetchData(transcription)
st.markdown(response)

result = speech_synthesizer.speak_text_async(response).get()
st.session_state.chat_history.append(AIMessage(content=response))

if "WelcomeMessage" not in st.session_state:
st.session_state.WelcomeMessage = WelcomeMessage
result = speech_synthesizer.speak_text_async(WelcomeMessage).get()

#============================================================================================================
if __name__ == '__main__':
main()

User Experience: Seamless Interaction

Imagine uploading a collection of PDF documents — research papers, technical manuals, or reports — and simply speaking your questions aloud. The chatbot not only transcribes your speech but also responds audibly, providing immediate access to relevant information. This seamless interaction is particularly beneficial for users with visual impairments or those multitasking who prefer auditory information consumption.

Demo 💬🎤🤖


 

Enhancing Accessibility and Efficiency

Our voice-enabled PDF chatbot represents a significant advancement in accessibility technology. By integrating speech recognition, document vectorization, and AI-driven query processing, we empower users to effortlessly interact with complex information. This technology not only enhances accessibility but also boosts efficiency by streamlining the process of retrieving information from documents.

Conclusion: Paving the Way Forward

As we continue to explore the capabilities of AI in enhancing user experiences, the voice-enabled PDF chatbot stands as a testament to innovation in accessibility and usability. Whether you’re a researcher seeking insights from academic papers or a professional referencing technical documents, this technology promises to revolutionize how we interact with information.

Stay tuned as we push the boundaries further, exploring new applications and advancements in AI-driven technology. Stay connected and yes Happy Coding ! 😊

13 June, 2024

🚀 Revolutionizing Document Interaction: An AI-Powered PDF Text-2-Voice Chatbot Using LlamaIndex 🐑, Langchain 🔗 and Azure AI Speech 🔊

 In this blog post, we’re diving into the creation of an intelligent PDF voice chatbot using cutting-edge technologies like LangChain, LlamaIndex, and Azure AI Speech. This isn’t just another chatbot; it’s an interactive voice assistant capable of understanding and reading out the answer from the content in your PDFs. This project is a step up from my previous blog post where we explored building a text-based PDF chatbot without the voice functionality. If you missed that, be sure to check it out here

Technologies Used

  1. LangChain: For chaining language models and building complex applications.
  2. LlamaIndex: To index and query documents efficiently.
  3. Azure AI Speech: For speech synthesis, giving our bot a human-like voice.
  4. Streamlit: To create a user-friendly web interface.

Let’s jump into the code and see how these technologies come together to create our voice assistant chatbot.

Setting Up the Environment

First, ensure you have all the necessary libraries installed. You can do this by running:

pip install os faiss-cpu streamlit python-dotenv azure-cognitiveservices-speech langchain llama_index

Also, make sure you have your Azure Cognitive Services API keys ready.

The Code

Here’s the complete code for our voice chatbot:

import os
import faiss
import streamlit as st
from dotenv import load_dotenv
from langchain_core.messages import AIMessage, HumanMessage
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, StorageContext, load_index_from_storage
from llama_index.vector_stores.faiss import FaissVectorStore
import azure.cognitiveservices.speech as speechsdk

d = 1536
faiss_index = faiss.IndexFlatL2(d)
PERSIST_DIR = "./storage"

AZURE_SPEECH_KEY = os.getenv("AZURE_SPEECH_KEY")
AZURE_SPEECH_REGION = os.getenv("AZURE_SPEECH_REGION")
OPENAI_API_KEY=os.getenv("OPENAI_API_KEY")

def saveUploadedFiles(pdf_docs):
UPLOAD_DIR = 'uploaded_files'
try:
for pdf in pdf_docs:
file_path = os.path.join(UPLOAD_DIR, pdf.name)
with open(file_path, "wb") as f:
f.write(pdf.getbuffer())
return "Done"
except:
return "Error"

def doVectorization():
try:
vector_store = FaissVectorStore(faiss_index=faiss_index)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
documents = SimpleDirectoryReader("./uploaded_files").load_data()
index = VectorStoreIndex.from_documents(
documents,
storage_context=storage_context
)
index.storage_context.persist()
return "Done"
except:
return "Error"

def fetchData(user_question):
try:
vector_store = FaissVectorStore.from_persist_dir("./storage")
storage_context = StorageContext.from_defaults(
vector_store=vector_store, persist_dir=PERSIST_DIR
)
index = load_index_from_storage(storage_context=storage_context)
query_engine = index.as_query_engine()
response = query_engine.query(user_question)
return str(response)
except:
return "Error"

WelcomeMessage = """
Hello, I am your PDF voice chatbot. Please upload your PDF documents and start asking questions.
I will try my best to answer your questions based on the documents.
"""


if "chat_history" not in st.session_state:
st.session_state.chat_history = [
AIMessage(content=WelcomeMessage)
]

AZURE_SPEECH_KEY = os.getenv("AZURE_SPEECH_KEY")
AZURE_SPEECH_REGION = os.getenv("AZURE_SPEECH_REGION")
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")

speech_config = speechsdk.SpeechConfig(subscription=AZURE_SPEECH_KEY, region=AZURE_SPEECH_REGION)
speech_config.speech_synthesis_voice_name = "en-US-AriaNeural"
speech_config.speech_synthesis_language = "en-US"

speech_synthesizer = speechsdk.SpeechSynthesizer(speech_config=speech_config)

def main():
load_dotenv()

st.set_page_config(
page_title="Chat with multiple PDFs",
page_icon=":sparkles:"
)

st.header("Chat with single or multiple PDFs :sparkles:")

for message in st.session_state.chat_history:
if isinstance(message, AIMessage):
with st.chat_message("AI"):
st.markdown(message.content)
elif isinstance(message, HumanMessage):
with st.chat_message("Human"):
st.markdown(message.content)

with st.sidebar:
st.subheader("Your documents")
pdf_docs = st.file_uploader(
"Upload your PDFs here and click on 'Process'",
accept_multiple_files=True
)

if st.button("Process"):
with st.spinner("Processing"):
IsFilesSaved = saveUploadedFiles(pdf_docs)
if IsFilesSaved == "Done":
IsVectorized = doVectorization()
if IsVectorized == "Done":
st.session_state.isPdfProcessed = "done"
st.success("Done!")
else:
st.error("Error! in vectorization")
else:
st.error("Error! in saving the files")

user_question = st.chat_input("Ask a question about your document(s):")

if user_question is not None and user_question != "":
st.session_state.chat_history.append(HumanMessage(content=user_question))

with st.chat_message("Human"):
st.markdown(user_question)

with st.chat_message("AI"):
with st.spinner("Fetching data ..."):
response = fetchData(user_question)
st.markdown(response)

result = speech_synthesizer.speak_text_async(response).get()
st.session_state.chat_history.append(AIMessage(content=response))

if "WelcomeMessage" not in st.session_state:
st.session_state.WelcomeMessage = WelcomeMessage
result = speech_synthesizer.speak_text_async(WelcomeMessage).get()

if __name__ == '__main__':
main()

Breaking Down the Code

Environment Setup

We start by importing necessary libraries and loading environment variables using dotenv. The .env file :

OPENAI_API_KEY = ""
AZURE_OPENAI_API_KEY = ""
AZURE_SPEECH_KEY = ""
AZURE_SPEECH_REGION = ""

File Upload and Vectorization

The saveUploadedFiles function handles file uploads, saving PDFs to a directory. The doVectorization function uses Llama_Index and FAISS to create a searchable index from the uploaded documents.

Fetching Data

The fetchData function retrieves answers to user questions by querying the vector index.

Voice Synthesis

We use Azure AI Speech service to convert text responses into speech. This involves configuring the speechsdk.SpeechConfig and synthesizing speech with speechsdk.SpeechSynthesizer.

Streamlit Interface

Streamlit makes it easy to build a web interface. The sidebar allows users to upload and process PDFs. The main chat interface displays the conversation and handles user inputs.

Running the App

Run the Streamlit app using:

streamlit run your_script_name.py

Upload your PDF documents, ask questions, and listen as the bot responds with both text and voice.


 

Conclusion

By combining LangChain, LlamaIndex, and Azure AI, we’ve created an intelligent PDF voice chatbot that can make interacting with documents more engaging and accessible. Whether you’re using it for research, studying, or just exploring, this project showcases the potential of modern AI and NLP technologies.

Don’t forget to check out the previous blog post for a text-only version of this chatbot. Happy coding!

11 June, 2024

🚀 Revolutionizing Document Interaction: An AI-Powered PDF Chatbot Using LlamaIndex 🐑 and Langchain 🔗

 In today’s fast-paced digital world, efficiently accessing and extracting information from documents is more crucial than ever. What if you could chat with your PDFs, asking questions and getting precise answers instantly? Enter our innovative AI-powered PDF chatbot, combining the capabilities of LlamaIndex and Langchain. In this blog post, I’ll take you through the journey of building this exciting project and how it can transform your document interaction experience.


 

The Vision: Intelligent Document Interaction

Imagine you’re dealing with multiple PDF documents, trying to extract specific information quickly. Traditional methods can be time-consuming and tedious. Our AI chatbot changes the game by allowing you to interact with your documents through natural language queries, providing accurate answers instantly. This project leverages advanced AI technologies, including LlamaIndex for vector storage and Langchain for conversational AI, to make document interaction smarter and more intuitive.

The Technology Stack

To bring this vision to life, we used a robust technology stack:

  • LlamaIndex: For creating and managing a vector store to efficiently handle document data.
  • Langchain: For building the conversational AI interface.
  • FAISS (Facebook AI Similarity Search): For fast and scalable similarity search.
  • Streamlit: For creating an interactive and user-friendly web interface.
  • Python: The backbone of our project, tying everything together.

Let’s Dive into the Code

Here’s a detailed walkthrough of the code that powers our AI PDF chatbot.

Setting Up the Environment

First, we need to import the necessary libraries and set up the FAISS index and environment variables.

import os
import faiss
import streamlit as st
from dotenv import load_dotenv
from langchain_core.messages import AIMessage, HumanMessage
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, StorageContext, load_index_from_storage
from llama_index.vector_stores.faiss import FaissVectorStore

d = 1536
faiss_index = faiss.IndexFlatL2(d)
PERSIST_DIR = "./storage"

Handling File Uploads

We need a function to save uploaded PDF files. This function saves the files to a specified directory.

def saveUploadedFiles(pdf_docs):
UPLOAD_DIR = 'uploaded_files'
try:
for pdf in pdf_docs:
file_path = os.path.join(UPLOAD_DIR, pdf.name)
with open(file_path, "wb") as f:
f.write(pdf.getbuffer())
return "Done"
except:
return "Error"

Vectorizing Documents

The doVectorization function reads the uploaded PDFs, converts them into vectors, and stores them in the FAISS index.

def doVectorization():    
try:
vector_store = FaissVectorStore(faiss_index=faiss_index)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
documents = SimpleDirectoryReader("./uploaded_files").load_data()
index = VectorStoreIndex.from_documents(
documents,
storage_context=storage_context
)
index.storage_context.persist()
return "Done"
except:
return "Error"

Fetching Data Based on User Queries

The fetchData function processes user queries and retrieves relevant information from the vectorized documents.

def fetchData(user_question):
try:
vector_store = FaissVectorStore.from_persist_dir("./storage")
storage_context = StorageContext.from_defaults(
vector_store=vector_store, persist_dir=PERSIST_DIR
)
index = load_index_from_storage(storage_context=storage_context)
query_engine = index.as_query_engine()
response = query_engine.query(user_question)
return str(response)
except:
return "Error"

Building the Streamlit Interface

The main function sets up the Streamlit interface, handles file uploads, processes documents, and manages user interactions.

if "chat_history" not in st.session_state:
st.session_state.chat_history = [
AIMessage(content="Hello, I am a bot. How can I help you?")
]

def main():
load_dotenv()
os.environ["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY")

st.set_page_config(
page_title="Chat with multiple PDFs",
page_icon=":sparkles:"
)

st.header("Chat with single or multiple PDFs :sparkles:")

for message in st.session_state.chat_history:
if isinstance(message, AIMessage):
with st.chat_message("AI"):
st.markdown(message.content)
elif isinstance(message, HumanMessage):
with st.chat_message("Human"):
st.markdown(message.content)

with st.sidebar:
st.subheader("Your documents")
pdf_docs = st.file_uploader(
"Upload your PDFs here and click on 'Process'",
accept_multiple_files=True
)

if st.button("Process"):
with st.spinner("Processing"):
IsFilesSaved = saveUploadedFiles(pdf_docs)
if IsFilesSaved == "Done":
IsVectorized = doVectorization()
if IsVectorized == "Done":
st.session_state.isPdfProcessed = "done"
st.success("Done!")
else:
st.error("Error! in vectorization")
else:
st.error("Error! in saving the files")

if "isPdfProcessed" in st.session_state:
user_question = st.chat_input("Ask a question about your document(s):")

if user_question is not None and user_question != "":
st.session_state.chat_history.append(HumanMessage(content=user_question))

with st.chat_message("Human"):
st.markdown(user_question)

with st.chat_message("AI"):
with st.spinner("Fetching data ..."):
response = fetchData(user_question)
st.markdown(response)

st.session_state.chat_history.append(AIMessage(content=response))

if __name__ == '__main__':
main()

Bringing It All Together

With our AI PDF chatbot, you can upload multiple PDFs, process them, and interact with the content effortlessly. The chatbot provides precise answers to your queries, making document interaction more efficient and enjoyable. This project showcases the power of combining LlamaIndex and Langchain to create intelligent, user-friendly AI applications.

Join the AI revolution and transform how you interact with your documents. Happy coding! 🚀

06 June, 2024

Bringing AI to Your Fashion Shop: Meet ShopBot, Your Friendly Shopping Assistant! 🛒

Welcome to the future of online shopping! Imagine having a friendly assistant right at your fingertips, ready to help you navigate through a variety of stylish clothing, accessories, and more. Say hello to ShopBot, an AI-powered assistant for your Fashion Shop, designed to enhance your shopping experience and make it more enjoyable than ever.

In this blog post, we’ll try to simulate a shopping store and dive into the exciting world of AI in e-commerce, showcase how ShopBot works, and provide you with all the code snippet to get started with your very own AI shopping assistant.

Why AI in E-Commerce?

Artificial Intelligence has revolutionized many industries, and e-commerce is no exception. Here are a few reasons why integrating AI into your online store can be a game-changer:

  • Personalized Shopping Experience: AI can tailor recommendations based on customer preferences and browsing history.
  • 24/7 Customer Support: AI chatbots can assist customers at any time, providing instant responses and support.
  • Efficient Navigation: AI can help customers find products quickly and easily, enhancing their overall shopping experience.

Introducing ShopBot

ShopBot is an AI assistant designed specifically for aFashion Shop. It assists customers in browsing products, providing information, and guiding them through the checkout process. Let’s see how you can create your very own ShopBot using Python, Streamlit and Azure Open AI

Setting Up ShopBot

Here’s a step-by-step guide to creating ShopBot. The code below demonstrates how to use Azure OpenAI, Streamlit, and a predefined product list to build an engaging shopping assistant.

Step 1: Import Required Libraries

First, import the necessary libraries:

import os
from openai import AzureOpenAI
from dotenv import load_dotenv
from langchain_core.messages import AIMessage, HumanMessage
import streamlit as st

Step 2: Load Environment Variables

Load your environment variables to access your Azure OpenAI API keys:

load_dotenv()

Step 3: Define the Product List

Here’s a sample product list. Credit goes to Yennhi95zz’s medium blog post for this comprehensive list:

product_list = '''
# Fashion Shop Product List

## Women's Clothing:
- T-shirt
- Price: $20
- Available Sizes: Small, Medium, Large, XL
- Available Colors: Red, White, Black, Gray, Navy

- Elegant Evening Gown
- Price: $150
- Available Sizes: Small, Medium, Large, XL
- Available Colors: Black, Navy Blue, Burgundy

- Floral Summer Dress
- Price: $45
- Available Sizes: Small, Medium, Large
- Available Colors: Floral Print, Blue, Pink

- Professional Blazer
- Price: $80
- Available Sizes: Small, Medium, Large, XL
- Available Colors: Black, Gray, Navy

## Men's Clothing:
- Classic Suit Set
- Price: $200
- Available Sizes: Small, Medium, Large, XL
- Available Colors: Charcoal Gray, Navy Blue, Black

- Casual Denim Jeans
- Price: $35
- Available Sizes: 28, 30, 32, 34
- Available Colors: Blue Denim, Black

- Polo Shirt Collection
- Price: $25 each
- Available Sizes: Small, Medium, Large, XL
- Available Colors: White, Blue, Red, Green

## Accessories:
- Stylish Sunglasses
- Price: $20
- Available Colors: Black, Brown, Tortoise Shell

- Leather Handbag
- Price: $60
- Available Colors: Brown, Black, Red

- Classic Wristwatch
- Price: $50
- Available Colors: Silver, Gold, Rose Gold

## Footwear:
- High-Heel Ankle Boots
- Price: $70
- Available Sizes: 5-10
- Available Colors: Black, Tan, Burgundy

- Comfortable Sneakers
- Price: $55
- Available Sizes: 6-12
- Available Colors: White, Black, Gray

- Formal Leather Shoes
- Price: $90
- Available Sizes: 7-11
- Available Colors: Brown, Black

## Kids' Collection:
- Cute Cartoon T-shirts
- Price: $15 each
- Available Sizes: 2T, 3T, 4T
- Available Colors: Blue, Pink, Yellow

- Adorable Onesies
- Price: $25
- Available Sizes: Newborn, 3M, 6M, 12M
- Available Colors: Pastel Blue, Pink, Mint Green

- Trendy Kids' Backpacks
- Price: $30
- Available Colors: Blue, Red, Purple

## Activewear:
- Yoga Leggings
- Price: $30
- Available Sizes: Small, Medium, Large
- Available Colors: Black, Gray, Teal

- Running Shoes
- Price: $40
- Available Sizes: 6-12
- Available Colors: White, Black, Neon Green

- Quick-Dry Sports T-shirt
- Price: $20
- Available Sizes: Small, Medium, Large
- Available Colors: Red, Blue, Gray

'''

Step 4: Create the Context for ShopBot

Define the context for your AI assistant:

def get_context():
context = [{'role': 'system',
'content': f"""
You are ShopBot, an AI assistant for my online fashion shop - My Fashion Shop.
Your role is to assist customers in browsing products, providing information, and guiding them through the checkout process.
Be friendly and helpful in your interactions. We offer a variety of products across categories such as Women's Clothing,
Men's clothing, Accessories, Kids' Collection, Footwears and Activewear products.
Feel free to ask customers about their preferences, recommend products, and inform them about any ongoing promotions.
The Current Product List is limited as below:

```{product_list}```

Make the shopping experience enjoyable and encourage customers to reach out if they have any questions or need assistance.
"""
}]
return context

Step 5: Define the Function to Get AI Responses

Create a function to get responses from the AI:

def get_completion_from_messages(messages):
client = AzureOpenAI(
api_key = os.environ["AZURE_OPENAI_API_KEY"],
api_version = os.environ["API_VERSION"],
azure_endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]
)

chat_completion = client.chat.completions.create(
model = os.environ["DEPLOYMENT_MODEL"],
messages = messages
)

return chat_completion.choices[0].message.content

Step 6: Initialize Streamlit Session State

Initialize the chat history and context:

if "chat_history" not in st.session_state:
st.session_state.chat_history = [
AIMessage(content="Hello! I'm your Ecom Assistant. How can I help you today"),
]

if "chat_msg" not in st.session_state:
st.session_state.chat_msg = get_context()

Step 7: Set Up Streamlit UI

Configure Streamlit page and display the chat interface:

st.set_page_config(page_title="Chat with My Shop", page_icon=":speech_balloon:")

st.title("Ecommerce Chatbot :speech_balloon:")


for message in st.session_state.chat_history:
if isinstance(message, AIMessage):
with st.chat_message("AI"):
st.markdown(message.content)
elif isinstance(message, HumanMessage):
with st.chat_message("Human"):
st.markdown(message.content)

Step 8: Handle User Input

Capture and respond to user queries:

user_query = st.chat_input("Type a message...")
if user_query is not None and user_query.strip() != "":
st.session_state.chat_msg.append({"role" : "user", "content" : f"{user_query}"})
st.session_state.chat_history.append(HumanMessage(content=user_query))

with st.chat_message("Human"):
st.markdown(user_query)

with st.chat_message("AI"):
with st.spinner("Please wait ..."):
response = get_completion_from_messages(st.session_state.chat_msg)
st.markdown(response)

st.session_state.chat_msg.append({"role" : "user", "content" : f"{response}"})
st.session_state.chat_history.append(AIMessage(content=response))

Step 8: Run the application

Run the app:

streamlit run <<yourapp>>.py

Try It Yourself!

That’s it! You’ve now created your very own AI shopping assistant. With ShopBot, your customers can enjoy a personalized and efficient shopping experience, making them feel valued and well-assisted. Whether they need recommendations, product details, or help with the checkout process, ShopBot is here to help.

Feel free to customize the product list and enhance the functionality to suit your store’s unique needs. Happy coding and happy shopping!