Build a GenAI Chatbot to Search Internal Docs with Snowflake Cortex + Streamlit + Azure WebApp

In this blog, you’ll learn how to build and deploy a GenAI chatbot that enables your internal users to query company documents like PDFs, SOPs, and manuals using natural language.

• Built with Snowflake Cortex LLM for embedding and completion

• UI built in Python using Streamlit

• Deployed as an Azure Web App with enterprise-grade SSO (Azure AD)

• Credentials managed in Azure Key Vault

• Connected to Snowflake using JWT authentication (no username/password)

🔍 Why This Matters

Internal knowledge is often locked away in:

• SharePoint folders

• Scattered PDFs and Word docs

• Static SOPs and manuals

This makes it hard for teams to access timely information, especially when they don't know where to look. A GenAI chatbot solves this by allowing users to just ask a question and get relevant answers instantly.

Use cases:

• Operations teams querying SOPs

• HR looking up policies

• Finance referencing checklists

• Sales asking product/pricing questions

🧰 Architecture Overview

• Azure App Service: Hosts the Streamlit frontend

• Azure AD: Authenticates internal users via SSO

• Azure Key Vault: Securely stores JWT private key + metadata

• Snowflake Cortex LLM: Handles embeddings and completions

• Snowflake Vector Table: Stores doc chunks + embeddings

🛠️ Step-by-Step Build

1. Upload and Chunk Internal Docs

• Convert your PDFs, DOCXs, and SOPs to clean text

• Chunk them into manageable pieces (e.g., 300–500 tokens per chunk)

• Store each chunk in Snowflake with metadata (source, title, section)

🛠️ Coming soon: sample code for chunking + loading to Snowflake

2. Generate Embeddings Using Snowflake Cortex

Store these embeddings in a dedicated vector table for retrieval.

🛠️ Coming soon: sample code for chunking + loading to Snowflake

3. Create a Semantic Search Function

When a user asks a question:

1. Generate embedding for the query

2. Use cosine similarity to find the most relevant document chunks

3. Pass both query and context to snowflake cortex functions for response generation

🛠️ Similarity query and prompt logic will be detailed in the final post

4. Build the Streamlit Front-End

Using Streamlit, create a chatbot UI:

• Input box for user questions

• Display area for LLM response

• Optional: show source documents for transparency

🛠️ UI code + screenshot coming soon

5. Deploy the Chatbot with Azure Web App

Instead of Snowflake’s Streamlit service, we deploy to Azure App Service:

• Upload your Streamlit app to Azure (via GitHub or ZIP)

• Use Managed Identity or Service Principal for secure Key Vault access

• Store JWT private key and Snowflake creds in Azure Key Vault

6. Secure Access with Azure AD SSO

Set up Azure AD authentication for the Web App:

• Restrict access to internal users

• Log user identity and group membership (for audit/tracking)

• Optionally pass user context to the chatbot for personalisation

🔐 Authentication Flow Summary

• Internal user logs into Azure Web App using Azure AD credentials

• App retrieves Snowflake credentials securely from Key Vault

• Connects to Snowflake using JWT-based authentication

• All logic and data remain within your enterprise boundary

🎯 Benefits

• No manual login or password sharing

• Credentials are managed securely

• Fine-grained access control via Snowflake roles + Azure groups

• Scalable and low-maintenance architecture

• Fast, accurate responses with contextually rich answers

💡 Best Practices

• Use a separate Snowflake role for the chatbot (read-only, scoped access)

• Refresh JWT tokens automatically before expiry

• Log user questions and LLM responses for monitoring + improvement

• Chunk documents with context (e.g., headers, page numbers)

• Use embeddings metadata for fine-tuned filtering (e.g., "only HR docs")

📌 Conclusion

You don’t need to set up your own LLM or vector DB to build internal search tools. With Snowflake Cortex, Streamlit, and Azure App Service, you can build and deploy a secure, enterprise-grade chatbot with minimal infrastructure.

It’s fast to build, easy to scale, and truly useful for internal teams.