Master Data Management Using Snowflake + Streamlit Apps

Learn how to build an internal Master Data Management (MDM) solution using Snowflake for logic and audit tracking, and Streamlit for a self-service business UI.

This low-code, cost-effective approach enables teams to:

• Clean and group inconsistent values

• Confirm or correct fuzzy-matched suggestions

• Track approvals with full audit trail

• Drive consistent reporting in Tableau, Power BI, and dbt models

All without paying for a full-scale MDM product.

💬 Why This Matters

In almost every business, free-text or poorly maintained master data causes:

• Duplicated records

• Messy dashboard filters

• Misaligned reporting metrics

• Manual cleanup before each reporting cycle

Whether you're standardising investment channels, client types, hotel amenities, or partner names, the need is the same:
📦 Raw values in → Cleaned values out → Auditable, confirmed, and trusted.

💡 Why Not Use a Traditional MDM Tool?

There are many great commercial Master Data Management tools (like Informatica MDM, Reltio, Ataccama, etc.) — but they come with:

• 💸 High licensing costs

• 🛠️ Complex implementations

• 🧠 Specialised skillsets and vendor lock-in

• 🕰️ Long timelines to onboard and scale

As a result, many growing companies still rely on:

• Excel spreadsheets

• CSV-based mapping files

• Manual adjustments during reporting

This is fragile, error-prone, and not scalable.

✅ A Practical Alternative

With Snowflake + Streamlit, you can build:

• ✅ A governed, auditable master data layer

• ✅ A low-code UI your business teams can actually use

• ✅ At zero licensing cost (compute only)

This approach is perfect for teams that need:

• 🧩 Lightweight governance

• 🧹 Self-service cleanup workflows

• ⚡ Fast time-to-value — without vendor lock-in

🧪 Use Case: Interactive Grouping for Investment Channels

Let’s say your CRM or onboarding flow captures free-text for "Channel":

"private_banker", "PB", "PrivateBank", "Retail", "direct", "Web-Direct", "Online Booking"

Your team needs to group and clean these into:

• "Private Banking"

• "Retail"

• "Online Direct"

The MDM app enables business users to:

• View raw values

• See system-suggested display names

• Edit or override them

• Confirm values with one click

• Push results to reporting — with full audit trail

🖥️ Key App Features

• Tab-based UI per master data domain

• Filters for confirmed/unconfirmed status

• Search on raw or display name

• Edit suggestions inline

• Checkboxes to confirm

• Audit columns for user + timestamp

• Snowflake-native — no external infra or storage needed

🛠️ Full app UI, code, and SQL snippets coming soon.

🧭 Types of MDM Workflows

1. Static Mapping (Controlled Reference Mapping)

For simple, fixed relationships where raw values should always resolve to the same group or label.

Examples:

• "EMEA" → "Europe, Middle East, and Africa"

• "US" → "United States"

✅ Best for: low-cardinality fields, controlled vocabularies, reference codes.

2. Interactive Grouping & Cleanup (Fuzzy Matching)

For messy, free-text fields with high variation across systems.

Supports:

• System-suggested “group” based on similarity (e.g., Soundex, Levenshtein, co-occurrence)

• Business user confirmation

• Inline editing of display names

• Logging of who confirmed what, and when

Example:

→ Raw values: "JP Morgan", "JPMorgan", "JPM", "JP Morgan Bank"

→ Suggested group: "J.P. Morgan"

✅ Best for: long-tail values, unstructured entry points (forms, CRM, legacy systems)

3. Hybrid and Contextual Mappings

You can mix both methods based on:

• Field type

• Source system

• Region or business unit

Example:

• Use static mapping for known internal codes

• Use interactive grouping for external vendor names

• Apply filters (e.g., geography) to scope reviews

🖥️ App Experience (Business UI)

The Streamlit app provides:

• Filterable list of values (confirmed/unconfirmed)

• Search by raw value or display name

• Inline editing and checkbox to confirm

• Real-time audit of who made changes

• Role-specific views (optional)

🛠️ Screenshots and code templates coming soon.

🔐 Governance & Audit Trail

All confirmations are saved with:

• display_name

• suggested_group (optional)

• confirmed_by (Snowflake CURRENT_USER)

• confirmed_date (timestamp)

• Optional: source_system, last_seen_date, is_active

📊 Analytics Integration

Downstream tools like Tableau, Power BI, or dbt models can:

• Pull only confirmed values (is_confirmed = TRUE)

• Highlight or exclude unconfirmed records

• Join to this table for trusted labels or groupings

🎯 Key Benefits

• ✅ Business-friendly interface for self-service cleanup

• ✅ Easy to extend across data domains

• ✅ Zero dependency on third-party tools or vendors

• ✅ Auditable, traceable, and real-time inside Snowflake

• ✅ Faster, cheaper, and more transparent than a big MDM rollout

📎 Artefacts

• GitHub repo with full Streamlit app template

• Fuzzy matching logic using Snowflake

• Bulk mapping upload/import and admin review tools

• Role-based access per data domain