Freelancers using Snowflake in San Francisco
Freelancers using Snowflake in San Francisco
Sign Up
Post a job
Sign Up
Log In
Filters
2
Projects
People
Metricgrain Lab
San Francisco, USA
Snowflake cost, data modeling & AI analytics
New to Contra
Follow
Message
Snowflake cost, data modeling & AI analytics
1
Data Ingestion Migration: $170K a Year Saved (cover: work_6.png) # Problem: At a $2B SaaS unicorn, Fivetran’s new pricing model was about to raise ingestion costs from ~$13K to ~$23K a month, with no change in data volume. Syncs from Google Cloud SQL read replicas also broke whenever a replica lagged or was rebuilt, and each break needed manual re-syncs and chat with Fivetran team (due to table size). # What I did - Benchmarked ELT alternatives on cost, connector coverage and sync reliability - Migrated production connectors to a modern ELT platform in about 30 days, running old and new pipelines in parallel - Built an end-to-end validation framework comparing every internal and customer-facing data product before cutover: row counts, values and schemas - Found and resolved data type inconsistencies between the two tools (timestamps, decimals, JSON and nested fields) plus edge cases in deletes and soft-deletes - Preserved full history: migrated SCD Type 2 tables and past deletes from the existing Snowflake tables, so no history was lost and nothing had to be re-synced from source - Re-pointed downstream dbt sources only after the outputs matched - Tuned sync frequency and schemas to stop loading tables nobody used - Set up automatic recovery for replica issues, so syncs resume on their own instead of failing and waiting for a manual re-sync # Result: - Ingestion cost down from ~$13K to ~$9K a month (about 30% lower) ~$170K a year saved against the projected new pricing - Zero changes to internal or customer-facing data products: every output validated identical before cutover - Full history kept: SCD Type 2 history and past deletes carried over with no historical re-sync - More reliable pipelines: replica hiccups now recover automatically instead of breaking syncs and paging the team
1
12
0
The open-source Snowflake Cost Scan
0
17
0
Title: Customer-Facing AI Analytics Chat on Snowflake # Problem 20K+ business customers wanted answers about their own business without building dashboards or waiting on support. Pointing an LLM straight at raw tables gave wrong joins, inconsistent metric definitions and numbers that didn’t match the product’s dashboards, so nobody trusted the answers. # What I did - Explored options for Semantic layers and integrated dbt Smentic layer to company stack. - Defined business metrics once in the dbt Semantic Layer, the single source of truth for internal, external analytics (this grew eventually to ~300 metrics) - Built a chat interface where the LLM picks governed metrics and dimensions from the semantic layer instead of writing free-form SQL. - Kept it multi-tenant and secure: each customer only ever sees their own data Built an evaluation framework to measure and protect accuracy: - A golden set of real customer questions, each with the expected metric, dimensions, filters and answer - Covers frequent lookups, time comparisons, filtered breakdowns and multi-metric questions - Automated scoring: correct metric and filter selection, plus result match against the expected answer. - Separate LLM agents evaluates the answer in predefined criteria and steps. # Result - 92% of eval questions answered correctly - Answers in under 10 seconds - 300+ governed metrics available in plain English to 20K+ customers - Same numbers in chat as in the dashboards; new metrics show up in chat as soon as they’re added to dbt. # Tools: - Snowflake, dbt Semantic Layer, Kimball dimensional modeling, LLMs, Python
0
26
0
Title: Snowflake Cost Optimization: $1.3M in Cumulative Savings Tools: Snowflake, dbt, Airflow, SQL # Problem A $2B SaaS unicorn’s Snowflake bill kept climbing as data volumes and dbt models grew. Spend was spread across CI runs, stale models, idle warehouses and storage nobody used, and no one owned it. # What I did Ran Snowflake cost as a quarterly program: attributed spend to dbt models and workloads, ranked every fix by ROI, and shipped 25+ initiatives over 3 months. - Retired a legacy data models: ~$240K/yr - Trimmed CI test data: ~$190K/yr - Removed expensive models with little or no usage: ~$180K/yr - Cleaned up stale, backup and dev databases: ~$175K/yr - Turned rarely queried tables into views: ~$145K/yr - Moved heavy models to incremental loads and fixed clustering: up to 90% cheaper per model. - Contract re-negotiation with Snowflake - Other # Result - $1.3M in cumulative annual savings across 25+ initiatives - Biggest single win ~$240K/yr; fastest win ~$65K/yr from a one-hour change
0
37
Snowflake
(4)
Follow
Message
Explore people
