catalog / saptrishi / portfolio fresh

TABLE portfolio.saptrishi

Saptrishi, data engineer.

I design and run reliable data platforms — streaming pipelines, lakehouses and the models analysts actually trust.

Get in touch →Selected work
eventsorders_dbcrm_apispark · dbtwarehousedashboardsml_features
fig. 1 — a typical pipeline I build, live.
2B+
events processed / day
120+
pipelines in production
-40%
warehouse spend
99.9%
SLA on critical jobs
01

about

bio TEXT · focus TEXT[]

I'm a data engineer who likes boring infrastructure and exciting outcomes. For the last few years I've been turning messy event streams and legacy cron jobs into well-tested, observable pipelines.

I care about data contracts, cost-aware warehouse design, and making on-call quiet. Outside work I tinker with DuckDB side projects and over-engineer my home weather station.

02

experience

company VARCHAR · role VARCHAR · period DATERANGE

  1. Senior Data Engineer

    Northwind Logistics2023 — Present

    Lead engineer on the real-time data platform serving pricing, ops and ML teams.

    • Built a Kafka → Flink → Iceberg streaming pipeline handling 2B events/day at p95 < 5s.
    • Cut Snowflake spend 40% with incremental models, clustering and warehouse right-sizing.
    • Introduced data contracts and dbt tests in CI, reducing data incidents by 70%.

  2. Data Engineer

    Contoso Retail2021 — 2023

    Modernised batch ETL and built the first CDC pipelines for the analytics org.

    • Migrated 300+ legacy cron jobs to Airflow on Kubernetes with retries, SLAs and lineage.
    • Shipped Postgres → warehouse CDC with Debezium, replacing nightly full dumps.
    • Grew the dbt project to 200+ tested models used by 60 analysts.

  3. Analytics Engineer

    Fabrikam2019 — 2021

    First data hire — stood up the warehouse and self-serve reporting.

    • Designed the initial BigQuery warehouse and dimensional models.
    • Automated weekly reporting, saving the finance team ~20 hours a week.
03

projects

name VARCHAR · lineage ARRAY · impact METRIC
model_01

Real-time clickstream pipeline

Exactly-once event ingestion from web and mobile into an Iceberg lakehouse, queryable in Trino within seconds.

Kafka→Flink→Iceberg→Trino

2B events/day · p95 < 5s

model_02

Lakehouse migration

Moved a creaking on-prem warehouse to a medallion architecture on Delta Lake with zero downtime for dashboards.

Databricks→Delta Lake→Spark→Terraform

-40% cost · 3× faster queries

model_03

Data quality & contracts

Schema-registry checks in CI, freshness and volume monitors, and Slack alerts that route to the owning team.

Great Expectations→dbt→GitHub Actions

-70% data incidents

model_04

Laptop-scale analytics stack

Open-source demo: an orchestrated weather ingestion pipeline with DuckDB, dbt and a static BI site.

Dagster→DuckDB→dbt→Python

Runs end-to-end for $0

Source ↗
04

skills

category VARCHAR · tools ARRAY<STRING>
languagesPythonSQLBash
processingSparkFlinkdbtPolars
streamingKafkaDebeziumKinesis
orchestrationAirflowDagsterPrefect
storageSnowflakeBigQueryDatabricksIcebergPostgres
platformAWSTerraformDockerKubernetesGitHub Actions
azureData FactorySynapse AnalyticsMicrosoft FabricADLS Gen2Event HubsAzure FunctionsCosmos DB
azure devopsAzure ReposAzure Pipelines (YAML)Azure BoardsAzure ArtifactsAzure Test PlansBicepARM templatesKey VaultAzure MonitorApplication InsightsAKSContainer Registry
dataopsCI/CD for dataGreat Expectationsdbt testsData contractsData observabilityOpenLineageGit
biTableauPower BI
aiRAGAzure OpenAIAzure AI Search
05

contact

channel VARCHAR · latency INTERVAL

Have data that needs moving?

INSERT INTO conversations (who, topic)
VALUES ('you', 'a data platform role');
-- typical response time: under 24h
hello@dreamymonk.xyzsaptrishiwork@gmail.comGitHubLinkedIn