Senior Data / Platform Lead

Senior Data / Platform Lead

Remote | LATAM | Full-time

Position Overview

Upstart AEC is hiring a Senior Data / Platform Lead for a full-time, fully remote role.

This is a hands-on technical leadership role focused on taking an Azure data platform from early architecture into pilot delivery, production serving, and operational handoff. The role contributes to initial architecture, becomes significantly more involved during the pilot, takes full ownership through the serving-slice build, and remains engaged through enablement.

The right person combines Azure data-platform depth with strong hands-on delivery skills across Python, SQL/T-SQL, Spark, PostgreSQL, infrastructure integration, secure multi-tenant serving, and production operations.

Key Responsibilities

• Contribute to early platform architecture and technical design, with particular focus on Azure data services, security, private networking, identity, and serving patterns.
• Take increasing hands-on ownership as the solution moves into the pilot phase, ensuring architecture decisions translate into reliable working components.
• Lead technical delivery during the serving-slice phase, coordinating ingestion, transformation, storage, serving, security, testing, and production-readiness work.
• Build and guide SQL Server ingestion using Change Tracking, including incremental loads, replay, backfill, reconciliation, and schema-drift handling.
• Develop and oversee data-processing workflows using Python, SQL/T-SQL, ADLS Gen2, Parquet/Delta, and PySpark/Spark.
• Design PostgreSQL serving marts that support downstream analytics and application consumption.
• Implement secure multi-tenant serving patterns, including identity-bound routing, tenant isolation, regional-residency controls, and automated failure testing.
• Lead integration of the data platform with AKS, Terraform, GitHub Actions/ArgoCD, Entra managed identity, Key Vault, private endpoints, and Azure Monitor.
• Partner with platform engineering resources on infrastructure-as-code, CI/CD, GitOps, networking, and deployment integration while remaining hands-on with data-platform implementation.
• Drive production readiness through automated testing, monitoring, troubleshooting, runbooks, operational documentation, and structured handoff.
• Support enablement and knowledge transfer after the core platform and serving capabilities are established.

Required Skills

• Strong hands-on experience delivering production Azure data platforms.
• Experience with Azure Kubernetes Service (AKS), Azure Data Lake Storage Gen2 (ADLS Gen2), private networking, managed identity, and Key Vault.
• Strong Python and SQL/T-SQL skills.
• Hands-on experience with SQL Server Change Tracking and incremental ingestion patterns.
• Strong understanding of replay, backfill, reconciliation, schema-drift handling, and production recovery scenarios.
• Experience working with Parquet/Delta and PySpark/Spark.
• Experience designing and implementing PostgreSQL serving marts.
• Experience taking data-platform solutions from technical design through implementation, serving, and operational handoff.
• Terraform and CI/CD/GitOps fluency sufficient to lead integration with platform engineering resources.
• Experience implementing secure multi-tenant architectures, including tenant isolation, identity-based routing, and regional-residency controls.
• Ability to lead technical implementation while remaining actively involved in development, integration, testing, and troubleshooting.

Nice-to-Have Skills

• SQLMesh.
• dlt.
• Cube or equivalent production semantic-layer experience.
• Dagster or equivalent orchestration experience.
• Experience with EAV or custom-field data models.
• Retail analytics experience.