Sr Data Engineer - Remote

We are seeking a Senior / Lead Data Engineer to join a mature enterprise data platform team built on Databricks, Spark, and DBT . This role is designed for an engineer who can lead, own, and drive work , not simply execute tasks.

The environment is past early build‑out and moving into scale, refinement, and optimization . The engineer will work across ingestion pipelines and refined (gold‑layer) assets , while also mentoring junior engineers and taking accountability for delivery.

This is not an order‑taker role . Candidates must bring ideas, make design decisions, and lead workstreams independently.

Key Responsibilities

• Lead the design, development, and optimization of data ingestion pipelines using Databricks and Spark
• Build and maintain refined (gold‑layer) data assets using DBT and SQL
• Author and manage DBT models and macros to create reusable, scalable transformations
• Work within a medallion architecture (bronze / silver / gold layers)
• Partner with product owners, engineers, and stakeholders to understand data use cases
• Take ownership of workstreams and mentor junior engineers, including task delegation and progress oversight
• Identify performance, scalability, and cost‑optimization opportunities in Databricks workloads
• Contribute to reusable ingestion and transformation frameworks used across teams

Required Technical Skills (Non‑Negotiable) Candidates must have strong, hands‑on experience with:

• Databricks (production experience required)
• Apache Spark / PySpark
• DBT, including DBT macros
• Python
• SQL (used primarily through DBT)
• Strong understanding of data warehousing and medallion architecture concepts

This role requires real implementation depth , not surface‑level exposure. Experience & Seniority Expectations

• Senior / Lead‑level engineer (title history as Lead is a strong plus)

• Proven ability to:

• Take ownership and accountability

• Drive solutions independently

• Lead and mentor junior engineers

• Comfortable operating in a team of 7–8 engineers

• Able to stay on a project long‑term (1+ year mindset strongly preferred)

Nice‑to‑Have Skills

• Airflow (primary orchestration tool)
• Experience working with hundreds of pipelines in large enterprise environments
• Prior experience building reusable data assets and frameworks
• Financial services exposure (banking, investments, insurance) — preferred but not required

Domain & Business Acumen (Important)

• Financial services experience is not a hard requirement

• Candidates must be:

• Willing to learn business context (banking, mutual funds, brokerage systems)

• Interested in becoming a data SME who can answer stakeholder questions

Ability to connect technical work to business value is a strong differentiator