Sr. Data Engineer

DUTIES and Responsibilities

Responsibilities

• Help build the AWS data platform: S3 lakehouse on open table formats (Iceberg/Delta) organized in medallion (bronze/silver/gold) layers, with a Redshift, Snowflake, or Databricks serving layer.

• Migrate Azure workloads (Data Factory, Synapse, Azure SQL, ADLS) to AWS: assess, re-platform pipelines and models, validate data parity, and support cutover.

• Build batch and event-driven pipelines (Glue, Step Functions, Lambda, Kinesis) from Salesforce, SmartSimple and other enterprise sources.

• Develop gold-layer star-schema models that support Tableau and self-service analytics.

• Build the data foundation for AI: vector pipelines, RAG, and agentic/LLM solutions using Amazon Bedrock, SageMaker, and OpenSearch.

• Deliver infrastructure as code and CI/CD (Terraform/CDK, Git) with governance, security, and data-quality controls.

• Tune and cost-optimize workloads; document architecture and migration mappings; report progress to stakeholders.

________________________________________

REQUIRED SKILLS

• Deep hands-on AWS data stack: S3, Glue, Redshift, Athena, Lambda, Step Functions, Lake Formation, IAM.

• Snowflake and/or Databricks development (Spark/SQL transformations, warehouse and lakehouse models).

• Data modeling: conceptual, logical, and physical design; dimensional (star/snowflake schema) modeling with conformed dimensions and slowly changing dimensions; normalized and Data Vault patterns; defining grain, keys, and semantic-layer models for BI.

• Modern techniques: medallion architecture, ELT, change data capture, incremental loads, dbt, data-quality testing.

• Cloud architecture: secure, resilient, cost-efficient design (VPC, identity, encryption, multi-account).

• Applied AI/GenAI: Bedrock or SageMaker, LLM integration, RAG, vector databases; agent frameworks/MCP a plus.

• Strong Python (PySpark) and advanced SQL; Parquet, Iceberg, and Delta formats.

• Translates business needs into technical solutions; Agile; strong communication and collaboration.

________________________________________

. REQUIRED EXPERIENCE

• 5+ years running production data platforms on AWS, including at least one cloud migration (Azure-to-AWS preferred).

• Production Snowflake and/or Databricks experience building medallion-layer lakehouses.

• Delivered at least one AI/ML or GenAI solution to production (e.g., RAG app, AI agent, ML pipeline).

• Azure data services (Data Factory, Synapse, Azure SQL, ADLS); Salesforce integration and Tableau a plus.

• Handling sensitive data and communicating technical concepts to executives and non-technical staff.

________________________________________