Lead Data Engineer/Data Architect

Role Overview: Data Architect/Data Engineering Lead

We are looking for a Data Architect who can provide strong technical leadership across data engineering, architecture, and cloud solutions. The role requires a hands-on architect who can make sound technical decisions, work closely with the customer and engineering teams, resolve technical roadblocks, and drive scalable data solutions from design through implementation.The ideal candidate should be versatile across modern data and AWS technologies, with strong experience in Python, Apache Spark, SQL, AWS, dbt, and Snowflake, along with a good understanding of event-driven architectures, CDC, data integration, and cloud-native services.

Key Responsibilities

• Own and drive data architecture and technical decisions across data engineering initiatives.
• Work closely with customer architects, engineering leaders, product teams, and developers to understand business and technical requirements and translate them into scalable solutions.
• Define data architecture, integration patterns, data models, pipelines, processing frameworks, and technology standards.
• Provide technical direction to data engineering teams and unblock complex implementation challenges.
• Review solution designs, code, data models, pipelines, and technical approaches to ensure quality, scalability, performance, and maintainability.
• Lead architecture discussions with the customer and provide clear recommendations, trade-offs, and technology choices.
• Design and optimize data pipelines using Python, Spark, SQL, dbt, and Snowflake.
• Define and implement cloud-native data solutions leveraging relevant AWS services.
• Work across batch and near-real-time data processing, including CDC, event-driven architecture, streaming, and data integration patterns.
• Provide guidance on services such as AWS Lambda, SNS/SQS, S3, EventBridge, Glue, Step Functions, IAM, and other relevant AWS capabilities.
• Establish patterns for data quality, governance, security, lineage, observability, and operational reliability.
• Identify technical risks, dependencies, and architectural gaps early and drive them to resolution.
• Collaborate with DevOps, application, security, and platform teams to ensure end-to-end solution alignment.
• Support technical estimation, roadmap planning, PoCs, and architecture decisions for new initiatives.
• Stay hands-on enough to prototype solutions, troubleshoot issues, and validate architectural decisions when required.
• Mentor engineers and help build stronger technical capabilities within the team.

Required Technical SkillsCore Data Engineering

• Strong hands-on experience with Python, Apache Spark, and SQL
• Strong understanding of data modeling, ETL/ELT, data pipelines, and data integration
• Experience with dbt and modern data transformation practices
• Strong experience with Snowflake, including data modeling, performance optimization, and scalable data processing

AWS / Cloud

• Strong AWS experience with the ability to design cloud-native data solutions
• Good working knowledge of services such as:
• S3
• Lambda
• Glue
• EventBridge
• SNS / SQS
• Step Functions
• IAM
• CloudWatch
• Understanding of cloud security, networking, scalability, reliability, and cost considerations

Data Architecture

• Experience designing modern data platforms and data architectures
• Strong understanding of CDC (Change Data Capture) and incremental data processing
• Experience with batch and event-driven architectures
• Understanding of APIs, file-based integrations, messaging, and streaming patterns
• Strong understanding of data quality, governance, lineage, and observability

Customer & Leadership Skills

• Strong ability to communicate architecture and technical decisions to both technical and non-technical stakeholders
• Comfortable working directly with customers and challenging assumptions when required
• Ability to evaluate multiple technical options and clearly articulate trade-offs and recommendations
• Strong problem-solving skills with a bias toward unblocking teams and moving delivery forward
• Ability to operate across architecture, engineering, cloud, data, and application teams
• Comfortable working in an environment where requirements and technical challenges evolve rapidly
• Strong ownership and accountability for technical outcomes

Experience

• 10+ years of experience in data engineering / data architecture / cloud data platforms
• Strong hands-on experience with Python, Spark, SQL, AWS, dbt, and Snowflake
• Proven experience working as a Data Architect / Lead Data Engineer / Data Engineering Architect
• Experience working directly with enterprise customers and distributed engineering teams
• Experience making architecture decisions and leading technical delivery across complex data initiatives

What Success Looks Like

• Engineering teams have clear technical direction and architecture guidance
• Technical blockers are identified and resolved quickly
• Customer and engineering teams are aligned on architecture, technology choices, and implementation approach
• Data solutions are scalable, secure, performant, and maintainable
• Architecture decisions translate effectively into working production solutions
• The team becomes increasingly self-sufficient through strong technical guidance and mentoring