Senior Data Engineer

**This is a Consulting role for upcoming client engagements**

At SQA Group, we help companies turn data into trusted, scalable products that improve decisions and business outcomes. Our teams modernize legacy data estates, build cloud and lakehouse platforms, create reliable batch and streaming pipelines, and establish the quality, governance, security, and delivery practices required for analytics and AI at scale.

We’re looking for a consultative-minded, hands-on Senior Data Engineer to support SQA Group’s in-house Data & Advanced Analytics Solutions team across client modernization, implementation, and managed-service engagements. The ideal candidate can assess an existing environment, recommend a pragmatic target architecture, and deliver secure, supportable solutions across cloud, hybrid, and on-premises platforms.

• Design, build and integrate data pipelines from various sources
• Assemble large, complex sets of data that meet non-functional and functional business requirements
• Ensure that data models run reliably in production

Typical Responsibilities:

• Design, build, and operate scalable cloud, lakehouse, warehouse, and hybrid data architectures that support batch and streaming workloads.
• Improve platform scalability, reliability, security, and cost efficiency through modular design, infrastructure as code, automated testing, CI/CD, version control, monitoring, and the retirement of manual processes.
• Operate data warehouses, lakehouses, data applications, and production pipelines, including ingestion, transformation, orchestration, data quality, lineage, monitoring, incident response, performance tuning, and delivery to analysts, data scientists, applications, and AI solutions.
• Design and implement ETL and ELT pipelines across databases, APIs, files, SaaS applications, event streams, and legacy platforms while preserving data integrity, security, lineage, and reconciliation throughout migrations and ongoing operations.
• Build required infrastructure for optimal extraction, transformation and loading of data from various data sources using various SQL & NoSQL databases, cloud and on-premises services, and data integration tools
• Build analytical tools that utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition
• Work directly with stakeholders including Executive, Product, Data and Design teams to support their data infrastructure needs while assisting with data-related technical issues.
• Lead technical discovery, profile source data, document current-state and target-state architectures, estimate delivery effort, communicate trade-offs to technical and business stakeholders, and produce runbooks, operational playbooks, and knowledge-transfer materials that enable effective client ownership.

What You Bring:

Required capabilities include:

• Advanced SQL and Python
• Dimensional, relational, and lakehouse data modeling
• Production pipeline design
• Automated testing
• Git-based development
• CI/CD
• Data quality and observability
• Cloud security and identity
• Performance and cost optimization
• Clear technical communication with client stakeholders.

You should demonstrate depth in at least one modern cloud data ecosystem and the ability to adapt quickly across client platforms.

Demonstrated understanding and experience using software and tools including:

• Understanding of relational and non-relational data models; structured, semi-structured, and unstructured data; and the unique challenges presented by big data sets
• Distributed processing and streaming technologies such as Apache Spark, PySpark, Kafka, and Structured Streaming; experience with Hadoop ecosystems is helpful for legacy modernization engagements.
• NoSQL databases including Cosmos, Couchbase, Cassandra, MongoDB, or Google Firestore
• Database platforms including PostgreSQL, MySQL, SQL Server, DB2, or Oracle
• Workflow management or pipeline tools such as Airflow, Luigi or Azkaban
• Cloud and lakehouse platforms, with particular value placed on Microsoft Azure and Fabric technologies such as Azure Data Factory, Azure Databricks, ADLS, Fabric Data Factory, Fabric Lakehouse, Fabric Warehouse, OneLake, and Eventstream; experience with AWS or Google Cloud is also valuable.
• Data ingestion and transformation tools such as dbt, Fivetran, Airbyte, Matillion, Azure Data Factory, Fabric Data Factory, IBM DataStage, or comparable platforms, with the ability to evaluate build-versus-buy trade-offs.
• Advanced Python and SQL are required; experience with PySpark, Scala, Java, C#, or another general-purpose language is valuable for distributed processing, application integration, and platform automation.

Additional relevant experience includes SQL Server modernization and administration, Databricks workspace and Delta Lake design, lakehouse table formats such as Delta or Iceberg, Power BI or semantic-model integration, REST APIs and CDC patterns, Terraform or Bicep, Docker, Azure DevOps or GitHub Actions, Microsoft Purview or comparable cataloging tools, and data quality or observability frameworks.

This role does not require experience with every platform listed above. We are seeking engineers with strong fundamentals, production delivery experience, depth in one or more modern data ecosystems, and the consulting judgment to learn new technologies and recommend solutions that fit each client’s architecture, operating model, and business goals.

Thank you for your interest in pursuing a career at SQA Group! This role is fully remote. SQA Group is an equal opportunity employer and supports workforce diversity. This job description is intended to describe the general nature and level of work being performed by the person assigned to this position. The primary duties and responsibilities above are intended to describe those functions that are essential to the performance of this job. This job description does not state or imply that the above are the only duties and responsibilities assigned to this position. All requirements are subject to possible modification to reasonably accommodate individuals with a disability.