Data Engineer

Data Engineer

Location: Atlanta, GA

Work Arrangement: Hybrid

Job Type: Contract

Overview

We are seeking a highly skilled Data Engineer to design, build, and support scalable data and streaming solutions that transform enterprise data into trusted, governed datasets for business intelligence and advanced analytics.

The Data Engineer will work across the full data engineering lifecycle—from gathering and clarifying requirements with business stakeholders and data modelers to designing lakehouse architectures, developing production pipelines, and supporting solutions in a cloud environment. The ideal candidate brings strong hands-on experience with Databricks, Delta Lake, Apache Spark, Kafka, AWS, and SQL, along with a strong engineering mindset focused on data quality, observability, scalability, and maintainability.

Responsibilities

• Own the architecture, design, development, and delivery of scalable data engineering and streaming solutions.
• Partner with business stakeholders, data modelers, and technical teams to understand requirements and translate business needs into durable data solutions.
• Design and implement lakehouse architectures and production-grade data pipelines using Databricks and Delta Lake.
• Develop ingestion and transformation processes using Delta Live Tables (DLT), Apache Spark, Spark SQL, and PySpark.
• Build and support real-time and streaming data solutions using Kafka, including managed platforms such as Confluent Cloud.
• Leverage AWS services, including S3 and related integration patterns, to support enterprise data solutions.
• Develop complex and efficient SQL for data transformation, analysis, and integration.
• Apply strong software engineering practices to ensure solutions are scalable, maintainable, reliable, and secure.
• Implement and maintain data quality, governance, monitoring, and observability practices.
• Troubleshoot production data pipelines and streaming processes and resolve performance and reliability issues.
• Collaborate across teams to define measurable outcomes and deliver data solutions that support BI and advanced analytics initiatives.
• Clearly communicate technical concepts, requirements, designs, and project status in both written and verbal formats.

Required Qualifications

• Bachelor’s degree required, preferably in Information Systems, Computer Science, Electrical Engineering, Computer Information Systems, or a related field.
• Advanced, hands-on experience with Databricks and modern data engineering practices.
• Strong experience with Delta Lake, Delta Live Tables, and Unity Catalog.
• Strong proficiency with Apache Spark, including Spark SQL and PySpark.
• Hands-on experience building streaming and event-driven data solutions using Kafka; Confluent Cloud experience is a plus.
• Strong experience with SQL and enterprise data transformation.
• Experience with AWS, particularly S3 and related data integration services/patterns.
• Experience designing and operating production-grade data pipelines and scalable analytics solutions.
• Strong understanding of data quality, governance, observability, and maintainability.
• Excellent written and verbal communication skills with the ability to work effectively across technical and business teams.

Preferred Qualifications

• Experience with Confluent Cloud and managed Kafka environments.
• Experience with enterprise lakehouse architecture and modern data platforms.
• Experience supporting BI, advanced analytics, or data science initiatives.
• Strong understanding of data modeling and data architecture concepts.
• Experience with CI/CD, source control, automated testing, and other modern engineering practices.