Senior Databricks & Spark Data Engineer

W2 ONLY — NO C2C / NO 1099 / NO THIRD-PARTY

Work Authorization: U.S. Citizens (USC) or Green Card Holders (GC) ONLY

📍 Richardson, TX or Chicago, IL | Hybrid

🚨 W2 ONLY — NO C2C / NO 1099 / NO THIRD-PARTY

🇺🇸 USC OR GREEN CARD HOLDERS (GC) ONLY

📍 Location: Chicago, IL OR Richardson, TX

🏢 Work Model: Hybrid

💼 Employment Type: W2

⏳ Experience: 8+ Years — Mandatory

Please read the requirements carefully before applying. This opportunity is strictly W2. Candidates seeking C2C, Corp-to-Corp, 1099, or third-party arrangements should NOT apply.

We are looking for a Senior Databricks & Spark Data Engineer with strong hands-on experience in Scala, Apache Spark, Databricks, Apache Kafka, CDC, and Unity Catalog.

Key Responsibilities

• Design, develop, and maintain scalable enterprise data pipelines using Scala, Apache Spark, and Databricks.

• Develop high-performance Scala/Spark applications for large-scale batch and real-time data processing.

• Build and optimize Databricks notebooks, jobs, workflows, and Lakehouse-based data solutions.

• Design and implement real-time and near-real-time streaming pipelines using Apache Kafka and Spark Structured Streaming.

• Develop and maintain Change Data Capture (CDC) solutions to efficiently process inserts, updates, and deletes.

• Implement incremental data ingestion and transformation patterns using CDC, Delta Lake, and Databricks.

• Design reliable and scalable ETL/ELT pipelines for large volumes of structured and semi-structured data.

• Implement enterprise data governance, security, access controls, and data lineage using Databricks Unity Catalog.

• Optimize Spark and Databricks workloads for performance, scalability, reliability, and cost efficiency.

• Troubleshoot Spark jobs, Kafka pipelines, production failures, data quality issues, and performance bottlenecks.

• Implement monitoring, logging, data quality checks, error handling, and recovery mechanisms.

• Collaborate with data architects, platform engineers, application teams, and business stakeholders.

Mandatory Skills

Scala | Apache Spark | Databricks | Apache Kafka | CDC | Unity Catalog

Non-Negotiable Requirements

✅ U.S. Citizen or Green Card Holder ONLY

✅ Strictly W2

✅ 8+ years of relevant experience

✅ Must be comfortable working hybrid in Richardson, TX or Chicago, IL

❌ No C2C

❌ No 1099

❌ No Corp-to-Corp

❌ No third-party/vendor submissions

Work authorization and employment eligibility will be verified during the hiring process.

📩 Qualified candidates: [email protected]

Please apply only if you personally meet ALL requirements and are genuinely interested in W2 employment.