Senior Data Engineer (Databricks) - US Citizen - DHS EOD Clearance

Job Summary

ETC Solutions is hiring a Senior Data Engineer to design and build the data pipelines behind a mission-critical analytics program at the Cybersecurity and Infrastructure Security Agency (CISA). You will lead the lakehouse engineering work on an enterprise Databricks platform, set the technical standard for the team, and mentor other engineers.

This role is for a senior engineer who has built production pipelines at scale, knows how to make them fast and affordable, and can architect solutions that hold up for years.

What You Will Do

• Design and optimize batch and streaming pipelines using PySpark, Lakeflow Spark Declarative Pipelines, Delta Lake, Auto Loader, and Workflows, with error handling, retries, and alerting
• Lead medallion architecture (bronze, silver, gold) ingestion and refinement, along with data lifecycle management and storage tiering
• Tune performance and cost through cluster sizing, serverless adoption, Photon, file sizing, Z-ORDER, partitioning, and caching
• Implement Unity Catalog governance, including RBAC and ABAC, row and column level security, governed tags, and lineage
• Build Git-based CI/CD using Databricks Repos, Asset Bundles, Terraform, and automated testing
• Design schemas for and administer relational and NoSQL systems such as PostgreSQL, Aurora, and DynamoDB
• Perform root cause analysis and production support for data pipelines
• Mentor engineers and review their code
• Define system requirements and architecture for enterprise-scale information systems, making sure solutions follow open systems standards and integrate cleanly across APIs and external systems

What You Bring

• 8+ years of total experience, with at least 7 years within the last 10 in data engineering, ETL/ELT pipeline development, or distributed data processing in enterprise cloud environments
• Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent experience
• Databricks Certified Data Engineer Professional (required)
• At least one cloud certification, such as AWS Certified Developer Associate or AWS Certified Solutions Architect Associate (required)
• Ability to obtain a favorable DHS/CISA Entry on Duty determination through an SF-85P Public Trust background investigation
• U.S. citizenship

Nice to Have

• Databricks Lakehouse Architect or Generative AI Engineer certification
• Experience with Databricks Model Serving, Genie, or Agent Bricks
• Active or prior DHS Public Trust
• Experience supporting federal agencies

Work Schedule

• On site in Arlington, VA (4200 Wilson Blvd and 1310 N. Courthouse Rd)
• Monday through Friday, 8:00 AM to 5:00 PM ET
• Telework only with CISA approval, and only within the U.S.

Start Timeline

The Public Trust investigation typically takes 30 days or more after a complete package is submitted. Candidates with active DHS suitability may start sooner.

Pay: $155,000.00 - $165,000.00 per year

Benefits:

• 401(k)
• 401(k) matching
• Dental insurance
• Health insurance
• Paid time off
• Retirement plan
• Vision insurance

License/Certification:

• Databricks Data Engineer Professional (Required)
• AWS or Azure Cloud Associate Certification (Required)

Security clearance:

• Secret (Preferred)

Work Location: Hybrid remote in Arlington, VA 22203