Senior Data Operations Engineer

Senior Data Operations Engineer _ Auburn Hills, MI (Hybrid)

Position Summary

We are seeking a Senior Data Operations Engineer with 8+ years of hands-on operational support experience to own and maintain batch jobs on our data engineering pipelines. This role owns operational stability of data platforms supporting domains and applications, ensuring reliable delivery, rapid incident resolution, and continuous performance improvement across the data ecosystem.

Key Responsibilities

Operations & Support

Monitor and support batch job execution across data engineering pipelines; proactively identify and resolve failures before they impact business operations.

Participate in on-call rotation schedules, providing timely response to production incidents.

Own and drive incident management, change ticket, and problem management processes following ITIL best practices.

Proactively monitor ticket queues and address assigned items immediately, ensuring SLAs are met.

Support administration activities across data platforms, including Tomcat application servers and related infrastructure.

Technical Ownership

Develop, troubleshoot, and optimize Python and Java code supporting batch and pipeline operations.

Identify and resolve performance bottlenecks; implement code and query optimizations for operational efficiency.

Support and maintain ETL/ELT data pipelines with deep understanding of data extraction, transformation, loading, and processing patterns end to end.

Manage and troubleshoot AWS Glue jobs, crawlers, and workflows supporting pipeline orchestration.

Support data storage and database layers including S3 (data lake tables), RDS databases, and query/performance tuning.

Support Oracle databases and OLTP systems, including query tuning and operational troubleshooting.

Administer and support Cloudera Hadoop ecosystem components including HDFS and Impala for distributed data storage and querying.

Work with Apache Spark for large-scale batch and distributed data processing troubleshooting and tuning.

Maintain and support BI tools, including Tableau and Power BI, for downstream reporting and analytics consumers.

Manage code libraries and dependencies, ensuring version currency and compatibility.

Governance, Quality & Security

Serve as gate reviewer for code being promoted to production, validating operational readiness and adherence to standards before release.

Ensure developers complete required operational validations (testing, documentation, rollback plans) prior to deployment.

Work within Git repositories, review pull requests, and enforce branching/merge standards for operational code changes.

Identify, track, and remediate security vulnerabilities in a timely manner in coordination with security teams.

Support end-to-end product lifecycle management, including deprecation, upgrades, and decommissioning of applications and pipelines.

Domain & Business Knowledge

Develop deep understanding of domain applications and business processes to inform troubleshooting and support decisions.

Partner with business and SME stakeholders to translate operational issues into actionable fixes.

Required Skills & Qualifications

8+ years of experience in data operations/production support roles.

Proficiency in Python and Java for pipeline support and automation.

Strong hands-on experience with AWS services, particularly AWS Glue (jobs, crawlers, catalogs), S3 (data lake table structures), and RDS databases.

Working knowledge of GCP cloud platform.

Deep experience with Cloudera Hadoop ecosystem: HDFS, Impala, and related components.

Experience with Oracle databases and OLTP transactional systems.

Hands-on experience with Apache Spark for distributed data processing.

Strong understanding of ETL/ELT design, processing, and troubleshooting.

Working knowledge of BI tools such as Tableau and Power BI, including report/dashboard troubleshooting.

Familiarity with Tomcat application server administration.

Strong understanding of Git version control, code review, and pull request workflows.

Deep knowledge of ITIL framework (incident, change, problem management).

Awareness of application security vulnerability management practices.

Excellent troubleshooting, analytical, and communication skills.

Ability to work in on-call rotation supporting production systems, including senior-level escalation ownership.

Preferred Qualifications

Prior experience supporting automotive or manufacturing domain applications.

Experience acting as a release/gate reviewer in a CI/CD environment.

Certifications in AWS, GCP, or Cloudera,.

Familiarity with DevSecOps practices is a plus, including containers (Docker), OpenShift environments, Infrastructure as Code with Terraform, SaaS product management, and CI/CD security integration with automated vulnerability scanning and secrets management.

Understanding of authentication and identity management platforms such as Microsoft Entra ID and IBM Security Access Manager (ISAM) is a plus.