Senior Data Engineer

The Challenge

The challenge involves leading the design, architecture, and implementation of complex, highly scalable, and reliable end-to-end data systems. This role requires deep expertise in distributed computing and driving the adoption of best practices to ensure data infrastructure operates without manual intervention.

What You ll Take On

• Lead the architectural design and implementation of stable, scalable data pipelines that cleanse, structure, and integrate disparate big data sets into an accessible format for end-user analyses and targeting using stream and batch processing architectures.

• Drive the improvement of the current data architecture, data quality, monitoring, and data availability, collaborating with labels to incorporate new data sources and ensure system reliability.

• Develop and lead the implementation of a comprehensive data quality framework to ensure the delivery of high-quality data and analyses to stakeholders.

• Overall responsibility for maintenance of enterprise wide data dictionary.

• Define, architect, and implement comprehensive monitoring and alerting policies for all mission-critical data solutions.

  What You Bring

• 5+ years of demonstrated experience with SQL

• Demonstrated experience with Python, particularly PySpark on Databricks

• Demonstrated experience with CI/CD systems

• Experience with large data sets and distributed computing. (Spark/Hive/Hadoop)

• Demonstrated expertise in Databricks for large-scale data processing, platform optimization, and pipeline orchestration.

• Ensure the effective integration of data storage, processing, and retrieval components.

• Design and implement fault-tolerant systems to ensure high availability and reliability of data services.

• Skilled in testing and monitoring data for anomalies, with a strong ability to troubleshoot and resolve issues.

• Significant experience working in a global environment, overseeing engineers, collaborating with cross-functional teams, and effectively reporting to managers in different time zones.

• Experienced in testing and monitoring data for anomalies and rectifying them.

• Knowledge of software coding practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations.

• Proven ability to lead projects, mentor junior team members, and drive best practices in coding standards, code reviews, and source management.

  Great to Have:

• Developing solutions in Databricks ecosystems

• Experience with CI/CD tools (e.g., Drone, Jenkins, Github Actions)

• Experience with CDPs or DMPs

• Experience with MCPs

• GDPR / CCPA compliance experience

• Familiarity with Marketing Technologies

• Experience working in an agile environment.

• Previous gaming industry experience

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.