Senior Data Engineer (TILDI)

• Analyze complex business requirements and design scalable, high-quality data solutions that align with long-term data platform strategy.

• Design and implement highly scalable, efficient data pipelines using data lake/data lakehouse architectures.

• Define, develop, and maintain data validation and data quality frameworks across technical and business rules.

• Lead the design and implementation of key data management services (data quality, monitoring, security, governance) within the DataOps framework.

• Monitor and ensure reliability, scalability, and performance of data pipelines and data platform clusters (e.g., Kubernetes, container orchestration).

• Collaborate effectively with cross-functional teams (SRE, Security, Data Science, AI/ML Engineering, Business teams, and leadership).

• Lead large-scale or complex projects end-to-end, either as a technical expert or as a project lead managing multiple contributors.

• Define engineering standards, best practices, design patterns, and development guidelines for the data engineering team.

• Review designs and solutions from Mid-level and Junior team members to ensure quality and alignment with standards.

• Mentor, coach, and guide team members to support their technical and professional growth.

• Influence key stakeholders and communicate complex technical concepts to both technical and non-technical audiences, including executives.

Qualifications:

• 5+ years of experience designing and implementing large-scale or complex data pipelines.

• Extensive experience with cloud platforms (GCP preferred).

• Strong expertise in Apache Spark, Apache Airflow, and distributed data processing.

• Proven experience leading or managing multiple concurrent projects.

• Strong proficiency in SQL and Python with deep understanding of software engineering best practices.

• Experience with containerization and orchestration technologies (Docker, Kubernetes, Helm).

• Strong understanding of coding standards, testing methodologies, and code review processes.

• Growth-oriented mindset and ability to adopt new tools, frameworks, and technologies.

• Understanding of data management concepts (data lineage, observability, data mesh, data governance, data security).

• Excellent communication, documentation, and stakeholder management skills.