Senior Data Engineer

Job Summary

This role helps extend and evolve digital products that support newsroom workflows, content discovery, and audience engagement.

Responsibilities

• Design and Develop: Develop robust data ingestion, transformation, and processing frameworks using technologies such as Databricks, Spark, Kafka, Airflow, DBT, and cloud-native services. Build and optimize data models, lakehouse architectures, and data products that serve analytics, reporting, machine learning, and operational use cases.
• Collaboration: Partner closely with product managers, analysts, data scientists, and software engineers to understand business requirements and translate them into scalable data solutions.
• Security and Governance: Ensure all data systems adhere to security, privacy, and governance requirements. Implement best practices around access controls, encryption, data lineage, and compliance.
• Documentation and Quality: Create and maintain technical design documents, data flow diagrams, source-to-target mappings, and operational runbooks. Develop comprehensive testing strategies and automation to ensure reliability and correctness of data products.

Requirements

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field with at least 4+ years of experience building large-scale data engineering solutions and data platforms.

Expertise

• Databricks, Snowflake, or modern cloud-native data ecosystems.
• Python, PySpark, SQL, and distributed data processing frameworks such as Apache Spark or Flink.
• Experience building both batch and streaming data pipelines using technologies such as Kafka, Kinesis, Spark Structured Streaming, or equivalent.
• Strong understanding of dimensional modeling, data warehousing, and lakehouse architecture patterns.
• Experience with workflow orchestration platforms such as Airflow or Astronomer.
• Experience working with AWS services such as S3, Lambda, Glue, DynamoDB, Redshift, EMR, and related cloud technologies.

Skills Competencies

• Problem Solving: Strong analytical and troubleshooting skills with the ability to identify bottlenecks, optimize performance, and solve complex data challenges at scale.
• Design: Experience designing scalable, resilient, and observable data systems with a strong focus on reliability, performance, and maintainability.
• Data and AI Enablement: Experience building data foundations that support advanced analytics, machine learning, and AI-driven applications. Exposure to modern data and AI platforms such as Databricks, MLflow, SageMaker, Vertex AI, or similar ecosystems is highly desirable.
• Ownership and Accountability: Self-driven mindset with a strong sense of ownership. Ability to take initiatives from concept through production deployment and ongoing operational support.
• Communication: Excellent communication and collaboration skills with the ability to work effectively across technical and non-technical stakeholders.

Good to Have

• Experience with containerization and orchestration technologies such as Docker and Kubernetes.
• Experience working with semi-structured or unstructured datasets and NoSQL databases.
• Experience building APIs and data services.
• Knowledge of data governance, cataloging, lineage, and metadata management solutions.
• Open-source contributions or active participation in the data engineering community.
• Interest in emerging technologies including AI, machine learning, and modern data platform architectures.

Disclaimer: This job description has been sourced from a public domain and may have been modified by Naukri.com to improve clarity for our users. We encourage job seekers to verify all details directly with the employer via their official channels before applying.