Senior Data Engineer

Job Summary

We are looking for a skilled Data Engineer to design, build, and maintain scalable data pipelines and data processing systems. The ideal candidate should have strong experience in Python, SQL, ETL/ELT, data warehousing, and cloud technologies, with the ability to work with large and complex datasets.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines.
• Build and optimize data pipelines for structured and unstructured data.
• Develop efficient and complex SQL queries for data extraction and transformation.
• Work with data warehouses, data lakes, and relational/non-relational databases.
• Perform data cleaning, validation, transformation, and quality checks.
• Optimize data pipelines and database performance.
• Collaborate with Data Scientists, Business Analysts, Software Engineers, and other stakeholders.
• Implement data integration solutions from multiple sources.
• Monitor data pipelines and troubleshoot data processing issues.
• Ensure data security, reliability, and availability.
• Document data pipelines, workflows, and technical processes.

Required Skills

• Strong programming experience in Python.
• Strong knowledge of SQL and database concepts.
• Hands-on experience with ETL/ELT processes and data pipelines.
• Experience with Apache Spark / PySpark.
• Experience with relational databases such as MySQL/PostgreSQL.
• Knowledge of Data Warehousing concepts.
• Experience with cloud platforms such as AWS / Azure / GCP.
• Familiarity with Airflow or other workflow orchestration tools.
• Understanding of data modeling, performance optimization, and data quality.
• Good problem-solving and analytical skills.

Good to Have

• Experience with Kafka or other streaming technologies.
• Experience with Databricks.
• Knowledge of AWS Glue, S3, Redshift or equivalent cloud data services.
• Experience with CI/CD and Git.
• Exposure to Docker/Kubernetes.
• Experience working with real-time data pipelines.

Pay: From ₹100,000.00 per year

Benefits:

• Flexible schedule
• Provident Fund

Work Location: Remote