Technical Data Engineer

Technical Data Engineer Databricks

Location: Noida
Work Mode: Hybrid
Experience: 5+ Years
Relevant Databricks Experience: 2 to 5 Years
Employment Type: Full-Time

Job Summary

We are looking for a skilled Technical Data Engineer with strong hands-on experience in Databricks, Python, PySpark, SQL, and AWS to design, develop, and maintain scalable data engineering solutions.
The ideal candidate will have experience building production-grade ETL/ELT pipelines, integrating data from databases, Amazon S3, files, and REST APIs, and working with Delta Lake, Unity Catalog, Medallion Architecture, and Databricks Workflows.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, Python, and SQL.
• Integrate data from multiple sources including databases, Amazon S3, files, and REST APIs.
• Build and manage data pipelines using Databricks Unity Catalog.
• Implement business logic, data transformations, and dimensional data models.
• Create, schedule, monitor, troubleshoot, and optimize Databricks Jobs and Workflows.
• Design and manage Delta Lake tables using Bronze, Silver, and Gold layers.
• Implement data quality checks, validations, error handling, logging, and monitoring.
• Optimize Spark workloads for performance, scalability, and reliability.
• Handle batch processing, incremental data loading, and large-volume data processing.
• Collaborate with engineering, business, and cross-functional teams to deliver production-ready data solutions.
• Follow best practices for version control, CI/CD, testing, documentation, and deployment.

Required Technical Skills

• Strong hands-on expertise in Python, PySpark, and Advanced SQL.

• Strong experience with the Databricks Lakehouse Platform.

• Hands-on knowledge of:

• Unity Catalog

• Delta Lake

• Databricks Jobs & Workflows

• Databricks Clusters

• Notebooks

• Repos

• Medallion Architecture

• Experience with REST API integration for data ingestion and data export.

• Strong understanding of ETL/ELT, batch processing, incremental loading, and data transformation.

• Experience with data modeling, including Star Schema, Snowflake Schema, Fact & Dimension tables, and SCD concepts.

• Good understanding of data warehousing concepts and best practices.

• Experience handling structured and semi-structured data such as CSV, JSON, Parquet, and Delta.

• Knowledge of partitioning, file optimization, Spark performance tuning, and query optimization.

• Experience with Git and CI/CD practices.

• Good understanding of data quality, monitoring, troubleshooting, and production support.

Good to Have

• Experience with Databricks Auto Loader.
• Knowledge of Spark Declarative Pipelines.
• Experience with Kafka, Airflow, or dbt.
• Databricks certification.
• Experience working in Agile development environments.

Experience & Qualifications

• 5+ years of overall experience in Data Engineering.
• Minimum 2+ years of hands-on experience with Databricks.
• Strong programming and analytical skills.
• Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field preferred.

What We are Looking For

The ideal candidate should be a hands-on Data Engineer who can independently build, optimize, troubleshoot, and maintain production-grade data pipelines on Databricks.
Strong practical experience in PySpark + SQL + Databricks + AWS + REST APIs is essential.

Location: Noida
Work Mode: Hybrid

Contact -

Call / What's App Resume

9718978697

Email - [email protected]

Manager Global Talent Acquisition

MOPTRA INFOTECH PVT LTD