Data Engineer

In this role, you will design and develop scalable ETL/ELT pipelines using Databricks and Snowflake. You will be responsible for architecting data lakehouse solutions following Medallion architecture and implementing Lambda patterns for batch and real-time processing. The ideal candidate will build and maintain a single source of truth across enterprise platforms while optimizing data models and query performance. You will collaborate with cross-functional teams to translate business requirements into reliable technical solutions.

Required Skills and Qualifications:

• Proficiency in designing and developing ETL/ELT pipelines using Databricks (PySpark/Spark SQL) and Snowflake
• Experience architecting data lakehouse solutions (Bronze, Silver, Gold layers)
• Ability to implement Lambda architecture patterns for batch and real-time data processing
• Skilled in developing data ingestion frameworks from databases, APIs, and streaming platforms
• Experience with orchestration tools such as Airflow, ADF, or Databricks Workflows
• Knowledge of CI/CD practices for data pipelines and infrastructure-as-code
• Familiarity with data quality frameworks and pipeline monitoring.

Roles and Responsibilities

• Experience with Delta Live Tables (DLT) or Unity Catalog
• Knowledge of Informatica, Streamsets, or other industry-standard ETL tools
• Familiarity with data governance and cataloging tools
• Understanding of security and compliance standards in data engineering