Opening For Data Engineer Databricks

Key Responsibilities

• ETL Pipeline Development: Design, build, and maintain batch and streaming data pipelines using Apache Spark, PySpark, and Delta Lake.
• Lakehouse Architecture: Implement medallion architecture (Bronze, Silver, Gold layers) to process data incrementally using tools like Delta Live Tables (DLT).
• Orchestration & Automation: Schedule and monitor data workflows using Databricks Jobs and Workflows.
• Data Governance & Security: Enforce data access policies, lineage, and security compliance utilizing Unity Catalog.
• Performance Tuning: Optimize Spark clusters, query performance, and infrastructure costs within cloud environments