Senior Software Engineer

*Azure Databricks Engineer*

Experience: 5+ Years
Location: Hybrid / Only Hyderabad
Employment Type: Full-Time

Job application: https://forms.gle/rePvD4UcB4dJtMYj9

Job Summary:

We are looking for an experienced Azure Databricks Engineer with strong hands-on expertise in developing scalable data pipelines and data engineering solutions using Azure Databricks, PySpark, Python, and SQL.

Key Responsibilities:

• Develop and maintain scalable data pipelines using Azure Databricks and PySpark.
• Design and implement ETL/ELT workflows for large-scale data processing.
• Build data solutions using Delta Lake, Azure Data Lake Storage (ADLS), and Azure Data Factory (ADF).
• Perform data transformation, cleansing, aggregation, and optimization using PySpark and Spark SQL.
• Develop and optimize Databricks notebooks, jobs, workflows, and clusters.
• Implement data ingestion from various sources and formats including CSV, JSON, Parquet, and Delta.
• Troubleshoot and optimize data pipelines for performance, reliability, and scalability.
• Collaborate with data engineers, architects, and business teams to deliver data solutions.
• Follow best practices for Git, version control, deployment, and data security.

Required Skills:

• 5+ years of experience in Data Engineering.
• Strong hands-on experience with Azure Databricks and PySpark.
• Strong proficiency in Python and SQL.
• Experience with Azure Data Factory (ADF) and ADLS.
• Strong understanding of ETL/ELT and data pipeline development.
• Experience with Delta Lake and Spark SQL.
• Good knowledge of data warehousing concepts.
• Experience with Git and CI/CD is preferred.
• Strong analytical and problem-solving skills.

Good to Have:

• Experience with Snowflake.
• Knowledge of Kafka / Spark Structured Streaming.
• Experience with Azure DevOps.
• Microsoft Azure Data Engineer certification.

Notice Period: Immediate Joiners Preferred

Work Location: Hybrid remote in Hyderabad, Telangana (Hyderabad, Hyderabad District)