Data Engineer (Databricks Specialist)_Virtual drive

Data Engineer (Databricks Specialist)

Exp: 6 to 12 years

Location: Hyderabad Only

Interview Mode: Virtual interview in Week days

Role & responsibilities

Desired Competencies (Technical/Behavioral Competency)

Must-Have**

(Ideally should not be more than 3-5)

• Databricks & Spark: Hands-on experience with Databricks, Apache Spark, and PySpark for big data processing.
• Cloud Platforms: Proficiency in Azure (preferred), AWS, or GCP for data engineering workflows
• SQL & Data Modeling: Strong knowledge of SQL, schema design, and dimensional data modeling.
• ETL/ELT Tools: Experience with tools like Azure Data Factory, Informatica, or Talend.
• Data Warehousing: Familiarity with technologies such as Azure Synapse, Snowflake, or BigQuery.
• Strong problem-solving and debugging skills.
• Excellent communication and teamwork abilities.
• Ability to work in a fast-paced, dynamic environment.

Good-to-Have

• Knowledge of Event Hub, IoT Hub, and streaming analytics.
• Exposure to Power BI, Azure DevOps, and CI/CD pipelines.
• Understanding of machine learning libraries (MLlib) and predictive analytics.

SN

Responsibility of / Expectations from the Role

1Design and maintain scalable data pipelines using Databricks and PySpark.

2Implement and optimize ETL processes for large datasets.

3Collaborate with cross-functional teams to define data requirements.

4Ensure data integrity and quality across the lifecycle.

5Integrate solutions with cloud platforms and manage large-scale datasets.

Preferred candidate profile