Databricks Engineer – Contract | Unity Catalog | Databricks Genie | AWS-contract

Databricks Engineer – Contract | Unity Catalog | Databricks Genie | AWS

Job Type: Contract / Temporary
Contract Duration: 3 Months (Extendable)
Experience: 6+ Years
Work Mode: Remote / Offshore
Working Hours: 1:00 PM – 10:00 PM IST
Budget: Up to ₹80,000 per month
No. of Positions: 1

Job Overview

We are looking for an experienced Databricks Engineer for a 3-month contract-based engagement, with the possibility of extension.

The role is part of a Knowledge / Context Layer program supporting Life Sciences Commercial Analytics and Brand Insights for a global pharmaceutical client.

The engineer will work extensively with Databricks, Unity Catalog, Databricks Genie, Collibra, AWS, PySpark, Spark SQL, and Delta Lake, integrating the Databricks environment with an enterprise Knowledge Layer and Agentic AI ecosystem.

Key Responsibilities

• Work with the Databricks platform team to align the Knowledge Layer with the existing Databricks data estate.
• Implement and manage Databricks Unity Catalog, including catalogs, schemas, tables, lineage, external locations, and access controls.
• Configure Databricks Genie, including metric views, certified tables, and grounding logic.
• Build and maintain metadata synchronization between Databricks Unity Catalog and Collibra.
• Extend the Knowledge Layer across brand, customer, activity, market, and performance datasets.
• Work within existing Bronze/Silver/Gold medallion architecture.
• Integrate Databricks with AWS services including Amazon S3 and IAM.
• Support integration with Databricks Vector Search / AI Search, Amazon Neptune, and Amazon Bedrock Knowledge Bases.
• Implement data governance and Unity Catalog security, including row-level and column-level access controls and masking.
• Develop production-grade data pipelines using PySpark, Spark SQL, and Delta Lake.
• Use Databricks REST APIs/SDK for automation and integration.
• Automate deployments and workflows using Databricks Asset Bundles, Terraform, or similar tools.
• Develop Python-based data pipeline and API automation.
• Create technical documentation and reusable Databricks assets.
• Collaborate with GenAI, Data Engineering, Knowledge Engineering, and client platform teams.

Required Skills

• 6+ years of overall Data Engineering experience.
• Minimum 2+ years hands-on Databricks experience.
• Strong experience with Databricks Unity Catalog.
• Hands-on experience with Databricks Genie.
• Strong PySpark, Spark SQL, and Delta Lake experience.
• Strong Python programming skills.
• Experience with AWS S3 and IAM.
• Experience with Databricks REST API/SDK.
• Experience with Collibra and enterprise data catalog integration.
• Experience with data governance, metadata management, business glossary, and KPI/metric definitions.
• Experience with Terraform and/or Databricks Asset Bundles is preferred.

Good to Have

• Life Sciences / Pharmaceutical experience.
• Amazon Neptune / Neo4j.
• Databricks Vector Search.
• Amazon Bedrock Knowledge Bases.
• RAG / Agentic AI / Knowledge Layer experience.
• Ontology / Taxonomy concepts such as RDF, OWL, SKOS.
• MLflow experience.
• Databricks Certified Data Engineer Professional certification.

Contract Details

Employment Type: Contract-Based
Contract Duration: 3 Months
Extension: Possible based on project requirements
Experience: 6+ Years
Working Hours: 1:00 PM – 10:00 PM IST
Budget: Up to ₹80,000/month
Work Mode: Remote / Offshore
Position: 1

Interested candidates can share their updated CV at:
[email protected]

Pay: ₹90,000.00 - ₹100,000.00 per year

Work Location: Remote