Data Engineer- Databricks + AI

Position: Data Engineer- Databricks + AI

Purpose of the Position:

Build and enhance AI Agents and modern pipelines leveraging Databricks platform features and services.

Location: Nagpur/ Pune/ Bangalore/ Chennai

Key Responsibilities:

• Work with account/project architects to understand and translate requirements into technical implementations following the AI First strategy.
• Develop, implement, and maintain Agents and pipelines using python/pyspark within Databricks eco system by using the native support.
• Code, test, and document new or modified data systems to create robust and scalable applications for data analytics.

Roles and Responsibilities

Work and Technical Experience:

Mandatory Skills:

• 6+ years of overall experience with 4+ years of hands-on Databricks experience, PySpark and Python
• Understanding of Databricks Agent framework Agent Bricks. Built RAG pipelines using langchain
• Capability to build Genie Agent Chatbot and fine tuning
• Worked on complex business processes and implementations of Big Data Projects
• Understand end to end lakehouse build up on Databricks and usage of Genie code.
• Thorough understanding of Databricks Unity Catalog, Notebook, Autoloader, Data apps, Genie and other key services/features
• Proficiency in SQL and data manipulation skills
• Understanding of data warehousing concepts and technologies.

Desirable Skills:

• Understanding of AWS platform and its key services like S3, Lambda etc.
• Familiarity with ETL processes and tools for data extraction, transformation, and loading.
• Familiarity with AI/BI dashboards in Databricks, Agentic workflows in Databricks

Qualifications:

• Bachelor’s degree in computer science, engineering, or related field
• Demonstrated continued learning through one or more technical certifications or related methods
• 6+ years of relevant experience in Data Engineering
• Relevant experience in Retail domain

Qualities:

• Self-motivated and focused on delivering outcomes for a fast-growing team and firm
• Able to communicate persuasively through speaking, writing, and client presentations
• Able to consult, write, and present persuasively
• Able to work in a self-organized and cross-functional team
• Able to iterate based on new information, peer reviews, and feedback
• Able to work with teams and clients in different time zones
• Research focused mindset