Databricks (Delta Lake, PySpark) AI Dev

Position Overview

The Data & AI Engineer role is responsible for building scalable data pipelines, developing AI agents, and delivering end-to-end data and AI applications on the enterprise data platform, with a primary focus on Databricks.

This role bridges data engineering, AI engineering, and data science to deliver production-grade solutions that power analytics, automation, and intelligent applications across Suntory Beverage & Food’s global operations.

As part of the GHQ AAA team, you are expected to contribute to the organisation’s enterprise AI architecture by delivering reusable, scalable, and high-performance data and AI solutions.

Key Responsibilities

Data Pipeline Development

Design, develop, and maintain

scalable data pipelines

using Databricks (Delta Lake, PySpark, workflows).

Build robust ETL/ELT processes to ingest, transform, and serve data from multiple enterprise sources (SAP, external data, APIs).

Ensure data quality, reliability, and performance optimisation across pipelines.

Implement data models aligned with analytics and AI use cases (e.g., feature-ready datasets).

Collaborate with data governance teams to ensure compliance with enterprise data standards.

AI Agent Development

Design and develop

AI agents and GenAI solutions

(e.g., knowledge assistants, automation agents) using Databricks and Azure AI capabilities.

Implement Retrieval-Augmented Generation (RAG), prompt engineering, and orchestration logic for enterprise AI use cases.

Integrate agents with enterprise data sources, vector databases, and APIs.

Collaborate with business teams to identify and deliver AI-driven automation and productivity use cases.

Ensure scalability, performance, and responsible AI practices in agent deployment.

Databricks App Development

Develop

end-to-end data and AI applications

using Databricks (e.g., notebooks, dashboards, apps, APIs).

Build interactive analytics or AI-driven applications for business users (e.g., demand planning tools, decision support apps).

Expose data and AI services via APIs (e.g., Databricks Model Serving, API integration).

Work with front-end or BI teams (e.g., Power BI, apps) to integrate backend logic into user-facing solutions.

Optimise application performance and ensure scalability for enterprise usage.

Data Science Development

Support development of machine learning models (forecasting, optimisation, classification).

Perform feature engineering and data preparation for modelling.

Collaborate with Data Scientists to productionise models within Databricks environment.

Integrate trained models into pipelines and applications for real-time or batch inference.

Ensure alignment between modelling outputs and business requirements.

Experiences & Skills Requirements

Bachelor’s or Master’s degree in Computer Science, Data Engineering, Data Science, or related field.

1–3 years of experience in data engineering, AI engineering, or related roles.

Strong hands-on experience with

Databricks (Delta Lake, PySpark, workflows, model serving)

.

Proficiency in

Python and SQL

for data processing and application development.

Experience building

data pipelines and ETL processes

in large-scale environments.

Practical experience in

AI/GenAI development (LLMs, RAG, prompt engineering)

.

Familiarity with

API development and integration

for data and AI services.

Understanding of

machine learning concepts and model deployment workflows

.

Experience with

MLOps / DataOps practices

(CI/CD, pipeline automation, monitoring) is preferred.

Knowledge of

Azure ecosystem (ADLS, ADF, Azure AI, API Management)

is a plus.

Strong problem-solving skills and ability to work across cross-functional teams.

Experience in FMCG, supply chain, or commercial analytics is an advantage.

Tata Consultancy Services is an IT services, consulting and business solutions organization that has been partnering with many of the world’s largest businesses in their transformation journeys for over 56 years. Its consulting-led, cognitive powered, portfolio of business, technology and engineering services and solutions is delivered through its unique Location Independent Agile™ delivery model, recognized as a benchmark of excellence in software development.

A part of the Tata group, India's largest multinational business group, TCS has over 601,000 of the world’s best-trained consultants in 55 countries. The company generated consolidated revenues of US $29 billion in the fiscal year ended March 31, 2024.

TCS' proactive stance on climate change and award-winning work with communities across the world have earned it a place in leading sustainability indices such as the MSCI Global Sustainability Index and the FTSE4Good Emerging Index.

At TCS, our values guide our decisions, shape our culture, and inspire us to achieve our shared vision. The TCS Values are integral to the TCS Way.

Leading Change: Embrace innovation and adapt rapidly to changes in technology and business, leading our organization and our customers to outperform and achieve sustainable growth.

Integrity: Be authentic, honest and transparent in all our interactions - building trust and credibility through all our actions.

Respect for Individual: Value the contribution of each and every team member - fostering a collaborative atmosphere, valuing diverse perspectives and ensuring that everyone feels welcome, as we strive and celebrate our success as a team.

Excellence: Be passionate in striving for the highest standards in everything we do, taking accountability for and exceeding the expectations of all our stakeholders.

Learning and Sharing: The world around us is changing rapidly and it is important to stay relevant. We will keep learning, seek new challenges, and take determined action to stay ahead and generously share our collective knowledge.