Platform Data Engineer

Job Title – Platform Data Engineer

Company – TCS (MEA)

Location – Johannesburg, South Africa

Job type – Full time

About Us:

Tata Consultancy Services (TCS) is an IT services, consulting and business solutions organization that has been partnering with many of the world’s largest businesses in their transformation journeys for over 50 years. TCS offers a consulting-led, cognitive powered, integrated portfolio of business, technology and engineering services and solutions. This is delivered through its unique Location Independent Agile™ delivery model, recognized as a benchmark of excellence in software development.

A part of the Tata group, India's largest multinational business group, TCS has over 616,171 of the world’s best-trained consultants with 157 nationalities in 53 countries. For more information, visit www.tcs.com and follow TCS news at @TCS_News.

Job Description:

Description

We are looking for an experienced Platform Data Engineer for their Save & Open API team in the CDX Division. The role involves both data engineering and Azure platform engineering functions. The candidate will be responsible for managing all stages of the data engineering process, including data ingestion, transformation, modelling, and warehousing, as well as designing, building, and maintaining the Azure data platform infrastructure and services required for creating data products for reporting and analytics. Additionally, the candidate will contribute towards enhance the One Data Platform (ODP) using their Azure Engineering knowledge, in line with the Data Strategy.

The role will include:

• Translating business, data, architecture designs, requirements/models into technical implementations.
• Building dynamic meta-data driven data ingestion patterns using Azure Data Factory and Databricks.
• Building and maintaining the Enterprise Data Warehouse (using Kimball methodology)
• Building and modelling source and business focused data products in Databricks and Microsoft Fabric.
• Implementing and delivering to all stages of the data engineering using Investec Data Experience (IDX) templates into ODP (One data platform).
• Implementing DevOps efficiencies where possible using a combination of third-party SaaS and PaaS products and custom developed tools.
• Implementing unit tests, perform integration testing, and debug applications to ensure code quality, identify and fix issues, and improve overall reliability pre- and post-deployment.
• Designing and managing of Azure templates and components to deploy out data related Azure technologies.
• Develop, implement, and maintain relevant documentation, guidelines, checklists, and policies to promote continuous integration, ensure and improve data security, and reduce the possibility of “human error.
• Sharing support and operational duties within the team.
• Working closely with end-users to understand their business and their data requirements.
• Adopting the Data Governance and Group Engineering practices and principles and applying that proactively to delivery high-quality data products for consumption by business and technology partners
• Active participation and contribution to the Group Data Engineering and Modeling practices

Experience, skill and capability

Experience in the following technologies is key to this role:

• Knowledge of distributed data mesh deployment methodology.
• 6+ years of experience as a platform and data engineer.
• Hands-on experience with Microsoft Fabric (Lakehouse, Warehouse, Data Pipelines, notebooks, and semantic models).
• Strong understanding of event streaming architectures using Azure Event Hubs.
• Hands-on experience with data governance frameworks and tooling, specifically Databricks Unity Catalog and/or Microsoft Purview.
• Experience designing and maintaining real-time data processing solutions with Azure Stream Analytics.
• Experience in building robust and performant ETL processes.
• Excellent data analysis and exploration using T-SQL
• Demonstrated expertise in developing and optimizing large-scale data processing applications using Apache Spark. Proficient in writing complex Spark jobs to process and analyze vast datasets efficiently.
• Skilled in leveraging Pandas for data manipulation and analysis tasks. Capable of transforming trusted data into actionable insights by utilizing Pandas' powerful data structures and functions.
• Knowledge and experience of data warehouse modelling methodologies (Data Vault 2.0, Kimball)
• Experience in Azure – one or more of the following: Microsoft Fabric, Azure Data Factory, Azure Event Hubs, Databricks, Azure Synapse Analytics, ADLS Gen2 and Azure Stream Analytics.
• Experience in Azure DevOps and managing source code repositories, building and releasing applications, and implementing continuous integration and delivery processes (CI/CD) and infrastructure automation, including automated deployment pipelines using ‘infrastructure-as-code’.
• Experienced in the Agile Methodology ways of work
• Understanding of web application security concepts, such as Azure Active Directory (Azure AD) and the ability to integrate, authentication and authorization mechanisms into applications using Azure AD, data validation, and protection against common vulnerabilities.
• Experience in Infrastructure as Code (IaC) - scripting and automation via Azure CLI, PowerShell, Bash, Bicep and JSON ARM and Bicep templates.
• Understanding and experience in Azure bicep templates for automating deployment of pipelines.
• C#, Python and SQL (stored procedures, functions)
• Effective communication and collaboration skills to work in cross-functional teams, participate in code reviews, and communicate technical concepts to non-technical stakeholders.
• Willingness to learn new technologies, keep up with industry trends, and adapt to evolving development practices and tools.
• Experience in Banking and Financial Services
• Experience in Open Banking technologies

Required Qualifications

• Bachelor’s degree in computer science or related field from an accredited university and/or equivalent work experience
• Microsoft Azure certifications (Mandatory AZ900, or DP600 or DP700 or DP203), Adventurous (Databricks Fundamentals, Databricks Data Engineer Associate or Databricks Platform Architect)

Application Deadline: 30-October-2026

Privacy Note:

https://www.tcs.com/connect-with-tcs/privacy-policy