Senior Data Engineer (AI Data Platforms & Automation)

We are looking for an experienced and hands-on Senior Data Engineer to build scalable data platforms and AI-ready data pipelines. The ideal candidate will be responsible for designing data architectures, metadata frameworks, ETL pipelines, and automation that support AI and data initiatives across the organization.

Key Responsibilities

Data Engineering

• Design and develop scalable ETL/ELT pipelines.

• Build robust data ingestion frameworks.

• Design enterprise data models.

• Implement data transformation and validation processes.

• Optimize data storage and retrieval.

AI Data Platforms

Build AI-ready datasets.

Design metadata frameworks and taxonomies.

Develop dataset versioning strategies.

Manage structured and unstructured datasets.

Support AI model training and evaluation pipelines.

Data Quality & Governance

Implement data quality checks.

Build validation and monitoring frameworks.

Define data standards and best practices.

Ensure data integrity and consistency.

Develop documentation and data lineage.

Automation

Automate repetitive data workflows.

Build Python-based automation tools.

Develop integrations with internal and external systems.

Create dashboards and monitoring utilities.

Platform Engineering

Work with SQL and NoSQL databases.

Optimize database performance.

Manage cloud-based data platforms.

Support scalable enterprise data architecture.

Required Technical Skills

Programming

Python (Expert)

SQL (Expert)

PySpark (Preferred)

Databases

PostgreSQL

MySQL

MongoDB

BigQuery / Snowflake

Vector databases (Preferred)

Data Engineering

ETL/ELT Development

Data Warehousing

Data Modeling

Metadata Management

Data Quality Frameworks

Cloud

Google Cloud Platform or AWS

Cloud Storage

BigQuery / Redshift

Airflow (Preferred)

Tools

Docker

Git

CI/CD

API Integration

Preferred Experience

AI data platforms

Metadata management

Dataset curation

Workflow automation

Enterprise data engineering

Cloud-native data platforms

AI data preparation

Interested candidates can share resume at [email protected] or call at 8595125050