Data Engineer, Specialist

Provides advanced data solutions by using software to process, store, and serve data to others. Tests data quality and optimizes data availability. Ensures that data pipelines are scalable, repeatable, and secure. Utilizes a deep dive analytical skillset on a variety of internal and external data.

Role Summary

The Data Engineer, Specialist is responsible for building and maintaining scalable data solutions using Databricks, Python, SQL, and cloud technologies. The role focuses on designing and optimizing data pipelines, implementing end-to-end data models in collaboration with business stakeholders, and deploying reliable data solutions that enable analytics, reporting, and business decision-making.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL processes using Databricks, Python, and SQL to support enterprise data needs.
• Manage end-to-end data modeling in collaboration with business stakeholders, translating business requirements into scalable and efficient data models.
• Build and optimize data ingestion, transformation, and processing workflows across multiple data sources and platforms.
• Develop reusable and maintainable solutions using Python (Functional Programming and Object-Oriented Programming principles).
• Deploy and manage data infrastructure and workflows using Terraform and cloud-native services.
• Work with AWS and/or GCP environments to build scalable and secure data solutions, with adherence to cloud security best practices.
• Partner with engineering teams and business stakeholders to deliver high-quality, trusted datasets that support reporting, analytics, and business initiatives.
• Monitor, troubleshoot, and optimize data pipelines to ensure performance, reliability, and data integrity.
• Contribute to the adoption of modern data engineering practices, automation, and platform enhancements within the organization.

Qualifications and Skills

• Minimum 5 years of relevant work experience, with at least 2 years in designing and maintaining data pipelines, ETL processes and database architectures.
• Bachelor s degree (B.E./B.Tech) in Computer Science or IT, or Bachelor s in Data Science, Statistics, or related field.
• Hands-on experience with Databricks, including development and deployment using Asset Bundles.
• Strong programming expertise in Python, including Functional Programming and Object-Oriented Programming (OOP) concepts.
• Advanced proficiency in SQL for data processing, transformation, optimization, and analytics.
• Experience managing end-to-end data modeling and collaborating with business stakeholders to design effective data solutions.
• Experience working with AWS and/or GCP cloud platforms; understanding of cloud security principles is highly preferred.
• Experience with infrastructure deployment and automation using Terraform.
• Good to Have: Experience with big data technologies, cloud data services, relational and NoSQL databases, and exposure to data architecture modernization, platform migration, and advanced analytics initiatives.

Location

This role is based in Hyderabad, Telangana at Vanguard s India office. Only qualified external applicants will be considered.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.