Sr Data Engineer

Job Summary

We are seeking a highly experienced Senior Data Engineer to join our team, with a primary focus on database architecture, optimization, and management. The ideal candidate will have at least 4+ years of robust, hands-on experience in PostgreSQL (preferably AWS RDS), advanced SQL, and database development, as well as deep expertise in designing and managing ETL data pipelines. Experience with both batch and event-driven data processing is essential. Strong Python skills for data engineering, and familiarity with AWS Glue, AWS Batch, AWS S3, CI/CD tools (Jenkins, Git), and AI-related technologies are also required.

What you will do

• Architect, develop, and maintain robust database structures (tables, views, procedures, functions, packages) with a focus on PostgreSQL (AWS RDS).
• Optimize database performance, write and debug complex SQL queries, and ensure reliability and scalability of the database layer.
• Design, build, and manage ETL pipelines using Python and AWS Glue/Batch, integrating with AWS S3 for data storage and retrieval.
• Develop and maintain procedures, packages, and scripts for data migration and processing large volumes of structured and unstructured data.
• Implement both batch and event-driven data processing architectures.
• Collaborate with stakeholders to deliver secure, high-quality data solutions that meet business requirements.
• Implement and maintain CI/CD pipelines, automate deployments with Jenkins, and manage code repositories using Git.
• Support AI and machine learning initiatives by preparing, processing, and managing data for model training and inference.
• Monitor, troubleshoot, and resolve data engineering and database issues as they arise.

What you will need

Strong IT professional with 4-6 yrs of experience in software development for data engineering using Python, AWS. The candidate should have strong problem-solving skills along with high on ownership and accountability.

Must have

• 4+ years of robust, hands-on experience in data engineering, with a strong focus on PostgreSQL (AWS RDS).
• Proven expertise in architecting, optimizing, and maintaining complex database solutions using PostgreSQL.
• Extensive experience developing and maintaining database objects (tables, views, procedures, functions, packages).
• Deep proficiency in designing and managing ETL data pipelines, leveraging Python and AWS Glue/Batch.
• Demonstrated experience with both batch and event-driven data processing architectures.
• Hands-on experience with AWS S3 for data storage and integration.
• Experience with data migration and processing of large datasets (structured and unstructured).
• Familiarity with CI/CD pipelines, Jenkins, and Git for automated deployments and version control.
• Strong problem-solving, communication, and collaboration skills.

Who you are

• Bachelor's degree or foreign equivalent degree in Computer Science or a related field required
• Excellent communication and prioritization skills.
• Able to work independently or within a team proactively in a fast-paced Agile-Scrum environment.
• Owns success - Takes responsibility for successful delivery of the solutions.
• Strong desire to improve upon their skills in software development, frameworks, and related areas.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.