Sr Software Engineer - Data Engineering

Job Title: Sr Software Engineer - Data Engineering

Experience: 6- 13 Years
Location: Hyderabad
Employment Type: Full-Time

Job Description

We are looking for an experienced Senior Software Engineer Data Engineering with strong hands-on expertise in AWS Glue, PySpark, Python, SQL, and cloud-based data engineering. The ideal candidate should have experience building scalable ETL/ELT pipelines, handling batch and streaming data, and working with multiple databases and AWS services.

Mandatory Skills:

• Strong hands-on experience with AWS Glue and PySpark.
• Proficiency in Python and SQL, including stored procedures.
• Experience with Oracle, MongoDB, and DynamoDB.
• Working knowledge of AWS ECS and Terraform.
• Experience building batch and streaming data pipelines.
• Knowledge of error handling, retry/replay mechanisms, and data quality validation.
• Understanding of AI/LLMs, prompt engineering, and AI-assisted development tools such as Cursor, Claude, or Devin AI.
• Ability to review AI-generated code and follow secure coding practices.

Key Responsibilities:

• Design, develop, and maintain scalable ETL/ELT pipelines using AWS Glue and PySpark.
• Build data integration workflows using AWS Lambda and Step Functions.
• Develop and optimize SQL queries and stored procedures.
• Work with relational and NoSQL databases, including Oracle, MongoDB, DynamoDB, and Aurora PostgreSQL.
• Implement data quality checks, error handling, retry mechanisms, and data replay processes.
• Support CI/CD automation using GitHub Actions or Jenkins.
• Troubleshoot production issues and optimize pipeline performance.
• Follow AWS security best practices, including IAM permissions and encryption.
• Leverage AI-assisted development tools responsibly to improve engineering productivity.

Preferred Skills:

• Healthcare payer domain experience.
• Knowledge of HIPAA, HL7, FHIR, or X12.
• Exposure to Amazon Athena, Redshift, Kinesis, or EMR.
• AWS Data Engineer or Analytics certifications.

Interested candidates can your resume with [email protected]