Python Pyspark & Automation Engineer

Dear Professionals
Greetings from Tata consultancy Services,
Job Title : Python PySpark & Automation Engineer
Experiernce: 6 to 10 Years
Location: PAN India
Mode of Work : Work from Office
Mode of Interview: Virtual

If you are Interested in the above opportunity kindly share your updated resume to [email protected] immediately with the details below (Mandatory)

Name:
Contact No.
Email id:
Skillset:
Total exp:
Relevant Exp:
Fulltime highest qualification (Year of completion with percentage scored):
Current organization details (Payroll company):
Current CTC:
Expected CTC:
Notice period:
Current location:
Preffered Location
Any gaps between your education or career (If yes pls specify the duration):
Will you be able to join within 30/45 days? (Yes/NO)

Key Responsibilities

Job Description: Python PySpark & Automation Engineer

Role: Python PySpark & Automation Engineer
Experience: 6 to 10 Years
Location: PAN India

Job Summary

We are seeking an experienced Python PySpark & Automation Engineer with strong expertise in designing, developing, and optimizing large-scale data processing solutions. The ideal candidate should have experience in automation frameworks, scripting, cloud platforms, and data engineering practices.

Required Skills

• Python Programming
• PySpark
• Automation Framework Development
• Data Engineering & ETL Processes
• SQL and Database Management
• Shell Scripting
• CI/CD Pipelines (Jenkins, GitHub Actions, Azure DevOps)
• REST API Development & Integration
• Cloud Platforms (AWS/Azure)
• Git Version Control
• Test Automation (PyTest, Robot Framework, Selenium preferred)

Key Responsibilities

• Develop and maintain scalable data pipelines using Python and PySpark.
• Design and implement automation solutions to improve operational efficiency.
• Build reusable automation frameworks and scripts.
• Optimize Spark jobs for performance and scalability.
• Develop and consume REST APIs for system integrations.
• Collaborate with data engineers, developers, and business stakeholders.
• Implement CI/CD pipelines and automation best practices.
• Troubleshoot production issues and provide timely resolutions.
• Ensure code quality through reviews, testing, and documentation.
• Follow Agile development methodologies.

Required Qualifications

• 6 to 10 years of IT experience.
• Strong hands-on experience in Python and PySpark.
• Experience in automation scripting and framework development.
• Strong knowledge of SQL and data processing concepts.
• Experience with AWS or Azure cloud services.
• Familiarity with CI/CD tools and DevOps practices.
• Excellent problem-solving and communication skills.

Preferred Skills

• Databricks
• Airflow
• Docker & Kubernetes
• Selenium Automation
• Terraform
• Machine Learning pipeline automation