Technical Lead – PySpark / Pune /

Technical Lead – PySpark

Location: Pune

Experience: 5–8 Years

Primary Skill: PySpark

Job Description

We are looking for a Technical Lead – PySpark with strong hands-on experience in building and optimizing scalable data processing solutions. The ideal candidate should have solid expertise in PySpark, Python, SQL, and data engineering , along with the ability to provide technical guidance to the development team.

Key Responsibilities

• Design, develop, and maintain scalable data processing solutions using PySpark .
• Build and optimize ETL/data pipelines for large datasets.
• Perform performance tuning and optimization of Spark/PySpark jobs.
• Troubleshoot complex data processing and application issues.
• Conduct code reviews and ensure development best practices.
• Provide technical guidance and mentoring to team members.
• Collaborate with cross-functional teams to translate requirements into technical solutions.
• Ensure solutions are scalable, reliable, and maintainable.

Required Skills

• Strong hands-on development experience in PySpark .
• Strong programming skills in Python .
• Good understanding of Apache Spark and distributed data processing.
• Strong knowledge of SQL and data transformations.
• Hands-on experience developing ETL/data pipelines .
• Experience in Spark/PySpark performance optimization.
• Good problem-solving and troubleshooting skills.
• Experience in technical leadership, code reviews, and team guidance.

Preferred Skills

• Exposure to AWS, Azure, or GCP data platforms.
• Understanding of big-data and modern data-engineering ecosystems.
• Familiarity with Git, CI/CD, and software development best practices.