Senior Pyspark Developer

We are looking for a skilled Senior Pyspark Developer with 4 to 9 years of experience to join our team at Luxoft India LLP. The ideal candidate will have expertise in developing scalable data processing pipelines using Pyspark.

Roles and Responsibility

• Design, develop, and deploy large-scale data processing pipelines using Pyspark.
• Collaborate with cross-functional teams to identify business requirements and design solutions.
• Develop and maintain complex SQL queries for data extraction and transformation.
• Troubleshoot and optimize performance issues in existing data pipelines.
• Implement automated testing and deployment scripts using tools like Jenkins or GitLab CI/CD.
• Participate in code reviews to ensure high-quality code standards.

Job Requirements

• Strong proficiency in Python programming language and its ecosystem.
• Experience with big data technologies such as Hadoop, Spark, and NoSQL databases.
• Knowledge of cloud-based platforms such as AWS or GCP.
• Familiarity with agile development methodologies and version control systems like Git.
• Excellent problem-solving skills and attention to detail.
• Ability to work collaboratively in a team environment and communicate effectively with stakeholders.