Python PySpark Developer

Job Description: Python + PySpark + SQL Developer

Job Title: Python / PySpark / SQL Developer

Experience: 4-9 Years (customize as needed)

Location: Chennai/Pune/Mumbai/Bangalore/Hyderabad

Job Summary

We are seeking a skilled Python, PySpark, and SQL Developer to design, develop, and optimize scalable data processing solutions. The ideal candidate should have strong experience in data engineering, ETL development, big data technologies, and database management while working with large-scale datasets.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Python and PySpark.
• Develop and optimize complex SQL queries, stored procedures, and database objects.
• Build, automate, and support ETL/ELT processes for data integration and transformation.
• Process and analyze large volumes of structured and unstructured data using Apache Spark.
• Perform data cleansing, validation, and transformation to ensure data quality.
• Monitor, troubleshoot, and optimize data processing jobs for performance and reliability.
• Collaborate with data analysts, data engineers, and business stakeholders to understand requirements.
• Implement data security, governance, and best practices for data management.
• Work with cloud platforms and distributed computing environments.
• Create technical documentation and support deployment activities.

Required Skills

• Strong proficiency in Python programming.
• Hands-on experience with PySpark and Apache Spark.
• Strong knowledge of SQL and database concepts.
• Experience in developing and maintaining ETL pipelines.
• Understanding of data warehousing concepts and dimensional modeling.
• Knowledge of performance tuning and query optimization.
• Familiarity with version control tools such as Git.
• Experience working with Linux/Unix environments.
• Strong problem-solving and analytical skills.

Preferred Skills

• Experience with cloud platforms such as Azure, AWS, or Google Cloud Platform (GCP).
• Knowledge of Databricks, Azure Data Factory (ADF), Azure Synapse, or similar data platforms.
• Experience with workflow orchestration tools such as Apache Airflow.
• Exposure to CI/CD pipelines and DevOps practices.
• Knowledge of data lake and data warehouse architectures.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or related field.
• Relevant experience in Python, PySpark, and SQL development.
• Certifications in Azure, Databricks, Spark, or cloud technologies are an added advantage.

Nice-to-Have Technologies

• Databricks
• Azure Data Factory (ADF)
• Azure Synapse Analytics
• Power BI
• Delta Lake
• Apache Kafka
• Airflow
• Snowflake