Data Engineer (Spark & Python)

Position: SparkDveloper

Experience: 5 to 8 years

Location: Pan India Opening

Employment Type: Full-Time

Roles & Responsibilities

• Design and develop scalable data processing applications using PySpark and Spark.
• Build and maintain ETL/ELT pipelines for large-scale datasets.
• Process structured and unstructured data from multiple sources.
• Optimize Spark jobs for performance, scalability, and reliability.
• Develop data transformations using Spark SQL and DataFrames.
• Integrate data from cloud and on-premises systems.
• Work with Hadoop, Hive, Databricks, and Data Lake architectures.
• Troubleshoot and resolve performance issues in Spark applications.
• Implement data quality checks and validation processes.
• Collaborate with data engineers, analysts, and business stakeholders.
• Support batch and real-time data processing solutions.
• Maintain technical documentation and coding standards.