Python-Spark Scala Professional

Educational Requirements

BTech, BCA, BSc, MSc, MCA, MTech

Service Line

Data Analytics Unit

Responsibilities

• 59 years of experience in Data Engineering / Big Data development.
• Strong hands-on experience with Python and Apache Spark (Scala PySpark).
• Deep understanding of Spark architecture (RDD, DataFrames, Spark SQL).
• Experience in distributed computing and big data processing.
• Strong knowledge of SQL and data modeling concepts.
• Experience with data pipeline development (ETL/ELT).
• Familiarity with Linux/Unix environments.
• Experience with version control tools (Git).

Additional Responsibilities

• Experience with cloud platforms (AWS, Azure, or GCP).
• Hands-on with Databricks / EMR / Spark clusters.
• Knowledge of streaming technologies (Kafka, Spark Streaming, Structured Streaming).
• Experience with workflow orchestration tools (Airflow, Oozie).
• Familiarity with Delta Lake / Lakehouse architecture.
• Exposure to NoSQL databases (MongoDB, Cassandra).
• Knowledge of CI/CD and DevOps practices.

Technical and Professional Requirements

• Primary skills:Technology- >Big Data - Data Processing- >Spark,Technology- >Functional Programming- >Scala,Technology- >Machine Learning- >Python

Preferred Skills

• Technology- >Functional Programming- >Scala
• Technology- >Big Data - Data Processing- >Spark- >SparkSQL
• Technology- >AI-Data science- >PYTHON