Big Data Engineer

Role & responsibilities

Must Have

• 6+ years data engineering experience

• Strong programming skills in Python

• Hands on experience with Big Data tech (e.g., Hadoop) and Spark

• Comfort working in Agile, highly collaborative teams

• Strong written and verbal communication skills

Good to have

• Experience with cloud platforms (AWS EMR, S3 Lamda, or Azure), Terraform (provisioning and Infra), Scala, Java, Databricks

• Performance tuning and code optimization experience

Must Have

8+yrs

• PySpark: Experience with Apache Spark using the PySpark for distributed data processing, including DataFrame operations, window functions, and Spark SQL.

• Python Programming: Strong knowledge of Python, including advanced features and best practices.

• AWS Services: Familiarity with AWS S3 (for data storage and access), EMR, Lambda, SQS, SNS, IAM etc and Boto3 (AWS SDK for Python).

• Data Engineering: Understanding of ETL processes, schema design, and data validation.

• SQL: Proficiency in writing and optimizing SQL queries, especially within Spark SQL

• Strong analytical and Problem solving skills

Preferred / Ideal to have

• Experience with other AWS services - SQS, SNS, IAM etc

• Infrastructure as Code (IaC) with tools like Terraform

• Experience with Git and branching strategies and Code quality tools (SonarQube)

• RESTful API development and documentation (e.g., Swagger/OpenAPI/Fast API)

• Data pipeline orchestration (Airflow, Step Functions)

• Scripting for automation (Bash, Python, PowerShell)

Good to have

• Any knowledge on Databricks is added advantage

• Data Quality and Validation Frameworks

WFO all 5days

Bangalore/Hyderabad

F2F interview

6+yrs

Immediate only

[email protected]