Data Engineer

Job Title: Senior Data Engineer

Duration: 6 months contract to Perm

Location: New York, NY 10001, (onsite)

Tech Stack: Python, Java, Spark, AWS, Kubernetes, Airflow, Glue, EMR, Athena, Kinesis, Parquet, Iceberg, CI/CD.

About the Role

We are seeking a Lead Software Engineer – Data Engineering to design and build scalable, cloud-native data platforms and pipelines. You will lead the development of robust data solutions, drive architectural decisions, and mentor engineers while ensuring reliability, security, and operational excellence.

Responsibilities

• Design and develop large-scale data pipelines and data platforms.
• Build cloud-native solutions using AWS services and Spark.
• Lead data modeling, warehousing, migration, and analytics initiatives.
• Improve system performance, resilience, monitoring, and automation.
• Collaborate with cross-functional teams and mentor junior engineers.

Required Qualifications

• Strong proficiency in Python and/or Java with Spark/PySpark.
• Experience with Data Pipelines, Data Modeling, Data Warehousing, and Data Migration.
• Hands-on experience with AWS (Glue, EMR, S3, Athena, Kinesis/MSK, Airflow, Lambda, Step Functions, Lake Formation).
• Knowledge of Kubernetes, CI/CD, Agile methodologies, and software design principles.
• Experience with data formats such as Parquet, Iceberg, Avro, or JSON.
• Understanding of AWS security concepts including IAM, KMS, Encryption, and Secrets Manager.

Preferred Qualifications

• Experience building REST APIs using Java.
• Knowledge of Hybrid Cloud architectures.
• Exposure to Jenkins, Git, Linux Scripting, JIRA, and TDD.
• Strong leadership, problem-solving, and communication skills.