Lead I - Software Engineering_Java spark

Role description

Detailed Job Description

Job Responsibilities:

• Responsible for ingesting data from multiple sources into a Data Lake using Java-based tools and frameworks such as Apache Spark, Spring Batch, and connectors for relational and NoSQL databases.

• Write transformation logic using Spark (Java API) and store processed data in HDFS or cloud storage solutions (e.g., AWS S3, GCP Storage, or on-prem equivalents).

• Responsible for writing stored procedures for relational databases (e.g., PostgreSQL, MySQL, or Oracle).

• Develop/re-develop existing ETL code and workflows into the modernized data pipeline.

• Experience with Batch and Realtime data processing and transformations.

  Minimum Job Requirements:

• Must have experience in writing SQL queries and Spark code using Java.

• Must have experience with Java frameworks for data processing and integration (e.g., Spring, Apache Spark, JDBC).

• Should be familiar with Data Warehousing concepts and data modeling techniques like star schema.

  Preferred Job Requirements:

• Good to have knowledge of Delta Lake or similar transactional storage formats in Spark.

• Knowledge of various Slowly Changing Dimensions (SCD) types.

• Basics of workflow orchestration tools (e.g., Apache Airflow, Oozie) and trigger mechanisms.

• Strong client communication skills.

  Educational Requirement - BE/BTech

  Experience Level (Min-Max) - 5+ years

Skills

java,etl,sql,aws,data processing,

About UST
UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world’s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients’ organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact—touching billions of lives in the process.