Scala

Job Requirements*

5+ years of professional experience in data engineering with strong focus on large-scale data solutions.
Advanced proficiency in Scala programming language.
Deep hands-on experience with Apache Spark (Core, SQL, Streaming) for batch and real-time data processing.
Extensive experience with at least one major cloud provider (AWS, Azure, or GCP) and their data services.
Strong understanding of data warehousing concepts, dimensional modeling, and ETL/ELT processes.
Expert-level SQL skills for data querying, manipulation, and optimization.
Experience with distributed systems and their challenges (consistency, fault tolerance, concurrency).
Proficiency with Git and collaborative development workflows.

Key Responsibilities*

Architect, build, and optimize robust, scalable, and efficient data pipelines using Scala and Apache Spark.
Develop solutions for ingesting high-volume, high-velocity data from various sources into data lake/warehouse.
Implement complex data transformations, aggregations, and feature engineering logic.
Identify and resolve performance bottlenecks in Spark jobs and data pipelines.
Implement data validation, monitoring, and alerting mechanisms to ensure data accuracy and consistency.
Leverage and optimize cloud services (AWS EMR/Glue, Azure Databricks, GCP DataProc/BigQuery).
Design and implement automated workflows for data pipelines using Apache Airflow or similar tools.
Contribute to data governance best practices across the organization.