Data Engineer

Skills: Digital : Python~Digital : PySpark

Descriptions: GX| Python|Pyspark|Databricks / Snowflake / Fabric

Candidate Profile Summary

• Core Technical Expertise:

• Python & PySpark: Strong proficiency in writing scalable, high-performance data processing code and building ETL/ELT pipelines.

• Data Quality & Governance: Experience with GX (Great Expectations) for data validation, testing, and profiling.

• Modern Data Platforms: Hands-on experience with at least one (ideally a combination) of the following cloud data ecosystems:

• Databricks (Lakehouse architecture, Spark optimization)

• Snowflake (Cloud data warehousing, SQL performance tuning)

• Microsoft Fabric (OneLake, Spark compute, data engineering workloads)

Key Responsibilities Expected from this Profile

• Designing, developing, and optimizing large-scale data pipelines using Python and PySpark.
• Implementing data quality frameworks and automated validation tests using Great Expectations (GX).
• Integrating and deploying data solutions on cloud platforms like Databricks, Snowflake, or Microsoft Fabric.
• Collaborating with cross-functional teams (Data Engineers, Data Analysts, and Architects) to deliver robust data infrastructure.