Snowflake Python Data Engineer

Key Responsibilities:

• Design, develop, and maintain scalable ETL/ELT data pipelines using modern data engineering technologies.
• Develop complex and optimized SQL queries for data extraction, transformation, and processing.
• Build and maintain data processing solutions using Python, PySpark, and Apache Spark.
• Design, develop, and optimize data solutions using Snowflake.
• Design and implement scalable data models to support analytical and reporting requirements.
• Develop data applications and interactive solutions using Streamlit. Perform data transformation, cleansing, validation, and quality checks to ensure data accuracy and consistency.
• Identify and resolve performance bottlenecks across SQL queries, Spark jobs, Snowflake workloads, and ETL pipelines.
• Provide technical guidance and mentoring to junior and mid-level team members.
  Take ownership of technical deliverables and ensure timely and high-quality delivery.
• Collaborate with business stakeholders, architects, developers, and other teams to understand requirements and translate them into scalable data solutions.
• Participate in technical discussions, design reviews, code reviews, and solution planning.
• Communicate technical concepts, project status, risks, and dependencies effectively to both technical and non-technical stakeholders.
• Follow engineering best practices for coding, testing, version control, documentation, and deployment.

Preferred candidate profile

• Bachelors degree in computer science, Information Technology, Engineering, or a related field.
• Mandatory: Strong hands-on experience with Snowflake and Snowpark including its data warehousing capabilities.
• Mandatory: Extensive hands-on experience with internal and external stages, file formats, data loading, Snowpipe, Streams, Tasks, stored procedures, and Snowflake security/access controls.
• Mandatory: Strong hands-on experience with Snowpark, particularly using Python/Snowpark for data processing and transformation within Snowflake.
• Mandatory: Strong understanding of Snowflake architecture, virtual warehouses, compute/storage separation, workload management, performance optimization, and cost optimization
• Mandatory: Familiarity with cloud storage technologies such as Amazon S3 or Azure Data Lake Storage.
• Mandatory: Strong hands-on experience with SQL.
• Mandatory: Strong programming experience in Python and hands-on experience with PySpark for large-scale data processing.
• Good understanding of data modeling concepts, including relational and dimensional modeling.
• Experience in designing and developing ETL/ELT pipelines.
• Strong understanding of data transformation, data integration, and data processing concepts.
• Good analytical and problem-solving skills.
• Ability to work independently as well as collaboratively in a team environment.