Data Engineer - Databricks

Key Responsibilities

• Work with business analysts and data architects to translate requirements into technical implementations.
• Design, develop, implement, and maintain pyspark code through Databricks UI that enables data and analytics use cases for the client
• Code, test, and document new or modified data systems to create robust and scalable applications for data analytics.
• Dive deep into performance, scalability, capacity, and reliability problems to resolve issues.
• Take on research projects and POC to improve data processing.

Must-Have Skills Work and Technical Experience:

• 5+ Hands on Experience with Databricks and PySpark
• Proficiency in SQL and data manipulation skills
• Good understanding of data warehousing concepts and technologies.

Good-to-Have Skills

• Understanding of Google Pub sub/Kafka/Mongo DB
• Familiarity with ETL processes and tools for data extraction, transformation, and loading.
• Knowledge of cloud platforms like Databricks, Snowflake, Google Cloud
• Familiarity with data governance and data quality best practices.

Qualifications

• Bachelor’s degree in computer science, engineering, or related field
• Demonstrated continued learning through one or more technical certifications or related methods
• 5+ years of relevant experience in Data Analytics
• Relevant experience in Retail domain.

Qualities

• Self-motivated and focused on delivering outcomes for a fast-growing team and firm
• Able to communicate persuasively through speaking, writing, and client presentations
• Able to consult, write, and present persuasively
• Able to work in a self-organized and cross-functional team
• Able to iterate based on new information, peer reviews, and feedback
• Able to work with teams and clients in different time zones
• Research focused mindset