Databricks Data Engineer

Job Title: Databricks Data Engineer 

Job Description: 
As a Databricks Data Engineer at KPI Partners, you will play a crucial role in designing, developing, and optimizing data processing pipelines on the Databricks platform. You will be responsible for transforming raw data into actionable insights and enabling data-driven decision-making across the organization. Your expertise in data engineering and proficiency with the Databricks environment will be essential in building scalable and efficient data workflows. 

Key Responsibilities: 

• Develop and maintain data pipelines using Databricks, Spark, and other data processing technologies. 
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and ensure data quality and accessibility. 
• Implement and optimize ETL processes to support analytics and reporting initiatives. 
• Monitor and troubleshoot data pipeline performance, ensuring timely delivery and reliability. 
• Contribute to the design and implementation of data architecture and data models. 
• Utilize Databricks notebooks for data exploration, visualization, and sharing insights with team members. 
• Stay updated with the latest trends and advancements in data engineering and the Databricks platform.

Skills and Tools Required: 

• Proficiency in Databricks and Apache Spark. 
• Strong programming skills in Python or Scala for data processing. 
• Experience with ETL processes and tools. 
• Knowledge of data modeling and data warehousing concepts. 
• Familiarity with cloud platforms such as AWS, Azure, or Google Cloud. 
• Experience with SQL and database technologies (e.g., MySQL, PostgreSQL, or others). 
• Familiarity with data visualization tools (e.g., Tableau, Power BI) is a plus. 
• Strong analytical and problem-solving skills. 
• Excellent communication and collaboration skills to work effectively within a team.

This position offers an opportunity to work in a dynamic environment and make a significant impact by enabling data-driven strategies at KPI Partners.