Databricks Engineer

Job Summary
We are seeking a dynamic and innovative Databricks Engineer to join our data engineering team. In this role, you will be responsible for designing, developing, and maintaining scalable big data solutions using Databricks Unified Analytics Platform. You will leverage your expertise in cloud-based data systems, data modeling, and software development to enable advanced analytics and business intelligence capabilities. Your contributions will directly impact our ability to harness vast datasets for strategic insights and operational excellence. This is an exciting opportunity for a passionate professional eager to work at the forefront of big data technology within a collaborative, fast-paced environment.

Responsibilities

• Develop and optimize ETL pipelines utilizing Databricks, Spark, Python, SQL, and other relevant tools to process large-scale datasets efficiently.
• Design and implement data models, including dimensional modeling and data warehousing strategies, to support business intelligence and analytics initiatives.
• Integrate diverse data sources such as AWS cloud storage, Azure Data Lake, Hadoop clusters, Oracle databases, Microsoft SQL Server, and other SQL databases into unified data platforms.
• Collaborate with cross-functional teams to understand data requirements and translate them into scalable technical solutions using cloud databases and big data systems.
• Build and maintain robust data management workflows using Informatica, Talend, or similar ETL tools; ensure seamless data flow across systems through RESTful APIs and shell scripting.
• Support model training, analysis activities, and query management to facilitate actionable insights for business stakeholders.
• Participate in Agile development cycles to deliver high-quality solutions rapidly while adhering to best practices in software development and data engineering.

Experience

• Proven experience working with cloud-based big data platforms such as Databricks, Azure Data Lake, or similar environments.
• Strong proficiency in Spark (PySpark or Scala), Java, Python, SQL (including complex query writing), and Bash or shell scripting for automation tasks.
• Hands-on experience designing data models using dimensional modeling techniques for enterprise data warehouses.
• Familiarity with big data ecosystems including Hadoop, Hive, Apache Spark, and related technologies.
• Knowledge of public cloud services such as AWS or Azure Data Lake for scalable storage and computing solutions.
• Experience integrating diverse systems like Oracle, Microsoft SQL Server, Informatica, Talend, or other ETL tools into cohesive pipelines.
• Understanding of business intelligence tools such as Looker or similar platforms for reporting and visualization purposes.
• Strong analytical skills with the ability to interpret complex datasets and communicate findings effectively.
• Experience working within Agile teams on large-scale projects involving data management integration and software development best practices.

Join us to leverage your expertise in big data systems and cloud technologies to transform raw information into strategic assets!

Pay: $96,400.00 - $117,000.00 per year

Work Location: In person