Databricks Engineer

Overview

We are seeking a dynamic and highly skilled Databricks Engineer to join our innovative data team. In this role, you will be at the forefront of designing, developing, and optimizing large-scale data solutions using cutting-edge cloud-based platforms. Your expertise will drive impactful insights and enable data-driven decision-making across the organization. This position offers an exciting opportunity to work with a diverse set of big data technologies, cloud environments, and business intelligence tools to transform complex data into actionable intelligence.

Responsibilities

• Design, develop, and maintain scalable ETL pipelines and data workflows utilizing Databricks, Spark, and related big data systems.
• Collaborate with cross-functional teams to implement data management integration strategies across cloud databases such as Azure Data Lake, AWS, and other public cloud platforms.
• Build and optimize data models using dimensional modeling techniques for efficient data warehousing and query management.
• Develop and enhance data architecture solutions leveraging Hadoop, Apache Hive, and SQL databases including Microsoft SQL Server and Oracle.
• Implement robust data security, governance, and quality standards to ensure reliable and compliant data assets.
• Automate deployment processes using scripting languages such as Python, Bash (Unix shell), and Shell Scripting within an Agile development environment.
• Work closely with business intelligence tools like Looker to create insightful dashboards and reports that support strategic decision-making.
• Participate in model training, analysis activities, and RESTful API integrations to extend data capabilities across platforms.

Experience

• Proven experience working with big data systems such as Hadoop, Spark, Apache Hive, and related technologies in a cloud environment (AWS, Azure).
• Strong background in software development with proficiency in Java, Python, SQL (including SQL databases like Oracle), and ETL pipeline development.
• Hands-on experience designing data models for data warehousing solutions using dimensional modeling principles.
• Familiarity with cloud databases including Azure Data Lake, cloud-based data warehousing design, and public cloud infrastructure management.
• Knowledge of business intelligence tools like Looker or similar platforms for creating dashboards and visualizations.
• Understanding of data management integration techniques involving Informatica, Talend or equivalent tools.
• Ability to work effectively within an Agile framework while managing multiple priorities in a fast-paced environment.
• Strong analysis skills coupled with experience in database design, query management, and analysis of large datasets.
• Additional skills such as VBA or Bash scripting are advantageous for automation tasks.

Join us to leverage your expertise in big data engineering while contributing to innovative solutions that empower our organization’s growth through smarter insights!

Pay: $100.00 - $110.00 per hour

Expected hours: 40.0 per week

Work Location: Hybrid remote in Ashburn, VA 20146