Databricks Engineer (Tax & Audit)

We are Improving, an IT services firm focused on AI, Data, and Applications. We modernize legacy systems, build cloud-native platforms, and deliver future-ready solutions through collaborative, long-term partnerships that drive measurable outcomes. In this role, we are looking for Databricks Engineers to support the Tax and Audit group. You will help design, build, and operate data ingestion, transformation, and analytics pipelines that enable reliable reporting and downstream consumption. The work centers on turning raw data into structured, governed datasets—leveraging Apache Spark, Databricks, Delta Lake/Delta Tables, and columnar formats—so business stakeholders can trust the outputs used for audit and tax-related analytics.

Responsibilities We are looking for Databricks Engineers to join our Tax and Audit group and strengthen the team’s delivery across the data lifecycle.

• Build and maintain Apache Spark solutions delivered on Databricks, including scalable transformations and production-ready workloads.
• Develop and optimize data ingestion, transformation, and analytics pipelines that handle small to large data volumes.
• Use advanced Python and PySpark to implement efficient data engineering patterns (e.g., batch processing, enrichment, and data quality steps).
• Apply strong understanding of columnar storage (e.g., Parquet) and deliver production data sets using Delta Lake / Delta Tables.
• Write and maintain unit tests for Databricks workloads to improve reliability and reduce regressions.
• Perform performance tuning and optimization (Spark execution efficiency, partitioning strategies, and query/read/write improvements).
• Collaborate with stakeholders to clarify requirements, translate analytics needs into technical designs, and ensure outputs meet accuracy and timeliness expectations.
• Own Databricks workflows end-to-end—from development through deployment and ongoing support.
• Optionally contribute to Databricks DevOps practices (nice to have), improving CI/CD, environment management, and release processes.

Goals: deliver dependable data products for Tax and Audit workflows, improve pipeline performance and maintainability, and ensure high-quality results through testing and careful attention to detail.

Required Skills & Experience We need someone with proven, hands-on software development experience and strong data engineering depth for Databricks-based delivery.

• 5+ years of hands-on software development experience.
• Strong expertise in Apache Spark, with a proven record of delivering on Databricks.
• Proficient in Python with advanced data engineering skills using PySpark, Pandas, and related libraries.
• Experience developing and maintaining unit tests for Databricks workloads.
• Solid understanding of columnar storage (e.g., Parquet).
• Hands-on experience with Delta Lake / Delta Tables.
• Ability to handle small to large data volumes, including performance tuning and optimizations.
• Demonstrated ability to build data ingestion, transformation, and analytics pipelines.
• Strong analytical and problem-solving skills with close attention to detail.
• Comfort working with Databricks workflows and delivering outcomes reliably.

How We Work (traits We Value)

• Quality mindset: careful validation, correctness, and attention to details—especially important for Tax and Audit contexts.
• Ownership: you take responsibility for pipeline stability, performance, and maintainability.
• Communication: we expect clear collaboration with technical and non-technical stakeholders to align data outputs with business needs.
• Continuous improvement: you actively look for ways to simplify pipelines, improve performance, and strengthen testing coverage.

Nice to have

• Familiarity with Databricks DevOps practices (e.g., CI/CD concepts, environment/version management, and deployment workflows).
• Experience contributing to data governance patterns such as dataset documentation, consistency checks, and maintainable conventions across teams.
• Prior exposure to testing strategies beyond unit tests (e.g., integration tests for pipelines) to further improve reliability.

Benefits

• Comprehensive medical, dental, vision, and life insurance
• Mental health support
• Savings Fund & Corporate Retirement Plan
• Paid time off, vacation bonus, and Christmas bonus
• Career development, certifications, and continuous learning
• TotalPass wellness program
• Collaborative culture with internal events and technical communities
• Opportunity to work on innovative global projects with cutting-edge technologies