Senior Data Engineer Python PySpark Databricks

Job Title : Senior Data Engineer Python/PySpark & Databricks

Work Location: Remote

Experience : 8Years

Job Summary: We are looking for an experienced Backend/Data Developer with hands-on experience in Databricks, PySpark, Python/Java, Web Services, and SQL to support Dealer Services (Data Edge). The ideal candidate will have strong backend and data engineering skills, experience designing and troubleshooting data-driven solutions, and the ability to work closely with architects, technical leads, business analysts, and operations teams.

Essential Job Functions

• Understand the overall system architecture and support operations with monitoring, dashboards, and system health activities.
• Work closely with architects and technical leads on solution design and implementation.
• Continuously monitor and optimize system performance, scalability, and cost.
• Partner with Business Analysts to understand and implement integration requirements.
• Troubleshoot and resolve performance, backend, data, and functional issues.
• Analyze application and data-processing issues and identify root causes.
• Maintain strong attention to detail and document key technical decisions, solutions, and project artifacts.
• Support fast-paced projects and participate in troubleshooting and resolving production issues when required.
• Quickly learn and adopt new technologies, tools, and development methodologies.
• Communicate promptly and effectively with technical and business stakeholders.
• Participate in code reviews and provide technical guidance and mentoring to junior engineers.
• Collaborate with cross-functional teams to deliver reliable, scalable, and high-quality solutions.

Minimum Qualifications & Requirements

• Minimum 7 years of IT development experience.

• Strong experience developing data-driven backend solutions.

• Strong hands-on experience with:

• Python / PySpark

• Databricks

• SQL / PL/SQL

• Java or C#

• Backend development and Web Services/API integration

• Strong SQL skills with experience working with complex queries and data processing.

• Experience designing technical solutions and participating in architecture/design discussions.

• Experience performing code reviews and mentoring junior engineers.

• Strong analytical, troubleshooting, and problem-solving skills.

• Ability to investigate performance issues and identify root causes.

• Strong understanding of software development lifecycle, coding standards, and best practices.

• Ability to work independently as well as effectively within a team.

• Strong verbal and written communication skills.

• Must be a collaborative team player with a customer-focused mindset.

Preferred Qualifications

Experience with one or more of the following is highly desirable:

• Microsoft Azure
• Apache Kafka
• Node.js
• Azure Data Factory
• Cloud-based data engineering and integration solutions
• Distributed data processing and large-scale data platforms
• Monitoring, logging, and performance optimization tools