Senior Data Engineer

About the Role:
Take your software engineering career to the next level as a Senior Product Software Data Engineer. You will play a key role in designing, developing, and troubleshooting software programs. Your technical skills and innovative mindset will contribute to the continuous improvement of our products, making a significant impact on user experience and satisfaction.

Responsibilities:

• Collaborate with cross-functional teams to review use cases and translate business requirements into scalable data solutions.
• Design and develop highly optimized, reliable, and secure enterprise-level data pipelines for real-time and batch processing.
• Analyze existing business processes, identify gaps, and recommend data-driven improvements.
• Build modern real-time data solutions using Scala, Apache Spark, and Azure Databricks.
• Partner with data scientists and business stakeholders to operationalize machine learning models.
• Champion data governance, metadata management, and operational excellence across data platforms.
• Ensure data quality and integrity by developing validation frameworks and monitoring mechanisms.
• Adhere to coding standards, participate in peer code reviews, and contribute to technical documentation.
• Mentor junior engineers and contribute to knowledge sharing across the team.

Skills:

• Software Engineering: The ability to design, develop, and maintain software systems and applications by applying principles and techniques of computer science, engineering, and mathematical analysis. This includes the capacity to understand user requirements, create and test the software, and resolve any software-related issues.
  • Software Development: The ability to design, write, test, and implement software programs, applications, and systems. This includes understanding various programming languages, software architecture, and software testing methods. It also involves problem-solving capabilities to fix software issues and improve functionality.
  • Problem Solving: The ability to understand a complex situation or issue and devise a solution by defining the problem, identifying potential strategies, and ultimately choosing and implementing the most effective course of action.
  • Analysis: The ability to examine complex situations or problems, break them down into smaller parts, and understand how these parts work together.
  • Testing: The skill of evaluating a system or process, often in software or product development. It involves the ability to identify problems, measure effectiveness, and ensure quality or functionality.
  • Agile: The ability to swiftly and effectively respond to changes, with an emphasis on continuous improvement and flexibility. In the context of project management, it denotes a methodology that promotes adaptive planning and encourages rapid and flexible responses to changes.
  • Source Code Repository: The ability to effectively use a source code repository, a file archive and web hosting facility where a large amount of source code, for software or for web pages, is kept, either publicly or privately. This skill involves the ability to manage and track changes to code, identify and fix issues, merge code from different branches, and collaborate with other developers.
  • Relational Database: The ability to design, implement, and manipulate a relational database, a type of database that stores and organizes data in a structured way and where data is logically interrelated. This skill often requires proficiency in SQL, database management systems, and understanding of database design principles.
  • Documentation: The ability to create, manage, organize, and maintain important information and documents in various physical and digital formats. This skill may include preparing reports, managing files, storing data, and keeping records organized and updated for easy retrieval and understanding.
• 7+ yearsof overall experience in data engineering or related fields.
• 5+ yearsof experience building data pipelines for structured and unstructured data.
• 2+ yearsof experience with big data technologies including PySpark, Hadoop, Kafka, EventHub, and Stream Analytics.
• 3+ yearsof experience in relational and dimensional data modeling using SQL Serverand data warehousing concepts.
• 3+ yearsof experience with T-SQL, stored procedures, and performance tuning.
• 3+yearsof experience with Azure Data Factory, Azure Data Lake, Azure SQL DB, and Azure Synapse Analytics.
• Good to have 2+ yearsof experience with Azure ML, Python, and Scala.
• Good to have 2+ yearsof experience with Power BIand Azure Analysis Services.
• Experience working with business stakeholders for requirements gathering and use case analysis.
• Strong communication, collaboration, and creative problem-solving skills.

Preferred Qualifications

• Bachelors degree in computer science, Engineering, or equivalent practical experience.
• Experience with Agile/Scrummethodologies and DevOps practices.
• Familiarity with CI/CD pipelines and version control systems (e.g., Git).
• Azure Data Engineer Certificationor equivalent is a strong advantage.