Senior Data Engineer

Description:
We are seeking a highly skilled and experienced Senior Data Engineer specializing in Microsoft SQL Server, Azure data services, and Microsoft Fabric. The ideal candidate will design, develop, and maintain enterprise data warehouses, lakehouses, and data pipelines, ensuring data reliability, performance, quality, and security. The Senior Data Engineer will collaborate closely with software developers, data analysts, and other stakeholders to deliver trusted data for reporting, analytics, and business processes.

Database Design and Development:
Design, implement, and maintain database structures and dimensional data models, including tables, views, indexes, and schemas using Microsoft SQL Server, Azure SQL, and Microsoft Fabric Warehouse and Lakehouse.

Develop complex SQL queries, stored procedures, functions, and triggers to support application development and data processing.

Optimize database performance through indexing, query optimization, and other performance tuning techniques.

Data Integration and ETL Processes:
Design and develop ETL and ELT processes using SQL Server Integration Services (SSIS), Azure Data Factory, and Microsoft Fabric Data Factory pipelines and notebooks to ingest and transform data from multiple sources.

Ensure data quality, consistency, and integrity across different systems and environments.

Monitor and troubleshoot data pipelines, implement logging and error handling, and ensure timely and accurate data processing.

Query Optimization:
Analyze and optimize SQL queries to improve execution times and reduce resource consumption.

Implement indexing strategies to speed up data retrieval operations.

Rewrite complex queries to make them more efficient and easier to maintain.

Collaborate with developers to optimize queries embedded within applications.

Azure and Microsoft Fabric Integration:
Design and maintain cloud data solutions using Azure SQL Database, Azure Data Lake Storage, and Microsoft Fabric OneLake, Lakehouse, and Warehouse.

Develop and maintain data integration solutions connecting SQL Server, Azure data services, and Microsoft Fabric, including incremental loads and bronze, silver, and gold data layers.

Apply best practices for Azure and Microsoft Fabric, including workspace organization, access controls, monitoring, and performance and cost optimization.

Collaboration and Support:
Work closely with software developers, data analysts, and other IT team members to understand requirements and provide database solutions.

Provide technical support and troubleshooting for database-related issues.

Mentor and guide junior data engineers and other team members.

Documentation and Compliance:
Create and maintain comprehensive documentation for database designs, processes, and procedures.

Ensure compliance with data governance, security policies, and industry regulations.

Qualifications:
Minimum of 9 years of experience in data engineering and database development with a focus on Microsoft technologies.

Proven experience with Microsoft SQL Server and T-SQL, SSIS, Azure data services, and Microsoft Fabric, with familiarity with SSRS and Power BI.

Expertise with Azure data services and Microsoft Fabric for enterprise data warehousing and lakehouse solutions.

Experience with Microsoft Fabric medallion architecture, Delta tables, incremental processing, and pipeline orchestration.

Knowledge of data modeling and database design principles.

Familiarity with Azure Synapse Analytics, Microsoft Purview, and Git-based version control and CI/CD for data solutions.

Strong analytical and problem-solving skills.

Excellent communication and teamwork abilities.

Ability to work independently and manage multiple tasks and projects simultaneously.

Detail-oriented with a focus on quality and accuracy.

Education:
Bachelor’s degree in computer science, Information Technology, or a related field. Master’s degree preferred.

The estimated annual salary for this position ranges from $109,000 to $152,000. Actual compensation will vary based on factors including but not limited to the candidate’s skills, experience, and qualifications. Geographic differentials may also apply depending on the position’s location. There is no application deadline for this role; recruitment will remain open until the position is filled.