Technology Lead - Data Engineer

We are seeking an experienced Data Engineer to design, develop, and support enterprise data engineering and analytics solutions across modern cloud platforms. The ideal candidate will have hands-on experience with the complete project life cycle — from requirements gathering and design through development, testing, deployment, and post-production support — along with strong technical expertise in Python, Snowflake, Azure Data Factory (ADF) or Microsoft Fabric, and advanced SQL. This role requires strong UNIX/Linux shell scripting skills to build, optimize, and troubleshoot complex data pipelines supporting business-critical data and analytics initiatives.

Required Qualifications:

• Candidates authorized to work for any employer in Canada without employer-based visa sponsorship are welcome to apply. Infosys is unable to provide immigration sponsorship for this role at this time.
• Candidate must be located within commuting distance of Mississauga, ON / Calgary, AB / Burnaby, BC or be willing to relocate to the areas.
• Bachelor's degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.
• At least 4 years of Information Technology experience, including hands-on data engineering or ETL/ELT development.

Preferred Qualifications:

• Strong hands-on experience with Python for data pipeline development, automation, and scripting (e.g., pandas, PySpark, or similar libraries).
• Experience building and managing data pipelines and warehousing solutions in Snowflake (e.g., Streams, Tasks, Pipes, warehouses, role-based access, performance/cost optimization).
• Hands-on experience with Azure Data Factory (ADF) and/or Microsoft Fabric for cloud-based data integration, orchestration, and transformation.
• Strong proficiency in SQL (complex queries, joins, stored procedures, performance tuning) on any major RDBMS or cloud data warehouse (Snowflake, Azure SQL, Synapse, Oracle, SQL Server, etc.).
• UNIX/Linux shell scripting for automation, job scheduling, and file/data handling.
• Experience with ETL/ELT performance tuning and troubleshooting.
• Knowledge of job scheduling/orchestration tools (e.g., Airflow, Control-M, Autosys, ADF triggers/pipelines) is a plus.
• Familiarity with data warehousing concepts, dimensional modeling, and data governance practices.
• End-to-end exposure to project phases: initiation, requirement analysis, design, build, testing, UAT, deployment, and warranty/support.
• Experience working in Agile/Scrum or Waterfall project environments.
• Ability to prepare and review technical design documents, test plans, and deployment plans.
• Experience coordinating with cross-functional teams (business, QA, infrastructure, DBA) to ensure on-time, quality delivery.
• Familiarity with change management, version control (e.g., Git), and release management practices.

Key Responsibilities:

• Participate in the complete project life cycle (PLC) — requirement gathering, analysis, design, development, unit testing, UAT support, deployment, and post-go-live support.
• Design, develop, and maintain data pipelines and workflows using Python, Snowflake, and Azure Data Factory (ADF) or Microsoft Fabric.
• Build and manage cloud-native data integration and transformation pipelines, including ingestion, staging, and curated/analytics-ready layers in Snowflake.
• Analyze business and functional requirements and translate them into technical design documents (source-to-target mappings, pipeline design specs).
• Develop and optimize complex SQL queries, stored procedures, and views for data extraction, transformation, and validation.
• Write and maintain UNIX/Linux shell scripts for job scheduling, file handling, automation, and workflow orchestration.
• Perform data profiling, data quality checks, and error handling within data pipelines.
• Conduct performance tuning of pipelines, warehouses, and SQL queries to improve throughput and reduce processing time/cost.
• Collaborate with business analysts, DBAs, QA teams, and project managers throughout all SDLC/PLC phases.
• Prepare and maintain technical documentation, including design documents, unit test cases, and deployment/run-books.
• Support migration/modernization activities onto Snowflake, ADF, and/or Microsoft Fabric platforms.
• Troubleshoot production issues, perform root cause analysis, and implement permanent fixes.
• Adhere to coding standards, version control practices, and change management/release processes.
• Provide effort estimates, status updates, and risk assessments to project managers and stakeholders.

Nice to Have:

• Experience with additional cloud data platforms (e.g., Azure Synapse, Databricks, or AWS Redshift).
• Exposure to API-based integrations and event-driven/streaming data pipelines.
• Knowledge of Informatica or other ETL tools as a complement to modern cloud-native pipelines.
• Experience with CI/CD pipelines for data engineering deployments.

Estimated annual compensation range for the candidate based in the below location will be:
Ontario: $ 92740 to $ 123375
British Columbia: $ 90450 to $ 136500

The job entails sitting as well as working at a computer for extended periods of time. Should be able to communicate by telephone, email or face to face. Travel may be required as per the job requirements