Software Engineer III

Job Summary

Software Engineer III

Are you excited to build scalable data platforms and transform complex data into actionable insights. Do you enjoy collaborating with cross-functional teams to deliver high-quality data solutions that drive business outcomes.

About the Role

As a Data Engineer, you will design, build, and optimize data pipelines and platforms that enable reliable, scalable, and secure data processing across enterprise environments. You will work with modern Azure technologies and collaborate with business analysts, data scientists, and application teams to deliver high-quality data solutions that support analytics, reporting, and operational needs.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory, Databricks, and Azure storage services.

• Ingest data from multiple sources including relational databases, APIs, and file-based systems into Azure data platforms.

• Implement data transformations on HPCC using ECL and on Azure platform using SparkSQL and PySpark in Azure Databricks.

• Develop and optimize notebooks and jobs using Azure Databricks.

• Write efficient Spark jobs for large-scale data processing.

• Design and implement data models on HPCC using ECL and also using Delta Lake, Parquet, or other columnar storage formats.

• Optimize ECL code, SQL, and Spark workloads for performance and scalability.

• Work closely with business analysts, data scientists, and application teams to understand data requirements and promote data engineering best practices.

  Requirements

• Strong proficiency in SQL and data transformation logic with Solid understanding of data warehousing concepts and ETL/ELT patterns.

• Hands-on experience with Azure Databricks, Spark SQL, and PySpark.

• Experience with Azure Data Factory, Azure Data Lake Storage (ADLS Gen2), and Azure SQL / Synapse.

• Experience working with large-scale, structured and semi-structured datasets.

• Experience with Agile development methodologies.

• Must have a positive attitude and desire to learn new processes and technologies.

• Knowledge of SDLC process like requirement gathering, design, implementation (Coding), testing, deployment, and maintenance. Solid understanding of software development principles, design patterns, and best practices. Proficient in requirement gathering and documentation.

• Excellent communication skills - verbal and written, Strong problem-solving skills with good knowledge of Insurance terminology.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.