Software Developer

Software Developer is needed to perform the following duties:

·Create business models, logical specifications and/or user requirements to develop solutions for the application environment–10%

o Build and maintain data transformation pipelines in Azure Databricks using Python, PySpark, Java, and Spark SQL.

o Use AI-assisted development tools and platforms, including GitHub Copilot, Claude, ChatGPT, and Azure AI Foundry, to support application and data engineering development, including code generation and refinement, PySpark and SQL optimization, troubleshooting, test development, documentation, and the development and integration of AI agent-based workflows to automate repetitive development activities and improve engineering efficiency.

o Work closely with Data Scientists, Cloud Architects, Data Engineers, Software Developers, and other stakeholders to understand data requirements, design solutions, participate in Agile development activities, and deliver reliable and accessible data for analytics and downstream applications.

· Design software applications, create system procedures and ensure that the developed applications function normally- 10%

o Design and develop data engineering solutions on the Microsoft Azure Cloud Platform, using Azure Data Factory, Azure Databricks, Azure Data Lake Storage, Azure Blob Storage, Azure SQL Database, and related Azure services to support data processing and analytics.

o Design and maintain Terraform-based Infrastructure-as-Code for cloud data infrastructure, including Databricks workspaces, Snowflake resources, Azure storage accounts, and networking components.

o Design and implement Snowflake data warehouse solutions, including schema design, data loading processes, virtual warehouse sizing, SQL development, and query performance tuning for analytical workloads.

· Develop and create business models, logical specifications and/or user requirement solutions for the application environment-15%

o Develop and manage Azure Data Factory (ADF) pipelines to orchestrate end-to-end data workflows, including scheduled and trigger-based executions, Databricks notebook activities, and downstream Snowflake data loads.

o Develop and maintain CI/CD pipelines using GitHub Actions to automate testing, validation, and deployment of ADF pipelines, Databricks notebooks, and Delta Live Tables across development, staging, and production environments.

o Develop and maintain cloud-based ETL solutions on Amazon Web Services (AWS) using AWS Glue, Amazon S3, Amazon EMR, Amazon EC2, and Amazon Redshift for large-scale data processing and warehousing.

· Implement and modify programs; make approved changes by amending flow charts, develop detailed programming logic, and coding changes-15

o Implement the medallion architecture to progressively cleanse, enrich, transform, and aggregate data across bronze, silver, and gold layers.

o Implement data storage and processing solutions on the Google Cloud Platform (GCP) using Google Cloud Storage, BigQuery, and Cloud SQL to support data integration, storage, and analytical workloads.

o Develop scalable Apache Spark applications using Python, Scala, and Java to process large datasets and perform transformations, joins, aggregations, and other distributed data processing tasks.

· Participate in scrum meetings and coordinate with Business Analysts to understand the business needs and implement the same into a functional design-20%

o Use Git and GitHub branching and pull request workflows to manage source code, conduct code reviews, and support collaborative development and controlled releases of data engineering applications.

o Build and maintain data ingestion and processing workflows using Apache Kafka and Azure event-driven services to capture, stream, and process data from multiple source systems and integrate it with downstream data engineering and analytics pipelines.

o Use shell scripting and automation tools to support data workflow scheduling, orchestration, monitoring, and operational activities.

· Write source code, prepare test data, tests and debug programs; revise and refine programs to improve the performance of the application software-15%

o Work with relational and NoSQL databases, including MySQL, PostgreSQL, Microsoft SQL Server, Netezza, Oracle, MongoDB, and Snowflake, developing SQL, DDL, DML, data models, and data integration processes.

o Support production operations by creating and managing ServiceNow incidents, problem records, and change requests, documenting root-cause analysis, remediation steps, deployment plans, approvals, and production changes.

o Develop data validation, testing, and error-handling processes to improve the quality, accuracy, completeness, and reliability of data throughout ingestion, transformation, and loading workflows.

· Perform execution of functional test plan, validate test results, prepare documentation & data for analysis-15%

o Monitor production ADF pipelines and Databricks jobs, troubleshoot failures and performance issues, identify root causes, and implement fixes to keep data processing workflows reliable and within established SLAs.

o Develop and maintain automated tests, including unit, functional, integration, and regression testing, for data processing applications and pipeline components to ensure reliable software releases.

o Develop AWS Glue ETL jobs to extract data from sources such as Amazon S3 and ORC, Parquet, and text files, apply transformations, joins, and pre-aggregations, and load the resulting data into Amazon Redshift.

o Design and maintain scalable data pipelines and data lake solutions to support the ingestion, transformation, storage, and processing of large structured and unstructured datasets.

Bachelor's Degree is required in Computer Science or Computer Engineering or Information Systems Technologies.

Work Location: In person