Data Engineer (Databricks, AWS, Power BI) Retail

Job Summary

We are looking for a Data Engineer to design, build, and maintain scalable data pipelines and modern data platforms that support business reporting, analytics, and decision-making. This role is responsible for integrating data from multiple sources, developing efficient ETL/ELT processes, and ensuring data is accurate, reliable, and readily accessible across the organization.

The ideal candidate has strong experience in Python, SQL, cloud-based data platforms, and data engineering best practices, with the ability to optimize data architecture, automate workflows, and collaborate with both technical and business stakeholders to deliver high-quality data solutions.

---

Key Responsibilities

• Design, develop, and maintain scalable data pipelines for collecting, transforming, and integrating data from multiple sources.
• Build and manage ETL/ELT processes to automate data ingestion, transformation, and loading.
• Maintain and optimize databases, data warehouses, and data lakes to ensure performance, scalability, and data integrity.
• Integrate data from APIs, cloud platforms, third-party applications, and internal systems.
• Monitor data quality by implementing validation, monitoring, and error-handling processes.
• Troubleshoot and resolve data pipeline issues, bottlenecks, and performance concerns.
• Optimize data architectures to improve processing speed, scalability, and cost efficiency.
• Develop automation scripts using Python, SQL, and related technologies to streamline data operations.
• Manage cloud-based data infrastructure and support data processing environments.
• Collaborate with Data, BI, Analytics, and business teams to understand requirements and deliver reliable data solutions.
• Support data extraction, reporting, and analytics initiatives as needed.
• Perform user testing, validation, and production verification to ensure solution quality before deployment.
• Document data pipelines, system architecture, workflows, and technical processes for knowledge sharing and support.
• Follow Agile development practices and participate in sprint planning, reviews, and continuous improvement initiatives.
• Stay updated on emerging data engineering technologies and recommend improvements to enhance platform performance and reliability.

---

Qualifications

• Bachelor's Degree in Computer Engineering, Computer Science, Information Technology, Data Science, Statistics, Physics, or a related field.

• At least 3–5 years of experience in Data Engineering, Data Integration, or Data Platform Development.

• Hands-on experience designing and maintaining scalable ETL/ELT pipelines.

• Strong programming skills in:

  • Python (Apache Spark preferred)
  • SQL
  • DAX

• Experience working with cloud platforms such as Azure and/or AWS.

• Experience using Databricks, GitHub, Power BI, Linux, and modern data engineering tools.

• Strong understanding of:

  • Data Pipelines
  • ETL/ELT
  • Data Modeling
  • Data Wrangling
  • Data Quality
  • Data Governance
  • API Integration
  • Code and Performance Optimization

• Familiarity with common data formats including Parquet, SQL, CSV, JSON, XLSX, and TXT.

• Experience working in Agile environments (Scrum or Kanban).

• Strong analytical, troubleshooting, and problem-solving skills.

• Excellent communication skills with the ability to collaborate effectively with technical and business stakeholders.

• Detail-oriented with strong documentation and organizational skills.

Nice to Have

• Experience with LS Central or Microsoft Dynamics 365 Business Central.
• Knowledge of Microsoft SQL Server or Tableau.
• Basic knowledge of JavaScript, HTML, CSS, or AL.
• Experience supporting Business Intelligence, analytics, or enterprise reporting initiatives.