Data Engineer- Azure Databricks Senior Associate

Industry/Sector

Not Applicable

Specialism

Product Innovation

Management Level

Senior Associate

Job Description & Summary

At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth.

In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data transformation solutions.

Years of Experience

Candidates with 4-8 years of hands on experience

Position Requirements:

Must Have:

• 4+ years of experience working in Azure ADLS, Data Bricks, Data Flows, HDInsight, Azure Analysis services
• Should be aware of Data Management concepts and Data Modelling
• Good knowledge of Big Data frameworks and related technologies
• Experience in Hadoop and Spark is mandatory
• Should have expertise in extracting data from different source systems like flat files, XML sources, JSON, Parquet files, Big data appliances, RDBMS, etc.
• Good knowledge of Big Data frameworks and related technologies
• Experience in Hadoop and Spark is mandatory
• Should have clear understanding of DW lifecycle and contributed in preparing
• Detail Level Design document
• Unit Test plans
• Code review reports
• Must have strong sql, understand joins, groups, functions & Stored procedures
• Must have experience in Unit and Integration Test creation and verification
• Proficient in creating documentation related to Project Architecture, Project end User Manual and Operations Hand off guide
• Should have good understanding of Data Migration processes, methods and project lifecycle
• Good in Unix Shell Scripting skills
• Strong analytical skills and troubleshooting skills
• Should be well versed with quality processes and implementation
• Excellent communication skills
• Strong understanding of Cloud data migration processes, methods and project lifecycle
• Good knowledge of Application DevOps tools (Git, CI/CD Frameworks)
• Experience in Jenkins or Gitlab with rich experience in source code management like Code Pipeline, Code Build and Code Commit

Desired Knowledge / Skills:

• Certifications in Azure / Databricks
• Experience in building stream-processing systems, using solutions such as Storm or Spark
• Streaming
• Experience in Big Data ML toolkits, such as Mahout, SparkML, or H2O
• Experience in Azure functions
• Knowledge in Python

The Opportunity

• When you join PwC Acceleration Centers (ACs), you step into a pivotal role focused on actively supporting various Acceleration Center services, from Advisory to Assurance, Tax and Business Services. In our innovative hubs, you’ll engage in challenging projects and provide distinctive services to support client engagements through enhanced quality and innovation. You’ll also participate in dynamic and digitally enabled training that is designed to grow your technical and professional skills.
• As part of the Data and Analytics Engineering team you will lead the end-to-end implementation of innovative Cloud data engineering solutions. As a Senior Associate, you will analyze complex problems, mentor junior team members, and build meaningful client relationships while navigating the intricacies of the data landscape. This role offers the chance to deepen your technical skills and enhance your professional brand in a dynamic and collaborative environment.

Responsibilities

• Design and develop data solutions using Azure Databricks
• Collaborate with cross-functional teams to deliver analytics projects
• Analyze data to derive actionable insights for clients
• Enhance data processing workflows for efficiency
• Mentor and support junior data engineers in their tasks
• Stay informed on advancements in data engineering technologies
• Maintain adherence to data governance and security protocols
• Conduct training sessions on recommended practices in data management

What You Must Have

• Bachelor's Degree in Engineering, MCA, M.Sc, M.E, M.Tech, MBA
• 4 years of experience
• Oral and written proficiency in English required
• BE / B.Tech / MCA / M.Sc / M.E / M.Tech / MBA

What Sets You Apart

• Certifications in Azure or Databricks
• Excelling in Azure cloud and data engineering solutions
• Demonstrating planning and organization skills
• Leading cloud development on agile teams
• Understanding data management concepts and data modeling
• Skilled in SQL and data extraction techniques
• Utilizing Big Data frameworks like Hadoop and Spark
• Knowledgeable in application DevOps tools and processes

Travel Requirements

Not Specified

Job Posting End Date