Bigdata Developer

Objectives of this role

• Designing, developing, and implementing scalable and efficient data processing pipelines using Big Data technologies.
• Collaborating with data architects, data scientists, and business analysts to understand and translate data requirements into technical solutions.
• Developing and optimizing ETL (Extract, Transform, Load) processes to ingest and transform large volumes of data from multiple sources.
• Implementing data integration solutions to ensure seamless data flow and accessibility across different platforms and systems.
• Building and maintaining data warehouses, data lakes, and data marts to store and organise structured and unstructured data.
• Designing and developing data models, schemas, and structures to support business analytics and reporting requirements.
• Monitoring and optimizing the performance of Big Data applications and infrastructure to ensure reliability, scalability, and efficiency.
• Troubleshooting and resolving data processing, quality, and system performance issues.

Your tasks

• Develop and deploy data processing applications using Big Data frameworks such as Hadoop, Spark, Kafka, or similar technologies.
• Write efficient and optimized code in programming languages like Java, Scala, Python, or SQL to manipulate and analyse data.
• Implement data security and privacy measures to protect sensitive information and comply with regulatory requirements.
• Collaborate with cross-functional teams to integrate data solutions with existing systems and applications.
• Conduct testing and validation of data pipelines and analytical solutions to ensure accuracy, reliability, and performance.
• Document technical specifications, deployment procedures, and operational guidelines for data solutions.
• Stay updated on industry trends, emerging technologies, and Big Data development and analytics best practices.

Experience - 6-10 yrs