Senior Data Engineer

Role & responsibilities

• Data Architecture: Design and implement scalable and robust backend data architectures to support business intelligence, analytics, and machine learning initiatives.
• ETL Development: Develop and maintain efficient Extract, Transform, Load (ETL) pieplines to move and transform data from diverse sources into our data warehouse.
• Database Management: Manage and optimize databases, ensuring data quality, integrity, and security. Implement indexing, partitioning, and other performance optimization strategies.
• Data Integration: Collaborate with cross-functional teams to integrate data from various sources and ensure a cohesive and accurate representation of business information.
• Data Modeling: Design and implement data models to support business requirements and facilitate data analysis.
• Coding and Scripting: Write efficient and maintainable code in languages such as Python, Java, or Scala to implement data pipelines and automation processes.
• Workflow Automation: Implement and manage workflow automation tools such as Apache Airflow to orchestrate and schedule data processing tasks.
• Monitoring and Optimization: Monitor system performance, troubleshoot issues, and implement optimizations for data processing and storage.
• Documentation: Create and maintain comprehensive documentation for data processes, data models, and infrastructure configurations.
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide support for their initiatives.

Preferred candidate profile

• Bachelor's degree B.Tech/BE in Computer Science, Information Systems, or a related field.
• Proven experience as a Backend Data Engineer with 4-8 years of hands-on experience in data engineering and backend development.
• Strong proficiency in programming languages like Python, Java, or Scala.
• Proficiency & hands-on on databases such as MongoDB, DynamoDB, Redis, Cassandra, HBase, ElasticSearch etc.
• Proficiency & hands-on on Message Broker Services such as Kafka, RabbitMQ, AWS SQS/SNS etc.
• Good knowledge working on cloud platforms such as AWS, Azure, or GCP.
• Good knowledge on Big data platforms such as Apache Spark, Hadoop, Cloudera, Snowflake, Databricks etc
• Experience with workflow automation tools such as Apache Airflow.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills.