Data Infrastructure Engineer

Who you are

• The ideal candidate has extensive experience with distributed data platforms such as Kafka, Hadoop/MapR, ultra-fast NoSQL databases (Redis, DynamoDB, Aerospike), and Linux-based infrastructure
• 4+ years of experience administering distributed Big Data Platforms
• Messaging Systems: Proven experience deploying and managing production grade Kafka infrastructure
• NoSQL Databases: Practical experience administering at least on major NoSQL database
• Scripting: Proficiency with shell scripting (Bash, Python, or similar)
• Experience with relational databases such as PostgreSQL or MySWL
• Linux Administration: Experience in Linux operating systems, including storage management, networking, security and performance troubleshooting
• Experience with monitoring platforms such as Grafana, Prometheus, or similar
• Experience with Kubernetes and containerized workloads
• Experience with Ceph or other distributed storage platforms
• Experience with Infrastructure as Code tools such as Terraform or Spacelift
• Experience with configuration management tools such as Puppet or Ansible
• Experience with cloud platforms (AWS)

What the job involves

• We are seeking a Data Platform Engineer (Hadoop) to join our Infrastructure Engineering team. This role is responsible for designing, building, maintaining, and modernizing our enterprise big data platforms that support large-scale data processing and analytics across the organization
• This individual will work closely with Infrastructure, SRE, Database and Engineering teams to ensure our platforms are scalable, highly available, secure and performant
• This is a hands-on engineering role requiring strong operational experience supporting mission-critical production environments while driving platform modernization initiatives
• Hadoop Management: Provision, configure, monitor and maintain robust distributed Hadoop clusters (e.g. MapR, Cloudera, Hortonworks)
• Linux Systems Engineering: Manage, optimize, and troubleshoot enterprise Linux servers (RedHat, CentOS, Rocky Linux) hosting the data platform
• Performance Tuning: Troubleshooting, root cause analysis for production environments
• High Availability: Maintain cluster backup, disaster recovery, production support and on-call rotations
• Infrastructure as Code (Iac)
• CI/CD Integration
• Kafka Event Streaming: Deploy, configure, tune and scale multi-node Kafka clusters, topics, partitions and consumer groups
• NoSQL Database Management: Administer, schema design, and performance-tune NoSQL databases (such as HBase, Aerospike, Cassandra, or MongoDB)
• Ecosystem Integration: Support and integrate core Hadoop ecosystem tools including Hive, Spark, and HDFS

Benefits

• Equity and Employee Stock Purchase Plan
• Pension and Retirement Savings Plan in Several Countries
• Comprehensive Healthcare Benefits for You and Your Family
• Generous Time Off, Holiday Breaks and Summer Fridays
• Family-Focused Leave Benefits
• Cell Phone Subsidy
• Performance Management, and Investment in Diversity Initiatives
• Bonusly Peer-to-Peer Recognition Program, Turning Recognition into Tangible Perks and Magnite Swag
• Community Service Events
• Wellness Coach—Meditate and Recharge with an Unlimited User Account for You and a Plus One