Senior Big Data Engineer

• We are seeking a Senior Big Data Engineer to join our Infrastructure Engineering team. This role is responsible for designing, building, maintaining, and modernizing our enterprise big data platforms that support large-scale data processing and analytics across the organization
• This individual will work closely with Infrastructure, SRE, Database, and Engineering teams to ensure our platforms are scalable, highly available, secure, and performant
• This is a hands-on engineering role requiring strong operational experience supporting mission-critical production environments while driving platform modernization initiatives
• Design, deploy, administer, and optimize enterprise Big Data platforms
• Manage and support distributed technologies including:
• Apache Kafka
• Hadoop / MapR
• Aerospike
• Build highly available, scalable, and resilient data infrastructure across multiple data centers
• Perform capacity planning, performance tuning, troubleshooting, and root cause analysis for production environments
• Automate infrastructure deployment, configuration management, and operational workflows using Infrastructure as Code and scripting
• Collaborate with Software Engineering, Data Engineering, SRE, and Database teams to support new applications and platform requirements
• Implement monitoring, alerting, and observability solutions to ensure platform health and availability
• Participate in production support and on-call rotations for critical infrastructure

Benefits

• Equity and Employee Stock Purchase Plan
• Pension and Retirement Savings Plan in Several Countries
• Comprehensive Healthcare Benefits for You and Your Family
• Generous Time Off, Holiday Breaks and Summer Fridays
• Family-Focused Leave Benefits
• Cell Phone Subsidy
• Performance Management, and Investment in Diversity Initiatives
• Bonusly Peer-to-Peer Recognition Program, Turning Recognition into Tangible Perks and Magnite Swag
• Community Service Events
• Wellness Coach—Meditate and Recharge with an Unlimited User Account for You and a Plus One- The ideal candidate has extensive experience with distributed data platforms such as Kafka, Hadoop/MapR, Aerospike, and Linux-based infrastructure
• 4+ years of experience administering distributed Big Data platforms
• Strong experience with:
• Hadoop (MapR, Cloudera, Apache Hadoop, or similar)
• Passion for automation, operational excellence, and continuous improvement
• Experience troubleshooting distributed systems in production environments
• MySQL
• Strong knowledge of networking fundamentals including DNS, TCP/IP, load balancing, and security
• Aerospike
• Experience with monitoring platforms such as Grafana, Prometheus, Graphite, or similar
• Ability to work independently in a fast-paced production environment
• Experience with relational databases such as:
• 4+ years of experience supporting large-scale Linux infrastructure
• Experience with shell scripting (Bash, Python, or similar)
• Apache Kafka
• PostgreSQL
• Strong Linux system administration experience (RHEL, Rocky Linux, CentOS)
• Preferred Qualifications
• Experience working in highly available, 24x7 production environments
• Experience with Kubernetes and containerized workloads
• Experience with Infrastructure as Code tools such as Terraform or Spacelift
• Experience with configuration management tools such as Puppet or Ansible
• Experience with Ceph or other distributed storage platforms
• Experience with cloud platforms (AWS)
• Familiarity with CI/CD pipelines and DevOps practices