Principal Data Engineer

Who you are

• Education & Experience: BS/MS in Computer Science, Data Engineering, or a related technical field (or equivalent practical experience) with 14+ years of software or data engineering experience with a strong focus on building distributed, high-throughput streaming data pipelines (e.g., GCP Cloud Pub/Sub, Bigtable, BigQuery)
• Cloud Infrastructure: Solid, hands-on experience designing and deploying cloud infrastructure on Google Cloud Platform (GCP)
• Observability: Deep expertise in observability standards, specifically OpenTelemetry (OTEL), and managing large-scale time-series, log, or trace databases
• Programming Skills: Exceptional programming skills in Go, Python, or Java
• Modern Systems Programming: Familiarity with the Rust programming language is highly desired
• Performance Monitoring: Experience with measuring data pipeline processing latency metrics (e.g., tracking P90/P99 latency)
• Specific Databases: Familiarity with managing and querying large-scale databases such as ClickHouse and Elasticsearch
• Infrastructure-as-Code: Proven experience with Infrastructure-as-Code (e.g., Terraform) and container orchestration platforms like Kubernetes
• Data Architecture: Strong understanding of data modeling, data governance, and optimization for distributed storage systems

What the job involves

• As a Data Pipeline Engineer, you will be the architect of our high-volume data ecosystem. You will build and scale the critical pipelines that ingest logs and telemetry from our high-performance engines, transforming massive streams of data into actionable intelligence. Your work will directly power our advanced threat detection capabilities by ensuring our security platforms have reliable, real-time access to network traces and operational metrics at massive scale
• Design, build, and scale high-throughput log and telemetry ingest pipelines capable of processing massive data streams from our distributed inline security engines with ultra-low latency
• Implement and standardize observability frameworks using OpenTelemetry (OTEL) to capture distributed tracing, metrics, and logs across complex, microservices-based architectures
• Architect and manage a highly performant trace lake to store, index, and query deep network forensics, facilitating rapid threat hunting and machine learning model training
• Develop and maintain robust benchmark infrastructure to continuously validate pipeline performance, latency, reliability, and data integrity under peak global loads
• Partner tightly with Data Plane and Machine Learning engineering teams to ensure seamless, efficient data flow from the edge directly into our centralized analytical platforms

Benefits

• Healthcare: We value your health and provide options so you can choose what best supports your lifestyle and personal health goals. Various nationwide plans — including Blue Shield of California, Kaiser Permanente, Delta Dental, VSP, and more — are available to you and your family. Our providers offer the flexibility you want and the coverage you need
• Wellness: With a large on-site gym and daily, instructor-led exercise classes at HQ in Santa Clara, Palo Alto Networks actively supports employee fitness. Our juice bar, Gym & Juice, features fresh juice blends, a quick and nutritious option pre- or post-workout. And our ongoing wellness speaker series highlights the latest trends in health and wellness
• Development: Professional development is serious business at Palo Alto Networks, where the Talent Development team offers 25+ courses to help employees boost their careers and develop leadership capability
• Financial: Most roles are eligible for equity grants, and we now offer new hires a quarterly vesting schedule right away. We want you to be fairly compensated and rewarded as soon as you make an impact – on our team, that happens immediately! And because we consider every employee a stakeholder in our long-term success, we offer an Employee Stock Purchase Plan (ESPP) with a 24-month lookback. From life insurance to health savings accounts, we provide ways to help you protect and grow your wealth. Palo Alto Networks offers both traditional and Roth 401(k) options, along with a company match of 50% to a max annual amount of $1,000
• Time Off: In addition to paid corporate holidays, Flexible Time Off programs empower our employees to balance their work schedules with personal time off. We offer 12 weeks of full pay for medical leave and 6 weeks of full pay for parental leave. As a company with strong ties to the military and veterans, we also offer 26 weeks of full pay for military leave
• Perks: Our corporate discount program provides employees with significant discounts on merchandise, travel and entertainment. We offer many on-site perks at headquarters, including cafeterias, fuel service, dry cleaning service, EV charging stations, and a company store. We also provide complimentary drinks, fresh fruit, and snacks in break rooms across our campus to help everyone power through the day.

Life at Palo Alto Network