Senior Data Engineer

Senior Data Engineer

One of the most innovative and high-profile teams in News Corp is looking for a seasoned data engineer to help accelerate its vision. The role will involve combining all of News Corps data across categories, countries and organizations into a singular view of a customer enabling engagement, measurement and targeting. The Dow Jones Senior Data Engineer will partner closely with product, data & engineering teams to help make this data available and accessible to our businesses, teams and partners in order to drive revenue and improve the customer experience.

Responsibilities

Architecture & Pipeline Development:

• Design, implement, and maintain high-performance big data infrastructure and processing pipelines capable of scaling to billions of structured and unstructured events daily.
• Architect and maintain deep, resilient integrations with upstream data systems.
• Translate complex business requirements into logical, scalable data pipelines and processes.

Optimization & Data Quality:

• Monitor, troubleshoot, and optimize data platform performance, compute resources, and query execution.
• Establish and monitor comprehensive data quality metrics across all systems to ensure accuracy, consistency, and completeness.
• Evaluate, prototype, and implement emerging data processing technologies to keep the stack modernized.

Collaboration & Leadership:

• Partner with product teams on the overall roadmap, ensuring updates to senior leadership are 100% technically accurate and feasible.
• Conduct targeted research (data collection, analysis, and synthesis) to address specific engineering or business requests.
• Think quickly, communicate clearly, and work collaboratively alongside product, engineering, QA, and operations teams.
• Bring a high energy level, a strong work ethic, and a team-first mindset.

Technologies we use

• Data Warehouse & Modeling: Snowflake, dbt (Core/Cloud)
• Cloud Infrastructure: AWS (EMR, Lambda, S3, SNS, EC2, MWAA Athena, Redshift, DynamoDB), GCP (BigQuery, Cloud Storage)
• Processing & Orchestration: Python, Spark (PySpark), Apache Airflow
• DevOps, CI/CD & AI Tools: Kubernetes, Docker, Terraform (Infrastructure as Code), GitHub Actions, GitHub Copilot, Claude Code, Git, Jira

Qualifications

• BS in Computer Science or other technical discipline
• 5+ years of experience designing and developing big data processing systems using distributed computing
• 1+ years of production experience working with modern cloud data warehouses (Snowflake), advanced dbt workflows
• Fluency in Python and Spark
• Expert knowledge in optimizing complex SQL queries and Hands on experience with NoSQL (DynamoDB, MongoDB)
• Experience with job orchestration tools like Airflow
• An affinity for automation
• Experience working with cloud platforms such as AWS
• Familiarity with networking and network application programming, including HTTP/HTTPS, JSON, and REST APIs
• Experience with at least one object oriented language (ex: Java)
• Strong OO design, data structure, and algorithm design skills
• Strong interest in emerging technologies: Spark, EKS

Nice to Have

• Experience in the digital advertising and marketing industry
• Contribution to Open Source projects