Databricks

JD:

Role
Build a large-scale content discovery platform, including data pipelines, services, and algorithms, using computational probability-based ranking, topic-driven search, content intelligence, telemetry analytics, and distributed data processing.

Responsibilities
 Develop Databricks and Spark pipelines for telemetry, analytics, ranking, and content intelligence.
 Design computational probability-based ranking and discovery algorithms.
 Build content enrichment, topic-tag extraction, and knowledge classification pipelines.
 Contribute to ontology and taxonomy design used to represent content semantics and discovery signals.
 Develop search, discovery, and ranking APIs.
 Design scalable data models and feature pipelines.
 Implement experimentation, measurement, and reporting frameworks.
 Build dashboards and monitoring for platform health and user engagement.
 Partner with product and engineering teams to translate business objectives into measurable ranking signals.

Required Qualifications
 BS/MS in Computer Science or related field.
 6+ years of software engineering experience.
 Strong Python and SQL skills.
 Strong Databricks and Apache Spark experience.
 Experience building production data pipelines.
 Experience designing REST services and distributed systems.
 Experience with large-scale analytics, ranking, search, or personalization systems.
 Experience working in AWS or Azure.

Preferred Qualifications
 Search, discovery, or ranking systems.
 Computational probability and statistical modeling.
 Ontology, taxonomy, metadata, or knowledge-modeling experience.
 User behavior analytics and telemetry.
 LLM-based metadata extraction and enrichment.
 Experimentation and A/B testing frameworks.
 Media, content, advertising, or consumer platforms.

Technical Skills

Required
 Python
 SQL
 Databricks
 Apache Spark 
 REST APIs
 Data Modeling
 Git
 Cache development

Preferred
 Airflow
 AWS / Azure
 Power BI
 Ontology / Taxonomy Design 
 LLM Integration
 Search & Ranking
 Feature Engineering
 Event Streaming