DATA Engineer (Pyspark)

Role Purpose

Develop, validate and productionize machine-learning, GenAI and agentic-AI solutions that deliver measurable business outcomes.

Key Responsibilities

• Translate business problems into analytical and AI approaches.
• Conduct data exploration, feature engineering, experimentation and model development.
• Build predictive, recommendation, NLP, RAG and conversational-AI solutions.
• Define evaluation metrics; perform error, bias, fairness and explainability analysis.
• Collaborate with engineering and MLOps teams on deployment, monitoring, retraining and continuous evaluation.

Required Skills

• Python, SQL, PySpark and scikit-learn
• XGBoost/LightGBM, NLP and LLM frameworks
• LangChain, LangGraph or Hugging Face
• Databricks, MLflow, Azure ML or Google Cloud
• Model explainability, fairness, drift and production monitoring Role & responsibilities

Preferred candidate profile