ETL_GCP_Bigquery

• Develop, implement, and optimize ETL/ELT pipelines for processing large datasets efficiently.
  • Work extensively with BigQuery for data processing, querying, and optimization.
  • Utilize Cloud Storage, Cloud Logging, Dataproc, and Pub/Sub for data ingestion, storage, and event-driven processing.
  • Perform performance tuning and testing of the ELT platform to ensure high efficiency and scalability.
  • Debug technical issues, perform root cause analysis, and provide solutions for production incidents.
  • Ensure data quality, accuracy, and integrity across data pipelines.
  • Collaborate with cross-functional teams to define technical requirements and deliver solutions.
  • Work independently on assigned tasks while maintaining high levels of productivity and efficiency.
    Skills Required:
  • Proficiency in SQL and PL/SQL for querying and manipulating data.
  • Experience in Python for data processing and automation.
  • Hands-on experience with Google Cloud Platform (GCP), particularly:
    o BigQuery (must-have)
    o Cloud Storage
    o Cloud Logging
    o Dataproc
    o Pub/Sub
  • Experience with GitHub and CI/CD pipelines for automation and deployment.
  • Performance tuning and performance testing of ELT processes.
  • Strong analytical and debugging skills to resolve data and pipeline issues efficiently.
  • Self-motivated and able to work independently as an individual contributor.
  • Good understanding of data modeling, database design, and data warehousing concepts.Role & responsibilities

Preferred candidate profile