Senior Python Developer - Web Scraping & Data Crawling

Hello,

Greetings from GlobalData!!!

Urgent job openings for Software Module Lead - PHP, Wordpress - GD @HYD

Job Description

We are looking for a seasoned Senior Python Developer with a proven track record in Data Crawling and Web Scraping. In this role, you will own the end-to-end data extraction lifecycle, maintain full-stack web crawling applications, and build automated data pipelines.

If you are an analytical thinker who thrives on transforming raw web data into structured business intelligence, we want to hear from you.

Role & responsibilities

• End-to-End Crawling: Architect, deploy, and scale robust data crawlers to extract massive datasets from complex web architectures.
• Application Management: Maintain and optimize the existing full-stack web crawling application to ensure high uptime and data integrity.
• Pipeline Automation: Design and execute Python automation scripts to monitor pipelines, handle errors, and streamline workflows.
• Data Engineering: Clean, manipulate, and structure unstructured web data using SQL and Python libraries for downstream analytics.
• Anti-Bot Navigation: Implement advanced strategies to bypass anti-scraping mechanisms, including proxy management, CAPTCHA solving, and user-agent rotation.

Requirements & Technical Skills:

• Core Language: Strong expertise in Python and Python Automation Scripting.
• Scraping Tech Stack: Advanced hands-on experience with Scrapy, Requests, BeautifulSoup, and browser automation tools (Selenium/Playwright).
• Databases: Proficient in SQL for data manipulation, storage, and retrieval.
• Core Experience: Mandatory hands-on experience leading data crawling projects for 5+ years.
• Soft Skills: Excellent analytical capabilities and strong professional communication skills.

Preferred candidate profile

• Exposure to NLP (Natural Language Processing) and NLTK.
• Basic understanding of Machine Learning (ML) data preprocessing workflows.

Thanks & Regards

Tejaswini.P

(Human Resource)