Velan | Research Executive

Research Executive

Python 3.x

Roles and Responsibilities

• Develop and maintain robust web scraping and data extraction solutions using Python.
• Build scrapers for websites with static and dynamic content.
• Work with technologies such as Requests, BeautifulSoup, Scrapy, Selenium, and Playwright.
• Handle JavaScript-rendered websites, pagination, authentication, sessions, cookies, and complex website structures.
• Extract, clean, transform, validate, and store structured data from various sources.
• Develop automation scripts for scheduled data collection and processing.
• Implement techniques to handle common scraping challenges such as rate limiting, retries, timeouts, proxy usage, and changing website structures.
• Integrate scraped data with REST APIs, databases, and internal applications.
• Design efficient data pipelines for processing large volumes of scraped data.
• Write clean, maintainable, reusable, and well-documented Python code.
• Debug and troubleshoot scraping failures and data-quality issues.
• Monitor scraper performance and ensure reliable execution.
• Collaborate with backend developers, QA engineers, and business teams to understand data requirements.
• Follow best practices related to security, performance, logging, and error handling.

Candidate Profile

• Bachelors degree in Computer Science, Information Technology, or a related field.
• 2+ years of professional Python development experience.
• Demonstrable experience in web scraping/data extraction projects.
• Strong knowledge of Python 3.x.
• Strong analytical and problem-solving skills.
• Good understanding of OOP, functions, modules, exception handling, and data structures.
• Experience writing production-quality Python applications and automation scripts.
• Strong hands-on web scraping experience is mandatory.
• Understanding of HTML, CSS selectors, XPath, DOM structure, and browser automation.
• Experience scraping dynamic JavaScript-based websites.
• Knowledge of handling pagination, infinite scrolling, forms, sessions, cookies, and authentication.
• Experience working with CAPTCHA challenges and anti-bot mechanisms.
• Experience handling scraping failures, retries, timeouts, and rate limits.
• Understanding of HTTP/HTTPS, request/response lifecycle, headers, status codes, and APIs.
• Experience with debugging, logging, exception handling, and performance optimization.
• Good communication and teamwork skills.
• Ability to work independently and meet project deadlines.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.