Description:
We are looking for a highly skilled Senior Python Developer with expertise in Automation, Selenium, and Web Scraping. The ideal candidate should have a deep understanding of browser automation, data extraction and handling dynamic web applications. You will be responsible for designing and implementing robust automation frameworks, building scalable web scrapers, and optimizing data collection processes.
Key Responsibilities:
- Develop, maintain and optimize web automation scripts using Python and Selenium/WebDriver.
- Design and implement scalable web scraping solutions to extract structured and unstructured data from various websites.
- Utilize Scrapy, BeautifulSoup, Requests and other Python libraries for efficient data extraction.
- Handle CAPTCHAs, proxies, user-agent rotation and other anti-scraping mechanisms.
- Develop and maintain test automation frameworks for web applications.
- Collaborate with cross-functional teams to integrate automation solutions into CI/CD pipelines.
- Ensure compliance with ethical web scraping practices and website policies.
- Optimize scraping speed, data accuracy and overall system performance.
- Troubleshoot and resolve issues in automation scripts and web crawlers.
Required Skills & Qualifications:
- 2+ years of experience in Python development with a focus on automation and web scraping.
- Strong experience with Selenium/WebDriver, Scrapy, BeautifulSoup, Requests and other scraping frameworks.
- Expertise in handling dynamic content (AJAX, JavaScript-heavy websites, Shadow DOM).
- Solid understanding of XPath, CSS selectors and Regular Expressions.
- Knowledge of headless browser automation (Chromium, Puppeteer, Playwright).
- Experience in dealing with CAPTCHA-solving techniques, proxy rotation and anti-bot countermeasures.
- Hands-on experience with CI/CD tools, version control (Git) and cloud platforms (AWS, GCP, Azure).
- Strong problem-solving skills and ability to optimize code for efficiency.
- Excellent written and verbal communication skills.
Preferred Qualifications:
- Experience with API development and integration.
- Knowledge of database management (SQL, NoSQL).
- Minimum 3 to 4 years of experience as a Senior Python Developer.
- Exposure to machine learning techniques for data extraction and automation.
- Experience in test automation frameworks such as PyTest, Robot Framework.
- Understanding of cybersecurity principles related to web scraping.