Description:
We are seeking a skilled and detail-oriented Web Scraping Engineer with 1-2 years of experience to join our team. You will play a critical role in developing and maintaining web scraping projects, ensuring seamless data extraction from a variety of sources while adhering to high-quality standards and business requirements.
Key Responsibilities:
- Design, develop, and maintain scalable web scraping solutions using tools like Scrapy, Selenium, Playwright, or similar frameworks.
- Extract, clean, and organize data from static and dynamic websites, including handling JavaScript-heavy content and AJAX.
- Implement efficient and reliable scraping pipelines that handle large-scale data extraction.
- Overcome challenges such as CAPTCHA bypassing and anti-bot measures.
- Troubleshoot and resolve scraping-related issues, ensuring consistent and accurate data delivery.
- Collaborate with cross-functional teams to define data requirements and provide actionable insights.
- Ensure compliance with website policies and data protection regulations.
Requirements:
- 1-2 years of hands-on experience with web scraping and data extraction.
- Strong programming skills in Python and experience with libraries/tools like Scrapy, Selenium, Playwright, BeautifulSoup, and Requests.
- Knowledge of working with dynamic websites and overcoming scraping challenges like CAPTCHAs and rate limits.
- Familiarity with APIs and data integration (RESTful services).
- Experience working with databases (SQL/NoSQL) and cloud platforms (AWS, GCP) is a plus.
- Excellent problem-solving skills and attention to detail.
- Strong communication and teamwork abilities.