Job Description

Data Engineer-Web Scraping

• Responsible for Creating and managing website scraping configurations on web
   scraping tool.
• Responsible for monitoring scraping configurations for potential errors and blockages.
• Responsible for monitoring data being scraped to identify potential issues and blockages.
• Responsible for coordinating with stakeholders to understand scraping task requirements and reporting issues.
• Responsible for preparing and sharing periodic scraping activity reports with stakeholders.
• Minimum 3-year experience of working as a data collection and quality engineer/lead.
• Should have hands-on experience of building and managing scraping configurations
   for large number of websites.
• Should have hands-on experience of working with third-party web scraping tools.
• Should have hands-on experience of working with open-source web scraping libraries
• Should have good understanding of data ingestion and processing pipelines.
• Should have hands-on experience of implementing and managing data quality
   checks.
• Should be familiar with programming Languages like Python and Go.
• Should be Fluent with web technology concepts like HTML, DOM, CSS, XPATH etc.
• Should be familiar with usage of regular expressions for data selection and cleaning